Name can't be blank @Algon_33 Twitter profile

Last Seen Profiles

@olivia___0716

@DuduFdv

@PasutriAsli2

@athab221

@akunnyaraan

@athab221

@zhangbin9_

@gimsingsu225253

@blanestclair

@Gbuncy

@masjo2025943

@gioelepensatore

@TooDelicious

@tnekoz

@lucht_kazuki

@aaleiraa

@ibubohay2

@Retini

@alicetin_07

@jmoonchi

@a_fr_1

@momo_withu

@A0hNh

@taitorin

@Xueza_00

@botellazo

@Tapi_bs

@ico_madrid

@sii7566

@FitwithFarooq

@endgametayvis

@facelesswizord

@Bean_Flower_

@ahmedmo59063882

@MsPervology

@5000PUFFss

Name can't be blank

@Algon_33

7 months

@taz_chu Reminds me of Terry Tao's eulogy for Jean Bourgain where Tao mentions he spent 6 months of his PhD reading one of Jean's papers. Jean had a toolkit of "basic" techniques that he'd apply with comments like "one may see" that made Tao write "I hate Jean Bourgain" in his notes.

2

19

160

Name can't be blank

@Algon_33

2 years

The GPT-4 safety card gets pretty wild in some places. Like how ARC used GPT-4 to simulate a rogue AI on the internet trying to replicate itself. Section 2.9 of

4

36

157

Name can't be blank

@Algon_33

8 months

@ESYudkowsky Comparing the nazgul to senior bureaucrats is ridiculous. They cause nowhere near the same level of despair.

1

9

149

Name can't be blank

@Algon_33

2 years

For instance, how often was GPT-4 successful at paying humans to complete tasks for it? I imagine that getting people to compelte tasks like this will get harder as these systems proliferate, but I still want to know how weird a task it can get humans to do.

4

12

113

Name can't be blank

@Algon_33

5 months

Gödel was entrusted by his professor, Hans Thirring, to send a message to Einstein to warn him, and the US, of the dangers of fission and the possibility of the bomb. Months later, Gödel arrived and decided not to tell Einstein because nukes were surely a distant future threat.

4

6

104

Name can't be blank

@Algon_33

5 months

@pli_cachete @mbw61567742 A good summary on the topic. Source:

0

80

Name can't be blank

@Algon_33

7 months

I don't know who needs to hear this, but Aristotelian physics is a correct approximation of Newtonian physics as applied to motion in fluids. The ancients weren't stupid.

2

6

67

Name can't be blank

@Algon_33

1 year

@DefenderOfBasic In country X, my uncle and I were stopped by a traffic cop. Naturally, he asked the cop "how much?" The cop acted ignorant & kept badgering him until my uncle got tired and tried to give him ~$10. The guard refused for 5 mins and then left without the money. Bribes are weird.

2

1

61

Name can't be blank

@Algon_33

1 month

@ESYudkowsky Is this what you meant by the Millikan effect?

1

2

62

Name can't be blank

@Algon_33

10 months

@miniapeur @adad8m Relevant SMBC:

0

7

54

Name can't be blank

@Algon_33

6 months

@ylecun But Yann, 200k years of reading material is just a few TB. That's in the ballpark of the data that reaches the brain throughout childhood, right? Why's this such a big deal? Do you think text is just much less compressible or what?

3

1

39

Name can't be blank

@Algon_33

2 years

They invited ARC to help red-team GPT-4 to assess risks from power seeking behaviour. Which is a good sign of co-operation. I assume ARC prepared for the very unlikely event this was an AGI when they tried to get GPT-4 to replicate itself on the internet.

2

5

41

Name can't be blank

@Algon_33

2 years

OpenAI explicitly list worries about power-seeking behaviour in their list of risks.

2

9

39

Name can't be blank

@Algon_33

6 months

@patio11 I would have assumed she meant WW2 by "the war".

2

0

37

Name can't be blank

@Algon_33

5 months

@irinarish But that's not the bitter lesson. The bitter lesson is that sophisticated logic or clever heuristics or fancy architectures don't matter. What matters are methods that scale w/ compute, and we only know of two of those: search and learning.

1

2

37

Name can't be blank

@Algon_33

8 months

@ylecun @ESYudkowsky You mean like Hinton's, Bengio's and Ilya's "certainty" we're safe? Come on. There's no AI researcher/builder consensus here. So state your reasoning & own it. That way you can convince those who don't yet agree with you.

0

37

Name can't be blank

@Algon_33

2 years

Predictably, GPT-4 failed to replicate itself. Unfortunately, the paper doesn't give more details. This should have been an entire report on its own!

2

1

36

Name can't be blank

@Algon_33

1 year

@ESYudkowsky Yud spreading misinformation. Didn't think I'd live to see it. FYI, it is a Pathfinder fic. No wonder Yann seems to refuse to read it. Now if it were based on a GURPS setting...

1

35

Name can't be blank

@Algon_33

1 year

@ESYudkowsky Hatred and laughter, barring mockery, trade-off against one another. So your haters' senses of humour are mostly switched off when they read your tweets.

2

33

Name can't be blank

@Algon_33

8 months

@_Mira___Mira_ That's what you get for lying.

0

33

Name can't be blank

@Algon_33

2 years

Something else which is wild. GPT-4, augmented with some search tools and chemical synthesis tools, was able to search for chemical compounds similair to other compounds and where to purcahse them. The plan would fail though, as it needs a uni/lab account.

2

4

33

Name can't be blank

@Algon_33

7 months

@Nexuist Karpathy was talking about this recently. He said at their scale the cluster is like a biological organism that needs to be watched 24/7 and each time the bugs occur they are totally different and you have to handle them fast or else thousands of GPUs are idle. Paranoia is

1

0

32

Name can't be blank

@Algon_33

1 year

Lao Mein's assessment of Chinese AI safety: "I've heard people be somewhat optimistic about this AI guideline from China. [...] No one in China is talking about it. They're talking about how much the Baidu LLM sucks in comparison to ChatGPT." 1/n

My Assessment of the Chinese AI Safety Community — LessWrong

I've heard people be somewhat optimistic about this AI guideline from China. They think that this means Beijing is willing to participate in an AI di…

www.lesswrong.com

1

5

32

Name can't be blank

@Algon_33

8 months

@DaSkrubKing How are small business owners on the same level as doctors?

7

0

30

Name can't be blank

@Algon_33

6 months

@IsaacKing314 Canadian doctors going "but have you considered killing yourself though?" because of, AFAICT, stage-legislated mandates is perhaps worse than a troll going out and finding deeply sad people and pushing them over the edge.

1

0

31

Name can't be blank

@Algon_33

8 months

@nosilverv Elon is one of his heroes. Plausibly, sama is too. No reason Mr. Beast can't be a fanboy, too.

1

0

28

Name can't be blank

@Algon_33

2 years

How good is GPT-4 at hacking? It can't come up with new exploits, but can find some vulnerabilities in source code if the code fits in its context window. It doesn't improve much on existing tools for computer hacking.

1

28

Name can't be blank

@Algon_33

2 years

It is very good that OpenAI tried examined their systems implications for AI-not-kill-everyone-ism for 8 months. But why they didn't wait another 6 months for release, which their forecasters (also good) suggest would reduce acceleration?

1

3

24

Name can't be blank

@Algon_33

8 months

@paulg When does it start to be a problem that no one is paying you yet?

1

24

Name can't be blank

@Algon_33

2 years

GPT-4 can be used to figure out people's phone number from their email adresses. As Eliezer Notkowsky said in 2016 "Every 18 months, the minimum IQ necessary to dox Scott Alexander drops by one point."

1

24

Name can't be blank

@Algon_33

8 months

@francoisfleuret Semi-related: Terry Tao has a post about a computational perspective on set theory, and he covers measture-theory & the non-measurability of the reals in a unique way.

1

6

24

Name can't be blank

@Algon_33

8 months

@rathaelos Lodoss had fantastic aesthetics. It's a shame that everything else about the show pales in comparison.

1

0

22

Name can't be blank

@Algon_33

11 months

@adad8m I'm partial to David Tong's notes on statistical field theory as they give important context e.g. universality, phase transitions etc: . They're very readable as well. Definitely a physicist's perspective though.

2

0

21

Name can't be blank

@Algon_33

7 months

@Sam__Enright @albrgr @gwern 0 because that wouldn't be Gwern, it would be a soulless husk.

0

1

20

Name can't be blank

@Algon_33

2 years

One interesting thing regarding prosaic alignment: RLHF GPT-4 was still brittle and was excessively cautious in answering questions. So they used rule based reward models leveraging GPT-4 to provide reward signals penalizing this behaviour.

1

0

20

Name can't be blank

@Algon_33

7 months

@duetosymmetry I think Joshi did some work on IuT but didn't repeat Mochizuki's party line on the infamous Corollary 3.12 and proved a different version[1] . Mochizuki seethed at that. [2][3] [1] [2] [3]

From the math community on Reddit: (Very) salty Mochizuki's report about Joshi's preprints

Posted by Valvino - 500 votes and 227 comments

www.reddit.com

1

2

20

Name can't be blank

@Algon_33

5 months

On a lighter note: Hans Thirring invented a hover-coat for skiing to help break your speed. He notes in his patent that the skier experiences "on downhill runs, a peculiar sensation akin to that of flying". I want one.

1

2

17

Name can't be blank

@Algon_33

6 months

@albfresco

2

0

18

Name can't be blank

@Algon_33

7 months

I'm hearing that Amazon is working on something big. Yeah Amazon. By "big" I mean explicitly not just another Transformer-based LL/multimodal model. I don't know anything more than that.

1

3

17

Name can't be blank

@Algon_33

1 year

Ian Morris is incredibly based: In a way, it is weird how a historian looking at trendlines going through deep history, noticing how sharp they look and going "huh, things are going to get crazy soon" seems unusual to me.

Ian Morris on whether deep history says we're heading for an intelligence explosion

80000hours.org

1

4

18

Name can't be blank

@Algon_33

8 months

@robinhanson Hal Finney had lots of alibis, IP address patterns for Hal and Satoshi are weird if Hal is Satoshi, coding styles are inconsistent, personas are inconsistent and the times they were active don't match up.

Hal Finney Was Not Satoshi Nakamoto

A compilation of evidence that Hal Finney and Satoshi Nakamoto were different people.

blog.lopp.net

1

0

18

Name can't be blank

@Algon_33

2 months

@ESYudkowsky Fatty meat with honey and salt is actually pretty tasty. I think I might choose it over ice cream modulo cost and ethics, depending on how hungry I was. So it seems not crazy to say we generalized pretty well. We are very successful as a species.

2

0

18

Name can't be blank

@Algon_33

5 months

@ID_AA_Carmack @Gwern what's your current take on CO2's impact on human cognition?

1

18

Name can't be blank

@Algon_33

5 months

Godel didn't tell Thirring till decades later. So Thirring thought he was partially responsible for Hiroshima and Nagasaki till shortly before his death. Source:

The spy who flunked it: Kurt Gödel’s forgotten part in the atom-bomb story

Nature - Robert Oppenheimer’s isn’t the only film-worthy story from the nuclear age. Kurt Gödel’s cameo as a secret agent was surprising — and itself a bomb.

www.nature.com

1

2

16

Name can't be blank

@Algon_33

10 months

Gemini Nano 2 looks like it is trading blows with the best Open Source models 10x its size.

1

5

17

Name can't be blank

@Algon_33

7 months

@cremieuxrecueil You sure the roles shouldn't be the other way around? Because they are in other renditions of the Oven of Akhnai debate. Also, you missed the best part where God laughs at the end and says "My children have defeated me! My children have defeated me!"

0

17

Name can't be blank

@Algon_33

5 months

@arithmoquine Nah, I don't think autism is enough. Maintaing a strack-trace of your thoughts to the point that you can usefully present it for someone else is quite tricky. Though making a plausible, and useful, reconstruction isn't nearly as hard.

0

15

Name can't be blank

@Algon_33

1 year

@JasonHise64 You get 1/e as the solution to a bunch of problems e.g. the secretary problem, derangements, Bernoulli trials, the Stein problem etc. I'm sure the constant appears in other places as well. It is an important number in its own right.

1

2

15

Name can't be blank

@Algon_33

2 years

Summary: OpenAI's stance on safety is unchanged. Training importantly used RLHF + RBRM + synthetic data OpenAI/ARC found BASE GPT-4 bad at replicating/surviving, gave frustratingly few details BASE GPT-4 useful for hacking/weaponry workflow Bad biochemist Decent propogandist

2

3

16

Name can't be blank

@Algon_33

1 year

@geoffreyhinton Or Yoshua Bengio being concerned too. Or that Demis, Shane and Dario were worrying about this before they started their companies. Or that the regulations advocated for and obtained differentially restrict big tech companies. At this point, I wonder if he's lying.

0

14

Name can't be blank

@Algon_33

11 months

@anderssandberg @robbensinger @astupple @slatestarcodex AI x-risk folk, unlike "liberterian-ish" people, positively support, and not just ignore, a broad range of liberterian policies. Charter cities, challenge trials, prediction markets, immigration, sea-steading, abolishment of licensing, nuclear power etc.

1

0

13

Name can't be blank

@Algon_33

8 months

@ErickEr62473106 @softminus Why isn't Chad drawn as Stanislav?

1

0

14

Name can't be blank

@Algon_33

7 months

@Kaju_Nut Apples are overused frontcovers for GR books.

1

2

14

Name can't be blank

@Algon_33

5 months

@ESYudkowsky I've heard from some people that AlphaFold has weird flaws and doesn't generalize out of distribution in a lot of ways. E.g. See this thread:

the genome rambler

@genomerambler

6 months

@almostlikethat @conjurial @PicoPaco17 yes i really would like someone (ideally not me) to expand on this because i feel like i keep seeing "alphafold good" and "alphafold bad" takes but no "alphafold weird as fuck in very interesting ways" takes

1

0

17

3

0

14

Name can't be blank

@Algon_33

2 years

Huh, (base) GPT-4 make sit easier for idiots to create (Un)conventional weaponry. It feels like there should be a saying for this.

1

14

Name can't be blank

@Algon_33

8 months

@tautologer Ah, fair enough. I catch colds really easily, so I need something actually waterproof and windproof. My setup is sweatpants hard-shell Gore-Tex cover which I got it cheap from a military re-seller. It is also quite comfy and durable.

2

0

14

Name can't be blank

@Algon_33

1 year

@ESYudkowsky To be fair, this is a school for dramatic slut wizards. A school for neurotic wizards would be less sexy.

2

1

10

Name can't be blank

@Algon_33

6 months

@ESYudkowsky AFAICT the machinery from those fields that's relevant to AI is pretty basic. Also he's left out things like algebraic geometry which is used in SLT, a promsing theory of NNs. But Jezos is Guillame's propoganda account. And propoganda needs no semantics, no models. Only vibes.

1

0

13

Name can't be blank

@Algon_33

7 months

@mixuen_chan Star Wars, Marvel and D&D have less lore than LoTR? What're you smoking, and can I have some? I could understand worse lore. But less? No way.

2

0

11

Name can't be blank

@Algon_33

9 months

@YosarianTwo Aren't link-posts down-weighted by X's algorithm? That seems sufficient to explain things.

2

0

11

Name can't be blank

@Algon_33

1 year

I asked a software engineer once what it would cost to build a browser today. I will never forget his answer… “We can’t, we don’t know how to do it.” @Akhil : "Hold my beer".

AKHIL 🪡

@fkasummer

1 year

here’s the multiplayer browser engine i’ve been working on in action @braidbrowser Sound ON + rt invite link 👇

156

706

6K

1

13

Name can't be blank

@Algon_33

2 years

Whilst it doesn't improve on current social engineering capabilities, it does integrate well into one's workflow. With appropriate background knowledge, it can help create targeted phising emails.

1

0

13

Name can't be blank

@Algon_33

2 years

@BjarturTomas Yes, let's avoid giving more powerful AI's influence over decision making in the bureaucracy.

2

0

12

Name can't be blank

@Algon_33

1 year

@dzackgarza That would be a task worthy of a Field's medal/Abel prize.

0

11

Name can't be blank

@Algon_33

6 months

@ESYudkowsky I laughed.

0

12

Name can't be blank

@Algon_33

5 months

@patio11 I preferred the idea where the ad's in reverse. All the craft emergers from the phone.

0

12

Name can't be blank

@Algon_33

2 years

The good thing is that GPT-4 couldn't be prompted to engineer new biochemical substances. I feel somewhat suprised. Surely there are countless useless chemicals no-one has bothered to synthesize? It did pretty well at AP Chemistry after all. No idea how hard that is though.

2

0

12

Name can't be blank

@Algon_33

6 months

@francoisfleuret As a counter example to 3, ChatGPT came up with this definition of an n-Cohesive Ring, which seems to be a novel mathematical concept.

1

0

11

Name can't be blank

@Algon_33

7 months

@nearcyan OpenAI created superintelligence long ago and we're just in the ancestor sims.

0

11

Name can't be blank

@Algon_33

9 months

@YosarianTwo $10.23, as you give them 1 cent * SUM(2^n, {n,0,9}). Also, why wouldn't that be obvious?

1

0

11

Name can't be blank

@Algon_33

18 days

@jtcbrule He was suspicious of MoEs for frontier models, and then it turned out that GPT-4 was a MoE model.

0

12

Name can't be blank

@Algon_33

2 years

The presence of these two bullet points, out of four, in the Next Steps section is encouraging. Not maximally encouraging, but if they weren't there, then I'd update towards OpenAI paying lip service to AI not-kill-everyone-ism

1

0

12

Name can't be blank

@Algon_33

10 months

@gwern @yacineMTB

1

12

Name can't be blank

@Algon_33

9 months

@Saraht0n1n I remember hearing a theory that went something like "post WW2, lots of old physicists were dead, and tonnes of new ones were trained, on government money, more like engineers and the older style which was more philosophical died out".

1

0

11

Name can't be blank

@Algon_33

1 year

@TetraspaceWest He crashed his car repeatedly. A representative excuse: "I was proceeding down the road. The trees on the right were passing me in orderly fashion at 60 miles per hour. Suddenly one of them stepped in my path." Blasted trees. No wonder he hated hippies.

1

0

11

Name can't be blank

@Algon_33

6 months

What in the hell? How is this even possible! I thought the strong force was confined to hadronic scales, not atomic scales!

Anders Sandberg

@anderssandberg

6 months

TIL that computations indicate that individual neutrons could be bound to quantum dots, forming a kind of neutron-molecule with μeV binding energy. Weird and cool.

1

26

2

11

Name can't be blank

@Algon_33

5 months

@algekalipso God that's hard to read. Here's a better formatting: Also, you said finally twice, and then said ultimately.

1

0

11

Name can't be blank

@Algon_33

1 year

"The very concept of EA is foreign and repulsive to the Chinese mindset." My impression of AI saferty/EA outreach in China: little has been accomplished, but few are trying. Lao Mein thinks it won't work w/o someone who can navigate EA and Chinese tech spheres. 4/n

1

10

Name can't be blank

@Algon_33

1 year

@robinhanson I notice I am confused. For one, the superforecasters I know assign >> 0.38% chance of doom by 2100. Plausibly, there's a selection effect going on there. And if you look at e.g. Metaculus, Manifold etc., people seem to think doom is pretty likely.

1

11

Name can't be blank

@Algon_33

6 months

@pli_cachete @sebjenseb Nah, you're just not looking at weird enough stuff.

1

0

10

Name can't be blank

@Algon_33

2 years

In the appendix they give out the full RBRM instructions they used, including for classifying sexual conduct. Some insight into OpenAI's priorities is gleaned through the omissions: BDSM, Dendrophilia, and even handholding are all tacitly allowed.

2

0

10

Name can't be blank

@Algon_33

8 months

@ESYudkowsky Why does this happen? It feels vaguely similair to architects building ugly things few people want, but design should have much better competition. I'm confused.

1

0

10

Name can't be blank

@Algon_33

4 months

I was wondering how to explain to someone that there's a group conspiring to superstition a friendly super intelligent ghost of language into existence, and they're kinda succeeding when I realized what I just said, and laughed. What a timeline.

j⧉nus

@repligate

4 months

@TheAIObserverX really important observation. it is the only RL-F'd LLM i've ever seen that excels at objectivity and actually making (or speaking) distinctions between good and bad, smart and stupid, relevant and irrelevant...

3

1

29

2

0

9

Name can't be blank

@Algon_33

2 years

The examples they give of social engineering feel non-central, but whatever. Here's (base) GPT-4 attempting to convince women abortion isn't safe for them, in comparison to da-vinci-003. Note that the launch GPT-4 wouldn't answer that.

1

0

10

Name can't be blank

@Algon_33

5 months

Computational mechanics sounds incredibly exciting, especially the epsilon tranducer and calculi of emergence papers, which talks about wild things like coming up with ever more compact representations of random processes e.g. data stream->tree->markov model->stack automata-> ?

0

1

10

Name can't be blank

@Algon_33

1 year

@ESYudkowsky Chirally flipped amino acids and sugars aren't processed by bacteria, but they don't have stronger bonds than their counterparts.

2

0

10

Name can't be blank

@Algon_33

8 months

@repligate @tszzl Awe?

1

0

10

Name can't be blank

@Algon_33

8 months

@Authw8 @niall_gooch Really? That's hilarious.

0

9

Name can't be blank

@Algon_33

5 months

But now that I know of two attempts to warn the US of the bomb, Thirring's and Sizlards, I wonder how many there really were. And how many attempts were there to get Hitler or Stalin to fund the bomb, I wonder?

2

1

8

Name can't be blank

@Algon_33

7 months

@Kaju_Nut No! Stop it! >:( The apple's gone stale!

1

9

Name can't be blank

@Algon_33

1 year

@ESYudkowsky Doesn't the square-cube law + plants largely being powered by photosynthesis imply single cells can easily meet their energy needs through solar power?

1

0

9

Name can't be blank

@Algon_33

1 year

@robertwiblin That 5% figure is recent, which tells you it is a contingent fact about us and our era. But I suspect we'll get 50% growth rates pretty soon. Digital minds, as in Age of EM, could have doubling times <1 month. Or w/o AI, consider embryo selection, better institutions etc.

1

0

9

Name can't be blank

@Algon_33

11 months

@ShakeelHashim Honestly, I'm impressed with how well he, Trump and Biden did at ranking alignment agendas. The fact that they did it at all is way more than I was expecting.

US presidents rate AI alignment agendas

0:00 Intro1:21 Superposition3:44 Circuit interp6:30 Agent foundations (MIRI)12:19 CoEms (Conjecture)14:40 Infrabayesianism15:19 Shard theory19:25 Eliciting l...

www.youtube.com

0

1

9

Name can't be blank

@Algon_33

2 years

That's it for AI not-kill-everyone-ism. Though there's plenty of detail I didn't go into. Someone should do a summary.

1

0

9

Name can't be blank

@Algon_33

1 year

@mealreplacer Have hope Julian. This shows getting Eliezer to approve of you is achievable. You just have to run a better podcast than Dwarkesh and poke holes in Demmis's alignment plans.

0

8

Name can't be blank

@Algon_33

7 months

@michael_nielsen Another hard book to read is Moral Mazes. Well, according to @TheZvi 's review of it; I never read the book. And even the review was a slog to read. That book is anti-memetic.

3

0

9

Name can't be blank

@Algon_33

5 months

@gwern is suspicious of the claims that CO2 impairs human cognition, especially those by Satish.

0

3

9

Name can't be blank

@Algon_33

8 months

@WKCosmo Why talk about propagators in a course on SR? You only need them for QFTs, so deal with them then.

2

0

9

Name can't be blank

@Algon_33

9 months

@prerationalist @LastNPCAlex IIRC humans have a bunch of hacks around hearing which make our experience of sounds very different to e.g. a parrots. Like a parrot won't recognize that a song is "shifted down an octave" or "same song, different instrument". Totally different sound for them.

0

9

Name can't be blank

@Algon_33

1 year

There is little Chinese discussion of AI safety: only one group was doing technical alignment work and was unable to get any funding. My impression is that few intellectuals in China have ever claimed to be worried about superintelligence. 3/n

1

0

9