Something I genuinely admire about effective altruists: since I started a blog criticizing effective altruism, the number of invitations I've received from EAs has gone sharply up. Honestly, this is a bit surreal to me.
Can we maybe agree on the base-level claim that human extinction would be very bad? I know plenty of fancy reasons to disagree with that claim, but most people don't buy them, right?
Over 2022 and 2023, OpenPhil has pulled $350m in planned funding from GiveWell. This money could save about 70,000 lives today. That's the price of longtermism.
Can we please (on all sides) stop relying so much on analogies to drive Twitter debates about AI risk? Analogies aren't arguments. Make arguments instead.
How did effective altruism move from "only the best randomized controlled trials will do" to "why won't scientists accept our papers" within the space of ten years?
I'm starting to see more signs of cross-engagement (from both sides) between the academic AI community and the EA AI safety community. I'm not sure how we're going to make this work, but I hope we manage it, and it makes me happy.
@daniel_271828
My prediction: you're going to find that those other people can't think, especially if you keep out any disciplines of sanity who could call them on the reasoning errors.
Sorry to keep on with the discourse-focused tweets, but ... please don't criticize people for their "date me" documents either. I've read some horrible comments today about a lovely person.
Something I would like to see: more AI safety papers published in reputable scientific journals, and fewer published on blogs. This would improve the quality of the research and make it possible for a wider audience to seriously engage with it.
I want to be honest. I was impressed by the EA Forum’s response to my post about a rather inappropriate April Fools thread. They corrected this well and explained themselves afterwards.
I've been asked to write an article on the TESCREAL critique for a magazine. I want to tread carefully here. Three questions: (1) What are some things the critique gets right, (2) What are some things the critique gets wrong, (3) How can we avoid the kind of polarizing discourse
In 2020, Slate Star Codex asked its readers: "How would you describe your opinion of the the idea of "human biodiversity", eg the belief that races differ genetically in socially relevant ways?" (1 = very unfavorable, 5 = very favorable).
🚨 Very excited to announce that I will be editing a special issue of Philosophical Studies on AI Safety! 🚨
Call for papers linked below — submissions are due in early November, and contributions can be about any aspect of AI Safety.
Please share!
I'm thinking about writing an academic book about longtermism. If I did, would you read it? What would you be most excited to see in the book? Would it help if the book were available open access?
New blog series about my paper "Mistakes in the moral mathematics of existential risk". This builds on my paper "Existential risk pessimism and the time of perils"
New post about the Carlsmith report drops tomorrow (Part 1 of 3). I really am trying to engage with AI safety work, and I think Carlsmith's report is one of the best pieces out there.
I'm happy to see voices calling for change at Manifest. This would be a good time for Manifold to make meaningful changes, or for effective altruists to reassess their relationship with Manifold if they do not.
I have a new logo! Many thanks to the wonderful Alex Savard (
@visably
) for putting it together, including a full kit of variations. I can't recommend Alex enough.
I wish people would give more context in talking about the underfunding of the biological weapons convention. Many of the people using this example know full well that money isn’t the chief bottleneck here.
On my way to EAGxCambridge. Catch me tomorrow at 5pm for a talk about what goes wrong if the Time of Perils hypothesis is false - and why it probably is.
A few days ago I was complaining about the need for increased transparency after the FTX scandal. I'll be darned if this isn't a good start. I'm genuinely impressed.
Sometimes we agree on how to make the world a better place. Thanks to EV and EA Funds for their generous support of a promising class of underrepresented students in global priorities research.
At the pluralisms in existential risk workshop (day 2 today). It's been really insightful. More diversity of perspectives on existential risk than I've seen in a long time. Props to Gideon, SJ and Anders for organizing.
Not even hiding it anymore. The goal is to produce work that would change their minds, not that would be judged as sound by an independent panel of specialists (there are none on the panel).
The EA Forum is getting better at responding to posts about misconduct. There's room to improve, but compared to a few months ago we've jumped light-years ahead. Credit where credit is due.
EAs: Let's talk about what constitutes a reputable source of information. Here is Yudkowsky on TIME magazine (he says it's not reliable enough even to be worth steelmanning)
The amazing Alex Savard (
@visably
, designer of the Giving What We Can logo) put together some logo concepts for Reflective Altruism. What do you think? (Information about Alex below as well).
At least one EA org (CEA) did the right thing: "We unequivocally condemn Nick Bostrom’s recklessly flawed and reprehensible words. We reject this unacceptable racist language, and the callous discussion of ideas that can and have harmed Black people.". But look at the responses.
Peddling discredited racist pseudoscience is the opposite of free and open inquiry. Those who do this use ignorance, prejudice and punditry to recast falsehoods as forbidden knowledge.
Planning my future writing: what do folks honestly think about the prospects for placing a paper about instrumental convergence into a leading philosophy journal? The paper would aim to clarify instrumental convergence and cast doubt on its ability to carry AI risk arguments.
At a recent discussion, a leading existential risk researcher told me there is no realistic chance of existential catastrophe from biological causes. A second said there's a sizable chance, but they couldn't tell me why (it's an infohazard). What am I to make of this situation?
The turn to longtermism has brought a rapid devolution in standards of evidence within EA. The movement went from very high standards ("where's your randomized controlled trial?") to rather low standards (self-published reports and Forum posts).
In my new paper (AI&Society), I develop the argument that misaligned AI poses an existential risk to humanity, including discussion of multiple common objections.
I'm working on a chapter of my longtermism book. The chapter focuses on risk and ambiguity. This seems like a good time to remind people *not* to make the cheapest arguments that risk/ambiguity aversion falsify longtermism:
Apparently some (many?) people think that AI should not be regulated in any way. That is quite a fringe view, and should not be treated as representative of skepticism about AI x-risk. It's also insane.
Feuding with the editorial board of Nature is not a good look for any movement that prides itself on the use of reason and evidence to solve the world's problems.
Not to defend OpenAI, but if their superalignment team was firing off doomsday predictions like this one I understand why OpenAI may have been uncomfortable employing them.
Virtually nobody is pricing in what's coming in AI.
I wrote an essay series on the AGI strategic picture: from the trendlines in deep learning and counting the OOMs, to the international situation and The Project.
SITUATIONAL AWARENESS: The Decade Ahead
@robertwiblin
The purpose of my blog and associated media account is to use academic research to drive positive change within and around the effective altruism movement. I think this is an important and impactful mission. Other people can (and do) write about other movements. That's a good
I've started writing a blog series based on my paper "Mistakes in the moral mathematics of existential risk". It's a follow-up to my "Existential risk pessimism and the time of perils", which explains some other common mistakes we in calculating the value of existential risk
Thanks all for your feedback on examples in EA (Biological Weapons Convention + Aum Shinrikyo). At least one person asked me to write this up into a proper blog post, so I wrote a draft. Should be out in 1-3 months.
Let me be clear. If OpenPhil were to add a panel of independent experts to their judging panel I suspect the panel would tell them that most of their preferred submissions are of poor quality. I think that OpenPhil should be be moved by that.
@NathanNobis
One good place to start would be my work. I have four published papers and dozens of blog posts, many of which are neither calling for small tweaks at the margins nor for changes that fall into categories (a)-(c).
There's a really interesting exchange between Richard Pettigrew (
@Wiglet1981
) and Andreas Mogensen on risk aversion and longtermism going on. Original paper:
Let's be clear. This is nonsense. People who have devoted their life to a subject (scholarship; investigative journalism ...) at the highest levels know something about it and should be listened to.
We're less than one day into the TIME magazine story about sexual harassment and abuse and already the EA Forum is trying to reveal identities redacted to protect vulnerable women. Don't do it.