New AI & international security paper just dropped. 🔥
Let's take a look at how confidence-building measures can mitigate some of the international security risks associated with foundation models.
I see that some NatSec dudes are crying because NATO allies criticized the Supreme Court decision.
This is an alliance based on liberal democratic norms. The U.S rolled back the clock on equality and you're shocked that the U.S continues to lose its global moral authority?
See, the thing about the promotion of norms abroad is that the entire enterprise begins to fall apart when you're suffering from internal institutional decay. Liberal democratic countries notice when their allies introduce differentiated citizenship.
I have never felt closer to my OpenAI colleagues than I do now
I left Canada--I miss my friends, family & country every day--to join OAI because I believe in the collective power we all have to advance safe & beneficial AGI
And this is just the start. We're stronger than ever.
Anyway, I feel bad for many of the U.S civil servants who will now have to engage in international discussions with disclaimers like "this is so embarrassing," which is what happened under the Administration that got us into this legal mess in the first place.
Because AI safety has been 'making the rounds' this week, I want to emphasize that you don't need to identify as longtermist or EA to tackle this issue.
Some professional news! I’m absolutely thrilled to join
@OpenAI
as a member of the policy research team. I’m looking forward to working on some of our most challenging questions with this ridiculously talented group of people.
I am super proud of our
@OpenAI
team! If you haven't already, check out our research papers--we have both a GPT-4 Technical paper *and* System Card! (Links below.)
@carmenmmachado
What struck me is that the 'stolen' details were fairly generic, while the details were very unlike her life. The author criticizes the fictional account for not being generous/accurate enough to their ex. Well, that's what makes it fiction, not biography.
If you work in AI & feel safe doing so, I really encourage you to speak out against Bostrom's very bad and defensive apology. We can't create a safe world while allowing false and harmful (Bostrom did both!) ideas to push bipoc people out of the field.
@LarryRosenthal
@beyerstein
I was waiting for someone would bring up Turkey--the awkward partner, to be sure! Probably best for the U.S to not emulate Turkey's transition to non-democracy. It does tell us that a NATO membership isn't a bulwark against authoritarian pressures.
I'm hiring! I'm looking for a research scientist who works at the intersection of AI and global politics to join my team.
Feel free to DM if you have any questions about the role. :)
Daniel is one of the smartest alignment researchers I know—and he thinks the board should resign. Rigorous safety research happens hand-in-hand with good governance.
I deeply regret my participation in the board's actions. I never intended to harm OpenAI. I love everything we've built together and I will do everything I can to reunite the company.
It would be a real shame if AI safety became a brand instead of a field of inquiry. Other research fields have intellectually suffered from being insular...let's not repeat the same mistakes.
I come into the AI safety discussion from IR, meaning that I'm concerned about dual-use, the undermining of international humanitarian law and human rights law, and international stability--all problems that are not historically associated with the aforementioned viewpoints.
@KevinLi26651602
The countries you mention are experiencing democratic rollbacks which, yes, also threaten NATO’s stability. These democratic rollbacks are not a stable, permanent features of the org’s history. It’s not correct to say that NATO’s moral system doesn’t play into existence.
@ESYudkowsky
@Miles_Brundage
What you are describing is effectively the status quo. “not signing an agreement” and aiming for superiority in military tech are common challenges in IR. So ask yourself what other tools currently exist to solve this very common problem?
So much of proposed AI governance comes down to technology-denial like export controls. More research should focus on understanding how these rules can worsen the global digital divide & how to improve access to innovation tools in a security environment clamping down on chips.
@GregCfollows
@carmenmmachado
Hi Greg, It turns out that I can read. I appreciate that you've chosen to frame our disagreement as one where you think the problem is my reading comprehension. Sometimes, ppl just disagree. The sooner you accept that, the quicker you'll drop your unearned condescension. Cheers!
We have reached an agreement in principle for Sam Altman to return to OpenAI as CEO with a new initial board of Bret Taylor (Chair), Larry Summers, and Adam D'Angelo.
We are collaborating to figure out the details. Thank you so much for your patience through this.
My initial reaction is very positive. These principles are not *exclusive* to autonomous weapons systems--meaning that USG acknowledges a wider range of high-risk AI applications.
There's a good chance that if you work on international relations or in an adjacent field then you haven't heard of EA or longtermism. That's a-okay. We still need you.
For my final Twitter act of 2021,
I’m assigning a novella by Ted Chiang called “The Lifecycle of Software Objects.”
Over 10 yrs old but the story and moral questions remain relevant. An important read if you work on AI. (Also contains a quick nod to a flu pandemic.)
I have a selfish motive here--I want to attract research talent. Most research fields aren't "EA" aligned. I don't want to risk closing the door to people who have great questions.
I think there's also a misconception about people who work in AI safety. If you're interested in joining an AI lab but worried that you won't 'fit in,' let me take the first step and say that I don't identify as EA or longtermist.
@ESYudkowsky
@Miles_Brundage
For better or worse, this is partially why the US plays the role of “global police officer.” So what you’re suggesting w. air strikes wouldn’t be the enforcement of an intl treaty, but a unilateral/multilateral club of nations exerting military might on the intl system.
Co-signed. On IR,
@swatipash
once said that the discipline is "deeply engaged with war yet seems to have an estranged relationship with it.”
The discipline struggles to think about bodies. Great 🧵
Indeed, the international relations literature is full of phrases like "use of force" or words such as "conflict", "aggression", and, obviously, "war".
But the word "killing"? Or the word "violence"? Much more rare.
@GregCfollows
@carmenmmachado
The short story writer points to the following similarities: age-gap relationship, working at a movie theatre & the name of the town. Those are 'identifying' details as we say in the research world, but fairly generic in terms of human experience.
While the "AI arms race" discourse lately is cringe from a scholarly/precise-language perspective (RIP my colleague
@SarahShoker
+ others who know/care about such things), the basic idea of AI competition getting more fierce lately, and that being worrying, is correct. 🧵
I'm getting this framed. I cannot emphasize enough how happy I am to see the mainstreaming of AI (and space exploration!) to a readership that was historically thought to be uninterested. Shoutout to the talented
@karenxcheng
for this cover.
I’m very happy that other AI labs are implementing CBRN red teaming.
AI safety isn’t zero sum, and I believe we’ll all be safer if we build on and recognize each other’s work.
Also shout out to
@_lamaahmad
who really set the tone on normalizing red teams for AI labs.
Glad Anthropic is investing in stuff like bio risks but unless I am missing something, there is no reference to
@SarahShoker
et al.’s seminal work with external red teamers like
@andrewwhite01
in the GPT-4 system card, which made very similar claims.
The one silver lining about the Y. article in Time is that it’s laid bare the international security theorizing that’s become popular in AI safety circles.
It feels odd to me that actor-network theory isn't more widely discussed in AI spaces *outside* academia. If we're trying to understand whether AI can be a 'collaborator' or 'act' on the world, then I would have expected Bruno Latour to be cited more often.
J&J has agreed to grant licenses to sell generics in countries with a high burden of TB--and this is all within the span of 1 week. That's nerdfighteria for you.❤️
That fact that Erin O'Toole was ousted in part because he wasn't sufficiently willing to embrace a convoy that included the waving of nazi flags isn't exactly making me feel optimistic about our democracy.
@ESYudkowsky
@Miles_Brundage
What you are suggesting has many problems (pre-emptive and relies on alignment theory rather than evidence to pursue punishment, for 1) and fundamentally misunderstands how international treaties work (usually voluntarily, actually!)
As far as I can tell, humans are extremely optimistic when it comes to estimating how much they can get done in a day, and really cynical about everything else.
Erm, I want to talk about "Rabbit Test" with people.
I'd be surprised if this story didn't become a classic. I can see why it was nominated for a Nebula. 👀
@UncannyMagazine
@ESYudkowsky
@Miles_Brundage
The “compliance” angle you’re pushing comes from a smaller club of nations who institutionalize certain non-proliferation rules (e.g NATO, Wassenaar etc.) that overlap with intl treaties. And even here the coercive response is not purely through kinetic military actions.
I suppose you could ask “in what way is AI like nuclear?”
But the better question is “how does AI interact with nuclear* risks?”
*and other international security risks
@ESYudkowsky
@Miles_Brundage
Finally, I’ll say that a military threat on critical infrastructure tends to incentivize the building of deterrence technologies (currently, nukes. For you, likely AGI.) And so, your game plan is actually escalatory & more likely to contribute to the very thing you fear.
It is extremely, extremely disappointing that there are people who are centering Bostrom as the victim (of his own words, really?)
I hear the word 'humanity' in this field a lot, but too many times there are some humans who have their humanity respected over others.
OpenAI’s mission is to ensure that artificial intelligence benefits all of humanity.
An important part of this effort is training AI systems to align with human intentions and human values.
Learn more about our alignment research:
I 💯 believe that Nerdfighteria can pull through on this. The community consistently meets its philanthropic health goals.
If you’re reading this, please consider supporting this effort.
Let’s end TB,
@JNJNews
. It’s in our power.
An open letter to
@JNJNews
. Stop pursuing the secondary patent of bedaquiline. YOU KNOW that millions of people will be unable to access lifesaving treatment for multidrug-resistant tuberculosis if you extend the patent. YOU KNOW it violates your credo.
I realize no one (probably even the party that called it) wants this election, but one thing I do miss are election night parties and the opportunity to rant together in front of a giant TV, avec snacks. Just one more thing Covid has cancelled, I guess. 😐
You may have recently come across some ‘takes’ on AI twitter that suggest AGI > nuclear war (and not even in comparison to so-called “tactical strikes.”) Steve has kindly written a 🧵that adds some much needed specificity.
Because we live in the dumbest timeline, I think we should take a moment to talk about nukes and why they are bad. Strap in friends.
1. Three are approximately 13,000 viable nuclear weapons known to exist. More likely exist
@dansukman
@KevinLi26651602
So to reiterate my point, most NATO states (and the biggest one, the U.S) now understand their membership as being connected to liberal norms. And yes, that makes it awkward for members who have allies that flirt and accept democratic rollbacks. Hopefully that clarifies things!
I’m not inclined to take intl security points from someone who both states he’s against an arms race while simultaneously arguing for unilateral air strikes against data centres. I cannot imagine discourse that is more likely to contribute to arms racing.
Is anyone aware of LLM research that addresses the political economy of the household? e.g. Can LLMs equalize time spent on housework and/or does it amplify existing household inequalities?
I realize this is still speculative, but I'm curious for early takes.
@ESYudkowsky
@Miles_Brundage
(Just going to repeat this in case you didn't see it.)
For solutions: I'm wondering why technology-denial through export controls (literally depriving an actor the means to build) is insufficient for you? 1/
@ESYudkowsky
@Miles_Brundage
A repudiation against a nuclear first strike but a willingness to tolerate nuclear exchange is a horrible condolence prize for civilians. That is my criticism, and that is why I asked about the costs you were wiling to tolerate.
Officially connected to the 5G network and it feels great. Big thanks to Moderna and Pfizer. I promise to use my new superpowers for good. Or neutral good. Or maybe just neutral.
@NathanpmYoung
@HaydnBelfield
Perhaps I’m misunderstanding you, but are you suggesting that the US/UN would back territorial succession based solely on a democratic vote? (Because they would absolutely not. But again, perhaps I misunderstand you.)
Can GPT-4 do drug discovery? No, but it can help. Let's walk through GPT-4 proposing new drugs. This is called knowledge-based screening. We're trying to fill a list of plausible compounds that could lead to new drugs based on research papers. 1/n
@ESYudkowsky
@Miles_Brundage
A repudiation against a nuclear first strike but a willingness to tolerate nuclear exchange is a horrible condolence prize for civilians. That is my criticism, and that is why I asked about the costs you were wiling to tolerate.
I had a dream that I joined a Twitter alternative that was perfectly user friendly and full of great people. If this is the height of my subconscious fantasies then god has truly abandoned me.
Our system card: This doc. outlines GPT-4's known risks & limitations. We've expanded the domains that we tested since Dall-E (and you may notice we included int'l security domains👀.)
Was going to make a spicy comment about Bayesian rationality being a veneer for what is more accurately described as techno spirituality but this meme does a better job ngl.
@ESYudkowsky
@Miles_Brundage
4 major export control arrangements continue to exist post-WW2. So, I do think they'd work for at least 15 years.
Re: ASML, there's a bilateral agreement b/w the U.S & the Netherlands restricting EUV exports, w. further discussions underway b/w U.S/Japan/Netherlands
To the non-Canadian folks who are following the protests in Ottawa: please know that most political observers in Canada do not recognize the version of events written in the New York Times.
If you want to know more, I’d recommend the following accounts:
@dansukman
@KevinLi26651602
Anyway, it's hard to cover everything over twitter. But the main point is that NATO isn't a static organization w. static members & membership. This is a challenge!
@ValerieInsinna
Rapinoe is extremely charismatic and stylish. She has “it,” as I’m sure the VS marketing department has figured out.
And even if she didn’t, it’s “interesting” that some Internet ppl see VS’ decision as a threat to the very foundations of womanhood.
There's ample IR lit. that finds a high correlation between women's equality and responsible state behaviour on the international stage. Just in case anyone's wondering about what kinds of pressures contribute to the decline of the rules-based international order.
@dansukman
@KevinLi26651602
Okay, I’ll engage b/c I think this is important to clarify. NATO’s original founding emphasized collective defence (though “western” values were implicit ). Obvious reason: the Cold War, different understanding of state security interests, and new int’l rules still in infancy.
As much as one can critique securitization theory, it’s obvious to me that the theory gets right the idea of “panic politics” leading to the socialization of exceptional measures in int’l security. This time, nothing less than an authoritarian response (track all GPUs!) will do.
@ESYudkowsky
@Miles_Brundage
I disagree that you mentioned what it takes to live; you've gestured at nuclear risk. US military ops use a collateral damage estimate that they then compare to achievable military gains before making a decision to strike. How many people are allowed to die to prevent AGI?
I hate the idea that one can “call upon” the ppl in their field, as though it were the draft. But if there is anyone in IR who’s confused about the state of int’l security in AI and wants to write on this topic, then my DMs are open.
I won't go into why Bostrom's apology was inadequate b/c many, many people have already done so. If you haven't, I encourage you to seek these explanations out--the ppl who've done this time-consuming work have been extremely charitable with their time.
@dansukman
@KevinLi26651602
How do you say 'no' to a democratic country that wants to join while also being pro self-determination?
(Portugal, which you mention, now cites its involvement using value-based language centered on democracy, btw.)