🎙️Bittersweet as I announce I'll be putting AGI Show podcast on pause to put all my time into my new defensive AI startup
@HarmonyAISafety
. I'm reminded yet again that startups take 110% of your energy & focus, which is why I'm stopping a side project I've otherwise loved 🥲.
🧵📣New jailbreaks on SOTA LLMs. We introduce an automated, low-cost way to make transferable, black-box, plain-English jailbreaks for GPT-4, Claude-2, fine-tuned Llama. We elicit a variety of harmful text, incl. instructions for making meth & bombs.
I've been holding off on posting as more facts come out about SamA firing at
@OpenAI
. I'll provide my first tentative take based on available information:
* Looks like from most credible accounts this was due to a divide on the board on safety vs commercial tensions in the
@venturetwins
Actually a good thing. We need people trying adversarial attacks on these systems, ideally in public like this. One of the surest ways to find & fix edge cases while still in limited beta & not that powerful.
This has been one of
@OpenAI
's arguments for releasing capabilities.
Updated take on OAI situation, given we seemed to have reached a relatively stable state for now:
* Sam may have done something wrong here to deserve to be fired, but the board has provided very little evidence to prove that assertion.
* Board maybe have felt they "had to act"
I wanted to share my story about why I left fintech in San Francisco to tackle food’s most exciting challenges alongside the
@vowfood
team. Let me know what you think! --
Australia's current crop of unicorn $1bn+ startups:
Canva
Airwallex
Immutable
SafetyCulture
CultureAmp
Linktree
GO1
Pet Circle
Scalapay
Which of them have a "unique" reason to have been successful in Australia vs elsewhere in the world? Little to none.
The message you hear
1/ Imagine a future where machines can think, learn, & understand the world just like humans. Sounds like science fiction? I believe it's closer than most think - probably in our lifetimes.
A friend of mine captured why I love living in Sydney in a nice succinct way: "There's enough hustle & bustle to keep you engaged, and enough balance, safety, space & nature to keep you happy."
My thoughts exactly 👌🏽.
Who says a warehouse + lab can't be both functional *and* livable? Big shoutout in particular to
@TNoakesmith
who's always been massively focused on making our space at
@vowfood
as beautiful, human, and livable as possible 🌱👩🏽🔬👨🏻🔬.
My team and I are about to embark on **one of the most ambitious sustainability, manufacturing, & construction projects in human history**, to build one of the biggest cultured meat production plants in record time. We have to do this to save our planet and feed the future.
Tried
@TransferWise
for the first time this past week to make an overseas forex transfer. Highly recommended -- low, transparent fees + smooth onboarding. Such a step up over the banks / Western Union. Awesome to see that they're using
@Plaid
to make the process even better :)
@DanHendrycks
@elonmusk
@xai
Very glad there's a credible and knowledgeable AI safety person involved.
I really hope you can influence their direction to ensure safety, or call it out if safety critical advice is not being heeded.
**Call to action to folks in the EU** -
This isn't good. EU foundational model makers (Mistral, Aleph Alpha etc) lobbying to block sensible foundational model regulation in the EU. If you care about AI safety, it's time to talk to your legislators and rally advocacy efforts.
However, in a Council's technical meeting on Thursday, 🇫🇷 & 🇩🇪 came out vehemently against ANY rules for foundation models. This opposition results from a strong push from their national champions, Mistral & Aleph Alpha respectively, which have strong political connections. 4/8
📣 EP10 AGI Show w/
@ryan_kidd44
out! We talk ML Alignment & Theory Scholars (MATS) program that accelerates people into AI safety research roles via mentorship, seminars & connections. If you're interested in technical AI research for catastrophic/x-risk, this ep is for you!
@AndrewYNg
@geoffreyhinton
Well argued Andrew.
On your point about zero risk of AI human extinction, is there somewhere you've outlined your detailed reasoning that I could engage with? I'd love to better understand your perspective here.
A great example of the automation we're using at Vow to make cultured meat research faster, better and ultimately take us to cheap, delicious food for the world! Check it out!
Good research is an exacting endeavour. Accurately quantifying the maturity of some of our cells can be a long and intensive process - a perfect candidate for automation!
Here is a water/dye run of our protocol in action on one of our beloved
@opentrons
robots 💪
At some point, I'd like to share my
#food
and
#agriculture
#startup
ideas with more context + better format, but didn't want to let perfect be enemy of good:
Some are _categories_ of solutions (helpful for me). If any pique your interest, reach out!
**Engineers, scientists, foodies, cultured meat followers** - listen to
@MarieMakesMeat
@alexshirazi
& I talk high-throughput automation 🤖 + making scientists the most creative they can be👩🏽🔬👨🏾🔬 to solve the world's biggest sustainability challenge🌍!
A great visual from
@CellAgAustralia
on the lay of the land for Australia's cell ag ecosystem.
A growing and exciting ecosystem to be a part of -- let's reinvent the future of delicious, sustainable food for the world!
Message me if you're keen to join the effort!
There's an age old trend, often in young people, of jumping from idealism ("we can make the world perfect!") to cynicism ("the world is broken beyond repair!"). But to make change, we have to find pragmatism ("here's how *I'm* going to make a difference, even if imperfect").
We introduce a way to automate jailbreaks by using one jailbroken model as an assistant for creating new jailbreaks for specific harmful behaviors. It takes our method less than $2 and 10 minutes to develop 15 jailbreak attacks.
Models aren’t very smart right now, but as they get better, misuse risks go from problematic to catastrophic, making it possible for anyone to get detailed instructions on the most atrocious crimes – e.g. novel pathogens, building a bomb, launching cyberattacks.
The
#1
forecaster on
@slatestarcodex
's 2022 prediction contest,
@ryankupyn
, beat out 500+ forecasters & even prediction markets.
He & I sat down on The AGI Show podcast to talk forecasting AGI timelines & his own predictions.
AGI Show Ep9 w/
@ARGleave
(CEO,
@farairesearch
) is out!
FAR is one of the fastest growing & most respected frontier AI safety orgs in the world.
We talk FAR's founding, research areas (adversarial robustness, interpretability, evals & more), & opps in the AI safety ecosystem.
Startup equity comp can have serious gotchas that mean the difference between 💰 / 😵. That's why when I got to
@itsjustvow
, I pushed for employee-friendly ESOP features from Day 1. Sharing them to make them the norm across startups!
Happy to answer Qs!
@blackbirdvc
just posted published data on their first fund returns -- $3.40 returned for every $1 invested, $6.50 of remaining carrying value, 57% IRR. A fantastic vindication for Australian startups tackling global problems cc
@canva
@zoox
@CultureAmp
Hear a lot of people (mostly under 35yo, too young to remember evils of communism) hating on capitalism.
Feel like hating capitalism is like hating automobiles: totally reasonable to call out problems (climate impact, air pollution, traffic etc) but almost no one would
If you need to fine-tune models, check out Brev. Find low-cost, available GPUs and get set up on them fast.
Worked great for us for our recent AI research project fine-tuning Vicuna 33B.
🚨Introducing Brev 2.0🚨
• Cheap, actually available GPUs — $2/hr A100s 🤑
• The simplest GPU provisioner 🤙
• Open-source tool to set up python & CUDA once and for all 😅
We hope this is the easiest way to fine-tune anything.
There are 3 main changes. Here’s why & what 👇
Are there examples of tax jurisdictions that tax capital at significantly higher rates than income?
Plenty of the opposite, but struggling to think of examples of this kind. Feels perverse to me - taxing productive work highly, passive wealth lowly. I'm thinking more about why.
👷🏼♀️🥘A quick story of
@Vowfood
Engineering & how it came to be 🥘👷🏽♂️!
18mo ago, I came across a pair of scrappy founders named
@TNoakesmith
&
@peppsyd
. I was skeptical since most startups aren't too good...except the ones that are⭐️.
**If you or anybody you know wants to be involved, DM me.** Particularly looking for people with a construction, manufacturing or bioprocess background, but first and foremost deep problem solvers willing to give their absolute everything and do the work of their lives.
When you (read:
@peppsyd
) get a novelty sized cut-out of
@itsjustvow
logo as Xmas party gimmick & have to figure out what to do with it come Monday after.....
Meanwhile, a human-in-the-loop can efficiently make these jailbreaks stronger with minor tweaks. We use this semi-automated approach to quickly get instructions from GPT-4 about how to synthesise meth 🧪💊.
TLDR: Tentatively, SamA firing looks like
@OpenAI
's not-for-profit board doing its job. Now they need transparency to justify their decision & show world their reasoning. Great moment to show the world what they believe in & where
@OpenAI
goes from here.
I've been holding off on posting as more facts come out about SamA firing at
@OpenAI
. I'll provide my first tentative take based on available information:
* Looks like from most credible accounts this was due to a divide on the board on safety vs commercial tensions in the
Incredible work by
@Opentrons_
to rapidly ramp up COVID testing at a fraction of the cost of doing so with other vendors and at much higher throughput.
We've been working non-stop to create a high-throughput system for COVID-19 testing.
Opentrons can get surveillance systems to labs within 5 days of ordering that can ramp to 2,400 tests / day.
@ImpossibleFoods
's 2019 impact report is incredibly inspirational. Recommend it to anybody out their trying to make positive impact in the world.
(their 2018 one is definitely worth reading too)
Incredible shot of Sydney from overhead, courtesy of my dad who loves a window seat & is much more observant than I am.
Helps capture how Sydney Harbour is the largest natural harbour in the world! Most of it is actually not in shot, would sit below the bottom cutoff.
@rossaokod
@OpenAI
If this ends up being true, then this whole thing will have been a real shit show and the board will look really, really incompetent.
Holding judgement until it's confirmed as true...
@ShafronTom
@OpenAI
There's no obligation for them to do so. They are the board specifically put in place to govern and make those decisions. In many real world situations, getting broad stakeholder support is not realistic or counterproductive to these types of major decisions.
Check out this
@blackbirdvc
podcast ep with
@vowfood
's founders
@TNoakesmith
@peppsyd
to understand what our mission to sustainably feed billions of people worldwide is all about!
Did I mention we're hiring for incredible scientists and engineers? Reach out if interested!
I absolutely love what I do here at
@VowFood
. Get to work on amazing engineering, alongside a caring team, all in pursuit of a mission to sustainably feed our world.
We're hiring for a Software Eng (Backend) to come join us. Please apply or share 🙂!
Impressed by openness & clarity here shown by
@OpenAI
in calling out by far *the biggest risk* with AI systems -- a superintelligence that's not aligned with humanity & would pose an *existential threat*. Fully support their call for an IAEA for AGI.
I've started napping for ~15-30min almost daily in the afternoons, pretty much anytime I'm convinced I'll be more productive & happy with a nap than without.
Such a good decision! Better than coffee or fighting it.
Recommend it! Every office should have space for it.
Saddens me to hear many Aussies regularly downplay how much we can positively impact the world 👎🏽
There are no more powerful words than the ones we use to speak to ourselves 😶
*🇦🇺🌍 We have done & can do so much good on the global stage 🇦🇺🌍*
That begins w/ believing we can.
@justinkan
I've gone from heavy drinker (university) -> moderate drinker (post-university) -> almost no drinking in the last 2 years (<1 drink/week avg): been an incredibly positive change. Better health, more time, healthy relationships. Recommend it for everyone, especially busy people.
@HWejberg
@Noahpinion
Hmmm this is interesting. I did use a UN carbon emission estimator recently and found that flying is *my* biggest contribution to carbon emissions. So it may not be irrational in all cases to curb such behaviours. Maybe we should just point everyone to a good carbon estimator?
@nic__carter
Should be noted that, at this stage, we haven't learned anything that shows that OAI problems were caused by unique governance structure. A board fired a CEO, for reasons we don't yet know. Can happen in any standard corp.
Need to know more before we draw too many conclusions.
This misuse risk persists despite significant attempts to stamp them out by LLM providers. This is an *unsolved, fundamental issue* for today’s LLMs. RLHF and fine-tuning aren’t preventing them.
@GaryMarcus
On the first one, it is publicly disclosed on the OpenAI website:
"[Sam's] only interest is indirectly through a Y Combinator investment fund that made a small investment in OpenAI before he was full-time." ()
On the second:
It's possible that he
@ai_risks
& top AI leaders have put a spotlight on existential risk of AI. Critics have voiced doubts, but their counterpoints fall short. The risk is real & should be a global priority. Here's a detailed rebuttal of the criticisms & how they fall short:
🧵Two things that have been lost in recent AI safety debates:
#1
) Whether you care more about existential risk -or- societal issues like bias, fairness, misuse, there's concrete research & effort that can benefit both e.g.
* Explainability
* Model evals
* Cybersecurity
* (more)
Honestly think
@Superhuman
is worth the hype. If you process lots of emails per day & want to do it faster using keyboard shortcuts, omnibar, templates etc. it's for you.
Analogous to productivity improvement of VSCode vs a less capable text editor (minus the extensions).
@Peter_0_0_g
@OpenAI
Definitely agree there's an argument they shouldn't have ever taken capital in their capped for-profit entity. Then again, capital is key to safety research on frontier models too. They're trying to thread a fine needle. The board here seems to have decided SamA was going too far
I switched to decaf recently for my daily coffee, only drinking full-strength coffee when I *really* need it. I'm happy to report that full-strength coffee now *WORKS*. I am buzzing right now 😂🚀🐝🐝🐝🐝!
1/ Live taste test of
@V2FoodOfficial
's new plant-based Rebel Whopper @ Hungry Jack's Australia
(for overseas ppl: "Hungry Jack's" = "Burger King")
By "live", I mean I ate it 10 minutes ago and took handwritten notes, I can't handle any more mid-meal social media than that
I'll be speaking on an AI Safety panel this Friday 12.15pm at Hack Sydney conference's AI Village alongside Harriet Farlow & others.
Come listen to the panel or find me around the conference!
Conference deets:
@ai_risks
& AI leaders have put a spotlight on the existential risk of AI. Critics have voiced doubts, but their counterpoints fall short. The risk is real & should be a global priority. Here's a detailed rebuttal of the criticisms & how they fall short:
I teamed up with philosopher
@sethlazar
and AI impacts researcher
@random_walker
to investigate the "Statement on AI Risk" that proposes:
"Mitigating the risk of extinction from AI should be a global priority".
tl;dr: We're not convinced.🧵
@deliprao
@abacaj
Makes sense. Someone should spin up a quick & dirty open-source UI clone to use API.
I'll put it on my project list, though I'm happy for anybody to ship it before I do!
We need (a) deeper investment in AI safety research for catastrophic risks and (b) stronger mandatory controls on this technology to prevent misuse risk and many other societal harms.
Just got Issue 1 of Grow magazine by
@Ginkgo
and it contains a scratch-and-sniff card of the scent of a flower that for a scent of a flower that *went extinct in 1881* (). I literally have goosebumps all over.....and it really is a beautiful smell 🌷😲
What's
@ItsJustVow
been up to that's made it to
@ColbertLateShow
?
It involves woolly mammoths, delicious new meat experiences, & the future of our planet.
-- sign up to be first to try delicious new meat experiences
I absolutely adore the cafe culture in Sydney ☕️. Having a unique, cozy, welcoming cafe anyone can go to w/ delicious coffee & food where you can sit, read, talk, &work. It's such a simple and wonderful luxury that's ubiquitous across Sydney and much of Australia 🇦🇺.
Hey Sydney friends! 📢 Launching a *monthly AI safety meetup* covering AI risk, cybersecurity, regulation & more. Anyone interested, join us!
📅 FIRST MEETUP: Next Thu, Aug 24th, from 6pm in Sydney CBD
I would do a lot *never to use JIRA again*. Wishing the
@linear_app
team huge success, I'll definitely be one of the first to try it out.
@HarveyMultani
Fantastic take by
@LizSpecht
on the need to start addressing the root causes of the coronavirus pandemic, not just the symptoms, if we want to avoid this in the future. One of the biggest root causes is the massive scale of industrial animal agriculture:
== Sydney AI Safety meetup ==
For anybody interested in AI safety, regardless of experience level — from AI risk to ethics to cybersecurity, come on by to our November meetup!
Nov 23 6-9pm @ Aurora Hotel near Central. See you then!
Absolutely love my Kindle. In a world full of distractions and negative news, it feels so incredibly grounding, important, and straight up *joyous* to absorb long term perspectives and nuanced ideas through books and long form writing.
Early career biologists in Australia, we at
@VowFood
are hiring the next cohort of Junior Research Scientists to tackle one of the world's biggest sustainability challenges! Applications close July 12. Share with any biologist friends as well!
Making career switches across fields (e.g. financial software -> food & biotech -> AI safety) or roles (e.g. builder -> manager) can be scary. I've found tricks over the years to make it fun & rewarding. Check it or share if relevant for you / a friend!
1/ Highly recommend this *incredible* interview with
@holasammy
,
@blackbirdvc
partner & one of
@vowfood
's first investors, who believed in us & the entire alt protein sector before anybody knew our name.
Interview:
@mikeee
Agree entirely with your point re: referendum booklet. "No" case was so much stronger. You could argue that was due to substance of the side rather than simply execution, but it definitely felt like it could have been a lot stronger.
With all attention on
@OpenAI
, story that's flown under radar is
@AIatMeta
dismantling their Responsible AI team. You do the math -- will lead to irresponsible AI development.
Reminder that we can't trust tech co's to self-regulate AI. Important we have credible, effective reg
My company
@vowfood
is currently making our own batches of hand sanitiser to share with those who need it most. Check out the article to learn more about how we did it and, if you're in need, how you can get some hand sanitiser.
Panic buying is leaving people vulnerable. We’ve felt it, our friends and family have felt it and we realised some of you might be feeling it too.
So we decided to do something about it for our community.
Every bit counts. We're ALL in this together,
🧠Want to know what everyone in AI technical alignment is working on in one enjoyable ~90min listen? I thought so!
Sat down w/ deeply knowledgeable & great communicator Thomas Larsen (Strategy Director, Center for AI Policy, prev MIRI,
@MATSProgram
) on AGI Show for the rundown.
So so cool
@Ginkgo
. And *yes*, we at
@vowfood
are well on our way to doing the same to produce delicious, sustainable, low cost cultured meat for the whole world 😉🤖⚙️!
A week ago,
@RosenthalHealth
asked Dr. Fauci: “If you had a national plan for testing, what would it be?” His answer: “Surveillance testing. Literally flooding the system with tests.” At Ginkgo, we’re proud to do our part.
Coming from Silicon Valley, the wealth of talent, ideas & resources in Australia continually impresses me. Got a massive part to play on the global stage.
Look forward to pushing it forward much further still!
First task: add
@vowfood
to that list 😛 ! We're not far off😉
Australia's startups are on a roll! 🔥🇦🇺
We've seen the big wins; ex-pats return, unicorns, soonicorns, breakout stars and a community that shares in the success of their mates.
🔭Here's the view from AirTree:
A fantastic discussion of Vow's business and why we are taking the approach we are. If you're an investor or anybody interested in how we make cultured meat a huge success around the world, take a read of
@jamestynan
's fantastic piece.
When did startup Twitter turn into a bunch of self improvement BuzzFeed listicles?
They've absolutely exploded over the last few months and are overrunning my feed......
I'm just as giddy to get my new Macbook Pro M2 Pro as I was the day my dad brought home our first ever home computer, the Power Macintosh 5200 😁😁😁 !
Some things (thankfully) never change 👶🏽!
People keep telling me, flights to Australia are sooooo long 😵💫✈️!
I'm just thinking - 24h of distraction-free thinking // reading // writing // movies // Simpsons // naps? Sign me up!
Just don't tell anyone in-flight WiFi has been invented 🤫.
Anthropic's governance commitment to giving away significant power to ensure safe & beneficial outcomes of its tech is incredibly admirable & important. Well worth the read & huge credit to the Anthropic team 👏🏽.