🎺 𝙒𝙤𝙧𝙡𝙙 𝙋𝙧𝙚𝙢𝙞𝙚𝙧𝙚 𝙤𝙛 𝙏𝙖𝙢𝙞𝙣𝙜 𝙎𝙞𝙡𝙞𝙘𝙤𝙣 𝙑𝙖𝙡𝙡𝙚𝙮! 🎺
“This might by the most important book you read this year” -
@AnnieDuke
September 18 at the New York Public Library
@nypl
, and via livestream (link below)
When I started working on AI four decades ago, it simply didn’t occur to me that one of the biggest use cases would be derivative mimicry, transferring value from artists and other creators to megacorporations, using massive amounts of energy.
This is not the AI I dreamed of.
OpenAI just released a model that can generate 1-minute videos.
You simply cannot argue that these models don't / won't compete with the content they're trained on, and the human creators behind that content.
What is the model trained on? Did the training data providers consent
I just wrote a great piece for WIRED predicting that the AI bubble will in collapse in 2025, and now I wish I hadn’t.
Clearly, I got the year wrong. It’s going to be days or weeks from now, not months.
Rough Translation: We won’t get fabulously rich if you don’t let us steal, so please don’t make stealing a crime!
Don’t make us pay 𝘭𝘪𝘤𝘦𝘯𝘴𝘪𝘯𝘨 fees, either!
Sure Netflix might pay billions a year in licensing fees, but *we* shouldn’t have to!
More money for us, moar!
Amazing! But, um, ants have six legs.
We are about to have a whole generation of children educated by fake videos that are completely plausible to naive audiences yet biologically incorrect. 🤯
Black Mirror has arrived, ahead of schedule.
An entire cast of deepfaked people tricked a CFO out of $25 million. “(In the) multi-person video conference, it turns out that everyone was fake”
Deepfaked shit is getting real.
The OpenAI board was right. Even on little things, Sam is not consistently candid.
Earlier today OpenAI said the resemblance to ScarJo was a “coincidence”.
It was not:
GPT-4o hot take:
• The speech synthesis is terrific, reminds me of Google Duplex (which never took off).
but
• If OpenAI had GPT-5, they have would shown it.
• They don’t have GPT-5 after 14 months of trying.
• The most important figure in the blogpost is attached below. And
SCOOP: OpenAI may lose $5B this year & may run out of cash in 12 months, unless they raise more $, per analysis
@theinformation
.
Investors should ask: What is their moat? Unique tech? What is their route in profitability when Meta is giving away similar tech for free? Do they
Quote of the day, maybe the year:
“*He is not an idiot, Elon Musk is very smart, very tech savvy.
He could do a simple Google search of any of the disinformation he has seen in the last week and see it's untrue, but instead he seeks to amplify it and perpetuate it”.
I will always stand in awe of Noam Chomsky. I just sent him my recent essay; 12 minutes later he replied with smart comments, including a point I should have thought to include. He’s 93.
Since
@OpenAI
still has not changed misleading blog post about "solving the Rubik's cube", I attach detailed analysis, comparing what they say and imply with what they actually did. IMHO most would not be obvious to nonexperts.
Please zoom in to read & judge for yourself.
𝗪𝗼𝘄! Incredibly damning new study on GenAI from Upwork, via
@baldurbjarnason
:
“Nearly half (47%) of workers using AI say they have no idea how to achieve the productivity gains their employers expect. Over three in four (77%) say AI tools have decreased their productivity
If all we had was ChatGPT, we could say, hmm “maybe hallucinations are just a bug”, and fantasize that they weren’t hard to fix.
If all we had was Gemini, we could say, hmm “maybe hallucinations are just a bug”.
If all we had was Mistral, we could say, hmm “maybe hallucinations
OMG! The GenAI honeymoon is definitely over.
“A pharma company stopped using Microsoft's Copilot AI tool, with an exec citing high cost and low value.
The company used Office 365 Copilot for 500 staff and compared it to middle-school presentations.”
-
@BusinessInsider
"This is the same climate of antisemitism that has led to the massacre of Jews throughout the centuries. This is not just harassment. This is our lives on the line."
@MIT
student Talia Khan highlights the rise of antisemitism at MIT.
Remember how Sam Altman told the US Senate he had no “direct” investment in OpenAI? and how they gushed over his apparent selflessness?
👉He didn’t mention that (presumably) he had equity in YC which likely has equity in OpenAI.
And now this:
👉“OpenAI Agreed to Buy $51 Million
Someday my children will ask me why anyone ever took seriously the thought that systems like this had anything to do with AGI.
I honestly have no idea how I will answer. Mass delusion?
(Example emailed to me by a friend.)
Congratulations,
@elonmusk
, on Starship’s successful There and Back Again!
You will not be forgotten, regardless of what certain myopic ML experts named
@Ylecun
might have implied.
You don’t need to publish to the change the world.
Now we know why Sam Altman went around the world last summer meeting world leaders: his company won’t make it big unless they can convince governments to give them one of the biggest handouts in history.
OpenAI lobbying:
(a) ChatGPT useless without copyright material,
(b) Requires special exemptions from the law.
So, LLMs are thinly disguised plagiarization algorithms and pilfering copyright is the only way to make them profitable, right?
@GaryMarcus
👉Seismic from
@geoffreyhinton
: ‘The best hope is for the world’s leading scientists to collaborate on ways of controlling the technology. “I don’t think they should scale this up more until they have understood whether they can control it”
Let us invent then a new breed of AI systems that mix an awareness of the past with values that represent the future that we aspire to.
Our focus should be on figuring on how to build AI that can represent and reason about *values*, rather than simply perpetuating past data.
I think
@elonmusk
owes Apple an apology. I won’t say this is perfect (hackers will inevitably find problems), but it seems to me to be quite thoughtfully conceived, and likely is state of the art.
This is from Apple's State of the Union
The local model is a 3B parameter SLM that uses adapters trained for each specific feature. Diffusion model does the same thing, adapter for each style.
Anything running locally or Apple's Secure Cloud is an Apple model, not OpenAI.
The number of people simply unable to imagine that the Board acted in a good faith with a legitimate and serious concern boggles my mind.
We don’t of course know what happened, but simply dismissing that possibility seems foolish to me, especially given that they were not
Prediction: By end of 2024 we will see
• 7-10 GPT-4 level models
• No massive advance (no GPT-5, or disappointing GPT-5)
• Price wars
• Very little moat for anyone
• No robust solution to hallucinations
• Modest lasting corporate adoption
• Modest profits, split 7-10 ways
Take this seriously:
@geoffreyhinton
on AI possibly wiping out humanity: ‘It's not inconceivable’
Is coding faster and having fun chatbots to play with worth a 1% risk of that coming true?
⚠️ Something terrifying just snapped in place:
• Sam Altman has acknowledged wanting to train on everyone's personal documents (Word files, email etc)
• ChatGPT has gathered unprecedented amounts of personal data
• Sam founded WorldCoin, known for their eye-scanning orb
•
Common view; almost certainly wrong. The board’s decision 𝘸𝘢𝘴𝘯’𝘵 a “𝖿𝗎𝖼𝗄 𝗎𝗉”. Instead, it was almost certainly a Hail Mary:
👉This Board’s sole job was to look out for humanity - NOT to protect the brand.
👉They must have seen danger in something Sam was doing
👉They
@ilex_ulmus
@GaryMarcus
But something here really doesn't make sense. It seems like a huge and pointless fuck up by the board and they seem more capable than that
I don’t think people quite realize the significance of this moment, so I will spell it out: OpenAI has been bluffing all along, and their CTO just gave away the fact that they are not holding aces.
75% chance that the GenAI bubble will have burst by 12 months from today.
Appalled that
@timnitgebru
, AI ethics icon, can’t lay off a lovely 87-year-old man, one of the great scientists of our time, whose journalist son was kidnapped and publicly beheaded in an act of political violence.
Her power has gone to her head;
her humanity has vanished.
A lot of people are missing the funny part. Let me break it down.
If OpenAI has actually achieved AGI, they get their software back from Microsoft.
Elon’s lawsuit has put them in a position having to prove that they *haven’t* reached AGI, even though OpenAI likes to hint that
@OpenAI
Dear
@openAI
This is baloney: “We are dedicated to the OpenAI mission and have pursued it every step of the way.”
The original mission was to be “unconstrained by a need to generate a financial return”, “not organized for … private gain”, “seek[ing] to open source technology
Just watched Noam Chomsky give a fascinating and up-to-the minute talk on deep learning, science, and the nature of human language.
I loved the first half and find myself deeply skeptical of the second
A 🧵summarizing what he said, and my own take.
I *am* an AI expert, and I don’t think AGI is coming soon. My track record is quite good:
- I anticipated the challenges of out of distribution generalization in 1998
- hallucination errors in 2001
- troubles w driverless cars in 2016
- that radiologists would not be quickly
If you don't agree that AGI is coming soon, you need to explain why your views are more informed than expert AI researchers. The experts might be wrong -- but it's irrational for you to assert with confidence that you know better than them.
Open letter to all European leaders,
The recent events at OpenAI are likely going to lead to considerable, unpredictable instability.
This highlights the fact that we can’t really trust the companies to self-regulate AI where even their own internal governance can be deeply
Hot take on Google Gemini and GPT-4:
👉Google Gemini seems to have by many measures matched (or slightly exceeded) GPT-4, but not to have blown it away.
👉From a commercial standpoint GPT-4 is no longer unique. That’s a huge problem for OpenAI, especially post drama, when many
The Gemini era is here. Thrilled to launch Gemini 1.0, our most capable & general AI model. Built to be natively multimodal, it can understand many types of info. Efficient & flexible, it comes in 3 sizes each best-in-class & optimized for different uses
"absolutely brilliant"
—Nobel Laureate Danny Kahneman
The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence
It's what I wish I had had time to say at the
#AIDebate
:)
Finally ready, free, on arXiv. Happy Reading!
No. Tesla recall, MSFT Bing fail, and Google Bard fail are NOT independent; each reflect the fact that you cannot build AI in the real world from Big Data and deep learning alone.
Too many edge cases and not enough reasoning. We need new approaches; current AI has been oversold.
No, we are not even close. AGI would require systems that
👉essentially never hallucinate
👉reliably reason over abstractions
👉can form long term plans
👉understand causality
👉reliably maintain models of the world
👉reliably handle outliers
We currently have none of that.
@GaryMarcus
Gary, I’m non-technical… but the hype around agi coming very soon and annihilating us all is kind of stressing me out. Is this a valid concern?
Holy shit! OpenAI just gave me sneak preview early access to GPT-5 (to do some red-teaming) — and it’s incredible!
What really makes me is happy is that they let me look at the training data, too, so I could do proper tests of its generalization. This thing is LIT!
And wow,
So many people are confused about the relation between human cognitive errors and LLM hallucinations that I wrote this short explainer:
Humans say things that aren't true for many different reasons
• Sometimes they lie
• Sometimes they misremember things
• Sometimes they fail
The number of great AI-generated feature films created in the next three years will be approximately equal to the number of great AI-generated novels that were produced in the last 14 months.
For people who suddenly think they're going to be 'filmmakers' with AI, how are you going to:
Block a scene?
Get coverage?
Choreograph a fight?
Move specific lights?
Give acting notes?
Deal with continuity?
The confidence vs cluelessness on display is something else.
Wake up.
If a single bug can take down
airlines, banks, retailers, media outlets, and more, what on earth makes you think we are ready for AGI?
The world needs to up its software game massively. We need to invest in improving software reliability and methodology, not rushing
If OpenAI does collapse, WeWork style, it will likely be seen as a tale of hubris.
They knew infringement was going to be a major issue.
They proceeded anyway.
That will not play well in front of juries.
In the end it could devour them.
btw this is not a joke, and here are the first two paragraphs of the piece on why the bubble would collapse soon, which I submitted earlier in the week.
The Generative AI Bubble Will Collapse in 2025
Generative AI took the world by storm in November 2022, with the release of
GPT-3 is a better bullshit artist than its predecessor, but it's still a bullshit artist.
an investigation,
@techreview
, co-authored with Ernest Davis.
OpenAI’s reply to
@ElonMusk
may turn out to be one of the biggest own-goals of all time, because (as explained below) it inadvertently revealed that they have been lying from the jump.
It may also put them in legal jeopardy over their 501(3)(c) non-profit filings.
Huge
What a mendacious company to the very core.
For years OpenAI promised to “open source technology for the public benefit” when possible, even in legal filings as a “charitable” organization.
And yet they never meant it, except as a recruiting ploy,
@AGRobBonta
@Public_Citizen
Mark my words, this reply from
@elonmusk
won’t age well.
(I am willing to put a million dollars against, if he is game, using the criteria I laid out in my 2023 essay “Dear Elon Musk”.)
I was watching this AI generated video, and at one point in it I saw an image that I immediately recognized where it was "inspired" from. It was Leonardo Dicaprio in Wolf of Wall Street. Left the AI generated image and right where I think it came from:
@GaryMarcus
@geoffreyhinton
OpenAI is in serious trouble.
👉The excerpt below is particularly damning, because the prompts that elicited the plagiarism in no way requested that the system draw on the NYT at all.
👉
@jason_kint
&
@CeciliaZin
largely converge on the overall seriousness of the suit.
👉OpenAI
Here are four examples. Again, the lawsuit includes one hundred of them. You get the point. I find this exhibit to be an incredibly powerful illustration for a lawsuit that will go before a jury of Americans. Again, it's impossible to argue with this. /14
“Cesspools of automatically-generated fake websites, rather than ChatGPT search, may ultimately come to be the single biggest threat that Google ever faces. After all, if users are left sifting through sewers full of useless misinformation, the value of search would go to
*Fabulous* question: How come smart assistants have virtually no ability to converse, despite all the spectacular progress with large language models?
Thread (and substack essay), inspired by a reader question from
@___merc___
Deep Learning Is Hitting a Wall. What would it take for artificial intelligence to make real progress?
#longread
in
@NautilusMag
on one of the key technical questions in AI.
Board is completely on the ropes, almost everyone hate them, they have no equity at stake, and are in a no-win situation.
Yet they are still apparently fighting to have a replacement board that respects the mission.
That’s integrity.
And suggests they can’t unsee something.
Francois Chollet: OpenAI has set back the progress towards AGI by 5-10 years because frontier research is no longer being published and LLMs are an offramp on the path to AGI
X has a moral duty to take steps to reduce bots,
@elonmusk
.
Something has clearly broken in the codebase in recent days,
Now getting pages and pages of these; most w no pic, no followers, just joined, blatantly autogenerated names.
5 minutes of coding could get rid of most.
Ok, game on. Since you asked, here are a few of my credentials; looking forward to comparing and hearing yours.
👉 My 2001 technical book on cognition and neural networks (The Algebraic Mind) anticipated many of the problems current AI is facing (including many that
@ylecun
now
What has
@GaryMarcus
built in the field of machine learning so far? How is he relevant in the AI industry? We need to stop celebrating these self-proclaimed AI experts who haven't even written a single technical AI book and yet speak on behalf of the AI community about AI
Midjourney’s Merry Christmas to all
- We are likely using copyrighted materials
- We will ban you if you try to find out
- We may sue you if you try to find out
- If you get sued because the output that you generated violates copyright or trademark laws, don’t look at us.
Counterpoint: scaling alone hasn’t even brought LLMs to reliable multi digit integer arithmetic.
Also: exponential progress in Go playing hasn’t led AlphaGo to take any interest whatsoever in human territory or even led it to ask what a stone is.
Satya Nadella says Windows PCs will have a photographic memory feature called Recall that will remember and understand everything you do on your computer by taking constant screenshots
must-read new study from
@Google
confirms all central claims of Deep Learning: A Critical Appraisal (2018):
- machine learning often generalizes poorly
- extrapolation beyond training data is key
- urgent need for better ways of adding in domain expertise
Top 10 reasons
#deeplearning
isn’t getting us to artificial general intelligence. A critique of deep learning, 5 years into its resurgence, by
@garymarcus
Gotta love how the opening of
@ylecun
’s recent lecture
- complete confirms that his NYU colleagues Davis and Marcus in 2019 (Rebooting AI) diagnosis and problem formulation was—and remains—completely correct.
- yet somehow entirely fails to cite his colleagues’s analysis.
Yann LeCun
@ylecun
delivered a lecture on Objective-Driven AI.
He began with a reality check: "Machine Learning falls short compared to humans and animals!"
Here's his insight on constructing AI systems that learn, reason, plan, and prioritize safety:
1/5
Remember how Sam Altman told the US Senate he had no “direct” investment in OpenAI? and how they gushed over his apparent selflessness?
👉He didn’t mention that (presumably) he had equity in YC which likely has equity in OpenAI.
And now this:
👉“OpenAI Agreed to Buy $51 Million
Within 12 months, the GenAI bubble will have burst.
• The economics don’t work
• The current approach has reached a plateau
• There is no killer app
• Hallucinations remain
• Boneheaded errors remain
• Nobody has a moat
• People are starting to realize all of the above.