Founder, CEO, (Serial) entrepreneur, engineer, stochastic parrot. Heidelberg, Germany. Ex Apple AI R&D - Personal, for inoffensive marketing:
@Aleph__Alpha
We're changing the game again. Tokenisation is the foundation for all LLMs, the reason why multi language models aren't fair across languages, why fine-tuning is hard and the (language) learning problems is so compute intensive. Super proud of the great work.
📢Forget about classical
#tokenizers
! Embed words via sparse activations over character triplets. (1) Similar downstream performance with fewer parameters. (2) Significant improvements in cross-lingual transfer learning. Joint work with
@Aleph__Alpha
👉
ChatGPT is basically useless for high-value work. An expert (e.g. lawyer) needs explainability, sources and a way to influence and control outcomes. I am proud of what our team achieved with AtMan. Making multimodal generative AI trustworthy.
@Tangrenin
Something like this can even happen to humans with a normal upbringing and environment. And it’s a ride you should feel lucky to be on if you get the chance (don’t make any rash decisions and keep the seatbelt on while it’s going)
New paper from our research team: First true multimodal image synthesis. You can prompt any combination of texts and images. Also shows the power of our multimodal embedding. Check out the Catronaut in the paper.
Explainability is now live. For every generated content, trace the origin of (factual) information through the inputs - even in multimodal space. Give it a try:
We believe strongly in the open-source community & the reproducibility of research. That's why I am happy to publish MAGMA code + checkpoint. All our language models now have full multimodal functionality = no reason to use pure language models ever again
It's here!
Upload *any paper* to Explainpaper and start instantly getting explanations! Ask follow up questions if you need a more in-depth answer.
Go to and go read all the papers you've been saving! 📝📝📝
"US Marines Defeat DARPA Robot by Hiding Under a Cardboard Box" - This is what happens if we build AI based on object perception without world knowledge:
Expect every data source to be contaminated from here on. There are some clear signs like the examples here, but I've seen many more less obvious. First, token distribution learned by LLMs converged towards all text - now, all text converges towards LLM distribution.
I did some coding this weekend. Trying different ways to work with AI. Verdict: It's awesome for people like me that are rusty but know many of the eng concepts. If you cannot read and fix code yourself you will hit a wall you cannot get through. Haven't had this fun for years.
@Aella_Girl
We cannot evaluate every claim ourselves all the time. A reputation system for heuristic confidence bias makes sense. And there are already non standard paths to reputation. Blind belief is ofc not a good idea but to some degree just how many humans work.
Nice Forbes article outlining our leading role in Europe for generative AI and LLMs. And while I want to remind everyone that model size is not everything, I am excited about our 300b luminous-world incoming in 2023 (already in test deployment).
Visual QA models are impressive, but what if you had the flexibility of a model like GPT-3 in a multimodal space? Here our model learns and applies new visual and language concepts with just one example. All the GPT power is included: This will change CV.
Um mit moderner KI gigantisches industrielles Potential zu heben fehlt den US-Tech-Giganten eine kritische Ressource: Branchenwissen - der Schatz auf dem wir in Europa sitzen! Egal wie viel zehntausende GPUs irgendwo stehen, unsere Kunden und Partner haben eine starke Position in
The current situation presents the best opportunity we'll ever get to solve one of the challenges of our generation: the digitalisation of our government.
* We can leapfrog a technological generation enabling a level of efficiency that was impossible with earlier technology
* A
Every social media should have a switch that would give me the opposite of what the site has learned is what I like. Sort the comments in inverse order to my preference, show me my anti-bubble. I want to see it.
I am not a huge fan of bans, but can see why people think this is a necessary step. Long-term the solution for Europe however cannot be developing into an anachronistic vacation destination or hiding behind cookie-accept-buttons.
I am still doing final interviews for candidates. Seeing the phenomenal talent that notices our work and wants to join, moving from all around the world to Heidelberg (or Berlin) is deeply humbling. With all the resources our communication takes - this makes it all worth.
Awesome work by
@kerstingAIML
,
@mayukh091
and Björn Deiseroth, Samuel Weinbach & Patrick Schramowski: some brilliant people that (for some unexplainable reason) don't have twitter
Ich bin stolz, dass wir wieder einen Schritt vor dem Markt mit entscheidenden Funktionen live sind. 2021 noch Multimodalität - jetzt ist Erklärbarkeit live. Die Herkunft jeder Information, halluziniert oder aus Quellen jetzt transparent + reproduzierbar!
China now has a net inflow of scientists. In contrast, the US, which greatly benefited from talent inflow in the past, saw an outflow in 2021. Data from
@OECD
This is one of the things Germany is in a great position to do. We can be at the top of the AI manufacturing revolution - few countries are as well positioned to make this work.
When people see the robot installation by country.
They think it'll be easy for the US to reindustrialize, or for underdeveloped countries like India to industrialize and catchup
Here are the reasons why, it's very difficult for both type of nations:
1/7🧵
Is my LLM dumber than a hamster? Are you wondering if LLMs can reason? Testing general reasoning capabilities of the best models I also interacted with GPT4 and want to share some funny and hopefully insightful results.
Congratz to
@EMostaque
and the whole
#stabilityAI
team for the awesome work and launch. A small correction: we are running >1000 A100, just "only" 512 in our own cluster :)
By combining open communites with serious compute resources,
@EMostaque
has set up
#StabilityAI
as the nexus of the present generative AI movement, while OpenAI and Google hold back. AI for the people, by the people: text, image, video, audio, vr, games, coding, medicine, tools
It's amazing how the company changed in the last year. I feel honoured to have many of the very first team members still with us and then there are so many world-class new joiners. In 12 months again we'll be a different company. To the fun and responsibility in 2024.
GPT does not understand chess! There was some back and forth going around on GPT playing chess and I was confused so I did my own experiment. And I found that there are valid board states for which GPT returns invalid moves, meaning that there seems to not be an intrinsic
Davos was crazy. While last year genAI certainly was in focus this year everybody seems all-in on AI. Phenomenal view from our (somewhat remote) apartment. These days were also the start of Jan (VP strat & coms - coming from Palantir) and so this is also onboarding pizza.
My econ professor told me decades ago: reducing something undesirable (cost, CO2, ...) alone cannot be your optimisation function because there would be an easy answer: close down the company and you have the optimal possible value: zero
Our data center is now ranked
#72
of all supercomputers worldwide. There is no other European commerical system like it. Thanks for the teams of
@HPE_AI
&
@NVIDIAAI
for the help during setup. The 512 A100 already go plenty of brrrrr.
Many people are still obsessed with installing residential solar. Looking at German spot power prices (that get negative regularly) I think spending more on residential storage (together with giving households access to spot power) will be a much bigger ecological impact.
Why no one has home storage — YET
Battery pack prices have totally collapsed — at high volumes, this means that enough batteries to *totally flatten* a home’s peak electricity demand cost no more than about $1,500. That means never paying for electricity from 4-9PM.
Don't let yourself be fooled by the tale of the 20-something college dropout millionaire entrepreneur. If you are 50+ and had a great career so far - now is the time to change the world.
Source: Success drivers for start-ups from
@AndreRetterath
Don't miss
@JonasAndrulis
next month at
#SuperAi
.
Jonas, CEO of
@Aleph__Alpha
, is on a mission to empower businesses and governments with sovereign generative AI technology that puts them in the driver seat.
Meet AI's foremost experts in Singapore.
A lot of smart people are thinking about how to best regulate AI. From a European perspective the first question should be: Hoe can we not just regulate but build the future? Our contribution can be much more than the cookie banner.
@Fliederlilo
@miele
Derivate sind nichts grundsätzlich verkehrtes. Oft werden sie gerade zur Risikominimierung eingesetzt. Es gibt Kennzahlen die Risikoexposure zeigen - die Nominalbeträge sind da aber eher irreführend.
@anonynaut
I would love to have the experience of visiting a cathedral for the first time. A medieval farmer walking weeks into a city, seeing the towers appear on the horizon. The light painted by colorful glass hearing the organ fill the room - I see that could be a day to remember.
I had a chatbot write a random paragraph & then asked llama3 if this is AI-generated content. Llama3 fitted with our explainability showing which tokens have most impacted the answer. No single token was overwhelmingly responsible but guess which two are strongest contributors.
@xlr8harder
We're already in a world where in business tons of text, reports, wikis are generated but never read. I recently had a conversation about why all legal documents should be symbolic. AI business processes will trim all the fat and just get shit done.
We're hiring. An opportunity to contribute to the most pivotal shift in tech history towards sovereignty of liberal democracies and world's best enterprises. Turbocharge your career and get boosted by the most inspiring group of people I have ever worked with.
Hello AI researchers, IPAI Aleph-Alpha Research is hiring! Following a great Series B round in autumn, we are looking for experienced researchers and software engineers to join us on our mission to build sovereign human centric ai in Germany.
One of the secrets of tech giants is that they’re ruthless and break rules whenever strategically optimal. European companies are often naive in that way. This “good guy” compliance sadly gets you nothing but we’re still doing it.
Next week Aug 11th
@Aleph__Alpha
and
@Robert_Baldock
are exited to welcome
@TheGregYang
of Microsoft Research to our series of expert ml seminars. Presenting muTransfer: phenomenal work on efficiently scaling up training for giant models.
@soniajoseph_
Some people may use EA, alignment or longtermism to try and invalidate current legitimate concerns. Ideologies already had this effect - attracting horrible people using this to cover their actions. That’s why I think we should never abandon deontological ethics fully.
The
@nytimestech
has a great piece on the multimodal
#GPT4
. Our multimodal models are out (and open source) since 2021 () and while GPT4 wants to make these ingredients into yogurt parfait, I like luminous' taste better.
Our large multimodal luminous model continues to amaze me. Not only does it read a hand-drawn treasure map, knows what the red "X" means (there's treasure there!) but can answer questions about it that automatically translate the contents if asked in German.
During our work with partners we found a capability of our explainability I have never seen somewhere else: link to conflicting information. You can now for every factual claim see if and where you have factual contradictions in our knowledgebase. Works for text and images.
The EU AI Act negotiations ended.
One contentious issue was the regulation of foundation models, particularly open source ones.
Kudos to the French, German, and Italian governments for not giving up on open source models.
Juicy part:
"The legislation ultimately included
Looking at past scientists warning the world from tech that is now ubiquitous (e.g. electricity) and past authority consensus condemning social rights we now see als obviously just (e.g. interracial marriage).
If this does not humble your conviction today, you should stop and
The week continues with high speed: tomorrow we will present the 2nd
#nittygritty
with
@PfeiffJo
. I am extra happy about this because adapters played a great role for MAGMA. Come join and learn about the next evolution for transformers.
@officialmcafee
Government: You owe us money. It’s called taxes.
Me: How much do I owe?
Gov’t: You have to figure that out.
Me: I just pay what I want?
Gov’t: Oh, no we know exactly how much you owe. But you have to guess that number too.
Me: What if I get it wrong?
Gov’t: You go to prison
Finally selling the MacBook I started Aleph Alpha with. Back then (2019) the tech vision was "totally unsupervised" going after
@ylecun
's cake. The ZX Spectrum was my 1st computer and I am happy we changed the logo (wdyt?). ML Summit was where we presented work at Apple.
Pattern recognition is so strong that it gives the impression of reasoning within distribution. Push the prompt/ context far enough out of distribution (of training data) you'll see errors that no human understanding the rules would make.
what is the best explanation of how gpt can both play chess at 1800 elo if the game evolves conventionally, but it totally falls apart when random moves are introduced, even suggesting illegal moves?
what's going on?
Sure people are crazy about AI right now, this has brought the world's best minds and a lot of capital together. Compared to the 2000s where Investment Banking was pulling all the brilliant tech talent to build structured finance, AI is already making the world a better place.
Wir sind mitten in der turbulentesten industriellen Revolution und die heutigen Entscheidungen werden die Welt von Morgen prägen. Das Umfeld für alle folgenden Generationen wird von KI gebaut und nur mit eigener Kompetenz können wir sicherstellen, dass unsere Werte darin sicher
Sie haben Fragen zum Thema Künstliche Intelligenz? Beim 4.
#Transformationsgespr
äch wird Bundesminister Robert
#Habeck
mit Prof. Dr. Judith Simon und
@JonasAndrulis
auf Fragen aus der Community eingehen. Stellen Sie jetzt Ihre Fragen in den Kommentaren!
#KI
Diese Entscheidung ist mir extrem schwer gefallen, aber es geht leider nicht anders: Wir müssen uns von unserem Schloss
#Heidelberg
trennen, die Instandhaltungskosten sind in diesen Zeiten einfach nicht mehr verantwortbar. Wir verhandeln aktuell mit mehreren ausländ. Investoren.
German innovation strategy in the past:
t-3: don't invest in huge experiments
t-2: complain - results of others for new method not convincing
t-1: realize a new market has already developed
t-0: try to build something ourselves
t+1: become US-customer
@alth0u
No. It’ll cause prohibition style gangsters dealing out dance clips in shadowy back alleys. Collapse of law and order. Perfectly choreographed shoutouts.
My 1st contact with deep learning was in supervised computer vision. Even ten years ago AI was better than humans for some diagnostic tasks, it's great to see the progress and I cannot wait to build a new generation of tech to support medical experts.
Danke für den holistischen Blick
@AnKaNez
auf das Thema.
Entscheider in Europa, in Industrie & Verwaltung müssen diese Chance ergreifen, Technologie nutzen & stark in eine neue Era starten. Dann wird
@Aleph__Alpha
weiter in der Weltspitze Boden gutmachen.
With AI developing a incredible speed it is difficult to adjust ethical requirements properly. Many established techniques are barely usable for a new generation of models. Based on our research I wrote an article about how to approach these challenges.
What to take from this? We cannot trust current AI for critical and complex problems. We need a way for humans to understand and verify. Responsibility can only ever be taken by humans and explainability is a key ingredient.
Eins meiner Highlights dieses Jahr: OMR Festival am 09. Mai in Hamburg. Mein Thema: "Der letzte Flaschenhals: Ist künstliche Intelligenz auf dem Weg zur eigenen Innovation?" Es gibt viele faszinierende Programmpunkte - kommt vorbei!
The best enterprises and governments are now all confronted with one reality: employees, partners, vendors, authorities, .... will put data into ChatGPT. IP will be compromised, confidential data leaked and mistakes will happen because 'the AI said so'.
Having spent a good part of my career with supervised computer vision I was ready to celebrate "Segment Anything" () as a "GPT3-Moment" for vision: segmentation with a strong context understanding/ implizit world model. Not perfect but neither was GPT3.
Sometimes it seems to me that we are building regulation under the assumption that we don't have the chance to shape the future ourselves anyway - so we have to regulate the tech we're buying. This is then threatening to become a self-fulfilling prophecy.
Europe, we have a major problem with NOT getting tech & AI
European citizens CANNOT be drawn into technological irrelevance by this type of stubborn behavior by 🇪🇺 regulators.
Short thread 🧵 ⬇️
@FooYunChee
@johnthornhillft
@smatthes
@NicholasVinocur
Mal ein Blick etwas weiter als auf den direkten ROI unserer Kunden. Wir haben in dieser Transformation eine Verantwortung nicht nur für die Sicherung des Wohlstands - wesentliche Dimensionen unserer Gesellschaft verändern sich in einer neuen Geschwindigkeit.
Panel 1 des
#FoGipf
: Welche Auswirkungen werden KI-Anwendungen auf Produktion und Arbeitsmarkt haben und wie schaffen wir die notwendigen Kompetenzen und Kooperationen für die KI-basierte Wirtschaft? Darüber sprechen wir mit
@JonasAndrulis
von
@Aleph__Alpha
. Stay tuned!
Our biggest model yet <luminous supreme> got great results in test - it will be available for everyone Monday. As all models is speaks five languages. Benchmarks are of questionable relevance for models that powerful but of course we ran some evals. Excited to hear your feedback.
Ich habe viele starke Ideen gesehen, wie wir wesentliche Themen mit KI souverän und innovativ aufstellen können. Im Gesundheitswesen haben wir eine besondere Verantwortung aber auch die Chance eine neue Innovations-platform zu schaffen, die vielen Menschen helfen wird.
Faszinierende Gespräche über KI mit drei der führenden Experten aus Deutschland:
@nettwerkerin
Prof. Katharina Zweig, Christopher Kränzler
@lengooTweets
, Jonas Andrulis
#AlephAlpha
. Auch im Vergleich zu ChatGPT hat Deutschland einiges zu bieten. KI mit Transparenz…
Ich hatte in Las Vegas nach unserem gemeinsamen announcement mit
@HPE
ein spannendes Gespräch mit
@mhohensee
, danke. Ich bin sicher die nächsten Jahre werden und nicht langweilig.
@strangestloop
Worrying is an anachronistic animal instinct. Its only justification is change in action. Worry without action is wasted & any suffering unnecessary. You can indeed short-cut this by not worrying and just taking the right action based on analytical planning if that works for you.
In the last days I spoke with some brilliant product people and had trouble compiling some of the advice and questions. Thinking about why these are my (preliminary) notes on our business. WDYT?
This is groundbreaking. We've collaborated with
@kerstingAIML
and
@TUDarmstadt
for years now, started a very successful PhD program and many innovations were created together. This is the turbocharger for this collaboration and a phenomenal opportunity for brilliant candidates.
We're thrilled to announce Lab 1141, a groundbreaking collaboration between Aleph Alpha Research and
@TUDarmstadt
, with the goal of pushing the boundaries of explainable, safe, and transparent GenAI.
🔗
#writtenbyalephalpha