Super excited to share that my daughters (high school seniors) published this cookbook for teens (and adults!). Kudos to their mom on motivating them that cooking is a life skill. Please do support them by buying a copy (great gift!) & leaving a review
Proud to say that this was architecture proposal was my biggest contribution at Google. Much cheaper at scale and scales much better. Needs excellent scaling software to take advantage of this system architecture. But ideal for very large LLMs.
Google has moved away from big chips to swarms of smaller, cheaper chips with huge interconnect & datacenter connect (Jupiter) to scale to millions in a run
Trillium has 32GB VRAM per chip but pods are 256 so 8.2 TB per pod (great for long context), likely each chip 1/3 H100
You know you live in the Geek valley aka Silicon Valley, when the local
@SCCgov
Santa Clara county government puts up signs like this.
#COVID19
#WearAMask
The optical switches connecting our TPU Pods give Google a massive scaling advantage for TPU Pods, that enables us to train large language models (LLMs) like Bard / LamDa at a much lower cost than GPUs.
Google designed and developed their own optical switches for their network!
This dramatically reduces their networking cost of ownership by 40% and capex by 30% and it improves performance 32%
The switch is based on MEMS mirror arrays
Only 3db loss is nutty!
$GOOGL
#OFC
#OFC2023
The
#AI
chips are falling. First Mythic laid off 50% of their staff in June, now Graphcore is in trouble. The cliff to build a competitive AI chip is very high, whether its training of inference
#MachineLearning
@melissasavenko
@Donn1nger
I agree. Agents provide a valuable service. But that service is worth $10-25K per agent. Not a percentage of the home price, which in my area is >$3M. Also I think a new system will develop soon that will work just fine, but realtors will no longer make a big payout
.
@Google
’s TPU v4 ML supercomputers set performance records on five benchmarks, with an average speedup of 1.42x over the next fastest non-Google submission, and 1.5x vs our MLPerf 1.0 submission.
#ml
#google
#deeplearning
CS is completely dominating mindshare among engineering programs in high schooler's minds - I see the anecdotal information as my own kids and their peers consider college. Its almost like the death of Moore's law is followed by the death of EE too!
Interesting project: "ZLUDA is a drop-in replacement for CUDA on Intel GPU. ZLUDA allows to run unmodified CUDA applications using Intel GPUs with near-native performance."
Now, Intel just has to produce a decent GPU .....
The interesting thing to see is that VCs spent over $2 to $3 Billion on new
#AI
#DeepLearning
chips (besides big investments from Intel, Qualcomm, others), and after 4+ years, none of them have a production chip in the market that is competitive. Really cuts through AI chip hype
One key premise for the long term future of
@Tesla
is based on leading in AI, given that electric cars are now a commodity. And now Tesla is way behind on AI and so Musk wants everyone else to slow down 🤣🤣
Despite advocating for an industry-wide halt to
#AI
training, Elon Musk has reportedly kicked off a major
#AI
project
@Twitter
They purchased ~10,000 GPUs and recruited AI talent from DeepMind for the project that involves a
#LLM
via
@tomshardware
Google Cloud announces our next generation TPU: Cloud TPU v5e
TPU v5e delivers up to 2x higher training performance per dollar and up to 2.5x inference performance per dollar for LLMs and gen AI models compared to Cloud TPU v4.
Some nuggests from
@Google
's Q2 2023 Earnings call:
- # of GenAI customers on Google Cloud grew 15x from April to June
- >70% of gen AI unicorns are Google Cloud customers
- GCP: Q2 revenue of $8 billion, up 28%, and operating profit of $395 M
Yesterday, we announced the IBM-H2O partnership to bring automated machine learning to the enterprise, and today, we announced the largest & fastest AI supercomputer "Summit" at Oak Ridge National Labs
@ORNL
@IBMPowerSystems
@H2oai
#MachineLearning
#AI
Just realized that next time POTUS has a technology council meeting with the CEOs of IBM, Microsoft, Google, they might want to order Indian food ... ;)
Current CEOs of notable tech companies:
Alphabet - Sundar Pichai
Microsoft - Satya Nadella
Adobe - Shantanu Narayen
MasterCard - Ajaypal Singh Banga
Vimeo - Anjali Sud
And now IBM - Arvind Krishna
Common link?
All their parents are still disappointed that they’re not doctors
OpenXLA is an open-source compiler framework that takes underlying compiler technology of TPUs (XLA) & makes it available to every machine learning HW accelerator.
@Google
partnered with many AI chip companies to enable leverage of compiler tech
Corporations using ChatGPT are giving their internal secrets to
@OpenAI
, as Samsung engineers did when they used ChatGPT to write code.
You will need to use Generative AI that comes with guarantees that the provider does not read your data.
John Kelly: the IBM Summit supercomputer is the most powerful in the world and is breaking
#AI
records. We're donating 5 racks to
#MITComputing
to power research across the institution.
#MITIBM
There is no business model for AI generated search results - for Perplexity and for Google.
And the internet starts dying as publishers / content developers stop developing content because they get no traffic.
@pankaj
First let me post some TPU links:
Software scaling with ML Pathways
@JeffDean
et al:
TPU's optical network paper:
Associated slides on optical net:
Resilience at scale:
#DeepLearning
training times are going up.
@IBMPowerSystems
with NVLink between
@NVIDIADC
GPUs and Power9 CPUs, enables large models to train ~4x faster than Intel-based
#GPU
systems, and with Distributed deep learning, you can use 100s of GPUs
Its ironic that China's zero Covid policy will push manufacturing out to countries like India and Vietnam. Add to this the increasing US-China isolation / conflict, we will see an acceleration of the manufacturing move.
There are some violet protests going on in Foxconn Zhengzhou, China as workers attempt to flee the factories they are locked in.
This is the largest Apple product factory complex in the world with >200k people working there at some parts of the year
$AAPL
We witnessed history last evening as Arvind was announced as new CEO of
@IBM
and Jim from
@redhat
as President. Thanks Ginni for your leadership; you will always remain a role model for all of us.
There was a decision made yesterday that I believe will be a historical decision. Unfortunately I can't tell anyone for a few years. Nevertheless recording the date on Twitter
The hype of Generative AI is finally starting to dampen as everyone realizes that after 1.5 years, there is no commercially viable enterprise use case. The GenAI chatbots have had little impact due to inaccuracies and costs of serving.
TL;DR -
@AWS
Cloud GPU servers are 3% slower and 1.5x more expensive than buying on-prem over 3 years TCO (gets closer to 2x more expensive in 5 years)
Tanmay Bakshi
@TajyMany
says IBM PowerAI that will empower domain experts to create more & better data much more quickly for the purpose of optimizing machine learning algorithms, reducing time it takes to train these systems from weeks or months to days
I am hiring Product Managers for machine learning infrastructure at
@Google
. Looking for folks with super strong product management expertise and experience. ML / analytics / similar experience very, very nice to have. Send me a message if you are interested.
#AI
#MachineLearning
Terrific report from
@Usefyi
- 90% of remote workers plan on remote work for rest of their careers
- 96% recommend remote work to friends
- 91% of remote workers say remote work is good fit for them
#COVID19
accelerating trend
#futureofwork
#worktrends
This is such a critical problem in so many large tech companies. its so attractive to bring in more "senior" experienced people, who don't understand the company culture and at the same time, you piss off internal star performers
When we hire from the outside, we pay more to get less.
Despite costing more, external hires perform worse and quit more. Promoting from within fuels individual growth and collective success.
If there are never strong internal candidates, we've failed at leadership development.
Big news:
@AMD
in advanced talks to buy
@Xilinx
. Massive consolidation happening in
#semiconductor
space, as companies use their stock valuations to buy others. Signs of a rapidly maturing (and slowing down) market
Looks like first
#AI
Startup with silicon for
#deeplearning
inference with remarkable performance results. 15,000 ResNet-50 images/second with 1.3ms latency, batch size 10 at 100W.
#MachineLearning
We are finally revealing some of the details behind our currently deployed TPU infrastructure, the TPU v4.
Detailed blog with link to ISCA paper here:
@CNET
write up here
This is indeed big news. Yet another specialized accelerator for HPC (NEC SX vector processors) bite the dust.
The commercial engine of ML and graphics accelerators (TPUs, GPUs) are so big, that they can innovate faster and with the latest memory / interconnect IPs.
Buried in GTC/GTP excitement, NEC announces ceasing R&D of their next gen accelerator aiming to be 10x improvement in power efficiency, citing that commercial accelerators will match their objectives. This could be the final nail in the coffin for their SX vector processors
Results from researcher Sinead Griffin (below) at Lawrence Berkeley national labs (LBNL) confirms LK-99 as a superconductor at room temperature and then mic drops and takes a long deserved nap.
Top500 list of supercomputers is out.
Top10 systems are more flops than bottom 490
Top 5 are 85% of the flops of the Top 10
This means, we have 5 really massive supercomputers, vs 100s of small ones. Large hyperscalars like Google don't enter Top500
#HPC
#supercomputing
#SC22
BOOM! The worst kept semi secret is out.
@Google
announces Axion, an
@Arm
-based general purpose processor. Already deployed internally on services like
@YouTube
, in preview for enterprise customers soon. More details later from
@MattKimball_MIS
.
Great talk from PGS about they built a 53K node GKE system in
@GoogleCloud
with 800K+ vCPUs making it the 24th largest super computer in the world. They moved from a 260,000-core Cray supercomputer.
#HPC
infrastructure
@rseroter
This is a great framework from Google DeepMind that defines the different levels of AGI. Many of us assume
#AGI
means what is defined Level 4 or Level 5 (exceeding most humans in most tasks). In reality, basic AGI can do lots of skilled / unskilled human tasks.
#GenAI
#LLMs
The OEMs and hyperscalars have gone from supporting just Intel CPUs to AMD and now ARM. There are two dynamics at play:
- AMD's strong roadmap
- Poor gen-to-gen perf improvements from CPUs (death of Moore's Law), prompting looking at alternate architectures (ARM, RISC-V)
Amazing white hat
#hacker
work by
@IBMSecurity
team on a new type of enterprise attack called "Warshipping". Beware of the $100 hacking device in our corporate gifts like Teddy Bears that can sniff your Wifi password
I have seen this video several times over the years & it is still a lesson in instruction. As technologists, we first gravitate to talking about speeds & feeds, rather than client value, our core value prop, our core product vision / what problem we are trying to solve.
Its chip week: - Google announced their custom ARM SoC, Axion & availability of TPU v5p:
- Intel announced Gaudi3 AI accelerator for training:
- Meta announced v2 of their training / inference AI chip, MTIA:
Benchmarks for AMD gpus showing up. Looks like great scaling and their investment in a much larger HBM memory leads to a dramatic reduction in the number of nodes required.
#HPC
what's really cool though is just how much more we can do with
@OLCFGOV
Frontier.
The same code runs on the AMD GPUs there (now using HIP instead of CUDA) and just works. The larger GPU memory means we can run on fewer nodes too! Here's strong scaling for this XRB run.
Building self-driving cars is super hard. I advised a startup in the
#ADAS
space; realized its not about
#AI
/
#DeepLearning
. about building a complex real-time control system doing complex sensor fusion with diverse sensors. Argo AI is first of many IMHO
Silicon Valley has healed. At Local Starbucks at 4:30 pm on Friday. Every table is two people clearly having a business / startup ideation meeting. Many with laptops open
hey
@levie
- weird way to send you a feature request, but ... if someone sends a Box folder link, and I don't have access, would be great to have a button "Request Access". Right now, we have to email the folder owner to get access.
Boggles my mind all the areas that
@IBMResearch
works on. In this work, IBM researchers working with researchers from Singapore develop new macromolecule that could counter antibiotic resistance
@TechCrunch