@pmddomingos
Try magnifying that scale. You can clearly see an upward slope, and given that a 1c average temp change is significant, your scale is way too compressed.
@Bubblebathgirl
@elonmusk
Visual pollution for who? Lizards? 🌵 20k sq miles of solar panels can power all of the US. The US is 3.8 million sq miles. That’s 0.5% of total land area (that humans don’t want to live in). If we can make maintenance automatic through robotics advancements, this is very much an
@eastdakota
@kevinroose
A few issues with these assumptions.
1. The cash cash investment is multi-tranche, not all upfront
2. A large chunk is the form of compute credits
3. MS market cap fluctuates > 10B on a normal day
4. The deal gives MS license rights to all IP so there are no IP issues
5. The
@cain151714
@pmddomingos
Except im not a climate doomer. I just think that someone with a PhD would know how to represent data without inserting their particular flavor of bias.
It’s amazing what you can accomplish in one year with the right team. Grateful for everyone at
@nutrisenseio
!
Jan 2020 => Dec 2020
Team: 3 => 12
ARR: $30k => $3mil (100x)
A few things we learned along the way👇
@bindureddy
Cisco is a bad comp. State of the art data center CPUs never became commoditized (it’s still just Intel and AMD) because it’s practically impossible to enter the market for a newcomer.
1. The chip design is difficult, and line for SOTA is constantly being pushed.
2. The
@nikitabier
You definitely don’t start from zero if you check out for a few years. The foundational knowledge is very deeply ingrained. The surface knowledge (the new state of the art, etc) typically takes < 6mo to catch up on. This is probably true for any skill where physical performance /
@Grady_Booch
With all due respect, why downtalk progress? These tools allow non technical people to do what they want quickly without learning an obscure notation or spending hours testing new tools. As an experienced engineer ChatGPT speeds me up by 10-50%. For non-technical folks, the
@8teAPi
I don’t know… that letter signed by over 700 people seems like pretty good way to absolve Microsoft from any wrongdoing (poaching claims) since everyone is leaving at will.
@jeremykauffman
In that case, Celsius is a 0-3 scale and is much simpler. The extra granularity is not necessary…
30's hot
20's pleasant
10's cold
0's freezing
@_Mira___Mira_
Things get much more complicated on non abstract machines (ISA, caching hierarchy, malloc, etc) which is why most interviewers will want you to assume that it’s an abstract machine.
And unfortunately all algorithms don’t automatically become O(1) on a finite memory machine,
@martyrmade
I understand why people think that this is the right time to re-evaluate the history of the conflict and criticize Israel...
Unfortunately, searching for justification in a terrorist attack is a slippery slope. You can argue that every single terrorist attack in history was in
@paultoo
Google is being pulled into this kicking and screaming. They don’t want to give up their 90+ margins on search for 50% margins on inference… but they will have to anyways 🤷♂️
@VictorTaelin
NVDA’s data center cards are not GPUs. It’s a colloquialism. The H100 will not run Doom or Quake or AAA games. It literally has no accelerated graphics output port. Anything unrelated to data center ML workloads was removed two generations ago. Groq barely beats the H100 on
@Austen
Virus infects 100m, 2% death rate = 2mil dead. Vaccine goes out to a billion people, 0.2% death rate = 2mil dead. The vaccine must be orders of magnitude safer than what it’s trying to prevent. This is the main reason.
@dhh
Complex skills (where excellence requires being good at many individual skills) are distributed log-normally instead of normally (since the probability of being good at any of the individual skill is multiplied out). This creates a long tail on the right side of the distribution,
@0xglitchbyte
The image lists 33 books for streetfighting computer scientists to read:
1. Dijkstra, A Discipline of Programming
2. Knuth, The Art of Computer Programming, especially volumes 1 and 4B
3. Wozniak, Hackers Delight
4. Stevens, Advanced Programming in the UNIX Environment
5.
@AliceCaterpill1
@RichardHanania
Following Putin’s logic, the whole world belongs to the Persian, Greek, or Roman Empires.
Historic borders do not and should not influence modern geopolitics, otherwise the wars will never end.
All land USED TO belong to someone else, that doesn’t mean that those nations can
@LeisureWine
@RichardHanania
Wars rooted in historical geopolitics are the least justifiable, as they perpetuate endlessly. Eventually, someone must choose to overlook the transgressions of predecessors and rise above disputes that span over millennia. Otherwise there will never be peace.
@jasonjoyride
@GillVerd
@BasedBeffJezos
I thought 40min was plenty of time for
@BasedBeffJezos
to explain the working principles and architecture of the chip… but now I think one or more of the following is true…
1. He wants to keep riding the line between “stealth deep tech” and “look, we’re working in public”
@ChrisRinvesting
@DanMulroyjr
You can also use tables for addition and multiplication, does that mean we should skip arithmetic? Understanding how those tables are built and why they’re useful leads to understanding instead of mere memorization to get a grade.
@ParikPatelCFA
It’s not about how often you’re wrong, it’s about the size of the move/bet when you’re right. You can be wrong 95% of the time and still win. VC wouldn’t exist otherwise. Jim’s been right on Tesla and NVDA for years. Those two alone have made lots of people rich.
@karpathy
@karpathy
do you believe that “embodied” agent architectures like Tesla’s (but for virtual instead of physical space) are necessary for progress towards AGI?
@8teAPi
What I’m wondering is why it’s so hard for the board to admit they made a mistake, bring Sam back, and resign. Unless they pierced the corporate veil in some way, nobody is personally liable. They also don’t seem to have skin in the game as far as financial incentives go (maybe
@stephen_wolfram
@OpenAI
What’s funny to me, is that we spent so many years developing clever ways for machines to communicate with one another, and in the end we ended up with English as the RPC protocol 🤔
@jamiebishop123
99% of embedded hardware runs C, plenty of these are critical systems: weapon systems, spacecraft, etc. Its about how you use the language. Just like JavaScript: The Good Parts, there is C++ The Good Parts. JPL has a 200 page coding standards book on what not to do in C++.
@randymullen
@Bubblebathgirl
@elonmusk
Good point, curious how much extra power you need to account for the line losses, if the power generation is concentrated in a few hot spots.
@Nexuist
Without the Transformer architecture it would have been impossible to train efficiently on datasets of current size. Efficiency improvements are sometimes enough to move the field forward for a long time. This has been true in CPU architectures for 70 years. You can argue that,
@VadimYuryev
There is a reason Intel kept node changes and architecture changes on a different release cadence. The node change is hard enough, no need to also lump in large changes in architecture changes. The core count increase is more than enough this go-around.
@paulg
@SievaKozinsky
Under 4 hour trip, train is more convenient IMO. Sure, the flight is 1 hour, but getting to the airport an hour early, dealing with the crazy line for a discount airline, and driving from the destination airport all add up to a pretty miserable experience.
@getnormality
This one seems like a reading comprehension problem because I have a hard time comprehending what it’s asking 😂 If it’s just asking for a worst case distribution of red circles, then the only guaranteed red circle in a ninja path is the first one, which would make the worst case
@corry_wang
30x on inference using FP4 and the new NVSwitch at the CLUSTER level. A lot of the system perf comes from the networking improvements. Nobody who is buying these is running a single B200 in their data center or cares much about songle chip performance increase.
@jasondeanlee
Let’s say you’re learning multiplication. You can memorize the answers for every input pair. For 2x 3-digit numbers, this works out to a ~12mb lookup table. Every incremental pair costs you ~12 bytes to memorize.
This approach is low compression, low Kolmogorov complexity, and
@bradpwyble
@andpru
@EricMTrautmann
If by that you mean, over promising and eventually delivering, I’ll take that over stagnation any day. SpaceX and Tesla have reignited progress in their respective industries. There’s no reason Neuralink won’t do the same with the unlimited resources and funding Musk can provide.
@karpathy
@deepwhitman
@AIatMeta
@lmsysorg
Chinchilla is essentially the 80/20 rule for LLMs. You can get to 80% performance fairly cheaply, but eeking out the last 20% is extremely hard. It applies to humans as well… with 5 years of on the job experience you can do most jobs competently, but true mastery requires a lot
1/ Why is Tether and fiat-collateralized stablecoins a huge problem for crypto? Because they rely on an authority to verify the balance of the collateral that makes them “stable”. Any stablecoin that trades on enough exchanges can be used to manipulate the price of all assets...
@zebulgar
Genes have to interact with an environment to produce a phenotype… so genes + economic factors are equally important in producing both human and dog breeds.
@kevinroose
Price goes up, shorts get squeezed, news spreads, price goes up some more. Pretty sure Reddit just singlehandedly proved the efficient market hypothesis ;) Pricing information is no less important than fundamental information.
@h3adsh0tzz
It’s a prototype, they said multiple times that they’re working on custom radios for other spectrums. Why they would reinvent the wheel to test on pigs is a better question.
@iamgingertrash
That’s not true, GUIs and screens are much more useful for output/feedback vs input. The computers of the future are “screen optional”.
@BronskiJoseph
@_drawthentic_
It’s not just motility. Thousands of sperm make it to the egg. The process to penetrate the egg wall is basically a gamete selection process based on the sperm’s surface protein makeup. It’s mostly designed to keep broken genes out though.
@8teAPi
Also a very good chance Adam can be sued for some sort of breach of fiduciary given that he runs a competing product. But again, very hard to prove since the non-profit has no obligations to the investors. Shitty situation all around.
8/ Celebrate the little things... 🥳
2020 sucked in more ways than one, but it created an opportunity for lots of focused work. Sometimes we were so caught up that we forgot to celebrate our progress. Let’s celebrate the little wins in 2021. Happy New Year!
2/ Focus on sales from the start... 💵
Nobody wanted to give us money in an untested market. $20k in pre-orders funded out first few months, paid for first hire, got us to 30k in ARR, into TechStars, and to our first raise. Prove that people are willing to pay for your product.
@ZombieDayTrader
@pmddomingos
The planet has also gone through 5 mass extinctions, all of which had to do with extreme cooling or heating. We’re currently in an ice age that’s been pushed back by high concentrations of CO2. Is this an issue? I don’t know. Maybe the human-driven warming is preventing an
7/ Keep moving... 🏃
Product market fit is the ultimate goal. If you haven’t found it, then you should be making big changes, breaking things, and being critical about everything that can be improved. Not micro optimizations, macro optimizations!
@lookoutitsbbear
The goal is always to compress the “signal” not the file. So the first step, that everyone seems to be missing is “find the fucking signal”. I don’t think 1khz low pass is what they’re looking for, but it’s better than most of the solutions trying to compress 1024 channels of
6/ Focus on what matters... 🧘♂️
You always have a million things you can be doing with your time. If it doesn’t have to do with selling or building your product, you should not be prioritizing it. Time is your most valuable asset. Say “no” to more.
@GergelyOrosz
Unit tests are at the bottom of the testing pyramid and are less impactful than e2e/system/integration/manual testing. This is just as applicable to business applications as it is to gaming.
4/ Hiring is hard... 👨👩👧👦
In a few hours you have to decide if a candidate is skilled, autonomous, organized, proactive... It's a big ask, and 50% is about the best odds you can hope for. We give ourselves 1 month to decide if they're working out. It’s either “hell yea” or “no”.
3/ Ship ship ship... 🚢
Can’t stress this enough. Ship if it isn't done. Ship it if it’s ugly. Ship it with bugs. The longer you wait to ship, the less feedback and iterations you have to improve the product. We shipped major updates 17 times this year. I wish we shipped more!
Just ran our
@nutrisense
Rails backend on
@TruffleRuby
20.2 and it 'just worked'. Um Wow! Big props to the
@graalvm
and
@TruffleRuby
teams for making this a reality. Been holding my fingers crossed for the last 5 years. Congrats!
5/ Raising is a Catch 22... 💰
Investors try to max upside and min risk. No cash? Nobody wants to give you any. Lots of cash? Everyone wants to give you more. That's reality. Early on, find those who believes in your team and vision. Then, let your numbers do the talking.
@mahemoff
@vqp
@BorisMPower
@MatjazLeonardis
The two nucleotide base pairs interact in 3 dimensions to form the double helix structure of DNA which is not possible with only one base pair. As far as space efficiency goes, a base-4 3-nucleotide codon allows for redundancy while still covering all 20 amino acids. In base-2
@WR4NYGov
Moats: Very fast iteration cycle (biggest moat), Drivers that work (AMD cards crash constantly on ML workloads), A distributed ML software stack that works (Megatron), Owning Mellanox (infiniband networking for large clusters), Getting volume (only second to Apple) from TSM on
@leecronin
Machine intelligence is not limited by memory, speed, or power efficiency the way human intelligence is. Sure, human minds can compute anything over an infinite time horizon, but how useful is that? Machines will be able to compute anything almost instantly.
@AdamPaigge
@greenboxal
@ID_AA_Carmack
Marx was born into a privileged and wealthy family. Manifestos and philosophical treatise come from people who can dedicate time to philosophy rather than survival.
@peterrhague
I recommend Sapolsky’s last book Determined. He covers this in some depth.
“Sapolsky concludes that collectively, studies in experimental settings have shown no difference in ethical behavior between atheists and theists and that once you control for sex, age, socioeconomic
@abacaj
Unified memory is the main reason. I’d need a huge rig to fit 4x3090s to match my $3500 M2 Max MacBook Pro with 96GB of unified memory. While the throughput is not as high, it’s fast enough to be usable even with 70B 8bit models.
@KikeljGregor
@_Mira___Mira_
That’s exactly my point. If you have a real valued function, the input space is infinitely deep as it is infinitely wide (non-countable infinity). So a finite memory machine cannot represent such a function as a lookup table and must perform the actual computation.
@0xglitchbyte
C and C++ is the
#1
reason people drop out of CS classes. C# and Go might ease that pain. Ocaml, Lisp/Racket require a mindset shift towards functional programming which isn’t easy for beginners. Python, Ruby, JS all have their quirks but are generally pretty well tolerated by
@jimcraddock
@eastdakota
@kevinroose
Microsoft isn’t Google (kinda weird to say this), but Satya has shown restraint in trying to absorb everything into their corporate amoeba. GitHub is a shining example of this.
@simonkalouche
I remember hearing the same things about the economics of reusable rockets. So a quick reminder that there is a big difference between research and industry. Not the least of which is funding. A research lab with a couple million $ is not a multi-billion dollar corporation. What
@VictorTaelin
And as far as general purpose processors vs ASICS go… ML is NOT sha256 hashing. Architectures are changing all the time, and general purpose compute will be necessary for change. Every ASIC startup in the near future will end up learning “the bitter lesson”.
3/ The existence of centralized stablecoins is a serious vulnerability in the crypto ecosystem. It opens up the whole system to manipulation by the stablecoin authorities and negates one of the primary tenets of decentralized currencies: resistance to centralized control
@SorenCookie
@bindureddy
They can’t and wouldn’t want to. Intel and AMD used to have their own foundries and they left that business. There is a huge difference between chip design and manufacturing. Being competitive in both is no longer possible.
Chip design is a small part of what NVDA does.
@kemar74
@Devon_Eriksen_
Our kids are also trained on our work, are supposed to do something new with it, and end up taking our jobs… what is your point?
@pmddomingos
The cool thing about the universe is that you don’t have to keep track of them all! Simple rules lead to complexity at each level of the system, but this complexity is often reducible to closed form and can be ignored at the next level of abstraction. It would be very hard to do
@GaryMarcus
Ugh, humans have been “befouling” the internet since the invention of the internet. Every new generation of kids on the internet are “befouling” it. Now, every new generation of machines on the internet are “befouling” it. We’ve been hearing the same story since the BBS days.