Back in the US after a very fun and informative few weeks in China with
@HaoruXue
. Met some brilliant people and learned a lot, much of which will be directly relevant to our direction at
@kscalelabs
, so stay tuned 😀 For now, some photos...
Life update, I left Meta earlier this week to work full-time on a startup idea. Going great so far. Woke up at 4:30 to check Twitter for 6 hours while simulcasting all 2,103 Joe Rogan podcast at 10x on infinite loop. Here's how that applies to the tech industry (1/913)
We started doing dinner lectures. So far it has just been interns presenting some topic I assign them, but probably we will start rotating through the whole team. They're very fun (at least for me, although I haven't been the presenter yet). We're debating streaming them.
I get asked about 3D printing all the time
We do a decent amount of non structural 3D printing today, but within our high production line almost none of the robot will be 3D printed
You just can’t beat the cost of traditional manufacturing methods (stamp, casting, molding) -
Excited to officially launch K-Scale on Y Combinator. It's been a fun couple months. I'm really pleased to with the progress we've made so far and I'm super excited to start open-sourcing what we've built. Stay tuned :)
After reading 6 hours worth of Twitter posts (doled out 3 hearty "hearts", almost retweeted something Paul Graham wrote about Bay Area politics) I've finally achieved product-market fit (2/913)
AI and Robotics is moving fast.
So, I share the most important research every week.
Here's everything you need to know and how to make sense out of it:
I was chatting with a former friend from Tesla about how he needs to keep the flickering candle of great engineering alive - there are a very, very small number of organizations that can capture this. Team at K-Scale feels Tesla-level imo
Very pleased with how much useful information has been added to in just 24 hours. A lot of robotics knowledge only exists in a few peoples' brains. It is cool to have an open repository of information on how to make humanoid robots do stuff
is just so good. Gets all the small UX stuff right in a way that you can't from either 1) hacking together a consumer app onto a research project (ChatGPT) or 2) shoehorning generated answers into existing search (Google)
3D printers aren't great at manufacturing at high volume. Where they win is by being ridiculously low opex (especially if you give away your design for anyone to print themselves, instead of setting up a factory to make it)
Modern ML approaches don't need rigid bodies or high-precision actuators to do great IK anymore. Analogy is that human actuators aren't that precise and we can still do impressive manipulation. This means a 3D printed skeleton is fine, you just need to account for the imprecision
Humanoid robots probably won't generate enough cash flow in the near term to justify injection molding. Upfront capex for making molds for a single robot is $1m+ (ignoring actually *using* them). And plastic injection molding is on the cheap end
We're experimenting with something like Craigslist for humanoid robots. Feels like it will be the fastest way to get a humanoid robot into peoples' hands. Would love feature requests on the Github page
is live (stub page right now). Running CI/CD from the repo here: An intern hacked together a feature-complete local version in 2 days using Rust templating on the 'rust' branch
Of all the robotics startups there is none I want to see succeed more than
@pollenrobotics
. You can literally open an OnShape link and see their full design (specifically this link: ) The world needs a LLaMa for humanoid robots
3:30 AM, alone in the dark except for the glow of graphics cards and street lamps, haven't eaten since breakfast, coding on my own project with only unit tests as code review - feels like remembering something i forgot about
The advent of print farms has pushed the break-even volume for doing injection molding up quite a bit. 3 years ago it might have been 1000 robots, now it's 100k+ and climbing (print farms are incredible btw)
Accumulating high fixed opex (like when you start a factory) means you have to thread the needle with customer demand - too early and you go broke, too late and Apple does it better
So sad to hear the news ()😰. The conclusion of our investigation:
1. Llama3-V can be run using MiniCPM-Llama3-V 2.5's code and config.json after changing param names
2. It behaves similarly to MiniCPM-Llama3-V 2.5 in unrevealed experimental features
Made a small Chrome extension to be able to use ChatGPT as a search engine (basically, just adding the missing "chat . openai . com / chat ? q=something" endpoint):
@NathanTylerP
Very sorry to hear this. If you or any of your coworkers are interested in building open-source humanoids, please send me an email - apply
@kscale
.dev We are hiring aggressively and really need an electrical engineer so I don't have to learn EAGLE :)
After 3 years
@tesla
and Optimus, I am thrilled to announce that I joined Hugging Face to start an ambitious open robotics project! (open as in open-source, not as in Open AI) Looking for engineers to build real robots in Paris 🇫🇷 🤖🤗
Sound robotics businesses today will look a lot more like promising generative AI startups (Suno, Midjourney, Pika, Luma, Runway, Jasper, Harvey) than like iRobot / Franka / Kuka
@tim_zaman
@gdb
Copilot + ChatGPT have probably made me 4x more efficient. Was working on a 3D geometry project over New Years (), Copilot wrote all the boilerplate and ChatGPT explains niche 3D geometry algos better than Google in >half of cases. 75%+ written by AI
@USTechWorkers
@debarghya_das
imagine living in the country with the world's best tech labor market and worrying about some random indian guy taking your job... maybe try working more than 30 minutes a day instead of being racist on twitter smh
Doing more startup-y stuff has landed me in a lot more "hyper-transactional status-anxiety" type relationships (maybe also just living in NYC). It really helps to be married to someone who doesn't really follow any of that stuff. Would recommend
That being said, on a 10 year horizon (i.e., by the time I'm almost 40) I do expect this to be the most important industry in the world, and there are sound ways to build a business on the way there. Building and selling robots is not one of them
IMO humanoid robot demand is obviously too small for the next five years to support high-volume manufacturing. Capex + opex burden is too high. Same is likely true for paying for robot teleoperators to collect a bunch of data. It's the self-driving story on repeat
I wrote some minimal code for training consistency models from
@DrYangSong
. It's really wonderful to read mathematically-oriented ML papers that actually work well and do what they say they do. So satisfying.
That's it for this week's AI and Robotics breakdown.
I share the latest research every week, so follow me
@benjamin_bolte
for more.
If you found this valuable, consider a like/retweet to spread the word.
Excited to share
@1x_tech
's latest software update:
1. all capabilities merged into a single set of goal-conditioned network weights
2. contact-rich manipulation skills like table wiping and t-shirt folding
3. general picking of diverse objects
I wrote a pure-PyTorch version of the one of the RWKV models (i.e., easier to understand what the WKV thing is doing than parsing the CUDA kernel), for other people who found the math somewhat mysterious ->
Made a small bullet journal CLI which 1) syncs to a Github gist 2) uses ChatGPT API to suggest a schedule for you 3) is a single dependency-free Python file (i.e., you can copy it to some bin/ directory and use it as a regular executable). Code ->
Scale of software engineers by hardcore-ness: 0. Chris Sawyer (creator of Roller Coaster Tycoon) 1. People who write CAD software 2. Ex-math PhDs ... N. The popular image of a 10x engineer (usually someone that knows some framework really well)
We do have a prenuptial agreement but the only clauses are 1) wash hands after using the bathroom, 2) no eating stuff out of the sink, and 3) live in Singapore for at least 1 year (this is to make our kids respect authority and not be spoiled brats)
regulation should take effect above a capability threshold.
AGI safety is really important, and frontier models should be regulated.
regulatory capture is bad, and we shouldn't mess with models below the threshold. open source models and small startups are obviously important.
If you consider the Hilbert curve to be an sorting function instead of a production rule you can generate curves for arbitrary point dimensions (although they're not as pretty)
Maybe I'm wrong, but I consider myself quite a bit more knowledgable about the current state of ML than other humanoid robot CEOs. No amount of boosterism about OpenAI is going to materialize a $1 billion product category out of nowhere, at least for 5+ years
Spent a couple days messing around with monotonic attention. Feels kind of old-school at this point, but it's easier to write high-performance kernels for PGMs thanks to Triton.
Can we use wearable devices to collect robot data without actual robots?
Yes! With a pair of gloves🧤!
Introducing DexCap, a portable hand motion capture system that collects 3D data (point cloud + finger motion) for training robots with dexterous hands
Everything open-sourced
Wrote a blog post about streaming 1D convolutions and transposed convolutions - i.e., if you are streaming an audio waveform, how can you process it in chunks. Probably something from some DSP 101 course somewhere but I couldn't find it on Google...
I realized that I've started subconsciously wanting to add "Thanks!" when asking ChatGPT a question... Wonder if my kids will have this instinct or not
You can install it via `pip install usa-net`. I tried to make the code pretty good quality so you won't pull your hair out. More info on the website here: This was done with Austin Wang,
@JimmyTYYang1
,
@mhmukadam
,
@mkalakrishnan
and
@chris_j_paxton
When everything in your life kinda just works out, you expect everything in everyone else's life to kinda just work out. Probably this is the root of a lot of empathy-voids
Today is the beginning of our moonshot to solve embodied AGI in the physical world. I’m so excited to announce Project GR00T, our new initiative to create a general-purpose foundation model for humanoid robot learning.
The GR00T model will enable a robot to understand multimodal
@Ilir_AI
@_mattfreed
@trentshumay
@dexhandv2
Oh hey :) Actually Rob was one of the first people we talked to back in January when we were thinking about what to work on, I am a big fan of his and have been for a while (in fact, just emailed him again last night). The short answer is that reinventing the wheel is fun
Wrote this post a month ago: Just realized I recently passed 100 consecutive days of micro-journaling. Was looking through some of the older ones, I think my project ideas are a bit all-over-the-place...
My sister-in-law is applying to colleges. Some require 8+ essays (50 to 250 words). For 20 colleges (potentially more) you can easily write 10,000+ words. I gave a few prompts to ChatGPT and essays were 💯, likely in the top 99% of high school senior writing quality.
Something surprisingly few people know about, you can build your own brain stimulator for ~10 bucks. This was a college hobby of mine. The only friend of mine who agreed to try it is now my wife. It's fun to check up on the subreddit from time to time:
@SenSanders
He took out $35k in loans so he could get a job making $20k? Did he make a calculated risk expecting to earn more and get bitten by bad timing, or did he just not consider that $35k is a lot to borrow for a degree he could get for substantially less at a community college?
My ego: Internal conflict is healthy for developing a moral compass and pushing yourself to lead a purposeful life, and is ultimately a good thing. My id: fuck i hate this
Had a GCP sales rep try and sell me on the "Google brand" being worth the premium over
@LambdaAPI
. When did cloud providers move so far away from just trying to solve their customers' problems...
"A few years ago, when very little had been heard of digital computers, it was possible to elicit much incredulity concerning them, if one mentioned their properties without describing their construction." (Turing, 1950)