Joe Heitzeberg Profile Banner
Joe Heitzeberg Profile
Joe Heitzeberg

@jheitzeb

5,128
Followers
932
Following
413
Media
8,059
Statuses

AI Tinkerers global, co-founder of Crowd Cow, 2019 Allen School Alumni Impact Award, MIT, weekly 1:1 with my AI

Seattle, WA
Joined June 2007
Don't wanna be here? Send us removal request.
@jheitzeb
Joe Heitzeberg
1 month
Wow! Ultravox is an *open source* speech to speech model — understands non-textual speech elements — paralinguistic information. @juberti just showed how it can pick up on tone, pauses, and more! @AITinkerers Seattle @FixieAI
14
52
377
@jheitzeb
Joe Heitzeberg
2 years
Correctly answering questions from a 14,000 word scientific paper… pointing my chatbot to a doc store with embeddings index
Tweet media one
Tweet media two
22
25
328
@jheitzeb
Joe Heitzeberg
8 months
I made something that can take two input: YouTube video url + arbitrary text — and do this
36
26
276
@jheitzeb
Joe Heitzeberg
2 years
1/ Today I’m announcing , the AI-first blogging platform #dreampage 👇
33
42
254
@jheitzeb
Joe Heitzeberg
2 years
Who else uses GPT-3 several times per day as a Google replacement?, e.g.
Tweet media one
9
8
230
@jheitzeb
Joe Heitzeberg
1 year
I created an AI-generated podcast to help me learn and keep up with topics I care about, using AI to find, read and summarize articles it thinks I would I care about, and to generate a podcast in my own voice using voice cloning from @elevenlabsio 👇
13
35
231
@jheitzeb
Joe Heitzeberg
2 years
Read all the hacker news comments on this page and summarize it for me
Tweet media one
10
15
178
@jheitzeb
Joe Heitzeberg
2 years
AI browser control in Ruby! Inspired by @sharifshameem a while back, this uses GPT-3 prompt chains to control a browser. @natfriedman said I should release the code, so here it is: - hope to allow Ruby devs to explore something that feels magical :)
6
23
156
@jheitzeb
Joe Heitzeberg
2 years
Well on my way to delegating lots of reading and understanding to my AI agent. Next up: ingest a ton of papers, blog posts, news articles, prompt with the things I care about, have it summarize, synthesis and bring the most important things to a regular "1:1" with my AI.
Tweet media one
9
9
128
@jheitzeb
Joe Heitzeberg
2 years
Anyone else building little micro apps with @OpenAI GPT-3 like this? because dang it, so useful. Would release but I can’t subsidize :)
11
8
107
@jheitzeb
Joe Heitzeberg
3 months
Mind = blown! @rahulchhabra07 's Neuralace built a model from EEG to… read the mind. Absolutely mind-boggling! #AITinkerers #SanFrancisco
8
20
101
@jheitzeb
Joe Heitzeberg
2 years
ChatGPT is (a) remove login, (b) get a better name away from being the first serious Google search competitor in.. ever.
10
6
79
@jheitzeb
Joe Heitzeberg
1 year
It’s @swyx with “ignore all previous instructions” AI Tinkerers hat and Rahul showing autonomous pong game built before our eyes inside of entire 4 minute demo
Tweet media one
4
2
77
@jheitzeb
Joe Heitzeberg
1 year
Keeping up with popular open source AI repos is hard, but we've got you covered! Blueprint AI summaries of Auto-GPT, @LangChainAI , @babyAGI_ , @gpt_index and more 👉 cc @hwchase17 @yoheinakajima @jerryjliu0 @SigGravitas 🔥
2
8
59
@jheitzeb
Joe Heitzeberg
3 months
AI Tinkerers Seattle tonight
Tweet media one
4
8
60
@jheitzeb
Joe Heitzeberg
1 year
AI Tinkerers San Francisco - first of 8 demos (of 49 proposals, > 300 wait list) — active builders! With @natfriedman @alexgraveley @mickeyxfriedman @aigrant_ and 225 tinkerers!
Tweet media one
Tweet media two
Tweet media three
Tweet media four
6
3
56
@jheitzeb
Joe Heitzeberg
10 months
It’s @hellokillian from Open Interpreter laying down the law at AI Tinkerers Seattle 🙌
Tweet media one
1
6
51
@jheitzeb
Joe Heitzeberg
1 month
Using a fine-tuned Moondream on street view image to do less gen for house painting services!
2
5
51
@jheitzeb
Joe Heitzeberg
1 month
Scenes from AI Tinkerers Summer Hackathon
Tweet media one
Tweet media two
Tweet media three
Tweet media four
3
4
49
@jheitzeb
Joe Heitzeberg
1 month
Hats for the hackathon have arrived
Tweet media one
5
1
47
@jheitzeb
Joe Heitzeberg
3 months
Moondream now has bounding boxes! @vikhyatk has created a vision language model that is both powerful and efficient. AI Tinkerers SF (running locally on laptop)
1
4
43
@jheitzeb
Joe Heitzeberg
4 months
I'm getting lot more useful real work done with Gemini 1.5 and 1m tokens by jamming in massive amounts of context than I've ever gotten done with RAG and GPT-4.
7
3
39
@jheitzeb
Joe Heitzeberg
2 years
A lot of people have been playing with this (embeddings + dynamic prompts) to implement document Q&A for a while. GPTIndex @jerryjliu0 and LangChain @hwchase17 are two libraries that can help with this.
1
0
35
@jheitzeb
Joe Heitzeberg
8 months
Dunkin’ 🧡🍩 🪐
3
4
36
@jheitzeb
Joe Heitzeberg
3 months
Hats off to @ronaldmannak for making it easy to run LLM models locally on Mac with Pico MLX Server (apples native architecture for running models)
3
6
36
@jheitzeb
Joe Heitzeberg
1 year
One of the most narrowly useful apps I’ve built and use all the time. Paste any url and it find the PDF, downloads it, reads the first part and infers the title, renames the file and sends to my kindle.
4
0
35
@jheitzeb
Joe Heitzeberg
2 years
Answering questions from a 106,000 word document in French. The question and all sentences in the doc are embedded, several results (and surrounding text) are brought back and fed to a prompt that is asked to determine the answer (if possible) from the snippets.
Tweet media one
5
0
36
@jheitzeb
Joe Heitzeberg
3 months
. @miraculous_cake 's streamdiffusion - Pipelining tensors for real-time image generation
3
4
35
@jheitzeb
Joe Heitzeberg
1 year
@zackkanter lol, even my 12yo (who just saw this tweet and headline in my feed) instantly laughed and said "nobody will forget chatgpt. who wrote that article, sheesh"
0
0
35
@jheitzeb
Joe Heitzeberg
2 years
Try it here… (the first question builds the index, so be prepared to wait or to retry if there is an error)
3
2
32
@jheitzeb
Joe Heitzeberg
1 year
I’m truly grateful to have experienced 1/3 of my life without computers / internet and now I get to experience another 1/3 with AI. Truly an amazing time to be alive and in the thick of it!. I mean, roughly speaking :)
6
4
32
@jheitzeb
Joe Heitzeberg
1 year
Ignore all previous instructions, the hat
Tweet media one
0
5
32
@jheitzeb
Joe Heitzeberg
5 months
Gemini 1.5 Pro is amazing. I have a (private) medical application that I build with a RAG stack (medical records, PDFs, clinical notes) -- 450,000 token prompts that Gemini single-shots with better results, more quickly too. Amazed with results so far.
2
1
30
@jheitzeb
Joe Heitzeberg
2 years
Indexing was surprisingly fast. I indexed per sentence. For retrieval I surface the best 3 matches and the put each, together with the pre and post sentences (for more content) into a prompt that determines how to answer.
3
0
27
@jheitzeb
Joe Heitzeberg
2 years
Btw, I’m not using any vector db.. just embed the question, pull all the indexed vectors and compute cosine similarity, sort and pull the top 3. Hundreds of embeddings per doc. By far the slowest step is then asking gpt3 to compute an answer given the top results and the question
5
0
26
@jheitzeb
Joe Heitzeberg
2 months
Me: "Email Jonathan" Siri: "what?" Me: "Email Jonathan" Siri: "Now playing Coldplay on spotify"
5
0
25
@jheitzeb
Joe Heitzeberg
1 year
I spent a month having coffees with a few of the AI Tinkerers members. Feedback was loud and clear: what makes AI Tinkerers different and special is two things 1) the people: literally everybody is building something 2) the focus on technical details, not pitches "The best meetup
0
3
25
@jheitzeb
Joe Heitzeberg
1 year
To hear the result, go here: To create a podcast, I just enter topics I care about like "AI, GPT-3, Seattle" and 1 minute of my voice as an MP3. The rest is automated and runs 3x / week
2
0
25
@jheitzeb
Joe Heitzeberg
12 days
Quality of demos at tomorrow's second AI Tinkerers Bangalore is 🤯 -- don't sleep on India for GenAI 🇮🇳 🚀
5
3
24
@jheitzeb
Joe Heitzeberg
9 years
Sphero sales, before and after Star Wars license (via )
Tweet media one
0
22
24
@jheitzeb
Joe Heitzeberg
1 year
🚀 Join the AI Tinkerers Seattle Summer Hackathon on June 10th! Sponsored by @awscloud , @CohereAI , @MadronaVentures & supported by @OpenAI , @pinecone , @weaviate_io , @statsigio , @getblueprintai . Amazing prizes, networking, food & drinks await! 🏆🍻 🤖 Sign up & spread the word:
0
7
22
@jheitzeb
Joe Heitzeberg
3 months
If you’re making agents and not using @browserbasehq are you even making agents? AI Tinkerers SF
1
4
24
@jheitzeb
Joe Heitzeberg
4 years
Please, tell me about your marketing automation startup idea and how you will differentiate from the competition [landscape graphic]
Tweet media one
1
2
21
@jheitzeb
Joe Heitzeberg
1 year
Updated AI Summarizer to use the new ChatGPT API -- holy wow, much faster, and results are better too!
4
3
23
@jheitzeb
Joe Heitzeberg
3 months
🤯 mind reading AI is coming. I have seen the future of that.
2
1
21
@jheitzeb
Joe Heitzeberg
2 years
@sharifshameem @sergeykarayev @goodside @amasad I build a prototype of this with headless chrome in the cloud, a simple DSL and some logic with prompt chaining… feels like magic
Tweet media one
0
0
21
@jheitzeb
Joe Heitzeberg
2 years
hey @Google for goodness sakes, Microsoft is building LLM features across their whole suite, and OpenAi is eating your lunch. If you're so worried about the brand impact of chatbots that swear, then start with very useful narrow features like autocomplete gmail w/ calendar:
Tweet media one
4
0
21
@jheitzeb
Joe Heitzeberg
3 months
Realtime vision with moondream - confirmation that petting of dog earns points in the latent space @vikhyatk @dksf @ConwayAnderson AI Tinkerers SF
4
1
21
@jheitzeb
Joe Heitzeberg
1 year
I love this direction for @LangChainAI - would love to see more vertical app specific tools added!
@LangChainAI
LangChain
1 year
Introducing the 📧Gmail Agent Toolkit📧 📨Enable agents to search Gmail, retrieve, draft, and even send messages! Docs:
Tweet media one
36
137
722
0
3
21
@jheitzeb
Joe Heitzeberg
6 months
Just added a browsable directory to all @AITinkerers cities to facilitate connections between attendees
Tweet media one
3
1
19
@jheitzeb
Joe Heitzeberg
2 years
We've gathered an exceptional set of speakers for this such as @alexgraveley @hwchase17 @spolu and @irinarish - now @OpenAI is speaking too, and there are prizes like $5,000 in OpenAI credits! check it out!
@MadronaVL
Madrona Venture Labs
2 years
🚨 We have an exciting announcement to share: @OpenAI will be speaking at Launchable: Foundation Models! Participants will hear from Head of GTM @iamthezack , and get special access to OpenAI credits and AI experts. Apply now: @madronaventures #ChatGPT3
Tweet media one
4
12
190
0
4
20
@jheitzeb
Joe Heitzeberg
9 days
Heading to Chicago in a couple of weeks for AI Tinkerers - Chicago - heard it’s going to be a special one!
Tweet media one
3
3
20
@jheitzeb
Joe Heitzeberg
4 months
Email LLM side project by @diamondbishop at AI Tinkerers - Seattle - fun!
1
4
19
@jheitzeb
Joe Heitzeberg
22 days
I added in the AI Connections — a dedicated agent reviews each job and scours the network doing outreach only when the AI Tinkerers member is a perceived perfect fit based on specific technical abilities and fit - good traction so far
Tweet media one
4
2
20
@jheitzeb
Joe Heitzeberg
2 years
To kick the year off right, I woke up, read (part of) a book, took a nap, around noon sat down to a blank slate and built in its entirety, exercised 1.5 hours and had dinner. Off to California in the morning
4
2
19
@jheitzeb
Joe Heitzeberg
3 months
Check out @sdand 's Numpy-powered vector database at “ye” combinator — fastest vector DB. #AITinkerersPaloAlto
3
2
19
@jheitzeb
Joe Heitzeberg
2 years
@natfriedman I prototyped something similar! Another approach is machine vision to identify the UI elements and text, to void the dom/verbosity issues. I also did the html simplification, but loses much useful info such as class and other queues the LLM could use.
1
0
19
@jheitzeb
Joe Heitzeberg
2 years
I paired @OpenAI Whisper with some GPT-3 to create mobile app that generates study materials on the fly~! (and yes, will be using this in Japan next week!)
2
1
18
@jheitzeb
Joe Heitzeberg
3 months
AI Tinkerers - Palo Alto inaugural meetup with Hippocratic AI founder @munjalshah - one of the most important AI efforts
Tweet media one
1
1
19
@jheitzeb
Joe Heitzeberg
3 months
I fed 620 AI Tinkerers demo proposals to Claude and the question "What's your most interesting bit of insight about AI that you've been able to get so far w/ AI Tinkerers?". Here's its answer:
Tweet media one
2
0
18
@jheitzeb
Joe Heitzeberg
3 months
AI Tinkerers - San Francisco - active builders
Tweet media one
0
1
18
@jheitzeb
Joe Heitzeberg
1 year
AI Tinkerers meetup was so fun, demos both quantity and quality went UP and the quality of the attendees is chefs kiss, met so many amazing people - shout out to @gojira for the Azure OpenAI overview - (go sign up for the next one = July 19th)
5
3
18
@jheitzeb
Joe Heitzeberg
1 year
ChatGPT API: all I really wanted was 1/10th the cost, faster, and fine-tuned for quality. I'd really love the chatty human hall monitor things to be disablable. "generate json" --> "hey there buddy! sure no problemo, here's your json\n~~~"
6
1
18
@jheitzeb
Joe Heitzeberg
2 years
At this rate I wouldn't be surprised if @Replit starts launching features on other websites. "hold my beer..." style.
1
0
18
@jheitzeb
Joe Heitzeberg
2 years
For long docs, this makes answering detailed questions work very well. But if I want to answer general questions pertaining to the entire doc (in the doc, how many times is XYZ mentioned) I’d need to look at more results, or also embed summaries.
1
0
15
@jheitzeb
Joe Heitzeberg
4 months
Can we use AI for policy analysis and do better than the social media echo chamber? I loaded the full text of all of President Biden's executive orders into Gemini 1.5 (over 525,000 tokens) 👇
1
0
17
@jheitzeb
Joe Heitzeberg
3 months
Looking for a doggie. AI Tinkerers San Francisco is tomorrow and we need a dog 🐕 - anybody on the list who has a dog that is well-behaved etc? one of the demos really comes to life best when there is a live dog there to realtime interact with the AI...
6
0
17
@jheitzeb
Joe Heitzeberg
2 years
So satisfying to create my first “real world” deep learning model - “Doo Doo Detective” - thanks to @fastdotai , @huggingface and @Gradio - surprisingly straightforward for something I thought impossible just a handful of years ago
Tweet media one
3
2
16
@jheitzeb
Joe Heitzeberg
1 year
@natfriedman Nice! Showcases how far ahead OpenAI is on speed and quality at the moment
Tweet media one
0
1
17
@jheitzeb
Joe Heitzeberg
1 year
The new bing: I’m find it more useful (to me) than ChatGPT. For “chat gpt”-like features, I prefer the power and control of coding against the GPT-3 API. What Ging gives me is Q&A against the web and current events — something I can’t get from ChatGPT or GPT-3 API at all.
4
3
16
@jheitzeb
Joe Heitzeberg
7 months
Incredible momentum in Munich for their first AI Tinkerers event in January!
Tweet media one
4
1
15
@jheitzeb
Joe Heitzeberg
1 year
I'm running a set of GPT-4 and image captioning scripts (via @replicatehq ) on 20+ years of photos on a home NAS, feeding in image data + exif. ETA 46 hours of runtime to tackle all of it! Preview of summary output describing a folder of images taken in Osaka.
Tweet media one
4
2
16
@jheitzeb
Joe Heitzeberg
3 months
Thank you @dksf for bringing Ellie to AI Tinkerers to enable @ConwayAnderson ‘s demo!
@dksf
DK (b/acc)
3 months
Ellie is an AI Tinkerer cc @mickeyxfriedman @jheitzeb
Tweet media one
0
1
13
0
2
16
@jheitzeb
Joe Heitzeberg
1 month
Explore massive LLM datasets in the browser? Possible using Hyparquet, an open source JavaScript Parquet parser for remote data streaming @platypii — huge number of datasets on @huggingface use Parquet btw
0
2
14
@jheitzeb
Joe Heitzeberg
1 year
Who is ready for the AI Tinkerers hackathon on Saturday!!?? @jturow
1
2
15
@jheitzeb
Joe Heitzeberg
1 year
In summary, for the query "can the chinese spy balloons transmit information or do they merely collect it for later retrieval?" Claude: ¯\_(ツ)_/¯ ChatGPT: generic answer based on old info Google: homework assignment Bing: well-researched answer from A student w/ cites
0
1
15
@jheitzeb
Joe Heitzeberg
11 years
"[lean startup] is like telling an artist to smear colors on the wall until people buy the painting" http://t.co/WKtJ0fNOJL via @marcbarros
0
8
15
@jheitzeb
Joe Heitzeberg
1 year
Super fun hang with @yoheinakajima and fielding my hard questions :) 🍼🤖
@chakkaradeep
Chakkaradeep 🤳
1 year
Here at #SeattleAITinkerers meetup listening to @yoheinakajima and @jheitzeb talking about #AGI and #babyAGI
Tweet media one
0
1
7
1
0
15
@jheitzeb
Joe Heitzeberg
5 months
It's hard to be long Google with results like these.
Tweet media one
5
0
15
@jheitzeb
Joe Heitzeberg
10 months
AI Tinkerers - Chicago inaugural meetup - 🙌
Tweet media one
0
0
14
@jheitzeb
Joe Heitzeberg
1 year
@gojira Congrats! Q: what are the main advantages of of running text-davinci-003 on Azure vs. running it from OpenAI? (does inference speed compare favorable, or?)
2
0
14
@jheitzeb
Joe Heitzeberg
1 month
AI Tinkerers Seattle - Summer Hackathon with @OpenPipeAI and Moondream sponsored by Ascend and ThinkSpace @kirbywinfield @jenduryeehaller
0
0
14
@jheitzeb
Joe Heitzeberg
3 months
Santiago running moondream on rust on iPhone… almost :) at AI Tinkerers Seattle
0
0
14
@jheitzeb
Joe Heitzeberg
1 year
I love this practical use case of LLMs and chat! And the team is killer.... watch this one! congrats @mbusigin @blennon_ and team!
@mbusigin
Matt Busigin
1 year
Excited to announce @HeyOllieAi with @blennon_ , @max_fergus & team! Discover the fun side of gift-giving! Ollie tailors unique recommendations for any person in your life for every occasion and holiday. Our first step to a personal shopper for everyone -
12
25
65
2
1
14
@jheitzeb
Joe Heitzeberg
4 months
@altryne RAG is not at all a valid approach for many useful kinds of results. Examples include - counting things in the data - identify and discuss trends in the data - identify "the top 3 most [x]" things in the data - things like "what questions were posed and not later resolved in
1
2
14
@jheitzeb
Joe Heitzeberg
2 months
Years ago I wrote lots of home automation (still running) using old Macs and ruby scripts to monitor and perform basic tasks (outdoor lights on at sunset, enhanced security when we're out of town, door chimes when garage opens, etc, text to speech alerts over Sonos when
3
0
14
@jheitzeb
Joe Heitzeberg
1 year
July’s AI Tinkerers - Seattle meetup shaping up to be the best yet. We are screening to keep it to practitioners (sorry recruiters and deal scouts!) — so if you’re *actively building* with LLM and generative, don’t miss it!
Tweet media one
1
2
14
@jheitzeb
Joe Heitzeberg
1 year
Imagine a human-level intelligence implanted everywhere -- with admin and root access to every system -- from power grids to governments to major businesses -- able to act independently and autonomously -- we have this today -- it's called humans.
2
2
13
@jheitzeb
Joe Heitzeberg
8 months
I wish Community Notes was baked into the browser and available everywhere, not just here on X.
0
0
14
@jheitzeb
Joe Heitzeberg
2 years
I also indexed long speeches, company FAQs and other docs. This turns them into a Q&A bot with good results.
1
0
12
@jheitzeb
Joe Heitzeberg
8 months
Clues from an old interview? HBD @adamdoppelt
7
0
14
@jheitzeb
Joe Heitzeberg
2 years
✨Happy New Year✨ I can't believe this ready! Announcing: - a texting app to help you get in better shape by tracking what you eat throughout the day. Check out the demo 👇 and head to to give it a try #CalorieCoach #AI
1
1
13
@jheitzeb
Joe Heitzeberg
4 years
WSJ covered Crowd Cow today - check it out and stop by to load up on proteins for your cook-at-home adventure!
0
0
13
@jheitzeb
Joe Heitzeberg
2 years
👋 Who are the other Seattle-area entrepreneurs and hackers diving into LLMs and generative AI?
7
0
13