How do you draw an owl:
1. draw some circles
2. click "Diffuse the rest"
We have made the greatest thing since MS Paint (using
#stablediffusion
):
Looking forward to seeing what you'll generate with it 😊
text-to-3D
text -> 2D image -> 3D point cloud
Step 1: describe what you want ->
Step 2: img generation with a diffusion model ->
Step 3: 3D point cloud generation using a second diffusion model using the img from step 2
gr.demo:
ML runs everywhere now because of 2 recent trends:
* 𝚓𝚊𝚟𝚊𝚜𝚌𝚛𝚒𝚙𝚝-𝚏𝚒𝚌𝚊𝚝𝚒𝚘𝚗 of ML (transformers.js, huggingface.js) for running on browsers
* 𝚌𝚙𝚙-𝚏𝚒𝚌𝚊𝚝𝚒𝚘𝚗 of ML (llama.cpp, bloomz.cpp, whisper.cpp) for running on embedded devices
text-to-image-editing
words -> mask -> replacement
describe what you want -> automatic mask generation with CLIPSeg -> replace mask with Stable Diffusion generation
gr.demo:
Pose 3D mode & render with Stable Diffusion (ControlNet)
(forget about ray-tracing, path-tracing, or any kind of tracing in that respect
#FutureIsNow
)
try gr.demo:
There are wrappers that are built on top of open-source LLMs to generate syntactically valid json, yaml, or context-free grammar langs, its kinda big deal.
is Gradio demo of Jsonformer project that lets you generate syntactically valid json
🧵
Did you know that the most used language in ML pull-requests is NOT python 🐍 but... emojis? We know that as well, so we've added support for reactions in
@huggingface
PRs /Discussions! You can now use 👍 🤝 😔 🤯 🤗 ❤️
Any other favorite emoji we should add?
At
@huggingface
, we are adding more support to GGUF (model format by
@ggerganov
). The number of GGUF models on the hub has been exploding & doesn't look like it is gonna slow down🔥
see more at:
14 million text-to-image prompt dataset with their hyperparameters (DiffusionDB dataset from
@polodataclub
)
Quite useful for both research & product at the same time
I created a browser based environment where humanoids are trained to walk through NeuroEvolution of Augmenting Topologies (NEAT)
@kenneth0stanley
| Tools used:
@TensorFlow
, Neataptic.js, Planck.js (a Box2D rewrite)
1/ Creating smart NPC (non-player character) is as easy as just calling
@huggingface
Inference API
#gamedev
#NLP
I've created this WebGL demo using 🤗 +
@playcanvas
Code and demo are available at 🤗 Spaces
Transformers work on different modalities (from text to vision to audio) because everything is a "sequence of tokens"
🧵 Thread on how any data type can be viewed as a "sequence of tokens"
1/
ᴇᴅɪᴛ ɪᴍaɢeꜱ ᴡɪᴛʜ ᴡᴏʀᴅꜱ
🤗Diffusers now supports InstructPix2Pix
#stablediffusion
model by Tim Brooks, Aleksander Holynski, et al.
Read more on how to use it with diffusers:
#MichelangeloTheGreatest
“When AI critics get together, they talk about sentience and consciousness and emergence. When AI scientists/engineers get together, they talk about where you can get cheap GPU” - Pablo Picasso 2023
“When art critics get together, they talk about form and structure and meaning. When artists get together, they talk about where you can buy cheap turpentine oil.”
—Pablo Picasso
As transformers generalized from NLP to other modalities, so did 🤗Hub
On 🤗Hub, you can find checkpoints in CV, audio, RL, & even table-question-answering.
Moreover, you can even play 🕹️with them:
The ongoing consolidation in AI is incredible. Thread: ➡️ When I started ~decade ago vision, speech, natural language, reinforcement learning, etc. were completely separate; You couldn't read papers across areas - the approaches were completely different, often not even ML based.
One of the perks of working from
@huggingface
Paris office 🇫🇷 is that you can watch Messi, Neymar, Mbappe, De Bryune, Mahrez, Grealish play in the same match ⚽️⚽️⚽️
context aware image transformation using
@diffuserslib
v0.10.0 (utilizing Depth2Image model)
example: transforming the initial image into "a photo of a bag of gummy bears on a basketball court"
gr.demo:
How do language models (like BERT or GPT) "see" words?
TLDR: whereas we see 𝚆𝚎̄𝚕𝚌𝚘́𝚖𝚎̂ 𝚝𝚘́ 𝚝𝚑𝚎̈ 🤗 𝚃𝚘̂𝚔𝚎́𝚗𝚒̄𝚣𝚎̄𝚛𝚜, language models see [𝟷0𝟷, 𝟼𝟷𝟼0, 𝟸000, 𝟷𝟿𝟿𝟼, 𝟷00, 𝟷𝟿𝟸0𝟺, 𝟷𝟽𝟼𝟸𝟿, 𝟸0𝟷𝟻, 𝟷0𝟸]
🧵 on Tokenization by examples
1/
RefinedWeb is a massive English web dataset on which falcon-40b was pretrained on. Atm, falcon-40b holds the first position on top of llama-65b on Open LLM Leaderboard.
Explore the dataset through datasets-viewer:
🤗 Hub & Transformers now support ᴢᴇʀᴏ-ꜱʜᴏᴛ ɪᴍᴀɢᴇ ᴄʟᴀꜱꜱɪꜰɪᴄᴀᴛɪᴏɴ
In this example, I'm using ʟᴀɪᴏɴ/ᴄʟɪᴘ-ᴠɪᴛ-ʟ-14-ʟᴀɪᴏɴ2ʙ-ꜱ32ʙ-ʙ82ᴋ, which is "the best performing open-source ViT CLIP model released" (more info below)
We just helped a team member from Mongolia get a work visa to France.
June 10: offer sent to candidate(&accepted)
June 16: appointment at the FR consulate in Ulaanbaatar, Mongolia 🇲🇳
June 22: visa application approved
June 30:
@mishig25
will travel to France 🎉
Go France!! 🇫🇷🇫🇷
🧵on how we implemented
Under the hood, we are using
@meilisearch
.
Decided to use it after finding its performance & feature-set to be quite impressive.
I don't wanna be that guy: but did I mention that meiliseach is written in Rust 🦀 & OSS.
1/N
PRs & Discussions
@huggingface
now support video uploading/viewing 🎥
We are super excited to add this feature since videos (screencasts) are usually a great aid to any discussion 😊
@karpathy
@osanseviero
@karpathy
you can use the webui to upload your model.bin file & use the download link afterwards as demonstrated in the video. Looking forward to your first repo on 🤗 Hub
"Textbooks Are All You Need" is making rounds:
reminding me of my earlier tweet :). TinyStories is also an inspiring read:
We'll probably see a lot more creative "scaling down" work: prioritizing data quality and diversity over
Terribly excited about open-source + on-device AI these days! Great to see
@Qualcomm
release 80+ models optimized and curated for their devices and chips on
@huggingface
:
Visual Question Answering widgets are live on the 🤗Hub!
In this example, the correct answer would have been "chilling", but ViLT🖼️📝answers are pretty close
We are open sourcing Dolly, a ChatGPT-like model that can do instruction following, created for $30, trained 3 hours on 1 server. The secret in magical human-like interactivity probably lies in a small dataset.
Catch me if you can! 🐰
@GroqInc
just added support for Gemma 7B. 🔥
Did up this simple
@Gradio
app to show how fast inference is, and imagine how smooth a LLM-powered autocomplete would feel with this.
In January, this prediction looked improbable 🎲
As of today, it looks inevitable 🎯
(with alpaca
@StanfordCRFM
, GPT-NeoXT-Chat
@togethercompute
, ChatGLM
@thukeg
among others)
Prediction:
there'll be 10 open reproductions of ChatGPT in 6 months
and if i understand correctly, models are quite smaller than LLMs so they might be significantly easier to deploy at scale (maybe even single-GPU)
📷 Introducing Open-Source Background Removal by
@BriaAI
📷
Now live on
@huggingface
, RMBG v1.4 excels in separating foreground from background across diverse categories, surpassing current open models.
See demo []
#BriaAI
#OpenSource
#AI
@briaai
Native MacOS app (in Swift) for running stable diffusion using
@Apple
Core ML framework. Might be interesting for privacy reasons (as the imgs are generated on-device).
obvvv, it is open-source: by
@pcuenq
“When AI critics get together, they talk about sentience and consciousness and emergence. When AI scientists/engineers get together, they talk about where you can get cheap GPU” - Pablo Picasso 2023
Downloading some quantized ggufs from & testing them on Vision Pro M2 neural engine. So far, the latency is good I'd say. whisper, mobileSam, & some llms (2b & 7b)
Believe you me on-device AI is going 🔥
Next event of Paris AI running club is happening on May 20th 17:00 with
@julien_c
& other members of
@huggingface
We invite anyone/everyone to join us 🤗