building the most efficient inference engine for diffusion models. head of eng/silicon at fal ai labs (
@fal
). (C)Python core developer,
@thePSF
fellow member.
day 6 & 7 of automating factorio -- i have massive mod updates. i restructured the mods & added a fuck ton of modding for simple atomic actions that an agent could take. 17 actions implemented so far (example video shows placing a furnace, without collisions).
we're cooking :)
uhm, does anyone have any idea on how to make model "unlearn" some stuff, if by any chance if it learnt stuff that it shouldn't include in its distribution?
Wait a minute, why is GPT4-o mini has the same price as GPT4-o for vision. One claims it is 255 tokens and the other one is 8500 tokens for the same input image. Like wtf?
OK guys, holy shit, gemini-1.5-pro-exp-0801 might be the captioning king (much better than GPT-4o). I need to caption a couple hundred million images
@google
where do i get the api keys
Whatup chads, I'll be joining
@FAL
and lead the research effort to develop open source / proprietary models.Ill work on boundaries of research / product, theory / practice and inference / training. I have couple head counts, dm if you wana join my team. Thx
you might ask why are we launching this now when the model is clearly undertrained instead of waiting for a few month? the reason is why iterate behind the closed doors and have no community feedback. what do we have to hide? let's build this model together. step by step.
@FAL
There is a lot more story to tell, but we are just releasing the intermediate checkpoint for now. What you are seeing is actually now what we have, and is merely a beta release!
Last month I promised if our tweet got 200 likes, we’d make whisper 20% faster (and we did it 120% faster while halving the price and maintaining WER).
Now I am saying I’ll double the performance of whisper v3 if we get 2000 likes. You go figure what will happen :)
LETS FUCKING GO MAN. LETS FUCKING GO. THIS IS JUST THE START, AURAGAN, AURAFLOW. If no one is releasing models, WE WILL. WHO CARES, it might not beat the closed source SOTA versions, but it will be our own and we'll keep developing it!!!
@FAL
@burkaygur
So much of the credit goes to
@isidentical
who did loads of technical works from data to infra management to make this happen. I dont know how to coordinate NFS amongst nodes nor preprocess massive dataset with ray... This model would have not been here without him, and btw this
We at
@fal
are sponsoring PyTorch Conference'2024! We are big users of torch here, and are continuing to push the frontiers of the provided tooling with PT2
@t3dotgg
@t3dotgg
did you see BiRefNet? it is a SOTA background removal model, even better than Bria 1.4, and open source. we also host it serverlessly FYI at
at
@fal
, we don't care about your fancy titles or ex-companies you worked with. if you are interested in diffusion models, can deliver high quality stuff, at a flash paced environment, in a consistent manner. DM me. we are hiring the best in their class.
Actively hiring an ML inference performance engineer. If you wanna squeeze the last bit of performance from our wide fleet of GPUs, work with nightly torch APIs and build SOTA tooling on top of it to achieve the extreme levels of performance shoot me a DM or an e-mail.
@inerati
it is not a design decision. It is the intuitive: expected behavior of different language constructs. e.g. how else should def fn(a=A()): … would behave? Do we give a shallow copy at every invocation? Or a deep one? Or maybe disallow it (what makes this different than a
A big welcome to freshly minted PyPy dev
@isidentical
, who got his first bug fix merged today and already opened a second feature merge request (f-string debugging expressions). Thank you!
> be me
> find out that gaudi chips are actually amazing for training, comparable perf between A100 and H100
> think intel is so much undervalued (without having any idea about other stuff)
> look at the chart and see it went down 50%, think it can't go any lower
> buy intel
Best paper award goes to Scaling Rectified Flow Transformers for High Resolution Image Synthesis by Esser & Rombach et al (aka SD3 paper). Well deserved
If meta didnt release llama, do you guys think mistral would have open sourced (still with NC license) the mistral large? Something tells me they never wanted to release a large model but meta pushed their hands
Built with WASM in a matter of hours. The entire site is defined in a single Python script (no CSS/JS, all the widgets/interactions/logic is in Python), and hosted entirely on GitHub pages. I wish somebody would have told me that it was really 'this' easy
nvidia bros, never fell into the H100 PCIE trap. there is a noticable difference in both the raw flops and memory bw. some providers sell 8x H100 NVLinked and you might think oh this must be SXM but no lol. always double check
really lucky that i can work at a company who sponsored tens of thousands of hours of H100 compute already and will continue to do so in multiples of that going forward! OPEN SOURCE FTW
Announcing fal Research Grants which provides free compute resources to researchers and developers working on cutting-edge open source initiatives.
Learn more here:
People sometimes compare companies by team sizes and think oh how are you guys supporting more traffic than 2x better funded / 2.5x more staffed companies.
A) our whole team is just cracked engineers. No BS roles
B) we work till job is done and more
C) its either dominate or
We've created an optimized endpoint just for loading multiple FLUX LoRAs.
Now running at
@fal
speed here:
Watch how much faster it is compared to our general purpose FLUX endpoint that supports LoRAs and ControlNets.
It is also half the price 💸
did you know you can train a FLUX LoRA cheaper than anywhere else on the internet while having literally higher quality thanks to our own framework? kinda crazy
@t3dotgg
It is MIT licensed, and can be self served but we are offering it as a serverless optimized endpoint and for ~2000 images per 1$ (literally free).
Ok so here is AuraFlow v0.2
Feature wise, nothing much, its pretrained bit more and spent longer time on highres-fine-tuning. (I undid couple mistakes made during fine-tuning.)
Also check out samples & comparisons on VERY COMPLEX PROMPTS: (AuraFlow vs
We are excited to announce the launch of Black Forest Labs. Our mission is to develop and advance state-of-the-art generative deep learning models for media and to push the boundaries of creativity, efficiency and diversity.
CVPR take: everyone is debating AR vs Diffusion for image generation. Am very pro on diffusion. Its beautiful. AR is a soulless rule (code) following method. Hate it.
We
@fal_ai_data
are hiring a senior+ level distributed ML systems engineer!
It's a fully remote position w/ extremely competitive pay. But more importantly you get to work with me hand to hand on the intersection of ML problems and computing
@edge
. DMs are open.
FLUX.1 [dev] is currently leading with a big margin, followed by FLUX.1 [schnell] and the long standing
#1
RealVis XL V4.0. Pro model currently needs more samples so go vote!!!
OMG
@ValDotTown
's Townie is crazy. Just showed a single example from
@fal
, and it built an AI image generation demo app under 1 minute from scratch 🤯🤯🤯🤯
i offered to pay. i'd literally pay for uv. am OK. free stuff ain't sustainable. uv changed my life. it is in the same league of software like dingboard and X for me. both of whom i am paying.
those devices are gonna get obsolete <3mo. not sustainable. on device inference is gonna be for super smol utility models but the general intellegince and main models will always be on cloud. network latencies in us is <50ms.
future of compute in a box approach
many startups are exploring the idea of providing an independent device for llm inference, like tinybox and truffle. but does this really make sense in the long term?
in the future, inference is likely to be predominantly cloud-based and
I collected a dataset of links to ~10 million public domain Flickr photos , to hopefully let neural networks learn about the visual world without relying on copyrighted material.
PixArt Sigma, SD3, Stable Cascade, XXX (*512x512). Better prompt following than cascade, better physical semantics than SD3. Aesthetics seem to be bad though, i assume we need a post-training finetuning pass?
I was going to apply for US visa to possibly attend PyCon 2022. But seems like the earliest interview date the embassy can offer is September of 2023 for non-immigrants. 1.5 years from now 😱 Guess I'll be at PyCon 2024 if I apply now?
Looking for a contractor (1-3 mo gig with the potentially to extend into a full time position) for ML-related data acquisition & processing job. Dealing with billions of images, batch processing them (e.g. feature extraction) on a distributed setup and essentially feeding the
a stylized cartoon image, yellow background, a smiling turtle holding a pizza, the turtle is wearing a white and brown cap, large eyes and a friendly expression, the turtle is standing on its hind legs, the pizza is held in both of its front legs, small clouds and trees on either
🔌 Plugins
Integrations with
@elevenlabsio
,
@DeepgramAI
,
@fal_ai_data
, and
@openai
make it easy to compose together multimodal AI applications.
We’re excited to build more integrations with the LiveKit community!