The Death of Instagram in one chart:
- 39.4% of your Instagram feed is suggested reels & posts.
- 20.8% is ads.
- Leaving only 39.8% percent from accounts you follow.
Absolutely pathetic. Captured and measured this today.
Been cooking on this for a while, but it's nearly ready. Ive built an iOS app using Snap Camera Kit to provide full body, hand, & face tracking into any app using VMC/OSC.
No calibration, no weird setup. Just set the IP of your PC &use your fav VTube app (Using Warudo here).
Every pixel of this video is synthetic and rendered in REAL-TIME.
Using a custom Metahuman in Unreal 5 paired with DeepFaceLive. A 3D base allows me to create the ideal person to deepfake onto (this body was geared for the first two shots).
My poor gpu...๐ฅ๐ฅ๐ฅ ๐
This new demo is running on the Dreamcast using the new raylib port.
The system really is quite capable and is rendering this 3D scan along with multiple realtime point lights moving around it. This demo wouldโve blown my mind in 1999.
VNyan - Yes
VSeeFace - Yes
Warudo - Yes
Animaze - Mostly. Some models joints are the wrong direction.
VTube Studio - No as there's no VMC support.
VTuber Plus - Not right now. Models are distorted.
My VTuber app will be called: SelfReflect
Sign up at to be notified when the beta will happen, and when the full launch will be!
(Tentatively hoping to launch by TwitchCon)
It appears the audience for this app has found this post. Woke up to around 5x the views and likes on this. Guess I need to hurry up and come up with a name for this app.
Been cooking on this for a while, but it's nearly ready. Ive built an iOS app using Snap Camera Kit to provide full body, hand, & face tracking into any app using VMC/OSC.
No calibration, no weird setup. Just set the IP of your PC &use your fav VTube app (Using Warudo here).
My VTuber app will be called: SelfReflect
Sign up at to be notified when the beta will happen, and when the full launch will be!
(Tentatively hoping to launch by TwitchCon)
Been cooking on this for a while, but it's nearly ready. Ive built an iOS app using Snap Camera Kit to provide full body, hand, & face tracking into any app using VMC/OSC.
No calibration, no weird setup. Just set the IP of your PC &use your fav VTube app (Using Warudo here).
So that โBlues Cluesโ clip going around reminded me I forgot to share this amazing gem here.
Steve was briefly on Cameo last Dec, and Nick Lutsko ordered one for me to thank me for the help I did on the โGremlins 3: Dawn of Desmondโ trailer ๐ .
This still makes me so happy!
@thijsniks
Great design?? This is a monumental failure. Getting things off of iOS devices is a complete mess and doing anything remotely complex between applications is way harder than it should be.
Finally, my first complete proof of concept of what AR filmmaking can be using Snapchat tech.
- New silky smooth 3D body tracking
- Face tracking
- 3D Camera tracking
- 6DOF Seurat environment
All running simultaneously on a phone. Just point your phone at a friend & film. (1/4)
I gotta finish some other projects tonight, so I'll leave it up to someone else to carry the torch. Here's a quick rigged FBX of Zuck's avatar head with ARKit blendshapes ready to go.
This full paper is excellent and confirms exactly what I suspected.
"We also identify cases where diffusion models, including the popular Stable Diffusion model, blatantly copy from their training data."
Full paper:
Since we used CamTrackAR to get camera movement data, the shots where the camera is moving couldn't be shot on a green screen. So it had to be manually rotoscoped. RunwayML helped provide a good start but it still needed some manual cleanup. But the effect is worth it.
Steam Deck running VR.
This is the PC version of VRChat streaming to Quest 2 via Virtual Desktop.
Considering itโs unoptimized, unsupported, and running on a battery, itโs not too bad.
Oculus Link over USB hasnโt worked yet.
Kevin Feige introduces "Phase Gremlin" at
#SDCC
!!
#SDCCMarvel
๐คฏ๐ฒ
I sure hope they have Writer/Director/Star of "Gremlins 3"
@NickLutsko
back at the helm for all of these!
AR restaurant menus are *INCREDIBLE* in AR glasses.
Here's an example I made for the Next Gen Spectacles.
The video does not do this justice. The food looks like it's right in front of you and looks delicious!
(Thx to Sketchfab market for the scans)
#spectacles
#snapchat
Facebook isnโt hiring 10,000 new XR engineers because they need them to build their platform.
Theyโre hiring them to deprive the rest of the industry of XR engineers.
Theyโre hiring them to stop any competition before it can even get started.
Got to quickly do a new 3D scan while I saw Nathan. The detail Im getting from this first pass is wild. Cant wait to see this in motion.
About 100 photos at 20MP on a Sony ZV-1. When doing photogrammetry, 4K video works in a pinch, but good photos can really elevate the quality.
I was able to generate a (rough) 3D model of the Joker stairs based on wide-angle footage from Youtube. Ended up being around 1 million polys, but Unreal handled it with ease. Here's a quick tour of the whole environment.
BREAKING NEWS:
Due to recent market conditions, Crypto. com Arena will be turning into a year-round Spirit Halloween, effective immediately!
#Cryptocrash
Song by
@NickLutsko
I made a special Snapchat AR Lens of Desmond dancing that uses Nick Lutskoโs โSongs on the Computerโ vinyl as a tracking marker. Works on Next Gen Spectacles too for a real glimpse at the future.
Would a one-time $49 price be too much for my VTuber app?
Trying to make sure I could generate enough revenue to support it well & add new features (like standalone mocap recording on device). And I hate subscriptions.
But I want to be mindful that most people VTube as a hobby
Updated my "Gremlin Man" Unreal 5 demo. The cape now has cloth physics, and you can fly around Denver thanks to the awesome plugin from
@CesiumJS
.
#UnrealEngine
Man, so many Vision Pro demos I'm seeing posted just consist of 2D screens all over the place. Where's the creative new ideas that really use the space?
All in Snapchat. Real-time body capture with smoothing, face capture, camera tracking and body in-painting. Cleaned up the โol TMNT lens and realized I donโt think Iโve quite showed off this whole combination of features at once.
If anyone is trying to claim that Beetlejuice is *not* still being held as a political prisoner, just send them the link to this very real 24/7 live camera feed of his cell.
Present me: "So in 2021 you'll be speaking at an augmented reality conf about producing holographic content and the future of wearables. You also rant a lot about NFTs"
Past me: "I don't understand any of that."
Present: "...and you're a girl."
Past: "Ah yea that makes sense"
It's so cool what we can create at home these days. This movie was one of my favorites as a kid.
Here's a test render I made in Unreal Engine 5. Toys scanned with Polycam Photo mode. Cleanup in Cinema 4D.
#3dscanning
#unrealengine
Desmond Snapchat Lens update!
Instead of just watching Desmond dance, now he can mimic YOUR dance moves.
Features full 3D body tracking, head rotation, and face animation starring Desmond from "Gremlins 3: Dawn of Desmond"!
@nicklutsko
#gremlins3
Huzzah! Iโve settled on a name for my upcoming VTuber/ Mocap app. And a good domain was still available for it. Time to do some branding work and get ready for the upcoming public beta test.
(It will be free during the beta test period)
Therapist: โWhat are some ways you have coped with depression in the past?โ
Me: โWell, so 5 years ago today, I had this inflatable T-Rex costumeโฆโ
I recently made a full 3D model of Desmond from "Gremlins 3:DoD".
I wanted to see how he would look in a video game so I tossed this proof of concept demo together in Unity (city from the asset store).
The cyberpunk game we deserve!
#Gremlins3
#unity3d
@NickLutsko
Did some digging into the data dataset used to train Stable Diffusion. Thereโs a lot of porn. A LOT.
I disabled the content filter on SD and gave it simple prompts like โnurseโ or โsecretaryโ. Big yikes. Stable Diffusion has an extremely distorted view of women.
House Republicans introduced a bill that would ban any discussion or material in federally funded programs about LGBTQ people in any event with children under 10 years old. This is a federal "Don't Say Gay" bill.
My newest Next Gen Spectacles demo is just so incredible to see in-person. This video does not do it justice.
The demo turns this Porter Robinson vinyl into a hologram concert. It really looks like he's standing there when looking through the glasses.
#snapchat
#spectacles
This livestreaming/Vtuber setup now has hands & face tracking on a single PC and webcam. Using a custom Unity app and Snap Camera Kit (for Web).
I also finally got smoothing and 60fps interpolation working well on the Snapchat tracking data-stream.
Itโs here! Iโm so honored I got asked to produce all of the visuals for this amazing song! Used Unreal and a 3090 to make rendering this much 3D possible on such a quick timeframe.
Iโll do a more technical write up later, but for now just enjoy!
Happy Halloween ๐ป ๐!
Exclusive new footage from "Titanic 2"!!
Starring Beetlejuice and the old lady from the first Titanic.
From the creative mind that brought you "Gremlins 3: Dawn of Desmond",
@NickLutsko
Now you too can lovingly toss a pillow a Mark Zuckerberg in AR with this Snapchat Lens.
Works with Next Gen Spectacles today!
Link:
#snapchat
#zuckerberg
#spectacles
@CorridorDigital
While I strongly disagree with the view in the video that AI training on images is the same as humans learning a style, there is also a *ton* of manual work here that you guys did a great job with.
Ebsynth is also a quiet unsung hero in VFX.
I was catching up on "Falcon & The Winter Soldier" and I couldn't believe that this character showed up!
First WB and now Marvel!
@NickLutsko
, did Marvel get permission to do this??
Messed around with the character model from Superman 64. Remeshed and rigged it to make it work better for animation. Then tested some subsurface materials & added on some of my own mocap.
Testing out a cool new Mediapipe plugin for Unreal that provides body & face with just a webcam. And using the Realsense T265 for camera tracking all on one PC.
Did all the mocap and rendering in about 15 hours yesterday. Made the characters between Mon & Tuesday. Rendered in Unreal. Tried my best to make it as spooky as possible.