Damien Henry @dh7net Twitter profile

Pinned Tweet

Damien Henry

1 year

I created the Google Cardboard almost ten years ago but quit VR soon after to focus on Machine Learning. VR is based on a ridiculous misunderstanding. Let me explain why in this thread.

338

496

4K

Last Seen Profiles

@lalsinghrath123

@cocopark_io

@stw_pdg

@cukienaknikmati

@stw_pdg

@yaawonne

@Anisogamyrules

@farersk

@pizzamiheart

@pocapoee

@nixthebratzdoll

@taeS2lele

@iwashi_saba

@pnikmttntebunda

@leepooool

@takataz7777777

@fdremoe

@brad76525

@EcemCem17

@404j74x

@Raymund3285999

@beltriski

@meepoing

@yuri27_violin

@chrisdonahuey

@KOBCOB2

@panduull

@buttermhlee

@Vetri37176519

@hm0d112

@Angy_Afj

@20190517S

@GayCostaRica2

@phillyfnslayer

@kldhny1815927

@meiidaren

Damien Henry

@dh7net

2 years

How works? A short thread to explain how realtime relighting of a picture is possible.

95

1K

8K

Damien Henry

@dh7net

1 year

Ironically, VR is a bad medium because the only immersion that matters is the immersion of our minds.

43

48

831

Damien Henry

@dh7net

1 year

VR takes the problem the wrong way. It tries to immerse us by fooling our sensory system. VR is lying to our bodies. That’s why most VR experiences are unpleasant after a couple of minutes.

24

44

711

Damien Henry

@dh7net

1 year

Reading a book is decoding black symbols on a white page. Our brains decode these abstract symbols, creating a virtual world inside our brains. There is no need for brain implants; we can already read the “Matrix”.

14

57

697

Damien Henry

@dh7net

1 year

The VR promise is to bring “a sense of presence” to computing. But the “sense of presence,” seeing things as if they were there, adds little value beyond the wahoo effect it generates. The misunderstanding is between "immersion" and a "sense of presence."

19

30

624

Damien Henry

@dh7net

1 year

Reading a book is real immersion. When immersed in a book, our senses are disconnected from our bodies, and we no longer feel our direct environment.

5

22

622

Damien Henry

@dh7net

1 year

Listening to a good story is immersion. Our brains' capacity to immerse ourselves in someone else idea is a superpower that allows us to co-create.

5

20

620

Damien Henry

@dh7net

6 months

Someone asked me recently why the Paris AI ecosystem is so 🔥 these days. On the surface, it looks like Paris became a major #AI center overnight, but it didn't. It took time and didn't appear by accident. Here is a story that started more than 10 years ago. 👇

18

110

601

Damien Henry

@dh7net

4 years

This code is so beautifully written, it almost hurts. 😍 Scraping + Analysis: 158 lines 😍 Server: 77 lines 😍 HTML: 50 lines 😍 JS: 61 lines 😍 CSS: 127 lines Total: 396 lines for a full app! Congrats @karpathy 🙏🙏🙏

GitHub - karpathy/covid-sanity: Aspires to help the influx of bioRxiv / medRxiv papers on COVID-19

Aspires to help the influx of bioRxiv / medRxiv papers on COVID-19 - karpathy/covid-sanity

github.com

9

91

594

Damien Henry

@dh7net

7 years

1h video 100% made by an algorithm, no editing, cut, post production: ; Music for 18 musicians by Steve Reich. v1.0

19

147

519

Damien Henry

@dh7net

4 years

I'm leaving Google and I'm starting an exciting adventure! Stay tuned!

38

8

519

Damien Henry

@dh7net

5 years

Was cleaning my basement today due to the #confinement and found the very first iterations of the Cardboard #VR headset.

15

53

509

Damien Henry

@dh7net

1 year

Our perceptions fade away when we are immersed. For instance, while reading a book in the tube. On the other end, VR uses our body's senses as the medium, distracting us from the story, the message, and the real thing we want to be immersed in.

7

14

417

Damien Henry

@dh7net

2 years

IMO, the coolest part of #dalle2 and other diffusion-based technology is not image generation, but image edition!

14

28

405

Damien Henry

@dh7net

1 year

OK, that's all for today. There would be so much more to say about VR, but I'll keep it for next time!

60

1

395

Damien Henry

@dh7net

2 years

I just found this video of a project I did 5 years ago. The screen is not displaying what the webcam sees but a prediction of what the webcam will see next.

12

25

374

Damien Henry

@dh7net

1 year

We are immersed when we are focused on our inner thoughts, controlled by some external signals.

3

10

349

Damien Henry

@dh7net

2 years

First, we trained a deep neural network to predict a depth map and a normal map from a picture.

12

13

342

Damien Henry

@dh7net

6 years

#NeurIPS2018 is over! Time to wrap up! In this thread I'll share what I found the most interesting in the field of ML and creativity. Everything below worth your time IMO!

1

82

290

Damien Henry

@dh7net

2 years

The depth map allows generating a 2.5D model (almost 3D, but we are only adding depth to the picture plane) The normal map is then applied to the 2.5D model.

4

15

267

Damien Henry

@dh7net

2 years

You probably see this paper, Pix2Pix I was curious to understand how it works so I dug a bit. 1/6

1

27

231

Damien Henry

@dh7net

3 years

The best thing about ML is simplicity. This paper explains how to create a speech to text system that works on all languages, without labelled data. It's fascinating to see that this task can be achieved with a simple concept.

Alexis Conneau

@alex_conneau

3 years

New work: "Unsupervised speech recognition" TL;DR: it's possible for a neural network to transcribe speech into text with very strong performance, without being given any labeled data. Paper: Blog: Code:

3

95

464

0

33

196

Damien Henry

@dh7net

2 years

You can try to relight your own pictures there: And have more technical info on our blog here:

Technicalities Behind Image Relighting

The technicalities behind our image relighting app

clipdrop.co

7

21

193

Damien Henry

@dh7net

7 years

Very proud of done at the @googlearts Lab!

Google Art & Culture Experiments - Art Palette

Find artworks that match your chosen color palette.

artsexperiments.withgoogle.com

3

51

186

Damien Henry

@dh7net

2 years

@YayaChanArtist Thanks for sharing our tool!. It was originally made for portraits. Here is how it works:

Damien Henry

@dh7net

2 years

How works? A short thread to explain how realtime relighting of a picture is possible.

95

1K

8K

4

7

168

Damien Henry

@dh7net

6 years

Very proud to work with @NASA and release today a new @googlearts experiment. We used NLP to make a semantic maps of the concepts extracted from NASA images metadata. Check it out:

NASA's Visual Universe

To celebrate 60 years of NASA, the Google Arts & Culture Lab explored NASA's archive of 127,000 historic images stretching back to 1915, and analyzed them using machine learning - bringing surprising...

artsexperiments.withgoogle.com

1

65

168

Damien Henry

@dh7net

2 years

Then we use @threejs to add virtual light to the scene.

1

2

144

Damien Henry

@dh7net

2 years

A normal map can be seen as a derivative of the depth map, but in practice, it adds a lot of information about how light should be reflected or not with much better precision than the depth map. More info if you are curious:

Normal mapping - Wikipedia

en.wikipedia.org

1

2

137

Damien Henry

@dh7net

1 year

Putting Google Brain under Deepming is a smart move, but it won't be enough to solve the main issue that Google is facing. The problem never was research. 1/N

12

7

139

Damien Henry

@dh7net

2 years

@Piles0fPeaches This is done by the @clipdropapp team! We explained how it worked here:

Damien Henry

@dh7net

2 years

How works? A short thread to explain how realtime relighting of a picture is possible.

95

1K

8K

2

8

127

Damien Henry

@dh7net

2 years

@izbubbles Fun fact: this tool was created for portraits initially! Here is how it works:

Damien Henry

@dh7net

2 years

How works? A short thread to explain how realtime relighting of a picture is possible.

95

1K

8K

0

15

134

Damien Henry

@dh7net

2 years

And that's it! That's the way you can create pictures like these ones:

Onur Tasar

@onurxtasar

2 years

With the app, you can transform your portraits into extraordinary visuals. 🪄 💫 ✨ Photo by Léa Genoud & @clipdropapp

2

35

3

2

130

Damien Henry

@dh7net

1 year

@mo1ok Every book lover will tell you that a book is better than a movie.

30

0

128

Damien Henry

@dh7net

6 years

I played a little bit with #keras . It's truly amazing how simple it is to test an idea with it.

1

13

116

Damien Henry

@dh7net

5 years

Don't fear the machines. Your are made by a *MUCH* superior technology.

Massimo

@Rainmaker1973

5 years

Molecular visualizations of DNA based on X-ray crystallographic models

7

289

1K

1

31

113

Damien Henry

@dh7net

6 years

Hey if you want to generate video based on next frame prediction, I published my code: #MadeWithMagenta

A train window

Inspired by Steve Reich’s Music for 18 musicians, I used machine learning to create a visual to go along with it: It uses videos recorded from train windo...

magenta.tensorflow.org

5

26

112

Damien Henry

@dh7net

6 years

Here a tutorial about Mixture Density Network. The notebook is highly inspired from @hardmaru MDN tutorial, but I adapted it so it can generate mixtures for multidimensional outputs.

2

31

117

Damien Henry

@dh7net

6 years

I'm so glad I have the chance to work with @WayneMcGregor . Using Machine Learning to build a choreographic tool that learns from his archive:

Google's latest experiment teaches AI to dance like a human

Together with Google Arts & Culture, British choreographer Wayne McGregor developed an AI capable of predicting dance moves in his particular style.

www.wired.com

1

28

102

Damien Henry

@dh7net

7 years

I'm now able to create videos using an ML algorithm. No other inputs than training data. Excited!

9

18

88

Damien Henry

@dh7net

6 years

GAN Demystified, WTH do they learn. An article from @henddkn who I had the chance to meet during #NeurIPS2018

GANs Demystified — What the hell do they learn?

This article is a summary of the research paper “GAN Dissection: Visualising And Understanding Generative Adversarial Networks”. The paper…

towardsdatascience.com

1

26

85

Damien Henry

@dh7net

7 years

A new home for @googlearts experiments!

0

36

86

Damien Henry

@dh7net

2 years

Edit: This thread is gaining popularity, so I'll make more content like this one. Follow me or @clipdropapp if you are interested.

4

0

82

Damien Henry

@dh7net

1 year

@ojoshe Music don't try to mimic reality the same way a book don't try to imitate reality. But it's a direct channel to your emotions. Sound engineering, 3d sound system, or better image quality are just nice to have.

6

2

82

Damien Henry

@dh7net

4 years

ML will change the way we create images and video! Amazing demo from @runwayml !

Cristóbal Valenzuela

@c_valenzuelab

4 years

Content editing with @runwayml

13

58

410

0

17

68

Damien Henry

@dh7net

1 year

I'm maybe a bit biased, but this is still the best VR headset IMO.

2

4

61

Damien Henry

@dh7net

2 years

#Dalle , #StableDiffusion , etc.. are amazing to imagine things like avocado chairs, but they can't draw specific objects. This is a major blocker for those who want to use this tech for advertising a real product. But this will change. 👇 This thread to explain how.

7

60

Damien Henry

@dh7net

3 years

If there is still someone in the room that don't understand why Computational Photography is a game changer, here another proof ;-)

Cyril Diagne

@cyrildiagne

3 years

Today I'm thrilled to launch ✨ A free tool to remove objects and defects from any picture. 💻 Try it: 🤖 Code: ⬆️ PH: 🥰 100% free & open-source thanks to @clipdropapp

20

266

1K

1

11

58

Damien Henry

@dh7net

3 years

An important step, but the journey has only just begun!

Clipdrop

@clipdropapp

3 years

We raised $1,5M to reinvent AI-first photo editing! Join us!

9

16

145

11

2

55

Damien Henry

@dh7net

5 years

Great artists can understand technology better than who is building it.

Omid Djalili

@omid9

5 years

Extraordinary clip from 20 years ago. David Bowie sharing what he feels about what the internet is going to turn into. And he’s not wrong. Paxman: it’s just a tool tho isn’t it? Bowie: No it’s not.

923

20K

64K

3

11

55

Damien Henry

@dh7net

6 years

We need to get rid of anthropomorphism to reveal it's real power. ML can do things that no human can do, from signals we can't sense. 2 ex: Infer a user identity from keystroke: Infer a user pose by analyzing radio signal:

Recurrent neural network-based user authentication for freely...

Keystroke dynamics-based user authentication (KDA) based on long and freely typed text is an enhanced user authentication method that can not only identify the validity of current users during...

arxiv.org

2

12

56

Damien Henry

@dh7net

1 year

@JungleSilicon AR can be a distraction, too. But simple headphones augment reality and are just amazing.

3

0

56

Damien Henry

@dh7net

7 years

How possible did I miss that: Takuya Kato doing face->ramen->face translation using CycleGan?

1

13

53

Damien Henry

@dh7net

6 years

Hey! I'm curating a list of videos done using ML. Can you please add your findings by replying to this thread? Please don't be shy and feel free to add your own work. I'm also curious about the tools used to produce them.

19

11

53

Damien Henry

@dh7net

2 years

Very proud of this new feature we are launching today!

Onur Tasar

@onurxtasar

2 years

🔥🔥 Today we, @clipdropapp are launching our 💡image relighting #AI application 💡 The app allows you to apply professional lights to your portrait images 📸 in real time ⚡ Try it now! it is free 🙂 #photography #MachineLearning

16

103

630

3

9

53

Damien Henry

@dh7net

5 years

Inspiring work: Teaching agents to paint inside their own dreams from @ReiiYoda via @hardmaru

Teaching agents to paint inside their own dreams

In this post, I talk about using World Models to train agents to paint with real painting software. This includes my thought process, approach, failures, and some future work in this direction I’m...

reiinakano.com

0

10

53

Damien Henry

@dh7net

4 months

@francoisfleuret @Plinz of course.

2

0

48

Damien Henry

@dh7net

4 years

I'm now COO of :-) I have the privilege to work with the 2 very best co-founder you can think of: @cyrildiagne & @jblanchefr I can't be more excited by the idea of helping people through machine learning. There are so many things waiting to be done!

1

0

52

Damien Henry

@dh7net

1 year

Clipdrop is Hiring. Come have fun with us, reimagining image editing and more.

Stability AI Careers: Join Our Team of Innovators — Stability AI

Explore exciting career opportunities at Stability AI. Join our team of innovators and contribute to cutting-edge products in AI and technology. Discover your path to a rewarding career with us.

stability.ai

3

9

50

Damien Henry

@dh7net

6 years

Our code related to gets 51 github stars after less than 1 month. Good to see that our work is useful to the creative community:

Google Art & Culture Experiments - Art Palette

Find artworks that match your chosen color palette.

artsexperiments.withgoogle.com

0

19

48

Damien Henry

@dh7net

1 year

It's easy to think that chatGPT is just an LLM with a simple UX on top of it. IT IS NOT. It's a PRODUCT. 2/N

2

4

50

Damien Henry

@dh7net

2 years

I like this image but could not find a good caption for it. Feel free to try.

15

3

48

Damien Henry

@dh7net

6 years

Generate video from a text

0

13

48

Damien Henry

@dh7net

8 months

Me in 2015 vs OpenAI in 2024

Greg Brockman

@gdb

8 months

Announcing Sora — our model which creates minute-long videos from a text prompt:

1K

4K

20K

2

3

49

Damien Henry

@dh7net

4 years

I told you! Computational photography/video is eating the world!

Jia-Bin Huang

@jbhuang0604

4 years

Annoyed by shaky videos? Check out our work on stabilizing shaky handheld videos *without cropping*! Neural Re-rendering for Full-frame Video Stabilization Project:

7

160

931

0

12

48

Damien Henry

@dh7net

7 years

Hey! There is an open call for an artistic residency with @somersethouse about ML, with mentoring from @googlearts and @memotv :

2

33

46

Damien Henry

@dh7net

4 years

Today we just released 5 new experiments with the amazings @bgirschig @carolinebuttet @jblanchefr @jonathan_tanant @onemorestudio @romaincazier @vogler_voice and more talented people at the @googlearts lab. Please have a look!

1

13

45

Damien Henry

@dh7net

4 years

Very few people knows that just after the #deepdream storm that @zzznah initiated, he continued to refine this technique and produced crazy beautiful artwork while the world was invaded by puppies eyes. I feel incredibly lucky and grateful to have this one at home. 😊

3

6

46

Damien Henry

@dh7net

1 year

UX x AI is not a nice interface on top of a nice model. It is about incorporating the UX inside the model at every possible level. 6/6

4

6

44

Damien Henry

@dh7net

6 years

Joelle Pineau talked about reproducibility in reinforcement learning. She gave a a checklist if you want to summit to #NeurIPS2019 and contribute to solve the reproducibility crisis.

2

9

44

Damien Henry

@dh7net

5 years

Let's face it: the Walkman and all it's mp3/offline/online based sequels are the very best AR devices you can think of. The layer added to reality is so powerful, it can transform the worse commute into the best moment of your day.

4

3

43

Damien Henry

@dh7net

7 years

It's going somewhere. More to come soon!

3

4

44

Damien Henry

@dh7net

6 months

In 2015, @ylecun created FAIR Paris. (Facebook AI Research, now Fundamental AI Research). Yann Le Cun is recognized as one of the 3 inventors of deep learning as we know it today. Deep learning was at least a third French thing from the beginning! (The other two are Canadians.)

3

1

43

Damien Henry

@dh7net

8 years

How to hack a Daydream controller:

How I hacked Google Daydream controller

Reversing and making it compatible with Apple iOS devices.

medium.com

3

24

45

Damien Henry

@dh7net

4 years

One of the best @googlearts experiment to end the year in beauty!

1

3

40

Damien Henry

@dh7net

1 year

In 2023, building a product is no more spending time in Figma to polish the "UX" The "UX" is embedded in the model itself. But a good "UX" won't emerge magically from random data or random refining. It needs to be opinionated. 4/N

3

1

41

Damien Henry

@dh7net

3 years

"Out Painting" is a technique to extend a picture by guessing what is around. Here are some examples. The right image is created by an algorithm from the small image on the left.

2

4

42

Damien Henry

@dh7net

1 year

How to do that? It's an entirely new field that is almost unexplored. There is only one way, the maker way: Do, try stuff, iterate, learn, fail, learn, repeat, retry. But always with the product you want to build in mind. 5/N

1

0

41

Damien Henry

@dh7net

2 years

@invissivni @Piles0fPeaches Great to see it's working on anime too! More info there:

Damien Henry

@dh7net

2 years

How works? A short thread to explain how realtime relighting of a picture is possible.

95

1K

8K

2

38

Damien Henry

@dh7net

6 years

The most beautiful paper you'll read this week: It opens the path to HI-REZ image generation using ML. Congrats @zzznah !!!

1

7

39

Damien Henry

@dh7net

3 years

I created the Google Cardboard and still is a VR enthusiast. But it's important to keep in mind what is real and what is not. I'll put my money on a trip to Iceland rather than on a new headset for sure.

ᴊᴏᴀɴɪᴇ ʟᴇᴍᴇʀᴄɪᴇʀ ⏚

@JoanieLemercier

3 years

Welcome to the Icelandverse 😂

1

4

18

1

38

Damien Henry

@dh7net

8 months

@Gia_Vang Meanwhile in Paris:

0

3

37

Damien Henry

@dh7net

1 year

@MasonStormAI @mo1ok Was it the visual or was it the music? In my experience music is what triggers emotions. Video without audio never does.

10

0

39

Damien Henry

@dh7net

6 months

In 2013 @Xavier75 created Ecole 42. This unusual school accepts students from many different backgrounds and trains them to code based on peer-to-peer learning. (The concept was imported into the state by Lambda School and some others later.)

3

0

37

Damien Henry

@dh7net

4 years

Twitter at its best: When someone ask a tech question, and the creator of the algorithm himself replies directly. 😘

0

6

36

Damien Henry

@dh7net

2 years

MakeAVideo is so "Yesterday" ;-) Look at what this other tech can do!

3

7

37

Damien Henry

@dh7net

2 years

@kohaku__0 Fun to see @clipdropapp used there! Here is more info about how it works:

Damien Henry

@dh7net

2 years

How works? A short thread to explain how realtime relighting of a picture is possible.

95

1K

8K

0

14

37

Damien Henry

@dh7net

5 years

Each dot is a sentence from Ulysses, James Joyce. Organized by similarity: Bert + t-SNE. Color is the sentence position in the book. It fascinating to see that NLP is able to discover some inner structure in this massive master piece.

3

5

37

Damien Henry

@dh7net

5 years

I have seen this picture a couple of time today. Each time IT MAKES ME SO HAPPY!!! This picture is *Perfect*.

MIT CSAIL

@MIT_CSAIL

5 years

Here's the moment when the first black hole image was processed, from the eyes of researcher Katie Bouman. #EHTBlackHole #BlackHoleDay #BlackHole (v/ @dfbarajas )

483

14K

49K

1

0

36

Damien Henry

@dh7net

2 years

Following @karpathy recommendation, I just read "Understand" I loved it. While most SF authors optimize their writings for video adaptation, Ted Chiang uses the written language medium to convey much more than any video would. Any recommendations for other books like this?

6

2

34

Damien Henry

@dh7net

2 years

Large Language Models (LLM) like GPT3 are great for completing a sentence or a paragraph. This is exactly what they have been trained for. A surprising consequence, they are quite good at question answering. But what if they can use a calculator? Or consult Wikipedia? 👇

2

4

33

Damien Henry

@dh7net

1 year

When building the primary Dataset, you must first consider the product you are creating. The loss of the model should come after that. Every choice that falls into research when you write a paper needs to be made with the product in mind at every step. 3/N

1

34

Damien Henry

@dh7net

5 years

OMG, I just rediscovered my old Flickr account. I found 360° panorama created using a DYI robot. In 2004 kids!

1

0

33

Damien Henry

@dh7net

2 years

“If a picture is worth 1,000 words, a prototype is worth 1,000 meetings.”

2

3

35

Damien Henry

@dh7net

5 years

A novel where the characters have to reprogram themselves, genuinely change their tastes, personality and behavior to escape an AI which is enslaving them.

1

0

33

Damien Henry

@dh7net

4 years

The @googlearts was also busy keeping the lab open despite most of us stuck at home. With @zachlieberman @onemorestudio @cyrildiagne and so many more!

Open Lab: Home Sprints by Google Arts & Culture Lab, MIT Media Lab - Experiments with Google

Since 2009, coders have created thousands of amazing experiments using Chrome, Android, AI, WebVR, AR and more. We're showcasing projects here, along with helpful tools and resources, to inspire...

experiments.withgoogle.com

0

6

32

Damien Henry

@dh7net

7 years

Funny sign you can find in the @googlearts lab. I think this one is from @kikko_fr

2

1

34

Damien Henry

@dh7net

2 years

Stable Diffusion can teleport any product anywhere! Which one is a real picture, and which ones are fake? Given the pace of research, do you think you'll be able to tell the difference in one week? ;-)