Hyper excited to announce our paper Glaze, which protects artists from AI mimicry. This is *far* from stopping AI once and for all, but we believe it is a necessary first step towards artist-centric protection tools while longer term (legal, regulatory) efforts take hold.
AI-generated images are winning art contests, adorning book covers and leaving human artists worried about their futures. “That data is my artwork, that’s my life. It feels like my identity,” one woman said.
A new tool is trying to protect human-made art.
In this project, we looked at how hard it is to have text-to-image models generate wrong images for everyday terms. Turns out, you just need to insert 50 poison data to have "dog" becomes "cat", "fantasy art" becomes "oil painting". More unexpected results in the paper.
By now, I'm guessing most have already seen the news on our new project, Nightshade. Lots of artists sharing it, but here's the article from MIT Technology Review (thank you to the wonderful
@Melissahei
), and a thread explaining its goals and design.
Very excited to finally release Glaze. We took the extra time to make the app useable for artist and can run locally on personal laptops.
Again, this is one of the first steps to stop invasive AI. We will ofc continually to improve the tool and continue this line of research.
It's a big day.
Glaze, our tool for protecting artists against AI art mimicry, is now available for download/use at
Glaze analyzes your art, and generates a modified version (with barely visible changes). This "cloaked" image disrupts AI mimicry process.
Just wanted to say thank you to all the wonderful artists who have helped make the Glaze project possible. We are honored to announced that Glaze received a Distinguished Paper Award at USENIX Security Symposium, and the 2023 USENIX Internet Defense Prize.
@ZakugaMignon
The exact same applies to researchers. I thought these models were the coolest thing when it came out. Till I realized the coolest are the artists, whose hard work are exploited by all these companies.
This type of poisoning can also be used for protection. Protecting copyrighted content from scrapers. (we use clean-label poisoning, so we can embed poison in benign contents on the Internet).
@SemiPerfectDren
That's a great question. Yes, it does. Except for this case, someone could upload a bunch of random sketches or pictures (not their finest art) and apply these artifacts on them. Then these AI trainers who are scraping entire Internet, will absorb these poison into their model.
@leos_ng
Currently, we plan to release user end apps. The main reason is that this way, the users' original art pieces never leaves their machine. But if there are enough interests, we are happy to host a service.
I never thought about this implication while building the tool: even Glaze is broken by adaptive attacks, the act of adaptive attack itself has legal and ethical consequences.
Have been wanting to do this for a while now. I finally put together a full list of Test of Time papers at Top 4 security conferences.
Hope this can help people (esp. junior folks like me) to appreciate and learn from these papers. 🤗
@Dreni_Drenstar
Unfortunately, protecting against stable diffusion requires us to have a deep understanding of how those models work. I have been genuinely impressed by these models, which has made me realize the damages they can cause to artists.
@ken57967735
Yes, we have tested on JPEG. But ofc, if you compress the images too much or add too much noise, the protection performance degrades. Then at the same time, those low-quality images are not very useful to mimicry the artist's style.
@_master117
Great questions. We discussed some of these limitations in the last section of our paper, if you are interested to take a look. In short, this is far from the perfect protection, but a step in this direction and hopefully motivates follow up work.
@ZakugaMignon
Yeah. Hope with all the actions (legal, glaze, etc) we are taking and actions we will take. We can get ethically build generative AIs in not-too-far-away future.
@SemiPerfectDren
Yup giving them incorrect ALT text would definitely mess up these models more. But we need to be careful here since people can easily give up on using ALT text and use some of those image captioning tools.
Hi all. Realized today we have not updated folks on Glaze things in a while. There's so much happening on Glaze and related topics. Some are definite, and many are still evolving. So here's a thread. 1/n
@severian22
@neilturkewitz
I would say most of these generative models have similar vulnerability. Here is a paper that explored similar space with LLMs.
@kortizart
@ravenben
@MystWolven
Haha I am enjoying myself with everything happening. All the wonderful support will keep me going. And oh yes, so many video games to catch up!!!
Despite
@ravenben
@heatherzheng
keeps saying they are social media managers, all of these would not be possible without their guidance over many many years (7 to be exact).
@Aeyga_X
Yes to both questions. We released the app fairly fast so ppl can use it asap but we have many ideas on further improvement that we will slowly build. Also incorporate any feedbacks from artist community.
Our paper on immunizing images to diffusion model-powered malicious manipulation is out ! This approach, combined with policy incentives, aims to raise the cost of such unauthorized image editing.
w/
@hadisalmanX
@Alaa_Khaddaj
@gpoleclerc
@andrew_ilyas
@Sketch_Ferret
Thank you Kim for your help!! This project would not be possible (literally) if you haven’t reached out to us after seeing our Fawkes project.
@Dreni_Drenstar
At the same time, there are many alternative initiative to protect artists (e.g. legal). Likely more to come in the near future. We are simply offering one possibility, which has its pros and cons.
@StephenLCasper
Awesome and very important work, Stephen!! I wonder whether we can take one step further to prove (probabilistically perhaps) that the generated images learns from the original work.
@NecroKuma3
Yeah those won’t work. But the fact companies are desperately adding them shows they are concerned about legal consequences. If they are confident in their “fair use” arguments, why bother?
@Rahll
Given everything happened recently, Wall E is probably a more optimistic view at this point. “In Time” is what I think it is gonna be like — very few controls all the resources and wealth, while the rest of us struggle to make a bare living.
@GSNotArt
@RikuNKMR1
Yes, Glaze is not designed for the generic case. We do have some early ideas on targeting the generic models, but the situation is more complex (even some potential ethical concerns). We will likely work along those lines after Glaze's app release.
@Alkraas_art
@TheGlazeProject
Yes, we are aware this. I am not familiar with Kasperky, but I believe there should be a way to "whitelist" Glaze while keep the anti-virus running on the rest of the files.
We are organizing the first
#ICLR2023
workshop focusing specifically on Backdoor Attacks and Defenses in Machine Learning (BANDS). The submission deadline is Jan 25, 2023. Please consider submitting your awesome work to our workshop and spread the word!
@TheGlazeProject
@axl99
Yes that is correct. Ideally, we will implement something smarter like automatically detect multiple styles and switch targets.
@TheEliRiots
@TheGlazeProject
@ZakugaMignon
Not linux specifically, but we are putting up a web service to host Glaze on our servers, so that it does not require any resource on your PC. Should be up at some point in the summer.
@chshibo
My first first author paper was rejected 3 times before it got in. Tbh, it is a blessing in disguise for me. I learnt so much from those rejects.
@ricpacca
@YingchenWang96
Super cool work
@ricpacca
. Now I just hope my usenix talk is not scheduled at the same time as yours. If so, no one will show up to my talk haha.
@Rahll
@palomamcclain
Same. I was so exhausted from all the twitter trolls during glaze initial release and had to uninstall twitter. Only start to getting back recently. Life is a lot better after I give less crap about random ppl on the internet.
@Rahll
These are soooo good!! Bad translation of course but they are delicious. They shred the dry meat somehow and make into fillings. We had way too much of these during Covid lockdown.
@poolio
One of those bugs that haunt me. To add more, cv2 resize, PIL image.resize, skimage.transform.resize are all slightly different, especially if the image is preprocessed.
@dataisgone
@TheGlazeProject
@ZakugaMignon
Yes absolutely. We are receiving feedbacks from a diverse set of artists and will improve accordingly. But it would be a longer term process, as we are a small team.
@florian_tramer
Then, of course, there is always the risk that some people think these tools are perfect and will solve all their problems. While inevitable, we could minimize them by better communicating limitations etc.
Researchers have a tool to help you post photos of your loved ones online without feeling guilty about making them identifiable to face recognition systems:
@florian_tramer
Might a bit off topic. At individual user level, I totally agree with what you said: user has no 2nd shoot & nothing we can come up is remotely future proof. In a broader sense, however, I believe these tools give user leverages to push back and balance the power dynamic (a bit).
@florian_tramer
@wielandbr
@aleks_madry
I am not sure where the incentive is for people to fix this? A fix probably won’t results in an additional publication, and I suspect people would rather work on the “next” broken defense.
@_ericzeng
@franziroesner
@yoshi_kohno
Haven’t read the paper, but I believe many of these ads are from several large platforms? Like host Google ads on your website and Google will distribute the ads. If so, is quality of ads highly correlate with the platforms?
@antsstyle
@TheGlazeProject
The GPU memory seems to spike when too many analysis data are saved. We had to do the batch workflow, so that we can offload some resources once the analyzing stage is finished.
I will limit the max number of images for now, till I figure out a way to limit GPU memory.
@florian_tramer
Balancing power dynamic can often leads to more fruitful progress (e.g. unionization/labor strikes lead to better labor laws). I would say we started to see some similar progress in the AI case.
@TheGlazeProject
@dead_em_
I suspect it is due to a corrupted resource file. Could you try deleting the Glaze resource folder? The folder is at C:\\USERNAME\.glaze. Once you delete that, reopen Glaze and it will redownload the resources. Hopefully this fixes the problem.
@TheFutman27
@TheGlazeProject
@ZakugaMignon
Yeah we tried. Turns out pytorch (one of the Python library we use) has a bug with M1 chips. So we weren't able to leverage yM1. Hope we can address this in the future.
@midiankai
@ZakugaMignon
@TheGlazeProject
In the meantime, I suspect the issue is with your image file. Would you mind trying to glaze a few random images you find online? See whether they also have the same problem.
@biggiobattista
@florian_tramer
I agree. I am a big fan of defense papers that acknowledge existing breaks, and discuss things learned and potential directions.