Baharan Mirzasoleiman @baharanm Twitter profile

Pinned Tweet

Baharan Mirzasoleiman

1 month

We’re thrilled by the amazing response to our #ICML2024 tutorial on “Foundations of data-efficient learning”! Over 1000 attendees joined us. Thank you all! 🙌🌱🌱🌱 ➡️ Slides: ➡️ Recording: will be available on Aug 22 🎊🎊

Baharan Mirzasoleiman

@baharanm

2 months

I'll be giving a 2-hour tutorial on data-efficient learning with my PhD student @sjoshi804 on Monday July 22 at #ICML2024 . Join us to learn more about this cool topic! ➡️ We can learn better from better data! ⬅️🙌🌱

6

24

281

6

18

212

Last Seen Profiles

@rorymcconnell

@bigcat_mia

@HoracekFilipe

@Inception_org

@RonKrahn

@rt_findompl

@CarlosAugustoMX

@Rin_ha_Rinri

@catterall_pp

@rachel_alexgo

@Nasty_CSA

@xyz_fucker

@P365sportsRDs

@softlyforhoon

@RouletteFantasy

@InkwellTattoos2

@MxEnImagenes

@Qzensixu

@dopegirllk

@DavidLien779201

@cicraa

@onedanilox

@raenyii

@derbynews1

@AiAyame51

@QuadrupleJ_vrs

@TheLilShorty_

@bokeplokalmalam

@KosalEng

@pasurtihot

@kaki_kambing

@kawiwiii

@YasarGK060

@Taanya_K

@kabo_lab

@jbarrez

Baharan Mirzasoleiman

@baharanm

2 months

I'll be giving a 2-hour tutorial on data-efficient learning with my PhD student @sjoshi804 on Monday July 22 at #ICML2024 . Join us to learn more about this cool topic! ➡️ We can learn better from better data! ⬅️🙌🌱

6

24

281

Baharan Mirzasoleiman

@baharanm

2 months

ML models are sensitive to distribution shift. Can we adapt a model with only a few examples from the target domain? In this #ICML2024 paper, @xue_yihao65785 proposes an effective way, with nice theoretical analysis🌱 🔗 Thu, July 25, Poster session 5, #800

7

22

120

Baharan Mirzasoleiman

@baharanm

1 year

One of our recent work that I’m most excited about: speed up your large (contrastive) pretraining by 2.5x without compromising accuracy! This can be done (rigorously) by pretraining on the “right” data points! Shout out to @sjoshi804 for his amazing work accepted to #ICML2023 !

Siddharth Joshi

@sjoshi804

1 year

Our paper ( @baharanm ) Data-Efficient Contrastive Learning has been accepted to #ICML2023 ! We propose the first method to select the most useful data subset to train on and provide rigorous theoretical generalization guarantees. 🧵 (1/5)

8

23

119

0

8

83

Baharan Mirzasoleiman

@baharanm

5 months

My PhD student @xue_yihao65785 has received one of the 50 @OpenAI Superalignment Fast Grants (out of 2700 applications)! Big congrats Yihao and looking forward to seeing more amazing work from you! 🎉🎉🌱🌱

Yihao Xue

@xue_yihao65785

5 months

Huge thanks to @OpenAI for the superalignment fast grants! Can't put into words how happy I am. Looking forward to diving into a seriously exciting project!

5

2

113

1

68

Baharan Mirzasoleiman

@baharanm

2 months

CLIP is highly sensitive to data poisoning and backdoor attacks. In this #ICML2024 paper, @WenhanYang0315 proposed an interesting way to pretrain CLIP robust to such attacks without compromising the performance! 🌱🌱 🔗 Thu, July 25, Poster session 6, #814

6

8

48

Baharan Mirzasoleiman

@baharanm

4 months

Why CLIP is more robust to distribution shift than supervised learning? This #ICLR2024 paper provides the first rigorous proof! TL;DR details specified in the captions allow learning more generalizable features from images. Check it out: Tue, PS #1 , P #113

Yihao Xue

@xue_yihao65785

6 months

Happy to share that I have two papers ( , ) accepted at #ICLR2024 ! ⬇️🧵 1/

6

12

82

1

6

44

Baharan Mirzasoleiman

@baharanm

2 months

Double descent confirms the benefit of larger models. But, when there is label noise in the data, larger model size can hurt the performance! We called this phenomenon "Final Ascent". Check out this interesting #UAI2024 spotlight by @xue_yihao65785 : 🙌🌱

0

5

41

Baharan Mirzasoleiman

@baharanm

1 year

Can we speedup deep learning 2.5x via data-selection with theoretical guarantees? Our #ICM2023 paper shows for the first time that this is indeed possible! Data points of increasing learning difficulty is what you need! An excellent work well worth reading ( @YUYANG_UCLA ,Hao)🎉🌱

Yu Yang

@YUYANG_UCLA

1 year

📢Our (Hao Kang, @baharanm ) paper () will appear at #ICML2023 ! Introducing CREST: the first coreset selection algorithm theoretically guaranteed to speed up training of deep neural networks!🚀(🧵1/7)

8

13

77

0

2

39

Baharan Mirzasoleiman

@baharanm

1 year

Neural networks can make right predictions for wrong reasons! Despite the body of efforts on mitigating spurious correlations,this problem is far from being solved on real-world datasets. We introduced a benchmark and two datasets to help you evaluate and develop better methods🎉

Siddharth Joshi

@sjoshi804

1 year

Introducing SpuCo: a Python package to standardize tackling spurious correlations in deep neural networks! 1️⃣ Modular implementation of current SOTA methods 2️⃣ Two new challenging and realistic datasets Paper: Github: 🧵(1/n)

3

4

47

1

37

Baharan Mirzasoleiman

@baharanm

1 year

Despite being more robust, large-scale contrastive vision-language pretraining can be still affected by spurious correlations. Check out our latest #ICML2023 paper to see how language can be leveraged to mitigate them during fine tuning. @YUYANG_UCLA @besanushi @hmd_palangi 🌱🎉

Yu Yang

@YUYANG_UCLA

1 year

🎙️We are thrilled to announce that we will be presenting our latest paper with @besanushi @hmd_palangi @baharanm () at #ICML2023 ! 🎉 Join us as we share insights and solutions for ✨spurious correlations in vision-language models✨. (🧵1/8)

10

5

94

0

34

Baharan Mirzasoleiman

@baharanm

4 months

Dataset distillation methods can only distill the early training dynamics. We showed in this #ICLR2024 paper that generating multiple synthetic data to capture different training stages improves the performance! Tue PS #2 Halle B #9 @YUYANG_UCLA @xxchenxx_ut

0

5

28

Baharan Mirzasoleiman

@baharanm

9 months

CLIP has enabled zero-shot classification. But pre-training CLIP is highly sensitive to data poisoning and backdoor attacks. RoCLIP protects your CLIP pre-training against up to 1% poison rate. Check out this amazing work by @WenhanYang0315 tomorrow at #NeurIPS23 🌱🎉🙌

WENHAN YANG

@WenhanYang0315

9 months

CLIP is such a great model for zero-shot inference…until it is poisoned! 0.0001% poison /0.01% backdoor rates are enough to compromise the model! Our work on multimodal robustness proposed RoCLIP to tackle this issue!

1

3

21

0

4

26

Baharan Mirzasoleiman

@baharanm

4 months

Why does projection head benefits representation learning? It allows learning a broader range of features, as we rigorously proved in this #ICLR2024 paper! @xue_yihao65785 also proposed an alternative for the project head! 🙌🎉🌱 Thu, PS #5 , Hall B #119

Yihao Xue

@xue_yihao65785

6 months

Happy to share that I have two papers ( , ) accepted at #ICLR2024 ! ⬇️🧵 1/

6

12

82

1

4

25

Baharan Mirzasoleiman

@baharanm

1 year

If you’re attending ICML, check out our papers: 1- Data-Efficient Contrastive Self-supervised Learning: Most Beneficial Examples for Supervised Learning Contribute the Least (Tue, 2-3:30pm, EH 1 #214 ) @sjoshi804

1

0

22

Baharan Mirzasoleiman

@baharanm

1 year

Do you know which features are learned by contrastive learning and what are its failure modes? Check out this beautiful rigorous results to learn how simplicity bias of SGD changes the picture! Congrats @xue_yihao65785 , @sjoshi804 ,Eric Gan, @pinyuchenTW on the #ICML2023 Oral!🎉🎉

Yihao Xue

@xue_yihao65785

1 year

We are excited to announce that our paper ( @sjoshi804 , Eric Gan, @pinyuchenTW , @baharanm ) "Which Features are Learned by Contrastive Learning? On the Role of Simplicity Bias in Class Collapse and Feature Suppression" has been accepted for oral presentation at ICML 2023!⬇️🧵 1/

2

4

32

0

3

20

Baharan Mirzasoleiman

@baharanm

4 months

Simplicity Bias (SB) makes deep models learn spurious correlations. But SB can be also used to eliminate them! Check out this nice #AISTATS2024 paper with @YUYANG_UCLA where this is rigorously proved and used to achieve SOTA worst-group acc: (Sat. P #67 ) 🌱

0

1

19

Baharan Mirzasoleiman

@baharanm

2 months

I’ll also present “SafeClip” on behalf of @WenhanYang0315 tomorrow at 1:30pm (poster session 6) #814 . See you there! 🙌

Baharan Mirzasoleiman

@baharanm

2 months

CLIP is highly sensitive to data poisoning and backdoor attacks. In this #ICML2024 paper, @WenhanYang0315 proposed an interesting way to pretrain CLIP robust to such attacks without compromising the performance! 🌱🌱 🔗 Thu, July 25, Poster session 6, #814

6

8

48

0

2

18

Baharan Mirzasoleiman

@baharanm

2 months

Graph Contrastive Learning (GCL) has shown a great promise in learning node representations. But, under heterophily, existing GCL methods fail. Check out this nice #UAI2024 paper by @WenhanYang0315 that addresses this problem using graph filters! 🙌🌱

0

2

17

Baharan Mirzasoleiman

@baharanm

1 year

@xue_yihao65785 @sjoshi804 @pinyuchenTW 3- Mitigating Spurious Correlations in Multi-modal Models during Fine-tuning (Thu, 10:30am-12pm EH 1 #106 ) @YUYANG_UCLA @besanushi @hmd_palangi 4-Towards Sustainable Learning: Coresets for Data-efficient Deep Learning (Thu, 1:30-3pm EH 1 #304 ) @YUYANG_UCLA Hao Kang

0

15

Baharan Mirzasoleiman

@baharanm

2 months

I’ll present “MixPro” on behalf of @xue_yihao65785 tomorrow at 11:30 (poster session 5) poster #800 . Come check it out 🙌

Baharan Mirzasoleiman

@baharanm

2 months

ML models are sensitive to distribution shift. Can we adapt a model with only a few examples from the target domain? In this #ICML2024 paper, @xue_yihao65785 proposes an effective way, with nice theoretical analysis🌱 🔗 Thu, July 25, Poster session 5, #800

7

22

120

0

1

16

Baharan Mirzasoleiman

@baharanm

4 months

Is CLIP data hungry? We rigorously showed that one can discard a good portion of CLIP’s massive (pre-)training data without harming its performance! Check out this awesome #AISTATS2024 paper with @sjoshi804 (Friday, poster #140 @ session 2) 🎉🎉🌱🌱 Paper:

Siddharth Joshi

@sjoshi804

4 months

Proud to be presenting Data-Efficient CLIP at AISTATS 2024! We propose the first theoretically rigorous method to select the most useful data subset to train CLIP models. On CC3M and CC12M, our subsets are upto 2.5x better than the next best baseline! 🧵👇

2

14

61

0

1

15

Baharan Mirzasoleiman

@baharanm

1 year

2-Which Features are Learned by Contrastive Learning? On the Role of Simplicity Bias in Class Collapse and Feature Suppression (Oral Tue 5:45-5:55pm Room 316 A3, Poster Thu 10:30am-12pm EH 1 #218 ) @xue_yihao65785 @sjoshi804 Eric Gan @pinyuchenTW

1

2

14

Baharan Mirzasoleiman

@baharanm

1 year

What I resonate with the most in the leaked Google doc is not the power of open source or better data (we knew these). It’s to (finally) see the following:we don’t need the largest model & largest data, we need to be able to iterate. Largest prevents this!

0

1

7

Baharan Mirzasoleiman

@baharanm

1 year

A great team effort with @sjoshi804 @YUYANG_UCLA @xue_yihao65785 @WenhanYang0315 🎉🌱

1

0

5

Baharan Mirzasoleiman

@baharanm

2 months

We’ll discuss recent theoretically-rigorous approaches for data-efficient learning, and explain why and when different heuristics work!

0

3

Baharan Mirzasoleiman

@baharanm

2 months

We’ll cover data-efficient learning for supervised learning, self-supervised contrastive learning, and for training foundation models such as Contrastive Language-Image Pretraining (CLIP) and Large Language Models (LLMs).

1

0

2

Baharan Mirzasoleiman

@baharanm

2 months

@rosikand @sjoshi804 It will be recorded and we’ll post the slides after the tutorial as well!

1

0

2

Baharan Mirzasoleiman

@baharanm