Professor for CS at the Tuebingen AI Center
@uni_tue
and affiliated Professor at MIT-IBM Watson AI lab
@MITIBMLab
- Multimodal learning and video understanding
Now that the
@NeurIPSConf
rebuttal is over, I finally have time to announce that in the middle of everything, I had the pleasure of starting my new position as a professor at the Tuebingen AI Center at the
@uni_tue
✌️🥳🎉!
Happy to finally share our paper about differentiable Top-K Learning by Sorting that didn’t make it to
#CVPR2022
, but was accepted for
#ICML2022
! We show that you can improve classification by actually considering top-1 + runner-ups… 1/6🧵
#ComputerVision
#AI
#MachineLearning
Ever asked yourself what's the best explainability method for ViT Transformer at the moment?
🚨 We present you LeGrad, a Layerwise Explainability GRADient method for large ViT transformer architectures. 🚨
So, grab your ☕️ and 🥐, release your inner 🇫🇷, and join us on our
Hi! I have a phd position in video understanding to fill! If you are interested in research, have a MSc in CS,
#ML
etc. and ideally already a bit of publication experience in
#cv
/ ML, reach out!
Also if you think about applying to
#ellis
and need a backup!
SVD 1.1 seems to have been released 👀
HF:
For those who can't wait for the official announcement, you can try it on this colab for free.
Colab:
1/3
#SVD
🚨 New Paper Alert! 🚨 *Mask Inversion*
What it does? -> It learns a token representation for a specific region in an image (e.g. a token for a mask, a BBox etc.)
project page:
paper:
code:
On my way to
#ICCV2023
! Looking forward to an exciting week packed with workshops, talks, and posters. Here's where you will find me on the first day:
We are organizing two workshops on Oct 2 (morning):
* What is Next in Multimodal Foundation Models? - P01 - 8:30-12:30 ->
So proud to be awarded the Mark Everingham Prize at
#ECCV2022
for our work on HMDB51, but even happier to share this prize with my dear colleagues Mubarak Shah and
@wjscheirer
Big kudos to my collaborators Hueihan &
@tserre
🙌 Couldn't have done it without you!
Let’s face it… Just because you don’t know what’s in your training data, you can not just call it zero-shot 🤷♀️
1) We just relabeled the old concept of zero-shot learning from attributes to “I have not checked my training data “ resp. “ I have not seen the dataset” (and even
So much for emergent magic.
If VCs understood the significance of this paper, they would make radically different choices. It really is driverless cars all over again.
We welcome submissions to our Multimodal Foundation Models and their applications Workshop (MMFM2) at
@CVPR
2024.
We have archival and non-archival tracks (including works accepted to CVPR without modification to the paper).
We expect a large attendance, and it’s a great
PAMI Helmholtz Prize in
@ICCV2021
to papers from
#ICCV2011
with significant impact on computer vision research
** ORB: An efficient alternative to SIFT or SURF
and
** HMDB: A large video database for human motion recognition
and
** DTAM: Dense tracking and mapping in real-time
Working on Multimodal foundation models (MMFMs)!
Busy with NeurIPS? ... and ... want to go to
@CVPR
?
Check out our
@CVPR
MMFM Challenge on understanding multimodal data etc:
Have some VL/MM model around? Give it a try!
@MIT_CSAIL
@berkeley_ai
Andrew Zissermann is giving a talk at the BMVA symposium on vision language about how to generate textual descriptions for movies. Great talk and great line of work to make movies accessible for everyone. Check out the papers:
📽️
Just as everyone is waiting for
#iccv2023
#decisions
, happy to announce the
Open Fine-grained Activity Detection Challenge (OpenFAD)
in conjunction with the
2nd Workshop on Fine Grained Activity Detection
@ICCVConference
Happy to announce that I will have open PhD slots for the IMPRS-IS program
@uni_tue
!
You can apply here:
Deadline is Nov 15th.
Ask me
@eccvconf
for any details!
Call for Women in Computer Vision
#WiCV
workshop organizer at
#ICCV2023
If you want to be an organizer for the
#WiCV
workshop at ICCV 2023, please fill out the form by Jan 20, 2023:
Please RT and share. We look forward to your application.
#CVPR2023
Great first talk at the Anticipating Human Behavior Workshop by Jürgen Gall happening today
@UniBonn
!
Looking forward to more talks today i.a. By
@SvenBehnke
and
@dimadamen
Happy to share the video to our
#ICML2022
paper on top-K classification learning via sorting loss! Check it out 👇
Paper:
Spotlight: Wed 1:25-1:30 p.m. EDT @ Ballroom 1 & 2
Poster: Wed 6:30-8:30 p.m. EDT @ Hall E
#328
@MITIBMLab
Currently happening at the poster session in Foyer Sud. Drop by our posters 4 + 5 … and yes, we have two posters right next to each other, and no, we didn’t plan for that 😁😁
@ICCVConference
#ICCV2023
@ninashv__
@anna_kukleva_
Info especially for the junior folks in the
#computervision
community 👉 Please subscribe to the PAMI TC newsletter:
It’s a monthly email with all relevant information about conferences and calls. NO SPAM! Just everything important in one place from the
Just some random theory I wanted to share for a while ... the "ACADEMIC IKEA" effect:
Idea: The more your supervisor is directly involved in your work, the more he/she will like it.
Just citing this to celebrate … two papers accepted to
#cvpr2022
🎉 !
First glimpse on how to get multimodal fusion transformer to work:
Will post more details after
#eccv2022
deadline 👻
Just because I'm back to academia ... Could we have a flag for ML conferences which says "Industry resources used? yes/no" to convey why we did not train 3 months on 1024 GPUs to search for good parameters for this "one" model? Also, not every company has FAAGN scale resources.
A bunch of people have requested the slides for my "Scholars & Big Models" CVPR workshop talk. I didn't have a script, but I wrote a rough version of what I probably said at the bottom of each slide. Feedback is welcome!
Happy to share our
#NeurIPS2022
paper on differentiable logic gates!
Want to find out how to run Neural Network only on binary decisions 0/1? How to make logic gate networks differentiable?
Here is a :thread:1/4
paper:
#computervision
#machinelearning
#ai
#ECCV2022
paper stats: 2 accepted, 2 rejected, 1 withdrawn. Check it out👇!
@AishaUroojKhan
: VQA Grounding in Vision Language Transformers
@WeiLinCV
: Domain Adaptation from Image to Video:
Big congrats to both first authors!
We could make this a new metric for faculty hiring… how many papers (absolute or ratio, whatever you want) of your scholar stack have a female first author that’s not you 👩 … ?
no better way to celebrate women’s day than submitting a paper with a female student as the first author. I’m sorry but it’s a big deal, it doesn’t happen very often. in fact, the first in 4.5 years! let’s hope, first of many 🤞🏽
I thought this was funny first… now I’m thinking about writing a thread about how research papers (which should first be out there to communicate new findings and to share knowledge with your community) being used as a pseudo metric for various performance measures where your
The NeurIPS high school track has made its way to Chinese social media.
The collaborator who sent me this said there is at least one example of a professor asking their PhD student to help write a paper for one of their kids so that it can help their college admissions abroad.
Shout out to
@MattNiessner
for tapping the barrel in one hit at the GCPR dinner tonight!
@gcpr_by_dagm
Thanks to everyone for being there! It was a great evening! 🍻🍗🥨
I'm not sure if this is the best possible outcome. I thought a lot about if I should write about this or not, but I hope there is some value in thinking about this a bit more. A longer 🧵 ...
If you made it till here, I'm also looking for some great PhDs and PostDocs 👀 to work on everything from data generation to large-scale multimodal retrieval.
If you have a solid Top-A track record (PostDoc) or just some first paper(s) and want to do more (PhD), please ping me
To my fellow
#CVPR
#CVPR2023
AC (after finishing my paper assignments): Please do NOT just click the first 10 ones with highest affinity score. Too much noise in the system. We all know how important communities are, so match the right papers with the right reviewers. Please rt!
@PhD_Genie
Once gave a talk in the lab of a close collaborator. Head of the group introduced me to the folks with “… you can talk with her about everything. I 100% trust her.“ … biggest compliment I ever got in academia.
Going to
@ICCVConference
? Drop by at our posters!
I'll be around at all of them!
#ICCV2023
MAXI: Unsupervised Finetuning for Zero-Shot Action Recognition with Language Knowledge
Paper:
Code:
Wed 4th, 10:30-12:30, Foyer Sud – ID
Hi
@NeurIPSConf
! Just to get this out off my system ... would there be any way to replace those checklists at the end of the paper with questions that you have to click and fill during submission? Just the same information in a more organized way ...
Very glad and honored to join as a mentor to discuss aspects of industry vs. academia and how to manage your supervisor 😇 ... sign up and let's chat!
#CVPR2022
I stand with my point to keep the social media ban for CVPR!
#CVPR2023
Having a paper on arxiv is a different story than having a full blown social media campaign as we have seen before.
DEADLINE EXTENDED for the Women in Computer Vision (WiCV) workshop @
#CVPR2022
.
Updated submission deadline: March 10, 2022 11:59 pm PT
More details here:
@CVPR
Please RT/amplify
Did you know that a pretrained VL foundation model is all you need for open vocabulary localization and segmentation ?
❗️No training needed❗️Just get your favorite model and localize everything! 💎
Pro-tip: try meta-clip
All you need to know is below 👇
Grounding
@MushtaqBilalPhD
Equally can feel like discrimination for those who where profiting from the inequality … But shoutout for the young woman who had the courage to stand up against a full panel at such an event. nsf or eu, give that woman a grant!
Why the
#EUAIAct
might be bad for
#AI
#Research
in the
#EU
and why we should care. A long 🧵 1/15
Disclaimer:
*I’m not a legal expert, but I work with
#AI
models
*I’m happy to discuss if you have some insights that I have missed
*I am part of
#LAION
*All opinions are my own
🚨
@CVPR
Challenge alert!🚨
Think you have a good MML for documents, infographics and text-rich images?
Challenge “What is Next in Multimodal Foundation Models?” extended the deadline and opened for all:
$10K in prizes
Deadline: 5th June 2024 (11 days)!
Hi everybody! As the new year is finally here 🎊, what was your coolest paper (with link if you want) of 2022?
And what do you think will be the most exciting topics in CV in 2023? 🌅
#ml
#computervision
Big congrats 🥳🥳 to
@mamatathota
for passing her PhD! And to
@geleonti
for graduating such a great PhD!
It was an honor and a pleasure to be part of this committee!
Delighted that
@mamatathota
passed her PhD viva today with minor corrections! Mamatha did a great PhD that focused on (continual & un/self-supervised) domain adaptation - Mamatha is the third PhD student of mine graduating this year & last one from my days at Lincoln University
Attending
#CVPR2024
? Consider attending the PAMI TC meeting (4pm Thurs. Summit Flex Hall ABC) where we'll be discussing important potential changes to the review process and the location of
#ICCV2029
. The TC meeting gives the community a voice on conference operations.
Interested in organizing a tutorial related to
#ComputerVision
? We are welcoming proposals for tutorials at the 2022 Asian Conference on Computer Vision! Details under
Join us by submitting your proposal until June 8th!
#ACCV2022
#CVPR
#ICCV
#ECCV
Call for Women in Computer Vision
#WiCV
workshop organizer at
#ECCV2024
If you want to be an organizer for the
#WiCV
workshop at ECCV 2024, please fill out the form by Feb 04, 2024:
Please RT and share. We look forward to your application.
Aloha ... 🐢
PCs at work to finalize the last details for
#WACV2024
CfP. Stay tuned! Hope to bring you a great conference in wonderful
#Hawaii
🏝️🌊🏝️ !
@MajiSubhransu
@CSProfKGD
and Vlad Morariu
Excited to introduce Whisper-Flamingo! Check out the demo video; Whisper-Flamingo can transcribe and translate speech with heavy background noise!
Paper:
Code, Pre-trained models, Colab:
Just to chime in here a bit why this might not be correct ...
1) We have reason to assume that web scrap data includes samples from or very similar to the used dataset, including INet () ... you can not do better than training on test data.
This is it! With
@NeurIPSConf
D&B, the last deadline for a major conference in 2024 is done! Congrats to everybody who gave it a try!🥳
... and now up to the 500 add-on evaluations promised for the supplementary! 🤣
Happy to represent academic AI research at this year’s reception for the 2024 annual report of the Commission of Experts for Research and Innovation
In german:
@EFI_Kommission
@UniBonn
@Michael_J_Black
TLDR: As long as we rely on voluntary reviewing, we can not enforce bans for paper submissions for bad reviewers.
Long version: I’m not sure if the reviewer system can be steered by negative extrinsic feedback. The natural reaction to this is that less people sign up for review
@taiyasaki
There was no big motivation to cheat bc there was not the same level of funding as today and no paper would get you anywhere near a 6-figure job.
Things changed when papers were used as surrogate metric for big hiring, and a few years we’re still not ready for this DDOS attack.
I think it might be worth discussing of a system between
@CVPR
and
@ICCVConference
or
@eccvconf
to allow to pass on previous reviews for the submission, but 100% optional.
This is not the same as a rolling submission, but even implementing this will take time as conferences