Simulations reveal that ducklings swimming in a single-file formation behind the parent can achieve a wave-riding benefit whereby the wave drag turns positive. Gives a new meaning to "get your ducks in a row" 🦆
@JFluidMech
#dynamic
#equilibrium
#swimming
Heard about all them new-fangled AI NWP systems like "GraphCast", "PanguWeather", and "FourCastNet" and want to try them out yourself? Let me help you with that ->
This is just a quick preview release; see README for details. Feedback very welcome!
ai-models-for-all v0.2.0 is now up - run PanguWeather, FourCastNet, or GraphCast with GFS initial conditions the moment they hit NOMADS/GCS.
Check out the README at for more details. As always, your feedback/questions/comments are very welcome!
Congrats to the whole DeepMind team! It's been very exciting seeing the steady progress of this system, and to observe its utility over the tropical season this year.
The Weather Enterprise is staring at a very real inflection point at the intersection of weather/tech...
Presenting GraphCast: our state-of-the-art AI model delivering 10-day weather forecasts with unprecedented accuracy in under one minute. 🌦️
It can even help predict the potential paths of cyclones further into the future.
Here's how it works. 🧵
All you
@dask_dev
and
@xarray_dev
users please keep up the social media sharing! I'm enjoying catching glimpses of all the cool stuff from the community while taking care of my 2-day old! :) She'll be a dask power user soon enough!!
Is there a good manuscript/article on why AI hasn’t led to a significant improvement in weather forecast quality (broadly)? If not, do folks have any hot takes or thoughts on this topic?
Pretty amazing to see weather forecasting on the cover of Science. Even more amazing to see transformative work in this domain from folks well outside the traditional weather community!
2024 is going to be very exciting in this field. I hope you're buckled in!
Trained on four decades of historical data,
#GraphCast
from
@GoogleDeepMind
is an
#AI
model that predicts global weather with greater speed and accuracy compared with traditional approaches.
Learn more in Science's last issue of 2023:
Arxiv preprint "Neural General Circulation Models" dropping this morning from
@shoyer
and team:
NeuralGCM is, "the first fully-differentiable hybrid [GCM] of the Earth's atmosphere;" the authors discuss both weather and AMIP-like climate experiments.
@GoogleDeepMind
sneaking in one more massive delivery on
#AI4NWP
before the end of the year to solidify 2023 as the breakthrough year for this tech - "GenCast: Diffusion-based ensemble forecasting for medium-range weather"
Congrats
@rabernat
and the broader
@pangeo_data
community! Amazing write-up and thrilled to see this work reaching a much broader audience than just the geosciences!
What if I told you that _today_, for less than the cost of one cup of Dunkin Donuts coffee, you could have a 10-day AI-NWP forecast initialized from the GFS(/GDAS) within 10 minutes of the data being posted on NOMADS?
A lot folks have asked what I’ve been up to since joining with
@Waymo
last year. Check out some of my team’s work in this awesome write-up from
@verge
!
(Anywhere in US)
Dentist: What do you do for work?
Me: I’m a meteorologist
Dentist: Cool, what TV channel?
(Boulder):
Dentist: “”
Me: “”
Dentist: Oh it must be fun working at NCAR!
Me: So about that…
Weather/climate folks: data engineering is a superpower. Learn how to wield it effectively and you’ll free up so much more time for science and research that you won’t know what to do with it!
Interesting short article on
@WIRED
this morning - but I don't think it really captures the challenge, opportunity, and pitfalls here. A short thread. (1/10)
#WxTwitter
and
#python
/
#datascience
folks - I’m working on an article about the “atmospheric data science” niche and career opportunities for undergrads/graduate students. Please reach out if you have thoughts or would like to contribute to the article!
Success! Integrated
@PrefectIO
with my team’s
@pangeo_data
/
@ProjectJupyter
cluster, which uses the
@dask_dev
dank-gateway to dynamically spin-up resources for jobs. Very excited to start automating more jobs with this stack!
Best part of
#AMS2021
? My Twitter feed is overloaded with meteorology, innovation, and science topics, so it's impossible to doom-scroll current events!
New
@NOAA
GFS GRIB kerchunking pipeline triggering on
@modal_labs
with the Aug18 00Z run this hour... will generate references for pseudo-cloud-native access to GFS data for all my forecast visualization/analysis pipelines. Fully serverless weather data processing, let's go!
Are you using AI to build end-to-end weather forecasting systems? If so, check out the session topic "Pure AI and Data-Driven Weather Forecasts" in the AI Conference at the
@ametsoc
' Annual Meeting and consider submitting an abstract!
Friday was my last day w/
@tomorrowio_
serving as Chief Scientist... very excited to share the exciting next steps in my career in technology/weather in the coming weeks!
Omg I just polished an insanely cool research result that I should squeeze into our AMS presentation in two weeks… really hope I can get review/approval, this is a game-changer for hyper-local weather / urban micro-climate characterization!
Something cool on this holiday weekend - 4DVar applied to the classic Lorenz (1963) “butterfly”, using JAX. Hope to have a blog post up soon with reference code and a write-up on how reverse-mode differentiation through ODE solvers could open a new world of DA applications
Some days I feel like I’m just ranting about the importance of “subject matter expertise” into the void. Is this exclusively a weather/climate thing, or do other technical scientific domains have similar issues?
I'm reviewing some Fortran on the train home and some techy hacker dude writing what looks like a webapp keeps gawking at my screen. Yes, kid, I'm debugging code that was written before you were born!
To say that the release of
@GoogleDeepMind
's GraphCast this week has shaken the weather enterprise to its core would be an understatement. My LinkedIn is inundated with weather companies claiming they've been "doing AI/ML for years". And while true... it's not the same thing!
Friendly reminder for
@ametsoc
attendees at
#AMS2024
looking to play with
#AINWP
models... I've got you covered! Check out
Happy to sit down and demo running
#GraphCast
for you 🙂
Excellent, the ensemble Kalman Filter I wrote in JAX seems to work (perfect obs experiment). Can’t wait to share the rabbit I’m going to pull out of this hat… fun holidays R&D project :)
This is a super important contribution to the broader weather community and will drive faster and more effective innovation in the nascent "pure AI NWP" space. Congratulations
@raspstephan
and team!
The weather forecast is improving… literally! Introducing WeatherBench 2, a benchmark for the next generation of data-driven, global weather forecast models, providing data, tools, & an evaluation platform. Learn how to use it and check out the website →
For those of you who have had trouble reaching me the past Q, this is why... a huge accomplishment for the team here at
@Waymo
! I can't wait to tell the story of how weather plays into this significant milestone for the company :)
We’re taking the next step in our journey—both in San Francisco and in Phoenix. Last week, our employees began taking fully autonomous rides in the City by the Bay, and soon we’ll be expanding our Waymo One Trusted Tester program into Downtown Phoenix.
Wow... Stumbled across
@blaylockbk
's Herbie package and saved myself a morning of hacking some one-off GRIB processing tools. Thanks for this fantastic toolkit!
Gobsmacked at how easy it was to integrate
@Superhuman
into my workflow. First time I’ve hit Inbox Zero and have everything properly captured in
@todoist
in months!
What a morning. GraphCast in press at Science, NeuralGCM preprint on arxiv, Fifth National Climate Assessment released to the public... there's only so much coffee in my house!
Thanks
@DrShepherd2013
for the shout out to
@climacell
on
@weatherbrains
this week! We agree about the huge opportunity for students to become "atmospheric data scientists"... Meteorologists who can build software and extract value from weather data. I'd love to help train them!
Super excited to share a project I worked on during my week off... started with messing around with Neural ODEs with Jax, and evolved into a super efficient toy 4D-VAR framework.
Will write some blogposts this week on the results - need to hack a Lorenz96 DA example first
Amazing to see 19 years of GEFSv12 reforecast data now available on S3, but super depressing and unfortunate to see it uploaded as GRIB format. The first group to cloud optimize this data is going to find amazing science applications for the data, I hope they share!
My session topic at the 2022
@ametsoc
Annual Meeting, “Towards Operational Precipitation and Convective Weather Nowcasting Leveraging Deep Learning” was accepted!
Please consider applying! CC
@DJGagneDos
@ralager_Wx
@profamymcgovern
I sincerely hope the
@ametsoc
reconsiders its policy and chooses to enforce a vaccine mandate for those who wish to attend the 2022 Annual Meeting in person. I’ll need to re-evaluate my plans to travel this year (been to every mtg in-person since 2010).
If you plan on joining us in Houston, we want you to know that your health and safety is of utmost importance to us. View our Commitment to Care Statement to learn more:
#AMS2022
So… VS Code continues to be the most amazing productivity booster imaginable. I mean, hovertips with PR details from orthogonal repos while you’re working in a different context?!?!
If you go to four different talks on "digital twins" you'll hear eight different definitions of what the heck they actually are.
Still some very fascinating work coming from the community in this space though!
Thrilled to participate in
#AMS2022
this week - if you’re at
@AMSEarlyCareer
/
@AMSStudentConf
and interested in careers at the intersection of data, software, tech and weather/climate, hmu - I’d be happy to talk and help you find some awesome and unique opportunities!
If you do any technical/coding work in your research or job, try to keep a few projects on GitHub. Even better - if you use a tool like
@xarray_dev
or
@matplotlib
, consider contributing to these libraries! You never know when a hiring manager will stumble across your work…
Happy Wednesday! We can hardly believe that it is almost May - and time for new grads to be hunting for jobs!
For
#ShareYourStory
this week, let's share some job hunting and interview tips for our new grads, and anyone lese looking for a job!
Might share a 2023 "Year in Review" post later but for now, it's all hands on deck for 2024 and the breakthrough year of AI for Weather and Climate!
If you're working in this area - we should chat!
The ongoing influx of smoke / biomass burning aerosol over CONUS is an incredible natural experiment in aerosol impacts on clouds and convection in a continental mid-latitude climate. Is anyone already looking at these potential impacts?
To better simulate the in-person
#AMS2021
experience I am going to commit to listening to all my talks standing up, just like I usually do standing in the back of the breakout rooms :)
So ai-models-for-all is easily the most well-received code/project I've ever released. Thanks for all the encouraging comments the past few days!
Two big targets for next steps are a post-processing workflow (auto-gen a forecast maps webpage!) and initializing with GDAS.
Folks - put your utility code on GitHub! A throw-away script I wrote like 8 years ago for a random PhD side-project just saved weeks of work for a colleague!
@LyssaFreese
MetPy has a few options for simple gridding ()
Verde () is a bit more complex but has more sophisticated methods. You may need to write your own wrappers to output xarray structs but should be straightforward
Just realized I’ve been an “atmospheric data scientist” for more than half my life now… started as a 16-year old volunteering at the NWS in Louisville and working on a hydrology project with a local software company. Cheers for another 16 years (at the minimum)!!
Episode 889 is LIVE tonight at 8pm ET/7pm CT! The folks from Waymo will join the show! They will talk about how and why they’re essentially employing their fleet of autonomously driven vehicles as mobile weather stations and what it enables! Watch live:
Been cranking through a bunch of AI/weather projects all weekend for fun, and it's amazing how simple this work is compared to just a few years ago. Four years ago it was a whole project just to set up the data and infrastructure for even a toy problem!
Hey Python fans! Enjoying
@ametsoc
#ams2021
? Our sessions start tomorrow! Please in sure you have watched the video files to get the best experience from the live sessions
Nice paper by Selz and Craig in GRL regarding how well Pure AI NWP systems handle perturbation error growth - . This will be an important issue as we inevitably move to very large ensembles built around these tools.
I actually got to do real science today. Was a lot of fun! I should probably do more of this. It would be fun to go back to being a full-time researcher...
Congratulations
@al_merose
and the rest of the team
@Google
for this significant contribution to the weather/climate community! Great example of building ready-to-use ARCO data for climate analysis!
@ametsoc
I applaud this decision by the
@ametsoc
and am excited to work with other session chairs and conference conveners to ensure an amazing virtual conference experience!
“The Staff Engineer’s Path” is really helping me understand how to be more effective in my current role. Academics considering a jump to industry should check it out - if you’re a technical problem solver, this could be the career path for you!
In case it wasn’t obvious, my visualization is heavily inspired by the amazing work of
@TropicalTidbits
,
@burgwx
,
@JackSillin
, and so many other world-class meteorologists who create and share these types of graphics to communicate our science to the broader public.
Thrilled that the parcel model I developed during my PhD is still alive, kicking and useful for researchers and educators in the field!
A
@JuliaLanguage
version should be available later this year.
A new
#BulletinAMS
paper introduces interactive worksheets for
#teaching
about atmospheric aerosol and cloud physics, and describes the set of programming tools used to create and deliver content. Learn more at .
Huge shout out to .
@ametsoc
team working diligently to kick off the
#AMS2021
meeting... We appreciate all your hard work so much!
I forecast smooth sailing from here on out :)
Today marks a historic moment for our company.
In addition to our series D funding, we are also changing the name of our company. Say hello to
@Shimonelkabetz2
explains it all:
Anyone want to join in on proposing a topic for the
#AMS100
meeting? I’m thinking about ideas ranging from “climate data science” to big data challenges/opportunities or even partnerships/engagement between the weather and tech worlds
I’m re-writing pyrcel () in Julia to better serve fundamental research into physical/chemical constraints on droplet nucleation and microphysics. If you would like to collaborate please reach out!
Down to the very final "to-do" items on my task list for work before moving on to new things for 2024!
It's going to be a _very_ exciting January. Stay tuned!
Sharing as a “sneak peak” of side project I’ve been working on since last last year… building out a cloud-based data system leveraging
@PrefectIO
for analysis and visualization of forecast model data.
@arianaelena97
Oh… and I’m happy to chat with anyone in the geosciences contemplating jumps to industry, especially into tech! There’s **so** much opportunity right now, it’s unbelievable… and you as a geoscientist are in **high demand**. 9/8
Wow… is an incredibly powerful tool for geospatial visualization, assuming you’ve prepared your data correctly. Definitely my bad to have not played with it before now!
Wife was complaining that baby was having a rave, so of course I asked Alexa to play Sandstorm. Might need to sleep in the car tonight but it was worth it.
Interesting perspective article in e360 about the potential climate impacts of microplastics:
Particularly intrigued by Zamin Kanji's reported lab results (are these published?)...
@obSarveshon
and
@DanCziczo
maybe we should reboot our CESM flyash work?