Waiting to Exascale: Prepping the EURO Model for the Next Generation in Computing Power
I need more power computing power. That is this has been a major focus for numerical weather prediction in hopes of creating more accurate and detailed prediction of systems. The highly touted European model has long been one of the leaders in numerical weather prediction performance as the European Centre for medium range. Weather forecasts prepares to integrate the next generation of supercomputers model efficiency may suffer as more data is computed at higher resolutions current processing capabilities and codes are not adapted to meet these standards. Today's guest is Dr Peter. Bauer and he'll outline how they're working with meteorological models computer scientists and hardware providers to make sure the euro model is ready for the upgrade. Peter Thank you for joining us on the weather. Geeks podcast. Thanks very much for having me. Muscial yeah this is really one of the great things about doing this podcast. I often get to reconnect with former colleagues and Peter. Bauer and I actually overlap for some time. I can't I believe it was the late nineties at NASA Goddard Space Flight Center scientists at Goddard and I believe was visiting from Europe. Let me just give some of your credentials. Peter Before we get into the discussion. Peter is the deputy director of the research of the Research Department and head of the scale ability program at SEM Wfan. You're GonNa hear that acronym quite a bit. That's for the European Centre for medium range weather forecasts for those of you that use the euro model. It's their model. He's held several other positions including guest scientists at NASA and he was also received his PhD from University of Hamburg in the nineties. Peter one of the things I like to do before we get into the meat of the conversation. If you will that asked my guess how did you get into your field of meteorology or whether modeling those at something as a kid or something later in life? I I guess it was a more general Inclination to do something with the signs to start with And I did not start immediately to study atmospheric physics metrology I started with Geo Physics But I learned the basic foundations of similar start with learning mathematical methods You learn the basics of physics which are very generic cross geosciences anyway And then only like two years into my studies. I actually moved over to atmospheric signs and that was because I really liked that. You could see what you're doing. You know you just looked at the sky you could see flow processes you can see the effects of dynamics and everything and and that that fascinated me because it was different to kind of earthquakes volcanoes. Que Nos but kind of everything that happens below the crust. You can't really see The atmosphere was more visual and And that fascinated me and I stuck with ever since I have to say a similar story geophysics because when I went off to Florida State University. Initially that was my interest as well geophysics but then ended up in the Meteorology Department but certainly as we both know our fields in this whole notion of earth system science or quite related. I want to now pivot because you work with the premier modeling group in in the world. Frankly ACM Wfan. Everyone knows about the euro model but before we get into the sort of details of that can you give a brief overview of how we use observations and equations to generate model forecast? Because I know Peter and I don't know if it's like this in Europe where you may have mainly matriculate when I ask people here just casually. How do we make a weather forecast? How do we know what it's going to be five days from now here in Atlanta or in New York? I find that people just don't that are not versed in our field. Don't really understand that. We're using models and observations to. Can you just give the public a little one? Oh one on that process. Even the the Tra model is not very clear to many people actually means you know when you when you say model. What do you mean you build something with with sticks and paper or what is it you know? And so the American modeling is actually what we do. And it's nothing but trying to find a way how he can actually do calculations of the actual physical law that happened in nature. There you know you see cloud condensation and the cloud forming or you have air coming game against the mountain and rising and creating some funny Features in the Lee of these mountains or big weather systems that you can see inset line pictures that that caused rain precipitation and storms so something like that's what we tried to represent models and that's that's no trivial task at all and what. I recently started to use as a term is. Actually you know if and that's probably appeals to young people like this this concept of like virtual reality so if you if you a simulation of ours that we run everyday and the same is done at the US or Japan everywhere so you take your your goals and what you see is actually what the model is doing. And then when you when you look for your goggles you can actually see these processes. They're not represented in the May. But it would look like as you look around yourself. Look like the real atmosphere Forming clouds or or air coming against the mountain or the storm forming above your around. You and the better models are the more realistic. That picture looks into more real. Your virtual reality would be and I think then people start to understand it. Because they're familiar with the concept of computer. Games and computer games simulate purchaser reality as well. Yeah that's a great way to think about things because I concur. We even just as academics and scholars. Even be in disciplines. People can be confused by the term model and I think for the public. They actually have this sort of mental model. If you will pun. Intended of what a model is so. I like the way you put. That often remind people that the atmosphere is a fluid. It's governed by fluid dynamics in Dhahran hammocks equations that we can simulate in model equations in the same way that we're modeling river flow or Pie. Af- water flowing through pipes for example how we optimize across turbines Formula One. 'cause you know it's the same fluid dynamics different way we have a bunch of additional processes Like MOIST PROCESSES. And as you said you know. Land surface and water water flowing along rivers and atmosphere composition of these things but fundamentally the underlying fluid dynamics is very similar. Let's discuss now circle back to what many of the listeners of this whether podcasts are interested in we're talking by the way with Dr Peter Bauer who's the SEM WF deputy director of research in the head of the scale ability pergram. Now I mean Hanau. Many people look to the Euro as the pinnacle the peak the best in terms of forecast models. And I think you all know that. But how do you view this perspective that people have concerning the euro modeling? Do you strive to be the best kind of pot of gender you know? I mean it's not about competition as much as that sounds though I think. The original idea of the foundation of the center was not to be globally based or something but to kind of funnel European expertise and resources into a single course and say look You can do this. Individually and people and centers national met services in Europe was still do that but we can create with joint efforts a single place in Europe where we put our best experts That will take in and out We locate some sincerest resources together and that allows us through a collaborative effort to create something that we individually can't do in terms of accuracy and reliability of focus. So that was the the the basic agenda at the onset of his efforts forty plus years ago and it worked you know it turned out that actually following this recipe and and centralizing excellence and resources. helps and cost us to be number one since then in terms of global medium range forecasting and success was also focused Because we off nasty you know what? Why is it that you better than others? So part from this this centralization arguments you can also say the focus. Many met services And no is included in German weather service as well or Medoff is very broad portfolio of the customers. They need to serve and the technical equipment and he to running shooting satellite programs. And such you know. The European Center was focused on medium-range prediction Goebel nothing regional limited area. It's global image prediction And that covers the chime range between day. Three and eight hundred fifteen. Let's say That has changed a bit over time. But that kind of focused helps because you have a single purpose. Yeah some great insight from Peter our here in the US. The European has a fan club. You guys have quite the following and certainly for for good reason but I should remind that most meteorologists and forecasters are looking at all of the models the USGF s the European. The Canadian model the came in office off a model and many others in it's a collaborative process but the reality is when we start looking at the metrics that determine sort of skill in there are lots of sort of Jargonese type things you have to trust me most listeners. And not interested in in terms of Anomaly Heights Gordon all those types of things one of the things that we know is that European model outperforms and it's been said that the data assimilation approach in your integrated forecasting system or AFS are at the heart of why you generally have superior performance first of all talking to sort of sort of our listeners. About what data simulation is and then what makes you different than what we're doing here at the US at the National Weather Service and other places large and in many cases we get through that you know? The difference is very large construction. But I think in the end. It's like the the experience the time you've spent over years and decades to kind of optimize the system but I think the fundamentals we all agree on just like for the models as we just sit earlier We all based on fluid dynamics same equations. Maybe slightly different ways of putting. It's putting the on the computer same. For data civilization we use kind of the same algorithms have slightly different flavors of over running it so data summation is the way to fundamentally create your initial conditions. Because you need for a good focused to basic ingredients. One is that you have a good model. Because the model is the only means you have to prediction to look into the future weather in Washington. Ten days or how a tropical cyclone evolves in the Pacific Ocean in five days So this unique model. That's your your looking glass and then you need to stop that model somewhere so you have to have a very good idea of what the president conditions are worldwide in three dimensions Across all the variables that are relevant to us like this pressure temperature moisture clouds surface conditions all that And these initial conditions. The starting point where you're the point. You launched your model from his create by data nation system and the term data simulation relates to that you using a lot of sedation for that and that makes sense because for for gauging what. The present state is a unique. You observed things And that you know links to another great point of international collaboration actually which is the exchange of observations that relate to whether the coordinated approach to design and run set like programs across continents to observe to make observations Chevy's observations in very short time period actually across Andrews And then Once you have that data that needs to be with us within like three hours between the satellite observing Shiites over the Antarctic and having that observation here in hours you put these observations together to create a three dimensional picture of the atmosphere but since the observations don't cover everything everywhere at the same time so they don't they don't measure temperature and moisture in clouds everywhere all the time. We actually have to use the model as well to kind of fill in the gaps loosely spoken. And how you do. That is complicated. Because you have to run the model back and forth you use your observations to adjust the modal weather model is quite a big wrong because the model is imperfect obviously sedation up perfect either. So it's a bit of optimization process between the information you get from the observations and the information you get from model edgier judge the model as much as you can so that they fit the of patients as best as possible and then you have followed your starting point phobe predictions and we are back on the weather. Geeks podcast. I'm talking with Dr Peter. Bauer of ACM. Wfan's director deputy director of research. I'm Dr Marshall Shepherd from the University of Georgia and also a brief colleague of Peter when we overlapped at NASA and we're talking about the European model some of its modeling programs. Forecasting data assimilation. All kinds of things that are at the heart of your day to day weather forecast. And you heard Dr. Bauer talking about satellite data symbol satellite data and assimilation into the models now. European model really gained fame at least in the US. World quarters if you will after Sandy. Because I think there were many sort of studies and sort of case studies that showed that the European model sniff out sandy's hard left turns well before the American model nine days before the actual turn. So you've done data denial studies or at least someone has that showed. The without those satellite data sets. That forecast would not have been as accurate. And you mentioned this earlier. So am I correct. Peter noting that there is a strong partnership between the US European Space Agency's and other space agencies in terms of the state of flow. And it's not just the space agency's is also a country with maintaining ground-based networks of radars of station networks aircraft sharing vacations That are measured along them. Crossing the Atlantic for example. So it's a it's a it's a global effort coordinating The observation of the atmosphere of the entire system actually and sharing of information very quickly. But you know since you mentioned the satellites Many billion. Us dollars spent in in satellite programs every year globally accumulators. And so that needs to be well well coordinated between countries to make sure that's best value for money greed and that's I think one of the ways we actually overlapped isn't a satellite mission. Call the tropical rainfall measuring mission and its subsequent follow on the global precipitation measurement G. PM mission now just before we continue. I WanNa give you a little bit more about Peter Bauer. He's the German Helm Hold Society International Fellow Award Winner Group Achievement Award to the GPS post launch team from NASA. He has a certification appreciation in recognition of outstanding contributions to the PEX program issued by the World Meteorological Organization and the Professor Doctor Veal Ho Vice Low award for the development and implementation of instruments and methods also issued by the World Meteorological Organization. So clearly we're talking with one of the world's top experts in this area. I want to mention something that you stated at the twenty nine thousand nine hundred international supercomputing conference he said X. scale systems present a vision for weather and climate prediction ten. We meet those challenges but then you posed a new question you said. Weather and climate prediction presents a vision for X. scale systems. Can we meet the challenges? What is what is what is exit scale computing. Because I'd just have this sneaky feeling that people listening may not know maybe a few people do what is the scale computing. And why is it? Relevant to numerical weather prediction. Obviously we were running very complex. Model model spill engages relation systems. We discussed earlier that We're trying to cost The physics that's happening out there into equations. This becomes more realistic. The better solution is The more complex you image and and and and represent physical processes in your model and the more data you use and you know as you go as you assimilate hundreds of millions of sedation per day and you do his over billions of quick points. It is clear that this is the computing task because he was one of the product steam. The focused in in very short timescale since for example we need to complete our focus within an hour. Real time That's fourteen day forecast at like nine kilometer global resolution. Thirty seven levels. And this kind of you know this kind of numbers. So it's I think computing task so traditionally On national mid services or or international. Mexico's like us have used computers for that and the next game in tone is excess. Gail next scale relates roughly to the number of complications Or calculations of floating point operations so number operations. If you wish that a such a system could feel perform within the second an extra relates to tend to the power of eighteen. So it's a it's a very big number and join us to these kind of numbers may be if you look at your the specs of your laptop And the processes that sit on the let left off. They have certain speed Indications of how many gigahertz processor operates it and that's very similar in comparison so excess galas a very big big number and the excess requirement Pretty much comes from when we say so. Today we run. Our system at ten kilometers is On the computer of size x if we would like to learn the same system at one kilometer to increase physical realism Location detail Simulate a thousand times more today since May be more complicated and complex and many more satellites than we would need. Maybe a factor of thousand more calculations per second to complete a focused within that same hour you know and if you extrapolate that you pretty much end up at like excess scale figures and this is where that term comes through the statements that that you later to earlier. Is You know you know? It's nice that that sounds and as as cleanly as you can work out such next Extrapolation oh models are simply not fit to run these resolutions and efficiencies that are even close to running on Xtra scale systems. We can put him there but they will run very inefficiently. And that's a challenge. We have And we need to breach within the next five or ten years because the requirements for more accurate forecast grow and grow. That's right. We talked with Peter. Bauer from SEM. Wf No one thing that I've noticed and I've actually written about this recently and Forbes the major modeling centers around the nation you all the United States. Uk Met Office. I think even the Canadians have announced massive budget allocations for increased computing power and obviously that relates to things that you said about where we need the resources and where things are going. I've always told people that to some degree. The skill of our weather forecast or directly related to computing power in the resources that we allocate to them. You all at the universe sorry at the University of thinking in the University of Georgia. Which is where I'm from by the way but you all ACM w. f. have it seems ample budgets. It seems at times that there needed to be events like a sandy or a hurricane harvey or others for the US to Congress and others to get our act together and get these supplemental bills and fundings whereas you're you have a funding model that allows you to just continue. It seems to the by the biggest fastest computers. How how what is your perspective on? How your budget model compares to the US budget model or knows where I think no at least has to allocate the weather service of supercomputers satellites radar and other things. Can you speak about the differences in how those models are? Funding models at lease are aligned. Yeah I think this is not presented. Quite right you know. I don't think we have a limited budgets. I think we work actually. Unquiet leading budget But I think what helps us to a certain degree? Is that focus that I mentioned earlier? So we focus is on medium-range global predictions so we don't have two quite spread as widely as other agencies have to do but I would claim actually that we use for daily focus. The forecast that created the Sunday forecast was probably run on similar. Hpcl location in terms of you know how many processes you on that thing on To to complete in an hour has been very similar to what other forecasting systems run on. Because that's how much it costs So it's a bit more complicated stories if he. I don't think you can translate to success of each isn't so uni directionally onto HP and the HP budget itself. Think the message needs to be slightly more complex. I think one advantage that we have and that we built in purposefully. You know our. Hvac budget is that we want to allocate a significant amount of resources for research. So if we buy a machine and we we produce procurement for us. It's very important that about half of that capacity of the system is allocated for research so think this is part of a recipe for for making sure that the recent researchers that work here can actually play with the operational configuration at that little change his testimony within a reasonable time so they have enough turnover cased and verify what they're doing. I think this is important. The operational location. So that's the you know. The Sandy focused allocation That stuff is only twenty. Five percent of the total capacity. And then we've missing twenty five which is allocated to member states. Don't forget we're member state organization And we allocate Some of the HP budget to them as well so why we're focused We have specific distribution of HP resources. But that would actually claim that today for sure The the major national services like the US so noah or metaphysics or DWDM Germany. Recently Have pretty similar. Hp systems as a whole and probably very similar allocations to the specific focus sweet that produces day this nation for the initial conditions and forecast. So I think we're pretty much poverty and we are back on the weather. Geeks podcast Dr Marshall Shepherd for the University of Georgia. And I speak with Dr Peter. Bauer of SEM Wfan. We're talking about Euro model and how SEM WF does business as compared to say the national weather service or Noah and so based on one of the things that I'm learning from. Dr bowers remarks is. It's not likely that you're going to see the. Wf For example develop an HR model which is high resolution model. That Noah runs. You have a very specific focus and you know I guess Soro Adage Find Your expertise and do it well and that's what you all do in your sort of focus in with that. Now I think you clarify for many people that you don't have unlimited budgets are very strategic in sort of optimize. How you utilize that budget. I want to kind of pivot now to your view of the world. I mean you. You all are upgrading. And continuing to upgrade from your vantage point as the director deputy director of the research department. What's next for you? What are your biggest challenges going forward to continue to improve weather forecasts and also interested in your thoughts pier on where the limit is I mean? How far can we get out with credible forecast and with resolution because I saw some recent workout at Penn state by I think the late fourteen Zang and his group That harks back to the old Lorenzo and chaos theory that there are these natural limits. An how far we can go so I want to get your thoughts on that but also what are your big challenges with the Euro model and he seemed. Wf going forward. I think in the day to day business we. We have the same challenges. Everybody else you know. We try to improve for solution. Of course Just described the the the realistic view of how physical processes operating there between now and day. Ten Day fifteen or up. Two seasons even So that's the same and I think we're investing into More complexity of your system at the same time so we want to couple atmospheres with oceans and land surfaces and sea ice and atmosphere composition because all these processes interact with weather And I think increasingly we have recognized that in the past five or ten years. I'd say pretty much today. Most of the operational global systems run couples Earth system models not as a couple and complex clement models are but still in the sense that they cover a couple at least a physical space of the of the system and has shown to produce a great improvement certainly the medium range but also the extended range and with extended range forecast into the month. Or even season's over in India. For example you see great benefits of that so these challenges are very similar to everybody and it sounds maybe triple saying that you know that you that you add aerosols in or interactive aerosols like dust particles in your atmosphere. And you get better focused. But all these posters interplay at different scales on different time scales some processes of first faster than others some of therefore limited by your time stepping or your spatial resolution or how aerosols indirect with clouds for example radiation. So it's a very complex interactive and Heine Jew and system. So you can't just add components and you see an immediate benefits. It's as low as low funds. It takes a lot of scientific research. Actually get the answer right across all scales globally and we verify globally every day. So that's that's like the incremental challenge if you asking for like the big time challenges come back to the computing. Actually you know. We talked about big numbers in budgets. And what is important to keep in mind and we set except floor access scale. Is The scale the reach For realizing high resolution simulations for example but right now and that is going to get worse than that access deals if we extrapolate into the future. The efficiency of our Global Code is only five percent on these systems. So we only use five percent of the calculation rates of these Confusing systems throwing. And that's and that's because you know we writing so many complex calculations and data communications beeping pulses and you know That's we just can't optimize system with the current programming and poetry and the way we map of physics problem onto the computer and so I think that number is really important for us to increase efficiency and have a better return on the investment. We make in the first place and this translates in in in the center but the same everywhere in to really you radical reformulation of computer codes for future architectures of processes as well so in the end. Ideally you want to use used an architecture like like you and I have an iphone so exactly. No we have a general purpose processor that you do your emails with You WanNa have a GP do your image processing with and you WANNA have a little air processor that you that you can play with If if you have the AI APPs for that so very specialized configuration of processing technology. And ideally your model dangerous mission system is able to use that and to do that for now and for the future issues. Shallow for for us but it will be the key to unleash. You know the the Games we can expect from resolution and model complexity. We're coming to almost an end of the podcast has been amazing discussion with my colleague. Dr Peter Bauer a couple of more questions. I want to get out. You have you on the line. We're doing this sort of transatlantic. I Know Peter Sitting somewhere. I are you reading right now. Yeah I remember visiting reading a few years ago at the European Meteorological Society meeting when President Obama side. Definitely quite nice weather when I was there but speaking of and by the way you see him. Wf Is based in reading. But you you said something earlier that prompted question because you mentioned that you are Member State Organization House Brexit all going to impact your operations or is it It's important. The member states are fall. Perations nothing how is brexit going to impact the European ACM WF operations or? Will it full dancer? The number one is formally doesn't because we're an independent European organization so I'll member-states stay as they are then that's not a brexit and the UK stays a member state of ours but we have indirect effects. Of course we're located located in the UK so if you're fit to hire scientists or computation scientists for something. People may not feel inclined right now to apply for position here if they don't know if they're wife's can work children can go to school in these things so we have that aspect the other is WE I involved in a lot of collaborative research in Europe which is funded by the European European Commission and to certain extend may become a problem because it's difficult to host European Commission funded projects outside the European Union to the to the negative effects that we see right now so you know there's a bit of time to deal with it but it's clearly something that affects us indirectly But formerly doesn't certainly it sounds like there will be no degradation or Fra problems with the the the model in the forecast which is quite good for all of us a as I remind everyone that we all look at all the models I want to draw to a close just kind of circling back to your role as deputy director of the research division. Tell us about what you do in that role what. Sem WWLTV's ten year strategy is in what the scale ability program is and then we'll draw to a close the I think it it. It's pretty much what we've discussed as my role as deputy director is that I compliment the director of research. The director of research is like more in charge of the day to day formulation and guidance. For Research at the center we have about one hundred. Scientists are working in the research departments across a few sections. I compliment that with the computational aspects And we put the two of US kind of design and discuss The research strategy for the next ten years. I should do this right now. Because we're formulating our next. In Your Strategy and computational science book playing important role in that so that's That's what I do the commission as I said You know. The computing problem is is a very large challenge for us Spoke ABOUT UNLEASHING. The politics of mortals. Data's mission systems to come And why it has to be recognized. That's already like Seven years ago I'd say no we fund it what we call it. The scale ability program actually To run a project across the individual departments and groups that we have here across research focused apartment computing department and so forth. You gotTA COORDINATE. The work that's needed of the developments that are needed to kind of overcome this threshold of five percent efficiency. How we deal with data The Petra Bites exit bytes of data in the future. How we get our member states involved in that and actually how we can reach out and and draw an expertise that we don't even have if he can go. Yes In terms of computational science and they're all within facilities also with the US in the Bay Lots Department of Energy Programs on X. scale computing But also exists computing aimed at weather and climate prediction And there's joined efforts there so you know just like the the set like collaborations that you mentioned earlier Along side that we have very good international. Collaboration on the computing Issue as well nothing really good and that is where we're going to have to end it but before we get out of here in that time of the podcast. It's time for our Geek of the week are Geek of the week. This Week is sky warned. Spotter Drake Caldwell. Drake is constantly posting his weather reports on twitter and also does his part for the community by helping investigate storm damage in the area that can be super helpful to the National Weather Service. Forecaster so thank you so much drake. He's most fascinated by hurricanes but the weather of it he remembers most with a Tornado. That traveled bright by his house aches. Drake also loves to help out with making the local forecast and so far he feels like he's done a pretty good job. Maybe there's a future in you for Meteorology Drake. Thanks for all of your hard work. A sky warn spotter. And congratulations on being this week in Geek of the week. Now if you know someone that would like to be the Geek of the week or you'd like to nominate or hey nate nominate yourself. Check out our social media pages on twitter and facebook. Peter Thank you so much for joining us on this episode of the weather Geeks podcast. It's been great opportunity. That was really enjoyable. Yeah really we think hopefully everyone has a better appreciation of the euro model. And how you do your business in. What's to come and this has been Dr Marshall Shepherd from the University of Georgia. And we'll see you next time on weather geeks me.