1 Episode results for "Jordan Hankins"
What's the Deal With AWS Billing...? With Corey Quinn and Pete Cheslock
"Do you pay money for me. To BE CLOUD ECONOMIST. They said yes we do and I said Yeah. I McLeod economists. It's time for arrested. Devas THE PODCAST. That helps you. Achieve understanding develop could practices and operate your team and Organization for maximum devops awesomeness. I met Stratton Jessica Care just because usually a lot more excited when she says her name. This is very subdued. I think I have Democrats getting to you. I being subtle and and it just lets may have more room to build later. Got It so. Subtlety is not going to be the theme of this show. As you'll find out to our guests but before we bring our guests then let's have a word from our sponsors. This episode is sponsored by Circle. Ci designed for modern software teams circles. He is continuous. Integration and delivery platform helps developers push code with confidence trusted by thousands of companies from four persons startups to Fortune. Five hundred businesses circle sky helps teams take their software from idea to delivery quickly safely and scale visit arrested devops dot com slash to learn. Why high performing devops teams you circle. Ci To automate and accelerate their CI CD pipelines. If you're like most of your friends and devops you probably prefer using open source solutions for observability. But you also wish you didn't have to sacrifice scale ability performance and simplicity with logs. I owe you get the best of both worlds for your cloud environment. You can use the tools you love at the scale. You need logs. Io is a fully managed service that offers complete cloud observability for engineers on one unified platform. Log Management and Cloud Sim based on elk and infrastructure monitoring based on Crisanto to give it a try for yourself. Sign up for a free fourteen day trial today at logs dot. Io Slash Ado and for your chance to win a free logs. Iot shirts the worst thing about the arrested devops. Podcast is when it ends. You're left wondering what to do next. What are you going to listen to on your commute home? How do you occupy your time when walking the dog? What are you going to listen to during the quarterly? All HANDS MEETING BUT FEAR. Not Dear Listener. There is a solution you need to subscribe to software defined. Talk Right now. It's a weekly podcast recaps. All the news in cloud computing devops enterprise software the hosts coattail Matt Ray and Brandon. Richard will keep you up to date on all things cloud while offering tips on how to optimize your Costco Hall and how powerpoint. It's a fun free flowing conversation. That will keep you entertained and informed. What are you waiting for? Subscribe to the podcast today by visiting software defined talk DOT COM or by searching for software defined talk in your favorite podcast APP. This episode is sponsored by check. Mix so as I said this is not going to be the episode about subtlety. But hopefully it's the episode that'll be somewhat informative and maybe hilarious joining us. We've got two great guests first of all pizzas lock. Pete say hi to people and also Corey Quin. Hello how are you? I'm pretty sure longtime listener. Even short time listeners of the show. Are maybe familiar with both of you. You've both been on the show before your prolific and prevalent on the twitter verses and everything but just for some quick background. Take a minute or so kind of just. What would you say you do here? I mean I guess I'll start. This one just goes I for the past. Six months did very little. It was kind of Nice I used to work for a company called chaos search which all caps so you're required legally to scream it got to scream at yet it was in my employment contract And My my Separation Agreement Hata screaming out loud everywhere you went and so I moved on their from being a product person into the wide world of consulting and You know when you're consulting. You're just you're just staying one step ahead of the game and I was just one step ahead of that whole world and now now I am. I cloud economist and I worked for the DUCK BILL GROUP. That was that was You know a big big chain. My life going back to the fulltime fulltime job. Yeah he went from doing nothing for six months to now doing nothing on someone else's dime which is really the best way to do nothing. If you're now if you're good at you nothing you WanNa do nothing with like a corporate expense account. That's a great time to do nothing. The problem is you don't want to wind up doing this during an an pandemic because then great you have the corporate expense account. But what the heck are you going to embezzlement for? You can't basically abuse the meal policy Sneaking upgrade you're not supposed to qualify for isn't really a thing and getting huge numbers of boxes delivered to your home is entirely too. So what the Hell is the cloud economists because we have two of them on the show right now we have you hope to clydes economist. Yes what is the plural? Noun of cloud economist. The plural noun the collective duck bill rate duck drill a duck bill of receipt of cloud of clouds economist. Twelve foot long almost reached the ground. So what exactly is a cloud economist? Whether there's more than one of you or not well. My definition is the title that they gave to me when I joined duck bill group like I said you were going to be a cloud economist and I said W. Sway so your definition of cloud economist is. It's the title that Corey gave me when I joined this company. Exactly and my response was do you. Do you pay money for me. To be a cloud economist. They said yes we do and I said Yeah I. I am a cloud economists. So it's like meritocracy if you have a meritocracy than you must have merit so everybody. Who's already there wins. I think everyone in their heart is a cloud economist. When you really think about it I feel like there's an old joke where a student is taking an exam and praying and Shakespeare suddenly materializes. Great what did you mean when you wrote this stanza and the answer was something completely banal like I was trying to impress a girl at it. Sort of worked. Great sort of the same story here. I I originally cloud economist. Because there are two words that no one can define cloud meaning a bunch of other people's computers at economists meaning someone who claims everything about money but still dresses like a flood victim so combine the two of them and who the hell was ever going to question me on it so. I looked around and realized. Oh there are other people call themselves clouds economists including Rogers over at four fifty one research who apparently has a PhD in cloud economics and he was super excited to meet me and I had a choice to make. Do I wind up passing up to the fact that I basically turned his? Life's work into a travesty. Or do I team up with him and get a book deal and Since I'm telling the story out loud prior to publication we know how that one worked out but fundamentally what we do is we look at companies. Aws bills because those tend to be the big ones and help them become smaller and less terrifying which sounds incredibly simple but there are hidden depths of nuance to it. That basically make us cynical an angry all the time hidden depths of nuance to aws bills. Yeah those those clouds are not white and fluffy no. They're stained with the blood of juniors of junior devops who have clicked on the wrong instance types when provisioning things just get a deploy a couple of you know e. m. r. clusters sure cluster and it fails. But it doesn't turn off the old one and there's no identi- check so it's sponsored a new one every time you run like the accounts creation system wells Fargo apparently. Yeah so you wind up with fifty clusters per engineer and turns out that you're not building the cloud for what you use rather for what you forget to turn off so we're talking about this. You alluded to it. You know that you just you just joined duck bill Recently so besides having an easily exploitable expense policy what really major interested in in making this move well other than the fact that Corey is the person who approves my expense reports and by approval. It's after I've spent the money already. He goes oh that's cool. I should get one of those too by the way if you're thinking about contracting with the duck billed group we are now talking about a company that is supposed to help you better with their money and clearly is terrible with it themselves back to how you buy real things with your money real things. You've met you joke but I have always made it a point since I started on this path at reinvent to explicitly not ever the caught at a slot machine because wait. Isn't that the person supposed to know all about the money suddenly pouring money into wow? He's a lot of drinks around him along as he did there. The whole point is that we want to have a veneer of making sound financial decisions and then we go ahead and completely different. You know hiring P exactly and it'll be it'll be interesting when they're you know the next reinvent rolls around and there's Pete the cloud economist at the craps table with a whole group of Amazon cloud you engineers around him because AWS REINVENT EXPENSE. Reports are especially full of dark nuance. I'm looking at this invoice. Here did you buy out the restaurant? Did you buy the restaurant? I could really make a strong argument either way now in serious from our side when I was starting this out a few years ago I was talking to a few folks that I knew interested about the expensive problems in the space and I talked to Pete which in hindsight was a terrible mistake because I almost wound up going in a completely different direction. It turns out. Pete was very in depth with this to the point where holy crap. I've got to be really freaking. Good if I want to help companies with their aws bills. It turns out that Pete was and is a bit of an outlier in that context so I studied up and loaded for bear and I walked into some of my early clients and Oh you haven't bought an Roi in eighteen months and nothing has ever been turned off cool. It turns out that a lot of that high level nuance while incredibly valuable is not necessarily step one as you walk through the maturity model of getting a company to a point of reasonable clouds spanning the governess. But yeah that's that's a really good starting point which is back to your question. Matt of. How did this whole thing happen a couple of years ago? When Corrie started doing this we met up and we had a great chat and of course. I'm just like Oh you got to think about this. You gotta think about this and from being in the cloud for almost ten years now I was just like oh all these other things and all these edge cases and everything else. You're kind of forgetting the fact of consulting. Which is you just need to know a little bit more than your first customer. Because then you go to your second customer and now you've got multiple companies of experience. And it just kind of snowballed from there but Corrie night chatted last summer or maybe a year ago or something like that and and we always check in each other's devops days or at at reinvent and you know whatever we're talking and always kind of ends up with you know at some point to be really great to do something together but it's the great challenge of growing companies is the tiny and luck and circumstance. I'll have to line up perfectly and then I had been doing consulting and I've been working on a couple of pretty interesting projects over the last few months and then a couple of them actually finished up early so I was kind of out there looking for my next thing and chatting with a few folks and I saw my Julian who is Yo- of Duck. Bill had tweeted out. Like hey like we're looking for some some folks so I just Kinda like slid into his said. Are you looking for part-time maybe consulting? I'm flexible. I thought you were going to. Are you looking for me? Do I turns out that we almost lost out entirely because due to a fund bug and tweet bought. It doesn't actually show group. Dm So it's sort of sat around for awhile. Mike one day logged into actual twitter. And Oh hey chess locks on the market. How about that? And we sort of snowballed from there. Because I am freaking terrible at following up on DMZ and whatnot. Because I mostly twitter as a transmit only mechanism actually was very hilarious and I sent the DM thinking like I al here back in like thirty seconds. Because they're always online they're always on twitter and it was like we had gone by and I was like what ever got that like. Do I send it again? And then but luckily there's a pandemic going on and I was stuck at home and and you know trying to take care of my kids and then everyone and so I wasn't really doing a whole lot in general so worked out kind of come not a whole lot here and it seemed like a great plan. Yeah exactly exactly so but yeah actually re originally. We're just GONNA do like a part time thing or a consulting thing and what happened is like the more we talked about it a just about you know. What the company's doing today and how important it is for a lot of companies to reduce their spend because it's it's in many cases a difference between do we lay awesome engineers that took US years to find and train up or do we turn off. You know all these servers that this team totally forgot about right and so as the economy started to really get questionable. I was like this is actually a lot more important thing and have meant in the Amazon ecosystem for ten years now. I think that was another big draw. Which is I got to continue on in that world and keep using some of that experience because it it. It's a lot of fun kind of chat with folks and all of these different stages of their growth and this is a good time to be in consulting if you're consulting is saving people money on their aws Phil. I mean. I would believe so but my first full day was Monday. So I don't have a lot of data points on this one but I think saving money and really never goes out of style. I think people are always interested in saving money but I think when when the April bill comes in when the you know the march bill came in April. Bill comes in people. Start looking at that going. Hold on a second. Like what's what is this. You could store oil in the cloud. Maybe if you evaporated badly that sounds like it might have environmental consequences but we went definitely have some dark nuance now a slightly more nuanced answer. Is that something that I've noticed for a while? Is that what customers and what customers actual pain points are historically aligned super. Well what happens? Is You wind up with someone in finance getting Amazon Bill and they see that looks like phone number. That isn't there so they start to worry and they wonder how many books engineering is buying you. Then have that moved through about five different levels of corporate telephone to the person who spun up a cluster but it's magically never allowed to see the bill and it turns out the pain is not that it's too expensive but rather that it's far too difficult to figure out what the cost drivers are and allocate that so it comes down to understanding optimizing and predicting it now that we're suddenly seeing a recession style. Pandemic event customers are a lot more accurate when they say so. What are you here to do? We're here to save money now. Suddenly that's true rather than just understanding it. So it's it's simplifying aspects of the consulting but it also means that they're that now there's a little bit less nuanced have those conversations and used as we used to say it. Looks like a dumb question but why do you care about the WS bell? Well now it actually is a dumb question because most of our customers are inbound folks. We're talking to. There's a very clear reason why suddenly all their projections magically shift. You mentioned that. The engineers turning on the instances do not get the feedback of how much thoughts costing as like. We did devops in order to give people the feedback loops of the consequences of their decisions and yet with cloud. We just took that away to some extent. Yeah so you've always had this problem with traditional data centers to the just was buried in year. Multi your cycles. So that's kind of the beauty of cloud for me. Anyway is that there is any hope of ever measuring how much it actually costs as opposed to the data center. People burying it in the cost of new projects because they're only allowed to get budget for new projects so they roll all that maintenance into it and you wonder why your data center projects are still ridiculously more expensive than they really are. Don't worry you can still get away with financial hijinks in the cloud to just tends to look slightly different. Exactly I mean. That's that's the interesting part with a lot of these projects like you might have people that are doing the devops and they you build it. You run it world and and it's a great way to move fast. You give each team the authority to do what they need to do. But in many cases like it's super nuanced I mean there. There are bugs that Corean and crew have found with Amazon network data transfer. I think my favorite being that it's cheaper to transfer data between one and two than it has in between availability zones. Something of which. I don't think anyone at Amazon realized or knew it was shocking right. But that's just one of many examples. I mean for for people who have legitimate spend on Amazon or probably any cloud provider Amazon. Being obviously the biggest the number of line items in their bill could be tens of thousands or millions. I mean the number of pages in their bill could be measured in hundreds of pages depending on how they're spending their money and what they're doing with it don't print it out and again you might wonder what a non-legitimate cloud spend is well. That's my ret. Conned origin story were ten years ago. I spun something up as a test that I've been paying twenty two cents a month ever since and so three years ago I snapped swore revenge. And here we are. I feel like that was the other reason I knew that this is something that I enjoyed was similar story. A few years ago I was cleaning out my Amazon account and I I went through and looked and I had this to sent glacier charge. Glacier in my Amazon account did not understand. It was two pennies. I just ignored it like I had been ignorant for many years but house like no. I'm GONNA solve this and I went through the most convoluted series of documents on the Amazon site. In order to find some legacy vault that had been sitting around since glacier was first announced Many years ago to then run a series of commands and I finally was able to delete it through like a bunch of Lupine and a bunch of really terrible code authored. The Amazon Command Line. It probably took me five hours to do that. Hassle we now know leading it. There was there was there was there was. I definitely got hit on the. Api calls. Because I had to do so many calls in order to find like every file and the vaulter whatever the scenario was but the end result was yeah. I spent like five hours to save you cents a month. But it's not on my bill anymore and you really can't put a price on it through manners. Well it's very much I've always felt like because I have that twenty two cent charge too and it's like the friends episode with Ross Cheddar like I want to quit the gym right like it's easier to quit Jim than it is to get rid of that last little bit of of your belt but if you have more than twenty two cent bill. So what are kind of the misconceptions? I don't WanNa say mistakes. But we're people missing when they're thinking about managing cloud spend and this can be a misconception. That's happening at a practitioner level or at a strategic level. Where people missing? You WanNa take this should I? You know I'll I'll start with one that is I think is where people miss out on. And it's not necessarily like you should do this and you'll save money. It's you should do this because a couple years from now you're going to look back and think yourself and that is having a clear concise way of tagging your usage of things in the cloud. But when you think about tagging you wanNA think about Basically how your company makes money by using the cloud and maybe I'm more biased. Because I worked for a lot of SAS companies but the SAS companies are the ones who are selling some sort of service and they're trying to figure out essentially. How much does it cost per user? What DOES OUR COST OF GOODS SOLD? And the reason why you know two year old you. It's smart for adding those tags is because current you is going to have the CFO role over to you one day and say what is our cost of good sold and you have to have a way of mutual. Ca slicing days that data. I think the more challenging part of that is and there's ways to do it in Amazon is to not only have structure policy. Assume your users are not gonNA listen to you or follow it at all and find a way to like enforce that policy so that if they don't create the proper tagging you just start deleting resources right under underneath them because if you start having stuff spin up then you start running reports and it's like oh the state is not tagged or all this all. These servers are not tagged. You're going to spend way too much time walking around and saying is this yours is this year's and everyone's going to say it's not mine and you say okay cool down and then he kind of approaches to block it off with security rules or tightening up permissions but no one understands. I am so yeah. Just turn it off. It's just turn it off. Yeah I take the scorched earth approach. We're in this case. It's not that hard to find another job. Come on another another more common approach. I've seen to across. The board is all the same month. I had two or three customers asked me about Alexa for business. Which is why. Why are you asking? It's a three dollar charge. Why are you? Oh that's right. The bills are alphabetical. And this is a microcosm of what we see across the board. Which is that. We still have all of these customers who are focusing on their internal narratives and the first thing that they see. I had a very early customer. That was focused on cutting the developer environment. Where they wanted to have something technical that would spend things obscene things down have a slack dot weighing in or just getting China's customers and they okay great. Let's go ahead and build that for you. But I was doing analysis and oh development spend is three percent of your bill now originally when you were building this out. It was your entire bill but then your product caught fire. Bill Traction wound up spending up something significant and now the development build. Yeah you could save some money but there is a lot of other things you can do. I don't require disrupting developer. Workflows and have a dramatically outsized impact relative to the entirety of the development environment spent so as focused on the right part of the story and sometimes at one of the most valuable things we can provide. Is that unbiased third party perspective where we haven't been exposed to the internal narratives around what the bill contains. Yeah I think that's I think that you kind of mentioned it before a Jessica. You're talking about the we. We gave everyone the feedback loops around when you deploy thing and there are some plug ins. I've seen for things like Tara Forum and other stuff that you can know what the cost of what you're about to do is gonNa cost you but I I really fear for the future in the Kuban any space because if we already have obstruction that people don't understand what it costs when you suddenly start shoving containers into thing where you don't even know what the need like the underlying servers. Yeah like how do you tag that you know? That's a interesting challenge. I also wonder even if you expose the cost that hey if you press this button developer. It's going to cost X. DOLLARS. Who The hell knows the context of what that amount means right so like to me so that could go both ways right. It could be considered. It could be abundances if you do this. It's going to cost five hundred dollars a month and I'm just sitting there going. Why don't WanNa pay five hundred dollars? That's a lot of money for me but for number one. Hopefully my organization has slightly deeper pockets than you know my my debit card. But but there's the context of what's expensive to me versus the corporation. Also the value right again the context in the nuance of this amounts in a vacuum. Just a dollar amount doesn't mean anything and when you tie it individual teams or users. It doesn't work out. Well then you have counting coming down asking who the hell is Jenkins. They're costing all the money and you know that's that's our continuous integration and build system. It's fine butler. Oh Okay and they leave and then Jordan. Hankins crawls out from under the desk crew. Thanks you really saved my bacon on that one but without nuance or context. It's impossible to look at a bill attributed the users and say oh that's super expensive versus that's normal Y is that person costing a king's ransom every month In aws charges. Oh because they work in data science. And that's what they do you can search through bass quantities of data to unlock the hidden. Promise of Alan I. Which is that they can find anything they want except the business model so as long as you're not expecting a return on it. Oh can data science do some amazing stuff but it bring money? You're going to need it so I feel like you might have already answered this but I I'm going to ask because I thought I was GONNA ask the question inside in Lake. Your answer. I didn't like I WANNA I wanNa see if you'd answer it the same way so I was Gonna ask and say okay. I only do one first of all. This is ridiculous. Because how could this ever be a situation? But let's let's make disciples article where some how this constraint exists where I can only do one thing to get better about my cloud. Spend and safely a walking micro service right exactly and that one thing is not to hire the duck mountain has only one to give our do allocate alley not written in Rust. Because then the only thing you could do would be to talk about. How Great Rust is. It's like cross fit but what is like that one thing that I feel like Pete. You might have said this when he said it's about tagging but maybe maybe let me put it put it on his thing instead of if. I'm trying to get an understanding 'cause I can't it can't be like okay. The first thing to do is by the time machine. Go back and tag all your shit right so if you do figure that one out please let me know if you do. There are better things to do with with that ends with the title exactly. But if I'm going to try to get some understanding and get my arms around Where these happened? Were words the first thing I should do? What's the most important thing I mean? The first thing that I would probably do is Amazon has a bunch of of reporting that you can enable for different things. So let's assume you just roll in haunt to accompany brand new first day. You own the Amazon bill which I hope is the case. I mean every company should have at least who owns that. There should be some. I mean it's it's line of it's a budget. Someone should have a number on their head in some way someone should care about it really but there's so many different great reporting usage statistics. You can turn on their not on by default and you can go through and turn those on and the building sections that will start to generate data. You can do it at like an hourly or daily basis but the beauty is it all goes into S. three which is basically free. As long as you're not story in you know hundreds of terabytes of data it's the the scale we're talking about glacier just hostile hours and hours. Yeah that two cents. I'm never getting that two cents back but you know enabled those reporting options because if you nail them now like the first day that you're in there some of them take weeks to actually generate stuff to generate useful insights. So that's the first step because if you can start getting there then useful insights. Does that involve data science? I don't know what that is. That's in the cloud snake oil. It's the it's the thing where it's like. You just turn some stuff on and you're just like two weeks later you're like nothing's in my bucket whereas the data and just magically someone at Amazon was like Oh that flashing light has been flashing. We should probably hit the button now. And then all your data ends up in your bucket. I think I think that's how it works. Scientifically when we sell more upscale snake oil. We tried refer to it as serpent Greece. Just be aware now. Now I think we we think there's a lot of fallacies around cloud when it comes to cost and like why and I hope that we've gotten past the conversations we had ten years ago. Which is that you go to the cloud to save money right you go to the cloud to shift. Capex Apax. That's the end of the story but we had all this. The promise was that are are being a little sarcastic with that. But that's at least that's the way we used to talk about it ten years ago when. I wanted to get some cloud because your cio read the inflight magazine. Yes but the promises the pay per drinking like your. Your consumption is more closely tied to value and all this beautiful stuff. How much do you see that actually being true? Are we just overspending everything and overbuilding? Shit just like we did in the data center ten years ago. Yeah I mean. I can't tell you how many times I worked with engineers who were like. Can we spin up one of those like x? Twenty four thirteen. You know triple x large boxes. I Wanna I just WanNa see what two terabytes of Ram is like like no. You can't have that thirteen dollars an hour. It's more than you make like respite in that up. You can have it for an hour right see. That's a common misconception. Oh you can spit up and have it for an hour. I forget it's multiple terabytes of Ram. It takes forever to boot. Secondly when you're done waiting for the boot you've already lost interest in this ridiculous thing so it's just sitting in the background all the time and all my stars and data turns into a reddit post because obviously if you suddenly have a bill surprise that cost three times your annual salary. The immediate thing to do is why like a small child on read it rather than I don't know opening a support ticket like a responsible adult might know the first thing you do. Is you delete all the emails for a while yet? And then you post on REDDIT BE FANCY. Mr My got my email account to work. Fine you truly are the ultimate devops. I think you're right though that I think you're right in that. Most people are probably still over. Spec they're just picking an instance. I mean. There is a challenge. Because you're like all right. I'm running the JVM APP. So I need some memory and maybe a couple of us and then you run it and you get the great line which is it slow so you double it and you make it bigger and then you know maybe the there's no metrics on us you don't even know if it's slow because of that or if it's slow do something else you know I would. I would walk around One of my old companies and just bug the developers and just say sure you need that are four two x large because the CPU hasn't moved in like four days. I'm just GONNA move it to a T. Two for you. How about that and so it was hilarious because I used to say I'm GonNa move your stuff to a t to and magically everything got really slow that they hadn't used in forever and then I just thought telling them because I was like you know if I don't tell though you know like we've secretly replaced your workload. Let's see if they notice they never did so. Lambda GonNA solve this for us. I feel like he's got a good feel like you're very far away. From the cat from micro asking that we could call that the silence of the lambs but yeah. We'll lambda solve all of these things for you know themselves a very different problem which is sure we could fix. Aws Services natively so it does what customers want but screw you go fix it yourself. Ideally with a lambda function. That is true. How many of the blog posts end up being just? I don't know if it's law fix it. You're damn south. Yeah that that is more or less. What a lot of those blogs does turn into the fan fiction of aws API's because this like Harry Potter Fan Fiction. It's all about trying to fix the plotlines. Generally not because very few fan fiction pieces tend to be quite that scatalogical. It does take a lot of Fox to get a land deployed. There's a whole queue of them. Oh yeah that's why I think. On some level there is going to be a job posting someday for service. Fuck Smith no really I care I CARE. I think they tried to call. Api Gateway originally but here we are so for those who don't pay attention to corey on twitter. And I was just GonNa say you should but you know the but that being said one one one of Corey's favorite favorite things to go after is poorly named offerings from aws. So which is which is fundamentally the worst the worst thing in aws period the worst named thing in aws. Pete you start with that one I. Wow there's two that come to mind one. Four complete lack of googling it first one I don't even know what they're thinking but anyway When Amazon snowball came out snowball service which admittedly. It's pretty awesome. You can mail hard. Drives the ban with was a joke. The bandwidth of a station wagon full of hard drives is faster than Internet connections are but it's great. You can mail one of them. You could mail thirty of them like they're pretty impressive and just how they're built just terrible name terrible name and don't name that glitter bomb. They're just there. I don't know anyway. Just it was bad when that first came out because of whatever they go google it and then you see what it's what the urban dictionary. So that's the lesson that every product manager out there urban dictionary your product. I the other one that I think is fourteen year old. Yeah that's you. Ask you ask your nephew yesterday. And Scott Technology exactly and then the other one too I think because they keep on building on it all of the AWS systems manager manager systems and it it just become a great meal at this point with how many of the different services of which I know Corey probably could name them all have some sort of then keep going and then challenge people which ones I made up even Amazon employees. That one right. I would've thought that was part of the interview cycle duck. Bill was to be able to rattle off the most ridiculously reclusive under stressful conditions. Were big believers in waterboard interviews out. So it's like like enterprise job now except in services it's at some level asking what the worst named aws services is like asking. Someone who their leased average child is while they're on trial for drowning them all on the top. It's a difficult question to ask because there are. So it's you're so rife with opportunity and choice Personally if I can take a slightly more serious bent on it. I think that any of the services starting with the word simple because the unspoken message there is that Oh this is easy and straightforward the. Fuck it is try using one of these things for anything non trivial. Then come back and tell me that genius. It's one of those. It feels like Elements of Google culture seeped into the naming early on and credit. Where do they renamed a couple of those things? Simple systems manager. I'm not getting used to be. It's a ridge. One of his early names Fortunately they they tend to rename things exactly once. And then stop on my Google which turns into a seasonal habit with their messing products. Although I had a good one earlier today Matt you you had mentioned before like it must be part of the interview process to come up with a long standing one. Well something that we just noticed came out today was. Aws cost categories which admittedly looks Kinda cool. But of course we need to find a way to make fun of it so I just said what about. Aws cost categories. It systems manager cost manager as a good alternative title. So I don't know and then turn out to actually be a service. Search like in preview in this Vanja cost manager manager systems vandross cost management in the cloud in preview. Contact your camp. I remember that the naming reminds me of when I was going through kind of a a the beginnings of devops renaissance or whatever and I was working primarily in Microsoft shops. And I was saying I was like it's very interesting because at the time at least and now Amazon is proving this differently All of the Microsoft products from an OPS perspective head very dry very descriptive names operations manager sequel server. Whatever and I said then then they go into like the open source cool world and like they're called Lake snowbird Gremlin doodle. You know or whatever right and I was like it so much now. Like how do you have a serious conversation right? And then we all know how that actually worked out for anybody but now it seems like Amazon's may be you know putting putting a lid on that and saying like no we're going to Microsoft. Hold my beer right. You thought you had ridiculous names for things. No but far and away the worst cloud product name that I have seen full stop and I'm not even slightly kidding on. This is as your devops and the reason is. I was talking to a hiring manager. Two years ago caused rested with what they said was. Look this look at this awful resume. Someone List Azure devops. As if it's a skill set let me junk pile that it's no no no no no that is an actual terribly named service when your service name is so shitty that it negatively impacts the careers of all those dare to mention it? That's how you know that it is the worst name ever now. Take a challenge lying down. I'm sure Oracle is currently working on a service that features a racial slur but until that gets still going to go with Azure devops. Being the worst so a little known fact is Microsoft bought. Get Hub so I believe you'll find it Kronos Jack up but please Microsoft Bucket hob just so they would be able to get rid of Asher devops right or at least be able to make up for it. Just just have an excuse to retire and say and the thing is like if you look at the not the etymology but sort of the the journey of all the things that led to the name Azure devops. It actually makes you say Azure devops. Isn't that bad? Because you think you think Amazon you think manager manager of the system manager stuff is ridiculous go look at all the product names of the various visual studio products because it would be like visual studio online server for source of the Visual Studio. You know whatever so it sounds. Amazing is your devops. It's like actually not bad but when when you seem to actively be going out of your way to like Bait people on twitter with your product name does not seem to be like a great in the inflight magazine. I mean it can't top cloud which like you read it in the inflight magazine and then you look out the among the clouds. Yeah that's pretty good exactly by devops man. Microsoft will sell you the as your devops say Blue Sky when you're looking at the plane window again. You can't buy devops so I sure as hell can sell it to you. I'm also curious about inflight magazines. And that's a little bit of show history so in the bear beginning we always said that the point of the show was for when your boss says I read about devops inflight magazine and then for a long. We've talked about like where these inflight magazines like. Hemispheres does not talk about devops or whatever and then somebody found what did what did say I found an inflight magazine. Something about devops so but we never they in first class because maybe they have different. Magazi- The longer that this pandemic drags on. I think you'll find that the better. The overall level of corporate decision making improves just due to lack of executive exposure to airport. Enterprise software ads. What do you think is the product that is not advertised in airports but should be? I know that I know Corey. You had forever wanted to do ridiculous advertisement at reinvent Just because you can yeah last week in. Aws ridiculous sarcastic newsletter. Because save money by choosing Oracle shouldn't be the funniest thing you read this week. I feel like that would be an interesting direction to take things in but I don't know during a pandemic trying to remember what being an airport was like now. It seems that it's been so long that yeah I have to deal with my family. I have recurring scheduled. It's relatively constant. I don't know what to do with myself. I have no excuse to not go to these meetings. Somebody asked me how I was dealing with it and I said what I need to start doing is getting little plastic cups and United Napkins deport. My Diet Coke into and then it would feel a little more comfortable right if I could be. Everyone's like oh you got to have the window. That looks like airplane. I'm like no I just need to have my pop and with the little. The little United Roller Die. Every time it comes up with some number or whatnot punt yourself in the face spillover yourself. I'd call it turbulence fair. So how how has the world to besides the fact that a lot of your customers actually want to save money now and not. Just talk about it have you. Have you seen anything interesting or or just? Has Anything changed in how? You're doing your work. Besides the fact that we aren't all hang out in airport lounges. All the time like everybody seems to think we do sure a big trend that I'm seeing as well is that people did a lot of projections and made commitments to various cloud providers on a multi-year basis. That assumed the numbers would always be up into the right forever. Now it's a question ha looks like things aren't quite working out like we anticipated. Can you help us? And that's been an interesting area of expiration for us. We've always had a pretty decent hand in the cloud contract negotiation space so that we don't generally talk about it but now we're seeing that come from a different angle rather than The finer points of deal. It's more about okay. We've made a commitment that now. We might not live up to in the way that we anticipated. What do we do so having strategy planning sessions around? That has been very interesting and wasn't something that I originally anticipated something else. That's surprising me as we sit here in. The midst of a of the plague more or less is looking at what customers traffic has been doing. And in some cases it skyrocketing and others. It's falling through the basement on the latter case. What's interesting is that even though we are seeing the reduction in customer traffic and usage were not seeing the bills the spend on the infrastructure decline with the same degree in some cases at all so oh people sort of misunderstood auto scaling to me and only ever scales up and they can be forgiven for that originally when you have customers piling on if you fail the scale up your dropping money on the floor and customers are getting angry and leaving if you fail to scale down while you're just burning extra money in your account for that our it's not the the risk profile was very different so no one spent a lot of time focusing on the downward side of that curve. Well now we're being forced to evaluate that near we are so it's kind of so you're saying you're kind of proving out what we talked about a little bit earlier. Which was the beauty and the promise of cloud was as he last city that I would only be paying for what I'm using and what you're finding is that I am paying for what I'm using and what I used yesterday to even though I don't need it today. Well here's a fun part about the billing system to the ties back to the idea of. Oh and you spend something up. You should know what it costs if you wind up pushing a deploy and your CIC system didn't tell you frame a minimum of eight hours that whether it was going to work or not you would slow down you lose productivity. If that were even slightly acceptable you would see actual customer adoption of aws code built the fact that you aren't tells me that it's not but the billing system for all of these cloud providers operates on at least an eight hour consistency model. So Hey that thing you did. Yesterday boy did that. Screw the pooch metaphorically and possibly figuratively. So what are we so great? There needs to be something that closes that loop cycle now. It's very much an after action reporter up. Here's the mistakes you've already made. You should've tried harder. It's not when you get the bill for that month or if that's the more common case if you don't think check the bill the next day but if you look at the billing system right after you spin up read if you do something that is phenomenally expensive it will not show up for eight hours or more eventual consistency. Yeah putting the eventual and eventual consistency. Your money is eventually consistent in that is consistently database. That is so slow. Blockchain is jealous at sensually consistently ours. Well this has been a lot of fun and I hope we`ve. We've learned the things while it may seem super terrible. They in fact are super terrible but this is also. The thing is expected no. It's a different terrible surprise prepared to be surprised. I what I feel bad for honestly out there are there. You know. There's people out there who have just re up. There is Savings plan couple months ago. But the reserved instances a couple of months ago. Maybe they just negotiated a private Pricey Enterprise Discount Program But then I think about all the companies that have it. They haven't done that yet. And right now and over. The next couple of months is probably a really good time to do that. Because if you're still pain retail pricing if you're spending money on Amazon money that you look at and go. Wow that's a lot doesn't have two different number for everyone. My guess is that you could probably pay less because no one out. There really pays a retail pricing. You can you can always get it cheaper if you really if you really try and I think even if you negotiate something and you had some good leveraging you'd go back and be like yeah about that I mean I will say I've been an operator of Amazon for awhile. I've made purchases of our is that I was like well. Those are not the right notes I need. Can I swap them out and they'll help you out. I mean they're always up for a conversation but as Corey said. I think what's most interesting is. That traffic is going down for a lot of these companies. But their bill isn't it's like they almost never planned for this. This is contraction definition of elastic. Is Not that it stretches. It's that it snaps back. After it stretches sometimes with lawsuits I would say speaking of lawsuits. This is the time when we'd say. Where can we see you in upcoming conferences but the answer is not but there have been a lot of fun virtual ones? I trying to think if there's anything interesting coming up or any of you doing any any anything. Besides the the random the live tweeting of Earnings reports or you know well. Those are certainly entertaining. I don't have anything on my best guard at the moment. For some reason people keep forgetting to invite me to submit for conferences just submit. Oh Yeah Hey we'd love to have your talk at this thing maybe propose something otherwise. I don't notice these things. Are there because it's difficult for me to figure out. Which are the real conferences? And which are just fake twitter hash tags that someone put up there to support me. Yeah Quarry I WanNa know where people can find you on video because this is the first time I've met you in accounts is meet these days on video and data. You remind me of someone and someone who's on. Msg Three K. Might be one of the robots say. Tom Cervo could soon face radio also. Your hair is the longest I've ever seen query debating. If I shave it off cut it or just let it grow free. It's the thing we may discover that which of us are in fact robots by how much are haircuts out. I'm a little worried that it's my hair is going to end up looking like it didn't high school and the nineties were weird time and I talk about it necessarily going gray in high school. That's the other downside of log. Hair that you just keep seeing more gray coming out. What is going on credibility man credibility so speaking of credibility? If you'd like to see the show notes of this episode of which they will be minimal. Go to arrest of upside com slash cloud costs. And if you go to the rest of upside com slash itunes leave us a review in the attitude in store. It's a thing you could do. We might read it on an episode and we might not. I think we only did that. Once and it was Michael Ducey who wrote it under a pseudonym and if you're into spotify an Iheartradio you can find just there. We're all over the place so you know but you. You already found out somehow. 'cause you're listening to this so. I don't know why I'm telling you where to listen to us when you already are listening. Maybe I'm listening on itunes. But now I'm going to go to spotify spotify. Maybe you're listening to this. Once the episode comes out and blasting it from the rooftops at three in the morning. This is the thing you should do. I always just amuses me by the way like I meet people who are not in the industry who are not like not people that work for a living and do real jobs and it comes up about doing positive like Oh. What's your podcast? I want to listen to it. I'm like no you don't I mean I appreciate it. Went like the funny thing is the episodes. I point people were not industry. People are the ones that are regular. Listeners cannot stand which are year end wrapup that are just. Yeah there's just US fucking around for an hour like to like real people. Those are good but like if you wanNA learn something yet. Don't but yeah Pete and Corey thank you for for taking the time I know you have a lot of places. You should be right now like I other room of your house. I have to take a long track downstairs But now I thank you so much for inviting us and having this come out I feel like you you Three people are the first three people I've talked to. That are not my family in quite a few days quite a few days. We also didn't ask you about the soup you made last night. Pete like the reason we couldn't record last night as he was soup. Yeah it was like. Hey do you want to record this podcast? I can't do it tonight because I just making some soup so I'm really slammed okay. I think he was soup kitchen or something. I don't know also everybody out there. So this is soup is the new sourdot so everyone. Who's like playing around with that? It's like no no like if you're legit you're making soup at this point so it's easy you started off. You made this hour okay. Great and then. A couple of days went by near like our does not. Actually that creates. Let's make some French toast out of it. Cool that works too. I got some French toast. Ready some old salary to I got as well but now you're at this stage this quarantine got a bunch of stuff in your fridge and you're like what do I do with it. You put into a pot and just cook it forever. That's what you do or you make for Totta. That's the other quarantine moves so if you're wondering plenty of time. Yeah we can just start doing some recipes right now. Forget about technology much cooking so much cooking going on so with that. I'm Matt's at Matt Stratton Jessica at Justice Ron. This is arrested devops and remember. There's always devops stand in the crowd. I implied magazine.