MongoDB in the Cloud with James Kovacs and Rachelle Palmer


This portion of dot net rocks is brought to you by couchbase a modern multi cloud edge sequel friendly jason document database for building applications with jill dougherty performance and scale if you're new to couchbase and would like to learn more the couchbase developer portal is the best place to start. It's loaded with tutorials videos documentation as well as best practice tips. Quick start guides and community resources including couchbase developer community forum to get started developing on couchbase. Visit couchbase dot com slash dot net. Rocks that's couchbase. Dot com slash. Dot any t. r. o. c. k. s. Welcome back to dot net rocks. This is carl franklin. And this is richard cable and boy. It's been a while since we recorded a show. It's been a few weeks you gonna give you time off. We do a bunch of shows all at once and you get a couple of weeks off. They do a bunch more. I like it. it's march first right now. I mean right now as you're listening to this but as we recording it's march first than night saw. The crocus is coming up poking on today. Saying he here i am here i am. Don't worry life is coming back. life is coming back. Yeah it is pretty pretty rough winter. No two ways about it is pretty cool. This is going to be a great show but before we get the guests and stuff and before better no framework. I have a little announcement to make. I have put out. What i think is going to be a very popular. Get repo. it's called blazer sliders and there's a new package for it as well. It's just what you think but sliders is sort of you think like a slider that goes back and forth. But i'm talking about. I think about a hamburger. But that's especially blazer sliders right. I mean something. I want to order at chili's or burger king sunday but anyway yeah so so. It's multiple panels horizontal and vertical split panels. Right so you have a sprayer in the middle and you can grab the splitter and move it left and right to the panels in the whole thing re sizes when you re size the browser you can make it. Take up the whole screen. It's blazer it's good you can estim- right now. You can only nest a horizontal inside a vertical. But i'm working on multiple nesting. I basically got it working. And now i put it out there. Good yeah now. i'm refactoring it and making it actually more efficient and less verbose and all. Now you get the me notes from people of course our whole product around your library and it's your fault there's and it's only been a couple of days and thirty download. Some not all the eric about it. Give them a couple more weeks to really get the hate on for you right. So here's my advice. If you're using one point zero point one that's gonna change. So but but the good news is that properties are being taken away not add so the you won't need to do so much manual set up it'll automatically discover who the children are and who the parents are in. It'll automatically figure out how to size itself right now. I do i have. I would require a little bit of setup in the parameters but that's going away so anyway enjoy blazers flyers works on s- a server and blazer wasim and Yeah but that's not my better framework. Oh okay will you better play the music then be yeah you better go now. You know you do it. No you fine. You got all right. Well this is actually pretty cool in the guys in one of the guys in Avonex slack room. Scott rowdy found this so microsoft uses this thing called code q. L. it's a semantic code analysis engine. That's part of get hub right okay. They open sourced it so in the end the ideas that you can use code q wealth to make queries to hunt for Soul solar gate activity so essentially. What you can do is you can have this thing. Where your code base and find the militias bits. Oh interesting yeah yeah. It's pretty cool. So i don't understand a whole lot of it but it is mind blowing that you know you can just imagine just checking your code in and then you know get hub says. Hey this is Or notifies you know malicious code so it's It's really cool. Yeah very interesting. And he's becoming a normal part of the pipeline. These days it seems for at softwares also looking for like accidental uses of somebody else's coat right like you don't know right pace things like don't necessarily know where sources come prime so you can trace analysis across the board just like where is this code from That whole dependency chain. Yeah yeah for sure. It's good. But i thought james would like that and Now over to you richard. What's who's talking to us today. Well considering alassio james twenty thirteen. It's very unfair to read like comments from eight nine years ago. We should anyway. Not that i wouldn't do that. But actually i found a really relevant comment for this show from adult compatriot of james's from jerry miller's from show sixteen fifty five. Just show we. Jeremy met back in october. Twenty nineteen talking about the dot net core ecosystem so that whole the what the open source community looked like there and so we talked about a lot of different things in that space. One of which was talking about non relational databases and crossed crafts comment about a year and a half old now where he says. Hey it's always interesting to hear from rockstars like miller but as a show was concentrated towards no sequel noise right. Some of the statements were converts. Wrote not will put me off a bit. Like when richard said why would it make the customer to decompose objects. Just store the object right. Of course. I think a little. Alex fair alex's i followed that with the customer store the object and then break it out into a relational database. A synchronous -ly right. After the customers already moved on the goes on to say though i understand that adding all the caveats the statements would make the show boring as a dvd worshiper inactive. No sequel developer for years. Here's my outcry. There is a serious cost associated with developing and supporting no sequel database for relational databases to Consider at least document type databases which most people mean when they say no sequel the aggregates designed for the most often operation reading data so persisting aggravates is not meant to be quick sensitive volt dated duplication maintaining reference out of the boundaries and so forth and about performance. Hey sequel is performed to the question is what are you trying to do. There are places where no sequel shines and they're also places where databases do just fine. I do like no sequel in much higher level of usage in the enterprise but modern hazy understanding of those equal obstructs adoption of the technology. You the pragmatic way as just another tool in your tool belt team. No sequel as the default option would remove the perception of it being an untouchable shrine. And get more people using it. Yeah he does reference a couple of blog posts including seven reasons not to use no sequel and the book no sequel to still by martin fowler. Which is a little stale. but it's martin fowler. You should read it. Yeah anyway. I thought it was interesting. You know our apps. We weren't firm enough on this whole in. I think the two things work together really. Well there are places where people may sense appraises where people may sense and most places where both will help you zoom time. Yup so alex. Thank you so much for your comment. A copy musical buys on its way to eun. If you'd like a copy of these go by write a comment on the website at dot net rocks dot com or on facebook were published every show and if you sit there and read on the show. I'll send you a copy music. Oh by and definitely follow us on twitter. I met carl franklin. He's at rich campbell. Send us a tweet and you know. Hurry up because times. Wasting not yeah. It's springtime get out there and play in the crocuses all right. So i'd like to introduce reintroduced james kovacs and introduced for the first time rachelle palmer so a long time ago in a galaxy far far away. James kovacs was a frequent guest on dot net rocks. He spoke at conferences wrote magazine articles and had an active twitter account. Then a career change landed him at mongo. Db where he has spent the last five years in self inflicted obscurity. I liked that. Self inflicted obscurity during that time he has worked on the technical support team assisting customers with a core database product and more recently on the driver's team building the language idiomatic drivers that allow developers to connect mongo db his current focus is implementing new features in the mongo db dot net c. Sharp driver which is used by millions of developers worldwide and for the first time on dot net rocks rochelle palmar is currently a senior product manager focused on developer experience at mongo db which includes drivers and integrations with all of manga. Db's officially supported programming languages. Which are c. Sharp dot net python corsi sharp slash dot net python ruby. Php no go swift and java. She's been with mommy. Db's since two thousand thirteen. i mean. Welcome you individually. Welcome james thank you. Welcome back good to be back and welcome thanks mongo. Db james I'll never forget that. I Show that you did with us. Low those many years ago In the absolute perfect description of ifc. And why you it and It just You know it was a great explanation. I consider it the best explanation of those things since before. Or since i appreciate that a lot of my goal of technology is to understand things at a fundamental level and then share it with others. And i'm hoping to bring some of that. Knowledge of non relational databases and mongo. Db two emeals the dot net community. And i think i heard about maga d be in the context of raven db which is i thing and i guess he he wanted to do something similar to mongo but i never really really got into it into raven but i did like the way he was talking about. How index is created on the fly. And all of that stuff and i imagine. It's very much the same in mongo. I end is a smart guy. And he was trying to build a native and non relational engine on windows for the dot net framework and like a lot of the ideas. He took from mongo. Db we are competitive product. Why will fully admit up but with mongo. Db you've got a much larger organization behind cut a lot more features and it's amazing all the things that de mortgage is used. That's been around for well over a decade shall do you remember when we were founded two thousand seven might have been two thousand nine somewhere somewhere around there so we've been around for a while and we've gone through. There's been a lot of different technologies steps along the way an improvement. So that's remember very angie ireland with that canadian. Who's been quiet. so far we were on the Scott net rocks was at scott net rocks. It was the england. Ireland and scotland tour that we did yes and so we were in dublin. And we're heading out to go meet a friend you know for some traditional music and right next door to the hotel was manga devi. But it was just really. It was really like a a soviet kind of experience. There was just the word mongo. Db in a very small font on the door. You remember this. Richard was like. Isn't that kind of unassuming right for this huge as a company to just have this little door with his little logo. We're not yeah. Yeah yeah no. What i'm saying is it was probably a big office but just under understand it at the front door little door. A little sign on a perfectly normal size ashley right. So what's new in mongo. Db are we talking about atlas. Yeah we can talk atlas. We can talk driver right. We can talk core. Server product shown anything dot the dot net itchy has not been the usual constituency for mongo. Right is normally think of mongo in the context of the lamps. No right the lyrics world austin's often we the the actual core server can run on a variety of operating systems including windows including including a variety of other ones mac. Os things like that. A lot of times. People will build applications in whatever their development languages. And that's something that we've seen a real uptake in is that people were developing on windows dot net applications but we're seeing more and more desire to move dot net applications over I'm for the simple reason. Cheaper to run run faster Full stop mythic end of conversation. Yeah yeah a lot of companies are interested in doing this So that's one and the other thing is sometimes people will question. How much is mugabe committed to the microsoft ecosystem. We have a law of customers. Running the microsoft stack that are connecting among d. b. r. c. Sharp driver team is one of our larger grabber teams. We've got five fulltime members on the team right now that are actively developing features The dot net driver itself has been around for over eight years and we are supporting more and more currently going through a rewrite of our link provider to provide better links support. And that's gonna be coming in a future version so we're very much invested in the dot net ecosystem and in microsoft developers. He had the link side of Curing mongo with link is very interesting is better or worse like is very relational oriented so sort of get this. You know there used to be an odbc driver for excel to. It didn't make it a good idea. My the hyun kidneys. Cram that the link query style into through a among database. There's kind of performance again gonna get from it Quite good actually because from a technical standpoint C. sharp with link is just an abstract syntax tree. It's ast and we take that tree. And then we translate it into the equivalent. Mql were mongo db query language. And so it is actually quite efficient and we can get very good performance Is one of my favorite ways of actually accessing mongo db in performing complex queries often doing aggregations group by operations really crunching. Data is a lot easier to link quiry than often hand Handcrafting the queries yourself. 'cause you still have relations. It's just they're just not you know so diabolically specked out. I mean you just figure them out. On the fly build indexes he the the friendly way to say that his structure he added that in air quote. I like diabolic. Better more accurate. Well one of the. That's one thing that allows good to perform really well compared to a relational database. Is that if you think about your typical sequel database. A lot of your relations are parent child relations. Right where one object fully owns another one. But because sequel a sequel and everything structured tables in you got foreign keys you have to break everything apart. So you're spending a lot of Computation resources breaking things apart and then putting them back together whereas with mongo. Db you've got an order which has a shipping address and a bunch of order line items and a discount. You just jam that all into one document and that can be queried as a whole mugabe also has a very advanced query language so that you can say give me all customers who received a twenty five percent discount right. Even though that percent discount is buried inside order document. But if you think about it you don't want that order documenta ever change because that was the order. That is the truth. This was the point i made on. Jerry miller show all that time ago right is like you know the actual reference to the truth is all the things at that moment. Store those things then. Decompose them later for analysis. And i've done sequel applications. That are like that. They need to know the state of what was the address of that particular time. And you have to put in a whole bunch of machinery and always ensure that new addresses are always inserted in your version in them and all of that becomes very complicated and with mongo db. You just store the current address. Here's a really good example. The dot net rocks database is it is a sequel database and we have a guest table and we have a shows table and you know the guests have a photo and description and whatever and a bio right in anytime that That bio changes it changes everywhere that its reference so you go back to show in back from two thousand to. It's got a bio you know. It doesn't have the original by when picture it's all been updated and so you're listening to reading bio and it's the old bio well. Let's look at it this way. James's i show on two thousand eight and his bio on that show now says he works for mongo dooby before. That's right exactly. I know people are going back. What rachel what's your role at mongo. So i moved. Over to products in twenty twenty and i currently own all of our drivers and ap is and then in addition Framework integration so for php. That's larry will You know for ruby. That's rails and arm rails odiham which is mongoloid so a whole lot of stuff is summary. Boss here yeah. It's it's pretty exciting. I mean i joined manga in two thousand thirteen and it was a really different company. Actually not the tom. We didn't make any money wise. And i actually was when i first joined. I was on the cloud team and it was a much different product. Then this was pre atlas and so i actually have the spreadsheet of win. We first started making money. And i basically backed out our own personal credit cards. And we discovered that someone who didn't work at mongo. Db had paid us and we have a little party. So that's how long it's been now. It's been pretty crazy every day since i haven't been to. The story are of becoming a real company and that's without being acquired without going public like just growing up and that's a rarity to simply grow up in via company. I really recommend it if you can. You can handle a stress much preferred to work in at a big enterprise company. Where you're a cog for. Which is what i did. Prior to mongo. Db big big machines. So we've obliquely referenced times. I guess we better tell that story. So when mongo db atlas so among it'd be atlas is our database as a service platform you can use manga deby atlas with any cloud provider. So that's azure. That's gcp that's aws and essentially you. You know create an account. We have a free tier and it's free forever and you create your own little manga database. And you're up and running and you get your connection string and you're off to the races. It's huge at this point I was also obviously. They are when it was first created wing and a prayer. Hell mary type of style. And now you know we have c. Sharp actually is really big on atlas. We have over ten thousand active projects. Which active is somebody has logged in in the last thirty days and done something. So that's it's it's pretty. It's pretty big enough specifically c. Sharp projects right. Yeah overall project is much much. Larger projects is over over a million. At this point i would. I would presume. C. sharp is still very much a minority player in the in the mongo ecosystem. Yeah yeah you'd be surprised You'd be surprised at chuck. It's fun fascinating to me. You know when. I took over this position. I have to say like i was a little nervous about see sharp because it's one of the languages i've never worked with. I didn't know anything about him. Like microsoft yeah links. I never windows never But actually it's been super fun and it's one of my favorite teams to work with the community is just so great like really great And out of the top. You know i think. Out of the top. Ten gaming companies eight of them use mongo db and a lot of them use atlas actually which is pretty exciting so yes that's an area of focus and then of course like a bunch of Enterprise and financial firms uc sharp. Because it's a serious language. Meant for serious work. It's not an obvious thing. In the way that some other languages are well if you look at our commercials support offering which is where we make a good portion of our money. We're companies pay us to assist them and troubleshoot their applications most of these as you can imagine enterprises and therefore they use enterprise languages. Top top tier is java turnout surprising java c. Sharp is actually really close. Second for me. Let's face it. Essentially have similar origins. Right oh exactly statically. Typed object oriented management memory language in development environments like pretty pretty similar they are brothers yep and then pass that is that no j. us and the rest of our languages fall into down below that but yeah the top. Three languages are java. Java sharp and j. us enterprise side jaw job as he sharp and you sort of classic open web dev at davos. It's going to be no apps. A lot of developers whether java warsi sharp liked to play around with no j. on side. So that's what i've seen in enterprises is that they'll be a microsoft shop and sharper their javale shop so that all be java but then they got a bunch of nausea applications and both teams do exactly that they'll play with no gs his. Everybody has to write a little javascript. It's kind of unavoidable. I'm looping back a bit to them on the to atlas here. So if i'm on azure i can i can by this as a service and is also running on azure so you guys are actually operating on the three major cloud providers yet and actually if if you're worried about it or if you're one of the unfortunate souls who survived one of the aws outages in the past few years. You can actually have one of your each of your manga to be nodes on a different cloud so you can run across cloud which i like that well. Any par par somewhat multi cloud mythology to. It's almost like a checklist. But it's i appreciate that. Yeah you've done that. And so i could sink across these infrastructures if i wanted to migrate or or to have a fail over to another cloud i just have it as a guy who builds a lot of fail over solutions over the past decades. This is disturbing. I haven't seen a lot of fielded multi clouds like we can fail from from. Aws azure or. Did you see like that's just doesn't seem to be much of that. Actually lots of people talking about it. A lot of people talk about it but not a lot of people either. Do it or actually need to do it. Yeah we're i think. Multi multi cloud really plays strong. Especially the cto's is the lack of lock in if you're running on azure and microsoft decides to drastically increase as prices. Yeah you can just port your entire solution over to gdp or it'll be just not going to be free like there's gonna be effort involved with at least doable but it's doable. You do not all of a sudden reconsidering rewrite. My entire data layer in order to achieve these things. Yeah so what you're really describing then is prozac for right. That's actually is. It's like. Keep the cto com gonna be fine. We drop a couple of these in his coffee. You'll be all right. I guess it also depends on how far you depends on how far you go right so you can have everything in another cloud ready to go just laying dormant just images right and then turn it on if something goes wrong rather than invest a whole lot of time and effort and money into automatic fail over stuff you know then then it just comes down to up. Somebody gets a phone call or an alarm and they have to do it really quickly. I still think it would be days like it's just not that trivial wanted to shift that stuff across and and hopefully you're using the m.'s. And and containers and things that are relatively portable. You could go across deeply. Invested in any of the different vendors. Service technologies should've their distinctive analogy. There's going to be hard to move. Yeah that's true well and that's something that when you're building a cloud based application you have to consider a cloud vendors do provide a wide variety of very convenient services but they lock you into their platform. So how locked in are you willing to be. How much risk are you willing to tolerate being locked into. Aws or another platform. And how much do you want the flexibility to move services around. I like Azures coober nettie service because you can use your own docker containers and everything so as long as you have your containers some place ready to go. You could do your own cuban netease in and in gcp or in In an amazon if you need to But while you're on asher you can enjoy all the the wonderful high level stuff of ak s as a sort of separate thing. I think one of the other benefits of atlas is that secure by default. You get less. Esa sale automatically default. You don't have to set up anything which is really nice because you know. That's that's hard you've also got a variety of secur security solutions around like you can log in with a username password which is properly hashed salted over the net. Like you. don't wanna pass. Anything clear taxed You can use l. doubt you can use a api keys. There's a variety of ways of actually authenticating with the atlas service as well as private networks. So that you can isolate your look if you have a azure deployed infrastructure. You can make sure that it goes through private links so you only going across azure controlled. Interfaces you're not ever going out onto the public. Bob that reminds me. I want you to tell people about f. l. e. while we're here before we dive into that. Let's take break for this very important message. If you've had automating your. Espn at deployments. On your to do list. Now's a great time to give octopus. Deploy a try the starter edition. Lets you install octopus on your own infrastructure in deployed i s web servers azure websites and pretty much anything from no d. Coober netease and they just made it free for small teams. Give your team a single place to release. Deploy and operate software with octopus. Deploy find out. More at octopus dot com. This episode of dot net rocks is brought to you by every plate. America's best value meal kit delivery service. Not only is the food fresh and amazing but each meal costs about as much as a cup of coffee. Recipes come together in about thirty minutes definitely faster than a trip to the grocery store and starting a meal from scratch. Every plate gives you easy to follow recipe cards and pre portioned ingredients. C-can spend less time prepping and cooking and more time enjoying good food with family or loved ones. I wanted to see if every plate was as good as it was cost effective and after subscribing and cooking a few awesome meals. I'm convinced that you can get the same deliciousness at a much lower price so experience fuller plates and fat wallet. Try every plate for just one ninety nine per meal plus an additional twenty percent off your next two boxes by going to every play dot com and entering code dot net one ninety nine. That's right with every play for just one ninety nine per meal plus an additional twenty percent off your next two boxes. that's a one hundred dollar value. Go to every plate dot com now and enter code dot net one ninety nine. That's dot net one ninety nine and we're back. It's dot net rocks. I'm richard campbell. That's my friend. Carl friedman and we're talking to james kovacs and rachelle palmer and talk a little mongo. Db all the cool offerings going on there. And james i just interrupt you talking about l. f. e. a. Plant side field level encryption. Yes f. l. e. so what what that is. It's a recent feature that was added to mongo db and it allows you to do your encryption on the client side so you actually acquire encryption keys from an encryption key provider either on azure we now support azure and a and aws. I think we ought as well recently. So any of the major cloud providers they provide a key containers so you can request keys and those are ever only ever seen on the client side. So any data that you read has been encrypted with your key so the server the monkeys can't read it. It's just no pick blob a bits once it gets to the client. It's it can be automatically decrypted so it looks like clear text. So if you're storing hassle you shouldn't be showing say passwords sleep that topic out up. Ppi right like or p. I personally identifiable information which sometimes people want to store. So if you're storing credit card details once again need you really need to But things like social security numbers. Which often you do need to have them stored in your database. But you don't want your cloud provider necessarily to have access to the us. Side field encryption to actually encrypt these fields and ensure that and if you read them it just looks like encrypted bits but if you have the decryption key then all of a sudden it's readable unusable and is your ability to share those keys between devices so that multiple devices could decrypt it even though the store cannot yes So you actually communicate with the the jerky store aka s and if your client is typically do it on a server but if that server has been allowed access to a ks what y'all control take aka infrastructure than that client would be able to decrypt encrypt data to and from that is coming from manga. Db but if it didn't have those encryption keys for instance if it was reporting app that could only report on high level patient information but none of the particulars than you wouldn't grant it access and it would not be able to read those fields but you can go very granular. You can go okay. I want these three fields encrypted. But i don't want these other twelve encrypted at all so you can it not like you have to encrypt the entire object. You can if you want to. But typically Field by field. What sensitive rights. He could protecting specific data. But we we battled this problem with encryption on sequel server to where he was easy to grip the whole thing. But then you you know you just crippled any kind of querying because you had decreased everything to figure out any of that right. Not just a just encrypting sensitive information you just the salary information. Just the identifiable but the index the index stuff important stuff. We've still legible well while the other interesting things that we play with is you. When you're designing your encryption scheme what you can do is you can say. Let's say you're encrypting the salary information. You can encrypt it all with the same key. So they're always encrypt to the same value. You don't know what that value is but it's always the same value right and that allows you to still query it you can't do range based queries but you can do give me everybody who earns fifty thousand dollars stink because you can trip that fifty thousand dollars to a known value and then query the database saying everybody with is known value. I not know what that value is. But i know it's the same across these records exactly exactly so. There's a lot of fun things you can do with that from an application designs chant white insecurity standpoint. Yeah if you if you think hard about it but this whole encrypted before it leaves the device is pretty compelling for a lot of folks right. i don't. i'm not transmitting. It counting on some kind of encryption during transmission and then encrypting it again as it goes to rest on the in the central store and crooked it. Before i left. That's right y'all got for that and that's that's the client side encryption. Then we also do a lot of when you're on asher the bits that actually get out to the actual volumes are encrypted as well so we do on disconnection as well right to ensure that is secure while you guys mungo went through the security ride. I did a run as back in twenty seventeen about the mongo exploit which is like they. Hey you know what would be careful with. Defaults defaults are hard. Yes that it's not. I think it's also you know your products important when it makes but it becomes a part of an exploit like that like and now you have to think about your product differently too. And that was an interesting growing pain going from focusing on startups to an enterprise company off because as a startup. We want people to just be able to spin up a mongo d on your laptop. Not don't have to worry about authenticating anything you just connect to a certain port and you're off to the races so we made it really easy for developers to get mongo. Db setup running. Yeah well that was the problem and we had we had. We had all of our documents at the time. I was working with technical support and we had all the documentation in how secure it a. Nobody bothered insist. The that's the reality because the default was unsecured guy. You kind of have to force them to do the right thing. Even though it impairs their ability to get started yet by the other the atlas is another solution of that is like hey you developed locally. You ready to go into production. Why don't you use are secure one in the cloud and no worry about scaling at an all. Those other things like life will be easier this way. And you're far more likely to stay safe exactly because when you even when you spend up an zero cluster. Which is our lowest free tier always free that is secured not less. It has password protection automatically turned on everything secure by default in atlas as you can try out. I actually spent last week building applications with my team just like let's build a senior application and see what it's like from a user perspective and we just spun up. An atlas database put some data in there and were able to securely connect and started reading a phone app actually right on the device and connecting to atlas so that was a lot of fun but yeah it was secure by default of the box and as you start to scale. That's the other nice thing about atlas is you're going you say okay. Now i need more horsepower behind it this. I've developed the v. next plants versus zombies game. Let's start scaling this now we can. It's literally a click of the button in the u. Is like okay. I need an m. thirty. Which is our dedicated tier. Okay i need something with even bigger up to fifty and one hundred and what we are doing on the back end is basically scaling up the vm's at the running and providing more resources to the back end. She something somebody could do. Your ops team could be scaling mongo themselves. But did they want to do they wanna learn how they this thing and there are definitely people who do that. We do have an in prescription where you can. You can also just do it on community if you want right a lot of your own tooling you can download community for free and start building at your own mogadishu infrastructure. But is that what you want specialized right. Do you get a lot of customer requests asking you to move their relational databases to mongo some some. Yeah i mean that. Is that a common thing. And that's a that's an area that it's an area that i know. Our consulting team does log working obsessive. One of the nice things about maggie is. There's a support team. There's a conservative team. There's an engineer that we got all these different teams can focus on different things. There is a desire to move relational workloads over tomago db depending on the application. Sometimes that can be a good thing and sometimes it is more work than is really warranted. Yeah i don't find that lift and shift is super popular. I think what tends to happen. Is that they sort of sidecar the to databases for a while. And then they'll move workload over but it tends to be that they re architect in the process. Which is why. It's usually a very big projects like let's download all the data and then just put it in mongo. It's it's more like a even considering lifting and shifting. Because you have relational data that now has to be documented. Alexa therefore does need to be reacted absolutely but do. Do you think that if you were going to make the case somebody who's got a hefty sequel azure bill every month. Because they're you know because it's expensive. Let's face it and you know. Maybe it turns out that the shape of their data with lend itself really well to a document database. What's the case. Is it a money in performance. Both make the case there. I think it's both one of the solutions that is is pretty recent. I can't remember what year we debuted our data lake project but it's basically You know you can query your data in s. three bucket and then the results of the query you can put into a manga debbie database. So that's how some companies shift there content or their their data basically to mongo to be atlas which is a pretty nice way to do it because then it's like i'm only moving the data that i need just kind of nice i worked in consulting four gb. I think that you know it's one of those projects that we would love to bid on as a consultancy firm because it will take multiple years. No cost you millions of dollars and then you may not actually get any your net. Your net money gained loss. Might not really be what you were sold. I think well you look at the price of an enterprise license of sequel server today Like it's it's far more than the hardware. The hardware price keeps going down. The the licensed price keeps going up. Right what a sixteen thousand core right now for enterprise sequel server like. It's no fooling like the you could buy a lotta sequel azure for that and i suspect you buy awful lot of atlas for two. Oh that's true totally. I mean that's one of the reasons to move atlas also it's a consumption based model so if you are gonna run on thirties then you which is a sort of a smaller instance is like and then you scale up when you need to or during specific periods of the year. Like if you're a retailer for black friday you'll skill up and then you'll skill back down around valentine's day and it's fine. Yeah i've tried doing that with hardware. It's really tough to take those rams don't do it's not a good idea just leave. Leave those michelle. You don't have hot swap. Richard you pull hard enough everyday talks. That's why yeah. But if you know to that point. This alaska this sort of op ex capital choices like instead of investing in that hardware. And i'm saying this on the it side. These contracts are expiring these hardware. This harbors coming out of warranty. You're looking at in some cases to common numbers worth of equipment in licenses and so forth and you can get an awful lot of variable cost resources for that money. Like i just put that in the bank pay it out monthly and get pretty comparable results and ultimately if i careful and do some tuning and like you said dial down dial up. Those times are can spend less now. Nothing's for free like there's effort involved but by there's nothing easy about buying and setting up new gear either like instead pilot or the other thing that's true and like i'm sure that my our sales team would literally murder me if they hear this podcast but you can also in efforts to developer friendly. We don't have long term contracts for support so if you're using atlas you can purchase a poor because you have a problem or you want a question answered and then you can. Of course cancel your support. So you don't get locked in to this sort of multi year Thousands and thousands of dollars type of deal which is really cool. Yeah and it depends on your organization as to how that works. I also know companies where it's like you don't spend this money you lose it from your budget so you know i've been on the vendor side of that where somebody's calling me saying. Listen i got to this month. What can you sell me once you guys. Give me some kind of contract that shows. I'll get value from you over the next year. I just need this money out of the my account. This you are my friends. I will make something special. i guarantee you. Yeah well and actually the reason we first started doing that is because what would happen. Customers would have some kind of incident or some kind of problem and they would want. I want to support. I don't care how much it cost. i'll pay it right now and we're like no well we'll have to send you to procurement and then you have to sign some papers and they're like can i. Just give you a critical. No you need to start talking now however much money you want to on my problems and also take my money. Yeah yes well again. This is back to the prozac statement. It's like i've been that contract where it's like you pay a retainer to me so that i say there it'll be okay and then go make it okay. Well hearkening back to an earlier comment was. There's a lot of place that the cloud really excels is elastic. Compute resources where you can scale up and scale down. There's a quite a number of business. Not all businesses. But there's a lot of businesses that are very cyclical in nature either seasonal Where black friday. Rush or a i worked with a particular customer who shall remain nameless. Wonderful folks to work with But they they're peaks were around sporting events and every time there was a super bowl or an mc w aa tournament or some big sporting event have a huge spike in traffic. And then it would die down to almost nothing and so with. They were actually on atlas in what they do. Is they scale. Up to very large instances a or two before the big event. Host the big event and yeah. They'd be spending a lot of money for those few days but they've also got a lot of income coming in because of those events and then once those events are done they would scale back down to minimal instances to keep the background traffic going. And there's a number of businesses gaming companies. Do this where they have like. Blizzard has a big launch. And all of a sudden. You're gonna have a lot of traffic hammered. Now we can scale up and then once interest dies down then you scale back down and that flexibility that you're not locked into physical hardware and having to pay too because we're basically with physical hardware you paying for your peak. Now you have to the po provision the pizza and then you've got that peak provision hardware for the other three hundred and sixty four days that you don't need yeah that's absolutely true and certainly this is the new era right. The utility computers that we can buy what we need when we need it and in reasonably short amount of time i mean. How long does it take to move to a higher instance and On atlas minutes. Yeah there's a lot of a we do a lot of Interesting things in the back end. Most of the cloud providers will allow you to tweak their hardware once every six hours right so if you just need a like a very comic database we live and die by aiops guy ops. So if you want to bump up from like a thousand to ten thousand aiops that'll take a few minutes. Yeah i still think if. I might be doing ops coming into black friday. It's like on wednesday. I turn up the nah right and then sort of poke things like everybody. Happy with the big instance and we all good. We don't want to wait until noon on friday for another thing that we were experimenting with Experiment because it's actually in production is auto scale you can turn on auto scaling on your allison. Is that if you see a certain peak load for an extended period of time it will auto scale up to the next instant size. What's the thing you're measuring is like number of transactions are is it i opposite you're measuring ago. This view. Bombardier instance right now. A cpu guys. We're are pinned. Yeah you're processor is pin. Pretty hard bottleneck for mound. Go like i think disk latency zero bottleneck remember. I think you can also do it on. Aiops sedans on your workload if you're doing a very heavy workload. Obviously disguise ops is going to be your primary driving factor if you are doing heavy duty. Educations of where you're doing grouping expressions that's where you can run into. Cbs resource end. Seaview run. We almost never see network run. Hot is typically depending on your workload. It can query heavy and using complex queries like if you're just searching by essentially primary key are underscore field than that takes. Virtually no seaview resort sources resources. But if you're doing a lot of sorting in aggregation in server memory than that can take up cebu you can actually have atlas auto scale so that it will bump up to the next tier up to a maximum the you set and then if it remains if you are below a certain value for typical previously days. I think we're we're getting a bit more aggressive. It can be on the order of ours if you see like your views really running low. You're not using law guy. Oh then you'll drop down your tears again down to a certain minimum set point as it were very much experimenting with this for customers to optimize their costs on atlas. If you could get into daily with is like if you're a retail outlet that's streaming data from transactions but like ten hour window where every store is close like being able to turn that knob all the way down for what is better more than a third of the day or add up over share. Yeah for sure. What about the long term storage side of things is archived. Like i find that. That's an excellent. Sequel azure and these kinds of products as datasets. Get big. they actually get really expensive. And you kinda wanna carve off old data and put it away right so where how. What is the cheapest storage out there in the world today. Blob block store. Yeah whether it be s. Three or azure block store. That is by far the cheapest per gigabyte that you can get so one of the things that we enable is atlas data lake where you can actually pull off your old archive data into an s. Three blob store or a azra blob store and the nice thing about that is although your performance isn't great it's to query -able right so it's still there you know. Is it going to be slower. Because we don't have the full indexes we actually have you can issue mongo db queries against this blob store. Which is quite cool. Yeah but it's it's still there and available for reporting purposes but you're not in incurring the cost of having it hot an available immediately. I should mention the azure sequel does make backups. And so if you screw something up you can just go online into the portal. Find the last backup and restore that to another sequel database. And you're off to the races. That is a very nice feature. But you know you you pay for it. Yep we have automatic backups as well an in atlas so you can actually you can establish a backup schedule a. The backups are snap shotted into asimov's door or orange s three and then you can pick a snapshot and then restore it either to your own. Cluster to cluster. So you have similar functionality. So that you can because you live in companies live and die by the data. Yep and you talk about keeping. Cto's com lose weight like that's bad that'd be cto's get upset win. But so having visibility into data that's old is great but releasing the cost of storing all. Data's because the traditional solution is just delete data. Right or i built archive systems. Auto remote stores is archive it onto tapes yet but having it all built into the platform and we are investing in this as well so to make it easier to right now last. I looked You could either query the data that's in atlas or the the data and data lake. We're one of our goals is to provide a holistic view of it so it will look like the same collection but data that is past a certain year at certain point old will actually exist in s. three and be queried for automate that shuffling i can run into toads Over do do a batch. Once a week this sorta hunts down the oldest stuff in pushes over that street. But i'd rather more me than it's your fault. The other real selling point. I think is that there is automatic upgrades of your manga debbie server version. Which you don't have to do that anymore. Right break right but yeah it's nice that it's you know upgrading. Gnarly takes effort. But you always doing it wrong. At least if if the the vendor is doing it they're probably doing it right and if they're doing around they're upset rags up people at once and hopefully no breaking changes of version to version. Yeah we have a process where you know for the entire platform like said there is over a million active projects so we don't roll it out to all one million at the same time we we basically roll it out to a randomized ten percent and wait and then another twenty percent and wait and then another twenty percent and wait and then the remainder so if we have to. We can rollback of version upgrade. If we see that there are any problems with the i. Ten percent and related to that is not only mogadishu upgrades but os upgrades. You have to run your server software out on us and it's always a hassle. There's a new windows version. There's a new lennox patch comes out and going through the rigmarole of having to upgrade all of your servers. That once again taken care of. Portia not you guys mentioned if free skew atlas what are you get. Oh i don't know we saw. James will have to look up the stats. It seems like a lot. There are a lot of people using it and not just for a tinkering and not for learning mongo db but there are actually people who are running production level apps in our free tier which is kind of fascinating. We did a survey we sort of assume that most people who were Running applications on our free tier sort of learners or students or beginners with mongo. Db but that didn't turn out to be true based on the responses to the survey so yes so you can start out at five hundred twelve megabytes storage for free. And i think there's also a limit on connections and a limit on aiops. But i don't recall what they are sure. So if you have your your stuff out there your clusters or your your shared. Cluster and In turns out lo and behold everybody wants to be on it you know and it becomes popular. You can sort of migrate that up. I suppose that's great. It's also i mean. I use the free tier buy-sell for you know some one of the things i did. This year was analyzing stack overflow developer survey data. So i just put it in mongo. Db and then did some charts so that could look like a cross sections of developer communities. That was cool for that. And then last year i did a health little app which was mobile in a web app and they used our free tier for that and then used it for a talk me world so it's nice to get something up quick quick and running. I think when it comes to my health. I'd want to impact other than but it's also especially with demos it's like i don't like it in the bill at the end of the month because i forgot to turn the flipping off not that. That's this is all good stuff guys very very good. What's next what's next for you james offer me. We're working so. I've been working on the sharp driver and we of new features coming one of the features as coming in the next few days is were introducing. Better lennox support. So if you've got a dot net core apps running on lenox We're going to be adding the ability to feel bubble encryption. Cfl and the fear that i implemented was actually kurosawa. Nice so we're actually talking to the gp shared library on lenox and doing all the funding perot's ticketing stuff and that was believe me. I understood. for bruce before because i knew tickets i new server tickets a new Your ticket to all that actually. Having to implement the bits on the wire is a total total level of understanding. How you can see sharpe seven. These days We actually recently upgraded to c sharp nine po. And we can we do this. Hey we can if if we use a c. Sharp nine feature that requires compiler Libraries sport than we get under myler so we can use all of the new cool features like record types and others that come with c. Sharp nine and eight at did. I did mean to say nine seven. Oh okay moving in the past. But ultimately i know there's a lot of lot of really nice language features that allow you run much clearer code in c sharp and just some of the things that we're used to from functional languages are definitely coming over to see sharp you were. So what's what's in your sharp team I really you know. I've mentioned the gaming before. I started digging in on that and so have asked some of our developer advocates and developer relations teams to do Unity tutorials for c. Sharp from from beginning for beginners so build off entire game in ninety minutes so they've started doing youtube Live streaming of building a game in. It's sort of paired programming exercise. I think it's going to be really cool And then the other thing that. I'm gonna probably ask the team to do is I keep an eye on whenever we have sort of a little swell of users asking for certain things or complaining about thing and so probably going to ask the team to do serialisation improvements ca notice. There's a few tickets that come in faster and make it better. Yes the there's always going to be those. Yeah never fast enough those thing but it gets you know you're the pm role you could see the this hit a critical mass where it's like. Better look at this as something narrada to prove well in the serialisation movements are interesting because we actually have unlike some of the other drivers in one of the I was interested in joining the c. Sharp team is that our serialisation infrastructure includes complete. Odium object document map. So it's you don't just have to work with documents you could actually say. I've got a customer i've gotten order. Are you actual pocos then automatically get translated into the equivalent document structures by the drive itself. So you can get your business layer abstraction level and what the driver worry. That is really cool james. Glad you're working on. That's very cool. Well james kovacs and shell palmer for spending this hour so much for having us anytime pleasure seeing you too. And we'll talk to you next time. Dot net rocks dot. Net rocks is brought to you by. Franklin's net and produced by plop studios a full service audio video and post production facility located physically in new london connecticut and of course the cloud online at p. w. o. p. dot com visit our website at dot n. t. c. k. Ks dot com for rss feeds downloads. Mobile apps comments and access to the full archives. Going back to show number one reported in september two thousand to make sure you check out our sponsors. They keep us in business. Now go write some code. Cnx time law.

Coming up next