The Good Parts of AWS with Daniel Vassallo

Automatic TRANSCRIPT

Can you just go a little bit deeper on why architecturally Dynamo DB is not well equipped to fulfill the same semantics of a sequel database. Oh, it was designed to dissuade I. Don't know if you know do the member simple to be what it used to be. A the dissenter of Dynamo DB back in I think it was launched in two thousand, ten nine. And it's. It's a significantly more ambitious than Donald. Debates was meant to be slow inequity. Relations was more like you know like Mongo documents based you documents so aquarians essentially the answer. And this is actually it's probably one of the few. I can't think of any any other service. One of the few implicated services from aws. It's technically supported of your salon and gets. You send using simple debate. API still work, but basically almost hit it under the carpet side. You don't find any and there were. You won't find us in the console. It's not it's not a new. The send things like that. And the problem was that. Amazon founded super hard to make this type of database Kale and to have predictable performance guarantees, one of the biggest problems that was happening. That would simply be. You might throw in some complex square. You might not have an index about it. And the declare would take two minutes, timeouts and lay. It was completely unpredictable. Some quays take two hundred milliseconds. Some will take minutes. And it was very high. On the service side to these about site to locate the sources so talking about capacity, so not the be was the answer to that and TWAS, radically different perspective like completely predictable versus completely unpredictable, so there's two operations gets put the listed going to bt index behind the scenes and updating single lighten very predictable. They all take. Just, a few single digits milliseconds identify typically, and there's this query API, which again just goes to the starting point of a beat the sequence of cards after megabyte so again like the the the upper bound per addicts, how much expensive Dakota can be ends attested dissolved, and that set of to to continue to participate you. You go with the next token. megabyte that I, so it became very easy. For the service provider to these about how expensive it can be how fast it can be how to allocate resources, and it became huge success successful, but numbers on itself, because I remember we used to on services on top of relational database address to have the same problem so sometimes the relational database at an all. It's a complex machine is my star choosing? It's it's my start to use suboptimal query, and suddenly acquitted that used to take a second is now taking twenty seconds and suddenly using all the memory. When we started thinking in terms of much more primitive technology, like beatings become easier to these in about as long as you managed to model your queries and what you needed to do. To its limitations. But then. Today's were you're fighting your database. Because suddenly spiking two hundred percents, appeal and everything is slowing down disappear so that that element of predictability is highly highly available, so they were defer the. It was designed to be this way that I just wasn't. Designed to be so inequity, of. Arbitrary complexity and will give you the answer. What do people do when they have built their infrastructure around Dynamo DB and it's not fulfilling the requirements that they have. I think you will struggle the limitations and up subsidizing you I. I think the problem is up become became being discovered very early in development. Like for example, if you're expecting to be doing lots of recommendations on amounts of data, doing development you to realize that you're going to be downloading everything out of Dynamo and doing it locally, not so. Hopefully yearly allies airily that this is worth considering Golden, considering that they should use another type of database or relational, database or something. I don't have any first hand experience for example where delimitations and and adopt sort of surprising later, which is I think is a good thing again like the fact that it's significant distinctive. Had few. It's very hard to. Abuse it sight and sort of expect more out of your life. You realize immediately that these are the limits. which again I think these tend to be sometimes that. And more sophisticated database aside because during development your. Attention like hundred minutes seconds, and then once you have lots of data or things are in Qatar. They start to become more unpredictable. Dynamo just elements that issue just just there's no unpredictability. It's actually incredibly predictable at the cost of the constraints the comes with. You right in some detail about s three and s three I think of for obvious use cases as slow file system. It's BLOB. Storage it static website hosting its data lake. Told me about the other applications of s three. Yes. Yes, I. Think One of the lists. Values of trees that you can think of as having infinite Benguet for all. Practical purposes that so, if you have terabytes of data, you could basically an estimate. You could download it as fast as you as you want to. Basically always many to that says he wanted many servers. You want to tell you can chunk it up in pieces and just download the terabyte like in a second. For example one of my biggest project Thomason was launching and working gone. Cloud Watch watchdogs incites, which is basically a monitoring tool that allows you to arbiter the complex queries against your log data. And much entirely built on top of the and this is it surprises? People because this unlike Donald to be, we actually chose to support. Give me an arbiter equality of complexity, including regular expressions and things that are super cost to evaluate and. To dissolve, and we built it literally on top of us today and in in a very cost effective way because we relies on the assumption that. For example log data tends to be. Very big generalized especially nowadays like application censored. Tonight's like gigabytes and terabytes of logs. You want to start them somewhere where it's cheap and us these the perfect place for that and you tend to Kuwait infrequently, though when there's a problem I want to something about your application. And I think one of the ideas that works with s threes, this technique where you separate compute from the data so basically once. Does no question. There's no compute so basically you can just have the data sitting in streeter, just paying the to censor gigabytes per month, and there's no other costs. And if you open up the consulate insights and you do Equa they. Spin up some. Is it Winston while I mean behind the scenes like some pool of warmest. But fundamentally you can think of about it does like spin some ephemeral instances and we enough such that we can download data. As they wanted to. And then you can sort of just turn over the data very quickly I just because. I can listen to such as your network

Coming up next