19 Episode results for "martin fowler"

160: DRY, WET, DAMP, AHA, and removing duplication from production code and test code

Test & Code : Python Testing

14:40 min | 2 weeks ago

160: DRY, WET, DAMP, AHA, and removing duplication from production code and test code

"Should your code be dry or damp or something completely different. How about your tax code to different rules apply but what do all these acronyms me. We'll get to all of these definitions and talked about how it applies to a production code and test code in this episode contestant discussion about dry dam in various other clever acronyms Should start with some definitions in a little history. Let's start with dry or don't repeat yourself. I i ran across dry during my first reading of the pragmatic programmer. Long ago. don't repeat yourself. The dry principle is every piece of knowledge must have a single unambiguous authoritative representation within a system. That sounds neat. A good summary from competed says when the dry principles applied successfully amount of -cation of any single element of a system does not require a change to other logically unrelated elements additionally elements that are logically related all change. Predictably and uniformly enter thus kept in sync besides using methods and subroutine in their code. Dave thomas and andy hunt. The authors of pragmatic programmer rely on co generators automatic build systems in scripting languages to observe the dry principle across layers by. That sounds great but time. It's not always easy temperament to end. It can go haywire if applied inappropriately. Let's look a little bit more in the same wikipedia article It also includes some funny description of Wet w et which could stem for write everything twice right. Every time we enjoy typing and waste everyone's time but really enjoy. We enjoy typing one. That's awesome derek. Also talks about a hough from Dodds and being a big fan of take me on as a kid. I definitely wanted to know. More ha is not part of the band a ha. It stands for avoid hasty abstractions Described by dodds as optimizing change. I n avoiding premature optimization end was influenced by sandy meta's preferred duplication over wrong abstractions. Okay scenario now. We're getting maybe at some people that don't think All the time now we're also getting it Some of the misapplication of dry. A ha- programming assumes that both wet and dry solutions inevitably create software that is rigid and difficult to maintain. It's not good instead of starting with Within abstraction or abstracting specific number of duplications software can be more flexible or robust Abstraction is done winning his needed or when the duplication itself has become the barrier and it is known how the abstraction needs to function not sure. I followed all of that but we definitely need to pull on this thread a little bit more to find out more but first this episode of testing code is brought to you by dog the monitoring insecurity platform for cloud scale applications data dog. Apm lets you automatically track the impact of every code deploy on your applications performance in real time leverage out of the box comparison graphs visualize requests errors. And latency down to specific. End points to decide whether you should ship a hot fix or rollback. Your code data dog. Also ties your code deployments to the underlying infrastructure in code profiles without clicking a button start monitoring your canary blue green and shadow deploys with a free trial at testing code dot com slash data dot Dog will send you a free t shirt. Okay we've defined dry wet and hot haha in it's not the take on me a high Avoid hasty abstractions end. We have some hints to at where the trouble with that. There might be trouble dry paradise. I'd like to bring up a some examples. I've seen from experience. A lot of people have heard of dry but not really understood it deeply. They remember the. Don't repeat yourself thing so weird things happen An example is in order. let's have two functions each with say thirty five and forty lines of code and they share some of that code with each other. let's say they share the first thirty lines of code and differ only in the last five to ten lines of code. Wouldn't you be following the dry principle by grabbing those first thirty lines sticking them in a separate function in calling that function from each of the original functions. Well that's not it's not that cut and dry get it dry anyway. It avoids duplication is about is more than just about avoiding duplication. It's also about abstractions in about how the code behaves when a changes needed. Yes exactly you might say if a change is needed now. I only have to change the code in one place not to rate but what if the change is only appropriate for one of the client functions. If you don't know that then you have a bug in the other function. If you do that then maybe he'll copy that shared function into a second function in. Change one of them and now you have two functions each calling their own separate helper function. You now have a deeper called tree Actually haven't saved anything however the first thirty lines really do representative an identical abstraction then you'd probably be correct in creating the helper function probably though maybe it software future requirements are to predict and also what it's not thirty lines if they share one line of code obviously it'd be silly to create one line helper function in replace it with one line calling the helper function. You haven't really saved anything and you hidden away implementation. I've seen this though and still have bad dreams about it okay. Thirty lines may be created helper function with that. But one line. Don't but where's the cutoff. Is two lines is a five is ten lines. i dunno. That's where abstractions in making sure. Your code is subjectively. Easy to read is important by the way you're a bad judge of whether or not your code is easy to read. So am i a bad judge of my own code. That's why Good let's look at a couple more. Demonic one is about the duplication thing. It's called the rule of three. I've also heard it as three strikes near out. And it's roughly two instances of similar code. Do not require Refactoring but when similar code is used three times it should be. It should be extracted into a new procedure. Martin fowler likes this role and or at least did And burn up in his refactoring book. And i'm i'm on the fence about it actually. It's relatively easy to apply when it's your code and you just wrote the other two duplications recently. It's harder to apply when it's a shared cut base and the duplications have been put in place at various times any might not even know they exist but it's important it is important as part of refactoring because this cut paste modify type of coating is very common in software and it's You know cut base modify is not considered doing architecture. So if you see a lot of duplication it's time to start a refactoring effort i think but maintainers need to be on board in the nba involved in the code reviews so they can understand the abstractions that you're putting in place any abstractions need to make sense okay. One more thing damp damp is descriptive in meaningful phrases in his about promoting readability that. Now we're getting somewhere And i learned about damp from his dock overflow discussion about applying dot ryan damp to testing. I'll read just section from chris edwards because it's great so far to maintain code you. I need to understand the code tender standard. You have to read it. Consider for a moment. How much time you spend reading code. It's a lot damp increases maintaining -bility by reducing the time necessary to read and understand. The code sounds good. Removing duplication ensures that every concept system has a single authoritative representation in the code. A change to business concept results in a single change to that to the code dry attempts to maintain maintain attempts maintainability by isolating change to only those parts of the system. That must change also sounds good. So why duplication more acceptable in tests. This is still part of the tests often contain inherent duplication because they are testing the same thing over and over again only with slightly different input values or setup code not following now however unlike duplication code unlike production code. This duplication is usually the isolated only to this narrow within a single test fixture file Because of this Has minimal an obvious which means it posted. The poses less risk to the project than other types of duplication again continue reading in a comeback. Furthermore removing this kind of duplication reduces the readability of tests the details that are previously duplicated in each tester now hidden away in some new method or class. Yeah that's bad to get the full picture of the test. You now have to mentally put all these pieces back together. Therefore since test code duplication often carries less risk and promotes readability. It's easy to see how it is considered acceptable as principal favorite dry in production code favored dantesque code. While both are equally important with a little wisdom. You can tip the balance in your favor into quote. And i also had some interjections in there but Some little more on the fence. He had me for a while. But then it kinda fell over a bit. I remember hearing kent beck once. Say that tests should be attest. Should tell story. I like that and i think it should be at the same abstraction but shouldn't your code in the functions in it tells stories also. I'm i guess. I'll just say that. How ideal at this. I don't really have good advice. I can tell you back. And tell you how i deal with. I do know that a long time ago tried to make I started to try to make tests as readable as possible. That's michael if there's duplication parameters ation fixtures in helper functions are great to clean that duplication up but only if you can still read the test quickly in understand it. I also wanted to make sure that anything that can fail. Generates in easy to understand failure. 'cause that's kind of a side thing from what we're talking about now but that's harder than it sounds. I also know that. I started when i started to do this for tests. Make sure my tests told the story. At a consistent level of abstraction i started wanting my production code to be like that too. So now i don't really have to rules for production code test code. I want it all to be readable. Another thing is that. I'm not that great at predicting. What kind of changes are needed in the future So my defense against all of this is A few things. I keep in mind dry. it's something to keep in mind. Even if i don't follow it all the time i make sure. My functions till story at a consistent level of abstraction without hiding important detail a voted avoid duplication when it makes sense doesn't detract from readability. I test everything everything gets tested. And don't be zealot about dry or really anything. I work with experienced engineers from all sorts of backgrounds in learning paths and some of Some people have read about all this stuff in some people have not I try to not be on my soapbox in. I learned from my fellow employees in try to have them learn for me. don't prematurely clump duplication into helper functions. Sometimes i need to write the same thing five times before i really understand what the abstraction is end in understand which of those Things a abstractions symptoms. It's too so sometimes. I have to write it several times before. I'm ready to factors big after. I really understand it and also don't be afraid to re factor other people's code. Your tests will help you to make sure that you don't make things up any version. Control system allows you to roll things back if you accidentally make these worse or possibly make it less readable. You'll know that will from code reviews anyway also have fun. This is software. Don't get too serious about it. Thank you patriot. Supporters join them at testing co dot com slash support. Thank you to dog for sponsoring get started with a free trial at testing dot com slash data. Dog in dog will send you a free t shirt. Those links are in the show notes ab testing code dot com slash. One six zero hundred sixty. That's all for now Something in may.

sandy meta andy hunt Dave thomas Dodds dodds ryan damp bility derek Martin fowler chris edwards nba kent beck michael
#275 Beautiful Pythonic Refactorings

Talk Python To Me

55:04 min | 1 year ago

#275 Beautiful Pythonic Refactorings

"Do you obsess about writing your code just the right way before you get started maybe some ugly code on your hands and you need to make it better. Either way re factoring could be your ticket to happier days. On this episode, we'll walk through a powerful example of iterative Lee re factoring some code until we eventually turn our ugly duckling into a python ick beauty on our hope is our guest in this episode to talk us through re factoring some web scraping Python Code. This is Talk Python Emmy Episode Two, hundred, seventy, five, recorded July, ninth, two, thousand, twenty. Nine. Welcome to talk by a weekly podcast on python the language, the libraries, the ecosystem in the personalities. This is your host Michael Kennedy. Follow me on twitter where I'm at in Kennedy, keep up with the show and listen to past episodes at Talk Python FM and all the show on twitter via at talked by on this episode is brought to you by us over at Talk Python. Training. Pythons acing imperilled program and support is highly underrated. Have you shied away from the amazing new eysenck keywords because you've heard, it's way too complicated or that it's just not worth the effort. But. The right workloads one hundred times speedup is totally possible with minor changes here code. But you do need to understand the internals and that's why our course acing techniques in examples and python show you how to write acing code successfully as well as how it works. Get started the AC can wait today with our course at talk by dot com slash ASE INC honor welcomed. Oh, talk by Sunday me. Thanks for having me on. Excited to be here. I'm excited to. It's going to be beautiful man. Hopefully. Hopefully. Yeah it's beautiful re factories. So I am a huge fan of re factoring I've seen so many people bri to overthink the code that they're writing like well, I gotta get it right and I gotta think about the Algorithms in the way and Ryan and all this stuff and what I found is you don't really end up with what you want in the end a lot of times anyway. If you just go in with an attitude of this code is plastic, it is malleable and I can just keep changing it and you always are on the lookout for making a better you end up in a good place. Yeah. I completely agree for is not a one time thing or something that happens only two years from when you initially write the code it's I heard once actually that it. Goes a lot in hand with legacy code and There's a number of different definitions for legacy co by one definition is like is he code is code that isn't actively being written. So if you write something and then you consider it done and then the next week like no one's working on it that technically according to that person's definition is legs Z code. So that can be re factored. You know you can refer something you wrote earlier in the day it doesn't have to be a year later or ten. Yeah absolutely. I mean you just you get it working you know A. Little bit more. You apply that learning back to it and with the tooling these days is really good. It's not just a matter of you know if you go back to nineteen, ninety nine, you read Martin Fowler's re in book he talks about these are the steps that you take by hand. Make sure you don't make a mistake and now the steps are highlight right click apply re factoring I mean that's not one hundred percent true. In the example we're GONNA talk through is not like that exactly but there are steps along the way where it is potentially definitely winters in static analyzers. Heavily. Underutilized I feel and so many of them will just automatically apply the changes that you WANNA do and it's fantastic for huge code bases. It would be almost impossible to do it by hand. Yeah. Absolutely. It would definitely be risky. So maybe that's why people sometimes avoid it. Now before we get into that though let's start with your story. How'd you get him into python? I know you're into languages weren't talk about that but by onto then also yes. So the the shorter it's a long story but the shorter. Version of it is I degree in university, which wasn't computer science required at least two introductory courses. So the first intercourse was in python the second one was in Java, and then I ended up really enjoying the classes I ended up taking a couple more but ultimately stuck with the career that I had entered into, which was actuarial science. That's like insurance statistics. So you were in some form of math program I'm guessing yeah. Yeah. It's very, very boring to explain but if you like math, it's a great career. Yeah Awesome and SO I ended up for my first job at a university. I ended up working at a software company basically that very simply explained created the insurance calculator that many insurance companies use, and after working there for about four or five years, I had just fallen in love with the Software Engineering side of my job and decided that I wanted to transition fulltime to like a purely technical company. So Mitzvah several years or a couple years later and now I work for video as A. Senior. Library software engineer and That's how I got into programming and our code base that we work on is completely open source and primarily uses was fourteen and python three that's python enter down like a dream job sounds awesome. Yeah. I absolutely love it. Yeah. So you working on the rapids team, right which works on doing a lot of the computation that might be in pandas but over on jeep use that roughly right? Yeah that that's a great description. So so yeah, within INVIDIA. I work for an organization called. Rapids we have a number of different projects. So specifically I work on Cudi. F that is see you D apps. So the coup is to letters see you from Khuda which is the parallel programming language that invidia has made and the F. stands for data frame, and so this is basically a very similar library two pandas the difference being that it runs on the GPS. So the one liner for rap is is it's a completely open source end to end data science pipeline that runs on the GP you so if You're using pandas and it works great for you like there's no reason to switch. But if you run into a situation where you have a performance bottleneck, Cudi can be like a great drop in replacement. We don't have a one hundred percent parody with like the pandas library, but we have enough that a lot of fortune five hundred companies that pick up and use us are able to very easily transition their existing code in pandas to right James and import line go much faster something incredible like that. That's the goal. At the dream. Yeah. I just recently got a new alienware high end alienware desktop and it's the first G force I've had in a long time. That's not like some amd Radi on a book or something like that. So I'm pretty excited to have a machine that I can now test some of these things out at some point. Yep Acceleration on different devices it is very exciting also. All right. Well, let's start by introducing a bill briefly a little bit about refractory and we talked a tiny bit about it in general, and then we're going to dive. Into a cool example that you put together that really brings a lot together and what I love about your examples. It's something you've just gone grabbed off the Internet. It's not by contrived like well, let's do this and then unwind the re factories until it does is like just found it Michael let's see what this thing does. That's GonNa be fun. Let's just start with a quick definition of refractory factoring maybe how howdy know when you need it how do you know when you need refracting for me? I have a sort of number of anti patterns in my. Head that when I, recognize them in the code, some people might refer to them as sort of technical debt. This idea that the first time you write things or maybe initially when you write things, you don't have the full picture in mind and then as time goes on, you start to build up technical debt in your code and re factoring can be reorganizing, restructuring your code or rewriting little bits of it to basically reduce tech to make it more readable, maintainable, scalable, and just in better in general better code. That's sort of the way I think of it. Yeah. It is pure sense right? It should not change the behavior at least in terms of like inputs outputs exactly as the easiest code factor is code with tests that's unit tests or regression tests or any the other number of tests that there are. If you have a code base that has zero tests re factoring is very, very dangerous because you can fact or something and completely changed the behavior and not know about it, which is not ideal at all somewhat suboptimal and indeed Mark Valor when he came up with idea of refractory or at least he publicized. Sure. The ideas were basically there before one of the things that struck me most was not the refractories but was this idea of code smells and it's like this aesthetic I look at the code and it works. But like your nose Kinda turns out, you're kind of like you know you but it still works right it's like not broken, but it's it's not nice and You know there's all sorts because Kuzma's like too many parameters long method things like that but they rarely have beer cutoffs right? It's over twelve lines, the functions to large but under that is totally fine right? Like that's not it's never really super clear cut so I, think this whole idea. Of Re factoring much like factor in itself requires like going over and over it as sort of through your career to refine like what the right aesthetic to achieve is it probably varies by language as well little. Yeah. If you start to do it consciously when you're looking at code and asking yourself like when you have that code smell feeling like something's not right here if you are conscientiously like paying attention to what it is like slowly over time, you will start to pick up on exactly what it is about it like a very, very small one for me and I think this is mentioned in maybe clean code or it might have been Martin Fowler's book. It's like a declaring a variable earlier than it needs to be declared. So you might declare like all your variables at the top of a function, but then like two of them you use immediately but the other three, you don't use until the last four lines of the function small things like that. It seems simple but I've made the change where I've put that a declaration closer to where it's get used, and then you realize, oh, wait a second. This isn't actually reference it set to something, but then it's not actually use later on. So I can just delete this is because it was at the top of the function you can't see where it's being declared. Or. If it's US somewhere else that like you actually have a phantom unused variable that can be deleted. It's simple things that lead to better changes later on well, and just mental overhead like you said, the technical debt side of things. For example, there's the variable that was at the top surely win the code has written it was being used, but it's been modified over the years and now no longer is it being used but it because it's separated from whereas declared towards us you don't. WanNa. Mess with that like if you start messing with that, you're earning more work are you're asking more is I'm just going to make the minor change I don't WanNa break anything. And then the next person that comes to try to understand that they gotta figure out will, why is there that like set count variable like I. Don't feel like it's being used, but it's there. You know you've got another thing to think about that's in the way for sure. Yeah. So certainly, I think it's viable. There are fantastic tools that will like highlight. This variable is unused or this assignment is it meaningless or or something like that? So there are options, but still it's better to not let that stuff live in the Code Yep. Agree. Let's talk about this example that you've got here and maybe should give a little background on your language enthusiasm and programming competition interests, and so on. Your interest in hosting competitions I think is probably worth touching on already. But then this example is from you trying to reach out and understand it and do some analysis of those environments are those ecosystems right. The background with this these different languages and coding competition. Yeah. So I initially got into competitive programming quote unquote. So just though the one sentence description is there's a number of websites online hacker rank league code code forces that they host these one, two, three hour contests where they have three to four or five problems that start out easy than get harder as you progress through them and you can choose any language you want to solve them in and The goal is just to get a solution that passes as quickly as possible. So it's not, necessarily about how efficient your coat is, it has to run within a certain time limit but if you can get it to run or pass python versus plus versus Java, any code solution works I started doing these to prepare for technical interviews. So if you're interviewing for companies like, Google facebook, etc. A lot of their interview questions are very similar to the questions on these websites, and so I at one point was looking for a resource online like for youtube videos that just explained the stuff. But at the time I couldn't really find any. So I started youtube channel covering the solutions to these problems and I thought it would be better to solve it in a number of languages than than opposed to just see plus plus. So I started solving them people's plus Python and Java, and that's Sorta what led into led to my interest and competitive programming even though I'm I'm not interviewing actively anymore I just find these Super Fun it keeps you on your toes in terms of your data structure in algorithms knowledge and you can treat them as like code cod. I'm not sure if you're. Familiar with the concept of just sort of writing one little small program and trying a couple of times in different languages and you learn different ways of solving the problem you might not would have initially solve the problem that way this example I decided to just figure out what the top languages that people use to solve these competitive programming problems on a given website. So the site that I chose was cut forces. Yeah, and you're like Hey I'm working on this new data frame library that's like pandas. Let me see how I can use pandas. To solve this problem and get some practice or something, right? Yeah. Yeah. So when I just started Invidia I knew that the panda's library existed but I had zero experience with it and I knew that it had the sort of group by reduction functionality that if you had a big table of elements, you could get these sort of statistics on you know what's the top language or what's the average time? It takes people to submit a very easily with this kind of library. So I thought what better way to learn pandas than. By, trying to build a simple example that uses this library for something that I'm interested in and so the first thing that I did was I google you know how to scrape each tables using pandas and brought me to this blog that at the end of the day has about sixty lines of code and it's a tutorial walk. So it walks you through how to get this code off of HD month table and basically the Pike on talk that I gave it came out of doing this I had no plans of giving A. Talk. Talk on this I just after having gone through it and sort of re factoring one by one. I realized that like I could give a pretty simple talk to like connor like five years ago that didn't know about any of the I didn't know about prevention I didn't know about numerate I didn't know about all the different techniques I was using and I figured it would be at least for some individuals out there it would be a useful talk highlighting the things that I didn't know when I first started coding in. Python. But now we're like second nature for me and that's where that came from. Yeah and it's really interesting example is cool. I do think that a lot of the factories were let's try to make a more python ick brandon of this more idiomatic version of this like miss understanding the four in loop for example and treat them. All right. So in a lot of ways, it's a corey factory, but also kind of leveraging more of the native bits of the language if you will absolutely now. Yeah. So you went and grabbed. His code and it does two basic things goes download some html and then pulls it apart using I. Think L. Xml html parter, and then it's going to loop over the results that it gets from the H.. melpar Sir and turn this into basically a list or a dictionary. Then you're gonNA feed that over pandas as pandas, pretty interesting questions and most of the challenge or most of the the MESSI. Code lived in this html by the things, right? Yeah. That's a pretty good description of what's happening. So let's go and just Talk through some of the issues you identified, and then the fix exactly knowing how did you identify that as a problem and then what fix did you apply to it? Now there's a lot of code and it's hard to talk about coding audio. So we'll maybe try to edge like as high levels possible talk about like the general patterns and what we fix. The first part of the code would go through and it would create an empty list recruit like index to keep track of where it was and the did a loop. Elements increment the index out of thing to the list pronounce information as it went right Yep and I think the first thing that you talked about was a code comments. Actually you're like, what is this code comment here? It just says we're looping over these things but what do you think a loop is and why don't we have comment? Yeah. Even worse was like arguably the second comment some might argue is add some value but the first. Comment. Above the line that creates an empty list, it says create empty list and it's only a what is that six characters if you don't include the spaces and I, think, that's definitely one of the things that's called out in a number of re factoring books as comments should add value that is not explicitly clear from the code. I think even beginners are able to tell that you're creating an empty lists there. There's no reason to basically state. What the code is doing typically, comments should say why if it's not clear why something is being done a certain way or something that's implicit and not explicitly clear from what the code is doing yeah. In terms of refractory and I love this idea of these comments are sort of almost warning signs because if I find myself writing one of these comments to make stuff more clear unlike wait a minute wait a minute if this is just describing. What's here something about what I'm doing is wrong. Maybe the variable name is not at all clear what the heck it is or maybe it could use a type annotation to say what types come in and said of here's a list of strings like how about list bracket string goes there to just type it is it's five three after all and you know from the the code smells book dollar had this great description of calling these types of Comments Deodorant for code smells something wrong. It smells a little as bad if we'd like, lay it out, set the stage but every time I see one of those like I just need to rename this function like a short version. This comment would say a rename this variable or like restructure embrace these things apart because if it needs a comment is probably just to there's an individual in these plus plus community his name's Tony Van and he. has a a rule or not a rule, but a a recommendation that you up code base for step one, step two, step three and and guaranteed. You're going to get like one or two matches and a lot of times. It's these steps of comments on top of pieces of code and like a larger function and odds are you could make that coat better by re factoring each of those steps into its own small function just whatever the step. Like if you put step wanting a description, you've already given that he's a code name. You just need to take the next step but in a functioning give that function that. Yes. Exactly. Exactly what you said exactly I think there was even some tool way way back in early days a C. Sharp that if you would highlight Kodori factor in you highlighted a comment would function name a fi who would try to guess the function by using the. Comment turning into function like a that would work as an identifier in the language. Anyway it was totally a good idea to there's a couple of things going on here. One is like wiser prince statement nobody needs us. Once you take that out though you were able to identify this. Well, let's take a step back I if you have an integer and your increment every time through the loop so that it stays in sync with the index of. The elementary looping over. That's probably not the best way to do it right like python has built in enumerate. Yeah. This is probably one of the most common things I see in Python. Sadly in in certain languages, they don't have this function but in Python, it's right there built into the language. As you mentioned, it's called the numerate. So you can pass whatever thing you're looping over to a numerate and that's going to bundle it with an. Index, that you can then in line d structure into an index and the element that you were getting a from your range for loop before. So anytime you see an index idea or I or something that's keeping track of the index and that's getting into the J.. Sometimes it's J. sometimes it's J. sometimes it's K. ex wife you're being really creative and yeah there is a Bilton pattern for basically avoiding that and it makes me extremely happy like. It happens actually not just once in this piece of go but twice where you can make out of a numerate and once you see it, it's very hard to unseat it but like I said this was something that I learned numerate from Python and this was not something that I knew of an I didn't learn school. So there's a lot of python developers developers in many languages out there that I think they're just not aware and as soon. As you tell them. I. Think they'll agree Oh yeah. This is way better than what I was doing before. Yeah. You just need to be aware of it. You always trying to these issues you got to create the variable than like wise variable there. Then you gotTa make sure you increment you incriminate before you work on that with the value increment after is zero basis at one based. All of these things are just like complexities that are like. What is happening here? Like what if you have a have a past continue skip the loop, but you forget the increment like there's all these little education and you can just with the numerous you can say you know it's always gonNa work. You can even set the start position to be one if you wanted to go one two, three, beautiful. Yeah. That's a great point. Yeah. There are use cases where you're gonNA run into bucks whereas with. The numerate you know at least you're not going to have a bug with that index, right? It's always gonNA be tied to the position with the starting place the way you want it. So yeah, that's really nice but it's not super discoverable, right? There's nothing in the language that screams and waves it's hands and says, yeah, you're in a for loop. We don't have this concept of a numerical for loop. This is actually better than this is what? You wanted you didn't even know you wanted it. Yeah. It has to be something that you stumble across. Interestingly, some language is go is the one that comes to mind they actually build in the enumerate into their range based for Liu. So in go, they have built in basically the destructuring, and if you don't want the index, if you just want range based for loop and you want to ignore the index, then you're just supposed to use the. Under bar to say I, don't need the index for this loop, but it's interesting that like go is a more or more recently created language than python at least when they decided like they thought it was such a common use case that they would think that most people need it more often than they would. So they built it into their for loop. So with that language, you can't avoid learning about it 'cause it's it's in there for about. Syntax air tonight at least they explicitly ignore this yup interesting I. Didn't know that about go to now. You've got this little cleaner you look at it again and you say, well, now what we're doing is we're creating a list an empty list which we commented great emptiness that was cool book that coming out, but it was very helpful in the beginning to help you understand now, and then you say we're GonNa loop over these items and then a pen something to that. List well as possible. But this is one of your anti patterns that you like to like the finding get rid of right. This is an anti pattern that I call initialized then modify and actually enumerate example previously also a falls into this anti anti pattern. So anytime you have a variable that it doesn't need to be a for loop many many times it is that inside each iteration of that four loop, you're then modifying what you just initialized outside that is initialising and. Then modifying and my ass rotations is that you should try to avoid this as much as possible when it comes to the pattern of initialising an empty list, and then in each iteration of your loop, you're calling append that is built in to the Python language as something that can be used as a list comprehension, which is so much more beautiful in my opinion compared to just a a roth four loop and then appending for each iteration. Yeah. Every now, and then there's Like a complicated enough set of tests or conditionals or something going on in there that maybe not. But I, agree with you most the time at just means what I really wanted to write was a list comprehension. It is though meal bracket item four item in such and such if such as I tried that, that's what you gotta do. Yeah. Let's comprehension. Once you start to use it moving to a language that doesn't have it makes you very sad because it's Joe's needs. He's totally makes Zad and. I really really wish list comprehension had some form of sorting clause because at that point, you're almost into like in memory data base type of behaviors, right? Like I would love to say projection thing transformed thing for thing in collection where the test is order by whatever I mean, you can always put assorted around it, but it's it'd be lovely if they're like it's already got those nice steps I like to ride it on three lions, right? The projection, the set and the conditional like just one more line but the order by in there but maybe wonder maybe I should put a pep in there. Who knows I was gonna say that sounds like a future pep. Definitely I mean it would be easy to implement. Transform went to SORTA impasse at his key or something like that. But anyway, it would be really cool. But they're they're very, very nice even without that and once you have it as a list comprehension then unlocks the ability to do some other interesting stuff which you didn't cover in yours because it didn't really matter. But if you have square brackets there and those brackets are turning a large data collection into a list if you put rounded brackets all the sudden, you have a much more efficient generator. Yup that is something I don't call out at that point but at. The end of the talk I- Lewd to article that was mentioned on the other podcast that you coz Python Bites. Yeah. Thanks for the shout out by the way. Yeah. No, it was a great article. But if I mentioned generator expressions right after it mentions list comprehension and I mentioned that these things go hand in hand that you should familiarize yourself because if at any point you're passing a list comprehension to like an algorithm like any or or something, you can drop the square brackets and then just pass it the generator and it'll it'll become much more efficient. So. With them and there's no way to go from four loop really quickly and easily to generator yield style of programming. Right there's not like for yield I in whatever right like there. But with comprehension, it's where brackets versus rounded BRI apprentices, right? It's. So it's so close that if that makes sense, it's like basically no effort to make it happen and okay. So we've got into a list comprehension which is beautiful, and then you say, all right, it's time to turn our attention to this doubly nested for loop and it's GonNa go over a bunch of the items and pull out an index, and then you know go and work with index. So it's another numerate and then I think another Thing, that's pretty interesting that you talk about I don't remember exactly where it came in the talk but you're like look what you're doing in this loop is actually looping from like a second onward for all the items and that really is just a slice. Yeah. Yeah. So in this nested for loop, the outer for loop is basically reads for J. in range of one to the length of your list. So you're basically creating a range of numbers from one to the length of your list, and then right inside that four loop you're creating a basically a variable. That's the J, th element of your list. So all you're doing is skipping the very first element of your list, but the way. You're doing this is generating explicit indexes indices based on the range function and the length function, and I thought at first that they must be doing this because we need access to the index later or we need access tr elements later but that wasn't the case it just seemed like the only reason they were doing all of this was to skip over the first element and so very nicely. Once again, Python has very, very many nice features. They have something called slicing where you can basically pass it the syntax, which is square bracket, and then something in the Middle End Square brackets and in order to skip the first one, you just go one colon one to the and that's beautiful because. He don't even have to check the length of the items. You just go to the end, which is avoids areas of like do I have to plus one here do I not is at minus one budget the ending piece, by John, Dory badges from skip, Vermont and the rest Yep, it's so convenient. You're you avoid making a call to lend you avoid making a call the range, and you avoid your local assignment on the first line of your. You can basically remove all of that and just use slicing and you're good to go and the slicing is slicing as a a really really awesome future actually comes from a super old language that was created in the sixties called AP L. and Python. Languages that has something called negative negative index slicing where you can pass out a negative one so that it wraps around to the last element, which is a super super. It's SORTA looks weird but once you use it, it's so much more convenient than doing like a len minus one or something like that. It's it's it is a little bit unreal but. Once, you know what it does. It's great. It's great. It's like I want the last three. I don't WanNa care how long it is I just want the last three and that's yeah it's fantastic. Slicing I think is fairly underused for people who come from other languages but yeah it and it fits the bill because there's so many of these little edge case. You talk about arison programming like off by one errors are a significant part of problems with programming right and they just skip that altogether it's beautiful up to the next thing to do is so you're parsing the stuff out of the Internet, which means you're working with one hundred percents strings, but some of the time you need numerical data. So you can ask questions like the sixth or seventh or whatever, and so they have. This is GonNa be fun to talk about they have try value equals in of data. So past the integer asked the potentially like data over to the. Initial eiser either that's going to work or it's going to throw an exception in which case you will say except pass will not you the original article had that rape? Do It's this try pars except? Otherwise it's going to be non or it's going to be set to the string value or something to that effect. So what do you think about this? How'd you feel when you saw that? Yes. So my initial reaction was that this is four lines of code that can potentially be done in a single line using something called a conditional expression. So in many other languages, they have something called a ternary operator, which is typically a question mark where you can do an assignment to a variable based on a conditional predicates is something that's just asking to reform. And if it's true, you assign at one value and if it's false, sign it another value. So in Python, they have something called a conditional expression, which has the syntax assigned to value using the equal sign ask your questions. On this case we just ask, is it an int- or story? So the first thing that returns it's actually backwards from Turner operator. So this is the reality. The line reads data equals into data if and then check your predicate and in Python, we can just call is numeric. Value which will return as true or false based on whether it's a number. So if that returns true than it'll end up assigning data into data otherwise, you can just sign it itself data and then it's not going to do any transformation on that variable because it's not numeric it's one line of code. It's more expressive in my opinion and avoid using trying except and it's preferable from from my point of view. I would say it's probably preferable from my point of view as well. I have mixed feelings about this but I, do think it's nice under certain circumstances one. For example, if you say tried to thing except pass a lot of lenders and high charm and whatnot will go this is too broad of a clause you're catching too much and you're like, okay. Well now to make the little squiggly in the scroll bar, Go hey. I have to put a Hashtag disable check whatever right now it's five lines one with a weird exception to say, no, no this time it's fine. So that's not ideal. I. Definitely. Think that that's more explicit more expressive tease this additional if one liner, the one situation where I might step back and go you know, let's just do the try is if there's more variability in the data, so this assumes that the data. Is Not none and that it's string like right. But if you got potentially objects back or you got none some the time, then you need a little bit more of a test mean you could always do if if data and data is numeric, that's okay. But then it's like if data and is instance of string data and like there's some level where there's enough task that it becomes you kind of find a crash, right? We'll just catch it and go but we were talking before we record also like there's a performance consideration potentially definitely and it's interesting. I'll let you speak to what what you found but on the Youtube comments of the Pike talk that was one of the probably the most discuss things was whether or not the conditional expression was less performance than the. Original trying except because a couple individuals commented that it was, it was more python to use the except in and therefore it might be more performance. You can share with what you found sure. Well I think in terms of the python inside like certainly from other languages like say c plus there's more of this it's easier to ask for forgiveness than. Style programming rather than alternative look before we leap right visiting like see, it could be a page fault in the programs because poof and goes away and something wrong. Where's this? It's just gonNA throw an exception you're gonNA catch it or something like that A. so there's like this tendency to do the style but in terms of performance I wrote a little program because I wanted to. I've Mike maybe this is faster may be slower. Let's think about that right by wrote little program which I linked to. There's a simple just linked to it in the show notes breeze one, million list with one million items at us the random seed that has always the same so there's no zero the area. Ability even though it's random, it's like predictable random and it builds of this list of either strings or numbers randomly a million of them about third strings, one third number, and then it goes through and it just tries both of Mrs like let's just convert this as many of them as we can over to editors and and do it either with the trikes pass or just do this is numeric test. It is six times I got. Yeah. About six point five times faster to do the test, the one line test than it is to let it brash and realized that it didn't work. Yeah. So there you go. You heard it here on hug my to me driving conditional expressions faster than trikes. Talk Python to me is partially supported by our training courses. How does your team keep their python skills sharp? How do you make sure new hire get started fast and learn the python ICK way. If the answer is a series of boring videos that don't inspire or a subscription service. You pay way too much for and use way too little listen up. At talk by on training, we have enterprise tears for all of courses get just the one course you need for your team with four reporting and monitoring or ditch that anew subscription for our correspondence, which include all the courses and you pay about the same price as subscription once. For details visit training talk by DOT, FM slash business or just email sales at Tom Dot FM. There's a lot of overhead to throw an exception and catching it and dealing with all that. Now, right this is a particular use case that varies and like all these benchmarks like might vary like if you've got ninety five percent numbers and five percent strings at might behave differently. So there's a lot of variations example you can play with in what seems like a reasonable example to me it's faster to do. The is numeric tests. So fast, right? Not like five percent faster six, hundred, fifty percent faster is worth thinking about out for sure. Let's see. So come through and in the end you had. A ton of stuff was here. It was like when he lines a code just for these two loops, and now you've got it down to four lines of code by basically an outer loop and inner loop grab the data independent with this little test that you've got much nicer. I Agree Yep. So you went I, think if you look at the overall program at this point, you were doing some analysis or like some reporting said, it started sixty lines of code and now it's down to twenty. Roughly. depending on if you count, you know lines and whatnot but it was about sixty down to about ten or twenty lines and At this point I had sort of pointed out that I had made a mistake. So this was fantastic at least I had thought that you know I I take a code I'd take a code snippet from a blog reduced it by. Roughly seventy five percent or sixty seven percent depending on how you measure it. But I had made an even bigger mistake than I had realized and it was that when I hit originally I'd shown googling for you know how to scrape html using pandas that I read the second results and the third result was actually what I should have chosen and it was that I had a pandas actually has a read html method in the library and so the point that I go on to make if you use that you go from you know ten or twenty lines down to like four lines of code and You're just invoking this one Pan Api read html and it's so much better. So you re factoring is fantastic but there's some quote about like the best code is no code. If you don't have to write anything to do what you WanNa do and you can just use an existing library. That's the best thing that you can do because that's going to be way more tested than the custom code that you've written. It's GONNA save you a ton of time and you're going to end up with ultimately less code to maintain yourself and better than having someone else maintained the Co that you're using for you. Exactly right. It gets better for for no effort on your part. Yeah. It might get faster where my handle more cases of like broken html or who knows but you don't have to keep maintaining that it's just read UNDIS- grades Tim Allen painters just probably getting maintained. Yeah. One of the things that I've echoed in some of the other talks that I've given is knowing your algorithms in C. Plus plus definitely there's a whole. Standard. Library. There's a lot of built in functions. I guess they're not so much called Algorithms. They call them built in functions and python. But like there's a whole page where I was just looking at it the other day and there's a ton of them that I'm just not aware of everyone else about map filter. Any all like I. Just saw I. Think it was called MoD which was a built in function for giving you. Both like the quotient and the remainder, which is like there's definitely been a couple of times where I've needed both of those in you do those operating separately and it's like off I just knew about it. You can in a single line you can D- structure it using the Uraba unpacking knowing your algorithms is great but also knowing your libraries knowing your collections like the more you get familiar with what exists out there the less. You have to right and the more readable your code is because if everybody knows about it, we have a common knowledge base that it's transferable from every project you work on your final version basically had to really meaningful lines. One was request dot get the other was hand as dot read html. You don't have to explain to anyone who has done almost anything with python what request get means like Oh yeah. Okay. So got it. Right. We all know that works. We know it's good work and so on and it's really nice. I. Think Though what you've touched on here really it's really important but it also shows why it's kind of hard to get really good at a language and the reason is there are so many packages right you go to. Let me. Let me try Pipe York. Now every time I go there is always Hundred and forty, five, thousand packages. If you WANNA, learn to be a good pilot program or you need to at least have awareness at a lot of those and probably some skill set in some of them because like pandas, one of those requests and other one writes the four lines solution that you came up with was building on those two really cool libraries and so to be a good programmer effective. Like keeping your eye on all those things and I think that's it's both amazing. But it's also kind of tricky because I'm really good with four loops functions great. You've got two hundred thousand packages of study go. There some quote that I've heard before where being a language expert is ten percent language, ninety percent ecosystem and it's you can't be a guru in insert any language if you don't know the tools if you don't know the libraries, it's so much more than just learning the syntax and learning the built in functions that come with your language it takes years and It definitely doesn't happen overnight. It's a challenge for all of us aperture. You know maybe it's worth a shoutout to awesome bash python dot com right now as well, which like as different categories you may be cared about and then we'll like highlight some of the more popular libraries in that area that. Sounds that's a good one. Yeah. For sure do you did nine different steps? You actually have those called out very clearly in your slides, you get the slides from the get hub repo associate with your talk which linked to the show nights of course. But all of this re factoring talk was really art of the journey to come up with a totally different answer, which was one of the most popular languages for these coding competitions. Yeah. Ultimate goal was to scrape the data and then to use pandas in order to do that analysis and at the end of the day I, believe the number one I definitely know the number one language was C. Plus Plus said about. Eighty nine percent, and that typically is the case because certain websites they give the same time limit per language so Like hacker rank, they vary by language. So Python your execution time that your allotted faster is ten times more for python. So even though python slower, they give you our amount of time, but most websites don't do that. So the code forces website, it gives like you I think two seconds execution time regardless of the language they use and so due to that most people choose the most performance language, which is C. Plus plus. But in second place was python and I know a lot of competitive programmers that for the problems where performance isn't an issue that you're trying to sell four, they always use python because it's about a fraction of the number of lines of code to. Solve it in python than it is in any language sometimes, you can solve a problem in one line in on and the next closest languages like five lines, which is a big deal when time mattress guy yeah. Are you optimizing execution time or developer time in this competition, right? Yeah. It definitely matters what you're trying to solve for C. C plus plus was first python was second Java was third and then there was a bunch of fringe languages. The top three were C. Sharp Paschal and Katelyn and yeah you can see a full list if you go watch the Pike on talk but it was add to find out what was what was used and what wasn't sure was and Cool to see the evolution of what you created the answer that question or just pretty neat. All right. Well, let's just talk a little bit about rapids because I know that people out. There are visual data scientists and they're probably interested in that project. Do we did mention a tiny bit that it's basically hake has as data frames apply something like that that API pretty close not one hundred percent identical and everything, but pretty close and it runs on. The wire GP's better like I've a really fast computer. I have a core I nine with six cores. I. Got a couple of years ago. That's a lot of course, right? So yeah it first thing I should highlight to is that rapids is more than just Cudi APPs. So could have the library I work on. We also have a Kuu, I o coup graph consignia who spatial cwm l., and each those sort of map to a different thing and like the data science ecosystem. So Cudi F-. Definitely is the analog pandas Kumhal I think this sort of analog you can think of is like psychic learn but also to like none of this is meant as as replacements, they're just meant as. Alternatives like if performance is not an issue for you like stick with what you have, there's no reason to switch day. I don't don't do it because for example, I couldn't run it on my macbook right? Because I've Radian right? Right. If you do want to try it out I think they're on the record. So if you go rapids day I, we have a link to a couple of examples using like, Google. Co Lab that are hooked up to like free GP that you can just take it for a spin and. You need the hardware, but you can't go try it out. But like our pitches sort of like this is useful for people that have issues with compute right and for different pieces. You'RE GONNA want different projects. So if you're doing pandas likes or data manipulation Cudi, F is what you want. But yeah, why are GP's faster is just a completely different device in a completely different model so a GP us. Typically. It's in the G. of the GPO were known for being great for graphics processing, which is why it's called the GPO. But at some points, someone coined the term he actually works on the the rapids team. Mark Harris, he coined the term gpgpu which stands for general processing GP compute. It's now typically referred to as just jeep you computing, but it's this idea that even though the GP model is Great. For Graphics Processing. There are other applications that a jeep to us are also amazing for the next best one is matrix multiplication, which is why they of became huge in neural nets and deep learning. But since then we've basically discovered that there's not really any domain that we can't find a use for GPS force. So there is a a standard library in the coup model called thrust. So if you're familiar with C.. Plus plus the standard libraries called S. t. l., and it has a suite of algorithms and data structures that you can use thrust the analog of that for Khuda, and it has reductions. It has scans and it basically has all the algorithms that you might find in your CPAP plus stl, and if you can build a program that uses those algorithms you've just GPA accelerated your code however using thrust isn't as easy. As some might like and a lot of data scientists they're currently operating in Python and R., and they don't want to go and learn C. Plus plus and then Khuda, and then master the thrust library just in order to accelerate their data science code the rapids goal is to basically bring this GP computing model for a sort of general her bis acceleration of data science compute or whatever compute you want to the Data scientists, and so if they're familiar with the pandas API, let's just do all that work for them. Put the. So so rapids is built heavily on top of thrust and Khuda, and so we're basically just doing all this work for the data scientists so that they can take their panda's code like you said, hopefully, just replace the import and you're off to the races and the performance wins are pretty impressive like. I'm not on the marketing side of things. But in the talk I mentioned, I just happened to be listening to a podcast held the invidia AI podcast and they had i. believe name was Nicholson and by swapping out cudi effort pandas for their model. They were able to get a hundred x performance win and a thirty x reduction in cost. That's thirty times. Not Thirty percent. Yes. So thirty is at right. Multiplication, valley, which is massive. That's the difference between something running. So if it's a hundred X in terms of performance at the difference between something running in sixty seconds or an hour and forty minutes, and if you can also save thirty x if that cost you a hundred bucks and now you only have to pay three dollars. It seems like a a no brainer for those individuals that are impacted by performance bottleneck. If you're hitting pandas and runs in a super short number seconds. It's probably not worth it to switch over. Yeah. Well, and you probably you tell me how realistic you think says, but you could probably do some kind of import conditional import like in the import, you could try to get the record stuff working. If that fails, you could just import pandas as the same thing. One is PD the speedy. Maybe it just falls back to just working on regular hardware but faster when it works pretty think that is definitely possible. There's going to be limitations to it though obviously if you have. A a sort of Cudi F. Data frame I don't think you do it piecemeal. But if you have a large product what I'm thinking if you wrote it for the rapids version but then let it fall back to pan does not the other way round you take arbitrary pass code you try to ratify it that might not work, but it seems like the other one may well work and that way if somebody tries to run it, they don't have the right setup is just slower also would there's definitely a way to do that to make that work it might require a little bit of. Some sort of boilerplate framework code that is doing some sort of checking you know is this compatible else? But like that definitely sounds automated like, yeah. Yeah. That sounds cool because that would be great to have it just fall back to like not not working just not so fast. Right The future of computing is headed to a place where we can dispatch compute to like different devices without having to like manually specify that like I need this code to run on the CPU versus the view versus the TPS versus in the future I'm sure there's GonNa be a cue for quantum processing unit like. Exactly. We all think Sierra Leone most of us that don't work it in video. We are serially and in terms of like the way that fuse do compute but I think in ten or twenty years were all going to be learning about different devices and it's going to be too much work to in our head always have to be keeping track of which devices is going to. At some point there's going to be a programming model that comes out just automatically handles like when it can go to the fast vice when we can send it to the seaview. Yeah, absolutely. So just a while you're talking I pulled it up on that alienware gaming machine got as a g force are. To seventy, which has two, thousand, three, hundred, and four cores. That's a lot that's a lot of course, and if you look somewhere Google claims that it achieves seven point five teraflops in the super increases that ten nine teraflops, which is just insane like a core I seven doing like going to three five. Point two, eight or something like that. So anyway, the numbers readers are like they boggle the mind you think of how much computation graphics cards do these days I think top of the line I might get this wrong but like the modern GP user are capable of fifty teraflops. It's an immense amount of compute. That's hard to fathom especially when coming from the CPU sort of way of thinking. Yeah, absolutely. Yeah. The only reason I didn't get a higher graphics card is every other version required water cooling. I'm like that sounds like more effort than I want for a computer could just go with us when. All right, well rapid sounds like super cool project, and maybe we should do another show on the the rapids team across these things to talk a little bit more deeply. But it sounds like a great project by working I, work on the C. Plus plus lower engine of it but I'd be happy to connect you with them. Some of the Python folks that that work on that side of things and I'm sure they'd love to come on. Yeah. That'd be fun. All right now before you get out of here, got to ask you the questions if you're gonNA write some Python Code, what editor do us so I am a vs Code convert. That's what I typically use day to day. Nice. Yeah. That's quite a popular one these days and then notable pipe package. Then you ran across Chicago people should know about this. Yeah. So I like to recommend there's a built in Standard Library which I'm pretty sure most hyphen developers are familiar with it or tools which has a ton of great functions but less well-known is a IP package called more hyphen editor tools and I'm not sure if this one's been recommended on the show before. But if you like what's in it or tools, you'll love what's in more tools. It has a a ton of my favorite algorithms chunked being. One of them you basically pass it a list and a number, and it gives you a list of lists consisting of that many things. It's like paging for yeah. Yeah and there's tons of neat functions. Another great one that's so simple but doesn't exist built in all underscore equal. It just checks given a list are all the elements, the same, and it's a simple thing to do. You can do it with all but you have check is every element equal to the first one of the last one. So there's just a ton of really convenient functions and algorithms in more O'Toole someone Iraq. Yeah that's cool and you can combine these with. Like generator expressions and stuff they are all these you know pull some element out of each object that's in their generate collection. Ask of all those are equal at they go to all these ideas go together well there. Yeah. They compose super nicely. Yeah for sure. All right. Final Call to action people are interested in doing and refractory and making their code better. Maybe even check out rapids but he say, I'd say if you're interested in what you heard on the podcast, check out the pike on talk, it's it's on Youtube. If you search for bike on twenty twenty, you'll find the youtube channel and if you're definitely am interested in the Day I check us out there I assume all this stuff will be in the show notes as well. So maybe that's Urging. Yeah it well, then also you talked about Youtube channel a little bit baby just tell people how to find that will put like in the show notes as well. So they can if they to watch you. Talk about some of these solutions in these competitions. Yes. So my online aliases code underscore report. If you search for that on twitter youtube or Google I'm sure all the links will come up and yet you can find me that way. Awesome. Are Willing to that as well. I Will Boehner. Thank you so much be in the show a lot of fun to talk about these things with you. Thanks for having me on. This was awesome. You Bet bye bye. This has been another episode of Talk Python to me or guest on this episode was Connor who? Has Been brought to you by US over at talk by on training. Wanting to level up your python. If you're just getting started, try my python jump start by building ten apps course or if you're looking for something more advanced check out our new eysenck course, the digs into all the different types of acing programming you can do in python and of course, if you're interested in more than one of these, be sure to check out our everything bundle. It's like a subscription. That never expires be sure to subscribe the show open your favorite podcast and search for Python. We right at the top you can also find the itunes feed it slash itunes the Google play fetus slash play in the direct RSS feed at slash ourselves on talk by phone Donovan. This is your host, Michael Kennedy thanks so much for listening I really appreciate it now get out there and write some Python Code.

Michael Kennedy youtube Google Martin Fowler Emmy GP twitter developer Khuda ASE INC Ryan Lee jeep Mark Harris US alienware
170 Navigating Uncertainty is Product Management

This is Product Management

28:16 min | 3 years ago

170 Navigating Uncertainty is Product Management

"This is product management. We feature the brightest minds across numerous disciplines that fuel modern product teams join over fifteen thousand weekly listeners to learn from product leaders, authors and founders from companies such as Spotify under armor in to it strike base camp and Airbnb, they'll share their insights and best practices for experimentation, user research, leadership, corporate innovation and more to receive updates on the latest episodes subscribe to our newsletter at this product management dot com. Your host, Mike fish bind. Welcome back on the Sepah soda. I'll be speaking with Ryan singer, head of strategy at base camp. He'll discuss how to manage projects on validated as sumptious, eliminate blockers, and avoid unnecessary meetings. This is product management is produced by alpha, the world's leading companies use ALPHA's on demand, user insights platform to make data driven decisions about users products and new markets requested demo at alpha HQ dot com. Slash t I p n. I've been working at base camp for fifteen years now. I was part of the small team that built v one starting in two thousand three. And when I started off, I was doing hands on you, I design and you. I development, you know, hooking up views and stuff to very early on. I reached a point where I was a little bit frustrated that I would have to come up with the design and then give it to a programmer textually see it in action, then figure out what needed to change. So David who's one of the founders here he created ruby on rails, which is what base camp is built on. And that actually made it a lot easier for us to sort of cross that bridge between design and programming. So started to learn to program starting to learn a lot from the programming world. And then as a few years went by, I had one foot the design world one for the programming world, which was kind of a natural step toward sort of how do we work together, what are the steps that we use to get projects done? On how do we design and develop. So then I ended up kind of working at a more product management level. And now these days my work also includes a lot of kind of ARIN, d and strategy work, figuring out what's important for us to work on. So not just getting things done on time or getting things done in the right sequence, but also sort of figuring out what's valuable and where we should next. Ryan, how filled the first version of base camp in two thousand and three and has been with the company ever since now works on strategy and are in Indy. I asked Ryan how we get started a base camp and what makes the company. So you need. When I first joined base camp, we were called thirty seven signals time, and it was Jason move founder and Jason freed, and he had another designer working with him at Lindemann. And the thing that really attracted me to Jason's work at the time was this was kind of earlier days in the web. So this was like late nineties. I think thirty seven signals was founded in ninety nine and there was a lot of flashy stuff that was very kind of graphic designing and then but with thirty seven signals was doing was really about the utility. It was about usability. It was about the right words in about whether things make sense or not. So at the time, thirty, seven signals was doing a lot of kind of usability focused redesigns of confusing websites and the sort of function focused clarity, simplicity thing. This was sort of a shared value that really attracted me to Jason's work. He was looking to hire. Our designer, and that's how I got on board. When David came in, he was originally doing some contracting for Jason, and then they started building the first version of base camps together. David brought in a whole lot of values from the software world that are still unique. Some things have caught on in turn into buzzwords like agile, but they don't really mean anything anymore or if they mean something, they don't mean something particularly good, but David inherited a lot of wisdom from what I think of as the small talk, Wendy, each of programming. So sort of the mount Martin Fowler, Kent back school of thought around how to deal with complexity. David brought a lot of discipline in a lot of knowledge about kind of how to take things how to bitings often small pieces. And so he was always pushing us from the very beginning. He was only working ten hours a week on base camp. So it created this very early culture of how do we make progress with what's the most meaningful, small step of progress that we can make. And so I kind of feel. Like that sort of where I see, how do you say that the culture was born out of that combination? Jason is an extremely intuitive experience oriented, like down to the core product guy who has just really powerful gut for what matters. Stripping things that don't matter on the experienced side. And then David is really strong in terms of how do we strip down what we need to do on the technical side so that we get the most leverage out of the fewest moving parts. That's kind of a I learned by learning from both of them. On episode of a one, twenty seven of this product management base camp CEO Jason freed shared how he fosters productive work environment without requiring team to work crazy hours. Ryan says that Jason and David created ruby on rails, encourage simplicity. How does Ryan manage projects with unknown requirements. We learned the hard way, you know, through experience that every time we planned out how we thought project should work. If we tried to perfectly plan out the interface in advance and plan out everything that every task that needed to happen on the back end side, it never worked. We would always end up being surprised by things that we had overlooked things that were more complicated than we expected. Things that didn't connect the way we thought they were going to connect or even just things that didn't look the way that we didn't like how they looked, even though we thought that we would. We learned early on that, and I think everybody in this business actually knows this. You know, if you've done the work that you keep getting surprised in the things that you think are gonna work, don't end up working. It's just that our processes, they don't take into account what we all know what we see in front of our faces every day. And I think a lot of this is is behind the enthusiasm for stuff like agile. Title where people are saying, don't plan and, but actually those processes don't really help you. Because if you look at at the way they're being practiced, it mainly comes down to doing the old school waterfall approach, trying to estimate everything that you're going to do, but just doing a week at a time or two weeks at a time, and that's not a meaningful difference. So what we're trying to do is we're trying to be honest with ourselves about what is it that we can know from where we stand when we first start a project. So let's say we're doing a project and we've budgeted eight weeks due to project, what do we know at the beginning and what are the things that were not going to know until we start to get our hands dirty. We start off with the concept of these are the basic flows in the interface. These are the basic pieces of code that we expect to be able to connect. This is how it's supposed to work, but there are on the interface side, many, many details where we don't know where the body should go. We don't know how we're going to deal with. It on the mobile version, right? We don't know what the exact right copywriting is going to be news, tons, tons of details like that. Right. And on the back end side, we might know that it's gotta be possible. This is always the attitude we have for programming. Of course, it's possible everything in software as possible, but we don't know what we're gonna run into when it comes to actually implementing it. This is because we're building things that we haven't just built before. If we were manufacturing shop and we were cooking out the exact same product over and over. Again, of course, we wouldn't have any of these knowns, but every time that we build something we haven't built before we're gonna run into surprises. So we're trying to do is deal with that. Right? Because the things that actually get projects stuck is not so much the fact that something turns out to be a little bit harder than you thought or you missed a few tasks or it's taking a little bit longer than you expected. Those things aren't really problems if you know what needs to be done, and it just takes a little bit longer. That's. Not a big problem. The problem happens when the project isn't moving and you don't know why. And what happens is somebody is stuck or somebody's going in circles on something and they don't know the solution and they're not raising their hands. That's where things really get stuck in where they really break down. Projects, rarely go as planned Ryan differentiates between figuring out what you need to do and actually doing projects get stuck when someone doesn't know the solution to a problem. Here's how he recommends voiding the situations. So the first thing is, I think it's important to recover a little bit of what's called design up front. You know, when there was a time in the programming world in an I think maybe the peak was in the nineties where you didn't start a project before somebody had filled two hundred pages with diagrams depicting the whole architecture of how everything was supposed to work. It was like a master blueprint and then it would go down to lesser beings to implement and this clearly didn't work, you know, planning everything in advance and then being surprised that the outcome wasn't what you wanted. So there was a reaction to that, and a lot of us now work in an environment that's shaped by the agile sort of movement, which was in some ways a response to that and what happened is the pendulum slung totally the other way where now that the religion that everybody preaches is no plan, no learn by iterating. And this is also. Wrong, because if you go in and you have no plan at all, and you're only plan is to just keep iterating that's called wandering. That's a random walk. That's not in order for iterating to be constructive. You have to have some kind of a measure of when are we getting warmer in winter, we getting colder. So you have to have some sense of what you're trying to do. The first thing is before we start a project, we actually do a bit of homework with the most experienced people to sketch out the solution. So instead of saying, we're going to build this feature and we're sure it's possible. So here's the bullet points of what it's gonna be. And now the team goes, does it? We say ourselves, well, let's produce some evidence that this is feasible as we think it is by rolling up our sleeves a little bit. So we're gonna actually sketch out a quick model of maybe how the pieces of code are gonna connect or what are all the moving parts that we're going to need to have to actually make this thing work. And we're going to sketch out from the interface point of view. What are the main elements would've key afford Ince's? How are you going to get from step aid to be to see and if it after we've done that preliminary work on kind of roughing out the outlines of sort of the solution. Now we've eliminated enough risk that if we give it to a team, there's still unknowns that can pop up and they're still things to get stuck on, but it's not like there's a giant gaping hole in the initial idea. So that's the first thing is as actually doing a little bit more of what you could call shaping the work. The pre work before you actually budget to team is really important, really valuable. The second part is once you've actually given it to the team, how do you manage it? Right? So if curiously a list of things that have to be done, then that's not gonna work because you can't estimate down to the very task how long things are going to take and what's going to be harder and what's going to be. Easier and get it all done in the timeframe that you estimated. So we use the two main techniques. First thing is that we give the team in overall budget of time. Let's say eight weeks, six weeks or eight weeks depending on the time of year that we're in and they have variable scope. So they know it's important. They know it needs to be there, but there's a lot of things that might be in the design that turn out to be expensive to build, but aren't actually the most important thing. So the team needs to be able to make judgment calls along the way about what's getting hard and what's getting difficult and what's taking time and contrasting that with what's actually central to the idea what's actually important to deliver the improvement that we're trying to give customers ler says, what is maybe just implementation idea or particular user interface concept. You know, there could be two equivalent ways to do the exact same interface. And one of them could be an order of magnitude more complicated just because it happened to be that that's the idea designer had. Day. So we have to have this push pull between the complexity that's coming up in the work and the value of the work is worth it. The way that we monitor that is through the hill chart thing, which is we're looking at as people are working on different chunks of work. Are they getting quickly to the solution? So are we say, are they moving uphill? Are they figuring out how to get it done and seeing what's going to work? And then getting to the point of execution on that piece of work, or is the work kind of stalled in this up hill phase where it's like, yeah, I'm not sure. I'm not quite sure about this part or we're still waiting to know about that part or we haven't solved this, and those are really the red flags that come up when there's a piece that that hasn't been solved, or they're not kind of clear how to move forward. And those are the places where we wanna be able to spot that's happening and then jump in and then makes decisions. Ryan recommends giving team time budget for variable scope using what he refers to as a hill chart to spot where pieces are stalled and solutions decisions are needed. How does he prevent projects from getting stuff when pieces aren't moving as quickly as planned. Every project that you take on kind of has its own anatomy. You know how the hip bone is connected to the leg bone. So we want to build a to do list feature in order to do the reordering. We have some Java script. There's some interface pieces that have deployed in here and you have to get to it from over here and we have to talk to the back end there. There's all these different moving parts. And if we try to solve them all at once, we're not gonna get anywhere. And the whole thing is going to be an unclear unfinished thing until the eleven our last day. That doesn't work. What we do is what we learned that good programmers. Do you know if you look at what a good programmer does. They don't just coat everything into one file. They separate their code into what they call separate concerns. So they have different functions, different classes, they separate things out by the responsibility that they have. And that's actually kind of the origin of where I learned how to do this. So I gave the example video. Although that I mentioned to you this, how we worked video number five, if we're building a piece of content management system for advertising events on a website, there's a back end part for populating the events, and there's a front end part for revealing the event with interface element. Now, the types of problems and the types of work that you have to do for those things are very disconnected from each other. You can pull this things apart and say, in order to get to the back end, somebody has to authenticate with the username and password because we don't want anybody to come in and post events on our CMS. If you don't think about that and it, there's just a task somewhere that says authentication, it could easily happen that somebody starts to work on that. And then they start to say, oh, why don't we hook this up to our internal identity system? And then it turns out that the internal identity system doesn't have the right permission flagged at a new service to it. And then before you know it a three. Month's project has gone by because they're refractory some back end service so that they can in the future Admiral indication options. And it's like, wait a minute. What project were we doing. I thought we were trying to show an event on the homepage of the website, but this is the kind of thing that happens all the time. So if we dentist front, okay, we have to have log in. We have to have this piece of Java script on the front end to show the event. We have some design challenge that we have to solve on mobile, and then we wanna do is we want to figure out which of these things contributes the most to the solution. So for example, if I skip on the design, that's the thing that everybody's gonna see. Okay, so I don't want to necessarily cut corners on the way to be event is presented on the homepage. However, if I skip on the center -cation and instead of connecting it to our back end identity system and improving the identity system to support arbitrary future projects, if I could just hard code password because maybe only two people need to create events and it's a small office or a small team with the office. So I could cut down the work by hundred x by realizing that this. One piece is less important and then also sequencing differently. So if I see that, what are the big unknowns here where we have a major unknown in how we're going to expose this on the phone, but say because we don't quite know how it's gonna fit into design and maybe we have an unknown with how the Java script client is going to deal with some sort of a caching thing, right? We have some sort of a performance concern with the technical idea. So what we want to do is we want to be able to isolate those two problems and then sequence those early in the project so that those of the things that people are working on and then we wanna know we can't ask just kind of are those tasks done because before somebody starts to actually do the work, they don't even really know what the tasks are. It's an open problem to solve. So we wanna know is, hey, are we getting closer to solving that? Are we figuring this out? How much is still unknown? How much of this is something where we can see the edges of it, and we know where we stand that sort of how we think about it. Put our guest advice into practice with ALPHA's on-demand user insights platform teams at the world leading companies use alpha to make data driven decisions about users products and new markets request a demo at alpha h q dot com. Slash t. I p m. Isolate unknowns, so you can get answers sooner and avoid blocking other pieces of a project. Ryan says, how can this approach help reduce wasting time in pointless meetings. If you don't have a lot of structure about kind of what are the pieces of work and where does each piece of work stand? You don't have an easy way to talk about that. Of course, you have to call a meeting to figure out what the heck is going on. So this is, I think what happens is people rely on meetings because they don't have enough structure around how they define an how they think about the work. So you could have a loose kind of wandering meeting where everybody goes around and says, hey, this is what I'm stuck on, and this is what I'm working on. I'm trying to figure this out and we're waiting on this and you have to listen to all those details. If you have looked at the anatomy of the work and you've found the parts that are can be worked on independently and you isolate those as separate scopes. And then each scope is tracked on a hill showing you whether it's on the uphill climb of figuring it out or at the top of the hill where it's known, but still needs execution or. The downhill where it's all solved. And it's just a question of a bit more labor to get it done. What I can do is I can look at the whole project. I can see six seven dots on a hill. Each one has a name that's referring to a scope and I can see how far along each thing is and where it stands in terms of being stuck or moving, or being known or being unknown. That's less than five minutes. That's like a two minute thing that I can look at that chart and see all the pieces of work that are in motion. And then if I see something that's at the bottom of the hill on the left side, something that hasn't started, it's on to- climb yet a total unknown. And I think that that's something that makes me nervous about the project or it's something that I think is fundamental to the project, then I can go to the team and I can write a message a synchronous Lee without even calling a meeting in saying, hey, this thing is important. What's going on with that? Or if I see something that is stuck on the same position on the hill for many days in a row, I can send them a message and say. It what's going on with this and they say, oh, turns out that, you know, we have to hook up with this API and so and so's out of town. So no problem. Let's break out new scope for dealing with that API and we can let that that previous piece of work continue to move over the hill because we've redefined a work. These are the types of we can have conversations about the strategically, what's being done in his at the right thing for the moment to be working on. And we also have conversations about, whereas something stuck, who were you waiting on or what problem doing to solve? And we can do that just with a synchronous messages by because we have a language and structure for seeing the whole project. So that's really powerful. And then of course, if we actually are spotting problem and there's something needs to be done about it, then we can just call an ad-hoc meeting with the two or three people who are actually relevant for solving that specific thing. And then we're. Using everybody's time. Well, the problem with recurring meeting is that if our plan is to meet on a weekly basis, let's say than people are gonna start queuing their questions up for that week. So if they get stuck on something, they're going to set it aside, and they're going to hold it for the next time that we meet or if there's a problem and there's an error in terms of the sequencing or somebody's not raising their hand on something they're stuck on. That's not going to get exposed until the next week comes by and with the schedules were working on. I mean, we, we wanna make progress. We actually ship work on a six or eight week basis all the time. I mean, we have a pretty fast cadence, and if we wanna do that, we don't want to sort of sit around waiting for issues to raise and then also interrupting everybody no matter where they are so that they can all sit around and have this long conversation, right? So basically the synchronous communication frees everybody up so that they can work on what they're working on. They don't need to be interrupted. And then if something comes up. That is a block or something that people are stuck on. We can deal with it in an ad hoc way as it arises. So this is a huge, huge productivity game. Brian has shared his approach to managing progress element projects, except that project rarely goes plan differentiate between the work of figuring out what to do and actually doing it isolate independent parts of a project and work on the PC that our stock or unknown next up is the benchmark. Let's your Ryan reflects the next questions. We ask all interviewees that's themselves. I would say probably the best intro to all this is the how we work video series. So if you go to, it's called getting real. That's the channel that we have on YouTube. So you go to getting real and you'll see videos called how we work. And in particular, how we work number five is a demo of defining project, breaking it out into separate scopes and then tracking the scopes on the hill, and you'll see how we do it in basecamp. And that'll kind of make all of this concrete. My absolute favorite way is using an interview technique that is is like a criminal interrogation. So you start with somebody who demonstrated that they have some skin in the game by making a purchase or investing time and energy into writing support ticket. And you take that moment the purchase of the product or the writing, the support ticket, and you trace it backward to figure out what led them up to doing that. So instead it's saying, would you like about the product or what do you not like, or what do you want to do? Or what do you not want? We say what happened? And it allows us to just get the facts, but we can connected into a chain of causing affect and then by tracing backward, we learn the situations that actually lead people to buy the product or replace the product of something else or request a feature. You know, because we have this problem where somebody says, hey, you guys should build a calendar into. Base camp and it's like, well, I don't know what that means. I don't know how much of a calendar join need the Bill to make that person happy. Right. But if I play the story backward and I figure out that they were trying to schedule a resource and they just needed to be able to see open spaces figure where they could schedule something. Now I've got a definition of a calendar that is at least ten x simpler because I know what to focus on to solve the problems for that. So tracing back to the situation is a really, really powerful technique. I have sort of nerdy books on my list right now. So I've been reading some papers by Harry crane. He's a mathematician, and he has some interesting ideas about sort of dealing with probability in evidence. I've been reading this product-development flow book which is kind of a, it's actually a little bit technical, but the guy has a major experience in product development in a lot of different businesses. And he talks about some of the stuff that we talked about today, but in a more technical way. So he talks about queueing theory and capacity utilization and how processes get stuck and clogged, and how to sequence work and sort of define the work to be done in a different way so that it flows in that it keeps moving without getting clogged basically. But he does it in a very precise technical ways. So that's been kind of interesting to flip through recently. New product that I've been using outside of work. It's funny actually, just recently had to replace. I had a kindle oasis, which is like the fancy kindle, and it had problems with since I got it. And I finally started having connectivity issues on it, and I just downgraded it to the basic paper white kindle, and it just arrived yesterday and it's funny. It's like half the price and it's way better. It's a much simpler product and it works much better. So that's been kind of good product experience. And that's our show you can connect on Twitter at RJ s and finally to the resources that Ryan recommended this product management dot com. We also recently released a report on how product team set goals in download that at this product management dot com. Slash goals.

Ryan singer Jason freed David ALPHA programmer Spotify Airbnb Indy Twitter Lindemann product development founder Martin Fowler CEO YouTube Ince Harry crane
101. Cloud Native Applications

Code[ish]

30:38 min | 8 months ago

101. Cloud Native Applications

"Hello and welcome to coach. An exploration of the lives of modern developers. Join us as we dive into. Topics like languages are frameworks data and event driven architectures and individual and team productivity all tailored to developers and engineering leaders this episode as part of our deeply technical series. Hello and welcome to the podcast. I'm joe kutner your host for today. I'm an architect working on the roku and salesforce platforms and with me is cornelia davis Today we're going to talk about cloud native the cloud cloud patterns. And hopefully today all learn if i am a cloud native developer so cornelia. Welcome to the podcast can tell us a little bit about yourself. Yeah my name is cornelius. Davis i am currently the cto at. We've works We've works just for those listeners. Who may not know it. You might know. Actually we've worked through a number of open source projects like flux and flagger and we've met in some of those things But we are really in the in the cloud native operations space We have coined the term ceo alexis has coined the term get ops. That's the space that i work in now. But i've spent the last decade or so working in cloud application platforms prior to joining. We verts at the beginning this year. I was at pivotal Where i worked on cloud foundry initially Helped bring that product market and worked with lot of customers to make them successful in the cloud using a cloud native application platform and then later on more in the last four years or so have really focused on cooper netease which ironically is wouldn't call it an application platform. I call it more of a cloud native infrustructure platform And so kind of bringing those two worlds together very cool stuff. Can you define get ops. That's a attorney mentioned in. Can you maybe give a formal definition and talk about what some of the implications are. Yeah so i think that. The simplest formal definition actually doesn't involve the word yet at all. It is cloud native. Operations is the way that i think of it now. Let me draw an analog there. In that i'm one of the things i didn't mention in my intros that also the author of a book called cloud native patterns and that book is targeted at developers software developers architects. The who are building these highly distributed applications these micro service based applications and helping them understand all the patterns that you have to put in place to be able to make these micro services based apps work in this ever changing environment that they run in and i think as an industry we have really come to understand those cloud native patterns from a software architecture perspective quite well so as an industry. If you remember a number of years ago martin fowler said. If you're gonna do micro service you have to be this tall to ride the ride right. And so we figured out with those patterns were too so that we could you know circuit breakers and retry patterns and things like that. We have not as an industry figured out what it means really to do. Operations in this cloud setting where everything is highly distributed and constantly changing. And that's really. What get ops is focused on. Its focused on a new paradigm for doing operations. Now the fact that get is in there. It's a snazzy name. And i'd like to say that guesses the central square on the buzzword bingo card these days so it's it's kind of a snazzy term but i'd like to emphasize the ops part more than the get part get does play a role in that a couple of the key patterns are declared of configuration and having a version immutable version. History of those declared of configurations and. It happens to be a really good tool to do that. So the gift is really kind of hinting. At one of these cloud native operational patterns. But i think of get ops is the whole broader spectrum of the set of patterns that were used to do operations in the cloud native setting. Yeah that makes sense so in your book You have a statement that i think is related to this and i really like that. The cloud is where we are doing. Computing and cloud. Native is how we're doing it. And so you consider gaps or those operational patterns as part of that. How the right exactly. And it's it's more again now on the the how of how we keep these things running in production. How do we upgrade them. How do we. So one of the patterns in the book actually starts to talk about different deployment things like blue. Green deployments or canary deployments. And it talks about those things. I actually talk about them in the book from the concept but then i talk about the architectural patterns that need to be in place in your application to support those operational patterns a progressive delivery. I consider the operational patterns to be. And i've actually heard somebody wants say that that the software architecture patterns that we have they referred to it as designing for operations and so it's designing for cloud native operations the day. I read that that statement. I had a doctor's appointment and i was talking to my doctor and he asked you know what what do you do for a living. What kind of software. And so. I told him i worked in the cloud and that led to him talking about where he stores his photos and i sort of just let them. Go like yeah. Sure not really the same thing. Yep but i feel like that's going to help me articulate. Just what. I do on a day to day basis better so thank you for that my pleasure. There's such a thing as running in the cloud and not being cloud native. Are there anti patterns. That were maybe falling into his traps. Yeah i mean that's absolutely a big part of what we've been doing is an industry is helping people understand that going to the cloud so going to the place of the cloud doesn't mean that you're doing things in the cloud native way. In fact i mean the first four words in chapter one of my book are. It's not amazon's fault and i start off by talking about an act outage that amazon had an how there were a whole bunch of online properties like imdb an even nest that were down for longer than the aws outage was because by the time the outage came back in the outage was like five hours By the time they recovered after the outage was resolved. They were offline for six seven. Eight hours but net flicks was they. Actually i have a quote in there. Were they in any blog post. They said yeah. We suffered a brief availability blip. I mean for them. It was literally a shoulder shrug. And that's the difference between cloud native and being in the cloud so if we don't follow some of these patterns and again going back to the operations in the software architecture patterns if we don't follow the software architecture patterns there's a whole host of things that can go wrong but then even if we do that in don't adopt these new operational practices than again. We're going to be in a world of hurt Because when things change when amazon has that outage which they're gonna have. They never promised you that they'd never have an outage or region outage. That's they give you multiple agencies or multiple regions. It's up to you to embrace those. And so yeah we do things i talk in the book about. Please don't use sticky sessions when you do sticky sessions than what that does is it binds your user experience to a particular node and now you can't even apply some of these operational patterns. You can't do a rolling upgrade or you can't do you have to take a maintenance window and you have to wait for things to drain away and all of that stuff. And how do you know when when sticky session can be drained away because it's no longer an open socket at something else and there's lots of mistakes that we can make and again. I think we're getting better at it from the architectural patterns. That were still still have a ways to go on the operation side. Yeah i think. That's that sticky sessions are a great example of of those anti patterns. And i think we're definitely seeing the industry start to move away from those And in my mind relates very much to the twelfth factor app because that was one of those patterns that when we would talk about the twelfth factor app and stateless processes Was something that we had to as a community as an industry. Sort of move ourselves forward from So i think there are some other examples that you talk about in your book different patterns. That are very much related. I think you have a great chapter Talking about configuration and environment variables Can you talk a little bit about the different types of configurations and really how i think maybe some. The ideas in your book have gone further than was originally stated in the twelfth factor out so on application configuration. What i think is super interesting about application. Configuration is the relationship to life cycle in. So that's a big part of what i did in in that chapter and i think application life cycle and configuration were to adjoining chapters if i remember correctly And i did that on purpose because they because of the relationship between them now even today. I still find what i consider application configuration deeply embedded in inside of a code base. You know there's there's some value that's in there now. We've gotten better at that. We've pulled that out at least put it into a properties file But then there's the question of is the properties file compiled into the binary that's distributed or is that something that is added later when you're using something like this spring framework. You can have other ways of doing that and now. The twelfth factor app actually suggested well. Let's use something completely neutral. Let's not worry about whether it is a dot properties file and java or something else for ruby or something else for c. shop or something like that. There's something that's uniform across all of these different environments and that's environment variables so let's use environment variables in what's cool about the environment. Variables is it allows you then to draw them from a number of different sources so you can use some of these operational patterns something like get ups for example you can use that to deliver environment variables or you could use some type of configuration service. Or there's a number of different ways that you can inject those things into environment variables and so it's nice from that perspective. The environment variables are nice from that perspective. In that it allows you from within your code to say look. I don't have to worry about the mechanism. I know that there's something that's ubiquitous across all of all of these environments the relationship though to application life cycle is really interesting because so when you change that environment variable. Let's say somebody comes along and wants to do credential rotation or something like that. You change a value. An environment variable. What what is the cycle picking up that environment variable from within the code and this is where the relationship to operational patterns is really interesting as well and tie back to the whole notion of twelve factor. Apps and statelessness is that let's take a containers based way. Let's say like something like cuban. Eddie's say my thing is running in that container and i want to deliver new application configuration. This is actually orthogonal to whether it's an environment variable or not. I want refresh the configuration one of the ways that i can do it as i can. Just say you know what. I'll just get rid of that pod and i'll stand up a new one it state lists and that allows us some flexibility. It allows us to have some application code. That maybe isn't designed in such a way that you can kick it when there's new configuration and have it re initialized itself. Maybe it only can do that on initialisation. But now you've applied some of these other patterns like statelessness that allow me to have an operational pattern. That i can use to refresh the configuration. I love that those ideas sort of hearken back to the the principles of the twelve factor app at i think you framed it as the neutrality of environment variables but still leaves space for accommodating. Some of these new concerns Like for example as we talk about containers. And i think start to see this on the roku platform in the future to other mechanisms for for providing in a neutral way secrets. And things like that. That gives you the flexibility to for example roll credentials without restarting the process. Those kinds of things. Which of course means that. The process has to embrace that that puts an onus on the process to be able to refresh people without having to be rebooted. You have these three categories in your book of cognitive apps cloud need a data and cloud native interactions and i see between like apps and interactions potentially that boundary where i think a platform like roku being able to handle certain interaction patterns for you but i'm not sure if it's as clear cut as all of the interaction patterns so i'm curious if you have a better framework or or definition for how to decompose. Yeah and what's interesting is that i have this very firm belief that like i said i have about thirty or so patterns that i cover and they're listed on the inside covers of the book and i cover all of them but honestly i think that the ones the developers responsible for implementing themselves is probably maybe only a third of those certainly not more than half of them because they can leverage a number of other things. For those implementations now the platform i think is one of the places that you can have that in your right that there are A number of patterns but even before we go there it's not only the platforms. It's also the frameworks and so one of the things that i did in the book was i was using the spring framework and the spring framework implements. Many of those are provides an implementation. Now it's you up to use the developer to know that implementation exists in the framework to included to configure it. The right way and all of that stuff. It's also by the way up to you as a developer to know exactly what the platform offers and so. That's why i say it's important for the developer to understand those thirty patterns but they don't necessarily have to implement them but one of the things. I find it really interesting here is that there are some examples in the book where i use the spring framework to the pattern in a concrete example. So i talked about the pattern. Explain it and then i use it. Concrete example that my readers can actually you know follow along. They couldn't pull it out of the get hub repository some of those patterns. If i was writing the book today if i was writing that chapter today as opposed to writing it two years ago or three years ago i would probably have the platform do so for example some of those patterns are now implemented in sto or implemented in a service mesh an so that kind of transition from even less responsibility on the developer to do the implementation in and then tested within their put. They're actually now using platform primitive to leverage that pattern to make their software resilient and have certain characteristics in the way that it runs in the cloud does that reflect a sort of maturity in the industry that these things are becoming more integral to the technologies would choose as part of our platform. I absolutely think so. I think it's it's an indication of maturity of platforms in general. And you know. I spend a lotta time with cooper nowadays and i think that the reason that cooper netease took off wasn't because it was the best orchestration container orchestration engine out there but because it was the best it was a platform for building platforms. It's designed in such a way that you can add something like service mesh adds something like sto an envoy to the mix and now some of the things that cloud foundry for example. We had a number of services that we created for you. That would tie in so here was where we were actually starting to take some of the things that netflix had done. So they had created for example eureka. We offered that as a service on the platform but then it in particular integrated best with the framework which is the spring framework so there were still a tie there. And you're right. I think i hadn't even thought of that but thinking about it. What we're seeing is kind of this next level of maturity. Where now in fact. I don't need to do anything on the code side. I take all of that. That was in the framework implement in the sidecar for example. But that does introduce some new patterns. Like now you're just talking to local host and you no longer have to worry. Your local code can just talk to local host in. It doesn't have to worry about doing this service discovery protocol because you've taken client side load balancing and said okay. I'm going to let the sidecar deal with that. I'm gonna let on void deal with that for example so going along with that. I think there is one thing in your book that might disagree with or at least terms of its certainty. And you made a statement that Writing cloud native software is complex. Just flat out complex and so. I'm curious as if it has to be that way because as these platforms mature as our as we sort of kick those old habits down the road you know. Is it possible that a software developer who is not solving necessarily complex problems but just trying to build some something that's valuable to their business can still have all cloud native characteristics like the resiliency in adapting to change without the software and the apps. Being complex is that possible or is it just inherently complex. Yeah no i in. You know what. We're not going to actually disagree on this. That might all. I guess it's a scoping question as a whole when you look at something like netflix. The netflix application or the facebook application. Or something like that. That is absolutely a complex system. It's got lots of moving parts and the only way that we've been able to manage things like that. At the scale that facebook and google in those properties reached was because they are really good at understanding what are the not only the software architecture patterns but the operational patterns these. Are you know the unicorns have figured out. Those cloud native operational patterns so as a whole bets there but i think that we have achieved the right thing if somebody who's working on a component within that system isn't burdened with that complexity and. I think that is absolutely achievable. And have we achieved that completely. I would say probably not. I think in places we have but again going back to the cooper nettie space today today. The developers are increasingly. Asked to not only worry about the the code in the processes in the multi threading that they have within their single micro service but they are also asked to understand what it looks like from a deployment perspective in those types of things. And i think that we haven't nailed that substrate yet that makes it easy for services to be consumed into the complex and maybe were sixty percent of the way they are to make the micro service developers life a little bit easier. So you mentioned micro services and a lot of these patterns are directly related to micro services whether it's like service discovery or circuit breakers. Maybe this is a restating of the question. I just asked before but is there room for the monoliths in a cognitive architecture and these cognitive patterns. Yeah i mean that's a really great question. I think that monoliths in general break so many of the patterns so right out of the gate. You're gonna be burdened with with problems because you've now you're you're now running. Let's say cloud native platform or in an environment like that. That assumes those patterns in that can do things based on those patterns based on that assumption of those patterns being there beyond that there are definitely monoliths there where the internal architecture is component sized. Done a good job creating separate services. And maybe you're even doing statelessness in if if i'm following this patterns. Why am i not calling that a micro services architecture. Well it has to do with even the way that you bundle those things together. and so. There's this idea of this monolithic bundle. I've done micro services on the inside and that type of monolith will probably work well reasonably. Well it depends on again how. Those components are deployed in those types of things. But it isn't going to solve some of the other things that these more loosely coupled micro services architecture do and things like having independent release cycles being able to do independent blue-green being able to create bulkheads between the different micro services. And so i would say that. The answer is i mean. I am not a purist. I'm a scientist. And i want to be able to take things that. Don't follow twelve factors. Maybe they only follow four factors. And i want to be able to bring some of those things over and whether they get this like negative label of monoliths or not. I'm very very interested. In bringing those into the fold and getting their incrementally so in short. I think that there is room for some level of monolithic architectures monolithic applications in the cloud. And starting to make its way into cloud native. And i think the reality is. I work a lot with large enterprises. The reality is that we can't just rebuild everything greenfield we need to get there incrementally. Okay so I'd like to come back to the twelfth factor app because Just love it so much. And i talk about it so much One of the things that i think you do really well in your book is cover some topics that may have been missed in the twelve factor app or like often say if we were going to write it again from scratch. Today there'd be a whole bunch of things that we would add in two of those are related their visibility and then logging and metrics. So i think in in visibility you talk about health and points. Can you talk about why that's important. And how it fits into the cloud native patterns. Yes so i love that you you talked about visibility with respect to These probes in. I'm going to use the term. That cooper netease they have you know. These health probes in these health points. Although you're as developer of these services you're responsible for implementing them would coober netease does is it allows you to tap into those do some interesting things with that in fact. I think that's really the answer to your question. Which is okay. I've got a bunch of different components and we talked about the fact that i've got lots of components that come together into a relatively complex whole you know whole Whol e. so there's a there's a a larger i'm digital offering that is made up of composed of all these little micro services and it's one thing to know whether one piece of that is working fine and that is something that from a monitoring perspective. You could just say okay. Well i'm going to monitor these components. What the health check does is. It allows us to actually start to get some behaviors where we stitch those things together. So it allows for example. A client micro service to. Let's say you've got some behavior in there. You wanted to find some behavior in the client micro service on consuming a another service so downstream service. You have a system going back to platforms. You have a system. Like cooper netease that is constantly watching those health points and then can actually help the upstream service do something different based on the way that that downstream service is appearing so used the term visibility. So it's not just visibility in terms of dashboard. I think a lot of times when we hear visibility. We think okay well. I'm just going to bubble things to dashboard. But visibility is super important for being able to orchestrate things automate things as well and that's a big part of what i see these probes in these health end points are about. I can tell you that ten years ago. I didn't really think about programming. Health end points so that some automated robot could do something on my behalf depending on what that value is. But that's exactly what kuban netease does. That's how cooper netease knows. Hey if there's something there's health probe that says if i'm not getting back in response from that health prob- i'm gonna throw away. This pod and i'm going to create a new pod. I'm going to create a new container instance for you. So that's the mindshift that we need to do as developers is to think about robots. The consumers not just a dashboard somewhere. So the subtitle of your book is Designing change tolerant software. And i really like how several times throughout the book you characterize cloudy patterns of apps as being able to adjust to new conditions in it in adapt to change. So i'm wondering if you can give you some examples of the kind of change that you're talking about. Maybe something that you've experienced. So i mean that is really what characterizes the cloud just about more than anything else. I mean highly distributed and. I've talked about that a few times. So highly distributed by services all over the place network latency networks going in and out and in fact that network latency in networks going in and out kind of hints at that change. That's one of the many many things that can change in the environment and so that changed. Tolerance is partly there to say. Hey stop assuming that your infrastructure is going to be stable because your infrastructure is going to change. We talked about the the example of. It's not amazon's fault. You know this. That's one of the things that i always say as if you as as an administrator as an operations person if you ever catch yourself thinking okay i'm gonna run the script and then i'll be done that that done word is a bad word in the cloud. Because you're never done. There's always some change that's happening but there's also changed. That is intentional. Change that you want to be able to enable so i'll give you a concrete example here one of the things that we used to think ten fifteen years ago. We used to think that the way that we achieved security in our software systems was to go through this rigorous process of analyzing everything we had the security office in. We had the change control people. And we this is part of what has slowed us down significantly in being able to do deployments but we did it. It was something we were willing to put up with because we felt like the way that we could be secure as to get everything set up in a secure way and then not allow it to change as we've gotten better and better at the cloud we started to realize that there's some really interesting patterns and we've started to realize that in fact security comes through constant change so for example the way that we have historically tried to protect ourselves from malware is to have malware scanners but the malware scanners depend on recognizing the signatures of malware signatures that are coming across networks or signatures. That are coming across in. I don't know computer profiles the start seeing spikes on some regular basis and those are the things that we tried to look for but what if instead we said you know what we recognize the fact that we're never going to be able to see those all those signatures. There's gonna be some clever hacker that gets malware in there that is going to be in detail. Undetectable what if we just throw away. That container instance every single day and so now the malware. Maybe it gets in there but it doesn't live there for six months collecting credit card numbers right and so that is something where this intentional change. This allows us a completely different security posture but it requires that we build our software in such a way that it can tolerate that changing condition in the changing condition being ham. Just gonna reboot you. Well thank you for joining us today. Cornelia a really been a pleasure to talk if you are interested in learning more about cloud native patterns and cutting of applications will include a link to your book coordinated patterns from manning In the show notes thanks again. Thanks so much for having me. It's been a real delight. Thanks for joining us for this episode of the podcast. Coach has produced by her roku the easiest way to deploy manage and scale your applications in the cloud. If you like to learn more about or any of whose podcasts please visit hiroko. Dot com slash podcast.

cooper netease joe kutner cornelia davis amazon netease martin fowler netflix cornelia salesforce cornelius alexis Davis Eddie facebook eureka cooper kuban netease greenfield google
Episode 189  Reactive DDD with Vaughn Vernon

The 6 Figure Developer Podcast

43:30 min | 4 months ago

Episode 189 Reactive DDD with Vaughn Vernon

"Came and welcome into six-figure developer. Podcast the podcast where we talk about new and exciting technologies professional development clean code advancement and more. I'm john calloway. I'm clayton hunt. And i'm john nash. With us today as vaughn. Vernon vaughn is an entrepreneur software developer and architect with more than thirty five years of experience in a broad range of business domains. Vaughn is leading expert in domain driven design and reactive and a champion of simplicity. Welcomes on thank you for having me on show. Yeah so before we kind of jump into the mita things. Would you give us a little Introduction to yourself perhaps tell us like How you got started in the industry. So i started all saum developing software especially fashionably month main. Three was very bad. Time economic I don't know how many people around still remembered after our jobs back while she had blow by experienced halsey. Ics or his job especially in the city where i was are won't financial stuff so ponson salk once so anyway i did finally get work and i would like to see. The best thing never happens will offer. john paul said. I landed no in mma company that well. They actually Overall on working sheri- Were during house in. the usa. saw off He sees the. Us were pulled possibles for church claim elections and while in nineteen meet three the such. The ideology Fuse were once. These gorgeous day of. I got started that way had went from. Cdc plus plus small talk c. Plus plus for awhile. Java mostly gianluigi sala on training subsidence pop. Okay so yeah. That kind of brings us to what water. What are the things you're working on these of the sort of problems you solvent and in cable first of all i said sign with the fives helping them understand. O meter design training. Salting own sue for understand the difference between sort of the traditional software development project. One that interfaces a lot of business with with this since and it is differ. Find seven different mindset. Were on hormones voyage. Second sort of pat on golf all the see of areas and all of those wounds work together this is the challenge of also challenging just understand language and how human imitation is so importance when it comes to writing software that really reflects who vision that we have and we don't necessarily get a lot of thought staff and so will force suit to step into a deal with the fact that this really is. What now anway talk about wrote about Recognize still some early seventies And he had it right. Get it right which proves it's true right because nobody Wants to believes you up of right. Anyway that's what it's all about mutation and get that right and reflected in new software than on ways law with south for you. We well than that strange dog. And i think you're considered one of the big influential people in the domain driven design space. And i think that all kind of ties in together with making sure that writing software that conforms to those ideas and in communicates those ideas you wanna give. Maybe just a brief overview of your thoughts on domain driven design. I probably was using the tenets of of injured design arm back in low for early long this and even though another ten years of so So it was really interviews. We ideas came from small talk. A lot of salt. Patters will a things like that on so just sort of the model shore way of doing things never been. Fury separated in business logic model. We try to make model understand human dynamics whatever your Knowledge was working also gave me a certain of course using onset on that has flirted. Listen also. I was doing a lot of this stuff. Welcome for we've thought seems were really really advocates. On for language that we use we think of as language at the moore's naming naming will very big thing and our will lose dot Was it was all but design and even bounded context which we had. No you're sort of just because the silo which we wrote software small opponents that Had eight the is between them of the signs. A lot of the cell phone on mark you saw was desktop also palm so we sort of have those values anyway. So you know. When whenever i learned about this formal yeah of the injured desire which i had never heard of stressed that way but reading was just like. Wow what. It actually came down. Sue is like this book was now sort of like something where i can hold up. What i've talked about a whole make. Tell me that. I was be fool because things i was talking about real more. When you can read it can say look or other people that have a similar background or different but using some of the same taliban from small world Now it's sorta hard to go like. Oh it either. Martin fowler four for that right so now. It's sorta like hard to say she. So yeah so just trying to be a voice for folks who may not be all that familiar but you mentioned the concept of bounded context. Would you just give a little bit of definition to understand that terminology for them when we have discussions about software. We tend to talk to people who know a certain subject will often referred to as subject matter. Experts business expert domain. Exeter's lump paulsen sprouts are vein lawler's and so what you do. Is you tend to gravity for certain people from apple expertise and have conversations about this very specific area and wanted contain now a model within that obligational boundary. That have where you're saying. Well when we talk about and hells he means very specific in this discussion. On this conversation we're having will be subject matter. Experts and accountants this. And so we're gonna put a foundry ramble going to start collecting. Were language around. You know by versing on this topic. And as we several more and more language that fits incidents related Authorizations that were having then these will then we keep adding to that language but then there's going to a point where he will start talking about An house as something navy. A little bit like what we were talking about over missile But not exactly like the mistake that most software projects will systems will will make is they will try to make those multiple conceptual cows together like in one place and instead though No they're differ so just put them in different contexts. You can call them all enel. I'd if there are. Three different sort of definitions are wasted. Faith about health than those into separate areas where those conversations have your boundaries contacts and. That's where that belongs. And then if you need to integrate clean those than you sort of translate between your language Which we exchange information am art form relationships with we've of teams of these over multiple It's a new sort of decide. Well you know we're a bit more independent. They are so they're they want to exchange information with us. Tell them what we know about our thing. But they don't get you know we don't have to chair about a million about their site of the world and so that puts us upstream so now we have the sole language a model was a hatter language for having these offices and the rations in collaboration and so everybody who's working in this kind of march system. Let's trying to fill large system for smaller subsystems. We all know what we're talking about with each other. Does i say well. This is our language in our anex and we have an open service and published language that you can integrate with but we're upstream so we're not gonna learn about your language so if you want you can translate our live with your language Eruption layer Business so we don't care what you can actually perform for model must see. These are all the discussions as As as natural language were very Technology and seeing positions. You will see all right that you can see how these concepts starts sap together again. It's just conway's law go back snakes and sixty seven and we're just finally learning and begging people to learn along with us and this is where it's worth how has dd evolved in the last fifteen years or so. What what are the do's and don'ts and what did they have right. Originally hasn't hasn't changed much since earlier. Definitions has evolved to encompass different ideas. Different thoughts well. Of course yes. I would say that anyone who offers the fog you you may have a slightly different slant on things. We all have our different ways of more mental models of things and also weaken those people in certain ways that i think of the year round with contact center on context happening harris so forth. Ugly much the same. There are a few new pads. Around that eric evans with something. Palpable arm fast. Which is where working legacy and not sort of subjects Lot of whom shipments knows with already in that system to make this a bubble around that I think there are other ensued or maybe slightly different won't find for example with anybody but shortly after his book was published than will handle that what was in their suit out. More more development started having more positions about the lady says because invest New very heavy all stressing things that happen as knowing have these valves round. Don't want don't want various funds heck's thinking other on sexting. What happened when apple. You think apple instead. We still okay this apple. Well we've had that onset around for a long time of some observer padded whatever so now it's just formalized as part of the having lane language being fined. Soon i would say also that earth talk about mostly generic of the names but some donate has now stand out more than just a generic sub domain. It's more like As our our concept of have a vein now there are various subcommittees been part of those subsidies navy in a legacy That's just Is implicit and they're not really explicitly. nice alary Had originally designed also. How do we deal is Hard that that sub domain is over here in this really tangled esi area. We also need to have arte that implemented now new area how we more exchange information make that will really smooth for the innovative area of a system. And i think the biggest thing that has changed is the number of different ways that hackers designed and that really gets a lot of people's attention because we're often talking programmers adopt naked Languages program herod is always doing our functional. Whatever for pull me year. And i don't say that critically it's just that in a sense what matters about the tackle implementation is that people understand and reflects the way people are thinking about the views of paradigm The any right host tactically that no one can really sees the business. You know that language in there who chairs like well a lot as you did it. I so so instead right. Whatever amuse make sure that you're making it very clear solicit what this model does that. Even this people nontechnical people from work at state Totally other seven priest. Lex way we've been talking Every so you know. I don't think anyone really had an idea of what he will be. Like for functional language act in two thousand three years something but now think about fat a lot so that has changed one of the refinements or extensions to dd over. The years has been What you call reactive dvd Could you explain what that is. Yes so well. It's really just lending reactive programming if you go to the reactive manifesto the acting manifesto dot or others explanation there. That what reactors are who react programs out and essentially what it boils down to is. That reactive is message drill foundation we have to save. We are sending messages from aids and aids of the might be subbing entity or were another vow you know about contacts Know just just think of here. Two components one sends a message to another and that is they by a reactive mountain house. You have this underlying foundation of In enables other characteristics up reactive which is so. Now because you're sending messages that are not strongly coupled to wo- some parts of the system make very Jail alcs wall us still back in than the more responsive has as low continues on the front end. Who were wherever access is made possible service. Let's say that our services also he reacted in also last has its elastic expanded contract as needed to load while that helps us continues response when we remain responsive and alaska. Because we're a resilient we know if something goes wrong over here how do we keep from no cascading Nail you're modeling across our service on processing of services use so a lot of people senators shy away from react as they consider it mma serious Message central message driven. He'll just go like whoa wait. I can't handle you know what what is found is first of all. There is nothing proactive. Anything coaxial on this fall. everything's the Think about when you say you're being active about something you're not reacting to something and you think you know and that's why i gave a ceelo last week by the title of chino was. Would i hope that Proactive is ally control injury. I think actually better title with control is in the right. So now what we're emitting. We'd like to make ourselves and everybody else who we're that. We are proactively in control than software and in reality we are and so what we end. Be ready for his that. Things will go wrong that we can think of everything. Go wrong or at least a lot of them than we can be cared. She react when those things go well. The other thing is message sending we've been doing message seven for years. Just you know. Small talk was all about message. Doesn't mean that it has to be synchronous just means that there is a decoupling between an object saying what wants another objects do and how that object action does that for us right and that is the thing is the messages sent an. It's the same way with an interface. Just think of an interface in c sharp or java or whatever as a as a protocol and all of that protocol says is the messages that something most how to handle. And if you send that message to it it will determine holiday with us and tell you how about where that becomes really crazy is all these debtors center right for. Nemec wear the people programme all time. Well their heart is really a message since he. Because it's obvious what's happening. Just hoping data into object in getting data out of doing things right. We always seem to rag things up to the next level up. Do something there in the mortgage strapped up to the next level above that and then we start pushing things back down for. We finally have an answer. We just set things rights number. No that's actually Of the which should you go back to. Any elementary object oriented way of of explanations of no. No that's that's exactly the opposite of what we should. So actually messenger is really familiar. And so is reacting. We want to understand react. Reacted this conversational have overreacted. Why because you asked me a question. I reacted. I'm probably overreacting. As i'm talking too much trying to help people understand themselves. Look at your light single thing you action. Just look at you. What happens when you prefer warmer. Growls event that says on click right. That's a message. People have been doing reacted over having for so long that they forget the they don't even realize that is exactly what reactive is it's just that now we're rating that so other parts of the software where we think that. We're in control of something that we're proactive. But we really are. What are your preferred tactical patterns for building. Dd systems. how can we make sure that we don't get into this habit of doing more than we didn't just creating a big ball of mud. Are there specific things that we should keep in mind as we're designing our systems. Well the big ball of mud has less sue. Proposals said more of out. He conversations that have about is generally ause by having submitting conversations in one place and having you know five x fruits from different areas of business having conversations altogether and we're just and we recognize the boundaries just pushing things into place and and then we start duplicated things where we thought there was nothing like this elsewhere so looking for knowledge which is is really what we should be actually preventing rise on himself. Because it's the knowledge that has leagues. it's not thinking about dry s own. Remove his code the code those exactly holds reflects the same knowledge that it does somewhere else fracturing entangle and so forth. All apple's very much we'll be terms of but knowing where to pump recessional valves are really as long as tackles that views are are able to stress the conversations in a meaningful way to team of business developers seven assault useful speaking of tools. I understand you've got a collection of of open source tools that they can help. Yes so you see my background here. We got a start off of any of lingo which is well as legally isis very much about language and we have a product teams. Zoom zoom is at its Absolutes not like zoom Haitian software we're using right now facility Sdk which actually has if Very unique in the sense that start off with some high level designs individuals desire. Not at all limiting orton training. And then you actually start out in sort of a love boat mode when you're happy about it since over in fort hood mode as the full software life cycle so point of this is actually list all of the overhead of liens surrounding architectural mechanisms. That usually said way too much time. Implementing we just nickel's worth for you. Even to the point where incoming rest requests were incoming delivery from message broker message process is automatically translate and the sach on the lame model so literally all you have to think about the domain model water. The business roles. Why is this incense and even to the point where you now to events. In the database event from and i haven't get residency so tactical will into a journal that only recieve view but essentially that bat will get broadcast others published. Outs others either now okay. I've got a hundred events slowing around the system. How do we know who's depends on what How do we keep the version incorrect. While we have allsteel registering out food will zoom that maintains the over zhen's so it's sort of a neutral way which will it yourself roll the textual language that those incident scene registry but when someone hands on that or language civic organization has compiled code in a special place. Where you can you can use. It can't change it a mile. Never benefits of type safety version control over the languages that are used changed. I mean thou concept's and event again the automatic translation of someone else's languages in coming on a message. Oh what do i do. If that one translate bats who have command command in our which i'll i'll just automatically to the For me model and half literally don't have to do that. Were out at some point when you're happy with that sort of low code Environment than venue. Just take over full source tells the service now you can start you need to add. More events has these from other contacts later on. Just ask Now you have a lot of examples and we even Comet bode all over the place with laces documentation that sleighs. How well once you. How does this work while in. Just go directly with us thought. Thought i know and was goal. What what resources will have for people who are trying to get into dvd Or or react with some of the topics we have. I know you mentioned that. You have some workshops That you do. What would the with something that would be good for our listeners. Worse i've taught thousands of people. I don't know we'll have albany before actions hog but probably wore than thousand already Will off down for of line. Avoided that for I teach anything from implementing design which is based on a work that i did read Name at my flagship workshop. For half days which we are I am a member of other ones veteran architecture. Relation with the texture. So for almost like a day we were shot. So am i teach a reactive. Were shot That actually sell. That had a very low cost as its so us importance. You get this message about reacted out there but it's not just good for No local optimization is actually good for loser and you think about you can run software at maximum efficiency. We're going exactly opposite direction. blockchain blockchain slight losing all the energy level You know like if you just think about about what watchings. Doing that way. We're going office. Direction were resume so and so we just want people to know about. This is really the way that computers were invested. Run since about two thousand through our eighteen years in or most people are still not doing so well. So yes so. My workshops definitely I have three books already published Another one on the actor model which is reacted. and then i have a was the one with Or by ho author smith's jesse harris and this focus entitled Mahrussa micro services driving change with Her argued that she so. This is really showing that real soaring of why software should use is for innovation. We need to drive business innovation with software. Let's get real each other if well logical rise model works well for michael services. Don't use it right. The waist leggings money just having fun. Saruman with with technique set Snake with business won't core or those brains cells that you're musings to try to tackle started for my one seven one but revenge right in china and you're trying to get hoover any word for you your company and it's like while we just keep our heads on this issue game on the hash instead to help them with that right so right rightly so they so they get. John asked to help them then they can just send their time making them. This challenge right. That means you're you're less who sort of less rockstar owner than you. Are thomas edison right more. Thomas edison and tesla people like back. While you're thinking in that direction alexander graham bell you know whoever was innovated around messaging light whatever earls. That's what we will be with softball shooting saw before hitting record. You'd you'd mentioned that you're also in addison. Wesley pearson signature author. Yes like Just finishing now just mentioned digi model is. Michael services is within a signature series. That i was so privileged. The asked bikers in. Addison wesley to sue. Were either editor of miles signature series. And i have to say it's embarrassing to needs to think about that. Very influential other people. That has she sues down his head. Back martin fowler on or no i got lots of Body was saying hand. We are hard very vest. Bow seriously Possible on the theme the general theme of organic software while offering longer where chappie ryan for. Never perfect. The all encompassing innovative. All in one point. You can't even have your on sect's absolutely correct. Wait now but organically as sort of like you know. I decided as a scientist who just who served mostly as lab jackson user and worm fear meeting and we find the right way an again. That's true scientists in innovator cerebral as how. That's all the lillian chu innovative ventures. That will pass earn things. Just try and like the wright brothers by giving up on the third crash alon must've crash. You know which. I recently moved from a mary. Kind as Forward also you'd are very critic after. She has worn fantastic software developer. Each year that is to engineer and mary poppins Like said i made a sort of funny awesome tough statement about not very much. as -ccomplish rocket science insulating sixteenth of. She was so wrong. The point well as well considering all the progress of the new other areas innovation other is not so much but when she really liked drew. My attention to what stays access has accomplished as it's mind boggling At what point was you know. How many rockets say crash yet. There and the that a government contracting never probably never be permitted that not. The one safe crashed one rocket. Right i mean very own it when it's not one soon so you know instead spacex like they just said in. We got a crash rockets so we learn what works what doesn't work and even recently right. I mean they didn't crash the walk but it blew up a some minutes after it landed and know what they're gonna learn simple paul and they already have moved out. So what if they got they. Own face diller. Also earl mova so anyway. Yes think of every book as having that mindset in different sort of technical and methodological or were paradigm areas and. That's where. I think we're going to help a lot. So what has been helpful in your career that you might share with those just getting started or those looking to level of their own careers and change. Well some people call it reinventing themselves. I don't know that that's the case it's really. It's really more like just having a better. Better view the Circumstances change all fine. The past year hasn't proved that newest nothing will on so so we've all anybody Survivors has become a better version of ourselves with the new circumstances that face and when that those new circumstances are technology face the greatest determine to win. I will messer or succeeding thomas but with humility. Not knowing that need health and the bar here turn it off of the teams do better work individuals than you can all go together. One of the biggest Vice that i can Any software developer is really have to al green Introvert hannah fry is not that you argument uber. Inter and you and you can. Are you always say no way talking. Too much energy ways. Like i'm sticking this Biden richard. But i have learned over fundus and i will be exhausted after this conversation. I will obvious. Go take back with something. But hey i make it work for what i needed to do. And everybody else do that. Out of seep. Go talk to business. People find a way to get along with them. He make a partnership for market research. Says what is eleven years ago now softwares eating world well yes and no software is eating world. One faces eating businesses in other cases in that bright The business that's trying to see this just going in the wrong way as they think they can separate whistles. People in software offers from things over cubicles to the software developers or the other way round august workout. And how many times does that feel for a word. It doesn't actually to Where can our listeners. Go to follow you and just to keep up with what you're working on on. Twitter i m at vaughn vernon and i am sorry that my name is so think of remember and so forth a. h. d. e. r. initial paths and an-and linked Same same slash in slash all lower fees through. And of course go take a look at hub on slash you linger And then also slash get dot com slash. Leo dash net dot. So we have we have. Jvm and dot net c. Sharp right now will also invest shirts for or you're gonna be amazed. At how actors are yvonne. Thanks so much really appreciate you taking time to speak with us today. Thanks for inviting. Newest vaughn vernon von is an entrepreneur software developer and architect with more than thirty five years of experience in a broad range of business domains. Vaughn is a leading expert in domain driven design in reactive and a champion of simplicity. If you liked this episode please like rate interview on itunes. Fine show notes blog posts and more six-figure dot com catch us live you tweak on twitch and be sure to follow us on twitter at six figure dev. This has been another episode of the six-figure developer. Podcast helping others reach a potential. I'm john calloway. I'm clayton hans. And i am nash.

john calloway clayton hunt Vernon vaughn apple gianluigi sala john nash halsey Martin fowler navy Nemec Vaughn vaughn eric evans john paul lawler aids Cdc usa Exeter taliban
#266 Refactoring your code, like magic with Sourcery

Talk Python To Me

57:05 min | 1 year ago

#266 Refactoring your code, like magic with Sourcery

"Pre factoring your code is a fundamental step on the path to professional maintainable software. We rarely have the perfect picture of what we need to build when we start writing code in attempts to overplay in an over design software more often lead to analysis paralysis rather than ideal outcomes. Join me as I discuss. Re factoring with Brennan McGinnis and Nick Saban as well as their tool sorcery which adds automatic re factoring in the Popular Python Editors. This is talk by any episode. Two hundred and sixty six recorded may twenty first two thousand twenty. Welcome to talk by me. A weekly podcast on python the language the libraries the ecosystem in the personalities. This is your host Michael Kennedy. Follow me on twitter. I'm at an Kennedy. Keep up with show and listen to past episodes at Talk Python Dot. Fm and follow the show on twitter via at by on this episode is sponsored by data dog inland owed please check out what they're offering during their segments. It really helps support the show Brinton. Nick Welcome to talk by me very much. Thank you for having a yeah. It's great to have you here. I'm such a huge fan of re factoring in code quality and all these these ways of like picking living software and making it evolve right. I think long gone are the days of. We have to play this perfectly. And then we're going to build the perfect thing that we've thought up right. And so having this idea of continuous evolving improving code. It just frees you from worrying about trying to get it all right you can just get started. I'm really excited to talk about sorcery in factory with you guys. We'll Shamir we think it's rations super important as well so trying to get a skeleton of the thing up and running and then to tidy up later yeah absolutely kind of how nine one really understands that the main when I wrote to cut it anyway you have to write the kite. Find That old mistakes that you've made and then tied playing time like you say it bows and say what looks nice qualification so aids of uses. Yeah I think that's a super good point because understanding you don't fully understand it until you've gotten mostly into it but if if you tried to understand it all like it's just so much work even if you get it right. It's so much work to get to that point that it you might as well just written it three times. So that's awesome but before we jump into your more that let's just get your stories in Nick. I guess we'll start you first. How y'all get into programming in python so I did was back in school on these graphics calculators. We had back in the nineties. Remember reading a book about complex numbers being super proud of getting I took about twelve set on my little calculator mental brought set or something cool like that could zoom in zoom in but it would take another twelve hours to show you we do it twice because then the battery run out pretty much my little bits of basics and stuff but I never department in a serious way until joining software. Company often investigate. What are you studying at? University says masks and philosophy okay. That's cool combination actually. Yeah and there's like a love logic which is quite cater to programming. The First Languages Island in the Sofa Company were this mainframe language called rbm. Pg I don't know if you've come across that one I could not look at RPG source code and tell you what it is like. I couldn't identify it from code. We have heard of it as a credible. So it's like punchcards. Everything has to be in the same in the right column kind of thing okay. 'cause it was this. Finance Committee Brendan Weather as well. It sounds useful. But I think I'll stick with python. Yeah this at this base from the eighty s. Okay Cool and you have to programmatic. Green screen terminal. So I didn't put me off if you go through that you're definitely good for the industry if you can make it through that. Siv Delfi which back on being she permitting Java and then it was only when I joined a period and doing so machine learning. Thi- gone to buy some Maga's mid twenty ten. Will you like what is it? This language have semi colons. What happened to it or just like a breath of fresh air early during Java. It seems so like why. I can't read this or symbols. All over is trying to tell me. Yeah Yeah Curly Raises Parentheses and semi colons and the way they do the libraries and things. Actually it seems to be all done a very nice way. Yeah indeed ice well Brennan. Have I programming coming did was in my final? Yeh Great so we had a project today. Which was to simulate various questions so one side ties with the hates equation and It would semi light one will bring in another world being cold various obstacles. That's way and let's say wools. And then after Ron which is child the temperature different places in the room. And the other thing off the plane with Quantum mechanics quantum mechanics so interesting and it's just like it. Seems like such a weird twilight zone alternative reality and yet it's it seems to be applied to real reality. It's such a weird world. I love quantum mechanics. It's amazing how accurate predictions and yet when you try some the sound it. It just has a nice making that up already. There's no way this is real. Yeah when you get smollet doesn't match up with what you understand how handicap for object in two places at the same time. What does I even name exactly? Yeah well they. I was just lighting single objects in the whites almost it so it was quite simple The way they did it was his mingy W. Guy probably come and say and then just into and we need nothing about how they might programs with just hundreds of lines statement absolutely nice options. Nothing I think. We've probably similar leaks in there as well as written. Oh yeah the whole thing. Yeah hoping was praised. Terrible under the project and used to me like it correctly and you think compare them now. I probably even if that's not entirely true. I think you do come away with this feeling of like. Oh my gosh look what I built like. This is so awesome. Even if it's not really it just like that feeling of creating. That thing is so cool in those days. When you're getting started I did I. I think yeah the time that I managed to output the hey equation on onto the screen but you say but different temperature across the written provided us for the chopping library see as I was advising wow I can. I can do anything. I'm not computers. Now Yeah Super Colo so yeah authorized finishing a join the same company then I'll PJ. And I let a Java and then I changed quite a senior role in the company. They ought to cut you manage to introduce scholar to the company which yeah we did a language comparison and it was between Jerry languages. So one of the language was a Jay Johnson We've been a guy for that little jerry baby because it wasn't on the phone so I really the main contenders job garnering clay and we had someone come in until about scholar. It was read or anything I'm taking just to to use that and So yeah also. I ended up late in the project brings scholar into the company and Riach sets a lot of the systems that when I left that company. I was very much down the functional programming rate so it's a moat pleasure. She's already functional. Pasco by then I I Join Nick Ontario College. London and That's where he he said he got into the into it. As well site hijackings reading crepe was all about learning and great. I'm wait right a pipe which was cool differential Niro computer and I have no in designing what it meant and I was like okay. I'M GONNA Guy GonNa incidents it so went home and I cry out Census and it to me wake some wakes than for that because I have no idea what was going on so just preventing more and more of these type is is is more reading grapes and attendance. They will a Atari playing rainfall ending algorithms. Just ready talk tonight. Okay cool I learn how to play break out? And then how on and right pitfall and there are never caught on pitiful. That's a good one nice. I'm really fascinated with these ways to train is around like video games. How me I'm blanking on one of the options. But there's there's a handful of libraries that you can kind of like plug you're a into the virtual world so it has somewhere to interact with things to interact with them. Yeah that's fascinating. I really just wanted to pass nights. And that's why thing every week and I mentioned around that time out the guy who came out as well and it was like a shock to the world. Lay this the hottest guy in the humanity can play his faith and Buchan Pizza. Yeah and it's one of the might have been the first real opponent that used like strategizing rather than just deep exploration of the the path right like the the chess one is like well. I can hold twelve steps ahead in every direction in my mind at more than the chessmaster. So we'll just play them all out all the possible futures out and then go down the best one right that. But that's not how go worked. Which is I think part of the magic school year. That's the interesting. Yeah Yeah Yeah it was. It was just back up. I ended up writing of Russian about the guy supply connect full which is actually pretty strong. It's that was bait. May I'm an average alcohol? Alcohol is so. What are you working on the same project yet again? Would you do in day today? What are you doing now? What bugs working on three full-time we kind of slide? Why can't back at the end of two thousand eighteen Brendan's flat so I turn up in them? Abc Beyond Bahamas cereal. Sit Down and away we kind of teddy so making societas. Gooda referring tools is compete. Really cool we'll get into. What sorcery is more laid? But what's the quick elevator pitch before we dive into more software stuff? I guess we've been pitching it. It's Kinda grandma for Code John. Graham leads it's like improves the style of the codes without changing the meaning of the content. I guess that's what we factoring in to improve the quality and the structure without changing the functionality. And that's what we aim to do. So is your writing it. We analyze it and we suggest reflecting improvements. So you go so maybe. I got a loop and doing some kind of accumulation into a list and it could say that could just be a list comprehension. Yeah exactly exactly. Yeah okay. That sounds awesome. It's easy to just get focused on writing code and then not really worrying about the quality it's also equal easily easy to get super obsessed with equality and not. Just get the thing done right. I kind of see like by modal distributions ones like I don't give a crap wrong is at work so I'm just GonNa right right right and you know. That's one group of people's phosphate. The other is there like really slow and super meticulous to get it just right because they wanNA ride it the best way or it sounds to me like the tool would let people kind of find the middle ground right like right it a little more loose and free but then it says oh by the way that things wrote actually could make that way better if you just let us. Yeah that's what we've been getting. That's people write code. Samoa quickly without worrying so much about the Colton them okay. Cool and the other aspect of it is Some people actually I call it. The is so if you have stopping out in price and you might be able to write solution that you necessarily not to rush it. Well so you may not even know about this comprehension yet so right right right and the benefit using socially not case as can teach you is on quay rushing. This portion of python is brought to you by data dog. Are you having trouble visualizing bottlenecks and latency in your APPS and you're not sure where the issue is coming from or how to solve it with data dogs end to end monitoring platform. You can use their customizable built in dashboard to collect metrics and visualize APP performance in real time. Did it all automatically correlates logs and traces at the level of individual requests allowing you to quickly troubleshoot your python application plus their service map automatically plots the flow of requests across your application architecture. So you understand. Dependencies can proactively monitor the performance of your APPs. Be The here that got your APP. Back on track at your company get started today with a free trial at talk by thon dot com slash data dog to. Maybe you don't necessarily know that deums. I think one of the Challenges Python in particular may all languages have this problem but python suffers more than others from it one is that it's so easy to learn that people feel like they can learn it in a weekend and then they just go right code in it right because it's a simple little language is not a whole lot to it which. I think is actually not true. Ran Still feel like I'm learning python every day like I'm getting out I didn't know or I should have done this right like there's all these things there's just there's so much nuance detail to it. But it's easy for people to come from a language like C or Java or something else in just do the Java style programming there or ceased programming there and not like you said not even know that there's this other component right the sale I crappy. It doesn't have you know. Four a numerical four loop. This is crummy right when really a better way to do. It would be to use enumerate collection where you get the index and the item right. You don't go back if you didn't know that that's multiple layers one you have to know. There's a foreign loop and then do you have to know that. Enumerate the thing and then you've got to know about to pull him back in like. That's a pretty complex set of topics if you've why spent the weekend now kind of note? Let's let's finish this project that Fossey later I mean like you said the learning the pipe and language and it's like each place Info. Nice people come pick it up and awake to a months time event. He's got all these items and that and not just the different bits of Syntax. So the and right. There's multiple libraries as the place and ever library built tend to the and main library but also of the libraries out there that you need to accomplish more complicated. Yeah just finding out. What what the Best Library is? The job is a difficult patient research. Often it is. Sometimes it's something built in like inner tools or something like that or other times. It's something you've never heard of. There's two hundred thousand options. They all have their their behaviors and whatnot. But if you would grab it like that would take twenty lines down to one. Probably be faster at the same time. It's like it's incredible right. That's kind of what I was thinking about like. Never Learning Python because like Oh. There's this other standard library module I discovered I was using counter in this way or whatever it is right. There's just all these options. We've definitely come across up so he. Yeah Yeah I can imagine. I can imagine coming from academic space. I'm sure you see that a lot there as well because there's probably a lot of people who don't see themselves as a developer but they're still touching on right coach kind of right once and then That look at again. It's kind of some of the different aiming mind so worried about re-use by the paper is right right in quite a challenge for us because we live hyphen together effectively at the same time and wave never wrestling by and large kite by apart. From all right so we have to learn all of these things selves. So I've a time we've rewrite some bit to the way we found out. Oh we can use this feature piping that makes it so much a and At one point we inscribed with my pie and say okay base and that was a big in pretty been and learn about these options and I think a lot of people out as part probably into that pocket as well then and learning. It says they're not learning it with people who can guide them. How best to use it and it's really hard on your right. I mean whereas they're develop is and I think way finding it really hard to do it right and to take full advantage of it. It is so it's cool to have like extensions for the ID that will sort of not quite appeared program partner but somewhat sort. They're the you know what that actually is not the right way to do it. But it's super easy to fix and I can take care of that for you right. Yeah exactly that's awesome to. We talked about code quality. And that's a little bit in either beholder and it's also a little bit in the trade off like Nikki talked about this concept of. I'm going to write a script and get an answer. Never run it again right. That has a different threshold for Code quality than you know the core trading engine at a bank right like it would probably be in improper to put that much energy into that script is going to be run once right. You should just right the thing and get to work and not worry too much about it but the same time. If you're building up the NEBI reused and is important to be run by lots of people you really want to get it right and so. I think there's a spectrum. People got to figure out where they live on. But no matter where you live you would like to have better code quality rather than less good good quality by like whatever applies to your situation right. Oh definitely news come out to quantify what sometimes when you sit. I guess it was reading the book. Clean Codes Property Martin. Yeah Yeah really crystallized it for me now. It was trying to get the graduate so Neil Company worked out to read it and eventually wants so I think I guess I mean I liked it worked you converted and they stole your book. The real real corvette is the cycle. Dakota's easy to read and understand that just reads like a story this medicine this. Yeah I really love this idea of clean code and the stuff that Martin talks about. He's got some really good ideas. I don't totally agree with everything. Says but I think there's a lot of good lessons to take from what he's doing. Yeah one of the really interesting ideas that comes. From one of his contemporaries Martin Fowler. Way Back the origins of factory. And so like I remember reading. The book called re factoring in nineteen ninety nine or something like that. Just go in. My mind is blown. Like I've had this problem of bad code quality and I've had this problem of like trying to write it well or to fix it and then I realized you know reading what he was talking about like. Oh there's this way to take the bad stuff you've already put down as sediment in this is Krista and like turn that into something that can be improved and grown over time and I just I remember. It really changed my way of thinking about programming digging into refractories. I'm just such a huge fan How'd you guys come across it? It's an interesting one because the based company was so huge and difficult to change the often. We didn't even try referring it. Just kind of he did a surgical approach. She went in and tried to understand that and make the smaller changes. You could. Please don't break. The new feature without Breaking Argus Brendan. You're actually so of data the other person said like okay. I'll just rewrite this. This whole bet. Yeah well I mean it was risk of price in that. The system was no UNBE test. It'll so I would. I May Day that on on ends compliance I the Ui but yeah sometimes it was. It was just a case of. I don't understand what's going on here. I can say well the functionality currently is I'm just GONNA reimplementation scrap right. You know if people are in that space like the whole area of what? Michael Feathers talked about with legacy code and like how to take these these huge systems. That are hard to change that. You don't actually know they don't have tasks and how to like. Break Off Little Bits. That are maintainable that such a cool book working effectively like as he could. I really do enjoy tinkering with codes tracking down bugs so improving it Michaela better. I think when I see a blank sheet of paper or blank screen. Someone's find difficult to brands bit better at that. So I I enjoy reflecting. It's one of the things we're trying to kind of achieve a suicides so of if there's a machine that can do the factoring for you can be less worried about being on the test because stunned pronounces because whenever I do reflect I break something. Ni- I would lean heavily on. The tests is almost never seems like a good idea to May to factoring manually. If there's some sort of like tool based way in which it will happen right. It's just you never know what little thing you're gonna all. There's that one that Krahn job being that we add that was also using that now. Apparently it's not gonNA take it anymore with with by on year. Have compiling right. So you're not gonNA catch the obvious stuff like I moved this function over here. It was like well. Don't run. That part is GONNA crash by one of the challenges with and I mean the way the price with associates. I is thing on my type annotations. Yeah I think they give enormous confidence when you factor in the kite so now is i. Don't say crazy crazy. Rewrites lie incrementally in pretty slow changes are they mentioned without once and realize it's not the white guy. Yeah they've ray okay is having tests Association you can move something anywhere in the system and she'll get told about Oliveira's but now have and then you can go and fix all it is and then you can do the next three five trading and up through that my soapbox from the bench we hadn't Added my two so I so I totally expected to hear automation from you. Guys and applying sorcery back unto itself. And things like this. We'll dig into the features in a second but I didn't expect to hear typing and my pie. I'm personally a huge fan of the type annotations on Python. I think they make working with Python Code so much easier you don't annotate everything but certain places like this function returns one of these just knowing like actually it expects to return is super helpful and it like it'll light up the editors as well right. They can all of a sudden auto complete for what they were before but now they know. Oh here's the five things you can do what you got back perfect well. How does my Pi fit into your world? It gets pretty interesting. Be Doing with that. We really only use internally because my skied out. There doesn't actually at my by type. Annotations right even if it has some type annotations. It's gotta be. There's like sort of a chain of annotations that have to be consistent like my stronger levels than just saying. Oh this function happens. We're trying to list. Yeah I mean one of the great things about type annotations. You can just look at a function out the definition but I'm the united need rate because without those associations you have to actually make the code and say Oh actually. This is a string. I'm this is an instrument on returns listed in Jisr or something like that. Yeah with really point. I like that so I think talk on passions form of documentation that really powerful just from a rate of one day. Then you get old of the security as well as nine when you probably can the coach when you've not something correctly one of the things that is really important to us. We're socially is waiting at the pike other people's type so we have to have extremely strong. Yeah so let's take a step back and let. Why don't you tell people about like what sorcery is? How do people use it? So you mentioned that. It's a plug Fridays. But it was a little bit more detail than we can talk about how it bike. You keep from breaking people's go which is probably people probably appreciate that after the plug into your IV we've got plug it in so they ask Tom Cutting away. It's just there reading okay and I think a change to function that you're working on will embrace Impreza kyw quality though suggested it She can review that suggestion. And if you like it you can accept it. And they'll apply that change it line on carry on kiting is what's on my famously and your work okay. That sounds awesome. Does it change the way the editor to work in other ways like for example does it change the auto completes or things like that or is it really more like a the code intentions like the Light Bulb in charm? Yes like a code intention pie. That's Kinda the thing we've gone with Sarah Underlying so as you're going along you're watch. There's little pop up. I should go see what this is about. Yeah exactly and it runs locally machine. That was quite concerned. People that code things into the cloud at Kennedy started it. Do it kind of on service in the cloud about to do a paper and get it running locally on the machine. Okay so somehow behind this like how does it make decisions about refractories? Is it like in a I saying powder matching like what is it doing inside pets Nine learning or I and the way it works is essentially pop matching. So it's looking for while it was at the level of the abstracts in type straight so takes the cards on the positive into structure on structure will have for instance and if not whether or not if it will have a function co on this night. I have a list of statements. It looks at those nights and it looks for the patterns like you say so for instance. It might look for apology. That has an excitement within the appends to list says okay comprehension. Waiting to be made right there. Yeah exactly and I guess. The clever bit is it kind of has his little. Lots of little tiny little patterns of improvements that can do they can compose those together until like bigger effect during and it's guided by light of code metrics we've kind of incorporate into it so we can get into those data cyclamen complexity and so those types of things. Yeah Yeah so we psychomatic. We do use cognitive bacteria which I think is like a trademark of Sal. Kibo something but it's a different metric we use a few we've written ourselves. And so I can kind of chain together little factoring seducing bigger so for example on the if you've seen the gilded drains re factoring Carter no tell people about it so it's kind of this big fantasy what's a Cada accounting for is a coding exercise to improve. You'll pregnant right right. And there are various ones as such around the gilded Complicated set of nested. If specically up about this fantasy in I think it takes maybe an hour to kind of manually the code out and refectory so of an exercise people to solve a mommy started sorcery was to this was like our initial target. Problem can kind of do at work at once by changing together. Lots of little report drinks so it can take the complex massive Spaghetti code an antenna into something understandable. Right instead of having to say okay. Here's a little if statement that could be improved and then applied again and say well now that we have this code. There's another thing we can improve the plan to get. It'll like chain altogether and go. Actually we could roll up exactly. Yeah because when you're doing manufacturing that's kind of Wolf Napkins. You do thing and then you realize oh now. I can do this if you might name in mind. We might not taught chaining meetings together in the end like. Oh now it says done could base. Yeah a pretty cool. Oh that's super useful. I know that the some re factoring building. The certain tools like has certainly factories. But they don't seem to take this more holistic approach right. They're like Oh this list. This list comprehension could be expanded to a for loop if you needed or something like that. But that's kind of as far as it goes and the kind of vary developer driven enough to know you want to do them. Yeah yesterday no way you can do them and then you have to do them so very useful. If you now you want to do something because it will do it for you like we made mistakes but didn't of an Italian good idea on so are kind of suggesting things that we think are good ideas to actually change. This portion of talk python is brought to you by. Leonod whether you're working on a personal project or managing your enterprises infrastructure. Leonard has the pricing support in scale that you need to take your project to the next level with eleven data centers worldwide including their newest data center in Sydney Australia. Enterprise Grade Hardware S. three compatible storage in the next generation network. Leno delivers the performance. That you expect at a price that you don't get started on the note today with a twenty dollar credit and you get access to native. Ssd storage a forty gigabit network. Industry-leading processors they're revamped cloud manager cloud DOT Llano dot com root access to your server along with their newest. Api and a Python. Cli just visit Talk Python Jonathan Slash on the node when creating a new lagnota account automatically get twenty dollars credit for your next project. Oh and one last thing. They're hiring. Go to dot com slash careers to find out more. Let him know that we sent you the mature one of my favorite concepts from refractory. And then ask you about some of your favorite factoring so there's all these different factories even in the early days. That mark talked about like okay. There's a God object in the here's how you break it down or there's a function with. That's too long. Here's what you do and so on but those are all kind of interestingly the most interesting concept run all that stuff to me was the concept of code smell right just like there's something wrong with this like it's it works but your nose kind of turns out when you look at you like there's something wrong with this part of my coder code probably inherited from somebody else right and the other thing was he would talk about comments and say often comments have value but they they're really just deodorant for these code. Smells like this really hard to understand. Because it's in badly so let me write a comment that tells people what it really means. Yeah and then just leave the bad stuff there right it Kinda deodorizers. The code smell a little bit in the idea of like if you have those comments. It's like this underlying thing of like you should start thinking of applying these different factories so my question to you with like site That out there is. What are some of the favourite refractories that you guys are seen possible with like this? This deeper integration. Right like obviously four loop to comprehension less competition to four loop by those are pretty straightforward but it sounds like there might be the things just love there might be like some more interesting larger factories. The main smell. I think socially is ready well with is eliminating to coach within a function. I'm the within. Different branches of the complicated Five s expressions. So you may have the saying Kay then to different. Places and sorcery can restructure the kind of and so it's just a single condition applies about Booker Code at school. I think in the next. May delete lines are carried is very pleasing so yeah yeah and delete conditionals as well right if possible or simplify them. The I feel like the more. The word legacy gets applied to a code base. The less you WANNA do those kinds of things like I'm pretty sure these things are these. Three things are the same. But I don't WanNa be responsible for what happens if I misunderstand that. These are actually slightly different and try to factor version. So as much as you can get software. Does you go. Actually No. This is totally safe. We got you interesting because One of the things that happens is way suggested race factor in the papal To five on this is incorrect as examples of eliminating kites and some final succession. It's kind of once you went through the logic. Yeah weightlifting discovered late so raise correct and tabs on. Your coach was very confusing possibly have dug in it but she acknowledged this is right right. You saw these two things. We're doing different stuff. But in fact it has no the the effect of this is not what you had in your mind. It's actual effect being misunderstood. What was actually happening. And Your Mental Model. Didn't match the factory in result. But that's because it wasn't actually doing that exactly and so I actually once it's done that race I train you can say Oh actually. There is a buck in my case. I can fix that. I have the trust in rate tonight. Correct before you willing type. That takes a little bit of usage to that trust to how do you guys check. That INS ARE VALID. So like we said this we have a library of small razorbacks rings and then we have such engine. That composes is the by impulsive. Thing is making sure each individual response in the libraries. Correct right right. 'cause composing a bunch of things that are correct is not to break anything exactly. Yeah so the time to try and make sure the individual ones direct. Sorry we have lots and lots of tests on my tests are the full hair of so I'm here. They expected so skied on freight train. There's an old state of ice. There's also a multitude of tasks of the phone. His face of social sky that looks similar. Today's other bits that you have five but you should have raised Batra because if you to refinance right if you do make the Chinese you're right because I'm expressing in there. Yeah that's true rape by train and it will tend to be things like your cooling function and say you calm. Swap face statements because one of them is actually global bearable cited. We have an awful lot of analysis which determines walk statements and depend on the other statements. Yeah sounds interesting. It turns out how the problem that we've tried this. All I can imagine. Have you guys looked at using things like hypothesis or other property based testing where it's like? Here's a block of code apply referring to it beat it a bunch of inputs to both versions and see long as you get the same outputs. Are things like that. Got A pizza plan that we have. We have a second floor almost in the AM which is as PA bill process. We run sorcery. Either a whole bunch of bucket libraries and five of them and then once I've been responses. We run that test themselves to tech We haven't broken any of that guy. 'cause people have already written a ton of tasks for sequel alchemy for requests. Or whatever right exactly so I mean back inside probably when we chase it identified loss officiates and since then it. Stop this releasing. Any new is pretty good way to just hit it with enough information that it's GonNa get caught in the issues. Have you found errors and other libraries because of this gotten back to them like what is actually we are stuff was broken but actually your stuff is broken. I mean have equal I mean. Sometimes the tests are failing and my office after a while we asked us to pick attack and stick with it. I'm actually we found the there's just a lot of back to be so we tend not to review it at the moment. Yeah it's not your job to check all open source libraries for correctness right exactly and unlike some of the libraries. Sql Out that we do on our absolutely enormous is hundreds of hours and multiple drivers to different ice seconds. Side takes a long time. Yeah we just rely on the test a good relationship but yeah they hung by testing is a very interesting I did. We have talks about the way. We're considering doing. It was exactly how you go about you. You write a piece of Pie on that. You put some infants and you take what the opposite and then you run so. I I'm the way we understood stood is actually also right generates. If it's all sky that takes be an initial taste of does random mutations to it to stall so here's the pace of socially should raise. Let's apply of than mutations to run. Soci- I've read check the inputs outputs. So yeah the generate. A that would have been quite interesting to riot is something that we considering the feature. Okay yeah it sounds cool. It sounds like it would take forever to run but it sounds like a cool project. Maybe you don't run it on every every safe all right. Well let's see. There's a bunch of things I want to ask you about that. I don't want to like over too much in time so I guess one area. That looks interesting to me. As we've talked about this being a plug in for an editor that's interactive you also talk about just applying to source libraries and on sorcery homepage. I see that Thursday. You'll get instant quality of your python. Could base like just pointed at your repo and it'll give you some answers. So is there more stuff than it does than just be a plugging. Jacques Cli way to like deal to use it for as also available. Get how thought. So you instill sorcery ball into You'll get by and every time you to over software review dot full requests and find any sense to the to any of the files have been touched by the request and it will cra- request on top of that thing. Hits the changes that you can make an improvement to it. Then you can just much older question straightaway when you first install also coming back to the whole library of all at once very cool all right. So let's talk about pricing so this is something that is free for some people but it's not free for some other people. What's the story with whole business? Model Open source cited things. Like what is what are you guys offering here because it sounds really useful to a lot of people but at the same time. You're charging some folks for us to that might that might influence people's opinion on how they feel about it. Yes Oh plug ins a free. At the moment we think in the future will probably introduce supreme. Indonesian still have a free version. I see so if I'm sitting here I'm going to write code on my macbook unpatriotic. I can just go get it for free. Go get it for free right now. Don't think that's whether open source so place also or anything. I could be working for a bank even you could. Just we have had Jesus from banks working on it right on but there is some business model where you guys charge money for something. So what are you? What is that side of the thanks? review free. Prep in source again. If you WANNA use on a closed source repository as a small charge pedal Alpa month bicycling not come something. We just released in the last few days. Basically all right cool so basically if I'M GONNA apply to my code base as a autonomous bought type of thing and I'm open source one hundred percent free right so if I was taking care of requests or seek welcome here flask or whatever I could just plug it into the flask repo and get hub and all of a sudden. It would solve those problems if like. You didn't seriously just give me a four. The Better Penn state listed you right. Yeah Yeah particularly useful. People maintaining large libraries because got requests and they might come in various standards. So does the initial coach nine the project. There's another one that is trying it out if you have a good health account on Einstein is simply to star of the CREPE and I'll get a little find you'll most popular piscine repository and send you pull request to refer to you. Okay Bye Click a star and Yoga. Aoki cool yeah seems to me really useful to have it just built in to get a automatically looking over the code because I you all have worked with different groups of people. Different companies different languages. My experience has been that people that care about code quality and re factoring and testing and maintainability and patterns. And all that kind of stuff. There's a massive spectrum on any given team some people it really matters to them and others. Those failing task that feeling build is just a nuisance. In how do I turn off the build so I don't have to hear about it again and so having it as part of the Repo means thy get kind of applies to everyone at least it suggests for everyone whereas if it's just in the editor there's going to be the people who love it and the people just like how do I install this or disabled as so it doesn't it's just I wrote my code to you? Know what I mean like. There's just it doesn't matter how much advocacy there is. There's going to be that so having kind of external is pretty cool. Yeah we started doing too because we thought that was kind of the way to really make you write code. Foster economy has been does the change but like a saying definitely. That's why be introduced the plug in because the courtesy not kind of brings things up to a level. Yeah so it gives you the benefit as a partner in the she got kind of a gaze but also they experienced a facie time during the coach of because there's order until during the simple steps has not dealing with the contextual elements of it is making function is not the yeah. I think that makes a lot of sense because it doesn't matter how good you are. You don't WanNa have to go think of the implications throughout the whole could base. We're just say the tool set is good so it's good much beautiful but I do think it's really important news editor because it teaches you could teach you at teaches you the idiomatic python acc- ways of writing things she'll like. I had no idea that I could create a four loop with Enuma that had to pull unpacking instead of trying to do bore over range in then pulling out the item and things like that so it seems like a really cool combination. I think definitely the educational things I WANNA folks loan. I can bring up combination so blog post recently. With few Ole ry factoring. You're doing like why we think they're good idea as opposed to just what we've done. Is that the one that is called Python. Refractories part one. I look through that so maybe you could give us a couple of factories out of there. Like hoisting is like one of the best things because anytime you go deep go away. Can increase mistakes released late to that would be like maybe of the same code in an if L. Statement yes and it's just duplicated so from people right so at the end of the same thing in the in the outsource lights valid. Even maybe 'cause happens in every branch it means. It was happened certify actually need to be in the condition that tool if you take it out exactly just put it at the end. It becomes kind of more clear what the conditional is doing. What is controlling causing all this extraneous thing in another one you have? There is converting from four loop which does a yield to a yield from that collection directly. Which is Pretty Nice? I mean it might even apply to code that was written long ago before yield from was introduced to the language but yield was there and you could say hey look yeah for sure. Gosh this old way could be you come into that. Didn't realize you could do that. Watch really and change. It may seem to us so bizarre. I think I'm sure there's pets that I don't read as well I to pick a single most favorite absolute love it re factoring. It has to be convert like a deeply nested set of code to something with guarding clauses. So it's like flat right and going if this is true then if this is true while this is true if this is true and you end up like writing starting on column forty to write your code if you negate them all and like return earlier breakout earlier something. Yeah it's just so much cleaner okay. That cases out that cases out that cases out now I focus on the essence avoiding nesting is one of our those kind of metrics anything. We didn't touch with his. How you get the computer to realize that as a code smell like Ryan Good Code that computer in quite difficult so there's yes metrics complex day which what's that about about avoiding conditionals basically number of decisions yet. How many branches would you potentially go down right kind of on Special Tackled cogan shift? Which is trying to get idea of. How telling Houston? Holding your head. Not really penalizes nesting. How many variables are play? How many other things like that as well right? Probably Yeah so like that. So penalizes nesting multiple. So that's kind of like household not to like nesting have written written metrics about. Oh I'd never really thought about it that way. But that's exactly. The problem is like the reason. It sucks so much as like that next test is piled on as a and and and and and this and that all the stuff that you've nested yourself into you've got think of all those at the same time while I'm in here because remember things you have to hold new head you know. A human holds six or seven things in that head at once. Measure exceptionally can do so like some of our metrics is focusing on how many variables you have to be thinking of. And how many conditionals you have to be thinking when you halfway down the function. And it's gone off to the right away so that we have the working memory metric. Yeah Oh cool. They mentioned a number of Arab bills. in skype at the current time. So we think yeah if you're raised in kind from top to Boston by the time you've got to the line of code if you've got seven variables and you'll head that he's done you've done inside the function anymore uconn. Yeah he pulled out in in your head on the on the stroll scroll back and forth instead of just reading it. Yeah yeah there's this really interesting saying from a friend of mine that talked about. It was then like when you write code. Debugging Code is harder than writing code. So if you write code at the very limit of what you're kind of able to write and do the most complicated of You. You probably can't bucket. Yeah because trying to think through it actually like a more complex than you kind of just pushed it over your limited. So there's anytime you Kinda dial that back a bit factory or other styles. That should be three functions. Then you won't have to think about so harsh three things in this part so much better. It's interesting figure. We found in scientific. Pei Pu analyze developments was so if they spent seventy percent always spend seventy percent of all time trying to understand the coach ninety five percent of the time actually typing seventy percent of the time. We need to cut down on by camel readable. Refractory in at you have the Mugabe's Zan Python right. I think that's why it's popular language is because it's like clean to read presents itself well right so don't undo that by writing bad code. I guess definitely all right. Well I think this is probably a good place to leave it you guys. It looks like a really cool project. If people are using by charm or using vs code they can just go get the plug it and give it a try right shoulder search for three in the marketplace of the ide-. Yeah Okay so you get a like as you go to the plug and marketplace charm or you do. The extensions envious K. U. as an computer source about as in Gandalf. Yeah Yeah Yeah I know if you guys saw website has instructions for installing by the plug INS and using it on get up. I'm to people have open source getup Repo. They should just drop it in there and it'll give them some ideas to try to the I'm links to occupation as well cool all right now before I let you. Outta here to questions always asked in the show so we'll just be quick since there's two of you have I go. I writes Python Code. What editor US I used been. I ended up with a rift injuries from rape factoring. Who's in control? And just right case too much so I decided to learn and ended up with that as well Longtime ago and had to like re juggle a lot of interesting stuff like funky hervey keyboards and all sorts of stuff. Yeah try to use hockey's rather than mouse a lot. Yeah it was turning into a real issue so I have to learn van with me down by about ten times ten weeks and then now I say this I've it. Yeah it's not my tip. It's awesome Nick Right. By Lanka's what some visually impaired in that high contrast mood is just really really good. Katie bit IRA starts up super-quick but it's difficult to say I've made the switch. I can imagine that will definitely pushed you over the then notable pipe package. Maybe not something that everyone necessarily knows but is like Oh cool. I found this the other day and you should check it out any ideas recommendations that way. She is this package cooled. Newt coca to spell by new. It guy and it takes you a kate's cross composite into say I man codes and creates the t spo- and that was without package sorcery just wouldn't exist as a running projects because you'd have to deploy as we're just delivering all of our source code with the Plug Ins and the extension interesting so your packaging it up. You're packing up sorcery with New York. Exactly Yeah so interesting. It's fake and of them that you use saying. I'M RACHEL THE IMPULSE WORKOUT. Which of the cut that needs to Compose the whole thing down some my windows on Linda's awesome as Nixon wrinkle. I had Kay Hagan from the project on for episode. One hundred seventy four which is like like year and a half two years ago. I don't know quite a long while ago. But yes super cool. I didn't realize that it was show flexible and packaging up APPs. I've thought of Moore's like sight on this little bit. We could make faster. So that's that's good to hear nice all right. I don't call the action. People interested in sorcery interested in factory. And what he tells them cry out. Now you have get house account you can star already And try out in five seconds or you can install that and got old of your requests. Refracted all if you're using touch on got an instilled it right now. Try Out. Got Your country. Does you wet? It's not I mean. We ready came to get feedback from people keep on on a hit better. Better bicycling awesome. Do you guys have like a get hub Poe or how should they give feedback or say you know? My favorite re factory is whatever you guys don't do. Had they make that happen? Do you ever go the sorcery I repo can raise issues or does the MLS free read answer every email. Hello I am. I'll get her is also very. I slash sorcery ankle awesome. Well Brennan Nick. Thank you both for being here and creating the school project looks awesome much. Thank you very much for having us. Mike. Okay you bet bye bye bye bye. This has been another episode of Talk Python to me. I guess in this episode were Brendan McGuinness. And Nick Saban in has been brought to you by data dog and winnowed data dog gives you visibility into the whole system running your code visit talk by Thon Dot FM Slash Data Dog. And see what you've been missing. There are in a free t shirt with your free trial. Start Your next python project. On the nodes state of the art cloud service just visit talk by Dot com slash loud l. I. In Od automatically get a twenty dollar credit when you create a new account one to level of python. If you're just getting started. Try My python jumpstart my building ten apps course or if you're looking for something more advanced check out our new. Ac Horse digs into all the different types of acing program you can do in Python and of course if you're interested in more than one of these be sure to check out our everything bundle. It's like a subscription that never expires. Be Sure to subscribe to the show. Open your favorite podcast and search for Python. We should be right at the top. You can also find the itunes feed it slash. I tunes the Google. Play play in the direct. Rss Feed at slash RSS. Talk by John. Him this is your host. Michael Kennedy. Thanks so much for listening. I really appreciate it. Now get out there and write some Python Code.

editor Brendan Weather Nick Right Michael Kennedy twitter Kay Hagan rape developer Nick partner Brennan McGinnis Shamir Jerry Martin Fowler First Languages Island Ron Siv Delfi Jay Johnson
Census: Data Accessibility with Boris Jabes

Software Engineering Daily

55:43 min | 3 months ago

Census: Data Accessibility with Boris Jabes

"A data warehouse is a data management system that contains large amounts of historical data and is used for business. Intelligence activities like analytics. A data warehouse centralized is customer data from multiple sources to be organizations single source of truth. Getting the data from your data warehouse into the different applications. Used by organization can be difficult. The company census simplifies sinking. Your data warehouse with other applications census works on top of existing infrastructure and lets users pick destination apps map the data and then handles maintaining live in sync metrics. Because since it's runs inside your data warehouse. Data remains secure and is not stored on their servers. In today's episode. We speak with boris job. As ceo census. He was previously a managing partner at polychrome and a senior director log. Me in we discussed the complexity of data warehouses and how the field of data analytics has grown over the years and just how census works rookie out empowers engineers to solve customer issues. Five times faster by making debugging easy and accessible in any environment empower your engineers to handle the complexity of modern applications by seeing into their code in real time. As it's running using non breaking breakpoints. Try free at roof out dot com. Today there's pain-free debugging and kuban netease. It's not a dream. You can empower your engineers to handle the complexity of modern applications by seeing into their code in real time as it's running using non breaking breakpoints. No coding restarts redeployments needed. Try for free at rook out dot com if you're doing lots of debugging and production with pain than try recount dot com. It's a solution for non breaking break points with no coating restarts or redeployment needed thanks to rook out dot com for being a sponsor or is welcome to the show. Hi thanks for having me. You work on census and we'll get into what census does. But i i want to talk a little bit about the state of data engineering and data platforms and a lot of work in data engineering has started to consolidate around the data warehouse. And you look back in. History of how data engineering evolved. It didn't have to go this way. There were all kinds of things that could happen. You had all the streaming systems. You had data lakes which you still have. But why did we get to a point. Where so much of the work is being done by. The data warehouse and so much consolidation is around the data warehouse. I think people really deep down. We're craving away to simplify. And i think the modern cloud data warehouse especially has really nailed. It doesn't truly solve every problem. They're still needs for all sorts of specialized technology. But for a lot of the general cases. It gives a single tool set that covers small companies startups. Even that are just kind of scaling up as well as massive sized companies right so. That's the first thing i think that they're able to do is to say you. Don't have to have breakage for various levels of scale and you don't have to invest upfront and we shouldn't understate how big that is. And then the other reason. I think people really started gravitating towards just do everything in the cloud. Two warehouse is it means learning like a lot less learning something bespoke. They are effectively. They're not perfectly standard but there are a lot closer to standards equal. And i think that's another thing. People really craving was a way to operate on their data without having to learn something proprietary. Or something you know kind of complex and so there's still room for those technologies. But i think those are the two reasons i would say. Why a lot of the gravity now resides on the cloud data warehouse and today for data into the data warehouse. Is it directly off of applications or does it typically go into kafka and then goes into the data warehouse or does it go into a data lake the data warehouse. How is the data warehouse aggregating. Its data yeah great question. I think i see a couple of patterns out there that are starting to become common so the first is getting application data into a warehouse especially when it's born out of kind of a. Let's call it a finance. Be i need so a lot of the initial use cases can be just pulling data down from your sales tools and finance tool so that you can do better reporting and for that i think really. We've reached the point of maturity now for tools like five tran or or stitch. Such that people really just use these very very easy lt tools to drop data warehouse and basically at the point where you don't even need what i would call like a data engineer to own that piece and then on the product data side or let's call it the live sides of the business. I see two different solutions. Based on where you are in terms of performance scale and latency needs at your company. I think there's actually a ton of people that do very casual twenty four hour kind of data dumps from their production databases into a warehouse. And they're getting away with that because they're comfortable with kind of that latency on their on the warehouse side and it's easy again. It's a simple solution. You do see people connecting their event streams to a warehouse as well and again the way i see. It is not everyone's kafka but you eventually graduate to something like that a lot of people who are not on that stack. It's because they're actually still able to do everything in segment snowplow and just have those natively push the data into the warehouse. And that way you don't have to invest in your own streaming architecture. But i think eventually you do right and so i would say broadly. Speaking the most mature organizations have some kind of streaming setup a kafka that the their service work texture is depending on and then the warehouse is a sink off of that and then all of the business applications are going directly to the warehouse and that combination becomes your your source input for all this. What's the role of the data lake in this model. That's a good question. I think there's like so much data being generated what the old adage right. We're doubling the amount of data we produce. Every i dunno eighteen months i guess our data moore's law and the rate at which companies are generating data is similarly increasing. And so i think when you're dealing in the trillions of events and the amount of scale that your product team generates you don't necessarily wanna have that go directly into your kind of high performance data warehouse so i think the data lake is that staging area floor immediately storing all the data. You have and that way. You don't have to worry in any way shape or form about the scale at which it operates and you can make decisions later on as to which parts of that data you want to bring up for analysis. I do think the warehouses in the data lakes are are trying to blend these two universes right where you're seeing a lot of those kinds of that kind of terminology emerged at where it's going to land honestly but i think again the desire for simplicity is there so i think the way an engineer or a analyst wants to deal with data is actually in one consistent way to access it and then with just differing levels of latency based on their needs. So i think as an end user. I don't really want to have to think about is the data in the data lake or is the data in a data warehouse. I want to be able to just do an analysis. And then i'd love to just to specify i'm okay with this analysis being twenty four hours behind or more take like one hour to complete right versus. I need this to be fresher data. I needs to be faster. Query and then a single system could define where data should live based on that right. That's kind of an ideal outcome. I think yeah. I mean i didn't interview with somebody from snowflake like three years ago or so and i remember talking to him. And it sounded like an snowflake is proprietary. Technology really knows. What's going on under the hood at least not anybody that is outside of snowflake or who who hasn't worked there but it sounded like from what he described this tiered system where it is basically at the lowest tier something approaching a data lake. Where you're you're throwing all your data in there and it's just getting tiered into you know lower lower down the memory hierarchy or whatever and into you know slower and slower access storage mediums. But it's like the long term goal of of snowflake is to be that unified solution. Much like you have the same thing going on with the data bricks world the data bricks delta. I think they're both coming at it from their own side. And let's even like step back on the snowflake side rate. Even just separating storage from compute. We take that for granted now it was the great innovation and props to big for having that as well pretty early on but once you do that you've already created that first level of separation between thinking about how your data stored in how performance you need that access from the computation. Think the true blend will be specifically not just what kind of data warehouse is going to do that computation. but how easily accessible is that data. And it's hard to tell you right at snowflake whether they're doing are they changing how they store the data based on your access patterns but if i were them i would be working on that right. I think that's definitely something if i were an engineer there. That's what i'd wanna pull off. That's what it sounded like. And that was what kind of blew me. Away was one of the more mind altering experiences that i've had on the podcast where i really started to understand. Oh this is. This is a real company like they've got some real problems that they are solving here because making that kind of tiered thing just invisible to the user is quite a challenge. It's a huge challenge. And by the way. I think it's something we've talked about and what we can get into later. But we've talked about this idea of in our product like magic scheduler because customers always want all these various schedules renters any etl type product. You're going to have effectively people. Wanting to redefine krahn and from the day the company started. We were kind of trying to think about. How do we get people away from the concept of schedules altogether and just try to. What if we could have a magic schedule. An adaptive schedule that determines when data should move based on prior patterns and. It is fantastically difficult to do that. Well and do that in such a way that where. Everyone's happy all the time. So yeah. I think you're right. These are capital e. hard engineering cross until you're saying in your description of data warehouse versus data like it sounded like you were saying companies that don't produce you know really really high transaction volumes. They don't need a data lake. You can just throw everything in the data warehouse is that is that. That's that's how. I think yeah i think a lot of people i mean. We work with customers of decent scale. you know. And they're able to operate in that in that way. I think the interesting question is a lot of people that i work with that. Have the two pieces of infrastructure separate kind of an oftentimes all in aws. I can't tell if it's provable. Because they've run the numbers in this actually works out to be significantly better. Or if it's just a remnant of will. They started working on that system five years ago. And you know you don't just throw it away. So i think most people who are getting building infrastructure now ken to a pretty high degree just rely on data warehouse exclusively which i think is part of the reason why people are so excited. So let's start to get into your building with census. So i've done a few shows with five tran. And you know i think of as a really easy way to move data between different places like get your data out of stripe and put it into your data warehouse for example or get your data out of segment and put it into your data warehouse and when i look at sense i kind of think of it as the opposite direction. Like you've got your data and your data warehouse and you're looking to get out into something else like a segment or a zen desk. Is that a good way of looking at it. I think from a pure it which direction the bits flowing i think. That's a very reasonable way to describe it. And it's our customers have kind of describe it. Since we started in two thousand eighteen. They started calling it like reverse five trend back then. I think what it doesn't describe is what people are doing right. I think when you think of five tran you're pulling data down into your database so that you can do analysis. But when people census they're pushing data out in order to drive business workflow anything. That's actually a very different job to be done. So even though yes bits are moving in. They're moving in the opposite direction. The needs of the users and what they're trying to accomplish is actually quite different. Say more about that. Sure so you know when you think of. Et l. e. l. t. as as the term. Nowadays you're bringing data down from stripe from net sweet from us and asking you're trying to analyze. How much revenue did we generate in this quarter based on all these very signals that we can get the one correct answer so that the cfo doesn't get in trouble when we present wall street with census. What you're doing is you're on the other end of the. Let's call it. The data refining process. So you are generating really interesting models and insights about users about about entities in their business and then what they wanna do is use that to change the way someone operates in the company. So let's take your example of zenda. Ask so everyone is to handle tickets from customers and one of the things that every company is trying to get better at it is provide not only better service but better service per agent so that they can cover more users without loss of customer support experience and the data team can have really useful information about what uses. They're doing to the point. We just described so. Data analyst might compute how much revenue they're generating from one company and combined that with how large the company is and how active they are in the product and determine. This is a vip customer in by pushing that data out to send us using census. The support team can now take advantage of that when they're prioritizing tickets and when they're responding to customers and so it's about making the rest of the business better using data rather than enabling analysis and so. I think that's the real difference here between what people are doing which changes the needs right in terms of latency in terms of correctness if building software is your passion. You'll love thought works technology podcast. It's a podcast. Four techies by techies. Their team of technologists. Take a deep dive into a tech topic. Each episode that's pique their interests. It could be. How machine learning is being used in astro physics or maybe how to succeed continuous delivery. They're always coming across fascinating ways. Technology is advancing and love to share what they learn. Whatever the topic. The discussions are always lively informative and opinionated. The team of co hosts are experienced technologist from across thought works including thought work. Cto dr rebecca parsons and renowned writer and speaker. Neil ford each episode. The podcast features a guest or to talk about their particular passion and areas of expertise past guests have included imminent technologists such as martin fowler. Marg and danah boyd. There are a wide variety of episodes on the podcast and a lot of them have to do with software architecture closure digital trust. These are topics that thought works has been thinking about for a very long time. And they're as well equipped as anybody to talk about them. So whether you want to broaden your knowledge on a specific topic or you just want to immerse yourself in the world of technology theraworx technology. Podcast has something for you to find out more. Just search thought works technology podcast on your podcast platform choice and make sure to subscribe. Can you go through another workflow example. Totally how people us census totally really doesn't so i'll give you another classic example. This is a really really common scenario. So we work. With a lot of freemium high scale b2b sas companies so these are companies like sigma or notion and they they have millions of users and many of them have the product for free. Some of them around like a pro version. And then there's an enterprise version and you can't call everyone right not cost effective. It's not the point. The whole point is to do scaled out software. But when you're dealing with the go to market and the sales funnel for companies like that you really want to identify great leads out of your user base and the team. That is most able to do that. Is the team. They have every single signal right landing in the data warehouse. They can combine that with external signals. That you have about companies like the kind of company that the person works at and then you can combine that with a regression analysis or just something more advanced on the data science side to determine what kinds of users what kinds of companies in the user base are most likely to convert to pay plan or to an enterprise plan. And then when you have that analysis done instead of having a meeting and say here's some users identified in my analysis you plug that in directly into your crm and generate the leads automatically so census will not just sink data. It will create objects in the serum and then the sales team can have that automatically routed to whoever's in charge and they can make calls so they save time having to do this kind of manual process between them and the data team and you get higher quality leads right because the data team is uniquely positioned to have the best information to do this. So in that example. Can you give more of a description of what the data team is. Actually doing what. The data team is building like a got. My you know there's like the sales team with the crm and then there's the data warehouse. That's ingesting lots of events. And then the data teams responsible for building the middleware using census to sink data from data warehouse into the crm. What is the data team doing. Yes i think they're doing a bunch of different things right. All which i put under the umbrella of product their data so the first thing is they have to model the data correctly and i think of that as coming up with what are the unique entities. The matter in the business so potentially that's users potentially that's companies. It could be other kinds of objects to you know. It could be a unification of invoice data for example and then once they have those unified models they have to start they structure them such that they can get all of the columns that you need to drive a business workflow and you don't have to. You don't have to solve that all in one giant table but we're increasingly seeing that as a pattern of lemme create out of my entire kind of data transformation pipeline. Let me finish with a table. That represents here all the users that we care about and then on the table you're gonna fix columns around how active they are. What are the various members of that workspace. How many workspaces they have and then you might have interesting scoring attributes. So in that example about lead qualification. You're gonna run a query or it could be the output of an internal process to but let's keep it simple and so you might have a query that uses as signals like. I said how big a company is how much they've been using the product. Have they hit this event or that event and then combine that into a score make that a column and then create a mapping from users that have a certain score into the crm such that generates leads and the rest is kind of hit schedule. Hit go and then maintain the pipeline and so in that process right. They've gone from kind of design and coding part of this which is really where they should spend most of their time in how they model users should two uses a have the same email. Be part of the same company if if you have a workspace with ten different emails of different like users like that. Don't all share the same domain who should be attributed workspace to. There's a lot of hard work to be done there. How do i stitch anonymous user data with identified user data. Right like all of that. Is that the data team has to do. And then when they're ready to deploy the results of their analysis right into a live workflow that's census comes in and they can be sure right that the data's valid before it goes out because the last thing you want to do is push invalid data into production systems. And then because you know the the hit screw this up as much worse than if you screw up a dashboard and then have a tool. Like census in the air to monitor how it's going right and so make sure that when the sales team decides they've they're changing their data models that you're pipelines break and everyone can adapt and that's kind of day in the life and the alternative to this would be that. The sale seems literally periodically going to the data team in an asking for like a report and then they have to like manually. Create the leads in the crm. There are only two alternatives to our world. One is you get your engineering team to build these pipelines right and we're back to your original question about what's happening. In data engineering and these are non trivial pipelines to build. Because all the usual reasons right. The api's change and if the pipeline breaks you're breaking alive workflow right. You're not breaking a a reporting system. You're breaking alive workflow so you might have live emails. Go out based on this data so it's really really important to have these pipelines. Be up to date and correct and the other alternative is. Your sales. Team goes to your. Bi tool they'll go to look or or some tools like mode and download a c and uploaded into their sierra. This happens all the time and that comes with all sorts of problems right one. it's not incremental. It doesn't have any kind of ability to integrate right. It's just a random report. They're going to download and it creates a whole bunch of duplicates because an a manual upload doesn't isn't really aware of what's happened in the data team is disconnected right like. That's really the most frustrating part of of what people are doing here is that they're moving data kind of independently and the point of sentences to say not. Only should we make your warehouse and operational hub. Not only should we make your data warehouse be the source of truth for these systems. But you want all of these pipelines of live connected to the source so that the people responsible for generating that information are aware when where they're used in the leaf nodes of the business. This episode of software engineering daily is brought to you by data dog. A full stack monitoring platform that integrates with over three hundred and fifty technologies like gremlin pager duty. Aws lambda spinnaker and more with rich visualizations and algorithms. Alerts data. Dog can help you. Monitor the effects of chaos experiments can also identify weaknesses and improve the reliability of your systems visit software engineering daily dot com slash data dog to start a free fourteen day trial and receive one of data dogs famously. Cozy t shirts that software engineering daily dot com slash data. Doc thank you to data dog for being a long running sponsor of software engineering daily in. Show let's talk a little bit more by from your point of view so when you're building infrastructure to facilitate this kind of export from data warehouse to crm. What's actually going on in your infrastructure. What what have you built yes. There's a lot going on. So let's see if. I can summarize it so the first thing is we connect to our customers data warehouses right so we have a credential to the data warehouse just like a bi tool and that credential will execute a query right they will execute a query. That says here are the you know. Let's call it sales qualified users and it will then do a comparison with the last time. Data was sinked to determine what has changed right. So here's a query that is like all sales qualified users and we don't want to rethink the hundred thousand sinked last week. So let's only sink the new the new ten thousand that that occurred. And so that's the first piece of interesting code in our product is like generating death and the way we do. That is really nice. We use our customers data warehouse to do that right. So none of the data has to leave our customers premises for this and then from the resulting data set we have to prepare that for upload into the sierra so that involves unloading it from the warehouse and stitching it together such that we can batch it into the destination system so you're probably aware that every single sass product is different. They have different kinds of. Api's different kinds of rate limits and different kinds of batch api's which is what we try to use exclusively in so for certain products that means we have to take the data determined. What fits in you know. Whether that's a ten megabyte file batch or a number of record batch and then have the job run so that it can bring that data all into the destination system extract from the destination system. Everything that went wrong so again. This is a distributed system and these third party apps are fairly unreliable so they fail in a number of different ways. Some of which the user can't do anything about some of which is in the users control so a system might reject an update for having a field. That's too long right stuff like that. So we bring all those areas back and record kind of how the sink was run like just kind of what was the the final tally of records that were created updated and rejected. And all of that kind of comes back into your warehouse and and we go quiet again and until we run the next time. I think that's probably in a nutshell. What sentences doing here and the interesting thing that we've started to add into that is if you think of census as this last mile were basically pushing data out to these systems of action as the last mile like this is really where the data has to be correct and a lot of what we've seen with our customers over the last few years is the more they use censused the more. They are empowered but also scared that they're gonna break systems and so we've started adding into census built in validation so that you're at least some level confident that your data on the way out is not malformed and is not going to break things so a good example of that just you're really classic simple thing that census can always validate because we understand the destination systems is something like duplicate records you might be able to into something like salesforce duplicates and salesforce. Well believe it or not silently accept the duplicates and you won't know which one was picked as the correct answer which is not a great experience for the user right and so those are validations that we do upfront incenses. So every time census runs it computes a delta it computes which records are valid and rejects upfront records. That are invalid based on the destination rules. And then it. You know batches the data so that it can push it into a destination system at maximum speed and then brings you back. All of the arizona occurred. There's also a data check obviously right so meta data breaks so your columns break your crm field. Names change or field. Api name name-change they get deleted and so every time. Of course there's also a meta data check to make sure that these pipelines can even still execute so it's a lot of work to maintain high quality data pipelines across. Let's say systems that are changing as well as unreliable and if they break. Do you just notify the engineer somehow. Yeah we do two things right one. Is i treat this. Lake a monitoring problem so a lot of how census designed and a lot of i think of what you see in this in our product is thought out is really applying the lessons that you and i know from the world of software engineering and devops into the world of data. So yeah it. First and foremost will stop. The the sank right send out an email to a number of users or logging system that you might have internally and tell you what to do right tell you sink. Can't run a map column was deleted right something back and then in addition to that sense and this is actually pretty fundamental and a lot of a lot of nontechnical. People at first are confused by this. But it's actually. I think the right way to design these kinds of systems census state based right. It's not an event based system so you're pointing census at a data model of what they call it earlier. I said let's call it. Sales qualified users rent. That model is not sales qualified users that like events like using a user. It's a stable table that defines or query that defines what users out of the entire userbase are qualified. And that way. If there's a failure census will no matter what rerun and when it reruns for all you know even if you didn't do anything as an engineer to fix it maybe the problem was legitimately transient then will heal right. When the next run occurs everything that was rejected. Everything that failed last time is automatically rerun and so i think that is where people get a lot of benefit relative to hand rolled solutions. It's all of this kind of tracking of the failures and retrying of the failures. Because this is going to be eventually consistent zooming out a little bit is one way to look at this like a zepe ear for your data warehouse because you think about all the the n by n problems to be solved and all the different integrations to right. The the analogy xabier is appropriate. It absolutely is actually. I think the reason it's appropriate is if you think about people use tools like that in lots of different ways right very broad kind of user base. But it's a lot of operationalizing automating some business workflow. And the way census designed is to solve those same problems but instead of using a point to point and by as you said integration model and an event based integration model. Where when something happens here do something. Their census is designed around this idea that build a unified correct data model. So think of it more like functional programming right and tell us where it should be put and we will make sure it arrives there and that way you don't have to think about event replay. You don't have to think about what action triggered that outcome. You can define the query right. You can write the functional program that says hey what inputs lead to what outputs right. So what rules defined whether a user is sales qualified and if those rules change then great than all of the old users that were not qualified before will now become qualified and census will sink them. Which is something. That is extremely difficult. If not impossible to do in these event based kind of connector tools i see. Can you talk a little bit more about how you keep up with all the integrations. I guess i'm a little unclear about that because you have so many of them. Yeah that is a line sheriff. What the team does. Obviously we have. You know my team has years of experience with this is not our first experience building these kinds of integrations. So today i think census has over thirty like probably nearing thirty five. And you know that's gonna grow to probably one hundred this year and so to build those. There's two things right i. The beauty of census is one side of the equation is fairly fixed rate. We support five or six data warehouses. Not and by an right so we don't have to support every combination of input every combination of output. We define the input as this this hub right like the data warehouse is the hub and our goal is to sink data out from a warehouse. So it's no redshift snowflake and data bricks. These are our bread and butter and on the output side. First of all some of these integrations are very much. Follow a power law in terms of popularity. So so that's one thing. So you know the top. Ten percent of our integrations get a lot of play. And they're the most robust and they actually you know kind of battle-tested in production for years and then the long tail. We have a lot of internal systems to make buildings integrations effective for it so there is first of all like internal generic components that we can use for any destination system so things around retry is things around rate limits all of that luckily shared logic in census so when a new service comes out and we just added on the other day that has unbelievably low rate limit instead of having to adapt to their weird amount. Like this is all just plugs into our existing infrastructure. So so that's one. There's there are things that we can reuse across every integration which is really nice. The second thing there's a lot of institutional experience the batch. Api's we're at the point. Now where for most of our integrations. We're actually giving them the feedback on how they need to fix them. Because most companies really implement poor batch. Api's which is something that sends us really thrives on is to try to move these sas applications to take data batch and then the last thing that really helps is number of years in production right like one of the unfortunate things about the space is unlike. Let's call it to rewind to what you asked at the beginning unlike. Let's call it five trend where you're bringing in raw data from a salesforce instance down into your warehouse senses dealing with bespoke user models into potentially very bespoke objects in a crm or in a support tool or marketing tool or any of those things and so a lot of this just means you just need years of customer use to see all the patterns that emerge. There's just there are just air messages that you would never be able to simulate on your side. Is the really honest answer. I refuse to believe that there is anyone who could generate every single way that let's call it a third party. Marquette instance could fail so you do as much testing as you can't internally and then the rest is being in production for a few years triple bite is a network of over two hundred thousand top engineers. It works with more than four. Hundred tech. Companies including coin base zoo cts dropbox and facebook dribble by this focused on matching high quality. Engineers with great jobs triple bite is the linked in for software engineers. It showcases your tech skills to recruiter. So you can focus on writing code not your resume. There's no more irrelevant outreach. From recruiters about roles. The you're not interested in with triple y you get roles that you are interested in and you can set granular preferences so that you could matched only with roles that you are interested in and you let the right roles. Just come to you. Top engineering roles from four hundred and fifty top tech companies. Hire their best engineering teams from triple bite and triple by is free for engineers companies. Pay triple bite because triple bite makes their hiring process more efficient visit triple by dot com slash s daily to take the quiz to get started as a triple bite engineer. You can take several different quizzes. Generalist quiz front mobile machine. Learning devops data science entry level. The entry level track is new. It's geared towards engineers looking for their first programming job. If you're interested in any of these good a triple bite dot com slash. Save daily to take the quiz. Thanks triple by for being a sponsor. Are there any big variations between the data. Warehouses like your interact with the various data. Warehouses like snowflake versus redshift versus big query. Yeah there's definitely some variance. They're not as i mean as any customer who tried to port for one to the other. We'll tell you these are not turn key migrations. Let's call it. The sequel is eighty percent the same but begin the ways in which we twist sequel here to do things. It is actually different on all of our warehouses including let me be clear between postcards and redshift those are also the same despite the fact that wretched will tell you it is so yeah i would say all of our integrations with warehouses are unique. I'll give you what's a good example. The way you unload data in data breaks is very different than the way you unload data in snowflake and then the formats in which they dump data even though it's csv there can be some mild differences between them and then the limits we're generating because we're remember. I told you earlier we do. The delta of what records changed between each run of our system. So we do that by doing. It's actually kind of a sequel except query in your warehouse and that's a very long generated query. That's doing it against a snapshot that has a good in your warehouse. And you know those don't always just work between two warehouses so there's some customization there between how it works in. Let's say snowflake versus big query and then finally authentication is slightly different and permissions are slightly different on every system. So yeah i would say the warehouse. Connectors are not the same. I'd like to zoom out a little bit and talk about some other subjects in shut engineering landscape. So i d bt. We've done shows on but maybe you could briefly talk about just give an overview for what. Dt as in why has dbt had such an impact on the world of data engineering and particularly your business absolutely. It's without a doubt the kind of the most significant change in the data landscape for the last year or two so. I think i told you this earlier. The way i see everything that we do. And most of the things that are happening in the realm of data as the application of software engineering and devops principles into the realm of data. So you can think about that at every stage. So census almost in a way as a cic de system right it is a deployment mechanism for your data to get out to where it's used and not really created to pretty huge yet simple but huge innovations right. The first thing is it gave people a macro language around sequel to do to reference to different sequel. Grace right so that you can start to decompose your transform logic into a series of of sequel and data models and the beauty of this is they're connected via the logic not via some ordering of events right so it's not like do this. Then do this query. It's this query. Depends on that query right. So you get a a distributed basically graph of dependencies and that way you get a separation of concerns in the way you do your data model so dvd gives users that which is a huge improvement over just writing a sequel career. Which is what most analysts were doing and second it allows you to use it to kind of store and version that coat and that's kind of the baseline for doing all engineering. So it's it allows you to treat your sequel analyses in your sequel data models as a piece of version able code and that was a huge change for people in the world. So now you can have versioned data models that can be tested. That can be code reviewed by others that you can put them in a plo requests. You can have different branches all the things that you and i take for granted and then census actually native to your dt repositories so that you can pull out all of that information all those data models directly out of your repository and then you can decide which models which branch which version of the model is the one you want to sink. And then census acts. As like i said kind of a cic d. and deployment system right so it will validate that data push that data into a dozen system and report back to you with basic monitoring of how that's going so the existing has really changed our business mostly because it finally gave people the building block. They needed to start treating their data. Not as a one off. I think really everything we're trying to do here. As a company is about helping data practitioners data analyst teams treat their job and their output as a product so that they are not just reacting and doing. Oh well let me read another query for you. Let me fix this report for you. But actually saying how do we start to get leverage out of our data investment. We spent all this money on the infrastructure. But how do. I leverage up kind of all of the analysis that we're doing and that is what. Dbt has really made possible here and census was as soon as we kind of. I guess we came across two years ago and as soon as was possible to start integrating with it natively great analysis of it another category is the workflow scheduling notably airflow. And then you have up. Commerce dag stir and prefect. Yeah what are the roles of the workflow scheduler. And why airflow. Insufficient wire their competitor's airflow so i think you're going to start to notice a pattern in my responses right so if you once again you go back to our trusty handbook of software engineering than we had code and versioned code for decades before we had tariff on right so you can think of dbt as really having laid the groundwork to give you version. Four the code which is the data models and what airflow and actor and prefect are and all these orchestration tools are doing are saying. How do i bring your the logic of how all this moves around and the state that you intend to exist across all your systems into code as well and you know airflow was born before. Ddt rights with of course airflow is also solving the problems. Dvd was solving around the dependency graph of sequel crees. People were doing that as a series of airflow jobs which some of that has been simplified. But at the end of the day you are eventually dealing with heterogeneous systems even inside your house right. You've got maybe an m. l. process happening with a totally different tool set. You've got some sequel happening in. Dt you might even have sufficiently large team that there's different teams doing that stuff. And there's higher level dependencies that you need to orchestrate. So that's you know what. I think those tools are there for to your question about. Why do we still see innovation there and wise airflow not enough. I think that honestly comes down to the audience for these tools is bit by bit and i think we're really at the beginning of a ten year kind of journey here becoming more sophisticated and i think they're becoming frustrated with tools. That are not as great as they can be for this in every category so they got fed up with doing sequel worksheet and they moved to dtn. I think even the people are using air floor. You know frustrated with some of the limitations in our or the user experience. They're trying to find something better. And so that's my high level answer for you as to why there's innovation in that space. I think deep down this audience is just learning about these techniques and as they become sophisticated they want better and better tools predictions for how data engineering is going to change in the next. You know five ten years. So i think the first self-serving answer i think is a lot of the simple ish tasks. Indeed engineering will become automated using technology. Like census right. So the kind of if all you're doing is need. Engineer is connecting salesforce down into snowflake like that's no longer a good time when there's tools like five train around and vice versa. Right if you're there just to try to create a pipeline that you maintain between snowflake end salesforce the other direction. That's also not a great use of your time. And there's a products like census to take care of that. That said i think the next five to ten years for data engineering is this. Push towards productized I think of traditional data team was actually split it would have bi under the cfo and they would be given effectively a warehouse with stuff in it and they would do analyses right and present it in tableau or something and the data engineering. Insofar as you had those people on staff lived under the engineering work right. And i hope they're not living under a cfo but it has happened. Based in some companies. That i've talked to which is unfortunate the new data team i think is reorganizing itself to be a like its own entity in the company. So i think that's the first significant change. So i think you're gonna start to see more and more this idea of a chief data officer arise and the reason for that is if data is not just a afterthought or like exhaust system right. It's not an afterthought it's probably more of a backwards looking team for a lot of people it's like. Hey what happened last quarter. What happened last year. And can we run a regression on users and those kinds of things. But as i as a and took over on the product side and then now with tools like census people are product -tising either analysis for everything the business right whether that's marketing sales support finance. All of these organizations are being enhanced using data. And so what you're going to see. I think is a restructuring of the data team to first of all merge these various practitioners right. So there's going to be data engineers analytics engineers data scientists living under one house and the court. The date engineering staff. I think is to become increasingly in charge of really high scale. Production data systems. So i don't think let's say you. Have you know to go back to our way at the beginning. You know you have your caucus setup for connecting various services within your life systems like what users directly experienced on your product side. That's going to be a significant part of what the data engineering team. Take care of right. So you know. This is the the pub sub infrastructure to make sure that your elastic search servers get every single update as quickly as possible from the other side of the business right. That's i think. Where a lot of data engineering folks it's going to be and potentially purchasing and owning core core infrastructure so buying confluence snowflake and maintaining those systems and the permissions around those systems. And all that stuff. Then you're gonna have this middle layer of. Let's call analytics engineers. This is the kind of bread and butter of dt has championed as well as census. So these are people who have historically not been engineers but are kind of coming more evident that they are becoming like engineers and their job is going to be to create the central models for the whole company. So these are there's a whole transmission pipeline here to try to build these unified. What are the users we care about. What are the companies we care about what is it de duplicated user. What is a company is an email domain. is it actually twelve. You combine the all these things that you have to do. And then tie every metric to them. And then you'll have the data science team that is there to kind of do ad hoc experimentation forward deployed analysts that are working with a specific team to kind of work on a process so like you might deploy an analyst out into marketing for a project and so to me the shift of the next ten years. Is this kind of unification of everything that data team does under one umbrella and it will likely be viewed similar to kind of a product technical team rather than hey. Bi team kind. Sounds like in your vision. There's a lot more stuff that can be bought or added onto enterprise products like confluence or elastic search or whatever that are going to save the time that currently is located to data engineers today. Yeah i think the way. I see it is as a business and as a human right. You wanna try to work on. What is differentiated and unique to you. I think there will always be some data engineering work. That is hard to buy. Some of that will just be like well. It's really weird database that none of the vendors want to support so we ourselves. That's obviously that's a whole chunk of the world right there but for core things. I think you're not going to build your own snowflake and you're not gonna build your own kafka so you're gonna do very little reason not to buy off the shelf. There that said as an engineer you will still find lots of ways to add. I think scaled out value and the emphasis should be on what is unique to your business same for an analyst right like the date. Engineering data analysts are both trying to find unique insights in their business. There's still a long ways to go. I think just connecting production systems will still be a lot of work for engineers and look at how much work there is still involved in deploying an ai model directly into product rain. It's still kind of a mess. So i think just running on the product side. There's a lot of work for dancing. So i think i haven't looked at the stats but i think the number of job titles with date engineers going up and that's a good sign that i think everyone is trying to invest in this all right well. Boris pleasure talking to you. It seems like a good place to wrap up and thanks for coming on the show. Thanks for having me.

krahn Cto dr rebecca parsons Neil ford polychrome boris martin fowler danah boyd xabier Marg salesforce moore ken arizona dropbox Marquette dbt plo
Exploring Event Modeling with Adam Dymitruk

Hanselminutes

35:58 min | 6 months ago

Exploring Event Modeling with Adam Dymitruk

"Today's episode of hansel minutes is sponsored by data dog a monitoring and analytics platform for cloud scale infrastructure and applications data dog provides dash boarding alerting application performance monitoring and log management in one tightly integrated platform. So you can get end to end visibility. Quickly be proactive with your monitoring strategy and catch issues before your clients are impacted. Start managing the overall health of your environment with a free data dog trial visit. Data dog h q dot com slash. Hansel minutes to get started. This is scott. Hanson was another episode of hansel minutes. Today i'm chatting with adam event. Modeling pinch me all of that. All things events. Are you sure. Good scott happy to be on the podcast. That's been years of waiting for this day. So thanks for having. I appreciate that will lower your expectations now but You talked to me about event. Modeling dot net fringe. Which is a conference in portland a number of years ago and we have a little youtube video up on my site. That's a couple of years old But if you go around and search for like event in google it'll auto complete a number of things it'll it'll autocomplete event modeling. It'll autocomplete events sourcing. Sometimes you'll get event storming. Maybe we can start by understanding this concept of events and explain this kind of this new way of thinking about system design. That's been actually not new at all. Yeah that's a. It's a interesting journey. That i started twelve years ago. By looking at the technical side of event driven systems and slowly realised over over a decade long time that It's really how we always thought about information systems long before digitisation I always go into the fact that digitization is just a small blip on human history. It's only been really the last fifty years of even that stretching at of modern And one of those things is application of of moore's law knots computing power but the storage and the transistor gave this giant leap for processing power. But we didn't have a transistor for storage of information and so there's this mismatch and this is where we came up with Tables normalization all these technical things to to make digitize information system significantly different than the way we treated regular information systems on pen and paper. And what's coming back now. Is that all of. The tech has caught up including storage. Now you can have a run on the entire company from the nineteen fifties global company from the nineteen fifties on your phone Just your phone. Has that much storage. So that moore's law aspect as apply to storage has caught up and so we no longer have to have Contrived abstractions ways of thinking to digitize systems yet. The ruts in the road are deep. And so that's the way we still learn Systems and it's really about on learning and thinking that hey humans think in stories how would civilization started is really about telling stories to the next generation so even before the written word You know a grandpa and grandma would be talking to to their grandkids around a fire in a cave and and this is how knowledge in memory got passed onto the generation so the story like nature of our minds and being able to key frames from stories or pictures or it's very interesting field of study just to see how we can with a very few data points alcohol. Him facts or events recreate Reconstructed entire story and so this is a really fundamental shift. In the way we designed systems whether they're manual or digital so that With human to talk to them. We already had clues about this. With the behavior driven development specification by example the reason these were successful is because they played in harmony with the the human minds worked. And the reason that you wasn't so highly adopted as because graphs and multiple documents and so that didn't stick around because it was hard to remember them and it was tedious to go back and do that but we can get into more of the other details as to why that's a little bit more onerous and how event driven thinking and event modeling. Help you of fat problem okay. So let's let's back up and level set for a little bit. 'cause you're a lot to unpack that. Oh yeah so. You said digitizing systems so systems can be a human interactions. They can be processes that one goes through. They are as you said a new thing in the last forty fifty years Assistant might have been someone comes in says. They want to check out a book from the library. Someone fills out a slip of paper that starts a process and in that paper becomes alleger of sorts. Then it gets put into the book nothing and every library in the world might have a little bit slightly different system. Yeah exactly so. I generally stopped talking about building software. Talk about automating information systems because how many people are doing brand new encryption scheme As their daily driver for in in their job very few most of us that are out there. Working in programming are automating information systems. Moving information from left to right calling on. Api making sure that they could the response in store that response and so on. There's very little you know craftiness to it in terms of being very smarten Coming up with brand new sorting algorithms such. So it's really the same thing over and over again. And would event mulling does distills down that molecule. Blood is smallest. Change that we can calculate and then have the fewest amount of patterns to describe any information system sort of like seeing different information systems as snowflakes different snowflakes. Your insurance companies much different than my insurance company yet on the very lower level level. We see that we have the same sort of state transitions. Someone does this type of action in this event has happened. Same on this In my company and so we can start to identify these tiny little pieces of patterns like small we have identified a core four patterns that basically describe any information system. And that's that's sort of like the answer to the patterns soup that you find if you buy all the martin fowler books and all of the different patterns books you're gonna have you know three hundred different patterns. And how applicable are those. You might exercise three or four of them. You might prefer one quite a lot in the specific language but there's really no underpinning sort of like a just like a nuclear physics particle soup in the early nineteenth century twentieth century. Sorry with With all of the different Subatomic particles but then someone said well. It must be something. That's common to all these these are made up of. They found that right. I forget what they're called corks or i forget. I'm not an expert field but there's a very interesting parallel to that. I'm insane with matter and energy with eagles quite a simple formula on top of all of this other research. That underpins all of that. So what we've done is sort of the same thing for information systems to identify these inputs and outputs in how these various small little patterns can be used over and over again to stretch out any complexity and described in very repeatable tiny steps that actually give you really good estimates and other things you can reuse your whole stack the same way. A brand new junior can go in and copied your previous step and just change the contents and continue on making sure workflow continues to act as illustrated by the way. I'm sure that folks will probably chat us on twitter or line about our physics and whatnot. That we're thinking about like the higgs boson that was discovered twenty twelve and then the the different particle. Oh or gorsuch analogy. And i'm sure we'll get totally slammed for the him. No strange quirks work. So when i was in the ninety s i was at big giant. Enterprise companies like nike. And we we had order management systems and it was like the height of object oriented programming. It was the height of of all those things. You described all the different factory patterns and stuff and you know it was. It was a time when you could roll into a a place with a whiteboard and start charging people money to teach them about the factory pattern and one of the things that we would do is we. Would you know argue about how to name things. and Fast forward where does object oriented programming and lovie entity relationship diagrams. You are m diagrams rather fit into this kind of his kind of world because fowler still uses. You know a very specific way to describe systems that is Language nonspecific has it bent. Modeling fit into that. It fits very much in the same way that it's non language specific in terms of The a specific type of Wording that use use a business wording for you events such as user registered or a book was told from the library and things like that however the underpinning of what you're describing is quite significantly different with object oriented programming paradigm with those enterprises. You're talking about from the nineties It goes back to that ability to store everything and we didn't really have ledgers back then. Something would add an archive table beside a table. To show know what the roads used to be like just for for for traceability. You know the unclogging a and logs would do that but it was really kind of You'd have to go dig for that. Information wasn't part of actual online transaction processing. It was for historical and for emergency type of operations However now that we do have a ton of storage at our disposal we can actually by default not throw away information right and so this is about more about events sourcing the event modeling but it's where the founding sort of ideas come from and so what What we would do in those days. We would need to have a third normal form. We may make sure we wouldn't repeat storage of an address. For example because storage was a concern. And also the distraction of of what is state. Now that was really important what it is now and we really didn't mead or thing that the history of how we got there was as important. That's why we ran into a lot of trouble in in software Now fast forward to event modeling of by default we keep everything and we actually are quite loose with what the state is. In fact the most responsible thing you can do when someone registers or takes another action. The system is just a store that fact and then let other systems later figure how to interpret that fact. That's the more responsible thing to do because we have fewer moving pieces to establish that. Hey this happened. And now we know what happened and we can move on and basically build All the other interpretations when of there needed to be sort of like the agony but in for an architectural perspective Hopefully that makes sense that we have a of an ability to include time in our abstractions whereas before we didn't have that ability because of storage constraints folks who may have english as second language or not understand what you just said yagudin. Knee is an acronym raise. You're not going to need it. That aren't gonna need it. So your your goal here is to take complex systems which we tended to turn into even more complicated software systems and express them in a way that is more natural for humans and thereby become as as complex as possible here using kind of You're changing the size of the lego pieces. Now that we can build complicated things but the the the corks the the legos the molecules they the the essence of what. You're doing that the size of the chewing the pieces that you're chewing off here are are limited inside. That's right and and your seems to be like. We always talked about single responsibility. Principle did object oriented programming. You're taking that that responsibility very seriously as absolutely and other Principles such as open closed principle. A lot of times these. The state transitions talking about those lego pieces you're talking about. We call them slices because we include the ui in an event model which is really important that you why you x. Part of the system even it was a back end system we add in the ui for monitoring screen half the people there are visual learners and half analytical right or some and there's obviously overlap both In the in the space of all of our architecture talks we really didn't talk about the. Us experience always import to tie it back to what these are experienced is overall in event. Modeling provides out of the first class citizens. I think that's one of the biggest differences between event modeling in any other kind of modeling or design discipline. That's their ties. Everything together in really does go from the boardroom. All the way through to implementation testing and maintenance of your systems version the systems and and migrating to new versions of all of those types of things are considered when you're dealing with that and so back to the lego pieces that you said a real good really good analogy. We established that. We're just gonna use these for different blocks in that way because we're using these four different blocks. We get a really high sample number for when we use them. And when you get a high sample number you can get a really good average. And so now he can do estimates empirically like from this team can do this many Workflow steps column or in a given time for a given project of a certain size and now we have something incredibly reliable. I haven't seen a more accurate. Estimation process in my whole history have been writing software for the commodore sixty four when i was a kid in the eighties and they have never seen a way to predict the of software that that match this which is really one of the i mean that alone should be gold for businesses and then i just want to make sure that we we touch on a very briefly for those again. Who aren't may not be familiar with these different terms. The open closed principle. The idea that something is is open for but is closed for modification and that seems to be pretty fundamental. Like you're you're taking some of these fundamental tenents of of solid solid principles of object design. Exactly and you're supplementing them and insisting upon them and then taking it to the next the next level exactly we're making that concept available to business because because we chose such small lego pieces it's easier to throw it away and replace it with the correct ones in go in and have to swim through and and try to fix something on. This is a very powerful methodology as well for rooting for doing a polyglot developments. For example if you starting to And for those. That don't know. Polyglot means using multiple languages in the same solution than even the service which is. How could you do that while. Imagine if you have to hire someone. That's an expert in your field. Maybe it's a library and This person is an expert c. Sharp developer your solution is going. Would you wanna send them on their way. Because the tech stack doesn't match or force them to learn cpr or vice versa. Or go lang. No we should be able to take these. Little pieces of state transitions have them done in any kind of stack that you want. I mean we already see a lot of this happening in the service world. Right which you're familiar with but that that doesn't stop us from being able to do that on traditional Topologies that we may still be using today. Okay so when. I talk to the regular joe and jane developers out there in the world the patterns that i tend to hear a lot about tend to be a little closer to the glass of the monitor. If they're things like model view controller and they're thinking about view models and things like that What kind of person. What kind of engineer or architect are thinking about the things that your level. This is kind of back end stuff as things that don't have a ui or does this cross cutting is incredibly cross cutting and it even cuts upwards into being able to use event modeling at the c. Level director level for being able to zoom out of that time. Line because event has a core timeline on it by example. It doesn't have splits in terms of pass that you could take like an activity diagram or some other flow chart. It says this is design of the entire system by example like reading a storybook. But if you zoom that out and just keep some key cornerstone events. Such as payment completed or library book signed out something that identifiable at the at the director level or sea level boardroom level decisions you can remove all the other ones and just have these small pieces of the small indications of how far you got and say that you know this twenty on the your twenty twenty two twenty twenty one a roadmap includes this part of the workflow but behind us. We see what happened in the previous five years and in front of us. We see what's going to happen the next five years there you can start to analyze that. Hey maybe digital currency is something. That's going to be more ubiquitous. It's going to move in from being an experiment. To being a product we can use Or at least some custom development that we can do with reliable libraries to do that with Or it's going to become a commodity words like email right but we can plan our idea of where a company is going to be in relation to all these but zoom into event model for the actual meat of it. We see that out. This is an example of joe. And bob using the system and the manager. Let's as a hotel system Hotel system and this is what each person sees in their email or each Each screen on the computer when they're at home and this is what the manager sees behind the counter when they check in and they have that story. I see all information that goes in and out of the system accurately in that also gives me other thing it gives me something called information completeness. Sappho i have a bit of the most important part of the design is really about making sure you have the storage places in you. Understand what data's in the system and this goes back to mythical man must with with a with fred brooks so mythical man month is a really old book from the nineteen sixties i believe and gets quoted for a bunch of other things but i think something that gets missed. Is that fred book said. Show me your tables in understand your entire system and how it works. Describe your Algorithms and schemes and will continue to be mystified. So state is that important. We've kind of been ignoring it. more-or-less in terms of where we spend time in design If we use state we actually have a much better anchor to reality to actually what we're going to need to implement and so for us it's been a driving force for the last five years at my company and we own the develop things this way if someone sits down with us and says that this event model is correct. This was is what. I'll be happy with or the or they actually do the event modeling with because the very easy to do. It's it's a very low fidelity easy to explain practice. Or at the least years of showing what screens people see as time moves forward and then what information gets stored along those same timelines and ensure the interactions by example. Say hey at the end of the year. If i have this system doing this i'm good. That's that passes and can slice that up into these little pieces that we talking about before to give a proper estimate and cost and time in all those things you know how your op. Steam keeps a pile of scripts and wicky documents explaining how to perform those routine and emergency tasks that keep your applications running. They might call them run books or playbook our friends at octopus deploy were thinking you know. Devops is about collaboration. So it doesn't make sense for run books to be automated from the same place as deployments. Well octopus deploy is now the first deployment tool with native run book support. The best thing. Is you run. Books can share configuration settings and automation. Steps with your deployments. Find out more at octopus dot com. So if we are a company that is already or a person is already familiar with like you. Mel unified modeling. Language is greedy bouches. Yep way of thinking about software And we're designing systems and we're used to doing things like swim lane. Johnson and stuff like that Expressing something in the context of event modeling is is going to be very natural. I absolutely is going to give you a fast. A the turbo boost for that approach Because of grady's excellent work is born on the best way to express what the program is doing. however it's a little bit owners because you almost have a one to mapping between your classes and methods in all the other things you want to show in your a mallender seven types of diagrams. So essentially we looked at the goals of what you suppose to be used for or any kind of design methodology in say can we achieve those same goals with fewer parts and the parts that stuck out as the biggest piece that we could remove is the how. I don't care how you store that information as long as it's stored taking that how the minutia of the implementation really reduced the amount of work for designed to be down to like five percents. I mean we've been doing event modeling sessions from an afternoon to two days for startups to large fortune. Five hundred companies There's an entire mind. The deepest mine in europe is is event money to coordinate sixty different subsystems that need to be coordinated together. We have large international banks like capital one using the coach. So event mullings Basically beyond me. Now it's taken off as a way that holy smokes. These guys proved that you can ignore this. really chunky. Part of design and reinvest that time into all these really awesome things like x. Ui test ability the a mindset of specification by example on a very large scale and basically break down walls make very wide bridges between all the different roles in the company that you're all speaking the same language a lot of times. I say this is the blueprint for when you're building a house. We're building a company will busy a system. We have a blueprint. We can rule out on the table and anyone can walk up to ask a question. Get an answer immediately. It serves designed documents. When you're building it serves as a map when you've already built where you wanna find. Things were things have happened. It applies to traditional systems as well as new event based systems so traditional system state might just be represented by By rose being shown at the bottom sorta like a sequence diagram but turned sideways. And there you would have the state of each column in each row that's Play right now when they get changed and that's also weigh in traditional systems. Couplings of big enemy. We have different pieces of functionality Coupled together by the same state and some table and you can show that you can show that in my workflow. I have this thing at one end doing something here in this other thing at the other end doing something almost completely different. But they're sharing this one row and column from this table. I can show that against say. Look if i need to change that part this is going to drag in with all depends in We've all been there. We walk on eggshells with some of these large systems where we start to you. Know tweak wonderful method here then oh my goodness we just all the integration tests fail. We've all been there. It would be nice to have a map to navigate that it's listen. We either have to augment something on the side but if we change this we have to consider you know five ten of these. A lot of times are done via you. Know a analysis of dependency graphs and other things. But some of these things are more insidious where they're just about the fact that something is stored somewhere in in state and it might have been kind of a side effect that someone someone's collegiate in some extra coating in the field in value in the field somewhere from you know as the systems grow bigger and as you get more into technical debt and other things you see more and more of these things. Needless to say You can basically use event modeling to get yourself out of those situations. Do some empirical strangler pattern stuff. I hate to use patterns again. I'm not trying to just get things down before patterns but For people that are Familiar with some of the works around legacy systems all the strategies. They're they're made much more easily. Accessible via event mulling because he can use the state approach that allows you to really remove some of the really burdensome Ideas about how and just empirically look at what happened in. That should fast track to getting somewhere a lot faster and get some better tests place other things or entirely you know start to implement things on the side and still integrate with old systems. Those are all possibilities. So one of the things that i think is worth noting As we get towards the end of the conversation here is that we've been talking about things. Referenced the nineties you've referenced tables you haven't said anything technology specific you haven't headset sequel server mongo or anything like that when we say things like cloud native or born on the cloud Even the cloud didn't exist when these systems were created. So you're not dictating. Where any of this state is stored at could strategist. It could be as your storage could be wherever this Nondenominational from this is a religion of yours. But it's a nondenominational religion that really supports all system as more like environmentalism right. We care about the plan in a good state and say it's more of that than religion every religion can agree on. It's being a responsible citizen in in your world of software to really open up the communication channels up and down be transparent Know exactly where the work's going to be if someone makes a decision See the impact of that. Make it not something has to go down the chain of command through four different roles and by that time you don't know who to hold accountable for why things went wrong or or why there's a change that needs to be back bubble back up because it might not be the best implementation So a lotta times Modeling diagram is shared and actually collaborated upon by the most amount rule. So that you can have people you know from. The us us on designers is a really bad idea. We shouldn't have these screens order. People hate that Which regular engineers have really don't care. I'm just making the is here And vice versa. Someone can suggest some. Some type of a workflow were screens are shown. This order and the engineer can pipe up and say you know this is actually quite bad. We're going to need to reallocate all this extra storage and whatever else. These estimates will be hard to implement but they can have a back and forth to say. Why you know they have a form for that. They have a canvas for that where where they can actually look at both sides of the of their concerns in in have a discussion. And it's not about us. It's about them standing shoulder to shoulder and talking this solution in front of them right. That's a big difference. Once he sta- start. Having the design documents. The designers hold the diagrams. The developers hold this strategy roadmap the boardroom holds and so on and so on right that creates islands and and really requires a lot of translation. What we want is a better form of communication. We want a collaborative medium that everyone can understand and have that so sometimes when we talk about what s delays or maybe security we can introduce the idea of overlay on top of the event model. We can say well. This'll jumped from this you. I hear to to actually Register that's you know that's got a security risk because it's going over the network. We have to make sure that. That path is encrypted. We can have overly were. That part is highlighted in red in the security. Overly right. it's just like you would have a blueprint for a house where you can start to put overlays for plumbing or electrical heating or whatever you have and you can have that discussion with respect to the whole house and still understand where you have to go with. Florrie have to be on to actually put that sock or whatever you're doing that's actually an interesting analogy right there. We know often in the last twenty thirty years people use construction related analogies. And they talk about how you know. Houses aren't falling over. And we all understand how to talk about and think about houses. Why can't we do the same for software. And then of course people go the naysayer. We'll go and say well. That's that's not a reasonable analogy. That's a trash analogy on. Software changes this and that other thing. But you're you're saying that in fact that blueprint that here's a piece of paper design house design. A system is actually possible. Yeah i think so. We have the right pieces. I don't think we had the right small enough pieces to do that. Analogy properly to have that. Parallelism shown in terms of effort of development to equal the cost of a brick or a spanner for bridge. Right that bridge analogies always been. It's tired thing that software is not like real construction site. Well no it is. It's not so far removed that it isn't than so what we're doing. Here is identifying the bricks that make up our structures and yes we a house who might not rebuild all the time but we certainly do maintenance on it We certainly want to be at a point where. We don't have to demolish the walls because halfway through you learned that we need a second floor which is kind of like the you know if i want to put as on the bad place. That's what i would say your ideas that you're going to build the first floor and then suddenly next sprint. You're gonna find out. We need a second floor so attempt to demolish those first walls and put up supporting walls. Because there's to be a second story sounds ridiculous but that's kind of how we have How far the pendulum has swung away from you. Amal and all that right. The pendulum has to come back to somewhere in the middle. And i think event modeling is really where that pendulum rests and why it's so successful in so many different companies. A totally took me by surprise because now this was just the way i wanted to work in my company because we found that this is the most efficient thing we can do. Were small startup. We know with a specific way of doing things that very few clients can appreciate because it's not widely known so we need it to be incredibly efficient and so have a little joke cartoon on my profile Where you know event modeling is all i need. All these other. People are buying in there costco shopping cart. The perspectives on the stand ups and this just giant pile of practices. What we're doing is that we had a really good to allow me to. I can't afford to wear a tool belt with fifty tools on it weighing me down. I need to find the efficiency of finding one tool that will replace ten of them right and make those efficient choices. And i think for the event mulling was that efficiency choice it came from actual struggles to build systems fast. Cheap with high quality with predictability on fixed costs like all of these interesting things like events sourcing event long. They all have come from not rich places where you could buy whatever you want it. They came from Either high performance demands or are limited budget demands or incredible time demands. Like every time. That i've done this with with a hackathon. The hackathon was one because in fifteen minutes can basically event modeling as get a rough design out there and with the first to finish hands down. If you've it's a secret weapon is not modeling and you go to a hackathon you can get your teams on the same page so fast you have you've just doubled or tripled. You're the chance that you'll that you'll win Simply because everyone knows what to do that ability to slice up your workflow into pieces in the fact that you've you have a contract in between all the state transitions as you move left to right. Through the example you actually have a very strong contract that will guarantee that things clicked together correctly in a very easy way And so you don't have to do a lot of rework with a hackathon nightmare of like. Oh i don't know what john did but his. Api can't call you know those things and struggling the last hour to debug a just a nightmare. The the event modeled ones are are way more sane and they just continue to slowly fill in that blueprint the the color with crayons which parts are done. You can have it on the whiteboard. It it's a really good test so these high stress places really do that. The good thing is that it actually scales so you can go from tricycle to bicycle too motorcycle because the infrastructure that makes state transition happen can be changed. So if i'm just doing stuff on like a shell script to make one st transition because i'm in the middle of a hackathon a company. I know if that state transitions the same. It'll run on like your scaled out. Server listing in asia problem. That's awesome guarantee right now so folks can learn about this at event modeling dot org again. It's not a product to buy. Anything can go and explore all the different posts up there. I would recommend that you go to a modeling oregon hit slash about to get a little bit of context. And then you've got a what is event. Modeling of mulling. What is it post from. June of nineteen is a very clean about a twenty minute. Read with lots of diagrams at a walk you through the entire thing. Step by step. And there's also workshops people can attend find videos of you online. Yeah there's lots of new. There's lots of news to happen all the time. We just finished the week long crazy workshop. That was forty hours of instruction in two time zones which was repeated which means eighty hours of of stuff. Eight nearly died during that workshop. But we're very committed to to helping as many people in the world as possible. Get more done for less with higher quality all the good things. We're all after. I'm almost finished the book. It's in draft stages. It'll be published Sometime january already mentioned the website. We have a slack community right now. Ten thousand people there but they're all helping each other out in ways to use event long. Describe their systems and And at aveiro ganic community. I i hardly ever mentioned. But more and more people seem to congregate there so. I hope that i see you there. I'm very excited. When they see new people always tried to say hi and help them immediately with whatever questions they have so More than the happy to that. That's the slack. Channel invite can be found on the resources page very cool on event. Modeling dot org. And if you don't remember the address if you if you just google event modeling or being event modeling it'll come up as the first hit all right. We've been talking with asked me trick you can check them out at a modeling dot org and this has been another episode of hansel minutes you again.

gorsuch moore yagudin scott Hansel hansel martin fowler fred brooks fred book Hanson fowler portland joe adam mead eagles youtube nike google
Episode 134  Startup Culture

The 6 Figure Developer Podcast

31:06 min | 1 year ago

Episode 134 Startup Culture

"Name Hello and welcome to the bigger developer. Podcast the podcast where we talk about new and exciting Technologies Professional Development Clean Code career advancement and more. I'm John Calloway. I'm Clayton Hunt. What do you think of when you hear that so and so has a startup culture? Do you think about the things. That are probably positive. Like fear of ping-pong maybe beanbag chairs or do you think about the negative things. Like Eighty Hour workweeks company offering you sweat equity for putting in those extra hours. Maybe even being forced to sleep in the office or crazy and unrealistic expectations. I think of kind of a mixture of those things typically like any of these discussions. It kind of depends on what the context is. Is this a recruiter? Talking about a particular organization has a startup. Culture is this an organization declaring that they themselves have startup culture. So what does it mean? In in what context the three of us have all kind of worked for a start up before I mean things. Things have their moments but the it wasn't sort of terribly chaotic. I think we were pretty successful at keeping sort of progress moving forward but changes happening and and etc in certainly depending on the aspects of the business or the asks of particular clients. It can be a little bit more chaotic at Times. Trying to make sure that we're delivering the value in a timely manner in an effective manner figuring out what the original ask is in figuring out if it's possible and how we can deliver that feature or pieces software. Yeah personally I don't. I don't think that I have worked at a couple of startups. But I don't feel like I have a huge amount of experience at startups. That would make me think that they're really any different than any other companies that I worked at. I mean I've worked at a huge companies that had ping pong tables because they knew they sucked and they wanted to draw the developers in somehow. I've slept in the office but not from working late hours. Just because I was really tired I usually stay away from companies that that have the long workweeks or like. Oh Hey we'll give you equity on the company that usually means that it's not the greatest place to work in my experience. Those things aren't unique to startups. They're just things that happen at companies. I think I would kind of Second that opinion I I feel like sort of like you were saying John when someone says hey this is a startup culture or we have started culture or this is a startup. I kind of look at. Who's saying it? And what are they trying to communicate with that the word startup or the idea of of in many ways in my mind. I don't think it's so cliche that I don't really have any real association to it and I just tried to sub in what I think that they're trying to convey if the if it's developer trying to warn me off in there like it's a startup called you know company in there probably friend warn me about the hours of the chaos and the you know long workweek or if it's a it's a recruiter or company itself they're trying to say. Hey We have a fun environment in we break things and you know but we do a lot of work so you know but we started to have fun at the same time. So I don't know that's the way I kinda take it and it seems like the industry is starting to mature a little bit. I still feel like the industry is relatively new. Relatively young still going through a fair amount of growing pains and figuring out what it is people enjoy figuring out what a good work environment is what benefits people enjoy in that type of thing years ago it was the googles and the facebooks saying that you know we we. We want to make sure that everyone's in the office at all times. You're putting in eighty hours. A week will make sure you're you're fed will bring in catering will offer dry cleaning services in things like that. Facebook was notorious for early on their motto was move fast and break things but I feel like too. Many companies try to emulate those same thoughts and ideals without truly understanding their purpose or their execution. And saying that. We're going to move fast. We're GONNA throw out all of our professional knowledge in working well and clean. All of our past lessons learned and we're going to just go ninety miles an hour. Dan Torpedoes I see that as as kind of a sign of our our immaturity as a profession. Nobody knows what works and nobody knows why. The things that are working are working and so we are all too quick to jump onto a trend like ooh. What's Agile we should do that? I don't know what it is but I'm going to get a five page book and we're GonNa do agile or you know startups they. They're just they're the boss. They're doing everything these days. We need to be a startup. I you know like even corporations like well we you know. We're we're trying to get that startup culture like they're just trying to fix the fact that nothing they've done is worked and I think the whole industry is doing that. It would be nice if we would stop jumping on the bandwagon and actually like learn from our mistakes. Understand the the thing that we're trying to achieve and before from there but if I it might be a little while yet before we are a mature enough to do that made the to to kind of a little bit. Come AGAINST THAT. A little bit Isn't that kind of what they're trying to do. Though isn't isn't the attempt to move to another new trend or whatever attempt to try to improve to an attempt to try to say okay. What does what does these things. What are these things doing right? We need to make ourselves better so we try to follow this trend so that we can get something right. I think I agree with you that the execution is poor most often. I think they're heart is in the right place right like they. They see a problem. They WANNA fix it. They won't be better. This other company has done really well. Let's imitate what they're doing right. I mean you you see somebody being successful you imitate them so that you can be successful. The problem is that we are. We're doing like that surface level imitation like you watch a movie and then you learn how to say the line from the movie the same way that the actor did that doesn't make you a great actor right because you don't understand all the process that goes into being a great actor and it's the same thing that we do we. We imitate what we see on the surface without understanding why the other company came to those decisions or why this practice matters and so we end up doing it poorly read a blog post. That's basically written at a third grade reading level and then we assume that that's going to contain all of the information necessary to implement and understand. Why decisions were even made in the first place. Yang. Exactly there's not going to be the cure. You both kind of alluded. You Think. The heart is in the right place. Is We know we can do better. We've heard we can do better if we adopt agile processes and procedures. We we know that we can do better. If we adopt micro services we we can do better if we insert buzzword but without truly understanding. What is those ideals helped to solve. What problem so those helped us off and the execution it takes to be successful with the with those mindsets have heard about low code and no code movements of heard about all kinds of different processes and procedures. That this is the new way to be successful in. I've been in those environments. Where we're we're moving thousand miles an hour. We don't have time to sit back. End gather our thoughts and work cleanly and when you when individuals bring up that. We know how to fix this. We need to slow down. We need to plan better. We need to effect change. We need to work cleanly. I've even been told that some people just aren't cut out to be startup engineers as a result of trying to bring up those issues. I've been told that startups can't be run like that. Even though those organizations aren't startups and that also is in attempt to negate my own personal experience of the startups. I've been involved with have been successful by working slowly by working cleanly by getting the trust of their staff getting the trust of their clients. I've also been told that some people are only cut out to be greenfield engineers too so some people are definitely cut out to only be Brownfield. Engineers the key thing that in in when you're in an actual startup like it's not moving fast and breaking things. It's not whether your agile or not. It's how quickly can you get your product to your customer. So that your customer can tell you what you've done wrong so that you can fix it so whatever. Whatever process facilitates that? That's probably what you need to do. That may involve moving fast and breaking things. It may involve agile. It may involve devops but it doesn't necessarily have to and hopefully you don't find yourself sleeping under the desk so was saying what I think. Is We kind of a little bit of the? We've talked about sort of the immaturity of our field that I think. Also the sort of where we are in our sights society and culture We really value these sort of like soundbite phrases right and so you think of the words move fast and break things you think of the words like low code no code right like. Oh we're going to code with no code right like that just sounds like mind blown right and so know the thing is we want. We want to have something. That sounds edgy. That sounds like we're doing something crazy but I think the reality is the move fast and break things is really an amato. At the hard things really is a great motto. It's just you have to break it down and understand what that mood. Fat Lucas means move fast. Means move well and do the things that are going to allow you to move quickly right break. Things means experimental experiment. And how a place in your in your workflows to be able to try. Try out new things and to say okay. This doesn't work and make sure that your system is in your systems. Both workflows and applications are resilient to change US right like that. That's really all that's saying but but if you just simply go we have to move fast. And that means whatever I defined how Get the fastest point. Two point. Eight e Okay. Go and throw out everything that we think it does not mean. Move like the Tasmanian Devil. Leaving a disaster area in your wake. Exactly yeah at face value. It's code with reckless abandon. That may have been the impression that they wanted. But that's not what they were actually doing Bingo. Yes that's correct and that's because we had like find some way to make it sound like you're doing something that you're not In the same way that low code or no code which no code really was the first movement that because someone finally had to say okay. Well actually there's coating involve again now. It came up with the low code movement. Right and the reality is like. There's still lots of coding vic evaluation and learning her. That's necessary for those things. You still have to learn the tooling. Yousof to learn how to handle all these things. It's just an. They may work great for some particular applications right but We wanted we wanted to sound like this sort of mind-blowing I'm doing something. That sounds impossible sort of thing. And that's that's what society really values. Martin Fowler tweeted out a link and discussion starter based on Elizabeth Hendrickson blog posts discussing momentum and urgency where Martin said lots of managers. Think that to get things done. You need to create a sense of urgency test obsessed. Which is Elizabeth Hendrickson? Twitter handle explains why it's more effective to create a sense of momentum on the twitter discussion. There's some discussion about. Momentum is greater than urgency and also even momentum is equal to purpose so teams be productive and feel that momentum away to help facilitate that is to give them a sense of purpose or or explain the sense of purpose. Slow and steady wins the race. I mean it's age old knowledge that that you need to move consistently on carefully if you WANNA get ahead right at the same time. I've seen that phrase be used throw breaks right into the middle of something that someone's going to make a change that would actually improve the workflow and actually make things better. But it's different than what you've done before end. Someone's like no no no we that's like that's much too fast. We need to like split. This take this to a committee you know sort of like put it on the back burner or let us think about it. Can Slow and steady wins the race. We just need to like you know. We'll think about that in. Maybe in seventeen months will actually come back to it and and you know implement it right. That's not really slow and steady though. That's like driving in first gear when you don't know how to use the clutch that's true but I'm just saying like we have to be careful that we don't sort of like throw out cliches that Kinda get worked the get thrown into that that other which is kind of went a large portion of this talk. Are This Conversation Yeah you can't sum up. The death practices in half a sense. It's a volume of information that needs to be considered you can't you can't just go but this guy said these three words and that's what. I'm going to base my entire life on. There's also a book that was released not too long ago by the the base camp guys Jason Freed and D. H. entitled. It doesn't have to be crazy at work where they have. A lot of. The soundbites crossed out on the cover. The eighty hour workweeks the PECs schedules. Super-busy endless meetings those types of things whereas on the back cover of the book it says how to run the COM- company eight hour days. Forty hour weeks plenty of time to myself comfortably paste days. No weakened work rarely meeting no rushing. I think that that kind of sums up my ideal environment is I. Don't enjoy having other people's stresses imposed on me. I'm fine with a sense of urgency. I'm fine with momentum. I'm fine with meeting a deadline. That is a true deadline that we must get to market before our competitors or we must get to market before a particular sales cycle or whatever the case may be we must start delivering value and start bringing in revenue. I understand that what I don't enjoy is we made a mess. And as a result we have to work nights and weekends. Yeah if you're having to put an overtime because it's impossible to get your work done because the code is a your way. That's that's a terrible thing. One of the sound bites. I guess you could say this kind of been ringing in my head. The last several weeks is I think it's a It's a Microsoft quote but it's in the either the Phoenix project or the Unicorn project and I can't remember the quote exactly but it's basically like the. Ceo of Microsoft told there the employees the developers if you have to choose between delivering features and increasing developer productivity. Choose to increase developer productivity. Because they know that if the developers can move faster by moving well that the features will happen faster so new so increasing one increases the other but increasing features only slows everything down because then the emphasis on getting done as fast as possible. Which makes you make a mess. Which slows you down. And that's a scary thing for a lot of businesses out there that idea of handing over. I think that's very scary for many many businesses in the best organizations that I've worked for the ones that I've enjoyed the most were the ones strive to make an environment that they themselves wanted to work in so whenever the the owners of their organization or the the leaders within those organizations set back in really thought about the environment that they they themselves would want to work in and tried to make that environment. Reality that's where most enjoyed working the converse to that is the projects and the teams and the companies that I disliked the most were those that were more focused on features features features features and didn't leave time for the innovation didn't leave time for the exploration because at that point all you're doing is driving towards the burn out that we discussed a couple of episodes ago absolutely in. You're not GonNa get the value that you could potentially get out of those professionals that you've employed it's a it's a very serious indicator for a company to pay attention to retention of developers and devops individuals. Because you know if you're if you're driving them into the ground and they're quitting then it's not going to be long before the software your company's GonNa Stop Working and whether you think your software company or not if you're hiring developer. You're a software company and without software. Your company will die. I think a lot of this really does kind of come down to like. How do we motivate people in have we we get those motivations? I think one of the things that you're bringing up earlier with Elizabeth's tweets the question of momentum and urgency i. I think when they're urgency is a real thing and allowing that to be shown to the teams. I think that's that's a sort of a. It'll it's part of the purpose right surgeons who becomes a part of that purpose Because the urgency is is being driven off of you know. Hey we have this product. We need to get it out there or we need to get this. This completed right men so it's sort of like really giving more transparency and trust to the team to say. Hey this is where we are in that gives the ability to make momentum and to to make completion towards towards that goal towards that Whatever whatever it is that were there working on and when what's damaging is when we tried to create either full senses of momentum or false sense of urgency so we're GONNA throw a deadline in front of you even though it isn't really urgent on our part we're going to hold you to hold your feet to the fire because we need to. We feel like we need to like get the whips out in order to get work out of you That just really saps all of the goodwill and motivation that you're going to get out of people you might be able to get work out of them if they are afraid of losing losing their jobs. Unfortunately it's not very effective in this kind of economy because there's so many opportunities out there that if you're half a decent developer they're probably going to be another place for you to go to even as an independent consultant in coming in and working with existing teams for for a number of different clients can of integrating with the team and and getting up to speed with their development practices bringing in my own development practices in explaining how I feel effective and how we can deliver value quickly together as a team as a cohesive team really kind of goes along way and gets guest that trust factor up relatively quickly once you start delivering the value delivering on your promises and showing the the people that are paying the bills whether that's your own company or client and showing them that we can be effective if we do the types of things that that were asking for that were that were saying we we want we can deliver the value we can commit to our promises and deliver up at Clayton. I'm reminded of the startup that you and I worked together. I Number of years ago when we were kind of figuring out on this this new renewed startup how to assemble the team and how to be cohesive as a unit in how to learn and grow together how to introduce test development town to introduce patterns and practices that maybe not everyone was familiar with how to gain the trust of the business to to figure out that we don't need death. Marches and endless sprints and nights and weekends in figuring out that we can commit to the things that is being asked of us. I forgot about the death march parts because we we did a really good job of teaching that that business that it didn't have to be like that very shortly after I got there. We started test driven development. Every we had like team launches. Every day is very cohesive team. There were some arguments disagreements here and there but for the most part everything about that team was great the memories that I have a working there and it would definitely was a startup small company. We were recreating the softer reimagining. It FROM FROM SCRATCH. Not Not even looking at the the previous stuff. All the members have are coming into work basically went. I wanted to being able to take longer lunches so that we can do training. If you had a problem you just grab another developer. Go into another room said at the whiteboard for a little while. Nobody really pressured US too much. We did have a deadline. We did have several deadlines but we even got to sit with the business person and collect our stories together. Form an estimate of when we thought we could get stuff done and then tell the businessperson. Yeah we think we can hit that date as shouldn't be a problem. We'll let you know if anything changes and even in the larger companies I've worked for. That's extremely unusual. I don't think I've been at any other companies where we've been able to to talk with the business like that so I never really think of that company as your your traditional startup. Because it didn't feel like that it was. It was a completely different feeling. You know like I wasn't sleeping at my desk wasn't none of that stuff we had a ping pong table. I guess but one one question I I'll ask is So you guys were. You're you're mentioning that you're tired of teaching this to the business and whatnot. So how did you get the buy in from the your team and that sort of thing? It was a little rocky. At first I was one of the relatively new developers on on the team third or fourth maybe Fifth Higher. Clayton joined. Maybe a couple of months after that. Yeah it was. It was a rocky time for sure that there was not a lot of specific direction and a lot of cohesiveness in the beginning. Because it was there are a million things that we need to get done. Everybody just go in without that unified vision without that shared momentum. Maria shared sense of urgency. Without that singular purpose we we were often spinning our wheels working in different directions working on different features in different subsets of of a bigger application. So it was a number of months to to really sit down and figure out a better way to work in that was a little bit rocky because we started with preteen of of Agile and scrum practices. We as an industry I think have started to figure out that if you if you apply those labels in the wrong time or the wrong context in there might be a little push back against it so if you can just kind of describe the the end goal and the process with which we want to get there without maybe using those over utilize terms then often the the end result is a little bit better so it was a number of months to to get everybody on board to see the value to meet with the proper people to pull out the requirements figuring out their requirements are necessary for assistant figuring out that you can't actually write software without requirements scramming. Agile where they say. We don't need requirements anymore is really not a thing. There are requirements whether you choose to discover them captured them right them or not. You must know what done looks like. Okay and then for sort of getting the businesses trust. I'm GONNA guess that a lot of that was really found when you're able to sort of again. Give them those deadlines or help them. Work on creating those deadlines and then delivering on them. zapped the only thing or is there was there other ways that you sort of Able to earn the businesses trust. There's definitely some give and take because a lot of times and I just got an email Aurora Lincoln notification that someone who was looking for a developer to do a thing in this amount of time for this amount of dollars. And if you if you define something within that box. There's there's really no wiggle room. There's nowhere to move so figuring out that there's there's a discussion to be had there. There's a finite amount of time. There's a financial amount of money. There's a finite amount of developer man our resources so figuring out. What's possible in the timeline? Or figuring out how expensive something is to implement and having that conversation and it is a process it is going to take time to develop that trust back and forth in coming to that agreement of what is feasible. What is possible? I think one of our big turnarounds at that company as far as convincing the business that we were on the right track we actually we took. I think it was a half a day. We took the the second half Friday and we had art day where we got some poster board. We got some string of markers and we drew grids on the poster board and we taped string to the board and we started tracking. Our our metrics are sprint. Metrics with string on a board on it like a piece of paper poster board that we had taped to the wall but when the when the business owner came for. The retrospect are for the demo. She looked over and she immediately. She was like what's that was like. Oh that's that's our progress. That's how we're doing and she got interested. And after a couple of sprints she could see that we were we were above the line. We we drew a line from start a project or start of of tracking to current number of story points. That had estimated we would bounce you know above and below that line a little bit but for the most part we were staying above the line and so she loved seeing that we were going to hit our target. Because we you know it was. It was a points versus time chart so we had the we have to finish by this date. So if we if we end up below this line that we're not gonNA make it but we had to. We had installed test development. We had installed user stories and a couple of things and so we were able to work through those without any real supervision and very quickly put ourselves above the line and stay above the line and I think the business person being able to see that gained us a lot of favor and then from then on we could. We could say well. We don't think we're GONNA make that and we had no. We have the data. Tobacco could show her on our chart that we weren't GonNa make it and she's like okay. Well what if we cut out this feature that feature whereas on most companies if you're like which one of these features is the most important because we don't think we can deliver all well. They're all priority number one. They all gotta get done. We can't there's no wiggle room but when you can show them with no uncertainty that we just can't get it done then they start going. Oh Dang Well that one's not real important. We could remove that right. It was it was actually like an amazing thing. Because I haven't had it happened since we've never been able to actually show the business people who are making the decisions you know that kind of thing since that that company but I think that was like the big turnaround was which is being able to display that data and have the business to be able to actually see it. That's why I really enjoyed information. Radiators in being able to show meaningful metrics in meaningful graphs in charge to people and explain to them what they're seeing in in how this provides value to them whether it's the health of our applications or the speed with which were able to deliver features all of that goes along way with gaining trust from co workers. Your Business Your clients and then You guys had mentioned that you introduce. Td and patterns and practices and whatnot. So it sounds like you also had a huge focus of your team side on doing following good programming development techniques and disciplines. Right that allow you to be allow you to reproduce the amount of work and continuously be sort of operating on a level that can be reproduced over and over and over and over again With a very good Predictable result. Yeah at that company even though you know the business logic is different from thing to thing. We had pretty knelt down process. We knew generally what was going to take to get that next page done and so we can estimate pretty good. If you can work cleanly and methodically you can. You can crank out some features pretty easily as long as you understand the requirements or the ask that is being made and with that dear listener. We'd like to hear from you. What's been your experience if you like this episode please like rate interview on items find show notes blog post more at six figurative dot com and be sure to follow us on twitter at six figure. Have this has been another episode of the six figure developer. Podcast helping others tip potential. I'm John Calloway. I'm Clayton Hunt and I'm John.

developer Clayton Hunt John Calloway Twitter US Elizabeth Hendrickson Facebook facebooks Dan Torpedoes Microsoft Phoenix Tasmanian Devil Lucas Martin Fowler business owner Maria Jason Freed Ceo Aurora Lincoln
Season 1, Episode 7: "How do I up-skill my team for digital transformation?" with Thomas Sweet, VP in IT Solutions @ GM Financial

Network Disrupted

26:03 min | 1 year ago

Season 1, Episode 7: "How do I up-skill my team for digital transformation?" with Thomas Sweet, VP in IT Solutions @ GM Financial

"How would you enabled your infrastructure fundamental change over the last five years and partnering a business is critical tools exist on the cloud change at the rate necessary to secure by designing this where this low. Hey, it's Andrew and welcome to network disrupted where I T leaders talk about navigating the disruption in our industry in this episode. We talked with our guests. Tom sweet over a GM Financial about how he's transforming his QA team into part of a unified engineering team and empowering his team to stay along for the ride. Tom is a VP and IT solutions at GM Financial and he's got a real focus on lifting people up in the workplace and to give credit to where credit is due. It also like to thank our friends at Linux Academy or rather a cloud Guru for introducing us Tom gets into the Tactical weeds explaining how he's doing what he's doing which is really valuable Kristoff. Thanks for that. And before we get into the show, I've been asked to say Tom's opinions are his own and do not reflect those of GM or dead. Financial let me know what you thought of this episode. You can tweet me at Network disrupted leave a review on Spotify or apple podcasts or email me at Andrew at Network disrupted, So Tom tell me a fact about yourself. I know you're GM Financial and you're very much involved in in the transformation of of Technology, but I'd love to hear it from you sure. I started off an it back in nineteen ninety seven prior to that. I worked in civil engineering which is traditional engineering roads and bridges worked on that big dig in Boston. I had an opportunity to go to any seed which at the time was NEC computer systems Division. I made his career jump because I just was excited about it back. Then I was really trying to get what you'd call a tester up today, but the recruiting agency recommended I get a key drop over at NEC testing laptops and the windows ninety-eight and n t for free installs right wage. Okay, I'll try that and that was an awesome job. You go back and say one of the David the best I've ever had. It was just a lot of fun to get to go to Japan and I've been in kind of the software divorce topic, uh space ever since I had an opportunity to go to Microsoft for a couple of years work on Office 2010 and SQL Server Kilimanjaro went to work in another country club Tech which was a software for mineral mining. So think of AutoCAD but for geology, right and then in the wild up at GM Financial, they needed someone to come in and take over the QA team and help reduce the growth of the q18 upscale then two more of a software development skill set and that was in 2016. And that's where I've been ever since we had a big project that was in place for about a year and half once that was done then we really started pushing this transformation which will get into and it's been a lot of fun job. And I've had a lot of personal reward from it because I really like investing in people growing them and and helping them become better. That's really one of my passions wage. That's fantastic and and and and transformation is is always fun at least from my perspective and and nowadays, you know, so critical as new skills required to deal with with new computer. And and I think that's that's around practice and and process and and skill sets. And you know, when when I look at it from my perspective, especially on the infrastructure side, I mean, everything is becoming software and software driven and and the quality aspect of that has has changed has changed quite significantly off, but you can give me some sense from from your perspective. I'm how how quality and and the process and practices around quality have have changed over the last few years. Sure. So I'm home. Big fan of jazz humble. I don't know if you know him but he's is active in a devops space and he says if you want to improve your quality, you have to go faster and that may seem counterintuitive. But in order for you to go faster, you must have automation. You must have really good processes in place. And so as we continue to shift the ship left that means more and more automation that's better pipelines for our devops tooling. We want to make sure we have a template or the pipeline of pipelines that everyone can Implement and make sure that we're all in alignment with the different scanning and security fans or sonarqube or check marks or whatever tool that we have in our pipeline, but it's it's shifting left and it's it's off his concept from a finding defects to reduce the introduction at defects. Yeah. I think that's critical and I think a lot of the processes and Tools around that, you know, the, you know build stuff that wage. Works and and that works from the quality perspective of you know, the functional side, but also, of course the non-functional side and security-related requirements and scalability and performance and and and and driving off continuously, right? Yeah, and that's what we really want to get too. And and you know, obviously that leaves a gap in in skills in some cases and I know obviously, you know as you stated before as a huge passion of your so I'm curious how you you know, what what what's your approach to to that skill-set Gap and and how long you working with your teams to ensure that they have the skills they need so they can help move everything left. Sure. We are doing an awful lot around that when I first started here I had about a hundred and Thursday if I team members and five were titled automation engineer and the rest were some sort of QA analyst and when we looked today with total headcount like 1:17 in yep. Be software developers is 63 analysts. So we're reducing the king size because we're getting more efficient and as maybe people who different rules are not necessarily backfilling them, but we're hiring software developers in place of queuing analyst security analyst is a legacy roll this end of life and we're transitioning queuing analyst to software developers were also hiring software developers off the street a lot of companies. You know, when I worked at Microsoft, I was called software development sneering test and I worked alongside the the platform developers working at Office 2010. Will Microsoft has since eliminated the pest control and everyone else towards unified engineering. Well, that's what we're going to do here. So either hiring software developers off the street who have the skills. I need and I willing to work on a QA team until it Blends into this unified engineering model, which were moving to I'm also hiring software developers out of college and they're excited. You know off University of Texas at Arlington is closed you TV, which is Dallas and we're hiring, you know graduates to come work with us. And then we're also training the QA analyst to become software developers and you promoted about 12 right pretty intense training program. That's internal part of what we do is I provide an hour a day for my team to Thursday, which is either used for Innovation. It can be used for Learning and I asked for this from the T I go because it's important that we reinvest wage to create this culture continuous learning and you know some companies, you know, give this time of the companies don't but we we feel it's important to to reinvest in allow team members to get birth. So that's fantastic. Yeah, so is it is do they is there they need to propose how they want to use that time or or is it really just left of them to to to pursue what they think might be interesting? Well, it's mostly free-form, but they have to upscale. So for the QA analyst specifically they need to learn that coding language such as Java and c-sharp, we're standardizing and c-sharp and potentially no j s They have to end they going to have assessment. So one thing I was doing with the QA analyst was I was going to have them do either an official Oracle certification or a Microsoft seat shops or you know provided by the vendor. And in a reason for that was as a third-party the is the perception that there's no bias in the exam because Microsoft and Oracle give this exam to all these different people around the world. Where is when I started looking in-depth at those two exams. There was a lot there were a lot of peculiarities or Oddities of the language. I didn't really find appropriate team. So we're creating the assessments for them that they they have to meet and we just had one at the end of February and we have another one coming up the end of April which has no concept such as audio problems going to design the year inheritance encapsulation polymorphism those Concepts and we're going to quiz them on that the end of April. And help you help them understand where they're at. And where they need to go for the you know, the software developers who are already in that role there. They have requirements for certification. So they have to do to Microsoft certifications this year and a sure so they can use that time to get better in Azure for those through the daisy 400 or the AC 203 certification with a gramming or we're also moving towards doctor and kubernetes so they can use that time towards advancing their knowledge of document kubernetes. So is there any sense that team, you know, you know for instance sometimes so this is happening a lot across many many different types of Tegan's and you know to speak to a lot of people in networking who have as individuals have a great deal of skill and what they're doing and it's not like people who were the traditional QA analyst role didn't have a great deal of skill and what they were doing and sometimes there's resistance and there's religious. Since not because there's not a desire to change maybe the resistance is in. I mean these these individuals have accumulated a great deal of skill and what they do they were intent wasn't to be a software developer, you know, maybe there's fear of change or fear of going from an expert in something to being a beginner in something else. Do you do you mentor and talk to your team about that? Did they reflect back to you with any of of those sorts of individual concerns? Oh sure. I mean that's an ongoing process when you look at a few analysts that that I have some of them come from QA organizations outside of the company others came from different business units and they they showed aptitude and desire and not move it to i, t as more of like a user acceptance testing type approach. I'm taking the team members and helping them completely transform. That's scary for someone and we use it at car approaches like awareness desire and knowledge ability and then repetition off. So my job as a leader is to provide any awareness. They have to provide the desire themselves. But what I do is I also see it with them a lot of what's going on in the DFW area. There's a lot of high-tech down here, especially North Dallas and A lot of these companies I'm not going to list them. But they've either Outsource their QA to India where they've gone to more of a unified engineering approach and they release their q18s and and laid off what I'm telling them is it's we are investing in you we are providing you with the future and this is what the future looks like and you can get on a bus or you can find another bus that goes someplace else but this process moving more towards unified engineering and you're part of that we have a seat for you on this bus we paid for your ticket. We just need you to get on the bus and and for the most part. Both of the team members are excited about that now everyone at a different pace, but some team members have decided. Hey, you know what? I really want to maybe move to a role in the business and that's fine. Right, We'll we'll give them a reference and and help them find a good path for them. Right it is hard and it can be programming can be overwhelming and now I get to them not only programming but we're talking cloud. We're talking command prompt and networking and and a number of different. Different parts of this job. I mean the job now is is a software development is devops. It's automated testing. It's networking it Cloud. If I did not say that there's all these different pieces to it. It's not just one thing and it's not only one language Java and c-sharp. It snowed is angular and they going to have to be able to pick up these different languages. And now we're talking about Go Google Go Lang someone who knows C sharp. Well, he or she just may have to learn go in the next couple of months to maybe work on a different project. So it's having this culture continuous lack of being excited about it. And and what we try to tell the team and we got this kind of a joke. It's like keep up with the cloud passions about the Kardashians cuz we got a we had this meeting like a staff meeting at home twenty questions that we had cloud-based questions that we had Kardashian questions and like everyone got the Kardashian questions, right? But not everyone got the cloud questions. Right and it I can appreciate if someone knows the name of Kim Kardashian small child, but I also need to know what platform is a services and how that differs than infrastructure-as-a-service ya know for sure and wage, you know, but I think I just just highlight a couple of things you said, you know, I think that for one and and I'm actually maybe just turns into more of a question. So often times when there is resistance to change when people start seeing the effect of the change and so that might be in their own skill sets, but it might also be in your ability as an organization to release things faster dog. Or you know to to you know, the the massive reduction of Escape defects cuz you had moved things left or so. What sort of what sort of metrics do you look at? And do you use those metrics to sort of create that enthusiasm and drive success? Sure. So we do a pretty big Matrix program that we're and I always say we're starting it. We always have always used metrics but we've used more of the traditional activity or task-based metrics what we're trying to move to is more into in values to metrics and in one mistake that that I guess I'm responsible for is I kept calling him that box metrics and I should have rebranded them internally as before value delivery metrics where it's your how often how long it takes you to go from checked in code until it gets a customer your frequency of delivery You're a Mean Time to resolve and your frequency of change failure. So the poor metrics that the cold war is going to happen devastated that popped off. We're trying to really move towards into end metrics because it doesn't matter. You know how quickly one person does things it is how quickly they move through the value stream in for example in the old waterfall World. It might look really good. If a development team dropped their time from four weeks to three but they've added six weeks to the QA team and then two more weeks to the monitoring team. Are you really saving anything? There's no but if you would even if but if you were to say hey, look at them being spent one more week and we can save six weeks down Downstream. That's much better. But unless you're really looking at the whole value screen. You don't always see those efficiency gains. I know for sure and I really like, you know using that term value and really driving this as value and and unfamiliar with those metrics and and and appreciate them quite a lot. It's our ability to release. It's our ability to get stuff to customers and and and you know, the faster we can do that the the better assuming we're meeting those quad. Requirements and and you know, so it's all it's all interwoven. But so so are there are there, you know, you you set off over time. You said Targets on these metrics. Is there a a drive to you know release in a week or I mean and I know there's many different platforms. You probably work with that have their own historical construction and other concerns and compliance concerns about changing but but often times or do you do you do you rally the team towards metrics or it's it's more of a month, you know over over-arching theme. It's an overarching theme because we're still trying to find the right way to collect them because we'll use one tool for my blinds in our cash management and such we also have What is that? It's we also have service now for our change management process right in trying to make sure that we have all these these metrics and we also have different schedules for the project intake process. So right now we're trying to find a way to actually collect all this data and make sure that data is correct. And then we can start really getting metrics towards that are working metrics and then work to improve them. And as we move more towards, you know, a product 18 model or something like Toyota calls it a factory team where you have everyone on the team has all the skills necessary to deliver it a product. It'll be a lot easier to focus on, you know for sure to measure now in in absolutely and and you know, I've I have my own share of mistakes in the page asked where where you know the old adage be careful what you measure because you if you don't necessarily know that driving that metric there is going to yield off. Overall quality or or or driving that metric towards that goal could potentially cause problems somewhere else like the example you gave great. We reduce the amount of time it took to code this boss increased QA. So who cares, you know, the net result isn't there and and so I think metrics are are measuring and metrics are very important. We always need to think about Iraq whether or not they they're you know, we should be challenging their their usefulness or maybe better said making sure we're looking at the the whole system, you know, the Gestalt of like this metric might be getting better. But what's happening in general? Yeah, you know and getting to that. Another point is I like it'll Martin Fowler. She's a really good program a much better than I'd ever be he's also won a signed up at the ad y'all made a fesco. He is a blog post about measuring programmer productivity and he talks like a really hard to measure programmer productivity. You can't look at my notes can't look at Dead. X cuz you could have met right so if you can read this post it's not that long but he makes a lot of sense. And so when people say how do you measure with individuals it's hard, right? You can look at rework you can look at attention to detail, but it it's hard to compare one person to another objectively only through metrics. So, you know, there's always Improvement right? No Mark Martin Fowler's great and I appreciate his blog and also his books. I mean the guy I'll miss quote this but like, you know page one line one, or maybe it's a forward introduction of of his refactoring book is you know, like if you can't test the system close this book and put it back on the Shelf, you know, and he's been a proponent of birth of you know, driving driving everything left for as far as I can remember in my career and and and certainly measuring productivity of an individual developer, by the way. Historical like, you know lines of code or something is is is in many ways nonsensical you have to look at, you know, you have to look at broader things. However, I'm not a fan of throwing out all metrics things need to be things need to be measurable for sure. What about different? So so now great. I was a QA analyst and or software developer and so I'm learning either new languages or new practices because now I'm driving, uh, you know, I'm testing with code. I'm building software and building software thoughts along with everything else. But there's also other skills that are necessary, right you mentioned earlier security and both in terms of writing secure code, which is important in that test back as well. But also, you know in ensuring that you're the software itself is meeting security requirements, but also with infrastructures a code and cloud in general there's everything from networking to in all aspects of networking, whether it's it's stuff near and dear to my heart like DNS or load balancing or application load balancing. I mean, there's these other Stacks that have, you know, great deal of you know, knowledge sometimes esoteric sometimes otherwise necessary to understand. So how do you infuse those different knowledge steps in your team's do cross-functional teams or you upskilling in non-programming areas as well? Yeah, exactly. So I have We pushed I started pushing the certifications and cloud back in 2018. And at that point it was a w s and we had about 10 then in 2019 by April 2019. It was a flat. No one had been certified in anything and I was talking my budget analyst and saying yeah, we're going to have to give back some money. We're not using it. Okay, so I came up with this idea for a contest to send people to Microsoft ignite wage, which was later in the year. And then I had to kind of guess like when I be able to buy tickets, which I guess a little bit too late, but I guess by August I can still get tickets for the November conference and I came up with this approach to weigh the different Microsoft Azure certifications based on my perception of how hard they were and give people rapping tickets based on how they starts they got and I opened it up to not only fifteen but the architecture team and some other development groups in the org and I was going to pay for the whole thing. So we wound up scented for people to Microsoft ignite. from the contest and that really drove 60 Cloud search and following that in October the cloud Center of Excellence needed some help because they were creating a system for our public Cloud which had, you know help spokesmodel had A lot of work infrastructures code the networking all those pieces and they say can I borrow a couple of people for two weeks? I'm like, okay, you can do that and two weeks later. They asked for two more people and like sure we can do that. And here it is in March and I have six people that been down in the cloud Center of Excellence since October or so, and they're acting up two of them are setting up the new firewalls. We just want You did something maybe a year ago. They couldn't no one would have considered any you know from one from my team capable of setting up firewalls, but is three people working on the cloud on it? And and two of them fell apart from my team and in one of them is the cloud architect, but the three of them are the ones that setting up firewalls and they've done all the the infrastructures code and terraformed and they're really squishy gone inside this and they've learned an incredible a lot. So even though that the certifications good and help provide them the space education, but the on the job training that they're getting down in that group is amazing and they probably going to want look at five months is a long time. They probably not coming back cuz it's not kidding myself right down the center of excellence until further notice and and that's been really good job. That's actually right back. Yeah for sure, but that's actually a sign of success right? I mean you're you're you're working with the team is everything I've been trying to do a hundred percent and like wage. You know at at my company, we have a support team that is that pushes hard in training in skills and they end up being like it opens up opportunities across the company for them. And so now you see them in Professional Services or sales engineering or the product development team and and and you know, it's a it's a shame when somebody has great at what they do isn't there anymore but it's way better if they're inside the company in a different position that that is exciting to them and also helpful for the company. So I think it's I think it's a great sign of success if off if the individuals are creating value outside of your team. Yeah, I mean and flip at the other side what if they didn't want them write what they said? Yeah take them back. We don't want them here. That's not the case, right? They want to write them. So that that's good. Yeah. Yeah. No that that's fantastic. All right. It was really good talking to Tom. Thank you. Thank you for listening. I'd love to know what you thought of this episode. And I'm all ears. If you have that guest recommendation. You can tweet a network disrupted leave a review on Spotify or apple podcasts or email me at Andrew and network disrupted. Com.

Microsoft QA analyst GM Financial Tom sweet analyst Andrew Spotify software developer Japan Boston NEC apple GM David Kim Kardashian Martin Fowler
Learning to Unlearn  Barry OReilly on The Product Experience

The Product Experience

37:37 min | 1 year ago

Learning to Unlearn Barry OReilly on The Product Experience

"Okay so there's hippos unseparated as seagulls. What are you talking about Guli? Some of my favorite product related Thames head pays the highest paid person's opinion and Zebra zero evidence but radiant. Oh you mean zebras okay. Yeah and their singles Those are the exacts that do the swoop poop just do one more thing for me please and I learned a new term from today's guests the reporting watermelon. Yes we're getting. Tell people what that one meets got doc. No I'm not telling. They need to listen. Well that's not a terror because we have a wonderful fiery a riley on this week. He has more jobs than I can shavkat. Just a cat like Autho consultion yet professor and adviser and even podcast host but he makes it very good guest as well. He came on this. We talk about the philosophy behind his book on learn and let's jump right into the the product. Experience is part of the mind the product network every week we talked to the best product people from around the globe about how we improve practice aside from conferences in London sovereign's escape Singapore Hamburg Manchester. That's also free product tanks in more than one hundred and eighty five cities. And there's probably one now you find out about them on the product dot com where you can also catch. Antropov past episodes videos from the conferences. Read great articles and learn about the training that we do Barry. Thank you so much joining us on the product experience podcast. It's weighty great to have you here thanks for having me so be really good if if you could give us a really quick kind of entry into your history in product and what your teeth. These days My name's Barry Riley. I'M A business advisor. Adviser Entrepreneur and author at primarily were around the intersection of business model innovation product development organizational design in culture Transformation in my advisory work a lot of my clients are executives leadership teams from Fortune five hundred companies or scaling up startups Arnold's here in San Francisco another areas. I've written two books if I won was lean enterprise at high-performance organizations Scale that was part of Eric Reece's lean series and most. Recently I wrote another book called on learn NETCO has success to achieve extraordinary results. And I'm also Faculty Singularity University here in San Francisco so I work a lot on there executive invoke accelerator programs and then I acid on the board of crafts with reaching the just acquired by Atlassian and just three things which is a start of based in under focused on organizational alignment Nice. I actually the just three things team. That's burnden awesome company. Check them out. Full End. We we're GONNA talk about your book on land so you know we often As product people talk about failed measure in Lan- where has Alan come from one of the interesting things when we goat lean enterprise. Not People had read being startled and were sort of like we need to experiment. But we're not a storrow and lean. Enterprise was really much more focused around when you're a larger organization under the practices that you need to do to innovate from her. Folio management to organizational organizational design to growing the culture that you aim for through a experimenting to find out what works well for you and you know it was amazing opportunity. Yeah I got to work. With some of the most talented people ever met in my entire life and Moment for me was while teaching these people to learn. New things was tough while what was even harder was to get them to under their existing behavior especially if it had made made them successful in the past so this was a real sort of Aha moment for me is that yes we all need to learn but it's not just about learning we also have to recognize what are thinking or behavior has become outdated or obsolete and we need to let go are to adopt and that's really bird on learning so it came from me so what what are the principles of unloading. How do you go about teaching? This will the way to think about it is a Just like products have features and they have to continuously innovate features to stay relevant in your Marcus at people have behaviors figures. So if you're not continuously adapting your behavior changing market technology situations or customer. Needs the the argument I always say is that it's not organizations. That get disrupted. The truth is it's really individuals. Individuals are the people who need organizations so really what people need to recognize is that they have to understand that at some times their existing behavior thinking could be limiting items uh-huh arranging for and desist directly nuys. When do I need to learn? How can I start to learn and experiment? The new behaviors to try and get the breakthroughs that I'm seeking. And he coached is more and more realized it was a system at the virtuous system the more people could recognize when to unlearn and then start to experiment with new behaviors and relearn to get breakthroughs in their performance or the behavior thinking at this Extraordinary results and you know pleasure. Working leadership teams from International Airlines group Beverage Airways NASA Circa McDonald's accountable and there's so many of these amazing organization so they've got a chance to work with and share examples stories in the book about how these leaders recognized when they need to learn on. The benefits system is helpful SABERI. What are the signs that you're using unlearn? Well rather than just getting lucky. It's very easy to get lucky sometimes but to attribute that to do the right thing so really interesting for me. A lot of people feel like understanding when you need to stop is serendipitous or it's intuitive to other people Started to recognize is that there's a system that you can learn and therefore make it intentional deliberate for other people so there's no way I would frame to you is how do you recognize you need to unlearn. So I'd I'd ask your both yourselves and maybe listeners. To sort of grab post it notes and see what answer they come up to to these series of questions because when people ask me I know I need to unlearn. He'll stay. I want you to think of a situation our challenge that you may be facing right now. Maybe there's somewhere you're not living up to the expectations faces. You have for yourself or you're not achieving the outcomes that you desire or maybe you're sort of struggling to resolve a certain situation or for a challenge that you're just totally avoiding or maybe you've tried everything you can think of on. You're still not getting the results you want so when you think about answers to those questions not having up to the expectations you have for yourself. Not achieve the outcomes you want situation. You're struggling to evolve to resolve her order avoiding or trying to think of something pop into your mind definitely does for me. Yeah absolutely and I struggle to find. Find somebody who didn't have a a reaction to to this especially in a world where so many of us have imposter syndrome and yet okay well so imposter syndrome impede an interesting challenge right like people start losses like handle. I present my struggle to present my ideas or feel uncomfortable at AH putting my opinions out there or you know D- Dis- shortage. Everybody has problems you know and the whole point is if our existing behaviors working. We wouldn't have problems because we be achieving all the Want you know what often happens. Insist that people sorta gets stuck. You know they they stick to the sense of behaviors that they know would understand or feel comfortable to them and and when those behaviors on driving the results of the outcomes that the aiming for often people's natural reaction is to point to other people. You know we would beat successful with the product launch. The engineering team has delivered on time or we would have had a greater success with that feature if we were Arabia to deploy us at ten times a day rather than once a month you know and this is sort of one of the challenges is if you're serious about making progress you've really gotta own the results you've gotta say what. What can I do? Different needs to change the comes getting and that often means it. You'RE GONNA have to get comfortable with being uncomfortable or trying things that don't feel obvious to you. And this is why the big steps talk a lot of those. When you're trying to earn is you've got to think big about the aspirational right from that? You're trying to achieve. But you start small as you start to read her and starting small crate safety. It makes it safe to fail. You get fast feedback loop you learn quickly but also you feel successful that you're trying Ryan newspapers to try and drive the that you want and this is sort of one of the patterns of thinking big starting small learning fast to try and get the brake choosing. You're looking for in your performance. I can see doing that for myself on an individual basis at I can see how somebody in a senior executive eh role can do that. Ankara safety for the large parts of the organization but so many of the product people I know where in essentially middle management roles roles and we can try and create safety locally. But there's this wider environment that's going on that's b. not be especially safe that not give us the permission our mission to do this. How do we use this? When we're famously? People who have all the responsibility even potentially none of the power so I think the most important wouldn't thing anybody can do is be responsible for transforming themselves in luckily work in these big companies. Everyone's always always worried that. What impact can I have just as an individual inside this huge organization under so many different competing priorities people pushing in different directions? Like how can I change and a simple thing is the thing you can do. The most is role model behavior. That you believe is going to be most successful in the organization because you can affect us and people might feel like that's small it actually can have a massive ripple effect across the organization can give. You can't just examples of this so in one of the groups. I work with really senior leadership team in an organization and they were trying to get more effective with their time writing. I ask the question you know where. Where do you spend your time so for? Most people in at product grows reporting into spend most of their time time in meetings. I was GonNa say this event and on a scale of one to ten. How effective do you think those meetings are? Oh will maybe a five. You're lucky right right if you're lucky This is like a fifty percent success rate for an implementation. If you're the product manager and I was only getting like fifty percent satisfaction rating with with something I want to change it. I'd be asking myself. How can I make doc better? I can make it a nine to make it a ten if we're really serious about improving And this is one of the challenges that lots of people face and yet most people just blindly walk into meetings and conduct them the same way every single time. They don't make any changes they they just go. That's the way it's always being so one of the things I have a right little. Learning statements were again leadership teams to describe a better outcome of you now. I want to be more effective. I spend a lot of times in means they are totally ineffective. So what's one small thing I could do in every meeting to try and make it batter and and we sat down on a district leader. We came up with this simple step so five minutes before meeting with end Vida would stop. Stop this team in the room. How effective do we think this meeting was? What are you taking away from this meeting? And then they'd sit Erin and basically shut up and go round the room and hear what people have to say anything. Sir sessions initially went. Why the people honest they absolutely This is sort of the thing starting to build it up because when you start small in a small meeting with a a small group of people and people start to realize that there's lots of different assumptions might take away was that we're going to implement the dray someone else's takeaway was actually. I felt we to implement Feature be someone else's I I don't think we made a decision on it like these questions never get asked sometimes sometimes a lot of people. Just leave meetings rushing from one thing to another you learn a huge amount about what's working and by just asking these simple questions like. How could we make this better next time? Eastern small little changes compound compound compound over time at the other interesting thing is when the leader starts to demonstrate new behavior of meetings pausing five minutes before the end and asking people have effective. Was the meeting. What you think starts to happen and grievement? Hopefully also they repeated the other people in the meeting take it back to their own teams and it spreads right and so so this is sort of like the network effect about when you start to role model these behaviors in your organization. You cannot have a systemic thing because his people see you're trying new things to improve that inspires them to say. Well if I want to be serious the better proving I need to start asking these questions about how effective affected my spending my time my investment. What's working what's not working hand? We make better do. We even need to have these meetings all these things start to. Oh below and so when people say that they can't influence for me the DOT needs to be on alert. You know you can have a huge impact or just role modeling behavior. You're asking other people to do and it can have a transformational impact across organizations. How how much of what you've experienced with the unleavened philosophy has been around behavior is just looking at price? So Ah frameworks that people are using and things like that. So like the core of on learning I describe it as the process of of letting go or reframing gene or moving away from once useful mindsets in acquired behaviors that where effective in the past. But now limit your success so it's not forgetting a removing living or discarding knowledge or experience. It's just a conscious of letting go outdated information and actively engaging in taking a new information to inform decision making an action so this is not you have to think of this like your knowledge is like a ram the computer at some points. You need to load certain in parts of this when you're performing certain tasks and other stages you need to take that out and replace it with something else. I think what you see the stupor high-performance leaders they recognize that when they're at an inflection point or win the behaviors are using are not working. They will change wjr. Am You mentioned this idea. Ruth Eric's ill measure learn cycle by decided experimentation with product. So when your product is not driving the customer customer comes at your aiming for you start to try and tweak it you try and change the product so it moves in the direction that you're aiming for and similarly with him a process you're trying to find a great way to onboard people into your organization and lots of people are joining. They still feel lost or not not working in well. It's been thinking about all of these things with product thinking. You know there's an outcome that you're aiming for. There's many options that you have to drive that. Oh come on really what you want. System to rapidly experiment with those options to see which ones are driving the outcomes that you want an amusing those options Shin's rather than sort of just seeing this is only a product thing or it's only a culture thing I think we really have to get better at recognizing at all of this is just a good experimentation and if you could read bed hey find success move with options to try and drive up success correct when things are moving in the right direction or nothing adopt. That's eighty degrees of what we're trying to do. In all aspects of her work people in organizations organizations are famously resistant to change. And if you don't see a reason to change your you're not really going to And a lot of people especially the the higher you go up in organizations they have a biased towards. Well what. I've been doing his work to get me here so far. What's the best time to introduce this to to start changing genesis? This one of those things tough never waste a good crisis and start modeling and introducing behavior then or can you do it more gradually before when when people don't even recognized recognized that there's a problem yet so being a senior executive in large companies really difficult that the quality of information you guess is often very very poor and yet most of the feedback mechanisms. You have in your organization is often telling you that your existing existing behavior is working. So you're getting promoted. You know you reports to people are sending to the organization all look like a beautiful green dashboard. And everything's going great in the company's growing. Why would I change ev own? My feedback mechanisms are telling me I'm doing the right thing you know. And this is a huge challenge when I start working with companies I joke and call it eating when I when I go in and ask to see their reports portfolio dash or it's to see a sea of the green Projects going well. I'm like this is a watermelon. I did screen on the outside. But we know it's red in the middle and the thing is if you're giving executives poor information and and they're making good decisions with bad information you're not gonNA get the results you want. Unnoticed creates a painful situation for everybody. So it's a two way street. Here of teams actually been sharing the correct information Asian leaders responding to that in a way that encourages able to keep sharing the correct information. See you're making good decisions based on real data and you've a higher chance of success so one point a really like all of this. I sort of have empathy for leadership but also recognize their parsh- in bringing correct information to people to make better decisions with and in terms of like the incentives of trying to get people to change their behavior. This is one of the reasons sway thinking big and starting small is so important like I often say like the worst time to try and change is when you're in a crisis because you so you have a do or die type scenario and you've never practiced had to adopt your behavior in if you think about a great athletes are Our great leaders is there constantly building muscle in practicing ways Abedin. They can experiment with the way. The company works in the way that they behave. It's so many of my favorite eaters. I work with are just great experiment. There's and I'm very curious. So one example is like the CEO of of HSBC. What it used to do was when the graduates came into the company every year he would go and sit with them and say to them? Here's some of the challenges. I'm working on. How would you try to solve these? And he would sit there and watch graduates would try and use different tools new techniques new technologies to try and talk like these is reading difficult problems that he's facing and that gave him a real great chance to learn and what's new on learned some of the existing notions he had it also. Has Its amazing coke zero reference for the company so you can imagine these big bureaucratic organizations. Some of the most senior people are willing to sit Dan. The most junior people and ask them had a solved problems. They're working on it. Transcends is huge sort of cultural artifact for the company Super Super Powerful all the best leaders I've worked with are the ones who aren't afraid to be vulnerable to who are able to put themselves out there and say I know I need to continuously adopt. I I'm not going to believe I have it all figured out. I'm going to need your help to get there. And another example sample might be a sewing work international airlines group so they own British Airways Iberia inviting early The sixth largest airline in the world. Well now they recognize that their approach to innovation especially for leadership innovation was struggling. They're using the classic send people in a two day training course. I expect him to come back with massive behavior. Change right so evil being on a two day training course in massively change your behavior right. Of course I ah on yet. This is what we would certification programs in our industry right like even the way we teach people had learned needs to be John learned so you know they recognize that that's not an effective method to drive the outcomes that they want but they were bold enough to say. Will we need to do something different rent. So I have a business called exact camp where I get executives to leave their business with the goal of launching new businesses disrupt the existing organization and as a part of the process essentially disrupt themselves so we took six of the most senior executive team Advi for eight weeks with the goal L. to try and launch six new ideas to disrupt the airline industry on themselves and when I say to people like you can get like the CTO British Airways to leave the company Buzney for eight weeks. That's insane but I know what why applauded at these organizations is they know insane is doing the same thing over and over and expecting expecting a different result like putting into day training courses in expecting. It's going to change their behavior so it off. You're pushing against all these. The existing norms of society. Says you're wrong. You should do it that way you know. So this is where I applaud like true sort of owning the problem. I'm getting uncomfortable about is and you know like that program was amazing. We launched the first ever blockchain identity management system for the airline we built the processing platforms. Could turn their customer data requests incompetence seconds in but we also came up with more ideas as we couldn't work on and that inspired British Airways to say I'd only open up all of our eyes and make them available for starters start building products and services. This is on top of our assets. Never ways that we can't even imagine. And that's the first venture capital firm in the airline industry called Hangar fifty. Do you want so now. An international airlines group is around three or four versions of this program for they have startups using Never Djing all their assets in ways that they could never have imagined ill new products and services broken up this whole new ecosystem of innovation in startups that they work with as well as delivering innovative programs programs internally because the leaders who've been tired of that program I've gone back into the company and their coaching other people had unlearn had to do product Development Linden innovation better. And if you have to look at their results over the last two years have they started to turn a process now really starting to power ahead relative to the other airline companies. I think a lot of that is just. They were bold enough brave enough to sort of take different path when when everyone said don't do do that. That's the power of of these sorts of things when companies really commit to getting comfortable with being uncomfortable but making On that point of kind of Bang bold and brave to take different paths. Do you think that this way of working all of unloading Planning do you think this has to come from the leadership team within a company because it might be the the behavior change that you're I'm trying to emulate or encourage will do yourself is then seen as insane unpredictable. I guess in an organization which is like no. This is the way we do it. And this is the way we waste on. It was always interesting to me. Am I remember when I joined. Thought it works. Maybe ten years ago and Martin Fowler. Who is one of the leaders? Najia community was That's inspired me to want to work there. And I remember I am meeting Martin and when the first thing he said was you know you got to change your company or you got to change your company and I loved us it because you know leadership obviously had a bias effect in organizations right. They're they're the most senior people in the company. They have the most influence on the whole system of the company so by definition when they start to change their behavior they start to be working the different way role model new behaviors that creates agency for more people to really go fresh in Working with them. are really well known banking company in London. They were going to transformation and one of the things that was most interesting for me is everybody you will come up to me and say you know what the most interesting part of this is at the. CEO's name is Chris. Chris is the person who's actually going for the most. He's trying to be the most outlandish to get there. And that was a hypocrite this huge agency and for them which I thought was really really interesting now now what does that mean for people who aren't CEO in the company will look. You've got to think big about to change you seeing the company and you're gonNA start small smell it. It's GonNa be you trying to show these new behaviors but by starting small again. It allows you to try lots of little things to see what works in the context that you're in and makes it safe to fail. You can try different thing. At the end of the beating asked people How effective this meeting? Yes or no maybe asked people maybe on the next user story. I don't try and write it as a hypothesis statement rather than a user story so maybe we can test it better you know maybe instead of at the end the iteration in showcasing it off to our stakeholder I don't mean invite some customers into showcase its customers at the end of the week these are all small tiny little changes. The what's the worst can happen. Somebody Says No. And if these things start to work than people see the value of these tactics and it starts wants to buy the mid to say yeah we maybe we should get customers into the office more often. You'll maybe maybe we should have because Moore signed the product off instead of waiting so manager is your sign it off it. They're very very powerful statement. So I guess the thing I would always ask listeners to take heart is you have way more influence than you realize. It's just do you WanNa take to sort of a uncomfortable step of trying things and you note by definition the more you try the more things are going to fail and some things will work and that's why starting small and iterating quickly just like you do. New Product Development by testing new features with prototypes or bringing them into the market with a small sub section of your customers are in Ktar strategies. The product managers know had managed risk. When you're trying to experiment and you can apply not your products to yourself into the processes in your in your company? So I don't think furious should embrace this on any should be coaching other people reduce and is that is that as time where you've seen too much change kind of going to fall down this rate of experimenting Do too many many things at once. is sweet spot of you try one new thing a week or something like that. So a key. Principles of the League thinking Inspired Eric's built answered learn. Is that you work. In small batches and the limit work in progress. This is a real important part of experimentation. That people probably get wrong if the hair experimentation. They're like let's go change fifty things all at the same time and then something breaks. Heidi Carly's are caused break. So this idea of like making limiting the amount experiments or changes that you have in progress at any one time being as you're turning the dials you can sorta see the effect of that change. And if you have too not much change happening. It's very difficult to correlate what led to success allotment in the direction. You didn't go or unintended consequences. So as you're saying your example of maybe if you're GonNa Start Yeah Start Small. It's just change one thing at a week. It's you know. Sit Down as a team go. What's the one thing if we if we changed this week that we think would have an impact? Let's do it for a week at see how we go underneath evaluate. Try the next thing. And and you know what I've learned is when teams ill dismissal and with a to be able to continuously adopts that makes him undefeatable because they're they're able to adopt to changing circumstances which will always happen. Would you technology innovations new customer needs just to changes in the markets that we operate with him and really the best skill to have is some sort of adaptability quote rather than just the great one at behavior one set skills are fix yourself really should be able to adopt an announced the most powerful thing when you're in a situation of high uncertainty so berry. This has been absolutely fantastic but I have to ask. was there something that happened to you at the beginning of this process. That made you say what I'm doing isn't working king and I need to change my approach. That will lead to the unlearn breakthrough for you. What I have what I call learning moments all the time you know Alan and the way I I identify the missed I've got good at describing black success? Or what are the outcomes really. Aiming for simple example was news and you know when I was riding on learn itself in my typical notion about how you had to write a book. Is You have to sit at a desk and you have to have have a roaring fire and a Purple Velvet Jacket on typing keys. Being this amazing at author person you know and and nothing could. It'd be further from the truth. Like I was sitting there staring at a screen for days in and getting like three sentences. You know so I was sort of like cotton Indus notion is that why can't say right. What's wrong with me? Maybe I'm not a right or maybe I suck. Maybe I'll never be able to write a book and I started selling reflecting a bit to say. Well what's what's the outcome here. I'm aiming for an and reading. What I was trying to do is create content? There's many ways to create content intensely type. You can talk. You can offer these things started to come into my mind realized that actually my better her way for me to create content was actually talk. So what I started to do is create like mini little at keynotes of each chapter. And what I would do is I would just I turn on the audio recording and I would just record myself talking throughout the chapter would be and then I would ship the audio filed to a transcription service. They they'd automate dot incentive acne. Pretty quick thirty and then suddenly I had like minimum viable chapter where it was just just my initial thoughts and I start reading the chapter and and then I said Nina iteration process which you know. So that gave me this. A huge breakthrough If I wanted to write a book I didn't have to type it. I could just talk quickly. Create any at early versions of the chapter. They you can start iterating get feedback from other people on and start to build the product. So you know. I think there's on learning moments happening for me all all the time. And it's just recognizing what are the outcomes that I made for other existing behaviors that are working or not for me. Automated trying to experiment or to get the breakthroughs amazing for I think the first experiment should have been just changing the color of the smoking jacket. I mean who wears there's purple. I really wanted to have a purple Krasnov as is to. I'm never going to picture you with that one. I think the next book. That's a has to be the author photo. I'll make sure it is very fantastic. Thank you so much for everything tell you. This has been a really great conversation. Yeah thanks for having me again. Looking forward to hear people's at various different than learning experiments in just just write Hashtag unlearn Out Find it in Keita's thanks very okay so be inspired but don't go changing young running everything tomorrow. That could be very problematic that randy. What are we going to unlearn on the PODCAST Maybe but how we read her L.. Truce the next week in the product experience is part of the mind. The product network hosts are meat. That's lilies and Randy. Andy Selva Emily. Our producer and Luke Smith is our editor. Emily is ours alone. But we're happy to share luke if you need someone to your own podcast. Hey you can't shut him too much. He's my husband Ozzy. Music is from Hamburg based band POW. That's P. A. U.. Thanks to ONIC HITLER WHO runs product tank on MTV engage in Hamburg and plays Bass in band for letting US use the music and sign up for your local product tank regular meet up over one hundred eighty five cities he's worldwide. There's probably one someone near you and if there's not you can start one yourself to find out more good in mind. The product com slash product tank. Here's global coordinator Ordinary Mar Abraham to tell you more about it. Products is a global community of meet ups in over one hundred fifty five cities across the world driven driven by and for for managers. Whether you have a group discussion or you're listening to speakers whole ideas to create a safe environment for productivity cheaper to come together and share that learnings

senior executive CEO product manager London Alan Product Development Barry Riley Ruth Eric San Francisco Eric Reece Andy Selva Emily Luke Smith randy Hamburg Martin Fowler Guli NETCO Autho consultion Singapore Hamburg Manchester
MongoDB in the Cloud with James Kovacs and Rachelle Palmer

.NET Rocks!

1:01:46 hr | 4 months ago

MongoDB in the Cloud with James Kovacs and Rachelle Palmer

"This portion of dot net rocks is brought to you by couchbase a modern multi cloud edge sequel friendly jason document database for building applications with jill dougherty performance and scale if you're new to couchbase and would like to learn more the couchbase developer portal is the best place to start. It's loaded with tutorials videos documentation as well as best practice tips. Quick start guides and community resources including couchbase developer community forum to get started developing on couchbase. Visit couchbase dot com slash dot net. Rocks that's couchbase. Dot com slash. Dot any t. r. o. c. k. s. Welcome back to dot net rocks. This is carl franklin. And this is richard cable and boy. It's been a while since we recorded a show. It's been a few weeks you gonna give you time off. We do a bunch of shows all at once and you get a couple of weeks off. They do a bunch more. I like it. it's march first right now. I mean right now as you're listening to this but as we recording it's march first than night saw. The crocus is coming up poking on today. Saying he here i am here i am. Don't worry life is coming back. life is coming back. Yeah it is pretty pretty rough winter. No two ways about it is pretty cool. This is going to be a great show but before we get the guests and stuff and before better no framework. I have a little announcement to make. I have put out. What i think is going to be a very popular. Get repo. it's called blazer sliders and there's a new package for it as well. It's just what you think but sliders is sort of you think like a slider that goes back and forth. But i'm talking about. I think about a hamburger. But that's especially blazer sliders right. I mean something. I want to order at chili's or burger king sunday but anyway yeah so so. It's multiple panels horizontal and vertical split panels. Right so you have a sprayer in the middle and you can grab the splitter and move it left and right to the panels in the whole thing re sizes when you re size the browser you can make it. Take up the whole screen. It's blazer it's good you can estim- right now. You can only nest a horizontal inside a vertical. But i'm working on multiple nesting. I basically got it working. And now i put it out there. Good yeah now. i'm refactoring it and making it actually more efficient and less verbose and all. Now you get the me notes from people of course our whole product around your library and it's your fault there's and it's only been a couple of days and thirty download. Some not all the eric about it. Give them a couple more weeks to really get the hate on for you right. So here's my advice. If you're using one point zero point one that's gonna change. So but but the good news is that properties are being taken away not add so the you won't need to do so much manual set up it'll automatically discover who the children are and who the parents are in. It'll automatically figure out how to size itself right now. I do i have. I would require a little bit of setup in the parameters but that's going away so anyway enjoy blazers flyers works on s- a server and blazer wasim and Yeah but that's not my better framework. Oh okay will you better play the music then be yeah you better go now. You know you do it. No you fine. You got all right. Well this is actually pretty cool in the guys in one of the guys in Avonex slack room. Scott rowdy found this so microsoft uses this thing called code q. L. it's a semantic code analysis engine. That's part of get hub right okay. They open sourced it so in the end the ideas that you can use code q wealth to make queries to hunt for Soul solar gate activity so essentially. What you can do is you can have this thing. Where your code base and find the militias bits. Oh interesting yeah yeah. It's pretty cool. So i don't understand a whole lot of it but it is mind blowing that you know you can just imagine just checking your code in and then you know get hub says. Hey this is Or notifies you know malicious code so it's It's really cool. Yeah very interesting. And he's becoming a normal part of the pipeline. These days it seems for at softwares also looking for like accidental uses of somebody else's coat right like you don't know right pace things like don't necessarily know where sources come prime so you can trace analysis across the board just like where is this code from That whole dependency chain. Yeah yeah for sure. It's good. But i thought james would like that and Now over to you richard. What's who's talking to us today. Well considering alassio james twenty thirteen. It's very unfair to read like comments from eight nine years ago. We should anyway. Not that i wouldn't do that. But actually i found a really relevant comment for this show from adult compatriot of james's from jerry miller's from show sixteen fifty five. Just show we. Jeremy met back in october. Twenty nineteen talking about the dot net core ecosystem so that whole the what the open source community looked like there and so we talked about a lot of different things in that space. One of which was talking about non relational databases and crossed crafts comment about a year and a half old now where he says. Hey it's always interesting to hear from rockstars like miller but as a show was concentrated towards no sequel noise right. Some of the statements were converts. Wrote not will put me off a bit. Like when richard said why would it make the customer to decompose objects. Just store the object right. Of course. I think a little. Alex fair alex's i followed that with the customer store the object and then break it out into a relational database. A synchronous -ly right. After the customers already moved on the goes on to say though i understand that adding all the caveats the statements would make the show boring as a dvd worshiper inactive. No sequel developer for years. Here's my outcry. There is a serious cost associated with developing and supporting no sequel database for relational databases to Consider at least document type databases which most people mean when they say no sequel the aggregates designed for the most often operation reading data so persisting aggravates is not meant to be quick sensitive volt dated duplication maintaining reference out of the boundaries and so forth and about performance. Hey sequel is performed to the question is what are you trying to do. There are places where no sequel shines and they're also places where databases do just fine. I do like no sequel in much higher level of usage in the enterprise but modern hazy understanding of those equal obstructs adoption of the technology. You the pragmatic way as just another tool in your tool belt team. No sequel as the default option would remove the perception of it being an untouchable shrine. And get more people using it. Yeah he does reference a couple of blog posts including seven reasons not to use no sequel and the book no sequel to still by martin fowler. Which is a little stale. but it's martin fowler. You should read it. Yeah anyway. I thought it was interesting. You know our apps. We weren't firm enough on this whole in. I think the two things work together really. Well there are places where people may sense appraises where people may sense and most places where both will help you zoom time. Yup so alex. Thank you so much for your comment. A copy musical buys on its way to eun. If you'd like a copy of these go by write a comment on the website at dot net rocks dot com or on facebook were published every show and if you sit there and read on the show. I'll send you a copy music. Oh by and definitely follow us on twitter. I met carl franklin. He's at rich campbell. Send us a tweet and you know. Hurry up because times. Wasting not yeah. It's springtime get out there and play in the crocuses all right. So i'd like to introduce reintroduced james kovacs and introduced for the first time rachelle palmer so a long time ago in a galaxy far far away. James kovacs was a frequent guest on dot net rocks. He spoke at conferences wrote magazine articles and had an active twitter account. Then a career change landed him at mongo. Db where he has spent the last five years in self inflicted obscurity. I liked that. Self inflicted obscurity during that time he has worked on the technical support team assisting customers with a core database product and more recently on the driver's team building the language idiomatic drivers that allow developers to connect mongo db his current focus is implementing new features in the mongo db dot net c. Sharp driver which is used by millions of developers worldwide and for the first time on dot net rocks rochelle palmar is currently a senior product manager focused on developer experience at mongo db which includes drivers and integrations with all of manga. Db's officially supported programming languages. Which are c. Sharp dot net python corsi sharp slash dot net python ruby. Php no go swift and java. She's been with mommy. Db's since two thousand thirteen. i mean. Welcome you individually. Welcome james thank you. Welcome back good to be back and welcome thanks mongo. Db james I'll never forget that. I Show that you did with us. Low those many years ago In the absolute perfect description of ifc. And why you it and It just You know it was a great explanation. I consider it the best explanation of those things since before. Or since i appreciate that a lot of my goal of technology is to understand things at a fundamental level and then share it with others. And i'm hoping to bring some of that. Knowledge of non relational databases and mongo. Db two emeals the dot net community. And i think i heard about maga d be in the context of raven db which is i thing and i guess he he wanted to do something similar to mongo but i never really really got into it into raven but i did like the way he was talking about. How index is created on the fly. And all of that stuff and i imagine. It's very much the same in mongo. I end is a smart guy. And he was trying to build a native and non relational engine on windows for the dot net framework and like a lot of the ideas. He took from mongo. Db we are competitive product. Why will fully admit up but with mongo. Db you've got a much larger organization behind cut a lot more features and it's amazing all the things that de mortgage is used. That's been around for well over a decade shall do you remember when we were founded two thousand seven might have been two thousand nine somewhere somewhere around there so we've been around for a while and we've gone through. There's been a lot of different technologies steps along the way an improvement. So that's remember very angie ireland with that canadian. Who's been quiet. so far we were on the Scott net rocks was at scott net rocks. It was the england. Ireland and scotland tour that we did yes and so we were in dublin. And we're heading out to go meet a friend you know for some traditional music and right next door to the hotel was manga devi. But it was just really. It was really like a a soviet kind of experience. There was just the word mongo. Db in a very small font on the door. You remember this. Richard was like. Isn't that kind of unassuming right for this huge as a company to just have this little door with his little logo. We're not yeah. Yeah yeah no. What i'm saying is it was probably a big office but just under understand it at the front door little door. A little sign on a perfectly normal size ashley right. So what's new in mongo. Db are we talking about atlas. Yeah we can talk atlas. We can talk driver right. We can talk core. Server product shown anything dot the dot net itchy has not been the usual constituency for mongo. Right is normally think of mongo in the context of the lamps. No right the lyrics world austin's often we the the actual core server can run on a variety of operating systems including windows including including a variety of other ones mac. Os things like that. A lot of times. People will build applications in whatever their development languages. And that's something that we've seen a real uptake in is that people were developing on windows dot net applications but we're seeing more and more desire to move dot net applications over I'm for the simple reason. Cheaper to run run faster Full stop mythic end of conversation. Yeah yeah a lot of companies are interested in doing this So that's one and the other thing is sometimes people will question. How much is mugabe committed to the microsoft ecosystem. We have a law of customers. Running the microsoft stack that are connecting among d. b. r. c. Sharp driver team is one of our larger grabber teams. We've got five fulltime members on the team right now that are actively developing features The dot net driver itself has been around for over eight years and we are supporting more and more currently going through a rewrite of our link provider to provide better links support. And that's gonna be coming in a future version so we're very much invested in the dot net ecosystem and in microsoft developers. He had the link side of Curing mongo with link is very interesting is better or worse like is very relational oriented so sort of get this. You know there used to be an odbc driver for excel to. It didn't make it a good idea. My the hyun kidneys. Cram that the link query style into through a among database. There's kind of performance again gonna get from it Quite good actually because from a technical standpoint C. sharp with link is just an abstract syntax tree. It's ast and we take that tree. And then we translate it into the equivalent. Mql were mongo db query language. And so it is actually quite efficient and we can get very good performance Is one of my favorite ways of actually accessing mongo db in performing complex queries often doing aggregations group by operations really crunching. Data is a lot easier to link quiry than often hand Handcrafting the queries yourself. 'cause you still have relations. It's just they're just not you know so diabolically specked out. I mean you just figure them out. On the fly build indexes he the the friendly way to say that his structure he added that in air quote. I like diabolic. Better more accurate. Well one of the. That's one thing that allows good to perform really well compared to a relational database. Is that if you think about your typical sequel database. A lot of your relations are parent child relations. Right where one object fully owns another one. But because sequel a sequel and everything structured tables in you got foreign keys you have to break everything apart. So you're spending a lot of Computation resources breaking things apart and then putting them back together whereas with mongo. Db you've got an order which has a shipping address and a bunch of order line items and a discount. You just jam that all into one document and that can be queried as a whole mugabe also has a very advanced query language so that you can say give me all customers who received a twenty five percent discount right. Even though that percent discount is buried inside order document. But if you think about it you don't want that order documenta ever change because that was the order. That is the truth. This was the point i made on. Jerry miller show all that time ago right is like you know the actual reference to the truth is all the things at that moment. Store those things then. Decompose them later for analysis. And i've done sequel applications. That are like that. They need to know the state of what was the address of that particular time. And you have to put in a whole bunch of machinery and always ensure that new addresses are always inserted in your version in them and all of that becomes very complicated and with mongo db. You just store the current address. Here's a really good example. The dot net rocks database is it is a sequel database and we have a guest table and we have a shows table and you know the guests have a photo and description and whatever and a bio right in anytime that That bio changes it changes everywhere that its reference so you go back to show in back from two thousand to. It's got a bio you know. It doesn't have the original by when picture it's all been updated and so you're listening to reading bio and it's the old bio well. Let's look at it this way. James's i show on two thousand eight and his bio on that show now says he works for mongo dooby before. That's right exactly. I know people are going back. What rachel what's your role at mongo. So i moved. Over to products in twenty twenty and i currently own all of our drivers and ap is and then in addition Framework integration so for php. That's larry will You know for ruby. That's rails and arm rails odiham which is mongoloid so a whole lot of stuff is summary. Boss here yeah. It's it's pretty exciting. I mean i joined manga in two thousand thirteen and it was a really different company. Actually not the tom. We didn't make any money wise. And i actually was when i first joined. I was on the cloud team and it was a much different product. Then this was pre atlas and so i actually have the spreadsheet of win. We first started making money. And i basically backed out our own personal credit cards. And we discovered that someone who didn't work at mongo. Db had paid us and we have a little party. So that's how long it's been now. It's been pretty crazy every day since i haven't been to. The story are of becoming a real company and that's without being acquired without going public like just growing up and that's a rarity to simply grow up in via company. I really recommend it if you can. You can handle a stress much preferred to work in at a big enterprise company. Where you're a cog for. Which is what i did. Prior to mongo. Db big big machines. So we've obliquely referenced times. I guess we better tell that story. So when mongo db atlas so among it'd be atlas is our database as a service platform you can use manga deby atlas with any cloud provider. So that's azure. That's gcp that's aws and essentially you. You know create an account. We have a free tier and it's free forever and you create your own little manga database. And you're up and running and you get your connection string and you're off to the races. It's huge at this point I was also obviously. They are when it was first created wing and a prayer. Hell mary type of style. And now you know we have c. Sharp actually is really big on atlas. We have over ten thousand active projects. Which active is somebody has logged in in the last thirty days and done something. So that's it's it's pretty. It's pretty big enough specifically c. Sharp projects right. Yeah overall project is much much. Larger projects is over over a million. At this point i would. I would presume. C. sharp is still very much a minority player in the in the mongo ecosystem. Yeah yeah you'd be surprised You'd be surprised at chuck. It's fun fascinating to me. You know when. I took over this position. I have to say like i was a little nervous about see sharp because it's one of the languages i've never worked with. I didn't know anything about him. Like microsoft yeah links. I never windows never But actually it's been super fun and it's one of my favorite teams to work with the community is just so great like really great And out of the top. You know i think. Out of the top. Ten gaming companies eight of them use mongo db and a lot of them use atlas actually which is pretty exciting so yes that's an area of focus and then of course like a bunch of Enterprise and financial firms uc sharp. Because it's a serious language. Meant for serious work. It's not an obvious thing. In the way that some other languages are well if you look at our commercials support offering which is where we make a good portion of our money. We're companies pay us to assist them and troubleshoot their applications most of these as you can imagine enterprises and therefore they use enterprise languages. Top top tier is java turnout surprising java c. Sharp is actually really close. Second for me. Let's face it. Essentially have similar origins. Right oh exactly statically. Typed object oriented management memory language in development environments like pretty pretty similar they are brothers yep and then pass that is that no j. us and the rest of our languages fall into down below that but yeah the top. Three languages are java. Java sharp and j. us enterprise side jaw job as he sharp and you sort of classic open web dev at davos. It's going to be no apps. A lot of developers whether java warsi sharp liked to play around with no j. on side. So that's what i've seen in enterprises is that they'll be a microsoft shop and sharper their javale shop so that all be java but then they got a bunch of nausea applications and both teams do exactly that they'll play with no gs his. Everybody has to write a little javascript. It's kind of unavoidable. I'm looping back a bit to them on the to atlas here. So if i'm on azure i can i can by this as a service and is also running on azure so you guys are actually operating on the three major cloud providers yet and actually if if you're worried about it or if you're one of the unfortunate souls who survived one of the aws outages in the past few years. You can actually have one of your each of your manga to be nodes on a different cloud so you can run across cloud which i like that well. Any par par somewhat multi cloud mythology to. It's almost like a checklist. But it's i appreciate that. Yeah you've done that. And so i could sink across these infrastructures if i wanted to migrate or or to have a fail over to another cloud i just have it as a guy who builds a lot of fail over solutions over the past decades. This is disturbing. I haven't seen a lot of fielded multi clouds like we can fail from from. Aws azure or. Did you see like that's just doesn't seem to be much of that. Actually lots of people talking about it. A lot of people talk about it but not a lot of people either. Do it or actually need to do it. Yeah we're i think. Multi multi cloud really plays strong. Especially the cto's is the lack of lock in if you're running on azure and microsoft decides to drastically increase as prices. Yeah you can just port your entire solution over to gdp or it'll be just not going to be free like there's gonna be effort involved with at least doable but it's doable. You do not all of a sudden reconsidering rewrite. My entire data layer in order to achieve these things. Yeah so what you're really describing then is prozac for right. That's actually is. It's like. Keep the cto com gonna be fine. We drop a couple of these in his coffee. You'll be all right. I guess it also depends on how far you depends on how far you go right so you can have everything in another cloud ready to go just laying dormant just images right and then turn it on if something goes wrong rather than invest a whole lot of time and effort and money into automatic fail over stuff you know then then it just comes down to up. Somebody gets a phone call or an alarm and they have to do it really quickly. I still think it would be days like it's just not that trivial wanted to shift that stuff across and and hopefully you're using the m.'s. And and containers and things that are relatively portable. You could go across deeply. Invested in any of the different vendors. Service technologies should've their distinctive analogy. There's going to be hard to move. Yeah that's true well and that's something that when you're building a cloud based application you have to consider a cloud vendors do provide a wide variety of very convenient services but they lock you into their platform. So how locked in are you willing to be. How much risk are you willing to tolerate being locked into. Aws or another platform. And how much do you want the flexibility to move services around. I like Azures coober nettie service because you can use your own docker containers and everything so as long as you have your containers some place ready to go. You could do your own cuban netease in and in gcp or in In an amazon if you need to But while you're on asher you can enjoy all the the wonderful high level stuff of ak s as a sort of separate thing. I think one of the other benefits of atlas is that secure by default. You get less. Esa sale automatically default. You don't have to set up anything which is really nice because you know. That's that's hard you've also got a variety of secur security solutions around like you can log in with a username password which is properly hashed salted over the net. Like you. don't wanna pass. Anything clear taxed You can use l. doubt you can use a api keys. There's a variety of ways of actually authenticating with the atlas service as well as private networks. So that you can isolate your look if you have a azure deployed infrastructure. You can make sure that it goes through private links so you only going across azure controlled. Interfaces you're not ever going out onto the public. Bob that reminds me. I want you to tell people about f. l. e. while we're here before we dive into that. Let's take break for this very important message. If you've had automating your. Espn at deployments. On your to do list. Now's a great time to give octopus. Deploy a try the starter edition. Lets you install octopus on your own infrastructure in deployed i s web servers azure websites and pretty much anything from no d. Coober netease and they just made it free for small teams. Give your team a single place to release. Deploy and operate software with octopus. Deploy find out. More at octopus dot com. This episode of dot net rocks is brought to you by every plate. America's best value meal kit delivery service. Not only is the food fresh and amazing but each meal costs about as much as a cup of coffee. Recipes come together in about thirty minutes definitely faster than a trip to the grocery store and starting a meal from scratch. Every plate gives you easy to follow recipe cards and pre portioned ingredients. C-can spend less time prepping and cooking and more time enjoying good food with family or loved ones. I wanted to see if every plate was as good as it was cost effective and after subscribing and cooking a few awesome meals. I'm convinced that you can get the same deliciousness at a much lower price so experience fuller plates and fat wallet. Try every plate for just one ninety nine per meal plus an additional twenty percent off your next two boxes by going to every play dot com and entering code dot net one ninety nine. That's right with every play for just one ninety nine per meal plus an additional twenty percent off your next two boxes. that's a one hundred dollar value. Go to every plate dot com now and enter code dot net one ninety nine. That's dot net one ninety nine and we're back. It's dot net rocks. I'm richard campbell. That's my friend. Carl friedman and we're talking to james kovacs and rachelle palmer and talk a little mongo. Db all the cool offerings going on there. And james i just interrupt you talking about l. f. e. a. Plant side field level encryption. Yes f. l. e. so what what that is. It's a recent feature that was added to mongo db and it allows you to do your encryption on the client side so you actually acquire encryption keys from an encryption key provider either on azure we now support azure and a and aws. I think we ought as well recently. So any of the major cloud providers they provide a key containers so you can request keys and those are ever only ever seen on the client side. So any data that you read has been encrypted with your key so the server the monkeys can't read it. It's just no pick blob a bits once it gets to the client. It's it can be automatically decrypted so it looks like clear text. So if you're storing hassle you shouldn't be showing say passwords sleep that topic out up. Ppi right like or p. I personally identifiable information which sometimes people want to store. So if you're storing credit card details once again need you really need to But things like social security numbers. Which often you do need to have them stored in your database. But you don't want your cloud provider necessarily to have access to the us. Side field encryption to actually encrypt these fields and ensure that and if you read them it just looks like encrypted bits but if you have the decryption key then all of a sudden it's readable unusable and is your ability to share those keys between devices so that multiple devices could decrypt it even though the store cannot yes So you actually communicate with the the jerky store aka s and if your client is typically do it on a server but if that server has been allowed access to a ks what y'all control take aka infrastructure than that client would be able to decrypt encrypt data to and from that is coming from manga. Db but if it didn't have those encryption keys for instance if it was reporting app that could only report on high level patient information but none of the particulars than you wouldn't grant it access and it would not be able to read those fields but you can go very granular. You can go okay. I want these three fields encrypted. But i don't want these other twelve encrypted at all so you can it not like you have to encrypt the entire object. You can if you want to. But typically Field by field. What sensitive rights. He could protecting specific data. But we we battled this problem with encryption on sequel server to where he was easy to grip the whole thing. But then you you know you just crippled any kind of querying because you had decreased everything to figure out any of that right. Not just a just encrypting sensitive information you just the salary information. Just the identifiable but the index the index stuff important stuff. We've still legible well while the other interesting things that we play with is you. When you're designing your encryption scheme what you can do is you can say. Let's say you're encrypting the salary information. You can encrypt it all with the same key. So they're always encrypt to the same value. You don't know what that value is but it's always the same value right and that allows you to still query it you can't do range based queries but you can do give me everybody who earns fifty thousand dollars stink because you can trip that fifty thousand dollars to a known value and then query the database saying everybody with is known value. I not know what that value is. But i know it's the same across these records exactly exactly so. There's a lot of fun things you can do with that from an application designs chant white insecurity standpoint. Yeah if you if you think hard about it but this whole encrypted before it leaves the device is pretty compelling for a lot of folks right. i don't. i'm not transmitting. It counting on some kind of encryption during transmission and then encrypting it again as it goes to rest on the in the central store and crooked it. Before i left. That's right y'all got for that and that's that's the client side encryption. Then we also do a lot of when you're on asher the bits that actually get out to the actual volumes are encrypted as well so we do on disconnection as well right to ensure that is secure while you guys mungo went through the security ride. I did a run as back in twenty seventeen about the mongo exploit which is like they. Hey you know what would be careful with. Defaults defaults are hard. Yes that it's not. I think it's also you know your products important when it makes but it becomes a part of an exploit like that like and now you have to think about your product differently too. And that was an interesting growing pain going from focusing on startups to an enterprise company off because as a startup. We want people to just be able to spin up a mongo d on your laptop. Not don't have to worry about authenticating anything you just connect to a certain port and you're off to the races so we made it really easy for developers to get mongo. Db setup running. Yeah well that was the problem and we had we had. We had all of our documents at the time. I was working with technical support and we had all the documentation in how secure it a. Nobody bothered insist. The that's the reality because the default was unsecured guy. You kind of have to force them to do the right thing. Even though it impairs their ability to get started yet by the other the atlas is another solution of that is like hey you developed locally. You ready to go into production. Why don't you use are secure one in the cloud and no worry about scaling at an all. Those other things like life will be easier this way. And you're far more likely to stay safe exactly because when you even when you spend up an zero cluster. Which is our lowest free tier always free that is secured not less. It has password protection automatically turned on everything secure by default in atlas as you can try out. I actually spent last week building applications with my team just like let's build a senior application and see what it's like from a user perspective and we just spun up. An atlas database put some data in there and were able to securely connect and started reading a phone app actually right on the device and connecting to atlas so that was a lot of fun but yeah it was secure by default of the box and as you start to scale. That's the other nice thing about atlas is you're going you say okay. Now i need more horsepower behind it this. I've developed the v. next plants versus zombies game. Let's start scaling this now we can. It's literally a click of the button in the u. Is like okay. I need an m. thirty. Which is our dedicated tier. Okay i need something with even bigger up to fifty and one hundred and what we are doing on the back end is basically scaling up the vm's at the running and providing more resources to the back end. She something somebody could do. Your ops team could be scaling mongo themselves. But did they want to do they wanna learn how they this thing and there are definitely people who do that. We do have an in prescription where you can. You can also just do it on community if you want right a lot of your own tooling you can download community for free and start building at your own mogadishu infrastructure. But is that what you want specialized right. Do you get a lot of customer requests asking you to move their relational databases to mongo some some. Yeah i mean that. Is that a common thing. And that's a that's an area that it's an area that i know. Our consulting team does log working obsessive. One of the nice things about maggie is. There's a support team. There's a conservative team. There's an engineer that we got all these different teams can focus on different things. There is a desire to move relational workloads over tomago db depending on the application. Sometimes that can be a good thing and sometimes it is more work than is really warranted. Yeah i don't find that lift and shift is super popular. I think what tends to happen. Is that they sort of sidecar the to databases for a while. And then they'll move workload over but it tends to be that they re architect in the process. Which is why. It's usually a very big projects like let's download all the data and then just put it in mongo. It's it's more like a even considering lifting and shifting. Because you have relational data that now has to be documented. Alexa therefore does need to be reacted absolutely but do. Do you think that if you were going to make the case somebody who's got a hefty sequel azure bill every month. Because they're you know because it's expensive. Let's face it and you know. Maybe it turns out that the shape of their data with lend itself really well to a document database. What's the case. Is it a money in performance. Both make the case there. I think it's both one of the solutions that is is pretty recent. I can't remember what year we debuted our data lake project but it's basically You know you can query your data in s. three bucket and then the results of the query you can put into a manga debbie database. So that's how some companies shift there content or their their data basically to mongo to be atlas which is a pretty nice way to do it because then it's like i'm only moving the data that i need just kind of nice i worked in consulting four gb. I think that you know it's one of those projects that we would love to bid on as a consultancy firm because it will take multiple years. No cost you millions of dollars and then you may not actually get any your net. Your net money gained loss. Might not really be what you were sold. I think well you look at the price of an enterprise license of sequel server today Like it's it's far more than the hardware. The hardware price keeps going down. The the licensed price keeps going up. Right what a sixteen thousand core right now for enterprise sequel server like. It's no fooling like the you could buy a lotta sequel azure for that and i suspect you buy awful lot of atlas for two. Oh that's true totally. I mean that's one of the reasons to move atlas also it's a consumption based model so if you are gonna run on thirties then you which is a sort of a smaller instance is like and then you scale up when you need to or during specific periods of the year. Like if you're a retailer for black friday you'll skill up and then you'll skill back down around valentine's day and it's fine. Yeah i've tried doing that with hardware. It's really tough to take those rams don't do it's not a good idea just leave. Leave those michelle. You don't have hot swap. Richard you pull hard enough everyday talks. That's why yeah. But if you know to that point. This alaska this sort of op ex capital choices like instead of investing in that hardware. And i'm saying this on the it side. These contracts are expiring these hardware. This harbors coming out of warranty. You're looking at in some cases to common numbers worth of equipment in licenses and so forth and you can get an awful lot of variable cost resources for that money. Like i just put that in the bank pay it out monthly and get pretty comparable results and ultimately if i careful and do some tuning and like you said dial down dial up. Those times are can spend less now. Nothing's for free like there's effort involved but by there's nothing easy about buying and setting up new gear either like instead pilot or the other thing that's true and like i'm sure that my our sales team would literally murder me if they hear this podcast but you can also in efforts to developer friendly. We don't have long term contracts for support so if you're using atlas you can purchase a poor because you have a problem or you want a question answered and then you can. Of course cancel your support. So you don't get locked in to this sort of multi year Thousands and thousands of dollars type of deal which is really cool. Yeah and it depends on your organization as to how that works. I also know companies where it's like you don't spend this money you lose it from your budget so you know i've been on the vendor side of that where somebody's calling me saying. Listen i got to this month. What can you sell me once you guys. Give me some kind of contract that shows. I'll get value from you over the next year. I just need this money out of the my account. This you are my friends. I will make something special. i guarantee you. Yeah well and actually the reason we first started doing that is because what would happen. Customers would have some kind of incident or some kind of problem and they would want. I want to support. I don't care how much it cost. i'll pay it right now and we're like no well we'll have to send you to procurement and then you have to sign some papers and they're like can i. Just give you a critical. No you need to start talking now however much money you want to on my problems and also take my money. Yeah yes well again. This is back to the prozac statement. It's like i've been that contract where it's like you pay a retainer to me so that i say there it'll be okay and then go make it okay. Well hearkening back to an earlier comment was. There's a lot of place that the cloud really excels is elastic. Compute resources where you can scale up and scale down. There's a quite a number of business. Not all businesses. But there's a lot of businesses that are very cyclical in nature either seasonal Where black friday. Rush or a i worked with a particular customer who shall remain nameless. Wonderful folks to work with But they they're peaks were around sporting events and every time there was a super bowl or an mc w aa tournament or some big sporting event have a huge spike in traffic. And then it would die down to almost nothing and so with. They were actually on atlas in what they do. Is they scale. Up to very large instances a or two before the big event. Host the big event and yeah. They'd be spending a lot of money for those few days but they've also got a lot of income coming in because of those events and then once those events are done they would scale back down to minimal instances to keep the background traffic going. And there's a number of businesses gaming companies. Do this where they have like. Blizzard has a big launch. And all of a sudden. You're gonna have a lot of traffic hammered. Now we can scale up and then once interest dies down then you scale back down and that flexibility that you're not locked into physical hardware and having to pay too because we're basically with physical hardware you paying for your peak. Now you have to the po provision the pizza and then you've got that peak provision hardware for the other three hundred and sixty four days that you don't need yeah that's absolutely true and certainly this is the new era right. The utility computers that we can buy what we need when we need it and in reasonably short amount of time i mean. How long does it take to move to a higher instance and On atlas minutes. Yeah there's a lot of a we do a lot of Interesting things in the back end. Most of the cloud providers will allow you to tweak their hardware once every six hours right so if you just need a like a very comic database we live and die by aiops guy ops. So if you want to bump up from like a thousand to ten thousand aiops that'll take a few minutes. Yeah i still think if. I might be doing ops coming into black friday. It's like on wednesday. I turn up the nah right and then sort of poke things like everybody. Happy with the big instance and we all good. We don't want to wait until noon on friday for another thing that we were experimenting with Experiment because it's actually in production is auto scale you can turn on auto scaling on your allison. Is that if you see a certain peak load for an extended period of time it will auto scale up to the next instant size. What's the thing you're measuring is like number of transactions are is it i opposite you're measuring ago. This view. Bombardier instance right now. A cpu guys. We're are pinned. Yeah you're processor is pin. Pretty hard bottleneck for mound. Go like i think disk latency zero bottleneck remember. I think you can also do it on. Aiops sedans on your workload if you're doing a very heavy workload. Obviously disguise ops is going to be your primary driving factor if you are doing heavy duty. Educations of where you're doing grouping expressions that's where you can run into. Cbs resource end. Seaview run. We almost never see network run. Hot is typically depending on your workload. It can query heavy and using complex queries like if you're just searching by essentially primary key are underscore field than that takes. Virtually no seaview resort sources resources. But if you're doing a lot of sorting in aggregation in server memory than that can take up cebu you can actually have atlas auto scale so that it will bump up to the next tier up to a maximum the you set and then if it remains if you are below a certain value for typical previously days. I think we're we're getting a bit more aggressive. It can be on the order of ours if you see like your views really running low. You're not using law guy. Oh then you'll drop down your tears again down to a certain minimum set point as it were very much experimenting with this for customers to optimize their costs on atlas. If you could get into daily with is like if you're a retail outlet that's streaming data from transactions but like ten hour window where every store is close like being able to turn that knob all the way down for what is better more than a third of the day or add up over share. Yeah for sure. What about the long term storage side of things is archived. Like i find that. That's an excellent. Sequel azure and these kinds of products as datasets. Get big. they actually get really expensive. And you kinda wanna carve off old data and put it away right so where how. What is the cheapest storage out there in the world today. Blob block store. Yeah whether it be s. Three or azure block store. That is by far the cheapest per gigabyte that you can get so one of the things that we enable is atlas data lake where you can actually pull off your old archive data into an s. Three blob store or a azra blob store and the nice thing about that is although your performance isn't great it's to query -able right so it's still there you know. Is it going to be slower. Because we don't have the full indexes we actually have you can issue mongo db queries against this blob store. Which is quite cool. Yeah but it's it's still there and available for reporting purposes but you're not in incurring the cost of having it hot an available immediately. I should mention the azure sequel does make backups. And so if you screw something up you can just go online into the portal. Find the last backup and restore that to another sequel database. And you're off to the races. That is a very nice feature. But you know you you pay for it. Yep we have automatic backups as well an in atlas so you can actually you can establish a backup schedule a. The backups are snap shotted into asimov's door or orange s three and then you can pick a snapshot and then restore it either to your own. Cluster to cluster. So you have similar functionality. So that you can because you live in companies live and die by the data. Yep and you talk about keeping. Cto's com lose weight like that's bad that'd be cto's get upset win. But so having visibility into data that's old is great but releasing the cost of storing all. Data's because the traditional solution is just delete data. Right or i built archive systems. Auto remote stores is archive it onto tapes yet but having it all built into the platform and we are investing in this as well so to make it easier to right now last. I looked You could either query the data that's in atlas or the the data and data lake. We're one of our goals is to provide a holistic view of it so it will look like the same collection but data that is past a certain year at certain point old will actually exist in s. three and be queried for automate that shuffling i can run into toads Over do do a batch. Once a week this sorta hunts down the oldest stuff in pushes over that street. But i'd rather more me than it's your fault. The other real selling point. I think is that there is automatic upgrades of your manga debbie server version. Which you don't have to do that anymore. Right break right but yeah it's nice that it's you know upgrading. Gnarly takes effort. But you always doing it wrong. At least if if the the vendor is doing it they're probably doing it right and if they're doing around they're upset rags up people at once and hopefully no breaking changes of version to version. Yeah we have a process where you know for the entire platform like said there is over a million active projects so we don't roll it out to all one million at the same time we we basically roll it out to a randomized ten percent and wait and then another twenty percent and wait and then another twenty percent and wait and then the remainder so if we have to. We can rollback of version upgrade. If we see that there are any problems with the i. Ten percent and related to that is not only mogadishu upgrades but os upgrades. You have to run your server software out on us and it's always a hassle. There's a new windows version. There's a new lennox patch comes out and going through the rigmarole of having to upgrade all of your servers. That once again taken care of. Portia not you guys mentioned if free skew atlas what are you get. Oh i don't know we saw. James will have to look up the stats. It seems like a lot. There are a lot of people using it and not just for a tinkering and not for learning mongo db but there are actually people who are running production level apps in our free tier which is kind of fascinating. We did a survey we sort of assume that most people who were Running applications on our free tier sort of learners or students or beginners with mongo. Db but that didn't turn out to be true based on the responses to the survey so yes so you can start out at five hundred twelve megabytes storage for free. And i think there's also a limit on connections and a limit on aiops. But i don't recall what they are sure. So if you have your your stuff out there your clusters or your your shared. Cluster and In turns out lo and behold everybody wants to be on it you know and it becomes popular. You can sort of migrate that up. I suppose that's great. It's also i mean. I use the free tier buy-sell for you know some one of the things i did. This year was analyzing stack overflow developer survey data. So i just put it in mongo. Db and then did some charts so that could look like a cross sections of developer communities. That was cool for that. And then last year i did a health little app which was mobile in a web app and they used our free tier for that and then used it for a talk me world so it's nice to get something up quick quick and running. I think when it comes to my health. I'd want to impact other than but it's also especially with demos it's like i don't like it in the bill at the end of the month because i forgot to turn the flipping off not that. That's this is all good stuff guys very very good. What's next what's next for you james offer me. We're working so. I've been working on the sharp driver and we of new features coming one of the features as coming in the next few days is were introducing. Better lennox support. So if you've got a dot net core apps running on lenox We're going to be adding the ability to feel bubble encryption. Cfl and the fear that i implemented was actually kurosawa. Nice so we're actually talking to the gp shared library on lenox and doing all the funding perot's ticketing stuff and that was believe me. I understood. for bruce before because i knew tickets i new server tickets a new Your ticket to all that actually. Having to implement the bits on the wire is a total total level of understanding. How you can see sharpe seven. These days We actually recently upgraded to c sharp nine po. And we can we do this. Hey we can if if we use a c. Sharp nine feature that requires compiler Libraries sport than we get under myler so we can use all of the new cool features like record types and others that come with c. Sharp nine and eight at did. I did mean to say nine seven. Oh okay moving in the past. But ultimately i know there's a lot of lot of really nice language features that allow you run much clearer code in c sharp and just some of the things that we're used to from functional languages are definitely coming over to see sharp you were. So what's what's in your sharp team I really you know. I've mentioned the gaming before. I started digging in on that and so have asked some of our developer advocates and developer relations teams to do Unity tutorials for c. Sharp from from beginning for beginners so build off entire game in ninety minutes so they've started doing youtube Live streaming of building a game in. It's sort of paired programming exercise. I think it's going to be really cool And then the other thing that. I'm gonna probably ask the team to do is I keep an eye on whenever we have sort of a little swell of users asking for certain things or complaining about thing and so probably going to ask the team to do serialisation improvements ca notice. There's a few tickets that come in faster and make it better. Yes the there's always going to be those. Yeah never fast enough those thing but it gets you know you're the pm role you could see the this hit a critical mass where it's like. Better look at this as something narrada to prove well in the serialisation movements are interesting because we actually have unlike some of the other drivers in one of the I was interested in joining the c. Sharp team is that our serialisation infrastructure includes complete. Odium object document map. So it's you don't just have to work with documents you could actually say. I've got a customer i've gotten order. Are you actual pocos then automatically get translated into the equivalent document structures by the drive itself. So you can get your business layer abstraction level and what the driver worry. That is really cool james. Glad you're working on. That's very cool. Well james kovacs and shell palmer for spending this hour so much for having us anytime pleasure seeing you too. And we'll talk to you next time. Dot net rocks dot. Net rocks is brought to you by. Franklin's net and produced by plop studios a full service audio video and post production facility located physically in new london connecticut and of course the cloud online at p. w. o. p. dot com visit our website at dot n. t. c. k. Ks dot com for rss feeds downloads. Mobile apps comments and access to the full archives. Going back to show number one reported in september two thousand to make sure you check out our sponsors. They keep us in business. Now go write some code. Cnx time law.

microsoft james kovacs rachelle palmer carl franklin jerry miller martin fowler jill dougherty richard cable james Scott rowdy Alex fair alex rich campbell James kovacs rochelle palmar mongo db Db james scott net rocks mugabe
105. Event Sourcing and CQRS

Code[ish]

33:41 min | 7 months ago

105. Event Sourcing and CQRS

"Hello and welcome to coach. An exploration of the lives of modern developers. Join us as we dive into. Topics like languages and frameworks data and event driven architectures an individual and team productivity all taylor to developers and engineering leaders. This episode is part of our deeply technical seats for codex. This is robert blooming. I am a devops. Engineer would salesforce. I have with me. Andrei makovsky a software architect at software mill. Which is a scale of development shop in poland. He writes in blogs extensively under an hour. We'll be talking about events. Sourcing insecure us andrei. Welcome to code ish. Hello robert hello everybody. Thank you for having me to get some basic concepts in place. Could you give us a brief review of the c. r. u. d. or crud model. Okay model. i think every programmer is familiar with this concept. Basically in classic system swift grads entities like user order etc are saved to the database as the most current state. And if you need to update something you'll simply override existing state. So i think this is the very short description of credit systems of course Systems you have usually operations like Create read update did of the entity end to be honest. This model is not the best idea if we need to implement very complex systems. We're talking about alternative model events sourcing. Let's start with the idea of an event. What is an event okay. Event is something that happened. Effect in the system or ed change to that location state. Now i think we can ask you. What is events sourcing. Let me start with the definition because some people would say the. This is a software design pattern. Others could say the disassociate architectural style. I think both terms are correct. However i would personally define even sourcing as the software designed butler that has a huge influence on the underlying system architecture. And maybe that's why programmers use these definitions interchangeably. Okay let's talk about as a design pattern and then we can look at architecture. Okay so i think the easiest way to explain this pattern is to show it. In comparison to something that is familiar tabby programmer like croats systems as i mentioned in classic systems entities are saved to the database as the most current state whereas in application based on even sourcing the most current state does not exist in the database. It can for performance reasons. It's called snap shoting but this is optional. Instead of state we are resisting a sequence of events. So as i mentioned fox in the systems and when you replay all these events for particle entity you'll get and the current state relatively easy concept and fun fact from greg presentation by the way greg yankees swallowed. The even sourcing evangelists is that humans have been using cuban sourcing for thousands of years. The i hate Implementation of sourcing was found on clay tablets on From mesopotamia nine thousand years bc and it was used for persisting market price transactions. So how many ships gray bradlow. Where both salt. And based on such tablets Merchant trader could very easily calculate the state of possession. Not necessarily by doing this. Molly for example by counting chips and sourcing for general bookkeeping has been used with great success until this. I love the sheep example. Could you think of another example that you could give us the entity and some typical events on that entity yet So for example we could have an order entity and order could be or the shaped or they're validated or they're rejected s things like that you've explained we don't update in place. We don't necessarily keep anywhere. What is the most current state you might have to look at every single event on the entity to find out what is the most current state now. Many business rules depend on the current state like he wants to make withdraw from an account. We have to see the current balance greater than zero. Does that mean when a change comes in you have to roll up the entire state history before you can determine if that change is valid. Yes exactly You need your domain gate to validate the input and to make a change to produce new events and unique to reply all the events. But i suppose the next question will be. Is the performance problem. Not necessary because you should do this. Only once when for example applications restarted and it's empty Without any state when you first requests for a given entity to roll up the state but then you should use the state from the memory which is extremely fast. Because you don't have to do any cooperation. So this israeli ready. I if you need to create low latency system for example if i log into my bank and it was using events included. Roll up my account balance when i create a session yet so that its present. I mean it depends if you need to change your bank account somehow so we drove money or something like that then yet it will need to replay all the events. But if you want to look at the current state you could optimize the whole protests and you'll some view mode the of this data so we've been talking about this one area where perhaps crowd is a bit better because the current value of the state is already there. What are some advantages of event. Sourcing can not do or that are better than with crowd first of all. Let's start with a complete lack of all states changes which is something extremely powerful if you never used it before you might be thinking Okay but do. I really need it. I don't have any business coments for all the drugs whatsoever. So how will i benefit from it and then you need to solve a very nasty back on production. You have no idea how this could happen or where the problem is. Usually in such cases you'll analyze baden of folks if you have them sometimes. The helpful sometimes completely useless. The bagging is also an option. If you know how to reproduce the problem however with even sourcing you have something much better. A complete lack of allstate changes in the form of events journal and you can replay events. Even by event analyze. What has happened with the usual replication. It's like machine. You can go back to the point in the past and verify entity state a priceless feature from my perspective. I use this kind of the bagging many times on production to prove that the system was working properly or to analyze what should be fixed in other cases and if you actually need an outlook because of the business or regulatory requirements than you will get it by design also from analytics perspective. If you don't lose any information about the system you can protest all the historical events you can create a real data driven organization. Could you give on. Example of an issue solved in production by using the is log. Okay so for example We got a ticket from a user that he thinks that application is was not working. Fine because he eh change something a month ago and we don't have locks basically our logging infrastructure. We can go back for two weeks something like that and we've been sourcing. We could read all the events So a month ago a year ago. That's not the problem. Could analyze the state and explain the user that everything was working. Okay now what happens if perhaps a bog in software. Maybe you have incorrect validation written incorrect event into the event log. How do you move forward to get to the state you want to be in that case. Okay if the state is wrong because made back in the source code and we produce wrong events so events are immutable objects wants persisted. They shouldn't be updated under any circumstances but the state is created from events also bet events in this case. So how can i fix it. Relatively easy we should you something called healing command which is either an existing comment or some additional semi business semi technical fink that will allow us to fix the entity. State discount probably Would produce a healing event or use an existing event but this time in a correct way. And that's the recipe for fixing the state once again. Updating events in database should be forbidden if for example a bank statement it showed an incorrect deposit. Then you don't remove the deposit. You would enter compensating withdrawal yet of the incorrect amount to get the deposit back to where it's correct. Executive wanted to talk a little bit about performance implications. Are there any advantages from a right standpoint to doing a pens versus in place updates definite even sourcing from the right path. Perspective is something that any database would love to handle append. only rights without the deletes is the easiest the fastest possible operations for majority of storage solutions. And i heard this on some presentation years ago that if you need to work effectively with the hard drive the best way is to look at if it were a cassette tape ends this way you will gain maximum performance from it so imagine that your hard drive is a cassette tape. You don't have to jump. Between many different places. You can write read from such deep without extra lax now. This metaphor is obviously correct for rotational. Hard rights with spinning days. Recording credit moved by some Surprisingly it is also correct Nowadays fast acidity the drives of course. In case of the the problem of writing reading data from different places much smaller but there is a famine. Call right amputation. About the details you can read on one of my books. But in general append only rights. Songwriting data chunks close to each other is a recipe for really low latency applications under. We've been talking about comparison between crowd. Sourcing you're trying to decide your architecture and system you're trying to decide which one to use. How are you going to go about making the decision of what's right. In your case. So i even sourcing is not a general purpose butler although sime indicators might help you with the decision if been sourcing suitable for the case let me start with domains. Were even sourcing. He's a good match so bookkeeping systems already dimension trading systems right now a lot of benefit from even sourcing actually any system. That handles money in some way would be a candidate for even sourcing Simply because sooner or later you'll need to provide a full audit log and even sourcing of drug is something that you will get almost for free. Also any non trivial domain could introduce even sourcing At some clarification for example you have simple crude application where the user sending requests deputations updating something into on the database and reply. We've the response. Don't go even sourcing. You don't need it but if demand is more complex and after persisting something to database you also need to send an email. Dan invoke a some internal micro service than external micro services to finally. Send the response as a website at to the. Because you don't want freezer to wait for all these steps to compete and you will do this asynchronicity. Maybe this domain with the complex process will be easier to implement. The we've been sourcing. I think the rule of thumb is that you don't have to use even sourcing everything even in your domain. You should choose. Some subset of this domain probably depart which brings money which is the most valuable and maybe this will be a good candidate for even sourcing update from a different angle. If you have a very special technical requirements like speed really low latency scaling capabilities. Dan you should also consider As possible implementation there is no magic cure. It doesn't mean that even source will give you all these features just like that although this pattern will at least not block you in this area of scaling speed end because of its mechanical sympathy. It will allow you to create really fast in really wrecked solutions on the other hand. Where even sourcing is not the best idea to us so simple kurt applications. That's for sure where you're events names would be like user update user deleted. Also i noticed that applications or parts of the applications were need to manage a lot of complex relationships between entities usually with very high level of consistency. Transactional are not the best match for even sourcing. Most of the popular web frameworks. Now give your crud pretty much out of the box. You can get a crowd application very quickly. Are you thinking in terms of you. Have a situation where it's a good match for events searching but there's a bit more cost to build so you have to decide if the cost is worth it to get the additional benefits yet. That's for sure to be honest. I don't believe in auto generated the systems because sooner or later you will pay the price for the very quick start of the Have a pretty good introduction to events. Sourcing i wanna move onto the next major topic see to our s which is going to build on top of events. Sourcing first question is what does it stand for. Okay come on query responsibility segregation. Diarrhea is to separate the part of the population responsible for handling comments and writes from the part responsible for handling queries and rates and this separation is not only on a software level but So different repositories different deployments but also the hardware levels of different hosts different databases c standing for command. Is that a reference to the well-known command pattern from the gang of four. Yes yes yeah. Give us a brief review of the command pattern. It's quite easy. You simply encapsulate a request from the user as a command object and this way you can build comment processors that will consume comments from different sources. Like h chris kafka messages whip socket messages cetera made part of. Cq arstan is the separation between the part of the system. That handles rights from other parts to handle reach. What's the motivation for that. Or where is a case where you would want to do that. Where you want to scale those parts independently or where you want to have a separate modo for rights and for comment handling and separate model for rates in data modeling in a relational database. And we're not necessarily always talking about a relational database but the trade off between a highly norm is modeled. It's more efficient for rights in d. normalized model. It's more efficient for reads and end up. Doing is making some kind of compromise of what you can live with for what you can live with for the rights. Are we trying to avoid that compromise here by giving each side its own model. it's optimized for what it needs to do exactly no more compromises like okay this single mother or single database. Ab it's not perfect but at least it can support dries rights and most of the queries. you can choose the best model in the best database for rights and the best solution for reads and scaling possibilities of such. A system are great. Because of course it won't be a piece of cake to scale it. You have a lot of moving pieces but at least it's possible. also you can do this completely independently. If only reads are the problematic part focused on them and add more resources for query services or for the database responsible for each or create a new mold that will allow you to have a very fast rates on the other hand. If comments processing is the bottleneck analyze. What could be done to fix this without the read side entirely the command processing. I'd which is the right model is going to be an event sourcing model with an append. Only database exactly and that is not necessarily a great model for doing a lot of queries against it depending on what kinda curry's but it might be very unsuitable for curry definitely so it will be really really hard to query events three and get some aggregations or some very specific queries yet. That's why even source against securities Are more or less used together. You mentioned different. Databases would an example of that be are gonna have part of my read model. Be a document database or graph database or a key value store that would be a specialized database. It's very optimized for a particular type of read and it doesn't even have to be the same technology as is processing the rights. Exactly for example. Do you have an application for managing users and their contacts friends and now on the comment handling a site be stored simply as a new entry in the user's friends list which is easy to modal insecure database and we need to also provide the way around very complex queries about users relationships Also bulbs from this application perspective. We are in the second level of relationship and we need to support. For the ford levels will such Will kill sequel database but for graph databases a pretty standard query and insecurities will create a dedicated rate model. That will work as a different view. On the same data generated from the komo unhinged excite any smart efficient to get the information about users relationships from such view than querying. A database responsible for handling comments is it also the case you could have multiple read models each one optimized for different use cases. You can have separate retail the for a single query. You can have separate database for a single query if that's the problem for you if that's the only way to solve the performance problem which is great because you can't use the best tool for the problem you want to solve in this case a very fast and efficient rates supposed deli meat is only in your pocket you can start with something simple even a single model and slowly evolve your obligation to meet new requirements some extra production It's obvious question. You have this right model. Which is the source of truth and you have line or more read models which are only useful if they have a reasonably up to date information. How does the data get from the right model into one or more read models. You should own something called a projection which will consume all the events from the common handling part and update various In real time. So we're talking about this in a conceptual way as more of an architecture pattern are there. Some well-known tools are frameworks. That you can build this without a lot of custom work yet. I could recommend aca persistence. Aqap assistant typed. Both solutions are really mature. Use it in many production solutions. And i can honestly recommend them so when the wheel use some existing solution and developed and tested by many programmers. Many users. there's gonna be some latency before data from the right model shows up in the read model if you adopt some of the more proven technologies and was some tuning. How far behind is the read model going to be. It should be less than i would say. Ten millisecond if you have bigger latency may be. The problem is with your original. Those may be very well. Those updates are slow or something like that but the whole process of moving. The events from one parts from the command handling site to the rate handing sites should be radio fast. If you use good tools for that many enterprises are doing something like this where they have transaction databases data gets pulled out of them into say dupe jobs and data warehouses. They not be calling it c. Cure is this putting a name in a bit of a sharper point on something that most organizations are doing anyway and just not calling it that probably because this pattern is really heavily used for example any database internals is using sputter so right something to the lock in. Dan update some models that could be queried by users. As so i suppose. Many companies are using even sourcing or similar solutions. Stephen sourcing without knowing that they are raising even sourcing. How spread is the adoption of secrecy. By people who've been influenced by the idea of secrecy may set out to do that. And if you ask them what they're doing they say seek yar s. Okay so my personal observation. Is that more. More systems require real time processing agency high throughput end with securities. And even sourcing. You can address all these moments or be ready for them when you hit a Certain threshold also stream processing streaming platforms as a way to build systems are getting more and more popular each month and forcing much as such solutions and from a completely different perspective. Functional programming committee has been growing from quite some time and here. Even sourcing is also pretty good much. So imitable events avoiding side effects not to mention about the state which is nothing more than fold left on apply even function also reactive manifesto. Probably you heard about it. It's built on top of asynchronous message processing so building elastic. Resilient responsive leverage the message driven approach effort creating software. And if your messages are events. It's even easier to do this right. I want to change direction on together. Talk a little bit about what type of persistent store. Would you put your events in. I can imagine you could put them in a lot of different types of persistent stores but are there some either specialized events sourcing or of the more well known open source. Databases are there some. That are a better fit. Let me start with relational databases. Because i wouldn't cross out sequel database as uneven store. You'd be shocked. How fast append on. The rights will be Bike classic sequel database of course sequel databases were designed to work well on the single Sooner or later you'll hit two possible limit. So the first one is of course the seapower and the second one is required disk space from my perspective. The second on is even more often the problem than the first one. This shouldn't be a surprise. Since in even sourcing systems. We are not deleting any data and even might be pretty flat structures of course vertically scaling spew or disk space is extremely expensive from some level however if you need to handle and moderate load than start with secret database it will save you a lot of time at probably a lot of money once you grow so big that secret database is not enough. You'll probably also have enough money to migrate to a different storage. Now if you know based on some data that storage size would be a problem. All you need to handle thousands of rides per second and then you should start thinking about distributed databases which can be scaled horizontally A perfect example of such database for even sourcing is cassandra or any database with similar concepts under the hood so still be the db etcetera under your wrote a blog post about event civilization. Why would i want to serialize my events. Well first of all you don't want to mop event attributes two columns because it's like the worst idea in most cases events will have a completely different fields. So you'll end up with a lot of notable columns also it's not very efficient Comb by comb from time to time you want to also change something in the event so at remove some fields or at a completely new event. Good luck with fiscal evolution. D- backwards forward compatibility when using a column per field approach. That's why you should learn an analyze serialisation for sourcing. What are some of the more popular civilization options so the two main categories are playtex formats like jason End binary formats like uproar About three others and most people usually jason for even series asian. Because it's well known anybody can read it. et cetera. It's okay but from my experience it's worth spending some time in at least analyze binary formats Why because they are faster when it comes to Dissertation produce smaller payloads is important. When you want to save this space also they will give you a very good support for schema evolution. So if you want to be always backward forward compatible it will be much easier with binary formats than weekdays. The last topic brought up a little earlier with snap. Shotting i can guess we're at is only describe what it is when you would use it. Slept choking is a performance of the mutation that consists in saving the current state every x. of evens it could be one thousand or ten thousand or more or less depending on your case. You don't have to load all events from database to get this current state. You just need to go. The most current snapshot and then all events created after snapshots. Great before we wrap up. Is there anything else you wanted listeners. To know about events sourcing are seek yar s that we haven't covered okay so i've been developing even source systems for over four five years and i must say that it was one of the best decision to try any been sourcing approach. It has really changed mind in the sense that when i'm thinking about developing a system or designing a system i'm not so focused on the state itself which is very common for many programmers. End leads to many problems. I pay more attention. To the events. The overall business protests the events flow in the whole system. Thanks to that. I have learned a lot of techniques to overcome many challenges when building a distributed system and these are not single use patterns day they are used everywhere or speed in scaling. The program even sourcing osama notice is turning your database inside out which is interesting idea because when when you're creating an event even source systems you're actually creating a very specific one of a kind database. That will solve your business problems. The inside out approach might also be a good candidate for the inter micro service as communication By making the events and they schema public each micro service can arbitrary use data emitted by others of course translation. Liar will be useful. We don't want to expose the internal scheme of events and other developer. I can only say that extending tool books. We've even sourcing is something that you won't regret end. You will benefit from it for a long time. I would like to find you on the internet. Were should they look. You can find me on linkedin. You can check my blocks on medium or my homepage. Thank you for speaking to coach. Thank you very much. Thanks for joining us for this episode of the coders. Podcast coach is produced by her roku the easiest way to deploy manage and scale your applications in the cloud. If you'd like to learn more about code ish or any of rubio's podcasts. Please visit peru dot com slash podcast.

robert blooming Andrei makovsky robert hello greg yankees salesforce andrei Dan chris kafka allstate poland Molly taylor greg fox Aqap Functional programming committ Diarrhea curry ford Stephen
Interfaz Podcast Episodio 138  NetCoreConf 2020 con Manuel Snchez, Txema Gonzlez, Adrin Daz y Robert Bermejo

Interfaz Podcast

34:39 min | 1 year ago

Interfaz Podcast Episodio 138 NetCoreConf 2020 con Manuel Snchez, Txema Gonzlez, Adrin Daz y Robert Bermejo

"Interface software several your most soft skills the NBA needles. Kc Amigos me number as the escort. Yesterday's open up. You saw your fast. Import cast the Arctic. Today's over this. Your software nervous signal. Hia IS SOFT SKILLS. The autonomous wimpy. So you'll seem precedent Squadron be does primarily is Manuel Sanchez. Skin Komo technical manager in Hispania. You're the masses Microsoft. Mvp Asher that mean contaminants percents HMO Gonzalez kidneys architecture software in CATALANO ACCIDENTALLY DUMP? Animals La Percents. Here we have the S. kipness. Mvp In lack agree are the office development putting more Robert Bermejo Kin- commented on Komo back and live in Sec. Ama legal international hub. Yeah the MASS MVP of Asher. Komo's Johnson Yours. Bienvenidos interface added listed. You ever see skype support Podcast THE CINCO PERSONAS IMPO bubbles poster. Iowa's guy you so you can know skype. In Oregon Antezana Circle know stamos. Take Nichols E. ULTIMA. Principal disappoint cast per focused on Commasso status for publishing circle. Manuela Cleo get one actually get these and Independence Day in the goal and drove it in your at Ila Unical when he's The reloaded got food. let's Olympics Olympic Kmart Clintonwas quintanillas for going into Lower Microsoft. And a personality. No here's you they sorta. They'll they'll hokey not paying for mine. I'm waiting on You Zealand. They went on a base. And then I got. I was also be that a move. Then I'm waiting excellent. Hmo Being Bonito women needle almost has to be proud of the episode Yoga Llamas Olympic Piece Omit McQuillan facility by Saudi off a coffee. Nosy be almost NICKELSVILLE. Snow mean we don't get only one without without any general because they feel anything. I knew it'd be good. I knew off on house because of the deductible. Then you're thinking bundy finalize Nicole. When they will not say Miller. Bishop will it yet? I just in again. And then I don't know and the Media Tab for se your local. Could I see associated with Manila parole? Tim Obama's Robert Come south road whereas the end which has already made no sense at guises on Rodeo Balchunas. Landau someone still doing it. Because they only go so bring eligible. I simply ankle on. What's going on it and say it is in focus. It he this a so most seminoles around those demos for savior stay as close up at our competitors. Excellent roller just relax. Nip Core Estar persist in Tokyo. Yes H. O. Individual Sasha this in Hispanic era I wanNA say Clinton was Canarsie star is they are Alimentos. Komo's who said your acquaintance Makino Qantas don't to see. The AD is known as an not feeling their laundry they did on my newman. Who Tim Tim? Tim Cover The timber thin Yankee at the Yankee by using Briseno signal and it's a moment he's Getting closer to los but regaining stole open as of May may be one of those. Your your letter your candidate when. Yeah when you're coming when come on Saturday. Those e E. Industrial Vega. Luckily you gotta almost have finished. He ended down on them so they don't pass the consistency. After the annual L. Premier Premiere Nickel consequences millionaire physicist jumping K. Communist Switzer EA admitted into inflammation Mississau-. But okay I'll contract settled on the success in the United States like Western. It'll be mad if Moscow blitzer. No we don't have been limited in the Senate in the last Movie than a bomb or Sunday's fatal kinda surprising when she wants it Komo Komo Ceelo lattice obscene in Hispania Cable. Claro speakers repented capacities internationalists. That up at all in the Komo. rentals independent Microsoft in his panacea. John O. Cocomo Kumasi Lewisville. They tell you you're you're gay. Primitive Industry said I let him go the speakers stop. Chi-china spend sallied papers the empathize and they will be A. Aw Italians by Gordon e Semana the neons and the different super obvious Donald Komo. Lola entered into a joy decor data. We need those. No one does into the empathy Nissan. Also Tracy escape your windows in method L. L. dollars globally children U. Mass. Go visit about it so when in Wendover Nevada. If you aren't you're talking about the game online and mobile eat much in does he balance in paramus. Cfo and thinking you know. If they've widened. Apple denied the grass. He hated adult demise. If you'RE GONNA LAGO and they're interested in August. I must have been ordered. Not It certainly has the theater. And then throughout the day the what the Russians mementos but but people and wounded endo indoor and they see what aspects on hasn't westbound instead of simply much opportunity then wind drug ammoniac stable industry which is not necessarily Appalachians. Local SCARES ATLANTA'S MORNING. That the is more revulsion. Alexi to Canton Mikhalkov. Allies in continuing through stays quiet role. Co-organised lorries why y'all know. Cnn tastes in Rowan cowardice. Us Tennessee Emeka. So Tim out. They don't get mad at me if I'm honest. Assist Right Gail. I'm more tougher. Atl See simplisafe. Can only supposed ki pessimism over Robert? He'll be miserable in my opinion young illegal. Even though party quiet through squad remain to seek supercup. Saxon voice began recording again. He's not let me see like I guess it you eighth in just seven does it must see the. Em necessay tackle Mooney. I'm going to get the biggest mass by denouncing my feeling muscle my Bush. I Know Marty astonished then. Walk as somebody because they won't cow. Nba lagging lagging. Is Michael Toscano in the local mall? Finale interrupted standing standing in. In when you're GONNA get him eating window. Fan Gate Danubian security Iraq when we are not our military get below you gotta. That's he'll be honest. Speaking looking at an wasn't going to be that may almost no I don't under no eat in but couple of days but it also must have seen as Roma's Nada but so I think it was in Vietnam. I'd be skiing Atlanta today at the APP but gave most weather in winter. Grundy SORTA come on now. I will meet APPS ASSIGNED THEM MELINDA. Goldman just call us and this gives him Rascal Taddeo. It's overkill throws of almost GONNA at Arthur Andersen was going to say that. Come all kind of started going to get back on show kind of the customs from necessarily semi item. Look for gay fool do become more because I love the guy who they're not take them get paid by lava young people build those those Lower Emelio cuisine priority than diet plans and duty to put the devil stylianos homebound better. Liam ammonia daddy up tennis shoes burying England bill on Sal assays at that is was the GonNa tell us under concert familiar. Marcus without ballot comedian absorb bitcoin Athenian dossier with one of the nameless case. You've been out on the GonNa Quintals. They stay a mental Proxima. Salvo quattro seen a lot of HR SANTOPAOLO OUR K. Alimentary stumbles on on them. Your global e. Were NOT LOOKING PIONEERS. And all those places months of thous-. Komo two step process will the Columbia less than the owner. Vento percents Seattle on the ladder. Saito being authorised sallow estimate. Parcel Persons Yell. Our gave our Komo Kumasi incident. One kids around littleton interested the entire value being. I'm not gay. Gives the fifty corn goal on the on the almost on the a A political wason value out of maize. gay that made up was no bus in the fall unless something on. Samos From yeah he was out of the House. Gus Houses. Listen I sister is on a per Technica Laporta Technica seal to all platform anastos style. Is this combative for Aguado. Button is in a lifestyle dot. They're does this. Tell but the bath Eliah the just an or Langa was Tele Scales reveals speakers around. GonNa vilely the Mossy similar industrial. Are you being gay until six on US giving us the woman's one hundred and linear told US pregnant? Well wouldn't ask lear. They'll cloudy cloudy but it's W. O. Poor Caffeine elementary is impossible ptolemies Monte impulses his Moscow model. It'll this career lassus. Yanni better la a poster it but it has a Pelosi Speaker Sundays and Amazon two. In Alaska token there is a Bloomberg stocks until eight at least not initially mass daily Dolphin. Mostly live that must've loved that our according Chubby Florida lesbians in April remorse case about ask kilos because people in order kid or at USC Demo second-oldest nil relocate. Nimmo's nagging was pure on we'll stay. Lesean insists Yanni's technique pencil now. Almost WANNA start a little stall. Oprah's theon of getting another building when he is that there's was almost by lots and lots of common sense unite for painting in look in an evil go. I blew him yell at the student. Or jimbo putting technology Jahmai says he has no. This is not gonNa Damborena Yahtzee. When it's much more important than the ebay Bruno Taut adequate Ghazi looking on its different. You're going to see on this day. I eat what I do. Anything rental. Nestle you've done windows. Doug Pederson pimples. They gave Luminous Ivy. This guy get past. I didn't get any moderated by miss about the so as soon as I have been Janika. Neal's nobody alison find out. I keep wanting. Ah almost rather to give him a little more. Still do talk. I'll stop let me tell. You must've tuna spending on seal aiden Speakers Mollison anybody else. I need you you being processing Canada. Okay it about the see. Some entail retired case that's mentioned under Saudi law into DOS. Maybe Nola Mental Equality Komo Wednesdays. Doodo they landed. Combs communicated nudity. Randall or your mentality test almost Christine the contents for Payroll GonNa Wednesdays Fatullah confidential. Can but you know you got denial critical sukhum window or money is fed a maximum lose the April forever. There's no anthem. Faye Daniels locate here H H whether or not. I think it'd be going into the game. Was Duke Niccolo. Get hope and really cargo in one of the less and is in the Middle East Daniel Quiz Nassir up when I get the group forty off the off. I'm thinking I'm nick for the benefit. Not as it's not food it will look alive. God with competitive is still in the number of people around me Malia. Get a they're gonNA those simple kipling by okay Tony Miami and install simpler tax total and get the Damascus donated. The they pass years. How WINDOWS THEY? I say I started grey. Okay can I mean karate? Total wait until my last breath as was does into lethal e competitive interesting people's sewn per gig note was a man man multiple only a you are professionalism Potato personality. The city possesses militarized because Dan ASEAN though in the form while. Carlino technologies have income to the mass. Still case can okay. Muench CMO IMPACTER Espana put in Salamanca baton impact. Oh Grow Elsie. And stay there in Nassir Corporal Pessimism Robert Aniseed. This is tempered. Pavel's commentator products Coachella. See ooh by Laboon US again. This recommendation only will come in the Commonwealth by boy. You've gotta come in that it would be wanting to me build demented mind. When I meet the equal kiss they actually wanted in in the PPO case leaving the show by Sunday asset over the last thousand theology on Africa. He gets on the rice aid from doing today is and gave a window boys out said his uncle wants the perfect woman. Scorned Timur Quint Donald Shimer. So he mas daily resume will hang out the and this this comes up. There is one of our negative over there. You wonder the debate among the bidders guests will not say that the Liberals who can forget I love it. Talk a little less. You can notice cancer. Perfecto Polka Afri. Less than the game. It was almost God. I got to boogie doll. They did Legally Melissa. Only make like a total he. Audio commendable Bono. GonNa look at the fucking cook it out. You'RE GONNA go get somebody to lean liking than the Chai music. That was luckier. But our bobby sally the bird and Dougherty El Mundo's okay. Okay Perfecto Moon lasorda. Co E parole Timo convert Mr Model Railroad. Canova you're in the factory in the market for let classical on Komo so where Brennan Diko and it was it got it set up a little bit damper sentiments about. I'm able to remain limited. Norian said okay. Give Me Missile. The aim at me mental in Boca when the Act New York Assembly Vinyl story we own antibodies dock labourers day Santiago sticky or go to extend the legal economically. And they're not GONNA say they'll say gave those who've Nano Ella Elementary Awfully estelito Eve as this real this up to your people elevator. Jabber Grow Imbaba Elliott semi over unawares. Look the muscles mobile APP name. Total import every factoring style. Seon awesomely renew classical classical Latin. Lahia celente recommend the refractory the Martin Fowler against his cheesy multi focal economist. Podcast symbol well is in courses impact O. E. animals with startled until the wing. Nedcorp lescot stairs when we'd be one which is a rain. Windy Wild Dem's technicals Kalomo. This will defend in this category. As Paddock insane. Knows this Cleveland Keynesian though Manuela. 'cause the IOT Amnesty Damian down again by Laporta the OFFI C. Sharp pointy the tolls. This causes causes of scores. So Avenue an empty mind. Your team isn't it isn't get ready in your area. I cannot go near Gloucester excellent. That was already there and their mass a circle this normally when she must get us border so deem poor e atoll's musters police Kuchar mammals ent mcgahn. The interface widow again in the thousands.

Komo United States Microsoft Tim Tim Nba MVP ATLANTA Donald Komo Asher Komo Kumasi Iowa Komo skype Oregon Antezana Circle Manuela Cleo S. kipness Yanni MASS
iOS Interview Questions For Senior Developers in 2020 Part 2

Inside iOS Dev

15:00 min | 1 year ago

iOS Interview Questions For Senior Developers in 2020 Part 2

"You're listening to inside. Iowa's Dev shod Real World Iowa's development two day with you your host Alex Bush and today's episode as part two to the previous episode about IOS interview questions for Senior Developers in two thousand twenty. The article that I wrote S- and this is the part two episode where I'm covering article so in this one. I'm going over questions next question. So we'll start with question number four. What is what is NBC? It's a very fundamental question again. Even though it's very basic question senior developers also need to know about and expected to give more detail on the show more depth of knowledge of such basic questions so NBC stands for Model View Controller. Software Design Pattern apple adopted. It came from small talk. Apple adopted its towards their own. Had has has its own reincarnation of it Model is responsible for data controller is responsible or views responsible rendering the dates and getting input from the user and control are supposed to glue views and models together right and be this coordinator control right at the end of the day. Apples flavor of it is More view skewed towards the view. That's why wouldn't really have controllers. Who Have you controllers what you should know? Besides there's this basics That that I just covered you also need to know and understand that model view controller. Nbc is not your application architecture. That's just a design pattern foot of you and if you want to architecture application especially architectures for scale you probably should go for something something else. Something with more flexibility. Something's more advanced at least. I'M VM MVP COORDINATOR. Design pattern or go for a heavy duty architectures such as viper and ribs but again as a as a senior developer. You should be aware of that so erratic flag that could raise here as well simply not knowing what an NBC is right. What what it stands for. And what that's fundamental Basic design pattern of APP Iowa's Apple Development in general I think they use it on all the platforms but nevertheless I again as I mentioned already as a senior. Dev rather the expectation is that you know. Other design patterns and architectures for further reading. Look at Apple. Apples article on their developer portal. It's called model dash view controller Then ABC's Dotto has a great article Cold lighter view controllers. It's from one of their previous issues of Went UP CIO. Had this Monthly I believe issue of their articles. Then there's another article on a con Lou con Lou. I think that's how I say it that come about massive you controller. Kennedy shoes with NBC. Another one is on a Napa soda actual this podcast inside. Iowa's Dev where and I covered the the issues with controller skull the problem with the problems with controllers another article you could read introduction to n. b. b. m. that's also obviously that article And then another episode of this podcast view models to the rescue As far as I recall Andrew in that episode. Goes over a great example of where? Mvp and could could save your bacon in the overblown view controller. Massive you controller code base and another article could read up on a model view presenter. It's MVP by Martin Fowler. It's on his website. March and following dot com. And as usual. You you'll find those links south. Put Him the Up put the link to the article in the show notes and they're in the further reading for that question. You can find links to those articles that I mentioned. The next question is number five. What do you know about? Singleton's where would you use one? And where would you not Singles Singleton Severi? It's very common design pattern. It'S CONSIDERED TO BE CO CORE. Competencies design pattern and it's used everywhere throughout all of the apple examples and They they actually utilizing soon bilton themselves in the API COCA API that they provide a four us. This jar short answer. Singleton Is a class that returns only one. And the same instance no matter how many times you requested what it really means at the end of the day that you have a type but say networking services and you implement a static or property rather. I think that's typically the implementation called shared or something like that instance and then in there you basically cash that instance the very first time. It's accessed you initials it and then you cash it and that's it. And then every other time when the in through that static property the instances rick. The SINGLETON has requested to return the same instance Singled and Sarah unfortunately anti pattern. They have a lot of issues with them and specifically they encourage global states. The they they really did. Sir and there are a lot of issues with dependency injection they. They really don't help at all with it. They make it more difficult. They you have to tie things to specific concrete types instead of relying and interfaces things like that so and in general even though apple promotes it. I do not recommend using singleton's and I know it's a it's a kind of a debatable topic if you will but there are a lot of prominent articles written about by prominence experienced developers Basically kind of debunking if you will the design pattern than explaining why. It's an anti pattern. So the Rad flak for this question as well you should actually know what a singleton the is and how to implement it and how to work with it even though it's a necessary and or unfortunate evil quite often and for further reading to to kind of get into the details of why. I'm saying it's an anti pattern. There are two resources you could go for. One is actually episode of this podcast. Called Y Singleton are evil and another one is actually similar styles. Single single are evil. It's by you can find this article on Viki Dot C two DOT com. They cover it more. They're in in more details in an an unwrap all the issues with singles. So the next questions number six. What's different between delegates and TV? Oh so this is a this is also quite common question on on. Irs INTERVIEWS THE DETAILED. Yar The cute. Let's let's start with What what is Ki? Ki Ki all right cave your sense. Four key valley observation so delicate design pattern and Kiva observation both are techniques for to establish relationships between objects and delegates establishes and declares using the delegates. You declare one to one relationship using protocols delegate protocols and Dan with Kiva labs operation you declare and establish a many to many relationship using other techniques but mostly. It's a kind of declarative key valor. While the that that's in the name Kiva Observation and your keys likely will be there strings or I believe you could use like selectors or something about for but the idea is if you want to when he needs to pass a message from one object to another you need to be aware are you in which type of relationship you are. Do you want to just send that message to one object than its delegates you want you wanNA use. Delegate Design Pattern and that object that receives the message will have to implement that protocol a delegate protocol and the delegate t will have to have a reference to that object that implements that protocol and with Kiva labs operation. You and then another one is Kiva variation if you want to send a message. But they're multiple receivers You you you could use key validation and then broadcast message and then whoever's listening and the the the broadcaster doesn't need to know who's listening there's a need to have a reference to them and then the listeners subscribe using What is it called the default censure notifications something something like that object and then they they use that to subscribe to those broadcasts that other objects and broadcast messages. And then one or mania of those or non actually could be observing that those broadcasts. That's that's overall the difference there is another TV always a very rudimentary implementation of observable design pattern effectively. And the way better implementations are Rx swift and reactive coca and now we we have combined which does the same dose are functional. Reactive programming libraries. That health with Observable design pattern and this broadcasting in a one to many broadcasting a red flag for this question spoil essentially a season developer should really know Not only the difference between the two But also what are that you know? These are all implementations of observable design pattern and what other options such as combine our our our Rx swift that you have available to implement them further reading for this question. delegation article on on Apples. Dev Portal Key valley observing an S. hipster dot com articles pretty good overview of a key key cave you'll and I highly recommend looking at Two others reactive x dot. Io that's a rx not just Iraq swift but in general Rx reactive ex absurd observable simplemente Shen and fundamental documentation for it and then another one combined. Take a look it up. Apple's documentation on combined framework as I mentioned before f. r. p. implementation kind of covers a lot of it as well and the last question for today number seven. What design patterns are commonly used an IOS APPS? This is sort of. Yeah I mean this is more or less fundamental question. All the developers should know that the children for this is NBC. Singleton delegates and observable observer. Those are the core so called core. Competencies design patterns used in coca coca touch applications. I already covered Singleton Nbc and delegates an observer actual basically covered all of them in other questions. Red Flag for this essentially. When you're interviewers asking this type of question in one form or another basically what they wanna hear is something besides NBC right. They want to know if you know any other designed parents against such a singleton delegate and so on that are commonly used in Iowa's APPs all besides just NBC because Mec's just so basic For everyone and the for further reading model view controller article on Apple Steph portal and them apples Dev portal actually has a another article about core competencies designed parents. The articles called coca design patterns. You can find more more about them. They're all right guys That set for today. This was part. Two of Ios interview questions for senior developers. In Twenty twenty I will. I will do more episodes and cover more questions from from the article. I also wrote a book called the Iowa Interview Guide that covers this questions and many other questions. It breaks down into groups such as you I- questions fundamental questions storage questions and so on so if you're interested checked Lincoln to show notes. Yeah if you WANNA reach us you can tweet us at inside our dev or email us at at how low at inside. Iowa's DOT COM and I'll see you next time.

NBC Apple Iowa developer Mvp Singleton COORDINATOR NBC Kiva Observation Twenty twenty Kiva Nbc Lou con Lou Alex Bush Napa Kiva labs CIO ABC
259: Continuous Deployment (dpl) with Hiro Asari

Ruby on Rails Podcast

17:34 min | 2 years ago

259: Continuous Deployment (dpl) with Hiro Asari

"You're listening to episode two hundred fifty nine of the ruby on rails podcast. And I'm your host, Rick, Mark. Hero. Sorry is a software developer Travis CI. He spoke at many conferences across the globe, including reveal djing rails top and Rubicon foster. He'll be here today to talk about will the Travis CI has called DPL. Welcome to the show hero. Thank you. Thank you. It's wonderful to have you now hero. Can you? Please. Tell us your developer origin story. No, I was a database administrator back in Madison Wisconsin wave out Gwen I had my day job. And I wrote a lot of Perl script back then. And I found that time where my creativity is not getting at a quick feed. So to speak and at the time ruby on rails got really really hot. This is groovy on rails one point two days. I don't know if the listeners remember. And Jay ruby was getting some tracks day would be before one point five at a full Rubio on support and Charles Nodar the. Maintain our of day, ruby wrote a book post about getting involved in Jay, ruby development. He wrote about how the code bases laid out hug you. Ruby methods to Jay ruby on. And I got heavily involved because I saw an opening in my creative activities. This is about this point ten ten years. And I am happy to say that I have found a job after job of writing code full-time since. That's amazing. So someone who's not particularly familiar with Jay, ruby what? Especially for you to contribute to that community Dini to have deep knowledge of both Java and ruby order to contribute. I think so. My jealous might understanding of Java is not particularly great. But it was good enough to get started on many of the aspects of ruby development and tally as well. As Tom Annable were very, very helpful and kind to get my contribution into the fold. That's asks on. Okay. So I wanna dive into a tool that you have been supporting Travis CI, which is called EPL, which is a deployments will need for continuous deployment. What inspired you and Travis CI to build maintain it? Travis CI as a continuous integration. Continuous delivery service. It is critical that we provide a wide range of deployment targets DPL is a utility written in ruby that makes it possible to do that the code is very module early constructed and can support many of the deployment targets the are quite common. These days, including getup releases Hiroko, it is also possible to write the user own script to do the deployment as well. Can you tell us a little more about the technology sack that it's built on? It is written in ruby for the most part. There are certain parts that allows us to use other tools like five ace. Cloud, foundry, etc. Because you know, ruby can make these tools available with external calls. Awesome. Well, looking over the read me for the project you support over forty different providers, including her Oku Lambda in blue mix. How are you able to support all those different varieties as we all know that they are each thrown special flower? Yes. Flower is a very very kind way to put it. I think. Aren't many headaches that goes with it because small things can change over time? And it is nearly impossible for me personally to get a stay on top of the entire array of development providers. And for that we rely on community contributions. So had there been contributors that have gotten involved in the project just by saying, hey, I'd really like you to support such and such provider. And you say, hey, we actually don't have support for them yet. But if you would be interested in writing that support than we'd be more than happy to review that code is that how a lot of contributors are getting started. Exactly. Cool. Are there any representatives from those companies that are working directly with you? Or is it really just a community effort? I would say it is mostly community effort. I do not have for example, have a direct contact with Google or. Well, I do have some contacting of yet. But those are the people who are not working on a blue mix, for example. Gotcha. Okay. That's interesting. Have you ever had a situation where one of those providers have broken DP L in some sort of way by releasing a new change or it just seems like a lot in order to keep up with a set, you know, forty different providers on an open source tool. Right. I can't think of a situation where that upland about. What can happen is? For example, when the deployment relies on ruby Jains. The gym that deal knees gets updated. And the version the co the views on DPL can be broken because of the new release, and then we would have to figure out how to support the current gem version. So that development does not break. So is DP L essentially dependency on Travis CI. It's basically a component is considered a micro service that you're maintaining an order for Travis CI as a whole to operate correctly. It is not perhaps surprisingly, it is a script that is independent of Laura service. For example, you can call deal on its own and run the deployment itself. In fact, this is how the deployment actually works. When you run it on Travis CI, my understanding is that get lab. The I for example, uses deal in its own way. I do not know the details of it. But the Dell are situations and support tickets on get hub where users are trying to figure out how to use deal on in the get love context. I cannot provide adequate support, but the it has been reported. That's that's gotta be. That's gotta make you feel good to know that your code is reaching a lot of people that are even outside of Travis CI. So does indeed have you seen a lot of humidity support around it the reason that I found out about the? Project was at ruby newsletter highlighted it a couple of weeks ago, and it seems like you've gotten a lot of stars a lot of issues. Pull requests how is the community feedback that in general? I'm really happy to hear that. There are many requests that you mentioned about adding more and more deployment targets, and if makes me feel good at the same time. I'm constantly reminding myself, I have to learn a lot I learned about you service providers, new kinds of service and soul on. I think they're four or five open PR right now in DPL that will aim to do just that. And I I want to, you know, learn all the things, but I don't have all the time. Is supporting DP L part of your core Java Travis or is that considered a side project? How does that integrate in your day to day? I would say it is a part of my job now. Very critical of our service. Gotcha. Well, you're living the dream of getting to do something that a lot of people get to enjoy your contributing to the open source community. But also helping your company as a whole signing that that's a pretty great position to be in. Thank you. So quoting from Martin Fowler. It's hard enough for software developers to write code that works on the Racine. But even when that sun, there's a long journey from their software to that's producing value since off only produces value in its in production, essentially for continuous appointments to work. Your code must always be production. Ready? Do you feel that this concept is championed or avoid in our current community? Yes. Yes. I think I think it has to be champion house. We championed why why? Because. The championing. We are not going to get. What the question is really asking is not whether or not continuous appointments that good idea. It's when you think the communities actually doing it. DPL enable to do it. But it's whether or not people actually have production ready code at the at the all the time, which I I feel they don't you think? You see how many tests fail on Travis cetera. You have that. I don't so. So so. Oftentimes pass or fail on the CI system is not exactly the indicator of whether or not the code is production. Ready? I it is just an indication of whether or not they passed. They are distinctly different. Metric so to speak. The when when the coast pass cold tests pass and goal on to deployment. It depends on how well your test. Tests are written. You know what I'm saying? I do. If you'll tests are not good. C-i-s CD will lose value. Absolutely. Your champion that concept if you can believe that code is production ready when they are not. Absolutely. Well, I mean, that's why it's so helpful to have something like DP L. When you have solid production ready code than your go. It's just having integrating that philosophy into your engineering team that, you know, your master ranch or whatever your production, which is always ready to be deployed kind can be a real mind bender for some development teams. And then does also a question about the details quality, right. Sharon it. It is not always production ready. New yet. There's weekend right tests, and we we can push through a master Bryant or whatever, and it can break things, and we have done that before. Partly because we don't we don't things like ruby gems not ruby gems. But the gym that deployment rely on can get, you know, if he for example. So what can this is due in order to help support the health of DP L, these I really do think it's a meaningful in awesome project. That's a very good question because we want. We do releases right? New releases on version x y z and they'll are things I want to test in order to Poland. Kiosks, but my development or or the range of my deployments or tests are very very limited. Because I don't know a lot about the the kind of deployment that users want to do. And I want them to test them but often I don't have enough of them. So to speak catch assing you need as any subset of users that regularly use a provider, they can almost be a Representative of. Hey, we have this new version of DPL that we wanna release, you know. My company's is blue max on having the person who tests the spate of hers, and for you know, for example. Yes. Yes. Exactly. So how can our listeners get involved in can I just go to the repo open issue or? How would you prefer? They do that. Yes. I think that is the best way to get involved in deal. It's get up dot com slash Travis. The eye with a DASS between Travis CI slash DP. L? Those are lettuce the P and L. Well, like this all up in the show. I promise. That'd be great. Yes. And Willie and PR's are the best way to get involved. If you can't event finding someone else might via that idea. How users are listeners who would be interested in actually implementing DP within the organization. How can they get started? Right. The easiest way is to use Travis CI. We have a deployment document that describes how to configure your deployment on Travis CI. Okay. As I mentioned before DPL can be used directly as a utility. And that is also described in reach me antastic asset will I will make sure to link all that as well. How can our listeners follow you hero? Twitter hero underscore Asari. I help not posted anything much lately about you might change. If we get a lot of if I get a lot of of or. That's good that that's a challenge to you listeners. But I will also link heroes get hub as well. Yes. Excellent. Okay. We'll thank you so much for joining me on the show today hero listeners. I will catch you next week. Thank you. Thank you very much.

Travis CI Jay ruby ruby newsletter Representative Rick developer djing software developer administrator Madison Wisconsin Rubicon Google Twitter Tom Annable Poland Hiroko Dini
Part VII: The Wife

Exit Scam

31:34 min | Last month

Part VII: The Wife

"This episode of exit scam is available. Everywhere you listen to podcasts. But if you want to hear the next episode right now it's available exclusively on the new odyssey app. Odyssey is your home for all the podcasts music news and sports that matter to you. That's a u. d. a. c. y. Jennifer robertson was all alone. Seven thousand miles from home was december eighth twenty eighteen and she was in jaipur india. Her husband gerald cotton had just died from complications of crohn's disease a weakened of their honeymoon. So jen left the hospital. She spent the night alone in her hotel room in the morning. She began the long trip. Home to canada with jerry's coffin She chartered a plane. Flew the casket to new delhi from. There's you took a fifteen hour flight to toronto and then finally another flight home to halifax on the way. She sent an email to the staff of the orphanage. The she and jerry had donated money to help bill. They'd planned to be there for the grand opening but now she wouldn't make this is what she wrote. And as before. This isn't jen's voice. Jerry passed away. He was a really beautiful human being. I am heartbroken but trying to stay strong for him so at the age of thirty after being married for only a few months. Jen was a widow earliest. She was acting like i'm aaron lamb. Welcome to exit scam a podcast about a mysterious death missing fortune. This is part seven. The white a lot of people think gerald cotton is still alive. They think that after stealing hundreds of millions of dollars from the customers of his bitcoin exchange. Kouadria jerry went to india and faked his death. But so far. Nobody's been able to prove there's only one person who really knows for sure what happened to jerry. That's his wife. Jan and jen says jerry is debt. She says she was with him in the hospital when he passed away she identified his body at the more and she escorted his coffin home. And despite all the questions that have been raised about. Jerry's death over the past two years. Jen hasn't changed her story. And maybe that's because she's telling the truth. Maybe jerry really is dead if so you have to feel bad for jim. Not only did she. Loser has been on the worst honeymoon of all time then. She came home and found out. He was bitcoins. Answer to bernie madoff but on the other hand. If you believe the jerry faked his death you probably have to believe john was on. His wife was with him when he died. Your have you found cases like this where the wife doesn't know. Well first of all i've done about seven hundred eight hundred Suspected fraudulent breath investigations of which more than ninety percent. Were in fact frauds. And hundreds of those. The spouse was the co-conspirator this is steven romblom. He's that private investigator who catches death fakers in this particular case got the classic situation where she was right there when he died she knows who she went into the hospital when she spread the nights in the room with him if if he faked his death The the possibility of are not being an active and willing participant is miniscule last six episodes. We've learned a lot about gerald cop and if someone asked me if he seemed like the kind of person who could fake his death and have to answer emphatically. Yes but what about jen does she. The kind of person who could help her husband fake his death and then stay quiet about it for the rest of her life denver. Robertson is kind of a mystery. We don't know much about her. She hasn't spoken any reporter since jerry's death and she declined our request to interview her for this podcast. But here's what we have been able to piece together. Jan was born in nineteen eight in halifax her given name. Jennifer kathleen margaret griffith. We don't know much about jen's early life but we do know that jerry was not her first husband she married another guy and twenty and a destination wedding in the dominican republic and she was twenty three or twenty four. She took his last name for forgery. But changed it back to griffith. After the marriage and at some point she moved to toronto. That's where she was living in november. Two thousand fourteen when she started dating jair. Jerry must have seemed like a catch. Janet pry never heard of bitcoin before but she knew he was an urdu running his own startup. It a great apartment and a nice car you loved to travel. He could even fly planes. They moved in together and twenty sixteen. Jerry bought the couple a house in a suburb of halifax called fall river. Next door to jen's mother and stepfather his name is thomas beasley. When jerry was running kouadria from home mr beasley would come over and the two would chat sometimes for hull. Afternoons allah counts. Jerry seemed like the perfect future husband and son in law but of course we know now that jerry was far from perfect. Who's moving as customers. Money off of his bitcoin exchange and into his own pockets. That's how he bought the house that he and jen lived but jan says she had no idea that any of that was happening in a sworn statement after his death. Jen said she had no involvement and no knowledge of what was happening behind the scenes at jerry's exchange but whether she knew it or not she made help jerry move dirty money but two years into their relationship. Jen changed her name from griffith to roberts. We don't know why she chose that name. But match the name of a real estate company that she and jerry registered around the same time robertson nova property management. Jerry used robertson nova to buy a dozen houses with money. He stole from his customers. He made gen the president of the company and hired her stepdad as a handyman for the properties but kept his own name off the paperwork in the two years before jerry's death he and jan lived larch jerry once bragged for one of jen's birthdays gave her a bank account with a million dollars in it. The couple traveled constantly and took international vacations often twice a month sometimes on private jets in the bio of for her instagram account which has since been deleted gen described herself as a travel addict and globetrotter. She posted pictures for more than a dozen trips. Sometimes the hashtag luxury travel the caption on a photo taken by a hotel pool in dubai. Red soup in singapore slings on my twenty eighth in twenty eighteen. Jerry and jen were legally married in canada but they flew both of their families to scotland for the wedding. They brought along a photographer. Who took dramatic shots of them in the scottish highlands. John had everything she could have ever want. Maybe she felt like giving something back because it was her who i reached out to angel house a charity that builds orphanages in india. She and jerry donated twenty thousand dollars to have an orphanage belt in a small village. Which is far as i can tell was the first time jerry ever gave to charity. They plan their honeymoon to coincide with the grand opening. But i three days before they left. Jerry signed a will the may gen the beneficiary of everything. He owned the houses the cars. His yacht is plain and quadri so the bitcoin exchange jerry had robbed and it soon to be furious. Customers were about to become jen's problem. Wanna tell you about our sponsor this week case fleet fleet make that's really useful for investigations of every kind whether you're a storyteller or a lawyer or a private investigator. I actually used it while i was putting together this show. Their software allows you to make a detailed chronology of events. Which if you've been listening up to this point you probably understand why i needed one. But not only. Does this chronology hall dates. It can pull dates directly out of pdf's even ones that use fuzzy dates so basically you're able to upload a bunch of documents and then refer to them when you need to on a time line it helps you. Connect the dots and find the hidden story within the story. If you're ready to give case fleet you can get a fourteen day free trial as a listener of this show at case fleet dot com slash exit. Scam even get ten percent off your first subscription. Thanks case fleet for helping me get organized and thank you for sponsoring the show. We all spend some portion of our lives on the internet and most the time decisions we make online are routine but sometimes a single click can change your life forever. I'm all fanning and welcome to season one of one click. I've teamed up with journalist. Jessica wapner to investigate a lethal diet. Drug that has managed to reappear time and again it's a story about how the all consuming wish to be lean has caused people to ingest a chemical that was once used to make bombs and world war one now it's an illicit diet drug that's flourishing online preying on people whose only barrier between life and excruciating death can be shattered with one click. Listen and follow one. Click a presentation of c. Thirteen originals a cadence. Thirteen studio in association with the spooky and the well in pictures available now on apple podcasts. Spotify or wherever you get your podcasts. After jerry died or allegedly died. Jen didn't have much time degreef. She had to keep his bitcoin exchange running. And even though jen barely understood. What a bitcoin exchange was. She knew she needed to find the passwords to quadri. Cold storage wallets. Which as far as she was aware still contain two hundred and fifteen million dollars so as soon as she got home from india. She let us search of her and jerry's house. The search party included her mother and stepfather. Who lived next door. There was also jerry's family. They flew in as soon as they heard he died. Jerry's parents bruce and cheryl cotton made it to halifax a day before gen and spent the night and jerry jen's fall river house as jerry's brother and his wife they were all there win jiang got home and there was alexander. Heynen quadri kaz lead programmer. Who came all the way from the uk. The got on the plane immediately and arrived shortly after gen. The group looked everywhere the bedrooms closets the basement attic jerry's home office and the pool house. They even cut open the furniture. Jen also gave alex jerry's laptops and phones plus access to his two factor authentication app to see if he could find anything but jerry was known store. His passwords using paper wallets. Fancy way to say he wrote down. The private keys to his wallets on paper even talked about an interview. The paper will have is a great way to store your bitcoins. So the best way to do is take your private key printed off store. Offline your safety deposit box ball to whatever and then take the public. Which is your address and use that to send money to it so that way you can never have your bitcoin stolen unless someone breaks into the back steals hearsay gets into your private key and so forth but jerry just said is if you use a paper wallet the only way someone can steal your crypto as if they physically break into where you keep that paper their private keys written on it and take an jerry's case we know where he probably would have stored that paper as i told you a couple of episodes ago. Jerry liked to keep his valuables locked in the safe in his attic. The question was in the that's what he mentions. it's a small saif. It's the kind of sites that you have in a hotel you know. It's not like the the big sites that we pitch going into movies that sits on the war and wideband trade towns this little saif. This is a little safety just have documents scene was bolted to the rafters that also. This is the anonymous quadri. Customer turned detective. Who calls himself. Qci aunt he's the one who uncovered gerry's teen-aged ponzi schemes. And he agreed to talk to us as long as we didn't reveal his identity. Where was this safe found. After jerry died will my understanding. This safe was found on the floor of the office. It'd being taken down from the rafters added being unbolted and it was on the floor of the office and From honest any was empty. Now here's where things get murky. We don't know who move jerry safe from the attic to the office. We don't know opened it and we don't know if there was anything inside but if there was someone took it it could have been jerry himself. Maybe he took whatever was inside to india with him. But maybe someone else robbed the safe because while officially everybody was searching the house for the password to the cold storage wallets. Some members of the search party may have been looking for something else. It seems pretty clear that at least at some point That was that was the saif that he used for his own. Personal october's these referred to as the nest egg. That's jerry's nasdaq. Remember when i told you about jerry's personal cryptocurrency investments. He bought a large amount of syria during the presale when the price was thirty one cents. Plus jerry probably also had some bitcoin from back in the day. But like the private keys for quadri. Cold-storage wallets the nest. Egg was never found or never found that we know of well. That was a that was an interesting thing is that his office was completely not what i expected. This is chris bryant. he's the cybersecurity specialist. We heard from back in episode. One gen hired him to find jerry's missing passwords so he interviewed a couple of the people involved with the search visited the house himself a couple of weeks later. What i learned was in our investigation. Was that someone had gone into the office and cleaned it up and tidied everything up Whereas when they left to go to. India alaska anybody. Seen that office with the you know. Papers and everything strewn everywhere sh- piled up the papers and files and and everything else but when they returned home after a couple of days i guess family members had gotten access to the office and they had cleaned his office off. What christie said. Was that when jerry gen left on their honeymoon. Jerry's office had been a mess is desk. Drawers were overflowing and there. Were papers everywhere. Jerry was a slop but when jen got home after jerry's death. She found that the office had been cleaned up. And a lot of stuff was gone though. Not everything you know. We can't even count. How many Gambling cards like Casino cards from memberships casinos and from casinos. All over the world. Some the most famous because gonna world there according to gen the only ones who had been in the house in between or jerry's family. His parents his brother and sister-in-law. It's hard to know exactly what to make of this. Maybe jerry's parents rated his office and popped his safe while was on her way back from india. Jerry's family isn't talking. They ignored our requests for interviews. So we don't know who cleaned out the office move the safe or whether they found anything maybe we never will. Jerry's funeral was held on the morning of friday december fourteenth. It was mostly for family and close friends but jen also invited quadra rigas freelance contractors and even though most of them had never met jerry in person. Four of them showed up in addition to alex. Heynen there were a couple of junior contractors plus aaron matthews the quadri guy customer service rep. You might remember from a couple of episodes ago. He leaked his chats with jerry. He drove in from new brunswick for the funeral. We spoke to one of the contractors. Who agreed to tell us what he remembered he said. Jerry's funeral was closed. Casket jen and her stepfather. Jerry's parents and his brother all gave eulogies and there was a smaller burial ceremony mostly for relatives under the circumstances it was a pretty normal funeral but later that day. Something strange happened. The contractors. Alex and erin and the other two were hanging around their hotel. When jerry's parents suddenly showed up the cons were upset and the contractors asked. Why from what i've been able to piece together. A group went back to the fall river house after the funeral an included jerry's parents and Jennifer and some of her friends and apparently there was there was a thought it was fought between jennifer and and jerry's parents jerry's parents told the contractors that as a result of this altercation with jen. She threw them out of her house. So they packed up their luggage and then departed for the nearest hotel. Which is weird. I ran into some of the quadri employees including alex hannan. And and that's how we kind of come to know you know this. This part of the story is because they ever heard what was relied to them at the hotel. According to our sources gen and the cottons stopped speaking. After that night. I called that number that showed up on my own. And when the woman answered she said yes my name is jennifer robertson This is denise sibley. She manages a tiny airport. Outside of halifax it's where jerry kept his airplane. Although that isn't her main gicc. I am radio host or do you wanna five in halifax. What kind of music do you play hazier. Okay your share of the many many people we asked. Denise is the only person who's ever spoken to jen and would agree to talk about her on tape but even she dealt with jerry are plants. I knew we had a fancy plane. I knew he was into something to do with crypto currency. He told me that he was going away to india to start an orphanage which i thought was quite impressive but then that sort of jimmy that kind of fell into place with everything that in my little head i was thinking. Oh this guy's got money he's giving back. What how noble you know. The news wished him good luck but then a couple of weeks later there was a storm coming in and she needed jerry's permission. The move is plain out of harm's way. So she called him. And jen picked up right away her name jennifer robertson and. That's when because your name wasn't caught. I would have assumed there was no relation. Just you know. And i said Are you jerry. And i didn't know she's wife. And then i'm like oh how's the orphanage going. Have paid it and she goes. I tell you something if you promised me that you won't tell. Share the information with anybody else course and She said that. Jerry passed away That they weren't he. I you know Opening this orphanage and they were on their honeymoon and then he passed his pass away. She sounded very distraught. Then i'm of course. I i'm like oh my god. I thought his tried to what to say. No she's founded choked up. And i had to promise that i wasn't anybody when when she asked you not to tell anyone about gerald cotton's death did she say why no she kitten I i i think. I guess she probably off the little bit by just. I'm dealing with a lot. And i do remember her saying you know he passed away. He left me with a phone. He left me with You know an airplane is about an airplane like can you help me sell it. Denise agreed to help gen sell jerry's plane but later after the shock of the phone call war off. She's started to get suspicious. Somebody get that. I'm not supposed to tell anybody about but this is just a nasty divorce. And she selling a plano. From of i said i don't know this guy i honestly said i don't know if this guy money to people i don't know if he's wasted a scam artists and she's not really still married to and she's got sell the plano from like i had all these crazed kooky scenario in my head. I watched too much. I i think did you really think he was dead. Honestly i was really suspicious. I did not believe he was dead. There was a reason. Jan didn't want people to know. Jerry was dead right away. It was because kouadria which she was now in charge of was still open for business and still accepting new deposits at you all the way back in episode one gen waited a whole month to make jerry's death public. That sounds sketch and it was sketch but jen said she was only following advice from alex. Heynen quadri goes lawyers. She said they wanted to keep jerry's death a secret until they secured the customers money to avoid a panic but of course there was no money to secure and finding the passwords wouldn't have made any difference. The search had been a wild goose chase. So finally jen announced jerry's death she shut down the exchange and on the lawyer's advice applied for creditor protection. Which is a kind of bankruptcy. It's a court order to prevent customers from sil. When those customers found out that jerry had been dead for a month and their money was gone and they couldn't even sue. They got angry and with no other outlet for their frustration. They focused on jan. Some accused her of helping jerry fake his death others called her a black widow and said she'd murdered jerry to get her inheritance even sent her death. Threats got so bad that she deleted her social media wanting to hide. In an affidavit filed with accord cons widow said she had received online threats. And what she called slanderous comments including questions about the nature of her husband's death and whether he is really dead a few months after jerry's alleged death we learned that quadras money wasn't just missing. Jared stolen used some of that money to buy houses a plane in a yacht physically. All the stuff you gave jannine as well so under mounting pressure jen reached a settlement with quadri kaz customers. She agreed to turn over everything. Jerry left her plus everything the couple owned in her name. R almost everything. She was allowed to keep the ninety thousand dollars in her personal checking account twenty fifteen jeep cherokee and her wedding ring the court. Filing there is a statement from gem again. This isn't really her voice. As a result of the monitor's investigation. I have agreed to return quits regis x. assets that i had previously thought were purchased with jerry's legitimately earned profits dividends and salary. I was upset and disappointed with jerry's activities. When i first learned of them and i continue to be as we conclude the settlement. This settlement will allow me to move on with the next chapter of my life is part of the deal. All of jerry. Jones houses were put up for sale. The help payback quadri customers. That included the house. They lived together and halifax it was listed on a bunch of canadian real-estate websites and looks like a pretty average suburban home with a pool in the backyard in the photos. The house was completely empty except for one corner of the basement. Where gen left behind a makeshift. Jim folded up treadmill and on a nearby wall. Someone had painted a motivational slogan. That said you don't get what you want. You get what you work for on the next episode of exit scam. Have you ever heard of the body getting into doomed in a case like this and more often than not. There's there's nothing in the cough. I did a fake case in mongolia. Lots of rocks in that particular case to two hundred pound of rocks. You know more more often than not. It's an empty coffin. But in the cases where there has been somebody in the coffin and they've taken dna Copy asleep did. Jake is up as a former fugitive hunter. First of all. I wanna stylish whether he's alive or dead. Exit scam is in original production of treats media. You can get next week's episode right now on the odyssey app. That's a u. d. a. c. y. Has all the podcasts and music news and sports that matter to you download it for free today from the app store or google play. Exit scam was written and produced by me. Aaron lamour and lane brown mixing and additional editing by martin fowler. Show art an art direction by mickey dj. The theme song is by francis and the lights and scoring from this episode was by mark allen piccolo ross seem anini martin fowler francis starlight and myself with additional music from epidemic sound and blue dot sessions are executive producers. Are max linski lane brown and myself for treats media additional sound editing and producing by james nicholson additional producing by jacqueline scurry very special. Thanks to the person who did a jen's voice in this episode. She didn't want her name used special. Thanks to jake. Schreier and garett had greco jae kang. Evan ratliff everyone at cadence thirteen. Thank you lizzie. Dente han everyone at odyssey. Thank you. Jd rally thanks to everyone at pineapple street studios and anyone. I've forgotten. thank you last episode coming next week.

jerry jen Jerry quadri gerald cotton Jen halifax jennifer robertson india kouadria crohn's disease aaron lamb Kouadria jerry steven romblom gerald cop Jennifer kathleen margaret gri Janet pry thomas beasley mr beasley
Fri. 03/12 - Netflix Is Cracking Down On Password Sharing

Techmeme Ride Home

23:11 min | 4 months ago

Fri. 03/12 - Netflix Is Cracking Down On Password Sharing

"Welcome to the tech at home for friday march. Twelve twenty twenty one bryan mccullough today. Net flicks is cracking down on password. Sharing apple sues a former employee for allegedly stealing trade secrets. Masa montesano is back baby. I guess we have to watch ads on our fifteen hundred dollars smart. Tv's now and of course. The weekend long reads suggestions. Here's what you missed today. In the world of tech the free ride is over and seemingly. Half of america is going to have to awkwardly cutoff. That boyfriend or girlfriend. They haven't spoken to in five years because netflix's apparently running a test that cracks down on password sharing prompting users to get their own account. If they don't live with the owner of an account cutting the struggle in the prompt customers are told that if you don't live with the owner of this account you need your own account to keep watching in order to continue. They need to verify the account with an email or text code or create a new account within a thirty day free trial. We've heard the test right. Now is only on tv devices. A net flicks. Most person told the stream abo- quote. This test is designed to help ensure that people using netflix accounts are authorized to do so. It isn't clear if users in the tests all need to be on the same. Ip address to be considered in the same household according to net flix terms and account can only be shared with members of your household. Quote the netflix service and any content viewed through our service are for your personal noncommercial use only and may not be shared with individuals beyond your household and quote until now. Netflix says not done anything to police. This except set limits on simultaneous streaming. Their basic plan allows streaming on a single device while the standard plan offers streaming on up to two devices and the premium plan on up to four devices. However they don't limit you on the number of devices a single account can be logged into. There has been talk. That companies will become more aggressive on password sharing as the industry becomes more mature and quote. Yeah we've known for years. That netflix has been aware for years. Just how much password sharing was actually going on. Don't think you were being that clever. So the interesting thing now is why. Now why is this the time that netflix's decides to do something about all this. Although some research suggests that the number of people digitally squatting as it were on other people's netflix account could be as high as a third of the net flicks user base. So you know that's not an insignificant number potentially tens of millions of users. Netflix could suddenly monetize for the very first time. Apple is suing its former materials lead simon lancaster for allegedly stealing trade secrets and leaking them to a tech reporter. Quoting nine to five mac. As first reported by apple insider the court documents detailing the lawsuit was made public on thursday. Lancaster allegedly used his senior role at apple to gain access to internal meetings and documents outside the scope end quote of his job as materials lead at apple in fact on his last day at apple. November i twenty nine thousand nine. He allegedly downloaded a substantial number of confidential apple documents. Lancaster departed apple after a decade to join the materials research and development company heiress which is actually an apple vendor. The trade secrets he stole from apple continued to benefit him in his new position. At arras the report explains lancaster described his role at arras as scratching a startup. Each but things get even more interesting when the media aspect of this story comes into play. The lawsuit explains that the reporter. I contacted lancaster. in november. Two thousand eighteen year before he departed apple apple. Trade secrets were allegedly exchanged throughout two thousand eighteen and twenty nine thousand nine the anonymous media correspondent who use the information in multiple articles and attributed the details to a source at apple. Details about the topics are unclear but lancaster attended multiple meetings about project x at apple and shared all of that information with the media correspondent and quote. So nobody knows who the reporter is who got this information though. Believe me those of us who traffic in these circles have spent all night gossiping about who we think it might be and were all assuming that project. X is referring to those ar and vr glasses being developed based on what we know about what lancaster worked on for apple. But all that friend of the show. Ed citron's sum up this whole thing from his twitter thread last night. Quote mad respect for the guy who stole a bunch of apple trade secrets and texted a reporter them from his company. Phone with stuff. Like here's the secret stuff. And i'm mad. Apple is secretly developing a competitor to one of my investors. When the guy in question resigned he texted the reporter about some secret documents. And the reporter said oh. Can you get me some secret documents. And the guy said sure which ones this guy rocks because he logged into apple's network after he left on his final day and downloaded a bunch of stuff. You know. I think apple might notice that dude. This guy also literally asked for positive coverage of his investments to in return for the secrets. Like did he not know. Apple had lawyers and quote kupang. The south korean e commerce company raised its ipo price yesterday above its range and then went. Public popping above the initial trading point giving the company initially a market valuation of about sixty billion dollars. But when all was said done kupang was worth over a hundred billion dollars and while this is yet another story of a tech company reaching public markets to great success. At the moment. i probably wouldn't have covered this except for one. Little detail quoting from bloomberg. Japanese conglomerate softbank kupang biggest shareholder has reaped a gain of more than sixteen billion dollars from the ipo burnishing the reputation of founder masayoshi sun in picking successful startups. Even after a number of missteps in november two thousand eighteen softbank's vision fund invested. Two billion dollars in the company in a deal that valued kupang nine billion dollars people familiar with the matter said at the time that funding followed one billion dollars from softbank itself. In two thousand fifteen valuing the startup about five billion dollars and quote so again after its pop in softbank's stake now represents a thirty three billion dollar windfall as we covered mussons stumbles with softbank. Last year with we work with wag with uber. That didn't pan out. As maybe massa hoped i guess we owe it to them to acknowledge that things have turned around recently. Softbank has done very well from its stakes in the like of door dash very well to the tune of eleven point. Two billion dollars and even uber has come back to the tune of around ten billion dollars. Give or take grab which softbank was very early on is talking about doing spec soon. So reminder folks investing is a hits business one or two big successes can make up for a lot of disappointment. Why have there been so many stories about spamming. Ads invading places. We don't want them are telecom. Companies selling are browsing gated advertisers. Google leveraging are spending to give us quote unquote special offers. And now. Chris welsh at the verge looks at the recent trend of ads showing up in your smart tv quote now. I'm fully aware that it's not unusual to see ads placed around the tv's home screen or main menu l. g. samsung roku video and others are all on this game. We live in an era when smart tvs can automatically recognize what you're watching and tv makers are building nice. Add businesses for themselves with all of the data that gets funneled in this stuff can come off as invasive but it's also partially steadily brought the prices down on even high. Mtv's i got this fifty five inch cx on sale for like fourteen hundred dollars and it's pretty much the best tv on the market for next gen gaming but even if this beautiful panel came cheaper than it might have without ads plastered in random places. The level of infiltration on display here is still disheartening to see. Lg recently announced it will be licensing web os to other tv brand so maybe the company is trying to see how far it can push things a random full on commercial just popping up in l. App store is there no escape from the stuff. We're just gonna cram adds into every corner of atv software Imagine if an auto play add started up while you were updating the apps in your smartphone and quote to which i'd say never say never i mean. Have you noticed that apple helpfully invites you to sign up to their various services inside other apps and even inside the settings app. What is that. If not an advertisement i told you about editor ex not too long ago. The design platform built specifically for designers and agencies where you can create complex sites while feeling like you're working on a visual design software platform editor. Ex is a wide workspace. Where you get total control powered by smooth drag and drop. The canvas is so intuitive it feels like a physical space where you can move things around as you want. You can create custom breakpoints to build unique web experiences every screen size. You'll get total. Accuracy of a responsive sizing units and docking to give developer level control than designers world combined with all of this editor x has a massive range of integrated business solutions. So your sights will not only be works of art. They'll serve the needs of any type of business now picture. A hassle-free client handover with editor xs code free. Cms your clients can update site content behind the scenes without messing up. Your design finally imagine inefficient workflow and collaboration process with editor xs. Recently launched sharable design libraries plus other exciting new features coming soon. You and your team can work more effectively together. I can talk about this all day but you should really go see it for yourself at editor x dot com and discover the new standard in web design editor x dot com if building software is your passion. Then you're gonna love thought works technology. Podcast it's a podcast. Four by techies. Their team of experience technologists a deep dive into a tech topic. That's peak their interest. It could be how machine learning is being used in astrophysics or maybe how to succeed at continuous delivery. They're always coming across fascinating ways. Technology is advancing and love to share what they learn. Whatever the topic. The discussions are always lively informative and opinionated. The team of co-hosts are experienced technologists from across thought works and include dot work dr rebecca persons and renowned writer and speaker. Neil ford past guests have included eminent technologists such as martin fowler mark richards and danah boyd. The episode i just listened to wasn't explainer on extended reality which is a blending of ar vr. Sort of along the lines of what we've been talking about on this show recently. So whether you went to broaden your knowledge on a specific topic or just want to immerse yourself in the world of tech thought works technology. Podcast has something for you to find out more just search for works technology on your podcast platform of choice and make sure you subscribe time for the weekend. Long reads suggestions and i up a couple of interviews. That people have been discussing all this week. The i was in an interview. That friend of the show. Noah smith who you'll recall. We did a recent weekend bonus episode with an interview. He did with stripe. co patrick. Collison patrick has a reputation for being a prodigious intellect. He founded stripe when he was twenty. Two years old. You'll recall but this interview which since no is an economist and is therefore interested in any number of big issues. Ranges among a number of fascinating topics really highlights. Why patrick has this reputation for being a brainiac. Like dude knows a lot about a lot. For example in the middle of this interview packages lays down the best description of the innovation roadmap for the entire tech industry over the next ten years that i've heard anywhere quote in terms of what the world needs. Improvements in medical. Technology are probably still number one. Climate change mitigation technology cleaner energy generation and co two sequestration and so on also quite high up more broadly. We need to make all of the things that you and i enjoy every day. Cheap inefficient enough for billions more people to afford with safety insecurity high on that list but need is a tough framing. There's obviously so much stuff that would be fabulously valuable. And it's hard to predict the magnitude of the impact upfront. Besides the obvious diseases better. Cures for depression and mental illness and other psychiatric conditions would be hugely beneficial hundred dollar robotic surgeries a machine for cheaply manufacturing food a three d. printer for nourishment into which you just insert elemental ink cartridges and not just for replicating already existing foods. The possible design space is very large flying cars. Obviously plus space-based earth to earth transportation. Fast growing trees. So that everywhere can be as blissfully are boreal as you like technology for comprehensively eliminating air pollution not just from internal combustion engines but also sand dust ubiquitous detectors for toxins like lead arsenic and benzene. Smart books that are better fit for purpose. A babel fish that works programming environments that are less hopefully primitive than those today. Take mathematica squeak genera and go far beyond them. Better education technology for everyone. What's khan academy but ten times better too cheap to meter water desalination batteries with so much energy density that they need never be recharged nanotechnology self repairing would flexible glass. Translucent steel quantum computers that accurately simulate physical chemistry completely new kinds of matter better catalysts for all major existing chemical processes and quote. so basically. There you have it. Make a successful company out of any of those areas mentioned and you can probably have a trillion dollar company and a couple of decades. These are the things. Technologist will be working on for the next few years laid out for you for free and then over at the verge neil. I spoke to cave on bake poor. Twitter's head of consumer product about all the things that twitter has been doing lately and also why they're suddenly doing all the things all the sudden. There are a lot of tea leaves to read in this one. Like i noticed how kevin was careful to tip toe pass any criticism of apple's tax were in an environment now where when someone elects not to pick a fight. It's probably more because it makes you wonder why couldn't apple acquisition of twitter be possible in this regulatory environment just leaving that there for now anyway quote fast forward to about a year ago we really started investing in audio and thinking about how we can enable audio as a new form factor for conversations on twitter. The same team. That's driving spaces. Today really started focusing on that. The first product that they built was what we call voice tweets which we put in market late last year on ios. Lets you record your voice in tweeted out. Basically around the same time they were thinking about the sort of conversational experience and this is when audio release started heating up and clubhouse was getting a lot of traction so we had a long and winding road to refocus back on that sort of multi person conversational format for audio but the team. That's building spaces. Now has had their heart in this for quite some time. Obviously hindsight is twenty twenty. We found much more customer. Success and impact and excitement around the multi party experienced than we did with the voice tweets experience. We still see a lot of really awesome use cases there but we've all of our focus to spaces now and quote next. I've said a bunch of times recently. How the whole no code low code movement takes inspiration from excel and how it trained. Millions of normies essentially the program without knowing they were programming. Well not boring. Takes a look at the spreadsheet that launched a million companies quote excel may be the most influential software ever built. It's a canonical example of steve jobs. Bicycle of the mind endowing. Its users with computational superpowers normally reserved for professional software engineers armed with those superpowers users can create fully functional software programs in the form of a humble spreadsheet to solve problems in a seemingly limitless number of domains. These programs often serve as high fidelity prototypes of domain specific applications. Just begging to be brought to market in a more polished form. Hundreds of b2b's startups have been built by taking a job. Currently being done in excel and trying to accomplish the job in more optimized purpose built to software every time you hear an entrepreneur say we're replacing silence spreadsheets and outdated processes with purpose built software. You're hearing the unbundling of excel in real time. Many popular sas applications fall into this category and yet despite being unbundled excel keeps getting stronger. That resiliency has inspired entrepreneurs to look more deeply at what makes excel tick and why adventurous builders are creating new software. That doesn't unbundle excel but is inspired by excel excels balance of usability and flexibility can be found in popular no code and low code products created over three decades since accel. I grace screen. The source of inspiration is less direct and more meta it is less about getting anything. Concrete that happens in excel and more about capturing the essence of what makes excel so successful. And i didn't do this as a full-fledged segment because i'm not convinced that there's really new news here but everyone has been discussing this. Mit technology review piece about how facebook allegedly sidelined ai. Experts and weakened initiatives internally to clean up misinformation because leadership didn't want to hurt mark zuckerberg's desire for growth. I'm linking to it though so that you're up on the conversation and i think that gideon litchfield decent summation of the peace quote the narrative. Facebook has been pushing over. These last years is that limiting misinformation and hate speech for billions of pieces of content. Posted daily is an incredibly hard technical challenge. That it's best minds are working tirelessly to solve. What karen. how figured out over months of reporting is that. This narrative is both true and false people at facebook like j. kenya. Narrow are indeed brilliant sensitive hardworking and deeply committed to doing the right thing but facebook itself cannot let them do it and the reason facebook cannot let people do the right thing and limit toxic content. Is that facebook is addicted to toxic content and its leadership is unwilling to make the one change that would allow it to kick the habit. Facebook has built content moderation systems to try to filter out the worst misinformation and hate but those systems are in a losing battle with the content recommendation. Algorithms that promote the stuff that doesn't get caught by the filters and quote and this month. We might talk about this more in coming months the one startup. That was new to me that has pushed its way onto my radar as potentially the next big thing or the latest. Next big thing is billy billy. What is billy billy. Well see. i didn't know either. So here's a great explainer intro piece. From the south china morning post. How chinese site dedicated to the anime subculture grew up with it's gen z users to become a mainstream success quote since launching in two thousand nine. Billy billy has grown into one of the most recognizable names in video streaming in china. It now fifty. Four million daily active users who spend an average of around seventy five minutes on the platform each day. Most of them are from. What billy billy calls. Gen z plus which the company to find those people who were born between nineteen eighty five and two thousand nine and many grew up with the site by the fourth quarter last year. Billy billy said it had one point nine million content creators who were contributing more than five point nine million videos each month some longtime users like qiu. Chang consider themselves part of a community made up of uploaded and their fans. She checks billy billy daily to see her favourite up have posted new content and quote and finally today a piece from wired that. I'm directing at whomever needs to hear it sometimes. It's okay to give up just because you devoted one hundred hours of your life to a given video game doesn't mean you have to finish it especially if it now bores you quote ultimately. I know i'm only playing this game. Because i've already played it for one hundred hours and giving up at this point feels tantamount to wasting four point one six seven days of my life. It's like avoiding breaking up with someone simply because you've already been dating for a year and they met your family and i no longer look forward to this game but if i stop now. What sense of accomplishment will i have. It becomes even more complicated when you think of the concept of game chores which anyone who's been playing animal crossing new horizons for the last year. We'll tell you as a whole thing. I can't begin to explain to you. How many times. I've opened that game just to check in at stores talk to villagers get my miles for the day and log off for weeks on end. It's the only interaction i had with the game. I wasn't playing or deriving entertainment just doing chores in my video game. I forgot to log in one day. Broke my nook mile streak and that's the very boring story of how i stopped playing animal crossing and quote no weekend bonus episode this weekend at all. Sorry i just need one weekend off. But also i'm gonna use this weekend to finalise the writing of the first elon. Musk miniseries episode. Never fear especially you ride home. Plus subscribers the next interesting raise episodes in the works for next weekend and for the weekend after that. We're gonna have the next office hours episode. This time with chris froehlich of first round capital. He's the guy that led first round investment in roablocks. So we're definitely going to hear some of the story behind that. But also chris. And i have been friendly for many years. He's the guy who. I got me involved with ted some years ago so i know for a fact. He has some interesting insights into investing and entrepreneurial success plus as all reveal. Dude has a full on gadget museum in his house. You'll hear i'll explain. Also we've got another regular bonus episode interview in the pipeline as well talked. Y'all on monday.

apple netflix softbank lancaster bryan mccullough Masa montesano simon lancaster Ed citron Lancaster masayoshi sun Chris welsh
Part II: The Co-Founder

Exit Scam

27:08 min | 2 months ago

Part II: The Co-Founder

"This episode of exit scam is available. Everywhere you listen to podcasts. But if you want to hear the next episode right now it's available exclusively on the new odyssey app. Odyssey is your home for all the podcast music news and sports that matter to you. That's a u. d. a. c. Why i'm twenty five and let me paint the picture. Mr kaden looks probably exactly what you think energy. This would look like. Are we hanging out with mark zuckerberg right now held. Unfortunately not the audio you're hearing comes from podcast. Called true bromance was hosted by a couple of guys from vancouver who usually just interviewed their friends serve. Anybody was doing something interesting around town. In march of twenty fourteen. Their guest was a twenty five year old entrepreneur by the name of gerald cotton who just a few months earlier had launched a bitcoin exchange our exchange launched on december twenty six. That's quite. That's quite drink in the last episode i told you about jerry's death or his alleged death in india and twenty eighteen. I told you that before he died. Jerry was running quadra riga all by himself from his personal laptop. Thou told you about his customers loss. Two hundred fifteen million dollars when the exchange folded. But that was the end of quadri in this episode. Ona tell you about the beginning. I'm erin lamour welcomed exit scam podcast about a mysterious death a missing fortune. This is part to the co founder. The see all of canada's biggest crypto currency exchanges died when he was on a trip to india neighbors colonial street where gerald cotton known to. Home are also stunned. He was the ceo of card. Riga if you wanna buy coins. I would recommend going to my site. Obviously so quadri x dot com complicated dreavers early days. The company wasn't just run off of jerry's mac. Book is a modest startup with an office in vancouver's gas town neighborhood and a small team of employees. How many employees do you have a clutch right now. Right now we have five over me. And then like programmers added other people who help out with things like compliance. Jerry also had a co founder. You'll notice he's not very excited to talk about him. And how many partners do you have in this in owning a business one partner. Oh who is basically. We shared the business. Okay and There's the employee is employee as the jerry is. Anonymous friend had good timing when they founded kouadria. Bitcoin wasn't quite new but the price just gone above a thousand bucks for the first time. Some early investors gotten rich so regular people were starting to become interested. The typical bitcoin user is probably not what you're imagining. a. I mean with my business. I've met a lot of my clients and it ranges from eighteen year. old college. Kids all the way up to seventy five year. Old grandma's right. We think it's cool to buy headquarters. Yeah but for jerry. Bitcoin was more than just something to invest in a whole new mindset lose. It's interesting because bitcoin itself is not entity. so it's not like we're actually promoting business per se. We're kind of promoting new thinking of how money works instead of using traditional currency which is basically an antiquity take so to enlighten canadians on this new better kind of money and to hopefully drum up some business while they were at it quadrupled sponsored logo. Bitcoin conventions and jerry became one of the directors the vancouver bitcoin co op which is basically a nonprofit that works to promote bitcoin every monday night actually ever bitcoin coop. Meeting in our office in gaston. Yeah so we've had times when fifty people show up and everyone's just interested in bitcoin. All were welcome. And while you were there jerry would sell you. Bitcoin on the spot. Quadri guy also announced plans to install bitcoin. Atm's businesses around vancouver. Bitcoin vending machine in the way. Go up put some cash. You scan your qr code which is basically for your bitcoin. Wallet and within a few seconds to have the coins in your wallet cool. It was a great video that still online of jerry. Showing a friend's young daughter how to use one of quadri as bitcoin. atm's okay jerry's social world revolved around. Bitcoin i met jerry and In a a bitcoin meet up. This is michael parkland. He's a bitcoin security expert back then This must have been twenty thirteen or two thousand fourteen being a regular in the bitcoin seen in toronto back. Then i i got to know. Most of the people and jerry was was a new face and what perked my ears up was. He was telling another group of people at this meet up about his bitcoin exchange and that struck me as interesting because at the time there were only those was only really one Bitcoin exchange so i gravitated towards that conversation introduced myself and That that's how we met. Were your early impressions of him. My early impressions of jerry were. He was careful reserved a professional and Humble he had a way of talking where it was clear he was thinking about each of his words before even he was saying them. Something that you typically see with like a politician or something like that He definitely knew enough about bitcoin that he wasn't like a marketing guy. Who's just repeating the bitcoin headlines. Jerry made a good spokesperson for quadri. This is a couple of years out of business school and he looked even younger than he was with wavy reddish blonde hair glasses is a bit socially awkward but he had a strange charisma. People like jerry. He reminded them of something. Hanging out with mark zuckerberg right now is unfortunately not jerry was. What a normal person thought a tech genius was supposed to be like somebody who could change the world's or at least get rich. Jerry made people feel lake if they bought bitcoin from him. Maybe they could get rich too but a year after launch. Jerry co workers weren't making much money yet. In fact they were losing between november. Twenty fourteen in january twenty fifteen poetry had operating costs of ninety thousand dollars on a revenue of only twenty. Two thousand dollars. They were basically doing. The business of an unsuccessful subway franchise bitcoin exchanges like riga earn only a small percentage when somebody buys or sells bitcoin enough. People were doing that yet but soon. Kouadria caught a lucky break. Their main competitor went out of business. See a vertex which until then had been the top. Bitcoin exchange in canada announced suddenly that it had been hacked. Not shutting down so practically overnight kouadria jumped to first place. Jerry happened to be aghast at a conference in toronto. The we it happen so you know. There's a reason. gerald y. You are now canada's largest bitcoin exchange. Do you care to tell the story. What happened to the former largest bitcoin exchange they just cited a variety of reasons banking troubles legal troubles but as a result of them shutting down. It's been great for me jerry. And his team were barely getting started. Now ready they had the biggest bitcoin exchanges in the country. Now they just needed to figure out how to turn a profit wanna to tell you about our sponsor this week case fleet case fleet makes software. That's really useful for investigations of every kind whether you're a storyteller or a lawyer or a private investigator i actually used it while i was putting together this show. Their software allows you to make a detailed chronology of events. Which if you've been listening up to this point you probably understand why needed one but not only does this chronology hall dates. It can pull dates directly out of. Pdf's even ones that used fuzzy dates. So basically you're able to upload a bunch of documents and then refer to them when you need to on a time line it helps you. Connect the dots and find the hidden story within the story. If you're ready to give case fleet a try you can get a fourteen day free trial as a listener of this show at case fleet dot com slash exit. Scam even get ten percent off your first subscription. Thanks case fleet for helping me get organized and thank you for sponsoring the show. Hey there this is. Aaron lamour the host of exit scam. I'm excited to share with you. A brand new podcast from pineapple street studios called my fugitive throughout the series host needed. Cv works to answer a question that haunted her family for decades. Whatever happened to howard mechanic. Howard was a college student in saint louis when he vanished after a riot left. A federal building burned to the ground. Meena's dad was the civil rights attorney who represented howard on our hunt for answers. She came across a cold war. Spiring conspiracy to assassinate a civil rights icon black bag jobs against activists whose lives were changed forever and the government's attempts to cover it. All new episodes of my fugitive are available every week. Wherever you get your podcasts. But why wait you can binge all the episodes now exclusively on the new odyssey app. That's a u. d. a. c. y. o. Onnor remembered jerry's mysterious co-founder wanna talk about in that interview. And how many partners to have in this one partner. Basically we share the business and yeah. There's the employees employers like the an employee's jerry won't say his name so i will his co founder. Was a guy named michael patron. Jerry wasn't just evasive about mike in interviews. He also kept mike. Name out of quadri rigas early public documents and i'll press releases. Jerry alone was listed as quadri as founder. Ceo and director. But kouadria was mike's company just as much as it was jerry's and mike wasn't keen on keeping himself hidden as jerry was. After a couple of years in business mike gave an interview about quadri. Got to a group from a co working space called decentral vancouver from google cuny such as good million with the district companies out there around why today sure well mid two thousand thirteen when decided to get more involved with bitcoin. The only exchange of notes in canada vertex state of monopoly which many believed they were using so the killed founder. Gerald nine believed that we could do better and clutch was born. Mike was six years older than jerry. He was more worldly and projected a macho self-confidence he practiced brazilian jujitsu and had bulging tattooed by abs- he claimed that he'd fought as a professional anime fighter. And won millions of dollars in poker tournaments. You can find pictures of him online driving lamborghinis and posing next to tigers. Starting when mike introduced himself at bitcoin meet ups. He told some people. He was from india others that he was from italy but he told everybody who would listen that. He was a serial entrepreneur with vast experience and digital currencies. Sure i've been working with digital currency since about two thousand. I've operated numerous exchanges merchant Casinos provided advisory services with respect to growth and development of other companies have been very very volatile turns for The better part of my adult life. Jerry and mike made an odd couple people who met them at bitcoin op meetings to jerry acted overly deferential to mike like his nerdy little brother. Some of those people wondered if mike not jerry was really the one in charge at tariq jerry may have been the spokesman but mike was the businessman businessman. He wasn't happy with all the money. Kouadria was losing so he came up with a plan to turn things around. You wanted to take the company public. Here's the announcement at a conference in toronto. The big announcement is we have announced that the be going public that ambitious. It's probably because it was not only was kouadria in the red with no path the profitability. No bitcoin exchange anywhere in the world had ever gone public before. Mike hired a vancouver businessman to raise money and for a while. Everything seemed to be on track but an early twenty sixteen. The whole thing fell apart and not only did quadri is bid to go public fail. It ripped the company in half. Mike and jerry had a blowout mega fight and mike left the company and then most of quad rigas board of directors quit leaving jerry completely in charge a lot of finger pointing afterwards make sued the businessman. He'd hired claiming the poor quality of his work was responsible. But mike also blamed. Jerry said jerry had suddenly decided to halt the listing process because it was too much work but to hear some people tell it there may have been another reason quadri. Guinevere went public. It was probably the same reason. Jerry wouldn't save. Mike's name in that interview. And why on paper jerry was listed as quadri only founder was a rumor the rumor that was spreading around the canadian bitcoin industry. The rumor was that michael patron wasn't really michael patron and that rumor was true. Michael patron had changed his name twice. Used to be omar patron and before that omar did not omar. Dhani had some dark secrets. Omar was born in canada. Nineteen eighty-three when he was a kid. He moved with his family to southern california town called fountain valley. Omar was into computers and in the early two thousands when he was in his late teens. He joined a ring of cyber thieves. That sold stolen identities and credit card. Numbers was called shadow crew show crew forums. Omar went by the screen. Name voeller just french for thief. There's been a lot of talk about how shadow crew was basically the ebay of cybercrime ghalem. That was that was screaming column fund. This is brad johnson. He built in iran shadow. It's kinda like a criminal field of dreams. If you build it. They will come so we were the first websites the first website of his car. It was just word of mouth. We didn't have read it or anything like that back. Then and at one point. I think we had a million hits velour. Aka omar was one of the shadow. Cruise money launders lot of members. Were using an ego. Is that precursor to bitcoin. The crypto currency when shadow cruise hackers wanted to cash out the prophets. They made selling stolen identities. They needed someone to convert their eagled into clean money. They could spend in the real world. What he did was he starts saying. I can cash out your eagle for a ten percent fee. Omar would exchange evolve for a prepaid debit card in two thousand and four. Omar bragged in a chat room that he was laundering between forty and one hundred thousand dollars a week in eagle pretty good. But here's the badness. The united states secret service was listening in on that chat room. We were Intercepting text messages from the secret service about them investigating shadow crew when agents stormed in on twenty year old omar as parents house if found a new mercedes in the driveway and a bunch of western union money order stuffed in a closet. Amar pleaded guilty to one count of conspiracy to transfer identification documents was given eighteen months and a maximum security. Prison is released from jail in may. Two thousand seven was deported. By the united states. He moved to canada. Omar changed his name to mike. But besides that not much was different. He went right back to his old hustle. This time he used a newer digital currency called liberty reserve which was popular with cyber criminals and drug cartels. Mike bought liberty reserve in bulk and sold it through a website. He'd set up called midas gold over the next five years. Eighteen point four million dollars flowed through his liberty reserve accounts but in twenty thirteen liberty reserve was shut down today. We announced charges against liberty reserve. Who for years have operated one of the world's most widely used digital currencies that's former. Us attorney preet berrara announcing that the justice department had raided the offices of liberty reserve costa rica and what may be the largest international money laundering case ever brought by the united states. Mike himself was in charge this time. But authorities seized midas to website. Now the liberty reserve had on hand became worthless was only six months later in december twenty thirteen but thirty one year old. Michael patron launched kouadria twenty five year old gerald cotton. The rest was history The first time. I heard the name. Michael patron is in the same breath today. Also heard omar to ninety alter ego. This is michael. Kirk lynn again. He's the bitcoin security expert. Who met jerry. At the mita ended up becoming friends. He introduced himself. Is the ceo of khwaja riga. And i never found that weird at all Until years later. I heard allegations that there was somebody else. michael patron or omar tanani who was the ceo Of kouadria Remember asking him about that. Actually it was. It was one night when he was over at my place playing board games. So it's like. Hey jerry i gotta ask you You're the ceo right. Yep i heard that. There's this other guy omar that he's the ceo At least somebody from vancouver mentioned to me that he's telling people in vancouver he's the ceo. What's up with that While he's definitely more involved in earlier days. But no i'm the ceo He's no longer involved company. Okay in two thousand nineteen. After jerry's alleged death was announced and kouadria customers learned that they're two hundred fifteen million dollars disappeared. It didn't take long for those customers to start asking about mike. What had been an open secret in the canadian. Bitcoin industry became public knowledge when reporters found the name change records. That proved michael. Patrick from kouadria used to be omar patron from midas gold and before that omar ninety from shadow. thousand found. omar's mugshot. And if you compare it to a photo of mike. It's pretty clear that they're the same person so for quadri customers. Mike instantly became a suspect. They wondered what if mike had been the secret mastermind behind quadra the whole taught because to them. Jerry didn't really seem like the kind of guy who could have stolen their two hundred and fifteen million dollars but mike ditka. Mike knew how to launder money and change his name. So maybe he helped or even forced jerry to steal their bitcoin and adopt a new identity but when reporters tracked mike down he said he found out about jerry's death the same way everyone else did from the internet. Mike said he'd been living in saigon and barely spoken to jerry since he quadri all the way back in twenty sixteen but when he was pressed on other points. Mike was evasive. Despite all the evidence that proves he and omar deny near the same person make denied it to reporters also tried to downplay his role at quadri. He said he was just an adviser not one of the founders and he refused to say how he and jerry had. I come to know each other. It only say that they met online around five years ago. Which would have been when they were starting kouadria together but we know now that that's not true. Mike and jerry went back a lot further than five years when the us justice department busted liberty reserve and twenty thirteen. They seized the domain of all the websites that sold it including michael patron site might scold. The evidence for the case has been online for years. There are records for all the websites that sold liberty reserve. How much of it moved through each site and how much each site was hauled at the time of the shutdown and the contact information for the people who ran each liberty reserve exchange. Nobody ever had much reason to go. Combing through these records until riga collapsed. And mike became a person of interest and when kouadria customers finally went looking. They've found was surprising because the contact for my goal wasn't michael patrick. And it wasn't omar patron or omar dhani either. The contact email for might as gold was gerald dot cotton at g. Mail dot com. That's right jerry. Had secret pass to jerry. And mike had dealt liberty reserve together before they sold bitcoin. Grandma's at riga the helped cybercriminals launder their money and midas gold. Next to mike. Jerry seemed pretty harmless but this man the maybe they weren't so different after all. Maybe behind the mask gerald cotton to the world. He was somebody else too the next episode of exit scam a few years ago. I was struggling with my financial portfolio. If all like there was no solution. That is until i found the quadra fund. Then the people would lost their money would just sort of think well. Maybe the money's coming. The money's coming only they just kinda give up hope. I'm excited. are you excited day. I am excited. Exit scam was written and produced by me. Aaron lamour and lane brown mixing additional editing by martin fowler show art by mickey dj theme song by francis and the lights music in this episode was by francis starlight mark allen piccolo ross seem anini martin de fowler and myself are executive producers. Are max linski lane brown and myself for treats media additional sound editing and producing by james nicholson additional producing by jacqueline scurry special. Thanks to jake. Schreier patty greco niraj wall majgaiya leisure status. Evan ratliff joe level and garrett and j can have you wanna listen to next week's episode right now can listen to it for free and the odyssey app. That's a u. d. a. c. y. Look it up in the app store of your choice. Listen to next week's episode right now. We'll be back next week.

jerry quadri Jerry mike gerald cotton vancouver michael patron Bitcoin Kouadria canada Mr kaden Omar Mike mark zuckerberg erin lamour Michael patron omar Quadri michael parkland