Exploring Event Modeling with Adam Dymitruk
Today's episode of hansel minutes is sponsored by data dog a monitoring and analytics platform for cloud scale infrastructure and applications data dog provides dash boarding alerting application performance monitoring and log management in one tightly integrated platform. So you can get end to end visibility. Quickly be proactive with your monitoring strategy and catch issues before your clients are impacted. Start managing the overall health of your environment with a free data dog trial visit. Data dog h q dot com slash. Hansel minutes to get started. This is scott. Hanson was another episode of hansel minutes. Today i'm chatting with adam event. Modeling pinch me all of that. All things events. Are you sure. Good scott happy to be on the podcast. That's been years of waiting for this day. So thanks for having. I appreciate that will lower your expectations now but You talked to me about event. Modeling dot net fringe. Which is a conference in portland a number of years ago and we have a little youtube video up on my site. That's a couple of years old But if you go around and search for like event in google it'll auto complete a number of things it'll it'll autocomplete event modeling. It'll autocomplete events sourcing. Sometimes you'll get event storming. Maybe we can start by understanding this concept of events and explain this kind of this new way of thinking about system design. That's been actually not new at all. Yeah that's a. It's a interesting journey. That i started twelve years ago. By looking at the technical side of event driven systems and slowly realised over over a decade long time that It's really how we always thought about information systems long before digitisation I always go into the fact that digitization is just a small blip on human history. It's only been really the last fifty years of even that stretching at of modern And one of those things is application of of moore's law knots computing power but the storage and the transistor gave this giant leap for processing power. But we didn't have a transistor for storage of information and so there's this mismatch and this is where we came up with Tables normalization all these technical things to to make digitize information system significantly different than the way we treated regular information systems on pen and paper. And what's coming back now. Is that all of. The tech has caught up including storage. Now you can have a run on the entire company from the nineteen fifties global company from the nineteen fifties on your phone Just your phone. Has that much storage. So that moore's law aspect as apply to storage has caught up and so we no longer have to have Contrived abstractions ways of thinking to digitize systems yet. The ruts in the road are deep. And so that's the way we still learn Systems and it's really about on learning and thinking that hey humans think in stories how would civilization started is really about telling stories to the next generation so even before the written word You know a grandpa and grandma would be talking to to their grandkids around a fire in a cave and and this is how knowledge in memory got passed onto the generation so the story like nature of our minds and being able to key frames from stories or pictures or it's very interesting field of study just to see how we can with a very few data points alcohol. Him facts or events recreate Reconstructed entire story and so this is a really fundamental shift. In the way we designed systems whether they're manual or digital so that With human to talk to them. We already had clues about this. With the behavior driven development specification by example the reason these were successful is because they played in harmony with the the human minds worked. And the reason that you wasn't so highly adopted as because graphs and multiple documents and so that didn't stick around because it was hard to remember them and it was tedious to go back and do that but we can get into more of the other details as to why that's a little bit more onerous and how event driven thinking and event modeling. Help you of fat problem okay. So let's let's back up and level set for a little bit. 'cause you're a lot to unpack that. Oh yeah so. You said digitizing systems so systems can be a human interactions. They can be processes that one goes through. They are as you said a new thing in the last forty fifty years Assistant might have been someone comes in says. They want to check out a book from the library. Someone fills out a slip of paper that starts a process and in that paper becomes alleger of sorts. Then it gets put into the book nothing and every library in the world might have a little bit slightly different system. Yeah exactly so. I generally stopped talking about building software. Talk about automating information systems because how many people are doing brand new encryption scheme As their daily driver for in in their job very few most of us that are out there. Working in programming are automating information systems. Moving information from left to right calling on. Api making sure that they could the response in store that response and so on. There's very little you know craftiness to it in terms of being very smarten Coming up with brand new sorting algorithms such. So it's really the same thing over and over again. And would event mulling does distills down that molecule. Blood is smallest. Change that we can calculate and then have the fewest amount of patterns to describe any information system sort of like seeing different information systems as snowflakes different snowflakes. Your insurance companies much different than my insurance company yet on the very lower level level. We see that we have the same sort of state transitions. Someone does this type of action in this event has happened. Same on this In my company and so we can start to identify these tiny little pieces of patterns like small we have identified a core four patterns that basically describe any information system. And that's that's sort of like the answer to the patterns soup that you find if you buy all the martin fowler books and all of the different patterns books you're gonna have you know three hundred different patterns. And how applicable are those. You might exercise three or four of them. You might prefer one quite a lot in the specific language but there's really no underpinning sort of like a just like a nuclear physics particle soup in the early nineteenth century twentieth century. Sorry with With all of the different Subatomic particles but then someone said well. It must be something. That's common to all these these are made up of. They found that right. I forget what they're called corks or i forget. I'm not an expert field but there's a very interesting parallel to that. I'm insane with matter and energy with eagles quite a simple formula on top of all of this other research. That underpins all of that. So what we've done is sort of the same thing for information systems to identify these inputs and outputs in how these various small little patterns can be used over and over again to stretch out any complexity and described in very repeatable tiny steps that actually give you really good estimates and other things you can reuse your whole stack the same way. A brand new junior can go in and copied your previous step and just change the contents and continue on making sure workflow continues to act as illustrated by the way. I'm sure that folks will probably chat us on twitter or line about our physics and whatnot. That we're thinking about like the higgs boson that was discovered twenty twelve and then the the different particle. Oh or gorsuch analogy. And i'm sure we'll get totally slammed for the him. No strange quirks work. So when i was in the ninety s i was at big giant. Enterprise companies like nike. And we we had order management systems and it was like the height of object oriented programming. It was the height of of all those things. You described all the different factory patterns and stuff and you know it was. It was a time when you could roll into a a place with a whiteboard and start charging people money to teach them about the factory pattern and one of the things that we would do is we. Would you know argue about how to name things. and Fast forward where does object oriented programming and lovie entity relationship diagrams. You are m diagrams rather fit into this kind of his kind of world because fowler still uses. You know a very specific way to describe systems that is Language nonspecific has it bent. Modeling fit into that. It fits very much in the same way that it's non language specific in terms of The a specific type of Wording that use use a business wording for you events such as user registered or a book was told from the library and things like that however the underpinning of what you're describing is quite significantly different with object oriented programming paradigm with those enterprises. You're talking about from the nineties It goes back to that ability to store everything and we didn't really have ledgers back then. Something would add an archive table beside a table. To show know what the roads used to be like just for for for traceability. You know the unclogging a and logs would do that but it was really kind of You'd have to go dig for that. Information wasn't part of actual online transaction processing. It was for historical and for emergency type of operations However now that we do have a ton of storage at our disposal we can actually by default not throw away information right and so this is about more about events sourcing the event modeling but it's where the founding sort of ideas come from and so what What we would do in those days. We would need to have a third normal form. We may make sure we wouldn't repeat storage of an address. For example because storage was a concern. And also the distraction of of what is state. Now that was really important what it is now and we really didn't mead or thing that the history of how we got there was as important. That's why we ran into a lot of trouble in in software Now fast forward to event modeling of by default we keep everything and we actually are quite loose with what the state is. In fact the most responsible thing you can do when someone registers or takes another action. The system is just a store that fact and then let other systems later figure how to interpret that fact. That's the more responsible thing to do because we have fewer moving pieces to establish that. Hey this happened. And now we know what happened and we can move on and basically build All the other interpretations when of there needed to be sort of like the agony but in for an architectural perspective Hopefully that makes sense that we have a of an ability to include time in our abstractions whereas before we didn't have that ability because of storage constraints folks who may have english as second language or not understand what you just said yagudin. Knee is an acronym raise. You're not going to need it. That aren't gonna need it. So your your goal here is to take complex systems which we tended to turn into even more complicated software systems and express them in a way that is more natural for humans and thereby become as as complex as possible here using kind of You're changing the size of the lego pieces. Now that we can build complicated things but the the the corks the the legos the molecules they the the essence of what. You're doing that the size of the chewing the pieces that you're chewing off here are are limited inside. That's right and and your seems to be like. We always talked about single responsibility. Principle did object oriented programming. You're taking that that responsibility very seriously as absolutely and other Principles such as open closed principle. A lot of times these. The state transitions talking about those lego pieces you're talking about. We call them slices because we include the ui in an event model which is really important that you why you x. Part of the system even it was a back end system we add in the ui for monitoring screen half the people there are visual learners and half analytical right or some and there's obviously overlap both In the in the space of all of our architecture talks we really didn't talk about the. Us experience always import to tie it back to what these are experienced is overall in event. Modeling provides out of the first class citizens. I think that's one of the biggest differences between event modeling in any other kind of modeling or design discipline. That's their ties. Everything together in really does go from the boardroom. All the way through to implementation testing and maintenance of your systems version the systems and and migrating to new versions of all of those types of things are considered when you're dealing with that and so back to the lego pieces that you said a real good really good analogy. We established that. We're just gonna use these for different blocks in that way because we're using these four different blocks. We get a really high sample number for when we use them. And when you get a high sample number you can get a really good average. And so now he can do estimates empirically like from this team can do this many Workflow steps column or in a given time for a given project of a certain size and now we have something incredibly reliable. I haven't seen a more accurate. Estimation process in my whole history have been writing software for the commodore sixty four when i was a kid in the eighties and they have never seen a way to predict the of software that that match this which is really one of the i mean that alone should be gold for businesses and then i just want to make sure that we we touch on a very briefly for those again. Who aren't may not be familiar with these different terms. The open closed principle. The idea that something is is open for but is closed for modification and that seems to be pretty fundamental. Like you're you're taking some of these fundamental tenents of of solid solid principles of object design. Exactly and you're supplementing them and insisting upon them and then taking it to the next the next level exactly we're making that concept available to business because because we chose such small lego pieces it's easier to throw it away and replace it with the correct ones in go in and have to swim through and and try to fix something on. This is a very powerful methodology as well for rooting for doing a polyglot developments. For example if you starting to And for those. That don't know. Polyglot means using multiple languages in the same solution than even the service which is. How could you do that while. Imagine if you have to hire someone. That's an expert in your field. Maybe it's a library and This person is an expert c. Sharp developer your solution is going. Would you wanna send them on their way. Because the tech stack doesn't match or force them to learn cpr or vice versa. Or go lang. No we should be able to take these. Little pieces of state transitions have them done in any kind of stack that you want. I mean we already see a lot of this happening in the service world. Right which you're familiar with but that that doesn't stop us from being able to do that on traditional Topologies that we may still be using today. Okay so when. I talk to the regular joe and jane developers out there in the world the patterns that i tend to hear a lot about tend to be a little closer to the glass of the monitor. If they're things like model view controller and they're thinking about view models and things like that What kind of person. What kind of engineer or architect are thinking about the things that your level. This is kind of back end stuff as things that don't have a ui or does this cross cutting is incredibly cross cutting and it even cuts upwards into being able to use event modeling at the c. Level director level for being able to zoom out of that time. Line because event has a core timeline on it by example. It doesn't have splits in terms of pass that you could take like an activity diagram or some other flow chart. It says this is design of the entire system by example like reading a storybook. But if you zoom that out and just keep some key cornerstone events. Such as payment completed or library book signed out something that identifiable at the at the director level or sea level boardroom level decisions you can remove all the other ones and just have these small pieces of the small indications of how far you got and say that you know this twenty on the your twenty twenty two twenty twenty one a roadmap includes this part of the workflow but behind us. We see what happened in the previous five years and in front of us. We see what's going to happen the next five years there you can start to analyze that. Hey maybe digital currency is something. That's going to be more ubiquitous. It's going to move in from being an experiment. To being a product we can use Or at least some custom development that we can do with reliable libraries to do that with Or it's going to become a commodity words like email right but we can plan our idea of where a company is going to be in relation to all these but zoom into event model for the actual meat of it. We see that out. This is an example of joe. And bob using the system and the manager. Let's as a hotel system Hotel system and this is what each person sees in their email or each Each screen on the computer when they're at home and this is what the manager sees behind the counter when they check in and they have that story. I see all information that goes in and out of the system accurately in that also gives me other thing it gives me something called information completeness. Sappho i have a bit of the most important part of the design is really about making sure you have the storage places in you. Understand what data's in the system and this goes back to mythical man must with with a with fred brooks so mythical man month is a really old book from the nineteen sixties i believe and gets quoted for a bunch of other things but i think something that gets missed. Is that fred book said. Show me your tables in understand your entire system and how it works. Describe your Algorithms and schemes and will continue to be mystified. So state is that important. We've kind of been ignoring it. more-or-less in terms of where we spend time in design If we use state we actually have a much better anchor to reality to actually what we're going to need to implement and so for us it's been a driving force for the last five years at my company and we own the develop things this way if someone sits down with us and says that this event model is correct. This was is what. I'll be happy with or the or they actually do the event modeling with because the very easy to do. It's it's a very low fidelity easy to explain practice. Or at the least years of showing what screens people see as time moves forward and then what information gets stored along those same timelines and ensure the interactions by example. Say hey at the end of the year. If i have this system doing this i'm good. That's that passes and can slice that up into these little pieces that we talking about before to give a proper estimate and cost and time in all those things you know how your op. Steam keeps a pile of scripts and wicky documents explaining how to perform those routine and emergency tasks that keep your applications running. They might call them run books or playbook our friends at octopus deploy were thinking you know. Devops is about collaboration. So it doesn't make sense for run books to be automated from the same place as deployments. Well octopus deploy is now the first deployment tool with native run book support. The best thing. Is you run. Books can share configuration settings and automation. Steps with your deployments. Find out more at octopus dot com. So if we are a company that is already or a person is already familiar with like you. Mel unified modeling. Language is greedy bouches. Yep way of thinking about software And we're designing systems and we're used to doing things like swim lane. Johnson and stuff like that Expressing something in the context of event modeling is is going to be very natural. I absolutely is going to give you a fast. A the turbo boost for that approach Because of grady's excellent work is born on the best way to express what the program is doing. however it's a little bit owners because you almost have a one to mapping between your classes and methods in all the other things you want to show in your a mallender seven types of diagrams. So essentially we looked at the goals of what you suppose to be used for or any kind of design methodology in say can we achieve those same goals with fewer parts and the parts that stuck out as the biggest piece that we could remove is the how. I don't care how you store that information as long as it's stored taking that how the minutia of the implementation really reduced the amount of work for designed to be down to like five percents. I mean we've been doing event modeling sessions from an afternoon to two days for startups to large fortune. Five hundred companies There's an entire mind. The deepest mine in europe is is event money to coordinate sixty different subsystems that need to be coordinated together. We have large international banks like capital one using the coach. So event mullings Basically beyond me. Now it's taken off as a way that holy smokes. These guys proved that you can ignore this. really chunky. Part of design and reinvest that time into all these really awesome things like x. Ui test ability the a mindset of specification by example on a very large scale and basically break down walls make very wide bridges between all the different roles in the company that you're all speaking the same language a lot of times. I say this is the blueprint for when you're building a house. We're building a company will busy a system. We have a blueprint. We can rule out on the table and anyone can walk up to ask a question. Get an answer immediately. It serves designed documents. When you're building it serves as a map when you've already built where you wanna find. Things were things have happened. It applies to traditional systems as well as new event based systems so traditional system state might just be represented by By rose being shown at the bottom sorta like a sequence diagram but turned sideways. And there you would have the state of each column in each row that's Play right now when they get changed and that's also weigh in traditional systems. Couplings of big enemy. We have different pieces of functionality Coupled together by the same state and some table and you can show that you can show that in my workflow. I have this thing at one end doing something here in this other thing at the other end doing something almost completely different. But they're sharing this one row and column from this table. I can show that against say. Look if i need to change that part this is going to drag in with all depends in We've all been there. We walk on eggshells with some of these large systems where we start to you. Know tweak wonderful method here then oh my goodness we just all the integration tests fail. We've all been there. It would be nice to have a map to navigate that it's listen. We either have to augment something on the side but if we change this we have to consider you know five ten of these. A lot of times are done via you. Know a analysis of dependency graphs and other things. But some of these things are more insidious where they're just about the fact that something is stored somewhere in in state and it might have been kind of a side effect that someone someone's collegiate in some extra coating in the field in value in the field somewhere from you know as the systems grow bigger and as you get more into technical debt and other things you see more and more of these things. Needless to say You can basically use event modeling to get yourself out of those situations. Do some empirical strangler pattern stuff. I hate to use patterns again. I'm not trying to just get things down before patterns but For people that are Familiar with some of the works around legacy systems all the strategies. They're they're made much more easily. Accessible via event mulling because he can use the state approach that allows you to really remove some of the really burdensome Ideas about how and just empirically look at what happened in. That should fast track to getting somewhere a lot faster and get some better tests place other things or entirely you know start to implement things on the side and still integrate with old systems. Those are all possibilities. So one of the things that i think is worth noting As we get towards the end of the conversation here is that we've been talking about things. Referenced the nineties you've referenced tables you haven't said anything technology specific you haven't headset sequel server mongo or anything like that when we say things like cloud native or born on the cloud Even the cloud didn't exist when these systems were created. So you're not dictating. Where any of this state is stored at could strategist. It could be as your storage could be wherever this Nondenominational from this is a religion of yours. But it's a nondenominational religion that really supports all system as more like environmentalism right. We care about the plan in a good state and say it's more of that than religion every religion can agree on. It's being a responsible citizen in in your world of software to really open up the communication channels up and down be transparent Know exactly where the work's going to be if someone makes a decision See the impact of that. Make it not something has to go down the chain of command through four different roles and by that time you don't know who to hold accountable for why things went wrong or or why there's a change that needs to be back bubble back up because it might not be the best implementation So a lotta times Modeling diagram is shared and actually collaborated upon by the most amount rule. So that you can have people you know from. The us us on designers is a really bad idea. We shouldn't have these screens order. People hate that Which regular engineers have really don't care. I'm just making the is here And vice versa. Someone can suggest some. Some type of a workflow were screens are shown. This order and the engineer can pipe up and say you know this is actually quite bad. We're going to need to reallocate all this extra storage and whatever else. These estimates will be hard to implement but they can have a back and forth to say. Why you know they have a form for that. They have a canvas for that where where they can actually look at both sides of the of their concerns in in have a discussion. And it's not about us. It's about them standing shoulder to shoulder and talking this solution in front of them right. That's a big difference. Once he sta- start. Having the design documents. The designers hold the diagrams. The developers hold this strategy roadmap the boardroom holds and so on and so on right that creates islands and and really requires a lot of translation. What we want is a better form of communication. We want a collaborative medium that everyone can understand and have that so sometimes when we talk about what s delays or maybe security we can introduce the idea of overlay on top of the event model. We can say well. This'll jumped from this you. I hear to to actually Register that's you know that's got a security risk because it's going over the network. We have to make sure that. That path is encrypted. We can have overly were. That part is highlighted in red in the security. Overly right. it's just like you would have a blueprint for a house where you can start to put overlays for plumbing or electrical heating or whatever you have and you can have that discussion with respect to the whole house and still understand where you have to go with. Florrie have to be on to actually put that sock or whatever you're doing that's actually an interesting analogy right there. We know often in the last twenty thirty years people use construction related analogies. And they talk about how you know. Houses aren't falling over. And we all understand how to talk about and think about houses. Why can't we do the same for software. And then of course people go the naysayer. We'll go and say well. That's that's not a reasonable analogy. That's a trash analogy on. Software changes this and that other thing. But you're you're saying that in fact that blueprint that here's a piece of paper design house design. A system is actually possible. Yeah i think so. We have the right pieces. I don't think we had the right small enough pieces to do that. Analogy properly to have that. Parallelism shown in terms of effort of development to equal the cost of a brick or a spanner for bridge. Right that bridge analogies always been. It's tired thing that software is not like real construction site. Well no it is. It's not so far removed that it isn't than so what we're doing. Here is identifying the bricks that make up our structures and yes we a house who might not rebuild all the time but we certainly do maintenance on it We certainly want to be at a point where. We don't have to demolish the walls because halfway through you learned that we need a second floor which is kind of like the you know if i want to put as on the bad place. That's what i would say your ideas that you're going to build the first floor and then suddenly next sprint. You're gonna find out. We need a second floor so attempt to demolish those first walls and put up supporting walls. Because there's to be a second story sounds ridiculous but that's kind of how we have How far the pendulum has swung away from you. Amal and all that right. The pendulum has to come back to somewhere in the middle. And i think event modeling is really where that pendulum rests and why it's so successful in so many different companies. A totally took me by surprise because now this was just the way i wanted to work in my company because we found that this is the most efficient thing we can do. Were small startup. We know with a specific way of doing things that very few clients can appreciate because it's not widely known so we need it to be incredibly efficient and so have a little joke cartoon on my profile Where you know event modeling is all i need. All these other. People are buying in there costco shopping cart. The perspectives on the stand ups and this just giant pile of practices. What we're doing is that we had a really good to allow me to. I can't afford to wear a tool belt with fifty tools on it weighing me down. I need to find the efficiency of finding one tool that will replace ten of them right and make those efficient choices. And i think for the event mulling was that efficiency choice it came from actual struggles to build systems fast. Cheap with high quality with predictability on fixed costs like all of these interesting things like events sourcing event long. They all have come from not rich places where you could buy whatever you want it. They came from Either high performance demands or are limited budget demands or incredible time demands. Like every time. That i've done this with with a hackathon. The hackathon was one because in fifteen minutes can basically event modeling as get a rough design out there and with the first to finish hands down. If you've it's a secret weapon is not modeling and you go to a hackathon you can get your teams on the same page so fast you have you've just doubled or tripled. You're the chance that you'll that you'll win Simply because everyone knows what to do that ability to slice up your workflow into pieces in the fact that you've you have a contract in between all the state transitions as you move left to right. Through the example you actually have a very strong contract that will guarantee that things clicked together correctly in a very easy way And so you don't have to do a lot of rework with a hackathon nightmare of like. Oh i don't know what john did but his. Api can't call you know those things and struggling the last hour to debug a just a nightmare. The the event modeled ones are are way more sane and they just continue to slowly fill in that blueprint the the color with crayons which parts are done. You can have it on the whiteboard. It it's a really good test so these high stress places really do that. The good thing is that it actually scales so you can go from tricycle to bicycle too motorcycle because the infrastructure that makes state transition happen can be changed. So if i'm just doing stuff on like a shell script to make one st transition because i'm in the middle of a hackathon a company. I know if that state transitions the same. It'll run on like your scaled out. Server listing in asia problem. That's awesome guarantee right now so folks can learn about this at event modeling dot org again. It's not a product to buy. Anything can go and explore all the different posts up there. I would recommend that you go to a modeling oregon hit slash about to get a little bit of context. And then you've got a what is event. Modeling of mulling. What is it post from. June of nineteen is a very clean about a twenty minute. Read with lots of diagrams at a walk you through the entire thing. Step by step. And there's also workshops people can attend find videos of you online. Yeah there's lots of new. There's lots of news to happen all the time. We just finished the week long crazy workshop. That was forty hours of instruction in two time zones which was repeated which means eighty hours of of stuff. Eight nearly died during that workshop. But we're very committed to to helping as many people in the world as possible. Get more done for less with higher quality all the good things. We're all after. I'm almost finished the book. It's in draft stages. It'll be published Sometime january already mentioned the website. We have a slack community right now. Ten thousand people there but they're all helping each other out in ways to use event long. Describe their systems and And at aveiro ganic community. I i hardly ever mentioned. But more and more people seem to congregate there so. I hope that i see you there. I'm very excited. When they see new people always tried to say hi and help them immediately with whatever questions they have so More than the happy to that. That's the slack. Channel invite can be found on the resources page very cool on event. Modeling dot org. And if you don't remember the address if you if you just google event modeling or being event modeling it'll come up as the first hit all right. We've been talking with asked me trick you can check them out at a modeling dot org and this has been another episode of hansel minutes you again.