The Problem with Software by Adam Barr
As a software engineer chances are you've crossed paths with mongo DB at some point with your building an app for millions of users or just figuring out your side hustle. It's the most popular non relational database mongo DB is intuitive incredibly easy for development teams to us. Now. Mongo DB atlas you can take advantage of mongo DB's, flexible document, data model as a fully automated, cloud service mongo DB atlas handled all the costly database operations and Adleman tacit. You'd rather not spend time on like security, high availability, data recovery monitoring, elastic scaling, try mongo DB. Atlas for free today. Visit WWW dot mongo DB dot com slash cloud. To learn more. That's mongo DB dot com slash cloud. Hi, this is Scott Hanson. Ansell minutes today. Chatting with Adam bar. He's the author of the problem was software. Why smart engineers write bad code worked at Microsoft for many years? Didn't you, sir? Yes, I worked there for a little over twenty three years in two different stints ten years and thirteen years as developer deadly Dev manager, and then a couple of other jobs, and you've you've since escaped and art consulting and hopefully enjoying yourself. Yes. I this things I've is about Microsoft, but overall I'm happy to be have a little more flexibility in my schedule. Well, when you took a sabbatical you in often wrote this book, the problem with software, and I was fortunate enough to get an early copy, and I've actually on the back of the cover there where they little quote about. Why I think this this book is great and one of the things that I I said about it was that it was an unflinching view of what sucks about software and how you can potentially solve it. It's a really dense book, and I say that in the kindest way of the word dense. There's a lot of information shoved into three hundred pages in this book. Right there deal is I wanted a book that was. Readable by non programmer, but also could be read by programmers. So the non programmers will get some insight into what was going on with programmers. And the programmers would totally get a little self reflection about what they could do better. So my concern with that. I'd make some comment about what example was I'd say, well, a bite is eight bits. And then some programmer in their mind would jump up and say, well, no, actually, no, there's this example referenced in the original KNRC book wherein, abide is nine bits. How can you say such an uninformed statement? So I tried to put the the details like that proving my programming chops into the end, though, it's because the programmers will reduce and then the regular people will just get the notes, and hopefully get a little more of a story without all these random tangents explaining these really esoteric details about stuff that only a program would but care about I'm trying to avoid the situation when you read a book about programming. Ng Clive Thompson, just come out of his book coders, and I was leafing through it. Even just reading it. He's he's not really a programmer. And and you do you see these things where you grit your teeth and you're like, oh, that's not quite accurate. Even I understand for wide audience. That's what you gotta do. So I tried to be accurate where possible that does lead to some digression tangents just to really explain well, this is actually what the real story is. I'm actually gonna have Clive on the show and a little while and I'm quoted multiple times in the book. And even though he had a fact checker I spoke to him at length. I spoke to his fact checker at length there's still a number of times. And I was like let's not exactly how I would feel. I saw you know, there's all these people talk about bugs and software. They're kind of merging together an actual bug like heart bleed where the program made a mistake. And then things like the seven thirty seven max aid problem, which is really technically not a programmer bug because the system is working as designed it was just assigned badly. And then he got problems like the the Ericsson certificate problem last fall that knocked out some cellphones and people call all this things bugging dogs. He's trying to give an example of bug and he gives a python syntax error. That's not any of those that. That's that's not at all what we're talking about. It's not a logic bug or design bug aren't operational bug. It's some minor thing that gets that gets cleared up quickly and never affects any code. That's checked in. So I was trying to be hey, I'm actually a programmer, I do understand this stuff. When I try to explain it for a man at least, I'm not making some basic mistake of. Mis categorizing something that would grate on a programmers is ball's. I was trying to explain meltdown inspector, you know, the Intel CPU exploits as you know, they're they're not really a bug as they were like you said design flaw was trying to explain them for the lay person. And I said, well, imagine that you are thinking about going and spending some money and by virtue of the fact that you thought about looking in your wallet. I stole the credit cards out of your wallet. You know, you're you're doing a predictive, you know, your there's there's two parallel universes. One where you left an open your wallet and went off. And did it in one way you didn't. And by virtue of the fact that you thought about it. I've stolen your money and people like either think that's a perfect analogy. Or they spend the rest of the day Rippin, the analogy apart is poli incompletely imperfect analogy, there's just no good way to make analogies around code, your honestly that analogy surveyed has some flaws, which up never mind. I thought I heard about that. And I thought, wow, that is really clever occasionally you have to look sometimes you look next like, wow. I mean, just to think of that and realize you could use that cash speed to predict stuff in realize what the what the processor had done is. I got more complicated is actually pretty darn clever. Yeah. And you refer to the software sausage, then how it's made. And what's what's clean, and what's not throughout the book? And one of the things that you call out kind of early on is this the education of a programmer, and whether or not we can catch these things early or maybe some of these bugs in software are really bugs and how we teach software. Yeah. I think the problem is you you could learn to program on your own and a lot of people do this really started once the personal computer became widespread and people could teach themselves to program without going to work for our company or go to college. Which is where the computers used to be and people talk themselves, and then they went to college and kinda use the same skills. They taught themselves and university is what really teaching the actual mechanics of writing code they tended to teach algorithms, and so people were basically all self taught even if you had a nominally had a software engineering or computer science degree. You actually tight yourself to code and you had tie yourself on various small programs, maybe with a couple other people working for three months on some assignment in class, and that is vastly different. From what you do at a large company where the software has to persist for years, and it's multiple people working out about groups at stitch together with API's the frustrating part of all this is IBM and other companies figure this out in the nineteen sixties and wrote a bunch of books about it. But that was essentially ignored by all these self taught people in the late eighties and early nineties. Because they could be successful with their self taught methodology so you really wind up, and it's still today that it takes a while working in industry for people to hopefully figure out oh API design is very important. It's not some afterthought stitching. These things together is really what matters not just cranking out some code in my office, and Microsoft, certainly despite that being known and ignored. They spent a long time figuring out things, for example that you couldn't test in quality you had to design and in. I heard Jerry Weinberg state that Charles Babbit JR. Working on the difference engine. One hundred fifty years ago stated that you cannot test quality and you have to design it in. So it's not like it's a new idea, but people habit to rediscover it on their own at great cost of time and frustration so okay. So let me ask you this this tough question because as I'm going to refer to you, and I are senior developers, and that could be a double entendre in the use of the word senior. How do we make sure that we as senior people are not 'gate-keeping and preventing younger people or self taught people from being successful. Like, there is this knowledge out there by virtue of the fact that they are either self taught or learned to code camp or maybe just learned at a university that didn't teach them these things. How do we set them up for success? When we bring them on our teams. Certainly we don't want them to simply not get involved in tech in the first place. We want them to be successful. Right. It's tricky to avoid big the grumpy old person shaking your fist at those kids today. I think the way I learned and I've started out by had a job before Microsoft, but my first real serious offer development was Microsoft, and I happened to be on the windows NT team under Dave Cutler. And luckily that was just the way things were done. You did do design work. You really did worry about quality you try to avoid being careless. That was the just the in the air in the group. And so I think what's important when people come into existing team at a college. It was just to say, look, this is what we do acknowledged that the people are smart Nicon, right code, Nick, debugged, small bits of code. But in a gentle way. Plane that industry software is different because the scale and the the time that has to last. There's certain ways we have to operate despite that your first code review will be hard to take for anybody. It will feel like personal criticism. No matter how much you prepared for it. I wish universities did code reviews. They had students doing code reviews of each other's code both for the educational aspect and also just getting used to it that that's part of working in softwares. Other people have opinions about your code. And you're not the the the one software genius in the world. It is hard to take that feedback though. Like, you have to be able to take like your like, I I used to say like, you know, my co doesn't suck. We might code sucks rather. I don't suck. Right. It's like when you write your first essay, or your first poem or anything, you put a piece of creativity out in the world and its sub optimal. You know? It doesn't work for whatever reason it not attaching your ego to what you re wrote is challenging for anyone especially young person. Right. A specially if you're in software where you're sorta succeeded without really having to learn a lot from other people and get feedback. I think another industry is certainly other engineering disciplines or areas like medicine, you're learning from other people all the time. Nobody goes through a medical program thinking, oh, I know everything about medicine. I'll just keep doing what I did in high school. So they've already been taught that yes, you have to learn if the stand on the shoulders of giants. And I think made an analogy in software Mark or maybe getting a little boost onto their big toe or something and most most stuff is just relearned by people rather than having understanding at that's the way the world works. You learn something from other people. Speaking of old people who shake their fists. You mentioned your Dykstra in the in the book, and you talk about in nineteen seventy five he said, it's impossible to teach good programming to students who've had an exposure to basic, they're already mutilated beyond hope of Janet regeneration. That's a little bit rough. I certainly hope we're not doing that today with Java script or other languages that are more accessible. Well, every time I make a comment about a specific language somebody yells at me. So I'll refrain from that. But Dykstra was really saying the quote is a good one about basic because well, Bill Gates is first LEGO, which was basic, so it's not funny, but the timings actually important it was nineteen seventy five. It was the year that Microsoft was founded to market a basic interpreter, and ducks are really trying to tell people languages like basic as they existed back, then not not visual, basic, which is somewhat modern those languages. Did they didn't have block? If statements they were riddled with Goto out of necessity, the seventeen ram ramblers. There were terrible languages, and he was saying there are actually better languages algal. In particular was a pretty good language, and it's really the the ancestor of of c c sharp and Java and most modern languages. He was desperately trying to tell people stop learning. Basic, stop learning fortran Cobos your first language, use better languages. Unfortunately. It was just the wrong timing the PC industry was happening. They all use basic is there built in language because you could fit it into one k rom or something. And so they went with basic all these people bought their PC's taught themselves basic, and maybe there weren't mentally mutilated. But they certainly learn some very bad habits. I always had an argument with somebody about well. It was my sister actually about how. You didn't really need haram at Hearst methods, and you could use a line number set of a name. I mean, you understood right. That's just a classic kind of thing. You say when you've only worked on small pieces of software for yourself, and you've never worked on something big. Because if you're writing a small piece of software for yourself Eddie lag, which is fine essentially 'cause it doesn't matter. How readable it is? You understand it? It's when other people have to understand your code that something like old school basic really becomes problematic cheesy program or to your family programming while she's actually not a program of choose taken a a program at class in in college. When I was a high school senior. Uh-huh. Audrey with my sister's probably not the you know, the place. Why applied the most logic tomorrow arguments us, probably mostly understood in proven her wrong. So I adopted this said ridiculous attitude that IBM PC basic was a was a good language. Your code is powerful. And so are you and when you consider the reach of your code. It's clear that to deliver quality code. You must also deliver secure code. So where can you start? A first step is educating yourself on basic secure coding principles and beginning to put these principles into practice. Every day. Vera codes secure coding toolkit is your go-to resource for all your secure coding needs the benefits of the toolkit include learning best practices for a secure coding understanding how to secure your DevOps environments and learning to combat the most common software, vulnerabilities stay secure and download this valuable asset today at VERA code dot com slash toolkit. That's VR a code dot com slash toolkit. You you also talk about the difference in programmers of different eras and what they worry about. And you point out that the programmers of your era worried about the efficiency at which they're programs. Is run. And I remember myself reading the book programming pearls, which is a collection of columns from Carnegie Mellon, computer, science professor, and it seems like I hear at least amongst applications that are commercial applications less and less about the concern around if fishy are we getting away with a lot because processors are so powerful and memories, so plentiful. Well, we probably are. And I think that's fantastic. Because when you worry about performance, you almost certainly make code more complicated and harder. For other people to understand what I teach my kids to drive. I said the number one way to drive safely's not to be in a herd. Because when you're in a hurry, you take shortcuts you you're not your best driver and same thing with Ziya Soffer. If you're worried about performance, it sounds like a noble thing. And maybe you could give it conceptually say, and there's a couple of samples probably where oh I removed the code, and it was simpler, and it ran faster. But that almost never happens Oregon about performance means put again some side channel or adding some weird API for that kicks in some behavior in certain situations of just generally writing more co handle the the common past separately from the Arab path or whatever just just it winds up with less understandable code. And so the fact that people can not worry about in most cases and just right clear. Simple code is. Actually, a great thing again when when Bentley was writing program or pearls that was a long time ago, and you did have to worry about performance morbid. Even he said, although programming pearls is mostly about performance. Even he said don't do this stuff. All the time. This is for the situations where you need to worry about performance. You can use these techniques, but don't just go blindly trying to make your software fast. Even though it feels good it's like scratching a niche. It's easy to measure you made something five percent faster. Figuring out the also made it onto percent more complicated is takes logger involves other people in more time, which are not something programmers generally want to deal with. So they can just sit there over. Optimizing something all day and feeling virtuous when in fact, they're probably not helping anything. We'll certainly making it. What's that? What's that old saying, right? Make it right. And then make it fast. Yes. I like canoes quote about premature optimization is the root of all evil, and I simplify that to say optimizations the root of all evil. Not you should never optimize. But a lot of bugs or they are because some was optimizing something, and they like said made the code more complicated or just had a bug programmers like to write caches, again caches seemed like a clever thing, but there's a lot of bugs in the cash itself which only manifest because you try to make things faster and often without. Really knowing if that was important in until they get something in front of a customer. You don't really know if this thing needs to be optimized and today with stuff in the cloud and the way you can scale so easily and storage is basically free and CPU is so cheap a lot. You don't have to worry about. Optimizing it. You just throw some more hardware at it. You know, it's it's interesting that you point that out because I've got a blog system that I worked on it's called dos blog DAS, like German for the blog, and it's been running for fourteen years, effectively unchanged. But I have a couple of bugs that I hit probably once or twice a month as well. As a lovely bug that I hit every time we fall back because the software never expected that you that time could go backwards, and they're all cashing related, and I and I probably didn't I did it because fifteen years ago we had less processor power than we do today. And I could probably turn the cash off in the thing would work a lot more reliably virtually every bug indus- blog is cash and a cache related. Yeah. I'd say within within within a rounding error. If someone suggested performance optimization, you should say, no, you'll be right enough that well the couple of cases where it's probably good idea. You can you can deal with. But. I think as I said, it's a programmer instinct end you've got these old timers grumbling about kids today. But I think there's far more poor thanks to concentrate on it in particular, make a code this, maintainable and readable. I appreciate that. Like we can joke about old programmer saying kids today. But there what what what what do you think kids today will get from a book like this? They'll there's historical context. I think you talk about the importance of agile and win these things started. And how it how it fits into? How we make software today. What do you think that a twenty year old would take away from this book as they are learning about things like agile and learning about things like design thinking, right? Hopefully, they'll they'll get the understanding that something like agile is useful in certain situations. But not all situations and the that language they like is good in some situations. But now that the situation so just be somewhat sceptical consumer of programming wisdom, they have likely. Limited experience, but all the broad exposure to bunch of different languages a bunch of different code bases. And so they may based on their own experience think that's something is the right way to do something. But that may or may not be true. And I think it's easy to kind of float through a see us undergrad program on your own wits. And never really come to terms of the fact, I'm to face with fact that other people actually are smart to and and. Your favorite language may not be the greatest language. I think though, just an appreciation that they might be suffering from a little bit of who Bris as they approach work in an industry, and those old grumpy programmers may have something to teach them. Yeah. I as someone myself who used to teach computer science at a at a state university. At least I always felt like there should have been a historical context of computer science or a history of computer science in like, let's learn about programming because I feel like I'm learning more now in two thousand nineteen about the history of programming from tweets where I learn about, you know, what, you know, a Lovelace dead or what or seeing a movie like Hidden Figures, and I go, wow, I didn't realize that. That's how that was done. Because no one ever taught me this in school. I don't know if did you have a history of computer science core. We had we had zero history at I went to school in the eighties. So a lot of this this there's a lot of impure. Studies done in the seventies. People actually studied commenting and variable names and method length all this stuff. That's just so debated endlessly. Now with no resolution actually studied in the seventies. And that wasn't that long ago. And I went to school. But we heard nothing about it was just here's some algorithms. Now, go write a compiler and figure out here's the here's the book of here's K and our go teach yourself see kinda thing. And that was that was what I heard from other people. It wasn't just why I went to school. And so yeah, there there's a there was there was really nothing. Historical sort of time began when when Unix was created may be it literally begin though. I mean, technically kind of begins at that moment. Right. We know wins zero was right, right? Yeah. That was essentially the message we got you know, she was CEO as the first programming language ever. You know, the first decent programming language. I feel like that like for people. A certain age. They did kind of imply that like see was the first language ever of any of any import. Here's the K in our book, they handed to you, and oh, by the way, there's a simpler language, but we only have to do that for one semester. Yeah. It really was. I mean, literally somebody asked him class as we're discussing our first programming assignment. Because they were one of the rare people who hadn't taught themselves to program in high school said, well, how do I learn the language and they said, oh, well, here's the book. Knock yourself out. Those were the days. So you you were at Microsoft through windows and NT through two thousand three and a lot of bugs started showing up slammer bugs slammer blaster worms like two thousand three four or five people, were slapping windows, NT and windows, generally around pretty pretty hard where they not. Yes, we did a an attempt to eradicate the bugs. In it had to be before mid two thousand because that's when I left the N T. And that's when I left marks off the first time. So that was where they actually sat down explained about buffer overruns and exactly how you could construct an exploit using a buffer overrun, which again to me seemed. Wow, that's really clever. But then of course, it was my software they're attacking so we have to fix it. But we did it with just with code reviews. And we didn't have the Sal adaptation language that would attempt to track. Some of these see problems the cost across function calls. You could actually get some sense of what was safe, and what wasn't and I think Jim Allchin made an unfortunate quote to comment to the press that oh, we found all the bugs, and and and fix them. And then of course, you had slammer and Zotov and blaster and like because he believed that there was an automated tool that could find all buffer overflows. I think he just thought you could find it by code reviews. And I mean finding stuff by coaches uses is really a classic testing in quality approach. I mean. Eric Raymond was running around claiming that open source software is more secure because there were more eyeballs looking at it, which is just completely not true. You make things more openness as hell. Right. You you make things more secure by by engineering in certain techniques. Avoid in this case in see such terrible language for buffer overruns by there's techniques avoided or you could not read stuff and see that would also work. But that might be a bridge too far for some people. But yeah, we I mean, it's it was I don't think I don't think my code ever actually had an exploit directly in my code. But I mean, I was writing code that faced the network that manipulated buffers received off the network. So I mean, I could easily have had an exploit that was directly my code. Do do people. Keep did people keep track of stuff like that. I mean, does it really matter whether it was in your code or do they their collective code ownership. Does it where people, you know, running blame and saying look it was Adam they they they wouldn't run blame. I think it was maybe this is being too forgiving the attitude was well, see is hard. And I faked that was Gerald. How even when I is hard. Yeah. The first Morris worm came out in nineteen Eighty-eight, which was the first remote see exploit the attitude was well, you can't protect against everything sort of, you know, attitude. I mean, why once I was talking to I actually worked in the trustworthy computing team. And he had. They add some analysis of all the major exploits and who had done the actual check in again. I don't think they were going to blame people. Who is just okay. Maybe there's something on his team. And he had the actual name of the person you've done the check in. And I sort of said well as anybody on that list twice. And he sorta laughed, and then we we ordered the excel by by user ID. And there was somebody who had actually managed to check in to exploits or the code behind to exploits. And with like motor have that a guy we looked them up? And of course, he was now director of development somewhere at Microsoft. But that that's an amusing story, but that doesn't mean necessarily it was a bad program. I mean, I think we all could have made mistakes. But. He just happened to to have made two of them. While certainly let me let he is. He or she who has filed the first bug through the first, you know, throw the first stone. I mean, any anyone can can make a bug like that. It's it you can fat finger something and off by one errors are are the bane of the entire internet. Aren't they? Yeah. These to there's a lot of unit code. Putting Yuna code into the guts of NT was a great saying because it meant you could handle all these languages. But there was a lot of these exploits that were just messing up the w char the number of to bide characters versus the number of bytes needed to store it. And so you had a lot of these. It wasn't off by one. It was off by a factor of two mistakes. And you just try to copy a two hundred and fifty six unit code character buffer, which is five hundred twelve bytes into a two hundred six bite stack buffer. And the thing would overrun I remember those days, very difficult. I remember as everything was switching over, you know, in every we were in school. It was baked into our brains that one bite equals one character. And then one day one bite equals one, you know, one or less than one character or a fraction of a character of the beginning of a statement that there might be. Actors coming in UT Efate at headers, and all these things became complicated and something we were counting bites in one hand and counting characters on the other hand. And it took quite a few years before everything shook itself out road actually walked through this in some detail in the book because I think it's important. I'm trying to explain why why c is a bad language, and hopefully, I'm not losing all my non-programme readers. But it really comes down to just make it a math mistake. I mean, you you can do it. Right. You can write buffer manipulation code that is not exploited by the fact that people use the term virus makes it sound like, oh, you you can't necessarily defend to get an against it. But you can write clean code. It's just a little tricky to do. Yeah. You call that a couple of other things that reminded me of some bugs that I had like you call that the Turkish code page and the concept of code pages where we we had two hundred and fifty six characters and we were trying to map other characters on top of those a remember one of my big bugs that actually I still think still may exist in my blogging software is the the legendary Turkish I where they have an eye letter. I with a dot above it. And when you capitalize that I turns into a capital, I with a dot above it, which is not a capital. I it's a Turkish capitali. Yeah. But you make you make comparisons. Add. Yeah, it's another example. I actually dig into it my book a bit because I'm trying to explain about. It's it's an API you call an upper casing API or maybe call a comparison. API y'all have to make sure you're doing properly in its I've just explained that. Again, API design is really important and understanding API is critical. And that's really somebody. Not exactly understanding what does in this particular case. I wind up railing against default parameters, which I think are really a scourge on on programming and and should be removed because you have to make an assumption about with the color of the API Watson, I think that's a bad idea. And yes, you can save a little typing with a default Prammer to which saving typing as bit of an obsession, among programmers. But if it leads to a bad bug, it, it probably isn't worth it. So I I don't like default parameters on on a P is well, there's a. Ton of great stories in the book, and like you point out, it is acceptable. This is not a book where one page is full of code and the other pages full of discussion of code. It is definitely readable. By the non programmer, it's not an overwhelming bunch of curly braces, that's going to stress you out. It's really a fun. A smart tour of many decades of experience from yourself and others. And I appreciate you coming on the show to chat with me about it. Thank you so much for having me. You can check out the problem with software. Why smart engineers write bad code? It's available now from MIT press. And I'm going to have links to it in the show notes. This has been another episode of handsome in again next week.