A highlight from #401: Migrating 3.8 Million Lines of Python


Ben, welcome to talk by the name. Thank you. Thank you so much for having me, Michael. We're going to talk a little bit of legacy code, a little bit of very, very large code bases, and how you might not have to permanently live in the past, which I think would be really welcome to a lot of people. I just talked a little bit about this before I hit record, but even though your topic is specifically how the story moved from python two to three, and this making your whole code base modern. I do think that this idea of how do I move from one code base to another code base is super relevant to lots of folks who might not be going from python two to three, but maybe from fast to fast API or vice versa or those types of things. So I think the techniques that you're going to talk about here are more broadly applicable than just a two to three migration and it's really cool how you all migrated 3.8 million lines of code without interrupting development. That's kind of nuts. Yeah, I did it and it still seems ridiculous. You love that and it seems like a dream. Amazing. Before we get to all that though, let's start with your story. How'd you get into programming and python? Took a job at Yelp. You all put a python chop before that. I had a couple of internships, and I went to Georgia Tech and I mostly did Java. So it was sort of a new experience for me. Python is one of those beginning languages that everyone loves to throw around. So I had done a dabbled in a little bit, but I first started really getting deep into the language when I started it at Yelp and I've definitely made it, it's sort of become like a sort of a local expert on it. So I've been able to build up a lot of knowledge about a lot of weird edge cases and stuff like that. You may be familiar with it. There's a T-shirt that's kind of a joke on me and it says, I learned python. It was a great weekend. Yeah. And yet, I've been doing python for many years, and I'm still learning new stuff, even today. I learned some interesting new python things. So which is it? Do you learn it in a day? Or is it this deep journey? I think most programming languages have some amount of width and depth. I think python definitely has the advantage of being a relatively straightforward language. One of the nice things obviously is that instead of using a lot of weird keywords, it has words like instead of being like, oh, I'm a pipe. Yeah, or yeah, or instead of double pipe and stuff like that. Those are the things that I think definitely help people personally, you know, me coming into it and what that wasn't like as big of a deal for me because I was already familiar with all that stuff, but you were coming from a very simple heavy world of Java, which is not assemble heavy as C++, but it's got a lot of abstractions in what it builds, for sure. Yeah. I will say, I think there are certain things that python does. I will say python is not like a perfect language bunny structure of the imagination, but I will say one thing about python that I think is really cool is it did sort of before many other languages managed to integrate a lot of functional paradigms like list comprehensions or comprehensions in general. I think one of those features where it's just like, this doesn't exist in a lot of other sort of more popular languages and really, really fluent and powerful in a way that you kind of miss when you don't have it, right? And so I think that that's something that's really cool about python. But python being a language with the legacy that it has, I mean, we're going to be talking about the two to three differences, which have their own nuances to them, but it's always going to have some weirdnesses to it. And some of those things are just like, oh, someone made a decision 30 years ago that still reverberates today. And so that means that it has this depth to it. You have to really learn the depth in order to fully understand all of the problems that exist. Like dealing with not necessarily problems with the language, but when you're building software, you run into problems. So you have to solve. And so that's like the main, I think that's true of all languages to an extent, especially popular ones and older ones. But yeah, I do think that python more than a lot of languages really is able to straddle the line of being like, oh, it's approachable, but also you can do a lot of really interesting and powerful stuff with it. Yeah, you compare that with Java. Java, you've got to understand functions. You've got to understand classes, possibly namespaces. Just to write the first line of code. Whereas python, you can work with it for like, you know what? This is really clumsy to repeat this. Maybe I'll learn what a function is. And then I can start using that, but you don't know what a class is. You don't care about it. You kind of slowly layer on the stuff as you need it rather than you've got to jump in and go with it all at once. Yeah, for sure. Yeah. So you're still at Yelp. And I'm in a farmer's room in our San Francisco office right now. Excellent. And what are you doing there? So I work on a team that's called core services. Our team is responsible for a lot of infrastructure, mostly python focused, though not exclusively. We do a lot of our internal python infrastructure. So we'll be responsible for making sure that we can upgrade to new python versions.

Coming up next