Railyard, Jason, Netease discussed on Software Engineering Daily

Automatic TRANSCRIPT

You know, my algorithm is like I want my false positive rate to be this, and after that maximize recall and figuring out also like, in some cases, we kind of compose models, like, you know, try and run this one, but fall back to that one all the stuff in the API. I think that really it's been like a really good force, and kind of having this automatic retraining on top of it to have sort of, like codify, our valuation metrics, because you kind of have to do that, if you want to automate, all of the stuff and now we have. Kind of like a little bit of an evaluation API that we've built as well that, rob. Probably say more about too, is when I think of an API. I think of something that is kind of an imperative interface. Like I'm executing an API right now. Like, I'm making API request computer. Go do something, the coober Netease way of things is more of a declarative like this should be the state of the world at all times. Tell me a perspective on declarative versus imperative. I realize it's a sliding scale it's kind of semantic, but I mean you could have made you could have done this declared. If you could have said like railyard is a declarative way of describing when and how you want to run your models. But instead, you said here is an API you make requests to it was was that deliberate or like told me about declarative versus imperative in your mind it was pretty deliberate. And it was it was very driven by the product teams in that they were very interested in colleagues API, I write they wanted this API to resist because. They wanted to build in a sort of the idea of we want to retrain, these models in evaluate them and then decide whether or not to put them in the production, they, they wanted to own that logic, because that is sort of their logic to own as, as a product team. They're the ones making the decisions about is this model better than the last one showed this model go into production that sort of thing. So from their perspective, what they really want. It was in the eye that they'd call like sort of dynamic API declared peace interesting questions so on Cuba, Netease. You know, we use like the Cuban daddy's jobs, interface in, there are pieces of that, that are declared it, right. You know, we sort of have this, this pile of Yambol that describes how we want to run the job. Right. So, you know, to run a job on Cooper, daddy's, you have this gamble description of I wanna run this job with these resources with this Docker file. Right. So that is very declared if and we build up one of these declared a specifications when we run the job within that job within that. Docker container where executing a bunch of imperative python code. Right. So it's sort of these pieces all work together. So we make API calls. We make this or this dynamic she API call with Jason specification for how we want to train the job. We build this declared a spec to Tokyo Bonetti's run this job with a Stocker file. And then we pass through this, Jason specification, too, I thought goad, which imperatively decides, you know, how to train the model of what data defect based on the specification. So it's sort of like a nice blend of the two. Like, we're, we're sort of like route were sort of using a little bit of declared if pieces in building our daddy's job on this specifying the Docker container away, also had this dynamic piece this API that are brought teams are using to, you know, sort of trade models on their terms of actively taking a step back. How did you come upon the insight of giving this unified experience through an API because, like it's, it's not necessarily like this is not. I just want to understand your thought process, a little bit better because this is not necessarily like a like, you know, aliens to be leads to see, you know, kind of insight this is a inside of there's all this disconnected stuff that's going on in the machine learning developers workflow..

Coming up next