The Evolution of ML and Furry Little Animals

Talking Machines


You are listening to talking machines Catherine Gorman Lawrence and Neil. We are again taping an episode in front of a live audience digitally recorded though on on talking machines. And if you want to be part of our live. Studio audience big quotes. You can follow us on twitter at Ti Okay. N. G. M. C. H. S. Or hit us up on the talking machines at and our guest today for this interview on talking. Machines is Dr Terence. Annouce key doctors and thank you so much for taking the time to join us today. I really appreciate it Great to be here so we ask all of our guests the same question I. How did you get where you are? What's been your academic and industrial journey. You're also very involved in the reps conference. Tell US everything well. A wise man once told me that careers are only made retrospectively and I have no idea how he got here. There was no plan. It went through a sequence of stages starting with graduate school at Princeton in theoretical physics. From there when I finished that I for reasons that have to do with the field of physics. At the time which was a little bit more bummed I went into neuroscience so that was a post doc and then from there that's when I met. Geoffrey Hinton and had changed my life because we met him at a small seminar here in San Diego and set nineteen seventy nine. We hit it off and From that over the next few years you know blossoms the the Boehner Sheen and back prop and you know. The rest was history. Terry who you post talking with where you post talking in San Diego no no. This was a post doc at Harvard. Medical School in the Department of Neurobiology with Stephen Kofler who was widely considered to be the founder of modern neurobiology and It was an experimental post. Doc I actually recorded from neurons. Subic seventy nine. You mentioning physics. It was a little bit more bond a in some sort of connection modeling. That was also a very quiet period. That wasn't a lot going on it. Was this sort of age of classical. Ai Right you're absolutely right. This was in fact. It was the neural network winter. The seventies and it was primarily because of the failure of the perception. That's neat because you say failure of the percents on I read about that a lot. Do you really did fail. All was the men's ski paper little. What the mid ski books are in Minsk. Eighty books have killed it but was it a fair representation. Well you know it's interesting. I think that that's the myth that that book killed it but I actually think that there are other things going on and and Rosenblatt had died as well which seems pretty significant. Yes well He. He was a pioneer. But you have to understand that digital computers were regally primitive back. Then you know that even the most expensive you know the biggest computers you could buy. Don't have the power of your wristwatch today. Rosenblatt actually had to build an analog device. It a million dollars in today's dollars to build a analog device that had potentially otters driven by motors for the weight sums the learning. Wasn't it potentially because you know digital computers? Were good at logic but they were terrible. Doing a floating point is amazing so he built that at Cornell. Right that's right yeah Funded by the owner. Any case by by the time that we were getting started computers was the vaccine era. It was becoming possible. Do Simulations You know they were small-scale by today's standards but but really meant we could explorer in a way that Frank Rosenblatt couldn't so what you're saying around the perceptual and so just forbid of context for Central and sixty one. Is that right? It was fifty nine. I think it was the book but you know it was in that era of early sixty zero and so then there's this period where the digital computer actually wasn't powerful enough to do much and then digital kind of overtook and divinity but these analog machines would just now impractical from a point of view of expense. So you're saying it's less the book and more of a shift to the Digital Machine. That in those early days wasn't powerful enough to simulate the perception. Yes so I I have you know. I have a feeling that history will show that A. I was like the blind man looking under the Lamppost. His keys and someone came along and said where did you lose your keys He said well somewhere else. But this is the only place right can see. I was reading Donald BACI quote. I recently At the beginning of his book about the I which is just a fascinating area and I guess he spent a lot of his career and he did work in in the wool on radar and he was talking about the Radio Club. Which is these early Cybernet assist and the potential of the analog or digital computer to be what represented the brain and his perspective was he. He was sure it wasn't a digital computer and he wasn't sure it was an analog computer either and he thought it was kind of somewhere in between but it feels like that in between is what you're saying is that was the difficult bit to look and perhaps a police were able to look now. That's right I you know. It's I think it's being driven. This is true all science that what you cannot understand is is really determined by the tools that you have for making measurements for doing simulations in it's really only this modern era that has given us enough tools both to make progress with understanding how the brain works and also with a because of the fact that we have a tremendous amount of power now but just to go back to that early era. I think you know I once asked L. Annual you know who is at Carnegie Mellon and it was a time when Geoff Hinton was an assistant professor and I was at Johns Hopkins and I you know he was at the first fifty six meeting at Dartmouth or a I was born and I I said well. Why was it that you didn't look at the brain and for for inspiration and he said well we did. But there wasn't very much known about the at the time to help us out so we just had make doing our own and he's right. That was a era. You know the the fifties was kind of the the beginning of what we now understand about the signals in the brain. Actually potential synoptic potentials. So you know in a sense. What what he was saying was that we basically use the tools we have available the time which was basically computers but what they were good at. What were they good at? They were good at logic at rules. A binary programming. So that you know that was In a sense they were forced to do that. That's a really. WanNa come back to nine hundred seventy nine in a moment but this is an interesting context to that because of course. Vena initially was someone who spread across. Both these areas of Norbert Vena who was at mit founded cybernetics spread across both these areas of the analog and digital he did his PhD thesis on Russell and Whitehead's book but one thing I was reading about recently is there was a big falling out between Vina. I'm McCulloch Pitts. And it's sort of interesting. That Vena wasn't there at the I. E. T. in fifty six and I sometimes wonder was that more about personalities and wanting this sort of old guard to stay away because you always feel veto with someone who who bridge these worlds it. You know that's the fascinating story. I actually wrote a review of a book about Warren McCulloch came up. They were friends. They actually had had been friends yet. It has something to do with their wife's. Yeah I think the lifestyle McCullough was not line with its a side story but but I guess the point you're making which I think is an I'd like us to take us back to seventy nine and the meeting with Jeff is and I think that that's true. Despite the story between humans the real factor that drove things then was the sudden available at a t of increasing cheap digital computer. And no longer the need to do this work that Rosenblatt and McCain and others had done having to wire together a bunch of analog circuits. That you couldn't reprogram to build system. Yeah I think that was a dead. End It for the very reason you gave. Which is that you know you. It's a special purpose device. That isn't good for anything else. And and really if you're trying to explore you need the flexibility of being able to try many ideas and that's in that really is a digital simulation allows you to

Coming up next