ud675 »

Contents

Definition of ML

  • Hi, Michael.
  • Hey, Charles, how's it going?
  • It's going quite well. How's it going with you?
  • Good. Good.
  • Good. Good. So today, I thought we would talk a little bit about the philosophy of machine learning.
  • Ooh. I hate philosophy.
  • I don't like it much either, although I'm a doctor of philosophy.
  • Oh, that's very impressive.
  • Aren't you a doctor of philosophy, too?
  • I am. It's kind of impressive.
  • It is. It is kind of impressive. So what we wanted to get across today was a little bit about why the class is structured the way it is. What the different parts are, and maybe go a little bit of back and forth about what we think you should be getting out of the course. That seem reasonable?
  • Sure.
  • Okay. Well so first off, by the way, before we get started, I wanted to thank you for coming down to Atlanta and joining me in these beautiful studios.
  • Well it's, it's very nice to be here. Thank you for inviting me.
  • Oh, no, no. Thank you for coming, Michael.
  • Thank you for asking me to do the course, this has been a lot of fun.
  • Oh, the whole point was to be able to do the course with you, Michael We like each other, and that's one of the things that we want you to get, want to get across in this class, because we like machine learning and lot of stuff in common, but I'm not sure we completely agree on the most important parts of machine learning and why we do the things that we do.
  • Hm. Alright.
  • So I think people in the outside world Michael would claim that you're more theoretical than I. But I am?
  • In theory.
  • In theory. And I'm more practical than you are.
  • Practically.
  • At least in practice. And hopefully some of that tension will come out in the class. But I think in order to see why that tension works that way, you have to understand what machine learning is. So, Michael.
  • Right.
  • What's machine learning?
  • It's about proving theorems.
  • [LAUGH] No.
  • No.
  • I would not say it's about proving theorems, although proving theorems is often important in machine learning.
  • I agree with that
  • Okay.
  • So we're on the same page.
  • We're partially on the same page. What is machine learning?
  • What is machine learning? Right.
  • Give me a definition.
  • So, it is computational statistics. How's that for a definition?
  • That is a definition. It is wrong on so many levels. However, a lot of people would agree with that statement. They would say that machine learning is really just applied statistics.
  • Not applied statistics, computational statistics.
  • Computationally applied statistics.
  • Psh.
  • I don't like that definition. I think that it's a bit too narrow. I think that machine learning is about this broader notion of building artifacts, computational artifacts typically that learn over time based on experience. And then in particular, it's not just the act of building these artifacts that matter, it's the math behind it, it's the science behind it, it's the engineering behind it, and it's the computing behind it. It's everything that goes into building intelligent artifacts that almost by necessity, have to learn over time. You buy that?
  • Yeah, so you, you have data, and you do analysis of the data and try to glean things from the data and you use various kinds of computational structure to do that. So, computational statistics.
  • I don't think that's computational statistics.

Supervised Learning

  • This class is divided into three subclasses
  • Mm-hm.
  • Three parts. They are supervised learning
  • Yeah.
  • Unsupervised learning, and reinforcement learning. So what do you think supervised learning is?
  • So I think of supervised learning as being the problem of taking labeled data sets Gleaning information from it so that you can label new data sets.
  • That's fair. I call that function approximation.
  • Mm.
  • So, here's an example of supervised learning. I'm going to give you an input and an output, and I'm going to give them to you as pairs, and I want you to guess what the function is.
  • Sure
  • Okay? Okay. 1:1.
  • Uh-huh. Two four.
  • Wait. Hang on. Is one input and one the output?
  • Yes.
  • And two the input and four the output?
  • Correct.
  • Alright, I'm on, I think I'm on to you.
  • Three.
  • Uh-huh.
  • Nine.
  • okay.
  • Four. Sixteen.
  • Nice.
  • Five. Twenty five. Six. Thirty six. Seven. Forty nine.
  • Nice. This is a very hip data set.
  • It is. What's a [UNKNOWN]?
  • It's hip to be squared.
  • Exactly. Maybe. Oh.
  • So if you believe that's true, then tell me if the input is ten, what's the output?
  • 100.
  • And that's right, if it turns out, in fact, that the function is X squared. But the truth is we have no idea where's the function's X squared, not really.
  • I have a pretty good idea.
  • You do?
  • Well, just.
  • Where did that idea come from?
  • It comes from having spoken with you over a long period of time. And plus you know, math.
  • And plus math, well, I'm going to
  • You can't say I'm wrong.
  • You're wrong.
  • uhh, you said I was wrong.
  • No, you've talked to me for a long time and plus math, I agree with that, but I'm going to claim that you're making a leap of faith.
  • Mm-mm
  • Despite being a scientist, by deciding that the input is 10 and the output is 100.
  • Sure, I would agree with that.
  • And what's that leap of faith?
  • Well, I mean, from what you told me, it's still consistent with lots of other mappings from input to output like ten gets mapped to 11.
  • Right, or everything's x squared except ten.
  • Sure.
  • Or everything's x, x squared up to ten.
  • Right, that would be mean.
  • That would be mean.
  • But it's not logically impossible. What'll be the medium.
  • [SOUND]
  • Thank you very much. I was saving that one up.

Induction and Deduction

  • Well, what you're doing in order to make that work, and what you end up doing in supervised learning and function approximation in general, is you make some fundamental assumptions about the world. Right? You decide that you have a well-behaved function that is consistent with the data that you're getting. And with that, you're able to generalize. And, in fact, that is the fundamental problem in Machine Learning. It is generalization. Now what's behind all of this, I'm going to claim. Michael, you jump in whenever you disagree.
  • I disagree! Oh, sorry. Too soon. Go ahead.
  • Is bias.
  • Bias.
  • And in particular, inductive bias.
  • Inductive bias.
  • Right. So all of Machine Learning, or certainly Supervised Learning, is about induction. As opposed to deduction
  • I see, induction of course being a problem of going from examples to a more general rule.
  • Right specifics to generalities. By contrast deduction is?
  • Would be the opposite. It would be going from a general rule to specific instances, basically like reasoning. Right, in fact a lot of the AI in the beginning was about productive reasoning about logic, programming, those sort of things where you have certain rules and you do only those things that follow mainly in those rules. For example you have A implies B, that's a rule in universe and then I tell you A so if you know A implies B in universe and I tell you A then you also know. That A implies b.
  • And therefore, you can.
  • And a.
  • Infer that.
  • B.
  • B, you have a implies b, you have a. That implies b.
  • Okay.
  • It's what we just said. That's deduction.
  • That's deduction but what we just did was not deduction. Before then when I asked you
  • That was induction.
  • Induction is more about did the Sun rise yesterday?
  • Yes.
  • Did the Sun rise the day before that?
  • Yes.
  • Did the Sun rise the day before that?
  • [UNKNOWN]. Did the sun rise the day before that?
  • Yes.
  • Yes. So the sun has risen every day. Is the sun going to rise tomorrow?
  • I sure hope so.
  • We all hope so. And we all act like it does because if it doesn't, then there are a whole bunch of other things we ought to be doing besides sitting in this studio and having this interview.
  • I think we should warn the plants.
  • [LAUGH] I don't think the plants are going to care.
  • They are! They really need sun. I think we all need some, Michael. So the idea there is that induction is crucial, and that inductive bias is crucial. And we'll talk about all this in the course. But that's kind of fundamental notions behind supervised learning and machine learning in general.
  • I agree with that. Yeah.
  • All right, so we're on the same page. So that's supervised learning. Supervised learning, you can talk about in these high muckety-muck ways. But at the end of the day It's function approximation. It's figuring out how to take a bunch of training examples, coming in with some function that generalizes beyond the data you see.
  • So why wouldn't you call it function induction then?
  • Because someone said supervised learning first. Well there is a difference.
  • No, no, no, no you said supervised learning is function approximation and I want to say supervised learning is. Function induction?
  • As opposed to function approximation?
  • Yeah.
  • Okay. It's
  • Approximate function induction.
  • Or induction of approximate, of.
  • Approximate functions.
  • Something like that, yeah.
  • You don't want to induce an approximate function, you want to induce the actual function.
  • Yeah, but sometimes you can't.
  • Yeah.
  • Because sometimes you think it's quadratic, but it's not. I have that as a plaque on my wall.
  • You do?
  • No.
  • Yeah, I didn't think so. Okay, so that's supervised learning.

Unsupervised Learning

  • What about unsupervised learning?
  • Right, so unsupervised learning we don't get those examples. We have just essentially something like inputs and we have to derive some structure from them just by looking at the relationship between the inputs themselves.
  • Right, so give me an example of that.
  • So, when you are studying different kinds of animals say, even as a kid.
  • Mm-hm.
  • You might start to say oh, there's these animals that all look kind of the same, they're four legged. I'm going to call all of them them dogs. Even if they happen to be horses or cows or whatever, but I have developed, without anyone telling me, this sort of notion that all these belong into the same class and it's different from things like trees.
  • Which don't have 4 legs.
  • Well some do, but I mean they have, they both bark, is all I'm saying.
  • [LAUGH] Did I really set you up for that?
  • Not on purpose.
  • I'm sorry, I want to apologize to each and every one of you for that. But that was pretty good. Michael is very good at word play, which I guess is often unsupervised as well.
  • No, I get a lot of supervision! [LAUGH]
  • You certainly get a lot of feedback.
  • Yeah, that's right. It's like, please stop doing that.
  • So if supervised learning is about function approximation, then unsupervised learning is about description. It's about taking a set of data and figuring out how you might divide it up in one way or the other.
  • Or maybe even summarization. It's not just a description, but it's a shorter description.
  • Yeah, it's usually a concise. Compression.
  • Compact, description. So I might take a bunch of pixels like I have here, and might say, male.
  • [LAUGH] Wait, wait, wait, wait. I am pixels now?
  • As far as we can tell.
  • That's fine.
  • I however, am not pixels. I know I am not pixels. I am pretty sure that rest of you are pixels.
  • That's right.
  • So, I have a bunch of pixels and I might say, male. And or I might say female, or I might say dog, or I might say tree, but the point is, I don't have a bunch of labels that say dog, tree, male or female. I just decide that pixels like this belong with pixels like this, as opposed to pixels like something else that I'm pointing to behind me.
  • Yeah, we're living in a world right now that is devoid of any other objects. Oh, chairs.
  • Chairs right?
  • Chairs.
  • So these pixels are very different from those pixels, because of where they are relative to the other pixels, say. Right? So if you were looking.
  • I'm not sure that's helping me understand unsupervised learning.
  • Go out and, go outside and look at a crowd of people and try to decide how you might divide them up. Maybe you'll divide them up by ethnicity, maybe you'll divide them up by whether they have purposely shaven their hair in order to mock the bald, or whether they have curly hair. Maybe you'll divide them up by whether they have goatees.
  • Facial hair.
  • Or whether they have grey hair. There's lots of things that you might do in order to
  • Did you just point at me and say grey hair?
  • I was pointing and your head happened to be there.
  • Pixels it's, its a two dimensional.
  • Oh, come on, where's the grey hair?
  • Right there. It's right where your spit curl is.
  • All right.
  • Okay, so, imagine you're dividing the world up that way. You could divide it up male/female, you could divide it up short/tall, wears hats/doesn't wear hats. All kinds of ways you can divide it up, and no one's telling you the right way to divide it up, at least not directly. That's unsupervised learning, that's description, because now,
  • Mm.
  • Rather than having to send pixels of everyone, or having to do a complete description of this crowd, you can say there were 57 males, and 23 females, say. Or, there were mostly people with beards, or whatever.
  • I like summarization for that.
  • I like summarization for that, it's a nice concise description. That's unsupervised learning.
  • > Good, very good. And it's different than supervised learning.
  • In fact. It's different than supervised learning. And it's different in a couple of ways. One way that it's different is, all of those ways that we could have just divided up the world, in some sense they're all equally good. So I could divide up by sex. Or I could divide up by height. Or I can divide up by clothing or whatever and they're all equally good, absent some other signal later telling you how you should be dividing up the world. But supervised learning directly tells you there's a signal, this is what it ought to be, and that's how you train. Those are very different.
  • Now, but I can ways that unsupervised learning could be helpful in the supervised setting. Alright, so if I do get a nice description and it's the right kind of description, it might help me map to, it might help me do the function approximation better.
  • Right. So instead of taking pixels at input, as input and then labels like male or female, I can just simply take a summarization of you like, how much hair you have, your relative height to weight. And various things like that that might help me do it. That's right. And by the way, in practice, this tends to turn out to be things like density estimation. We do end up turning it into statistics at the end of the day. Often.
  • It was statistics from the beginning. But when you say density estimation.
  • Yes.
  • Are you saying I'm stupid?
  • No.
  • Alright, so what is density estimation?
  • Well, they'll have to take the class to find out.
  • I see.
  • Okay.

Reinforcement Learning

  • All right. So, that's supervised learning and unsupervised learning. That's pretty good. The last one is reinforcement learning.
  • Whoo!
  • Now, reinforcement learning is what we both do. So, Michael does a little bit of reinforcement learning here and there. You've got how manypapers published in reinforcement learning?
  • All of them.
  • [LAUGH]
  • Several. I have several The man has like 100 papers on reinforcement learning. . In fact, he wrote, with his colleagues, the great summary journal article bringing everyone up to date what reinforcement learning was like back in 19?
  • Like 112 years ago.
  • 1992 or something like that?
  • People were saying, yeah we should probably, somebody should write a new one because the other one is getting a little long in the tooth.
  • But there have books written on machine learning since then.
  • That's right.
  • It's a very popular field. That's why we're both in it. Michael tends to prove a lot of things.
  • It is not, that is not why I'm in it.
  • Why, I'm, wait, what?
  • You said it's a very popular field, and that's why we're in it.
  • No, no, no, no, no, no. Did I say that?
  • That's what I heard.
  • I didn't mean to say that.
  • Ooooh! Let's run it back, let's see.
  • Yeah, let's do that again. Because I did not mean to say that. It is a very popular field. Perhaps because you're in it, Michael.
  • I don't think that's it. When I was an undergraduate I thought, the thing that I really want to understand, I liked AI, I liked the whole idea of AI. But what I really want to understand is, how can you learn to be better from experience? Like, I built a tic-tac-toe playing program. And like, I want this tic-tac-toe playing program to get really good at tic-tac-toe. Because I was always interested in the most practical, society impacting problems.
  • Tic-tac-toe generalizes pretty well to world hunger.
  • Eventually. So that is what got me interested in it. And I was, I didn't even know what it was called for a long time. So I started doing reinforcement learning and then discovered that it was interesting and popular,
  • Right. Well. I certainly wouldn't suggest we are doing the science that we are doing because it is popular. We are doing it because we are interested in it.
  • Yes.
  • And I'm interested in reinforcement learning because, in some sense it kind of encapsulates all the things I happen to care about. I come from a sort of general AI background, and I care about modeling people. I care about building smart agents that have to live in a world with other smart agents, thousands of them, hundreds of thousands of them, thousands of them. Some of whom might be human. And I have to field some way to predict what to do over time. So, from a sort of technical point of view, if we can think of supervised learning as function approximation, and unsupervised learning as, you know, concise, compact description, what's the difference between something like reinforcement learning and those two? Supervised learning in particular.
  • > So, often the that supervised learning, sorry, reinforcement learning is described is as learning from delayed reward. So instead of the feedback that you get in supervised learning, which is, here's what you should do, and the feedback you get in unsupervised learning, which is [INAUDIBLE], the feedback in reinforcement learning may come several steps after the decisions that you've actually made. So, a good example of that, or an easy example of that would be actually your tic-tac-toe program, right? So, you do something in tic-tac-toe. You put an X in the center and then you put let's say, an O over here.
  • Ooh.
  • And then I put an X right here.
  • Nice.
  • And then you ridiculously put an O in the center.
  • Aw, come on.
  • Which allows me to put an X over here. And I win.
  • All right.
  • Now what's interesting about that is, I didn't tell you what happened until the very end, when I said, X wins.
  • Right, and now I know I made a mistake somewhere along the way, but I don't know exactly where. I may have to kind of roll back the game in my mind and eventually figure out where it is that I went off track, and what it is that I did wrong.
  • And in the full generality of reinforcement learning, you may have never made a mistake. It may simply be that, that's the way games go. But you would like to know which of the moves you made mattered. Now, if it were a supervised learning problem, I would have put the X here, you would put the O there, and would have been told, that's good. I would have put the X here, and then when you put the O there, it would have been, that's bad. The O goes here.
  • Or something like that, or were told where he should have put the O. But, here all he gets is, eventually some kind of signal saying, you did something well, you did something poorly. And even then it's only relevant to the other signals that you might have gotten.
  • Right, so then reinforcement learning is, in some sense, harder, because nobody's telling you what to do. You have to work it out on your own.
  • Yeah, it's like playing a game without knowing any of the rules. Or at least knowing how you win or lose. But being told every once in awhile that you've won or you've lost. Okay, now.
  • Sometimes I feel like that.
  • I know, man.

Comparison of these parts of ML

  • Okay, so, we have got three little bits of a machine line here and there are lot of tools and techniques that are inside that.
  • Mm-hm.
  • And I think that's great and we are going to be trying to teach you a lot of those tools and techniques and sort of ways to connect them together. So, by the way, as Michael is pointing out, there are kind of ways that these might help each other. Unsupervised learning might help supervised learning, it's actually much deeper than that. It turns out you, even though unsupervised learning is clearly not the same as supervised learning, at the level we've described it, in some ways they're exactly the same thing. Supervised learning you have some bias, oh it's a quadratic function, induction makes sense, all these kind of assumptions you make. And in unsupervised learning, I told you that we don't know whether this cluster's better than this cluster, dividing by sex is better than dividing by height, or or hair color or whatever. But ultimately, you make some decision about how to cluster, and that means implicitly there's some assumed set. There's some assumed set of labels that you can possess. Oh, I think things that look alike should somehow be clustered. Things that are near one another should be clustered together. So, in some ways, it's still kind of like supervised learning. You can certainly turn any supervised learning problem into an unsupervised learning problem.
  • Mm, mm.
  • Right. So in fact, all of these problems are really the same kind of problem.
  • Yeah, well there's two things that I'd want to add to that. One is that, in some sense, in many cases, you can formulate all of these different problems as some form of optimization. In supervised learning, you want something that, that labels data well, and so your, the thing you're trying to optimize is, find me a function that, that does that. We're going to score it. In reinforcement learning, we're trying to find a behavior that scores well. And in unsupervised learning, we usually have to make up some kind of a criterion, and then we find a way of clustering the data, organizing the data so that it scores well. So that was the first point I wanted to make. The other one is. If you divide things by sex and you're a virgin, then there's numerical instability issues.
  • Did you learn about that on the street?
  • I learned in the math book.
  • That's ugh. I, I'm going to move on with her. So here's the thing.
  • Alright.
  • Everything Michael just said, except the last part, is true. But there's actually a sort of deeper thing going on, to me. If you think about the commonalities of everything we just said, it boils down to one thing, data. Data, data, data, data, data. Data is king in machine work. Now Michael would call himself a computer scientist.
  • Oh, yeah.
  • And I would call myself a computationalist.
  • What?
  • I'm in a college of computing at a Department of Computer Science. I believe in computing and computation as being the ultimate thing. So I'd call myself a computationalist and Michael would probably agree with that just to keep this discussion moving.
  • Let's say.
  • Right, so we're computationalists, we believe in computing. That's a good thing.
  • Sure.
  • Many of our colleagues, who do computation, think in terms of algorithms. They think in terms of, what are the series of steps I need to do in order to solve some problem?
  • I think about algorithms.
  • Or they might think in terms of theorems. If I try to describe this problem in a particular way, is it solvable implicitly by some algorithm?
  • Yeah.
  • And truthfully, machine learning is a lot of that. But the difference between the person who's trying to solve a problem as an AI person or as a computing person and someone who's trying to solve a problem as a machine learning person is that the algorithm stops being central, the data starts being central. And so what I hope you get out of this class or at least part of the stuff that you do, is understanding that you have to believe the data, you have to do something with the data, you have to be consistent with the data. The algorithms that fall out of all that are algorithms but they're algorithms that take the data as primary. Or at least important.
  • I'm going to go with coequal.
  • So algorithms and data are coequal.
  • Coequal.
  • Well, if you believe in lists they the same thing.
  • Exactly.
  • They knew back in the seventies.
  • So it turns out we do agree on those things.
  • Whew that was close.
  • Excellent. So, the rest of the semester will go exactly like this except you won't see us. You'll see our hands though
  • This side, this side. There you go.
  • You'll see our hands though. Thank you Michael
  • S'alright.
  • [LAUGH] Well.
  • What? [LAUGH]
  • That was good. That took me back to when I was four.
  • Señor, Señor Wences.
  • Hm?
  • It's called Señor Wences.
  • Yes I know. I remember that.
  • Yeah, okay. Mm-hm.
  • I'm not that much younger than you. Ten, 12 years old.
  • Come on.
  • You count gray hairs. Anyway, the point is the rest of the semester will go like this. We will talk about supervised learning and a whole series of algorithms, step back a little bit and talk about the theory behind them. And try to connect theory of machine learning with theory of computing notions. Or at least that kind of basic idea. What does it mean to be a hard problem versus an easier problem. We'll move into randomized optimization and unsupervised learning. Where we will talk about all the issues that we brought up here and try to connect them back to some of the things that we did in the section on supervised learning. And then finally, we will spend our time on reinforcement learning and the generalization of these traditional reinforcement learning which involves multiple agents. So we'll talk about a little bit.
  • Hm.
  • Of game theory which Michael loves to talk about, I love to talk about. And the applications of all the stuff that we've been learning To solving problems. How to actually act in the world. How to build that robot to do something, or build that agent to play a game, or to teach you how to do whatever you need to be taught how to do. But at the end of the day we're going to teach you to think about data, how to think about algorithms, and how to build artifacts that you know will learn.
  • Let's do this thing.
  • Excellent. Alright, well thank you Micheal.
  • Sure.
  • I will see you next time we're in the same place, at the same time.