Good. Good. So today, I thought we would
talk a little bit about the philosophy of machine learning.
Ooh. I hate philosophy.
I don't like it much either, although I'm a doctor of philosophy.
Oh, that's very impressive.
Aren't you a doctor of philosophy, too?
I am. It's kind of impressive.
It is. It is kind of impressive. So
what we wanted to get across today was a
little bit about why the class is structured the
way it is. What the different parts are, and maybe
go a little bit of back and forth about what we
think you should be getting out of the course. That seem reasonable?
Okay. Well so first off, by the way, before we get started, I wanted
to thank you for coming down to
Atlanta and joining me in these beautiful studios.
Well it's, it's very nice to be here. Thank you for inviting me.
Oh, no, no. Thank you for coming, Michael.
Thank you for asking me to do the course, this has been a lot of fun.
Oh, the whole point was to be able
to do the course with you, Michael We like each
other, and that's one of the things that we
want you to get, want to get across in this
class, because we like machine learning and lot of
stuff in common, but I'm not sure we completely
agree on the most important parts of machine learning
and why we do the things that we do.
So I think people in the outside world Michael
would claim that you're more theoretical than I. But I am?
In theory. And I'm more practical than you are.
At least in practice. And hopefully some of that
tension will come out in the class. But I think in
order to see why that tension works that way,
you have to understand what machine learning is. So, Michael.
What's machine learning?
It's about proving theorems.
I would not say it's about proving theorems,
although proving theorems is often important in machine learning.
I agree with that
So we're on the same page.
We're partially on the same page. What is machine learning?
What is machine learning? Right.
Give me a definition.
it is computational statistics. How's that for a definition?
That is a definition. It is wrong on so many levels. However, a lot of
people would agree with that statement. They would
say that machine learning is really just applied statistics.
Not applied statistics, computational statistics.
Computationally applied statistics.
I don't like that definition. I think that it's a
bit too narrow. I think that machine learning is about this broader
notion of building artifacts, computational artifacts typically that
learn over time based on experience. And then in
particular, it's not just the act of building these
artifacts that matter, it's the math behind it, it's
the science behind it, it's the engineering behind
it, and it's the computing behind it. It's everything
that goes into building intelligent artifacts that almost by
necessity, have to learn over time. You buy that?
Yeah, so you,
you have data, and you do analysis of the data and try to glean things from
the data and you use various kinds of
computational structure to do that. So, computational statistics.
I don't think that's computational statistics.
This class is divided into three subclasses
Three parts. They are supervised learning
Unsupervised learning, and reinforcement learning. So
what do you think supervised learning is?
So I think of supervised learning as being the problem of taking labeled
data sets Gleaning information from it so that you can label new data sets.
I call that function approximation.
So, here's an example of supervised learning. I'm
going to give you an input and an output, and
I'm going to give them to you as pairs, and
I want you to guess what the function is.
So if you believe that's true, then tell
me if the input is ten, what's the output?
And that's right, if it turns out, in fact, that the function is X
squared. But the truth is we have no
idea where's the function's X squared, not really.
I have a pretty good idea.
Where did that idea come from?
comes from having spoken with you over a
long period of time. And plus you know, math.
And plus math, well, I'm going to
You can't say I'm wrong.
uhh, you said I was wrong.
No, you've talked to me for a long time and plus math, I
agree with that, but I'm going to claim that you're making a leap of faith.
Despite being a scientist, by deciding that
the input is 10 and the output is 100.
Sure, I would agree
And what's that leap of faith?
Well, I mean, from what you told me, it's still consistent with
lots of other mappings from input to output like ten gets mapped to 11.
Right, or everything's x squared except ten.
Or everything's x, x squared up to ten.
Right, that would be mean.
That would be mean.
But it's not logically impossible. What'll be the medium.
Thank you very much. I was saving that one up.
Induction and Deduction
Well, what you're doing in order to make that work, and
what you end up doing in supervised learning and function approximation
in general, is you make some fundamental assumptions about the world.
Right? You decide that you have a well-behaved function that is
consistent with the data that you're getting. And with that, you're
able to generalize. And, in fact, that is the fundamental problem
in Machine Learning. It is generalization. Now what's behind all of
this, I'm going to claim. Michael, you jump in whenever you disagree.
I disagree! Oh,
sorry. Too soon. Go ahead.
And in particular, inductive bias.
Right. So all of Machine Learning, or certainly
Supervised Learning, is about induction. As opposed to deduction
I see, induction of course being a problem
of going from examples to a more general rule.
Right specifics to generalities. By contrast deduction is?
Would be the opposite. It would be
going from a general rule to specific instances,
basically like reasoning. Right, in fact a lot of the
AI in the beginning was about productive reasoning about logic,
programming, those sort of things where you have certain rules
and you do only those things that follow mainly in those
rules. For example you have A implies B, that's a
rule in universe and then I tell you A so
if you know A implies B in universe and I
tell you A then you also know. That A implies b.
therefore, you can.
B, you have a implies b, you have a. That implies b.
It's what we just said. That's deduction.
That's deduction but what we just did
was not deduction. Before then when I asked you
That was induction.
Induction is more about did the Sun rise yesterday?
Did the Sun rise the day before that?
Did the Sun rise the day before that?
Did the sun rise the day before that?
Yes. So the sun has risen every day. Is the sun going to rise tomorrow?
I sure hope so.
We all hope so. And we all act
like it does because if it doesn't, then there are
a whole bunch of other things we ought to be
doing besides sitting in this studio and having this interview.
I think we should warn the plants.
[LAUGH] I don't think the plants are going to care.
They are! They really need sun.
I think we all need some, Michael. So the idea there
is that induction is crucial, and that inductive bias is crucial.
And we'll talk about all this in the course. But that's
kind of fundamental notions behind supervised
learning and machine learning in general.
I agree with that. Yeah.
All right, so we're on the same page. So that's supervised learning.
Supervised learning, you can talk about in these high muckety-muck ways. But at
the end of the day It's function approximation. It's figuring out how to
take a bunch of training examples, coming in with some function that generalizes
beyond the data you see.
So why wouldn't you call it function induction then?
Because someone said supervised learning
first. Well there is a difference.
No, no, no, no you said supervised learning is
function approximation and I want to
say supervised learning is. Function induction?
As opposed to function approximation?
Approximate function induction.
Or induction of approximate, of.
Something like that, yeah.
You don't want to induce
an approximate function, you want to induce the actual function.
Yeah, but sometimes you can't.
Because sometimes you think it's quadratic, but it's
not. I have that as a plaque on my wall.
Yeah, I didn't think so. Okay, so that's supervised learning.
What about unsupervised learning?
Right, so unsupervised learning we don't get those
examples. We have just essentially something like inputs and
we have to derive some structure from them just
by looking at the relationship between the inputs themselves.
Right, so give me an example of that.
So, when you are studying different kinds of animals say, even as a kid.
You might start to say oh, there's these animals that all look kind of the
same, they're four legged. I'm going to call all
of them them dogs. Even if they happen to
be horses or cows or whatever, but I
have developed, without anyone telling me, this sort
of notion that all these belong into the
same class and it's different from things like trees.
Which don't have 4 legs.
Well some do, but I mean they have, they both bark, is all I'm saying.
[LAUGH] Did I really set you up for that?
Not on purpose.
I'm sorry, I want to
apologize to each and every one of you for that. But that was pretty good.
Michael is very good at word play, which I guess is often unsupervised as well.
So if supervised learning is about function approximation,
then unsupervised learning is about description. It's about taking
a set of data and figuring out how you
might divide it up in one way or the other.
Or maybe even summarization. It's not just
a description, but it's a shorter description.
Yeah, it's usually a concise. Compression.
Compact, description. So I might take a bunch
of pixels like I have here, and might say, male.
[LAUGH] Wait, wait, wait, wait. I am pixels now?
As far as we can tell.
I however, am not pixels. I know I am not
pixels. I am pretty sure that rest of you are pixels.
So, I have a bunch of pixels and I might say, male. And or I
might say female, or I might say dog, or I might say tree, but the point
is, I don't have a bunch of labels that say
dog, tree, male or female. I just decide that pixels
like this belong with pixels like this, as opposed to
pixels like something else that I'm pointing to behind me.
Yeah, we're living in a world right now
that is devoid of any other objects. Oh, chairs.
So these pixels are very different from those pixels, because of where
they are relative to the other pixels, say. Right? So if you were looking.
I'm not sure that's helping me understand unsupervised learning.
Go out and, go outside and
look at a crowd of people and try to decide how you might divide
them up. Maybe you'll divide them up
by ethnicity, maybe you'll divide them up by
whether they have purposely shaven their hair in order to mock the bald, or
whether they have curly hair. Maybe you'll
divide them up by whether they have goatees.
Or whether they have grey hair. There's lots
of things that you might do in order to
Did you just point at me and say grey hair?
I was pointing and your head happened to be there.
Pixels it's, its a two dimensional.
Oh, come on, where's the
Right there. It's right where your spit curl is.
Okay, so, imagine you're dividing the world up that way. You could divide
it up male/female, you could divide it
up short/tall, wears hats/doesn't wear hats. All kinds
of ways you can divide it up, and no one's telling you the right way
to divide it up, at least not
directly. That's unsupervised learning, that's description, because now,
Rather than having to send pixels of everyone, or having
to do a complete description of this crowd, you can say there
were 57 males, and 23 females, say. Or,
there were mostly people with beards, or whatever.
I like summarization for that.
I like summarization for that, it's
a nice concise description. That's unsupervised learning.
> Good, very good. And it's different than supervised learning.
In fact. It's different than supervised learning. And it's
different in a couple of ways. One way that it's different
is, all of those ways that we could have just
divided up the world, in some sense they're all equally good.
So I could divide up by sex. Or I could divide up by height. Or I can
divide up by clothing or whatever and they're
all equally good, absent some other signal later telling
you how you should be dividing up the
world. But supervised learning directly tells you there's a
signal, this is what it ought to be,
and that's how you train. Those are very different.
Now, but I can ways that unsupervised
learning could be helpful in the supervised setting.
Alright, so if I do get a nice
description and it's the right kind of description,
it might help me map to, it might help me do the function approximation better.
Right. So instead of taking pixels at input, as
input and then labels like male or female, I can
just simply take a summarization of you like, how much
hair you have, your relative height to weight. And various things
like that that might help me do it. That's right.
And by the way, in practice, this tends to turn out
to be things like density estimation. We do end up
turning it into statistics at the end of the day. Often.
It was statistics from the beginning. But when you say
Are you saying I'm stupid?
Alright, so what is density estimation?
Well, they'll have to take the class to find out.
All right. So, that's supervised learning and unsupervised learning.
That's pretty good. The last one is reinforcement learning.
Now, reinforcement learning is what we both do. So, Michael does a little
bit of reinforcement learning here and there.
You've got how manypapers published in reinforcement learning?
All of them.
Several. I have several The man
has like 100 papers on reinforcement learning.
. In fact, he wrote, with his
colleagues, the great summary journal article bringing
everyone up to date what reinforcement learning was like back in 19?
Like 112 years ago.
1992 or something like that?
People were saying, yeah we should probably, somebody should write a new
one because the other one is getting a little long in the tooth.
But there have books written on machine learning since then.
It's a very popular field. That's why we're both
in it. Michael tends to prove a lot of things.
It is not, that is not why I'm in it.
Why, I'm, wait, what?
You said it's a very popular field, and that's why we're in it.
No, no, no, no, no, no. Did I say that?
That's what I heard.
I didn't mean
to say that.
Ooooh! Let's run it back, let's see.
Yeah, let's do that again. Because I did not mean to say
that. It is a very popular field. Perhaps because you're in it, Michael.
I don't think that's it. When I was
an undergraduate I thought, the thing that I really want
to understand, I liked AI, I liked the whole
idea of AI. But what I really want to understand
is, how can you learn to be better from
experience? Like, I built a tic-tac-toe playing program. And like,
I want this tic-tac-toe playing program to get really good
at tic-tac-toe. Because I was always interested in the most
practical, society impacting problems.
Tic-tac-toe generalizes pretty well to world hunger.
Eventually. So that is what got me interested in
it. And I was, I didn't even know what it
was called for a long time. So I started doing
reinforcement learning and then discovered that it was interesting and popular,
Right. Well. I certainly wouldn't suggest
we are doing the science that we are
doing because it is popular. We are doing it because we are interested in it.
And I'm interested in reinforcement learning
because, in some sense it kind of encapsulates all the things I
happen to care about. I come from a sort of general AI
background, and I care about modeling people. I care about building smart
agents that have to live in a world with other smart agents, thousands
of them, hundreds of thousands of them, thousands of them. Some of
whom might be human. And I have to field some way to predict
what to do over time. So, from a sort of technical point
of view, if we can think of supervised learning as function approximation, and
unsupervised learning as, you know,
concise, compact description, what's the difference
between something like reinforcement learning and
those two? Supervised learning in particular.
> So, often the that supervised
learning, sorry, reinforcement learning is described is
as learning from delayed reward. So instead of the feedback that you get in
supervised learning, which is, here's what you
should do, and the feedback you get
in unsupervised learning, which is [INAUDIBLE], the
feedback in reinforcement learning may come several
steps after the decisions that you've actually made. So, a good example
of that, or an easy example of that would be actually your tic-tac-toe
program, right? So, you do something in tic-tac-toe. You put an X
in the center and then you put let's say, an O over here.
And then I put an X right here.
And then you ridiculously put an O in the center.
Aw, come on.
Which allows me to put an X over here. And I win.
Now what's interesting about that is, I didn't
tell you what happened until the very end, when I said, X wins.
Right, and now I know I made a mistake somewhere along the
way, but I don't know exactly where. I may have to kind of
roll back the game in my mind and eventually figure out where it
is that I went off track, and what it is that I did wrong.
And in the full generality of reinforcement learning, you may have never
made a mistake. It may simply be that, that's the way games go. But
you would like to know which of the moves you made mattered. Now,
if it were a supervised learning problem, I would have put the X here,
you would put the O there, and would have
been told, that's good. I would have put the X
here, and then when you put the O there,
it would have been, that's bad. The O goes here.
Or something like that, or were told where he should have
put the O. But, here all he gets is, eventually some kind of
signal saying, you did something well, you did something poorly. And even
then it's only relevant to the other signals that you might have gotten.
Right, so then reinforcement learning is, in some
sense, harder, because nobody's telling you what to do.
You have to work it out on your own.
Yeah, it's like playing a game without knowing any of the rules.
Or at least knowing how you win or lose. But being told
every once in awhile that you've won or you've lost. Okay, now.
Sometimes I feel like that.
I know, man.
Comparison of these parts of ML
Okay, so, we have got three little bits of a machine line
here and there are lot of tools and techniques that are inside that.
And I think that's great and we are going to be
trying to teach you a lot of those tools and techniques
and sort of ways to connect them together. So, by the
way, as Michael is pointing out, there are kind of ways
that these might help each other. Unsupervised learning might help supervised
learning, it's actually much deeper than that. It turns out you,
even though unsupervised learning is clearly not the same as supervised
learning, at the level we've described it, in some ways they're exactly
the same thing. Supervised learning you have some bias, oh
it's a quadratic function, induction makes sense, all these kind
of assumptions you make. And in unsupervised learning, I told
you that we don't know whether this cluster's better than
this cluster, dividing by sex is better than dividing by
height, or or hair color or whatever. But ultimately, you
make some decision about how to cluster, and that means
implicitly there's some assumed set. There's some assumed set of labels
that you can possess. Oh, I think things that look alike should
somehow be clustered. Things that are near one another should be clustered
together. So, in some ways, it's still kind of like supervised learning. You
can certainly turn any supervised learning
problem into an unsupervised learning problem.
Right. So in fact, all of these
problems are really the same kind of problem.
Yeah, well there's two things that I'd
want to add to that. One is that, in some
sense, in many cases, you can formulate all
of these different problems as some form of optimization.
In supervised learning, you want something that, that labels data well, and
so your, the thing you're trying to optimize is, find me a function
that, that does that. We're going to score it. In reinforcement learning, we're
trying to find a behavior that scores well. And in unsupervised learning, we
usually have to make up some kind of a criterion, and then
we find a way of clustering the data, organizing the data so that
it scores well. So that was the first point I wanted to
make. The other one is. If you divide things by sex and you're
a virgin, then there's numerical instability issues.
Did you learn about that on the street?
I learned in the math book.
That's ugh. I, I'm going to move on with her. So here's the thing.
Everything Michael just said, except the last part, is
true. But there's actually a sort of deeper thing going
on, to me. If you think about the commonalities of
everything we just said, it boils down to one thing, data.
Data, data, data, data, data. Data is king in
machine work. Now Michael would call himself a computer scientist.
And I would call myself a computationalist.
I'm in a college of computing at a Department
of Computer Science. I believe in computing and computation as being
the ultimate thing. So I'd call myself a computationalist and Michael
would probably agree with that just to keep this discussion moving.
Right, so we're computationalists, we
believe in computing. That's a good thing.
Many of our colleagues, who do computation,
think in terms of algorithms. They think in terms
of, what are the series of steps I need to do in order to solve some problem?
I think about algorithms.
Or they might think in terms of theorems. If I try to
describe this problem in a particular way,
is it solvable implicitly by some algorithm?
And truthfully, machine learning is a lot
of that. But the difference between the person who's
trying to solve a problem as an AI
person or as a computing person and someone who's
trying to solve a problem as a machine learning person is
that the algorithm stops being central, the data starts being central.
And so what I hope you get out of this class
or at least part of the stuff that you do, is understanding
that you have to believe the data, you have to do
something with the data, you have to be consistent with the data.
The algorithms that fall out of all that are algorithms but
they're algorithms that take the data as primary. Or at least important.
I'm going to go with coequal.
So algorithms and
data are coequal.
Well, if you believe in lists they the same thing.
They knew back in the seventies.
So it turns out we do agree on those things.
Whew that was close.
Excellent. So, the rest of the semester will go exactly
like this except you won't see us. You'll see our hands though
This side, this side. There you go.
You'll see our hands though. Thank you Michael
That was good. That took me back to when I was four.
SeÃ±or, SeÃ±or Wences.
It's called SeÃ±or Wences.
Yes I know. I remember that.
Yeah, okay. Mm-hm.
I'm not that much younger than you. Ten, 12 years old.
You count gray hairs. Anyway, the point is
the rest of the semester will go like this.
We will talk about supervised learning and a whole
series of algorithms, step back a little bit and talk
about the theory behind them. And try to connect theory of machine
learning with theory of computing notions. Or at least that kind of
basic idea. What does it mean to be a hard problem versus
an easier problem. We'll move into
randomized optimization and unsupervised learning. Where we
will talk about all the issues that we brought up here and
try to connect them back to some of the things that we
did in the section on supervised learning. And then finally, we will
spend our time on reinforcement learning
and the generalization of these traditional reinforcement
learning which involves multiple agents. So we'll talk about a little bit.
Of game theory which Michael loves to talk about, I love
to talk about. And the applications of all the stuff that we've
been learning To solving problems. How to actually act in the world.
How to build that robot to do something, or build that agent
to play a game, or to teach you how to do whatever
you need to be taught how to do. But at the end
of the day we're going to teach you to think about data, how
to think about algorithms, and how to build artifacts that you know
Let's do this thing.
Excellent. Alright, well thank you Micheal.
I will see you next time we're in the same place, at the same time.