8.  Probability

These are draft notes extracted from subtitles. Feel free to improve them. Contributions are most welcome. Thank you!
Please check the wiki guide for some tips on wiki editing.


01 Probability

In this and the following units, we will talk about probability.Probability is just the opposite of statistics,and there's a yin-yang relationship between both.Put differently, in statistics we are given dataand try to infer possible causes that relate to the data,whereas in probability we are given the description of the causesand we'd like to predict the data.The reason why we now study probability and not statisticsis because it gives us a languageto describe the relationship between data and the underlying causes.So, enough of the theory; let's dive in.


02 Flipping Coins

I have here a U.S. dollar coin.It has two sides, one showing a head and one showing what's called tails.In probability, I'm giving a description of this coin, and I'm making data.We just make data.[sound of coin spinning]So if we look at the coin, it came up heads.So I just made a data point of flipping the coin once, and it came up heads.Let me do it again.[sound of coin spinning]And--wow! It came up heads again.So my new data is {heads, heads}.And you can see how it relates to the data we studied beforewhen we talked about histograms and pie charts and so on.Let me give it a third try.[sound of coin spinning]And, unbelievably, it comes up once again heads.So let me ask a statistical question to test your intuition.Do you think if I twist this coin more frequently will it always come up heads?And say I try to twist it as fairly as I possibly can.


03 Flipping Coins Solution

And you can debate it, but I think the best answer is no.This is what's called a fair coin,and that means it really has a 50% chance of coming up tails.So let me spin it again.[sound of coin spinning]And, not surprisingly, it actually came up tails this time.So probability is a method of describing the anticipated outcomeof these coin flips.

04 Fair Coin

Let's talk about a fair coin.The probability of the coin coming up heads is written in this P notation.This reads probability of the coin coming up heads.And in a fair coin, the chances are 50%.That is, in half the coin flips, the coin should come up heads.In probability we often write 0.5,which is half of 1.So a probability of 1 means it always occurs.A probability of 0.5 means it occurs half the time.And let me just ask you what do you think, for this coin, is the probability of tails?

unnamed (1).jpg

05 Fair Coin Solution

And I would say the answer is 0.5.Let me now go to a coin that is what is called "loaded."

06 Loaded Coin 1

A loaded coin is one that comes up with one of the two much more frequently than the other.So, for example, suppose I have a coin that always comes up heads.What probability would I assess for this coin to come up heads?What would be the right number over here?

unnamed (2).jpg

07 Loaded Coin 1 Solution

And the number is 1.That's the same as 100%.1 just means it always comes up in heads.

08 Loaded Coin 2

And, given that, what number would you now assessthe probability of tails to be?

unnamed (3).jpg

09 Loaded Coin 2 Solution

And, yes, the answer is zero.And we find a little law here we just want to point out,which is the probability of heads plus the probability of tailsequals 1.And the reason why that's the case is the coin either comes up heads or tails.There is no other a choice.So no matter what happens, if I look at heads and tails combinedthe chances of either of those occurring is 1,because we know it's going to happen.So we can use this lawto compute the probability of tails for other examples.

10 Loaded Coin 3

So suppose the probability of heads is 0.75,that is, 3 out of 4 times we're going to get heads.What is the probability of tails?

unnamed (4).jpg

11 Loaded Coin 3 Solution

And the answer is 0.25,which is 1 - 0.75 using the law down here.As you can verify, 0.75 + 0.25 =1.

12 Complementary Outcomes

So we just learned something important.There's a probability for an outcome; I'm going to call it A, for now.And we learned that the probability of the opposite outcome,which we're going to call ¬A(this over here just means "not")is 1 minus the probability as expressed right over here.That's a very basic law of probability, which will become handy as we go forward,so please remember it.

unnamed (5).jpg

13 Two Flips 1

In our example, we observed heads twice.So now I want to ask you a really tricky question:What's the probability of observing heads and headsif you flip the same unbiased coin twice?This means in each flip we assume the probability of heads is 0.5.Please answer here.

unnamed (6).jpg

14 Two Flips 1 Solution

That was a tricky question, and you couldn't really know the answerif you've never seen probability before,but the answer is 0.25.And I will derive it for you using something called a truth table.In a truth table, you draw out every possible outcomeof the experiment that you conducted.There were two coin flips--flip 1 and flip 2--and each hada possible outcome of heads, heads; heads, tails; tail, heads; and tail, tail.So when we look at this table, you can see every possible outcome of these two coin flips.There happens to be four of them.And I would argue because heads and tails are equally likely,each of these outcomes is equally likely.Because we know that the probability of all outcomes has to add up to 1,we find that each outcome has a chance of a quarter, or, 0.25.Another way to look at this is the probability of heads followed by headsis the product .What are the chances of the first outcome to be headsmultiplied by the probability of the second outcome to be heads?The first is 0.5, as is the second.And if you multiply these two numbers, it's 0.25, or, a quarter.

15 Two Flips 2

Let me now challenge youand give you a loaded coin I flipped twice.And for this loaded coin, I assumed the probability of heads is 0.6.That really changes all of the numbers in the table so far,but you can apply the same method of truth tablesto arrive at an answer for what is the probability of seeing heads twiceunder the assumption that the probability of heads equals 0.6?And I want to do this in steps,so rather than asking the question directly,let me help you derive it by first asking: What's the probability of tails?

unnamed (7).jpg

16 Two Flips 2 Solution

And the answer is 0.4 becauseheads comes up 0.6,and 1 - 0.6 = 0.4.

17 Two Flips 3

And our Ps fill out the entire truth table.There are four values over here, so please compute them for me.

unnamed (8).jpg

18 Two Flips 3 Solution

And the answer using our product rule isheads, heads comes out to 0.6 0.6, which is 0.36.Heads followed by tails is 0.6 0.4, which is 0.24.Tails followed by heads is, again, 0.24.And tails followed by tails is 0.16, which is 0.4 0.4.

19 Two Flips 4

If you add up these numbers over here--please go ahead and add them upand tell me what the sum of those numbers is.

unnamed (9).jpg

20 Two Flips 4 Solution

And, not surprisingly, it's 1.That is, the truth table always has a probability that adds up to 1because it considers all possible cases,and all possible cases together have a probability of 1.So we just check this and make sure it's correct.Reading from this table, we find that the probability of (H,H) is 0.36.And you can do the same over here.0.6 0.6 = 0.36So that's our correct answer.

21 Two Flips 5

Let's now go to the extreme, and this is a challenging probability question.Suppose the probability of heads is 1,so my coin always comes up with heads.What is the probability of (H,H)?

unnamed (10).jpg

22 Two Flips 5 Solution

And the answer is 1.To see this, we know that the probability of tails is 0.All the probability goes to heads.1 1 = 11 0 = 00 1 = 0And 0 0 = 0.And it's easy to verify that all these things add up to 1.Our (H,H) is just 1.

23 One Head 1

The truth table gets more interesting when we ask different questions.Suppose we flip our coin twice.What we care about is that exactly one of the two things is heads,and thereby exactly the other one is tails.For a fair coin, what do you think the probability would bethat if I flip it twice we would see heads exactly once?

unnamed (11).jpg

24 One Head 1 Solution

And the answer shall be 0.5.And this is a nontrivial question.Let's do the truth table.So, for flip-1, we have the outcomes of heads, heads, tails, tails.For flip-2, heads and tails and heads and tails.These are all possible outcomes.And we know for the fair coin each outcome was equally likely.That is, exactly one quarter.

25 One Head 2

Given that, we now have to associate a truth tablewith the question we're asking.So where exactly is, in the outcome, heads represented once?Please check the corresponding cases.

unnamed (12).jpg

26 One Head 2 Solution

And, yes, it's in the second case and in the third case.The extreme cases of heads, heads and tails, tails don't satisfy this condition.So the trick now has been to take the 0.25 probability of these two casesand add them up, which gives us 0.25 + 0.25 = 0.5.This is the number which is correct for this inquiry.

27 One Of Three 1

Let me now make it really, really challenging for you.I take a fair coin and flip it 3 times, and I want to knowthe probability that exactly 1 of those 3 flips comes up heads.

unnamed (13).jpg

28 One Of Three 1 Solution

And this answer is tricky. We will derive it through the truth table. Now there's eight possible cases.Flip one can come of heads or tail; same for flip two, heads, tail, heads, tail;and the same for flip three and if you look at this every possible combination is represented.For example, these are heads, tail, tail.Now each of those outcomes has the same probability of an eighth, because it's eight cases.So 8 x 1/8 sums up to 1. In how many cases do we have exactly one H?It turns out that it's true for only three cases.The H could be in the first position, in the second position, or in the third position.So three out of eight cases have a single H.Each of those carries a probability so we sum those cases up to carry a total of 3/8 of a probability.These are the same as 0.375.

29 One Of Three 2

Now that was a challenging question.I'm going to make it even more challenging for you now.I'll give you a loaded coin--the probability for H is 0.6.I expect this will take you awhile on a piece of paperto really calculate this probability over here.But you can do exactly the same thing.You go through the truth table.You apply the multiplication I showed you beforeto calculate the probability of each outcome; they're not the same anymore.H, H, H is clearly more likely than T, T, T.And when you've done this, add the corresponding figures up,and tell me what the answer is.

unnamed (14).jpg

30 One Of Three 2 Solution

And my answer is 0.288. How do I get that? Let's look at the three critical cases.H T T is 0.6 for H times 0.4 for tails times another 0.4 for tails again and it gives me 0.096.Now it turns out this case over here has the same property because all we do is we order it0.4 x 0.6 x 0.4 and we know that in multiplication the order doesn't matter,so you get the same 0.096, and by the same logic, if third one also gets me 0.096.So adding this 0.096s together, if we get them, gives me 0.288.So I did not have to fill the entire truth table, which you might have done in the duration.I only have to fill out the cases I care about, yet they give me their correct result.

31 Even Roll

So let's do one final exercise.Now I am throwing dice.The difference between dice and coins is that there are now 6 possible outcomes.Let me just draw them, and say it's a fair die,which means each of the different sides comes up with a probability over 6for any of the numbers you can plug in over here.What do you think the probability is the die comes up with an even number?I'm going to write this as the outcome of the die is even.And you can once again use a truth table to calculate that number.

unnamed (15).jpg

32 Even Roll Solution

In truth table-speak, there are 6 outcomes, 1 to 6.Each has the same probability over six.Half of those numbers are even--2, 4, and 6,so if we add those up, we get 3 1/6--the same as a half.The outcomes is 0.5.Now I'm finally going to make, as my final quiz,a really challenging question for you.

33 Doubles

Suppose we throw a fair die twice. What do you think the probability of a double is?Double means both outcomes are identical with the same number regardless of what that number is.The actually an important number because in many games involving two dice,have different rules when these come up with the same number.So, it might be important to know what the probability is.

unnamed (16).jpg

34 Doubles Solution

And once again, we can answer this using a truth table.Now the truth table will have 36 different entries,six for the first throw times six for the second throw,and on the space under this tablet, we draw all the 36 entries.So, let me just draw the ones in any manner, one-one, two-two, and so on all the way to a six-six.So, each one of those is a probability of 1/6 for the first outcome times 1/6 for the second,which gives me 1/36, and the same logic applies everywhere.So, for all of these six outcomes, I have 1/36 of a chance this outcome would materialize.Adding them all up gives me 1/6, why?Because, I get 6 times 36 and I can simply this back to 1/6 that's just the same as 0.16667.So, 1/6 times, you will get a double?Now, when you're play a game like backgammon, which is played with two dice,it might not feel like this, I can swear I don't get a double of 1/6 moves,but it's actually true that that's the right--that's the correct probability.

35 Summary

So let me summarize, you've actually learned quite a bit.You learned about probability of an event, such as the outcome of a coin flip.You learned that the probability of the opposite event is 1 minus the probability of the event.And you learned about the probability of a composite event,which was in the form P P …. P.Technically speaking, this thing over here is called independence,which means nothing else, but that the outcome of the second coin flip didn't really depend on the outcome of the first coin flip.In our next unit, we will talk about dependence where there are bizarre dependencies between different outcomes.But for the time being, you really managed to get a very basic understanding of probability.So, let's take the next unit and let's jointly dive much deeper into the rabbit hole of probability.