9.  Conditional Probability


These are draft notes extracted from subtitles. Feel free to improve them. Contributions are most welcome. Thank you!
Please check the wiki guide for some tips on wiki editing.

Contents

01 Dependent Things

In real life, things depend on each other.Say you can be born smart or dumb and for the sake of simplicity, let's assumewhether you're smart or dumb is just nature's flip of a coin.Now whether you become a professor at Standford is non-entirely independent.I would argue becoming a professor in Standford is generally not very likely,so probability might be 0.001 but it also depends on whether you're born smart or dumb.If you are born smart the probability might be larger, whereas if you're born dumb,the probability might be marked more smaller.Now this just is an example, but if you can think of the most two consecutive coin flips.The first is whether you are born smart or dumb.The second is whether you get a job on a certain time.And now if we take them in these two coin flips, they are not independent anymore.So whereas in our last unit, we assumed that the coin flips were independent,that is, the outcome of the first didn't affect the outcome of the second.From now on, we're going to study the more interesting cases where theoutcome of the first does impact the outcome of the second,and to do so you need to use more variables to express these cases.

Selection_001.png

02 Cancer Example 1

To do so, let's study a medical example--supposed there's a patientin the hospital who might suffer from a medical condition like cancer.Let's say the probability of having this cancer is 0.1.Thatt means you can tell me what's the probability of being cancer free.

unnamed.jpg

03 Cancer Example 1 Solution

And the answer is 0.9 with just 1 minus the cancer.

04 Cancer Example 2

Of course, in reality, we don't know whether a person suffers cancer,but we can run a test like a blood test.The outcome of it blood test may be positive or negative, but like any good test,it tells me something about the thing I really care about--whether the person has cancer or not.Let's say, if the person has the cancer, the test comes up positive with the probability of 0.9,and that implies if the person has cancer, the negative outcome will have 0.1 probabilityand that's because these two things have to add to 1.I've just given you a fairly complicated notation that says the outcome of the test dependson whether the person has cancer or not.That is more complicated than everything else we've talked about so far.We call this thing over here a conditional probability,and the way to understand this is a very funny notation.There's a bar in the middle, and the bar says what's the probability of the stuff on the leftgiven that we assume the stuff on the right is actually the case.Now, in reality, we don't know whether the person has cancer or not, and in a later unit,we're going to reason about whether the person has cancer given a certain data set,but for now, we assume we have god-like capabilities.We can tell with absolute certainty that the person has cancer,and we can determine what the outcome of the test is.This is a test that isn't exactly deterministic--it makes mistakes,but it only makes a mistake in 10% of the cases, as illustrated by the 0.1 down here.Now, it turns out, I haven't fully specified the test.The same test might also be applied to a situation where the person does not have cancer.So this little thing over here is my shortcut of not having cancer.And now, let me say the probability of the test giving me a positive results--a false positive resultwhen there's no cancer is 0.2.You can now tell me what's the probability of a negative outcome in casewe know for a fact the person doesn't have cancer, so please tell me.

unnamed (1).jpg

05 Cancer Example 2 Solution

And the answer is 0.8.As I'm sure you noticed in the case where there is cancer, the possible test outcomes add up to 1.In the where there isn't cancer, the possible test outcomes add up to 1.So 1 - 0.2 = 0.8.

06 Cancer Example 3

Look at this, this is very nontrivial but armed with this,we can now build up the truth table for all the cases of the two different variables,cancer and non-cancer and positive and negative tests also.So, let me write down cancer and test and let me go through different possibilities.We could have cancer or not, and the test may come up positive or negative.So, please give me the probability of the combination of those for the very first one,and as a hint, it's kind of the same as before where we multiply two things,but you have to find the right things to multiple in this table over here.This is not an easy question.

unnamed (2).jpg

07 Cancer Example 3 Solution

And the answer is probability of cancer is 0.1, probability of test being positive given that he hascancer is the one over here--0.9, multiplying those two together gives us 0.09.

08 Cancer Example 4

Moving to the next case--what do you think the probability is thatthe person does have cancer but the test comes back negative?What's the combined probability of these two cases?

unnamed (3).jpg

09 Cancer Example 4 Solution

And once again, we'd like to refer the corresponding numbers over here on the right side0.1 for the cancer times the probability of getting a negative result conditioned on havingcancer and that is 0.1 0.1, which is 0.01.

10 Cancer Example 5

Moving on to the next case. What do you think the answer is?

unnamed (4).jpg

11 Cancer Example 5 Solution

And here the answer is 0.18 by multiplying the probability of not having cancer, which is 0.9,with the probability of getting a positive test result for a non-cancer patient 0.2.Multiplying 0.9 with 0.2 gives me 0.18.

12 Cancer Example 6

Let's just quickly do the final one,because it's the most likely one.

unnamed (5).jpg

13 Cancer Example 6 Solution

Here you get 0.72,which is the product ofnot having cancer in the first place0.9and the probability of getting anegative test resultunder the condition of not having cancer.

14 Cancer Example 7

Now quickly, do me a favor and add all of those up.What do you get?

unnamed (6).jpg

15 Cancer Example 7 Solution

And as usual, the answer is 1.That is, we study in the truth table all possible cases.When you add up the properties,you should always get the answer of 1.

16 Cancer Example 8

Now let me ask you a really tricky question.What is the probability of a positive test result?Can you sum or determine,irrespective of whether there's cancer or not,what is the probability you get a positive test result?

unnamed (7).jpg

17 Cancer Example 8 Solution

And the result, once again, is following the truth table,which is why this table is so powerful.Let's look at where in the truth table we get a positive test result.I would say it is right here,right here.If you take corresponding probabilities of0.9 and 0.18,and add them up,we get 0.27,and that's the correct answer for getting a positive result.

18 Total Probability

Putting all of this into mathematical notationwe've given the probability of having cancerand from there, it follows the probability of not having cancer.And they give me 2 condition probabilitythat are the test being positive.If we have have cancer, from which we can now predict the probabilityof the test being negative of having cancer.And the probability of the test being positive can be cancer freewhich can complete the probability ofa negative test result in the cancer-free case.So these things are just easily inferredby the 1 minus rule.Then when we read this,you complete the probability of a positive test resultas the sum ofa positive test result given cancertimes the probability of cancer,which is our truth table entry for the combination of P and Cplus the same can be done of cancer.Now this notation is confusing and complicatedif we entered that deep probability, that's calledtotal probability,but it's useful to know that this isvery, very intuitiveand to further develop an equation that can just give youanother exercise of exactly the same type.

unnamed(8).jpg

19 Two Coins 1

This time around, we have a bag,and in the bag are 2 coins,coin 1 and coin 2.And in advance, we know that coin 1 is fair.So P of coin 1 of coming up heads is 0.5whereas coin 2 is loaded, that is, P of coin 2coming up heads is 0.9.Quickly, give me the following numbersof the probability of coming up tailsfor coin 1 and for coin 2.

unnamed (8).jpg

20 Two Coins 1 Solution

And the answer is 0.5 for coin 1and 0.1 for coin 2,because these things have to add up to 1for each of the coins.

21 Two Coins 2

So now what happens is, I'm going toremove one of the coins from this bag,and each coin, coin 1 or coin 2,is being picked with equal probability.Let me now flip that coin once,and I want you to tell me,what's the probability that this coinwhich could be 50% chance fair coinand 50% chance a loaded coin.What's the probability that this coin comes up heads?Again, this is an exercise in conditional probability.

[Image:agpzfnVkYWNpdHl1chILEglXaWtpSW1hZ2UYkZXXXgw;size=500]]

22 Two Coins 2 Solution

And let’s do the truth table.You have a pick event followed by a flip eventWe can pick coin 1 or coin 2.There is a 0.5 chance for each of the coins.Then we can flip and get headsor tails for the coin we've chosen.Now what are the probabilities?I'd argue picking 1 at 0.5and once I pick the fair coin, I knowthat the probability of heads is, once again, 0.5which makes it 0.25The same is true for picking the fair coinand expecting tailsbut as we pick the unfair coin with a 0.5 chancewe get a 0.9 chance of headsSo 0.5 times 0.95 gives you 0.45whereas the unfair coin, the probability of tails is 0.1multiply by the probability of picking it at 0.5 gives us 0.05Now when they ask you, what's the probability of headswe'll find that 2 of those cases indeed come up with headsso if you add 0.25 and 0.45 and we get 0.7.So this example is a 0.7 chancethat we might generate heads.

23 Two Coins 3

Now let me up the ante by flipping this coin twice.Once again, I'm drawing a coin from this bag,and I pick one at 50% chance.I don't know which one I have picked.It might be fair or loaded.And in flipping it twice,I get first heads,and then tails.What's the probability that if I do the following,I draw a coin at random with the probabilities shown,and then I flip it twice, that same coin.I just draw it once and then flip it twice.What's the probability of seeing heads firstand then tails?Again, you might derive this using truth tables.

unnamed (10).jpg

24 Two Coins 3 Solution

This is a non-trivial question,and the right way to do this is to go through the truth table,which I've drawn over here.There's 3 different things happening.We've taken initial pick of the coin,which can take coin 1 or coin 2 with equal probability,and then you go flip it for the first time,and there's heads or tails outcomes,and we flip it for the second time with the second outcome.So these different cases summarize my truth table.I now need to observe just the cases wherehead is followed by tail.This one right here and over here.Then we compute the probability for those 2 cases.The probability of picking coin 1 is 0.5.For the fair coin, we get 0.5 for heads,followed by 0.5 for tails.They're together is 0.125.Let's do it with the second case.There's a 0.5 chance of taking coin 2.Now that one comes up with heads at 0.9.It comes up with tails at 0.1.So multiply these together, gives us 0.045,a smaller number than up here.Adding these 2 things together results in 0.17,which is the right answerto the question over here.That was really non-trivial,and I'd be amazed if you got this correct.

25 Two Coins 4

Let me do this once again.There are 2 coins in the bag,coin 1 and coin 2.And as before, taking coin 1 at 0.5 probability.But now I'm telling youthat coin 1 is loaded, so give you heads with probability of 1.Think of it as a coin that only has heads.And coin 2 is also loaded.It gives you heads with 0.6 probability.Now work out for me into this experiment,what's the probability of seeing tails twice?

unnamed (11).jpg

26 Two Coins 4 Solution

And the answer is depressing.If you, once again, draw the truth table,you find, for the different combinations,that if you've drawn coin 1,you'd never see tails.So this case over here, version D has tails, tails.We have 0 probability.We can work this out probability of drawing the first coin at 0.5,but the probability of tails given the first coinmust be 0, because the probability of heads is 1,so 0.5 times 0 times 0,that is 0.So the only case where you might see tails/tails iswhen you actually drew coin 2,and this has a probability of0.5 times the probability of tailsgiven that we drew the second coin,which is 0.4 times 0.4 again,and that's the same as 0.08would have been the correct answer.

27 Summary

In the important lessons in what we just learned,the key thing is we talked about conditional probabilities.We said that the outcome in a variable, like a testis actually not like the random coin flipbut it depends on something else,like a disease.When we looked at this,we were able to predictwhat's the probability of a test outcomeeven if we don't know whether the person has a disease or not.And we did this using the truth table,and in the truth table,we summarized multiple lines.For example, we multiplied the probability of a test outcomecondition on this unknown variable,whether the person is diseasedmultiplied by the probability of the disease being present.Then we added a second row of the truth table,where our unobserved disease variabletook the opposite value of not diseased.Written this way, it looks really clumsy,but that's effectively what we did when we went to the truth table.So we now understand that certain coin flipsare dependent on other coin flips,so if god, for example, flips the coin of us having a disease or not,then the medical test againhas a random outcome,but its probability really depends on whether we have the disease or not.We have to consider this when we do probabilistic inference.In the next unit,we're going to ask the real question.Say we really care about whether we have a disease like cancer or not.What do you think the probability is,given that our doctorjust gave us a positive test result?And I can tell you,you will be in for a surprise.

unnamed(12).jpg