st101 » week-4 »

21.  Manipulating Normals

These are draft notes extracted from subtitles. Feel free to improve them. Contributions are most welcome. Thank you!
Please check the wiki guide for some tips on wiki editing.


01 Raises

So welcome back!You just made it to one of the hairy parts of this class when we talk about normal distributions.Now we've come back into something intuitive, which is how to manipulate normal distributions.Like I said in the previous class, you can flip coins up to better to the values from the normal distribution just the same way you flip coins.So now I'd like to understand what happens if I manipulate normal distributions.So my very first example will be the following.Suppose all the salaries in my company Udacity are normally distributed.That is to say, If I were to look at the salaries, there is a well-defined mean of course.Fewer and fewer people deviate from the mean by a larger amount where the frequency of deviations captured really well by this normal distributions as most people are really near the mean and there's a small number of people are farther from mean.And just for the sake of the argument, suppose the mean is $60,000 per year and the standard deviation is 10,000--these are obviously not accurate numbers.Suppose now, I give everyone a raise of say $10,000, what do think is the new mean and the new standard deviation following µ' and σ'--can you answer these questions?


02 Raises Solution

And the answer for the µ‘ is 70,000 of this given salary increase and the new σ stays the same.We already talked about this example earlier in class, but now it's in the content of normal distribution so let me take a second to show this to you.In our normal, ignoring normalization constant,we know that everyone's salary is drawn from a distribution that looks like this.This is plugging in the mean and the variance as that defined it,and now we know, we set the new salary x' to be the old salary plus 10,000 That is the same as saying that the old salary was the new salary minus 10,000.We now substitute this x over here with this expression over here--what we got is something that looks looks smallest, so you can see I took out the x and replaced it by x' minus 10,000 and if you look at this carefully, you'll find this is the same as (x'-70,000)²/10,000².So that proves to you that the variance of the same deviation doesn't change and the mean has effect it by just increasing it to 70,000 but definitely the µ‘ is now 70,000 and the σ' remains 10,000.

03 Double Salary

Now let's say in the same company now with a mean salary of 70,000 and a standard deviation of 10,000, I decide to double everyone salary,and please don't tell my people about that--I'm contemplating with.Don't discuss this in the forums. That's kind of secret, but what does this mean?What do you think is the new mean and what's the new standard deviation important?I'm not asking about the variance but just the square of this just the standard deviation.

unnamed (1).jpg

04 Double Salary Solution

And the answer is the mean will double and so with the standard deviation. To see put this again into our formula using our original mean, and now we realize that the new salaries x' are twice the x salaries or put differently x will be a half of x'.If I substitute this into the formula over here, I get the same exponential expression,but now with half x' where the x used to be.Now the trick is to deal with the half and that's more complicated than our case before.The very first thing is I have to bring the half out of the parenthesis over here and there's a trick whereby I rewrite 70,000 as a half of a 140,000 and you can see where this is going.With that, I can now bring the half out of the square over here, which gives me a quarter,you have to square the half with x'-140,000)²/10,000²and now, all I have to do is I have to bring the quarter into the 10,000.So quarter up here is the same as a 4 down here.Now, you have to bring it inside the square, so we have to take the square root so it ends up to be the same as the new 2, the square of 2 into the square over here.So this transformation shows that doubling the salary doubles the µ but also doubles the σ.It doubles the standard deviation of those salaries--I've proven it to you.

05 Throwing

Let's ask you a different question--suppose we're on the fields and we learn how to throw ball.On average, we're able to throw it 30 m, but because there's randomness in the ball,we have a standard deviation of 5 m.And now we trained, we improved our performance by 10%.We didn't just step forward by 2 more meters before we throw and that gives us an additional improvement of 2 m.For Gaussian outcome like this, how does it affect all µ' and σ'after both of these improvements are materialized.

unnamed (2).jpg

06 Throwing Solution

And the answer is relatively straightforward a 10% boost in performance goes from 30 to 33 m and a subsequent 2 m gives us 35 m as our new mean.Our σ increases in proportion to the first term to 5.5.The fixed offset of 2 m doesn't affect the spread so this all just stays the same.

07 Golf 1

Here's another challenge--this time you are a golfer.You hit the ball really hard, it goes flying and it lands with some uncertainty that's Gaussian where the distance in expectation is a 100 m and your variance is 30 m².And now you do exactly the same thing again--you hit the ball really hard,it goes flying and has a final position.Again, you expect it to fly a 100 m with a variance of 40 m².Without diving into the mathematics, I'd like you to answer the question first.For the combined stroke, what do you expect the distance to be?

unnamed (3).jpg

08 Golf 1 Solution

And the answer is 200 m. Those just add up.

09 Golf 2

What's the combined variance?

unnamed (4).jpg

10 Golf 2 Solution

And the answer is 60 and it turns out the uncertainties from both of these experiments just add up.The second one is very non-trivial. I'm not going to prove it to you.There's a proof in my book that takes marginal pages, but it is interesting to see when you add Gaussian variables, the means add up and the variance add up.The important to this standard deviation is don't add up.

11 Golf 3

Let me modify my question and let me say this time I expressed things in terms of standard deviations and not variances.Each of these has a standard deviation of 10 m.You ought to know that the combined µ will be 200 m, but what about the new standard deviation.You can actually calculate this and give me at least an approximate answer.

unnamed (5).jpg

12 Golf 3 Solution

And the answer is 14.14 m, which is the same as 10 m √2.And you see, we know that the old variance would be a 100 m², the µ thinks therefore, it's 200 m² because I told you that just add up.If we go to the standard deviation, we take the square root of this.This factors into 2×100 m², so we take the square root of this, we get back the 10 m on the right side and you get the √2 on the left side,which gives me the 14.14 m for the standard deviation.That's interesting--standard deviations don't add up, variances add up and that was a non-trivial question.

13 Golf 4

So at first we draw values from a normal distribution with µ and σ² parameters and say it equals a value A, then what's the mean and the variance of aA+b where a and b are constants.I'll give you possible answers down here from 1 to 7 and you have to plug an expression to either one of those by giving me the corresponding number.So if you think the expressions over here would be for example σ²+b², you type in 3 over here.If you think it's over here aσ², you type in 7, so either one of those, you have a number in mind 7 and as a hint, once an expression is used up, you can use it again.

unnamed (6).jpg

14 Golf 4 Solution

This was a trick question, which was totally nontrivial. Multiplying something with A+b modifies the mean in a very straightforward way.So this is the valid expression here, aµ+b. So 6 would've been the right answer over here.You already know that adding a constant doesn't change the spread of a distribution.So the result should not have a b inside. That takes out this term and all those terms.We also know that the spread has nothing to do with the mean so this term is out.So the question really is, is it aσ² or a²σ²?And if you remember correctly when we did the salary raise by a factor of 10%,we noticed that the standard deviation goes up by this factor.Therefore, the variance goes up by the square of this factor and 1 is the right answer here and not 7.And that's interesting because in doing arithmetic with the mean and the variance,you have to be very conscientious as to whether you use the variance or the standard deviation.If you get that wrong, then you've got the wrong answer.

15 Adding Normals

Let me do a final quiz. This is related to playing golf.We have a normal distribution with µ and σ² from which we draw A.We do the same with B using the same µ and σ². Now we combine A+B and you just add them up.Now I would like to know how the new µ and the new σ² looks like.And as before, I give you a number of possibilities.So there are seven options in total from 2µ all the way to 0.Please put your green number into these fields over here.So this will be a number within 1 and 7,and it indicates that the corresponding term is the correct answer for µ‘ and σ‘².

unnamed (7).jpg

16 Adding Normals Solution

You already learned that the means add up. So we get 2µ, which is 1 for the µ‘. And we learned that variances add up in our golf example.This is just 2σ² or 3 over here. That's the correct answer.

17 Subtracting Normals

And now, let me play a trick on you. I want to take A-B.  So I'm subtracting A from B.Think of it as playing golf all the way over,picking up the ball, and playing the ball back to where you started.What do you think the µ‘ will be and what do you think the σ‘² will be?I'm appealing to intuition here.

unnamed (8).jpg

18 Subtracting Normals Solution

And let's say in expectation, when you go forward 100 m and come back 100 m, you are to be where you started.So 7 will be the correct mean. Now σ‘² is more difficult. It turns out the variances still add up.And the way to see this is if we started over here at a well-defined zero position,and you go to over here and you have some uncertainty as to where you might be.And then you hit the ball back over here, your uncertainty will increase. It won't decrease.That makes 2σ² the correct answer.So even though we substituted the plus sign by a minus sign,the uncertainty that comes in for either of these steps over here will add up just the way it did before.And I have to grant you, this wasn't an easy question and I hope you got it right.If not, think about it and perhaps do the same quiz again.There's some deep insight here, which is for the means the arithmetic I'm giving you always apply straight, but for the variances things are counter-intuitive.When we add to random variables, the variances always just add up no matter what the operation is plus or minus.

19 Summary

So this completes this unit. Congratulations! I'd say this was nontrivial. I didn't derive every piece of math for you. But you've learned a number of things.If X is normal with parameters µ and σ², then aX+b will yield parameters aµ+b and a²σ².You also learned if you combined X and Y, that the combined thing adds up the means in this case two µ's and adds up the variances as well.So we did some very basic math on normals. We got a feel for how they changed as we turned them into problems and manipulate them.I'll leave it at this. This was possibly the most insightful and deep probability consideration.You now go back to statistics, and we worry about confidence and other things.