Bayes Rule
Last updated
Last updated
Bayesian statistics expresses probability as a degree of belief in an event which can change as new information is gathered rather than a fixed value based on frequency.
Example: 1% of the population has cancer. Given that there is a 90% chance that you will test positive if you have cancer and that there is a 90% chance you will test negative if you don't have cancer, what is the probability that you have cancer if you test positive?
The pink area are all the people that tested positive. That's the only thing that matters.
Let's look at the math:
Prior belief = Our initial prediction
This doesn't add up to 1! So what happened?
Example:
If we take the oposite example, what's the probability of having cancer if I have Negative test?
Same methodology applies. So P(C|Neg) (prob of having cancer, knowing that my test was neg) = 0.0011.
So before the test, 1% of the population has cancer so I may have 1% chance of having cancer. Now that I passed the test, it's neg, I know that I have .1% chance of having cancer. The test really helped us increase our confidence that we are cancer free.
In our last example, our sensitivity was high (0.9), and our specificity was high (0.9). What happens when our specificity is lower (0.5).
Same thing but different numbers.
To summarize
We have a hidden variable we care about, Cancer for example. We can't measure it exactly, so instead we have a test.
We have a prioir of how frequent having the disease is true P(C), and the test in generally characterized by how often it is positive when P(C) is true (= Probability: P(Pos|C), and how often it is neg when P(C) is not true (probability: P(Neg | notC).
To see it again slightly differently:
But the green and red area don't equal 1. So we normalize the red and green area, which means dividing them both by the total area:
So doing this gets rid of all the other cases.
New exercise:
The step-by-step breakdown of the solution is pretty quick. Let's recap what's covered in the solution video.
Let's start with what we know:
Prior Probabilities
The robot is perfectly ignorant about where it is, so prior probabilities are as follows:
Conditional Probabilities
The robot's sensors are not perfect. Just because the robot sees red does not mean the robot is at red.
Posterior Probabilities
From these prior and posterior probabilities we are asked to calculate the following posterior probabilities after the robot sees red:
But we still have one unknown! What was the probability that we would see red? The answer is 0.5 and there are two ways I can convince myself of that. The first is intuitive and the second is mathematical.
Why is P(see red) 0.5?
Argument 1: Intuitive
Of course it's 0.5! What else could it be? The robot had a 50% belief that it was in red and a 50% belief that it was in green. Sure, its sensors are unreliable but that unreliability is symmetric and not biased towards mistakenly seeing either color.
So whatever the probability of seeing red is, that will also be the probability of seeing green. Since these two colors are the only possible colors the probability MUST be 50% for each!
Argument 2: Mathematical (Law of Total Probability)
There are exactly two situations where the robot would see red.
When the robot is in a red square and its sensors work correctly.
When the robot is in a green square and its sensors make a mistake.
I just need to add up these two probabilities to get the total probability of seeing red.
What happens when instead of having 2 outcomes, R or G, Cancer, Non-cancer, there are 3?
Same but to calculate P(R), since there are 3 states, you need to calculate the probability of seeing R when either being in A, B or C. 3 steps instead of 2.
We know a little bit about the map of the road that our car is on (pictured above). We also have an initial GPS measurement; the GPS signal says the car is at the red dot. However, this GPS measurement is inaccurate up to about 5 meters. So, the vehicle could be located anywhere within a 5m radius circle around the dot.
Then we gather data from the car's sensors. Self-driving cars mainly use three types of sensors to observe the world:
Camera, which records video,
Lidar, which is a light-based sensor, and
Radar, which uses radio waves.
All of these sensors detect surrounding objects and scenery.
Autonomous cars also have lots of internal sensors that measure things like the speed and direction of the car's movement, the orientation of its wheels, and even the internal temperature of the car!
Suppose that our sensors detect some details about the terrain and the way our car is moving, specifically:
The car could be anywhere within the GPS 5m radius circle,
The car is moving upwards on this road,
There is a tree to the left of our car, and
The car’s wheels are pointing to the right.
Knowing only these sensor measurements, examine the map below and answer the following quiz question.
Given the sensor info, we know we're in position A.
Once we gather sensor data about the car's surroundings and its movements, we can use this information to improve our initial position prediction.
If we sens lane markers and specific terrain. We know from previous knowledge that if we see lane markers on the left we know we must be at the center of the lane. If sense that our tires are pointing to the right, we're in a cuved section. Bayes rules gives us a mathematic way to correct our measurements and lets us move from an uncertain prior belief, to something more and more probable.
