{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Exercise: Checking the sum and product rules, and their consequences\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Goal: Check using a very simple example that the Bayesian rules are consistent with standard probabilities based on frequencies. Also check notation and vocabulary.\n",
"\n",
"Physicist-friendly references:\n",
"\n",
"* R. Trotta, [*Bayes in the sky: Bayesian inference and model selection in cosmology*](https://www.tandfonline.com/doi/abs/10.1080/00107510802066753) {cite}`Trotta:2008qt`\n",
" \n",
"* D.S. Sivia and J. Skilling, [*Data Analysis: A Bayesian Tutorial, 2nd edition*](https://www.amazon.com/Data-Analysis-Bayesian-Devinderjit-Sivia/dp/0198568320/ref=mt_paperback?_encoding=UTF8&me=&qid=) {cite}`Sivia2006`\n",
" \n",
"* P. Gregory,\n",
" [*Bayesian Logical Data Analysis for the Physical Sciences: A Comparative Approach with Mathematica® Support*](https://www.amazon.com/Bayesian-Logical-Analysis-Physical-Sciences/dp/0521150124/ref=sr_1_1?s=books&ie=UTF8&qid=1538587731&sr=1-1&keywords=gregory+bayesian) {cite}`Gregory2005`\n",
"\n",
"$% Some LaTeX definitions we'll use.\n",
"\\newcommand{\\pr}{\\textrm{p}}\n",
"$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Bayesian rules of probability as principles of logic \n",
"\n",
"Notation: $p(x \\mid I)$ is the probability (or pdf) of $x$ being true\n",
"given information $I$\n",
"\n",
"1. **Sum rule:** If set $\\{x_i\\}$ is exhaustive and exclusive, \n",
" \n",
" $$ \n",
" \\sum_i p(x_i \\mid I) = 1 \\quad \\longrightarrow \\quad \\color{red}{\\int\\!dx\\, p(x \\mid I) = 1} \n",
" $$ \n",
" \n",
" * cf. complete and orthonormal \n",
" * implies *marginalization* (cf. inserting complete set of states or integrating out variables - but be careful!)\n",
" \n",
" $$\n",
" p(x \\mid I) = \\sum_j p(x,y_j \\mid I) \n",
" \\quad \\longrightarrow \\quad\n",
" \\color{red}{p(x \\mid I) = \\int\\!dy\\, p(x,y \\mid I)} \n",
" $$\n",
" \n",
" \n",
"2. **Product rule:** expanding a joint probability of $x$ and $y$ \n",
"\n",
" $$\n",
" \\color{red}{ p(x,y \\mid I) = p(x \\mid y,I)\\,p(y \\mid I)\n",
" = p(y \\mid x,I)\\,p(x \\mid I)}\n",
" $$\n",
"\n",
" * If $x$ and $y$ are mutually independent: $p(x \\mid y,I) = p(x \\mid I)$, then \n",
" \n",
" $$\n",
" p(x,y \\mid I) \\longrightarrow p(x \\mid I)\\,p(y \\mid I)\n",
" $$\n",
" \n",
" * Rearranging the second equality yields Bayes' Rule (or Theorem)\n",
" \n",
" $$\n",
" \\color{blue}{p(x \\mid y,I) = \\frac{p(y \\mid x,I)\\, \n",
" p(x \\mid I)}{p(y \\mid I)}}\n",
" $$\n",
"\n",
"See Cox for the proof."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Answer the questions in *italics*. Check answers with your neighbors. Ask for help if you get stuck or are unsure."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"| TABLE 1 | Blue | Brown | Total |\n",
"| :-------------: | :----------: | :-----------: | :-----------: |\n",
"| Tall | 1 | 17 | 18 |\n",
"| Short | 37 | 20 | 57 |\n",
"| Total | 38 | 37 | 75 |\n",
"\n",
"| TABLE 2 | Blue | Brown | Total |\n",
"| :-------------: | :----------: | :-----------: | :-----------: |\n",
"| Tall | | | |\n",
"| Short | | | |\n",
"| Total | | | |"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"1. Table 1 shows the number of blue- or brown-eyed and tall or short individuals in a population of 75.\n",
"*Fill in the blanks in Table 2 with probabilities (in decimals with three places, not fractions) based on the usual \"frequentist\" interpretations of probability* (which would say that the probability of randomly drawing an ace from a deck of cards is 4/52 = 1/13). *Add x's in the row and/or column that illustrates the sum rule.*\n",
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"2. *What is $\\pr(short, blue)$? Is this a joint or conditional probability? What is $\\pr(blue)$? \n",
"
From the product rule, what is $\\pr(short | blue)$? Can you read this result directly from the table?*\n",
"
\n",
"
\n",
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"3. *Apply Bayes' theorem to find $\\pr(blue | short)$ from your answers to the last part.*\n",
"
\n",
"
\n",
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"4. *What rule does the second row (the one starting with \"Short\") illustrate? Write it out in $\\pr(\\cdot)$ notation.* \n",
"
\n",
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"5. *Are the probabilities of being tall and having brown eyes mutually independent? Why or why not?*\n",
"
\n",
"
\n",
"
\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.10"
}
},
"nbformat": 4,
"nbformat_minor": 2
}