399 lines
13 KiB
Plaintext
399 lines
13 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Optional Lab: Model Representation\n",
|
|
"\n",
|
|
"<figure>\n",
|
|
" <img src=\"./images/C1_W1_L3_S1_Lecture_b.png\" style=\"width:600px;height:200px;\">\n",
|
|
"</figure>"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Goals\n",
|
|
"In this lab you will:\n",
|
|
"- Learn to implement the model $f_{w,b}$ for linear regression with one variable"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Notation\n",
|
|
"Here is a summary of some of the notation you will encounter. \n",
|
|
"\n",
|
|
"|General <img width=70/> <br /> Notation <img width=70/> | Description<img width=350/>| Python (if applicable) |\n",
|
|
"|: ------------|: ------------------------------------------------------------||\n",
|
|
"| $a$ | scalar, non bold ||\n",
|
|
"| $\\mathbf{a}$ | vector, bold ||\n",
|
|
"| **Regression** | | | |\n",
|
|
"| $\\mathbf{x}$ | Training Example feature values (in this lab - Size (1000 sqft)) | `x_train` | \n",
|
|
"| $\\mathbf{y}$ | Training Example targets (in this lab Price (1000s of dollars)).) | `y_train` \n",
|
|
"| $x^{(i)}$, $y^{(i)}$ | $i_{th}$Training Example | `x_i`, `y_i`|\n",
|
|
"| m | Number of training examples | `m`|\n",
|
|
"| $w$ | parameter: weight, | `w` |\n",
|
|
"| $b$ | parameter: bias | `b` | \n",
|
|
"| $f_{w,b}(x^{(i)})$ | The result of the model evaluation at $x^{(i)}$ parameterized by $w,b$: $f_{w,b}(x^{(i)}) = wx^{(i)}+b$ | `f_wb` | \n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Tools\n",
|
|
"In this lab you will make use of: \n",
|
|
"- NumPy, a popular library for scientific computing\n",
|
|
"- Matplotlib, a popular library for plotting data"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import numpy as np\n",
|
|
"import matplotlib.pyplot as plt\n",
|
|
"plt.style.use('./deeplearning.mplstyle')"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Problem Statement\n",
|
|
"<img align=\"left\" src=\"./images/C1_W1_L3_S1_trainingdata.png\" style=\" width:380px; padding: 10px; \" /> \n",
|
|
"\n",
|
|
"As in the lecture, you will use the motivating example of housing price prediction. \n",
|
|
"This lab will use a simple data set with only two data points - a house with 1000 square feet(sqft) sold for \\\\$300,000 and a house with 2000 square feet sold for \\\\$500,000. These two points will constitute our *data or training set*. In this lab, the units of size are 1000 sqft and the units of price are $1000's of dollars.\n",
|
|
"\n",
|
|
"| Size (1000 sqft) | Price (1000s of dollars) |\n",
|
|
"| -------------------| ------------------------ |\n",
|
|
"| 1.0 | 300 |\n",
|
|
"| 2.0 | 500 |\n",
|
|
"\n",
|
|
"You would like to fit a linear regression model (shown above as the blue straight line) through these two points, so you can then predict price for other houses - say, a house with 1200 sqft.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Please run the following code cell to create your `x_train` and `y_train` variables. The data is stored in one-dimensional NumPy arrays."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# x_train is the input variable (size in 1000 square feet)\n",
|
|
"# y_train in the target (price in 1000s of dollars)\n",
|
|
"x_train = np.array([1.0, 2.0])\n",
|
|
"y_train = np.array([300.0, 500.0])\n",
|
|
"print(f\"x_train = {x_train}\")\n",
|
|
"print(f\"y_train = {y_train}\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
">**Note**: The course will frequently utilize the python 'f-string' output formatting described [here](https://docs.python.org/3/tutorial/inputoutput.html) when printing. The content between the curly braces is evaluated when producing the output."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Number of training examples `m`\n",
|
|
"You will use `m` to denote the number of training examples. Numpy arrays have a `.shape` parameter. `x_train.shape` returns a python tuple with an entry for each dimension. `x_train.shape[0]` is the length of the array and number of examples as shown below."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# m is the number of training examples\n",
|
|
"print(f\"x_train.shape: {x_train.shape}\")\n",
|
|
"m = x_train.shape[0]\n",
|
|
"print(f\"Number of training examples is: {m}\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"One can also use the Python `len()` function as shown below."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# m is the number of training examples\n",
|
|
"m = len(x_train)\n",
|
|
"print(f\"Number of training examples is: {m}\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Training example `x_i, y_i`\n",
|
|
"\n",
|
|
"You will use (x$^{(i)}$, y$^{(i)}$) to denote the $i^{th}$ training example. Since Python is zero indexed, (x$^{(0)}$, y$^{(0)}$) is (1.0, 300.0) and (x$^{(1)}$, y$^{(1)}$) is (2.0, 500.0). \n",
|
|
"\n",
|
|
"To access a value in a Numpy array, one indexes the array with the desired offset. For example the syntax to access location zero of `x_train` is `x_train[0]`.\n",
|
|
"Run the next code block below to get the $i^{th}$ training example."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"i = 0 # Change this to 1 to see (x^1, y^1)\n",
|
|
"\n",
|
|
"x_i = x_train[i]\n",
|
|
"y_i = y_train[i]\n",
|
|
"print(f\"(x^({i}), y^({i})) = ({x_i}, {y_i})\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Plotting the data"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"You can plot these two points using the `scatter()` function in the `matplotlib` library, as shown in the cell below. \n",
|
|
"- The function arguments `marker` and `c` show the points as red crosses (the default is blue dots).\n",
|
|
"\n",
|
|
"You can also use other functions in the `matplotlib` library to display the title and labels for the axes."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Plot the data points\n",
|
|
"plt.scatter(x_train, y_train, marker='x', c='r')\n",
|
|
"# Set the title\n",
|
|
"plt.title(\"Housing Prices\")\n",
|
|
"# Set the y-axis label\n",
|
|
"plt.ylabel('Price (in 1000s of dollars)')\n",
|
|
"# Set the x-axis label\n",
|
|
"plt.xlabel('Size (1000 sqft)')\n",
|
|
"plt.show()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Model function\n",
|
|
"\n",
|
|
"<img align=\"left\" src=\"./images/C1_W1_L3_S1_model.png\" style=\" width:380px; padding: 10px; \" > As described in lecture, the model function for linear regression (which is a function that maps from `x` to `y`) is represented as \n",
|
|
"\n",
|
|
"$$ f_{w,b}(x^{(i)}) = wx^{(i)} + b \\tag{1}$$\n",
|
|
"\n",
|
|
"The formula above is how you can represent straight lines - different values of $w$ and $b$ give you different straight lines on the plot. <br/> <br/> <br/> <br/> <br/> \n",
|
|
"\n",
|
|
"Let's try to get a better intuition for this through the code blocks below. Let's start with $w = 100$ and $b = 100$. \n",
|
|
"\n",
|
|
"**Note: You can come back to this cell to adjust the model's w and b parameters**"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"w = 100\n",
|
|
"b = 100\n",
|
|
"print(f\"w: {w}\")\n",
|
|
"print(f\"b: {b}\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now, let's compute the value of $f_{w,b}(x^{(i)})$ for your two data points. You can explicitly write this out for each data point as - \n",
|
|
"\n",
|
|
"for $x^{(0)}$, `f_wb = w * x[0] + b`\n",
|
|
"\n",
|
|
"for $x^{(1)}$, `f_wb = w * x[1] + b`\n",
|
|
"\n",
|
|
"For a large number of data points, this can get unwieldy and repetitive. So instead, you can calculate the function output in a `for` loop as shown in the `compute_model_output` function below.\n",
|
|
"> **Note**: The argument description `(ndarray (m,))` describes a Numpy n-dimensional array of shape (m,). `(scalar)` describes an argument without dimensions, just a magnitude. \n",
|
|
"> **Note**: `np.zero(n)` will return a one-dimensional numpy array with $n$ entries \n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"def compute_model_output(x, w, b):\n",
|
|
" \"\"\"\n",
|
|
" Computes the prediction of a linear model\n",
|
|
" Args:\n",
|
|
" x (ndarray (m,)): Data, m examples \n",
|
|
" w,b (scalar) : model parameters \n",
|
|
" Returns\n",
|
|
" y (ndarray (m,)): target values\n",
|
|
" \"\"\"\n",
|
|
" m = x.shape[0]\n",
|
|
" f_wb = np.zeros(m)\n",
|
|
" for i in range(m):\n",
|
|
" f_wb[i] = w * x[i] + b\n",
|
|
" \n",
|
|
" return f_wb"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now let's call the `compute_model_output` function and plot the output.."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"tmp_f_wb = compute_model_output(x_train, w, b,)\n",
|
|
"\n",
|
|
"# Plot our model prediction\n",
|
|
"plt.plot(x_train, tmp_f_wb, c='b',label='Our Prediction')\n",
|
|
"\n",
|
|
"# Plot the data points\n",
|
|
"plt.scatter(x_train, y_train, marker='x', c='r',label='Actual Values')\n",
|
|
"\n",
|
|
"# Set the title\n",
|
|
"plt.title(\"Housing Prices\")\n",
|
|
"# Set the y-axis label\n",
|
|
"plt.ylabel('Price (in 1000s of dollars)')\n",
|
|
"# Set the x-axis label\n",
|
|
"plt.xlabel('Size (1000 sqft)')\n",
|
|
"plt.legend()\n",
|
|
"plt.show()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"As you can see, setting $w = 100$ and $b = 100$ does *not* result in a line that fits our data. \n",
|
|
"\n",
|
|
"### Challenge\n",
|
|
"Try experimenting with different values of $w$ and $b$. What should the values be for a line that fits our data?\n",
|
|
"\n",
|
|
"#### Tip:\n",
|
|
"You can use your mouse to click on the triangle to the left of the green \"Hints\" below to reveal some hints for choosing b and w."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"<details>\n",
|
|
"<summary>\n",
|
|
" <font size='3', color='darkgreen'><b>Hints</b></font>\n",
|
|
"</summary>\n",
|
|
" <p>\n",
|
|
" <ul>\n",
|
|
" <li>Try $w = 200$ and $b = 100$ </li>\n",
|
|
" </ul>\n",
|
|
" </p>"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Prediction\n",
|
|
"Now that we have a model, we can use it to make our original prediction. Let's predict the price of a house with 1200 sqft. Since the units of $x$ are in 1000's of sqft, $x$ is 1.2.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"w = 200 \n",
|
|
"b = 100 \n",
|
|
"x_i = 1.2\n",
|
|
"cost_1200sqft = w * x_i + b \n",
|
|
"\n",
|
|
"print(f\"${cost_1200sqft:.0f} thousand dollars\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Congratulations!\n",
|
|
"In this lab you have learned:\n",
|
|
" - Linear regression builds a model which establishes a relationship between features and targets\n",
|
|
" - In the example above, the feature was house size and the target was house price\n",
|
|
" - for simple linear regression, the model has two parameters $w$ and $b$ whose values are 'fit' using *training data*.\n",
|
|
" - once a model's parameters have been determined, the model can be used to make predictions on novel data."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.7.6"
|
|
},
|
|
"toc-autonumbering": false
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|