"
]
},
{
"cell_type": "markdown",
"id": "ba1e2404",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"### Optional Lab - W1: Brief Introduction to Python and Jupyter Notebooks\n",
"Welcome to the first optional lab! \n",
"Optional labs are available to:\n",
"- provide information - like this notebook\n",
"- reinforce lecture material with hands-on examples\n",
"- provide working examples of routines used in the graded labs"
]
},
{
"cell_type": "markdown",
"id": "7eed3933",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"#### Goals\n",
"In this lab, you will:\n",
"- Get a brief introduction to Jupyter notebooks\n",
"- Take a tour of Jupyter notebooks\n",
"- Learn the difference between markdown cells and code cells\n",
"- Practice some basic python\n"
]
},
{
"cell_type": "markdown",
"id": "da73e262",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"The easiest way to become familiar with Jupyter notebooks is to take the tour available above in the Help menu:"
]
},
{
"cell_type": "markdown",
"id": "4957c87e",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"
\n",
"
\n",
""
]
},
{
"cell_type": "markdown",
"id": "466abf8c",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Jupyter notebooks have two types of cells that are used in this course. Cells such as this which contain documentation called `Markdown Cells`. The name is derived from the simple formatting language used in the cells. You will not be required to produce markdown cells. Its useful to understand the `cell pulldown` shown in graphic below. Occasionally, a cell will end up in the wrong mode and you may need to restore it to the right state:"
]
},
{
"cell_type": "markdown",
"id": "e25d5702",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"
\n",
" \n",
""
]
},
{
"cell_type": "markdown",
"id": "30d554fe",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"The other type of cell is the `code cell` where you will write your code:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f35db920",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"#This is a 'Code' Cell\n",
"print(\"This is code cell\")"
]
},
{
"cell_type": "markdown",
"id": "21f16015",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"### Python\n",
"You can write your code in the code cells. \n",
"To run the code, select the cell and either\n",
"- hold the shift-key down and hit 'enter' or 'return'\n",
"- click the 'run' arrow above\n",
"
\n",
" \n",
"\n",
"\n",
" "
]
},
{
"cell_type": "markdown",
"id": "47759d5b",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"#### Print statement\n",
"Print statements will generally use the python f-string style. \n",
"Try creating your own print in the following cell. \n",
"Try both methods of running the cell."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8babb554",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"# print statements\n",
"variable = \"right in the strings!\"\n",
"print(f\"f strings allow you to embed variables {variable}\")"
]
},
{
"cell_type": "markdown",
"id": "66da1cd0",
"metadata": {},
"source": [
"### Practice Quiz "
]
},
{
"cell_type": "markdown",
"id": "b7f8d20a",
"metadata": {},
"source": [
"#### Quiz - 1"
]
},
{
"cell_type": "markdown",
"id": "3be811d0",
"metadata": {},
"source": [
"
\n"
]
},
{
"cell_type": "markdown",
"id": "c071f6e5",
"metadata": {},
"source": [
"## Module - 2"
]
},
{
"cell_type": "markdown",
"id": "af7db867",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"### Optional Lab W2: Python, NumPy and Vectorization\n",
"A brief introduction to some of the scientific computing used in this course. In particular the NumPy scientific computing package and its use with python.\n",
"\n",
"#### Outline\n",
"- [1.1 Goals ](#toc_40015_1.1)\n",
"- [1.2 Useful References ](#toc_40015_1.2)\n",
"- [2 Python and NumPy ](#toc_40015_2)\n",
"- [3 Vectors ](#toc_40015_3)\n",
"- [3.1 Abstract ](#toc_40015_3.1)\n",
"- [3.2 NumPy Arrays ](#toc_40015_3.2)\n",
"- [3.3 Vector Creation ](#toc_40015_3.3)\n",
"- [3.4 Operations on Vectors ](#toc_40015_3.4)\n",
"- [4 Matrices ](#toc_40015_4)\n",
"- [4.1 Abstract ](#toc_40015_4.1)\n",
"- [4.2 NumPy Arrays ](#toc_40015_4.2)\n",
"- [4.3 Matrix Creation ](#toc_40015_4.3)\n",
"- [4.4 Operations on Matrices ](#toc_40015_4.4)\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5ba156ba",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"import numpy as np # it is an unofficial standard to use np for numpy\n",
"import time"
]
},
{
"cell_type": "markdown",
"id": "ba562829",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"#### 1.1 Goals \n",
"\n",
"In this lab, you will:\n",
"- Review the features of NumPy and Python that are used in Course 1"
]
},
{
"cell_type": "markdown",
"id": "8eed19e7",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"#### 1.2 Useful References\n",
"\n",
"- NumPy Documentation including a basic introduction: [NumPy.org](https://NumPy.org/doc/stable/)\n",
"- A challenging feature topic: [NumPy Broadcasting](https://NumPy.org/doc/stable/user/basics.broadcasting.html)\n"
]
},
{
"cell_type": "markdown",
"id": "4feabd06",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"#### 2 Python and NumPy \n",
"\n",
"Python is the programming language we will be using in this course. It has a set of numeric data types and arithmetic operations. NumPy is a library that extends the base capabilities of python to add a richer data set including more numeric types, vectors, matrices, and many matrix functions. NumPy and python work together fairly seamlessly. Python arithmetic operators work on NumPy data types and many NumPy functions will accept python data types.\n"
]
},
{
"cell_type": "markdown",
"id": "c5e38bfd",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"#### 3 Vectors\n",
""
]
},
{
"cell_type": "markdown",
"id": "3afb8d41",
"metadata": {},
"source": [
"##### 3.1 Abstract\n",
"\n",
"\n",
"
\n",
" \n",
"
\n",
"
\n",
" Vectors, as you will use them in this course, are ordered arrays of numbers. In notation, vectors are denoted \n",
" with lower case bold letters such as $\\mathbf{x}$. The elements of a vector are all the same type. A vector \n",
" does not, for example, contain both characters and numbers. The number of elements in the array is often \n",
" referred to as the *dimension* though mathematicians may prefer *rank*. The vector shown has a dimension of $n$. The elements of a vector can be referenced with an index. In math settings, indexes typically run from 1 to n. In computer science and these labs, indexing will typically run from 0 to n-1. In notation, elements of a vector, when referenced individually will indicate the index in a subscript, for example, the $0^{th}$ element, of the vector $\\mathbf{x}$ is $x_0$. Note, the x is not bold in this case. \n",
"
\n"
]
},
{
"cell_type": "markdown",
"id": "c9e73ec6",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### 3.2 NumPy Arrays\n",
"\n",
"\n",
"NumPy's basic data structure is an indexable, n-dimensional *array* containing elements of the same type (`dtype`). Right away, you may notice we have overloaded the term 'dimension'. Above, it was the number of elements in the vector, here, dimension refers to the number of indexes of an array. A one-dimensional or 1-D array has one index. In Course 1, we will represent vectors as NumPy 1-D arrays. \n",
"\n",
" - 1-D array, shape (n,): n elements indexed [0] through [n-1]\n",
" "
]
},
{
"cell_type": "markdown",
"id": "4c557076",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### 3.3 Vector Creation\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "34a5e347",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Data creation routines in NumPy will generally have a first parameter which is the shape of the object. This can either be a single value for a 1-D result or a tuple (n,m,...) specifying the shape of the result. Below are examples of creating vectors using these routines."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "25aac0da",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"# NumPy routines which allocate memory and fill arrays with value\n",
"a = np.zeros(4); print(f\"np.zeros(4) : a = {a}, a shape = {a.shape}, a data type = {a.dtype}\")\n",
"a = np.zeros((4,)); print(f\"np.zeros(4,) : a = {a}, a shape = {a.shape}, a data type = {a.dtype}\")\n",
"a = np.random.random_sample(4); print(f\"np.random.random_sample(4): a = {a}, a shape = {a.shape}, a data type = {a.dtype}\")"
]
},
{
"cell_type": "markdown",
"id": "3caf3a13",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Some data creation routines do not take a shape tuple:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8cd7c0b3",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"# NumPy routines which allocate memory and fill arrays with value but do not accept shape as input argument\n",
"a = np.arange(4.); print(f\"np.arange(4.): a = {a}, a shape = {a.shape}, a data type = {a.dtype}\")\n",
"a = np.random.rand(4); print(f\"np.random.rand(4): a = {a}, a shape = {a.shape}, a data type = {a.dtype}\")"
]
},
{
"cell_type": "markdown",
"id": "f107c53e",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"values can be specified manually as well. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9a4cc9b8",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"# NumPy routines which allocate memory and fill with user specified values\n",
"a = np.array([5,4,3,2]); print(f\"np.array([5,4,3,2]): a = {a}, a shape = {a.shape}, a data type = {a.dtype}\")\n",
"a = np.array([5.,4,3,2]); print(f\"np.array([5.,4,3,2]): a = {a}, a shape = {a.shape}, a data type = {a.dtype}\")"
]
},
{
"cell_type": "markdown",
"id": "1580f398",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"These have all created a one-dimensional vector `a` with four elements. `a.shape` returns the dimensions. Here we see a.shape = `(4,)` indicating a 1-d array with 4 elements. "
]
},
{
"cell_type": "markdown",
"id": "8f8a0cf6",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### 3.4 Operations on Vectors\n",
"\n",
"Let's explore some operations using vectors.\n",
"\n",
"###### 3.4.1 Indexing\n",
"\n",
"\n",
"Elements of vectors can be accessed via indexing and slicing. NumPy provides a very complete set of indexing and slicing capabilities. We will explore only the basics needed for the course here. Reference [Slicing and Indexing](https://NumPy.org/doc/stable/reference/arrays.indexing.html) for more details. \n",
"**Indexing** means referring to *an element* of an array by its position within the array. \n",
"**Slicing** means getting a *subset* of elements from an array based on their indices. \n",
"NumPy starts indexing at zero so the 3rd element of an vector $\\mathbf{a}$ is `a[2]`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "aed264a5",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"#vector indexing operations on 1-D vectors\n",
"a = np.arange(10)\n",
"print(a)\n",
"\n",
"#access an element\n",
"print(f\"a[2].shape: {a[2].shape} a[2] = {a[2]}, Accessing an element returns a scalar\")\n",
"\n",
"# access the last element, negative indexes count from the end\n",
"print(f\"a[-1] = {a[-1]}\")\n",
"\n",
"#indexs must be within the range of the vector or they will produce and error\n",
"try:\n",
" c = a[10]\n",
"except Exception as e:\n",
" print(\"The error message you'll see is:\")\n",
" print(e)"
]
},
{
"cell_type": "markdown",
"id": "09a98b39",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"###### 3.4.2 Slicing\n",
"\n",
"Slicing creates an array of indices using a set of three values (`start:stop:step`). A subset of values is also valid. Its use is best explained by example:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1d4b3666",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"#vector slicing operations\n",
"a = np.arange(10)\n",
"print(f\"a = {a}\")\n",
"\n",
"#access 5 consecutive elements (start:stop:step)\n",
"c = a[2:7:1]; print(\"a[2:7:1] = \", c)\n",
"\n",
"# access 3 elements separated by two \n",
"c = a[2:7:2]; print(\"a[2:7:2] = \", c)\n",
"\n",
"# access all elements index 3 and above\n",
"c = a[3:]; print(\"a[3:] = \", c)\n",
"\n",
"# access all elements below index 3\n",
"c = a[:3]; print(\"a[:3] = \", c)\n",
"\n",
"# access all elements\n",
"c = a[:]; print(\"a[:] = \", c)"
]
},
{
"cell_type": "markdown",
"id": "7bd222c2",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"###### 3.4.3 Single vector operations\n",
"\n",
"\n",
"There are a number of useful operations that involve operations on a single vector."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "52c244d1",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"a = np.array([1,2,3,4])\n",
"print(f\"a : {a}\")\n",
"# negate elements of a\n",
"b = -a \n",
"print(f\"b = -a : {b}\")\n",
"\n",
"# sum all elements of a, returns a scalar\n",
"b = np.sum(a) \n",
"print(f\"b = np.sum(a) : {b}\")\n",
"\n",
"b = np.mean(a)\n",
"print(f\"b = np.mean(a): {b}\")\n",
"\n",
"b = a**2\n",
"print(f\"b = a**2 : {b}\")"
]
},
{
"cell_type": "markdown",
"id": "408fcc40",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"###### 3.4.4 Vector Vector element-wise operations\n",
"\n",
"\n",
"Most of the NumPy arithmetic, logical and comparison operations apply to vectors as well. These operators work on an element-by-element basis. For example \n",
"$$ \\mathbf{a} + \\mathbf{b} = \\sum_{i=0}^{n-1} a_i + b_i $$"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0169f2ba",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"a = np.array([ 1, 2, 3, 4])\n",
"b = np.array([-1,-2, 3, 4])\n",
"print(f\"Binary operators work element wise: {a + b}\")"
]
},
{
"cell_type": "markdown",
"id": "c3d4350f",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Of course, for this to work correctly, the vectors must be of the same size:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2118077b",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"#try a mismatched vector operation\n",
"c = np.array([1, 2])\n",
"try:\n",
" d = a + c\n",
"except Exception as e:\n",
" print(\"The error message you'll see is:\")\n",
" print(e)"
]
},
{
"cell_type": "markdown",
"id": "4ee67081",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"###### 3.4.5 Scalar Vector operations\n",
"\n",
"\n",
"Vectors can be 'scaled' by scalar values. A scalar value is just a number. The scalar multiplies all the elements of the vector."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7bff1040",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"a = np.array([1, 2, 3, 4])\n",
"\n",
"# multiply a by a scalar\n",
"b = 5 * a \n",
"print(f\"b = 5 * a : {b}\")"
]
},
{
"cell_type": "markdown",
"id": "8651d003",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"###### 3.4.6 Vector Vector dot product\n",
"\n",
"\n",
"The dot product is a mainstay of Linear Algebra and NumPy. This is an operation used extensively in this course and should be well understood. The dot product is shown below."
]
},
{
"cell_type": "markdown",
"id": "615a4d74",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
" "
]
},
{
"cell_type": "markdown",
"id": "36aa2cda",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"The dot product multiplies the values in two vectors element-wise and then sums the result.\n",
"Vector dot product requires the dimensions of the two vectors to be the same. "
]
},
{
"cell_type": "markdown",
"id": "31cf42f3",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Let's implement our own version of the dot product below:\n",
"\n",
"**Using a for loop**, implement a function which returns the dot product of two vectors. The function to return given inputs $a$ and $b$:\n",
"$$ x = \\sum_{i=0}^{n-1} a_i b_i $$\n",
"Assume both `a` and `b` are the same shape."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4cccf6b1",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"def my_dot(a, b): \n",
" \"\"\"\n",
" Compute the dot product of two vectors\n",
" \n",
" Args:\n",
" a (ndarray (n,)): input vector \n",
" b (ndarray (n,)): input vector with same dimension as a\n",
" \n",
" Returns:\n",
" x (scalar): \n",
" \"\"\"\n",
" x=0\n",
" for i in range(a.shape[0]):\n",
" x = x + a[i] * b[i]\n",
" return x"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "95c40306",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"# test 1-D\n",
"a = np.array([1, 2, 3, 4])\n",
"b = np.array([-1, 4, 3, 2])\n",
"print(f\"my_dot(a, b) = {my_dot(a, b)}\")"
]
},
{
"cell_type": "markdown",
"id": "8aeeabc3",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Note, the dot product is expected to return a scalar value. \n",
"\n",
"Let's try the same operations using `np.dot`. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b50e32ca",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"# test 1-D\n",
"a = np.array([1, 2, 3, 4])\n",
"b = np.array([-1, 4, 3, 2])\n",
"c = np.dot(a, b)\n",
"print(f\"NumPy 1-D np.dot(a, b) = {c}, np.dot(a, b).shape = {c.shape} \") \n",
"c = np.dot(b, a)\n",
"print(f\"NumPy 1-D np.dot(b, a) = {c}, np.dot(a, b).shape = {c.shape} \")\n"
]
},
{
"cell_type": "markdown",
"id": "ba8b9cdc",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Above, you will note that the results for 1-D matched our implementation."
]
},
{
"cell_type": "markdown",
"id": "abcfcc14",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"###### 3.4.7 The Need for Speed: vector vs for loop\n",
"\n",
"\n",
"We utilized the NumPy library because it improves speed memory efficiency. Let's demonstrate:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "427fae1d",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"np.random.seed(1)\n",
"a = np.random.rand(10000000) # very large arrays\n",
"b = np.random.rand(10000000)\n",
"\n",
"tic = time.time() # capture start time\n",
"c = np.dot(a, b)\n",
"toc = time.time() # capture end time\n",
"\n",
"print(f\"np.dot(a, b) = {c:.4f}\")\n",
"print(f\"Vectorized version duration: {1000*(toc-tic):.4f} ms \")\n",
"\n",
"tic = time.time() # capture start time\n",
"c = my_dot(a,b)\n",
"toc = time.time() # capture end time\n",
"\n",
"print(f\"my_dot(a, b) = {c:.4f}\")\n",
"print(f\"loop version duration: {1000*(toc-tic):.4f} ms \")\n",
"\n",
"del(a);del(b) #remove these big arrays from memory"
]
},
{
"cell_type": "markdown",
"id": "21897335",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"So, vectorization provides a large speed up in this example. This is because NumPy makes better use of available data parallelism in the underlying hardware. GPU's and modern CPU's implement Single Instruction, Multiple Data (SIMD) pipelines allowing multiple operations to be issued in parallel. This is critical in Machine Learning where the data sets are often very large."
]
},
{
"cell_type": "markdown",
"id": "fc56006f",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"###### 3.4.8 Vector Vector operations in Course 1\n",
"\n",
"\n",
"Vector Vector operations will appear frequently in course 1. Here is why:\n",
"- Going forward, our examples will be stored in an array, `X_train` of dimension (m,n). This will be explained more in context, but here it is important to note it is a 2 Dimensional array or matrix (see next section on matrices).\n",
"- `w` will be a 1-dimensional vector of shape (n,).\n",
"- we will perform operations by looping through the examples, extracting each example to work on individually by indexing X. For example:`X[i]`\n",
"- `X[i]` returns a value of shape (n,), a 1-dimensional vector. Consequently, operations involving `X[i]` are often vector-vector. \n",
"\n",
"That is a somewhat lengthy explanation, but aligning and understanding the shapes of your operands is important when performing vector operations."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9eac936c",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"# show common Course 1 example\n",
"X = np.array([[1],[2],[3],[4]])\n",
"w = np.array([2])\n",
"c = np.dot(X[1], w)\n",
"\n",
"print(f\"X[1] has shape {X[1].shape}\")\n",
"print(f\"w has shape {w.shape}\")\n",
"print(f\"c has shape {c.shape}\")"
]
},
{
"cell_type": "markdown",
"id": "05e58839",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"#### 4 Matrices\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "9e5b0b36",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### 4.1 Abstract\n",
"\n",
"\n",
"Matrices, are two dimensional arrays. The elements of a matrix are all of the same type. In notation, matrices are denoted with capitol, bold letter such as $\\mathbf{X}$. In this and other labs, `m` is often the number of rows and `n` the number of columns. The elements of a matrix can be referenced with a two dimensional index. In math settings, numbers in the index typically run from 1 to n. In computer science and these labs, indexing will run from 0 to n-1. \n",
"
\n",
"
\n",
" Generic Matrix Notation, 1st index is row, 2nd is column \n",
""
]
},
{
"cell_type": "markdown",
"id": "88c4fda5",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### 4.2 NumPy Arrays\n",
"\n",
"\n",
"NumPy's basic data structure is an indexable, n-dimensional *array* containing elements of the same type (`dtype`). These were described earlier. Matrices have a two-dimensional (2-D) index [m,n].\n",
"\n",
"In Course 1, 2-D matrices are used to hold training data. Training data is $m$ examples by $n$ features creating an (m,n) array. Course 1 does not do operations directly on matrices but typically extracts an example as a vector and operates on that. Below you will review: \n",
"- data creation\n",
"- slicing and indexing"
]
},
{
"cell_type": "markdown",
"id": "45cba502",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### 4.3 Matrix Creation\n",
"\n",
"\n",
"The same functions that created 1-D vectors will create 2-D or n-D arrays. Here are some examples\n"
]
},
{
"cell_type": "markdown",
"id": "15f7050e",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Below, the shape tuple is provided to achieve a 2-D result. Notice how NumPy uses brackets to denote each dimension. Notice further than NumPy, when printing, will print one row per line.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1269c0be",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"a = np.zeros((1, 5)) \n",
"print(f\"a shape = {a.shape}, a = {a}\") \n",
"\n",
"a = np.zeros((2, 1)) \n",
"print(f\"a shape = {a.shape}, a = {a}\") \n",
"\n",
"a = np.random.random_sample((1, 1)) \n",
"print(f\"a shape = {a.shape}, a = {a}\") "
]
},
{
"cell_type": "markdown",
"id": "f17b4454",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"One can also manually specify data. Dimensions are specified with additional brackets matching the format in the printing above."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "203da595",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"# NumPy routines which allocate memory and fill with user specified values\n",
"a = np.array([[5], [4], [3]]); print(f\" a shape = {a.shape}, np.array: a = {a}\")\n",
"a = np.array([[5], # One can also\n",
" [4], # separate values\n",
" [3]]); #into separate rows\n",
"print(f\" a shape = {a.shape}, np.array: a = {a}\")"
]
},
{
"cell_type": "markdown",
"id": "fcebc352",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### 4.4 Operations on Matrices\n",
"\n",
"\n",
"Let's explore some operations using matrices."
]
},
{
"cell_type": "markdown",
"id": "0852ee28",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"###### 4.4.1 Indexing\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "ca8c4a30",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Matrices include a second index. The two indexes describe [row, column]. Access can either return an element or a row/column. See below:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bb8fa67f",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"#vector indexing operations on matrices\n",
"a = np.arange(6).reshape(-1, 2) #reshape is a convenient way to create matrices\n",
"print(f\"a.shape: {a.shape}, \\na= {a}\")\n",
"\n",
"#access an element\n",
"print(f\"\\na[2,0].shape: {a[2, 0].shape}, a[2,0] = {a[2, 0]}, type(a[2,0]) = {type(a[2, 0])} Accessing an element returns a scalar\\n\")\n",
"\n",
"#access a row\n",
"print(f\"a[2].shape: {a[2].shape}, a[2] = {a[2]}, type(a[2]) = {type(a[2])}\")"
]
},
{
"cell_type": "markdown",
"id": "5d66eadf",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"It is worth drawing attention to the last example. Accessing a matrix by just specifying the row will return a *1-D vector*."
]
},
{
"cell_type": "markdown",
"id": "2d803091",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"**Reshape** \n",
"The previous example used [reshape](https://numpy.org/doc/stable/reference/generated/numpy.reshape.html) to shape the array. \n",
"`a = np.arange(6).reshape(-1, 2) ` \n",
"This line of code first created a *1-D Vector* of six elements. It then reshaped that vector into a *2-D* array using the reshape command. This could have been written: \n",
"`a = np.arange(6).reshape(3, 2) ` \n",
"To arrive at the same 3 row, 2 column array.\n",
"The -1 argument tells the routine to compute the number of rows given the size of the array and the number of columns.\n"
]
},
{
"cell_type": "markdown",
"id": "118df767",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"###### 4.4.2 Slicing\n",
"\n",
"\n",
"Slicing creates an array of indices using a set of three values (`start:stop:step`). A subset of values is also valid. Its use is best explained by example:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "934be8fa",
"metadata": {
"pycharm": {
"name": "#%%\n"
},
"scrolled": true
},
"outputs": [],
"source": [
"#vector 2-D slicing operations\n",
"a = np.arange(20).reshape(-1, 10)\n",
"print(f\"a = \\n{a}\")\n",
"\n",
"#access 5 consecutive elements (start:stop:step)\n",
"print(\"a[0, 2:7:1] = \", a[0, 2:7:1], \", a[0, 2:7:1].shape =\", a[0, 2:7:1].shape, \"a 1-D array\")\n",
"\n",
"#access 5 consecutive elements (start:stop:step) in two rows\n",
"print(\"a[:, 2:7:1] = \\n\", a[:, 2:7:1], \", a[:, 2:7:1].shape =\", a[:, 2:7:1].shape, \"a 2-D array\")\n",
"\n",
"# access all elements\n",
"print(\"a[:,:] = \\n\", a[:,:], \", a[:,:].shape =\", a[:,:].shape)\n",
"\n",
"# access all elements in one row (very common usage)\n",
"print(\"a[1,:] = \", a[1,:], \", a[1,:].shape =\", a[1,:].shape, \"a 1-D array\")\n",
"# same as\n",
"print(\"a[1] = \", a[1], \", a[1].shape =\", a[1].shape, \"a 1-D array\")\n"
]
},
{
"cell_type": "markdown",
"id": "3624960f",
"metadata": {},
"source": [
"### Practice Quiz "
]
},
{
"cell_type": "markdown",
"id": "a1516b3a",
"metadata": {},
"source": [
"#### Quiz-1 "
]
},
{
"cell_type": "markdown",
"id": "cc8002d6",
"metadata": {},
"source": [
"
\n",
""
]
},
{
"cell_type": "markdown",
"id": "6d64f2fc",
"metadata": {},
"source": [
"### Assignment W2: \n"
]
},
{
"cell_type": "markdown",
"id": "30e6a206",
"metadata": {},
"source": [
"#### Practice Lab: Linear Regression\n",
"\n",
"Welcome to your first practice lab! In this lab, you will implement linear regression with one variable to predict profits for a restaurant franchise.\n",
"\n",
"\n",
"##### Outline\n",
"- [ 1 - Packages ](#1)\n",
"- [ 2 - Linear regression with one variable ](#2)\n",
"- [ 2.1 Problem Statement](#2.1)\n",
"- [ 3 Dataset](#3)\n",
"- [ 4 Refresher on linear regression](#4)\n",
"- [ 5 Compute Cost](#5)\n",
" - [ Exercise 1](#ex01)\n",
"- [ 6 Gradient descent ](#6)\n",
" - [ Exercise 2](#ex02)\n",
" - [ 6.1 Learning parameters using batch gradient descent ](#6.1)\n"
]
},
{
"cell_type": "markdown",
"id": "22a5e94a",
"metadata": {},
"source": [
"#### 1 - Packages \n",
"\n",
"\n",
"First, let's run the cell below to import all the packages that you will need during this assignment.\n",
"- [numpy](www.numpy.org) is the fundamental package for working with matrices in Python.\n",
"- [matplotlib](http://matplotlib.org) is a famous library to plot graphs in Python.\n",
"- ``utils.py`` contains helper functions for this assignment. You do not need to modify code in this file.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ebb0fd1d",
"metadata": {},
"outputs": [],
"source": [
"import sys\n",
"#add modules from the path\n",
"sys.path.append(\"/home/amitk/my_web/Machine-Learning-Andrew-Ng/source/source_files/Supervised_Machine_Learning_Regression_and_Classification/week2/C1W2A1\")\n",
"\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"\n",
"from utils import *\n",
"import copy\n",
"import math\n",
"%matplotlib inline \n",
"#to show graphs inline"
]
},
{
"cell_type": "markdown",
"id": "425b1ed5",
"metadata": {},
"source": [
"#### 2 - Problem Statement\n",
"\n",
"Suppose you are the CEO of a restaurant franchise and are considering different cities for opening a new outlet.\n",
"- You would like to expand your business to cities that may give your restaurant higher profits.\n",
"- The chain already has restaurants in various cities and you have data for profits and populations from the cities.\n",
"- You also have data on cities that are candidates for a new restaurant. \n",
" - For these cities, you have the city population.\n",
" \n",
"Can you use the data to help you identify which cities may potentially give your business higher profits?\n",
"\n",
"#### 3 - Dataset\n",
"\n",
"You will start by loading the dataset for this task. \n",
"- The `load_data()` function shown below loads the data into variables `x_train` and `y_train`\n",
" - `x_train` is the population of a city\n",
" - `y_train` is the profit of a restaurant in that city. A negative value for profit indicates a loss. \n",
" - Both `X_train` and `y_train` are numpy arrays."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "cb386022",
"metadata": {},
"outputs": [],
"source": [
"# load the dataset\n",
"x_train, y_train = load_data()"
]
},
{
"cell_type": "markdown",
"id": "492f2c9c",
"metadata": {},
"source": [
"##### View the variables\n",
"Before starting on any task, it is useful to get more familiar with your dataset. \n",
"- A good place to start is to just print out each variable and see what it contains.\n",
"\n",
"The code below prints the variable `x_train` and the type of the variable."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d19e9005",
"metadata": {},
"outputs": [],
"source": [
"# print x_train\n",
"print(\"Type of x_train:\",type(x_train))\n",
"print(\"First five elements of x_train are:\\n\", x_train[:5]) "
]
},
{
"cell_type": "markdown",
"id": "bb820aa5",
"metadata": {},
"source": [
"`x_train` is a numpy array that contains decimal values that are all greater than zero.\n",
"- These values represent the city population times 10,000\n",
"- For example, 6.1101 means that the population for that city is 61,101\n",
" \n",
"Now, let's print `y_train`"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4e65298e",
"metadata": {},
"outputs": [],
"source": [
"# print y_train\n",
"print(\"Type of y_train:\",type(y_train))\n",
"print(\"First five elements of y_train are:\\n\", y_train[:5]) "
]
},
{
"cell_type": "markdown",
"id": "4338e255",
"metadata": {},
"source": [
"Similarly, `y_train` is a numpy array that has decimal values, some negative, some positive.\n",
"- These represent your restaurant's average monthly profits in each city, in units of \\$10,000.\n",
" - For example, 17.592 represents \\$175,920 in average monthly profits for that city.\n",
" - -2.6807 represents -\\$26,807 in average monthly loss for that city."
]
},
{
"cell_type": "markdown",
"id": "f0bd0ace",
"metadata": {},
"source": [
"##### Check the dimensions of your variables\n",
"\n",
"Another useful way to get familiar with your data is to view its dimensions.\n",
"\n",
"Please print the shape of `x_train` and `y_train` and see how many training examples you have in your dataset."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "da8ceba3",
"metadata": {},
"outputs": [],
"source": [
"print ('The shape of x_train is:', x_train.shape)\n",
"print ('The shape of y_train is: ', y_train.shape)\n",
"print ('Number of training examples (m):', len(x_train))"
]
},
{
"cell_type": "markdown",
"id": "1edcdd3f",
"metadata": {},
"source": [
"The city population array has 97 data points, and the monthly average profits also has 97 data points. These are NumPy 1D arrays."
]
},
{
"cell_type": "markdown",
"id": "65fcd323",
"metadata": {},
"source": [
"##### Visualize your data\n",
"\n",
"It is often useful to understand the data by visualizing it. \n",
"- For this dataset, you can use a scatter plot to visualize the data, since it has only two properties to plot (profit and population). \n",
"- Many other problems that you will encounter in real life have more than two properties (for example, population, average household income, monthly profits, monthly sales).When you have more than two properties, you can still use a scatter plot to see the relationship between each pair of properties.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fd2f807a",
"metadata": {},
"outputs": [],
"source": [
"# Create a scatter plot of the data. To change the markers to red \"x\",\n",
"# we used the 'marker' and 'c' parameters\n",
"plt.scatter(x_train, y_train, marker='x', c='r') \n",
"\n",
"# Set the title\n",
"plt.title(\"Profits vs. Population per city\")\n",
"# Set the y-axis label\n",
"plt.ylabel('Profit in $10,000')\n",
"# Set the x-axis label\n",
"plt.xlabel('Population of City in 10,000s')\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "1294b699",
"metadata": {},
"source": [
"Your goal is to build a linear regression model to fit this data.\n",
"- With this model, you can then input a new city's population, and have the model estimate your restaurant's potential monthly profits for that city."
]
},
{
"cell_type": "markdown",
"id": "a86e4912",
"metadata": {},
"source": [
"#### 4 - Refresher on linear regression\n",
"\n",
"\n",
"\n",
"In this practice lab, you will fit the linear regression parameters $(w,b)$ to your dataset.\n",
"- The model function for linear regression, which is a function that maps from `x` (city population) to `y` (your restaurant's monthly profit for that city) is represented as \n",
" $$f_{w,b}(x) = wx + b$$\n",
" \n",
"\n",
"- To train a linear regression model, you want to find the best $(w,b)$ parameters that fit your dataset. \n",
"\n",
" - To compare how one choice of $(w,b)$ is better or worse than another choice, you can evaluate it with a cost function $J(w,b)$\n",
" - $J$ is a function of $(w,b)$. That is, the value of the cost $J(w,b)$ depends on the value of $(w,b)$.\n",
" \n",
" - The choice of $(w,b)$ that fits your data the best is the one that has the smallest cost $J(w,b)$.\n",
"\n",
"\n",
"- To find the values $(w,b)$ that gets the smallest possible cost $J(w,b)$, you can use a method called **gradient descent**. \n",
" - With each step of gradient descent, your parameters $(w,b)$ come closer to the optimal values that will achieve the lowest cost $J(w,b)$.\n",
" \n",
"\n",
"- The trained linear regression model can then take the input feature $x$ (city population) and output a prediction $f_{w,b}(x)$ (predicted monthly profit for a restaurant in that city)."
]
},
{
"cell_type": "markdown",
"id": "0819a0e6",
"metadata": {},
"source": [
"#### 5 - Compute Cost\n",
"\n",
"\n",
"Gradient descent involves repeated steps to adjust the value of your parameter $(w,b)$ to gradually get a smaller and smaller cost $J(w,b)$.\n",
"- At each step of gradient descent, it will be helpful for you to monitor your progress by computing the cost $J(w,b)$ as $(w,b)$ gets updated. \n",
"- In this section, you will implement a function to calculate $J(w,b)$ so that you can check the progress of your gradient descent implementation.\n",
"\n",
"##### Cost function\n",
"As you may recall from the lecture, for one variable, the cost function for linear regression $J(w,b)$ is defined as\n",
"\n",
"$$J(w,b) = \\frac{1}{2m} \\sum\\limits_{i = 0}^{m-1} (f_{w,b}(x^{(i)}) - y^{(i)})^2$$ \n",
"\n",
"- You can think of $f_{w,b}(x^{(i)})$ as the model's prediction of your restaurant's profit, as opposed to $y^{(i)}$, which is the actual profit that is recorded in the data.\n",
"- $m$ is the number of training examples in the dataset\n",
"\n",
"##### Model prediction\n",
"\n",
"- For linear regression with one variable, the prediction of the model $f_{w,b}$ for an example $x^{(i)}$ is representented as:\n",
"\n",
"$$ f_{w,b}(x^{(i)}) = wx^{(i)} + b$$\n",
"\n",
"This is the equation for a line, with an intercept $b$ and a slope $w$\n",
"\n",
"##### Implementation\n",
"\n",
"Please complete the `compute_cost()` function below to compute the cost $J(w,b)$."
]
},
{
"cell_type": "markdown",
"id": "ed5704b1",
"metadata": {},
"source": [
"#### Exercise 1\n",
"\n",
"\n",
"Complete the `compute_cost` below to:\n",
"\n",
"* Iterate over the training examples, and for each example, compute:\n",
" * The prediction of the model for that example \n",
" $$\n",
" f_{wb}(x^{(i)}) = wx^{(i)} + b \n",
" $$\n",
" \n",
" * The cost for that example $$cost^{(i)} = (f_{wb} - y^{(i)})^2$$\n",
" \n",
"\n",
"* Return the total cost over all examples\n",
"$$J(\\mathbf{w},b) = \\frac{1}{2m} \\sum\\limits_{i = 0}^{m-1} cost^{(i)}$$\n",
" * Here, $m$ is the number of training examples and $\\sum$ is the summation operator\n",
"\n",
"If you get stuck, you can check out the hints presented after the cell below to help you with the implementation."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "575a6d9e",
"metadata": {},
"outputs": [],
"source": [
"# UNQ_C1\n",
"# GRADED FUNCTION: compute_cost\n",
"\n",
"def compute_cost(x, y, w, b): \n",
" \"\"\"\n",
" Computes the cost function for linear regression.\n",
" \n",
" Args:\n",
" x (ndarray): Shape (m,) Input to the model (Population of cities) \n",
" y (ndarray): Shape (m,) Label (Actual profits for the cities)\n",
" w, b (scalar): Parameters of the model\n",
" \n",
" Returns\n",
" total_cost (float): The cost of using w,b as the parameters for linear regression\n",
" to fit the data points in x and y\n",
" \"\"\"\n",
" # number of training examples\n",
" m = x.shape[0] \n",
" \n",
" # You need to return this variable correctly\n",
" total_cost = 0\n",
"\n",
" ### START CODE HERE ###\n",
" cost=0\n",
" for i in range(m):\n",
" f_wb = w*x[i]+b\n",
" cost += (f_wb - y[i])**2\n",
" \n",
" total_cost = cost/(2*m)\n",
" \n",
" ### END CODE HERE ### \n",
"\n",
" return total_cost"
]
},
{
"cell_type": "markdown",
"id": "4bdd9017",
"metadata": {},
"source": [
"\n",
" Click for hints\n",
" \n",
" \n",
" * You can represent a summation operator eg: $h = \\sum\\limits_{i = 0}^{m-1} 2i$ in code as follows:\n",
" ```python \n",
" h = 0\n",
" for i in range(m):\n",
" h = h + 2*i\n",
" ```\n",
" \n",
" * In this case, you can iterate over all the examples in `x` using a for loop and add the `cost` from each iteration to a variable (`cost_sum`) initialized outside the loop.\n",
"\n",
" * Then, you can return the `total_cost` as `cost_sum` divided by `2m`.\n",
" \n",
" \n",
" Click for more hints\n",
" \n",
" * Here's how you can structure the overall implementation for this function\n",
" ```python \n",
" def compute_cost(x, y, w, b):\n",
" # number of training examples\n",
" m = x.shape[0] \n",
" \n",
" # You need to return this variable correctly\n",
" total_cost = 0\n",
" \n",
" ### START CODE HERE ### \n",
" # Variable to keep track of sum of cost from each example\n",
" cost_sum = 0\n",
" \n",
" # Loop over training examples\n",
" for i in range(m):\n",
" # Your code here to get the prediction f_wb for the ith example\n",
" f_wb = \n",
" # Your code here to get the cost associated with the ith example\n",
" cost = \n",
" \n",
" # Add to sum of cost for each example\n",
" cost_sum = cost_sum + cost \n",
"\n",
" # Get the total cost as the sum divided by (2*m)\n",
" total_cost = (1 / (2 * m)) * cost_sum\n",
" ### END CODE HERE ### \n",
"\n",
" return total_cost\n",
" ```\n",
" \n",
" If you're still stuck, you can check the hints presented below to figure out how to calculate `f_wb` and `cost`.\n",
" \n",
" \n",
" Hint to calculate f_wb For scalars $a$, $b$ and $c$ (x[i], w and b are all scalars), you can calculate the equation $h = ab + c$ in code as h = a * b + c\n",
" \n",
" More hints to calculate f You can compute f_wb as f_wb = w * x[i] + b \n",
" \n",
" \n",
"\n",
" \n",
" Hint to calculate cost You can calculate the square of a variable z as z**2\n",
" \n",
" More hints to calculate cost You can compute cost as cost = (f_wb - y[i]) ** 2\n",
" \n",
" \n",
" \n",
" \n",
"\n",
"\n",
" \n"
]
},
{
"cell_type": "markdown",
"id": "ac5bcb29",
"metadata": {},
"source": [
"You can check if your implementation was correct by running the following test code:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e52a6e0b",
"metadata": {},
"outputs": [],
"source": [
"# Compute cost with some initial values for paramaters w, b\n",
"initial_w = 2\n",
"initial_b = 1\n",
"\n",
"cost = compute_cost(x_train, y_train, initial_w, initial_b)\n",
"print(type(cost))\n",
"print(f'Cost at initial w (zeros): {cost:.3f}')\n",
"\n",
"# Public tests\n",
"from public_tests import *\n",
"compute_cost_test(compute_cost)"
]
},
{
"cell_type": "markdown",
"id": "1ff4d7d1",
"metadata": {},
"source": [
"**Expected Output**:\n",
"
\n",
"
\n",
"
Cost at initial w (zeros): 75.203
\n",
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"id": "39fdb569",
"metadata": {},
"source": [
"#### 6 - Gradient descent \n",
"\n",
"\n",
"In this section, you will implement the gradient for parameters $w, b$ for linear regression. "
]
},
{
"cell_type": "markdown",
"id": "c22f102d",
"metadata": {},
"source": [
"As described in the lecture videos, the gradient descent algorithm is:\n",
"\n",
"$$\\begin{align*}& \\text{repeat until convergence:} \\; \\lbrace \\newline \\; & \\phantom {0000} b := b - \\alpha \\frac{\\partial J(w,b)}{\\partial b} \\newline \\; & \\phantom {0000} w := w - \\alpha \\frac{\\partial J(w,b)}{\\partial w} \\tag{1} \\; & \n",
"\\newline & \\rbrace\\end{align*}$$\n",
"\n",
"where, parameters $w, b$ are both updated simultaniously and where \n",
"$$\n",
"\\frac{\\partial J(w,b)}{\\partial b} = \\frac{1}{m} \\sum\\limits_{i = 0}^{m-1} (f_{w,b}(x^{(i)}) - y^{(i)}) \\tag{2}\n",
"$$\n",
"$$\n",
"\\frac{\\partial J(w,b)}{\\partial w} = \\frac{1}{m} \\sum\\limits_{i = 0}^{m-1} (f_{w,b}(x^{(i)}) -y^{(i)})x^{(i)} \\tag{3}\n",
"$$\n",
"* m is the number of training examples in the dataset\n",
"\n",
" \n",
"* $f_{w,b}(x^{(i)})$ is the model's prediction, while $y^{(i)}$, is the target value\n",
"\n",
"\n",
"You will implement a function called `compute_gradient` which calculates $\\frac{\\partial J(w)}{\\partial w}$, $\\frac{\\partial J(w)}{\\partial b}$ "
]
},
{
"cell_type": "markdown",
"id": "5162e280",
"metadata": {},
"source": [
"##### Exercise 2\n",
"\n",
"\n",
"Please complete the `compute_gradient` function to:\n",
"\n",
"* Iterate over the training examples, and for each example, compute:\n",
" * The prediction of the model for that example \n",
" $$\n",
" f_{wb}(x^{(i)}) = wx^{(i)} + b \n",
" $$\n",
" \n",
" * The gradient for the parameters $w, b$ from that example \n",
" $$\n",
" \\frac{\\partial J(w,b)}{\\partial b}^{(i)} = (f_{w,b}(x^{(i)}) - y^{(i)}) \n",
" $$\n",
" $$\n",
" \\frac{\\partial J(w,b)}{\\partial w}^{(i)} = (f_{w,b}(x^{(i)}) -y^{(i)})x^{(i)} \n",
" $$\n",
" \n",
"\n",
"* Return the total gradient update from all the examples\n",
" $$\n",
" \\frac{\\partial J(w,b)}{\\partial b} = \\frac{1}{m} \\sum\\limits_{i = 0}^{m-1} \\frac{\\partial J(w,b)}{\\partial b}^{(i)}\n",
" $$\n",
" \n",
" $$\n",
" \\frac{\\partial J(w,b)}{\\partial w} = \\frac{1}{m} \\sum\\limits_{i = 0}^{m-1} \\frac{\\partial J(w,b)}{\\partial w}^{(i)} \n",
" $$\n",
" * Here, $m$ is the number of training examples and $\\sum$ is the summation operator\n",
"\n",
"If you get stuck, you can check out the hints presented after the cell below to help you with the implementation."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dd59d520",
"metadata": {},
"outputs": [],
"source": [
"# UNQ_C2\n",
"# GRADED FUNCTION: compute_gradient\n",
"def compute_gradient(x, y, w, b): \n",
" \"\"\"\n",
" Computes the gradient for linear regression \n",
" Args:\n",
" x (ndarray): Shape (m,) Input to the model (Population of cities) \n",
" y (ndarray): Shape (m,) Label (Actual profits for the cities)\n",
" w, b (scalar): Parameters of the model \n",
" Returns\n",
" dj_dw (scalar): The gradient of the cost w.r.t. the parameters w\n",
" dj_db (scalar): The gradient of the cost w.r.t. the parameter b \n",
" \"\"\"\n",
" \n",
" # Number of training examples\n",
" m = x.shape[0]\n",
" \n",
" # You need to return the following variables correctly\n",
" dj_dw = 0\n",
" dj_db = 0\n",
" \n",
" ### START CODE HERE ### \n",
" for i in range(m):\n",
" f_wb = w*x[i]+b\n",
" dj_db += f_wb - y[i]\n",
" dj_dw += (f_wb - y[i])*x[i]\n",
" dj_dw /= m\n",
" dj_db /= m\n",
" \n",
" ### END CODE HERE ### \n",
" \n",
" return dj_dw, dj_db"
]
},
{
"cell_type": "markdown",
"id": "03265cc0",
"metadata": {},
"source": [
"\n",
" Click for hints\n",
" \n",
" * You can represent a summation operator eg: $h = \\sum\\limits_{i = 0}^{m-1} 2i$ in code as follows:\n",
" ```python \n",
" h = 0\n",
" for i in range(m):\n",
" h = h + 2*i\n",
" ```\n",
" \n",
" * In this case, you can iterate over all the examples in `x` using a for loop and for each example, keep adding the gradient from that example to the variables `dj_dw` and `dj_db` which are initialized outside the loop. \n",
"\n",
" * Then, you can return `dj_dw` and `dj_db` both divided by `m`. \n",
" \n",
" Click for more hints\n",
" \n",
" * Here's how you can structure the overall implementation for this function\n",
" ```python \n",
" def compute_gradient(x, y, w, b): \n",
" \"\"\"\n",
" Computes the gradient for linear regression \n",
" Args:\n",
" x (ndarray): Shape (m,) Input to the model (Population of cities) \n",
" y (ndarray): Shape (m,) Label (Actual profits for the cities)\n",
" w, b (scalar): Parameters of the model \n",
" Returns\n",
" dj_dw (scalar): The gradient of the cost w.r.t. the parameters w\n",
" dj_db (scalar): The gradient of the cost w.r.t. the parameter b \n",
" \"\"\"\n",
" \n",
" # Number of training examples\n",
" m = x.shape[0]\n",
" \n",
" # You need to return the following variables correctly\n",
" dj_dw = 0\n",
" dj_db = 0\n",
" \n",
" ### START CODE HERE ### \n",
" # Loop over examples\n",
" for i in range(m): \n",
" # Your code here to get prediction f_wb for the ith example\n",
" f_wb = \n",
" \n",
" # Your code here to get the gradient for w from the ith example \n",
" dj_dw_i = \n",
" \n",
" # Your code here to get the gradient for b from the ith example \n",
" dj_db_i = \n",
" \n",
" # Update dj_db : In Python, a += 1 is the same as a = a + 1\n",
" dj_db += dj_db_i\n",
" \n",
" # Update dj_dw\n",
" dj_dw += dj_dw_i\n",
" \n",
" # Divide both dj_dw and dj_db by m\n",
" dj_dw = dj_dw / m\n",
" dj_db = dj_db / m\n",
" ### END CODE HERE ### \n",
" \n",
" return dj_dw, dj_db\n",
" ```\n",
" \n",
" If you're still stuck, you can check the hints presented below to figure out how to calculate `f_wb` and `cost`.\n",
" \n",
" \n",
" Hint to calculate f_wb\n",
" You did this in the previous exercise! For scalars $a$, $b$ and $c$ (x[i], w and b are all scalars), you can calculate the equation $h = ab + c$ in code as h = a * b + c\n",
" \n",
" More hints to calculate f\n",
" You can compute f_wb as f_wb = w * x[i] + b \n",
" \n",
" \n",
" \n",
" \n",
" Hint to calculate dj_dw_i\n",
" For scalars $a$, $b$ and $c$ (f_wb, y[i] and x[i] are all scalars), you can calculate the equation $h = (a - b)c$ in code as h = (a-b)*c\n",
" \n",
" More hints to calculate f\n",
" You can compute dj_dw_i as dj_dw_i = (f_wb - y[i]) * x[i] \n",
" \n",
" \n",
" \n",
" \n",
" Hint to calculate dj_db_i\n",
" You can compute dj_db_i as dj_db_i = f_wb - y[i] \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"\n",
" \n"
]
},
{
"cell_type": "markdown",
"id": "247d6609",
"metadata": {},
"source": [
"Run the cells below to check your implementation of the `compute_gradient` function with two different initializations of the parameters $w$,$b$."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b9b34b8a",
"metadata": {},
"outputs": [],
"source": [
"# Compute and display gradient with w initialized to zeroes\n",
"initial_w = 0\n",
"initial_b = 0\n",
"\n",
"tmp_dj_dw, tmp_dj_db = compute_gradient(x_train, y_train, initial_w, initial_b)\n",
"print('Gradient at initial w, b (zeros):', tmp_dj_dw, tmp_dj_db)\n",
"\n",
"compute_gradient_test(compute_gradient)"
]
},
{
"cell_type": "markdown",
"id": "cd7603c1",
"metadata": {},
"source": [
"Now let's run the gradient descent algorithm implemented above on our dataset.\n",
"\n",
"**Expected Output**:\n",
"
"
]
},
{
"cell_type": "markdown",
"id": "d13b681c",
"metadata": {},
"source": [
"##### 6.1 Learning parameters using batch gradient descent \n",
"\n",
"\n",
"You will now find the optimal parameters of a linear regression model by using batch gradient descent. Recall batch refers to running all the examples in one iteration.\n",
"- You don't need to implement anything for this part. Simply run the cells below. \n",
"\n",
"- A good way to verify that gradient descent is working correctly is to look\n",
"at the value of $J(w,b)$ and check that it is decreasing with each step. \n",
"\n",
"- Assuming you have implemented the gradient and computed the cost correctly and you have an appropriate value for the learning rate alpha, $J(w,b)$ should never increase and should converge to a steady value by the end of the algorithm."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "47307808",
"metadata": {},
"outputs": [],
"source": [
"def gradient_descent(x, y, w_in, b_in, cost_function, gradient_function, alpha, num_iters): \n",
" \"\"\"\n",
" Performs batch gradient descent to learn theta. Updates theta by taking \n",
" num_iters gradient steps with learning rate alpha\n",
" \n",
" Args:\n",
" x : (ndarray): Shape (m,)\n",
" y : (ndarray): Shape (m,)\n",
" w_in, b_in : (scalar) Initial values of parameters of the model\n",
" cost_function: function to compute cost\n",
" gradient_function: function to compute the gradient\n",
" alpha : (float) Learning rate\n",
" num_iters : (int) number of iterations to run gradient descent\n",
" Returns\n",
" w : (ndarray): Shape (1,) Updated values of parameters of the model after\n",
" running gradient descent\n",
" b : (scalar) Updated value of parameter of the model after\n",
" running gradient descent\n",
" \"\"\"\n",
" \n",
" # number of training examples\n",
" m = len(x)\n",
" \n",
" # An array to store cost J and w's at each iteration — primarily for graphing later\n",
" J_history = []\n",
" w_history = []\n",
" w = copy.deepcopy(w_in) #avoid modifying global w within function\n",
" b = b_in\n",
" \n",
" for i in range(num_iters):\n",
"\n",
" # Calculate the gradient and update the parameters\n",
" dj_dw, dj_db = gradient_function(x, y, w, b ) \n",
" \n",
" # Update Parameters using w, b, alpha and gradient\n",
" w = w - alpha * dj_dw \n",
" b = b - alpha * dj_db \n",
"\n",
" # Save cost J at each iteration\n",
" if i<100000: # prevent resource exhaustion \n",
" cost = cost_function(x, y, w, b)\n",
" J_history.append(cost)\n",
"\n",
" # Print cost every at intervals 10 times or as many iterations if < 10\n",
" if i% math.ceil(num_iters/10) == 0:\n",
" w_history.append(w)\n",
" print(f\"Iteration {i:4}: Cost {float(J_history[-1]):8.2f} \")\n",
" \n",
" return w, b, J_history, w_history #return w and J,w history for graphing"
]
},
{
"cell_type": "markdown",
"id": "3302fed3",
"metadata": {},
"source": [
"Now let's run the gradient descent algorithm above to learn the parameters for our dataset."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c16c485b",
"metadata": {},
"outputs": [],
"source": [
"# initialize fitting parameters. Recall that the shape of w is (n,)\n",
"initial_w = 20\n",
"initial_b = 5\n",
"\n",
"# some gradient descent settings\n",
"iterations = 15000\n",
"alpha = 0.01\n",
"\n",
"w,b,_,_ = gradient_descent(x_train ,y_train, initial_w, initial_b, compute_cost, compute_gradient, alpha, iterations)\n",
"print(\"w,b found by gradient descent:\", w, b)"
]
},
{
"cell_type": "markdown",
"id": "fe7c4c79",
"metadata": {},
"source": [
"**Expected Output**:\n",
"
\n",
"
\n",
"
w, b found by gradient descent
\n",
"
1.16636235 -3.63029143940436
\n",
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"id": "0c0460e9",
"metadata": {},
"source": [
"We will now use the final parameters from gradient descent to plot the linear fit. \n",
"\n",
"Recall that we can get the prediction for a single example $f(x^{(i)})= wx^{(i)}+b$. \n",
"\n",
"To calculate the predictions on the entire dataset, we can loop through all the training examples and calculate the prediction for each example. This is shown in the code block below."
]
},
{
"cell_type": "markdown",
"id": "908a3ada",
"metadata": {},
"source": [
"##### Assignment2: My solution"
]
},
{
"cell_type": "markdown",
"id": "35d5b587",
"metadata": {},
"source": [
"\n",
" My Solution\n",
" \n",
" ```python\n",
" \n",
"#my solution: Dictate learning rate automatically,costrain parameter within boundry\n",
"\n",
"import sys\n",
"#add modules from the path\n",
"sys.path.append(\"/home/amitk/my_web/Machine-Learning-Andrew-Ng/source/source_files/Supervised_Machine_Learning_Regression_and_Classification/week2/C1W2A1\")\n",
"\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"\n",
"from utils import *\n",
"import copy\n",
"import math\n",
"%matplotlib inline \n",
"\n",
"#to show graphs inline\n",
"# load the dataset\n",
"#x_train, y_train = load_data()\n",
"x_train=np.linspace(5,25,100)\n",
"y_train= 3*x_train-8 + np.random.normal(0,1,len(x_train)) \n",
"\n",
"\n",
"def model(x,theta):\n",
" w,b=theta\n",
" return w*x+b\n",
"\n",
"def dmodel_w(x,theta): \n",
" w,b=theta\n",
" return x\n",
"\n",
"def dmodel_b(x,theta): \n",
" w,b=theta\n",
" return 1.\n",
"\n",
"\n",
"def cost(x,theta,y):\n",
" cf= ( model(x,theta) - y)**2\n",
" return np.sum(cf)/2/np.shape(x_train)[0]\n",
"\n",
"def dcost_w(x,theta,y):\n",
" return np.sum((model(x,theta)-y)*dmodel_w(x,theta))/len(x)\n",
"\n",
"def dcost_b(x,theta,y):\n",
" return np.sum((model(x,theta)-y)*dmodel_b(x,theta))/len(x)\n",
" \n",
"def compute_gradient(x,theta,y):\n",
" return dcost_w(x,theta,y),dcost_b(x,theta,y)\n",
"\n",
"np.set_printoptions(precision=2)\n",
"def gradient_decent(x,y,theta,alpha,niter):\n",
" w,b=theta\n",
" if theta[1]>0: #constraining parameters\n",
" b=-theta[1]\n",
" cost_i=np.zeros(niter)\n",
" for i in np.arange(niter):\n",
" if i>1:\n",
" if np.abs((cost_i[i]-cost_i[i-1])/cost_i[i])<0.05:\n",
" alpha/=2\n",
" \n",
" dcw,dcb= compute_gradient(x,theta,y)\n",
" w = w-alpha*dcw\n",
" b = b-alpha*dcb\n",
" theta=w,b\n",
" cost_i[i]=cost(x,theta,y)\n",
" if i>1:\n",
" if cost_i[i]>cost_i[i-1]:\n",
" alpha/=2\n",
" #print(cost_i[i],alpha)\n",
" #print(theta) \n",
" return cost_i,theta\n",
"\n",
" \n",
" \n",
"niter=10000\n",
"Win=20\n",
"Bin=5\n",
"alpha=0.5\n",
"theta_in=Win,Bin\n",
"grad_dec_result,theta_f=gradient_decent(x_train,y_train,theta_in,alpha,niter) \n",
"\n",
"wf,bf=theta_f\n",
"print(wf,bf,grad_dec_result[-1])\n",
"#print(compute_gradient(x_train,y_train,0.2,0.2))\n",
"ax=plt.subplot(121)\n",
"\n",
"plt.plot(np.arange(niter),grad_dec_result,\".\")\n",
"plt.yscale(\"log\")\n",
"plt.xlabel(\"No of steps\")\n",
"plt.ylabel(\"Cost function\")\n",
"plt.ylim(bottom=0.01)\n",
"#plt.xlim(0,100)\n",
"#plt.show()\n",
"m = x_train.shape[0]\n",
"predictedamit = np.zeros(m)\n",
"\n",
"for i in range(m):\n",
" predictedamit[i] = wf * x_train[i] + bf\n",
"\n",
" \n",
"ax=plt.subplot(122) \n",
"# Plot the linear fit\n",
"#plt.plot(x_train, predicted, c = \"b\")\n",
"plt.plot(x_train, predictedamit, c = \"g\",label=\"Predcited model\")\n",
"\n",
"# Create a scatter plot of the data. \n",
"plt.scatter(x_train, y_train, marker='x', c='r') \n",
"\n",
"# Set the title\n",
"plt.title(\"Model fit\")\n",
"# Set the y-axis label\n",
"plt.ylabel('training data')\n",
"# Set the x-axis label\n",
"plt.xlabel('training input') \n",
"plt.legend()\n",
"plt.tight_layout()\n",
" \n",
" ```\n",
"\n",
"\n",
"\n",
"\n",
" \n"
]
},
{
"cell_type": "markdown",
"id": "88f79b65",
"metadata": {},
"source": [
"\n",
" How to write summary\n",
" \n",
" ```python\n",
" import math\n",
" %matplotlib inline \n",
" plt.xlabel('Area of triangle')\n",
" ```\n",
"\n",
" \n",
" See hints\n",
" \n",
" ```python\n",
" import math\n",
" %matplotlib inline \n",
" plt.xlabel('Area of triangle')\n",
" ```\n",
" \n",
" \n",
"\n",
" \n",
"\n",
"\n",
"\n",
" \n",
" Hint to calculate f_wb For scalars $a$, $b$ and $c$ (x[i], w and b are all scalars), you can calculate the equation $h = ab + c$ in code as h = a * b + c\n",
" \n",
" More hints to calculate f You can compute f_wb as f_wb = w * x[i] + b \n",
" \n",
" \n",
"\n",
" \n",
" Hint to calculate cost You can calculate the square of a variable z as z**2\n",
" \n",
" More hints to calculate cost You can compute cost as cost = (f_wb - y[i]) ** 2\n",
" \n",
" \n",
"\n",
"\n",
" \n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1180e202",
"metadata": {},
"outputs": [],
"source": [
"m = x_train.shape[0]\n",
"predicted = np.zeros(m)\n",
"\n",
"for i in range(m):\n",
" predicted[i] = w * x_train[i] + b\n",
" \n",
" "
]
},
{
"cell_type": "markdown",
"id": "c973eb09",
"metadata": {},
"source": [
"We will now plot the predicted values to see the linear fit."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "238e66d0",
"metadata": {},
"outputs": [],
"source": [
"# Plot the linear fit\n",
"plt.plot(x_train, predicted, c = \"b\")\n",
"#plt.plot(x_train, predictedamit, c = \"g\")\n",
"\n",
"# Create a scatter plot of the data. \n",
"plt.scatter(x_train, y_train, marker='x', c='r') \n",
"\n",
"# Set the title\n",
"plt.title(\"Profits vs. Population per city\")\n",
"# Set the y-axis label\n",
"plt.ylabel('Profit in $10,000')\n",
"# Set the x-axis label\n",
"plt.xlabel('Population of City in 10,000s')"
]
},
{
"cell_type": "markdown",
"id": "e3f2dba9",
"metadata": {},
"source": [
"Your final values of $w,b$ can also be used to make predictions on profits. Let's predict what the profit would be in areas of 35,000 and 70,000 people. \n",
"\n",
"- The model takes in population of a city in 10,000s as input. \n",
"\n",
"- Therefore, 35,000 people can be translated into an input to the model as `np.array([3.5])`\n",
"\n",
"- Similarly, 70,000 people can be translated into an input to the model as `np.array([7.])`\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0395000c",
"metadata": {},
"outputs": [],
"source": [
"predict1 = 3.5 * w + b\n",
"print('For population = 35,000, we predict a profit of $%.2f' % (predict1*10000))\n",
"\n",
"predict2 = 7.0 * w + b\n",
"print('For population = 70,000, we predict a profit of $%.2f' % (predict2*10000))"
]
},
{
"cell_type": "markdown",
"id": "4029bda9",
"metadata": {},
"source": [
"**Expected Output**:\n",
"
\n"
]
},
{
"cell_type": "markdown",
"id": "f2acf972",
"metadata": {},
"source": [
"## Module - 3"
]
},
{
"cell_type": "markdown",
"id": "d5c72459",
"metadata": {},
"source": [
"### Optional Lab W3"
]
},
{
"cell_type": "markdown",
"id": "3941ceb8",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"#### Optional Lab - 3.1: Classification\n",
"\n",
"In this lab, you will contrast regression and classification."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "cbd4f7cc",
"metadata": {},
"outputs": [],
"source": [
"import os,sys\n",
"proj_path=f\"{os.environ['HOME']}/my_web/Machine-Learning-Andrew-Ng\"\n",
"module3=f\"{proj_path}/source/source_files/Supervised_Machine_Learning_Regression_and_Classification/\"\n",
"os.chdir(module3)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2572523f",
"metadata": {},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"plt.style.use(\"week3/OptionalLabs/deeplearning.mplstyle\")\n",
"sys.path.append(f\"{module3}/week3/OptionalLabs\")\n",
"from lab_utils_common import dlc, plot_data\n",
"from plt_one_addpt_onclick import plt_one_addpt_onclick\n",
"import numpy as np"
]
},
{
"cell_type": "markdown",
"id": "f31ffb13",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Classification Problems\n",
" Examples of classification problems are things like: identifying email as Spam or Not Spam or determining if a tumor is malignant or benign. In particular, these are examples of *binary* classification where there are two possible outcomes. Outcomes can be described in pairs of 'positive'/'negative' such as 'yes'/'no, 'true'/'false' or '1'/'0'. \n",
"\n",
"Plots of classification data sets often use symbols to indicate the outcome of an example. In the plots below, 'X' is used to represent the positive values while 'O' represents negative outcomes. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ef30f6ed",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"x_train = np.array([0., 1, 2, 3, 4, 5])\n",
"y_train = np.array([0, 0, 0, 1, 1, 1])\n",
"X_train2 = np.array([[0.5, 1.5], [1,1], [1.5, 0.5], [3, 0.5], [2, 2], [1, 2.5]])\n",
"y_train2 = np.array([0, 0, 0, 1, 1, 1])"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "555a8709",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"pos = y_train == 1\n",
"neg = y_train == 0\n",
"\n",
"fig,ax = plt.subplots(1,2,figsize=(8,3))\n",
"#plot 1, single variable\n",
"ax[0].scatter(x_train[pos], y_train[pos], marker='x', s=80, c = 'red', label=\"y=1\")\n",
"ax[0].scatter(x_train[neg], y_train[neg], marker='o', s=100, label=\"y=0\", facecolors='none', edgecolors=dlc[\"dlblue\"],lw=3)\n",
"\n",
"ax[0].set_ylim(-0.08,1.1)\n",
"ax[0].set_ylabel('y', fontsize=12)\n",
"ax[0].set_xlabel('x', fontsize=12)\n",
"ax[0].set_title('one variable plot')\n",
"ax[0].legend()\n",
"\n",
"#plot 2, two variables\n",
"plot_data(X_train2, y_train2, ax[1])\n",
"ax[1].axis([0, 4, 0, 4])\n",
"ax[1].set_ylabel('$x_1$', fontsize=12)\n",
"ax[1].set_xlabel('$x_0$', fontsize=12)\n",
"ax[1].set_title('two variable plot')\n",
"ax[1].legend()\n",
"plt.tight_layout()\n",
"plt.show()\n"
]
},
{
"cell_type": "markdown",
"id": "4110c136",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Note in the plots above:\n",
"- In the single variable plot, positive results are shown both a red 'X's and as y=1. Negative results are blue 'O's and are located at y=0.\n",
" - Recall in the case of linear regression, y would not have been limited to two values but could have been any value.\n",
"- In the two-variable plot, the y axis is not available. Positive results are shown as red 'X's, while negative results use the blue 'O' symbol.\n",
" - Recall in the case of linear regression with multiple variables, y would not have been limited to two values and a similar plot would have been three-dimensional."
]
},
{
"cell_type": "markdown",
"id": "5590635c",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Linear Regression approach\n",
"In the previous week, you applied linear regression to build a prediction model. Let's try that approach here using the simple example that was described in the lecture. The model will predict if a tumor is benign or malignant based on tumor size. Try the following:\n",
"- Click on 'Run Linear Regression' to find the best linear regression model for the given data.\n",
" - Note the resulting linear model does **not** match the data well. \n",
"One option to improve the results is to apply a *threshold*. \n",
"- Tick the box on the 'Toggle 0.5 threshold' to show the predictions if a threshold is applied.\n",
" - These predictions look good, the predictions match the data\n",
"- *Important*: Now, add further 'malignant' data points on the far right, in the large tumor size range (near 10), and re-run linear regression.\n",
" - Now, the model predicts the larger tumor, but data point at x=3 is being incorrectly predicted!\n",
"- to clear/renew the plot, rerun the cell containing the plot command."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "de29056d",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"w_in = np.zeros((1))\n",
"b_in = 0\n",
"plt.close('all') \n",
"addpt = plt_one_addpt_onclick( x_train,y_train, w_in, b_in, logistic=False)"
]
},
{
"cell_type": "markdown",
"id": "4a7b4314",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"The example above demonstrates that the linear model is insufficient to model categorical data. The model can be extended as described in the following lab."
]
},
{
"cell_type": "markdown",
"id": "ab2364fc",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"In this lab you:\n",
"- explored categorical data sets and plotting\n",
"- determined that linear regression was insufficient for a classification problem."
]
},
{
"cell_type": "markdown",
"id": "9ffe5098",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"#### Optional Lab - 3.2: Logistic Regression\n",
"\n",
"In this ungraded lab, you will \n",
"- explore the sigmoid function (also known as the logistic function)\n",
"- explore logistic regression; which uses the sigmoid function"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7c92ccec",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"import numpy as np\n",
"%matplotlib widget\n",
"import matplotlib.pyplot as plt\n",
"from plt_one_addpt_onclick import plt_one_addpt_onclick\n",
"from lab_utils_common import draw_vthresh\n",
"plt.style.use('week3/OptionalLabs/deeplearning.mplstyle')"
]
},
{
"cell_type": "markdown",
"id": "39a36487",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Sigmoid or Logistic Function\n",
"As discussed in the lecture videos, for a classification task, we can start by using our linear regression model, $f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) = \\mathbf{w} \\cdot \\mathbf{x}^{(i)} + b$, to predict $y$ given $x$. \n",
"- However, we would like the predictions of our classification model to be between 0 and 1 since our output variable $y$ is either 0 or 1. \n",
"- This can be accomplished by using a \"sigmoid function\" which maps all input values to values between 0 and 1. \n",
"\n",
"\n",
"Let's implement the sigmoid function and see this for ourselves.\n",
"\n",
"##### Formula for Sigmoid function\n",
"\n",
"The formula for a sigmoid function is as follows - \n",
"\n",
"$g(z) = \\frac{1}{1+e^{-z}}\\tag{1}$\n",
"\n",
"In the case of logistic regression, z (the input to the sigmoid function), is the output of a linear regression model. \n",
"- In the case of a single example, $z$ is scalar.\n",
"- in the case of multiple examples, $z$ may be a vector consisting of $m$ values, one for each example. \n",
"- The implementation of the sigmoid function should cover both of these potential input formats.\n",
"Let's implement this in Python."
]
},
{
"cell_type": "markdown",
"id": "256fca46",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"NumPy has a function called [`exp()`](https://numpy.org/doc/stable/reference/generated/numpy.exp.html), which offers a convenient way to calculate the exponential ( $e^{z}$) of all elements in the input array (`z`).\n",
" \n",
"It also works with a single number as an input, as shown below."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fa442685",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"# Input is an array. \n",
"input_array = np.array([1,2,3])\n",
"exp_array = np.exp(input_array)\n",
"\n",
"print(\"Input to exp:\", input_array)\n",
"print(\"Output of exp:\", exp_array)\n",
"\n",
"# Input is a single number\n",
"input_val = 1 \n",
"exp_val = np.exp(input_val)\n",
"\n",
"print(\"Input to exp:\", input_val)\n",
"print(\"Output of exp:\", exp_val)"
]
},
{
"cell_type": "markdown",
"id": "e9596531",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"The `sigmoid` function is implemented in python as shown in the cell below."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dc319850",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"def sigmoid(z):\n",
" \"\"\"\n",
" Compute the sigmoid of z\n",
"\n",
" Args:\n",
" z (ndarray): A scalar, numpy array of any size.\n",
"\n",
" Returns:\n",
" g (ndarray): sigmoid(z), with the same shape as z\n",
" \n",
" \"\"\"\n",
"\n",
" g = 1/(1+np.exp(-z))\n",
" \n",
" return g"
]
},
{
"cell_type": "markdown",
"id": "14c4ba5e",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Let's see what the output of this function is for various value of `z`"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dac07d87",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"# Generate an array of evenly spaced values between -10 and 10\n",
"z_tmp = np.arange(-10,11)\n",
"\n",
"# Use the function implemented above to get the sigmoid values\n",
"y = sigmoid(z_tmp)\n",
"\n",
"# Code for pretty printing the two arrays next to each other\n",
"np.set_printoptions(precision=3) \n",
"print(\"Input (z), Output (sigmoid(z))\")\n",
"print(np.c_[z_tmp, y])"
]
},
{
"cell_type": "markdown",
"id": "5a2287bb",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"The values in the left column are `z`, and the values in the right column are `sigmoid(z)`. As you can see, the input values to the sigmoid range from -10 to 10, and the output values range from 0 to 1. \n",
"\n",
"Now, let's try to plot this function using the `matplotlib` library."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3e9e89ad",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"# Plot z vs sigmoid(z)\n",
"fig,ax = plt.subplots(1,1,figsize=(5,3))\n",
"ax.plot(z_tmp, y, c=\"b\")\n",
"\n",
"ax.set_title(\"Sigmoid function\")\n",
"ax.set_ylabel('sigmoid(z)')\n",
"ax.set_xlabel('z')\n",
"draw_vthresh(ax,0)"
]
},
{
"cell_type": "markdown",
"id": "1ee2fdc7",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"As you can see, the sigmoid function approaches `0` as `z` goes to large negative values and approaches `1` as `z` goes to large positive values.\n"
]
},
{
"cell_type": "markdown",
"id": "5b6be211",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Logistic Regression\n",
" A logistic regression model applies the sigmoid to the familiar linear regression model as shown below:\n",
"\n",
"$$ f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) = g(\\mathbf{w} \\cdot \\mathbf{x}^{(i)} + b ) \\tag{2} $$ \n",
"\n",
" where\n",
"\n",
" $g(z) = \\frac{1}{1+e^{-z}}\\tag{3}$\n"
]
},
{
"cell_type": "markdown",
"id": "59745ca1",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
" \n",
"Let's apply logistic regression to the categorical data example of tumor classification. \n",
"First, load the examples and initial values for the parameters.\n",
" \n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5db9b025",
"metadata": {
"pycharm": {
"name": "#%%\n"
},
"tags": []
},
"outputs": [],
"source": [
"x_train = np.array([0., 1, 2, 3, 4, 5])\n",
"y_train = np.array([0, 0, 0, 1, 1, 1])\n",
"\n",
"w_in = np.zeros((1))\n",
"b_in = 0"
]
},
{
"cell_type": "markdown",
"id": "076378af",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Try the following steps:\n",
"- Click on 'Run Logistic Regression' to find the best logistic regression model for the given training data\n",
" - Note the resulting model fits the data quite well.\n",
" - Note, the orange line is '$z$' or $\\mathbf{w} \\cdot \\mathbf{x}^{(i)} + b$ above. It does not match the line in a linear regression model.\n",
"Further improve these results by applying a *threshold*. \n",
"- Tick the box on the 'Toggle 0.5 threshold' to show the predictions if a threshold is applied.\n",
" - These predictions look good. The predictions match the data\n",
" - Now, add further data points in the large tumor size range (near 10), and re-run logistic regression.\n",
" - unlike the linear regression model, this model continues to make correct predictions"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c0f8764c",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"plt.close('all') \n",
"addpt = plt_one_addpt_onclick( x_train,y_train, w_in, b_in, logistic=True)"
]
},
{
"cell_type": "markdown",
"id": "60ac40e2",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"You have explored the use of the sigmoid function in logistic regression."
]
},
{
"cell_type": "markdown",
"id": "c164236e",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"#### Optional Lab - 3.3: Logistic Regression, Decision Boundary\n"
]
},
{
"cell_type": "markdown",
"id": "7f045e80",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Goals\n",
"In this lab, you will:\n",
"- Plot the decision boundary for a logistic regression model. This will give you a better sense of what the model is predicting.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f69c660a",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"import numpy as np\n",
"%matplotlib widget\n",
"import matplotlib.pyplot as plt\n",
"from lab_utils_common import plot_data, sigmoid, draw_vthresh\n",
"#plt.style.use('week3/OptionalLabs/deeplearning.mplstyle')"
]
},
{
"cell_type": "markdown",
"id": "1205b0c6",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Dataset\n",
"\n",
"Let's suppose you have following training dataset\n",
"- The input variable `X` is a numpy array which has 6 training examples, each with two features\n",
"- The output variable `y` is also a numpy array with 6 examples, and `y` is either `0` or `1`"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3642026f",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"X = np.array([[0.5, 1.5], [1,1], [1.5, 0.5], [3, 0.5], [2, 2], [1, 2.5]])\n",
"y = np.array([0, 0, 0, 1, 1, 1]).reshape(-1,1) "
]
},
{
"cell_type": "markdown",
"id": "8be4a974",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Plot data \n",
"\n",
"Let's use a helper function to plot this data. The data points with label $y=1$ are shown as red crosses, while the data points with label $y=0$ are shown as blue circles. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "199f1848",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"fig,ax = plt.subplots(1,1,figsize=(4,4))\n",
"plot_data(X, y, ax)\n",
"\n",
"ax.axis([0, 4, 0, 3.5])\n",
"ax.set_ylabel('$x_1$')\n",
"ax.set_xlabel('$x_0$')\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "74003f78",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Logistic regression model\n",
"\n",
"\n",
"* Suppose you'd like to train a logistic regression model on this data which has the form \n",
"\n",
" $f(x) = g(w_0x_0+w_1x_1 + b)$\n",
" \n",
" where $g(z) = \\frac{1}{1+e^{-z}}$, which is the sigmoid function\n",
"\n",
"\n",
"* Let's say that you trained the model and get the parameters as $b = -3, w_0 = 1, w_1 = 1$. That is,\n",
"\n",
" $f(x) = g(x_0+x_1-3)$\n",
"\n",
" (You'll learn how to fit these parameters to the data further in the course)\n",
" \n",
" \n",
"Let's try to understand what this trained model is predicting by plotting its decision boundary"
]
},
{
"cell_type": "markdown",
"id": "c287e48f",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Refresher on logistic regression and decision boundary\n",
"\n",
"* Recall that for logistic regression, the model is represented as \n",
"\n",
" $$f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) = g(\\mathbf{w} \\cdot \\mathbf{x}^{(i)} + b) \\tag{1}$$\n",
"\n",
" where $g(z)$ is known as the sigmoid function and it maps all input values to values between 0 and 1:\n",
"\n",
" $g(z) = \\frac{1}{1+e^{-z}}\\tag{2}$\n",
" and $\\mathbf{w} \\cdot \\mathbf{x}$ is the vector dot product:\n",
" \n",
" $$\\mathbf{w} \\cdot \\mathbf{x} = w_0 x_0 + w_1 x_1$$\n",
" \n",
" \n",
" * We interpret the output of the model ($f_{\\mathbf{w},b}(x)$) as the probability that $y=1$ given $\\mathbf{x}$ and parameterized by $\\mathbf{w}$ and $b$.\n",
"* Therefore, to get a final prediction ($y=0$ or $y=1$) from the logistic regression model, we can use the following heuristic -\n",
"\n",
" if $f_{\\mathbf{w},b}(x) >= 0.5$, predict $y=1$\n",
" \n",
" if $f_{\\mathbf{w},b}(x) < 0.5$, predict $y=0$\n",
" \n",
" \n",
"* Let's plot the sigmoid function to see where $g(z) >= 0.5$"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "06536178",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"# Plot sigmoid(z) over a range of values from -10 to 10\n",
"z = np.arange(-10,11)\n",
"\n",
"fig,ax = plt.subplots(1,1,figsize=(5,3))\n",
"# Plot z vs sigmoid(z)\n",
"ax.plot(z, sigmoid(z), c=\"b\")\n",
"\n",
"ax.set_title(\"Sigmoid function\")\n",
"ax.set_ylabel('sigmoid(z)')\n",
"ax.set_xlabel('z')\n",
"draw_vthresh(ax,0)"
]
},
{
"cell_type": "markdown",
"id": "31154efa",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"* As you can see, $g(z) >= 0.5$ for $z >=0$\n",
"\n",
"* For a logistic regression model, $z = \\mathbf{w} \\cdot \\mathbf{x} + b$. Therefore,\n",
"\n",
" if $\\mathbf{w} \\cdot \\mathbf{x} + b >= 0$, the model predicts $y=1$\n",
" \n",
" if $\\mathbf{w} \\cdot \\mathbf{x} + b < 0$, the model predicts $y=0$\n",
" \n",
" \n",
" \n",
"##### Plotting decision boundary\n",
"\n",
"Now, let's go back to our example to understand how the logistic regression model is making predictions.\n",
"\n",
"* Our logistic regression model has the form\n",
"\n",
" $f(\\mathbf{x}) = g(-3 + x_0+x_1)$\n",
"\n",
"\n",
"* From what you've learnt above, you can see that this model predicts $y=1$ if $-3 + x_0+x_1 >= 0$\n",
"\n",
"Let's see what this looks like graphically. We'll start by plotting $-3 + x_0+x_1 = 0$, which is equivalent to $x_1 = 3 - x_0$.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7575f8d5",
"metadata": {},
"outputs": [],
"source": [
"#plotting some decision boundry\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"x0 = np.arange(0,2.1,0.01)\n",
"\n",
"x1 = np.sqrt(4 - x0**2)\n",
"fig,ax = plt.subplots(1,1,figsize=(5,4))\n",
"# Plot the decision boundary\n",
"ax.plot(x0,x1, c=\"b\")\n",
"ax.axis([0, 4, 0, 4])\n",
"\n",
"# Fill the region below the line\n",
"ax.fill_between(x0,x1, alpha=0.2)\n",
"\n",
"# Plot the original data\n",
"ax.set_ylabel(r'$x_1$')\n",
"ax.set_xlabel(r'$x_0$')\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b33eeb88",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"# Choose values between 0 and 6\n",
"x0 = np.arange(0,6)\n",
"\n",
"x1 = 3 - x0\n",
"fig,ax = plt.subplots(1,1,figsize=(5,4))\n",
"# Plot the decision boundary\n",
"ax.plot(x0,x1, c=\"b\")\n",
"ax.axis([0, 4, 0, 3.5])\n",
"\n",
"# Fill the region below the line\n",
"ax.fill_between(x0,x1, alpha=0.2)\n",
"\n",
"# Plot the original data\n",
"plot_data(X,y,ax)\n",
"ax.set_ylabel(r'$x_1$')\n",
"ax.set_xlabel(r'$x_0$')\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "4b86da6b",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"* In the plot above, the blue line represents the line $x_0 + x_1 - 3 = 0$ and it should intersect the x1 axis at 3 (if we set $x_1$ = 3, $x_0$ = 0) and the x0 axis at 3 (if we set $x_1$ = 0, $x_0$ = 3). \n",
"\n",
"\n",
"* The shaded region represents $-3 + x_0+x_1 < 0$. The region above the line is $-3 + x_0+x_1 > 0$.\n",
"\n",
"\n",
"* Any point in the shaded region (under the line) is classified as $y=0$. Any point on or above the line is classified as $y=1$. This line is known as the \"decision boundary\".\n",
"\n",
"As we've seen in the lectures, by using higher order polynomial terms (eg: $f(x) = g( x_0^2 + x_1 -1)$, we can come up with more complex non-linear boundaries."
]
},
{
"cell_type": "markdown",
"id": "38168e17",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"You have explored the decision boundary in the context of logistic regression."
]
},
{
"cell_type": "markdown",
"id": "976d73e3",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"#### Optional Lab - 3.4: Logistic Regression, Logistic Loss\n",
"\n",
"In this ungraded lab, you will:\n",
"- explore the reason the squared error loss is not appropriate for logistic regression\n",
"- explore the logistic loss function"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bf9a0b64",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"id": "d02e05b0",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"import numpy as np\n",
"%matplotlib widget\n",
"import matplotlib.pyplot as plt\n",
"from plt_logistic_loss import plt_logistic_cost, plt_two_logistic_loss_curves, plt_simple_example\n",
"from plt_logistic_loss import soup_bowl, plt_logistic_squared_error\n",
"plt.style.use('week3/OptionalLabs/deeplearning.mplstyle')"
]
},
{
"cell_type": "markdown",
"id": "3be5d79c",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Squared error for logistic regression?\n",
" Recall for **Linear** Regression we have used the **squared error cost function**:\n",
"The equation for the squared error cost with one variable is:\n",
" $$J(w,b) = \\frac{1}{2m} \\sum\\limits_{i = 0}^{m-1} (f_{w,b}(x^{(i)}) - y^{(i)})^2 \\tag{1}$$ \n",
" \n",
"where \n",
" $$f_{w,b}(x^{(i)}) = wx^{(i)} + b \\tag{2}$$\n"
]
},
{
"cell_type": "markdown",
"id": "5f552ee0",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Recall, the squared error cost had the nice property that following the derivative of the cost leads to the minimum."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3f97dafd",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"soup_bowl()"
]
},
{
"cell_type": "markdown",
"id": "01265df7",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"This cost function worked well for linear regression, it is natural to consider it for logistic regression as well. However, as the slide above points out, $f_{wb}(x)$ now has a non-linear component, the sigmoid function: $f_{w,b}(x^{(i)}) = sigmoid(wx^{(i)} + b )$. Let's try a squared error cost on the example from an earlier lab, now including the sigmoid.\n",
"\n",
"Here is our training data:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d7a0079c",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"x_train = np.array([0., 1, 2, 3, 4, 5],dtype=np.longdouble)\n",
"y_train = np.array([0, 0, 0, 1, 1, 1],dtype=np.longdouble)\n",
"plt_simple_example(x_train, y_train)"
]
},
{
"cell_type": "markdown",
"id": "5ef9cf54",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Now, let's get a surface plot of the cost using a *squared error cost*:\n",
" $$J(w,b) = \\frac{1}{2m} \\sum\\limits_{i = 0}^{m-1} (f_{w,b}(x^{(i)}) - y^{(i)})^2 $$ \n",
" \n",
"where \n",
" $$f_{w,b}(x^{(i)}) = sigmoid(wx^{(i)} + b )$$\n"
]
},
{
"cell_type": "markdown",
"id": "5b9b2478",
"metadata": {},
"source": [
"###### Plot logistic squared error "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3c78eb3c",
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"from matplotlib import cm\n",
"x_train = np.array([0., 1, 2, 3, 4, 5],dtype=np.longdouble)\n",
"y_train = np.array([0, 0, 0, 1, 1, 1],dtype=np.longdouble)\n",
"\n",
"wx,by=np.meshgrid(np.linspace(-6,12,100),np.linspace(10,-20,100))\n",
"\n",
"\n",
"def logistic_model(x,w,b):\n",
" return 1/(1+np.exp(-(w*x+b)))\n",
"\n",
"def cost_fn_logistic(x,w,b,y):\n",
" return np.sum((logistic_model(x,w,b)-y)**2)/2/len(x)\n",
"\n",
"\n",
"cost_f=np.zeros(wx.shape)\n",
"for wi in range(wx.shape[0]):\n",
" for wj in range(wx.shape[1]):\n",
" w,b=wx[wi,wj],by[wi,wj]\n",
" #print(cost_fn_logistic(x_train,w,b,y_train))\n",
" cost_f[wi,wj]=cost_fn_logistic(x_train,w,b,y_train)\n",
" \n",
" \n",
"fig = plt.figure()\n",
"fig.canvas.toolbar_visible = False\n",
"fig.canvas.header_visible = False\n",
"fig.canvas.footer_visible = False\n",
"ax = fig.add_subplot(1, 1, 1, projection='3d')\n",
"ax.plot_surface(wx, by, cost_f, alpha=0.6,cmap=cm.coolwarm)\n",
" \n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "10726c44",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"plt.close('all')\n",
"plt_logistic_squared_error(x_train,y_train)\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "a8165594",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"While this produces a pretty interesting plot, the surface above not nearly as smooth as the 'soup bowl' from linear regression! \n",
"\n",
"Logistic regression requires a cost function more suitable to its non-linear nature. This starts with a Loss function. This is described below."
]
},
{
"cell_type": "markdown",
"id": "c6b0b89a",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Logistic Loss Function\n",
"\n",
"\n",
" "
]
},
{
"cell_type": "markdown",
"id": "5f552d09",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Logistic Regression uses a loss function more suited to the task of categorization where the target is 0 or 1 rather than any number. \n",
"\n",
">**Definition Note:** In this course, these definitions are used: \n",
"**Loss** is a measure of the difference of a single example to its target value while the \n",
"**Cost** is a measure of the losses over the training set\n",
"\n",
"\n",
"This is defined: \n",
"* $loss(f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}), y^{(i)})$ is the cost for a single data point, which is:\n",
"\n",
"\\begin{equation}\n",
" loss(f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}), y^{(i)}) = \\begin{cases}\n",
" - \\log\\left(f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right) & \\text{if $y^{(i)}=1$}\\\\\n",
" - \\log \\left( 1 - f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right) & \\text{if $y^{(i)}=0$}\n",
" \\end{cases}\n",
"\\end{equation}\n",
"\n",
"\n",
"* $f_{\\mathbf{w},b}(\\mathbf{x}^{(i)})$ is the model's prediction, while $y^{(i)}$ is the target value.\n",
"\n",
"* $f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) = g(\\mathbf{w} \\cdot\\mathbf{x}^{(i)}+b)$ where function $g$ is the sigmoid function.\n",
"\n",
"The defining feature of this loss function is the fact that it uses two separate curves. One for the case when the target is zero or ($y=0$) and another for when the target is one ($y=1$). Combined, these curves provide the behavior useful for a loss function, namely, being zero when the prediction matches the target and rapidly increasing in value as the prediction differs from the target. Consider the curves below:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "edb72317",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"plt_two_logistic_loss_curves()"
]
},
{
"cell_type": "markdown",
"id": "277f5476",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Combined, the curves are similar to the quadratic curve of the squared error loss. Note, the x-axis is $f_{\\mathbf{w},b}$ which is the output of a sigmoid. The sigmoid output is strictly between 0 and 1."
]
},
{
"cell_type": "markdown",
"id": "f5c96b6e",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"The loss function above can be rewritten to be easier to implement.\n",
" $$loss(f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}), y^{(i)}) = (-y^{(i)} \\log\\left(f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right) - \\left( 1 - y^{(i)}\\right) \\log \\left( 1 - f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right)$$\n",
" \n",
"This is a rather formidable-looking equation. It is less daunting when you consider $y^{(i)}$ can have only two values, 0 and 1. One can then consider the equation in two pieces: \n",
"when $ y^{(i)} = 0$, the left-hand term is eliminated:\n",
"$$\n",
"\\begin{align}\n",
"loss(f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}), 0) &= (-(0) \\log\\left(f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right) - \\left( 1 - 0\\right) \\log \\left( 1 - f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right) \\\\\n",
"&= -\\log \\left( 1 - f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right)\n",
"\\end{align}\n",
"$$\n",
"and when $ y^{(i)} = 1$, the right-hand term is eliminated:\n",
"$$\n",
"\\begin{align}\n",
" loss(f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}), 1) &= (-(1) \\log\\left(f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right) - \\left( 1 - 1\\right) \\log \\left( 1 - f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right)\\\\\n",
" &= -\\log\\left(f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right)\n",
"\\end{align}\n",
"$$\n",
"\n",
"OK, with this new logistic loss function, a cost function can be produced that incorporates the loss from all the examples. This will be the topic of the next lab. For now, let's take a look at the cost vs parameters curve for the simple example we considered above:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4022f38d",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"plt.close('all')\n",
"cst = plt_logistic_cost(x_train,y_train)"
]
},
{
"cell_type": "markdown",
"id": "beb51ef1",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"This curve is well suited to gradient descent! It does not have plateaus, local minima, or discontinuities. Note, it is not a bowl as in the case of squared error. Both the cost and the log of the cost are plotted to illuminate the fact that the curve, when the cost is small, has a slope and continues to decline. Reminder: you can rotate the above plots using your mouse."
]
},
{
"cell_type": "markdown",
"id": "5b367649",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"You have:\n",
" - determined a squared error loss function is not suitable for classification tasks\n",
" - developed and examined the logistic loss function which **is** suitable for classification tasks.\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "5d717b07",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"#### Optional Lab - 3.5: Cost Function for Logistic Regression\n",
"\n",
"##### Goals\n",
"In this lab, you will:\n",
"- examine the implementation and utilize the cost function for logistic regression."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4a4fdc5e",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"import numpy as np\n",
"%matplotlib widget\n",
"import matplotlib.pyplot as plt\n",
"from lab_utils_common import plot_data, sigmoid, dlc\n",
"plt.style.use('week3/OptionalLabs/deeplearning.mplstyle')"
]
},
{
"cell_type": "markdown",
"id": "c075181d",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"###### Dataset \n",
"Let's start with the same dataset as was used in the decision boundary lab."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6ef2e3dc",
"metadata": {
"pycharm": {
"name": "#%%\n"
},
"tags": []
},
"outputs": [],
"source": [
"X_train = np.array([[0.5, 1.5], [1,1], [1.5, 0.5], [3, 0.5], [2, 2], [1, 2.5]]) #(m,n)\n",
"y_train = np.array([0, 0, 0, 1, 1, 1]) #(m,)"
]
},
{
"cell_type": "markdown",
"id": "d3d3cba7",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"We will use a helper function to plot this data. The data points with label $y=1$ are shown as red crosses, while the data points with label $y=0$ are shown as blue circles."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bf605c65",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"fig,ax = plt.subplots(1,1,figsize=(4,4))\n",
"plot_data(X_train, y_train, ax)\n",
"\n",
"# Set both axes to be from 0-4\n",
"ax.axis([0, 4, 0, 3.5])\n",
"ax.set_ylabel('$x_1$', fontsize=12)\n",
"ax.set_xlabel('$x_0$', fontsize=12)\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "f22e896e",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Cost function\n",
"\n",
"In a previous lab, you developed the *logistic loss* function. Recall, loss is defined to apply to one example. Here you combine the losses to form the **cost**, which includes all the examples.\n",
"\n",
"\n",
"Recall that for logistic regression, the cost function is of the form \n",
"\n",
"$$ J(\\mathbf{w},b) = \\frac{1}{m} \\sum_{i=0}^{m-1} \\left[ loss(f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}), y^{(i)}) \\right] \\tag{1}$$\n",
"\n",
"where\n",
"* $loss(f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}), y^{(i)})$ is the cost for a single data point, which is:\n",
"\n",
" $$loss(f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}), y^{(i)}) = -y^{(i)} \\log\\left(f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right) - \\left( 1 - y^{(i)}\\right) \\log \\left( 1 - f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right) \\tag{2}$$\n",
" \n",
"* where m is the number of training examples in the data set and:\n",
"$$\n",
"\\begin{align}\n",
" f_{\\mathbf{w},b}(\\mathbf{x^{(i)}}) &= g(z^{(i)})\\tag{3} \\\\\n",
" z^{(i)} &= \\mathbf{w} \\cdot \\mathbf{x}^{(i)}+ b\\tag{4} \\\\\n",
" g(z^{(i)}) &= \\frac{1}{1+e^{-z^{(i)}}}\\tag{5} \n",
"\\end{align}\n",
"$$\n",
" "
]
},
{
"cell_type": "markdown",
"id": "6c505662",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Code Description\n",
"\n",
"\n",
"\n",
"The algorithm for `compute_cost_logistic` loops over all the examples calculating the loss for each example and accumulating the total.\n",
"\n",
"Note that the variables X and y are not scalar values but matrices of shape ($m, n$) and ($𝑚$,) respectively, where $𝑛$ is the number of features and $𝑚$ is the number of training examples.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "22a5ddbe",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"def compute_cost_logistic(X, y, w, b):\n",
" \"\"\"\n",
" Computes cost\n",
"\n",
" Args:\n",
" X (ndarray (m,n)): Data, m examples with n features\n",
" y (ndarray (m,)) : target values\n",
" w (ndarray (n,)) : model parameters \n",
" b (scalar) : model parameter\n",
" \n",
" Returns:\n",
" cost (scalar): cost\n",
" \"\"\"\n",
"\n",
" m = X.shape[0]\n",
" cost = 0.0\n",
" for i in range(m):\n",
" z_i = np.dot(X[i],w) + b\n",
" f_wb_i = sigmoid(z_i)\n",
" cost += -y[i]*np.log(f_wb_i) - (1-y[i])*np.log(1-f_wb_i)\n",
" \n",
" cost = cost / m\n",
" return cost\n"
]
},
{
"cell_type": "markdown",
"id": "180dec69",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Check the implementation of the cost function using the cell below."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f2bb9ca6",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"w_tmp = np.array([1,1])\n",
"b_tmp = -3\n",
"print(compute_cost_logistic(X_train, y_train, w_tmp, b_tmp))"
]
},
{
"cell_type": "markdown",
"id": "0d1f4ef7",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"**Expected output**: 0.3668667864055175"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7315c0e3",
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"from matplotlib import cm\n",
"\n",
"wx,by=np.meshgrid(np.linspace(-6,12,100),np.linspace(10,-20,100))\n",
"\n",
"\n",
"def logistic_model(x,w,b):\n",
" return np.array([ (1/(1+np.exp(-(np.dot(w,i)+b)))) for i in x])\n",
" \n",
"def cost_fn_logistic(x,w,b,y):\n",
" return np.sum(-y*np.log(logistic_model(x,w,b))-(1-y)*np.log(1-logistic_model(x,w,b)))/len(x)\n",
"cost_fn_logistic(X_train,np.array([1,1]),-4,y_train)\n"
]
},
{
"cell_type": "markdown",
"id": "ac31da9f",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Example\n",
"Now, let's see what the cost function output is for a different value of $w$. \n",
"\n",
"* In a previous lab, you plotted the decision boundary for $b = -3, w_0 = 1, w_1 = 1$. That is, you had `b = -3, w = np.array([1,1])`.\n",
"\n",
"* Let's say you want to see if $b = -4, w_0 = 1, w_1 = 1$, or `b = -4, w = np.array([1,1])` provides a better model.\n",
"\n",
"Let's first plot the decision boundary for these two different $b$ values to see which one fits the data better.\n",
"\n",
"* For $b = -3, w_0 = 1, w_1 = 1$, we'll plot $-3 + x_0+x_1 = 0$ (shown in blue)\n",
"* For $b = -4, w_0 = 1, w_1 = 1$, we'll plot $-4 + x_0+x_1 = 0$ (shown in magenta)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dfff9371",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"\n",
"# Choose values between 0 and 6\n",
"x0 = np.arange(0,6)\n",
"\n",
"# Plot the two decision boundaries\n",
"x1 = 3 - x0\n",
"x1_other = 4 - x0\n",
"\n",
"fig,ax = plt.subplots(1, 1, figsize=(4,4))\n",
"# Plot the decision boundary\n",
"ax.plot(x0,x1, c=dlc[\"dlblue\"], label=\"$b$=-3\")\n",
"ax.plot(x0,x1_other, c=dlc[\"dlmagenta\"], label=\"$b$=-4\")\n",
"ax.axis([0, 4, 0, 4])\n",
"\n",
"# Plot the original data\n",
"plot_data(X_train,y_train,ax)\n",
"ax.axis([0, 4, 0, 4])\n",
"ax.set_ylabel('$x_1$', fontsize=12)\n",
"ax.set_xlabel('$x_0$', fontsize=12)\n",
"plt.legend(loc=\"upper right\")\n",
"plt.title(\"Decision Boundary\")\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "c30b9c56",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"You can see from this plot that `b = -4, w = np.array([1,1])` is a worse model for the training data. Let's see if the cost function implementation reflects this."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4d2a4e61",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"w_array1 = np.array([1,1])\n",
"b_1 = -3\n",
"w_array2 = np.array([1,1])\n",
"b_2 = -4\n",
"\n",
"print(\"Cost for b = -3 : \", compute_cost_logistic(X_train, y_train, w_array1, b_1))\n",
"print(\"Cost for b = -4 : \", compute_cost_logistic(X_train, y_train, w_array2, b_2))"
]
},
{
"cell_type": "markdown",
"id": "68a25495",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"**Expected output**\n",
"\n",
"Cost for b = -3 : 0.3668667864055175\n",
"\n",
"Cost for b = -4 : 0.5036808636748461\n",
"\n",
"\n",
"You can see the cost function behaves as expected and the cost for `b = -4, w = np.array([1,1])` is indeed higher than the cost for `b = -3, w = np.array([1,1])`"
]
},
{
"cell_type": "markdown",
"id": "8fb5b5be",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"In this lab you examined and utilized the cost function for logistic regression."
]
},
{
"cell_type": "markdown",
"id": "6beb4eb1",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"#### Optional Lab - 3.6: Gradient Descent for Logistic Regression"
]
},
{
"cell_type": "markdown",
"id": "984999b1",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Goals\n",
"In this lab, you will:\n",
"- update gradient descent for logistic regression.\n",
"- explore gradient descent on a familiar data set"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0247ebcd",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"import copy, math\n",
"import numpy as np\n",
"%matplotlib widget\n",
"import matplotlib.pyplot as plt\n",
"from lab_utils_common import dlc, plot_data, plt_tumor_data, sigmoid, compute_cost_logistic\n",
"from plt_quad_logistic import plt_quad_logistic, plt_prob\n",
"plt.style.use('week3/OptionalLabs/deeplearning.mplstyle')"
]
},
{
"cell_type": "markdown",
"id": "7a1e36bb",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"###### Data set \n",
"Let's start with the same two feature data set used in the decision boundary lab."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4fc1fdc5",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"X_train = np.array([[0.5, 1.5], [1,1], [1.5, 0.5], [3, 0.5], [2, 2], [1, 2.5]])\n",
"y_train = np.array([0, 0, 0, 1, 1, 1])"
]
},
{
"cell_type": "markdown",
"id": "8df6592b",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"As before, we'll use a helper function to plot this data. The data points with label $y=1$ are shown as red crosses, while the data points with label $y=0$ are shown as blue circles."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bd3a0530",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"fig,ax = plt.subplots(1,1,figsize=(4,4))\n",
"plot_data(X_train, y_train, ax)\n",
"\n",
"ax.axis([0, 4, 0, 3.5])\n",
"ax.set_ylabel('$x_1$', fontsize=12)\n",
"ax.set_xlabel('$x_0$', fontsize=12)\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "f893d122",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Logistic Gradient Descent\n",
"\n",
"\n",
"Recall the gradient descent algorithm utilizes the gradient calculation:\n",
"$$\\begin{align*}\n",
"&\\text{repeat until convergence:} \\; \\lbrace \\\\\n",
"& \\; \\; \\;w_j = w_j - \\alpha \\frac{\\partial J(\\mathbf{w},b)}{\\partial w_j} \\tag{1} \\; & \\text{for j := 0..n-1} \\\\ \n",
"& \\; \\; \\; \\; \\;b = b - \\alpha \\frac{\\partial J(\\mathbf{w},b)}{\\partial b} \\\\\n",
"&\\rbrace\n",
"\\end{align*}$$\n",
"\n",
"Where each iteration performs simultaneous updates on $w_j$ for all $j$, where\n",
"$$\\begin{align*}\n",
"\\frac{\\partial J(\\mathbf{w},b)}{\\partial w_j} &= \\frac{1}{m} \\sum\\limits_{i = 0}^{m-1} (f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) - y^{(i)})x_{j}^{(i)} \\tag{2} \\\\\n",
"\\frac{\\partial J(\\mathbf{w},b)}{\\partial b} &= \\frac{1}{m} \\sum\\limits_{i = 0}^{m-1} (f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) - y^{(i)}) \\tag{3} \n",
"\\end{align*}$$\n",
"\n",
"* m is the number of training examples in the data set \n",
"* $f_{\\mathbf{w},b}(x^{(i)})$ is the model's prediction, while $y^{(i)}$ is the target\n",
"* For a logistic regression model \n",
" $z = \\mathbf{w} \\cdot \\mathbf{x} + b$ \n",
" $f_{\\mathbf{w},b}(x) = g(z)$ \n",
" where $g(z)$ is the sigmoid function: \n",
" $g(z) = \\frac{1}{1+e^{-z}}$ \n",
" \n"
]
},
{
"cell_type": "markdown",
"id": "8dad45d1",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Gradient Descent Implementation\n",
"The gradient descent algorithm implementation has two components: \n",
"- The loop implementing equation (1) above. This is `gradient_descent` below and is generally provided to you in optional and practice labs.\n",
"- The calculation of the current gradient, equations (2,3) above. This is `compute_gradient_logistic` below. You will be asked to implement this week's practice lab.\n",
"\n",
"###### Calculating the Gradient, Code Description\n",
"Implements equation (2),(3) above for all $w_j$ and $b$.\n",
"There are many ways to implement this. Outlined below is this:\n",
"- initialize variables to accumulate `dj_dw` and `dj_db`\n",
"- for each example\n",
" - calculate the error for that example $g(\\mathbf{w} \\cdot \\mathbf{x}^{(i)} + b) - \\mathbf{y}^{(i)}$\n",
" - for each input value $x_{j}^{(i)}$ in this example, \n",
" - multiply the error by the input $x_{j}^{(i)}$, and add to the corresponding element of `dj_dw`. (equation 2 above)\n",
" - add the error to `dj_db` (equation 3 above)\n",
"\n",
"- divide `dj_db` and `dj_dw` by total number of examples (m)\n",
"- note that $\\mathbf{x}^{(i)}$ in numpy `X[i,:]` or `X[i]` and $x_{j}^{(i)}$ is `X[i,j]`"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ded2e5ce",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"id": "2e692682",
"metadata": {},
"source": [
"#### My solution\n",
"\n",
"\n",
" Logistic regression(1 variable) \n",
" \n",
" ```python\n",
"\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"from matplotlib import cm\n",
"x_train = np.array([0., 1, 2, 3, 4, 5],dtype=np.longdouble)\n",
"y_train = np.array([0, 0, 0, 1, 1, 1],dtype=np.longdouble)\n",
"\n",
"#wx,by=np.meshgrid(np.linspace(-6,12,100),np.linspace(10,-20,100))\n",
"\n",
"def model(x,theta):\n",
" w,b=theta\n",
" sigmoid=np.zeros(len(x))\n",
" for i in range(len(x)):\n",
" if np.isscalar(w):\n",
" w=np.array(w)\n",
" if w.shape!=x[i].shape:\n",
" print(\"Shape of W and X dosn't match\")\n",
" sys.exit() \n",
" sigmoid[i]=1/(1+np.exp(-(np.dot(w,x[i])+b)))\n",
" return sigmoid\n",
"\n",
"def dmodel_w(x,theta): \n",
" w,b=theta\n",
" return x\n",
"\n",
"def dmodel_b(x,theta): \n",
" w,b=theta\n",
" return 1.\n",
"\n",
"def cost(x,theta,y):\n",
" w,b=theta\n",
" cf= -y*np.log(model(x,theta))-(1-y)*np.log(1-model(x,theta))\n",
" return np.sum(cf)/np.shape(x)[0]\n",
"\n",
"def dcost_w(x,theta,y):\n",
" return np.sum((model(x,theta)-y)*dmodel_w(x,theta))/len(x)\n",
"\n",
"def dcost_b(x,theta,y):\n",
" return np.sum((model(x,theta)-y)*dmodel_b(x,theta))/len(x)\n",
" \n",
"def compute_gradient(x,theta,y):\n",
" return dcost_w(x,theta,y),dcost_b(x,theta,y)\n",
"\n",
"np.set_printoptions(precision=2)\n",
"def gradient_decent(x,y,theta,alpha,niter):\n",
" w,b=theta\n",
" if theta[1]>0: #constraining parameters\n",
" b=-theta[1]\n",
" cost_i=np.zeros(niter)\n",
" for i in np.arange(niter):\n",
" if i>1:\n",
" if np.abs((cost_i[i]-cost_i[i-1])/cost_i[i])<0.05:\n",
" alpha/=2\n",
" \n",
" dcw,dcb= compute_gradient(x,theta,y)\n",
" w = w-alpha*dcw\n",
" b = b-alpha*dcb\n",
" theta=w,b\n",
" cost_i[i]=cost(x,theta,y)\n",
" if i>1:\n",
" if cost_i[i]>cost_i[i-1]:\n",
" alpha/=2\n",
" #print(cost_i[i],alpha)\n",
" #print(theta) \n",
" return cost_i,theta\n",
"\n",
" \n",
" \n",
"niter=1000\n",
"Win=20\n",
"Bin=5\n",
"alpha=0.5\n",
"theta_in=Win,Bin\n",
"grad_dec_result,theta_f=gradient_decent(x_train,y_train,theta_in,alpha,niter) \n",
"\n",
"wf,bf=theta_f\n",
"print(wf,bf,grad_dec_result[-1])\n",
"\n",
"\n",
"\n",
"plt.figure(figsize=(8,4))\n",
"ax=plt.subplot(121)\n",
"plt.plot(np.arange(niter),grad_dec_result,\".\")\n",
"plt.yscale(\"log\")\n",
"plt.xlabel(\"No of steps\")\n",
"plt.ylabel(\"Cost function\")\n",
"plt.ylim(bottom=0.01)\n",
"\n",
"\n",
"\n",
"ax=plt.subplot(1,2,2) \n",
"plt.plot(x_train, model(x_train,theta_f), c = \"g\",label=\"Predcited model\")\n",
"plt.scatter(x_train, y_train, marker='x', c='r') \n",
"# Set the title\n",
"plt.title(\"Model fit\")\n",
"# Set the y-axis label\n",
"plt.ylabel('training data')\n",
"# Set the x-axis label\n",
"plt.xlabel('training input') \n",
"plt.legend()\n",
"plt.tight_layout()\n",
"\n",
" \n",
" ```\n",
"\n",
"\n",
"\n",
"\n",
"\n",
" Logistic regression(2 variables) \n",
"\n",
" ```python\n",
"\n",
"import numpy as np,sys\n",
"import matplotlib.pyplot as plt\n",
"from matplotlib import cm\n",
"x_train = np.array([[0.5, 1.5], [1,1], [1.5, 0.5], [3, 0.5], [2, 2],[0.5,0.5],[2.7,1.5], [1, 2.5]])\n",
"y_train = np.array([1, 1, 1, 1, 1,0,1, 1],dtype=np.longdouble)\n",
"\n",
"\n",
"\n",
"#wx,by=np.meshgrid(np.linspace(-6,12,100),np.linspace(10,-20,100))\n",
"\n",
"def model(x,theta):\n",
" w,b=theta\n",
" if np.isscalar(x):\n",
" x=np.array(x)\n",
" if np.isscalar(w):\n",
" w=np.array([w])\n",
" elif isinstance(w,tuple):\n",
" w=np.array(w)\n",
" sigmoid=np.zeros(len(x))\n",
" for i in range(len(x)):\n",
" if w.shape!=x[i].shape:\n",
" print(\"Shape of W and X dosn't match\", w.shape,x[i].shape)\n",
" sys.exit() \n",
" sigmoid[i]=1/(1+np.exp(-(np.dot(w,x[i])+b)))\n",
" return sigmoid\n",
"\n",
"def dmodel_w(x,theta):\n",
" w,b=theta\n",
" return x\n",
"\n",
"def dmodel_b(x,theta): \n",
" w,b=theta\n",
" return 1.\n",
"\n",
"def cost(x,theta,y):\n",
" w,b=theta\n",
" cf= -y*np.log(model(x,theta))-(1-y)*np.log(1-model(x,theta))\n",
" return np.sum(cf)/np.shape(x)[0]\n",
"\n",
"def dcost_w(x,theta,y):\n",
" w,b=theta\n",
" if np.isscalar(w):\n",
" w=np.array([w])\n",
" elif isinstance(w,tuple):\n",
" w=np.array(w)\n",
" dcost_w_result=np.zeros(w.shape) \n",
" for wi in range(len(w)):\n",
" dcost_w_result[wi]=np.sum((model(x,theta)-y)*dmodel_w(x,theta)[:,wi])/len(x) \n",
" return dcost_w_result \n",
"\n",
"def dcost_b(x,theta,y):\n",
" return np.sum((model(x,theta)-y)*dmodel_b(x,theta))/len(x)\n",
" \n",
"def compute_gradient(x,theta,y):\n",
" return dcost_w(x,theta,y),dcost_b(x,theta,y)\n",
"\n",
"np.set_printoptions(precision=2)\n",
"\n",
"def gradient_decent(x,y,theta,alpha,niter):\n",
" w,b=theta\n",
" if np.isscalar(w):\n",
" w=np.array(w)\n",
" elif isinstance(w, tuple):\n",
" w=np.array(w)\n",
"\n",
" if theta[1]>0: #constraining parameters\n",
" b=-theta[1]\n",
" cost_i=np.zeros(niter)\n",
" for i in np.arange(niter):\n",
" if i>1:\n",
" if np.abs((cost_i[i]-cost_i[i-1])/cost_i[i])<0.05:\n",
" alpha/=2\n",
" dcw,dcb= compute_gradient(x,theta,y)\n",
" \n",
" w = w-alpha*dcw\n",
" b = b-alpha*dcb\n",
" theta=w,b\n",
" cost_i[i]=cost(x,theta,y)\n",
" if i>1:\n",
" if cost_i[i]>cost_i[i-1]:\n",
" alpha/=2\n",
" #print(cost_i[i],alpha)\n",
" #print(theta) \n",
" return cost_i,theta\n",
"\n",
" \n",
" \n",
"niter=10000\n",
"Win=np.array([2.,3.])\n",
"Bin=1.\n",
"\n",
"alpha=0.5\n",
"theta_in=Win,Bin\n",
"grad_dec_result,theta_f=gradient_decent(x_train,y_train,theta_in,alpha,niter) \n",
"\n",
"wf,bf=theta_f\n",
"print(wf,bf,grad_dec_result[-1])\n",
"\n",
"\n",
"\n",
"plt.figure(figsize=(8,4))\n",
"ax=plt.subplot(121)\n",
"plt.plot(np.arange(niter),grad_dec_result,\".\")\n",
"plt.yscale(\"log\")\n",
"plt.xlabel(\"No of steps\")\n",
"plt.ylabel(\"Cost function\")\n",
"plt.ylim(bottom=0.01)\n",
"\n",
"\n",
"ax=plt.subplot(1,2,2) \n",
"#plt.plot(x_train, model(x_train,theta_f), c = \"g\",label=\"Predcited model\")\n",
"ax.plot((-bf/wf[0],0),(0,-bf/wf[1]),label=\"Predicted model\")\n",
"pos=y_train>0.5\n",
"neg=y_train<0.5\n",
"plt.scatter(x_train[:,0][pos],x_train[:,1][pos] , marker='x', c='r') \n",
"plt.scatter(x_train[:,0][neg],x_train[:,1][neg] , marker='o', c='b') \n",
"ax.set_ylabel(r'$x_1$')\n",
"ax.set_xlabel(r'$x_0$') \n",
"ax.axis([0, 4, 0, 3.5])\n",
"# Set the title\n",
"plt.title(\"Model fit\")\n",
"# Set the y-axis label\n",
"plt.ylabel('training data')\n",
"# Set the x-axis label\n",
"plt.xlabel('training input') \n",
"plt.legend()\n",
"plt.tight_layout()\n",
"\n",
"\n",
"x_train, model(x_train,theta_f),y_train\n",
"\n",
" ```\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fe38d2d9",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"def compute_gradient_logistic(X, y, w, b): \n",
" \"\"\"\n",
" Computes the gradient for linear regression \n",
" \n",
" Args:\n",
" X (ndarray (m,n): Data, m examples with n features\n",
" y (ndarray (m,)): target values\n",
" w (ndarray (n,)): model parameters \n",
" b (scalar) : model parameter\n",
" Returns\n",
" dj_dw (ndarray (n,)): The gradient of the cost w.r.t. the parameters w. \n",
" dj_db (scalar) : The gradient of the cost w.r.t. the parameter b. \n",
" \"\"\"\n",
" m,n = X.shape\n",
" dj_dw = np.zeros((n,)) #(n,)\n",
" dj_db = 0.\n",
"\n",
" for i in range(m):\n",
" f_wb_i = sigmoid(np.dot(X[i],w) + b) #(n,)(n,)=scalar\n",
" err_i = f_wb_i - y[i] #scalar\n",
" for j in range(n):\n",
" dj_dw[j] = dj_dw[j] + err_i * X[i,j] #scalar\n",
" dj_db = dj_db + err_i\n",
" dj_dw = dj_dw/m #(n,)\n",
" dj_db = dj_db/m #scalar\n",
" \n",
" return dj_db, dj_dw"
]
},
{
"cell_type": "markdown",
"id": "5ed1faef",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Check the implementation of the gradient function using the cell below."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ddd41a33",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"X_tmp = np.array([[0.5, 1.5], [1,1], [1.5, 0.5], [3, 0.5], [2, 2], [1, 2.5]])\n",
"y_tmp = np.array([0, 0, 0, 1, 1, 1])\n",
"w_tmp = np.array([2.,3.])\n",
"b_tmp = 1.\n",
"dj_db_tmp, dj_dw_tmp = compute_gradient_logistic(X_tmp, y_tmp, w_tmp, b_tmp)\n",
"print(f\"dj_db: {dj_db_tmp}\" )\n",
"print(f\"dj_dw: {dj_dw_tmp.tolist()}\" )"
]
},
{
"cell_type": "markdown",
"id": "745399c8",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"**Expected output**\n",
"``` \n",
"dj_db: 0.49861806546328574\n",
"dj_dw: [0.498333393278696, 0.49883942983996693]\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "c4cfd3fc",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Gradient Descent Code \n",
"The code implementing equation (1) above is implemented below. Take a moment to locate and compare the functions in the routine to the equations above."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "77f30888",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"def gradient_descent(X, y, w_in, b_in, alpha, num_iters): \n",
" \"\"\"\n",
" Performs batch gradient descent\n",
" \n",
" Args:\n",
" X (ndarray (m,n) : Data, m examples with n features\n",
" y (ndarray (m,)) : target values\n",
" w_in (ndarray (n,)): Initial values of model parameters \n",
" b_in (scalar) : Initial values of model parameter\n",
" alpha (float) : Learning rate\n",
" num_iters (scalar) : number of iterations to run gradient descent\n",
" \n",
" Returns:\n",
" w (ndarray (n,)) : Updated values of parameters\n",
" b (scalar) : Updated value of parameter \n",
" \"\"\"\n",
" # An array to store cost J and w's at each iteration primarily for graphing later\n",
" J_history = []\n",
" w = copy.deepcopy(w_in) #avoid modifying global w within function\n",
" b = b_in\n",
" \n",
" for i in range(num_iters):\n",
" # Calculate the gradient and update the parameters\n",
" dj_db, dj_dw = compute_gradient_logistic(X, y, w, b) \n",
"\n",
" # Update Parameters using w, b, alpha and gradient\n",
" w = w - alpha * dj_dw \n",
" b = b - alpha * dj_db \n",
" \n",
" # Save cost J at each iteration\n",
" if i<100000: # prevent resource exhaustion \n",
" J_history.append( compute_cost_logistic(X, y, w, b) )\n",
"\n",
" # Print cost every at intervals 10 times or as many iterations if < 10\n",
" if i% math.ceil(num_iters / 10) == 0:\n",
" print(f\"Iteration {i:4d}: Cost {J_history[-1]} \")\n",
" \n",
" return w, b, J_history #return final w,b and J history for graphing\n"
]
},
{
"cell_type": "markdown",
"id": "720c6fdd",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Let's run gradient descent on our data set."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ed3a1ce9",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"w_tmp = np.zeros_like(X_train[0])\n",
"b_tmp = 0.\n",
"alph = 0.1\n",
"iters = 10000\n",
"\n",
"w_out, b_out, _ = gradient_descent(X_train, y_train, w_tmp, b_tmp, alph, iters) \n",
"print(f\"\\nupdated parameters: w:{w_out}, b:{b_out}\")"
]
},
{
"cell_type": "markdown",
"id": "3acea32f",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Let's plot the results of gradient descent:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "454c6748",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"fig,ax = plt.subplots(1,1,figsize=(5,4))\n",
"# plot the probability \n",
"plt_prob(ax, w_out, b_out)\n",
"\n",
"# Plot the original data\n",
"ax.set_ylabel(r'$x_1$')\n",
"ax.set_xlabel(r'$x_0$') \n",
"ax.axis([0, 4, 0, 3.5])\n",
"plot_data(X_train,y_train,ax)\n",
"\n",
"# Plot the decision boundary\n",
"x0 = -b_out/w_out[0]\n",
"x1 = -b_out/w_out[1]\n",
"ax.plot([0,x0],[x1,0], c=dlc[\"dlblue\"], lw=1)\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "db45e5dd",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"In the plot above:\n",
" - the shading reflects the probability y=1 (result prior to decision boundary)\n",
" - the decision boundary is the line at which the probability = 0.5\n",
" "
]
},
{
"cell_type": "markdown",
"id": "434a95de",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"###### Another Data set\n",
"Let's return to a one-variable data set. With just two parameters, $w$, $b$, it is possible to plot the cost function using a contour plot to get a better idea of what gradient descent is up to."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "555b0836",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"x_train = np.array([0., 1, 2, 3, 4, 5])\n",
"y_train = np.array([0, 0, 0, 1, 1, 1])"
]
},
{
"cell_type": "markdown",
"id": "61a539b3",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"As before, we'll use a helper function to plot this data. The data points with label $y=1$ are shown as red crosses, while the data points with label $y=0$ are shown as blue circles."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "64f0f080",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"fig,ax = plt.subplots(1,1,figsize=(4,3))\n",
"plt_tumor_data(x_train, y_train, ax)\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "f6ea41f2",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"In the plot below, try:\n",
"- changing $w$ and $b$ by clicking within the contour plot on the upper right.\n",
" - changes may take a second or two\n",
" - note the changing value of cost on the upper left plot.\n",
" - note the cost is accumulated by a loss on each example (vertical dotted lines)\n",
"- run gradient descent by clicking the orange button.\n",
" - note the steadily decreasing cost (contour and cost plot are in log(cost) \n",
" - clicking in the contour plot will reset the model for a new run\n",
"- to reset the plot, rerun the cell"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6983ece1",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"w_range = np.array([-1, 8])\n",
"b_range = np.array([1, -18])\n",
"quad = plt_quad_logistic( x_train, y_train, w_range, b_range )"
]
},
{
"cell_type": "markdown",
"id": "0e72b476",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"You have:\n",
"- examined the formulas and implementation of calculating the gradient for logistic regression\n",
"- utilized those routines in\n",
" - exploring a single variable data set\n",
" - exploring a two-variable data set"
]
},
{
"cell_type": "markdown",
"id": "619d67d0",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"#### Optional Lab - 3.7: Ungraded Lab: Logistic Regression using Scikit-Learn\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "fbb02dcf",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Goals\n",
"In this lab you will:\n",
"- Train a logistic regression model using scikit-learn.\n"
]
},
{
"cell_type": "markdown",
"id": "825654b3",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"###### Dataset \n",
"Let's start with the same dataset as before."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8b4efcaa",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"import numpy as np\n",
"\n",
"X = np.array([[0.5, 1.5], [1,1], [1.5, 0.5], [3, 0.5], [2, 2], [1, 2.5]])\n",
"y = np.array([0, 0, 0, 1, 1, 1])"
]
},
{
"cell_type": "markdown",
"id": "3dcb0174",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Fit the model\n",
"\n",
"The code below imports the [logistic regression model](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html#sklearn.linear_model.LogisticRegression) from scikit-learn. You can fit this model on the training data by calling `fit` function."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d13eeeff",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"from sklearn.linear_model import LogisticRegression\n",
"\n",
"lr_model = LogisticRegression()\n",
"lr_model.fit(X, y)"
]
},
{
"cell_type": "markdown",
"id": "d8f1d0ec",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Make Predictions\n",
"\n",
"You can see the predictions made by this model by calling the `predict` function."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bcf0a8d6",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"y_pred = lr_model.predict(X)\n",
"\n",
"print(\"Prediction on training set:\", y_pred)"
]
},
{
"cell_type": "markdown",
"id": "1df576c2",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Calculate accuracy\n",
"\n",
"You can calculate this accuracy of this model by calling the `score` function."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9728cd8c",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"print(\"Accuracy on training set:\", lr_model.score(X, y))"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b07d490f",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"id": "6ec838ec",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"#### Optional Lab - 3.8: Ungraded Lab: Logistic Regression using Scikit-Learn\n"
]
},
{
"cell_type": "markdown",
"id": "e895d21e",
"metadata": {},
"source": [
"##### Ungraded Lab: Overfitting \n",
"\n",
"\n",
"\n",
"\n",
"\n",
"###### Goals\n",
"In this lab, you will explore:\n",
"- the situations where overfitting can occur\n",
"- some of the solutions\n",
"\n",
"\n",
"##### Overfitting\n",
"The week's lecture described situations where overfitting can arise. Run the cell below to generate a plot that will allow you to explore overfitting. There are further instructions below the cell.\n",
"\n",
" ```python\n",
"plt.close(\"all\")\n",
"display(output)\n",
"ofit = overfit_example(False)\n",
" ```\n",
"In the plot above you can:\n",
"- switch between Regression and Categorization examples\n",
"- add data\n",
"- select the degree of the model\n",
"- fit the model to the data \n",
"\n",
"Here are some things you should try:\n",
"- Fit the data with degree = 1; Note 'underfitting'.\n",
"- Fit the data with degree = 6; Note 'overfitting'\n",
"- tune degree to get the 'best fit'\n",
"- add data:\n",
" - extreme examples can increase overfitting (assuming they are outliers).\n",
" - nominal examples can reduce overfitting\n",
"- switch between `Regression` and `Categorical` to try both examples.\n",
"\n",
"To reset the plot, re-run the cell. Click slowly to allow the plot to update before receiving the next click.\n",
"\n",
"Notes on implementations:\n",
"- the 'ideal' curves represent the generator model to which noise was added to achieve the data set\n",
"- 'fit' does not use pure gradient descent to improve speed. These methods can be used on smaller data sets. \n",
"\n",
"You have developed some intuition about the causes and solutions to overfitting. In the next lab, you will explore a commonly used solution, Regularization.\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "6428c6a2",
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"%matplotlib widget\n",
"import matplotlib.pyplot as plt\n",
"from ipywidgets import Output\n",
"import sys\n",
"sys.path.append(\"week3/OptionalLabs\")\n",
"plt.style.use('week3/OptionalLabs/deeplearning.mplstyle')\n",
"from plt_overfit import overfit_example, output"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "5b657b94",
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "af12f81533e941d494086be48ae8e298",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Output()"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "c4c372a238644bdda01e4f48e2a06862",
"version_major": 2,
"version_minor": 0
},
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAyAAAAJYCAYAAACadoJwAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAA9hAAAPYQGoP6dpAAC1DElEQVR4nOzdd3QUVRvH8W9CQmiBEBJISKU3Q0c6UqRXEQUFRERUUIqI2MVXkaKgqAiigg1ULEgRaSJIUwEpiog0Q+8QSAglZd8/rgkpuymw2d0kv885eyBzZ3eenZ2dnWduc4uKirIgIiIiIiLiAO7ODkBERERERPIPJSAiIiIiIuIwSkBERERERMRhlICIiIiIiIjDKAERERERERGHUQIiIiIiIiIOowREREREREQcRgmIiIiIiIg4jBIQERERERFxGCUgIiIiIiLiMEpARERERETEYZSAiIiIiIiIwygBERERERERh1ECIiIiIiIiDqMEREREREREHEYJiIiIiIiIOIwSEBERERERcRglICIiIiIi4jBKQERERERExGGUgIiIiIiIiMMoAREREREREYdRAiIiIiIiIg6jBERERERERBxGCYiIiIiIiDiMEhAREREREXEYJSAiIiIiIuIwSkBERERERMRhlICIiIiIiIjDKAERERERERGHUQIiIiIiIiIOowREREREREQcRgmIiIiIiIg4jBIQERERERFxGCUgIiIiIiLiMEpARERERETEYZSAiIiIiIiIwygBERERERERh1ECIiIiIiIiDqMEREREREREHEYJiIiIiIiIOIwSEBERERERcRglICIiIiIi4jBKQERERERExGGUgIiIiIiIiMMoAREREREREYdRAiIiIiIiIg6jBERERERERBxGCYiIiIiIiDiMEhAREREREXEYJSAiIiIiIuIwSkBERERERMRhlICIZMFPP/1Er169KFeuHGXKlKFevXqMHTuWqKgop8XUuXNnfHx80j2qV6+eXD5kyJDk9detW8eECRNITExM9ToHDx5kwoQJREZGpttGREQEEyZMyNH3YY2195X0+P777x0ejz2l/VxySnR0NK+//jrNmjUjKCiIMmXKcOutt/LMM89w+PDhbL1WVFQUEyZMYPv27TkTrB2tW7cOHx8fDh486HKv7ePjw9y5c7P9vOnTp7No0aIb2mZOs3Vesae056Hvv/+eadOmWY3Fx8eHNWvW5FgsWXXw4EF8fHxYt26ds0MRcUkezg5AxNVNmTKFV155hc6dO/P2229TsmRJtm/fzltvvcWiRYtYvHgxwcHBTomtRo0aTJ06NdWyggULAiZuLy+v5OXr169n0qRJPPnkk7i7X7/3cOjQISZNmkTjxo0JDw9P9Vpz5szBz88vx+LPyL333svAgQPTLa9UqZITosldjh07Ro8ePTh16hQPPfQQjRo1wsPDg127dvHpp5/y559/ZiuRu3DhApMmTSIoKIjatWvnXOB2UKtWLVauXElAQICzQ7GbGTNm0LhxY7p16+bsUNKxdV6xp7TnoSVLlvDzzz/z2GOP5cj27CEgIICVK1dSpUoVZ4ci4pKUgIhkYO3atYwbN44hQ4akugPXrFkzunTpQsuWLXnkkUccelf+6tWryYmFt7c3DRo0sLpe1apVb3pbtWrVuunXuFFly5a1+d4kYw8//DCnTp3ip59+onz58snLW7RowUMPPcSyZcucGF3OSEhIwGKxULx4cR03eYwzz0M3ysvLS8ehSAbUBEskA0k1HmPHjk1XFh4ezuOPP8769evZsmULAA0bNqR///7p1t2yZQs+Pj6pLvz+/PNP+vTpQ1hYGAEBAbRv356NGzemet6QIUOoXr06mzZtol27dgQEBPDiiy9mKfaUTX0mTJjApEmTAPDz80tuzrRu3Tq6du0KQI8ePVIth/RNHyZMmICPjw/79u3jzjvvpGzZstSqVYsZM2ak2/6aNWto3rw5ZcqUoU6dOnz66acMGTKEzp07Zyn+zHz66afpmmQlJCTQoUMHateuTXR0NAAHDhzgoYceombNmgQEBFCrVi1GjRqVrvlc0r7etm1b8r6uX78+y5cvB2DatGlEREQQEhLCPffcw5kzZ1I938fHh1deeYXJkydTvXp1AgIC6NixI3/88Uem7yUyMpLBgwdToUIFSpcuTbNmzVi8ePEN7ZctW7awbt06nnjiiVTJRxJ3d3c6deqU/Pe3335L165dqVChAkFBQTRv3pzPP/88ufzgwYPJF4DDhw9PPkZSNiVatGgRt99+O4GBgYSGhjJgwIB0zbxiY2MZNWoU5cqVIzg4mL59+/Lbb7+la6ZisVh49913qV+/Pv7+/lSpUoUnn3ySixcvpnq9pP395ptvUrNmTfz9/fnrr79sNpP65JNPaNGiBQEBAYSFhdGpUyd+++235PLx48fTokULQkNDKV++PF27dmXz5s3Z2fXJEhISGDduHFWqVCEwMJDOnTvz999/p1tv69at3HfffcnHS/369Xn55Ze5fPly8joREREcPnyYr776KnnfJ32vs3psW7Nv3z769u1LxYoVKVOmDLfccgsDBgwgPj4+eZ2zZ88yatQoqlWrRunSpWnQoAEff/xxcrmt84otvXv3pnv37sl/JyYmEh4eTunSpYmNjU1ePnDgQNq2bZtqHySdh4YMGcIXX3zBsWPHkrcXERGRajuXLl1i1KhRhIeHU6lSJUaOHJnq9W3x8fFh3LhxvPvuu9xyyy2EhITQs2fPdMdSXFwc48aNIyIiAn9/fyIiIhg3bhxxcXHJ61hrgrVq1SratWtHaGgoQUFB1K9fP3n/JcnK74JIXqAaEBEb4uPj2bBhA506daJQoUJW1+nYsSNjx45l7dq11K9fn969ezNx4kSioqJS/RDPmzcPPz8/br/9dgC2b99Op06dqFWrFm+//TaFCxdm9uzZ9OjRgxUrVqRq5nLx4kUeeOABhg0bxgsvvEDhwoXTxZlSgQIFcHNzS7Xsvvvu49ixY3z22WcsW7aMAgUKAFClShUmT57M6NGjmTRpEnXr1k1enpH+/ftz7733MmzYMJYuXcozzzxDlSpVaN26NQC7d+/m7rvvpl69esyaNYu4uDhef/11Ll68SFhYWIavncRisaR7bwAeHh7J7+mnn35i+PDh1K1bl7JlyzJp0iR+//13li1bhre3NwDHjx+nbNmyjB8/npIlS3Ls2DGmTZvGXXfdxcqVK1O9dnR0NI888giPPfYYgYGBTJkyhfvuu48HH3yQffv2MXnyZE6dOsWzzz7L6NGjU12MAXz55ZcEBwfz2muvce3aNcaPH0/37t3ZunUrJUuWtPo+jxw5wu23346/vz/jx4/Hz8+P+fPnc9999zF37tzkZCEpEXjqqad45plnbO63n3/+GTDHZlb8+++/dO3alREjRuDp6cm2bdt48sknuXLlCg888AABAQF89tln9O/fn1GjRiW/brly5QCYPXs2o0aNol+/fjz11FNER0czceJEOnfuzIYNG5I/h5EjR7JgwQKefvpp6tSpw88//8zgwYPTxfPKK6/wxhtvMHjwYDp06MDu3bsZP348O3fuZMmSJama+Xz++eeEh4fzyiuvULRoUQIDA9MlKgDPP/8806ZNo3///jzzzDO4u7uzefNmjhw5QsOGDQHTbG3IkCEEBQVx9epVli5dSqdOnVi9ejW33HJLlvZlkgkTJjBlyhQeffRRWrduzbZt27jnnnvSrXf48GFuueUW+vTpQ/HixTlw4ABvvvkmkZGRzJ49GzDNj+6++25uueUWnn76aYDk5kjZObbT6t27NyVKlGDKlCmUKlWK48ePs3LlyuS+HBcvXqR9+/ZcuXKFp59+mrCwMFatWsWoUaO4evUqDz/8sM3zii3NmjVj/PjxybW4f/75JxcuXKBgwYL8+uuvyeePDRs20K9fP6uvMWbMGM6ePcvWrVv54osvgOvNTpM8/fTTdOjQgdmzZ7N3717Gjh2Lr69vlm7efPnll1SuXJnXXnuNuLg4XnjhBR566KHkGxFgkqDvvvuOUaNG0bhxYzZt2sTkyZOJjIzkww8/tPq6kZGR3HPPPXTv3p0xY8bg6enJgQMHUvW9y87vgkhupwRExIZz585x+fJlQkNDba6TVHb06FEA7rrrLl555RW+++675P4LcXFxzJ8/nzvvvDP54vnFF18kODiYhQsXJv94tmnThsaNG/Paa6+lugMdExPDzJkzrdYc/Prrr+n6aLz99tvcd999qZYFBQVRtmxZAOrXr58cB1xPNqpUqZLlJgOPPvpo8gVCy5YtWbduHQsWLEi+gJg8eTLe3t58++23FClSBIDGjRtTq1atLCcgU6ZMYcqUKemWR0ZGJid3U6dOpVmzZjz00EM8/fTTTJkyhRdeeIF69eolr9+0aVOaNm2a/Hd8fDyNGjUiIiKCHTt2pGreER0dzRtvvJG8fkBAAM2aNWP58uX89ttvyRdYf//9N++//z4JCQmpLrouX77M/PnzKVq0KAD16tWjXr16vPvuuzz//PNW3+fEiROxWCwsWbIEX19fwBwLR48eZfz48ckJiJubGwUKFMi0nX3SsRgSEpLheklGjx6d/H+LxULTpk2Ji4tj1qxZPPDAA3h5eVGzZk3A1PqlPEZiYmJ46aWX6Nu3b6pOwfXq1aN+/fp89tlnDB06lL179/L111/z0ksvMWLECABatWpFbGws77//fvLzzp8/z7vvvss999zD66+/nrwv/Pz8ePjhh1m2bFmq2huLxcL8+fNTJeX//PNPqvd34MABpk+fztChQxk/fnzy8vbt26daL2X8iYmJtGrVigMHDvDZZ5+lu0udkaioKGbMmMH999/PuHHjAGjdujUFChTgpZdeSrVuytoAgEaNGlGxYkU6d+7M5MmT8fX1pVatWhQsWJBSpUql+35m59hO6ezZs+zfv5/PP/881f686667kv//3nvvcfjwYTZu3EiFChUA811P6g80aNCgDM8r1jRv3pzLly+zefNmmjVrxrp165JrV9atW0fr1q3ZvXs3p06donnz5lZfo1y5cpQqVYqCBQvaPF81bdo0+fhp3bo1+/btY8GCBVlKQAoWLMi8efPw9PRMXjZgwACOHTtG2bJl2bVrF998802qGwFJn++rr77KyJEjrSasO3bs4Nq1a0yZMoXixYsDcNttt6VaJzu/CyK5nZpgidhgsViy/ZyQkBCaNm3KvHnzkpf9+OOPnD17NvkO6OXLl9mwYQPdu3fH3d2d+Ph44uPjsVgs3Hbbbemq2z08POjQoYPV7d1yyy2sXr061cNeTZwykvbirVq1askXvgCbN2+mbdu2yckHmIv5W2+9Ncvb6NevX7r3tnr16uQ76mCaTHz44Yf88ssv3HnnnTRr1ozhw4enep24uDjeeustGjZsSNmyZfHz80tusrFv375U6xYtWjTVBV3lypUBc+GVMtGoXLky8fHxnDhxItXz27Ztm5x8AISFhdGgQYMMm/KsWrWKtm3bUrx48eRjIT4+njZt2rBz587kO/qhoaGcPXuWp556Kkv7L6v+/fdfHnroIWrUqIGfnx9+fn68+uqr6faNNZs3b+bixYvcfffdqWIPCgqiUqVKycfyli1bsFgs6S640/69efNmrl69Su/evVMtT0reN2zYkGp5mzZt0tUIprVmzRoSExO5//77M1xv7dq1dO/enYoVK+Lr64ufnx9r1qzJ0n5I6a+//uLSpUv06NEj1fKePXumWzc6OpqXX36ZunXrUqZMGfz8/OjUqRMWi4X9+/dnuq3sHNsp+fr6Eh4ezv/+9z8++eQTq9tatWoV9erVIywsLN1xee7cOXbv3p1pfGnVrFkTHx8f1q5dC5hRq1q0aEHz5s1TLfP09EyumboRac9P1atXT3V+ykirVq1SJR9JowomPT/pmE57jN59990A6Y7RJBEREXh6ejJo0CAWLlzI6dOnU5Vn93dBJLdTDYiIDaVKlaJw4cIcOnTI5jpJZUFBQcnLevfuzbBhw4iMjCQ8PJx58+ZRuXJl6tSpA5i7vAkJCbz++uvJd+nSSkxMTL7T7e/vb7NpQ7FixZJf15HSNicqWLAgV65cSf775MmT+Pv7p3te6dKl01202xIQEJCl99agQQMqVarE7t27GTJkSLoagpdffpkZM2bwxBNP0LhxY4oXL05iYiK33357qpgBSpQoke59AenatSddoFy9ejXd+0vL398/w4u106dP8+WXX/Lll19aLT937lzyHdOsSDoWDx8+TMWKFTNcNyYmhu7du1OwYEGeffZZypcvT6FChViyZAmTJ0/OdFtJF1FpE4kkSfvt5MmTAOmOibT76/z58wCUKVMm1XIPDw98fX2Ty5NkZaSrc+fOASTfqbdmx44d9OrVi+bNm/PGG28QGBiIh4cHr776aqr+GFmR9F7Tvjdrx8Zjjz3GqlWreOqpp6hVqxbe3t4cOXKE/v37pzs2rcnOsZ2Sm5sbCxYsYMKECfzvf//j3LlzhIWFMXz4cAYNGgSYz/bAgQM2R8FL2q/Z4e7uTpMmTVi3bh0JCQn88ssv3HfffcnNDy9evMi6deuoX79+qpsX2WXt/JT2u5qd5wLJ+9PWMZr0d9pjNEn58uX59ttveeutt3j44Ye5evUqdevW5X//+x/NmjXL9u+CSG6nBETEBg8PD5o0acLq1au5cuWK1X4gS5cuBczoQkmS2vh+9dVXPPLIIyxbtownn3wyubxEiRK4u7vz4IMPWm0XDqT6kUnbnyM3KFOmTLo7fACnTp2y+7YmTpzI/v37qVGjBk8//TRNmzZNVUvy7bff0qdPn1T9Jv7991+7xwHW39/p06cJDAy0+RxfX18aN27MyJEjrZZn9FxrbrvtNl555RWWLl3KsGHDMlx3y5YtHDp0iB9++IEmTZokL1+yZEmWtpXUZGz69OlUq1YtXXmxYsWA6xdnp0+fTlVDlHZ/JV38nTp1KtXrxcfHc+7cueTtJcnKd6NUqVKA6S9hawjnxYsXU6BAAb744otU/QkuXbqU7Qu+pPea9j2kfa9Xrlzh+++/56mnnkr1OVnrw2LLzRzb4eHhzJw5E4vFwp9//skHH3zAE088QWhoKG3btsXX1xd/f38mTpxo9fmZJbe2NG/enLFjx7Jx40ZiYmJo2rQpxYoVo3DhwmzYsIENGzbwwAMP3NBrO0LKYzSpHxRcTzzTHqMptWjRghYtWnD16lV+/fVXJkyYQO/evfnjjz+y/bsgktvpaBbJwPDhwzl37hwvv/xyurLIyEimTp1KkyZNqF+/fvLy4sWL07FjR+bNm8eCBQu4evVqcvU8mGY+jRs3ZufOndSqVYs6deqke+SEpKF7097RtbX8ZjRo0ICVK1emGnnmxIkTqUYdsoeNGzfyxhtv8MILL/Dll19y/vx5nnjiiVTrxMbGpmpSAaZjb05YuXIlly5dSv774MGDbN68OcO+NW3atOGvv/6iatWqVo+FlHO5ZEX9+vWT7+QfOHAgXbnFYklOnJM+n5T759q1a3z99depnmPrGLn11lvx9vbmwIEDVmNPuuCvX78+bm5uLFy4MNXzFyxYkOrvBg0a4OXlxbfffptq+fz584mPj0/VPC6rWrZsibu7e7oBA1KKjY1N179mz549bNq0Kdvbq1GjBkWLFk333ubPn5/q72vXrpGQkJClY9PLy8vq99Mex7abmxs1a9bk1VdfBUgeratNmzbs2bOH4OBgq59tUpKf3fNH8+bNuXbtGpMnT6ZWrVr4+Pjg4eFBo0aNeO+99zh79qzN/h9JbO0PR0g6BtMeo0nfmZSJvC1eXl7cdtttDB8+nEuXLnHw4EGn/S6IOItqQEQycNttt/Hss88yfvx4Dh06RJ8+ffDx8WHHjh1MnTqV4sWLM3PmzHTP6927N/Pnz2fixIk0bdo0XYfgV199lc6dO9OzZ0/69+9PmTJlOHv2LDt27MDNzY0XXnjB7u8lqbP5tGnTaNu2LQUKFKBOnTpUrFgRDw8P5syZQ8mSJfHy8qJixYqpahGya/To0SxcuJA777yTxx57jGvXrvH6669TunTpLN/FO3bsmNW+EyEhIQQEBBAVFcVDDz1E8+bNGTZsGG5ubrz11lvcf//93H777clJX9u2bfniiy+oWrUqVapU4aeffsryHf7sKly4MD179mTYsGFcu3aNCRMm4O3tzaOPPmrzOc8++yxt2rShU6dODB48mNDQUKKiovj77785fPgwb7/9NmCa+9WpU4cxY8Zk2g9k5syZdO/endatW/Pwww8nT0S4e/duPvnkE3x8fOjYsSO33norxYsX54knnuCZZ54hLi6O6dOnp/uMSpcuja+vL/Pnz0++wA4LC8PX15eXX36Z0aNHc/bsWW6//XaKFy/O8ePHWb9+PS1btqRnz55UqlSJu+66i1dffZXExERq167N2rVrk4elTtpeyZIlefTRR3njjTcoUqQI7dq1459//uHVV1+lcePG6dr2Z0W5cuUYOnQo7777LjExMXTs2JECBQrw+++/U7lyZXr27Enbtm2ZPn06Dz/8MP379+fw4cNMmjQpyx35U0oaJnfKlCkUK1aM1q1bs3XrVj777LNU6xUvXpyGDRvyzjvv4O/vT2BgIN99913ykN4pValShV9++YVly5ZRpkwZfH19CQsLu+Fje+fOnTz99NP07NmT8uXLk5CQwOeff46Hh0dybe7QoUP57rvv6NixI0OHDqVixYrExsayd+9efv311+RhmG2dV2ypUaMGpUqV4ueff07VXyupZsTLyyvTvmJVqlTh/PnzzJo1KzlJr1GjRqbv2x6qVatGr169mDhxIvHx8TRs2JBNmzbx+uuv06tXL5sjps2ePZuNGzfStm1bgoKCOHv2LG+++SaBgYHJNWXO+F0QcRYlICKZGDNmDHXr1mX69Ok8+uijXL58meDgYPr06cOoUaOsDq+aNHLPsWPHrA6ZWrt2bX766ScmTZrEU089xcWLF/Hz86NmzZo8+OCDOfI+OnTowIMPPsisWbN47bXXsFgsREVF4evry+uvv87UqVPp3LkzCQkJLF68ONO7kBmpWrUqX331FS+88AIDBw4kMDCQkSNH8uOPP2b5NT7//HOro7688sorDBs2jBEjRnD58mXee++95KY4PXr0oH///owePZpbb72V8PBwXnvtNdzc3JIvGJo2bcr8+fNzZHKzPn36UKRIkeShQuvWrcusWbNsDsELJqFavXo1EydO5JVXXuHMmTP4+vpSrVo1+vbtm7yexWIhISEheZjUjJQtW5affvqJGTNmsHDhQt5++20SExMJCwujXbt2yfNI+Pn58fnnn/P8888zcOBA/Pz86Nu3L0FBQakuDt3d3Xn77bd55ZVX6NGjB/Hx8bz77rv07duXgQMHEhQUxNtvv80333xDXFwcgYGBNGnSJNXF2NSpUylWrBhvvfUWcXFxNG/enMmTJ9O7d+9UfVxeeOEFSpUqxUcffcSsWbPw9fWlT58+vPjiizfcBGXcuHGUL1+eDz/8kC+++IIiRYpQo0aN5FHbWrVqxZQpU3jnnXf4/vvvqVChAq+88gorVqzIsA+YLUnf+U8//ZQPPviAevXq8eWXX9KoUaNU633wwQc8+eSTPPvss7i7u9O+fXs++ugjWrZsmWq9sWPHMmLECAYOHMjly5e55557mDFjxg0f22XKlCE4OJh3332XY8eO4eXlRfXq1Zk3b17yUK8lSpRgxYoVTJo0ialTp3L8+HFKlChBpUqVUnWwt3VescXNzY2mTZuyaNGiVE1Xk8439evXtznseZL77ruPLVu28PLLL3PhwgVCQkL4888/M3yOPc2YMYPw8HDmzJnD5MmTCQgIYOTIkRneGLjllltYuXIlL7/8MqdPn6ZkyZI0atSIDz74IHkgBWf8Log4i1tUVFT2h/oREcmmmJgY6taty4MPPsiYMWOcHY7d+fj4MHr0aJvD7Up6b7/9NhMnTuTAgQOZXnSKiEjeoRoQEckRTz75JA0bNiQgIIATJ07w3nvvceXKFaszxUvet2zZMv7++29q1qyJm5sbGzduZNq0aQwePFjJh4hIPqMERERyxJUrVxg7diynT5+mUKFCNGzYkCVLlmR7VCfJG4oVK8bixYt58803k5sxPvHEE4waNcrZoYmIiIOpCZaIiIiIiDiMhuEVERERERGHUQIiIiIiIiIOowREREREREQcRgmIiIiIiIg4jBIQERERERFxGCUgIiIiIiLiMJoHxEUVLVoUd3flhyIiIiKuIDExkUuXLjk7jDxBCYiLcnd3VwIiIiIiInmOrnBFRERERMRhlIBkYMyYMURERODj48OuXbtsrvfpp59St25dateuzYgRI4iPj08uW7ZsGQ0aNKBOnTr079+fmJgYR4QuIiIiIuKSlIBkoHv37ixbtoyQkBCb60RGRjJ+/HiWLVvGtm3bOHnyJJ999hkAMTExDBs2jLlz57Jt2zYCAgKYMmWKo8IXEREREXE5SkAy0LRpU4KCgjJcZ9GiRXTp0oXSpUvj5ubGAw88wDfffAPAjz/+SJ06dahcuTIAgwYNSi4T+/jjLAz8CcrNgULvQ7EPoO7X8No2OHfl+notF5pHSm4z4KXN2dvemqPmed/sv/nYXcH9P0H4nBt77vSd8PHurK8//ndY8O+NbSsnWTs2rEn67NccvbHtvLTZPP9GfL4Hpu64sedmV2bH+GPrbvx9ZOd4y8r38+PdZr2kx5nL18ssFvhyLzT/Dkp/ZM4PwZ9C++/hwxQV2pEXzXMnb888Jmuf4bUEeORnCPwECrwHtb+CY5fMutvPpH+NqTtsx2zL879B6Gfg8R74zMp8/Ztx/0/mPJoV4XPM+o7w8hao/iUkWhyzPYC4BPjfZvM+vWZC1S/gnT/Tr9d/FfRYevPbu5HfJMj4eBNxVeqEfpMOHz6cqoYkNDSUI0eO2Cw7fvw4iYmJ6mBuBx/sgqHroEoJeLI2VC8JcYmw5TS89xf8chK+62D7+b/cAcHFHBZunjN9J/gVhvurZm398VuhVwXoUS5n48qLPt8HO8/ByFrOjsQ1zW8PgUXAx+v6smd+g0nbYHA1c37wLggHo+Gno7AwEh6snv3tPFgNOqSpEJ/xF8zcBe80g3r+UMzTXBD+bwuEe0Ntv9Tr96kEjcrAh3/DrCwk8Av/hVe3wnN1oWMoeBXIfty53bFL5qbSx63B3c1x2x26Dj7bA680gAalYflhGLEeoq/Bs/Wur/dSfaj6Jfx0BFoHOy6+JBkdbyKuSgmIHbi5XT8jWiwWm2ViP7+cgCFroW0wLOiY+ke5bQg8UQuWHcr4NRoF5GyMIuIYdfwgvPj1vy/Hw9Q/4L7K8H7L1OveX/XG76IHF0t/02LnOSjsAY9FXF+25ZTt1wgoYh7LDmdtmzvPmX+HR0DpItmL15bYOCjiaZ/XcoS3/jDJZc/yjtvmX+dg1t/wakN4so5Z1jIIzl6BcVvhkRrgW8gsr1DCJKYTtzknARHJjXQb/iaFhIRw6ND1K93Dhw8THBxstezQoUMEBgaq9sMOxm8FNzdzcWHtjmDBAtAtkzvt1qq7j8bAQ2sg5FMoOBPKfgK9lsPJWNuvc/GaadZR5mPYdDLjbR6Khn4/miYhXjOh2hcwZXvqC6KUTULe2GGalxX7ABrPh19PZPz6cL1ZysrDpnma72wo+gF0/QEOXMz8+Vfi4ZlfzXYLzoSgT+HRtRB19fo64XPgr/Pw87HrTUkyalrjNgMuxcMn/1xfP2Wzp51noftSKDnLNJWp/RV8ksXmXe/uhBYLzD4t+gFEzDN3S+MSUq9nsZjlYZ+ZbdT9GpYetP6au89Dh++hyAfg95FpYhMdl7V4AJYcNO/Ba6bZj7aa92Ql9pYLzesdjE7ddCfJ/zZDw2/N51z8Q/O+Zv1t3q+jJP63b6t+Yd5z6Y/gvlVwJAtjbly8BoPXQKnZ5jjv8D3sibq5eC7FwdUECCxqvdzWXfTMvm9pm2C5zTA1GZfjr38uH++GBt+a8oGrry+/kaY14XPg+U3m/2U+Sf06Wd3nLRfCLV/C2mPQZL45ph9Yk/m2/zoHbRaZ49L/I9P0LjaT70DSuScyzXnGVvPFH4+YbRT/0MTV9DtYdST1OtcSTE3RvZWuf24WC1T63Jx304qJgxKzzDnrZiz4FyzAwDQ1vAOrms877Q2u/pXN+9l/IfPXzuoxv++COYdX+tzsn6BPzXn8z7PX11lzNOPjbcsp6LPSHEuF3zf/3rPSnE9EnEk1IDepW7dudOjQgTFjxuDv78/s2bO58847AWjTpg2jR49mz549VK5cmVmzZiWXyY1LSDTNKOr5QYgdm1AdjTEn8rhEeLYu1Cxl7nYtPwznr0IZK3cfj8RApyVwLRF+6Qnli6dfJ8npy9DkO/OD+sqtprr8+4Mw+hfYfxGmt0i9/rs7oaoPTG1q/n5hE3T6Af7tCyW80r18OoPWmBqiz2+HwzHmQqblQvjj7tRNVVKyWKDHMlh1FJ6pA80DTT+bsZtNk7ZfepqE77v20GsFlCgI05ub52bUNOSXO6D1YmhVFl74r+lC8YLm33/Om/1SujC83QxKFYI5e+D+1XDyMoypk/H73H/BXJyU8zaJ544zpsnK7iiY3er6ev/bYh6DqpqmYIdjYPDPkGCBKj7X1zsZC7ctBE93897KFIa5e80FWFasOmKSqcZl4Mu25vVf2249ic1K7NObw0M/m2Pku/bpXyMyGh6uDqHe5u9fT8Kw9XD0ErxYP2sxW5NogfjE9MutJTZD1sL7u+CxW6BLmInphU2w5hhs7WWa6lljsZi28xtPwov1TDOXDSeg45IbjxvM9iqWMM0ESxeGTqHmM86oQvpGvm+/3AGv/A6rj8FPXc2ywKLwUStzMfh8PegcapbfSHPP79qbuGbthmWdzfct6XWys8+Px0K/VTCmNoxvmHkzprhEc157uDo8XQc2njB3/Q9Gw+JO2X8f1szZYxKm7uXgk9bm+zZzl0kqlneBNv/VJPx20pyHW5W9/lw3Nxh2C4zcAHujoJLP9bJP/zEX+I/ecn2ZtePYmgJu14+RnefAv5CprUqpZqnr5Sm1LGsSlh8OwbAIbMrOMX/skjkfTmwI/oXh3FVzE6fhfNjWC6qUhLr+GR9vkdGmmXKfiuDrZY6FGX9Bg29gVx/b381cJyYKivk4OwrJBiUgGRg9ejQ//PADJ0+epEePHhQtWpRt27YxbNgwOnbsSKdOnQgPD+eZZ56hffv2JCYm0qJFC/r37w+At7c3b7/9Nn379iU+Pp7q1aszY8YN9t6UZGeuQGw8lMvgYv9GvLjZvPaOu6FayevL765off3tZ6DzD1ChOCzocL063pY3dpiLwt96wq1lzLL2oeYC9b2/YGRNqOxzfX1vT/i+ExT4r8KsbFG49VtYesi0I89MfX+YleICvIavucP47k54rp7156w4bBKu1xpdb3bQNsQker1Xmh/3wdWhjj8ULmCSiKw0ZWsUYKpb/QunX/+lLSaBW939ekLZKQyirpmE4eHqGSdcbzS9/v9Ei0maShUyP8hTmkBJL1N7M2kb3FEOPrSyT1ImIG/+YZLFbXdBrf/aU3cMg3aL4VAW7ug/95tJVld2hUL/nWHbh1ivIcpK7NV9TcLoVcD6vv6oderXaFnWXOS89adJ9m60FWjvlVlbb/d5cyE8tAa80/z68jp+5kLpzT9MMxZrlh82F+9vNYXhNc2ytiFQ0B2e23RjcSf5/HZTe/nERvPw9oTbysLdFaBf5fT75Ua+b40CzDHtTurP5hZf82+F4jfX1LOO//ULyXr+1y8Ws7vPz12Fr9tlvXnQtUTTjDXlZ+L532ey4Tg0Dbzx9wSmJmXEepM4peyn1ynM1OA9+xv89l+sv/xXq1zXP/VrDKxqbqq8uxOmNru+/N2dJlmp7nt9mefMrMX1UavrfdrOXrF+Ti/qaY7Ps1dTLy9dBIKKmmQiowQkO8d8i7LmkSQh0SQYNeaZZO2NpuYcnNHx1quCeaR8jS5hpsb+873XY8i14q7Bh2Ng3bfwwU4oWsLZEUkWKQHJwOTJk5k8eXK65e+8806qvwcMGMCAAQOsvkanTp3o1MlOt4wkRy09BK2CUicftiw/bC4Su4SZjpFZ6Rj601HTUT4p+UhyfxVzR+qno6kTkM5h1y+G4Pqdt4NZnEqmb5qLpiYBEOYNq4/aTkB++q+JRNqO5XdVgAdWm5qRwTfQeTcjPx2FNkHpa7Pur2I+k19OQodQ28/fdtrU0Gw4YS60UtoTBQ3LmD5DVxJs75OUVh81iUlS8pHk3kqwMk3zkLQuxcHm0+bCsFCKs6t3Qegabu5eZjf2zPx0xDRJ3Hza3PlN6dRl6zV3WTGpEbS2Mgjg69vhqxQjZK22cczcWsZ8l1YdsZ2AJD23b+XUy++tdPMJSIPSsO9ec3ytPW4Gp1h11NQ6frUfFnVMnYTc7PfNkbK7z0t6Zb9vgq3PZPWxm09ANp40x/uAKulrJzqEmqZll+LMxf6xS+AG+KVJBrwLmiTk43/Mey3qab4Lu86bGuaUNmex4UHam1oZ5e7WykoXNjXpGcnOMR+faPbFnL2mOVZcin319/mMt5MkJg5e2QLfHjC1IQkpajCz+hou7doV2LYKOj8Kd9wFn38OfuqJnxsoAZFcx68QFPGAf7PQnyE7Tl+BYBttxtNa8K9pBzykRtZHpTl7xTS7Sqts0evlKZVK84ObtJ3L8WRJ2qYDAAGF09+5Sxujh7u5q5uSm5t5vbQx2sPZK2YEo7Rs7ZeUDkVD8wWmBuOtZmb/FioAm07Bo+uu76uk92xrn6SNx1rtmrXnpnX+qqmFsLqdNMuyGntGNp2Edt+bWo8PbjN3ywu6m+Pz1a1ZP1asKV8c6pdOvzztsZG0b61+hkUyvoBPOt7SHutZ2ddZ4VnA1DK2D72+vV7LTRKy9JC5457kZr9vjpTdfW5tvYxk9JnY4xyQ1Byx1wrb65y7apKKywmm9iVlcphkWARM22maSD5U3fw/uCh0D0+9XlZHhiqQIqsoVcj6sLaX4kwNka+VWtlCBUy8GcnOMT9qo6nReaq2qb0r6WWazz24JvPtJLl3pUm8X/ivuVfxgiZ56rQk66/hkjb9ACFVIbA8vLwCAv+rKvL3h9OnlYTkAkpAJNcp4G7umC89bPpg2GsoXf9CcORS1tZ9synM22fa7X7XAdrZnqsyWalCpv1tWsf+22baO3w364SVbZ24DBUzaLpWqpC563b6cuoLTYvFvF4DKxekN+tm9suCf03n9vkdUtdkpL1wKPXfxYKtfZIyMSxVyMZ6GQxEkKSkl/lxz8rzsxp7Rr7cZy7Ovu+UusbFkfOtJO3b47Hpv4vHYjP+/JKOt7NXUl+QZWVf34hShUxTxzXHTBv+lAlIbpLdfZ7dVngZfSZpL5xTKvRf0nY1zYXtmTRJS1J87zQzQxJbU6bw9XWvJV6vEUmpYgkzNPG7O82/iyLhfw3SJys30gQrwtd8v07Epk4OkjqA3+Kb/vnnrlq/yZRSdo75OXvMSG7jG6VefuaK7X58KV24apLtsfXh6brXl19NSF/jmmskxMPHL8C8idD7Keg++nrykURJSK6g4ZgkV3qmrrkoHrzGdOpOKy4BFkdm7zU7hprq8X+yUC1dqICZe6BLGHRbasbqz0ybINM8YOvp1Ms/3WMuEFplPOdlts3dm/rvjSdMJ9KWGWwnqePnnD2pl397wFwst0nxXK8C2bs7bGv9NkGmmcyxNMnfp3tMTZetCxS43oQmZS2UxQIf/J16vUYB5jOztU9SahVkRgDakSYR+DzNc60p6gm3lob5/5rRxJJEX0t/PGY1dgAvd+v7zs3N3E1Neef2cryZu8BRkppppT1mNp8yTTzaZHC8JR3zc9M8Nyv7OiNxCbbv1Cc1OymbxdrOG5HTtSc3s8+zytZn0rJs+nWTJF18/5Gmg/aiyNR/Nw0En4LmfFi/tPVHwf/2YVUf8+9+GzXeIyLMQBkDVpnvweBq6dfZfGfWHl3Drz+nezlzXk7bbPLjf8ywy2mbhcYnmoEtqmfShDc7x7wb6WvYlxw0fQlTsnW8ubmZjvFpX+PDv1M3xco1zh6DMW3g69dh8OvQ7QmTbFjj7w9nNDOjK1MNiORKjQNgRgszUVS9b0xTqBq+po3sttPw/t/mDlXKH5TMvNzANMtosdCMghXhazpCLzsEo2pB1TQ/LJ4F4Iu2pjq81wr4tDXck0Hn8MdrmYvqzj+YbYV5mx+T6TtN/Cn7f9jDltPw4GrTf+NwjGlfHFTU9E+wpW2w6TD91K+mP0HTFKNg1fGD/lWurxtRytwhnLfPNNcpVMAssyWilLnzvDjSNAnx9jSjuIytb+7StVpoRm3y9TKJwpKDpjN8Rh3Q2wabJkf3rDQj/FxJMP1pzqe5u1fSC0bXhnG/p94nL21J3/RhZE2Y/bf5nMbden0UrN1RtuNI6ZVbocMSaLsYnqhtOn1O2m6Sk5R3HbMae9K+m/8vzNhpOiO7u5mLtM6hZnCDe380TVDOXoHJO6w3Cxy02lxM7e+bvt9Lhs6cgb59Ye5cq3cUq5Q0237nTxNXx9DrIzKFFDPHvS3tQqBFIIz51SS49f1Nf5ibTaAuXDOd/u+qALcHmzhi4szx99Yfpp9EzxycELNCcXOROnev2VYxT5Pw2CvpuZl9nhUF3WHKDrPPGpS+PgpWx1BolkH/jwalTZPC0RvNBXlJL/juX1h/PPV6xTxN5/kBP5nvRK/ypv/E6cuw46z5d8ZtZt2kGya/nrzeLyeltiHmon/1MTO4gLW5Uqw1JcxMDV8YVM2c+wq4mfe24rDp/D/u1vQd1P84awZHyexGUnaO+S5hJuGp6mPe+++nTR+stE2FMzreWgSa5/gVMnPl/HzMDNPtUzD1axyMhgpzTb+clIOXuIz4OBjVHOKuwpSfoUZTaG9lWMCU+vaF5csdE59kmxIQybUGVzd3m9/8w4xwdCLWNEep7AP3Vkw9MVhWBBWDTXeaH5yJ28zFnH9haBZgvb0vmB//WS3NxXS/VaaZgK0Zlv0Lw8Y7zAzNz/xmLvDLF4fXGpsEx95mtTQ/an1WwtVEMzLMW80yHq3Lzc2M6PXSFvjoH9OPwK+QGeN+fMPUF7b/a2CagAxeY+bICPOGyH62X/utpqZvQ5+V5of6trKwpru5mNp4hxn5JqnvQ7WSqZtD2FK1JHzb3oyG03O5adJwbyWzP9MOa/lyAyjqAdP/Mvulakl4r4W5YE8poAj83MOM0jNkramFuaMcTGsG3ZdlHA+YC6IFHUxMvVeY1xt6i3lf/9tyY7GPqGlqZZ79zVxcWwDLENOxeHYrc/x3XWoSzMHVzMXcoDWpXyPBYh7Zmh/kzJnrdxiTmjWQPgmZ0cJcBM362zSHKVHQ3CGe0DDjJjvubqYz+KiNprPttURoGgA/dDIzS9+o4gXN8bnqqNlnJy+bu8nlipsE86k6OTsRXxFPmN3SfN7tvjc3RsbWh5ca2G8bN7rPsyKpWd/w9SbxKFzAHFevN874eQXcYXFHeGw9PLLW1Nz1qQTTmpuEPqV+lSG0mBmi+uH/5tkpXdj017g/xY2OkGJmdLiF/5qky5q7K5hz1mO3WC+/UdObm+/UO3+a35dwb3MOtTbK1YJ/zbkys+a42Tnm32pmPosJ20wyWNfPNNlMmhsmSUbH2+e3w4gNJuGJ/29bK7um/zwsluvnCJcSH2eaXXkVhhEzoXwt8PnvnDR3ru0akKRycVluUVFRrna4CWYIX01YKDfi491mhK7Nd97YnT8RIHXykZILta1OOtb33WsSYI9ccMpMutB7eYuZQ+T0/XloLoYc8u1+Myz0wX7mRlFa9b8xCebmXg4PDTC1nBU/NzcQbI34Jjfg1GEY3wfCqsPjH1hfx8HnqcTERKKjNYujPeSC07WIiDiUrR91cMm21RU/Nx2Nz1x2diSZe+sPE+srvzs7ktyjZ3nTBGrCtuvLLl4zTcOe/dU0TbI1tLgjzNljaiierO28GPKcXxbDkNpw+jC0G2h7PT+//2pmU3ChmyRim5pgiYhIan37Zl7uAm2ru4annuMhKyMDOdu9lVL3o8gNMTubm5sZZnpRpBnm2t3NDObRapFpbja2PvTIwT49mUkE5t6uz9IuLBaY+QTMfxMad4MnPoLiVoYcSykpCcmgr5q4HjXBclFqgiUiTpNRDQjoDqOI5JwPn4JSZaHH8NSzhboANcGyHyUgLkoJiIg4VS7oAyIiecSquXA1FjoNdnYkGVICYj+6whURkfTUtlpEclpsNLw2ACb1g382OzsacSAlICIiYl1SEtKunZIPEbGvPb/D0LqwYT6M+RQef9/ZEYkDqQmWi1ITLBEREcmznu0IF07Ds19CUEVnR5MlaoJlP0pAXJQSEBEREclT9m41fT1uaQZRp6FoCfAsmPnzXIQSEPvRFa6IiIiI5JxrV+HjF2DYrfD162aZj3+uSj7EvjQPiIiIiIjkjD1bYPJAOLwb+r0IvZ92dkTiApSAiIiIiIj9JSTA+HugiDdM2wIVajk7InERSkBERERExH52b4IS/hBYDsYvg9Kh4OHp7KjEhagPiIiIiIjcvGtXzEzmIxvD/DfMsrIVlHxIOqoBEREREZGb8/evpq/HiQMwYBzc/aSzIxIXpgRERERERG5cTBQ83RZCq8G7WyG8hrMjEhenJljiOGfOQPv25l8RERHJ3f7+DS5fgmI+8PpqmLpRyYdkiRIQcYwzZ8DfH1asMP8qCREREcmdrsTCzCdMX48l75llletDATWskaxRApKJ/fv3065dO+rVq0fr1q3ZvXt3unW++OILmjVrlvwoX748/fr1A+DgwYOUKlUqVfm///7r6LfhXEnJR0pKQkRERHKfnethSG1Y9C48+BrcMdLZEUku5BYVFWVxdhCurGvXrvTp04e+ffuycOFCpk2bxsqVKzN8TpMmTXjqqafo3r07Bw8epFWrVhw4cCBb2/X29sbdPQ/kh9aSj5ROnwY/P8fFIyIiIjfm8D/wYDWo1ghGzYbQqs6OyKESExOJjo52dhh5Qh64ws05p0+fZseOHfTu3RuAbt26cfDgQQ4ePGjzOb///junTp2iU6dOjgrTtfXte3PlIiIi4lz7t4PFAiFVYNwPMGVdvks+xL6UgGTg6NGjBAYG4uFh2jS6ubkRHBzMkSNHbD7ns88+o3fv3nh6Xh/zOjo6mlatWtGiRQsmTZpEQkJCjsfuMubOvblyERERcY7Ll+Dd4TCkDmxcaJY16AAFCjg3Lsn1lIBkws3NLdXfFovtFmuxsbHMnz+f/v37Jy8LCAhg165drF69moULF/LLL78wbdq0HIvX5fj5mWZW1qj5lYiIiGvasQYeqQnLPoQhU6FRV2dHJHmIEpAMBAUFcezYMeLj4wGTfBw9epTg4GCr6y9cuJAqVapQter1akkvLy/8/+sDUbJkSfr168fGjRtzPnhXYi0JUfIhIiLimrb+CE+2Ar9geO8PuGOEaj3ErpSAZMDf35+IiAjmzZsHwKJFiwgNDSUsLMzq+nPmzElV+wGmH0lcXBwAV69eZfHixdSsWTNnA3dFSUlIu3ZKPvIrzQMjIuLajv83SmetVvD0XDO3R1BF58YkeZJGwcrE3r17GTp0KOfOncPb25sZM2ZQrVo1hg0bRseOHZM7m//77780b96cv//+G29v7+TnL1q0iAkTJuDu7k5CQgLNmzdn3LhxeHl5ZbjdPDMKlgikHw1NSaiIiOuIjYYPn4If3ofpW6F8PrxRmgUaBct+lIC4KCUgkmfYGopZSYiIiPNt/RHefBAunDHzenR5BHT9YZUSEPvRESYiOSejeWA0GaWIiHOt+BiebguBFWDmn9BtqJIPcQgdZSKSczQPjIiI6zl/yvzbqBs8/iFM+hECyzk3JslX1ATLRakJluQJGdWAgJphiYg40qULMPMJ2PAdzNoNPhmcnyUdNcGyHw9nByAieVjS6GfqAyIi4lyblsLUhyD2Ajw0BUro/CvOo1vsIpKzNA+MiIhzfTEenu8EYdXh/Z3QaTCkmWhZxJFUAyIiOS8pCenbF+bOVfIhIuIIl2OgcDFo2AV8ykCHB5R4iEtQHxAXpT4gIiIickOiz8OMkbBvK7z7O3gWdHZEeYL6gNiPakBERERE8opfFsFbj8DVWBgyFTw8nR2RSDpKQERERETygukjYMHb0LAzjJgJfkHOjkjEKiUgIiIiIrlZ3DXTzKpWK6hUD27vr74e4tLUB8RFqQ+IiIiIZOjCGZg+HBLi4fmvnB1Nnqc+IPajK1wRERGR3Gb9fBhcA7Ysg8bdwaL7yZJ7qAmWiIiISG5hscDEfrD6c2jSA4bPAN8AZ0clki2qARHXcuYMtG9v/hURERHj8iW4dsX07ahQG575AsbOV/IhuZL6gLiofNkH5MwZ8Pe//rdmyxYRkfzOYoGfPodZT0HXoXDPs86OKN9SHxD7yWdXuOKy0iYfYP5WTYiIiORX/2yGx5vCpH5QtRG0vMfZEYnYhRIQcT5ryUcSJSHibGoWKCLOsG8bDLsVrlyC136CF7+BwHLOjkrELpSAiPP17Xtz5SI5JSk5XrFCybCI5LxrV2D1l+b/FWrDSwvg3d+hditnRiVid+oD4qIc1gfk1GHwD3buhEUZ1YCA+oKIc9g6LnU8ioi9WSywYQF8MBpOHYKZf0JoVWdHJWmoD4j9qAYkP7t6GYY3hCdbmapeZ/HzMxd11uhiT5xBzQJFxFH+/ROeuh1e7glBlZV8SL6gBCQ/8yoMoz+CqFPwaD14czCcP+mcWKwlIUo+xFnULFBEHGXDd3DmCIxbAuOXKvmQfEFNsFyUQ4fhjY+DJTPh07EQUhWmbnDMdq05c8Zc3M2dq+RDnEfNAkUkp8THwffvwdVY6P3Uf3N7uINnQWdHJplQEyz7UQ1IJvbv30+7du2oV68erVu3Zvfu3enWWbduHYGBgTRr1iz5cfny5eTyZcuW0aBBA+rUqUP//v2JiYlx5FvInIcndH8MPtoLI983y/bvgBWfQEKCY2Px84Ply3VxJ86lZoEikhN+XwlDasOMEabWA6BgISUfku8oAcnEyJEjGTBgAL///jsjRoxg2LBhVterUqUK69evT34ULlwYgJiYGIYNG8bcuXPZtm0bAQEBTJkyxZFvIeuK+0J4DfP/TT/A5PthaB3YvMx0kBPJT9QsUETs5dpVGNsdnmkHxf3MyFaPvuPsqEScRglIBk6fPs2OHTvo3bs3AN26dePgwYMcPHgwy6/x448/UqdOHSpXrgzAoEGD+Oabb3IkXru65xl46xco6gPPdYSn28Kx/c6OSsSxkpKQdu2UfIhI9l2OMTfwCnpB6VB4/iuYvAYq1nF2ZCJOpQQkA0ePHiUwMBAPDw8A3NzcCA4O5siRI+nW3bdvHy1atKBVq1Z8+OGHycsPHz5MSEhI8t+hoaEcP36cxMTEnH8DN6taI5jyM/xvoTmJFvY2y69dcW5cIo6kZoEikl0JCbB0FgyoAOu+NcsefQda3OXcYe9FXISHswNwdW5pThQWK02RatWqxV9//UWJEiU4evQod911F6VKleKOO+6w+hq5ipsbNO5mHgAxUTC4BrTsA/c8Z5ptiYiI5FXZGRzFYoHflsCsp+HgX9DqXqja0DFxiuQiqgHJQFBQEMeOHSM+Ph4wycfRo0cJDg5OtV7x4sUpUaJE8nN69erFxo0bAQgJCeHQoUPJ6x46dIjAwEDHjXBlbx4FofMj8MP7cH8F+Op11YiIiEjelDQi3ooVWZsDaM08eLEr+JSGdzbBM3OhdEjGzxHJh3LpVbBj+Pv7ExERwbx58wBYtGgRoaGhhIWFpVrvxIkTyU2qoqOjWb58OTVr1gSgTZs2bN26lT179gAwa9Ys7rzzTge+CzsrVAT6vQAf7zN3dmY/AxM1J4KIiOQx1objtpaEHNoNy2ab/ze9A8Yvh9dWQZUGjolTJBfSPCCZ2Lt3L0OHDuXcuXN4e3szY8YMqlWrxrBhw+jYsSOdOnXi/fffZ/bs2RQoUICEhAS6d+/O008/ndz06ocffmDs2LHEx8dTvXp1ZsyYQfHixTPcrkPnAbkZR/aYGpDyNeGfzRB9Huq3c3ZUIiIiNy4rcwElxsJn/4OVH0NAOXj/L9PZXPIszQNiP0pAXFSuSUBSeudRWDwd6raFwa9BhdrOjkhERCT72rc3za5saRsGnsehSAm49znTNFnJR56nBMR+lIC4qFyZgFgssHEhzHoKju41TbQengIlyzg7MhERkayzVgNSAEiam3fSQChbDnqOhCLeDg5OnEUJiP1oFCyxHzc3aNoDGnaGZbNMbUjBQqbsSqzpPyIiIuLqkuYA8vc3vWVDgHBgD/CH5gQSuVm57Ba75AoentDlEXhvBxQtAedPQb9QmD4Czp90dnQiIiKZK14M5kyApkBF4BSw5g8lHyJ2oAREck7S/CeFisAdI2HlJ3BfeXj/SZOUiIiIuKq1X8Nnz8Htd4F7M1h7GipHODsqkTxBfUBcVK7sA5KStYmbLp6Db9+AhW9DrVZmhnURERFXcDkGFs+As0dhyFSIj4OTByGoorMjExehPiD2owTEReXqBCRt573TadrLXjwHly5AYDnYtBT+WAO9RoNPBkMeioiI5IRLF2HRNHODLPYidBgEw6Zfr8UX+Y8SEPtRJ3SxL1sTN6VMQor7mgfAmSOms/rCadD9MSUiIiLiOFdiYWAluBQFHR6E3k9B6VBnRyWS56kGxEXlyhqQrEzcZK3z3sWz5s7TgrchMRFeXQo1W+RcnCIikn9dPAtLZsKdT5i5O1Z/CRHNwS/I2ZGJi1MNiP3ksitccWl9+95YefFSMPBV+CzS3H2qXN8s//kr0/5WRETkZkWdhllPQ/9w+PxV2LfVLG/VR8mHiIOpBsRF5asaEGvirsGACnD+hJnQsPdTEFbdPnGKiEj+8t3bMPsZcHeHbo/BnaPU3FeyTTUg9qMExEXlygQEbCch2Uk+kly+BMs+hK9fhzNHoVlPeG4eFFDXJRERycSZo+ZmVmA52LgQ9v4Od4wwte4iN0AJiP0oAXFRuTYBgcxHwcquuGuwag5E7oRH3oCEBNi1EW5pplFKREQktVOHYN4kcwOr+V3w9BxnRyR5hBIQ+1EC4qJydQIC1ucBsZfflsALXaBqQ+jzDDTqaqrVRUQk/zp7DD4daya9LVLcdDLv9igULe7syCSPUAJiP0pAXFSuT0ByksUCW5bDF+Nh5zoIqwEPTIDGXZ0dmYiIONq5E+AbAGePw8jG0HWoeRQu5uzIJI9RAmI/usKV3MfNDRp0gDfWwhvroEyYufMFcP6kmVRKRCQnnTkD7dubf8XxEuJh/XwY1QIerglXL0OpQPjkANw9RsmHiItTAiK52y3NYNwS6PyQ+fvjF6BvCLw/Gk4ddm5sIpI3JfVzW7HC/JufkhBnJ14JCfD1ZLi/Irx8J1gSYcRM8ChoytVyQCRXUBMsF6UmWDfozFEzq/qS9+ByDLS4G+5/BQLLOzsyEckL7DnSX25j7wFGsuP4AQgoZ/4/qrk5p/cYAZXrOWb7IqgJlj0pAXFRSkBu0uUYWDYbFrwF45dDUEU4th/KhEOBAs6OTkRyI3vOdZTbOCPxSkyE31fAd2/BlmXw5gao0cTUgug8Lk6gBMR+lIC4KCUgdpKYaKrkExJMlb2bG3QfBu0HQjEfZ0cnIrlJ+/am2ZUt7drB8uWOi8dRnJF4/fiZma38yD9Qsa6Zv+O23lDQy77bEckGJSD2oytcyduSkrgCBeD5r6B6E5j1FNwbDO88ampKRESyYu7cmyvPrfr2vbnyrDoRCRfPmv8f2w/lIsxAI+9ugbb3KfkQyUNUA+KiVAOSg84ehyUzYftPMHmNSVK2/QQRzcHD09nRiYgry499QHKyBsRigT/XmmZWvyyE/v+Dvs+b5ZpoVlyMakDsRwmIi1IC4kAnIuG+cmYc+Q4PQqeHoHSIs6MSEVflzM7YzpITidf21fDe43BgB4RWM53K2/SDwkVvLlaRHKIExH50hSsSEA4z/4CmPU2n9fvCYVJ/Z0clIq7Kz89ceLdrlz+SD7j+nlO6kfd+/ADs3Wr+7+EJfkEwYQV88Bd0eVjJh0g+oRqQTOzfv58hQ4Zw9uxZSpQowfTp06latWqqdX7++WdefvllYmJicHd3p1OnTjz//PO4ublx8OBB6tatS7Vq1ZLX/+yzzyhXrlyG21UNiJPERsNPn0PsRbj7SYiJgh/eh3YDwSeDJggiIvnBmTOmz8fcuVlPPq5dgQ0LYNmHsG0V1G8P45flaJgiOUE1IPajBCQTXbt2pU+fPvTt25eFCxcybdo0Vq5cmWqdHTt2UKJECcLDw7ly5Qo9evRg0KBB3HXXXRw8eJBWrVpx4MCBbG1XCUg23ciPYlZsWQFjuwEWaHYndBliJj9U22QRkczt2QLPtIfoc3BLc+j4IDTvBYWKODsykWxTAmI/usLNwOnTp9mxYwe9e/cGoFu3bhw8eJCDBw+mWq9WrVqEh4cDUKhQISIiIoiMjHRwtPlYTs5KXL8dfHEUBk4wP6RPtIC3h9jv9UVE8pLLMbB0Fsx5xfwdWt30q5u1G95Ya0azUvIhku95ODsAV3b06FECAwPx8DC7yc3NjeDgYI4cOUJYWJjV55w8eZKFCxfy1VdfJS+Ljo6mVatWJCQk0LlzZ0aPHk0BTaJkH9Y6Rvr727dddvFS0GsU9BxpRs4q7G2Wb14GKz+B9g9AnTbXh/wVEclPLBbYvck0sVrzJVy5BE16mOWFisCgCc6OUERcjBKQTLilaWpjsdhusXbx4kX69OnD8OHDqV27NgABAQHs2rULf39/zp8/z8CBA5k2bRojRozIybDzh4yGhrR3EgImwah7+/W/46+Z0VueaQelQ6Hd/SYZKWM9ORURyVPi40xH8nMn4PEm4BcMdz1pzoWlQ50dnYi4MN2yzUBQUBDHjh0jPj4eMMnH0aNHCQ4OTrdudHQ0vXr1omPHjjz22GPJy728vPD/7yK5ZMmS9OvXj40bNzrmDeR1jpocy5bG3czILW/9AvXaw/w3Yf18UxZ1WpMcikjek5hoOpKPvwceqAIJ8VAqEN7eBJ8cgH4vKvkQkUwpAcmAv78/ERERzJs3D4BFixYRGhqarvlVTEwMvXr1onXr1owZMyZV2enTp4mLiwPg6tWrLF68mJo1azrmDWTAYoF/L8Jf5+ByvLOjuUGuMCuxmxtUawSPvw9fHIdOg83yL16Fu8vAq31g40K4djXnYxERySlx12DG4zCgAjx1O+zfDt0eNbUgAJXrgSs0LT5zBtq3t29fQBGxO42ClYm9e/cydOhQzp07h7e3NzNmzKBatWoMGzaMjh070qlTJyZPnszEiRNTDc/bo0cPRo8ezaJFi5gwYQLu7u4kJCTQvHlzxo0bh5eXV4bbzclRsH4+Bo/8DLuj/tuWJzxRC56vBwVyW0rqqrMSH//XtIX++Us48AcU84GRH0CLXs6LSUQkq67Ewu8r4O9f4MFJZtmYNhBUCW7vD9WbuN5ogPlxgkhxKI2CZT9KQFxUTiUgG45D60VwLTF92fAIeKuZ3TeZ81z9R+fgLlj9BbTsA+E14PuZcHAntLwHqjd2vR9xEcmf4q7BT3Nh4wKTfFy7AmHV4c0N5iaKK3PVm1GSpygBsR8lIC4qpxKQht/CplO2y3f3gSol7b7ZnJdT84DkhO/egq9eg7PHTIf1ln3MMJWB5Z0dmYjkN8f2wz+boVUfSEiAe4OgbEVo3B2adIfgys6OMHMZDUgCSkLEbpSA2I8SEBeVEwnIpTgo9mHG67zXAh6uYdfNijUJCbBznakZWfcNPDUHbu1ommt5FYGgis6OUETyIosF9m41tRwbF0DkTihUFOadgMLFTNOr3DZPR/v2Zh4oW9q1g+XLHReP5FlKQOxHw/DmIwlZSDWzso7YQYECUKuleTz6Drj/13nzkxfhl4VQpQHc1gda9ga/IGdGKiK5XXwcHN4N5SJMgvF4U5NkNOwC/V+C+u1N8gG5L/k4cwauXct4HUcMSCIi2aIaEBeVU02wqn4B/0TZLt98J9QvbffNSlZdiYXfvjc1I5t/MBcO/1sEjbqYO5fqLyIiWXHpImxZZkbh27QEEhPg69NQsBDs32H6dnh4OjvKm5NZ0ytQ8yuxK9WA2I8SEBeVUwnIt/uhl42a6h7l4LsOdt+k3KhLF2DDd2ZG4WI+8NYjcPIgNL3DtM0uWcbZEYqIK0lqPhV9HvoEQtxVKF/LnEOadIcKtfPOTQwlH+IESkDsRwmIi8rJYXhn/w0jNkBM3PVlvSvCrJZQNJffEMvTVnwMKz+FP9eCJdEMg/nIm6a5lojkT4d2m74cvyw0A1t8FmmSjFVzzTkisJyzI7S/zJKPli3h66+VfIjdKQGxH/UByYceqAZ3VYD1x+FyAtQsBRVLODsqyVS7+83jwhnTTGvDd1DsvyHL5k+FM0egUVeo0RQK6KstkqedPwVPtIAj/4BXYajfATo/YppaFfCANn2dHWHO6ZvJeytYUMmHiItTDYiLyskaEMmD5rwM38+AcyfAuyTU7wh3P2maXIhI7nbtCmz/yfTnOHMExi0xfcLeHw01b4M6t+e+zuM3Q8PuipOoBsR+lIC4KCUgkm2JibD3d/h1sXk8Ns3Uhqz+Eo7tgzptTHMt1Y6I5A7nT8K7w2DzUrgcA2UrmP4cgyaZkfTyM008KE6gBMR+lIC4KCUgYjefjjVNtGIvQhFviGgBPUdBndbOjkxEUjp12PTlOHcCBo4zM5M/297UcDTuDuE18k4ncntIm4Qo+ZAcpgTEfpSAuCglIGJXCfGw53fYvso05egxAhp3hbVfw7pvoXZrU0MSWD77Fzi5aRZ6EVdz8Swsmm4Sj72/mxrK+h3g5UVKNrJC5x9xICUg9qMExEUpARGHWPs1fDMF9mw2TbjKhMHdT0HXIVmbd0R3IEWyJyEB/toAZ49Cq3vMkLn3V4S6bc1QuQ06mmG3RcTlKAGxHyUgLkoJiDjUpQvwx1pTQ1KjGbToBb9+D7OeMrUjtduYWdtTXhipDbZI1sRGw6Yf4Pflpn/WhTMQfgvM/MMk+Qnx6pslkgsoAbEfJSAuSgmION3erWZkre0/wfED4O4OHQfDiPfg1EkIDIBEG89VEiL5lcUCh/+BnevAoyC0GwDH/4UB5SG02n+TAvaAyvXNd0pEcg0lIPajBMRFKQERl3IiEratAm9faHYHtGsEib9BFHAO8+8Frick7drB8uXOiVXEGfZsgS/Gw871cOG0SS6a3wXPfWmSkqhTULKMs6MUkZugBMR+lIC4KCUg4tL2/QWtbwFfoCTgCUQDv/5X/v0sqNsSAsqpI624rhvpwBx9Hv7ZBH//avpy1G1r5tz5Z7OZl+OW5hDRHKo1hqLFczZ+EXEoJSD2owTERSkBkXRcbbSXlH1AimGSkPPAP3/AYzXN8hL+UPVWqNIQ7h4DBb2cFKxIGlkZQOHSRdi3FYIrQ6my8MUE+OhZU1a8FFRrBLffB7fd7bi4RcRplIDYjxIQF6UERFJx1dGmbMV14Yy5S7z7N/M4th8+2mNqQ8Z2hwKeUKkeVKpr/i3hAu9F8o+MBlDYvgy2LDNNqo78Y5YPfw+6PAz7tsHBXWZCz6BKqt0TyWeUgNiPEhAX5YwEJD4RIqNNc+VyxcFD+Y9rcPXRprJSM5NySN/Zz5rmK/u2mtG3AN76xdxN3vYTRJ+DsOrmAs/D0zHvQfKP06chrDQUxTyKAd7AVuAaML4fnNhnOolXqm/+DamqmcdFRAmIHSkBcVGOTkDm7IGnfoVjl8zfgUVgYiO4r4rDQhBrbCUfSVwlCbkRiYlmdK19W+HWzlC4KEzsBz/NNeUenhBUGe59zsyXcOGM6dxbtqISE8nclVhTg3F4Nxz6G67GwkOTzQAJcStNk8F44BJwEfgXuIoGUBARm5SA2I8SEBflyATk490wcLX1sg9bwqBqDglDrGnfHlassF2eFy+Wok7Dwb9MU5eDf0HTO6Du7fDDBzD1oeuJSXgNMz9J54dMDUtCvBKT/MZiMYnpob9NouEfDLd2gr9/gxGNrq/nGwgV68C4JSapr+Bvko2rVl4zNyf1IpKjlIDYT55JQO644w6+++47Z4dhN45KQK7EQ9CncM7aDzHgUxCO3gdFdF3nHHm5BiS7YqLgwA6I/Ou/BOUvCK0Ow6fD2ePQL9R0Fg6rYZpvBVaAtgNM05mszOourishAU4dNIlGhdrgFwSLZ8DHz5sme2CGve38CAx713Qe3zDfNJ0KqZp+ZnFXb9YoIi5JCYj95LoE5NVXX023zGKx8M0337B9+3bHB5RDHJWAbDoJDednvM6GO6BJQI6HIrboYilz0edh9RcmKTm0C47ug9iL8F2USTyGNzIJTGB5KFsBAspD8zuhdCjEXTM1J0pQnMdiMZ/h6cNw6hA06mI+j9nPwqYlcGQPXLti1n1iNrQfCH+ugz9+NglGaDXTNC87o6y56sAOInnc3ih4dyf8eQ68PaFbuGnunRv6nSoBsR8PZweQXR999BGvvPIKFkvqvKlIkSI5sr39+/czZMgQzp49S4kSJZg+fTpVq1ZNt96nn37K1KlTSUxM5LbbbmPKlCl4eJjdu2zZMl544QXi4+O55ZZbmDFjBsWKFcuReLMrPgvpZ7yt2a7FMfz8zMWRLpZs8y4J3YamXhYfdz2p6PigqTk5cQC2r4YTs8zoW6VDYc7/YP6b4B9qmvCUCoKGnc3QqpcvmaSmhJ+ZhLFoCSUq2RV3Dc6fgHMn4PxJiDoJRX2gRS+TFI5sYpKOK5euP+erU+DjbxLDao2h3UCTZIRWA79gs07Ef/Nt3Kik75UrDW0tecaJWPjzLHgVgPr+akWQZMVhuGMZxMZfX7YwEubth4UdoFCuuyqVG5XrakA6duzIrFmzKFu2bKrlDz30EO+//77dt9e1a1f69OlD3759WbhwIdOmTWPlypWp1omMjKRDhw6sXbsWf39/7rnnHtq3b8/AgQOJiYmhTp06LFmyhMqVK/Pkk09SrFgxxo4dm+F2HVUDcuEqBH4Kl+OtlxcqAMcGQElN3+B8rjYPSG5msZiHu7uZQO6vDeYi+MwROHMUGnWFPk/DXxvh8abXn+dewNSgzP5veNZ3HoVrl6G4n5kXongp0wehVFm4eNYkQd6+4FnQOe/T3iwWuBxjEoeY86a5W4EC8Ov3pnnc+ZMm2Th/EnoMN7VMy2bDG4NSv069djBhuRmIYOYo8A8xyaB/qPm3ZBnz2YjkMtcS4PENMHMXJPx3dVXSC95uBv0qOzc2Zzt/FcrPgahr1sufrQuvNnRsTNmlGhD7yXUJiMViwc1BdyBPnz5NvXr1OHDgAB4eHlgsFqpUqcLKlSsJCwtLXu/tt9/m0KFDTJ48GYAVK1bw1ltvsWTJEhYsWMDnn3/OV199BcDu3bu56667+PPPPzPctiM7oY/dBC//br3subowzsVPCCI55kosHN1jkomkR2ICdH/MlE/sB8f2meXRZ82F+eSfzZ35D5+Cr14z6xUuZpKTtvfDfS/BiUiY9bRJTDy9zKOwNwyaYNZfNtsMUZxU5ukFtVqaGpoTkXAy8vpyd3dToxAQbmobDv5lEp9rV8wj7opJqNzcYMMC05ciqezaFZMkVK4P21aZjv5Jz7l2BcrXgqFvmT4V94WbmBJTVInOOwklS8PLd8Kfa6FkgEkefMqYZlJ1b4fTRyByJ/gGmOU+/lBAtzklb3pgNXy023rZ1+2gVwXHxuNKZv8Ng9bYLi9TGE7c76BgbpASEPvJFb8CTz/9NBMmTMDNzc1hyQfA0aNHCQwMTG5K5ebmRnBwMEeOHEmVgBw+fJiQkJDkv0NDQzly5IjNsuPHj5OYmOgyEw2ObQBxifD6juvNrQq4wRO14OVbnRubiFMVKmI6Pdvy9JzUfyckXP9/x8FQo5lJTJKSl/Bb/lsvDi6egbir1x8eBYH/EpClH5iL9rirJpkA+N8ik4Cs/hw+ei71dlvcBc9/ZWoehtZNH+f3V0z/iAVvm4khCxa6/qhUzyQgVy+bmAoWgkLFTK2OX5B5fuFicPdTJtEp9t8j6f8AL3xju2maf7B5iORxu8/bTj4AnvkN7iyff1txHrmUcfnJyxCXAJ6acidfyBUJyLp167jnnnuYNWsWRYsWTV5+5coVZs2axaOPPppj206b8KTte2JtvbTrODJpuhHubjC+EQyLgE2nwALcWhrKFs30qSKSUsrJ6oIqmoc1QZVg0o+2X+etX67/PzHxvwTlv0bkXYeahCPumlluSYRiJU1ZyTIwbYupYfAqbJIJz0LXm4C9/pPtbTbqYh7WuLtD76dsP9fFz3F2paaQYsMvJzMu33cBzlwB/8KOicfVBGVyTVG6sJKP/MQ1bsFnYvny5VgsFjp06MDx48e5evUq06dPp1atWsyYMSPHthsUFMSxY8eIjzcdJCwWC0ePHiU4OPXdvJCQEA4dOpT89+HDh5PXSVt26NAhAgMDXab2I6XAotC9HPQop+RDxGW4u5tkIqnZUjEfk8CE14BKdU3tRdn/2nV4FoTK9aBCLTMkcelQ00QqPyUIOSlp5KwVK8y/Z844OyJxIVn5lrnn46/iHeWgeAbd4e7XxMf5iutdBVtRrFgxvvzySxo1akTr1q2pVasWM2fO5JlnnmHbtm05tl1/f38iIiKYN28eAIsWLSI0NDRV8yuAbt268f3333Pq1CksFguzZ8/mzjvvBKBNmzZs3bqVPXv2ADBr1qzkMhERySWsDYetJERSaB6YcXm1kuCbjwd08S0EX9xuBrdJq2VZeKmB42MS58kVndCvXr3KJ598wltvvcW1a9e4cOEC3377Lc2b38QQjFm0d+9ehg4dyrlz5/D29mbGjBlUq1aNYcOG0bFjRzp16gTAJ598kjwMb4sWLXjjjTfw9DRNJn744QfGjh1LfHw81atXZ8aMGRQvXjzD7TqyE7qIiGRAE4JKFg1fD+9YGWPGDfi+E3QKS1+W3/x93uyjneeg2H/zgAyqmjuaX6kTuv3kigSkSpUqeHp6MmrUKPr3789XX33FM888w+uvv07v3r2dHV6OUAIiIuIi2rc3za5sadcOli93XDzishIS4YVN8OYfcOW/MSmCisK7zU0TZ8ndlIDYT65IQD744AMGDBhAwYLXGw+uX7+eAQMGMHDgQJ5//nknRpczlICIiLgI1YBINl24Cv9EmYkIa/jmjlm+JXNKQOwnVyQgthw4cIDevXuzefNmZ4did0pARERciK0kRMmHSL6hBMR+cnUCAhAVFYWPj4+zw7A7JSAiIi4mbRKi5EMkX1ECYj+5/go3LyYfIiLigvz8TNLRrp2SDxGRm5Dra0DyKtWAiIiIiLgO1YDYT66YCV1ERERE8p/1x+HV3+H306ZTf7sQeLkBBBVzdmRyM1QD4qJUAyIiIiL52bf74e6VkJjmSjWgCGy8A8plPKWa3akGxH50hSsiIjfnzBkzV4ZmBRcRO7l4DR5Ykz75ADgRC0PXOjwksSM1wRIRkRuXcmQof391zhYRu1h52CQhtiw7DNHXwLug7XXEdakGREREboy1uTH8/TOuCVFtiYhkwfmrma8TlUGCIq5NNSAiIpJ9Gc0ObqsmRLUlIpJFlX0yLvf2hIDCDglFcoBqQCRPio2Dt/+Ansug+1IzgsbZK86OSsR+TsbC2mOw7TQkJDohgL59s1d+I7UlIpJvNQuEuhncn3j0FvAs4Lh4xL40CpaL0ihYN+70ZWi7GHacTb08uCis6pb5XRURVxYbB4+th0/+ud45M8wbZrWENsEODCSjGhBIXbuRnXVFRP5zMBo6fA+7o1Iv710RPm0NBR2cgGgULPtRAuKilIDcuDuXwfx/rZfV9oOtvcDNzbExidiDxQI9lsGiyPRlBd1hbQ9oWMaBAdlKLNImFO3bw4oVtl+nXTtYvtz+8YlIrnc5Hr7aB1vPmPNc+1BoE+Sc33ElIPajBMRFKQG5MWcuQ+mPIaODetOd0KC0oyISsZ+fj0HLhbbLWweZWj6HSpuEZNb3wxrVgIhILqAExH50hSt5ypFLGScfYKp0RXKj9cczL7c4+paSn59JINq1s51IJK1jjZIPEZF8RwmI5CmBRTJfp2zRnI9DJF/x8zNNqDJKJKwlIUo+RETyJSUgkqeUKQLtQmyXVy4BjRzZRl7EjpoEZF7u0v2bslJbIiIieZ4SEMlzPmwJ5bzTL/f1gnntwD2HL9BOxsLOsxCVhUmURLKjZVnoHGa9zNMdxjd0bDw3JCu1JSIikqcpAZE8J6QYbOkFL9SDZgHQuAyMrAnb7zajYOWUyItw+yII+AQivgK/j2DAKrigRETsxM0N5rWFAVVSJ9IhxWBJJ2gcgGYad1X6XEREkmkULBelUbByl5OxUO8bOHopfVnTAPipm+PHK5e87fglMza+t6dJrD3cydqIVOJ4+lxE8gSNgmU/usIVsYOJ26wnHwAbTsCX+xwbj+R9gUWhVRDUL20j+QDNNO4K9LmIiKSjBMSG2NhYBg0aRJ06dahXrx6LFi2yut7x48fp2bMn9evXp0mTJtx///2cP38+uTwiIoIGDRrQrFkzmjVrxvz58x31FsSBVh/NuPynTMpFbkpG82zoYtd59LmIiFilBMSGd955h4IFC7Jt2za+/fZbRo8eTVRUVLr1ChQowJNPPsmWLVvYuHEjISEhvPTSS6nW+eSTT1i/fj3r16+nZ8+ejnkD4lDxiTdXLnJT+va9uXLJGfpcRESsUgJiw3fffcfgwYMBCA8Pp0mTJixZsiTdeqVLl6Zx48bJf9evX5/IyEhHhSkuIrOZ1W/VzOuSk+bOvblyyRn6XERErFICYsORI0cICbk+oURoaChHjhzJ8DkJCQl88MEHdOjQIdXywYMH06RJE4YNG8YZVbnnSU/XgaIe1ssqFIf7qzo2HslnNNO4a9LnIiJiVb5NQDp27Ej58uWtPpISDbcUM3pZLBkPFmaxWHjiiScoUaIEDz/8cPLyH374gQ0bNvDzzz/j6+vLkCFDcuYNiVNVKQnLu5jhUFOq7w8ru0Lxgs6JS/IRzTTumvS5iIikY+Oebd63dOnSDMuDg4M5dOgQfv/9SBw+fJi2bdvaXH/MmDEcPXqUuXPnpho+N6kWxdPTkyFDhlC/fn07RC+uqGkg7L/XjHp16jKUK24SEJeemVrylqSL3b59TfMeXeS6Bn0uIiKpaB4QGyZMmMChQ4eYMWMGkZGRtG3blk2bNlGyZMl0644ZM4YDBw4wd+5cvLy8kpdfunSJuLg4fHx8AJg2bRpLlizJNPkBzQNib9tOw5t/wM5zZt6EbuHwWAR4aW4OERERyQLNA2I/SkBsuHTpEo899hjbt2/H3d2dF198ke7duwMwe/Zsjh8/znPPPcevv/5Khw4dqFy5MgULmnY2YWFhzJ07l8jISPr3709CQkLy8okTJxIWFpbp9pWA2M+3++GeHyEuzUhUjcuY5lFFPZ0Tl4iIiOQeSkDsRwmIi1ICYh+nYqHcXIiNt14+qhZMaeLYmERERG7EuSvw7k745SS4AbeVhUdqqJ+hoygBsR8lIC5KCYh9zPwLHllru9yvEJwe6Lh4REREbsT+C9B6ERyKSb28cglY3R3KFnVOXPmJEhD70RWu5GnHYzMuP3NFkwSKiIhrs1hMU+K0yQfAngvwwGrHxyRyM/LtKFiSP4R5Z1weVBQ8lIbnG1tOwYStsPWMGYCgbTC8UA9KF3F2ZCIitu08B5tP2S5ffhgOx6QfCl7EVenSS/K0nuWgVCHb5Q9Vd1ws4lxLDkKT72D+vxAZDf9EwbSdUP9b88Od21ks5n3sOgexcc6ORkTsKSvnqLxwHpP8QwmI5GklvOCbdtY76HUPh2fqODwkcYJLcdB/VfqR0MD8aA/NoJ9QbvD7aaj3DYR+BjXmgf/HMOYXiEtwdmQiYg9BWejfkZV1RFyFmmBJntcyCP68G977C/46D8X+mwfkrgrgrkkC84WVR+D8VdvlSw5CTJw5NnKbP8/CbQvgUoqR3mLj4fXtcDIWPm6tyTBFcruapaC2H2w/Y728VdnMmxyLuBIlIJIvhHrD+EbOjkKc5czljMstmOEtc2MC8tSvqZOPlD7dA0NvgYZlHBuTiNiXmxt8cTu0WgQn0gyuEu5tbjSI5CZKQEQkz6tYIuPyIh4QkAs7olsssPpoxuusPqoERCQvqFoStt8Fb/0BG0+aNvQtysLwCPDNoK+jiCtSAiIieV6LshDhC3+es17+cHUoWMCxMdlLfCYzOWVWLiK5R5kiqs2XvEGd0EUkz3N3g/kdoHzx9GVdwmB8Q8fHZA9ubtDAP+N1MisXERFxNM2E7qI0E7qI/UVfgzl7zKhRXgWgbYgZkCA3D0bw4xFot9j0Y0mrRaCZITk3vz8REVehmdDtRwmIi1ICIiJZ9c1+GLwGoq5dX9Y5DD5rAyW9nBaWiEieogTEfpSAuCglICKSHZfiYN1xiI6DGiWhuq+zIxIRyVuUgNiPOqGLiOQBRT2hQ6izoxAREcmcbrGLiIiIiIjDKAERERERERGHUQIiIiIiIiIOoz4gIiIikmttPgWrj0KCBRqVgZZlzRw5IuK6lICI5DLxiTB9J8zbD2evQMUS8Ngt6oAsIvlLXAIMWgOf7Um9vG0wfNMeihd0RlQikhUahtdFaRhesSYuAbothWWH05e91gierOP4mEREnGH0Rpiyw3pZz3LwbQfHxiN5n4bhtR9d4YrkIm/9aT35AHjqV9hxxrHxiIg4Q/Q1eHen7fL5/8K+C46LR0SyRwmISC4yd6/tMgvw5T6HhSIi4jT7L8KVhIzX+eucY2IRkexTAiKSi5yMzbj81GXHxCEi4kzenpmvUywL64iIcygBsSE2NpZBgwZRp04d6tWrx6JFi2yu6+PjQ5MmTWjWrBnNmjVj48aNyWX79++nXbt21KtXj9atW7N7925HhC95VPniGZeXy6RcRCQvKF8cqpe0Xe7rBU0CHBePiGSPRsGy4Z133qFgwYJs27aNyMhI2rVrR4sWLfDx8bG6/ooVKyhWrFi65SNHjmTAgAH07duXhQsXMmzYMFauXJnD0UteNaQGbDhhvayIB9xX2bHxiIg4g5sbfNgSWi9K3xTLDXj/NiisKxwRl6UaEBu+++47Bg8eDEB4eDhNmjRhyZIl2XqN06dPs2PHDnr37g1At27dOHjwIAcPHrR7vJI/3FsJnqiVfnlhD/j8dgj1dnxMIiLO0DgA1t8BtweDpzsUcDPzgKzoCndWcHZ0IpIR3R+w4ciRI4SEhCT/HRoaypEjR2yu36VLF+Li4rjtttt47rnnKFq0KEePHiUwMBAPD7Ob3dzcCA4O5siRI4SFheX4e5C8x80NJjeBruEwbx+cuQKVSsCgapk3zxIRyWvq+cPKrpBoAYsFCui2qkiukG8TkI4dO/LPP/9YLVu7di1gEoYkFovt6VL+/PNPQkJCuHTpEo8//jgvvvgiU6ZMSfcamb2OSFbdVtY8REQE3N0wba9EJFfItwnI0qVLMywPDg7m0KFD+Pn5AXD48GHatm1rdd2kmpKiRYvy4IMPMnLkSACCgoI4duwY8fHxeHh4YLFYOHr0KMHBwfZ7IyIiIiIiuYgqK23o3r07H3zwAQCRkZFs2LCBTp06pVsvKiqK2FgzNmpiYiLz588nIiICAH9/fyIiIpg3bx4AixYtIjQ0VM2vRERERCTfcouKilKbICsuXbrEY489xvbt23F3d+fFF1+ke/fuAMyePZvjx4/z3HPPsWnTJkaOHImbmxsJCQnUrFmTSZMmUbKkGR9w7969DB06lHPnzuHt7c2MGTOoVq1aptv39vbG3V35oYiIiIgrSExMJDo62tlh5AlKQFyUEhARERER16EExH50hSsiIiIiIg6jBERERERERBwm346CJSLiSIkWWHcc/j4PPgWhfSiU9HJ2VCIiIo6nBEREJIcdjoGey2DL6evLinjA+7dB38rOi0tERMQZ1ARLRCQHXU2Ajt+nTj4AYuOh/yr48Yhz4hIREXEWJSAiIjlo/gH467z1Mgsw7neHhiMiIuJ0SkBERHLQtjMZl289nXG5iIhIXqMEREQkBxUqcHPlIiIieY0SEBGRHNQuJOPy9qGOiUNERMRVKAEREclBTQPg/irWy0oXhvENHRuPiIiIsykBERHJQW5u8EFLeKl+6nk/2ofAxjsgpJjTQhNxuKsJsPMs7DoHcQnOjkZEnMUtKirK4uwgJD1vb2/c3ZUfiuQlCYlw6jIULwhFPZ0djYjjWCww4y94cTOcvWKWBRSB1xtDP82F45ISExM5efIk8fHxzg7FaTw8PChTpkzy9VhiYiLR0dFOjipv0ESEIiIOUsAdAos6OwoRx3v7Txi5IfWyE7FmLhyLBfrbaKYoznPy5Em8vb0pViz/VtPGxMRw8uRJAgMDnR1KnqNb7CIiIpJjLlyF536zXT76F7im5lguJz4+Pl8nHwDFihXL1zVAOUkJiIiIiOSY30/DpQyu4U5dhr9tTNYpInmTEhARERHJMQlZ6GmalXVEJO9QAiIiIiI5prYfeGZwtVG8IFT1cVg4kkvVrl2by5cvWy0LDw9n586dN/zaa9asoX79+jf8fMk+JSAiIiKSY/wLwxO1bJe/UA+KaFQ4ycT27dspXLiws8MQO1ECIiIiIjnq1YYwpjZ4pLjq8CoA427NODkRSeLm5kZMTAwA69atIyIigltvvZXHHnsMi+V6G769e/fSuXNnGjRoQK1atZg+fXpyWb9+/ahfvz41a9akS5cunDp1yuHvQwwNwysiIiI5yt0NJjWGx2uZTunuwK1loFQhZ0cmWXb2OJw7nnpZsZIQWA6uXYGDu9I/p1Jd8+/hf+DKpdRlZcKhuG+2w7h69Sp9+vRh7ty5tGzZkq+++op3330XgISEBO69914+++wzqlatSmxsLI0aNaJRo0bUrVuXqVOn4ufnB8DEiRN5+eWXmTZtWrZjkJunBEREREQcIqAIdA5zdhRyQ5bMhDn/S72sdV94eg6cPgKP1kv/nBX/1UxMvh/+/jV12ZjP4PZ+2Q7jn3/+oUiRIrRs2RKAu+++m4ceeii57K+//qJPnz7J60dHR7Nr1y7q1q3L3Llz+eyzz7h69SqXL18mICAg29sX+1ACIiIiIiIZ6/wwNO6WelmxkuZf/2B493fbzx39sfUakBuQsrmVtTI/Pz+2b9+ermz9+vVMmzaNjRs34u/vz6JFi3j55ZdvKAa5eUpAbIiNjWXYsGFs3boVd3d3xo4dS7du3dKtt3v3bh588MHkvy9cuEB0dDSRkZEAREREUKhQIby8vAAYNWoUPXv2dMh7EBEREbGLUoHmYU3BQtebW1kTYr+p7qtWrcrly5dZu3YtLVq04JtvvuHChQsAVKlShSJFivDpp59y3333AbBv3z58fX05f/48xYsXx9fXl2vXrjFz5ky7xSTZpwTEhnfeeYeCBQuybds2IiMjadeuHS1atMDHxyfVelWrVmX9+vXJfz/55JPpXuuTTz6hevXqOR2yiIiISJ7m5eXFF198wdChQylcuDAtW7YkNDQUAA8PDxYvXszjjz/O5MmTSUhIwN/fn7lz59KxY0fmzJlD1apVCQ4OpkmTJixfvtzJ7yb/couKitL0P1Y0atSI6dOnU7euyejvv/9+2rZtS9++fW0+5+rVq1SpUoVFixZRs2ZNwNSAzJs3L9sJiLe3N+7uGqRMREREHO/w4cOEhIQ4OwynS7kfEhMTiY6OdnJEeYNqQGw4cuRIqi9eaGgoR44cyfA5ixcvJiwsLDn5SDJ48GAsFgv16tVj7NixySMwiIiIiOR3cQlwLBYuXIVEoLCHGbCgREFnRyY5Jd8mIB07duSff/6xWrZ27VrAjDmdJKNOT0nmzJlD//79Uy374YcfCAkJIS4ujnHjxjFkyBC+/vrrm4hc7OHCVVgUaU54ocWgazgU00RYIiIiDnUtAXZHmX+TRF8zj3Bv8NPcg3lSvk1Ali5dmmF5cHAwhw4dSq6tOHz4MG3btrW5/qFDh9i0aRMff/xxquVJtSienp4MGTKE+vXr31zgctOWHYJ7VkLUtevL/AvBt+2heVnnxSUiIpLfHL6UOvlI6WCMqQXxLODYmCTnqZOBDd27d+eDDz4AIDIykg0bNtCpUyeb68+dO5cuXbqk6qR+6dIloqKikv/+5ptviIiIyKmQJQt2nYM7lqVOPgBOX4EuS+GQmnaKiIg4hMUCUVczLr9wzXa55F75tgYkM8OHD+exxx6jTp06uLu7M3nyZEqWNONdz549m+PHj/Pcc88BpnnW559/njwTZ5LTp0/Tv39/EhJMah8WFsZ7773n2DciqUz9A67YuNNy8Rq8u9PM1iv5x94oeG07bD4Fnu7QOgjG1NEMzSIiOS3RYpKMjMRrqKQ8SaNguSiNgpUzGs+HX0/aLu8QAku7OC4eca6NJ6D99xATl3p5mDes6wEhxZwSloiI0zliFCyLBf44C3GJttepWAJ8vHI0jAxpFKycoStcyVeKZlLnV1Qd0fONuAToszJ98gFwMBoeWuPwkERE8hU3NyhTxHZ5oQJQ3A4jYfXq1YuyZcvi5uZGTEzMzb+g3DQlIJKvdA7LuLxLJuWSd6w7Docz+B1adhjOXnFcPLnJsUuw4wyc0/4RkbTOnIH27c2/WVCmsPWRrrwKmNoPd7f0Zdn1yCOPsH379pt/IbEb9QGRfGVIDZi3D347lb7s9mDoV9nxMYlznIjNfJ1Tl9UXJKX9F2DQGvj5mPm7gBv0rgjvNnduEwkRZ0m0wNJD5jthsUCTADOsu0d+vb175gz4+5v/+/vD6dOQydxnbm7/DbfrZQaISbRAEQ8o6QUF/tuPr7/+Ovv27WPmzJkAREVFUbFiRfbs2YOvr2+mYd1+++039bbE/pSASL5SyAN+7AbP/Aqf/APRceYk92A1eLlBPv7RyIfKFc+43MMdgoo6Jpbc4EQsNF8Ax1MkbgkW+HwvREbD6m5QUENlSj4SEwc9l8HKlHMU74CmAbC4k/ltyVdSJh9JspiEABQraB7WDB48mCpVqvDaa69RokQJZs2aRffu3Tlx4gStW7e2+pw6derw0UcfZfddiIOoE7qLUif0nGexmB+QYp7mDozkL4kWqPs17Dhrvfy+yvBJG8fG5MpGrIe3/7Rd/klruK+K4+IRcbYBq+DTPdbLuofDgo4ODcfustUJ3VrykVIWk5CMDB06lCpVqjB8+HAqVarE119/TZ06dbL1Gm5ubkRHR1OsWNZHGFEn9JyhGhDJt9zcwDubndtOxZq7XTFxcIuvqW5X8pI7ubvBN+3h9sWm03lKjcvA282cE5erWn0083IlIJJfnL4Mc/baLl8YCQcuQvlMalrzjL59My9fvvymNjF8+HB69OhBhQoVKFOmDHXq1GHXrl3ce++9VtdXDYhrUwIikkXTd8LjG+BaiuECmwWYGdRLZzCKh7iuiiVg+13w4d+p5wHpVzn3NSfaGwUvbIY1R03TqIZl4KX6UL+0fV4/IZO68szKRfKSfRdMLWpG9kTlowRk7tyMa0Dmzr3pTVStWpXw8HCGDBnCa6+9BkD16tXVuTyXUhsfkSz4Zj88ui518gGw/gR0WwoJGYxhLq7NxwtG14Z57WDO7fBAtdyXfPxxFhp8awZYOHkZzlyBJQeh6Xew4rB9ttGwzM2Vi+QlWenfka/6gPj5mWZW1tih+VWSwYMHEx8fT69evbL1vG7duhEcHAxAlSpVaNmypV3ikRunBEQkC1753XbZb6fSdEIUcSCLBQb+BBeupS+7lggDfoJrCTe/nafrmP5S1lQqAQPU/ErykSo+UKOk7fLQYlAvgwqBPMlaEmLH5ANg1apVDB06FE/P7E3atWjRIo4cOYLFYuHo0aOsWbPGbjHJjVECIpKJ+ET400ZH5STbsjbcuYjdHYyGrRkcfydi4ZeTN7+dyj6woouZJT6lRmXgx662kxORvMjNDT5qbf249yoAn7bJp6MqJiUh7drZNfk4duwYVatWZfv27YwcOdIurynOpT4gIplwdzN9A9I2v0rJK5c12ZG84/xV+6yTFY0DYN+98MsJM0dK+eJQ208DMUj+1KA0/NYTxm6GtcdNn5AmAabvVZ38VvuRkp/fTXc4T6ts2bLs3r3brq8pzqUERCQT7m7QNsS0qbelfRZHKnQ0iwUs2Gcm2YxcijM/wnP3mgvTct5m0seRNa9PJCU5o1xxkyDHZZAgV/Gx3/Y83KF5Wfu9nkhuVt0Xvm7v7ChEch9dGohkweTGUMLGkL2P14QamU/E6lAHo+HuFeD9IXjOhJrz4MsMhoy8GZfjofUimLLDNPdJtMD+izD6Fxi42iRBknN8vOCBqrbL24VAtQzaqouIiDiaEhCRLKhaEjbeAS1T3Pn1LwSvN4bJTZwXlzWRF+HWb+Hr/XAp3iQEf56De36EKdvtv703d8CmU9bLPtsDPxyy/zYltTeaQKfQ9MsblIY5mkxRRERcjJpgiWRRdV9Y3R2ir0FsPPgXzvmmTTfi8Y2mGZQ1T/0Kd1WAUG/r5TdiYWTG5Qv+hc5h9tuepFfEExZ3guWHYM2x6/OA9AgHT/VPEhERF6MERCSbvAtmfwZ1R4lPzLivSoLF1Eg8UsN+27xoZfjX7JTb065zsDsKfApC08CbHxwgqflYbuhk7e4GHcPMQ0RExJUpARHJQ64mZNwZGSAmzr7brF7SXPTb4oj+MWcuQ/9VsCzFpHtlCsPHraGDlaZJmdlyCp79zYxsY7GY0Z9evdUkNZKz4hJg6h9mQIMTsaaT/cPVzTwjuSERFMnPzl2B9cfhaiLU9YMKJZwdkfHbb7/x8MMPExsbS0hICHPmzCEwUCd0Z1IfEJE8pKgnhGfSvCqjybNuxOO1wNZ1oU9BGJRBB2l7iEuALj+kTj7AzAjebSn8ls05MNYeg2YLzOSSVxPM8Ms/H4OWi2CZ+rPkqPhE6L4MxvwKO86az/DXk2YwgxEbnB2diGTkrT8g6FPzHb57BVT8HO7/Ca7EOzcui8VC3759mTp1Knv27KFjx46MGjXKuUGJEhCRvGZMbdtltUqZUZHsqVkgfNQKCqY5m5QqBAs7QlAx+24vrcUHzWz01sQlwkubs/5aiRYYtMYkHmnFJ5qyODvMKi7WvfcXLLWR5L3zJ/x4xLHxiEjWzPobRm6AK2nOj5/8A4N/vvnXf/3113n44YeT/46KisLPz49z585l+twtW7bg5eVFy5YtAXj44YdZsGABcXF2bg4g2aImWCJ5zCM14HAMTNiWenmELyzqmDPzcgyoCm2C4dsD1+cB6VXBDBGbmdg4mLkLfjpqEob6/vBYBAQUydq2M6vh+DUbNSB/n4d9F2yXH7sEW06bJllif1/tz6R8H9we7JhYRCRrEhLhxQxu9MzZA8/XhSo3Ufs+ePBgqlSpwmuvvUaJEiWYNWsW3bt358SJE7Ru3drqc+rUqcNHH33EoUOHCAu73jnO29sbb29vjh8/TmjoDbTRFbtQAiKSx7i5wfhG0K+y6XAeG2+Sjy5hOTsiUnAxGFEze885ewXaLDLNbZIsPwwf/A0/doWIUpm/RoFM+gV4ZCPhykqHeUd2qs9vzl3JuPxsJuUi4njHYs3NmYxsPn1zCYiPjw933nknH3/8McOHD2fGjBl8/fXXVK9ene3bt2f6fLc0HcgsmqDK6ZSAiORR1X3Nw5U9ui518pHk1GXotRx29cm8xqZlUPranlTl2Zi1u7KPSVjibXTkd8P192luVrEE/HXednklH4eFIiJZ5JmFmzwedhhAYvjw4fTo0YMKFSpQpkwZ6tSpw65du7j33nutrp9UAxIaGkpkZGTy8ujoaKKjo9UJ3cmUgIiIU1y4aiZLtGXPBdh4AppnkkDcHgzdwmFRZPqy4gVh3K1Zj6lUITOr+Pu7rJf3qQghOdynJT8bFmF7XplCBeDBag4NR0SyoExhqFbSNGG1poAbtMjGjSBbqlatSnh4OEOGDOG1114DyFINSL169bhy5Qpr1qyhZcuWzJw5kx49euDp6XnzQckNUyd0Gz777DOaNGlCqVKleP/99zNcd8uWLTRr1ox69erRrVs3Tpw4kVy2f/9+2rVrR7169WjdujW7d+/O6dBFcoVjsabTd0YOxWT+Ou5u8GVbGFIjdUf4un7wc/fsV/tPbQp3lEu/vH0IzLwte68l2dMm2Oz/tBN8FvGAL9qaGhIRcS1ubvB2M9vNYZ+vB2WL2mdbgwcPJj4+nl69emX5Oe7u7syZM4cRI0ZQuXJllixZwpQpU+wTkNwwt6ioKDWEs+LPP/+kYMGCvPHGG9SrV4+HHnrI6noWi4W6devy9ttv07x5c9555x22b9/OrFmzAOjatSt9+vShb9++LFy4kGnTprFy5cpMt+/t7Y27u/JDybvOX4VSsyGjE9Ca7nBbNu6cXbgKkdGm83tosRufN8JigXXHzcMCNC4DrYM0D4Wj/HUOvthrhuEN94b+lSE0k+GlRcS+Dh8+TEhI1odNXHEYHlsHe/8byMPXC56tC6Nq2e/cOXToUAIDA3nhhRfs84JZkHI/JCYmEh0d7bBt52VKQDIxZMgQ6tSpYzMB2bp1K0OHDuXXX38FTNvCSpUqcfjwYaKioqhXrx4HDhzAw8MDi8VClSpVWLlyZaoRGaxRAiL5wZ3LYP6/1svKF4c99+TMqF0iIpKx7CYgYGq1D0Wb+ZPCvaGgnQY+OXbsGK1bt8bX15fly5fj7e24OxJKQHKG+oDcpLRfUG9vb4oVK8aJEyc4e/YsgYGBeHiY3ezm5kZwcDBHjhzJNAERyQ+mt4C/o9K3Hfb1gq/bKfkQEclN3N0gvLj9X7ds2bJqwp7H5NsEpGPHjvzzzz9Wy9auXUtwcNYHm89oeDcN/SZiW5ki8FtPmLYTVh0x84A0KA0ja5phfUVERCTvybcJyNKlS+3yOiEhIRw6dH3q3ujoaGJiYggICKBQoUIcO3aM+Pj45CZYR48ezVZyI5LXeReEZ+qah4iIiOR9auBwk2rXrs2VK1dYt24dAB9//DGdO3fG09MTf39/IiIimDdvHgCLFi0iNDRUza9ERETEpXl4eBATk4WhCPOwmJiY5Gb0Yl/qhG7DvHnz+N///kdUVBSenp4ULVqUL774glq1ajF79myOHz/Oc889B8CmTZt4/PHHuXLlCoGBgbz//vuULWuG7tm7dy9Dhw7l3LlzeHt7M2PGDKpVy3wwe3VCFxEREWdJTEzk5MmTxMfHOzsUp/Hw8KBMmTLJ12PqhG4/SkBclBIQEREREdehBMR+dIUrIiIiIiIOo4ZtLioxMdHZIYiIiIjIf3RtZj9KQFzUpUuXnB2CiIiIiIjdqQmWiIiIiIg4jBIQERERERFxGCUgIiIiIiLiMEpARERERETEYZSAiIiIiIiIwygBERERERERh1ECkofFxsYyaNAg6tSpQ7169Vi0aJHV9Xbv3k2zZs2SHxEREYSHhyeXR0RE0KBBg+Ty+fPnO+gdOF9W9yGAj48PTZo0Sd5PGzduTC7bv38/7dq1o169erRu3Zrdu3c7InyXkNV9ePz4cXr27En9+vVp0qQJ999/P+fPn08uz4/HYVaPm08//ZS6detSu3ZtRowYQXx8fHLZsmXLaNCgAXXq1KF///7ExMQ4KnyXkJV9+PPPP9OmTRsaNmxI48aNeeWVV7BYLAAcPHiQUqVKpTpH/vvvv45+G06Xlf24bt06AgMDU+2ry5cvJ5frWMx8H37xxRep9l/58uXp168foGMRYMyYMURERODj48OuXbtsrqdzoutzi4qKsjg7CMkZkyZNIjIykhkzZhAZGUm7du3YtGkTPj4+GT7vySefBOD1118HzIXfvHnzqF69ek6H7HKysw99fHw4cuQIxYoVS1fWtWtX+vTpQ9++fVm4cCHTpk1j5cqVDngHzpfVfXjq1Cn2799P48aNAXjhhRe4ePEib731FpA/j8OsHDeRkZF06NCBtWvX4u/vzz333EP79u0ZOHAgMTEx1KlThyVLllC5cmWefPJJihUrxtixY530jhwvK/twx44dlChRgvDwcK5cuUKPHj0YNGgQd911FwcPHqRVq1YcOHDASe/ANWRlP65bt44XXniBNWvWpHu+jsUb+x1o0qQJTz31FN27d9exCGzYsIHw8HA6dOhg8/dA58TcQTUgedh3333H4MGDAQgPD6dJkyYsWbIkw+dcvXqVr7/+mv79+zsiRJd3I/swrdOnT7Njxw569+4NQLdu3Th48CAHDx60e7yuKKv7sHTp0snJB0D9+vWJjIx0VJguJ6vHzaJFi+jSpQulS5fGzc2NBx54gG+++QaAH3/8kTp16lC5cmUABg0alFyWH2R1H9aqVSu51rdQoUJERETk62MvLXucw3QsZn8f/v7775w6dYpOnTo5KkyX17RpU4KCgjJcR+fE3EEJSB525MgRQkJCkv8ODQ3lyJEjGT5n8eLFhIWFUbNmzVTLBw8eTJMmTRg2bBhnzpzJkXhdUXb3YZcuXWjatCnPPvts8mz2R48eJTAwEA8PDwDc3NwIDg7O9LPIK27kOExISOCDDz6gQ4cOqZbnp+Mwq8fN4cOHbe5fa2XHjx8nMTHRAe/A+W7ku3fy5EkWLlxI27Ztk5dFR0fTqlUrWrRowaRJk0hISMjx2F1Jdvbjvn37aNGiBa1ateLDDz9MXq5jMfvH4meffUbv3r3x9PRMXpbfj8Ws0Dkxd/BwdgBy4zp27Mg///xjtWzt2rWAOcklSWrTnJE5c+akq/344YcfCAkJIS4ujnHjxjFkyBC+/vrrm4jcddhzH/7555+EhIRw6dIlHn/8cV588UWmTJmS7jUye53cxt7HocVi4YknnqBEiRI8/PDDycvz8nFoS1aPm4z2b9rXyG+y8927ePEiffr0Yfjw4dSuXRuAgIAAdu3ahb+/P+fPn2fgwIFMmzaNESNG5GTYLicr+7FWrVr89ddflChRgqNHj3LXXXdRqlQp7rjjDquvkd9k51iMjY1l/vz5rFixInmZjsWs0znR9SkBycWWLl2aYXlwcDCHDh3Cz88PMJl/yrt6aR06dIhNmzbx8ccfp1qedLfA09OTIUOGUL9+/ZsL3IXYcx8m7aeiRYvy4IMPMnLkSACCgoI4duwY8fHxeHh4YLFYOHr0KMHBwfZ7I05k7+NwzJgxHD16lLlz5+Lufr2SNi8fh9Zk9bgJCQnh0KFDyX8fPnw4eZ2QkBDWrVuXXHbo0CECAwNT7de8LDvfvejoaHr16kXHjh157LHHkpd7eXnh7+8PQMmSJenXrx9ff/11vrroy+p+LF68eKrn9OrVi40bN3LHHXfoWMzm78DChQupUqUKVatWTV6mYzFrdE7MHbTH87Du3bvzwQcfAKZT1oYNGzJsSzp37ly6dOmSqnPwpUuXiIqKSv77m2++ISIiIqdCdjlZ3YdRUVHExsYCkJiYyPz585P3k7+/f3IHajDtU0NDQwkLC3PQu3Cu7ByHY8aM4d9//2XOnDkULFgweXl+PA6zetx069aN77//nlOnTmGxWJg9ezZ33nknAG3atGHr1q3s2bMHgFmzZiWX5QdZ3YcxMTH06tWL1q1bM2bMmFRlp0+fJi4uDjB95BYvXpyuiWpel9X9eOLEieSmLNHR0Sxfvjx5X+lYzN7vgLXWCDoWs0bnxNxBo2DlYZcuXeKxxx5j+/btuLu78+KLL9K9e3cAZs+ezfHjx3nuuecAU0VZs2ZN3n33XVq0aJH8GpGRkfTv3z+5nWlYWBgTJ07MNxfPWd2HmzZtYuTIkbi5uZGQkEDNmjWZNGkSJUuWBGDv3r0MHTqUc+fO4e3tzYwZM6hWrZoz35rDZHUf/vrrr3To0IHKlSsnJx9hYWHMnTs33x6Hto6bYcOG0bFjx+RE7pNPPmHq1KkkJibSokUL3njjjeR24z/88ANjx44lPj6e6tWrM2PGjFR3qvO6rOzDyZMnM3HixFR3m3v06MHo0aNZtGgREyZMwN3dnYSEBJo3b864cePw8vJy4rtyvKzsx/fff5/Zs2dToEABEhIS6N69O08//XRykxcdi1n7Pv/77780b96cv//+G29v7+Tn61iE0aNH88MPP3Dy5ElKlSpF0aJF2bZtm86JuZASEBERERERcRg1wRIREREREYdRAiIiIiIiIg6jBERERERERBxGCYiIiIiIiDiMEhAREREREXEYJSAiIiIiIuIwSkBERERERMRhlICIiIiIiIjDKAERERG7eP/996lcuTKXL18GIDo6miZNmjBmzBgnRyYiIq5ECYiIiNjFgAED8PT05JNPPiExMZEHH3yQ4OBgJkyY4OzQRETEhXg4OwAREckbvLy8GDVqFFOmTGH//v0cOXKEZcuWUaBAAWeHJiIiLsQtKirK4uwgREQkb7h27RrVq1enQIECrFq1iuDg4OSyefPm8eGHHwLw/PPPc9tttzkrTBERcSLVgIiIiN18/fXXXLhwgZIlS1KqVKnk5RcuXGDq1Kn89NNPXL58mc6dO7N+/XrVjoiI5EPqAyIiInaxfv16nnrqKb755hsKFSrErFmzkst+//13GjZsSOHChfH19SU4OJh9+/Y5MVoREXEWJSAiInLTDhw4wH333cdrr73GbbfdxuOPP87bb79NbGwsAGfPnsXHxyd5fR8fH86ePeukaEVExJmUgIiIyE2Jiori7rvvZsCAAdx7770A9O3bF09Pz+RaEF9fX6KiopKfc+HCBXx9fZ0RroiIOJk6oYuISI6LioqiY8eOrF69msuXL9OpUyfWrVuHh4e6IoqI5Dc684uISI7z8fFh+PDhdOnSBTc3N8aPH6/kQ0Qkn1INiIiIiIiIOIz6gIiIiIiIiMMoAREREREREYdRAiIiIiIiIg6jBERERERERBxGCYiIiIiIiDiMEpBc6MqVK9x///2UL1+ehg0bOjucVFq0aMHPP//sktv+9ddfqVGjBgDffvstAwcOdFRokkLLli1ZsGCB1bIePXqwZs0ah8YjIiIijqVB2HOhDz/8kJiYGPbs2eNy4+ivXbvWZbf9999/c8stt5CYmMjLL7/M559/7qDIJElCQgK7d++matWqVstHjx7NM888w7p16xwcmYiIiDiKakByoVWrVtGtWzeXSz5c3a5du7jllltYsWIFJUuWTK4NcTXr1q2jc+fOzg4jW7Ia8969e0lMTKRixYpWy5s2bcqFCxf49ddf7R2iiIiIuAglILlIQkIC5cqVY/Xq1Tz99NMEBwcTFRXl0BiOHz/O/fffT6VKlQgODqZJkyYcPHgQgP+3d+dxVdX5H8ffF1FRwEuCiqihSKIouO9bamqSmuaGe2o2lqON2aTpWJnmOOaaa07llrtkYSmmhUtp7hFIpZa4K4iASrhx+f1xf96RQEWBcwVfz8fjPh5y7vd8z4fDMN0357ts2LBBzZo1s7VduHChatasKW9vb73//vtq166d1qxZY2vbuHFjvf/++6pcubJ8fHy0Zs0a7dmzRy1btlSZMmXUp08fpaam2vr79NNPVadOHZUrV05t27bVL7/8Ynvvr9eWpMWLF6t27dry9vbW7NmzbQFk06ZNGdpWqlRJERERkqQDBw7Izc1NISEhkqRTp07J29tbSUlJd70vU6dO1dNPP620NOu+nqGhoapSpYpiYmIe9BZnWV6sOSoqSr6+vpo/f778/Pzk6+urhQsX2t43mUxq1qyZNm3alGs1AAAA+yKA5CEFChTQnj175ODgoKNHj+r06dNyc3N76P5GjhypJ5988q6v3bt3Zzhn+PDhqlKlig4fPqyYmBjNmDFDnp6ekqSff/5ZgYGBkqQ5c+Zo8eLFWrVqlY4dO6ZTp05p//79qlatmq3tH3/8ocqVK+vw4cN67bXX9NZbb+mTTz7R2rVrtX//fu3YsUPff/+9JGnWrFn65JNPtGrVKp04cUItW7bUoEGDbHXdeW1JmjdvnhYsWKClS5fq2LFjio6O1r59+1StWjVFRkbqqaeeSvd9PfHEE7p69aok6cMPP5Sfn58t3P33v/9Vz549ZTab73ovX331VV24cEEbNmzQDz/8oDfeeEOrV69W+fLlH/CnknV5sebIyEgdP35cLi4u+vnnn/XRRx9pzJgxOn78uK2Nn5+fIiMjc60GAABgXwSQPCYiIkK+vr5ydna+b9sPPvjgnn9JnjZtmk6ePHnXV8OGDTOc8/vvv+vWrVu6ceOGHB0dVb9+fRUuXFjS/0LA5cuXNWXKFNuH4oIFC6p79+5ycHBQpUqVbG0HDhyoLl26qECBAqpWrZpu3bqlmTNnyt3dXaVLl5aHh4csFosSEhI0depU/fe//5Wvr68cHBzUq1cvRUdHKyUlJd21Jeny5cuaPHmy5s+fr6pVq6pgwYIKDg6Wo6OjKlasqMTERBUrVizd9+Xm5qarV68qJiZGkZGRCg4OVmJiolJSUrRixQoNGTLE1nbJkiVq06aNnnvuOdvTgqJFi2r06NF65513NHDgQH366afpAlFm52RXVmq+efOm2rZtqyeffFJffvlluvPvVfO9zsuOqKgode3aVQMGDFDhwoXVqlUrPfnkk4qOjra1KVasmOFP9gAAgHEIIHlMREREug+29xIVFSV/f/8cvf5HH32kiIgIVa1aVb169dKRI0ds70VGRiowMFA7d+5U6dKlVadOHdt7ly5dkp+fn23eSmRkpNq1a2d7Pzo6Wk8//bSKFi0qSfrzzz914sQJ+fv7a8+ePfLy8kr3vcTHx+uJJ55QkSJF0l1bss5H8PDwUM2aNW3t4+Li5O/vLwcHB7m5ueny5cvpvq/bH+bnzp2rv/3tb3Jzc1NiYqJWr16txo0b254KJCQkaOnSpdq4caMmTJigd99919ZH9erVdfz4cfXq1UtNmjSxHb/XObfd+TQqODhYP/74432fRmWlZkdHRy1ZskSvvPJKpj/Pu9V8v/MetuaoqCi98MIL6Y5dvHhRJUqUsH19+fLlbD3ZAwAAjzYCSB5z51/69+7dq3feeUeS1KVLF82ePVuS1KFDB12/fl1Hjx7VvHnz1KZNG40YMSJDXyNGjFCZMmXu+tq1a1eGc+rWras1a9bo8OHDKly4sP79739Lsn6IPH/+vKpVq6b4+PgMHyC//vpr2/Crixcv6ty5c+mCVEREhKpXr277+vDhw/Lw8FCpUqUy7W/Dhg22JzR3XluyhhMPD4907UNCQmzvBwQE6OjRo+neN5vNOnnypDZu3Kg+ffqoWLFiSkhI0MKFCzV06FBbu/3796tJkyZydHRUrVq1dOzYMUlSTEyMevXqpcGDB2v16tW2JzP3OudOdz6NWrVqlRo0aHDfp1FZqdlkMtmGyP3VvWq+13kPW3NcXJwuXLigkiVL2o59++23Klq0qGrVqmU79ttvvykgIOCe1wYAAHkXASSPufMJiNls1tWrV/XLL7+oaNGiSkpK0vbt29W4cWPdvHlT586d0/Dhw/XNN98oIiJCcXFx6fqaMWOGzpw5c9dXo0aN0rUPDQ3Vb7/9JovFopSUFF28eNEWGn7++Wf5+PjIxcVFVapUUUREhCIiInTt2jXNmzdPoaGh6eZ/VKhQId0wqL8GkDu/rlWrliIjIxUZGanr169r1apVWrx4scaOHZvh2pJ1DsFPP/2kgwcPKiUlRbNnz1ZYWJjt+s8++2yGZV7d3Nz00UcfKTg4WM7OznJ1ddWWLVvk7OysevXq2dolJiamC0MWi0WxsbHq3LmzRo8erQ8++EBeXl5asGDBPc/JCVmtOTP3qzk3REVFydHRUWvWrJHFYlFkZKRGjhyp9957L92Kbjt37lTbtm1ztRYAAGA/BJA8JDExUSdPnrT9ddhsNuvKlStauHChRo4cqcTERC1btkyDBg3S4cOH1bFjR5UpU0aSVKRIERUqVChb19+zZ486d+6ssmXLqkWLFqpfv76GDRsmKf2Tmbp162rIkCHq0KGDateurbS0NHl6eqp27dq2tneGjatXr+r3339Pd+zO/qpUqaKJEyeqZ8+eqlixoj777DOFhITYltH96wT0+vXra8iQIerUqZPq1aunU6dOqWTJkrYA0qZNG8XHx6ebd+Dm5qZLly7p5ZdflmSdh3DhwoV0Tz9ut7tzZank5GR16dJFvXv3Vr9+/SRJ48aN08yZM5WQkJDpOQ4OOfNrl9Wa/+ry5cv3rTk3REVFqX379kpKSlL58uU1cOBAjRkzRt27d7e12bVrl1xcXDKEXwAAkH+YEhMT0+xdBB7OtWvX1LFjR/n4+Gj27Nnq2LGjnnrqKX344Yf65JNPFB0drWnTpunw4cOaMWOGPv74Y7vUeeDAAXXr1k1Hjx5VgQIF7FLDX61bt05ff/21Fi1a9EDnJSQkqHv37goLC1NkZKRmzJihJUuW5Pg5Oe3f//63/P399fzzzxty3sN64YUXNGzYMLVo0cKQ6wEAAOMRQPI4T09PbdiwQXXr1pWnp6e2bNmigIAAjRgxQg4ODrp06ZJSU1M1bdq0dBN9c9P+/ftVvHhxVahQQRERERo8eLAGDx5s+0t9Xvfpp59q1apVcnR01Jw5c+Tj45Mr5+SUF198UYcOHZKzs7OeeeYZvffee7l6HgAAwL0QQJDjlixZovfee0/Xrl1TuXLlNHjwYA0cOFAmk8nepQEAAMDOCCAAAAAADMMkdAAAAACGIYAAAAAAMAwBBAAAAIBhCCAAAAAADEMAAQAAAGAYAggAAAAAwxBAAAAAABiGAAIAAADAMAQQAAAAAIYhgAAAAAAwDAEEAAAAgGEIIAAAAAAMQwABAAAAYBgCCAAAAADDEEAAAAAAGIYAAgAAAMAwBBAAAAAAhiGAAAAAADAMAQQAAACAYQggAAAAAAxDAAEAAABgGAIIAAAAAMMQQAAAAAAYhgACAAAAwDAEEAAAAACGIYAAAAAAMAwBBAAAAIBhCCAAAAAADEMAAQAAAGAYAggAAAAAwxBAAAAAABiGAAIAAADAMAQQAAAAAIYhgAAAAAAwDAEEAAAAgGEIIAAAAAAMQwABAAAAYBgCCAAAAADDEEAAAAAAGIYAAgAAAMAwBBAAAAAAhiGAAAAAADAMAQQAAACAYQggAAAAAAxDAAEAAABgGAIIAAAAAMMQQAAAAAAYhgACAAAAwDAEEAAAAACGIYAAAAAAMAwBBAAAAIBhCCAAAAAADEMAAQAAAGAYAggAAAAAwxBAAAAAABiGAJJDli9fLjc3N9urVKlSqlSpktq3b6/p06crLi7O3iUCAAAAdudo7wLym7lz56pSpUq6efOm4uLi9OOPP2rmzJmaPXu2Fi1apKefftreJQIAAAB2QwDJYf7+/qpZs6bt6+eff16vvvqq2rVrp759++rAgQMqWbKkYfWkpKSoSJEihl0PAAAAuBeGYBmgXLlymjhxoq5cuaJFixbZjh86dEjBwcEqX768SpUqpaZNm2r9+vUZzt+9e7dat26tUqVKqUqVKpo4caKWLl0qNzc3nThxwtYuICBAPXr0UGhoqJo2bapSpUrpP//5jyTpwoUL+sc//iF/f3+VKFFCgYGBmjx5sm7dupXuWjdu3NAHH3ygunXrqmTJkqpYsaJeffVVXbx4MZfuDgAAAB4nPAExSOvWrVWgQAHt2rVLkrRjxw517dpVtWvX1owZM1SsWDGFhIRowIAB+vPPP9W7d29JUlRUlDp37qyKFStq/vz5KlKkiBYtWqQ1a9Zkep2IiAj99ttveuONN+Tt7S1nZ2dduHBBrVq1kslk0ptvvqkKFSpo7969mjp1qk6ePKl58+ZJkiwWi3r16qXdu3dr+PDhql+/vk6ePKnJkyerffv2Cg8P52kKAAAAsoUAYhBnZ2e5u7vr/PnzkqQ33nhDlStX1oYNG+ToaP0xtGrVSvHx8ZowYYJ69uwpBwcHTZ06VQUKFFBoaKjc3d0lSW3btlWjRo0yvU5cXJz27NkjX19f27ERI0YoMTFRu3fvVrly5SRJzZs3l5OTk8aNG6fhw4ercuXKWr9+vbZu3aqlS5eqY8eOtvMDAgLUokULrVixQoMGDcqV+wMAAIDHA0OwDJSWliZJ+uOPP3TkyBF169ZNknTr1i3bq02bNjp//ryOHj0qSfrhhx/UtGlTW/iQJAcHB3Xq1CnTa1StWjVd+JCkzZs3q0mTJipdunS6a7Vu3dp2jdvtzGaz2rVrl65dQECASpUqpe+//z5H7wcAAAAePzwBMUhycrIuXbokf39/xcbGSpLGjRuncePGZdo+Pj5eknTp0qVMJ63fbSK7p6dnhmOxsbEKCwuTh4fHPa8VGxurpKQklShR4p7tAAAAgIdFADHIN998o9TUVDVp0sT2NOP1119Xhw4dMm1/+ylG8eLFbYHlThcuXMj0PJPJlOGYu7u7qlatetewczu0uLu7q3jx4goJCcm0nYuLS6bHAQAAgKwigBjg1KlTGjdunIoVK6YBAwbIw8NDFStWVFRUlN5+++17ntu4cWNt2bJF8fHxtuBisVj05ZdfZvn6bdu21ZYtW1ShQgW5ubnds11ISIhSU1NVp06dLPcPAAAAZBUBJIdFR0fb5k7ExcVp9+7dWr58uQoUKKDPPvvMNgxqxowZ6tatm1544QX16tVLpUuXVkJCgo4cOaKIiAgtWbJEkjRy5EiFhYWpY8eOeuONN+Tk5KRFixYpOTlZknU+yP2MGTNG4eHhatOmjf72t7/J19dX169f18mTJ7VlyxZNnz5dZcqUUZcuXbR27Vp169ZNQ4YMUe3atVWwYEGdOXNGO3fuVFBQ0F2f2AAAAABZYUpMTEyzdxH5wfLlyzV06FDb14UKFZLZbFalSpXUqlUr9evXL8McjKioKE2bNk3ff/+9EhMTVbx4cfn5+alz584aMGCArd3u3bs1btw4RUZGys3NTT169JC7u7veeecdnThxQmazWZJ1tSp/f3+tXr06Q33x8fGaMmWKwsLCdPbsWbm4uMjb21vPPPOMRowYIWdnZ0nWCfHz58/X6tWrdezYMTk6OsrLy0uNGzfWsGHD5OPjkxu3DwAAAI8JAkge1blzZ508eVIHDhywdykAAABAljEEKw8YM2aMAgMDVaZMGSUkJGjt2rUKDw/X7Nmz7V0aAAAA8EAIIHlAamqqJk2apNjYWJlMJvn5+emjjz5Sjx497F0aAAAA8EAYggUAAADAMOyEDgAAAMAwBBAAAAAAhmEOCJAD6m5w1qUb5HkjFC9k0b4OyfYuAwAAPCQCCJADLt1wUPx1AggAAMD9EEByiLOzc5Z2Jcejz2Kx2Haax+OB39/8g99fAHj0EUByiIODAx9ggDyK318AAIzDf3EBAAAAGIYAAgAAAMAwBBAAAAAAhiGAAAAAADAMAQQAAACAYQggAAAAAAxDAAEAAABgGAIIAAAAAMMQQHJBamqqxo8fr0uXLmX6/ueff67vvvsu29cZPny4ypcvL5PJpKioqGz3BwAAAOQ2dkLPBcuXL9e7776rGTNmqEGDBipUqJDtvbNnz+rgwYMqVKiQOnfurJUrVz70dbp27ao333xTTZo0yYmyAQAAgFzHE5Bc0LdvX3l7eyspKUlxcXFavHixQkND1b9/f0VERMjV1VXbtm1TcnJytq7TrFkzlS1bNoeqBgAAAHIfASQXmEwmBQQE6JVXXtHBgwfVunVrLV68WMHBwSpatKg2b96sBg0a2LtMAAAAwHAMwcolJpNJc+fOlSTNnz9fAwYMkKurK+EDAAAAjzWegOQik8mkli1bysHBepu9vLxUqVIlO1cFAAAA2A8BJBeFhISoZ8+ecnFxUefOnfXbb7+pdevWd10dCwAAAMjvCCC55OzZs+nmfISEhKSbE5ITIWTo0KEqW7asTp8+rWeeeUa+vr45UDkAAACQewgguWDTpk06ePBgugnnt+eE3BlCUlNTs3WduXPn6vTp07p165bOnz+vY8eO5dB3ALvZv1qaUFV6rYj0qkk69ZP01bvWf99p+zxp9+LsX+9Vk7X/B5V41nreqZ+yXwMAAHisMAk9F9SsWVPu7u6qXLmyJk2alO69tLQ0eXt76+rVqypevLidKsQj6UqctKSv5P+s1GOeVLCwVKqS1Pglqeqz6dvumCe5eEgNX7RLqUo6K20cL7mXl8rVsE8NAAAgTyKA5AJPT0/FxsbKZDJl+n5aWtpd38NjLPaIlHpTqtdHqtT8f8cLFZWeYL8XAACQPxBAcsm9AgbhAxksfVH6cYn135/0sL6eai6N2GYd6rRxvDQvzfr+v8pLl05Y/317aFZxb2lizN37T7ksfT5S+ulz6dZ1qWITqduHGdvFHpPC3pd+/15KPCMVfUIqV0t6fpJUJsDa5sg2aWYL67+XDbC+JCnoHan9u9KJ/dLWqdLxH6UrFyTXUpJPQ+n5yZK790PeIGP17dtXSUlJOdaf2WzWsmXLcqw/AADyMgII8ChoN07yrietHip1nCT5tZCcimXe9m/rpf92lYqYpeB51mOOhe/ed1qa9FEn6Y9dUtDbkndd6fcfpLntMrZNOis5u0udJksuJaTkS9KeJdKU+tKYQ1IpP2sg6bvIGjza/Uuq9pz1XLf/f0oTH2NtVztYci4uJZ2Tds6X/lNXejvaOnTsEZeUlKTQ0NAc669jx4451hcAAHkdAQR4FJSoKJX2t/675FNShXtsVlmuplSwiDWg3KvdbdGbpSPhUrdZUovh1mNVWkuOhaTQsenbPtXM+rrNkioFPGedGL/zI6nrdKlIMcmrmvV9j4oZa6jV1fpK10d7aXQpad+K/9XwmCtfvrycnJzk5OSklJQUDRgwQKNHj7Z3WRmcPXtWvXv3Vnh4uL1LAQDkEwQQIL878v8fHOv2Tn+8bq+MAST1lrRlirT3MynumHVOym3nf8na9a5dlTZNkH4KsT4Nsdyx2ltW+3hMrFu3TtWqVdPZs2fl7++vli1bql69etnu99atW3J0zJn/e/fy8iJ8AAByFMvwAvldcrzk4Ci5uKc/XswzY9uQ16UN46TqnaRXNkhv7pFG7ZPKVpdupmTteot6SdvnSI1ekv6+WXpzr7UPlxJZ7+Mx4+XlJT8/P504cULnz59X9+7dVa9ePQUGBurtt9+2tdu5c6cCAgIUGBioYcOGydvbW1FRUZKsT1Tef/99tWjRQv3799fNmzc1evRo1atXTzVq1FBwcLASExMlSR9//LH8/f1Vo0YNBQQEaM+ePbJYLPr73/+uypUrq3r16qpdu7auXbummJgYeXj8b9hcWFiYatWqpcDAQDVv3lzR0dGSpG3btqlGjRp69dVXVb16dVWtWlX79+837iYCAPIMnoAA+Z2zu2S5JV2NTx9CLp/P2HbvZ1L9ftZJ53e6elEq4nb/a6UkSVFfWSekt71jONHN69Kf2d98M7/69ddfdfHiRT399NPq06ePxo4dq2bNmunWrVtq37691q9fr6CgIPXs2VMrV65U06ZNtX79es2ZMyddPydPntR3330nk8mkSZMmycXFRXv37pUkTZgwQe+8845mzZqlkSNH6pdffpGXl5du3ryp69evKyIiQt9++62io6Pl4OCgpKQkFSpUKF3/sbGx6tOnj8LDwxUQEKDly5ere/futhB0+PBhffzxx5o3b54WLFigsWPHavPmzcbcRABAnkEAycOuXbum4OBgRUdHq2jRovL09NSCBQtUvnx5e5eG3OZYOOtPEyq1sA6r2rc8/fyLfSsytjWZMk5oj/zauiJWCd/015cyqcFknfT+1z52fZx+KBYkSV27dpXJZNJvv/2mGTNmqGjRovruu+904cIFW5urV6/q119/VcWKFVWkSBE1bdpUktS5c2e5ubml62/AgAG2Vfa++OILXb58WevWrZMk3bhxQxUrVpQktWzZUv369VOHDh3Url07VapUST4+Prp586YGDhyoFi1a6LnnnpODQ/qH5Hv27LE9NZGk3r17a+jQoTp37pwkyc/PT3Xq1JEkNWzYUFOnTs3hOwYAyA8IILkkK8t45sTSnC+//LLatWsnk8mkOXPm6OWXX9Y333yTrT6RB5QJkPavsu6c7uEjFXT63zK5f1WljeTbTFr/pnQ9WfKuY10Fa28m/9ur1l76cbHkWVkqEyidPCBt+eB/K1zdVqKidSL8vuWSZxWpsItk9pLcvKzX2vqBdbUr9/LS0e3Srk+y9gTlMXN7DsjWrVvVoUMHtWzZUiaTSfv27VPBggXTtY2IiLjvEt4uLi62f6elpWnevHlq2bJlhnaff/65Dhw4oG3btikoKEgTJ05UcHCwDh8+rO3btys8PFxvvfWWduzYkW4uyd32MLp9zMnJyXasQIECunXrVtZuBADgsUIAySVZWcYzu0tzOjk5KSgoyPZ1gwYNNHPmzGz1iTziufHW5W1XDJauXbn3PiAODtIrodK6161PQlJvSD6NpaEbpfGV07ftNksqUFDa/G/p+lXrkrsvfy5t+Ff6doWKSn0/lb4eL81uY52sfnsfkIErpLWvWQOP5Zb1WsO2SPOey407kS8888wzeuWVV/Svf/1LTZs21eTJkzVu3DhJ1lWoLBaLKleurOTkZP3www9q3LixvvzyS9ucjsx07NhR06dPV4MGDVS0aFH9+eefOn78uPz8/BQTE6M6deqoTp06unjxovbu3atWrVqpQIECatOmjVq3bq3t27crOjpagYGBtj4bNmyoQYMG6ZdfflGVKlW0atUqlS1bVp6envr1119z+zYBAPIJAkg+8uGHH6pDhw72LgMPq9LT/9ts8E7t37W+7uTuLQ17gLH1RcxS30+srzv99XpF3aQ+H2c8f8S2jMfqBFtff+VWRhq8LuPxe22UCI0bN06+vr4KDQ3V3LlzbcOcXFxctGDBApUtW1YrVqzQkCFDVKRIEbVo0UKlSpWS2WzOtL/Ro0dr/Pjxql+/vu0JxahRo+Tr66sBAwYoISFBjo6OKlGihBYtWqRTp05p8ODBunnzpiwWixo1aqR27drpzJkztj5LlCihZcuWqXfv3kpNTZWbm5vWrFmT+zcHAJCvmBITEzP5xIMH5erqmm68dMeOHbP0BCSnNjubNGmSNmzYoG+//VZFixbNkT4fVxaLRVeuXHmgcyqGuCr+OovKGcG9sEW/d3mwn8/9PMzv74PIqf6uXLkiV1dXSVJ4eLj69++vmJiYDHM1HmcP8/sLADAWT0DygalTp+rzzz/X1q1bCR9APhYSEqIZM2bIYrGocOHCWrlyJeEDAJDnEEDyuOnTp2vlypXaunVrhhVxAOQvL774ol588UV7lwEAQLYQQPKw06dPa+TIkfLx8VGLFi0kSYULF9aePXvsXBkAAACQOQJIHla2bFmlpTGFBwAAAHkHAQQA/sJsNmd7mey/9gcAAKwIILkkKx9g+FACPJqyu0EoAAC4OwJILuEDDAAAAJAR6zcCAAAAMAwBBAAAAIBhCCAAAAAADEMAAQAAAGAYAggAAAAAw7AKVh7Xpk0bnT9/Xg4ODnJ1ddXs2bNVo0YNe5cFAAAAZIoAkkv69u2rpKSke7Yxm83ZXq53zZo1cnNzkyR98cUXGjhwoA4ePJitPgEAAIDcQgDJJUlJSQoNDb1nm5zYafl2+Lh9TQcHRtUBAADg0UUAyQf69eun8PBwSVJYWJidqwEAAADujj+X5wNLly7VqVOnNHHiRP3zn/+0dzkAAADAXRFA8pH+/fsrPDxc8fHx9i4FyNOSk5Mfyb4AAMgPCCB52OXLl3X27Fnb1+vXr5e7u7uKFy9ux6qAvO3cuXPy8/PTlClTZLFYHrofi8WiKVOmyM/PT+fPn8/BCgEAyNuYA5KHJSUlqUuXLkpJSZGDg4NKlCihr776SiaTyd6lAXnW+fPnlZCQoFGjRmnatGmqUaOGChcu/EB9XL9+XT/99JNiY2Pl5OSkQYMG6euvv86ligEAyFsIIHlYuXLltHfvXnuXAeQrNWvWVJMmTeTo6KiNGzcqOjpaq1atUuPGjbN0/g8//KAePXooNjZWQUFBWrp0qQYMGJDLVQMAkHcQQIAcULzQww/VwYMx4l4XLlxYX3zxhaZNm6a33npLzZs316RJk/TGG2/cdalri8WiqVOnasyYMZKkKVOmaOTIkfdcGvvWrVuaNGmSVqxYoQIFCig1NVXNmjXTlClT0i2xfadt27bpxo0batOmTba/zwcRFBSk2bNnq2LFig91fvny5fXVV1+pWrVqOVwZACCvIYDkErPZfN99Psxms0HVILft68BE4/zGwcFB//znP9WoUSMFBwdr1KhR2r59u5YsWSIPD490bS9evKh+/fpp06ZNKlu2rFavXq1GjRrd9xqDBg3SpUuXtHv3bj3xxBOyWCwKCQnRpUuX7hlArl69algAuT0PZuPGjYZcDwCQ/xFAckl2dzgH8Gho3LixDh06pP79+2vjxo2qWbNmuiFZt4dcnTlzxjbkyt3d/b79Hjt2TGvXrtXJkyf1xBNPSLKGnm7duun8+fNq0aKFLl++rGvXrqlVq1aaNWuWIiIitGDBAlksFm3dulUvvPCC3n77bW3evFkTJkxQSkqKHB0d9cEHH6hZs2aSpLFjx2r16tVyd3dXs2bNFB4erv3790uyPqVZsmSJHBwcFBgYqHnz5slsNuvdd9/V77//ruTkZB07dkybNm1S48aNbU8wzpw5o9dee01HjhyRJD3//POaMGGCVqxYoVmzZunGjRtKS0vTpEmTFBQUlBs/FgBAHkYAAYD78PDw0IYNG9INyZo4caJMJpPGjh0rKWtDru508OBBPfXUUxmepkiSm5ubNmzYIBcXF6Wmpur5559XSEiIunbtqiFDhujq1auaOnWqJOmPP/7Q+PHjFRYWpmLFiunYsWNq3ry5YmJiFBYWpq+++koREREqUqSIunbtarvGpk2btGjRIu3evVtubm56+eWXNWbMGM2dO1eSFB4eroMHD6pkyZIZ6uvTp4+CgoK0bt06SVJcXJwkqW3bturZs6dMJpNiYmLUqFEjnThxQgULFnyAuw0AyO8IIACQBXcOyerWrZveeustSZKXl5fWrl2bpSFXWWWxWDRq1Ch9//33SktLU2xsrGrUqJEuQNwWFhamY8eO2Z543Hbq1CmFh4ere/fucnZ2lmTdK2jChAmSpK1bt6p37962oV6vvPKKgoODbee3b98+0/Bx9epV7dq1S1u2bLEdK1GihCTp+PHj6t27t06fPi1HR0ddvHhRJ06ckK+vb/ZuCAAgXyGAAEA2pKWlKS0t7YHPq1Wrlo4ePar4+PgMQ7amT5+u+Ph47dmzR05OTnr99dd17dq1u17/2Wef1dKlSzN9727Lcmf23p1fu7i4POi3pODgYE2dOlWdOnWSJBUvXvyudQMAHl9sRAgAWXB7Y8HmzZsrNjZW//nPfzR58mTFxsaqefPmD7xxoa+vr7p06aJBgwYpMTFRkjUULF26VAcOHJCnp6ecnJx04cIFrV271nZesWLFlJSUZPu6TZs2CgsLU1RUlO3Y7eW5W7RoobVr1+rPP/+UxWJJNzetdevWWrVqla5cuSJJWrhwoZ555pn71u3i4qImTZpoxowZtmO3h2AlJCSofPnykqTPPvtMCQkJWb4fAIDHB09AAOA+7rXKVZMmTe67StbdfPrpp5o4caLq168vR0dHpaWlqVmzZpo5c6a6deumGjVqqEyZMumCQefOnbVs2TLVqFHDNgn9s88+00svvaSUlBTduHFDtWrV0vLly9WxY0ft2rVL1atXl5eXlxo0aGALBe3atVNkZKQaNmwok8lkm4SeFcuWLdOwYcNUtWpVOTo6qlOnTho/frxmzZqlzp07q0yZMmrYsKGefPLJB7zTAIDHgSkxMfHBxw4gA1dX1yxPPsWjzWKx2P4qjMfDX39/O3bsqNDQUElZW+Xq4sWLtlWyypYtm2Hjwjv7M9qVK1fk6uoqi8Wil156SV5eXpo4caJdajECv78A8OjjCUgu6du3b7phEpkxm805slzv+PHj9e677yoyMpJNvoAc8iAbC2a2Stb9Ni40Sr9+/RQTE6OUlBTVqlVLb775pl3rAQCAAJJLkpKS7vsXz/ttVJgVBw8e1I8//shQByAHXb9+Xe3bt3+gjQXvtXGhPa1fv96u1wcA4K8YM5SHXb9+XUOHDtW8efPuutINgAdz6NAh7dixQ5s2bVJQUJB++umnB1pi9/bGhUFBQbaNC+/3NBQAgMcJASQPe/vtt9WnTx9VqFDB3qUA+Ubp0qUlWYdcbdiwIUu7mv/V7SFZU6ZMUVpamgoXLpzTZQIAkGcxBCuP2r17t/bt26fJkyfbuxQgX/H09FSnTp20c+dO7dy5M9v9BQYGPlSIAQAgvyKA5FHbt2/Xr7/+anv6cfr0abVt21Yff/yx2rVrZ+fqgLxt5cqV9i4BAIB8iyFYedTo0aN19uxZxcTEKCYmRmXLltXmzZsJHwAAAHikEUAAAAAAGIYhWPlETEyMvUsAAAAA7osAkkvMZvN99/kwm80GVQMAAAA8GkyJiYlp9i4iP3B1dbX7jsfIGRaLRVeuXLF3GTAQv7/5B7+/APDo47+4AAAAAAxDAAEAAABgGAIIAAAAAMMQQAAAAAAYhgACAAAAwDAEEAAAAACGYR+QXNK3b18lJSXds43ZbNayZcuydZ3y5cvLyclJTk5OkqS33npLPXr0yFafAAAAQG4hgOSSpKQkhYaG3rPN/TYqzKp169apWrVqOdIXAAAAkJsYggUAAADAMASQfKB3794KCAjQSy+9pLi4OHuXAwAAANwVASSP27FjhyIiInTw4EG5u7urf//+9i4JAAAAuCvmgORxTz75pCSpYMGC+sc//qFKlSrZuSIAAADg7gggeVhycrJu3rwpNzc3SdLKlStVs2ZN+xYF5EEWi8XeJSCH8LMEgEcfASQPu3Dhgrp06aLU1FSlpaXJx8dHS5cutXdZQJ6TnJxs7xIAAHhsEEDyMB8fHx06dMjeZQAAAABZRgDJJWaz+b77fJjNZoOqAQAAAB4NpsTExDR7F5EfuLq6ysGBRcXyA4vFoitXrti7DAAAgHyJT8wAAAAADEMAAQAAAGAYAggAAAAAwxBAAAAAABiGAAIAAADAMAQQAAAAAIYhgORx169f19///nc99dRTqlq1qvr06WPvkgAAAIC7YiPCXNK3b18lJSXds43ZbNayZcuydZ3Ro0fLwcFBR44ckclk0rlz57LVHwAAAJCbCCC5JCkpSaGhofdsc7+d0u8nOTlZixYt0unTp2UymSRJpUuXzlafAAAAQG5iCFYe9vvvv8vd3V0TJ05UnTp11LRpU3377bf2LgsAAAC4KwJIHnbz5k398ccf8vf31/79+zVnzhwFBwcrLi7O3qUBAAAAmSKA5GHe3t5ycHBQ7969JUnVq1dXhQoVdPjwYTtXBgAAAGSOAJKHeXh4qFWrVtq8ebMk6cSJEzp+/Lj8/PzsXBkAAACQOSah53ELFizQwIEDNWrUKBUoUEALFy5kIjoAAAAeWQSQPM7Hx0fbtm2zdxkAAABAlhBAconZbL7vMrtms9mgagAAAIBHgykxMTHN3kXkB66urnJwYEpNfmCxWHTlyhV7lwEAAJAv8YkZAAAAgGEIIAAAAAAMQwABAAAAYBgCCAAAAADDEEAAAAAAGIYAAgAAAMAwBBAAAAAAhiGAAAAAADAMO6HnEIvFYu8SkEP4WQIAAOQeAkgOSU5OtncJAAAAwCOPIVgAAAAADEMAAQAAAGAYAggAAAAAwxBAAAAAABiGAAIAAADAMAQQAAAAAIYhgAAAAAAwDAEEAAAAgGEIIAAAAAAMQwABAAAAYBgCCAAAAADDEEAAAAAAGIYAAgAAAMAw/wclqUacnKfDxAAAAABJRU5ErkJggg==",
"text/html": [
"\n",
"
\n",
"
\n",
" Figure\n",
"
\n",
" \n",
"
\n",
" "
],
"text/plain": [
"Canvas(footer_visible=False, header_visible=False, toolbar=Toolbar(toolitems=[('Home', 'Reset original view', …"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plt.close(\"all\")\n",
"display(output)\n",
"ofit = overfit_example(False)"
]
},
{
"cell_type": "markdown",
"id": "3a1ffc89",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"#### Optional Lab - 3.9 - Regularized Cost and Gradient"
]
},
{
"cell_type": "markdown",
"id": "20519af2",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"###### Goals\n",
"In this lab, you will:\n",
"- extend the previous linear and logistic cost functions with a regularization term.\n",
"- rerun the previous example of over-fitting with a regularization term added.\n"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "f50f471a",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"import numpy as np,sys,os\n",
"%matplotlib widget\n",
"import matplotlib.pyplot as plt\n",
"proj_path=f\"{os.environ['HOME']}/my_web/Machine-Learning-Andrew-Ng\"\n",
"os.chdir(f\"{proj_path}/source/source_files/Supervised_Machine_Learning_Regression_and_Classification/\")\n",
"sys.path.append(\"week3/C1W3A1\")\n",
"sys.path.append(\"week3/OptionalLabs\")\n",
"\n",
"from plt_overfit import overfit_example, output\n",
"from lab_utils_common import sigmoid\n",
"np.set_printoptions(precision=5)"
]
},
{
"cell_type": "markdown",
"id": "6fa743d1",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Adding regularization\n",
"\n",
"\n",
"\n",
"The slides above show the cost and gradient functions for both linear and logistic regression. Note:\n",
"- Cost\n",
" - The cost functions differ significantly between linear and logistic regression, but adding regularization to the equations is the same.\n",
"- Gradient\n",
" - The gradient functions for linear and logistic regression are very similar. They differ only in the implementation of $f_{wb}$."
]
},
{
"cell_type": "markdown",
"id": "49ea942b",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Cost functions with regularization\n",
"###### Cost function for regularized linear regression\n",
"\n",
"The equation for the cost function regularized linear regression is:\n",
"$$J(\\mathbf{w},b) = \\frac{1}{2m} \\sum\\limits_{i = 0}^{m-1} (f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) - y^{(i)})^2 + \\frac{\\lambda}{2m} \\sum_{j=0}^{n-1} w_j^2 \\tag{1}$$ \n",
"where:\n",
"$$ f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) = \\mathbf{w} \\cdot \\mathbf{x}^{(i)} + b \\tag{2} $$ \n",
"\n",
"\n",
"Compare this to the cost function without regularization (which you implemented in a previous lab), which is of the form:\n",
"\n",
"$$J(\\mathbf{w},b) = \\frac{1}{2m} \\sum\\limits_{i = 0}^{m-1} (f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) - y^{(i)})^2 $$ \n",
"\n",
"The difference is the regularization term, \n",
" $\\frac{\\lambda}{2m} \\sum_{j=0}^{n-1} w_j^2$ \n",
" \n",
"Including this term encourages gradient descent to minimize the size of the parameters. Note, in this example, the parameter $b$ is not regularized. This is standard practice.\n",
"\n",
"Below is an implementation of equations (1) and (2). Note that this uses a *standard pattern for this course*, a `for loop` over all `m` examples."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c6e3a2d3",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"def compute_cost_linear_reg(X, y, w, b, lambda_ = 1):\n",
" \"\"\"\n",
" Computes the cost over all examples\n",
" Args:\n",
" X (ndarray (m,n): Data, m examples with n features\n",
" y (ndarray (m,)): target values\n",
" w (ndarray (n,)): model parameters \n",
" b (scalar) : model parameter\n",
" lambda_ (scalar): Controls amount of regularization\n",
" Returns:\n",
" total_cost (scalar): cost \n",
" \"\"\"\n",
"\n",
" m = X.shape[0]\n",
" n = len(w)\n",
" cost = 0.\n",
" for i in range(m):\n",
" f_wb_i = np.dot(X[i], w) + b #(n,)(n,)=scalar, see np.dot\n",
" cost = cost + (f_wb_i - y[i])**2 #scalar \n",
" cost = cost / (2 * m) #scalar \n",
" \n",
" reg_cost = 0\n",
" for j in range(n):\n",
" reg_cost += (w[j]**2) #scalar\n",
" reg_cost = (lambda_/(2*m)) * reg_cost #scalar\n",
" \n",
" total_cost = cost + reg_cost #scalar\n",
" return total_cost #scalar"
]
},
{
"cell_type": "markdown",
"id": "f2fd183e",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Run the cell below to see it in action."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5281d9a1",
"metadata": {
"pycharm": {
"name": "#%%\n"
},
"tags": []
},
"outputs": [],
"source": [
"np.random.seed(1)\n",
"X_tmp = np.random.rand(5,6)\n",
"y_tmp = np.array([0,1,0,1,0])\n",
"w_tmp = np.random.rand(X_tmp.shape[1]).reshape(-1,)-0.5\n",
"b_tmp = 0.5\n",
"lambda_tmp = 0.7\n",
"cost_tmp = compute_cost_linear_reg(X_tmp, y_tmp, w_tmp, b_tmp, lambda_tmp)\n",
"\n",
"print(\"Regularized cost:\", cost_tmp)"
]
},
{
"cell_type": "markdown",
"id": "6a21f014",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"**Expected Output**:\n",
"
\n",
"
\n",
"
Regularized cost: 0.07917239320214275
\n",
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"id": "fda319e0",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Cost function for regularized logistic regression\n",
"For regularized **logistic** regression, the cost function is of the form\n",
"$$J(\\mathbf{w},b) = \\frac{1}{m} \\sum_{i=0}^{m-1} \\left[ -y^{(i)} \\log\\left(f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right) - \\left( 1 - y^{(i)}\\right) \\log \\left( 1 - f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right) \\right] + \\frac{\\lambda}{2m} \\sum_{j=0}^{n-1} w_j^2 \\tag{3}$$\n",
"where:\n",
"$$ f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) = sigmoid(\\mathbf{w} \\cdot \\mathbf{x}^{(i)} + b) \\tag{4} $$ \n",
"\n",
"Compare this to the cost function without regularization (which you implemented in a previous lab):\n",
"\n",
"$$ J(\\mathbf{w},b) = \\frac{1}{m}\\sum_{i=0}^{m-1} \\left[ (-y^{(i)} \\log\\left(f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right) - \\left( 1 - y^{(i)}\\right) \\log \\left( 1 - f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right)\\right] $$\n",
"\n",
"As was the case in linear regression above, the difference is the regularization term, which is \n",
" $\\frac{\\lambda}{2m} \\sum_{j=0}^{n-1} w_j^2$ \n",
"\n",
"Including this term encourages gradient descent to minimize the size of the parameters. Note, in this example, the parameter $b$ is not regularized. This is standard practice. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4641aca8",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"def compute_cost_logistic_reg(X, y, w, b, lambda_ = 1):\n",
" \"\"\"\n",
" Computes the cost over all examples\n",
" Args:\n",
" Args:\n",
" X (ndarray (m,n): Data, m examples with n features\n",
" y (ndarray (m,)): target values\n",
" w (ndarray (n,)): model parameters \n",
" b (scalar) : model parameter\n",
" lambda_ (scalar): Controls amount of regularization\n",
" Returns:\n",
" total_cost (scalar): cost \n",
" \"\"\"\n",
"\n",
" m,n = X.shape\n",
" cost = 0.\n",
" for i in range(m):\n",
" z_i = np.dot(X[i], w) + b #(n,)(n,)=scalar, see np.dot\n",
" f_wb_i = sigmoid(z_i) #scalar\n",
" cost += -y[i]*np.log(f_wb_i) - (1-y[i])*np.log(1-f_wb_i) #scalar\n",
" \n",
" cost = cost/m #scalar\n",
"\n",
" reg_cost = 0\n",
" for j in range(n):\n",
" reg_cost += (w[j]**2) #scalar\n",
" reg_cost = (lambda_/(2*m)) * reg_cost #scalar\n",
" \n",
" total_cost = cost + reg_cost #scalar\n",
" return total_cost #scalar"
]
},
{
"cell_type": "markdown",
"id": "ed8316cf",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Run the cell below to see it in action."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3e592cf7",
"metadata": {
"pycharm": {
"name": "#%%\n"
},
"tags": []
},
"outputs": [],
"source": [
"np.random.seed(1)\n",
"X_tmp = np.random.rand(5,6)\n",
"y_tmp = np.array([0,1,0,1,0])\n",
"w_tmp = np.random.rand(X_tmp.shape[1]).reshape(-1,)-0.5\n",
"b_tmp = 0.5\n",
"lambda_tmp = 0.7\n",
"cost_tmp = compute_cost_logistic_reg(X_tmp, y_tmp, w_tmp, b_tmp, lambda_tmp)\n",
"\n",
"print(\"Regularized cost:\", cost_tmp)"
]
},
{
"cell_type": "markdown",
"id": "9959f568",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"**Expected Output**:\n",
"
\n",
"
\n",
"
Regularized cost: 0.6850849138741673
\n",
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"id": "4531d5d5",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Gradient descent with regularization\n",
"The basic algorithm for running gradient descent does not change with regularization, it is:\n",
"$$\\begin{align*}\n",
"&\\text{repeat until convergence:} \\; \\lbrace \\\\\n",
"& \\; \\; \\;w_j = w_j - \\alpha \\frac{\\partial J(\\mathbf{w},b)}{\\partial w_j} \\tag{1} \\; & \\text{for j := 0..n-1} \\\\ \n",
"& \\; \\; \\; \\; \\;b = b - \\alpha \\frac{\\partial J(\\mathbf{w},b)}{\\partial b} \\\\\n",
"&\\rbrace\n",
"\\end{align*}$$\n",
"Where each iteration performs simultaneous updates on $w_j$ for all $j$.\n",
"\n",
"What changes with regularization is computing the gradients."
]
},
{
"cell_type": "markdown",
"id": "7936754c",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Computing the Gradient with regularization (both linear/logistic)\n",
"The gradient calculation for both linear and logistic regression are nearly identical, differing only in computation of $f_{\\mathbf{w}b}$.\n",
"$$\\begin{align*}\n",
"\\frac{\\partial J(\\mathbf{w},b)}{\\partial w_j} &= \\frac{1}{m} \\sum\\limits_{i = 0}^{m-1} (f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) - y^{(i)})x_{j}^{(i)} + \\frac{\\lambda}{m} w_j \\tag{2} \\\\\n",
"\\frac{\\partial J(\\mathbf{w},b)}{\\partial b} &= \\frac{1}{m} \\sum\\limits_{i = 0}^{m-1} (f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) - y^{(i)}) \\tag{3} \n",
"\\end{align*}$$\n",
"\n",
"* m is the number of training examples in the data set \n",
"* $f_{\\mathbf{w},b}(x^{(i)})$ is the model's prediction, while $y^{(i)}$ is the target\n",
"\n",
" \n",
"* For a **linear** regression model \n",
" $f_{\\mathbf{w},b}(x) = \\mathbf{w} \\cdot \\mathbf{x} + b$ \n",
"* For a **logistic** regression model \n",
" $z = \\mathbf{w} \\cdot \\mathbf{x} + b$ \n",
" $f_{\\mathbf{w},b}(x) = g(z)$ \n",
" where $g(z)$ is the sigmoid function: \n",
" $g(z) = \\frac{1}{1+e^{-z}}$ \n",
" \n",
"The term which adds regularization is the $\\frac{\\lambda}{m} w_j $."
]
},
{
"cell_type": "markdown",
"id": "dbc80d53",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Gradient function for regularized linear regression"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "56709277",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"def compute_gradient_linear_reg(X, y, w, b, lambda_): \n",
" \"\"\"\n",
" Computes the gradient for linear regression \n",
" Args:\n",
" X (ndarray (m,n): Data, m examples with n features\n",
" y (ndarray (m,)): target values\n",
" w (ndarray (n,)): model parameters \n",
" b (scalar) : model parameter\n",
" lambda_ (scalar): Controls amount of regularization\n",
" \n",
" Returns:\n",
" dj_dw (ndarray (n,)): The gradient of the cost w.r.t. the parameters w. \n",
" dj_db (scalar): The gradient of the cost w.r.t. the parameter b. \n",
" \"\"\"\n",
" m,n = X.shape #(number of examples, number of features)\n",
" dj_dw = np.zeros((n,))\n",
" dj_db = 0.\n",
"\n",
" for i in range(m): \n",
" err = (np.dot(X[i], w) + b) - y[i] \n",
" for j in range(n): \n",
" dj_dw[j] = dj_dw[j] + err * X[i, j] \n",
" dj_db = dj_db + err \n",
" dj_dw = dj_dw / m \n",
" dj_db = dj_db / m \n",
" \n",
" for j in range(n):\n",
" dj_dw[j] = dj_dw[j] + (lambda_/m) * w[j]\n",
"\n",
" return dj_db, dj_dw"
]
},
{
"cell_type": "markdown",
"id": "bacf0f56",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Run the cell below to see it in action."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6b729786",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"np.random.seed(1)\n",
"X_tmp = np.random.rand(5,3)\n",
"y_tmp = np.array([0,1,0,1,0])\n",
"w_tmp = np.random.rand(X_tmp.shape[1])\n",
"b_tmp = 0.5\n",
"lambda_tmp = 0.7\n",
"dj_db_tmp, dj_dw_tmp = compute_gradient_linear_reg(X_tmp, y_tmp, w_tmp, b_tmp, lambda_tmp)\n",
"\n",
"print(f\"dj_db: {dj_db_tmp}\", )\n",
"print(f\"Regularized dj_dw:\\n {dj_dw_tmp.tolist()}\", )"
]
},
{
"cell_type": "markdown",
"id": "f753293f",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"**Expected Output**\n",
"```\n",
"dj_db: 0.6648774569425726\n",
"Regularized dj_dw:\n",
" [0.29653214748822276, 0.4911679625918033, 0.21645877535865857]\n",
" ```"
]
},
{
"cell_type": "markdown",
"id": "fd9ea7e5",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Gradient function for regularized logistic regression"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "04553ad7",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"def compute_gradient_logistic_reg(X, y, w, b, lambda_): \n",
" \"\"\"\n",
" Computes the gradient for linear regression \n",
" \n",
" Args:\n",
" X (ndarray (m,n): Data, m examples with n features\n",
" y (ndarray (m,)): target values\n",
" w (ndarray (n,)): model parameters \n",
" b (scalar) : model parameter\n",
" lambda_ (scalar): Controls amount of regularization\n",
" Returns\n",
" dj_dw (ndarray Shape (n,)): The gradient of the cost w.r.t. the parameters w. \n",
" dj_db (scalar) : The gradient of the cost w.r.t. the parameter b. \n",
" \"\"\"\n",
" m,n = X.shape\n",
" dj_dw = np.zeros((n,)) #(n,)\n",
" dj_db = 0.0 #scalar\n",
"\n",
" for i in range(m):\n",
" f_wb_i = sigmoid(np.dot(X[i],w) + b) #(n,)(n,)=scalar\n",
" err_i = f_wb_i - y[i] #scalar\n",
" for j in range(n):\n",
" dj_dw[j] = dj_dw[j] + err_i * X[i,j] #scalar\n",
" dj_db = dj_db + err_i\n",
" dj_dw = dj_dw/m #(n,)\n",
" dj_db = dj_db/m #scalar\n",
"\n",
" for j in range(n):\n",
" dj_dw[j] = dj_dw[j] + (lambda_/m) * w[j]\n",
"\n",
" return dj_db, dj_dw \n"
]
},
{
"cell_type": "markdown",
"id": "b58a224a",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Run the cell below to see it in action."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7747ecab",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"np.random.seed(1)\n",
"X_tmp = np.random.rand(5,3)\n",
"y_tmp = np.array([0,1,0,1,0])\n",
"w_tmp = np.random.rand(X_tmp.shape[1])\n",
"b_tmp = 0.5\n",
"lambda_tmp = 0.7\n",
"dj_db_tmp, dj_dw_tmp = compute_gradient_logistic_reg(X_tmp, y_tmp, w_tmp, b_tmp, lambda_tmp)\n",
"\n",
"print(f\"dj_db: {dj_db_tmp}\", )\n",
"print(f\"Regularized dj_dw:\\n {dj_dw_tmp.tolist()}\", )"
]
},
{
"cell_type": "markdown",
"id": "00f3b856",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"**Expected Output**\n",
"```\n",
"dj_db: 0.341798994972791\n",
"Regularized dj_dw:\n",
" [0.17380012933994293, 0.32007507881566943, 0.10776313396851499]\n",
" ```"
]
},
{
"cell_type": "markdown",
"id": "442e5abd",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Rerun over-fitting example"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6464dc89",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"plt.close(\"all\")\n",
"display(output)\n",
"ofit = overfit_example(True)"
]
},
{
"cell_type": "markdown",
"id": "5817863b",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"In the plot above, try out regularization on the previous example. In particular:\n",
"- Categorical (logistic regression)\n",
" - set degree to 6, lambda to 0 (no regularization), fit the data\n",
" - now set lambda to 1 (increase regularization), fit the data, notice the difference.\n",
"- Regression (linear regression)\n",
" - try the same procedure."
]
},
{
"cell_type": "markdown",
"id": "8e73bb4f",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"You have:\n",
"- examples of cost and gradient routines with regularization added for both linear and logistic regression\n",
"- developed some intuition on how regularization can reduce over-fitting"
]
},
{
"cell_type": "markdown",
"id": "1c887ea3",
"metadata": {},
"source": [
"### Practice Quiz "
]
},
{
"cell_type": "markdown",
"id": "a79ab212",
"metadata": {},
"source": [
"#### Quiz-1 "
]
},
{
"cell_type": "markdown",
"id": "268b0a6d",
"metadata": {},
"source": [
"
\n",
""
]
},
{
"cell_type": "markdown",
"id": "f97644cb",
"metadata": {},
"source": [
"### Assignment W3: \n"
]
},
{
"cell_type": "markdown",
"id": "990eb3c7",
"metadata": {},
"source": [
"##### Logistic Regression\n",
"\n",
"In this exercise, you will implement logistic regression and apply it to two different datasets. \n",
"\n",
"\n",
"###### Outline\n",
"- [ 1 - Packages ](#1)\n",
"- [ 2 - Logistic Regression](#2)\n",
" - [ 2.1 Problem Statement](#2.1)\n",
" - [ 2.2 Loading and visualizing the data](#2.2)\n",
" - [ 2.3 Sigmoid function](#2.3)\n",
" - [ 2.4 Cost function for logistic regression](#2.4)\n",
" - [ 2.5 Gradient for logistic regression](#2.5)\n",
" - [ 2.6 Learning parameters using gradient descent ](#2.6)\n",
" - [ 2.7 Plotting the decision boundary](#2.7)\n",
" - [ 2.8 Evaluating logistic regression](#2.8)\n",
"- [ 3 - Regularized Logistic Regression](#3)\n",
" - [ 3.1 Problem Statement](#3.1)\n",
" - [ 3.2 Loading and visualizing the data](#3.2)\n",
" - [ 3.3 Feature mapping](#3.3)\n",
" - [ 3.4 Cost function for regularized logistic regression](#3.4)\n",
" - [ 3.5 Gradient for regularized logistic regression](#3.5)\n",
" - [ 3.6 Learning parameters using gradient descent](#3.6)\n",
" - [ 3.7 Plotting the decision boundary](#3.7)\n",
" - [ 3.8 Evaluating regularized logistic regression model](#3.8)\n"
]
},
{
"cell_type": "markdown",
"id": "c7d0ad3c",
"metadata": {},
"source": [
"#### 1 - Packages \n",
"\n",
"\n",
"First, let's run the cell below to import all the packages that you will need during this assignment.\n",
"- [numpy](www.numpy.org) is the fundamental package for scientific computing with Python.\n",
"- [matplotlib](http://matplotlib.org) is a famous library to plot graphs in Python.\n",
"- ``utils.py`` contains helper functions for this assignment. You do not need to modify code in this file."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "a5c62d8a",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'/home/amit/my_web/Machine-Learning-Andrew-Ng/source/source_files/Supervised_Machine_Learning_Regression_and_Classification'"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": []
},
{
"cell_type": "code",
"execution_count": 1,
"id": "5c2efd93",
"metadata": {},
"outputs": [],
"source": [
"import numpy as np,os,sys\n",
"import matplotlib.pyplot as plt\n",
"import subprocess,os\n",
"from pathlib import Path\n",
"home_path = str(Path.home())\n",
"proj_path=home_path+\"/my_web/Machine-Learning-Andrew-Ng/source/source_files/Supervised_Machine_Learning_Regression_and_Classification\"\n",
"sys.path.append(f\"{proj_path}/week3/C1W3A1\")\n",
"#os.chdir(proj_path)\n",
"from utils import *\n",
"import copy\n",
"import math\n",
"\n",
"%matplotlib inline"
]
},
{
"cell_type": "markdown",
"id": "2610c7f7",
"metadata": {},
"source": [
"#### 2 - Logistic Regression\n",
"\n",
"\n",
"In this part of the exercise, you will build a logistic regression model to predict whether a student gets admitted into a university.\n",
"\n",
"##### 2.1 Problem Statement\n",
"\n",
"\n",
"Suppose that you are the administrator of a university department and you want to determine each applicant’s chance of admission based on their results on two exams. \n",
"* You have historical data from previous applicants that you can use as a training set for logistic regression. \n",
"* For each training example, you have the applicant’s scores on two exams and the admissions decision. \n",
"* Your task is to build a classification model that estimates an applicant’s probability of admission based on the scores from those two exams. \n",
"\n",
"##### 2.2 Loading and visualizing the data\n",
"\n",
"\n",
"You will start by loading the dataset for this task. \n",
"- The `load_dataset()` function shown below loads the data into variables `X_train` and `y_train`\n",
" - `X_train` contains exam scores on two exams for a student\n",
" - `y_train` is the admission decision \n",
" - `y_train = 1` if the student was admitted \n",
" - `y_train = 0` if the student was not admitted \n",
" - Both `X_train` and `y_train` are numpy arrays.\n"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "15b6df01",
"metadata": {},
"outputs": [],
"source": [
"# load dataset\n",
"X_train, y_train = load_data(\"week3/C1W3A1/data/ex2data1.txt\")"
]
},
{
"cell_type": "markdown",
"id": "0b1f0d8a",
"metadata": {},
"source": [
"###### View the variables\n",
"Let's get more familiar with your dataset. \n",
"- A good place to start is to just print out each variable and see what it contains.\n",
"\n",
"The code below prints the first five values of `X_train` and the type of the variable."
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "701d523f",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"First five elements in X_train are:\n",
" [[34.62365962 78.02469282]\n",
" [30.28671077 43.89499752]\n",
" [35.84740877 72.90219803]\n",
" [60.18259939 86.3085521 ]\n",
" [79.03273605 75.34437644]]\n",
"Type of X_train: \n"
]
}
],
"source": [
"print(\"First five elements in X_train are:\\n\", X_train[:5])\n",
"print(\"Type of X_train:\",type(X_train))"
]
},
{
"cell_type": "markdown",
"id": "78850227",
"metadata": {},
"source": [
"Now print the first five values of `y_train`"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "e93bae7c",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"First five elements in y_train are:\n",
" [0. 0. 0. 1. 1.]\n",
"Type of y_train: \n"
]
}
],
"source": [
"print(\"First five elements in y_train are:\\n\", y_train[:5])\n",
"print(\"Type of y_train:\",type(y_train))"
]
},
{
"cell_type": "markdown",
"id": "4f7d0260",
"metadata": {},
"source": [
"###### Check the dimensions of your variables\n",
"\n",
"Another useful way to get familiar with your data is to view its dimensions. Let's print the shape of `X_train` and `y_train` and see how many training examples we have in our dataset."
]
},
{
"cell_type": "code",
"execution_count": 19,
"id": "9a2a991c",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"The shape of X_train is: (100, 2)\n",
"The shape of y_train is: (100,)\n",
"We have m = 100 training examples\n"
]
}
],
"source": [
"print ('The shape of X_train is: ' + str(X_train.shape))\n",
"print ('The shape of y_train is: ' + str(y_train.shape))\n",
"print ('We have m = %d training examples' % (len(y_train)))"
]
},
{
"cell_type": "markdown",
"id": "588956c1",
"metadata": {},
"source": [
"###### \n",
"Visualize your data\n",
"\n",
"Before starting to implement any learning algorithm, it is always good to visualize the data if possible.\n",
"- The code below displays the data on a 2D plot (as shown below), where the axes are the two exam scores, and the positive and negative examples are shown with different markers.\n",
"- We use a helper function in the ``utils.py`` file to generate this plot. \n",
"\n",
"\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 20,
"id": "f49c2b82",
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAjsAAAGwCAYAAABPSaTdAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAA9hAAAPYQGoP6dpAABbUElEQVR4nO3de1xUdf4/8NcB4QSCgIIwKAmKV7xj66ULWmllpeW2llZqtiymW5CVZlYMZXjpl6vdkUwt16xvarvtbt5KSHMpBa9oKIqKOggpF1MaFD6/P9w5MVxkBuZy5szr+XjMI+ecw8znnAbOez6f9+f9kYQQAkREREQa5eHsBhARERHZE4MdIiIi0jQGO0RERKRpDHaIiIhI0xjsEBERkaYx2CEiIiJNY7BDREREmtbK2Q1Qg5qaGpw9exb+/v6QJMnZzSEiIiILCCFw8eJFhIeHw8Oj8f4bBjsAzp49i4iICGc3g4iIiJqhsLAQHTt2bHQ/gx0A/v7+AK5drDZt2ji5NURERGSJiooKREREKPfxxjDYAZShqzZt2jDYISIicjFNpaAwQZmIiIg0jcEOERERaRqDHSIiItI05uwQEZHDVVdX48qVK85uBqmcl5cXPD09W/w6DHaIiMhhhBAoKipCWVmZs5tCLiIwMBBhYWEtqoPn1GDn+++/x5tvvons7GwYDAZs2LABDzzwgLJfCIGUlBQsW7YMpaWlGDx4MN577z3ExMQoxxiNRjz//PP47LPPUFlZiTvuuAPvv//+defbExGRc5gCnfbt28PX15eFXKlRQghcvnwZxcXFAACdTtfs13JqsHPp0iX069cPTzzxBP74xz/W279o0SIsXrwYK1euRLdu3TBv3jyMHDkSeXl5ypz6pKQkfP3111i7di3atWuH5557Dvfddx+ys7Nt0vVFRES2UV1drQQ67dq1c3ZzyAX4+PgAAIqLi9G+fftm39edGuzcc889uOeeexrcJ4TAkiVLMHfuXIwbNw4AsGrVKoSGhmLNmjVISEhAeXk5li9fjk8//RR33nknAGD16tWIiIjA1q1bcddddzX42kajEUajUXleUVFh4zMjIqK6TDk6vr6+Tm4JuRLT5+XKlSvNDnZUOxuroKAARUVFGDVqlLJNlmXExcVh586dAIDs7GxcuXLF7Jjw8HD07t1bOaYh8+fPR0BAgPLgUhFERI7DoSuyhi0+L6oNdoqKigAAoaGhZttDQ0OVfUVFRfD29kZQUFCjxzRkzpw5KC8vVx6FhYU2br1jCFGN0tIMnDv3GUpLMyBEtbObREREpDqqn41VN6ITQjQZ5TV1jCzLkGXZJu1zlpKS9cjPT4TReFrZJssdER29FCEh45zYMiIiInVRbc9OWFgYANTroSkuLlZ6e8LCwlBVVYXS0tJGj9GikpL1yM19yCzQAQCj8Qxycx9CScl6J7WMiIhM9Ho9+vfv36LXOHHiBCRJwt69e23SJksMHz4cSUlJDns/R1BtsBMVFYWwsDBs2bJF2VZVVYXMzEwMGzYMABAbGwsvLy+zYwwGAw4ePKgcozVCVCM/PxGAaGgvACA/P4lDWkSkaQaDAXq9HgaDwaHvu3PnTnh6euLuu+92yPtFRETAYDCgd+/eAICMjAxIklSvTpEWAxRbcmqw8+uvv2Lv3r1KxFpQUIC9e/fi1KlTkCQJSUlJSE1NxYYNG3Dw4EFMmTIFvr6+mDhxIgAgICAATz75JJ577jl8++232LNnDx577DH06dNHmZ2lNWVl2+v16JgTMBoLUVa23WFtIiJyNIPBgJSUFIcHOx9//DGefvpp7NixA6dOnbL7+3l6eiIsLAytWqk+60TVnBrs7N69GwMGDMCAAQMAADNnzsSAAQPw6quvAgBmzZqFpKQkTJ8+HYMGDcKZM2ewefNmpcYOAPztb3/DAw88gPHjx+Pmm2+Gr68vvv76a83W2KmqsuwX29LjiIjIMpcuXcIXX3yBp556Cvfddx9Wrlxptn/BggUIDQ2Fv78/nnzySfz2229m+6dMmYIHHngAqampCA0NRWBgIFJSUnD16lW88MILaNu2LTp27IiPP/5Y+Znaw1gnTpzAiBEjAABBQUGQJAlTpkzBlClTkJmZiaVLl0KSJEiShBMnTgAADh06hNGjR8PPzw+hoaF4/PHH8csvv5id06RJk+Dn5wedToe33nrLPhfPyZwa7AwfPhxCiHoP0wdIkiSlm/K3335DZmam0pVncsMNN+Cdd97B+fPncfnyZXz99deankru7W1ZBUlLjyMichUGgwE5OTnKA4DZc3v38nz++efo3r07unfvjsceewwrVqyAENfSB7744gskJyfjjTfewO7du6HT6fD+++/Xe43vvvsOZ8+exffff4/FixdDr9fjvvvuQ1BQEH788UdMmzYN06ZNa3CWcEREBNatWwcAyMvLg8FgwNKlS7F06VIMHToU8fHxMBgMMBgMyvBXXFwc+vfvj927d2Pjxo04d+4cxo8fr7zmCy+8gG3btmHDhg3YvHkzMjIykJ2dbacr6ESCRHl5uQAgysvLnd2UJtXUXBU7d3YU27ZJYts2NPCQxM6dEaKm5qqzm1rP2bNnRXJysjh79qyzm0JkN/ycN66yslIcOnRIVFZWNuvnk5OTBa4lJzb4SE5Otm2D6xg2bJhYsmSJEEKIK1euiODgYLFlyxYhhBBDhw4V06ZNMzt+8ODBol+/fsrzyZMni06dOonq6mplW/fu3cWtt96qPL969apo3bq1+Oyzz4QQQhQUFAgAYs+ePUIIIbZt2yYAiNLSUrP3iouLE4mJiWbbXnnlFTFq1CizbYWFhQKAyMvLExcvXhTe3t5i7dq1yv7z588LHx+feq/lTNf73Fh6/1ZtgjI1TJI8ER291PSs7l4AQHT0EkiS+obxnDXGTuRI/JzbT0JCArKzs5GdnY309HQAQHp6urItISHBbu+dl5eHn376CY888ggAoFWrVnj44YeVIafDhw9j6NChZj9T9zkAxMTEwMPj91tvaGgo+vTpozz39PREu3btlPWgWiI7Oxvbtm2Dn5+f8ujRowcA4NixYzh27BiqqqrM2tm2bVt07969xe+tNsx4ckEhIeMQE/NlI3V2lrDODhFpkk6nq7cY5MCBAzFw4EC7v/fy5ctx9epVdOjQQdkmhICXl1e98ifX4+XlZfZckqQGt9XU1LSswQBqampw//33Y+HChfX26XQ6HD16tMXv4SoY7LiokJBxCA4ei7Ky7aiqMsDbW4fAwFtV16NjGj8GYDbGbtLQHy8iV8PPubZdvXoVn3zyCd566y2z5YkA4I9//CP+/ve/o2fPnsjKysKkSZOUfVlZWTZvi7e3N4Bri6rW3V5328CBA7Fu3TpERkY2OJsrOjoaXl5eyMrKwo033ggAKC0txZEjRxAXF2fztjsTgx0XJkmeCAoa7uxmXFdaWhpSUlLMtsXHxyv/Tk5Ohl6vd3CrSO0MBgPS0tKQkJDgEkECP+eOp9PpkJyc7JDPx7/+9S+UlpbiySefREBAgNm+hx56CMuXL8eLL76IyZMnY9CgQbjlllvw97//Hbm5uejcubNN29KpUydIkoR//etfGD16NHx8fODn54fIyEj8+OOPOHHiBPz8/NC2bVvMmDED6enpmDBhAl544QUEBwcjPz8fa9euRXp6Ovz8/PDkk0/ihRdeQLt27RAaGoq5c+eaDbNphfbOiFTFmWPs5LpcLe/FnT/nzirup9PpoNfrHRLsLF++HHfeeWe9QAe41rOzd+9edO3aFa+++ipmz56N2NhYnDx5Ek899ZTN29KhQwekpKTgxRdfRGhoKP76178CAJ5//nl4enqiV69eCAkJwalTpxAeHo4ffvgB1dXVuOuuu9C7d28kJiYiICBACWjefPNN3HbbbRgzZgzuvPNO3HLLLYiNjbV5u51NEkI0VIrXrVRUVCAgIADl5eVo06aNs5ujWTk5OYiNjUV2drZDxthJ/RrrwXHlz4ort705rDnf3377DQUFBYiKisINN9zgoBaSq7ve58bS+zeHsYjIaUw9OGPGjFGeA8x7qc3VhvSI1IjBDjmMI8fYyfVoJe/F1p/z2gGhWn53mJBNrobBjooJUa362VbWMI2xk3tr7EY5dOhQrF69GsHBwSgsLER8fDzS09OVoRFXuXm6w+dcK4EpuQ8GOypVUrK+kTo6S1lHh1yaJTdK07CWo2qoqI3ae04SEhKU/0c5OTkuG5iS+2Cwo0IlJeuRm/sQrlVA/53ReAa5uQ8hJuZLBjzksiy5UbrKLCx7UXvPiTOL+xE1B4MdlRGiGvn5iagb6PxvLwAJ+flJCA4e69JDWuS+LL1RunN+F3tOiGyLwY7KlJVtNxu6qk/AaCxEWdl21RcUJGoud8h7uR5X6jnhxANyBQx2VKaqyrLue0uPI1Iz3ihdn7sHpuQaWEFZZby9Lfujb+lxRGrmyCq4rooBIVlq5cqVCAwMbPHrSJKEr776qsWvY6kpU6bggQcesOt7MNhRmcDAWyHLHQFIjRwhQZYjEBh4qyObRUROwoCwPiGqUVqagXPnPkNpaQaEqG76h1pgypQpkCQJCxYsMNv+1VdfQZIa+1vdsMjISCxZssSGrbM9g8GAe+65BwBw4sQJSJKEvXv3mh3jiADFlhjsqIwkeSI6eqnpWd29AIDo6CVMTiYit1RSsh5ZWZHYt28EDh+eiH37RiArKxIlJevt+r433HADFi5ciNLSUru+jxqEhYVBlmVnN8OmGOyoUEjIOMTEfAlZ7mC2XZY7cto5EbktU1mOupM4TGU57Bnw3HnnnQgLC8P8+fOve9y6desQExMDWZYRGRmJt956S9k3fPhwnDx5Es8++ywkSbpur9DixYvRp08ftG7dGhEREZg+fTp+/fVXs2NWrlyJG2+8Eb6+vnjwwQdx/vx5s/16vR79+/fHxx9/jBtvvBF+fn546qmnUF1djUWLFiEsLAzt27fHG2+8YfZztYexoqKiAAADBgyAJEkYPnw49Ho9Vq1ahX/84x/KeWRkZAAAzpw5g4cffhhBQUFo164dxo4dixMnTiivXV1djZkzZyIwMBDt2rXDrFmz4IglOhnsqFRIyDgMGXIC/fptQ8+ea9Cv3zYMGVLAQIeI3FLTZTmA/Pwkuw1peXp6IjU1Fe+88w5On254xmx2djbGjx+PRx55BAcOHIBer8crr7yClStXAgDWr1+Pjh074rXXXjMrHNkQDw8PvP322zh48CBWrVqF7777DrNmzVL2//jjj5g6dSqmT5+OvXv3YsSIEZg3b1691zl27Bi++eYbbNy4EZ999hk+/vhj3HvvvTh9+jQyMzOxcOFCvPzyy8jKymqwHT/99BMAYOvWrTAYDFi/fj2ef/55jB8/HnfffbdyHsOGDcPly5cxYsQI+Pn54fvvv8eOHTvg5+eHu+++G1VVVQCAt956Cx9//DGWL1+OHTt24MKFC9iwYYNF/w9agrOxVEySPDm9nIgI6ijL8eCDD6J///5ITk7G8uXL6+1fvHgx7rjjDrzyyisAgG7duuHQoUN48803MWXKFLRt2xaenp7w9/dHWFjYdd8rKSlJ+XdUVBRef/11PPXUU3j//fcBAEuXLsVdd92FF198UXmvnTt3YuPGjWavU1NTg48//hj+/v7o1asXRowYgby8PPznP/+Bh4cHunfvjoULFyIjIwNDhgyp146QkBAAQLt27cza7OPjA6PRaLZt9erV8PDwwEcffaT0Wq1YsQKBgYHIyMjAqFGjsGTJEsyZMwd//OMfAQAffvghNm3adN1rYQvs2SEiItVTS1mOhQsXYtWqVTh06FC9fYcPH8bNN99stu3mm2/G0aNHUV1tXY/Ttm3bMHLkSHTo0AH+/v6YNGkSzp8/j0uXLinvNXToULOfqfscuJYQ7e/vrzwPDQ1Fr1694OHhYbatuLjYqvY1JDs7G/n5+fD394efnx/8/PzQtm1b/Pbbbzh27BjKy8thMBjM2tmqVSsMGjSoxe/dFAY7RESkemopy3HbbbfhrrvuwksvvVRvnxCiXh5Oc/JRTp48idGjR6N3795Yt24dsrOz8d577wEArly5YtXrenl5mT2XJKnBbTU1NVa3s66amhrExsZi7969Zo8jR45g4sSJLX79lmCwQ0TkYAaDAXq93u3XALOGmspyLFiwAF9//TV27txptr1Xr17YsWOH2badO3eiW7du8PS8NoPW29u7yV6e3bt34+rVq3jrrbcwZMgQdOvWDWfPnq33XnXzbBrLu2kJb29vAKjX5obOY+DAgTh69Cjat2+P6Ohos0dAQAACAgKg0+nM2nn16lVkZ2fbvN11MdghInIwg8GAlJQUBjtWUFNZjj59+uDRRx/FO++8Y7b9ueeew7fffovXX38dR44cwapVq/Duu+/i+eefV46JjIzE999/jzNnzuCXX35p8PW7dOmCq1ev4p133sHx48fx6aef4sMPPzQ75plnnsHGjRuxaNEiHDlyBO+++269fB1baN++PXx8fLBx40acO3cO5eXlynns378feXl5+OWXX3DlyhU8+uijCA4OxtixY7F9+3YUFBQgMzMTiYmJSlJ3YmIiFixYgA0bNuDnn3/G9OnTUVZWZvN218Vgh4iIXIKaynK8/vrr9YaSBg4ciC+++AJr165F79698eqrr+K1117DlClTlGNee+01nDhxAl26dFGSf+vq378/Fi9ejIULF6J37974+9//Xm/K+5AhQ/DRRx/hnXfeQf/+/bF582a8/PLLNj/PVq1a4e2330ZaWhrCw8MxduxYAEB8fDy6d++OQYMGISQkBD/88AN8fX3x/fff48Ybb8S4cePQs2dPTJ06FZWVlWjTpg2AawHhpEmTMGXKFAwdOhT+/v548MEHbd7uuiThiAnuKldRUYGAgACUl5cr/0OIiGyp9lTjxlYy13qV5N9++w0FBQWIiorCDTfc0OzXEaIaZWXbUVVlgLe3DoGBt7LQqoZd73Nj6f2bU8+JiBwgLS0NKSkpZtvi4+OVfycnJ2t+Qc3i4mKUlZWhqqqqRcEOy3KQtRjsEBE5QEJCAsaMGQOg8Z4drSspKUF5eTmuXr3q7KaQm2GwQ0TkAA0NUw0cOFAJdojIfhjsEBGR3dTOVTp06BBCQkLw22+/KcXxvLy8lOnNRPbC2VhERA6m0+mQnJzsFkNXaWlpiI2NRWxsLF5++WUIIXD27FkcPnwYhw8fbnT6NZGJLeZRcTYWOBuLiMheavfs7N27F+3atUPfvn3Rvn17AOzZoaadP38excXFZsUZTTgbi4iInK5urtJ7772H9u3bw9fXF23atEFNTQ1+++03J7aQ1EoIgcuXL6O4uBiBgYH1Ah1rMNghIiKHWbFiBQDgr3/9Ky5cuODk1pArCAwMbHKV+KYw2CEiciMGgwFpaWlISEhweM6QTqdDfHw8li1bhunTpyMqKsqh70+ux8vLq0U9OiYMdoiI3IhpXa4xY8Y4LNipnbdz0003YdmyZdizZ4+yQrg7VI8m52KwQ0REdsXq0eRsqp96fvHiRSQlJaFTp07w8fHBsGHDsGvXLmW/EAJ6vR7h4eHw8fHB8OHDkZub68QWExGpi8FgQE5OjvIAYPbc3quvJyQkIDs7G9nZ2UhPTwcApKenK9sSEhLs+v5Equ/Z+fOf/4yDBw/i008/RXh4OFavXo0777wThw4dQocOHbBo0SIsXrwYK1euRLdu3TBv3jyMHDkSeXl58Pf3d3bziYicztk9K6weTc6m6jo7lZWV8Pf3xz/+8Q/ce++9yvb+/fvjvvvuw+uvv47w8HAkJSVh9uzZAACj0YjQ0FAsXLjQ4m8LrLNDRFpkSkZ+4IEHUFNTA8D5K67n5OQgNjYW2dnZDHaoxSy9f6t6GOvq1auorq6utzquj48PduzYgYKCAhQVFWHUqFHKPlmWERcXh507dzb6ukajERUVFWYPIldmMBig1+vtPhxBrsWUjFxTU6P0pJgCjNrPHZkc7E7Vo0k9VB3s+Pv7Y+jQoXj99ddx9uxZVFdXY/Xq1fjxxx9hMBhQVFQEAAgNDTX7udDQUGVfQ+bPn4+AgADlERERYdfzILI3002NwQ6pnU6ng16vZ7BDDqX6nJ1PP/0UU6dORYcOHeDp6YmBAwdi4sSJSpIdAGX6ookQot622ubMmYOZM2cqzysqKhjwEJEm1J7mXTsZ2cTDw4M9K+R2VB/sdOnSBZmZmbh06RIqKiqg0+nw8MMPIyoqSqmoWFRUZPaLW1xcXK+3pzZZliHLst3bTmRPTd3UWLvEPTk7GZlaxplFH7VM1cNYtbVu3Ro6nQ6lpaXYtGkTxo4dqwQ8W7ZsUY6rqqpCZmYmhg0b5sTWkhaoPQ+m9mrSpptZfHy8si0tLc3JLSRn4DRv18YhaftQfc/Opk2bIIRA9+7dkZ+fjxdeeAHdu3fHE088AUmSkJSUhNTUVHTt2hVdu3ZFamoqfH19MXHiRGc3nVycMyrNWiMhIQFjxowB0PgMG3I/nOZNVJ/qg53y8nLMmTMHp0+fRtu2bfHHP/4Rb7zxBry8vAAAs2bNQmVlJaZPn47S0lIMHjwYmzdvZo0d0jze1Ii0gUPS9qf6YGf8+PEYP358o/slSYJer+cYNNkE/+iQlnCat2tgnpX9qbqooKOwqCCZ6PX6en90alPrHx0mNRK5rrpfspxZ9NHVWHr/ZrADBjv0O/7RIXfDQNnxrnfNWWHaOpbev1U/jEXkSMyDIXej9kR8LeI1dzyXmXpO7kXt076JiOyBeVb2wZ4dUiU1fPPhHx3SKibiO56l19y0nAbZFnt2iBrBNXxIq1iQ0vG0cs1dtdedPTukGvy2SeQYLEjpeFq55mrodW8OBjukGqw1oV2c8aMuTMR3PF5z52KwQ6qhlW8+VJ+rfhskIm30ujPYIdXgNx8i+2qoh42J+I7natdcC73uDHaIyC608G2wKa42PNdQDxtn/zieq11zLfS6M9ghVXK1bz5Unxa+DTaFw3PkDrTQ685gh1TJ1b75UH1a+DaoBe7Qw0bUFAY7RGQXWvg22BBXCx7coYeNHMdVe90Z7BARWcFVggdTPtEDDzzAHjayGVftdWewQ0R256rfBhviKsNztfOJ6vamaaGHjcgaDHaIyO5c9dtgQ7Q6PEekZQx2iIg0oql8Ig8PD830sBFZg8EOEVEzqW14zlXyiYgcTRJCCGc3wtkqKioQEBCA8vJytGnTxtnNISJqlro9Ow3lE6klMCOyBUvv3+zZISLSCOYTETXMw9kNICIiIrInBjtERBqktnwiImdizg6Ys0NEROSKLL1/s2eHiIiINI3BDhEREWkagx0iIiLSNAY7REREpGkMdoiIiEjTGOwQERGRpjHYISIiIk1jsENERESaxmCHiIiINI3BDhFphsFggF6vV1b+JiICGOwQkYYYDAakpKQw2CEiMwx2iIiISNNaObsBREQtYTAYlJ6cnJwcs/8C11b/5srfRO6NwQ4RubS0tDSkpKSYbYuPj1f+nZycDL1e7+BWEZGaqHoY6+rVq3j55ZcRFRUFHx8fdO7cGa+99hpqamqUY4QQ0Ov1CA8Ph4+PD4YPH47c3FwntpqIHCkhIQHZ2dnIzs5Geno6ACA9PV3ZlpCQ4OQWEpGzqbpnZ+HChfjwww+xatUqxMTEYPfu3XjiiScQEBCAxMREAMCiRYuwePFirFy5Et26dcO8efMwcuRI5OXlwd/f38lnQET21tAw1cCBAzFw4EAntYiI1EbVPTv//e9/MXbsWNx7772IjIzEQw89hFGjRmH37t0ArvXqLFmyBHPnzsW4cePQu3dvrFq1CpcvX8aaNWuc3HoiIiJSA1UHO7fccgu+/fZbHDlyBACwb98+7NixA6NHjwYAFBQUoKioCKNGjVJ+RpZlxMXFYefOnY2+rtFoREVFhdmDiFyfTqdDcnIyE5KJyIyqh7Fmz56N8vJy9OjRA56enqiursYbb7yBCRMmAACKiooAAKGhoWY/FxoaipMnTzb6uvPnz6+X0EjqIEQ1ysq2o6rKAG9vHQIDb4UkeTq7WeQidDodk5GJqB5VBzuff/45Vq9ejTVr1iAmJgZ79+5FUlISwsPDMXnyZOU4SZLMfk4IUW9bbXPmzMHMmTOV5xUVFYiIiLD9CZBVSkrWIz8/EUbjaWWbLHdEdPRShISMc2LLiIjIlak62HnhhRfw4osv4pFHHgEA9OnTBydPnsT8+fMxefJkhIWFAbjWw1O727q4uLheb09tsixDlmX7Np6sUlKyHrm5DwEQZtuNxjPIzX0IMTFfMuAhIqJmUXXOzuXLl+HhYd5ET09PZep5VFQUwsLCsGXLFmV/VVUVMjMzMWzYMIe2lZpPiGrk5yeibqDzv70AgPz8JAhR7dB2ETkD1/cisj1VBzv3338/3njjDfz73//GiRMnsGHDBixevBgPPvgggGvDV0lJSUhNTcWGDRtw8OBBTJkyBb6+vpg4caKTW0+WKivbbjZ0VZ+A0ViIsrLtDmsTkbNwfS8i21P1MNY777yDV155BdOnT0dxcTHCw8ORkJCAV199VTlm1qxZqKysxPTp01FaWorBgwdj8+bNbltjxxUTfKuqLPujbulx5J4MBgPS0tKQkJDA2VhEZEYSQjQ0duBWKioqEBAQgPLycrRp08bZzWk2V03wLS3NwL59I5o8rl+/bQgKGm739pBrysnJQWxsLLKzs12uoGDd9b3i4+ORnp6unAfX9yJqmKX3b1UPY5HlTAm+dYeDTAm+JSXrndSypgUG3gpZ7gigsRl0EmQ5AoGBtzqyWUQOk5aWhtjYWMTGxirresXHxyvb0tLSnNxCItem6mEsskzTCb4S8vOTEBw8VpVDWpLkiejopf+bjSXB/DyuBUDR0UtU2XZyLq2seJ6QkIAxY8YAaLxnh4iaj8GOBliT4KvWYaCQkHGIifmykWG4JaoehiPn0cqK51zfi8i+GOxogFYSfENCxiE4eKzLJViT87BHhMgcE/UbxmBHA7y9LftAW3qcM0mSp2p7n0h9tNgjwvW9qCVMpQvGjBnDz1AtDHY0wJTgazSeQcN5OxJkuSMTfIlcANf3IrI9BjsawARfIvaIaBWHZZqmlUR9e2KdHWi9zk6EQxN8XbGoIRGplyvXT3IUvV5fL1G/NldJ1G8OS+/f7NnREGcn+LpqUUMiIlfGRP2mMdjRGGcl+HLV8qZptTteq+dFzsNhGetoMVHf1lhBmVqMq5ZbRqsLPGr1vMh5WFGabI09Oxrg7DwZLRQ1JCL14LBM8zFRv2EMdlycGvJktFLU0B602h2v1fOilrHVkCaHZZqPpQsaxmEsF6aWxT+1VNTQ1rTaHa/V86KW4ZAmqRV7dlyUmhb/ZFHDxmm1O16r50Xqw2EZsgUGOy5KTXkyLGrYOK12x2v1vMh69h7S5LAM2QKHsVyU2vJkTKuWy3IHs+2y3JHTzok0jEOa5ArYs+Oi1Jgn4+yihmqn1e54rZ4XWYZDmuQKuFwEXHO5CCGqkZUV2WSezJAhBQw2iMghGlvagYUnyV4svX9zGMtFmfJk/ves7l4A7psnQ0Tqwlla5GwMdlwY82SISE04pElqxZwdF8c8GSJSi9ozp1h4ktSEwY4GOGvxTyKixqSlpSElJcVsm2m2FgAkJydzSjk5TLODnfz8fBw7dgy33XYbfHx8IISAJNXNHSG1cfY6WuTemKjqPjhLi9TE6mDn/PnzePjhh/Hdd99BkiQcPXoUnTt3xp///GcEBgbirbfeskc7yQbUsI4WuTdTouqYMWPMbnYMgrSHhSdJTaxOUH722WfRqlUrnDp1Cr6+vsr2hx9+GBs3brRp48h21LKOFlFDOFuHiOzJ6p6dzZs3Y9OmTejYsaPZ9q5du+LkyZM2axjZjprW0SL3Y0miKmkbZ2m5NzX03Fod7Fy6dMmsR8fkl19+gSzLNmkU2Zaa1tEi99NUoupf/vIX3HTTTQA4W0eruL6Ve2ts+NqRrB7Guu222/DJJ58ozyVJQk1NDd58802MGDHCpo0j21DbOlrkXhISEpCdnY3s7Gykp6cDANLT0/GXv/wFALBs2TKuqUREdmV1z86bb76J4cOHY/fu3aiqqsKsWbOQm5uLCxcu4IcffrBHG6mF1LiOFrmPxhJV7733XiQkJADgbB0irVFbnSWrg51evXph//79+OCDD+Dp6YlLly5h3LhxmDFjBv84qVRg4K2Q5Y5NrqMVGHiro5tGboyzdYi0S211lqwKdq5cuYJRo0Y1eBKkXqZ1tHJzH8K1dbNqBzz2XUeLdX2oNiaqErkHtdVZsirY8fLywsGDB1k80AWZ1tFquM7OErvU2WFdH6qrsURVBkFE2qK2nltJCNHQuEajnnvuOXh5eWHBggX2apPDWbpEvBY4qqfFVNen/rDZtUCZC5USEbmHnJwcxMbGIjs72+bBjqX3b6tzdqqqqvDRRx9hy5YtGDRoEFq3bm22f/Hixda3lhzGEetosa4PERGZqKHn1upg5+DBg0pkduTIEbN9HN4igHV9iIjod2qos2R1sLNt2zZ7tIM0hHV9iLRBDZVviWzB6qKCtZ0+fRpnzpyxVVtII1jXh0gbuGYZaYXVwU5NTQ1ee+01BAQEoFOnTrjxxhsRGBiI119/HTU1NTZvYGRkJCRJqveYMWMGAEAIAb1ej/DwcPj4+GD48OHIzc21eTvIcqa6PqZk5PokyHIE6/pQkwwGA/R6PW+2RNQiVgc7c+fOxbvvvosFCxZgz549yMnJQWpqKt555x288sorNm/grl27lEqMBoMBW7ZsAQD86U9/AgAsWrQIixcvxrvvvotdu3YhLCwMI0eOxMWLF23eFrKMqa7P/57V3QvAfnV9SFvYs2C9lgaIBoMBOTk5ygOA2XP+v2gcg3MVE1bS6XTiH//4R73tX331lQgPD7f25ayWmJgounTpImpqakRNTY0ICwsTCxYsUPb/9ttvIiAgQHz44YcWv2Z5ebkAIMrLy+3RZLdVXLxO7NzZUWzbBuWxc2eEKC5e5+ymkYvIzs4WAER2drazm+IyWnrNkpOTBa5Nm2zwkZycbNsGawg/r45n6f3b6gTlCxcuoEePHvW29+jRAxcuXGh20GWJqqoqrF69GjNnzoQkSTh+/DiKioowatQo5RhZlhEXF4edO3cq6+7UZTQaYTQalecVFRV2bbe7CgkZh+Dgsayg3AzunBiqtjV13I3aKt8S2YLVwU6/fv3w7rvv4u233zbb/u6776Jfv342a1hDvvrqK5SVlWHKlCkAgKKiIgBAaGio2XGhoaE4efJko68zf/58LnfhII6o66NFpuGbMWPGuN3NRW1r6rgCWwaIaqt8q3YMzl2D1cHOokWLcO+992Lr1q0YOnQoJEnCzp07UVhYiP/85z/2aKNi+fLluOeeexAeHm62vW59HyHEdWv+zJkzBzNnzlSeV1RUICIiwraNJaJmYc+C9RwZILpzr2NDGJy7BquDnbi4OOTl5eH999/Hzz//DCEExo0bh+nTp9cLQmzp5MmT2Lp1K9avX69sCwsLA3Cth6f2L11xcXG93p7aZFmGLMt2aytRc/Ab4jXsWbCevQLEhirfunOvY0PcKTi3JtBVXVDskAwiG0hOThZhYWHiypUryjZTgvLChQuVbUajkQnK5JKYGFofEz6tZ+9rxv8njdP6tbHm/Bx1LeyWoLxixQr4+fkpU79N/u///g+XL1/G5MmTmx95NaKmpgYrVqzA5MmT0arV702WJAlJSUlITU1F165d0bVrV6SmpsLX1xcTJ060eTuI7MmdviFaSg1r6hB7Hcn1WR3sLFiwAB9++GG97e3bt8df/vIXuwQ7W7duxalTpzB16tR6+2bNmoXKykpMnz4dpaWlGDx4MDZv3gx/f3+bt4PInjh8U58a1tRxNfYIEJmXYhktBufWBLpqDoolIURDS1M36oYbbsDPP/+MyMhIs+0nTpxAz549UVlZacv2OYSlS8QTOUpOTg5iY2ORnZ3t1sEOqUPdm1hDvY5ausHT7/R6/XVnL9cOdK051lYsvX9b3bPTvn177N+/v16ws2/fPrRr187qhhJRfVr8hkiui72O7sua4XU1D8VbHew88sgjeOaZZ+Dv74/bbrsNAJCZmYnExEQ88sgjNm8gkTvi8A0RqYE1ga6ag2Krg5158+bh5MmTuOOOO5Rk4ZqaGkyaNAmpqak2byAREakHex3JFVmds2Ny9OhR7N27Fz4+PujTpw86depk67Y5DHN2iIiIrk+NdXYsvX83O9gxqa6uxoEDB9CpUycEBQW15KWchsEOERGR67H0/u1h7QsnJSVh+fLlAK4FOnFxcRg4cCAiIiKQkZHR7AYT2YoQ1SgtzcC5c5+htDQDQlQ7u0lEROREVgc7X375pbLg59dff43jx4/j559/RlJSEubOnWvzBhJZo6RkPbKyIrFv3wgcPjwR+/aNQFZWJEpK1jf9w0R1GAwG6PV6Zdo1Ebkmq4OdX375RVmT6j//+Q/Gjx+Pbt264cknn8SBAwds3kAiS5WUrEdu7kMwGk+bbTcazyA39yEGPGQ10zpQDHaIXJvVwU5oaCgOHTqE6upqbNy4EXfeeScA4PLly/D09LR5A4ksIUQ18vMTcW0ZqXp7AQD5+Ukc0iIickNWTz1/4oknMH78eOh0OkiShJEjRwIAfvzxR/To0cPmDSSyRFnZ9no9OuYEjMZClJVtR1DQcAe1ilyRmkveE1HzWB3s6PV69O7dG4WFhfjTn/4EWZYBAJ6ennjxxRdt3kAiS1RVWTbMYOlx5L64DhTZkqOmYNP1tXjquRZw6rnrKy3NwL59I5o8rl+/bezZoeviOlBkS85c584dAi27rY1FpEaBgbdCljvCaDyDhvN2JMhyRwQG3uroppGLUXPJeyJrmBLsx4wZo9lgx1IMdqjZhKhGWdl2VFUZ4O2tQ2DgrZAk5ySpS5InoqOXIjf3IQASzAMeCQAQHb3Eae0jIvfBvC/1YbBDzVJSsh75+YlmScGy3BHR0UsREjLOKW0KCRmHmJgvG2nXEqe1Sy3coUvb1rgOFDWHM/O+GGg1jDk7YM6OtUz1bOoPF13rQYmJ+dKpgYWaepzUxJm5A0TuxJl5X3q9vl6gVZvWEuyZs0N20XQ9Gwn5+UkIDh7r1CEtJiETkbM4M+8rISEBY8aMAdB4oOWOrAp23n//faxfvx5t27bFtGnTcPvttyv7fvnlF/zhD3/A8ePHbd5IUg/Ws3Et7NImci9MsG+YxRWU3377bbzwwgvo0aMHZFnG6NGjMX/+fGV/dXU1Tp48aZdGknqwno1rSUtLQ2xsLGJjY5Wcgfj4eGVbWlqak1tIpG3M+1IHi3t20tLSkJ6ejokTJwIApk+fjgceeACVlZV47bXX7NZAUhdvb8t+YS09juyLXdpEzqXT6ZyWI8NA63cWBzsFBQUYNmyY8nzo0KH47rvvcMcdd+DKlStISkqyR/tIZVjPxrWwS5vIfTkz0FIbi4Od4OBgFBYWIjIyUtkWExOD7777DrfffjvOnDljj/aRyrCeDRERuRqLc3ZuueUWrFu3rt72Xr164dtvv8XGjRtt2jBSL1M9G1nuYLZdljs6fdq5FghRjdLSDJw79xlKSzNstlI7u7SJyF1ZXGdn//79yM7OxhNPPNHg/tzcXHz55ZdITk62aQMdgXV2mof1bGxPjcUaiYjUytL7N4sKgsEOqYPaizUSEamNpfdvi4exiMh+mi7WCOTnJ9lsSIuIyJ0w2CFSAWuKNRIRkXUY7BCpAIs1EhHZD4MdIhVgsUYiIvthsEOkAqZijaZk5PokyHIEizUSETWD1cHO+fPnMWPGDPTq1QvBwcFo27at2YOIrGcq1vi/Z3X3AmCxRiKi5rJq1XMAeOyxx3Ds2DE8+eSTCA0NhSQ19k2UiKxhKtbYcJ2dJZx2TkTUTFbX2fH398eOHTvQr18/e7XJ4Vhnh9SExRqJiCxjtzo7PXr0QGVlZYsaR0SNkyRPBAUNR/v24wEAxcVf2HTZCLo+g8EAvV4Pg4Ez34iaS22/R1YHO++//z7mzp2LzMxMnD9/HhUVFWYPImq5kpL1yMqKxL59I3D48ETs2zcCWVmRKClZ7+ymaZ7BYEBKSopq/kgTuSK1/R5ZnbMTGBiI8vJy3H777WbbhRCQJAnV1fz26Swc/tCGxpaNMBrPIDf3IS4bQURkJauDnUcffRTe3t5Ys2YNE5RVhAtIakPTy0ZIyM9PQnDwWAayNmQwGJRvoDk5OWb/Ba6tGM/V4omuT82/R1YnKPv6+mLPnj3o3r27vdrkcK6eoMwFJLWjtDQD+/aNaPK4fv22IShouN3b4y70ej1SUlIa3Z+cnAy9Xu+4BhG5IGf8Hll6/7a6Z2fQoEEoLCzUVLDjytgToD4tGU7kshHOkZCQgDFjxgC49k00Pj4e6enpGDhwIACwV4fIAmr+PbI62Hn66aeRmJiIF154AX369IGXl5fZ/r59+9qscQBw5swZzJ49G9988w0qKyvRrVs3LF++HLGxsQCu5QqlpKRg2bJlKC0txeDBg/Hee+8hJibGpu1QK2sWkGRPgP21dDiRy0Y4R0Pd6wMHDlT+SBNR09T8e2R1sPPwww8DAKZOnapskyTJLgnKpaWluPnmmzFixAh88803aN++PY4dO4bAwEDlmEWLFmHx4sVYuXIlunXrhnnz5mHkyJHIy8uDv7+/zdqiVuwJUA9bJBablo0wGs/Ue51rJMhyRy4bQURkBauDnYKCAnu0o0ELFy5EREQEVqxYoWyLjIxU/i2EwJIlSzB37lyMG3ftJrJq1SqEhoZizZo1SEhIaPB1jUYjjEaj8tyVp8yzJ0AdbDWcaFo24lrQJNV5PccuG2EwGJCWloaEhAS3GsbR6XRITk52q3MmsjW1/R5ZnaDsSL169cJdd92F06dPIzMzEx06dMD06dMRHx8PADh+/Di6dOmCnJwcDBgwQPm5sWPHIjAwEKtWrWrwdRtLonLFBGUhqpGVFdlkT8CQIQXM2bEjWycWNzwcFuHQZSNycnIQGxuL7OxsVXRDExHVZbcEZZNDhw7h1KlTqKqqMttuSk6yhePHj+ODDz7AzJkz8dJLL+Gnn37CM888A1mWMWnSJBQVFQEAQkNDzX4uNDQUJ0+ebPR158yZg5kzZyrPKyoqEBERYbN2O5KaegLcma2HE0NCxiE4eCzrJhER2YDVwc7x48fx4IMP4sCBA0quDgCl3o4tc3ZqamowaNAgpKamAgAGDBiA3NxcfPDBB5g0aZJyXN1aP6b8ocbIsgxZlm3WTmfjApLOZ4/hRNOyEY6k5joZRETNZfVyEYmJiYiKisK5c+fg6+uL3NxcfP/99xg0aBAyMjJs2jidTodevXqZbevZsydOnToFAAgLCwMApYfHpLi4uF5vj9aFhIzDkCEn0K/fNvTsuQb9+m3DkCEFDHQcxJRYbOpNq0+CLEeoPrE4LS0NsbGxiI2NVYaL4+PjlW1paWlObiERkfWs7tn573//i++++w4hISHw8PCAh4cHbrnlFsyfPx/PPPMM9uzZY7PG3XzzzcjLyzPbduTIEXTq1AkAEBUVhbCwMGzZskXJ2amqqkJmZiYWLlxos3a4Cmf0BNA1WhlOVHOdDCKi5rI62Kmuroafnx8AIDg4GGfPnkX37t3RqVOneoFJSz377LMYNmwYUlNTMX78ePz0009YtmwZli1bBuDa8FVSUhJSU1PRtWtXdO3aFampqfD19cXEiRNt2haipmhhOFHNdTKIiJrL6mCnd+/e2L9/Pzp37ozBgwdj0aJF8Pb2xrJly9C5c2ebNu6mm27Chg0bMGfOHLz22muIiorCkiVL8OijjyrHzJo1C5WVlZg+fbpSVHDz5s1uUWOH1IeJxURE6mP11PNNmzbh0qVLGDduHI4fP4777rsPP//8M9q1a4fPP/+83mrorsDV18Yisgd3rbNDRK7D0vu3TersXLhwAUFBQS67AjqDHSIiItdj6f3b6tlY586dq7etbdu2kCQJ+/fvt/bliIiIiOzK6mCnT58++Oc//1lv+//7f/8PgwcPtkmjiIiIiGzF6mBn9uzZePjhhzFt2jRUVlbizJkzuP322/Hmm2/i888/t0cbiYiIbMJgMECv1yvFM8k9WB3sPPfcc8jKysIPP/yAvn37om/fvvDx8cH+/fttulQEERGRrRkMBqSkpDDYcTNWBzsA0LlzZ8TExODEiROoqKjA+PHj3a5iMbkGIapRWpqBc+c+Q2lpBoSw3XImRETkGqyus/PDDz/gscceQ7t27bB//3788MMPePrpp/Hvf/8baWlpCAoKskc7iazW8MrhHREdvdQlCvwRkeWuVyqBa76R1VPPZVnGs88+i9dffx1eXl4AgGPHjuHxxx/HqVOncPr06SZeQX049Vx7SkrW/2/phrof72vlEWJivmTAQ6QhOTk5iI2NRXZ2dr2K33q9HikpKY3+bHJyMvR6vZ1bSPZg6f3b6p6dzZs3Iy4uzmxbly5dsGPHDrzxxhvWt5TIxoSoRn5+IuoHOvjfNgn5+UkIDh7LysZEboBrvpHVwU7dQMfEw8MDr7zySosbRNRSZWXbzYau6hMwGgtRVradC6cSuTBLh6e45htZnKA8evRolJeXK8/feOMNlJWVKc/Pnz+PXr162bRxRM1RVWXZLAtLjyMidUpLS0NsbCxiY2MRHx8PAIiPj1e2paWlObmFpBYW9+xs2rQJRqNReb5w4UJMmDABgYGBAICrV6/afNVzoubw9rasS9rS44hInZozPKXT6ZCcnMyhKzdjcbBTN4/ZBktqEdlFYOCtkOWOMBrPoOG8HQmy3BGBgbc6umlEZEPNGZ7S6XRMRnZDzaqzQ6RmkuSJ6Oilpmd19wIAoqOXMDmZiMhNWBzsSJJUb1VzV13lnLQvJGQcYmK+hCx3MNsuyx057ZxIgzg8RddjcZ0dDw8P3HPPPZBlGQDw9ddf4/bbb0fr1q0BAEajERs3bkR1tetVqGWdHe0SohplZdtRVWWAt7cOgYG3skeHiEgjLL1/WxzsPPHEExa98YoVKyxroYow2CEiInI9Ni8q6IpBDBERERETlImIiEjTGOwQERGRpjHYISIiIk1jsENERESaxmCHiIiINI3BDhEREWkagx0iIhsxGAzQ6/UwGAzObgoR1cJgh4jIRgwGA1JSUhjsEKmMxUUFicg1aGGJDC2cAxGpB4MdohZS0425pGQ98vMTYTSeVrbJckdERy91mcVPXe0cDAaD0pOTk5Nj9l/g2gKVXJySyLksXhtLy7g2FjWXmm7MJSXrkZv7EIC6v9ISALjEau+ueA56vR4pKSmN7k9OToZer3dcg4jciM0XAtUyBjvUHGq6MQtRjaysSLOgq26bZLkjhgwpUO1wkKueQ92enfj4eKSnp2PgwIEA2LNDZE82XwiUiH4nRDXy8xNRP9DB/7ZJyM9PQnDwWIfcmMvKtl8nSLjWJqOxEGVl2xEUNNzu7WkOVz2HhoKZgQMHKsEOETkfZ2MRNYM1N2ZHqKqybPaPpcc5gxbOgYjUicEOUTOo7cbs7W3ZMImlxzmDFs5Bp9MhOTmZw1ZEKsNgh6gZ1HZjDgy8FbLcEaZ8ofokyHIEAgNvdUh7mkML56DT6aDX6xnsEKkMgx2iZlDbjVmSPBEdvVR577ptAYDo6CWqSuytSwvnQETqxGCHqBnUeGMOCRmHmJgvIcsdzLbLckdVTtluSHDwWHTqpEerVkFm213pHIhIfTj1HJx6Ts3XcJ2dCERHL3HajVlNRQ6t0dC1bNWqLTp2TESnTnNd4hyIyLFYZ8cKDHaoJZoKLlw1+HAkNdUsIiLXYen9W9XDWHq9HpIkmT3CwsKU/UII6PV6hIeHw8fHB8OHD0dubq4TW/w7IapRWpqBc+c+Q2lpBoSodnaTyE4kyRNBQcMRGjoBQUHDzQKZkpL1yMqKxL59I3D48ETs2zcCWVmRKClZ77T2qk3TNYuA/Pwk/g4RUbOpOtgBgJiYGKVCqcFgwIEDB5R9ixYtwuLFi/Huu+9i165dCAsLw8iRI3Hx4kUntpg3OLrG1FtRtx6P0XgGubkP8fPwP2qrWURE2qP6YKdVq1YICwtTHiEhIQCu9eosWbIEc+fOxbhx49C7d2+sWrUKly9fxpo1a677mkajERUVFWYPW+ENjgD2VlhDbTWLiEh7VB/sHD16FOHh4YiKisIjjzyC48ePAwAKCgpQVFSEUaNGKcfKsoy4uDjs3Lnzuq85f/58BAQEKI+IiAibtJU3ODJhb4Xl1FaziIi0R9XBzuDBg/HJJ59g06ZNSE9PR1FREYYNG4bz58+jqKgIABAaGmr2M6Ghocq+xsyZMwfl5eXKo7Cw0Cbt5Q2OTNhbYTm11SwiIu1R9UKg99xzj/LvPn36YOjQoejSpQtWrVqFIUOGAAAkyfwPpBCi3ra6ZFmGLMs2by9vcGRiaS9EZeVRO7dE/Uw1i67NxpJg3jPKYoJE1HKq7tmpq3Xr1ujTpw+OHj2qzMqq24tTXFxcr7fHUdgdTyZN91Zcc+JEMvO4oI2CiESkXi4V7BiNRhw+fBg6nQ5RUVEICwvDli1blP1VVVXIzMzEsGHDnNI+dseTiXmF5eseyTyu/wkJGYchQ06gX79t6NlzDfr124YhQwoY6NB1GQwG6PV6GAzsMafGqTrYef7555GZmYmCggL8+OOPeOihh1BRUYHJkydDkiQkJSUhNTUVGzZswMGDBzFlyhT4+vpi4sSJTmmvGpcQIOcJCRmHTp30TRzFPK7arleziKghBoMBKSkpDHboulSds3P69GlMmDABv/zyC0JCQjBkyBBkZWWhU6dOAIBZs2ahsrIS06dPR2lpKQYPHozNmzfD39/faW02dcfXX0Kgo1OXECDn8PXtatFxzOMiIrIfVQc7a9euve5+SZKg1+uh1+sd0yALhYSMQ3DwWC4RQMzjIrIDU5FZAMjJyTH7LwDodDrodPydot+pOthxZabueHJvpjwuo/EMGq6/JEGWOzKPi1THYDAgLS0NCQkJqgsc0tLSkJKSYrYtPj5e+XdycrLqvgSTc6k6Z4fI1TGPi1yVmnNhEhISkJ2djezsbKSnpwMA0tPTlW0JCQlObiGpDXt2NI4rbjsf87iIbKuhYaqBAwdi4MCBTmoRqR2DHQ0rKVnfyA12KW+wDna9PC4GpKQWzIUhrWKwo1GmBUnr5omYFiRloTbHayiPiwGp63CHoNQVc2F0Oh2Sk5MZhNF1SUKIhrIm3UpFRQUCAgJQXl6ONm3aOLs5LSZENbKyIq+zTte1pNghQwo098falTQWkJpyeRiQqoerB6WWJhvX7dmJj49Henq6MjzEnh1SG0vv3+zZ0SBrFiTljDHnEKIa+fmJaHiGloCpsnJw8FgGpE6mhV5SU7LxmDFjrhusMBeGtIqzsTSIC5KqnzUBKTlP00EpuNwHkQtgz44GsZCd+l2ru9M0BqTO5cq9pC1NNmYuDGkJgx0NYiE7dSspWY9jx5616FgGpM7lyr2kLU021ul0qktGJmouDmNpEAvZqZcp/+PKlZImjpQgyxEMSJ3MlXtJWXiPTLgyPIMdzTIVspPlDmbbZbmjSyRUatH18z/qY0DqfKZe0vpfGkzUG5TqdDoludiUYFz7OYenHMfZwYaaq2E7CoexNIwLkqpL0/kf13h5haBbtw8ZkKqAqZf02mwsCeaBKntJyTKWzoYj+2Gwo3FckFQ9LM3riI7+GwMdFdHCch9MNnY/rIZtjsEOkYNYnv/RoemDyKFcvZdUDcnGal5F3R6cHWy4YjVse2IFZWivgjKp0++Vra8/S46VrUmLcnJyEBsbi+zsbLcoUqjX6+sFG7XZO9hwl2rYrKDsRtxhzR4tYP4HkftISEjAmDFjADQebNgTq2GbY7Dj4lx9zR53o4X8DyJLOXsox5kYbKgLgx07s2evixbW7HFHrp7/QWQp5o2oAxPUmbMDwH45O/bsdeHK5kSkdu6SN9IUd0vOdiRL798MdmCfYKexXhdTboalvS6N9QyVlmZg374RTf58v37bOPWciJzO0QnKDDDcg6X3b1ZQtgNbrZRcUrIeWVmR2LdvBA4fnoh9+0YgKysSJSXrXXrNHiIie1Nz1WBnV1R2Rwx27MCalZIbY+oZqvs6pnycy5ePWtQWNa7ZQ0Tuh3kjv1NzIKZVTFC2g5b2ujTdMyShqCgd3t4dUFV1tpHjuLI5UUuxrIPtNKewobXX351nf9H1Mdixg5aulGxZz9BpREam4MQJPVizhVyBqwUOLOvgXM25/mqe/cVAzLmYoAzbJyi3tFLuuXOf4fDhiU2+T8+ea+DhITfwByGCNVtIVVwtcLDVBANqnuZefzXP/nJ2RWWtYgVlJ2pppVxreoaCgoazZgupmqvVg7JkGDk/PwnBwWP5e2YHLbn+ai7k5+yKyu6OwY6dtKRSbmDgrZDljk32DJnycbiyOamVKwYO1kww4O+d7Wn1+qs5EHMHDHbsqLmVcrmGEmmFK964WNbBuWx1/Tn7i2pjsGNnze114RpKpAWuGDi0dIKBmrhaUjhgu+vfnNlfjsJAzPEY7KgY11AiV+eKgYO1w8hq5WpJ4SZauf7Xo+ZATKtYVFDlTD1DoaETEBQ0nIEOuRTTjcs0/FqfBFmOcPqNS4hqlJZm4Ny5z1BWth1duixW2mfONYaRmypKWlKy3kkta5ppGP9/z+ruBaD+60/qw2CHiOzGFW5cDS3LcuzYTEREPA9Z7mB2rCx3VN3ssbpstVyNM5mG8V3x+pM6sc4O7LfqORFd0/CQivPrQTVVz6VXry/g5RXsUsPIWlok2BVzjsixWGeHiFRDjflnlkyLP3ZsZqPFP9XKFZPCGyNJnggMvFX53JSVbXf654ZcE4MdInIItdWDcsVp8ZZwxaTwxrhqkjWpD3N2iMgtaakHpDZXSQpviisnWZP6MNghIrekpR6Q2lwhKbwpWkiyJnVxqWBn/vz5kCQJSUlJyjYhBPR6PcLDw+Hj44Phw4cjNzfXeY0kIpeglR6Qhrj6bCZrhhiJLOEyOTu7du3CsmXL0LdvX7PtixYtwuLFi7Fy5Up069YN8+bNw8iRI5GXlwd/f38ntZaI1E7ry7KoMSncUlodYiTncYmenV9//RWPPvoo0tPTERQUpGwXQmDJkiWYO3cuxo0bh969e2PVqlW4fPky1qxZ48QWE7mG2sX0Sksz3G5YwNV7QJriqkVJtTrESM7jEj07M2bMwL333os777wT8+bNU7YXFBSgqKgIo0aNUrbJsoy4uDjs3LkTCQkJDb6e0WiE0WhUnldUVNiv8UQqxZku17hyD4hWucOSEeRYqu/ZWbt2LXJycjB//vx6+4qKigAAoaGhZttDQ0OVfQ2ZP38+AgIClEdERIRtG02kcpzpYs5Ve0C0SgtJ1qQuqg52CgsLkZiYiNWrV+OGG25o9DhJMv9lEELU21bbnDlzUF5erjwKCwtt1mYiteNMF3IFWh9ibAl3H35uDlUPY2VnZ6O4uBixsbHKturqanz//fd49913kZeXB+BaD49O9/vYbXFxcb3entpkWYYsy/ZrOJGKabWYHmkPhxjr4/Bz86g62Lnjjjtw4MABs21PPPEEevTogdmzZ6Nz584ICwvDli1bMGDAAABAVVUVMjMzsXDhQmc0mUj1ONOFXInaKm87U2NruZmGn63t8XKntcdUHez4+/ujd+/eZttat26Ndu3aKduTkpKQmpqKrl27omvXrkhNTYWvry8mTpzojCYTqR5nuhC5HkvWcsvPT0Jw8FiLAhZ36yFSdbBjiVmzZqGyshLTp09HaWkpBg8ejM2bN7PGDlEjONOFyPXYcvjZ1j1ErkASQjT0186tWLpEPJFW/P7HDmiomJ4W/9gRubJz5z7D4cNNj1j07LkGoaETGt0vRDWysiKvEzhd+7IzZEiBSwxpWXr/VvVsLCKyD850IXItthp+dtelOFx+GIuImoczXYhch62Gn911ggKDHSI3xpkuRK7BVmu5uesEBQ5jERERuQBbDD+beojqV6Y2kSDLEZqboMCeHSIiIhfR0uFnW/UQuRoGO0RERC6kpcPPph6ihuvsLNHkBAUGO0RERG7G3SYoMNghIiJyQ+40QYEJykRERKRp7NkhchPutOgfEVFtDHaI3IC7LfpHRFQbh7GINM60DlbdEvGmRf9KStY7qWVERI7BYIdIw4SoRn5+IhouL39tW35+EoSodmi7iIgcicEOkYa566J/RES1Mdgh0jB3XfSPiKg2BjtEGuaui/4REdXGYIdIw9x10T8iotoY7BBpmGnRv/89q7sXgDYX/SMiqo3BDpHGmRb9k+UOZttluSNiYr5knR0i0jwWFSRyA+626B8RUW0MdojchDst+kdEVBuHsYiIiEjTGOwQERGRpjHYISIiIk1jsENERESaxmCHiIiINI3BDhEREWkagx0iIiLSNAY7REREpGkMdoiIiEjTGOwQERGRpjHYISIiIk1jsENERESaxmCHiIiINI2rnhMRaZgQ1Sgr246qKgO8vXUIDLwVkuTp7GYRORSDHSIijSopWY/8/EQYjaeVbbLcEdHRSxESMs6JLSNyLA5jERFpUEnJeuTmPmQW6ACA0XgGubkPoaRkvZNaRuR4DHaIiDRGiGrk5ycCEA3tBQDk5ydBiGqHtovIWVQd7HzwwQfo27cv2rRpgzZt2mDo0KH45ptvlP1CCOj1eoSHh8PHxwfDhw9Hbm6uE1tMROR8ZWXb6/XomBMwGgtRVrbdYW0iciZVBzsdO3bEggULsHv3buzevRu33347xo4dqwQ0ixYtwuLFi/Huu+9i165dCAsLw8iRI3Hx4kUnt5yIyHmqqgw2PY7I1ak62Ln//vsxevRodOvWDd26dcMbb7wBPz8/ZGVlQQiBJUuWYO7cuRg3bhx69+6NVatW4fLly1izZo2zm05E5DTe3jqbHkfk6lQd7NRWXV2NtWvX4tKlSxg6dCgKCgpQVFSEUaNGKcfIsoy4uDjs3Lnzuq9lNBpRUVFh9iAi0orAwFshyx0BSI0cIUGWIxAYeKsjm0XkNKoPdg4cOAA/Pz/Isoxp06Zhw4YN6NWrF4qKigAAoaGhZseHhoYq+xozf/58BAQEKI+IiAi7tZ+IyNEkyRPR0UtNz+ruBQBERy9hvR1yG6oPdrp37469e/ciKysLTz31FCZPnoxDhw4p+yXJ/BdZCFFvW11z5sxBeXm58igsLLRL24mInCUkZBxiYr6ELHcw2y7LHRET8yXr7JBbUX1RQW9vb0RHRwMABg0ahF27dmHp0qWYPXs2AKCoqAg63e/jzsXFxfV6e+qSZRmyLNuv0UREKhASMg7BwWNZQZncnup7duoSQsBoNCIqKgphYWHYsmWLsq+qqgqZmZkYNmyYE1tIRKQekuSJoKDhCA2dgKCg4Qx0yC2pumfnpZdewj333IOIiAhcvHgRa9euRUZGBjZu3AhJkpCUlITU1FR07doVXbt2RWpqKnx9fTFx4kRnN52IiIhUQtXBzrlz5/D444/DYDAgICAAffv2xcaNGzFy5EgAwKxZs1BZWYnp06ejtLQUgwcPxubNm+Hv7+/klhMREZFaSEKIhuqJu5WKigoEBASgvLwcbdq0cXZziIiIyAKW3r9dLmeHiIiIyBoMdoiIiEjTGOwQERGRpjHYISIiIk1jsENERESaxmCHiIiINE3VdXYcxTT7nqufExERuQ7TfbupKjoMdgBcvHgRALj6ORERkQu6ePEiAgICGt3PooIAampqcPbsWfj7+ze5Yro1KioqEBERgcLCQrctVuju18Ddzx/gNQB4Ddz9/AFeA3udvxACFy9eRHh4ODw8Gs/MYc8OAA8PD3Ts2NFur9+mTRu3/HDX5u7XwN3PH+A1AHgN3P38AV4De5z/9Xp0TJigTERERJrGYIeIiIg0jcGOHcmyjOTkZMiy7OymOI27XwN3P3+A1wDgNXD38wd4DZx9/kxQJiIiIk1jzw4RERFpGoMdIiIi0jQGO0RERKRpDHaIiIhI0xjstNAHH3yAvn37KoWShg4dim+++UbZL4SAXq9HeHg4fHx8MHz4cOTm5jqxxfY3f/58SJKEpKQkZZvWr4Ner4ckSWaPsLAwZb/Wzx8Azpw5g8ceewzt2rWDr68v+vfvj+zsbGW/1q9BZGRkvc+AJEmYMWMGAO2f/9WrV/Hyyy8jKioKPj4+6Ny5M1577TXU1NQox2j9GgDXli1ISkpCp06d4OPjg2HDhmHXrl3Kfq1dg++//x73338/wsPDIUkSvvrqK7P9lpyv0WjE008/jeDgYLRu3RpjxozB6dOnbdtQQS3yz3/+U/z73/8WeXl5Ii8vT7z00kvCy8tLHDx4UAghxIIFC4S/v79Yt26dOHDggHj44YeFTqcTFRUVTm65ffz0008iMjJS9O3bVyQmJirbtX4dkpOTRUxMjDAYDMqjuLhY2a/1879w4YLo1KmTmDJlivjxxx9FQUGB2Lp1q8jPz1eO0fo1KC4uNvv/v2XLFgFAbNu2TQih/fOfN2+eaNeunfjXv/4lCgoKxP/93/8JPz8/sWTJEuUYrV8DIYQYP3686NWrl8jMzBRHjx4VycnJok2bNuL06dNCCO1dg//85z9i7ty5Yt26dQKA2LBhg9l+S8532rRpokOHDmLLli0iJydHjBgxQvTr109cvXrVZu1ksGMHQUFB4qOPPhI1NTUiLCxMLFiwQNn322+/iYCAAPHhhx86sYX2cfHiRdG1a1exZcsWERcXpwQ77nAdkpOTRb9+/Rrc5w7nP3v2bHHLLbc0ut8drkFdiYmJokuXLqKmpsYtzv/ee+8VU6dONds2btw48dhjjwkh3OMzcPnyZeHp6Sn+9a9/mW3v16+fmDt3ruavQd1gx5LzLSsrE15eXmLt2rXKMWfOnBEeHh5i48aNNmsbh7FsqLq6GmvXrsWlS5cwdOhQFBQUoKioCKNGjVKOkWUZcXFx2LlzpxNbah8zZszAvffeizvvvNNsu7tch6NHjyI8PBxRUVF45JFHcPz4cQDucf7//Oc/MWjQIPzpT39C+/btMWDAAKSnpyv73eEa1FZVVYXVq1dj6tSpkCTJLc7/lltuwbfffosjR44AAPbt24cdO3Zg9OjRANzjM3D16lVUV1fjhhtuMNvu4+ODHTt2uMU1qM2S883OzsaVK1fMjgkPD0fv3r1tek0Y7NjAgQMH4OfnB1mWMW3aNGzYsAG9evVCUVERACA0NNTs+NDQUGWfVqxduxY5OTmYP39+vX3ucB0GDx6MTz75BJs2bUJ6ejqKioowbNgwnD9/3i3O//jx4/jggw/QtWtXbNq0CdOmTcMzzzyDTz75BIB7fAZq++qrr1BWVoYpU6YAcI/znz17NiZMmIAePXrAy8sLAwYMQFJSEiZMmADAPa6Bv78/hg4ditdffx1nz55FdXU1Vq9ejR9//BEGg8EtrkFtlpxvUVERvL29ERQU1OgxtsBVz22ge/fu2Lt3L8rKyrBu3TpMnjwZmZmZyn5JksyOF0LU2+bKCgsLkZiYiM2bN9f7RlOblq/DPffco/y7T58+GDp0KLp06YJVq1ZhyJAhALR9/jU1NRg0aBBSU1MBAAMGDEBubi4++OADTJo0STlOy9egtuXLl+Oee+5BeHi42XYtn//nn3+O1atXY82aNYiJicHevXuRlJSE8PBwTJ48WTlOy9cAAD799FNMnToVHTp0gKenJwYOHIiJEyciJydHOUbr16Cu5pyvra8Je3ZswNvbG9HR0Rg0aBDmz5+Pfv36YenSpcpsnLrRaXFxcb1I15VlZ2ejuLgYsbGxaNWqFVq1aoXMzEy8/fbbaNWqlXKuWr8OtbVu3Rp9+vTB0aNH3eJzoNPp0KtXL7NtPXv2xKlTpwDALa6BycmTJ7F161b8+c9/Vra5w/m/8MILePHFF/HII4+gT58+ePzxx/Hss88qvb3ucA0AoEuXLsjMzMSvv/6KwsJC/PTTT7hy5QqioqLc5hqYWHK+YWFhqKqqQmlpaaPH2AKDHTsQQsBoNCof7i1btij7qqqqkJmZiWHDhjmxhbZ1xx134MCBA9i7d6/yGDRoEB599FHs3bsXnTt3dovrUJvRaMThw4eh0+nc4nNw8803Iy8vz2zbkSNH0KlTJwBwi2tgsmLFCrRv3x733nuvss0dzv/y5cvw8DC/pXh6eipTz93hGtTWunVr6HQ6lJaWYtOmTRg7dqzbXQNLzjc2NhZeXl5mxxgMBhw8eNC218Rmqc5uas6cOeL7778XBQUFYv/+/eKll14SHh4eYvPmzUKIa9PuAgICxPr168WBAwfEhAkTXHqaoaVqz8YSQvvX4bnnnhMZGRni+PHjIisrS9x3333C399fnDhxQgih/fP/6aefRKtWrcQbb7whjh49Kv7+978LX19fsXr1auUYrV8DIYSorq4WN954o5g9e3a9fVo//8mTJ4sOHTooU8/Xr18vgoODxaxZs5RjtH4NhBBi48aN4ptvvhHHjx8XmzdvFv369RN/+MMfRFVVlRBCe9fg4sWLYs+ePWLPnj0CgFi8eLHYs2ePOHnypBDCsvOdNm2a6Nixo9i6davIyckRt99+O6eeq83UqVNFp06dhLe3twgJCRF33HGHEugIcW3qXXJysggLCxOyLIvbbrtNHDhwwIktdoy6wY7Wr4OpdoSXl5cIDw8X48aNE7m5ucp+rZ+/EEJ8/fXXonfv3kKWZdGjRw+xbNkys/3ucA02bdokAIi8vLx6+7R+/hUVFSIxMVHceOON4oYbbhCdO3cWc+fOFUajUTlG69dACCE+//xz0blzZ+Ht7S3CwsLEjBkzRFlZmbJfa9dg27ZtAkC9x+TJk4UQlp1vZWWl+Otf/yratm0rfHx8xH333SdOnTpl03ZKQghhu34iIiIiInVhzg4RERFpGoMdIiIi0jQGO0RERKRpDHaIiIhI0xjsEBERkaYx2CEiIiJNY7BDREREmsZgh4iIiDSNwQ4RERFpGoMdIrLYlClTIElSvcfdd9/t7KY1KDExEbGxsZBlGf3793d2c4jISVo5uwFE5FruvvturFixwmybLMtOas31CSEwdepU/Pjjj9i/f7+zm1NPVVUVvL29nd0MIs1jzw4RWUWWZYSFhZk9goKCAAAZGRnw9vbG9u3blePfeustBAcHw2AwAAA2btyIW265BYGBgWjXrh3uu+8+HDt2TDn+xIkTkCQJX3zxBW699Vb4+PjgpptuwpEjR7Br1y4MGjQIfn5+uPvuu1FSUnLdtr799tuYMWMGOnfubPH56fV63HjjjZBlGeHh4XjmmWeUfUajEbNmzUJERARkWUbXrl2xfPlyZX9mZib+8Ic/QJZl6HQ6vPjii7h69aqyf/jw4fjrX/+KmTNnIjg4GCNHjgQAHDp0CKNHj4afnx9CQ0Px+OOP45dffrG4zUR0fQx2iMhmhg8fjqSkJDz++OMoLy/Hvn37MHfuXKSnp0On0wEALl26hJkzZ2LXrl349ttv4eHhgQcffBA1NTVmr5WcnIyXX34ZOTk5aNWqFSZMmIBZs2Zh6dKl2L59O44dO4ZXX33Vpu3/8ssv8be//Q1paWk4evQovvrqK/Tp00fZP2nSJKxduxZvv/02Dh8+jA8//BB+fn4AgDNnzmD06NG46aabsG/fPnzwwQdYvnw55s2bZ/Yeq1atQqtWrfDDDz8gLS0NBoMBcXFx6N+/P3bv3o2NGzfi3LlzGD9+vE3Pjcit2XQNdSLStMmTJwtPT0/RunVrs8drr72mHGM0GsWAAQPE+PHjRUxMjPjzn/983dcsLi4WAMSBAweEEEIUFBQIAOKjjz5Sjvnss88EAPHtt98q2+bPny+6d+9uUbuTk5NFv379mjzurbfeEt26dRNVVVX19uXl5QkAYsuWLQ3+7EsvvSS6d+8uampqlG3vvfee8PPzE9XV1UIIIeLi4kT//v3Nfu6VV14Ro0aNMttWWFgoAIi8vLwm20xETWPPDhFZZcSIEdi7d6/ZY8aMGcp+b29vrF69GuvWrUNlZSWWLFli9vPHjh3DxIkT0blzZ7Rp0wZRUVEAgFOnTpkd17dvX+XfoaGhAGDWyxIaGori4mKbntuf/vQnVFZWonPnzoiPj8eGDRuUYai9e/fC09MTcXFxDf7s4cOHMXToUEiSpGy7+eab8euvv+L06dPKtkGDBpn9XHZ2NrZt2wY/Pz/l0aNHDwAwG94jouZjgjIRWaV169aIjo6+7jE7d+4EAFy4cAEXLlxA69atlX33338/IiIikJ6ejvDwcNTU1KB3796oqqoyew0vLy/l36YAou62ukNfLRUREYG8vDxs2bIFW7duxfTp0/Hmm28iMzMTPj4+1/1ZIYRZoGPaVrv9AMyuBQDU1NTg/vvvx8KFC+u9pmnoj4hahj07RGRTx44dw7PPPov09HQMGTIEkyZNUoKS8+fP4/Dhw3j55Zdxxx13oGfPnigtLXVyi835+PhgzJgxePvtt5GRkYH//ve/OHDgAPr06YOamhpkZmY2+HO9evXCzp07lQAHuBb0+fv7o0OHDo2+38CBA5Gbm4vIyEhER0ebPeoGRkTUPAx2iMgqRqMRRUVFZg/TzKHq6mo8/vjjGDVqFJ544gmsWLECBw8exFtvvQUACAoKQrt27bBs2TLk5+fju+++w8yZM+3W1vz8fOzduxdFRUWorKxUht3q9iKZrFy5EsuXL8fBgwdx/PhxfPrpp/Dx8UGnTp0QGRmJyZMnY+rUqfjqq69QUFCAjIwMfPHFFwCA6dOno7CwEE8//TR+/vln/OMf/0BycjJmzpwJD4/G/9TOmDEDFy5cwIQJE/DTTz/h+PHj2Lx5M6ZOnYrq6mq7XBcit+PknCEiciGTJ08WAOo9TInCKSkpQqfTiV9++UX5ma+++kp4e3uLPXv2CCGE2LJli+jZs6eQZVn07dtXZGRkCABiw4YNQojfE5RNxwshxLZt2wQAUVpaqmxbsWKFCAgIuG574+LiGmxvQUFBg8dv2LBBDB48WLRp00a0bt1aDBkyRGzdulXZX1lZKZ599lmh0+mEt7e3iI6OFh9//LGyPyMjQ9x0003C29tbhIWFidmzZ4srV66YtScxMbHe+x45ckQ8+OCDIjAwUPj4+IgePXqIpKQks2RnImo+SYhafa5EREREGsNhLCIiItI0BjtERESkaQx2iIiISNMY7BAREZGmMdghIiIiTWOwQ0RERJrGYIeIiIg0jcEOERERaRqDHSIiItI0BjtERESkaQx2iIiISNP+P23CLUB8hyGaAAAAAElFTkSuQmCC",
"text/plain": [
"
"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Plot examples\n",
"plot_data(X_train, y_train[:], pos_label=\"Admitted\", neg_label=\"Not admitted\")\n",
"\n",
"# Set the y-axis label\n",
"plt.ylabel('Exam 2 score') \n",
"# Set the x-axis label\n",
"plt.xlabel('Exam 1 score') \n",
"plt.legend(loc=\"upper right\")\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "734e4cff",
"metadata": {},
"source": [
"Your goal is to build a logistic regression model to fit this data.\n",
"- With this model, you can then predict if a new student will be admitted based on their scores on the two exams."
]
},
{
"cell_type": "markdown",
"id": "41a18ab6",
"metadata": {},
"source": [
"##### 2.3 Sigmoid function\n",
"\n",
"\n",
"Recall that for logistic regression, the model is represented as\n",
"\n",
"$$ f_{\\mathbf{w},b}(x) = g(\\mathbf{w}\\cdot \\mathbf{x} + b)$$\n",
"where function $g$ is the sigmoid function. The sigmoid function is defined as:\n",
"\n",
"$$g(z) = \\frac{1}{1+e^{-z}}$$\n",
"\n",
"Let's implement the sigmoid function first, so it can be used by the rest of this assignment.\n",
"\n",
"###### Exercise 1\n",
"\n",
"\n",
"Please complete the `sigmoid` function to calculate\n",
"\n",
"$$g(z) = \\frac{1}{1+e^{-z}}$$\n",
"\n",
"Note that \n",
"- `z` is not always a single number, but can also be an array of numbers. \n",
"- If the input is an array of numbers, we'd like to apply the sigmoid function to each value in the input array.\n",
"\n",
"If you get stuck, you can check out the hints presented after the cell below to help you with the implementation."
]
},
{
"cell_type": "code",
"execution_count": 21,
"id": "b21a4ef0",
"metadata": {},
"outputs": [],
"source": [
"# UNQ_C1\n",
"# GRADED FUNCTION: sigmoid\n",
"\n",
"def sigmoid(z):\n",
" \"\"\"\n",
" Compute the sigmoid of z\n",
"\n",
" Args:\n",
" z (ndarray): A scalar, numpy array of any size.\n",
"\n",
" Returns:\n",
" g (ndarray): sigmoid(z), with the same shape as z\n",
" \n",
" \"\"\"\n",
" \n",
" ### START CODE HERE ### \n",
" g = 1/(1+np.exp(-z))\n",
" ### END SOLUTION ### \n",
" \n",
" return g"
]
},
{
"cell_type": "markdown",
"id": "b63b9554",
"metadata": {},
"source": [
"\n",
" Click for hints\n",
" \n",
"`numpy` has a function called [`np.exp()`](https://numpy.org/doc/stable/reference/generated/numpy.exp.html), which offers a convinient way to calculate the exponential ( $e^{z}$) of all elements in the input array (`z`).\n",
" \n",
"\n",
" Click for more hints\n",
" \n",
" - You can translate $e^{-z}$ into code as `np.exp(-z)` \n",
" \n",
" - You can translate $1/e^{-z}$ into code as `1/np.exp(-z)` \n",
" \n",
" If you're still stuck, you can check the hints presented below to figure out how to calculate `g` \n",
" \n",
" \n",
" Hint to calculate g\n",
" g = 1 / (1 + np.exp(-z))\n",
" \n",
"\n",
"\n",
""
]
},
{
"cell_type": "markdown",
"id": "6f2932ee",
"metadata": {},
"source": [
"When you are finished, try testing a few values by calling `sigmoid(x)` in the cell below. \n",
"- For large positive values of x, the sigmoid should be close to 1, while for large negative values, the sigmoid should be close to 0. \n",
"- Evaluating `sigmoid(0)` should give you exactly 0.5. \n"
]
},
{
"cell_type": "code",
"execution_count": 22,
"id": "2e8ef3d4",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"sigmoid(0) = 0.5\n"
]
}
],
"source": [
"print (\"sigmoid(0) = \" + str(sigmoid(0)))"
]
},
{
"cell_type": "markdown",
"id": "c07b16cd",
"metadata": {},
"source": [
"**Expected Output**:\n",
"
\n",
"
\n",
"
sigmoid(0)
\n",
"
0.5
\n",
"
\n",
"
\n",
" \n",
"- As mentioned before, your code should also work with vectors and matrices. For a matrix, your function should perform the sigmoid function on every element."
]
},
{
"cell_type": "code",
"execution_count": 23,
"id": "42b4570d",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"sigmoid([ -1, 0, 1, 2]) = [0.26894142 0.5 0.73105858 0.88079708]\n",
"\u001b[92mAll tests passed!\n"
]
}
],
"source": [
"print (\"sigmoid([ -1, 0, 1, 2]) = \" + str(sigmoid(np.array([-1, 0, 1, 2]))))\n",
"\n",
"# UNIT TESTS\n",
"from public_tests import *\n",
"sigmoid_test(sigmoid)"
]
},
{
"cell_type": "markdown",
"id": "90682034",
"metadata": {},
"source": [
"**Expected Output**:\n",
"
\n",
"
\n",
"
sigmoid([-1, 0, 1, 2])
\n",
"
[0.26894142 0.5 0.73105858 0.88079708]
\n",
"
\n",
" \n",
"
"
]
},
{
"cell_type": "markdown",
"id": "f05dce11",
"metadata": {},
"source": [
"##### 2.4 Cost function for logistic regression\n",
"\n",
"\n",
"In this section, you will implement the cost function for logistic regression.\n",
"\n",
"###### Exercise 2\n",
"\n",
"\n",
"Please complete the `compute_cost` function using the equations below.\n",
"\n",
"Recall that for logistic regression, the cost function is of the form \n",
"\n",
"$$ J(\\mathbf{w},b) = \\frac{1}{m}\\sum_{i=0}^{m-1} \\left[ loss(f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}), y^{(i)}) \\right] \\tag{1}$$\n",
"\n",
"where\n",
"* m is the number of training examples in the dataset\n",
"\n",
"\n",
"* $loss(f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}), y^{(i)})$ is the cost for a single data point, which is - \n",
"\n",
" $$loss(f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}), y^{(i)}) = (-y^{(i)} \\log\\left(f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right) - \\left( 1 - y^{(i)}\\right) \\log \\left( 1 - f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right) \\tag{2}$$\n",
" \n",
" \n",
"* $f_{\\mathbf{w},b}(\\mathbf{x}^{(i)})$ is the model's prediction, while $y^{(i)}$, which is the actual label\n",
"\n",
"* $f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) = g(\\mathbf{w} \\cdot \\mathbf{x^{(i)}} + b)$ where function $g$ is the sigmoid function.\n",
" * It might be helpful to first calculate an intermediate variable $z_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) = \\mathbf{w} \\cdot \\mathbf{x^{(i)}} + b = w_0x^{(i)}_0 + ... + w_{n-1}x^{(i)}_{n-1} + b$ where $n$ is the number of features, before calculating $f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) = g(z_{\\mathbf{w},b}(\\mathbf{x}^{(i)}))$\n",
"\n",
"Note:\n",
"* As you are doing this, remember that the variables `X_train` and `y_train` are not scalar values but matrices of shape ($m, n$) and ($𝑚$,1) respectively, where $𝑛$ is the number of features and $𝑚$ is the number of training examples.\n",
"* You can use the sigmoid function that you implemented above for this part.\n",
"\n",
"If you get stuck, you can check out the hints presented after the cell below to help you with the implementation."
]
},
{
"cell_type": "code",
"execution_count": 24,
"id": "dc682c2d",
"metadata": {},
"outputs": [],
"source": [
"# UNQ_C2\n",
"# GRADED FUNCTION: compute_cost\n",
"def compute_cost(X, y, w, b, lambda_= 1):\n",
" \"\"\"\n",
" Computes the cost over all examples\n",
" Args:\n",
" X : (ndarray Shape (m,n)) data, m examples by n features\n",
" y : (array_like Shape (m,)) target value \n",
" w : (array_like Shape (n,)) Values of parameters of the model \n",
" b : scalar Values of bias parameter of the model\n",
" lambda_: unused placeholder\n",
" Returns:\n",
" total_cost: (scalar) cost \n",
" \"\"\"\n",
"\n",
" m, n = X.shape\n",
" \n",
" ### START CODE HERE ###\n",
" cost = 0\n",
" for i in range(m):\n",
" z = np.dot(X[i],w) + b\n",
" f_wb = sigmoid(z)\n",
" cost += -y[i]*np.log(f_wb) - (1-y[i])*np.log(1-f_wb)\n",
" total_cost = cost/m\n",
" \n",
" ### END CODE HERE ### \n",
"\n",
" return total_cost"
]
},
{
"cell_type": "markdown",
"id": "af06ab44",
"metadata": {},
"source": [
"\n",
" Click for hints\n",
" \n",
" \n",
" * You can represent a summation operator eg: $h = \\sum\\limits_{i = 0}^{m-1} 2i$ in code as follows:\n",
" ```python \n",
" h = 0\n",
" for i in range(m):\n",
" h = h + 2*i\n",
" ```\n",
" \n",
" * In this case, you can iterate over all the examples in `X` using a for loop and add the `loss` from each iteration to a variable (`loss_sum`) initialized outside the loop.\n",
"\n",
" * Then, you can return the `total_cost` as `loss_sum` divided by `m`.\n",
" \n",
" \n",
" Click for more hints\n",
" \n",
" * Here's how you can structure the overall implementation for this function\n",
" ```python \n",
" def compute_cost(X, y, w, b, lambda_= 1):\n",
" m, n = X.shape\n",
" \n",
" ### START CODE HERE ###\n",
" loss_sum = 0 \n",
" \n",
" # Loop over each training example\n",
" for i in range(m): \n",
" \n",
" # First calculate z_wb = w[0]*X[i][0]+...+w[n-1]*X[i][n-1]+b\n",
" z_wb = 0 \n",
" # Loop over each feature\n",
" for j in range(n): \n",
" # Add the corresponding term to z_wb\n",
" z_wb_ij = # Your code here to calculate w[j] * X[i][j]\n",
" z_wb += z_wb_ij # equivalent to z_wb = z_wb + z_wb_ij\n",
" # Add the bias term to z_wb\n",
" z_wb += b # equivalent to z_wb = z_wb + b\n",
" \n",
" f_wb = # Your code here to calculate prediction f_wb for a training example\n",
" loss = # Your code here to calculate loss for a training example\n",
" \n",
" loss_sum += loss # equivalent to loss_sum = loss_sum + loss\n",
" \n",
" total_cost = (1 / m) * loss_sum \n",
" ### END CODE HERE ### \n",
" \n",
" return total_cost\n",
" ```\n",
" \n",
" If you're still stuck, you can check the hints presented below to figure out how to calculate `z_wb_ij`, `f_wb` and `cost`.\n",
" \n",
" Hint to calculate z_wb_ij\n",
" z_wb_ij = w[j]*X[i][j] \n",
" \n",
" \n",
" \n",
" Hint to calculate f_wb\n",
" $f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) = g(z_{\\mathbf{w},b}(\\mathbf{x}^{(i)}))$ where $g$ is the sigmoid function. You can simply call the `sigmoid` function implemented above.\n",
" \n",
" More hints to calculate f\n",
" You can compute f_wb as f_wb = sigmoid(z_wb) \n",
" \n",
" \n",
"\n",
" \n",
" Hint to calculate loss\n",
" You can use the np.log function to calculate the log\n",
" \n",
" More hints to calculate loss\n",
" You can compute loss as loss = -y[i] * np.log(f_wb) - (1 - y[i]) * np.log(1 - f_wb)\n",
" \n",
" \n",
" \n",
" \n",
"\n",
""
]
},
{
"cell_type": "markdown",
"id": "603ffa12",
"metadata": {},
"source": [
"Run the cells below to check your implementation of the `compute_cost` function with two different initializations of the parameters $w$"
]
},
{
"cell_type": "code",
"execution_count": 25,
"id": "5811e870",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Cost at initial w (zeros): 0.693\n"
]
}
],
"source": [
"m, n = X_train.shape\n",
"\n",
"# Compute and display cost with w initialized to zeroes\n",
"initial_w = np.zeros(n)\n",
"initial_b = 0.\n",
"cost = compute_cost(X_train, y_train, initial_w, initial_b)\n",
"print('Cost at initial w (zeros): {:.3f}'.format(cost))"
]
},
{
"cell_type": "markdown",
"id": "1d6e1746",
"metadata": {},
"source": [
"**Expected Output**:\n",
"
"
]
},
{
"cell_type": "markdown",
"id": "e6eff0f4",
"metadata": {},
"source": [
"##### 2.5 Gradient for logistic regression\n",
"\n",
"\n",
"In this section, you will implement the gradient for logistic regression.\n",
"\n",
"Recall that the gradient descent algorithm is:\n",
"\n",
"$$\\begin{align*}& \\text{repeat until convergence:} \\; \\lbrace \\newline \\; & b := b - \\alpha \\frac{\\partial J(\\mathbf{w},b)}{\\partial b} \\newline \\; & w_j := w_j - \\alpha \\frac{\\partial J(\\mathbf{w},b)}{\\partial w_j} \\tag{1} \\; & \\text{for j := 0..n-1}\\newline & \\rbrace\\end{align*}$$\n",
"\n",
"where, parameters $b$, $w_j$ are all updated simultaniously"
]
},
{
"cell_type": "markdown",
"id": "cfdbf87b",
"metadata": {},
"source": [
"###### Exercise 3\n",
"\n",
"\n",
"\n",
"Please complete the `compute_gradient` function to compute $\\frac{\\partial J(\\mathbf{w},b)}{\\partial w}$, $\\frac{\\partial J(\\mathbf{w},b)}{\\partial b}$ from equations (2) and (3) below.\n",
"\n",
"$$\n",
"\\frac{\\partial J(\\mathbf{w},b)}{\\partial b} = \\frac{1}{m} \\sum\\limits_{i = 0}^{m-1} (f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) - \\mathbf{y}^{(i)}) \\tag{2}\n",
"$$\n",
"$$\n",
"\\frac{\\partial J(\\mathbf{w},b)}{\\partial w_j} = \\frac{1}{m} \\sum\\limits_{i = 0}^{m-1} (f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) - \\mathbf{y}^{(i)})x_{j}^{(i)} \\tag{3}\n",
"$$\n",
"* m is the number of training examples in the dataset\n",
"\n",
" \n",
"* $f_{\\mathbf{w},b}(x^{(i)})$ is the model's prediction, while $y^{(i)}$ is the actual label\n",
"\n",
"\n",
"- **Note**: While this gradient looks identical to the linear regression gradient, the formula is actually different because linear and logistic regression have different definitions of $f_{\\mathbf{w},b}(x)$.\n",
"\n",
"As before, you can use the sigmoid function that you implemented above and if you get stuck, you can check out the hints presented after the cell below to help you with the implementation."
]
},
{
"cell_type": "code",
"execution_count": 27,
"id": "ec8a88b0",
"metadata": {},
"outputs": [],
"source": [
"# UNQ_C3\n",
"# GRADED FUNCTION: compute_gradient\n",
"def compute_gradient(X, y, w, b, lambda_=None): \n",
" \"\"\"\n",
" Computes the gradient for logistic regression \n",
" \n",
" Args:\n",
" X : (ndarray Shape (m,n)) variable such as house size \n",
" y : (array_like Shape (m,1)) actual value \n",
" w : (array_like Shape (n,1)) values of parameters of the model \n",
" b : (scalar) value of parameter of the model \n",
" lambda_: unused placeholder.\n",
" Returns\n",
" dj_dw: (array_like Shape (n,1)) The gradient of the cost w.r.t. the parameters w. \n",
" dj_db: (scalar) The gradient of the cost w.r.t. the parameter b. \n",
" \"\"\"\n",
" m, n = X.shape\n",
" dj_dw = np.zeros(w.shape)\n",
" dj_db = 0.\n",
"\n",
" ### START CODE HERE ### \n",
" for i in range(m):\n",
" f_wb_i = sigmoid(np.dot(X[i],w) + b) \n",
" err_i = f_wb_i - y[i] \n",
" for j in range(n):\n",
" dj_dw[j] = dj_dw[j] + err_i * X[i,j] \n",
" dj_db = dj_db + err_i\n",
" dj_dw = dj_dw/m \n",
" dj_db = dj_db/m \n",
" \n",
" ### END CODE HERE ###\n",
"\n",
" \n",
" return dj_db, dj_dw"
]
},
{
"cell_type": "markdown",
"id": "fcb8beee",
"metadata": {},
"source": [
" \n",
" Click for hints\n",
" \n",
" \n",
"* Here's how you can structure the overall implementation for this function\n",
" ```python \n",
" def compute_gradient(X, y, w, b, lambda_=None): \n",
" m, n = X.shape\n",
" dj_dw = np.zeros(w.shape)\n",
" dj_db = 0.\n",
" \n",
" ### START CODE HERE ### \n",
" for i in range(m):\n",
" # Calculate f_wb (exactly as you did in the compute_cost function above)\n",
" f_wb = \n",
" \n",
" # Calculate the gradient for b from this example\n",
" dj_db_i = # Your code here to calculate the error\n",
" \n",
" # add that to dj_db\n",
" dj_db += dj_db_i\n",
" \n",
" # get dj_dw for each attribute\n",
" for j in range(n):\n",
" # You code here to calculate the gradient from the i-th example for j-th attribute\n",
" dj_dw_ij = \n",
" dj_dw[j] += dj_dw_ij\n",
" \n",
" # divide dj_db and dj_dw by total number of examples\n",
" dj_dw = dj_dw / m\n",
" dj_db = dj_db / m\n",
" ### END CODE HERE ###\n",
" \n",
" return dj_db, dj_dw\n",
" ```\n",
" \n",
" If you're still stuck, you can check the hints presented below to figure out how to calculate `f_wb`, `dj_db_i` and `dj_dw_ij` \n",
" \n",
" \n",
" Hint to calculate f_wb\n",
" Recall that you calculated f_wb in compute_cost above — for detailed hints on how to calculate each intermediate term, check out the hints section below that exercise\n",
" \n",
" More hints to calculate f_wb\n",
" You can calculate f_wb as\n",
"
\n",
" for i in range(m): \n",
" # Calculate f_wb (exactly how you did it in the compute_cost function above)\n",
" z_wb = 0\n",
" # Loop over each feature\n",
" for j in range(n): \n",
" # Add the corresponding term to z_wb\n",
" z_wb_ij = X[i, j] * w[j]\n",
" z_wb += z_wb_ij\n",
" \n",
" # Add bias term \n",
" z_wb += b\n",
" \n",
" # Calculate the prediction from the model\n",
" f_wb = sigmoid(z_wb)\n",
"
\n",
" \n",
" \n",
" \n",
" Hint to calculate dj_db_i\n",
" You can calculate dj_db_i as dj_db_i = f_wb - y[i]\n",
" \n",
" \n",
" \n",
" Hint to calculate dj_dw_ij\n",
" You can calculate dj_dw_ij as dj_dw_ij = (f_wb - y[i])* X[i][j]\n",
" \n",
"\n",
""
]
},
{
"cell_type": "markdown",
"id": "d5a86862",
"metadata": {},
"source": [
"Run the cells below to check your implementation of the `compute_gradient` function with two different initializations of the parameters $w$"
]
},
{
"cell_type": "code",
"execution_count": 28,
"id": "ca500a49",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"dj_db at initial w (zeros):-0.1\n",
"dj_dw at initial w (zeros):[-12.00921658929115, -11.262842205513591]\n"
]
}
],
"source": [
"# Compute and display gradient with w initialized to zeroes\n",
"initial_w = np.zeros(n)\n",
"initial_b = 0.\n",
"\n",
"dj_db, dj_dw = compute_gradient(X_train, y_train, initial_w, initial_b)\n",
"print(f'dj_db at initial w (zeros):{dj_db}' )\n",
"print(f'dj_dw at initial w (zeros):{dj_dw.tolist()}' )"
]
},
{
"cell_type": "markdown",
"id": "1643f15d",
"metadata": {},
"source": [
"**Expected Output**:\n",
"
"
]
},
{
"cell_type": "markdown",
"id": "50b93479",
"metadata": {},
"source": [
"##### 2.6 Learning parameters using gradient descent \n",
"\n",
"\n",
"Similar to the previous assignment, you will now find the optimal parameters of a logistic regression model by using gradient descent. \n",
"- You don't need to implement anything for this part. Simply run the cells below. \n",
"\n",
"- A good way to verify that gradient descent is working correctly is to look\n",
"at the value of $J(\\mathbf{w},b)$ and check that it is decreasing with each step. \n",
"\n",
"- Assuming you have implemented the gradient and computed the cost correctly, your value of $J(\\mathbf{w},b)$ should never increase, and should converge to a steady value by the end of the algorithm."
]
},
{
"cell_type": "code",
"execution_count": 30,
"id": "d27a2ef7",
"metadata": {},
"outputs": [],
"source": [
"def gradient_descent(X, y, w_in, b_in, cost_function, gradient_function, alpha, num_iters, lambda_): \n",
" \"\"\"\n",
" Performs batch gradient descent to learn theta. Updates theta by taking \n",
" num_iters gradient steps with learning rate alpha\n",
" \n",
" Args:\n",
" X : (array_like Shape (m, n)\n",
" y : (array_like Shape (m,))\n",
" w_in : (array_like Shape (n,)) Initial values of parameters of the model\n",
" b_in : (scalar) Initial value of parameter of the model\n",
" cost_function: function to compute cost\n",
" alpha : (float) Learning rate\n",
" num_iters : (int) number of iterations to run gradient descent\n",
" lambda_ (scalar, float) regularization constant\n",
" \n",
" Returns:\n",
" w : (array_like Shape (n,)) Updated values of parameters of the model after\n",
" running gradient descent\n",
" b : (scalar) Updated value of parameter of the model after\n",
" running gradient descent\n",
" \"\"\"\n",
" \n",
" # number of training examples\n",
" m = len(X)\n",
" \n",
" # An array to store cost J and w's at each iteration primarily for graphing later\n",
" J_history = []\n",
" w_history = []\n",
" \n",
" for i in range(num_iters):\n",
"\n",
" # Calculate the gradient and update the parameters\n",
" dj_db, dj_dw = gradient_function(X, y, w_in, b_in, lambda_) \n",
"\n",
" # Update Parameters using w, b, alpha and gradient\n",
" w_in = w_in - alpha * dj_dw \n",
" b_in = b_in - alpha * dj_db \n",
" \n",
" # Save cost J at each iteration\n",
" if i<100000: # prevent resource exhaustion \n",
" cost = cost_function(X, y, w_in, b_in, lambda_)\n",
" J_history.append(cost)\n",
"\n",
" # Print cost every at intervals 10 times or as many iterations if < 10\n",
" if i% math.ceil(num_iters/10) == 0 or i == (num_iters-1):\n",
" w_history.append(w_in)\n",
" print(f\"Iteration {i:4}: Cost {float(J_history[-1]):8.2f} \")\n",
" \n",
" return w_in, b_in, J_history, w_history #return w and J,w history for graphing"
]
},
{
"cell_type": "markdown",
"id": "6a70e252",
"metadata": {},
"source": [
"Now let's run the gradient descent algorithm above to learn the parameters for our dataset.\n",
"\n",
"**Note**\n",
"\n",
"The code block below takes a couple of minutes to run, especially with a non-vectorized version. You can reduce the `iterations` to test your implementation and iterate faster. If you have time, try running 100,000 iterations for better results."
]
},
{
"cell_type": "code",
"execution_count": 31,
"id": "ea6e6ab2",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Iteration 0: Cost 1.01 \n",
"Iteration 1000: Cost 0.31 \n",
"Iteration 2000: Cost 0.30 \n",
"Iteration 3000: Cost 0.30 \n",
"Iteration 4000: Cost 0.30 \n",
"Iteration 5000: Cost 0.30 \n",
"Iteration 6000: Cost 0.30 \n",
"Iteration 7000: Cost 0.30 \n",
"Iteration 8000: Cost 0.30 \n",
"Iteration 9000: Cost 0.30 \n",
"Iteration 9999: Cost 0.30 \n"
]
}
],
"source": [
"np.random.seed(1)\n",
"intial_w = 0.01 * (np.random.rand(2).reshape(-1,1) - 0.5)\n",
"initial_b = -8\n",
"\n",
"\n",
"# Some gradient descent settings\n",
"iterations = 10000\n",
"alpha = 0.001\n",
"\n",
"w,b, J_history,_ = gradient_descent(X_train ,y_train, initial_w, initial_b, \n",
" compute_cost, compute_gradient, alpha, iterations, 0)"
]
},
{
"cell_type": "markdown",
"id": "70b0aac4",
"metadata": {},
"source": [
"\n",
"\n",
" Expected Output: Cost 0.30, (Click to see details):\n",
"\n",
"\n",
" # With the following settings\n",
" np.random.seed(1)\n",
" intial_w = 0.01 * (np.random.rand(2).reshape(-1,1) - 0.5)\n",
" initial_b = -8\n",
" iterations = 10000\n",
" alpha = 0.001\n",
" #\n",
"\n",
"```\n",
"Iteration 0: Cost 1.01 \n",
"Iteration 1000: Cost 0.31 \n",
"Iteration 2000: Cost 0.30 \n",
"Iteration 3000: Cost 0.30 \n",
"Iteration 4000: Cost 0.30 \n",
"Iteration 5000: Cost 0.30 \n",
"Iteration 6000: Cost 0.30 \n",
"Iteration 7000: Cost 0.30 \n",
"Iteration 8000: Cost 0.30 \n",
"Iteration 9000: Cost 0.30 \n",
"Iteration 9999: Cost 0.30 \n",
"```"
]
},
{
"cell_type": "markdown",
"id": "73bc1fc7",
"metadata": {},
"source": [
"##### 2.7 Plotting the decision boundary\n",
"\n",
"\n",
"We will now use the final parameters from gradient descent to plot the linear fit. If you implemented the previous parts correctly, you should see the following plot: \n",
"\n",
"\n",
"We will use a helper function in the `utils.py` file to create this plot."
]
},
{
"cell_type": "code",
"execution_count": 32,
"id": "4f008e19",
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAigAAAGdCAYAAAA44ojeAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAA9hAAAPYQGoP6dpAABU80lEQVR4nO3dd3xUVfrH8c8QyBCQEikpgFLiWmiSoAJKs6CrgCtWsMCqMSgqsayIrCaogFgQlV0M0Z+irFhWQOxgAVQENaGLCIouwgSiJoBlE0ju74+zGUlIIGVm7r0z3/frNS+YmQt55maS+8w5z3mOx7IsCxEREREHqWd3ACIiIiIVKUERERERx1GCIiIiIo6jBEVEREQcRwmKiIiIOI4SFBEREXEcJSgiIiLiOEpQRERExHHq2x1AbZSWlrJjxw6aNGmCx+OxOxwRERGpBsuy2Lt3L4mJidSrd+gxElcmKDt27KBdu3Z2hyEiIiK1sG3bNtq2bXvIY1yZoDRp0gQwL7Bp06Y2RyMiIiLVsWfPHtq1a+e/jh+KKxOUsmmdpk2bKkERERFxmeqUZ6hIVkRERBxHCYqIiIg4jhIUERERcRwlKCIiIuI4SlBERETEcWqcoCxbtowhQ4aQmJiIx+NhwYIF5Z63LIvMzEwSExOJiYlhwIABbNiwodwxRUVF3HTTTbRs2ZLGjRszdOhQfvjhhzq9EBEREQkfNU5Qfv31V7p3786MGTMqff7BBx9k2rRpzJgxg88//5z4+HjOOuss9u7d6z8mPT2d+fPn8+KLL/Lxxx/zyy+/MHjwYEpKSmr/SkRERCRseCzLsmr9jz0e5s+fz1/+8hfAjJ4kJiaSnp7OuHHjADNaEhcXx9SpU0lLS2P37t20atWK559/nksvvRT4ozPsW2+9xdlnn33Yr7tnzx6aNWvG7t271QdFRETEJWpy/Q5oDcrWrVvJy8tj0KBB/se8Xi/9+/dn+fLlAOTk5LBv375yxyQmJtKlSxf/MRUVFRWxZ8+ecje3sqwSCgqWsHPnXAoKlmBZGjUSERGpKKCdZPPy8gCIi4sr93hcXBzff/+9/5jo6GhiY2MPOqbs31c0ZcoUJk6cGMhQbZGfP48tW8ZSVPRHvY3X25akpMdo1WqYjZGJiIg4S1BW8VRsYWtZ1mHb2h7qmPHjx7N7927/bdu2bQGLNVTy8+exYcNF5ZITgKKi7WzYcBH5+fNsikxERMR5ApqgxMfHAxw0ErJr1y7/qEp8fDzFxcUUFBRUeUxFXq/Xv++OG/ffsawStmwZC1RW7mMe27IlXdM9IiIi/xPQBKVDhw7Ex8ezePFi/2PFxcUsXbqUPn36AJCSkkKDBg3KHePz+Vi/fr3/mHBTWPjRQSMn5VkUFW2jsPCjkMUkIiLiZDWuQfnll1/YsmWL//7WrVtZvXo1Rx55JEcddRTp6elMnjyZY445hmOOOYbJkyfTqFEjRowYAUCzZs245ppruO2222jRogVHHnkkt99+O127duXMM88M3CtzkOJiX0CPExERCXc1TlC++OILBg4c6L9/6623AjBy5EieffZZ7rjjDn7//XduuOEGCgoKOOWUU1i0aBFNmjTx/5tHH32U+vXrc8kll/D7779zxhln8OyzzxIVFRWAl+Q80dEJAT1ORALL5/ORlZVFWloaCQn6ORRxgjr1QbGL2/qgWFYJK1a0p6hoO5XXoXjwetvSq9dWPB7nJGn6pS2RIjc3l5SUFHJyckhOTrY7HJGwZVsfFKmcxxNFUtJjZfcqPgtAUtJ0RyUnYBKUiRMn4vNp6klEREIroH1QpGqtWg2jc+d/V9EHZbr6oIiEmM/n8yffubm55f4ESEhI0MihiI2UoIRQq1bDaNnyfAoLP6K42Ed0dALNm/d11MiJfmlLbbltSjArK+ugBpCpqan+v2dkZJCZmRniqESkjGpQpJzMzMxDdu3VL22pKhFxWx1HxWQ8NTWV7Oxsf+zhnIy7LZmU8FGT67dGUKSctLQ0hg4dClT9S1siW1lt0tChQ139fqgsAUlOTq5zcuWGi3+4fA8lvClBkXKC9UtbwpOmBA+mi79IYChBCSDLKnF0fYlIbVWViHz++efMmjWr3LFurONISEggIyMjrBMKJZPiNkpQAiQcdyqOhF/aUj2HKyi97rrrOOmkk1w7JZiQkFCnRMoNF38VBYvbqEg2AMp2Kj64CZvpcdK5879dm6SIQPUKSn0+n6uKZAPJDcXlkVwULM6hItkQOvxOxR62bEmnZcvzNd0jrlWd2qRIbujnhuJy1ZeJ2yhBqaOa7FQcGzsgRFGJhF4kTwnq4i8SeEpQ6kg7FUukqSoRqWsdh4ROJCeT4h5KUOpIOxVLpFEicmhuuPjreyhuoCLZOnLrTsUiIiKhpt2MQ8itOxWLiIg4mRKUACjbqdjrbVPuca+3rZYYi4iI1IJqUALEDTsVi4iIuIUSlADyeKK0lFhERCQANMVTwf79dkcgIk7j8/nIzMyM6GZ0IqGmBOUA+/ZBv34wYQIUF9sdjYg4RdkOxUpQREJHCcoBXn8dPv0UJk+GXr3gyy/tjkhERCQyKUE5wLBh8MorcOSRsGoVJCfD9OlQWmp3ZCISaj6fj9zcXP8NKHc/EkZTNLUldlKCUsFFF8H69XDOOVBUBLfcAmedBdu22R2ZiIRSVlYWKSkppKSkkJqaCkBqaqr/saysLJsjDD5NbYmdlKBUIiEB3noLZs6ERo3ggw+ga1d44QVwX99dEamNtLQ0cnJyyMnJITs7G4Ds7Gz/Y2lpaTZHKBLetMy4Ch4PjB4Np58OV14Jn30Gl18Or71mEpcjj7Q7QhEJpkjdodjn8/lHTA6c2ipT2XkRCQaNoBzGn/4En3wCEydCVBS8/LIZTVm0yO7IREQCT1Nb4hRKUKqhfn245x6zwudPf4IdO+Dss+Gmm+C33+yOTkSCzQ07FAdKZVNbQ4YM4Z133tHUloSUdjOuod9+g3HjYMYMc//YY+H55+Gkk0IahohI0OXm5pKSkgJATk5O2E9vSfBpN+MgatQInngC3nnHFNNu2gS9e8O996oLrYgEnpb6SqRSglJLZ59tliNffDGUlEBGBpx2Gnz9td2RiUg4sWOpb1kPmPz8fIYMGQJEXg8YsZ8SlDo48kh46SWYMweaNYOVK6FHD7PKx30TZyIiRlmh7DnnnMPrr78OqFBWQk/LjOvI4zHLj/v1g1GjTM+UG24wbfOfftpMA4mI1ITdS33T0tIYOnSo/+umpqaSnZ3tr0GJhGJhsZ+KZAOotBQefxzuvNN0oW3RArKy4MIL7Y5MRNzC5/MxfPhwli5dWuUxGRkZZGZmhiSeskJZFclKIKhI1ib16kF6OuTkmKmen34yrfNHjoTdu+2OTpxEhY9SFZ/Px9KlS5kzZ4662EpEU4ISBJ07w4oVcNddJml57jno1g2WLLE7MnEK7XEih3P88ceX61xb9vfk5OSQTrFEUg8YcRYlKEESHQ2TJsGyZdCxI/znP6Zt/u23w3//a3d0IuIkh9o5eePGjbbGlpCQQGZmphKUKmg0NHhUJBtkp54Kq1fDbbdBdjY88gi8+65Z+dO9u93RhRefz0dWVhZpaWmO/GVqd+GjOFdWVhYTJ04s91hZm3mA/v37673hUGWjoUOHDtX3KMA0ghICTZrArFmwcCG0bm36p5x0EkydanqoSGA4fdpEe5xIVQ63c/LcuXN18ZOIoxGUEBoyBNatg+uuM7si33knvPGGqVHp0MHu6CTYtHRTqhKpOye7lUZDQ0MJSoi1bg3z58Mzz8DYsfDxx6aA9vHHTR8Vj8fuCN3FTb8odBESCQ+Hm5IL5TLwcKYpHht4PHD11bB2rWmP/8sv5v4FF8CuXXZH5y6aNpFwc7hVMyrKDL2K5/xwU3JaBh4YatRms5ISePhhuPtu2LfPjLA89ZSZDpLDqziCUtm0iVNGUA7k9IJecS41Tgu9Q51zfT9qpibXb03x2CwqCsaNM5sPXnEFbNgAQ4dCaipMmwZHHGF3hNVnx0XXrdMmZUs3RUSkcpricYgTT4QvvoBbbzVTQNnZZhny8uV2R1Z9Tl9FI+JWh+qTot2Fg6O659wNjexcOy1oudDu3bstwNq9e7fdoQTFBx9YVrt2lgWWVa+eZd11l2UVFdkd1eHl5ORYgJWTk2PL19+xY4eVkZFh7dixw5avLxIsGRkZFlDlLSMjw+4Qw044nXO7fzcfqCbXb9WgONTu3XDTTfD88+Z+jx6mudsJJ9gbV0VurQGRw1OdjHPo5yz0wumcO6lORjUoYaBZM9MfZehQSEuDVasgOdk0d7vpJrPHjxNouV34UodM53BrrZWbuf2cu6kFQ1UccpmTqlx0kek8e845UFRkdkseNAi2bbM7MkPL7USCy7X1A2KrcGjBoBEUF0hIgLfegiefNHv6vP8+dO0K//wnDB9ub3M3t3/KkPLC4VPXobhx2qriSJYbijLDjRvPeVh0rg56RUwQhHuR7KFs2mRZJ59sCmjBsi691LJ++snuqAwnFWJJ7YRTYWBl3PgedWPM4ixOeg/V5PqtERSX+dOf4JNPYPJkuPdeeOkl+Ogj0zp/0CB7Y3PjpwwpLyw+dYWBcB/JEqkOJSguVL8+3HMP/PnPprnb11+bRm833miKaBs1sicuNR9zv3CcsnPLxf7A6ScVn0sgufXDo5YZu9xvv5lOtDNmmPvHHmuWI/fsaW9c4n5OWppYF5mZmQdd7A/klIv9gec7ISEhbJa4ihxIy4wjSKNG8MQTMHgw/PWvsGkT9O5t9va56y4z2iJSG2791FWRG6etwnEkS6SmdPkKE2efDevWwfXXwyuvQEaGWfnz/PNwzDF2RyduFC5Tdk6+2Fdn+kkkUqkPShhp0cIUzc6ZYxq9rVxp9vh58kmz5kdEnKU6vSrCZSRLpKZUgxKmtm2DUaPggw/M/T//GZ5+2vRUEYlUTuuDEk7t1EWqoybXbyUoYay0FB5/HO6803ShbdECsrLgwgvtjkxEKgqXomSRQ6nJ9VtTPGGsXj3TGj8nx0z1/PSTaZ0/cqTZjFBERMSplKBEgM6dTT3K+PEmaXnuOejWDZYutTsyESmjWhOR8jTFE2E++QSuugq+/dbs4XPbbXDffdCwod2RiYhIuNMUj1Tp1FNh9Wq49lqzsufhh+Gkk2DNGrsjExER+YMSlAjUpAlkZ8Nrr0Hr1rB+vUlSHnwQSkrsjk5EREQJSkQbOtQ0dxs6FPbtMy3zBw6ErVvtjkxERCKdEpQI17o1LFhgeqQccYTZGblbN7M7svuqk0REJFwoQRE8Hrj6alOHctpp8Msv5v6wYZCfb3d0IiISiZSgiF/HjrBkCTzwADRoYEZWunSBN96wOzIREYk0SlCknKgoU4vy2Wemf8quXTBkCFx3nRlZEQkkn89HZmamv927iEgZJShSqRNPhC++gFtvNVNA2dnmseXL7Y5MwonP52PixIlKUETkIEpQpEoNG8Ijj8D770O7dvDNN9C3L0yYAMXFdkcnIiLhLOAJyv79+/n73/9Ohw4diImJoWPHjtx7772Ulpb6j7Esi8zMTBITE4mJiWHAgAFs2LAh0KFIgAwcCGvXwpVXmg0IJ0+GXr3gyy/tjkzcyOfzkZub678B5e5rNEVEALAC7P7777datGhhvfHGG9bWrVutV155xTriiCOs6dOn+4954IEHrCZNmlivvvqqtW7dOuvSSy+1EhISrD179lTra+zevdsCrN27dwc6fDmMl1+2rCOPtCywLK/XsqZPt6ySErujEjfJyMiwgCpvGRkZdocoIkFSk+t3wPfiGTx4MHFxcTz99NP+xy688EIaNWrE888/j2VZJCYmkp6ezrhx4wAoKioiLi6OqVOnkpaWdtivob147LVjh1mG/O675v4ZZ5i+Ke3a2RuXuIPP5/OPkuTm5pKamkp2djbJycmA2TRPG+aJhCdb9+I57bTTeP/99/n6668BWLNmDR9//DHnnnsuAFu3biUvL49Bgwb5/43X66V///4sVwWmKyQmwttvwz//CTExpkala1d44QW7IxM3SEhIIDk52X8Dyt1XciIiEIQEZdy4cQwfPpzjjjuOBg0a0KNHD9LT0xk+fDgAeXl5AMTFxZX7d3Fxcf7nKioqKmLPnj3lbmIvjweuv95sPHjSSRa7d8Pll8OwYbv46Sdt6CMiInUT8ATlpZdeYs6cObzwwgvk5uYye/ZsHn74YWbPnl3uOI/HU+6+ZVkHPVZmypQpNGvWzH9rp7kEx4iNnccjj3Rk1KgM6tXbz/z5rTnuuHxeeeVju0MTF0hISCAjI0OjJiJykIDXoLRr144777yTMWPG+B+7//77mTNnDl999RXffvstnTp1Ijc3lx49eviPOf/882nevPlBiQyYEZSioiL//T179tCuXTvVoNgsP38eGzZchKlthK++6snkyXPYtu1YAK69dguPPZZEo0Y2BikiIo5haw3Kb7/9Rr165f/bqKgo/zLjDh06EB8fz+LFi/3PFxcXs3TpUvr06VPp/+n1emnatGm5m9jLskrYsmUsZckJwHHHfcGsWT34y19mAPDUU0kkJ1t88YVNQYqEiDriigRewBOUIUOGMGnSJN58802+++475s+fz7Rp07jgggsAM7WTnp7O5MmTmT9/PuvXr2fUqFE0atSIESNGBDocx7OsEgoKlrBz51wKCpZgWe6o3ygs/Iiioh8Oerxhw98ZO/Ympk49mxYtdrBpk4feveG++2D/fhsCFQkBdcQVCbz6gf4Pn3jiCe6++25uuOEGdu3aRWJiImlpadxzzz3+Y+644w5+//13brjhBgoKCjjllFNYtGgRTZo0CXQ4jpafP48tW8aWu9B7vW1JSnqMVq2G2RjZ4RUXH/oX8cknL+Lpp7vy9NOreP31o7jnHnjzTXj+eTjmmBAFKY7n8/nIysoiLS1NdSgiUk7Aa1BCIRz6oFSs3/iDKRTu3Pnfjk5SCgqWsGbNwMMe163bh7z99gDGjIHdu6FRI5g2zWw+WEVNtESQ3NxcUlJSyMnJ8S85dgv1cxGpOVtrUOTwKqvfOOBZALZsSXf0dE/z5n3xettSllAdzIPX247Y2L5cfjmsWwennw6//QajR8PgwaDRcHGzrKwsUlJSSElJITU1FYDU1FT/Y1lZWTZHKOJuSlBsUFX9xh8sioq2UVj4UchiqimPJ4qkpMfK7lV8FoCkpOl4PFGA6TK7eDE8+ih4vfDWW6a526uvhi5mcYZw2YsnLS2NnJwccnJyyM7OBiA7O9v/WHW6YouAiqyrogTFBoer36jpcXZp1WoYnTv/G6+3TbnHvd62lU5R1asH6emQkwMnngg//QQXXQQjR5rpH4kM4TLyoI64Eigqsq5cwItk5fCio6v3i6u6x9mpVathtGx5PoWFH1Fc7CM6OoHmzfv6R04q07kzrFwJmZkwdSo89xwsWWL+7N8/ZKGLTdLS0hg6dChQde2GiIgSFBuU1W8UFW2n8joUD15vW5o37xvq0GrF44kiNnZAjf5NdDRMngznnQdXXQXffgsDB8Jtt5klyQ0bBiVUcYDKikcPHIVwI3XELU+rsw6vYpH1gX+CiqxBUzy2qGn9RjDZ3Yfl1FPNfj7XXguWBQ8/DCefDGvXhjQMkTpJSEggMzMz4i8oZTRlcXjhMtUZTEpQbFLT+o1gyM+fx4oV7VmzZiAbN45gzZqBrFjRnvz8eUH/2gdq0gSys+G116BVK7Pip2dPePBBKHHuQiZXc0pRnkYeJFKpyPrw1AfFZpZVUqP6jUBxah+WXbsgNRUWLjT3+/Y1tSnt24c8FCB8h6rd3H9EnEl9YWovkn4e1QfFBcqmVnbtehmA1q0vITZ2QMimdZzah6V1a1iwAJ5+Go44Aj76CLp1g2efNVNAoaahapHq0ZSFBJqKZG1gd4v7mvRhqWnxayB4PHD11TBggCmg/eQT+OtfzRTQrFlmGkhqTkV5UlEgRwi1Oqv2NNVZOSUoIVbV1EpR0XY2bLgoJFMrbunD0rEjLF0KDz0E99xjRlaWLzejK4MHB+/rhuuFPCsri4kTJ5Z7rOyTLkBGRgaZmZkhjkrsVDZCOHTo0Dq/p8NxdVaolBVZS3ma4gkhp0ytuKkPS1QU3HknfPaZ6Z+yaxcMGQJpafDLL8H5muE6VK2iPBFxE42ghJBTplbc2IflxBPhiy9gwgSz2eCsWfD++6aAtk+fwH6tcB2q1idcgdCMEGrKQgJBCUoIOWVqpawPi5lq8lA+SQltH5aaaNgQHnnETO+MHAnffGNW+Ywfb6aAoqMD83V0IZdwVp2pvrS0tDrVpmjKQgJBUzwh5KSpFSf0YamtgQNNI7crr4TSUpg0CXr3hi+/tDsy99An3MhVnak+rV4TJ9AISgg5bWqlNvvoOEXz5mZ6Z8gQGD0acnMhJcXs7XPjjWZjwkAI1wu5PuFGruqMECoxESdQghJCTpxaqc0+Ok5y8cWmXf7VV8O778LYsabJ27PPQtu2df//dSGXSBGuq9fEvTTFE2JunlpxqsREePtt+Oc/ISbGFM927Qpz59odmYizHThCGK6r18S91OreJtVtcW9XK3y32rTJ1KZ8/rm5f9ll8I9/wJFH2huXW4Vrq385mFrVSyjU5PqtBMXB7O4461b79sHkyXDffWazwcREM+Vz1ll2R+Y+Ve0RosQlvEXS3jASWtqLJwyUdZyt2DelrONsqHccdpMGDSAjw3Sd/dOfYMcOGDQIbr4ZfvvN7ujCg1Z5iEiwKUFxIKd0nHW7k0+GVatgzBhz/4knzEqfL76wNy6n8/l85Obm+m9AuftKSsJfuK5ek+rz+XxkZmba+vOuKR4HKihYwpo1Aw97XPfuH7p6BU4ovfOOWenj80H9+qax2/jx5u9SXmZm5kGNvA503XXXcdJJJ6lGQSSMBWuaT1M8LueUjrPh5JxzYN06syx5/36ToJx2GmzebHdkzlNVI6/rrrsOgFmzZmmVh4gEnT4/OpCTOs6GkxYt4KWX4PzzzbTPypVmj59p0+C668DjsTtCZ6iqkdd5553n31AwnPYoEhHDab1wlKA4kN0dZ8N5abPHA5dfbvbwGTUKPvzQdKJduBCeegp0ja2a9igSCW/V2acplI0rlaA4kJ0dZyNlafNRR8F778Fjj5lalLfeMs3dZs2CYeHzMutMxZIikcNpO7mrSNbBKk8W2pGUND0oyULZ0uaDR21MUhSunW43bIArroDVq839kSNN4tKsma1hOZr6oIiENycUySpBcbhQTbdYVgkrVrQ/qO/KH8y0Uq9eW8NmuudAxcWQmWk2GywthaOPhtmzoX9/uyMTqRkljxIITkhQtIrH4co284uLG05s7ICgJQeFhR8dIjkBsCgq2kZh4UdB+fp2i4423WeXLYOOHeH772HgQPjb36CoyO7oRKpPTfQkEJwwvasERQAtbS5z6qlmqufaa8Gy4OGH4aSTYO1auyMTEQmdsp3claCI7bS0+Q9NmkB2Nrz2GrRqZfqnnHQSPPSQ2dtHquaE7pNuE4hzpu6/taP3q7MpQRHgj6XNZQWxB/Pg9bYL2tJmJxo6FNavN38WF8Mdd5hpn+++szsy59L0Qs0F4pxlZWX5G+apiV716f3qbEpQBPhjafP/7lV8Fgje0mYna90aFiwwPVKOOAI++gi6dTO7Ix9YXq5PYmKnqrr/lj1W1mBPxE3UB0X8WrUaRufO/66iD0pwlja7gccD11xjRk+uugo++QT++lfT3C0ry0wDlX0SGzp0aMStnHBa90k3CPQ5UxO96tP71UUsF9q9e7cFWLt377Y7lLBUWrrf+vnnD628vBesn3/+0Cot3W93SI6xf79lTZliWQ0aWBZYVlycZb3+umXl5ORYgJWTk2N3iCGXkZFhYZrnVHrLyMiwO0THCeY5q/he3LFjh5WRkWHt2LEjQNG7W6S9X6v7/Q/V+6Qm12/1QRGphVWr4LLL9vH11w0A6NdvI8uWnUR29vSI29234idS7XJ8eME8ZxX7oASrn4VbRdr7tbrf/1C9T2py/dYUj0gt9OgBF188lUmTYoDbWLbseGA1qalXAqZIMdT7VthF0ws1F8xzVrY8VCqn96t7KEERqaUxY65h2DAfn332NRMmJPLzz0l4PJ9w9dU7SU3N46ij4u0OUSKU6iwiW3W//45/nwR1silIVIMiTrNkyWoLnrPM2h7LSk62rC+/tDuq0FO9Q80F45xFWp1FbYXr+7W633873ieqQREJsbL52wce+IYHH+zIzz9Dw4Zmb58bb4R6WtAvIRRpdRZSXnW//3a8T1SDIhJiZftWXHVVDFdeCVdfDe++C2PHwuuvwzPPQNu2dkcpkUJ1FpGtut9/p79P9LlOJAAO3LciMRHefhv+8Q+IiYH33oOuXWHuXLujFBFxDyUoIkHg8cANN5jlyCedBIWFMGIEDB8OP/9sd3QSSZywK63Yp7rffye+T1SDIrazrBIKCz+iuNhHdHQCzZv3DauW+vv2weTJcN99ZrPBNm3MlM9ZZ9kdWXiq2AdERJyjJtdvjaCIrfLz57FiRXvWrBnIxo0jWLNmICtWtCc/f57doQVMgwaQkQHLl8Of/gTbt8OgQXDzzfDbb3ZHF360AZxIeFCCIrbJz5/Hhg0Xldv3B6CoaDsbNlwUVkkKwMknmymfMWPM/SeegJQU+OILe+MSEXEiJShiC8sqYcuWsZjl9gc9C8CWLelYVklI4wq2Ro1gxgxTRJuQAF99Bb17w/33w/79dkfnXj6fj9zcXP8NKHdfoyki7qMERWxRWPjRQSMn5VkUFW2jsPCjkMUUSuecA+vWwUUXmcTk7ruhb1/YvNnuyNwpKyuLlJQUUlJSSE01Ww2kpqb6H8vKyrI5QnELn89HZmamLUmtnV/biZSgiC2Ki6v3A1jd49yoRQt4+WV4/nlo2hRWrIATT4SsLNOPVqovLS2NnJwccnJyyM7OBiA7O9v/WFpams0RilvYWcOk+qny1KgtgjhptUx0dPVWV1T3OLfyeOCKK6BfPxg1Cj78EEaPhoUL4emnIV7b+VSL0xtOiUjNKUGJEPn589iyZWy5aRWvty1JSY/RqtWwkMfTvHlfvN62FBVtp/I6FA9eb1uaN+8b6tBscdRRpqHbY4/B+PHw1ltwwgmlnHXWK0yf3k/LZUWCyM5N8xy/YZ+N1AclApStljk4EfAA0Lnzv21JUv6IC8rHZm9cdlu/3oyqrFlj7g8e/BNz5rSgWTN743IL9UGRmsrMzGTixIlVPp+RkUFmZmbYfW071OT6rQQlzFlWCStWtD9EQaoZqejVa6st0z2Vj+y0IylpekQmJ2WKi2H06DyeeaYVEMXRR8Ps2dC/v92RiYQfOzdXjLSNHbVZoPjVZLVMbOyAEEX1h1athtGy5fmOqY2x24G/rPr0yeWZZ56lZcu3+P77pgwcaDF69K88+ugReL02ByoSRuysYVL9VNWUoIQ5N6yW8XiibEmOnCgrK+ug4d4ff2wDPIplXcvMmUfw8ccwZw5062ZPjCIioaBlxmFOq2XcpfLlso+Sk5PMtGnf0KJFCevWmQ0IH3rI7O0jIoFj56Z5Ttywz06qQQlzf9SgHHq1jF01KOEgWMu3c3NzSUlJIScnxz/cu2sXpKaaZchglifPng3t29f5y4mIBJ02CxQ/jyeKpKTHyu5VfBaApKTpSk5qKdSbHbZuDQsWwFNPwRFHwLJlZqrn2WfV3E1EwosSlAjQqtUwOnf+N15vm3KPe71tI3YpbyAEe7PDqoZ7PR645hqzDPnUU2HvXvjrX+HCCyE/v05fUkTEMTTFE0Gc1EnW7ZyyfLukxNSi3HMP7NsHcXGmA+155wXtS4qI1JqmeKRSZatl4uKGExs7QMlJHThls8OoKLjzTli5Ek44AXbuhMGDIS0NfvklqF9aRCSolKCI1ILTlm/36AE5OXDrreb+rFlm48FPPw3JlxcRCTglKCK14MTl2w0bwiOPwPvvQ7t28M03cNppcPfdZvpHRMRNlKCI1ELZZocHr4wq48HrbWfLZoennw5r15r9fEpL4f77oVcv2Lgx5KGIiNSaEhSRWnD68u3mzeH55+GllyA2FnJzITkZHn/cJC0iIk6nBEWkltywfPuSS8zuyGefDf/9L4wda/7+w6Hqe0VEHEAJikgdtGo1jF69vqN79w857rg5dOr0KB07TqF+/SOxLGf0oU9MhLffhn/8A2Ji4L33oGtXmDvX7sgCw+fzkZmZ6d9kUURqzok/R0pQROrI44li//6f2br1Tr755hY2brwi6B1la8rjgRtugFWrzD4+hYUwYgQMHw4FBXZHVzc+n4+JEyc66heriNs48edICYrNLKuEgoIl7Nw5l4KCJY751C3VF+yOsoF07LHwySeQkWF6qLz4ohlNWbzY7shERMqrb3cAkSw/fx5btowtd2HzetuSlPSYI+oX5PAsq4QtW8ZS+UaMFuBhy5Z0WrY83zGN8Ro0gMxMOPdcs9Jn82YYNAhuvhkeeMBMAzmdz+fzf9LLzc0t9yeYbQK0I6zIoTn95ygoIyjbt2/niiuuoEWLFjRq1IgTTzyRnJwc//OWZZGZmUliYiIxMTEMGDCADRs2BCMUx3LTp+5wV5dRLKd0lK2Nk082Uz433GDuP/64WelzwI+qY2VlZZGSkkJKSgqpqakApKam+h/LysqyOUIR53P6z1HA9+IpKCigR48eDBw4kOuvv57WrVvzzTff0L59ezp16gTA1KlTmTRpEs8++yx/+tOfuP/++1m2bBmbNm2iSZMmh/0abt+Lxyn7uEjdR7F27pzLxo0jDnvc8ce/QFzc8DrFGkzvvGM2HMzLg/r1zRTQnXeavztRxU9+qampZGdnk5ycDNj/yU/EDez4OarJ9Tvgv36mTp1Ku3bteOaZZ/yPtW/f3v93y7KYPn06EyZMYNgwcwGYPXs2cXFxvPDCC6SlpQU6JMepyafu2NgBIYoq8pSNYlWcnikbxarOUmGndZT1+XxkZWWRlpZWo18s55xjliOPHg3//rfpPvvmm6aXSlJSEAOupcp+cSYnJ/t/sYrI4Tn95yjgUzwLFy6kZ8+eXHzxxbRu3ZoePXqQnZ3tf37r1q3k5eUxaNAg/2Ner5f+/fuzfPnySv/PoqIi9uzZU+7mZk7bxyUSHb52BLZsST/sdI/TOsrWpRK/RQt4+WWTlDRtCitWQPfukJUF7tvzXETcLuAJyrfffsvMmTM55phjePfddxk9ejQ333wzzz33HAB5eXkAxMXFlft3cXFx/ucqmjJlCs2aNfPf2rVrF+iwQ8ppn7ojUaBqR5zeUbamPB5TOLtuHQwcCL/9ZkZVBg820z9OlJCQQEZGhqZ0ROrAiT9HAU9QSktLSU5OZvLkyfTo0YO0tDRSU1OZOXNmueM8nvK/zC3LOuixMuPHj2f37t3+27Zt2wIddkg57VN3JArkKJbdHWV9Ph+5ubn+G1Dufm1GU446yjR0mzYNvF546y3o0gXmObB2OyEhgczMTEf9YhVxGyf+HAU8QUlISOCEE04o99jxxx/Pf/7zHwDi4+MBDhot2bVr10GjKmW8Xi9NmzYtd3OzcPvU7UaBHsU6sKPs8ce/QPfuH9Kr19aQLBcPViV+vXpwyy3wxRdmquenn+DCC2HUKNi9O4AvQESkEgFPUE499VQ2bdpU7rGvv/6ao48+GoAOHToQHx/P4gM6QxUXF7N06VL69OkT6HAcy+5P3ZEuGKNYHk8UsbEDiIsbTmzsgJAlmGlpaeTk5JCTk+Ov98rOzvY/VtfC8y5dYOVKs6rH44HZs03CsmxZIKIXEalcwFfx3HLLLfTp04fJkydzySWX8NlnnzFr1ixmzZoFmKmd9PR0Jk+ezDHHHMMxxxzD5MmTadSoESNGHH65Zjhp1WoYLVueT2HhRxQX+4iOTqB5874aOQmBslEss4rHQ/liWXeNYoWiEt/rhSlT4Lzz4KqrYOtWGDAAbr8d7rvPPC8iEkgBH0E56aSTmD9/PnPnzqVLly7cd999TJ8+ncsvv9x/zB133EF6ejo33HADPXv2ZPv27SxatKhaPVDCjV2fukWjWLVx2mmwZg1cc41Z2fPQQ2Zvn7Vr7Y5MRMJNwBu1hYLbG7WJs1hWSdiMYtW2D0ptvPYapKZCfj5ER8P998Ott5o9fkREKlOT67cSFBGptV27TJKycKG536+fqVE5oDejiIhfTa7f2s1YRGqtdWtYsACeegoaNzaFs926mSTFfR99xKl8Ph+ZmZm1WjIv7qUERUTqxOMxNSlr1kCfPrB3r1mKfNFF8OOPdkcn4aAuHZLFvZSgiKPUZWdhsVenTmYEZfJks8ngvHlmifKbb9odmYi4kUP3KpVIVNedhcV+UVEwfrzZfPCKK+DLL02b/LQ0ePhhOOIIuyMUJzlUUXfFnXYP/BO0Y3Uk0AiKOELZzsIV98cp21k4P9+BPdalSj16QE6O6UQLZsPBHj3MBoQiZQ41dROsDsniHkpQxHaB2llYnKVhQ7OXz/vvQ9u2sGULnHoq3H037Ntnd3TidMHukCzOpykesV1NdhaOjR0QoqgkUE4/3eyOfNNNMGeO6Zfy9tvw/PNw/PF2RyehVt2pm1B0SBZn0wiK2C6QOwuLMzVvbhKSl16C2Fgz/ZOcDI8/DqWldkcnoaSpG6kuJShiu0DvLCzOdcklsH49DBoE//0vjB0LZ58NPxxqAE3CSm2mbhISEsjIyFBRbIRRJ1mxnWWVsGJFe4qKtlN5HYoHr7ctvXptdW0LeinPsuCf/4S//Q1+/92MsMycCZddZndkEkq5ubmkpKSQk5OjqZsIoU6y4iplOwv/717FZwH37Cws1ePxwJgxsGqV2WywsBCGDze3ggK7oxMRJ1CCIo6gnYUj07HHwiefQEaG6aHy4ovQtSu8957dkUkoaOpGDkVTPOIo4bSzsNTMZ5+Z5m6bN5v7N98MDzwAMTH2xiUigaMpHnEtjyeK2NgBxMUNJzZ2gJKTCHLyyWbK54YbzP3HHzcrfXJy7I1LROyhBEVEHKNxY/jHP+CttyA+Hr76Cnr1gkmTYP9+u6MTkVBSgiIijvPnP5vlyBddZBKTv/8d+vY13WhFJDIoQRERR2rRAl5+2TR4a9rU7OPTvTvMmmWWKYtIeFOCIiKO5fGYwtl162DAAPjtN7Mz8pAhkJdnd3QiEkxKUETE8Y46ymw6+MgjEB0Nb74JXbrAvDpucu3z+cjMzKx0N10RsZcSFBEHsKwSCgqWsHPnXAoKlmjn5krUqwe33mpW9XTvDj/9BBdeCH/9K+zZU7v/0+fzMXHiRCUoIg6kBEUikpMSgvz8eaxY0Z41awayceMI1qwZyIoV7cnPr+PwQIiF6px26QIrV8Kdd5opoGefhW7dYNmyoHw5EbFJfbsDEAm1/Px5bNkylqKiP3ao83rbkpT0WMg71ubnz2PDhououAdRUdF2Nmy4yDVddEN9Tr1emDIFzjsPrrwSvvvO1Kjcfjvcd595vio+n88/YpKbm1vuTzDdTdXZVMR+6iQrEaWqhKBsz59QJgR/bJJY1Va+7tgk0e5zuncv3HILPP20ud+tm1n5061b5cdnZmYyceLEKv+/jIwMMjMzAx+oiKiTrEhlLKuELVvGUvmOyeaxLVvSQzbdU1j40SGSExNTUdE2Cgs/Ckk8teGEc9qkCTz1FCxYAK1awdq1ZgPChx6Ckkq+bFpaGjk5OeTk5JCdnQ1Adna2/7G0tLSgxSoi1acERSKG0xKC4uLqFWZW9zg7OOmcnn++WY48ZAgUF8Mdd8Dpp5vpnwMlJCSQnJzsvwHl7mt6R8QZlKBIxHBaQhAdXb0LYXWPs4PTzmlcHLz2GmRnm7b5y5aZqZ7Zs9XcTcRtlKBIxHBaQtC8eV+83raU1WoczIPX247mzfuGJJ7acNo5BbOy59prYc0a6NPH1KiMGmXa5v/4Y/ljExISyMjI0KiJiAMpQZGI4bSEwOOJIinpMf/XrhgLQFLSdEcXyDZr1ocGDVod4gj7kqxOncwIyuTJUL++aerWpYvZiLBMQkICmZmZSlBEHEgJikQMJyYErVoNo3Pnf+P1tin3uNfb1vFLjPPz57FyZSf27cuv4gj7k6yoKBg/Hj77DE44AXbuNEuTR4+GX36xJSQRqSYtM5aIU3nPjnYkJU33JwSWVUJh4UcUF/uIjk6gefO+Qb3Ihvrr1VXVS4v/UPGc2u2//4W77oJHHzX3k5LMcuReveyNSySS1OT6rQTlAG67SEjtHep77aRGbk50+P4t0KBBK3r3/oF69aJDGFn1vP++qUn54QfTPn/CBLj7bmjQwO7IRMKfEpRa0EVJwP6mY25QULCENWsGHva47t0/JDZ2QNDjqY3CQrjxRvjXv8z9lBQzmnL88baGJRL21KithsouShU/EZa1G3fbnihSO05oOuYGTltaXBvNm8OcOfDSSxAbazYgTE6GJ56A0lK7oxMRUIKii5L4OanpmJM5cWlxbV1yiWnuNmiQqVG5+WY45xzYvt3uyEQk4hMUXZSkTDiMDISC05Zr11WbNvDOOzBjBsTEwOLFZjnySy/ZHZlIZIv4BEUXJSlT3U/8v/++OciROJsTl2vXlccDY8bAqlVmH5/CQrjsMhgxAgoK7I5OJDJFfIISTsPVUjfNm/clOrrtYY/bsSM74qf83Ny/5VCOPRY++QQyMkwPlblzoWtXeO89uyMLHz6fj8zMTHw+feiTQ4v4BCXchqul9jyeKBISUg97XHHxD5rywyQpvXp9R/fuH3L88S/QvfuH9Oq11bXJSZkGDSAz0yQqxxxj6lHOOgvGjoXff7c7Ovfz+XxMnDhRCYocVsQnKOE4XC2116jRMdU6TlN+hscTRWzsAOLihhMbOyCsfk5OOcVM+Vx/vbn/+ONmOXJOjr1xiUSKiE9QIHyHq6XmNOUnB2rcGP75T7N/T3w8bNxoOs9OmgT799sdXXlOnjrx+Xzk5ub6b0C5+06MWeynRm0HcEonWafEEYn+6JK6ncqXnnvwetvSq9dWfU8izI8/mj18Xn3V3O/dG557zrTMd4Lc3FxSUlLIyckhOTnZ7nDKyczMZOLEiVU+n5GRQWZmZugCEtvU5PpdP0QxuULZcLWd1NHWXmVTfqabrIfyScofU35gOqoqiYwcLVvCK6+YBm833giffgonngjTpkFqqlkJJJVLS0tj6NChgEmkUlNTyc7O9idS2k1aKqMExUGqarNe1tFW002hUTblV3miOB3goL1olEQ6U6BHIz0euPJK6N8fRo6EJUsgLQ0WLoSnnjLTQKHk8/n80yMHTp2USUhIcMTFv7I4kpOTHTfSI86iGhSHUEdbZ6lqhQqgbRFcIj9/HitWtGfNmoFs3DiCNWsGsmJF+4B8j446ymw6+MgjEB0Nb75pliPPnx+AwP+nOjUlWVlZpKSkkJKSQmqqWYGWmprqfywrKytwAYmEmGpQHCIcNmALd4ffxVf1KU4Ryk0f16+HK66ANWvM/VGj4LHHoK6/mqpTU1JxBKWyqRMnjKAcyOfzkZWVRVpamuNik+DTZoEupI62zldQsETbIrhAqEcju3SBlSvhzjvNFNCzz0K3brBsWUD++0NKSEjwT5WUJSUH3ndiApCQkEBmZqYjYxNnUYLiEFre6mz5+fP48stLqnWskkh72bG/ltcLU6bA0qXQvj18/z0MGADjxkFRUfX/Hy3HFfmDEhSHUEdb5yqbLti//+dqHa8k0l52jkb27Wumeq6+GiwLHnwQTj7Z7JhcHXWpKUlISCAjI0MjE2HCyX1tQkUJikOoo60zHXq6oCIlkU5g92hk06bw9NOwYAG0agVr10LPnvDww1BymFmltLQ0cnJyyMnJITs7G4Ds7Gz/Y2lpaVX+W02dBI4TkgNtCaAExVHU0dZ5Dj9dUJ6SSPs5ZTTy/PPNyMmQIVBcDH/7G5xxhpn+qYoba0rCkZIDZ1AfFIdp1WoYLVuer06yDlHdaYD69Y/k2GOzlUQ6QHWb7YXiZyouDl57zYyopKebGpWuXWHGDNNPxenN3bTiJrTc0tcmVDSC4kDhvAGb21R3GqBz55eVnDiIk0YjPR649lpTm9KnD+zda5q8XXSRaZ9fFSfUlETSSIITCpTV16Y89UGxifbbcQftzeNuTvs5KykxhbP33GM2G4yLg//7Pzj3XNtCOiQn7+8TaE7YL8iNfW1qSnvx1EEofqFpvx33cNJ0gdScE/bXOlBUFIwfD+ecY5q7ffklnHee2YTw4YfN7sl2i9RpBifsF6QtAcpTgnKAUCQO2m/HfQ63N4++X1JTPXrAF1/AXXfB9Onw5JPw3nvw/PPQq5e9sWVlZR00klA23QDhu/OwkgPn0RTP/wSqNfahRmDUKt3dnDZdIOHh/fdNe/wffoB69WDCBLj7bmjQwJ547JpmcFJBrhOmtpx0PgKpJtdvJSgELnE43AiM9tsRkcoUFsKNN8K//mXup6TAnDlw3HG2hhXSC7UTkoIylSUH4ZowhJr24qmhQLTGLhuBOdQut9pvRyS4LKuEgoIl7Nw5l4KCJa7Z/bt5c5OQvPQSxMZCTo6ZBnriCSgttTu6yFNZ07tIWtHkFKpBoe6tsQ+/OZmHLVvSOfbYZ6r1ddQqXaTmwqH4/JJL4NRTTav8RYvg5pvh9dfhmWegTZvD//tAq+lS55pOg0ZqQa5UjxIU6t4au7ojMB6P+YV5uCWrapUuTuCmmptwKj5v0wbeeQf++U/TfXbxYtPcbeZMuPTS0MZSNpJQHbVJEJ1ekKsEyl6qQaHuvS527pzLxo0jDvt1jj/+BerV8/7vFylUtmTVTb9IJXy5aTQinIvPv/rKdJz94gtzf/hw+Mc/zDSQk9R2kYHT+344oTdKuFENSg3VdaO+mozAOKnDpUhlqlNP5SSBqCFzquOOg+XLTWO3qCiYO9eMprz3nt2R/eHwU9ywZUt6pfVATt97qC6bN0rdaYrnf+rS66Jsc7LqTt1ovx1xqurWU7Vseb5j3q/hVHxe2bRagwZRTJxous1eeSVs3gxnnQVjx8KUKRATY2/MNUkQ3bY6Ub1R7KUE5QC1TRxq023UaR0uRcCdF5u61pA5xeGm1U45BVatMnUpM2fCY4+ZQto5c8DO62WgEkQn7D0kzqIpngpqu1Gfpm4kHLhpNKJsSXFx8XYaNGh5iCM9eL3tHF18Xt1ptcaNTfHsW29BfDxs3AinnAKTJpm9fewQqASxsqW9TqIEKvRUJBtgblr5IFKRW5oJVjbaUDnnF5/Xtsj3xx/NHj6vvmru9+4Nzz0HSUnBj/lA2lBTakJFsjaq7QiMiBOU1VMdXCxexv7RiKpGGyrjhhHM2hb5tmwJr7xikpKmTeHTT+HEEyE7G0L5sfPQiwwMbagptaEERUT86rqiLdgOXcQL4KFBg1Ycd9wcunf/kF69tjo6OYG6Tat5PKZwdu1aGDAAfv0VrrsOhg6FvLwAB3oIZVPc9esfedBzlT0mUh1KUESkHCfXU1VntGHfvny83jauGcEMRA3H0UebTQcfeQSio+GNN8xy5AULAhRkNe3f/1Mlj/3syOXp4nxKUETkIK1aDaNXr+/o3v1Djj/+BceMRripiLe6AjWtVq8e3HqraerWvbupUbngAtM2f8+egIddzh8jW5U+C1TdC0WkKkFPUKZMmYLH4yE9Pd3/mGVZZGZmkpiYSExMDAMGDGDDhg3BDkVEasCJ9VThsqT4QIGeVuvaFVauhHHjzBTQM8+YhGXZssDFXFE4N8sT+wQ1Qfn888+ZNWsW3bp1K/f4gw8+yLRp05gxYwaff/458fHxnHXWWezduzeY4YiIy7mhiLc2Aj2t5vXCAw/A0qXQvj18952pURk3DoqKAha2XziObIn9gpag/PLLL1x++eVkZ2cTe8DGEZZlMX36dCZMmMCwYcPo0qULs2fP5rfffuOFF14IVjgiYaGs98fOnXMpKFgScUPmTi/irYtgTKv17Qtr1phpHsuCBx+Ek0+GdesCGDjhObIl9gtagjJmzBjOO+88zjzzzHKPb926lby8PAYNGuR/zOv10r9/f5YvX17p/1VUVMSePXvK3UQiTX7+PFasaM+aNQPZuHEEa9YMZMWK9hFXfOjkIt66Csa0WtOm8PTTpmC2VSuz4qdnT3j4YSgJUH4briNbYq+gJCgvvvgiubm5TJky5aDn8v639i0uLq7c43Fxcf7nKpoyZQrNmjXz39q1axf4oEUczG0b+AWbU4t4nez8883IyeDBUFxsWuafcQZ8/33d/+9wHtkKhEgf+aytgCco27ZtY+zYscyZM4eGDRtWeZzHU/5NbFnWQY+VGT9+PLt37/bftm3bFtCYRZysLrvFhjMnFvE6XVwcLFxomrk1bmxqVLp1M83e6trcLZxHtupCI5+1F/BW9wsWLOCCCy4gKuqPXxYlJSV4PB7q1avHpk2bSEpKIjc3lx49eviPOf/882nevDmzZ88+7Ndwcqt7kUBzS/t5cZdvvjFN3j791Ny/8EJ48knTobYutN3HH8pGPg/+cFG7LRjC4dza2ur+jDPOYN26daxevdp/69mzJ5dffjmrV6+mY8eOxMfHs3jxYv+/KS4uZunSpfTp0yfQ4Yi4nlZISDB06mSWHk+aBPXrmz19unaFt9+u2/+rkS0j0COfkTgSE/AEpUmTJnTp0qXcrXHjxrRo0YIuXbr4e6JMnjyZ+fPns379ekaNGkWjRo0YMWJEoMMRcT2tkJBgqV8f7rrL9E054QTTHv/cc+H6603bfKm9QPaGidQaNFs6yd5xxx2kp6dzww030LNnT7Zv386iRYto0qSJHeGIOJpWSEiwJSebDrRl/TSffNJsPLhypZ1RuVugRj4juQYt4DUooaAaFIk0f8xlQ/lfVLWbyxapyvvvw6hR8MMPEBVlRljuvhsaNLA7MncJVO1YuNWg2VqDIiKBpxUSEipnnGGWI19+uemTct990Ls3fPWV3ZG5S6BGPiO5Bk0JiohLqPeHhErz5jBnDrz4IsTGQk4O9OgBM2ZAaand0blDoHrDRHINmhIUERfRCgkJpUsvNaMpgwbBf/8LN90E55wD27fbHZk7BGLkM5Jr0FSDIiIih2RZ8M9/wu23m0QlNhZmzjQJjBxeXfuXhFMNWk2u30pQRESkWr76yjR3++ILc3/ECDPtc8B+sBIk+fnz2LJlbLmlxl5vO5KSprsmOQElKCIiEiT79sH995sGbyUl0KYNPPssVNgXVoIg0jrJKkERcahw+GUk4WvlSjOasnmzuT92LEyZAjEx9sYlzqZlxiIuF4ltrcVdTjkFVq0yXWcBHnsMUlIgN9feuCR8KEERcZhIbWst7tO4sSmefestiI+HjRtN4jJ5Muzfb3d04nZKUEQcJJLbWot7/fnPZjnyhReaxGTCBOjXz+yYLFJbSlBEHCSQG4yJhFLLlvDKK/Dcc9C0KXz6KXTvDtnZZpmySE0pQRFxkEhuay3u5/GYwtm1a6F/f7Mj8nXXwdChsHOn3dGJ2yhBEXGQSG5rLeHj6KPhgw/g4YchOhreeAO6dIEFC+yOTNxECYqIg0RyW2sJL/XqwW23maZu3bvDjz/CBRfA1VfDnj12RyduoARFxEECtcGYiFN07Wp6powbZ6aAnnnGJCwfqYxKDkMJiojDBGKDMREn8XrhgQdg6VJo3x6++87UqIwbB0VFdkcnTqVOsiIOpU6yEo727IFbboH/+z9zv1s3mDPHjLRI+FMnWZEw4PFEERs7gLi44cTGDlByImGhaVN4+mmYP98sTV67Fnr2hEcegdJSu6MTJ1GCIiIiIfeXv8D69TB4MBQXw+23w+mnw/ff2x2ZOIUSFBERsUVcHCxcaJq5NW5salS6dTPN3txXfCCBpgRFRERs4/HAtdfCmjXQu7epURk5Ei6+2CxNlsilBEVERGzXqRMsWwaTJkH9+vDqq6Zw9u237Y5M7KIERUREHKF+fbjrLtM35fjjIS8Pzj0Xrr/etM2XyKIERUREHCU5GXJyID3d3H/ySejRwyQuEjmUoIiIiOPExMCjj8J770HbtrB5M5x6KmRkwL59dkcnoaAERUREHOuMM0yvlBEjoKQE7r0X+vSBr76yOzIJNiUoIiIOYlklFBQsYefOuRQULMGySuwOyXaxsfCvf8GLL5q/f/GFmfKZMUPN3cKZWt2LiDhEfv48tmwZS1HRD/7HvN62JCU9pj2Y/mf7dvjrX2HxYnN/0CDTNr9Nm0P/O3EGtboXEXGZ/Px5bNhwUbnkBKCoaDsbNlxEfv48myJzljZt4J134IknoGFDWLTILEd+6SW7I5NAU4IiImIzyyphy5axQGUD2uaxLVvSNd3zP/XqwY03wqpVZh+fggK47DK4/HLzdwkPSlBERGxWWPjRQSMn5VkUFW2jsPCjkMXkBscdB8uXwz33QFQUvPCCaZX//vt2RyaBoARFRMRmxcW+gB4XSRo0gIkT4eOPISkJfvgBzjzT9FD5/Xe7o5O6UIIiImKz6OiEgB4XiXr1gtWrYfRoc/+xxyAlBXJzbQ1L6kAJioiIzZo374vX2xbwVHGEB6+3Hc2b9w1lWK7TuDHMnAlvvgnx8bBxI5xyCkyeDPv32x2d1JQSFBERm3k8USQlPVZ2r+KzACQlTcfjiQppXG517rmwbh1ceKFJTCZMgH794Jtv7I5MakIJioiIA7RqNYzOnf+N11u+oYfX25bOnf+tPig11LIlvPIKPPccNG0Kn34K3btDdja4r/tXZFKjNhERB7GsEgoLP6K42Ed0dALNm/fVyEkdff89jBwJS5ea+4MHw1NPQVycvXFFIjVqExFxKY8nitjYAcTFDSc2doCSkwA4+mj44AN4+GGIjoY33oAuXWDBArsjk0NRgiIiImGvXj247Tazj0+3bvDjj3DBBXDNNbBnj93RSWWUoIiISMTo2hU++wzGjQOPx+zj0707fKQeeI6jBEVERCKK1wsPPGBqUtq3h+++g/794c47oajI7uikjBIUERGJSH37wpo1Zndky4KpU03flPXr7Y5MQAmKiIhEsKZNzTTP/PlmafKaNaYD7SOPQGmp3dFFNiUoIiIS8f7yFzNyMngwFBfD7bfDGWeYJcpiDyUoIiIimL4oCxfCrFmmbf6SJWbFz3PPqbmbHZSgiIiI/I/HA6mpZqqnd2+zBHnkSLj4YrM0WUJHCYqIiEgFnTrBsmUwaRLUrw+vvmqWKL/9tt2RRQ4lKCIiIpWoXx/uugtWroTjj4e8PLMR4fXXw6+/2h1d+FOCIiIicgjJyZCTA2PHmvtPPgk9epjERYJHCYqIiMhhxMTA9OmweDG0bQubN8Opp0JGBuzbZ3d04UkJioiISDWdeSasXQsjRkBJCdx7L/TpA5s22R1Z+FGCIiIiUgOxsfCvf8GLL0Lz5mYDwh49YMYMLUcOJCUoIiIitXDppaa521lnwe+/w003wTnnwPbtdkcWHpSgiIiI1FKbNvDOO/DEE9CwISxaZJYjv/yy3ZG5nxIUERGROqhXD268EVatMvv4FBSY0ZUrroDCQrujcy8lKCIiIgFw3HHw6adwzz0QFWXqVLp2hffftzsyd1KCIiIiEiANGsDEifDxx5CUBD/8YFb+3HKLqVOR6lOCIiIiEmC9esHq1TB6tLk/fTr07GmmgaR6lKCIiIgEQePGMHMmvPkmxMfDl1/CySfD5Mmmh4ocmhIUERGRIDr3XFi3DoYNg/37YcIE6NcPvvnG7sicTQmKiIhIkLVsCf/+N8yeDU2awPLl0L07ZGeruVtVlKCIiIiEgMcDV11lRlP69TM7Il93HQwdCjt32h2d8yhBERERCaGjj4YPPoCHHoLoaHjjDbMcecECuyNzFiUoIiIiIRYVBbffbvbx6dYN8vPhggvgmmtg7167o3MGJSgiIiI26doVPvsM7rjDTAH93/+Z2pSPP7Y7MvspQREREbGR1wtTp8KSJdC+PWzdampU7rwTiorsjs4+SlBEREQcoF8/WLMG/vpXs7Jn6lQ45RSzY3IkUoIiIiLiEE2bmmmeefPM0uQ1a8wGhNOmQWmp3dGFlhIUERERh7ngArMcefBgKC6G226DM86A77+3O7LQUYIiIiLiQPHxsHAhzJpl2uYvWWJW/Dz/fGQ0dwt4gjJlyhROOukkmjRpQuvWrfnLX/7Cpk2byh1jWRaZmZkkJiYSExPDgAED2LBhQ6BDERERcTWPB1JTzcaDvXvDnj2m2dsll8BPP9kdXXAFPEFZunQpY8aMYcWKFSxevJj9+/czaNAgfv31V/8xDz74INOmTWPGjBl8/vnnxMfHc9ZZZ7FXi79FREQOkpQEy5bBpElQv75pm9+lC7zzjt2RBY/HsoI7UJSfn0/r1q1ZunQp/fr1w7IsEhMTSU9PZ9y4cQAUFRURFxfH1KlTSUtLO+z/uWfPHpo1a8bu3btp2rRpMMMXERFxlNxcuOIK2LjR3L/+etOVtnFje+Oqjppcv4Neg7J7924AjjzySAC2bt1KXl4egwYN8h/j9Xrp378/y5cvD3Y4IiIirpacDDk5MHasuT9zJvToAStX2htXoAU1QbEsi1tvvZXTTjuNLl26AJCXlwdAXFxcuWPj4uL8z1VUVFTEnj17yt1EREQiVUwMTJ8OixdD27aweTOceipkZMC+fXZHFxhBTVBuvPFG1q5dy9y5cw96zuPxlLtvWdZBj5WZMmUKzZo189/atWsXlHhFRETc5MwzYe1aGDECSkrg3nuhTx+osDbFlYKWoNx0000sXLiQDz/8kLZt2/ofj4+PBzhotGTXrl0HjaqUGT9+PLt37/bftm3bFqywRUREXCU2Fv71L5g7F5o3NxsQ9ugBM2a4ezlywBMUy7K48cYbmTdvHh988AEdOnQo93yHDh2Ij49n8eLF/seKi4tZunQpffr0qfT/9Hq9NG3atNxNRERE/nDZZaYt/llnwe+/w003wTnnwI4ddkdWOwFPUMaMGcOcOXN44YUXaNKkCXl5eeTl5fH7778DZmonPT2dyZMnM3/+fNavX8+oUaNo1KgRI0aMCHQ4IiIiEaNNG7P0+PHHoWFDWLTILEd++WW7I6u5gC8zrqqO5JlnnmHUqFGAGWWZOHEiWVlZFBQUcMopp/CPf/zDX0h7OFpmLCIicmhffWWWI+fkmPuXX26mfZo3ty+mmly/g94HJRiUoIiIiBzevn1w332mwVtpqVnxM3s2nH66PfE4qg+KiIiI2KNBA7Oy55NPTDfaH34wmw7ecoupU3EyJSgiIiJhrlcvs5/P6NHm/vTp0LMnrFplZ1SHpgRFREQkAjRubLrOvvkmxMXBl1/CKafAlCmmh4rTKEERERGJIOeea5YjDxtmalTuugv69YNvv7U7svKUoIiIiESYli3NjsizZ0OTJrB8OXTrBk895ZzmbkpQREREIpDHA1ddZVrl9+sHv/4Kqalw/vmwc6fd0SlBERERiWjt28MHH8BDD0F0NLz+OnTtCq+9Zm9cSlBEREQiXFQU3H672cenWzfIz4eRI6GgwL6Y6tv3pUVERMRJunaFzz6De+4xiUpsrH2xKEERERERP68Xpk61OwpN8YiIiIgDKUERERERx1GCIiIiIo6jBEVEREQcRwmKiIiIOI4SFBEREXEcJSgiIiLiOEpQRERExHGUoIiIiIjjKEERERERx1GCIiIiIo6jBEVEREQcRwmKiIiIOI4rdzO2LAuAPXv22ByJiIiIVFfZdbvsOn4orkxQ9u7dC0C7du1sjkRERERqau/evTRr1uyQx3is6qQxDlNaWsqOHTto0qQJHo8noP/3nj17aNeuHdu2baNp06YB/b/dINJfP+gcRPrrB50D0DmI9NcPwTkHlmWxd+9eEhMTqVfv0FUmrhxBqVevHm3btg3q12jatGnEvilBrx90DiL99YPOAegcRPrrh8Cfg8ONnJRRkayIiIg4jhIUERERcRwlKBV4vV4yMjLwer12h2KLSH/9oHMQ6a8fdA5A5yDSXz/Yfw5cWSQrIiIi4U0jKCIiIuI4SlBERETEcZSgiIiIiOMoQRERERHHicgEZebMmXTr1s3ffKZ37968/fbb/uctyyIzM5PExERiYmIYMGAAGzZssDHi4JoyZQoej4f09HT/Y+F+DjIzM/F4POVu8fHx/ufD/fWX2b59O1dccQUtWrSgUaNGnHjiieTk5PifD+fz0L59+4PeAx6PhzFjxgDh/drL7N+/n7///e906NCBmJgYOnbsyL333ktpaan/mHA/D3v37iU9PZ2jjz6amJgY+vTpw+eff+5/Ptxe/7JlyxgyZAiJiYl4PB4WLFhQ7vnqvN6ioiJuuukmWrZsSePGjRk6dCg//PBD4IO1ItDChQutN99809q0aZO1adMm66677rIaNGhgrV+/3rIsy3rggQesJk2aWK+++qq1bt0669JLL7USEhKsPXv22Bx54H322WdW+/btrW7dulljx471Px7u5yAjI8Pq3Lmz5fP5/Lddu3b5nw/3129ZlvXzzz9bRx99tDVq1Chr5cqV1tatW6333nvP2rJli/+YcD4Pu3btKvf9X7x4sQVYH374oWVZ4f3ay9x///1WixYtrDfeeMPaunWr9corr1hHHHGENX36dP8x4X4eLrnkEuuEE06wli5dam3evNnKyMiwmjZtav3www+WZYXf63/rrbesCRMmWK+++qoFWPPnzy/3fHVe7+jRo602bdpYixcvtnJzc62BAwda3bt3t/bv3x/QWCMyQalMbGys9dRTT1mlpaVWfHy89cADD/if++9//2s1a9bMevLJJ22MMPD27t1rHXPMMdbixYut/v37+xOUSDgHGRkZVvfu3St9LhJev2VZ1rhx46zTTjutyucj5TyUGTt2rNWpUyertLQ0Yl77eeedZ1199dXlHhs2bJh1xRVXWJYV/u+B3377zYqKirLeeOONco93797dmjBhQti//ooJSnVeb2FhodWgQQPrxRdf9B+zfft2q169etY777wT0PgicornQCUlJbz44ov8+uuv9O7dm61bt5KXl8egQYP8x3i9Xvr378/y5cttjDTwxowZw3nnnceZZ55Z7vFIOQebN28mMTGRDh06cNlll/Htt98CkfP6Fy5cSM+ePbn44otp3bo1PXr0IDs72/98pJwHgOLiYubMmcPVV1+Nx+OJmNd+2mmn8f777/P1118DsGbNGj7++GPOPfdcIPzfA/v376ekpISGDRuWezwmJoaPP/447F9/RdV5vTk5Oezbt6/cMYmJiXTp0iXg5yRiE5R169ZxxBFH4PV6GT16NPPnz+eEE04gLy8PgLi4uHLHx8XF+Z8LBy+++CK5ublMmTLloOci4RyccsopPPfcc7z77rtkZ2eTl5dHnz59+OmnnyLi9QN8++23zJw5k2OOOYZ3332X0aNHc/PNN/Pcc88BkfE+KLNgwQIKCwsZNWoUEDmvfdy4cQwfPpzjjjuOBg0a0KNHD9LT0xk+fDgQ/uehSZMm9O7dm/vuu48dO3ZQUlLCnDlzWLlyJT6fL+xff0XVeb15eXlER0cTGxtb5TGB4srdjAPh2GOPZfXq1RQWFvLqq68ycuRIli5d6n/e4/GUO96yrIMec6tt27YxduxYFi1adNAnhwOF8zn485//7P97165d6d27N506dWL27Nn06tULCO/XD1BaWkrPnj2ZPHkyAD169GDDhg3MnDmTq666yn9cuJ8HgKeffpo///nPJCYmlns83F/7Sy+9xJw5c3jhhRfo3Lkzq1evJj09ncTEREaOHOk/LpzPw/PPP8/VV19NmzZtiIqKIjk5mREjRpCbm+s/Jpxff2Vq83qDcU4idgQlOjqapKQkevbsyZQpU+jevTuPPfaYfyVHxUxw165dB2WVbpWTk8OuXbtISUmhfv361K9fn6VLl/L4449Tv359/+sM53NQUePGjenatSubN2+OiPcAQEJCAieccEK5x44//nj+85//AETMefj+++957733uPbaa/2PRcpr/9vf/sadd97JZZddRteuXbnyyiu55ZZb/COrkXAeOnXqxNKlS/nll1/Ytm0bn332Gfv27aNDhw4R8foPVJ3XGx8fT3FxMQUFBVUeEygRm6BUZFkWRUVF/jfl4sWL/c8VFxezdOlS+vTpY2OEgXPGGWewbt06Vq9e7b/17NmTyy+/nNWrV9OxY8ewPwcVFRUVsXHjRhISEiLiPQBw6qmnsmnTpnKPff311xx99NEAEXMennnmGVq3bs15553nfyxSXvtvv/1GvXrlLwNRUVH+ZcaRch7AfEhJSEigoKCAd999l/PPPz+iXj9U7/udkpJCgwYNyh3j8/lYv3594M9JQEtuXWL8+PHWsmXLrK1bt1pr16617rrrLqtevXrWokWLLMsyy6yaNWtmzZs3z1q3bp01fPhwVy8rq44DV/FYVvifg9tuu81asmSJ9e2331orVqywBg8ebDVp0sT67rvvLMsK/9dvWWaJef369a1JkyZZmzdvtv71r39ZjRo1subMmeM/JtzPQ0lJiXXUUUdZ48aNO+i5cH/tlmVZI0eOtNq0aeNfZjxv3jyrZcuW1h133OE/JtzPwzvvvGO9/fbb1rfffmstWrTI6t69u3XyySdbxcXFlmWF3+vfu3evtWrVKmvVqlUWYE2bNs1atWqV9f3331uWVb3XO3r0aKtt27bWe++9Z+Xm5lqnn366lhkHytVXX20dffTRVnR0tNWqVSvrjDPO8CcnlmWWWmVkZFjx8fGW1+u1+vXrZ61bt87GiIOvYoIS7uegbG1/gwYNrMTERGvYsGHWhg0b/M+H++sv8/rrr1tdunSxvF6vddxxx1mzZs0q93y4n4d3333XAqxNmzYd9Fy4v3bLsqw9e/ZYY8eOtY466iirYcOGVseOHa0JEyZYRUVF/mPC/Ty89NJLVseOHa3o6GgrPj7eGjNmjFVYWOh/Ptxe/4cffmgBB91GjhxpWVb1Xu/vv/9u3XjjjdaRRx5pxcTEWIMHD7b+85//BDxWj2VZVmDHZERERETqRjUoIiIi4jhKUERERMRxlKCIiIiI4yhBEREREcdRgiIiIiKOowRFREREHEcJioiIiDiOEhQRERFxHCUoIiIi4jhKUERERMRxlKCIiIiI4yhBEREREcf5f/GyBCxmOJylAAAAAElFTkSuQmCC",
"text/plain": [
"
"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plot_decision_boundary(w, b, X_train, y_train)"
]
},
{
"cell_type": "markdown",
"id": "e948ffa2",
"metadata": {},
"source": [
"##### 2.8 Evaluating logistic regression\n",
"\n",
"\n",
"We can evaluate the quality of the parameters we have found by seeing how well the learned model predicts on our training set. \n",
"\n",
"You will implement the `predict` function below to do this.\n"
]
},
{
"cell_type": "markdown",
"id": "2ce67b86",
"metadata": {},
"source": [
"##### Exercise 4\n",
"\n",
"\n",
"\n",
"Please complete the `predict` function to produce `1` or `0` predictions given a dataset and a learned parameter vector $w$ and $b$.\n",
"- First you need to compute the prediction from the model $f(x^{(i)}) = g(w \\cdot x^{(i)})$ for every example \n",
" - You've implemented this before in the parts above\n",
"- We interpret the output of the model ($f(x^{(i)})$) as the probability that $y^{(i)}=1$ given $x^{(i)}$ and parameterized by $w$.\n",
"- Therefore, to get a final prediction ($y^{(i)}=0$ or $y^{(i)}=1$) from the logistic regression model, you can use the following heuristic -\n",
"\n",
" if $f(x^{(i)}) >= 0.5$, predict $y^{(i)}=1$\n",
" \n",
" if $f(x^{(i)}) < 0.5$, predict $y^{(i)}=0$\n",
" \n",
"If you get stuck, you can check out the hints presented after the cell below to help you with the implementation."
]
},
{
"cell_type": "code",
"execution_count": 33,
"id": "432d08e4",
"metadata": {},
"outputs": [],
"source": [
"# UNQ_C4\n",
"# GRADED FUNCTION: predict\n",
"\n",
"def predict(X, w, b): \n",
" \"\"\"\n",
" Predict whether the label is 0 or 1 using learned logistic\n",
" regression parameters w\n",
" \n",
" Args:\n",
" X : (ndarray Shape (m, n))\n",
" w : (array_like Shape (n,)) Parameters of the model\n",
" b : (scalar, float) Parameter of the model\n",
"\n",
" Returns:\n",
" p: (ndarray (m,1))\n",
" The predictions for X using a threshold at 0.5\n",
" \"\"\"\n",
" # number of training examples\n",
" m, n = X.shape \n",
" p = np.zeros(m)\n",
" \n",
" ### START CODE HERE ### \n",
" # Loop over each example\n",
" for i in range(m): \n",
" z_wb = np.dot(X[i],w) \n",
" # Loop over each feature\n",
" for j in range(n): \n",
" # Add the corresponding term to z_wb\n",
" z_wb += 0\n",
" \n",
" # Add bias term \n",
" z_wb += b\n",
" \n",
" # Calculate the prediction for this example\n",
" f_wb = sigmoid(z_wb)\n",
"\n",
" # Apply the threshold\n",
" p[i] = 1 if f_wb>0.5 else 0\n",
" \n",
" ### END CODE HERE ### \n",
" return p"
]
},
{
"cell_type": "markdown",
"id": "ba0c79a5",
"metadata": {},
"source": [
"\n",
" Click for hints\n",
" \n",
" \n",
"* Here's how you can structure the overall implementation for this function\n",
" ```python \n",
" def predict(X, w, b): \n",
" # number of training examples\n",
" m, n = X.shape \n",
" p = np.zeros(m)\n",
" \n",
" ### START CODE HERE ### \n",
" # Loop over each example\n",
" for i in range(m): \n",
" \n",
" # Calculate f_wb (exactly how you did it in the compute_cost function above) \n",
" # using a couple of lines of code\n",
" f_wb = \n",
"\n",
" # Calculate the prediction for that training example \n",
" p[i] = # Your code here to calculate the prediction based on f_wb\n",
" \n",
" ### END CODE HERE ### \n",
" return p\n",
" ```\n",
" \n",
" If you're still stuck, you can check the hints presented below to figure out how to calculate `f_wb` and `p[i]` \n",
" \n",
" \n",
" Hint to calculate f_wb\n",
" Recall that you calculated f_wb in compute_cost above — for detailed hints on how to calculate each intermediate term, check out the hints section below that exercise\n",
" \n",
" More hints to calculate f_wb\n",
" You can calculate f_wb as\n",
"
\n",
" for i in range(m): \n",
" # Calculate f_wb (exactly how you did it in the compute_cost function above)\n",
" z_wb = 0\n",
" # Loop over each feature\n",
" for j in range(n): \n",
" # Add the corresponding term to z_wb\n",
" z_wb_ij = X[i, j] * w[j]\n",
" z_wb += z_wb_ij\n",
" \n",
" # Add bias term \n",
" z_wb += b\n",
" \n",
" # Calculate the prediction from the model\n",
" f_wb = sigmoid(z_wb)\n",
"
\n",
" \n",
" \n",
" \n",
" Hint to calculate p[i]\n",
" As an example, if you'd like to say x = 1 if y is less than 3 and 0 otherwise, you can express it in code as x = y < 3 . Now do the same for p[i] = 1 if f_wb >= 0.5 and 0 otherwise. \n",
" \n",
" More hints to calculate p[i]\n",
" You can compute p[i] as p[i] = f_wb >= 0.5\n",
" \n",
" \n",
"\n",
""
]
},
{
"cell_type": "markdown",
"id": "b5cf416b",
"metadata": {},
"source": [
"Once you have completed the function `predict`, let's run the code below to report the training accuracy of your classifier by computing the percentage of examples it got correct."
]
},
{
"cell_type": "code",
"execution_count": 34,
"id": "c01a976c",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Output of predict: shape (4,), value [0. 1. 1. 1.]\n",
"\u001b[92mAll tests passed!\n"
]
}
],
"source": [
"# Test your predict code\n",
"np.random.seed(1)\n",
"tmp_w = np.random.randn(2)\n",
"tmp_b = 0.3 \n",
"tmp_X = np.random.randn(4, 2) - 0.5\n",
"\n",
"tmp_p = predict(tmp_X, tmp_w, tmp_b)\n",
"print(f'Output of predict: shape {tmp_p.shape}, value {tmp_p}')\n",
"\n",
"# UNIT TESTS \n",
"predict_test(predict)"
]
},
{
"cell_type": "markdown",
"id": "3c73ab64",
"metadata": {},
"source": [
"**Expected output** \n",
"\n",
"
\n",
"
\n",
"
Output of predict: shape (4,),value [0. 1. 1. 1.]
\n",
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"id": "eb8c0f6a",
"metadata": {},
"source": [
"Now let's use this to compute the accuracy on the training set"
]
},
{
"cell_type": "code",
"execution_count": 35,
"id": "0cfe1116",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Train Accuracy: 92.000000\n"
]
}
],
"source": [
"#Compute accuracy on our training set\n",
"p = predict(X_train, w,b)\n",
"print('Train Accuracy: %f'%(np.mean(p == y_train) * 100))"
]
},
{
"cell_type": "markdown",
"id": "5fd9bc6e",
"metadata": {},
"source": [
"
\n",
"
\n",
"
Train Accuracy (approx):
\n",
"
92.00
\n",
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"id": "b46e747b",
"metadata": {},
"source": [
"#### 3 - Regularized Logistic Regression\n",
"\n",
"\n",
"In this part of the exercise, you will implement regularized logistic regression to predict whether microchips from a fabrication plant passes quality assurance (QA). During QA, each microchip goes through various tests to ensure it is functioning correctly. \n",
"\n",
"##### 3.1 Problem Statement\n",
"\n",
"\n",
"Suppose you are the product manager of the factory and you have the test results for some microchips on two different tests. \n",
"- From these two tests, you would like to determine whether the microchips should be accepted or rejected. \n",
"- To help you make the decision, you have a dataset of test results on past microchips, from which you can build a logistic regression model.\n",
"\n",
"##### 3.2 Loading and visualizing the data\n",
"\n",
"\n",
"\n",
"Similar to previous parts of this exercise, let's start by loading the dataset for this task and visualizing it. \n",
"\n",
"- The `load_dataset()` function shown below loads the data into variables `X_train` and `y_train`\n",
" - `X_train` contains the test results for the microchips from two tests\n",
" - `y_train` contains the results of the QA \n",
" - `y_train = 1` if the microchip was accepted \n",
" - `y_train = 0` if the microchip was rejected \n",
" - Both `X_train` and `y_train` are numpy arrays."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "125e300e",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": 36,
"id": "728ac6da",
"metadata": {},
"outputs": [],
"source": [
"# load dataset\n",
"%matplotlib widget\n",
"import matplotlib.pyplot as plt\n",
"import sys\n",
"sys.path.append(\"week3/OptionalLabs\")\n",
"sys.path.append(\"week3/C1W3A1\")\n",
"from utils import *\n",
"plt.style.use('week3/OptionalLabs/deeplearning.mplstyle')\n",
"from plt_overfit import overfit_example, output\n",
"\n",
"X_train, y_train = load_data(\"week3/C1W3A1/data/ex2data2.txt\")"
]
},
{
"cell_type": "markdown",
"id": "0365b1a4",
"metadata": {},
"source": [
"###### View the variables\n",
"\n",
"The code below prints the first five values of `X_train` and `y_train` and the type of the variables.\n"
]
},
{
"cell_type": "code",
"execution_count": 37,
"id": "3716097e",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"X_train: [[ 0.05 0.7 ]\n",
" [-0.09 0.68]\n",
" [-0.21 0.69]\n",
" [-0.38 0.5 ]\n",
" [-0.51 0.47]]\n",
"Type of X_train: \n",
"y_train: [1. 1. 1. 1. 1.]\n",
"Type of y_train: \n"
]
}
],
"source": [
"# print X_train\n",
"print(\"X_train:\", X_train[:5])\n",
"print(\"Type of X_train:\",type(X_train))\n",
"\n",
"# print y_train\n",
"print(\"y_train:\", y_train[:5])\n",
"print(\"Type of y_train:\",type(y_train))"
]
},
{
"cell_type": "markdown",
"id": "2e174fe0",
"metadata": {},
"source": [
"###### Check the dimensions of your variables\n",
"\n",
"Another useful way to get familiar with your data is to view its dimensions. Let's print the shape of `X_train` and `y_train` and see how many training examples we have in our dataset."
]
},
{
"cell_type": "code",
"execution_count": 38,
"id": "0e6fb2f1",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"The shape of X_train is: (118, 2)\n",
"The shape of y_train is: (118,)\n",
"We have m = 118 training examples\n"
]
}
],
"source": [
"print ('The shape of X_train is: ' + str(X_train.shape))\n",
"print ('The shape of y_train is: ' + str(y_train.shape))\n",
"print ('We have m = %d training examples' % (len(y_train)))"
]
},
{
"cell_type": "markdown",
"id": "e68a2775",
"metadata": {},
"source": [
"###### Visualize your data\n",
"\n",
"The helper function `plot_data` (from `utils.py`) is used to generate a figure like Figure 3, where the axes are the two test scores, and the positive (y = 1, accepted) and negative (y = 0, rejected) examples are shown with different markers.\n",
"\n",
""
]
},
{
"cell_type": "code",
"execution_count": 39,
"id": "dee26d62",
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "2b07931d87014b7f881c17310c7d83a5",
"version_major": 2,
"version_minor": 0
},
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAoAAAAHgCAYAAAA10dzkAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAA9hAAAPYQGoP6dpAABctklEQVR4nO3de1xUdf4/8NfgZSxFh2TUyQFJibwBAl1cb4l2UTMo09TU+qUSWaJ+81KrFuDWbrW22WoqsbmWmlkuLppm2SYrqaybGFtqKioIOOEkYKKJAvP7g+bgyAAzcM7MubyejwcPnTNn5nzOHGbmzefz/rw/urKyMhuIiIiISDN8vN0AIiIiIvIsBoBEREREGsMAkIiIiEhjGAASERERaQwDQCIiIiKNYQBIREREpDEMAImIiIg0hgEgERERkcYwACQiIiLSGAaARERERBrDAJCIiIhIYxgAEhEREWkMA0AiIiIijWEASERERKQxDACJiIiINIYBIBEREZHGMAAkIiIi0hgGgEREREQawwCQiIiISGMYABIRERFpDANAIiIiIo1hAEhERESkMQwAiYiIiDSGASARERGRxjAAJCIiItIYBoBEREREGsMAkIiIiEhjGAASERERaQwDQCIiIiKNYQBIREREpDEMAImIiIg0hgEgERERkcYwACQiIiLSGAaARERERBrDAJCIiIhIYxgAEhEREWkMA0AiIiIijWEASERERKQxDACJiIiINIYBIBEREZHGMAAkIiIi0hgGgEREREQawwCQiIiISGMYABIRERFpDANAIiIiIo1hAEhERESkMQwAiYiIiDSGASARERGRxjAAJCIiItIYBoBEREREGsMAkIiIiEhjGAASERERaQwDQCIiIiKNYQBIREREpDEMAImIiIg0hgEgERERkcYwACQiIiLSGAaARERERBrDAJCIiIhIY1p6uwFq0bZtW/j4MJ4mIiKSq+rqaly6dMnbzZAFBoAi8fHxYQBIREREisCIhYiIiEhj2ANIRETkYdXV1SguLkZlZaW3m6JqLVu2ROfOnTlC5wQDQCIiIg8rLi6Gr68v2rVr5+2mqFp5eTmKi4thMpm83RTZYUhMRETkYZWVlQz+PKBdu3bsZa0HA0AiIiIijWEASERERKQxDACJiIgUwmKxICkpCRaLRfTnvnjxItq1a4fp06eL/tyNycjIwJdfftnkx955550it0j9GAASkehstiqUlmaguHgjSkszYLNVebtJRKpgsViQnJwsSQD48ccfIzIyEv/4xz9QXl4u+vM3pDkBIDUNA0AiEpXVmoasrCDk5ETj6NEnkJMTjaysIFitad5uGhE14P3338eLL76IwYMH45NPPgEAXLhwAdOnT0doaCjCw8MxdepUAMDVq1cxf/58YfuIESOE51m6dCnuvvtuREZGYtSoUSgoKAAAJCUl4fHHH8eoUaPQt29fxMTEoLS0FN999x1Wr16NDz/8EP369cOSJUsAAF988QUGDRqEqKgo3HPPPdizZ49wjMWLFyM4OBj33nsvPvvsM0+9RKrCMjBEJBqrNQ2HD48FYHPYXlFRhMOHx6JPn80wGsd4p3FECmWxWIQev+zsbId/AcBkMjW7zMnhw4dRUFCAESNGoLKyEm+++SamTp2KOXPmoF27dsjJyYGPjw+sVisA4E9/+hNOnjyJb7/9Fnq9Xtj+0Ucf4fjx49i/fz9atGiBdevWYebMmUhPTwcAZGZm4rvvvkPnzp3x3HPPYdGiRVi5ciWeffZZlJeXY+nSpQCAU6dOITk5GTt37kT79u2Rm5uLe++9F3l5edi5cye2bt2K7777DjfddBMeffTRZp27VjEAJCJR2GxVyM2djRuDv9/uBaBDbu4c+PvHQqdr4eHWESlXSkoKkpOTHbbFxcUJ/09MTERSUlKzjvH+++/jySefRIsWLfDQQw/h2WefxdGjR/HZZ5/h4MGDQiFlo9EIAPjss8/w1ltvQa/XO2z/5z//iW+//RZRUVEAgKqqKrRoUft+Hz16NDp37gwAeOaZZ/D44487bc/OnTuRm5uLIUOGOGwvKCjA7t27MX78eKGMztSpU/Hqq6826/y1iAEgEYmirCwTFRWFDexhQ0VFAcrKMuHnN9RDrVInm60KZWWZuHrVgtatTTAYBjOoVrH4+HjExMQAqOn5i4uLQ2pqKiIjIwGg2b1/165dw/r169GqVSts3LgRAHD58mWsWbPG7eey2WxYvHixMFTcGJ1OV+/zjBgxAh9++KHT+6j5mANIRKK4etW1pHRX9yPnmGOpPSaTCZGRkcIPAIfbzQ0A09PT0b17dxQVFSEvLw95eXnYu3cvPvzwQ8TExODPf/4zqqurAUAY6o2JicGyZctQUVFRZ/vKlStRUlICoCa4PHTokHCs7du349y5cwBqeh3vu+8+AED79u1x4cIFYb8HHngAO3fuxA8//CBsO3DgAABg+PDh+OSTT3Dp0iVUVVVh7dq1zTp/rWIASESiaN3atS8hV/ejuuw5ljf2tNpzLBkEUlO8//77mDRpksO2vn374tZbb8W9996Ly5cvo2/fvujXrx8WLlwIAHjxxRfRo0cPREREoF+/fnjqqacAAFOmTMHkyZMxdOhQhIeHo1+/fti9e7fwvMOHD8e0adPQt29f5OfnC0O3jz76KL799lthEsjtt9+O9evXY/r06QgPD0evXr3wzjvvAKgZRh49ejTCw8MxbNgwhIWFeeJlUh1dWVkZ+1JF4Ovry8WmSdNstipkZQWhoqIIzvMAddDrzejf/zSHK5ug9vWtb5idr6+SFBQUICAgwO3HWSwWpKSkID4+XnHr2yYlJTlM9PCU61/r6upqXLx40aPHlytGLEQkCp2uBYKD37HfuvFeAEBw8DIGJ03kTo4lqZfJZEJSUpLigj+SH04CISLRGI1j0KfPZuTmznYIVvR6M4KDl7EETDMwx5KUrrkzlUlcDACJSFRG4xj4+8dylqrImGNJRGJiAEhEotPpWrDUi8gMhsHQ682N5lgaDIM93TQiUiDmABIRKQBzLIlITAwAiYgUwp5jqdd3ddiu15u5zJ4G2GxVKC3NQHHxRpSWZsBmq/J2k0jBOARMRKQgzLHUJqs1rZ7JVe+IEvgHBQWhTZs2aNOmDX799Vc8/fTTeOmllxp8zKhRo7B8+XL06NGjScdcu3YtBgwYgJCQELcf662SMmrCAJCISGGYY6kt9gLgN+Z+2guAi9X7u3nzZvTt2xdnz55F7969MWzYMNx999317r9jx45mHW/t2rXw9/dvUgBIzcchYCIiIpmy2aqQmzsbzif+1GzLzZ0j6nDwrbfeijvuuAP5+fn46aef8Pjjj+Puu+9GWFgYXnnlFWG/oKAgYam2hvY7evQoHnzwQYSFhSEsLAyrV6/G3/72N3z77beYNWsW+vXrJwSTS5cuxd13343IyEiMGjUKBQUFAIALFy5g7Nix6N27Nx588EHk5uaKdr5axQCQiIhIprxRAPzHH3/Ezz//jKFDh+Kpp57CzJkzceDAAWRnZ+PAgQPYsmVLncfUt19lZSViY2Mxbdo0/O9//8P//vc/jB07FtOnT8edd96Jv/71r/juu+8watQofPTRRzh+/Dj279+P7OxsTJw4ETNnzgQALFmyBO3bt8eRI0ewYcMG7NmzR7Tz1SoOARMREcmUJwuAjx07FjqdDseOHcPbb7+Nm2++GV9//TWKi4uFfcrLy/Hjjz86PO7SpUv17hcSEoLKyko8/vjjwn3+/v5Oj//Pf/4T3377LaKiogAAVVVVaNGiJrd19+7dWL58ufD4MWM44am5GAASERHJlCcLgNtzAL/66is8/PDDGDZsGHQ6Hf773/+iVatW9T6uurq63v0OHz7s8vFtNhsWL16MqVOnOr2PxMUhYFItlkxQPl5D0jp7AfC6tR/tdNDrA0QtAH7fffdhxowZWLx4MQYPHozXX39duO/s2bMoLHQckvb19a13vzvuuAOtW7fGp59+Ktz3888/AwDat2+PCxcuCNtjYmKwcuVKlJSUAACuXbuGQ4cOAQCGDx+Ov//97wCAkpISp8PQ5B5VBYALFixAaGgoDAYDjhw5Uu9+H374ISIjI9GvXz/Mnj0blZWVwn07d+7EXXfdhYiICEyZMgXl5eWeaDqJzGpNQ1ZWEHJyonH06BPIyYlGVlYQrNY0bzdNdaQK0ngNibxXAPzll1/GN998gyVLluDo0aMIDQ1FaGgoHnvsMZw/f/669tW0YcOGDU73a9myJdLT0/Hee+8hNDQUYWFh+Mc//gEAeOaZZ7BkyRJhEsiUKVMwefJkDB06FOHh4ejXrx92794ttKe0tBS9e/fGpEmTcP/994t6vlqkKysrU02/6t69exEUFIQRI0Zg06ZN6N27d5198vLyMGLECOzZswdGoxETJ07Egw8+iKeffhrl5eWIiIjA9u3bERISgvnz56Ndu3ZITExs9Ni+vr7w8VFVPK1Y9ZVMsH9YsmCueKSqTcZrSGpXUFCAgIAAl/d3/l4LQHDwMq+8FyorK9GxY0ccO3YMXbp08fjx3XH9a11dXY2LFy96uUXyoKqIZeDAgejatWuD+2zduhWjR49Gp06doNPpMHXqVGzevBkA8NVXXyEiIkKoSTRt2jThPlIGb5RM0Cp7kHbjDEV7bbKm9tTxGhLVZTSOQf/+eQgP341evT5CePhu9O9/2ivB39mzZ9GzZ09MnjxZ9sEf1U9zk0Bu/KsrMDBQyGdwdp/FYkF1dTV79xTCnZIJLKTbdI0HaTrk5s6Bv3+s20NTvIZEzsmlAPitt97KOnwqoMmoxp6zANSdWXT9faQ8niyZoGVS1ibjNSQikp7mAsCAgACcOXNGuF1QUACz2ez0vjNnzsBkMrH3T0E8WTJBy6QM0ngNiYikp7nIJiYmBp999hnOnTsHm82GNWvW4LHHHgNQM808Ozsbx48fBwC8//77wn2kDN4omaBFUgZpvIakFaxtJz2+xvVTVQA4b9489O7dG2fPnsUjjzyCiIgIAEBCQoKwzmBQUBB+//vf48EHH0S/fv1gNBoxZcoUADUzef/6179i0qRJiIiIwNmzZ/HCCy947XzIfd4qmeAJFosFSUlJsFi8P/QpZZCm5mtIZNehQwehHh5J5+eff0aHDh283QxZUlUZGG9iGRh5kVvJBDFkZ2cjKioKBw8eRGRkpLebc12pFsBxMog4pVrUeA2Jrnfu3DlUVFR4uxmqptfr0alTJ+E2y8DU0twsYNIGo3EM/P1jUVaWiatXLWjd2gSDYTB7jURkNI5Bnz6b66kD2PwgjdeQ1O76wITI0xgAkmrJpWRCc1gsFmHINzs72+FfADCZTDCZvDcZQuogTQ3XkIhIjjgELBIOAZMUkpKSkJycXO/9iYmJSEpK8lyDqFE2WxV7LYlkikPAtRgAioQBIEnhxh7AuLg4pKamCjmA3u4BJEdSLY1HROJgAFiLQ8BETeSJnh5nAV5kZKQsJoHInad74upbv9i+NB7XLyYiOWEASNQE7OmRN09fHymXxiMikgLHLIncZO/puXEpNHtPj9WaJslxTSYTEhMTOeTbCG9cHymXxiMikgIDQFI8m60KpaUZKC7eiNLSDNhsVZIeq+GeHiA3d44kbTCZTEhKSmIA2ABvXR+uX0xESsMhYFI0Tw/1udPTw/Ilnuet68P1i4lIadgDSIrljaE+9vTIm7euD9cvJiKlYQBIiuStoT729Mibt64P1y9WLk+mkBDJCQNAUiRvJd2zp0fevHl97Evj6fVdHbbr9WaWgJEpqzUNWVlByMmJxtGjTyAnJxpZWUGSTeQikhPmAJIieWuoz97TU1PvTQfHHkj29Hibt68P1y9WDtZtJK1jDyApkjeHYtnTI2/evj729Ys7d54IP7+hDP5kyJuz+YnkgkvBiYRLwXmWzVaFrKwgVFQUwfmHuA56vRn9+5+W7AuYa77KG68P1ae0NAM5OdGN7hcevpuz+VWGS8HV4hAwKZK3h/rsbeCXg3zx+lB9OJufiEPApGDeHuojImXibH4iDgGLhkPA3sOhPiJyhxxSSMg7OARci0PApHgc6iMid8ghhYTI29hlRUREmsMUEtI6DgGLhEPARETKwxQSbeEQcC0OARMRkVNaCI6YQkJaxQCQiIjqsFrTkJs722HJRb3ejODgdzg8SqQCHLMkIiIH9mXSblxv275MGtfKJVI+BoBE5BUWiwVJSUmwWFhsV064TBqRNjAAJCKvsFgsSE5OZgAoM2VlmXV6/hzZUFFRgLKyTI+1iYjExwCQiATslSMuk0akDZwEQkQCe69cTEwMTCbxl8GyWCxCcJmdne3wLwCYTCZJjkuu4zJpRNrAAJCIPCYlJQXJyckO2+Li4oT/JyYmIikpycOtousZDIOh15sbXSbNYBjs6aYRkYgYABJpnCd75eLj4xETEyMcIy4uDqmpqYiMjBSO5Q3eqHcn1xp7XCaNSBu4EohIuBIIicVisSAlJQXx8fEeCYiSkpLq9MpdT6peuezsbERFReHgwYNCAOgN3qh3p4Qae87bGIDg4GWyaSORu7gSSC0GgCJhAEhi8XRgdGMPoLNeOSkCUTkEgPZ6d3WHOmt6uqRYE9Ybx2wqufZSEjUVA8BaHAIm0jhnAV5kZKTkQZnJZEJiYqJXh30brnenQ27uHPj7x4oW9HjjmM3BZdKI1IsBIJEMaHF2rMlk8uqED3fq3YkVBHnjmErAnkYiz2MASCQDcpkd6+1eOU/yRr071tirSwn5kERqxACQSAbkMjvW271ynuSNenesseeovnxI+5rDcsqHJFIb1QWAJ0+exIwZM3D+/Hl06NABK1euRM+ePR322bhxI959913h9tmzZzFgwACsX78e+fn5iIyMRK9evYT7161bh9tuu81j50Da4608PC3zRr071tirpbR8SCK1Ud201Tlz5uCpp57CwYMHMXv2bCQkJNTZZ+LEifjmm2+Eny5dumDcuHHC/R06dHC4n8EfkfrY6939duvGewGIX+/OG8eUK645TORdqgoArVYrcnJyMH78eABATEwM8vPzkZ+fX+9jDh48iHPnzmHUqFGeaiZRg7SUh+dtRuMY9OmzGXp9V4fter1ZsuFHbxxTjryRD2mzVaG0NAPFxRtRWpoBm61KtOcmUhpVDQEXFRXBZDKhZcua09LpdDCbzSgsLES3bt2cPmbdunUYP348WrVqJWy7ePEioqOjUVVVhYceegjz5s1Dixbq/4uc5EFLeXhyYDSOgb9/rEdnoXrjmHLj6XxITjYhcqSqABCoCfquZ7PVX+f68uXLSEtLw5dffils69KlC44cOQKj0YjS0lI8/fTTWLFiBWbPni1Zm4nIu7xR707rNfY8mQ/JySZEdalqCLhr1644e/YsKisrAdQEf0VFRTCbzU73T09Pxx133OEwSUSv18NoNAIA/Pz8MHnyZOzbt0/6xhMRaYin8iEbn2wC5ObO4XAwaY6qAkCj0YjQ0FBs2rQJALB161YEBgbWO/y7fv16TJkyxWGb1WrFtWvXAAAVFRXYtm0bwsLCpG04OSWnfB05tYVILTyRD8nJJkTOqW4IeNmyZXjuuefwl7/8Bb6+vli1ahUAICEhASNHjhQme5w+fRo5OTn4+OOPHR6/f/9+/OlPf4KPjw+qqqowePBgzJs3z+PnoXVyyteRU1uI1EbqfEgW3yZyTldWVlZ/khy5zNfXFz4+qupQ9Zr68nXsw0KezNeRU1uIyH2lpRnIyYludL/w8N2azsnUiurqaly8eNHbzZAFRiwkK3LK15FTW6RksViQlJQkrEWsFmo9L3KPfbJJ3TxDOx30+gBNFN8muh4DQHLg7Vw3OeXryKktTeVKEGSxWJCcnKy6QEmt5+UqBsA1WHybyDkGgCSwWtOQlRWEnJxoHD36BHJyopGVFQSrNc1jbZBTvo6c2tJUng6CGHTIh9YD4Oux+DZRXaqbBEJNI5c6WZ4uDivGMTzRFrFZLBYhMMjOznb4F3C+NrGrz5ucnIyYmBivrGQi1XmR8rH4NpEjBoAkq0XZPVkcVkltcYcrQVBKSgqSk5MdHhcXFyf8PzExUZGrkaj1vFzFALhhWi++TXQ9BoDkVq6b1B+e9nydmt5IHRwDL8/m68ipLe5wJQiKj49HTEwMgJoAIS4uDqmpqYiMjAQAt4IEOQUdYp6XEnk7ALZYLEhJSUF8fLzqX2tvsdmq2ItJomAZGJEouQxMcfFGHD36RKP79er1ETp3nuiBFtVXey8AwcHLZFIH0DttccWNAZmzIOj6L+fs7GxERUXh4MGDwj7uSEpKqhN0XM9bvW7NPS8lcvfai02Lr7knsSZp87EMTC32AJIsc93klK8jp7a4wtmXfGRkpGRfyFrvdZMTT1978hy55GmTejAAJNnmuskpX0dObRGbyWRCYmJikwM1uQYdzT0vco2cUgDUSk552qQeyhyzJFGxTpZ6aTkIMplMSEpK0uS5A5679ikpKYiKikJUVJSQbxgXFydsS0lJkfT4UvN2bVRAHTVJSX6YAygSJecA2ikt143EIWbeFicBaI+38w6lJJecOznmaSsVcwBrcQiYBErLdXMVgxLPsfe6kXbINQWgueSUcyfHPG1SPgaA5ECNuW7eLk4sR8zbIqqf3HLu5JqnTcqm7DFLImoStedtaYUc8tOup5acU7nl3DFPm6TAHkBSJfZwNYylW5RPLvlp11NLCoAc1wG3r2fs/JozT5vcxwCQVMnbKyLInVrztrRCTvlpaiTXnDu15mmTdzAAJFViDxepldzy09RIzjl3aszTJu9gAEiqxB4u16klb0sr5LR2txI0Ze1cpa4DTuQOTgIhj7FYLEhKShJy80getF4wWWnkmJ8mV1ZrGrKygpCTE42jR59ATk40srKCYLWmNfpYe86dXt/VYbteb+YQO6kCewDJY7xVjoU9XKQmcs1Pkxsx8iSZc0dqxgCQVE8tMxOJAHnnp8mFmHmSzLkjteIQMEnKYrEgOztb+AHgcJvDwUTucbUm3E8/ndNsyoXc6vgRyREDQJIUCw6TWsgph9WV/DR7yoUc2utpzJMkahyHgElSLMdCaiG3JQWZn1Y/5kkSNY4BIEmK5ViIpHNjfhpXwKnBPEmixjEAJCKqh9ICKq6AU4N1/IgapysrK3P25xG5ydfXFz4+TKlsiMViQUpKCuLj42X1pUniUds1TkpKqhNQXU9uAdWNAauzlAs1XBdXOV8vOYBr52pYdXU1Ll686O1myAIDQJEoMQBsSoV80iZXA7vs7GxERUXh4MGDqhjmV3JApbZr0VT8nKPrMQCsxSFgjXL+l7EZwcHv8C9jqkNuEyA8xdUcVrX1fKoJ6/gROccAUIPEqJBPZKe0PDkpyDFA5go4RNQQBoAaI2aFfFI3VwM7rUw8UFpAxRVwiKghzAEUiVJyAEtLM5CTE93ofuHhuzlsonGuToBQcp5cc4hx3hw6JvIs5gDWYg+gxrBCPrnK1SLeWq31KEbPpxyHjolIGxgAagwr5JOrtBrYuYqr3BCRkjEA1BhWyCcpKS1PrjmaGiBz0gwBLE9D3if/pDU3nTx5Eg888ACioqIwbNgw/Pjjj3X2yczMhMlkwqBBg4SfX3/9Vbh/586duOuuuxAREYEpU6agvLzck6cgKXuF/N9u3XgvAOkr5NtsVSgtzUBx8UaUlmbAZquS7FieYLFYhFw4tXI1sLNPPGAAU7+UlBRERUUhKipKGDKOi4sTtqWkpHi5hSQ1qzUNWVlByMmJxtGjTyAnJxpZWUGwWtO83TTSENVNAnn44YcxYcIETJo0Cenp6VixYgV27drlsE9mZiZefvllZGRk1Hl8eXk5IiIisH37doSEhGD+/Plo164dEhMTGzyuUiaB2HmrQr4a6w+y4C65M5lDq5NmqEZ9Zbjsf4CzDJe0OAmklqqGgK1WK3JycrBlyxYAQExMDObPn4/8/Hx069bNpef46quvEBERgZCQEADAtGnTMG7cuEYDQKUxGsfA3z/Wo0MQrD9IauVOyRXmVmoXy3CRnKgqACwqKoLJZELLljWnpdPpYDabUVhYWCcAzM3NxZAhQ9CiRQtMmjQJ06dPBwAUFBQgICBA2C8wMBAWiwXV1dWK6uFzhScr5Kvtg495XESep/SyOWVlmQ6jH3XZUFFRgLKyTJbhIsmpKgAEaoK+69lsdQOO8PBwHD58GB06dEBRURHGjRuHjh074tFHH3X6HNR8avvg00rxY5KWlibNiEHpZXNYhovkRFVdWl27dsXZs2dRWVkJoCb4Kyoqgtlsdtivffv26NChg/CYsWPHYt++fQCAgIAAnDlzRtj3zJkzMJlMquv98zS1ffDFx8fj4MGDOHjwIFJTUwEAqampwrb4+PhmH0MLk0u0jpNmtIVluEhOVBXVGI1GhIaGYtOmTQCArVu3IjAwsM7w708//YTq6moAwMWLF/HFF18gLCwMADB8+HBkZ2fj+PHjAID3338fjz32mAfPQp2k/ODzRqBkMpmEvC177tb1t8X4Qrf3djAAJC2zWCzIzs4WfgA43FbS+8NehqtuBQY7HfT6AJbhIo9QVQAIAMuWLcPatWsRFRWFt99+G8uXLwcAJCQkYMeOHQBqAsMBAwZg4MCBuP/++zF06FBMnjwZQM1s3r/+9a+YNGkSIiIicPbsWbzwwgteOx+1kPKDj4ESkXqpqWyOHMpwEdmprgyMtyitDIw31M4CBhwngzSv/IG3y7CImZjOEiFEjtT4nvBWGS5iGZjrMQAUCQNA14j1wafGLwUASEpKqjO55HqcXEJa5u0/9sTElUC8gwFgLQaAImEA6DoxPvjUGiipNbAlEoNSAkCll6tRMwaAtSQNAC9evIi2bdvWCYyqq6tRVFTkUG9P6RgAepYWAiWlfNkReYpSAiu+d+WLAWAtSeoAnj9/HlOnTkVmZibatGmDJ554AkuWLMHNN98MAPj5558RHh6OkpISKQ5PGsDVFIi0x50VV4ioYZIEgK+88gouXbqEL774Aj///DP+/Oc/4+GHH0ZaWppQf89ZgWYiqsUiwUTKwdWBSGkkGQLu2bMnNm7ciIiICADA1atXERcXh9zcXKSnp6O6uho9e/ZUVQ8gh4C9RynDQkSkXmrNS1YbDgHXkiQANJvNyMjIQHBwsLCturoazz77LLKzs5Gamorhw4czACQiIlXQQl6yGjAArCXJEHCPHj3w3XffOQSAPj4+SElJwcyZMzFu3DgpDktEROQVzEsmpZGkyyomJgaffvppne06nQ7vvvsuRo8ezRxAIiIiIi9hHUCRcAiYSBrM8SSl4e+sfHEIuBYDQJEwACSSBmuqEZFYGADWYsRCREREpDGSTAIhImoO1lQjIpIWh4BFwiFgIvGwphoRSYFDwLUkjVg2btyIioqKOtuvXr2KjRs3SnloIlKw+Ph4HDx4EAcPHkRqaioAIDU1VdgWHx/v5RYSESmbpD2At9xyC44dOwaj0eiwvaSkBMHBwSwETUSN4iQQIs+x2apQVpaJq1ctaN3aBINhMHS6Ft5ulmjYA1hL0hxAm80GnU5XZ9uBAwfg5+cn5aGJiIjIDVZrGnJzZ6OiolDYptebERz8DozGMV5sGUlBkgDQz88POp0OOp0OISEhTveZM2eOFIcmohsovSaZyWRCYmKiIttOpBRWaxoOHx4LwHFQsKKiCIcPj0WfPpsZBKqMJEPA//73v2Gz2fDoo49i7dq1MBgMwn0tW7ZEQEAAAgMDxT6sV3EImOSKQ6hE1BCbrQpZWUEOPX+OdNDrzejf/7Tih4M5BFxLkh7Ae++9FwCQk5ODgICAOsPAREREJA9lZZkNBH8AYENFRQHKyjLh5zfUQ60iqUmaA/jDDz8gPz8fgwcPBgCsWLECGzZswO2334633nqrzuQQIhIH6+gRkauuXrWIuh8pg6RjlsnJybh69SoA4LvvvsNrr72GiRMn4pdffsFLL70k5aHJiywWC5KSkoQAhDwvJSUFUVFRiIqKQlxcHAAgLi5O2JaSkuLlFhKRXLRu7dofg67uR8ogaRkYk8mE//znPwgMDMQf/vAHFBQU4L333sPhw4cRExODkydPSnVoj2MOYC3mnHnfjT2AcXFxSE1NFa4HewCJyK42B7AIN04CqcEcQDWSdAi4Xbt2KCsrQ2BgIL7++mvMmDEDAHDTTTfhypUrUh6aSNOcBXiRkZEMyImoDp2uBYKD3/ltFrAOjkFgTQ5/cPAyxQd/5EjSAHDkyJGYNWsWwsLCcPLkSTzwwAMAgO+//x5BQUFSHlrRlFiIkzlnRETKZTSOQZ8+m+upA7iMJWBUSNIh4IqKCqxevRpFRUWYMGGC0PuwcuVKtG3bFk899ZRUh/Y4sYaAlVqIk2u31pBjzT05tomI5EmJHRDu4BBwLUkDQC0RIwCsrxCnvQtezoU4mXNWg/mPRETyxQCwlqRDwADwz3/+Ex988AHy8/ORnp6OgIAArFmzBoGBgbjvvvukPrxi2GxVyM2dDecJuDYAOuTmzoG/f6ws/xpjzhkREZFySDpt9W9/+xsWLFiAwYMHw2KxoKqqCgDQpk0bLFu2TMpDK447hThdYbNVobQ0A8XFG1FamgGbrUqchlIdFosF2dnZwg8Ah9ssh0NERHIjaQCYkpKCFStW4IUXXkCLFrW9VpGRkTh8+LCUh1YcMQtxWq1pyMoKQk5ONI4efQI5OdHIygqC1ZrW3Ga6RGtrt7LmHhERKY2kQ8CFhYXo2bNnne06nQ4VFRVSHlpxxCrEKYcFvU0mkyYmfNjFx8cjJiYGQP35j0REYlD7JA3yHEkDwJ49eyIzMxOTJk1y2P7pp58iPDxcykMrjsEwGHq9udFCnAbD4HqfQ+l5hErF/Eci8gSlVokgeZIkAHz++efx+uuvIzExEU8++SSOHTuGyspKrF+/HidOnMAXX3yBLVu2SHFoxRKjECcX9CYiUic5jO6QukiSA7hx40ZcuXIFQ4cOxb/+9S+UlJSgd+/e2LZtG1q2bImdO3fid7/7nRSHVjR7IU69vqvDdr3e7NKbmwt6e5/W8h+J5ELNa5A3ProD5ObO4WQ/coskdQD9/Pxw/PhxGI1GsZ9atsRcC7ipOR6lpRnIyYludL/w8N3sASQiVVFzDU5+touHdQBrSZYDWFRU1Oh6vwEBAVIdXtF0uhZNehOLkUdIRETywtEdkoJkAeCwYcPqvc9ms0Gn06GkpET04548eRIzZszA+fPn0aFDB6xcubLOTOR///vfWLJkCcrLy+Hj44NRo0Zh8eLF0Ol0yM/PR2RkJHr16iXsv27dOtx2222it1VsXNCbiLREK2uQi1Ulguh6kg0B79q1Cx07dmxwPymCqocffhgTJkzApEmTkJ6ejhUrVmDXrl0O++Tk5KBDhw4ICgrClStX8Mgjj2DatGkYN24c8vPzER0djVOnTrl1XDGHgJvL+UyxAC7oTUSqopU1yG22KmRlBTU6utO//2n+gd8IDgHXkiQAvOWWW3Ds2DGP5wBarVZERUXh1KlTaNmyJWw2G+644w7s2rUL3bp1q/dx8+fPR6dOnTB//nxVBIAAa0URkfppaQ3y2lnAgLPRHc4Cdg0DwFqSDAHbbKLHlC4pKiqCyWRCy5Y1p6XT6WA2m1FYWFhvAFhcXIz09HR88sknwraLFy8iOjoaVVVVeOihhzBv3jyHlUyUoKl5hERESqGlGpz2KhHO6wBydIfcJ0kAOHHiRLRp00aKp26UTqdzuN1QMPrLL79gwoQJmDVrFvr16wcA6NKlC44cOQKj0YjS0lI8/fTTWLFiBWbPni1ls4mIiBpkNI6Bv38sR3dIFJKMWa5cuRK+vr5SPHWDunbtirNnz6KyshJATfBXVFQEs9lcZ9+LFy9i7NixGDlyJGbOnCls1+v1wtC1n58fJk+ejH379nnmBIiIqEm0UoPTPrrTufNE+PkNZfBHTSafpDURGI1GhIaGYtOmTQCArVu3IjAwsM7wb3l5OcaOHYthw4ZhwYIFDvdZrVZcu3YNAFBRUYFt27YhLCzMMydAXqPmIrJEWmBfg1ztASCRWFQVAALAsmXLsHbtWkRFReHtt9/G8uXLAQAJCQnYsWMHAGD16tU4ePAgPvvsMwwaNAiDBg3C0qVLAQD79+/HkCFDMHDgQNx7773o1KkT5s2b57XzIefEDtgsFguSk5MZABIRkSZIMgtYi+Q2C1jtxK76r+ZVBIiIqAZnAdeSrBD09X788UecOHECAHD77bfXKcxM5A1aKSJLRER0I0kDwKKiIjz77LP45ptvYDAYAAAXLlzAwIEDsWrVKqeTM4jqI3bAlpKSUqeIbFxcnPB/tRSRJSIiupGkQ8AxMTGorKzEu+++K6z6cfr0aSQkJMDHxwdbt26V6tAexyFg6Yld9V9LRWSJiIhDwNeTNADs0qUL/vWvf6FPnz4O27///nvcf//9+Omnn6Q6tMcxAJSelAEbcwCJiNSPAWAtSYeAb7/9dvz88891tpeUlKBHjx5SHppUSEtV/4mIiKQkaQA4d+5czJ07FwkJCYiIiIBOp0N2djZWrFiBhQsXIi8vT9g3KChIyqYQNUjNRWQtFgtSUlIQHx+vyvMjIiL3SToE7Ofn1/DBdTrYbDbodDqUlJRI1QyP4BCwZzGocR2Ht4mIanAIuJakPYA5OTlSPj1pmL3qPxFRU/CPSNI6SQPAwMBAKZ+eiOrBGodEDbOv/hMTE8P3AmmS6AHgunXr8Pjjj0Ov12PdunUN7jtlyhSxD09EYI1DIiJqmOg5gGFhYcjIyMAtt9yCsLCw+g+s06lqiJg5gCQnrHFIVBffF8QcwFpcC1gkDABJrjgJhKiG2MXkSXkYANbyyFrARERE3hYfH4+YmBgA9fcAEmmFpAHgtWvXsHbtWuzduxdWqxXV1dUO93/++edSHp6IoO4ah0TuYDF5olqSBoD/93//hx07diA2NhZ33HEHdDqdlIcjIidYMoeIiG4kaQC4bds2bNiwAYMGDZLyMERERG5hzzhpnaQBoMFggL+/v5SHICIicht7xknrJJ22umTJEiQnJ+P8+fNSHoaIiIhUxGarQmlpBoqLN6K0NAM2W5W3m6Q6ovcA9unTxyHX7/z58wgJCYG/vz9atWrlsO8PP/wg9uGJiIhIwazWNOTmzkZFRaGwTa83Izj4HRiNY7zYMnURPQBctGiR2E9JREREMmCzVaGsLBNXr1rQurUJBsNg6HQtRHt+qzUNhw+PBeBYoriiogiHD49Fnz6bGQSKhIWgRcJC0EREpGZS98zZbFXIygpyeH5HOuj1ZvTvf7rJQScLQdeSNGL54osv8PXXX9fZ/vXXX2PXrl1SHpqIiIhEYu+ZuzE4s/fMWa1pzT5GWVlmA8EfANhQUVGAsrLMZh+LJA4AExMTYbPV7WDU6XR45ZVXpDw0ERERicBmq0Ju7mzcOCz7270AgNzcOc2eqHH1qkXU/ahhkgaAeXl56NGjR53t3bt3R15enpSHJiIiIhF4qmeudWvXajK6uh81TNIA0N/fH4cPH66z/X//+x/8/PykPDQRERGJwFM9cwbDYOj1ZgD1rRqmg14fAINhcLOOQzUkLQQ9ceJEzJs3D9XV1cJqIJmZmXjppZfwxBNPSHloIiIiEoGneuZ0uhYIDn7nt1nAOjgOOdcEhcHBy0SddaxlkgaAL730EqqrqxEXF4erV68CAPR6PZ5//nn8/ve/l/LQREREJAJ7z1xFRRGc5wHWzM4Vo2fOaByDPn021zPbeBlLwIjII2Vgrly5glOnTsFms6FHjx5o06aN1If0OLWUgZG6xhMRESlPbX0+wFnPnNj1+aT6LmIZmFqS9gDaXbt2DZWVlcL/1RgAqgGrrxMRkTOe7pnT6VrAz2+oqM9JjiTtAfz111+xcOFCrF+/XggAW7VqhcmTJ+O1117DTTfdJNWhPU7pPYD1VV+X6q87IiJSHqWPErEHsJakEcuCBQuwZ88efPzxx8jPz8eZM2fw0UcfYc+ePXjppZekPDS5wVM1ntTAYrEgKSkJFgvrUBGR9th75jp3ngg/v6GKCv7IkaQB4LZt27By5UoMHz4c7du3h6+vL+677z6sWLEC6enpUh6a3MDq666zWCxITk6WJABkcElERJ4iaQBYWVnpdJi3TZs2qKpib5JcsPq6PEgZXBIREV1P0kkgw4YNw9y5c7Fy5UrcfvvtAIDjx49jwYIFGDZsmJSHJjew+nrDLBaLEJRlZ2c7/AsAJpMJJpM2XxsiUi+l5/tRwyQNAN966y0888wzuPvuu+Hr6wsAKC8vR3R0NN566y0pD01u8GSNJyVKSUlBcnKyw7a4uDjh/4mJiUhKSmrSczO4JDWzWCxISUlBfHw8f48VhlUh1E/SWcAXLlzAzTffjLy8PJw4cQI2mw0hISFCb6CaqGcWMOCJGk9K+mK4MUiLi4tDamoqIiMjATQvSEtKSqoTXF6vOcElkbdlZ2cjKioKBw8eFN4vJH9qrgrBWcC1JAsAKysrYTKZsG/fPo8GfCdPnsSMGTNw/vx5dOjQAStXrkTPnj3r7Pfhhx9i2bJlqK6uxr333ou33noLLVvWdIju3LkTL7/8MiorK9G3b1+sWrUK7dq1a/C4Sg8Agfr+4guQpMaTUr8YxG63lMElkbcp9X2uZTZbFbKyghqYGFgzItS//2lFDgczAKwl2RBwy5Yt0aNHD5SVlUl1CKfmzJmDp556CpMmTUJ6ejoSEhKwa9cuh33y8vLwxz/+EXv27IHRaMTEiROxbt06PP300ygvL0dCQgK2b9+OkJAQzJ8/H2+99RYSExM9eh7eYDSOgb9/LHM+PMhZgBcZGckvS1IspjUomztVIVioWdkk7bJ69dVXsXjxYuzfvx/l5eWorq52+BGb1WpFTk4Oxo8fDwCIiYlBfn4+8vPzHfbbunUrRo8ejU6dOkGn02Hq1KnYvHkzAOCrr75CREQEQkJCAADTpk0T7tMCKWs8WSwWZGdnCz8AHG4rYfaryWRCYmIiv8CI6pGSkoKoqChERUUJubJxcXHCtpSUFC+3kBrCqhDaIekkkHHjxgEAHnroIaf3l5SUiHq8oqIimEwmYShXp9PBbDajsLAQ3bp1E/YrKChAQECAcDswMBCFhYX13mexWFBdXa34IV5vk3IyhaeYTCbJ2ihmcKmkHEtSl/j4eMTExACoP62B5ItVIbRD0gBw27ZtUj69UzqdzuG2zeY8xfH6/W7c58bnIHHwi6FhYgaX9pqCMTExmn9dybOUlNbAMid1sSqEdkgaAA4aNEjKp6+ja9euOHv2LCorK9GyZUvYbDYUFRXBbDY77BcQEIAzZ84ItwsKCoR9AgICkJlZu+LFmTNnYDKZ2PsnAiV9MRCRurHMiXM6XQsEB7/z2yxgHZxVhQgOXqb5QFkNJI1q1q1b53TJt/T0dGzYsEH04xmNRoSGhmLTpk0AanL9AgMDHYZ/gZrcwM8++wznzp2DzWbDmjVr8NhjjwEAhg8fjuzsbBw/fhwA8P777wv3EcmZGnIsSV3kmjNrL3Ny42SHiooiHD48FlZrmpdaJg9G4xj06bMZen1Xh+16vVnRJWDIkaR1ACMiIrB8+fI6PYH79+/H888/7zAzTCwnTpzAc889h5KSEvj6+mLVqlXo1asXEhISMHLkSIwaNQoA8MEHHwhlYIYMGYK//OUvaNWqFQBgx44dSExMRGVlJXr37o1Vq1ahffv2DR5XDWVgPIk5auJjTUGixqm9zImY1DhEzjIwtSQNADt37oz//Oc/CAoKctiel5eH/v3746effpLq0B7HAJC8jTUFiRpXWpqBnJzoRvcLD9/NMicqxACwlqQ5gF27dsX+/fvrBID79u1Dly5dpDw0keYwx5KocSxzQlRD0gDwmWeewYsvvoiSkhIMHDgQAPDNN9/gzTffxO9//3spD02kWGocdiGSC5Y5IaohaQD47LPP4qabbsLSpUuxePFiAIDZbMarr76KJ598UspDk0wwmHGPWDMT5Zp8T+RtLHNCVEPSHMDrlZeXw2azwdfX1xOH8zjmANbFMgvuUfMC7ERyUvteA5yVOeF7Tb2YA1jLYxFLu3btVBv8UV0ss+Aem60Kubmz4bxHomZbbu4c2GxVHm0XkRqxzAmRBD2Affv2xZ49e3DLLbegT58+Da6q8cMPP4h5aK9SUg+g1MOyLLPgPs5MJPI8taaoqPW8xMAewFqi5wAuXLgQbdu2BQAsWrRI7KenZvLEsGxZWWYDwR8A2FBRUYCyskwGM7/hzEQiz9PpWqjuM4ipN+Qq0QPAJ554wun/yfvqyzGzD8uKNfTBYMZ9nJlIRM3lqc94UgdJZgEXFBS4tF9AQIAUhycnGs8x0yE3dw78/WObPVTAYMZ9nJlIRM3hyc94UgdJAsDw8HDh/zZb7S+jPR/QZrNBp9OhpKREisOTE54clmUw4z4uwE5EzcHUG3KXJAHgzTffDIPBgPHjxyM2Npazf2XAk8OyDGaaxj4z0Xn+zjIO3RBRvZh6Q+6SJAA8fvw4tm3bho8//hjvvfceRowYgYkTJyI6OrrBWcEkHU8PyzKYaRqjcQz8/WM5g4+I3MLUG3KX5IWgz549i08++QSffPIJSkpKMG7cOLzyyito1aqVlIf1OLmXgaktzdLwsKzYpVlYjoCISHre+oxXGpaBqeWxlUAOHTqERYsWISsrC6dOnYLBYPDEYT1G7gEgwOr3RERqxs/4xjEArCVpxFJUVIS3334b99xzD5544glERkYiMzNTdcGfUmi9+r3NVoXS0gwUF29EaWkGV9UgIlXR+mc8uUeSHsANGzZg06ZNOHToEEaOHInx48cjOjpa9j1kzaGEHkA7LQ7LsjgqEWmFFj/jXcUewFqSBIB+fn7o2rUrHn74YbRr167e/dS0UoiSAkCtqa84KodFiMjOYrEgJSUF8fHxMJk4UUKtGADWkmQW8IABA6DT6fD999/Xuw9nA5MnsDgqEbnCYrEgOTkZMTExDABJEyQJALdv3y7F0xK5jcVRiYiI6pIkACSSCxZHJaL6WCwWWCw17/3s7GyHfwHAZDKxN5BUi0lrpGosjkoWiwVJSUnCFz2RXUpKCqKiohAVFYW4uDgAQFxcnLAtJSXFyy0kko7H6gCqHSeByBOLo1J2djaioqJw8OBBREZGers5JCM39gDGxcUhNTVV+D1hD6D6cBJILQ4Bk6pxXWIiqo+zAC8yMpJ/KJAmMAAk1eO6xNrD3C4iooZxzJI0wWgcg/798xAevhu9en2E8PDd6N//NIM/D/JkLh5zu+RLrjmZJpMJiYmJov9hINfzJWIOoEiYA0jNoYUitJ7MxWNul3xpLSdTa+crd8wBrMUhYCIZYBFacTG3i4ioYQwAiUgyzMUjQHu/B3I/X64VTAADQHITPzjEI/cvCTGkpKQgOTnZYZs9Jw8AEhMTkZSUJGkbpMrtItfJ4ffAk+R8vlZrWj0T4t5hTrTGMAdQJFrIAeQHh7iSkpLqfElcz50vCbnmEDIXjwDt/R7I9Xyt1rTfSmLd+LVfUxKrT5/Nqv8sZw5gLQaAIlF7AMgPDvGJ+SWhhERzJbSRpKe13wO5nG9tUfz61kbXRlF8BoC1OARMjbLZqpCbOxvOV9KwAdAhN3cO/P1jVf3BITZOVCAiTykry2wg+AMAGyoqClBWlgk/v6EeahV5EwNAahQ/OORJaTmEzMUTn1yH/huitd8DuZzv1auu1SF0dT9SPg4Bi0TNQ8DFxRtx9OgTje7Xq9dH6Nx5ogdapD5N+SIXM4eQlEkuw4skf6WlGcjJiW50v/Dw3ar+Q55DwLXYA0iNat3atYDE1f2oLpPJ5HawFh8fj5iYGAD15xASUeOU2JPqLoNhMPR6MyoqiuA8nacmB9BgGOzpppGXqCYAvHz5MhISEpCdnQ0fHx8kJiYKX47Xs1gseP7553HmzBm0bt0aISEhePvtt+Hn5wcACA0NRZs2baDX6wEAL7zwAsaM0fbkBn5wyBNzCLVJaUP/SqCFQuw6XQsEB7/z22Q+HRw/y2sm8wUHL2Met4aoZsxy+fLlaN26NQ4dOoR//OMfmDdvHsrKyurs16JFC8yfPx/ffvst9u3bh4CAgDo9Lx988AG++eYbfPPNN6oI/my2KpSWZqC4eCNKSzNgs1W59Xj7B8dvt268F4B2Pjia+1oSNRfXOaamMhrHoE+fzdDruzps1+vNrOSgQarpAdyyZQtWrlwJAAgKCsKAAQOwfft2TJo0yWG/Tp06oVOnTsLtO++8E2vWrPFoWz1JrNp99g8O58+1TBMfHHKugyiXRHOSHof+xaHVnlSjcQz8/WNZ0J/UMwnEbDbj0KFDMBqNAIBXXnkFbdu2xYsvvljvY6qqqhAbG4uHHnoIM2bMAFAzBNy+fXvYbDZERUUhMTER/v7+jR5fjpNApKjdp9WVQFgHkeSIk0CajpOotImTQGoppgdw5MiROHbsmNP79uzZAwDQ6WqHJ222huNam82GuXPnokOHDoiPjxe279ixAwEBAbh27RpeffVVzJgxA59++qkIZ+BZUtXu0+laqHqG2PXsieHPPDMd+fnaroOohSR50hb2pJLWKSYA/Pzzzxu832w248yZM0JvXUFBAe6///5691+wYAGKioqwYcMGh567gIAAAECrVq0wY8YM3HnnnSK03vNYu6/57Inho0bdqvnXUgtJ8krEof+m4yQq0jp5jVk2Q2xsLFJTUwEAeXl52Lt3L0aNGuV03wULFuD06dNYv349WrduLWy/dOmSw8SRzZs3IzQ0VNJ2S4VFP8Vjs/3s0n58LcnT7OWDGAASkbsU0wPYmFmzZmHmzJmIiIiAj48Pli5dKpR2WbNmDSwWCxYtWoSsrCy89957CAkJwfDhwwEA3bp1w4YNG2C1WjFlyhRUVVUJ21evXu21c2oO1u5rGmeJ4bm5pejataFH1VDba6nVJHnSHvakkhapZhKIt8ltEkjtwt8N1+5T+8Lf7nKWGO7jA2zcCPj71/y/LnW+lkySJyK14SSQWgwARSK3ABC4fuYq4Kzop1pnrjZnpvKNvV72xPDQ0DL8+ut81FdAVY2vZX2vxfVJ8uwxISIlYQBYSzVDwFSXFmv3NbdWX0OJ4VZrd029lkySJyJSLwaAKqelop/11eqrqCjC4cNjm91Lp6XXkoiI1I0BoAZooXafFHUPnSWGa+G1dIZJ8kRE6sIcQJHIMQdQS0pLM5CTE93ofuHhuzUZwBEREXMAr8eIhVSBdQ+JiIhcxwCQVIF1D4mIiFzHAJBUwWAYDL3eDHtZlrp00OsDYDAM9mSziIiIZIkBIKmCTtcCwcHv2G/deC8AIDh4GWfsEhERgQEgqYi97qFe77hum15vVmWhZiIioqbiLGCRcBawfDRnJRAiIlIvzgKuxTqApDpardUnJgbRRMrD9y25gwEgETlo7nJ6ROR5fN+SuzgELBIOAZMa1Lecnn0iDXMpieSH71vXcQi4FiMWIgLgynJ6QG7uHNhsVR5tFxHVj+9baioGgEQEACgry3QYPqrLhoqKApSVZXqsTUTUML5vqakYABIRAC6nR6REfN9SUzEAJCIAXE6PSIn4vqWmYgBIRADcX07PYrEgKSkJFgt7Foi8hctgUlMxACRqgM1WhdLSDBQXb0RpaYaqE6ndXU7PYrEgOTnZKwEgg0+iGlwGk5qKASBRPazWNGRlBSEnJxpHjz6BnJxoZGUFwWpN83bTJKOU5fS8GXyKjcEsNZdS3rckLywETR6jpCr19dXVqqgowuHDY1X9oWo0joG/f6zTa2WxWIRAJTs72+FfADCZTDCZmGvkDnswGxMTo7rXTknveaVr6H1L5AwDQPIIJVWpb7yulg65uXPg7x+r2g/X+pbTS0lJQXJyssO2uLg44f+JiYlISkqSpE0MPpVFSe95teAymOQOrgQiEq4EUj+lVakvLc1ATk50o/uFh+/W3IftjUFYXFwcUlNTERkZCUDaICwpKalO8Hk9KYNPsXnzdfQEpb3nSTu4Ekgt9gCSpJTYm8a6WvVzFphERkYKgYuU4uPjERMTA6D+oEkpvNmTKjUlvueJtIgBIEnKnSr1culNY10tefJm8Ck2NQWzN1Lie55IixgAkqSU2Jtmr6tVUVEE570YOuj1Zk3V1XKWzG8ymZCYmKjoYMVb1BTM3kiJ73kiLWIASJJSYm+ava5WTQ6TDo5BoPbqajWUzO+tYUoGn/KlxPc8kRZx1gJJSqlV6llXq4Y9mf/GIT17ORxv1UQ0mUxISkpSRQCotmBWqe95Iq3hLGCRcBZw/WpnBALOetPkHFBpuY6ZzVaFrKygBvK5aobC+/c/rZnXhFyj5Pc8qRtnAddixEKSU3Jvmr2uVufOE+HnN1RTgY47yfxE11Pye55IK5gDSB7BKvXKw2R+ag6+54nkjQEgeQyr1CsLk/mpufieJ5IvDgETkVNM5iciUi8GgETklL0czm+3brwXgLbK4RARqQkDQCKqF5P5iYjUSTVlYC5fvoyEhARkZ2fDx8cHiYmJwlJLNzIYDOjdu7dQtuXNN9/EgAEDAAAnT57EjBkzcP78eXTo0AErV65Ez549Gz0+y8CQmmm5HA4RqQfLwNRSzSSQ5cuXo3Xr1jh06BDy8vLwwAMPYMiQITAYDE73//LLL9GuXbs62+fMmYOnnnoKkyZNQnp6OhISErBr1y6JW08kb0zmJyJSF9V0WW3ZsgVxcXEAgKCgIAwYMADbt2936zmsVitycnIwfvx4AEBMTAzy8/ORn58venuJiIiIvEU1AWBhYSECAgKE24GBgSgsrL+I7ejRozFw4EAsXLgQly5dAgAUFRXBZDKhZcuajlGdTgez2dzg8xCR+lksFiQlJcFiYc1D8j6brQqlpRkoLt6I0tIM2GxV3m4SKZBiAsCRI0eie/fuTn/sAZpOVztT0WarP7Xx+++/R0ZGBr788kv8/PPPeOWVV4T7rn+Oxp6HiLTBYrEgOTmZAaCHMfCuy2pNQ1ZWEHJyonH06BPIyYlGVlaQ19blJuVSTAD4+eef49SpU05/zGYzzGYzzpw5I+xfUFAAs9ns9LnsPYVt27bF9OnTsX//fgBA165dcfbsWVRWVgKoCf6KiorqfR4iIpIOA29H9jWWb1yisaKiCIcPj2UQSG5RTADYmNjYWKSmpgIA8vLysHfvXowaNarOfmVlZbh8+TKAmtlAaWlpCA0NBQAYjUaEhoZi06ZNAICtW7ciMDAQ3bp189BZEJFcWCwWZGdnCz8AHG4zKCFPstmqkJs7G4CzUamabbm5czgcTC5TzSzgWbNmYebMmYiIiICPjw+WLl0KPz8/AMCaNWtgsViwaNEiHD9+HHPmzIFOp0NVVRXCwsLwxhtvCM+zbNkyPPfcc/jLX/4CX19frFq1ylunRERelJKSguTkZIdt9olmAJCYmIikpCQPt0p5LBYLUlJSEB8fD5Op8WUDLRaLEFxfH3jbmUwml55HTO6egxTKyjLr9Pw5sqGiogBlZZmcsU8uUU0dQG9jHUAidbkxEImLi0NqaioiIyMBeCcQUaLs7GxERUXh4MGDwmvXkKSkpDqB9/W8EXi7ew5SKC7eiKNHn2h0v169PkLnzhM90CJlYh3AWqrpASQiEpOzAC8yMtJrAYBWxMfHC0X86wu81c5Z4fXWrV07b1f3I2IASEREomrOMK5cAm9vDUVbrWnIzZ3tMNyr15vRo8fb0OvNqKgogvM8QB30ejMMhsGit4nUiWOWRESNMJlMSExM1ETvkxhSUlIQFRWFqKgoIW8yLi5O2JaSkuLlFjbOG+fQ0CzfI0ceR6dO9qFd3Q2PrLkdHLyMSzSSy5gDKBLmABKRWjR30oNY+ZPenHzh6RxQm60KWVlBDUz00P3WE/gWTp584YYewgAEBy+D0ThGtPaoFXMAazEAFAkDQCLXOctxUkrPhZLbDrjWfjEnPchhAkVzeeIcSkszkJMT3eh+4eG7YTAMVvTvoDcxAKzFHEAi8qj6cpyCg9+RfQ+GktsOKL/9anb1qmt1Ja9etUCna8FSL9RsDACJyGPsOU43JrHbVzLo02ezbAMRJbcdaLz9t96aivLyCADiTnpQQ/6kJ86Bs3zJ0zgELBIOARM1zNUcp/79T8tuOEvJbQdca/+VK7546KFfUF3tfA9P1N9T+vB6c9Reo4Zn+cr1d0wpOARciz2AROQRSl7JQMltB1xrf5s2v2D//hS0bHmnV+rvaX14WqdrgeDgd37rpdXBMQjkLF8SH7usiMgj3Mlxkpvmtt1isSApKclr6we72v5u3Xwdau7Z/x8ZGSlpANhQ+ZPDh8fCak2T7NhyYjSOQZ8+m6HXd3XYrtebZZ9iQMrDHkAi8ggl5zg1t+0WiwXJycmIiYnxSi6cnF97m60Kubmz4XzY0wZAh9zcOfD3j9VE75fROAb+/rGaHQonz2EASEQeYTAMVuxKBkpuO+B++z05cUPpw+tS4Cxf8gQGgETkEUrOcWpK2721lJgz7rbfZDJJPuHDTsmpAURKxhxAIvIYJec4udt2uS2HJtfXXs7D00RqxjIwImEZGCLXKbnch6tt9/RSYq6S22vP8ifkSSwDU4tDwEQKJ7cvdFcoOcfJ1bY7C/Cun2HrLXJ77ZWcGkCkZAwAiRRM67XTSB3sw9POf5eX8XeZSAIcAhYJh4DJ0+pb2sveayL3nDotsVgsSElJQXx8vKKXRJOaEnuzSVk4BFyLAaBIGACSJyl9aTIiIm9gAFiLEQuRArlTO42IiOhGDACJFIi104iIqDkYABIpEGunERFRczAAJFIg+9Je9gkfdemg1wfIdmkyIiLyLgaARApkr532260b7wXA2mlERFQ/BoBECiXXpb2IiEj+WAZGJCwDQ97C2mlERK5hGZhaXAmESOHktrQXERHJH7usiIiIiDSGASARERGRxjAAJCIiItIYBoBEREREGsMAkIiIiEhjGAASERERaQzLwBARkaqwNiZR4xgAEhGRalitacjNnY2KikJhm15vRnDwO1wdh+g6HAImIiJVsFrTcPjwWIfgDwAqKopw+PBYWK1pXmoZkfyopgfw8uXLSEhIQHZ2Nnx8fJCYmIiYmJg6+/3444+YPn26cPvChQu4ePEi8vLyAAChoaFo06YN9Ho9AOCFF17AmDH8q5FITThEqD42WxVyc2cDcLa6qQ2ADrm5c+DvH8trTQQVBYDLly9H69atcejQIeTl5eGBBx7AkCFDYDAYHPbr2bMnvvnmG+H2/Pnz6zzXBx98gN69e0vdZCLyAg4RqlNZWWadnj9HNlRUFKCsLJNLJxJBRUPAW7ZsQVxcHAAgKCgIAwYMwPbt2xt8TEVFBT799FNMmTLFE00kIi9TyxChzVaF0tIMFBdvRGlpBmy2Km83yeuuXrWIuh+R2qmmB7CwsBABAQHC7cDAQBQWNvTXILBt2zZ069YNYWFhDtvj4uJgs9kQFRWFxMRE+Pv7S9JmIvIctQwRsgfTudatTaLuR6R2iukBHDlyJLp37+70xx7o6XQ6YX+bzdmHvKP169fX6f3bsWMH9u7di3//+9+45ZZbMGPGDHFPhIi8wp0hQrlSSw+mFAyGwdDrzQB09eyhg14fAINhsCebRSRbiukB/Pzzzxu832w248yZM0JvXUFBAe6///569z9z5gwOHDiAtWvXOmy39yK2atUKM2bMwJ133tm8hhORLCh9iFAtPZhS0elaIDj4HRw+PBY1QeD1r1NNUBgcvEyTrw2RM4rpAWxMbGwsUlNTAQB5eXnYu3cvRo0aVe/+GzZswOjRox0miVy6dAllZWXC7c2bNyM0NFSqJhORByl9iFANPZhSMxrHoE+fzdDruzps1+vN6NNns6aHyIlupJgewMbMmjULM2fOREREBHx8fLB06VL4+fkBANasWQOLxYJFixYBqBke/uijj/Duu+86PIfVasWUKVNQVVWTUN2tWzesXr3asydCRJKwDxFWVBTBeS+aDnq9WbZDhErvwfQUo3EM/P1jWeaHqBG6srKyxpPlqFG+vr7w8VFNhyqRKtlz6GrUHSKUcy9RaWkGcnKiG90vPHw3y5wQ1aO6uhoXL170djNkgRELEWmGkocIOcmBiMTEHkCRsAeQSDmUuhKIknswieSAPYC1GACKhAEgEXmC8zqAAQgOXsbgj6gRDABrMQAUCQNAIvIUpfZgEnkbA8BaqpkFTESkFTpdC070IKJmYZcVERERkcYwACQiIiLSGAaARERERBrDAJCIiIhIYxgAEhEREWkMA0AiIiIijWEASERERKQxrAMokurqam83gYiIiBrA7+paDABFcunSJW83gYiIiMglHAImIiIi0hgGgEREREQawwCQiIiISGMYABIRERFpDANAIiIiIo1hAEhERESkMQwAVeLy5cuYNm0aIiIiEBUVha1btzrd78cff8SgQYOEn9DQUAQFBQn3h4aG4q677hLuT0tL89AZuM7VcwUAg8GAAQMGCOezb98+4b6TJ0/igQceQFRUFIYNG4Yff/zRE813i6vnarFYMGbMGNx5550YMGAA/t//+38oLS0V7pfzdXX1Onz44YeIjIxEv379MHv2bFRWVgr37dy5E3fddRciIiIwZcoUlJeXe6r5bnHlXP/9739j+PDhuOeee/C73/0Of/jDH2Cz2QAA+fn56Nixo8N7+PTp054+DZe4cq6ZmZkwmUwO5/Prr78K9yvlugKune/GjRsdzrV79+6YPHkyAOVc2wULFiA0NBQGgwFHjhypdz81vF/VTldWVmbzdiOo+d544w3k5eVh1apVyMvLwwMPPIADBw7AYDA0+Lj58+cDAP785z8DqAkUNm3ahN69e0vd5CZz51wNBgMKCwvRrl27Ovc9/PDDmDBhAiZNmoT09HSsWLECu3bt8sAZuM7Vcz137hxOnjyJ3/3udwCAl19+Gb/88gveeecdAPK+rq5ch7y8PIwYMQJ79uyB0WjExIkT8eCDD+Lpp59GeXk5IiIisH37doSEhGD+/Plo164dEhMTvXRG9XPlXHNyctChQwcEBQXhypUreOSRRzBt2jSMGzcO+fn5iI6OxqlTp7x0Bq5z5VwzMzPx8ssvIyMjo87jlXRdgaZ9ngwYMAAvvvgiYmNjFXNt9+7di6CgIIwYMaLezxS1vF/Vjj2AKrFlyxbExcUBAIKCgjBgwABs3769wcdUVFTg008/xZQpUzzRRNE05VxvZLVakZOTg/HjxwMAYmJikJ+fj/z8fNHb2xyunmunTp2E4A8A7rzzTuTl5XmqmU3m6nXYunUrRo8ejU6dOkGn02Hq1KnYvHkzAOCrr75CREQEQkJCAADTpk0T7pMTV881PDxc6JVv06YNQkNDFXEtryfG+0sp1xVo2vkePHgQ586dw6hRozzVTFEMHDgQXbt2bXAfNbxftYABoEoUFhYiICBAuB0YGIjCwsIGH7Nt2zZ069YNYWFhDtvj4uIwYMAAJCQk4Oeff5akvc3h7rmOHj0aAwcOxMKFC4UVW4qKimAymdCyZc1iODqdDmazudHXzNOacl2rqqqQmpqKESNGOGyX43V19ToUFBTU+zo4u89ischuyaem/M4VFxcjPT0d999/v7Dt4sWLiI6OxpAhQ/DGG2+gqqpK8ra7y51zzc3NxZAhQxAdHY2//e1vwnalXFegadd23bp1GD9+PFq1aiVsU8K1dYUa3q9awKXgFGLkyJE4duyY0/v27NkDoOZDx86eM9SQ9evX1+n927FjBwICAnDt2jW8+uqrmDFjBj799NNmtNx9Yp7r999/j4CAAFy6dAn/93//h1deeQVvvfVWnedo7HmkIvZ1tdlsmDt3Ljp06ID4+Hhhuxyua31cvQ4NvQ43PodcufM798svv2DChAmYNWsW+vXrBwDo0qULjhw5AqPRiNLSUjz99NNYsWIFZs+eLWWzm8SVcw0PD8fhw4fRoUMHFBUVYdy4cejYsSMeffRRp88hZ+5c28uXLyMtLQ1ffvmlsE1J19YVani/qh0DQIX4/PPPG7zfbDbjzJkz8Pf3B1DzV9b1vQY3OnPmDA4cOIC1a9c6bLf/ZdaqVSvMmDEDd955Z/Ma3gRinqv9fNq2bYvp06djzpw5AICuXbvi7NmzqKysRMuWLWGz2VBUVASz2SzeibhA7Ou6YMECFBUVYcOGDfDxqe3gl8N1dcbV6xAQEIAzZ84ItwsKCoR9AgICkJmZKdx35swZmEwmh/OXA3d+5y5evIixY8di5MiRmDlzprBdr9fDaDQCAPz8/DB58mR8+umnsgsSXD3X9u3bOzxm7Nix2LdvHx599FHFXFfA/c+T9PR03HHHHejZs6ewTSnX1hVqeL9qAV9xlYiNjUVqaiqAmgTcvXv3NphbsmHDBowePdphMsGlS5dQVlYm3N68eTNCQ0OlanKTuXquZWVluHz5MgCguroaaWlpwvkYjUZhYgRQk7MSGBiIbt26eegsXOPOdV2wYAFOnz6N9evXo3Xr1sJ2OV9XV69DTEwMPvvsM5w7dw42mw1r1qzBY489BgAYPnw4srOzcfz4cQDA+++/L9wnJ66ea3l5OcaOHYthw4ZhwYIFDvdZrVZcu3YNQE0O77Zt2+qkcMiBq+f6008/CUN/Fy9exBdffCGcj1KuK+D+54mz0RelXFtXqOH9qgWcBawSly5dwsyZM/Hdd9/Bx8cHr7zyCmJjYwEAa9asgcViwaJFiwDUdMeHhYXh3XffxZAhQ4TnyMvLw5QpU4S8k27duuH111+XXVDk6rkeOHAAc+bMgU6nQ1VVFcLCwvDGG2/Az88PAHDixAk899xzKCkpga+vL1atWoVevXp589TqcPVcs7KyMGLECISEhAjBX7du3bBhwwbZX9f6rkNCQgJGjhwpBLwffPABli1bhurqagwZMgR/+ctfhPypHTt2IDExEZWVlejduzdWrVrl0LskF66c69KlS/H666879A498sgjmDdvHrZu3Yo//elP8PHxQVVVFQYPHoxXX30Ver3ei2flnCvn+t5772HNmjVo0aIFqqqqEBsbi5deekkYIlTKdQVc/z0+ffo0Bg8ejKNHj8LX11d4vFKu7bx587Bjxw4UFxejY8eOaNu2LQ4dOqTK96vaMQAkIiIi0hgOARMRERFpDANAIiIiIo1hAEhERESkMQwAiYiIiDSGASARERGRxjAAJCIiItIYBoBEREREGsMAkIg8bsOGDejdu7esjx8aGooPP/zQQy0iIvIsBoBEJIqHHnoIBoMB69atc9h+6dIlmM1mGAwG5OfnAwDGjBmDPXv2eKOZLtu9ezfGjRvXpMfOmDEDBoOh3p8ZM2Y06XkzMjIclm+sz9atWxETE4PAwEAYDAZUVlY26XhEpF4MAIlINF27dhXWQ7Xbtm0bOnTo4LDtpptugr+/f5OOce3aNdhs0i9g5O/vj5tuuqlJj3399ddx7NgxHDt2DLt27QIAfP3118K2119/Xcym1vHrr79iyJAhmDNnjqTHISLlYgBIRKIZPXo0Dh06hIKCAmHbxx9/jMcff9xhP2dDsKtXr0ZERAQ6deqE8PBwYfg1MzMTBoMBu3btwj333IMuXbrgwoULOHfuHJ588kl07doV3bp1w8yZM3Hp0iXh+SorK/Haa6+hb9++6Ny5M+6++258/vnnDsdMT09HWFgYAgMD8fzzz6OiokK47/oh4Pz8fBgMBmzZsgWDBg1C586d8fDDD6OwsNDp69ChQwd07twZnTt3RseOHQEAHTt2FLYdOnQIQ4cORZcuXRAVFYXU1FThsVeuXMGsWbMQHByMLl264K677sJnn32G/Px8PPLIIwAg9CRu2LDB6fHHjx+PefPm4a677nJ6PxERA0AiEo2vry9GjhyJTz75BABw9uxZ/Pe//0VsbGyDj/vggw/w2muvYe7cufjPf/6D5cuXw9fX12GfN998E3/961+xb98+3HzzzYiPj0dRURG2b9+Ojz/+GPv27cPChQuF/f/0pz/hww8/xB//+EdkZWXhtddeExajB4CSkhJ89NFH2LhxI9avX48dO3Zg7dq1DbbzD3/4A5YsWYKvvvoKlZWViI+Pd/MVAk6cOIEpU6Zg6tSpQrveeOMNpKWlAQBSUlLw3XffYfPmzcjKysIf//hH+Pr6wmw2C+2z9ySOGTPG7eMTEQFAS283gIjUZcKECVi4cCHmzp2LTz75BCNGjED79u0bfMyf//xnLFiwAJMnTwYA3HbbbXX2SUxMxD333AMAOH78OHbv3o2srCz07NkTQE2AOGHCBCxZsgStW7fGihUr8N577yEmJsbpc1ZUVGD58uXo1KkTACA2NhZ79+5tMKibOXMmhg0bBgB49913ERkZiSNHjrg1oWXZsmV46qmn8OSTTwIAgoKCMGPGDHzwwQcYM2YMCgsLERYWhn79+gn329nz/zp37uzy8YiInGEASESiio6OxoULF5CdnY1NmzZhyZIlDe5/8eJFFBYWYtCgQQ3uFxYWJvz/xIkT8PX1FYI/ALjrrrtQWVmJ06dPo1WrVqioqGjwOf39/YXgDwA6deqEY8eONdiGyMhI4f/du3eHwWDAiRMn3AoAjxw5giNHjuDvf/+7sK2yshJdunQBUBNAP/roo/j+++8xfPhwxMTECMEgEZFYGAASkahatGiBsWPHYtGiRTh//jyGDRsmzP51xtUJHTfffHODj9HpdG495/XDwfbHN/a464/RVJcuXcLzzz8v9HbatWjRAgAQFRWFnJwcfPHFF/jXv/6FBx98EIsXL0ZCQkKzj01EZMccQCIS3cSJE7F//36MHTtWCGzq0759e5jNZnzzzTcuP39ISAguXryIH3/8Udh24MABtGzZErfddht69OgBvV7v1nO6Ijs7W/j/6dOnUVZWhttvv92t5+jbty9OnDiB7t27O/x069ZN2MfPzw8TJkxAamoqFi5ciPXr1wMAWras+Zu9qqpKhLMhIi1jDyARia5v3744deoU2rZt69L+8+fPx+LFi3HLLbdg4MCBOHv2LM6dOyfMer1RSEgIhg0bhpkzZ2Lp0qW4cuUKXnzxRUyaNEkoOTNz5ky8+OKL0Ol0CAsLw6lTp1BdXY377ruvyee1YsUK3HbbbejYsSNeeuklDBgwwO2C1rNmzcIDDzyAV199FWPHjoXNZkN2djZ+/fVXTJ8+He+++y66du2K0NBQXLlyBV9//TWCg4MBAAEBAQCAL7/8EnfffTfatWsHvV5f5xilpaUoKCjA6dOnAQA//PADfHx80L17d7Rr167J509E6sEAkIgkccstt7i871NPPYXy8nK88cYb+Omnn3Drrbdi7ty5DT5m9erVmDt3Lh566CG0aNECMTEx+OMf/yjc//vf/x4AsGDBApSWliIoKKjRfMTGLFy4EAsXLkRubi7uuusurF692u3n6NevH7Zs2YJXX30VK1asgF6vR58+fYSafW3btsWbb76J06dPo02bNrj33nvxxhtvAAC6deuGOXPm4Pnnn0dJSQneffddTJo0qc4xduzYgeeff164PXToUAA1NRkHDx7s/okTkeroysrKpK+oSkSkYPn5+QgPD0d2dja6d+/u7eYQETUbcwCJiIiINIYBIBEREZHGcAiYiIiISGPYA0hERESkMQwAiYiIiDSGASARERGRxjAAJCIiItIYBoBEREREGsMAkIiIiEhjGAASERERaQwDQCIiIiKNYQBIREREpDEMAImIiIg0hgEgERERkcYwACQiIiLSGAaARERERBrDAJCIiIhIYxgAEhEREWkMA0AiIiIijWEASERERKQx/x9WDqQISD4CkwAAAABJRU5ErkJggg==",
"text/html": [
"\n",
"
\n",
"
\n",
" Figure\n",
"
\n",
" \n",
"
\n",
" "
],
"text/plain": [
"Canvas(toolbar=Toolbar(toolitems=[('Home', 'Reset original view', 'home', 'home'), ('Back', 'Back to previous …"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Plot examples\n",
"plot_data(X_train, y_train[:], pos_label=\"Accepted\", neg_label=\"Rejected\")\n",
"\n",
"# Set the y-axis label\n",
"plt.ylabel('Microchip Test 2') \n",
"# Set the x-axis label\n",
"plt.xlabel('Microchip Test 1') \n",
"plt.legend(loc=\"upper right\")\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "a5b2e048",
"metadata": {},
"source": [
"Figure 3 shows that our dataset cannot be separated into positive and negative examples by a straight-line through the plot. Therefore, a straight forward application of logistic regression will not perform well on this dataset since logistic regression will only be able to find a linear decision boundary.\n"
]
},
{
"cell_type": "markdown",
"id": "f9dda84b",
"metadata": {},
"source": [
"##### 3.3 Feature mapping\n",
"\n",
"\n",
"One way to fit the data better is to create more features from each data point. In the provided function `map_feature`, we will map the features into all polynomial terms of $x_1$ and $x_2$ up to the sixth power.\n",
"\n",
"$$\\mathrm{map\\_feature}(x) = \n",
"\\left[\\begin{array}{c}\n",
"x_1\\\\\n",
"x_2\\\\\n",
"x_1^2\\\\\n",
"x_1 x_2\\\\\n",
"x_2^2\\\\\n",
"x_1^3\\\\\n",
"\\vdots\\\\\n",
"x_1 x_2^5\\\\\n",
"x_2^6\\end{array}\\right]$$\n",
"\n",
"As a result of this mapping, our vector of two features (the scores on two QA tests) has been transformed into a 27-dimensional vector. \n",
"\n",
"- A logistic regression classifier trained on this higher-dimension feature vector will have a more complex decision boundary and will be nonlinear when drawn in our 2-dimensional plot. \n",
"- We have provided the `map_feature` function for you in utils.py. "
]
},
{
"cell_type": "code",
"execution_count": 40,
"id": "6905b1de",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Original shape of data: (118, 2)\n",
"Shape after feature mapping: (118, 27)\n"
]
}
],
"source": [
"print(\"Original shape of data:\", X_train.shape)\n",
"\n",
"mapped_X = map_feature(X_train[:, 0], X_train[:, 1])\n",
"print(\"Shape after feature mapping:\", mapped_X.shape)"
]
},
{
"cell_type": "markdown",
"id": "f5cdea39",
"metadata": {},
"source": [
"Let's also print the first elements of `X_train` and `mapped_X` to see the tranformation."
]
},
{
"cell_type": "code",
"execution_count": 41,
"id": "ef06b33e",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"X_train[0]: [0.05 0.7 ]\n",
"mapped X_train[0]: [5.13e-02 7.00e-01 2.63e-03 3.59e-02 4.89e-01 1.35e-04 1.84e-03 2.51e-02\n",
" 3.42e-01 6.91e-06 9.43e-05 1.29e-03 1.76e-02 2.39e-01 3.54e-07 4.83e-06\n",
" 6.59e-05 9.00e-04 1.23e-02 1.68e-01 1.82e-08 2.48e-07 3.38e-06 4.61e-05\n",
" 6.29e-04 8.59e-03 1.17e-01]\n"
]
}
],
"source": [
"print(\"X_train[0]:\", X_train[0])\n",
"print(\"mapped X_train[0]:\", mapped_X[0])"
]
},
{
"cell_type": "markdown",
"id": "79288cae",
"metadata": {},
"source": [
"While the feature mapping allows us to build a more expressive classifier, it is also more susceptible to overfitting. In the next parts of the exercise, you will implement regularized logistic regression to fit the data and also see for yourself how regularization can help combat the overfitting problem.\n",
"\n",
"##### 3.4 Cost function for regularized logistic regression\n",
"\n",
"\n",
"In this part, you will implement the cost function for regularized logistic regression.\n",
"\n",
"Recall that for regularized logistic regression, the cost function is of the form\n",
"$$J(\\mathbf{w},b) = \\frac{1}{m} \\sum_{i=0}^{m-1} \\left[ -y^{(i)} \\log\\left(f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right) - \\left( 1 - y^{(i)}\\right) \\log \\left( 1 - f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right) \\right] + \\frac{\\lambda}{2m} \\sum_{j=0}^{n-1} w_j^2$$\n",
"\n",
"Compare this to the cost function without regularization (which you implemented above), which is of the form \n",
"\n",
"$$ J(\\mathbf{w}.b) = \\frac{1}{m}\\sum_{i=0}^{m-1} \\left[ (-y^{(i)} \\log\\left(f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right) - \\left( 1 - y^{(i)}\\right) \\log \\left( 1 - f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right)\\right]$$\n",
"\n",
"The difference is the regularization term, which is $$\\frac{\\lambda}{2m} \\sum_{j=0}^{n-1} w_j^2$$ \n",
"Note that the $b$ parameter is not regularized."
]
},
{
"cell_type": "markdown",
"id": "786f8919",
"metadata": {},
"source": [
"###### Exercise 5\n",
"\n",
"\n",
"Please complete the `compute_cost_reg` function below to calculate the following term for each element in $w$ \n",
"$$\\frac{\\lambda}{2m} \\sum_{j=0}^{n-1} w_j^2$$\n",
"\n",
"The starter code then adds this to the cost without regularization (which you computed above in `compute_cost`) to calculate the cost with regulatization.\n",
"\n",
"If you get stuck, you can check out the hints presented after the cell below to help you with the implementation."
]
},
{
"cell_type": "code",
"execution_count": 42,
"id": "137c8eaa",
"metadata": {},
"outputs": [],
"source": [
"# UNQ_C5\n",
"def compute_cost_reg(X, y, w, b, lambda_ = 1):\n",
" \"\"\"\n",
" Computes the cost over all examples\n",
" Args:\n",
" X : (array_like Shape (m,n)) data, m examples by n features\n",
" y : (array_like Shape (m,)) target value \n",
" w : (array_like Shape (n,)) Values of parameters of the model \n",
" b : (array_like Shape (n,)) Values of bias parameter of the model\n",
" lambda_ : (scalar, float) Controls amount of regularization\n",
" Returns:\n",
" total_cost: (scalar) cost \n",
" \"\"\"\n",
"\n",
" m, n = X.shape\n",
" \n",
" # Calls the compute_cost function that you implemented above\n",
" cost_without_reg = compute_cost(X, y, w, b) \n",
" \n",
" # You need to calculate this value\n",
" reg_cost = 0.\n",
" \n",
" ### START CODE HERE ###\n",
" reg_cost = sum(np.square(w))\n",
" ### END CODE HERE ### \n",
" \n",
" # Add the regularization cost to get the total cost\n",
" total_cost = cost_without_reg + (lambda_/(2 * m)) * reg_cost\n",
"\n",
" return total_cost"
]
},
{
"cell_type": "markdown",
"id": "fa69610d",
"metadata": {},
"source": [
"\n",
" Click for hints\n",
" \n",
" \n",
"* Here's how you can structure the overall implementation for this function\n",
" ```python \n",
" def compute_cost_reg(X, y, w, b, lambda_ = 1):\n",
" \n",
" m, n = X.shape\n",
" \n",
" # Calls the compute_cost function that you implemented above\n",
" cost_without_reg = compute_cost(X, y, w, b) \n",
" \n",
" # You need to calculate this value\n",
" reg_cost = 0.\n",
" \n",
" ### START CODE HERE ###\n",
" for j in range(n):\n",
" reg_cost_j = # Your code here to calculate the cost from w[j]\n",
" reg_cost = reg_cost + reg_cost_j\n",
"\n",
" ### END CODE HERE ### \n",
" \n",
" # Add the regularization cost to get the total cost\n",
" total_cost = cost_without_reg + (lambda_/(2 * m)) * reg_cost\n",
"\n",
" return total_cost\n",
" ```\n",
" \n",
" If you're still stuck, you can check the hints presented below to figure out how to calculate `reg_cost_j` \n",
" \n",
" \n",
" Hint to calculate reg_cost_j\n",
" You can use calculate reg_cost_j as reg_cost_j = w[j]**2 \n",
" \n",
" \n",
" \n",
"\n",
"
"
]
},
{
"cell_type": "markdown",
"id": "b22ef148",
"metadata": {},
"source": [
"##### 3.5 Gradient for regularized logistic regression\n",
"\n",
"\n",
"In this section, you will implement the gradient for regularized logistic regression.\n",
"\n",
"\n",
"The gradient of the regularized cost function has two components. The first, $\\frac{\\partial J(\\mathbf{w},b)}{\\partial b}$ is a scalar, the other is a vector with the same shape as the parameters $\\mathbf{w}$, where the $j^\\mathrm{th}$ element is defined as follows:\n",
"\n",
"$$\\frac{\\partial J(\\mathbf{w},b)}{\\partial b} = \\frac{1}{m} \\sum_{i=0}^{m-1} (f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) - y^{(i)}) $$\n",
"\n",
"$$\\frac{\\partial J(\\mathbf{w},b)}{\\partial w_j} = \\left( \\frac{1}{m} \\sum_{i=0}^{m-1} (f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) - y^{(i)}) x_j^{(i)} \\right) + \\frac{\\lambda}{m} w_j \\quad\\, \\mbox{for $j=0...(n-1)$}$$\n",
"\n",
"Compare this to the gradient of the cost function without regularization (which you implemented above), which is of the form \n",
"$$\n",
"\\frac{\\partial J(\\mathbf{w},b)}{\\partial b} = \\frac{1}{m} \\sum\\limits_{i = 0}^{m-1} (f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) - \\mathbf{y}^{(i)}) \\tag{2}\n",
"$$\n",
"$$\n",
"\\frac{\\partial J(\\mathbf{w},b)}{\\partial w_j} = \\frac{1}{m} \\sum\\limits_{i = 0}^{m-1} (f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) - \\mathbf{y}^{(i)})x_{j}^{(i)} \\tag{3}\n",
"$$\n",
"\n",
"\n",
"As you can see,$\\frac{\\partial J(\\mathbf{w},b)}{\\partial b}$ is the same, the difference is the following term in $\\frac{\\partial J(\\mathbf{w},b)}{\\partial w}$, which is $$\\frac{\\lambda}{m} w_j \\quad\\, \\mbox{for $j=0...(n-1)$}$$ \n",
"\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "61898e54",
"metadata": {},
"source": [
"###### Exercise 6\n",
"\n",
"\n",
"Please complete the `compute_gradient_reg` function below to modify the code below to calculate the following term\n",
"\n",
"$$\\frac{\\lambda}{m} w_j \\quad\\, \\mbox{for $j=0...(n-1)$}$$\n",
"\n",
"The starter code will add this term to the $\\frac{\\partial J(\\mathbf{w},b)}{\\partial w}$ returned from `compute_gradient` above to get the gradient for the regularized cost function.\n",
"\n",
"\n",
"If you get stuck, you can check out the hints presented after the cell below to help you with the implementation."
]
},
{
"cell_type": "code",
"execution_count": 44,
"id": "76cef830",
"metadata": {},
"outputs": [],
"source": [
"# UNQ_C6\n",
"def compute_gradient_reg(X, y, w, b, lambda_ = 1): \n",
" \"\"\"\n",
" Computes the gradient for linear regression \n",
" \n",
" Args:\n",
" X : (ndarray Shape (m,n)) variable such as house size \n",
" y : (ndarray Shape (m,)) actual value \n",
" w : (ndarray Shape (n,)) values of parameters of the model \n",
" b : (scalar) value of parameter of the model \n",
" lambda_ : (scalar,float) regularization constant\n",
" Returns\n",
" dj_db: (scalar) The gradient of the cost w.r.t. the parameter b. \n",
" dj_dw: (ndarray Shape (n,)) The gradient of the cost w.r.t. the parameters w. \n",
"\n",
" \"\"\"\n",
" m, n = X.shape\n",
" \n",
" dj_db, dj_dw = compute_gradient(X, y, w, b)\n",
"\n",
" ### START CODE HERE ### \n",
" for j in range(n):\n",
" dj_dw[j] = dj_dw[j] + (lambda_/m) * w[j]\n",
" ### END CODE HERE ### \n",
" \n",
" return dj_db, dj_dw"
]
},
{
"cell_type": "markdown",
"id": "bab72c69",
"metadata": {},
"source": [
"\n",
" Click for hints\n",
" \n",
" \n",
"* Here's how you can structure the overall implementation for this function\n",
" ```python \n",
" def compute_gradient_reg(X, y, w, b, lambda_ = 1): \n",
" m, n = X.shape\n",
" \n",
" dj_db, dj_dw = compute_gradient(X, y, w, b)\n",
"\n",
" ### START CODE HERE ### \n",
" # Loop over the elements of w\n",
" for j in range(n): \n",
" \n",
" dj_dw_j_reg = # Your code here to calculate the regularization term for dj_dw[j]\n",
" \n",
" # Add the regularization term to the correspoding element of dj_dw\n",
" dj_dw[j] = dj_dw[j] + dj_dw_j_reg\n",
" \n",
" ### END CODE HERE ### \n",
" \n",
" return dj_db, dj_dw\n",
" ```\n",
" \n",
" If you're still stuck, you can check the hints presented below to figure out how to calculate `dj_dw_j_reg` \n",
" \n",
" \n",
" Hint to calculate dj_dw_j_reg\n",
" You can use calculate dj_dw_j_reg as dj_dw_j_reg = (lambda_ / m) * w[j] \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"\n",
" \n"
]
},
{
"cell_type": "markdown",
"id": "f05e0b4e",
"metadata": {},
"source": [
"Run the cell below to check your implementation of the `compute_gradient_reg` function."
]
},
{
"cell_type": "code",
"execution_count": 45,
"id": "5b26b0dd",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"dj_db: 0.07138288792343662\n",
"First few elements of regularized dj_dw:\n",
" [-0.010386028450548701, 0.011409852883280122, 0.0536273463274574, 0.003140278267313462]\n",
"\u001b[92mAll tests passed!\n"
]
}
],
"source": [
"X_mapped = map_feature(X_train[:, 0], X_train[:, 1])\n",
"np.random.seed(1) \n",
"initial_w = np.random.rand(X_mapped.shape[1]) - 0.5 \n",
"initial_b = 0.5\n",
" \n",
"lambda_ = 0.5\n",
"dj_db, dj_dw = compute_gradient_reg(X_mapped, y_train, initial_w, initial_b, lambda_)\n",
"\n",
"print(f\"dj_db: {dj_db}\", )\n",
"print(f\"First few elements of regularized dj_dw:\\n {dj_dw[:4].tolist()}\", )\n",
"\n",
"# UNIT TESTS \n",
"compute_gradient_reg_test(compute_gradient_reg)\n"
]
},
{
"cell_type": "markdown",
"id": "c1d9f328",
"metadata": {},
"source": [
"**Expected Output**:\n",
"
"
]
},
{
"cell_type": "markdown",
"id": "0b4a8a31",
"metadata": {},
"source": [
"##### 3.6 Learning parameters using gradient descent\n",
"\n",
"\n",
"Similar to the previous parts, you will use your gradient descent function implemented above to learn the optimal parameters $w$,$b$. \n",
"- If you have completed the cost and gradient for regularized logistic regression correctly, you should be able to step through the next cell to learn the parameters $w$. \n",
"- After training our parameters, we will use it to plot the decision boundary. \n",
"\n",
"**Note**\n",
"\n",
"The code block below takes quite a while to run, especially with a non-vectorized version. You can reduce the `iterations` to test your implementation and iterate faster. If you have time, run for 100,000 iterations to see better results."
]
},
{
"cell_type": "markdown",
"id": "7c0683be",
"metadata": {},
"source": [
"###### Regularised Gradient Descent "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3aef0b72",
"metadata": {
"scrolled": true
},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"id": "10233bd1",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": 46,
"id": "6e1c8da6",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Iteration 0: Cost 0.72 \n",
"Iteration 1000: Cost 0.59 \n",
"Iteration 2000: Cost 0.56 \n",
"Iteration 3000: Cost 0.53 \n",
"Iteration 4000: Cost 0.51 \n",
"Iteration 5000: Cost 0.50 \n",
"Iteration 6000: Cost 0.48 \n",
"Iteration 7000: Cost 0.47 \n",
"Iteration 8000: Cost 0.46 \n",
"Iteration 9000: Cost 0.45 \n",
"Iteration 9999: Cost 0.45 \n"
]
}
],
"source": [
"# Initialize fitting parameters\n",
"np.random.seed(1)\n",
"initial_w = np.random.rand(X_mapped.shape[1])-0.5\n",
"initial_b = 1.\n",
"\n",
"# Set regularization parameter lambda_ to 1 (you can try varying this)\n",
"lambda_ = 0.01; \n",
"# Some gradient descent settings\n",
"iterations = 10000\n",
"alpha = 0.01\n",
"\n",
"w,b, J_history,_ = gradient_descent(X_mapped, y_train, initial_w, initial_b, \n",
" compute_cost_reg, compute_gradient_reg, \n",
" alpha, iterations, lambda_)"
]
},
{
"cell_type": "markdown",
"id": "97f9d12d",
"metadata": {},
"source": [
"\n",
"\n",
" Expected Output: Cost < 0.5 (Click for details)\n",
"\n",
"\n",
"```\n",
"# Using the following settings\n",
"#np.random.seed(1)\n",
"#initial_w = np.random.rand(X_mapped.shape[1])-0.5\n",
"#initial_b = 1.\n",
"#lambda_ = 0.01; \n",
"#iterations = 10000\n",
"#alpha = 0.01\n",
"Iteration 0: Cost 0.72 \n",
"Iteration 1000: Cost 0.59 \n",
"Iteration 2000: Cost 0.56 \n",
"Iteration 3000: Cost 0.53 \n",
"Iteration 4000: Cost 0.51 \n",
"Iteration 5000: Cost 0.50 \n",
"Iteration 6000: Cost 0.48 \n",
"Iteration 7000: Cost 0.47 \n",
"Iteration 8000: Cost 0.46 \n",
"Iteration 9000: Cost 0.45 \n",
"Iteration 9999: Cost 0.45 \n",
" \n",
"```"
]
},
{
"cell_type": "markdown",
"id": "25ea15f9",
"metadata": {},
"source": [
"##### 3.7 Plotting the decision boundary\n",
"\n",
"\n",
"To help you visualize the model learned by this classifier, we will use our `plot_decision_boundary` function which plots the (non-linear) decision boundary that separates the positive and negative examples. \n",
"\n",
"- In the function, we plotted the non-linear decision boundary by computing the classifier’s predictions on an evenly spaced grid and then drew a contour plot of where the predictions change from y = 0 to y = 1.\n",
"\n",
"- After learning the parameters $w$,$b$, the next step is to plot a decision boundary similar to Figure 4.\n",
"\n",
""
]
},
{
"cell_type": "code",
"execution_count": 47,
"id": "6d59cdd7",
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "2b07931d87014b7f881c17310c7d83a5",
"version_major": 2,
"version_minor": 0
},
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAoAAAAHgCAYAAAA10dzkAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAA9hAAAPYQGoP6dpAAB4hElEQVR4nO3deVhUZcMG8HtYZUdBlGVQRNFEFLWs3MAldzHNHdMyy9wKTX3Ldj/T0lLeNK0sU4RcykxTc3tzX3IdF1zQcRvAHUFEZJv5/kAGxxlgYM7MmeX+XRcX8pwz5zwMznDzrJLMzEwViIiIiMhm2IldASIiIiIyLQZAIiIiIhvDAEhERERkYxgAiYiIiGwMAyARERGRjWEAJCIiIrIxDIBERERENoYBkIiIiMjGMAASERER2RgGQCIiIiIbwwBIREREZGMYAImIiIhsDAMgERERkY1hACQiIiKyMQyARERERDaGAZCIiIjIxjAAEhEREdkYBkAiIiIiG8MASERERGRjGACJiIiIbAwDIBEREZGNYQAkIiIisjEMgEREREQ2hgGQiIiIyMYwABIRERHZGAZAIiIiIhvDAEhERERkYxgAiYiIiGwMAyARERGRjWEAJCIiIrIxDIBERERENoYBkIiIiMjGMAASERER2RgGQCIiIiIbwwBIREREZGMYAImIiIhsDAMgERERkY1hACQiIiKyMQyARERERDaGAZCIiIjIxjAAEhEREdkYBkAiIiIiG8MASERERGRjGACJiIiIbAwDIBEREZGNYQAkIiIisjEMgEREREQ2hgGQiIiIyMYwABIRERHZGAZAIiIiIhvDAEhERERkYxgAiYiIiGwMAyARERGRjbGpADh16lRERETA29sbZ86c0XnOnj174O/vj7Zt26o/cnNzTVxTIiIiIuNxELsCptSnTx+8++676NatW7nnNWzYEDt37jRNpYiIiIhMzKYCYJs2bcSuAhEREZHobKoLWF8XL15E+/bt0aFDB/z0009iV4eIiIhIUDbVAqiPZs2aITk5GV5eXkhLS8OAAQPg4+ODvn37il01IiIiIkGwBfApnp6e8PLyAgAEBgaif//+2L9/v8i1IiIiIhIOWwCfcuPGDfj5+cHOzg7Z2dnYsmULhg0bVuHj3NzcYGfHPE1ERGSulEolcnJyxK6GWbCpADh58mRs2rQJN2/exMsvvww3NzccP34cEyZMQPfu3dGjRw+sX78eS5Ysgb29PYqKitCnTx+9AqCdnR0DIBEREVkESWZmpkrsSlgDDw8PBkAiIiIzplQqkZ2dLXY1zIJNtQASERGZA6VSiZs3b6KwsFDsqlg1BwcH1KpViw00OjAAEhERmdjNmzfh4eEBd3d3sati1R48eICbN2/C399f7KqYHUZiIiIiEyssLGT4MwF3d3e2spaBAZCIiIjIxjAAEhEREdkYBkAiIiILcf36dXz22We4fv264NfOzs6Gu7s7Ro0aJfi1K7Jz505s3bq1yo999tlnBa6R9WMAJCIishDXr1/H559/bpQAuHLlSrRo0QJr1qzBgwcPBL9+eQwJgFQ1DIBERESEn3/+Gf/5z3/Qrl07rF69GgCQlZWFUaNGISIiAs2aNcPIkSMBAPn5+ZgyZYq6vFu3burrfP3112jVqhVatGiBHj16QKFQAAA+++wzDBw4ED169ECTJk0QExODe/fuQSaT4fvvv0dCQgIiIyMxffp0AMCWLVvQtm1btGzZEs8//zx2796tvsdHH32E+vXrIyoqChs2bDDVU2RVuAwMERGRGbt+/bq6xe/YsWManwHA39/f4GVOkpOToVAo0K1bNxQWFmL27NkYOXIk4uLi4O7ujhMnTsDOzg63b98GAMyaNQtyuRxHjhyBs7OzuvzXX39FSkoKDhw4AHt7eyxfvhzjx4/HunXrAAB79uyBTCZDrVq1MHbsWHz44YdYuHAh3n77bTx48ABff/01AODSpUv4/PPPsXnzZnh6euLixYuIiorClStXsHnzZqxfvx4ymQwuLi7o27evQd+7rWIAJCIiMmM//PADPv/8c42yN998U/3vTz/9FJ999plB9/j5558xfPhw2Nvbo2fPnnj77bdx9uxZbNiwAUePHlUvpFyzZk0AwIYNG/DNN9/A2dlZo/zPP//EkSNH0LJlSwBAUVER7O3t1ffp1asXatWqBQB46623MHDgQJ312bx5My5evIj27dtrlCsUCuzYsQODBg1SL6MzcuRIzJgxw6Dv3xYxABIREZmx0aNHIyYmBkBxy9+bb76JxYsXo0WLFgBgcOtfQUEBEhMT4ejoiBUrVgAAHj58iCVLllT6WiqVCh999JG6q7giEomkzOt069YNCQkJOo+R4TgGkIiIyIz5+/ujRYsW6g8AGl8bGgDXrVuHevXqIS0tDVeuXMGVK1ewb98+JCQkICYmBnPmzIFSqQQAdVdvTEwM4uPjkZeXp1W+cOFCZGRkACgOl8ePH1ffa+PGjbh16xaA4lbHzp07AwA8PT2RlZWlPq9Lly7YvHkzTp8+rS47dOgQAKBTp05YvXo1cnJyUFRUhKVLlxr0/dsqBkAiIiIb9vPPPyM2NlajrEmTJggICEBUVBQePnyIJk2aIDIyEtOmTQMA/Oc//0FoaCiaN2+OyMhIjBgxAgDw6quvYtiwYYiOjkazZs0QGRmJHTt2qK/bqVMnvPHGG2jSpAmuXr2q7rrt27cvjhw5op4E0qBBAyQmJmLUqFFo1qwZnnnmGfz3v/8FUNyN3KtXLzRr1gwdO3ZE06ZNTfE0WR1JZmYm21IF4OHhwc2miYhILwqFAlKptNKPu379On744QeMHj3a4va3/eyzzzQmepjKk8+1UqlEdna2Se9vrjgGkIiIyEL4+/sbPOGDCGAAJCIiIhNgcDUv7LMkIiIisjEMgEREREQ2hgGQiIiIyMZwDCAREZEFUKmKkJm5B/n51+Hk5A9v73aQSOwrfiCRDmwBJCIiMnO3b/+Bgwfr4sSJDjh7dihOnOiAgwfr4vbtPwS5ft26ddGoUSNERkaiYcOG+PLLLyt8TI8ePSCXy6t8z6VLlyIlJaVKj/3ss88wefLkKt+b2AJIRERk1m7f/gPJyf0BaC7bm5eXhuTk/ggP/x01a/Yz+D6///47mjRpgvT0dDRu3BgdO3ZEq1atyjx/06ZNBt1v6dKl8PX1RVhYmEHXoaphCyAREZGZUqmKcPHiu3g6/D0+CgC4eDEOKlWRYPcMCAhAw4YNcfXqVdy4cQMDBw5Eq1at0LRpU3zyySfq8+rWraveqq28886ePYuuXbuiadOmaNq0Kb7//nv89NNPOHLkCN555x1ERkaqw+TXX3+NVq1aoUWLFujRowcUCgUAICsrC/3790fjxo3RtWtXXLx4UbDv11YxABIREZmpzMw9yMtLLecMFfLyFMjM3CPYPc+dO4c7d+4gOjoaI0aMwPjx43Ho0CEcO3YMhw4dwtq1a7UeU9Z5hYWF6NOnD9544w2cPHkSJ0+eRP/+/TFq1Cg8++yz+PbbbyGTydCjRw/8+uuvSElJwYEDB3Ds2DEMGTIE48ePBwBMnz4dnp6eOHPmDJKSkrB7927Bvl9bxS5gIiIiM5Wff13Q88rTv39/SCQSnD9/HvPmzYOrqyv++ecf3Lx5U33OgwcPcO7cOY3H5eTklHleWFgYCgsLMXDgQPUxX19fnff/888/ceTIEbRs2RIAUFRUBHv74kkuO3bswPz589WP79fP8C5vW8cASEREZKacnPTb71ff88pTMgZw+/bt6N27Nzp27AiJRILDhw/D0dGxzMcplcoyz0tOTtb7/iqVCh999BFGjhyp8xgJi13AREREZsrbux2cnYMASMo4QwJnZym8vdsJds/OnTtjzJgx+Oijj9CuXTuNGcHp6elITdXskvbw8CjzvIYNG8LJyQm//fab+tidO3cAAJ6ensjKylKXx8TEYOHChcjIyAAAFBQU4Pjx4wCATp064ZdffgEAZGRk6OyGpsphACQiIjJTEok96tf/b8lXTx8FANSvHy/4eoAff/wx9u7di+nTp+Ps2bOIiIhAREQEXnnlFdy9e/eJ+hXXISkpSed5Dg4OWLduHX788UdERESgadOmWLNmDQDgrbfewvTp09WTQF599VUMGzYM0dHRaNasGSIjI7Fjxw51fe7du4fGjRsjNjYWL730kqDfry2SZGZmsl1VAB4eHrCzY54mIqKKKRQKSKVSvc+/ffsPXLz4rsaEEGdnKerXjxdkCZjKKiwshI+PD86fP4/atWub/P6V8eRzrVQqkZ2dLXKNzAPHABIREZm5mjX7wde3j1nsBJKeno727dtj2LBhZh/+qGwMgERERBZAIrFH9erRYlcDAQEBXIfPCrDPkoiIiMjGMAASERER2RgGQCIiIhFwbTvj43NcNgZAIiIiE/Py8lKvh0fGc+fOHXh5eYldDbPESSBEREQm5unpiVu3bkGhUIhdFavm7OwMT09PsathlhgAiYiIRODn5yd2FciGsQuYiIiIyMYwABIRERHZGAZAIiIiIhvDAEhERERkYxgAiYiIiGwMAyARERGRjWEAJCIiIrIxDIBERERENoYBkIiIiMjGMAASERER2RhuBUdEakplPtLSFiI3Vw4Xl1AEBo6FnZ2T2NUiIiKBSTIzM1ViV8IaeHh4wM6ODapkueTyqVAo5gIoeqLUHlLpJISGzharWkREglEqlcjOzha7GmaBLYBE9Dj8zdFxpEhdzhBIRGQ92GRFZOOUyvzHLX9lUyjmQqnMN1GNiIjI2BgAiWxcWtpCaHb76lL0+DwqDszxSEmZAIUinsGYiCwSu4CJbFxurlzQ86yZrnGScvlkjpMkIovDAEhk41xcQgU9z1pxnCQRWRN2ARPZuMDAsQDsKzjL/vF5tonjJInI2jAAEtk4OzsnSKWTyj1HKp1k0+sBcpwkEVkbdgETkbrrkusA6sZxkkRkbRgAiQhAcQgMCZnBnUB04DhJIrI23AlEINwJhMh6KZX52L3bFeV3A9ujffuHDMxEZow7gZRiYiEiqgDHSRKRtWEXMBGRHjhOkoisCbuABcIuYCLboFTmc5wkkYViF3ApBkCBMAASERGZNwbAUkwsRERERDaGYwDJqrB7Tjx87omILIdNdQFPnToVf//9NxQKBfbv34/GjRvrPC8hIQHx8fFQKpWIiorCN998AweH8rMyu4DFV7xXKwfoV4ZQoY3PPRFZAnYBl7KpxNKnTx9s3rwZUqm0zHOuXLmCmTNnYvPmzTh+/Dhu3ryJ5cuXm7CWVBXFAWQOtNdpK4JCMQdy+VQxqmXW5PKp2L3bFXL5RKSnL4BcPvHx15V7rvjcExFZHpsKgG3atEFgYGC556xfvx69evWCn58fJBIJRo4cid9//91ENaSqUCrzH7c+lU2hmAulMt9ENTJ/QoU2PvdERJbJpgKgPhQKhUYLYXBwMFJTU0WsEVUkLW0hyt+hAQCKHp9HQoY2PvdERJaJAVAHiUSi/rdKZTNDJC1Wbq5c0POsnZChjc89EZFlYgB8ilQqxbVr19RfKxQKBAUFiVgjqoiLS6ig55maTCZDdHQ0ZDKZSe4nZGiz9OeeiMhWMQA+JSYmBhs2bMCtW7egUqmwZMkSvPLKK2JXi8oRGDgWgH0FZ9k/Ps/8JCcnY9euXUhOTjbJ/YQMbZb+3BMR2SqbCoCTJ09G48aNkZ6ejpdffhnNmzcHAEyYMAGbNm0CANStWxcffPABunbtisjISNSsWROvvvqqmNWmCtjZOUEqnVTuOVLpJK5J95iQoY3PPRGRZbKpdQCNiesAis+S1qKTyWTqFr+tW7ciISEBw4cPR5cuXQAA4eHhiIyMNNr9S2cB6yaVTqnUc2bOzz0XqCaiElwHsBQDoEAYAM2Dpfyyj46Oxq5du8o8HhUVhZ07dxq1DkKHNnN87s05mBKR6TEAlmIAFAgDIFWG2C2AJcwxtAlF6FZOIrJ8DIClGAAFwgBovYwdkpKSkjBs2DAkJiYiNjZWsOtaAmM9t0plPnbvdkX5y93Yo337h1YTeImoYgyApcrf4JbIxunqQpTLJ7MLUQDGfG4rs9ahVBpn0L2IiCwRAyBZNGO2zpXdhVikLhciBIaHhyMqKgrh4eEGX8tSGPu55QLVRETlYxewQNgFbHrGHODPLkTjMcVzq1DEQy6fWOF5oaHz2AJIZEPYBVyKiYUsUmkL0tMhorgFSS6fatD1ucet8ZjiueUC1aZXvMd0PFJSJkChiNdrL2kiEg8DIFmc4l80c8s9R6GYa9AvIHYhGo8pnlsuUG1acvlU7N7tCrl8ItLTF0Aun/j4a8P+ECMi42EAJItjihYk7nFrPKZ6bkNDZ0MqnQLtlkB7LgEjIGO3xhORcXAMoEA4BtB0UlImID19QYXnBQSMR1jY/Crdg2MAjcfUz601r3UoNr5OyNJwDGApJhayOKZoQWIXovGY+rktvl8cwsLmQyqN489MQBwrS2S5GADJ4phqgD+7EI2Hz6114FhZIsvFdQDJ4pS0IJW/zZcwLUihobMREjKDXYhGwOfW8nGsLJHl4hhAgXAMoOkZcx1AIqoYxwCSpeEYwFIMgAJhABSHNQ/wL1QWIutRFrLysjQ+Zz7K1Cp7WPgQeYV5yC/KR15RHvIK83R+LlJq/qKWSCSl/0bpv+0kdnB2cIazvTOcHZxRzaGa+t9PfnZxcIGHswc8nDzg6ewJD+fHn5/62tPZE97VvGEn4WvE2pS9q0sxdumTOWEALMUAKBAGQNJX5qNMpN1Pw40HNzQ/cjS/vvPwjthVFZSdxA7Vq1WHr6svfF194ePqA18XX82vXX1R2702gjyDUNu9NgOjhWBrPFkKBsBSDIACYQCkp93LvYfk28lIvpVc/Pnxv2/m3BS7ahbByd4JwV7BqOtdF3W86qCud131Rx2vOgjwCIC9XUWTgSyHpbdmW3r9yTYwAJZiABQIA6DtKigqwKlbp3A0/ahG0Lv+4LrYVbNqzvbOqF+jPhr6NkRYjTCE+YQV/9snDL6uvmJXr1LYgkZkGgyApRgABcIAaDuuZ1/HwdSDOJh6EAdSD+BI+hHkFuaKXS16Qg2XGgjzKQ6FjXwaoYlfE0TUikAdrzoa4x7NAcfQEZkOA2ApBkCBMABap/yifBy/fhwHUg+oQ9/VrKtiV0tDNYdqcHdy1zlJ4+nPDnalKz+pUPrSV6k03wYKlYXIL8rHo8JHGpNIbmfcxvVb1+Hl6wU7RzvkFubiUeEjk32vhvJw8kC4Xzgi/CIQ4RehDoZitRhyFi2RaTEAlmIAFAgDoPWQZ8ix+eJmbJZvxo7LO5BTkGOS+3o4eaC2e22ND19XX3g5e8GrmleZn53sNYOBTCZDXFwc4uPjERkZKWgdk5KSMGzYMCQmJiI2NhZAcRd4dn42svOycT/vPrLzH39+4uvMR5m4+/Au7uTewZ2Hd4r//fAObj+8bRYBsrZ7bUT4RaClf0s8G/AsWga0NElroUIRD7l8YoXnhYbOg1QaZ9S6ENkCBsBSXAiabF5Ofg52XNmBzRc3Y4t8Cy5mXDTKffzd/RHuF47wmuFoUKMB/D381UGvllstuDm5CXKf5ORk7Nq1C8nJyYIEQJlMhuTkZADA1q1bNT4DQHh4OCIjI1HDpUaVrv+w4KE6FN7KuYX07HQo7itwJfOK+kNxX4FCZaHB30tZSmZeb7u0TV3m4+JTHAafCIVST6mgoZA7aRCRWBgAySaduX0Gmy5swuaLm7Hn2h7kF+ULdu1abrXUQS+8ZjjC/cLRuGbjKgckscXFxWHXrl0aZQkJCUhISAAAREVFYefOnVW+vqujK4K9ghHsFazzuFKZj2uKBbiacRK3Cz2Q49gCVzKv4ULGBZy/ex4pd1NwP+9+le9flru5d7FFvgVb5FvUZTVda+LZgGfRWtoa7YLboVVgK7g4ulT5HtxJg4jEwi5ggbAL2PzdfHATPx//Gb+e+hXJt5MFuWb1atXxQtAL6o+W/i3h4+qj81xjds0+3UqXkJCA4cOHo0uXLgBKW+nM7doV0Wd2rEqlwq2cW+oweP7OeaRkpCDlbgouZlw0asuho50jWga0RFtpW3QJ7YJ2ddqhmkM1vR/PMYBEpsUu4FIMgAJhADRf5++cx9wDc7HsxDLkFeVV+Tp2EjtE+EXghaAX8GLQi3gh6AU08Gmg92LFusbPCSU6Olqrle5JhrbSlTDm9/A0IWbHFhQVIOVuCk7dOoVTN0/h9O3TOHXzFC5nXha6ugAAFwcXdAjpgG6h3dC9QXfUr1G/wsdwFjCR6TAAlmIXMFkllUqFfYp9mLN/DtafX1+la0ggQcuAlugW2g0dQzriucDn4O7kLnBNhREfH19hK50lUSrzH7f8lU2hmIuQkBnltow52jsWd8f7hWNwk8Hq8qzcu0ja5ofLOUpczgHkD4ALD4Cc8hri9JBbmItNFzZh04VNwGagoU9D9G3UF32f6YtnA57V+cdCSbgz1jqAXKCZiHRhACSrUqQswp/n/sTXB77GwdSDlX68n5sfuoZ2Rbf63fBSvZdQ062mQfXRdwKFoSIjIzWuk5CQgC5dugjeShceHo6oqCijB8q0tIUov1sUAIqQlrawSrNj799ZjsaeSjT2LC1TqoD0XCDlAXA+G0jJBi4+dMaDgqq3Gp+/ex5f7vsSX+77EoEegejTsA9ebvQyoutGw9HeUX1eaOhshITMEDyo6epCl8snc4FpImIAJOtQpCxC0qkkTN81HfJ7+s+YtJfYo7W0NbrV74Zu9bshsnakoPvPGnsChalFRkaapL7Gnh2r63F2EiDItfijo19xWW3/UZD4TMDR60dxJP0IDqcfxuG0w1UaSpCWnYaFRxZi4ZGF8K7mjV5hvfByw5fRrX43uDm5wc7OSdClXsruWi5SlzMEEtkuBkCyeNvk2zBl2xScuHlCr/MlkKBr/a4Y2mQoYhrGwKual9HqJkbXrKla6YzJ2LNj9X2cm2t9SH0boqFvQwyNGAoAyCvMw9HrR7H32l7svbYXe67tQeajzErdP/NRJhJPJiLxZCKqOVRD9/rdMTRiKHo26GnQrOISQnWhE5H14iQQgXASiOmduHECU7dPxVb51opPRvHesa9FvoaJL0xEQ9+GRq6dNlNOoLB0xp4dK+T1i5RFOJx+GH9f+Bub5ZtxOO2wxi4rleHp7Il+z/TD0CZD0TGkI+zt7Kt0HS4wTaQbJ4GUYgsgWRxFlgIf7/gYCScS9PpFW8OlBsY9Nw7jW42Hn5ufCWpIhiruDp1UwezYSVVuvRLy+vZ29uplgD7v8Dlu59zGFvkW/HnuT/x98W88LHiod73u593HUtlSLJUtRS23WhjcZDCGRgzFcwHPVWoBalMsMM3JJUSWjS2AAmELoPEVKYswa+8sfLHnC722D6tXvR4mvTAJr0W+JtguG4Yw5jqA1kqfdQDN+fq5BbnYfmk7/jz3J9anrMedh3eqdJ0wnzCMfXYsXot8Ta8hC8ZuATT280ZkLGwBLMUAKBAGQONKu5+G2D9isetq2Wvdlahfoz5mdJiB/o37V7kLjcyHsVuaTNWSVagsxH7Ffqw9uxZrz63F1ayrlb6Gh5MHxjw7BhNfnIja7rXLPM+YXehct5AsGQNgKQZAgTAAGodSmY+kfyfg3V0JuJdXfqufr6svPo36FKNbjtZYYqOy92O3FhmbSqXCiZsnsObMGqw4vaJSM9eB4vGsr0e+jiltpqBe9Xo6zzFGUOPOJWTpGABLMQAKhAFQeHL5VPz34NdYcLH8kX4uDi6Y9OIkTG0zFZ7OnuWcWfH92K1FpqZSqXA4/TB+PfUrVp5eiZs5N/V+rL3EHoObDMb7bd9HE78mWseF/j/NySVk6RgASzGxkFm6cHEK3t8xB/PLCX8SSDAyciRSJqRgRscZAoS/OdBu2SheM00un1rla5uKTCZDdHQ0ZDKZ2FXRmyXWWWgSiQStAlshvls8UielYuuwrXgt8jV4OHlU+NgiVfH6lxGLIhCzIkZr8fPQ0Nlo3/4hQkPnISBgPEJD5z3+ump/0JhicgkRmQYDIOmleF2xeKSkTIBCEQ+lMt9o98rJy8IbW77G72lln1O7GrBv5G783OdnBHkGGXQ/fddMM+b3rI+KwlJycjJ27dqlXnfQElhinStiSKh1sHPAS6Ev4Zc+v+Dm5JtY1X8V2tdpr9dj/0r5Cy/+/CI6LOuAbfJtUKmK/3QqWWA6LGw+pNI4g7pmjb0+IxGZDgMgVUgun4rdu10hl09EevoCyOUTH38tfKvY7ZzbiFoSiT3lTJZs7wssbgkE4Ygg96zMtmNiMnZYYmucMIT6Obk4umBg+EDsem0Xjr51FAMaD4AEFS8Fs/PKTnRJ7IKopVHYd22fQXV4WmDgWAAVTayyf3weEZkzrgNI5TLldlIX7l5A96TukN+7UuY5b4UAg6WARCJcN5Mld2sJudfwk8HFmMvUmGp/ZGvSwr8FVg9YjfN3zmP2vtlYfnI5CpQF5T5mz7U9aPtLW/QO642ZnWbqHCNYWcZen5GITIcBkMpkyu2kzt85j7a/tC1znTRHCfB+o9I9WgHhupnMuVurorA0d+5cHDt2TOMx5r7XsLXtjwyYLtQ29G2ICXUmIHlhMuoPq4+1irUVLjT9V8pf2HRhEya+MBGfRX9m8JqYJX/wWcKEKc7qJyobZwELxBpnAZtqxl9Gbgae/+l5XMy4qPO4hwMwIxxo6v1kqXBLTZjz0hbR0dFaYelJLVq0wKRJkwCUvddwecHj6eBS2cdXhRj3NLaKfk5ChtontxTs2rcrvv33W8w/NF+v/YjretfFop6L0K1+N4PrYe7hirP6SRfOAi7FFkAqkym6RguVhRj0+6Ayw59/NeDLCCDYVbNcyG4mc+7Wio+Pr1RYSkhIQJcuXfTea1iM1rjIyEiD6myO9Pk5GYOvqy+md5iOKa2n4IejP+Dr/V+Xu4zMlcwr6J7UHUOaDMG8rvNQy71Wle9dMrnEHJly6AqRpWIApDKZomt00pZJ2H5pu85jzXz88WnYTVR3Uj5Rapy/4M21W8vYYUms4GJtjP1z0qeLeXLryRjz7BjMPzQfX+79Ell5WWVeb8XpFdh8cTPmvDQHI5uPrNQ+w+bOlENXiCwZAyCVKTBwLOTyyaioa7SqM/5+OPID5h+ar/PY84HP458R/6CavYPJuplCQ2cjJGSGWXdrlcfe3h5eXl6wt9d/+zuxW+PCw8MRFRXFoFkBfVtq3Zzc8H7b9/FWy7fw1d6vMO/gvDIni9x7dA+j/hqFVcmrsPTlpQjwCDBa/U3ZXVyZWf3m2oJJZAoMgFQmY3aN7r22F+P/Hq/zWKBHINYOWgtXR9fH94ir9PWrypy7tSoKS0VFRcjKykJRUUW//MxHZGSkxU34qIgxQm1lW2pruNTAVy99hRGRIzB6w2jsvba3zGtvu7QNEYsisLj3YvR7pp9gdS6hayyeXD7ZaC3rljyrn8iUGACpXMboGs18lInYP2JRqCzUOubi4IJ1g9fB38O/ijUuJpPJEBcXh/j4eIubUFAWY4cltsYJwxg/p6q21Dau2Ri7XtuFn479hKnbppbZLZyRm4FXVr+CkZEjEd8tHh7OFe9Cog8xxuKZ86x+InPCAEgVErJrVKVSYczGMbiWdU3n8aUvL0XLgJaGVtlka9qJTcjlR6yxNc7YzH0mLADYSezwVsu30DusN97d/C5+O/NbmecukS3Brqu7sLL/Sjwb8KxB9xVrLJ6xh64QWQsGQNKLUF2jf577EytPr9R57OP2H2Ng+ECD72FLrHFNPUth6q7NElVtqfX38MfqAauxIWUD3t7wNtKyde+1KL8nR7tf2mHZy8sMej2KNRbPnGf1E5kTBkAyGaVKiY93fKzzWIe6HfBZ9GcGXd8Wd5jgLF5xiLnMiKEttb3CeuHUmFN4e+PbWJ28Wuc5jwofYdDvg3D29ll8EvVJlWYJizkWz1xn9ROZEy4ELRBrXAhaaGvOrEH/3/prlVevVh0nx5xEkGeQQdc35WK85ujJBYIteU09c2fOC4dXhkqlQuLJRIzbNA7Z+WUvjDsofBB+6fMLXBxdKtXlbaqF5MtjCV30ZFpcCLoUWwDJJJQqJabvnq7z2JyX5hgc/gC2hpFpWMsyIxKJBK82exXt6rTDq2tfLXOm8KrkVbh07xL++8KzyLv7I/Tt8jaHsXjmPKufSGwMgGQSf53/CydvntQqD/EOwfBmwwW5h9hr2omNs3hNw9qWGanrXRc7RuzAlK1TEP9vvM5zDqcfxssbD+PrpoBUY1eesru8ORaPyLyxz5KMTqVSldn6N63dNDjaO5q4RtapZGyYtY1zNDf6Lh9y+3Y1I9dEOA52DpjXbR6+7/k9HOx0twvcygPelQEXH2gfUyjmQqnM1yoPDZ0NqXQKgKcXJ7eHVDqFY/GIRCRqAMzOzoZSqdQqVyqVUCgUItSIjGHThU04dv2YVrlbgRsczzgiKSkJMplM0HuyNcx2yGQyREdHC/5/qCzFXZZl77aiUgE3bwJXrjQxSX2ENPrZ0dgybAuqV6uu8/i9AiBOBpzWWk6w6HHXuLbQ0Nlo3/4hQkPnISBgPEJD5z3+muGPSEyiBMC7d++iT58+qFOnDoKCgjB58mQ8fPhQffzOnTto1qyZGFUjgZXX+pezJQevvfoahg0bhri4OEHvy9Yw2/Hkmo+mUNK1WZ7vvgMsdYRNx5COODjqIMJ8wnQezykCppwETmZqlpfX5V0yFi8sbD6k0jh2+xKZAVHeoT755BPk5ORgy5YtuHPnDubMmYPevXvjjz/+gJeXF4Di4ECWb59iHw6lHdI+cB+IbRyL7nHdAXCCBlmWspYZKSiQ4MCBZ7BnzxmEhFjuEkRhPmE4+MZB9FrRC/sV+7WOP1IC004D30YC9dyLy7izBpFlEWUZmEaNGmHFihVo3rw5ACA/Px9vvvkmLl68iHXr1kGpVKJRo0bIyMgwddWqjMvA6Pbu3+/i20Pfah/YBCS+w+VKqGqeXvNR14xvUwSukmVG4uPn4H//S8epU4COUS0WuwRRTn4OXl7VB9sv/U/ncV8nYEFzoFY181/2hgjgMjBPEiUABgUFYefOnahfv766TKlU4u2338axY8ewePFidOrUiQHQwqlUKtT9b12tbd9c7VzxcPpDJC5lADQnlrR/srmt+WgugdQY8grzEJMQga2KCzqP13EF1vQcj5aN55u4ZkSVxwBYSpQu4NDQUMhkMo0AaGdnhx9++AHjx4/HgAEDxKgWPSbU4qnHrh/Tuedvx8COyG6TzW5fE9In3FnS/snmtuajNS9B5OzgjI2vnUHMsib4+9p5reNXHwLvHDyG7Q1y4eLoIkINiagqRGmyiomJwW+/aW9ILpFI8N1336FXr14cAygSuXwqdu92hVw+EenpCyCXT3z89dRKX+uPs3/oLH+j9RucoGFipp4oYWyRkZGIjY1FbGysOvSVBK6S0GXKmcHWzsHOAWtfPYHoOlE6j+9X7MfgNYNRqCw0cc2IqKpEaQF87733yj0eHx+P+Ph401SG1ITe33TtubVaZS4OLugS2qWqVSSBWev+yWK2ZlrrEkTODs74c/A6tF/aXuei7uvPr8fYjWPxQ68fqrR3MBGZlmWuU0CCUyrzH89oLJtCMRchITP06g4+d+cczt45q1XevUF3uDq66ngECU2fcBcXF6c1li4hIQEJCQkALGPyghCBS8jxjyVLEFkjr2pe+Dv2b7T+uTWuZl3VOr742GIEeATgs+jPTF85IqoUBkACIPz+pmvParf+AUDfRn0rXTeqGn3CnbmNpauKksAlk8mQlJQEoPKtmZY0/lFsAR4B2DJsC9osaYO7uXe1jn++63NE+EXglcaviFA7ItIXAyABEH5/0y3yLVplDnYO6NmgZ6XqRVWnT7izpskL1tCaaSka+jbEhqEb0HFZR+QW5modH7tpLKLqRsHX1VfQ+wo1QY2IbCwAyuVyjBkzBnfv3oWXlxcWLlyIRo0aaZyzZ88eDBw4EKGhpYuabtu2DS4u1j27Td9FXPU5T6VS4cTNE1rl7eu0R3WX0i2mLO3N3JKWSQGse2aqLpVtzbTW8Y+m8kLQC1g9YDVeXvkyilSavQe3cm4hbnMcEvslCna/4jHKmgtvy+WTIZVO4rZyRFUgagBcsWIF+vXrB2dnZ43y/Px8rFmzBkOGDBH0fnFxcRgxYgRiY2Oxbt06TJgwAdu2bdM6r2HDhjbXUhAYOBZy+WSU3w1s/3gf1PKlZach81GmVvmz/s+q/22Jb+bW3k1o6ZMXKht42WJouF5hvbCw50KM3jBa61jSqSQMCh+E3g17G3wfoSeoEZFIy8CUGDduHO7fv69V/uDBA4wbN07Qe92+fRsnTpzAoEGDABQvRXP16lVcvao9kNkW6bO/qVQ6Sa8WOl0zBAGgaa2mAJ58M386bBa/mVdlyRkqnz7hztb2T46Pj0diYiISExMxfPhwAMDw4cPVZba0EoFMJqvysjlvtngTMQ1jdB57e+PbOv8YrAx9J6gplfkG3YfI1ojaAqhSqbSWC1CpVDh06BCqV69exqOqJi0tDf7+/nBwKP6WJRIJgoKCkJqaijp16mice/HiRbRv3x729vaIjY3FqFGjBK2LuSprf1PAvlItc+UFQKFnGxubtXQTWvPMVF30Dby21EVeHkNatyUSCRb1XIRdV3YhKy9L41h6djomb52Mn2J+qnLdhJ6gRkTFRAmA1atXh0QigUQiQVhYmM5z4uLiBL+vrrD5tGbNmiE5ORleXl5IS0vDgAED4OPjg759bWP2amjobISEzDBobN6pW6e0yhzsHNDQt6HFvZkbo5vQ0sYSWiJbC7xiC/AIwLyu8zBy/UitYz8f/xmDwgfhpdCXqnRtoSeoEVExUQLgn3/+CZVKhb59+2Lp0qXw9vYurZCDA6RSKYKDgwW9Z2BgINLT01FYWAgHBweoVCqkpaUhKChI4zxPT0+Nx/Tv3x/79++3mQAIlHQHx1X58bpaAJ/xfQZO9k4W92ZujGVSrH0soSWy9PGPVSF06/Zrka9hZfJKbJVv1Tr25l9v4tSYU/Bw9qh0PYWcoEZEpUQJgFFRxdsJnThxAlKp1CSrxtesWRMRERFYtWoVYmNjsX79egQHB2t1/964cQN+fn6ws7NDdnY2tmzZgmHDhhm9ftYivygf5+6c0yqPqBUBQPg3c2O3prGb0DbYYouh0K3bEokEP/b6EU0WNcGD/Acax65mXcWM3TPw1UtfVbqeQk5QI6JSoo4BPH36NK5evYp27doBABYsWICkpCQ0aNAA33zzDWrWrCno/eLj4zF27FjMnTsXHh4eWLRoEQBgwoQJ6N69O3r06IH169djyZIlsLe3R1FREfr06cMAWAmKLIXO/UAj/IoDoNBv5pbSmmYtYwnJehijdbuOdx3M7jwbYzdpv37/++9/Mfa5sajjXUfHI8tWMkFN9yzgYvpOUCOiUpLMzEztgXAm8vzzz2PmzJno1KkTZDIZunfvjg8++AD//PMPfHx88PPPP4tVtUrz8PCAnZ2ok6rNwpH0I3hu8XNa5cteXobhzYpnWpa9pEMxqXSK3hNOkpKSMGzYMCQmJhq9Vc6Q1sbo6Git1pYncckREpOQryOlSomOyzpi11Xt/+9DI4YiqV9Sla6ra+moyk5QI1IqlcjOzha7GmZB1BbAa9euoUGDBgCAv/76C71798Y777yDTp06ISZG97ICZN7KWvLBu5q3+t+GzjYWqzXNkG5Ca9hyjUgfdhI7LOixAM2+bwalSqlx7NdTvyLu+Tg8F6j9R2KJsv7QEmKCGhGVEjUAuru7IzMzE8HBwfjnn38wZswYAICLiwsePXokZtWoiu7l3tNZXr2a5rI+hryZW+ICvhxLSOZM6EkwTfya4I3mb2DxscVaxyZvm4ydI3aWOfa7vGEdhk5QI6JSogbA7t2745133kHTpk0hl8vVrSGnTp1C3bp1xawaVZE+LYAlqvpmztY0ImEZYxLM59Gf49dTvyKnIEejfPfV3Vh3fh1ebvSyoPcjosoRNQDOmTMH33//PdLS0vDnn3+ql4NJS0vDW2+9JWbVqIruPSqjBdBFuIW9Lb01zRaXHCHb4+/hj6ltpuLTnZ9qHZu6bSp6NugJR3tHAJwkRSQGUSeBWBNOAik27X/TMGvvLK3y7A+y4e7kLvj9TDkJhIgqJyc/Bw3mN8D1B9e1js3vPh/jW40HwElSZDqcBFJK9MTy559/om/fvmjRogUUCgUAYMmSJdi+fbvINaOq0NUFbC+xh5ujm1Hux9Y0IvPl5uSGGR1n6Dz21b6vUFBUAID7MhOJQdQA+NNPP2Hq1Klo164drl+/jqKi4hmh1apV4wveQhUptdf3c7R3NNpi3yVjl9g9ZB5kMhmio6Mhk8nErgqZiRHNRqBpraZa5an3U7Hy9EoAxa/j2NhYxMbGqsfylgzriI2N5eubyAhEDYA//PADFixYgEmTJsHe3l5d3qJFC/V4ELIszg7OWmV5hXk6910m6/PkDE4iALC3s8cXHb/QeezbQ9+auDZEVELUAJiamopGjRpplUskEuTl5YlQIzJUNYdqWmUqqHTuDkJEtqFHgx4Ir6k9TONI+hEcv35co4zDOohMQ9RZwI0aNcKePXu0Bu//9ttvaNasmUi1IkPoCoAA8KjwkXrGH1kXzuCkithJ7BD3Qhze/OtNrWOLjy3Gwp4L1V/b4r7MRGIQZRbwuHHj8OWXX+Lo0aMYPnw4XnvtNfz4448YP348Lly4gC1btmDt2rV48cUXTV21KuMs4GJf7P4CH+34SKv81uRbqOkm7N7OZB44g5P08SD/Afy/8ceD/Aca5R5OHrj+3nW4ORlnohjRkzgLuJQoiWXFihV49OgRoqOj8b///Q8ZGRlo3Lgx/vrrLzg4OGDz5s0WFf6oVFktgHlF7NK3VpzBSfpwd3JHbIT2Uk3Z+dlYlbxKhBpVnlKZD4UiHikpE6BQxEOpzBe7SkRVJkoX8JMTAho0aIAFCxaIUQ0ygvK6gMk6WfrC3GQ6b7V8Cz8c/UGrfPGxxRjZfKQINdKfXD5Va/9yuXyyXvuXE5kj0fos09LSoFAoyv0gy1NWAHy624eqhsuskCVr4d8CLf1bapUfTD2Ia1nXRKiRforD3xw8Gf6KFUGhmAO5fKoY1SIyiGiTQDp27FjmMZVKBYlEgoyMDBPWiITg5+anszw9Ox2RtSNNWxkr9OQyK+Y4sYIzOKkibzR/A0evH9Uq//Pcn3jn+XdEqFH5irt955Z7jkIxFyEhM2Bn52SiWhEZTrQAuHXrVvj4+Ih1ezKSIM8gneWp91NNXBMSA2dwUkX6PtMX4zaNgwqa8w/XnltrlgEwLW0htFv+nlaEtLSFkErjTFAjImGIEgAlEgnq1KmDmjU5K9TaBHoG6iyfOX8mWsW1MstWK3PHZVbImtR2r43W0tbYp9inUb776m7czrltdqsF5ObKBT2PyFyIPgmErIuvqy+c7J2QX6Q5O+7qvatm221p7uLi4rSWWUlISEBCQgIALrNClqffM/20AqBSpcT68+vxRos3RKqVbi4uoYKeR2QuRAmAQ4YMQbVquicLUOUolflIS1uI3Fw5XFxCERg4VtRxKHYSOwR4BOBK5hXNA56iVMcqxMfHa7QAJiQkYPjw4eo9UznejixN30Z98d7W97TK155ba3YBMDBwLOTyySi/G9gegYFjTVUlIkGIEgAXLlxY8UlUIXNdliDIM0hnALSWbkuZTIa4uDjEx8eb5HvgMitkbUKqh6B57eY4fkNzG7htl7bhft59eDqbz1+MdnZOkEonPZ4FrJtUOokTQMjicOsKC2XOyxIEeugYB+hZHFyGDRuGYcOGIS4uzuT1EsqTM3GJqGr6NuqrVZZflI8tF7eIUJvyhYbOhlQ6BYD9U0fsIZVO4TqAZJFE3QuYqkaoZQmM1X0c7BWsXVgNGPD6APTp1AcAuy2risuskLXo90w/fLLzE63yHVd2YED4ABFqVL7Q0NkICZlhVkNuiAzBAGiBhFiWwJjdx+E1dYeT0BdDLbbb0lxm4nKZFbIWjWs2RoBHANKz0zXKd17ZKU6F9FDcHRwndjWIBGEWAfDcuXO4cOECgOKt4Ro1aiRyjcybocsSlHYfP61IXW5ICGzi10RnuSLfcnd34UxcImFJJBJ0qNsBSaeSNMrP3jmLGw9uoLZ7bZFqRmQbRA2AaWlpePvtt7F37154e3sDALKystCmTRssWrQIQUG6FxW2dYYsS2CKVe0b12wMCSRaC70+cLXc7eA4E5dIeLoCIFDcCji4yWCDr29uqyQQmRNRJ4GMGTMGRUVFOHbsGC5fvozLly/j6NGjUKlUGDuWU+rLUrzcwNODkZ+me1mCynQfV5WLowvq16ivVX696HqVrym2yMhIxMbGIjY2Vh36SmbixsbGWuyMZiIxRdeN1lkuRDewXD4Vu3e7Qi6fiPT0BZDLJz7+mvv2EgEiB8BDhw5hzpw5CAkJUZeFhIRg1qxZOHTokIg1M28lyxKUp6xlCUy1qn1ErQitsuRbyVCqlAZdl4hMRyaTITo6GjKZzCjXr1e9HqSeUq3yHVd2GHRdc14lgchciBoAGzRogDt37miVZ2RkIDSUq6qXp6rLEphqVfsmNbXHAeYU5GivD2iBOBOXbIWxlzySSCQ6WwFT7qZoTQ7Rl77DXJTK/HLPIbJ2ogbA9957D++99x6WLVuGkydP4tSpU1i2bBkmT56MyZMn48qVK+oP0hYaOhvt2z9EaOg8BASMR2jovMdflz2Bw5Du48ooayLI0fSjBl3XHJTMxGW3L5HhOtTtoLP8gOJAla5nimEuRNZA1Ekgr7/+OgDoXBR45MiRkEgkUKlUkEgkyMjIMHHtLENllyUw1ar2Lfxb6Czfp9hnlmt8EVExUy951Da4rc7y4zeO45XGr1T6eqYa5kJk6UQNgCdOnBDz9jarpIXw6XUAi7uPhdlGrl71evBz88OtnFsa5U9vAE9E5sXUSx6F1giFh5MHsvOzNcqf3iZOX6Ya5kJk6SSZmZmqik+jinh4eMDOzrJ21jP2Egn9VvXD2nNrNcrsJfbIej8Lbk5ugt2HiITzdAugriWPhB7+0O6Xdth7ba9Gmb+7P9Lfq/w4QKUyH7t3u6L8bmB7tG//kEvC2CClUons7OyKT7QBJm8BXL58OQYOHAhnZ2csX7683HNfffVVE9XKNhl7Vfs20jZaAbBIVYRDaYfQIUT3uB8iEldkZKRGwEtISFAveWQszWs31wqA1x9cx80HN1HLvValrmWqYS5Els7kAXDOnDno2bMnnJ2dMWdO2S9QiUTCAGjhWktb6yzfp9gnegCUyWSIi4tDfHw8J3MQiax57eY6y2U3ZOhav2ulr2eKYS5Els7kAfDkyZM6/03Wp4V/CzjbOyOvKE+j3NBxgEKEtyeXt2AAJNLNVEseNffXHQCP3zhepQAIFIfAkJAZ3AmEqAxmsRcwWSdnB2c8F/icVtfOfsV+FCoL4WBXtf9+DG9EplGy5JGxNa7ZGI52jihQFmiUV3UiSAljD3MhsmSiBsCCggIsXboU+/btw+3bt6FUau4S8ffff4tUMxJKG2kbrQB4P+8+DqUdKrOL2FhMvbwFEenHyd4JjWs2xombmitDXLh7QaQaEVk/UQPgxIkTsWnTJvTp0wcNGzaERCIRszpkBJ3rdcZX+77SKt98cXOlAqAQ4c3Uy1sQkf7CfMK0AqD8nly9FiwRCUvUZWDq1KmDpKQktG2reyFQS2KJy8CYwqPCR/CZ7YOHBQ81yp8LeA6H3tR/v+fo6Git8PYkfcKbGMtbEJF+3t/+vs4/Fu9OvYsaLjVEqBFZIy4DU0rUFkBvb2/4+vqKWQUysmoO1dChbgdsvLBRo/xI+hHczrmNmm419bpOfHx8heGtImIsb0FE+qlXvZ7O8kv3LjEAEhmBqE1W06dPx+eff467d++KWQ0ysm71u2mVqaDCtkvb9L5GZGQkYmNjERsbqw59JeEtNjbWZlruZDIZoqOjIZPJxK4KkaDKCoDyDG7ZRmQMJm8BDA8P1xjPcffuXYSFhcHX1xeOjo4a554+fdrU1SMj0BUAgeJxgEMjhpq4NsVMtbyF0DgDmqxVeS2ARCQ8kwfADz/80NS3JJHVr1EfodVDIb+n+Zf8FvkWKFVK2Ekq1xAtRHgz1fIWRLZAiLU5pZ5S2EvsUaTS3MKNAZDIOEweAIcOFafFh8TVrX43fHf4O42yWzm38G/qv3hR+mKlrmVr4Y3L15C5E6Jl2tHeEcFewbiceVmj/ErWFcMrSERaRJ0EsmXLFjg6OqJjx44a5f/88w+Kiorw0ksviVQzElr3+t21AiAA/Hbmt0oHQFvD5WvIVgR6BmoFwNs5t0WqDZF1EzUAfvrpp/jiiy+0yiUSCT755BMGQCvSuV5neDp74n7efY3y38/8jm+6fMN1vsohxAxoIqEZo2Xa11V7VYi7uZwkSGQMogbAK1euIDQ0VKu8Xr16uHLliukrREbj7OCMmIYxSDyZqFGuuK/A4fTDaBXYSqSamT8uX0PmyBgt0z4uPlpldx7e4WLQREYgagD09fVFcnIy6tatq1F+8uRJVK9eXZxKkdH0f6a/VgAEgN+Sf2MAJLIwxmiZ1tUC+KjwER4WPISbk5thFSYiDaIGwCFDhmDy5MlQKpXq3UD27NmD999/n5NFrFDX+l3h7uSOB/kPNMp/P/s7Zr80m3/h68FSl68h62OMlmldARAo7gZmACQSlqgB8P3334dSqcSbb76J/Px8AICzszPGjRuHDz74QMyqkRFUc6iG3mG9seL0Co3yK5lXcOz6MbQMaClSzSyHrc2AJtuiqwsYKO4GDvYKNnFtiKybqAHQ3t4eH3/8MaZMmYJLly5BpVIhNDQU1apVE7NaZET9G/fXCoAAsPL0SgZAIgslVMt0WS2Adx7eMei6lkypzEda2kLk5srh4hKKwMCxsLNzErtaZAVEDYAlCgoKUFhYqP43A6D16l6/O9wc3ZBTkKNRnngqEbM6z4KDnVn8lySiShCqZbq6i+6x31mPsgy+tiWSy6dCoZgLoOiJssmQSichNHS2eBUjqyDqXsC5ubmYOHEi6tWrh6ioKERFRSE0NBSTJk1Cbm6umFUjI3FxdEGfRn20ym88uIGt8q06HkFEtsLZ3llneYGywMQ1qTylMh8KRTxSUiZAoYiHUplv0PWKw98cPBn+ihVBoZgDuXyqQdcnEjUATp06Fbt378bKlStx9epVXLt2Db/++it2796N999/X8yqkRG91uw1neWLjy02bUWIyKw42jvqLC8oMu8AKJdPxe7drpDLJyI9fQHk8omPv65aSCsOk3PLPUehmGtwyCTbJmoA/Ouvv7Bw4UJ06tQJnp6e8PDwQOfOnbFgwQKsW7dOzKqREXUM6YggzyCt8vXn1+Na1jURakRE5sDJXvfYtvwi8w06xmipS0tbqON6Tyt6fB5R1YgaAAsLC+Hi4qJVXq1aNRQVVfSfnyyVvZ09Xo98XatcqVJi0eFFItSIiMyBpQVAY7XU5ebKBT2PSBdRA2DHjh3x3nvv4cKFC+qylJQUTJ06VWt/YLIuo1uO1jnhY/GxxXhU+EiEGhGR2BztyugCNtMxgMZqqXNx0d4hy5DziHQRNQB+8803cHd3R6tWrRAcHIzg4GC88MIL8PDwwDfffCNm1cjIAj0D0e+Zflrld3PvYsUp7WViiMj6WVoLoLFa6gIDxwKwr+As+8fnEVWNqGtuODk5YfXq1bhy5QouXLgAlUqFsLAwNGjQQMxqkQ7GWItq/HPjsTp5tVb5Nwe+wYjIEbCTiPr3CRGZWFmTQMw1ABqrpc7OzglS6aTHYwt1k0oncT1AMohoAbCwsBD169fH/v370aBBA4Y+M2astajaBrdFs1rNcOLmCY3y5NvJ2JiyEb0b9q7ytYnI8hQpdXenmuv6oIGBYyGXT0b53cBVa6kreW99+r0XsOc6gCQI0ZpYHBwcEBoaiszMTLGqQHow5lpUEokE7734ns5js/bOgkqlqvK1DSWTyRAdHQ2ZTCZaHYhsTVnjf6s5mOfmACUtdeUxpKUuNHQ22rd/iNDQeQgIGI/Q0HmPv2b4I8OJ2sc2Y8YMfPTRRzhw4AAePHgApVKp8UHiMsVaVIObDNa5x+eB1APYe21vla9rqOTkZOzatQvJycmCXI+BkqhilhYAgeKQJpVOgfaYPXtIpVMMDmvFITMOYWHzIZXGsduXBCNqu/qAAQMAAD179tR5PCMjw5TVoadUZoabVBpXpXs42jti8ouT8c7md7SOfbnvS7Sr065K1zU3TwbKyMhIsatDZJYsMQACxSEwJGQG9+wliyJqAPzrr7/EvD1VwFRrUb3R4g1M3z1da8P3TRc24eTNk2haq6lB19eXTCZTt/ht3bpV4zNQvOE9wxuR8VhSANQ1Ma6qfwgTiUHUANi2bVsxb08VMNVaVK6Ornin1Tv4ZOcnWse+2vcVkvolGXR9fcXFxWHXrl0aZQkJCUhISAAAREVFVWrDewZKMgWZTIa4uDjEx8db/P8nSwmAxpoYR2RKogbA5cuXw9PTE3369NEoX7duHR48eIDY2FhB7yeXyzFmzBjcvXsXXl5eWLhwIRo1aqR1XkJCAuLj46FUKhEVFYVvvvkGDg7mOQvNmIw5w61EyS+vz2d/DjdHN+QU5GgcX3l6JT6N+hRhPmFVvoe+4uPjNQJbQkIChg8fji5dugAoDmyVIXSgJNLFmoYXWEIALJ0Y97QidTlDIFkCUSeBzJ07Fz4+Plrlfn5+RlkIOi4uDiNGjMDRo0fx7rvvYsKECVrnXLlyBTNnzsTmzZtx/Phx3Lx5E8uXLxe8LpbA2DPcgNJfXqkXUjG65Wit40qVEp/v+rzK16+MyMhIxMbGIjY2Vh36unTpoi6r7C/X+Ph4JCYmIjExEcOHDwcADB8+XF0WHx8v8HdAZNmy87N1lrs4aG8ZKgZTTIwjMhVRm7XS09MRFBSkVe7v74/09HRB73X79m2cOHECa9euBQDExMRgypQpuHr1KurUqaM+b/369ejVqxf8/PwAACNHjsR///tfvP669t61tsCUa1FNfHEiFhxeoLXo64pTK/BB2w/QxK+JYPcyhcjISI3QmJCQoA6URIaw1uEFt3Nu6yyv6VbTxDXRzRQT44hMRdQAGBgYiAMHDqBu3boa5fv370ft2rUFvVdaWhr8/f3VXbkSiQRBQUFITU3VCIAKhQJSqVT9dXBwMFJTUwWti6UReoZbeb+8otyjsC1rm8b5Kqjw2c7P8PvA3w34LionPDwcUVFRle72NRZrGudFhrPW4QW3cm7pLK/pah4B0FQT44hMQdQA+NZbb+E///kPMjIy0KZNGwDA3r17MXv2bHzwwQeC308ikWh8XdZCw0+eJ+ZixOakZC0qIZT7y8sdsJtoB6W95jqQa86uwbHrx9DCv4UgdahIZGSkoL9ADQ2U1jTOiwwn9HhVQwm1VeTth9otgHYSO9RwqSFENQ1mqolxRKYgagB8++234eLigq+//hofffQRACAoKAgzZsxQj5kSSmBgINLT01FYWAgHBweoVCqkpaVpdUFLpVJcu3ZN/bVCodDZTU1VV9Evr63YioSLCVqP+2THJ9gwdINJ6yoUoQMl2TZzGl4g5IxYXS2APi4+sLd7epFlcZhiYhyRqYg+tXXEiBEYMWIEHjx4AJVKBQ8PD6Pcp2bNmoiIiMCqVasQGxuL9evXIzg4WKP7FygeG9itWzdMnToVNWvWxJIlS/DKK68YpU62qqJfXl1yuuCPb//Ag/wHGo/beGEjDigO4EXpi6asrmisdZwXWQ+hZ8TqagH0c/OravUEVzIxTvf3XMzQiXFEpiJ6ACzh7u5u9HvEx8dj7NixmDt3Ljw8PLBo0SIAwIQJE9C9e3f06NEDdevWxQcffICuXbtCqVSiffv2ePXVV41eNypV060m3n3+XXyx5wutYx/t+AjbX92u1Z1vjax1nBcJS6zxqvrOiA0JmaF3INLVAmguE0BKmHJiHJExSTIzM006yK1JkybYvXs3atSogfDw8HJ/kZ8+fdqENTOMh4cH7OxEXVXHIpU1ueFe7j2E/DcEWXlZWo/5a8hf6BXWy4S1FEZlx0k93QKoa5wXWwBJLApFPOTyiRWeFxo6T+/xwzXn1NTaEWhA4wFYPWB1VapoVEKNeyTTUiqVyM7WvdyQrTF5C+C0adPg5uYGAPjwww9NfXsSka43zLLGxlV3qY7JrSfj4x0fax17d/O76BjSEa6OriaotTCqMk7KnMZ5ET1N6Bmx2XnZWuEPAAI8AipVL1MRcmIckRhMHgCHDh2q899k3aoSgN59/l0sOLQAN3NuapRfuncJM3bPwMxOM41ZZcFw5wCyRkLPiL2ceVlneWh1zqglMgZR+iwVCoVeH2QdSgPQ0zPnigOQXD5V5+M8nD3wfx3+T+exOfvn4NTNU8JW1AiE2jnA3NYlJCqe6VrR7Fz9Z8TKM3S3FNarXq9yFSMivYgyCaRZs2bqfz+5zl7JeECVSgWJRIKMjAyT183WCT2uxdCB4m+0eAO/yH7BgdQDGuWFykKM3jAae0fuhZ3EfMdeCrVzAJeRIXMj9IxY+T3dATC0hvFaADmOj2yZKAHQ1dUV3t7eGDRoEPr06WO0pV+ocoRcz6uEoQHITmKHH3r9gBY/tkChslDj2IHUA/jx6I94+9m3q1Q3U+DOAWTNhJwRq6sFUAIJQrxDDKxlGfczwvsdkSURpekkJSUFH3/8MY4dO4aePXti5syZuHz5MurWrYuQkBD1B5lOVbtpKyJEAIqoFYHJL07Weez97e/jevb1KtXNFLhzAFm70NDZaN/+IUJD5yEgYDxCQ+c9/rpyIepS5iWtsiDPIDg7OAtVVTVjvd8RWRJRAqCrqysGDRqEtWvX4t9//0V4eDg++ugjPPPMM/j4449RUFAgRrVsllDj1HQRKgB9HPWxzpaArLwsjNk4xmy37BN6nBSROSqZERsWNh9SaVyVulF1tQAaY/yfMd/viCyJ6IOnAgICEBcXh++++w716tXDd999h5ycHLGrZVMq001bWUIFIFdHVyzquUjnsXXn1yHpVFKl62YKJeOkysOdA8jW5Rbk6pwFbIwZwMZ8vyOyJKIGwLS0NMybNw/PP/88hg4dihYtWmDPnj3w9vYWs1o2x5jj1IQMQF3rd8WQJkN0Hpvw9wSk3U+rdP1MITR0NqTSKdAOwvaQSqdwvBHZvNO3TkOpUmqVN/FrIvi9OC6XqJgok0CSkpKwatUqHD9+HN27d8fMmTPRoUMH7qQhEmOPUxNyoPh/u/0X2y9t19ozNPNRJt7a8BY2DNlgltvEhYbORkjIDM44JNJBdkOmszyydqTg9+K4XKJiJt8KDgCqV6+OwMBA9O7du9w9gC1ppxBL3gpOqczH7t2uKL9bxB7t2z80eEkYIQLQH2f/wCurX9F57OeYnzGy+UjB70lExjNu4zgsPKLd5Xp36l3UcKkh6L1M9X5H5olbwZUSpQWwdevWkEgkOHWq7IV8zbEVx1oJvZ5X+feJM+gaANDvmX4YGjEUv576VetY3OY4dK7XGcFewVzmgchCyG7KtMqknlLBwx9guvc7InMnSgugNbLkFsASugJTVbppTSEjNwPhC8Nx48ENrWOd63XGohcjkZr6dZmP59g7IuOQyWSIi4tDfHy8xl7WZVGqlPCc5YmcAs3Jf73DemP9kPVGqqVlvd+RcNgCWEqUFkAyT5Y0Tq2GSw382OtHxKyM0Tq2/dJ2zLf7H/oGlv348nYfIaKqS05Oxq5du5CcnKxXAJRnyLXCH2Cc8X9PsqT3OyJjYAAkDUJ105pC74a9MaLZCCw7sUzr2CK5Cs28gHplDjGtePs1IjK+Y9eP6Sw3dgAELOv9jkhoDIBk0eK7xWP7pe1Iy9ZcAqZABXx6BljUAnAv4385l3kwH5XtNiTzIpPJkJycDADYunWrxmcACA8PL/PnuufaHp3lzWs3F7aSRKSBAZAsmnc1b/wc8zO6JXXTOpaaC8w4C3zRBLDXMaeIyzyYj8p2G5J5iYuLw65duzTKEhISkJCQAACIiorCzp07dT5WVwD0d/dHXe+6QleTiJ7AAEgWr2v9rpj4wkTMOzhP69i/GcDSK8AbWrvIcfs1IqHEx8drtAAmJCRg+PDh6NKlC4DiFkBdMh9l4tRN7dUg2tVpx5UgiIyMAZCswqxOs7BPsQ+H0g5pHUu8BtR3B6JqlpZxmQfxGdJtSOYlMjJS42eVkJCALl26IDY2ttzH7bu2DypoL0TRLrid0FUkoqdY9rolRI85Ozjjj4F/oJZbLZ3HvzwHXHoAcPu1ypPJZIiOjoZMJhP0unFxcRg2bBiGDRum7ipMSEhQl8XFxQl6P1thrJ+XMZQ1/q8kAFrS90JkaRgAyWoEegZizcA1cLRz1Dr2SAl8nuKD8GcVJg9/lv5L7MnxeUKKj49HYmIiEhMTMXz4cADA8OHD1WXx8fGC3s9WGOvnpa/w8HBERUWV2e37JF0B0MvZS70HsNjfC5E1YxcwWZU2wW2woMcCjN4wWuvYtey76P/7YGwdthXODs4mqxMnOOhW1W5DMm+RkZFlTvh4Um5BLg6nHdYqbxvcFvZ29kaoWeVwG0mydgyAZDBze6N8q+VbOHb9GH44+oPWsd1Xd+PNv97EspeXcZB5OTg+z7JY4s9r55WdKFAWaJWHOoYiKSkJgHjfC7eRJFvAreAEYg1bwVWFuW6nlF+Uj47LOmKfYp/O49Ojp+PjqI+Ndv+nfyHrmhVZ3i8xsdfFi46O1lrW40nlLetRFWJ/v5bO1D8vIYzfNB7fHf5Oq7z50eY4/tfxMh9n7O+l+D2tvH2COYbYknEruFIMgAKxxQBo7m+UNx/cRKufWuFa1jWdx5P6JWFoxFCj3NvQX8hJSUkYNmwYEhMTRekSNTTAkmlZ2s9LpVKh3rf1cCXzikZ5Tdea2Nx5M86eOQvA9N+LUpmP3btdofkH7dPs0b79Q3YHWygGwFLsAqYqUSrzH7f8lU3s/XZrudfCxqEb0WZJG9zPu691/PV1ryPYKxhtg9sKfu+qrotmLjg+Tz/m0nJpaT+vc3fOaYU/AOjeoDtaNG+BFs1bqMtM+b2kpS1E+eEP4DaSZC0YAKlKLOWNsolfE/w24Df0SOqBIpVmffOL8vHyypdxcNRB1K9RX9D7VuUXsiWO47J1nOBTNRsvbNRZ3rNBTxPXRJO+20NyG0myBrbVZ0mCsaQ3yi6hXfBdD+2xRgBwN/cuuizvgrT7aTqPm5K5rotXmWU9SHxi/rz0XfJIVwC0l9ijS2gXjTJTfy/6bg/JbSTJGrAFkKrE0t4oRz87GhcyLuCbA99oHbuceRkdEzpix4gdCPAIEPze+v4SM9duY32X9bAV5t5SK+bPS58W0axHWdh7ba9WeZvgNvCu5q1RZurvJTBwLOTyyahoDCC3kSRrwABIVVrGxRLfKGe/NBuX7l3C2nNrtY6l3E1B9NJo7BixA4GegQCEW95G319iljaOy1bFxcVpTfBJSEhQt9qa44xbc7LpwiYUKgu1ysXu/gUAOzsnSKWTKpjcxm0kyTowANq4qq53ZYlvlHYSOyT2S0SHZR107hl8IeMCopcVh8C8299yHTDSyVxbasVS2RbRFadX6LyOOQRAAOrXtzkub0UkJC4DIxBLXAZGiGVczHUdwPJk5GYgemk0Tt06pfN4sLs35oRnwq+a7sebYnkbc5ldSuUTe7kec1CZJY8ycjNQ++vaWgtAN/RpiLPjzprV4uzmtsA9CYPLwJRiC6CNEmoZl9DQ2QgJmWFRb5Q1XGpg+/Dt6LisI5Jva+8xeu1BJiaeAOY2A2rpCIGmWN5G6LFPDJRkLJVpEf3j7B86d/8Y0mSIWYU/oKSXI07sahAZDQOgjRJyGRdLfKP0c/PDjhE70Cmhk86WwPRHUIfA2lohUPzlbSqLy5UYB2dIV27salndv0MihhirekRUBsvqsyTBWNIyLsZS060m/jf8f2haq6nO49cfh8Abj7SPWfPzQvoraallqK7Y9ezr2HF5h1Z5S/+WCPMJE6FGRLaNLYA2ytKWcdFHZcfslAxeH+M6BtPypuGe8z2tc248AuJkxS2BAS6l5ZbwvJj7ciVkfcprEV2dvBoqaA85H9KErX9EYuAkEIFY2iQQa9vzsiqTUTQGr7sAGA7AX/f1fZyA2RFAPffi61rC82LofsREQilSFiF8YTjO3z2vUS6BBNcmXkOQZ5BINSNbw0kgpdgCaKMscRmXspQ9m7lIXa4rBOoavF4jrgYynDO0zr2bD0yQAZ88A/SPtIznhcuVkLn47cxvWuEPANrVacfwRyQSBkAbZg3rXRkym1nX4PWZjWZi8aPFOHr9qNZ1HhYB05IlcA2SYoL59wBzYWkyC0XKIkzfNV3nsdEtR5u4NkRUwnL6LMkoQkNno337hwgNnYeAgPEIDZ33+GvzD39A5WYz68Pd3h3bXt2GVoGtdB5XqlR4Z/M7GL9pvM7dDIhI029nfsPZO2e1yuvXqI+B4QNFqBERAWwBJFjmMi4lhJrN/OTg9eou1bH91e14edXL+OfyPzrP/+7wd7iQcQGr+6+GVzWvStfb1LhcCYmhvNa/j9t/DAc7/goiEgsngQjE0iaBWAuFIh5y+cQKzwsNnVfpkJtXmIfRG0Zj2YllZZ7zjO8z2DB0A+pVr1epaxPZgpWnV2LIGu1ZvvVr1MfZcWcZAMnkOAmkFBMLWbTAwLEA7Cs4y/7xeZXj7OCMX/r8glmdZpV5ztk7Z/H8T89j77W9lb4+kTVj6x+ReWMAJItWMpu5PIbMZpZIJHi/7ftYM3ANXBxcdJ5z5+EddFzWEYuPLq7SPYis0erk1WWO/RsaMVSEGhHRkxgAyeKFhs6GVDoF2i2B9pBKpwgyoaXfM/2w5/U98HfXvVBggbIAb214C6P/Go28wjyD70dkyR4VPsK0f6bpPMbWPyLzwDGAAuEYQPFVdieQqki9n4qYFTE4fuN4mee09G+JX1/5VfTtrUzxfBDpMmvPLJ0B0BzG/vF1Yds4BrAUA6BAGABtx4P8Bxj2xzCsO7+uzHPcHN0wv/t8vBb5GiQSiQlrV6wqO6MQCSE9Ox1h88OQU5CjdWx53+UY1nSYCLUqxtcFMQCWYmIhqiR3J3f8MegPfND2gzLPySnIwcj1IzFkzRBkPso0XeXw5M4oT6+PWLwzilw+1aT1Idsy7X/TdIa/VoGtRB37x9cFkSa2AAqELYC26fczv+O1P1/T+QuvRB2vOkjql4Q2wW2MXh9r2+OZLMvhtMNo9ZPuRdQPvHEALwS9YOIaFePrgkqwBbAUEwuRAfo37o9/R/2LxjUbl3nO1ayraL+0PT7f+bnRdw8RemcUIn2pVCrEbYnTeSw2Ila08AfwdUGkCwMgkYHC/cJx+M3DeLvl22Weo1Qp8dmuzxC9NBqX7l0yWl303Rnl0qWDiI6OhkwmM1pdyLasPL0S+xX7tcpdHV3xZecvRahRKaF2DCKyJgyARI8plflQKOKRkjIBCkU8lMp8vR/r6uiKRb0WYe2gtajhUqPM8/Yp9qHpoqb4/sj3UKmEH33h4hKq13k3bjhi165dSE5OFrwOT5PJZAybVi7rURambJui89j7bd5HkGeQiWukSd/Xhb7nEVkDBkAyOUOClrHI5VOxe7cr5PKJSE9fALl84uOvKzcw/OVGL+Pk2yfRoW6HMs/JKcjBmI1j0DGhI87e1l4o1xD67oxSWNhZ0PuWJzk52WRh0xCWGlTN4fU0ZdsUpGWnaZUHewVjcuvJJq/P04y5YxCRpWIAJJMSKmgJXSchZwcGegZi26vbMKvTrHLXO9t5ZSeafd8M0/43DQ8LHla+4jqUtzOKSlX8cetWN2zd+g8AYOvWrUhKSkJSUpLFBR+hWUpQfZI5vJ62XNyCxcd074Izu/NsuDjq3kHHlIy9YxCRJeJy7GQypUHraUXqclOvxVXcejK33HMUirkICZlRqV8O9nb2eL/t++gY0hFD1wyF/J7usUUFygLM2jsLv576FfO7z0fvhr0rVX9dSp7Dp9c7y8kBZs8G9uzZqC5LSEhAQkICACAqKgo7d+40+P5AcWtaSZDaunWrxmcACA8PR2RkpCD3slXm8HpKuZuCwWsG6zzWuV5nDAwfaNT7V0ZZrwuuA0i2isvACITLwJTPXJdhUCjiIZdPrPC80NB5kErjqnSP7LxsTNk2BT8c/aHCc/s07INvu3+LYK/gKt3rSU/veHD7dmucPXsBQHEYS0hIwPDhw9GlSxcAwoay6Oho7Nq1q8zjQoZNQzwdVI35nAjJHF5P93Lv4YWfX0DK3RStY26Objg99jTqetc1yr0NwZ1AbBuXgSnFACgQBsDymSJoVUVKygSkpy+o8LyAgPEIC5tv0L22XNyCUX+NQur91HLPc3V0xadRn2LiCxPhaO9o0D3LkpSUhGHDhiExMRGxsbGCX99SgpWlBNWnif16KlQWontSd2y/tF3n8QXdF2Bcq3GC35fIUAyApdgFTCZhrsswmHJ2YNf6XXF23Fl8vvNzzDs4D0Uq3a03Dwse4j/b/4NlJ5ZhUc9FaF+nvcH3frrVA6hu8DXLExkZqRHwEhIS0KVLF6OETUPEx8dXGFTNkdivp4mbJ5YZ/gY3GYyxz3EyBZG5YwAkkzDXZRgCA8dCLp+MirrShJod6O7kjjld5mB4s+EYu2ks9l7bW+a5Z26fQdTSKPRt1BefR3+OiFoRVbqnrv1P/f3t8M47QWYbcEzFUoLq08R8Pc3/dz4WHNbdav5cwHNYErNElP2viahy2GdJJmGuyzCINTswolYEdr22C0tilsDHxafcc9eeW4tm3zfD0DVDdY63Kk9ZM5zt7JTo2zcVHh6/VrbqlRYeHo6oqCibD5tCEuv1tEy2DO9sfkd3nTwC8efgP81i1i8RVYwBkEzCnJdhCA2dDal0CrR/odpDKp1itNmBdhI7vN78dZwffx5vtniz3HNVUGHF6RV45rtnMHLdSFzJvFLh9fWd4WzsdeMiIyOxc+dOsxjzVx5LCqpivJ7WnFmDketH6jzm4uCCdYPXIcAjQLD7EZFxcRKIQDgJRD+6uiPNZRkGsWcHHlAcwJiNY3Di5okKz3W0c8SoFqPwYbsPEegZqPMcsScKkPGZ6vW0+eJmxKyIQYGyQOfx1f1XY0D4AMHuR2QsnARSigFQIAyA+hM7aJmzQmUhFhxagE93for7efcrPN/Z3hljnh2D99u+j1rutTSOmXKGM4nH2K+n3Vd3o1tiN+QW5uo8/vVLX+O91u8Jdj8iY2IALMUAKBAGQBLSnYd3MHvfbCw4tKDMX7xPcnV0xZhnxyDuhTj1vqtsASRDHUk/go7LOiI7X/cvzE+jPsVn0Z+ZtlJEBmAALMUAKBAGQDKG69nXMWvvLPxw9AfkF1U8Vs/RzhGxTWMxpfUUNPKpL/piwWS5jqQfQdfErsjIzdB5fOILE/FNl28445csCgNgKZtILA8fPsQbb7yB5s2bo2XLlli/fn2Z53p7e6N169Zo27Yt2rZti/3795uwpkSa/D388W33b3FhwgW82eJN2EvKn/lZoCzAUtlShC8MR++VfXHFvi9U5fyJx/1PSZflJ5aj3S/tygx/o5qPYvgjsnA20QL41Vdf4cqVK1i0aBGuXLmCLl264NChQ/D29tY619vbG6mpqXB3d6/UPdgCSKZwMeMipu+ajsSTiVBBv5duqGcN9PK7h661VXBR50fzmHhD5uV+3n2M3TgWSaeSyjxncJPBSOybCHu7ipahITI/bAEsZROJZe3atXjzzeJlNurWrYvWrVtj48aNIteKqPLq16iPhL4JOD32NAY01m/Wpfx+Bv57UYXBh6ph2Y1mkPh+gPbtH1pF+JPJZIiOjoZMJhO7KhbvYOpBRH4fWW746x3WGwkvJ1hU+CteDikeKSkToFDEG33ZIyJLYRMBMDU1FVKpVP11cHAwUlPL3o+1V69eaNOmDaZNm4acnBxTVJGoUhrXbIzVA1ZDNlqGwU0Gw05S8Uv5fv4jLD1/AtFrZiFqWScsOb4E2XmW/ZdwcnIydu3apd7OjfTzZHAuUhbhi91foO2StricebnMx3Sr3w2rB6w22v7UxiCXT8Xu3a6QyyciPX0B5PKJj7+eKnbViERnFQGwe/fuqFevns6PkqD35FgVVTmDok6dOoWdO3di69atuHPnDj755BOj15+oqprVboYVr6zAxQkXMe65cXBx0G8Xhr3X9uKN9W/A/xt/vL7udey5uqfc1wVZl5LgvFu2Gx0TOuKjHR+VuTc1AMQ9H4e/hvyFag7VTFhLw5S1Cw5QBIViDkMg2Tyr2Av477//Lvd4UFAQrl27Bl9fXwCAQqHASy+9pPPckpZCNzc3jBo1CnFxcYLWlcgYQqqHYEGPBfgs+jMsOrwIi44swvUH1yt8XE5BDpbKlmKpbCnq16iP1yNfx/Bmw9VLyZgjmUymbvHbunWrxmegeEcPc991xCw8A0xTTEOOsuxeDu9q3ljcezH6N+5vwooZTt9dcEJCZnASFNksm5gEMmvWLFy7dk09CeSll17CoUOHUL16dY3zMjMz4eTkBFdXVyiVSkybNg337t3DDz/8UOE9OAmEzEl+UT7+OPsHvv33WxxIPVCpx9pJ7NAltAtebfoqeof1hoezh5FqWTXR0dHYtWtXmcejoqKwc+dO01XIDMhkMsTFxSE+Pr7M8FsSnLOLsvHNiW9w0fNiuddsX6c9EvsmQuolLfc8IesoFK6BSWXhJJBSVtECWJF33nkH48ePR/PmzWFnZ4evv/5aHf6WLFmC69ev48MPP0RKSgri4uIgkUhQVFSEpk2b4quvvhK59kSV52TvhMFNBmNwk8E4nHYY8w/Nx6rkVXqtJahUKbH54mZsvrgZ1RyqoXv97hjQeAB6NOgBr2peJqh9+eLj4zVaABMSEjB8+HB06dIFACxiL1+hPTkWsqxwNWHSBOwt2Au0BeBZ9rXsJfb4LPozfND2A0Ene+hTR6Hk5soFPY/IGtlEAHRzc8Mvv/yi89jIkaWbm7dq1Yrr/pHVeS7wOST0TcC8rvOw4vQKLDm+BMdvHNfrsY8KH2HtubVYe24tHO0cEV03Gn0a9kFMwxjBWoYqKzIyUiNAJCQkoEuXLoiNjRWlPubuXu49LDi0AKdfOg1UkP/retfFr/1+xYvSF01TOYE8vR2es3OwXo9zcQk1cs2IzJdNBEAiAnxcfTC+1XiMbzUeshsy/HL8FySeSixzsd+nFSgLsO3SNmy7tA3j/x6PFv4tEBMWg55hPdG8dnOLWhrEGlQ0FtIv1A/bsrdh0ZFFeJD/oMLrDY0YioU9FgraymuK8ZrFkz3mQnOyhx0ACVDuWpn2CAwca9C9iSyZTYwBNAWOASRLlFeYh79S/sKS40uwRb4FSpWyStepXq06OoZ0ROd6ndEppBPq16hvkl0iTDmuzJiq8n2UORayOoA2gKS5BCr7it/e3Z3csajnIgxrOqxylTakjo8ZOl6zdKZv5UmlU6xiLUyqHI4BLMUAKBAGQLJ0affTkHAiActOLMP5u+cNulawVzA6h3RG53qd0TGkI2q51xKolro93QUYGDjWrGZ3VlS/pKQkDBs2DImJiXp3ZT/Zuvb31r+R9G8S6vSug2vu1/TeJSa6djR+Hvgz6lWvV/lvqpJ1LGu8ZlWDu1KZr8de1yV/hDz5fHAXHFvGAFiKAVAgDIBkLVQqFU7fOo3Vyaux+sxqpNxNMfiaEX4RiK4bjReDXsSL0hdRx6uOYC2EursAzeeXvD71q0oALCgqwPZL27EqeRV+O/UbHiof6l2ndsHt8GXnL9Fa2roS34lhqvI9lkffmb716s2GROJotn8ckGkxAJbiGEAi0iCRSBBRKwIRtSIwvcN0bDn2BtYk/4J9d4CzVXzfPHXrFE7dOoX5h+YDAGq71y4Og48DYUv/lnBx1G8R6yeV3QVYpC4XMwSWV79r1+bg6NEzKCgYovf4uEJlIXZc3oFVyauw9txavcdvlogObIQ3GzVGuzrtEBj4bGW/HbOi7wzeR4+uISxsvpFrQ2R52AIoELYAkjV6upstIx/YfxfYdwc4eg8oEOjdw8HOAc1rN8eLQS/iucDnEOEXgUa+jeDs4Kx33XSzR/v2D0Vp8amofioVcOsWMHQooNQx9LJkfNz9vPs4mHoQa8+uxZqza3D74e1K1UMCCboFh6Gv7wU08HjyRqZtJRV6vCbX+qOqYAtgKQZAgTAAkjUq75dsbhFwKgs4dg84/TAQyRlpgt7bwc4BYT5heMb3GTSu2Vj9OcwnDC6OLlUOAKaaOKJv/W7eHIZNm+yQkJCAoSOGIqxdGC49uoRMl0xczruM5NvJVZqc42DngOFNh2OwVALH+z+XeZ6lToYw9z8AyDwxAJZiFzARlam8bjYXe6BVjeKPgIC+qB74Cf65/A+2X9qO7Ze340rmFYPuXagsxJnbZ3Dm9hmsObtGXS6BBCHVQxDgXAgvCVDbGfCrBvg5A7WcAV9nwPGJv8We/h5MtSBxec9dvhK4nQfcegTcc7uGy81UwBhgZa2VUKZWbSZ2iSDPIAxpMgQTWk1AoEetxyGpbJa6JZqdnROk0knlzgKWSidZ3PdFZCoMgERUJn0XynVxCUVNt5oY1GQQBjUZBAC4dO9ScRi8tB17ru3BjQc3BKmTCipcuncJl8o4LgHg4wTUehwK6905iTrX/w81XGqghksNnMw5CQQANwtu4l7uPXhV84KdxPDWe6VKidyCXOQU5OBB/gOcy3bA8dvAzUfFYe9m3uPPj4B7BU8+cnfxp1qAElULf/7u/hjQeAAGhg/Ei9IX1d+PQhGP8lvIAKAIaWkLLbKbtKTl0pwnARGZK3YBC4RdwGSNhOpmU6lUuJp1FQcUB3AgtfhDdkOGQmWh4HWuLAkk8HT0hJerFxztHOFg5wAHOwc42hf/u6Ss5OsiZREe5D9ATkEOcvJz1IHvYYH+s3CFUNO1Jvo37o9B4YPQNritzoW4U1ImID19QYXXCggYb9ETJcx9GSAyH+wCLsUWQCIqk1DdbBKJBHW966Kud10MiRgCAHhY8BBH04+qA+EBxQHczLkpaP31oYIKWQVZyMrKMvm9K6uGSw288swrGBQ+CFF1o+BgV/5beGVacC1Z8f/TOLGrQWRR2AIoELYAklDMsTXDFGvtqVQqpGWn4dTNUzh58yRO3TqFs3fO4tydcyZvXTMXYT5haOnfEi8EvYDW0taIrB1ZYeh7EidKEGliC2ApBkCBMACSEMx5UWOxgqlSpcS1rGs4e/sszt45i7O3z+L83fO4mnUVaffTUKSqaIybZSgJey39W+LZgGfR3L85PJ09Db5uRdulWeosYKKqYAAsxQAoEAZAMhR/UVdeobIQ17Ov41rWNY2Pq1lX1f/OyjOPrl1fV19IPaUI9gpWfy75CPcLFyTslcWc/7AgMiUGwFIMgAJhACRDsKvOeAqKCpD5KBMZuRnIyM3AkeQjWPjLQvQe2BsuNVyKyx9lILcgF4XKQhQqC1GgLCj+XFSgVWYnsYOboxvcnNw0Prs7uWt87ensCamXFFJPKaReUrg6lr8ci7GZ49ACIlNjACzFACgQBkAyBHc1ICIyPgbAUkwsRGZA331N9T2PiIioPAyARGbAVpbrICIi88AASGQGAgPHAtBeyFeT/ePziIiIDMMASGQGShZcLg/3NSUiIqFwJxAiM8F9TYmIyFQ4C1ggnAVMQuFyHURExsFZwKUYAAXCAEhERGTeGABLMbEQERER2RgGQCIiIiIbwwBIREREZGM4C5iIiMrFiUlE1oeTQATCSSBEZI3k8qlcmoisBieBlGILIBGZFbY2mY/i8DdHx5EidTlDIJFlYgugQNgCSGQ4tjaZD6UyH7t3u0LzZ/E0e7Rv/5ABnSwGWwBLMbEQkVkobW16OnAUtzbJ5VPFqJYWpTIfCkU8UlImQKGIh1KZL3aVjCItbSHKD38AUPT4PCKyNOwCJiLRFYequeWeo1DMRUjIDFFbm3S1UMrlk62yhTI3Vy7oeURkXtgCSESis4TWJktpoRSKi0uooOcRkXlhACQi0Zl7a5O+LZTW1B0cGDgWgH0FZ9k/Po+ILA0DIBGJztxbmyyhhVJodnZOkEonlXuOVDqJE0CILBQDIBGJztxbm8y9hdJYQkNnQyqdAu2fjT2k0ilWN+6RyJZwEggRia6ktUn3mnPFxGxtMvcWSmMKDZ2NkJAZXJuRyMpwHUCBcB1AIsOZ6zqAXBOPyDpwHcBSDIACYQAkEoa57gRS9q4YxdglSmT+GABLMQAKhAGQyPqZawslEemHAbAUA6BAGACJbIO5tlASUcUYAEsxAAqEAZCIiMi8MQCWYmIhIiIisjEMgEREREQ2hgGQiIiIyMYwABIRERHZGAZAIiIiIhvDAEhERERkYxgAiYiIiGwMAyARERGRjWEAJCIiIrIxDIBERERENoYBkIiIiMjGMAASERER2RgGQCIiIiIbwwBIREREZGMYAImIiIhsDAMgERERkY1hACQiIiKyMQyARERERDaGAZCIiIjIxjAAEhEREdkYBkAiIiIiG8MASERERGRjGACJiIiIbIxNBMDly5ejdevW8PHxwY8//ljuuUeOHEHbtm3RsmVLxMTE4MaNGyaqJREREZFpSDIzM1ViV8LYTp06BScnJ8ydOxctW7bEW2+9pfM8lUqFFi1a4Ntvv0W7du0wf/58yGQy/PzzzxXew83NDXZ2NpGniYiILJJSqUROTo7Y1TALDmJXwBQiIiIAoMKAdvz4cTg7O6Ndu3YAgNdeew0NGjRAQUEBHB0dy30s/0MRERGRpWCT1RMUCgWkUqn6aw8PD7i7u7MbmIiIiKyKVbQAdu/eHefPn9d5bPfu3QgKCtL7WhKJRONrlcrqe8iJiIjIxlhFAPz7778FuY5UKsW1a9fUX2dnZ+PBgweoXbu2INcnIiIiMgfsAn5CZGQkHj16hD179gAAli5dip49e1Y4/o+IiIjIktjELOBVq1bh888/R2ZmJhwdHeHm5oYVK1agWbNmWLJkCa5fv44PP/wQAHDo0CFMnDgRjx49gr+/P3788UcEBASI/B0QERERCccmAiARERERlWIXcCVxUWnz8fDhQ7zxxhto3rw5WrZsifXr15d5rre3N1q3bo22bduibdu22L9/vwlrap3kcjm6dOmCli1bomPHjjh37pzO8xISEtCiRQtERkbi3XffRWFhoYlrav30+Vns2bMH/v7+6tdA27ZtkZubK0JtrdvUqVMREREBb29vnDlzpszz+LowPn1+Frb8umAArKTIyEj88ssv6N+/f7nnqVQqvPnmm5g1axaOHj2Kl156Sd3NTMKYP38+nJyccPz4caxZswaTJ09GZmZmmedv3boVe/fuxd69e9G6dWvTVdRKxcXFYcSIETh69CjeffddTJgwQeucK1euYObMmdi8eTOOHz+OmzdvYvny5SLU1rrp87MAgIYNG6pfA3v37oWLi4uJa2r9+vTpg82bN2ssKfY0vi5MQ5+fBWC7rwsGwEqKiIhAw4YNq7So9MaNG1FQUGCKatqEtWvX4s033wQA1K1bF61bt8bGjRtFrpVtuH37Nk6cOIFBgwYBAGJiYnD16lVcvXpV47z169ejV69e8PPzg0QiwciRI/H777+LUWWrpe/PgkyjTZs2CAwMLPccvi5MQ5+fhS1jADQSLiptfKmpqRrPcXBwMFJTU8s8v1evXmjTpg2mTZvGnVsMlJaWBn9/fzg4FK8kJZFIEBQUpPX8P/06qOhnRJWn788CAC5evIj27dujQ4cO+Omnn0xdVXqMrwvzYquvC6tYB1BIXFTafFT0swA0n+Pynt9Tp05BKpUiJycHEydOxCeffIJvvvlG2ArbGH3/f+v7M6Kq0+dn0axZMyQnJ8PLywtpaWkYMGAAfHx80LdvX1NVk57A14V5sOXXBQPgU7iotPmo6GcRFBSEa9euwdfXF0DxX9UvvfSSznNL/tp2c3PDqFGjEBcXJ2hdbU1gYCDS09NRWFgIBwcHqFQqpKWlaf2B9PTrQKFQVOqPKKqYvj8LT09Pjcf0798f+/fvt4lfdOaGrwvzYcuvC3YBGwkXlTa+Pn36YPHixQCKB1Xv27cPPXr00DovMzMTDx8+BAAolUr88ccfiIiIMGldrU3NmjURERGBVatWASge0xQcHIw6deponBcTE4MNGzbg1q1bUKlUWLJkCV555RUxqmy19P1Z3LhxA0qlEkDxH6RbtmxB06ZNTV5f4uvCnNjy64LrAFYSF5U2Hzk5ORg/fjxkMhns7OzwySefoE+fPgCg8bM4dOgQ4uLiIJFIUFRUhKZNm+Krr75C9erVRf4OLNuFCxcwduxYZGRkwMPDA4sWLcIzzzyDCRMmoHv37uowvmzZMsTHx0OpVKJ9+/aYO3cu/xASmD4/ix9//BFLliyBvb09ioqK0KdPH7z//vta3cdkmMmTJ2PTpk24efMmfHx84ObmhuPHj/N1IQJ9fha2/LpgACQiIiKyMewCJiIiIrIxDIBERERENoYBkIiIiMjGMAASERER2RgGQCIiIiIbwwBIREREZGMYAImIiIhsDAMgEZlcUlISGjdubNb3j4iIQEJCgolqRERkWgyARCSInj17wtvbG8uXL9coz8nJQVBQELy9vXH16lUAQL9+/bB7924xqqm3HTt2YMCAAVV67JgxY+Dt7V3mx5gxY6p03Z07d8Lb27vC89avX4+YmBgEBwfD29sbhYWFVbofEVkvBkAiEkxgYKB6T9oSf/31F7y8vDTKXFxc4OvrW6V7FBQUQKUy/gZGvr6+cHFxqdJjv/zyS5w/fx7nz5/Htm3bAAD//POPuuzLL78UsqpacnNz0b59e8TFxRn1PkRkuRgAiUgwvXr1wvHjx6FQKNRlK1euxMCBAzXO09UF+/3336N58+bw8/NDs2bN1N2ve/bsgbe3N7Zt24bnn38etWvXRlZWFm7duoXhw4cjMDAQderUwfjx45GTk6O+XmFhIb744gs0adIEtWrVQqtWrfD3339r3HPdunVo2rQpgoODMW7cOOTl5amPPdkFfPXqVXh7e2Pt2rVo27YtatWqhd69eyM1NVXn8+Dl5YVatWqhVq1a8PHxAQD4+Pioy44fP47o6GjUrl0bLVu2xOLFi9WPffToEd555x3Ur18ftWvXxnPPPYcNGzbg6tWrePnllwFA3ZKYlJSk8/6DBg3C5MmT8dxzz+k8TkTEAEhEgvHw8ED37t2xevVqAEB6ejoOHz6MPn36lPu4ZcuW4YsvvsB7772Hf//9F/Pnz4eHh4fGObNnz8a3336L/fv3w9XVFaNHj0ZaWho2btyIlStXYv/+/Zg2bZr6/FmzZiEhIQEzZ87EwYMH8cUXX8DR0VF9PCMjA7/++itWrFiBxMREbNq0CUuXLi23nv/3f/+H6dOnY/v27SgsLMTo0aMr+QwBFy5cwKuvvoqRI0eq6/XVV1/hjz/+AAD88MMPkMlk+P3333Hw4EHMnDkTHh4eCAoKUtevpCWxX79+lb4/EREAOIhdASKyLoMHD8a0adPw3nvvYfXq1ejWrRs8PT3LfcycOXMwdepUDBs2DAAQEhKidc6nn36K559/HgCQkpKCHTt24ODBg2jUqBGA4oA4ePBgTJ8+HU5OTliwYAF+/PFHxMTE6LxmXl4e5s+fDz8/PwBAnz59sG/fvnJD3fjx49GxY0cAwHfffYcWLVrgzJkzlZrQEh8fjxEjRmD48OEAgLp162LMmDFYtmwZ+vXrh9TUVDRt2hSRkZHq4yVKxv/VqlVL7/sREenCAEhEgurQoQOysrJw7NgxrFq1CtOnTy/3/OzsbKSmpqJt27blnte0aVP1vy9cuAAPDw91+AOA5557DoWFhbh8+TIcHR2Rl5dX7jV9fX3V4Q8A/Pz8cP78+XLr0KJFC/W/69WrB29vb1y4cKFSAfDMmTM4c+YMfvnlF3VZYWEhateuDaA4QPft2xenTp1Cp06dEBMTow6DRERCYQAkIkHZ29ujf//++PDDD3H37l107NhRPftXF30ndLi6upb7GIlEUqlrPtkdXPL4ih735D2qKicnB+PGjVO3dpawt7cHALRs2RInTpzAli1b8L///Q9du3bFRx99hAkTJhh8byKiEhwDSESCGzJkCA4cOID+/furg01ZPD09ERQUhL179+p9/bCwMGRnZ+PcuXPqskOHDsHBwQEhISEIDQ2Fs7Nzpa6pj2PHjqn/ffnyZWRmZqJBgwaVukaTJk1w4cIF1KtXT+OjTp066nOqV6+OwYMHY/HixZg2bRoSExMBAA4OxX+zFxUVCfDdEJEtYwsgEQmuSZMmuHTpEtzc3PQ6f8qUKfjoo49Qo0YNtGnTBunp6bh165Z61uvTwsLC0LFjR4wfPx5ff/01Hj16hP/85z+IjY1VLzkzfvx4/Oc//4FEIkHTpk1x6dIlKJVKdO7cucrf14IFCxASEgIfHx+8//77aN26daUXtH7nnXfQpUsXzJgxA/3794dKpcKxY8eQm5uLUaNG4bvvvkNgYCAiIiLw6NEj/PPPP6hfvz4AQCqVAgC2bt2KVq1awd3dHc7Ozlr3uHfvHhQKBS5fvgwAOH36NOzs7FCvXj24u7tX+fsnIuvBAEhERlGjRg29zx0xYgQePHiAr776Cjdu3EBAQADee++9ch/z/fff47333kPPnj1hb2+PmJgYzJw5U338gw8+AABMnToV9+7dQ926dSscj1iRadOmYdq0abh48SKee+45fP/995W+RmRkJNauXYsZM2ZgwYIFcHZ2Rnh4uHrNPjc3N8yePRuXL19GtWrVEBUVha+++goAUKdOHcTFxWHcuHHIyMjAd999h9jYWK17bNq0CePGjVN/HR0dDaB4TcZ27dpV/hsnIqsjyczMNP6KqkREFuzq1ato1qwZjh07hnr16oldHSIig3EMIBEREZGNYQAkIiIisjHsAiYiIiKyMWwBJCIiIrIxDIBERERENoYBkIiIiMjGMAASERER2RgGQCIiIiIbwwBIREREZGMYAImIiIhsDAMgERERkY1hACQiIiKyMQyARERERDaGAZCIiIjIxjAAEhEREdkYBkAiIiIiG8MASERERGRjGACJiIiIbAwDIBEREZGNYQAkIiIisjH/DyOfgipfrYexAAAAAElFTkSuQmCC",
"text/html": [
"\n",
"
\n",
"
\n",
" Figure\n",
"
\n",
" \n",
"
\n",
" "
],
"text/plain": [
"Canvas(toolbar=Toolbar(toolitems=[('Home', 'Reset original view', 'home', 'home'), ('Back', 'Back to previous …"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plot_decision_boundary(w, b, X_mapped, y_train)"
]
},
{
"cell_type": "code",
"execution_count": 48,
"id": "a9f5c8f2",
"metadata": {},
"outputs": [],
"source": [
"def sig(z):\n",
" return 1/(1+np.exp(-z))\n",
"\n",
"def plot_decision_boundary(w, b, X, y):\n",
" # Credit to dibgerge on Github for this plotting code\n",
"\n",
" plot_data(X[:, 0:2], y)\n",
"\n",
" if X.shape[1] <= 2:\n",
" print(\"HI\")\n",
" plot_x = np.array([min(X[:, 0]), max(X[:, 0])])\n",
" plot_y = (-1. / w[1]) * (w[0] * plot_x + b)\n",
"\n",
" plt.plot(plot_x, plot_y, c=\"b\")\n",
"\n",
" else:\n",
" u = np.linspace(-1, 1.5, 50)\n",
" v = np.linspace(-1, 1.5, 50)\n",
"\n",
" z = np.zeros((len(u), len(v)))\n",
"\n",
" # Evaluate z = theta*x over the grid\n",
" for i in range(len(u)):\n",
" for j in range(len(v)):\n",
" z[i,j] = sig(np.dot(map_feature(u[i], v[j]), w) + b)\n",
"\n",
" # important to transpose z before calling contour \n",
" z = z.T\n",
" print(z,z.shape)\n",
" # Plot z = 0\n",
" plt.contour(u,v,z, levels = [0.5], colors=\"g\")\n",
" plt.show()"
]
},
{
"cell_type": "markdown",
"id": "36468e29",
"metadata": {},
"source": [
"##### 3.8 Evaluating regularized logistic regression model\n",
"\n",
"\n",
"You will use the `predict` function that you implemented above to calculate the accuracy of the regulaized logistic regression model on the training set"
]
},
{
"cell_type": "code",
"execution_count": 49,
"id": "36559ced",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Train Accuracy: 82.203390\n"
]
}
],
"source": [
"#Compute accuracy on the training set\n",
"p = predict(X_mapped, w, b)\n",
"\n",
"print('Train Accuracy: %f'%(np.mean(p == y_train) * 100))"
]
},
{
"cell_type": "markdown",
"id": "15f64b07",
"metadata": {},
"source": [
"**Expected Output**:\n",
"
\n",
"
\n",
"
Train Accuracy:~ 80%
\n",
"
"
]
},
{
"cell_type": "markdown",
"id": "6b7715cd",
"metadata": {},
"source": [
"##### My Solution"
]
},
{
"cell_type": "code",
"execution_count": 50,
"id": "066eceb6",
"metadata": {
"scrolled": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"The cost is 2.0028840493199764\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"/tmp/ipykernel_6349/2829149365.py:82: RuntimeWarning: divide by zero encountered in scalar divide\n",
" if np.abs((cost_i[i]-cost_i[i-1])/cost_i[i])<0.05:\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"The cost is 0.6893248648821225\n",
"The cost is 0.5947327181536963\n",
"The cost is 0.5594964140290553\n",
"The cost is 0.5439204525169686\n",
"The cost is 0.5365941656209957\n",
"The cost is 0.5329892209020973\n",
"The cost is 0.5311518234465994\n",
"The cost is 0.5301878305603884\n",
"The cost is 0.5296693737410955\n",
"The cost is 0.5293843854020753\n",
"The cost is 0.5292246537838604\n",
"The cost is 0.5291335540683393\n",
"The cost is 0.5290807827627604\n",
"The cost is 0.5290497884840468\n",
"The cost is 0.5290313609587369\n",
"The cost is 0.5290202869125287\n",
"The cost is 0.5290135693988894\n",
"The cost is 0.5290094612865763\n",
"The cost is 0.5290069311969307\n",
"[ 0.62 1.18 -2.02 -0.92 -1.43 0.13 -0.37 -0.36 -0.17 -1.46 -0.05 -0.61\n",
" -0.27 -1.19 -0.24 -0.2 -0.04 -0.27 -0.29 -0.46 -1.04 0.03 -0.28 0.02\n",
" -0.32 -0.14 -0.93] 1.2715618096527268 0.5290053880660263\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"No artists with labels found to put in legend. Note that artists whose label start with an underscore are ignored when legend() is called with no argument.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Train Accuracy: 0.000000\n",
"(array([ 0.62, 1.18, -2.02, -0.92, -1.43, 0.13, -0.37, -0.36, -0.17,\n",
" -1.46, -0.05, -0.61, -0.27, -1.19, -0.24, -0.2 , -0.04, -0.27,\n",
" -0.29, -0.46, -1.04, 0.03, -0.28, 0.02, -0.32, -0.14, -0.93]), 1.2715618096527268)\n"
]
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "2516291ceb8a486e86cf3517f8d513c4",
"version_major": 2,
"version_minor": 0
},
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAyAAAAGQCAYAAABWJQQ0AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAA9hAAAPYQGoP6dpAACZlklEQVR4nOzdeVxU1fsH8M8MoLgguODGLu6KimAq7pYGKmBqrpllaZmAS2qLVpaaWpqCkpXlr1IT0zTB3L/uW+5m7gu7Gy4guLLM74/bjAyzMMudlc/79eKFnHvn3jMjeu9zz3meI8nOzpaBiIiIiIjIDKSW7gAREREREZUdDECIiIiIiMhsGIAQEREREZHZMAAhIiIiIiKzYQBCRERERERmwwCEiIiIiIjMhgEIERERERGZDQMQIiIiIiIyGwYgRERERERkNgxAiIiIiIjIbBiAEBERERGR2TAAISIiItLDypUr4ebmBjc3N1y5ckVl+759+xTbd+/eLco5U1NT4ebmhn379un92oCAAMyePbvU/TZv3oyQkBDUqlULbm5uyM7ORu/evTFmzBjFPvv27cPs2bNRVFSkdz+I5BiAEBERERnAxcUFCQkJKu0JCQlwcXGxQI8MV1BQgNGjR6NOnTpYt24dtm/fDhcXF8yfPx9TpkxR7Ld//37MnTuXAQgZhQEIERERkQH69OmD33//HTKZTNH2+PFjJCUlITw83II909/169eRm5uLvn37okOHDmjTpg0cHBzQuHFj+Pn5Wbp7ZGcYgBAREREZYPDgwUhPT8ehQ4cUbRs3bkRhYSEiIiLUvmb16tXo0KEDatWqhXr16mH06NG4efOm0j6PHj3C+++/Dz8/P3h4eGDw4MG4fv262uMlJibipZdeQp06deDt7Y0RI0YgPT1dr/cxe/ZstGjRAgAQHR0NNzc3xbSr4lOwZs+ejblz5wIAatSooZhmRqQvBiBEREREBvDy8kJISAhWr16taEtISEDv3r1RqVIllf1//vlnvPPOO2jUqBGWL1+O6dOnY+fOnejduzfy8vIU+40fPx6//vorxo4di+XLl6NBgwYYNWqUyvGWLVuG119/HY0bN8avv/6KhQsX4vz58+jduzdyc3N1fh+vv/46fvnlFwDApEmTsH37dqVpV8X3Gz58OABgy5Yt2L59O7Zv367zeYjkHC3dASIiIiJbNXjwYEybNg1z585FdnY2du/ejbVr16rsV1hYiFmzZqFjx45YtmyZor1BgwYICwvDihUr8O677+Ly5ctYu3YtPvnkE0yYMAEA0L17dzx8+FDpdXl5eZg+fTqGDRuGxYsXK9qDgoIQHByM5cuX47333tPpPXh4eODp06cAAD8/P7Rp00bjfnXr1gUABAcHw9GRt5FkGI6AEBERERmob9++ePbsGbZs2YI1a9agVq1a6NKli8p+ly9fRlZWFgYOHKjU3r59e3h5eeHAgQMAgGPHjqGoqAivvPKK0n79+vVT+vno0aN48OABBg4ciIKCAsWXh4cHGjRogIMHD4r8TonEw9CViIiIyEAuLi7o3bs3EhISkJaWhldffRVSqerz3fv37wMAatWqpbKtVq1aiu23bt0CALi7uyvtU7NmTaWfs7KyAACRkZFq+8XcDLJmDECIiIiIjDB48GAMHDgQRUVF+Omnn9TuU7VqVQDPA4zibt26hcDAQADPA5SsrCylPJLbt28rvaZatWoAgG+//RZNmjRROWblypUNeCdE5sEAhIiIiMgI3bp1wyuvvAJXV1e1wQAg5HrUrFkT69atw+uvv65o//vvv5Geno6oqCgAQm6FVCrF+vXrFTkgALBu3Tql473wwgtwcXHBtWvXMHToUBO8K/XKly8PQCg3bGtrnZD1YABCREREZAQHBweNIx/F9/n4448xfvx4jB49GgMHDsT169cxc+ZM+Pv7Y9iwYQCEQGXAgAH48ssvUVRUhNatW2PXrl3Ytm2b0vGqVKmCL774ApMmTcLdu3fx0ksvoUqVKrhx4wb279+Prl27quSNiKFRo0YAgMWLF6NHjx5wcHBQjN4Q6YoBCBEREZEZvPHGG6hQoQLi4uIwdOhQVKpUCT169MAXX3yhNGVq4cKFqFy5MhYtWoT8/Hx06tQJP/74I0JDQ5WO9+abb8LDwwNxcXFYu3Yt8vPzUadOHYSEhKB58+YmeQ+hoaF4++238dNPP+Grr76CTCZDdna2Sc5F9kuSnZ0tK303IiIiIiIi47EMLxERERERmQ0DECIiIiIiMhsGIEREREREZDYMQIiIiIiIyGwYgBARERERkdkwACEiIiIiIrNhAEJERERERGbDAISIiIiIiMyGAQgREREREZmNo6U7YA0qVaoEqVS3WGzlZeD9g0CRDJBKgPkhwLAGJu4gEZGJFRUV4eHDh5buhk3R59pBRGRvjLluMAABIJVKdbqIZOQBb+wSgg+5N3cB3TwAz8om7CAREVkdXa8dRESkjP9z6uFyjnLwAQCFMuBKjmX6Q0RERERkaxiA6KGBqzDtqjgpgPquFukOEREREZHN4RQsPXhWBn7oAozaDcgHQmQAtqYDbzWxYMeIiIiIiIxUVFSEW7duoaCgQGWbo6MjatWqJcrUUwYgenrZC5BAOQB5Z4/QzjwQIiIiIrJVt27dgouLCypXVr2pzcvLw61bt1CnTh2jz8MpWHq6nAMUlWhjHggRERER2bqCggK1wQcAVK5cWe3IiCEYgOhJXR6Ig4R5IEREREREumAAoifPysDwhsptrzXk9CsiIiIiIl0wANFTRh6w/JJy24pLQjsRERGVLTJZ6fsQkTIGIHriWiBERERlW24uEBMD+PkBXl7C95gYoZ2ISscqWHpq4KpcBQsQfmYOCBERkf3LzQXatwfOnweKilWliY8Hdu4EDh0CXFws1z8iW8AREBFISt+FiIiI7MDUqarBByD8fP48MG2aZfplSpxmVnY4OjoiL099XkFeXh4cHcUZu2AAoqfLOcqjH4BQlpdTsIiIiOxfUpJq8CFXVAQkJpq3P6bCaWZlU61atZCbm4v09HSVr9zcXNSqVUuU83AKlp7kZXiL54GwDC8REZH9k8mA/Hzt++TnC/tJbHh6BKeZlV1SqVSUhQZLPY/Jz2BnWIaXiIiobJJIACcn7fs4Odl28AGUzWlmZF4MQPTEMrxERERlV3g4INVw9ySVAhER5u2PKZSVaWZkOQxA9MQyvERERGXXrFlAkyaqQYhUKrTPnGmZfolFn2lmRIZiAKIneQ5IScdum78vREREZF4uLkIORFQU4OsLeHgI36Oi7CM3oqxMMyPLYgCiJ8/KwJy2qu0f/s1pWERERGWBiwsQGwskJwPp6cL32FjbDz7kysI0M7IsBiAGCK6p2sZpWERERGWPPY4E2Ps0M7I8BiAGkK+GXhxXQyciIiJ7YO/TzMjyuA6ISOzwAQgRERGVUfJpZrGxtr+uCVkfjoAYgKuhExERUVnB4IPExgDEAJyCRURERERkGAYgRERERERkNgxADKBuCpYMQOw/lugNERGRZXFROt3wcyISMAAxgLopWACw4B+uBUJEZE5TpkxBQEAA3NzccO7cOY37/frrr2jdujVatWqFcePGoaCgQLFty5YtaNOmDQIDAzF8+HDk5fE/cl3k5gIxMYCfH+DlJXyPiRHa7Zm+QURZ/ZyItGEAYgDPysD7LVXbuRYIEZF5RUZGYsuWLfDy8tK4T0pKCr788kts2bIFJ0+exK1bt7B8+XIAQF5eHqKjo7Fy5UqcPHkStWvXxvz5883VfZuVmwu0bw/ExwMpKUBmpvA9Pl5ot7eba0ODiLL2ORHpigGIgca1YCI6EZGldejQAR4eHlr3SUxMRJ8+fVCzZk1IJBKMHDkSa9euBQDs2LEDgYGBaNiwIQDgrbfeUmwjzaZOBc6fB4qKlNuLioT2adMs0y9TMCaIKEufE5E+GICIiFXqiIisT3p6utIIibe3NzIyMjRuu3HjBopK3jGSkqQk1ZtquaIiIDHRjJ3JyQESEpTbEhKEdhEYE0RY1edEZEUYgBiIa4EQEdkOSbGFDGQlJvFLuMiBXmQyID9f+z75+WZKuM7JAUJDgSFDgLg4oS0uTvg5NFSUIMTQIMKqPiciK8MAxECVNawhX4lryxMRWRUvLy+kpaUpfk5PT4enp6fabWlpaahTpw6kUl4eNZFIACcn7fs4OZlh8Tp58HH4sPDzuHFA/frCd0BoNzIIMSaIsJrPicgK8X9YA+UVqG///ap5+0FERNpFRERg48aNuH37NmQyGZYtW4b+/fsDAF588UWcOHECly5dAgD89NNPim2kWXg4oClGk0qBiAgTd6Bk8CF3tcRF2MggxNggwuKfE5GVYgBiIJbiJSKyvEmTJqFp06a4fv06+vbti8DAQABAdHQ0Nm3aBADw9fXFRx99hJdffhmtWrWCu7s7hg8fDgBwcXFBXFwchg0bhsDAQFy/fh0TJ0602PuxFbNmAU2aqN5cS6VC+8yZJu7A5s2qwYcmhw8L+xvImCDC4p8TkZWSZGdnl/nZhy4uLgYNt08+CMw7rdq+KwLoqr0oCxGRVSkqKkIua4LqxdBrh73IzRUSsBMThWlITk7CzfjMmYCLixk6EBf3fLqVNrGxQs1cA8mrYJVMRJcHEYcOaX+/Fv+ciEzEmOsGAxAYfhE5egt4YZ1q+5F+QJtaInSMiMhMGIDor6wHIMXJZBbKZahfX3XaVXH+/sCVK0afRqwgwlKfk8X+fsiuGXPd4P+cRtCUB/JQQzsREZE9ssjNbVyc9uADELbLq2MZwcVFGEhJTgbS04XvsbH6j2CY83PiCuxkzVizyQishEVERGQ8vZ/QJyToNv0KEParWRMYPNigvpVkjiDC2BELTdPG4uOBnTtLnzZGZGocATFCsoanCCl8ukBERKSVUU/ow8KAdu10O1G7dsL+Vk7MEQuuwE7Wzi5zQJ4+fYrx48fDxcUFRUVFmDdvntb9DZ3H+/sVYNB21fZ3mwJLuuh9OCIii2EOiP7Kcg6IqZ7Q65rYDUB9KV5/f+VpWe3aAVu2AK6uhnfWDET5PIrx8wNSUjRv9/UVppERGcOuc0CmTJmCgIAAuLm54dy5c0rbrl69ip49eyIoKAjdu3fHhQsXAABJSUno0KEDvvrqK7i5ueHIkSMm6VtIbfXtS8+zFC8REdkXq3tC7+oqBBfykZDYWCHhPDZW+NlGgg9A3BELrsBOtsDqA5DIyEhs2bIFXl5eKtvGjx+PESNG4Pjx4xg3bhyio6MBCKvcent7AwB8fHyQnp5ukr55VgYmtVRtL5QBVwxfeJWIiMiqyJ/Qx8cLT9YzM4Xv8fFCu75BSFKS6s22XFGRUG1KJ/IgZNWq56V2Y2KEn20k+ABE/DzAFdjJNlh9ANKhQwd4eKguqpGVlYXTp09j0KBBAISVblNTU5GamgoPDw9F0JGWlgZPT0+T9W+gv/p2JqITEZG9sOon9K6uqgnmgwfbTPBhihELrsCuG44CWY7VByCaZGZmok6dOnB0FO70JRIJPD09kZGRgYiICOzfvx8ff/wx7t69i7Zt25qsH5oS0ZddMNkpiYiIzIpP6E3HFJ8HV2DXjOWJrYNNP6eXlPjXKPsvlHV2dsaSJUss0SWF788BU4OEaVpERES2Sp8n9LreJIeHC9O31AU1ZfEJvdifh4uLkLjOFdiVsTyx9bDZERAPDw9cv34dBQXCqn8ymQyZmZkmnW6ljqZEdBmAQzfN2hUiIiLR8Qm96Zni8xBr8UR7wvLE1sNmAxB3d3cEBARg9erVAIDExER4e3vDx8fHrP3wrAyMbmLWUxIREZmV2DkF8if0UVFCSVgPD+F7VFTZfApt6s+jrExnK42YUwnJOFa/DsikSZOwadMm3Lp1C9WrV0elSpVw8uRJAMDly5fx3nvv4d69e3BxccGSJUvQpIn+0YCxtdyP3gJeWKfafqQf0KaWwYclIjIbrgOiv7K0DojY61SUZOy6IvaGn4f4ZDIh5yMzU/M+Hh7CaBE/e90Yc92w+gDEHIy9iHBBQiKydQxA9FeWAhBACEKYU0C2jAs0isuuFyK0Zd+d44KERERkH5hTQLaO5YmtBwMQEWhKRAeAWcfN1w8iIiJz4BQVskUsfmA9GICIwLMyMLS++m3fcxSEiIiIyOJY/MB6MAARSaSf+naW4yUiIioDcnKAhATltoQEoZ2sZtVxTiW0DgxARKJtGtY3p83XDyIiIjKznBwgNBQYMgSIixPa4uKEn0NDy2wQYu2rjnMqoeWwChbEq2Tyzm7gh/Pqt7EkLxFZM1bB0l9Zq4Jlt3JygM2bgcGDn7clJABhYYCrq26vDw0FDh9+3ubvD1y9+vzndu2ALVt0O56dMHXpZrI8VsGyEp8Ea942g8noRERE1sXYkQt1wQegHHwAwvYyNhLCVcdJGwYgIvKsDPT1Vb8tKZXJ6ERERFajZPAwbhxQv77wHdAtaNi8WTX40OTwYWH/MoKrjpM2DEBENqSB5m0syUtERGQFxBq5GDxYyGDWRWys8jQvOyaTCYtVapOfbz2J6WR+DEBEpi0ZnQsTEhERGUmMalNijlzExAg5H9r4+wv7lRESCeDkpH0fJycmgZdlDEBE5lkZGN1E8/aITebrCxERkV0Rq9qUmCMXcXGqIyclXb36vL9GEnPUwJQjEFx1nLRhFSyIX8kkIw/wWq55+9TWwMy2op2OiMhorIKlP1bBMjNTVJuqX1978ODvD1y5onl7QoIQ/Ohq1SqDpmHl5gpJ3UlJwtQlJyfhBn/WLP0rSYl5rNLOwypY9s2Y6wYDEJjmIjL1MPDlSc3b04cLoyVERNaAAYj+GICYkaacDXV0DULi4p4nnGsTG6t5+pQp+lWCmDfy5g4KcnOFaleJic+DnYgIYOZMBh/2gGV4rdCsdkD9Kpq3B68xX1+IiIhsmtjVphISdAs+AGG/kjkncq6uQlDRrp1ye8mcECPWARGznK25S+OKveo4k9btBwMQE/rtJc3bbj0BPH82W1eIiIhsl9jVpsLCVIMGTdq1E/bXpGQQEhsrTNuS99fIRQjFLGdrydK4hiacW/tq6mQYTsGCaYfRe/8FbErTvL22M3DjTZOcmohIZ7Y8Bevq1asYM2YM7t69C1dXV3z77bdo3Lix0j6rVq1CfHy84ufr168jJCQEK1asQGpqKlq3bo0mTZ5XEFm+fDn8/Py0npdTsCzA2JyN4sTOKTF2RXU1ZDLhpjszU/M+Hh7C6EJpN/hiHstcmEdi3TgFy4r91RtoqGUq1s0nQIXvgaO3zNcnIiJ7Mn78eIwYMQLHjx/HuHHjEB0drbLPkCFDsH//fsVX7dq18eqrryq2u7q6Km0vLfggCxC72pTYIxeurqojL4MHGxx8AOKWs7XF0rhcTd1+MQAxg/9Fat/+pAh4YR0Qss48/SEishdZWVk4ffo0Bg0aBACIiIhAamoqUlNTNb7m+PHjuH37Nnr16mWubpKxxMrZKEkehKxa9TzRPCZG+NmIaVNiErOcra2VxuVq6vbL0dIdKAs8KwM/dgXe3q19v0O3gIo/AHsigTa1zNEzIiLblpmZiTp16sDRUbicSSQSeHp6IiMjAz4+Pmpfs3z5cgwaNAhOxR4H5+bmolu3bigsLETv3r0xadIkODg4mOU9kA7kORu6VpvSlrNRkqaRCwM8zn+M7CfZeFzwGI/zHxv0Pb8oH84OzqjoVBEVnCqgUu+KqJFZAVmZFSF7VhEoqADkV4SksAK861ZE/+iKOJdVQdjfUfheuVxlSNQMZcyaBezcqXlK08yZBr1tk9BnNXV9Rm303Z9MgwGImbzVBHjZC2ixGrj/TPN+jwuF0RD/KsCqlxiIEBGVpuSNlkxLqZxHjx5h3bp12LZtm6Ktdu3aOHfuHNzd3XH//n28+eabWLx4Mcbp+sSdTE8+UiH2OiB6yH2ai4wHGapfuc//fO/xPdHPCwBo8d9XMTIAKQC6rFTd3dnRGZ5VPOFVxQterl7wruINL1cveFXxwk9JXvi/hd7YmljFqkvjijllzFxrn5DumIQO8ycSVv0JyNYShBTnUxlY05OBCBGZlq0moWdlZSEoKAjXrl2Do6MjZDIZGjVqhO3bt6sdAVm1ahWWLVuG7du3azzm2rVrsWbNGqxevVrruZmEbgHFE8fl63PI1/MwIviQyWTIzM3Ev7f/VRtkpD9Ix4OnD0zwhiynSvkq8KriBW9Xb0WgIv/Z180Xvm6+akdRzCkmBoiPVz8NSyoFoqJKL47GRHbTMea6wREQC7j/FuC3HEjJK33f1DxhRKR6eWBee+CNJqW/hoiorHB3d0dAQABWr16NYcOGITExEd7e3hqnX61YsQLDhw9XasvKyoKbmxucnJzw9OlTJCUloUWLFmpfX9aYcrqKQceWj4QUrzYVEwPUrKlztanCokJcunsJp64dxMm//8TJqk9w6uYp3Hl0R/83YcMePH2As1lncTbrrNrtbs5uCKoThOC6wYovH1cfswYlYkwZ0yWRXdcKzyQejoDAck+xYvYBi/7V7zVOAF7yAt5rBvRhkRYiEomtjoAAwOXLl/Hee+/h3r17cHFxwZIlS9CkSRNER0cjLCxMkWyenJyMTp064fz583Ap9sgzMTERs2fPhlQqRWFhITp16oSZM2eifPnyWs9rryMgppyuYu6pMI/zH+Pfa4dxctdvOOnpiFO3TuGfW//gUf4j8U9WBlR3roZgqSeCg8IVQYnHX/sg6dXLZAn7xq6m7ucHpKRo3u7rKyyQSPoz5rrBAASWvYhk5AGBvwN3nur/2vIA2tcF3m/BYISIjGPLAYil2GMAYsrpKqaeCpPzJAfHbxzHyRsncerWKZy8cRIX7lxAoazQ8IOaUbkCoEIB4AQpnkqK8NAJKLKBX69aeUDwIzcEhY9GsF9HBNcNRh2XOiY5lyEJ57a29oktYQBiJGu4iLy4Adh53fDXlwdQvyrg7gz08ARebyxU3yIi0gUDEP1Zw7UDEHealBhz7s11bJlMhlM3T2Hzlc3YfGUzDqUfMnuw4QAp6uYUwbOoMjxfeAme1f3gXtFdUcFKXpVK/mel74/zUWHoCFQ4fBzOBYBDibsxGYB8B+CxI/CoXWs8/nUZHjk74FH+IzzOfyx8LxC+F2/LfZaL67nXkf4gHek56Uh/kI4nBU/M+rnUdamLbr7d0K9JP4TWD0VFp4pmPX9xHAExHQYgRrKWi8jRW8D7B4F9N8U5nnt5oGZFBiVEVDoGIPqz5LXDVFOZTHmzJsax7z++j+3XtmPzlc3YcmULbuaJdMFUw6kQ8HB2h2edRvB0rgXPrQfheeEGPB8AXg8Az6o+qHU29XngoG8SfEICMGSI7h1atcqg8sAymQx3Ht1RBCRpOWnCn4v9fD33usmCt4pOFRFWPwz9m/RH74a9UaW8ltWZTcCUQXVZxwDESNYSgMhl5AGjdgNb0sU/dp0KQGUnoJwDABngJAV6egHRLRicEJVlDED0Z6lrh6mmMplyuooxx/739r/YcGGDMMqRcQhFMg0r0xmo6mMg8AbQ6iYQeBNofAfwygHcZy2AdNx45cpbpdE3CJFX8CqNvOKXiRQUFeBm3k2k5aQhNTsV/9z6B8duHMOx68eQ/SRbtPOUdyiPHv490L9Jf/Sr+yKq7DygHFQlJOhcTEBXrIJlOgxAjGRtAYhcRh6w/CLw5Qkgr8D056vjDFRzBp4VPQ9QnhUJFwOZDPCozHwTInvFAER/FitgYsInutYyAlIkK8Kmy5sw/9B87E7ZbdgJ1fDKEYKNwJvPgw7vHEAlnvL3B65cEf5s6pGK+vWV1zLR1hczk8lkuHb/Go4tm4Fj237BsbrA8bpArvb6DDqpXCDFm8eLEN1jKhpMmClKOWVNjE1kJ/UYgBjJWgOQ4jYmAx//DZy5b+mePM83kQco6oKV4m26budoDJHlMADRn6WuHaYMEiydAzJ3/hMsP70c3xz+BhfuXDDsRACkEika12iMVkfSEHg5TxFsVH+sx0GKjzqYaqTCSkZAdPJfoFQkAS5XA47Vff51og7wqJxhh5XIgF6ZFTFu5yO8dO2/YNCEC0paeiV0S59fTAxAjGQLAYicfFQkKQU4cQd4Ku5ItNVwLw9UdTY8mNFlO49lvecqC8cy5bkMCeYZgOjPEtcOU1f1sVQVrAYt76LfnMX48XQ8sh5l6X1sJ6kTOvt0Rmj9UHTy7oSAWgGouORH3W7utSk+miH2SIWZckBEUUqgVCgBLtQAjk0ehiNNXfHX5b+QmpOq92ma3gZi/gaG/wNUDDJdEGJu9roSOwMQI9lSAFLSxmTgm9NAxkPh67FtVBskIjP5sSvwlg4LmDIA0Z89joAApp2uUvLYjk5FqPfqUpys9hGyn+o3xO/j6oOw+mEIaxCG7n7dUblcsWhb35t7dYo/hTfFSIUpc0vEZECgJBs0CCdunMAf5//AH+f/wKW7l/Q6ZdXHwKjjwPsjvkPN197Rs8PWxZ5zUBiAGMmWA5CSNiYDS84CabnC6AiDEqKyTQogdXjpIyEMQPRnjzkgJZU6XSQnR3lVckDnROKTN05hzF/v4u/Mv3XqSzmHcujs01kIOuqHoXGNxppX5dbn5h4AHB2BgmLJlpUrA2fPAt7eph2pUNdPf3/lkRZLBh+A0YGSTCbD2ayz+OOcEIycuX1G51O7ObthRrcZeDf4XThKHQ3pvUFspbS1pTEAMZI9BSDqyIOSm4+AZ4VCYOLsAKTlATn5lu4dEZnargigq4f2fRiA6M/eqmDprfiNqfzJvw6JxLlPc/Hprk8RdyROp4pWXlW8MK7tOLzd+m24OutxE67uxtnXV/vwUXHy9wCYdqTCwM/RrEQMlE7dPIXYyZ3xm28unukYU7Ss1RLxveLRwbuDAZ3XjS2WtrY0BiBGsvcARJujt4AFp4HTdwGH/+aRywMU+Z8dJEByrv3mmxDZM46AmI6l1wGxaFUfA25IZTIZ1p5bi/Fbx+N6bukr7wbVCcL77d/HgKYD4OTgZHw/Y2OBmjUNG80w9UiFESNJZiNWoPTfa25XAr4PAr5tA9zU8Xd2RMsRmPvSXNSqXMu491KCLZa2tgYMQIxUlgMQfRTPNykeoJQMVgpl+m/naAyRaTAHxHSs5dph9qo6BkzJuVp4B2M3jcXWq1tLfUmvBr0wJWQKOvt01jzFSt/+Fr+5NzSfwxZGKkxMlp0DyRYjAiU109meOQC/NwNi2wLHShmpBQDX8q6Y0W0GxrQZI9q0LFstbW1pDECMZC0XkbLu6C3gh3PAiSwgN9/wYEbX7TyW9Z6rLBzLlOdylAIvewFRrIJlUmX22qFHTsRTB2DuwgH4MjsJTwufat23XtV6iO8Vj9D6oWL0UjtDK1rZwkiFyESdmqQleJUBOOQFfB0C/KnDQ5MWtVogvlc8Onp31LMTqmy1tLWlMQAxUpm9iBAR/YcBiP7K9LVDh1GE//kB771ZE5eKbmvdr5xDOXzQ4QN81PEjVHCqIGYv1bOltTcszCRTk3SYzrajdxNE9yzEhfulV88a3mI4vurxFWpXrq1nRwS2XNra0oy5bpTR/zmJiIjKoJwc4Yl9cQkJQrs+YmKEm0Y1blYGhvYHXhqBUoOPF/1exD/v/oMvun1hnuAjIUH3tUHGjVP9rMqYqVNVb5wB4efz54U8JL25ugrT1dq1E36OjRVGm+TDAO3a4aWVh3B67Bl89dJXqORUSevhlv+zHI0WN0Ls4VgUFBVo3VcdiUQY1dHGycnwaY4uLkKQERUljKR4eAjfo6JsO/gwFkdAUMafYhERgSMghrC5a4eYOQxqRhEKJcCSNsDU7sADZ+0vr1WpFr55+RsMaT5EnDwPXdnK2htWwqT5CzpOZ8t4kIFJ2yZh9dnVpR4yoGYAFvdajM4+nVW2acuVsqrS1jaEU7CMZHMXESIikTEA0Z9NXTvErOKkJgfkWF3g3T7A8braXyqBBO+1eQ8zu8+Em7Obfu9BLLaw9oYV0Hlq0tcJkPQyfQ7MzuSdiNoUhfN3zpe672stXsNXL32FyqijU/6KPU+TMiVOwSIiIiL1ND31L5mEffiwsF9p07HCwhTTZ2QQkobbvl168BFUJwhHRh3B4l6LLRd8ADpNAbKZ4EOsKXVq6DQ16XEOJEOH6PZ7Y6Tuft1x6t1T+LrH18qr3qux4p8VaLS4ERoOX4TF8TKkpAiBVEqKMNLRvr0QdMhxmpT5cQQENvYUi4jIBDgCoj+buXaYYiXvnBw86dUT79Q8gl9bad+1Svkq+LL7l3g3+F04SB1074ep2XpFKzOUBdY6NQkFiMJixGKC0GDGwC3zQSYmb5+MVf+uKn3nf4YBG34CCssrmkqbVmVP06RMiVOwjGQzFxEiIhNhAKI/m7p2iFz56WbeTfT7LRKHbhzRut+Q5kMwv+d81HGpo2tPSRdmmkaWmwu0f6EQ5y8ARXgePEpRgCY4j0MIgQvyRD2nPnYl70LU5iicyzqnfcfUjsDq9cCjGoomW15/w1owADGSTV1EiIhMgAGI/mzu2mHo2hclnM86j5dXvIz0B+ka92lYtT7i+yzBS/VeMqSnpI2ZE+lz/28tpo3MQCIikI9ycMIzRCARM/GJcvAhp8sImojyC/Ox6MgifLb7M+Q9U9MfuXv+wIotwL36AGx7BXJrwRwQIiIi0iwuTnvwAQjb4+K07vLg6QP0/q23xuBDKpHisy6f4Z/3/rXP4MOEORc627xZt+ADEPbbvNmo07m8OQCxsVIkwx/p8EIy/BGLCeqDj9hYswYfAODk4ISJ7SfiYtRFDA0YqnnHaleB118EqmQIrzOitC4ZjwEIERGRPRNx7YvozdFIzlY/b6VK+SpIGpKE6V2no7xjebX72ARNQUZamjDyMGTI80AtLk742QxJ2AqDB+teE1asgOC/dV+03q/7+1t04ca6LnWxst9KbB62GVXKV1G/k1saMLwnJJXuICLCvP0jZZyCBRscRiciEhmnYOnPZq4dIk3ZSfg3AUP+UJ/M7l/VH4lDEtHUvamxvbUsbYndlSsDecWe+lu6dK+uU+rESKy3sdXjz2WdQ68VvZH6IEXt9gr32uDKp/9D3eosb2UM5oAYyWYuIkREJsIARH+WunYYVKHHyKTltJw0tFjSAjlPVZ/yv+DxAjYN3YTqFavr2Skro8tnVBpzBSG6BgT9+gHr1hnXL1NUUTOD2w9vI2x5H5y4dVTt9u5+3fHX0L/g7FjKqpmkEXNAiIiI7FhurvBg2c9PWBzOz0/4WedrvxFrXxQWFWL4+uFqgw/X8q74fcDv9hl8APoFH4AoORel0mdK3bp1wvfia7zom8dSbN2XUrVrJ+xvBWpWqokdI7YioGaA2u07k3di6B9DUVBUYOaeEcAREAAcASEi4giI/sx17RB1lWYD1r6YvW82Pt75sdptq/qvwuDmln/abTR9n/JrYo4pSPpMqSspOFgYPjt6VL+1Q2x49fgbuTfQ8f864tr9a2q3j2w1Ej9G/AgJM9L1xilYRmIAQkRlHQMQ/Znr2qF1MbhSFlQz1tHMowhZFqL2KfHwFsPx6yu/mubElqDrtCZNdCxjLApjgpDi9AkizLDwoalcu38NHZZ1wM28m2q3v9/+fXzd42sGIXpiAGIkBiBEVNYxANGfua4dfn5ASorm7aZaUC3vWR5af98al+9dVu2Tmx9OvXtKc7UhW1VaYndpzJmEXTwgkOd6iKG0IMRGV48/c+sMOv/cGdlPstVu/7L7l/io00fm7ZSNYw4IERGRHZLJgPx87fvk5wv7iW3Clglqgw+pRIoV/VaIE3xYw7oacrqslVKaUsoYi0qe17NqFfDHH8Johhi05bG4uqommA8ebPXBBwAE1ArAX0P/QkWnimq3f7zzY3x/7Hsz96rsYgBCRERkpSQSYcE0bUyxoNr68+vx48kf1W6b1mkaQrxCjD+J/Am+pdfVAPRL7NbG3EnY8oBAjOBJzgKLCZpLiFcI1g1cByep+n9UY/4ag9/P/m7w8U3xIMBeMQAhIiKbdvXqVfTs2RNBQUHo3r07Lly4oLLPvn37UKdOHXTs2FHx9fjxY8X2LVu2oE2bNggMDMTw4cORl6dmlWcLCQ8Xcj3UkUph9IJqJW+arudex9tJb6vdt51nO3zS5RPjTgio5jCMGydMf5IHAcWrNpmDPpWe5EqOOFgqD0Ks4Amw+GKCejFw9Ozl+i9j+SvLIVGzrKIMMry27jVsvbJV524YXaGujGIAQkRENm38+PEYMWIEjh8/jnHjxiE6Olrtfo0aNcL+/fsVXxUqVAAA5OXlITo6GitXrsTJkydRu3ZtzJ8/35xvQatZs4RqVyWDEHkVrJkz9T+mppumnAdFGPHnCNx7fE/lNZXLVcaKV1bAUepo4Dv5j64lb80ZhJQsUyxXMsioXFn4rkcZY5MzJHjS5OrV56NR1szI0bNBzQdhSe8larflF+Wj3+/9cCj9UKndkFeoi48X8rQyM4Xv8fFCO4MQzRiAEBGRzcrKysLp06cxaNAgAEBERARSU1ORmpqq8zF27NiBwMBANGzYEADw1ltvYe3atSbpryFcXIRSu1FRQsK5h4fwPSpKzxK8/9F209RoxELsuLZD7esWhy2GfzUR8gw2b9a9epOmfART5I7oslbK2bNCzoV8lCAmRvjZkhWgdA2edGXOPBZDiDR69k7wO5jVfZbabY/yH6HXb71w5tYZrceYOlW1PDYg/Hz+PDBtmk7vqExiAEJERDYrMzMTderUgaOj8FReIpHA09MTGRkZKvteuXIFnTt3Rrdu3fDjj8/zG9LT0+Hl5aX42dvbGzdu3ECRurq3FuLiItwHJycD6enC99hY/YMPQMtNU81TuNVcfRWggc0G4nXfSHFu+gcP1r1usLp8BFPmjhRP7FYXZHh7W2cSdmnBU3Aw0KaNbseyosUEVYg8evZRx48wsd1Etduyn2Sj54qeGtcPAYCkJPXlsQGhPTFR6+nLNAYgRERk00rW7pepyQRt2bIlzp49i71792LFihVYtmwZ1q9fr/EY1szYrqq9aXJ6BPQfCjg+U9nfs4onvuv0FSRhYeLd9MfElP6EXl0+gjlyR2y10pO24GnHDmD79tJHSax8PQ9RRs+KkUgkmNdzHt5s9aba7TfzbiJ8VTieFDxR2WbJCnX2gAEIERHZLA8PD1y/fh0FBcJCeTKZDJmZmfD09FTar0qVKnD976bKw8MDAwYMwMGDBwEAXl5eSEtLU+yblpaGOnXq2OX6UBpvmnpMAdzPqzRLIMHyHktQte9gcW/6danaVDIfwRpzR6yNtuBJlylm1hx8AMaPnqkhkUjwQ/gP6Nu4r9rt57LO4ZOdqoUXLFWhzl7Y3/+uRERUZri7uyMgIACrV68GACQmJsLb2xs+Pj5K+928eVMxpSo3Nxdbt25FixYtAAAvvvgiTpw4gUuXLgEAfvrpJ/Tv39+M78J81N40+e4CXohXu/8HwePRddQscW/69anaVDwfQeSn32VSaVPMrDn4kDN09EwLR6kjVvVfhe5+3dVun39oPg5nqP7umbpCnT1jAEJERDZt4cKF+PnnnxEUFIQFCxZg0aJFAIDo6Ghs2rQJgBCYhISEoEOHDujRowe6du2K1157DYCwonlcXByGDRuGwMBAXL9+HRMnqp8Xbg9UbppC5qndL6hOED5/0Fr8m359qjYVz0cwwdPvMslWp5jJGTJ6pgNnR2f8OehPtKrdSmWbDDJ8uGUSZKtWKbXPClyLJg0LRa1QV1ZIsrOzy/zsNBcXF7scaici0lVRURFyWTNSL7Z67ZBXwTp/HiiqeBOY6AlIC5X2qehUESdGn0CjGo2EGzldRixiY3V/6qxuOpW/v/KNpaYpQfXra78B9fcXphaR/UlIEPKOdLVqld6B6NnbZxH0QxCeFj5V2bb9V+Clcf/9nv/37yK3TXdMa70JiVvLIz9fGGGMiBCCD0OKRNgSY64btvc/JxERERmseFnfah3WqwQfAPBBhw+E4AMwyZQXg/MRTPT0m2yEoaNnemhWsxmmdVZfP3dqd0BWIgfK5ehOxJ7uiuRTOUZXqCtLGIAQERGVMfKyvh3f2KJ2+4iWI57/YKqbfn3zEQzNHSH7oeuaJ0Ym1I9vNx41K9VUaT/iCSQ1gsYcKMmDMlj4wEAMQIiIiMqgZ4XPsDN5p0p76zqt4eP2XxK/qW/69clHMMPTb7IBZqjmVblcZXzUUf2aOJ90A4rUVbZi4QO9OBrz4oKCAly5cgVZWVkqCzZ16dLFqI4RERGR6RxKP4S8Z3kq7S/7v/z8B/lNvy6J6Ka+6ZffeBqaO0L2Q/67sHnz8wA2JgaoWVP4HRTh7/7d4Hcx7+A8ZOZmKrX/UxtY0xQYdLbEC1j4QC8Gj4Ds3bsXLVq0QPv27REREYG+ffsqvl555RUx+0hEREQi23p1q9p2pQDETFNedGaLa1nk5IizgjwpM3E1L2dHZ3zSWXX9DwD4tBtQUPwOWt8cKDI8AJk8eTJ69uyJCxcu4P79+0pf9+7dE7OPRERkIx4/fozp06cjMDAQNWvWRLVq1ZS+yHpsuaKa/+FSzgXtvdorN1rbTb8trWUhr/Yl1gryZFYjA0einrS6SvulGsDyFsUaWPhAbwaX4fXw8MD+/fvh5+cndp/MzlZLKRIRiUWsMryTJk3C/v37MWXKFIwdOxZz587FzZs38csvv+Cjjz5SrL1hD2z52nEr7xZqz6+t0h7ZKBJ/Dv5T/YtycpSnvADCk3yRprzYHWNKDZN1SEjA8tlD8Ho/1U0+2cClRUC54kXkDCj7a8ssUoa3R48eOHr0qKEvJyIiO/TXX39hwYIF6NevHxwdHdGhQwdMmTIFM2bMwJo1ayzdPfrP9mvb1bYrTb8qydYXsDMndcEHYNwK8mR+YWEYWrEtmmSpbkp1A35sXayBhQ/0YnASenBwMD755BMcPXoUTZs2haOj8qGGDx9udOeIiMi25OXlwcPDAwDg6uqKrKws+Pv7o3Xr1oiKirJw70hOY/5HfS0BCOlu82b9V5AvQ0/ObYarKxy2bMUXI9rgVffLKptndgbeOAVUDOJIlr4MDkB++OEHlC9fHlu3bsXWrcr/kUkkEgYgRERlUOPGjXH+/Hl4e3ujVatW+Pbbb1G5cmX88ssvqFu3rqW7RwCKZEXYekU1AGlQrQHqVa1ngR7ZocGDgdu3dV9BnsGH9XJ1Rb+f/0bgF5446fpIadMNF+Db/t6Y9B2DD30ZnANiT2x5Hi8RkRjEygHZvHkznj17hsjISFy8eBFDhgxBcnIy3NzcsHTpUrz00ksi9NY62Oq148SNEwj6IUilPapNFBb1WmSBHtmx+vW1L+Lo7y8k9JPV++vU7+izYZBKe3XnakgenwKX8mVv6XOL5IAUl52djezsbDEORURENiwsLAyRkZEAgEaNGuHEiRO4evUqrl69alfBhy1TN/oB2Pn0K0uUwjXVCvJkEb1avor2nu1V2u8+uYeFhxearR8yOxk2MDgAKSoqwoIFC9CgQQPUq1cP9erVQ8OGDbFw4UKVRQmJiKhsCA8PV3kgVa1aNeTl5SE8PNwynSIl6vI/yjmUQ1ffrubvjDlYohSuqVeQJ7OTSCSY1X2W2m3zDs3DvcemW4IiN1eoNO3nB3h5Cd9jYoR2W2VwADJ9+nQsWbIEH3/8Mfbu3Yu9e/fiww8/xLfffosvvvhCzD4SEZGN2L9/P/Lz81Xanzx5gr///tsCPaLiHjx9gAPpB1TaO3p3ROVylS3QIxMrWY1q3DhhWpQ8ODBVFSr5CvK6YPUkm9HNrxu6+3VXaX/w9AHmHZxnknPm5gLt2wPx8UBKCpCZKXyPjxfabTUIMTgJPSEhAUuWLMGLL76oaGvevDm8vLwwduxYTJ8+XYz+ERGRDVi1apXiz+vXr4eLy/P50IWFhTh48KBdrBtl6/al7kNBUYFKu9byu7ZK31K4YlYxki+WyHVA7M6s7rPQ/ifVqViLjizCBx0+gKuzuH+XU6cC588DJScXFRUJ7dOmPV8T1JYYHIDk5uYqSi0W5+npiby8PKM6RUREtmXmzJmKPy9YsEApOdvR0RFeXl6YP3++JbpGxRYQvHT3ktpdetTrYeZOmYGlS+GWDEJiY4V5M3FxwggMgw+b1M6zHfo07IONlzYqtec9y8Oyk8swof0EUc+XlKQafMgVFQGJiWUsAGnXrh2++OILLFmyBK7//ePJzs7GjBkz0E7XYUciIrILZ8+eBQD06dMHK1asgJubm2U7RILiowC3b+NW01tqd2tQvYGZO2YG1lAKVx6EFA9uYmKAmjW5grwN+7zr5yoBCCCMgsS0jYGD1EGU88hkgJoZrUry84X9JBJRTmk2BpfhvXbtGoYMGYL09HTUq1dP0ebj44PffvvNpobabbWUIhGRWMQqw1uWWP21Q80UpDeGu+AXf+W/54qOFfFw6kNz9858WAqXTKDrz12xJ3WPSvv6QevRt3Ff0c7j5yfkfGji6wskJ4t2Or0Yc90weASkXr16OHToEHbu3IkrV65AJpOhYcOG6N69OyS2FoYREZFozp8/j40bNyIzM1MlIT0+Pt5CvSpjNOQ/3CpSvVmo9aBQ2N+ST+OLTRNTSEgwfpRAn1K4MTGGn4fKnPHtxqsNQBYeXihqABIeLiScq5uGJZUCERGincqsjHp0I5VK8dJLL+Hdd9/FmDFj8OKLLzL4ICIqw9avX4+uXbvi77//xm+//Ybbt2/j77//RlJSktrqWGWN2Wr4a8h/uKWm0FXtu0+F/S3FVGVyWQqXTCi8YTj83FRn++xJ3YNTN0+Jdp5Zs4AmTYRgozipVGgvln5nU/SagjVr1ixMmDABFStWxKxZ6mshy02dOtXozpmL1Q+jExGZmFhTsEJCQjB69Gi88cYb8PT0xP79++Hj44MpU6bAxcUFn376qQi9tQ66Xjtyc4VKNklJwnxtJyfhqeasWYCLKRdPlic7F+MxEbheRXm3vk4BWP/xPybsiBbqRmrEqhSlqQqWOkwIJwMsOLQAE7dNVGl/o9Ub+L/I/xPtPLm5QrWrxMTn/4dERAjBh0n/DymFMdcNvQKQ4smFffr00XxQiQRJSUkGdcgSGIAQUVknVgBSt25dHDp0CD4+PvD398eGDRvQvHlzXLlyBS+//DKuljYdxobocu2Q1/AvWUZT/vTy0CET30AUy38okgDlpwEFJfJj3wl6B9/1+c6EndDAHAGCKQMcKvNynuTAc4En8p4pV38t51AOaePTUKtyLdHPaU0J58ZcN/S66964caOissnGjRs1ftlS8EFEROKpWbMm7t+/DwDw9vbGwYMHAQBXr15FkaZaknZMlxr+2hg1ZatE/sO9CqrBBwDUOptixEmMYEiZXH3Jq1DJq3PGxgoJ5/K6pQw+yAiuzq4Y2WqkSvuzwmf4/vj3JjmntQQfxjL4sf/YsWPVRj0PHz7E2LFjjeoUERHZprCwMOzYsQMAMGrUKEybNg2dO3fGm2++iUGDBlm4d+anSw3/knJzhXxoPz/Ay0v4HhOj54rHavIfblVSv2utP7ZaJv9h8GDdFzAwpkyuPAhZtep5onlMjPAzgw8yUnTbaEigGhV8e/RbPC14aoEe2QaDy/BWq1YNFy9ehLu7u1L7nTt30LhxY9y5c0eUDpoDp2ARUVlnqjK8Bw8exPHjx+Hn56d16q4tKu3aIZMJAURmpuZjeHgA6enPn2qKNmVLzdSjnX7AiyNUd117sgH6/3LUcjfiLJNLNi5iVQSSLqnO/vml7y94veXrFuiReZi1DO+BAwcAADKZDEeOHFFabKqwsBB79+5F3bp1DeoMERHZl5CQEISEhFi6GxYhkQjJoto4OSlPqdBlypZOgwYlV+GG5hGQ2rMXWS74YJlcEpOpyjmXYny78WoDkIWHF2J4i+FKFWKtKYfDkvQOQORPsCQSCV577TWlbVKpFB4eHpgxY4Y4vSMiIqs3d+5cnff94IMPTNgT66NvDX9dpmzpOmupZBByc3hf4PGfKrvVquWv4wFFpm+Z3Jo1TbNaOdmH4qN+t28LAau8EpyJc326+XZDQM0AnLl9Rqn95M2T2J+2H62qdbJMJTwrpvcUrMLCQshkMrRu3Ro7duxA9erVFdscHMRZet7cOAWLiMo6Y4bSw8LClH7+999/UVhYCD8/oUZ+cnIyHB0d0axZM2y25HoTIhO7CpYhU7Z08t9T4Q9rnMLcA6rB4oMPH8ClvAXuglgml8RiBdXOfjrxE95OelulPaJ+P1yd/YflKuGZkNmqYAFCkOHo6Ih//vkHNWvWhIODg+KLiIjKns2bNyu+wsLC0LVrV5w7dw4HDhzAgQMHcO7cOXTr1g0vv/yypbtqdi4uwg1GVBTg6ysEEL6+ws8lbzwMmbKlE1dXYPBg3Hp4S2VTBccKqFxOzeqE5lCyQpWcf4kRGQYfpI2mQLbk1L7Dh41b2LIUQwOGokbFGirtSZf/xLnrKQZXwrNXBj/2nzx5MpYuXarSvnTp0jI3xE5ERIJFixZh2rRpSvmBbm5u+Pjjj7F48WLLdcyCXFyEaVPJycLoRXKy8LO6p57h4aorHsupm7Klj1t5qgFIrcq1lOanmx3L5JKxzFHOWQcVnCrg3aB3VdplkiLIWqpflFBTJbyywOAAJDExEW3btlVpb9euHf78809j+kRERDYqPz8fFy9eVGm/cOECCgoKTHLOq1evomfPnggKCkL37t1x4cIFlX327NmDF198EW3btkX79u0xY8YMyP5bZCM1NRXVq1dHx44dFV/Jyckm6Wtp9/qzZgnTMkoGIfLpGjNnGn7um3k3VdpqV65t+AHFwjK5ZAxzlXPWwZg2Y+AoVZNe3WSdxtfk5xu53o+N0jsJXS4nJwcVKlRQaXd2dkaOiYa3iIjIur311luIiorC6dOnERgYCIlEghMnTmDp0qUYNWqUSc45fvx4jBgxAsOGDcOGDRsQHR2N7du3K+3j5uaGn376Cb6+vnjy5An69u2LtWvX4tVXXwUAuLq6Yv/+/Sbpnz7kU7amTROejMoTViMihODDmLni2U+yVdqqV6iuuqMl/DdNTAkTzklX8oTz0so5m7iSWl2XuujdoDc2XNygvKHWv0Cl28DDmiqvMWhapR0weASkefPm+P3331XaV69ejSZNmhjVKSIisk2ffPIJ5syZgz179mDs2LF47733sHv3bsyePRuffPKJ6OfLysrC6dOnFYscRkREIDU1FampqUr7tWzZEr6+vgCEB2UBAQFISUkRvT9i0GfKlj6cHZ1V2p4VPjPuoETWQJ9yzibWu0Fv9Ru8Dqo0GTut0pYZPAIydepUDBo0CGfOnEHHjh0BAPv378euXbuQYIkVVYmIyCoMHToUQ4cONcu5MjMzUadOHTg6CpcziUQCT09PZGRkwMfHR+1rbt26hQ0bNig9RMvNzUW3bt1QWFiI3r17Y9KkSVZRXEXMJ6Pqks3znuWJd4LiLLQeA5VBVlbOuaN3R/UbfPYDF/oqfhRjWqUtM3gEpFu3bti7dy+qVq2KtWvXYs2aNahatSr27t2Lbt26idlHIiIijUomUcu0TKh+8OABBg8ejJiYGLRq1QoAULt2bZw7dw67du3Chg0bcOjQIbtMmDc4AMnJEW7yiktI0FxNSF6VaMiQ50+c4+KEn01YhYjKqLAw1UpqmrRrJ+xvQo1qNEK1CtVU2mu3OVBqJbyyxOAREABo3LgxlixZIlZfiIiI9OLh4YHr16+joKAAjo6OkMlkyMzMhKenp8q+ubm5GDBgAMLCwhAVFaVoL1++PNzd3QEAVatWxWuvvYY1a9ZgnK5PVW2EurU+Sg1A9F3crWRJ1HHjlKfHyEuhMrmcxFJiwU0FM68DIieVSNHBq4PKyuh3yx1HzqXHcHasUCZzPkoyavW9goICXLhwAfv27cOePXuUvoiIiEzN3d0dAQEBWL16NQChQqO3t7fK9Ku8vDwMGDAA3bt3x5QpU5S2ZWVlIT8/HwDw9OlTJCUloUWLFuZ5A2ak9wiIumCifv3n011KrqtgJesxUBlkZeWcO3h1UGnLL8rH0etHGXz8x+ARkL179+Ldd9/FjRs3VLZJJBLcu3fPqI4RERHpYuHChXjvvffwzTffwMXFRTEyHx0djbCwMPTq1Qvfffcdjh8/jocPH2Ljxo0AgL59+2LSpEk4dOgQZs+eDalUisLCQnTq1AmTJk2y5FsyicpOqgFI7jMNqxjrG0xs2WLYegysdEVikQchxX+vYmKEnA8z5x518FYNQADgQNoBdPbpbLZ+WDNJdna2QdWH5bXUP/roI9SqVUvsfpmVi4sLpJpWfiIiKgOKioqQm6vhZpTUsrVrx/tb38c3h79Rac//JF917YKEBCFnQ1erVgk3ffLpWaWJjTV5SVQiS3lS8ASuc1xVqsz1btAbG4dutFCvxGfMdcPgEZCMjAyMGzfO5oMPIiIST58+fdSurC2RSFC+fHn4+vpi0KBBCA4OtkDvyjZ1U7AA4OGzh3B1LvF0ePBgIedD12Ci+BNnK1iPQQWrcpEZOTs6I7huMA6mK5fePZh+EEWyIkgltvPgwlQM/gR69OiBo0ePitkXIiKycYGBgTh9+jSePXuGpk2bomnTpsjPz8epU6fg6+uLy5cv4+WXX8amTZss3dUyR1MAojEPJCZGCBa0KRlMWNF6DAqsykUWoC4P5P6T+zifdd4CvbE+Bo+ABAcH45NPPsHRo0fRtGlTRQ12ueHDhxvdOSIisi137tzBuHHj8P777yu1L1iwAJcuXcKff/6JefPmYc6cOejVq5eFelk2qauCBWgJQPQJJmJirG49BgCsykUW09G7I74++LVK+4H0A2hWs5kFemRdDM4B0VYhRCKR4PTp0wZ3ytxsbR4vEZHYxMoB8fT0xJ49e+Bf4sn51atX0aVLF2RkZODq1avo3LkzMjMzjT6fJdnatWPFPyswfL3qw8Fjo44hqG6QcqMhOSBhYeoT19UxR1UiTYn0luoPWZ4Zp+LdeXQH7l+7q7S/3vJ1/NL3F1HPZSnGXDcM/p/zn3/+0fhlS8EHERGJx83NTVFlqriNGzfCzc0NAPDo0SNUrqx+OhCZjl5TsAxZ3K1kKVS5ktO4zHWzb0hVLrJfZp6KV6NiDTSq3kilfX/aflHPY6ts59ENERFZvS+++AIzZsxAWFgYPvroI3z88cfo1asXZsyYgS+++AIAcPjwYYSZeDViUqVXAGJoMGFN6zEMHvz8vKUpnkhP9kffNW1E0tG7o0rbtfvXcDPvpqjnsUUGT8F65513tG7//vvvDeqQJdjaMDoRkdjELMN79epV/Prrr7hy5QpkMhkaNGiAESNGoF69eqIc31rY2rXjcMZhtP+pvUr7qv6rMLi5hpvv4jdu8tK52lZCL/46a6k6Vb9+6VW5rlwxX3/IvCw4Fe//Tv4fRiaOVGlf++pa9G/aX5RzWJJFyvA6ODgo/VxQUICzZ88iPT0dffr0MfSwRERk4/z9/fH5559buhtUgt5VsADDF3dzdVUdUbDECIO+ifRkfyy4QKamBQn3p+23iwDEGAYHIN9++63a9hkzZkAmM2hQhYiI7MDdu3dx/PhxZGVloaioSGkbKyRajks59VWw7j++r/2F1hJM6Msaq3KR+Rm6po0IGlRrAPeK7sh6lKXUfuT6EdHOYasMnoKlybVr19C9e3ekpKSIeViTsrVhdCIisYk1BWvdunUYO3YspFIpqlWrprQooa1VSCyNrV07Huc/RqUvK0EG5cv+6Naj8X247Uyb1hmrYFFxFpqKF74qHBsvKRfmqOpcFXen3FW7aKstsUgVLE12796NihUrin1YIiKyAdOnT8e4ceOQlpaGM2fOsEKiFangVAG+br4q7efv2OnCaNZWlYssx4ILZDat0VSl7f6T+7j18Jbo57IlBk/BUlfB5NatW0hJScGXX35pVKeIiMg23b9/H4MHD1bJEyTr0MS9CZKzk5Xa7DYAAZ4HIYYk0pN9sPBUvCbuTdS2n886j9qVa4t2HltjcADSpUsXpZ+lUilq1KiB9u3bo0kT9R82ERHZtwEDBmDLli149913Ld0VUqNx9cbYdHmTUtudR3dw59Ed1KhYw0K9MjFDE+nJPsjXtNF1Kp7IJcKb1NAQgNw5j25+3UQ9ly3RKwAZO3Ys5syZAxcXF/j4+KBfv34oX768qfpGREQ2pkqVKpg9ezZ27dqFpk2bwtFR+TIzdepUC/WMAM1PYy/cuaB2zQK7YauJ9GS8kqNgcv7+ytOyTDQapunf3Lmsc6Kex1gyGWDOlBS9ckDWrFmDhw8fAhCCkQcPHpikU0REZJuOHz+OgIAAPHz4EEePHsWhQ4cUX4d1LYVJJtO4RmO17eez7HgaFpEFF8isUr4KPFw8VNqtYepjbq4wGOjnB3h5Cd9jYoR2U9NrBKRBgwaYPn06OnfuDJlMhvXr18PFRX1ZvyFDhojSQSIish0bN24sfSeyGE3TQS7cuWDmnhCZmQWn4jV1b4rM3EylNjGDfkNGL3JzgfbtgfPngeLV0uPjgZ07gUOHAA23+KLQqwzviRMnMH36dKSkpCAjIwO1atVSW4JQIpHg33//FbWjpmRrpRSJiMQm5kroZYWtXjvcv3bHnUd3lNrC6odh07BNGl5BRMYYt3kc4o6oVti6/8F9uDm7GXTM3Fxg6lQgKQnIzwecnIDwcGDWLN0Ch5gYIdgosVQTAEAqBaKing8QaWK2ldBbt26NxMREAEDVqlWxd+9euLu7G3RiIiKyD2FhYVi1ahXc3NzUVkgsbvPmzWbqFWnSpEYT7Evbp9RmDdNBiOyVtkpY7b3a6308MUYvkpLUBx+A0J6YWHoAYgyDq2Ddv1/KyqlERFQmdOnSBeXKlVP8maybugAkNTsVj/IfoaIT1/EiEltTd9W1QAAh8DckAJk6VTX4AISfz58Hpk3THjzIZMKoiTb5+aZNTDc4ACEiIgKADz/8UO2fyTqpS0SXQYZLdy+hVe1W5u8QkZ3TlHtlaCUsY0cvJBJhypY2Tk6mrYple5NXiYjI6j169AipqalISUlR+iLL0zYdhIjE517JHdUrVFdpN2Tqoz6jF9qEhwu5HupIpUBEhN5d0wtHQIiISDRnz55FVFQUTp8+DQCQyWSQSCSK7/fu3bNwD0lTKV5WwiIynabuTVVzrwwI+sUavZg1S8gXKTmVSyoFmjQBZs7Uu2t6MXgEJD09HTI14ZVMJkN6erpRnSIiIts0duxY1KpVC1u3bsXJkydx+vRpnDp1SvGdxFfak86SvF29UcGxgkq7WRLRc3KAhATltoQEoZ3IjqmbhpWSnYJH+Y/0PpYYoxcuLkKyelQU4OsLeHgI36OiTF+CFzAiAGnZsiXu3Lmj0n7//n20bNnSqE4REZFtunTpEr788ku0adMGPj4+8Pb2VvoicRizgJhUIkWjGo1U2k0egOTkCKtRDxkCxP1XkjQuTvg5NFS3IIQBDNkodVMfZZDh4p2Leh9r1ixhlKJkEKLv6IWLi5ArkpwMpKcL32NjTR98AEYEIPLh9JLu3r2LSpUqGdUpIiKyTS+88AIuXbpk6W7YNXkJzvh4ICUFyMwUvsfHC+26BCHqnsZeunsJTwueit5fAM+Dj8OHhZ/HjQPq1xe+A0J7aUGIGAEMkYVoq4SlL1OMXpgy4VwdvXNA+vTpA4lEAolEgtdeew1OxSaiFRYW4sqVK+jQoYOonSQiItswcOBAfPTRR7h8+TKaNm0KR0flywzL9BrP2BKcANDMvZlK27PCZziccRhdfEX+OyoZfMhdvar8szwI2bJFdVVqdQFMXNzzY2h7LVm/nBzlFcoBYWTLxCuUm1PD6g3Vtmc+yFTbXhr56EVsrGnL5ZqK3gFIu3btAAD79+9H69atUbHi85rhTk5OeO211xAZGSleDw2QnZ2NadOmYdeuXTh79qxF+0JEVJaMHTsWAPDpp5+qbGMSujjEWECsc43Watt3XtgkfgCyebNq8KHJ4cOqN6JiBDBkvYr//d6+LcwljIsTgsx27ezm77N25dpq22/m3TToeMWDDlsLPgBAkp2drWf6muC3335D//79Ub58eVE6MmXKFGzevBnp6ek4ePAgmjZ9PlR19epVjBkzBnfv3oWrqyu+/fZbNG6svopHcZGRkdiwYUOp+7m4uECqKZuHiKgMKCoqQq4uc3dIwRLXDplMyPnI1PLQ1MNDmM+t8aYkJwfPwnqiarcjeFROeVOH+y7Y/3m6+Dd88hvK0sTGCjegxSUkCNOsdLVqlXIAQ9ZLXXDp768cXNpREFJ1blVkP8lWahsaMBQr+63U6fW5ucIIaFKSUGrXyUlISJ81yzx5GyUZc90w+H/OkJAQ3Lz5PGo7fPgwJk+ejB9++EFtdazSREZGYsuWLfDy8lLZNn78eIwYMQLHjx/HuHHjEB0dDQC4ffs2IiMjlb7UPXUjIiKyB0aX4Pzvhq/coSPolKa6+e8qucjr3UP8fIqYGOHGUht/f9XgAxCCidKGdORiYxl82Ap9R7bsIMdH3SiIriMgYuR+WROD1wEZNWoU3nnnHfj4+CAzMxMDBgxASEgINm3ahBs3buCzzz7T63ia8kaysrJw+vRprF+/HgAQERGByZMnIzU1FT4+PjqNcBARkenMmjULEyZMQMWKFTFr1iyt+06dOlX08+s6Sv7rr79i4cKFKCoqQpcuXTB//nxFjsqWLVvwySefoKCgAM2bN8eSJUtQuXJl0fsqhvBw4aZD3TQsrSU4S9zwvXgN2FpfeZcCB2DfraMIE3sqU/F8DU2uXhX2UxeEyKflaDuGpgCGrJOxU/NsUO3KtVXW29E1ABEj98uaGDwCcvHiRbRuLcwhXbduHYKDg/H7779j6dKlWLNmjWgdzMzMRJ06dRQXCYlEAk9PT2RkZGh93YQJE3D58mVMmDABycnJovWHiIiUHTp0CM+ePVP8WdPXYV1vNvSkaZS8uJSUFHz55ZfYsmULTp48iVu3bmH58uUAgLy8PERHR2PlypU4efIkateujfnz55ukr2IwuARniRu+7houjTv98PyGTwwJCbpNvwKE/UqW2QX0C2CKY9le61UGR7aMGQHRJffLloiyEvru3bsRFhYGAPD09MTdu3fFOKxCyXK/ukzxWrBggah9ICIi9TZu3Kj2z+ZQ2ii5XGJiIvr06YOaNWsCAEaOHInY2Fi8+eab2LFjBwIDA9GwoVCl5q233sKrr76q90i+uchLcE6bJtx0yOeCR0QIwYfGueCDBwtJvv8FA61uAlUfA/dLrEn4v3oQ94YvLEyYx69LANqunbB/cfoGMDVrCn0vI8nNNq2MjWzVrqQagNx7fA9PC56ivKPmnGqZTPh3rk1+vm1VwzJ4BOSFF17AvHnz8Pvvv2P//v0IDQ0FACQnJ6N2bfWZ/obw8PDA9evXUVBQAEAIPjIzM+Hp6SnaOYiIyDbpOkqenp6ulGPo7e2t2Efdths3bqBI0+NGK2DwAmLFcjEcZEDXFNVdTtUGbo0cJF5nXV2Fm/3/qmgqlMwJ0RQUyAMYXcgDGDHWHSHTM3Rky0ZpqoR16+Etra8zOvfLChkcgMybNw937tzBwoULMXv2bMWTpq1bt+Kll14SrYPu7u4ICAjA6tWrAQhPsby9vZWebBERkfXYsWMHoqOj0a9fP4SHhyt9mYKuo+TF9yu5j7qFdW2FXl0vccP34jXVXWQS4M8lIj9xLhmExMYCV648n4KjbURC3wAGKHPJzTZJjKl5RjCgXpLRjCnFGx6uOu1STmvul5UyOADx9fXF77//joMHD2LkyJGK9i+//BJff/213sebNGkSmjZtiuvXr6Nv374IDAxUbFu4cCF+/vlnBAUFYcGCBVi0aJGh3SYiIhP6/vvv8eabb8LJyQn79++Hn58fpFIpTp8+rcgbFJOuo+ReXl5IS3te9ik9PV2xT8ltaWlpqFOnjv2VZ1dzw9dHw6L1f5z5XfQbPkUgsWrV8yk1MTHCz6VNh9IngDEkuZnMz5CRLSPl5gq/cn5+QjlrPz/hZ3NVkDImADE498tKGbwOCCAk7q1evRqXL18GADRs2BADBw602sohmnAdECIq68RaByQ4OBiffPIJIiMj4enpiX379sHPzw9ff/01MjIyEGuCMi29e/fG0KFDMWzYMGzYsAGLFi3Cjh07lPZJSUlBaGgo9u7dC3d3dwwZMgQ9e/bEyJEjkZubi8DAQGzatAkNGzbE5MmTUalSJUyfPl3reW3u2qGh7GmbUcAxD+VdHYskuPXeNVSr5Wu+/ulC1xWzjVl3hMzHjOuAyMvYlqwkJb+BP3TI9GtpnL55Gq2+b6XS/n2f7zE6aHSpr8/NNSD3y4Qssg7IiRMn0LJlSyxYsAAZGRnIyMjAN998g1atWuHUqVOGHpaIiGzY9evXFSPYFStWRM5/01sGDBigSBQXm6ZR8ujoaGzatAmAMGr/0Ucf4eWXX0arVq3g7u6O4cOHAxACibi4OAwbNgyBgYG4fv06Jk6caJK+WpSGqUz9b1VT2bVAKkPijd1m6pgeXF1Vk+MHD1a9OTVm3REyH2Om5ulJlzK2plbHpY7adl0rYRmc+2WFDB4B6dGjB5o1a4b58+fDwcEBAFBYWIiJEyfi/Pnz2LZtm6gdNSWbe4pFRCQysUZA2rZti0WLFuGFF15AWFgYunTpgg8//BC//fYbpk+fjkuXNMz5sUE2e+0o/tQ5NhaXhoWi0eJGKrv1adgHSUOSLNBBEXAExLboOrJlBD8/YeE+TXx9hRt6UyosKkT5meVRKCtUah8TPAbf9v7WtCc3AYuMgPzzzz8YO3asIvgAAAcHB0RFReGff/4x9LBERGTDBg8ejGPHjgEQ1udYsGABfH19ER0djffee8/CvSMAKrkYDas3RPOazVV223Z1Gx48fWCBDupObSKxhZObyQC6jmwZSJ8ytqbkIHVAzUo1Vdp1HQGxJwYHIO7u7moDjdOnT6NGjRpGdYqIiGzThAkTFIHGyy+/jL///huxsbHYtWsXxo8fb9nOkYKsivINX/8m/VX2eVb4DH9d+suc3dJJqYnEFkhuJutmTWVsjVmM0J4YHICMHj0aMTEx+Pzzz7Fx40Zs3LgR06dPx4QJE/DOO++I2UciIrIB+fn56Ny5s6IwCSDkXkRGRqJFixYW7BkB2m/c1QUgAPDbv7+ZuZfayROJ4+OF6TSZmcL3+HihPTcXxq87QnbJWsrYMgARGByAxMTEYOHChdi/fz+ioqIQFRWFAwcOYOHChYiOjhazj0REZAOcnJyQlZVl1Qv42TpDp4iUduPuU6E5GlRroPK6vy79hav3Slkozox0TiQ2Y3Iz2QZrKWPrXsldpe3+k/sWWZfEkozKnnv11Vexfft2pKSkICUlBdu3b0f//uqfohARkf2LiorC119/jcePH1u6K3ZDjLULSrtx/+QTCYYGDFV5nQwyLDpiPWtvJSWpvge5oiKhPKmCMeuOkN1xcRFK7UZFCQnnHh7C96go85TglSsnLafSlpNbYJF1SSxJ7ypY169fx5IlSzB58mRUqVJFaduDBw/w9ddfY+zYsahdW/1iK9bIZiuZEBGJRKwqWGFhYThz5gykUinq1auHChUqKG3fbEeLvpnj2iHW2gW6VAA6dOYmvBd4I79IOVvXpZwLMiZmoEr5KupfbCYymRCAZWZq3sfDQyhPasML25OZyGSW+T0Zs3EMvjv+nXJjvjMwS3hoY851SYxlzHXDUd8XxMbGoqioSCX4AIAqVarg6dOnWLhwIebMmWNQh4iIyHZ16dIFXbp0sXQ37IYuU45KW9tR1wpAtSrVxpCAIfj19K9K23Kf5WLZyWUY3268/m9ARNaUSEy2z1K/J45SNbfe0gLFH/X5t23L9A5Adu7ciSVLlmjcPmjQICahExGVUR9++KGlu2BXdJlyVNpNij437uPajlMJQAAg7u84RL8QDQepg5pXm094uJC3ou4zMWciMZGh1AYgDgUAZACEqEjXf9u2TO+x4/T0dNStW1fjdnd3d2RqGx8lIiK71bJlS9y7d0+lPTs7Gy1btrRAj2yXmGsX6FoBqHWd1ujs01lln+TsZCRdsvyihNaSSExkKLUBCABIlKNqc6xLYkl6ByBVq1ZFenq6xu3Xrl2Dm5ubMX0iIiIblZaWhsLCQpX2x48f4+bNsldq0hhiTjnS58Z9fNvxao+x8PDC0k9kYtaSSExkKI0BSLFpWID9TyfUewpWt27dsHDhQqxatUrt9oULF6Jbt25Gd4yIiGzH3LlzAQASiQSLFi1CpUqVFNsKCwtx7NgxNGnSxFLds1liTTmS37hPmyZM7cjPF25wIiKE4KP4jXtEowj4uvkiJTtF6Rh7Uvfg5I2TCKwTaPgbEoGLizA1JTbWconERIbSGoAUlhf+WAamE+pdBSstLQ3dunWDv78/xo4dC///Fva5cuUKlixZgsuXL2PXrl3w8fExSYdNgVWwiKisM7YKVth/q0kfPnwYQUFBcCr26N7R0RFeXl6Ijo62qyDElqpglVTajfuCQwswcdtElfYRLUfg574/G3VsIovLyQE2bwYGD37elpAAhIWZvDzz57s/x/Q901U3zM4GnrqWmSpYegcgAHD58mVMmjQJ+/btU2rv1KkTvv76azRs2NCgzlgKAxAiKuvEKsP73nvvYc6cOWorJdobc107cnN1G7kQU86THHgu8ETeszyl9nIO5ZA6PlVlNefcXKFiV1LS8z6GhwtTv6z9JorKmJwcIDQUOHxYGEaLiQHi4oBx48yyQOWsvbMwbdc0lfbav96Bc1F1k//bFpPZAxC5e/fuITk5GQDg5+eHatWqGXooi2IAQkRlnVgBSFliiWuHOUcXxm0eh7gjcSrtn3X5DNO7Tlf8bKpRGiLRFQ8+5Pz9gatXn/9s4iBk7v65+PB/qtUCb0y8idoutUxyTlMx5rph1P+c1apVQ1BQEIKCgmw2+CAiIrIV5pzaFN02GhKonnDJsSV4UvBE8bMua5WQjnJyhKlAxSUkCO1kHHXBB6AcfADC9tBQk33mmnJACmUFatvtFR/7ExERkYr61eojvFG4Svvth7eR8O/zm2Rd1iohHchvkIcMEaYEAcL3IUNMekNcZmzerBp8aHL4sLC/CWgKQAqKGIAQERERaS3JK5PJRF2rpEwr+XR+3Digfn3hO2Dyp/JlwuDBuq/sFxurnKAuIgYgAgYgREREpFZX365oUauFSvvpW6exJ3WPqGuVlFlWMjWoTIiJEXI+tPH3F/YzEQYgAgYgREREpJZEItE4CjJn/xwAuq+ybpfEyNmwkqlBVsOUeTBxcaqBXUlXrz6fAmeC/jEAETAAISIiIo2GBAyBe0V3lfatV7di29Vteq2ybhRrS9AWK2fDSqYGWQVT5sEkJDyf0laaceNUf9dE6p9Ew3BgoaxQt77ZCQYgREREpJGzozPea/Oe2m3vb3sfFSsV4tAhICoK8PUFPDyE71FRIpbgtbYEbbFzNqxgapDFmToPJixMKLGri3bthP1N0L/7j++rbXctb9oFEK0NAxAiIiLSany78ahRsYZK+7+3/8Wyk8vg4iI8nE9OBtLThe+xsSIHH9aSoG2KnA0xpgbZMnPkwbi6Cut7lAxCSgZ+6tYBEbF/WY+y1Lar+/dlzxiAEBERkVZuzm74vOvnardN2zUNuU+fL0YmasK5ITd+pp6qJXbOhhhTg2ydufJgSgYhsbHAlSvPp8BpWoRQxP7deXRHpc3Z0RkVnSqq3d9eK8gxACEiIrIzprhpGR00Go1rNFZpv/3wtiIhXXT63vj98Yfpp2qJnbNh7NQge2DOPBh5ELJq1fMpbTExws+aVkAXsX/qAhD3iu5KuSG5uUKX/PwALy/he0yM0G4vJNnZ2XYaW+nOxcUFUk0lPIiIyoCioiLk2tPVzQys7dqRmyusSp6UJKy94eQkVKiaNUukqVAA/rr0F/qs6qPSXt6hPC5GXYSPm484JyouLk63EYI5c4A//1QOWPz9lUdLND3hNkT9+tqnTfn7C0/XdaFupMeUfbdWYn6mpiBC/zou64gD6QeU2gJrB+LEOycACP+O27cHzp9XXuBTXtRBtLwqERhz3bCe/zmJiIjIIPKblvh4ICUFyMwUvsfHC+1ixZa9GvTCS/VeUml/WvgUH+/8WJyTlKRLgravr2rwAZhuLQ2xczYMnRpkauasPGbteTAi9U9dDkjx/I+pU1WDD0D4+fx5YNo0nXts1RiAEBER2Thz3bRIJBLM7zkfEqgmevx25jf8nfG3OCcqTpcbv5QU862lYaqcDUOmBpmSOSuPWXsejIj9UzcFq3gAkpSk+u9YrqgISEzUrRvWjgEIERGRjTPnTUuLWi3wVuBbardN3DYRMjETUPS58dOVsTkEpszZcHVV7dvgwZYLPsxVecyceTCGjOqI1L+CogK1ZXjl6+zIZML0SW3y8+0jMZ0BCBERkQ2zxE3LjO4zUMmpkkr7wfSDWHturXgn0vfGz89P+z5irKVhTDlXW2COkrglmeszNXRUR6T+3Xt8DzKo/kOUj4BIJELuljZOTiJXmrMQBiBEREQ2zBI3LbUr18ZHHT9Su23Kjil4nP9YnBPpc+PXt6+wAIk2YuUQWGvOhhjMVRK3JFN/psaO6ojQP3XTrwDlKVjh4ULCuTpSKRARofHwNoUBCBER2aRHjx7hrbfeQmBgIIKCgpCoYZ7RjRs30K9fPwQHByMkJARvvPEG7t9/Pg0iICAAbdq0QceOHdGxY0esW7fOXG9BZ6WNXljipmVi+4nwquKl0p6SnYLPdn+m0zF0GpXR5cZv1Cjgww9167hYOQTWlrMhFnOWxC3JVJ+pWKM6RvZPUwDiXsld8edZs4RqVyX/PcurYM2cqfUUNoNleGF9pRSJiMzNFsvwzp07FykpKViyZAlSUlLQs2dPHDlyBG5ubkr73b59G1evXkX79u0BAJ988gkePHiA2P9usgICArB69Wo0bdpUr/Ob+tqhT1ldS5XuXPnPSry2/jWVdgkk2D9yP0K8QoSbuc2bFTequbnA1FcvIul8feQXOuheLrjEcQAIgYR8vr26G0x1SntSre08thpUGMLaS+LqIyFBmGalq1WrxA2s/rPu/Dr0/72/SvuuEbvQ1ber4ufcXKFwRGLi83/7ERFC8GEtJXgBluElIqIyaP369Rg1ahQAwNfXFyEhIfjrr79U9qtZs6Yi+ACA4OBgpKSkmKubBtG3rK6LixBkREUJFWk9PITvUVGmXTdgSMAQtPVoq9Iugwxv/PkGHt25oTTnPjcXaN/wDuK31kdKmoN+5YK1JWiLlUNgzspP1szaS+Lqy5KjOsVkPVQtwQsoT8EChH+vsbHCjML0dOF7bKx1BR/GYgBCREQ2KSMjA15ez6cAeXt7IyMjQ+trCgsLsXTpUoSGhiq1jxo1CiEhIYiOjsadO+qnSZiTIWV1LXHTIpVI8WPEjyjnUE5l2+V7lzF1cmulOfdTPX/G+ZtVUQQHpX1FKRds7Bx9c1d+slbWXhLXULqsJyNGkQItdMkBKckeEs7VYQBCRERWKSwsDPXq1VP7JQ80JMWuzqWVf5XJZHj//ffh6uqKd955R9G+adMmHDhwAHv27EG1atUwZswY07whPRhbVtecNy3NazbH510/V7st1ucmkho+/znpQReV4ENOlHLBhs7Rt0TlJ2tlzpK45mQFozqaApDqFaqb7JzWigEIERFZpc2bN+PatWtqvzw9PeHp6Ym0tDTF/unp6fD09NR4vClTpiAzMxP/93//p5S7IR9FcXJywpgxY3Do0CHTvSkd2OJaAJNCJqmfiiUBhgwATtUGZADyob1clyjvy5C1NCxV+cka2WOZYSsZ1bmRd0Olzc3ZDU4OpZSxs0MMQIiIyCZFRkZi6dKlAICUlBQcOHAAvXr1UrvvlClTkJycjBUrVqBcuefThR4+fIjs7GzFz2vXrkVAQIBJ+10aW1wLwFHqiJ/7/ozyDuVVtj0sB4QPAW5WBpygPbKy2PuykhwBq2FvZYatZFTn1M1TKm3qKsmVBQxAiIjIJsXExODJkycIDAxE//79MW/ePFStWhUAsGzZMsyaNQsAcPjwYfzwww9IS0vDiy++iI4dO2LYsGEAgKysLISHhyMkJAQhISE4cOAAvvvuO4u9JzlbWgtAPmLRuEZjfN3ja7X7ZLgCkUOAUKe1kKJA7T4Wf19WkCNgVeypzLAVjOrkPs3FpbuXVNpb12kt+rlsAcvwgmV4iYhssQyvpZny2mGpsrr69E9dieCZM2WYtOcdLD2xVO3r+p51wKW1J3BB1hRFcFS0W8X7iovTbZpObGzZCULsTfFcH/nfo/zv3cSjOvtS96Hzz51V2uNC4xDdNtok5zQ1Y64bDEDAAISIiAGI/syxDog1rgVQWnC0d38+Xl0aiJ2Pzqp9/aS9Tni28yskIgL5KAcnPENEaD5m/t7Icu/LStaJIDOw0DovCw8vxIStE1Ta97+5Hx28O5jsvKbEAMRIDECIqKxjAKI/c147ZDLryfmIiRHW7VBXpUsqFdYemf5xKtrNb4xLlZ6oPcav64Dh/wiJ6RJryCfQVAVLHWvoL9mc19e/juX/LFdqk0qkePDhA1QqV8lCvTIOFyIkIiKyY9YSfAC6lQiuWssHG987gGrPHNXu93YEsN/bSoIPwCpyBMi+Hb9xXKWtcY3GNht8GIsBCBEREelEnxLBDXxbY93QP+FUpBo9PXME+r7miKsJS6znZt7eKj+R1Xj47CEu3Lmg0l5WE9ABBiBERESkI31LBHdp1hvfv7xI7X53yxUgfONQZD/JFreTxrCnyk9kNU7fOo0imeqwYVCdIAv0xjowACEiIiKd6Vsi+M2Qsfigwwdq9z9/5zwGrhmI/MJShlXMyZCFDIm0OH5ddfoVwACEiIiISCezZgnVrkoGIfIqWDNnqr7myxe/RN/GfdUeb/u17YjZHAOZNS3tTiQidfkfEkjQqnYr83fGSjAAISIiIp25uAjrdURFAb6+gIeH8D0qSvM6HlKJFCteWYHA2oFqj/nd8e8Q93ecSftNZCknbpxQaWtYvSFcyluwnraFsQwvWIaXiIhlePXHa4dAnxLBmQ8y8cKPL+B67nWVbVKJFImDE9G7YW+Re6g/ayp7TLbtcf5juMx2QaGsUKl9aMBQrOy30qZ/11iGl4iIiCxCn5snjyoeSBqShIpOFVW2FcmKMPiPwThz64yIvdNdbq6Qb+7nB3h5Cd9jYoR2IkOdvnVaJfgAgNTDrcv07xoDECIiIjKb1nVaY8UrK9Ruy3uWhz6r+qgdITEl+eru8fFASgqQmSl8j48X2svSjSGJS930KwA4uDaoTP+uMQAhIiIis3qlySuY8+IctdvSctIQ8lMITt08Zbb+TJ0KnD+vusBiUZHQPm2a2bpCdkZTBSzZdeV8qLL2u8YAhIiIiMxuSocpeLPVm2q3peakIuSnEKw6s8osfdFldXeFhAQgJ8cs/SLbJpPJcCD9gOqGu/WBp6plnVV+1+wYAxAiIiIyO4lEgu/6fIfOPp3Vbn9c8BhD1w3FpG2TUFBUYLJ+6LS6+/1cyGQA4uKAIUOA0FDDg5CcHCGIKY5BjV06fes0Lt69qLrherDG1+TnC7+T9o4BCBEREVlEOYdyWDdwHZq5N9O4z/xD8xG6IhR3Ht0xSR90Wt095w4kDeoD48YJDYcPGxaE5OQIrxsyRAhmAHGCGn0xCDKL3878pn7DpT4aX+PkZLtVsfTBAISIiIjMqvgT3uoVq2PXiF0aR0IA4H/J/0PwD8EmywvRuro7ChCBDcDVq8ob9A1C5MHH4cPCz+PGAfVFCGr0ZS1BkJ0rkhVh1b+qUwgdZRUhuRSp9jVSKRARYeqeWQcGIERERGWUOad6aCtz617JHTuG70D0C9EaXy/PC9H4VNkIGld3RwGa4Dxm4hP1Lzx8GNi8ufQTlAw+5IwNavRlLUFQKexhCtLe1L3IeJCh0v5Kk75oWr+y6u+aVPgdnDnTTB20MAYgREREZYgl1rvQpcytk4MT4sLi8HPkzyjvUF7tcR4XPMawdcPw/tb3Rc0LUVnd3TUXvriGKCzGIYTABXnqXxgbCwweXPoJNm9WDT400TWo0Ze1BEEa2Ns6LCv/Wam2fUTgUOXfNQ/he1SU8DvoUkYWR+dK6OBqtkREXAldf7Z47ZAHAiVLzsqfvprqBigmRgg21FWakkqFm6/Y2Odtx64fQ7/V/ZD+IF3jMV/0exEJAxJQo2IN0fsrk0HI+Sh5c16cvz9w5YruB42Lez7SoE1srPCBiS0hQZhmpatVq3QLrkRgqd9LU3la8BS159dG9pNspfYaFWvg+sTrcHJ4nnTEldCJiIjIrllqvQu9ytwCCK4bjGOjj6GLTxeNx5TnhZy8cVLEngoki+K0Bx+AsF2eQ6GLmBghaNHG3980wQcgBBPFozxtdB3ZEYm9rcOy+cpmleADAAY2HagUfAC2G3wYiwEIERFRGaFvICAGncrcqik9WrNSTWwfvh0xL2i+IU/NSUWHZR3EzQtJSFAaqdA6TWTcONVqUprEmSCo0ZelgyANLPF7aUorz6iffjU0YKiZe2K9GIAQERGVAYYGAsbSqcythtKjTg5OiA2LxS99fyk1L2Ti1oni5IWEhSG3TXfEYCH8cA1eSIcfriEGC5GLysr7tmsHhIWVfswSQY1W+gQ1+rKGIKgES/1emsqDpw+QdDFJpd3XzRchXiEW6JF1YgBCRERUBhgTCBhLa5lbHUqPvt7ydRwYeQBeVbw07rPg8AK8vOJlo9cLyZW6on3uNsQjCinwQyY8kQI/xGMs2uPQ8yCkXTtgyxbAVXVFaxVhYcL+utA1qNGXtQRBJVjy99IU1p1fh6eFT1XahzYfComtvAkzYABCRERURhgbCBhKY5lbPUqPBtUNKjUvZGfyTgT/EIy/M/42uK9TpwLnLzmgCA5K7UVwxHlJE0zDDP2CD0DYb8sW1SCk5HQofY+rD2sIgjSw1O+l2GQyGf7v1P+p3TasxTAz98a6MQAhIiIqI8QIBAyhUubWwNKj+uSFfL77c4OmZGnNR5A5INH9bcOChJJBSGysUEVLnhhuyuBD3fnlzBkEaWCp30uxbby0EXtT96q0t6zVEk3dm1qgR9aLZXhhm6UUiYjExDK8+rPVa0durlBVKDFRmFvv5CQ8YZ4503ylTsUoPfrr6V8xOmm02ukucm092mJFvxWoX62+zv3y8hLWKdHEwwNITzei/zk5wjofxatMJSQIIw7muOkvvh6IvOSvvESwBYIPOWv4vTTG04KnaPZtM1y9r5pj83WPrzEpZJIFemVaxlw3GIDAdi8iRERiYQCiP3u4dtjyGgQAcPz6cbyy+hWt64VUcqqE2NBYjAwcqdMcfD8/YZFETXx9geRk/ftqVSwdBJXCFn8v5+yfg4/+95FKe81KNXEp6hJcnS3/uYqN64AQERGR3mztJq+koLpBOD76OLr6dtW4z8P8h3g76W10/aUrDmeUvhq5veQjaOXqqrrOx+DBVhF8ALb3e5nxIAMz96qfJzb7xdl2GXwYiyMgsI+nWERExuAIiP547bAeBUUFmLN/Dj7fU3reR/8m/TGr+yw0qtFI7XZ7W5WbTCvvWR46/19nnLypuiBmcN1g/P3235BK7PP/CY6AEBFRmfPo0SO89dZbCAwMRFBQEBK1rFbm5uaGkJAQdOzYER07dsTBgwcV265evYqePXsiKCgI3bt3x4ULF8zRfRKRo9QR0zpPw8GRB9GwekOt+/5x/g80+7YZxmwcgxu5N1S2i5UwT/avoKgAg9YOUht8AEBcaJzdBh/G4ggI+BSLiMgWR0Dmzp2LlJQULFmyBCkpKejZsyeOHDkCNzc3lX3d3NyQkZGBypUrq2wLDw/H4MGDMWzYMGzYsAGLFy/G9u3bSz0/rx3W6eGzh5i0bRK+O/5dqftWdKqI99u/j0khk1ClfBW1+1htPoKV53HYO5lMhjF/jcH3x79Xu314i+H49ZVfzdwr8+IICBERlTnr16/HqFGjAAC+vr4ICQnBX3/9pdcxsrKycPr0aQwaNAgAEBERgdTUVKSmporeXzKPSuUqYUmfJUgcnIg6leto3fdR/iPM2DsD9ePqY9Hfi/Cs8JnKPlYbfISGAkOGPF+1PC5O+Dk0VNhuj3JyVBdITEiwyPv96sBXGoOPxjUaIzY01sw9si0MQIiIyCZlZGTAy+v5ytje3t7IyMjQuH+fPn3QoUMHfPzxx3j48CEAIDMzE3Xq1IGjoyMAQCKRwNPTU+txyDaENwrH5ejLmNltJlzKaZ83lfUoCzFbYtAkvgkS/k1AkUzDQiDWoHgZXUAon1u//vNVzg8fts8gxIqCroR/E/Dh/z5Uu61mpZrYNHQTqlaoqvH1sjI/94gBCBERWamwsDDUq1dP7Zc8QCheVlWm5ap+5swZ7N69G9u2bcOdO3fw6aefKraVLM2q7ThkWyqVq4SpnafiasxVjGs7Dk5SJ637X7t/DUP+GILm3zbHL6d+UTsiYlElgw+5qyXWnrDFIETb6IY5gi4dR1f2p+3HiD9HqD1ERaeK2DhkI/yq+qlsy80Vllzx8xPWmvHzE362sZmvomEAQkREVmnz5s24du2a2i9PT094enoiLS1NsX96ejo8PT3VHks+UlKpUiW8/fbbOHToEADAw8MD169fR0GBUDlJJpMhMzNT43HINrlXcsfC0IW4EHUBQ5oPKXX/83fO440Nb6B+XH3EHo7Fw2cPzdBLHWzerBp8aHL4sLC/LdA2utGjB/DSS6YNunQcXbl45yIiEyLVBqZSiRSr+q9CG482KtvkldXi44U1ZjIzhe/x8UJ7WQxCGIAQEZFNioyMxNKlSwEAKSkpOHDgAHr16qWyX3Z2Nh49egRASJpct24dAgICAADu7u4ICAjA6tWrAQCJiYnw9vaGj4+Pmd4FmVO9qvXwW//fcHz0cbxU76VS909/kI7xW8fDe6E3Pt/9Oe4+umuGXmoxeLCwerkuYmNV1/qwRqWNbhw9Chw7ptuxDAm6dBxduX3jCnr91gv3Ht9Te5jY0FhENFK/SMzUqaplnQHh5/PnhRXgyxpWwQIrmRAR2WIVrIcPHyIqKgqnTp2CVCrFp59+isjISADAsmXLcOPGDUydOhVHjhzB+PHjIZFIUFhYiBYtWmDu3LmoWlWYo3358mW89957uHfvHlxcXLBkyRI0adKk1PPz2mH7tl3dhg92fIBTN0/ptH9Fp4oY3Xo0JrafCC9Xr9JfYCr166uOABTn7w9cuWK+/hhK05QyQ8XGCvOaRD7/Iyeg+9jK+NstT+32ie0mYv7L8zW+3s9PGPHQxNcXSE7Wob9WxpjrBgMQ8CJCRGSLAYil8dphH4pkRVh1ZhWm7ZqGlOwUnV7jKHXEay1ew5SQKWjiXnqwKqq4uOdP57XR92ZcTLqWCE5IEKY5icGQoEuH8xdKgFcHAus1/DX3b9Ifv7/6u8b1PoqKAG9vYdqVJh4eQHq6lVZc04JleImIiIgMIJVIMazFMFyKuoTlryxHM/dmpb6moKgAP5/6GU2/bYpXVr+CvzP+NkNPIdww6xJ8AMJ+JZOqzUGfalX6TCkrzdWrz8+nKx3OP6mn5uCjnWc7LH9luUrwUTzh3NsbuHlTezecnGwv+DAWR0DAp1hERBwB0R+vHfapSFaETZc3Yfb+2TiYflDn13X17YoPO3yInv49VSqriUafKUvt2gFbtph3UUJ1/fP3V54upq5fpU0p08eqVfrnvmg4f1xbYFyY+pf4V/XHobcOwb2Su1K7POFcXc6HOlIpEBUlXhxmThwBISIiIhKBVCJFn4Z9cGDkAex7cx96N+it0+t2p+xG6MpQtP6hNZafXo7H+Y/F75yrq3Dz3q6dcru/v/LP1hJ8AKVXq4qLEy/4aNdOmOalDw3n/7MxMD5U/UuqVaiGTcM2qQQfgOaEc3WkUqBJE2DmTP26bA84AgI+xSIi4giI/njtKDv+ufUPvjrwFRL+TUChrFCn11R1roo3Wr2B0UGj0bhGY3E7VPxmX57rERcH2bhxkFgi+ACAhATIhgyBzmM/q1YJ3w3NAdFlZKU0GnJAjngAXd8AHqtZNqa8Q3n87/X/oYN3B7WHLC3h3NERqFVLmHYVESEEHy7a18m0WkxCNxIvIkRU1jEA0R+vHWVP8v1kzD80Hz+d/AlPCp7o/Lquvl3xTtA7eKXxKyjvWF6czvyX6J3bezCmTgWSkoD8nEdwcimP8EgHzJplnhvb3Fw8P//9XDjl3EE4EjEL0+AC9VWjFEGTPlPK2rQRlhA/dkwp6MK4cYaP+Kg5/6XqQKc3gduV1b/k9wG/49Vmr6rdJpMJiwyWlnCeliaMftg6BiBG4kWEiMo6BiD647Wj7Lr98Dbi/o5D/NF4ZD/J1vl1bs5uCKsfhohGEQitHwo3Zzej+qEp30A+tefQIcClSMeKVGKeHwVoggs4hPaqQUjJalX65I0AwObNkA0a/Dxp29j38t/5b505jPgXgNi2wANn9bt+9dJXmNxhstbD2WvJXXWYA0JERERkJjUr1cTM7jORNj4N83rMQ6Wiujq9LvtJNlb9uwpD/hgC96/d8eKvLyL2cCyu3b9mUD9KXeBu8lPdK1KJeX444jwaYxpmqL6oZLWqknktsbFCgCLPyv4v+MiVuiLmE1f4fTQYXl7CjX5MDJDbe7BRgdT5Z9cxalJD+EyUYEYXzcHHmOAxmBQyqdTjhYdrHt2QSoVpV8QREAB8ikVExBEQ/fHaQXK+/k+R6rYcaBsL1PrXoGM0c2+G8IbhiGgUgRc8XoCD1KHU15T6tL3cdSQ/83jeIEbehD7nxzUkw1/9xpLVqrSsHZIrdS19pEeP6WYymQx7Uvdg3sF5+OvyX6Xu37tBb/w5+E84Sh1L3VenUSkbzfkoiVOwjMSLCBGVdQxA9MdrBwEl5/3LAK9DQPB3QLPfAcenBh3TtbwrOnh3QCfvTujk3QnBdYNVckd0yjdABtLhpT0x3MAgxKjz63nOmBggPl59ZSl9ytgWFBVg7bm1mHdwHo7fOK7TuYPqBGH3G7tRuZyGpBA1cnOBadOAxEQgP98+Es7VYQBiJF5EiKisYwCiP147SE7tSECFe0DLX4Dg74EaF406vrOjM9p6tBUCEp9OaO/ZHi7lXXQYgUhGMuqVfgJD1s6ALiMgyUj272H0qIuxeRW5T3Px44kfsfDvhUjLSdP5vIObD8Z3vb+Dq7PhU7xkMvtdZJA5IEREREQWonbe/+NqwOEJkHx7Hv0f7MWEdhPgX1XDdKRSPCl4gj2pezBz30y8vOJlVJ1bFcE/BMNt0ARImqwHKmapvEYqKUQENpR+8NhYg4IPoJR8BxQgonOO2nwOfYIPmUwYRdAmP1/Yr6SMBxmYsn0KvBZ4YeK2iToFHxJI0LdxXxwYeQCr+q8yKvgA7Df4MBZHQMCnWEREHAHRH68dJKfrvH+ZTIYLdy4g8WIiki4l4WD6Qcgg0m1YVmMgrROQ2gmSzPZoUtsPhx+1hkvyP5pfU7IilZ40v28ZmtTNwaFzbs+nHBlRrUqfERCZTIbTt07jm0PfYNW/q1BQVKDTOZwdnfFmqzcxod0ENKjeQO8+lkWcgmUkXkSIqKxjAKI/Xjusnzmnvxgy7z/rYRY2Xd6ExEuJ2HplKx7mPxStPxXghCbX89EsC2h2G2h+G2iWBXjnANLid37yNTUMZI58B205IJLKWQh/5yha9zmKo9eFr9sPb+t8bPeK7oh+IRpj2oxBjYo1xOlwGcEAxEi8iBBRWccARH+8dlgnpYXx/rshDg+H2RbmAwwLfJ4UPMHulN3YnbIb+9L24WjmUeQXlTL3yACVngFNSwQlzT5dDM/X3oPEyGjNVAGffKTl3NUHkNU+DngcBeoeFb67pRp0zEbVG+H99u/jtRavoYJTBZF7XDYwADESLyJEVNYxANEfrx3Wx55KoD7Of4wjmUewN3Uv9qXtw8H0g6KOkJRUpXwVNHVvimbuzdC4RmPUqlQLNSrWgHsld7hXdEeNijVQqVwlk52/pCcFT3D65mkcyTyCo9eP4u+Mo7h09yIgMe62tYtPF0wKmYReDXpBKuG/X2MwADESLyJEVNYxANEfrx3WR6xyrdaooKgAp26ewr7UfdiXJnzdeXTHrH2o4FgB7pWEYMS9orvw5wrKQYp7JXdUda6Kp4VP8Sj/kcrXw2cPVdsLlH++9/gezmWd0zl/ozQOEge82uxVvN/+fQTXDRblmMQAxGi8iBBRWccARH+8dlgfY8u12hKZTIaLdy9iX+o+7E3bi32p+5CaY9h0JHtVyakSRrUehXHtxsHXzdfS3bE7DECMxIsIEZV1DED0x2uHddFpYTwPID3dfkujpuek4+/Mv3H29lmczTqLf2//i8v3Los2kmDtKjhWQOs6rdGmbhu09WyL0PqhcHN2s3S37JYx143S15QnIiIisnISiZBwro2Tk/0GHwDg5eoFL1cvDGg6QNH2rPAZLt29pBSUnM06iyv3rqBIpmaumo1wlDqiRa0WaFO3jfDl0QZN3ZvCUcpbW1vAvyUiIiKyC+Hh2nNAIiLM3ydLK+dQDs1rNkfzms2V2p8UPMHFOxdxNusszt4+i3+z/sXZ22dx7f418dYmEYkEEjSq0Ugp2GhVuxWcHZ0t3TUyEKdggcPoREScgqU/Xjusjz1VwbKUx/mPcTPvJrIeZeHOozvIepil8ufiP+c8zTHqfOUcyqGiU0VUdKqISk6VUNGpIio4VYBXFS8E1w1Gm7ptEFQ3CFXKVxHpHZJYmANiJF5EiKisYwCiP147rJM5Fsaj554VPsPdR3eFwOS/AOXB0wdwdnRWCirUfVVwqsApUzaMAYiReBEhorKOAYj+eO2wfuZcCZ2orDHmusH/OYmIiMguMfggsk4MQIiIiIiIyGwYgBARERERkdkwACEiIiIiIrNhAEJERERERGbDAISIiIiI7JKszNd6tU4MQIiIiIjIbuTmAjExgJ8f4OUlfI+JEdrJOnAdELCWOxGRLa4D8ujRI0RHR+PEiROQSqX47LPPEBERobLfhQsX8Pbbbyt+zsnJQW5uLlJSUgAAAQEBcHZ2Rvny5QEAEydORL9+/Uo9P68dZE5c00Q3ublA+/bA+fNAUdHzdqkUaNIEOHSIC1KKxZjrBpefJCIim7Ro0SKUK1cOJ0+eREpKCnr27InOnTvDzc1Nab/GjRtj//79ip8nT56scqxffvkFTZs2NXWXifSSmwtMnQokJT1f1T08HJg1izfRmkydqhp8AMLP588D06YBsbGW6Rs9x0c3RERkk9avX49Ro0YBAHx9fRESEoK//vpL62uePn2KNWvWYPjw4eboIpHB5E/y4+OBlBQgM1P4Hh8vtJtzwNKW8iiSklSDD7miIiAx0bz9IfUYgBARkU3KyMiAl5eX4mdvb29kZGRofU1SUhJ8fHzQokULpfZRo0YhJCQE0dHRuHPnjkn6S6QPXZ7km5It5lHIZMJIkTb5+bYVUNkrTsEiIiKrFBYWhosXL6rdtnfvXgCApNikeJkOdxUrVqxQGf3YtGkTvLy8kJ+fj5kzZ2LMmDFYs2aNET0nMp4uT/JNNZVIUx5FfDywc6f15lFIJMI0NW2cnJhLYw0YgBARkVXavHmz1u2enp5IS0tDjRo1AADp6eno0aOHxv3T0tJw5MgR/Pzzz0rt8lEUJycnjBkzBsHBwcZ1nMhI+jzJN8XNtC3nUYSHC4GSuuBNKgXU1KkgC+AULCIiskmRkZFYunQpACAlJQUHDhxAr169NO6/cuVK9OnTRylJ/eHDh8jOzlb8vHbtWgQEBJiqy0Q6sfSTfFvOo5g1S6h2VbJAnbwK1syZlukXKWMAQkRENikmJgZPnjxBYGAg+vfvj3nz5qFq1aoAgGXLlmHWrFmKfWUyGX777Te89tprSsfIyspCeHg4QkJCEBISggMHDuC7774z6/sgUic8XPUmWs6UT/JtPY/CxUWYIhYVBfj6Ah4ewveoKOudOlYWcR0QsJY7EZEtrgNiabx2kClZcj0LPz+h4pYmvr5AcrJpzi02rp9iOsZcN/g/JxEREZGVseSTfEuNvpgCgw/rxBEQ8CkWERFHQPTHaweZkzmf5HM1cdIFR0CIiIiI7Jg5n+Qzj4JMjSMg4FMsIiKOgOiP1w4qK5hHQepwBISIiIiITILBB4mNAQgREREREZkNAxAiIiIiIjIbBiBERERERGQ2DECIiIiIiMhsHC3dAVPYv38/EhIS8PTpU7i5ueHrr7+2dJeIiIiIiAhWNAIyZcoUBAQEwM3NDefOnVPadvXqVfTs2RNBQUHo3r07Lly4oPVYHTt2xOLFi7F06VKkp6eztCQRERERkZWwmhGQyMhIjBs3DqGhoSrbxo8fjxEjRmDYsGHYsGEDoqOjsX37dty+fRujRo1S2rdly5b44osvAADbtm1Do0aN4FLKijlFxZf5JCIqg/j/oP74mRFRWWbM/4FWtxBhQEAAVq9ejaZNmwIAsrKyEBQUhGvXrsHR0REymQyNGjXC9u3b4ePjo/E4v/zyC65fv46PPvrIXF0nIiIiIqJSWM0ULE0yMzNRp04dODoKgzUSiQSenp7IyMjQ+JoNGzbgq6++wu3btzFhwgTcuXPHXN0lIiIiIiItrGYKljaSEktwymTaB20iIyMRGRlpyi4REREREZEBrH4ExMPDA9evX0dBQQEAIfjIzMyEp6enhXtGRERERET6svoAxN3dXZEXAgCJiYnw9vbWmv9BRERERETWyWqS0CdNmoRNmzbh1q1bqF69OipVqoSTJ08CAC5fvoz33nsP9+7dg4uLC5YsWYImTZpYuMdERERERKQvqxkBmTdvHs6dO4e7d+/i0qVLiuADABo0aIDt27fj+PHj2L17t0WDD33XJLF2T548wdChQxEUFISOHTuif//+SE1NBSBUIOvfvz9at26N9u3b49ChQ4rXPXr0CG+99RYCAwMRFBSExMRES70Fg8yZM0dpzRl7fK9Pnz7F5MmT0bp1a7Rr1w6jR48GYJ/vFQD+97//oUuXLujUqRPat2+P3377DYB9vF9N6yQZ+t6KioowefJktGrVCoGBgfjxxx/N+n5Ima6/hxcuXEDHjh0VXwEBAfD19VVsDwgIQJs2bRTb161bZ7a+AYCbmxtCQkIU5z948KBimymunbr27caNG+jXrx+Cg4MREhKCN954A/fv31dsF+tz0/U9/vrrr2jdujVatWqFcePGKaaYA8CWLVvQpk0bBAYGYvjw4cjLyzOoL/r2a8+ePXjxxRfRtm1btG/fHjNmzFDk26ampqJ69epKv3vJyclG90vXvu3btw916tRROv/jx48V203xmenat1WrVin1q169enjttdcAmO5z07ZuXnHm/j3Tl9UEILZCvibJ8ePHMW7cOERHR1u6S0Z74403cOzYMezfvx8vv/wyxo8fDwCYPn06goODceLECcTHx2PUqFGKX+BFixahXLlyOHnyJP744w9MmjQJ2dnZlnsTejh16hSOHTumlEdkj+91+vTpkEgkOH78OA4fPqxYH8ce36tMJsPbb7+N+Ph47Nu3DwkJCZgwYQJyc3Pt4v1GRkZiy5Yt8PLyUmo39L2tXr0aFy9exPHjx7Fz507Exsbi0qVL5n5b9B9dfw8bN26M/fv3K75CQ0Px6quvKu3zyy+/KLb369fPbH2T27Ztm+L8ISEhinZTXDt17ZuDgwMmT56MY8eO4eDBg/Dy8sL06dOV9hHjc9PlPaakpODLL7/Eli1bcPLkSdy6dQvLly8HAOTl5SE6OhorV67EyZMnUbt2bcyfP9+gvujbLzc3N/z000/4+++/sWvXLhw4cABr165VbHd1dVX63fPz8zO6X7r2DQAaNWqkdP4KFSoAMN1npmvfhgwZotSv2rVrK/2bNMXnpul6UJwlfs/0xQBED1lZWTh9+jQGDRoEAIiIiEBqaqpixMAWOTs7o2fPnopKY23atEFKSgoA4M8//1Qs9Ni6dWvUrFlT8YR1/fr1im2+vr4ICQnBX3/9Zf43oCf5qMC8efOUqqvZ23t9+PAhVq5ciU8//VTxPmvXrg3A/t5rcTk5OQCA3NxcVKtWDeXLl7eL99uhQwd4eHiotBv63tavX4+RI0fCwcEBVatWxSuvvII//vjDTO+GSjLk9/Dp06dYs2YNhg8fbnV9K8lU105d+1azZk20b99e8XNwcLDiOicWXd9jYmIi+vTpg5o1a0IikWDkyJGKG/0dO3YgMDAQDRs2BAC89dZbSkGAKfvVsmVLxWias7MzAgICRP+MDO2bNqb4zAzt2/Hjx3H79m306tXL6PNro+l6UJy5f88MwQBED4asSWJrvvvuO4SGhuLevXsoKipCjRo1FNu8vb0V7zUjI0Mp+i6+zZp9+eWXGDhwoNK0BXt8r8nJyahWrRrmzZuHrl27IiwsDHv27LHL9woI/xZ//vlnDB8+HM2bN0dYWBiWLFmCvLw8u3y/gHG/t7b8vu2RIX8fSUlJ8PHxQYsWLZTaR40ahZCQEERHR4uyBpa+fevTpw86dOiAjz/+GA8fPgRgumunIZ9bYWEhli5ditDQUKV2Yz83Xd9jenq6xj6r23bjxg2jVps25LO/desWNmzYgB49eijacnNz0a1bN3Tu3Blz585FYWGhwX0ypG9XrlxB586d0a1bN6Upo6b4zPTtm9zy5csxaNAgODk5KdpM8bnpwty/Z4awiXVArIm+a5LYkvnz5+PatWtYsGABnjx5Uup7Lb7dFj6HI0eO4MSJEypD70Dpf6+29l4LCgqQkpKCRo0aYfr06Thz5gz69u2LQ4cO2d17BYT3+8033+C3335Du3btcOLECQwbNgwHDhywy/crZ8x7s+X3bWvCwsJw8eJFtdv27t0LQP+/jxUrVqiMfmzatAleXl7Iz8/HzJkzMWbMGKxZs8ZsfTtz5gy8vLzw8OFDTJgwAZ9++qliaoch106xPzeZTIb3338frq6ueOeddxTthnxu6uj6HnX9dykWfT77Bw8eYPDgwYiJiUGrVq0ACKPn586dg7u7O+7fv48333wTixcvxrhx48zSt5YtW+Ls2bNwdXVFZmYmXn31VVSvXh2vvPKK2mOIRZ/P7dGjR1i3bh22bdumaDPl56YLc/+e6YsjIHqw5zVJFi1ahKSkJKxZswYVK1ZEtWrVAEDpSVB6errivXp6eiItLU3tNmt14MABXL58GS1atEBAQACuX7+O/v374/jx4wDs6716eXlBKpVi4MCBAIQkSx8fH8XF3J7eKyDc+Ny8eRPt2rUDIExHql27Nv79918A9vd+ARj1b9SW37ct2rx5M65du6b2y9PTU++/j7S0NBw5cgQDBgxQapc/1XRycsKYMWOUihKYo2/y81eqVAlvv/224vyGXjvF/tymTJmCzMxM/N///R+k0ue3P4Z8biXp+h69vLw09rnktrS0NNSpU0epr6bqFyA8rR8wYADCwsIQFRWlaC9fvjzc3d0BAFWrVsVrr72mVGDA1H2rUqUKXF1dFa8ZMGCA4vym+Mz06Zvchg0b0KhRIzRu3FjRZqrPTRfm/j0zBAMQPdjrmiSLFy/G2rVr8eeff8LNzU3RHhkZiaVLlwIATpw4gdu3byvm0RbflpKSggMHDph83qOxJkyYgAsXLuDMmTM4c+YM6tatiz/++AM9evSwu/davXp1dOnSBf/73/8ACP/BpKamokGDBnb3XoHnF4vLly8DAK5du4bk5GTUr1/fLt+vnKHvLTIyEv/3f/+HwsJC3L9/H+vWrRMlYZkMo+/v4cqVK9GnTx+l/68fPnyolIC9du1aBAQEmK1v2dnZePToEQChytq6desU5zfVtVOfz23KlClITk7GihUrUK5cOUW7WJ+bru8xIiICGzduxO3btyGTybBs2TL0798fAPDiiy/ixIkTioIQP/30k2KboXTtV15eHgYMGIDu3btjypQpStuysrKQn58PQMg9SkpKUpn6Z8q+3bx5UzE9KDc3F1u3blWc3xSfmT59k1M3Immqz00X5v49M4TVrANiK+xtTZLMzEw0a9YMvr6+qFy5MgAhav/f//6H27dv45133kFqairKlSuHefPmoWPHjgCE/7SjoqJw6tQpSKVSfPrpp4iMjLTkW9Gb/D+Xpk2b2uV7TUlJwdixY3H//n1IpVJ88MEHCA8Pt8v3Cgg3Dt988w0kEoliqkX//v3t4v1qWifJ0PdWWFiIDz74ADt27AAAvPfee4oyzWR+2v6uli1bhhs3bmDq1KkAhCexLVq0QHx8PDp37qw4RkpKCoYPH66YY+7j44M5c+YYfZOva9+OHDmC8ePHQyKRoLCwEC1atMDcuXNRtWpVAKa5durat8OHDyM0NBQNGzZUBB8+Pj5YuXKlqJ+bpvcYHR2NsLAwRXD0yy+/YOHChSgqKkLnzp3xzTffKPIGNm3ahM8++wwFBQVo2rQplixZgipVqhj1OenSr3nz5mHOnDlKT/D79u2LSZMmITExEbNnz4ZUKkVhYSE6deqEmTNnonz58kb1S9e+/fDDD1i2bBkcHBxQWFiIyMhIfPjhh4ppRKb4zHTtGyDkXHbq1Annz5+Hi4uL4vWm+tw0XQ8s/XumLwYgRERERERkNpyCRUREREREZsMAhIiIiIiIzIYBCBERERERmQ0DECIiIiIiMhsGIEREREREZDYMQIiIiIiIyGwYgBARERERkdkwACEys/nz56N+/fpwc3PDvn37LN0dIiIygd69e2PmzJk67z979myEhoaasEfmOQeRLrgQIZVJvXv3xoEDB7Bu3Tp0795d0T569Gg4ODhgyZIlJjlvWloaWrZsieXLl6NNmzaoWrWqYmVeXZi6f0REZVloaCi6dOmCjz76yOhj3b9/H05OTqhcubJO++fl5SE/P1+xerwpmOMcJe3evRt9+/ZFdna22c5J1s/R0h0gshRnZ2fMnDlTKQAxtdTUVMhkMvTu3RsSicRs5yUiInE8e/ZMpwdH+t7k6xqoGMMc5yDSBadgUZk1aNAgXLx4ERs3btS4z+3bt/H666/Dw8MDPj4+iIqKwsOHD7UeNy4uDs2aNUPNmjXx0ksv4fjx4wCAlStXIjw8HIBwYXJzc1P7+lOnTiE0NBR169aFj48PevXqhezsbMyePRu///47Vq1aBTc3N6XX7969G127dkXt2rURFBSEpUuXKralpqbCzc0N69evR8eOHVGrVi2Eh4cjIyNDsc+uXbvQqVMn1K5dG/Xq1cPAgQNL+/iIiOzKmDFjcPjwYcydOxdubm4ICAgA8Hza0uLFi9G4cWPFQ6tvvvkGL7zwAurUqYPWrVvju+++UzpeySlYbm5uWLlyJSIjI1GnTh106dIF//77r2J7yelRvXv3xqefforx48fD09MTAQEB+OOPP5TOsW7dOjRv3hx169bFu+++i2nTpqF3794a36O+59i3bx/c3Nywbds2BAUFoXbt2njttdeURjMCAgLw66+/Kp3Hzc0Nu3fvRmpqKvr27atok38GRAxAqMxyd3fHu+++iy+//BJFRUVq93nnnXeQmZmJv/76CwkJCTh48CA+/vhjjcdcs2YN5syZg88++wz79u1Ds2bN8Oqrr+LBgwfo168ffv75ZwDAxYsXcfHiRY3nbNu2LQ4ePIgtW7YogoHo6GhERETglVdeUXr95cuXMXz4cIwcORKHDx/GrFmzMHfuXKxbt07puDNmzMAXX3yBHTt2oKCgAO+88w4AoKCgACNGjMDQoUNx5MgRJCYmomvXrvp8lERENm/OnDkICgpCVFQULl68iF27dim2nTlzBidOnMD69evx008/AQDKlSuH2NhYHDp0CJ988glmzJiBbdu2aT3HV199hdGjR2Pfvn2oXbs2xo4dq3X/n3/+GQ0aNMDevXsxdOhQjB07FllZWQCAq1evYtSoURg5ciT27NmD+vXr45dfftH7fWs7h9zs2bOxZMkSJCUl4fLlyzpPUfP09FS57vXr10/vPpL9YQBCZVp0dDQyMzNVnioBwKVLl7Br1y7Ex8ejVatWaN++Pb766iusXLkSOTk5ao/3/fff4+2338bAgQPRqFEjzJ8/HxUqVMDq1atRoUIFxahFrVq1UKtWLbXHyMzMxMsvvwxfX180adIEb7zxBtzc3FC5cmU4OzvD2dlZ6fULFy7EiBEj8Prrr8PX1xehoaEYM2aMyoUoKioK3bt3R0BAAOLj43HgwAGcO3cODx48wIMHDxAeHg5vb280b94c7733nhGfKhGR7XF1dYWTkxMqVaqEWrVqoUaNGoptEokEcXFxaNKkCRo1agRA+D+1ffv28PX1xSuvvIKBAwdiw4YNWs8xcuRI9O7dG/Xr18f777+P06dPIy8vT+P+bdu2xdixY1GvXj1MnjwZUqlUMar+yy+/oE2bNpg4cSIaNGiASZMmoUmTJnq/b23nkJs6dSpeeOEFtGnTBnPnzsWaNWt0yulwcHBQue5VqFBB7z6S/WEAQmWam5sboqOjMXv2bBQUFChtu3z5MlxcXNC4cWNFW5s2bVBQUIDk5GS1x7t8+TKCg4MVPzs6OqJVq1a4fPmyzn0aNWoU+vXrh6FDh2Lp0qW4e/eu1v3PnTuHpUuXwsPDQ/H11VdfISUlRWm/1q1bK/5cr149uLm54fLly6hWrRr69++PkJAQjBw5EitWrNB6QSQiKmvq1aunkj+xdetWhIaGokGDBvDw8MCKFSuQmZmp9ThNmzZV/LlmzZoAgDt37ui0v6OjI6pVq6YYnbhy5QpatmyptH+rVq10ej+6nkMuKChI8efWrVujoKBA5RpDpA8GIFTmvfvuu3jw4AF+++03pXaZTLVAnDkSxz/77DPs3LkTbdq0werVqxEcHIyrV69q3P/hw4cYO3Ys9u3bp/g6dOgQEhMTlfbT1veffvoJ69atQ/369bF48WK0b98e9+7dE+09ERHZsooVKyr9nJKSguHDh6NTp074/fffsXfvXgwePBj5+flaj+Pk5KT4s/z/ZE1TgEvuL3+N/Nokk8lEuSZpO0fJvpb8MwBIpVKl/Uv7DIgABiBEqFy5MiZMmICvvvoKT58+VbQ3bNgQubm5uHDhgqLtyJEjcHR0hJ+fn9pjNWjQAMeOHVP8XFBQgFOnTqFhw4Z69alp06aYMGECduzYgVq1aikS5R0dHVFYWKi0b/PmzXH58mXUq1dP6cvHx0dpvxMnTij+nJycjOzsbDRo0EDRFhwcjI8//hh79+5FTk4O9uzZo1efiYhsnbr/Y9U5ffo0nJ2dMXXqVAQGBsLf3x+pqalm6OFzDRo0wOnTp1X6ZQrFp2SdOHECjo6O8PX1BQDUqFEDt27dUmw/e/as0msdHYWCq7p8rlR2MAAhAvDWW29BJpNh69atiraGDRuie/fuiIqKwqlTp3D48GF88MEHGDZsGFxdXdUe591338VPP/2ENWvW4NKlS3j//ffx+PFjnatKPX78GFOmTMHBgweRlpaGbdu2ISMjA/Xr1wcAeHl54cyZM0hNTVVMzYqJicHWrVsxc+ZMXLhwAefPn8fKlSvx448/Kh178eLF2L17N86cOYOoqCiEhISgadOmSElJwYwZM3Ds2DGkpaXhzz//xMOHD1GvXj1DPkoiIpvl5eWFY8eO4fr161pzHHx9fZGbm4uVK1fi2rVr+Prrr3Hy5EnzdRTAiBEjcOTIESxcuBBXrlzBggULcO7cOZOM1M+aNQvHjh3DsWPH8OGHH2LAgAGK3I6QkBCsWLECJ06cwIkTJ/DZZ58pvdbLywsAsG3bNty9e1fpQR+VXQxAiCCsCTJp0iQ8efJEqf27775DnTp10Lt3bwwcOBDt27fHl19+qfE4AwYMwAcffIDPPvsMHTt2xNmzZ7FmzRpUqVJFp344ODjgzp07ePvttxEcHIzJkydjypQpirKKI0aMgJubG9q1awd/f38Awpzf9evX48CBA+jatStCQ0OxcuVKeHt7Kx37448/xscff4wXX3xR8d4AYWrBuXPnMHToULRp0wbz58/H4sWLVeYWExHZu+joaNy7dw+tWrVCp06dNO7XsmVLfPLJJ/jss8/QpUsXpKWl4Y033jBfRwH4+/vj+++/x48//ojOnTvjwoULGDRoEMqXLy/6uSZPnoxRo0ahd+/e8PPzw5w5cxTb3n//fQQEBCAiIgKjR4/GxIkTlV7r4+OD8ePHY+zYsfD398fatWtF7x/ZHq6ETmTnUlNT0bJlS5w4cYKjGkREdiwyMhINGjTAvHnzRDnevn37EB4ejjt37iimUhGJgb9NRERERDbohx9+QLt27VCpUiWsX78ee/fu1bpWFZG1YABCREREZIPOnj2Lr7/+Gnl5efD398evv/6Ktm3bWrpbRKXiFCwiIiIiIjIbJqETEREREZHZMAAhIiIiIiKzYQBCRERERERmwwCEiIiIiIjMhgEIERERERGZDQMQIiIiIiIyGwYgRERERERkNgxAiOj/269jAQAAAIBB/taz2FUWAQBsBAQAANgICAAAsBEQAABgE3SmLf5bG+YFAAAAAElFTkSuQmCC",
"text/html": [
"\n",
"
![missing](\"week1/Optional_Labs/images/C1W1L1_Tour.PNG\")
![missing](\"week1/Optional_Labs/images/C1W1L1_Markdown.PNG\")
\n",
"![](\"week1/Optional_Labs/images/C1W1L1_Run.PNG\")
\n",
"![missing](\"week1/Practice_quiz/ss1.png\")
![missing](\"week1/Practice_quiz/ss2.png\")
![missing](\"week1/Practice_quiz/ss3.png\")
![](\"week2/Optional_Labs/images/C1_W2_Lab04_dot_notrans.gif\")
"
]
},
{
"cell_type": "markdown",
"id": "36aa2cda",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"The dot product multiplies the values in two vectors element-wise and then sums the result.\n",
"Vector dot product requires the dimensions of the two vectors to be the same. "
]
},
{
"cell_type": "markdown",
"id": "31cf42f3",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Let's implement our own version of the dot product below:\n",
"\n",
"**Using a for loop**, implement a function which returns the dot product of two vectors. The function to return given inputs $a$ and $b$:\n",
"$$ x = \\sum_{i=0}^{n-1} a_i b_i $$\n",
"Assume both `a` and `b` are the same shape."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4cccf6b1",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"def my_dot(a, b): \n",
" \"\"\"\n",
" Compute the dot product of two vectors\n",
" \n",
" Args:\n",
" a (ndarray (n,)): input vector \n",
" b (ndarray (n,)): input vector with same dimension as a\n",
" \n",
" Returns:\n",
" x (scalar): \n",
" \"\"\"\n",
" x=0\n",
" for i in range(a.shape[0]):\n",
" x = x + a[i] * b[i]\n",
" return x"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "95c40306",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"# test 1-D\n",
"a = np.array([1, 2, 3, 4])\n",
"b = np.array([-1, 4, 3, 2])\n",
"print(f\"my_dot(a, b) = {my_dot(a, b)}\")"
]
},
{
"cell_type": "markdown",
"id": "8aeeabc3",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Note, the dot product is expected to return a scalar value. \n",
"\n",
"Let's try the same operations using `np.dot`. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b50e32ca",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"# test 1-D\n",
"a = np.array([1, 2, 3, 4])\n",
"b = np.array([-1, 4, 3, 2])\n",
"c = np.dot(a, b)\n",
"print(f\"NumPy 1-D np.dot(a, b) = {c}, np.dot(a, b).shape = {c.shape} \") \n",
"c = np.dot(b, a)\n",
"print(f\"NumPy 1-D np.dot(b, a) = {c}, np.dot(a, b).shape = {c.shape} \")\n"
]
},
{
"cell_type": "markdown",
"id": "ba8b9cdc",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Above, you will note that the results for 1-D matched our implementation."
]
},
{
"cell_type": "markdown",
"id": "abcfcc14",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"###### 3.4.7 The Need for Speed: vector vs for loop\n",
"\n",
"\n",
"We utilized the NumPy library because it improves speed memory efficiency. Let's demonstrate:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "427fae1d",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"np.random.seed(1)\n",
"a = np.random.rand(10000000) # very large arrays\n",
"b = np.random.rand(10000000)\n",
"\n",
"tic = time.time() # capture start time\n",
"c = np.dot(a, b)\n",
"toc = time.time() # capture end time\n",
"\n",
"print(f\"np.dot(a, b) = {c:.4f}\")\n",
"print(f\"Vectorized version duration: {1000*(toc-tic):.4f} ms \")\n",
"\n",
"tic = time.time() # capture start time\n",
"c = my_dot(a,b)\n",
"toc = time.time() # capture end time\n",
"\n",
"print(f\"my_dot(a, b) = {c:.4f}\")\n",
"print(f\"loop version duration: {1000*(toc-tic):.4f} ms \")\n",
"\n",
"del(a);del(b) #remove these big arrays from memory"
]
},
{
"cell_type": "markdown",
"id": "21897335",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"So, vectorization provides a large speed up in this example. This is because NumPy makes better use of available data parallelism in the underlying hardware. GPU's and modern CPU's implement Single Instruction, Multiple Data (SIMD) pipelines allowing multiple operations to be issued in parallel. This is critical in Machine Learning where the data sets are often very large."
]
},
{
"cell_type": "markdown",
"id": "fc56006f",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"###### 3.4.8 Vector Vector operations in Course 1\n",
"\n",
"\n",
"Vector Vector operations will appear frequently in course 1. Here is why:\n",
"- Going forward, our examples will be stored in an array, `X_train` of dimension (m,n). This will be explained more in context, but here it is important to note it is a 2 Dimensional array or matrix (see next section on matrices).\n",
"- `w` will be a 1-dimensional vector of shape (n,).\n",
"- we will perform operations by looping through the examples, extracting each example to work on individually by indexing X. For example:`X[i]`\n",
"- `X[i]` returns a value of shape (n,), a 1-dimensional vector. Consequently, operations involving `X[i]` are often vector-vector. \n",
"\n",
"That is a somewhat lengthy explanation, but aligning and understanding the shapes of your operands is important when performing vector operations."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9eac936c",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"# show common Course 1 example\n",
"X = np.array([[1],[2],[3],[4]])\n",
"w = np.array([2])\n",
"c = np.dot(X[1], w)\n",
"\n",
"print(f\"X[1] has shape {X[1].shape}\")\n",
"print(f\"w has shape {w.shape}\")\n",
"print(f\"c has shape {c.shape}\")"
]
},
{
"cell_type": "markdown",
"id": "05e58839",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"#### 4 Matrices\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "9e5b0b36",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### 4.1 Abstract\n",
"\n",
"\n",
"Matrices, are two dimensional arrays. The elements of a matrix are all of the same type. In notation, matrices are denoted with capitol, bold letter such as $\\mathbf{X}$. In this and other labs, `m` is often the number of rows and `n` the number of columns. The elements of a matrix can be referenced with a two dimensional index. In math settings, numbers in the index typically run from 1 to n. In computer science and these labs, indexing will run from 0 to n-1. \n",
"![\"Image\"](\"week2/Optional_Labs/images/C1_W2_Lab04_Vectors.PNG\")
\n",
"![missing](\"week2/Optional_Labs/images/C1_W2_Lab04_Matrices.PNG\")
![missing](\"week2/Practice_quiz/ss1.png\")
![missing](\"week2/Practice_quiz/ss2.png\")
![missing](\"week2/Practice_quiz/ss3.png\")
![](\"week3/OptionalLabs/images/C1_W3_Classification.png\")
Examples of classification problems are things like: identifying email as Spam or Not Spam or determining if a tumor is malignant or benign. In particular, these are examples of *binary* classification where there are two possible outcomes. Outcomes can be described in pairs of 'positive'/'negative' such as 'yes'/'no, 'true'/'false' or '1'/'0'. \n",
"\n",
"Plots of classification data sets often use symbols to indicate the outcome of an example. In the plots below, 'X' is used to represent the positive values while 'O' represents negative outcomes. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ef30f6ed",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"x_train = np.array([0., 1, 2, 3, 4, 5])\n",
"y_train = np.array([0, 0, 0, 1, 1, 1])\n",
"X_train2 = np.array([[0.5, 1.5], [1,1], [1.5, 0.5], [3, 0.5], [2, 2], [1, 2.5]])\n",
"y_train2 = np.array([0, 0, 0, 1, 1, 1])"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "555a8709",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"pos = y_train == 1\n",
"neg = y_train == 0\n",
"\n",
"fig,ax = plt.subplots(1,2,figsize=(8,3))\n",
"#plot 1, single variable\n",
"ax[0].scatter(x_train[pos], y_train[pos], marker='x', s=80, c = 'red', label=\"y=1\")\n",
"ax[0].scatter(x_train[neg], y_train[neg], marker='o', s=100, label=\"y=0\", facecolors='none', edgecolors=dlc[\"dlblue\"],lw=3)\n",
"\n",
"ax[0].set_ylim(-0.08,1.1)\n",
"ax[0].set_ylabel('y', fontsize=12)\n",
"ax[0].set_xlabel('x', fontsize=12)\n",
"ax[0].set_title('one variable plot')\n",
"ax[0].legend()\n",
"\n",
"#plot 2, two variables\n",
"plot_data(X_train2, y_train2, ax[1])\n",
"ax[1].axis([0, 4, 0, 4])\n",
"ax[1].set_ylabel('$x_1$', fontsize=12)\n",
"ax[1].set_xlabel('$x_0$', fontsize=12)\n",
"ax[1].set_title('two variable plot')\n",
"ax[1].legend()\n",
"plt.tight_layout()\n",
"plt.show()\n"
]
},
{
"cell_type": "markdown",
"id": "4110c136",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Note in the plots above:\n",
"- In the single variable plot, positive results are shown both a red 'X's and as y=1. Negative results are blue 'O's and are located at y=0.\n",
" - Recall in the case of linear regression, y would not have been limited to two values but could have been any value.\n",
"- In the two-variable plot, the y axis is not available. Positive results are shown as red 'X's, while negative results use the blue 'O' symbol.\n",
" - Recall in the case of linear regression with multiple variables, y would not have been limited to two values and a similar plot would have been three-dimensional."
]
},
{
"cell_type": "markdown",
"id": "5590635c",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Linear Regression approach\n",
"In the previous week, you applied linear regression to build a prediction model. Let's try that approach here using the simple example that was described in the lecture. The model will predict if a tumor is benign or malignant based on tumor size. Try the following:\n",
"- Click on 'Run Linear Regression' to find the best linear regression model for the given data.\n",
" - Note the resulting linear model does **not** match the data well. \n",
"One option to improve the results is to apply a *threshold*. \n",
"- Tick the box on the 'Toggle 0.5 threshold' to show the predictions if a threshold is applied.\n",
" - These predictions look good, the predictions match the data\n",
"- *Important*: Now, add further 'malignant' data points on the far right, in the large tumor size range (near 10), and re-run linear regression.\n",
" - Now, the model predicts the larger tumor, but data point at x=3 is being incorrectly predicted!\n",
"- to clear/renew the plot, rerun the cell containing the plot command."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "de29056d",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"w_in = np.zeros((1))\n",
"b_in = 0\n",
"plt.close('all') \n",
"addpt = plt_one_addpt_onclick( x_train,y_train, w_in, b_in, logistic=False)"
]
},
{
"cell_type": "markdown",
"id": "4a7b4314",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"The example above demonstrates that the linear model is insufficient to model categorical data. The model can be extended as described in the following lab."
]
},
{
"cell_type": "markdown",
"id": "ab2364fc",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"In this lab you:\n",
"- explored categorical data sets and plotting\n",
"- determined that linear regression was insufficient for a classification problem."
]
},
{
"cell_type": "markdown",
"id": "9ffe5098",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"#### Optional Lab - 3.2: Logistic Regression\n",
"\n",
"In this ungraded lab, you will \n",
"- explore the sigmoid function (also known as the logistic function)\n",
"- explore logistic regression; which uses the sigmoid function"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7c92ccec",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"import numpy as np\n",
"%matplotlib widget\n",
"import matplotlib.pyplot as plt\n",
"from plt_one_addpt_onclick import plt_one_addpt_onclick\n",
"from lab_utils_common import draw_vthresh\n",
"plt.style.use('week3/OptionalLabs/deeplearning.mplstyle')"
]
},
{
"cell_type": "markdown",
"id": "39a36487",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Sigmoid or Logistic Function\n",
"![](\"week3/OptionalLabs/images/C1_W3_LogisticRegression_left.png\")
As discussed in the lecture videos, for a classification task, we can start by using our linear regression model, $f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) = \\mathbf{w} \\cdot \\mathbf{x}^{(i)} + b$, to predict $y$ given $x$. \n",
"- However, we would like the predictions of our classification model to be between 0 and 1 since our output variable $y$ is either 0 or 1. \n",
"- This can be accomplished by using a \"sigmoid function\" which maps all input values to values between 0 and 1. \n",
"\n",
"\n",
"Let's implement the sigmoid function and see this for ourselves.\n",
"\n",
"##### Formula for Sigmoid function\n",
"\n",
"The formula for a sigmoid function is as follows - \n",
"\n",
"$g(z) = \\frac{1}{1+e^{-z}}\\tag{1}$\n",
"\n",
"In the case of logistic regression, z (the input to the sigmoid function), is the output of a linear regression model. \n",
"- In the case of a single example, $z$ is scalar.\n",
"- in the case of multiple examples, $z$ may be a vector consisting of $m$ values, one for each example. \n",
"- The implementation of the sigmoid function should cover both of these potential input formats.\n",
"Let's implement this in Python."
]
},
{
"cell_type": "markdown",
"id": "256fca46",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"NumPy has a function called [`exp()`](https://numpy.org/doc/stable/reference/generated/numpy.exp.html), which offers a convenient way to calculate the exponential ( $e^{z}$) of all elements in the input array (`z`).\n",
" \n",
"It also works with a single number as an input, as shown below."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fa442685",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"# Input is an array. \n",
"input_array = np.array([1,2,3])\n",
"exp_array = np.exp(input_array)\n",
"\n",
"print(\"Input to exp:\", input_array)\n",
"print(\"Output of exp:\", exp_array)\n",
"\n",
"# Input is a single number\n",
"input_val = 1 \n",
"exp_val = np.exp(input_val)\n",
"\n",
"print(\"Input to exp:\", input_val)\n",
"print(\"Output of exp:\", exp_val)"
]
},
{
"cell_type": "markdown",
"id": "e9596531",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"The `sigmoid` function is implemented in python as shown in the cell below."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dc319850",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"def sigmoid(z):\n",
" \"\"\"\n",
" Compute the sigmoid of z\n",
"\n",
" Args:\n",
" z (ndarray): A scalar, numpy array of any size.\n",
"\n",
" Returns:\n",
" g (ndarray): sigmoid(z), with the same shape as z\n",
" \n",
" \"\"\"\n",
"\n",
" g = 1/(1+np.exp(-z))\n",
" \n",
" return g"
]
},
{
"cell_type": "markdown",
"id": "14c4ba5e",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Let's see what the output of this function is for various value of `z`"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dac07d87",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"# Generate an array of evenly spaced values between -10 and 10\n",
"z_tmp = np.arange(-10,11)\n",
"\n",
"# Use the function implemented above to get the sigmoid values\n",
"y = sigmoid(z_tmp)\n",
"\n",
"# Code for pretty printing the two arrays next to each other\n",
"np.set_printoptions(precision=3) \n",
"print(\"Input (z), Output (sigmoid(z))\")\n",
"print(np.c_[z_tmp, y])"
]
},
{
"cell_type": "markdown",
"id": "5a2287bb",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"The values in the left column are `z`, and the values in the right column are `sigmoid(z)`. As you can see, the input values to the sigmoid range from -10 to 10, and the output values range from 0 to 1. \n",
"\n",
"Now, let's try to plot this function using the `matplotlib` library."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3e9e89ad",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"# Plot z vs sigmoid(z)\n",
"fig,ax = plt.subplots(1,1,figsize=(5,3))\n",
"ax.plot(z_tmp, y, c=\"b\")\n",
"\n",
"ax.set_title(\"Sigmoid function\")\n",
"ax.set_ylabel('sigmoid(z)')\n",
"ax.set_xlabel('z')\n",
"draw_vthresh(ax,0)"
]
},
{
"cell_type": "markdown",
"id": "1ee2fdc7",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"As you can see, the sigmoid function approaches `0` as `z` goes to large negative values and approaches `1` as `z` goes to large positive values.\n"
]
},
{
"cell_type": "markdown",
"id": "5b6be211",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Logistic Regression\n",
"![](\"week3/OptionalLabs/images/C1_W3_LogisticRegression_right.png\")
A logistic regression model applies the sigmoid to the familiar linear regression model as shown below:\n",
"\n",
"$$ f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) = g(\\mathbf{w} \\cdot \\mathbf{x}^{(i)} + b ) \\tag{2} $$ \n",
"\n",
" where\n",
"\n",
" $g(z) = \\frac{1}{1+e^{-z}}\\tag{3}$\n"
]
},
{
"cell_type": "markdown",
"id": "59745ca1",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
" \n",
"Let's apply logistic regression to the categorical data example of tumor classification. \n",
"First, load the examples and initial values for the parameters.\n",
" \n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5db9b025",
"metadata": {
"pycharm": {
"name": "#%%\n"
},
"tags": []
},
"outputs": [],
"source": [
"x_train = np.array([0., 1, 2, 3, 4, 5])\n",
"y_train = np.array([0, 0, 0, 1, 1, 1])\n",
"\n",
"w_in = np.zeros((1))\n",
"b_in = 0"
]
},
{
"cell_type": "markdown",
"id": "076378af",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Try the following steps:\n",
"- Click on 'Run Logistic Regression' to find the best logistic regression model for the given training data\n",
" - Note the resulting model fits the data quite well.\n",
" - Note, the orange line is '$z$' or $\\mathbf{w} \\cdot \\mathbf{x}^{(i)} + b$ above. It does not match the line in a linear regression model.\n",
"Further improve these results by applying a *threshold*. \n",
"- Tick the box on the 'Toggle 0.5 threshold' to show the predictions if a threshold is applied.\n",
" - These predictions look good. The predictions match the data\n",
" - Now, add further data points in the large tumor size range (near 10), and re-run logistic regression.\n",
" - unlike the linear regression model, this model continues to make correct predictions"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c0f8764c",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"plt.close('all') \n",
"addpt = plt_one_addpt_onclick( x_train,y_train, w_in, b_in, logistic=True)"
]
},
{
"cell_type": "markdown",
"id": "60ac40e2",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"You have explored the use of the sigmoid function in logistic regression."
]
},
{
"cell_type": "markdown",
"id": "c164236e",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"#### Optional Lab - 3.3: Logistic Regression, Decision Boundary\n"
]
},
{
"cell_type": "markdown",
"id": "7f045e80",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Goals\n",
"In this lab, you will:\n",
"- Plot the decision boundary for a logistic regression model. This will give you a better sense of what the model is predicting.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f69c660a",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"import numpy as np\n",
"%matplotlib widget\n",
"import matplotlib.pyplot as plt\n",
"from lab_utils_common import plot_data, sigmoid, draw_vthresh\n",
"#plt.style.use('week3/OptionalLabs/deeplearning.mplstyle')"
]
},
{
"cell_type": "markdown",
"id": "1205b0c6",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Dataset\n",
"\n",
"Let's suppose you have following training dataset\n",
"- The input variable `X` is a numpy array which has 6 training examples, each with two features\n",
"- The output variable `y` is also a numpy array with 6 examples, and `y` is either `0` or `1`"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3642026f",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"X = np.array([[0.5, 1.5], [1,1], [1.5, 0.5], [3, 0.5], [2, 2], [1, 2.5]])\n",
"y = np.array([0, 0, 0, 1, 1, 1]).reshape(-1,1) "
]
},
{
"cell_type": "markdown",
"id": "8be4a974",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Plot data \n",
"\n",
"Let's use a helper function to plot this data. The data points with label $y=1$ are shown as red crosses, while the data points with label $y=0$ are shown as blue circles. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "199f1848",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"fig,ax = plt.subplots(1,1,figsize=(4,4))\n",
"plot_data(X, y, ax)\n",
"\n",
"ax.axis([0, 4, 0, 3.5])\n",
"ax.set_ylabel('$x_1$')\n",
"ax.set_xlabel('$x_0$')\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "74003f78",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Logistic regression model\n",
"\n",
"\n",
"* Suppose you'd like to train a logistic regression model on this data which has the form \n",
"\n",
" $f(x) = g(w_0x_0+w_1x_1 + b)$\n",
" \n",
" where $g(z) = \\frac{1}{1+e^{-z}}$, which is the sigmoid function\n",
"\n",
"\n",
"* Let's say that you trained the model and get the parameters as $b = -3, w_0 = 1, w_1 = 1$. That is,\n",
"\n",
" $f(x) = g(x_0+x_1-3)$\n",
"\n",
" (You'll learn how to fit these parameters to the data further in the course)\n",
" \n",
" \n",
"Let's try to understand what this trained model is predicting by plotting its decision boundary"
]
},
{
"cell_type": "markdown",
"id": "c287e48f",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Refresher on logistic regression and decision boundary\n",
"\n",
"* Recall that for logistic regression, the model is represented as \n",
"\n",
" $$f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) = g(\\mathbf{w} \\cdot \\mathbf{x}^{(i)} + b) \\tag{1}$$\n",
"\n",
" where $g(z)$ is known as the sigmoid function and it maps all input values to values between 0 and 1:\n",
"\n",
" $g(z) = \\frac{1}{1+e^{-z}}\\tag{2}$\n",
" and $\\mathbf{w} \\cdot \\mathbf{x}$ is the vector dot product:\n",
" \n",
" $$\\mathbf{w} \\cdot \\mathbf{x} = w_0 x_0 + w_1 x_1$$\n",
" \n",
" \n",
" * We interpret the output of the model ($f_{\\mathbf{w},b}(x)$) as the probability that $y=1$ given $\\mathbf{x}$ and parameterized by $\\mathbf{w}$ and $b$.\n",
"* Therefore, to get a final prediction ($y=0$ or $y=1$) from the logistic regression model, we can use the following heuristic -\n",
"\n",
" if $f_{\\mathbf{w},b}(x) >= 0.5$, predict $y=1$\n",
" \n",
" if $f_{\\mathbf{w},b}(x) < 0.5$, predict $y=0$\n",
" \n",
" \n",
"* Let's plot the sigmoid function to see where $g(z) >= 0.5$"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "06536178",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"# Plot sigmoid(z) over a range of values from -10 to 10\n",
"z = np.arange(-10,11)\n",
"\n",
"fig,ax = plt.subplots(1,1,figsize=(5,3))\n",
"# Plot z vs sigmoid(z)\n",
"ax.plot(z, sigmoid(z), c=\"b\")\n",
"\n",
"ax.set_title(\"Sigmoid function\")\n",
"ax.set_ylabel('sigmoid(z)')\n",
"ax.set_xlabel('z')\n",
"draw_vthresh(ax,0)"
]
},
{
"cell_type": "markdown",
"id": "31154efa",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"* As you can see, $g(z) >= 0.5$ for $z >=0$\n",
"\n",
"* For a logistic regression model, $z = \\mathbf{w} \\cdot \\mathbf{x} + b$. Therefore,\n",
"\n",
" if $\\mathbf{w} \\cdot \\mathbf{x} + b >= 0$, the model predicts $y=1$\n",
" \n",
" if $\\mathbf{w} \\cdot \\mathbf{x} + b < 0$, the model predicts $y=0$\n",
" \n",
" \n",
" \n",
"##### Plotting decision boundary\n",
"\n",
"Now, let's go back to our example to understand how the logistic regression model is making predictions.\n",
"\n",
"* Our logistic regression model has the form\n",
"\n",
" $f(\\mathbf{x}) = g(-3 + x_0+x_1)$\n",
"\n",
"\n",
"* From what you've learnt above, you can see that this model predicts $y=1$ if $-3 + x_0+x_1 >= 0$\n",
"\n",
"Let's see what this looks like graphically. We'll start by plotting $-3 + x_0+x_1 = 0$, which is equivalent to $x_1 = 3 - x_0$.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7575f8d5",
"metadata": {},
"outputs": [],
"source": [
"#plotting some decision boundry\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"x0 = np.arange(0,2.1,0.01)\n",
"\n",
"x1 = np.sqrt(4 - x0**2)\n",
"fig,ax = plt.subplots(1,1,figsize=(5,4))\n",
"# Plot the decision boundary\n",
"ax.plot(x0,x1, c=\"b\")\n",
"ax.axis([0, 4, 0, 4])\n",
"\n",
"# Fill the region below the line\n",
"ax.fill_between(x0,x1, alpha=0.2)\n",
"\n",
"# Plot the original data\n",
"ax.set_ylabel(r'$x_1$')\n",
"ax.set_xlabel(r'$x_0$')\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b33eeb88",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"# Choose values between 0 and 6\n",
"x0 = np.arange(0,6)\n",
"\n",
"x1 = 3 - x0\n",
"fig,ax = plt.subplots(1,1,figsize=(5,4))\n",
"# Plot the decision boundary\n",
"ax.plot(x0,x1, c=\"b\")\n",
"ax.axis([0, 4, 0, 3.5])\n",
"\n",
"# Fill the region below the line\n",
"ax.fill_between(x0,x1, alpha=0.2)\n",
"\n",
"# Plot the original data\n",
"plot_data(X,y,ax)\n",
"ax.set_ylabel(r'$x_1$')\n",
"ax.set_xlabel(r'$x_0$')\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "4b86da6b",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"* In the plot above, the blue line represents the line $x_0 + x_1 - 3 = 0$ and it should intersect the x1 axis at 3 (if we set $x_1$ = 3, $x_0$ = 0) and the x0 axis at 3 (if we set $x_1$ = 0, $x_0$ = 3). \n",
"\n",
"\n",
"* The shaded region represents $-3 + x_0+x_1 < 0$. The region above the line is $-3 + x_0+x_1 > 0$.\n",
"\n",
"\n",
"* Any point in the shaded region (under the line) is classified as $y=0$. Any point on or above the line is classified as $y=1$. This line is known as the \"decision boundary\".\n",
"\n",
"As we've seen in the lectures, by using higher order polynomial terms (eg: $f(x) = g( x_0^2 + x_1 -1)$, we can come up with more complex non-linear boundaries."
]
},
{
"cell_type": "markdown",
"id": "38168e17",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"You have explored the decision boundary in the context of logistic regression."
]
},
{
"cell_type": "markdown",
"id": "976d73e3",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"#### Optional Lab - 3.4: Logistic Regression, Logistic Loss\n",
"\n",
"In this ungraded lab, you will:\n",
"- explore the reason the squared error loss is not appropriate for logistic regression\n",
"- explore the logistic loss function"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bf9a0b64",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"id": "d02e05b0",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"import numpy as np\n",
"%matplotlib widget\n",
"import matplotlib.pyplot as plt\n",
"from plt_logistic_loss import plt_logistic_cost, plt_two_logistic_loss_curves, plt_simple_example\n",
"from plt_logistic_loss import soup_bowl, plt_logistic_squared_error\n",
"plt.style.use('week3/OptionalLabs/deeplearning.mplstyle')"
]
},
{
"cell_type": "markdown",
"id": "3be5d79c",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Squared error for logistic regression?\n",
"![](\"week3/OptionalLabs/images/C1_W3_SqErrorVsLogistic.png\")
Recall for **Linear** Regression we have used the **squared error cost function**:\n",
"The equation for the squared error cost with one variable is:\n",
" $$J(w,b) = \\frac{1}{2m} \\sum\\limits_{i = 0}^{m-1} (f_{w,b}(x^{(i)}) - y^{(i)})^2 \\tag{1}$$ \n",
" \n",
"where \n",
" $$f_{w,b}(x^{(i)}) = wx^{(i)} + b \\tag{2}$$\n"
]
},
{
"cell_type": "markdown",
"id": "5f552ee0",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Recall, the squared error cost had the nice property that following the derivative of the cost leads to the minimum."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3f97dafd",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"soup_bowl()"
]
},
{
"cell_type": "markdown",
"id": "01265df7",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"This cost function worked well for linear regression, it is natural to consider it for logistic regression as well. However, as the slide above points out, $f_{wb}(x)$ now has a non-linear component, the sigmoid function: $f_{w,b}(x^{(i)}) = sigmoid(wx^{(i)} + b )$. Let's try a squared error cost on the example from an earlier lab, now including the sigmoid.\n",
"\n",
"Here is our training data:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d7a0079c",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"x_train = np.array([0., 1, 2, 3, 4, 5],dtype=np.longdouble)\n",
"y_train = np.array([0, 0, 0, 1, 1, 1],dtype=np.longdouble)\n",
"plt_simple_example(x_train, y_train)"
]
},
{
"cell_type": "markdown",
"id": "5ef9cf54",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Now, let's get a surface plot of the cost using a *squared error cost*:\n",
" $$J(w,b) = \\frac{1}{2m} \\sum\\limits_{i = 0}^{m-1} (f_{w,b}(x^{(i)}) - y^{(i)})^2 $$ \n",
" \n",
"where \n",
" $$f_{w,b}(x^{(i)}) = sigmoid(wx^{(i)} + b )$$\n"
]
},
{
"cell_type": "markdown",
"id": "5b9b2478",
"metadata": {},
"source": [
"###### Plot logistic squared error "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3c78eb3c",
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"from matplotlib import cm\n",
"x_train = np.array([0., 1, 2, 3, 4, 5],dtype=np.longdouble)\n",
"y_train = np.array([0, 0, 0, 1, 1, 1],dtype=np.longdouble)\n",
"\n",
"wx,by=np.meshgrid(np.linspace(-6,12,100),np.linspace(10,-20,100))\n",
"\n",
"\n",
"def logistic_model(x,w,b):\n",
" return 1/(1+np.exp(-(w*x+b)))\n",
"\n",
"def cost_fn_logistic(x,w,b,y):\n",
" return np.sum((logistic_model(x,w,b)-y)**2)/2/len(x)\n",
"\n",
"\n",
"cost_f=np.zeros(wx.shape)\n",
"for wi in range(wx.shape[0]):\n",
" for wj in range(wx.shape[1]):\n",
" w,b=wx[wi,wj],by[wi,wj]\n",
" #print(cost_fn_logistic(x_train,w,b,y_train))\n",
" cost_f[wi,wj]=cost_fn_logistic(x_train,w,b,y_train)\n",
" \n",
" \n",
"fig = plt.figure()\n",
"fig.canvas.toolbar_visible = False\n",
"fig.canvas.header_visible = False\n",
"fig.canvas.footer_visible = False\n",
"ax = fig.add_subplot(1, 1, 1, projection='3d')\n",
"ax.plot_surface(wx, by, cost_f, alpha=0.6,cmap=cm.coolwarm)\n",
" \n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "10726c44",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"plt.close('all')\n",
"plt_logistic_squared_error(x_train,y_train)\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "a8165594",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"While this produces a pretty interesting plot, the surface above not nearly as smooth as the 'soup bowl' from linear regression! \n",
"\n",
"Logistic regression requires a cost function more suitable to its non-linear nature. This starts with a Loss function. This is described below."
]
},
{
"cell_type": "markdown",
"id": "c6b0b89a",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Logistic Loss Function\n",
"![](\"week3/OptionalLabs/images/C1_W3_LogisticLoss_a.png\")
\n",
"![](\"week3/OptionalLabs/images/C1_W3_LogisticLoss_b.png\")
\n",
"![](\"week3/OptionalLabs/images/C1_W3_LogisticLoss_c.png\")
"
]
},
{
"cell_type": "markdown",
"id": "5f552d09",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Logistic Regression uses a loss function more suited to the task of categorization where the target is 0 or 1 rather than any number. \n",
"\n",
">**Definition Note:** In this course, these definitions are used: \n",
"**Loss** is a measure of the difference of a single example to its target value while the \n",
"**Cost** is a measure of the losses over the training set\n",
"\n",
"\n",
"This is defined: \n",
"* $loss(f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}), y^{(i)})$ is the cost for a single data point, which is:\n",
"\n",
"\\begin{equation}\n",
" loss(f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}), y^{(i)}) = \\begin{cases}\n",
" - \\log\\left(f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right) & \\text{if $y^{(i)}=1$}\\\\\n",
" - \\log \\left( 1 - f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right) & \\text{if $y^{(i)}=0$}\n",
" \\end{cases}\n",
"\\end{equation}\n",
"\n",
"\n",
"* $f_{\\mathbf{w},b}(\\mathbf{x}^{(i)})$ is the model's prediction, while $y^{(i)}$ is the target value.\n",
"\n",
"* $f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) = g(\\mathbf{w} \\cdot\\mathbf{x}^{(i)}+b)$ where function $g$ is the sigmoid function.\n",
"\n",
"The defining feature of this loss function is the fact that it uses two separate curves. One for the case when the target is zero or ($y=0$) and another for when the target is one ($y=1$). Combined, these curves provide the behavior useful for a loss function, namely, being zero when the prediction matches the target and rapidly increasing in value as the prediction differs from the target. Consider the curves below:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "edb72317",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"plt_two_logistic_loss_curves()"
]
},
{
"cell_type": "markdown",
"id": "277f5476",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Combined, the curves are similar to the quadratic curve of the squared error loss. Note, the x-axis is $f_{\\mathbf{w},b}$ which is the output of a sigmoid. The sigmoid output is strictly between 0 and 1."
]
},
{
"cell_type": "markdown",
"id": "f5c96b6e",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"The loss function above can be rewritten to be easier to implement.\n",
" $$loss(f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}), y^{(i)}) = (-y^{(i)} \\log\\left(f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right) - \\left( 1 - y^{(i)}\\right) \\log \\left( 1 - f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right)$$\n",
" \n",
"This is a rather formidable-looking equation. It is less daunting when you consider $y^{(i)}$ can have only two values, 0 and 1. One can then consider the equation in two pieces: \n",
"when $ y^{(i)} = 0$, the left-hand term is eliminated:\n",
"$$\n",
"\\begin{align}\n",
"loss(f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}), 0) &= (-(0) \\log\\left(f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right) - \\left( 1 - 0\\right) \\log \\left( 1 - f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right) \\\\\n",
"&= -\\log \\left( 1 - f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right)\n",
"\\end{align}\n",
"$$\n",
"and when $ y^{(i)} = 1$, the right-hand term is eliminated:\n",
"$$\n",
"\\begin{align}\n",
" loss(f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}), 1) &= (-(1) \\log\\left(f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right) - \\left( 1 - 1\\right) \\log \\left( 1 - f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right)\\\\\n",
" &= -\\log\\left(f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right)\n",
"\\end{align}\n",
"$$\n",
"\n",
"OK, with this new logistic loss function, a cost function can be produced that incorporates the loss from all the examples. This will be the topic of the next lab. For now, let's take a look at the cost vs parameters curve for the simple example we considered above:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4022f38d",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"plt.close('all')\n",
"cst = plt_logistic_cost(x_train,y_train)"
]
},
{
"cell_type": "markdown",
"id": "beb51ef1",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"This curve is well suited to gradient descent! It does not have plateaus, local minima, or discontinuities. Note, it is not a bowl as in the case of squared error. Both the cost and the log of the cost are plotted to illuminate the fact that the curve, when the cost is small, has a slope and continues to decline. Reminder: you can rotate the above plots using your mouse."
]
},
{
"cell_type": "markdown",
"id": "5b367649",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"You have:\n",
" - determined a squared error loss function is not suitable for classification tasks\n",
" - developed and examined the logistic loss function which **is** suitable for classification tasks.\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "5d717b07",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"#### Optional Lab - 3.5: Cost Function for Logistic Regression\n",
"\n",
"##### Goals\n",
"In this lab, you will:\n",
"- examine the implementation and utilize the cost function for logistic regression."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4a4fdc5e",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"import numpy as np\n",
"%matplotlib widget\n",
"import matplotlib.pyplot as plt\n",
"from lab_utils_common import plot_data, sigmoid, dlc\n",
"plt.style.use('week3/OptionalLabs/deeplearning.mplstyle')"
]
},
{
"cell_type": "markdown",
"id": "c075181d",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"###### Dataset \n",
"Let's start with the same dataset as was used in the decision boundary lab."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6ef2e3dc",
"metadata": {
"pycharm": {
"name": "#%%\n"
},
"tags": []
},
"outputs": [],
"source": [
"X_train = np.array([[0.5, 1.5], [1,1], [1.5, 0.5], [3, 0.5], [2, 2], [1, 2.5]]) #(m,n)\n",
"y_train = np.array([0, 0, 0, 1, 1, 1]) #(m,)"
]
},
{
"cell_type": "markdown",
"id": "d3d3cba7",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"We will use a helper function to plot this data. The data points with label $y=1$ are shown as red crosses, while the data points with label $y=0$ are shown as blue circles."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bf605c65",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"fig,ax = plt.subplots(1,1,figsize=(4,4))\n",
"plot_data(X_train, y_train, ax)\n",
"\n",
"# Set both axes to be from 0-4\n",
"ax.axis([0, 4, 0, 3.5])\n",
"ax.set_ylabel('$x_1$', fontsize=12)\n",
"ax.set_xlabel('$x_0$', fontsize=12)\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "f22e896e",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Cost function\n",
"\n",
"In a previous lab, you developed the *logistic loss* function. Recall, loss is defined to apply to one example. Here you combine the losses to form the **cost**, which includes all the examples.\n",
"\n",
"\n",
"Recall that for logistic regression, the cost function is of the form \n",
"\n",
"$$ J(\\mathbf{w},b) = \\frac{1}{m} \\sum_{i=0}^{m-1} \\left[ loss(f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}), y^{(i)}) \\right] \\tag{1}$$\n",
"\n",
"where\n",
"* $loss(f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}), y^{(i)})$ is the cost for a single data point, which is:\n",
"\n",
" $$loss(f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}), y^{(i)}) = -y^{(i)} \\log\\left(f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right) - \\left( 1 - y^{(i)}\\right) \\log \\left( 1 - f_{\\mathbf{w},b}\\left( \\mathbf{x}^{(i)} \\right) \\right) \\tag{2}$$\n",
" \n",
"* where m is the number of training examples in the data set and:\n",
"$$\n",
"\\begin{align}\n",
" f_{\\mathbf{w},b}(\\mathbf{x^{(i)}}) &= g(z^{(i)})\\tag{3} \\\\\n",
" z^{(i)} &= \\mathbf{w} \\cdot \\mathbf{x}^{(i)}+ b\\tag{4} \\\\\n",
" g(z^{(i)}) &= \\frac{1}{1+e^{-z^{(i)}}}\\tag{5} \n",
"\\end{align}\n",
"$$\n",
" "
]
},
{
"cell_type": "markdown",
"id": "6c505662",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Code Description\n",
"\n",
"\n",
"\n",
"The algorithm for `compute_cost_logistic` loops over all the examples calculating the loss for each example and accumulating the total.\n",
"\n",
"Note that the variables X and y are not scalar values but matrices of shape ($m, n$) and ($𝑚$,) respectively, where $𝑛$ is the number of features and $𝑚$ is the number of training examples.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "22a5ddbe",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"def compute_cost_logistic(X, y, w, b):\n",
" \"\"\"\n",
" Computes cost\n",
"\n",
" Args:\n",
" X (ndarray (m,n)): Data, m examples with n features\n",
" y (ndarray (m,)) : target values\n",
" w (ndarray (n,)) : model parameters \n",
" b (scalar) : model parameter\n",
" \n",
" Returns:\n",
" cost (scalar): cost\n",
" \"\"\"\n",
"\n",
" m = X.shape[0]\n",
" cost = 0.0\n",
" for i in range(m):\n",
" z_i = np.dot(X[i],w) + b\n",
" f_wb_i = sigmoid(z_i)\n",
" cost += -y[i]*np.log(f_wb_i) - (1-y[i])*np.log(1-f_wb_i)\n",
" \n",
" cost = cost / m\n",
" return cost\n"
]
},
{
"cell_type": "markdown",
"id": "180dec69",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Check the implementation of the cost function using the cell below."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f2bb9ca6",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"w_tmp = np.array([1,1])\n",
"b_tmp = -3\n",
"print(compute_cost_logistic(X_train, y_train, w_tmp, b_tmp))"
]
},
{
"cell_type": "markdown",
"id": "0d1f4ef7",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"**Expected output**: 0.3668667864055175"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7315c0e3",
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"from matplotlib import cm\n",
"\n",
"wx,by=np.meshgrid(np.linspace(-6,12,100),np.linspace(10,-20,100))\n",
"\n",
"\n",
"def logistic_model(x,w,b):\n",
" return np.array([ (1/(1+np.exp(-(np.dot(w,i)+b)))) for i in x])\n",
" \n",
"def cost_fn_logistic(x,w,b,y):\n",
" return np.sum(-y*np.log(logistic_model(x,w,b))-(1-y)*np.log(1-logistic_model(x,w,b)))/len(x)\n",
"cost_fn_logistic(X_train,np.array([1,1]),-4,y_train)\n"
]
},
{
"cell_type": "markdown",
"id": "ac31da9f",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Example\n",
"Now, let's see what the cost function output is for a different value of $w$. \n",
"\n",
"* In a previous lab, you plotted the decision boundary for $b = -3, w_0 = 1, w_1 = 1$. That is, you had `b = -3, w = np.array([1,1])`.\n",
"\n",
"* Let's say you want to see if $b = -4, w_0 = 1, w_1 = 1$, or `b = -4, w = np.array([1,1])` provides a better model.\n",
"\n",
"Let's first plot the decision boundary for these two different $b$ values to see which one fits the data better.\n",
"\n",
"* For $b = -3, w_0 = 1, w_1 = 1$, we'll plot $-3 + x_0+x_1 = 0$ (shown in blue)\n",
"* For $b = -4, w_0 = 1, w_1 = 1$, we'll plot $-4 + x_0+x_1 = 0$ (shown in magenta)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dfff9371",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"\n",
"# Choose values between 0 and 6\n",
"x0 = np.arange(0,6)\n",
"\n",
"# Plot the two decision boundaries\n",
"x1 = 3 - x0\n",
"x1_other = 4 - x0\n",
"\n",
"fig,ax = plt.subplots(1, 1, figsize=(4,4))\n",
"# Plot the decision boundary\n",
"ax.plot(x0,x1, c=dlc[\"dlblue\"], label=\"$b$=-3\")\n",
"ax.plot(x0,x1_other, c=dlc[\"dlmagenta\"], label=\"$b$=-4\")\n",
"ax.axis([0, 4, 0, 4])\n",
"\n",
"# Plot the original data\n",
"plot_data(X_train,y_train,ax)\n",
"ax.axis([0, 4, 0, 4])\n",
"ax.set_ylabel('$x_1$', fontsize=12)\n",
"ax.set_xlabel('$x_0$', fontsize=12)\n",
"plt.legend(loc=\"upper right\")\n",
"plt.title(\"Decision Boundary\")\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "c30b9c56",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"You can see from this plot that `b = -4, w = np.array([1,1])` is a worse model for the training data. Let's see if the cost function implementation reflects this."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4d2a4e61",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"w_array1 = np.array([1,1])\n",
"b_1 = -3\n",
"w_array2 = np.array([1,1])\n",
"b_2 = -4\n",
"\n",
"print(\"Cost for b = -3 : \", compute_cost_logistic(X_train, y_train, w_array1, b_1))\n",
"print(\"Cost for b = -4 : \", compute_cost_logistic(X_train, y_train, w_array2, b_2))"
]
},
{
"cell_type": "markdown",
"id": "68a25495",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"**Expected output**\n",
"\n",
"Cost for b = -3 : 0.3668667864055175\n",
"\n",
"Cost for b = -4 : 0.5036808636748461\n",
"\n",
"\n",
"You can see the cost function behaves as expected and the cost for `b = -4, w = np.array([1,1])` is indeed higher than the cost for `b = -3, w = np.array([1,1])`"
]
},
{
"cell_type": "markdown",
"id": "8fb5b5be",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"In this lab you examined and utilized the cost function for logistic regression."
]
},
{
"cell_type": "markdown",
"id": "6beb4eb1",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"#### Optional Lab - 3.6: Gradient Descent for Logistic Regression"
]
},
{
"cell_type": "markdown",
"id": "984999b1",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Goals\n",
"In this lab, you will:\n",
"- update gradient descent for logistic regression.\n",
"- explore gradient descent on a familiar data set"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0247ebcd",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"import copy, math\n",
"import numpy as np\n",
"%matplotlib widget\n",
"import matplotlib.pyplot as plt\n",
"from lab_utils_common import dlc, plot_data, plt_tumor_data, sigmoid, compute_cost_logistic\n",
"from plt_quad_logistic import plt_quad_logistic, plt_prob\n",
"plt.style.use('week3/OptionalLabs/deeplearning.mplstyle')"
]
},
{
"cell_type": "markdown",
"id": "7a1e36bb",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"###### Data set \n",
"Let's start with the same two feature data set used in the decision boundary lab."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4fc1fdc5",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"X_train = np.array([[0.5, 1.5], [1,1], [1.5, 0.5], [3, 0.5], [2, 2], [1, 2.5]])\n",
"y_train = np.array([0, 0, 0, 1, 1, 1])"
]
},
{
"cell_type": "markdown",
"id": "8df6592b",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"As before, we'll use a helper function to plot this data. The data points with label $y=1$ are shown as red crosses, while the data points with label $y=0$ are shown as blue circles."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bd3a0530",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"fig,ax = plt.subplots(1,1,figsize=(4,4))\n",
"plot_data(X_train, y_train, ax)\n",
"\n",
"ax.axis([0, 4, 0, 3.5])\n",
"ax.set_ylabel('$x_1$', fontsize=12)\n",
"ax.set_xlabel('$x_0$', fontsize=12)\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "f893d122",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Logistic Gradient Descent\n",
"![](\"week3/OptionalLabs/images/C1_W3_Logistic_gradient_descent.png\")
\n",
"\n",
"Recall the gradient descent algorithm utilizes the gradient calculation:\n",
"$$\\begin{align*}\n",
"&\\text{repeat until convergence:} \\; \\lbrace \\\\\n",
"& \\; \\; \\;w_j = w_j - \\alpha \\frac{\\partial J(\\mathbf{w},b)}{\\partial w_j} \\tag{1} \\; & \\text{for j := 0..n-1} \\\\ \n",
"& \\; \\; \\; \\; \\;b = b - \\alpha \\frac{\\partial J(\\mathbf{w},b)}{\\partial b} \\\\\n",
"&\\rbrace\n",
"\\end{align*}$$\n",
"\n",
"Where each iteration performs simultaneous updates on $w_j$ for all $j$, where\n",
"$$\\begin{align*}\n",
"\\frac{\\partial J(\\mathbf{w},b)}{\\partial w_j} &= \\frac{1}{m} \\sum\\limits_{i = 0}^{m-1} (f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) - y^{(i)})x_{j}^{(i)} \\tag{2} \\\\\n",
"\\frac{\\partial J(\\mathbf{w},b)}{\\partial b} &= \\frac{1}{m} \\sum\\limits_{i = 0}^{m-1} (f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) - y^{(i)}) \\tag{3} \n",
"\\end{align*}$$\n",
"\n",
"* m is the number of training examples in the data set \n",
"* $f_{\\mathbf{w},b}(x^{(i)})$ is the model's prediction, while $y^{(i)}$ is the target\n",
"* For a logistic regression model \n",
" $z = \\mathbf{w} \\cdot \\mathbf{x} + b$ \n",
" $f_{\\mathbf{w},b}(x) = g(z)$ \n",
" where $g(z)$ is the sigmoid function: \n",
" $g(z) = \\frac{1}{1+e^{-z}}$ \n",
" \n"
]
},
{
"cell_type": "markdown",
"id": "8dad45d1",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Gradient Descent Implementation\n",
"The gradient descent algorithm implementation has two components: \n",
"- The loop implementing equation (1) above. This is `gradient_descent` below and is generally provided to you in optional and practice labs.\n",
"- The calculation of the current gradient, equations (2,3) above. This is `compute_gradient_logistic` below. You will be asked to implement this week's practice lab.\n",
"\n",
"###### Calculating the Gradient, Code Description\n",
"Implements equation (2),(3) above for all $w_j$ and $b$.\n",
"There are many ways to implement this. Outlined below is this:\n",
"- initialize variables to accumulate `dj_dw` and `dj_db`\n",
"- for each example\n",
" - calculate the error for that example $g(\\mathbf{w} \\cdot \\mathbf{x}^{(i)} + b) - \\mathbf{y}^{(i)}$\n",
" - for each input value $x_{j}^{(i)}$ in this example, \n",
" - multiply the error by the input $x_{j}^{(i)}$, and add to the corresponding element of `dj_dw`. (equation 2 above)\n",
" - add the error to `dj_db` (equation 3 above)\n",
"\n",
"- divide `dj_db` and `dj_dw` by total number of examples (m)\n",
"- note that $\\mathbf{x}^{(i)}$ in numpy `X[i,:]` or `X[i]` and $x_{j}^{(i)}$ is `X[i,j]`"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ded2e5ce",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"id": "2e692682",
"metadata": {},
"source": [
"#### My solution\n",
"\n",
"![](\"week3/OptionalLabs/images/C1_W3_Overfitting_a.png\")
\n",
"![](\"week3/OptionalLabs/images/C1_W3_Overfitting_b.png\")
\n",
"![](\"week3/OptionalLabs/images/C1_W3_Overfitting_c.png\")
\n",
"\n",
"###### Goals\n",
"In this lab, you will explore:\n",
"- the situations where overfitting can occur\n",
"- some of the solutions\n",
"\n",
"\n",
"##### Overfitting\n",
"The week's lecture described situations where overfitting can arise. Run the cell below to generate a plot that will allow you to explore overfitting. There are further instructions below the cell.\n",
"\n",
" ```python\n",
"plt.close(\"all\")\n",
"display(output)\n",
"ofit = overfit_example(False)\n",
" ```\n",
"In the plot above you can:\n",
"- switch between Regression and Categorization examples\n",
"- add data\n",
"- select the degree of the model\n",
"- fit the model to the data \n",
"\n",
"Here are some things you should try:\n",
"- Fit the data with degree = 1; Note 'underfitting'.\n",
"- Fit the data with degree = 6; Note 'overfitting'\n",
"- tune degree to get the 'best fit'\n",
"- add data:\n",
" - extreme examples can increase overfitting (assuming they are outliers).\n",
" - nominal examples can reduce overfitting\n",
"- switch between `Regression` and `Categorical` to try both examples.\n",
"\n",
"To reset the plot, re-run the cell. Click slowly to allow the plot to update before receiving the next click.\n",
"\n",
"Notes on implementations:\n",
"- the 'ideal' curves represent the generator model to which noise was added to achieve the data set\n",
"- 'fit' does not use pure gradient descent to improve speed. These methods can be used on smaller data sets. \n",
"\n",
"You have developed some intuition about the causes and solutions to overfitting. In the next lab, you will explore a commonly used solution, Regularization.\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "6428c6a2",
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"%matplotlib widget\n",
"import matplotlib.pyplot as plt\n",
"from ipywidgets import Output\n",
"import sys\n",
"sys.path.append(\"week3/OptionalLabs\")\n",
"plt.style.use('week3/OptionalLabs/deeplearning.mplstyle')\n",
"from plt_overfit import overfit_example, output"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "5b657b94",
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "af12f81533e941d494086be48ae8e298",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Output()"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "c4c372a238644bdda01e4f48e2a06862",
"version_major": 2,
"version_minor": 0
},
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAyAAAAJYCAYAAACadoJwAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAA9hAAAPYQGoP6dpAAC1DElEQVR4nOzdd3QUVRvH8W9CQmiBEBJISKU3Q0c6UqRXEQUFRERUUIqI2MVXkaKgqAiigg1ULEgRaSJIUwEpiog0Q+8QSAglZd8/rgkpuymw2d0kv885eyBzZ3eenZ2dnWduc4uKirIgIiIiIiLiAO7ODkBERERERPIPJSAiIiIiIuIwSkBERERERMRhlICIiIiIiIjDKAERERERERGHUQIiIiIiIiIOowREREREREQcRgmIiIiIiIg4jBIQERERERFxGCUgIiIiIiLiMEpARERERETEYZSAiIiIiIiIwygBERERERERh1ECIiIiIiIiDqMEREREREREHEYJiIiIiIiIOIwSEBERERERcRglICIiIiIi4jBKQERERERExGGUgIiIiIiIiMMoAREREREREYdRAiIiIiIiIg6jBERERERERBxGCYiIiIiIiDiMEhAREREREXEYJSAiIiIiIuIwSkBERERERMRhlICIiIiIiIjDKAERERERERGHUQIiIiIiIiIOowREREREREQcRgmIiIiIiIg4jBIQERERERFxGCUgIiIiIiLiMEpARERERETEYZSAiIiIiIiIwygBERERERERh1ECIiIiIiIiDqMEREREREREHEYJiIiIiIiIOIwSEBERERERcRglICIiIiIi4jBKQERERERExGGUgIiIiIiIiMMoAREREREREYdRAiIiIiIiIg6jBERERERERBxGCYiIiIiIiDiMEhAREREREXEYJSAiIiIiIuIwSkBERERERMRhlICIZMFPP/1Er169KFeuHGXKlKFevXqMHTuWqKgop8XUuXNnfHx80j2qV6+eXD5kyJDk9detW8eECRNITExM9ToHDx5kwoQJREZGpttGREQEEyZMyNH3YY2195X0+P777x0ejz2l/VxySnR0NK+//jrNmjUjKCiIMmXKcOutt/LMM89w+PDhbL1WVFQUEyZMYPv27TkTrB2tW7cOHx8fDh486HKv7ePjw9y5c7P9vOnTp7No0aIb2mZOs3Vesae056Hvv/+eadOmWY3Fx8eHNWvW5FgsWXXw4EF8fHxYt26ds0MRcUkezg5AxNVNmTKFV155hc6dO/P2229TsmRJtm/fzltvvcWiRYtYvHgxwcHBTomtRo0aTJ06NdWyggULAiZuLy+v5OXr169n0qRJPPnkk7i7X7/3cOjQISZNmkTjxo0JDw9P9Vpz5szBz88vx+LPyL333svAgQPTLa9UqZITosldjh07Ro8ePTh16hQPPfQQjRo1wsPDg127dvHpp5/y559/ZiuRu3DhApMmTSIoKIjatWvnXOB2UKtWLVauXElAQICzQ7GbGTNm0LhxY7p16+bsUNKxdV6xp7TnoSVLlvDzzz/z2GOP5cj27CEgIICVK1dSpUoVZ4ci4pKUgIhkYO3atYwbN44hQ4akugPXrFkzunTpQsuWLXnkkUccelf+6tWryYmFt7c3DRo0sLpe1apVb3pbtWrVuunXuFFly5a1+d4kYw8//DCnTp3ip59+onz58snLW7RowUMPPcSyZcucGF3OSEhIwGKxULx4cR03eYwzz0M3ysvLS8ehSAbUBEskA0k1HmPHjk1XFh4ezuOPP8769evZsmULAA0bNqR///7p1t2yZQs+Pj6pLvz+/PNP+vTpQ1hYGAEBAbRv356NGzemet6QIUOoXr06mzZtol27dgQEBPDiiy9mKfaUTX0mTJjApEmTAPDz80tuzrRu3Tq6du0KQI8ePVIth/RNHyZMmICPjw/79u3jzjvvpGzZstSqVYsZM2ak2/6aNWto3rw5ZcqUoU6dOnz66acMGTKEzp07Zyn+zHz66afpmmQlJCTQoUMHateuTXR0NAAHDhzgoYceombNmgQEBFCrVi1GjRqVrvlc0r7etm1b8r6uX78+y5cvB2DatGlEREQQEhLCPffcw5kzZ1I938fHh1deeYXJkydTvXp1AgIC6NixI3/88Uem7yUyMpLBgwdToUIFSpcuTbNmzVi8ePEN7ZctW7awbt06nnjiiVTJRxJ3d3c6deqU/Pe3335L165dqVChAkFBQTRv3pzPP/88ufzgwYPJF4DDhw9PPkZSNiVatGgRt99+O4GBgYSGhjJgwIB0zbxiY2MZNWoU5cqVIzg4mL59+/Lbb7+la6ZisVh49913qV+/Pv7+/lSpUoUnn3ySixcvpnq9pP395ptvUrNmTfz9/fnrr79sNpP65JNPaNGiBQEBAYSFhdGpUyd+++235PLx48fTokULQkNDKV++PF27dmXz5s3Z2fXJEhISGDduHFWqVCEwMJDOnTvz999/p1tv69at3HfffcnHS/369Xn55Ze5fPly8joREREcPnyYr776KnnfJ32vs3psW7Nv3z769u1LxYoVKVOmDLfccgsDBgwgPj4+eZ2zZ88yatQoqlWrRunSpWnQoAEff/xxcrmt84otvXv3pnv37sl/JyYmEh4eTunSpYmNjU1ePnDgQNq2bZtqHySdh4YMGcIXX3zBsWPHkrcXERGRajuXLl1i1KhRhIeHU6lSJUaOHJnq9W3x8fFh3LhxvPvuu9xyyy2EhITQs2fPdMdSXFwc48aNIyIiAn9/fyIiIhg3bhxxcXHJ61hrgrVq1SratWtHaGgoQUFB1K9fP3n/JcnK74JIXqAaEBEb4uPj2bBhA506daJQoUJW1+nYsSNjx45l7dq11K9fn969ezNx4kSioqJS/RDPmzcPPz8/br/9dgC2b99Op06dqFWrFm+//TaFCxdm9uzZ9OjRgxUrVqRq5nLx4kUeeOABhg0bxgsvvEDhwoXTxZlSgQIFcHNzS7Xsvvvu49ixY3z22WcsW7aMAgUKAFClShUmT57M6NGjmTRpEnXr1k1enpH+/ftz7733MmzYMJYuXcozzzxDlSpVaN26NQC7d+/m7rvvpl69esyaNYu4uDhef/11Ll68SFhYWIavncRisaR7bwAeHh7J7+mnn35i+PDh1K1bl7JlyzJp0iR+//13li1bhre3NwDHjx+nbNmyjB8/npIlS3Ls2DGmTZvGXXfdxcqVK1O9dnR0NI888giPPfYYgYGBTJkyhfvuu48HH3yQffv2MXnyZE6dOsWzzz7L6NGjU12MAXz55ZcEBwfz2muvce3aNcaPH0/37t3ZunUrJUuWtPo+jxw5wu23346/vz/jx4/Hz8+P+fPnc9999zF37tzkZCEpEXjqqad45plnbO63n3/+GTDHZlb8+++/dO3alREjRuDp6cm2bdt48sknuXLlCg888AABAQF89tln9O/fn1GjRiW/brly5QCYPXs2o0aNol+/fjz11FNER0czceJEOnfuzIYNG5I/h5EjR7JgwQKefvpp6tSpw88//8zgwYPTxfPKK6/wxhtvMHjwYDp06MDu3bsZP348O3fuZMmSJama+Xz++eeEh4fzyiuvULRoUQIDA9MlKgDPP/8806ZNo3///jzzzDO4u7uzefNmjhw5QsOGDQHTbG3IkCEEBQVx9epVli5dSqdOnVi9ejW33HJLlvZlkgkTJjBlyhQeffRRWrduzbZt27jnnnvSrXf48GFuueUW+vTpQ/HixTlw4ABvvvkmkZGRzJ49GzDNj+6++25uueUWnn76aYDk5kjZObbT6t27NyVKlGDKlCmUKlWK48ePs3LlyuS+HBcvXqR9+/ZcuXKFp59+mrCwMFatWsWoUaO4evUqDz/8sM3zii3NmjVj/PjxybW4f/75JxcuXKBgwYL8+uuvyeePDRs20K9fP6uvMWbMGM6ePcvWrVv54osvgOvNTpM8/fTTdOjQgdmzZ7N3717Gjh2Lr69vlm7efPnll1SuXJnXXnuNuLg4XnjhBR566KHkGxFgkqDvvvuOUaNG0bhxYzZt2sTkyZOJjIzkww8/tPq6kZGR3HPPPXTv3p0xY8bg6enJgQMHUvW9y87vgkhupwRExIZz585x+fJlQkNDba6TVHb06FEA7rrrLl555RW+++675P4LcXFxzJ8/nzvvvDP54vnFF18kODiYhQsXJv94tmnThsaNG/Paa6+lugMdExPDzJkzrdYc/Prrr+n6aLz99tvcd999qZYFBQVRtmxZAOrXr58cB1xPNqpUqZLlJgOPPvpo8gVCy5YtWbduHQsWLEi+gJg8eTLe3t58++23FClSBIDGjRtTq1atLCcgU6ZMYcqUKemWR0ZGJid3U6dOpVmzZjz00EM8/fTTTJkyhRdeeIF69eolr9+0aVOaNm2a/Hd8fDyNGjUiIiKCHTt2pGreER0dzRtvvJG8fkBAAM2aNWP58uX89ttvyRdYf//9N++//z4JCQmpLrouX77M/PnzKVq0KAD16tWjXr16vPvuuzz//PNW3+fEiROxWCwsWbIEX19fwBwLR48eZfz48ckJiJubGwUKFMi0nX3SsRgSEpLheklGjx6d/H+LxULTpk2Ji4tj1qxZPPDAA3h5eVGzZk3A1PqlPEZiYmJ46aWX6Nu3b6pOwfXq1aN+/fp89tlnDB06lL179/L111/z0ksvMWLECABatWpFbGws77//fvLzzp8/z7vvvss999zD66+/nrwv/Pz8ePjhh1m2bFmq2huLxcL8+fNTJeX//PNPqvd34MABpk+fztChQxk/fnzy8vbt26daL2X8iYmJtGrVigMHDvDZZ5+lu0udkaioKGbMmMH999/PuHHjAGjdujUFChTgpZdeSrVuytoAgEaNGlGxYkU6d+7M5MmT8fX1pVatWhQsWJBSpUql+35m59hO6ezZs+zfv5/PP/881f686667kv//3nvvcfjwYTZu3EiFChUA811P6g80aNCgDM8r1jRv3pzLly+zefNmmjVrxrp165JrV9atW0fr1q3ZvXs3p06donnz5lZfo1y5cpQqVYqCBQvaPF81bdo0+fhp3bo1+/btY8GCBVlKQAoWLMi8efPw9PRMXjZgwACOHTtG2bJl2bVrF998802qGwFJn++rr77KyJEjrSasO3bs4Nq1a0yZMoXixYsDcNttt6VaJzu/CyK5nZpgidhgsViy/ZyQkBCaNm3KvHnzkpf9+OOPnD17NvkO6OXLl9mwYQPdu3fH3d2d+Ph44uPjsVgs3Hbbbemq2z08POjQoYPV7d1yyy2sXr061cNeTZwykvbirVq1askXvgCbN2+mbdu2yckHmIv5W2+9Ncvb6NevX7r3tnr16uQ76mCaTHz44Yf88ssv3HnnnTRr1ozhw4enep24uDjeeustGjZsSNmyZfHz80tusrFv375U6xYtWjTVBV3lypUBc+GVMtGoXLky8fHxnDhxItXz27Ztm5x8AISFhdGgQYMMm/KsWrWKtm3bUrx48eRjIT4+njZt2rBz587kO/qhoaGcPXuWp556Kkv7L6v+/fdfHnroIWrUqIGfnx9+fn68+uqr6faNNZs3b+bixYvcfffdqWIPCgqiUqVKycfyli1bsFgs6S640/69efNmrl69Su/evVMtT0reN2zYkGp5mzZt0tUIprVmzRoSExO5//77M1xv7dq1dO/enYoVK+Lr64ufnx9r1qzJ0n5I6a+//uLSpUv06NEj1fKePXumWzc6OpqXX36ZunXrUqZMGfz8/OjUqRMWi4X9+/dnuq3sHNsp+fr6Eh4ezv/+9z8++eQTq9tatWoV9erVIywsLN1xee7cOXbv3p1pfGnVrFkTHx8f1q5dC5hRq1q0aEHz5s1TLfP09EyumboRac9P1atXT3V+ykirVq1SJR9JowomPT/pmE57jN59990A6Y7RJBEREXh6ejJo0CAWLlzI6dOnU5Vn93dBJLdTDYiIDaVKlaJw4cIcOnTI5jpJZUFBQcnLevfuzbBhw4iMjCQ8PJx58+ZRuXJl6tSpA5i7vAkJCbz++uvJd+nSSkxMTL7T7e/vb7NpQ7FixZJf15HSNicqWLAgV65cSf775MmT+Pv7p3te6dKl01202xIQEJCl99agQQMqVarE7t27GTJkSLoagpdffpkZM2bwxBNP0LhxY4oXL05iYiK33357qpgBSpQoke59AenatSddoFy9ejXd+0vL398/w4u106dP8+WXX/Lll19aLT937lzyHdOsSDoWDx8+TMWKFTNcNyYmhu7du1OwYEGeffZZypcvT6FChViyZAmTJ0/OdFtJF1FpE4kkSfvt5MmTAOmOibT76/z58wCUKVMm1XIPDw98fX2Ty5NkZaSrc+fOASTfqbdmx44d9OrVi+bNm/PGG28QGBiIh4cHr776aqr+GFmR9F7Tvjdrx8Zjjz3GqlWreOqpp6hVqxbe3t4cOXKE/v37pzs2rcnOsZ2Sm5sbCxYsYMKECfzvf//j3LlzhIWFMXz4cAYNGgSYz/bAgQM2R8FL2q/Z4e7uTpMmTVi3bh0JCQn88ssv3HfffcnNDy9evMi6deuoX79+qpsX2WXt/JT2u5qd5wLJ+9PWMZr0d9pjNEn58uX59ttveeutt3j44Ye5evUqdevW5X//+x/NmjXL9u+CSG6nBETEBg8PD5o0acLq1au5cuWK1X4gS5cuBczoQkmS2vh+9dVXPPLIIyxbtownn3wyubxEiRK4u7vz4IMPWm0XDqT6kUnbnyM3KFOmTLo7fACnTp2y+7YmTpzI/v37qVGjBk8//TRNmzZNVUvy7bff0qdPn1T9Jv7991+7xwHW39/p06cJDAy0+RxfX18aN27MyJEjrZZn9FxrbrvtNl555RWWLl3KsGHDMlx3y5YtHDp0iB9++IEmTZokL1+yZEmWtpXUZGz69OlUq1YtXXmxYsWA6xdnp0+fTlVDlHZ/JV38nTp1KtXrxcfHc+7cueTtJcnKd6NUqVKA6S9hawjnxYsXU6BAAb744otU/QkuXbqU7Qu+pPea9j2kfa9Xrlzh+++/56mnnkr1OVnrw2LLzRzb4eHhzJw5E4vFwp9//skHH3zAE088QWhoKG3btsXX1xd/f38mTpxo9fmZJbe2NG/enLFjx7Jx40ZiYmJo2rQpxYoVo3DhwmzYsIENGzbwwAMP3NBrO0LKYzSpHxRcTzzTHqMptWjRghYtWnD16lV+/fVXJkyYQO/evfnjjz+y/bsgktvpaBbJwPDhwzl37hwvv/xyurLIyEimTp1KkyZNqF+/fvLy4sWL07FjR+bNm8eCBQu4evVqcvU8mGY+jRs3ZufOndSqVYs6deqke+SEpKF7097RtbX8ZjRo0ICVK1emGnnmxIkTqUYdsoeNGzfyxhtv8MILL/Dll19y/vx5nnjiiVTrxMbGpmpSAaZjb05YuXIlly5dSv774MGDbN68OcO+NW3atOGvv/6iatWqVo+FlHO5ZEX9+vWT7+QfOHAgXbnFYklOnJM+n5T759q1a3z99depnmPrGLn11lvx9vbmwIEDVmNPuuCvX78+bm5uLFy4MNXzFyxYkOrvBg0a4OXlxbfffptq+fz584mPj0/VPC6rWrZsibu7e7oBA1KKjY1N179mz549bNq0Kdvbq1GjBkWLFk333ubPn5/q72vXrpGQkJClY9PLy8vq99Mex7abmxs1a9bk1VdfBUgeratNmzbs2bOH4OBgq59tUpKf3fNH8+bNuXbtGpMnT6ZWrVr4+Pjg4eFBo0aNeO+99zh79qzN/h9JbO0PR0g6BtMeo0nfmZSJvC1eXl7cdtttDB8+nEuXLnHw4EGn/S6IOItqQEQycNttt/Hss88yfvx4Dh06RJ8+ffDx8WHHjh1MnTqV4sWLM3PmzHTP6927N/Pnz2fixIk0bdo0XYfgV199lc6dO9OzZ0/69+9PmTJlOHv2LDt27MDNzY0XXnjB7u8lqbP5tGnTaNu2LQUKFKBOnTpUrFgRDw8P5syZQ8mSJfHy8qJixYqpahGya/To0SxcuJA777yTxx57jGvXrvH6669TunTpLN/FO3bsmNW+EyEhIQQEBBAVFcVDDz1E8+bNGTZsGG5ubrz11lvcf//93H777clJX9u2bfniiy+oWrUqVapU4aeffsryHf7sKly4MD179mTYsGFcu3aNCRMm4O3tzaOPPmrzOc8++yxt2rShU6dODB48mNDQUKKiovj77785fPgwb7/9NmCa+9WpU4cxY8Zk2g9k5syZdO/endatW/Pwww8nT0S4e/duPvnkE3x8fOjYsSO33norxYsX54knnuCZZ54hLi6O6dOnp/uMSpcuja+vL/Pnz0++wA4LC8PX15eXX36Z0aNHc/bsWW6//XaKFy/O8ePHWb9+PS1btqRnz55UqlSJu+66i1dffZXExERq167N2rVrk4elTtpeyZIlefTRR3njjTcoUqQI7dq1459//uHVV1+lcePG6dr2Z0W5cuUYOnQo7777LjExMXTs2JECBQrw+++/U7lyZXr27Enbtm2ZPn06Dz/8MP379+fw4cNMmjQpyx35U0oaJnfKlCkUK1aM1q1bs3XrVj777LNU6xUvXpyGDRvyzjvv4O/vT2BgIN99913ykN4pValShV9++YVly5ZRpkwZfH19CQsLu+Fje+fOnTz99NP07NmT8uXLk5CQwOeff46Hh0dybe7QoUP57rvv6NixI0OHDqVixYrExsayd+9efv311+RhmG2dV2ypUaMGpUqV4ueff07VXyupZsTLyyvTvmJVqlTh/PnzzJo1KzlJr1GjRqbv2x6qVatGr169mDhxIvHx8TRs2JBNmzbx+uuv06tXL5sjps2ePZuNGzfStm1bgoKCOHv2LG+++SaBgYHJNWXO+F0QcRYlICKZGDNmDHXr1mX69Ok8+uijXL58meDgYPr06cOoUaOsDq+aNHLPsWPHrA6ZWrt2bX766ScmTZrEU089xcWLF/Hz86NmzZo8+OCDOfI+OnTowIMPPsisWbN47bXXsFgsREVF4evry+uvv87UqVPp3LkzCQkJLF68ONO7kBmpWrUqX331FS+88AIDBw4kMDCQkSNH8uOPP2b5NT7//HOro7688sorDBs2jBEjRnD58mXee++95KY4PXr0oH///owePZpbb72V8PBwXnvtNdzc3JIvGJo2bcr8+fNzZHKzPn36UKRIkeShQuvWrcusWbNsDsELJqFavXo1EydO5JVXXuHMmTP4+vpSrVo1+vbtm7yexWIhISEheZjUjJQtW5affvqJGTNmsHDhQt5++20SExMJCwujXbt2yfNI+Pn58fnnn/P8888zcOBA/Pz86Nu3L0FBQakuDt3d3Xn77bd55ZVX6NGjB/Hx8bz77rv07duXgQMHEhQUxNtvv80333xDXFwcgYGBNGnSJNXF2NSpUylWrBhvvfUWcXFxNG/enMmTJ9O7d+9UfVxeeOEFSpUqxUcffcSsWbPw9fWlT58+vPjiizfcBGXcuHGUL1+eDz/8kC+++IIiRYpQo0aN5FHbWrVqxZQpU3jnnXf4/vvvqVChAq+88gorVqzIsA+YLUnf+U8//ZQPPviAevXq8eWXX9KoUaNU633wwQc8+eSTPPvss7i7u9O+fXs++ugjWrZsmWq9sWPHMmLECAYOHMjly5e55557mDFjxg0f22XKlCE4OJh3332XY8eO4eXlRfXq1Zk3b17yUK8lSpRgxYoVTJo0ialTp3L8+HFKlChBpUqVUnWwt3VescXNzY2mTZuyaNGiVE1Xk8439evXtznseZL77ruPLVu28PLLL3PhwgVCQkL4888/M3yOPc2YMYPw8HDmzJnD5MmTCQgIYOTIkRneGLjllltYuXIlL7/8MqdPn6ZkyZI0atSIDz74IHkgBWf8Log4i1tUVFT2h/oREcmmmJgY6taty4MPPsiYMWOcHY7d+fj4MHr0aJvD7Up6b7/9NhMnTuTAgQOZXnSKiEjeoRoQEckRTz75JA0bNiQgIIATJ07w3nvvceXKFaszxUvet2zZMv7++29q1qyJm5sbGzduZNq0aQwePFjJh4hIPqMERERyxJUrVxg7diynT5+mUKFCNGzYkCVLlmR7VCfJG4oVK8bixYt58803k5sxPvHEE4waNcrZoYmIiIOpCZaIiIiIiDiMhuEVERERERGHUQIiIiIiIiIOowREREREREQcRgmIiIiIiIg4jBIQERERERFxGCUgIiIiIiLiMJoHxEUVLVoUd3flhyIiIiKuIDExkUuXLjk7jDxBCYiLcnd3VwIiIiIiInmOrnBFRERERMRhlIBkYMyYMURERODj48OuXbtsrvfpp59St25dateuzYgRI4iPj08uW7ZsGQ0aNKBOnTr079+fmJgYR4QuIiIiIuKSlIBkoHv37ixbtoyQkBCb60RGRjJ+/HiWLVvGtm3bOHnyJJ999hkAMTExDBs2jLlz57Jt2zYCAgKYMmWKo8IXEREREXE5SkAy0LRpU4KCgjJcZ9GiRXTp0oXSpUvj5ubGAw88wDfffAPAjz/+SJ06dahcuTIAgwYNSi4T+/jjLAz8CcrNgULvQ7EPoO7X8No2OHfl+notF5pHSm4z4KXN2dvemqPmed/sv/nYXcH9P0H4nBt77vSd8PHurK8//ndY8O+NbSsnWTs2rEn67NccvbHtvLTZPP9GfL4Hpu64sedmV2bH+GPrbvx9ZOd4y8r38+PdZr2kx5nL18ssFvhyLzT/Dkp/ZM4PwZ9C++/hwxQV2pEXzXMnb888Jmuf4bUEeORnCPwECrwHtb+CY5fMutvPpH+NqTtsx2zL879B6Gfg8R74zMp8/Ztx/0/mPJoV4XPM+o7w8hao/iUkWhyzPYC4BPjfZvM+vWZC1S/gnT/Tr9d/FfRYevPbu5HfJMj4eBNxVeqEfpMOHz6cqoYkNDSUI0eO2Cw7fvw4iYmJ6mBuBx/sgqHroEoJeLI2VC8JcYmw5TS89xf8chK+62D7+b/cAcHFHBZunjN9J/gVhvurZm398VuhVwXoUS5n48qLPt8HO8/ByFrOjsQ1zW8PgUXAx+v6smd+g0nbYHA1c37wLggHo+Gno7AwEh6snv3tPFgNOqSpEJ/xF8zcBe80g3r+UMzTXBD+bwuEe0Ntv9Tr96kEjcrAh3/DrCwk8Av/hVe3wnN1oWMoeBXIfty53bFL5qbSx63B3c1x2x26Dj7bA680gAalYflhGLEeoq/Bs/Wur/dSfaj6Jfx0BFoHOy6+JBkdbyKuSgmIHbi5XT8jWiwWm2ViP7+cgCFroW0wLOiY+ke5bQg8UQuWHcr4NRoF5GyMIuIYdfwgvPj1vy/Hw9Q/4L7K8H7L1OveX/XG76IHF0t/02LnOSjsAY9FXF+25ZTt1wgoYh7LDmdtmzvPmX+HR0DpItmL15bYOCjiaZ/XcoS3/jDJZc/yjtvmX+dg1t/wakN4so5Z1jIIzl6BcVvhkRrgW8gsr1DCJKYTtzknARHJjXQb/iaFhIRw6ND1K93Dhw8THBxstezQoUMEBgaq9sMOxm8FNzdzcWHtjmDBAtAtkzvt1qq7j8bAQ2sg5FMoOBPKfgK9lsPJWNuvc/GaadZR5mPYdDLjbR6Khn4/miYhXjOh2hcwZXvqC6KUTULe2GGalxX7ABrPh19PZPz6cL1ZysrDpnma72wo+gF0/QEOXMz8+Vfi4ZlfzXYLzoSgT+HRtRB19fo64XPgr/Pw87HrTUkyalrjNgMuxcMn/1xfP2Wzp51noftSKDnLNJWp/RV8ksXmXe/uhBYLzD4t+gFEzDN3S+MSUq9nsZjlYZ+ZbdT9GpYetP6au89Dh++hyAfg95FpYhMdl7V4AJYcNO/Ba6bZj7aa92Ql9pYLzesdjE7ddCfJ/zZDw2/N51z8Q/O+Zv1t3q+jJP63b6t+Yd5z6Y/gvlVwJAtjbly8BoPXQKnZ5jjv8D3sibq5eC7FwdUECCxqvdzWXfTMvm9pm2C5zTA1GZfjr38uH++GBt+a8oGrry+/kaY14XPg+U3m/2U+Sf06Wd3nLRfCLV/C2mPQZL45ph9Yk/m2/zoHbRaZ49L/I9P0LjaT70DSuScyzXnGVvPFH4+YbRT/0MTV9DtYdST1OtcSTE3RvZWuf24WC1T63Jx304qJgxKzzDnrZiz4FyzAwDQ1vAOrms877Q2u/pXN+9l/IfPXzuoxv++COYdX+tzsn6BPzXn8z7PX11lzNOPjbcsp6LPSHEuF3zf/3rPSnE9EnEk1IDepW7dudOjQgTFjxuDv78/s2bO58847AWjTpg2jR49mz549VK5cmVmzZiWXyY1LSDTNKOr5QYgdm1AdjTEn8rhEeLYu1Cxl7nYtPwznr0IZK3cfj8RApyVwLRF+6Qnli6dfJ8npy9DkO/OD+sqtprr8+4Mw+hfYfxGmt0i9/rs7oaoPTG1q/n5hE3T6Af7tCyW80r18OoPWmBqiz2+HwzHmQqblQvjj7tRNVVKyWKDHMlh1FJ6pA80DTT+bsZtNk7ZfepqE77v20GsFlCgI05ub52bUNOSXO6D1YmhVFl74r+lC8YLm33/Om/1SujC83QxKFYI5e+D+1XDyMoypk/H73H/BXJyU8zaJ544zpsnK7iiY3er6ev/bYh6DqpqmYIdjYPDPkGCBKj7X1zsZC7ctBE93897KFIa5e80FWFasOmKSqcZl4Mu25vVf2249ic1K7NObw0M/m2Pku/bpXyMyGh6uDqHe5u9fT8Kw9XD0ErxYP2sxW5NogfjE9MutJTZD1sL7u+CxW6BLmInphU2w5hhs7WWa6lljsZi28xtPwov1TDOXDSeg45IbjxvM9iqWMM0ESxeGTqHmM86oQvpGvm+/3AGv/A6rj8FPXc2ywKLwUStzMfh8PegcapbfSHPP79qbuGbthmWdzfct6XWys8+Px0K/VTCmNoxvmHkzprhEc157uDo8XQc2njB3/Q9Gw+JO2X8f1szZYxKm7uXgk9bm+zZzl0kqlneBNv/VJPx20pyHW5W9/lw3Nxh2C4zcAHujoJLP9bJP/zEX+I/ecn2ZtePYmgJu14+RnefAv5CprUqpZqnr5Sm1LGsSlh8OwbAIbMrOMX/skjkfTmwI/oXh3FVzE6fhfNjWC6qUhLr+GR9vkdGmmXKfiuDrZY6FGX9Bg29gVx/b381cJyYKivk4OwrJBiUgGRg9ejQ//PADJ0+epEePHhQtWpRt27YxbNgwOnbsSKdOnQgPD+eZZ56hffv2JCYm0qJFC/r37w+At7c3b7/9Nn379iU+Pp7q1aszY8YN9t6UZGeuQGw8lMvgYv9GvLjZvPaOu6FayevL765off3tZ6DzD1ChOCzocL063pY3dpiLwt96wq1lzLL2oeYC9b2/YGRNqOxzfX1vT/i+ExT4r8KsbFG49VtYesi0I89MfX+YleICvIavucP47k54rp7156w4bBKu1xpdb3bQNsQker1Xmh/3wdWhjj8ULmCSiKw0ZWsUYKpb/QunX/+lLSaBW939ekLZKQyirpmE4eHqGSdcbzS9/v9Ei0maShUyP8hTmkBJL1N7M2kb3FEOPrSyT1ImIG/+YZLFbXdBrf/aU3cMg3aL4VAW7ug/95tJVld2hUL/nWHbh1ivIcpK7NV9TcLoVcD6vv6oderXaFnWXOS89adJ9m60FWjvlVlbb/d5cyE8tAa80/z68jp+5kLpzT9MMxZrlh82F+9vNYXhNc2ytiFQ0B2e23RjcSf5/HZTe/nERvPw9oTbysLdFaBf5fT75Ua+b40CzDHtTurP5hZf82+F4jfX1LOO//ULyXr+1y8Ws7vPz12Fr9tlvXnQtUTTjDXlZ+L532ey4Tg0Dbzx9wSmJmXEepM4peyn1ynM1OA9+xv89l+sv/xXq1zXP/VrDKxqbqq8uxOmNru+/N2dJlmp7nt9mefMrMX1UavrfdrOXrF+Ti/qaY7Ps1dTLy9dBIKKmmQiowQkO8d8i7LmkSQh0SQYNeaZZO2NpuYcnNHx1quCeaR8jS5hpsb+873XY8i14q7Bh2Ng3bfwwU4oWsLZEUkWKQHJwOTJk5k8eXK65e+8806qvwcMGMCAAQOsvkanTp3o1MlOt4wkRy09BK2CUicftiw/bC4Su4SZjpFZ6Rj601HTUT4p+UhyfxVzR+qno6kTkM5h1y+G4Pqdt4NZnEqmb5qLpiYBEOYNq4/aTkB++q+JRNqO5XdVgAdWm5qRwTfQeTcjPx2FNkHpa7Pur2I+k19OQodQ28/fdtrU0Gw4YS60UtoTBQ3LmD5DVxJs75OUVh81iUlS8pHk3kqwMk3zkLQuxcHm0+bCsFCKs6t3Qegabu5eZjf2zPx0xDRJ3Hza3PlN6dRl6zV3WTGpEbS2Mgjg69vhqxQjZK22cczcWsZ8l1YdsZ2AJD23b+XUy++tdPMJSIPSsO9ec3ytPW4Gp1h11NQ6frUfFnVMnYTc7PfNkbK7z0t6Zb9vgq3PZPWxm09ANp40x/uAKulrJzqEmqZll+LMxf6xS+AG+KVJBrwLmiTk43/Mey3qab4Lu86bGuaUNmex4UHam1oZ5e7WykoXNjXpGcnOMR+faPbFnL2mOVZcin319/mMt5MkJg5e2QLfHjC1IQkpajCz+hou7doV2LYKOj8Kd9wFn38OfuqJnxsoAZFcx68QFPGAf7PQnyE7Tl+BYBttxtNa8K9pBzykRtZHpTl7xTS7Sqts0evlKZVK84ObtJ3L8WRJ2qYDAAGF09+5Sxujh7u5q5uSm5t5vbQx2sPZK2YEo7Rs7ZeUDkVD8wWmBuOtZmb/FioAm07Bo+uu76uk92xrn6SNx1rtmrXnpnX+qqmFsLqdNMuyGntGNp2Edt+bWo8PbjN3ywu6m+Pz1a1ZP1asKV8c6pdOvzztsZG0b61+hkUyvoBPOt7SHutZ2ddZ4VnA1DK2D72+vV7LTRKy9JC5457kZr9vjpTdfW5tvYxk9JnY4xyQ1Byx1wrb65y7apKKywmm9iVlcphkWARM22maSD5U3fw/uCh0D0+9XlZHhiqQIqsoVcj6sLaX4kwNka+VWtlCBUy8GcnOMT9qo6nReaq2qb0r6WWazz24JvPtJLl3pUm8X/ivuVfxgiZ56rQk66/hkjb9ACFVIbA8vLwCAv+rKvL3h9OnlYTkAkpAJNcp4G7umC89bPpg2GsoXf9CcORS1tZ9synM22fa7X7XAdrZnqsyWalCpv1tWsf+22baO3w364SVbZ24DBUzaLpWqpC563b6cuoLTYvFvF4DKxekN+tm9suCf03n9vkdUtdkpL1wKPXfxYKtfZIyMSxVyMZ6GQxEkKSkl/lxz8rzsxp7Rr7cZy7Ovu+UusbFkfOtJO3b47Hpv4vHYjP+/JKOt7NXUl+QZWVf34hShUxTxzXHTBv+lAlIbpLdfZ7dVngZfSZpL5xTKvRf0nY1zYXtmTRJS1J87zQzQxJbU6bw9XWvJV6vEUmpYgkzNPG7O82/iyLhfw3SJys30gQrwtd8v07Epk4OkjqA3+Kb/vnnrlq/yZRSdo75OXvMSG7jG6VefuaK7X58KV24apLtsfXh6brXl19NSF/jmmskxMPHL8C8idD7Keg++nrykURJSK6g4ZgkV3qmrrkoHrzGdOpOKy4BFkdm7zU7hprq8X+yUC1dqICZe6BLGHRbasbqz0ybINM8YOvp1Ms/3WMuEFplPOdlts3dm/rvjSdMJ9KWGWwnqePnnD2pl397wFwst0nxXK8C2bs7bGv9NkGmmcyxNMnfp3tMTZetCxS43oQmZS2UxQIf/J16vUYB5jOztU9SahVkRgDakSYR+DzNc60p6gm3lob5/5rRxJJEX0t/PGY1dgAvd+v7zs3N3E1Neef2cryZu8BRkppppT1mNp8yTTzaZHC8JR3zc9M8Nyv7OiNxCbbv1Cc1OymbxdrOG5HTtSc3s8+zytZn0rJs+nWTJF18/5Gmg/aiyNR/Nw0En4LmfFi/tPVHwf/2YVUf8+9+GzXeIyLMQBkDVpnvweBq6dfZfGfWHl3Drz+nezlzXk7bbPLjf8ywy2mbhcYnmoEtqmfShDc7x7wb6WvYlxw0fQlTsnW8ubmZjvFpX+PDv1M3xco1zh6DMW3g69dh8OvQ7QmTbFjj7w9nNDOjK1MNiORKjQNgRgszUVS9b0xTqBq+po3sttPw/t/mDlXKH5TMvNzANMtosdCMghXhazpCLzsEo2pB1TQ/LJ4F4Iu2pjq81wr4tDXck0Hn8MdrmYvqzj+YbYV5mx+T6TtN/Cn7f9jDltPw4GrTf+NwjGlfHFTU9E+wpW2w6TD91K+mP0HTFKNg1fGD/lWurxtRytwhnLfPNNcpVMAssyWilLnzvDjSNAnx9jSjuIytb+7StVpoRm3y9TKJwpKDpjN8Rh3Q2wabJkf3rDQj/FxJMP1pzqe5u1fSC0bXhnG/p94nL21J3/RhZE2Y/bf5nMbden0UrN1RtuNI6ZVbocMSaLsYnqhtOn1O2m6Sk5R3HbMae9K+m/8vzNhpOiO7u5mLtM6hZnCDe380TVDOXoHJO6w3Cxy02lxM7e+bvt9Lhs6cgb59Ye5cq3cUq5Q0237nTxNXx9DrIzKFFDPHvS3tQqBFIIz51SS49f1Nf5ibTaAuXDOd/u+qALcHmzhi4szx99Yfpp9EzxycELNCcXOROnev2VYxT5Pw2CvpuZl9nhUF3WHKDrPPGpS+PgpWx1BolkH/jwalTZPC0RvNBXlJL/juX1h/PPV6xTxN5/kBP5nvRK/ypv/E6cuw46z5d8ZtZt2kGya/nrzeLyeltiHmon/1MTO4gLW5Uqw1JcxMDV8YVM2c+wq4mfe24rDp/D/u1vQd1P84awZHyexGUnaO+S5hJuGp6mPe+++nTR+stE2FMzreWgSa5/gVMnPl/HzMDNPtUzD1axyMhgpzTb+clIOXuIz4OBjVHOKuwpSfoUZTaG9lWMCU+vaF5csdE59kmxIQybUGVzd3m9/8w4xwdCLWNEep7AP3Vkw9MVhWBBWDTXeaH5yJ28zFnH9haBZgvb0vmB//WS3NxXS/VaaZgK0Zlv0Lw8Y7zAzNz/xmLvDLF4fXGpsEx95mtTQ/an1WwtVEMzLMW80yHq3Lzc2M6PXSFvjoH9OPwK+QGeN+fMPUF7b/a2CagAxeY+bICPOGyH62X/utpqZvQ5+V5of6trKwpru5mNp4hxn5JqnvQ7WSqZtD2FK1JHzb3oyG03O5adJwbyWzP9MOa/lyAyjqAdP/Mvulakl4r4W5YE8poAj83MOM0jNkramFuaMcTGsG3ZdlHA+YC6IFHUxMvVeY1xt6i3lf/9tyY7GPqGlqZZ79zVxcWwDLENOxeHYrc/x3XWoSzMHVzMXcoDWpXyPBYh7Zmh/kzJnrdxiTmjWQPgmZ0cJcBM362zSHKVHQ3CGe0DDjJjvubqYz+KiNprPttURoGgA/dDIzS9+o4gXN8bnqqNlnJy+bu8nlipsE86k6OTsRXxFPmN3SfN7tvjc3RsbWh5ca2G8bN7rPsyKpWd/w9SbxKFzAHFevN874eQXcYXFHeGw9PLLW1Nz1qQTTmpuEPqV+lSG0mBmi+uH/5tkpXdj017g/xY2OkGJmdLiF/5qky5q7K5hz1mO3WC+/UdObm+/UO3+a35dwb3MOtTbK1YJ/zbkys+a42Tnm32pmPosJ20wyWNfPNNlMmhsmSUbH2+e3w4gNJuGJ/29bK7um/zwsluvnCJcSH2eaXXkVhhEzoXwt8PnvnDR3ru0akKRycVluUVFRrna4CWYIX01YKDfi491mhK7Nd97YnT8RIHXykZILta1OOtb33WsSYI9ccMpMutB7eYuZQ+T0/XloLoYc8u1+Myz0wX7mRlFa9b8xCebmXg4PDTC1nBU/NzcQbI34Jjfg1GEY3wfCqsPjH1hfx8HnqcTERKKjNYujPeSC07WIiDiUrR91cMm21RU/Nx2Nz1x2diSZe+sPE+srvzs7ktyjZ3nTBGrCtuvLLl4zTcOe/dU0TbI1tLgjzNljaiierO28GPKcXxbDkNpw+jC0G2h7PT+//2pmU3ChmyRim5pgiYhIan37Zl7uAm2ru4annuMhKyMDOdu9lVL3o8gNMTubm5sZZnpRpBnm2t3NDObRapFpbja2PvTIwT49mUkE5t6uz9IuLBaY+QTMfxMad4MnPoLiVoYcSykpCcmgr5q4HjXBclFqgiUiTpNRDQjoDqOI5JwPn4JSZaHH8NSzhboANcGyHyUgLkoJiIg4VS7oAyIiecSquXA1FjoNdnYkGVICYj+6whURkfTUtlpEclpsNLw2ACb1g382OzsacSAlICIiYl1SEtKunZIPEbGvPb/D0LqwYT6M+RQef9/ZEYkDqQmWi1ITLBEREcmznu0IF07Ds19CUEVnR5MlaoJlP0pAXJQSEBEREclT9m41fT1uaQZRp6FoCfAsmPnzXIQSEPvRFa6IiIiI5JxrV+HjF2DYrfD162aZj3+uSj7EvjQPiIiIiIjkjD1bYPJAOLwb+r0IvZ92dkTiApSAiIiIiIj9JSTA+HugiDdM2wIVajk7InERSkBERERExH52b4IS/hBYDsYvg9Kh4OHp7KjEhagPiIiIiIjcvGtXzEzmIxvD/DfMsrIVlHxIOqoBEREREZGb8/evpq/HiQMwYBzc/aSzIxIXpgRERERERG5cTBQ83RZCq8G7WyG8hrMjEhenJljiOGfOQPv25l8RERHJ3f7+DS5fgmI+8PpqmLpRyYdkiRIQcYwzZ8DfH1asMP8qCREREcmdrsTCzCdMX48l75llletDATWskaxRApKJ/fv3065dO+rVq0fr1q3ZvXt3unW++OILmjVrlvwoX748/fr1A+DgwYOUKlUqVfm///7r6LfhXEnJR0pKQkRERHKfnethSG1Y9C48+BrcMdLZEUku5BYVFWVxdhCurGvXrvTp04e+ffuycOFCpk2bxsqVKzN8TpMmTXjqqafo3r07Bw8epFWrVhw4cCBb2/X29sbdPQ/kh9aSj5ROnwY/P8fFIyIiIjfm8D/wYDWo1ghGzYbQqs6OyKESExOJjo52dhh5Qh64ws05p0+fZseOHfTu3RuAbt26cfDgQQ4ePGjzOb///junTp2iU6dOjgrTtfXte3PlIiIi4lz7t4PFAiFVYNwPMGVdvks+xL6UgGTg6NGjBAYG4uFh2jS6ubkRHBzMkSNHbD7ns88+o3fv3nh6Xh/zOjo6mlatWtGiRQsmTZpEQkJCjsfuMubOvblyERERcY7Ll+Dd4TCkDmxcaJY16AAFCjg3Lsn1lIBkws3NLdXfFovtFmuxsbHMnz+f/v37Jy8LCAhg165drF69moULF/LLL78wbdq0HIvX5fj5mWZW1qj5lYiIiGvasQYeqQnLPoQhU6FRV2dHJHmIEpAMBAUFcezYMeLj4wGTfBw9epTg4GCr6y9cuJAqVapQter1akkvLy/8/+sDUbJkSfr168fGjRtzPnhXYi0JUfIhIiLimrb+CE+2Ar9geO8PuGOEaj3ErpSAZMDf35+IiAjmzZsHwKJFiwgNDSUsLMzq+nPmzElV+wGmH0lcXBwAV69eZfHixdSsWTNnA3dFSUlIu3ZKPvIrzQMjIuLajv83SmetVvD0XDO3R1BF58YkeZJGwcrE3r17GTp0KOfOncPb25sZM2ZQrVo1hg0bRseOHZM7m//77780b96cv//+G29v7+TnL1q0iAkTJuDu7k5CQgLNmzdn3LhxeHl5ZbjdPDMKlgikHw1NSaiIiOuIjYYPn4If3ofpW6F8PrxRmgUaBct+lIC4KCUgkmfYGopZSYiIiPNt/RHefBAunDHzenR5BHT9YZUSEPvRESYiOSejeWA0GaWIiHOt+BiebguBFWDmn9BtqJIPcQgdZSKSczQPjIiI6zl/yvzbqBs8/iFM+hECyzk3JslX1ATLRakJluQJGdWAgJphiYg40qULMPMJ2PAdzNoNPhmcnyUdNcGyHw9nByAieVjS6GfqAyIi4lyblsLUhyD2Ajw0BUro/CvOo1vsIpKzNA+MiIhzfTEenu8EYdXh/Z3QaTCkmWhZxJFUAyIiOS8pCenbF+bOVfIhIuIIl2OgcDFo2AV8ykCHB5R4iEtQHxAXpT4gIiIickOiz8OMkbBvK7z7O3gWdHZEeYL6gNiPakBERERE8opfFsFbj8DVWBgyFTw8nR2RSDpKQERERETygukjYMHb0LAzjJgJfkHOjkjEKiUgIiIiIrlZ3DXTzKpWK6hUD27vr74e4tLUB8RFqQ+IiIiIZOjCGZg+HBLi4fmvnB1Nnqc+IPajK1wRERGR3Gb9fBhcA7Ysg8bdwaL7yZJ7qAmWiIiISG5hscDEfrD6c2jSA4bPAN8AZ0clki2qARHXcuYMtG9v/hURERHj8iW4dsX07ahQG575AsbOV/IhuZL6gLiofNkH5MwZ8Pe//rdmyxYRkfzOYoGfPodZT0HXoXDPs86OKN9SHxD7yWdXuOKy0iYfYP5WTYiIiORX/2yGx5vCpH5QtRG0vMfZEYnYhRIQcT5ryUcSJSHibGoWKCLOsG8bDLsVrlyC136CF7+BwHLOjkrELpSAiPP17Xtz5SI5JSk5XrFCybCI5LxrV2D1l+b/FWrDSwvg3d+hditnRiVid+oD4qIc1gfk1GHwD3buhEUZ1YCA+oKIc9g6LnU8ioi9WSywYQF8MBpOHYKZf0JoVWdHJWmoD4j9qAYkP7t6GYY3hCdbmapeZ/HzMxd11uhiT5xBzQJFxFH+/ROeuh1e7glBlZV8SL6gBCQ/8yoMoz+CqFPwaD14czCcP+mcWKwlIUo+xFnULFBEHGXDd3DmCIxbAuOXKvmQfEFNsFyUQ4fhjY+DJTPh07EQUhWmbnDMdq05c8Zc3M2dq+RDnEfNAkUkp8THwffvwdVY6P3Uf3N7uINnQWdHJplQEyz7UQ1IJvbv30+7du2oV68erVu3Zvfu3enWWbduHYGBgTRr1iz5cfny5eTyZcuW0aBBA+rUqUP//v2JiYlx5FvInIcndH8MPtoLI983y/bvgBWfQEKCY2Px84Ply3VxJ86lZoEikhN+XwlDasOMEabWA6BgISUfku8oAcnEyJEjGTBgAL///jsjRoxg2LBhVterUqUK69evT34ULlwYgJiYGIYNG8bcuXPZtm0bAQEBTJkyxZFvIeuK+0J4DfP/TT/A5PthaB3YvMx0kBPJT9QsUETs5dpVGNsdnmkHxf3MyFaPvuPsqEScRglIBk6fPs2OHTvo3bs3AN26dePgwYMcPHgwy6/x448/UqdOHSpXrgzAoEGD+Oabb3IkXru65xl46xco6gPPdYSn28Kx/c6OSsSxkpKQdu2UfIhI9l2OMTfwCnpB6VB4/iuYvAYq1nF2ZCJOpQQkA0ePHiUwMBAPDw8A3NzcCA4O5siRI+nW3bdvHy1atKBVq1Z8+OGHycsPHz5MSEhI8t+hoaEcP36cxMTEnH8DN6taI5jyM/xvoTmJFvY2y69dcW5cIo6kZoEikl0JCbB0FgyoAOu+NcsefQda3OXcYe9FXISHswNwdW5pThQWK02RatWqxV9//UWJEiU4evQod911F6VKleKOO+6w+hq5ipsbNO5mHgAxUTC4BrTsA/c8Z5ptiYiI5FXZGRzFYoHflsCsp+HgX9DqXqja0DFxiuQiqgHJQFBQEMeOHSM+Ph4wycfRo0cJDg5OtV7x4sUpUaJE8nN69erFxo0bAQgJCeHQoUPJ6x46dIjAwEDHjXBlbx4FofMj8MP7cH8F+Op11YiIiEjelDQi3ooVWZsDaM08eLEr+JSGdzbBM3OhdEjGzxHJh3LpVbBj+Pv7ExERwbx58wBYtGgRoaGhhIWFpVrvxIkTyU2qoqOjWb58OTVr1gSgTZs2bN26lT179gAwa9Ys7rzzTge+CzsrVAT6vQAf7zN3dmY/AxM1J4KIiOQx1objtpaEHNoNy2ab/ze9A8Yvh9dWQZUGjolTJBfSPCCZ2Lt3L0OHDuXcuXN4e3szY8YMqlWrxrBhw+jYsSOdOnXi/fffZ/bs2RQoUICEhAS6d+/O008/ndz06ocffmDs2LHEx8dTvXp1ZsyYQfHixTPcrkPnAbkZR/aYGpDyNeGfzRB9Huq3c3ZUIiIiNy4rcwElxsJn/4OVH0NAOXj/L9PZXPIszQNiP0pAXFSuSUBSeudRWDwd6raFwa9BhdrOjkhERCT72rc3za5saRsGnsehSAm49znTNFnJR56nBMR+lIC4qFyZgFgssHEhzHoKju41TbQengIlyzg7MhERkayzVgNSAEiam3fSQChbDnqOhCLeDg5OnEUJiP1oFCyxHzc3aNoDGnaGZbNMbUjBQqbsSqzpPyIiIuLqkuYA8vc3vWVDgHBgD/CH5gQSuVm57Ba75AoentDlEXhvBxQtAedPQb9QmD4Czp90dnQiIiKZK14M5kyApkBF4BSw5g8lHyJ2oAREck7S/CeFisAdI2HlJ3BfeXj/SZOUiIiIuKq1X8Nnz8Htd4F7M1h7GipHODsqkTxBfUBcVK7sA5KStYmbLp6Db9+AhW9DrVZmhnURERFXcDkGFs+As0dhyFSIj4OTByGoorMjExehPiD2owTEReXqBCRt573TadrLXjwHly5AYDnYtBT+WAO9RoNPBkMeioiI5IRLF2HRNHODLPYidBgEw6Zfr8UX+Y8SEPtRJ3SxL1sTN6VMQor7mgfAmSOms/rCadD9MSUiIiLiOFdiYWAluBQFHR6E3k9B6VBnRyWS56kGxEXlyhqQrEzcZK3z3sWz5s7TgrchMRFeXQo1W+RcnCIikn9dPAtLZsKdT5i5O1Z/CRHNwS/I2ZGJi1MNiP3ksitccWl9+95YefFSMPBV+CzS3H2qXN8s//kr0/5WRETkZkWdhllPQ/9w+PxV2LfVLG/VR8mHiIOpBsRF5asaEGvirsGACnD+hJnQsPdTEFbdPnGKiEj+8t3bMPsZcHeHbo/BnaPU3FeyTTUg9qMExEXlygQEbCch2Uk+kly+BMs+hK9fhzNHoVlPeG4eFFDXJRERycSZo+ZmVmA52LgQ9v4Od4wwte4iN0AJiP0oAXFRuTYBgcxHwcquuGuwag5E7oRH3oCEBNi1EW5pplFKREQktVOHYN4kcwOr+V3w9BxnRyR5hBIQ+1EC4qJydQIC1ucBsZfflsALXaBqQ+jzDDTqaqrVRUQk/zp7DD4daya9LVLcdDLv9igULe7syCSPUAJiP0pAXFSuT0ByksUCW5bDF+Nh5zoIqwEPTIDGXZ0dmYiIONq5E+AbAGePw8jG0HWoeRQu5uzIJI9RAmI/usKV3MfNDRp0gDfWwhvroEyYufMFcP6kmVRKRCQnnTkD7dubf8XxEuJh/XwY1QIerglXL0OpQPjkANw9RsmHiItTAiK52y3NYNwS6PyQ+fvjF6BvCLw/Gk4ddm5sIpI3JfVzW7HC/JufkhBnJ14JCfD1ZLi/Irx8J1gSYcRM8ChoytVyQCRXUBMsF6UmWDfozFEzq/qS9+ByDLS4G+5/BQLLOzsyEckL7DnSX25j7wFGsuP4AQgoZ/4/qrk5p/cYAZXrOWb7IqgJlj0pAXFRSkBu0uUYWDYbFrwF45dDUEU4th/KhEOBAs6OTkRyI3vOdZTbOCPxSkyE31fAd2/BlmXw5gao0cTUgug8Lk6gBMR+lIC4KCUgdpKYaKrkExJMlb2bG3QfBu0HQjEfZ0cnIrlJ+/am2ZUt7drB8uWOi8dRnJF4/fiZma38yD9Qsa6Zv+O23lDQy77bEckGJSD2oytcyduSkrgCBeD5r6B6E5j1FNwbDO88ampKRESyYu7cmyvPrfr2vbnyrDoRCRfPmv8f2w/lIsxAI+9ugbb3KfkQyUNUA+KiVAOSg84ehyUzYftPMHmNSVK2/QQRzcHD09nRiYgry499QHKyBsRigT/XmmZWvyyE/v+Dvs+b5ZpoVlyMakDsRwmIi1IC4kAnIuG+cmYc+Q4PQqeHoHSIs6MSEVflzM7YzpITidf21fDe43BgB4RWM53K2/SDwkVvLlaRHKIExH50hSsSEA4z/4CmPU2n9fvCYVJ/Z0clIq7Kz89ceLdrlz+SD7j+nlO6kfd+/ADs3Wr+7+EJfkEwYQV88Bd0eVjJh0g+oRqQTOzfv58hQ4Zw9uxZSpQowfTp06latWqqdX7++WdefvllYmJicHd3p1OnTjz//PO4ublx8OBB6tatS7Vq1ZLX/+yzzyhXrlyG21UNiJPERsNPn0PsRbj7SYiJgh/eh3YDwSeDJggiIvnBmTOmz8fcuVlPPq5dgQ0LYNmHsG0V1G8P45flaJgiOUE1IPajBCQTXbt2pU+fPvTt25eFCxcybdo0Vq5cmWqdHTt2UKJECcLDw7ly5Qo9evRg0KBB3HXXXRw8eJBWrVpx4MCBbG1XCUg23ciPYlZsWQFjuwEWaHYndBliJj9U22QRkczt2QLPtIfoc3BLc+j4IDTvBYWKODsykWxTAmI/usLNwOnTp9mxYwe9e/cGoFu3bhw8eJCDBw+mWq9WrVqEh4cDUKhQISIiIoiMjHRwtPlYTs5KXL8dfHEUBk4wP6RPtIC3h9jv9UVE8pLLMbB0Fsx5xfwdWt30q5u1G95Ya0azUvIhku95ODsAV3b06FECAwPx8DC7yc3NjeDgYI4cOUJYWJjV55w8eZKFCxfy1VdfJS+Ljo6mVatWJCQk0LlzZ0aPHk0BTaJkH9Y6Rvr727dddvFS0GsU9BxpRs4q7G2Wb14GKz+B9g9AnTbXh/wVEclPLBbYvck0sVrzJVy5BE16mOWFisCgCc6OUERcjBKQTLilaWpjsdhusXbx4kX69OnD8OHDqV27NgABAQHs2rULf39/zp8/z8CBA5k2bRojRozIybDzh4yGhrR3EgImwah7+/W/46+Z0VueaQelQ6Hd/SYZKWM9ORURyVPi40xH8nMn4PEm4BcMdz1pzoWlQ50dnYi4MN2yzUBQUBDHjh0jPj4eMMnH0aNHCQ4OTrdudHQ0vXr1omPHjjz22GPJy728vPD/7yK5ZMmS9OvXj40bNzrmDeR1jpocy5bG3czILW/9AvXaw/w3Yf18UxZ1WpMcikjek5hoOpKPvwceqAIJ8VAqEN7eBJ8cgH4vKvkQkUwpAcmAv78/ERERzJs3D4BFixYRGhqarvlVTEwMvXr1onXr1owZMyZV2enTp4mLiwPg6tWrLF68mJo1azrmDWTAYoF/L8Jf5+ByvLOjuUGuMCuxmxtUawSPvw9fHIdOg83yL16Fu8vAq31g40K4djXnYxERySlx12DG4zCgAjx1O+zfDt0eNbUgAJXrgSs0LT5zBtq3t29fQBGxO42ClYm9e/cydOhQzp07h7e3NzNmzKBatWoMGzaMjh070qlTJyZPnszEiRNTDc/bo0cPRo8ezaJFi5gwYQLu7u4kJCTQvHlzxo0bh5eXV4bbzclRsH4+Bo/8DLuj/tuWJzxRC56vBwVyW0rqqrMSH//XtIX++Us48AcU84GRH0CLXs6LSUQkq67Ewu8r4O9f4MFJZtmYNhBUCW7vD9WbuN5ogPlxgkhxKI2CZT9KQFxUTiUgG45D60VwLTF92fAIeKuZ3TeZ81z9R+fgLlj9BbTsA+E14PuZcHAntLwHqjd2vR9xEcmf4q7BT3Nh4wKTfFy7AmHV4c0N5iaKK3PVm1GSpygBsR8lIC4qpxKQht/CplO2y3f3gSol7b7ZnJdT84DkhO/egq9eg7PHTIf1ln3MMJWB5Z0dmYjkN8f2wz+boVUfSEiAe4OgbEVo3B2adIfgys6OMHMZDUgCSkLEbpSA2I8SEBeVEwnIpTgo9mHG67zXAh6uYdfNijUJCbBznakZWfcNPDUHbu1ommt5FYGgis6OUETyIosF9m41tRwbF0DkTihUFOadgMLFTNOr3DZPR/v2Zh4oW9q1g+XLHReP5FlKQOxHw/DmIwlZSDWzso7YQYECUKuleTz6Drj/13nzkxfhl4VQpQHc1gda9ga/IGdGKiK5XXwcHN4N5SJMgvF4U5NkNOwC/V+C+u1N8gG5L/k4cwauXct4HUcMSCIi2aIaEBeVU02wqn4B/0TZLt98J9QvbffNSlZdiYXfvjc1I5t/MBcO/1sEjbqYO5fqLyIiWXHpImxZZkbh27QEEhPg69NQsBDs32H6dnh4OjvKm5NZ0ytQ8yuxK9WA2I8SEBeVUwnIt/uhl42a6h7l4LsOdt+k3KhLF2DDd2ZG4WI+8NYjcPIgNL3DtM0uWcbZEYqIK0lqPhV9HvoEQtxVKF/LnEOadIcKtfPOTQwlH+IESkDsRwmIi8rJYXhn/w0jNkBM3PVlvSvCrJZQNJffEMvTVnwMKz+FP9eCJdEMg/nIm6a5lojkT4d2m74cvyw0A1t8FmmSjFVzzTkisJyzI7S/zJKPli3h66+VfIjdKQGxH/UByYceqAZ3VYD1x+FyAtQsBRVLODsqyVS7+83jwhnTTGvDd1DsvyHL5k+FM0egUVeo0RQK6KstkqedPwVPtIAj/4BXYajfATo/YppaFfCANn2dHWHO6ZvJeytYUMmHiItTDYiLyskaEMmD5rwM38+AcyfAuyTU7wh3P2maXIhI7nbtCmz/yfTnOHMExi0xfcLeHw01b4M6t+e+zuM3Q8PuipOoBsR+lIC4KCUgkm2JibD3d/h1sXk8Ns3Uhqz+Eo7tgzptTHMt1Y6I5A7nT8K7w2DzUrgcA2UrmP4cgyaZkfTyM008KE6gBMR+lIC4KCUgYjefjjVNtGIvQhFviGgBPUdBndbOjkxEUjp12PTlOHcCBo4zM5M/297UcDTuDuE18k4ncntIm4Qo+ZAcpgTEfpSAuCglIGJXCfGw53fYvso05egxAhp3hbVfw7pvoXZrU0MSWD77Fzi5aRZ6EVdz8Swsmm4Sj72/mxrK+h3g5UVKNrJC5x9xICUg9qMExEUpARGHWPs1fDMF9mw2TbjKhMHdT0HXIVmbd0R3IEWyJyEB/toAZ49Cq3vMkLn3V4S6bc1QuQ06mmG3RcTlKAGxHyUgLkoJiDjUpQvwx1pTQ1KjGbToBb9+D7OeMrUjtduYWdtTXhipDbZI1sRGw6Yf4Pflpn/WhTMQfgvM/MMk+Qnx6pslkgsoAbEfJSAuSgmION3erWZkre0/wfED4O4OHQfDiPfg1EkIDIBEG89VEiL5lcUCh/+BnevAoyC0GwDH/4UB5SG02n+TAvaAyvXNd0pEcg0lIPajBMRFKQERl3IiEratAm9faHYHtGsEib9BFHAO8+8Frick7drB8uXOiVXEGfZsgS/Gw871cOG0SS6a3wXPfWmSkqhTULKMs6MUkZugBMR+lIC4KCUg4tL2/QWtbwFfoCTgCUQDv/5X/v0sqNsSAsqpI624rhvpwBx9Hv7ZBH//avpy1G1r5tz5Z7OZl+OW5hDRHKo1hqLFczZ+EXEoJSD2owTERSkBkXRcbbSXlH1AimGSkPPAP3/AYzXN8hL+UPVWqNIQ7h4DBb2cFKxIGlkZQOHSRdi3FYIrQ6my8MUE+OhZU1a8FFRrBLffB7fd7bi4RcRplIDYjxIQF6UERFJx1dGmbMV14Yy5S7z7N/M4th8+2mNqQ8Z2hwKeUKkeVKpr/i3hAu9F8o+MBlDYvgy2LDNNqo78Y5YPfw+6PAz7tsHBXWZCz6BKqt0TyWeUgNiPEhAX5YwEJD4RIqNNc+VyxcFD+Y9rcPXRprJSM5NySN/Zz5rmK/u2mtG3AN76xdxN3vYTRJ+DsOrmAs/D0zHvQfKP06chrDQUxTyKAd7AVuAaML4fnNhnOolXqm/+DamqmcdFRAmIHSkBcVGOTkDm7IGnfoVjl8zfgUVgYiO4r4rDQhBrbCUfSVwlCbkRiYlmdK19W+HWzlC4KEzsBz/NNeUenhBUGe59zsyXcOGM6dxbtqISE8nclVhTg3F4Nxz6G67GwkOTzQAJcStNk8F44BJwEfgXuIoGUBARm5SA2I8SEBflyATk490wcLX1sg9bwqBqDglDrGnfHlassF2eFy+Wok7Dwb9MU5eDf0HTO6Du7fDDBzD1oeuJSXgNMz9J54dMDUtCvBKT/MZiMYnpob9NouEfDLd2gr9/gxGNrq/nGwgV68C4JSapr+Bvko2rVl4zNyf1IpKjlIDYT55JQO644w6+++47Z4dhN45KQK7EQ9CncM7aDzHgUxCO3gdFdF3nHHm5BiS7YqLgwA6I/Ou/BOUvCK0Ow6fD2ePQL9R0Fg6rYZpvBVaAtgNM05mszOourishAU4dNIlGhdrgFwSLZ8DHz5sme2CGve38CAx713Qe3zDfNJ0KqZp+ZnFXb9YoIi5JCYj95LoE5NVXX023zGKx8M0337B9+3bHB5RDHJWAbDoJDednvM6GO6BJQI6HIrboYilz0edh9RcmKTm0C47ug9iL8F2USTyGNzIJTGB5KFsBAspD8zuhdCjEXTM1J0pQnMdiMZ/h6cNw6hA06mI+j9nPwqYlcGQPXLti1n1iNrQfCH+ugz9+NglGaDXTNC87o6y56sAOInnc3ih4dyf8eQ68PaFbuGnunRv6nSoBsR8PZweQXR999BGvvPIKFkvqvKlIkSI5sr39+/czZMgQzp49S4kSJZg+fTpVq1ZNt96nn37K1KlTSUxM5LbbbmPKlCl4eJjdu2zZMl544QXi4+O55ZZbmDFjBsWKFcuReLMrPgvpZ7yt2a7FMfz8zMWRLpZs8y4J3YamXhYfdz2p6PigqTk5cQC2r4YTs8zoW6VDYc7/YP6b4B9qmvCUCoKGnc3QqpcvmaSmhJ+ZhLFoCSUq2RV3Dc6fgHMn4PxJiDoJRX2gRS+TFI5sYpKOK5euP+erU+DjbxLDao2h3UCTZIRWA79gs07Ef/Nt3Kik75UrDW0tecaJWPjzLHgVgPr+akWQZMVhuGMZxMZfX7YwEubth4UdoFCuuyqVG5XrakA6duzIrFmzKFu2bKrlDz30EO+//77dt9e1a1f69OlD3759WbhwIdOmTWPlypWp1omMjKRDhw6sXbsWf39/7rnnHtq3b8/AgQOJiYmhTp06LFmyhMqVK/Pkk09SrFgxxo4dm+F2HVUDcuEqBH4Kl+OtlxcqAMcGQElN3+B8rjYPSG5msZiHu7uZQO6vDeYi+MwROHMUGnWFPk/DXxvh8abXn+dewNSgzP5veNZ3HoVrl6G4n5kXongp0wehVFm4eNYkQd6+4FnQOe/T3iwWuBxjEoeY86a5W4EC8Ov3pnnc+ZMm2Th/EnoMN7VMy2bDG4NSv069djBhuRmIYOYo8A8xyaB/qPm3ZBnz2YjkMtcS4PENMHMXJPx3dVXSC95uBv0qOzc2Zzt/FcrPgahr1sufrQuvNnRsTNmlGhD7yXUJiMViwc1BdyBPnz5NvXr1OHDgAB4eHlgsFqpUqcLKlSsJCwtLXu/tt9/m0KFDTJ48GYAVK1bw1ltvsWTJEhYsWMDnn3/OV199BcDu3bu56667+PPPPzPctiM7oY/dBC//br3subowzsVPCCI55kosHN1jkomkR2ICdH/MlE/sB8f2meXRZ82F+eSfzZ35D5+Cr14z6xUuZpKTtvfDfS/BiUiY9bRJTDy9zKOwNwyaYNZfNtsMUZxU5ukFtVqaGpoTkXAy8vpyd3dToxAQbmobDv5lEp9rV8wj7opJqNzcYMMC05ciqezaFZMkVK4P21aZjv5Jz7l2BcrXgqFvmT4V94WbmBJTVInOOwklS8PLd8Kfa6FkgEkefMqYZlJ1b4fTRyByJ/gGmOU+/lBAtzklb3pgNXy023rZ1+2gVwXHxuNKZv8Ng9bYLi9TGE7c76BgbpASEPvJFb8CTz/9NBMmTMDNzc1hyQfA0aNHCQwMTG5K5ebmRnBwMEeOHEmVgBw+fJiQkJDkv0NDQzly5IjNsuPHj5OYmOgyEw2ObQBxifD6juvNrQq4wRO14OVbnRubiFMVKmI6Pdvy9JzUfyckXP9/x8FQo5lJTJKSl/Bb/lsvDi6egbir1x8eBYH/EpClH5iL9rirJpkA+N8ik4Cs/hw+ei71dlvcBc9/ZWoehtZNH+f3V0z/iAVvm4khCxa6/qhUzyQgVy+bmAoWgkLFTK2OX5B5fuFicPdTJtEp9t8j6f8AL3xju2maf7B5iORxu8/bTj4AnvkN7iyff1txHrmUcfnJyxCXAJ6acidfyBUJyLp167jnnnuYNWsWRYsWTV5+5coVZs2axaOPPppj206b8KTte2JtvbTrODJpuhHubjC+EQyLgE2nwALcWhrKFs30qSKSUsrJ6oIqmoc1QZVg0o+2X+etX67/PzHxvwTlv0bkXYeahCPumlluSYRiJU1ZyTIwbYupYfAqbJIJz0LXm4C9/pPtbTbqYh7WuLtD76dsP9fFz3F2paaQYsMvJzMu33cBzlwB/8KOicfVBGVyTVG6sJKP/MQ1bsFnYvny5VgsFjp06MDx48e5evUq06dPp1atWsyYMSPHthsUFMSxY8eIjzcdJCwWC0ePHiU4OPXdvJCQEA4dOpT89+HDh5PXSVt26NAhAgMDXab2I6XAotC9HPQop+RDxGW4u5tkIqnZUjEfk8CE14BKdU3tRdn/2nV4FoTK9aBCLTMkcelQ00QqPyUIOSlp5KwVK8y/Z844OyJxIVn5lrnn46/iHeWgeAbd4e7XxMf5iutdBVtRrFgxvvzySxo1akTr1q2pVasWM2fO5JlnnmHbtm05tl1/f38iIiKYN28eAIsWLSI0NDRV8yuAbt268f3333Pq1CksFguzZ8/mzjvvBKBNmzZs3bqVPXv2ADBr1qzkMhERySWsDYetJERSaB6YcXm1kuCbjwd08S0EX9xuBrdJq2VZeKmB42MS58kVndCvXr3KJ598wltvvcW1a9e4cOEC3377Lc2b38QQjFm0d+9ehg4dyrlz5/D29mbGjBlUq1aNYcOG0bFjRzp16gTAJ598kjwMb4sWLXjjjTfw9DRNJn744QfGjh1LfHw81atXZ8aMGRQvXjzD7TqyE7qIiGRAE4JKFg1fD+9YGWPGDfi+E3QKS1+W3/x93uyjneeg2H/zgAyqmjuaX6kTuv3kigSkSpUqeHp6MmrUKPr3789XX33FM888w+uvv07v3r2dHV6OUAIiIuIi2rc3za5sadcOli93XDzishIS4YVN8OYfcOW/MSmCisK7zU0TZ8ndlIDYT65IQD744AMGDBhAwYLXGw+uX7+eAQMGMHDgQJ5//nknRpczlICIiLgI1YBINl24Cv9EmYkIa/jmjlm+JXNKQOwnVyQgthw4cIDevXuzefNmZ4did0pARERciK0kRMmHSL6hBMR+cnUCAhAVFYWPj4+zw7A7JSAiIi4mbRKi5EMkX1ECYj+5/go3LyYfIiLigvz8TNLRrp2SDxGRm5Dra0DyKtWAiIiIiLgO1YDYT66YCV1ERERE8p/1x+HV3+H306ZTf7sQeLkBBBVzdmRyM1QD4qJUAyIiIiL52bf74e6VkJjmSjWgCGy8A8plPKWa3akGxH50hSsiIjfnzBkzV4ZmBRcRO7l4DR5Ykz75ADgRC0PXOjwksSM1wRIRkRuXcmQof391zhYRu1h52CQhtiw7DNHXwLug7XXEdakGREREboy1uTH8/TOuCVFtiYhkwfmrma8TlUGCIq5NNSAiIpJ9Gc0ObqsmRLUlIpJFlX0yLvf2hIDCDglFcoBqQCRPio2Dt/+Ansug+1IzgsbZK86OSsR+TsbC2mOw7TQkJDohgL59s1d+I7UlIpJvNQuEuhncn3j0FvAs4Lh4xL40CpaL0ihYN+70ZWi7GHacTb08uCis6pb5XRURVxYbB4+th0/+ud45M8wbZrWENsEODCSjGhBIXbuRnXVFRP5zMBo6fA+7o1Iv710RPm0NBR2cgGgULPtRAuKilIDcuDuXwfx/rZfV9oOtvcDNzbExidiDxQI9lsGiyPRlBd1hbQ9oWMaBAdlKLNImFO3bw4oVtl+nXTtYvtz+8YlIrnc5Hr7aB1vPmPNc+1BoE+Sc33ElIPajBMRFKQG5MWcuQ+mPIaODetOd0KC0oyISsZ+fj0HLhbbLWweZWj6HSpuEZNb3wxrVgIhILqAExH50hSt5ypFLGScfYKp0RXKj9cczL7c4+paSn59JINq1s51IJK1jjZIPEZF8RwmI5CmBRTJfp2zRnI9DJF/x8zNNqDJKJKwlIUo+RETyJSUgkqeUKQLtQmyXVy4BjRzZRl7EjpoEZF7u0v2bslJbIiIieZ4SEMlzPmwJ5bzTL/f1gnntwD2HL9BOxsLOsxCVhUmURLKjZVnoHGa9zNMdxjd0bDw3JCu1JSIikqcpAZE8J6QYbOkFL9SDZgHQuAyMrAnb7zajYOWUyItw+yII+AQivgK/j2DAKrigRETsxM0N5rWFAVVSJ9IhxWBJJ2gcgGYad1X6XEREkmkULBelUbByl5OxUO8bOHopfVnTAPipm+PHK5e87fglMza+t6dJrD3cydqIVOJ4+lxE8gSNgmU/usIVsYOJ26wnHwAbTsCX+xwbj+R9gUWhVRDUL20j+QDNNO4K9LmIiKSjBMSG2NhYBg0aRJ06dahXrx6LFi2yut7x48fp2bMn9evXp0mTJtx///2cP38+uTwiIoIGDRrQrFkzmjVrxvz58x31FsSBVh/NuPynTMpFbkpG82zoYtd59LmIiFilBMSGd955h4IFC7Jt2za+/fZbRo8eTVRUVLr1ChQowJNPPsmWLVvYuHEjISEhvPTSS6nW+eSTT1i/fj3r16+nZ8+ejnkD4lDxiTdXLnJT+va9uXLJGfpcRESsUgJiw3fffcfgwYMBCA8Pp0mTJixZsiTdeqVLl6Zx48bJf9evX5/IyEhHhSkuIrOZ1W/VzOuSk+bOvblyyRn6XERErFICYsORI0cICbk+oURoaChHjhzJ8DkJCQl88MEHdOjQIdXywYMH06RJE4YNG8YZVbnnSU/XgaIe1ssqFIf7qzo2HslnNNO4a9LnIiJiVb5NQDp27Ej58uWtPpISDbcUM3pZLBkPFmaxWHjiiScoUaIEDz/8cPLyH374gQ0bNvDzzz/j6+vLkCFDcuYNiVNVKQnLu5jhUFOq7w8ru0Lxgs6JS/IRzTTumvS5iIikY+Oebd63dOnSDMuDg4M5dOgQfv/9SBw+fJi2bdvaXH/MmDEcPXqUuXPnpho+N6kWxdPTkyFDhlC/fn07RC+uqGkg7L/XjHp16jKUK24SEJeemVrylqSL3b59TfMeXeS6Bn0uIiKpaB4QGyZMmMChQ4eYMWMGkZGRtG3blk2bNlGyZMl0644ZM4YDBw4wd+5cvLy8kpdfunSJuLg4fHx8AJg2bRpLlizJNPkBzQNib9tOw5t/wM5zZt6EbuHwWAR4aW4OERERyQLNA2I/SkBsuHTpEo899hjbt2/H3d2dF198ke7duwMwe/Zsjh8/znPPPcevv/5Khw4dqFy5MgULmnY2YWFhzJ07l8jISPr3709CQkLy8okTJxIWFpbp9pWA2M+3++GeHyEuzUhUjcuY5lFFPZ0Tl4iIiOQeSkDsRwmIi1ICYh+nYqHcXIiNt14+qhZMaeLYmERERG7EuSvw7k745SS4AbeVhUdqqJ+hoygBsR8lIC5KCYh9zPwLHllru9yvEJwe6Lh4REREbsT+C9B6ERyKSb28cglY3R3KFnVOXPmJEhD70RWu5GnHYzMuP3NFkwSKiIhrs1hMU+K0yQfAngvwwGrHxyRyM/LtKFiSP4R5Z1weVBQ8lIbnG1tOwYStsPWMGYCgbTC8UA9KF3F2ZCIitu08B5tP2S5ffhgOx6QfCl7EVenSS/K0nuWgVCHb5Q9Vd1ws4lxLDkKT72D+vxAZDf9EwbSdUP9b88Od21ks5n3sOgexcc6ORkTsKSvnqLxwHpP8QwmI5GklvOCbdtY76HUPh2fqODwkcYJLcdB/VfqR0MD8aA/NoJ9QbvD7aaj3DYR+BjXmgf/HMOYXiEtwdmQiYg9BWejfkZV1RFyFmmBJntcyCP68G977C/46D8X+mwfkrgrgrkkC84WVR+D8VdvlSw5CTJw5NnKbP8/CbQvgUoqR3mLj4fXtcDIWPm6tyTBFcruapaC2H2w/Y728VdnMmxyLuBIlIJIvhHrD+EbOjkKc5czljMstmOEtc2MC8tSvqZOPlD7dA0NvgYZlHBuTiNiXmxt8cTu0WgQn0gyuEu5tbjSI5CZKQEQkz6tYIuPyIh4QkAs7olsssPpoxuusPqoERCQvqFoStt8Fb/0BG0+aNvQtysLwCPDNoK+jiCtSAiIieV6LshDhC3+es17+cHUoWMCxMdlLfCYzOWVWLiK5R5kiqs2XvEGd0EUkz3N3g/kdoHzx9GVdwmB8Q8fHZA9ubtDAP+N1MisXERFxNM2E7qI0E7qI/UVfgzl7zKhRXgWgbYgZkCA3D0bw4xFot9j0Y0mrRaCZITk3vz8REVehmdDtRwmIi1ICIiJZ9c1+GLwGoq5dX9Y5DD5rAyW9nBaWiEieogTEfpSAuCglICKSHZfiYN1xiI6DGiWhuq+zIxIRyVuUgNiPOqGLiOQBRT2hQ6izoxAREcmcbrGLiIiIiIjDKAERERERERGHUQIiIiIiIiIOoz4gIiIikmttPgWrj0KCBRqVgZZlzRw5IuK6lICI5DLxiTB9J8zbD2evQMUS8Ngt6oAsIvlLXAIMWgOf7Um9vG0wfNMeihd0RlQikhUahtdFaRhesSYuAbothWWH05e91gierOP4mEREnGH0Rpiyw3pZz3LwbQfHxiN5n4bhtR9d4YrkIm/9aT35AHjqV9hxxrHxiIg4Q/Q1eHen7fL5/8K+C46LR0SyRwmISC4yd6/tMgvw5T6HhSIi4jT7L8KVhIzX+eucY2IRkexTAiKSi5yMzbj81GXHxCEi4kzenpmvUywL64iIcygBsSE2NpZBgwZRp04d6tWrx6JFi2yu6+PjQ5MmTWjWrBnNmjVj48aNyWX79++nXbt21KtXj9atW7N7925HhC95VPniGZeXy6RcRCQvKF8cqpe0Xe7rBU0CHBePiGSPRsGy4Z133qFgwYJs27aNyMhI2rVrR4sWLfDx8bG6/ooVKyhWrFi65SNHjmTAgAH07duXhQsXMmzYMFauXJnD0UteNaQGbDhhvayIB9xX2bHxiIg4g5sbfNgSWi9K3xTLDXj/NiisKxwRl6UaEBu+++47Bg8eDEB4eDhNmjRhyZIl2XqN06dPs2PHDnr37g1At27dOHjwIAcPHrR7vJI/3FsJnqiVfnlhD/j8dgj1dnxMIiLO0DgA1t8BtweDpzsUcDPzgKzoCndWcHZ0IpIR3R+w4ciRI4SEhCT/HRoaypEjR2yu36VLF+Li4rjtttt47rnnKFq0KEePHiUwMBAPD7Ob3dzcCA4O5siRI4SFheX4e5C8x80NJjeBruEwbx+cuQKVSsCgapk3zxIRyWvq+cPKrpBoAYsFCui2qkiukG8TkI4dO/LPP/9YLVu7di1gEoYkFovt6VL+/PNPQkJCuHTpEo8//jgvvvgiU6ZMSfcamb2OSFbdVtY8REQE3N0wba9EJFfItwnI0qVLMywPDg7m0KFD+Pn5AXD48GHatm1rdd2kmpKiRYvy4IMPMnLkSACCgoI4duwY8fHxeHh4YLFYOHr0KMHBwfZ7IyIiIiIiuYgqK23o3r07H3zwAQCRkZFs2LCBTp06pVsvKiqK2FgzNmpiYiLz588nIiICAH9/fyIiIpg3bx4AixYtIjQ0VM2vRERERCTfcouKilKbICsuXbrEY489xvbt23F3d+fFF1+ke/fuAMyePZvjx4/z3HPPsWnTJkaOHImbmxsJCQnUrFmTSZMmUbKkGR9w7969DB06lHPnzuHt7c2MGTOoVq1aptv39vbG3V35oYiIiIgrSExMJDo62tlh5AlKQFyUEhARERER16EExH50hSsiIiIiIg6jBERERERERBwm346CJSLiSIkWWHcc/j4PPgWhfSiU9HJ2VCIiIo6nBEREJIcdjoGey2DL6evLinjA+7dB38rOi0tERMQZ1ARLRCQHXU2Ajt+nTj4AYuOh/yr48Yhz4hIREXEWJSAiIjlo/gH467z1Mgsw7neHhiMiIuJ0SkBERHLQtjMZl289nXG5iIhIXqMEREQkBxUqcHPlIiIieY0SEBGRHNQuJOPy9qGOiUNERMRVKAEREclBTQPg/irWy0oXhvENHRuPiIiIsykBERHJQW5u8EFLeKl+6nk/2ofAxjsgpJjTQhNxuKsJsPMs7DoHcQnOjkZEnMUtKirK4uwgJD1vb2/c3ZUfiuQlCYlw6jIULwhFPZ0djYjjWCww4y94cTOcvWKWBRSB1xtDP82F45ISExM5efIk8fHxzg7FaTw8PChTpkzy9VhiYiLR0dFOjipv0ESEIiIOUsAdAos6OwoRx3v7Txi5IfWyE7FmLhyLBfrbaKYoznPy5Em8vb0pViz/VtPGxMRw8uRJAgMDnR1KnqNb7CIiIpJjLlyF536zXT76F7im5lguJz4+Pl8nHwDFihXL1zVAOUkJiIiIiOSY30/DpQyu4U5dhr9tTNYpInmTEhARERHJMQlZ6GmalXVEJO9QAiIiIiI5prYfeGZwtVG8IFT1cVg4kkvVrl2by5cvWy0LDw9n586dN/zaa9asoX79+jf8fMk+JSAiIiKSY/wLwxO1bJe/UA+KaFQ4ycT27dspXLiws8MQO1ECIiIiIjnq1YYwpjZ4pLjq8CoA427NODkRSeLm5kZMTAwA69atIyIigltvvZXHHnsMi+V6G769e/fSuXNnGjRoQK1atZg+fXpyWb9+/ahfvz41a9akS5cunDp1yuHvQwwNwysiIiI5yt0NJjWGx2uZTunuwK1loFQhZ0cmWXb2OJw7nnpZsZIQWA6uXYGDu9I/p1Jd8+/hf+DKpdRlZcKhuG+2w7h69Sp9+vRh7ty5tGzZkq+++op3330XgISEBO69914+++wzqlatSmxsLI0aNaJRo0bUrVuXqVOn4ufnB8DEiRN5+eWXmTZtWrZjkJunBEREREQcIqAIdA5zdhRyQ5bMhDn/S72sdV94eg6cPgKP1kv/nBX/1UxMvh/+/jV12ZjP4PZ+2Q7jn3/+oUiRIrRs2RKAu+++m4ceeii57K+//qJPnz7J60dHR7Nr1y7q1q3L3Llz+eyzz7h69SqXL18mICAg29sX+1ACIiIiIiIZ6/wwNO6WelmxkuZf/2B493fbzx39sfUakBuQsrmVtTI/Pz+2b9+ermz9+vVMmzaNjRs34u/vz6JFi3j55ZdvKAa5eUpAbIiNjWXYsGFs3boVd3d3xo4dS7du3dKtt3v3bh588MHkvy9cuEB0dDSRkZEAREREUKhQIby8vAAYNWoUPXv2dMh7EBEREbGLUoHmYU3BQtebW1kTYr+p7qtWrcrly5dZu3YtLVq04JtvvuHChQsAVKlShSJFivDpp59y3333AbBv3z58fX05f/48xYsXx9fXl2vXrjFz5ky7xSTZpwTEhnfeeYeCBQuybds2IiMjadeuHS1atMDHxyfVelWrVmX9+vXJfz/55JPpXuuTTz6hevXqOR2yiIiISJ7m5eXFF198wdChQylcuDAtW7YkNDQUAA8PDxYvXszjjz/O5MmTSUhIwN/fn7lz59KxY0fmzJlD1apVCQ4OpkmTJixfvtzJ7yb/couKitL0P1Y0atSI6dOnU7euyejvv/9+2rZtS9++fW0+5+rVq1SpUoVFixZRs2ZNwNSAzJs3L9sJiLe3N+7uGqRMREREHO/w4cOEhIQ4OwynS7kfEhMTiY6OdnJEeYNqQGw4cuRIqi9eaGgoR44cyfA5ixcvJiwsLDn5SDJ48GAsFgv16tVj7NixySMwiIiIiOR3cQlwLBYuXIVEoLCHGbCgREFnRyY5Jd8mIB07duSff/6xWrZ27VrAjDmdJKNOT0nmzJlD//79Uy374YcfCAkJIS4ujnHjxjFkyBC+/vrrm4hc7OHCVVgUaU54ocWgazgU00RYIiIiDnUtAXZHmX+TRF8zj3Bv8NPcg3lSvk1Ali5dmmF5cHAwhw4dSq6tOHz4MG3btrW5/qFDh9i0aRMff/xxquVJtSienp4MGTKE+vXr31zgctOWHYJ7VkLUtevL/AvBt+2heVnnxSUiIpLfHL6UOvlI6WCMqQXxLODYmCTnqZOBDd27d+eDDz4AIDIykg0bNtCpUyeb68+dO5cuXbqk6qR+6dIloqKikv/+5ptviIiIyKmQJQt2nYM7lqVOPgBOX4EuS+GQmnaKiIg4hMUCUVczLr9wzXa55F75tgYkM8OHD+exxx6jTp06uLu7M3nyZEqWNONdz549m+PHj/Pcc88BpnnW559/njwTZ5LTp0/Tv39/EhJMah8WFsZ7773n2DciqUz9A67YuNNy8Rq8u9PM1iv5x94oeG07bD4Fnu7QOgjG1NEMzSIiOS3RYpKMjMRrqKQ8SaNguSiNgpUzGs+HX0/aLu8QAku7OC4eca6NJ6D99xATl3p5mDes6wEhxZwSloiI0zliFCyLBf44C3GJttepWAJ8vHI0jAxpFKycoStcyVeKZlLnV1Qd0fONuAToszJ98gFwMBoeWuPwkERE8hU3NyhTxHZ5oQJQ3A4jYfXq1YuyZcvi5uZGTEzMzb+g3DQlIJKvdA7LuLxLJuWSd6w7Docz+B1adhjOXnFcPLnJsUuw4wyc0/4RkbTOnIH27c2/WVCmsPWRrrwKmNoPd7f0Zdn1yCOPsH379pt/IbEb9QGRfGVIDZi3D347lb7s9mDoV9nxMYlznIjNfJ1Tl9UXJKX9F2DQGvj5mPm7gBv0rgjvNnduEwkRZ0m0wNJD5jthsUCTADOsu0d+vb175gz4+5v/+/vD6dOQydxnbm7/DbfrZQaISbRAEQ8o6QUF/tuPr7/+Ovv27WPmzJkAREVFUbFiRfbs2YOvr2+mYd1+++039bbE/pSASL5SyAN+7AbP/Aqf/APRceYk92A1eLlBPv7RyIfKFc+43MMdgoo6Jpbc4EQsNF8Ax1MkbgkW+HwvREbD6m5QUENlSj4SEwc9l8HKlHMU74CmAbC4k/ltyVdSJh9JspiEABQraB7WDB48mCpVqvDaa69RokQJZs2aRffu3Tlx4gStW7e2+pw6derw0UcfZfddiIOoE7qLUif0nGexmB+QYp7mDozkL4kWqPs17Dhrvfy+yvBJG8fG5MpGrIe3/7Rd/klruK+K4+IRcbYBq+DTPdbLuofDgo4ODcfustUJ3VrykVIWk5CMDB06lCpVqjB8+HAqVarE119/TZ06dbL1Gm5ubkRHR1OsWNZHGFEn9JyhGhDJt9zcwDubndtOxZq7XTFxcIuvqW5X8pI7ubvBN+3h9sWm03lKjcvA282cE5erWn0083IlIJJfnL4Mc/baLl8YCQcuQvlMalrzjL59My9fvvymNjF8+HB69OhBhQoVKFOmDHXq1GHXrl3ce++9VtdXDYhrUwIikkXTd8LjG+BaiuECmwWYGdRLZzCKh7iuiiVg+13w4d+p5wHpVzn3NSfaGwUvbIY1R03TqIZl4KX6UL+0fV4/IZO68szKRfKSfRdMLWpG9kTlowRk7tyMa0Dmzr3pTVStWpXw8HCGDBnCa6+9BkD16tXVuTyXUhsfkSz4Zj88ui518gGw/gR0WwoJGYxhLq7NxwtG14Z57WDO7fBAtdyXfPxxFhp8awZYOHkZzlyBJQeh6Xew4rB9ttGwzM2Vi+QlWenfka/6gPj5mWZW1tih+VWSwYMHEx8fT69evbL1vG7duhEcHAxAlSpVaNmypV3ikRunBEQkC1753XbZb6fSdEIUcSCLBQb+BBeupS+7lggDfoJrCTe/nafrmP5S1lQqAQPU/ErykSo+UKOk7fLQYlAvgwqBPMlaEmLH5ANg1apVDB06FE/P7E3atWjRIo4cOYLFYuHo0aOsWbPGbjHJjVECIpKJ+ET400ZH5STbsjbcuYjdHYyGrRkcfydi4ZeTN7+dyj6woouZJT6lRmXgx662kxORvMjNDT5qbf249yoAn7bJp6MqJiUh7drZNfk4duwYVatWZfv27YwcOdIurynOpT4gIplwdzN9A9I2v0rJK5c12ZG84/xV+6yTFY0DYN+98MsJM0dK+eJQ208DMUj+1KA0/NYTxm6GtcdNn5AmAabvVZ38VvuRkp/fTXc4T6ts2bLs3r3brq8pzqUERCQT7m7QNsS0qbelfRZHKnQ0iwUs2Gcm2YxcijM/wnP3mgvTct5m0seRNa9PJCU5o1xxkyDHZZAgV/Gx3/Y83KF5Wfu9nkhuVt0Xvm7v7ChEch9dGohkweTGUMLGkL2P14QamU/E6lAHo+HuFeD9IXjOhJrz4MsMhoy8GZfjofUimLLDNPdJtMD+izD6Fxi42iRBknN8vOCBqrbL24VAtQzaqouIiDiaEhCRLKhaEjbeAS1T3Pn1LwSvN4bJTZwXlzWRF+HWb+Hr/XAp3iQEf56De36EKdvtv703d8CmU9bLPtsDPxyy/zYltTeaQKfQ9MsblIY5mkxRRERcjJpgiWRRdV9Y3R2ir0FsPPgXzvmmTTfi8Y2mGZQ1T/0Kd1WAUG/r5TdiYWTG5Qv+hc5h9tuepFfEExZ3guWHYM2x6/OA9AgHT/VPEhERF6MERCSbvAtmfwZ1R4lPzLivSoLF1Eg8UsN+27xoZfjX7JTb065zsDsKfApC08CbHxwgqflYbuhk7e4GHcPMQ0RExJUpARHJQ64mZNwZGSAmzr7brF7SXPTb4oj+MWcuQ/9VsCzFpHtlCsPHraGDlaZJmdlyCp79zYxsY7GY0Z9evdUkNZKz4hJg6h9mQIMTsaaT/cPVzTwjuSERFMnPzl2B9cfhaiLU9YMKJZwdkfHbb7/x8MMPExsbS0hICHPmzCEwUCd0Z1IfEJE8pKgnhGfSvCqjybNuxOO1wNZ1oU9BGJRBB2l7iEuALj+kTj7AzAjebSn8ls05MNYeg2YLzOSSVxPM8Ms/H4OWi2CZ+rPkqPhE6L4MxvwKO86az/DXk2YwgxEbnB2diGTkrT8g6FPzHb57BVT8HO7/Ca7EOzcui8VC3759mTp1Knv27KFjx46MGjXKuUGJEhCRvGZMbdtltUqZUZHsqVkgfNQKCqY5m5QqBAs7QlAx+24vrcUHzWz01sQlwkubs/5aiRYYtMYkHmnFJ5qyODvMKi7WvfcXLLWR5L3zJ/x4xLHxiEjWzPobRm6AK2nOj5/8A4N/vvnXf/3113n44YeT/46KisLPz49z585l+twtW7bg5eVFy5YtAXj44YdZsGABcXF2bg4g2aImWCJ5zCM14HAMTNiWenmELyzqmDPzcgyoCm2C4dsD1+cB6VXBDBGbmdg4mLkLfjpqEob6/vBYBAQUydq2M6vh+DUbNSB/n4d9F2yXH7sEW06bJllif1/tz6R8H9we7JhYRCRrEhLhxQxu9MzZA8/XhSo3Ufs+ePBgqlSpwmuvvUaJEiWYNWsW3bt358SJE7Ru3drqc+rUqcNHH33EoUOHCAu73jnO29sbb29vjh8/TmjoDbTRFbtQAiKSx7i5wfhG0K+y6XAeG2+Sjy5hOTsiUnAxGFEze885ewXaLDLNbZIsPwwf/A0/doWIUpm/RoFM+gV4ZCPhykqHeUd2qs9vzl3JuPxsJuUi4njHYs3NmYxsPn1zCYiPjw933nknH3/8McOHD2fGjBl8/fXXVK9ene3bt2f6fLc0HcgsmqDK6ZSAiORR1X3Nw5U9ui518pHk1GXotRx29cm8xqZlUPranlTl2Zi1u7KPSVjibXTkd8P192luVrEE/HXednklH4eFIiJZ5JmFmzwedhhAYvjw4fTo0YMKFSpQpkwZ6tSpw65du7j33nutrp9UAxIaGkpkZGTy8ujoaKKjo9UJ3cmUgIiIU1y4aiZLtGXPBdh4AppnkkDcHgzdwmFRZPqy4gVh3K1Zj6lUITOr+Pu7rJf3qQghOdynJT8bFmF7XplCBeDBag4NR0SyoExhqFbSNGG1poAbtMjGjSBbqlatSnh4OEOGDOG1114DyFINSL169bhy5Qpr1qyhZcuWzJw5kx49euDp6XnzQckNUyd0Gz777DOaNGlCqVKleP/99zNcd8uWLTRr1ox69erRrVs3Tpw4kVy2f/9+2rVrR7169WjdujW7d+/O6dBFcoVjsabTd0YOxWT+Ou5u8GVbGFIjdUf4un7wc/fsV/tPbQp3lEu/vH0IzLwte68l2dMm2Oz/tBN8FvGAL9qaGhIRcS1ubvB2M9vNYZ+vB2WL2mdbgwcPJj4+nl69emX5Oe7u7syZM4cRI0ZQuXJllixZwpQpU+wTkNwwt6ioKDWEs+LPP/+kYMGCvPHGG9SrV4+HHnrI6noWi4W6devy9ttv07x5c9555x22b9/OrFmzAOjatSt9+vShb9++LFy4kGnTprFy5cpMt+/t7Y27u/JDybvOX4VSsyGjE9Ca7nBbNu6cXbgKkdGm83tosRufN8JigXXHzcMCNC4DrYM0D4Wj/HUOvthrhuEN94b+lSE0k+GlRcS+Dh8+TEhI1odNXHEYHlsHe/8byMPXC56tC6Nq2e/cOXToUAIDA3nhhRfs84JZkHI/JCYmEh0d7bBt52VKQDIxZMgQ6tSpYzMB2bp1K0OHDuXXX38FTNvCSpUqcfjwYaKioqhXrx4HDhzAw8MDi8VClSpVWLlyZaoRGaxRAiL5wZ3LYP6/1svKF4c99+TMqF0iIpKx7CYgYGq1D0Wb+ZPCvaGgnQY+OXbsGK1bt8bX15fly5fj7e24OxJKQHKG+oDcpLRfUG9vb4oVK8aJEyc4e/YsgYGBeHiY3ezm5kZwcDBHjhzJNAERyQ+mt4C/o9K3Hfb1gq/bKfkQEclN3N0gvLj9X7ds2bJqwp7H5NsEpGPHjvzzzz9Wy9auXUtwcNYHm89oeDcN/SZiW5ki8FtPmLYTVh0x84A0KA0ja5phfUVERCTvybcJyNKlS+3yOiEhIRw6dH3q3ujoaGJiYggICKBQoUIcO3aM+Pj45CZYR48ezVZyI5LXeReEZ+qah4iIiOR9auBwk2rXrs2VK1dYt24dAB9//DGdO3fG09MTf39/IiIimDdvHgCLFi0iNDRUza9ERETEpXl4eBATk4WhCPOwmJiY5Gb0Yl/qhG7DvHnz+N///kdUVBSenp4ULVqUL774glq1ajF79myOHz/Oc889B8CmTZt4/PHHuXLlCoGBgbz//vuULWuG7tm7dy9Dhw7l3LlzeHt7M2PGDKpVy3wwe3VCFxEREWdJTEzk5MmTxMfHOzsUp/Hw8KBMmTLJ12PqhG4/SkBclBIQEREREdehBMR+dIUrIiIiIiIOo4ZtLioxMdHZIYiIiIjIf3RtZj9KQFzUpUuXnB2CiIiIiIjdqQmWiIiIiIg4jBIQERERERFxGCUgIiIiIiLiMEpARERERETEYZSAiIiIiIiIwygBERERERERh1ECkofFxsYyaNAg6tSpQ7169Vi0aJHV9Xbv3k2zZs2SHxEREYSHhyeXR0RE0KBBg+Ty+fPnO+gdOF9W9yGAj48PTZo0Sd5PGzduTC7bv38/7dq1o169erRu3Zrdu3c7InyXkNV9ePz4cXr27En9+vVp0qQJ999/P+fPn08uz4/HYVaPm08//ZS6detSu3ZtRowYQXx8fHLZsmXLaNCgAXXq1KF///7ExMQ4KnyXkJV9+PPPP9OmTRsaNmxI48aNeeWVV7BYLAAcPHiQUqVKpTpH/vvvv45+G06Xlf24bt06AgMDU+2ry5cvJ5frWMx8H37xxRep9l/58uXp168foGMRYMyYMURERODj48OuXbtsrqdzoutzi4qKsjg7CMkZkyZNIjIykhkzZhAZGUm7du3YtGkTPj4+GT7vySefBOD1118HzIXfvHnzqF69ek6H7HKysw99fHw4cuQIxYoVS1fWtWtX+vTpQ9++fVm4cCHTpk1j5cqVDngHzpfVfXjq1Cn2799P48aNAXjhhRe4ePEib731FpA/j8OsHDeRkZF06NCBtWvX4u/vzz333EP79u0ZOHAgMTEx1KlThyVLllC5cmWefPJJihUrxtixY530jhwvK/twx44dlChRgvDwcK5cuUKPHj0YNGgQd911FwcPHqRVq1YcOHDASe/ANWRlP65bt44XXniBNWvWpHu+jsUb+x1o0qQJTz31FN27d9exCGzYsIHw8HA6dOhg8/dA58TcQTUgedh3333H4MGDAQgPD6dJkyYsWbIkw+dcvXqVr7/+mv79+zsiRJd3I/swrdOnT7Njxw569+4NQLdu3Th48CAHDx60e7yuKKv7sHTp0snJB0D9+vWJjIx0VJguJ6vHzaJFi+jSpQulS5fGzc2NBx54gG+++QaAH3/8kTp16lC5cmUABg0alFyWH2R1H9aqVSu51rdQoUJERETk62MvLXucw3QsZn8f/v7775w6dYpOnTo5KkyX17RpU4KCgjJcR+fE3EEJSB525MgRQkJCkv8ODQ3lyJEjGT5n8eLFhIWFUbNmzVTLBw8eTJMmTRg2bBhnzpzJkXhdUXb3YZcuXWjatCnPPvts8mz2R48eJTAwEA8PDwDc3NwIDg7O9LPIK27kOExISOCDDz6gQ4cOqZbnp+Mwq8fN4cOHbe5fa2XHjx8nMTHRAe/A+W7ku3fy5EkWLlxI27Ztk5dFR0fTqlUrWrRowaRJk0hISMjx2F1Jdvbjvn37aNGiBa1ateLDDz9MXq5jMfvH4meffUbv3r3x9PRMXpbfj8Ws0Dkxd/BwdgBy4zp27Mg///xjtWzt2rWAOcklSWrTnJE5c+akq/344YcfCAkJIS4ujnHjxjFkyBC+/vrrm4jcddhzH/7555+EhIRw6dIlHn/8cV588UWmTJmS7jUye53cxt7HocVi4YknnqBEiRI8/PDDycvz8nFoS1aPm4z2b9rXyG+y8927ePEiffr0Yfjw4dSuXRuAgIAAdu3ahb+/P+fPn2fgwIFMmzaNESNG5GTYLicr+7FWrVr89ddflChRgqNHj3LXXXdRqlQp7rjjDquvkd9k51iMjY1l/vz5rFixInmZjsWs0znR9SkBycWWLl2aYXlwcDCHDh3Cz88PMJl/yrt6aR06dIhNmzbx8ccfp1qedLfA09OTIUOGUL9+/ZsL3IXYcx8m7aeiRYvy4IMPMnLkSACCgoI4duwY8fHxeHh4YLFYOHr0KMHBwfZ7I05k7+NwzJgxHD16lLlz5+Lufr2SNi8fh9Zk9bgJCQnh0KFDyX8fPnw4eZ2QkBDWrVuXXHbo0CECAwNT7de8LDvfvejoaHr16kXHjh157LHHkpd7eXnh7+8PQMmSJenXrx9ff/11vrroy+p+LF68eKrn9OrVi40bN3LHHXfoWMzm78DChQupUqUKVatWTV6mYzFrdE7MHbTH87Du3bvzwQcfAKZT1oYNGzJsSzp37ly6dOmSqnPwpUuXiIqKSv77m2++ISIiIqdCdjlZ3YdRUVHExsYCkJiYyPz585P3k7+/f3IHajDtU0NDQwkLC3PQu3Cu7ByHY8aM4d9//2XOnDkULFgweXl+PA6zetx069aN77//nlOnTmGxWJg9ezZ33nknAG3atGHr1q3s2bMHgFmzZiWX5QdZ3YcxMTH06tWL1q1bM2bMmFRlp0+fJi4uDjB95BYvXpyuiWpel9X9eOLEieSmLNHR0Sxfvjx5X+lYzN7vgLXWCDoWs0bnxNxBo2DlYZcuXeKxxx5j+/btuLu78+KLL9K9e3cAZs+ezfHjx3nuuecAU0VZs2ZN3n33XVq0aJH8GpGRkfTv3z+5nWlYWBgTJ07MNxfPWd2HmzZtYuTIkbi5uZGQkEDNmjWZNGkSJUuWBGDv3r0MHTqUc+fO4e3tzYwZM6hWrZoz35rDZHUf/vrrr3To0IHKlSsnJx9hYWHMnTs33x6Hto6bYcOG0bFjx+RE7pNPPmHq1KkkJibSokUL3njjjeR24z/88ANjx44lPj6e6tWrM2PGjFR3qvO6rOzDyZMnM3HixFR3m3v06MHo0aNZtGgREyZMwN3dnYSEBJo3b864cePw8vJy4rtyvKzsx/fff5/Zs2dToEABEhIS6N69O08//XRykxcdi1n7Pv/77780b96cv//+G29v7+Tn61iE0aNH88MPP3Dy5ElKlSpF0aJF2bZtm86JuZASEBERERERcRg1wRIREREREYdRAiIiIiIiIg6jBERERERERBxGCYiIiIiIiDiMEhAREREREXEYJSAiIiIiIuIwSkBERERERMRhlICIiIiIiIjDKAERERG7eP/996lcuTKXL18GIDo6miZNmjBmzBgnRyYiIq5ECYiIiNjFgAED8PT05JNPPiExMZEHH3yQ4OBgJkyY4OzQRETEhXg4OwAREckbvLy8GDVqFFOmTGH//v0cOXKEZcuWUaBAAWeHJiIiLsQtKirK4uwgREQkb7h27RrVq1enQIECrFq1iuDg4OSyefPm8eGHHwLw/PPPc9tttzkrTBERcSLVgIiIiN18/fXXXLhwgZIlS1KqVKnk5RcuXGDq1Kn89NNPXL58mc6dO7N+/XrVjoiI5EPqAyIiInaxfv16nnrqKb755hsKFSrErFmzkst+//13GjZsSOHChfH19SU4OJh9+/Y5MVoREXEWJSAiInLTDhw4wH333cdrr73GbbfdxuOPP87bb79NbGwsAGfPnsXHxyd5fR8fH86ePeukaEVExJmUgIiIyE2Jiori7rvvZsCAAdx7770A9O3bF09Pz+RaEF9fX6KiopKfc+HCBXx9fZ0RroiIOJk6oYuISI6LioqiY8eOrF69msuXL9OpUyfWrVuHh4e6IoqI5Dc684uISI7z8fFh+PDhdOnSBTc3N8aPH6/kQ0Qkn1INiIiIiIiIOIz6gIiIiIiIiMMoAREREREREYdRAiIiIiIiIg6jBERERERERBxGCYiIiIiIiDiMEpBc6MqVK9x///2UL1+ehg0bOjucVFq0aMHPP//sktv+9ddfqVGjBgDffvstAwcOdFRokkLLli1ZsGCB1bIePXqwZs0ah8YjIiIijqVB2HOhDz/8kJiYGPbs2eNy4+ivXbvWZbf9999/c8stt5CYmMjLL7/M559/7qDIJElCQgK7d++matWqVstHjx7NM888w7p16xwcmYiIiDiKakByoVWrVtGtWzeXSz5c3a5du7jllltYsWIFJUuWTK4NcTXr1q2jc+fOzg4jW7Ia8969e0lMTKRixYpWy5s2bcqFCxf49ddf7R2iiIiIuAglILlIQkIC5cqVY/Xq1Tz99NMEBwcTFRXl0BiOHz/O/fffT6VKlQgODqZJkyYcPHgQgP+3d+dxVdX5H8ffF1FRwEuCiqihSKIouO9bamqSmuaGe2o2lqON2aTpWJnmOOaaa07llrtkYSmmhUtp7hFIpZa4K4iASrhx+f1xf96RQEWBcwVfz8fjPh5y7vd8z4fDMN0357ts2LBBzZo1s7VduHChatasKW9vb73//vtq166d1qxZY2vbuHFjvf/++6pcubJ8fHy0Zs0a7dmzRy1btlSZMmXUp08fpaam2vr79NNPVadOHZUrV05t27bVL7/8Ynvvr9eWpMWLF6t27dry9vbW7NmzbQFk06ZNGdpWqlRJERERkqQDBw7Izc1NISEhkqRTp07J29tbSUlJd70vU6dO1dNPP620NOu+nqGhoapSpYpiYmIe9BZnWV6sOSoqSr6+vpo/f778/Pzk6+urhQsX2t43mUxq1qyZNm3alGs1AAAA+yKA5CEFChTQnj175ODgoKNHj+r06dNyc3N76P5GjhypJ5988q6v3bt3Zzhn+PDhqlKlig4fPqyYmBjNmDFDnp6ekqSff/5ZgYGBkqQ5c+Zo8eLFWrVqlY4dO6ZTp05p//79qlatmq3tH3/8ocqVK+vw4cN67bXX9NZbb+mTTz7R2rVrtX//fu3YsUPff/+9JGnWrFn65JNPtGrVKp04cUItW7bUoEGDbHXdeW1JmjdvnhYsWKClS5fq2LFjio6O1r59+1StWjVFRkbqqaeeSvd9PfHEE7p69aok6cMPP5Sfn58t3P33v/9Vz549ZTab73ovX331VV24cEEbNmzQDz/8oDfeeEOrV69W+fLlH/CnknV5sebIyEgdP35cLi4u+vnnn/XRRx9pzJgxOn78uK2Nn5+fIiMjc60GAABgXwSQPCYiIkK+vr5ydna+b9sPPvjgnn9JnjZtmk6ePHnXV8OGDTOc8/vvv+vWrVu6ceOGHB0dVb9+fRUuXFjS/0LA5cuXNWXKFNuH4oIFC6p79+5ycHBQpUqVbG0HDhyoLl26qECBAqpWrZpu3bqlmTNnyt3dXaVLl5aHh4csFosSEhI0depU/fe//5Wvr68cHBzUq1cvRUdHKyUlJd21Jeny5cuaPHmy5s+fr6pVq6pgwYIKDg6Wo6OjKlasqMTERBUrVizd9+Xm5qarV68qJiZGkZGRCg4OVmJiolJSUrRixQoNGTLE1nbJkiVq06aNnnvuOdvTgqJFi2r06NF65513NHDgQH366afpAlFm52RXVmq+efOm2rZtqyeffFJffvlluvPvVfO9zsuOqKgode3aVQMGDFDhwoXVqlUrPfnkk4qOjra1KVasmOFP9gAAgHEIIHlMREREug+29xIVFSV/f/8cvf5HH32kiIgIVa1aVb169dKRI0ds70VGRiowMFA7d+5U6dKlVadOHdt7ly5dkp+fn23eSmRkpNq1a2d7Pzo6Wk8//bSKFi0qSfrzzz914sQJ+fv7a8+ePfLy8kr3vcTHx+uJJ55QkSJF0l1bss5H8PDwUM2aNW3t4+Li5O/vLwcHB7m5ueny5cvpvq/bH+bnzp2rv/3tb3Jzc1NiYqJWr16txo0b254KJCQkaOnSpdq4caMmTJigd99919ZH9erVdfz4cfXq1UtNmjSxHb/XObfd+TQqODhYP/74432fRmWlZkdHRy1ZskSvvPJKpj/Pu9V8v/MetuaoqCi98MIL6Y5dvHhRJUqUsH19+fLlbD3ZAwAAjzYCSB5z51/69+7dq3feeUeS1KVLF82ePVuS1KFDB12/fl1Hjx7VvHnz1KZNG40YMSJDXyNGjFCZMmXu+tq1a1eGc+rWras1a9bo8OHDKly4sP79739Lsn6IPH/+vKpVq6b4+PgMHyC//vpr2/Crixcv6ty5c+mCVEREhKpXr277+vDhw/Lw8FCpUqUy7W/Dhg22JzR3XluyhhMPD4907UNCQmzvBwQE6OjRo+neN5vNOnnypDZu3Kg+ffqoWLFiSkhI0MKFCzV06FBbu/3796tJkyZydHRUrVq1dOzYMUlSTEyMevXqpcGDB2v16tW2JzP3OudOdz6NWrVqlRo0aHDfp1FZqdlkMtmGyP3VvWq+13kPW3NcXJwuXLigkiVL2o59++23Klq0qGrVqmU79ttvvykgIOCe1wYAAHkXASSPufMJiNls1tWrV/XLL7+oaNGiSkpK0vbt29W4cWPdvHlT586d0/Dhw/XNN98oIiJCcXFx6fqaMWOGzpw5c9dXo0aN0rUPDQ3Vb7/9JovFopSUFF28eNEWGn7++Wf5+PjIxcVFVapUUUREhCIiInTt2jXNmzdPoaGh6eZ/VKhQId0wqL8GkDu/rlWrliIjIxUZGanr169r1apVWrx4scaOHZvh2pJ1DsFPP/2kgwcPKiUlRbNnz1ZYWJjt+s8++2yGZV7d3Nz00UcfKTg4WM7OznJ1ddWWLVvk7OysevXq2dolJiamC0MWi0WxsbHq3LmzRo8erQ8++EBeXl5asGDBPc/JCVmtOTP3qzk3REVFydHRUWvWrJHFYlFkZKRGjhyp9957L92Kbjt37lTbtm1ztRYAAGA/BJA8JDExUSdPnrT9ddhsNuvKlStauHChRo4cqcTERC1btkyDBg3S4cOH1bFjR5UpU0aSVKRIERUqVChb19+zZ486d+6ssmXLqkWLFqpfv76GDRsmKf2Tmbp162rIkCHq0KGDateurbS0NHl6eqp27dq2tneGjatXr+r3339Pd+zO/qpUqaKJEyeqZ8+eqlixoj777DOFhITYltH96wT0+vXra8iQIerUqZPq1aunU6dOqWTJkrYA0qZNG8XHx6ebd+Dm5qZLly7p5ZdflmSdh3DhwoV0Tz9ut7tzZank5GR16dJFvXv3Vr9+/SRJ48aN08yZM5WQkJDpOQ4OOfNrl9Wa/+ry5cv3rTk3REVFqX379kpKSlL58uU1cOBAjRkzRt27d7e12bVrl1xcXDKEXwAAkH+YEhMT0+xdBB7OtWvX1LFjR/n4+Gj27Nnq2LGjnnrqKX344Yf65JNPFB0drWnTpunw4cOaMWOGPv74Y7vUeeDAAXXr1k1Hjx5VgQIF7FLDX61bt05ff/21Fi1a9EDnJSQkqHv37goLC1NkZKRmzJihJUuW5Pg5Oe3f//63/P399fzzzxty3sN64YUXNGzYMLVo0cKQ6wEAAOMRQPI4T09PbdiwQXXr1pWnp6e2bNmigIAAjRgxQg4ODrp06ZJSU1M1bdq0dBN9c9P+/ftVvHhxVahQQRERERo8eLAGDx5s+0t9Xvfpp59q1apVcnR01Jw5c+Tj45Mr5+SUF198UYcOHZKzs7OeeeYZvffee7l6HgAAwL0QQJDjlixZovfee0/Xrl1TuXLlNHjwYA0cOFAmk8nepQEAAMDOCCAAAAAADMMkdAAAAACGIYAAAAAAMAwBBAAAAIBhCCAAAAAADEMAAQAAAGAYAggAAAAAwxBAAAAAABiGAAIAAADAMAQQAAAAAIYhgAAAAAAwDAEEAAAAgGEIIAAAAAAMQwABAAAAYBgCCAAAAADDEEAAAAAAGIYAAgAAAMAwBBAAAAAAhiGAAAAAADAMAQQAAACAYQggAAAAAAxDAAEAAABgGAIIAAAAAMMQQAAAAAAYhgACAAAAwDAEEAAAAACGIYAAAAAAMAwBBAAAAIBhCCAAAAAADEMAAQAAAGAYAggAAAAAwxBAAAAAABiGAAIAAADAMAQQAAAAAIYhgAAAAAAwDAEEAAAAgGEIIAAAAAAMQwABAAAAYBgCCAAAAADDEEAAAAAAGIYAAgAAAMAwBBAAAAAAhiGAAAAAADAMAQQAAACAYQggAAAAAAxDAAEAAABgGAIIAAAAAMMQQAAAAAAYhgACAAAAwDAEEAAAAACGIYAAAAAAMAwBBAAAAIBhCCAAAAAADEMAAQAAAGAYAggAAAAAwxBAAAAAABiGAJJDli9fLjc3N9urVKlSqlSpktq3b6/p06crLi7O3iUCAAAAdudo7wLym7lz56pSpUq6efOm4uLi9OOPP2rmzJmaPXu2Fi1apKefftreJQIAAAB2QwDJYf7+/qpZs6bt6+eff16vvvqq2rVrp759++rAgQMqWbKkYfWkpKSoSJEihl0PAAAAuBeGYBmgXLlymjhxoq5cuaJFixbZjh86dEjBwcEqX768SpUqpaZNm2r9+vUZzt+9e7dat26tUqVKqUqVKpo4caKWLl0qNzc3nThxwtYuICBAPXr0UGhoqJo2bapSpUrpP//5jyTpwoUL+sc//iF/f3+VKFFCgYGBmjx5sm7dupXuWjdu3NAHH3ygunXrqmTJkqpYsaJeffVVXbx4MZfuDgAAAB4nPAExSOvWrVWgQAHt2rVLkrRjxw517dpVtWvX1owZM1SsWDGFhIRowIAB+vPPP9W7d29JUlRUlDp37qyKFStq/vz5KlKkiBYtWqQ1a9Zkep2IiAj99ttveuONN+Tt7S1nZ2dduHBBrVq1kslk0ptvvqkKFSpo7969mjp1qk6ePKl58+ZJkiwWi3r16qXdu3dr+PDhql+/vk6ePKnJkyerffv2Cg8P52kKAAAAsoUAYhBnZ2e5u7vr/PnzkqQ33nhDlStX1oYNG+ToaP0xtGrVSvHx8ZowYYJ69uwpBwcHTZ06VQUKFFBoaKjc3d0lSW3btlWjRo0yvU5cXJz27NkjX19f27ERI0YoMTFRu3fvVrly5SRJzZs3l5OTk8aNG6fhw4ercuXKWr9+vbZu3aqlS5eqY8eOtvMDAgLUokULrVixQoMGDcqV+wMAAIDHA0OwDJSWliZJ+uOPP3TkyBF169ZNknTr1i3bq02bNjp//ryOHj0qSfrhhx/UtGlTW/iQJAcHB3Xq1CnTa1StWjVd+JCkzZs3q0mTJipdunS6a7Vu3dp2jdvtzGaz2rVrl65dQECASpUqpe+//z5H7wcAAAAePzwBMUhycrIuXbokf39/xcbGSpLGjRuncePGZdo+Pj5eknTp0qVMJ63fbSK7p6dnhmOxsbEKCwuTh4fHPa8VGxurpKQklShR4p7tAAAAgIdFADHIN998o9TUVDVp0sT2NOP1119Xhw4dMm1/+ylG8eLFbYHlThcuXMj0PJPJlOGYu7u7qlatetewczu0uLu7q3jx4goJCcm0nYuLS6bHAQAAgKwigBjg1KlTGjdunIoVK6YBAwbIw8NDFStWVFRUlN5+++17ntu4cWNt2bJF8fHxtuBisVj05ZdfZvn6bdu21ZYtW1ShQgW5ubnds11ISIhSU1NVp06dLPcPAAAAZBUBJIdFR0fb5k7ExcVp9+7dWr58uQoUKKDPPvvMNgxqxowZ6tatm1544QX16tVLpUuXVkJCgo4cOaKIiAgtWbJEkjRy5EiFhYWpY8eOeuONN+Tk5KRFixYpOTlZknU+yP2MGTNG4eHhatOmjf72t7/J19dX169f18mTJ7VlyxZNnz5dZcqUUZcuXbR27Vp169ZNQ4YMUe3atVWwYEGdOXNGO3fuVFBQ0F2f2AAAAABZYUpMTEyzdxH5wfLlyzV06FDb14UKFZLZbFalSpXUqlUr9evXL8McjKioKE2bNk3ff/+9EhMTVbx4cfn5+alz584aMGCArd3u3bs1btw4RUZGys3NTT169JC7u7veeecdnThxQmazWZJ1tSp/f3+tXr06Q33x8fGaMmWKwsLCdPbsWbm4uMjb21vPPPOMRowYIWdnZ0nWCfHz58/X6tWrdezYMTk6OsrLy0uNGzfWsGHD5OPjkxu3DwAAAI8JAkge1blzZ508eVIHDhywdykAAABAljEEKw8YM2aMAgMDVaZMGSUkJGjt2rUKDw/X7Nmz7V0aAAAA8EAIIHlAamqqJk2apNjYWJlMJvn5+emjjz5Sjx497F0aAAAA8EAYggUAAADAMOyEDgAAAMAwBBAAAAAAhmEOCJAD6m5w1qUb5HkjFC9k0b4OyfYuAwAAPCQCCJADLt1wUPx1AggAAMD9EEByiLOzc5Z2Jcejz2Kx2Haax+OB39/8g99fAHj0EUByiIODAx9ggDyK318AAIzDf3EBAAAAGIYAAgAAAMAwBBAAAAAAhiGAAAAAADAMAQQAAACAYQggAAAAAAxDAAEAAABgGAIIAAAAAMMQQHJBamqqxo8fr0uXLmX6/ueff67vvvsu29cZPny4ypcvL5PJpKioqGz3BwAAAOQ2dkLPBcuXL9e7776rGTNmqEGDBipUqJDtvbNnz+rgwYMqVKiQOnfurJUrVz70dbp27ao333xTTZo0yYmyAQAAgFzHE5Bc0LdvX3l7eyspKUlxcXFavHixQkND1b9/f0VERMjV1VXbtm1TcnJytq7TrFkzlS1bNoeqBgAAAHIfASQXmEwmBQQE6JVXXtHBgwfVunVrLV68WMHBwSpatKg2b96sBg0a2LtMAAAAwHAMwcolJpNJc+fOlSTNnz9fAwYMkKurK+EDAAAAjzWegOQik8mkli1bysHBepu9vLxUqVIlO1cFAAAA2A8BJBeFhISoZ8+ecnFxUefOnfXbb7+pdevWd10dCwAAAMjvCCC55OzZs+nmfISEhKSbE5ITIWTo0KEqW7asTp8+rWeeeUa+vr45UDkAAACQewgguWDTpk06ePBgugnnt+eE3BlCUlNTs3WduXPn6vTp07p165bOnz+vY8eO5dB3ALvZv1qaUFV6rYj0qkk69ZP01bvWf99p+zxp9+LsX+9Vk7X/B5V41nreqZ+yXwMAAHisMAk9F9SsWVPu7u6qXLmyJk2alO69tLQ0eXt76+rVqypevLidKsQj6UqctKSv5P+s1GOeVLCwVKqS1Pglqeqz6dvumCe5eEgNX7RLqUo6K20cL7mXl8rVsE8NAAAgTyKA5AJPT0/FxsbKZDJl+n5aWtpd38NjLPaIlHpTqtdHqtT8f8cLFZWeYL8XAACQPxBAcsm9AgbhAxksfVH6cYn135/0sL6eai6N2GYd6rRxvDQvzfr+v8pLl05Y/317aFZxb2lizN37T7ksfT5S+ulz6dZ1qWITqduHGdvFHpPC3pd+/15KPCMVfUIqV0t6fpJUJsDa5sg2aWYL67+XDbC+JCnoHan9u9KJ/dLWqdLxH6UrFyTXUpJPQ+n5yZK790PeIGP17dtXSUlJOdaf2WzWsmXLcqw/AADyMgII8ChoN07yrietHip1nCT5tZCcimXe9m/rpf92lYqYpeB51mOOhe/ed1qa9FEn6Y9dUtDbkndd6fcfpLntMrZNOis5u0udJksuJaTkS9KeJdKU+tKYQ1IpP2sg6bvIGjza/Uuq9pz1XLf/f0oTH2NtVztYci4uJZ2Tds6X/lNXejvaOnTsEZeUlKTQ0NAc669jx4451hcAAHkdAQR4FJSoKJX2t/675FNShXtsVlmuplSwiDWg3KvdbdGbpSPhUrdZUovh1mNVWkuOhaTQsenbPtXM+rrNkioFPGedGL/zI6nrdKlIMcmrmvV9j4oZa6jV1fpK10d7aXQpad+K/9XwmCtfvrycnJzk5OSklJQUDRgwQKNHj7Z3WRmcPXtWvXv3Vnh4uL1LAQDkEwQQIL878v8fHOv2Tn+8bq+MAST1lrRlirT3MynumHVOym3nf8na9a5dlTZNkH4KsT4Nsdyx2ltW+3hMrFu3TtWqVdPZs2fl7++vli1bql69etnu99atW3J0zJn/e/fy8iJ8AAByFMvwAvldcrzk4Ci5uKc/XswzY9uQ16UN46TqnaRXNkhv7pFG7ZPKVpdupmTteot6SdvnSI1ekv6+WXpzr7UPlxJZ7+Mx4+XlJT8/P504cULnz59X9+7dVa9ePQUGBurtt9+2tdu5c6cCAgIUGBioYcOGydvbW1FRUZKsT1Tef/99tWjRQv3799fNmzc1evRo1atXTzVq1FBwcLASExMlSR9//LH8/f1Vo0YNBQQEaM+ePbJYLPr73/+uypUrq3r16qpdu7auXbummJgYeXj8b9hcWFiYatWqpcDAQDVv3lzR0dGSpG3btqlGjRp69dVXVb16dVWtWlX79+837iYCAPIMnoAA+Z2zu2S5JV2NTx9CLp/P2HbvZ1L9ftZJ53e6elEq4nb/a6UkSVFfWSekt71jONHN69Kf2d98M7/69ddfdfHiRT399NPq06ePxo4dq2bNmunWrVtq37691q9fr6CgIPXs2VMrV65U06ZNtX79es2ZMyddPydPntR3330nk8mkSZMmycXFRXv37pUkTZgwQe+8845mzZqlkSNH6pdffpGXl5du3ryp69evKyIiQt9++62io6Pl4OCgpKQkFSpUKF3/sbGx6tOnj8LDwxUQEKDly5ere/futhB0+PBhffzxx5o3b54WLFigsWPHavPmzcbcRABAnkEAycOuXbum4OBgRUdHq2jRovL09NSCBQtUvnx5e5eG3OZYOOtPEyq1sA6r2rc8/fyLfSsytjWZMk5oj/zauiJWCd/015cyqcFknfT+1z52fZx+KBYkSV27dpXJZNJvv/2mGTNmqGjRovruu+904cIFW5urV6/q119/VcWKFVWkSBE1bdpUktS5c2e5ubml62/AgAG2Vfa++OILXb58WevWrZMk3bhxQxUrVpQktWzZUv369VOHDh3Url07VapUST4+Prp586YGDhyoFi1a6LnnnpODQ/qH5Hv27LE9NZGk3r17a+jQoTp37pwkyc/PT3Xq1JEkNWzYUFOnTs3hOwYAyA8IILkkK8t45sTSnC+//LLatWsnk8mkOXPm6OWXX9Y333yTrT6RB5QJkPavsu6c7uEjFXT63zK5f1WljeTbTFr/pnQ9WfKuY10Fa28m/9ur1l76cbHkWVkqEyidPCBt+eB/K1zdVqKidSL8vuWSZxWpsItk9pLcvKzX2vqBdbUr9/LS0e3Srk+y9gTlMXN7DsjWrVvVoUMHtWzZUiaTSfv27VPBggXTtY2IiLjvEt4uLi62f6elpWnevHlq2bJlhnaff/65Dhw4oG3btikoKEgTJ05UcHCwDh8+rO3btys8PFxvvfWWduzYkW4uyd32MLp9zMnJyXasQIECunXrVtZuBADgsUIAySVZWcYzu0tzOjk5KSgoyPZ1gwYNNHPmzGz1iTziufHW5W1XDJauXbn3PiAODtIrodK6161PQlJvSD6NpaEbpfGV07ftNksqUFDa/G/p+lXrkrsvfy5t+Ff6doWKSn0/lb4eL81uY52sfnsfkIErpLWvWQOP5Zb1WsO2SPOey407kS8888wzeuWVV/Svf/1LTZs21eTJkzVu3DhJ1lWoLBaLKleurOTkZP3www9q3LixvvzyS9ucjsx07NhR06dPV4MGDVS0aFH9+eefOn78uPz8/BQTE6M6deqoTp06unjxovbu3atWrVqpQIECatOmjVq3bq3t27crOjpagYGBtj4bNmyoQYMG6ZdfflGVKlW0atUqlS1bVp6envr1119z+zYBAPIJAkg+8uGHH6pDhw72LgMPq9LT/9ts8E7t37W+7uTuLQ17gLH1RcxS30+srzv99XpF3aQ+H2c8f8S2jMfqBFtff+VWRhq8LuPxe22UCI0bN06+vr4KDQ3V3LlzbcOcXFxctGDBApUtW1YrVqzQkCFDVKRIEbVo0UKlSpWS2WzOtL/Ro0dr/Pjxql+/vu0JxahRo+Tr66sBAwYoISFBjo6OKlGihBYtWqRTp05p8ODBunnzpiwWixo1aqR27drpzJkztj5LlCihZcuWqXfv3kpNTZWbm5vWrFmT+zcHAJCvmBITEzP5xIMH5erqmm68dMeOHbP0BCSnNjubNGmSNmzYoG+//VZFixbNkT4fVxaLRVeuXHmgcyqGuCr+OovKGcG9sEW/d3mwn8/9PMzv74PIqf6uXLkiV1dXSVJ4eLj69++vmJiYDHM1HmcP8/sLADAWT0DygalTp+rzzz/X1q1bCR9APhYSEqIZM2bIYrGocOHCWrlyJeEDAJDnEEDyuOnTp2vlypXaunVrhhVxAOQvL774ol588UV7lwEAQLYQQPKw06dPa+TIkfLx8VGLFi0kSYULF9aePXvsXBkAAACQOQJIHla2bFmlpTGFBwAAAHkHAQQA/sJsNmd7mey/9gcAAKwIILkkKx9g+FACPJqyu0EoAAC4OwJILuEDDAAAAJAR6zcCAAAAMAwBBAAAAIBhCCAAAAAADEMAAQAAAGAYAggAAAAAw7AKVh7Xpk0bnT9/Xg4ODnJ1ddXs2bNVo0YNe5cFAAAAZIoAkkv69u2rpKSke7Yxm83ZXq53zZo1cnNzkyR98cUXGjhwoA4ePJitPgEAAIDcQgDJJUlJSQoNDb1nm5zYafl2+Lh9TQcHRtUBAADg0UUAyQf69eun8PBwSVJYWJidqwEAAADujj+X5wNLly7VqVOnNHHiRP3zn/+0dzkAAADAXRFA8pH+/fsrPDxc8fHx9i4FyNOSk5Mfyb4AAMgPCCB52OXLl3X27Fnb1+vXr5e7u7uKFy9ux6qAvO3cuXPy8/PTlClTZLFYHrofi8WiKVOmyM/PT+fPn8/BCgEAyNuYA5KHJSUlqUuXLkpJSZGDg4NKlCihr776SiaTyd6lAXnW+fPnlZCQoFGjRmnatGmqUaOGChcu/EB9XL9+XT/99JNiY2Pl5OSkQYMG6euvv86ligEAyFsIIHlYuXLltHfvXnuXAeQrNWvWVJMmTeTo6KiNGzcqOjpaq1atUuPGjbN0/g8//KAePXooNjZWQUFBWrp0qQYMGJDLVQMAkHcQQIAcULzQww/VwYMx4l4XLlxYX3zxhaZNm6a33npLzZs316RJk/TGG2/cdalri8WiqVOnasyYMZKkKVOmaOTIkfdcGvvWrVuaNGmSVqxYoQIFCig1NVXNmjXTlClT0i2xfadt27bpxo0batOmTba/zwcRFBSk2bNnq2LFig91fvny5fXVV1+pWrVqOVwZACCvIYDkErPZfN99Psxms0HVILft68BE4/zGwcFB//znP9WoUSMFBwdr1KhR2r59u5YsWSIPD490bS9evKh+/fpp06ZNKlu2rFavXq1GjRrd9xqDBg3SpUuXtHv3bj3xxBOyWCwKCQnRpUuX7hlArl69algAuT0PZuPGjYZcDwCQ/xFAckl2dzgH8Gho3LixDh06pP79+2vjxo2qWbNmuiFZt4dcnTlzxjbkyt3d/b79Hjt2TGvXrtXJkyf1xBNPSLKGnm7duun8+fNq0aKFLl++rGvXrqlVq1aaNWuWIiIitGDBAlksFm3dulUvvPCC3n77bW3evFkTJkxQSkqKHB0d9cEHH6hZs2aSpLFjx2r16tVyd3dXs2bNFB4erv3790uyPqVZsmSJHBwcFBgYqHnz5slsNuvdd9/V77//ruTkZB07dkybNm1S48aNbU8wzpw5o9dee01HjhyRJD3//POaMGGCVqxYoVmzZunGjRtKS0vTpEmTFBQUlBs/FgBAHkYAAYD78PDw0IYNG9INyZo4caJMJpPGjh0rKWtDru508OBBPfXUUxmepkiSm5ubNmzYIBcXF6Wmpur5559XSEiIunbtqiFDhujq1auaOnWqJOmPP/7Q+PHjFRYWpmLFiunYsWNq3ry5YmJiFBYWpq+++koREREqUqSIunbtarvGpk2btGjRIu3evVtubm56+eWXNWbMGM2dO1eSFB4eroMHD6pkyZIZ6uvTp4+CgoK0bt06SVJcXJwkqW3bturZs6dMJpNiYmLUqFEjnThxQgULFnyAuw0AyO8IIACQBXcOyerWrZveeustSZKXl5fWrl2bpSFXWWWxWDRq1Ch9//33SktLU2xsrGrUqJEuQNwWFhamY8eO2Z543Hbq1CmFh4ere/fucnZ2lmTdK2jChAmSpK1bt6p37962oV6vvPKKgoODbee3b98+0/Bx9epV7dq1S1u2bLEdK1GihCTp+PHj6t27t06fPi1HR0ddvHhRJ06ckK+vb/ZuCAAgXyGAAEA2pKWlKS0t7YHPq1Wrlo4ePar4+PgMQ7amT5+u+Ph47dmzR05OTnr99dd17dq1u17/2Wef1dKlSzN9727Lcmf23p1fu7i4POi3pODgYE2dOlWdOnWSJBUvXvyudQMAHl9sRAgAWXB7Y8HmzZsrNjZW//nPfzR58mTFxsaqefPmD7xxoa+vr7p06aJBgwYpMTFRkjUULF26VAcOHJCnp6ecnJx04cIFrV271nZesWLFlJSUZPu6TZs2CgsLU1RUlO3Y7eW5W7RoobVr1+rPP/+UxWJJNzetdevWWrVqla5cuSJJWrhwoZ555pn71u3i4qImTZpoxowZtmO3h2AlJCSofPnykqTPPvtMCQkJWb4fAIDHB09AAOA+7rXKVZMmTe67StbdfPrpp5o4caLq168vR0dHpaWlqVmzZpo5c6a6deumGjVqqEyZMumCQefOnbVs2TLVqFHDNgn9s88+00svvaSUlBTduHFDtWrV0vLly9WxY0ft2rVL1atXl5eXlxo0aGALBe3atVNkZKQaNmwok8lkm4SeFcuWLdOwYcNUtWpVOTo6qlOnTho/frxmzZqlzp07q0yZMmrYsKGefPLJB7zTAIDHgSkxMfHBxw4gA1dX1yxPPsWjzWKx2P4qjMfDX39/O3bsqNDQUElZW+Xq4sWLtlWyypYtm2Hjwjv7M9qVK1fk6uoqi8Wil156SV5eXpo4caJdajECv78A8OjjCUgu6du3b7phEpkxm805slzv+PHj9e677yoyMpJNvoAc8iAbC2a2Stb9Ni40Sr9+/RQTE6OUlBTVqlVLb775pl3rAQCAAJJLkpKS7vsXz/ttVJgVBw8e1I8//shQByAHXb9+Xe3bt3+gjQXvtXGhPa1fv96u1wcA4K8YM5SHXb9+XUOHDtW8efPuutINgAdz6NAh7dixQ5s2bVJQUJB++umnB1pi9/bGhUFBQbaNC+/3NBQAgMcJASQPe/vtt9WnTx9VqFDB3qUA+Ubp0qUlWYdcbdiwIUu7mv/V7SFZU6ZMUVpamgoXLpzTZQIAkGcxBCuP2r17t/bt26fJkyfbuxQgX/H09FSnTp20c+dO7dy5M9v9BQYGPlSIAQAgvyKA5FHbt2/Xr7/+anv6cfr0abVt21Yff/yx2rVrZ+fqgLxt5cqV9i4BAIB8iyFYedTo0aN19uxZxcTEKCYmRmXLltXmzZsJHwAAAHikEUAAAAAAGIYhWPlETEyMvUsAAAAA7osAkkvMZvN99/kwm80GVQMAAAA8GkyJiYlp9i4iP3B1dbX7jsfIGRaLRVeuXLF3GTAQv7/5B7+/APDo47+4AAAAAAxDAAEAAABgGAIIAAAAAMMQQAAAAAAYhgACAAAAwDAEEAAAAACGYR+QXNK3b18lJSXds43ZbNayZcuydZ3y5cvLyclJTk5OkqS33npLPXr0yFafAAAAQG4hgOSSpKQkhYaG3rPN/TYqzKp169apWrVqOdIXAAAAkJsYggUAAADAMASQfKB3794KCAjQSy+9pLi4OHuXAwAAANwVASSP27FjhyIiInTw4EG5u7urf//+9i4JAAAAuCvmgORxTz75pCSpYMGC+sc//qFKlSrZuSIAAADg7gggeVhycrJu3rwpNzc3SdLKlStVs2ZN+xYF5EEWi8XeJSCH8LMEgEcfASQPu3Dhgrp06aLU1FSlpaXJx8dHS5cutXdZQJ6TnJxs7xIAAHhsEEDyMB8fHx06dMjeZQAAAABZRgDJJWaz+b77fJjNZoOqAQAAAB4NpsTExDR7F5EfuLq6ysGBRcXyA4vFoitXrti7DAAAgHyJT8wAAAAADEMAAQAAAGAYAggAAAAAwxBAAAAAABiGAAIAAADAMAQQAAAAAIYhgORx169f19///nc99dRTqlq1qvr06WPvkgAAAIC7YiPCXNK3b18lJSXds43ZbNayZcuydZ3Ro0fLwcFBR44ckclk0rlz57LVHwAAAJCbCCC5JCkpSaGhofdsc7+d0u8nOTlZixYt0unTp2UymSRJpUuXzlafAAAAQG5iCFYe9vvvv8vd3V0TJ05UnTp11LRpU3377bf2LgsAAAC4KwJIHnbz5k398ccf8vf31/79+zVnzhwFBwcrLi7O3qUBAAAAmSKA5GHe3t5ycHBQ7969JUnVq1dXhQoVdPjwYTtXBgAAAGSOAJKHeXh4qFWrVtq8ebMk6cSJEzp+/Lj8/PzsXBkAAACQOSah53ELFizQwIEDNWrUKBUoUEALFy5kIjoAAAAeWQSQPM7Hx0fbtm2zdxkAAABAlhBAconZbL7vMrtms9mgagAAAIBHgykxMTHN3kXkB66urnJwYEpNfmCxWHTlyhV7lwEAAJAv8YkZAAAAgGEIIAAAAAAMQwABAAAAYBgCCAAAAADDEEAAAAAAGIYAAgAAAMAwBBAAAAAAhiGAAAAAADAMO6HnEIvFYu8SkEP4WQIAAOQeAkgOSU5OtncJAAAAwCOPIVgAAAAADEMAAQAAAGAYAggAAAAAwxBAAAAAABiGAAIAAADAMAQQAAAAAIYhgAAAAAAwDAEEAAAAgGEIIAAAAAAMQwABAAAAYBgCCAAAAADDEEAAAAAAGIYAAgAAAMAw/wclqUacnKfDxAAAAABJRU5ErkJggg==",
"text/html": [
"\n",
" ![](\"week3/OptionalLabs/images/C1_W3_LinearGradientRegularized.png\")
\n",
"![](\"week3/OptionalLabs/images/C1_W3_LogisticGradientRegularized.png\")
\n",
"\n",
"The slides above show the cost and gradient functions for both linear and logistic regression. Note:\n",
"- Cost\n",
" - The cost functions differ significantly between linear and logistic regression, but adding regularization to the equations is the same.\n",
"- Gradient\n",
" - The gradient functions for linear and logistic regression are very similar. They differ only in the implementation of $f_{wb}$."
]
},
{
"cell_type": "markdown",
"id": "49ea942b",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"##### Cost functions with regularization\n",
"###### Cost function for regularized linear regression\n",
"\n",
"The equation for the cost function regularized linear regression is:\n",
"$$J(\\mathbf{w},b) = \\frac{1}{2m} \\sum\\limits_{i = 0}^{m-1} (f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) - y^{(i)})^2 + \\frac{\\lambda}{2m} \\sum_{j=0}^{n-1} w_j^2 \\tag{1}$$ \n",
"where:\n",
"$$ f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) = \\mathbf{w} \\cdot \\mathbf{x}^{(i)} + b \\tag{2} $$ \n",
"\n",
"\n",
"Compare this to the cost function without regularization (which you implemented in a previous lab), which is of the form:\n",
"\n",
"$$J(\\mathbf{w},b) = \\frac{1}{2m} \\sum\\limits_{i = 0}^{m-1} (f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) - y^{(i)})^2 $$ \n",
"\n",
"The difference is the regularization term, \n",
" $\\frac{\\lambda}{2m} \\sum_{j=0}^{n-1} w_j^2$ \n",
" \n",
"Including this term encourages gradient descent to minimize the size of the parameters. Note, in this example, the parameter $b$ is not regularized. This is standard practice.\n",
"\n",
"Below is an implementation of equations (1) and (2). Note that this uses a *standard pattern for this course*, a `for loop` over all `m` examples."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c6e3a2d3",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"def compute_cost_linear_reg(X, y, w, b, lambda_ = 1):\n",
" \"\"\"\n",
" Computes the cost over all examples\n",
" Args:\n",
" X (ndarray (m,n): Data, m examples with n features\n",
" y (ndarray (m,)): target values\n",
" w (ndarray (n,)): model parameters \n",
" b (scalar) : model parameter\n",
" lambda_ (scalar): Controls amount of regularization\n",
" Returns:\n",
" total_cost (scalar): cost \n",
" \"\"\"\n",
"\n",
" m = X.shape[0]\n",
" n = len(w)\n",
" cost = 0.\n",
" for i in range(m):\n",
" f_wb_i = np.dot(X[i], w) + b #(n,)(n,)=scalar, see np.dot\n",
" cost = cost + (f_wb_i - y[i])**2 #scalar \n",
" cost = cost / (2 * m) #scalar \n",
" \n",
" reg_cost = 0\n",
" for j in range(n):\n",
" reg_cost += (w[j]**2) #scalar\n",
" reg_cost = (lambda_/(2*m)) * reg_cost #scalar\n",
" \n",
" total_cost = cost + reg_cost #scalar\n",
" return total_cost #scalar"
]
},
{
"cell_type": "markdown",
"id": "f2fd183e",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"Run the cell below to see it in action."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5281d9a1",
"metadata": {
"pycharm": {
"name": "#%%\n"
},
"tags": []
},
"outputs": [],
"source": [
"np.random.seed(1)\n",
"X_tmp = np.random.rand(5,6)\n",
"y_tmp = np.array([0,1,0,1,0])\n",
"w_tmp = np.random.rand(X_tmp.shape[1]).reshape(-1,)-0.5\n",
"b_tmp = 0.5\n",
"lambda_tmp = 0.7\n",
"cost_tmp = compute_cost_linear_reg(X_tmp, y_tmp, w_tmp, b_tmp, lambda_tmp)\n",
"\n",
"print(\"Regularized cost:\", cost_tmp)"
]
},
{
"cell_type": "markdown",
"id": "6a21f014",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"**Expected Output**:\n",
"
\n",
" ![missing](\"week3/practice_quiz/gradient_decent_for_logistic.png\")
![missing](\"week3/practice_quiz/costfnlogistic.png\")
![](\"week3/C1W3A1/images/figure)
\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 20,
"id": "f49c2b82",
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAjsAAAGwCAYAAABPSaTdAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAA9hAAAPYQGoP6dpAABbUElEQVR4nO3de1xUdf4/8NcB4QSCgIIwKAmKV7xj66ULWmllpeW2llZqtiymW5CVZlYMZXjpl6vdkUwt16xvarvtbt5KSHMpBa9oKIqKOggpF1MaFD6/P9w5MVxkBuZy5szr+XjMI+ecw8znnAbOez6f9+f9kYQQAkREREQa5eHsBhARERHZE4MdIiIi0jQGO0RERKRpDHaIiIhI0xjsEBERkaYx2CEiIiJNY7BDREREmtbK2Q1Qg5qaGpw9exb+/v6QJMnZzSEiIiILCCFw8eJFhIeHw8Oj8f4bBjsAzp49i4iICGc3g4iIiJqhsLAQHTt2bHQ/gx0A/v7+AK5drDZt2ji5NURERGSJiooKREREKPfxxjDYAZShqzZt2jDYISIicjFNpaAwQZmIiIg0jcEOERERaRqDHSIiItI05uwQEZHDVVdX48qVK85uBqmcl5cXPD09W/w6DHaIiMhhhBAoKipCWVmZs5tCLiIwMBBhYWEtqoPn1GDn+++/x5tvvons7GwYDAZs2LABDzzwgLJfCIGUlBQsW7YMpaWlGDx4MN577z3ExMQoxxiNRjz//PP47LPPUFlZiTvuuAPvv//+defbExGRc5gCnfbt28PX15eFXKlRQghcvnwZxcXFAACdTtfs13JqsHPp0iX069cPTzzxBP74xz/W279o0SIsXrwYK1euRLdu3TBv3jyMHDkSeXl5ypz6pKQkfP3111i7di3atWuH5557Dvfddx+ys7Nt0vVFRES2UV1drQQ67dq1c3ZzyAX4+PgAAIqLi9G+fftm39edGuzcc889uOeeexrcJ4TAkiVLMHfuXIwbNw4AsGrVKoSGhmLNmjVISEhAeXk5li9fjk8//RR33nknAGD16tWIiIjA1q1bcddddzX42kajEUajUXleUVFh4zMjIqK6TDk6vr6+Tm4JuRLT5+XKlSvNDnZUOxuroKAARUVFGDVqlLJNlmXExcVh586dAIDs7GxcuXLF7Jjw8HD07t1bOaYh8+fPR0BAgPLgUhFERI7DoSuyhi0+L6oNdoqKigAAoaGhZttDQ0OVfUVFRfD29kZQUFCjxzRkzpw5KC8vVx6FhYU2br1jCFGN0tIMnDv3GUpLMyBEtbObREREpDqqn41VN6ITQjQZ5TV1jCzLkGXZJu1zlpKS9cjPT4TReFrZJssdER29FCEh45zYMiIiInVRbc9OWFgYANTroSkuLlZ6e8LCwlBVVYXS0tJGj9GikpL1yM19yCzQAQCj8Qxycx9CScl6J7WMiIhM9Ho9+vfv36LXOHHiBCRJwt69e23SJksMHz4cSUlJDns/R1BtsBMVFYWwsDBs2bJF2VZVVYXMzEwMGzYMABAbGwsvLy+zYwwGAw4ePKgcozVCVCM/PxGAaGgvACA/P4lDWkSkaQaDAXq9HgaDwaHvu3PnTnh6euLuu+92yPtFRETAYDCgd+/eAICMjAxIklSvTpEWAxRbcmqw8+uvv2Lv3r1KxFpQUIC9e/fi1KlTkCQJSUlJSE1NxYYNG3Dw4EFMmTIFvr6+mDhxIgAgICAATz75JJ577jl8++232LNnDx577DH06dNHmZ2lNWVl2+v16JgTMBoLUVa23WFtIiJyNIPBgJSUFIcHOx9//DGefvpp7NixA6dOnbL7+3l6eiIsLAytWqk+60TVnBrs7N69GwMGDMCAAQMAADNnzsSAAQPw6quvAgBmzZqFpKQkTJ8+HYMGDcKZM2ewefNmpcYOAPztb3/DAw88gPHjx+Pmm2+Gr68vvv76a83W2KmqsuwX29LjiIjIMpcuXcIXX3yBp556Cvfddx9Wrlxptn/BggUIDQ2Fv78/nnzySfz2229m+6dMmYIHHngAqampCA0NRWBgIFJSUnD16lW88MILaNu2LTp27IiPP/5Y+Znaw1gnTpzAiBEjAABBQUGQJAlTpkzBlClTkJmZiaVLl0KSJEiShBMnTgAADh06hNGjR8PPzw+hoaF4/PHH8csvv5id06RJk+Dn5wedToe33nrLPhfPyZwa7AwfPhxCiHoP0wdIkiSlm/K3335DZmam0pVncsMNN+Cdd97B+fPncfnyZXz99deankru7W1ZBUlLjyMichUGgwE5OTnKA4DZc3v38nz++efo3r07unfvjsceewwrVqyAENfSB7744gskJyfjjTfewO7du6HT6fD+++/Xe43vvvsOZ8+exffff4/FixdDr9fjvvvuQ1BQEH788UdMmzYN06ZNa3CWcEREBNatWwcAyMvLg8FgwNKlS7F06VIMHToU8fHxMBgMMBgMyvBXXFwc+vfvj927d2Pjxo04d+4cxo8fr7zmCy+8gG3btmHDhg3YvHkzMjIykJ2dbacr6ESCRHl5uQAgysvLnd2UJtXUXBU7d3YU27ZJYts2NPCQxM6dEaKm5qqzm1rP2bNnRXJysjh79qyzm0JkN/ycN66yslIcOnRIVFZWNuvnk5OTBa4lJzb4SE5Otm2D6xg2bJhYsmSJEEKIK1euiODgYLFlyxYhhBBDhw4V06ZNMzt+8ODBol+/fsrzyZMni06dOonq6mplW/fu3cWtt96qPL969apo3bq1+Oyzz4QQQhQUFAgAYs+ePUIIIbZt2yYAiNLSUrP3iouLE4mJiWbbXnnlFTFq1CizbYWFhQKAyMvLExcvXhTe3t5i7dq1yv7z588LHx+feq/lTNf73Fh6/1ZtgjI1TJI8ER291PSs7l4AQHT0EkiS+obxnDXGTuRI/JzbT0JCArKzs5GdnY309HQAQHp6urItISHBbu+dl5eHn376CY888ggAoFWrVnj44YeVIafDhw9j6NChZj9T9zkAxMTEwMPj91tvaGgo+vTpozz39PREu3btlPWgWiI7Oxvbtm2Dn5+f8ujRowcA4NixYzh27BiqqqrM2tm2bVt07969xe+tNsx4ckEhIeMQE/NlI3V2lrDODhFpkk6nq7cY5MCBAzFw4EC7v/fy5ctx9epVdOjQQdkmhICXl1e98ifX4+XlZfZckqQGt9XU1LSswQBqampw//33Y+HChfX26XQ6HD16tMXv4SoY7LiokJBxCA4ei7Ky7aiqMsDbW4fAwFtV16NjGj8GYDbGbtLQHy8iV8PPubZdvXoVn3zyCd566y2z5YkA4I9//CP+/ve/o2fPnsjKysKkSZOUfVlZWTZvi7e3N4Bri6rW3V5328CBA7Fu3TpERkY2OJsrOjoaXl5eyMrKwo033ggAKC0txZEjRxAXF2fztjsTgx0XJkmeCAoa7uxmXFdaWhpSUlLMtsXHxyv/Tk5Ohl6vd3CrSO0MBgPS0tKQkJDgEkECP+eOp9PpkJyc7JDPx7/+9S+UlpbiySefREBAgNm+hx56CMuXL8eLL76IyZMnY9CgQbjlllvw97//Hbm5uejcubNN29KpUydIkoR//etfGD16NHx8fODn54fIyEj8+OOPOHHiBPz8/NC2bVvMmDED6enpmDBhAl544QUEBwcjPz8fa9euRXp6Ovz8/PDkk0/ihRdeQLt27RAaGoq5c+eaDbNphfbOiFTFmWPs5LpcLe/FnT/nzirup9PpoNfrHRLsLF++HHfeeWe9QAe41rOzd+9edO3aFa+++ipmz56N2NhYnDx5Ek899ZTN29KhQwekpKTgxRdfRGhoKP76178CAJ5//nl4enqiV69eCAkJwalTpxAeHo4ffvgB1dXVuOuuu9C7d28kJiYiICBACWjefPNN3HbbbRgzZgzuvPNO3HLLLYiNjbV5u51NEkI0VIrXrVRUVCAgIADl5eVo06aNs5ujWTk5OYiNjUV2drZDxthJ/RrrwXHlz4ort705rDnf3377DQUFBYiKisINN9zgoBaSq7ve58bS+zeHsYjIaUw9OGPGjFGeA8x7qc3VhvSI1IjBDjmMI8fYyfVoJe/F1p/z2gGhWn53mJBNrobBjooJUa362VbWMI2xk3tr7EY5dOhQrF69GsHBwSgsLER8fDzS09OVoRFXuXm6w+dcK4EpuQ8GOypVUrK+kTo6S1lHh1yaJTdK07CWo2qoqI3ae04SEhKU/0c5OTkuG5iS+2Cwo0IlJeuRm/sQrlVA/53ReAa5uQ8hJuZLBjzksiy5UbrKLCx7UXvPiTOL+xE1B4MdlRGiGvn5iagb6PxvLwAJ+flJCA4e69JDWuS+LL1RunN+F3tOiGyLwY7KlJVtNxu6qk/AaCxEWdl21RcUJGoud8h7uR5X6jnhxANyBQx2VKaqyrLue0uPI1Iz3ihdn7sHpuQaWEFZZby9Lfujb+lxRGrmyCq4rooBIVlq5cqVCAwMbPHrSJKEr776qsWvY6kpU6bggQcesOt7MNhRmcDAWyHLHQFIjRwhQZYjEBh4qyObRUROwoCwPiGqUVqagXPnPkNpaQaEqG76h1pgypQpkCQJCxYsMNv+1VdfQZIa+1vdsMjISCxZssSGrbM9g8GAe+65BwBw4sQJSJKEvXv3mh3jiADFlhjsqIwkeSI6eqnpWd29AIDo6CVMTiYit1RSsh5ZWZHYt28EDh+eiH37RiArKxIlJevt+r433HADFi5ciNLSUru+jxqEhYVBlmVnN8OmGOyoUEjIOMTEfAlZ7mC2XZY7cto5EbktU1mOupM4TGU57Bnw3HnnnQgLC8P8+fOve9y6desQExMDWZYRGRmJt956S9k3fPhwnDx5Es8++ywkSbpur9DixYvRp08ftG7dGhEREZg+fTp+/fVXs2NWrlyJG2+8Eb6+vnjwwQdx/vx5s/16vR79+/fHxx9/jBtvvBF+fn546qmnUF1djUWLFiEsLAzt27fHG2+8YfZztYexoqKiAAADBgyAJEkYPnw49Ho9Vq1ahX/84x/KeWRkZAAAzpw5g4cffhhBQUFo164dxo4dixMnTiivXV1djZkzZyIwMBDt2rXDrFmz4IglOhnsqFRIyDgMGXIC/fptQ8+ea9Cv3zYMGVLAQIeI3FLTZTmA/Pwkuw1peXp6IjU1Fe+88w5On254xmx2djbGjx+PRx55BAcOHIBer8crr7yClStXAgDWr1+Pjh074rXXXjMrHNkQDw8PvP322zh48CBWrVqF7777DrNmzVL2//jjj5g6dSqmT5+OvXv3YsSIEZg3b1691zl27Bi++eYbbNy4EZ999hk+/vhj3HvvvTh9+jQyMzOxcOFCvPzyy8jKymqwHT/99BMAYOvWrTAYDFi/fj2ef/55jB8/HnfffbdyHsOGDcPly5cxYsQI+Pn54fvvv8eOHTvg5+eHu+++G1VVVQCAt956Cx9//DGWL1+OHTt24MKFC9iwYYNF/w9agrOxVEySPDm9nIgI6ijL8eCDD6J///5ITk7G8uXL6+1fvHgx7rjjDrzyyisAgG7duuHQoUN48803MWXKFLRt2xaenp7w9/dHWFjYdd8rKSlJ+XdUVBRef/11PPXUU3j//fcBAEuXLsVdd92FF198UXmvnTt3YuPGjWavU1NTg48//hj+/v7o1asXRowYgby8PPznP/+Bh4cHunfvjoULFyIjIwNDhgyp146QkBAAQLt27cza7OPjA6PRaLZt9erV8PDwwEcffaT0Wq1YsQKBgYHIyMjAqFGjsGTJEsyZMwd//OMfAQAffvghNm3adN1rYQvs2SEiItVTS1mOhQsXYtWqVTh06FC9fYcPH8bNN99stu3mm2/G0aNHUV1tXY/Ttm3bMHLkSHTo0AH+/v6YNGkSzp8/j0uXLinvNXToULOfqfscuJYQ7e/vrzwPDQ1Fr1694OHhYbatuLjYqvY1JDs7G/n5+fD394efnx/8/PzQtm1b/Pbbbzh27BjKy8thMBjM2tmqVSsMGjSoxe/dFAY7RESkemopy3HbbbfhrrvuwksvvVRvnxCiXh5Oc/JRTp48idGjR6N3795Yt24dsrOz8d577wEArly5YtXrenl5mT2XJKnBbTU1NVa3s66amhrExsZi7969Zo8jR45g4sSJLX79lmCwQ0TkYAaDAXq93u3XALOGmspyLFiwAF9//TV27txptr1Xr17YsWOH2badO3eiW7du8PS8NoPW29u7yV6e3bt34+rVq3jrrbcwZMgQdOvWDWfPnq33XnXzbBrLu2kJb29vAKjX5obOY+DAgTh69Cjat2+P6Ohos0dAQAACAgKg0+nM2nn16lVkZ2fbvN11MdghInIwg8GAlJQUBjtWUFNZjj59+uDRRx/FO++8Y7b9ueeew7fffovXX38dR44cwapVq/Duu+/i+eefV46JjIzE999/jzNnzuCXX35p8PW7dOmCq1ev4p133sHx48fx6aef4sMPPzQ75plnnsHGjRuxaNEiHDlyBO+++269fB1baN++PXx8fLBx40acO3cO5eXlynns378feXl5+OWXX3DlyhU8+uijCA4OxtixY7F9+3YUFBQgMzMTiYmJSlJ3YmIiFixYgA0bNuDnn3/G9OnTUVZWZvN218Vgh4iIXIKaynK8/vrr9YaSBg4ciC+++AJr165F79698eqrr+K1117DlClTlGNee+01nDhxAl26dFGSf+vq378/Fi9ejIULF6J37974+9//Xm/K+5AhQ/DRRx/hnXfeQf/+/bF582a8/PLLNj/PVq1a4e2330ZaWhrCw8MxduxYAEB8fDy6d++OQYMGISQkBD/88AN8fX3x/fff48Ybb8S4cePQs2dPTJ06FZWVlWjTpg2AawHhpEmTMGXKFAwdOhT+/v548MEHbd7uuiThiAnuKldRUYGAgACUl5cr/0OIiGyp9lTjxlYy13qV5N9++w0FBQWIiorCDTfc0OzXEaIaZWXbUVVlgLe3DoGBt7LQqoZd73Nj6f2bU8+JiBwgLS0NKSkpZtvi4+OVfycnJ2t+Qc3i4mKUlZWhqqqqRcEOy3KQtRjsEBE5QEJCAsaMGQOg8Z4drSspKUF5eTmuXr3q7KaQm2GwQ0TkAA0NUw0cOFAJdojIfhjsEBGR3dTOVTp06BBCQkLw22+/KcXxvLy8lOnNRPbC2VhERA6m0+mQnJzsFkNXaWlpiI2NRWxsLF5++WUIIXD27FkcPnwYhw8fbnT6NZGJLeZRcTYWOBuLiMheavfs7N27F+3atUPfvn3Rvn17AOzZoaadP38excXFZsUZTTgbi4iInK5urtJ7772H9u3bw9fXF23atEFNTQ1+++03J7aQ1EoIgcuXL6O4uBiBgYH1Ah1rMNghIiKHWbFiBQDgr3/9Ky5cuODk1pArCAwMbHKV+KYw2CEiciMGgwFpaWlISEhweM6QTqdDfHw8li1bhunTpyMqKsqh70+ux8vLq0U9OiYMdoiI3IhpXa4xY8Y4LNipnbdz0003YdmyZdizZ4+yQrg7VI8m52KwQ0REdsXq0eRsqp96fvHiRSQlJaFTp07w8fHBsGHDsGvXLmW/EAJ6vR7h4eHw8fHB8OHDkZub68QWExGpi8FgQE5OjvIAYPbc3quvJyQkIDs7G9nZ2UhPTwcApKenK9sSEhLs+v5Equ/Z+fOf/4yDBw/i008/RXh4OFavXo0777wThw4dQocOHbBo0SIsXrwYK1euRLdu3TBv3jyMHDkSeXl58Pf3d3bziYicztk9K6weTc6m6jo7lZWV8Pf3xz/+8Q/ce++9yvb+/fvjvvvuw+uvv47w8HAkJSVh9uzZAACj0YjQ0FAsXLjQ4m8LrLNDRFpkSkZ+4IEHUFNTA8D5K67n5OQgNjYW2dnZDHaoxSy9f6t6GOvq1auorq6utzquj48PduzYgYKCAhQVFWHUqFHKPlmWERcXh507dzb6ukajERUVFWYPIldmMBig1+vtPhxBrsWUjFxTU6P0pJgCjNrPHZkc7E7Vo0k9VB3s+Pv7Y+jQoXj99ddx9uxZVFdXY/Xq1fjxxx9hMBhQVFQEAAgNDTX7udDQUGVfQ+bPn4+AgADlERERYdfzILI3002NwQ6pnU6ng16vZ7BDDqX6nJ1PP/0UU6dORYcOHeDp6YmBAwdi4sSJSpIdAGX6ookQot622ubMmYOZM2cqzysqKhjwEJEm1J7mXTsZ2cTDw4M9K+R2VB/sdOnSBZmZmbh06RIqKiqg0+nw8MMPIyoqSqmoWFRUZPaLW1xcXK+3pzZZliHLst3bTmRPTd3UWLvEPTk7GZlaxplFH7VM1cNYtbVu3Ro6nQ6lpaXYtGkTxo4dqwQ8W7ZsUY6rqqpCZmYmhg0b5sTWkhaoPQ+m9mrSpptZfHy8si0tLc3JLSRn4DRv18YhaftQfc/Opk2bIIRA9+7dkZ+fjxdeeAHdu3fHE088AUmSkJSUhNTUVHTt2hVdu3ZFamoqfH19MXHiRGc3nVycMyrNWiMhIQFjxowB0PgMG3I/nOZNVJ/qg53y8nLMmTMHp0+fRtu2bfHHP/4Rb7zxBry8vAAAs2bNQmVlJaZPn47S0lIMHjwYmzdvZo0d0jze1Ii0gUPS9qf6YGf8+PEYP358o/slSYJer+cYNNkE/+iQlnCat2tgnpX9qbqooKOwqCCZ6PX6en90alPrHx0mNRK5rrpfspxZ9NHVWHr/ZrADBjv0O/7RIXfDQNnxrnfNWWHaOpbev1U/jEXkSMyDIXej9kR8LeI1dzyXmXpO7kXt076JiOyBeVb2wZ4dUiU1fPPhHx3SKibiO56l19y0nAbZFnt2iBrBNXxIq1iQ0vG0cs1dtdedPTukGvy2SeQYLEjpeFq55mrodW8OBjukGqw1oV2c8aMuTMR3PF5z52KwQ6qhlW8+VJ+rfhskIm30ujPYIdXgNx8i+2qoh42J+I7natdcC73uDHaIyC608G2wKa42PNdQDxtn/zieq11zLfS6M9ghVXK1bz5Unxa+DTaFw3PkDrTQ685gh1TJ1b75UH1a+DaoBe7Qw0bUFAY7RGQXWvg22BBXCx7coYeNHMdVe90Z7BARWcFVggdTPtEDDzzAHjayGVftdWewQ0R256rfBhviKsNztfOJ6vamaaGHjcgaDHaIyO5c9dtgQ7Q6PEekZQx2iIg0oql8Ig8PD830sBFZg8EOEVEzqW14zlXyiYgcTRJCCGc3wtkqKioQEBCA8vJytGnTxtnNISJqlro9Ow3lE6klMCOyBUvv3+zZISLSCOYTETXMw9kNICIiIrInBjtERBqktnwiImdizg6Ys0NEROSKLL1/s2eHiIiINI3BDhEREWkagx0iIiLSNAY7REREpGkMdoiIiEjTGOwQERGRpjHYISIiIk1jsENERESaxmCHiIiINI3BDhFphsFggF6vV1b+JiICGOwQkYYYDAakpKQw2CEiMwx2iIiISNNaObsBREQtYTAYlJ6cnJwcs/8C11b/5srfRO6NwQ4RubS0tDSkpKSYbYuPj1f+nZycDL1e7+BWEZGaqHoY6+rVq3j55ZcRFRUFHx8fdO7cGa+99hpqamqUY4QQ0Ov1CA8Ph4+PD4YPH47c3FwntpqIHCkhIQHZ2dnIzs5Geno6ACA9PV3ZlpCQ4OQWEpGzqbpnZ+HChfjwww+xatUqxMTEYPfu3XjiiScQEBCAxMREAMCiRYuwePFirFy5Et26dcO8efMwcuRI5OXlwd/f38lnQET21tAw1cCBAzFw4EAntYiI1EbVPTv//e9/MXbsWNx7772IjIzEQw89hFGjRmH37t0ArvXqLFmyBHPnzsW4cePQu3dvrFq1CpcvX8aaNWuc3HoiIiJSA1UHO7fccgu+/fZbHDlyBACwb98+7NixA6NHjwYAFBQUoKioCKNGjVJ+RpZlxMXFYefOnY2+rtFoREVFhdmDiFyfTqdDcnIyE5KJyIyqh7Fmz56N8vJy9OjRA56enqiursYbb7yBCRMmAACKiooAAKGhoWY/FxoaipMnTzb6uvPnz6+X0EjqIEQ1ysq2o6rKAG9vHQIDb4UkeTq7WeQidDodk5GJqB5VBzuff/45Vq9ejTVr1iAmJgZ79+5FUlISwsPDMXnyZOU4SZLMfk4IUW9bbXPmzMHMmTOV5xUVFYiIiLD9CZBVSkrWIz8/EUbjaWWbLHdEdPRShISMc2LLiIjIlak62HnhhRfw4osv4pFHHgEA9OnTBydPnsT8+fMxefJkhIWFAbjWw1O727q4uLheb09tsixDlmX7Np6sUlKyHrm5DwEQZtuNxjPIzX0IMTFfMuAhIqJmUXXOzuXLl+HhYd5ET09PZep5VFQUwsLCsGXLFmV/VVUVMjMzMWzYMIe2lZpPiGrk5yeibqDzv70AgPz8JAhR7dB2ETkD1/cisj1VBzv3338/3njjDfz73//GiRMnsGHDBixevBgPPvgggGvDV0lJSUhNTcWGDRtw8OBBTJkyBb6+vpg4caKTW0+WKivbbjZ0VZ+A0ViIsrLtDmsTkbNwfS8i21P1MNY777yDV155BdOnT0dxcTHCw8ORkJCAV199VTlm1qxZqKysxPTp01FaWorBgwdj8+bNbltjxxUTfKuqLPujbulx5J4MBgPS0tKQkJDA2VhEZEYSQjQ0duBWKioqEBAQgPLycrRp08bZzWk2V03wLS3NwL59I5o8rl+/bQgKGm739pBrysnJQWxsLLKzs12uoGDd9b3i4+ORnp6unAfX9yJqmKX3b1UPY5HlTAm+dYeDTAm+JSXrndSypgUG3gpZ7gigsRl0EmQ5AoGBtzqyWUQOk5aWhtjYWMTGxirresXHxyvb0tLSnNxCItem6mEsskzTCb4S8vOTEBw8VpVDWpLkiejopf+bjSXB/DyuBUDR0UtU2XZyLq2seJ6QkIAxY8YAaLxnh4iaj8GOBliT4KvWYaCQkHGIifmykWG4JaoehiPn0cqK51zfi8i+GOxogFYSfENCxiE4eKzLJViT87BHhMgcE/UbxmBHA7y9LftAW3qcM0mSp2p7n0h9tNgjwvW9qCVMpQvGjBnDz1AtDHY0wJTgazSeQcN5OxJkuSMTfIlcANf3IrI9BjsawARfIvaIaBWHZZqmlUR9e2KdHWi9zk6EQxN8XbGoIRGplyvXT3IUvV5fL1G/NldJ1G8OS+/f7NnREGcn+LpqUUMiIlfGRP2mMdjRGGcl+HLV8qZptTteq+dFzsNhGetoMVHf1lhBmVqMq5ZbRqsLPGr1vMh5WFGabI09Oxrg7DwZLRQ1JCL14LBM8zFRv2EMdlycGvJktFLU0B602h2v1fOilrHVkCaHZZqPpQsaxmEsF6aWxT+1VNTQ1rTaHa/V86KW4ZAmqRV7dlyUmhb/ZFHDxmm1O16r50Xqw2EZsgUGOy5KTXkyLGrYOK12x2v1vMh69h7S5LAM2QKHsVyU2vJkTKuWy3IHs+2y3JHTzok0jEOa5ArYs+Oi1Jgn4+yihmqn1e54rZ4XWYZDmuQKuFwEXHO5CCGqkZUV2WSezJAhBQw2iMghGlvagYUnyV4svX9zGMtFmfJk/ves7l4A7psnQ0Tqwlla5GwMdlwY82SISE04pElqxZwdF8c8GSJSi9ozp1h4ktSEwY4GOGvxTyKixqSlpSElJcVsm2m2FgAkJydzSjk5TLODnfz8fBw7dgy33XYbfHx8IISAJNXNHSG1cfY6WuTemKjqPjhLi9TE6mDn/PnzePjhh/Hdd99BkiQcPXoUnTt3xp///GcEBgbirbfeskc7yQbUsI4WuTdTouqYMWPMbnYMgrSHhSdJTaxOUH722WfRqlUrnDp1Cr6+vsr2hx9+GBs3brRp48h21LKOFlFDOFuHiOzJ6p6dzZs3Y9OmTejYsaPZ9q5du+LkyZM2axjZjprW0SL3Y0miKmkbZ2m5NzX03Fod7Fy6dMmsR8fkl19+gSzLNmkU2Zaa1tEi99NUoupf/vIX3HTTTQA4W0eruL6Ve2ts+NqRrB7Guu222/DJJ58ozyVJQk1NDd58802MGDHCpo0j21DbOlrkXhISEpCdnY3s7Gykp6cDANLT0/GXv/wFALBs2TKuqUREdmV1z86bb76J4cOHY/fu3aiqqsKsWbOQm5uLCxcu4IcffrBHG6mF1LiOFrmPxhJV7733XiQkJADgbB0irVFbnSWrg51evXph//79+OCDD+Dp6YlLly5h3LhxmDFjBv84qVRg4K2Q5Y5NrqMVGHiro5tGboyzdYi0S211lqwKdq5cuYJRo0Y1eBKkXqZ1tHJzH8K1dbNqBzz2XUeLdX2oNiaqErkHtdVZsirY8fLywsGDB1k80AWZ1tFquM7OErvU2WFdH6qrsURVBkFE2qK2nltJCNHQuEajnnvuOXh5eWHBggX2apPDWbpEvBY4qqfFVNen/rDZtUCZC5USEbmHnJwcxMbGIjs72+bBjqX3b6tzdqqqqvDRRx9hy5YtGDRoEFq3bm22f/Hixda3lhzGEetosa4PERGZqKHn1upg5+DBg0pkduTIEbN9HN4igHV9iIjod2qos2R1sLNt2zZ7tIM0hHV9iLRBDZVviWzB6qKCtZ0+fRpnzpyxVVtII1jXh0gbuGYZaYXVwU5NTQ1ee+01BAQEoFOnTrjxxhsRGBiI119/HTU1NTZvYGRkJCRJqveYMWMGAEAIAb1ej/DwcPj4+GD48OHIzc21eTvIcqa6PqZk5PokyHIE6/pQkwwGA/R6PW+2RNQiVgc7c+fOxbvvvosFCxZgz549yMnJQWpqKt555x288sorNm/grl27lEqMBoMBW7ZsAQD86U9/AgAsWrQIixcvxrvvvotdu3YhLCwMI0eOxMWLF23eFrKMqa7P/57V3QvAfnV9SFvYs2C9lgaIBoMBOTk5ygOA2XP+v2gcg3MVE1bS6XTiH//4R73tX331lQgPD7f25ayWmJgounTpImpqakRNTY0ICwsTCxYsUPb/9ttvIiAgQHz44YcWv2Z5ebkAIMrLy+3RZLdVXLxO7NzZUWzbBuWxc2eEKC5e5+ymkYvIzs4WAER2drazm+IyWnrNkpOTBa5Nm2zwkZycbNsGawg/r45n6f3b6gTlCxcuoEePHvW29+jRAxcuXGh20GWJqqoqrF69GjNnzoQkSTh+/DiKioowatQo5RhZlhEXF4edO3cq6+7UZTQaYTQalecVFRV2bbe7CgkZh+Dgsayg3AzunBiqtjV13I3aKt8S2YLVwU6/fv3w7rvv4u233zbb/u6776Jfv342a1hDvvrqK5SVlWHKlCkAgKKiIgBAaGio2XGhoaE4efJko68zf/58LnfhII6o66NFpuGbMWPGuN3NRW1r6rgCWwaIaqt8q3YMzl2D1cHOokWLcO+992Lr1q0YOnQoJEnCzp07UVhYiP/85z/2aKNi+fLluOeeexAeHm62vW59HyHEdWv+zJkzBzNnzlSeV1RUICIiwraNJaJmYc+C9RwZILpzr2NDGJy7BquDnbi4OOTl5eH999/Hzz//DCEExo0bh+nTp9cLQmzp5MmT2Lp1K9avX69sCwsLA3Cth6f2L11xcXG93p7aZFmGLMt2aytRc/Ab4jXsWbCevQLEhirfunOvY0PcKTi3JtBVXVDskAwiG0hOThZhYWHiypUryjZTgvLChQuVbUajkQnK5JKYGFofEz6tZ+9rxv8njdP6tbHm/Bx1LeyWoLxixQr4+fkpU79N/u///g+XL1/G5MmTmx95NaKmpgYrVqzA5MmT0arV702WJAlJSUlITU1F165d0bVrV6SmpsLX1xcTJ060eTuI7MmdviFaSg1r6hB7Hcn1WR3sLFiwAB9++GG97e3bt8df/vIXuwQ7W7duxalTpzB16tR6+2bNmoXKykpMnz4dpaWlGDx4MDZv3gx/f3+bt4PInjh8U58a1tRxNfYIEJmXYhktBufWBLpqDoolIURDS1M36oYbbsDPP/+MyMhIs+0nTpxAz549UVlZacv2OYSlS8QTOUpOTg5iY2ORnZ3t1sEOqUPdm1hDvY5ausHT7/R6/XVnL9cOdK051lYsvX9b3bPTvn177N+/v16ws2/fPrRr187qhhJRfVr8hkiui72O7sua4XU1D8VbHew88sgjeOaZZ+Dv74/bbrsNAJCZmYnExEQ88sgjNm8gkTvi8A0RqYE1ga6ag2Krg5158+bh5MmTuOOOO5Rk4ZqaGkyaNAmpqak2byAREakHex3JFVmds2Ny9OhR7N27Fz4+PujTpw86depk67Y5DHN2iIiIrk+NdXYsvX83O9gxqa6uxoEDB9CpUycEBQW15KWchsEOERGR67H0/u1h7QsnJSVh+fLlAK4FOnFxcRg4cCAiIiKQkZHR7AYT2YoQ1SgtzcC5c5+htDQDQlQ7u0lEROREVgc7X375pbLg59dff43jx4/j559/RlJSEubOnWvzBhJZo6RkPbKyIrFv3wgcPjwR+/aNQFZWJEpK1jf9w0R1GAwG6PV6Zdo1Ebkmq4OdX375RVmT6j//+Q/Gjx+Pbt264cknn8SBAwds3kAiS5WUrEdu7kMwGk+bbTcazyA39yEGPGQ10zpQDHaIXJvVwU5oaCgOHTqE6upqbNy4EXfeeScA4PLly/D09LR5A4ksIUQ18vMTcW0ZqXp7AQD5+Ukc0iIickNWTz1/4oknMH78eOh0OkiShJEjRwIAfvzxR/To0cPmDSSyRFnZ9no9OuYEjMZClJVtR1DQcAe1ilyRmkveE1HzWB3s6PV69O7dG4WFhfjTn/4EWZYBAJ6ennjxxRdt3kAiS1RVWTbMYOlx5L64DhTZkqOmYNP1tXjquRZw6rnrKy3NwL59I5o8rl+/bezZoeviOlBkS85c584dAi27rY1FpEaBgbdCljvCaDyDhvN2JMhyRwQG3uroppGLUXPJeyJrmBLsx4wZo9lgx1IMdqjZhKhGWdl2VFUZ4O2tQ2DgrZAk5ySpS5InoqOXIjf3IQASzAMeCQAQHb3Eae0jIvfBvC/1YbBDzVJSsh75+YlmScGy3BHR0UsREjLOKW0KCRmHmJgvG2nXEqe1Sy3coUvb1rgOFDWHM/O+GGg1jDk7YM6OtUz1bOoPF13rQYmJ+dKpgYWaepzUxJm5A0TuxJl5X3q9vl6gVZvWEuyZs0N20XQ9Gwn5+UkIDh7r1CEtJiETkbM4M+8rISEBY8aMAdB4oOWOrAp23n//faxfvx5t27bFtGnTcPvttyv7fvnlF/zhD3/A8ePHbd5IUg/Ws3Et7NImci9MsG+YxRWU3377bbzwwgvo0aMHZFnG6NGjMX/+fGV/dXU1Tp48aZdGknqwno1rSUtLQ2xsLGJjY5Wcgfj4eGVbWlqak1tIpG3M+1IHi3t20tLSkJ6ejokTJwIApk+fjgceeACVlZV47bXX7NZAUhdvb8t+YS09juyLXdpEzqXT6ZyWI8NA63cWBzsFBQUYNmyY8nzo0KH47rvvcMcdd+DKlStISkqyR/tIZVjPxrWwS5vIfTkz0FIbi4Od4OBgFBYWIjIyUtkWExOD7777DrfffjvOnDljj/aRyrCeDRERuRqLc3ZuueUWrFu3rt72Xr164dtvv8XGjRtt2jBSL1M9G1nuYLZdljs6fdq5FghRjdLSDJw79xlKSzNstlI7u7SJyF1ZXGdn//79yM7OxhNPPNHg/tzcXHz55ZdITk62aQMdgXV2mof1bGxPjcUaiYjUytL7N4sKgsEOqYPaizUSEamNpfdvi4exiMh+mi7WCOTnJ9lsSIuIyJ0w2CFSAWuKNRIRkXUY7BCpAIs1EhHZD4MdIhVgsUYiIvthsEOkAqZijaZk5PokyHIEizUSETWD1cHO+fPnMWPGDPTq1QvBwcFo27at2YOIrGcq1vi/Z3X3AmCxRiKi5rJq1XMAeOyxx3Ds2DE8+eSTCA0NhSQ19k2UiKxhKtbYcJ2dJZx2TkTUTFbX2fH398eOHTvQr18/e7XJ4Vhnh9SExRqJiCxjtzo7PXr0QGVlZYsaR0SNkyRPBAUNR/v24wEAxcVf2HTZCLo+g8EAvV4Pg4Ez34iaS22/R1YHO++//z7mzp2LzMxMnD9/HhUVFWYPImq5kpL1yMqKxL59I3D48ETs2zcCWVmRKClZ7+ymaZ7BYEBKSopq/kgTuSK1/R5ZnbMTGBiI8vJy3H777WbbhRCQJAnV1fz26Swc/tCGxpaNMBrPIDf3IS4bQURkJauDnUcffRTe3t5Ys2YNE5RVhAtIakPTy0ZIyM9PQnDwWAayNmQwGJRvoDk5OWb/Ba6tGM/V4omuT82/R1YnKPv6+mLPnj3o3r27vdrkcK6eoMwFJLWjtDQD+/aNaPK4fv22IShouN3b4y70ej1SUlIa3Z+cnAy9Xu+4BhG5IGf8Hll6/7a6Z2fQoEEoLCzUVLDjytgToD4tGU7kshHOkZCQgDFjxgC49k00Pj4e6enpGDhwIACwV4fIAmr+PbI62Hn66aeRmJiIF154AX369IGXl5fZ/r59+9qscQBw5swZzJ49G9988w0qKyvRrVs3LF++HLGxsQCu5QqlpKRg2bJlKC0txeDBg/Hee+8hJibGpu1QK2sWkGRPgP21dDiRy0Y4R0Pd6wMHDlT+SBNR09T8e2R1sPPwww8DAKZOnapskyTJLgnKpaWluPnmmzFixAh88803aN++PY4dO4bAwEDlmEWLFmHx4sVYuXIlunXrhnnz5mHkyJHIy8uDv7+/zdqiVuwJUA9bJBablo0wGs/Ue51rJMhyRy4bQURkBauDnYKCAnu0o0ELFy5EREQEVqxYoWyLjIxU/i2EwJIlSzB37lyMG3ftJrJq1SqEhoZizZo1SEhIaPB1jUYjjEaj8tyVp8yzJ0AdbDWcaFo24lrQJNV5PccuG2EwGJCWloaEhAS3GsbR6XRITk52q3MmsjW1/R5ZnaDsSL169cJdd92F06dPIzMzEx06dMD06dMRHx8PADh+/Di6dOmCnJwcDBgwQPm5sWPHIjAwEKtWrWrwdRtLonLFBGUhqpGVFdlkT8CQIQXM2bEjWycWNzwcFuHQZSNycnIQGxuL7OxsVXRDExHVZbcEZZNDhw7h1KlTqKqqMttuSk6yhePHj+ODDz7AzJkz8dJLL+Gnn37CM888A1mWMWnSJBQVFQEAQkNDzX4uNDQUJ0+ebPR158yZg5kzZyrPKyoqEBERYbN2O5KaegLcma2HE0NCxiE4eCzrJhER2YDVwc7x48fx4IMP4sCBA0quDgCl3o4tc3ZqamowaNAgpKamAgAGDBiA3NxcfPDBB5g0aZJyXN1aP6b8ocbIsgxZlm3WTmfjApLOZ4/hRNOyEY6k5joZRETNZfVyEYmJiYiKisK5c+fg6+uL3NxcfP/99xg0aBAyMjJs2jidTodevXqZbevZsydOnToFAAgLCwMApYfHpLi4uF5vj9aFhIzDkCEn0K/fNvTsuQb9+m3DkCEFDHQcxJRYbOpNq0+CLEeoPrE4LS0NsbGxiI2NVYaL4+PjlW1paWlObiERkfWs7tn573//i++++w4hISHw8PCAh4cHbrnlFsyfPx/PPPMM9uzZY7PG3XzzzcjLyzPbduTIEXTq1AkAEBUVhbCwMGzZskXJ2amqqkJmZiYWLlxos3a4Cmf0BNA1WhlOVHOdDCKi5rI62Kmuroafnx8AIDg4GGfPnkX37t3RqVOneoFJSz377LMYNmwYUlNTMX78ePz0009YtmwZli1bBuDa8FVSUhJSU1PRtWtXdO3aFampqfD19cXEiRNt2haipmhhOFHNdTKIiJrL6mCnd+/e2L9/Pzp37ozBgwdj0aJF8Pb2xrJly9C5c2ebNu6mm27Chg0bMGfOHLz22muIiorCkiVL8OijjyrHzJo1C5WVlZg+fbpSVHDz5s1uUWOH1IeJxURE6mP11PNNmzbh0qVLGDduHI4fP4777rsPP//8M9q1a4fPP/+83mrorsDV18Yisgd3rbNDRK7D0vu3TersXLhwAUFBQS67AjqDHSIiItdj6f3b6tlY586dq7etbdu2kCQJ+/fvt/bliIiIiOzK6mCnT58++Oc//1lv+//7f/8PgwcPtkmjiIiIiGzF6mBn9uzZePjhhzFt2jRUVlbizJkzuP322/Hmm2/i888/t0cbiYiIbMJgMECv1yvFM8k9WB3sPPfcc8jKysIPP/yAvn37om/fvvDx8cH+/fttulQEERGRrRkMBqSkpDDYcTNWBzsA0LlzZ8TExODEiROoqKjA+PHj3a5iMbkGIapRWpqBc+c+Q2lpBoSw3XImRETkGqyus/PDDz/gscceQ7t27bB//3788MMPePrpp/Hvf/8baWlpCAoKskc7iazW8MrhHREdvdQlCvwRkeWuVyqBa76R1VPPZVnGs88+i9dffx1eXl4AgGPHjuHxxx/HqVOncPr06SZeQX049Vx7SkrW/2/phrof72vlEWJivmTAQ6QhOTk5iI2NRXZ2dr2K33q9HikpKY3+bHJyMvR6vZ1bSPZg6f3b6p6dzZs3Iy4uzmxbly5dsGPHDrzxxhvWt5TIxoSoRn5+IuoHOvjfNgn5+UkIDh7LysZEboBrvpHVwU7dQMfEw8MDr7zySosbRNRSZWXbzYau6hMwGgtRVradC6cSuTBLh6e45htZnKA8evRolJeXK8/feOMNlJWVKc/Pnz+PXr162bRxRM1RVWXZLAtLjyMidUpLS0NsbCxiY2MRHx8PAIiPj1e2paWlObmFpBYW9+xs2rQJRqNReb5w4UJMmDABgYGBAICrV6/afNVzoubw9rasS9rS44hInZozPKXT6ZCcnMyhKzdjcbBTN4/ZBktqEdlFYOCtkOWOMBrPoOG8HQmy3BGBgbc6umlEZEPNGZ7S6XRMRnZDzaqzQ6RmkuSJ6Oilpmd19wIAoqOXMDmZiMhNWBzsSJJUb1VzV13lnLQvJGQcYmK+hCx3MNsuyx057ZxIgzg8RddjcZ0dDw8P3HPPPZBlGQDw9ddf4/bbb0fr1q0BAEajERs3bkR1tetVqGWdHe0SohplZdtRVWWAt7cOgYG3skeHiEgjLL1/WxzsPPHEExa98YoVKyxroYow2CEiInI9Ni8q6IpBDBERERETlImIiEjTGOwQERGRpjHYISIiIk1jsENERESaxmCHiIiINI3BDhEREWkagx0iIhsxGAzQ6/UwGAzObgoR1cJgh4jIRgwGA1JSUhjsEKmMxUUFicg1aGGJDC2cAxGpB4MdohZS0425pGQ98vMTYTSeVrbJckdERy91mcVPXe0cDAaD0pOTk5Nj9l/g2gKVXJySyLksXhtLy7g2FjWXmm7MJSXrkZv7EIC6v9ISALjEau+ueA56vR4pKSmN7k9OToZer3dcg4jciM0XAtUyBjvUHGq6MQtRjaysSLOgq26bZLkjhgwpUO1wkKueQ92enfj4eKSnp2PgwIEA2LNDZE82XwiUiH4nRDXy8xNRP9DB/7ZJyM9PQnDwWIfcmMvKtl8nSLjWJqOxEGVl2xEUNNzu7WkOVz2HhoKZgQMHKsEOETkfZ2MRNYM1N2ZHqKqybPaPpcc5gxbOgYjUicEOUTOo7cbs7W3ZMImlxzmDFs5Bp9MhOTmZw1ZEKsNgh6gZ1HZjDgy8FbLcEaZ8ofokyHIEAgNvdUh7mkML56DT6aDX6xnsEKkMgx2iZlDbjVmSPBEdvVR577ptAYDo6CWqSuytSwvnQETqxGCHqBnUeGMOCRmHmJgvIcsdzLbLckdVTtluSHDwWHTqpEerVkFm213pHIhIfTj1HJx6Ts3XcJ2dCERHL3HajVlNRQ6t0dC1bNWqLTp2TESnTnNd4hyIyLFYZ8cKDHaoJZoKLlw1+HAkNdUsIiLXYen9W9XDWHq9HpIkmT3CwsKU/UII6PV6hIeHw8fHB8OHD0dubq4TW/w7IapRWpqBc+c+Q2lpBoSodnaTyE4kyRNBQcMRGjoBQUHDzQKZkpL1yMqKxL59I3D48ETs2zcCWVmRKClZ77T2qk3TNYuA/Pwk/g4RUbOpOtgBgJiYGKVCqcFgwIEDB5R9ixYtwuLFi/Huu+9i165dCAsLw8iRI3Hx4kUntpg3OLrG1FtRtx6P0XgGubkP8fPwP2qrWURE2qP6YKdVq1YICwtTHiEhIQCu9eosWbIEc+fOxbhx49C7d2+sWrUKly9fxpo1a677mkajERUVFWYPW+ENjgD2VlhDbTWLiEh7VB/sHD16FOHh4YiKisIjjzyC48ePAwAKCgpQVFSEUaNGKcfKsoy4uDjs3Lnzuq85f/58BAQEKI+IiAibtJU3ODJhb4Xl1FaziIi0R9XBzuDBg/HJJ59g06ZNSE9PR1FREYYNG4bz58+jqKgIABAaGmr2M6Ghocq+xsyZMwfl5eXKo7Cw0Cbt5Q2OTNhbYTm11SwiIu1R9UKg99xzj/LvPn36YOjQoejSpQtWrVqFIUOGAAAkyfwPpBCi3ra6ZFmGLMs2by9vcGRiaS9EZeVRO7dE/Uw1i67NxpJg3jPKYoJE1HKq7tmpq3Xr1ujTpw+OHj2qzMqq24tTXFxcr7fHUdgdTyZN91Zcc+JEMvO4oI2CiESkXi4V7BiNRhw+fBg6nQ5RUVEICwvDli1blP1VVVXIzMzEsGHDnNI+dseTiXmF5eseyTyu/wkJGYchQ06gX79t6NlzDfr124YhQwoY6NB1GQwG6PV6GAzsMafGqTrYef7555GZmYmCggL8+OOPeOihh1BRUYHJkydDkiQkJSUhNTUVGzZswMGDBzFlyhT4+vpi4sSJTmmvGpcQIOcJCRmHTp30TRzFPK7arleziKghBoMBKSkpDHboulSds3P69GlMmDABv/zyC0JCQjBkyBBkZWWhU6dOAIBZs2ahsrIS06dPR2lpKQYPHozNmzfD39/faW02dcfXX0Kgo1OXECDn8PXtatFxzOMiIrIfVQc7a9euve5+SZKg1+uh1+sd0yALhYSMQ3DwWC4RQMzjIrIDU5FZAMjJyTH7LwDodDrodPydot+pOthxZabueHJvpjwuo/EMGq6/JEGWOzKPi1THYDAgLS0NCQkJqgsc0tLSkJKSYrYtPj5e+XdycrLqvgSTc6k6Z4fI1TGPi1yVmnNhEhISkJ2djezsbKSnpwMA0tPTlW0JCQlObiGpDXt2NI4rbjsf87iIbKuhYaqBAwdi4MCBTmoRqR2DHQ0rKVnfyA12KW+wDna9PC4GpKQWzIUhrWKwo1GmBUnr5omYFiRloTbHayiPiwGp63CHoNQVc2F0Oh2Sk5MZhNF1SUKIhrIm3UpFRQUCAgJQXl6ONm3aOLs5LSZENbKyIq+zTte1pNghQwo098falTQWkJpyeRiQqoerB6WWJhvX7dmJj49Henq6MjzEnh1SG0vv3+zZ0SBrFiTljDHnEKIa+fmJaHiGloCpsnJw8FgGpE6mhV5SU7LxmDFjrhusMBeGtIqzsTSIC5KqnzUBKTlP00EpuNwHkQtgz44GsZCd+l2ru9M0BqTO5cq9pC1NNmYuDGkJgx0NYiE7dSspWY9jx5616FgGpM7lyr2kLU021ul0qktGJmouDmNpEAvZqZcp/+PKlZImjpQgyxEMSJ3MlXtJWXiPTLgyPIMdzTIVspPlDmbbZbmjSyRUatH18z/qY0DqfKZe0vpfGkzUG5TqdDoludiUYFz7OYenHMfZwYaaq2E7CoexNIwLkqpL0/kf13h5haBbtw8ZkKqAqZf02mwsCeaBKntJyTKWzoYj+2Gwo3FckFQ9LM3riI7+GwMdFdHCch9MNnY/rIZtjsEOkYNYnv/RoemDyKFcvZdUDcnGal5F3R6cHWy4YjVse2IFZWivgjKp0++Vra8/S46VrUmLcnJyEBsbi+zsbLcoUqjX6+sFG7XZO9hwl2rYrKDsRtxhzR4tYP4HkftISEjAmDFjADQebNgTq2GbY7Dj4lx9zR53o4X8DyJLOXsox5kYbKgLgx07s2evixbW7HFHrp7/QWQp5o2oAxPUmbMDwH45O/bsdeHK5kSkdu6SN9IUd0vOdiRL798MdmCfYKexXhdTboalvS6N9QyVlmZg374RTf58v37bOPWciJzO0QnKDDDcg6X3b1ZQtgNbrZRcUrIeWVmR2LdvBA4fnoh9+0YgKysSJSXrXXrNHiIie1Nz1WBnV1R2Rwx27MCalZIbY+oZqvs6pnycy5ePWtQWNa7ZQ0Tuh3kjv1NzIKZVTFC2g5b2ujTdMyShqCgd3t4dUFV1tpHjuLI5UUuxrIPtNKewobXX351nf9H1Mdixg5aulGxZz9BpREam4MQJPVizhVyBqwUOLOvgXM25/mqe/cVAzLmYoAzbJyi3tFLuuXOf4fDhiU2+T8+ea+DhITfwByGCNVtIVVwtcLDVBANqnuZefzXP/nJ2RWWtYgVlJ2pppVxreoaCgoazZgupmqvVg7JkGDk/PwnBwWP5e2YHLbn+ai7k5+yKyu6OwY6dtKRSbmDgrZDljk32DJnycbiyOamVKwYO1kww4O+d7Wn1+qs5EHMHDHbsqLmVcrmGEmmFK964WNbBuWx1/Tn7i2pjsGNnze114RpKpAWuGDi0dIKBmrhaUjhgu+vfnNlfjsJAzPEY7KgY11AiV+eKgYO1w8hq5WpJ4SZauf7Xo+ZATKtYVFDlTD1DoaETEBQ0nIEOuRTTjcs0/FqfBFmOcPqNS4hqlJZm4Ny5z1BWth1duixW2mfONYaRmypKWlKy3kkta5ppGP9/z+ruBaD+60/qw2CHiOzGFW5cDS3LcuzYTEREPA9Z7mB2rCx3VN3ssbpstVyNM5mG8V3x+pM6sc4O7LfqORFd0/CQivPrQTVVz6VXry/g5RXsUsPIWlok2BVzjsixWGeHiFRDjflnlkyLP3ZsZqPFP9XKFZPCGyNJnggMvFX53JSVbXf654ZcE4MdInIItdWDcsVp8ZZwxaTwxrhqkjWpD3N2iMgtaakHpDZXSQpviisnWZP6MNghIrekpR6Q2lwhKbwpWkiyJnVxqWBn/vz5kCQJSUlJyjYhBPR6PcLDw+Hj44Phw4cjNzfXeY0kIpeglR6Qhrj6bCZrhhiJLOEyOTu7du3CsmXL0LdvX7PtixYtwuLFi7Fy5Up069YN8+bNw8iRI5GXlwd/f38ntZaI1E7ry7KoMSncUlodYiTncYmenV9//RWPPvoo0tPTERQUpGwXQmDJkiWYO3cuxo0bh969e2PVqlW4fPky1qxZ48QWE7mG2sX0Sksz3G5YwNV7QJriqkVJtTrESM7jEj07M2bMwL333os777wT8+bNU7YXFBSgqKgIo0aNUrbJsoy4uDjs3LkTCQkJDb6e0WiE0WhUnldUVNiv8UQqxZku17hyD4hWucOSEeRYqu/ZWbt2LXJycjB//vx6+4qKigAAoaGhZttDQ0OVfQ2ZP38+AgIClEdERIRtG02kcpzpYs5Ve0C0SgtJ1qQuqg52CgsLkZiYiNWrV+OGG25o9DhJMv9lEELU21bbnDlzUF5erjwKCwtt1mYiteNMF3IFWh9ibAl3H35uDlUPY2VnZ6O4uBixsbHKturqanz//fd49913kZeXB+BaD49O9/vYbXFxcb3entpkWYYsy/ZrOJGKabWYHmkPhxjr4/Bz86g62Lnjjjtw4MABs21PPPEEevTogdmzZ6Nz584ICwvDli1bMGDAAABAVVUVMjMzsXDhQmc0mUj1ONOFXInaKm87U2NruZmGn63t8XKntcdUHez4+/ujd+/eZttat26Ndu3aKduTkpKQmpqKrl27omvXrkhNTYWvry8mTpzojCYTqR5nuhC5HkvWcsvPT0Jw8FiLAhZ36yFSdbBjiVmzZqGyshLTp09HaWkpBg8ejM2bN7PGDlEjONOFyPXYcvjZ1j1ErkASQjT0186tWLpEPJFW/P7HDmiomJ4W/9gRubJz5z7D4cNNj1j07LkGoaETGt0vRDWysiKvEzhd+7IzZEiBSwxpWXr/VvVsLCKyD850IXItthp+dtelOFx+GIuImoczXYhch62Gn911ggKDHSI3xpkuRK7BVmu5uesEBQ5jERERuQBbDD+beojqV6Y2kSDLEZqboMCeHSIiIhfR0uFnW/UQuRoGO0RERC6kpcPPph6ihuvsLNHkBAUGO0RERG7G3SYoMNghIiJyQ+40QYEJykRERKRp7NkhchPutOgfEVFtDHaI3IC7LfpHRFQbh7GINM60DlbdEvGmRf9KStY7qWVERI7BYIdIw4SoRn5+IhouL39tW35+EoSodmi7iIgcicEOkYa566J/RES1Mdgh0jB3XfSPiKg2BjtEGuaui/4REdXGYIdIw9x10T8iotoY7BBpmGnRv/89q7sXgDYX/SMiqo3BDpHGmRb9k+UOZttluSNiYr5knR0i0jwWFSRyA+626B8RUW0MdojchDst+kdEVBuHsYiIiEjTGOwQERGRpjHYISIiIk1jsENERESaxmCHiIiINI3BDhEREWkagx0iIiLSNAY7REREpGkMdoiIiEjTGOwQERGRpjHYISIiIk1jsENERESaxmCHiIiINI2rnhMRaZgQ1Sgr246qKgO8vXUIDLwVkuTp7GYRORSDHSIijSopWY/8/EQYjaeVbbLcEdHRSxESMs6JLSNyLA5jERFpUEnJeuTmPmQW6ACA0XgGubkPoaRkvZNaRuR4DHaIiDRGiGrk5ycCEA3tBQDk5ydBiGqHtovIWVQd7HzwwQfo27cv2rRpgzZt2mDo0KH45ptvlP1CCOj1eoSHh8PHxwfDhw9Hbm6uE1tMROR8ZWXb6/XomBMwGgtRVrbdYW0iciZVBzsdO3bEggULsHv3buzevRu33347xo4dqwQ0ixYtwuLFi/Huu+9i165dCAsLw8iRI3Hx4kUnt5yIyHmqqgw2PY7I1ak62Ln//vsxevRodOvWDd26dcMbb7wBPz8/ZGVlQQiBJUuWYO7cuRg3bhx69+6NVatW4fLly1izZo2zm05E5DTe3jqbHkfk6lQd7NRWXV2NtWvX4tKlSxg6dCgKCgpQVFSEUaNGKcfIsoy4uDjs3Lnzuq9lNBpRUVFh9iAi0orAwFshyx0BSI0cIUGWIxAYeKsjm0XkNKoPdg4cOAA/Pz/Isoxp06Zhw4YN6NWrF4qKigAAoaGhZseHhoYq+xozf/58BAQEKI+IiAi7tZ+IyNEkyRPR0UtNz+ruBQBERy9hvR1yG6oPdrp37469e/ciKysLTz31FCZPnoxDhw4p+yXJ/BdZCFFvW11z5sxBeXm58igsLLRL24mInCUkZBxiYr6ELHcw2y7LHRET8yXr7JBbUX1RQW9vb0RHRwMABg0ahF27dmHp0qWYPXs2AKCoqAg63e/jzsXFxfV6e+qSZRmyLNuv0UREKhASMg7BwWNZQZncnup7duoSQsBoNCIqKgphYWHYsmWLsq+qqgqZmZkYNmyYE1tIRKQekuSJoKDhCA2dgKCg4Qx0yC2pumfnpZdewj333IOIiAhcvHgRa9euRUZGBjZu3AhJkpCUlITU1FR07doVXbt2RWpqKnx9fTFx4kRnN52IiIhUQtXBzrlz5/D444/DYDAgICAAffv2xcaNGzFy5EgAwKxZs1BZWYnp06ejtLQUgwcPxubNm+Hv7+/klhMREZFaSEKIhuqJu5WKigoEBASgvLwcbdq0cXZziIiIyAKW3r9dLmeHiIiIyBoMdoiIiEjTGOwQERGRpjHYISIiIk1jsENERESaxmCHiIiINE3VdXYcxTT7nqufExERuQ7TfbupKjoMdgBcvHgRALj6ORERkQu6ePEiAgICGt3PooIAampqcPbsWfj7+ze5Yro1KioqEBERgcLCQrctVuju18Ddzx/gNQB4Ddz9/AFeA3udvxACFy9eRHh4ODw8Gs/MYc8OAA8PD3Ts2NFur9+mTRu3/HDX5u7XwN3PH+A1AHgN3P38AV4De5z/9Xp0TJigTERERJrGYIeIiIg0jcGOHcmyjOTkZMiy7OymOI27XwN3P3+A1wDgNXD38wd4DZx9/kxQJiIiIk1jzw4RERFpGoMdIiIi0jQGO0RERKRpDHaIiIhI0xjstNAHH3yAvn37KoWShg4dim+++UbZL4SAXq9HeHg4fHx8MHz4cOTm5jqxxfY3f/58SJKEpKQkZZvWr4Ner4ckSWaPsLAwZb/Wzx8Azpw5g8ceewzt2rWDr68v+vfvj+zsbGW/1q9BZGRkvc+AJEmYMWMGAO2f/9WrV/Hyyy8jKioKPj4+6Ny5M1577TXU1NQox2j9GgDXli1ISkpCp06d4OPjg2HDhmHXrl3Kfq1dg++//x73338/wsPDIUkSvvrqK7P9lpyv0WjE008/jeDgYLRu3RpjxozB6dOnbdtQQS3yz3/+U/z73/8WeXl5Ii8vT7z00kvCy8tLHDx4UAghxIIFC4S/v79Yt26dOHDggHj44YeFTqcTFRUVTm65ffz0008iMjJS9O3bVyQmJirbtX4dkpOTRUxMjDAYDMqjuLhY2a/1879w4YLo1KmTmDJlivjxxx9FQUGB2Lp1q8jPz1eO0fo1KC4uNvv/v2XLFgFAbNu2TQih/fOfN2+eaNeunfjXv/4lCgoKxP/93/8JPz8/sWTJEuUYrV8DIYQYP3686NWrl8jMzBRHjx4VycnJok2bNuL06dNCCO1dg//85z9i7ty5Yt26dQKA2LBhg9l+S8532rRpokOHDmLLli0iJydHjBgxQvTr109cvXrVZu1ksGMHQUFB4qOPPhI1NTUiLCxMLFiwQNn322+/iYCAAPHhhx86sYX2cfHiRdG1a1exZcsWERcXpwQ77nAdkpOTRb9+/Rrc5w7nP3v2bHHLLbc0ut8drkFdiYmJokuXLqKmpsYtzv/ee+8VU6dONds2btw48dhjjwkh3OMzcPnyZeHp6Sn+9a9/mW3v16+fmDt3ruavQd1gx5LzLSsrE15eXmLt2rXKMWfOnBEeHh5i48aNNmsbh7FsqLq6GmvXrsWlS5cwdOhQFBQUoKioCKNGjVKOkWUZcXFx2LlzpxNbah8zZszAvffeizvvvNNsu7tch6NHjyI8PBxRUVF45JFHcPz4cQDucf7//Oc/MWjQIPzpT39C+/btMWDAAKSnpyv73eEa1FZVVYXVq1dj6tSpkCTJLc7/lltuwbfffosjR44AAPbt24cdO3Zg9OjRANzjM3D16lVUV1fjhhtuMNvu4+ODHTt2uMU1qM2S883OzsaVK1fMjgkPD0fv3r1tek0Y7NjAgQMH4OfnB1mWMW3aNGzYsAG9evVCUVERACA0NNTs+NDQUGWfVqxduxY5OTmYP39+vX3ucB0GDx6MTz75BJs2bUJ6ejqKioowbNgwnD9/3i3O//jx4/jggw/QtWtXbNq0CdOmTcMzzzyDTz75BIB7fAZq++qrr1BWVoYpU6YAcI/znz17NiZMmIAePXrAy8sLAwYMQFJSEiZMmADAPa6Bv78/hg4ditdffx1nz55FdXU1Vq9ejR9//BEGg8EtrkFtlpxvUVERvL29ERQU1OgxtsBVz22ge/fu2Lt3L8rKyrBu3TpMnjwZmZmZyn5JksyOF0LU2+bKCgsLkZiYiM2bN9f7RlOblq/DPffco/y7T58+GDp0KLp06YJVq1ZhyJAhALR9/jU1NRg0aBBSU1MBAAMGDEBubi4++OADTJo0STlOy9egtuXLl+Oee+5BeHi42XYtn//nn3+O1atXY82aNYiJicHevXuRlJSE8PBwTJ48WTlOy9cAAD799FNMnToVHTp0gKenJwYOHIiJEyciJydHOUbr16Cu5pyvra8Je3ZswNvbG9HR0Rg0aBDmz5+Pfv36YenSpcpsnLrRaXFxcb1I15VlZ2ejuLgYsbGxaNWqFVq1aoXMzEy8/fbbaNWqlXKuWr8OtbVu3Rp9+vTB0aNH3eJzoNPp0KtXL7NtPXv2xKlTpwDALa6BycmTJ7F161b8+c9/Vra5w/m/8MILePHFF/HII4+gT58+ePzxx/Hss88qvb3ucA0AoEuXLsjMzMSvv/6KwsJC/PTTT7hy5QqioqLc5hqYWHK+YWFhqKqqQmlpaaPH2AKDHTsQQsBoNCof7i1btij7qqqqkJmZiWHDhjmxhbZ1xx134MCBA9i7d6/yGDRoEB599FHs3bsXnTt3dovrUJvRaMThw4eh0+nc4nNw8803Iy8vz2zbkSNH0KlTJwBwi2tgsmLFCrRv3x733nuvss0dzv/y5cvw8DC/pXh6eipTz93hGtTWunVr6HQ6lJaWYtOmTRg7dqzbXQNLzjc2NhZeXl5mxxgMBhw8eNC218Rmqc5uas6cOeL7778XBQUFYv/+/eKll14SHh4eYvPmzUKIa9PuAgICxPr168WBAwfEhAkTXHqaoaVqz8YSQvvX4bnnnhMZGRni+PHjIisrS9x3333C399fnDhxQgih/fP/6aefRKtWrcQbb7whjh49Kv7+978LX19fsXr1auUYrV8DIYSorq4WN954o5g9e3a9fVo//8mTJ4sOHTooU8/Xr18vgoODxaxZs5RjtH4NhBBi48aN4ptvvhHHjx8XmzdvFv369RN/+MMfRFVVlRBCe9fg4sWLYs+ePWLPnj0CgFi8eLHYs2ePOHnypBDCsvOdNm2a6Nixo9i6davIyckRt99+O6eeq83UqVNFp06dhLe3twgJCRF33HGHEugIcW3qXXJysggLCxOyLIvbbrtNHDhwwIktdoy6wY7Wr4OpdoSXl5cIDw8X48aNE7m5ucp+rZ+/EEJ8/fXXonfv3kKWZdGjRw+xbNkys/3ucA02bdokAIi8vLx6+7R+/hUVFSIxMVHceOON4oYbbhCdO3cWc+fOFUajUTlG69dACCE+//xz0blzZ+Ht7S3CwsLEjBkzRFlZmbJfa9dg27ZtAkC9x+TJk4UQlp1vZWWl+Otf/yratm0rfHx8xH333SdOnTpl03ZKQghhu34iIiIiInVhzg4RERFpGoMdIiIi0jQGO0RERKRpDHaIiIhI0xjsEBERkaYx2CEiIiJNY7BDREREmsZgh4iIiDSNwQ4RERFpGoMdIrLYlClTIElSvcfdd9/t7KY1KDExEbGxsZBlGf3793d2c4jISVo5uwFE5FruvvturFixwmybLMtOas31CSEwdepU/Pjjj9i/f7+zm1NPVVUVvL29nd0MIs1jzw4RWUWWZYSFhZk9goKCAAAZGRnw9vbG9u3blePfeustBAcHw2AwAAA2btyIW265BYGBgWjXrh3uu+8+HDt2TDn+xIkTkCQJX3zxBW699Vb4+PjgpptuwpEjR7Br1y4MGjQIfn5+uPvuu1FSUnLdtr799tuYMWMGOnfubPH56fV63HjjjZBlGeHh4XjmmWeUfUajEbNmzUJERARkWUbXrl2xfPlyZX9mZib+8Ic/QJZl6HQ6vPjii7h69aqyf/jw4fjrX/+KmTNnIjg4GCNHjgQAHDp0CKNHj4afnx9CQ0Px+OOP45dffrG4zUR0fQx2iMhmhg8fjqSkJDz++OMoLy/Hvn37MHfuXKSnp0On0wEALl26hJkzZ2LXrl349ttv4eHhgQcffBA1NTVmr5WcnIyXX34ZOTk5aNWqFSZMmIBZs2Zh6dKl2L59O44dO4ZXX33Vpu3/8ssv8be//Q1paWk4evQovvrqK/Tp00fZP2nSJKxduxZvv/02Dh8+jA8//BB+fn4AgDNnzmD06NG46aabsG/fPnzwwQdYvnw55s2bZ/Yeq1atQqtWrfDDDz8gLS0NBoMBcXFx6N+/P3bv3o2NGzfi3LlzGD9+vE3Pjcit2XQNdSLStMmTJwtPT0/RunVrs8drr72mHGM0GsWAAQPE+PHjRUxMjPjzn/983dcsLi4WAMSBAweEEEIUFBQIAOKjjz5Sjvnss88EAPHtt98q2+bPny+6d+9uUbuTk5NFv379mjzurbfeEt26dRNVVVX19uXl5QkAYsuWLQ3+7EsvvSS6d+8uampqlG3vvfee8PPzE9XV1UIIIeLi4kT//v3Nfu6VV14Ro0aNMttWWFgoAIi8vLwm20xETWPPDhFZZcSIEdi7d6/ZY8aMGcp+b29vrF69GuvWrUNlZSWWLFli9vPHjh3DxIkT0blzZ7Rp0wZRUVEAgFOnTpkd17dvX+XfoaGhAGDWyxIaGori4mKbntuf/vQnVFZWonPnzoiPj8eGDRuUYai9e/fC09MTcXFxDf7s4cOHMXToUEiSpGy7+eab8euvv+L06dPKtkGDBpn9XHZ2NrZt2wY/Pz/l0aNHDwAwG94jouZjgjIRWaV169aIjo6+7jE7d+4EAFy4cAEXLlxA69atlX33338/IiIikJ6ejvDwcNTU1KB3796oqqoyew0vLy/l36YAou62ukNfLRUREYG8vDxs2bIFW7duxfTp0/Hmm28iMzMTPj4+1/1ZIYRZoGPaVrv9AMyuBQDU1NTg/vvvx8KFC+u9pmnoj4hahj07RGRTx44dw7PPPov09HQMGTIEkyZNUoKS8+fP4/Dhw3j55Zdxxx13oGfPnigtLXVyi835+PhgzJgxePvtt5GRkYH//ve/OHDgAPr06YOamhpkZmY2+HO9evXCzp07lQAHuBb0+fv7o0OHDo2+38CBA5Gbm4vIyEhER0ebPeoGRkTUPAx2iMgqRqMRRUVFZg/TzKHq6mo8/vjjGDVqFJ544gmsWLECBw8exFtvvQUACAoKQrt27bBs2TLk5+fju+++w8yZM+3W1vz8fOzduxdFRUWorKxUht3q9iKZrFy5EsuXL8fBgwdx/PhxfPrpp/Dx8UGnTp0QGRmJyZMnY+rUqfjqq69QUFCAjIwMfPHFFwCA6dOno7CwEE8//TR+/vln/OMf/0BycjJmzpwJD4/G/9TOmDEDFy5cwIQJE/DTTz/h+PHj2Lx5M6ZOnYrq6mq7XBcit+PknCEiciGTJ08WAOo9TInCKSkpQqfTiV9++UX5ma+++kp4e3uLPXv2CCGE2LJli+jZs6eQZVn07dtXZGRkCABiw4YNQojfE5RNxwshxLZt2wQAUVpaqmxbsWKFCAgIuG574+LiGmxvQUFBg8dv2LBBDB48WLRp00a0bt1aDBkyRGzdulXZX1lZKZ599lmh0+mEt7e3iI6OFh9//LGyPyMjQ9x0003C29tbhIWFidmzZ4srV66YtScxMbHe+x45ckQ8+OCDIjAwUPj4+IgePXqIpKQks2RnImo+SYhafa5EREREGsNhLCIiItI0BjtERESkaQx2iIiISNMY7BAREZGmMdghIiIiTWOwQ0RERJrGYIeIiIg0jcEOERERaRqDHSIiItI0BjtERESkaQx2iIiISNP+P23CLUB8hyGaAAAAAElFTkSuQmCC",
"text/plain": [
"
\n",
" 
\n",
" 
\n",
"