chapter05_fundamentals-of-ml.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dJlvY6QDTk2x"
      },
      "source": [
        "이 노트북은 [케라스 창시자에게 배우는 딥러닝 2판](https://tensorflow.blog/kerasdl2/)의 예제 코드를 담고 있습니다.\n",
        "\n",
        "<table align=\"left\">\n",
        "    <tr>\n",
        "        <td>\n",
        "            <a href=\"https://colab.research.google.com/github/rickiepark/deep-learning-with-python-2nd/blob/main/chapter05_fundamentals-of-ml.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
        "        </td>\n",
        "    </tr>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# 코랩에서 메모리 부족을 피하기 위해 부동소수점 정밀도를 낮춥니다.\n",
        "import tensorflow as tf\n",
        "\n",
        "tf.keras.backend.set_floatx('float16')"
      ],
      "metadata": {
        "id": "RcrnxWzGPFqG"
      },
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mPvlk_EMTk24"
      },
      "source": [
        "# 머신 러닝의 기본 요소"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QIwdb7e7Tk25"
      },
      "source": [
        "## 일반화: 머신 러닝의 목표"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tPMwQAH7Tk29"
      },
      "source": [
        "### 과소적합과 과대적합"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pGWJKdoUTk2-"
      },
      "source": [
        "#### 잡음 섞인 훈련 데이터"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6px_zXDbTk2_"
      },
      "source": [
        "#### 불확실한 특성"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QgElP-miTk3A"
      },
      "source": [
        "#### 드문 특성과 가짜 상관관계"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VfRNGJB8Tk3B"
      },
      "source": [
        "**MNIST에 백색 잡음 픽셀과 0 픽셀을 추가하기**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T01:01:24.254476Z",
          "iopub.status.busy": "2022-05-21T01:01:24.253958Z",
          "iopub.status.idle": "2022-05-21T01:01:27.373051Z",
          "shell.execute_reply": "2022-05-21T01:01:27.372074Z"
        },
        "id": "KBz18RQVTk3C"
      },
      "outputs": [],
      "source": [
        "from tensorflow.keras.datasets import mnist\n",
        "import numpy as np\n",
        "\n",
        "(train_images, train_labels), _ = mnist.load_data()\n",
        "train_images = train_images.reshape((60000, 28 * 28))\n",
        "train_images = train_images.astype(\"float32\") / 255\n",
        "\n",
        "train_images_with_noise_channels = np.concatenate(\n",
        "    [train_images, np.random.random((len(train_images), 784))], axis=1)\n",
        "\n",
        "train_images_with_zeros_channels = np.concatenate(\n",
        "    [train_images, np.zeros((len(train_images), 784))], axis=1)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T01:01:27.377455Z",
          "iopub.status.busy": "2022-05-21T01:01:27.376945Z",
          "iopub.status.idle": "2022-05-21T01:01:27.388938Z",
          "shell.execute_reply": "2022-05-21T01:01:27.387917Z"
        },
        "id": "8cUOncJ1Tk3F",
        "outputId": "9488b6f7-0119-41d3-eb48-c516448ac779",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(60000, 1568)"
            ]
          },
          "metadata": {},
          "execution_count": 3
        }
      ],
      "source": [
        "train_images_with_noise_channels.shape"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Eh9dzVI3Tk3I"
      },
      "source": [
        "**백색 잡음과 0을 추가한 MNIST 데이터에서 모델 훈련하기**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T01:01:27.392783Z",
          "iopub.status.busy": "2022-05-21T01:01:27.392444Z",
          "iopub.status.idle": "2022-05-21T01:02:25.502977Z",
          "shell.execute_reply": "2022-05-21T01:02:25.502078Z"
        },
        "id": "H8BUR0xETk3J",
        "outputId": "96dd32d9-6c0b-4554-b335-f3b39d681e26",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "375/375 [==============================] - 7s 4ms/step - loss: 0.5972 - accuracy: 0.8198 - val_loss: 0.3325 - val_accuracy: 0.8945\n",
            "Epoch 2/10\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.2515 - accuracy: 0.9395 - val_loss: 0.2339 - val_accuracy: 0.9307\n",
            "Epoch 3/10\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.1688 - accuracy: 0.9673 - val_loss: 0.1870 - val_accuracy: 0.9482\n",
            "Epoch 4/10\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.1223 - accuracy: 0.9839 - val_loss: 0.1940 - val_accuracy: 0.9429\n",
            "Epoch 5/10\n",
            "375/375 [==============================] - 2s 4ms/step - loss: 0.0887 - accuracy: 0.9907 - val_loss: 0.1495 - val_accuracy: 0.9575\n",
            "Epoch 6/10\n",
            "375/375 [==============================] - 1s 4ms/step - loss: 0.0646 - accuracy: 0.9932 - val_loss: 0.1267 - val_accuracy: 0.9653\n",
            "Epoch 7/10\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.0483 - accuracy: 0.9966 - val_loss: 0.1335 - val_accuracy: 0.9639\n",
            "Epoch 8/10\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.0356 - accuracy: 0.9985 - val_loss: 0.1169 - val_accuracy: 0.9702\n",
            "Epoch 9/10\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.0245 - accuracy: 0.9980 - val_loss: 0.1260 - val_accuracy: 0.9688\n",
            "Epoch 10/10\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.0163 - accuracy: 0.9985 - val_loss: 0.1464 - val_accuracy: 0.9648\n",
            "Epoch 1/10\n",
            "375/375 [==============================] - 2s 4ms/step - loss: 0.3020 - accuracy: 0.9272 - val_loss: 0.1681 - val_accuracy: 0.9521\n",
            "Epoch 2/10\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.1254 - accuracy: 0.9829 - val_loss: 0.1091 - val_accuracy: 0.9712\n",
            "Epoch 3/10\n",
            "375/375 [==============================] - 2s 4ms/step - loss: 0.0822 - accuracy: 0.9912 - val_loss: 0.0963 - val_accuracy: 0.9731\n",
            "Epoch 4/10\n",
            "375/375 [==============================] - 1s 4ms/step - loss: 0.0595 - accuracy: 0.9961 - val_loss: 0.0807 - val_accuracy: 0.9795\n",
            "Epoch 5/10\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.0448 - accuracy: 0.9976 - val_loss: 0.0767 - val_accuracy: 0.9785\n",
            "Epoch 6/10\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.0340 - accuracy: 0.9980 - val_loss: 0.0783 - val_accuracy: 0.9766\n",
            "Epoch 7/10\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.0256 - accuracy: 0.9985 - val_loss: 0.0786 - val_accuracy: 0.9800\n",
            "Epoch 8/10\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.0199 - accuracy: 0.9995 - val_loss: 0.0726 - val_accuracy: 0.9849\n",
            "Epoch 9/10\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.0145 - accuracy: 0.9995 - val_loss: 0.0828 - val_accuracy: 0.9805\n",
            "Epoch 10/10\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.0110 - accuracy: 1.0000 - val_loss: 0.0725 - val_accuracy: 0.9829\n"
          ]
        }
      ],
      "source": [
        "from tensorflow import keras\n",
        "from tensorflow.keras import layers\n",
        "\n",
        "def get_model():\n",
        "    model = keras.Sequential([\n",
        "        layers.Dense(512, activation=\"relu\"),\n",
        "        layers.Dense(10, activation=\"softmax\")\n",
        "    ])\n",
        "    model.compile(optimizer=\"rmsprop\",\n",
        "                  loss=\"sparse_categorical_crossentropy\",\n",
        "                  metrics=[\"accuracy\"])\n",
        "    return model\n",
        "\n",
        "model = get_model()\n",
        "history_noise = model.fit(\n",
        "    train_images_with_noise_channels, train_labels,\n",
        "    epochs=10,\n",
        "    batch_size=128,\n",
        "    validation_split=0.2)\n",
        "\n",
        "model = get_model()\n",
        "history_zeros = model.fit(\n",
        "    train_images_with_zeros_channels, train_labels,\n",
        "    epochs=10,\n",
        "    batch_size=128,\n",
        "    validation_split=0.2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Vx3EJaPaTk3K"
      },
      "source": [
        "**검증 정확도 비교 그래프 그리기**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T01:02:25.507068Z",
          "iopub.status.busy": "2022-05-21T01:02:25.506610Z",
          "iopub.status.idle": "2022-05-21T01:02:26.352048Z",
          "shell.execute_reply": "2022-05-21T01:02:26.351089Z"
        },
        "id": "FZnn2Uw4Tk3L",
        "outputId": "7f853610-bd0b-4827-dd01-7485ec300b22",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 472
        }
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkAAAAHHCAYAAABXx+fLAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAACMvElEQVR4nO3dd1hT59sH8G/YIEMUZDgAKW7FKmrVIg5anBWrdQvOto66aqtWratW26p1r/6sWpx11m0Rt3UrjrpwW0RxgqAMyfP+8bwJRIKAAgfI93NduUienJxzn5NAbp6pEkIIEBERERkQI6UDICIiIsprTICIiIjI4DABIiIiIoPDBIiIiIgMDhMgIiIiMjhMgIiIiMjgMAEiIiIig8MEiIiIiAwOEyAiIiIyOEyAKENxcXHo3bs3nJ2doVKpMHjwYADAgwcP0K5dOxQvXhwqlQozZsxQNM7syOic8sK+ffugUqmwb9++PDtmVixduhQqlQonT55UOpQcoTmfW7duKR2KYtzd3dG9e3ft4+x89ho2bIiGDRvmaDzjxo2DSqXK0X0SvSsTpQOgvLV06VL06NEjw+ePHDmCDz74AADw448/YunSpRgzZgw8PT1RsWJFAMCQIUOwa9cujB07Fs7OzvDx8cnxOH/88UdUqlQJgYGBOb5ffedERO/mxYsX+Pnnn3MlgSLKDUyADNSECRPg4eGRrvy9997T3t+zZw8++OADjB07VmebPXv2oHXr1hg2bFiuxffjjz+iXbt2OZ4AZXROeaFBgwZ4+fIlzMzM8vzYZNjy4rP34sULjB8/HgDSJUCjR4/GiBEjcu3YRG+DCZCBatasWaY1N9HR0ahUqZLe8qJFi+ZSZLkro3PKC0ZGRrCwsFDk2GTYlP7smZiYwMSEXzeZSUhIgJmZGYyM2DslL/AqUzqa/gI3b97Etm3boFKpoFKptH0rhBCYO3eutlzj2bNnGDx4MEqXLg1zc3O89957+Omnn6BWq3X2r1arMXPmTFStWhUWFhZwdHRE06ZNtX1QVCoV4uPjsWzZMu0x0vZn0Cc6Ohq9evWCk5MTLCws4O3tjWXLlmV6Tm/qJ6JSqTBgwABs2rQJVapUgbm5OSpXroydO3em2/bMmTNo1qwZbG1tYW1tjSZNmuDo0aN6r2vafhgRERFo27YtnJ2dYWFhgVKlSqFjx46IiYnRee3y5ctRs2ZNWFpaolixYujYsSPu3r37xmuiERkZiV69esHV1RXm5ubw8PBA3759kZSUpLNdYmIihg4dCkdHRxQpUgRt2rTBw4cPdbb566+/0KJFC+2+PD09MXHiRKSkpOhs17BhQ1SpUgUXL15Eo0aNYGVlhZIlS+Lnn3/We03+/PNPTJo0CaVKlYKFhQWaNGmCa9eupTuXY8eOoWnTprCzs4OVlRX8/Pxw+PDhTK/ByZMnERAQAAcHB1haWsLDwwM9e/bM0vWbN28eKleuDHNzc7i6uqJ///549uzZW52vPlWqVEGjRo3SlavVapQsWRLt2rXTlk2dOhX16tVD8eLFYWlpiZo1a2LdunWZHiOjPkCLFi2Cp6cnLC0tUbt2bRw8eDDda5OSkvD999+jZs2asLOzQ5EiReDr64u9e/dqt7l16xYcHR0BAOPHj9f+fo0bNw6A/j5Ar169wsSJE+Hp6Qlzc3O4u7vju+++Q2Jios527u7uaNmyJQ4dOoTatWvDwsICZcuWxR9//JHpeQPZu2bLly9H7dq1YWVlBXt7ezRo0AB///23zjY7duyAn58fbGxsYGtri1q1amHlypU68er7e/V606DmPVm9ejVGjx6NkiVLwsrKCrGxsXjy5AmGDRuGqlWrwtraGra2tmjWrBnOnj2bbr8JCQkYN24cypUrBwsLC7i4uODTTz/F9evXIYSAu7s7Wrdurfd1dnZ2+OKLL7J0HQslQQZlyZIlAoDYvXu3ePjwoc7t0aNHQggh7t+/L0JCQoSDg4OoXr26CAkJESEhIeLChQsiJCREABAfffSRtlwIIeLj40W1atVE8eLFxXfffScWLFgggoKChEqlEoMGDdKJoXv37gKAaNasmZgxY4aYOnWqaN26tZg9e7YQQoiQkBBhbm4ufH19tcf4559/MjynFy9eiIoVKwpTU1MxZMgQMWvWLOHr6ysAiBkzZrzxnOLi4jLcLwDh7e0tXFxcxMSJE8WMGTNE2bJlhZWVlfZaCSHEhQsXRJEiRbTbTZkyRXh4eAhzc3Nx9OhR7XZ79+4VAMTevXuFEEIkJiYKDw8P4erqKn744Qfxv//9T4wfP17UqlVL3Lp1S/u6H374QahUKtGhQwcxb948MX78eOHg4CDc3d3F06dP3/h+R0ZGCldXV2FlZSUGDx4sFixYIMaMGSMqVqyofa3mM/H++++Lxo0bi9mzZ4uvv/5aGBsbi/bt2+vsLzAwULRv31788ssvYv78+eKzzz4TAMSwYcN0tvPz8xOurq6idOnSYtCgQWLevHmicePGAoDYvn17umvy/vvvi5o1a4pff/1VjBs3TlhZWYnatWvr7DMsLEyYmZmJunXrimnTpolff/1VVKtWTZiZmYljx45pt9Ocz82bN4UQQjx48EDY29uLcuXKiV9++UX89ttvYtSoUaJixYpvvHZCCDF27FgBQPj7+4vZs2eLAQMGCGNjY1GrVi2RlJSU7fPVZ8KECcLIyEhERUXplO/fv18AEGvXrtWWlSpVSvTr10/MmTNHTJ8+XdSuXVsAEFu3btV5rZubmwgODk53nTWfPSGE+N///icAiHr16olZs2aJwYMHi6JFi4qyZcsKPz8/7XYPHz4ULi4uYujQoWL+/Pni559/FuXLlxempqbizJkzQggh4uLixPz58wUA0aZNG+3v19mzZ3WuY1rBwcECgGjXrp2YO3euCAoKEgBEYGBgunMpX768cHJyEt99952YM2eOqFGjhlCpVOLChQtvvLbZuWbjxo3TXo9ffvlFzJw5U3Tu3FkMHz5cu82SJUuESqUSVapUEZMmTRJz584VvXv3Ft26dcvw2mv4+fnpXFfNe1KpUiVRvXp1MX36dDF58mQRHx8vTpw4ITw9PcWIESPEwoULxYQJE0TJkiWFnZ2diIyM1O7j1atXokmTJgKA6Nixo5gzZ46YPHmyaNy4sdi0aZMQQohRo0YJU1NT8fjxY514/vzzTwFAHDhwINNrWFgxATIwmi8HfTdzc3Odbd3c3ESLFi3S7QOA6N+/v07ZxIkTRZEiRcTVq1d1ykeMGCGMjY3FnTt3hBBC7NmzRwAQAwcOTLdftVqtvV+kSBG9f0T0mTFjhgAgli9fri1LSkoSdevWFdbW1iI2NjbTc9IHgDAzMxPXrl3Tlp09e1YA0CZrQsikwMzMTFy/fl1bdu/ePWFjYyMaNGigLXv9S+jMmTPpvuBed+vWLWFsbCwmTZqkU37+/HlhYmKSrvx1QUFBwsjISJw4cSLdc5rrrflM+Pv767wHQ4YMEcbGxuLZs2fashcvXqTbzxdffCGsrKxEQkKCtszPz08AEH/88Ye2LDExUTg7O4u2bdtqyzTXpGLFiiIxMVFbPnPmTAFAnD9/Xhurl5eXCAgI0InxxYsXwsPDQ3z00UfastcToI0bNwoAeq/Bm0RHRwszMzPx8ccfi5SUFG35nDlzBADx+++/Z/t89bly5Uq6z5QQQvTr109YW1vrXPPXr39SUpKoUqWKaNy4sU55ZglQUlKSKFGihKhevbrOdV+0aJEAoPNF/erVK51thBDi6dOnwsnJSfTs2VNb9vDhQwFAjB07Nt05vp4AhYeHCwCid+/eOtsNGzZMABB79uzROZfXv6ijo6OFubm5+Prrr9Md63VZuWYRERHCyMhItGnTRue9FiL19+TZs2fCxsZG1KlTR7x8+VLvNpp4s5MAlS1bNl2MCQkJ6eK4efOmMDc3FxMmTNCW/f777wKAmD59errjaWLSfL7mz5+v8/wnn3wi3N3ddWI3NGwCM1Bz585FaGiozm3Hjh1vvb+1a9fC19cX9vb2ePTokfbm7++PlJQUHDhwAACwfv16qFQqvZ2Q33aY7Pbt2+Hs7IxOnTppy0xNTTFw4EDExcVh//79b3dSAPz9/eHp6al9XK1aNdja2uLGjRsAgJSUFPz9998IDAxE2bJltdu5uLigc+fOOHToEGJjY/Xu287ODgCwa9cuvHjxQu82GzZsgFqtRvv27XWuq7OzM7y8vHSaIV6nVquxadMmtGrVSm9/r9ev9+eff65T5uvri5SUFNy+fVtbZmlpqb3//PlzPHr0CL6+vnjx4gUuX76ssz9ra2t07dpV+9jMzAy1a9fWXru0evToodNB19fXFwC024aHhyMiIgKdO3fG48ePtdchPj4eTZo0wYEDB9I1tWpo+qtt3boVycnJerfRZ/fu3UhKSsLgwYN1+mT06dMHtra22LZt21ufb1rlypVD9erVsWbNGm1ZSkoK1q1bh1atWulc87T3nz59ipiYGPj6+uL06dNZPi9ANglGR0fjyy+/1Lnu3bt3134uNYyNjbXbqNVqPHnyBK9evYKPj0+2j6uxfft2AMDQoUN1yr/++msASHdtK1WqpP1MAICjoyPKly+f6bUFsnbNNm3aBLVaje+//z5d/xvN70RoaCieP3+OESNGpOtP9S5D/IODg3ViBABzc3NtHCkpKXj8+DGsra1Rvnx5nbjXr18PBwcHfPXVV+n2q4mpXLlyqFOnDlasWKF97smTJ9ixYwe6dOli0NMTsFeagapdu3aODl+PiIjAuXPntP0AXhcdHQ0AuH79OlxdXVGsWLEcO/bt27fh5eWV7g+XZoh72i/w7CpTpky6Mnt7ezx9+hQA8PDhQ7x48QLly5dPt13FihWhVqtx9+5dVK5cOd3zHh4eGDp0KKZPn44VK1bA19cXn3zyCbp27ar9EoqIiIAQAl5eXnrjMzU1zTD2hw8fIjY2FlWqVHmrc7W3twcA7bkCwL///ovRo0djz5496RK71/stlSpVKt0fV3t7e5w7dy7bx46IiAAgvywyEhMTo31dWn5+fmjbti3Gjx+PX3/9FQ0bNkRgYCA6d+4Mc3PzDPen+dy8/t6amZmhbNmy6T5X2Tnf13Xo0AHfffcdIiMjUbJkSezbtw/R0dHo0KGDznZbt27FDz/8gPDwcJ2+Mtn9EtPE/vrnytTUVCeR11i2bBmmTZuGy5cv6ySR+kaSZvX4RkZGOqNOAcDZ2RlFixZNd20z+z18k6xcs+vXr8PIyOiNAySuX78OAFn+fcoqfddQ009y3rx5uHnzpk4fu+LFi+vEVL58+Uw7mAcFBWHAgAG4ffs23NzcsHbtWiQnJ6Nbt245dyIFEBMgyhFqtRofffQRvv32W73PlytXLo8jyhnGxsZ6y4UQObL/adOmoXv37vjrr7/w999/Y+DAgZg8eTKOHj2KUqVKQa1WQ6VSYceOHXpjsba2zpE4gMzP9dmzZ/Dz84OtrS0mTJgAT09PWFhY4PTp0xg+fHi6GpjsXLvMttXs+5dffkH16tX1bpvRtVCpVFi3bh2OHj2KLVu2YNeuXejZsyemTZuGo0eP5tg1fJfPSocOHTBy5EisXbsWgwcPxp9//gk7Ozs0bdpUu83BgwfxySefoEGDBpg3bx5cXFxgamqKJUuW6HTCzWnLly9H9+7dERgYiG+++QYlSpSAsbExJk+erE0K3lZWE7e3vbZKXLOMziklJUXvebxe+wPIaUDGjBmDnj17YuLEiShWrBiMjIwwePDgDGs636Rjx44YMmQIVqxYge+++w7Lly+Hj4+P3n/cDAkTIMoRnp6eiIuLg7+/f6bb7dq1C0+ePHljLVB2/qN1c3PDuXPnoFardWqBNE0ybm5uWd5Xdjk6OsLKygpXrlxJ99zly5dhZGSE0qVLv3EfVatWRdWqVTF69Gj8888/qF+/PhYsWIAffvgBnp6eEELAw8Mj20mko6MjbG1tceHChWy9LiP79u3D48ePsWHDBjRo0EBbfvPmzRzZ/5tomiFtbW0z/Yxl5IMPPsAHH3yASZMmYeXKlejSpQtWr16N3r17691e87m5cuWKTq1IUlISbt68+dZx6OPh4YHatWtjzZo1GDBgADZs2IDAwECdGqr169fDwsICu3bt0ilfsmRJto+nObeIiAg0btxYW56cnIybN2/C29tbW7Zu3TqULVsWGzZs0Pm9fL0ZO7u/s2q1GhERETqTkT548ADPnj3Lsd/ZrF4zT09PqNVqXLx4McMEW/MZvHDhQrqaq7Ts7e3TjRIEZK2Xvto1fdatW4dGjRph8eLFOuXPnj2Dg4ODTkzHjh1DcnLyG2uDixUrhhYtWmDFihXo0qULDh8+XKBm8M8t7ANEOaJ9+/Y4cuQIdu3ale65Z8+e4dWrVwCAtm3bQgihnTAtrbT/zRUpUkTvHxF9mjdvjvv37+v0oXj16hVmz54Na2tr+Pn5ZfNsss7Y2Bgff/wx/vrrL50h9Q8ePMDKlSvx4YcfwtbWVu9rY2NjtddFo2rVqjAyMtJW1X/66acwNjbG+PHj0/23K4TA48ePM4zNyMgIgYGB2LJli95lLrJbi6X57zXt65KSkjBv3rxs7edt1KxZE56enpg6dSri4uLSPf/6cP20nj59mu5cNV9yrw+5Tsvf3x9mZmaYNWuWzusXL16MmJgYtGjRIptn8WYdOnTA0aNH8fvvv+PRo0fpmr+MjY2hUql0mkNu3bqFTZs2ZftYPj4+cHR0xIIFC3SmQ1i6dGm63zt97/uxY8dw5MgRne2srKwAIEu/t82bNweAdF/C06dPB4Acu7ZZvWaBgYEwMjLChAkT0tWwaM77448/ho2NDSZPnoyEhAS92wAyKTl69KjOdd26dWuWp63QxP36Z3bt2rWIjIzUKWvbti0ePXqEOXPmpNvH66/v1q0bLl68iG+++QbGxsbo2LFjluMprFgDZKB27NiRrtMqANSrVy/L/6Wk9c0332Dz5s1o2bIlunfvjpo1ayI+Ph7nz5/HunXrcOvWLTg4OKBRo0bo1q0bZs2ahYiICDRt2hRqtRoHDx5Eo0aNMGDAAADyC2/37t2YPn06XF1d4eHhgTp16ug99ueff46FCxeie/fuOHXqFNzd3bFu3Trtfzk2NjbZPp/s+OGHHxAaGooPP/wQ/fr1g4mJCRYuXIjExMQ3zgOzZ88eDBgwAJ999hnKlSuHV69eISQkBMbGxmjbti0A+cf0hx9+wMiRI3Hr1i0EBgbCxsYGN2/exMaNG/H555+/cUbuH3/8EX///Tf8/Pzw+eefo2LFioiKisLatWtx6NChbE1oWa9ePdjb2yM4OBgDBw6ESqVCSEhIjjUHvomRkRH+97//oVmzZqhcuTJ69OiBkiVLIjIyEnv37oWtrS22bNmi97XLli3DvHnz0KZNG3h6euL58+f47bffYGtrq/0i1sfR0REjR47E+PHj0bRpU3zyySe4cuUK5s2bh1q1aul0eM4J7du3x7BhwzBs2DAUK1YsXQ1TixYtMH36dDRt2hSdO3dGdHQ05s6di/feey9L/YzSMjU1xQ8//IAvvvgCjRs3RocOHXDz5k0sWbIk3e9/y5YtsWHDBrRp0wYtWrTAzZs3sWDBAlSqVEknGbW0tESlSpWwZs0alCtXDsWKFUOVKlX09pnx9vZGcHAwFi1apG1aPX78OJYtW4bAwEC98yK9jaxes/feew+jRo3CxIkT4evri08//RTm5uY4ceIEXF1dMXnyZNja2uLXX39F7969UatWLXTu3Bn29vY4e/YsXrx4oZ13rHfv3li3bh2aNm2K9u3b4/r161i+fLnOYIrMtGzZEhMmTECPHj1Qr149nD9/HitWrEj33gQFBeGPP/7A0KFDcfz4cfj6+iI+Ph67d+9Gv379dOb/adGiBYoXL461a9eiWbNmKFGixDte3UIgL4eckfLeNAwegFiyZIl22+wMgxdCiOfPn4uRI0eK9957T5iZmQkHBwdRr149MXXqVJ05U169eiV++eUXUaFCBWFmZiYcHR1Fs2bNxKlTp7TbXL58WTRo0EBYWloKAJkOiX/w4IHo0aOHcHBwEGZmZqJq1ao655LZOemT0XnqG+Z6+vRpERAQIKytrYWVlZVo1KhRurmLXh+KfOPGDdGzZ0/h6ekpLCwsRLFixUSjRo3E7t270x1z/fr14sMPPxRFihQRRYoUERUqVBD9+/cXV65cyfQ8bt++LYKCgoSjo6MwNzcXZcuWFf3799cObdZ8Jl4fJq5v7pjDhw+LDz74QFhaWgpXV1fx7bffil27dqXbzs/PT1SuXDldLMHBwcLNzS3dMV6fCuDmzZvpPo9CyKkDPv30U1G8eHFhbm4u3NzcRPv27UVYWJh2m9eHwZ8+fVp06tRJlClTRpibm4sSJUqIli1bipMnT2Z67YSQw94rVKggTE1NhZOTk+jbt2+6+Zeyer6ZqV+/vt7h4RqLFy8WXl5ewtzcXFSoUEEsWbJE7xw7WZkHSAgh5s2bp52zysfHRxw4cCDdcG21Wi1+/PFH4ebmJszNzcX7778vtm7dqvfc/vnnH1GzZk1hZmamMyReX4zJycli/PjxwsPDQ5iamorSpUuLkSNH6kynoDkXfb+zr8eZkaxeMyHksPL3339fmJubC3t7e+Hn5ydCQ0N1ttm8ebOoV6+esLS0FLa2tqJ27dpi1apVOttMmzZNlCxZUpibm4v69euLkydPZjgMXt80GAkJCeLrr78WLi4uwtLSUtSvX18cOXJE7zm/ePFCjBo1SnsdnZ2dRbt27XSm5dDo16+fACBWrlyZ6XUzBCoh8uDfNyIiIlLUkCFDsHjxYty/f1/bZGnI2AeIiIiokEtISMDy5cvRtm1bJj//j32AiIiICqno6Gjs3r0b69atw+PHjzFo0CClQ8o3mAAREREVUhcvXkSXLl1QokQJzJo1K8Nh/oaIfYCIiIjI4LAPEBERERkcJkBERERkcNgHSA+1Wo179+7BxsbGoFfKJSIiKkiEEHj+/DlcXV3TLZD9OiZAety7dy/T9ZuIiIgof7p79y5KlSr1xm2YAOmhWTrh7t27Ga7jRERERPlLbGwsSpcunaUlkJgA6aFp9rK1tWUCREREVMBkpfsKO0ETERGRwWECRERERAaHCRAREREZHCZAREREZHCYABEREZHBYQJEREREBocJEBERERkcJkBERERkcJgAERERkcFhAkREREQGhwkQERERGRwmQERERGRwmAAREVGhJgQQG6t0FJTfMAEiIqJC7cMPATs7wNsbmDABuHhR6YgoP2ACREREhcbjx8DixYBanVrm4yN/njsHjB0LVK4MVKwIjBkDhIfLGiIyPCoh+Na/LjY2FnZ2doiJiYGtra3S4RAR0RskJgJbtwIhIcD27UByMrBnD9CokXz+4UMgIQEICwPWrQNCQ4GkJPlcpUrAv/8qFzvlrOx8f5vkUUxEREQ5Rq0GDh+WSc/atcCzZ6nPVa8ukyANR0f5s3t3eYuJkQnT+vVArVqp2714IR9//DHQrh1Qty5gxHaSQos1QHqwBoiIKH87dgz44IPUx6VKAV26AF27AlWqvN0+N24EPv009bGLi3zcti3g6wuYsMog38vO9zcTID2YABER5R8PHwKrV8umrmHDZJkQQM2asmNzt26Anx9gbPxux3n5Evj7b9lMtnmz7sgxR0fgjz+Apk3f7RiUu5gAvSMmQEREynr5EtiyRTZx7dwJvHoF2NsDUVGAubncRghApcqd4ycmpvYZ+usv4MkTICICeO89+fyRI7LM3z81HlIe+wAREVGBdPQo8L//yX49aWtgataUNT0pKalluZX8ADKpad5c3pKTZZObJvkBgF9+kU1mtrZAq1aymaxpU8DSMvdiKqiEAG7dkqPwzp6VPz/6CPjiC2XjYgJERESKSluTs2WLHMYOAGXKyD49XbvKYetKMTWVcwml5eUFuLoC9+4BK1bIW5EiMmH67DN5M0TPn8uasho1Usv8/eWovLSMjJgAERGRAXrwAFi1Cli+XM7N06qVLA8Kks916yY7HufXUVg//QRMnixrrNavl01ld+7Imqu7d3UToJcvC1/NkFoN3LypW6tz9ixw44Z8z+LjAQsLue177wGHDskpB7y9gWrV5Ag7pbEPkB7sA0RElPNevJD9aUJCZGdjTXNW+/bAmjXKxvauhABOnZKJUMWKQHCwLH/6FChdGmjYUDaTtW4NFCumaKjZFhsLnD8P1KmTOhKuVy/g99/1b+/qCuzfn9pk+OyZrB0zNc2LWNkHiIiI8omEBODLL2VNSVxcanmdOrJ5q0MH5WLLKSqVnHFaM+u0RliYrA3Ztk3eTEzkBI3t2gGBgUCJEoqEq5daDVy/Lmtz0tbs3Lwpn794MbUpsmJF2U+qcmVZo6Op2alWDXBw0N1v0aJ5ehpZxhogPVgDRET0bu7dkzUBGtWryy9UD4/Ufj3lyikWXp66eDG1mezcudRyIyNZ1qZN3scUEyNjqVo1NUGZOBH4/nv925csKfs5+fnJxy9eAGZm+W9uJNYAERFRnouKAlaulE1cERGyL4+1tXzul19kP5j69XN39FZ+VKmSvI0ZI6/L+vXydvq0bl+YzZtlDUzbtrIDeE5ISZH71NTmaGp2bt9OPaam/1WVKrLfTuXKqTU63t4ySSpeXHe/VlY5E5+SWAOkB2uAiIiyJj5eDgcPCQF2705dhNTUVM7f07ixsvHlZ/fvA87OqY/9/WWTGSCX5GjXTiZDnp5Z29/Tp7KvjoeH7HcEyIS0Sxf925cuDfz8M9Cxo3ycnCyT0/xWq5Md2fn+Vrx//dy5c+Hu7g4LCwvUqVMHx48fz3Db5ORkTJgwAZ6enrCwsIC3tzd27typs01KSgrGjBkDDw8PWFpawtPTExMnTgTzPCKinLVlC+DkJEds/f23TH7q1QPmz5df7kx+3ixt8gPIZTcaNJBJyIkTwPDhsiPx++/LUWcaKSnA5cvAn38Co0bJGpwyZWTnaj8/2aymUa2arHmrVUt2XJ41C9i3T07ieOdOavIDyKS1ICc/2SYUtHr1amFmZiZ+//138e+//4o+ffqIokWLigcPHujd/ttvvxWurq5i27Zt4vr162LevHnCwsJCnD59WrvNpEmTRPHixcXWrVvFzZs3xdq1a4W1tbWYOXNmluOKiYkRAERMTMw7nyMRUWGgVgtx5owQp06llt2+LQQghKenEOPGCXHtmmLhFSpRUULMny+Ev78QxsbyGn/8cerzx47JMn03NzchZsxI3TYlRYhXr/L8FBSTne9vRZvA6tSpg1q1amHOnDkAALVajdKlS+Orr77CiBEj0m3v6uqKUaNGoX///tqytm3bwtLSEsuXLwcAtGzZEk5OTlismUlLzzaZYRMYEZH033+y8+vy5cCFC0CzZsD27anPX7gg+4wYWr+evPLokeyn4+qaug5ZfLxcqLVSJd0RWGk7NBuqAtEJOikpCadOncLIkSO1ZUZGRvD398eRI0f0viYxMREWmpmV/p+lpSUOHTqkfVyvXj0sWrQIV69eRbly5XD27FkcOnQI06dPzzCWxMREJCYmah/Hpp1/nYjIwOzZA+zaBezdC5w8KesWADnqx9ZWNnVpJih825XXKWscHICePXXLihSRc+vk10kiCwrFEqBHjx4hJSUFTk5OOuVOTk64fPmy3tcEBARg+vTpaNCgATw9PREWFoYNGzYgJc3iMCNGjEBsbCwqVKgAY2NjpKSkYNKkSeiSUS8wAJMnT8b48eNz5sSIFBAfLztAFi9e+Gacpdz14oWcwM/XN7Xsl19kB2YNX1/Zz6ddO7kgKSmPyc+7K1DdnWbOnIk+ffqgQoUKUKlU8PT0RI8ePfB7muko//zzT6xYsQIrV65E5cqVER4ejsGDB8PV1RXBmqk5XzNy5EgMHTpU+zg2NhalNV3oifKxyEjg11+BhQtTJ5j77DPZORKQ/7m3by+rxYsXl7dixVJ/li4tR4yQ4UhIkMs37N0rb0ePytE///0n53oBZKLj4iIn7GvcOLWcqDBRLAFycHCAsbExHjx4oFP+4MEDOL/eNf7/OTo6YtOmTUhISMDjx4/h6uqKESNGoGzZstptvvnmG4wYMQId/79re9WqVXH79m1Mnjw5wwTI3Nwc5ubmOXRmRLkvIkIOX/3jDyApSZYZGcmmibTN3nFxuiNCXtemDbBhg7wvhFzg0dZWN0nS/KxcGQgISH3t48eAnZ2BjRopwLZtA6ZOBY4cAdK0+AOQifDt26mJTq9e8kZUmCn2p8vMzAw1a9ZEWFgYAgMDAchO0GFhYRgwYMAbX2thYYGSJUsiOTkZ69evR/v27bXPvXjxAkav1Q0aGxtDrZmcgqiAE0ImLv/+Kx/7+gIjRsgOkvHxqesrAYCxMTB3rkxWnjyRP9Ped3NL3TYuTk6YlpE2bVITICHkEN5Xr2QS9Hrt0gcfAF99lfrasDDAxib1eTs7VuHnluRk2W9n71657lTlyrL8+XM5/BlIrd3R3MqWZSdmMjyK/u82dOhQBAcHw8fHB7Vr18aMGTMQHx+PHj16AACCgoJQsmRJTJ48GQBw7NgxREZGonr16oiMjMS4ceOgVqvx7bffavfZqlUrTJo0CWXKlEHlypVx5swZTJ8+HT1f70VGVEAIARw4ANSuLfv3qFTAt9/KVadHjJAz62rY2Oi+1soK6Ncva8exsJDNIZrk6PWftWunbhsXJ5MfQE6pHxMjV4HWSEhITYCEkMmZZntAJj/29jIZ8veXSZrG7NmyJsLLS86Bwj5Nb/bqFXDmTGqT1qFDqc2hKlVqAtSkCTBvnmzSKleOCQ+RoglQhw4d8PDhQ3z//fe4f/8+qlevjp07d2o7Rt+5c0enNichIQGjR4/GjRs3YG1tjebNmyMkJARF04z7mz17NsaMGYN+/fohOjoarq6u+OKLL/B9RgucEOVTajWwdSswebJMTObOTU1mgoLkLSeZmsrFKbPCxkbWNDx9mr5W6ckTWaOg8fKlHJ6r2S4+Xp6b5nHaUURCAEOG6NZilS4tkyEvL1nb9YbxDAbn0iVZ2/b6wNVixeTq45rkBwAcHYG+ffM0PKJ8jUth6MF5gEhJycnA6tVy5ldNM5e5uZzxdcwYZWPLCYmJMknSJEw2NnKmW0D2Z+rRQ/ZxioiQQ33TStvBW62WX/BpE6Ry5eRPd3eZ0BUGarWca2ffPlnDU6GCTIoBeb3s7eW5+vmlNmlVrcomRjJM2fn+ZgKkBxMgUoIQsonil19SFyq0tZW1PoMHyyUHDIkQMkHSJEMREbK2qEMH+fydO7p9mNIyNga++CK1aU0IIDRUJkdlysjn87OLF1ObtPbvl5PhaZQvL5dB0IiIkDVu+f2ciPJCgZgIkYh0qVRy8rnbt4ESJWTS06+f7DBsiFQqOQmcg4PuitkaJUrIvlFpE6SrV4Fr12SzW9r+UPfupXbgNjOTCYOm1sjLS65fVa1a3pzX64SQ62a5uKSWffKJbof0IkWADz+UzVqNGum+3ssrT8IkKnSYABEp5P59YMYMmeSUKSPLvv9edhju0YOdfzNjYSH7BKWdwA+QTUb37ukOz3/6VC4bcP26bIK7fFm3FmXYMFnzBgAPH8rao7QJkpeXTFByouOwEMDNm6k1PHv3yhFaT56kxtyihWz+1DRp1apVeJr0iPILJkBEeez6dfllu3Sp/DJ++RKYOVM+5+Mjb/T2jIyAUqV0y6pUkQlFSoqc8O/qVd2ao7Qj3C5fBjZuTL/fIkXkqLQhQwDNlGJJSbKfkqNj5snR338DK1fKhOfOHd3nzMxkHBUryseazwMR5R4mQER55OxZ2bF5zRpZSwHIppdmzZSNy5AYG8t+Q25uwEcf6d/G3V0mIGkTpFu35Oi1s2fl0hEaJ0/KaQjs7NLXGAFA8+ZyRBYgR/ItWybvm5jIEXeaGp66dVnjR5TXmAAR5YGOHWXio9GsmZzDx9eX87HkN6VLAwMH6pYlJclmq4gIOcJK47//5PsXEyOToZMndV+3dq1cVgIAWrWStX2NGsmkqUiR3D0PInozJkBEuUAztlKT3JQqJZtm2rcHhg8HqldXLDR6C2ZmcvRV+fK65e3by8Tmxg3dGqOICDkZZNqRWe+/nzrcn4iUx2HwenAYPL2tV6/kPDVTpgBz5gANGsjy6Gg5Wd177ykbHxFRYZad729OlUWUAxISgPnz5UR8XboA58/LEV4aJUow+SEiyk/YBEb0DmJiZOIzYwbw4IEsc3BIncOHiIjyJyZARO/A3z+142uZMnI+mV695CKkRESUfzEBIsqGW7fkhHjm5vLx55/LYdHDhwOdOnGyOiKigoJ9gIiy4MIFoGtX2Y/njz9Sy3v0kP19goKY/BARFSRMgIje4J9/5DDnqlWBFSvkTMJp53oxMeGq20REBRH/dBPpsXMn4OcnJ6zbulXO59OunUx+Fi5UOjoiInpX7ANEpMeMGXKlcVNT2bz17bdyiDsRERUOTIDI4CUmyn49LVoArq6ybPRooHJlYOhQoGRJZeMjIqKcxwSIDE5sLHDtmlyu4MIFYPFiICpKDmH/5Re5zYcfyhsRERVOTICoUIqPl0mOWp26/lJiolwFXDNhYVqlSnGmZiIiQ8IEiAo0IYC//kq/EGVkpHy+cWMgLEzeNzdPXZyyRAnAy0veGjaUc/iYmSlyCkREpAAmQJSvJScDN28CV6+mJjd2dsDkyfJ5lQr48kv9tTr29oCNjW7Z3r2Ak5PcBxERGS4mQKS4V6+AR48AZ+fUsi5dgGPH5MzLKSm627u7pyZAAPDJJ8Dz56k1Oppb8eLpj8WRXEREBDABojx09y5w+XL65qqbN+U6WteupW574wZw/bq8b2UlE5py5eTP8uV197toUd6dAxERFQ5MgCjHCAHcu5ea2Dx+DIwYkfp827bAiRP6X/vggawJMvn/T+TkybJ5y8tLrr2lUuV+/EREZDiYANE7mTcP2LNHJjzXrsmFQTVMTOTQck1SU6WK/qYqLy85CivtkhING+bpaRARkYFhAkTv5MABYP361MfGxrKPjqbJ6uXL1I7Iv/+uSIhERETpMAGibHn0SDZ1OTrKx127Ah98kFqT4+HBVdGJiCj/YwJEWSYE0KOH7MezerVspmrZUumoiIiIso8JEGXZwoVyZXQzM6BYMaWjISLKnFoNbNoE/P23nBqjVCm5vl+pUvLm4KDb/5AMBxMgypLLl+XCoAAwZQpQrZqy8RARvUlSErBiBfDTT8CVKxlvZ2YmE6K0SVHaW8mSco4yE35bFjp8SylTSUlyYsKXLwF/f2DQIKUjIiLSLz4e+O03YNo04L//ZJmdHRAcLGuD/vsv9fbggfz7dvOmvGXEyEhOx/F6YpT2saurXG6HCg4mQJSpsWOB06dls9eyZawuJqL858kTYPZsYNYseR+QNTdDhwJffAHY2qZ/TXIyEBWlmxT9959cS1Bz/949OUdZZKS8HTuWcQwlSry5JqlUKaBIkdw5f8o+JkD0RgcPyipkQP5X5eqqbDxERGlFRgLTp8s+ivHxsqxsWeDbb2Wtj4VFxq81NZWz0Jcpk/E2KSlAdHT6xOj1W2Ki3C46GjhzJuP9FS365pqkUqVkjRUnf819KiGEUDqI/CY2NhZ2dnaIiYmBrb5/GwxIbCzw1Vey/XvxYqWjISKSrl4Ffv4Z+OMPWZMDAN7ecvb5du3yts+OELLWSV9ipEma7t4F4uKytj8rq4xrkcqUkZPKsk+Sftn5/mYCpAcToPTSLlNBRKSU06flUjnr18vEAwB8fYGRI4GmTfN3zUlsrP5mtrQ3TfPdm9jbAwEBQPPm8pw187IRE6B3xgRI/nfl5ZW//5gQkWEQAti/XyY+f/+dWt6ypazxqV9fudhy2suX+pMjTVlEBBATk7q9SgXUri2ToebNgRo1DLufJhOgd2ToCdCtW7Iq+cMPgZUrZXs0EVFeU6uBLVtk4qPpfGxsDHTsCAwfDlStqmx8Snj1Cjh+HNi2Ddi+HQgP133eyQlo1kwmQx99JPscGRImQO/IkBOglBQ5w/OhQ0C9evK/LjZ9EVFeSk4GVq2SAzAuXpRl5uZAr17A11/LTs4kRUYCO3fKhCg0VLefkbGxrB1r0UImRJUrF/5afSZA78iQE6BJk4DRo+UCpmfPyrW9iIjywosXcrDF1KnAnTuyzNYW6NcPGDxY1m5QxpKS5D+v27fLhOjyZd3nS5eWiVCLFkDjxoVzSD4ToHdkqAnQiROy1ufVKznfT1CQ0hERkSF4+hSYNw+YORN4+FCWlSgBDBkC9O3LZvi3deMGsGOHTIj27AESElKfMzOTtf2ahOi99xQLM0cxAXpHhpgAxcXJznMREUD79nKx08JeVUpEyoqKAn79FViwAHj+XJa5u8s5fLp3BywtlYyucHnxAti3L7V26NYt3ee9vFI7Uvv5FdxZrZkAvSNDTIC+/FJOJFaqFHDunBxmSUSUG65fl3P4LF0qm20AObfNiBFAhw7sd5jbhJDro2k6Uh84IGv+Nays5LJHzZvLDtVvmigyv2EC9I4MMQE6elQ2eS1cCDRqpHQ0RFQYhYfLjs1//ilHeAGy2X3kSNkMw1pnZcTGArt3y2Ro+3ZZM5dWlSqpHanr1pUzaOdXTIDekSEmQAAnOySinCeE7Jg7ebLsj6LRrJlMfHx9lYuN0hNCDoDR1A4dPZqarAKyP1baSRjzW8d0JkDvyFASILVatgNzSCkR5TQh5Jfo5MnAP//IMiMj2cdw+HCgenVFw6MsevxYTj65fbtMYB8/1n3exye1dsjHR/lJGJkAvSNDSYBmzpRt7jNmyNWSiYje1atXwJo1wJQpwIULsszMDOjRAxg2rPCMNjJEKSlytLCmqezUKd3nHR1lrVDz5rKWSIm+pEyA3pEhJEDnzwO1askVjOfOlfNsEFHOOHZM3pycUheydHXN330n3tXLl8CSJcAvv6SOMLK2lsPYhwwBXFwUDY9yQVSUnIRx+3ZZSxQbm/qckZHs36UZWVatWt708WIC9I4KewKUkCDXjjl/XlZdbtnCzodEOeHuXTmEe/Xq9M+pVLoJUcmS+lf7LmhDv2NigPnzZU3ygweyzMFBTlzYrx9HlBqK5GTg8OHU2qF//9V9vmTJ1GSoSRM52W5uYAL0jgp7AjR0qJx7w9FRJkH5rRMbUUHz8qWs+ZgyRd5XqWQTQHx86mKWyclZ21exYukTo9eTpvzwZ+nBA5n0zJuX+p9/mTLAN98APXvKodRkuG7fTk2GwsLk74WGqamca6h9e6BPn5w9LhOgd1SYE6DQUODjj+X9LVvkaspE9HaEANatk31bNEs3fPih7F9Xo0bqdmo18OhR+pW9X7+9eJG149rY6E+M0t6KFcudmt2bN+VSFb//njqzcKVKsj9hx46Fu5mP3k5CglxXUjMJ4/XrsrxDB/21pe+CCdA7KqwJ0JMncvXke/dku/y8eUpHRFRwnT0LDBok/7ADcp2lX36R/9W+TeIhhGxO0pcYpU2Ynj3L2v4sLDJOjjTlTk5ZH7Vz/rycw2f1atkZFgDq1JFD2Vu1Un70DxUMQsgVB7Zvl/ML+fvn7P6ZAL2jwpoApaSk/gE7epRV1ERv49EjYMwYYNEiWbNjYSH7/Qwfnje/U3FxqQlRRjVJmvW0MmNiIjtnZ9TUVqqU7Nc0ZQqwdWvq6z7+WCY+fn7sP0j5CxOgd1RYEyCNpCQ5LJWIsi45WdaajhuXWgvTvr1c0sHNTcnI0ktMlDW9b6pJiorSneAuMyoV0K6dbOpK27xHlJ9k5/ub8/4agKgoORLDwkI+ZvJDlD2hoXJU08WL8rG3t+zn4+enaFgZMjcHPDzkLSOvXgH377+5JikyUiY+QUGyc3O5cnl3DkS5jQlQIZecDLRpI0ejrF0LVKigdEREBce1a8DXXwObN8vHxYsDkyYBvXsDxsbKxvauTExSm7kyolbLpnN2bKbCiAlQIffDD3JCNjs79vkhyqrnz2Wi8+uvssnY2BgYMAAYO9aw5rUxMmLnZiq8mAAVYv/8IxMgAFiwQM7RQUQZU6uBkBDZz+X+fVn28ccyEapUSdnYiChnMQEqpGJjga5d5R/0rl3l/BxElLFjx4CBA4Hjx+VjT0+Z+LRsyZFORIURKzcLqYED5YRlbm7AnDlKR0OUf927Jzv5fvCBTH6sreV0Ef/+K+e3YfJDVDixBqgQWrsWWLZMtt0vXy77/xCRroQEuZTDDz/IQQIA0L078OOPXLiTyBAwASqEatcGGjQAfH3ltPxElEoIOapr6FDgxg1ZVqcOMGuW/N0hIsPABKgQcnMD9uyRf+iJKNW//8r5fHbvlo9dXGRzV5cuHO1EZGj4K1+IaBZjBOSwXROmt0QAgKdPZb84b2+Z/JiZyaUcrl4FunVj8kNkiBT/tZ87dy7c3d1hYWGBOnXq4LhmCIYeycnJmDBhAjw9PWFhYQFvb2/s3Lkz3XaRkZHo2rUrihcvDktLS1StWhUnT57MzdNQ3JkzgJeX/COfnKx0NET5w6tXwPz58ndj9mw5qV9gIHDpkuzrY22tdIREpBRFE6A1a9Zg6NChGDt2LE6fPg1vb28EBAQgOjpa7/ajR4/GwoULMXv2bFy8eBFffvkl2rRpgzNnzmi3efr0KerXrw9TU1Ps2LEDFy9exLRp02BfiGcve/FCVuEnJclaINb8EAH79gE1awL9+gGPHwOVK8slLTZuBMqWVTo6IlKaoouh1qlTB7Vq1cKc/x+nrVarUbp0aXz11VcYMWJEuu1dXV0xatQo9O/fX1vWtm1bWFpaYvny5QCAESNG4PDhwzh48OBbx1XQFkMdMACYOxdwdgbOnwccHJSOiEg5t27JdavWrZOP7e2BCROAL7/kPwdEhV12vr8VqwFKSkrCqVOn4O/vnxqMkRH8/f1x5MgRva9JTEyEhWZFz/9naWmJQ4cOaR9v3rwZPj4++Oyzz1CiRAm8//77+O23394YS2JiImJjY3VuBcX27TL5AYClS5n8kOGKjwe+/x6oWFEmP0ZGsvYnIkL+k8Dkh4jSUiwBevToEVJSUuDk5KRT7uTkhPuaOehfExAQgOnTpyMiIgJqtRqhoaHYsGEDoqKitNvcuHED8+fPh5eXF3bt2oW+ffti4MCBWLZsWYaxTJ48GXZ2dtpb6dKlc+Ykc1l0NNCjh7w/aBAQEKBsPERKEAJYuRIoXx6YOFHO79OwoewXN3euXMCUiOh1ineCzo6ZM2fCy8sLFSpUgJmZGQYMGIAePXrAKM0QDrVajRo1auDHH3/E+++/j88//xx9+vTBggULMtzvyJEjERMTo73dvXs3L07nnQghV6SOjgaqVAGmTFE6IqK8d+qUnO+qSxcgMhJwdwfWr5fTQFSrpnR0RJSfKZYAOTg4wNjYGA8ePNApf/DgAZydnfW+xtHREZs2bUJ8fDxu376Ny5cvw9raGmXT9Gh0cXFBpddWLaxYsSLupB0j/hpzc3PY2trq3PI7lUqu8eXsDKxYAbzWMkhUqD14IP8BqFULOHwYsLKSMzpfvAh8+imXryCizCmWAJmZmaFmzZoICwvTlqnVaoSFhaFu3bpvfK2FhQVKliyJV69eYf369WjdurX2ufr16+PKlSs621+9ehVubm45ewL5QPv2cr0v/qdLhiIpCZg2DShXDli8WNaEdu4MXLkCjBoFWFoqHSERFRSKdgscOnQogoOD4ePjg9q1a2PGjBmIj49Hj//v2BIUFISSJUti8uTJAIBjx44hMjIS1atXR2RkJMaNGwe1Wo1vv/1Wu88hQ4agXr16+PHHH9G+fXscP34cixYtwqJFixQ5x5yWlAQ8ewaUKCEfs+aHDMX27cCQIXLyQkAOcZ85E6hfX9m4iKhgUjQB6tChAx4+fIjvv/8e9+/fR/Xq1bFz505tx+g7d+7o9O9JSEjA6NGjcePGDVhbW6N58+YICQlB0aJFtdvUqlULGzduxMiRIzFhwgR4eHhgxowZ6NKlS16fXq74/nvg99/lYqfNmikdDVHuu3JFrtu1fbt8XKIEMHmyXLiUMzgT0dtSdB6g/Cq/zgO0bx/QuLGs9l+/XvZ1ICqsYmLkqK6ZM+WMzqamcrTj6NGAnZ3S0RFRfpSd72/OjFFAPH0KBAXJ5KdnTyY/VHilpMg5rb77To5yBIAWLYDp02XfHyKinMAEqAAQQk7odvcu4Okp/yMmKoyOHwf69gVOn5aPy5cHfv2Vzb1ElPOYABUAK1YAq1fLFd5XrOACjlQ4HToEfPSRnMjQ1hYYNw7o31+u3E5ElNOYAOVzt27JLwEAGDsWqFNH0XCIcsW5c0DLljL5CQgA/vgjdaQjEVFuYAKUzxUvDrRrB1y+DIwcqXQ0RDnvxg2Z9MTEAB9+CGzYICc2JCLKTUyA8jkbGznh28uXXMyRCp/792Wz1/37ckLPLVuY/BBR3uAsGvnU/fuy87MGZ7ilwubZM1nzc+MGULYssHMnkGZKLyKiXMUEKB+KiwMaNACaN08dBkxUmLx8CXzyiez74+QE/P034OKidFREZEjYqJIPDR0KRETILwlTU6WjIcpZyclAhw7AwYNyQsNdu+T0DkREeYk1QPnMpk3Ab7/J1az/+AOwt1c6IqKco1bLVdy3bJHr2G3ZAnh7Kx0VERkiJkD5SFSU/HIAgGHDgEaNlI2HKCcJIT/Xf/wh57Rauxbw9VU6KiIyVEyA8gm1Wi7u+PgxUL26XAOJqDCZMkXO6gzIBX1btlQ2HiIybEyA8onZs2VHUAsLYOVKwNxc6YiIcs5vv8m1vQC5pldQkLLxEBGxE3Q+Ubcu4OUlV7uuWFHpaIhyzvr1wJdfyvsjRwJDhigbDxERwAQo36hdGwgP53w/VLjs2QN07iybePv0ASZNUjoiIiKJTWAKe/Ag9b6VlRz9RVQYnDwJtG4NJCUBbdsC8+fz801E+QcTIAXt3g24uwMzZigdCVHOunwZaNZMTurZuDGwYoUc+UVElF8wAVLI48dAcLBc/frqVaWjIco5d+8CH38MPHoE+PjIua3YqZ+I8hsmQAoQAvjiC+DePaBCBWDqVKUjIsoZjx/L9b3u3gXKlwe2b5cL+hIR5TdMgBSwZIkcGWNqKpsGuPo1FQZxcXL9ukuXgFKl5LQOjo5KR0VEpB8ToDx27RowcKC8P3EiUKOGsvEQ5YTERODTT4Hjx4FixWTyU6aM0lEREWWMCVAeSk4GunYF4uMBPz+5LABRQZeSIic2DA0FihSRzV6cy4qI8jsmQHnIyAgIDJTNApr1kIgKMiGAAQOAP/+UTbobNgB16igdFRFR5pgA5SFjY2DECOD6dTYPUOEwdiywYIGc32f5cjn6i4ioIGACpACOiqHCYNas1EV7584F2rdXNh4iouxgAkRE2bZihVy3DgAmTAD69lU2HiKi7GICRETZsmMH0L27vP/VV8Do0YqGQ0T0VpgAEVGW/fOPXNfr1Su5yOmMGVzfi4gKJiZARJQl588DLVoAL1/Kdb6WLpUjG4mICiL++SKiTN28KZe4ePYMqFcPWLdODnsnIiqomAAR0Rs9eCCHt0dFAVWqAFu3cvkWIir4mAARUYZiYoCmTeUSLu7uwK5dgL290lEREb07JkBEpNfLl8AnnwDh4UCJEnJ9L1dXpaMiIsoZTICIKJ1Xr4COHYEDBwBbW2DnTsDLS+moiIhyDhMgItIhBPD558DmzYC5ufz5/vtKR0VElLOYABGRjuHDgSVL5BD3NWsAPz+lIyIiynlMgMhgPXsGfPSRvK1eDSQmKh2R8n7+GfjlF3n/f/8DWrdWNh4iotxionQAREoQAggOBnbvlo937waKF5dLPPTpA5Qvr2h4ili8WNb+ADIJ6tFD2XiIiHITa4DIIE2dKvu2mJkBQ4YApUoBjx8D06YBFSoADRsCK1cCCQlKR5o3Nm6U/X4A4NtvgWHDlI2HiCi3MQEig3PgADBypLw/axYwfbqc6XjLFqBVK9n3Zf9+oEsXoGRJYOhQ4NIlZWPOTXv3Ap06AWo10KsXMGWK0hEREeW+bCdA7u7umDBhAu7cuZMb8RDlqvv3gQ4dgJQUoFu31FoPExOgZUtZK3T7NjB+PFC6NPDkCfDrr0ClSkCDBsCKFYWrVuj0adnPJzERCAwEFizg4qZEZBiynQANHjwYGzZsQNmyZfHRRx9h9erVSGTvUSoAXr2SNR337wOVKwPz5+v/si9VCvj+e1krtG2bTBCMjYGDB4GuXeVkgEOGABcv5v055KSrV+Usz8+fyya/VatkIkhEZAhUQgjxNi88ffo0li5dilWrViElJQWdO3dGz549UaNGjZyOMc/FxsbCzs4OMTExsLW1VTocyiHffQdMngxYWwMnT2avo3NkpBwa/ttvQNrKzw8/lLVI7doBlpY5H3NuiYwE6teXtV01ashmMH7Uiaigy87391snQBrJycmYN28ehg8fjuTkZFStWhUDBw5Ejx49oCqgdelMgAqfrVtl/x5ADnnv0OHt9pOSIpeEWLRI9hlKSZHlRYsCQUEyGapcOUdCzjVPngC+vrIGy8sLOHRILnVBRFTQ5UkClJycjI0bN2LJkiUIDQ3FBx98gF69euG///7D3Llz0bhxY6xcufKtTkBpTIAKl5s3ZS3Hs2fAV1/Jjs854d691Fqh27dTy+vVk4nQZ5/lv1XT4+MBf3/g6FHZlHf4sFzklIioMMjVBOj06dNYsmQJVq1aBSMjIwQFBaF3796oUKGCdpsLFy6gVq1aePny5dudgcKYABUeiYmyqefUKaBOHTkCzMwsZ4+hVgOhobJW6K+/dGuFunWT8wpVrZqzx3wbSUlycVPNiu4HD+b/2ioiouzIzvd3tjtB16pVCxEREZg/fz4iIyMxdepUneQHADw8PNCxY8fs7pooxw0ZIpOf4sWBP//M+eQHkMPmAwKA9euBu3eBH38EPDxkjdPs2UC1arJWaOlS4MWLnD9+VqjVcpLHXbtkrdS2bUx+iMiwZbsG6Pbt23Bzc8utePIF1gAVDitWyFFbKhWwfbsc8ZRX1GogLEzWCm3aJEegAYCdnYypTx/A2ztvYhECGDgQmDNHjvLasiVvrwURUV7J1Rqg6OhoHDt2LF35sWPHcPLkyezujihX/Ptv6hw/Y8bk/Re+kZFcY2ztWuC//+Tos7JlgZgYYO5coHp14IMPgN9/l/1yctOECTL5UamAP/5g8kNEBLxFAtS/f3/cvXs3XXlkZCT69++fI0ERvYvnz4G2bWVzk7+/nNNHSU5OwIgRQESE7Cv02WeyJubYMTnzsosL0K8fEB6e88eeOxcYN07enzVLzoNERERvkQBdvHhR71w/77//Pi4W9JnhqMATQtb8XLkil7FYuVJOYpgfGBnJhOzPP2Wt0E8/AZ6eMmGbPx94/32gdm25Cntc3Lsfb9UqOeoNAMaOBQYMePd9EhEVFtlOgMzNzfHgwYN05VFRUTDhNLKksHnz5Dw/JiYy0XB0VDoi/Zyc5KKjV6/KlejbtwdMTYETJ2T/IBcX4Msv5VIVb2PXLjkvkRBA//4yASIiolTZ7gTdqVMnREVF4a+//oKdnR0A4NmzZwgMDESJEiXw559/5kqgeYmdoAumY8fkBH/JyXL9rsGDlY4oe6KjgWXLZMfpa9dSy2vWBL74AujYEbCxyXw/R48CTZrIJsCOHWVncCMue0xEBiBX5wGKjIxEgwYN8PjxY7z//vsAgPDwcDg5OSE0NBSlS5d++8jzCSZABc/jx7IJ6e5d2f9n7dqCu6inEMC+fTIRWr9eJnSAXMKjc2fZxFezpv7X/vuvTAKfPpVD8zdvzp2h/0RE+VGuzwQdHx+PFStW4OzZs7C0tES1atXQqVMnmJqavnXQ+QkToIJFrQZatAB27pRLO5w8WXjWtXr4UI7cWrRINpdp1KghE6FOnVLP9dYtOenjvXtyhNnu3UCRIoqETUSkiDxdC6wwYgJUsPzwgxzqbmEhm8GqVVM6opwnhJzFetEiYN06OaszIBOcTp3k2mb9+smRZpUqyVmeixVTNmYioryWJwnQxYsXcefOHSRp/hL/v08++eRtdpevMAEqOHbvBj7+WCYIS5bI2Y4Lu0ePUmuFrlzRfa5MGbm+V6lSysRGRKSkXE2Abty4gTZt2uD8+fNQqVTQvFyz8nuKZiGkAowJUMEQGSn7/Tx8KOfT+d//lI4obwkha3o0tUL29sD+/UC5ckpHRkSkjFydCXrQoEHw8PBAdHQ0rKys8O+//+LAgQPw8fHBvn373jZmomxJTpbNPg8fylmVZ89WOqK8p1IBDRoAy5fL6xARweSHiCirsj1xz5EjR7Bnzx44ODjAyMgIRkZG+PDDDzF58mQMHDgQZ86cyY04iXSMGCGbeuzsZO2HpaXSESkrK8PjiYgoVbZrgFJSUmDz/39tHRwccO/ePQCAm5sbrrzeIYEoF2zYAEyfLu8vXSpnUyYiIsqObNcAValSBWfPnoWHhwfq1KmDn3/+GWZmZli0aBHKli2bGzESaUVEAD16yPvDhgGBgYqGQ0REBVS2E6DRo0cj/v+Xr54wYQJatmwJX19fFC9eHGvWrMnxAIk0Xr4E2rUDYmOBDz8EfvxR6YiIiKigypF5gJ48eQJ7e3vtSLCCjqPA8qdevYDffwdKlADOnAFcXZWOiIiI8pNcGwWWnJwMExMTXLhwQae8WLFi75T8zJ07F+7u7rCwsECdOnVw/PjxN8YwYcIEeHp6wsLCAt7e3ti5c2eG20+ZMgUqlQqDC9rCUKTj99/lzchIrnLO5IeIiN5FthIgU1NTlClTJkfn+lmzZg2GDh2KsWPH4vTp0/D29kZAQACio6P1bj969GgsXLgQs2fPxsWLF/Hll1+iTZs2ekefnThxAgsXLkS1wjg1sAEJD5crmgPAxIlA48aKhkNERIVAtkeBjRo1Ct999x2ePHmSIwFMnz4dffr0QY8ePVCpUiUsWLAAVlZW+P333/VuHxISgu+++w7NmzdH2bJl0bdvXzRv3hzTpk3T2S4uLg5dunTBb7/9Bnt7+xyJlfJeTIzs95OQADRvLoe/ExERvatsd4KeM2cOrl27BldXV7i5uaHIa6stnj59Osv7SkpKwqlTpzBy5EhtmZGREfz9/XHkyBG9r0lMTISFhYVOmaWlJQ4dOqRT1r9/f7Ro0QL+/v744Ycf3hhHYmIiEhMTtY9jY2OzfA6Ue4SQI76uXwfc3ICQENkERkRE9K6ynQAF5uC440ePHiElJQVOTk465U5OTrh8+bLe1wQEBGD69Olo0KABPD09ERYWhg0bNug0y61evRqnT5/GiRMnshTH5MmTMX78+Lc/EcoVv/4KbNwImJoCa9dycU8iIso52U6Axo4dmxtxZNnMmTPRp08fVKhQASqVCp6enujRo4e2yezu3bsYNGgQQkND09UUZWTkyJEYOnSo9nFsbCxKly6dK/FT1hw6BHz7rbw/YwZQq5ai4RARUSGjaIOCg4MDjI2N8eDBA53yBw8ewNnZWe9rHB0dsWnTJsTHx+P27du4fPkyrK2ttZMwnjp1CtHR0ahRowZMTExgYmKC/fv3Y9asWTAxMdHbgdvc3By2trY6N1JOdLRc5yslBejUCejbV+mIiIiosMl2AmRkZARjY+MMb9lhZmaGmjVrIiwsTFumVqsRFhaGunXrvvG1FhYWKFmyJF69eoX169ejdevWAIAmTZrg/PnzCA8P1958fHzQpUsXhIeHZztGyluapOfePaBiRbnSeSGZXoqIiPKRbDeBbdy4UedxcnIyzpw5g2XLlr1VP5qhQ4ciODgYPj4+qF27NmbMmIH4+Hj0+P/1DoKCglCyZElMnjwZAHDs2DFERkaievXqiIyMxLhx46BWq/Ht/7eX2NjYoEqVKjrHKFKkCIoXL56unPKfceOAPXuAIkWA9esBa2ulIyIiosIo2wmQpqYlrXbt2qFy5cpYs2YNevXqla39dejQAQ8fPsT333+P+/fvo3r16ti5c6e2Y/SdO3dglGboT0JCAkaPHo0bN27A2toazZs3R0hICIoWLZrdU6F8ZscOQDNgb9EiWQNERESUG3JkKQwAuHHjBqpVq4a4uLic2J2iuBRG3rt9G6hRA3jyBOjXD5g7V+mIiIiooMm1pTAy8vLlS8yaNQslS5bMid2RgUlMBNq3l8mPjw8wfbrSERERUWGX7Saw1xc9FULg+fPnsLKywvLly3M0ODIMX38NHD8O2NvL+X7MzZWOiIiICrtsJ0C//vqrTgJkZGQER0dH1KlTh0tOULatWpXa3BUSAri7KxoOEREZiGwnQN27d8+FMMgQXboE9Okj748aBbRooWw8RERkOLLdB2jJkiVYu3ZtuvK1a9di2bJlORIUFX5xcUDbtkB8vFzdnSuREBFRXsp2AjR58mQ4ODikKy9RogR+/PHHHAmKCjchgC+/lDVALi7AypUA56ckIqK8lO0E6M6dO/Dw8EhX7ubmhjt37uRIUFS4LVwIrFghk541a4DX1sIlIiLKddlOgEqUKIFz586lKz979iyKFy+eI0FR4XXyJDBokLw/ZQrg66tsPEREZJiynQB16tQJAwcOxN69e5GSkoKUlBTs2bMHgwYNQseOHXMjRioknjwB2rUDkpKAwEA5/J2IiEgJ2R4FNnHiRNy6dQtNmjSBiYl8uVqtRlBQEPsAUYbUaiAoSM747OkJLFnCRU6JiEg5b70URkREBMLDw2FpaYmqVavCzc0tp2NTDJfCyHmTJwPffScnOTx6FKheXemIiIiosMnO93e2a4A0vLy84OXl9bYvJwOydy8werS8P3cukx8iIlJetvsAtW3bFj/99FO68p9//hmfffZZjgRFhce9e0DHjrIJrHt3oGdPpSMiIiJ6iwTowIEDaN68ebryZs2a4cCBAzkSFBUOycky+YmOBqpWlbU/7PdDRET5QbYToLi4OJiZmaUrNzU1RWxsbI4ERYXDqFHAwYOAjQ2wbh1gZaV0RERERFK2E6CqVatizZo16cpXr16NSpUq5UhQVPBt2gT88ou8//vvQLlyioZDRESkI9udoMeMGYNPP/0U169fR+PGjQEAYWFhWLlyJdatW5fjAVLBc/267O8DAEOGyLl/iIiI8pNsJ0CtWrXCpk2b8OOPP2LdunWwtLSEt7c39uzZg2LFiuVGjFSAvHwpE56YGKBePUBPf3kiIiLFvfU8QBqxsbFYtWoVFi9ejFOnTiElJSWnYlMM5wF6e59/Dvz2G+DgAJw5A5QqpXRERERkKLLz/Z3tPkAaBw4cQHBwMFxdXTFt2jQ0btwYR48efdvdUSGwbJlMflQqucI7kx8iIsqvstUEdv/+fSxduhSLFy9GbGws2rdvj8TERGzatIkdoA3cuXNA377y/rhxwEcfKRoOERHRG2W5BqhVq1YoX748zp07hxkzZuDevXuYPXt2bsZGBURsrOz38/IlEBCQOuszERFRfpXlGqAdO3Zg4MCB6Nu3L5fAIC0hgF69gIgIoHRpYPlywOitG1aJiIjyRpa/qg4dOoTnz5+jZs2aqFOnDubMmYNHjx7lZmxUAMyaJSc5NDUF/vxTdn4mIiLK77KcAH3wwQf47bffEBUVhS+++AKrV6+Gq6sr1Go1QkND8fz589yMk/KhI0eAYcPk/WnTgA8+UDYeIiKirHqnYfBXrlzB4sWLERISgmfPnuGjjz7C5s2bczI+RXAYfOYePgRq1AD++w9o3x5YvZrrfBERkbLyZBg8AJQvXx4///wz/vvvP6xatepddkUFiFoNdOkik5/y5YH//Y/JDxERFSzvPBFiYcQaoDfbvBlo3RqwtASOHweqVFE6IiIiojysASLDtHSp/Nm3L5MfIiIqmJgAUbY8egRs3SrvaxY8JSIiKmiYAFG2rFoFJCfLDtBVqyodDRER0dthAkTZsmyZ/BkcrGwcRERE74IJEGXZhQvAqVNy0sPOnZWOhoiI6O0xAaIs09T+tGjBGZ+JiKhgYwJEWfLqlVznC2DnZyIiKviYAFGW/P03cP++rPlp1kzpaIiIiN4NEyDKEk3zV+fOgJmZsrEQERG9KyZAlKmnT4FNm+R9Nn8REVFhwASIMrVmDZCUJOf9qV5d6WiIiIjeHRMgypSm+at7dy56SkREhQMTIHqjK1eAo0cBY2O5AjwREVFhwASI3khT+9OsGeDkpGwsREREOYUJEGUoJQX44w95n0tfEBFRYcIEiDK0Zw8QGQnY2wOtWikdDRERUc5hAkQZWrpU/uzUCTA3VzQUIiKiHMUEiPSKjQU2bpT32fxFRESFDRMg0mvtWuDlS6BiRaBWLaWjISIiyllMgEgvTfNXcDDn/iEiosKHCRClc+0acOgQYGQEdO2qdDREREQ5jwkQpaMZ+v7RR0DJksrGQkRElBuYAJEOtTo1AeLCp0REVFgxASId+/cDt28DdnZA69ZKR0NERJQ7mACRDs3SF+3bA5aWysZCRESUW5gAkVZcHLBunbzP5i8iIirMmACR1vr1QHw84OUF1K2rdDRERES5hwkQaWmavzj3DxERFXZMgAgAcOsWsHevTHy6dVM6GiIiotzFBIgAACEh8mfjxkCZMsrGQkRElNuYABGE0G3+IiIiKuyYABEOHwauXwesrYFPP1U6GiIiotzHBIi0tT+ffQYUKaJsLERERHmBCZCBe/ECWLNG3mfzFxERGQomQAZu0ybg+XPAwwPw9VU6GiIioryRLxKguXPnwt3dHRYWFqhTpw6OHz+e4bbJycmYMGECPD09YWFhAW9vb+zcuVNnm8mTJ6NWrVqwsbFBiRIlEBgYiCtXruT2aRRIS5fKn0FBgFG++DQQERHlPsW/8tasWYOhQ4di7NixOH36NLy9vREQEIDo6Gi9248ePRoLFy7E7NmzcfHiRXz55Zdo06YNzpw5o91m//796N+/P44ePYrQ0FAkJyfj448/Rnx8fF6dVoHw33/A7t3yflCQsrEQERHlJZUQQigZQJ06dVCrVi3MmTMHAKBWq1G6dGl89dVXGDFiRLrtXV1dMWrUKPTv319b1rZtW1haWmL58uV6j/Hw4UOUKFEC+/fvR4MGDTKNKTY2FnZ2doiJiYGtre1bnln+N3ky8N13QIMGchV4IiKigiw739+K1gAlJSXh1KlT8Pf315YZGRnB398fR44c0fuaxMREWFhY6JRZWlri0KFDGR4nJiYGAFCsWLEM9xkbG6tzK+zSzv3DhU+JiMjQKJoAPXr0CCkpKXByctIpd3Jywv379/W+JiAgANOnT0dERATUajVCQ0OxYcMGREVF6d1erVZj8ODBqF+/PqpUqaJ3m8mTJ8POzk57K1269LudWAFw/Dhw5QpgZQW0a6d0NERERHlL8T5A2TVz5kx4eXmhQoUKMDMzw4ABA9CjRw8YZdCDt3///rhw4QJWr16d4T5HjhyJmJgY7e3u3bu5FX6+oen8/OmngI2NoqEQERHlOUUTIAcHBxgbG+PBgwc65Q8ePICzs7Pe1zg6OmLTpk2Ij4/H7du3cfnyZVhbW6Ns2bLpth0wYAC2bt2KvXv3olSpUhnGYW5uDltbW51bYZaQAGjyQTZ/ERGRIVI0ATIzM0PNmjURFhamLVOr1QgLC0PdunXf+FoLCwuULFkSr169wvr169G6dWvtc0IIDBgwABs3bsSePXvg4eGRa+dQEG3eDDx7BpQuDTRqpHQ0REREec9E6QCGDh2K4OBg+Pj4oHbt2pgxYwbi4+PRo0cPAEBQUBBKliyJyZMnAwCOHTuGyMhIVK9eHZGRkRg3bhzUajW+/fZb7T779++PlStX4q+//oKNjY22P5GdnR0sLS3z/iTzGU3nZ879Q0REhkrxBKhDhw54+PAhvv/+e9y/fx/Vq1fHzp07tR2j79y5o9O/JyEhAaNHj8aNGzdgbW2N5s2bIyQkBEWLFtVuM3/+fABAw4YNdY61ZMkSdDfwNp+oKEAzbyTn/iEiIkOl+DxA+VFhngdo6lTgm2+AevXkKvBERESFRYGZB4jyVtq5f7jwKRERGTImQAbk9GngwgXA3Bxo317paIiIiJTDBMiAaGp/2rQB0nSZIiIiMjhMgAxEUhKwcqW8z+YvIiIydEyADMS2bcDjx4CrK/DRR0pHQ0REpCwmQAZCs/RF166AsbGioRARESmOCZABiI4Gtm+X99n8RURExATIIKxcCbx6BdSqBVSqpHQ0REREymMCZAA0o78MfBJsIiIiLSZAhdzZs0B4OGBmBnTsqHQ0RERE+QMToEJOU/vTqhVQrJiysRAREeUXTIAKseRkYMUKeZ/NX0RERKmYABViO3fKEWAlSgABAUpHQ0RElH8wASrENM1fXbsCpqbKxkJERJSfMAEqpB4/BjZvlvc59w8REZEuJkCF1OrVsg/Q++8D1aopHQ0REVH+wgSokNI0f7H2h4iIKD0mQIXQxYvAiROAiQnQubPS0RAREeU/TIAKIU3tT4sWgKOjsrEQERHlR0yACplXr4CQEHmfzV9ERET6MQEqZHbvBqKigOLFZQ0QERERpccEqJBZulT+7NxZrv9FRERE6TEBKkSePQM2bZL3ufQFERFRxpgAFSJ//gkkJgJVqsj5f4iIiEg/JkCFiKb5q3t3QKVSMhIiIqL8jQlQIXH1KnDkCGBsDHTponQ0RERE+RsToEJCM/dPQADg7KxsLERERPkdE6BCICUF+OMPeZ+dn4mIiDLHBKgQ2LsX+O8/oGhRoFUrpaMhIiLK/5gAFQKa5q9OnQALC2VjISIiKgiYABVwsbHA+vXyPpe+ICIiyhomQAXcunXAy5dAhQpA7dpKR0NERFQwMAEq4DTNX8HBnPuHiIgoq5gAFWA3bgAHDsjEp2tXpaMhIiIqOJgAFWCaoe8ffQSUKqVsLERERAUJE6ACSq3Wbf4iIiKirGMCVEAdPAjcugXY2gKBgUpHQ0REVLAwASqgNLU/7dsDVlbKxkJERFTQMAEqgOLjgbVr5X0ufUFERJR9TIAKoA0bgLg44L33gHr1lI6GiIio4GECVAAtXSp/cu4fIiKit8MEqIC5fVsufgoA3bopGwsREVFBxQSogAkJAYQAGjUC3NyUjoaIiKhgYgJUgAiROvkhOz8TERG9PSZABciRI0BEBFCkCPDpp0pHQ0REVHAxASpANJ2fP/sMsLZWNBQiIqICjQlQAfHyJbBmjbzPpS+IiIjeDROgAmLTJiA2FnB3Bxo0UDoaIiKigo0JUAGhWfoiKAgw4rtGRET0TvhVWgBERgKhofJ+UJCysRARERUGTIAKgOXLAbUa8PUFPD2VjoaIiKjgM1E6AHozIVKbv9j52TClpKQgOTlZ6TCIiBRnamoKY2PjHNkXE6B87sQJ4NIlwNJSDn8nwyGEwP379/Hs2TOlQyEiyjeKFi0KZ2dnqN5xMUwmQPmcpvbn008BW1tlY6G8pUl+SpQoASsrq3f+ZSciKsiEEHjx4gWio6MBAC4uLu+0PyZA+VhiIrBqlbzPpS8MS0pKijb5KV68uNLhEBHlC5aWlgCA6OholChR4p2aw9gJOh/bsgV4+hQoVUoufkqGQ9Pnx8rKSuFIiIjyF83fxXftG8kEKB/TLH0RFATkUJ8vKmDY7EVEpCun/i4yAcqnHjwAdu6U9zn3Dxmahg0bYvDgwdrH7u7umDFjxhtfo1KpsGnTpnc+dk7th3JHVj4L48aNQ/Xq1fMknrRu3boFlUqF8PDwPD82AHTv3h2BgYGKHDu79u3bB5VKpeggDyZA+dSKFUBKCvDBB0D58kpHQ5Q1rVq1QtOmTfU+d/DgQahUKpw7dy7b+z1x4gQ+//zzdw1PR0ZfklFRUWjWrFmOHotyzuufhfyUsJYuXRpRUVGoUqWK0qFQFjAByoeESG3+YudnKkh69eqF0NBQ/Pfff+meW7JkCXx8fFCtWrVs79fR0THP+kM5OzvD3Nw8T46VnyQlJSkdQpbk5Wchu4yNjeHs7AwTE44vKgiYAOVD4eHA+fOAuTnQoYPS0RBlXcuWLeHo6Iilmgz+/8XFxWHt2rXo1asXHj9+jE6dOqFkyZKwsrJC1apVsUoz3DEDrzd7REREoEGDBrCwsEClSpUQqlkrJo3hw4ejXLlysLKyQtmyZTFmzBhtp8mlS5di/PjxOHv2LFQqFVQqlTbm12sUzp8/j8aNG8PS0hLFixfH559/jri4OO3zmmaHqVOnwsXFBcWLF0f//v3f2EHz+vXraN26NZycnGBtbY1atWph9+7dOtskJiZi+PDhKF26NMzNzfHee+9h8eLF2uf//fdftGzZEra2trCxsYGvry+uX78OIH0TIgAEBgaie5r/qNzd3TFx4kQEBQXB1tZWW6vypuumsWXLFtSqVQsWFhZwcHBAmzZtAAATJkzQW/tRvXp1jBkzRu+18PHxwdSpU3XiNDU11V7j//77DyqVCteuXdPGrfksuLu7AwDatGkDlUqlfawREhICd3d32NnZoWPHjnj+/LneGAD5mShatCh27dqFihUrwtraGk2bNkVUVJR2G7VajQkTJqBUqVIwNzdH9erVsVPTVwHpm8CePn2KLl26wNHREZaWlvDy8sKSJUu029+9exft27dH0aJFUaxYMbRu3Rq3bt3KMEbgze+7xps+iyEhIfDx8YGNjQ2cnZ3RuXNn7bByILVpKiwsDD4+PrCyskK9evVw5coV7Taa2tM3XV+1Wo3JkyfDw8MDlpaW8Pb2xrp16zI8r9u3b6NVq1awt7dHkSJFULlyZWzfvv2N1+JdMQHKhzTfHYGBQNGiCgZC+YoQQHy8MjchshajiYkJgoKCsHTpUog0L1q7di1SUlLQqVMnJCQkoGbNmti2bRsuXLiAzz//HN26dcPx48ezdAy1Wo1PP/0UZmZmOHbsGBYsWIDhw4en287GxgZLly7FxYsXMXPmTPz222/49ddfAQAdOnTA119/jcqVKyMqKgpRUVHooOe/jfj4eAQEBMDe3h4nTpzA2rVrsXv3bgwYMEBnu7179+L69evYu3cvli1bhqVLl6ZLAtOKi4tD8+bNERYWhjNnzqBp06Zo1aoV7ty5o90mKCgIq1atwqxZs3Dp0iUsXLgQ1tbWAIDIyEg0aNAA5ubm2LNnD06dOoWePXvi1atXWbqGGlOnToW3tzfOnDmjTVDedN0AYNu2bWjTpg2aN2+OM2fOICwsDLVr1wYA9OzZE5cuXcKJEye02585cwbnzp1Djx499Mbg5+eHffv2AZDzvBw8eBBFixbFoUOHAAD79+9HyZIl8d5776V7reY4S5YsQVRUlM5xr1+/jk2bNmHr1q3YunUr9u/fjylTprzxerx48QJTp05FSEgIDhw4gDt37mDYsGHa52fOnIlp06Zh6tSpOHfuHAICAvDJJ58gIiJC7/7GjBmDixcvYseOHbh06RLmz58PBwcHAHIEU0BAAGxsbHDw4EEcPnxYm3RlVBuXlfc9s89icnIyJk6ciLNnz2LTpk24deuWTmKsMWrUKEybNg0nT56EiYkJevbsqfN8Ztd38uTJ+OOPP7BgwQL8+++/GDJkCLp27Yr9+/frPbf+/fsjMTERBw4cwPnz5/HTTz9pP++5RlA6MTExAoCIiYnJ82MnJgrh4CAEIMT27Xl+eMonXr58KS5evChevnypLYuLk58LJW5xcVmP/dKlSwKA2Lt3r7bM19dXdO3aNcPXtGjRQnz99dfax35+fmLQoEHax25ubuLXX38VQgixa9cuYWJiIiIjI7XP79ixQwAQGzduzPAYv/zyi6hZs6b28dixY4W3t3e67dLuZ9GiRcLe3l7EpbkA27ZtE0ZGRuL+/ftCCCGCg4OFm5ubePXqlXabzz77THTo0CHDWPSpXLmymD17thBCiCtXrggAIjQ0VO+2I0eOFB4eHiIpKUnv869fPyGEaN26tQgODtY+dnNzE4GBgZnG9fp1q1u3rujSpUuG2zdr1kz07dtX+/irr74SDRs2zHD7zZs3Czs7O/Hq1SsRHh4unJ2dxaBBg8Tw4cOFEEL07t1bdO7cWSduzWdBCKH3fR87dqywsrISsbGx2rJvvvlG1KlTJ8M4lixZIgCIa9euacvmzp0rnJyctI9dXV3FpEmTdF5Xq1Yt0a9fPyGEEDdv3hQAxJkzZ4QQQrRq1Ur06NFD7/FCQkJE+fLlhVqt1pYlJiYKS0tLsWvXLr2vyex9f5vP4okTJwQA8fz5cyGEEHv37hUAxO7du7XbbNu2TQDQ/j3K7PomJCQIKysr8c8//+gcq1evXqJTp046x3n69KkQQoiqVauKcePGZRhnWvr+Pmpk5/s7X9QAzZ07F+7u7rCwsECdOnXe+J9gcnIyJkyYAE9PT1hYWMDb21unCvJt9pmfbN8OPHoEuLgAH32kdDRE2VehQgXUq1cPv//+OwDg2rVrOHjwIHr16gVATvI4ceJEVK1aFcWKFYO1tTV27dqlU/vxJpcuXULp0qXh6uqqLatbt2667dasWYP69evD2dkZ1tbWGD16dJaPkfZY3t7eKFKkiLasfv36UKvVOk0ClStX1pmQzcXFRadZ4XVxcXEYNmwYKlasiKJFi8La2hqXLl3SxhceHg5jY2P4+fnpfX14eDh8fX1hamqarfN5nY+PT7qyzK5beHg4mjRpkuE++/Tpg1WrViEhIQFJSUlYuXJlutqDtHx9ffH8+XOcOXMG+/fvh5+fHxo2bKitFdq/fz8aNmyY7XNzd3eHjY2N9nFm7wkg55fxTLPidNrXxMbG4t69e6hfv77Oa+rXr49Lly7p3V/fvn2xevVqVK9eHd9++y3++ecf7XNnz57FtWvXYGNjA2tra1hbW6NYsWJISEhI16SlkZX3PbPP4qlTp9CqVSuUKVMGNjY22s/Y678bafvqaWZcTrufN13fa9eu4cWLF/joo4+052ZtbY0//vgjw3MbOHAgfvjhB9SvXx9jx459q8ES2aV4ArRmzRoMHToUY8eOxenTp+Ht7Y2AgIAMP6ijR4/GwoULMXv2bFy8eBFffvkl2rRpgzNnzrz1PvMTzdIXXbsC7EdHaVlZAXFxytyy2+e0V69eWL9+PZ4/f44lS5bA09NT+4f2l19+wcyZMzF8+HDs3bsX4eHhCAgIyNFOuEeOHEGXLl3QvHlzbN26FWfOnMGoUaNyraPv619IKpUKarU6w+2HDRuGjRs34scff8TBgwcRHh6OqlWrauPTzHabkcyeNzIy0mmCBPRPGpc2sQOydt0yO3arVq1gbm6OjRs3YsuWLUhOTka7du0y3L5o0aLw9vbGvn37tMlOgwYNcObMGVy9ehUREREZJoJvkt33JKPXvH4ds6NZs2a4ffs2hgwZgnv37qFJkybaJrW4uDjUrFkT4eHhOrerV6+ic+fOeveX2bXP6Bw0561p0rW1tcWKFStw4sQJbNy4EUD6TvBp96OZdyft9XvTcTT9t7Zt26ZzbhcvXsywH1Dv3r1x48YNdOvWDefPn4ePjw9mz56d6fm+C8UToOnTp6NPnz7o0aMHKlWqhAULFsDKykr73+PrQkJC8N1336F58+YoW7Ys+vbti+bNm2PatGlvvc/84uFDYOtWeZ8rv9PrVCqgSBFlbtmdd6x9+/YwMjLCypUr8ccff6Bnz57aP6KHDx9G69at0bVrV3h7e6Ns2bK4evVqlvddsWJF3L17V6dz6tGjR3W2+eeff+Dm5oZRo0bBx8cHXl5euH37ts42ZmZmSElJyfRYZ8+eRXx8vLbs8OHDMDIyQvl3mJ/i8OHD6N69O9q0aYOqVavC2dlZp/Nr1apVoVarM+wvUa1aNRw8eDDDjtaOjo461yclJQUXLlzINK6sXLdq1aohLCwsw32YmJggODgYS5YswZIlS9CxY8dMv7j9/Pywd+9eHDhwAA0bNkSxYsVQsWJFTJo0CS4uLihXrlyGrzU1Nc30fcwJtra2cHV1xeHDh3XKDx8+jEqVKmX4OkdHRwQHB2P58uWYMWMGFi1aBACoUaMGIiIiUKJECbz33ns6Nzs7O737yux9z8zly5fx+PFjTJkyBb6+vqhQoUKuVAxUqlQJ5ubmuHPnTrpzK126dIavK126NL788kts2LABX3/9NX777bccjy0tRROgpKQknDp1Cv7+/toyIyMj+Pv748iRI3pfk5iYCAsLC50yS0tLbYe5t9lnfrFqFfDqFeDjA1SurHQ0RG/P2toaHTp0wMiRIxEVFaXTydLLywuhoaH4559/cOnSJXzxxRd48OBBlvft7++PcuXKITg4GGfPnsXBgwcxatQonW28vLxw584drF69GtevX8esWbO0/+lquLu74+bNmwgPD8ejR4+QmJiY7lhdunSBhYUFgoODceHCBezduxdfffUVunXrBicnp+xdlNfi27BhA8LDw3H27Fl07txZ579rd3d3BAcHo2fPnti0aRNu3ryJffv24c8//wQADBgwALGxsejYsSNOnjyJiIgIhISEaJvlGjdujG3btmHbtm24fPky+vbtm6UJ57Jy3caOHYtVq1Zh7NixuHTpkrbDalq9e/fGnj17sHPnzjc2f2k0bNgQu3btgomJCSpUqKAtW7FiRaa1P+7u7ggLC8P9+/fx9OnTTI/1Lr755hv89NNPWLNmDa5cuYIRI0YgPDwcgwYN0rv9999/j7/++gvXrl3Dv//+i61bt6JixYoA5GfLwcEBrVu3xsGDB7Xv8cCBA/VOIwFk/r5npkyZMjAzM8Ps2bNx48YNbN68GRMnTny7i/EGNjY2GDZsGIYMGYJly5bh+vXrOH36NGbPno1lmmaO1wwePBi7du3CzZs3cfr0aezdu1d7rXKLognQo0ePkJKSku4PiZOTE+7fv6/3NQEBAZg+fToiIiKgVqsRGhqKDRs2aP/beZt9JiYmIjY2VuemBM3ngrU/VBj06tULT58+RUBAgE5/ndGjR6NGjRoICAhAw4YN4ezsnK3Za42MjLBx40a8fPkStWvXRu/evTFp0iSdbT755BMMGTIEAwYMQPXq1fHPP/+kG4bdtm1bNG3aFI0aNYKjo6PeofhWVlbYtWsXnjx5glq1aqFdu3Zo0qQJ5syZk72L8Zrp06fD3t4e9erVQ6tWrRAQEIAaNWrobDN//ny0a9cO/fr1Q4UKFdCnTx9tTVTx4sWxZ88exMXFwc/PDzVr1sRvv/2mbZbo2bMngoODERQUBD8/P5QtWxaNsrCgYFauW8OGDbF27Vps3rwZ1atXR+PGjdP1sfTy8kK9evVQoUIF1KlTJ9Pj+vr6Qq1W6yQ7DRs2REpKSqb9f6ZNm4bQ0FCULl0a77//fqbHehcDBw7E0KFD8fXXX6Nq1arYuXMnNm/eDC8vL73bm5mZYeTIkahWrRoaNGgAY2NjrF69GoD8bB04cABlypTBp59+iooVK6JXr15ISEiAra2t3v1l9r5nRjNFxdq1a1GpUiVMmTJFZwqCnDRx4kSMGTMGkydPRsWKFdG0aVNs27YNHh4eerdPSUlB//79tduWK1cO8+bNy5XYtLLU5TqXREZGCgDpeop/8803onbt2npfEx0dLVq3bi2MjIyEsbGxKFeunOjXr5+wsLB4632OHTtWAEh3y8tRYOfOydE2pqZCPHqUZ4elfOpNoxyI8ju1Wi08PT3FtGnTlA6FCqFCMQrMwcEBxsbG6aq/Hzx4AGdnZ72vcXR0xKZNmxAfH4/bt2/j8uXLsLa2RtmyZd96nyNHjkRMTIz2dvfu3Rw4u+zR1P60agUUL57nhyciyhEPHz7EnDlzcP/+/Qzn/iHKDxRNgMzMzFCzZk2dDnVqtRphYWF6h7WmZWFhgZIlS+LVq1dYv349Wrdu/db7NDc3h62trc4tL716BSxfLu9z6QsiKshKlCiBCRMmYNGiRbC3t1c6HKIMKT7QeujQoQgODoaPjw9q166NGTNmID4+XvufQ1BQEEqWLInJkycDAI4dO4bIyEhUr14dkZGRGDduHNRqNb799tss7zO/2bVLrv7u6AhksI4kEVGBIN5h2DhRXlI8AerQoQMePnyI77//Hvfv39euraLpxHznzh0YGaVWVCUkJGD06NG4ceMGrK2t0bx5c4SEhKBomjUjMttnfqOZpbxrV+Ad5zUjIiKiLFAJpuvpxMbGws7ODjExMbneHPbkiZz1OSlJLoLq7Z2rh6MCIiEhATdv3oSHh0e6aR+IiAzZm/4+Zuf7W/GJEA3dmjUy+fH2ZvJDRESUV5gAKUzT/MXOz0RERHmHCZCCLl0Cjh+Xa35lsPQLERER5QImQArSzP3TvDlQooSysRARERkSJkAKSUkBQkLkfS59QaSrYcOGGDx4sPaxu7s7ZsyY8cbXqFQqbNq06Z2PnVP7odyRlc/CuHHjUL169TyJJz9ZunSpzojo/E7p3zUmQArZvRu4d0/O+tyypdLREOWMVq1aoWkGk1kdPHgQKpUK586dy/Z+T5w4gc8///xdw9OR0ZdkVFQUmjVrlqPHopzz+mdB6S9RKriYAClE0/zVqRNgZqZsLEQ5pVevXggNDdW7mvWSJUvg4+ODatWqZXu/jo6OsLKyyokQM+Xs7Axzc/M8OVZ+kpSUpHQIWZKXn4V3UVCupyFjAqSAmBhg40Z5n81fVJi0bNlSu+J0WnFxcVi7di169eqFx48fo1OnTihZsiSsrKxQtWpVvSuxp/V6s0dERAQaNGgACwsLVKpUCaGhoeleM3z4cJQrVw5WVlYoW7YsxowZg+TkZACyqWD8+PE4e/YsVCoVVCqVNubXaxTOnz+Pxo0bw9LSEsWLF8fnn3+OuLg47fPdu3dHYGAgpk6dChcXFxQvXhz9+/fXHkuf69evo3Xr1nBycoK1tTVq1aqF3bt362yTmJiI4cOHo3Tp0jA3N8d7772HxYsXa5//999/0bJlS9ja2sLGxga+vr64fv06gPRNiAAQGBiI7mmGm7q7u2PixIkICgqCra2ttlblTddNY8uWLahVqxYsLCzg4OCANm3aAAAmTJiAKlWqpDvf6tWrp1tVXsPHx0dnRfLAwECYmppqr/F///0HlUqFa9euaePWfBbc3d0BAG3atIFKpdI+1ggJCYG7uzvs7OzQsWNHPH/+XG8Mmmum+Sykvd26dQsA8OzZM/Tu3RuOjo6wtbVF48aNcfbsWe3rNTWK//vf/3Tmp7lz5w5at24Na2tr2Nraon379jprVZ49exaNGjWCjY0NbG1tUbNmTZw8eTLDOJ89e4YvvvgCTk5OsLCwQJUqVbB161adbXbt2oWKFSvC2toaTZs2RVRUlPa5EydO4KOPPoKDgwPs7Ozg5+eH06dP67xepVLhf//7H9q0aQMrKyt4eXlh8+bN2uf37dsHlUqFsLAw+Pj4wMrKCvXq1cOVK1d09vPXX3+hRo0asLCwQNmyZTF+/Hi8evVK73klJSVhwIABcHFxgYWFBdzc3LQrQOQWJkAK+PNPICEBqFwZqFlT6WiooImPz/iWkJD1bV++zNq22WFiYoKgoCAsXbpUZ0mEtWvXIiUlBZ06dUJCQgJq1qyJbdu24cKFC/j888/RrVs3HD9+PEvHUKvV+PTTT2FmZoZjx45hwYIFGD58eLrtbGxssHTpUly8eBEzZ87Eb7/9hl9//RWAnC3+66+/RuXKlREVFYWoqCh06NAh3T7i4+MREBAAe3t7nDhxAmvXrsXu3bsxYMAAne327t2L69evY+/evVi2bBmWLl2aLglMKy4uDs2bN0dYWBjOnDmDpk2bolWrVrhz5452m6CgIKxatQqzZs3CpUuXsHDhQlhbWwMAIiMj0aBBA5ibm2PPnj04deoUevbsmeGXS0amTp0Kb29vnDlzRpugvOm6AcC2bdvQpk0bNG/eHGfOnEFYWBhq164NAOjZsycuXbqEEydOaLc/c+YMzp07l+FSRH5+fti3bx8AuYzGwYMHUbRoURw6dAgAsH//fpQsWRLvvfdeutdqjrNkyRJERUXpHPf69evYtGkTtm7diq1bt2L//v2YMmVKhtdiw4YN2s9CVFQUPv30U5QvX167gsBnn32G6Oho7NixA6dOnUKNGjXQpEkTPHnyRLuPa9euYf369diwYQPCw8OhVqvRunVrPHnyBPv370doaChu3Lih81nr0qULSpUqhRMnTuDUqVMYMWIETDNYEkCtVqNZs2Y4fPgwli9fjosXL2LKlCkwNjbWbvPixQtMnToVISEhOHDgAO7cuYNhw4Zpn3/+/DmCg4Nx6NAhHD16FF5eXmjevHm65HD8+PFo3749zp07h+bNm6NLly465woAo0aNwrRp03Dy5EmYmJigZ8+e2ucOHjyIoKAgDBo0CBcvXsTChQuxdOlSTJo0Se+5zZo1C5s3b8aff/6JK1euYMWKFekS2hyXw6vUFwoxMTECgIiJicmV/devLwQgxM8/58ruqRB4+fKluHjxonj58mW654CMb82b625rZZXxtn5+uts6OOjfLrsuXbokAIi9e/dqy3x9fUXXrl0zfE2LFi3E119/rX3s5+cnBg0apH3s5uYmfv31VyGEELt27RImJiYiMjJS+/yOHTsEALFx48YMj/HLL7+ImjVrah+PHTtWeHt7p9su7X4WLVok7O3tRVxcnPb5bdu2CSMjI3H//n0hhBDBwcHCzc1NvHr1SrvNZ599Jjp06JBhLPpUrlxZzJ49WwghxJUrVwQAERoaqnfbkSNHCg8PD5GUlKT3+devnxBCtG7dWgQHB2sfu7m5icDAwEzjev261a1bV3Tp0iXD7Zs1ayb69u2rffzVV1+Jhg0bZrj95s2bhZ2dnXj16pUIDw8Xzs7OYtCgQWL48OFCCCF69+4tOnfurBO35rMghND7vo8dO1ZYWVmJ2NhYbdk333wj6tSpk+n5CiHE9OnTRdGiRcWVK1eEEEIcPHhQ2NraioSEBJ3tPD09xcKFC7XHNDU1FdHR0drn//77b2FsbCzu3LmjLfv3338FAHH8+HEhhBA2NjZi6dKlWYpr165dwsjISBvX65YsWSIAiGvXrmnL5s6dK5ycnDLcZ0pKirCxsRFbtmzRlgEQo0eP1j6Oi4sTAMSOHTuEEELs3btXABC7d+/WbrNt2zYBQPs3q0mTJuLHH3/UOVZISIhwcXHROY7mvfvqq69E48aNhVqtzuwyvPHvY3a+v1kDlMciIoDDhwEjI7n2F1FhU6FCBdSrVw+///47APlf8cGDB9GrVy8AQEpKCiZOnIiqVauiWLFisLa2xq5du3RqP97k0qVLKF26NFxdXbVldevWTbfdmjVrUL9+fTg7O8Pa2hqjR4/O8jHSHsvb2xtFihTRltWvXx9qtVqnur9y5co6/4W7uLggOjo6w/3GxcVh2LBhqFixIooWLQpra2tcunRJG194eDiMjY3h5+en9/Xh4eHw9fXNsKYgq3x8fNKVZXbdwsPD0aRJkwz32adPH6xatQoJCQlISkrCypUrdWoGXufr64vnz5/jzJkz2L9/P/z8/NCwYUNtrdD+/fvRsGHDbJ+bu7s7bGxstI8ze080duzYgREjRmDNmjUoV64cANlMFRcXh+LFi8Pa2lp7u3nzprbZEQDc3Nzg6Oiofaz5rJYuXVpbVqlSJRQtWhSXLl0CIBfv7t27N/z9/TFlyhSd/b0uPDwcpUqV0salj5WVFTw9PTM87wcPHqBPnz7w8vKCnZ0dbG1tERcXl+53I21fvSJFisDW1jbd9Uu7jYuLCwBotzl79iwmTJigc7369OmDqKgovHjxIl3c3bt3R3h4OMqXL4+BAwfi77//zvAcc4rii6Eamj/+kD8DAuQaYETZlab7STppvoMBAG/6e2/02r8//9/VIUf06tULX331FebOnYslS5bA09NT+2X+yy+/YObMmZgxYwaqVq2KIkWKYPDgwTnaafTIkSPo0qULxo8fj4CAANjZ2WH16tWYNm1ajh0jrdcTEZVKBbVaneH2w4YNQ2hoKKZOnYr33nsPlpaWaNeunfYaWFpavvF4mT1vZGSUblV2fX2S0iZ2QNauW2bHbtWqFczNzbFx40aYmZkhOTkZ7dq1y3D7okWLwtvbG/v27cORI0fw0UcfoUGDBujQoQOuXr2KiIiIDBPBN8nuewIAFy9eRMeOHTFlyhR8/PHH2vK4uDi4uLhok7LX49d4/Xpmxbhx49C5c2ds27YNO3bswNixY7F69Wptv6q0Mrv2gP7zTvtZCA4OxuPHjzFz5ky4ubnB3NwcdevWTff7l5Xrl3YblUoFANpt4uLiMH78eHz66afpYtS3vmGNGjVw8+ZN7NixA7t370b79u3h7++PdevWZXrOb4sJUB5Sq1MTIC59QW8rO39jc2vbzLRv3x6DBg3CypUr8ccff6Bv377aP5CHDx9G69at0fX/q0DVajWuXr2KSpUqZWnfFStWxN27dxEVFaX9r/Po0aM62/zzzz9wc3PDqFGjtGW3b9/W2cbMzAwpKSmZHmvp0qWIj4/XfrkdPnwYRkZGKF++fJbi1efw4cPo3r279ksuLi5O29kWAKpWrQq1Wo39+/fD398/3eurVauGZcuWITk5WW8tkKOjo07H15SUFFy4cAGNGjV6Y1xZuW7VqlVDWFhYhn16TExMEBwcjCVLlsDMzAwdO3bM9Ivbz88Pe/fuxfHjxzFp0iQUK1YMFStWxKRJk+Di4vLGGg9TU9NM38esePToEVq1aoW2bdtiyJAhOs/VqFED9+/fh4mJSbb6pWg+q3fv3tXWAl28eBHPnj3T+byXK1cO5cqVw5AhQ9CpUycsWbJEbwJUrVo1/Pfff7h69eobr8mbHD58GPPmzUPz5s0BAHfv3sWjR4/eal9vUqNGDVy5ckVv362M2NraokOHDujQoQPatWuHpk2b4smTJyhWrFiOxwewE3Se2rcPuHMHKFoU+OQTpaMhyj3W1tbo0KEDRo4ciaioKJ3RR15eXggNDcU///yDS5cu4YsvvtAZFZMZf39/lCtXDsHBwTh79iwOHjyo84WtOcadO3ewevVqXL9+HbNmzcJGzdDL/+fu7o6bN28iPDwcjx49QmJiYrpjdenSBRYWFggODsaFCxewd+9efPXVV+jWrZu2c+zb8PLy0naUPXv2LDp37qzz37W7uzuCg4PRs2dPbNq0CTdv3sS+ffvw559/AgAGDBiA2NhYdOzYESdPnkRERARCQkK0zXKNGzfGtm3bsG3bNly+fBl9+/bFs2fPshRXZtdt7NixWLVqFcaOHYtLly7h/Pnz+Omnn3S26d27N/bs2YOdO3e+sflLo2HDhti1axdMTExQoUIFbdmKFSsyrf1xd3dHWFgY7t+/j6dPn2Z6rIy0bdsWVlZWGDduHO7fv6+9paSkwN/fH3Xr1kVgYCD+/vtv3Lp1C//88w9GjRr1xhFb/v7+qFq1Krp06YLTp0/j+PHjCAoKgp+fH3x8fPDy5UsMGDAA+/btw+3bt3H48GGcOHECFStW1Ls/Pz8/NGjQAG3btkVoaKi2xmTnzp1ZPk8vLy+EhITg0qVLOHbsGLp06ZKlmqXs+v777/HHH39g/Pjx+Pfff3Hp0iWsXr0ao0eP1rv99OnTsWrVKly+fBlXr17F2rVr4ezsnKsTOzIBykP//SeTn44dAT01gESFSq9evfD06VMEBATo9NcZPXo0atSogYCAADRs2BDOzs4IDAzM8n6NjIywceNGvHz5ErVr10bv3r3TjSz55JNPMGTIEAwYMADVq1fHP//8k24Ydtu2bdG0aVM0atQIjo6OeofiW1lZYdeuXXjy5Alq1aqFdu3aoUmTJpgzZ072LsZrpk+fDnt7e9SrVw+tWrVCQEAAatSoobPN/Pnz0a5dO/Tr1w8VKlRAnz59EP//w/KKFy+OPXv2IC4uDn5+fqhZsyZ+++03bW1Qz549ERwcrP2yLVu2bKa1P0DWrlvDhg2xdu1abN68GdWrV0fjxo3TjeDz8vJCvXr1UKFCBdSpUyfT4/r6+kKtVuskOw0bNkRKSkqm/X+mTZuG0NBQlC5dGu+//36mx8rIgQMHcOHCBbi5ucHFxUV7u3v3LlQqFbZv344GDRqgR48eKFeuHDp27Ijbt2+/MRFWqVT466+/YG9vjwYNGsDf3x9ly5bFmjVrAADGxsZ4/PgxgoKCUK5cObRv3x7NmjXD+PHjM9zn+vXrUatWLXTq1AmVKlXCt99+m60asMWLF+Pp06eoUaMGunXrhoEDB6JELqzFFBAQgK1bt+Lvv/9GrVq18MEHH+DXX3+Fm5ub3u1tbGzw888/w8fHB7Vq1cKtW7ewfft2GL3eVp+DVOL1hmJCbGws7OzsEBMTA1tb2xzdd0KC7MPh4JCju6VCJiEhATdv3tSZT4SooBBCwMvLC/369cPQoUOVDocKmTf9fczO9zf7AOUxCwvW/hBR4fXw4UOsXr0a9+/fz7CfEFF+wASIiIhyTIkSJeDg4IBFixbB3t5e6XCIMsQEiIiIcgx7VVBBwU7QREREZHCYABEREZHBYQJElI+xOYGISFdO/V1kAkSUD2nmc9G3Zg4RkSHT/F1817Xw2AmaKB8yNjZG0aJFtQsLWllZaZeSICIyREIIvHjxAtHR0ShatKjOAsRvgwkQUT7l7OwMAFlawZqIyFAULVpU+/fxXTABIsqnVCoVXFxcUKJECb0reRMRGRpTU9N3rvnRYAJElM8ZGxvn2C88ERFJ7ARNREREBocJEBERERkcJkBERERkcNgHSA/NJEuxsbEKR0JERERZpfnezspkiUyA9Hj+/DkAoHTp0gpHQkRERNn1/Plz2NnZvXEbleBc++mo1Wrcu3cPNjY2nHwuA7GxsShdujTu3r0LW1tbpcMxeHw/8he+H/kL34/8J7feEyEEnj9/DldXVxgZvbmXD2uA9DAyMkKpUqWUDqNAsLW15R+UfITvR/7C9yN/4fuR/+TGe5JZzY8GO0ETERGRwWECRERERAaHCRC9FXNzc4wdOxbm5uZKh0Lg+5Hf8P3IX/h+5D/54T1hJ2giIiIyOKwBIiIiIoPDBIiIiIgMDhMgIiIiMjhMgIiIiMjgMAGiLJs8eTJq1aoFGxsblChRAoGBgbhy5YrSYdH/mzJlClQqFQYPHqx0KAYtMjISXbt2RfHixWFpaYmqVavi5MmTSodlkFJSUjBmzBh4eHjA0tISnp6emDhxYpbWiaJ3d+DAAbRq1Qqurq5QqVTYtGmTzvNCCHz//fdwcXGBpaUl/P39ERERkWfxMQGiLNu/fz/69++Po0ePIjQ0FMnJyfj4448RHx+vdGgG78SJE1i4cCGqVaumdCgG7enTp6hfvz5MTU2xY8cOXLx4EdOmTYO9vb3SoRmkn376CfPnz8ecOXNw6dIl/PTTT/j5558xe/ZspUMzCPHx8fD29sbcuXP1Pv/zzz9j1qxZWLBgAY4dO4YiRYogICAACQkJeRIfh8HTW3v48CFKlCiB/fv3o0GDBkqHY7Di4uJQo0YNzJs3Dz/88AOqV6+OGTNmKB2WQRoxYgQOHz6MgwcPKh0KAWjZsiWcnJywePFibVnbtm1haWmJ5cuXKxiZ4VGpVNi4cSMCAwMByNofV1dXfP311xg2bBgAICYmBk5OTli6dCk6duyY6zGxBojeWkxMDACgWLFiCkdi2Pr3748WLVrA399f6VAM3ubNm+Hj44PPPvsMJUqUwPvvv4/ffvtN6bAMVr169RAWFoarV68CAM6ePYtDhw6hWbNmCkdGN2/exP3793X+btnZ2aFOnTo4cuRInsTAxVDprajVagwePBj169dHlSpVlA7HYK1evRqnT5/GiRMnlA6FANy4cQPz58/H0KFD8d133+HEiRMYOHAgzMzMEBwcrHR4BmfEiBGIjY1FhQoVYGxsjJSUFEyaNAldunRROjSDd//+fQCAk5OTTrmTk5P2udzGBIjeSv/+/XHhwgUcOnRI6VAM1t27dzFo0CCEhobCwsJC6XAI8h8DHx8f/PjjjwCA999/HxcuXMCCBQuYACngzz//xIoVK7By5UpUrlwZ4eHhGDx4MFxdXfl+EJvAKPsGDBiArVu3Yu/evShVqpTS4RisU6dOITo6GjVq1ICJiQlMTEywf/9+zJo1CyYmJkhJSVE6RIPj4uKCSpUq6ZRVrFgRd+7cUSgiw/bNN99gxIgR6NixI6pWrYpu3bphyJAhmDx5stKhGTxnZ2cAwIMHD3TKHzx4oH0utzEBoiwTQmDAgAHYuHEj9uzZAw8PD6VDMmhNmjTB+fPnER4err35+PigS5cuCA8Ph7GxsdIhGpz69eunmxri6tWrcHNzUygiw/bixQsYGel+zRkbG0OtVisUEWl4eHjA2dkZYWFh2rLY2FgcO3YMdevWzZMY2ARGWda/f3+sXLkSf/31F2xsbLTttHZ2drC0tFQ4OsNjY2OTrv9VkSJFULx4cfbLUsiQIUNQr149/Pjjj2jfvj2OHz+ORYsWYdGiRUqHZpBatWqFSZMmoUyZMqhcuTLOnDmD6dOno2fPnkqHZhDi4uJw7do17eObN28iPDwcxYoVQ5kyZTB48GD88MMP8PLygoeHB8aMGQNXV1ftSLFcJ4iyCIDe25IlS5QOjf6fn5+fGDRokNJhGLQtW7aIKlWqCHNzc1GhQgWxaNEipUMyWLGxsWLQoEGiTJkywsLCQpQtW1aMGjVKJCYmKh2aQdi7d6/e74zg4GAhhBBqtVqMGTNGODk5CXNzc9GkSRNx5cqVPIuP8wARERGRwWEfICIiIjI4TICIiIjI4DABIiIiIoPDBIiIiIgMDhMgIiIiMjhMgIiIiMjgMAEiIiIig8MEiIgoAyqVCps2bVI6DCLKBUyAiChf6t69O1QqVbpb06ZNlQ6NiAoBrgVGRPlW06ZNsWTJEp0yc3NzhaIhosKENUBElG+Zm5vD2dlZ52Zvbw9ANk/Nnz8fzZo1g6WlJcqWLYt169bpvP78+fNo3LgxLC0tUbx4cXz++eeIi4vT2eb3339H5cqVYW5uDhcXFwwYMEDn+UePHqFNmzawsrKCl5cXNm/erH3u6dOn6NKlCxwdHWFpaQkvL690CRsR5U9MgIiowBozZgzatm2Ls2fPokuXLujYsSMuXboEAIiPj0dAQADs7e1x4sQJrF27Frt379ZJcObPn4/+/fvj888/x/nz57F582a89957OscYP3482rdvj3PnzqF58+bo0qULnjx5oj3+xYsXsWPHDly6dAnz58+Hg4ND3l0AInp7ebbsKhFRNgQHBwtjY2NRpEgRndukSZOEEEIAEF9++aXOa+rUqSP69u0rhBBi0aJFwt7eXsTFxWmf37ZtmzAyMhL3798XQgjh6uoqRo0alWEMAMTo0aO1j+Pi4gQAsWPHDiGEEK1atRI9evTImRMmojzFPkBElG81atQI8+fP1ykrVqyY9n7dunV1nqtbty7Cw8MBAJcuXYK3tzeKFCmifb5+/fpQq9W4cuUKVCoV7t27hyZNmrwxhmrVqmnvFylSBLa2toiOjgYA9O3bF23btsXp06fx8ccfIzAwEPXq1XurcyWivMUEiIjyrSJFiqRrksoplpaWWdrO1NRU57FKpYJarQYANGvWDLdv38b27dsRGhqKJk2aoH///pg6dWqOx0tEOYt9gIiowDp69Gi6xxUrVgQAVKxYEWfPnkV8fLz2+cOHD8PIyAjly5eHjY0N3N3dERYW9k4xODo6Ijg4GMuXL8eMGTOwaNGid9ofEeUN1gARUb6VmJiI+/fv65SZmJhoOxqvXbsWPj4++PDDD7FixQocP34cixcvBgB06dIFY8eORXBwMMaNG4eHDx/iq6++Qrdu3eDk5AQAGDduHL788kuUKFECzZo1w/Pnz3H48GF89dVXWYrv+++/R82aNVG5cmUkJiZi69at2gSMiPI3JkBElG/t3LkTLi4uOmXly5fH5cuXAcgRWqtXr0a/fv3g4uKCVatWoVKlSgAAKysr7Nq1C4MGDUKtWrVgZWWFtm3bYvr06dp9BQcHIyEhAb/++iuGDRsGBwcHtGvXLsvxmZmZYeTIkbh16xYsLS3h6+uL1atX58CZE1FuUwkhhNJBEBFll0qlwsaNGxEYGKh0KERUALEPEBERERkcJkBERERkcNgHiIgKJLbeE9G7YA0QERERGRwmQERERGRwmAARERGRwWECRERERAaHCRAREREZHCZAREREZHCYABEREZHBYQJEREREBocJEBERERmc/wP+8tuHjUtzwQAAAABJRU5ErkJggg==\n"
          },
          "metadata": {}
        }
      ],
      "source": [
        "import matplotlib.pyplot as plt\n",
        "val_acc_noise = history_noise.history[\"val_accuracy\"]\n",
        "val_acc_zeros = history_zeros.history[\"val_accuracy\"]\n",
        "epochs = range(1, 11)\n",
        "plt.plot(epochs, val_acc_noise, \"b-\",\n",
        "         label=\"Validation accuracy with noise channels\")\n",
        "plt.plot(epochs, val_acc_zeros, \"b--\",\n",
        "         label=\"Validation accuracy with zeros channels\")\n",
        "plt.title(\"Effect of noise channels on validation accuracy\")\n",
        "plt.xlabel(\"Epochs\")\n",
        "plt.ylabel(\"Accuracy\")\n",
        "plt.legend()\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ve-7nJARTk3N"
      },
      "source": [
        "### 딥러닝에서 일반화의 본질"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mMxFZdiQTk3O"
      },
      "source": [
        "**랜덤하게 섞은 레이블로 MNIST 모델 훈련하기**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T01:02:26.358294Z",
          "iopub.status.busy": "2022-05-21T01:02:26.357787Z",
          "iopub.status.idle": "2022-05-21T01:05:33.171731Z",
          "shell.execute_reply": "2022-05-21T01:05:33.170427Z"
        },
        "id": "8ocMHIMMTk3O",
        "outputId": "0415c269-69ef-4d23-9313-be55e0f5f10a",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/100\n",
            "375/375 [==============================] - 2s 5ms/step - loss: inf - accuracy: 0.1014 - val_loss: 2.2969 - val_accuracy: 0.1068\n",
            "Epoch 2/100\n",
            "375/375 [==============================] - 1s 4ms/step - loss: inf - accuracy: 0.1168 - val_loss: 2.3008 - val_accuracy: 0.1052\n",
            "Epoch 3/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: inf - accuracy: 0.1246 - val_loss: 2.3203 - val_accuracy: 0.0989\n",
            "Epoch 4/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: inf - accuracy: 0.1365 - val_loss: 2.3242 - val_accuracy: 0.1040\n",
            "Epoch 5/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: inf - accuracy: 0.1466 - val_loss: 2.3418 - val_accuracy: 0.1005\n",
            "Epoch 6/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: inf - accuracy: 0.1621 - val_loss: 2.3516 - val_accuracy: 0.1008\n",
            "Epoch 7/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: inf - accuracy: 0.1726 - val_loss: 2.3535 - val_accuracy: 0.1013\n",
            "Epoch 8/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: inf - accuracy: 0.1880 - val_loss: 2.3770 - val_accuracy: 0.0991\n",
            "Epoch 9/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: inf - accuracy: 0.2058 - val_loss: 2.3887 - val_accuracy: 0.1000\n",
            "Epoch 10/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: inf - accuracy: 0.2213 - val_loss: 2.4062 - val_accuracy: 0.1031\n",
            "Epoch 11/100\n",
            "375/375 [==============================] - 2s 4ms/step - loss: inf - accuracy: 0.2350 - val_loss: 2.4316 - val_accuracy: 0.0953\n",
            "Epoch 12/100\n",
            "375/375 [==============================] - 1s 4ms/step - loss: inf - accuracy: 0.2507 - val_loss: 2.4609 - val_accuracy: 0.0992\n",
            "Epoch 13/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: inf - accuracy: 0.2681 - val_loss: 2.4902 - val_accuracy: 0.0996\n",
            "Epoch 14/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: inf - accuracy: 0.2866 - val_loss: 2.5078 - val_accuracy: 0.0956\n",
            "Epoch 15/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: inf - accuracy: 0.3018 - val_loss: 2.5566 - val_accuracy: 0.1011\n",
            "Epoch 16/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: inf - accuracy: 0.3186 - val_loss: 2.5645 - val_accuracy: 0.0996\n",
            "Epoch 17/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: inf - accuracy: 0.3337 - val_loss: 2.6152 - val_accuracy: 0.0980\n",
            "Epoch 18/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: inf - accuracy: 0.3479 - val_loss: 2.6445 - val_accuracy: 0.0992\n",
            "Epoch 19/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: inf - accuracy: 0.3679 - val_loss: 2.6777 - val_accuracy: 0.0994\n",
            "Epoch 20/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: inf - accuracy: 0.3816 - val_loss: 2.7246 - val_accuracy: 0.0996\n",
            "Epoch 21/100\n",
            "375/375 [==============================] - 2s 4ms/step - loss: inf - accuracy: 0.3943 - val_loss: 2.7578 - val_accuracy: 0.1008\n",
            "Epoch 22/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: inf - accuracy: 0.4106 - val_loss: 2.8008 - val_accuracy: 0.0928\n",
            "Epoch 23/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: inf - accuracy: 0.4250 - val_loss: 2.8711 - val_accuracy: 0.0988\n",
            "Epoch 24/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: inf - accuracy: 0.4395 - val_loss: 2.8809 - val_accuracy: 0.0997\n",
            "Epoch 25/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: inf - accuracy: 0.4541 - val_loss: 2.9434 - val_accuracy: 0.0963\n",
            "Epoch 26/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: inf - accuracy: 0.4670 - val_loss: 2.9902 - val_accuracy: 0.0997\n",
            "Epoch 27/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: inf - accuracy: 0.4814 - val_loss: 3.0273 - val_accuracy: 0.0959\n",
            "Epoch 28/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: inf - accuracy: 0.4919 - val_loss: 3.0801 - val_accuracy: 0.0959\n",
            "Epoch 29/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: inf - accuracy: 0.5068 - val_loss: 3.1152 - val_accuracy: 0.0958\n",
            "Epoch 30/100\n",
            "375/375 [==============================] - 2s 4ms/step - loss: inf - accuracy: 0.5171 - val_loss: 3.1680 - val_accuracy: 0.0963\n",
            "Epoch 31/100\n",
            "375/375 [==============================] - 2s 4ms/step - loss: inf - accuracy: 0.5273 - val_loss: 3.2344 - val_accuracy: 0.0942\n",
            "Epoch 32/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: inf - accuracy: 0.5361 - val_loss: 3.2852 - val_accuracy: 0.0993\n",
            "Epoch 33/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 1.3389 - accuracy: 0.5454 - val_loss: 3.3223 - val_accuracy: 0.1019\n",
            "Epoch 34/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 1.3115 - accuracy: 0.5557 - val_loss: 3.3848 - val_accuracy: 0.0964\n",
            "Epoch 35/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 1.2803 - accuracy: 0.5625 - val_loss: 3.4160 - val_accuracy: 0.0977\n",
            "Epoch 36/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 1.2598 - accuracy: 0.5747 - val_loss: 3.4473 - val_accuracy: 0.0972\n",
            "Epoch 37/100\n",
            "375/375 [==============================] - 1s 4ms/step - loss: 1.2363 - accuracy: 0.5859 - val_loss: 3.5391 - val_accuracy: 0.0967\n",
            "Epoch 38/100\n",
            "375/375 [==============================] - 2s 4ms/step - loss: 1.2070 - accuracy: 0.5972 - val_loss: 3.5781 - val_accuracy: 0.0984\n",
            "Epoch 39/100\n",
            "375/375 [==============================] - 2s 5ms/step - loss: 1.1934 - accuracy: 0.6055 - val_loss: 3.6406 - val_accuracy: 0.0941\n",
            "Epoch 40/100\n",
            "375/375 [==============================] - 1s 4ms/step - loss: 1.1611 - accuracy: 0.6152 - val_loss: 3.6738 - val_accuracy: 0.0963\n",
            "Epoch 41/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 1.1377 - accuracy: 0.6216 - val_loss: 3.7656 - val_accuracy: 0.0981\n",
            "Epoch 42/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 1.1133 - accuracy: 0.6318 - val_loss: 3.7930 - val_accuracy: 0.1001\n",
            "Epoch 43/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 1.0918 - accuracy: 0.6362 - val_loss: 3.8398 - val_accuracy: 0.0992\n",
            "Epoch 44/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 1.0684 - accuracy: 0.6455 - val_loss: 3.8848 - val_accuracy: 0.0986\n",
            "Epoch 45/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 1.0488 - accuracy: 0.6514 - val_loss: 3.9512 - val_accuracy: 0.0992\n",
            "Epoch 46/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 1.0254 - accuracy: 0.6572 - val_loss: 3.9844 - val_accuracy: 0.0961\n",
            "Epoch 47/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 1.0039 - accuracy: 0.6694 - val_loss: 4.0312 - val_accuracy: 0.0987\n",
            "Epoch 48/100\n",
            "375/375 [==============================] - 1s 4ms/step - loss: 0.9854 - accuracy: 0.6738 - val_loss: 4.0938 - val_accuracy: 0.0992\n",
            "Epoch 49/100\n",
            "375/375 [==============================] - 2s 4ms/step - loss: 0.9678 - accuracy: 0.6787 - val_loss: 4.1719 - val_accuracy: 0.0966\n",
            "Epoch 50/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.9507 - accuracy: 0.6914 - val_loss: 4.2070 - val_accuracy: 0.1010\n",
            "Epoch 51/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.9282 - accuracy: 0.6938 - val_loss: 4.2695 - val_accuracy: 0.0953\n",
            "Epoch 52/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.9160 - accuracy: 0.7046 - val_loss: 4.3242 - val_accuracy: 0.0988\n",
            "Epoch 53/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.8979 - accuracy: 0.7119 - val_loss: 4.3750 - val_accuracy: 0.0974\n",
            "Epoch 54/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.8774 - accuracy: 0.7188 - val_loss: 4.4180 - val_accuracy: 0.0989\n",
            "Epoch 55/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.8638 - accuracy: 0.7266 - val_loss: 4.4922 - val_accuracy: 0.0989\n",
            "Epoch 56/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.8433 - accuracy: 0.7329 - val_loss: 4.5586 - val_accuracy: 0.0947\n",
            "Epoch 57/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.8262 - accuracy: 0.7373 - val_loss: 4.6094 - val_accuracy: 0.0986\n",
            "Epoch 58/100\n",
            "375/375 [==============================] - 2s 4ms/step - loss: 0.8140 - accuracy: 0.7441 - val_loss: 4.6719 - val_accuracy: 0.1001\n",
            "Epoch 59/100\n",
            "375/375 [==============================] - 1s 4ms/step - loss: 0.8013 - accuracy: 0.7451 - val_loss: 4.7461 - val_accuracy: 0.0947\n",
            "Epoch 60/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.7773 - accuracy: 0.7485 - val_loss: 4.7734 - val_accuracy: 0.0995\n",
            "Epoch 61/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.7700 - accuracy: 0.7559 - val_loss: 4.8398 - val_accuracy: 0.0987\n",
            "Epoch 62/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.7524 - accuracy: 0.7554 - val_loss: 4.8789 - val_accuracy: 0.0940\n",
            "Epoch 63/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.7422 - accuracy: 0.7622 - val_loss: 4.9492 - val_accuracy: 0.1021\n",
            "Epoch 64/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.7280 - accuracy: 0.7671 - val_loss: 4.9922 - val_accuracy: 0.0984\n",
            "Epoch 65/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.7139 - accuracy: 0.7705 - val_loss: 5.0312 - val_accuracy: 0.0986\n",
            "Epoch 66/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.7007 - accuracy: 0.7754 - val_loss: 5.0977 - val_accuracy: 0.0983\n",
            "Epoch 67/100\n",
            "375/375 [==============================] - 1s 4ms/step - loss: 0.6885 - accuracy: 0.7773 - val_loss: 5.1484 - val_accuracy: 0.1004\n",
            "Epoch 68/100\n",
            "375/375 [==============================] - 2s 4ms/step - loss: 0.6733 - accuracy: 0.7832 - val_loss: 5.1953 - val_accuracy: 0.0984\n",
            "Epoch 69/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.6611 - accuracy: 0.7881 - val_loss: 5.2891 - val_accuracy: 0.0978\n",
            "Epoch 70/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.6528 - accuracy: 0.7871 - val_loss: 5.3086 - val_accuracy: 0.1039\n",
            "Epoch 71/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.6396 - accuracy: 0.7915 - val_loss: 5.3945 - val_accuracy: 0.0946\n",
            "Epoch 72/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.6294 - accuracy: 0.7974 - val_loss: 5.4492 - val_accuracy: 0.0989\n",
            "Epoch 73/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.6157 - accuracy: 0.7979 - val_loss: inf - val_accuracy: 0.0959\n",
            "Epoch 74/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.6050 - accuracy: 0.8027 - val_loss: inf - val_accuracy: 0.0997\n",
            "Epoch 75/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.5977 - accuracy: 0.8052 - val_loss: inf - val_accuracy: 0.1002\n",
            "Epoch 76/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.5850 - accuracy: 0.8120 - val_loss: inf - val_accuracy: 0.0980\n",
            "Epoch 77/100\n",
            "375/375 [==============================] - 2s 4ms/step - loss: 0.5762 - accuracy: 0.8140 - val_loss: inf - val_accuracy: 0.0993\n",
            "Epoch 78/100\n",
            "375/375 [==============================] - 1s 4ms/step - loss: 0.5645 - accuracy: 0.8188 - val_loss: inf - val_accuracy: 0.0974\n",
            "Epoch 79/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.5532 - accuracy: 0.8271 - val_loss: inf - val_accuracy: 0.0996\n",
            "Epoch 80/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.5459 - accuracy: 0.8252 - val_loss: inf - val_accuracy: 0.0986\n",
            "Epoch 81/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.5356 - accuracy: 0.8252 - val_loss: inf - val_accuracy: 0.0993\n",
            "Epoch 82/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.5249 - accuracy: 0.8267 - val_loss: inf - val_accuracy: 0.0983\n",
            "Epoch 83/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.5181 - accuracy: 0.8320 - val_loss: inf - val_accuracy: 0.0987\n",
            "Epoch 84/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.5083 - accuracy: 0.8364 - val_loss: inf - val_accuracy: 0.0965\n",
            "Epoch 85/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.5024 - accuracy: 0.8379 - val_loss: inf - val_accuracy: 0.0967\n",
            "Epoch 86/100\n",
            "375/375 [==============================] - 1s 4ms/step - loss: 0.4919 - accuracy: 0.8433 - val_loss: inf - val_accuracy: 0.0981\n",
            "Epoch 87/100\n",
            "375/375 [==============================] - 2s 4ms/step - loss: 0.4827 - accuracy: 0.8506 - val_loss: inf - val_accuracy: 0.0983\n",
            "Epoch 88/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.4734 - accuracy: 0.8525 - val_loss: inf - val_accuracy: 0.1005\n",
            "Epoch 89/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.4670 - accuracy: 0.8560 - val_loss: inf - val_accuracy: 0.0980\n",
            "Epoch 90/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.4570 - accuracy: 0.8574 - val_loss: inf - val_accuracy: 0.1002\n",
            "Epoch 91/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.4504 - accuracy: 0.8574 - val_loss: inf - val_accuracy: 0.0987\n",
            "Epoch 92/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.4426 - accuracy: 0.8608 - val_loss: inf - val_accuracy: 0.1008\n",
            "Epoch 93/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.4343 - accuracy: 0.8662 - val_loss: inf - val_accuracy: 0.0963\n",
            "Epoch 94/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.4287 - accuracy: 0.8652 - val_loss: inf - val_accuracy: 0.0967\n",
            "Epoch 95/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.4216 - accuracy: 0.8726 - val_loss: inf - val_accuracy: 0.0980\n",
            "Epoch 96/100\n",
            "375/375 [==============================] - 2s 4ms/step - loss: 0.4146 - accuracy: 0.8711 - val_loss: inf - val_accuracy: 0.0960\n",
            "Epoch 97/100\n",
            "375/375 [==============================] - 1s 4ms/step - loss: 0.4070 - accuracy: 0.8799 - val_loss: inf - val_accuracy: 0.0964\n",
            "Epoch 98/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.3994 - accuracy: 0.8794 - val_loss: inf - val_accuracy: 0.0978\n",
            "Epoch 99/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.3960 - accuracy: 0.8828 - val_loss: inf - val_accuracy: 0.0974\n",
            "Epoch 100/100\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.3884 - accuracy: 0.8818 - val_loss: inf - val_accuracy: 0.0970\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.src.callbacks.History at 0x7bd523fc4280>"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ],
      "source": [
        "(train_images, train_labels), _ = mnist.load_data()\n",
        "train_images = train_images.reshape((60000, 28 * 28))\n",
        "train_images = train_images.astype(\"float32\") / 255\n",
        "\n",
        "random_train_labels = train_labels[:]\n",
        "np.random.shuffle(random_train_labels)\n",
        "\n",
        "model = keras.Sequential([\n",
        "    layers.Dense(512, activation=\"relu\"),\n",
        "    layers.Dense(10, activation=\"softmax\")\n",
        "])\n",
        "model.compile(optimizer=\"rmsprop\",\n",
        "              loss=\"sparse_categorical_crossentropy\",\n",
        "              metrics=[\"accuracy\"])\n",
        "model.fit(train_images, random_train_labels,\n",
        "          epochs=100,\n",
        "          batch_size=128,\n",
        "          validation_split=0.2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oXAyqCrXTk3P"
      },
      "source": [
        "#### 매니폴드 가설"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "L0Uvbr43Tk3P"
      },
      "source": [
        "#### 일반화의 원천인 보간"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pHMR5mNZTk3P"
      },
      "source": [
        "#### 딥러닝이 작동하는 이유"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eb0YYZR4Tk3P"
      },
      "source": [
        "#### 가장 중요한 훈련 데이터"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PunfvZ7uTk3Q"
      },
      "source": [
        "## 머신 러닝 모델 평가"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5P4VJfMFTk3Q"
      },
      "source": [
        "### 훈련, 검증, 테스트 세트"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mrwB851-Tk3Q"
      },
      "source": [
        "#### 단순 홀드아웃 검증"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1sEzzYjjTk3R"
      },
      "source": [
        "#### K-겹 교차 검증"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XYrpFDZQTk3R"
      },
      "source": [
        "#### 셔플링을 사용한 반복 K-겹 교차 검증"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kngFjE0QTk3R"
      },
      "source": [
        "### 상식 수준의 기준점을 넘기"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "czziT5WVTk3R"
      },
      "source": [
        "### 모델 평가에 대해 유념해야 할 점"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "p6nofQfTTk3R"
      },
      "source": [
        "## 훈련 성능 향상하기"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XtWaoT4BTk3S"
      },
      "source": [
        "### 경사 하강법의 핵심 파라미터 튜닝하기"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pRY-dxWTTk3S"
      },
      "source": [
        "**잘못된 높은 학습률로 MNIST 모델 훈련하기**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T01:05:33.177460Z",
          "iopub.status.busy": "2022-05-21T01:05:33.177117Z",
          "iopub.status.idle": "2022-05-21T01:05:52.585061Z",
          "shell.execute_reply": "2022-05-21T01:05:52.584241Z"
        },
        "id": "ZqvcFMPRTk3T",
        "outputId": "b5c66a83-edb6-4a56-ffd7-5999ba89a2dd",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "375/375 [==============================] - 2s 4ms/step - loss: nan - accuracy: 0.0988 - val_loss: nan - val_accuracy: 0.0995\n",
            "Epoch 2/10\n",
            "375/375 [==============================] - 1s 3ms/step - loss: nan - accuracy: 0.0988 - val_loss: nan - val_accuracy: 0.0995\n",
            "Epoch 3/10\n",
            "375/375 [==============================] - 2s 4ms/step - loss: nan - accuracy: 0.0988 - val_loss: nan - val_accuracy: 0.0995\n",
            "Epoch 4/10\n",
            "375/375 [==============================] - 1s 4ms/step - loss: nan - accuracy: 0.0986 - val_loss: nan - val_accuracy: 0.0995\n",
            "Epoch 5/10\n",
            "375/375 [==============================] - 1s 3ms/step - loss: nan - accuracy: 0.0988 - val_loss: nan - val_accuracy: 0.0995\n",
            "Epoch 6/10\n",
            "375/375 [==============================] - 1s 3ms/step - loss: nan - accuracy: 0.0983 - val_loss: nan - val_accuracy: 0.0995\n",
            "Epoch 7/10\n",
            "375/375 [==============================] - 1s 3ms/step - loss: nan - accuracy: 0.0986 - val_loss: nan - val_accuracy: 0.0995\n",
            "Epoch 8/10\n",
            "375/375 [==============================] - 1s 3ms/step - loss: nan - accuracy: 0.0987 - val_loss: nan - val_accuracy: 0.0995\n",
            "Epoch 9/10\n",
            "375/375 [==============================] - 1s 3ms/step - loss: nan - accuracy: 0.0983 - val_loss: nan - val_accuracy: 0.0995\n",
            "Epoch 10/10\n",
            "375/375 [==============================] - 1s 3ms/step - loss: nan - accuracy: 0.0987 - val_loss: nan - val_accuracy: 0.0995\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.src.callbacks.History at 0x7bd53c7b7940>"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ],
      "source": [
        "(train_images, train_labels), _ = mnist.load_data()\n",
        "train_images = train_images.reshape((60000, 28 * 28))\n",
        "train_images = train_images.astype(\"float32\") / 255\n",
        "\n",
        "model = keras.Sequential([\n",
        "    layers.Dense(512, activation=\"relu\"),\n",
        "    layers.Dense(10, activation=\"softmax\")\n",
        "])\n",
        "model.compile(optimizer=keras.optimizers.RMSprop(1.),\n",
        "              loss=\"sparse_categorical_crossentropy\",\n",
        "              metrics=[\"accuracy\"])\n",
        "model.fit(train_images, train_labels,\n",
        "          epochs=10,\n",
        "          batch_size=128,\n",
        "          validation_split=0.2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "77iXsoAYTk3U"
      },
      "source": [
        "**같은 모델을 적절한 학습률로 훈련하기**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T01:05:52.588549Z",
          "iopub.status.busy": "2022-05-21T01:05:52.588260Z",
          "iopub.status.idle": "2022-05-21T01:06:11.526008Z",
          "shell.execute_reply": "2022-05-21T01:06:11.525112Z"
        },
        "id": "TQAqlYm0Tk3U",
        "outputId": "7d881a98-9bb5-48d9-8a0c-cdcc39e2e79e",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "375/375 [==============================] - 2s 4ms/step - loss: 0.3750 - accuracy: 0.9238 - val_loss: 0.1970 - val_accuracy: 0.9404\n",
            "Epoch 2/10\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.1256 - accuracy: 0.9854 - val_loss: 0.1469 - val_accuracy: 0.9648\n",
            "Epoch 3/10\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.0949 - accuracy: 0.9912 - val_loss: 0.1501 - val_accuracy: 0.9678\n",
            "Epoch 4/10\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.0817 - accuracy: 0.9941 - val_loss: 0.1388 - val_accuracy: 0.9692\n",
            "Epoch 5/10\n",
            "375/375 [==============================] - 2s 4ms/step - loss: 0.0656 - accuracy: 0.9961 - val_loss: 0.1426 - val_accuracy: 0.9746\n",
            "Epoch 6/10\n",
            "375/375 [==============================] - 1s 4ms/step - loss: 0.0600 - accuracy: 0.9976 - val_loss: 0.1639 - val_accuracy: 0.9746\n",
            "Epoch 7/10\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.0491 - accuracy: 0.9980 - val_loss: 0.1685 - val_accuracy: 0.9707\n",
            "Epoch 8/10\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.0479 - accuracy: 0.9985 - val_loss: 0.1835 - val_accuracy: 0.9731\n",
            "Epoch 9/10\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.0406 - accuracy: 0.9985 - val_loss: 0.2007 - val_accuracy: 0.9780\n",
            "Epoch 10/10\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.0349 - accuracy: 0.9985 - val_loss: 0.1790 - val_accuracy: 0.9780\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.src.callbacks.History at 0x7bd5386b2920>"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ],
      "source": [
        "model = keras.Sequential([\n",
        "    layers.Dense(512, activation=\"relu\"),\n",
        "    layers.Dense(10, activation=\"softmax\")\n",
        "])\n",
        "model.compile(optimizer=keras.optimizers.RMSprop(1e-2),\n",
        "              loss=\"sparse_categorical_crossentropy\",\n",
        "              metrics=[\"accuracy\"])\n",
        "model.fit(train_images, train_labels,\n",
        "          epochs=10,\n",
        "          batch_size=128,\n",
        "          validation_split=0.2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9IVJeVeOTk3V"
      },
      "source": [
        "### 구조에 대해 더 나은 가정하기"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Br1F2jcwTk3V"
      },
      "source": [
        "### 모델 용량 늘리기"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ubvdcv_-Tk3V"
      },
      "source": [
        "**MNIST 데이터를 사용한 간단한 로지스틱 회귀 모델**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T01:06:11.530131Z",
          "iopub.status.busy": "2022-05-21T01:06:11.529560Z",
          "iopub.status.idle": "2022-05-21T01:06:25.955536Z",
          "shell.execute_reply": "2022-05-21T01:06:25.954654Z"
        },
        "id": "xl5ejzSRTk3W",
        "outputId": "e4b3a284-857c-4caf-bc7b-db8795aabb8d",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/20\n",
            "375/375 [==============================] - 2s 4ms/step - loss: 0.6670 - accuracy: 0.8389 - val_loss: 0.3584 - val_accuracy: 0.9067\n",
            "Epoch 2/20\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.3513 - accuracy: 0.9136 - val_loss: 0.3086 - val_accuracy: 0.9155\n",
            "Epoch 3/20\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.3174 - accuracy: 0.9209 - val_loss: 0.2930 - val_accuracy: 0.9180\n",
            "Epoch 4/20\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.3018 - accuracy: 0.9292 - val_loss: 0.2844 - val_accuracy: 0.9219\n",
            "Epoch 5/20\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.2937 - accuracy: 0.9292 - val_loss: 0.2793 - val_accuracy: 0.9219\n",
            "Epoch 6/20\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.2866 - accuracy: 0.9321 - val_loss: 0.2749 - val_accuracy: 0.9238\n",
            "Epoch 7/20\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.2822 - accuracy: 0.9331 - val_loss: 0.2722 - val_accuracy: 0.9268\n",
            "Epoch 8/20\n",
            "375/375 [==============================] - 1s 4ms/step - loss: 0.2783 - accuracy: 0.9385 - val_loss: 0.2700 - val_accuracy: 0.9268\n",
            "Epoch 9/20\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.2749 - accuracy: 0.9375 - val_loss: 0.2703 - val_accuracy: 0.9268\n",
            "Epoch 10/20\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.2722 - accuracy: 0.9385 - val_loss: 0.2676 - val_accuracy: 0.9292\n",
            "Epoch 11/20\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.2700 - accuracy: 0.9404 - val_loss: 0.2668 - val_accuracy: 0.9282\n",
            "Epoch 12/20\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.2671 - accuracy: 0.9414 - val_loss: 0.2673 - val_accuracy: 0.9292\n",
            "Epoch 13/20\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.2661 - accuracy: 0.9438 - val_loss: 0.2654 - val_accuracy: 0.9268\n",
            "Epoch 14/20\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.2649 - accuracy: 0.9429 - val_loss: 0.2649 - val_accuracy: 0.9312\n",
            "Epoch 15/20\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.2627 - accuracy: 0.9438 - val_loss: 0.2637 - val_accuracy: 0.9268\n",
            "Epoch 16/20\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.2620 - accuracy: 0.9414 - val_loss: 0.2632 - val_accuracy: 0.9307\n",
            "Epoch 17/20\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.2603 - accuracy: 0.9438 - val_loss: 0.2632 - val_accuracy: 0.9292\n",
            "Epoch 18/20\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.2610 - accuracy: 0.9434 - val_loss: 0.2627 - val_accuracy: 0.9292\n",
            "Epoch 19/20\n",
            "375/375 [==============================] - 1s 4ms/step - loss: 0.2590 - accuracy: 0.9399 - val_loss: 0.2620 - val_accuracy: 0.9287\n",
            "Epoch 20/20\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.2576 - accuracy: 0.9438 - val_loss: 0.2625 - val_accuracy: 0.9282\n"
          ]
        }
      ],
      "source": [
        "model = keras.Sequential([layers.Dense(10, activation=\"softmax\")])\n",
        "model.compile(optimizer=\"rmsprop\",\n",
        "              loss=\"sparse_categorical_crossentropy\",\n",
        "              metrics=[\"accuracy\"])\n",
        "history_small_model = model.fit(\n",
        "    train_images, train_labels,\n",
        "    epochs=20,\n",
        "    batch_size=128,\n",
        "    validation_split=0.2)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T01:06:25.959685Z",
          "iopub.status.busy": "2022-05-21T01:06:25.959399Z",
          "iopub.status.idle": "2022-05-21T01:06:26.277516Z",
          "shell.execute_reply": "2022-05-21T01:06:26.276506Z"
        },
        "id": "-yiFhvT4Tk3W",
        "outputId": "ad980ca7-d656-4844-88f0-7a189b625f53",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 472
        }
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkAAAAHHCAYAAABXx+fLAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABmc0lEQVR4nO3deVhUZfsH8O+A7LvsKILiihsJgriXKCrlnuhriktq7oaZmr9csiLX3NdSUysVt0wNU1JfU1ySTE0jM9wF3FhVQOb5/XFeRkaGVeDMMN/Pdc3FcOY5z9xnzgxzc55NIYQQICIiItIjBnIHQERERFTRmAARERGR3mECRERERHqHCRARERHpHSZAREREpHeYABEREZHeYQJEREREeocJEBEREekdJkBERESkd5gA6Yj09HS8++67cHFxgUKhwMSJEwEAiYmJ6NOnD+zt7aFQKLB48WJZ4yyJgo5JE09PTwwePLjCYitMVFQUfHx8YGpqCoVCgeTkZADA5s2bUb9+fRgZGcHW1hYA0L59e7Rv377Ez6FQKDBr1qwyi7kyGDx4MDw9PUu1b2nPgy7Sps+KNnr5vXD9+nUoFAps3LixyH1f5T1YkI0bN0KhUOD69etlWm9x6Pt7pYrcAeizjRs3YsiQIQU+HhMTgxYtWgAAPv/8c2zcuBEff/wxvLy80KBBAwDA+++/j4MHD2LmzJlwcXGBn59fmcf5+eefw9vbGz169CjzejUdkzZ7+PAh+vbti4YNG2LFihUwMTGBhYUF/vrrLwwePBidO3fG1KlTYW5uLneoRfruu++QlJRUaOJJuu/y5cvYvn17uXx5U/GV199RKj0mQFrgk08+Qc2aNfNtr127tur+L7/8ghYtWmDmzJlqZX755Rd0794dH3zwQbnF9/nnn6NPnz5l/sEt6Jg0iYuLg4GB/Bcsz549i7S0NMyZMwdBQUGq7UePHoVSqcSSJUvUztvPP/9cqud5+vQpqlQp34/nd999h0uXLjEBqmRe/qxcvnwZs2fPRvv27ZkAaeDh4YGnT5/CyMioXJ+noL+jAwcORL9+/WBiYlKuz0/5MQHSAl26dCnyyk1SUhK8vb01bs9tbtE1BR2TJtryxyEpKQkA8r3mBW03NjYu1fOYmpqWaj8ibfms6AqFQiHr583Q0BCGhoayPb8+k/9fairU0aNHoVAoEB8fj/3790OhUKjaqxUKBYQQWLFihWp7ruTkZEycOBHu7u4wMTFB7dq1MXfuXCiVSrX6c69aNG7cGKampnB0dETnzp3x22+/AZD+OGRkZOCbb75RPUdRbcZJSUkYNmwYnJ2dYWpqiqZNm+Kbb74p8pgKawN/ua069/hPnDiB8PBwODo6wsLCAj179sT9+/fV9v3tt98QHBwMBwcHmJmZoWbNmhg6dGi+eI4ePaq238t9A9q3b4+wsDAAQPPmzVWvhaenp+oqlqOjo1r/HU19T549e4ZZs2ahbt26MDU1haurK3r16oVr166pymjqA3Tnzh0MHToUzs7OMDExQcOGDbF+/Xq1MrnHsn37dnz22WeoXr06TE1N0aFDB/zzzz+qcu3bt8f+/ftx48YN1etf1NUBhUKBsWPHIjIyEt7e3jAzM0NgYCAuXrwIAFizZg1q164NU1NTtG/fXuP5jIyMhK+vL8zMzODg4IB33nkHd+7cyVduz549aNSoEUxNTdGoUSPs3r1bY0xKpRKLFy9Gw4YNYWpqCmdnZ4wcORKPHz8u9FgKs2XLFvj7+8Pc3Bx2dnZo27at2pW8H374ASEhIXBzc4OJiQm8vLwwZ84c5OTkqNXTvn17NGrUCOfOnUPLli1V773Vq1erlcvKysKMGTPg6+sLGxsbWFhYoE2bNjhy5IjG4y3s8wqof1Y2btyIt99+GwDw+uuvq8710aNHERYWBgcHB2RnZ+d7nk6dOqFevXpFvlbFOZ+DBw+GpaUl7ty5gx49esDS0hKOjo744IMP8r1mL3vzzTdRq1YtjY8FBgaq/eO4YcMGvPHGG3BycoKJiQm8vb2xatWqIo+hoD5AxX0PLliwAC1btoS9vT3MzMzg6+uLHTt2qJUp7O9oQX2AVq5ciYYNG8LExARubm4YM2aMqr9hrtz32OXLl/H666/D3Nwc1apVw7x584o87oL8+++/ePvtt1G1alWYm5ujRYsW2L9/f75yy5YtQ8OGDVWfEz8/P3z33Xeqx9PS0jBx4kR4enrCxMQETk5O6NixI2JjY0sdW1njFSAtkJKSggcPHqhtUygUsLe3R4MGDbB582a8//77qF69OiZNmgQAeO2117B582YMHDgQHTt2xKBBg1T7PnnyBO3atcOdO3cwcuRI1KhRAydPnsS0adNw7949tY7Sw4YNw8aNG9GlSxe8++67eP78OY4fP45Tp07Bz88Pmzdvxrvvvgt/f3+MGDECAODl5VXgsTx9+hTt27fHP//8g7Fjx6JmzZqIjIzE4MGDkZycjAkTJhR4TI6OjiV+7caNGwc7OzvMnDkT169fx+LFizF27Fhs27YNgJSMderUCY6Ojpg6dSpsbW1x/fp17Nq1q8TPNX36dNSrVw9r165VNVt6eXmhR48e2LRpE3bv3o1Vq1bB0tISTZo00VhHTk4O3nzzTURHR6Nfv36YMGEC0tLScOjQIVy6dKnA1zYxMREtWrRQJSGOjo746aefMGzYMKSmpuZrxvriiy9gYGCADz74ACkpKZg3bx4GDBiA06dPq44lJSUFt2/fxpdffgkAsLS0LPI1OH78OPbu3YsxY8YAACIiIvDmm2/iww8/xMqVKzF69Gg8fvwY8+bNw9ChQ/HLL7+o9s3t89a8eXNEREQgMTERS5YswYkTJ/D777+rrp79/PPP6N27N7y9vREREYGHDx9iyJAhqF69er54Ro4cqap3/PjxiI+Px/Lly/H777/jxIkTJW7WmD17NmbNmoWWLVvik08+gbGxMU6fPo1ffvkFnTp1Uh2HpaUlwsPDYWlpiV9++QUzZsxAamoq5s+fr1bf48eP0bVrV/Tt2xf9+/fH9u3bMWrUKBgbG6uS8NTUVHz11Vfo378/hg8fjrS0NHz99dcIDg7GmTNn4OPjo6qvqM/ry9q2bYvx48dj6dKl+Oijj1T97Bo0aICBAwdi06ZNOHjwIN58803VPgkJCfjll1+KbJou7vkEpPd9cHAwAgICsGDBAhw+fBgLFy6El5cXRo0aVeBzhIaGYtCgQTh79iyaN2+u2n7jxg2cOnVK7fVetWoVGjZsiG7duqFKlSr48ccfMXr0aCiVStX7tbhK8h5csmQJunXrhgEDBiArKwtbt27F22+/jX379iEkJAQASvx3dNasWZg9ezaCgoIwatQoxMXFYdWqVTh79my+9/Xjx4/RuXNn9OrVC3379sWOHTswZcoUNG7cGF26dCnRcScmJqJly5Z48uQJxo8fD3t7e3zzzTfo1q0bduzYgZ49ewIA1q1bh/Hjx6NPnz6YMGECnj17hgsXLuD06dP4z3/+AwB47733sGPHDowdOxbe3t54+PAhfv31V1y5cgXNmjUrUVzlRpBsNmzYIABovJmYmKiV9fDwECEhIfnqACDGjBmjtm3OnDnCwsJC/P3332rbp06dKgwNDcXNmzeFEEL88ssvAoAYP358vnqVSqXqvoWFhQgLCyvWMS1evFgAEFu2bFFty8rKEoGBgcLS0lKkpqYWeUyaeHh4qMWQ+9oFBQWpxfr+++8LQ0NDkZycLIQQYvfu3QKAOHv2bIF1HzlyRAAQR44cUdseHx8vAIgNGzbke96X65s5c6YAIO7fv6+2vV27dqJdu3aq39evXy8AiEWLFuWLI+9xABAzZ85U/T5s2DDh6uoqHjx4oLZPv379hI2NjXjy5InasTRo0EBkZmaqyi1ZskQAEBcvXlRtCwkJER4eHhpfE01y35fx8fGqbWvWrBEAhIuLi9q5nTZtmgCgKpuVlSWcnJxEo0aNxNOnT1Xl9u3bJwCIGTNmqLb5+PgIV1dX1TkUQoiff/5ZAFCL9/jx4wKA+Pbbb9XijIqKyrf95fOgydWrV4WBgYHo2bOnyMnJUXss77nJfa3zGjlypDA3NxfPnj1Te04AYuHChaptmZmZwsfHRzg5OYmsrCwhhBDPnz9XO1dCCPH48WPh7Owshg4dqtpW3M/ry5+VyMhIje/vnJwcUb16dREaGqq2fdGiRUKhUIh///033/PkKsn5DAsLEwDEJ598olbHa6+9Jnx9fQt8DiGESElJESYmJmLSpElq2+fNmycUCoW4ceOGapum8xIcHCxq1aqltu3l94Kmz3lx34OanjcrK0s0atRIvPHGG2rbC/o7mvs3JfezkpSUJIyNjUWnTp3U3ofLly8XAMT69evVjgWA2LRpk2pbZmamcHFxEb179873XC97+b0yceJEAUAcP35ctS0tLU3UrFlTeHp6quLp3r27aNiwYaF129jY5Ptu0jZsAtMCK1aswKFDh9RuP/30U6nri4yMRJs2bWBnZ4cHDx6obkFBQcjJycF///tfAMDOnTuhUCg0/qeXtzmtJA4cOAAXFxf0799ftc3IyAjjx49Heno6jh07VrqDKsCIESPUYm3Tpg1ycnJw48YNAC/65Ozbt0/jpf6KtnPnTjg4OGDcuHH5HivoNRdCYOfOnXjrrbcghFA7p8HBwUhJScl3WXnIkCFq/Y/atGkDQLq8/So6dOig1lQWEBAAAOjduzesrKzybc99vt9++w1JSUkYPXq0Wn+LkJAQ1K9fX3WJ/d69ezh//jzCwsJgY2OjKtexY8d8/cUiIyNhY2ODjh07qr0mvr6+sLS01NiEVJg9e/ZAqVRixowZ+Trc5z03ZmZmqvtpaWl48OAB2rRpgydPnuCvv/5S269KlSoYOXKk6ndjY2OMHDkSSUlJOHfuHACpD0juuVIqlXj06BGeP38OPz8/tfNa1p9XAwMDDBgwAHv37kVaWppq+7fffouWLVtqHJiRq7jnM6/33ntP7fc2bdoU+X60trZGly5dsH37dgghVNu3bduGFi1aoEaNGqptec9L7lX1du3a4d9//0VKSkqhz5NXSd6DLz/v48ePkZKSgjZt2pS6qefw4cPIysrCxIkT1d6Hw4cPh7W1db7X1tLSEu+8847qd2NjY/j7+5fqs37gwAH4+/ujdevWavWPGDEC169fx+XLlwFIf1dv376Ns2fPFliXra0tTp8+jbt375Y4jorCBEgL+Pv7IygoSO32+uuvl7q+q1evIioqCo6Ojmq33FFLuR12r127Bjc3N1StWrVMjgOQLk3XqVMn3xdI7qX33MSkrOT9AwgAdnZ2AKDqA9KuXTv07t0bs2fPhoODA7p3744NGzYgMzOzTOMormvXrqFevXolGuF1//59JCcnY+3atfnOae40CrnnNFdRr0tpvVxv7heEu7u7xu25z5d73jX1K6lfv77q8dyfderUyVfu5X2vXr2KlJQUODk55Xtd0tPT870mRbl27RoMDAyK7Jj/559/omfPnrCxsYG1tTUcHR1VX0Avf9G6ubnBwsJCbVvdunUBQK3PxzfffIMmTZrA1NQU9vb2cHR0xP79+9XqK4/P66BBg/D06VNV/5a4uDicO3cOAwcOLHS/4p7PXLn9lfKys7Mr1vsxNDQUt27dQkxMDADpdTh37hxCQ0PVyp04cQJBQUGwsLCAra0tHB0d8dFHHwHIf16Kc2zFeQ8C0j9XLVq0gKmpKapWrQpHR0esWrWqRM+p6flffi5jY2PUqlUr32tbvXr1fAlwcV9bTc+t6Rhf/vs9ZcoUWFpawt/fH3Xq1MGYMWNw4sQJtX3mzZuHS5cuwd3dHf7+/pg1a9Yr/wNW1tgHqBJSKpXo2LEjPvzwQ42P5/4BrgwKGj2R+9+iQqHAjh07cOrUKfz44484ePAghg4dioULF+LUqVOwtLQs8L/nojpoVpTcjuvvvPOOqhP2y17uc1TU61JaBdVbXs9XGKVSCScnJ3z77bcaHy9Nn7KiJCcno127drC2tsYnn3wCLy8vmJqaIjY2FlOmTMk3yKA4tmzZgsGDB6NHjx6YPHkynJycYGhoiIiICLWO8eXB29sbvr6+2LJlCwYNGoQtW7bA2NgYffv2LdPneZVRTm+99RbMzc2xfft2tGzZEtu3b4eBgYGqczcgJUUdOnRA/fr1sWjRIri7u8PY2BgHDhzAl19+WarzUhzHjx9Ht27d0LZtW6xcuRKurq4wMjLChg0b1DoElyc5PnsNGjRAXFwc9u3bh6ioKOzcuRMrV67EjBkzMHv2bABA37590aZNG+zevRs///wz5s+fj7lz52LXrl0l7ptUXpgAVUJeXl5IT09Xm6emoHIHDx7Eo0ePCv2vsiSX1z08PHDhwgUolUq1q0C5TQMeHh7FrqsstWjRAi1atMBnn32G7777DgMGDMDWrVvx7rvvqq6OvDzCoqyvVgHSa3769GlkZ2cXu4Ouo6MjrKyskJOTU+Q5LYnSNnOWRu55j4uLwxtvvKH2WFxcnOrx3J9Xr17NV0dcXJza715eXjh8+DBatWql1gxRWl5eXlAqlbh8+bJax+O8jh49iocPH2LXrl1o27atant8fLzG8nfv3kVGRobaVaC///4bAFRNiTt27ECtWrWwa9cutXPyclNXcT+vLyvqPA8aNAjh4eG4d+8evvvuO4SEhKg+EwUp7vksCxYWFnjzzTcRGRmJRYsWYdu2bWjTpg3c3NxUZX788UdkZmZi7969alcpS9oMCpTsPbhz506Ympri4MGDatMPbNiwId++xf285X1t846Ay8rKQnx8fJn+DdD03C8fI6D577eFhQVCQ0MRGhqKrKws9OrVC5999hmmTZumahZ1dXXF6NGjMXr0aCQlJaFZs2b47LPPtCYBYhNYJdS3b1/ExMTg4MGD+R5LTk7G8+fPAUj9NoQQqow9r7z/PVhYWORLDgrStWtXJCQkqEZhAcDz58+xbNkyWFpaol27diU8mlfz+PHjfP8J5X655TaDeXh4wNDQUNU3KtfKlSvLPJ7evXvjwYMHWL58eb7HCvqPzdDQEL1798bOnTtx6dKlfI+/POy/uCwsLEp9mb6k/Pz84OTkhNWrV6s1P/7000+4cuWKarSMq6srfHx88M0336jFdujQIVX/g1x9+/ZFTk4O5syZk+/5nj9/Xuz3bK4ePXrAwMAAn3zySb4rBrnnJve/7bznKisrq8D3yvPnz7FmzRq1smvWrIGjoyN8fX0LrPP06dOqJp9cxf28viw3+Sro9ejfvz8UCgUmTJiAf//9V60/SUGKez7LSmhoKO7evYuvvvoKf/zxR77mL02vYUpKisZEpCgleQ8aGhpCoVCoXS2+fv069uzZk6/e4v4dDQoKgrGxMZYuXap2PF9//TVSUlLK/LXNq2vXrjhz5ozaey8jIwNr166Fp6enqnn44cOHavsZGxvD29sbQghkZ2cjJycn398WJycnuLm5ydb9QBNeAdICP/30U77OkwDQsmXLAufAKMzkyZOxd+9evPnmmxg8eDB8fX2RkZGBixcvYseOHbh+/TocHBzw+uuvY+DAgVi6dCmuXr2Kzp07Q6lU4vjx43j99dcxduxYAICvry8OHz6MRYsWwc3NDTVr1lR1cn3ZiBEjsGbNGgwePBjnzp2Dp6cnduzYgRMnTmDx4sVqHWUrwjfffIOVK1eiZ8+e8PLyQlpaGtatWwdra2t07doVgNRf5e2338ayZcugUCjg5eWFffv2lbgPSXEMGjQImzZtQnh4OM6cOYM2bdogIyMDhw8fxujRo9G9e3eN+33xxRc4cuQIAgICMHz4cHh7e+PRo0eIjY3F4cOH8ejRoxLH4uvri23btiE8PBzNmzeHpaUl3nrrrVc9RI2MjIwwd+5cDBkyBO3atUP//v1Vw6Y9PT3x/vvvq8pGREQgJCQErVu3xtChQ/Ho0SPVnCPp6emqcu3atcPIkSMRERGB8+fPo1OnTjAyMsLVq1cRGRmJJUuWoE+fPsWOsXbt2pg+fTrmzJmDNm3aoFevXjAxMcHZs2fh5uaGiIgItGzZEnZ2dggLC8P48eOhUCiwefPmAhMQNzc3zJ07F9evX0fdunWxbds2nD9/HmvXrlVdAXzzzTexa9cu9OzZEyEhIYiPj8fq1avh7e2tdrzF/by+zMfHB4aGhpg7dy5SUlJgYmKimi8HgGouocjISNja2hbrC7Yk57MsdO3aFVZWVvjggw9U/xDk1alTJxgbG+Ott97CyJEjkZ6ejnXr1sHJyQn37t0r8fMV9z0YEhKCRYsWoXPnzvjPf/6DpKQkrFixArVr18aFCxfU6izu31FHR0dMmzYNs2fPRufOndGtWzfExcVh5cqVaN68ebES1NKaOnUqvv/+e3Tp0gXjx49H1apV8c033yA+Ph47d+5UXdXv1KkTXFxc0KpVKzg7O+PKlStYvnw5QkJCYGVlheTkZFSvXh19+vRB06ZNYWlpicOHD+Ps2bNYuHBhucVfYhU76IzyKmwYPF4allmSYfBCSEMXp02bJmrXri2MjY2Fg4ODaNmypViwYIFq+K0Q0hDc+fPni/r16wtjY2Ph6OgounTpIs6dO6cq89dff4m2bdsKMzMzAaDIIfGJiYliyJAhwsHBQRgbG4vGjRurHUtRx6RJQcPgXx6O/vKQ9tjYWNG/f39Ro0YNYWJiIpycnMSbb74pfvvtN7X97t+/L3r37i3Mzc2FnZ2dGDlypLh06VKZD4MXQho2O336dFGzZk1hZGQkXFxcRJ8+fcS1a9dUZfDSMHghpNd1zJgxwt3dXbVfhw4dxNq1a/Mdf2RkpNq+mob6pqeni//85z/C1tZW4/Del2l6r+XWO3/+fLXtBcWxbds28dprrwkTExNRtWpVMWDAAHH79u18z7Vz507RoEEDYWJiIry9vcWuXbtEWFiYxhjXrl0rfH19hZmZmbCyshKNGzcWH374obh7966qTHGGwedav369KkY7OzvRrl07cejQIdXjJ06cEC1atBBmZmbCzc1NfPjhh+LgwYP5hpq3a9dONGzYUPz2228iMDBQmJqaCg8PD7F8+XK151MqleLzzz8XHh4ewsTERLz22mti3759Go+3OJ/Xlz8rQgixbt06UatWLWFoaKhxSPz27dsFADFixIhivUa5inM+w8LChIWFRb59cz8zxTVgwADV1Bea7N27VzRp0kSYmpoKT09PMXfuXNW0E3mnbijOMHghiv8e/Prrr0WdOnWEiYmJqF+/vtiwYYPGYyvo7+jLw+BzLV++XNSvX18YGRkJZ2dnMWrUKPH48WO1MrnvsZcV9Fl5mab3yrVr10SfPn2Era2tMDU1Ff7+/mLfvn1qZdasWSPatm0r7O3thYmJifDy8hKTJ08WKSkpQghpKP7kyZNF06ZNhZWVlbCwsBBNmzYVK1euLDKmiqQQohx7ShER6an27dvjwYMHGpsttc0PP/yAHj164L///a9qygSiyo59gIiI9Ny6detQq1YttflfiCo79gEiItJTW7duxYULF7B//34sWbKkQkcGEsmNCRARkZ7q378/LC0tMWzYMIwePVrucIgqFPsAERERkd5hHyAiIiLSO0yAiIiISO+wD5AGSqUSd+/ehZWVFTsFEhER6QghBNLS0uDm5pZvUe6XMQHS4O7du/lWtyYiIiLdcOvWLVSvXr3QMkyANMhdruHWrVuwtraWORoiIiIqjtTUVLi7uxdr2SUmQBrkNntZW1szASIiItIxxem+wk7QREREpHeYABEREZHeYQJEREREeod9gIiIqNzl5OQgOztb7jBIxxkZGcHQ0LBM6pI9AVqxYgXmz5+PhIQENG3aFMuWLYO/v7/Gsrt27cLnn3+Of/75B9nZ2ahTpw4mTZqEgQMHqpW7cuUKpkyZgmPHjuH58+fw9vbGzp07UaNGjYo4JCIi+h8hBBISEpCcnCx3KFRJ2NrawsXF5ZXn6ZM1Adq2bRvCw8OxevVqBAQEYPHixQgODkZcXBycnJzyla9atSqmT5+O+vXrw9jYGPv27cOQIUPg5OSE4OBgAMC1a9fQunVrDBs2DLNnz4a1tTX+/PNPmJqaVvThERHpvdzkx8nJCebm5pxclkpNCIEnT54gKSkJAODq6vpK9cm6GGpAQACaN2+O5cuXA5BmYHZ3d8e4ceMwderUYtXRrFkzhISEYM6cOQCAfv36wcjICJs3by51XKmpqbCxsUFKSgqHwRMRlVJOTg7+/vtvODk5wd7eXu5wqJJ4+PAhkpKSULdu3XzNYSX5/patE3RWVhbOnTuHoKCgF8EYGCAoKAgxMTFF7i+EQHR0NOLi4tC2bVsAUgK1f/9+1K1bF8HBwXByckJAQAD27NlTaF2ZmZlITU1VuxER0avJ7fNjbm4ucyRUmeS+n161T5lsCdCDBw+Qk5MDZ2dnte3Ozs5ISEgocL+UlBRYWlrC2NgYISEhWLZsGTp27AgASEpKQnp6Or744gt07twZP//8M3r27IlevXrh2LFjBdYZEREBGxsb1Y3LYBARlR02e1FZKqv3k+ydoEvKysoK58+fR3p6OqKjoxEeHo5atWqhffv2UCqVAIDu3bvj/fffBwD4+Pjg5MmTWL16Ndq1a6exzmnTpiE8PFz1e+5U2kRERFQ5yXYFyMHBAYaGhkhMTFTbnpiYCBcXlwL3MzAwQO3ateHj44NJkyahT58+iIiIUNVZpUoVeHt7q+3ToEED3Lx5s8A6TUxMVMtecPkLIiIqC+3bt8fEiRNVv3t6emLx4sWF7qNQKIrstlEcZVVPYWbNmgUfH59yfY7yJFsCZGxsDF9fX0RHR6u2KZVKREdHIzAwsNj1KJVKZGZmqups3rw54uLi1Mr8/fff8PDwKJvAiYioUnvrrbfQuXNnjY8dP34cCoUCFy5cKHG9Z8+exYgRI141PDUFJSH37t1Dly5dyvS5KhtZm8DCw8MRFhYGPz8/+Pv7Y/HixcjIyMCQIUMAAIMGDUK1atVUV3giIiLg5+cHLy8vZGZm4sCBA9i8eTNWrVqlqnPy5MkIDQ1F27Zt8frrryMqKgo//vgjjh49KschEhGRjhk2bBh69+6N27dvo3r16mqPbdiwAX5+fmjSpEmJ63V0dCyrEItUWEsKSWRdCiM0NBQLFizAjBkz4OPjg/PnzyMqKkrVMfrmzZu4d++eqnxGRgZGjx6Nhg0bolWrVti5cye2bNmCd999V1WmZ8+eWL16NebNm4fGjRvjq6++ws6dO9G6desKPz5NHj0Cbt2SOwoiIirIm2++CUdHR2zcuFFte3p6OiIjIzFs2DA8fPgQ/fv3R7Vq1WBubo7GjRvj+++/L7Tel5vArl69irZt28LU1BTe3t44dOhQvn2mTJmCunXrwtzcHLVq1cLHH3+sGv20ceNGzJ49G3/88QcUCgUUCoUq5pebwC5evIg33ngDZmZmsLe3x4gRI5Cenq56fPDgwejRowcWLFgAV1dX2NvbY8yYMSUaaaVUKvHJJ5+gevXqMDExgY+PD6KiolSPZ2VlYezYsXB1dYWpqSk8PDxUFziEEJg1axZq1KgBExMTuLm5Yfz48cV+7tKQvRP02LFjMXbsWI2PvXzV5tNPP8Wnn35aZJ1Dhw7F0KFDyyK8MvX118C77wIhIcC+fXJHQ0Qkn4yMgh8zNATyzl1bWFkDA8DMrOiyFhbFj61KlSoYNGgQNm7ciOnTp6tGHUVGRiInJwf9+/dHeno6fH19MWXKFFhbW2P//v0YOHAgvLy8ClzNIC+lUolevXrB2dkZp0+fRkpKilp/oVxWVlbYuHEj3NzccPHiRQwfPhxWVlb48MMPERoaikuXLiEqKgqHDx8GANjY2OSrIyMjA8HBwQgMDMTZs2eRlJSEd999F2PHjlVL8o4cOQJXV1ccOXIE//zzD0JDQ+Hj44Phw4cX63VbsmQJFi5ciDVr1uC1117D+vXr0a1bN/z555+oU6cOli5dir1792L79u2oUaMGbt26hVv/uyKwc+dOfPnll9i6dSsaNmyIhIQE/PHHH8V63lITlE9KSooAIFJSUsq03kOHhACEqF+/TKslItJKT58+FZcvXxZPnz7N9xhQ8K1rV/Wy5uYFl23XTr2sg4PmciV15coVAUAcOXJEta1NmzbinXfeKXCfkJAQMWnSJNXv7dq1ExMmTFD97uHhIb788kshhBAHDx4UVapUEXfu3FE9/tNPPwkAYvfu3QU+x/z584Wvr6/q95kzZ4qmTZvmK5e3nrVr1wo7OzuRnp6uenz//v3CwMBAJCQkCCGECAsLEx4eHuL58+eqMm+//bYIDQ0tMJaXn9vNzU189tlnamWaN28uRo8eLYQQYty4ceKNN94QSqUyX10LFy4UdevWFVlZWQU+X67C3lcl+f7mavAVyMtL+hkfD/xvxD4REWmh+vXro2XLlli/fj0A4J9//sHx48cxbNgwANIs13PmzEHjxo1RtWpVWFpa4uDBg4WOOM7rypUrcHd3h5ubm2qbpgFA27ZtQ6tWreDi4gJLS0v83//9X7GfI+9zNW3aFBZ5LoO1atUKSqVSbdBQw4YN1WZWdnV1VS07UZTU1FTcvXsXrVq1UtveqlUrXLlyBYDUzHb+/HnUq1cP48ePx88//6wq9/bbb+Pp06eoVasWhg8fjt27d+P58+clOs6SYgJUgdzdgSpVgMxM4M4duaMhIpJPenrBt5071csmJRVc9qef1Mtev665XGkMGzYMO3fuRFpaGjZs2AAvLy/VfHLz58/HkiVLMGXKFBw5cgTnz59HcHAwsrKySvdkGsTExGDAgAHo2rUr9u3bh99//x3Tp08v0+fIy8jISO13hUKhml+vLDRr1gzx8fGYM2cOnj59ir59+6JPnz4AAHd3d8TFxWHlypUwMzPD6NGj0bZt21ee7bkwTIAqUJUqgKendP/aNVlDISKSlYVFwbeX164urGze/j+FlS2Nvn37wsDAAN999x02bdqEoUOHqvoDnThxAt27d8c777yDpk2bolatWvj777+LXXeDBg1w69YttYE+p06dUitz8uRJeHh4YPr06fDz80OdOnVw48YNtTLGxsbIyckp8rn++OMPZOTpIHXixAkYGBigXr16xY65MNbW1nBzc8OJEyfUtp84cUJtbj5ra2uEhoZi3bp12LZtG3bu3IlHjx4BAMzMzPDWW29h6dKlOHr0KGJiYnDx4sUyiU8TJkAVLLcZjAkQEZF2s7S0RGhoKKZNm4Z79+5h8ODBqsfq1KmDQ4cO4eTJk7hy5QpGjhyZb2LfwgQFBaFu3boICwvDH3/8gePHj2P69OlqZerUqYObN29i69atuHbtGpYuXYrdu3erlfH09ER8fDzOnz+PBw8eqObFy2vAgAEwNTVFWFgYLl26hCNHjmDcuHEYOHBgvuWoXsXkyZMxd+5cbNu2DXFxcZg6dSrOnz+PCRMmAAAWLVqE77//Hn/99Rf+/vtvREZGwsXFBba2tti4cSO+/vprXLp0Cf/++y+2bNkCMzOzcp3DjwlQBWMCRESkO4YNG4bHjx8jODhYrb/O//3f/6FZs2YIDg5G+/bt4eLigh49ehS7XgMDA+zevRtPnz6Fv78/3n33XXz22WdqZbp164b3338fY8eOVS3r9PHHH6uV6d27Nzp37ozXX38djo6OGofim5ub4+DBg3j06BGaN2+OPn36oEOHDli+fHnJXowijB8/HuHh4Zg0aRIaN26MqKgo7N27F3Xq1AEgjWibN28e/Pz80Lx5c1y/fh0HDhyAgYEBbG1tsW7dOrRq1QpNmjTB4cOH8eOPP8Le3r5MY8xLIYQQ5Va7jkpNTYWNjQ1SUlLKfFmMPXuA6GigSxega9cyrZqISKs8e/YM8fHxqFmzJkxfbtciKqXC3lcl+f6WfR4gfdOjh3QjIiIi+bAJjIiIiPQOEyAZJCcD584BT5/KHQkREZF+YgIkA29vwM8PuHRJ7kiIiIj0ExMgGXAkGBHpE461obJUVu8nJkAyYAJERPogd2bhJ0+eyBwJVSa576eXZ64uKY4CkwETICLSB4aGhrC1tVWtJ2Vubq6aSZmopIQQePLkCZKSkmBra6u2bllpMAGSARMgItIXLi4uAFDsRTWJimJra6t6X70KJkAyYAJERPpCoVDA1dUVTk5O5bqwJekHIyOjV77yk4sJkAxyE6A7d6Sh8C8v5kdEVNkYGhqW2RcXUVlgAiQDe3tg4kTA3R0oYhFfIiIiKgdMgGSgUABffil3FERERPqLw+CJiIhI7/AKkEwyMoC4OMDAAPDxkTsaIiIi/cIrQDL5/nvA1xf46CO5IyEiItI/TIBkUquW9JND4YmIiCoeEyCZ5A6Fj4/nSDAiIqKKxgRIJtWrA0ZGQHY2cPu23NEQERHpFyZAMjE0BGrWlO6zGYyIiKhiMQGSEZfEICIikgcTIBkxASIiIpIH5wGSUc+egIcH0KaN3JEQERHpFyZAMnrjDelGREREFYtNYERERKR3mADJ7MIFYNcuIC1N7kiIiIj0BxMgmYWEAL17A5cvyx0JERGR/mACJDOOBCMiIqp4TIBkxgSIiIio4jEBkhkXRSUiIqp4TIBkxitAREREFY8JkMyYABEREVU8JkAyy02A7t0DnjyRNxYiIiJ9wZmgZVa1KrBoEVCjBmDAdJSIiKhCMAHSAu+/L3cERERE+oXXHIiIiEjv8AqQFrh3Dzh9GrC0BIKC5I6GiIio8uMVIC3w009Az57A/PlyR0JERKQfmABpAQ6FJyIiqlhMgLRAbgJ04wbw/Lm8sRAREekDJkBawM0NMDGRkp9bt+SOhoiIqPJjAqQFDAyAmjWl+2wGIyIiKn9MgLQE+wERERFVHCZAWoIJEBERUcXhPEBaYuBAoHVroFkzuSMhIiKq/JgAaQk/P+lGRERE5Y9NYERERKR3mABpkX37gC+/BJKT5Y6EiIiocmMTmBYZMwa4eRMICABatpQ7GiIiosqLV4C0CEeCERERVQwmQFqECRAREVHFYAKkRZgAERERVQwmQFokNwH691954yAiIqrstCIBWrFiBTw9PWFqaoqAgACcOXOmwLK7du2Cn58fbG1tYWFhAR8fH2zevLnA8u+99x4UCgUWL15cDpGXLV4BIiIiqhiyJ0Dbtm1DeHg4Zs6cidjYWDRt2hTBwcFISkrSWL5q1aqYPn06YmJicOHCBQwZMgRDhgzBwYMH85XdvXs3Tp06BTc3t/I+jDKRmwAlJgLp6fLGQkREVJnJngAtWrQIw4cPx5AhQ+Dt7Y3Vq1fD3Nwc69ev11i+ffv26NmzJxo0aAAvLy9MmDABTZo0wa+//qpW7s6dOxg3bhy+/fZbGBkZVcShvDIbG2DHDuDsWcDUVO5oiIiIKi9ZE6CsrCycO3cOQUFBqm0GBgYICgpCTExMkfsLIRAdHY24uDi0bdtWtV2pVGLgwIGYPHkyGjZsWGQ9mZmZSE1NVbvJpXdvaUmMKpyhiYiIqNzImgA9ePAAOTk5cHZ2Vtvu7OyMhISEAvdLSUmBpaUljI2NERISgmXLlqFjx46qx+fOnYsqVapg/PjxxYojIiICNjY2qpu7u3vpDoiIiIh0gk5eZ7CyssL58+eRnp6O6OhohIeHo1atWmjfvj3OnTuHJUuWIDY2FgqFolj1TZs2DeHh4arfU1NTZUuC/voLOHgQcHEBQkNlCYGIiKjSkzUBcnBwgKGhIRITE9W2JyYmwsXFpcD9DAwMULt2bQCAj48Prly5goiICLRv3x7Hjx9HUlISatSooSqfk5ODSZMmYfHixbh+/Xq++kxMTGBiYlI2B/WKTp0CJk4EgoKYABEREZUXWZvAjI2N4evri+joaNU2pVKJ6OhoBAYGFrsepVKJzMxMAMDAgQNx4cIFnD9/XnVzc3PD5MmTNY4U0zYcCk9ERFT+ZG8CCw8PR1hYGPz8/ODv74/FixcjIyMDQ4YMAQAMGjQI1apVQ0REBACpv46fnx+8vLyQmZmJAwcOYPPmzVi1ahUAwN7eHvb29mrPYWRkBBcXF9SrV69iD64UchOgmzeB7GxARwawERER6RTZE6DQ0FDcv38fM2bMQEJCAnx8fBAVFaXqGH3z5k0YGLy4UJWRkYHRo0fj9u3bMDMzQ/369bFlyxaEVpL2IldXwMwMePoUuHED+F9LHxEREZUhhRBCyB2EtklNTYWNjQ1SUlJgbW1d4c/fqBHw559AVBQQHFzhT09ERKSTSvL9LftEiJQf1wQjIiIqX0yAtBA7QhMREZUv2fsAUX6jRgH9+gF168odCRERUeXEBEgL1akjdwRERESVG5vAiIiISO8wAdJSK1cCEyYADx7IHQkREVHlwwRIS82fDyxdKq0NRkRERGWLCZCW4kgwIiKi8sMESEsxASIiIio/TIC0FBMgIiKi8sMESEsxASIiIio/TIC0FBMgIiKi8sMESEvlJkAPHgCpqfLGQkREVNkwAdJSVlbA6dNAUpJ0n4iIiMoOl8LQYv7+ckdARERUOfEKEBEREekdJkBa7MwZIDxcWhaDiIiIyg4TIC3211/Al18CO3fKHQkREVHlwgRIi3EoPBERUflgAqTFchOgW7eArCx5YyEiIqpMmABpMWdnwMICUCqB69fljoaIiKjyYAKkxRQKoFYt6T6bwYiIiMoOEyAtx35AREREZY8JkJbLTYBu3JA3DiIiospEIYQQcgehbVJTU2FjY4OUlBRYW1vLGktiovTTyUlqEiMiIiLNSvL9zaUwtJyzs9wREBERVT5sAiMiIiK9wwRIB0ydCvToASQkyB0JERFR5cAESAdERgI//AD8/bfckRAREVUOTIB0AIfCExERlS0mQDqACRAREVHZYgKkA5gAERERlS0mQDqACRAREVHZYgKkA5gAERERlS0mQDogd0HUZ8+AtDR5YyEiIqoMOBO0DrC0BO7dk2aF5nIYREREr44JkI5wcZE7AiIiosqDTWBERESkd5gA6YhDh4BevYBPPpE7EiIiIt3HBEhHJCYCu3cDR47IHQkREZHuYwKkIzgUnoiIqOwwAdIRuQnQ7dtAZqa8sRAREek6JkA6wtFRGg4vBBAfL3c0REREuo0JkI5QKNgMRkREVFaYAOkQJkBERERlgwmQDvHyAkxNgfR0uSMhIiLSbQohhJA7CG2TmpoKGxsbpKSkwNraWu5wVJ49A4yNAQOmrURERPmU5PubS2HoEFNTuSMgIiKqHHgtgYiIiPQOEyAdM2AA0KyZNB8QERERlQ4TIB1z5gzw++/AP//IHQkREZHuYgKkYzgUnoiI6NUxAdIxTICIiIheHRMgHcMEiIiI6NUxAdIxTICIiIheHRMgHcMEiIiI6NUxAdIxtWpJEyK6uAAZGXJHQ0REpJs4E7SOMTeXEh8uh0FERFR6WvE1umLFCnh6esLU1BQBAQE4c+ZMgWV37doFPz8/2NrawsLCAj4+Pti8ebPq8ezsbEyZMgWNGzeGhYUF3NzcMGjQINy9e7ciDqVCMPkhIiJ6NbJ/lW7btg3h4eGYOXMmYmNj0bRpUwQHByMpKUlj+apVq2L69OmIiYnBhQsXMGTIEAwZMgQHDx4EADx58gSxsbH4+OOPERsbi127diEuLg7dunWryMMiIiIiLSb7avABAQFo3rw5li9fDgBQKpVwd3fHuHHjMHXq1GLV0axZM4SEhGDOnDkaHz979iz8/f1x48YN1KhRo8j6tHU1+Fw7dwKffw60bg0sWSJ3NERERNqhJN/fsl4BysrKwrlz5xAUFKTaZmBggKCgIMTExBS5vxAC0dHRiIuLQ9u2bQssl5KSAoVCAVtb27IIW3ZZWUBsLHD+vNyREBER6SZZO0E/ePAAOTk5cHZ2Vtvu7OyMv/76q8D9UlJSUK1aNWRmZsLQ0BArV65Ex44dNZZ99uwZpkyZgv79+xeYDWZmZiIzM1P1e2pqaimOpuJwKDwREdGr0clRYFZWVjh//jzS09MRHR2N8PBw1KpVC+3bt1crl52djb59+0IIgVWrVhVYX0REBGbPnl3OUZedWrWkn3fuAE+fAmZm8sZDRESka2RtAnNwcIChoSESExPVticmJsLFxaXA/QwMDFC7dm34+Phg0qRJ6NOnDyIiItTK5CY/N27cwKFDhwptC5w2bRpSUlJUt1u3br3agZUze3sg93Di4+WNhYiISBfJmgAZGxvD19cX0dHRqm1KpRLR0dEIDAwsdj1KpVKtCSs3+bl69SoOHz4Me3v7Qvc3MTGBtbW12k2bKRRsBiMiInoVsjeBhYeHIywsDH5+fvD398fixYuRkZGBIUOGAAAGDRqEatWqqa7wREREwM/PD15eXsjMzMSBAwewefNmVRNXdnY2+vTpg9jYWOzbtw85OTlISEgAIA2hNzY2ludAy5iXF/D770yAiIiISkP2BCg0NBT379/HjBkzkJCQAB8fH0RFRak6Rt+8eRMGeWb+y8jIwOjRo3H79m2YmZmhfv362LJlC0JDQwEAd+7cwd69ewEAPj4+as915MiRfP2EdJW3N1C3LmBiInckREREukf2eYC0kbbPA0RERET56cw8QERERERyYAJEREREeocJkI4SAmjTBqhaFbh+Xe5oiIiIdAsTIB2lUAD37wOPH3MkGBERUUkxAdJhnAuIiIiodJgA6TAmQERERKXDBEiHMQEiIiIqHSZAOix3UVQmQERERCXDBEiH5b0CxOksiYiIik/2pTCo9GrVAurUkX4+fQqYm8sdERERkW5gAqTDTE2Bv/+WOwoiIiLdwyYwIiIi0jtMgCqJ58/ljoCIiEh3MAHScRs2AA4OwLBhckdCRESkO5gA6Thzc+DhQw6FJyIiKgkmQDqOkyESERGVXKkSoFu3buH27duq38+cOYOJEydi7dq1ZRYYFU9uApSQAGRkyBsLERGRrihVAvSf//wHR44cAQAkJCSgY8eOOHPmDKZPn45PPvmkTAOkwtnZSTcA+PdfeWMhIiLSFaVKgC5dugR/f38AwPbt29GoUSOcPHkS3377LTZu3FiW8VExsBmMiIioZEqVAGVnZ8PExAQAcPjwYXTr1g0AUL9+fdy7d6/soqNi4ZpgREREJVOqBKhhw4ZYvXo1jh8/jkOHDqFz584AgLt378Le3r5MA6SitWgBdOwIuLnJHQkREZFuUAhR8mU0jx49ip49eyI1NRVhYWFYv349AOCjjz7CX3/9hV27dpV5oBUpNTUVNjY2SElJgbW1tdzhEBERUTGU5Pu7VAkQAOTk5CA1NRV2uT1wAVy/fh3m5uZwcnIqTZVagwkQERGR7inJ93epmsCePn2KzMxMVfJz48YNLF68GHFxcTqf/OiytDRAqZQ7CiIiIu1XqgSoe/fu2LRpEwAgOTkZAQEBWLhwIXr06IFVq1aVaYBUNCEADw/A2hq4fl3uaIiIiLRfqRKg2NhYtGnTBgCwY8cOODs748aNG9i0aROWLl1apgFS0RQKwMJCus+RYEREREUrVQL05MkTWFlZAQB+/vln9OrVCwYGBmjRogVu3LhRpgFS8XAuICIiouIrVQJUu3Zt7NmzB7du3cLBgwfRqVMnAEBSUhI7DcuECRAREVHxlSoBmjFjBj744AN4enrC398fgYGBAKSrQa+99lqZBkjFwwSIiIio+KqUZqc+ffqgdevWuHfvHpo2bara3qFDB/Ts2bPMgqPiYwJERERUfKVKgADAxcUFLi4uqlXhq1evrlofjCpe3gRICKljNBEREWlWqiYwpVKJTz75BDY2NvDw8ICHhwdsbW0xZ84cKDkRjSw8PYGgIGDAACAzU+5oiIiItFuprgBNnz4dX3/9Nb744gu0atUKAPDrr79i1qxZePbsGT777LMyDZKKZmICHDokdxRERES6oVRLYbi5uWH16tWqVeBz/fDDDxg9ejTu3LlTZgHKgUthEBER6Z5yXwrj0aNHqF+/fr7t9evXx6NHj0pTJZWR9HTgq6+A58/ljoSIiEh7lSoBatq0KZYvX55v+/Lly9GkSZNXDopKRwjA1xcYPhzYvVvuaIiIiLRXqfoAzZs3DyEhITh8+LBqDqCYmBjcunULBw4cKNMAqfgUCqBfP+CTT4DFi4G335Y7IiIiIu1UqitA7dq1w99//42ePXsiOTkZycnJ6NWrF/78809s3ry5rGOkEhg1CjAyAk6eBM6elTsaIiIi7VSqTtAF+eOPP9CsWTPk5OSUVZWy0PVO0IMGAZs3S0Pit2yROxoiIqKKUe6doEm7TZgg/dy2Dbh7V95YiIiItBEToErI1xdo3VoaCbZqldzREBERaR8mQJXUxIlSp2heASIiIsqvRKPAevXqVejjycnJrxILlaHu3YGrV1+sEUZEREQvlCgBsrGxKfLxQYMGvVJAVDaqVGHyQ0REVJASJUAbNmworzioHF2/Djx7BmiYvJuIiEgvsQ9QJbdunXQl6MMP5Y6EiIhIezABquTatgWUSmDfPuCff+SOhoiISDswAark6tUDunSR1glbtkzuaIiIiLQDEyA9MHGi9HPDBiA1VdZQiIiItAITID3QsSPQoAGQlgasXy93NERERPJjAqQHFIoXV4GWLQN0fKk2IiKiV8YESE+88w5QtSrw4AHw119yR0NERCSvEs0DRLrL3Bz48UegcWPAykruaIiIiOTFBEiPtGwpdwRERETagU1gekgIaZ0wIiIifcUESM88fgw0ayY1hd2/L3c0RERE8mACpGdsbQFjYyAzE1izRu5oiIiI5MEESM8oFMCECdL9FSuArCx54yEiIpIDEyA91KcP4OYGJCQA27fLHQ0REVHF04oEaMWKFfD09ISpqSkCAgJw5syZAsvu2rULfn5+sLW1hYWFBXx8fLB582a1MkIIzJgxA66urjAzM0NQUBCustevirExMHq0dH/xYqlTNBERkT6RPQHatm0bwsPDMXPmTMTGxqJp06YIDg5GUlKSxvJVq1bF9OnTERMTgwsXLmDIkCEYMmQIDh48qCozb948LF26FKtXr8bp06dhYWGB4OBgPHv2rKIOS+uNGAGYmgLnzgEnT8odDRERUcVSCCHv//8BAQFo3rw5li9fDgBQKpVwd3fHuHHjMHXq1GLV0axZM4SEhGDOnDkQQsDNzQ2TJk3CBx98AABISUmBs7MzNm7ciH79+hVZX2pqKmxsbJCSkgJra+vSH5yWe/dd4OuvgTFjgP+9/ERERDqrJN/fsl4BysrKwrlz5xAUFKTaZmBggKCgIMTExBS5vxAC0dHRiIuLQ9u2bQEA8fHxSEhIUKvTxsYGAQEBBdaZmZmJ1NRUtZs++PBDYN8+YOlSuSMhIiKqWLLOBP3gwQPk5OTA2dlZbbuzszP+KmTBqpSUFFSrVg2ZmZkwNDTEypUr0bFjRwBAQkKCqo6X68x97GURERGYPXv2qxyKTqpbV7oRERHpG9n7AJWGlZUVzp8/j7Nnz+Kzzz5DeHg4jh49Wur6pk2bhpSUFNXt1q1bZResjnj2TLoRERHpA1kTIAcHBxgaGiIxMVFte2JiIlxcXArcz8DAALVr14aPjw8mTZqEPn36ICIiAgBU+5WkThMTE1hbW6vd9MnKlUCNGsCGDXJHQkREVDFkTYCMjY3h6+uL6Oho1TalUono6GgEBgYWux6lUonMzEwAQM2aNeHi4qJWZ2pqKk6fPl2iOvVJTo60LMaSJYBSKXc0RERE5U/2JrDw8HCsW7cO33zzDa5cuYJRo0YhIyMDQ4YMAQAMGjQI06ZNU5WPiIjAoUOH8O+//+LKlStYuHAhNm/ejHfeeQcAoFAoMHHiRHz66afYu3cvLl68iEGDBsHNzQ09evSQ4xC13uDBgLU1EBcH/Pyz3NEQERGVP1k7QQNAaGgo7t+/jxkzZiAhIQE+Pj6IiopSdWK+efMmDAxe5GkZGRkYPXo0bt++DTMzM9SvXx9btmxBaGioqsyHH36IjIwMjBgxAsnJyWjdujWioqJgampa4cenC6ysgGHDgC+/lCZG7NxZ7oiIiIjKl+zzAGkjfZkHKK/4eKB2bakJ7PJloEEDuSMiIiIqGZ2ZB4i0R82aQLdu0n3OC0RERJUdEyBSmThR+rlpE5CWJmsoRERE5YoJEKm0bQvMmgWcOSP1CyIiIqqsZO8ETdpDoQBmzpQ7CiIiovLHK0BUoJwcuSMgIiIqH0yAKJ8bN4B33gE6dJA7EiIiovLBJjDKx8QE2L4dyM6W+gP5+8sdERERUdniFSDKx8UF6NdPur9kibyxEBERlQcmQKTRhAnSz+3bgbt35Y2FiIiorDEBIo18fYHWrYHnz6XV4omIiCoTJkBUoNyJEdesAZ4+lTUUIiKiMsUEiArUvTtQowbw4AHw3XdyR0NERFR2OAqMClSlijQx4oMHQM+eckdDRERUdpgAUaGGDpU7AiIiorLHJjAiIiLSO0yAqFi2bwdatQL++UfuSIiIiF4dEyAqlo0bgZMngWXL5I6EiIjo1TEBomLJHRK/fj3w77+yhkJERPTKmABRsXTsCAQGAunp0oiwjAy5IyIiIio9JkBULAqF1A/IyQm4cAEYPhwQQu6oiIiISocJEBVb9epAZKQ0P9D33wNffil3RERERKXDBIhKpG3bF4lPfLy8sRAREZUWJ0KkEhszBmjUCGjfXu5IiIiISodXgKjEFAr15Of5c+DZM9nCISIiKjEmQPRKHj4EOncG3n2XnaKJiEh3sAmMXsnly8DRo0BODtC8OTBhgtwRERERFY1XgOiVtGkDLFwo3Z80SUqGiIiItB0TIHpl48cD77wjXQXq2xe4eVPuiIiIiArHBIhemUIBrFkDvPYacP8+0KsX8PSp3FEREREVjAkQlQlzc2DXLsDeHjh3Dnj/fbkjIiIiKhgTICoznp7Atm1A3brSXEFERETaiqPAqEx16ABcugQYGckdCRERUcF4BYjKXN7k5+RJ4PZt+WIhIiLShAkQlZvISGnG6N69gcxMuaMhIiJ6gQkQlRs/P8DKCjhzRuoTxJmiiYhIWzABonJTsyawdStgYAB8/TWwdq3cEREREUmYAFG56tgRiIiQ7o8bJ/UJIiIikhsTICp3kycDb78NZGdL/YHu3pU7IiIi0ndMgKjcKRTA+vVAo0ZAQgKbwoiISH6cB4gqhKUlsHs38MMPQHi43NEQEZG+YwJEFaZ2bWnFeCIiIrmxCYxkkZEBDBwInDoldyRERKSPmACRLGbNArZskTpFJyTIHQ0REekbJkAkixkzgAYNpBFhb78NZGXJHREREekTJkAkCysrYM8ewNoa+PVXdowmIqKKxQSIZFO3LvDtt9L9FSuADRvkjYeIiPQHEyCS1ZtvArNnS/dHjQLOnZM3HiIi0g9MgEh2//d/QLduUnOYi4vc0RARkT7gPEAkOwMDYNMm4PJloFq1F9tzcgBDQ/niIiKiyotXgEgr2NgAgYEvft+1C/D3B65dky8mIiKqvJgAkdZ5/hyYMgWIjQWaNQN27JA7IiIiqmyYAJHWqVIFOHIEaNUKSE2V5gkaNw7IzJQ7MiIiqiyYAJFWql5dSoKmTpV+X74caN0a+PdfeeMiIqLKgQkQaS0jIyAiAti/H6haFfjtN6lJjEtnEBHRq+IoMNJ6XbsC588DoaGAry+HyhMR0atjAkQ6wd0dOHYMEOLFtnv3pH5Bnp6yhUVERDqKTWCkM4yMAGNj6f7z50D//sBrrwE//CBvXEREpHuYAJFOSk6Wrv4kJwM9ekiLqXJFeSIiKi4mQKSTHBykJrHcVeS//BJo2xa4cUPeuIiISDfIngCtWLECnp6eMDU1RUBAAM6cOVNg2XXr1qFNmzaws7ODnZ0dgoKC8pVPT0/H2LFjUb16dZiZmcHb2xurV68u78MgGRgbAwsXAnv2ALa2wOnTUpPYjz/KHRkREWk7WROgbdu2ITw8HDNnzkRsbCyaNm2K4OBgJCUlaSx/9OhR9O/fH0eOHEFMTAzc3d3RqVMn3LlzR1UmPDwcUVFR2LJlC65cuYKJEydi7Nix2Lt3b0UdFlWw7t2lWaObNwcePwamTZP6CBERERVEIUTecTUVKyAgAM2bN8fy5csBAEqlEu7u7hg3bhym5s6AV4icnBzY2dlh+fLlGDRoEACgUaNGCA0Nxccff6wq5+vriy5duuDTTz8tVlypqamwsbFBSkoKrK2tS3FkJIesLOCjj4ChQwFvb7mjISKiilaS72/ZrgBlZWXh3LlzCAoKehGMgQGCgoIQExNTrDqePHmC7OxsVK1aVbWtZcuW2Lt3L+7cuQMhBI4cOYK///4bnTp1KrCezMxMpKamqt1I9xgbAwsWqCc/K1cCBw7IFxMREWkn2RKgBw8eICcnB87OzmrbnZ2dkVDMqX6nTJkCNzc3tSRq2bJl8Pb2RvXq1WFsbIzOnTtjxYoVaNu2bYH1REREwMbGRnVzd3cv3UGRVjl7Fhg/HggJkZbUyM6WOyIiItIWsneCLq0vvvgCW7duxe7du2FqaqravmzZMpw6dQp79+7FuXPnsHDhQowZMwaHDx8usK5p06YhJSVFdbt161ZFHAKVsyZNgFGjpPtz5wKvv85RYkREJJFtJmgHBwcYGhoiMTFRbXtiYiJciljrYMGCBfjiiy9w+PBhNGnSRLX96dOn+Oijj7B7926EhIQAAJo0aYLz589jwYIFaleK8jIxMYGJickrHhFpGxMTYNkyoF07YNgw4MQJoGFDaX2xMWMAA51N/4mI6FXJ9hVgbGwMX19fREdHq7YplUpER0cjMDCwwP3mzZuHOXPmICoqCn5+fmqPZWdnIzs7GwYvfbMZGhpCqVSW7QGQzujTRxol1ro1kJEhNYt17aq+rAYREekXWdcCCw8PR1hYGPz8/ODv74/FixcjIyMDQ4YMAQAMGjQI1apVQ0REBABg7ty5mDFjBr777jt4enqq+gpZWlrC0tIS1tbWaNeuHSZPngwzMzN4eHjg2LFj2LRpExYtWiTbcZL8vLykiRNXrwamTAHefBNQKOSOioiI5CJrAhQaGor79+9jxowZSEhIgI+PD6KiolQdo2/evKl2NWfVqlXIyspCnz591OqZOXMmZs2aBQDYunUrpk2bhgEDBuDRo0fw8PDAZ599hvfee6/Cjou0k4EBMHq0tHRG3lbWkyeBKlUAf3/ZQiMiogom6zxA2orzAOmPjAyps/T168CECcCcOYCFhdxRERFRaejEPEBE2iA7G2jZElAqpfXEGjcGChkwSERElQQTINJrtrbA5s3SZInu7kB8PNCxozSb9OPHckdHRETlhQkQEYAuXYA//wTGjpU6R2/YADRoAORZZo6IiCoRJkBE/2NlJc0b9OuvQP36QEAA4OYmd1RERFQeZB0FRqSNWrYEfv9d6iCdO1T+wQPgxx+BwYM5fJ6IqDLgFSAiDUxNAXv7F7+//77UL6hDB+Cff+SLi4iIygYTIKIiCAE0awaYmQFHjkgjxebPB54/lzsyIiIqLSZAREVQKKQrQJcuSVeAnj0DPvwQaNEC+OMPuaMjIqLSYAJEVEy1agGHDgHr10vD58+dA/z8gD175I6MiIhKigkQUQkoFMCQIcCVK0Dv3oCjI9C+vdxRERFRSTEBIioFFxdgxw5ptJitrbQtJwfo2RP46isgPV3W8IiIqAhMgIhewf/W7QUgNY/t2QMMHw64ugIjRgBnz0qdqImISLswASIqI76+wLx5QJ060hWgdeukFeZfew1YsQJISZE7QiIiysUEiKiMODoCkycDcXHA0aPAgAGAiYk0UmzsWKnfEBERaQeFELxA/7LU1FTY2NggJSUF1tbWcodDOuzRI2DLFml5jW3bXswi/fnnUnIUFgY4OMgbIxFRZVGS728mQBowAaLylJ4urTGWlgYYGUkdp4cPB954AzDgNVkiolIryfc3/9wSVTBDQ2DBAmkOoexsYPt2oGNHqe/Q558D9+7JHSERUeXHBIiogpmZvRghFhsLjB4NWFsD//4LTJ8OrFwpd4RERJUfEyAiGeWOELt3D9i4EWjdGhg27MXj0dHAzJnAjRuyhUhEVCmxD5AG7ANE2uKtt4B9+6TO0x06SLNP9+ghTcRIRETq2AeIqJIYNEjqHC0EcPgwMGqU1IG6TRtg6VJOskhEVFpMgIi02NtvS81g164Bc+dKEysKIQ2r//77F8PqAeDuXfniJCLSNWwC04BNYKTNbt0Cdu+WmsH69pW2PX4MODkB9esDvXpJTWWNG6snSERElR3nAXpFTIBI1/z8MxASAjx//mJb7dpSMtSrl3TliMkQEVV27ANEpGc6dQKSkoBNm4Du3QFTU+Cff6S1yVq0ANavlztCIiLtwgSIqJKwswMGDpRWpL9/X5pgMTRUmmOoa9cX5bZskeYhOngQyMoq/7hycqQJH4mItAmbwDRgExhVJtnZ0pIbuYKCpI7VAGBrKw2179ULcHYGjI2lVe1zffutdGXp6dP8NwcH6QpTrrffBi5ffvH4kyfSz+xsoEoV4M4dqZ8SEVF5YR+gV8QEiCqz6Ghgxw6pI3ViovpjTZsC58+/+L1OHakpTZNataTRabmaNQN+/11z2aAg4NChF79v3gy0aiXVQURUVkry/V2lgmIiIi3RoYN0W74cOHkS2LULiIqSmsOqV1cvGxIiXQEyM8t/c3RUL7tiBfDsmeayJiYvyt29CwwZIjWNBQVJC8H26CFdfSIiqii8AqQBrwARlZ/Ll4GJE9WvCDk4AGFhUjJUr55soRGRjuMoMCLSWt7e0rD93MVf3dyABw+AhQuleYx27JA7QiLSB0yAiEgWNWsCn34qLfS6d6/UGdvKCujY8UWZEyeACxfki5GIKi8mQEQkqypVpORn717g9m3AxubFYxMnSh2zAwKAr74C0tNlC5OIKhkmQESkNfI22T99Cnh6SkP4z5yR+ge5ukpzGJ09y4VgiejVMAEiIq1kZgZERkpXhebPB+rWla4ArVsnLe0xfrzcERKRLuMweCLSak5OwAcfAJMmAcePSwlQZKQ0lD/XsWPAjz8C1apJQ/mrVZNubm7qk0ASEeViAkREOkGhANq2lW5LlwKWli8eO35cGkWmaR8nJ2l5kBYtpG0XLgB//KGeLFlYlF2cQkjNdw8fAo8evbgFB7+IeetW6fboEZCaKvVz6tpVWtPNzq7sYiGigjEBIiKd83KSEBgIhIdLy23cvi39vHNHWoYjMVG9b9EPPwAzZqjvb2v7IiH68kugQQNp++3bUiKjUORPaB4+BKZNexHL3LnAkiXSY5mZ+WO+eBFo1Ei6f/WqFEeuP/6QFrI1MJAStY0bpVm4iaj8MAEiIp2XO7t1XkqlNL/Q7duAl9eL7TVqSDNQ374t3dLTgeRk6fbnn+p1bNiQP1nKKyzsRQKUlQXcu/fisSpVAHt7oGpV6aZQvHisSxdpJm17e2mW7OPHgZ9+kp7/7Fmps3eu3buB58+l6QFsbUvwohBRoTgTtAacCZpIf6Smql856tsXMDeXHvv0U6m5TaGQkpi8CY29vTRM391dKnv7trRsSG4ZS0v1pKc4btyQrgZ16/Zim68vEBsLGBoCLVtKyVOXLlKzWUnrJ6rsuBjqK2ICRETaQKkEPvwQOHAAuHJF/TFXV+A//wEWLJAnNiJtxKUwiIgqAQMDKcG5fBmIjwdWrpQmjTQ3l5rb8ja5CSFdrbpwgXMkERUHrwBpwCtARKTNnj2T+g3Z2QF+ftK2S5eAxo2l+9WqvWgq69BBfXZtosqMTWCviAkQEema338HPv4Y+OUXaRh+XhYWwPr1Uv+m3LJr1uTv05R7v0YN9WkGiHRFSb6/OQqMiKgSeO01YN8+6erQsWPSqLIDB6Qh9xkZ0szauS5flhKggmzYAAweLN0/dgx4//2Ck6WAgBfTBhDpEiZARESViKmpNOlicDCweLE0vP/BA8DZ+UWZRo2AmTPV5zTKe9/e/kXZ27elK0YFWbPmRQJ0+bK0aG2TJtLN21uKh0gbMQEiIqrEbG3zzx/UtKl0K4433pCuJL2cJOX+9Pd/UfbECWkiyVwGBtIabrkJUb9+6nMyEcmJfYA0YB8gIqKSi4kBtm2TRqJduCAlSHlFR0sJFQAcOiQtUZKbHDVqBFhZVXjIVMmwDxAREVW4wEDpBkhD8RMSXiRDFy5IiU6uw4elYf151az5IiEaNUp9RmyissYrQBrwChARUfk6cgSIipISo4sXpVm487pxQxqNBkiTQX7/fcEdsUeMeDHU//59aemQqlWlZUZIv/AKEBERabXXX5duuR4+lBKhCxeAv/56scQIANy9+2LtNk0GDXqRAM2ZAyxbJt23sMifMC1bBri4SI/HxkrPWZDevV9MB/Dbb/nXisure/cXfa3On5eWNKlSRZqrKe/z29pKy5qQ/JgAERGR7OztgfbtpdvL5s2T1l3T1BH70SMpsciVmSl1vlYqpeH/GRnArVsvHl+x4sX9DRuA5csLjqlduxcJ0PbtwPz5BZe9dOlFArRnDzB7tuZyCgXw66/Sum4A8OOPwNatmteaq1oVaNiQczKVFyZARESk1dzcpFtxrFkDrFolLXKrKWGys3tR1tNTmi27IHnnTqpXr/CyeZOUOnWkstnZ6s+flib1jco7Ki82Fvjuu4LrPX4caN1auv/VV8DcuS+SJCcnoHp1aebv6tWBVq3UpzCgwrEPkAbsA0RERGUtOxt4/FhKwoyMpG2nT0vTB2iaj+nRI+DgQSn5AoBZswq+sgRIk1a2bSvd/+476WpX3gTp5Z9VyvgSyNOngLHxiya+8+eBs2c1H9fIkdJivmWNfYCIiIi0jJGRdNUmr4AA6VYcI0cCQUEvEonERKnz+O3b0s/cTuOANCnlyZMF1/Xf/wJt2kj3o6KAvXvVEyQ3N6kz+aNHQIsWLya0/OEHacZxTUnNs2dS/63chG3XLqlPliaamjorGhMgIiIiHeDqWvypAcLCpMku79xRT5Jyf1av/qLsr79KzYYFyZvUnDsnNcUVJO/cT02aAN26aR65l3dKBLkwASIiIqpk6tSRbpq83PGlQwepc3beJOnuXWkagapVgaysF2WDgqRmroKmJMg7mWWfPtJNW7EPkAbsA0RERKR7SvL9bVBBMRVoxYoV8PT0hKmpKQICAnDmzJkCy65btw5t2rSBnZ0d7OzsEBQUpLH8lStX0K1bN9jY2MDCwgLNmzfHzZs3y/MwiIiISIfImgBt27YN4eHhmDlzJmJjY9G0aVMEBwcjKSlJY/mjR4+if//+OHLkCGJiYuDu7o5OnTrhTp4pRK9du4bWrVujfv36OHr0KC5cuICPP/4YplySmIiIiP5H1iawgIAANG/eHMv/NxOVUqmEu7s7xo0bh6lTpxa5f05ODuzs7LB8+XIMGjQIANCvXz8YGRlh8+bNpY6LTWBERES6RyeawLKysnDu3DkEBQW9CMbAAEFBQYiJiSlWHU+ePEF2djaq/m8aUKVSif3796Nu3boIDg6Gk5MTAgICsGfPnvI4BCIiItJRsiVADx48QE5ODpydndW2Ozs7IyEhoVh1TJkyBW5ubqokKikpCenp6fjiiy/QuXNn/Pzzz+jZsyd69eqFY8eOFVhPZmYmUlNT1W5ERERUeensMPgvvvgCW7duxdGjR1X9e5RKJQCge/fueP/99wEAPj4+OHnyJFavXo127dpprCsiIgKzC5tek4iIiCoV2a4AOTg4wNDQEImJiWrbExMT4ZK7VG8BFixYgC+++AI///wzmuSZTcnBwQFVqlSBt7e3WvkGDRoUOgps2rRpSElJUd1u5V05j4iIiCod2RIgY2Nj+Pr6Ijo6WrVNqVQiOjoagYGBBe43b948zJkzB1FRUfDz88tXZ/PmzREXF6e2/e+//4aHh0eBdZqYmMDa2lrtRkRERJWXrE1g4eHhCAsLg5+fH/z9/bF48WJkZGRgyJAhAIBBgwahWrVqiIiIAADMnTsXM2bMwHfffQdPT09VXyFLS0tY/m8p3smTJyM0NBRt27bF66+/jqioKPz44484evSoLMdIRERE2kfWBCg0NBT379/HjBkzkJCQAB8fH0RFRak6Rt+8eRMGBi8uUq1atQpZWVno89Lc2jNnzsSsWbMAAD179sTq1asRERGB8ePHo169eti5cydat25dYcdFRERE2o1LYWjAeYCIiIh0j07MA0REREQkFyZAREREpHeYABEREZHe0dmJEMtTbrcozghNRESkO3K/t4vTvZkJkAZpaWkAAHd3d5kjISIiopJKS0uDjY1NoWU4CkwDpVKJu3fvwsrKCgqFQu5wyk1qairc3d1x69YtvRjtpk/Hy2OtnPTpWAH9Ol4ea9kQQiAtLQ1ubm5q0+howitAGhgYGKB69epyh1Fh9G32a306Xh5r5aRPxwro1/HyWF9dUVd+crETNBEREekdJkBERESkd5gA6TETExPMnDkTJiYmcodSIfTpeHmslZM+HSugX8fLY6147ARNREREeodXgIiIiEjvMAEiIiIivcMEiIiIiPQOEyAiIiLSO0yAKqmIiAg0b94cVlZWcHJyQo8ePRAXF1foPhs3boRCoVC7mZqaVlDEr2bWrFn5Yq9fv36h+0RGRqJ+/fowNTVF48aNceDAgQqK9tV4enrmO1aFQoExY8ZoLK9L5/W///0v3nrrLbi5uUGhUGDPnj1qjwshMGPGDLi6usLMzAxBQUG4evVqkfWuWLECnp6eMDU1RUBAAM6cOVNOR1AyhR1vdnY2pkyZgsaNG8PCwgJubm4YNGgQ7t69W2idpfksVISizu3gwYPzxd25c+ci69XGc1vUsWr6/CoUCsyfP7/AOrX1vBbnu+bZs2cYM2YM7O3tYWlpid69eyMxMbHQekv7WS8JJkCV1LFjxzBmzBicOnUKhw4dQnZ2Njp16oSMjIxC97O2tsa9e/dUtxs3blRQxK+uYcOGarH/+uuvBZY9efIk+vfvj2HDhuH3339Hjx490KNHD1y6dKkCIy6ds2fPqh3noUOHAABvv/12gfvoynnNyMhA06ZNsWLFCo2Pz5s3D0uXLsXq1atx+vRpWFhYIDg4GM+ePSuwzm3btiE8PBwzZ85EbGwsmjZtiuDgYCQlJZXXYRRbYcf75MkTxMbG4uOPP0ZsbCx27dqFuLg4dOvWrch6S/JZqChFnVsA6Ny5s1rc33//faF1auu5LepY8x7jvXv3sH79eigUCvTu3bvQerXxvBbnu+b999/Hjz/+iMjISBw7dgx3795Fr169Cq23NJ/1EhOkF5KSkgQAcezYsQLLbNiwQdjY2FRcUGVo5syZomnTpsUu37dvXxESEqK2LSAgQIwcObKMIyt/EyZMEF5eXkKpVGp8XFfPKwCxe/du1e9KpVK4uLiI+fPnq7YlJycLExMT8f333xdYj7+/vxgzZozq95ycHOHm5iYiIiLKJe7Sevl4NTlz5owAIG7cuFFgmZJ+FuSg6VjDwsJE9+7dS1SPLpzb4pzX7t27izfeeKPQMrpwXoXI/12TnJwsjIyMRGRkpKrMlStXBAARExOjsY7SftZLileA9ERKSgoAoGrVqoWWS09Ph4eHB9zd3dG9e3f8+eefFRFembh69Src3NxQq1YtDBgwADdv3iywbExMDIKCgtS2BQcHIyYmprzDLFNZWVnYsmULhg4dWujCvbp8XnPFx8cjISFB7bzZ2NggICCgwPOWlZWFc+fOqe1jYGCAoKAgnTvXgPQ5VigUsLW1LbRcST4L2uTo0aNwcnJCvXr1MGrUKDx8+LDAspXl3CYmJmL//v0YNmxYkWV14by+/F1z7tw5ZGdnq52n+vXro0aNGgWep9J81kuDCZAeUCqVmDhxIlq1aoVGjRoVWK5evXpYv349fvjhB2zZsgVKpRItW7bE7du3KzDa0gkICMDGjRsRFRWFVatWIT4+Hm3atEFaWprG8gkJCXB2dlbb5uzsjISEhIoIt8zs2bMHycnJGDx4cIFldPm85pV7bkpy3h48eICcnJxKca6fPXuGKVOmoH///oUuIFnSz4K26Ny5MzZt2oTo6GjMnTsXx44dQ5cuXZCTk6OxfGU5t9988w2srKyKbBLShfOq6bsmISEBxsbG+ZL2ws5TaT7rpcHV4PXAmDFjcOnSpSLbiwMDAxEYGKj6vWXLlmjQoAHWrFmDOXPmlHeYr6RLly6q+02aNEFAQAA8PDywffv2Yv1npau+/vprdOnSBW5ubgWW0eXzSpLs7Gz07dsXQgisWrWq0LK6+lno16+f6n7jxo3RpEkTeHl54ejRo+jQoYOMkZWv9evXY8CAAUUOTNCF81rc7xptwStAldzYsWOxb98+HDlyBNWrVy/RvkZGRnjttdfwzz//lFN05cfW1hZ169YtMHYXF5d8oxASExPh4uJSEeGViRs3buDw4cN49913S7Sfrp7X3HNTkvPm4OAAQ0NDnT7XucnPjRs3cOjQoUKv/mhS1GdBW9WqVQsODg4Fxl0Zzu3x48cRFxdX4s8woH3ntaDvGhcXF2RlZSE5OVmtfGHnqTSf9dJgAlRJCSEwduxY7N69G7/88gtq1qxZ4jpycnJw8eJFuLq6lkOE5Ss9PR3Xrl0rMPbAwEBER0erbTt06JDalRJtt2HDBjg5OSEkJKRE++nqea1ZsyZcXFzUzltqaipOnz5d4HkzNjaGr6+v2j5KpRLR0dE6ca5zk5+rV6/i8OHDsLe3L3EdRX0WtNXt27fx8OHDAuPW9XMLSFdwfX190bRp0xLvqy3ntajvGl9fXxgZGamdp7i4ONy8ebPA81Saz3ppg6dKaNSoUcLGxkYcPXpU3Lt3T3V78uSJqszAgQPF1KlTVb/Pnj1bHDx4UFy7dk2cO3dO9OvXT5iamoo///xTjkMokUmTJomjR4+K+Ph4ceLECREUFCQcHBxEUlKSECL/sZ44cUJUqVJFLFiwQFy5ckXMnDlTGBkZiYsXL8p1CCWSk5MjatSoIaZMmZLvMV0+r2lpaeL3338Xv//+uwAgFi1aJH7//XfVqKcvvvhC2Nraih9++EFcuHBBdO/eXdSsWVM8ffpUVccbb7whli1bpvp969atwsTERGzcuFFcvnxZjBgxQtja2oqEhIQKP76XFXa8WVlZolu3bqJ69eri/Pnzap/jzMxMVR0vH29RnwW5FHasaWlp4oMPPhAxMTEiPj5eHD58WDRr1kzUqVNHPHv2TFWHrpzbot7HQgiRkpIizM3NxapVqzTWoSvntTjfNe+9956oUaOG+OWXX8Rvv/0mAgMDRWBgoFo99erVE7t27VL9XpzP+qtiAlRJAdB427Bhg6pMu3btRFhYmOr3iRMniho1aghjY2Ph7OwsunbtKmJjYys++FIIDQ0Vrq6uwtjYWFSrVk2EhoaKf/75R/X4y8cqhBDbt28XdevWFcbGxqJhw4Zi//79FRx16R08eFAAEHFxcfke0+XzeuTIEY3v29zjUSqV4uOPPxbOzs7CxMREdOjQId9r4OHhIWbOnKm2bdmyZarXwN/fX5w6daqCjqhwhR1vfHx8gZ/jI0eOqOp4+XiL+izIpbBjffLkiejUqZNwdHQURkZGwsPDQwwfPjxfIqMr57ao97EQQqxZs0aYmZmJ5ORkjXXoynktznfN06dPxejRo4WdnZ0wNzcXPXv2FPfu3ctXT959ivNZf1WK/z0xERERkd5gHyAiIiLSO0yAiIiISO8wASIiIiK9wwSIiIiI9A4TICIiItI7TICIiIhI7zABIiIiIr3DBIiIqAAKhQJ79uyROwwiKgdMgIhIKw0ePBgKhSLfrXPnznKHRkSVQBW5AyAiKkjnzp2xYcMGtW0mJiYyRUNElQmvABGR1jIxMYGLi4vazc7ODoDUPLVq1Sp06dIFZmZmqFWrFnbs2KG2/8WLF/HGG2/AzMwM9vb2GDFiBNLT09XKrF+/Hg0bNoSJiQlcXV0xduxYtccfPHiAnj17wtzcHHXq1MHevXtVjz1+/BgDBgyAo6MjzMzMUKdOnXwJGxFpJyZARKSzPv74Y/Tu3Rt//PEHBgwYgH79+uHKlSsAgIyMDAQHB8POzg5nz55FZGQkDh8+rJbgrFq1CmPGjMGIESNw8eJF7N27F7Vr11Z7jtmzZ6Nv3764cOECunbtigEDBuDRo0eq5798+TJ++uknXLlyBatWrYKDg0PFvQBEVHplurQqEVEZCQsLE4aGhsLCwkLt9tlnnwkhpNWj33vvPbV9AgICxKhRo4QQQqxdu1bY2dmJ9PR01eP79+8XBgYGqlXG3dzcxPTp0wuMAYD4v//7P9Xv6enpAoD46aefhBBCvPXWW2LIkCFlc8BEVKHYB4iItNbrr7+OVatWqW2rWrWq6n5gYKDaY4GBgTh//jwA4MqVK2jatCksLCxUj7dq1QpKpRJxcXFQKBS4e/cuOnToUGgMTZo0Ud23sLCAtbU1kpKSAACjRo1C7969ERsbi06dOqFHjx5o2bJlqY6ViCoWEyAi0loWFhb5mqTKipmZWbHKGRkZqf2uUCigVCoBAF26dMGNGzdw4MABHDp0CB06dMCYMWOwYMGCMo+XiMoW+wARkc46depUvt8bNGgAAGjQoAH++OMPZGRkqB4/ceIEDAwMUK9ePVhZWcHT0xPR0dGvFIOjoyPCwsKwZcsWLF68GGvXrn2l+oioYvAKEBFprczMTCQkJKhtq1KliqqjcWRkJPz8/NC6dWt8++23OHPmDL7++msAwIABAzBz5kyEhYVh1qxZuH//PsaNG4eBAwfC2dkZADBr1iy89957cHJyQpcuXZCWloYTJ05g3LhxxYpvxowZ8PX1RcOGDZGZmYl9+/apEjAi0m5MgIhIa0VFRcHV1VVtW7169fDXX38BkEZobd26FaNHj4arqyu+//57eHt7AwDMzc1x8OBBTJgwAc2bN4e5uTl69+6NRYsWqeoKCwvDs2fP8OWXX+KDDz6Ag4MD+vTpU+z4jI2NMW3aNFy/fh1mZmZo06YNtm7dWgZHTkTlTSGEEHIHQURUUgqFArt370aPHj3kDoWIdBD7ABEREZHeYQJEREREeod9gIhIJ7H1noheBa8AERERkd5hAkRERER6hwkQERER6R0mQERERKR3mAARERGR3mECRERERHqHCRARERHpHSZAREREpHeYABEREZHe+X9Q5ctMJWEKKQAAAABJRU5ErkJggg==\n"
          },
          "metadata": {}
        }
      ],
      "source": [
        "import matplotlib.pyplot as plt\n",
        "val_loss = history_small_model.history[\"val_loss\"]\n",
        "epochs = range(1, 21)\n",
        "plt.plot(epochs, val_loss, \"b--\",\n",
        "         label=\"Validation loss\")\n",
        "plt.title(\"Effect of insufficient model capacity on validation loss\")\n",
        "plt.xlabel(\"Epochs\")\n",
        "plt.ylabel(\"Loss\")\n",
        "plt.legend()\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T01:06:26.281161Z",
          "iopub.status.busy": "2022-05-21T01:06:26.280863Z",
          "iopub.status.idle": "2022-05-21T01:06:48.235589Z",
          "shell.execute_reply": "2022-05-21T01:06:48.234644Z"
        },
        "id": "TYz3NAseTk3X",
        "outputId": "16b03190-69e2-4287-f4c9-e40545e3dcc9",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/20\n",
            "375/375 [==============================] - 2s 4ms/step - loss: 0.3762 - accuracy: 0.9053 - val_loss: 0.1875 - val_accuracy: 0.9473\n",
            "Epoch 2/20\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.1644 - accuracy: 0.9712 - val_loss: 0.1346 - val_accuracy: 0.9634\n",
            "Epoch 3/20\n",
            "375/375 [==============================] - 1s 4ms/step - loss: 0.1145 - accuracy: 0.9858 - val_loss: 0.1104 - val_accuracy: 0.9692\n",
            "Epoch 4/20\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.0876 - accuracy: 0.9927 - val_loss: 0.0990 - val_accuracy: 0.9731\n",
            "Epoch 5/20\n",
            "375/375 [==============================] - 1s 4ms/step - loss: 0.0710 - accuracy: 0.9941 - val_loss: 0.0976 - val_accuracy: 0.9707\n",
            "Epoch 6/20\n",
            "375/375 [==============================] - 1s 4ms/step - loss: 0.0570 - accuracy: 0.9951 - val_loss: 0.1163 - val_accuracy: 0.9678\n",
            "Epoch 7/20\n",
            "375/375 [==============================] - 2s 5ms/step - loss: 0.0480 - accuracy: 0.9976 - val_loss: 0.0894 - val_accuracy: 0.9766\n",
            "Epoch 8/20\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.0404 - accuracy: 0.9976 - val_loss: 0.0867 - val_accuracy: 0.9780\n",
            "Epoch 9/20\n",
            "375/375 [==============================] - 1s 4ms/step - loss: 0.0341 - accuracy: 0.9985 - val_loss: 0.0899 - val_accuracy: 0.9761\n",
            "Epoch 10/20\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.0283 - accuracy: 0.9980 - val_loss: 0.0892 - val_accuracy: 0.9780\n",
            "Epoch 11/20\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.0244 - accuracy: 0.9985 - val_loss: 0.1011 - val_accuracy: 0.9731\n",
            "Epoch 12/20\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.0197 - accuracy: 0.9995 - val_loss: 0.0994 - val_accuracy: 0.9775\n",
            "Epoch 13/20\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.0184 - accuracy: 0.9995 - val_loss: 0.0910 - val_accuracy: 0.9814\n",
            "Epoch 14/20\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.0150 - accuracy: 1.0000 - val_loss: 0.0967 - val_accuracy: 0.9800\n",
            "Epoch 15/20\n",
            "375/375 [==============================] - 1s 4ms/step - loss: 0.0124 - accuracy: 0.9995 - val_loss: 0.1055 - val_accuracy: 0.9795\n",
            "Epoch 16/20\n",
            "375/375 [==============================] - 2s 5ms/step - loss: 0.0102 - accuracy: 1.0000 - val_loss: 0.1036 - val_accuracy: 0.9785\n",
            "Epoch 17/20\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.0092 - accuracy: 1.0000 - val_loss: 0.1039 - val_accuracy: 0.9795\n",
            "Epoch 18/20\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.0079 - accuracy: 1.0000 - val_loss: 0.1063 - val_accuracy: 0.9800\n",
            "Epoch 19/20\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.0067 - accuracy: 1.0000 - val_loss: 0.1090 - val_accuracy: 0.9814\n",
            "Epoch 20/20\n",
            "375/375 [==============================] - 1s 3ms/step - loss: 0.0055 - accuracy: 0.9995 - val_loss: 0.1136 - val_accuracy: 0.9795\n"
          ]
        }
      ],
      "source": [
        "model = keras.Sequential([\n",
        "    layers.Dense(96, activation=\"relu\"),\n",
        "    layers.Dense(96, activation=\"relu\"),\n",
        "    layers.Dense(10, activation=\"softmax\"),\n",
        "])\n",
        "model.compile(optimizer=\"rmsprop\",\n",
        "              loss=\"sparse_categorical_crossentropy\",\n",
        "              metrics=[\"accuracy\"])\n",
        "history_large_model = model.fit(\n",
        "    train_images, train_labels,\n",
        "    epochs=20,\n",
        "    batch_size=128,\n",
        "    validation_split=0.2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xsjm0jXuTk3Y"
      },
      "source": [
        "## 일반화 성능 향상하기"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SCOzA1LKTk3Y"
      },
      "source": [
        "### 데이터셋 큐레이션"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cDNJQp2yTk3Y"
      },
      "source": [
        "### 특성 공학"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BA30Ys7CTk3Y"
      },
      "source": [
        "### 조기 종료 사용하기"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FS0RwwIpTk3Z"
      },
      "source": [
        "### 모델 규제하기"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g1lVn3NcTk3Z"
      },
      "source": [
        "#### 네트워크 크기 축소"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "j3r8_V04Tk3Z"
      },
      "source": [
        "**원본 모델**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T01:06:48.240327Z",
          "iopub.status.busy": "2022-05-21T01:06:48.239635Z",
          "iopub.status.idle": "2022-05-21T01:07:01.599335Z",
          "shell.execute_reply": "2022-05-21T01:07:01.598428Z"
        },
        "id": "yZ__XxRqTk3Z",
        "outputId": "ced47d1e-8a5f-434a-9b0c-1aa4b460abdb",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/20\n",
            "30/30 [==============================] - 3s 55ms/step - loss: 0.5938 - accuracy: 0.7188 - val_loss: 0.4775 - val_accuracy: 0.8223\n",
            "Epoch 2/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.3855 - accuracy: 0.8823 - val_loss: 0.3445 - val_accuracy: 0.8809\n",
            "Epoch 3/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.2793 - accuracy: 0.9102 - val_loss: 0.2954 - val_accuracy: 0.8882\n",
            "Epoch 4/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.2231 - accuracy: 0.9224 - val_loss: 0.3018 - val_accuracy: 0.8760\n",
            "Epoch 5/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.1873 - accuracy: 0.9380 - val_loss: 0.2815 - val_accuracy: 0.8838\n",
            "Epoch 6/20\n",
            "30/30 [==============================] - 0s 12ms/step - loss: 0.1600 - accuracy: 0.9487 - val_loss: 0.2976 - val_accuracy: 0.8809\n",
            "Epoch 7/20\n",
            "30/30 [==============================] - 0s 14ms/step - loss: 0.1401 - accuracy: 0.9546 - val_loss: 0.2854 - val_accuracy: 0.8882\n",
            "Epoch 8/20\n",
            "30/30 [==============================] - 0s 14ms/step - loss: 0.1210 - accuracy: 0.9629 - val_loss: 0.3162 - val_accuracy: 0.8809\n",
            "Epoch 9/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.1069 - accuracy: 0.9673 - val_loss: 0.3098 - val_accuracy: 0.8823\n",
            "Epoch 10/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.0941 - accuracy: 0.9717 - val_loss: 0.3347 - val_accuracy: 0.8794\n",
            "Epoch 11/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.0831 - accuracy: 0.9775 - val_loss: 0.3406 - val_accuracy: 0.8809\n",
            "Epoch 12/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.0732 - accuracy: 0.9805 - val_loss: 0.3789 - val_accuracy: 0.8745\n",
            "Epoch 13/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.0630 - accuracy: 0.9858 - val_loss: 0.3777 - val_accuracy: 0.8770\n",
            "Epoch 14/20\n",
            "30/30 [==============================] - 0s 12ms/step - loss: 0.0560 - accuracy: 0.9849 - val_loss: 0.3965 - val_accuracy: 0.8750\n",
            "Epoch 15/20\n",
            "30/30 [==============================] - 0s 14ms/step - loss: 0.0495 - accuracy: 0.9883 - val_loss: 0.4143 - val_accuracy: 0.8726\n",
            "Epoch 16/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.0435 - accuracy: 0.9907 - val_loss: 0.4888 - val_accuracy: 0.8647\n",
            "Epoch 17/20\n",
            "30/30 [==============================] - 0s 12ms/step - loss: 0.0355 - accuracy: 0.9946 - val_loss: 0.4541 - val_accuracy: 0.8745\n",
            "Epoch 18/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.0328 - accuracy: 0.9941 - val_loss: 0.4729 - val_accuracy: 0.8726\n",
            "Epoch 19/20\n",
            "30/30 [==============================] - 0s 16ms/step - loss: 0.0266 - accuracy: 0.9976 - val_loss: 0.4976 - val_accuracy: 0.8711\n",
            "Epoch 20/20\n",
            "30/30 [==============================] - 0s 16ms/step - loss: 0.0252 - accuracy: 0.9966 - val_loss: 0.5156 - val_accuracy: 0.8711\n"
          ]
        }
      ],
      "source": [
        "from tensorflow.keras.datasets import imdb\n",
        "(train_data, train_labels), _ = imdb.load_data(num_words=10000)\n",
        "\n",
        "def vectorize_sequences(sequences, dimension=10000):\n",
        "    results = np.zeros((len(sequences), dimension))\n",
        "    for i, sequence in enumerate(sequences):\n",
        "        results[i, sequence] = 1.\n",
        "    return results\n",
        "train_data = vectorize_sequences(train_data)\n",
        "\n",
        "model = keras.Sequential([\n",
        "    layers.Dense(16, activation=\"relu\"),\n",
        "    layers.Dense(16, activation=\"relu\"),\n",
        "    layers.Dense(1, activation=\"sigmoid\")\n",
        "])\n",
        "model.compile(optimizer=\"rmsprop\",\n",
        "              loss=\"binary_crossentropy\",\n",
        "              metrics=[\"accuracy\"])\n",
        "history_original = model.fit(train_data, train_labels,\n",
        "                             epochs=20, batch_size=512, validation_split=0.4)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ur2zKeT6Tk3a"
      },
      "source": [
        "**작은 용량의 모델**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T01:07:01.603485Z",
          "iopub.status.busy": "2022-05-21T01:07:01.603168Z",
          "iopub.status.idle": "2022-05-21T01:07:09.910749Z",
          "shell.execute_reply": "2022-05-21T01:07:09.909893Z"
        },
        "id": "J07S8kRUTk3a",
        "outputId": "fe7073fa-336f-4ed3-bcea-f881fc9b8c02",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/20\n",
            "30/30 [==============================] - 2s 53ms/step - loss: 0.6519 - accuracy: 0.5933 - val_loss: 0.6040 - val_accuracy: 0.7178\n",
            "Epoch 2/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.5728 - accuracy: 0.7461 - val_loss: 0.5542 - val_accuracy: 0.7925\n",
            "Epoch 3/20\n",
            "30/30 [==============================] - 0s 12ms/step - loss: 0.5244 - accuracy: 0.8154 - val_loss: 0.5210 - val_accuracy: 0.7886\n",
            "Epoch 4/20\n",
            "30/30 [==============================] - 0s 12ms/step - loss: 0.4868 - accuracy: 0.8511 - val_loss: 0.4944 - val_accuracy: 0.8511\n",
            "Epoch 5/20\n",
            "30/30 [==============================] - 0s 12ms/step - loss: 0.4570 - accuracy: 0.8818 - val_loss: 0.4780 - val_accuracy: 0.8369\n",
            "Epoch 6/20\n",
            "30/30 [==============================] - 0s 12ms/step - loss: 0.4326 - accuracy: 0.9004 - val_loss: 0.4685 - val_accuracy: 0.8369\n",
            "Epoch 7/20\n",
            "30/30 [==============================] - 0s 12ms/step - loss: 0.4116 - accuracy: 0.9146 - val_loss: 0.4497 - val_accuracy: 0.8799\n",
            "Epoch 8/20\n",
            "30/30 [==============================] - 0s 12ms/step - loss: 0.3926 - accuracy: 0.9292 - val_loss: 0.4441 - val_accuracy: 0.8696\n",
            "Epoch 9/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.3770 - accuracy: 0.9365 - val_loss: 0.4385 - val_accuracy: 0.8696\n",
            "Epoch 10/20\n",
            "30/30 [==============================] - 0s 14ms/step - loss: 0.3616 - accuracy: 0.9463 - val_loss: 0.4299 - val_accuracy: 0.8784\n",
            "Epoch 11/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.3477 - accuracy: 0.9541 - val_loss: 0.4536 - val_accuracy: 0.8530\n",
            "Epoch 12/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.3340 - accuracy: 0.9580 - val_loss: 0.4399 - val_accuracy: 0.8657\n",
            "Epoch 13/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.3203 - accuracy: 0.9648 - val_loss: 0.4412 - val_accuracy: 0.8638\n",
            "Epoch 14/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.3042 - accuracy: 0.9658 - val_loss: 0.4087 - val_accuracy: 0.8818\n",
            "Epoch 15/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.2827 - accuracy: 0.9707 - val_loss: 0.4180 - val_accuracy: 0.8706\n",
            "Epoch 16/20\n",
            "30/30 [==============================] - 0s 12ms/step - loss: 0.2534 - accuracy: 0.9736 - val_loss: 0.3831 - val_accuracy: 0.8794\n",
            "Epoch 17/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.2211 - accuracy: 0.9771 - val_loss: 0.3713 - val_accuracy: 0.8784\n",
            "Epoch 18/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.1882 - accuracy: 0.9785 - val_loss: 0.3606 - val_accuracy: 0.8774\n",
            "Epoch 19/20\n",
            "30/30 [==============================] - 1s 19ms/step - loss: 0.1589 - accuracy: 0.9805 - val_loss: 0.3386 - val_accuracy: 0.8823\n",
            "Epoch 20/20\n",
            "30/30 [==============================] - 0s 15ms/step - loss: 0.1340 - accuracy: 0.9824 - val_loss: 0.3340 - val_accuracy: 0.8833\n"
          ]
        }
      ],
      "source": [
        "model = keras.Sequential([\n",
        "    layers.Dense(4, activation=\"relu\"),\n",
        "    layers.Dense(4, activation=\"relu\"),\n",
        "    layers.Dense(1, activation=\"sigmoid\")\n",
        "])\n",
        "model.compile(optimizer=\"rmsprop\",\n",
        "              loss=\"binary_crossentropy\",\n",
        "              metrics=[\"accuracy\"])\n",
        "history_smaller_model = model.fit(\n",
        "    train_data, train_labels,\n",
        "    epochs=20, batch_size=512, validation_split=0.4)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "k6040fZeTk3a"
      },
      "source": [
        "**큰 용량의 모델**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T01:07:09.914619Z",
          "iopub.status.busy": "2022-05-21T01:07:09.914070Z",
          "iopub.status.idle": "2022-05-21T01:07:55.440909Z",
          "shell.execute_reply": "2022-05-21T01:07:55.439855Z"
        },
        "id": "gRyaH_CkTk3b",
        "outputId": "22c447e8-58d8-45b9-c6e2-f3aa71437290",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/20\n",
            "30/30 [==============================] - 2s 53ms/step - loss: 0.5781 - accuracy: 0.7070 - val_loss: 0.3630 - val_accuracy: 0.8560\n",
            "Epoch 2/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.3296 - accuracy: 0.8623 - val_loss: 0.3074 - val_accuracy: 0.8687\n",
            "Epoch 3/20\n",
            "30/30 [==============================] - 0s 14ms/step - loss: 0.2310 - accuracy: 0.9097 - val_loss: 0.3416 - val_accuracy: 0.8569\n",
            "Epoch 4/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.1981 - accuracy: 0.9180 - val_loss: 0.3533 - val_accuracy: 0.8462\n",
            "Epoch 5/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.1459 - accuracy: 0.9448 - val_loss: 0.3137 - val_accuracy: 0.8672\n",
            "Epoch 6/20\n",
            "30/30 [==============================] - 0s 14ms/step - loss: 0.0991 - accuracy: 0.9658 - val_loss: 0.3301 - val_accuracy: 0.8823\n",
            "Epoch 7/20\n",
            "30/30 [==============================] - 0s 14ms/step - loss: 0.0792 - accuracy: 0.9736 - val_loss: 0.4399 - val_accuracy: 0.8687\n",
            "Epoch 8/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.0660 - accuracy: 0.9766 - val_loss: 0.7881 - val_accuracy: 0.7910\n",
            "Epoch 9/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.0575 - accuracy: 0.9834 - val_loss: 0.4260 - val_accuracy: 0.8774\n",
            "Epoch 10/20\n",
            "30/30 [==============================] - 0s 14ms/step - loss: 0.0781 - accuracy: 0.9785 - val_loss: 0.3845 - val_accuracy: 0.8857\n",
            "Epoch 11/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.0100 - accuracy: 0.9995 - val_loss: 0.4487 - val_accuracy: 0.8857\n",
            "Epoch 12/20\n",
            "30/30 [==============================] - 0s 14ms/step - loss: 0.0807 - accuracy: 0.9819 - val_loss: 0.4062 - val_accuracy: 0.8838\n",
            "Epoch 13/20\n",
            "30/30 [==============================] - 0s 14ms/step - loss: 0.0057 - accuracy: 1.0000 - val_loss: 0.4863 - val_accuracy: 0.8857\n",
            "Epoch 14/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.0322 - accuracy: 0.9922 - val_loss: 0.5298 - val_accuracy: 0.8545\n",
            "Epoch 15/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.0066 - accuracy: 0.9990 - val_loss: 0.4905 - val_accuracy: 0.8848\n",
            "Epoch 16/20\n",
            "30/30 [==============================] - 0s 12ms/step - loss: 0.0024 - accuracy: 1.0000 - val_loss: 0.5449 - val_accuracy: 0.8818\n",
            "Epoch 17/20\n",
            "30/30 [==============================] - 0s 14ms/step - loss: 0.0022 - accuracy: 1.0000 - val_loss: 1.5303 - val_accuracy: 0.7710\n",
            "Epoch 18/20\n",
            "30/30 [==============================] - 0s 14ms/step - loss: 0.0626 - accuracy: 0.9834 - val_loss: 0.4951 - val_accuracy: 0.8833\n",
            "Epoch 19/20\n",
            "30/30 [==============================] - 0s 16ms/step - loss: 0.0018 - accuracy: 1.0000 - val_loss: 0.5664 - val_accuracy: 0.8838\n",
            "Epoch 20/20\n",
            "30/30 [==============================] - 1s 17ms/step - loss: 0.0452 - accuracy: 0.9902 - val_loss: 0.4851 - val_accuracy: 0.8774\n"
          ]
        }
      ],
      "source": [
        "model = keras.Sequential([\n",
        "    layers.Dense(512, activation=\"relu\"),\n",
        "    layers.Dense(512, activation=\"relu\"),\n",
        "    layers.Dense(1, activation=\"sigmoid\")\n",
        "])\n",
        "model.compile(optimizer=\"rmsprop\",\n",
        "              loss=\"binary_crossentropy\",\n",
        "              metrics=[\"accuracy\"])\n",
        "history_larger_model = model.fit(\n",
        "    train_data, train_labels,\n",
        "    epochs=20, batch_size=512, validation_split=0.4)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HkMwsNWiTk3b"
      },
      "source": [
        "#### 가중치 규제 추가하기"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "slmc7NwkTk3b"
      },
      "source": [
        "**모델에 L2 가중치 추가하기**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T01:07:55.445676Z",
          "iopub.status.busy": "2022-05-21T01:07:55.444826Z",
          "iopub.status.idle": "2022-05-21T01:08:04.225182Z",
          "shell.execute_reply": "2022-05-21T01:08:04.224432Z"
        },
        "id": "BbKqHmXtTk3b",
        "outputId": "2926e882-da20-4d4e-9490-44291894b789",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/20\n",
            "30/30 [==============================] - 2s 54ms/step - loss: 0.6064 - accuracy: 0.7861 - val_loss: 0.4951 - val_accuracy: 0.8433\n",
            "Epoch 2/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.4094 - accuracy: 0.8936 - val_loss: 0.4023 - val_accuracy: 0.8770\n",
            "Epoch 3/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.3438 - accuracy: 0.9092 - val_loss: 0.3677 - val_accuracy: 0.8906\n",
            "Epoch 4/20\n",
            "30/30 [==============================] - 0s 12ms/step - loss: 0.3059 - accuracy: 0.9248 - val_loss: 0.3633 - val_accuracy: 0.8887\n",
            "Epoch 5/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.2842 - accuracy: 0.9292 - val_loss: 0.3992 - val_accuracy: 0.8672\n",
            "Epoch 6/20\n",
            "30/30 [==============================] - 0s 12ms/step - loss: 0.2688 - accuracy: 0.9375 - val_loss: 0.3679 - val_accuracy: 0.8838\n",
            "Epoch 7/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.2593 - accuracy: 0.9404 - val_loss: 0.3628 - val_accuracy: 0.8872\n",
            "Epoch 8/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.2510 - accuracy: 0.9414 - val_loss: 0.3997 - val_accuracy: 0.8711\n",
            "Epoch 9/20\n",
            "30/30 [==============================] - 0s 14ms/step - loss: 0.2360 - accuracy: 0.9526 - val_loss: 0.3992 - val_accuracy: 0.8721\n",
            "Epoch 10/20\n",
            "30/30 [==============================] - 0s 14ms/step - loss: 0.2443 - accuracy: 0.9438 - val_loss: 0.3770 - val_accuracy: 0.8838\n",
            "Epoch 11/20\n",
            "30/30 [==============================] - 0s 12ms/step - loss: 0.2307 - accuracy: 0.9502 - val_loss: 0.3809 - val_accuracy: 0.8818\n",
            "Epoch 12/20\n",
            "30/30 [==============================] - 0s 12ms/step - loss: 0.2235 - accuracy: 0.9546 - val_loss: 0.3838 - val_accuracy: 0.8799\n",
            "Epoch 13/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.2229 - accuracy: 0.9531 - val_loss: 0.4050 - val_accuracy: 0.8745\n",
            "Epoch 14/20\n",
            "30/30 [==============================] - 0s 12ms/step - loss: 0.2180 - accuracy: 0.9580 - val_loss: 0.4011 - val_accuracy: 0.8774\n",
            "Epoch 15/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.2188 - accuracy: 0.9551 - val_loss: 0.4431 - val_accuracy: 0.8672\n",
            "Epoch 16/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.2150 - accuracy: 0.9575 - val_loss: 0.4329 - val_accuracy: 0.8706\n",
            "Epoch 17/20\n",
            "30/30 [==============================] - 0s 12ms/step - loss: 0.2054 - accuracy: 0.9614 - val_loss: 0.4363 - val_accuracy: 0.8696\n",
            "Epoch 18/20\n",
            "30/30 [==============================] - 1s 20ms/step - loss: 0.2084 - accuracy: 0.9595 - val_loss: 0.4292 - val_accuracy: 0.8735\n",
            "Epoch 19/20\n",
            "30/30 [==============================] - 1s 17ms/step - loss: 0.1978 - accuracy: 0.9634 - val_loss: 0.4160 - val_accuracy: 0.8774\n",
            "Epoch 20/20\n",
            "30/30 [==============================] - 1s 21ms/step - loss: 0.2053 - accuracy: 0.9595 - val_loss: 0.4197 - val_accuracy: 0.8770\n"
          ]
        }
      ],
      "source": [
        "from tensorflow.keras import regularizers\n",
        "model = keras.Sequential([\n",
        "    layers.Dense(16,\n",
        "                 kernel_regularizer=regularizers.l2(0.002),\n",
        "                 activation=\"relu\"),\n",
        "    layers.Dense(16,\n",
        "                 kernel_regularizer=regularizers.l2(0.002),\n",
        "                 activation=\"relu\"),\n",
        "    layers.Dense(1, activation=\"sigmoid\")\n",
        "])\n",
        "model.compile(optimizer=\"rmsprop\",\n",
        "              loss=\"binary_crossentropy\",\n",
        "              metrics=[\"accuracy\"])\n",
        "history_l2_reg = model.fit(\n",
        "    train_data, train_labels,\n",
        "    epochs=20, batch_size=512, validation_split=0.4)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "766AbsD9Tk3c"
      },
      "source": [
        "**케라스에서 사용할 수 있는 가중치 규제**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T01:08:04.229728Z",
          "iopub.status.busy": "2022-05-21T01:08:04.228866Z",
          "iopub.status.idle": "2022-05-21T01:08:04.237099Z",
          "shell.execute_reply": "2022-05-21T01:08:04.235990Z"
        },
        "id": "tT6wJynITk3c",
        "outputId": "32ad3b41-e0fe-4fbd-f0f3-324c7b0b7e5d",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.src.regularizers.L1L2 at 0x7bd538d04ac0>"
            ]
          },
          "metadata": {},
          "execution_count": 16
        }
      ],
      "source": [
        "from tensorflow.keras import regularizers\n",
        "regularizers.l1(0.001)\n",
        "regularizers.l1_l2(l1=0.001, l2=0.001)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ultTwtPKTk3c"
      },
      "source": [
        "#### 드롭아웃 추가"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MWW1i75kTk3c"
      },
      "source": [
        "**IMDB 모델에 드롭아웃 추가하기**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T01:08:04.241161Z",
          "iopub.status.busy": "2022-05-21T01:08:04.240762Z",
          "iopub.status.idle": "2022-05-21T01:08:13.113623Z",
          "shell.execute_reply": "2022-05-21T01:08:13.112554Z"
        },
        "id": "K6C938cRTk3d",
        "outputId": "bf684db2-2db1-447e-c6a0-dbac601268bf",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/20\n",
            "30/30 [==============================] - 3s 53ms/step - loss: 0.6196 - accuracy: 0.6484 - val_loss: 0.4907 - val_accuracy: 0.8433\n",
            "Epoch 2/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.4905 - accuracy: 0.7754 - val_loss: 0.3745 - val_accuracy: 0.8735\n",
            "Epoch 3/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.4167 - accuracy: 0.8252 - val_loss: 0.3311 - val_accuracy: 0.8848\n",
            "Epoch 4/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.3608 - accuracy: 0.8628 - val_loss: 0.2959 - val_accuracy: 0.8896\n",
            "Epoch 5/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.3181 - accuracy: 0.8843 - val_loss: 0.2827 - val_accuracy: 0.8906\n",
            "Epoch 6/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.2900 - accuracy: 0.8975 - val_loss: 0.2896 - val_accuracy: 0.8823\n",
            "Epoch 7/20\n",
            "30/30 [==============================] - 0s 14ms/step - loss: 0.2563 - accuracy: 0.9111 - val_loss: 0.2737 - val_accuracy: 0.8926\n",
            "Epoch 8/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.2261 - accuracy: 0.9243 - val_loss: 0.2917 - val_accuracy: 0.8896\n",
            "Epoch 9/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.2070 - accuracy: 0.9321 - val_loss: 0.2847 - val_accuracy: 0.8911\n",
            "Epoch 10/20\n",
            "30/30 [==============================] - 0s 12ms/step - loss: 0.1917 - accuracy: 0.9380 - val_loss: 0.2971 - val_accuracy: 0.8896\n",
            "Epoch 11/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.1716 - accuracy: 0.9443 - val_loss: 0.3088 - val_accuracy: 0.8887\n",
            "Epoch 12/20\n",
            "30/30 [==============================] - 0s 14ms/step - loss: 0.1566 - accuracy: 0.9497 - val_loss: 0.3286 - val_accuracy: 0.8872\n",
            "Epoch 13/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.1407 - accuracy: 0.9546 - val_loss: 0.3684 - val_accuracy: 0.8823\n",
            "Epoch 14/20\n",
            "30/30 [==============================] - 0s 13ms/step - loss: 0.1340 - accuracy: 0.9575 - val_loss: 0.3696 - val_accuracy: 0.8882\n",
            "Epoch 15/20\n",
            "30/30 [==============================] - 0s 14ms/step - loss: 0.1214 - accuracy: 0.9600 - val_loss: 0.3962 - val_accuracy: 0.8872\n",
            "Epoch 16/20\n",
            "30/30 [==============================] - 1s 22ms/step - loss: 0.1153 - accuracy: 0.9619 - val_loss: 0.4092 - val_accuracy: 0.8872\n",
            "Epoch 17/20\n",
            "30/30 [==============================] - 0s 17ms/step - loss: 0.1052 - accuracy: 0.9668 - val_loss: 0.4021 - val_accuracy: 0.8857\n",
            "Epoch 18/20\n",
            "30/30 [==============================] - 1s 22ms/step - loss: 0.0999 - accuracy: 0.9678 - val_loss: 0.4463 - val_accuracy: 0.8862\n",
            "Epoch 19/20\n",
            "30/30 [==============================] - 0s 16ms/step - loss: 0.0917 - accuracy: 0.9697 - val_loss: 0.4695 - val_accuracy: 0.8857\n",
            "Epoch 20/20\n",
            "30/30 [==============================] - 0s 12ms/step - loss: 0.0877 - accuracy: 0.9717 - val_loss: 0.5142 - val_accuracy: 0.8857\n"
          ]
        }
      ],
      "source": [
        "model = keras.Sequential([\n",
        "    layers.Dense(16, activation=\"relu\"),\n",
        "    layers.Dropout(0.5),\n",
        "    layers.Dense(16, activation=\"relu\"),\n",
        "    layers.Dropout(0.5),\n",
        "    layers.Dense(1, activation=\"sigmoid\")\n",
        "])\n",
        "model.compile(optimizer=\"rmsprop\",\n",
        "              loss=\"binary_crossentropy\",\n",
        "              metrics=[\"accuracy\"])\n",
        "history_dropout = model.fit(\n",
        "    train_data, train_labels,\n",
        "    epochs=20, batch_size=512, validation_split=0.4)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JuocOmx8Tk3d"
      },
      "source": [
        "## 요약"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "chapter05_fundamentals-of-ml.i",
      "provenance": [],
      "gpuType": "T4"
    },
    "kernelspec": {
      "display_name": "default:Python",
      "language": "python",
      "name": "conda-env-default-py"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.10"
    },
    "accelerator": "GPU"
  },
  "nbformat": 4,
  "nbformat_minor": 0
}