{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2024年 6月24日 月曜日 14時51分16秒 JST\n",
      "Python 3.9.6\n"
     ]
    }
   ],
   "source": [
    "!date\n",
    "!python --version"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# classification_report, ConfusionMatrixDisplayの実行例\n",
    "分類タスクにおける代表的な評価を出力するコード例。\n",
    "\n",
    "基本的には (1) 教師ラベルのリスト、(2) 予測結果のリスト、(3) 表示用ラベルの3点を用意してから以下の関数を利用するだけで良い。本ノートブックでは教師ラベルのリストを y_true、予測結果のリストを y_predicted、表示用ラベルを labels として用意し、出力した例を示す。\n",
    "\n",
    "- 参考\n",
    "    - [classification_report](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.classification_report.html)\n",
    "    - [ConfusionMatrixDisplay](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.ConfusionMatrixDisplay.html)）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import classification_report, confusion_matrix, ConfusionMatrixDisplay\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## データの用意"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['negative', 'normal', 'positive']"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 教師ラベルと推測ラベルを用意\n",
    "y_true = [\"negative\", \"positive\", \"positive\", \"normal\"]\n",
    "y_predicted = [\"negative\", \"positive\", \"negative\", \"negative\"] # 3,4番目が推定誤り\n",
    "\n",
    "# ラベル名一覧を用意\n",
    "# classification_report, confusion_matrixではラベル名がソート順で出力されるのでソートすると良い\n",
    "labels = [\"positive\", \"normal\", \"negative\"]\n",
    "sorted(labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## classification_report\n",
    "ラベル毎の precision, recall, f1-score, support を出力し、それに基づく総合評価もまとめて出力してくれる。\n",
    "\n",
    "- precision（適合率）：正と予測した結果がどれだけ正しかったか。\n",
    "- recall（再現率）：実際に正であったサンプルのうち、モデルが正と予測したサンプルの割合。\n",
    "- F1-score（F1スコア）：適合率と再現率の調和平均。\n",
    "- accuracy（正解率）：全てのサンプルのうち、正しく分類されたサンプルの割合。\n",
    "- macro avg：サンプル数に基づいたスコア（単純平均）。\n",
    "    - 例示した実行結果の場合、次のように求める。\n",
    "    - precision: (1.00 + 0.00 + 0.33) / 3 = 0.44\n",
    "    - recall: (0.50 + 0.00 + 1.00) / 3 = 0.50\n",
    "    - f1-score: (0.67 + 0.00 + 0.50) / 3 = 0.39\n",
    "- weighted avg：サンプル数に基づき重み調整したスコア。サンプル数の多いクラスのスコアをより重視する。\n",
    "    - 例示した実行結果の場合、次のように求める。\n",
    "    - precision: (1.00*2 + 0.00*1 + 0.33*1) / (2+1+1) = 0.58\n",
    "    - recall: (0.50*2 + 0.00*1 + 1.00*1) / (2+1+1) = 0.50\n",
    "    - f1-score: (0.67*2 + 0.00*1 + 0.50*1) / (2+1+1) = 0.46\n",
    "    - サンプル数が多いクラス（positive）のスコアが相対的に高いため、総合評価も良くなっている。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classification Report:\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "    positive       1.00      0.50      0.67         2\n",
      "      normal       0.00      0.00      0.00         1\n",
      "    negative       0.33      1.00      0.50         1\n",
      "\n",
      "    accuracy                           0.50         4\n",
      "   macro avg       0.44      0.50      0.39         4\n",
      "weighted avg       0.58      0.50      0.46         4\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/tnal/.venv/dm/lib/python3.9/site-packages/sklearn/metrics/_classification.py:1344: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.\n",
      "  _warn_prf(average, modifier, msg_start, len(result))\n",
      "/Users/tnal/.venv/dm/lib/python3.9/site-packages/sklearn/metrics/_classification.py:1344: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.\n",
      "  _warn_prf(average, modifier, msg_start, len(result))\n",
      "/Users/tnal/.venv/dm/lib/python3.9/site-packages/sklearn/metrics/_classification.py:1344: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.\n",
      "  _warn_prf(average, modifier, msg_start, len(result))\n"
     ]
    }
   ],
   "source": [
    "# classification_report出力\n",
    "# 教師ラベル、推測ラベル、表示用ラベルを与えて実行。\n",
    "report = classification_report(y_true, y_predicted, labels=labels)\n",
    "print(\"Classification Report:\")\n",
    "print(report)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ConfusionMatrixDisplay\n",
    "各ラベルに該当するサンプルをどのように推定したのかを確認する混合行列を出力する。\n",
    "- 行の「True label」が教師データ。列の「Predicted label」が予測結果。\n",
    "    - 例えば左上の1は、「教師ラベルpositiveとなるサンプル1つをpositiveと予測した（正解した）」ことを意味する。対角成分は正解したサンプル数を示すことになる。\n",
    "    - 真ん中の行は「教師ラベルnormalとなるサンプル1つをpositiveと予測した」ことを意味している。\n",
    "    - 下の行は「教師ラベルnegativeとなるサンプルは2つあり（行方向の総和=2）、一つはpositiveと推測、もう一つはnegativeと推測した」ことを意味している。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<sklearn.metrics._plot.confusion_matrix.ConfusionMatrixDisplay at 0x122c418e0>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAioAAAG2CAYAAABRfK0WAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABF0ElEQVR4nO3deVyVZf7/8fcBZZPVUFBC0VyCcjcdMMMKw7ExsykdpVTGZSxJky+lfkvRXJhp0my3NLd+mvbNMkszjQlzyzW1GQ33YFJcxgXREDnn/v3heOokKniAc255PR+P+/Ho3Oe+7utzzh3w8XNd93VbDMMwBAAA4IY8XB0AAADA1ZCoAAAAt0WiAgAA3BaJCgAAcFskKgAAwG2RqAAAALdFogIAANwWiQoAAHBbJCoAAMBtkagAAAC3RaICAACu65tvvlG3bt1Ut25dWSwWLVmy5LptsrKy1Lp1a3l7e6tRo0aaM2dOmfslUQEAANd17tw5tWjRQm+++Wapjj948KAefPBB3Xvvvdq+fbueeeYZDRw4UF9++WWZ+rXwUEIAAFAWFotFn3zyiR5++OGrHjNy5EgtW7ZM//znP+37/vSnP+n06dNasWJFqfuq5kygKF82m02HDx9WQECALBaLq8MBAJSRYRg6e/as6tatKw+Pihu0KCwsVFFRkdPnMQzjir833t7e8vb2dvrcGzZsUEJCgsO+xMREPfPMM2U6D4mKGzl8+LAiIyNdHQYAwEm5ubm69dZbK+TchYWFalDfX3nHrE6fy9/fXwUFBQ770tPTNW7cOKfPnZeXp7CwMId9YWFhys/P188//yxfX99SnYdExY0EBARIkn7cFqVAf6YP3ex6NGnm6hAAlLNiXdRaLbf/Pq8IRUVFyjtm1Y9boxQYcON/K/LP2lS/zSHl5uYqMDDQvr88qinliUTFjVwuvwX6ezj1Px/MoZqluqtDAFDe/jvrszKG7/0DLPIPuPF+bPrv35zAQIdEpbyEh4fr6NGjDvuOHj2qwMDAUldTJBIVAABMyWrYZHXidhirYSu/YEoQGxur5cuXO+xbtWqVYmNjy3Qe/tkOAIAJ2WQ4vZVFQUGBtm/fru3bt0u6dPvx9u3blZOTI0kaPXq0+vbtaz9+yJAhOnDggJ577jn98MMPeuutt/Thhx9qxIgRZeqXRAUAAFzXli1b1KpVK7Vq1UqSlJqaqlatWmns2LGSpCNHjtiTFklq0KCBli1bplWrVqlFixaaMmWKZs6cqcTExDL1y9APAAAmZJNNzgzelLV1p06ddK2l10padbZTp0767rvvyhqaAxIVAABMyGoYsjqxZqszbSsTQz8AAMBtUVEBAMCEbmRC7G/bmwGJCgAAJmSTIWsVSFQY+gEAAG6LigoAACbE0A8AAHBb3PUDAADgYlRUAAAwIdt/N2famwGJCgAAJmR18q4fZ9pWJhIVAABMyGrIyacnl18sFYk5KgAAwG1RUQEAwISYowIAANyWTRZZZXGqvRkw9AMAANwWFRUAAEzIZlzanGlvBiQqAACYkNXJoR9n2lYmhn4AAIDboqICAIAJVZWKCokKAAAmZDMsshlO3PXjRNvKxNAPAABwW1RUAAAwIYZ+AACA27LKQ1YnBkas5RhLRSJRAQDAhAwn56gYzFEBAABwDhUVAABMiDkqAADAbVkND1kNJ+aomGQJfYZ+AACA26KiAgCACdlkkc2JeoNN5iipkKgAAGBCVWWOCkM/AADAbVFRAQDAhJyfTMvQDwAAqCCX5qg48VBChn4AAACcQ0UFAAATsjn5rB/u+gEAABWGOSoAAMBt2eRRJdZRYY4KAABwW1RUAAAwIathkdVwYsE3J9pWJhIVAABMyOrkZForQz8AAADOoaICAIAJ2QwP2Zy468fGXT8AAKCiMPQDAADgYlRUAAAwIZucu3PHVn6hVCgSFQAATMj5Bd/MMahijigBAECVREUFAAATcv5ZP+aoVZCoAABgQjZZZJMzc1RYmRa4pu+/raH/e6u29n7vp5NHqyv9vYOK+/0ZV4eFCtKt/wk9+uQx1axVrAO7fPXWCxHK3u7n6rBQQbjeFa+qVFTMEWU5ysrKksVi0enTp695XFRUlKZNm1YpMVVVhec91PCOn5Uy+d+uDgUVLP6hUxqcfljzp4ZraGITHdjlo0kLDijolouuDg0VgOuN8lTlEpW4uDgdOXJEQUFBkqQ5c+YoODj4iuM2b96swYMHV3J0Vctd951V/5F56kAV5ab3yOATWrGgplYuqqmcvT56beStuvCzRYm9T7o6NFQArnfluLzgmzObGZgjynLk5eWl8PBwWSzXHpurVauW/PwoUwLOqlbdpsbNz2vbmgD7PsOw6Ls1AYppc96FkaEicL0rj82wOL2ZgVsmKp06dVJKSopSUlIUFBSk0NBQjRkzRsZ/n0tw6tQp9e3bVyEhIfLz89Pvf/977d27197+xx9/VLdu3RQSEqIaNWrojjvu0PLlyyU5Dv1kZWUpOTlZZ86ckcVikcVi0bhx4yQ5Dv306dNHvXr1cojx4sWLCg0N1bx58yRJNptNGRkZatCggXx9fdWiRQt99NFHFfxNAe4vsKZVntWk08cdp8SdOlFNIbWKXRQVKgrXG+XNbSfTzp07VwMGDNCmTZu0ZcsWDR48WPXq1dOgQYPUv39/7d27V0uXLlVgYKBGjhyprl27ateuXapevbqGDh2qoqIiffPNN6pRo4Z27dolf3//K/qIi4vTtGnTNHbsWGVnZ0tSicclJSXpscceU0FBgf39L7/8UufPn1ePHj0kSRkZGfp//+//afr06WrcuLG++eYbPf7446pVq5bi4+NL/IwXLlzQhQsX7K/z8/Od/t4AAFWDzcnhG7Ms+Oa2iUpkZKReeeUVWSwWNW3aVN9//71eeeUVderUSUuXLtW6desUFxcnSZo/f74iIyO1ZMkSPfbYY8rJydEf//hHNWvWTJLUsGHDEvvw8vJSUFCQLBaLwsPDrxpLYmKiatSooU8++URPPPGEJGnBggV66KGHFBAQoAsXLmjy5Mn66quvFBsba+9z7dq1euedd66aqGRkZGj8+PE3/B0BZpB/0lPWYin4N/+aDgkt1qnjbvsrCDeI6115nH96sjkSFbeN8ne/+53DPJLY2Fjt3btXu3btUrVq1dS+fXv7e7fccouaNm2q3bt3S5KGDRumiRMnqkOHDkpPT9fOnTudiqVatWrq2bOn5s+fL0k6d+6cPv30UyUlJUmS9u3bp/Pnz6tz587y9/e3b/PmzdP+/fuvet7Ro0frzJkz9i03N9epOAF3VHzRQ3t3+qnV3Wft+ywWQy3vLtCurcwDu9lwvVHebsr0duDAgUpMTNSyZcu0cuVKZWRkaMqUKXr66adv+JxJSUmKj4/XsWPHtGrVKvn6+qpLly6SpIKCAknSsmXLFBER4dDO29v7quf09va+5vs3u5/PeejwwV8+f16ul/b/01cBwcWqfSu3Md5MPn43VGnTcrVnh5+yv/NTj0HH5eNn08qFNV0dGioA17tyWGWR1YlF25xpW5ncNlHZuHGjw+tvv/1WjRs3VkxMjIqLi7Vx40b70M9//vMfZWdnKyYmxn58ZGSkhgwZoiFDhmj06NGaMWNGiYmKl5eXrFbrdeOJi4tTZGSkFi1apC+++EKPPfaYqlevLkmKiYmRt7e3cnJyrjrMgyvt2eGn5x5tZH/9zrhLSV7nnieVNi3HVWGhAqxeGqKgW6zq+2yeQmoV68C/fPV8UgOdPlHd1aGhAnC9K0dVGfpx20QlJydHqamp+stf/qJt27bp9ddf15QpU9S4cWN1795dgwYN0jvvvKOAgACNGjVKERER6t69uyTpmWee0e9//3s1adJEp06d0tdff63o6OgS+4mKilJBQYEyMzPVokUL+fn5XfW25D59+mj69Onas2ePvv76a/v+gIAApaWlacSIEbLZbLr77rt15swZrVu3ToGBgerXr1/5f0E3gRZxBfry8HZXh4FKsnR2qJbODnV1GKgkXG+UF7dNp/r27auff/5Z7dq109ChQzV8+HD7AmyzZ89WmzZt9Ic//EGxsbEyDEPLly+3VzisVquGDh2q6OhodenSRU2aNNFbb71VYj9xcXEaMmSIevXqpVq1aumll166akxJSUnatWuXIiIi1KFDB4f3JkyYoDFjxigjI8Pe77Jly9SgQYNy+kYAAPiFVb8M/9zYZg4W4/LiJG6kU6dOatmyZZVbwj4/P19BQUE6taehAgPcNodEOUms29LVIQAoZ8XGRWXpU505c0aBgYEV0sflvxUvfPuAfPxvfDitsOCiJv5uZYXGWh7cdugHAABcHQ8lBAAA+I0333xTUVFR8vHxUfv27bVp06ZrHj9t2jQ1bdpUvr6+ioyM1IgRI1RYWFjq/tyyopKVleXqEAAAcGuGLLI5cYuxcQNtFy1apNTUVE2fPl3t27fXtGnTlJiYqOzsbNWuXfuK4xcsWKBRo0Zp1qxZiouL0549e9S/f39ZLBZNnTq1VH1SUQEAwIQuD/04s5XV1KlTNWjQICUnJysmJkbTp0+Xn5+fZs2aVeLx69evV4cOHdSnTx9FRUXpgQceUO/eva9bhfk1EhUAAKqw/Px8h+3Xz6D7taKiIm3dulUJCQn2fR4eHkpISNCGDRtKbBMXF6etW7faE5MDBw5o+fLl6tq1a6njc8uhHwAAcG02wyKbceNDP5fbRkZGOuxPT0/XuHHjrjj+xIkTslqtCgsLc9gfFhamH374ocQ++vTpoxMnTujuu++WYRgqLi7WkCFD9L//+7+ljpNEBQAAE7I6+fTky21zc3Mdbk8uz0e7ZGVlafLkyXrrrbfUvn177du3T8OHD7evPVYaJCoAAFRhgYGBpVpHJTQ0VJ6enjp69KjD/qNHjyo8PLzENmPGjNETTzyhgQMHSpKaNWumc+fOafDgwXr++efl4XH9RIs5KgAAmNDloR9ntrLw8vJSmzZtlJmZ+UsMNpsyMzMVGxtbYpvz589fkYx4enpKkkq73iwVFQAATMgmD9mcqDfcSNvU1FT169dPbdu2Vbt27TRt2jSdO3dOycnJki49/iYiIkIZGRmSpG7dumnq1Klq1aqVfehnzJgx6tatmz1huR4SFQAAUCq9evXS8ePHNXbsWOXl5ally5ZasWKFfYJtTk6OQwXlhRdekMVi0QsvvKCffvpJtWrVUrdu3TRp0qRS9+mWz/qpqnjWT9XCs36Am09lPuvnyTWPyNuJZ/1cKLiotzt+zLN+AABA+Suv25PdHYkKAAAmZBgesjnxYEGDhxICAAA4h4oKAAAmZJVFViceSuhM28pEogIAgAnZDOfmmdhMcisNQz8AAMBtUVEBAMCEbE5OpnWmbWUiUQEAwIRsssjmxDwTZ9pWJnOkUwAAoEqiogIAgAlZDYusTkymdaZtZSJRAQDAhKrKHBVzRAkAAKokKioAAJiQTU4+68ckk2lJVAAAMCHDybt+DBIVAABQUarK05OZowIAANwWFRUAAEyoqtz1Q6ICAIAJMfQDAADgYlRUAAAwoaryrB8SFQAATIihHwAAABejogIAgAlVlYoKiQoAACZUVRIVhn4AAIDboqICAIAJVZWKCokKAAAmZMi5W4yN8gulQpGoAABgQlWlosIcFQAA4LaoqAAAYEJVpaJCogIAgAlVlUSFoR8AAOC2qKgAAGBCVaWiQqICAIAJGYZFhhPJhjNtKxNDPwAAwG1RUQEAwIRssji14JszbSsTiQoAACZUVeaoMPQDAADcFhUVAABMqKpMpiVRAQDAhKrK0A+JCgAAJlRVKirMUQEAAG6LigoAACZkODn0Y5aKCokKAAAmZEgyDOfamwFDPwAAwG1RUQEAwIRsssjCyrQAAMAdcdcPAACAi1FRAQDAhGyGRRYWfAMAAO7IMJy868ckt/0w9AMAANwWFRUAAEyoqkymJVEBAMCESFQAAIDbqiqTaZmjAgAA3BYVFQAATKiq3PVDogIAgAldSlScmaNSjsFUIIZ+AACA26KiAgCACXHXDwAAcFvGfzdn2psBQz8AAMBtUVEBAMCEGPoBAADuq4qM/TD0AwCAGf23onKjm26wovLmm28qKipKPj4+at++vTZt2nTN40+fPq2hQ4eqTp068vb2VpMmTbR8+fJS90dFBQAAlMqiRYuUmpqq6dOnq3379po2bZoSExOVnZ2t2rVrX3F8UVGROnfurNq1a+ujjz5SRESEfvzxRwUHB5e6TxIVAABMyBUr006dOlWDBg1ScnKyJGn69OlatmyZZs2apVGjRl1x/KxZs3Ty5EmtX79e1atXlyRFRUWVqU+GfgAAMCFnhn1+PRE3Pz/fYbtw4UKJ/RUVFWnr1q1KSEiw7/Pw8FBCQoI2bNhQYpulS5cqNjZWQ4cOVVhYmO68805NnjxZVqu11J+TRAUAgCosMjJSQUFB9i0jI6PE406cOCGr1aqwsDCH/WFhYcrLyyuxzYEDB/TRRx/JarVq+fLlGjNmjKZMmaKJEyeWOj6GfgAAMCMnJsTa20vKzc1VYGCgfbe3t7ezkdnZbDbVrl1b7777rjw9PdWmTRv99NNP+vvf/6709PRSnYNEBQAAEyqvOSqBgYEOicrVhIaGytPTU0ePHnXYf/ToUYWHh5fYpk6dOqpevbo8PT3t+6Kjo5WXl6eioiJ5eXldt1+GfgAAwHV5eXmpTZs2yszMtO+z2WzKzMxUbGxsiW06dOigffv2yWaz2fft2bNHderUKVWSIpGoAABgTkY5bGWUmpqqGTNmaO7cudq9e7eefPJJnTt3zn4XUN++fTV69Gj78U8++aROnjyp4cOHa8+ePVq2bJkmT56soUOHlrrPUg39LF26tNQnfOihh0p9LAAAuDGuWEK/V69eOn78uMaOHau8vDy1bNlSK1assE+wzcnJkYfHLzWQyMhIffnllxoxYoSaN2+uiIgIDR8+XCNHjix1nxbDuP4I1687vebJLJYy3XIER/n5+QoKCtKpPQ0VGECx62aXWLelq0MAUM6KjYvK0qc6c+ZMqeZ93IjLfyvqvTtWHn4+N3we2/lC5Qx+sUJjLQ+lqqj8emwJAAC4CZM8r8cZTt31U1hYKB+fG8/mAADAjakqT08u8/iC1WrVhAkTFBERIX9/fx04cECSNGbMGL333nvlHiAAACiBCybTukKZE5VJkyZpzpw5eumllxxuLbrzzjs1c+bMcg0OAABUbWVOVObNm6d3331XSUlJDgu4tGjRQj/88EO5BgcAAK7GUg6b+yvzHJWffvpJjRo1umK/zWbTxYsXyyUoAABwHc4O39ysQz8xMTFas2bNFfs/+ugjtWrVqlyCAgAAkG6gojJ27Fj169dPP/30k2w2mz7++GNlZ2dr3rx5+vzzzysiRgAA8FtUVErWvXt3ffbZZ/rqq69Uo0YNjR07Vrt379Znn32mzp07V0SMAADgty4/PdmZzQRuaB2Vjh07atWqVeUdCwAAgIMbXvBty5Yt2r17t6RL81batGlTbkEBAIBrM4xLmzPtzaDMicq///1v9e7dW+vWrVNwcLAk6fTp04qLi9PChQt16623lneMAADgt5ijUrKBAwfq4sWL2r17t06ePKmTJ09q9+7dstlsGjhwYEXECAAAqqgyV1RWr16t9evXq2nTpvZ9TZs21euvv66OHTuWa3AAAOAqnJ0Qe7NOpo2MjCxxYTer1aq6deuWS1AAAODaLMalzZn2ZlDmoZ+///3vevrpp7Vlyxb7vi1btmj48OF6+eWXyzU4AABwFVXkoYSlqqiEhITIYvmlRHTu3Dm1b99e1apdal5cXKxq1arpz3/+sx5++OEKCRQAAFQ9pUpUpk2bVsFhAACAMmGOyi/69etX0XEAAICyqCK3J9/wgm+SVFhYqKKiIod9gYGBTgUEAABwWZkn0547d04pKSmqXbu2atSooZCQEIcNAABUgioymbbMicpzzz2nf/zjH3r77bfl7e2tmTNnavz48apbt67mzZtXETECAIDfqiKJSpmHfj777DPNmzdPnTp1UnJysjp27KhGjRqpfv36mj9/vpKSkioiTgAAUAWVuaJy8uRJNWzYUNKl+SgnT56UJN1999365ptvyjc6AABQsst3/TizmUCZE5WGDRvq4MGDkqTbb79dH374oaRLlZbLDykESuP7b2tobN8G6t3qDiXWban1XwS5OiRUoG79T2juxl367MBOvfr5XjVted7VIaECcb0r3uWVaZ3ZzKDMiUpycrJ27NghSRo1apTefPNN+fj4aMSIEXr22WfLPUAzi4qKYg2aayg876GGd/yslMn/dnUoqGDxD53S4PTDmj81XEMTm+jALh9NWnBAQbdc+TgOmB/XG+WpzHNURowYYf/vhIQE/fDDD9q6dasaNWqk5s2bl2twuLnddd9Z3XXfWVeHgUrwyOATWrGgplYuqilJem3krWp3f74Se5/Uh2+EuTg6lDeudyVhHZXSqV+/vurXr18esVS6oqIieXl5uToM4KZWrbpNjZuf18I3atv3GYZF360JUEwbhgNuNlxvlLdSJSqvvfZaqU84bNiwGw7mejp16qTmzZvLx8dHM2fOlJeXl4YMGaJx48ZJknJycvT0008rMzNTHh4e6tKli15//XWFhV3K4MeNG6clS5YoJSVFkyZN0o8//iibzSaLxaLp06frs88+0z/+8Q/Vr19fs2bNUq1atTRw4EBt3rxZLVq00Pvvv6/bbrtNkrR//36lpqbq22+/1blz5xQdHa2MjAwlJCRU2OcHzCiwplWe1aTTxx1/3Zw6UU2RjS64KCpUFK535bHIyacnl1skFatUicorr7xSqpNZLJYKTVQkae7cuUpNTdXGjRu1YcMG9e/fXx06dND999+v7t27y9/fX6tXr1ZxcbGGDh2qXr16KSsry95+3759Wrx4sT7++GN5enra90+YMEFTp07V1KlTNXLkSPXp00cNGzbU6NGjVa9ePf35z39WSkqKvvjiC0lSQUGBunbtqkmTJsnb21vz5s1Tt27dlJ2drXr16pXqs1y4cEEXLvzyg5ufn18+XxIAADeJUiUql+/ycQfNmzdXenq6JKlx48Z64403lJmZKUn6/vvvdfDgQUVGRkqS5s2bpzvuuEObN2/WXXfdJenScM+8efNUq1Yth/MmJyerZ8+ekqSRI0cqNjZWY8aMUWJioiRp+PDhSk5Oth/fokULtWjRwv56woQJ+uSTT7R06VKlpKSU6rNkZGRo/PjxN/I1AKaRf9JT1mIpuFaxw/6Q0GKdOu706DPcDNe7ElWRhxKW+a4fV/vthN06dero2LFj2r17tyIjI+1JiiTFxMQoODhYu3fvtu+rX7/+FUnKb897eaioWbNmDvsKCwvtVY+CggKlpaUpOjpawcHB8vf31+7du5WTk1PqzzJ69GidOXPGvuXm5pa6LWAWxRc9tHenn1rd/cvEaYvFUMu7C7Rrq58LI0NF4HpXIlamdU/Vq1d3eG2xWGSz2UrdvkaNGtc9r8Viueq+y32lpaVp1apVevnll9WoUSP5+vrq0UcfveIhjdfi7e0tb2/vUh9/s/n5nIcOH/zl8+flemn/P30VEFys2rdyG+PN5ON3Q5U2LVd7dvgp+zs/9Rh0XD5+Nq1cWNPVoaECcL1RnkyXqFxNdHS0cnNzlZuba6+q7Nq1S6dPn1ZMTEy597du3Tr1799fPXr0kHSpwnLo0KFy7+dmtmeHn557tJH99TvjIiRJnXueVNq00lem4P5WLw1R0C1W9X02TyG1inXgX756PqmBTp+ofv3GMB2udyXh9mRzSUhIULNmzZSUlKRp06apuLhYTz31lOLj49W2bdty769x48b6+OOP1a1bN1ksFo0ZM6ZMlR1ILeIK9OXh7a4OA5Vk6exQLZ0d6uowUEm43hXP2dVlb9qVad2VxWLRp59+qpCQEN1zzz1KSEhQw4YNtWjRogrpb+rUqQoJCVFcXJy6deumxMREtW7dukL6AgCgqrIYhlHmnGrNmjV65513tH//fn300UeKiIjQ+++/rwYNGujuu++uiDirhPz8fAUFBenUnoYKDLhpckhcRWLdlq4OAUA5KzYuKkuf6syZMwoMDKyQPi7/rYiaOEkePj43fB5bYaEOvfB8hcZaHsr813Dx4sVKTEyUr6+vvvvuO/s6IGfOnNHkyZPLPUAAAFCCKnLXT5kTlYkTJ2r69OmaMWOGw10xHTp00LZt28o1OAAAULWVeTJtdna27rnnniv2BwUF6fTp0+UREwAAuA4m015FeHi49u3bd8X+tWvXqmHDhuUSFAAAuI7LK9M6s5lAmROVQYMGafjw4dq4caMsFosOHz6s+fPnKy0tTU8++WRFxAgAAH6risxRKfPQz6hRo2Sz2XT//ffr/Pnzuueee+Tt7a20tDQ9/fTTFREjAACoosqcqFgsFj3//PN69tlntW/fPhUUFCgmJkb+/v4VER8AAChBVZmjcsMr03p5eVXI0vQAAKAUWEK/ZPfee6/9AX0l+cc//uFUQAAAAJeVOVFp2bKlw+uLFy9q+/bt+uc//6l+/fqVV1wAAOBanBz6uWkrKq+88kqJ+8eNG6eCggKnAwIAAKVQRYZ+yu2BMo8//rhmzZpVXqcDAAC48cm0v7Vhwwb5OPFwJAAAUAZVpKJS5kTlkUcecXhtGIaOHDmiLVu2aMyYMeUWGAAAuDpuT76KoKAgh9ceHh5q2rSpXnzxRT3wwAPlFhgAAECZEhWr1ark5GQ1a9ZMISEhFRUTAACApDJOpvX09NQDDzzAU5IBAHC1KvKsnzLf9XPnnXfqwIEDFRELAAAopctzVJzZzKDMicrEiROVlpamzz//XEeOHFF+fr7DBgAAUF5KPUflxRdf1P/8z/+oa9eukqSHHnrIYSl9wzBksVhktVrLP0oAAHAlk1RFnFHqRGX8+PEaMmSIvv7664qMBwAAlAbrqDgyjEufKD4+vsKCAQAA+LUy3Z58racmAwCAysOCbyVo0qTJdZOVkydPOhUQAAAoBYZ+rjR+/PgrVqYFAACoKGVKVP70pz+pdu3aFRULAAAopaoy9FPqdVSYnwIAgBtx0cq0b775pqKiouTj46P27dtr06ZNpWq3cOFCWSwWPfzww2Xqr9SJyuW7fgAAQNW0aNEipaamKj09Xdu2bVOLFi2UmJioY8eOXbPdoUOHlJaWpo4dO5a5z1InKjabjWEfAADchQsqKlOnTtWgQYOUnJysmJgYTZ8+XX5+fpo1a9ZV21itViUlJWn8+PFq2LBhmfss8xL6AADA9crrWT+/fRTOhQsXSuyvqKhIW7duVUJCgn2fh4eHEhIStGHDhqvG+eKLL6p27doaMGDADX1OEhUAAMyonCoqkZGRCgoKsm8ZGRkldnfixAlZrVaFhYU57A8LC1NeXl6JbdauXav33ntPM2bMuOGPWaa7fgAAwM0lNzdXgYGB9tfe3t7lct6zZ8/qiSee0IwZMxQaGnrD5yFRAQDAjMppwbfAwECHROVqQkND5enpqaNHjzrsP3r0qMLDw684fv/+/Tp06JC6detm32ez2SRJ1apVU3Z2tm677bbr9svQDwAAJlRec1RKy8vLS23atFFmZqZ9n81mU2ZmpmJjY684/vbbb9f333+v7du327eHHnpI9957r7Zv367IyMhS9UtFBQAAlEpqaqr69euntm3bql27dpo2bZrOnTun5ORkSVLfvn0VERGhjIwM+fj46M4773RoHxwcLElX7L8WEhUAAMzIBc/66dWrl44fP66xY8cqLy9PLVu21IoVK+wTbHNycuThUb6DNSQqAACYkKuW0E9JSVFKSkqJ72VlZV2z7Zw5c8rcH3NUAACA26KiAgCAGblg6McVSFQAADCjKpKoMPQDAADcFhUVAABMyPLfzZn2ZkCiAgCAGVWRoR8SFQAATMhVtydXNuaoAAAAt0VFBQAAM2LoBwAAuDWTJBvOYOgHAAC4LSoqAACYUFWZTEuiAgCAGVWROSoM/QAAALdFRQUAABNi6AcAALgvhn4AAABci4oKAFSCLw9vd3UIqAT5Z20KaVI5fTH0AwAA3FcVGfohUQEAwIyqSKLCHBUAAOC2qKgAAGBCzFEBAADui6EfAAAA16KiAgCACVkMQxbjxssizrStTCQqAACYEUM/AAAArkVFBQAAE+KuHwAA4L4Y+gEAAHAtKioAAJgQQz8AAMB9VZGhHxIVAABMqKpUVJijAgAA3BYVFQAAzIihHwAA4M7MMnzjDIZ+AACA26KiAgCAGRnGpc2Z9iZAogIAgAlx1w8AAICLUVEBAMCMuOsHAAC4K4vt0uZMezNg6AcAALgtKioAAJgRQz8AAMBdVZW7fkhUAAAwoyqyjgpzVAAAgNuiogIAgAkx9AMAANxXFZlMy9APAABwW1RUAAAwIYZ+AACA++KuHwAAANeiogIAgAkx9AMAANwXd/0AAAC4FhUVAABMiKEfAADgvmzGpc2Z9iZAogIAgBkxRwUAAMC1qKgAAGBCFjk5R6XcIqlYJCoAAJgRK9MCAAC4FokKAAAmdPn2ZGe2G/Hmm28qKipKPj4+at++vTZt2nTVY2fMmKGOHTsqJCREISEhSkhIuObxJSFRAQDAjIxy2Mpo0aJFSk1NVXp6urZt26YWLVooMTFRx44dK/H4rKws9e7dW19//bU2bNigyMhIPfDAA/rpp59K3SeJCgAAKJWpU6dq0KBBSk5OVkxMjKZPny4/Pz/NmjWrxOPnz5+vp556Si1bttTtt9+umTNnymazKTMzs9R9kqgAAGBCFsNwepOk/Px8h+3ChQsl9ldUVKStW7cqISHBvs/Dw0MJCQnasGFDqWI+f/68Ll68qJo1a5b6c5KoAABgRrZy2CRFRkYqKCjIvmVkZJTY3YkTJ2S1WhUWFuawPywsTHl5eaUKeeTIkapbt65DsnM93J4MAEAVlpubq8DAQPtrb2/vCunnr3/9qxYuXKisrCz5+PiUuh2JCgAAJvTr4ZsbbS9JgYGBDonK1YSGhsrT01NHjx512H/06FGFh4dfs+3LL7+sv/71r/rqq6/UvHnzMsXJ0A8AAGZUyXf9eHl5qU2bNg4TYS9PjI2Njb1qu5deekkTJkzQihUr1LZt27J1KioqAACYkwtWpk1NTVW/fv3Utm1btWvXTtOmTdO5c+eUnJwsSerbt68iIiLs81z+9re/aezYsVqwYIGioqLsc1n8/f3l7+9fqj5JVAAAQKn06tVLx48f19ixY5WXl6eWLVtqxYoV9gm2OTk58vD4ZbDm7bffVlFRkR599FGH86Snp2vcuHGl6pNEBQAAE3JmddnL7W9ESkqKUlJSSnwvKyvL4fWhQ4durJNfYY7KVYwbN04tW7Z0dRg3te+/raGxfRuod6s7lFi3pdZ/EeTqkFCBuvU/obkbd+mzAzv16ud71bTleVeHhArAz3Ulujz048xmAiQqkiwWi5YsWeKwLy0trUwr56HsCs97qOEdPytl8r9dHQoqWPxDpzQ4/bDmTw3X0MQmOrDLR5MWHFDQLRddHRrKGT/XKG8M/VxFWSb64Mbcdd9Z3XXfWVeHgUrwyOATWrGgplYuurQa5Wsjb1W7+/OV2PukPnwj7DqtYSb8XFcei+3S5kx7M3BpRaVTp04aNmyYnnvuOdWsWVPh4eEOk2tOnz6tgQMHqlatWgoMDNR9992nHTt2OJxj4sSJql27tgICAjRw4ECNGjXKYchm8+bN6ty5s0JDQxUUFKT4+Hht27bN/n5UVJQkqUePHrJYLPbXvx76WblypXx8fHT69GmHvocPH6777rvP/nrt2rXq2LGjfH19FRkZqWHDhuncuXNOf0+AmVWrblPj5ue1bU2AfZ9hWPTdmgDFtGH4B7hhDP1Ujrlz56pGjRrauHGjXnrpJb344otatWqVJOmxxx7TsWPH9MUXX2jr1q1q3bq17r//fp08eVLSpYcdTZo0SX/729+0detW1atXT2+//bbD+c+ePat+/fpp7dq1+vbbb9W4cWN17dpVZ89eyvg3b94sSZo9e7aOHDlif/1r999/v4KDg7V48WL7PqvVqkWLFikpKUmStH//fnXp0kV//OMftXPnTi1atEhr16696oQjSbpw4cIVz1gAbjaBNa3yrCadPu5YwD11oppCahW7KCoAZuHyoZ/mzZsrPT1dktS4cWO98cYbyszMlK+vrzZt2qRjx47Zl/N9+eWXtWTJEn300UcaPHiwXn/9dQ0YMMB+//bYsWO1cuVKFRQU2M//64qHJL377rsKDg7W6tWr9Yc//EG1atWSJAUHB191ZT1PT0/96U9/0oIFCzRgwABJUmZmpk6fPq0//vGPkqSMjAwlJSXpmWeesX+W1157TfHx8Xr77bdLXC44IyND48ePv9GvDgBQld3Aom1XtDcBl1dUfruUbp06dXTs2DHt2LFDBQUFuuWWW+zzRfz9/XXw4EHt379fkpSdna127do5tP/t66NHj2rQoEFq3LixgoKCFBgYqIKCAuXk5JQpzqSkJGVlZenw4cOSLlVzHnzwQQUHB0uSduzYoTlz5jjEmpiYKJvNpoMHD5Z4ztGjR+vMmTP2LTc3t0wxAWaQf9JT1mIp+DfVk5DQYp067vJ/KwGmVV5PT3Z3Lv8tUb16dYfXFotFNptNBQUFqlOnzhX3ZEuyJwel0a9fP/3nP//Rq6++qvr168vb21uxsbEqKioqU5x33XWXbrvtNi1cuFBPPvmkPvnkE82ZM8f+fkFBgf7yl79o2LBhV7StV69eief09vausIc/Ae6i+KKH9u70U6u7z2rDiku3qloshlreXaClc25xcXQA3J3LE5Wrad26tfLy8lStWjX7BNffatq0qTZv3qy+ffva9/12jsm6dev01ltvqWvXrpIuPSXyxIkTDsdUr15dVqv1ujElJSVp/vz5uvXWW+Xh4aEHH3zQId5du3apUaNGpf2IVd7P5zx0+OAviVperpf2/9NXAcHFqn0rt63eTD5+N1Rp03K1Z4efsr/zU49Bx+XjZ9PKhTVdHRrKGT/XlcgFS+i7gtsmKgkJCYqNjdXDDz+sl156SU2aNNHhw4e1bNky9ejRQ23bttXTTz+tQYMGqW3btoqLi9OiRYu0c+dONWzY0H6exo0b6/3331fbtm2Vn5+vZ599Vr6+vg59RUVFKTMzUx06dJC3t7dCQkJKjCkpKUnjxo3TpEmT9OijjzpUQ0aOHKnf/e53SklJ0cCBA1WjRg3t2rVLq1at0htvvFExX5LJ7dnhp+ce/SWxe2dchCSpc8+TSptWtqE5uLfVS0MUdItVfZ/NU0itYh34l6+eT2qg0yeqX78xTIWf60pkSHLmFmNz5Cnum6hYLBYtX75czz//vJKTk3X8+HGFh4frnnvusT9TICkpSQcOHFBaWpoKCwvVs2dP9e/fX5s2bbKf57333tPgwYPVunVrRUZGavLkyUpLS3Poa8qUKUpNTdWMGTMUERFx1SV/GzVqpHbt2mnTpk2aNm2aw3vNmzfX6tWr9fzzz6tjx44yDEO33XabevXqVa7fy82kRVyBvjy83dVhoJIsnR2qpbNDXR0GKhg/15XH2XkmZpmjYjEMk0RaSp07d1Z4eLjef/99V4dSZvn5+QoKCtKpPQ0VGODyec6oYIl1W7o6BFQi/nhXDflnbQppckBnzpxRYGBgxfTx378V97UapWqeV95RWlrF1kL947u/Vmis5cFtKyqlcf78eU2fPl2JiYny9PTUBx98oK+++sq+DgsAADctQ07OUSm3SCqUqROVy8NDkyZNUmFhoZo2barFixcrISHB1aEBAFCxmEzr/nx9ffXVV1+5OgwAAFBBTJ2oAABQZdkkWZxsbwIkKgAAmFBVueuHW0sAAIDboqICAIAZMZkWAAC4rSqSqDD0AwAA3BYVFQAAzKiKVFRIVAAAMCNuTwYAAO6K25MBAABcjIoKAABmxBwVAADgtmyGZHEi2bCZI1Fh6AcAALgtKioAAJgRQz8AAMB9OZmoyByJCkM/AADAbVFRAQDAjBj6AQAAbstmyKnhG+76AQAAcA4VFQAAzMiwXdqcaW8CJCoAAJgRc1QAAIDbYo4KAACAa1FRAQDAjBj6AQAAbsuQk4lKuUVSoRj6AQAAbouKCgAAZsTQDwAAcFs2myQn1kKxmWMdFYZ+AACA26KiAgCAGTH0AwAA3FYVSVQY+gEAAG6LigoAAGZURZbQJ1EBAMCEDMMmw4knIDvTtjKRqAAAYEaG4VxVhDkqAAAAzqGiAgCAGRlOzlExSUWFRAUAADOy2SSLE/NMTDJHhaEfAADgtqioAABgRgz9AAAAd2XYbDKcGPoxy+3JDP0AAAC3RUUFAAAzYugHAAC4LZshWW7+RIWhHwAA4LaoqAAAYEaGIcmZdVTMUVEhUQEAwIQMmyHDiaEfg0QFAABUGMMm5yoq3J4MAABuMm+++aaioqLk4+Oj9u3ba9OmTdc8/v/+7/90++23y8fHR82aNdPy5cvL1B+JCgAAJmTYDKe3slq0aJFSU1OVnp6ubdu2qUWLFkpMTNSxY8dKPH79+vXq3bu3BgwYoO+++04PP/ywHn74Yf3zn/8sdZ8kKgAAmJFhc34ro6lTp2rQoEFKTk5WTEyMpk+fLj8/P82aNavE41999VV16dJFzz77rKKjozVhwgS1bt1ab7zxRqn7ZI6KG7k8sSm/wBzjhnBOsXHR1SGgEuWf5ee6Krj8+7syJqoW66JT670V69LvoPz8fIf93t7e8vb2vuL4oqIibd26VaNHj7bv8/DwUEJCgjZs2FBiHxs2bFBqaqrDvsTERC1ZsqTUcZKouJGzZ89Kkuq3PuTaQFBJDrg6AFSikCaujgCV6ezZswoKCqqQc3t5eSk8PFxr88o216Mk/v7+ioyMdNiXnp6ucePGXXHsiRMnZLVaFRYW5rA/LCxMP/zwQ4nnz8vLK/H4vLy8UsdIouJG6tatq9zcXAUEBMhisbg6nEqTn5+vyMhI5ebmKjAw0NXhoAJxrauOqnqtDcPQ2bNnVbdu3Qrrw8fHRwcPHlRRUZHT5zIM44q/NyVVU1yJRMWNeHh46NZbb3V1GC4TGBhYpX6hVWVc66qjKl7riqqk/JqPj498fHwqvJ9fCw0Nlaenp44ePeqw/+jRowoPDy+xTXh4eJmOLwmTaQEAwHV5eXmpTZs2yszMtO+z2WzKzMxUbGxsiW1iY2MdjpekVatWXfX4klBRAQAApZKamqp+/fqpbdu2ateunaZNm6Zz584pOTlZktS3b19FREQoIyNDkjR8+HDFx8drypQpevDBB7Vw4UJt2bJF7777bqn7JFGBy3l7eys9Pd3txkVR/rjWVQfX+ubUq1cvHT9+XGPHjlVeXp5atmypFStW2CfM5uTkyMPjl8GauLg4LViwQC+88IL+93//V40bN9aSJUt05513lrpPi2GWxf4BAECVwxwVAADgtkhUAACA2yJRAQAAbotEBS6TlZUli8Wi06dPX/O4qKgoTZs2rVJigjnx/4i5jRs3Ti1btnR1GHBTJCpwmbi4OB05csS+ONKcOXMUHBx8xXGbN2/W4MGDKzk6ABXBYrFc8ZyXtLS0K9baAC7j9mS4zOXnVVxPrVq1KiEaVKSioiJ5eXm5Ogy4KX9/f/n7+7s6DLgpKiq4pk6dOiklJUUpKSkKCgpSaGioxowZY38y6KlTp9S3b1+FhITIz89Pv//977V37157+x9//FHdunVTSEiIatSooTvuuEPLl196kNavh36ysrKUnJysM2fOyGKxyGKx2B+K9euyfp8+fdSrVy+HGC9evKjQ0FDNmzdP0qWVEjMyMtSgQQP5+vqqRYsW+uijjyr4m7q5dOrUScOGDdNzzz2nmjVrKjw83OEhZTk5Oerevbv8/f0VGBionj17OiyTfbmUP3PmTDVo0MC+1LfFYtE777yjP/zhD/Lz81N0dLQ2bNigffv2qVOnTqpRo4bi4uK0f/9++7n279+v7t27KywsTP7+/rrrrrv01VdfVdp3cbO43jU9ffq0Bg4cqFq1aikwMFD33XefduzY4XCOiRMnqnbt2goICNDAgQM1atQohyGbzZs3q3PnzgoNDVVQUJDi4+O1bds2+/tRUVGSpB49eshisdhf/3roZ+XKlfLx8bliSHj48OG677777K/Xrl2rjh07ytfXV5GRkRo2bJjOnTvn9PcE90OiguuaO3euqlWrpk2bNunVV1/V1KlTNXPmTElS//79tWXLFi1dulQbNmyQYRjq2rWrLl689PjwoUOH6sKFC/rmm2/0/fff629/+1uJ/3KKi4vTtGnTFBgYqCNHjujIkSNKS0u74rikpCR99tlnKigosO/78ssvdf78efXo0UOSlJGRoXnz5mn69On617/+pREjRujxxx/X6tWrK+LruWnNnTtXNWrU0MaNG/XSSy/pxRdf1KpVq2Sz2dS9e3edPHlSq1ev1qpVq3TgwIErEsh9+/Zp8eLF+vjjj7V9+3b7/gkTJqhv377avn27br/9dvXp00d/+ctfNHr0aG3ZskWGYSglJcV+fEFBgbp27arMzEx999136tKli7p166acnJzK+ipuGle7ppL02GOP6dixY/riiy+0detWtW7dWvfff79OnjwpSZo/f74mTZqkv/3tb9q6davq1aunt99+2+H8Z8+eVb9+/bR27Vp9++23aty4sbp27Wp/MvzmzZslSbNnz9aRI0fsr3/t/vvvV3BwsBYvXmzfZ7VatWjRIiUlJUm6lLx26dJFf/zjH7Vz504tWrRIa9eudfj/BjcRA7iG+Ph4Izo62rDZbPZ9I0eONKKjo409e/YYkox169bZ3ztx4oTh6+trfPjhh4ZhGEazZs2McePGlXjur7/+2pBknDp1yjAMw5g9e7YRFBR0xXH169c3XnnlFcMwDOPixYtGaGioMW/ePPv7vXv3Nnr16mUYhmEUFhYafn5+xvr16x3OMWDAAKN3795l/vxVVXx8vHH33Xc77LvrrruMkSNHGitXrjQ8PT2NnJwc+3v/+te/DEnGpk2bDMMwjPT0dKN69erGsWPHHM4hyXjhhRfsrzds2GBIMt577z37vg8++MDw8fG5Znx33HGH8frrr9tf//r/EZTsWtd0zZo1RmBgoFFYWOjw/m233Wa88847hmEYRvv27Y2hQ4c6vN+hQwejRYsWV+3TarUaAQEBxmeffWbfJ8n45JNPHI5LT093OM/w4cON++67z/76yy+/NLy9ve2/KwYMGGAMHjzY4Rxr1qwxPDw8jJ9//vmq8cCcqKjgun73u985PAY8NjZWe/fu1a5du1StWjW1b9/e/t4tt9yipk2bavfu3ZKkYcOGaeLEierQoYPS09O1c+dOp2KpVq2aevbsqfnz50uSzp07p08//dT+L619+/bp/Pnz6ty5s33c29/fX/PmzXMYTsD1NW/e3OF1nTp1dOzYMe3evVuRkZGKjIy0vxcTE6Pg4GD7dZek+vXrlzi/6NfnvbzsdrNmzRz2FRYWKj8/X9KlikpaWpqio6MVHBwsf39/7d69m4rKDbjaNd2xY4cKCgp0yy23OPzcHDx40P5zk52drXbt2jm0/+3ro0ePatCgQWrcuLGCgoIUGBiogoKCMl+rpKQkZWVl6fDhw5IuVXMefPBB+2T7HTt2aM6cOQ6xJiYmymaz6eDBg2XqC+6PybSoUAMHDlRiYqKWLVumlStXKiMjQ1OmTNHTTz99w+dMSkpSfHy8jh07plWrVsnX11ddunSRJPuQ0LJlyxQREeHQjmeOlE316tUdXlssFtlstlK3r1GjxnXPezkBLmnf5b7S0tK0atUqvfzyy2rUqJF8fX316KOPqqioqNSx4JKrXdOCggLVqVNHWVlZV7Qp6U68q+nXr5/+85//6NVXX1X9+vXl7e2t2NjYMl+ru+66S7fddpsWLlyoJ598Up988onmzJljf7+goEB/+ctfNGzYsCva1qtXr0x9wf2RqOC6Nm7c6PD68thzTEyMiouLtXHjRsXFxUmS/vOf/yg7O1sxMTH24yMjIzVkyBANGTJEo0eP1owZM0pMVLy8vGS1Wq8bT1xcnCIjI7Vo0SJ98cUXeuyxx+y/gGNiYuTt7a2cnBzFx8c787FxFdHR0crNzVVubq69qrJr1y6dPn3a4bqXl3Xr1ql///72OUgFBQU6dOhQufdTlbVu3Vp5eXmqVq2afYLrbzVt2lSbN29W37597ft+O8dk3bp1euutt9S1a1dJUm5urk6cOOFwTPXq1Uv1c56UlKT58+fr1ltvlYeHhx588EGHeHft2qVGjRqV9iPCxBj6wXXl5OQoNTVV2dnZ+uCDD/T6669r+PDhaty4sbp3765BgwZp7dq12rFjhx5//HFFRESoe/fukqRnnnlGX375pQ4ePKht27bp66+/VnR0dIn9REVFqaCgQJmZmTpx4oTOnz9/1Zj69Omj6dOna9WqVfZhH0kKCAhQWlqaRowYoblz52r//v3atm2bXn/9dc2dO7d8v5gqKiEhQc2aNVNSUpK2bdumTZs2qW/fvoqPj1fbtm3Lvb/GjRvbJ+Tu2LFDffr0KVNlB9eXkJCg2NhYPfzww1q5cqUOHTqk9evX6/nnn9eWLVskSU8//bTee+89zZ07V3v37tXEiRO1c+dOh2Hhxo0b6/3339fu3bu1ceNGJSUlydfX16GvqKgoZWZmKi8vT6dOnbpqTJf//5o0aZIeffRRh4royJEjtX79eqWkpGj79u3au3evPv30UybT3qRIVHBdffv21c8//6x27dpp6NChGj58uH0BttmzZ6tNmzb6wx/+oNjYWBmGoeXLl9srHFarVUOHDlV0dLS6dOmiJk2a6K233iqxn7i4OA0ZMkS9evVSrVq19NJLL101pqSkJO3atUsRERHq0KGDw3sTJkzQmDFjlJGRYe932bJlatCgQTl9I1WbxWLRp59+qpCQEN1zzz1KSEhQw4YNtWjRogrpb+rUqQoJCVFcXJy6deumxMREtW7dukL6qqosFouWL1+ue+65R8nJyWrSpIn+9Kc/6ccff7TPI0pKStLo0aOVlpam1q1b6+DBg+rfv7/91nNJeu+993Tq1Cm1bt1aTzzxhIYNG6batWs79DVlyhStWrVKkZGRatWq1VVjatSokdq1a6edO3c6/GNEujTXZvXq1dqzZ486duyoVq1aaezYsapbt245fitwFxbD+O+CGEAJOnXqpJYtW7I8OYArdO7cWeHh4Xr//fddHQpuYsxRAQBc1/nz5zV9+nQlJibK09NTH3zwgb766iv7OixARSFRAQBc1+XhoUmTJqmwsFBNmzbV4sWLlZCQ4OrQcJNj6AcAALgtJtMCAAC3RaICAADcFokKAABwWyQqAADAbZGoAHDQv39/Pfzww/bXnTp10jPPPFPpcWRlZclisej06dNXPcZisWjJkiWlPue4cePUsmVLp+I6dOiQLBaLtm/f7tR5AJQOiQpgAv3795fFYpHFYpGXl5caNWqkF198UcXFxRXe98cff6wJEyaU6tjSJBcAUBasowKYRJcuXTR79mxduHBBy5cv19ChQ1W9enWNHj36imOLiork5eVVLv3WrFmzXM4DADeCigpgEt7e3goPD1f9+vX15JNPKiEhQUuXLpX0y3DNpEmTVLduXTVt2lTSpafX9uzZU8HBwapZs6a6d+/u8ORhq9Wq1NRUBQcH65ZbbtFzzz2n3y6t9NuhnwsXLmjkyJGKjIyUt7e3GjVqpPfee0+HDh3SvffeK0kKCQmRxWJR//79JUk2m00ZGRlq0KCBfH191aJFC3300UcO/SxfvlxNmjSRr6+v7r333ht6QvLIkSPVpEkT+fn5qWHDhhozZowuXrx4xXHvvPOOIiMj5efnp549e+rMmTMO78+cOVPR0dHy8fHR7bffftXnUwGoeCQqgEn5+vqqqKjI/jozM1PZ2dlatWqVPv/8c128eFGJiYkKCAjQmjVrtG7dOvn7+6tLly72dlOmTNGcOXM0a9YsrV27VidPntQnn3xyzX779u2rDz74QK+99pp2796td955R/7+/oqMjNTixYslSdnZ2Tpy5IheffVVSVJGRobmzZun6dOn61//+pdGjBihxx9/XKtXr5Z0KaF65JFH1K1bN23fvl0DBw7UqFGjyvydBAQEaM6cOdq1a5deffVVzZgxQ6+88orDMfv27dOHH36ozz77TCtWrNB3332np556yv7+/PnzNXbsWE2aNEm7d+/W5MmTNWbMGJ6+DbiKAcDt9evXz+jevbthGIZhs9mMVatWGd7e3kZaWpr9/bCwMOPChQv2Nu+//77RtGlTw2az2fdduHDB8PX1Nb788kvDMAyjTp06xksvvWR//+LFi8att95q78swDCM+Pt4YPny4YRiGkZ2dbUgyVq1aVWKcX3/9tSHJOHXqlH1fYWGh4efnZ6xfv97h2AEDBhi9e/c2DMMwRo8ebcTExDi8P3LkyCvO9VuSjE8++eSq7//973832rRpY3+dnp5ueHp6Gv/+97/t+7744gvDw8PDOHLkiGEYhnHbbbcZCxYscDjPhAkTjNjYWMMwDOPgwYOGJOO77767ar8Ayg9zVACT+Pzzz+Xv76+LFy/KZrOpT58+GjdunP39Zs2aOcxL2bFjh/bt26eAgACH8xQWFmr//v06c+aMjhw5ovbt29vfq1atmtq2bXvF8M9l27dvl6enp+Lj40sd9759+3T+/Hl17tzZYX9RUZFatWolSdq9e7dDHJIUGxtb6j4uW7RokV577TXt379fBQUFKi4uVmBgoMMx9erVU0REhEM/NptN2dnZCggI0P79+zVgwAANGjTIfkxxcbGCgoLKHA8A55GoACZx77336u2335aXl5fq1q2ratUcf3xr1Kjh8LqgoEBt2rTR/PnzrzhXrVq1bigGX1/fMrcpKCiQJC1btswhQZAuzbspLxs2bFBSUpLGjx+vxMREBQUFaeHChZoyZUqZY50xY8YViZOnp2e5xQqg9EhUAJOoUaOGGjVqVOrjW7durUWLFql27dpXVBUuq1OnjjZu3Kh77rlH0qXKwdatW9W6desSj2/WrJlsNptWr15d4lNzL1d0rFarfV9MTIy8vb2Vk5Nz1UpMdHS0fWLwZd9+++31P+SvrF+/XvXr19fzzz9v3/fjjz9ecVxOTo4OHz6sunXr2vvx8PBQ06ZNFRYWprp16+rAgQNKSkoqU/8AKgaTaYGbVFJSkkJDQ9W9e3etWbNGBw8eVFZWloYNG6Z///vfkqThw4frr3/9q5YsWaIffvhBTz311DXXQImKilK/fv305z//WUuWLLGf88MPP5Qk1a9fXxaLRZ9//rmOHz+ugoICBQQEKC0tTSNGjNDcuXO1f/9+bdu2Ta+//rp9guqQIUO0d+9ePfvss8rOztaCBQs0Z86cMn3exo0bKycnRwsXLtT+/fv12muvlTgx2MfHR/369dOOHTu0Zs0aDRs2TD179lR4eLgkafz48crIyNBrr72mPXv26Pvvv9fs2bM1derUMsUDoHyQqAA3KT8/P33zzTeqV6+eHnnkEUVHR2vAgAEqLCy0V1j+53/+R0888YT69eun2NhYBQQEqEePHtc879tvv61HH31UTz31lG6//XYNGjRI586dkyRFRERo/PjxGjVqlMLCwpSSkiJJmjBhgsaMGaOMjAxFR0erS5cuWrZsmRo0aCDp0ryRxYsXa8mSJWrRooWmT5+uyZMnl+nzPvTQQxoxYoRSUlLUsmVLrV+/XmPGjLniuEaNGumRRx5R165d9cADD6h58+YOtx8PHDhQM2fO1OzZs9WsWTPFx8drzpw59lgBVC6LcbVZcwAAAC5GRQUAALgtEhUAAOC2SFQAAIDbIlEBAABui0QFAAC4LRIVAADgtkhUAACA2yJRAQAAbotEBQAAuC0SFQAA4LZIVAAAgNsiUQEAAG7r/wM6VJB6uX2WuwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# ConfusionMatrixDisplay出力\n",
    "# 事前に近藤行列を作成。出力時に近藤行列と表示用ラベルを与えて実行。\n",
    "cm = confusion_matrix(y_true, y_predicted)\n",
    "disp = ConfusionMatrixDisplay(confusion_matrix=cm, display_labels=labels)\n",
    "disp.plot()\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "dm",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}