{"cells":[{"cell_type":"code","execution_count":2,"id":"a863de04","metadata":{"id":"a863de04","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1747889791150,"user_tz":-540,"elapsed":224,"user":{"displayName":"TOMA Naruaki","userId":"11747312442870110137"}},"outputId":"17a87388-1bb8-4b86-a537-0e58db449125"},"outputs":[{"output_type":"stream","name":"stdout","text":["Python 3.11.12\n","Thu May 22 04:56:30 AM UTC 2025\n"]}],"source":["!python --version\n","!date"]},{"cell_type":"markdown","id":"71c33fbe","metadata":{"id":"71c33fbe"},"source":["# JGLUE（日本語言語理解ベンチマーク） + BoW\n","## JGLUEとは\n","2023年に構築された[日本語言語理解ベンチマーク](https://doi.org/10.5715/jnlp.30.63)の一つ。複数のタスクがある中から今回は JNLI (NLI = Natural Language Inference, 自然言語推論) を例に、事前学習済みBERTで分類学習するコード例を示す。\n","\n","NLIとは、前提文と仮説文の文ペアが与えられたときに、前提文（premise sentence）が仮説文（hypothesis sentence）に対して持つ推論関係を「含意（entailment）」「矛盾（contradiction）」「中立（neutral）」の3値で識別するタスクである。\n"]},{"cell_type":"markdown","id":"ef5bab7c","metadata":{"id":"ef5bab7c"},"source":["## 前提\n","[preprocessing.ipynb](./preprocessing.ipynb)を実行すると前処理された3つのpklファイル（preprocessed_{train,val,test}.pkl）が作成される。この3つのファイルをPCにダウンロードしておき、本コードを実行する環境で参照できるようにアップロードすること。"]},{"cell_type":"markdown","id":"6fabf61f","metadata":{"id":"6fabf61f"},"source":["## データ準備\n","本データセットは [datsets.load_dataset()](https://huggingface.co/docs/datasets/loading) 経由でダウンロードできる。ダウンロードした後に以下の前処理を行い、学習データ、検証用データ、テストデータに整理し直している。実装に興味のある人は `report4_preprocessing.ipynb` と `utils.py` を参照しよう。\n","- get_dataで取得できるデータ説明\n","    - `sent1`: 前提文。\n","    - `sent2`: 仮説文。\n","    - `bow`: Bag-of-Words用に分かち書きした文。\n","    - `w2v`: word2vec用の分散ベクトル。\n","    - `label`: 教師ラベル。\n","        - \"entailment\" (含意): 0\n","        - \"contradiction\" (矛盾): 1\n","        - \"neutral\" (中立): 2\n","- ラベルの付け直し\n","    - 元データでは「含意（entailment）」「矛盾（contradiction）」「中立（neutral）」。これを 0, 1, 2 と付け直した。\n","- データセットの分割\n","    - 提供先で train（学習用）, validation（検証用）が用意されており、最終的な評価用のtestは非公開となっている。ここでは便宜上validationを評価用に用いることとする。"]},{"cell_type":"code","execution_count":3,"id":"bd24e08f","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":459},"id":"bd24e08f","executionInfo":{"status":"ok","timestamp":1747889793484,"user_tz":-540,"elapsed":2330,"user":{"displayName":"TOMA Naruaki","userId":"11747312442870110137"}},"outputId":"9b2f1edf-1e86-4adb-99ef-cb63f20b4706"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["                    sent1                   sent2  \\\n","0  二人の男性がジャンボジェット機を見ています。    2人の男性が、白い飛行機を眺めています。   \n","1    2人の男性が、白い飛行機を眺めています。  二人の男性がジャンボジェット機を見ています。   \n","2      男性が子供を抱き上げて立っています。    坊主頭の男性が子供を抱いて立っています。   \n","3    坊主頭の男性が子供を抱いて立っています。      男性が子供を抱き上げて立っています。   \n","4       草地の上で牛と男性が立っています。     農場で白い牛が男性の隣に立っています。   \n","\n","                                                 bow  \\\n","0  二人 の 男性 が ジャンボジェット 機 を 見 て い ます 。 [SEP] 2人 の 男...   \n","1  2人 の 男性 が 、 白い 飛行機 を 眺め て い ます 。 [SEP] 二人 の 男性...   \n","2  男性 が 子供 を 抱き上げ て 立っ て い ます 。 [SEP] 坊主頭 の 男性 が ...   \n","3  坊主頭 の 男性 が 子供 を 抱い て 立っ て い ます 。 [SEP] 男性 が 子供...   \n","4  草地 の 上 で 牛 と 男性 が 立っ て い ます 。 [SEP] 農場 で 白い 牛 ...   \n","\n","                                                 w2v  label  \n","0  [0.050462592, -0.13604076, 0.010860913, -0.197...      2  \n","1  [0.088041924, -0.08648207, 0.014981923, -0.196...      2  \n","2  [0.056786165, -0.1304283, 0.013041455, -0.1172...      2  \n","3  [0.042833548, -0.12648883, 0.002992845, -0.119...      0  \n","4  [0.13754506, -0.074033, 0.013317642, -0.165398...      2  "],"text/html":["\n","  <div id=\"df-59e9b11b-67a6-4fc9-81c6-ff61d5374f3b\" class=\"colab-df-container\">\n","    <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>sent1</th>\n","      <th>sent2</th>\n","      <th>bow</th>\n","      <th>w2v</th>\n","      <th>label</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>二人の男性がジャンボジェット機を見ています。</td>\n","      <td>2人の男性が、白い飛行機を眺めています。</td>\n","      <td>二人 の 男性 が ジャンボジェット 機 を 見 て い ます 。 [SEP] 2人 の 男...</td>\n","      <td>[0.050462592, -0.13604076, 0.010860913, -0.197...</td>\n","      <td>2</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>2人の男性が、白い飛行機を眺めています。</td>\n","      <td>二人の男性がジャンボジェット機を見ています。</td>\n","      <td>2人 の 男性 が 、 白い 飛行機 を 眺め て い ます 。 [SEP] 二人 の 男性...</td>\n","      <td>[0.088041924, -0.08648207, 0.014981923, -0.196...</td>\n","      <td>2</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>男性が子供を抱き上げて立っています。</td>\n","      <td>坊主頭の男性が子供を抱いて立っています。</td>\n","      <td>男性 が 子供 を 抱き上げ て 立っ て い ます 。 [SEP] 坊主頭 の 男性 が ...</td>\n","      <td>[0.056786165, -0.1304283, 0.013041455, -0.1172...</td>\n","      <td>2</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>坊主頭の男性が子供を抱いて立っています。</td>\n","      <td>男性が子供を抱き上げて立っています。</td>\n","      <td>坊主頭 の 男性 が 子供 を 抱い て 立っ て い ます 。 [SEP] 男性 が 子供...</td>\n","      <td>[0.042833548, -0.12648883, 0.002992845, -0.119...</td>\n","      <td>0</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>草地の上で牛と男性が立っています。</td>\n","      <td>農場で白い牛が男性の隣に立っています。</td>\n","      <td>草地 の 上 で 牛 と 男性 が 立っ て い ます 。 [SEP] 農場 で 白い 牛 ...</td>\n","      <td>[0.13754506, -0.074033, 0.013317642, -0.165398...</td>\n","      <td>2</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","    <div class=\"colab-df-buttons\">\n","\n","  <div class=\"colab-df-container\">\n","    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-59e9b11b-67a6-4fc9-81c6-ff61d5374f3b')\"\n","            title=\"Convert this dataframe to an interactive table.\"\n","            style=\"display:none;\">\n","\n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n","    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n","  </svg>\n","    </button>\n","\n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    .colab-df-buttons div {\n","      margin-bottom: 4px;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","    <script>\n","      const buttonEl =\n","        document.querySelector('#df-59e9b11b-67a6-4fc9-81c6-ff61d5374f3b button.colab-df-convert');\n","      buttonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","      async function convertToInteractive(key) {\n","        const element = document.querySelector('#df-59e9b11b-67a6-4fc9-81c6-ff61d5374f3b');\n","        const dataTable =\n","          await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                    [key], {});\n","        if (!dataTable) return;\n","\n","        const docLinkHtml = 'Like what you see? Visit the ' +\n","          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","          + ' to learn more about interactive tables.';\n","        element.innerHTML = '';\n","        dataTable['output_type'] = 'display_data';\n","        await google.colab.output.renderOutput(dataTable, element);\n","        const docLink = document.createElement('div');\n","        docLink.innerHTML = docLinkHtml;\n","        element.appendChild(docLink);\n","      }\n","    </script>\n","  </div>\n","\n","\n","    <div id=\"df-858c2409-4a1d-4dbc-ad97-4e3c3d71b439\">\n","      <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-858c2409-4a1d-4dbc-ad97-4e3c3d71b439')\"\n","                title=\"Suggest charts\"\n","                style=\"display:none;\">\n","\n","<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","     width=\"24px\">\n","    <g>\n","        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n","    </g>\n","</svg>\n","      </button>\n","\n","<style>\n","  .colab-df-quickchart {\n","      --bg-color: #E8F0FE;\n","      --fill-color: #1967D2;\n","      --hover-bg-color: #E2EBFA;\n","      --hover-fill-color: #174EA6;\n","      --disabled-fill-color: #AAA;\n","      --disabled-bg-color: #DDD;\n","  }\n","\n","  [theme=dark] .colab-df-quickchart {\n","      --bg-color: #3B4455;\n","      --fill-color: #D2E3FC;\n","      --hover-bg-color: #434B5C;\n","      --hover-fill-color: #FFFFFF;\n","      --disabled-bg-color: #3B4455;\n","      --disabled-fill-color: #666;\n","  }\n","\n","  .colab-df-quickchart {\n","    background-color: var(--bg-color);\n","    border: none;\n","    border-radius: 50%;\n","    cursor: pointer;\n","    display: none;\n","    fill: var(--fill-color);\n","    height: 32px;\n","    padding: 0;\n","    width: 32px;\n","  }\n","\n","  .colab-df-quickchart:hover {\n","    background-color: var(--hover-bg-color);\n","    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n","    fill: var(--button-hover-fill-color);\n","  }\n","\n","  .colab-df-quickchart-complete:disabled,\n","  .colab-df-quickchart-complete:disabled:hover {\n","    background-color: var(--disabled-bg-color);\n","    fill: var(--disabled-fill-color);\n","    box-shadow: none;\n","  }\n","\n","  .colab-df-spinner {\n","    border: 2px solid var(--fill-color);\n","    border-color: transparent;\n","    border-bottom-color: var(--fill-color);\n","    animation:\n","      spin 1s steps(1) infinite;\n","  }\n","\n","  @keyframes spin {\n","    0% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","      border-left-color: var(--fill-color);\n","    }\n","    20% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    30% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","      border-right-color: var(--fill-color);\n","    }\n","    40% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    60% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","    }\n","    80% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-bottom-color: var(--fill-color);\n","    }\n","    90% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","    }\n","  }\n","</style>\n","\n","      <script>\n","        async function quickchart(key) {\n","          const quickchartButtonEl =\n","            document.querySelector('#' + key + ' button');\n","          quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n","          quickchartButtonEl.classList.add('colab-df-spinner');\n","          try {\n","            const charts = await google.colab.kernel.invokeFunction(\n","                'suggestCharts', [key], {});\n","          } catch (error) {\n","            console.error('Error during call to suggestCharts:', error);\n","          }\n","          quickchartButtonEl.classList.remove('colab-df-spinner');\n","          quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n","        }\n","        (() => {\n","          let quickchartButtonEl =\n","            document.querySelector('#df-858c2409-4a1d-4dbc-ad97-4e3c3d71b439 button');\n","          quickchartButtonEl.style.display =\n","            google.colab.kernel.accessAllowed ? 'block' : 'none';\n","        })();\n","      </script>\n","    </div>\n","\n","    </div>\n","  </div>\n"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"dataframe","variable_name":"df_train","summary":"{\n  \"name\": \"df_train\",\n  \"rows\": 20073,\n  \"fields\": [\n    {\n      \"column\": \"sent1\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 16983,\n        \"samples\": [\n          \"\\u7537\\u6027\\u304c\\u3001\\u30d5\\u30ea\\u30b9\\u30d3\\u30fc\\u3092\\u6295\\u3052\\u3088\\u3046\\u3068\\u3057\\u3066\\u3044\\u307e\\u3059\\u3002\",\n          \"\\u767d\\u3044\\u98db\\u884c\\u6a5f\\u304c\\u6ed1\\u8d70\\u8def\\u306b\\u99d0\\u6a5f\\u3057\\u3066\\u3044\\u307e\\u3059\\u3002\",\n          \"\\u30aa\\u30d5\\u30ed\\u30fc\\u30c9\\u30d0\\u30a4\\u30af\\u306b\\u4e8c\\u4eba\\u4e57\\u308a\\u3092\\u3057\\u3066\\u3044\\u307e\\u3059\\u3002\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"sent2\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 17009,\n        \"samples\": [\n          \"\\u30ea\\u30d5\\u30a9\\u30fc\\u30e0\\u3057\\u3066\\u30c8\\u30a4\\u30ec\\u306e\\u7a93\\u304b\\u3089\\u5149\\u304c\\u3055\\u3059\\u3088\\u3046\\u306b\\u3057\\u305f\\u3044\\u3002\",\n          \"\\u30cd\\u30c3\\u30c8\\u306e\\u5411\\u3053\\u3046\\u5074\\u3067\\u3001\\u91ce\\u7403\\u306e\\u8a66\\u5408\\u304c\\u884c\\u308f\\u308c\\u3066\\u3044\\u307e\\u3059\\u3002\",\n          \"\\u7537\\u6027\\u304c\\u30b8\\u30e3\\u30f3\\u30d7\\u3057\\u3066\\u30d5\\u30ea\\u30b9\\u30d4\\u30fc\\u3092\\u53d6\\u3063\\u3066\\u3044\\u307e\\u3059\\u3002\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"bow\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 19992,\n        \"samples\": [\n          \"\\u99ac \\u306b \\u4e57\\u3063 \\u305f \\u9a0e\\u624b \\u304c \\u969c\\u5bb3\\u7269 \\u3092 \\u98db\\u3073\\u8d8a\\u3048 \\u306a\\u304c\\u3089 \\u62f3\\u9283 \\u3092 \\u69cb\\u3048 \\u3066 \\u3044 \\u307e\\u3059 \\u3002 [SEP] \\u4eba \\u3092 \\u4e57\\u305b \\u305f \\u99ac \\u304c \\u969c\\u5bb3\\u7269 \\u3092 \\u30b8\\u30e3\\u30f3\\u30d7 \\u3057 \\u3066 \\u3044 \\u307e\\u3059 \\u3002\",\n          \"\\u30b5\\u30d0\\u30f3\\u30ca \\u306b \\u4e00 \\u982d \\u306e \\u30ad\\u30ea\\u30f3 \\u304c \\u305f\\u3063 \\u3066 \\u3044 \\u307e\\u3059 \\u3002 [SEP] \\u67af\\u308c\\u8349 \\u3082 \\u76ee\\u7acb\\u3064 \\u8349\\u539f \\u306b \\u30ad\\u30ea\\u30f3 \\u304c \\u4e00 \\u982d \\u305f\\u3063 \\u3066 \\u3044 \\u307e\\u3059 \\u3002\",\n          \"\\u9ed2 \\u3068 \\u30aa\\u30ec\\u30f3\\u30b8 \\u306b \\u5857\\u3089 \\u308c \\u305f \\u30d0\\u30a4\\u30af \\u304c \\u5c55\\u793a \\u3055 \\u308c \\u3066 \\u3044 \\u307e\\u3059 \\u3002 [SEP] \\u30e2\\u30fc\\u30bf\\u30fc\\u30b7\\u30e7\\u30fc \\u3067 \\u30d0\\u30a4\\u30af \\u304c \\u5c55\\u793a \\u3055 \\u308c \\u3066 \\u3044 \\u307e\\u3059 \\u3002\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"w2v\",\n      \"properties\": {\n        \"dtype\": \"object\",\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"label\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 2,\n        \"num_unique_values\": 3,\n        \"samples\": [\n          2,\n          0,\n          1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"}},"metadata":{},"execution_count":3}],"source":["from utils import *\n","\n","# 前処理済みデータ読み込み\n","df_train = get_data(\"preprocessed_train.pkl\")\n","df_val = get_data(\"preprocessed_val.pkl\")\n","df_test = get_data(\"preprocessed_test.pkl\")\n","\n","# 中身の確認\n","df_train.head()"]},{"cell_type":"code","execution_count":4,"id":"4d624db4","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"4d624db4","executionInfo":{"status":"ok","timestamp":1747889793494,"user_tz":-540,"elapsed":8,"user":{"displayName":"TOMA Naruaki","userId":"11747312442870110137"}},"outputId":"77465665-cdcf-418e-f6dc-081d7f8ba7b7"},"outputs":[{"output_type":"stream","name":"stdout","text":["train_labels:  Counter({2: 11193, 1: 6004, 0: 2876})\n","val_labels:  Counter({2: 1350, 1: 735, 0: 349})\n","test_labels:  Counter({2: 1365, 1: 776, 0: 367})\n"]}],"source":["# データセット毎のラベル分布チェック\n","\n","import collections\n","print(\"train_labels: \", collections.Counter(df_train[\"label\"]))\n","print(\"val_labels: \", collections.Counter(df_val[\"label\"]))\n","print(\"test_labels: \", collections.Counter(df_test[\"label\"]))"]},{"cell_type":"markdown","id":"7f25f2a4","metadata":{"id":"7f25f2a4"},"source":["## case 1: BoW (TF-IDF) + ロジスティック回帰\n","最も素朴な実装例として以下の通り実装した。なおpipelineを用いているためこれまでに示したコード例ともやや異なる点に注意。（これまでのコード例ベースで書き直してもOK）\n","- (1) 前提文と仮説文をそれぞれ分かち書きする。\n","- (2) 前提文と仮説文を「前提文[SEP]仮説文」として1つの文に結合する。ここまでの処理は report4_preprocessing.ipynb で実行済み。\n","- (3) 結合した文集合をTfidfVectorizerでベクトル化。\n","- (4) シンプルな分類モデル（LogisticRegression）で学習。\n","\n","学習後、scikit-learnで用意されている [classification_report](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.classification_report.html) と、utils.py で用意した混同行列により大雑把な評価結果を出力している。それぞれの出力結果の読み方は[classification_report, ConfusionMatrixDisplayの実行例](classification_evaluations.ipynb)を参照。"]},{"cell_type":"code","execution_count":5,"id":"e55afaaa","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":659},"id":"e55afaaa","executionInfo":{"status":"ok","timestamp":1747889798110,"user_tz":-540,"elapsed":4605,"user":{"displayName":"TOMA Naruaki","userId":"11747312442870110137"}},"outputId":"bbe511c6-2988-4ecb-d647-013d47d37b6b"},"outputs":[{"output_type":"stream","name":"stdout","text":["=== BoW + LogisticRegression ===\n","               precision    recall  f1-score   support\n","\n","   entailment       0.22      0.03      0.06       367\n","contradiction       0.54      0.29      0.37       776\n","      neutral       0.59      0.88      0.70      1365\n","\n","     accuracy                           0.57      2508\n","    macro avg       0.45      0.40      0.38      2508\n"," weighted avg       0.52      0.57      0.51      2508\n","\n"]},{"output_type":"display_data","data":{"text/plain":["<Figure size 640x480 with 2 Axes>"],"image/png":"iVBORw0KGgoAAAANSUhEUgAAAlYAAAHHCAYAAAB9dxZkAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAciNJREFUeJzt3XdYFFfbBvB7F2TpVWmCgIqKvSv2jrEEu6hRsMfeW2yIhYi991fRaBITjbErdkXFEjEWREVRoyBRBASk7nx/8DFxBddBliLev1x7xZ1z5syZZcuzzzlnViYIggAiIiIiyjV5QXeAiIiIqKhgYEVERESkIQysiIiIiDSEgRURERGRhjCwIiIiItIQBlZEREREGsLAioiIiEhDGFgRERERaQgDKyIiIiINYWBFhcKDBw/Qpk0bmJiYQCaTYd++fRptPzw8HDKZDNu2bdNou1+yZs2aoVmzZhpt89mzZ9DV1UVgYKBG2yXpvLy84OjoqLH28uJ58rXIj/ed1NRU2NvbY+3atXl2DMoZBlYkCgsLw9ChQ1G6dGno6urC2NgYDRs2xIoVK/Du3bs8Pbanpydu3bqF+fPnY8eOHahdu3aeHi8/eXl5QSaTwdjYONvH8cGDB5DJZJDJZFi8eHGO23/x4gW8vb0RHBysgd7mjo+PD+rVq4eGDRuK2zLPP/Omra0Ne3t7eHh44O7duzk+RlRUFGQyGcaMGZOlbMyYMZDJZJg9e3aWsn79+qFYsWJITEzM8TE/x5kzZyCTyfD777/ny/Fy4+7du/D29kZ4eLjG2nz/b575/G/atCkOHTqksWMQUKxYMYwfPx7z589HUlJSQXeHAGgXdAeocDh06BC6d+8OhUKBfv36oXLlykhJScGFCxcwadIk3LlzBxs3bsyTY7979w6XLl3C9OnTMXLkyDw5hoODA969e4dixYrlSfufoq2tjcTERBw4cAA9evRQKdu5cyd0dXU/+03xxYsXmDNnDhwdHVG9enXJ+x0/fvyzjvcx//77L/z9/eHv75+lTKFQYPPmzQCAtLQ0hIWFYf369Th69Cju3r0LW1tbycextLSEs7MzLly4kKUsMDAQ2tra2WbMAgMDUaNGDejr6+fgrL48mzZtglKpzNE+d+/exZw5c9CsWbMs2a7cPE9at26Nfv36QRAEPHnyBOvWrUPHjh1x5MgRuLm5fXa7X4r8et/p378/pk6dil27dmHAgAF5eiz6NAZWhMePH8PDwwMODg44deoUbGxsxLIRI0bg4cOHefot899//wUAmJqa5tkxZDIZdHV186z9T1EoFGjYsCF+/vnnLIHVrl270L59e+zZsydf+pKYmAh9fX3o6OhotN2ffvoJ2tra6NixY5YybW1tfPfddyrb6tevjw4dOuDQoUMYPHhwjo7VqFEjbN++HfHx8TA0NAQAJCQk4ObNm+jRowf279+P9PR0aGlpAQAiIiLw6NEjuLu75/i8MoONL2UYWdMf4rl5npQrV07l7961a1dUrFgRK1asyPfAKiEhAQYGBvl6zPx63zE1NUWbNm2wbds2BlaFAIcCCX5+foiPj8eWLVtUgqpMZcuWVRl2SUtLw9y5c1GmTBkoFAo4Ojrihx9+QHJyssp+jo6O6NChAy5cuIC6detCV1cXpUuXxvbt28U63t7ecHBwAABMmjQJMplM/Mb8sbki3t7ekMlkKtsCAgLQqFEjmJqawtDQEOXLl8cPP/wgln9srsOpU6fQuHFjGBgYwNTUFO7u7ggJCcn2eA8fPoSXlxdMTU1hYmKC/v3752hYqXfv3jhy5AhiYmLEbVevXsWDBw/Qu3fvLPWjo6MxceJEVKlSBYaGhjA2NsY333yDmzdvinXOnDmDOnXqAMj41po57JJ5ns2aNUPlypVx/fp1NGnSBPr6+uLj8uHcGU9PT+jq6mY5fzc3N5iZmeHFixdqz2/fvn2oV6+eGOh8irW1NYCMoOt9jx49Qvfu3WFubg59fX3Ur18/S2DfqFEjpKen4/Lly+K2oKAgpKWlYeLEiYiPj1cZGs3MYDVq1EhS3/KTlPMFgCdPnuDbb7+FgYEBLC0tMW7cOBw7dgwymQxnzpwR62X3uvnll19Qq1YtGBkZwdjYGFWqVMGKFSsAANu2bUP37t0BAM2bNxefQ5ltZjfHKikpCd7e3ihXrhx0dXVhY2ODLl26ICwsTO25uri4oHjx4lnqJScnY/bs2ShbtiwUCgXs7e0xefLkLO8p7969w+jRo1G8eHEYGRnh22+/xfPnzyGTyeDt7S3Wy3zN3r17F71794aZmZnK3/6nn35CrVq1oKenB3Nzc3h4eODZs2cqx3rw4AG6du0Ka2tr6Orqws7ODh4eHoiNjRXrFKb3ndatW+PChQuIjo5W+zegvMeMFeHAgQMoXbo0GjRoIKn+oEGD4O/vj27dumHChAkICgqCr68vQkJC8Mcff6jUffjwIbp164aBAwfC09MT//vf/+Dl5YVatWqhUqVK6NKlC0xNTTFu3Dj06tUL7dq1k/zBnOnOnTvo0KEDqlatCh8fHygUCjx8+PCTE6hPnDiBb775BqVLl4a3tzfevXuHVatWoWHDhvjrr7+yfDj16NEDTk5O8PX1xV9//YXNmzfD0tISCxculNTPLl264Pvvv8fevXvFb5W7du1ChQoVULNmzSz1Hz16hH379qF79+5wcnLCy5cvsWHDBjRt2lQcPnNxcYGPjw9mzZqFIUOGoHHjxgCg8rd8/fo1vvnmG3h4eOC7776DlZVVtv1bsWIFTp06BU9PT1y6dAlaWlrYsGEDjh8/jh07dqgdrktNTcXVq1cxbNiwj9Z59eoVACA9PR2PHj3ClClTYGFhgQ4dOoh1Xr58iQYNGiAxMRGjR4+GhYUF/P398e233+L3339H586dAfwXIF24cAGtWrUCkBE8lStXDjVq1ICdnR0CAwNRq1Ytsez9/QoLqeebkJCAFi1aICIiAmPGjIG1tTV27dqF06dPf/IYAQEB6NWrF1q2bCk+V0NCQhAYGIgxY8agSZMmGD16NFauXIkffvgBLi4uACD+/0Pp6eno0KEDTp48CQ8PD4wZMwZv375FQEAAbt++jTJlyny0L7GxsXjz5o1KHaVSiW+//RYXLlzAkCFD4OLiglu3bmHZsmW4f/++ykIWLy8v7N69G3379kX9+vVx9uxZtG/f/qPH6969O5ydnbFgwQIIggAAmD9/PmbOnIkePXpg0KBB+Pfff7Fq1So0adIEN27cgKmpKVJSUuDm5obk5GSMGjUK1tbWeP78OQ4ePIiYmBiYmJgUuvedWrVqQRAEXLx4UeU1RQVAoK9abGysAEBwd3eXVD84OFgAIAwaNEhl+8SJEwUAwqlTp8RtDg4OAgDh3Llz4raoqChBoVAIEyZMELc9fvxYACAsWrRIpU1PT0/BwcEhSx9mz54tvP/UXbZsmQBA+Pfffz/a78xjbN26VdxWvXp1wdLSUnj9+rW47ebNm4JcLhf69euX5XgDBgxQabNz586ChYXFR4/5/nkYGBgIgiAI3bp1E1q2bCkIgiCkp6cL1tbWwpw5c7J9DJKSkoT09PQs56FQKAQfHx9x29WrV7OcW6amTZsKAIT169dnW9a0aVOVbceOHRMACPPmzRMePXokGBoaCp06dfrkOT58+FAAIKxatSrb8weQ5VayZEnh+vXrKnXHjh0rABDOnz8vbnv79q3g5OQkODo6qjwelpaW4mMpCILg5uYm9O/fXxAEQejRo4fQvXt3sax27dqCs7PzJ88jO02bNhU8PT1zvN/p06cFAMJvv/320TpSz3fJkiUCAGHfvn1ivXfv3gkVKlQQAAinT58Wt3/4uhkzZoxgbGwspKWlfbQfv/32W5Z2Mn34PPnf//4nABCWLl2apa5SqRT/DUAYOHCg8O+//wpRUVHCtWvXhLZt22Z5nu/YsUOQy+Uqj4EgCML69esFAEJgYKAgCIJw/fp1AYAwduxYlXpeXl4CAGH27NnitszXbK9evVTqhoeHC1paWsL8+fNVtt+6dUvQ1tYWt9+4ceOTf7vC9r7z4sULAYCwcOHCj/aH8geHAr9ycXFxAAAjIyNJ9Q8fPgwAGD9+vMr2CRMmAECWIYyKFSuKWRQAKFGiBMqXL49Hjx59dp8/lDk3688//5Q8aTciIgLBwcHw8vKCubm5uL1q1apo3bq1eJ7v+/7771XuN27cGK9fvxYfQyl69+6NM2fOIDIyEqdOnUJkZGS2w4BAxrwsuTzjJZqeno7Xr1+Lww1//fWX5GMqFAr0799fUt02bdpg6NCh8PHxQZcuXaCrq4sNGzZ8cr/Xr18DAMzMzLIt19XVRUBAAAICAnDs2DFs2LABhoaGaNeuHe7fvy/WO3z4MOrWrauSWTI0NMSQIUMQHh6usoqwYcOGCAoKQnp6OpRKJS5fvixm6ho2bChmDhITExEcHCwpW5WamopXr16p3FJTU5GcnJxle04niGdH6vkePXoUJUuWxLfffivW09XVlTQ3zdTUFAkJCQgICMh1fwFgz549KF68OEaNGpWl7MMh+i1btqBEiRKwtLRE7dq1cfLkSUyePFnl/eO3336Di4sLKlSooPL4tmjRAgDErNzRo0cBAMOHD1c5Rnb9yPTha3bv3r1QKpXo0aOHyrGsra3h7OwsHsvExAQAcOzYsY8O9xe2953M115mZpgKDgOrr5yxsTEA4O3bt5LqP3nyBHK5HGXLllXZbm1tDVNTUzx58kRle6lSpbK0YWZmhjdv3nxmj7Pq2bMnGjZsiEGDBsHKygoeHh7YvXu32je7zH6WL18+S5mLiwtevXqFhIQEle0fnkvmG1lOzqVdu3YwMjLCr7/+ip07d6JOnTpZHstMSqUSy5Ytg7OzMxQKBYoXL44SJUrg77//Vpnn8SklS5bM0QTkxYsXw9zcHMHBwVi5ciUsLS0l7yv8/3DLh7S0tNCqVSu0atUKbdq0wZAhQ3DixAnExsZi2rRpYr0nT5589G+SWZ6pUaNG4lyq27dvIzY2VrzMQ4MGDfDixQuEh4eLc6+kBFaBgYEoUaKEyu3ixYv45Zdfsmx/+vSp5MflY6Se75MnT1CmTJksgcvHnjvvGz58OMqVK4dvvvkGdnZ2GDBggBikfI6wsDCUL18+y9y47Li7uyMgIACHDh0S5wwlJiaKXxiAjLlMd+7cyfL4litXDkDG5TWA/957nJycVI6h7jH4sO6DBw8gCAKcnZ2zHC8kJEQ8lpOTE8aPH4/NmzejePHicHNzw5o1a1Red4XtfSfztffhc4TyH+dYfeWMjY1ha2uL27dv52g/qS/ezFVZH/rYB7CUY6Snp6vc19PTw7lz53D69GkcOnQIR48exa+//ooWLVrg+PHjH+1DTuXmXDIpFAp06dIF/v7+ePTokcqE2w8tWLAAM2fOxIABAzB37lyYm5tDLpdj7NixOcqW6OnpSa4LADdu3BA/YG7duoVevXp9ch8LCwsAOQsy7ezsUL58eZw7dy5H/cv0/jwrHR0dmJubo0KFCgCA6tWrQ19fHxcuXMDjx49V6qtTrVq1LJmdCRMmwNraGpMmTVLZnjn5vrCztLREcHAwjh07hiNHjuDIkSPYunUr+vXrl+2lMTTJzs5OnAPXrl07FC9eHCNHjkTz5s3RpUsXABlfIKpUqYKlS5dm24a9vf1nH//D575SqYRMJsORI0eyfT2/P79zyZIl8PLywp9//onjx49j9OjR8PX1xeXLl2FnZ1fo3ncyX3vFixfXyHHp8zGwInTo0AEbN27EpUuX4Orqqraug4MDlEolHjx4oDK59eXLl4iJiRFX+GmCmZmZygq6TB9mxQBALpejZcuWaNmyJZYuXYoFCxZg+vTpOH36tPjG/uF5AEBoaGiWsnv37qF48eJ5tjS7d+/e+N///ge5XA4PD4+P1vv999/RvHlzbNmyRWV7TEyMypunJr+hJiQkoH///qhYsSIaNGgAPz8/dO7cWVx5+DGlSpWCnp6eGMRIlZaWhvj4ePG+g4PDR/8mmeWZatasKQZPCoUCrq6u4mOhra2NOnXqIDAwEI8fP4alpaWYAVHHzMwsy/PFzMwMNjY22T6Pckvq+To4OODu3bsQBEHl7/3w4UNJx9HR0UHHjh3RsWNHKJVKDB8+HBs2bMDMmTNRtmzZHD2HypQpg6CgIKSmpub40g5Dhw7FsmXLMGPGDHTu3BkymQxlypTBzZs30bJlS7X9yHzvefz4MZydncXtUh+DzL4LggAnJydJz4cqVaqgSpUqmDFjBi5evIiGDRti/fr1mDdvHoDC9b6T+dr72KIDyj8cCiRMnjwZBgYGGDRoEF6+fJmlPCwsTFya3a5dOwDA8uXLVepkfttUt0Inp8qUKYPY2Fj8/fff4raIiIgsKw+zW16ceaHMD5drZ7KxsUH16tXh7++vErzdvn0bx48fF88zLzRv3hxz587F6tWr1WY9tLS0snwr/e233/D8+XOVbZlvxNkFoTk1ZcoUPH36FP7+/li6dCkcHR3h6en50ccxU7FixVC7dm1cu3ZN8rHu37+P0NBQVKtWTdzWrl07XLlyBZcuXRK3JSQkYOPGjXB0dETFihXF7dra2qhXrx4CAwMRGBiYZVVrgwYNcO7cOVy+fFnlSvCFidTzdXNzw/Pnz7F//36xXlJSEjZt2vTJY2TOf8skl8tRtWpVAP+9PnLyHOratStevXqF1atXZyn7VPZWW1sbEyZMQEhICP78808AGavenj9/nu25vHv3Thway7zu1Yc/3bJq1apP9jlTly5doKWlhTlz5mTpqyAI4mMVFxeHtLQ0lfIqVapALpeLj1lhe9+5fv06ZDLZJ78cU95jxopQpkwZ7Nq1Cz179oSLi4vKldcvXryI3377DV5eXgAyhko8PT2xceNGxMTEoGnTprhy5Qr8/f3RqVMnNG/eXGP98vDwwJQpU9C5c2eMHj0aiYmJWLduHcqVK6cyedvHxwfnzp1D+/bt4eDggKioKKxduxZ2dnZqh38WLVqEb775Bq6urhg4cKC47NnExETtEF1uyeVyzJgx45P1OnToAB8fH/Tv3x8NGjTArVu3sHPnTpQuXVqlXpkyZWBqaor169fDyMgIBgYGqFevXpb5JZ9y6tQprF27FrNnzxYv/7B161Y0a9YMM2fOhJ+fn9r93d3dMX36dMTFxYlz9zKlpaXhp59+ApAxHBMeHo7169dDqVSq/PzM1KlT8fPPP+Obb77B6NGjYW5uDn9/fzx+/Bh79uxRmZsDZAzvZU44/jB4atCgAXx9fcV6BWXPnj1iBup9np6eks936NChWL16NXr16oUxY8bAxsZGvGI/oD5rOWjQIERHR6NFixaws7PDkydPsGrVKlSvXl3MblSvXh1aWlpYuHAhYmNjoVAo0KJFi2zn1/Xr1w/bt2/H+PHjceXKFTRu3BgJCQk4ceIEhg8f/smLsHp5eWHWrFlYuHAhOnXqhL59+2L37t34/vvvcfr0aTRs2BDp6em4d+8edu/ejWPHjqF27dqoVasWunbtiuXLl+P169fi5RYyFz9IybqVKVMG8+bNw7Rp0xAeHo5OnTrByMgIjx8/xh9//IEhQ4Zg4sSJOHXqFEaOHInu3bujXLlySEtLw44dO6ClpYWuXbsCKHzvOwEBAWjYsKE4LE8FqGAWI1JhdP/+fWHw4MGCo6OjoKOjIxgZGQkNGzYUVq1aJSQlJYn1UlNThTlz5ghOTk5CsWLFBHt7e2HatGkqdQQh43IL7du3z3KcD5dvf+xyC4IgCMePHxcqV64s6OjoCOXLlxd++umnLJdbOHnypODu7i7Y2toKOjo6gq2trdCrVy/h/v37WY7x4SUJTpw4ITRs2FDQ09MTjI2NhY4dOwp3795VqZN5vA+XVW/dulUAIDx+/Pijj6kgqF5u4WM+drmFCRMmCDY2NoKenp7QsGFD4dKlS9leJuHPP/8UKlasKGhra6ucZ9OmTYVKlSple8z324mLixMcHByEmjVrCqmpqSr1xo0bJ8jlcuHSpUtqz+Hly5eCtra2sGPHjiznjw8utWBsbCy0bNlSOHHiRJZ2wsLChG7dugmmpqaCrq6uULduXeHgwYPZHjPz8hDa2tpCQkKCStnr168FmUwmABCCgoLU9l2d3F5u4WO3zMsLSD3fR48eCe3btxf09PSEEiVKCBMmTBD27NkjABAuX74s1vvwcgu///670KZNG8HS0lLQ0dERSpUqJQwdOlSIiIhQaX/Tpk1C6dKlBS0tLZVLL2T3fEtMTBSmT58uvgdYW1sL3bp1E8LCwsQ6AIQRI0Zk+9h4e3urHCMlJUVYuHChUKlSJUGhUAhmZmZCrVq1hDlz5gixsbHifgkJCcKIESMEc3Nz8VIgoaGhAgDhxx9/FOt97DWbac+ePUKjRo0EAwMDwcDAQKhQoYIwYsQIITQ0VHysBwwYIJQpU0bQ1dUVzM3NhebNm6s8XwvT+05MTIygo6MjbN68OdvzpfwlE4QczLwlIlJj4MCBuH//Ps6fP1/QXfkqLF++HOPGjcM///yDkiVLFnR3CkRwcDBq1KiBn376CX369Cno7hSI5cuXw8/PD2FhYTlerEKax8CKiDTm6dOnKFeuHE6ePFlo5zV9qd69e6fyoZmUlIQaNWogPT1d5VpgRdmHjwGQMbS4Y8cOhIeH52oF4ZcqNTUVZcqUwdSpU7Nc44sKBudYEZHGlCpVCklJSQXdjSKpS5cuKFWqFKpXr47Y2Fj89NNPuHfvHnbu3FnQXcs3fn5+uH79Opo3bw5tbW3x8hFDhgz5KoMqIGPhiCauqUaaw4wVEdEXYPny5di8eTPCw8ORnp6OihUrYvLkyejZs2dBdy3fBAQEYM6cObh79y7i4+NRqlQp9O3bF9OnT5d0wVKi/MDAioiIiEhDeB0rIiIiIg1hYEVERESkIRyUJgAZF2188eIFjIyM+COeRERfIEEQ8PbtW9ja2ma5mK6mJCUlISUlRSNt6ejoiBe5LUoYWBEA4MWLF1/tqhoioqLk2bNnsLOz03i7SUlJ0DOyANISNdKetbU1Hj9+XOSCKwZWBAAwMjICADx8/AxGH/wcCRU9qWnKgu4C5aO7z2MLuguUDxLi36Jzkyri+7mmpaSkAGmJUFT0BLR0ctdYegoi7/ojJSWFgRUVTZnDf0bGxll+542KHgZWXxeDWC7+/prk+XQObV3IchlYCbKiO8WbgRURERFJJwOQ2+CtCE/lZWBFRERE0snkGbfctlFEFd0zIyIiIspnzFgRERGRdDKZBoYCi+5YIAMrIiIiko5DgWoV3TMjIiIiymfMWBEREZF0HApUi4EVERER5YAGhgKL8IBZ0T0zIiIionzGjBURERFJx6FAtRhYERERkXRcFahW0T0zIiIionzGjBURERFJx6FAtRhYERERkXQcClSLgRURERFJx4yVWkU3ZCQiIiLKZ8xYERERkXQcClSLgRURERFJJ5NpILDiUCARERERfQIzVkRERCSdXJZxy20bRRQDKyIiIpKOc6zUKrpnRkRERJTPmLEiIiIi6XgdK7UYWBEREZF0HApUq+ieGREREVE+Y8aKiIiIpONQoFoMrIiIiEg6DgWqxcCKiIiIpGPGSq2iGzISERER5TNmrIiIiEg6DgWqxcCKiIiIpONQoFpFN2QkIiIiymfMWBEREVEOaGAosAjndRhYERERkXQcClSr6IaMRERERPmMGSsiIiKSTibTwKrAopuxYmBFRERE0vFyC2oV3TMjIiIiymfMWBEREZF0nLyuFgMrIiIiko5DgWoxsCIiIiLpmLFSq+iGjERERET5jBkrIiIiko5DgWoxsCIiIiLpOBSoVtENGYmIiIjyGTNWREREJJlMJoOMGauPYmBFREREkjGwUo9DgUREREQawowVERERSSf7/1tu2yiiGFgRERGRZBwKVI9DgUREREQawowVERERScaMlXoMrIiIiEgyBlbqMbD6TNu2bcPYsWMRExMDAPD29sa+ffsQHBxcoP2irAL/eohVO07g5r2niHwVh58WDUb7ZtUAAKlp6Zi37gACAu/gyfPXMDbURdO6FTB75LewKWFasB2nXFu5PQDz1h3AkB5NMW9cV5UyQRDQa/x6nLocgm0/DkK7plULqJckxf7jV7D/+BW8/DcGAOBgZ4m+3ZqhXo1yAICDJ67i1IW/8eBxBBLfJePPrT/A0EBPpY2de8/g8l/3ERYeCW1tLezfNj2/T6NIYGCl3lc/x8rb2xvVq1fP8X49e/bE/fv3Nd8hDXB0dMTy5csLuhuFRuK7ZFQuVxKLJvfMWpaUgr/vPcOkgd/gzI4p2O43GA+fvETvCRsKoKekSTfuPsH2fYGoWNY22/INv5zJ/YcD5Zvi5sYY3LsN1v04DGt9v0eNyk6Y5bcL4c9eAgCSk1NRp7ozendu8tE2UtPS0bR+ZXRsUye/uk0acu7cOXTs2BG2traQyWTYt2+fSrkgCJg1axZsbGygp6eHVq1a4cGDByp1oqOj0adPHxgbG8PU1BQDBw5EfHy8Sp2///4bjRs3hq6uLuzt7eHn55fjvn71gdXn0tPTg6WlZUF3gyRo3bASZgzriA7Nq2UpMzHUwx9rRqFz65pwdrRCnSpO8JvUA8Ehz/AsMroAekuaEJ+YjGHe27Fkai+YGulnKb91/x+s+/kUlk/vXQC9o8/RoHYF1KtZDnY2FrC3LY6BvVpDT1cHdx/8AwDo2r4BenVqAhdn+4+24dWjJbp1aACnUlb51e2iSaahWw4kJCSgWrVqWLNmTbblfn5+WLlyJdavX4+goCAYGBjAzc0NSUlJYp0+ffrgzp07CAgIwMGDB3Hu3DkMGTJELI+Li0ObNm3g4OCA69evY9GiRfD29sbGjRtz1NcvPrBSKpXw9fWFk5MT9PT0UK1aNfz+++8AgDNnMr6Rnjx5ErVr14a+vj4aNGiA0NBQABnDeXPmzMHNmzfF1Oa2bdsAAEuXLkWVKlVgYGAAe3t7DB8+XCWy3bZtG0xNTT/aLy8vL3Tq1AkLFiyAlZUVTE1N4ePjg7S0NEyaNAnm5uaws7PD1q1bVfZ79uwZevToAVNTU5ibm8Pd3R3h4eFZ2l28eDFsbGxgYWGBESNGIDU1FQDQrFkzPHnyBOPGjdNMuvYrFBf/DjKZDCaGep+uTIXS1MW/oXWDSmhat3yWssSkFAyb7Y8fJ3aHlYVxAfSOcitdqcSpwL+RlJyCiuU+HkhR3sj8bMntLSe++eYbzJs3D507d85SJggCli9fjhkzZsDd3R1Vq1bF9u3b8eLFCzGzFRISgqNHj2Lz5s2oV68eGjVqhFWrVuGXX37BixcvAAA7d+5ESkoK/ve//6FSpUrw8PDA6NGjsXTp0hz19YsPrHx9fbF9+3asX78ed+7cwbhx4/Ddd9/h7NmzYp3p06djyZIluHbtGrS1tTFgwAAAGcN5EyZMQKVKlRAREYGIiAj07JkxXCSXy7Fy5UrcuXMH/v7+OHXqFCZPnpyjvp06dQovXrzAuXPnsHTpUsyePRsdOnSAmZkZgoKC8P3332Po0KH455+Mb1ypqalwc3ODkZERzp8/j8DAQBgaGqJt27ZISUkR2z19+jTCwsJw+vRp+Pv7Y9u2bWJAuHfvXtjZ2cHHx0c8J5IuKTkV3qv/RNc2tWDMwOqL9EfAddwKfYbpwzpmWz5z+V7UqeKEb5pwTtWX5tHTSLTvOxdte8/B8k0HMGdibzjaceTga/f48WNERkaiVatW4jYTExPUq1cPly5dAgBcunQJpqamqF27tlinVatWkMvlCAoKEus0adIEOjo6Yh03NzeEhobizZs3kvvzRU9eT05OxoIFC3DixAm4uroCAEqXLo0LFy5gw4YNYopv/vz5aNq0KQBg6tSpaN++PZKSkqCnpwdDQ0Noa2vD2tpape2xY8eK/3Z0dMS8efPw/fffY+3atZL7Z25ujpUrV0Iul6N8+fLw8/NDYmIifvjhBwDAtGnT8OOPP+LChQvw8PDAr7/+CqVSic2bN4vR/NatW2FqaoozZ86gTZs2AAAzMzOsXr0aWlpaqFChAtq3b4+TJ09i8ODBMDc3h5aWFoyMjLKc04ePXXJysng/Li5O8nkVValp6eg/bQsEQcCSqVnnY1Hh9/zlG0xfthe/rRwOXUWxLOVHz9/ChesPcNI/Z1+SqHCwty2OjYuGIyExCecu38HCNXuwdM5ABlf5TCaDBiavZ/zvw88ehUIBhUKRo6YiIyMBAFZWqkO8VlZWYllkZGSW6Tva2towNzdXqePk5JSljcwyMzMzSf35ogOrhw8fIjExEa1bt1bZnpKSgho1aoj3q1b975upjY0NACAqKgqlSpX6aNsnTpyAr68v7t27h7i4OKSlpSEpKQmJiYnQ1886ZyM7lSpVglz+X1LQysoKlStXFu9raWnBwsICUVFRAICbN2/i4cOHMDIyUmknKSkJYWFhKu1qaWmpnNOtW7ck9SmTr68v5syZk6N9irLMoOpZ5BvsXzuK2aov1M17z/DqzVu08lokbktPV+JScBi27DkPr86NEP78FZzbTFHZb8APW1C/WhnsWzs6v7tMOVBMWxslrS0AAOVKl0Ro2HPsPXwJ44e4F3DPvi4yaGKaScb+9vaqQ7mzZ8+Gt7d3LtsuWF90YJU55+nQoUMoWbKkSplCoRCDkWLF/vvmmvlkUCqVH203PDwcHTp0wLBhwzB//nyYm5vjwoULGDhwIFJSUiQHVu8fN/PY2W3L7Et8fDxq1aqFnTt3ZmmrRIkSattVdz7ZmTZtGsaPHy/ej4uLy/IE/1pkBlVhT//FgfWjYW5qWNBdos/UpHY5nP1pqsq2MfN3oayDJUZ91wrmpobo16mBSnnT737E3DFd0KZRZdCXRakUkJqaXtDdoFx49uwZjI3/m+uY02wVAHF05uXLl2LyJPN+5qp/a2trMYmRKS0tDdHR0eL+1tbWePnypUqdzPvqRoA+9EUHVhUrVoRCocDTp0/Fob73vZ/l+RgdHR2kp6u+MK9fvw6lUoklS5aIGafdu3drptNq1KxZE7/++issLS1Vnmg5ld05fehz0q1fqvjEZDx+9q94/8mL17gV+g9MTfRhXdwEnlM24+a9Z/hl2fdITxfw8lVGatrMRB86xb7ol8hXx9BAFy5lVC+voK+rA3NjA3F7dhPWS1qZwcHWIl/6SJ9n867jqFu9HCyLmyAxKRmnLvyNm3fD8eP0fgCA6Ji3iI6Jx/PI1wCAR09fQl9PAcviJjA2zPgy/PJVDN7Gv0PUq1golUo8DM+Yg1rS2hx6ul/H+6EmaPI6VsbGxrn6vAMAJycnWFtb4+TJk2IgFRcXh6CgIAwbNgwA4OrqipiYGFy/fh21atUCkDEPWqlUol69emKd6dOnIzU1VUxgBAQEoHz58pKHAYEvPLAyMjLCxIkTMW7cOCiVSjRq1AixsbEIDAyEsbExHBwcPtmGo6MjHj9+jODgYNjZ2cHIyAhly5ZFamoqVq1ahY4dOyIwMBDr16/P8/Pp06cPFi1aBHd3d/j4+MDOzg5PnjzB3r17MXnyZNjZ2Ulqx9HREefOnYOHhwcUCgWKFy+exz0v3IJDnqDj9yvF+9OX7QUA9GpfD1OHtMORcxnDqE36/Kiy34H1o9GoVrn86ygRfdSb2AT8uGYPot+8hYG+Lko7WOHH6f1Qu2pZAMCB41ex/ffTYv1xs7cAACYN74y2zWoCALb9egrHz94Q6wydnDFndsnsAaheSXVuDanxGZdLyLaNHIiPj8fDhw/F+5mf2+bm5ihVqhTGjh2LefPmwdnZGU5OTpg5cyZsbW3RqVMnAICLiwvatm2LwYMHY/369UhNTcXIkSPh4eEBW9uML129e/fGnDlzMHDgQEyZMgW3b9/GihUrsGzZshz19YsOrABg7ty5KFGiBHx9ffHo0SOYmpqiZs2a+OGHHyQNj3Xt2hV79+5F8+bNERMTg61bt8LLywtLly7FwoULMW3aNDRp0gS+vr7o169fnp6Lvr4+zp07hylTpqBLly54+/YtSpYsiZYtW+Yoovfx8cHQoUNRpkwZJCcnQxCEPOx14deoVjm8ubr6o+XqyujL96l5U1GXVqotp8Jh0rCsy+zf59mjBTx7tFBbZ8qILpgyoosmu0X55Nq1a2jevLl4P3Mqi6enJ7Zt24bJkycjISEBQ4YMQUxMDBo1aoSjR49CV1dX3Gfnzp0YOXIkWrZsCblcjq5du2Llyv9e/yYmJjh+/DhGjBiBWrVqoXjx4pg1a5bKta6kkAlf+6cuAchIm5qYmODl69hcp2Wp8EtNy9mcPPqy3XoWW9BdoHyQEB+HNjUdERubN+/jmZ8TZr22QK4jba7xxyhTEvHm54F51teC9MVnrIiIiCj/aGKOVVG+eDUDKyIiIpKMgZV6X/yV14mIiIgKC2asiIiISLoCWBX4JWFgRURERJJxKFA9DgUSERERaQgzVkRERCQZM1bqMbAiIiIiyRhYqcehQCIiIiINYcaKiIiIJGPGSj0GVkRERCQdL7egFocCiYiIiDSEGSsiIiKSjEOB6jGwIiIiIskYWKnHwIqIiIgkY2ClHudYEREREWkIM1ZEREQkHVcFqsXAioiIiCTjUKB6HAokIiIi0hBmrIiIiEgyZqzUY2BFREREksmggcCqCE+y4lAgERERkYYwY0VERESScShQPQZWREREJB0vt6AWhwKJiIiINIQZKyIiIpKMQ4HqMbAiIiIiyRhYqcfAioiIiCSTyTJuuW2jqOIcKyIiIiINYcaKiIiIJMvIWOV2KFBDnSmEGFgRERGRdBoYCuTlFoiIiIjok5ixIiIiIsm4KlA9BlZEREQkGVcFqsehQCIiIiINYcaKiIiIJJPLZZDLc5dyEnK5f2HGwIqIiIgk41CgehwKJCIiItIQZqyIiIhIMq4KVI+BFREREUnGoUD1GFgRERGRZMxYqcc5VkREREQawowVERERScaMlXoMrIiIiEgyzrFSj0OBRERERBrCjBURERFJJoMGhgJRdFNWDKyIiIhIMg4FqsehQCIiIiINYcaKiIiIJOOqQPUYWBEREZFkHApUj0OBRERERBrCjBURERFJxqFA9ZixIiIiIskyhwJze8uJ9PR0zJw5E05OTtDT00OZMmUwd+5cCIIg1hEEAbNmzYKNjQ309PTQqlUrPHjwQKWd6Oho9OnTB8bGxjA1NcXAgQMRHx+viYdFxMCKiIiIJMvMWOX2lhMLFy7EunXrsHr1aoSEhGDhwoXw8/PDqlWrxDp+fn5YuXIl1q9fj6CgIBgYGMDNzQ1JSUlinT59+uDOnTsICAjAwYMHce7cOQwZMkRjjw3AoUAiIiIq5C5evAh3d3e0b98eAODo6Iiff/4ZV65cAZCRrVq+fDlmzJgBd3d3AMD27dthZWWFffv2wcPDAyEhITh69CiuXr2K2rVrAwBWrVqFdu3aYfHixbC1tdVIXxlYkQpBEFRSq1Q0PX2dWNBdoHx0Mvx1QXeB8kFSwtv8OZAGVgVmXng9Li5OZbNCoYBCochSvUGDBti4cSPu37+PcuXK4ebNm7hw4QKWLl0KAHj8+DEiIyPRqlUrcR8TExPUq1cPly5dgoeHBy5dugRTU1MxqAKAVq1aQS6XIygoCJ07d87lSWVgYEVERESSaXLyur29vcr22bNnw9vbO0v9qVOnIi4uDhUqVICWlhbS09Mxf/589OnTBwAQGRkJALCyslLZz8rKSiyLjIyEpaWlSrm2tjbMzc3FOprAwIqIiIgKxLNnz2BsbCzezy5bBQC7d+/Gzp07sWvXLlSqVAnBwcEYO3YsbG1t4enpmV/dlYSBFREREUmmyQuEGhsbqwRWHzNp0iRMnToVHh4eAIAqVargyZMn8PX1haenJ6ytrQEAL1++hI2Njbjfy5cvUb16dQCAtbU1oqKiVNpNS0tDdHS0uL8mcFUgERERSVYQqwITExMhl6uGLFpaWlAqlQAAJycnWFtb4+TJk2J5XFwcgoKC4OrqCgBwdXVFTEwMrl+/LtY5deoUlEol6tWr97kPRxbMWBEREVGh1rFjR8yfPx+lSpVCpUqVcOPGDSxduhQDBgwAkBHsjR07FvPmzYOzszOcnJwwc+ZM2NraolOnTgAAFxcXtG3bFoMHD8b69euRmpqKkSNHwsPDQ2MrAgEGVkRERJQDBfFbgatWrcLMmTMxfPhwREVFwdbWFkOHDsWsWbPEOpMnT0ZCQgKGDBmCmJgYNGrUCEePHoWurq5YZ+fOnRg5ciRatmwJuVyOrl27YuXKlbk7mQ/IBK6tJ2SkTE1MTBD5KkbSeDd92R5FJRR0Fygf7b2ruRVPVHglJbzFgs41ERsbmyfv45mfE/XnHYW2rkGu2kpLSsDlGW3zrK8FiXOsiIiIiDSEQ4FEREQkGX+EWT0GVkRERCRZQcyx+pIwsCIiIiLJmLFSj3OsiIiIiDSEGSsiIiKSjEOB6jGwIiIiIsk4FKgehwKJiIiINIQZKyIiIpJMBg0MBWqkJ4UTAysiIiKSTC6TQZ7LyCq3+xdmHAokIiIi0hBmrIiIiEgyrgpUj4EVERERScZVgeoxsCIiIiLJ5LKMW27bKKo4x4qIiIhIQ5ixIiIiIulkGhjKK8IZKwZWREREJBknr6vHoUAiIiIiDWHGioiIiCST/f9/uW2jqGJgRURERJJxVaB6HAokIiIi0hBmrIiIiEgyXiBUPUmB1f79+yU3+O233352Z4iIiKhw46pA9SQFVp06dZLUmEwmQ3p6em76Q0RERPTFkhRYKZXKvO4HERERfQHkMhnkuUw55Xb/wixXc6ySkpKgq6urqb4QERFRIcehQPVyvCowPT0dc+fORcmSJWFoaIhHjx4BAGbOnIktW7ZovINERERUeGROXs/trajKcWA1f/58bNu2DX5+ftDR0RG3V65cGZs3b9Zo54iIiIi+JDkOrLZv346NGzeiT58+0NLSErdXq1YN9+7d02jniIiIqHDJHArM7a2oyvEcq+fPn6Ns2bJZtiuVSqSmpmqkU0RERFQ4cfK6ejnOWFWsWBHnz5/Psv33339HjRo1NNIpIiIioi9RjjNWs2bNgqenJ54/fw6lUom9e/ciNDQU27dvx8GDB/Oij0RERFRIyP7/lts2iqocZ6zc3d1x4MABnDhxAgYGBpg1axZCQkJw4MABtG7dOi/6SERERIUEVwWq91nXsWrcuDECAgI03RciIiKiL9pnXyD02rVrCAkJAZAx76pWrVoa6xQREREVTnJZxi23bRRVOQ6s/vnnH/Tq1QuBgYEwNTUFAMTExKBBgwb45ZdfYGdnp+k+EhERUSGhiaG8ojwUmOM5VoMGDUJqaipCQkIQHR2N6OhohISEQKlUYtCgQXnRRyIiIqIvQo4zVmfPnsXFixdRvnx5cVv58uWxatUqNG7cWKOdIyIiosKnCCecci3HgZW9vX22FwJNT0+Hra2tRjpFREREhROHAtXL8VDgokWLMGrUKFy7dk3cdu3aNYwZMwaLFy/WaOeIiIiocMmcvJ7bW1ElKWNlZmamEl0mJCSgXr160NbO2D0tLQ3a2toYMGAAOnXqlCcdJSIiIirsJAVWy5cvz+NuEBER0ZeAQ4HqSQqsPD0987ofRERE9AXgT9qo99kXCAWApKQkpKSkqGwzNjbOVYeIiIiIvlQ5DqwSEhIwZcoU7N69G69fv85Snp6erpGOERERUeEjl8kgz+VQXm73L8xyvCpw8uTJOHXqFNatWweFQoHNmzdjzpw5sLW1xfbt2/Oij0RERFRIyGSauRVVOc5YHThwANu3b0ezZs3Qv39/NG7cGGXLloWDgwN27tyJPn365EU/iYiIiAq9HGesoqOjUbp0aQAZ86mio6MBAI0aNcK5c+c02zsiIiIqVDJXBeb2VlTlOGNVunRpPH78GKVKlUKFChWwe/du1K1bFwcOHBB/lJmoMLn410Os+ukkbt57ishXcdjhNwjtm1UTy+MTk+Gz5k8cOnsLb2ITUMrWAkN7NEX/ro0KsNckxdbdp3H60h2E/xMFhU4xVHVxwCivb+BoVwIAEPs2ERt2BuDyjQd4+W8MTE0M0Kx+JQz7rg0MDXQBAPcfvcC238/i5t1wxMQlwMbSDF2/qYde7vz7Fybnjl/ChRNBKtvMS5jh+0kZq9YP7zmB8AfPEB8Xj2IKHdg52KB5u0Yobmmuss/f1+4g6NwNRL96A4VCBxWqOqNt5xb5dh5FgSaG8opwXJXzwKp///64efMmmjZtiqlTp6Jjx45YvXo1UlNTsXTp0rzoY6Emk8nwxx9/oFOnTggPD4eTkxNu3LiB6tWrf1Z7mmiDVCUkJaOyc0n06Vgf/aZszlI+Y/lenL92Hxvm9EMpG3OcCrqHSX67YV3CBN80qVIAPSap/rr9GN3b10dFZ3ukp6djzfZjGDlzC35bNx56ujr493Uc/o2Ow9gB7VC6lBUiot7Ad80+/Ps6Dn4/fAcACHn4HOYmBvCZ0BNWJUzwd8hTzF+9F3K5HD07NijgM6T3FbeyQO8hXcT7cvl/gy42Ja1QuUYFGJsaISkxGecDLuOXzX9g+NT+Yr2gc38h6Nx1tGzfGLalrJGakoqY6Lh8Pw8q2nIcWI0bN078d6tWrXDv3j1cv34dZcuWRdWqVTXaudxydHTE2LFjMXbs2Hw5nr29PSIiIlC8eHFJ9b28vBATE4N9+/Z9dhv0aa0bVELrBpU+Wn7l78fwaF8PjWo5AwC8OjeE/x+B+OvOEwZWhdwqnwEq973HdUfrPvMQ8vAf1KxcGmUdrbHoh75iuZ2NBYb3a4OZi39FWno6tLW04N6mjkobdtYWuHXvCU5fus3AqpCRy2UwNDLItqxG/fdeq+ZA07au2LxsJ2LfxMHMwhTvEpNw9thFdPf6Fk7OpcSqljYl8rrbRQ5XBaqX4zlWH3JwcECXLl0KXVAlVXp6OpRKpUba0tLSgrW1tfhTPwXVBuVM3apOOHruFl5ExUAQBJy/dh9hT6PQvF6Fgu4a5VB8QhIAwNhQX20dA31daGtpqa2jrg0qGG9exWDl3E1Y++P/8OeuI4h9k322KSUlFTev3oWpuTGMTYwAAI8fPIUgCIiPi8eGxf5YNX8z9v50CHExb/PzFIqEgloV+Pz5c3z33XewsLCAnp4eqlSpovK7xYIgYNasWbCxsYGenh5atWqFBw8eqLQRHR2NPn36wNjYGKamphg4cCDi4+Nz+5CokBRYrVy5UvItJ5RKJfz8/FC2bFkoFAqUKlUK8+fPBwDcunULLVq0gJ6eHiwsLDBkyBCVk/fy8kKnTp2wePFi2NjYwMLCAiNGjEBqaioAoFmzZnjy5AnGjRunMlFu27ZtMDU1xf79+1GxYkUoFAo8ffoUV69eRevWrVG8eHGYmJigadOm+Ouvv1T6++DBAzRp0gS6urqoWLEiAgICVMrDw8Mhk8kQHBwsbrtz5w46dOgAY2NjGBkZoXHjxggLC4O3tzf8/f3x559/iv07c+ZMtm2cPXsWdevWhUKhgI2NDaZOnYq0tDSxvFmzZhg9ejQmT54Mc3NzWFtbw9vbO0d/i6/ZwondUN7JGpU7zIRVg7HoPmYd/CZ1R4OaZQu6a5QDSqUSSzYdRLWKDijraJ1tnZjYBGz+5RQ6t6370XZuhjzB8fN/o4uaOpT/SpayRoeebeAxqBPadm6BmDdx2LHuNyQn/XeR6usXb2LRjDVYPGMNwkLD0WtwF2hpZwTQMdGxEAQBgaeuonXHpujyXXskJSZh16a9SE/j9RdzoiAmr7958wYNGzZEsWLFcOTIEdy9exdLliyBmZmZWMfPzw8rV67E+vXrERQUBAMDA7i5uSEpKUms06dPH9y5cwcBAQE4ePAgzp07hyFDhmjssQEkDgUuW7ZMUmMymQyjR4+WfPBp06Zh06ZNWLZsGRo1aoSIiAjcu3cPCQkJcHNzg6urK65evYqoqCgMGjQII0eOxLZt28T9T58+DRsbG5w+fRoPHz5Ez549Ub16dQwePBh79+5FtWrVMGTIEAwePFjluImJiVi4cCE2b94MCwsLWFpa4tGjR/D09MSqVasgCAKWLFmCdu3a4cGDBzAyMoJSqUSXLl1gZWWFoKAgxMbGfnKI8fnz52jSpAmaNWuGU6dOwdjYGIGBgUhLS8PEiRMREhKCuLg4bN26FQBgbm6OFy9eZGmjXbt28PLywvbt23Hv3j0MHjwYurq6KsGTv78/xo8fj6CgIFy6dAleXl5o2LAhWrdunW3fkpOTkZycLN6Pi/t65xls3H0O126HY9eSIbC3NsfFGw8xedFvsC5hgmZ1mbX6Uixc9yfCnkRis9+wbMvjE5MwZs42lC5liaG9W2Vb52F4JCbM3Y7BvVqifs1yedldyqEyFZzEf1valIBtKWus8f0fQv6+j+p1KwMAKtWoACfnUoh/m4Cgs3/hj58Oo9/wHtAupg1BEKBMV6KNezOULucAAHDv/Q1Wzt2EJ2HPULq8Y0GcFkm0cOFC2Nvbi5+XAODk9N9zQhAELF++HDNmzIC7uzsAYPv27bCyssK+ffvg4eGBkJAQHD16FFevXkXt2rUBAKtWrUK7du2wePFi2NraaqSvkgKrx48fa+Rg73v79i1WrFiB1atXi79FWKZMGTRq1AibNm1CUlIStm/fDgODjPH01atXo2PHjli4cCGsrKwAAGZmZli9ejW0tLRQoUIFtG/fHidPnsTgwYNhbm4OLS0tGBkZwdpa9dtramoq1q5di2rV/lsZ1qKF6qqQjRs3wtTUFGfPnkWHDh1w4sQJ3Lt3D8eOHRMf/AULFuCbb7756DmuWbMGJiYm+OWXX1CsWDEAQLly/71Z6+npITk5OUv/3rd27VrY29tj9erVkMlkqFChAl68eIEpU6Zg1qxZ4qTMqlWrYvbs2QAAZ2dnrF69GidPnvxoYOXr64s5c+Z89Lhfi3dJKZi39gB2+A1Cm0b//+bsXBK37j/H6p9OMbD6Qixc9ycuXL2HjT8OhVVxkyzlCYnJGD3rfzDQU2DR9L7Q1s46DPjo6UsMn7EZndvWxSCPlvnRbcoFXT1dmBc3w5vXMe9tU0BXTwHzEmYoWcoGS2evQ+jth6hUo4I4N+v9VYIGhvrQM9BDLIcDc0SO3M8jytz/wy/1CoUCCoUiS/39+/fDzc0N3bt3x9mzZ1GyZEkMHz5cTJw8fvwYkZGRaNXqvy9NJiYmqFevHi5dugQPDw9cunQJpqamYlAFZMwVl8vlCAoKQufOnXN5Vqrnlu9CQkKQnJyMli2zvoGFhISgWrVqYlAFAA0bNoRSqURoaKi4rVKlStB6b56EjY0NoqKiPnlsHR2dLHPCXr58icGDB8PZ2RkmJiYwNjZGfHw8nj59KvbJ3t5eJaJ1dXVVe5zg4GA0btxYDKo+R0hICFxdXVXSpg0bNkR8fDz++ecfcduH5/Opx2LatGmIjY0Vb8+ePfvsPn7JUtPSkZqWDplcNS2tpSWHUhAKqFcklSAIWLjuT5y5dAfr5g9GSWvzLHXiE5MwcuYWaGtrYenMflDoZH09hj15ie9/2IT2LWpiRD+3/Og65VJKcgrevI756GR2AQIE/Pcza3aOGe/dr/99I9Z5l5iEdwnvYGLG37jNCU0OBdrb28PExES8+fr6ZnvMR48eYd26dXB2dsaxY8cwbNgwjB49Gv7+/gCAyMhIABATL5msrKzEssjISFhaWqqUa2trw9zcXKyjCQU2Q1pPTy/XbXwYsMhkMkkT0fX09LKM73p6euL169dYsWIFHBwcoFAo4OrqmuVHpnNCE+coVU4fi499KyiK4hOT8fiff8X7T168xq37/8DMWB921uZoWLMsZq/8E3oKHdhbmyHwxkP8evgK5o3RzLcXyjsL1/2Jo2eDsWRGP+jrK/DqTUbmwVBfF7qKYmJQlZScirkT+yL+XTLi32UMgZsZG0BLS46H4ZEYNn0T6tcshz6dG4ttaMllMDMxLLBzI1UnD55DWZfSMDEzQnxcAs4FXIZMLkfF6uXx5nUsQm6GwqmcA/QN9PA2Nh6XTl9DsWLa4hCiRQkzlKtUGgH7z6Jd15bQ0dXBmSOBsLA0g0MZuwI+u6/Xs2fPYGz8X2D7sc8lpVKJ2rVrY8GCBQCAGjVq4Pbt21i/fr046lVYFFhg5ezsDD09PZw8eRKDBg1SKXNxccG2bduQkJAgZq0CAwMhl8tRvnx5ycfQ0dGR/KPQgYGBWLt2Ldq1awcg44/96tUrlT49e/YMERERsLGxAQBcvnxZbZtVq1aFv78/UlNTs81aSemfi4sL9uzZA0EQxGAwMDAQRkZGsLPjm4EUwSFP8e2w/xZWzFj+BwCgV/u6WDO7LzbP6w+ftfsxdJY/3sQlwt7aDNO/78ALhH4Bfj+c8RocOm2jyvbZY7uhY6vauPfwOW6HZmRjOw1epFJn/5bJsLUyx8nAjAvDHjl9A0dO3xDLbSxNceB/U/P4DEiquNh4/LnrCN4lJkHfUA92jrbwGtkTBob6UKbH49njF7hyIRhJ75JgYKiPUk4l0W94Dxi8t7qzY083nDhwDru3Ziwasi9dEh4DO6uMfNCnyWSA/DNW9X3YBpDxCy7vB1YfY2Njg4oVK6psy/x8BCBOqXn58qX4GZ15P/OakNbW1llGctLS0hAdHa12Sk5OFVhgpauriylTpmDy5MnQ0dFBw4YN8e+//+LOnTvo06cPZs+eDU9PT3h7e+Pff//FqFGj0Ldv3yxpPnUcHR1x7tw5eHh4QKFQqL02lLOzM3bs2IHatWsjLi4OkyZNUsk4tWrVCuXKlYOnpycWLVqEuLg4TJ8+Xe3xR44ciVWrVsHDwwPTpk2DiYkJLl++jLp166J8+fJwdHTEsWPHEBoaCgsLC5iYZJ0bMnz4cCxfvhyjRo3CyJEjERoaitmzZ2P8+PEqF8ejj2tUyxnRV1Z9tNyquDHWzPouH3tEmnLt4I9qy2tXLfPJOkP7tMbQPtnPRaTCo3Ofdh8tMzIxRM+BnT7ZhkJXgfbdW6N9d/69c0OugcAqp/s3bNhQZSoQANy/fx8ODhkLEZycnGBtbY2TJ0+KgVRcXByCgoIwbFjGghZXV1fExMTg+vXrqFWrFgDg1KlTUCqVqFevXu5O6D0F+sk8c+ZMTJgwAbNmzYKLiwt69uyJqKgo6Ovr49ixY4iOjkadOnXQrVs3tGzZEqtXr85R+z4+PggPD0eZMmVQooT6i8Bt2bIFb968Qc2aNdG3b1+MHj1aZSxWLpfjjz/+wLt371C3bl0MGjRIvDTEx1hYWODUqVOIj49H06ZNUatWLWzatEnMXg0ePBjly5dH7dq1UaJECQQGBmZpo2TJkjh8+DCuXLmCatWq4fvvv8fAgQMxY8aMHD0WREREX6px48bh8uXLWLBgAR4+fIhdu3Zh48aNGDFiBICM6S9jx47FvHnzsH//fty6dQv9+vWDra0tOnXqBCAjw9W2bVsMHjwYV65cQWBgIEaOHAkPDw+NrQgEAJkg5HyG7vnz57FhwwaEhYXh999/R8mSJbFjxw44OTmhUSMOn3yJ4uLiYGJigshXMZLSsvRlexSVUNBdoHy0967mJuZS4ZWU8BYLOtdEbGxsnryPZ35OjPjlGhT6uZt/mJwYjzUetXPU14MHD2LatGl48OABnJycMH78eJXLKQmCgNmzZ2Pjxo2IiYlBo0aNsHbtWpXV+NHR0Rg5ciQOHDgAuVyOrl27YuXKlTA01Nx8yhwPBe7Zswd9+/ZFnz59cOPGDfFaSLGxsViwYAEOHz6ssc4RERFR4VIQQ4EA0KFDB3To0OGj5TKZDD4+PvDx8floHXNzc+zatSvnB8+BHA8Fzps3D+vXr1cZ0gIyxj8/vFI5ERER0dckxxmr0NBQNGnSJMt2ExMTxMTEaKJPREREVEh97m/9fdhGUZXjjJW1tTUePnyYZfuFCxdQunRpjXSKiIiICie5TKaRW1GV48Bq8ODBGDNmDIKCgiCTyfDixQvs3LkTEydOFJc0EhERUdEk19CtqMrxUODUqVOhVCrRsmVLJCYmokmTJlAoFJg4cSJGjRqVF30kIiIi+iLkOLCSyWSYPn06Jk2ahIcPHyI+Ph4VK1bU6FJFIiIiKpw4x0q9z77yuo6OTpbLyxMREVHRJkfu50jJUXQjqxwHVs2bN8/yA8bvO3XqVK46RERERPSlynFglfkbPJlSU1MRHByM27dvF7pfmCYiIiLN4lCgejkOrJYtW5btdm9vb8THx+e6Q0RERFR4FdSV178UGlvx+N133+F///ufppojIiIi+uJ89uT1D126dAm6urqaao6IiIgKIZkMuZ68zqHA93Tp0kXlviAIiIiIwLVr1zBz5kyNdYyIiIgKH86xUi/HgZWJiYnKfblcjvLly8PHxwdt2rTRWMeIiIiIvjQ5CqzS09PRv39/VKlSBWZmZnnVJyIiIiqkOHldvRxNXtfS0kKbNm0QExOTR90hIiKiwkymof+KqhyvCqxcuTIePXqUF30hIiKiQi4zY5XbW1GV48Bq3rx5mDhxIg4ePIiIiAjExcWp3IiIiIi+VpLnWPn4+GDChAlo164dAODbb79V+WkbQRAgk8mQnp6u+V4SERFRocA5VupJDqzmzJmD77//HqdPn87L/hAREVEhJpPJ1P5msNQ2iirJgZUgCACApk2b5llniIiIiL5kObrcQlGOMImIiOjTOBSoXo4Cq3Llyn0yuIqOjs5Vh4iIiKjw4pXX1ctRYDVnzpwsV14nIiIiogw5Cqw8PDxgaWmZV30hIiKiQk4uk+X6R5hzu39hJjmw4vwqIiIi4hwr9SRfIDRzVSARERERZU9yxkqpVOZlP4iIiOhLoIHJ60X4pwJzNseKiIiIvm5yyCDPZWSU2/0LMwZWREREJBkvt6Bejn+EmYiIiIiyx4wVERERScZVgeoxsCIiIiLJeB0r9TgUSERERKQhzFgRERGRZJy8rh4DKyIiIpJMDg0MBRbhyy1wKJCIiIhIQ5ixIiIiIsk4FKgeAysiIiKSTI7cD3cV5eGyonxuRERERPmKGSsiIiKSTCaTQZbLsbzc7l+YMbAiIiIiyWT/f8ttG0UVAysiIiKSjFdeV49zrIiIiIg0hBkrIiIiypGim2/KPQZWREREJBmvY6UehwKJiIiINIQZKyIiIpKMl1tQj4EVERERScYrr6tXlM+NiIiIKF8xY0VERESScShQPWasiIiISDKZhm6f68cff4RMJsPYsWPFbUlJSRgxYgQsLCxgaGiIrl274uXLlyr7PX36FO3bt4e+vj4sLS0xadIkpKWl5aIn2WNgRURERF+Eq1evYsOGDahatarK9nHjxuHAgQP47bffcPbsWbx48QJdunQRy9PT09G+fXukpKTg4sWL8Pf3x7Zt2zBr1iyN95FDgaRCEDJuVLQZ6fKl/zVZMHl5QXeB8oGQnpIvxymoocD4+Hj06dMHmzZtwrx588TtsbGx2LJlC3bt2oUWLVoAALZu3QoXFxdcvnwZ9evXx/Hjx3H37l2cOHECVlZWqF69OubOnYspU6bA29sbOjo6uTqf9zFjRURERJLJNXTLqREjRqB9+/Zo1aqVyvbr168jNTVVZXuFChVQqlQpXLp0CQBw6dIlVKlSBVZWVmIdNzc3xMXF4c6dO5/Rm4/j11YiIiKSTJMZq7i4OJXtCoUCCoUiS/1ffvkFf/31F65evZqlLDIyEjo6OjA1NVXZbmVlhcjISLHO+0FVZnlmmSYxY0VEREQFwt7eHiYmJuLN19c3S51nz55hzJgx2LlzJ3R1dQuglznDjBURERFJlttVfZltABlBk7Gxsbg9u2zV9evXERUVhZo1a4rb0tPTce7cOaxevRrHjh1DSkoKYmJiVLJWL1++hLW1NQDA2toaV65cUWk3c9VgZh1NYcaKiIiIJMv8Eebc3gDA2NhY5ZZdYNWyZUvcunULwcHB4q127dro06eP+O9ixYrh5MmT4j6hoaF4+vQpXF1dAQCurq64desWoqKixDoBAQEwNjZGxYoVNfr4MGNFREREhZaRkREqV66sss3AwAAWFhbi9oEDB2L8+PEwNzeHsbExRo0aBVdXV9SvXx8A0KZNG1SsWBF9+/aFn58fIiMjMWPGDIwYMSLbYC43GFgRERGRZHLIIM/lYGBu9//QsmXLIJfL0bVrVyQnJ8PNzQ1r164Vy7W0tHDw4EEMGzYMrq6uMDAwgKenJ3x8fDTaD4CBFREREeXA+0N5uWkjN86cOaNyX1dXF2vWrMGaNWs+uo+DgwMOHz6cuwNLwDlWRERERBrCjBURERFJJvv//3LbRlHFwIqIiIgkKwxDgYUZhwKJiIiINIQZKyIiIpJMpoFVgRwKJCIiIgKHAj+FgRURERFJxsBKPc6xIiIiItIQZqyIiIhIMl5uQT0GVkRERCSZXJZxy20bRRWHAomIiIg0hBkrIiIikoxDgeoxsCIiIiLJuCpQPQ4FEhEREWkIM1ZEREQkmQy5H8orwgkrBlZEREQkHVcFqsehQCIiIiINYcaKiIiIJOOqQPUYWBEREZFkXBWoHgMrIiIikkyG3E8+L8JxFedYEREREWkKM1ZEREQkmRwyyHM5licvwjkrBlZEREQkGYcC1eNQIBEREZGGMGNFRERE0jFlpRYDKyIiIpKM17FSj0OBRERERBrCjBURERFJp4ELhBbhhBUDKyIiIpKOU6zU41AgERERkYYwY0VERETSMWWlFgMrIiIikoyrAtVjYEVERESSyTQweT3Xk98LMc6xIiIiItIQZqyIiIhIMk6xUo+BFREREUnHyEotDgUSERERaQgzVkRERCQZVwWqx8CKiIiIJOOqQPU4FEhERESkIcxYERERkWScu64eAysiIiKSjpGVWhwKJCIiItIQZqyIiIhIMq4KVI+BFREREUnGVYHqMbAiIiIiyTjFSj3OsSIiIiLSEGasqMhbtu04Dp65iQdPXkJPUQx1qjhh9kh3ODtYZakrCAJ6jluHk5dCsN1vENo3rVYAPSaprv4dhi27z+D2g+f493Uc1szxQquGlbOtO2v57/j14GVMG/YtvLo2yVKekpKG7qNW4l7YC+xbPw4uZUvmdfdJjQY1ymBU31aoVqEUbEqYoM/EjTh89m+xvEPzaujfpRGqVygFc1MDNO7ji9v3n6u04ViyOOaO6Yz61UtDp5g2Tl4KwZTFv+Hf6LcAgIY1nXFww5hsj9/C0w837j7NuxP8kjFlpRYzVkWUo6Mjli9fXtDdKBQu3niIgd0a4/iWCdizcgTS0tLRbfQaJLxLzlJ3/S+ni/SkyqImMSkF5UvbYvaozmrrBVy4hZshT2FpYfzROn6bDqotp/ylr6fA7fvPMcnv12zLDXR1cPlmGLxX78t+f10d7F09AgIEuA9bhW8GLYNOMS38vHQoZP8/wefK349Qvu00lZv/vkCEP3/FoEoNmYb+K6qYsSokmjVrhurVqzMYygO/rRiucn/1rO9Qvu0PuHnvGRrUKCtuv3X/H6zZeRon/SehYrvp+d1N+gxN67qgaV0XtXVevorF3NX7sOXHwRg6fUu2dc5eCUHg9ftYNbsfzl25lxddpRw6cfEuTly8+9HyX49cBQDY25hnW16vWmmUsrFA0+8W4m1CEgBguPcOPD7lhyZ1yuHslVCkpqUj6vVbcR9tLTnaNamKjbvPavBM6GvDjNUXRBAEpKWlFXQ3vnhx8RlvsmbG+uK2xKQUDJnpD79J3WHFrEWRoVQqMenHXRjYoxmcHa2zrfPqzVvMXPo7/Kb0gq5CJ597SHlFoaMNQRCQnPLfe2ZSShqUSgH1q5XJdp9vmlSFuYkBdh24nF/d/CJlrgrM7a2oYmAlQbNmzTB69GhMnjwZ5ubmsLa2hre3t1geExODQYMGoUSJEjA2NkaLFi1w8+ZNsdzLywudOnVSaXPs2LFo1qyZWH727FmsWLECMpkMMpkM4eHhOHPmDGQyGY4cOYJatWpBoVDgwoULCAsLg7u7O6ysrGBoaIg6dergxIkT+fBIfPmUSiWmL9uDelVLw6WMrbh9xrK9qFvVCe2aVi3A3pGmbfrlNLS1tNCvc6NsywVBwFS/X+DRwRVVytvnc+8oL129FY7EpBR4j3KHnqIY9HV1MHdMZ2hra8G6ePZfnvq6u+LU5RC8iIrJ385+YWQauhVVDKwk8vf3h4GBAYKCguDn5wcfHx8EBAQAALp3746oqCgcOXIE169fR82aNdGyZUtER0dLanvFihVwdXXF4MGDERERgYiICNjb//cmP3XqVPz4448ICQlB1apVER8fj3bt2uHkyZO4ceMG2rZti44dO+LpU+lzApKTkxEXF6dy+xpMWvQbQh5FYNM8L3HbkXO3cP7afcwf17XgOkYad/v+P9j+xwX4Tuopzqn50I59F5CQmIyhvVrkc+8or72OiYfX1C1o27gy/jm3BE9OL4KJkR6CQ55CqRSy1Le1NEWL+i7Y8eelAugtfYqvry/q1KkDIyMjWFpaolOnTggNDVWpk5SUhBEjRsDCwgKGhobo2rUrXr58qVLn6dOnaN++PfT19WFpaYlJkyZpfCSIc6wkqlq1KmbPng0AcHZ2xurVq3Hy5Eno6enhypUriIqKgkKhAAAsXrwY+/btw++//44hQ4Z8sm0TExPo6OhAX18f1tZZhyt8fHzQunVr8b65uTmqVftvtdrcuXPxxx9/YP/+/Rg5cqSk8/H19cWcOXMk1S0qJi/ajeMXbuPghjEoaWUmbj9/7T4eP3+F0q0mq9T3mroFrtXLYP+67FcNUeF27dYjvI6JR/Pe88Vt6UolFm44gO17z+PUzum4fOMhgkOeoMo3U1X27Tp8BTq2rIGFU3rld7dJg04H3UPNznNgbmKAtHQl4uLf4d7RBQg/fj1L3d4d6yM6NgFHzv2dTUukogBWBZ49exYjRoxAnTp1kJaWhh9++AFt2rTB3bt3YWBgAAAYN24cDh06hN9++w0mJiYYOXIkunTpgsDAQABAeno62rdvD2tra1y8eBERERHo168fihUrhgULFuTyhP7DwEqiqlVVh4hsbGwQFRWFmzdvIj4+HhYWFirl7969Q1hYmEaOXbt2bZX78fHx8Pb2xqFDhxAREYG0tDS8e/cuRxmradOmYfz48eL9uLg4lSxZUSIIAqYs/g2Hzv6N/WtHw8G2uEr5GM/W6OvuqrKtUW9fzBvbBW0bZ790nwo/91a10KCms8q2gVM3wb1VLXRpWwcAMGNEJ4zt31Ysj3odh4FTN2HZjO9QzaVUvvaX8k50bAIAoHHtcihhZogj529lqdOnY338cvgK0tKV+d29L05B/KTN0aNHVe5v27YNlpaWuH79Opo0aYLY2Fhs2bIFu3btQosWGRnorVu3wsXFBZcvX0b9+vVx/Phx3L17FydOnICVlRWqV6+OuXPnYsqUKfD29oaOjmbmWDKwkqhYsWIq92UyGZRKJeLj42FjY4MzZ85k2cfU1BQAIJfLIQiqqefU1FTJx86MxjNNnDgRAQEBWLx4McqWLQs9PT1069YNKSkpkttUKBRihq2om7RoN/Ycu46fFg2GoYEuXr7OGPY0NtCFnq4OrCyMs52wbmdtliUIo8Il4V0ynj5/Jd7/JyIaIQ+fw8RIH7ZWZjAzUX3tFNPWQnFzI5S2twQA2L6XuQQylvgDQClbC1iXMM3bzpNaBno6cLIvId53sLVA5XIlERObiH9evoGpsT7srM1gU9wEAMTr0kW9jhNX+vXuWB/3H0fi1Zt41K3qBN/x3bD259N4+CRK5VhN6pSDY8ni2LHvYj6dHWX6cBqK1M+m2NhYABkjOABw/fp1pKamolWrVmKdChUqoFSpUrh06RLq16+PS5cuoUqVKrCy+u8ahm5ubhg2bBju3LmDGjVqaOKUGFjlVs2aNREZGQltbW04OjpmW6dEiRK4ffu2yrbg4GCVYE1HRwfp6emSjhkYGAgvLy907pxx7Z74+HiEh4d/Vv+/Blv3XAAAfDtspcr2VTP7oHeH+gXRJdKQ26HP0G/ievG+7/r9AIDObWrjx8keBdUt0oDqLg4qF+9cMD5jDuSug5cxYs5P+KZJFayd3Vcs/9+CAQCAHzcexsJNhwEAzg6WmDXiW5gZ6+Ppi2gs2XoMa3edynKsvt82QNDNMDx48jJLGWWlyd8K/HCkZPbs2SqLw7KjVCoxduxYNGzYEJUrZ4wqREZGQkdHR0xoZLKyskJkZKRY5/2gKrM8s0xTGFjlUqtWreDq6opOnTrBz88P5cqVw4sXL3Do0CF07twZtWvXRosWLbBo0SJs374drq6u+Omnn3D79m2V6NjR0RFBQUEIDw+HoaGhGIVnx9nZGXv37kXHjh0hk8kwc+ZMKJVMX3/M66BV+bIP5b961csi9MRiyfVP7VR/fTI7a/MctUd5J/CvBzCr8/E5oz8fDMLPB4PUtjFn9X7MWb3/k8caPHNbTrv3VdPkFKtnz57B2Pi/EQMp2aoRI0bg9u3buHDhQi57kTe4KjCXZDIZDh8+jCZNmqB///4oV64cPDw88OTJEzESdnNzw8yZMzF58mTUqVMHb9++Rb9+/VTamThxIrS0tFCxYkWUKFFC7XyppUuXwszMDA0aNEDHjh3h5uaGmjVr5ul5EhERAdDo9RaMjY1Vbp8KrEaOHImDBw/i9OnTsLOzE7dbW1sjJSUFMTExKvVfvnwpLgqztrbOskow8352C8c+l0z4cPIPfZXi4uJgYmKCiH9jVL49UNH06m3Wn/Ohoqt8q4kF3QXKB0J6CpJvbUJsbGyevI9nfk5cfxABQ6PctR//Ng61nG0k91UQBIwaNQp//PEHzpw5A2dn1YUpsbGxKFGiBH7++Wd07ZoxbBwaGooKFSqIc6yOHDmCDh06ICIiApaWGfMsN27ciEmTJqms7M8tDgUSERGRZAWxKnDEiBHYtWsX/vzzTxgZGYlzokxMTKCnpwcTExMMHDgQ48ePh7m5OYyNjTFq1Ci4urqifv2MubRt2rRBxYoV0bdvX/j5+SEyMhIzZszAiBEjNLqYi4EVERERSaeJn6TJ4f7r1q0DAPEXSzJt3boVXl5eAIBly5ZBLpeja9euSE5OhpubG9auXSvW1dLSwsGDBzFs2DC4urrCwMAAnp6e8PHxyc2ZZMHAioiIiAo1KbOWdHV1sWbNGqxZs+ajdRwcHHD48GFNdi0LBlZEREQkWQFceP2LwsCKiIiIpGNkpRYvt0BERESkIcxYERERkWQFsSrwS8LAioiIiCTT5E/aFEUcCiQiIiLSEGasiIiISDLOXVePgRURERFJx8hKLQZWREREJBknr6vHOVZEREREGsKMFREREUkmgwZWBWqkJ4UTAysiIiKSjFOs1ONQIBEREZGGMGNFREREkvECoeoxsCIiIqIc4GCgOhwKJCIiItIQZqyIiIhIMg4FqsfAioiIiCTjQKB6HAokIiIi0hBmrIiIiEgyDgWqx8CKiIiIJONvBarHwIqIiIik4yQrtTjHioiIiEhDmLEiIiIiyZiwUo+BFREREUnGyevqcSiQiIiISEOYsSIiIiLJuCpQPQZWREREJB0nWanFoUAiIiIiDWHGioiIiCRjwko9BlZEREQkGVcFqsehQCIiIiINYcaKiIiIciD3qwKL8mAgAysiIiKSjEOB6nEokIiIiEhDGFgRERERaQiHAomIiEgyDgWqx8CKiIiIJONP2qjHoUAiIiIiDWHGioiIiCTjUKB6DKyIiIhIMv6kjXocCiQiIiLSEGasiIiISDqmrNRiYEVERESScVWgehwKJCIiItIQZqyIiIhIMq4KVI+BFREREUnGKVbqMbAiIiIi6RhZqcU5VkREREQawowVERERScZVgeoxsCIiIiLJOHldPQZWBAAQBAEA8PZtXAH3hPLD2/jkgu4C5SMhPaWgu0D5IPPvnPl+nlfi4nL/OaGJNgorBlYEAHj79i0AoFzpUgXcEyIiyo23b9/CxMRE4+3q6OjA2toazk72GmnP2toaOjo6GmmrMJEJeR3a0hdBqVTixYsXMDIygqwo52g/EBcXB3t7ezx79gzGxsYF3R3KQ/xbfz2+1r+1IAh4+/YtbG1tIZfnzdq0pKQkpKRoJgOqo6MDXV1djbRVmDBjRQAAuVwOOzu7gu5GgTE2Nv6q3oC/Zvxbfz2+xr91XmSq3qerq1skgyFN4uUWiIiIiDSEgRURERGRhjCwoq+aQqHA7NmzoVAoCrorlMf4t/568G9NBYmT14mIiIg0hBkrIiIiIg1hYEVERESkIQysiIiIiDSEgRV9NbZt2wZTU1Pxvre3N6pXr15g/aHCQSaTYd++fQCA8PBwyGQyBAcHf3Z7mmiDCidHR0csX768oLtBhRwDK/rifG5A1LNnT9y/f1/zHdIAvmFnL78fF3t7e0RERKBy5cqS6nt5eaFTp065aoPyTrNmzTB27NiC7gZ9ZXjldfpq6OnpQU9Pr6C7QRqWnp4OmUymkZ/w0NLSgrW1dYG3QflHEASkp6dDW5sfh6QZzFhRvlMqlfD19YWTkxP09PRQrVo1/P777wCAM2fOQCaT4eTJk6hduzb09fXRoEEDhIaGAsgYzpszZw5u3rwJmUwGmUyGbdu2AQCWLl2KKlWqwMDAAPb29hg+fDji4+PF4344FPihzOzDggULYGVlBVNTU/j4+CAtLQ2TJk2Cubk57OzssHXrVpX9nj17hh49esDU1BTm5uZwd3dHeHh4lnYXL14MGxsbWFhYYMSIEUhNTQWQ8a36yZMnGDdunHhOXwqlUgk/Pz+ULVsWCoUCpUqVwvz58wEAt27dQosWLaCnpwcLCwsMGTJE5e/xuY9L5t9x//79qFixIhQKBZ4+fYqrV6+idevWKF68OExMTNC0aVP89ddfKv198OABmjRpAl1dXVSsWBEBAQEq5dkN4925cwcdOnSAsbExjIyM0LhxY4SFhcHb2xv+/v74888/xf6dOXMm2zbOnj2LunXrQqFQwMbGBlOnTkVaWppY3qxZM4wePRqTJ0+Gubk5rK2t4e3trYk/UaH1qXOOiYnBoEGDUKJECRgbG6NFixa4efOmWJ5dtnDs2LFo1qyZWH727FmsWLFC/PuEh4eL7zFHjhxBrVq1oFAocOHCBYSFhcHd3R1WVlYwNDREnTp1cOLEiXx4JKioYWBF+c7X1xfbt2/H+vXrcefOHYwbNw7fffcdzp49K9aZPn06lixZgmvXrkFbWxsDBgwAkDGcN2HCBFSqVAkRERGIiIhAz549AWT83uHKlStx584d+Pv749SpU5g8eXKO+nbq1Cm8ePEC586dw9KlSzF79mx06NABZmZmCAoKwvfff4+hQ4fin3/+AQCkpqbCzc0NRkZGOH/+PAIDA2FoaIi2bduq/FDp6dOnERYWhtOnT8Pf3x/btm0TA8K9e/fCzs4OPj4+4jl9KaZNm4Yff/wRM2fOxN27d7Fr1y5YWVkhISEBbm5uMDMzw9WrV/Hbb7/hxIkTGDlypMr+n/u4JCYmYuHChdi8eTPu3LkDS0tLvH37Fp6enrhw4QIuX74MZ2dntGvXDm/fvgWQEQR26dIFOjo6CAoKwvr16zFlyhS15/f8+XM0adIECoUCp06dwvXr1zFgwACkpaVh4sSJ6NGjB9q2bSv2r0GDBtm20a5dO9SpUwc3b97EunXrsGXLFsybN0+lnr+/PwwMDBAUFAQ/Pz/4+PhkCfyKGnXn3L17d0RFReHIkSO4fv06atasiZYtWyI6OlpS2ytWrICrqysGDx4s/n3s7e3F8qlTp+LHH39ESEgIqlativj4eLRr1w4nT57EjRs30LZtW3Ts2BFPnz7Nk3OnIkwgykdJSUmCvr6+cPHiRZXtAwcOFHr16iWcPn1aACCcOHFCLDt06JAAQHj37p0gCIIwe/ZsoVq1ap881m+//SZYWFiI97du3SqYmJiI9z9sx9PTU3BwcBDS09PFbeXLlxcaN24s3k9LSxMMDAyEn3/+WRAEQdixY4dQvnx5QalUinWSk5MFPT094dixYyrtpqWliXW6d+8u9OzZU7zv4OAgLFu27JPnVJjExcUJCoVC2LRpU5ayjRs3CmZmZkJ8fLy47dChQ4JcLhciIyMFQfj8x2Xr1q0CACE4OFht/9LT0wUjIyPhwIEDgiAIwrFjxwRtbW3h+fPnYp0jR44IAIQ//vhDEARBePz4sQBAuHHjhiAIgjBt2jTByclJSElJyfYYnp6egru7u8q2D9v44YcfsjxH1qxZIxgaGorPtaZNmwqNGjVSaadOnTrClClT1J7jl0zdOZ8/f14wNjYWkpKSVMrLlCkjbNiwQRCE7B/7MWPGCE2bNlU5xpgxY1TqZL7H7Nu375N9rFSpkrBq1Srx/pf4OqX8x0FlylcPHz5EYmIiWrdurbI9JSUFNWrUEO9XrVpV/LeNjQ0AICoqCqVKlfpo2ydOnICvry/u3buHuLg4pKWlISkpCYmJidDX15fUv0qVKqnM1bGyslKZhKylpQULCwtERUUBAG7evImHDx/CyMhIpZ2kpCSEhYWptKulpaVyTrdu3ZLUp8IqJCQEycnJaNmyZbZl1apVg4GBgbitYcOGUCqVCA0NhZWVFYDPf1x0dHRUniMA8PLlS8yYMQNnzpxBVFQU0tPTkZiYKGYcQkJCYG9vD1tbW3EfV1dXtccJDg5G48aNUaxYsU/26WNCQkLg6uqqMsTbsGFDxMfH459//hGf0x+ej42Njfg8K6o+ds43b95EfHw8LCwsVMrfvXun8rrKjdq1a6vcj4+Ph7e3Nw4dOoSIiAikpaXh3bt3zFhRjjGwonyVOcfm0KFDKFmypEqZQqEQ3zTf/yDL/EBSKpUfbTc8PBwdOnTAsGHDMH/+fJibm+PChQsYOHAgUlJSJAdWH36AymSybLdl9iU+Ph61atXCzp07s7RVokQJte2qO58vgSYWAnzu46Knp5dlLpqnpydev36NFStWwMHBAQqFAq6uripDsjmVn4sdiuJz5FM+ds7x8fGwsbHBmTNnsuyTOU9SLpdD+OAX2TLn50nxftAPABMnTkRAQAAWL16MsmXLQk9PD926dcvV84e+TgysKF+9P9m4adOmWcqlfBvV0dFBenq6yrbr169DqVRiyZIlYsZp9+7dmum0GjVr1sSvv/4KS0tLGBsbf3Y72Z1TYefs7Aw9PT2cPHkSgwYNUilzcXHBtm3bkJCQIH6ABQYGQi6Xo3z58pKPkZPHJTAwEGvXrkW7du0AZCwqePXqlUqfnj17hoiICDELevnyZbVtVq1aFf7+/khNTc02ayWlfy4uLtizZw8EQRCDwcDAQBgZGcHOzk7SuX1tatasicjISGhra8PR0THbOiVKlMDt27dVtgUHB6v8nXL6/PHy8kLnzp0BZHxpen8RCpFUnLxO+crIyAgTJ07EuHHj4O/vj7CwMPz1119YtWoV/P39JbXh6OiIx48fIzg4GK9evUJycjLKli2L1NRUrFq1Co8ePcKOHTuwfv36PD4boE+fPihevDjc3d1x/vx5PH78GGfOnMHo0aPFCe5SODo64ty5c3j+/LlKMFCY6erqYsqUKZg8eTK2b9+OsLAwXL58GVu2bEGfPn2gq6sLT09P3L59G6dPn8aoUaPQt29fcRhQipw8Ls7OztixYwdCQkIQFBSEPn36qGScWrVqhXLlysHT0xM3b97E+fPnMX36dLVtjhw5EnFxcfDw8MC1a9fw4MED7NixQ1yl6ujoiL///huhoaF49epVthmT4cOH49mzZxg1ahTu3buHP//8E7Nnz8b48eM1comIoqhVq1ZwdXVFp06dcPz4cYSHh+PixYuYPn06rl27BgBo0aIFrl27hu3bt+PBgweYPXt2lkDL0dERQUFBCA8Px6tXr9RmAJ2dnbF3714EBwfj5s2b6N27d5HPGFLe4Kua8t3cuXMxc+ZM+Pr6wsXFBW3btsWhQ4fg5OQkaf+uXbuibdu2aN68OUqUKIGff/4Z1apVw9KlS7Fw4UJUrlwZO3fuhK+vbx6fCaCvr49z586hVKlS6NKlC1xcXDBw4EAkJSXlKIPl4+OD8PBwlClTRmUIsbCbOXMmJkyYgFmzZsHFxQU9e/ZEVFQU9PX1cezYMURHR6NOnTro1q0bWrZsidWrV+eo/Zw8Llu2bMGbN29Qs2ZN9O3bF6NHj4alpaVYLpfL8ccff+Ddu3eoW7cuBg0aJF4a4mMsLCxw6tQpxMfHo2nTpqhVqxY2bdokZkUGDx6M8uXLo3bt2ihRogQCAwOztFGyZEkcPnwYV65cQbVq1fD9999j4MCBmDFjRo4ei6+JTCbD4cOH0aRJE/Tv3x/lypWDh4cHnjx5Igbmbm5umDlzJiZPnow6derg7du36Nevn0o7EydOhJaWFipWrIgSJUqonS+1dOlSmJmZoUGDBujYsSPc3NxQs2bNPD1PKppkwoeD1ERERET0WZixIiIiItIQBlZEREREGsLAioiIiEhDGFgRERERaQgDKyIiIiINYWBFREREpCEMrIiIiIg0hIEVERUaXl5e6NSpk3i/WbNmGDt2bL7348yZM5DJZIiJifloHZlMhn379klu09vbG9WrV89Vv8LDwyGTyRAcHJyrdogo7zCwIiK1vLy8IJPJIJPJoKOjg7Jly8LHxwdpaWl5fuy9e/di7ty5kupKCYaIiPIaf4SZiD6pbdu22Lp1K5KTk3H48GGMGDECxYoVw7Rp07LUTUlJgY6OjkaOa25urpF2iIjyCzNWRPRJCoUC1tbWcHBwwLBhw9CqVSvs378fwH/Dd/Pnz4etrS3Kly8PAHj27Bl69OgBU1NTmJubw93dHeHh4WKb6enpGD9+PExNTWFhYYHJkyfjw1/Y+nAoMDk5GVOmTIG9vT0UCgXKli2LLVu2IDw8HM2bNwcAmJmZQSaTwcvLCwCgVCrh6+sLJycn6OnpoVq1avj9999VjnP48GGUK1cOenp6aN68uUo/pZoyZQrKlSsHfX19lC5dGjNnzsz2R5k3bNgAe3t76Ovro0ePHoiNjVUp37x5M1xcXKCrq4sKFSpg7dq1Oe4LERUcBlZElGN6enpISUkR7588eRKhoaEICAjAwYMHkZqaCjc3NxgZGeH8+fMIDAyEoaEh2rZtK+63ZMkSbNu2Df/73/9w4cIFREdH448//lB73H79+uHnn3/GypUrERISgg0bNsDQ0BD29vbYs2cPACA0NBQRERFYsWIFAMDX1xfbt2/H+vXrcefOHYwbNw7fffcdzp49CyAjAOzSpQs6duyI4OBgDBo0CFOnTs3xY2JkZIRt27bh7t27WLFiBTZt2oRly5ap1Hn48CF2796NAwcO4OjRo7hx4waGDx8ulu/cuROzZs3C/PnzERISggULFmDmzJnw9/fPcX+IqIAIRERqeHp6Cu7u7oIgCIJSqRQCAgIEhUIhTJw4USy3srISkpOTxX127NghlC9fXlAqleK25ORkQU9PTzh27JggCIJgY2Mj+Pn5ieWpqamCnZ2deCxBEISmTZsKY8aMEQRBEEJDQwUAQkBAQLb9PH36tABAePPmjbgtKSlJ0NfXFy5evKhSd+DAgUKvXr0EQRCEadOmCRUrVlQpnzJlSpa2PgRA+OOPPz5avmjRIqFWrVri/dmzZwtaWlrCP//8I247cuSIIJfLhYiICEEQBKFMmTLCrl27VNqZO3eu4OrqKgiCIDx+/FgAINy4ceOjxyWigsU5VkT0SQcPHoShoSFSU1OhVCrRu3dveHt7i+VVqlRRmVd18+ZNPHz4EEZGRirtJCUlISwsDLGxsYiIiEC9evXEMm1tbdSuXTvLcGCm4OBgaGlpoWnTppL7/fDhQyQmJqJ169Yq21NSUlCjRg0AQEhIiEo/AMDV1VXyMTL9+uuvWLlyJcLCwhAfH4+0tDQYGxur1ClVqhRKliypchylUonQ0FAYGRkhLCwMAwcOxODBg8U6aWlpMDExyXF/iKhgMLAiok9q3rw51q1bBx0dHdja2kJbW/Wtw8DAQOV+fHw8atWqhZ07d2Zpq0SJEp/VBz09vRzvEx8fDwA4dOiQSkADZMwb05RLly6hT58+mDNnDtzc3GBiYoJffvkFS5YsyXFfN23alCXQ09LS0lhfiShvMbAiok8yMDBA2bJlJdevWbMmfv31V1haWmbJ2mSysbFBUFAQmjRpAiAjM3P9+nXUrFkz2/pVqlSBUqnE2bNn0apVqyzlmRmz9PR0cVvFihWhUCjw9OnTj2a6XFxcxIn4mS5fvvzpk3zPxYsX4eDggOnTp4vbnjx5kqXe06dP8eLFC9ja2orHkcvlKF++PKysrGBra4tHjx6hT58+OTo+ERUenLxORBrXp08fFC9eHO7u7jh//jweP36MM2fOYPTo0fjnn38AAGPGjMGPP/6Iffv24d69exg+fLjaa1A5OjrC09MTAwYMwL59+8Q2d+/eDQBwcHCATCbDwYMH8e+//yI+Ph5GRkaYOHEixo0bB39/f4SFheGvv/7CqlWrxAnh33//PR48eIBJkyYhNDQUu3btwrZt23J0vs7Oznj69Cl++eUXhIWFYeXKldlOxNfV1YWnpydu3ryJ8+fPY/To0ejRowesra0BAHPmzIGvry9WrlyJ+/fv49atW9i6dSuWLl2ao/4QUcFhYEVEGqevr49z586hVKlS6NKlC1xcXDBw4EAkJSWJGawJEyagb9++8PT0hKurK4yMjNC5c2e17a5btw7dunXD8OHDUaFCBQwePBgJCQkAgJIlS2LOnDmYOnUqrKysMHLkSADA3LlzMXPmTPj6+sLFxQVt27bFoUOH4OTkBCBj3tOePXuwb98+VKtWDevXr8eCBQtydL7ffvstxo0bh5EjR6J69eq4ePEiZs6cmaVe2bJl0aVLF7Rr1w5t2rRB1apVVS6nMGjQIGzevBlbt25FlSpV0LRpU2zbtk3sKxEVfjLhYzNFiYiIiChHmLEiIiIi0hAGVkREREQawsCKiIiISEMYWBERERFpCAMrIiIiIg1hYEVERESkIQysiIiIiDSEgRURERGRhjCwIiIiItIQBlZEREREGsLAioiIiEhDGFgRERERacj/ATdA8Oc/82mHAAAAAElFTkSuQmCC\n"},"metadata":{}}],"source":["# ===============================\n","# ✅ 1. BoW(TF-IDF) + ロジスティック回帰\n","# ===============================\n","from sklearn.feature_extraction.text import TfidfVectorizer\n","from sklearn.linear_model import LogisticRegression\n","from sklearn.pipeline import Pipeline\n","\n","pipe_bow = Pipeline([\n","    ('vect', TfidfVectorizer()),\n","    ('clf', LogisticRegression(max_iter=200))\n","])\n","\n","pipe_bow.fit(df_train[\"bow\"], df_train[\"label\"])\n","pred_bow = pipe_bow.predict(df_test[\"bow\"])\n","\n","print(\"=== BoW + LogisticRegression ===\")\n","print(classification_report(df_test[\"label\"],\n","                            pred_bow,\n","                            target_names=label2id.keys()))\n","plot_confusion(\"BoW + LogisticRegression\",\n","               df_test[\"label\"],\n","               pred_bow,\n","               labels=list(label2id.keys()))\n"]},{"cell_type":"code","execution_count":6,"id":"dc78a411","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":1000},"id":"dc78a411","executionInfo":{"status":"ok","timestamp":1747889798423,"user_tz":-540,"elapsed":312,"user":{"displayName":"TOMA Naruaki","userId":"11747312442870110137"}},"outputId":"d25a587d-0023-4711-cd13-479334bd3d93"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["                               sent1                         sent2  \\\n","0                大型の警察用のバイクが置かれています。           大型のオートバイが１台駐輪しています。   \n","1              文机の上にノートパソコンが置いてあります。          つくえのうえにパソコンがおかれています。   \n","2        スーツ姿の男性がスキー板を履いて雪の上に立っています。        スーツ姿の男性がスキー板をつけて立っている。   \n","3                 草原の木立の中にキリンが一頭います。            草原の木立の中にキリンが二頭います。   \n","4        路線バスの前を男性が乗ったリカンベントが走っています。  路線バスの後ろを男性が乗ったリカンベントが走っています。   \n","5               スタジアムで野球の試合が行われています。           スタジアムで陸上競技が行われています。   \n","6              車道脇の歩道に荷車を引く馬がたっています。       乗用車の横に、荷台を引く白い馬が立っています。   \n","7                     駅に機関車が停車しています。          蒸気機関車が駅のホームにとまっています。   \n","8  ふきんの上にさかさまに伏せられたたくさんのグラスが置かれています。         流し台にたくさんのグラスが置かれています。   \n","\n","   true_label  pred_label true_label_name pred_label_name  \n","0           0           2      entailment         neutral  \n","1           0           2      entailment         neutral  \n","2           0           2      entailment         neutral  \n","3           1           0   contradiction      entailment  \n","4           1           2   contradiction         neutral  \n","5           1           2   contradiction         neutral  \n","6           2           1         neutral   contradiction  \n","7           2           0         neutral      entailment  \n","8           2           1         neutral   contradiction  "],"text/html":["\n","  <div id=\"df-8974561d-33f0-44da-88d3-19783c7002b0\" class=\"colab-df-container\">\n","    <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>sent1</th>\n","      <th>sent2</th>\n","      <th>true_label</th>\n","      <th>pred_label</th>\n","      <th>true_label_name</th>\n","      <th>pred_label_name</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>大型の警察用のバイクが置かれています。</td>\n","      <td>大型のオートバイが１台駐輪しています。</td>\n","      <td>0</td>\n","      <td>2</td>\n","      <td>entailment</td>\n","      <td>neutral</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>文机の上にノートパソコンが置いてあります。</td>\n","      <td>つくえのうえにパソコンがおかれています。</td>\n","      <td>0</td>\n","      <td>2</td>\n","      <td>entailment</td>\n","      <td>neutral</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>スーツ姿の男性がスキー板を履いて雪の上に立っています。</td>\n","      <td>スーツ姿の男性がスキー板をつけて立っている。</td>\n","      <td>0</td>\n","      <td>2</td>\n","      <td>entailment</td>\n","      <td>neutral</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>草原の木立の中にキリンが一頭います。</td>\n","      <td>草原の木立の中にキリンが二頭います。</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>contradiction</td>\n","      <td>entailment</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>路線バスの前を男性が乗ったリカンベントが走っています。</td>\n","      <td>路線バスの後ろを男性が乗ったリカンベントが走っています。</td>\n","      <td>1</td>\n","      <td>2</td>\n","      <td>contradiction</td>\n","      <td>neutral</td>\n","    </tr>\n","    <tr>\n","      <th>5</th>\n","      <td>スタジアムで野球の試合が行われています。</td>\n","      <td>スタジアムで陸上競技が行われています。</td>\n","      <td>1</td>\n","      <td>2</td>\n","      <td>contradiction</td>\n","      <td>neutral</td>\n","    </tr>\n","    <tr>\n","      <th>6</th>\n","      <td>車道脇の歩道に荷車を引く馬がたっています。</td>\n","      <td>乗用車の横に、荷台を引く白い馬が立っています。</td>\n","      <td>2</td>\n","      <td>1</td>\n","      <td>neutral</td>\n","      <td>contradiction</td>\n","    </tr>\n","    <tr>\n","      <th>7</th>\n","      <td>駅に機関車が停車しています。</td>\n","      <td>蒸気機関車が駅のホームにとまっています。</td>\n","      <td>2</td>\n","      <td>0</td>\n","      <td>neutral</td>\n","      <td>entailment</td>\n","    </tr>\n","    <tr>\n","      <th>8</th>\n","      <td>ふきんの上にさかさまに伏せられたたくさんのグラスが置かれています。</td>\n","      <td>流し台にたくさんのグラスが置かれています。</td>\n","      <td>2</td>\n","      <td>1</td>\n","      <td>neutral</td>\n","      <td>contradiction</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","    <div class=\"colab-df-buttons\">\n","\n","  <div class=\"colab-df-container\">\n","    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-8974561d-33f0-44da-88d3-19783c7002b0')\"\n","            title=\"Convert this dataframe to an interactive table.\"\n","            style=\"display:none;\">\n","\n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n","    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n","  </svg>\n","    </button>\n","\n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    .colab-df-buttons div {\n","      margin-bottom: 4px;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","    <script>\n","      const buttonEl =\n","        document.querySelector('#df-8974561d-33f0-44da-88d3-19783c7002b0 button.colab-df-convert');\n","      buttonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","      async function convertToInteractive(key) {\n","        const element = document.querySelector('#df-8974561d-33f0-44da-88d3-19783c7002b0');\n","        const dataTable =\n","          await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                    [key], {});\n","        if (!dataTable) return;\n","\n","        const docLinkHtml = 'Like what you see? Visit the ' +\n","          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","          + ' to learn more about interactive tables.';\n","        element.innerHTML = '';\n","        dataTable['output_type'] = 'display_data';\n","        await google.colab.output.renderOutput(dataTable, element);\n","        const docLink = document.createElement('div');\n","        docLink.innerHTML = docLinkHtml;\n","        element.appendChild(docLink);\n","      }\n","    </script>\n","  </div>\n","\n","\n","    <div id=\"df-701fe9b3-f41b-47a2-9f07-3531f5a0c89c\">\n","      <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-701fe9b3-f41b-47a2-9f07-3531f5a0c89c')\"\n","                title=\"Suggest charts\"\n","                style=\"display:none;\">\n","\n","<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","     width=\"24px\">\n","    <g>\n","        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n","    </g>\n","</svg>\n","      </button>\n","\n","<style>\n","  .colab-df-quickchart {\n","      --bg-color: #E8F0FE;\n","      --fill-color: #1967D2;\n","      --hover-bg-color: #E2EBFA;\n","      --hover-fill-color: #174EA6;\n","      --disabled-fill-color: #AAA;\n","      --disabled-bg-color: #DDD;\n","  }\n","\n","  [theme=dark] .colab-df-quickchart {\n","      --bg-color: #3B4455;\n","      --fill-color: #D2E3FC;\n","      --hover-bg-color: #434B5C;\n","      --hover-fill-color: #FFFFFF;\n","      --disabled-bg-color: #3B4455;\n","      --disabled-fill-color: #666;\n","  }\n","\n","  .colab-df-quickchart {\n","    background-color: var(--bg-color);\n","    border: none;\n","    border-radius: 50%;\n","    cursor: pointer;\n","    display: none;\n","    fill: var(--fill-color);\n","    height: 32px;\n","    padding: 0;\n","    width: 32px;\n","  }\n","\n","  .colab-df-quickchart:hover {\n","    background-color: var(--hover-bg-color);\n","    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n","    fill: var(--button-hover-fill-color);\n","  }\n","\n","  .colab-df-quickchart-complete:disabled,\n","  .colab-df-quickchart-complete:disabled:hover {\n","    background-color: var(--disabled-bg-color);\n","    fill: var(--disabled-fill-color);\n","    box-shadow: none;\n","  }\n","\n","  .colab-df-spinner {\n","    border: 2px solid var(--fill-color);\n","    border-color: transparent;\n","    border-bottom-color: var(--fill-color);\n","    animation:\n","      spin 1s steps(1) infinite;\n","  }\n","\n","  @keyframes spin {\n","    0% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","      border-left-color: var(--fill-color);\n","    }\n","    20% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    30% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","      border-right-color: var(--fill-color);\n","    }\n","    40% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    60% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","    }\n","    80% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-bottom-color: var(--fill-color);\n","    }\n","    90% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","    }\n","  }\n","</style>\n","\n","      <script>\n","        async function quickchart(key) {\n","          const quickchartButtonEl =\n","            document.querySelector('#' + key + ' button');\n","          quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n","          quickchartButtonEl.classList.add('colab-df-spinner');\n","          try {\n","            const charts = await google.colab.kernel.invokeFunction(\n","                'suggestCharts', [key], {});\n","          } catch (error) {\n","            console.error('Error during call to suggestCharts:', error);\n","          }\n","          quickchartButtonEl.classList.remove('colab-df-spinner');\n","          quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n","        }\n","        (() => {\n","          let quickchartButtonEl =\n","            document.querySelector('#df-701fe9b3-f41b-47a2-9f07-3531f5a0c89c button');\n","          quickchartButtonEl.style.display =\n","            google.colab.kernel.accessAllowed ? 'block' : 'none';\n","        })();\n","      </script>\n","    </div>\n","\n","  <div id=\"id_0a60b741-6fb5-413e-9184-f9250c9a16b8\">\n","    <style>\n","      .colab-df-generate {\n","        background-color: #E8F0FE;\n","        border: none;\n","        border-radius: 50%;\n","        cursor: pointer;\n","        display: none;\n","        fill: #1967D2;\n","        height: 32px;\n","        padding: 0 0 0 0;\n","        width: 32px;\n","      }\n","\n","      .colab-df-generate:hover {\n","        background-color: #E2EBFA;\n","        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","        fill: #174EA6;\n","      }\n","\n","      [theme=dark] .colab-df-generate {\n","        background-color: #3B4455;\n","        fill: #D2E3FC;\n","      }\n","\n","      [theme=dark] .colab-df-generate:hover {\n","        background-color: #434B5C;\n","        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","        fill: #FFFFFF;\n","      }\n","    </style>\n","    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('mis_df')\"\n","            title=\"Generate code using this dataframe.\"\n","            style=\"display:none;\">\n","\n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","       width=\"24px\">\n","    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n","  </svg>\n","    </button>\n","    <script>\n","      (() => {\n","      const buttonEl =\n","        document.querySelector('#id_0a60b741-6fb5-413e-9184-f9250c9a16b8 button.colab-df-generate');\n","      buttonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","      buttonEl.onclick = () => {\n","        google.colab.notebook.generateWithVariable('mis_df');\n","      }\n","      })();\n","    </script>\n","  </div>\n","\n","    </div>\n","  </div>\n"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"dataframe","variable_name":"mis_df","summary":"{\n  \"name\": \"mis_df\",\n  \"rows\": 9,\n  \"fields\": [\n    {\n      \"column\": \"sent1\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 9,\n        \"samples\": [\n          \"\\u99c5\\u306b\\u6a5f\\u95a2\\u8eca\\u304c\\u505c\\u8eca\\u3057\\u3066\\u3044\\u307e\\u3059\\u3002\",\n          \"\\u6587\\u673a\\u306e\\u4e0a\\u306b\\u30ce\\u30fc\\u30c8\\u30d1\\u30bd\\u30b3\\u30f3\\u304c\\u7f6e\\u3044\\u3066\\u3042\\u308a\\u307e\\u3059\\u3002\",\n          \"\\u30b9\\u30bf\\u30b8\\u30a2\\u30e0\\u3067\\u91ce\\u7403\\u306e\\u8a66\\u5408\\u304c\\u884c\\u308f\\u308c\\u3066\\u3044\\u307e\\u3059\\u3002\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"sent2\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 9,\n        \"samples\": [\n          \"\\u84b8\\u6c17\\u6a5f\\u95a2\\u8eca\\u304c\\u99c5\\u306e\\u30db\\u30fc\\u30e0\\u306b\\u3068\\u307e\\u3063\\u3066\\u3044\\u307e\\u3059\\u3002\",\n          \"\\u3064\\u304f\\u3048\\u306e\\u3046\\u3048\\u306b\\u30d1\\u30bd\\u30b3\\u30f3\\u304c\\u304a\\u304b\\u308c\\u3066\\u3044\\u307e\\u3059\\u3002\",\n          \"\\u30b9\\u30bf\\u30b8\\u30a2\\u30e0\\u3067\\u9678\\u4e0a\\u7af6\\u6280\\u304c\\u884c\\u308f\\u308c\\u3066\\u3044\\u307e\\u3059\\u3002\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"true_label\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 2,\n        \"num_unique_values\": 3,\n        \"samples\": [\n          0,\n          1,\n          2\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"pred_label\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 2,\n        \"num_unique_values\": 3,\n        \"samples\": [\n          2,\n          0,\n          1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"true_label_name\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 3,\n        \"samples\": [\n          \"entailment\",\n          \"contradiction\",\n          \"neutral\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"pred_label_name\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 3,\n        \"samples\": [\n          \"neutral\",\n          \"entailment\",\n          \"contradiction\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"}},"metadata":{},"execution_count":6}],"source":["# 検証データに対する失敗事例分析\n","pred_bow = pipe_bow.predict(df_val[\"bow\"])\n","mis_df, mis_all_df = show_misclassified_examples(\n","    df_val[\"sent1\"],\n","    df_val[\"sent2\"],\n","    df_val[\"label\"],\n","    pred_bow,\n","    id2label,\n","    samples_per_class=3, # ラベル辺りの出力件数\n","    random_seed=2025     # Noneにすると毎回ランダムになる\n",")\n","\n","mis_df"]},{"cell_type":"code","source":["# 検証データに対する全失敗事例をExcelファイルとして出力\n","\n","true_label = df_val[\"label\"].map(id2label)\n","pred_label = pd.Series(pred_bow).map(id2label)\n","\n","mis_all_df = pd.DataFrame({\n","    \"sent1\":df_val[\"sent1\"],\n","    \"sent2\":df_val[\"sent2\"],\n","    \"true_label\":df_val[\"label\"],\n","    \"bow_pred_label\":pred_bow,\n","    \"true_label_name\":true_label,\n","    \"bow_pred_label_name\":pred_label\n","})\n","\n","mis_all_df.to_excel(\"mis_all_df_bow.xlsx\")"],"metadata":{"id":"lLT8Eu7MZ03l","executionInfo":{"status":"ok","timestamp":1747891980356,"user_tz":-540,"elapsed":912,"user":{"displayName":"TOMA Naruaki","userId":"11747312442870110137"}}},"id":"lLT8Eu7MZ03l","execution_count":19,"outputs":[]},{"cell_type":"code","source":[],"metadata":{"id":"rSZi7pS_Z1Kj"},"id":"rSZi7pS_Z1Kj","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"deep","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.9.6"},"colab":{"provenance":[]}},"nbformat":4,"nbformat_minor":5}