{"cells":[{"cell_type":"code","execution_count":null,"id":"a863de04","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"a863de04","executionInfo":{"status":"ok","timestamp":1747900168590,"user_tz":-540,"elapsed":222,"user":{"displayName":"TOMA Naruaki","userId":"11747312442870110137"}},"outputId":"4aa88dc8-5e77-461a-9b72-5a04252f7f51"},"outputs":[{"output_type":"stream","name":"stdout","text":["Python 3.11.12\n","Thu May 22 07:49:28 AM UTC 2025\n"]}],"source":["!python --version\n","!date"]},{"cell_type":"markdown","id":"71c33fbe","metadata":{"id":"71c33fbe"},"source":["# JGLUE（日本語言語理解ベンチマーク）"]},{"cell_type":"markdown","source":["## 注意\n","### GPUを指定する\n","LLMを用いた学習を行っている都合上、デフォルト（CPU）実行すると極めて時間がかかる。おそらく数時間要するだろう。自身でも動作確認したい人は、GPUを指定して実行することをお勧めする。\n","\n","GPUを指定するには以下の手順を取る。\n","\n","- 「ランタイム」から「ランタイムのタイプの変更」を選ぶ。\n","- 「ハードウェア アクセラレータ」からGPUを選ぶ。\n"," - T4 GPU を選ぶと良い。この中では低スペックだが十分早い。T4 GPUなら、1エポックの学習が約15分で終了する。\n","\n","### リソース使用制限\n","Google Colabは無料で利用できるが、利用度合いに応じてリソースが制限されることがある。特にGPUは使えなくなることが多いため、不必要に何度も実行することは避けよう。\n","\n","詳細: [最適な Colab のプランを選択する](https://colab.research.google.com/signup/pricing?hl=ja)"],"metadata":{"id":"tzL_s1MbsL0s"},"id":"tzL_s1MbsL0s"},{"cell_type":"markdown","id":"ef5bab7c","metadata":{"id":"ef5bab7c"},"source":["## 前提\n","- [report4_preprocessing.ipynb](./report4_preprocessing.ipynb)を実行すると前処理された3つのpklファイル（preprocessed_{train,val,test}.pkl）が作成される。この3つのファイルをPCにダウンロードしておき、本コードを実行する環境で参照できるようにアップロードすること。\n","- `utils.py` も別途アップロードすること。"]},{"cell_type":"markdown","id":"80d0d789","metadata":{"id":"80d0d789"},"source":["## 環境構築"]},{"cell_type":"code","execution_count":null,"id":"09f1eefb","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"09f1eefb","executionInfo":{"status":"ok","timestamp":1747900176622,"user_tz":-540,"elapsed":8030,"user":{"displayName":"TOMA Naruaki","userId":"11747312442870110137"}},"outputId":"13c7d117-a0a0-4804-b73f-5d6ac986f7ef"},"outputs":[{"output_type":"stream","name":"stdout","text":["Requirement already satisfied: fugashi[unidic-lite] in /usr/local/lib/python3.11/dist-packages (1.4.0)\n","Requirement already satisfied: unidic-lite in /usr/local/lib/python3.11/dist-packages (from fugashi[unidic-lite]) (1.0.8)\n","Requirement already satisfied: accelerate in /usr/local/lib/python3.11/dist-packages (1.7.0)\n","Requirement already satisfied: numpy<3.0.0,>=1.17 in /usr/local/lib/python3.11/dist-packages (from accelerate) (2.0.2)\n","Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.11/dist-packages (from accelerate) (24.2)\n","Requirement already satisfied: psutil in /usr/local/lib/python3.11/dist-packages (from accelerate) (5.9.5)\n","Requirement already satisfied: pyyaml in /usr/local/lib/python3.11/dist-packages (from accelerate) (6.0.2)\n","Requirement already satisfied: torch>=2.0.0 in /usr/local/lib/python3.11/dist-packages (from accelerate) (2.6.0+cu124)\n","Requirement already satisfied: huggingface-hub>=0.21.0 in /usr/local/lib/python3.11/dist-packages (from accelerate) (0.31.2)\n","Requirement already satisfied: safetensors>=0.4.3 in /usr/local/lib/python3.11/dist-packages (from accelerate) (0.5.3)\n","Requirement already satisfied: filelock in /usr/local/lib/python3.11/dist-packages (from huggingface-hub>=0.21.0->accelerate) (3.18.0)\n","Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.11/dist-packages (from huggingface-hub>=0.21.0->accelerate) (2025.3.0)\n","Requirement already satisfied: requests in /usr/local/lib/python3.11/dist-packages (from huggingface-hub>=0.21.0->accelerate) (2.32.3)\n","Requirement already satisfied: tqdm>=4.42.1 in /usr/local/lib/python3.11/dist-packages (from huggingface-hub>=0.21.0->accelerate) (4.67.1)\n","Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.11/dist-packages (from huggingface-hub>=0.21.0->accelerate) (4.13.2)\n","Requirement already satisfied: networkx in /usr/local/lib/python3.11/dist-packages (from torch>=2.0.0->accelerate) (3.4.2)\n","Requirement already satisfied: jinja2 in /usr/local/lib/python3.11/dist-packages (from torch>=2.0.0->accelerate) (3.1.6)\n","Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.4.127 in /usr/local/lib/python3.11/dist-packages (from torch>=2.0.0->accelerate) (12.4.127)\n","Requirement already satisfied: nvidia-cuda-runtime-cu12==12.4.127 in /usr/local/lib/python3.11/dist-packages (from torch>=2.0.0->accelerate) (12.4.127)\n","Requirement already satisfied: nvidia-cuda-cupti-cu12==12.4.127 in /usr/local/lib/python3.11/dist-packages (from torch>=2.0.0->accelerate) (12.4.127)\n","Requirement already satisfied: nvidia-cudnn-cu12==9.1.0.70 in /usr/local/lib/python3.11/dist-packages (from torch>=2.0.0->accelerate) (9.1.0.70)\n","Requirement already satisfied: nvidia-cublas-cu12==12.4.5.8 in /usr/local/lib/python3.11/dist-packages (from torch>=2.0.0->accelerate) (12.4.5.8)\n","Requirement already satisfied: nvidia-cufft-cu12==11.2.1.3 in /usr/local/lib/python3.11/dist-packages (from torch>=2.0.0->accelerate) (11.2.1.3)\n","Requirement already satisfied: nvidia-curand-cu12==10.3.5.147 in /usr/local/lib/python3.11/dist-packages (from torch>=2.0.0->accelerate) (10.3.5.147)\n","Requirement already satisfied: nvidia-cusolver-cu12==11.6.1.9 in /usr/local/lib/python3.11/dist-packages (from torch>=2.0.0->accelerate) (11.6.1.9)\n","Requirement already satisfied: nvidia-cusparse-cu12==12.3.1.170 in /usr/local/lib/python3.11/dist-packages (from torch>=2.0.0->accelerate) (12.3.1.170)\n","Requirement already satisfied: nvidia-cusparselt-cu12==0.6.2 in /usr/local/lib/python3.11/dist-packages (from torch>=2.0.0->accelerate) (0.6.2)\n","Requirement already satisfied: nvidia-nccl-cu12==2.21.5 in /usr/local/lib/python3.11/dist-packages (from torch>=2.0.0->accelerate) (2.21.5)\n","Requirement already satisfied: nvidia-nvtx-cu12==12.4.127 in /usr/local/lib/python3.11/dist-packages (from torch>=2.0.0->accelerate) (12.4.127)\n","Requirement already satisfied: nvidia-nvjitlink-cu12==12.4.127 in /usr/local/lib/python3.11/dist-packages (from torch>=2.0.0->accelerate) (12.4.127)\n","Requirement already satisfied: triton==3.2.0 in /usr/local/lib/python3.11/dist-packages (from torch>=2.0.0->accelerate) (3.2.0)\n","Requirement already satisfied: sympy==1.13.1 in /usr/local/lib/python3.11/dist-packages (from torch>=2.0.0->accelerate) (1.13.1)\n","Requirement already satisfied: mpmath<1.4,>=1.1.0 in /usr/local/lib/python3.11/dist-packages (from sympy==1.13.1->torch>=2.0.0->accelerate) (1.3.0)\n","Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.11/dist-packages (from jinja2->torch>=2.0.0->accelerate) (3.0.2)\n","Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.11/dist-packages (from requests->huggingface-hub>=0.21.0->accelerate) (3.4.2)\n","Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.11/dist-packages (from requests->huggingface-hub>=0.21.0->accelerate) (3.10)\n","Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.11/dist-packages (from requests->huggingface-hub>=0.21.0->accelerate) (2.4.0)\n","Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.11/dist-packages (from requests->huggingface-hub>=0.21.0->accelerate) (2025.4.26)\n","Requirement already satisfied: datasets in /usr/local/lib/python3.11/dist-packages (3.6.0)\n","Requirement already satisfied: filelock in /usr/local/lib/python3.11/dist-packages (from datasets) (3.18.0)\n","Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.11/dist-packages (from datasets) (2.0.2)\n","Requirement already satisfied: pyarrow>=15.0.0 in /usr/local/lib/python3.11/dist-packages (from datasets) (18.1.0)\n","Requirement already satisfied: dill<0.3.9,>=0.3.0 in /usr/local/lib/python3.11/dist-packages (from datasets) (0.3.7)\n","Requirement already satisfied: pandas in /usr/local/lib/python3.11/dist-packages (from datasets) (2.2.2)\n","Requirement already satisfied: requests>=2.32.2 in /usr/local/lib/python3.11/dist-packages (from datasets) (2.32.3)\n","Requirement already satisfied: tqdm>=4.66.3 in /usr/local/lib/python3.11/dist-packages (from datasets) (4.67.1)\n","Requirement already satisfied: xxhash in /usr/local/lib/python3.11/dist-packages (from datasets) (3.5.0)\n","Requirement already satisfied: multiprocess<0.70.17 in /usr/local/lib/python3.11/dist-packages (from datasets) (0.70.15)\n","Requirement already satisfied: fsspec<=2025.3.0,>=2023.1.0 in /usr/local/lib/python3.11/dist-packages (from fsspec[http]<=2025.3.0,>=2023.1.0->datasets) (2025.3.0)\n","Requirement already satisfied: huggingface-hub>=0.24.0 in /usr/local/lib/python3.11/dist-packages (from datasets) (0.31.2)\n","Requirement already satisfied: packaging in /usr/local/lib/python3.11/dist-packages (from datasets) (24.2)\n","Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.11/dist-packages (from datasets) (6.0.2)\n","Requirement already satisfied: aiohttp!=4.0.0a0,!=4.0.0a1 in /usr/local/lib/python3.11/dist-packages (from fsspec[http]<=2025.3.0,>=2023.1.0->datasets) (3.11.15)\n","Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.11/dist-packages (from huggingface-hub>=0.24.0->datasets) (4.13.2)\n","Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.11/dist-packages (from requests>=2.32.2->datasets) (3.4.2)\n","Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.11/dist-packages (from requests>=2.32.2->datasets) (3.10)\n","Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.11/dist-packages (from requests>=2.32.2->datasets) (2.4.0)\n","Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.11/dist-packages (from requests>=2.32.2->datasets) (2025.4.26)\n","Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.11/dist-packages (from pandas->datasets) (2.9.0.post0)\n","Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.11/dist-packages (from pandas->datasets) (2025.2)\n","Requirement already satisfied: tzdata>=2022.7 in /usr/local/lib/python3.11/dist-packages (from pandas->datasets) (2025.2)\n","Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /usr/local/lib/python3.11/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2025.3.0,>=2023.1.0->datasets) (2.6.1)\n","Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.11/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2025.3.0,>=2023.1.0->datasets) (1.3.2)\n","Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.11/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2025.3.0,>=2023.1.0->datasets) (25.3.0)\n","Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.11/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2025.3.0,>=2023.1.0->datasets) (1.6.0)\n","Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.11/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2025.3.0,>=2023.1.0->datasets) (6.4.3)\n","Requirement already satisfied: propcache>=0.2.0 in /usr/local/lib/python3.11/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2025.3.0,>=2023.1.0->datasets) (0.3.1)\n","Requirement already satisfied: yarl<2.0,>=1.17.0 in /usr/local/lib/python3.11/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2025.3.0,>=2023.1.0->datasets) (1.20.0)\n","Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.11/dist-packages (from python-dateutil>=2.8.2->pandas->datasets) (1.17.0)\n"]}],"source":["!pip install fugashi[unidic-lite]\n","!pip install accelerate -U\n","\n","# 2025年5月22日現在: datasetsが古く、numpy2系列とブッキングするようです。\n","!pip install --upgrade datasets"]},{"cell_type":"markdown","source":["## 前処理済みデータの読み込み"],"metadata":{"id":"Ssr8TNBBr5VB"},"id":"Ssr8TNBBr5VB"},{"cell_type":"code","execution_count":null,"id":"bd24e08f","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":459},"id":"bd24e08f","executionInfo":{"status":"ok","timestamp":1747900176919,"user_tz":-540,"elapsed":291,"user":{"displayName":"TOMA Naruaki","userId":"11747312442870110137"}},"outputId":"1b445e00-3615-4e81-d5db-6491edd656ba"},"outputs":[{"output_type":"execute_result","data":{"text/plain":[" sent1 sent2 \\\n","0 二人の男性がジャンボジェット機を見ています。 2人の男性が、白い飛行機を眺めています。 \n","1 2人の男性が、白い飛行機を眺めています。 二人の男性がジャンボジェット機を見ています。 \n","2 男性が子供を抱き上げて立っています。 坊主頭の男性が子供を抱いて立っています。 \n","3 坊主頭の男性が子供を抱いて立っています。 男性が子供を抱き上げて立っています。 \n","4 草地の上で牛と男性が立っています。 農場で白い牛が男性の隣に立っています。 \n","\n"," bow \\\n","0 二人 の 男性 が ジャンボジェット 機 を 見 て い ます 。 [SEP] 2人 の 男... \n","1 2人 の 男性 が 、 白い 飛行機 を 眺め て い ます 。 [SEP] 二人 の 男性... \n","2 男性 が 子供 を 抱き上げ て 立っ て い ます 。 [SEP] 坊主頭 の 男性 が ... \n","3 坊主頭 の 男性 が 子供 を 抱い て 立っ て い ます 。 [SEP] 男性 が 子供... \n","4 草地 の 上 で 牛 と 男性 が 立っ て い ます 。 [SEP] 農場 で 白い 牛 ... \n","\n"," w2v label \n","0 [0.050462592, -0.13604076, 0.010860913, -0.197... 2 \n","1 [0.088041924, -0.08648207, 0.014981923, -0.196... 2 \n","2 [0.056786165, -0.1304283, 0.013041455, -0.1172... 2 \n","3 [0.042833548, -0.12648883, 0.002992845, -0.119... 0 \n","4 [0.13754506, -0.074033, 0.013317642, -0.165398... 2 "],"text/html":["\n","
\n","
\n","\n","\n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n","
sent1sent2boww2vlabel
0二人の男性がジャンボジェット機を見ています。2人の男性が、白い飛行機を眺めています。二人 の 男性 が ジャンボジェット 機 を 見 て い ます 。 [SEP] 2人 の 男...[0.050462592, -0.13604076, 0.010860913, -0.197...2
12人の男性が、白い飛行機を眺めています。二人の男性がジャンボジェット機を見ています。2人 の 男性 が 、 白い 飛行機 を 眺め て い ます 。 [SEP] 二人 の 男性...[0.088041924, -0.08648207, 0.014981923, -0.196...2
2男性が子供を抱き上げて立っています。坊主頭の男性が子供を抱いて立っています。男性 が 子供 を 抱き上げ て 立っ て い ます 。 [SEP] 坊主頭 の 男性 が ...[0.056786165, -0.1304283, 0.013041455, -0.1172...2
3坊主頭の男性が子供を抱いて立っています。男性が子供を抱き上げて立っています。坊主頭 の 男性 が 子供 を 抱い て 立っ て い ます 。 [SEP] 男性 が 子供...[0.042833548, -0.12648883, 0.002992845, -0.119...0
4草地の上で牛と男性が立っています。農場で白い牛が男性の隣に立っています。草地 の 上 で 牛 と 男性 が 立っ て い ます 。 [SEP] 農場 で 白い 牛 ...[0.13754506, -0.074033, 0.013317642, -0.165398...2
\n","
\n","
\n","\n","
\n"," \n","\n"," \n","\n"," \n","
\n","\n","\n","
\n"," \n","\n","\n","\n"," \n","
\n","\n","
\n","
\n"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"dataframe","variable_name":"df_train","summary":"{\n \"name\": \"df_train\",\n \"rows\": 20073,\n \"fields\": [\n {\n \"column\": \"sent1\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 16983,\n \"samples\": [\n \"\\u7537\\u6027\\u304c\\u3001\\u30d5\\u30ea\\u30b9\\u30d3\\u30fc\\u3092\\u6295\\u3052\\u3088\\u3046\\u3068\\u3057\\u3066\\u3044\\u307e\\u3059\\u3002\",\n \"\\u767d\\u3044\\u98db\\u884c\\u6a5f\\u304c\\u6ed1\\u8d70\\u8def\\u306b\\u99d0\\u6a5f\\u3057\\u3066\\u3044\\u307e\\u3059\\u3002\",\n \"\\u30aa\\u30d5\\u30ed\\u30fc\\u30c9\\u30d0\\u30a4\\u30af\\u306b\\u4e8c\\u4eba\\u4e57\\u308a\\u3092\\u3057\\u3066\\u3044\\u307e\\u3059\\u3002\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"sent2\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 17009,\n \"samples\": [\n \"\\u30ea\\u30d5\\u30a9\\u30fc\\u30e0\\u3057\\u3066\\u30c8\\u30a4\\u30ec\\u306e\\u7a93\\u304b\\u3089\\u5149\\u304c\\u3055\\u3059\\u3088\\u3046\\u306b\\u3057\\u305f\\u3044\\u3002\",\n \"\\u30cd\\u30c3\\u30c8\\u306e\\u5411\\u3053\\u3046\\u5074\\u3067\\u3001\\u91ce\\u7403\\u306e\\u8a66\\u5408\\u304c\\u884c\\u308f\\u308c\\u3066\\u3044\\u307e\\u3059\\u3002\",\n \"\\u7537\\u6027\\u304c\\u30b8\\u30e3\\u30f3\\u30d7\\u3057\\u3066\\u30d5\\u30ea\\u30b9\\u30d4\\u30fc\\u3092\\u53d6\\u3063\\u3066\\u3044\\u307e\\u3059\\u3002\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"bow\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 19992,\n \"samples\": [\n \"\\u99ac \\u306b \\u4e57\\u3063 \\u305f \\u9a0e\\u624b \\u304c \\u969c\\u5bb3\\u7269 \\u3092 \\u98db\\u3073\\u8d8a\\u3048 \\u306a\\u304c\\u3089 \\u62f3\\u9283 \\u3092 \\u69cb\\u3048 \\u3066 \\u3044 \\u307e\\u3059 \\u3002 [SEP] \\u4eba \\u3092 \\u4e57\\u305b \\u305f \\u99ac \\u304c \\u969c\\u5bb3\\u7269 \\u3092 \\u30b8\\u30e3\\u30f3\\u30d7 \\u3057 \\u3066 \\u3044 \\u307e\\u3059 \\u3002\",\n \"\\u30b5\\u30d0\\u30f3\\u30ca \\u306b \\u4e00 \\u982d \\u306e \\u30ad\\u30ea\\u30f3 \\u304c \\u305f\\u3063 \\u3066 \\u3044 \\u307e\\u3059 \\u3002 [SEP] \\u67af\\u308c\\u8349 \\u3082 \\u76ee\\u7acb\\u3064 \\u8349\\u539f \\u306b \\u30ad\\u30ea\\u30f3 \\u304c \\u4e00 \\u982d \\u305f\\u3063 \\u3066 \\u3044 \\u307e\\u3059 \\u3002\",\n \"\\u9ed2 \\u3068 \\u30aa\\u30ec\\u30f3\\u30b8 \\u306b \\u5857\\u3089 \\u308c \\u305f \\u30d0\\u30a4\\u30af \\u304c \\u5c55\\u793a \\u3055 \\u308c \\u3066 \\u3044 \\u307e\\u3059 \\u3002 [SEP] \\u30e2\\u30fc\\u30bf\\u30fc\\u30b7\\u30e7\\u30fc \\u3067 \\u30d0\\u30a4\\u30af \\u304c \\u5c55\\u793a \\u3055 \\u308c \\u3066 \\u3044 \\u307e\\u3059 \\u3002\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"w2v\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"label\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0,\n \"min\": 0,\n \"max\": 2,\n \"num_unique_values\": 3,\n \"samples\": [\n 2,\n 0,\n 1\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}"}},"metadata":{},"execution_count":15}],"source":["from utils import *\n","\n","# 前処理済みデータ読み込み\n","df_train = get_data(\"preprocessed_train.pkl\")\n","df_val = get_data(\"preprocessed_val.pkl\")\n","df_test = get_data(\"preprocessed_test.pkl\")\n","\n","# 中身の確認\n","df_train.head()"]},{"cell_type":"code","execution_count":null,"id":"4d624db4","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"4d624db4","executionInfo":{"status":"ok","timestamp":1747900176927,"user_tz":-540,"elapsed":7,"user":{"displayName":"TOMA Naruaki","userId":"11747312442870110137"}},"outputId":"3bd13370-9e28-46dc-c63f-95d768863e6a"},"outputs":[{"output_type":"stream","name":"stdout","text":["train_labels: Counter({2: 11193, 1: 6004, 0: 2876})\n","val_labels: Counter({2: 1350, 1: 735, 0: 349})\n","test_labels: Counter({2: 1365, 1: 776, 0: 367})\n"]}],"source":["# データセット毎のラベル分布チェック\n","\n","import collections\n","print(\"train_labels: \", collections.Counter(df_train[\"label\"]))\n","print(\"val_labels: \", collections.Counter(df_val[\"label\"]))\n","print(\"test_labels: \", collections.Counter(df_test[\"label\"]))"]},{"cell_type":"markdown","id":"955058cc","metadata":{"id":"955058cc"},"source":["## case 3: BERT + classification\n","### データセットの準備\n","Transformerのデファクトスタンダードとなっている Huggingface Datasets 型としてデータを用意すると諸々の処理を楽に実行することができる。このため、pandasのデータフレーム型から Datasets 型に変換し直している。"]},{"cell_type":"code","execution_count":null,"id":"0292ddc3","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"0292ddc3","executionInfo":{"status":"ok","timestamp":1747900176981,"user_tz":-540,"elapsed":53,"user":{"displayName":"TOMA Naruaki","userId":"11747312442870110137"}},"outputId":"33b900f1-ea32-492c-8866-3f33d64a61f7"},"outputs":[{"output_type":"stream","name":"stdout","text":["train_ds[0]={'sentence1': '二人の男性がジャンボジェット機を見ています。', 'sentence2': '2人の男性が、白い飛行機を眺めています。', 'labels': 2}\n"]}],"source":["# ===============================\n","# ✅ ライブラリ\n","# ===============================\n","from transformers import AutoTokenizer, AutoModelForSequenceClassification, TrainingArguments, Trainer\n","from datasets import Dataset\n","import torch\n","\n","# ===============================\n","# ✅ Dataset構築\n","# ===============================\n","def create_dataset(sent1, sent2, labels):\n"," data = {\n"," \"sentence1\": sent1,\n"," \"sentence2\": sent2,\n"," \"labels\": labels\n"," }\n"," return Dataset.from_dict(data)\n","\n","train_ds = create_dataset(df_train[\"sent1\"], df_train[\"sent2\"], df_train[\"label\"])\n","val_ds = create_dataset(df_val[\"sent1\"], df_val[\"sent2\"], df_val[\"label\"])\n","test_ds = create_dataset(df_test[\"sent1\"], df_test[\"sent2\"], df_test[\"label\"])\n","\n","print(f\"{train_ds[0]=}\")"]},{"cell_type":"markdown","source":["## LLM用の前処理\n","\n","### LLM用にデータを整形\n","- トークン系列分用意するデータ（同じ長さで揃える必要がある）\n"," - `input_ids`: 入力文をトークナイズした結果（トークンのID列）。\n"," - `tokey_type_ids`: BERT系では2文の対で用意することを前提としている。その2文を区別するためのフィルタ設定。単一文の場合は全て0（今回はこれ）。2文ある場合、文2のトークンは1になる。\n"," - `attention_mask`: 一般的には「どのトークンに注意を向けるか」を0（無視する）,1（注意を向ける）で設定している。ここでは実トークンがある部分を1に、パディングで意味の無い部分を0として設定している。\n"," - `labels`: Trainerと連動した設定。train_dataset=train_enc と設定していると、train_enc内の\"labels\"を自動で参照し、損失を計算するようになっている。違う名目でラベルを設定したい場合には DataCollatorもしくはTrainerをカスタマイズする必要があります。\n","\n","### モデルの用意\n","- ここでは \"tohoku-nlp/bert-base-japanese-v3\" を利用。\n","- 長文だが、最初の出力「`Some weights of BertForSequenceClassification were not initialized from the model checkpoint at tohoku-nlp/bert-base-japanese-v3 and are newly initialized: ['classifier.bias', 'classifier.weight'] You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.`」は、このままでは利用できない（ので学習が必要だろう）ということを指摘している。この理由は、[AutoModelForSequenceClassification](https://huggingface.co/docs/transformers/main/en/model_doc/auto#transformers.AutoModelForSequenceClassification.from_pretrained)が「LLMの最後尾に新たな線形層（[torch.nn.Linear](https://pytorch.org/docs/stable/generated/torch.nn.Linear.html)）を追加しているためだ。この線形層はクラス数と同数のユニットを持つように設定されており（引数num_labelsで設定）、各ユニットに対するスコアを求めるようにモデルを拡張している。\n","- このようなにモデルを拡張しているということは、拡張した部分（Linear層のパラメータ）については重みがまっさらな状態である。このままでは当然推定できない（でたらめになる）ため、「You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.」と注意を促している。\n"],"metadata":{"id":"CMs0Zfy_thW_"},"id":"CMs0Zfy_thW_"},{"cell_type":"code","execution_count":null,"id":"31604dec","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":218,"referenced_widgets":["9cbda31570034704bf5d27805db6601c","7409761697934ddda9efedf4f662ba0a","5528a60527724dbeb3de8599a0390879","13a5f22eb6454b19aeda5991ad804df5","125429ee6cd24efd9fe53c1a9ae8a9f4","fec95af60e42488596aa97a2c3546cab","1951f0d6323e48efa5578905888c32e0","5940f885e5f94f30ac722eb5b0d877e3","6fc40462f2b8417c833169472e598590","a48d578797834f2b847ad288b7135e37","6a6fc4561b41403c94e5e688fc42827f","1846836b9f434c769bd8c0eee6e56bc3","a0ae15cc12e74645957be9706c906bf1","55ab9342b2bf46b497a7d20afd7170f3","9daf16e8f2a341019ce302b513995c35","cfb85f1c42024e6c8b56593486f22b8b","ac4687fa8a374c04bd2a1e79f9e7203d","9058799eaa0e41fda66a776134cdfb06","07f45f1b5ec14c2999d70d20d84fbfc5","3bbcf597ef244b4f991c572f6429c363","6c9c6b834a5348fb84c95718dbe83c6e","0fa4d46ba53f4b25a761ab5039c11535","84d927c94551447a875aa584c256a163","6550acf3cece4444a623e2288eb81169","11156708cbf84c2f82f339f58a394a9a","f50a59d98b8d4d6bbd409f3c9dfcab6d","272efb41f56d4dd3aacc4fe529e90698","c510aef81f47445c95f43219b5057d08","bf1edd12a5704b9c970a76ec5f4e505e","4ec025499441479b9c0a08b03d7eb3ef","20d17fd09d07498592386850daf97438","6ae28d0c6eeb4a37918ee4e1b355c872","aceca98d40e54ee98a9066e1cc10e545"]},"id":"31604dec","executionInfo":{"status":"ok","timestamp":1747900189507,"user_tz":-540,"elapsed":12529,"user":{"displayName":"TOMA Naruaki","userId":"11747312442870110137"}},"outputId":"0cfd8405-b620-4213-d626-e0ae41f91f91"},"outputs":[{"output_type":"stream","name":"stdout","text":["🔧 実行デバイス: cuda\n"]},{"output_type":"stream","name":"stderr","text":["Some weights of BertForSequenceClassification were not initialized from the model checkpoint at cl-tohoku/bert-base-japanese-v3 and are newly initialized: ['classifier.bias', 'classifier.weight']\n","You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"]},{"output_type":"display_data","data":{"text/plain":["Map: 0%| | 0/20073 [00:00\n","before: train_enc[0]={'sentence1': '二人の男性がジャンボジェット機を見ています。', 'sentence2': '2人の男性が、白い飛行機を眺めています。', 'labels': 2, 'input_ids': [2, 14269, 464, 13341, 430, 27914, 16753, 3107, 500, 5538, 456, 422, 12995, 385, 3, 33, 680, 464, 13341, 430, 384, 16517, 13208, 3107, 500, 29887, 456, 422, 12995, 385, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], 'token_type_ids': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}\n"]}],"source":["# GPUチェック（ない場合はCPU実行）\n","device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n","print(f\"🔧 実行デバイス: {device}\")\n","\n","# ===============================\n","# ✅ モデル・トークナイザーを用意\n","# ===============================\n","model_name = \"cl-tohoku/bert-base-japanese-v3\"\n","tokenizer = AutoTokenizer.from_pretrained(model_name)\n","model = AutoModelForSequenceClassification.from_pretrained(model_name, num_labels=3)\n","model.to(device) # GPU設定時にはGPU実行\n","\n","# ===============================\n","# ✅ トークン化\n","# ===============================\n","def tokenize_function(examples):\n"," return tokenizer(examples[\"sentence1\"], examples[\"sentence2\"],\n"," padding=\"max_length\", truncation=True, max_length=128)\n","\n","train_enc = train_ds.map(tokenize_function, batched=True)\n","val_enc = val_ds.map(tokenize_function, batched=True)\n","test_enc = test_ds.map(tokenize_function, batched=True)\n","\n","print(f\"before: {type(train_enc[0])=}\")\n","print(f\"before: {train_enc[0]=}\")\n","\n","# LLMへの入力形式に変換。\n","# - Hugging Face の Dataset オブジェクトを、PyTorch tensor 形式に変換\n","# 元データは dict 型。\n","# これを input_ids, attention_mask, labels を含む tensor に変換。\n","# input_ids: 入力文のトークンid系列。\n","#\n","train_enc.set_format(\"torch\", columns=[\"input_ids\", \"attention_mask\", \"labels\"])\n","val_enc.set_format(\"torch\", columns=[\"input_ids\", \"attention_mask\", \"labels\"])\n","test_enc.set_format(\"torch\", columns=[\"input_ids\", \"attention_mask\", \"labels\"])\n","\n","#print(f\"after: {train_enc[0]=}\")\n","#print(f\"after: {type(train_enc[0])=}\")\n","#print(f\"after: {train_enc.with_format('python')[0]=}\")"]},{"cell_type":"code","execution_count":null,"id":"20d14d01","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":35},"id":"20d14d01","executionInfo":{"status":"ok","timestamp":1747900189514,"user_tz":-540,"elapsed":6,"user":{"displayName":"TOMA Naruaki","userId":"11747312442870110137"}},"outputId":"0f9771c8-36b2-477b-96a2-8abd4dccd200"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["'[CLS] 二人 の 男性'"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"string"}},"metadata":{},"execution_count":19}],"source":["tokenizer.decode([2, 14269, 464, 13341])"]},{"cell_type":"markdown","source":["## 学習\n","ステップ毎に出力しているログは、学習データに対する損失(training loss)、検証データに対する損失(validation loss)。\n","\n","ログを確認すると、学習データに対する損失が途中で頭打ちになっているように見える。しかし検証データに対してはまだ若干改善し続けているようにも見えるため、過学習かどうかは判断しづらい。ここでは特別なことはせずに指定したエポック数の学習を続け、最終モデルを用いた検証を行うこととした。\n","\n","- TrainingArgumentsの補足\n"," - 設定可能な項目多数のため、詳細は[huggingfaceの公式ドキュメント](https://huggingface.co/docs/transformers/ja/main_classes/trainer)推奨。コード内では利用したパラメータだけ補足している。"],"metadata":{"id":"tKWDdn3cvIeT"},"id":"tKWDdn3cvIeT"},{"cell_type":"code","execution_count":null,"id":"8566a11b","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":603},"id":"8566a11b","executionInfo":{"status":"ok","timestamp":1747903134978,"user_tz":-540,"elapsed":644423,"user":{"displayName":"TOMA Naruaki","userId":"11747312442870110137"}},"outputId":"ee0a8e7e-616c-47c9-ea4f-750dcb08c4be"},"outputs":[{"output_type":"stream","name":"stderr","text":[":31: FutureWarning:\n","\n","`tokenizer` is deprecated and will be removed in version 5.0.0 for `Trainer.__init__`. Use `processing_class` instead.\n","\n"]},{"output_type":"display_data","data":{"text/plain":[""],"text/html":["\n","
\n"," \n"," \n"," [1255/1255 10:43, Epoch 1/1]\n","
\n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n","
StepTraining LossValidation Loss
1000.2635000.418176100
2000.2175000.563925200
3000.2256000.481815300
4000.1946000.515818400
5000.2050000.534839500
6000.1304000.520754600
7000.1812000.479634700
8000.1682000.476028800
9000.1401000.455427900
10000.1594000.4838431000
11000.1730000.4363041100
12000.2531000.4204741200

"]},"metadata":{}},{"output_type":"execute_result","data":{"text/plain":["TrainOutput(global_step=1255, training_loss=0.19688097509254973, metrics={'train_runtime': 643.8758, 'train_samples_per_second': 31.175, 'train_steps_per_second': 1.949, 'total_flos': 1320368908513536.0, 'train_loss': 0.19688097509254973, 'epoch': 1.0, 'step': 1255})"]},"metadata":{},"execution_count":33}],"source":["from transformers import TrainerCallback\n","\n","# ログ記録用\n","train_log_history = []\n","\n","class LoggingCallback(TrainerCallback):\n"," def on_log(self, args, state, control, logs=None, **kwargs):\n"," if logs is not None:\n"," logs[\"step\"] = state.global_step\n"," train_log_history.append(logs.copy())\n","\n","# ===============================\n","# ✅ 学習設定\n","# ===============================\n","training_args = TrainingArguments(\n"," output_dir=\"./results\", # 出力フォルダ\n"," per_device_train_batch_size=16, # 学習時のバッチサイズ\n"," per_device_eval_batch_size=32, # テスト時のバッチサイズ\n"," num_train_epochs=1, # エポック数\n"," eval_strategy=\"steps\", # eval_datasetによる評価タイミング（ステップ毎）\n"," eval_steps=100, # 評価タイミング\n"," logging_strategy=\"steps\", # ログ（loss, learning_rate, epoch等）生成タイミング（ステップ毎）\n"," logging_steps=100, # ログ生成ステップ数\n"," save_strategy=\"no\", # 学習中にcheckpointを保存しない\n"," report_to=\"none\" # wandb等への出力をしない\n",")\n","\n","# ===============================\n","# ✅ 学習 & 評価\n","# ===============================\n","trainer = Trainer(\n"," model=model,\n"," args=training_args,\n"," train_dataset=train_enc,\n","# train_dataset=test_enc, # 動作確認時のみ利用\n"," eval_dataset=val_enc,\n"," tokenizer=tokenizer,\n"," callbacks=[LoggingCallback()] # ログ用コールバック\n",")\n","\n","trainer.train()"]},{"cell_type":"code","source":["# train_log_historyへのログ保存方法確認\n","train_log_history[:5]"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"ZXGabU_iKDFa","executionInfo":{"status":"ok","timestamp":1747903942534,"user_tz":-540,"elapsed":6,"user":{"displayName":"TOMA Naruaki","userId":"11747312442870110137"}},"outputId":"41058627-86f6-402d-e878-2ca14c0bb89e"},"id":"ZXGabU_iKDFa","execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/plain":["[{'loss': 0.2635,\n"," 'grad_norm': 4.145387649536133,\n"," 'learning_rate': 4.605577689243028e-05,\n"," 'epoch': 0.0796812749003984,\n"," 'step': 100},\n"," {'eval_loss': 0.41817551851272583,\n"," 'eval_runtime': 16.6703,\n"," 'eval_samples_per_second': 146.008,\n"," 'eval_steps_per_second': 4.619,\n"," 'epoch': 0.0796812749003984,\n"," 'step': 100},\n"," {'loss': 0.2175,\n"," 'grad_norm': 11.141556739807129,\n"," 'learning_rate': 4.2071713147410365e-05,\n"," 'epoch': 0.1593625498007968,\n"," 'step': 200},\n"," {'eval_loss': 0.5639246106147766,\n"," 'eval_runtime': 16.9137,\n"," 'eval_samples_per_second': 143.907,\n"," 'eval_steps_per_second': 4.553,\n"," 'epoch': 0.1593625498007968,\n"," 'step': 200},\n"," {'loss': 0.2256,\n"," 'grad_norm': 19.625856399536133,\n"," 'learning_rate': 3.808764940239044e-05,\n"," 'epoch': 0.23904382470119523,\n"," 'step': 300}]"]},"metadata":{},"execution_count":44}]},{"cell_type":"code","source":["# 学習曲線の描画\n","import pandas as pd\n","import plotly.graph_objects as go\n","\n","# ログ履歴（train_log_history）は Trainer実行時に記録されたものを前提\n","log_df = pd.DataFrame(train_log_history)\n","\n","# train log と eval log が混在しているため、分割\n","log_train_df = log_df[log_df[\"loss\"].notna()]\n","log_eval_df = log_df[log_df[\"eval_loss\"].notna()]\n","\n","# 描画オブジェクト\n","fig = go.Figure()\n","\n","# Train Loss\n","if \"loss\" in log_df.columns:\n"," fig.add_trace(go.Scatter(x=log_train_df[\"step\"], y=log_train_df[\"loss\"],\n"," mode='lines+markers', name='Train Loss'))\n","\n","# Eval Loss\n","if \"eval_loss\" in log_df.columns:\n"," fig.add_trace(go.Scatter(x=log_eval_df[\"step\"], y=log_eval_df[\"eval_loss\"],\n"," mode='lines+markers', name='Eval Loss'))\n","\n","# Eval Accuracy\n","if \"eval_accuracy\" in log_df.columns:\n"," fig.add_trace(go.Scatter(x=log_eval_df[\"step\"], y=log_eval_df[\"eval_accuracy\"],\n"," mode='lines+markers', name='Eval Accuracy'))\n","\n","# レイアウト\n","fig.update_layout(\n"," title=\"Training & Evaluation Metrics\",\n"," xaxis_title=\"Training Step\",\n"," yaxis_title=\"Metric Value\",\n"," legend_title=\"Metrics\",\n"," template=\"plotly_white\"\n",")\n","\n","fig.show()\n"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":542},"id":"PvtQ4_mxLkgj","executionInfo":{"status":"ok","timestamp":1747903798857,"user_tz":-540,"elapsed":13,"user":{"displayName":"TOMA Naruaki","userId":"11747312442870110137"}},"outputId":"90564e03-0843-407f-b905-8a7f67a94c31"},"id":"PvtQ4_mxLkgj","execution_count":null,"outputs":[{"output_type":"display_data","data":{"text/html":["\n","\n","\n","

\n","
\n","\n",""]},"metadata":{}}]},{"cell_type":"code","execution_count":null,"id":"30d38a1c","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":659},"id":"30d38a1c","executionInfo":{"status":"ok","timestamp":1747903152485,"user_tz":-540,"elapsed":17467,"user":{"displayName":"TOMA Naruaki","userId":"11747312442870110137"}},"outputId":"a48622fd-879f-47fc-b246-331eb2640fe9"},"outputs":[{"output_type":"display_data","data":{"text/plain":[""],"text/html":[]},"metadata":{}},{"output_type":"stream","name":"stdout","text":["=== BERT Fine-tuned ===\n"," precision recall f1-score support\n","\n"," entailment 0.71 0.72 0.72 367\n","contradiction 0.93 0.88 0.90 776\n"," neutral 0.89 0.91 0.90 1365\n","\n"," accuracy 0.87 2508\n"," macro avg 0.84 0.84 0.84 2508\n"," weighted avg 0.88 0.87 0.87 2508\n","\n"]},{"output_type":"display_data","data":{"text/plain":["
"],"image/png":"iVBORw0KGgoAAAANSUhEUgAAAlYAAAHHCAYAAAB9dxZkAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAcA1JREFUeJzt3XdYFFfbBvB7F2TpVaogYENs2H2xYUGxRqPGkJCIxhZjib3EhlhI7NhjiaDRRE3U2BVFRVFRUYwFURGsFBsiInXn+4OPiSuwLrLSvH+55rqyZ86ceWYX2MdzzpyRCIIggIiIiIiKTFrSARARERGVF0ysiIiIiNSEiRURERGRmjCxIiIiIlITJlZEREREasLEioiIiEhNmFgRERERqQkTKyIiIiI1YWJFREREpCZMrIhUdPv2bXTs2BFGRkaQSCTYvXu3WtuPjY2FRCJBQECAWtsty9q0aYM2bdqotc0HDx5AW1sboaGham23NPsY72N55eDggP79+4uvDx06BH19fTx58qTkgqIyhYkVlSnR0dEYOnQoqlSpAm1tbRgaGqJFixbw9/fHmzdvPuq5vb29cfXqVcydOxebN29G48aNP+r5ilP//v0hkUhgaGiY7/t4+/ZtSCQSSCQSLFy4sNDtP378GD4+PoiIiFBDtEXj6+uLZs2aoUWLFmJZ7vXnbpqamrCzs4Onpydu3LihcPyJEycU6r67/fnnn2JdBwcHhX16enpo2rQpNm3aBOC/ZFqVLTY2tsBrKugYKysr9b55H+DAgQPw8fEp6TA+WKdOnVCtWjX4+fmVdChURmiWdABEqtq/fz+++OILyGQy9OvXD3Xq1EFGRgZOnz6NCRMm4Pr161i7du1HOfebN29w9uxZTJ06FSNGjPgo57C3t8ebN29QoUKFj9L++2hqaiI1NRV79+5F3759FfZt2bIF2traSEtL+6C2Hz9+jFmzZsHBwQH169dX+bgjR4580PkK8uTJEwQGBiIwMDDPPplMhvXr1wMAsrKyEB0djTVr1uDQoUO4ceMGbGxsFOqPGjUKTZo0ydOOq6urwuv69etj3LhxAIC4uDisX78e3t7eSE9Px9dff43Nmzcr1F+0aBEePnyIJUuWKJSbm5srvbYOHTqgX79+CmU6OjoA1P8+FsaBAwewcuXKMp1cDR06FOPHj8esWbNgYGBQ0uFQKcfEisqEmJgYeHp6wt7eHsHBwbC2thb3DR8+HHfu3MH+/fs/2vlzhwGMjY0/2jkkEgm0tbU/WvvvI5PJ0KJFC/zxxx95EqutW7eia9eu+Pvvv4slltTUVOjq6kJLS0ut7f7+++/Q1NRE9+7d8+zT1NTEN998o1D2v//9D926dcP+/fsxePBghX2tWrVCnz593nvOSpUqKbTbv39/VKlSBUuWLMHgwYPznPPPP//Eixcv8pS/T40aNQo8Rt3v46emd+/eGDlyJHbs2IHvvvuupMOhUo5DgVQmzJ8/HykpKdiwYYNCUpWrWrVq+PHHH8XXWVlZmD17NqpWrQqZTAYHBwf89NNPSE9PVzjOwcEB3bp1w+nTp9G0aVNoa2ujSpUq4lANAPj4+MDe3h4AMGHCBEgkEjg4OADI+ZLM/f+3+fj4QCKRKJQFBQWhZcuWMDY2hr6+PpycnPDTTz+J+wuaYxUcHIxWrVpBT08PxsbG6NGjByIjI/M93507d9C/f38YGxvDyMgIAwYMQGpqasFv7Du+/vprHDx4EElJSWLZhQsXcPv2bXz99dd56j9//hzjx49H3bp1oa+vD0NDQ3Tu3BlXrlwR65w4cULs2RkwYIA4TJV7nW3atEGdOnUQHh6O1q1bQ1dXV3xf3p0b5O3tDW1t7TzX7+HhARMTEzx+/Fjp9e3evRvNmjWDvr6+Su9H7lCapqb6/g1qbm6OmjVrIjo6Wm1tvs+772PucOb27dsxd+5c2NraQltbG+3bt8edO3fyHB8WFoZOnTrByMgIurq6cHNzU2mOWv/+/bFy5UoAisOVb8dw4sQJhWPy+z3o378/9PX18ejRI/Ts2RP6+vowNzfH+PHjkZ2drXC8XC7H0qVLUbt2bWhra8PS0hJDhw7FixcvFOoJgoA5c+bA1tYWurq6aNu2La5fv57vdVhYWKBevXr4559/3nvNREysqEzYu3cvqlSpgubNm6tUf9CgQZgxYwYaNmyIJUuWwM3NDX5+fvD09MxT986dO+jTpw86dOiARYsWwcTEBP379xf/yPbq1Usclvnqq6+wefNmLF26tFDxX79+Hd26dUN6ejp8fX2xaNEifPbZZ+/9cjp69Cg8PDyQmJgIHx8fjB07FmfOnEGLFi3ynXPTt29fvHr1Cn5+fujbty8CAgIwa9YslePs1asXJBIJdu7cKZZt3boVNWvWRMOGDfPUv3v3Lnbv3o1u3bph8eLFmDBhAq5evQo3NzcxyXF2doavry8AYMiQIdi8eTM2b96M1q1bi+08e/YMnTt3Rv369bF06VK0bds23/j8/f1hbm4Ob29v8Qv1119/xZEjR7B8+fI8w3Vvy8zMxIULF/K9jlxPnz7F06dPkZCQgLNnz2LMmDEwMzNDt27d8tR99eqVWP/tTRCEAtsHcpL+hw8fwsTERGm9wkpLS8sTy7v/kHjXzz//jF27dmH8+PGYMmUKzp07By8vL4U6wcHBaN26NZKTkzFz5kzMmzcPSUlJaNeuHc6fP6+0/aFDh6JDhw4AIH7u7w59qio7OxseHh4wMzPDwoUL4ebmhkWLFuUZ/h86dCgmTJggzr0cMGAAtmzZAg8PD2RmZor1ZsyYgenTp8PFxQULFixAlSpV0LFjR7x+/Trf8zdq1Ahnzpz5oNjpEyMQlXIvX74UAAg9evRQqX5ERIQAQBg0aJBC+fjx4wUAQnBwsFhmb28vABBCQkLEssTEREEmkwnjxo0Ty2JiYgQAwoIFCxTa9Pb2Fuzt7fPEMHPmTOHtX68lS5YIAIQnT54UGHfuOTZu3CiW1a9fX7CwsBCePXsmll25ckWQSqVCv3798pzvu+++U2jz888/F8zMzAo859vXoaenJwiCIPTp00do3769IAiCkJ2dLVhZWQmzZs3K9z1IS0sTsrOz81yHTCYTfH19xbILFy7kubZcbm5uAgBhzZo1+e5zc3NTKDt8+LAAQJgzZ45w9+5dQV9fX+jZs+d7r/HOnTsCAGH58uX5Xj+APFulSpWE8PBwhbrHjx/Pt27uFhcXJ9a1t7cXOnbsKDx58kR48uSJcPXqVeHbb78VAAjDhw/PN86uXbvm+zOlTEGx5L7f776Pudfg7OwspKeni+X+/v4CAOHq1auCIAiCXC4XqlevLnh4eAhyuVysl5qaKjg6OgodOnR4b2zDhw9X+F14N4bjx48rlOf3e5D7+bz9MyUIgtCgQQOhUaNG4utTp04JAIQtW7Yo1Dt06JBCeWJioqClpSV07dpV4bp++uknAYDg7e2dJ9558+YJAISEhIT3XjN92thjRaVecnIyAKg8afTAgQMAgLFjxyqU504gfncuVq1atdCqVSvxtbm5OZycnHD37t0PjvlduXOz/vnnH8jlcpWOiYuLQ0REBPr37w9TU1OxvF69eujQoYN4nW/7/vvvFV63atUKz549E99DVXz99dc4ceIE4uPjERwcjPj4+HyHAYGceVlSac6fkezsbDx79kwc5rx06ZLK55TJZBgwYIBKdTt27IihQ4fC19cXvXr1gra2Nn799df3Hvfs2TMAKLCnSFtbG0FBQQgKCsLhw4fx66+/Ql9fH126dMGtW7fy1J8xY4ZY/+3t7c8KyJk4bm5uDnNzc9StWxebN2/GgAEDsGDBApWuV1U9evTIE4uHh4fSYwYMGKAw/yr39yD3Zz8iIkIcBn727JnYE/b69Wu0b98eISEhKv88q0N+P99v/57u2LEDRkZG6NChg0LPXaNGjaCvr4/jx48DyOkJzsjIwMiRIxWG7EePHl3guXN/bp4+farGK6LyiJPXqdQzNDQEkDP0oop79+5BKpWiWrVqCuVWVlYwNjbGvXv3FMorV66cpw0TE5M8czKK4ssvv8T69esxaNAgTJ48Ge3bt0evXr3Qp08fMTHJ7zoAwMnJKc8+Z2dnHD58GK9fv4aenp5Y/u615H4ZvHjxQnwf36dLly4wMDDAtm3bEBERgSZNmqBatWr5Dj3K5XL4+/tj1apViImJUZjvYmZmptL5gJwJ3oWZYL1w4UL8888/iIiIwNatW2FhYaHysUIBQ3UaGhpwd3dXKOvSpQuqV6+OKVOm5Jm4X7du3Tz189OsWTPMmTMH2dnZuHbtGubMmYMXL16ofUK5ra2tSvG8TdnPC5CzzAaQM7etIC9fvoSenh6eP3+uUG5ubg4NDY1CxaOMtrZ2njsj3/09vX37Nl6+fFngz0NiYiKA/363qlevnifmghLv3J+bd+dOEr2LiRWVeoaGhrCxscG1a9cKdZyqfwAL+uNf0BewKud4d0Ktjo4OQkJCcPz4cezfvx+HDh3Ctm3b0K5dOxw5ckRtX0BFuZZcMpkMvXr1QmBgIO7evav0Nvl58+Zh+vTp+O677zB79myYmppCKpVi9OjRherJyF0WQFWXL18WvySvXr2Kr7766r3H5CZ6hUmYbW1t4eTkhJCQkELF97aKFSuKCY+Hhwdq1qyJbt26wd/fP0+vanF7389L7me4YMGCApfJ0NfXR2hoaJ55cTExMfne2JFL1d+d98X6NrlcDgsLC2zZsiXf/e9bskKZ3J+bihUrfnAb9GlgYkVlQrdu3bB27VqcPXs2zzpB77K3t4dcLsft27fh7OwslickJCApKUm8w08dTExMFO6gy/VurxgASKVStG/fHu3bt8fixYsxb948TJ06FcePH8+3pyE3zqioqDz7bt68iYoVKyr0VqnT119/jd9++w1SqTTfCf+5/vrrL7Rt2xYbNmxQKE9KSlL4AlLnv/Jfv36NAQMGoFatWmjevDnmz5+Pzz//PN81pd5WuXJl6OjoICYmplDny8rKQkpKSlFCVtC1a1e4ublh3rx5GDp06Ef7DNWhatWqAHL+caOsN8zFxQVBQUEKZbl3VBb02ef2DL37+5Pf746qqlatiqNHj6JFixZKk/Xc363bt2+jSpUqYvmTJ08KTLxjYmJQsWLFIiVn9GngHCsqEyZOnAg9PT0MGjQICQkJefZHR0fD398fQM7wDYA8d+4tXrwYQM4Xm7pUrVoVL1++xL///iuWxcXFYdeuXQr13h0mASD2ABR055a1tTXq16+PwMBAhS+fa9eu4ciRI+J1fgxt27bF7NmzsWLFCqWrd2toaOTpDduxYwcePXqkUJabPOSXhBbWpEmTcP/+fQQGBmLx4sVwcHAQF9xUpkKFCmjcuDEuXryo8rlu3bqFqKgouLi4FDVsBZMmTcKzZ8+wbt06tbarbo0aNULVqlWxcOHCfJPL3PXdTExM4O7urrDlrslW0Gdvb28PDQ2NPL2Bq1at+uB4+/bti+zsbMyePTvPvqysLDEGd3d3VKhQAcuXL1f4+VV2t294ePh7/1FHBLDHisqIqlWrYuvWrfjyyy/h7OyssPL6mTNnsGPHDvH5Xi4uLvD29sbatWuRlJQENzc3nD9/HoGBgejZs2eBt/J/CE9PT0yaNAmff/45Ro0ahdTUVKxevRo1atRQmLzt6+uLkJAQdO3aFfb29khMTMSqVatga2uLli1bFtj+ggUL0LlzZ7i6umLgwIF48+YNli9fDiMjo4+6krVUKsW0adPeW69bt27w9fXFgAED0Lx5c1y9ehVbtmxR6AUAcj4/Y2NjrFmzBgYGBtDT00OzZs3g6OhYqLiCg4OxatUqzJw5U1w2YePGjWjTpg2mT5+O+fPnKz2+R48emDp1KpKTk/PMOcvKysLvv/8OIGdIKTY2FmvWrIFcLsfMmTPztHXq1Kl8V6KvV68e6tWrpzSOzp07o06dOli8eDGGDx9eYqvtv49UKsX69evRuXNn1K5dGwMGDEClSpXw6NEjHD9+HIaGhti7d6/SNho1agQgZ6V6Dw8PaGhowNPTE0ZGRvjiiy+wfPlySCQSVK1aFfv27ROHeD+Em5sbhg4dCj8/P0RERKBjx46oUKECbt++jR07dsDf3x99+vQR18Dy8/NDt27d0KVLF1y+fBkHDx7Md6gvMTER//77L4YPH/7BsdEnpATvSCQqtFu3bgmDBw8WHBwcBC0tLcHAwEBo0aKFsHz5ciEtLU2sl5mZKcyaNUtwdHQUKlSoINjZ2QlTpkxRqCMIObfDd+3aNc953r09vaDlFgRBEI4cOSLUqVNH0NLSEpycnITff/89z3ILx44dE3r06CHY2NgIWlpago2NjfDVV18Jt27dynOOd5ckOHr0qNCiRQtBR0dHMDQ0FLp37y7cuHFDoU7u+d5dzmHjxo0CACEmJqbA91QQFJdbKEhByy2MGzdOsLa2FnR0dIQWLVoIZ8+ezXeZhH/++UeoVauWoKmpmWcpgNq1a+d7zrfbSU5OFuzt7YWGDRsKmZmZCvXGjBkjSKVS4ezZs0qvISEhQdDU1BQ2b96c5/rxzlIFhoaGQvv27YWjR48q1H3fcgszZ84U6xb08yUIghAQEJDv5/2hyy0UtHyDIBS83MKOHTsU6hX0M3j58mWhV69egpmZmSCTyQR7e3uhb9++wrFjx94bW1ZWljBy5EjB3NxckEgkCr8XT548EXr37i3o6uoKJiYmwtChQ4Vr167lu9xCfj+f7/6e5Vq7dq3QqFEjQUdHRzAwMBDq1q0rTJw4UXj8+LFYJzs7W5g1a5b4s9umTRvh2rVrgr29fZ7lFlavXi3o6uoKycnJ771eIokgFGJWKxFRGTdw4EDcunULp06dKulQqIxo0KAB2rRpk+f5jUT5YWJFRJ+U+/fvo0aNGjh27BhatGhR0uFQKXfo0CH06dMHd+/eLdSyHvTpYmJFREREpCa8K5CIiIhITZhYEREREakJEysiIiIiNWFiRURERKQmXCCUAOQsiPj48WMYGBjwIaNERGWQIAh49eoVbGxsCny4e1GlpaUhIyNDLW1paWmJK/SXJ0ysCADw+PFj2NnZlXQYRERURA8ePICtra3a201LS4OOgRmQlaqW9qysrBATE1PukismVgQAMDAwAAAcPHsDevoGJRwNfWxONobvr0Tlxqs3mSUdAhWDV69eoYGzo/j3XN0yMjKArFTIankDGlpFayw7A/E3ApGRkcHEisqn3OE/PX0D6BvwS7e8e/c5eVTOVWBi9Sn56NM5NLUhKWJiJUjK7xRvJlZERESkOgmAoiZv5XgqLxMrIiIiUp1EmrMVtY1yqvxeGREREVExY48VERERqU4iUcNQYPkdC2RiRURERKrjUKBS5ffKiIiIiIoZEysiIiJSXe5QYFG3QggJCUH37t1hY2MDiUSC3bt3i/syMzMxadIk1K1bF3p6erCxsUG/fv3w+PFjhTaeP38OLy8vGBoawtjYGAMHDkRKSopCnX///RetWrWCtrY27OzsMH/+/EK/PUysiIiIqBCk/w0HfuhWyPTj9evXcHFxwcqVK/PsS01NxaVLlzB9+nRcunQJO3fuRFRUFD777DOFel5eXrh+/TqCgoKwb98+hISEYMiQIeL+5ORkdOzYEfb29ggPD8eCBQvg4+ODtWvXFipWzrEiIiKiUq1z587o3LlzvvuMjIwQFBSkULZixQo0bdoU9+/fR+XKlREZGYlDhw7hwoULaNy4MQBg+fLl6NKlCxYuXAgbGxts2bIFGRkZ+O2336ClpYXatWsjIiICixcvVkjA3oc9VkRERKQ6NQ4FJicnK2zp6elqCfHly5eQSCQwNjYGAJw9exbGxsZiUgUA7u7ukEqlCAsLE+u0bt0aWlr/rSrv4eGBqKgovHjxQuVzM7EiIiIi1RV1GPCtuwrt7OxgZGQkbn5+fkUOLy0tDZMmTcJXX30lPr4rPj4eFhYWCvU0NTVhamqK+Ph4sY6lpaVCndzXuXVUwaFAIiIiKhEPHjxQeHapTCYrUnuZmZno27cvBEHA6tWrixreB2FiRURERKpT4wKhhoaGansofG5Sde/ePQQHByu0a2VlhcTERIX6WVlZeP78OaysrMQ6CQkJCnVyX+fWUQWHAomIiEh1ahwKVJfcpOr27ds4evQozMzMFPa7uroiKSkJ4eHhYllwcDDkcjmaNWsm1gkJCUFmZqZYJygoCE5OTjAxMVE5FiZWREREpLoSWMcqJSUFERERiIiIAADExMQgIiIC9+/fR2ZmJvr06YOLFy9iy5YtyM7ORnx8POLj45GRkQEAcHZ2RqdOnTB48GCcP38eoaGhGDFiBDw9PWFjYwMA+Prrr6GlpYWBAwfi+vXr2LZtG/z9/TF27NhCxcqhQCIiIirVLl68iLZt24qvc5Mdb29v+Pj4YM+ePQCA+vXrKxx3/PhxtGnTBgCwZcsWjBgxAu3bt4dUKkXv3r2xbNkysa6RkRGOHDmC4cOHo1GjRqhYsSJmzJhRqKUWACZWREREVBgl8KzANm3aQBCEAvcr25fL1NQUW7duVVqnXr16OHXqVKFiexcTKyIiIlKdRKKGxKqIk99LMc6xIiIiIlIT9lgRERGR6qSSnK2obZRTTKyIiIhIdSUwx6osKb9XRkRERFTM2GNFREREqlPjyuvlERMrIiIiUh2HApUqv1dGREREVMzYY0VERESq41CgUkysiIiISHUcClSKiRURERGpjj1WSpXflJGIiIiomLHHioiIiFTHoUClmFgRERGR6jgUqFT5TRmJiIiIihl7rIiIiKgQ1DAUWI77dZhYERERkeo4FKhU+U0ZiYiIiIoZe6yIiIhIdRKJGu4KLL89VkysiIiISHVcbkGp8ntlRERERMWMPVZERESkOk5eV4qJFREREamOQ4FKMbEiIiIi1bHHSqnymzISERERFTP2WBEREZHqOBSoFBMrIiIiUh2HApUqvykjERERUTFjjxURERGpTCKRQMIeqwIxsSIiIiKVMbFSjkOBRERERGrCHisiIiJSneT/t6K2UU4xsSIiIiKVcShQOQ4FEhEREakJe6yIiIhIZeyxUo6JFREREamMiZVyTKw+UEBAAEaPHo2kpCQAgI+PD3bv3o2IiIgSjetTt+mvEzhx7jruP3wCLVkF1HWqjB+8O8G+krlCvas37+PXLUdw49YDSKVSVHe0xtKZAyCTVQAA9Bo8H/FPkhSO+f5bD/Tr7VZcl0Jq8up1Guat2Yd9J67g6YsU1K1hi5/H9UHD2vYlHRoVUUpqGhatP4jDp67i6YsU1K5eCT6jPoeLc2UAwJLfDmFv8GU8TkxCBU0N1HWyxYTBXdGgFj/7omBipdwnn1h9aEL05ZdfokuXLh8nqCJycHDA6NGjMXr06JIOpdhdvh6D3p3/B+fqtsjOlmPN70cw2mcjti4fDR1tLQA5SdVY3434tncbjB3cHRoaUtyJiYNEqviLPvgrd3zWsYn4WldHVqzXQurx45ytiIx+jDWzvGFtboTtB8+j5/DlOLd9GmwsjEs6PCqCSb9sQ1RMHJZM9YJlRUPsOhIOr7GrcXTTJFiZG8PRzhy+o3uhso0Z0tIzsX77SXw7bg1O/jEVZsb6JR0+lVOcvP6BdHR0YGFhUdJh0DuWzByAru0boUplS1R3tMa0Ub2R8CQJN6MfiXWW/bYfX3Rtjn693VClsiXsK5mjfct60Kqg+O8MXR0ZzEwMxC03MaOy401aBvYcj4DPqJ5o0bAaqtiZY/KQrqhiZ47f/j5V0uFREaSlZ+BgyL+YMqw7mtWvCgdbc4z5rhPsK1XE5t1nAAA9OzRCy8ZOqGxTETUcrTF9RE+8ep2GyOjHJRx9GSdR01ZOlfnESi6Xw8/PD46OjtDR0YGLiwv++usvAMCJEycgkUhw7NgxNG7cGLq6umjevDmioqIA5AznzZo1C1euXBG7NgMCAgAAixcvRt26daGnpwc7Ozv88MMPSElJEc8bEBAAY2PjAuPq378/evbsiXnz5sHS0hLGxsbw9fVFVlYWJkyYAFNTU9ja2mLjxo0Kxz148AB9+/aFsbExTE1N0aNHD8TGxuZpd+HChbC2toaZmRmGDx+OzMxMAECbNm1w7949jBkzRj3dtWXc69R0AIChvg4A4HlSCq7fegATIz0MmbQGXb3n4oepa3HlRmyeYzfvPIlO386G95jl2LIrBFnZ2cUZOqlBVrYc2dlyaGtVUCjXllXAuYjoEoqK1CH3s5Xl89levHo3T/2MzCxs3XMWhvraqFXVprjCLJdyv1uKupVXZT6x8vPzw6ZNm7BmzRpcv34dY8aMwTfffIOTJ0+KdaZOnYpFixbh4sWL0NTUxHfffQcgZzhv3LhxqF27NuLi4hAXF4cvv/wSACCVSrFs2TJcv34dgYGBCA4OxsSJEwsVW3BwMB4/foyQkBAsXrwYM2fORLdu3WBiYoKwsDB8//33GDp0KB4+fAgAyMzMhIeHBwwMDHDq1CmEhoZCX18fnTp1QkZGhtju8ePHER0djePHjyMwMBABAQFiQrhz507Y2trC19dXvKZPlVwux9IN+1DP2R5V7a0AAI8TngMANmw7hs86NsHimQPgVKUSRs3YgAePn4rHftGtOXzHeWLF7EHo6dEUm/46gZWBh0rkOujDGehpo0ldRyzYcBBxT5KQnS3HtgPnceFqDBKeJpd0eFQE+rraaFjbAcsDjyDh6UtkZ8ux88hFXLoei8Rn/322x85ch7PHJNRwn4gNO07i90XDYMphQPqIyvQcq/T0dMybNw9Hjx6Fq6srAKBKlSo4ffo0fv31VwwZMgQAMHfuXLi55Uw6njx5Mrp27Yq0tDTo6OhAX18fmpqasLKyUmj77flJDg4OmDNnDr7//nusWrVK5fhMTU2xbNkySKVSODk5Yf78+UhNTcVPP/0EAJgyZQp+/vlnnD59Gp6enti2bRvkcjnWr18vZvMbN26EsbExTpw4gY4dOwIATExMsGLFCmhoaKBmzZro2rUrjh07hsGDB8PU1BQaGhowMDDIc03vvnfp6eni6+Tk8vcls2jtHty9l4A1fkPFMkEQAAA9OzZFt/aNAABOVWxw8d9o7DsWjmHfegAAvurRUjymmoM1Kmhq4JfVuzHsW488Q4ZUuv3q2w8jfLegVpdp0NCQwsXJDr07NsaVm/dLOjQqoqXTvDDh5z/RtJcPNDSkqFPdFp+1b4irUQ/EOq4NquHghvF4/vI1/th7Dj/MDMQ/v45GRRODkgu8jJNIoIbJ6+qJpTQq098Qd+7cQWpqKjp06KBQnpGRgQYNGoiv69WrJ/6/tbU1ACAxMRGVK1cusO2jR4/Cz88PN2/eRHJyMrKyspCWlobU1FTo6uqqFF/t2rUhlf7XKWhpaYk6deqIrzU0NGBmZobExEQAwJUrV3Dnzh0YGCj+wqelpSE6+r9hi9q1a0NDQ0Phmq5evapSTLn8/Pwwa9asQh1TlixauwehF6Kwat5gWFQ0EsvN/v+PqYOd4vw4B1tzJLxzF+DbatWwQ3a2HHGJL/LcYUilm6OtOfavHY3Xb9Lx6nUarCoa4bspv8G+UsWSDo2KyL5SRWxfPgKp///ZWlY0wvCZgahsYybW0dWRwcHWHA625mhY2wFuX83Ftv1hGP6NewlGXrZJoI6hvPKbWZXpxCp3ztP+/ftRqVIlhX0ymUxMRipU+G8MPveHQS6XF9hubGwsunXrhmHDhmHu3LkwNTXF6dOnMXDgQGRkZKicWL193txz51eWG0tKSgoaNWqELVu25GnL3Py/L3NlbahqypQpGDt2rPg6OTkZdnZ2hWqjNBIEAYvX7cXJczewcs4g2FiaKuy3tjBBRVND3H/0VKH8/uOncG1Yo8B2b8fEQSqVwMSIQwhllZ6ODHo6MiQlp+LYuUjMGtmjpEMiNdHVkUFXR4aXr1IRcuEmpnzfvcC6ckFARkZWMUZHn5oynVjVqlULMpkM9+/fF4f63vZ2L09BtLS0kP3OpOTw8HDI5XIsWrRI7HHavn27eoJWomHDhti2bRssLCxgaGj4we3kd03vkslkkMnK3/IBC3/dg6CQK/jlp2+gqyPDsxevAOTMx5DJKkAikcCrZyus//MoqjlaoYajDQ4EX8K9R08wd+LXAHKWY7hx6wEa1q0CXR0ZrkXdh/9v++HhVl+cBE9lx7GzNyAIQHV7C9x9+AQz/HejhoMlvD5zLenQqIhOnr8JQRBQxc4C9x49xbzVe1C1siW+6NIMqW/SsWLzUbi3qA0LM0O8ePkagbtOI+HpS3Rt61LSoZdpXMdKuTKdWBkYGGD8+PEYM2YM5HI5WrZsiZcvXyI0NBSGhoawt3//InAODg6IiYlBREQEbG1tYWBggGrVqiEzMxPLly9H9+7dERoaijVr1nz06/Hy8sKCBQvQo0cP+Pr6wtbWFvfu3cPOnTsxceJE2NraqtSOg4MDQkJC4OnpCZlMhooVP50hj12HwgAAw6etVyifOrI3uv7/nKovP2uB9MwsLNtwAMkpqajmYA1/n+9ga50zfKBVQQNHT/+LDX8eQ0ZWFmwsTODZvQU835p3RWVHckoafFfuwePEJJgY6qJ7u/qY9kN3VNDUeP/BVKq9SnmDX9buR/yTJBgZ6KKzmwsmDO6CCpoayM6W4869BPx16AJevEyBsaEeXGpWxo7lI1HD0bqkQy/b1LFcQvnNq8p2YgUAs2fPhrm5Ofz8/HD37l0YGxujYcOG+Omnn1QaHuvduzd27tyJtm3bIikpCRs3bkT//v2xePFi/PLLL5gyZQpat24NPz8/9OvX76Nei66uLkJCQjBp0iT06tULr169QqVKldC+fftC9WD5+vpi6NChqFq1KtLT08UJ25+CM7vnqVSvX2+3AldRd6paCevmD1NnWFSCPu/QEJ93aFjSYdBH0K1dA3Rr1yDffdqyClg797tijogIkAif0rcuFSg5ORlGRkYIufoA+gYfPgxJZYNzJX7Gn5LkN5klHQIVg1fJyahmWxEvX74s0nSSguR+T5h8tQFSLdXmGhdEnpGKF38M/GixlqQy32NFRERExUcdc6y4QCgRERERSmbl9ZCQEHTv3h02NjaQSCTYvXu3wn5BEDBjxgxYW1tDR0cH7u7uuH37tkKd58+fw8vLC4aGhjA2NsbAgQMVnqgCAP/++y9atWoFbW1t2NnZYf78+YV+f5hYERERUan2+vVruLi4YOXKlfnunz9/PpYtW4Y1a9YgLCwMenp68PDwQFpamljHy8sL169fR1BQEPbt24eQkBBxIXEgZ6izY8eOsLe3R3h4OBYsWAAfHx+sXbu2ULFyKJCIiIhUVwJ3BXbu3BmdO3fOd58gCFi6dCmmTZuGHj1y1qfbtGkTLC0tsXv3bnh6eiIyMhKHDh3ChQsX0LhxYwDA8uXL0aVLFyxcuBA2NjbYsmULMjIy8Ntvv0FLSwu1a9dGREQEFi9erJCAvQ97rIiIiEhl6hwKTE5OVtjeftSaqmJiYhAfHw939/9W0zcyMkKzZs1w9uxZAMDZs2dhbGwsJlUA4O7uDqlUirCwMLFO69atoaWlJdbx8PBAVFQUXrx4oXI8TKyIiIioRNjZ2cHIyEjc/Pz8Ct1GfHw8gJzHxr3N0tJS3BcfHw8LC8VHmWlqasLU1FShTn5tvH0OVXAokIiIiFSmzrsCHzx4oLDcQnl4IggTKyIiIlKZOhMrQ0PDIq9jZWVlBQBISEiAtfV/q+onJCSgfv36Yp3ExESF47KysvD8+XPxeCsrKyQkJCjUyX2dW0cVHAokIiKiMsvR0RFWVlY4duyYWJacnIywsDC4uuY8E9TV1RVJSUkIDw8X6wQHB0Mul6NZs2ZinZCQEGRm/regblBQEJycnGBiYqJyPEysiIiISGUlsY5VSkoKIiIiEBERAQDiM37v378PiUSC0aNHY86cOdizZw+uXr2Kfv36wcbGBj179gQAODs7o1OnThg8eDDOnz+P0NBQjBgxAp6enrCxsQEAfP3119DS0sLAgQNx/fp1bNu2Df7+/hg7dmyhYuVQIBEREamuBJZbuHjxItq2bSu+zk12vL29ERAQgIkTJ+L169cYMmQIkpKS0LJlSxw6dAja2triMVu2bMGIESPQvn17SKVS9O7dG8uWLRP3GxkZ4ciRIxg+fDgaNWqEihUrYsaMGYVaagHgswLp//FZgZ8WPivw08JnBX4aiutZgZYDNqvlWYEJG7/lswKJiIjo08ZnBSrHxIqIiIhUxsRKOSZWREREpDImVsrxrkAiIiIiNWGPFREREamuBO4KLEuYWBEREZHKOBSoHIcCiYiIiNSEPVZERESkMvZYKcfEioiIiFQmgRoSq3I8yYpDgURERERqwh4rIiIiUhmHApVjYkVERESq43ILSnEokIiIiEhN2GNFREREKuNQoHJMrIiIiEhlTKyUY2JFREREKpNIcraitlFecY4VERERkZqwx4qIiIhUltNjVdShQDUFUwoxsSIiIiLVqWEokMstEBEREdF7sceKiIiIVMa7ApVjYkVEREQq412BynEokIiIiEhN2GNFREREKpNKJZBKi9blJBTx+NKMiRURERGpjEOBynEokIiIiEhN2GNFREREKuNdgcoxsSIiIiKVcShQOSZWREREpDL2WCnHOVZEREREasIeKyIiIlIZe6yUY2JFREREKuMcK+U4FEhERESkJuyxIiIiIpVJoIahQJTfLismVkRERKQyDgUqx6FAIiIiIjVhjxURERGpjHcFKsfEioiIiFTGoUDlOBRIREREpCbssSIiIiKVcShQOSZWREREpDIOBSrHxIqIiIhUxh4r5TjHioiIiEhN2GNFCqpa6sPQUL+kw6CPbPCfV0o6BCpGy3vVKekQqBhU0CimvhI1DAWW44XXmVgRERGR6jgUqByHAomIiIjUhD1WREREpDLeFagcEysiIiJSGYcCleNQIBEREZGaMLEiIiIileUOBRZ1K4zs7GxMnz4djo6O0NHRQdWqVTF79mwIgiDWEQQBM2bMgLW1NXR0dODu7o7bt28rtPP8+XN4eXnB0NAQxsbGGDhwIFJSUtTxtoiYWBEREZHKcocCi7oVxi+//ILVq1djxYoViIyMxC+//IL58+dj+fLlYp358+dj2bJlWLNmDcLCwqCnpwcPDw+kpaWJdby8vHD9+nUEBQVh3759CAkJwZAhQ9T23gCcY0VERESl3JkzZ9CjRw907doVAODg4IA//vgD58+fB5DTW7V06VJMmzYNPXr0AABs2rQJlpaW2L17Nzw9PREZGYlDhw7hwoULaNy4MQBg+fLl6NKlCxYuXAgbGxu1xMoeKyIiIlKZOnuskpOTFbb09PR8z9m8eXMcO3YMt27dAgBcuXIFp0+fRufOnQEAMTExiI+Ph7u7u3iMkZERmjVrhrNnzwIAzp49C2NjYzGpAgB3d3dIpVKEhYWp7f1hjxURERGpTJ3LLdjZ2SmUz5w5Ez4+PnnqT548GcnJyahZsyY0NDSQnZ2NuXPnwsvLCwAQHx8PALC0tFQ4ztLSUtwXHx8PCwsLhf2ampowNTUV66gDEysiIiJSmTqXW3jw4AEMDQ3FcplMlm/97du3Y8uWLdi6dStq166NiIgIjB49GjY2NvD29i5SLOrGxIqIiIhKhKGhoUJiVZAJEyZg8uTJ8PT0BADUrVsX9+7dg5+fH7y9vWFlZQUASEhIgLW1tXhcQkIC6tevDwCwsrJCYmKiQrtZWVl4/vy5eLw6cI4VERERqawklltITU2FVKqYsmhoaEAulwMAHB0dYWVlhWPHjon7k5OTERYWBldXVwCAq6srkpKSEB4eLtYJDg6GXC5Hs2bNPvDdyIs9VkRERKSyklh5vXv37pg7dy4qV66M2rVr4/Lly1i8eDG+++47sb3Ro0djzpw5qF69OhwdHTF9+nTY2NigZ8+eAABnZ2d06tQJgwcPxpo1a5CZmYkRI0bA09NTbXcEAkysiIiIqJRbvnw5pk+fjh9++AGJiYmwsbHB0KFDMWPGDLHOxIkT8fr1awwZMgRJSUlo2bIlDh06BG1tbbHOli1bMGLECLRv3x5SqRS9e/fGsmXL1BqrRHh72VL6ZCUnJ8PIyAiPEl+oNN5NZduwHVdLOgQqRst71SnpEKgYJCcnw97aFC9fvvwof8dzvyda/xIETR29IrWV9eY1QiZ1+GixliT2WBEREZHKpBIJpEUcCizq8aUZJ68TERERqQl7rIiIiEhl6lwgtDxiYkVEREQqK4m7AssSJlZERESkMqkkZytqG+UV51gRERERqQl7rIiIiEh1EjUM5ZXjHismVkRERKQyTl5XjkOBRERERGrCHisiIiJSmeT//ytqG+UVEysiIiJSGe8KVI5DgURERERqwh4rIiIiUhkXCFVOpcRqz549Kjf42WeffXAwREREVLrxrkDlVEqsevbsqVJjEokE2dnZRYmHiIiIqMxSKbGSy+UfOw4iIiIqA6QSCaRF7HIq6vGlWZHmWKWlpUFbW1tdsRAREVEpx6FA5Qp9V2B2djZmz56NSpUqQV9fH3fv3gUATJ8+HRs2bFB7gERERFR65E5eL+pWXhU6sZo7dy4CAgIwf/58aGlpieV16tTB+vXr1RocERERUVlS6MRq06ZNWLt2Lby8vKChoSGWu7i44ObNm2oNjoiIiEqX3KHAom7lVaHnWD169AjVqlXLUy6Xy5GZmamWoIiIiKh04uR15QrdY1WrVi2cOnUqT/lff/2FBg0aqCUoIiIiorKo0D1WM2bMgLe3Nx49egS5XI6dO3ciKioKmzZtwr59+z5GjERERFRKSP5/K2ob5VWhe6x69OiBvXv34ujRo9DT08OMGTMQGRmJvXv3okOHDh8jRiIiIioleFegch+0jlWrVq0QFBSk7liIiIiIyrQPXiD04sWLiIyMBJAz76pRo0ZqC4qIiIhKJ6kkZytqG+VVoROrhw8f4quvvkJoaCiMjY0BAElJSWjevDn+/PNP2NraqjtGIiIiKiXUMZRXnocCCz3HatCgQcjMzERkZCSeP3+O58+fIzIyEnK5HIMGDfoYMRIRERGVCYXusTp58iTOnDkDJycnsczJyQnLly9Hq1at1BocERERlT7luMOpyAqdWNnZ2eW7EGh2djZsbGzUEhQRERGVThwKVK7QQ4ELFizAyJEjcfHiRbHs4sWL+PHHH7Fw4UK1BkdERESlS+7k9aJu5ZVKPVYmJiYK2eXr16/RrFkzaGrmHJ6VlQVNTU1899136Nmz50cJlIiIiKi0UymxWrp06UcOg4iIiMoCDgUqp1Ji5e3t/bHjICIiojKAj7RR7oMXCAWAtLQ0ZGRkKJQZGhoWKSAiIiKisqrQidXr168xadIkbN++Hc+ePcuzPzs7Wy2BERERUekjlUggLeJQXlGPL80KfVfgxIkTERwcjNWrV0Mmk2H9+vWYNWsWbGxssGnTpo8RIxEREZUSEol6tvKq0D1We/fuxaZNm9CmTRsMGDAArVq1QrVq1WBvb48tW7bAy8vrY8RJREREVOoVusfq+fPnqFKlCoCc+VTPnz8HALRs2RIhISHqjY6IiIhKldy7Aou6lVeF7rGqUqUKYmJiULlyZdSsWRPbt29H06ZNsXfvXvGhzESlyca/TyFgZyjux+XMCaxZxRrjvusE9+a1AACbdofi78Ph+DfqAVJS03En6GcYGeiWZMhUCMY6mujtYoM61gbQ0pAiMSUdAWEPcO/FGwCATFOKXvWs0cDWEHpamnj6OgPBt57iZHTeOaIAMKq1I+raGGLlqRhEPEouzkuhQop7koQ5q/bg+LlIvEnLhINtRSz56WvUd64MANh/4go27Q7F1agHeJGciqCNE1Cnhm0JR132qWMorxznVYXvsRowYACuXLkCAJg8eTJWrlwJbW1tjBkzBhMmTFB7gKWdRCLB7t27AQCxsbGQSCSIiIj44PbU0QYpsrEwxrTh3XE0YAKOBkxAy0Y10G/iOty8GwcASE3LQDtXZ4zu37GEI6XC0q2ggUnu1ZEtF+B/8i5mHozCjojHSM387yaavg1ykq715+5jxsGbOHrrCb5qVAkuNnnvYHavUbE4w6ciSEpOxWff+6OCpga2LPoeJ7dMwcwRPWH81j+KUtMy0KxeFUwd9lkJRkqfmkL3WI0ZM0b8f3d3d9y8eRPh4eGoVq0a6tWrp9bgisrBwQGjR4/G6NGji+V8dnZ2iIuLQ8WKqv1x7t+/P5KSksTE7EPaoPfzaFVX4fXUYd0QsOs0Ll6LRc0q1vjesy0AIDT8dkmER0XQydkCL1IzEHD+gVj29LXiEjBVzXRxJvY5biW+BgCcin4Ot6pmcDTTxZXH//VI2Rlro2NNc8w5chuLetYungugD7Zyy1HYWBhj6dT/5vVWtjFTqPNFpyYAgAdx+fdO0ofhXYHKFWkdKwCwt7eHvb29OmIpEdnZ2ZBIJJBKC915l4eGhgasrKxKvA0qWHa2HHuCLyP1TTqa1HUo6XCoiFwqGeJ6/CsMbW6PGhZ6SHqThRO3n+LU3edinehnqahvY4TQu8+R9CYLThZ6sDSQYdvlx2IdLQ0JBrnaY0v4IySnZZXEpVAhHT59DW2a1sTgaRtx9vIdWJsbwbtXS3zzWfOSDq3c41CgciplE8uWLVN5Kwy5XI758+ejWrVqkMlkqFy5MubOnQsAuHr1Ktq1awcdHR2YmZlhyJAhSElJEY/t378/evbsiYULF8La2hpmZmYYPnw4MjMzAQBt2rTBvXv3MGbMGIWJcgEBATA2NsaePXtQq1YtyGQy3L9/HxcuXECHDh1QsWJFGBkZwc3NDZcuXVKI9/bt22jdujW0tbVRq1YtBAUFKezPbxjv+vXr6NatGwwNDWFgYIBWrVohOjoaPj4+CAwMxD///CPGd+LEiXzbOHnyJJo2bQqZTAZra2tMnjwZWVn//fFv06YNRo0ahYkTJ8LU1BRWVlbw8fEp1GdR3t248xj2bcejUuuxGP/LdgT8MghOjtYlHRYVkbm+FtpUM0NiSjqWnojBiTtP4dmwElwdTMQ6f4Q/wuPkNCzoURur+9bDj25VsDX8EW4/eS3W6dugEqKfvsYVzqkqM+4/foZNu0PhaFsRfywZhn6ft8T0JTux/cD5kg6t3OPkdeVU6rFasmSJSo1JJBKMGjVK5ZNPmTIF69atw5IlS9CyZUvExcXh5s2beP36NTw8PODq6ooLFy4gMTERgwYNwogRIxAQECAef/z4cVhbW+P48eO4c+cOvvzyS9SvXx+DBw/Gzp074eLigiFDhmDw4MEK501NTcUvv/yC9evXw8zMDBYWFrh79y68vb2xfPlyCIKARYsWoUuXLrh9+zYMDAwgl8vRq1cvWFpaIiwsDC9fvnzvEOOjR4/QunVrtGnTBsHBwTA0NERoaCiysrIwfvx4REZGIjk5GRs3bgQAmJqa4vHjx3na6NKlC/r3749Nmzbh5s2bGDx4MLS1tRWSp8DAQIwdOxZhYWE4e/Ys+vfvjxYtWqBDhw75xpaeno709HTxdXJy+f5CqWZvgeObJuHV6zfYExyBkb6/45/Vo5hclXESALEv3mDXv/EAgAdJb1DJSBtu1cxwNvYFAKBd9YqoYqaL5SExePY6AzUs9PB1o0pIepOJyIQUuNgYoqalPmYfvlWCV0KFJZcLcKlph5++7w4AqFvDFlF347Bpdyj6dmlawtHRp0ylxComJkbtJ3716hX8/f2xYsUK8VmEVatWRcuWLbFu3TqkpaVh06ZN0NPTAwCsWLEC3bt3xy+//AJLS0sAgImJCVasWAENDQ3UrFkTXbt2xbFjxzB48GCYmppCQ0MDBgYGeYbWMjMzsWrVKri4uIhl7dq1U6izdu1aGBsb4+TJk+jWrRuOHj2Kmzdv4vDhw7CxsQEAzJs3D507dy7wGleuXAkjIyP8+eefqFChAgCgRo0a4n4dHR2kp6crHfpbtWoV7OzssGLFCkgkEtSsWROPHz/GpEmTMGPGDHEIs169epg5cyYAoHr16lixYgWOHTtWYGLl5+eHWbNmFXje8kargiaq2JkDAFxqVkbEjftYu+0kFk32LOHIqChepmUh7mWaQllccjoa2hoDACpoSPB5PSusOh2Lq3GvAACPXqbBzlgHHWuaIzIhBTUt9WGurwX/XnUU2hnWwgG3n77GwuDoYrkWKhwLM0PUcFD821ndwRL7T1wpoYg+HVJ8wJ1v+bRRXhV5jtWHioyMRHp6Otq3b5/vPhcXFzGpAoAWLVpALpcjKipKTKxq164NDQ0NsY61tTWuXr363nNraWnlmWifkJCAadOm4cSJE0hMTER2djZSU1Nx//59MSY7OzsxqQIAV1dXpeeJiIhAq1atxKTqQ0RGRsLV1VWh27RFixZISUnBw4cPUblyzm3F716PtbU1EhMTC2x3ypQpGDt2rPg6OTkZdnZ2HxxnWSMXBKRncC5NWXfn6WtYGcoUyiwNZHiWmjOBXUMigaaGFMI7x8kFiL9TByMTFeZkAcCszk7Ydvkx/n1cvntyy7Km9Rxx577i37jo+4mwtTIp4AhSF3UM5X3yQ4Efg46OTpHbeDdhkUgkkMvlKp373Q/V29sbz549g7+/P+zt7SGTyeDq6prnIdOFoY5rVFVh3wuZTAaZTFbg/vJk9qo9aO9aC7aWJkhJTcffRy4i9NIdbF86DACQ8CwZic+ScffhEwDAjeg46OvKYGtpAhMjPWVNUwk7GvUEk9yro0stC1y4nwRHM120rmqKzRceAgDSsuSISkxBHxdrZGTL8fx1JmpY6MHVwQTbI3KG3ZPTsvKdsP48NSPPHYZUegz5sg26D10K/8Aj+Kx9A1y+cQ+/7zmLBRO/FOu8SH6NR/EvkPD0JYCcxAvI6e2yMMu73AaROpRYYlW9enXo6Ojg2LFjGDRokMI+Z2dnBAQE4PXr12KvVWhoKKRSKZycnFQ+h5aWlsoPhQ4NDcWqVavQpUsXAMCDBw/w9OlThZgePHiAuLg4WFvnzMs5d+6c0jbr1auHwMBAZGZm5ttrpUp8zs7O+PvvvyEIgpgMhoaGwsDAALa2XOhOFU9fpGDErN+R8OwlDPV1UKuqDbYvHYY2zWoCAAJ3nsaCDYfE+p997w8AWDbNC191a1YiMZNqYp+/werTMfi8njW61bbE05QMbLv0GGH3ksQ6a8/cQ6961hj0P3voaWngWWoGdl+Nw8k7vAW/LKvvbI/f/AZi3pp9WBJwGHbWZvD98XP09mgs1jly6hpGz9sqvv5+ZiAAYNx3nTB+YMHTOEg5iQSQ8q7AApVYYqWtrY1JkyZh4sSJ0NLSQosWLfDkyRNcv34dXl5emDlzJry9veHj44MnT55g5MiR+Pbbb8VhQFU4ODggJCQEnp6ekMlkSteGql69OjZv3ozGjRsjOTkZEyZMUOhxcnd3R40aNeDt7Y0FCxYgOTkZU6dOVXr+ESNGYPny5fD09MSUKVNgZGSEc+fOoWnTpnBycoKDgwMOHz6MqKgomJmZwcjIKE8bP/zwA5YuXYqRI0dixIgRiIqKwsyZMzF27Fi1LBHxKfCf+rXS/RMHd8HEwV2KKRpSt38fv8K/j18VuD85LUthnStVDP6T83TKgg4t6qBDizoF7v+yazN82ZX/OFI3qRoSqw85/tGjR5g0aRIOHjyI1NRUVKtWDRs3bkTjxjnJtCAImDlzJtatW4ekpCS0aNECq1evRvXq1cU2nj9/jpEjR2Lv3r2QSqXo3bs3/P39oa+vX7QLekuJfjNPnz4d48aNw4wZM+Ds7Iwvv/wSiYmJ0NXVxeHDh/H8+XM0adIEffr0Qfv27bFixYpCte/r64vY2FhUrVoV5ubmSutu2LABL168QMOGDfHtt99i1KhRsLCwEPdLpVLs2rULb968QdOmTTFo0CBxaYiCmJmZITg4GCkpKXBzc0OjRo2wbt06sfdq8ODBcHJyQuPGjWFubo7Q0NA8bVSqVAkHDhzA+fPn4eLigu+//x4DBw7EtGnTCvVeEBERlVUvXrxAixYtUKFCBRw8eBA3btzAokWLYGLy35y6+fPnY9myZVizZg3CwsKgp6cHDw8PpKX9d4OLl5cXrl+/jqCgIOzbtw8hISEYMmSIWmOVCILw7rzO9zp16hR+/fVXREdH46+//kKlSpWwefNmODo6omXLlmoNkIpHcnIyjIyM8CjxBQwNOfegvBu24/03eVD5sbxXwb06VH4kJyfD3toUL1++/Ch/x3O/J4b/eREy3aL18KSnpmClZ2OVY508eTJCQ0Nx6tSpfPcLggAbGxuMGzcO48ePBwC8fPkSlpaWCAgIgKenJyIjI1GrVi1cuHBB7OU6dOgQunTpgocPHyrcnFYUhe6x+vvvv+Hh4QEdHR1cvnxZXAvp5cuXmDdvnlqCIiIiotIpdyiwqBuQk6y9vb29vuLb9uzZg8aNG+OLL76AhYUFGjRogHXr1on7Y2JiEB8fD3d3d7HMyMgIzZo1w9mzZwEAZ8+ehbGxsZhUATnTfKRSKcLCwtT3/hT2gDlz5mDNmjUKQ1pAzhIA765UTkRERFQQOzs7GBkZiZufn1++9e7evSvOlzp8+DCGDRuGUaNGITAw54aE+PicRYLfnYdtaWkp7ouPj1eY4gMAmpqaMDU1FeuoQ6Enr0dFRaF169Z5yo2MjJCUlKSOmIiIiKiUUuezAh88eKAwFFjQMkByuRyNGzcWR8YaNGiAa9euYc2aNeIi46VFoXusrKyscOfOnTzlp0+fRpUqVdQSFBEREZVOUolELRsAGBoaKmwFJVbW1taoVauWQpmzs7O4iHfuE0wSEhIU6iQkJIj7rKys8iycnZWVhefPnyt9AkphFTqxGjx4MH788UeEhYVBIpHg8ePH2LJlC8aPH49hw4apLTAiIiIqfaRq2gqjRYsWiIqKUii7desW7O3tAQCOjo6wsrLCsWPHxP3JyckICwsTn5Li6uqKpKQkhIeHi3WCg4Mhl8vRrJn6luUo9FDg5MmTIZfL0b59e6SmpqJ169aQyWQYP348Ro4cqbbAiIiIiABgzJgxaN68OebNm4e+ffvi/PnzWLt2LdauXQsg52kjo0ePxpw5c1C9enU4Ojpi+vTpsLGxQc+ePQHk9HB16tQJgwcPxpo1a5CZmYkRI0bA09NTbXcEAh+QWEkkEkydOhUTJkzAnTt3kJKSglq1aql1cS0iIiIqndQ5x0pVTZo0wa5duzBlyhT4+vrC0dERS5cuhZeXl1hn4sSJeP36NYYMGYKkpCS0bNkShw4dgra2tlhny5YtGDFiBNq3by8uELps2bKiXcw7PmgdKyp/uI7Vp4XrWH1auI7Vp6G41rGa8NclyPSKuI7V6xQs6NPwo8VakgrdY9W2bVulT6UODg4uUkBEREREZVWhE6v69esrvM7MzERERASuXbtW6m55JCIiIvUqiaHAsqTQidWSJUvyLffx8UFKSkqRAyIiIqLSq6QewlxWqO0hzN988w1+++03dTVHREREVOYUuseqIGfPnlWYeU9ERETlj0QCcYHPorRRXhU6serVq5fCa0EQEBcXh4sXL2L69OlqC4yIiIhKH86xUq7QiZWRkZHCa6lUCicnJ/j6+qJjx45qC4yIiIiorClUYpWdnY0BAwagbt26MDEx+VgxERERUSnFyevKFWryuoaGBjp27IikpKSPFA4RERGVZhI1/VdeFfquwDp16uDu3bsfIxYiIiIq5XJ7rIq6lVeFTqzmzJmD8ePHY9++fYiLi0NycrLCRkRERPSpUnmOla+vL8aNG4cuXboAAD777DOFR9sIggCJRILs7Gz1R0lERESlAudYKadyYjVr1ix8//33OH78+MeMh4iIiEoxiUSi9JnBqrZRXqmcWAmCAABwc3P7aMEQERERlWWFWm6hPGeYRERE9H4cClSuUIlVjRo13ptcPX/+vEgBERERUenFldeVK1RiNWvWrDwrrxMRERFRjkIlVp6enrCwsPhYsRAREVEpJ5VIivwQ5qIeX5qpnFhxfhURERFxjpVyKi8QmntXIBERERHlT+UeK7lc/jHjICIiorJADZPXy/GjAgs3x4qIiIg+bVJIIC1iZlTU40szJlZERESkMi63oFyhH8JMRERERPljjxURERGpjHcFKsfEioiIiFTGdayU41AgERERkZqwx4qIiIhUxsnryjGxIiIiIpVJoYahwHK83AKHAomIiIjUhD1WREREpDIOBSrHxIqIiIhUJkXRh7vK83BZeb42IiIiomLFHisiIiJSmUQigaSIY3lFPb40Y2JFREREKpP8/1bUNsorJlZERESkMq68rhznWBERERGpCXusiIiIqFDKb39T0TGxIiIiIpVxHSvlOBRIREREpCbssSIiIiKVcbkF5ZhYERERkcq48rpy5fnaiIiIiIoVe6yIiIhIZRwKVI6JFREREamMK68rx6FAIiIiIjVhjxUpyMiSIz1LXtJh0Ee2sk/dkg6BipGl66iSDoGKgZCdUSzn4VCgckysiIiISGW8K1C58nxtREREpGa5PVZF3T7Uzz//DIlEgtGjR4tlaWlpGD58OMzMzKCvr4/evXsjISFB4bj79++ja9eu0NXVhYWFBSZMmICsrKwPjqMgTKyIiIioTLhw4QJ+/fVX1KtXT6F8zJgx2Lt3L3bs2IGTJ0/i8ePH6NWrl7g/OzsbXbt2RUZGBs6cOYPAwEAEBARgxowZao+RiRURERGpTKKmrbBSUlLg5eWFdevWwcTERCx/+fIlNmzYgMWLF6Ndu3Zo1KgRNm7ciDNnzuDcuXMAgCNHjuDGjRv4/fffUb9+fXTu3BmzZ8/GypUrkZGh3rlpTKyIiIhIZbkPYS7qVljDhw9H165d4e7urlAeHh6OzMxMhfKaNWuicuXKOHv2LADg7NmzqFu3LiwtLcU6Hh4eSE5OxvXr1z/sjSgAJ68TERFRiUhOTlZ4LZPJIJPJ8tT7888/cenSJVy4cCHPvvj4eGhpacHY2Fih3NLSEvHx8WKdt5Oq3P25+9SJPVZERESkMikkatkAwM7ODkZGRuLm5+eX53wPHjzAjz/+iC1btkBbW7u4L7fQ2GNFREREKvvQobx32wBykiZDQ0OxPL/eqvDwcCQmJqJhw4ZiWXZ2NkJCQrBixQocPnwYGRkZSEpKUui1SkhIgJWVFQDAysoK58+fV2g3967B3Drqwh4rIiIiKhGGhoYKW36JVfv27XH16lVERESIW+PGjeHl5SX+f4UKFXDs2DHxmKioKNy/fx+urq4AAFdXV1y9ehWJiYlinaCgIBgaGqJWrVpqvSb2WBEREZHKJP//X1HbUJWBgQHq1KmjUKanpwczMzOxfODAgRg7dixMTU1haGiIkSNHwtXVFf/73/8AAB07dkStWrXw7bffYv78+YiPj8e0adMwfPjwfJO5omBiRURERCpT51CguixZsgRSqRS9e/dGeno6PDw8sGrVKnG/hoYG9u3bh2HDhsHV1RV6enrw9vaGr6+vegMBEysiIiIqY06cOKHwWltbGytXrsTKlSsLPMbe3h4HDhz4yJExsSIiIqJCkLx1V19R2iivmFgRERGRykrjUGBpwsSKiIiIVMbESjkut0BERESkJuyxIiIiIpUV93ILZQ0TKyIiIlKZVJKzFbWN8opDgURERERqwh4rIiIiUhmHApVjYkVEREQq412BynEokIiIiEhN2GNFREREKpOg6EN55bjDiokVERERqY53BSrHoUAiIiIiNWGPFREREamMdwUqx8SKiIiIVMa7ApVjYkVEREQqk6Dok8/LcV7FOVZERERE6sIeKyIiIlKZFBJIiziWJy3HfVZMrIiIiEhlHApUjkOBRERERGrCHisiIiJSHbuslGJiRURERCrjOlbKcSiQiIiISE3YY0VERESqU8MCoeW4w4qJFREREamOU6yU41AgERERkZqwx4qIiIhUxy4rpZhYERERkcp4V6ByTKyIiIhIZRI1TF4v8uT3UoxzrIiIiIjUhD1WREREpDJOsVKOiRURERGpjpmVUhwKJCIiIlIT9lgRERGRynhXoHJMrIiIiEhlvCtQOQ4FEhEREakJe6yIiIhIZZy7rhwTKyIiIlIdMyulOBRIREREpCbssSIiIiKV8a5A5ZhYERERkcp4V6ByTKyIiIhIZZxipRznWBERERGpCXusqNz7X59ZeBj/Ik+59+ctMXdcH/QZsRznIqIV9n3Tozl+ntC3uEKkj2TZpiDMXb0Xg/u6Yc6Y3mL5hasx8Pt1Hy5dvwcNqQR1atjizyXDoKOtVYLR0tuaN6iKkd+6w6VmZVibG8Fr/FocOPkvAEBTQ4ppw7qjQ4vasK9khuSUNJw8fxOzVuxB/NOXedrSqqCJowHjUbeGLVp5+eHarUd56jjaVsTJ3ydDLpfDod3Ej359ZRq7rJRiYlVOOTg4YPTo0Rg9enRJh1Li9q8bh2y5XHwddTcOX41Zja5tXcSyr7u7YvygzuJrfsGWfZdv3MOm3aGoVc1GofzC1Rh8NWY1RvXrgHlj+0BTQ4rrtx9BKi3Hf+nLIF0dGa7deoTf95zF7wuGKO7T1kK9mnZYsOEgrt1+BGMDXfiN64Oti4ainff8PG3NGtUD8U9eom4N23zPpakhxfq5A3AuIhpN6zl+lOspTzh5XTkmVqVEmzZtUL9+fSxdurSkQyl3zEz0FV6v/P0o7CtVhGuDamKZjnYFWJgZFndo9JG8Tk3HDz6bsGjyV1gacFhh3wz/nRj0hRtG9esgllWztyzuEOk9jp65gaNnbuS7L/l1GnqNWKFQNnHBdgQHToStpQkeJvzXQ+3evBbaNnOG96T16NCidr7tTRvWHbdjE3DyQhQTKyoyzrEqQwRBQFZWVkmHUaZlZGZh55FweHZtBslbt6XsCgpH3a5T0f7bn+G3Zi/epGWUYJRUVJMX7oB789pwa+qkUP7k+Stcun4PFU310XXwYtTuMhU9h/kj7Ep0AS1RWWGorwO5XI6XKW/EMnNTAyz96St8P3MTUgv4nW7VuAZ6uDfAhPnbiyvUMi/3rsCibuUVEysVtGnTBqNGjcLEiRNhamoKKysr+Pj4iPuTkpIwaNAgmJubw9DQEO3atcOVK1fE/f3790fPnj0V2hw9ejTatGkj7j958iT8/f0hkUggkUgQGxuLEydOQCKR4ODBg2jUqBFkMhlOnz6N6Oho9OjRA5aWltDX10eTJk1w9OjRYngnyr7DIVeRnPIGX3RpKpb17NAIy6Z/g+3LhmP4t+74+/BFjPT9vQSjpKLYFRSOf6MeYOqw7nn23Xv8FACwcP1BfNOjOf5c8j3qOtmhz8gVuPsgsbhDJTWRaWnCZ0QP/H0kHK9ep4nlq2Z+g407TyMi8n6+x5kY6WHVzG8wfNZmheNIOYmatvKKiZWKAgMDoaenh7CwMMyfPx++vr4ICgoCAHzxxRdITEzEwYMHER4ejoYNG6J9+/Z4/vy5Sm37+/vD1dUVgwcPRlxcHOLi4mBnZyfunzx5Mn7++WdERkaiXr16SElJQZcuXXDs2DFcvnwZnTp1Qvfu3XH/fv5/PPKTnp6O5ORkhe1T8Of+c2jbzBlWFY3Esm96NEebZs5wrmqDXh0bw3+aFw6F/IvYR09LMFL6EI8SXmDakp1YNasftGUV8uwX5AIA4NueLfBVt/+hrpMdZo/uhaqVLbF177niDpfUQFNDio1+AyGRSDDu521i+ZAv3aCvq40lAUcKPNZ/6lf46/BFnLnMHktSHyZWKqpXrx5mzpyJ6tWro1+/fmjcuDGOHTuG06dP4/z589ixYwcaN26M6tWrY+HChTA2NsZff/2lUttGRkbQ0tKCrq4urKysYGVlBQ0NDXG/r68vOnTogKpVq8LU1BQuLi4YOnQo6tSpg+rVq2P27NmoWrUq9uzZo/L1+Pn5wcjISNzeTuTKq4fxz3Hq4i181f1/Sus1qGUPAIh9+KQ4wiI1unLzAZ6+eIUO/RfApuVo2LQcjTOX72D9jhDYtBwNc1MDAICTo5XCcdUdLPEoIe+do1S65SZVdlYm+HzECoVep9aNa6BJXUckhC7Fk7P+uLRzJgDgeOBErJr5rVhnhFd7PDnrjydn/bF8mheMDHTx5Kw/vN7zd+KTVgJdVn5+fmjSpAkMDAxgYWGBnj17IioqSqFOWloahg8fDjMzM+jr66N3795ISEhQqHP//n107doVurq6sLCwwIQJE9Q+xYaT11VUr149hdfW1tZITEzElStXkJKSAjMzM4X9b968QXS0ev4V1LhxY4XXKSkp8PHxwf79+xEXF4esrCy8efOmUD1WU6ZMwdixY8XXycnJ5T652rY/DBVNDNDetZbSetdv59yKbWFmpLQelT6tG9fAid8nK5SNnrsV1ewtMOIbd9hXqgirika4c09x2O/u/US0e8/PBZUuuUlV1crm6P79Mrx4+Vph/+SFf2Humn3ia6uKRti5YgS++2kjwq/HAgA6frcIGhr/9S90aV0Po/q5o9OgxXicmFQcl1EmlcRdgSdPnsTw4cPRpEkTZGVl4aeffkLHjh1x48YN6OnpAQDGjBmD/fv3Y8eOHTAyMsKIESPQq1cvhIaGAgCys7PRtWtXWFlZ4cyZM4iLi0O/fv1QoUIFzJs3r0jX8zYmViqqUEFxWEEikUAulyMlJQXW1tY4ceJEnmOMjY0BAFKpFIIgKOzLzMxU+dy5PzS5xo8fj6CgICxcuBDVqlWDjo4O+vTpg4wM1Sdcy2QyyGQyleuXdXK5HNsPnEefTk2gqflfb2Dso6fYHRSOdv+rBRMjXURGx2HWsl1oVr9qntv0qfTT19OGc1XFz01XWwsmhnpi+Q9e7bBg/UHUrm6DOtVtse3Aedy5l4gN874riZCpAHo6WnC0Mxdf29uYoU6NSkh6mYr4py8R+MsguNS0g+eYNdDQkMDCLKc38sXLVGRmZefcGfhWZ0VKajoAIObREzFpuhWr2JtR37kyBEFAZHTcx704KrRDhw4pvA4ICICFhQXCw8PRunVrvHz5Ehs2bMDWrVvRrl07AMDGjRvh7OyMc+fO4X//+x+OHDmCGzdu4OjRo7C0tET9+vUxe/ZsTJo0CT4+PtDSUs8yO0ysiqhhw4aIj4+HpqYmHBwc8q1jbm6Oa9euKZRFREQoJGtaWlrIzs5W6ZyhoaHo378/Pv/8cwA5PVixsbEfFP+n4tTFW3iU8AKeXZsplGtpauDUxVtYv/0k3qRlwNrCGJ3buOBH744lFCl9bEM92yI9Iwsz/HfhRXIqalezwfZlP8DB1vz9B1Oxqe9sj32//ii+njc2Z4HXrfvO4ee1B9DFLWcU4dTWKQrHdRvqj9BLt4sv0E9QaXhW4MuXOQvBmpqaAgDCw8ORmZkJd3d3sU7NmjVRuXJlnD17Fv/73/9w9uxZ1K1bF5aW/y2v4uHhgWHDhuH69eto0KBB0YL6f0ysisjd3R2urq7o2bMn5s+fjxo1auDx48fYv38/Pv/8czRu3Bjt2rXDggULsGnTJri6uuL333/HtWvXFD5EBwcHhIWFITY2Fvr6+uIPS36qV6+OnTt3onv37pBIJJg+fTrkby2ASXm5Na2Jh6eX5im3sTTB3ytGFn9AVGx2rRqVp2xUvw4K61hR6RN66TZMmowocL+yffl5EPf8vcf8sS8Mf+wLK1S7nyJ1Lrz+7o1TqoymyOVyjB49Gi1atECdOnUAAPHx8dDS0hJHinJZWloiPj5erPN2UpW7P3efunDyehFJJBIcOHAArVu3xoABA1CjRg14enri3r174gfm4eGB6dOnY+LEiWjSpAlevXqFfv36KbQzfvx4aGhooFatWjA3N1c6X2rx4sUwMTFB8+bN0b17d3h4eKBhw4Yf9TqJiIgAqHXyup2dncKNVH5+fu89/fDhw3Ht2jX8+eef6r0uNZEI707+oU9ScnIyjIyMEPP4GQwMuQJ5eVdBg/+m+pRYuubttaPyR8jOQPrVdXj58iUMP8Lf8dzvifDbcdA3KFr7Ka+S0ai6NR48eKAQ6/t6rEaMGIF//vkHISEhcHT8b5X84OBgtG/fHi9evFDotbK3t8fo0aMxZswYzJgxA3v27EFERIS4PyYmBlWqVMGlS5fUNhTIv65ERESkMoma/gMAQ0NDha2gpEoQBIwYMQK7du1CcHCwQlIFAI0aNUKFChVw7NgxsSwqKgr379+Hq6srAMDV1RVXr15FYuJ/dwUHBQXB0NAQtWqp765gzrEiIiIi1anjkTSFPH748OHYunUr/vnnHxgYGIhzooyMjKCjowMjIyMMHDgQY8eOhampKQwNDTFy5Ei4urrif//LWZOsY8eOqFWrFr799lvMnz8f8fHxmDZtGoYPH67Wu+SZWBEREVGptnr1agAQHwWXa+PGjejfvz8AYMmSJZBKpejduzfS09Ph4eGBVatWiXU1NDSwb98+DBs2DK6urtDT04O3tzd8fX3VGisTKyIiIlKZOu8KVJUq08G1tbWxcuVKrFy5ssA69vb2OHDgQCHPXjhMrIiIiEh1JZFZlSGcvE5ERESkJuyxIiIiIpWVxLMCyxImVkRERKSy0vBIm9KMQ4FEREREasIeKyIiIlIZ564rx8SKiIiIVMfMSikmVkRERKQyTl5XjnOsiIiIiNSEPVZERESkMgnUcFegWiIpnZhYERERkco4xUo5DgUSERERqQl7rIiIiEhlXCBUOSZWREREVAgcDFSGQ4FEREREasIeKyIiIlIZhwKVY2JFREREKuNAoHIcCiQiIiJSE/ZYERERkco4FKgcEysiIiJSGZ8VqBwTKyIiIlIdJ1kpxTlWRERERGrCHisiIiJSGTuslGNiRURERCrj5HXlOBRIREREpCbssSIiIiKV8a5A5ZhYERERkeo4yUopDgUSERERqQl7rIiIiEhl7LBSjokVERERqYx3BSrHoUAiIiIiNWGPFRERERVC0e8KLM+DgUysiIiISGUcClSOQ4FEREREasLEioiIiEhNOBRIREREKuNQoHJMrIiIiEhlfKSNchwKJCIiIlIT9lgRERGRyjgUqBwTKyIiIlIZH2mjHIcCiYiIiNSEPVZERESkOnZZKcXEioiIiFTGuwKV41AgERERkZqwx4qIiIhUxrsClWNiRURERCrjFCvlmFgRERGR6phZKcU5VkRERERqwh4rIiIiUhnvClSOiRURERGpjJPXlWNiRQAAQRAAAK9eJZdwJFQcNDU4C+BTImRnlHQIVAxyP+fcv+cfS3Jy0b8n1NFGacXEigAAr169AgDUc3Is4UiIiKgoXr16BSMjI7W3q6WlBSsrK1R3tFNLe1ZWVtDS0lJLW6WJRPjYqS2VCXK5HI8fP4aBgQEk5bmP9h3Jycmws7PDgwcPYGhoWNLh0EfEz/rT8al+1oIg4NWrV7CxsYFU+nF6pdPS0pCRoZ4eUC0tLWhra6ulrdKEPVYEAJBKpbC1tS3pMEqMoaHhJ/UH+FPGz/rT8Sl+1h+jp+pt2tra5TIZUidOtCAiIiJSEyZWRERERGrCxIo+aTKZDDNnzoRMJivpUOgj42f96eBnTSWJk9eJiIiI1IQ9VkRERERqwsSKiIiISE2YWBERERGpCRMr+mQEBATA2NhYfO3j44P69euXWDxUOkgkEuzevRsAEBsbC4lEgoiIiA9uTx1tUOnk4OCApUuXlnQYVMoxsaIy50MToi+//BK3bt1Sf0BqwD/Y+Svu98XOzg5xcXGoU6eOSvX79++Pnj17FqkN+njatGmD0aNHl3QY9Inhyuv0ydDR0YGOjk5Jh0Fqlp2dDYlEopZHeGhoaMDKyqrE26DiIwgCsrOzoanJr0NSD/ZYUbGTy+Xw8/ODo6MjdHR04OLigr/++gsAcOLECUgkEhw7dgyNGzeGrq4umjdvjqioKAA5w3mzZs3ClStXIJFIIJFIEBAQAABYvHgx6tatCz09PdjZ2eGHH35ASkqKeN53hwLfldv7MG/ePFhaWsLY2Bi+vr7IysrChAkTYGpqCltbW2zcuFHhuAcPHqBv374wNjaGqakpevTogdjY2DztLly4ENbW1jAzM8Pw4cORmZkJIOdf1ffu3cOYMWPEayor5HI55s+fj2rVqkEmk6Fy5cqYO3cuAODq1ato164ddHR0YGZmhiFDhih8Hh/6vuR+jnv27EGtWrUgk8lw//59XLhwAR06dEDFihVhZGQENzc3XLp0SSHe27dvo3Xr1tDW1katWrUQFBSksD+/Ybzr16+jW7duMDQ0hIGBAVq1aoXo6Gj4+PggMDAQ//zzjxjfiRMn8m3j5MmTaNq0KWQyGaytrTF58mRkZWWJ+9u0aYNRo0Zh4sSJMDU1hZWVFXx8fNTxEZVa77vmpKQkDBo0CObm5jA0NES7du1w5coVcX9+vYWjR49GmzZtxP0nT56Ev7+/+PnExsaKf2MOHjyIRo0aQSaT4fTp04iOjkaPHj1gaWkJfX19NGnSBEePHi2Gd4LKGyZWVOz8/PywadMmrFmzBtevX8eYMWPwzTff4OTJk2KdqVOnYtGiRbh48SI0NTXx3XffAcgZzhs3bhxq166NuLg4xMXF4csvvwSQ87zDZcuW4fr16wgMDERwcDAmTpxYqNiCg4Px+PFjhISEYPHixZg5cya6desGExMThIWF4fvvv8fQoUPx8OFDAEBmZiY8PDxgYGCAU6dOITQ0FPr6+ujUqZPCg0qPHz+O6OhoHD9+HIGBgQgICBATwp07d8LW1ha+vr7iNZUVU6ZMwc8//4zp06fjxo0b2Lp1KywtLfH69Wt4eHjAxMQEFy5cwI4dO3D06FGMGDFC4fgPfV9SU1Pxyy+/YP369bh+/TosLCzw6tUreHt74/Tp0zh37hyqV6+OLl264NWrVwByksBevXpBS0sLYWFhWLNmDSZNmqT0+h49eoTWrVtDJpMhODgY4eHh+O6775CVlYXx48ejb9++6NSpkxhf8+bN822jS5cuaNKkCa5cuYLVq1djw4YNmDNnjkK9wMBA6OnpISwsDPPnz4evr2+exK+8UXbNX3zxBRITE3Hw4EGEh4ejYcOGaN++PZ4/f65S2/7+/nB1dcXgwYPFz8fOzk7cP3nyZPz888+IjIxEvXr1kJKSgi5duuDYsWO4fPkyOnXqhO7du+P+/fsf5dqpHBOIilFaWpqgq6srnDlzRqF84MCBwldffSUcP35cACAcPXpU3Ld//34BgPDmzRtBEARh5syZgouLy3vPtWPHDsHMzEx8vXHjRsHIyEh8/W473t7egr29vZCdnS2WOTk5Ca1atRJfZ2VlCXp6esIff/whCIIgbN68WXBychLkcrlYJz09XdDR0REOHz6s0G5WVpZY54svvhC+/PJL8bW9vb2wZMmS915TaZKcnCzIZDJh3bp1efatXbtWMDExEVJSUsSy/fv3C1KpVIiPjxcE4cPfl40bNwoAhIiICKXxZWdnCwYGBsLevXsFQRCEw4cPC5qamsKjR4/EOgcPHhQACLt27RIEQRBiYmIEAMLly5cFQRCEKVOmCI6OjkJGRka+5/D29hZ69OihUPZuGz/99FOen5GVK1cK+vr64s+am5ub0LJlS4V2mjRpIkyaNEnpNZZlyq751KlTgqGhoZCWlqawv2rVqsKvv/4qCEL+7/2PP/4ouLm5KZzjxx9/VKiT+zdm9+7d742xdu3awvLly8XXZfH3lIofB5WpWN25cwepqano0KGDQnlGRgYaNGggvq5Xr574/9bW1gCAxMREVK5cucC2jx49Cj8/P9y8eRPJycnIyspCWloaUlNToaurq1J8tWvXVpirY2lpqTAJWUNDA2ZmZkhMTAQAXLlyBXfu3IGBgYFCO2lpaYiOjlZoV0NDQ+Garl69qlJMpVVkZCTS09PRvn37fPe5uLhAT09PLGvRogXkcjmioqJgaWkJ4MPfFy0tLYWfEQBISEjAtGnTcOLECSQmJiI7Oxupqalij0NkZCTs7OxgY2MjHuPq6qr0PBEREWjVqhUqVKjw3pgKEhkZCVdXV4Uh3hYtWiAlJQUPHz4Uf6bfvR5ra2vx56y8Kuiar1y5gpSUFJiZmSnsf/PmjcLvVVE0btxY4XVKSgp8fHywf/9+xMXFISsrC2/evGGPFRUaEysqVrlzbPbv349KlSop7JPJZOIfzbe/yHK/kORyeYHtxsbGolu3bhg2bBjmzp0LU1NTnD59GgMHDkRGRobKidW7X6ASiSTfstxYUlJS0KhRI2zZsiVPW+bm5krbVXY9ZYE6bgT40PdFR0cnz1w0b29vPHv2DP7+/rC3t4dMJoOrq6vCkGxhFefNDuXxZ+R9CrrmlJQUWFtb48SJE3mOyZ0nKZVKIbzzRLbc+XmqeDvpB4Dx48cjKCgICxcuRLVq1aCjo4M+ffoU6eeHPk1MrKhYvT3Z2M3NLc9+Vf41qqWlhezsbIWy8PBwyOVyLFq0SOxx2r59u3qCVqJhw4bYtm0bLCwsYGho+MHt5HdNpV316tWho6ODY8eOYdCgQQr7nJ2dERAQgNevX4tfYKGhoZBKpXByclL5HIV5X0JDQ7Fq1Sp06dIFQM5NBU+fPlWI6cGDB4iLixN7Qc+dO6e0zXr16iEwMBCZmZn59lqpEp+zszP+/vtvCIIgJoOhoaEwMDCAra2tStf2qWnYsCHi4+OhqakJBweHfOuYm5vj2rVrCmUREREKn1Nhf3769++Pzz//HEDOP5revgmFSFWcvE7FysDAAOPHj8eYMWMQGBiI6OhoXLp0CcuXL0dgYKBKbTg4OCAmJgYRERF4+vQp0tPTUa1aNWRmZmL58uW4e/cuNm/ejDVr1nzkqwG8vLxQsWJF9OjRA6dOnUJMTAxOnDiBUaNGiRPcVeHg4ICQkBA8evRIIRkozbS1tTFp0iRMnDgRmzZtQnR0NM6dO4cNGzbAy8sL2tra8Pb2xrVr13D8+HGMHDkS3377rTgMqIrCvC/Vq1fH5s2bERkZibCwMHh5eSn0OLm7u6NGjRrw9vbGlStXcOrUKUydOlVpmyNGjEBycjI8PT1x8eJF3L59G5s3bxbvUnVwcMC///6LqKgoPH36NN8ekx9++AEPHjzAyJEjcfPmTfzzzz+YOXMmxo4dq5YlIsojd3d3uLq6omfPnjhy5AhiY2Nx5swZTJ06FRcvXgQAtGvXDhcvXsSmTZtw+/ZtzJw5M0+i5eDggLCwMMTGxuLp06dKewCrV6+OnTt3IiIiAleuXMHXX39d7nsM6ePgbzUVu9mzZ2P69Onw8/ODs7MzOnXqhP3798PR0VGl43v37o1OnTqhbdu2MDc3xx9//AEXFxcsXrwYv/zyC+rUqYMtW7bAz8/vI18JoKuri5CQEFSuXBm9evWCs7MzBg4ciLS0tEL1YPn6+iI2NhZVq1ZVGEIs7aZPn45x48ZhxowZcHZ2xpdffonExETo6uri8OHDeP78OZo0aYI+ffqgffv2WLFiRaHaL8z7smHDBrx48QINGzbEt99+i1GjRsHCwkLcL5VKsWvXLrx58wZNmzbFoEGDxKUhCmJmZobg4GCkpKTAzc0NjRo1wrp168RekcGDB8PJyQmNGzeGubk5QkND87RRqVIlHDhwAOfPn4eLiwu+//57DBw4ENOmTSvUe/EpkUgkOHDgAFq3bo0BAwagRo0a8PT0xL1798TE3MPDA9OnT8fEiRPRpEkTvHr1Cv369VNoZ/z48dDQ0ECtWrVgbm6udL7U4sWLYWJigubNm6N79+7w8PBAw4YNP+p1UvkkEd4dpCYiIiKiD8IeKyIiIiI1YWJFREREpCZMrIiIiIjUhIkVERERkZowsSIiIiJSEyZWRERERGrCxIqIiIhITZhYEVGp0b9/f/Ts2VN83aZNG4wePbrY4zhx4gQkEgmSkpIKrCORSLB7926V2/Tx8UH9+vWLFFdsbCwkEgkiIiKK1A4RfTxMrIhIqf79+0MikUAikUBLSwvVqlWDr68vsrKyPvq5d+7cidmzZ6tUV5VkiIjoY+NDmInovTp16oSNGzciPT0dBw4cwPDhw1GhQgVMmTIlT92MjAxoaWmp5bympqZqaYeIqLiwx4qI3ksmk8HKygr29vYYNmwY3N3dsWfPHgD/Dd/NnTsXNjY2cHJyAgA8ePAAffv2hbGxMUxNTdGjRw/ExsaKbWZnZ2Ps2LEwNjaGmZkZJk6ciHefsPXuUGB6ejomTZoEOzs7yGQyVKtWDRs2bEBsbCzatm0LADAxMYFEIkH//v0BAHK5HH5+fnB0dISOjg5cXFzw119/KZznwIEDqFGjBnR0dNC2bVuFOFU1adIk1KhRA7q6uqhSpQqmT5+e70OZf/31V9jZ2UFXVxd9+/bFy5cvFfavX78ezs7O0NbWRs2aNbFq1apCx0JEJYeJFREVmo6ODjIyMsTXx44dQ1RUFIKCgrBv3z5kZmbCw8MDBgYGOHXqFEJDQ6Gvr49OnTqJxy1atAgBAQH47bffcPr0aTx//hy7du1Set5+/frhjz/+wLJlyxAZGYlff/0V+vr6sLOzw99//w0AiIqKQlxcHPz9/QEAfn5+2LRpE9asWYPr169jzJgx+Oabb3Dy5EkAOQlgr1690L17d0RERGDQoEGYPHlyod8TAwMDBAQE4MaNG/D398e6deuwZMkShTp37tzB9u3bsXfvXhw6dAiXL1/GDz/8IO7fsmULZsyYgblz5yIyMhLz5s3D9OnTERgYWOh4iKiECERESnh7ews9evQQBEEQ5HK5EBQUJMhkMmH8+PHifktLSyE9PV08ZvPmzYKTk5Mgl8vFsvT0dEFHR0c4fPiwIAiCYG1tLcyfP1/cn5mZKdja2ornEgRBcHNzE3788UdBEAQhKipKACAEBQXlG+fx48cFAMKLFy/EsrS0NEFXV1c4c+aMQt2BAwcKX331lSAIgjBlyhShVq1aCvsnTZqUp613ARB27dpV4P4FCxYIjRo1El/PnDlT0NDQEB4+fCiWHTx4UJBKpUJcXJwgCIJQtWpVYevWrQrtzJ49W3B1dRUEQRBiYmIEAMLly5cLPC8RlSzOsSKi99q3bx/09fWRmZkJuVyOr7/+Gj4+PuL+unXrKsyrunLlCu7cuQMDAwOFdtLS0hAdHY2XL18iLi4OzZo1E/dpamqicePGeYYDc0VEREBDQwNubm4qx33nzh2kpqaiQ4cOCuUZGRlo0KABACAyMlIhDgBwdXVV+Ry5tm3bhmXLliE6OhopKSnIysqCoaGhQp3KlSujUqVKCueRy+WIioqCgYEBoqOjMXDgQAwePFisk5WVBSMjo0LHQ0Qlg4kVEb1X27ZtsXr1amhpacHGxgaamop/OvT09BRep6SkoFGjRtiyZUuetszNzT8oBh0dnUIfk5KSAgDYv3+/QkID5MwbU5ezZ8/Cy8sLs2bNgoeHB4yMjPDnn39i0aJFhY513bp1eRI9DQ0NtcVKRB8XEysiei89PT1Uq1ZN5foNGzbEtm3bYGFhkafXJpe1tTXCwsLQunVrADk9M+Hh4WjYsGG+9evWrQu5XI6TJ0/C3d09z/7cHrPs7GyxrFatWpDJZLh//36BPV3Ozs7iRPxc586de/9FvuXMmTOwt7fH1KlTxbJ79+7lqXf//n08fvwYNjY24nmkUimcnJxgaWkJGxsb3L17F15eXoU6PxGVHpy8TkRq5+XlhYoVK6JHjx44deoUYmJicOLECYwaNQoPHz4EAPz444/4+eefsXv3bty8eRM//PCD0jWoHBwc4O3tje+++w67d+8W29y+fTsAwN7eHhKJBPv27cOTJ0+QkpICAwMDjB8/HmPGjEFgYCCio6Nx6dIlLF++XJwQ/v333+P27duYMGECoqKisHXrVgQEBBTqeqtXr4779+/jzz//RHR0NJYtW5bvRHxtbW14e3vjypUrOHXqFEaNGoW+ffvCysoKADBr1iz4+flh2bJluHXrFq5evYqNGzdi8eLFhYqHiEoOEysiUjtdXV2EhISgcuXK6NWrF5ydnTFw4ECkpaWJPVjjxo3Dt99+C29vb7i6usLAwACff/650nZXr16NPn364IcffkDNmjUxePBgvH79GgBQqVIlzJo1C5MnT4alpSVGjBgBAJg9ezamT58OPz8/ODs7o1OnTti/fz8cHR0B5Mx7+vvvv7F79264uLhgzZo1mDdvXqGu97PPPsOYMWMwYsQI1K9fH2fOnMH06dPz1KtWrRp69eqFLl26oGPHjqhXr57CcgqDBg3C+vXrsXHjRtStWxdubm4ICAgQYyWi0k8iFDRTlIiIiIgKhT1WRERERGrCxIqIiIhITZhYEREREakJEysiIiIiNWFiRURERKQmTKyIiIiI1ISJFREREZGaMLEiIiIiUhMmVkRERERqwsSKiIiISE2YWBERERGpCRMrIiIiIjX5P25sCLA23zMMAAAAAElFTkSuQmCC\n"},"metadata":{}}],"source":["# ===============================\n","# ✅ 評価結果 + 混同行列\n","# ===============================\n","from sklearn.metrics import classification_report\n","\n","pred_result = trainer.predict(test_enc)\n","pred_bert = pred_result.predictions.argmax(axis=1)\n","\n","print(\"=== BERT Fine-tuned ===\")\n","print(classification_report(df_test[\"label\"],\n"," pred_bert,\n"," target_names=label2id.keys()))\n","plot_confusion(\"BERT Fine-tuned\",\n"," df_test[\"label\"],\n"," pred_bert,\n"," labels=list(label2id.keys()))"]},{"cell_type":"code","execution_count":null,"id":"4daf6a78","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":872},"id":"4daf6a78","executionInfo":{"status":"ok","timestamp":1747903169130,"user_tz":-540,"elapsed":16645,"user":{"displayName":"TOMA Naruaki","userId":"11747312442870110137"}},"outputId":"b012f494-692a-4422-f760-face1f555074"},"outputs":[{"output_type":"display_data","data":{"text/plain":[""],"text/html":[]},"metadata":{}},{"output_type":"stream","name":"stdout","text":["pred_bert_val[0]=np.int64(2)\n"]},{"output_type":"execute_result","data":{"text/plain":[" sent1 sent2 \\\n","0 トランクのなかでネコが伏せています。 トランクのなかでねこが伏せています。 \n","1 木々の生える草原を歩く小さい小象と大きい象がいます。 広い草原の中に２頭の象が歩いている。 \n","2 黒い犬に寄り掛かっているテディベア。 黒い犬に熊のぬいぐるみがもたれさせてあります。 \n","3 線路上で鉄道車両が通過待ちをしています。 線路上を鉄道車両が通過するところです。 \n","4 駅構内に消火栓が設置されています。 草地の中に消火栓が設置されています。 \n","5 駅のプラットホームを電車が進んでいます。 駅のプラットホームに電車が停車しています。 \n","6 クマがコンクリートの上に座っています。 クマが壁にもたれて座っています。 \n","7 子どもたちがテーブルの前で食事をしています。 子供たちがケーキを前に笑っています。 \n","8 ピンクの服を着た人と緑色の服を着た人の前の机に箱に入ったドーナツが置かれています。 ドーナツの前で女性が男性と話しています。 \n","\n"," true_label pred_label true_label_name pred_label_name \n","0 0 1 entailment contradiction \n","1 0 2 entailment neutral \n","2 0 2 entailment neutral \n","3 1 0 contradiction entailment \n","4 1 2 contradiction neutral \n","5 1 0 contradiction entailment \n","6 2 1 neutral contradiction \n","7 2 1 neutral contradiction \n","8 2 1 neutral contradiction "],"text/html":["\n","
\n","
\n","\n","\n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n","
sent1sent2true_labelpred_labeltrue_label_namepred_label_name
0トランクのなかでネコが伏せています。トランクのなかでねこが伏せています。01entailmentcontradiction
1木々の生える草原を歩く小さい小象と大きい象がいます。広い草原の中に２頭の象が歩いている。02entailmentneutral
2黒い犬に寄り掛かっているテディベア。黒い犬に熊のぬいぐるみがもたれさせてあります。02entailmentneutral
3線路上で鉄道車両が通過待ちをしています。線路上を鉄道車両が通過するところです。10contradictionentailment
4駅構内に消火栓が設置されています。草地の中に消火栓が設置されています。12contradictionneutral
5駅のプラットホームを電車が進んでいます。駅のプラットホームに電車が停車しています。10contradictionentailment
6クマがコンクリートの上に座っています。クマが壁にもたれて座っています。21neutralcontradiction
7子どもたちがテーブルの前で食事をしています。子供たちがケーキを前に笑っています。21neutralcontradiction
8ピンクの服を着た人と緑色の服を着た人の前の机に箱に入ったドーナツが置かれています。ドーナツの前で女性が男性と話しています。21neutralcontradiction
\n","
\n","
\n","\n","
\n"," \n","\n"," \n","\n"," \n","
\n","\n","\n","
\n"," \n","\n","\n","\n"," \n","
\n","\n","
\n"," \n"," \n"," \n","
\n","\n","
\n","
\n"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"dataframe","variable_name":"mis_df","summary":"{\n \"name\": \"mis_df\",\n \"rows\": 9,\n \"fields\": [\n {\n \"column\": \"sent1\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 9,\n \"samples\": [\n \"\\u5b50\\u3069\\u3082\\u305f\\u3061\\u304c\\u30c6\\u30fc\\u30d6\\u30eb\\u306e\\u524d\\u3067\\u98df\\u4e8b\\u3092\\u3057\\u3066\\u3044\\u307e\\u3059\\u3002\",\n \"\\u6728\\u3005\\u306e\\u751f\\u3048\\u308b\\u8349\\u539f\\u3092\\u6b69\\u304f\\u5c0f\\u3055\\u3044\\u5c0f\\u8c61\\u3068\\u5927\\u304d\\u3044\\u8c61\\u304c\\u3044\\u307e\\u3059\\u3002\",\n \"\\u99c5\\u306e\\u30d7\\u30e9\\u30c3\\u30c8\\u30db\\u30fc\\u30e0\\u3092\\u96fb\\u8eca\\u304c\\u9032\\u3093\\u3067\\u3044\\u307e\\u3059\\u3002\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"sent2\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 9,\n \"samples\": [\n \"\\u5b50\\u4f9b\\u305f\\u3061\\u304c\\u30b1\\u30fc\\u30ad\\u3092\\u524d\\u306b\\u7b11\\u3063\\u3066\\u3044\\u307e\\u3059\\u3002\",\n \"\\u5e83\\u3044\\u8349\\u539f\\u306e\\u4e2d\\u306b\\uff12\\u982d\\u306e\\u8c61\\u304c\\u6b69\\u3044\\u3066\\u3044\\u308b\\u3002\",\n \"\\u99c5\\u306e\\u30d7\\u30e9\\u30c3\\u30c8\\u30db\\u30fc\\u30e0\\u306b\\u96fb\\u8eca\\u304c\\u505c\\u8eca\\u3057\\u3066\\u3044\\u307e\\u3059\\u3002\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"true_label\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0,\n \"min\": 0,\n \"max\": 2,\n \"num_unique_values\": 3,\n \"samples\": [\n 0,\n 1,\n 2\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"pred_label\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0,\n \"min\": 0,\n \"max\": 2,\n \"num_unique_values\": 3,\n \"samples\": [\n 1,\n 2,\n 0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"true_label_name\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 3,\n \"samples\": [\n \"entailment\",\n \"contradiction\",\n \"neutral\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"pred_label_name\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 3,\n \"samples\": [\n \"contradiction\",\n \"neutral\",\n \"entailment\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}"}},"metadata":{},"execution_count":36}],"source":["# 検証データに対する失敗事例分析\n","# trainer.predictでは「ラベルに対する確信度スコア」が出力されるだけであり、ラベルにはなっていない。\n","# 確信度スコアから最大確信度（argmax）となるインデックス（=予測ラベル）に変換してから分析する必要がある。\n","pred_bert = trainer.predict(val_enc)\n","pred_bert_val = pred_bert.predictions.argmax(axis=1)\n","print(f\"{pred_bert_val[0]=}\")\n","\n","mis_df = show_misclassified_examples(\n"," df_val[\"sent1\"], df_val[\"sent2\"], df_val[\"label\"], pred_bert_val, id2label\n",")\n","\n","mis_df"]},{"cell_type":"code","execution_count":null,"id":"24e4088a","metadata":{"id":"24e4088a"},"outputs":[],"source":["# 検証データに対する全失敗事例をExcelファイルとして出力\n","\n","true_label = df_val[\"label\"].map(id2label)\n","pred_label = pd.Series(pred_bert.label_ids).map(id2label)\n","\n","mis_all_df = pd.DataFrame({\n"," \"sent1\":df_val[\"sent1\"],\n"," \"sent2\":df_val[\"sent2\"],\n"," \"true_label\":df_val[\"label\"],\n"," \"bert_pred_label\":pred_bert.label_ids,\n"," \"true_label_name\":true_label,\n"," \"bert_pred_label_name\":pred_label\n","})\n","\n","mis_all_df.to_excel(\"mis_all_df_bert.xlsx\")"]}],"metadata":{"kernelspec":{"display_name":"Python 3","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.9.6"},"colab":{"provenance":[],"gpuType":"T4","toc_visible":true},"accelerator":"GPU","widgets":{"application/vnd.jupyter.widget-state+json":{"9cbda31570034704bf5d27805db6601c":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_7409761697934ddda9efedf4f662ba0a","IPY_MODEL_5528a60527724dbeb3de8599a0390879","IPY_MODEL_13a5f22eb6454b19aeda5991ad804df5"],"layout":"IPY_MODEL_125429ee6cd24efd9fe53c1a9ae8a9f4"}},"7409761697934ddda9efedf4f662ba0a":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_fec95af60e42488596aa97a2c3546cab","placeholder":"​","style":"IPY_MODEL_1951f0d6323e48efa5578905888c32e0","value":"Map: 100%"}},"5528a60527724dbeb3de8599a0390879":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_5940f885e5f94f30ac722eb5b0d877e3","max":20073,"min":0,"orientation":"horizontal","style":"IPY_MODEL_6fc40462f2b8417c833169472e598590","value":20073}},"13a5f22eb6454b19aeda5991ad804df5":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_a48d578797834f2b847ad288b7135e37","placeholder":"​","style":"IPY_MODEL_6a6fc4561b41403c94e5e688fc42827f","value":" 20073/20073 [00:07<00:00, 2884.21 examples/s]"}},"125429ee6cd24efd9fe53c1a9ae8a9f4":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"fec95af60e42488596aa97a2c3546cab":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"1951f0d6323e48efa5578905888c32e0":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"5940f885e5f94f30ac722eb5b0d877e3":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6fc40462f2b8417c833169472e598590":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"a48d578797834f2b847ad288b7135e37":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6a6fc4561b41403c94e5e688fc42827f":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"1846836b9f434c769bd8c0eee6e56bc3":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_a0ae15cc12e74645957be9706c906bf1","IPY_MODEL_55ab9342b2bf46b497a7d20afd7170f3","IPY_MODEL_9daf16e8f2a341019ce302b513995c35"],"layout":"IPY_MODEL_cfb85f1c42024e6c8b56593486f22b8b"}},"a0ae15cc12e74645957be9706c906bf1":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_ac4687fa8a374c04bd2a1e79f9e7203d","placeholder":"​","style":"IPY_MODEL_9058799eaa0e41fda66a776134cdfb06","value":"Map: 100%"}},"55ab9342b2bf46b497a7d20afd7170f3":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_07f45f1b5ec14c2999d70d20d84fbfc5","max":2434,"min":0,"orientation":"horizontal","style":"IPY_MODEL_3bbcf597ef244b4f991c572f6429c363","value":2434}},"9daf16e8f2a341019ce302b513995c35":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_6c9c6b834a5348fb84c95718dbe83c6e","placeholder":"​","style":"IPY_MODEL_0fa4d46ba53f4b25a761ab5039c11535","value":" 2434/2434 [00:00<00:00, 2838.81 examples/s]"}},"cfb85f1c42024e6c8b56593486f22b8b":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ac4687fa8a374c04bd2a1e79f9e7203d":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9058799eaa0e41fda66a776134cdfb06":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"07f45f1b5ec14c2999d70d20d84fbfc5":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"3bbcf597ef244b4f991c572f6429c363":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"6c9c6b834a5348fb84c95718dbe83c6e":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"0fa4d46ba53f4b25a761ab5039c11535":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"84d927c94551447a875aa584c256a163":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_6550acf3cece4444a623e2288eb81169","IPY_MODEL_11156708cbf84c2f82f339f58a394a9a","IPY_MODEL_f50a59d98b8d4d6bbd409f3c9dfcab6d"],"layout":"IPY_MODEL_272efb41f56d4dd3aacc4fe529e90698"}},"6550acf3cece4444a623e2288eb81169":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_c510aef81f47445c95f43219b5057d08","placeholder":"​","style":"IPY_MODEL_bf1edd12a5704b9c970a76ec5f4e505e","value":"Map: 100%"}},"11156708cbf84c2f82f339f58a394a9a":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_4ec025499441479b9c0a08b03d7eb3ef","max":2508,"min":0,"orientation":"horizontal","style":"IPY_MODEL_20d17fd09d07498592386850daf97438","value":2508}},"f50a59d98b8d4d6bbd409f3c9dfcab6d":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_6ae28d0c6eeb4a37918ee4e1b355c872","placeholder":"​","style":"IPY_MODEL_aceca98d40e54ee98a9066e1cc10e545","value":" 2508/2508 [00:00<00:00, 2850.46 examples/s]"}},"272efb41f56d4dd3aacc4fe529e90698":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c510aef81f47445c95f43219b5057d08":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"bf1edd12a5704b9c970a76ec5f4e505e":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"4ec025499441479b9c0a08b03d7eb3ef":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"20d17fd09d07498592386850daf97438":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"6ae28d0c6eeb4a37918ee4e1b355c872":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"aceca98d40e54ee98a9066e1cc10e545":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}}}}},"nbformat":4,"nbformat_minor":5}