{
"cells": [
{
"cell_type": "markdown",
"id": "9479f1e0",
"metadata": {},
"source": [
"# 量子神经网络的表达能力\n",
"\n",
"Copyright (c) 2021 Institute for Quantum Computing, Baidu Inc. All Rights Reserved."
]
},
{
"cell_type": "markdown",
"id": "c352b2aa",
"metadata": {},
"source": [
"## 概览"
]
},
{
"cell_type": "markdown",
"id": "3a195bcc",
"metadata": {},
"source": [
"在量子机器学习中,量子神经网络的**表达能力**是决定量子机器学习任务能否成功的关键因素。一般来说,量子神经网络拟设的表达能力越强,量子机器学习能够搜索到全局最优解的可能性就越大。本教程首先介绍量子神经网络表达能力的基本概念,随后将在量桨中展示如何通过布洛赫球感受不同拟设表达能力的差别。最后介绍一种定量分析量子神经网络表达能力的方法,并评估量桨提供的量子神经网络模板在不同深度下的表达能力。"
]
},
{
"cell_type": "markdown",
"id": "0f4583d7",
"metadata": {},
"source": [
"## 基本概念"
]
},
{
"cell_type": "markdown",
"id": "c03d7dcf",
"metadata": {},
"source": [
"我们先来回顾一下量子机器学习算法的基本流程。在量子机器学习中,我们往往设计一个损失函数 $\\mathcal{L}$,并通过优化一个酉变换 $U$ 使得损失函数最小化:\n",
"\n",
"$$\n",
"\\min_U\\mathcal{L}(U)=\\min_U \\text{tr}[HU\\rho_{in}U^\\dagger],\\tag{1}\n",
"$$\n",
"\n",
"算法背后的数学原理保证当我们遍历完所有可能的酉变换后,损失函数能取到的最小值就对应于我们问题的解。而在实际操作中,我们采用量子神经网络将酉变换参数化:\n",
"\n",
"$$\n",
"U=U(\\vec{\\theta})=U_D(\\vec{\\theta}_D)\\dots U_1(\\vec{\\theta}_1),\\tag{2}\n",
"$$\n",
"\n",
"其中每个 $U_j(\\vec{\\theta}_j),j\\in[1,D]$ 代表一层量子神经网络,$\\vec{\\theta}_j$ 代表该层对应的参数。此时,通过调整量子神经网络中的参数 $\\vec{\\theta}$,我们就可以进行对酉变换 $U$ 的优化,进而最小化损失函数 $\\mathcal{L}$:\n",
"\n",
"$$\n",
"\\min_{\\vec{\\theta}}\\mathcal{L}(\\vec{\\theta})=\\min_{\\vec{\\theta}} \\text{tr}[HU(\\vec{\\theta})\\rho_{in}U(\\vec{\\theta})^\\dagger].\\tag{3}\n",
"$$\n",
"\n",
"然而,细心的读者此时可能已经发现了量子神经网络的一个不足:对于一个给定的神经网络拟设,**遍历所有的参数并不一定可以保证遍历所有的酉变换**。作为一个简单的例子,如果我们只允许使用一个 $R_Y$ 旋转门作为单比特量子神经网络 $U(\\theta)=R_Y(\\theta)$,显然(除去全局相位)$U(\\theta)$ 不能表示任何矩阵元含虚部的复酉矩阵。而当允许使用 $R_Y$ 和 $R_Z$ 旋转门时,如果我们搭建量子神经网络为 $U(\\vec{\\theta})=R_Z(\\theta_1)R_Y(\\theta_2)R_Z(\\theta_3)$,$U(\\vec{\\theta})$ (除去全局相位)将能够表示所有的单比特酉矩阵 [1]。\n",
"\n",
"如果我们将神经网络的表达能力定义为**在遍历电路参数 $\\vec{\\theta}$ 时电路能够表达的酉变换的多少**,那么,一个表达能力强的量子神经网络将更有可能包含那些使得损失函数 $\\mathcal{L}$ 取到全局最小值的酉变换;相反地,如果一个量子神经网络 $U_{weak}$ 的表达能力太弱以至于不包含任何能将损失函数最小化的酉变换,那么基于优化 $U_{weak}$ 的量子机器学习任务就很可能会失败。\n",
"\n",
"接下来我们基于量桨,通过观察单量子比特酉门遍历布洛赫球的能力直观地感受量子神经网络的表达能力。"
]
},
{
"cell_type": "markdown",
"id": "3412d8c6",
"metadata": {},
"source": [
"## 直观感受表达能力:遍历布洛赫球"
]
},
{
"cell_type": "markdown",
"id": "5bd49ecf",
"metadata": {},
"source": [
"对于单量子比特的简单情况,我们可以直接观察量子神经网络如何将固定输入遍历布洛赫球表面。对于一个给定的神经网络拟设 $U(\\vec{\\theta})$,由于网络的输入往往是固定的(不妨设为 $|0\\rangle$),通过均匀地采样神经网络参数 $\\vec{\\theta}$,神经网络的输出态 $U(\\vec{\\theta})|0\\rangle$ 将散落在布洛赫球表面。显然,如果输出态在球面分布地越广越均匀,那么神经网络拟设 $U$ 的表达能力也就越强,包含损失函数全局最优的可能性也就越大。\n",
"\n",
"为在量桨实现这一功能,首先引入必要的包:"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "e05a5803",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/home/zl/miniconda3/envs/pq/lib/python3.8/site-packages/openfermion/hamiltonians/hartree_fock.py:11: DeprecationWarning: Please use `OptimizeResult` from the `scipy.optimize` namespace, the `scipy.optimize.optimize` namespace is deprecated.\n",
" from scipy.optimize.optimize import OptimizeResult\n",
"/home/zl/miniconda3/envs/pq/lib/python3.8/site-packages/paddle/tensor/creation.py:125: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe. \n",
"Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations\n",
" if data.dtype == np.object:\n",
"/home/zl/miniconda3/envs/pq/lib/python3.8/site-packages/paddle/tensor/creation.py:125: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe. \n",
"Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations\n",
" if data.dtype == np.object:\n"
]
}
],
"source": [
"import numpy as np\n",
"import paddle\n",
"import paddle_quantum as pq\n",
"from paddle_quantum.ansatz.circuit import Circuit\n",
"from paddle_quantum.visual import plot_state_in_bloch_sphere"
]
},
{
"cell_type": "markdown",
"id": "14f70b6b",
"metadata": {},
"source": [
"首先,我们只允许使用一个 $R_Y$ 旋转门作为单比特量子神经网络 $U(\\theta)=R_Y(\\theta)$。通过在 $[0,2\\pi]$ 均匀采样参数 $\\theta$ 并将 $U(\\theta)$ 作用在固定输入 $|0\\rangle$ 上,我们便得到量子神经网络 $U(\\theta)$ 的输出分布。通过量桨内置的 plot_bloch_sphere_from_input 函数,我们可以直接观察 $U(\\theta)|0\\rangle$ 在布洛赫球面上的分布:"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "04344561",
"metadata": {
"scrolled": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/home/zl/miniconda3/envs/pq/lib/python3.8/site-packages/paddle/fluid/framework.py:1104: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.\n",
"Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations\n",
" elif dtype == np.bool:\n"
]
}
],
"source": [
"num_qubit = 1 # 设定量子比特数\n",
"num_sample = 2000 # 设定采样次数\n",
"outputs_y = list() # 储存采样电路输出\n",
"\n",
"for _ in range(num_sample):\n",
" # 初始化量子神经网络\n",
" pq.set_backend('density_matrix')\n",
" cir = Circuit(num_qubit)\n",
" # 作用 Ry 旋转门,旋转角度随机\n",
" cir.ry(0)\n",
" # 输出态的密度矩阵\n",
" rho = cir(pq.state.zero_state(num_qubit))\n",
" outputs_y.append(rho)\n",
" \n",
"# 量桨内置的 plot_bloch_sphere_from_input 函数\n",
"# plot_state_in_bloch_sphere(outputs_y, save_gif=True, filename='figures/bloch_y.gif')"
]
},
{
"cell_type": "markdown",
"id": "97e80f54",
"metadata": {},
"source": [
"![bloch_y.gif](./figures/expressibility-fig-bloch_y.gif)"
]
},
{
"cell_type": "markdown",
"id": "7198779e",
"metadata": {},
"source": [
"可见,量子神经网络 $U(\\theta)=R_Y(\\theta)$ 的输出只能分布于布洛赫球面上的一个圆环(尽管在圆环上的分布是均匀的)。类似地,我们考虑包含两个参数的神经网络 $U(\\vec{\\theta})=R_Y(\\theta_1)R_Z(\\theta_2)$ 和 三个参数的神经网络 $U(\\vec{\\theta})=R_Y(\\theta_1)R_Z(\\theta_2)R_Y(\\theta_3)$ 的输出分布:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "0530af37",
"metadata": {},
"outputs": [],
"source": [
"outputs_yz = list() # 储存采样电路输出\n",
"for _ in range(num_sample):\n",
" # 初始化量子神经网络\n",
" cir = Circuit(num_qubit)\n",
" # 作用 Ry 和 Rz 旋转门,旋转角度随机\n",
" cir.ry(0)\n",
" cir.rz(0)\n",
" # 输出态的密度矩阵\n",
" rho = cir(pq.state.zero_state(num_qubit))\n",
" outputs_yz.append(rho)\n",
"\n",
"# plot_state_in_bloch_sphere(outputs_yz, save_gif=True, filename='figures/bloch_yz.gif')\n",
"\n",
"\n",
"outputs_yzy = list() # 储存采样电路输出\n",
"for _ in range(num_sample):\n",
" # 初始化量子神经网络\n",
" cir = Circuit(num_qubit)\n",
" # 作用 Ry,Rz,Ry 旋转门,旋转角度随机\n",
" cir.ry(0)\n",
" cir.rz(0)\n",
" cir.ry(0)\n",
" # 输出态的密度矩阵\n",
" rho = cir(pq.state.zero_state(num_qubit))\n",
" outputs_yzy.append(rho)\n",
" \n",
"# plot_state_in_bloch_sphere(outputs_yzy, save_gif=True, filename='figures/bloch_yzy.gif')"
]
},
{
"cell_type": "markdown",
"id": "8dbde635",
"metadata": {},
"source": [
"![bloch_yz.gif](./figures/expressibility-fig-bloch_yz.gif)"
]
},
{
"cell_type": "markdown",
"id": "58b0d055",
"metadata": {},
"source": [
"![bloch_yzy.gif](./figures/expressibility-fig-bloch_yzy.gif)"
]
},
{
"cell_type": "markdown",
"id": "4e195d35",
"metadata": {},
"source": [
"可见,神经网络 $U(\\vec{\\theta})=R_Y(\\theta_1)R_Z(\\theta_2)$ 的输出可以分布在整个布洛赫球表面了,虽然在两级($|0\\rangle$ 和 $|1\\rangle$)附近的分布会更加密集;而神经网络 $U(\\vec{\\theta})=R_Y(\\theta_1)R_Z(\\theta_2)R_Y(\\theta_3)$ 的输出在球面的分布是比较均匀的。"
]
},
{
"cell_type": "markdown",
"id": "7972ed41",
"metadata": {},
"source": [
"在单量子比特的低维情形下我们可以借助布洛赫球定性观察量子神经网络的表达能力。而在一般的多量子比特应用中,我们必须借助统计数学的工具对表达能力定量分析。接下来我们将引入量子态之间保真度分布的 K-L 散度作为量化神经网络的表达能力的指标,并计算一种常见拟设的表达能力。"
]
},
{
"cell_type": "markdown",
"id": "11ca0aeb",
"metadata": {},
"source": [
"## 定量分析表达能力:K-L 散度"
]
},
{
"cell_type": "markdown",
"id": "1275e46a",
"metadata": {},
"source": [
"### 保真度分布与 K-L 散度"
]
},
{
"cell_type": "markdown",
"id": "54ae96a6",
"metadata": {},
"source": [
"在文献 [2] 中,作者提出了基于神经网络输出态之间的保真度概率分布的表达能力量化方法。对任意量子神经网络 $U(\\vec{\\theta})$,采样两次神经网络参数(设为 $\\vec{\\phi}$ 和 $\\vec{\\psi}$),则两个量子电路输出态之间的保真度 $F=|\\langle0|U(\\vec{\\phi})^\\dagger U(\\vec{\\psi})|0\\rangle|^2$ 服从某个概率分布:\n",
"$$\n",
"F\\sim{P}(f).\\tag{4}\n",
"$$\n",
"文献 [2] 指出,量子神经网络 $U$ 能够均匀地分布在所有酉矩阵上时(此时称 $U$ 服从哈尔分布),保真度的概率分布 $P_\\text{Haar}(f)$ 满足\n",
"$$\n",
"P_\\text{Haar}(f)=(2^{n}-1)(1-f)^{2^n-2}.\\tag{5}\n",
"$$\n",
"\n",
"量桨提供了直接从哈尔分布采样酉矩阵的函数。观察哈尔分布酉矩阵输出的量子态保真度服从的概率分布:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "b9dbeb68",
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEWCAYAAACXGLsWAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAbQElEQVR4nO3de7ReVXnv8e9PbpqCoiaKJoR4QU6xxwumKHqK2FaLVOBUUaGI1aEHi3jUHqxFBemwxTO01YqXI0ZFBYHSo6jRYr0VS/WINSCiEJWYRkBQkDvGSoDn/LFW6Otm773eJHu9+/J+P2O8Y7xrrrnWemZ2dp7MOdeaK1WFJEnTuc9sByBJmvtMFpKkTiYLSVInk4UkqZPJQpLUyWQhSepkstC8lWRDkt+f7ThGbbp2J/mdJD8Y4hxfT/LEjjoPTbI2yU5bG6sWDpOFNAOSPCPJ+UluSbJhtuKoqn+tqr0G4rpXYklyMHBbVX2741w/A84Hju4lWM0rJgupQ5Lth6j2C+A04M97Dmcm/ClwxpB1zwRe0WMsmidMFprvnpDk0vZ/9OckuS9Akgcm+VyS65Pc1H5ftvmgJC9th1huS7I+ySsG9h2Q5Ookf5Hkp8BHuoKoqn+rqjOA9cMEneSoJD9OckOSNw32AJJ8NMlfT4xnwil+O8nlbds+MtDue+omOQNYDnw2ye1JXp9kR+B3gX8ZOP++SdYkuTXJz5K8c+A63wQemWSPYdqlhctkofnuBcCBwCOAxwEvacvvQ/OP/B40/2D+EnjvwHHXAc8B7g+8FPi7JPsM7N8NeFB7/IwOwyTZG3g/cBTwcODBwLJpD7q3I4E/AB4FPAY4YWKFqjoKuBI4uKp2rqq3A3sCd1fVYPI5BTilqu7fnu8fBs5xJ7AOePwWxqcFxmSh+e7dVXVNVd0IfBZ4AkBV3VBVn6yqjVV1G3Ay8PTNB1XVP1bVj6rxL8AXgd8ZOO/dwElV9auq+uUMx3wY8LmquqCqfgWc2F5vS7y3qq5q230ycMSQx+0K3DahbBPw6CSLq+r2qrpwwv7b2uM0xkwWmu9+OvB9I7AzQJJFST7QDvXcClwA7Jpku3b/s5NcmOTGJDcDBwGLB851fVX9R08xPxy4avNGVf0CuGELz3HVwPcft+ccxk3ALhPKXkbTO/l+km8lec6E/bsAN29hfFpgTBZaqI4D9gKe3A6v7N+Wp70V9JPA3wIPrapdgfOADBzf53LM1wK7b95IsohmKGqzXwCLBrZ3m+Qcuw98Xw5cM8W1JrZjXXPJLL2nQtUVVXUE8BDgbcAnkvxGG9v2wKOB70zXIC18JgstVLvQzFPcnORBwEkD+3YEdgKuB+5M8mzgWV0nTFJJDphi333aSeYdms3ct51MnswngOck+W9tnbfw67+LlwAHJXlQkt2A105yjmOTLGvb9ibgnCmu9TPgkZs3quoO4MsMDMkleVGSJVV1N//Zg9g8LLYvsKGqfjzF+TUmTBZaqN4F3A/4OXAh8E+bd7RzGK+mmci9CfhjYPV0J0uyO83Y/XenqLI/TXI6j/+cUP/iZBWr6jLgWOAsml7GTcDghPMZNP+T39CeY7JEcFa7bz3wI+CvJ6kD8L+BE5LcnOR1bdkHaCbXNzsQuCzJ7TST3YcPzNMcCZw6xbk1RuLLj6RuSV4EPLaq3tDT+TcAL6+qL/dx/kmu93XgVdM9mJfkITS32D6xx/kbzRMmC2kOGHWykLaUw1CSpE72LCRJnexZSJI6DbNA2ry0ePHiWrFixWyHIUnzxkUXXfTzqloy2b4FmyxWrFjBmjVrZjsMSZo3kkz5PI3DUJKkTiYLSVInk4UkqZPJQpLUyWQhSepkspAkdTJZSJI6mSwkSZ1MFpKkTgv2Ce5x99znvowNG66bdN+KFQ/h3HM/POKIJM1nJosFasOG61i69LNT7Dt4xNFImu8chpIkdTJZSJI6mSwkSZ1MFpKkTiYLSVInk4UkqZPJQpLUyWQhSepkspAkdTJZSJI6mSwkSZ1MFpKkTiYLSVInk4UkqZPJQpLUyWQhSepkspAkdeotWSTZPcn5SS5PclmS10xS54AktyS5pP28uS1fkuRrSb6X5L8P1P9Mkof3FbMkaXJ9vlb1TuC4qro4yS7ARUm+VFWXT6j3r1X1nAllRwCnAucC5wGfTnIw8O2quqbHmCVJk+gtWVTVtcC17ffbkqwFlgITk8VkNgGLgJ2Au5JsD7wW8OXRkjQL+uxZ3CPJCuCJwDcn2b1fku8A1wCvq6rLgLPaz9HAXwCvBM6oqo0d1zm6PYbly5fPWPxzwXOf+zI2bLjuXuUrVjyEc8/98CxENL6m+lmAPw8tXL0niyQ7A58EXltVt07YfTGwR1XdnuQg4NPAnlV1C/CH7fEPBI4H/ijJB4EHAu+oqm9MvFZVrQJWAaxcubJ6atKs2LDhOpYu/ewk5Xa2Rm2qn0Wzz5+HFqZe74ZKsgNNojizqs6duL+qbq2q29vv5wE7JFk8odqJwMk08xhfA/4E+Ms+45Yk/bo+74YK8GFgbVW9c4o6u7X1SLJvG88NA/v3BJZV1Vdp5jDuBgq4X19xS5Lurc9hqKcBRwHfTXJJW/ZGYDlAVZ0KHAYck+RO4JfA4VU1OHx0MvCm9vvZNMNUxwNv7jHueWX9+h+xzz73HvpYv/7HLF06CwFJ28D5oLmrz7uhvgako857gfdOs/8FA9+vA546YwEuEHfcwaTj59///t6zEI20bZwPmrt8gluS1Gkkt85q/Hirr7SwmCzUC2/1lRYWh6EkSZ1MFpKkTg5DaexNNb/y059ezW67LbtX+Tjfluxc1PgyWWjsTTW/8v3v782TnuRtyYOcixpfDkNJkjqZLCRJnUwWkqROJgtJUieThSSpk3dDSSOwpbecuvrq6PlnPj2ThTQCW3rLqauvjp5/5tNzGEqS1MlkIUnq5DDUDFjoSyAs9PZJW2ucfjdMFjNgoS+BsNDbJ22tcfrdcBhKktTJZCFJ6uQw1CTGaRxSUrf163/EPvvce2hpnJarN1lMYpzGISV1u+MOplzGflw4DCVJ6mTPokdTdV1hfIe0pvoz2dJlL6Z6i91055K09UwWPZqq6wrjO6Q11Z/Jli57MdVb7KY7l6St5zCUJKmTyUKS1MlhqC3g7XPz21TzH+P68/MWcW2J3pJFkt2B04GHAgWsqqpTJtQJcApwELAReElVXZxkL+AsYAfgFVX1jSTbA/8EHFJVG/uKezrePje/TTf/MY68RVxbos9hqDuB46pqb+ApwLFJJv5WPhvYs/0cDby/LX8F8BqaJPK6tuwY4OOzlSgkaZz11rOoqmuBa9vvtyVZCywFLh+odihwelUVcGGSXZM8DNgELGo/m5LsChwMHNhXvJKkqY1kziLJCuCJwDcn7FoKXDWwfXVb9j6aIaydaHoZJwJvraq7O65zNE0PheXLl89E6FKvtmYebEufVdHcMZ9f3dp7skiyM/BJ4LVVdeswx1TVlcAB7fGPBpYBa5OcAewInFhVP5zkuFXAKoCVK1fWjDRA6tHWzINt6bMqmjvm86tbe711NskONInizKo6d5IqPwF2H9he1pYNOhk4AXg18CHg9cBJMx+tJGkqvSWL9k6nDwNrq+qdU1RbDbw4jacAt7RzHZvP8XTgmqq6gmb+4u72s6ivuCVJ99bnMNTTgKOA7ya5pC17I7AcoKpOBc6jueNpHc2tsy/dfHCbbE4AXtgWrQLObGM+pse4F7yFPuY9m8/DzMVrT7WO1ihi8lmO4c3138s+74b6GpCOOgUcO82+Zw5srwX2mckYx9VCH/Oezedh5uq1J1tHaxQx+SzH8Ob676XLfUiSOrnch+7hcibSzBjF79Koh/hMFrqHy5lIM2MUv0ujHuJzGEqS1MlkIUnq5DDUHOMy2pLmIpPFHOMy2pLmIoehJEmdTBaSpE4OQ82ShfBMw1RtaPbNn3ZoYZpq/m82lz+Zz0wWs2QhPNMwVRtgfrVDC9N083+ztfzJfNY5DJXkoiTHJnngKAKSJM09w/QsXkizGuy3kqwBPgJ8sV3oT5oRC2FYTlrIOpNFVa0D3pTkROA5wGnAXUk+ApxSVTf2HKPGwEIYlpMWsqHuhkryOOAdwN/QvPnu+cCtwD/3F5okaa7o7FkkuQi4meatd8dX1a/aXd9M8rQeY5MkzRHDzFk8v6rWT7ajqp47w/FImoe2dM5pa+ao5vqb5Ba6YZLFy5O8vapuBmjvijquqk7oNTJJ88aWzjltzRzVXH+T3EI3zJzFszcnCoCquonmvdmSpDExTLLYLslOmzeS3A/YaZr6kqQFZphhqDOBr7S3ykLzzMXH+gtJkobnMzqjMcxzFm9Lcinwe23RX1XVF/oNS5KG4zM6ozHU2lBV9Xng8z3HIkmao4ZZG+q5Sa5IckuSW5PcluTWUQQnSZobhulZvB04uKrW9h2MJGluGuZuqJ+ZKCRpvA3Ts1iT5Bzg08DmpT6oqnP7CkqSNLcMkyzuD2wEnjVQVoDJQpLGxDC3zr50a06c5DSaJc2vq6rfmmT/AcBngH9vi86tqrckWQJ8CtgVOKGqPt3W/wxwTFVdszXxSJK23jB3Qz0myVeSfK/dflySYdaF+ihwYEedf62qJ7Sft7RlRwCnAvsCr22veTDwbROFJM2OYSa4Pwi8AdgEUFWXAod3HVRVFwBb82KkTcAimiVF7kqyPU3SePtWnEuSNAOGSRaLqurfJpTdOUPX3y/Jd5J8Pslj27KzgEOBLwFvBV4JnFFVG7tOluToJGuSrLn++utnKERJ0jDJ4udJHkUzqU2Sw4BrZ+DaFwN7VNXjgffQ3G1FVd1SVX9YVSvbOgcDn0jywSSfSLLfVCesqlVVtbKqVi5ZsmQGQpQkwXDJ4ljgA8B/SfITmiGhY7b1wlV1a1Xd3n4/D9ghyeIJ1U4ETqaZx/ga8CfAX27rtSVJW2aYu6HWA7+f5DeA+1TVbTNx4SS70TzwV0n2pUlcNwzs3xNYVlVfTfJ44D9oejf3m4nrS5KGN8w7uN88YRuAgbuXpjrubOAAYHGSq4GTgB3aY08FDgOOSXIn8Evg8KqqgVOcDLyp/X42zTDV8cCvxSNpZk215Hezz2W/x9UwD+X9YuD7fWmenehc/qOqjujY/17gvdPsf8HA9+uAp3ZGKmmbTbXkN7js9zgbZhjqHYPbSf4W8H0WkjRGhpngnmgRsGymA5EkzV3DzFl8l/a2WWA7YAkw7XyFJGlhGWbO4jkD3++kuYNpph7KkyTNA8Mki4m3yt5/8x1RAFW1NUt6SJLmkWGSxcXA7sBNQGhWg72y3VfAI3uJTJI0Zwwzwf0lmteqLq6qB9MMS32xqh5RVSYKSRoDwySLp7TLcQBQVZ/HZx4kaawMMwx1Tfv+io+320cCvldCksbIMMniCJqlOj5FM0dxQVsmSerZVMuvjHrplWGe4L4ReE2S36iqX3TVlyTNnKmWXxn10ivDvFb1qUkup10PKsnjk/yf3iOTJM0Zw0xw/x3wB7TLh1fVd4D9+wxKkjS3DLU2VFVdNaHorh5ikSTNUcNMcF+V5KlAJdkBeA1DLFEuSVo4hulZ/CnNq1WXAj8BntBuS5LGxLQ9iyTbAadU1ZEjikeSNAdN27OoqruAPZLsOKJ4JElz0DBzFuuBrydZzcArVqvqnb1FJUmaU6bsWSQ5o/16CPC5tu4uAx9J0piYrmfxpCQPp1mO/D0jikeSNAdNlyxOBb4CPAJYM1AefI+FJI2VKYehqurdVfWbwEeq6pEDH99jIUljpvM5i6o6ZhSBSJLmrqGW+5AkjTeThSSpk8lCktTJZCFJ6tRbskhyWpLrknxviv1J8u4k65JcmmSftnyvJBe1Zfu1Zdsn+XKSRX3FK0maWp89i48CB06z/9nAnu3naOD9bfkraJZBPwh4XVt2DPDxqtrYS6SSpGkNszbUVqmqC5KsmKbKocDpVVXAhUl2TfIwYBOwqP1sSrIrcDDTJx5JUo96SxZDWAoMvoHv6rbsfcDpwE40vYwTgbdW1d1dJ0xyNE0vheXLl890vJI0tubcBHdVXVlVB1TVfsBGYBmwNskZSc5J8phpjl1VVSurauWSJUtGFrMkLXSzmSx+Auw+sL2sLRt0MnAC8GrgQ8DrgZNGEp0k6R6zmSxWAy9u74p6CnBLVV27eWeSpwPXVNUVNPMXd7cf74iSpBHrbc4iydnAAcDiJFfT9Ah2AKiqU4HzaO54Wkcz3PTSgWND06N4YVu0Cjizjde1qiRpxPq8G+qIjv0FHDvNvmcObK8F9pnRACVJQ5tzE9ySpLnHZCFJ6mSykCR1MllIkjqZLCRJnUwWkqROJgtJUieThSSpk8lCktTJZCFJ6mSykCR1MllIkjqZLCRJnUwWkqROJgtJUieThSSpk8lCktTJZCFJ6mSykCR1MllIkjqZLCRJnUwWkqROJgtJUieThSSpk8lCktTJZCFJ6mSykCR1MllIkjr1miySHJjkB0nWJTl+kv0vSXJ9kkvaz8vb8r2SXJTk0iT7tWXbJ/lykkV9xixJurft+zpxku2A9wHPBK4GvpVkdVVdPqHqOVX1qgllrwBeA2wATgGeBxwDfLyqNvYVsyRpcn32LPYF1lXV+qq6A/h74NAhj90ELGo/m5LsChwMnN5HoJKk6fXWswCWAlcNbF8NPHmSes9Lsj/wQ+DPquoqmh7J6cBONL2ME4G3VtXd010wydHA0QDLly/f5gZIkhqzPcH9WWBFVT0O+BLwMYCqurKqDqiq/YCNwDJgbZIzkpyT5DGTnayqVlXVyqpauWTJklG1QZIWvD6TxU+A3Qe2l7Vl96iqG6rqV+3mh4AnTXKek4ETgFe3dV4PnDTj0UqSptRnsvgWsGeSRyTZETgcWD1YIcnDBjYPAdZO2P904JqquoJm/uLu9uMdUZI0Qr3NWVTVnUleBXwB2A44raouS/IWYE1VrQZeneQQ4E7gRuAlm49PEpoexQvbolXAmW3Mx/QVtyTp3vqc4KaqzgPOm1D25oHvbwDeMMWxRXPb7ebttcA+/UQqSZrObE9wS5LmAZOFJKmTyUKS1MlkIUnqZLKQJHUyWUiSOpksJEmdTBaSpE4mC0lSJ5OFJKmTyUKS1MlkIUnqZLKQJHUyWUiSOpksJEmdTBaSpE4mC0lSJ5OFJKmTyUKS1MlkIUnqZLKQJHUyWUiSOpksJEmdTBaSpE4mC0lSJ5OFJKmTyUKS1MlkIUnq1GuySHJgkh8kWZfk+En275TknHb/N5OsaMufluTSJGuS7NmW7Zrki0lMcJI0Yr39w5tkO+B9wLOBvYEjkuw9odrLgJuq6tHA3wFva8uPAw4CXgv8aVt2AvDWqrq7r5glSZPr83/p+wLrqmp9Vd0B/D1w6IQ6hwIfa79/Avi9JAE2AYvaz6YkjwJ2r6qv9hivJGkKqap+TpwcBhxYVS9vt48CnlxVrxqo8722ztXt9o+AJwPLgFOBXwJHAX8LnFhVV3Rc82jg6HZzL+AHWxn+YuDnW3nsfGWbF75xay/Y5i21R1UtmWzH9lsfT3+q6hLgKQBJ9geubb7mHJpex3FV9bNJjlsFrNrW6ydZU1Urt/U884ltXvjGrb1gm2dSn8NQPwF2H9he1pZNWifJ9sADgBs272yHpE4A/go4CXg98EHg1b1FLUm6lz6TxbeAPZM8IsmOwOHA6gl1VgN/0n4/DPjn+vVxsRcD51XVjTTzF3e3n0U9xi1JmqC3YaiqujPJq4AvANsBp1XVZUneAqypqtXAh4EzkqwDbqRJKAAkWQS8BHhWW/RO4DzgDuCP+4q7tc1DWfOQbV74xq29YJtnTG8T3JKkhcMH3CRJnUwWkqROY50stnY5kvlqiPb+rySXt0utfCXJHrMR50zqavNAveclqSTz/jbLYdqc5AXtz/qyJGeNOsaZNsTf7eVJzk/y7fbv90GzEedMSXJakuvaZ9Um258k727/PC5Nss82X7SqxvJDM+n+I+CRwI7Ad4C9J9R5JXBq+/1w4JzZjrvn9j4DWNR+P2Y+t3fYNrf1dgEuAC4EVs523CP4Oe8JfBt4YLv9kNmOewRtXgUc037fG9gw23FvY5v3B/YBvjfF/oOAzwOheWbtm9t6zXHuWWzLciTzUWd7q+r8qtrYbl5I82zMfDbMzxia53jeBvzHKIPryTBt/h/A+6rqJoCqum7EMc60YdpcwP3b7w8ArhlhfDOuqi6guYN0KocCp1fjQmDXJA/blmuOc7JYClw1sH11WzZpnaq6E7gFePBIopt5w7R30Mto/mcyn3W2ue2e715V/zjKwHo0zM/5McBjknw9yYVJDhxZdP0Yps1/CbwoydU0t+D/z9GENmu29Pe905xc7kOzK8mLgJXA02c7lj61y92/k+Z5nnGyPc1Q1AE0vccLkvzXqrp5NoPq2RHAR6vqHUn2o3m+67fKVayHNs49i21ejmSeGaa9JPl94E3AIVX1qxHF1peuNu8C/Bbw1SQbaMZ2V8/zSe5hfs5XA6uralNV/TvwQ5rkMV8N0+aXAf8AUFXfAO5Ls+DeQjXU7/uWGOdkMRPLkcwnne1N8kTgAzSJYr6PY0NHm6vqlqpaXFUrqmoFzTzNIVW1ZnbCnRHD/L3+NE2vgiSLaYal1o8wxpk2TJuvBH4PIMlv0iSL60ca5WitBl7c3hX1FOCWqrp2W044tsNQtY3Lkcw3Q7b3b4Cdgf/bzuNfWVWHzFrQ22jINi8oQ7b5C8CzklwO3AX8eVXN1x7zsG0+Dvhgkj+jmex+yTz+jx9JzqZJ+IvbeZiTgB0AqupUmnmZg4B1wEbgpdt8zXn85yVJGpFxHoaSJA3JZCFJ6mSykCR1MllIkjqZLCRJnUwW0hZIcleSSwY+K5L8vynqfjTJYR3nu6dOkg8l2bv9/saZj17aemP7nIW0lX5ZVU+YUPbUmThxVb18YPONwFtn4rzSTDBZSNsoye1VtXO7IvF7gGfSLOJ2x0CdJ9GsQ7Uz8HOah8KunXCerwKvo1kt4H5JLgEuo1l++8aqeldb72Tguqo6pd+WSf/JYShpy9xvYAjqUxP2/RGwF837El5M2+NIsgNNEjmsqp4EnAacPNUFqup42h5MVR3Z1n9xe6770Kwk8PGZbZY0PXsW0paZbBhqs/2Bs6vqLuCaJP/clu9Fs2Dhl9plVLYDhl6np6o2JLmhXbvrocC35/PyHJqfTBZS/wJcVlX7bcM5PkSzlPpuND0NaaQchpJmzgXAC5Ns176V7Blt+Q+AJe17FEiyQ5LHdpxrUzt8tdmngAOB36ZZME8aKXsW0sz5FPC7wOU0S2J/A6Cq7mhvj313kgfQ/N69i2byeiqrgEuTXFxVR7bnOB+4uR3mkkbKVWeleaCd2L4YeH5VXTHb8Wj8OAwlzXHtg3rrgK+YKDRb7FlIkjrZs5AkdTJZSJI6mSwkSZ1MFpKkTiYLSVKn/w/NNNca1foKLAAAAABJRU5ErkJggg==",
"text/plain": [
"