###                                                                                       神经网络入门介绍

原文链接：[mnist_blog.ipynb](https://github.com/bhimmetoglu/talks-and-lectures/blob/master/MachineLearning/mnist/mnist_blog.ipynb)

![img](https://burakhimmetoglu.files.wordpress.com/2017/02/neuron.png)

如今我们可以在网络上很轻松的找到很多关于神经网络的教程。只要需要搜索神经网络这几个字就会在Github Gist上很轻松的找到很多结果。
即使网络上已经优了大量的教程和案例，我仍然想以自己的看法和见解来写一篇神经网络的介绍   

这里是[神经网络入门介绍](https://github.com/bhimmetoglu/talks-and-lectures/blob/master/MachineLearning/mnist/mnist_blog.ipynb)的原文链接，希望你会喜欢   

## 正文开始

在这篇教程中我将会解释如何从头开始实现一个神经网络并且使用Scikit-Learn中提供的MNIST数据集版本来测试。在我的教程中，我会特意地尝试说明如何使用Python的类来将神经网络中的层定义为对象。每一层的对象都会有正向传播和反向传播两种能够使得代码更加紧凑易读的方法。
在编写本篇教程期间，我深受 [Peter Roelants](http://peterroelants.github.io/)的文章的启发   

In [1]: 

```python
## import
import numpy as np
from sklearn.datasets import load_digits
```

#### 数据准备

在载入数据之后， 我们将数据分为三个部分：训练集，验证集以及测试集。验证集被用来确定超参数(即隐藏层的数量和大小以及正则化参数)，测试集是用来评估最终模型的性能的单独数据

In [2]:

```python
## Load the data and reshape images
digits = load_digits()
n_samples = len(digits['images'])
data = digits['images'].reshape((n_samples, -1)); targets = digits['target']

## Train-test splitting
from sklearn.model_selection import train_test_split
X, X_test, y, y_test = train_test_split(data, targets, test_size=0.33, random_state=111)

## Train and validation splitting
X_train, X_valid, y_train, y_valid = train_test_split(X, y, test_size=0.40, random_state=123)
```

### 激活函数和分类器

我们将会在神经网络层上使用sigmoid激活函数：![$\sigma(x) = 1/(1+e^{-x})$](https://render.githubusercontent.com/render/math?math=%5Csigma%28x%29%20%3D%201%2F%281%2Be%5E%7B-x%7D%29&mode=inline)，对于分类器而言，我们将使用能够导出类的概率的softmax函数: 
![$$ {\rm softmax}_i({\bf Y}) = \frac{e^{Y_i}}{\sum_{k=1}^K\, e^{Y_k}} $$](https://render.githubusercontent.com/render/math?math=%7B%5Crm%20softmax%7D_i%28%7B%5Cbf%20Y%7D%29%20%3D%20%5Cfrac%7Be%5E%7BY_i%7D%7D%7B%5Csum_%7Bk%3D1%7D%5EK%5C%2C%20e%5E%7BY_k%7D%7D&mode=display)

这里 ![$Y_i$](https://render.githubusercontent.com/render/math?math=Y_i&mode=inline)是给定例子中神经网络输出的向量， ![$K$](https://render.githubusercontent.com/render/math?math=K&mode=inline)是类的数量，两个函数都在下例中被实现   
In [3]

```python
## Define the activation function, the Softmax classifier 
# Sigmoid function
def sigmoid(X):
    return 1.0 / (1.0 + np.exp(-X))

# Softmax function
def softmax(X):
    temp = np.max(X, axis = 0) # Determine the maximum of each column
    X = X - temp               # Subtract the max from each: does not change outcome
    return np.exp(X) / np.sum(np.exp(X), axis = 0)
```

在上述所有的应用之中，假设两个函数的参数都是按照列矩阵进行投射，因此每列都能够代表一个例子 ![$ {\bf X} \in {\mathbb R}^{n_{\rm feat} \times N}$](https://render.githubusercontent.com/render/math?math=%7B%5Cbf%20X%7D%20%5Cin%20%7B%5Cmathbb%20R%7D%5E%7Bn_%7B%5Crm%20feat%7D%20%5Ctimes%20N%7D&mode=inline),这里的N是
实例的数目 ![$n_{\rm feat}$](https://render.githubusercontent.com/render/math?math=n_%7B%5Crm%20feat%7D&mode=inline) 是要素的数量(在我们的例子中是像素的数量),在softmax函数中，我们已经从训练的样例函数中排除了每一列中的最大值，这是一个不会改变结果的函数，只是为了数据的稳定性。

现在我们有了我们的数据，激活函数和分类器，我们就能够来勾结神经网络的层了。

### 线性更新

首先我们，定义LinearUpdate类来执行当前层中(派生)特征线性变换：![$${\bf Y}^{(i)} = {\bf W}^{T} \cdot A^{(i)} + {\bf b}$$](https://render.githubusercontent.com/render/math?math=%7B%5Cbf%20Y%7D%5E%7B%28i%29%7D%20%3D%20%7B%5Cbf%20W%7D%5E%7BT%7D%20%5Ccdot%20A%5E%7B%28i%29%7D%20%2B%20%7B%5Cbf%20b%7D&mode=display)
在这个方程中 ![${\bf A}^{(i)} \in {\mathbb R}^{n_{\rm in}} $](https://render.githubusercontent.com/render/math?math=%7B%5Cbf%20A%7D%5E%7B%28i%29%7D%20%5Cin%20%7B%5Cmathbb%20R%7D%5E%7Bn_%7B%5Crm%20in%7D%7D&mode=inline)代表了当前层的状态(输入层的特征和隐藏层的派生特征)其中![$n_{\rm in}$](https://render.githubusercontent.com/render/math?math=n_%7B%5Crm%20in%7D&mode=inline)是神经元的数量。 ![$Y^{(i)} \in {\mathbb R}^{n_{\in out}}$](https://render.githubusercontent.com/render/math?math=Y%5E%7B%28i%29%7D%20%5Cin%20%7B%5Cmathbb%20R%7D%5E%7Bn_%7B%5Cin%20out%7D%7D&mode=inline)是当前层的线性输出(这个值在之后将会成为激活函数的参数)，并将和 ![$n_{\rm out}$](https://render.githubusercontent.com/render/math?math=n_%7B%5Crm%20out%7D&mode=inline)一起传递作为下一层的输入。上标 i 代表了将被考虑的训练实例。我们将其放到每一列都是训练实例向量值的矩阵中，而不是使用一个for循环来遍历整个训练实例： 

​                                               ![$$Y^{(i)}_j \rightarrow Y_{ji} : {\bf Y} \in {\mathbb R}^{n_{\rm out} \times N}$$](https://render.githubusercontent.com/render/math?math=Y%5E%7B%28i%29%7D_j%20%5Crightarrow%20Y_%7Bji%7D%20%3A%20%7B%5Cbf%20Y%7D%20%5Cin%20%7B%5Cmathbb%20R%7D%5E%7Bn_%7B%5Crm%20out%7D%20%5Ctimes%20N%7D&mode=display)。

其中N是训练实例的数量，相似的![$A^{(i)}_j \rightarrow A_{ji} : {\bf A} \in {\mathbb R}^{n_{\rm in} \times N}$](https://render.githubusercontent.com/render/math?math=A%5E%7B%28i%29%7D_j%20%5Crightarrow%20A_%7Bji%7D%20%3A%20%7B%5Cbf%20A%7D%20%5Cin%20%7B%5Cmathbb%20R%7D%5E%7Bn_%7B%5Crm%20in%7D%20%5Ctimes%20N%7D&mode=inline)   

###### 正向传递

上述的等式可以简单地用矩阵表示法表示为 ![${\mathbf Y} = {\mathbf W}^T \cdot {\bf A} + {\bf b}$](https://render.githubusercontent.com/render/math?math=%7B%5Cmathbf%20Y%7D%20%3D%20%7B%5Cmathbf%20W%7D%5ET%20%5Ccdot%20%7B%5Cbf%20A%7D%20%2B%20%7B%5Cbf%20b%7D&mode=inline)。这其中的权重 ![${\bf W} \in {\mathbb R}^{n_{\rm in} \times n_{\rm out}}$](https://render.githubusercontent.com/render/math?math=%7B%5Cbf%20W%7D%20%5Cin%20%7B%5Cmathbb%20R%7D%5E%7Bn_%7B%5Crm%20in%7D%20%5Ctimes%20n_%7B%5Crm%20out%7D%7D&mode=inline)和偏置  ![${\bf b} \in {\mathbb R}^{n_{\rm in}}$](https://render.githubusercontent.com/render/math?math=%7B%5Cbf%20b%7D%20%5Cin%20%7B%5Cmathbb%20R%7D%5E%7Bn_%7B%5Crm%20in%7D%7D&mode=inline)将会在训练代价函数最小化的时候决定的。

线性更新对象是用![$n_{\rm in}$](https://render.githubusercontent.com/render/math?math=n_%7B%5Crm%20in%7D&mode=inline), ![$n_{\rm out}$](https://render.githubusercontent.com/render/math?math=n_%7B%5Crm%20out%7D&mode=inline),，weights和biases初始化而成。之后的正向方法实现了上面的等式

###### 反向传递

反向传递的方法可以简单的采用相对于下一层(![$\nabla_{\bf Y} L$](https://render.githubusercontent.com/render/math?math=%5Cnabla_%7B%5Cbf%20Y%7D%20L&mode=inline))的状态的损失函数的梯度来描述，并计算出相对于当前层(![$\nabla_{\bf A}L$](https://render.githubusercontent.com/render/math?math=%5Cnabla_%7B%5Cbf%20A%7DL&mode=inline))，权重(![$\nabla_{\bf W}L$](https://render.githubusercontent.com/render/math?math=%5Cnabla_%7B%5Cbf%20W%7DL&mode=inline))，偏执(![$\nabla_{\bf b}L$](https://render.githubusercontent.com/render/math?math=%5Cnabla_%7B%5Cbf%20b%7DL&mode=inline))的梯度。反向传递的规则可以通过注意到以下的内容推得：

​                                      ![$$ Y_{ji} = \sum_{k=1}^{n_{\rm in}}\, \left( W^T \right)_{jk}\, A_{ki} + b_j $$](https://render.githubusercontent.com/render/math?math=Y_%7Bji%7D%20%3D%20%5Csum_%7Bk%3D1%7D%5E%7Bn_%7B%5Crm%20in%7D%7D%5C%2C%20%5Cleft%28%20W%5ET%20%5Cright%29_%7Bjk%7D%5C%2C%20A_%7Bki%7D%20%2B%20b_j&mode=display)

首先，和权重W有关的损失函数L的梯度可以写成这样：

​                                       ![$$ \frac{\partial L}{\partial W_{ln}} = \sum_{i=1}^N\, \sum_{j=1}^{n_{\rm out}}\, \frac{\partial L}{\partial Y_{ji}}\, \frac{\partial Y_{ji}}{\partial W_{ln}} = \sum_{i=1}^N\, \frac{\partial L}{\partial Y_{ni}}\, A_{li}$$](https://render.githubusercontent.com/render/math?math=%5Cfrac%7B%5Cpartial%20L%7D%7B%5Cpartial%20W_%7Bln%7D%7D%20%3D%20%5Csum_%7Bi%3D1%7D%5EN%5C%2C%20%5Csum_%7Bj%3D1%7D%5E%7Bn_%7B%5Crm%20out%7D%7D%5C%2C%20%5Cfrac%7B%5Cpartial%20L%7D%7B%5Cpartial%20Y_%7Bji%7D%7D%5C%2C%20%5Cfrac%7B%5Cpartial%20Y_%7Bji%7D%7D%7B%5Cpartial%20W_%7Bln%7D%7D%20%3D%20%5Csum_%7Bi%3D1%7D%5EN%5C%2C%20%5Cfrac%7B%5Cpartial%20L%7D%7B%5Cpartial%20Y_%7Bni%7D%7D%5C%2C%20A_%7Bli%7D&mode=display)

我们将下一层 ![$\frac{\partial L}{\partial Y_{ji}}$](https://render.githubusercontent.com/render/math?math=%5Cfrac%7B%5Cpartial%20L%7D%7B%5Cpartial%20Y_%7Bji%7D%7D&mode=inline)的梯度描述为 ![$\nabla_{\bf Y} L$](https://render.githubusercontent.com/render/math?math=%5Cnabla_%7B%5Cbf%20Y%7D%20L&mode=inline)上述的矩阵中的式子也就变成了这样：

​                                        ![$$ \nabla_{\bf W} L = {\bf A} \cdot (\nabla_{\bf Y} L)^T$$](https://render.githubusercontent.com/render/math?math=%5Cnabla_%7B%5Cbf%20W%7D%20L%20%3D%20%7B%5Cbf%20A%7D%20%5Ccdot%20%28%5Cnabla_%7B%5Cbf%20Y%7D%20L%29%5ET&mode=display)

相似的，我们可以计算出和A还有b相关的梯度，并且通过相似的步骤，我们可以获得：

​                                        ![$$ ( \nabla_{\bf b} L )_l = \sum_{i=1}^N\, (\nabla_{\bf Y} L)_{li} $$$$ \nabla_{\bf A} L = {\bf W} \cdot (\nabla_{\bf Y} L) $$](https://render.githubusercontent.com/render/math?math=%28%20%5Cnabla_%7B%5Cbf%20b%7D%20L%20%29_l%20%3D%20%5Csum_%7Bi%3D1%7D%5EN%5C%2C%20%28%5Cnabla_%7B%5Cbf%20Y%7D%20L%29_%7Bli%7D%20%24%24%24%24%20%5Cnabla_%7B%5Cbf%20A%7D%20L%20%3D%20%7B%5Cbf%20W%7D%20%5Ccdot%20%28%5Cnabla_%7B%5Cbf%20Y%7D%20L%29&mode=display)

上诉三种反向传递的方法都在下面这个LinearUpdate类中的方法中实现

```python
## Define Linear Update
class LinearUpdate(object):
    def __init__ (self, n_in, n_out, W = None, b = None):
        # Initialize W randomly
        eps = np.sqrt(6.0) / np.sqrt(n_in + n_out)
        self.W = (np.random.uniform(-eps, eps, (n_in, n_out)) if W is None else W)
        
        # Initialize biases as zero
        self.b = np.zeros((n_out,1))
        
    def forward(self, A):
        """ Forward propagation method: A is current state, method results in next state linearly
            Y <- W.T * A + b
        """
        return np.dot(self.W.T, A) + self.b
    
    def backward(self, A, gY):
        """ Backward propagation method: A is current state, gY is backpropagated derivative from 
            next layer.
        """
        gW = np.dot(A, gY.T)       # dL/dW_ab = sum_i A_ai * (gY)_bi
        gA = np.dot(self.W, gY)    # dL/dA_ab = sum_j ( (W)_aj * gY_jb )  
        gb = np.sum(gY, axis=1)    # dL/db_l = sum_i (gY)_li
        
        return gA, gW, gb
```

### 逻辑更新

现在我们实现了获取线性更新结果的对象，并且使用激活函数将他转换了。我们对激活函数的选择是上面所定义的sigmoid函数。这个激活函数将输入带入层中(让我们称它为Z)并将生成传递给下一层的结果 ![$\sigma(Z)$](https://render.githubusercontent.com/render/math?math=%5Csigma%28Z%29&mode=inline)，具体的来说我们会有如下表达式：

​                                          ![$$ {\bf Z} = {\bf W}^T \cdot {\bf A} + {\bf b}$$$$ {\bf Y} = \sigma({\bf Z})$$](https://render.githubusercontent.com/render/math?math=%7B%5Cbf%20Z%7D%20%3D%20%7B%5Cbf%20W%7D%5ET%20%5Ccdot%20%7B%5Cbf%20A%7D%20%2B%20%7B%5Cbf%20b%7D%24%24%24%24%20%7B%5Cbf%20Y%7D%20%3D%20%5Csigma%28%7B%5Cbf%20Z%7D%29&mode=display)

i.e.,这个线性的输出结果Z将会被激活函数转换然后传递给下一层

正想传递和反向传递的方法我们很容易就可以获得:

![$$ {\bf Y} = \sigma(Z), \quad \frac{\partial L}{\partial {\bf Z}} = \frac{\partial L}{\partial {\bf Y}} \cdot \frac{\partial {\bf Y}}{\partial {\bf Z}} = {\bf Y} \odot (1-{\bf Y}) \odot (\nabla_{\bf Y}L)$$](https://render.githubusercontent.com/render/math?math=%7B%5Cbf%20Y%7D%20%3D%20%5Csigma%28Z%29%2C%20%5Cquad%0A%5Cfrac%7B%5Cpartial%20L%7D%7B%5Cpartial%20%7B%5Cbf%20Z%7D%7D%20%3D%20%5Cfrac%7B%5Cpartial%20L%7D%7B%5Cpartial%20%7B%5Cbf%20Y%7D%7D%20%5Ccdot%20%5Cfrac%7B%5Cpartial%20%7B%5Cbf%20Y%7D%7D%7B%5Cpartial%20%7B%5Cbf%20Z%7D%7D%20%3D%20%7B%5Cbf%20Y%7D%20%5Codot%20%281-%7B%5Cbf%20Y%7D%29%20%5Codot%20%28%5Cnabla_%7B%5Cbf%20Y%7DL%29&mode=display)这里的Y是下一层的输入， ![$\nabla_{\bf Y}L$](https://render.githubusercontent.com/render/math?math=%5Cnabla_%7B%5Cbf%20Y%7DL&mode=inline)是和下一层输入相关的梯度![$\odot$](https://render.githubusercontent.com/render/math?math=%5Codot&mode=inline)表示元素乘法

```python

## Define Logistic Update
class LogisticUpdate(object):
    def forward(self, Z):
        """ Sigmoid activation: """
        return sigmoid(Z) #  Y = sigmoid(Z)
    
    def backward(self, Y, grad_out):
        """ Backward propagation: """
        return np.multiply(Y * (1 - Y), grad_out)   # dL/dZ = dL/dY * Y * (1-Y)
```

### 分类器

输出层将会是属于他自己类的分类器。正向的方法依然是将会是输出类概率的sigmoid函数，反向的方法实现是相对于网络输出的损失函数的梯度。最后crossEntropy方法定义了损失函数：

​                                                ![$$ L = -\frac{1}{N}\, \sum_{i=1}^{N}\, \sum_{k=1}^K\, \log(P_{ki})\, I(T_i = k) $$](https://render.githubusercontent.com/render/math?math=L%20%3D%20-%5Cfrac%7B1%7D%7BN%7D%5C%2C%20%5Csum_%7Bi%3D1%7D%5E%7BN%7D%5C%2C%20%5Csum_%7Bk%3D1%7D%5EK%5C%2C%20%5Clog%28P_%7Bki%7D%29%5C%2C%20I%28T_i%20%3D%20k%29&mode=display)

其中的 ![$P_{ki}$](https://render.githubusercontent.com/render/math?math=P_%7Bki%7D&mode=inline)是训练样本(i)属于k类的计算概率，当目标(i.e. 数字的实际值)属于k类的时候，其中函数![$I$](https://render.githubusercontent.com/render/math?math=I&mode=inline)是1否则的话为0.评估损失函数相对于输出层状态Y的导数：

​                                                  ![$$ \frac{\partial L}{\partial {\bf Y}}  = \frac{1}{N}\, ({\bf P} - {\bf I}) $$](https://render.githubusercontent.com/render/math?math=%5Cfrac%7B%5Cpartial%20L%7D%7B%5Cpartial%20%7B%5Cbf%20Y%7D%7D%20%20%3D%20%5Cfrac%7B1%7D%7BN%7D%5C%2C%20%28%7B%5Cbf%20P%7D%20-%20%7B%5Cbf%20I%7D%29&mode=display)这些方法在下面都被实现了

```python
# Define Softmax Classifier layer
class SoftmaxClassifier(object):
    def forward (self, A): 
        """ Given state A, produces output probs P by softmax """
        return softmax(A)
    
    def backward(self, P, T):
        """ Given output probs P and targets T, produces output gradient """
        expansionMatrix = np.eye(P.shape[0], dtype = int)
        expandedTarget = expansionMatrix[:, T]
        
        return P - expandedTarget # No division by number of samples yet.
    
    def crossEntropy(self, P, T):
        """ Computes cross entropy """
        expansionMatrix = np.eye(P.shape[0], dtype = int)
        expandedTarget = expansionMatrix[:, T]
        
        CE = -np.sum(expandedTarget * np.log(P + 1e-30))/P.shape[1]
        return CE
```

### 神经元

现在我们已经定义了在层中的状态如何正向传播的(显示线性的，然后被激活函数激活)，我们就能使用LinearUpdate和LogisticUpdate这两个类来定义Layer类。Layer类首先线性的转化当前状态向量A然后将他们输入到激活层，通过正向方法来产生下一层的输入Y。在反向方法中输入的梯度首先通过逻辑更新反向传播，然后通过线性更新来产生和当前层状态，权重，偏执相关的梯度。

仅仅为了进行健全性检查，我们可以明确地评估A的反向传播，如下所示

![$$ {\bf Y} = \sigma({\bf Z}), \quad {\bf Z} = {\bf W}^T \cdot {\bf A} + {\bf b}$$](https://render.githubusercontent.com/render/math?math=%7B%5Cbf%20Y%7D%20%3D%20%5Csigma%28%7B%5Cbf%20Z%7D%29%2C%20%5Cquad%20%7B%5Cbf%20Z%7D%20%3D%20%7B%5Cbf%20W%7D%5ET%20%5Ccdot%20%7B%5Cbf%20A%7D%20%2B%20%7B%5Cbf%20b%7D&mode=display)

![$$ \frac{\partial L}{\partial A_{ln}} = \sum_{i=1}^N\, \sum_{j=1}^{n_{\rm out}}\, \frac{\partial L}{\partial Y_{ji}}\, \frac{\partial Y_{ji}}{\partial Z_{ji}}\, \frac{\partial Z_{ji}}{\partial A_{ln}} = \sum_{j=1}^{n_{\rm out}}\, (\nabla_{\bf Y} L)_{jn}\, Y_{jn}\, (1-Y_{jn})\, W_{lj}$$](https://render.githubusercontent.com/render/math?math=%5Cfrac%7B%5Cpartial%20L%7D%7B%5Cpartial%20A_%7Bln%7D%7D%20%3D%20%5Csum_%7Bi%3D1%7D%5EN%5C%2C%20%5Csum_%7Bj%3D1%7D%5E%7Bn_%7B%5Crm%20out%7D%7D%5C%2C%20%5Cfrac%7B%5Cpartial%20L%7D%7B%5Cpartial%20Y_%7Bji%7D%7D%5C%2C%20%5Cfrac%7B%5Cpartial%20Y_%7Bji%7D%7D%7B%5Cpartial%20Z_%7Bji%7D%7D%5C%2C%20%5Cfrac%7B%5Cpartial%20Z_%7Bji%7D%7D%7B%5Cpartial%20A_%7Bln%7D%7D%20%3D%20%5Csum_%7Bj%3D1%7D%5E%7Bn_%7B%5Crm%20out%7D%7D%5C%2C%20%28%5Cnabla_%7B%5Cbf%20Y%7D%20L%29_%7Bjn%7D%5C%2C%20Y_%7Bjn%7D%5C%2C%20%281-Y_%7Bjn%7D%29%5C%2C%20W_%7Blj%7D&mode=display)

![$$ (\nabla_{\bf A} L) =  {\bf W} \cdot \left[{\bf Y} \odot (1-{\bf Y}) \odot (\nabla_{\bf Y}L) \right]$$](https://render.githubusercontent.com/render/math?math=%28%5Cnabla_%7B%5Cbf%20A%7D%20L%29%20%3D%20%20%7B%5Cbf%20W%7D%20%5Ccdot%20%5Cleft%5B%7B%5Cbf%20Y%7D%20%5Codot%20%281-%7B%5Cbf%20Y%7D%29%20%5Codot%20%28%5Cnabla_%7B%5Cbf%20Y%7DL%29%20%5Cright%5D&mode=display)

这是分两步实施的：首先 ![${\bf Y} \odot (1-{\bf Y}) \odot (\nabla_{\bf Y}L)$](https://render.githubusercontent.com/render/math?math=%7B%5Cbf%20Y%7D%20%5Codot%20%281-%7B%5Cbf%20Y%7D%29%20%5Codot%20%28%5Cnabla_%7B%5Cbf%20Y%7DL%29&mode=inline)被作为LogisticUpdate的反向方法，它被送入LinearUpdate的后向方法，以产生带有W的点积

```python
## Define Layes (combine linear and logistic updates)
class Layer(object):
    def __init__ (self, n_in, n_out):
        self.linear = LinearUpdate(n_in, n_out)
        self.logistic = LogisticUpdate()
        
    def forward(self, A):
        """ Forward propagation method """
        return self.logistic.forward(self.linear.forward(A))
    
    def backward(self, A, Y, grad_out):
        """ Backward propagation method """
        # First the derivative of the logistic State
        gZ = self.logistic.backward(Y, grad_out)
        # Then, gZ becomes gY for the linear state
        gA, gW, gb = self.linear.backward(A, gZ) 
        
        return gA, gW, gb
```

### 构建网络

现在我们拥有了Layers和Classifier，我们能够构建神经网络了。我们还在损失函数中添加了一个正则化项，该函数将由下面的验证集性能确定。正则化属于通过以下形式给出

![$$ L_{\rm reg} = \frac{\lambda}{2 N}\, \sum_{l=1}^{N_{\rm layers}}\, \vert {\bf W}_l \vert^2 $$](https://render.githubusercontent.com/render/math?math=L_%7B%5Crm%20reg%7D%20%3D%20%5Cfrac%7B%5Clambda%7D%7B2%20N%7D%5C%2C%20%5Csum_%7Bl%3D1%7D%5E%7BN_%7B%5Crm%20layers%7D%7D%5C%2C%20%5Cvert%20%7B%5Cbf%20W%7D_l%20%5Cvert%5E2&mode=display)

这是所有层的总和，并使用其平方范数惩罚所有层中的权重：

​                                                        ![$$ \vert {\bf W}_l \vert^2 = \sum_{i,j}\, (W_l)_{ij}^2 $$](https://render.githubusercontent.com/render/math?math=%5Cvert%20%7B%5Cbf%20W%7D_l%20%5Cvert%5E2%20%3D%20%5Csum_%7Bi%2Cj%7D%5C%2C%20%28W_l%29_%7Bij%7D%5E2&mode=display)

```python
## Construct a two hidden layer network
class nnet(object):
    def __init__ (self, nInput, nHidden1, nHidden2, K):
        """ Initiate method for the net """
        self.inputLayer = Layer(nInput, nHidden1)     # Input layer
        self.hiddenLayer1 = Layer(nHidden1, nHidden2) # 1st hidden layers
        self.hiddenLayer2 = Layer(nHidden2, K)        # 2nd hidden layer
        self.classifier = SoftmaxClassifier()         # Output: classification
    
    def forward(self, input_train):
        """ Perform forward propagation through all layers """
        A1 = input_train.T                 # Initial data
        A2 = self.inputLayer.forward(A1)   # inp -> hid1 
        A3 = self.hiddenLayer1.forward(A2) # hid1 -> hid2
        A4 = self.hiddenLayer2.forward(A3) # hid2 -> out
        P = self.classifier.forward(A4)    # output probabilities
        
        return A1, A2, A3, A4, P
    
    def backward(self, P, A4, A3, A2, A1, T, lam = 0.0):
        """ Back propagation method through all layers """
        grad_out = self.classifier.backward(P, T)                    # output grads
        gA3, gW3, gb3 = self.hiddenLayer2.backward(A3, A4, grad_out) # hid2 grads
        gA2, gW2, gb2 = self.hiddenLayer1.backward(A2, A3, gA3)      # hid1 grads
        gA1, gW1, gb1 = self.inputLayer.backward(A1, A2, gA2)        # input grads
        
        # Add regularization terms to the gradients
        if (lam > 0.0):
            gW3 += lam * self.hiddenLayer2.linear.W
            gW2 += lam * self.hiddenLayer1.linear.W
            gW1 += lam * self.inputLayer.linear.W
        
        return grad_out, gW3, gb3, gW2, gb2, gW1, gb1
    
    def Loss(self, P, T, lam = 0.0):
        """ Method for computing loss """        
        # Regularization term
        reg = np.sum(self.inputLayer.linear.W**2) + np.sum(self.hiddenLayer1.linear.W**2) + \
              np.sum(self.hiddenLayer2.linear.W**2)
        
        # Full loss
        return self.classifier.crossEntropy(P, T) + (0.5*lam/P.shape[1]) * reg
```

下面是一个能被用作训练网络的函数。我们使用动量法来更新权重的梯度，如下所示  

![$$ {\bf V} \leftarrow \mu\, {\bf V} - \alpha\, \nabla_{\bf W}\, L $$$$ {\bf W} \leftarrow {\bf W} + {\bf V} $$](https://render.githubusercontent.com/render/math?math=%7B%5Cbf%20V%7D%20%5Cleftarrow%20%5Cmu%5C%2C%20%7B%5Cbf%20V%7D%20-%20%5Calpha%5C%2C%20%5Cnabla_%7B%5Cbf%20W%7D%5C%2C%20L%20%24%24%24%24%20%7B%5Cbf%20W%7D%20%5Cleftarrow%20%7B%5Cbf%20W%7D%20%2B%20%7B%5Cbf%20V%7D&mode=display)

这中的 ![$\mu$](https://render.githubusercontent.com/render/math?math=%5Cmu&mode=inline)是动量， ![$\alpha$](https://render.githubusercontent.com/render/math?math=%5Calpha&mode=inline)是斜率。相似的更新方法也适用于偏置。所有的更新速度一开始都被设置为你可以在 [这里](http://sebastianruder.com/optimizing-gradient-descent/index.html#momentum)学习更多关于动量的知识。

这个函数返回的是训练完的网络，计算完的训练以及验证损失和验证的准确性。旁注：我们现在使用所有的训练集数据来计算梯度，这是因为训练集足够小，在真实情况下，建议你们批量训练。你可以在 [这里](http://sebastianruder.com/optimizing-gradient-descent/index.html#batchgradientdescent)了解更多。

```python
def trainNetwork(epochs, learningRate, momentum, lam, n_hidden_1, n_hidden_2, K):
    """ Function for training the network """
    
    ## Construct network
    network = nnet(n_input, n_hidden_1, n_hidden_2, K)
    
    ## Initiate velocities
    v_W1 = np.zeros_like(network.inputLayer.linear.W) 
    v_W2 = np.zeros_like(network.hiddenLayer1.linear.W)
    v_W3 = np.zeros_like(network.hiddenLayer2.linear.W)
    v_b1 = np.zeros_like(network.inputLayer.linear.b) 
    v_b2 = np.zeros_like(network.hiddenLayer1.linear.b)
    v_b3 = np.zeros_like(network.hiddenLayer2.linear.b)
    
    ## Train
    losses = {'train':[], 'validation':[]}
    for epoch in range(epochs):
        # Feed forward 
        A1, A2, A3, A4, P = network.forward(X_train)
        
        # Backprop
        grad_out, gW3, gb3, gW2, gb2, gW1, gb1 = network.backward(P, A4, A3, A2, A1, y_train, lam)
    
        # Update weights by momentum method (Divide by the number of examples in the batch)
        v_W1 = momentum * v_W1 - learningRate * gW1/n_train
        v_W2 = momentum * v_W2 - learningRate * gW2/n_train
        v_W3 = momentum * v_W3 - learningRate * gW3/n_train
    
        network.hiddenLayer2.linear.W += v_W3
        network.hiddenLayer1.linear.W += v_W2
        network.inputLayer.linear.W += v_W1
    
        # Update biases by momentum method (Divide by the number of examples in the batch)
        v_b1 = momentum * v_b1 - learningRate * gb1[:,None]/n_train
        v_b2 = momentum * v_b2 - learningRate * gb2[:,None]/n_train
        v_b3 = momentum * v_b3 - learningRate * gb3[:,None]/n_train
    
        network.hiddenLayer2.linear.b += v_b3
        network.hiddenLayer1.linear.b += v_b2
        network.inputLayer.linear.b += v_b1
    
        # Compute loss on training set
        loss_train = network.Loss(P, y_train, lam)
        losses['train'].append(loss_train)
    
        # Compute loss on validation set
        A1, A2, A3, A4, P = network.forward(X_valid)
        loss_valid = network.Loss(P, y_valid, lam)
        losses['validation'].append(loss_valid)
        
        # Compute validation accuracy
        pred = np.argmax(P, axis = 0)
        
        # Accuracy
        acc = np.sum((pred == y_valid).astype(float))/y_valid.shape[0]

    # Return the trained network and the losses
    return network, losses, acc
```

### 确定超参数

现在，我们将针对超参数的一系列值训练网络，并根据它们在验证集上的性能确定最佳值。为了说明，我们只选择优化正则化参数 ![$\lambda$](https://render.githubusercontent.com/render/math?math=%5Clambda&mode=inline) 并保持其他所有参数不变

````python
## Setup: For this example, just vary the regularization parameter
epochs=1000
learningRate = 0.2
momentum = 0.9

# Network structure
n_hidden_1 = 48
n_hidden_2 = 32
K = 10
n_input = X_train.shape[1]
n_train = X_train.shape[0]

# Values of lambda to try
lambda_l2 = np.logspace(-2,2,10)

# Train
dict_acc = {'lambda':[], 'acc':[]}; count=1
for lam in lambda_l2:
    network, _, acc = trainNetwork(epochs, learningRate, momentum, lam, n_hidden_1, n_hidden_2, K)
    dict_acc['lambda'].append(lam)
    dict_acc['acc'].append(acc)
    print "Trained %d of %d networks" % (count, lambda_l2.shape[0])
    count +=1
````

Trained 1 of 10 networks

Trained 2 of 10 networks

Trained 3 of 10 networks

Trained 4 of 10 networks

Trained 5 of 10 network

Trained 6 of 10 networks

Trained 7 of 10 networks

Trained 8 of 10 networks

Trained 9 of 10 networks

Trained 10 of 10 networks

让我们将准确度绘制为正则化参数的函数：

```python
## Plot the training and validation losses
%matplotlib inline
import matplotlib.pyplot as plt

plt.plot(dict_acc['lambda'],dict_acc['acc'])
plt.xlabel('lambda')
plt.ylabel('Accuracy')
```

<matplotlib.text.Text at 0x7efc663c6a90>



![img](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAh8AAAF5CAYAAAA/GEgLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz%0AAAAPYQAAD2EBqD+naQAAIABJREFUeJzt3Xl4lOW9//F3AEHBBSqQoAWP6ynqaTWRU/etitZdq9VU%0AjxWrFTeCtlV7Wpcee47n9LjvetCKVePP1lrUuuBSLdalmuC+1KrgggIitIgUBfL74zu5GMeEJDPP%0AzDOTeb+uK03mmWcyXx9o+OR+7vt7gyRJkiRJkiRJkiRJkiRJkiRJkiRJkiRJkiRJkiRJkiRJkiSp%0Am3YE7gLeA5YD+3fjNTsBLcBi4A3guKJVJ0mSEtUn7QKAgcB04MTM47Yuzl8fuAd4FNgC+C/gUuCg%0AYhUoSZJ6r+XAfl2c8z/ASznHrgIeL0pFkiQpUeUw8tFT2wBTc45NBbYC+pa+HEmS1BOVGD5qgdk5%0Ax2YD/YChpS9HkiT1RL+0CyiREZkPSZLUM+9nPhJTieHjA6Au51gtsBT4sIPzR6yzzjqzZs2aVfTC%0AJEnqhd4DxpBgAKnE8PEEsG/OsbHA08CyDs4fMWvWLG666SZGjx5d9OIUJk6cyMUXX5x2GVXFa156%0AXvPS85qX1iuvvMIRRxyxLnH3oFeFj0HAxlmPNyCW0M4D3gHOA9YBvpt5/mrgJOACYBIxAfVo4LCV%0Avcno0aOpr69PtHB1bvDgwV7vEvOal57XvPS85r1DOYSPMcDDma/bgAszX99AhIo6YGTW+TOAvYCL%0AiN4g7wEnA3cUv1RJklSocggfj7DyVTfjOjj2R6ChKNVIkqSiqsSltpIkqYIZPlQUjY2NaZdQdbzm%0Apec1Lz2vee9Qk3YBJVAPtLS0tDhJSZKkHmhtbaWhoQFiqkNrUt+3akY+2tpg8mSYOTPtSiRJqm5V%0AEz5mzICjjoKddoK33067GkmSqlfVhI9HHoGBA6GmBr7xDbDhqSRJ6aia8PHoo7DHHvDww7B4Mey2%0AG8yZk3ZVkiRVn6oJHy+8APvvD+uvHwHko49g993jsyRJKp2qCR8Ae+8dnzfZBB56KG69jB0LCxem%0AW5ckSdWkasJHTQ0MHbri8WabwdSp0NICd96ZXl2SJFWbqgkf1177xWNbbglrrunkU0mSSqlqwkdn%0A/cVqa2H27NLWIklSNaua8NGZujr44IO0q5AkqXpUffiorTV8SJJUSlUfPurqvO0iSVIpGT687SJJ%0AUklVffiorYV58+Czz9KuRJKk6lD14aOuLna8nTs37UokSaoOho+6+OytF0mSSqPqw0dtbXx20qkk%0ASaVR9eFj+PD47MiHJEmlUfXhY8AAGDLE8CFJUqlUffgAe31IklRKhg/s9SFJUikZPnBzOUmSSsnw%0AgSMfkiSVkuEDw4ckSaVk+CBuuyxYAEuWpF2JJEm9n+GDFV1OnfchSVLxGT6wy6kkSaVk+MD9XSRJ%0AKiXDBzBsGNTUGD4kSSoFwwfQrx8MHeptF0mSSsHwkeFyW0mSSsPwkWH4kCSpNAwfGbZYlySpNAwf%0AGY58SJJUGoaPDEc+JEkqDcNHRl0dLFwIixalXYkkSb2b4SPDFuuSJJWG4SPDFuuSJJWG4SPDFuuS%0AJJWG4SNj7bWhb1/DhyRJxWb4yOjTB4YP97aLJEnFZvjIYq8PSZKKz/CRxV4fkiQVn+EjiyMfkiQV%0An+Eji+FDkqTiM3xkab/t0taWdiWSJPVeho8sdXWweHG0WZckScVh+MjS3uXUWy+SJBVPuYSPE4C3%0AgMXAM8D2XZx/JPA8sAiYBVwPfKnQItzfRZKk4iuH8HEocBFwLrAFMA24FxjZyfk7E2HjWmBT4BBg%0ADDCp0EJssS5JUvGVQ/g4lQgO1wOvAacA7wDHd3L+VsAM4HJgJvAnIohsVWghgwdD//6OfEiSVExp%0Ah4/+QD0wNef4VGDbTl4zFagFvgnUZL4+BLi70GJqamLehyMfkiQVT9rhYyjQF8gda5gD1HXymueJ%0AOR+/BpYA7wPzgAlJFGSvD0mSiivt8JGPrYEbgLOJUZM9gQ2Aq5P45rZYlySpuPql/P4fAsuIWyfZ%0AaokRjY6cAtwPXJB5/CKx6mUa8BO+OIoCwMSJExk8ePDnjjU2NtLY2Pi5Y3V18Oyz3f8PkCSpN2hu%0Abqa5uflzxxYsWFCU90o7fHwKtABjgSlZx3cH7ujkNTVEYMm2POu5Dl188cXU19d3WZAjH5KkatTR%0AL+Stra00NDQk/l5phw+AC4FfEf09ngS+D3yZFbdRzgPWAb6befw74rbLeGLy6QjgYuApoODZGu1z%0APtraYgKqJElKVjmEj9uAtYGziCDxArAXsdwWYuJpds+PW4C1gJOIWy8LgIeA05Mopq4OPvsM5s+H%0ALxXctkySJOUqh/ABcFXmoyPjenh+QdpbrM+ebfiQJKkYKnG1S1HZ5VSSpOIyfOQwfEiSVFyGjxyr%0Arw6rreaKF0mSisXwkaOmxi6nkiQVk+GjA/b6kCSpeAwfHXDkQ5Kk4jF8dMDwIUlS8Rg+OuBtF0mS%0Aisfw0YG6OpgzB5bl7iAjSZIKZvjoQG1tBI9589KuRJKk3sfw0YH2RmPeepEkKXmGjw7Y5VSSpOIx%0AfHQge3M5SZKULMNHBwYOhDXWcORDkqRiMHx0wl4fkiQVh+GjE/b6kCSpOAwfnXDkQ5Kk4jB8dMKR%0AD0mSisPw0QlHPiRJKg7DRyfq6uDDD2Hp0rQrkSSpdzF8dKK2FtraYO7ctCuRJKl3MXx0wi6nkiQV%0Ah+GjE4YPSZKKw/DRieHD47MrXiRJSpbhoxMDBsCQIY58SJKUNMPHStjrQ5Kk5Bk+VsJeH5IkJc/w%0AsRKGD0mSkmf4WAlvu0iSlDzDx0o48iFJUvIMHytRVwfz58OSJWlXIklS72H4WIna2vg8Z066dUiS%0A1JsYPlbCLqeSJCXP8LES7SMfTjqVJCk5ho+VGDYMamoc+ZAkKUmGj5VYZRUYOtTwIUlSkgwfXbDX%0AhyRJyTJ8dMFeH5IkJcvw0QXDhyRJyTJ8dMHbLpIkJcvw0QVHPiRJSpbhowu1tbBwIXzySdqVSJLU%0AOxg+utDe5dRbL5IkJcPw0QVbrEuSlCzDRxdssS5JUrIMH11Ye23o29eRD0mSkmL46ELfvrHHiyMf%0AkiQlw/DRDS63lSQpOYaPbjB8SJKUHMNHN9jlVJKk5Bg+usGRD0mSklMu4eME4C1gMfAMsH0X5w8A%0A/hOYAfwD+CswrljFtYePtrZivYMkSdWjX9oFAIcCFwHHA38CxgP3ApsC73TymtuAYcDRRPAYDqxS%0ArAJra2HxYvj4Y1hjjWK9iyRJ1aEcwsepwCTg+szjU4A9iDDy7x2cvyewI7A+sCBz7O1iFpjd5dTw%0AIUlSYdK+7dIfqAem5hyfCmzbyWv2I27NnAG8C7wG/C+wapFqtMupJEkJSnvkYyjQF8j9Z30OUNfJ%0AazYg5oQsBg4gbr9cCaxN3IZJnPu7SJKUnLTDRz76AMuBw4GFmWOnAr8hbtUs6ehFEydOZPDgwZ87%0A1tjYSGNjY5dvOGQIrLKK4UOS1Hs1NzfT3Nz8uWMLFizo5OzCpB0+PgSWAbU5x2uB9zt5zfvALFYE%0AD4BXgRrgy8AbHb3o4osvpr6+Pq8ia2rs9SFJ6t06+oW8tbWVhoaGxN8rnzkfM4CzgVEJvP+nQAsw%0ANuf47sDjnbzmMWAdYFDWsU2I0ZB3E6ipQ/b6kCQpGfmEjwuA/Ym+HA8AjUTfjXxdCBxD9OkYTSy7%0A/TJwdeb584DJWeffAswDfpk5f0diwul1dHLLJQmGD0mSkpFP+LiMWKFSD7ycefwBcAWQz9jMbcBE%0A4CxgOjGZdC9W9PioA0Zmnb+IGBkZTKx6uQmYAkzI4727zdsukiQlo5Clts8BTcAI4GfA94CngWcz%0AX9f04HtdRfTtWBUYQ9xaaTcO2DXn/NeIWzWDiNs/P6KIox7gyIckSUkpZMJpf+BAIhzsBjxJNAob%0AAfw8c6zrpSQVon3ko60tJqBKkqT85BM+GojA0UisVLmRuG3yatY5U4hbIr1GXR18+iksWBBLbyVJ%0AUn7yCR9PExNNxxMh49MOznkLuLWAuspOdqMxw4ckSfnLJ3xsQCy3XZlFwFF5fO+yld1iffTodGuR%0AJKmS5TPhdDjw9Q6Obw1sVVg55csW65IkJSOf8HEFsG4Hx9fNPNcrrbEGrLaa4UOSpELlEz5GE/04%0Ack0HNiusnPJli3VJkpKRT/hYQiynzVUHLC2snPJmrw9JkgqXT/h4APgvosNouyFEG/QHkiiqXDny%0AIUlS4fIJHz8k2p3PBP4APEIsra0FfpBYZWXIkQ9JkgqXz1Lbd4GvAd8BtgAWE51Nm4HPkiut/Bg+%0AJEkqXL7t1T8Grk2ykEpQWwtz5sDy5dCnkF1xJEmqYvmGjxpi1csoYo+XbHcWVFEZq6uDZctg3jwY%0ANiztaiRJqkz5dji9A/iXTp7vtWMC2V1ODR+SJOUnn6BwCdFevZZoo745sCOxkdzOSRVWjuxyKklS%0A4fIZ+dgG2BWYCywndrZ9DDiDCCZbJlZdmWkf+TB8SJKUv3xGPvoSE04BPgTWyXz9NvCVJIoqV4MG%0Aweqr2+tDkqRC5DPy8RLwVeBN4CngNOBT4LjMsV7N5baSJBUmn/Dxc2Bg5uszgbuAacA84LCE6ipb%0Ahg9JkgqTT/i4L+vrN4BNgbWB+cQckF7NFuuSJBWmp3M++hObx22ec3weVRA8wJEPSZIK1dPw8Skx%0AsbRvEWqpCI58SJJUmHxWu/yc2MF27YRrqQh1dTB3LixdmnYlkiRVpnzmfJwMbATMIna2XZT1XBtQ%0An0BdZauuDtraIoCMGJF2NZIkVZ58wseUlTzXlm8hlSK7xbrhQ5KknssnfJyTdBGVxBbrkiQVptdu%0AAlcstliXJKkw+Yx8rGxJbRu9fCXMgAEweLArXiRJylc+4eOgnMerAFsA36VKbsnY60OSpPzlEz5+%0A18GxXxN7vhwKTCqoogpgrw9JkvKX5JyPPwO7Jfj9ypYjH5Ik5S+p8DEQOAl4L6HvV9YMH5Ik5S+f%0A2y7zcx7XAGsAnwBHFFxRBfC2iyRJ+csnfJyS83g5MBd4Cvio4IoqQF0dfPQRLFkSq18kSVL35RM+%0Abki6iErT3mhszhwYOTLdWiRJqjT5zPk4Gjikg+OHEMtte71NN4WaGpg6Ne1KJEmqPPmEjx8Dczo4%0APhf498LKqQzrrQd77w2XXhqbzEmSpO7LJ3yMJHazzTUTWK+wcipHUxM8/zw8+mjalUiSVFnyCR9z%0AgK91cPyrwLzCyqkc3/gGbLYZXHJJ2pVIklRZ8gkftwKXArsS+7j0Bb6ROXZrcqWVt5oamDABpkyB%0At95KuxpJkipHPuHjTOBJ4EHgH5mPqcBDVMmcj3ZHHBGbzF1+edqVSJJUOfIJH0uIPVy+AhxObDS3%0AIbEKZklypZW/gQPh2GPhuuvg44/TrkaSpMpQSHv1vwC3AXcBMxKppgKdeGIEj8mT065EkqTKkE/4%0A+C3wow6On0bsbltVRo2CAw+MZbfLl6ddjSRJ5S+f8LEDcG8Hx+8FdiqsnMrU1AR/+Qvcf3/alUiS%0AVP7yCR+rA591cHwpsGZh5VSm7baD+nqX3UqS1B35hI+XgMM6OH4o8HJh5VSmmpoY/bj/fnj11bSr%0AkSSpvOUTPv4D+ClwI7GXy3eBX2WOnZtcaZXl0EOhtjbmfkiSpM7lEz7uBA4ANgKuBC4A1iUajVVt%0Au60BA2D8+Fj1Mn9+2tVIklS+8l1q+3tgW2AQEUJuBy4CWhKqqyKNHw+ffRZ9PyRJUscK6fPxDeAW%0A4D3gZOAeYKskiqpUdXVw2GHR8XTp0rSrkSSpPPU0fHyZaK/+JjHa8XdgAPAtYs7H9DzrOIG4ZbMY%0AeAbYvpuv245YZZPv+yauqQlmzoQ770y7EkmSylNPwsd9RCfTPYCfASOA8UBb5iNfhxK3bM4FtgCm%0AET1DRnbxusHEpNcHC3z/RDU0xNJbl91KktSxnoSPscScjrOAycQoRRJOBSYB1wOvAacA7wDHd/G6%0Aq4GbgCeAmoRqSURTE/zxj/Dss2lXIklS+elJ+NgOeAH4HfA68GNi9KMQ/YF6YlfcbFOJCa2dGQf8%0AEzECU1bBA6Ld+siRjn5IktSRnoSPJ4BjgHWA84B9gJlAX2JUZI083n9o5vWzc47PAeo6ec3Gmfc/%0AAijL3VT69YsN5265BebMSbsaSZLKSz6rXT4mbpFsB/wLcD5wBjCX2OG2mPoSK2zOBv5a5PcqyLHH%0AQt++cM01aVciSVJ5SeqWRT9iJORoYL8evK4/sAg4GJiSdfwS4KvALjnnDwY+ApZlHetD/HcsA3YH%0AHsl5TT3QssMOOzB48ODPPdHY2EhjY2MPyu2Z446LVS8zZ0L//kV7G0mSCtbc3Exzc/Pnji1YsIBp%0A06YBNACtSb1XOcyXeJKYyHpi1rGXgTuAn+ScWwOMzjl2IrArsdx3BvBJzvP1QEtLSwv19fUJldw9%0AL70Em28ON90Ehx9e0reWJKlgra2tNDQ0QMLho5AmY0m5kJhLMo4IFhcR/USuzjx/HrG6BmJJ7cs5%0AH3OBf2S+zg0eqdpsM9htt5h42lY2i4ElSUpXOYSP24CJxBLe6USDsb2I5bYQE09X1vOj0D4jRdXU%0ABE8/DU8+mXYlkiSVh3IIHwBXAesDqwJjgMeynhtH3FbpzM+IWytlaa+9YKONXHYrSVK7cgkfvVaf%0APnDyyfCb38C776ZdjSRJ6TN8lMBRR8HAgXDllWlXIklS+gwfJbDmmnD00XDttbA4qab0kiRVKMNH%0AiZx8Mnz0Edx8c9qVSJKULsNHiWy4Ieyzj8tuJUkyfJRQUxO8+CL84Q9pVyJJUnoMHyW0667ReOzS%0AS9OuRJKk9Bg+SqimBiZMiP1e3nwz7WokSUqH4aPEjjgChgyByy9PuxJJktJh+CixgQPh2GPhuutg%0A4cK0q5EkqfQMHyk48URYtAgmT+76XEmSehvDRwpGjoSDDoqJp8uXp12NJEmlZfhISVMTvP463Hdf%0A2pVIklRaho+UbLstNDS4260kqfoYPlJSUxOjH1OnwiuvpF2NJEmlY/hI0be/DbW1Nh2TJFUXw0eK%0ABgyA44+HG2+E+fPTrkaSpNIwfKRs/HhYuhQmTUq7EkmSSsPwkbLaWjjssOh4unRp2tVIklR8ho8y%0A0NQEb78NU6akXYkkScVn+CgD9fWw/fYuu5UkVQfDR5loaoJp02D69LQrkSSpuAwfZeKAA2DUKEc/%0AJEm9n+GjTPTrFxvONTfD7NlpVyNJUvEYPsrIMcdA375wzTVpVyJJUvEYPsrIl74ERx4JV10Fn36a%0AdjWSJBWH4aPMTJgAH3wAt92WdiWSJBWH4aPMbLop7L57TDxta0u7GkmSkmf4KENNTfDMM/DEE2lX%0AIklS8gwfZeib34SNN3bZrSSpdzJ8lKE+feDkk+H22+Gdd9KuRpKkZBk+ytRRR8GgQXDllWlXIklS%0AsgwfZWqNNeDoo+Haa+GTT9KuRpKk5Bg+ytjJJ8P8+XDzzWlXIklScgwfZWyDDWDffV12K0nqXQwf%0AZa6pCV56CR5+OO1KJElKhuGjzO2yC2y+uctuJUm9h+GjzNXURMv1u++GN95IuxpJkgpn+KgAhx8O%0AQ4bA5ZenXYkkSYUzfFSAgQPh+9+H66+HhQvTrkaSpMIYPirECSfAokVwww1pVyJJUmEMHxVi5Ej4%0A1rfgsstg+fK0q5EkKX+GjwrS1ASvvw733pt2JZIk5c/wUUG22Qa22splt5Kkymb4qCA1NTH68cAD%0A8PLLaVcjSVJ+DB8V5tvfhro6uPTStCuRJCk/ho8K078/HH883HgjfPRR2tVIktRzho8KdNxxsGwZ%0ATJqUdiWSJPWc4aMC1dZCY2N0PF26NO1qJEnqGcNHhWpqgnfegd/9Lu1KJEnqGcNHhdpyS9hhB5fd%0ASpIqTzmFjxOAt4DFwDPA9is59yDgAWAO8DfgcWBssQssN01N8Nhj0NqadiWSJHVfuYSPQ4GLgHOB%0ALYBpwL3AyE7O3wG4H/gmUA88DNyVeW3V2H9/GDXK0Q9JUmUpl/BxKjAJuB54DTgFeAc4vpPzTwHO%0AB1qAN4CfAq8D+xa90jLSrx+cdBLceivMnp12NZIkdU85hI/+xOjF1JzjU4Ftu/k9+gBrAPMSrKsi%0AHHNMhJCrr067EkmSuqccwsdQoC+Q+7v7HKCum9/jB8BA4LYE66oIQ4bAkUfCVVfBkiVpVyNJUtfK%0AIXwUqhE4m5g38mHKtaRiwoS47XJb1UUvSVIl6pd2AURgWAbU5hyvBd7v4rWHEnNFDiYmnXZq4sSJ%0ADB48+HPHGhsbaWxs7FGx5Wj0aBg7NiaeHnFEbEAnSVJPNDc309zc/LljCxYsKMp7lcs/U08Sk0dP%0AzDr2MnAH8JNOXtMIXEcEkLtW8r3rgZaWlhbq6+sTKLU83XMP7L13LL3dbru0q5Ek9Qatra00NDQA%0ANACJNXYol9suFwLHAOOA0cSy2y8D7dMozwMmZ53/HeBGYq7H08TckDpgzRLVW3b23BM22cRlt5Kk%0A8lcu4eM2YCJwFjCdaDC2F7HcFiJYZPf8OJao/QpgVtbHxSWqt+z06QMnnwy//W20XZckqVyVS/gA%0AuApYH1gVGAM8lvXcOGDXrMe7ECtk+uR8HF2SSsvUd78LgwbF6pfbb4dFi9KuSJKkLyqn8KECrbEG%0AXHcdzJ8PBx8Mw4bBt74Ft9wCf/972tVJkhQMH73MwQfDs8/CX/8K55wD774Lhx8eQWSffeCXv4R5%0AVdeKTZJUTgwfvdSGG8Jpp8FTT8Hbb8MvfhGjH9/7HtTWwu67R1fUDz5Iu1JJUrUxfFSBkSNjB9w/%0A/hFmzYLLL4/jJ50E66wDO+4Yq2ScqCpJKgXDR5Wpq4Px4+GBB6Ir6nXXwZprxijJqFHw9a/HKMkb%0Ab6RdqSSptzJ8VLG114Zx4+Duu2HOHLj5Zlh33ZgrstFGsMUWcO658PLLaVcqSepNDB8CYK214Dvf%0AiT4hc+fCr38dbdt/8QvYbLP4+qc/henToa0t7WolSZXM8KEvGDQoVs00N0cQufPOuB1z5ZVQXx+j%0AIj/6UUxmXb487WolSZXG8KGVWnVV2HdfuOGGmCNy//2w225w442w9daw3norJrMuW5Z2tZKkSmD4%0AULetskrsnnvNNbFq5pFH4MADo5vqTjvFypn2yayffZZ2tZKkcmX4UF769o3Aceml0UfkiSeirfvU%0AqRFQ6upWTGZdsiTtaiVJ5cTwoYL16RO3YP73f2OJbmsrHH88PPlk3LIZNiwms95+O3zySdrVSpLS%0AZvhQompqYMst4ec/h1degZdegh/+MD4ffDAMHep+M5JU7QwfKqpNN4WzzoLnnoPXX4ezz45Oqrn7%0AzXz0UdqVSpJKxfChktloIzj9dPjzn2HmTPif/4G//S32mxk+PPabueaaWFUjSeq9DB9KxahRMHEi%0ATJsG770Hl10WzctOPBFGjFgxmfXdd9OuVJKUNMOHUjdiRExQffDB2GV30iRYffWYKzJy5IrJrG++%0AmXalkqQkGD5UVoYOhaOPht//Prqr3nRT9A856yzYcMPPT2aVJFUmw4fK1lprxcTU3/4WPvww9pv5%0Ayldirsimm8bHmWfCs8+634wkVRLDhypC7n4zU6bAmDFw+eUxGrLxxnDaabHfjEFEksqb4UMVZ9VV%0AYb/9YPLkWBlz332w666x/8zWW8dkVvebkaTyZfhQRevfH/bYA669Ft5/H/7wBzjgAPjNb2LFzLrr%0Axn4zDz7ofjOSVC4MH+o1+vaFnXeOZbvvvAOPPw5HHBE78e6+e+w30z6Z1f1mJCk9hg/1Sn36wDbb%0AwPnnxxLdlpYYAXn88eiqOnz4isms7jcjSaVl+FCvV1MD9fXwn/8ZS3RffBFOPRVeeCH2mRk2bMVk%0AVvebkaTiM3yoqtTUwGabxR4zzz8Pf/lLLNedOTN23h02LHbiveEG95uRpGIxfKiqbbwxnHEGPP00%0AzJgB//3fMH9+zA2prYWxY91vRpKSZviQMtZbD045BR57LPabueSSWKp74onRZdX9ZiQpGYYPqQMj%0ARsAJJ8BDD8V+M9deG43OsvebOf98eOuttCuVpMpj+JC6MHQofO97cM89MGcO/OpXEU7OPBM22GDF%0AZNZXX027UkmqDIYPqQcGD47eIXfcEW3eb7sNNtkk5oqMHh2TWc86C557zjbvktQZw4eUp9VXh0MO%0AgVtvjRGRKVOgoSHmhWyxRUxmPf10+POfDSKSlM3wISVgtdViv5kbb4wgcu+9sMsucP318PWvx2TW%0AiRNh2jT3m5GkfmkXIPU2/fvDnnvGx1VXReC4/fa4RXPJJbGE98ADo8FZQ0P0HpGkcrRwYXG+bzX8%0A2KsHWlpaWqivr0+7FlWx5cvhyScjiNx+ezQ2k6Ty1go0kPmf1qS+qyMfUon06QPbbhsf558P06fH%0AvjOSVK7efDPmriXN8CGloH2/GQfjJJWz1sTGOj7PCaeSJKmkDB+SJKmkDB+SJKmkDB+SJKmkDB+S%0AJKmkDB+SJKmkDB+SJKmkDB+SJKmkDB+SJKmkDB+SJKmkDB+SJKmkDB+SJKmkDB+SJKmkDB+SJKmk%0AyiV8nAC8BSwGngG27+L8nYCWzPlvAMcVtTr1WHNzc9olVB2veel5zUvPa947lEP4OBS4CDgX2AKY%0ABtwLjOzk/PWBe4BHM+f/F3ApcFDRK1W3+QOi9Lzmpec1Lz2vee9QDuHjVGAScD3wGnAK8A5wfCfn%0AjwdmZF73GnBd5rU/LHahkiSpcGmHj/5APTA15/hUYNtOXrNNJ+dvBfRNtDpJkpS4tMPHUCIwzM45%0APgeo6+Q1tR2cPxvol/l+kiSpjPVLu4BSeeWVV9IuoaosWLCA1tbWtMuoKl7z0vOal57XvLSK9W9n%0ATVG+a/f1BxYBBwNTso5fAnwV2KWD1zwKTAcmZh07EPh/wGrAspzzRwBPA+smU7IkSVXlPWAM8H5S%0A3zDtkY9PiSWzY/l8+NgduKOT1zwB7JtzbCwRMHKDB8TFGkOEEEmS1DPvk2DwKBffBpYA44DRxLLb%0Av7Niqe0mSfYNAAAG40lEQVR5wOSs8/8J+Bi4IHP+0ZnXH1iaciVJUm9wPNFk7B/ECEZ2k7FfAg/n%0AnL8jMWLyD6LJ2PdLUKMkSZIkSZIkSZIkSZJKo6cb1qn7fkzMz/k70eTtDmCTDs47h1im9QnwB2DT%0AEtVXDc4AlhOTtLOdg9c8SesCNwEfEq0BphOdmbOdg9c8KasQCw3eIq7nG8CZfLE1xDl4zfO1I3AX%0Acf2WA/t3cM45rPz6DgAuA+YSi0CmYEsLIDasW0Kshvln4gf0QjrfsE49cy9wJLHi6KvEX+QZwMCs%0Ac04HFgAHAJsBzcRf5tVLWWgvNQZ4E3gWuDDruNc8WUOIv9fXEVs4jCL6D22QdY7XPFlnE/+gfZO4%0A3t8ifsmZkHWO17wwewL/QVy/5cB+Oc935/peRezDtiuxyetDRDBPu3N66p4Crsg59jKxC66SN5T4%0AS9w+ulRDrAv/UdY5/YH5uDqpUKsTmyruSvxG0h4+vObJ+2+isWFnvObJuwv4v5xjt7Oi5YLXPFm5%0A4aM713ct4pf7Q7LOGQEsJfpudam3JpR8NqxTYQZnPn+U+bw+sQ9P9p/Bp8QPcv8MCnMFcDexBD17%0AKNprnrz9iGX9vyZuL7YCx2Q97zVP3t3AbsDGmcdfA7YD7sk89poXV3eubwNxeyz7nPeBF+nmn0Ha%0AHU6LJZ8N65S/GuK21jRidAlWXOeO/gxGlaiu3ugwYohzTOZxW9ZzXvPkbUD0IboA+Dnwr8ClxA/j%0AG/GaF8M1RDPJ14jfpPsC/05soQFe82LrzvWtI/4/8Lecc2YTwaVLvTV8qLQuJ+4LdndCb1vXp6gD%0AI4l9j3Yj/o8PEfy6s0eT1zw/fYA/Az/NPH4O2BwYT4SPlfGa52cCcBQRtF8CtgQuJn6z9pqnK7Hr%0A21tvu3xI7POSm8Bq6YX96VN2GbAPMQlvVtbxDzKfO/oz+ADlowEYRgz9f5b52JH4Yf0pXvNimMWK%0A0bx2r7LiN0CvefJ+ApwL3EaEj5uIkdUfZ573mhdXd67vB8T0hrVyzqmjm38GvTV8ZG9Yl2134PHS%0Al9Mr1RAjHgcQEx9n5jz/FvGXMPvPoD+wE/4Z5OtB4rfur2U+tiCWkN+U+dprnrw/AV/JObYJsQIG%0AvObFUMMXNwldzooRPq95cXXn+rYQv/xknzOCGAGv+j+DrjasU2GuJGY/70ik3faPVbPOOS1zzgHE%0AP5q3AO8Cg0paae/2CJ/v8+E1T9ZWxC8zPwY2Ar5D9DRozDrHa56sa4klnHsRcz8OJOYbnJd1jte8%0AMIOIX1i2IILdxMzX7f8+duf6Xgm8TfzyuSWx1LaV7t0G7vVWtmGdCrOc+O1kec7HkTnnnU0MXS/G%0ARkDFkL3Utp3XPFl7A88T1/Ml4HsdnOM1T84g4HxWNBn7K9GTIneOotc8fzuz4md29s/x67PO6er6%0A9icmX7c337PJmCRJkiRJkiRJkiRJkiRJkiRJkiRJkiRJkiRJklTN+qZdgKSy9gixj8z9ZfgeM4hd%0ANp9KuB5JRdZbN5aTlIw2ir9Neb7vUYraJBWB4UOSJJWU4UNSd/0b8AyxO/T7wM3AsKzndyY2pxoL%0ATCc2BXsYqAX2AV4B/pZ53Wo533sV4HJiJ80PgXNznh8O3JX5nm8Ch3dQ36nEBnAfE7ttXoG7nEqS%0AVHGyd80dB+xBbHP+deBx4PdZ5+5MhI8/AdsQW3T/BXiUmM/xNWJn6bnElt3tHiECzYXAxqzYtv6Y%0ArHPuAZ7LvG898Bixk+aErHOagJ2AUcAuRNi5Ip//aEmSlJ7s8JFrDBE2BmYe75x5vEvWOadnjv1T%0A1rGrgHuzHj8CvJjzvc8jtq8H2CTzPcZkPf/PmWMT6NwhRNCRVGa87SKpu7YEphCrTP5OBJM2YqQh%0A2/NZX88hbpXMyDk2POtxG/Bkzvd4khgFqQFGA0uBp7Oefw1YkPOaXYAHgHcz9U0GvgSs2sV/l6QS%0AM3xI6o6BwFTiH/XDga2AA4lw0D/n3M+yvm7Ledx+LPdnT02B9a1H3Jp5HjiIuDVzYif1SUpZv7QL%0AkFQRvgKsDZwBvJc59q8Jfe8aYi5Htq2J+SJtwKvEz6oxrBj9+GdgcNb5WxGB5gdZxw5LqD5JCXPk%0AQ9LKtI9IvA18Ssyx2ADYDzgzwfcZBVxAhIpG4CTgksxzrwH3Af9HBJ4GYBKwOOv1bxArZtrr+zfg%0AuATrk5Qgw4eklWlv4vUhcBQxifMlYrXKD/hik6+Omn51dE5bzuPJxPLbp4DLgEuJsNFuHPAOsXLm%0AN8A1xNyRds8SS21PB14gAsyPO6lHkiRJkiRJkiRJkiRJkiRJkiRJkiRJkiRJkiRJkiRJkiRJkiRJ%0AkiRJkiSpuv1/+1LYdYsgdQAAAAAASUVORK5CYII=%0A)

### 最终的神经网络

现在我们已经使用验证集确定了 ![$\lambda$](https://render.githubusercontent.com/render/math?math=%5Clambda&mode=inline)的值，我们可以最后一次训练网络并在测试集上进行预测。这将为我们提供网络的预测准确性。

```python
# Best lambda
lam_best = lambda_l2[np.argmax(dict_acc['acc'])]
best_validation_acc = max(dict_acc['acc'])

# One final training with the best model
network, losses, acc = trainNetwork(epochs, learningRate, momentum, lam_best, n_hidden_1, n_hidden_2, K)

## Predict on the test set
A1, A2, A3, A4, P = network.forward(X_test)

# Predict digits
pred_test = np.argmax(P, axis = 0)
        
# Accuracy
acc_test = np.sum((pred_test == y_test).astype(float))/y_test.shape[0]

print "Best lambda: %.6f, Validation set accuracy: %.6f" % (lam_best, acc)
print "Test set accuracy: %.6f" % acc_test
```

Best lambda: 0.599484, Validation set accuracy: 0.964730

Test set accuracy: 0.959596

很棒

最后，让我们在训练期间绘制训练和验证损失值。由于我们使用整批来训练模型（即所有训练集），因此我们没有观察到任何振荡和损失函数的平滑减少。

```python

## Plot the training and validation losses
%matplotlib inline
import matplotlib.pyplot as plt

plt.plot(losses['train'], label='Training loss')
plt.plot(losses['validation'], label='Validation loss')
plt.legend()
```

<matplotlib.legend.Legend at 0x7efc6637c790>

![img](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAg8AAAFkCAYAAACn/timAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz%0AAAAPYQAAD2EBqD+naQAAIABJREFUeJzs3Xd8FVX+//HXvekJaZSEhN5LIEBCk95VBARUNAqL1F11%0Av/7Y1RV3hbWAsuIqrAWVVYqFiK6giF2QJiAQQpMqSE0oBgghQEj7/TEp96bf5Cb3Jnk/H495zL0z%0AZ2Y+5KHkzZwzZ0BERERERERERERERERERERERERERERERERERERERERERERERERERERERERERKRA%0Afwe2A1eAc8BKoKUNx/cE0oBY+5cmIiIizuhr4A9AGyAc+AI4DniX4NgA4CjwDbCznOoTERGRCmAq%0Aw7G1gfNAH2BTMW0/Ag4BGcBIoFMZrisiIiIOZC7DsQFZ64vFtJsANAaepWxhRURERJyAaymPMwHz%0AgI3A/iLatQDmAL0w7jqUREjWIiIiIraJz1rKVWnDw+tAGEYoKIwLsAx4Gvi1hOcNCQ0NjYuLiytl%0AWSIiItXaGaAL5RwgStON8BowAmOsw4ki2gVgdGmkW2wzZ10zHRgMrMtzTAQQ88EHH9CmTZtSlCal%0AMW3aNObPn+/oMqoV/cwrnn7mFU8/84p14MABxo4dCxBJOT+cYMudBxNGcLgT6EfRwQEgEWiXZ9sj%0AwADgLownNQrUpk0bIiIibChNyiIgIEA/7wqmn3nF08+84ulnXnXZEh7eAKIwwkMyUDdr+2XgRtbn%0AOUAoMB7IJP94iAtZbYsaJyEiIiJOzJanLf4E+GF0NcRZLGMs2tQFGhRxjsysRURERCopW+48lCRo%0ATChm/7NZi4iIiFRSZZnnQaqIqKgoR5dQ7ehnXvH0M694+plXXc42aVMEEBMTE6NBNiIiIjbYuXMn%0AkZGR4GRPW4iIVDpHjhwhKSnJ0WWIlJmvry8tWrRwdBmAwoOIVGFHjhyhZUtbXv4r4twOHz7sFAFC%0A4UFEqqzsOw6aeE4qu+wJoJzlLppThofMTD3NKSL2o4nnROzLKZ+2iImPcXQJIiIiUginDA9Ldi1x%0AdAkiIiJSCKcMD1tObeHYpWOOLkNEREQK4JThwcfdR3cfREREnJRThodbm93Kkl1LSM9IL76xiEg1%0AYzabS7Rs2LChTNd55plnMJtL92ti3bp1dqmhsl27unDKpy1GtBrBim0rWPvbWgY3G+zockREnMrW%0ArVtzPmdmZjJr1izWrVvH2rVrrdqV9fHUKVOmMHTo0FIdGxkZydatW/WIbBXllOGhbe12tKndhkW7%0AFik8iIjk0bVrV6vvtWvXxmQy5due1/Xr1/Hy8irxderVq0e9evVKVaOvr2+x9Ujl5ZTdFmvWmJjY%0AaSIrD6zk0vVLji5HRKTS6devH+3bt2fDhg306NEDHx8fJk2aBMDy5csZMmQIoaGheHt707ZtW/7+%0A979z7do1q3MU1G3RuHFjhg8fzjfffENERATe3t60adOGxYsXW7UrqOvgwQcfxNfXl6NHjzJ06FB8%0AfX1p2LAhjz/+ODdv3rQ6/vTp09x99934+fkRGBjI2LFj2b59O2azmaVLl5bqZ7Jq1SpuueUWfHx8%0A8PPzY8iQIVZ3cQAuXLjA1KlTadiwIZ6engQFBdGrVy/WrFmT0yY2NpZhw4YRHByMp6cn9erVY9iw%0AYZw5c6ZUdVVGThkeVq+GseFjSctII3pftKPLERGpdEwmE/Hx8YwbN46xY8fy9ddf8/DDDwPGtN23%0A334777zzDt9++y3Tpk3j448/Zvjw4QWeJ+/33bt38/jjj/PYY4+xatUqwsPDmTRpEhs3biy2rtTU%0AVIYPH87gwYNZtWoVEydOZN68ebz44os5bZKTk+nfvz/r169n7ty5fPLJJwQFBXHvvfcWWFNJLFu2%0AjJEjRxIQEMBHH33Eu+++y6VLl+jXrx8//fRTTrtx48bx+eef8/TTT/PDDz/w7rvvMmjQIC5evJhT%0A2+DBg7lw4QILFizghx9+YP78+TRq1MhpZn+sCE7ZbbF1K7il1OWOlnewKHYRD3d52NEliUg1cO0a%0AHDxYvtdo3Rq8vcv3GmCMhbh48SKffvopffv2tdo3Y8YMq3a33HILrVu3pl+/fuzdu5f27dtb7c97%0A3oSEBDZv3kz9+vUB6N27N2vWrGHZsmX07t27yLpu3rzJrFmzuOuuuwDo378/O3bsYNmyZcycOROA%0ApUuXcvToUb755huGDBkCwKBBg7h+/Tpvv/22zT+LjIwM/va3vxEeHs7XX3+ds33o0KE0a9aM6dOn%0As2nTJgA2b97MlClTcu7SAFah6uDBg1y8eJHFixdbbb/nnntsrqsyc8rwkJEBn38OE3pMYNTyUew5%0At4fw4HBHlyUiVdzBg2C80bj8xMRARc2UXbNmzXzBAeDYsWPMmDGDH3/8kfPnz1sFhIMHD1qFh4J0%0A7NgxJzgAeHh40LJlS06ePFlsTSaTKd8djvbt21sN9ly/fn1Ot4KlqKioUoWHQ4cOER8fz1//+ler%0A7T4+PowePZqFCxdy48YNPD096dq1K4sXL6ZmzZoMHDiQyMhI3Nzcco5p0aIFgYGBPPHEE8TFxdG7%0Ad2/atm1rc02VnVOGh7Aw+PZb+GD8HQT5BLE4djHzbpvn6LJEpIpr3dr45V7e16goISEh+bZdvXqV%0A3r174+3tzfPPP0/Lli3x9vbm5MmTjB49muvXrxd73lq1auXb5u7uXqJjfXx8cHd3t9rm4eHBjRs3%0Acr4nJCQQHByc79igoKBiz1+QhIQEoOCfR2hoKBkZGVy6dImQkBCWL1/O7Nmzeeedd5g5cyY1atRg%0A1KhRzJ07l+DgYPz8/Fi/fj3PP/88//jHP3KOmzJlCjNmzMDV1Sl/rdqdU/4pu3eHTz8FM27cG3Yv%0AH+//mJdvfRmzySmHaIhIFeHtXXF3BSpCQWMD1q5dS3x8POvXr7fqYsju0y9vJXnxYa1atdi+fXu+%0A7WfPni3VNbPDTlxcXL59cXFxmM1mAgMDc9rOmzePefPmcfr0aT7//HOefPJJzp8/n9Pl0a5dO6Kj%0AjfF4e/bsYcmSJTz33HN4eXkxffr0UtVY2Tjlb+NbboFLl2DHDrg37F7ikuL46eRPxR8oIlJNlXQQ%0AYXa7vP/6L013QGmUpM5+/fqRlJTEN998Y7X9o48+KtU1W7VqRb169Vi2bJnV9uTkZD799FN69OiB%0Ap6dnvuPq16/PI488wqBBg4iNjS3w3OHh4bzyyiv4+/sX2qYqcso7D+3agZ8ffPcdPDXjFur51uPj%0AXz6md6OiB+KIiFRXBf2LvqBtPXv2JDAwkD/96U88/fTTuLq68uGHH7Jnz55yqaE0bcaPH8+8efMY%0AO3Yss2fPplmzZnz99dd89913ADbPemk2m5k7dy4PPPAAw4YNY+rUqaSkpPDSSy9x5coV/vWvfwGQ%0AmJjIgAEDuP/++2nVqhW+vr5s376db7/9NmeA5+rVq1mwYAGjRo2iSZMmZGZmsmLFChITExk8uPrM%0AS+SUdx5cXWHgQGPcg9lkZkzYGP534H+arlpEpAAmk6nARyoL+ld+zZo1+fLLL/H29mbs2LFMmjQJ%0APz8/li9fXuLz2lJDSWrKu93b25u1a9fSr18/nnjiCe6++25Onz7NggULAAgICCiwhqKuHRUVxWef%0AfUZCQgL33XcfEydOJCAggB9//JEePXoA4OXlRbdu3Xj//fcZO3YsQ4cOZdGiRTz55JP897//BaBl%0Ay5YEBgYyd+5c7rzzTsaMGcOuXbtYunSp1RMaVZ3tD8uWrwggJiYmhu3bI3jkEUhIgANJW7nl3Vv4%0AcfyP9Gvcz9E1ikglsXPnTiIjI4mJiSGiKg1mqKZeeOEFZs6cyalTpwgNDXV0ORWqJP8tZ7cBIoGd%0A5VmPU3ZbAAwZAunpsHYtjBzZjQZ+DVhxYIXCg4hINfD6668D0Lp1a1JTU1m7di2vvfYa48aNq3bB%0AwRk5ZbcFQJMm0KKF0XVhMpkY1nIYXx75skT9ZSIiUrn5+PiwcOFCRo8ezciRI3OeenjnnXccXZrg%0AxHceAG691ZiqOjMThrYYyps73uRwwmFa1W7l6NJERKQcTZgwgQkTJji6DCmE0955AKPr4vhx+PVX%0AGNBkAB4uHnx55EtHlyUiIlKtOXV46N8f3Nzg++/B282b/k36KzyIiIg4mC3h4e/AduAKcA5YCbQs%0A5phewE/A78A14ADwl5JesEYNY8KorEd7uaPFHWw8sZErKVdsKFtERETsyZbw0Ad4DegGDMYYL/Ed%0AUNT74a4CrwK9gdbAbGAW8MeSXnTIEOOJi9RUY9xDakYqa46tKf5AERERKRe2hIfbgfcw7h7sASYA%0ADTHmZijMLmB51jEngQ+Bb4EeJb3okCGQlAQ//wxNA5vSJKAJPx7/0YayRURExJ7KMuYhe4ovW96m%0A0gkjOHxf0gMiIqBmTWPcA0C/xv1Yf2K9DZcUEREReypteDAB84CNwP4StD8N3AB2AG8BH5T0Qi4u%0AMGhQ7riHvo36svfcXi5er5g3wImIiIi10oaH14EwIKqE7XtiTJf5R2AaxYx5mDZtGiNGjMhZDh0a%0Awc8/R3PpEvRt3JdMMtlwYkMpSxcRqdxGjx6Nt7c3iYmJhbZ54IEHcHd358KFCyU+r9ls5tlnn835%0Avm7dOsxmMxs2FP/37YMPPkiTJk1KfC1LCxYsYOnSpfm2Hz9+HLPZzHvvvVeq85bFM888Y/MLuCpS%0AdHS01e/JESNGMG3atAq7fmkmiXoNGIYxgDL/y9ELdiJr/QsQDDwOFPr+1/nz51vN3X3iBDRubAyc%0AvOuuxjTyb8T64+sZ2XpkKcoXEancJk+ezGeffcayZct46KGH8u1PTExk5cqVDB8+nDp16th0bssX%0ASkVGRrJ161batGlj87G2WLBgAXXq1GH8+PFW20NDQ9m6dSvNmjUr1XnLqrR/nooQFRVFVJT1v98t%0A3m1R7myJVSaMOw4jgQHkBoLSXNOmONeoEbRqldt10adRHzad2lTKy4uIVG633XYboaGhLFq0qMD9%0A0dHR3Lhxg4kTJ5bpOr6+vnTt2hVfX98Stbf36wPc3d3p2rUrtWrVsut5S0qvQyicLb/E3wAeyFqS%0AgbpZi6dFmzmA5b2nRzDuUrTIWiYAjwHv21rokCFGeMjMhO71u7P77G5upN2w9TQiIpWe2WzmwQcf%0AJCYmhn379uXbv3jxYkJDQxk6dCgXLlzg4YcfJiwsDF9fX4KDgxk4cCCbNhX/D7DCui2WLFlCq1at%0A8PT0pG3btrz/fsF/pT/77LN069aNWrVq4e/vT2RkZL7A07hxY/bv38/69esxm82YzWaaNm0K5HZb%0A5O3S2LRpEwMHDsTPzw8fHx969uzJV199la9Gs9nMunXreOihh6hTpw61a9fmrrvuIj4+vtg/e0Ey%0AMjKYO3curVu3xtPTk+DgYMaPH8+ZM2es2sXGxjJs2DCCg4Px9PSkXr16DBs2zKrdJ598Qrdu3QgI%0ACMDHx4dmzZpVqld62xIe/gT4AeswuiuylzEWbeoCDSy+mzACRSzGBFOPANOBZ7FR9lTVR49Ct3rd%0ASM1IJTY+1tbTiIhUCRMnTsRkMuX7Zbx//362b9/O+PHjMZlMXLp0CYCZM2fy5ZdfsmTJEpo2bUq/%0Afv1Yv972J9eWLFnCxIkTCQsLY8WKFcyYMYNZs2bx448/5rvNf/z4caZOncry5ctZuXIlo0eP5tFH%0AH2XWrFk5bT777DOaNm1KREQEW7duZevWraxcudLqPJbnXb9+PQMGDCApKYlFixYRHR2Nr68vw4cP%0A5+OPP85X7+TJk/Hw8CA6Opq5c+eybt06xo4da/OfG+Chhx7iySef5NZbb+WLL75g1qxZfPPNN/To%0A0YOEhAQAkpOTGTx4MBcuXGDBggX88MMPzJ8/n0aNGpGUlATA5s2buffee2nevDnLly/nq6++4p//%0A/Cfp6emlqssRbBnzUJKgkfctJq9nLWXWty+4uhp3H6b8MRwPFw9+PvMztzS4xR6nFxHhWuo1Dv5+%0AsFyv0bp2a7zdippbr2SaNm1K3759+eCDD5g7dy6ursZf59lhIrvLomXLlixYsCDnuPT0dAYPHszx%0A48d59dVX6du3b4mvmZGRwVNPPUXnzp1ZsWJFzvZevXrRokUL6tWrZ9V+8eLFVsf26dOHjIwMXn31%0AVWbOnAlAx44d8fT0xM/Pj65duxZbw5NPPkmtWrVYt24d3t7Gz3HYsGF07NiRxx9/nDFjxli1v/32%0A25k/f37O94sXL/LEE09w/vx5goKCSvxnP3jwIP/973955JFH+M9//pOzvVOnTnTr1o158+Yxe/Zs%0ADh48yMWLF1m8eDHDhw/PaXfPPffkfN6yZQsAb731Vk6XUN++ffON+XBmTv1WTUu+vtCjhxEeHn7Y%0AjYiQCH4+87OjyxKRKuTg7weJXFi+A85ipsYQEVLU3HolN2nSJMaNG8eqVasYPXo0aWlpfPDBB/Tp%0A08dqkOFbb73FwoULOXDgACkpKTnbSzoQMtuhQ4eIj4/n8ccft9resGFDevTowYkT1kPh1q5dywsv%0AvMCOHTu4ciX3tQImk4kLFy7YPJgzOTmZbdu28fDDD+cEBzC6ccaNG8f06dM5fPgwLVvmvjlhxIgR%0AVudo3749ACdOnLApPPz4ozE54YMPPmi1vUuXLrRp04Y1a9Ywe/ZsmjdvTmBgIE888QRxcXH07t2b%0Atm3bWh2THZLuueceJk6cSM+ePfMFL2dXacIDGF0Xc+caU1V3q9eNVYdXObokEalCWtduTczUmHK/%0Ahr3cdddd/PnPf2bx4sWMHj2ar776ivPnz/PSSy/ltHnllVd4/PHHeeihh3j++eepXbs2ZrOZmTNn%0AcvCgbXdZsm/N161bN9++4OBgq/Cwbds2br31Vvr3788777xD/fr1cXd3Z+XKlTz//PNcv37d5j/v%0ApUuXyMzMJCQkJN++7G3ZNWbLO9jSw8MDwObrZ5+3sGufOnUKAH9/f9avX8/zzz/PP/7xDy5dukRI%0ASAhTpkxhxowZuLq60rt3bz777DNeffVVxo8fT0pKCmFhYTz11FPcd999NtXlKJUuPMyYAdu2Qbf6%0A3Zj/83wuJF+gjo9t6VVEpCDebt52uytQETw9Pbn//vtZuHAhZ8+eZdGiRfj5+VndIv/ggw/o378/%0Ab7zxhtWxlncCSir7F3FBAw7Pnj1r9f2jjz7C3d2d1atX4+7unrPdsrvDVoGBgZjNZuLi8s8SkL2t%0Adu3apT5/UbL/7HFxcYSGhua7tuV127VrR3R0NAB79uxhyZIlPPfcc3h5eTF9+nSAnLkZUlNT2bJl%0AC3PmzOGBBx6gcePGdO/evVz+DPbkvDNgFCB7qurvvoOu9YzbPtvObHNwVSIijjNp0iTS09N56aWX%0A+Oqrr7jvvvvw9Mx9CM5sNlv98gbjF1p2v7stWrVqRUhISM4vxmwnTpxg8+bNVttMJhMuLi5WEy1d%0Av36d999/P9/ASg8PD65du1bs9X18fOjWrRsrVqzgxo3cp+0yMjL44IMPaNCgAS1atLD5z1USAwcO%0ABIwwZmn79u0cPHgwZ39e4eHhvPLKK/j7+xMbm3+Qv5ubG3369OFf//oXmZmZ7Nq1y/7Fl4NKdefB%0AxQUGDjTCwzPPNKG2d21+PvMzd7S8w9GliYg4REREBB06dGDevHkA+R73GzZsGLNmzeKZZ56hT58+%0AHDp0iFmzZtG0aVPS0tJsupbZbGbWrFlMnjyZUaNGMXnyZC5fvsyzzz5LSEiI1bwIw4YNY968edx/%0A//1MmTKFhIQE/v3vf+Pp6Zlv/oTw8HA++ugjli9fTtOmTfH09MwZm5DXnDlzGDx4MP379+fxxx/H%0Azc2NBQsWsH///nyhxp5atmzJ1KlTee211zCbzdx2220cP36cmTNn0rBhQ/7yl78AsHr1ahYsWMCo%0AUaNo0qQJmZmZrFixgsTERAYPHgzAP//5T86cOcPAgQOpV68ely9f5j//+Q/u7u42DWB1pEoVHgD6%0A94dHH4Vr10x0Ce1CTHz59k+KiDi7SZMm8eijj9KuXTu6dOlite+pp57i2rVrvPvuu8ydO5ewsDDe%0AfvttVqxYUaJHNfPeJch+iuPFF1/krrvuokmTJjz11FOsW7fO6nz9+/dn0aJFvPjii4wYMYL69esz%0AZcoU6tSpw+TJk63O+eyzzxIfH8/UqVNJSkqicePGHDt2rMB6+vTpw9q1a3n66ad58MEHycjIoGPH%0AjqxatYqhQ4cWWXtx2/O2ydvuzTffpFmzZrz77ru88cYb+Pv7c/vttzNnzhwCAwMBI2QEBgYyd+5c%0A4uLicHd3p3Xr1ixdupRx48YB0L17d9544w2mT5/OhQsXCAgIoEuXLqxdu9bmQayO4mxzb0YAMTEx%0AMVbTU1v65Rdo1w7WrIEfMv/B0t1LOfPXMwW2FZHqLXu63qL+ThGpDEry37LF9NSRwM7yrKdSjXkA%0AaNPGGPewYQN0qtuJuKQ4ziefd3RZIiIi1UalCw9mM/TunRUeQjoBaKZJERGRClTpwgNAnz6wZQvU%0A92mKr7svu85WjtGpIiIiVUGlDA+9e8ONGxC700yHuh2IPas7DyIiIhWlUoaHTp3Axyd33IPCg4iI%0ASMWplOHB1dV4z8WmTUZ4OJJwhKs3rzq6LBERkWqhUoYHgG7djGmqO9btRCaZ7Dm3x9EliYiIVAuV%0AbpKobF27wuzZ4HujLW5mN2LjY+nRoIejyxIRJ3TgwAFHlyBSJs7233ClDQ/Zk6jtinEnLChMT1yI%0ASD6+vr4AjB071sGViNhH9n/TjlZpw0PdutCgAWzfDp16aNCkiOTXokULDh8+TFJSkqNLESkzX1/f%0Acnvxl60qbXgA4+7Dtm0wanRHlu1dRlpGGq7mSv1HEhE7c5a/bEWqkko7YBKMcQ87dkC7Oh1ISU/h%0AcMJhR5ckIiJS5VXq8NClC1y9Cp5XjFe36okLERGR8lepw0NkJJhMcHhXTer71Vd4EBERqQCVOjz4%0A+0PLlkbXRYfgDuw+t9vRJYmIiFR5lTo8gDFV9e7dEB4crjsPIiIiFaDSh4eOHY3w0K5OOKevnObi%0A9YuOLklERKRKqxLhISkJaqaGAxo0KSIiUt6qRHgAuPJbSzxcPBQeREREylmlDw/BwcZsk3t3uxIW%0AFKbwICIiUs4qfXgA4+7Drl0aNCkiIlIRbA0Pfwe2A1eAc8BKoGUxx4wGvgfOA4nAZmCIjdctUocO%0AWU9cBIWz7/w+0jPS7Xl6ERERsWBreOgDvAZ0AwZjvBvjO8C7iGN6A98CtwMRwFrgC6CjrcUWpmNH%0AOHUKGnt34HradX69+Ku9Ti0iIiJ52PoWqdvzfJ+AcUchAthUyDF/yfN9BjASGA7Y5T3a2YMmM+Nz%0Ap6luVbuVPU4tIiIieZR1zENA1tqWyRXMgC+QUMZr52jRAry84Pj+OoTUCNG4BxERkXJUlvBgAuYB%0AG4H9Nhz3GEY3x8dluLYVFxcID88dNKlpqkVERMqPrd0Wll4HwoBeNhwTBTwNjAB+L6zRtGnTCAgI%0AsNoWFRVFVFRUoSfu2BE2b4bbx3Vg+S/LbShJRESkcomOjiY6Otpq2+XLlyvs+qZSHvcaRgDoA5wo%0A4TH3AouAu4GvC2kTAcTExMQQERFhU0FvvQX/93/w358/ZMIXY7k8/TL+nv42nUNERKSy2rlzJ5GR%0AkQCRwM7yvJat3RYmjDsOI4EBlDw4RAGLgfsoPDiUSYcOkJYG3kmaplpERKQ82Roe3gAeyFqSgbpZ%0Ai6dFmznAUovv9wPvYYx12G5xjF/pSi5Ye+NBCxKPtsLN7KbwICIiUk5sDQ9/wvilvw6Is1jGWLSp%0ACzSw+D4l6zpv5DlmfqkqLkSNGtC0KRz8xZ22ddpq0KSIiEg5sXXAZEnCxoQ83/vbeI1Sa98e9u2D%0A8F7h7D2/t6IuKyIiUq1UiXdbZGvXzggP7YPas/fcXjIyMxxdkoiISJVTpcJD+/YQFwdNvMNJTk3m%0At0u/ObokERGRKqdKhYd27Yy1y+/GExfquhAREbG/KhUeWrYENzc4c6gutb1r64kLERGRclClwoOb%0AG7RuDb/8YiI8OFzhQUREpBxUqfAAxriHvXuNQZMKDyIiIvZX5cJD7hMX4fx68VeSbyY7uiQREZEq%0ApcqFh/btITERgk3hZJLJLxd+cXRJIiIiVUqVCw/ZT1yknmmL2WRm7zk9cSEiImJPVS48NGxoTFX9%0A6wFvWtRsoXEPIiIidlblwoPZbNx92LsX44mL8woPIiIi9lTlwgNYT1O959weMjMzHV2SiIhIlVEl%0Aw0P79rB/P4TVDufi9YvEJcU5uiQREZEqo0qGh3btICUF/K5rmmoRERF7q5LhoX17Y33peCN83X01%0AaFJERMSOqmR4qFMHgoJg/y9m2gdrpkkRERF7qpLhASAszBg0GR6kd1yIiIjYU5UNDzlPXAS358Dv%0AB7iZftPRJYmIiFQJVTo8HDkCrQLDSctI49DvhxxdkoiISJVQpcNDejp4XDZGT6rrQkRExD6qbHgI%0ACzPWJw7508i/kcKDiIiInVTZ8ODvDw0aZA2a1DTVIiIidlNlwwPkn6ZaREREyq5ahIfw4HDikuJI%0AuJbg6JJEREQqvSofHo4fh2Y1NE21iIiIvVT58ACQEt8CDxcPdV2IiIjYQZUOD23agMkEB/e7EhYU%0ApvAgIiJiB7aEh78D24ErwDlgJdCymGPqAsuAQ0A6MK8UNZaalxc0b27xxIXCg4iISJnZEh76AK8B%0A3YDBgCvwHeBdxDEewHlgFrAbyCxdmaVn+cTFvvP7SM9Ir+gSREREqhRbwsPtwHvAAWAPMAFoCEQU%0AccwJYBrwAZBYyhrLxPKJi+tp1zl66agjyhAREakyyjLmISBrfdEehZSXdu3g7Flo4NYRgNj4WAdX%0AJCIiUrmVNjyYMMYvbAT2268c+8t+4uLssdo09G9ITHyMYwsSERGp5FxLedzrQBjQy461lIsWLcDN%0Azei66BzamR1xOxxdkoiISKVWmvDwGjAMYwBlnH3LMUybNo2AgACrbVFRUURFRdl8Ljc3aN3aCA+R%0A90cy96eFnAdIAAAgAElEQVS5ZGRmYDZV6adURUSkCouOjiY6Otpq2+XLlyvs+raEBxNGcLgT6Icx%0AGLJczJ8/n4iIosZh2iZ70OSo0M4kpiRy9OJRWtRqYbfzi4iIVKSC/kG9c+dOIiMjK+T6tvzz+w3g%0AgawlGWMOh7qAp0WbOcDSPMd1zFp8gaCsz21LWW+pZIeHiLrGD1VdFyIiIqVnS3j4E+AHrMPorshe%0Axli0qQs0yHPczqylE3B/1ufVpSu3dNq1g8uXIeVyLRoHNNagSRERkTKwpduiJEFjQimPK1fZT1xo%0A0KSIiEjZOfwXe0Vo3Bi8vbMGTYZEsjN+JxmZGY4uS0REpFKqFuHBbIawsNw7D0k3kziScMTRZYmI%0AiFRK1SI8gMWgyRDjKQ6NexARESmdahUefvkFAjxq0jSwqcY9iIiIlFK1Cg/Xr8Nvv2nQpIiISFlU%0Aq/AAuYMmY8/G6vXcIiIipVBtwkNICAQG5g6avHrzKocTDju6LBERkUqn2oQHk8m4+7B3rwZNioiI%0AlEW1CQ8A4eGwezcEeAbQomYLfj79s6NLEhERqXSqVXjo1AkOHYLkZOhevzs/n1F4EBERsVW1Cw+Z%0AmUbXRff63Yk9G8v11OuOLktERKRSqVbhISwMXF0hNtYID2kZacSejXV0WSIiIpVKtQoPHh7Qtq0R%0AHtoHtcfL1Yutp7c6uiwREZFKpVqFBzC6LmJjwc3Fjc6hnRUeREREbFQtw8PevZCaanRdKDyIiIjY%0AplqGh5QU46mL7vW7c+rKKc5cOePoskRERCqNahceOnQw1tmDJgE9sikiImKDahce/P2haVMjPIT6%0AhtLAr4G6LkRERGxQ7cID5A6aBI17EBERsVW1DQ+7dhkTRnWv350dcTtITU91dFkiIiKVQrUMDx07%0AwuXLcOKEER6up11nz7k9ji5LRESkUqiW4aFTJ2MdG2u8YdPdxZ2fTv3k2KJEREQqiWoZHkJCICjI%0ACA+erp50q9eNjSc3OrosERGRSqFahgeTyXrQZJ9GfdhwYgOZmZmOLUxERKQSqJbhAazDQ++GvTmf%0AfJ7DCYcdW5SIiEglUG3DQ0QEnDkDZ89CjwY9MJvM6roQEREpgWobHrp2Ndbbt4Ovhy8RIRFsOLHB%0AsUWJiIhUAtU2PDRsCHXqGOEBjK4LhQcREZHi2RIe/g5sB64A54CVQMsSHNcXiAGuA0eBP9pYY7kw%0AmaBLl9zw0KdRH04knuBk4knHFiYiIuLkbAkPfYDXgG7AYMAV+A7wLuKYJsBXwHqgI/AC8CowujTF%0A2lt2eMjMhF4NewGw8YTGPYiIiBTFlvBwO/AecADYA0wAGgIRRRzzJ+A48FfgEPAusAh4vBS12l2X%0ALpCQAMePQ23v2oTVCWP9ifWOLktERMSplWXMQ0DW+mIRbW7BuDth6TugM+BShmvbRZcuxjq762JA%0AkwGs+W2N4woSERGpBEobHkzAPGAjsL+IdsEY4yMsncPo8qhdymvbTVCQMXBy2zbj+6Cmgzh26RjH%0ALh1zbGEiIiJOzLWUx70OhAG97FhLjmnTphEQEGC1LSoqiqioKLtfy3LQZN9GfXExufDDsR+YGjnV%0A7tcSERGxh+joaKKjo622Xb58ucKubyrFMa8BIzAGUJ4opu16IBaYZrFtFLAc8ALS87SPAGJiYmKI%0AiChqKIX9zJ0Lzz0HiYng4gI9F/Wknm89Pr7n4wq5voiIiD3s3LmTyMhIgEhgZ3ley5ZuCxPGHYeR%0AwACKDw4AWzCezLA0BOORz7zBwSG6dIHkZDh40Pg+qMkg1vy2hozMDMcWJiIi4qRsCQ9vAA9kLclA%0A3azF06LNHGCpxfe3gEbAy0AbYGLW8u/Sl2xfkZHGnA/ZXReDmg7i4vWLxMbHOrYwERERJ2VLePgT%0A4AesA+IsljEWbeoCDSy+HweGAv0wui+eAv4PY4Ipp+DnB61a5YaHbvW74ePmww/HfnBsYSIiIk7K%0AlgGTJQkaEwrYtgGj/8Vpde0KW7can91d3OnbuC8//PYD03tNd2xhIiIiTqjavtvCUo8esHu3MfYB%0AYHDTwWw8sZGrN686tjAREREnpPCAER7S03O7Loa1HEZKeoq6LkRERAqg8AC0bWuMfdi82fjevGZz%0AWtduzerDqx1bmIiIiBNSeMCY36F799zwADC85XBWH16tRzZFRETyUHjI0qMHbNkCGVlZYVjLYZxL%0APseOuB2OLUxERMTJKDxk6dEDLl6Ew4ezvjfoQaBnIF8c+sKxhYmIiDgZhYcs3bqB2ZzbdeFqdmVo%0Ai6F8cVjhQURExJLCQxY/P2jfHn76KXfb8JbD2X1uNycTTzquMBERESej8GChRw/rQZO3Nb8Ndxd3%0AVh5wmgkxRUREHE7hwUKPHsYLshISjO/+nv7c2uxWPt6vN2yKiIhkU3iw0KOHsc6eqhpgTNgYNp/a%0AzKnEU44pSkRExMkoPFho0gSCg2HTptxtI1qNwMPFg//t/5/jChMREXEiCg8WTCbo0wc2bMjd5ufh%0Ax23Nb1PXhYiISBaFhzz69YNt23JfkgVwT9t72Hp6q566EBERQeEhn759IS3NmG0y2/BWw/Fw8eCT%0AXz5xXGEiIiJOQuEhj7ZtoXZtWLcud5ufhx/DWg7j/T3vO6wuERERZ6HwkIfJZHRdrF9vvX18h/Hs%0APrebXWd3OaQuERERZ6HwUIC+feHnn+HatdxttzW/jSCfIJbuWuq4wkRERJyAwkMB+vWD1FTr+R7c%0AXNwY234sH+79kNT0VIfVJiIi4mgKDwVo2xZq1bIe9wAwvuN4Lly7wNe/fu2QukRERJyBwkMBzGaj%0A6yJveAgPDqdT3U4s2bXEEWWJiIg4BYWHQgwYYHRbXL1qvX1Cxwl8cfgL4pPiHVOYiIiIgyk8FGLI%0AEGPcQ96nLsZ1GIe7izvv7HzHMYWJiIg4mMJDIZo3h8aN4bvvrLcHeAbwQPsHeDvmbdIy0hxSm4iI%0AiCMpPBTCZDLuPnz7bf59D3d5mDNJZ1h1aFXFFyYiIuJgCg9FGDIEDh2CEyest3es25EeDXqwYPsC%0AxxQmIiLiQAoPRRgwwHjy4vvv8+97uPPDrPltDQcuHKj4wkRERBxI4aEIgYHQrVv+cQ8Ad7e9m1Df%0AUP69+d8VX5iIiIgD2Roe+gBfAGeADODOEhzzCHAAuAYcBMbZeE2HGjIEfvgB0tOtt3u4ejCt2zTe%0A3/M+cUlxjilORETEAWwND95ALEYgAMgspv1DwAvAP4G2wNPAG8AwG6/rMEOGwKVLEBOTf9/UyKl4%0AuXkxf+v8ii9MRETEQWwND99gBIHPSth+HPAW8AlwHFgOvAtMt/G6DtO1K/j5FfzUhb+nPw91foi3%0AdrzF5RuXK744ERERByjvMQ/uQEqebTeAroBLOV/bLlxdYdAg+LqQ11n8v27/j5T0FN7c/mbFFiYi%0AIuIg5R0evgUmAxGACegMTARcgdrlfG27GT7cmKr6/Pn8+0J8Q5jcaTIvbX6JxBuJFV+ciIhIBXMt%0A5/PPAuoCWzHCw1lgMfAExoDLAk2bNo2AgACrbVFRUURFRZVfpUUYOtRYf/klTJiQf/+MPjNYvGsx%0AL295mef6P1exxYmISLUTHR1NdHS01bbLlyuu+9xUhmMzgJFASaZZdAGCgXjgT8C/AP8C2kUAMTEx%0AMURERJShNPvr0QPq1oUVKwre/8T3T/Dmjjc59ugx6vjUqdjiRESk2tu5cyeRkZEAkcDO8rxWRc3z%0AkA7EYTydcR/G456VyogRxnwPN24UvH96z+mYMDFn05yKLUxERKSC2RoefICOWQtA06zPDbK+zwGW%0AWrRvAYzNWncFPsJ4ZPMfpazXYYYPh+Rk+PHHgvfX8q7F33r8jTe2v8GvF3+t2OJEREQqkK3hoQvG%0ArZCdGHcRXsn6/GzW/rrkBgkwuiv+CuwCvsN4+qIHcLL0JTtG27bQpAl8UcQ9k8d6PEbdGnX567d/%0ArbjCREREKpit4WFd1jFmjGCQ/Xli1v4JwACL9gcxxjH4AAHAaOBI6ct1HJPJuPuwahVkFDLU09vN%0Am5eHvMwXh7/g6yOFPNspIiJSyendFjYYNQrOnIFt2wpvc1ebuxjQZAD/75v/R0pa3ikuREREKj+F%0ABxv07g3BwfDxx4W3MZlMvHrbq/x2+Tee3/h8xRUnIiJSQRQebODiAnffDf/7X+FdFwBhQWHM6D2D%0AFza+wM74cn1aRkREpMIpPNjonnvg1Cn4+eei2/29999pF9SOCZ9P4Gb6zYopTkREpAIoPNioVy9j%0Asqiiui4A3F3cWTJyCfsv7OeZdc9USG0iIiIVQeHBRiXtugDoWLcjz/V7jjmb5vDNr99UTIEiIiLl%0ATOGhFO65B06fhi1bim87vdd0bm9+O2NXjOVU4qnyL05ERKScKTyUQq9e0KABvP9+8W3NJjPvjXoP%0ALzcvxvxvDDfSCpnfWkREpJJQeCgFsxnGjYPlywt/14Wl2t61+d89/2PX2V1M/HwimZmZ5V+kiIhI%0AOVF4KKU//AEuXzZmnCyJbvW78d7I94jeF83T654u3+JERETKkcJDKbVqBd27w9KlxbfNdk/YPcwZ%0AOIdZG2bxxrY3yq84ERGRcuTq6AIqs/Hj4c9/hrNnjcc3S2J6z+mcTz7Pn7/+Mx6uHkyOmFy+RYqI%0AiNiZ7jyUwb33gqsrfPhhyY8xmUy8PORlHur8EFO/mMr7u0sw6lJERMSJKDyUQWAgjBwJ//0v2DIG%0A0mQy8frQ15nYaSLjPxvPgu0Lyq9IERERO1O3RRk99BD06wc//ggDBhTbPIfZZGbh8IX4uvvyyFeP%0AcPbqWZ7t9ywmk6ncahUREbEHhYcy6tMH2raFN9+0LTyAESBeufUVQnxDmP7DdM5cOcOCOxbg4epR%0APsWKiIjYgbotyshkMu4+rFwJcXGlOd7EEz2f4L2R7/Hh3g/pu6Qvp6+ctn+hIiIidqLwYAfjxoGH%0AhzH2odTn6DCOjRM2cibpDJELI1n721r7FSgiImJHCg924O8PY8fCwoVwswxv3+5SrwsxU2NoF9SO%0AQe8N4vHvHtd01iIi4nQUHuzk0UeNbovo6LKdJ8gniO/Hfc9Lg1/itW2v0eW/Xdh1dpd9ihQREbED%0AhQc7CQuDYcPgpZeKf1V3ccwmM4/1eIwdU3ZgNpnpvLAzj337GEkpSfYpVkREpAwUHuzoiSfgl1/g%0A66/tc772we3ZPmU7swfM5s0db9LmjTZ88ssnerGWiIg4lMKDHfXqZbzvYu5c+53T3cWdJ3s9yYFH%0ADtA5tDNj/jeGnot6svHERvtdRERExAYKD3ZkMhl3HzZsgM2b7XvuRgGN+Oy+z/h+3PekpKfQZ0kf%0Ahi0bxp5ze+x7IRERkWIoPNjZnXdCu3bwz3+Wz/kHNR3E9inb+eiujzj4+0E6vNWB4dHD2XzKzmlF%0ARESkEAoPdmY2w6xZsGaNMWV1uVzDZObedvdy4JEDLB25lKMXj9JzUU/6LunLqkOrSM9IL58Li4iI%0AoPBQLu68EyIjYeZM216YZSs3Fzf+0OEP7Ht4HyvvXcnN9Jvc+dGdNH21KS9sfIFzV8+V38VFRKTa%0AUngoByYTzJ4NP/0E33xT/tczm8yMbD2SLZO2sH3KdgY1GcSsDbNoMK8BUZ9G8d3R73Q3QkRE7MbW%0A8NAH+AI4A2QAd5bgmD8Ae4BkIA5YBNS08bqVzq23Qu/eMH06pKVV3HU7h3bm3Tvf5cxfz/DioBeJ%0AjY/l1g9upcG8Bvztu79pgKWIiJSZreHBG4gFHsn6XtxN+X4YYWEh0Ba4B+gCvGPjdSsdkwnmzYN9%0A+8r2zovSqulVk7/c8hcOPHKAbZO3cVebu1iyewkd3upAh7c6MHvDbPZf2K85I0RExGamMhybAYwE%0AVhXR5nHgT0Bzi23/B/wNaFhA+wggJiYmhoiIiDKU5jwmToRVq+DwYajp4PstqempfPPrNyzbt4zV%0Ah1dz9eZVWtZqyajWoxjVehRd6nXBbFJPlohIZbRz504iIyMBIoGd5Xmt8v5N8R0QDNyOEVSCMe4+%0ArC7n6zqNF16AlBR45hlHV2IMsBzeajjRd0Vz4W8XWB21mt4Ne/Nu7Lt0f7c7wf8O5v5P72fJriWc%0AuXLG0eWKiIiTci3n8+/BGPPwCeCedb3PgUfL+bpOo25dY86HJ5+E8eONpzCcgaerJ3e0vIM7Wt7B%0AWxlvsfnUZr799Vu+P/Y9H+37iEwyaVunLYOaDKJPoz70bNiTujXqOrpsERFxAuXdbdEd+BZ4Lmsd%0ACrwEbAcmF9A+Aojp3bs3AQEBVjuioqKIiooqQ7mOk5oKXbsaj21u3w5ubo6uqGgJ1xJY89savjv6%0AHWt/W8tvl38DoHnN5vRq2IteDXrRq2EvWtZqiclUlv+ERESkNKKjo4nO8xrny5cvs3HjRqiAbovy%0ADg/Ls64xxmJbT2AjEALknYigyo15yBYTYwSI55837kJUJmeunOGnUz+x6eQmNp3cxO5zu8nIzKCW%0AVy261OtC55DOdA7tTJd6XQj1DXV0uSIi1VJFjnko724LE5B3goEMi33VRmQkPPaYMfZh5Eho3drR%0AFZVcPb96jAkbw5gwIwNeSbnC1tNb2XxqMzvidrBw50Jmb5wNQEiNEDqHGmGiQ3AH2ge3p3FAYw3E%0AFBGpQmz9Be4DtMj6vBP4K7AOSABOAXMwuibGZ7W5H1iCMcbhO4y7DfOBNOCWAs5fZe88AFy7ZoQI%0ALy/YsgU8PBxdkX1kZmZy+sppdsTtYEfcDrbHbWdH3A4u3bgEgLebN2F1wmgf1J52Qe1oF9SOtnXa%0AEuobqm4PERE7ceY7D12AtVmfM4FXsj4vASYCdYEGFu2XAf7An4GXgcvAGmB66cqt3Ly9Ydky6NYN%0AnnoK/v1vR1dkHyaTiQb+DWjg34BRbUYBRqCIS4pj3/l97Du/j73n97L73G6i90VzPe06YISKFjVb%0A0KJWC2Nt8TnIJ0jBQkTESTnb385V+s5DtldeMbowvv0WhgxxdDUVKz0jnd8u/8aBCwc4cvEIRxKO%0AcPjiYY4kHOHUlVM57XzdfWkU0IhG/llLQCMa+jfM+Vy3Rl11hYiIWHDmOw9iB9Omwfffw/33w44d%0A0LixoyuqOC5mF5rXbE7zms3z7bueep2jl45yJOEIv178lROJJziZeJJNpzbx4d4PSUxJzGnrZnaj%0AgX8D6vnWI9Q3lJAaIYT4hlh9DqkRQoBngO5giIjYmcKDA5jN8OGH0LkzjBplvEDL29vRVTmel5tX%0AzpiIgiTeSORk4klOJJ7gxOUTnEg8QVxSHPFX49l9bjfxSfFWAQOM+Syyw0TdGnWp412HIJ+g3LVP%0A7vda3rVwNet/CRGR4uhvSgepWRM++wxuuQUmTzbChP6BXDR/T3/ae7anfXD7QttcS71GfFI88Vfj%0AiU+KzwkX8VfjOXf1HL9d+o0L1y5wPvk8N9NvWh1rwkRNr5pWgSJv0KjlVYuaXjVzlhruNXRnQ0Sq%0AHYUHBwoPh8WL4d57oW1bmDHD0RVVft5u3jSr2YxmNZsV2S4zM5Okm0mcTz7PhWQjTFy4dsHq8/nk%0A8xy9dDSnTWpGar7zuJpdc4JE3mCRdwn0DMTf0x9/D3/8Pf3xdPUsrx+DiEi5UnhwsDFj4OBBmDkT%0AQkJg0iRHV1Q9mEwm/Dz88PPwK3D8RV6ZmZkkpiRy8frFYpdfL/6a8znhegJpGQW/k93DxcMqTFit%0AC9pWwFoBREQcQeHBCcycCfHxMHUq1KkDI0Y4uiLJy2QyEeAZQIBnAE0Dm5b4uMzMTJJTk40gcS2B%0AxJREEm8kFrzO+hyXFGe1Lzk1udDzu7u44+vui6+Hb8Frd1/8PPwK3++R28bH3UdPsIhIiSg8OAGT%0ACV5/Hc6fN7owVq+GgQMdXZXYg8lkooZ7DWq416Chf0FvoS9eWkYaV1KuFBo6klKSSLqZlLu+mUTi%0AjUROXzmds+1KyhWSUpJIz8w74au1Gu41ig0ZRe33cfehhnsNfNx88HbzxsXsUqo/s4g4N4UHJ+Hi%0AYgyaHDUKhg0zBlPeequjqxJnYDmuoiwyMzO5kXbDOmjkWWeHjLxh5MyVM/naXUu9Vuw1PV09c8KE%0AZbDwcffBx82n+H1Zn/Pu93L10kBVEQdSeHAinp5GaLj7bqPrYuVKGDrU0VVJVWEymfBy88LLzYsg%0An6Ayny89I52rN69ahYrkm8kkpyaTfDOZqzev5nxOTs36bvH5wrULHL98PN++koQSE6ZCg0XOZ4vv%0A2dtyPltst9ynrhuRklF4cDIeHvDpp0b3xZ13wrvvwh/+4OiqRPJzMbsYgzc9/e163ozMDK6lXisw%0AdOQNJQUGlNRkLl2/lPPZ8vi8j+cWxMvVq9iQkfdz3rZ5g4q3m7dCiVQpCg9OyN0dPv4YHn4Yxo+H%0AEyeMxzh1l1aqA7PJnPOL195upt+0ChrZS3YQsQwcVvtSjc/ZoSRvu8KeqLFkeXej0ACS57uvh2/O%0AOJSCvmtMiTiKwoOTcnODhQuNqatnzIDjx+HNN41gISKl4+7ijruXO4FegXY7Z2ZmphFKLMJEsWHk%0A5lWupuZ+/v3a7wW2zcjMKPLaXq5exQYMy+2Ftsn67uHiobEkUiIKD07MZDLevtmwoTH/w8GD8Mkn%0AEBrq6MpEJJvJZMLD1QMPV48yD2q1lJmZyfW068a4kqwxJdmfs8eaFPg962mb7EGulm1S0lOKvKar%0A2bXogOFmvT1vGy83LzxdPfF09cTDxSPns6erJ+4u7gomVYjCQyUwbhy0aGEMpIyIMLo0+vRxdFUi%0AUp5MJhPebt54u3nbZYArQGp6ar7gUdJQcjLxZL5tV29eten6eQOFp6snHq4FbCuonYsH7i7ueLga%0A67xL9v5824to7+biprEopaTwUEl07w47dxoDKQcMgFmz4IknjEc8RURKws3FjUCvQLt122QPbs0O%0AITfSbpCSlsKNtBv5lpT0QrZnt0/P3XbpxqUCz3Uz/abVUtCU8bZyNbuWPYSY3XBzcbNau7u459tm%0AuXZ3cS/TPkePd1F4qESCgoxXef/zn0Z3xldfwXvvQZMmjq5MRKojy8GtIYRU+PWzx5sUtKSkpxS8%0APa2Q7UW1z7DediXlSs55UtJTSE1PJTUj1WqdHW6yt5VkUK0tTJjyhQvi7XqJIik8VDKurvDCC3D7%0A7UZ3RocO8OqrxlMZ6k4UkerEcryJs8vMzCQtI63QcFFU8Mi7vpl+s8B9Jw+eZCELK+TPo/BQSfXu%0ADXv2wKOPwoQJEB0NCxZAs6JfJikiIg5gMmXdKXBxA7fyucbOGjsrLDxopEgl5ucHS5YY78I4dAja%0AtYPnn4ebxc+DIyIiUmoKD1XAHXfAL78YdyGefhrCw+GLLyAz09GViYhIVaTwUEX4+MCLL0JsLNSr%0AZ7wbY+BA4wkNERERe1J4qGLat4cffoAvv4Rz5yAyEqKi4MABR1cmIiJVhcJDFWQyGW/j3L0b3n4b%0AfvoJwsLgvvuM7g0REZGyUHiowlxdYepUOHLEeC/Gli3GnYm77zY+i4iIlIbCQzXg4QF//KMRIt5+%0A23jEs0cPuOUW410Zafadu0RERKo4hYdqxN0dpkwxXrC1ahV4ecGYMdC8OfzrX3D2rKMrFBGRykDh%0AoRoym2H4cFi71ngao29fePZZaNAARo+Gr7+G9HRHVykiIs7K1vDQB/gCOANkAHcW035JVru8yz4b%0AryvlpFMnWLoU4uJg3jw4etQYbNmkCcyYoQGWIiKSn63hwRuIBR7J+l7cNESPAnUtlgbAReBjG68r%0A5SwwEP78Z9i1C7Ztg9tugzfeMGatDA+HOXPgt98cXaWIiDgDW8PDN8A/gc9K2P4KcN5i6QIEAott%0AvK5UEJMJunSBhQuNMRCff2485jl7NjRtaux77jkjZGgGSxGR6qmixzxMAr4HTlXwdaUUPDyMmSqj%0Ao+H8eVi2zAgQL79sdHc0agQPP2yMkbh2zdHViohIRanIt2qGArcBURV4TbETHx9jpsqoKOPFWxs2%0AGE9sfPGFMYeEu7vx6OfAgcbSpQu4ldOb40RExLEq8s7DeOASJe/yECfl7g6DBsGrr8KxY7BvH7z0%0AEvj7w7//DT17Qq1axsDL2bNhzRpISnJ01SIiYi+mMhybAYwEVpXwOoez2j5WRLsIIKZ3794EBARY%0A7YiKiiIqSjctnF1amvH455o1xt2JrVvh8mXj8dD27Y3Jqbp1M7o92rTR3QkRkdKIjo4mOjraatvl%0Ay5fZuHEjQCRQrq9FrKjw0A9YC7QD9hfRLgKIiYmJISIiogylibPIyDAmpdqyBTZvNtbZL+ny8DCe%0A5ujUCTp2NNbh4VCjhmNrFhGpjHbu3ElkZCRUQHiwdcyDD9DC4ntToCOQgDEIcg7G2IbxeY6bBGyl%0A6OAgVZDZDG3bGsukSca2K1eMl3bFxhrL9u2wZEnuNNkNGhh3Jdq0gdatcz/XqWM8DSIiIo5la3jo%0AgnEHAYw5Hl7J+rwEmEjuXA6W/IHRGHM+iODnB717G0u2lBTYv99478bBg8bdia+/htdfz53tsmZN%0AaNXKeOIj7xIaagQVEREpf7aGh3UUPchyQgHbEjHuWIgUysPD6Lbo1Ml6+82b8OuvRpg4cMB4udex%0AY/Djj8asmJbHN2liBIlGjaB+feMORva6Xj3jXR4iIlJ2FfmopojN3N1zuz3yun4djh83wsTRo7nr%0AzZvh1Cm4eNG6fe3a1qGifn0ICYG6dSE42FgHBRmvMhcRkcLpr0mptLy8csdDFOTaNTh92lhOnbJe%0AFxYwTCbjMdO6da1DRfbnOnWMEFKrlrGuUUPjMESk+lF4kCrL2xtatjSWwty8acyeefassZw7Z/35%0A5EljQOfZs8ZAz7zc3IwQYRko8q5r1zbGawQEGO8QCQgw7qiIiFRWCg9Srbm753ZhFOf6dfj9d0hI%0AMNaFfT52LHdbYdN2e3sbIcIyUGSvi9vm56fBoSLiWAoPIiXk5WWMl2iQ93miIly/boSKhARITIRL%0Al1m7C9sAAAnvSURBVIxJs7LXlp9PnjQeYc3edvVqwec0mcDX15jR08/PWAr6XNx+b291uYhI6Sg8%0AiJQjL6+S39nIKy3NOnBkh4pLl4wulOwlMdFYJyQYr0233FbUC8tcXIoOF76+hS9593t4KIiIVCcK%0ADyJOytXVGDdRq1bpz5GWVnDQyF4X9PnsWTh0yHgfSfaSnFx8rYUFi+KCR96lRg0j2IiI81J4EKnC%0AXF2NwZo1a5btPOnpRoCwDBRXrlh/L2xfXFz+fdmziRbGx6d0waOg/borImJ/Cg8iUizLLo6yysw0%0AZhQtSfDIu8TF5d9vr7siJQkmNWposKoIKDyISAUzmcDT01jq1Cn7+SzvihQVPArab3lXJHtfWe+K%0ABATkdjflXWrW1CRkUjXoP2MRqdQs74rUq1e2cxV1V6S4bpozZ4zPly4Zg1cLmhcEjAGp2WGidm1j%0AArLQUGO205CQ3M916xpdLiLOSOFBRCSLPe+KpKYaM5hmP6pruWRvv3DBeCHcmjUQH28cY6lWLSNI%0AZD+xU69e/s8BARrTIRVP4UFEpBy4uRlTmgcHl6x9RoYRKuLijCARH298josz7mrs2gWrVxszn2Zm%0A5h7n7V14sMj+HBSksRpiXwoPIiJOwGzOnc48PLzwdqmpRrA4fdoIFdnvbzlzxpjddMMGI3BY3sVw%0AczO6QyxDhWXICA01ruvjo7sYUjIKDyIilYibGzRsaCyFycgwukQKChinTxt3MU6fzj+JmLt77qO9%0A2QM8864tJxHLHmuS/dnNrXz/7OI8FB5ERKoYszm3yyQysuA2mZnGpGCnTxt3KrLHYmSPx8he79+f%0A+/3SJSOYFMbT0/rJk+y1t7f14uWVf1tRbTw8cpf/3969xshVlgEc/0+73RZKKEQsLVACBEG0ERAq%0AFki5BAyiwRJU1A9E0BjxA/GLAl7CRq4Jl3InQMBQiI33IARM4zV4CQqLGLFowGKFLrfgYovb3W23%0AfHjOZM4cZnfnzJxzdrv7/yUne+Y978yeeXZ25pn3vBdbRqYHkwdJmoVqtcaia8uXt3efsbHmYbH1%0AESjpWUyzx7ZsianVBwaipaPVlkdvb3MysWBB8+08ZfPnx+PNmxdbfb9VWbvHe3pmR4Jj8iBJasuc%0AOY15Lfbbr5jH3LkTtm2LJGJoqHVyMTwcdYaHm7dWZenywcHJ646OwshIcyfUbtWTiXaTj56e2O/p%0A6W5/YKC45zAZkwdJ0pSp1eISxW67Te157NgRScToaCOhSP8cb7+I49u3xzY6GknO1q2xXy9LH59o%0Af3i4uniZPEiSZr25c6dHEtON/v7x+7gUzZG/kiQpF5MHSZKUi8mDJEnKxeRBkiTlYvIgSZJyMXmQ%0AJEm5mDxIkqRcTB7EunXrpvoUZh1jXj1jXj1jPnPlTR5WAQ8BLwFjwCfauM984ErgBWAb8Bxwfs7f%0AqxL5D149Y149Y149Yz5z5Z1hcnfgKeAe4CdAO7OB/wB4N3ABkTgsBly4VZKkXVTe5OHnydauM4jW%0AioOBwaRsU87fKUmSppGy+zycBTwBXAK8CPwDuBZYUPLvlSRJJSl7YaxDgBOBIWA1cfniduBdxGWM%0AljZs2FDyaSltcHCQ/v7+qT6NWcWYV8+YV8+YV6vKz85aF/cdIxKCn01QZz1wArAE2JKUnQ38iOg/%0AkV1AdCnwZ2D/Ls5LkqTZ6iVgBTBQ5i8pu+VhANhMI3EAeJZIWg4Anm9RfwWRREiSpHwGKDlxgPKT%0Ah98BnwQWAm8lZYcRrRYvjnOfSp64JEmqxkLgqGQbA76a7C9Ljl8N3Jepv4kYrnkEMfLin8CdFZ2v%0AJEmaYicTScMYsCO1f29y/LvArzL3OZzo+/AWkUhcS0wcJUmSJEmSJEmSJEmSNBN8BdhITCj1BDG5%0AlDpzKTFfxv+AV4CfEqNcsvqIMcH/B34NvC9zfD5wC/AasBV4EOfgaMclRF+gNZnyPox30fYHHgBe%0AJ/pVPQV8MFOnD+NelHlEx/iNRDyfB77NO+cM6sOYd6KdxSf76D62ewP3E8tGDAJrgUVFPIGqnUtM%0AGHUB0cFyDTE3xLKJ7qRxPQqcR4xw+QDxYnyBmJir7mLiRbMaeD+wjnhB7pGqcwfwH+BUYlTNL4k3%0AZ5dyH98K4F/AX4AbUuXGu3h7E6/re4BjgQOBU4iZbeuMe7EuIz6UPkrE+xziS8pFqTrGvHNnAN8h%0AYjdGLPGQVlRsHwWeBo4DPgz8lYknfJy2Hgduy5T9HbhqCs5lJtqHeCHWW3NqxFwaX0vV6QX+C3wp%0Aub2ISOg+laqzFNgOfKTMk92F7UGs33Iq8Y2gnjwY73JcA/x2guPGvXgPAXdnyn5MY4i+MS9ONnko%0AKrZHJI+9IlXnuKSsVQt1S9Mhy+slmhnXZ8rXA8dXfzoz0l7JzzeSnwcD+9Ic8xHijbge82OIJsp0%0AnQHgb/h3Gc9twMPEcOV0M67xLsdZwJPAD4nLc/3AF1PHjXvxHgZOA96T3D6SWILgkeS2MS9Pt7Fd%0AmdxeCbxJXNquezwpW0mbyp5hsh37AHOJf/60V4k1MdSdGnEZ6DGiNQcacW0V8wNTdUaIF1TaK8QL%0AWM0+QzQR1rP5naljxrschwAXAtcDVwAfAm4m4rgW416GO4GDiBa27cR79zeA7yfHjXl5uo3tklSd%0AV1s8fq7P3OmQPKhctxLXxtrtgLpz8irKWAbcRHwjG0nKarS38Jzx7twc4E/At5LbTwPLgS8TycNE%0AjHtnLgI+TyTLzwBHAzcS326N+dSZLLbdLILZ0nS4bPE6MVtlNuPcF9e46NYtwMeJTmSbU+UvJz9b%0AxfzlVJ1e3tkDd0mqjsIxxHLz/cBosq0i3mhHMN5l2UyjNa3uWRrfwox78b4JXE4sOfAMMdJlDTHC%0AC4x5mbqJbbbO4haPv5gc8Z8OycMIcd0y21HmdOAP1Z/OjFAjWhxWE533/p05vpF4kaRj3gucRCPm%0ATxIfguk6S4lWDP8uzX5BfOM9MtmOIoYbP5DsG+9y/B54b6bsMGIEBhj3MtSIL3tpYzS+2Rrz8hQV%0A2z8SyUW2w+QidsH4f5roIXo+0RN0DTH8x6Ganbmd6IG7isjm69uCVJ2vJ3VWEx983yNWOl2YeZxN%0ARAJyNDHkp58SmsBmoN/QPM+D8S7escSXj0uBQ4HPEePaP5uqY9yLdRcxDPBMou/D2cS18qtTdYx5%0A5yZbfLKo2D5CDCdPD9V8sIwnVIULicxqG9EL1EmiOpdduKy+nZepdxnR9DtE68lGeokOaPUJeJzI%0ApX3poZp1xrt4HyPe+IaIZvQvtKhj3IuzELiOxiRRzxHzEmT7zxnzzpzMxItPQjGx3YuYJOrNZFsL%0A7Fnc05AkSZIkSZIkSZIkSZIkSZIkSZIkSZIkSZIkSZIkSZIkSZIkSdJs9jZDzNHw6ay/GAAAAABJ%0ARU5ErkJggg==%0A)

