Inception3 - last years baseline

From: https://www.kaggle.com/feichin/inception3-last-years-baseline

Author: Florian Eichin

Score: 0.51697

iNaturalist Competition 2018 Training Code

I did not write the following code myself. I copied this from last years baseline and only did some refactorization to make the code run in this kernel. Actually, I was just exploring the dataset and trying to see how past solutions do. All the credit goes to user macaoda.

So far, I managed to make the pre trained model run and also trained it on a few epochs over the training data. As you can see, the scores do not match this years baseline yet. According to user zz, scores of at least 0.22 are possible with slight changes. Sadly, I will not be able to spend more time with this competition.

Hope, this kernel helps you in some way.

Step 0: Import modules

In [1]:
import os
import shutil
import time
import numpy as np

import torch
import torch.nn as nn
import torch.backends.cudnn as cudnn
import torch.optim
import torch.utils.data
#import torchvision.models as models

import urllib.request

import torch.nn.functional as F
import torch.utils.model_zoo as model_zoo

import torch.utils.data as data
from PIL import Image
import os
import json
from torchvision import transforms
import random
import numpy as np

Step 1: Download pre-trained model and define global variables

In [2]:
# uncomment this to work with last years baseline model
# performance will however be bad, with this commented out, we will just work
# on a pre trained image net model
# print('Downloading model files...', end="")
# url = 'http://vision.caltech.edu/~macaodha/inat2018/iNat_2018_InceptionV3.pth.tar'  
# urllib.request.urlretrieve(url, 'iNat_2018_InceptionV3.pth.tar')
# print('Done.')
In [3]:
class Params:
    # arch = 'inception_v3'
    num_classes = 1010
    workers = 8
    epochs = 4
    start_epoch = 0
    batch_size = 64  # might want to make smaller 
    lr = 0.0045
    lr_decay = 0.94
    epoch_decay = 4
    momentum = 0.9
    weight_decay = 1e-4
    print_freq = 100

    resume = 'iNat_2018_InceptionV3.pth.tar'    # path to trained model
    train_file = '../input/train2019.json'      # path to train file
    val_file = '../input/test2019.json'         # path to test file
    data_root_train = '../input/train_val2019/' # path to train images
    data_root_test = '../input/test2019/'       # path to test images
    op_file_name = 'submission.csv'             # submission filename

    # set evaluate to True to run the test set
    evaluate = True
    save_preds = True

Step 2: Model initialization

I simply copied the code from the different modules and did some refactorization to execute it in this kernel.

In [4]:
# This code was copied from https://github.com/macaodha/inat_comp_2018/blob/master/inception.py

# Same as the version from the official start_epoch
# ttps://github.com/pytorch/vision/blob/master/torchvision/models/inception.py
# Only change being that it can take variable sized inputs
# See line 122

__all__ = ['Inception3', 'inception_v3']


model_urls = {
    # Inception v3 ported from TensorFlow
    'inception_v3_google': 'https://download.pytorch.org/models/inception_v3_google-1a9a5a14.pth',
}


def inception_v3(pretrained=False, **kwargs):
    r"""Inception v3 model architecture from
    `"Rethinking the Inception Architecture for Computer Vision" <http://arxiv.org/abs/1512.00567>`_.
    Args:
        pretrained (bool): If True, returns a model pre-trained on ImageNet
    """
    if pretrained:
        if 'transform_input' not in kwargs:
            kwargs['transform_input'] = True
        model = Inception3(**kwargs)
        model.load_state_dict(model_zoo.load_url(model_urls['inception_v3_google']))
        return model

    return Inception3(**kwargs)


class Inception3(nn.Module):

    def __init__(self, num_classes=1000, aux_logits=True, transform_input=False):
        super(Inception3, self).__init__()
        self.aux_logits = aux_logits
        self.transform_input = transform_input
        self.Conv2d_1a_3x3 = BasicConv2d(3, 32, kernel_size=3, stride=2)
        self.Conv2d_2a_3x3 = BasicConv2d(32, 32, kernel_size=3)
        self.Conv2d_2b_3x3 = BasicConv2d(32, 64, kernel_size=3, padding=1)
        self.Conv2d_3b_1x1 = BasicConv2d(64, 80, kernel_size=1)
        self.Conv2d_4a_3x3 = BasicConv2d(80, 192, kernel_size=3)
        self.Mixed_5b = InceptionA(192, pool_features=32)
        self.Mixed_5c = InceptionA(256, pool_features=64)
        self.Mixed_5d = InceptionA(288, pool_features=64)
        self.Mixed_6a = InceptionB(288)
        self.Mixed_6b = InceptionC(768, channels_7x7=128)
        self.Mixed_6c = InceptionC(768, channels_7x7=160)
        self.Mixed_6d = InceptionC(768, channels_7x7=160)
        self.Mixed_6e = InceptionC(768, channels_7x7=192)
        if aux_logits:
            self.AuxLogits = InceptionAux(768, num_classes)
        self.Mixed_7a = InceptionD(768)
        self.Mixed_7b = InceptionE(1280)
        self.Mixed_7c = InceptionE(2048)
        self.fc = nn.Linear(2048, num_classes)

        for m in self.modules():
            if isinstance(m, nn.Conv2d) or isinstance(m, nn.Linear):
                import scipy.stats as stats
                stddev = m.stddev if hasattr(m, 'stddev') else 0.1
                X = stats.truncnorm(-2, 2, scale=stddev)
                values = torch.Tensor(X.rvs(m.weight.data.numel()))
                values = values.view(m.weight.data.size())
                m.weight.data.copy_(values)
            elif isinstance(m, nn.BatchNorm2d):
                m.weight.data.fill_(1)
                m.bias.data.zero_()

    def forward(self, x):
        if self.transform_input:
            x = x.clone()
            x[:, 0] = x[:, 0] * (0.229 / 0.5) + (0.485 - 0.5) / 0.5
            x[:, 1] = x[:, 1] * (0.224 / 0.5) + (0.456 - 0.5) / 0.5
            x[:, 2] = x[:, 2] * (0.225 / 0.5) + (0.406 - 0.5) / 0.5
        # 299 x 299 x 3
        x = self.Conv2d_1a_3x3(x)
        # 149 x 149 x 32
        x = self.Conv2d_2a_3x3(x)
        # 147 x 147 x 32
        x = self.Conv2d_2b_3x3(x)
        # 147 x 147 x 64
        x = F.max_pool2d(x, kernel_size=3, stride=2)
        # 73 x 73 x 64
        x = self.Conv2d_3b_1x1(x)
        # 73 x 73 x 80
        x = self.Conv2d_4a_3x3(x)
        # 71 x 71 x 192
        x = F.max_pool2d(x, kernel_size=3, stride=2)
        # 35 x 35 x 192
        x = self.Mixed_5b(x)
        # 35 x 35 x 256
        x = self.Mixed_5c(x)
        # 35 x 35 x 288
        x = self.Mixed_5d(x)
        # 35 x 35 x 288
        x = self.Mixed_6a(x)
        # 17 x 17 x 768
        x = self.Mixed_6b(x)
        # 17 x 17 x 768
        x = self.Mixed_6c(x)
        # 17 x 17 x 768
        x = self.Mixed_6d(x)
        # 17 x 17 x 768
        x = self.Mixed_6e(x)
        # 17 x 17 x 768
        if self.training and self.aux_logits:
            aux = self.AuxLogits(x)
        # 17 x 17 x 768
        x = self.Mixed_7a(x)
        # 8 x 8 x 1280
        x = self.Mixed_7b(x)
        # 8 x 8 x 2048
        x = self.Mixed_7c(x)
        # 8 x 8 x 2048
        x = F.adaptive_avg_pool2d(x, 1)
        #x = F.avg_pool2d(x, kernel_size=8)
        # 1 x 1 x 2048
        x = F.dropout(x, training=self.training)
        # 1 x 1 x 2048
        x = x.view(x.size(0), -1)
        # 2048
        x = self.fc(x)
        # 1000 (num_classes)
        if self.training and self.aux_logits:
            return x, aux
        return x


class InceptionA(nn.Module):

    def __init__(self, in_channels, pool_features):
        super(InceptionA, self).__init__()
        self.branch1x1 = BasicConv2d(in_channels, 64, kernel_size=1)

        self.branch5x5_1 = BasicConv2d(in_channels, 48, kernel_size=1)
        self.branch5x5_2 = BasicConv2d(48, 64, kernel_size=5, padding=2)

        self.branch3x3dbl_1 = BasicConv2d(in_channels, 64, kernel_size=1)
        self.branch3x3dbl_2 = BasicConv2d(64, 96, kernel_size=3, padding=1)
        self.branch3x3dbl_3 = BasicConv2d(96, 96, kernel_size=3, padding=1)

        self.branch_pool = BasicConv2d(in_channels, pool_features, kernel_size=1)

    def forward(self, x):
        branch1x1 = self.branch1x1(x)

        branch5x5 = self.branch5x5_1(x)
        branch5x5 = self.branch5x5_2(branch5x5)

        branch3x3dbl = self.branch3x3dbl_1(x)
        branch3x3dbl = self.branch3x3dbl_2(branch3x3dbl)
        branch3x3dbl = self.branch3x3dbl_3(branch3x3dbl)

        branch_pool = F.avg_pool2d(x, kernel_size=3, stride=1, padding=1)
        branch_pool = self.branch_pool(branch_pool)

        outputs = [branch1x1, branch5x5, branch3x3dbl, branch_pool]
        return torch.cat(outputs, 1)


class InceptionB(nn.Module):

    def __init__(self, in_channels):
        super(InceptionB, self).__init__()
        self.branch3x3 = BasicConv2d(in_channels, 384, kernel_size=3, stride=2)

        self.branch3x3dbl_1 = BasicConv2d(in_channels, 64, kernel_size=1)
        self.branch3x3dbl_2 = BasicConv2d(64, 96, kernel_size=3, padding=1)
        self.branch3x3dbl_3 = BasicConv2d(96, 96, kernel_size=3, stride=2)

    def forward(self, x):
        branch3x3 = self.branch3x3(x)

        branch3x3dbl = self.branch3x3dbl_1(x)
        branch3x3dbl = self.branch3x3dbl_2(branch3x3dbl)
        branch3x3dbl = self.branch3x3dbl_3(branch3x3dbl)

        branch_pool = F.max_pool2d(x, kernel_size=3, stride=2)

        outputs = [branch3x3, branch3x3dbl, branch_pool]
        return torch.cat(outputs, 1)


class InceptionC(nn.Module):

    def __init__(self, in_channels, channels_7x7):
        super(InceptionC, self).__init__()
        self.branch1x1 = BasicConv2d(in_channels, 192, kernel_size=1)

        c7 = channels_7x7
        self.branch7x7_1 = BasicConv2d(in_channels, c7, kernel_size=1)
        self.branch7x7_2 = BasicConv2d(c7, c7, kernel_size=(1, 7), padding=(0, 3))
        self.branch7x7_3 = BasicConv2d(c7, 192, kernel_size=(7, 1), padding=(3, 0))

        self.branch7x7dbl_1 = BasicConv2d(in_channels, c7, kernel_size=1)
        self.branch7x7dbl_2 = BasicConv2d(c7, c7, kernel_size=(7, 1), padding=(3, 0))
        self.branch7x7dbl_3 = BasicConv2d(c7, c7, kernel_size=(1, 7), padding=(0, 3))
        self.branch7x7dbl_4 = BasicConv2d(c7, c7, kernel_size=(7, 1), padding=(3, 0))
        self.branch7x7dbl_5 = BasicConv2d(c7, 192, kernel_size=(1, 7), padding=(0, 3))

        self.branch_pool = BasicConv2d(in_channels, 192, kernel_size=1)

    def forward(self, x):
        branch1x1 = self.branch1x1(x)

        branch7x7 = self.branch7x7_1(x)
        branch7x7 = self.branch7x7_2(branch7x7)
        branch7x7 = self.branch7x7_3(branch7x7)

        branch7x7dbl = self.branch7x7dbl_1(x)
        branch7x7dbl = self.branch7x7dbl_2(branch7x7dbl)
        branch7x7dbl = self.branch7x7dbl_3(branch7x7dbl)
        branch7x7dbl = self.branch7x7dbl_4(branch7x7dbl)
        branch7x7dbl = self.branch7x7dbl_5(branch7x7dbl)

        branch_pool = F.avg_pool2d(x, kernel_size=3, stride=1, padding=1)
        branch_pool = self.branch_pool(branch_pool)

        outputs = [branch1x1, branch7x7, branch7x7dbl, branch_pool]
        return torch.cat(outputs, 1)


class InceptionD(nn.Module):

    def __init__(self, in_channels):
        super(InceptionD, self).__init__()
        self.branch3x3_1 = BasicConv2d(in_channels, 192, kernel_size=1)
        self.branch3x3_2 = BasicConv2d(192, 320, kernel_size=3, stride=2)

        self.branch7x7x3_1 = BasicConv2d(in_channels, 192, kernel_size=1)
        self.branch7x7x3_2 = BasicConv2d(192, 192, kernel_size=(1, 7), padding=(0, 3))
        self.branch7x7x3_3 = BasicConv2d(192, 192, kernel_size=(7, 1), padding=(3, 0))
        self.branch7x7x3_4 = BasicConv2d(192, 192, kernel_size=3, stride=2)

    def forward(self, x):
        branch3x3 = self.branch3x3_1(x)
        branch3x3 = self.branch3x3_2(branch3x3)

        branch7x7x3 = self.branch7x7x3_1(x)
        branch7x7x3 = self.branch7x7x3_2(branch7x7x3)
        branch7x7x3 = self.branch7x7x3_3(branch7x7x3)
        branch7x7x3 = self.branch7x7x3_4(branch7x7x3)

        branch_pool = F.max_pool2d(x, kernel_size=3, stride=2)
        outputs = [branch3x3, branch7x7x3, branch_pool]
        return torch.cat(outputs, 1)


class InceptionE(nn.Module):

    def __init__(self, in_channels):
        super(InceptionE, self).__init__()
        self.branch1x1 = BasicConv2d(in_channels, 320, kernel_size=1)

        self.branch3x3_1 = BasicConv2d(in_channels, 384, kernel_size=1)
        self.branch3x3_2a = BasicConv2d(384, 384, kernel_size=(1, 3), padding=(0, 1))
        self.branch3x3_2b = BasicConv2d(384, 384, kernel_size=(3, 1), padding=(1, 0))

        self.branch3x3dbl_1 = BasicConv2d(in_channels, 448, kernel_size=1)
        self.branch3x3dbl_2 = BasicConv2d(448, 384, kernel_size=3, padding=1)
        self.branch3x3dbl_3a = BasicConv2d(384, 384, kernel_size=(1, 3), padding=(0, 1))
        self.branch3x3dbl_3b = BasicConv2d(384, 384, kernel_size=(3, 1), padding=(1, 0))

        self.branch_pool = BasicConv2d(in_channels, 192, kernel_size=1)

    def forward(self, x):
        branch1x1 = self.branch1x1(x)

        branch3x3 = self.branch3x3_1(x)
        branch3x3 = [
            self.branch3x3_2a(branch3x3),
            self.branch3x3_2b(branch3x3),
        ]
        branch3x3 = torch.cat(branch3x3, 1)

        branch3x3dbl = self.branch3x3dbl_1(x)
        branch3x3dbl = self.branch3x3dbl_2(branch3x3dbl)
        branch3x3dbl = [
            self.branch3x3dbl_3a(branch3x3dbl),
            self.branch3x3dbl_3b(branch3x3dbl),
        ]
        branch3x3dbl = torch.cat(branch3x3dbl, 1)

        branch_pool = F.avg_pool2d(x, kernel_size=3, stride=1, padding=1)
        branch_pool = self.branch_pool(branch_pool)

        outputs = [branch1x1, branch3x3, branch3x3dbl, branch_pool]
        return torch.cat(outputs, 1)


class InceptionAux(nn.Module):

    def __init__(self, in_channels, num_classes):
        super(InceptionAux, self).__init__()
        self.conv0 = BasicConv2d(in_channels, 128, kernel_size=1)
        self.conv1 = BasicConv2d(128, 768, kernel_size=5)
        self.conv1.stddev = 0.01
        self.fc = nn.Linear(768, num_classes)
        self.fc.stddev = 0.001

    def forward(self, x):
        # 17 x 17 x 768
        x = F.avg_pool2d(x, kernel_size=5, stride=3)
        # 5 x 5 x 768
        x = self.conv0(x)
        # 5 x 5 x 128
        x = self.conv1(x)
        # 1 x 1 x 768
        x = x.view(x.size(0), -1)
        # 768
        x = self.fc(x)
        # 1000
        return x


class BasicConv2d(nn.Module):

    def __init__(self, in_channels, out_channels, **kwargs):
        super(BasicConv2d, self).__init__()
        self.conv = nn.Conv2d(in_channels, out_channels, bias=False, **kwargs)
        self.bn = nn.BatchNorm2d(out_channels, eps=0.001)

    def forward(self, x):
        x = self.conv(x)
        x = self.bn(x)
        return F.relu(x, inplace=True)
In [5]:
# The following was copied from https://github.com/macaodha/inat_comp_2018/blob/master/train_inat.py

# Adapted from https://github.com/pytorch/examples/blob/master/imagenet/main.py

best_prec3 = 0.0  # store current best top 3

def train(train_loader, model, criterion, optimizer, epoch):
    batch_time = AverageMeter()
    data_time = AverageMeter()
    losses = AverageMeter()
    top1 = AverageMeter()
    top3 = AverageMeter()

    # switch to train mode
    model.train()

    end = time.time()
    print('Epoch:{0}'.format(epoch))
    print('Itr\t\tTime\t\tData\t\tPrec@1\t\tPrec@3')
    for i, (input, im_id, target, tax_ids) in enumerate(train_loader):
        # measure data loading time
        data_time.update(time.time() - end)

        input = input.cuda()
        target = target.cuda(async=True)
        input_var = torch.autograd.Variable(input)
        target_var = torch.autograd.Variable(target)

        # compute output
        output = model(input_var)
        loss = criterion(output, target_var)

        # measure accuracy and record loss
        prec1, prec3 = accuracy(output.data, target, topk=(1, 3))
        # losses.update(loss.data[0], input.size(0))
        top1.update(prec1[0], input.size(0))
        top3.update(prec3[0], input.size(0))

        # compute gradient and do SGD step
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

        # measure elapsed time
        batch_time.update(time.time() - end)
        end = time.time()

        if i % args.print_freq == 0:
            print('[{0}/{1}]\t'
                '{batch_time.val:.2f} ({batch_time.avg:.2f})\t'
                '{data_time.val:.2f} ({data_time.avg:.2f})\t'
                '{top1.val:.2f} ({top1.avg:.2f})\t'
                '{top3.val:.2f} ({top3.avg:.2f})'.format(
                i, len(train_loader), batch_time=batch_time,
                data_time=data_time, top1=top1, top3=top3))


def validate(val_loader, model, criterion, save_preds=False):
    batch_time = AverageMeter()
    losses = AverageMeter()
    top1 = AverageMeter()
    top3 = AverageMeter()

    # switch to evaluate mode
    model.eval()

    end = time.time()
    pred = []
    im_ids = []

    print('Validate:\tTime\t\tPrec@1\t\tPrec@3')
    for i, (input, im_id, target, tax_ids) in enumerate(val_loader):
        input = input.cuda()
        target = target.cuda(async=True)
        input_var = torch.autograd.Variable(input, volatile=True)
        target_var = torch.autograd.Variable(target, volatile=True)

        # compute output
        output = model(input_var)
        # loss = criterion(output, target_var)

        if save_preds:
            # store the top K classes for the prediction
            im_ids.append(im_id.cpu().numpy().astype(np.int))
            _, pred_inds = output.data.topk(5,1,True,True)
            pred.append(pred_inds.cpu().numpy().astype(np.int))

        # measure accuracy and record loss
        prec1, prec3 = accuracy(output.data, target, topk=(1, 3))
        # losses.update(loss.data[0], input.size(0))

        top1.update(prec1[0], input.size(0))
        top3.update(prec3[0], input.size(0))

        # measure elapsed time
        batch_time.update(time.time() - end)
        end = time.time()
        
        if i % args.print_freq == 0:
            print('[{0}/{1}]\t'
                  '{batch_time.val:.2f} ({batch_time.avg:.2f})\t'
                  '{top1.val:.2f} ({top1.avg:.2f})\t'
                  '{top3.val:.2f} ({top3.avg:.2f})'.format(
                   i, len(val_loader), batch_time=batch_time,
                   top1=top1, top3=top3))

    print(' * Prec@1 {top1.avg:.3f} Prec@3 {top3.avg:.3f}'
          .format(top1=top1, top3=top3))

    if save_preds:
        return top3.avg, np.vstack(pred), np.hstack(im_ids)
    else:
        return top3.avg


def save_checkpoint(state, is_best, filename='checkpoint.pth.tar'):
    torch.save(state, filename)
    if is_best:
        print("\tSaving new best model")
        shutil.copyfile(filename, 'model_best.pth.tar')


class AverageMeter(object):
    """Computes and stores the average and current value"""
    def __init__(self):
        self.reset()

    def reset(self):
        self.val = 0
        self.avg = 0
        self.sum = 0
        self.count = 0

    def update(self, val, n=1):
        self.val = val
        self.sum += val * n
        self.count += n
        self.avg = self.sum / self.count


def adjust_learning_rate(optimizer, epoch):
    """Sets the learning rate to the initial LR decayed by 10 every 30 epochs"""
    lr = args.lr * (0.1 ** (epoch // 30))
    for param_group in optimizer.param_groups:
        param_group['lr'] = lr


def accuracy(output, target, topk=(1,)):
    """Computes the precision@k for the specified values of k"""
    maxk = max(topk)
    batch_size = target.size(0)

    _, pred = output.topk(maxk, 1, True, True)
    pred = pred.t()
    correct = pred.eq(target.view(1, -1).expand_as(pred))

    res = []
    for k in topk:
        correct_k = correct[:k].view(-1).float().sum(0, keepdim=True)
        res.append(correct_k.mul_(100.0 / batch_size))
    return res
In [6]:
# The following was copied from https://github.com/macaodha/inat_comp_2018/blob/master/inat2018_loader.py

def default_loader(path):
    return Image.open(path).convert('RGB')

def load_taxonomy(ann_data, tax_levels, classes):
    # loads the taxonomy data and converts to ints
    taxonomy = {}

    if 'categories' in ann_data.keys():
        num_classes = len(ann_data['categories'])
        for tt in tax_levels:
            tax_data = [aa[tt] for aa in ann_data['categories']]
            _, tax_id = np.unique(tax_data, return_inverse=True)
            taxonomy[tt] = dict(zip(range(num_classes), list(tax_id)))
    else:
        # set up dummy data
        for tt in tax_levels:
            taxonomy[tt] = dict(zip([0], [0]))

    # create a dictionary of lists containing taxonomic labels
    classes_taxonomic = {}
    for cc in np.unique(classes):
        tax_ids = [0]*len(tax_levels)
        for ii, tt in enumerate(tax_levels):
            tax_ids[ii] = taxonomy[tt][cc]
        classes_taxonomic[cc] = tax_ids

    return taxonomy, classes_taxonomic


class INAT(data.Dataset):
    def __init__(self, root, ann_file, is_train=True):

        # load annotations
        print('Loading annotations from: ' + os.path.basename(ann_file))
        with open(ann_file) as data_file:
            ann_data = json.load(data_file)

        # set up the filenames and annotations
        self.imgs = [aa['file_name'] for aa in ann_data['images']]
        self.ids = [aa['id'] for aa in ann_data['images']]

        # if we dont have class labels set them to '0'
        if 'annotations' in ann_data.keys():
            self.classes = [aa['category_id'] for aa in ann_data['annotations']]
        else:
            self.classes = [0]*len(self.imgs)

        # load taxonomy
        self.tax_levels = ['id', 'genus', 'family', 'order', 'class', 'phylum', 'kingdom']
                           #8142, 4412,    1120,     273,     57,      25,       6
        self.taxonomy, self.classes_taxonomic = load_taxonomy(ann_data, self.tax_levels, self.classes)

        # print out some stats
        print('\t' + str(len(self.imgs)) + ' images')
        print('\t' + str(len(set(self.classes))) + ' classes')

        self.root = root
        self.is_train = is_train
        self.loader = default_loader

        # augmentation params
        self.im_size = [299, 299]  # can change this to train on higher res
        self.mu_data = [0.485, 0.456, 0.406]
        self.std_data = [0.229, 0.224, 0.225]
        self.brightness = 0.4
        self.contrast = 0.4
        self.saturation = 0.4
        self.hue = 0.25

        # augmentations
        self.center_crop = transforms.CenterCrop((self.im_size[0], self.im_size[1]))
        self.scale_aug = transforms.RandomResizedCrop(size=self.im_size[0])
        self.flip_aug = transforms.RandomHorizontalFlip()
        self.color_aug = transforms.ColorJitter(self.brightness, self.contrast, self.saturation, self.hue)
        self.tensor_aug = transforms.ToTensor()
        self.norm_aug = transforms.Normalize(mean=self.mu_data, std=self.std_data)

    def __getitem__(self, index):
        path = self.root + self.imgs[index]
        im_id = self.ids[index]
        img = self.loader(path)
        species_id = self.classes[index]
        tax_ids = self.classes_taxonomic[species_id]

        if self.is_train:
            img = self.scale_aug(img)
            img = self.flip_aug(img)
            img = self.color_aug(img)
        else:
            img = self.center_crop(img)

        img = self.tensor_aug(img)
        img = self.norm_aug(img)

        return img, im_id, species_id, tax_ids

    def __len__(self):
        return len(self.imgs)

Step 3: Train and Predict

In [7]:
# The following was copied from https://github.com/macaodha/inat_comp_2018/blob/master/inat2018_loader.py

args = Params()

# load pretrained model
print("Using pre-trained inception_v3")
model = inception_v3(pretrained=True)
model.fc = nn.Linear(2048, args.num_classes)
model.aux_logits = False
model = model.cuda()

# define loss function (criterion) and optimizer
criterion = nn.CrossEntropyLoss().cuda()
optimizer = torch.optim.SGD(model.parameters(), args.lr,
                            momentum=args.momentum,
                            weight_decay=args.weight_decay)

# optionally resume from a checkpoint
if args.resume:
    if os.path.isfile(args.resume):
        print("=> loading checkpoint '{}'".format(args.resume))
        checkpoint = torch.load(args.resume)
        args.start_epoch = checkpoint['epoch']
        best_prec3 = checkpoint['best_prec3']
        model.load_state_dict(checkpoint['state_dict'])
        optimizer.load_state_dict(checkpoint['optimizer'])
        print("=> loaded checkpoint '{}' (epoch {})"
               .format(args.resume, checkpoint['epoch']))
    else:
        print("=> no checkpoint found at '{}'".format(args.resume))

    cudnn.benchmark = True

# data loading code
train_dataset = INAT(args.data_root_train, args.train_file,
                     is_train=True)
val_dataset = INAT(args.data_root_test, args.val_file,
                     is_train=False)

train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=args.batch_size,
                   shuffle=True, num_workers=args.workers, pin_memory=True)
val_loader = torch.utils.data.DataLoader(val_dataset,
                                         batch_size=args.batch_size, shuffle=False,
                                         num_workers=args.workers, pin_memory=True)

for epoch in range(args.start_epoch, args.epochs):
    adjust_learning_rate(optimizer, epoch)

    # train for one epoch
    train(train_loader, model, criterion, optimizer, epoch)

    # evaluate on validation set
    prec3 = validate(val_loader, model, criterion, False)

    # remember best prec@1 and save checkpoint
    is_best = prec3 > best_prec3
    best_prec3 = max(prec3, best_prec3)
    save_checkpoint({
        'epoch': epoch + 1,
        #'arch': args.arch,
        'state_dict': model.state_dict(),
        'best_prec3': best_prec3,
        'optimizer' : optimizer.state_dict(),
    }, is_best)

if args.evaluate:
    prec3, preds, im_ids = validate(val_loader, model, criterion, True)
    # write predictions to file
    if args.save_preds:
        with open(args.op_file_name, 'w') as opfile:
            opfile.write('id,predicted\n')
            for ii in range(len(im_ids)):
                opfile.write(str(im_ids[ii]) + ',' + ' '.join([str(j) for j in preds[ii, :]]) + '\n')
Using pre-trained inception_v3
Downloading: "https://download.pytorch.org/models/inception_v3_google-1a9a5a14.pth" to /tmp/.torch/models/inception_v3_google-1a9a5a14.pth
108857766it [00:07, 14436811.27it/s]
=> no checkpoint found at 'iNat_2018_InceptionV3.pth.tar'
Loading annotations from: train2019.json
	265213 images
	1010 classes
Loading annotations from: test2019.json
	35350 images
	1 classes
Epoch:0
Itr		Time		Data		Prec@1		Prec@3
[0/4144]	21.64 (21.64)	11.95 (11.95)	0.00 (0.00)	0.00 (0.00)
[100/4144]	0.52 (1.60)	0.01 (0.98)	1.56 (0.84)	4.69 (1.75)
[200/4144]	5.92 (1.56)	5.32 (0.98)	0.00 (1.73)	9.38 (4.18)
[300/4144]	0.53 (1.52)	0.00 (0.97)	4.69 (2.52)	9.38 (6.13)
[400/4144]	3.52 (1.51)	3.00 (0.96)	4.69 (3.43)	12.50 (8.10)
[500/4144]	0.51 (1.50)	0.00 (0.96)	7.81 (4.35)	14.06 (9.77)
[600/4144]	0.54 (1.50)	0.01 (0.96)	9.38 (5.13)	17.19 (11.34)
[700/4144]	0.52 (1.50)	0.00 (0.96)	6.25 (5.82)	15.62 (12.86)
[800/4144]	0.51 (1.49)	0.01 (0.96)	10.94 (6.45)	21.88 (14.24)
[900/4144]	0.50 (1.50)	0.00 (0.96)	12.50 (7.19)	26.56 (15.65)
[1000/4144]	0.64 (1.49)	0.00 (0.96)	21.88 (7.87)	31.25 (16.95)
[1100/4144]	0.55 (1.49)	0.00 (0.96)	14.06 (8.54)	34.38 (18.15)
[1200/4144]	0.68 (1.49)	0.13 (0.95)	18.75 (9.12)	31.25 (19.29)
[1300/4144]	0.54 (1.49)	0.00 (0.96)	21.88 (9.75)	39.06 (20.35)
[1400/4144]	0.67 (1.49)	0.15 (0.95)	23.44 (10.34)	39.06 (21.37)
[1500/4144]	0.58 (1.49)	0.00 (0.96)	20.31 (10.87)	32.81 (22.29)
[1600/4144]	0.59 (1.49)	0.00 (0.95)	20.31 (11.42)	39.06 (23.21)
[1700/4144]	0.51 (1.49)	0.00 (0.95)	17.19 (11.94)	39.06 (24.02)
[1800/4144]	0.55 (1.49)	0.01 (0.95)	21.88 (12.46)	42.19 (24.86)
[1900/4144]	0.56 (1.49)	0.00 (0.95)	29.69 (12.96)	46.88 (25.66)
[2000/4144]	0.55 (1.49)	0.01 (0.95)	23.44 (13.39)	42.19 (26.39)
[2100/4144]	0.55 (1.49)	0.00 (0.95)	21.88 (13.85)	46.88 (27.13)
[2200/4144]	0.55 (1.49)	0.00 (0.95)	23.44 (14.30)	40.62 (27.81)
[2300/4144]	0.60 (1.49)	0.00 (0.95)	31.25 (14.72)	46.88 (28.48)
[2400/4144]	0.53 (1.49)	0.01 (0.95)	23.44 (15.19)	40.62 (29.19)
[2500/4144]	0.55 (1.49)	0.01 (0.95)	31.25 (15.58)	51.56 (29.78)
[2600/4144]	1.94 (1.48)	1.41 (0.95)	21.88 (15.97)	35.94 (30.33)
[2700/4144]	0.50 (1.49)	0.00 (0.95)	23.44 (16.33)	43.75 (30.90)
[2800/4144]	4.43 (1.49)	3.91 (0.95)	23.44 (16.71)	48.44 (31.46)
[2900/4144]	0.52 (1.48)	0.00 (0.95)	21.88 (17.05)	40.62 (32.00)
[3000/4144]	5.72 (1.48)	5.20 (0.95)	28.12 (17.39)	45.31 (32.51)
[3100/4144]	0.50 (1.48)	0.01 (0.95)	25.00 (17.72)	48.44 (32.99)
[3200/4144]	7.22 (1.48)	6.70 (0.95)	28.12 (18.06)	50.00 (33.48)
[3300/4144]	0.50 (1.48)	0.00 (0.95)	25.00 (18.41)	35.94 (33.96)
[3400/4144]	5.39 (1.48)	4.87 (0.95)	29.69 (18.72)	57.81 (34.43)
[3500/4144]	0.52 (1.48)	0.00 (0.95)	31.25 (19.04)	48.44 (34.88)
[3600/4144]	0.54 (1.48)	0.00 (0.95)	28.12 (19.36)	39.06 (35.33)
[3700/4144]	0.53 (1.48)	0.00 (0.95)	21.88 (19.66)	43.75 (35.77)
[3800/4144]	0.51 (1.48)	0.00 (0.95)	26.56 (19.96)	46.88 (36.17)
[3900/4144]	0.50 (1.48)	0.00 (0.95)	34.38 (20.27)	59.38 (36.59)
[4000/4144]	1.16 (1.48)	0.63 (0.95)	39.06 (20.56)	60.94 (36.98)
[4100/4144]	0.57 (1.48)	0.01 (0.95)	23.44 (20.84)	53.12 (37.37)
Validate:	Time		Prec@1		Prec@3
/opt/conda/lib/python3.6/site-packages/ipykernel_launcher.py:75: UserWarning: volatile was removed and now has no effect. Use `with torch.no_grad():` instead.
/opt/conda/lib/python3.6/site-packages/ipykernel_launcher.py:76: UserWarning: volatile was removed and now has no effect. Use `with torch.no_grad():` instead.
[0/553]	6.85 (6.85)	0.00 (0.00)	0.00 (0.00)
[100/553]	0.15 (0.87)	0.00 (0.00)	0.00 (0.03)
[200/553]	5.54 (0.86)	0.00 (0.00)	0.00 (0.03)
[300/553]	0.16 (0.84)	0.00 (0.00)	0.00 (0.02)
[400/553]	5.58 (0.84)	0.00 (0.00)	0.00 (0.02)
[500/553]	0.13 (0.83)	0.00 (0.00)	0.00 (0.02)
 * Prec@1 0.000 Prec@3 0.017
	Saving new best model
Epoch:1
Itr		Time		Data		Prec@1		Prec@3
[0/4144]	12.66 (12.66)	12.00 (12.00)	29.69 (29.69)	46.88 (46.88)
[100/4144]	0.53 (1.58)	0.00 (1.03)	34.38 (33.35)	65.62 (55.31)
[200/4144]	7.83 (1.55)	7.19 (1.01)	32.81 (33.90)	56.25 (55.74)
[300/4144]	0.52 (1.52)	0.01 (0.98)	32.81 (33.71)	59.38 (55.32)
[400/4144]	7.92 (1.52)	7.40 (0.99)	28.12 (33.70)	48.44 (55.16)
[500/4144]	0.54 (1.51)	0.00 (0.98)	34.38 (33.68)	53.12 (55.06)
[600/4144]	4.16 (1.51)	3.60 (0.98)	35.94 (33.77)	57.81 (55.10)
[700/4144]	0.51 (1.50)	0.00 (0.96)	35.94 (33.75)	50.00 (55.17)
[800/4144]	0.52 (1.50)	0.00 (0.97)	26.56 (33.94)	51.56 (55.28)
[900/4144]	0.50 (1.49)	0.00 (0.96)	35.94 (34.15)	48.44 (55.52)
[1000/4144]	1.33 (1.50)	0.73 (0.96)	42.19 (34.26)	53.12 (55.63)
[1100/4144]	0.58 (1.49)	0.00 (0.96)	34.38 (34.40)	56.25 (55.80)
[1200/4144]	0.54 (1.49)	0.01 (0.95)	32.81 (34.51)	54.69 (55.86)
[1300/4144]	0.54 (1.49)	0.01 (0.95)	43.75 (34.63)	54.69 (56.03)
[1400/4144]	0.52 (1.49)	0.00 (0.95)	40.62 (34.72)	57.81 (56.14)
[1500/4144]	0.52 (1.48)	0.01 (0.95)	25.00 (34.87)	51.56 (56.27)
[1600/4144]	0.52 (1.49)	0.01 (0.95)	34.38 (34.93)	48.44 (56.33)
[1700/4144]	0.59 (1.49)	0.00 (0.96)	31.25 (34.99)	54.69 (56.47)
[1800/4144]	0.47 (1.50)	0.00 (0.96)	45.31 (35.08)	60.94 (56.51)
[1900/4144]	0.50 (1.50)	0.00 (0.96)	32.81 (35.19)	56.25 (56.61)
[2000/4144]	0.48 (1.50)	0.00 (0.96)	39.06 (35.28)	60.94 (56.75)
[2100/4144]	0.51 (1.50)	0.01 (0.96)	37.50 (35.34)	57.81 (56.80)
[2200/4144]	0.54 (1.50)	0.00 (0.96)	35.94 (35.48)	54.69 (56.93)
[2300/4144]	0.55 (1.49)	0.00 (0.96)	35.94 (35.55)	62.50 (57.03)
[2400/4144]	0.63 (1.49)	0.00 (0.96)	40.62 (35.66)	65.62 (57.15)
[2500/4144]	0.52 (1.49)	0.01 (0.95)	34.38 (35.74)	59.38 (57.26)
[2600/4144]	0.79 (1.49)	0.29 (0.96)	31.25 (35.80)	48.44 (57.35)
[2700/4144]	0.47 (1.49)	0.00 (0.95)	43.75 (35.88)	59.38 (57.47)
[2800/4144]	0.58 (1.49)	0.00 (0.95)	42.19 (35.97)	54.69 (57.56)
[2900/4144]	0.52 (1.48)	0.00 (0.95)	40.62 (36.07)	57.81 (57.68)
[3000/4144]	0.51 (1.48)	0.00 (0.95)	32.81 (36.15)	51.56 (57.77)
[3100/4144]	0.54 (1.48)	0.00 (0.95)	39.06 (36.28)	70.31 (57.89)
[3200/4144]	0.52 (1.48)	0.00 (0.95)	35.94 (36.35)	56.25 (57.97)
[3300/4144]	0.51 (1.47)	0.01 (0.94)	31.25 (36.45)	56.25 (58.06)
[3400/4144]	1.14 (1.47)	0.68 (0.94)	46.88 (36.57)	70.31 (58.16)
[3500/4144]	0.49 (1.47)	0.00 (0.94)	42.19 (36.65)	65.62 (58.22)
[3600/4144]	0.69 (1.47)	0.20 (0.94)	35.94 (36.78)	54.69 (58.34)
[3700/4144]	0.51 (1.46)	0.00 (0.94)	35.94 (36.89)	51.56 (58.46)
[3800/4144]	6.62 (1.46)	6.11 (0.94)	28.12 (36.97)	56.25 (58.53)
[3900/4144]	0.57 (1.46)	0.00 (0.93)	45.31 (37.06)	70.31 (58.61)
[4000/4144]	7.73 (1.46)	7.21 (0.93)	45.31 (37.12)	68.75 (58.70)
[4100/4144]	0.60 (1.46)	0.00 (0.93)	39.06 (37.22)	68.75 (58.79)
Validate:	Time		Prec@1		Prec@3
[0/553]	7.50 (7.50)	0.00 (0.00)	0.00 (0.00)
[100/553]	0.15 (0.85)	0.00 (0.03)	0.00 (0.14)
[200/553]	3.97 (0.83)	0.00 (0.02)	0.00 (0.12)
[300/553]	0.16 (0.81)	0.00 (0.03)	0.00 (0.13)
[400/553]	3.35 (0.81)	0.00 (0.02)	0.00 (0.13)
[500/553]	0.18 (0.80)	0.00 (0.02)	0.00 (0.13)
 * Prec@1 0.025 Prec@3 0.144
	Saving new best model
Epoch:2
Itr		Time		Data		Prec@1		Prec@3
[0/4144]	13.66 (13.66)	13.02 (13.02)	45.31 (45.31)	60.94 (60.94)
[100/4144]	0.52 (1.46)	0.01 (0.93)	46.88 (42.87)	71.88 (64.36)
[200/4144]	4.43 (1.43)	3.83 (0.91)	48.44 (42.70)	60.94 (63.90)
[300/4144]	0.47 (1.41)	0.00 (0.88)	40.62 (42.37)	67.19 (63.86)
[400/4144]	3.01 (1.40)	2.51 (0.87)	51.56 (42.20)	67.19 (63.71)
[500/4144]	0.65 (1.42)	0.00 (0.90)	50.00 (42.40)	71.88 (63.86)
[600/4144]	0.52 (1.43)	0.00 (0.90)	48.44 (42.53)	71.88 (63.98)
[700/4144]	0.86 (1.44)	0.35 (0.91)	40.62 (42.59)	59.38 (63.98)
[800/4144]	1.60 (1.44)	1.08 (0.91)	28.12 (42.62)	56.25 (63.97)
[900/4144]	0.51 (1.45)	0.01 (0.92)	48.44 (42.63)	64.06 (64.05)
[1000/4144]	1.54 (1.46)	0.96 (0.93)	37.50 (42.67)	48.44 (64.07)
[1100/4144]	1.36 (1.47)	0.85 (0.93)	45.31 (42.81)	64.06 (64.17)
[1200/4144]	0.55 (1.47)	0.00 (0.94)	42.19 (42.88)	68.75 (64.21)
[1300/4144]	2.42 (1.48)	1.80 (0.94)	31.25 (42.92)	56.25 (64.32)
[1400/4144]	0.51 (1.48)	0.01 (0.94)	42.19 (42.96)	62.50 (64.36)
[1500/4144]	3.27 (1.48)	2.67 (0.94)	35.94 (43.00)	64.06 (64.40)
[1600/4144]	0.53 (1.48)	0.00 (0.95)	29.69 (43.02)	50.00 (64.41)
[1700/4144]	4.65 (1.48)	4.06 (0.95)	54.69 (43.09)	71.88 (64.49)
[1800/4144]	0.50 (1.49)	0.00 (0.95)	35.94 (43.11)	59.38 (64.48)
[1900/4144]	4.12 (1.49)	3.50 (0.95)	43.75 (43.09)	57.81 (64.49)
[2000/4144]	0.82 (1.48)	0.33 (0.95)	51.56 (43.13)	59.38 (64.53)
[2100/4144]	6.73 (1.49)	6.27 (0.95)	37.50 (43.19)	57.81 (64.62)
[2200/4144]	0.50 (1.48)	0.00 (0.95)	39.06 (43.25)	64.06 (64.63)
[2300/4144]	6.02 (1.48)	5.41 (0.95)	43.75 (43.32)	68.75 (64.68)
[2400/4144]	0.52 (1.48)	0.00 (0.94)	51.56 (43.36)	65.62 (64.70)
[2500/4144]	6.85 (1.48)	6.34 (0.94)	31.25 (43.42)	67.19 (64.81)
[2600/4144]	0.50 (1.47)	0.01 (0.94)	40.62 (43.45)	56.25 (64.83)
[2700/4144]	7.74 (1.47)	7.15 (0.94)	50.00 (43.51)	79.69 (64.89)
[2800/4144]	0.46 (1.47)	0.00 (0.93)	45.31 (43.53)	68.75 (64.91)
[2900/4144]	6.95 (1.47)	6.36 (0.93)	56.25 (43.54)	82.81 (64.94)
[3000/4144]	0.50 (1.46)	0.00 (0.93)	37.50 (43.60)	57.81 (64.99)
[3100/4144]	7.09 (1.46)	6.57 (0.93)	43.75 (43.62)	73.44 (65.02)
[3200/4144]	0.50 (1.46)	0.00 (0.92)	45.31 (43.66)	65.62 (65.04)
[3300/4144]	7.73 (1.45)	7.19 (0.92)	43.75 (43.75)	54.69 (65.09)
[3400/4144]	0.50 (1.45)	0.00 (0.92)	50.00 (43.76)	70.31 (65.10)
[3500/4144]	6.98 (1.45)	6.51 (0.92)	32.81 (43.81)	65.62 (65.15)
[3600/4144]	1.14 (1.45)	0.61 (0.92)	32.81 (43.83)	56.25 (65.18)
[3700/4144]	8.40 (1.45)	7.88 (0.92)	46.88 (43.88)	71.88 (65.24)
[3800/4144]	0.63 (1.45)	0.12 (0.92)	48.44 (43.92)	67.19 (65.25)
[3900/4144]	8.18 (1.45)	7.67 (0.92)	40.62 (43.97)	71.88 (65.34)
[4000/4144]	0.54 (1.45)	0.00 (0.92)	54.69 (44.01)	79.69 (65.37)
[4100/4144]	6.83 (1.46)	6.23 (0.93)	45.31 (44.05)	71.88 (65.41)
Validate:	Time		Prec@1		Prec@3
[0/553]	7.20 (7.20)	0.00 (0.00)	0.00 (0.00)
[100/553]	0.13 (0.90)	0.00 (0.11)	0.00 (0.22)
[200/553]	2.20 (0.87)	0.00 (0.09)	0.00 (0.23)
[300/553]	0.15 (0.86)	0.00 (0.08)	0.00 (0.20)
[400/553]	4.03 (0.86)	0.00 (0.07)	0.00 (0.19)
[500/553]	0.15 (0.86)	0.00 (0.07)	0.00 (0.18)
 * Prec@1 0.071 Prec@3 0.190
	Saving new best model
Epoch:3
Itr		Time		Data		Prec@1		Prec@3
[0/4144]	15.26 (15.26)	14.53 (14.53)	45.31 (45.31)	67.19 (67.19)
[100/4144]	0.51 (1.63)	0.00 (1.09)	42.19 (46.35)	59.38 (67.50)
[200/4144]	7.78 (1.60)	7.13 (1.06)	59.38 (46.74)	76.56 (67.81)
[300/4144]	0.53 (1.57)	0.00 (1.04)	45.31 (46.85)	67.19 (67.97)
[400/4144]	7.26 (1.57)	6.70 (1.04)	50.00 (46.91)	54.69 (68.17)
[500/4144]	0.58 (1.56)	0.00 (1.03)	45.31 (46.77)	70.31 (68.10)
[600/4144]	3.95 (1.56)	3.30 (1.02)	53.12 (46.95)	73.44 (68.22)
[700/4144]	0.51 (1.55)	0.00 (1.02)	43.75 (47.09)	68.75 (68.36)
[800/4144]	5.70 (1.55)	5.03 (1.02)	60.94 (47.01)	79.69 (68.39)
[900/4144]	0.52 (1.55)	0.00 (1.02)	57.81 (47.00)	71.88 (68.34)
[1000/4144]	8.32 (1.55)	7.66 (1.02)	42.19 (47.19)	57.81 (68.46)
[1100/4144]	0.54 (1.55)	0.00 (1.01)	51.56 (47.20)	64.06 (68.43)
[1200/4144]	9.32 (1.55)	8.73 (1.02)	51.56 (47.16)	73.44 (68.33)
[1300/4144]	0.56 (1.55)	0.00 (1.02)	56.25 (47.19)	85.94 (68.34)
[1400/4144]	6.88 (1.55)	6.29 (1.02)	54.69 (47.26)	68.75 (68.37)
[1500/4144]	1.79 (1.55)	1.28 (1.02)	54.69 (47.30)	68.75 (68.44)
[1600/4144]	6.66 (1.55)	6.05 (1.02)	56.25 (47.35)	65.62 (68.47)
[1700/4144]	0.55 (1.55)	0.01 (1.01)	54.69 (47.38)	70.31 (68.54)
[1800/4144]	7.53 (1.55)	7.00 (1.01)	50.00 (47.42)	75.00 (68.60)
[1900/4144]	0.52 (1.55)	0.01 (1.01)	43.75 (47.47)	60.94 (68.59)
[2000/4144]	8.83 (1.55)	8.21 (1.02)	50.00 (47.51)	71.88 (68.64)
[2100/4144]	0.48 (1.55)	0.00 (1.01)	46.88 (47.52)	71.88 (68.68)
[2200/4144]	9.08 (1.55)	8.47 (1.01)	48.44 (47.54)	60.94 (68.70)
[2300/4144]	0.52 (1.55)	0.01 (1.01)	40.62 (47.60)	65.62 (68.70)
[2400/4144]	9.24 (1.55)	8.71 (1.01)	39.06 (47.63)	67.19 (68.70)
[2500/4144]	0.48 (1.55)	0.00 (1.01)	34.38 (47.68)	59.38 (68.72)
[2600/4144]	8.75 (1.55)	8.22 (1.01)	50.00 (47.77)	71.88 (68.77)
[2700/4144]	0.53 (1.55)	0.01 (1.01)	46.88 (47.80)	70.31 (68.79)
[2800/4144]	8.14 (1.55)	7.52 (1.01)	48.44 (47.84)	68.75 (68.84)
[2900/4144]	0.51 (1.55)	0.01 (1.01)	51.56 (47.87)	67.19 (68.85)
[3000/4144]	8.26 (1.55)	7.70 (1.01)	42.19 (47.89)	64.06 (68.87)
[3100/4144]	0.57 (1.55)	0.00 (1.01)	57.81 (47.89)	79.69 (68.88)
[3200/4144]	3.08 (1.55)	2.58 (1.01)	43.75 (47.91)	65.62 (68.89)
[3300/4144]	0.51 (1.55)	0.01 (1.01)	40.62 (47.92)	67.19 (68.90)
[3400/4144]	6.10 (1.55)	5.61 (1.01)	39.06 (47.91)	57.81 (68.88)
[3500/4144]	0.52 (1.55)	0.00 (1.01)	64.06 (47.93)	81.25 (68.90)
[3600/4144]	6.01 (1.55)	5.48 (1.01)	53.12 (47.93)	71.88 (68.91)
[3700/4144]	0.58 (1.55)	0.00 (1.01)	53.12 (47.96)	71.88 (68.91)
[3800/4144]	5.78 (1.55)	5.27 (1.01)	48.44 (47.98)	62.50 (68.92)
[3900/4144]	0.53 (1.55)	0.00 (1.01)	43.75 (47.99)	68.75 (68.95)
[4000/4144]	0.95 (1.54)	0.36 (1.01)	57.81 (48.02)	76.56 (68.99)
[4100/4144]	0.53 (1.54)	0.01 (1.01)	53.12 (48.03)	75.00 (69.00)
Validate:	Time		Prec@1		Prec@3
[0/553]	7.53 (7.53)	0.00 (0.00)	0.00 (0.00)
[100/553]	0.14 (0.89)	0.00 (0.06)	0.00 (0.22)
[200/553]	4.23 (0.87)	0.00 (0.07)	0.00 (0.19)
[300/553]	0.15 (0.85)	0.00 (0.06)	0.00 (0.17)
[400/553]	5.84 (0.86)	0.00 (0.06)	0.00 (0.16)
[500/553]	0.15 (0.85)	0.00 (0.06)	0.00 (0.17)
 * Prec@1 0.059 Prec@3 0.173
Validate:	Time		Prec@1		Prec@3
[0/553]	7.79 (7.79)	0.00 (0.00)	0.00 (0.00)
[100/553]	0.15 (0.89)	0.00 (0.06)	0.00 (0.22)
[200/553]	1.85 (0.85)	0.00 (0.07)	0.00 (0.19)
[300/553]	0.17 (0.84)	0.00 (0.06)	0.00 (0.17)
[400/553]	2.78 (0.84)	0.00 (0.06)	0.00 (0.16)
[500/553]	0.18 (0.83)	0.00 (0.06)	0.00 (0.17)
 * Prec@1 0.059 Prec@3 0.173