Keras Data Augmentation & Visualize

From: https://www.kaggle.com/hsinwenchang/keras-data-augmentation-visualize

Author: Beans

Score: 0.96392

This kernel is base on Alexander Teplyuk here I applied Data Augmentation technic from Udacity as following:

  • random 45 degree rotation
  • random zoom of up to 50%
  • random horizontal flip
  • width shift of 0.15
  • height shfit of 0.15
  • Model: vgg16
    Apply transfer learning skill from pretrained model using Keras. Using vgg16 with a flatten layer followed by 2 fully connected layer with 1024 units and 1010 units. The output class probabilities based on 1010 classes which is done by the softmax activation function. Add a layer use a relu activation function. Add Dropout layers with a probability of 50%, where appropriate.
In [1]:
import os
import cv2
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
import json
from keras.models import Sequential
from keras.layers import Dense, Flatten, Activation, Dropout, Conv2D,MaxPooling2D
from keras.optimizers import Adam
from keras.preprocessing.image import ImageDataGenerator
from keras.applications import VGG16
from keras.applications import ResNet50
from keras import optimizers
Using TensorFlow backend.

Train data

In [2]:
ann_file = '../input/train2019.json'
with open(ann_file) as data_file:
        train_anns = json.load(data_file)
In [3]:
train_anns_df = pd.DataFrame(train_anns['annotations'])[['image_id','category_id']]
train_img_df = pd.DataFrame(train_anns['images'])[['id', 'file_name']].rename(columns={'id':'image_id'})
df_train_file_cat = pd.merge(train_img_df, train_anns_df, on='image_id')
df_train_file_cat['category_id']=df_train_file_cat['category_id'].astype(str)
df_train_file_cat.head()
Out[3]:
image_id file_name category_id
0 0 train_val2019/Plants/400/d1322d13ccd856eb4236c... 400
1 1 train_val2019/Plants/570/15edbc1e2ef000d8ace48... 570
2 2 train_val2019/Reptiles/167/c87a32e8927cbf4f06d... 167
3 3 train_val2019/Birds/254/9fcdd1d37e96d8fd94dfdc... 254
4 4 train_val2019/Plants/739/ffa06f951e99de9d220ae... 739
In [4]:
len(df_train_file_cat['category_id'].unique())
Out[4]:
1010
In [5]:
# Example of images for category_id = 400
img_names = df_train_file_cat[df_train_file_cat['category_id']=='400']['file_name'][:30]

plt.figure(figsize=[15,15])
i = 1
for img_name in img_names:
    img = cv2.imread("../input/train_val2019/%s" % img_name)[...,[2, 1, 0]]
    plt.subplot(6, 5, i)
    plt.imshow(img)
    i += 1
plt.show()

Validation data

In [6]:
valid_ann_file = '../input/val2019.json'
with open(valid_ann_file) as data_file:
        valid_anns = json.load(data_file)
In [7]:
valid_anns_df = pd.DataFrame(valid_anns['annotations'])[['image_id','category_id']]
valid_anns_df.head()
Out[7]:
image_id category_id
0 265213 644
1 265214 597
2 265215 883
3 265216 300
4 265217 881
In [8]:
valid_img_df = pd.DataFrame(valid_anns['images'])[['id', 'file_name']].rename(columns={'id':'image_id'})
valid_img_df.head()
Out[8]:
image_id file_name
0 265213 train_val2019/Plants/644/716a69838526f3ada3b2f...
1 265214 train_val2019/Plants/597/0942cc64d2e759c5ee059...
2 265215 train_val2019/Plants/883/acfdbfd9fa675f1c84558...
3 265216 train_val2019/Birds/300/5f3194ff536c7dd31d80b7...
4 265217 train_val2019/Plants/881/76acaf0b2841f91982d21...
In [9]:
df_valid_file_cat = pd.merge(valid_img_df, valid_anns_df, on='image_id')
df_valid_file_cat['category_id']=df_valid_file_cat['category_id'].astype(str)
df_valid_file_cat.head()
Out[9]:
image_id file_name category_id
0 265213 train_val2019/Plants/644/716a69838526f3ada3b2f... 644
1 265214 train_val2019/Plants/597/0942cc64d2e759c5ee059... 597
2 265215 train_val2019/Plants/883/acfdbfd9fa675f1c84558... 883
3 265216 train_val2019/Birds/300/5f3194ff536c7dd31d80b7... 300
4 265217 train_val2019/Plants/881/76acaf0b2841f91982d21... 881
In [10]:
nb_classes = 1010
batch_size = 128
img_size = 128
nb_epochs = 10

In the cell below, use ImageDataGenerator to create a transformation that rescales the images by 255 here I applied Data Augmentation as following:

  • random 45 degree rotation
  • random zoom of up to 50%
  • random horizontal flip
  • width shift of 0.15
  • height shfit of 0.15
In [11]:
%%time
train_datagen=ImageDataGenerator(rescale=1./255, rotation_range=45, 
                    width_shift_range=.15, 
                    height_shift_range=.15, 
                    horizontal_flip=True, 
                    zoom_range=0.5)

train_generator=train_datagen.flow_from_dataframe(
    dataframe=df_train_file_cat,
    directory="../input/train_val2019",
    x_col="file_name",
    y_col="category_id",
    batch_size=batch_size,
    shuffle=True,
    class_mode="sparse",    
    target_size=(img_size,img_size))
Found 265213 images belonging to 1010 classes.
CPU times: user 5.5 s, sys: 7.65 s, total: 13.2 s
Wall time: 1min 30s
In [12]:
# udacity_intro_to_tensorflow_for_deep_learning/l05c04_exercise_flowers_with_data_augmentation_solution.ipynb#scrollTo=jqb9OGoVKIOi
# This function will plot images in the form of a grid with 1 row and 5 columns where images are placed in each column.
def plotImages(images_arr):
    fig, axes = plt.subplots(1, 5, figsize=(20,20))
    axes = axes.flatten()
    for img, ax in zip( images_arr, axes):
        ax.imshow(img)
    plt.tight_layout()
    plt.show()
    
    
augmented_images = [train_generator[0][0][0] for i in range(5)]
plotImages(augmented_images)
In [13]:
%%time
test_datagen = ImageDataGenerator(rescale=1./255)

valid_generator=test_datagen.flow_from_dataframe(
    dataframe=df_valid_file_cat,
    directory="../input/train_val2019",
    x_col="file_name",
    y_col="category_id",
    batch_size=batch_size,
    class_mode="sparse",    
    target_size=(img_size,img_size))
Found 3030 images belonging to 1010 classes.
CPU times: user 64 ms, sys: 80 ms, total: 144 ms
Wall time: 1.03 s
In [14]:
vgg16_net = VGG16(weights='imagenet', 
                  include_top=False, 
                  input_shape=(img_size, img_size, 3))
vgg16_net.trainable = False
WARNING:tensorflow:From /opt/conda/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
Downloading data from https://github.com/fchollet/deep-learning-models/releases/download/v0.1/vgg16_weights_tf_dim_ordering_tf_kernels_notop.h5
58892288/58889256 [==============================] - 1s 0us/step
In [15]:
resnet = ResNet50(include_top=False, weights='imagenet',
               input_shape=(img_size,img_size,3))
resnet.trainable = False
/opt/conda/lib/python3.6/site-packages/keras_applications/resnet50.py:265: UserWarning: The output shape of `ResNet50(include_top=False)` has been changed since Keras 2.2.0.
  warnings.warn('The output shape of `ResNet50(include_top=False)` '
Downloading data from https://github.com/fchollet/deep-learning-models/releases/download/v0.2/resnet50_weights_tf_dim_ordering_tf_kernels_notop.h5
94658560/94653016 [==============================] - 6s 0us/step

Model

In [16]:
model = Sequential()
model.add(vgg16_net)
model.add(Flatten())
model.add(Dense(1024))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(nb_classes, activation='softmax'))

model.compile(optimizers.rmsprop(lr=0.0001, decay=1e-6),loss='sparse_categorical_crossentropy',metrics=['accuracy'])
WARNING:tensorflow:From /opt/conda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:3445: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.
In [17]:
%%time
history = model.fit_generator(generator=train_generator, 
                              
                    steps_per_epoch=500,
                              
                    validation_data=valid_generator, 
                              
                    validation_steps=100,
                              
                    epochs=nb_epochs,
                    verbose=0)
WARNING:tensorflow:From /opt/conda/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
CPU times: user 3h 42min 11s, sys: 6min 16s, total: 3h 48min 28s
Wall time: 4h 22min 20s
In [18]:
with open('history.json', 'w') as f:
    json.dump(history.history, f)

history_df = pd.DataFrame(history.history)
history_df[['loss', 'val_loss']].plot()
history_df[['acc', 'val_acc']].plot()
Out[18]:
<matplotlib.axes._subplots.AxesSubplot at 0x7f96f03ef828>
In [19]:
acc = history.history['acc']
val_acc = history.history['val_acc']

loss = history.history['loss']
val_loss = history.history['val_loss']

epochs_range = range(nb_epochs)

plt.figure(figsize=(8, 8))
plt.subplot(1, 2, 1)
plt.plot(epochs_range, acc, label='Training Accuracy')
plt.plot(epochs_range, val_acc, label='Validation Accuracy')
plt.legend(loc='lower right')
plt.title('Training and Validation Accuracy')

plt.subplot(1, 2, 2)
plt.plot(epochs_range, loss, label='Training Loss')
plt.plot(epochs_range, val_loss, label='Validation Loss')
plt.legend(loc='upper right')
plt.title('Training and Validation Loss')
plt.show()

Test data

In [20]:
test_ann_file = '../input/test2019.json'
with open(test_ann_file) as data_file:
        test_anns = json.load(data_file)
In [21]:
test_img_df = pd.DataFrame(test_anns['images'])[['id', 'file_name']].rename(columns={'id':'image_id'})
test_img_df.head()
Out[21]:
image_id file_name
0 268243 test2019/e295f3c7046b1f1e80c0301401324aa9.jpg
1 268244 test2019/ad3dcbb6846ed0b4dab58d7b1a4210ba.jpg
2 268245 test2019/e697be8e296b4b140cff4f96f85c364f.jpg
3 268246 test2019/7e7ba55e6aa26ba99e814d63b15d0121.jpg
4 268247 test2019/6cb6372079d23702511c06923970f13f.jpg
In [22]:
%%time
test_datagen = ImageDataGenerator(rescale=1./255.)
test_generator = test_datagen.flow_from_dataframe(      
    
        dataframe=test_img_df,    
    
        directory = "../input/test2019",    
        x_col="file_name",
        target_size = (img_size,img_size),
        batch_size = 1,
        shuffle=False,
        class_mode = None
        )
Found 35350 images.
CPU times: user 1.62 s, sys: 756 ms, total: 2.38 s
Wall time: 13.2 s

Prediction

In [23]:
%%time
test_generator.reset()
predict=model.predict_generator(test_generator, steps = len(test_generator.filenames),verbose=1)
10647/35350 [========>.....................] - ETA: 28:49
In [24]:
len(predict)
Out[24]:
35350
In [25]:
predicted_class_indices=np.argmax(predict,axis=1)
In [26]:
labels = (train_generator.class_indices)
labels = dict((v,k) for k,v in labels.items())
predictions = [labels[k] for k in predicted_class_indices]
In [27]:
sam_sub_df = pd.read_csv('../input/kaggle_sample_submission.csv')
sam_sub_df.head()
Out[27]:
id predicted
0 268243 842
1 268244 139
2 268245 988
3 268246 612
4 268247 468
In [28]:
filenames=test_generator.filenames
results=pd.DataFrame({"file_name":filenames,
                      "predicted":predictions})
df_res = pd.merge(test_img_df, results, on='file_name')[['image_id','predicted']]\
    .rename(columns={'image_id':'id'})

df_res.head()
Out[28]:
id predicted
0 268243 5
1 268244 326
2 268245 976
3 268246 263
4 268247 373
In [29]:
df_res.to_csv("submission.csv",index=False)

Hope you like it and finds this kernel helpful :)!