Keras MobileNet Data Augmentation & Visualize

From: https://www.kaggle.com/hsinwenchang/keras-mobilenet-data-augmentation-visualize

Author: Beans

Score: 0.97509

This kernel is base on Alexander Teplyuk here I applied Data Augmentation technic from Udacity as following:

  • random 45 degree rotation
  • random zoom of up to 50%
  • random horizontal flip
  • width shift of 0.15
  • height shfit of 0.15
  • Model: MobileNet
    Apply transfer learning skill from pretrained model using Keras. Using MobileNet.
In [1]:
import os
import cv2
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
import json
from keras.models import Sequential
from keras.layers import Dense, Flatten, Activation, Dropout, Conv2D,MaxPooling2D
from keras.optimizers import Adam
from keras.preprocessing.image import ImageDataGenerator
from keras.applications import MobileNet
from keras import optimizers
Using TensorFlow backend.

Train data

In [2]:
ann_file = '../input/train2019.json'
with open(ann_file) as data_file:
        train_anns = json.load(data_file)
In [3]:
train_anns_df = pd.DataFrame(train_anns['annotations'])[['image_id','category_id']]
train_img_df = pd.DataFrame(train_anns['images'])[['id', 'file_name']].rename(columns={'id':'image_id'})
df_train_file_cat = pd.merge(train_img_df, train_anns_df, on='image_id')
df_train_file_cat['category_id']=df_train_file_cat['category_id'].astype(str)
df_train_file_cat.head()
Out[3]:
image_id file_name category_id
0 0 train_val2019/Plants/400/d1322d13ccd856eb4236c... 400
1 1 train_val2019/Plants/570/15edbc1e2ef000d8ace48... 570
2 2 train_val2019/Reptiles/167/c87a32e8927cbf4f06d... 167
3 3 train_val2019/Birds/254/9fcdd1d37e96d8fd94dfdc... 254
4 4 train_val2019/Plants/739/ffa06f951e99de9d220ae... 739
In [4]:
len(df_train_file_cat['category_id'].unique())
Out[4]:
1010
In [5]:
# Example of images for category_id = 400
img_names = df_train_file_cat[df_train_file_cat['category_id']=='400']['file_name'][:30]

plt.figure(figsize=[15,15])
i = 1
for img_name in img_names:
    img = cv2.imread("../input/train_val2019/%s" % img_name)[...,[2, 1, 0]]
    plt.subplot(6, 5, i)
    plt.imshow(img)
    i += 1
plt.show()

Validation data

In [6]:
valid_ann_file = '../input/val2019.json'
with open(valid_ann_file) as data_file:
        valid_anns = json.load(data_file)
In [7]:
valid_anns_df = pd.DataFrame(valid_anns['annotations'])[['image_id','category_id']]
valid_anns_df.head()
Out[7]:
image_id category_id
0 265213 644
1 265214 597
2 265215 883
3 265216 300
4 265217 881
In [8]:
valid_img_df = pd.DataFrame(valid_anns['images'])[['id', 'file_name']].rename(columns={'id':'image_id'})
valid_img_df.head()
Out[8]:
image_id file_name
0 265213 train_val2019/Plants/644/716a69838526f3ada3b2f...
1 265214 train_val2019/Plants/597/0942cc64d2e759c5ee059...
2 265215 train_val2019/Plants/883/acfdbfd9fa675f1c84558...
3 265216 train_val2019/Birds/300/5f3194ff536c7dd31d80b7...
4 265217 train_val2019/Plants/881/76acaf0b2841f91982d21...
In [9]:
df_valid_file_cat = pd.merge(valid_img_df, valid_anns_df, on='image_id')
df_valid_file_cat['category_id']=df_valid_file_cat['category_id'].astype(str)
df_valid_file_cat.head()
Out[9]:
image_id file_name category_id
0 265213 train_val2019/Plants/644/716a69838526f3ada3b2f... 644
1 265214 train_val2019/Plants/597/0942cc64d2e759c5ee059... 597
2 265215 train_val2019/Plants/883/acfdbfd9fa675f1c84558... 883
3 265216 train_val2019/Birds/300/5f3194ff536c7dd31d80b7... 300
4 265217 train_val2019/Plants/881/76acaf0b2841f91982d21... 881
In [10]:
nb_classes = 1010
batch_size = 128
img_size = 128
nb_epochs = 10

In the cell below, use ImageDataGenerator to create a transformation that rescales the images by 255 here I applied Data Augmentation as following:

  • random 45 degree rotation
  • random zoom of up to 50%
  • random horizontal flip
  • width shift of 0.15
  • height shfit of 0.15
In [11]:
%%time
train_datagen=ImageDataGenerator(rescale=1./255, rotation_range=45, 
                    width_shift_range=.15, 
                    height_shift_range=.15, 
                    horizontal_flip=True, 
                    zoom_range=0.5)

train_generator=train_datagen.flow_from_dataframe(
    dataframe=df_train_file_cat,
    directory="../input/train_val2019",
    x_col="file_name",
    y_col="category_id",
    batch_size=batch_size,
    shuffle=True,
    class_mode="sparse",    
    target_size=(img_size,img_size))
Found 265213 images belonging to 1010 classes.
CPU times: user 5.7 s, sys: 8.46 s, total: 14.2 s
Wall time: 2min 36s
In [12]:
# udacity_intro_to_tensorflow_for_deep_learning/l05c04_exercise_flowers_with_data_augmentation_solution.ipynb#scrollTo=jqb9OGoVKIOi
# This function will plot images in the form of a grid with 1 row and 5 columns where images are placed in each column.
def plotImages(images_arr):
    fig, axes = plt.subplots(1, 5, figsize=(20,20))
    axes = axes.flatten()
    for img, ax in zip( images_arr, axes):
        ax.imshow(img)
    plt.tight_layout()
    plt.show()
    
    
augmented_images = [train_generator[0][0][0] for i in range(5)]
plotImages(augmented_images)
In [13]:
%%time
test_datagen = ImageDataGenerator(rescale=1./255)

valid_generator=test_datagen.flow_from_dataframe(
    dataframe=df_valid_file_cat,
    directory="../input/train_val2019",
    x_col="file_name",
    y_col="category_id",
    batch_size=batch_size,
    class_mode="sparse",    
    target_size=(img_size,img_size))
Found 3030 images belonging to 1010 classes.
CPU times: user 68 ms, sys: 104 ms, total: 172 ms
Wall time: 1.61 s
In [14]:
model = MobileNet(input_shape=(img_size, img_size, 3), alpha=1., weights=None, classes=nb_classes)

model.compile(optimizers.rmsprop(lr=0.0001, decay=1e-6),loss='sparse_categorical_crossentropy',metrics=['accuracy'])
print(model.summary())
WARNING:tensorflow:From /opt/conda/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From /opt/conda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:3445: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_1 (InputLayer)         (None, 128, 128, 3)       0         
_________________________________________________________________
conv1_pad (ZeroPadding2D)    (None, 129, 129, 3)       0         
_________________________________________________________________
conv1 (Conv2D)               (None, 64, 64, 32)        864       
_________________________________________________________________
conv1_bn (BatchNormalization (None, 64, 64, 32)        128       
_________________________________________________________________
conv1_relu (ReLU)            (None, 64, 64, 32)        0         
_________________________________________________________________
conv_dw_1 (DepthwiseConv2D)  (None, 64, 64, 32)        288       
_________________________________________________________________
conv_dw_1_bn (BatchNormaliza (None, 64, 64, 32)        128       
_________________________________________________________________
conv_dw_1_relu (ReLU)        (None, 64, 64, 32)        0         
_________________________________________________________________
conv_pw_1 (Conv2D)           (None, 64, 64, 64)        2048      
_________________________________________________________________
conv_pw_1_bn (BatchNormaliza (None, 64, 64, 64)        256       
_________________________________________________________________
conv_pw_1_relu (ReLU)        (None, 64, 64, 64)        0         
_________________________________________________________________
conv_pad_2 (ZeroPadding2D)   (None, 65, 65, 64)        0         
_________________________________________________________________
conv_dw_2 (DepthwiseConv2D)  (None, 32, 32, 64)        576       
_________________________________________________________________
conv_dw_2_bn (BatchNormaliza (None, 32, 32, 64)        256       
_________________________________________________________________
conv_dw_2_relu (ReLU)        (None, 32, 32, 64)        0         
_________________________________________________________________
conv_pw_2 (Conv2D)           (None, 32, 32, 128)       8192      
_________________________________________________________________
conv_pw_2_bn (BatchNormaliza (None, 32, 32, 128)       512       
_________________________________________________________________
conv_pw_2_relu (ReLU)        (None, 32, 32, 128)       0         
_________________________________________________________________
conv_dw_3 (DepthwiseConv2D)  (None, 32, 32, 128)       1152      
_________________________________________________________________
conv_dw_3_bn (BatchNormaliza (None, 32, 32, 128)       512       
_________________________________________________________________
conv_dw_3_relu (ReLU)        (None, 32, 32, 128)       0         
_________________________________________________________________
conv_pw_3 (Conv2D)           (None, 32, 32, 128)       16384     
_________________________________________________________________
conv_pw_3_bn (BatchNormaliza (None, 32, 32, 128)       512       
_________________________________________________________________
conv_pw_3_relu (ReLU)        (None, 32, 32, 128)       0         
_________________________________________________________________
conv_pad_4 (ZeroPadding2D)   (None, 33, 33, 128)       0         
_________________________________________________________________
conv_dw_4 (DepthwiseConv2D)  (None, 16, 16, 128)       1152      
_________________________________________________________________
conv_dw_4_bn (BatchNormaliza (None, 16, 16, 128)       512       
_________________________________________________________________
conv_dw_4_relu (ReLU)        (None, 16, 16, 128)       0         
_________________________________________________________________
conv_pw_4 (Conv2D)           (None, 16, 16, 256)       32768     
_________________________________________________________________
conv_pw_4_bn (BatchNormaliza (None, 16, 16, 256)       1024      
_________________________________________________________________
conv_pw_4_relu (ReLU)        (None, 16, 16, 256)       0         
_________________________________________________________________
conv_dw_5 (DepthwiseConv2D)  (None, 16, 16, 256)       2304      
_________________________________________________________________
conv_dw_5_bn (BatchNormaliza (None, 16, 16, 256)       1024      
_________________________________________________________________
conv_dw_5_relu (ReLU)        (None, 16, 16, 256)       0         
_________________________________________________________________
conv_pw_5 (Conv2D)           (None, 16, 16, 256)       65536     
_________________________________________________________________
conv_pw_5_bn (BatchNormaliza (None, 16, 16, 256)       1024      
_________________________________________________________________
conv_pw_5_relu (ReLU)        (None, 16, 16, 256)       0         
_________________________________________________________________
conv_pad_6 (ZeroPadding2D)   (None, 17, 17, 256)       0         
_________________________________________________________________
conv_dw_6 (DepthwiseConv2D)  (None, 8, 8, 256)         2304      
_________________________________________________________________
conv_dw_6_bn (BatchNormaliza (None, 8, 8, 256)         1024      
_________________________________________________________________
conv_dw_6_relu (ReLU)        (None, 8, 8, 256)         0         
_________________________________________________________________
conv_pw_6 (Conv2D)           (None, 8, 8, 512)         131072    
_________________________________________________________________
conv_pw_6_bn (BatchNormaliza (None, 8, 8, 512)         2048      
_________________________________________________________________
conv_pw_6_relu (ReLU)        (None, 8, 8, 512)         0         
_________________________________________________________________
conv_dw_7 (DepthwiseConv2D)  (None, 8, 8, 512)         4608      
_________________________________________________________________
conv_dw_7_bn (BatchNormaliza (None, 8, 8, 512)         2048      
_________________________________________________________________
conv_dw_7_relu (ReLU)        (None, 8, 8, 512)         0         
_________________________________________________________________
conv_pw_7 (Conv2D)           (None, 8, 8, 512)         262144    
_________________________________________________________________
conv_pw_7_bn (BatchNormaliza (None, 8, 8, 512)         2048      
_________________________________________________________________
conv_pw_7_relu (ReLU)        (None, 8, 8, 512)         0         
_________________________________________________________________
conv_dw_8 (DepthwiseConv2D)  (None, 8, 8, 512)         4608      
_________________________________________________________________
conv_dw_8_bn (BatchNormaliza (None, 8, 8, 512)         2048      
_________________________________________________________________
conv_dw_8_relu (ReLU)        (None, 8, 8, 512)         0         
_________________________________________________________________
conv_pw_8 (Conv2D)           (None, 8, 8, 512)         262144    
_________________________________________________________________
conv_pw_8_bn (BatchNormaliza (None, 8, 8, 512)         2048      
_________________________________________________________________
conv_pw_8_relu (ReLU)        (None, 8, 8, 512)         0         
_________________________________________________________________
conv_dw_9 (DepthwiseConv2D)  (None, 8, 8, 512)         4608      
_________________________________________________________________
conv_dw_9_bn (BatchNormaliza (None, 8, 8, 512)         2048      
_________________________________________________________________
conv_dw_9_relu (ReLU)        (None, 8, 8, 512)         0         
_________________________________________________________________
conv_pw_9 (Conv2D)           (None, 8, 8, 512)         262144    
_________________________________________________________________
conv_pw_9_bn (BatchNormaliza (None, 8, 8, 512)         2048      
_________________________________________________________________
conv_pw_9_relu (ReLU)        (None, 8, 8, 512)         0         
_________________________________________________________________
conv_dw_10 (DepthwiseConv2D) (None, 8, 8, 512)         4608      
_________________________________________________________________
conv_dw_10_bn (BatchNormaliz (None, 8, 8, 512)         2048      
_________________________________________________________________
conv_dw_10_relu (ReLU)       (None, 8, 8, 512)         0         
_________________________________________________________________
conv_pw_10 (Conv2D)          (None, 8, 8, 512)         262144    
_________________________________________________________________
conv_pw_10_bn (BatchNormaliz (None, 8, 8, 512)         2048      
_________________________________________________________________
conv_pw_10_relu (ReLU)       (None, 8, 8, 512)         0         
_________________________________________________________________
conv_dw_11 (DepthwiseConv2D) (None, 8, 8, 512)         4608      
_________________________________________________________________
conv_dw_11_bn (BatchNormaliz (None, 8, 8, 512)         2048      
_________________________________________________________________
conv_dw_11_relu (ReLU)       (None, 8, 8, 512)         0         
_________________________________________________________________
conv_pw_11 (Conv2D)          (None, 8, 8, 512)         262144    
_________________________________________________________________
conv_pw_11_bn (BatchNormaliz (None, 8, 8, 512)         2048      
_________________________________________________________________
conv_pw_11_relu (ReLU)       (None, 8, 8, 512)         0         
_________________________________________________________________
conv_pad_12 (ZeroPadding2D)  (None, 9, 9, 512)         0         
_________________________________________________________________
conv_dw_12 (DepthwiseConv2D) (None, 4, 4, 512)         4608      
_________________________________________________________________
conv_dw_12_bn (BatchNormaliz (None, 4, 4, 512)         2048      
_________________________________________________________________
conv_dw_12_relu (ReLU)       (None, 4, 4, 512)         0         
_________________________________________________________________
conv_pw_12 (Conv2D)          (None, 4, 4, 1024)        524288    
_________________________________________________________________
conv_pw_12_bn (BatchNormaliz (None, 4, 4, 1024)        4096      
_________________________________________________________________
conv_pw_12_relu (ReLU)       (None, 4, 4, 1024)        0         
_________________________________________________________________
conv_dw_13 (DepthwiseConv2D) (None, 4, 4, 1024)        9216      
_________________________________________________________________
conv_dw_13_bn (BatchNormaliz (None, 4, 4, 1024)        4096      
_________________________________________________________________
conv_dw_13_relu (ReLU)       (None, 4, 4, 1024)        0         
_________________________________________________________________
conv_pw_13 (Conv2D)          (None, 4, 4, 1024)        1048576   
_________________________________________________________________
conv_pw_13_bn (BatchNormaliz (None, 4, 4, 1024)        4096      
_________________________________________________________________
conv_pw_13_relu (ReLU)       (None, 4, 4, 1024)        0         
_________________________________________________________________
global_average_pooling2d_1 ( (None, 1024)              0         
_________________________________________________________________
reshape_1 (Reshape)          (None, 1, 1, 1024)        0         
_________________________________________________________________
dropout (Dropout)            (None, 1, 1, 1024)        0         
_________________________________________________________________
conv_preds (Conv2D)          (None, 1, 1, 1010)        1035250   
_________________________________________________________________
act_softmax (Activation)     (None, 1, 1, 1010)        0         
_________________________________________________________________
reshape_2 (Reshape)          (None, 1010)              0         
=================================================================
Total params: 4,264,114
Trainable params: 4,242,226
Non-trainable params: 21,888
_________________________________________________________________
None
In [15]:
%%time
history = model.fit_generator(generator=train_generator, 
                              
                    steps_per_epoch=500,
                              
                    validation_data=valid_generator, 
                              
                    validation_steps=100,
                              
                    epochs=nb_epochs,
                    verbose=0)
WARNING:tensorflow:From /opt/conda/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
WARNING:tensorflow:From /opt/conda/lib/python3.6/site-packages/tensorflow/python/ops/math_grad.py:102: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Deprecated in favor of operator or tf.math.divide.
CPU times: user 3h 45min 2s, sys: 13min 10s, total: 3h 58min 13s
Wall time: 4h 29min 17s
In [16]:
with open('history.json', 'w') as f:
    json.dump(history.history, f)

history_df = pd.DataFrame(history.history)
history_df[['loss', 'val_loss']].plot()
history_df[['acc', 'val_acc']].plot()
Out[16]:
<matplotlib.axes._subplots.AxesSubplot at 0x7fdbfb857a58>
In [17]:
acc = history.history['acc']
val_acc = history.history['val_acc']

loss = history.history['loss']
val_loss = history.history['val_loss']

epochs_range = range(nb_epochs)

plt.figure(figsize=(8, 8))
plt.subplot(1, 2, 1)
plt.plot(epochs_range, acc, label='Training Accuracy')
plt.plot(epochs_range, val_acc, label='Validation Accuracy')
plt.legend(loc='lower right')
plt.title('Training and Validation Accuracy')

plt.subplot(1, 2, 2)
plt.plot(epochs_range, loss, label='Training Loss')
plt.plot(epochs_range, val_loss, label='Validation Loss')
plt.legend(loc='upper right')
plt.title('Training and Validation Loss')
plt.show()

Test data

In [18]:
test_ann_file = '../input/test2019.json'
with open(test_ann_file) as data_file:
        test_anns = json.load(data_file)
In [19]:
test_img_df = pd.DataFrame(test_anns['images'])[['id', 'file_name']].rename(columns={'id':'image_id'})
test_img_df.head()
Out[19]:
image_id file_name
0 268243 test2019/e295f3c7046b1f1e80c0301401324aa9.jpg
1 268244 test2019/ad3dcbb6846ed0b4dab58d7b1a4210ba.jpg
2 268245 test2019/e697be8e296b4b140cff4f96f85c364f.jpg
3 268246 test2019/7e7ba55e6aa26ba99e814d63b15d0121.jpg
4 268247 test2019/6cb6372079d23702511c06923970f13f.jpg
In [20]:
%%time
test_datagen = ImageDataGenerator(rescale=1./255.)
test_generator = test_datagen.flow_from_dataframe(      
    
        dataframe=test_img_df,    
    
        directory = "../input/test2019",    
        x_col="file_name",
        target_size = (img_size,img_size),
        batch_size = 1,
        shuffle=False,
        class_mode = None
        )
Found 35350 images.
CPU times: user 2.2 s, sys: 572 ms, total: 2.77 s
Wall time: 15.3 s

Prediction

In [21]:
%%time
test_generator.reset()
predict=model.predict_generator(test_generator, steps = len(test_generator.filenames),verbose=1)
11178/35350 [========>.....................] - ETA: 27:53
In [22]:
len(predict)
Out[22]:
35350
In [23]:
predicted_class_indices=np.argmax(predict,axis=1)
In [24]:
labels = (train_generator.class_indices)
labels = dict((v,k) for k,v in labels.items())
predictions = [labels[k] for k in predicted_class_indices]
In [25]:
sam_sub_df = pd.read_csv('../input/kaggle_sample_submission.csv')
sam_sub_df.head()
Out[25]:
id predicted
0 268243 842
1 268244 139
2 268245 988
3 268246 612
4 268247 468
In [26]:
filenames=test_generator.filenames
results=pd.DataFrame({"file_name":filenames,
                      "predicted":predictions})
df_res = pd.merge(test_img_df, results, on='file_name')[['image_id','predicted']]\
    .rename(columns={'image_id':'id'})

df_res.head()
Out[26]:
id predicted
0 268243 523
1 268244 320
2 268245 577
3 268246 73
4 268247 738
In [27]:
df_res.to_csv("submission.csv",index=False)

Hope you like it and finds this kernel helpful :)!