Image Classification of Fresh Fish and Non Fresh Fish


Intent

The fish sorting process carried out by fishermen or sellers, to select fish based on quality is still using manual methods and sometimes misses due to the limited sense of sight when tired. So far the examination has only been seen physically. As a result, the fish will often be damaged when consumed. This research tries to apply the Convolutional Neural Network (CNN) algorithm to distinguish between fresh and non-fresh fish.

This work has been published in the Indonesian Journal of Applied Informatics - Universitas Sebelas Maret and is Nationally accredited with SINTA 4. To view articles from this research journal, click on the following link.

Import The Library

Import some required libraries. In this project, the required library is TensorFlow to create a Convolution Neural Network (CNN) model.

import os
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D, Flatten, Dense, Activation
from keras.preprocessing.image import ImageDataGenerator
from keras.metrics import categorical_accuracy
from tensorflow.keras.preprocessing.image import ImageDataGenerator
import warnings
warnings.filterwarnings('ignore')

size = 64
base_dir = '/content/drive/My Drive/fresh and non-fresh fish'
train_dir = os.path.join(base_dir, 'train')
val_dir = os.path.join(base_dir, 'val')
print("content of train folder:",os.listdir(train_dir))
print("content of val folder:",os.listdir(val_dir))
content of train folder: ['fresh', 'non-fresh']
content of val folder: ['non-fresh', 'fresh']

Dataset Preview

The data used are fresh fisheye images and fresh fisheye images. Each image is divided into training data and test data based on the the folder which has previously been adjusted to the data label.

import glob
import matplotlib.pyplot as plt

def images_preview(x):
  a = glob.glob(x)

  w=10
  h=10
  fig=plt.figure(figsize=(8, 8))
  columns = 3
  rows = 1
  for i in range(1, columns*rows +1):
      img = plt.imread(a[i])
      fig.add_subplot(rows, columns, i)
      plt.imshow(img)
  plt.show()

print("images preview of fresh fish train")
images_preview("/content/drive/My Drive/fresh and non-fresh fish/train/fresh/*")
print("\nimages preview of non-fresh fish train")
images_preview("/content/drive/My Drive/fresh and non-fresh fish/train/non-fresh/*")
images preview of fresh fish train

images preview of non-fresh fish train

print("images preview of fresh fish validation")
images_preview("/content/drive/My Drive/fresh and non-fresh fish/val/fresh/*")
print("\nimages preview of non-fresh fish validation")
images_preview("/content/drive/My Drive/fresh and non-fresh fish/val/non-fresh/*")
images preview of fresh fish validation
images preview of non-fresh fish validation

Image Augmentation

Because the number of datasets is limited, which is only 40 datasets. Then the augmentation process is carried out on the dataset

train = ImageDataGenerator(rescale = 1./255)
test = ImageDataGenerator(rescale = 1./255)
train_data = train.flow_from_directory(
    train_dir,
    target_size = (size, size),
    batch_size = 4,
    class_mode = 'binary')
test_data = test.flow_from_directory(
    val_dir,
    target_size = (size, size),
    batch_size = 4,
    class_mode = 'binary')
Found 20 images belonging to 2 classes.
Found 20 images belonging to 2 classes.

Define The CNN Model

The CNN model used has several layers with convolution, max-pooling, flatten, and dense layer types.

model = Sequential()

model.add(Conv2D(4, (3, 3), input_shape = (size, size, 3), activation = 'relu'))
model.add(MaxPooling2D(pool_size = (2, 2)))
model.add(Flatten())

model.add(Dense(units = 8, activation = 'relu'))
model.add(Dense(units = 1, activation = 'sigmoid'))

model.summary()
Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d (Conv2D)              (None, 62, 62, 4)         112       
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 31, 31, 4)         0         
_________________________________________________________________
flatten (Flatten)            (None, 3844)              0         
_________________________________________________________________
dense (Dense)                (None, 8)                 30760     
_________________________________________________________________
dense_1 (Dense)              (None, 1)                 9         
=================================================================
Total params: 30,881
Trainable params: 30,881
Non-trainable params: 0
_________________________________________________________________
              

Function to Visualize The Accuracy and Loss Value

Create a function to visualize the accuracy and loss values ​​of the training and testing process on the dataset with the model defined in the previous step.

plt.style.use('seaborn-whitegrid')

def plot_acc(history):
acc = history.history['binary_accuracy']
val_acc = history.history['val_binary_accuracy']
epochs = range(len(acc))
plt.subplot(1, 2, 1)
acc_plot, = plt.plot(epochs, acc, 'r')
val_acc_plot, = plt.plot(epochs, val_acc, 'b')
plt.title('Training and Validation Accuracy')
plt.legend([acc_plot, val_acc_plot], ['Training Accuracy', 'Validation Accuracy'])


def plot_loss(history):
loss = history.history['loss']
val_loss = history.history['val_loss']
epochs = range(len(loss))
plt.subplot(1, 2, 2)
loss_plot, = plt.plot(epochs, loss, 'r')
val_loss_plot, = plt.plot(epochs, val_loss, 'b')
plt.title('Training and Validation Loss')
plt.legend([loss_plot, val_loss_plot], ['Training Loss', 'Validation Loss'])

def plot_graph(history):
plt.figure(figsize=(15,5))
plot_acc(history)
plot_loss(history)

Train and Test The Model

After carrying out several stages such as exploratory data analysis, modeling, and creating functions to visualize the accuracy and function values, the next step is to run the model. The model is compiled to carry out the training process and model testing on the defined data.

model.compile(
    optimizer = 'adam',
    loss = 'binary_crossentropy',
    metrics = ['binary_accuracy'])

accuracy_graph = model.fit_generator(
    train_data,
    steps_per_epoch = 50,
    epochs = 20,
    validation_data = test_data,
    validation_steps = 50)
Epoch 1/20
50/50 [==============================] - 6s 122ms/step - loss: 0.1147 - binary_accuracy: 1.0000 - val_loss: 0.2753 - val_binary_accuracy: 0.8500
Epoch 2/20
50/50 [==============================] - 6s 121ms/step - loss: 0.1082 - binary_accuracy: 1.0000 - val_loss: 0.2762 - val_binary_accuracy: 0.8500
Epoch 3/20
50/50 [==============================] - 6s 119ms/step - loss: 0.1043 - binary_accuracy: 1.0000 - val_loss: 0.2789 - val_binary_accuracy: 0.8516
Epoch 4/20
50/50 [==============================] - 6s 122ms/step - loss: 0.1005 - binary_accuracy: 1.0000 - val_loss: 0.2810 - val_binary_accuracy: 0.8516
Epoch 5/20
50/50 [==============================] - 6s 118ms/step - loss: 0.1028 - binary_accuracy: 1.0000 - val_loss: 0.2834 - val_binary_accuracy: 0.8500
Epoch 6/20
50/50 [==============================] - 6s 119ms/step - loss: 0.0962 - binary_accuracy: 1.0000 - val_loss: 0.2880 - val_binary_accuracy: 0.8500
Epoch 7/20
50/50 [==============================] - 6s 118ms/step - loss: 0.0936 - binary_accuracy: 1.0000 - val_loss: 0.2933 - val_binary_accuracy: 0.8484
Epoch 8/20
50/50 [==============================] - 6s 119ms/step - loss: 0.0930 - binary_accuracy: 1.0000 - val_loss: 0.2981 - val_binary_accuracy: 0.8484
Epoch 9/20
50/50 [==============================] - 6s 120ms/step - loss: 0.0888 - binary_accuracy: 1.0000 - val_loss: 0.2923 - val_binary_accuracy: 0.8531
Epoch 10/20
50/50 [==============================] - 6s 119ms/step - loss: 0.0882 - binary_accuracy: 1.0000 - val_loss: 0.3065 - val_binary_accuracy: 0.8469
Epoch 11/20
50/50 [==============================] - 6s 121ms/step - loss: 0.0834 - binary_accuracy: 1.0000 - val_loss: 0.3051 - val_binary_accuracy: 0.8500
Epoch 12/20
50/50 [==============================] - 6s 120ms/step - loss: 0.0838 - binary_accuracy: 1.0000 - val_loss: 0.3049 - val_binary_accuracy: 0.8516
Epoch 13/20
50/50 [==============================] - 6s 120ms/step - loss: 0.0816 - binary_accuracy: 1.0000 - val_loss: 0.3146 - val_binary_accuracy: 0.8500
Epoch 14/20
50/50 [==============================] - 6s 120ms/step - loss: 0.0787 - binary_accuracy: 1.0000 - val_loss: 0.3112 - val_binary_accuracy: 0.8516
Epoch 15/20
50/50 [==============================] - 6s 117ms/step - loss: 0.0767 - binary_accuracy: 1.0000 - val_loss: 0.3189 - val_binary_accuracy: 0.8500
Epoch 16/20
50/50 [==============================] - 6s 119ms/step - loss: 0.0755 - binary_accuracy: 1.0000 - val_loss: 0.1327 - val_binary_accuracy: 0.9500
Epoch 17/20
50/50 [==============================] - 6s 119ms/step - loss: 0.0743 - binary_accuracy: 1.0000 - val_loss: 0.1088 - val_binary_accuracy: 0.9500
Epoch 18/20
50/50 [==============================] - 6s 120ms/step - loss: 0.0690 - binary_accuracy: 1.0000 - val_loss: 0.1915 - val_binary_accuracy: 0.9500
Epoch 19/20
50/50 [==============================] - 6s 120ms/step - loss: 0.0707 - binary_accuracy: 1.0000 - val_loss: 0.1895 - val_binary_accuracy: 0.9516
Epoch 20/20
50/50 [==============================] - 6s 122ms/step - loss: 0.0676 - binary_accuracy: 1.0000 - val_loss: 0.1932 - val_binary_accuracy: 0.9500
              

Visualize The Accuracy

After completing training and validating the model. The next step is to visualize the accuracy and loss values ​​obtained. The graph below shows the comparison of the accuracy value and the loss value obtained from the training and validation process.

plot_graph(accuracy_graph)

Predict Image

Create a function to convert the index obtained from the prediction results into a label that will display whether the entered image is a fresh fish image or not. Then I try to load the image from my storage and try the model which has been trained and validated. So that it produces predictions like the one below.

def switch_dict_key_values(this_dict):
    return dict((v,k) for k,v in this_dict.items())

    nama_train_data = switch_dict_key_values(train_data.class_indices)
    print(nama_train_data)
    

import numpy as np
from google.colab import files
from keras.preprocessing import image
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
%matplotlib inline
 
uploaded = files.upload()
 
for fn in uploaded.keys():
 
  # predicting images
  path = fn

  def tampilkan_(ini):
    fig = plt.figure(figsize=(2, 2))
    img = plt.imread(ini)
    plt.imshow(img)
    plt.show()

  tampilkan_(path)
  img = image.load_img(path, target_size = (size, size))
  img = image.img_to_array(img)
  img = np.expand_dims(img, axis = 0)

  hasil = model.predict_classes(img)
  
  print(fn)
  if hasil==0:
    jenis = "segar"
  else:
    jenis = "tidak segar"

  print("Predicted as ", hasil, "labeled as ",jenis)
Saving fresh fish sample.jpg to fresh fish sample.jpg
fresh fish sample.jpg
Predicted as  [[0]] labeled as  segar

Conclusion

From the training and validation processes, both produce high accuracy values, the training process produces an accuracy value of 100% and the validation process is 95%. Then the model is tested to predict the eye pupil images of fresh fish taken from file explorer and identified as fresh fish. This shows that the model has good performance.