How to correctly use ImageDataGenerator in Keras?

I think the documentation can be quite confusing and I imagine the behavior is different depending on your Tensorflow and Keras version. For example, in this post, the user is describing the exact behavior you are expecting. Generally, the flow_from_directory() method allows you to read the images directly from a directory and augment them while your model is being trained and as already stated here, it iterates for every sample in each folder every epoch. Using the following example, you can check that this is the case (on TF 2.7) by looking at the steps per epoch in the progress bar:

import tensorflow as tf

BATCH_SIZE = 64

flowers = tf.keras.utils.get_file(
    'flower_photos',
    'https://storage.googleapis.com/download.tensorflow.org/example_images/flower_photos.tgz',
    untar=True)

img_gen = tf.keras.preprocessing.image.ImageDataGenerator(
    rescale=1./255,
    horizontal_flip=True,
)

train_ds = img_gen.flow_from_directory(flowers, batch_size=BATCH_SIZE, shuffle=True, class_mode='sparse')
num_classes = 5

model = tf.keras.Sequential([
  tf.keras.layers.Conv2D(16, 3, padding='same', activation='relu', input_shape=(256, 256, 3)),
  tf.keras.layers.MaxPooling2D(),
  tf.keras.layers.Conv2D(32, 3, padding='same', activation='relu'),
  tf.keras.layers.MaxPooling2D(),
  tf.keras.layers.Conv2D(64, 3, padding='same', activation='relu'),
  tf.keras.layers.MaxPooling2D(),
  tf.keras.layers.Flatten(),
  tf.keras.layers.Dense(128, activation='relu'),
  tf.keras.layers.Dense(num_classes)
])

model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True))

epochs=10
history = model.fit(
  train_ds,
  epochs=epochs
)

Found 3670 images belonging to 5 classes.
Epoch 1/10
 6/58 [==>...........................] - ETA: 3:02 - loss: 2.0608

If you wrap flow_from_directory with tf.data.Dataset.from_generator like this:

train_ds = tf.data.Dataset.from_generator(
    lambda: img_gen.flow_from_directory(flowers, batch_size=BATCH_SIZE, shuffle=True, class_mode='sparse'),
    output_types=(tf.float32, tf.float32))

You will notice that the progress bar looks like this because steps_per_epoch has not been explicitly defined:

Epoch 1/10
Found 3670 images belonging to 5 classes.
     29/Unknown - 104s 4s/step - loss: 2.0364

And if you add this parameter, you will see the steps in the progress bar:

history = model.fit(
  train_ds,
  steps_per_epoch = len(from_directory),
  epochs=epochs
)

Found 3670 images belonging to 5 classes.
Epoch 1/10
 3/58 [>.............................] - ETA: 3:19 - loss: 4.1357

Finally, to your question:

How to use this generator correctly with function fit to have all data in my training set, including original, non-augmented images and augmented images, and to cycle through it several times/step?

You can simply increase the steps_per_epoch beyond number of samples // batch_size by multiplying by some factor:

history = model.fit(
  train_ds,
  steps_per_epoch = len(from_directory)*2,
  epochs=epochs
)

Found 3670 images belonging to 5 classes.
Epoch 1/10
  1/116 [..............................] - ETA: 12:11 - loss: 1.5885

Now instead of 58 steps per epoch you have 116.

How to correctly use ImageDataGenerator in Keras?

Related

Recent Posts