Split tf tf.data.Dataset tuple into several datasets
I have a tf.data.Dataset with the following shape:
<ConcatenateDataset shapes: ((None, None, 12), (None, 5)), types: (tf.float64, tf.float64)>
Can I split this Dataset to get two datasets looking like this:
<Dataset shapes: (None, None, 12), types: tf.float64>
<Dataset shapes: (None, 5), types: tf.float64>
Solution 1:
You can use map
function to split them.
Demo:
import tensorflow as tf
# Create a random tensorflow dataset.
dataset1 = tf.data.Dataset.from_tensor_slices((tf.random.uniform([40, 10, 12]), tf.random.uniform([40, 5]))).batch(16)
dataset2 = tf.data.Dataset.from_tensor_slices((tf.random.uniform([40, 12, 12]), tf.random.uniform([40, 5]))).batch(16)
dataset = dataset1.concatenate(dataset2)
dataset
>> <ConcatenateDataset shapes: ((None, None, 12), (None, 5)), types: (tf.float32, tf.float32)>
In order to split:
data = dataset.map(lambda x, y: x)
labels = dataset.map(lambda x, y: y)