The whole process is simplied using the Dataset API. Here are both the parts: (1): Convert numpy array to tfrecords and (2): read the tfrecords to generate batches. 1. Creation of tfrecords from a numpy array: Example arrays: inputs = np.random.normal(size=(5, 32, 32, 3)) labels = np.random.randint(0,2,size=(5,)) def npy_to_tfrecords(inputs, labels, filename): with tf.io.TFRecordWriter(filename) as writer: … Read more
I hope this helps: filename_queue = tf.train.string_input_producer([‘/Users/HANEL/Desktop/tf.png’]) # list of files to read reader = tf.WholeFileReader() key, value = reader.read(filename_queue) my_img = tf.image.decode_png(value) # use decode_png or decode_jpeg decoder based on your files. init_op = tf.initialize_all_variables() with tf.Session() as sess: sess.run(init_op) # Start populating the filename queue. coord = tf.train.Coordinator() threads = tf.train.start_queue_runners(coord=coord) for i … Read more
In tensorflow 2.0.0, this will work: import tensorflow as tf raw_dataset = tf.data.TFRecordDataset(“input_file.tfrecord”) shards = 10 for i in range(shards): writer = tf.data.experimental.TFRecordWriter(f”output_file-part-{i}.tfrecord”) writer.write(raw_dataset.shard(shards, i))
The whole process is simplied using the Dataset API. Here are both the parts: (1): Convert numpy array to tfrecords and (2): read the tfrecords to generate batches. 1. Creation of tfrecords from a numpy array: Example arrays: inputs = np.random.normal(size=(5, 32, 32, 3)) labels = np.random.randint(0,2,size=(5,)) def npy_to_tfrecords(inputs, labels, filename): with tf.io.TFRecordWriter(filename) as writer: … Read more
The whole process is simplied using the Dataset API. Here are both the parts: (1): Convert numpy array to tfrecords and (2,3,4): read the tfrecords to generate batches. 1. Creation of tfrecords from a numpy array: def npy_to_tfrecords(…): # write records to a tfrecords file writer = tf.python_io.TFRecordWriter(output_file) # Loop through all the features you … Read more