Split .tfrecords file into many .tfrecords files

by

In tensorflow 2.0.0, this will work:

import tensorflow as tf

raw_dataset = tf.data.TFRecordDataset("input_file.tfrecord")

shards = 10

for i in range(shards):
    writer = tf.data.experimental.TFRecordWriter(f"output_file-part-{i}.tfrecord")
    writer.write(raw_dataset.shard(shards, i))

More Related Contents:

  1. Numpy to TFrecords: Is there a more simple way to handle batch inputs from tfrecords?
  2. Is there a simpler way to handle batch inputs from tfrecords?
  3. how to convert numpy to tfrecords and then generate batches?
  4. TensorFlow: training on my own image
  5. Tensorflow : logits and labels must have the same first dimension
  6. Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2
  7. How to tell if tensorflow is using gpu acceleration from inside python shell?
  8. Using a pre-trained word embedding (word2vec or Glove) in TensorFlow
  9. Tensorflow – ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type float)
  10. What is the difference between ‘SAME’ and ‘VALID’ padding in tf.nn.max_pool of tensorflow?
  11. Could not find a version that satisfies the requirement tensorflow
  12. Tensorflow read images with labels
  13. TensorFlow version 1.0.0-rc2 on Windows: “OpKernel (‘op: “BestSplits” device_type: “CPU”‘) for unknown op: BestSplits” with test code
  14. Which seeds have to be set where to realize 100% reproducibility of training results in tensorflow?
  15. Difference between Variable and get_variable in TensorFlow
  16. Tensorflow: How to get a tensor by name?
  17. TypeError: __array__() takes 1 positional argument but 2 were given (Image classification Keras)
  18. Tensorflow 2.0 – AttributeError: module ‘tensorflow’ has no attribute ‘Session’
  19. “zsh: illegal hardware instruction python” when installing Tensorflow on macbook pro M1 [duplicate]
  20. Illegal instruction (core dumped) after running import tensorflow
  21. ‘Library not loaded: @rpath/libcudart.7.5.dylib’ TensorFlow Error on Mac
  22. Why is Tensorflow not recognizing my GPU after conda install?
  23. Keras: .predict returns percentages instead of classes
  24. Keras Custom loss function to pass arguments other than y_true and y_pred
  25. Unknown initializer: GlorotUniform when loading Keras model
  26. Cartesian Product in Tensorflow
  27. How to fix “AttributeError: module ‘tensorflow’ has no attribute ‘get_default_graph'”?
  28. What is right batch normalization function in Tensorflow?
  29. How do I plot a Keras/Tensorflow subclassing API model?
  30. Strange behaviour of the loss function in keras model, with pretrained convolutional base

Leave a Comment