Inspect Parquet from command line

by

You can use parquet-tools with the command cat and the --json option in order to view the files without a local copy and in the JSON format.

Here is an example:

parquet-tools cat --json hdfs://localhost/tmp/save/part-r-00000-6a3ccfae-5eb9-4a88-8ce8-b11b2644d5de.gz.parquet

This prints out the data in JSON format:

{"name":"gil","age":48,"city":"london"}
{"name":"jane","age":30,"city":"new york"}
{"name":"jordan","age":18,"city":"toronto"}

Disclaimer: this was tested in Cloudera CDH 5.12.0

More Related Contents:

  1. Spark Dataframe validating column names for parquet writes
  2. Reading DataFrame from partitioned parquet file
  3. How do you control the size of the output file?
  4. Is it better to have one large parquet file or lots of smaller parquet files?
  5. Why are Spark Parquet files for an aggregate larger than the original?
  6. Reading parquet files from multiple directories in Pyspark
  7. How to read a Parquet file into Pandas DataFrame?
  8. What are the differences between feather and parquet?
  9. Spark lists all leaf node even in partitioned data
  10. What are the pros and cons of parquet format compared to other formats?
  11. How to read partitioned parquet files from S3 using pyarrow in python
  12. How to convert a csv file to parquet

Leave a Comment