How to skip CSV header in Hive External Table?

by

As of Hive v0.13.0, you can use skip.header.line.count table property:

create external table testtable (name string, message string)
row format delimited 
fields terminated by '\t' 
lines terminated by '\n' 
location '/testtable'
TBLPROPERTIES ("skip.header.line.count"="1");

Use ALTER TABLE for an existing table:

ALTER TABLE tablename
SET TBLPROPERTIES ("skip.header.line.count"="1");

Please note that while it works it comes with its own issues. When there is more than one output file generated i.e. reducers are greater than 1, it skips the first record for each and every file which might not necessarily be the desired behaviour.

More Related Contents:

  1. How to set variables in HIVE scripts
  2. How to create n number of external tables with a single hdfs path using Hive
  3. Create Table in Hive with one file
  4. Specify minimum number of generated files from Hive insert
  5. Hive Explode / Lateral View multiple arrays
  6. java.lang.RuntimeException:Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
  7. When creating an external table in hive can I point the location to specific files in a directory?
  8. How to update partition metadata in Hive , when partition data is manualy deleted from HDFS
  9. Update , SET option in Hive
  10. SQL split comma separated row [duplicate]
  11. HIVE select count(*) non null returns higher value than select count(*)
  12. How to create SparkSession with Hive support (fails with “Hive classes are not found”)?
  13. How to control partition size in Spark SQL
  14. What is the difference between partitioning and bucketing a table in Hive ?
  15. How do I output the results of a HiveQL query to CSV?
  16. How to export a Hive table into a CSV file?
  17. How to connect to remote hive server from spark [duplicate]
  18. Sqoop import : composite primary key and textual primary key
  19. Use collect_list and collect_set in Spark SQL
  20. Hive query performance for high cardinality field
  21. java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
  22. How to convert .txt file to Hadoop’s sequence file format
  23. Multiple Spark applications with HiveContext
  24. Find last day of a month in Hive
  25. Hive – Unpivot functionality in hive
  26. Overwrite only some partitions in a partitioned spark Dataset
  27. Export as csv in beeline hive
  28. How do I limit the number of rows per field value in SQL?
  29. Can Hive recursively descend into subdirectories without partitions or editing hive-site.xml?
  30. How to calculate Median in spark sqlContext for column of data type double

Leave a Comment