How to explode an array into multiple columns in Spark

by

Use apply:

import org.apache.spark.sql.functions.col

df.select(
  col("id") +: (0 until 3).map(i => col("DataArray")(i).alias(s"col$i")): _*
)

More Related Contents:

  1. Encoder error while trying to map dataframe row to updated row
  2. Spark Scala: How to convert Dataframe[vector] to DataFrame[f1:Double, …, fn: Double)]
  3. How to define schema for custom type in Spark SQL?
  4. Specifying the filename when saving a DataFrame as a CSV [duplicate]
  5. NullPointerException in Scala Spark, appears to be caused be collection type?
  6. What does setMaster `local[*]` mean in spark?
  7. Spark unionAll multiple dataframes
  8. How can I pass extra parameters to UDFs in Spark SQL?
  9. How to split a dataframe into dataframes with same column values?
  10. Spark performance for Scala vs Python
  11. How to use regex to include/exclude some input files in sc.textFile?
  12. How to define a custom aggregation function to sum a column of Vectors?
  13. Spark losing println() on stdout
  14. How to write spark streaming DF to Kafka topic
  15. Stackoverflow due to long RDD Lineage
  16. Better way to convert a string field into timestamp in Spark
  17. Spark UDAF with ArrayType as bufferSchema performance issues
  18. Apache Spark how to append new column from list/array to Spark dataframe
  19. How to find spark RDD/Dataframe size?
  20. How to get ID of a map task in Spark?
  21. Spark: produce RDD[(X, X)] of all possible combinations from RDD[X]
  22. Encode an ADT / sealed trait hierarchy into Spark DataSet column
  23. Provide schema while reading csv file as a dataframe in Scala Spark
  24. Spark RDD default number of partitions
  25. Why does Spark RDD partition has 2GB limit for HDFS?
  26. Spark : how to run spark file from spark shell
  27. How can I connect to a postgreSQL database into Apache Spark using scala?
  28. Spark Dataframe Random UUID changes after every transformation/action
  29. Spark column string replace when present in other column (row)
  30. How to vectorize DataFrame columns for ML algorithms?

Leave a Comment