Spark Row to JSON

by

Spark 2.1 should have native support for this use case (see #15354).

import org.apache.spark.sql.functions.to_json
df.select(to_json(struct($"c1", $"c2", $"c3")))

More Related Contents:

  1. Read multiline JSON in Apache Spark
  2. how to convert json string to dataframe on spark
  3. Reading JSON with Apache Spark – `corrupt_record`
  4. Spark 2.0 Dataset vs DataFrame
  5. Spark – load CSV file as DataFrame?
  6. What JSON library to use in Scala? [closed]
  7. How to define partitioning of DataFrame?
  8. How to parse JSON in Scala using standard Scala classes?
  9. SparkSQL: How to deal with null values in user defined function?
  10. How to aggregate values into collection after groupBy?
  11. Dropping a nested column from Spark DataFrame
  12. How to read records in JSON format from Kafka using Structured Streaming?
  13. Spark Dataframe :How to add a index Column : Aka Distributed Data Index
  14. Filling gaps in timeseries Spark
  15. How to use COGROUP for large datasets
  16. How to turn json to case class when case class has only one field
  17. Spark Dataframes UPSERT to Postgres Table
  18. What are possible reasons for receiving TimeoutException: Futures timed out after [n seconds] when working with Spark [duplicate]
  19. get TopN of all groups after group by using Spark DataFrame
  20. Why does join fail with “java.util.concurrent.TimeoutException: Futures timed out after [300 seconds]”?
  21. Create new Dataframe with empty/null field values
  22. Perform a typed join in Scala with Spark Datasets
  23. DataFrame-ified zipWithIndex
  24. Derive multiple columns from a single column in a Spark DataFrame
  25. Spark: Add column to dataframe conditionally
  26. What is going wrong with `unionAll` of Spark `DataFrame`?
  27. How to sort by column in descending order in Spark SQL?
  28. Access Array column in Spark
  29. Why is join not possible after show operator?
  30. Difference between two rows in Spark dataframe

Leave a Comment