To do it in a spark-shell (Scala), you can use spark.time()
.
See another response by me: https://stackoverflow.com/a/50289329/3397114
df = sqlContext.sql(query)
spark.time(df.show())
The output would be:
+----+----+
|col1|col2|
+----+----+
|val1|val2|
+----+----+
Time taken: xxx ms
Related: On Measuring Apache Spark Workload Metrics for Performance Troubleshooting.