Spark Unable to find JDBC Driver

by

This person was having similar issue: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-use-DataFrame-with-MySQL-td22178.html

Have you updated your connector drivers to the most recent version? Also did you specify the driver class when you called load()?

Map<String, String> options = new HashMap<String, String>();
options.put("url", "jdbc:mysql://localhost:3306/video_rcmd?user=root&password=123456");
options.put("dbtable", "video");
options.put("driver", "com.mysql.cj.jdbc.Driver"); //here
DataFrame jdbcDF = sqlContext.load("jdbc", options);

In spark/conf/spark-defaults.conf, you can also set spark.driver.extraClassPath and spark.executor.extraClassPath to the path of your MySql driver .jar

More Related Contents:

  1. How to use SQL query to define table in dbtable?
  2. SPARK SQL – update MySql table using DataFrames and JDBC
  3. Partitioning in spark while reading from RDBMS via JDBC
  4. Does spark predicate pushdown work with JDBC?
  5. How to optimize partitioning when migrating data from JDBC source?
  6. What is the meaning of partitionColumn, lowerBound, upperBound, numPartitions parameters?
  7. Finding duplicates from large data set using Apache Spark
  8. How to make good reproducible Apache Spark examples
  9. How to select the first row of each group?
  10. Querying Spark SQL DataFrame with complex types
  11. How to convert rdd object to dataframe in spark
  12. How to define and use a User-Defined Aggregate Function in Spark SQL?
  13. While writing to hdfs path getting error java.io.IOException: Failed to rename
  14. Multiple Aggregate operations on the same column of a spark dataframe
  15. How to access element of a VectorUDT column in a Spark DataFrame?
  16. Caused by: java.lang.NullPointerException at org.apache.spark.sql.Dataset
  17. How to save DataFrame directly to Hive?
  18. About how to add a new column to an existing DataFrame with random values in Scala
  19. Pyspark ‘NoneType’ object has no attribute ‘_jvm’ error
  20. Cast column containing multiple string date formats to DateTime in Spark
  21. spark broadcast variable Map giving null value
  22. How to explode multiple columns of a dataframe in pyspark
  23. What is the maximum size for a broadcast object in Spark?
  24. How to write unit tests in Spark 2.0+?
  25. PySpark: multiple conditions in when clause
  26. How does createOrReplaceTempView work in Spark?
  27. PySpark – get row number for each row in a group
  28. Spark Row to JSON
  29. Exploding nested Struct in Spark dataframe
  30. Retain keys with null values while writing JSON in spark

Leave a Comment