Spark java.lang.OutOfMemoryError: Java heap space
I have a few suggestions: If your nodes are configured to have 6g maximum for Spark (and are leaving a little for other processes), then use 6g rather than 4g, spark.executor.memory=6g. Make sure you’re using as much memory as possible by checking the UI (it will say how much mem you’re using) Try using more … Read more