User reading a large size json file (280GB) using sqlContext.json.
val loadJsonFile = sqlContext.jsonFile("/data/filename.txt")
Error
org.apache.spark.SparkException: Job aborted
Caused by: java.lang.OutOfMemoryError: Java heap space
Executed application with an increased driver memory for fix the issue
spark-shell --driver-memory 3g
val loadJsonFile = sqlContext.jsonFile("/data/filename.txt")
Error
org.apache.spark.SparkException: Job aborted
Caused by: java.lang.OutOfMemoryError: Java heap space
Executed application with an increased driver memory for fix the issue
spark-shell --driver-memory 3g