在spark中遇到org.apache.spark.SparkException:该如何解决问题

[weixin_38069804]

运行大量数据集就会遇到问题，怎么办？？？   Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 33322.0 failed 1 times, most recent failure: Lost task 0.0 in stage 33322.0 (TID 925, localhost): ExecutorLostFailure (executor driver exited caused by one of the running tasks) Reason: Executor heartbeat timed out after 159469 ms Driver stacktrace:     at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1431)     at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1419)     at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1418)     at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)     at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)     at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1418)     at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:799)     at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:799)     at scala.Option.foreach(Option.scala:236)     at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:799)     at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1640)     at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1599)     at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1588)     at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)     at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:620)     at org.apache.spark.SparkContext.runJob(SparkContext.scala:1832)     at org.apache.spark.SparkContext.runJob(SparkContext.scala:1952)     at org.apache.spark.rdd.RDD$$anonfun$reduce$1.apply(RDD.scala:1025)     at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)     at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:111)     at org.apache.spark.rdd.RDD.withScope(RDD.scala:316)     at org.apache.spark.rdd.RDD.reduce(RDD.scala:1007)     at org.apache.spark.graphx.impl.VertexRDDImpl.count(VertexRDDImpl.scala:90)     at com.dbcs.CommunityUtil.communityInfluence(CommunityUtil.scala:427)     at com.dbcs.CommunityUtil$$anonfun$execute$1.apply(CommunityUtil.scala:542)     at com.dbcs.CommunityUtil$$anonfun$execute$1.apply(CommunityUtil.scala:530)     at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)     at scala.collection.mutable.ArrayOps$ofLong.foreach(ArrayOps.scala:168)     at com.dbcs.CommunityUtil.execute(CommunityUtil.scala:530)     at com.dbcs.LouvainHarness.run(LouvainHarness.scala:82)     at com.dbcs.Main$.main(Main.scala:137)     at com.dbcs.Main.main(Main.scala) 17/09/21 11:32:28 WARN QueuedThreadPool: 5 threads could not be stopped