ERROR CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM

wcx945123 2018-04-21 10:28:08

程序大概就是通过spark读取hbase中的数据，对数据进行处理以后，将处理后的数据存到hbase的另一张表中
但是在Executor的日志中有报错，但是Driver里面没有，大家看看是什么原因
错误信息：
8/04/21 21:30:49 INFO ZooKeeper: Session: 0x161cfd1f3362a1b closed
18/04/21 21:30:49 INFO ClientCnxn: EventThread shut down
18/04/21 21:30:49 INFO SparkHadoopMapRedUtil: No need to commit output of task because needsTaskCommit=false: attempt_20180421213012_0000_m_000000_0
18/04/21 21:30:49 INFO Executor: Finished task 0.0 in stage 0.0 (TID 0). 1734 bytes result sent to driver
18/04/21 21:30:49 INFO CoarseGrainedExecutorBackend: Driver commanded a shutdown
18/04/21 21:30:49 ERROR CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM

代码如下：

Driver日志：

Executor日志：

...全文

4096 1 打赏收藏转发到动态举报

写回复

1 条回复

切换为时间正序

请发表友善的回复…

发表回复

house1978 2018-04-23

打赏
举报

回复

看日志y应该是Driver挂了

Blind equalization (BE) technology is a new adaptive technology. BE only uses the prior information of received signals to equalize the channel characteristics, so train- ing sequence is not needed. The output sequence is close to the transmitted sequence. Inter-symbol interference is overcome effectively and the quality of communication is improved by BE. Neural network (NN) is a cross-edge discipline of neural sci- ence, information science, and computer science. NN has the following abilities such as massively parallel, distributed storage and processing, self-organizing, adaptive, self-learning, and highly fault tolerant. The combination of NN and BE can improve convergence performance and equalization effect. The combination of NN and BE is a hot research topic in communication, signal, and information processing. It has important theoretical significance and practical value. This book was written by the author and his doctors and masters, namely, Yun- shan Sun, Xiaoqin Zhang, Rui Lu, Xiaowei Niu, Yu Bai, Haiqing Cheng, Fengmei Jia, Yanling Jia, Yuan Li, Yong Liu, and Yanqi Kang. This book was also suppor- ted by the following research funds: Shanxi Province Natural Science Fund project “Mobile communication blind equalizer” (20011035), China Postdoctoral Project Sci- ence Foundation “Fuzzy neural network used in blind equalization technology” (20060390170), Shanxi Provincial Natural Science Foundation Project “The blind equalization technique based on neural network” (20051038), Tianjin High School Science and Technology Fund Project “Research on evolution neural network blind equalization algorithm” (20060610), and “Medical image blind restoration algorithm based on Zigzag transform” (20110709) Tianjin Research Program of Application Foundation and Advanced Technology “Research on Integration issues of Medical MRI Image Three-Dimensional Implementation” (13JCYBJC15600), and “Low-dose Medical CT Image Blind Restoration Reconstruction Algorithm based on Bayesian Compressed Sensing” (16JCYBJC28800). This book was translated by Yu Bai (Chapters 1–2), Xiaoqin Zhang (Chapters 3–5), and Yunshan Sun (Chapters 6–8). The NN and BE algorithms are combined, and the new neural network is system- atically studied. Some research results have been published in important academic journals and also in international and domestic conferences. This book is a sum- mary of the results of these studies. The latest research trends and frontiers in neural network blind equalization algorithm in domestic and international are reflected basically in the book. This book is divided into eight chapters. The first chapter is introduction. The significance and application fields of blind equalization are given. The classification and research status of NN blind equalization algorithms are summarized. The research background and main work in this book are pointed out. The second chapter describes the fundamental theory of NN. The concept, struc- ture, algorithms, and equalization criterion of blind equalization are introduced. The fundamental principles and learning methods of NN blind equalization algorithm are elaborated. The evaluation of blind equalization algorithm is analyzed. The third chapter is about the research of blind equalization algorithms based on FFNN. BE algorithm based on feed-forward neural networks (four-layer, three-layer, and five-layer) are studied. BE algorithms based on momentum term, time-varying momentum term, and time-varying step are studied, respectively. The fourth chapter is about the research of blind equalization algorithms based on FBNN. BE algorithms based on bilinear recurrent NN, diagonal recurrent NN, and quasi-diagonal recurrent NN are studied, respectively. The blind equalization algorithms based on mean square error nonlinear function with time-varying step diagonal recurrent NN and with time-varying step quasi-diagonal recurrent NN are studied, respectively. The fifth chapter is the research of blind equalization algorithms based on FNN. The blind equalization algorithm based on fuzzy NN filter, fuzzy NN controller, and fuzzy NN classifier is studied, respectively. The sixth chapter is blind equalization algorithm based on evolutionary neural network. The blind equalization algorithms based on optimization NN weights and structure optimized by genetic algorithm are studied, respectively. The seventh chapter describes blind equalization algorithm based on wavelet NN. The blind equalization algorithms based on feed-forward NN and feedback wavelet NN are studied, respectively. The eighth chapter provides the application of blind equalization algorithm neural network in medical image processing. The application of blind equalization in CT image restoration is mainly studied. We would like to express our sincere thanks to Professor Jianfu Teng, the doc- toral tutor in Tianjin University; Professor Dingguo Sha, the doctoral tutor in Beijing Institute of Technology; and Professor Huakui Wang, the doctoral tutor in Taiyuan University of Technology for their help and support. We wish to thank Yanqin Li who is responsible for proofreading and revision. We are also grateful to the scholars at home and abroad whose published literature are referred to in this book. Due to the limited level of the author, there are some oversights and inadequacies inevitably in the book, the readers are welcome to give suggestions.

PyTorch版的YOLOv5是轻量而高性能的实时目标检测方法。利用YOLOv5训练完自己的数据集后，如何向大众展示并提供落地的服务呢？本课程将提供相应的解决方案，具体讲述如何使用Web应用程序框架Flask进行YOLOv5的Web应用部署。用户可通过客户端浏览器上传图片，经服务器处理后返回图片检测数据并在浏览器中绘制检测结果。本课程的YOLOv5使用ultralytics/yolov5，在Ubuntu系统上做项目演示，并提供在Windows系统上的部署方式文档。本项目采取前后端分离的系统架构和开发方式，减少前后端的耦合。课程包括：YOLOv5的安装、 Flask的安装、YOLOv5的检测API接口python代码、 Flask的服务程序的python代码、前端html代码、CSS代码、Javascript代码、系统部署演示、生产系统部署建议等。本人推出了有关YOLOv5目标检测的系列课程。请持续关注该系列的其它视频课程，包括：《YOLOv5(PyTorch)目标检测实战：训练自己的数据集》Ubuntu系统 https://edu.csdn.net/course/detail/30793 Windows系统 https://edu.csdn.net/course/detail/30923 《YOLOv5(PyTorch)目标检测：原理与源码解析》https://edu.csdn.net/course/detail/31428 《YOLOv5(PyTorch)目标检测实战：Flask Web部署》https://edu.csdn.net/course/detail/31087 《YOLOv5(PyTorch)目标检测实战：TensorRT加速部署》https://edu.csdn.net/course/detail/32303

问题现象：spark应用能正常计算得到结果，但是查看executor有很多是dead，如下图查看executorstderr日志发现有错误日志：ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM 除此之外没有其他任何错误信息问题原因: 由于使能动态资源分配executors数(spark.dy...

Spark提交任务执行遇到如下错误： 1.命令行错误： 2.yarn 日志错误：（yarn日志不知如何获取的请参考上一篇博客 yarn日志打印） ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL 15: SIGTERM 19/12/05 11:03:53 INFO netty.NettyBlockTransferService...

1,261

社区成员

1,169

社区内容

发帖

与我相关

我的任务

社区管理员

加入社区

近7日
近30日
至今

加载中

查看更多榜单

社区公告

暂无公告

试试用AI创作助手写篇文章吧

+ 用AI写文章