求问：集群启动成功，但是LiveNode只有一个而且在两台机器中来回变……

Ato42 2014-10-20 04:44:41
我的集群里有两台虚拟主机：master和node1，通过start-all.ssh启动集群成功了，但是在master：50070页面中只显示LiveNode数为1，点进去刷新的话会发现这个LiveNode一会儿是master，一会儿是node1……求大神解答这是怎么回事啊……
...全文
1601 17 打赏 收藏 转发到动态 举报
写回复
用AI写文章
17 条回复
切换为时间正序
请发表友善的回复…
发表回复
chenghu1982 2015-09-25
  • 打赏
  • 举报
 回复
以前遇到多次,主机原来是单机版改成集群后就这问题,需要重新bin/hdfs namenode -format 就ok了
曹宇 2014-10-30
  • 打赏
  • 举报
 回复
Last Contact说明这个节点的datanode 宕了 或者说网络问题等等 反正是master找不到它了 至于相同的dfs dir 那是没问题的. 目录是相同的 但是是两个机器啊 大家都是这么配置的. 你那个livenode来回变应该是连接问题 看看ssh ping 网络环境是否正常 看jps datanode是否挂掉 看datanode日志 解决不了帮顶了
skyWalker_ONLY 2014-10-21
  • 打赏
  • 举报
 回复
两个datanode中dfs.data.dir所指向路径中存的数据相同的时候，这句话指的是路径相同还是目录下的文件内容相同？
Ato42 2014-10-21
  • 打赏
  • 举报
 回复
引用 12 楼 wulinshishen 的回复:
[quote=引用 9 楼 u011136825 的回复:] [quote=引用 8 楼 sky_walker85 的回复:] 应该是NameNode在启动的时候报错了。你先执行一下hadoop fsck检查一下HDFS文件系统，或者执行hadoop dfsadmin -report查看一下块。在启动完成后，先不要立即在web查看DataNode，在退出safe模式后再查看
额……我刚才试了下，好像是因为两个datanode的hdfs-site.xml中，dir.data.dir所指向路径中数据相同造成的……改掉之后就没问题了……但是我不懂为什么……为什么数据不能相同呢？[/quote] DataNode的上的配置应该都是一样的，如果是两个DataNode的话LiveNode正常情况应该是2[/quote] 是两个datanode，但是当两个datanode中dfs.data.dir所指向路径中存的数据相同的时候，LiveNode只会显示1，这个1一会儿是master，一会儿是node1，而且LastContact数字在增加。当我把node1中dfs.data.dir指向路径中的数据清空后就好了，变成了2个LiveNode，Contact也正常了
Ato42 2014-10-21
  • 打赏
  • 举报
 回复
引用 11 楼 sky_walker85 的回复:
[quote=引用 9 楼 u011136825 的回复:] [quote=引用 8 楼 sky_walker85 的回复:] 应该是NameNode在启动的时候报错了。你先执行一下hadoop fsck检查一下HDFS文件系统，或者执行hadoop dfsadmin -report查看一下块。在启动完成后，先不要立即在web查看DataNode，在退出safe模式后再查看
额……我刚才试了下，好像是因为两个datanode的hdfs-site.xml中，dir.data.dir所指向路径中数据相同造成的……改掉之后就没问题了……但是我不懂为什么……为什么数据不能相同呢？[/quote] 如果是dfs.data.dir的话，当然可以相同了。你说两个datanode，不是一个吗？你slaves文件中是什么吗[/quote] slaves中是master node1 我把master作为namenode和datanode，node1作为secondarynamenode和datanode
人生偌只如初见 2014-10-21
  • 打赏
  • 举报
 回复
引用 9 楼 u011136825 的回复:
[quote=引用 8 楼 sky_walker85 的回复:] 应该是NameNode在启动的时候报错了。你先执行一下hadoop fsck检查一下HDFS文件系统，或者执行hadoop dfsadmin -report查看一下块。在启动完成后，先不要立即在web查看DataNode，在退出safe模式后再查看
额……我刚才试了下，好像是因为两个datanode的hdfs-site.xml中，dir.data.dir所指向路径中数据相同造成的……改掉之后就没问题了……但是我不懂为什么……为什么数据不能相同呢？[/quote] DataNode的上的配置应该都是一样的，如果是两个DataNode的话LiveNode正常情况应该是2
skyWalker_ONLY 2014-10-21
  • 打赏
  • 举报
 回复
引用 9 楼 u011136825 的回复:
[quote=引用 8 楼 sky_walker85 的回复:] 应该是NameNode在启动的时候报错了。你先执行一下hadoop fsck检查一下HDFS文件系统，或者执行hadoop dfsadmin -report查看一下块。在启动完成后，先不要立即在web查看DataNode，在退出safe模式后再查看
额……我刚才试了下，好像是因为两个datanode的hdfs-site.xml中，dir.data.dir所指向路径中数据相同造成的……改掉之后就没问题了……但是我不懂为什么……为什么数据不能相同呢？[/quote] 如果是dfs.data.dir的话，当然可以相同了。你说两个datanode，不是一个吗？你slaves文件中是什么吗
skyWalker_ONLY 2014-10-21
  • 打赏
  • 举报
 回复
dir.data.dir是什么？楼主拼错了吧
Ato42 2014-10-21
  • 打赏
  • 举报
 回复
引用 8 楼 sky_walker85 的回复:
应该是NameNode在启动的时候报错了。你先执行一下hadoop fsck检查一下HDFS文件系统，或者执行hadoop dfsadmin -report查看一下块。在启动完成后，先不要立即在web查看DataNode，在退出safe模式后再查看
额……我刚才试了下，好像是因为两个datanode的hdfs-site.xml中，dir.data.dir所指向路径中数据相同造成的……改掉之后就没问题了……但是我不懂为什么……为什么数据不能相同呢？
skyWalker_ONLY 2014-10-20
  • 打赏
  • 举报
 回复
应该是NameNode在启动的时候报错了。你先执行一下hadoop fsck检查一下HDFS文件系统，或者执行hadoop dfsadmin -report查看一下块。在启动完成后，先不要立即在web查看DataNode，在退出safe模式后再查看
Ato42 2014-10-20
  • 打赏
  • 举报
 回复
core-site.xml 

<configuration>
<property>
    <name>fs.default.name</name>
<value>hdfs://master:9000</value>
  </property>
  <property>
    <name>fs.checkpoint.dir</name>
    <value>/data/hadoop/hdfs/namesecondary</value>
  </property>
  <property>
    <name>fs.checkpoint.period</name>
<value>1800</value>
  </property>
  <property>
    <name>fs.checkpoint.size</name>
<value>33554432</value>
  </property>
  <property>
    <name>io.compression.codecs</name>
<value>org.apache.hadoop.io.compress.DefaultCodec,org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.BZip2Codec</value>
  </property>
  <property>
    <name>fs.trash.interval</name>
    <value>1440</value>
  </property>
</configuration>
Ato42 2014-10-20
  • 打赏
  • 举报
 回复
引用 4 楼 sky_walker85 的回复:
把配置贴出来看看吧，还有后台NameNode和DataNode日志的输出信息
master 的datanode日志 

[code=text]
************************************************************/
2014-10-20 03:22:28,760 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting DataNode
STARTUP_MSG:   host = master/192.168.1.100
STARTUP_MSG:   args = []
STARTUP_MSG:   version = 1.0.3
STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r 1335192; compiled by 'hortonfo' on Tue May  8 20:31:25 UTC 2012
************************************************************/
2014-10-20 03:22:29,015 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2014-10-20 03:22:29,032 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered.
2014-10-20 03:22:29,038 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2014-10-20 03:22:29,038 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode metrics system started
2014-10-20 03:22:29,192 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi registered.
2014-10-20 03:22:29,754 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Registered FSDatasetStatusMBean
2014-10-20 03:22:29,839 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Opened info server at 50010
2014-10-20 03:22:29,842 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Balancing bandwith is 1048576 bytes/s
2014-10-20 03:22:29,984 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2014-10-20 03:22:30,250 INFO org.apache.hadoop.http.HttpServer: Added global filtersafety (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
2014-10-20 03:22:30,260 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: dfs.webhdfs.enabled = false
2014-10-20 03:22:30,260 INFO org.apache.hadoop.http.HttpServer: Port returned by webServer.getConnectors()[0].getLocalPort() before open() is -1. Opening the listener on 50075
2014-10-20 03:22:30,261 INFO org.apache.hadoop.http.HttpServer: listener.getLocalPort() returned 50075 webServer.getConnectors()[0].getLocalPort() returned 50075
2014-10-20 03:22:30,261 INFO org.apache.hadoop.http.HttpServer: Jetty bound to port 50075
2014-10-20 03:22:30,261 INFO org.mortbay.log: jetty-6.1.26
2014-10-20 03:22:30,292 WARN org.mortbay.log: Can't reuse /tmp/Jetty_0_0_0_0_50075_datanode____hwtdwq, using /tmp/Jetty_0_0_0_0_50075_datanode____hwtdwq_2018615579564531004
2014-10-20 03:22:30,781 INFO org.mortbay.log: Started SelectChannelConnector@0.0.0.0:50075
2014-10-20 03:22:30,784 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source jvm registered.
2014-10-20 03:22:30,785 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source DataNode registered.
2014-10-20 03:22:31,873 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source RpcDetailedActivityForPort50020 registered.
2014-10-20 03:22:31,874 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source RpcActivityForPort50020 registered.
2014-10-20 03:22:31,876 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: dnRegistration = DatanodeRegistration(master:50010, storageID=DS-1696742302-192.168.1.100-50010-1413704373290, infoPort=50075, ipcPort=50020)
2014-10-20 03:22:31,884 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Starting asynchronous block report scan
2014-10-20 03:22:31,892 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Finished asynchronous block report scan in 8ms
2014-10-20 03:22:31,923 INFO org.apache.hadoop.ipc.Server: Starting SocketReader
2014-10-20 03:22:31,924 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(192.168.1.100:50010, storageID=DS-1696742302-192.168.1.100-50010-1413704373290, infoPort=50075, ipcPort=50020)In DataNode.run, data = FSDataset{dirpath='/data/hadoop/hdfs/data/current'}
2014-10-20 03:22:31,924 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2014-10-20 03:22:31,925 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 50020: starting
2014-10-20 03:22:31,942 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 50020: starting
2014-10-20 03:22:31,944 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on 50020: starting
2014-10-20 03:22:31,944 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: using BLOCKREPORT_INTERVAL of 3600000msec Initial delay: 0msec
2014-10-20 03:22:31,947 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Reconciled asynchronous block report against current state in 1 ms
2014-10-20 03:22:31,956 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: BlockReport of 102 blocks took 1 msec to generate and 9 msecs for RPC and NN processing
2014-10-20 03:22:31,956 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Starting Periodic block scanner.
2014-10-20 03:22:31,957 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 50020: starting
2014-10-20 03:22:31,966 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Generated rough (lockless) block report in 9 ms
2014-10-20 03:22:31,966 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Reconciled asynchronous block report against current state in 0 ms
2014-10-20 03:22:32,669 INFO org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification succeeded for blk_3918413853414315562_2376
2014-10-20 03:22:46,957 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
2014-10-20 03:22:46,958 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Starting asynchronous block report scan
2014-10-20 03:22:46,964 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Finished asynchronous block report scan in 5ms
2014-10-20 03:22:49,964 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
2014-10-20 03:22:52,980 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
2014-10-20 03:22:55,984 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
2014-10-20 03:22:58,982 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
2014-10-20 03:23:01,981 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
2014-10-20 03:23:04,981 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
2014-10-20 03:23:07,993 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
2014-10-20 03:23:11,001 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
2014-10-20 03:23:14,001 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
2014-10-20 03:23:17,000 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
2014-10-20 03:23:20,006 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
2014-10-20 03:23:23,003 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
2014-10-20 03:23:26,006 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
2014-10-20 03:23:29,015 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
2014-10-20 03:23:32,013 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
2014-10-20 03:23:35,033 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
2014-10-20 03:23:38,017 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
2014-10-20 03:23:41,021 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
2014-10-20 03:23:44,035 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
2014-10-20 03:23:47,033 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
2014-10-20 03:23:50,040 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
2014-10-20 03:23:53,038 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
2014-10-20 03:23:56,044 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
2014-10-20 03:23:59,041 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
2014-10-20 03:24:02,044 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
2014-10-20 03:24:05,046 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
2014-10-20 03:24:08,047 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
2014-10-20 03:24:11,050 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
2014-10-20 03:24:14,050 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
2014-10-20 03:24:17,058 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
2014-10-20 03:24:20,068 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeCommand action: DNA_REGISTER
[/code]
Ato42 2014-10-20
  • 打赏
  • 举报
 回复
引用 4 楼 sky_walker85 的回复:
把配置贴出来看看吧，还有后台NameNode和DataNode日志的输出信息
namenode的richie（部分） 

2014-10-20 03:27:42,614 FATAL org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.getDatanode: Data node 192.168.1.101:50010 is attempting to report storage ID DS-1696742302-192.168.1.100-50010-1413704373290. Node 192.168.1.100:50010 is expected to serve this storage.
2014-10-20 03:27:42,617 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.registerDatanode: node registration from 192.168.1.101:50010 storage DS-1696742302-192.168.1.100-50010-1413704373290
2014-10-20 03:27:42,618 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.registerDatanode: node 192.168.1.100:50010 is replaced by 192.168.1.101:50010 with the same storageID DS-1696742302-192.168.1.100-50010-1413704373290
2014-10-20 03:27:42,618 INFO org.apache.hadoop.net.NetworkTopology: Removing a node: /default-rack/192.168.1.100:50010
2014-10-20 03:27:42,618 INFO org.apache.hadoop.net.NetworkTopology: Adding a new node: /default-rack/192.168.1.101:50010
2014-10-20 03:27:44,234 FATAL org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.getDatanode: Data node 192.168.1.100:50010 is attempting to report storage ID DS-1696742302-192.168.1.100-50010-1413704373290. Node 192.168.1.101:50010 is expected to serve this storage.
2014-10-20 03:27:44,535 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.registerDatanode: node registration from 192.168.1.100:50010 storage DS-1696742302-192.168.1.100-50010-1413704373290
2014-10-20 03:27:44,536 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.registerDatanode: node 192.168.1.101:50010 is replaced by 192.168.1.100:50010 with the same storageID DS-1696742302-192.168.1.100-50010-1413704373290
2014-10-20 03:27:44,536 INFO org.apache.hadoop.net.NetworkTopology: Removing a node: /default-rack/192.168.1.101:50010
2014-10-20 03:27:44,536 INFO org.apache.hadoop.net.NetworkTopology: Adding a new node: /default-rack/192.168.1.100:50010
2014-10-20 03:27:45,615 FATAL org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.getDatanode: Data node 192.168.1.101:50010 is attempting to report storage ID DS-1696742302-192.168.1.100-50010-1413704373290. Node 192.168.1.100:50010 is expected to serve this storage.
2014-10-20 03:27:45,618 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.registerDatanode: node registration from 192.168.1.101:50010 storage DS-1696742302-192.168.1.100-50010-1413704373290
2014-10-20 03:27:45,618 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.registerDatanode: node 192.168.1.100:50010 is replaced by 192.168.1.101:50010 with the same storageID DS-1696742302-192.168.1.100-50010-1413704373290
2014-10-20 03:27:45,618 INFO org.apache.hadoop.net.NetworkTopology: Removing a node: /default-rack/192.168.1.100:50010
2014-10-20 03:27:45,619 INFO org.apache.hadoop.net.NetworkTopology: Adding a new node: /default-rack/192.168.1.101:50010
2014-10-20 03:27:45,939 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:hadoop cause:org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot delete /data/hadoop/mapred/mrsystem. Name node is in safe mode.
The ratio of reported blocks 0.9902 has not reached the threshold 0.9990. Safe mode will be turned off automatically.
2014-10-20 03:27:45,939 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 9000, call delete(/data/hadoop/mapred/mrsystem, true) from 192.168.1.100:59461: error: org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot delete /data/hadoop/mapred/mrsystem. Name node is in safe mode.
The ratio of reported blocks 0.9902 has not reached the threshold 0.9990. Safe mode will be turned off automatically.
org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot delete /data/hadoop/mapred/mrsystem. Name node is in safe mode.
The ratio of reported blocks 0.9902 has not reached the threshold 0.9990. Safe mode will be turned off automatically.
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.deleteInternal(FSNamesystem.java:1994)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:1974)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.delete(NameNode.java:792)
        at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
2014-10-20 03:27:47,238 FATAL org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.getDatanode: Data node 192.168.1.100:50010 is attempting to report storage ID DS-1696742302-192.168.1.100-50010-1413704373290. Node 192.168.1.101:50010 is expected to serve this storage.
2014-10-20 03:27:47,241 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.registerDatanode: node registration from 192.168.1.100:50010 storage DS-1696742302-192.168.1.100-50010-1413704373290
2014-10-20 03:27:47,241 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.registerDatanode: node 192.168.1.101:50010 is replaced by 192.168.1.100:50010 with the same storageID DS-1696742302-192.168.1.100-50010-1413704373290
2014-10-20 03:27:47,241 INFO org.apache.hadoop.net.NetworkTopology: Removing a node: /default-rack/192.168.1.101:50010
2014-10-20 03:27:47,241 INFO org.apache.hadoop.net.NetworkTopology: Adding a new node: /default-rack/192.168.1.100:50010
2014-10-20 03:27:48,618 FATAL org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.getDatanode: Data node 192.168.1.101:50010 is attempting to report storage ID DS-1696742302-192.168.1.100-50010-1413704373290. Node 192.168.1.100:50010 is expected to serve this storage.
2014-10-20 03:27:48,621 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.registerDatanode: node registration from 192.168.1.101:50010 storage DS-1696742302-192.168.1.100-50010-1413704373290
2014-10-20 03:27:48,622 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.registerDatanode: node 192.168.1.100:50010 is replaced by 192.168.1.101:50010 with the same storageID DS-1696742302-192.168.1.100-50010-1413704373290
2014-10-20 03:27:48,622 INFO org.apache.hadoop.net.NetworkTopology: Removing a node: /default-rack/192.168.1.100:50010
2014-10-20 03:27:48,622 INFO org.apache.hadoop.net.NetworkTopology: Adding a new node: /default-rack/192.168.1.101:50010
2014-10-20 03:27:50,238 FATAL org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.getDatanode: Data node 192.168.1.100:50010 is attempting to report storage ID DS-1696742302-192.168.1.100-50010-1413704373290. Node 192.168.1.101:50010 is expected to serve this storage.
2014-10-20 03:27:50,239 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.registerDatanode: node registration from 192.168.1.100:50010 storage DS-1696742302-192.168.1.100-50010-1413704373290
2014-10-20 03:27:50,239 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.registerDatanode: node 192.168.1.101:50010 is replaced by 192.168.1.100:50010 with the same storageID DS-1696742302-192.168.1.100-50010-1413704373290
2014-10-20 03:27:50,239 INFO org.apache.hadoop.net.NetworkTopology: Removing a node: /default-rack/192.168.1.101:50010
2014-10-20 03:27:50,239 INFO org.apache.hadoop.net.NetworkTopology: Adding a new node: /default-rack/192.168.1.100:50010
2014-10-20 03:27:51,630 FATAL org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.getDatanode: Data node 192.168.1.101:50010 is attempting to report storage ID DS-1696742302-192.168.1.100-50010-1413704373290. Node 192.168.1.100:50010 is expected to serve this storage.
2014-10-20 03:27:51,632 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.registerDatanode: node registration from 192.168.1.101:50010 storage DS-1696742302-192.168.1.100-50010-1413704373290
2014-10-20 03:27:51,632 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.registerDatanode: node 192.168.1.100:50010 is replaced by 192.168.1.101:50010 with the same storageID DS-1696742302-192.168.1.100-50010-1413704373290
2014-10-20 03:27:51,632 INFO org.apache.hadoop.net.NetworkTopology: Removing a node: /default-rack/192.168.1.100:50010
2014-10-20 03:27:51,632 INFO org.apache.hadoop.net.NetworkTopology: Adding a new node: /default-rack/192.168.1.101:50010
2014-10-20 03:27:53,240 FATAL org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.getDatanode: Data node 192.168.1.100:50010 is attempting to report storage ID DS-1696742302-192.168.1.100-50010-1413704373290. Node 192.168.1.101:50010 is expected to serve this storage.
2014-10-20 03:27:53,240 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.registerDatanode: node registration from 192.168.1.100:50010 storage DS-1696742302-192.168.1.100-50010-1413704373290
2014-10-20 03:27:53,240 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.registerDatanode: node 192.168.1.101:50010 is replaced by 192.168.1.100:50010 with the same storageID DS-1696742302-192.168.1.100-50010-1413704373290
2014-10-20 03:27:53,240 INFO org.apache.hadoop.net.NetworkTopology: Removing a node: /default-rack/192.168.1.101:50010
2014-10-20 03:27:53,241 INFO org.apache.hadoop.net.NetworkTopology: Adding a new node: /default-rack/192.168.1.100:50010
master datanode
skyWalker_ONLY 2014-10-20
  • 打赏
  • 举报
 回复
把配置贴出来看看吧，还有后台NameNode和DataNode日志的输出信息
Ato42 2014-10-20
  • 打赏
  • 举报
 回复
引用 2 楼 sky_walker85 的回复:
是不是配置有问题？
我也觉得是配置有问题……但是我不知道那有问题啊
skyWalker_ONLY 2014-10-20
  • 打赏
  • 举报
 回复
是不是配置有问题？
Ato42 2014-10-20
  • 打赏
  • 举报
 回复
问题补充：当只启动master时候没有问题，但是当两台机器同时启动就会出现这种问题，而且Last Contact一直在增加中……
Spark 3.0.0集群启动原理和源码详解
本课程讲解Spark 3.0.0 集群启动原理和源码详解的内容，包括：Master 启动原理和源码详解: Master 启动的原理详解、 Master 启动的源码详解、 Master HA 双机切换、 Master 的注册机制和状态管理解密； Worker 启动...
PHP面试题(一)
双端队列中的元素可以从两端弹出，其限定插入和删除操作在表的两端进行。双向队列（双端队列）就像是一 个队列，但是你可以在任何一端添加或移除元素。而双端队列是一种数据结构，定义如下： A deque is a data ...
Linux 高可用（HA）集群之Pacemaker详解
大纲说明：本来我不想写这篇博文的，因为前几篇博文都有介绍pacemaker，但是我觉得还是得写一下，试想应该会有博友需要，特别是pacemaker 1.1.8（CentOS 6.4）以后，pacemaker 有些特别要注意的变化 ，最后想说，敬...
corosync pacemaker 配置高可用集群
corosync 是集群管理套件的一部分,它在传递信息的时候可以通过一个简单的配置文件 来定义信息传递的方式和协议等。 也就是说 corosync 是 Messaging Layer 集群信息层软件,需要 pacemaker 资源管理 器( CRM ),...
KVM详解，教科书般的经典讲解
RedHat Linux 最多支持物理 CPU 内核总数的10倍数目的虚拟CPU，但是不支持在一个虚机上分配超过物理CPU内核总数的虚拟CPU。 支持 KSM （Kenerl Same-page Merging 内核同页合并）  RedHat Linux KVM 有如下两...
Hadoop生态社区

20,808

社区成员

4,690

社区内容

发帖
与我相关
我的任务
社区描述
Hadoop生态大数据交流社区，致力于有Hadoop，hive，Spark，Hbase，Flink，ClickHouse，Kafka，数据仓库，大数据集群运维技术分享和交流等。致力于收集优质的博客
社区管理员
  • 分布式计算/Hadoop社区
  • 涤生大数据
加入社区
  • 近7日
  • 近30日
  • 至今

加载中

查看更多榜单
社区公告
暂无公告

试试用AI创作助手写篇文章吧

+ 用AI写文章