Java 为什么运行hadoop时数据节点关闭？
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/18300940/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Why does data node shut down when I run hadoop?
提问by Wakko
I have hadoop 1.0.4 installed on ubuntu 11.0.4 in VirtualBox(same as my hostname), somehow the data node shuts down giving the following error in the log files
我在 VirtualBox 的 ubuntu 11.0.4 上安装了 hadoop 1.0.4（与我的主机名相同），不知何故数据节点关闭，在日志文件中出现以下错误
/************************************************************
STARTUP_MSG: Starting DataNode
STARTUP_MSG: host = VirtualBox/127.0.1.1
STARTUP_MSG: args = []
STARTUP_MSG: version = 1.0.4
STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r 1393290; compiled by 'hortonfo' on Wed Oct 3 05:13:58 UTC 2012
************************************************************/
2013-08-18 19:52:21,301 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2013-08-18 19:52:21,394 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered.
2013-08-18 19:52:21,412 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2013-08-18 19:52:21,417 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode metrics system started
2013-08-18 19:52:23,207 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi registered.
2013-08-18 19:52:23,276 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already exists!
2013-08-18 19:52:26,887 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException: Incompatible namespaceIDs in /app/hadoop/tmp/dfs/data: namenode namespaceID = 457871323; datanode namespaceID = 2066655210
at org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:232)
at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:147)
at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:385)
at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:299)
at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1582)
at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1521)
at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1539)
at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1665)
at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1682)
2013-08-18 19:52:26,903 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down DataNode at VirtualBox/127.0.1.1
************************************************************/
Any idea why?How can I fix it?
知道为什么吗？我该如何解决？
采纳答案by Andrew Martin
I've had this happen a few times. If restarting the data node doesn't help, then do the following:
我遇到过几次这种情况。如果重新启动数据节点没有帮助，请执行以下操作：
- Restart Hadoop
- Go to /app/hadoop/tmp/dfs/name/current
- Open VERSION (i.e. by
vim VERSION) - Record namespaceID
- Go to /app/hadoop/tmp/dfs/data/current
- Open VERSION (i.e. by
vim VERSION) - Replace the namespaceID with the namespaceID you recorded in step 4.
- 重启Hadoop
- 转到/app/hadoop/tmp/dfs/name/current
- 打开版本（即通过
vim VERSION） - 记录命名空间ID
- 转到/app/hadoop/tmp/dfs/data/current
- 打开版本（即通过
vim VERSION） - 将 namespaceID 替换为您在步骤 4 中记录的 namespaceID。
This should fix the problem.
这应该可以解决问题。
回答by SSaikia_JtheRocker
This usually results when you format the namenode and don't do a datanode clean up and you namespace changes.
这通常会在您格式化 namenode 并且不清理 datanode 并且您的命名空间更改时导致。
Probable solution:
可能的解决方案：
Try deleting the /app/hadoop/tmp/dfs/datadirectory and restart the datanode
尝试删除/app/hadoop/tmp/dfs/data目录并重启datanode
回答by Ramana
This is well known problem when we format the name node. Please refer below url:
当我们格式化名称节点时，这是众所周知的问题。请参考以下网址：
回答by minhas23
I lost my data on namenode. I had to format my namenode with
我在 namenode 上丢失了数据。我必须用
hadoop namenode -format
After starting namenode, datanode was failing to start because of VERSION problem
启动namenode后，由于VERSION问题导致datanode启动失败
Then I need to remove all the data from datanodes also.
然后我还需要从数据节点中删除所有数据。
I can simply clean data folder in dfs (/{hadoop.tmp.dir}/dfs/data/*) and restart datanode
我可以简单地清理 dfs (/{hadoop.tmp.dir}/dfs/data/*) 中的数据文件夹并重新启动 datanode
i.e rm -rf /{hadoop.tmp.dir}/dfs/data/*
IE rm -rf /{hadoop.tmp.dir}/dfs/data/*
OR
或者
delete the VERSION file (/{hadoop.tmp.dir}/dfs/data/current/VERSION) and restart datanode
删除VERSION文件（/{hadoop.tmp.dir}/dfs/data/current/VERSION）并重启datanode
回答by Vijay Kumar
Restart Hadoop
Go to /app/hadoop/tmp/dfs/name/current
Open VERSION (i.e. by vim VERSION)
Record namespaceID
Go to /app/hadoop/tmp/dfs/data/current
Open VERSION (i.e. by vim VERSION)
Replace the namespaceID with thve namespaceID you recorded in step 4.
to do this above steps, i dont have any files in the data folder /tmp/data
要执行上述步骤，我在数据文件夹 /tmp/data 中没有任何文件
