java Hadoop:TaskTracker 和 JobTracker 不是从 start-dfs.sh 开始的

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/13677517/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-10-31 13:34:06  来源:igfitidea点击:

Hadoop: TaskTracker and JobTracker don't start with start-dfs.sh

javahadoop

提问by Raunak Agarwal

I am trying to set up hadoop 0.22.0 on my laptop for learning purpose using this link

我正在尝试使用此链接在我的笔记本电脑上设置 hadoop 0.22.0 以进行学习

http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/

http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/

When I run the script start-dfs.sh this is the ouput

当我运行脚本 start-dfs.sh 这是输出

starting namenode, logging to /usr/local/hadoop/bin/../logs/hadoop-raunak-namenode-ubuntu.out
localhost: starting datanode, logging to /usr/local/hadoop/bin/../logs/hadoop-raunak-datanode-ubuntu.out
localhost: starting secondarynamenode, logging to /usr/local/hadoop/bin/../logs/hadoop-raunak-secondarynamenode-ubuntu.out

The output of jpsis:

的输出jps是:

18106 Jps
17269 NameNode
17556 DataNode
17845 SecondaryNameNode

But I can't see the TaskTrackeror JobTrackerrunning. So, can anyone please tell me what could I be missing. I am a total newbie and don't know what information is required. So, please let me know.

但我看不到TaskTrackerJobTracker正在运行。所以,谁能告诉我我会错过什么。我是一个完全新手,不知道需要什么信息。所以,请告诉我。

回答by Chris White

I agree with Chris Gerken's comment, you're running the start-dfs.shscript, which only starts the HDFS services. You'll need to run start-mapred.shif you want the map reduce services (job tracker and task trackers).

我同意 Chris Gerken 的评论,您正在运行start-dfs.sh仅启动 HDFS 服务的脚本。start-mapred.sh如果您想要地图缩减服务(作业跟踪器和任务跟踪器),则需要运行。

Also, the deprecated message from start-all.shhints towards this:

此外,来自start-all.sh提示的已弃用消息:

echo "This script is Deprecated. Instead use start-dfs.sh and start-mapred.sh"

回答by rajiv_

In case you are using the newer version of hadoop, you might find it a little different from the tutorial you mentioned. Now all the conf/files will be in hadoop-installation/etc/hadoopdirectory.

如果您使用的是较新版本的 hadoop,您可能会发现它与您提到的教程略有不同。现在所有conf/文件都在hadoop-installation/etc/hadoop目录中。

All the hadoop daemons will be in sbin/directory.

所有的 hadoop 守护进程都将在sbin/目录中。

Also, when you run sbin/start-all.shthe script will say that this command is deprecated and will ask you to run start-dfs.shand start-yarn.shin my case.

此外,当您运行sbin/start-all.sh脚本时,会说此命令已弃用,并会要求您运行start-dfs.shstart-yarn.sh在我的情况下。

start-dfs.shstarted the namenodeon master and data-nodeson slaveand secondarynamenodeon localhost.

start-dfs.sh开始了namenode对船长和data-nodesslavesecondarynamenodelocalhost

start-yarn.shstarted two new processes namely ResourceManagerand NodeManager. Therefore I guess that the ResourceManageris the JobTrackerand NodeManageris the TaskTrackerin this case.

start-yarn.sh启动了两个新进程,即ResourceManagerNodeManager。因此,我猜测,ResourceManagerJobTrackerNodeManagerTaskTracker在这种情况下。

回答by Shubham Sharma

JobTracker and TaskTracker are 2 essential process involved in MapReduce execution in MRv1 (or Hadoop version 1). Both processes are now deprecated in MRv2 (or Hadoop version 2) and replaced by Resource Manager, Application Master and Node Manager Daemons. If you will just use start-all.sh then it will start your following services .

JobTracker 和 TaskTracker 是 MRv1(或 Hadoop 版本 1)中 MapReduce 执行所涉及的 2 个基本过程。这两个进程现在在 MRv2(或 Hadoop 版本 2)中都已弃用,并由资源管理器、应用程序主进程和节点管理器守护进程取代。如果您将只使用 start-all.sh 那么它将启动您的以下服务。

7120 ResourceManager
6787 DataNode
6981 SecondaryNameNode
7573 Jps
6678 NameNode
7229 NodeManager

回答by shazin

Use the JobTracker web interface default is http://localhost:50030/and TaskTracker web interface default is http://localhost:50060/. May be it is not showing in JPS for some reason.

使用 JobTracker Web 界面默认为http://localhost:50030/,TaskTracker Web 界面默认为http://localhost:50060/。可能由于某种原因它没有显示在 JPS 中。