需要帮助运行 Cloudera 提供的 WordCount.java
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/11918942/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Need assistance with running the WordCount.java provided by Cloudera
提问by anonymous123
Hey guys so I am trying to run the WordCount.java example, provided by cloudera. I ran the command below and am getting the exception that I have put below the command. So do you have any suggestions on how to proceed. I have gone through all the steps provided by cloudera.
大家好，我正在尝试运行由 cloudera 提供的 WordCount.java 示例。我运行了下面的命令，并得到了我在命令下面放置的异常。那么您对如何进行有什么建议吗？我已经完成了 cloudera 提供的所有步骤。
Thanks in advance.
提前致谢。
hadoop jar ~/Desktop/wordcount.jar org.myorg.WordCount ~/Desktop/input
~/Desktop/output
Error:
错误：
ERROR security.UserGroupInformation: PriviledgedActionException
as:root (auth:SIMPLE)
cause:org.apache.hadoop.mapred.InvalidInputException: Input path does
not exist: hdfs://localhost/home/rushabh/Desktop/input
Exception in thread "main"
org.apache.hadoop.mapred.InvalidInputException: Input path does not
exist: hdfs://localhost/home/rushabh/Desktop/input
at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:194)
at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:205)
at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:977)
at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:969)
at org.apache.hadoop.mapred.JobClient.access0(JobClient.java:170)
at org.apache.hadoop.mapred.JobClient.run(JobClient.java:880)
at org.apache.hadoop.mapred.JobClient.run(JobClient.java:833)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1177)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:833)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:807)
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1248)
at org.myorg.WordCount.main(WordCount.java:55)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.apache.hadoop.util.RunJar.main(RunJar.java:197)
采纳答案by anonymous123
So I added the input folder to HDFS using the following command
所以我使用以下命令将输入文件夹添加到 HDFS
hadoop dfs -put /usr/lib/hadoop/conf input/
回答by Hisham Muneer
Your input and output files should be at hdfs. Atleast input should be at hdfs.
您的输入和输出文件应该在 hdfs。至少输入应该在 hdfs。
use the following command:
使用以下命令：
hadoop jar ~/Desktop/wordcount.jar org.myorg.WordCount hdfs:/input
hdfs:/output
HDFS:/输出
To copy a file from your linux to hdfs use the following command:
要将文件从 linux 复制到 hdfs，请使用以下命令：
hadoop dfs -copyFromLocal ~/Desktop/input hdfs:/
and check your file using :
并使用以下命令检查您的文件：
hadoop dfs -ls hdfs:/
Hope this will help.
希望这会有所帮助。
回答by somnathchakrabarti
The error clearly states that your input path is local. Please specify the input path to something on HDFS rather than on local machine. My guess
该错误明确指出您的输入路径是本地的。请指定 HDFS 上而不是本地机器上的输入路径。我猜
hadoop jar ~/Desktop/wordcount.jar org.myorg.WordCount ~/Desktop/input
~/Desktop/output
needs to be changed to
需要改为
hadoop jar ~/Desktop/wordcount.jar org.myorg.WordCount <hdfs-input-dir>
<hdfs-output-dir>
NOTE: To run MapReduce job, the input directory should be in HDFS, not local.
注意：要运行 MapReduce 作业，输入目录应该在 HDFS 中，而不是本地。
Hope this helps.
希望这可以帮助。
回答by Rahul Mahajan
When I tried to run wordcount MapReduce code, I was getting error as:
当我尝试运行 wordcount MapReduce 代码时，出现以下错误：
ERROR security.UserGroupInformation: PriviledgedActionException as:hduser cause:org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: file:/user/hduser/wordcount
ERROR security.UserGroupInformation: PriviledgedActionException as:hduser cause:org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: file:/user/hduser/wordcount
I was trying to execute the wordcount MapReduce java code with input and output path as /user/hduser/wordcount and /user/hduser/wordcount-output. I just added 'fs.default.name' from core-site.xml before this path and it ran perfectly.
我试图执行 wordcount MapReduce java 代码，输入和输出路径为 /user/hduser/wordcount 和 /user/hduser/wordcount-output。我只是在此路径之前从 core-site.xml 添加了 'fs.default.name' 并且它运行得很好。
回答by Stephen C
The error message says that this file does not exist: "hdfs://localhost/home/rushabh/Desktop/input".
错误消息说这个文件不存在：“hdfs://localhost/home/rushabh/Desktop/input”。
Check that the file does exist at the location you've told it to use.
检查该文件是否存在于您告诉它使用的位置。
Check the hostname is correct. You are using "localhost" which most likely resolves to a loopback IP address; e.g. 127.0.0.1. That always means "this host" ... in the context of the machine that you are running the codeon.
检查主机名是否正确。您正在使用“localhost”，它很可能解析为环回 IP 地址；例如 127.0.0.1。在您运行代码的机器的上下文中，这始终意味着“此主机” 。
回答by user1617057
Check the ownership of the files in hdfs to ensure that the owner of the job (root) has read privileges on the input files. Cloudera provides an hdfs viewer that you can use to view the filespace; open a web browser to either localhost:50075 or {fqdn}:50075 and click on "Browse the filesystem" to view the Input directory and input files. Check the ownership flags; just like *nix filesystem.
检查 hdfs 中文件的所有权，以确保作业的所有者（root）对输入文件具有读取权限。Cloudera 提供了一个 hdfs 查看器，您可以使用它来查看文件空间；打开 Web 浏览器到 localhost:50075 或 {fqdn}:50075，然后单击“浏览文件系统”以查看输入目录和输入文件。检查所有权标志；就像 *nix 文件系统一样。
