尝试从 Java 中的 HDFS 读取文件时出现“错误的 FS... 预期：file:///”
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/32078441/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
"Wrong FS... expected: file:///" when trying to read file from HDFS in Java
提问by gwg
I am unable to read a file from HDFS using Java:
我无法使用 Java 从 HDFS 读取文件：
String hdfsUrl = "hdfs://<ip>:<port>";
Configuration configuration = new Configuration();
configuration.set("fs.defaultFS", hdfsUrl);
FileSystem fs = FileSystem.get(configuration);
Path filePath = new Path(hdfsUrl + "/projects/harmonizome/data/achilles/attribute_list_entries.txt.gz");
FSDataInputStream fsDataInputStream = fs.open(filePath);
SEVERE: Servlet.service() for servlet [edu.mssm.pharm.maayanlab.Harmonizome.api.DownloadAPI] in context with path [/Harmonizome] threw exception
java.lang.IllegalArgumentException: Wrong FS: hdfs://146.203.54.165:8020/projects/harmonizome/data/achilles/attribute_list_entries.txt.gz, expected: file:///
at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:310)
at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:47)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:357)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:245)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:125)
at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:356)
at edu.mssm.pharm.maayanlab.Harmonizome.api.DownloadAPI.readLines(DownloadAPI.java:37)
at edu.mssm.pharm.maayanlab.Harmonizome.api.DownloadAPI.doGet(DownloadAPI.java:27)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:622)
...
I didn't setup our HDFS, so I don't know what I don't know. Any help is appreciated.
我没有设置我们的 HDFS，所以我不知道我不知道什么。任何帮助表示赞赏。
回答by java_bee
Try this:
试试这个：
Configuration configuration = new Configuration();
FileSystem fs = FileSystem.get(new URI(<url:port>), configuration);
Path filePath = new Path(<path/to/file>);
FSDataInputStream fsDataInputStream = fs.open(filePath);
BufferedReader br = new BufferedReader(new InputStreamReader(fsDataInputStream));
Please refer to http://techidiocy.com/java-lang-illegalargumentexception-wrong-fs-expected-file/
请参考http://techidiocy.com/java-lang-illegalargumentexception-wrong-fs-expected-file/
A similar problem is addressed.
解决了类似的问题。
回答by Sandeep Khot
Also below Configuration works!
也在下面配置工作！
Configuration conf = new Configuration();
conf.addResource(new Path("/usr/hdp/2.6.3.0-235/hadoop/etc/hadoop/core-site.xml"));
FileSystem fs = FileSystem.get(conf);
