java 用于编写和运行 hadoop 作业的 IDE?

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/15337464/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-10-31 19:17:44  来源:igfitidea点击:

IDE for writing and running hadoop jobs?

javahadoopide

提问by Ashish Banerjee

I have recently started learning the basics of Hadoop map-reduce programming in Java. So far, the only way I found is writing the code in a file using vi or emacs which seems awful, primitive and very painful.

我最近开始学习 Java 中 Hadoop map-reduce 编程的基础知识。到目前为止,我发现的唯一方法是使用 vi 或 emacs 在文件中编写代码,这看起来很糟糕、原始且非常痛苦。

Is there any IDE for writing, compiling and running Hadoop programs?

是否有任何用于编写、编译和运行 Hadoop 程序的 IDE?

采纳答案by DanGar

You should also check out the IntelliJ IDEA as the Apache Hadoop wiki pages have instructions on how to set up your development environment using it.

您还应该查看 IntelliJ IDEA,因为 Apache Hadoop wiki 页面有关于如何使用它设置开发环境的说明。

References:

参考:

http://www.jetbrains.com/idea/

http://www.jetbrains.com/idea/

http://wiki.apache.org/hadoop/HadoopUnderIDEA

http://wiki.apache.org/hadoop/HadoopUnderIDEA

回答by Amar

Use Eclipse. Cloudera has a great screencast herefor configuring eclipse for hadoop development. Also debugging a hadoop job locally is pretty cool using eclipse (though not trivial). To know more on that, see here.

使用Eclipse。Cloudera在这里有一个很棒的截屏视频用于为 hadoop 开发配置 eclipse。使用 eclipse 在本地调试 hadoop 作业也很酷(虽然不是微不足道的)。要了解更多信息,请参阅此处

回答by Praveen Sripati

Hereis a blog for Debugging/Developing Hadoop Jobs on Eclipse/Linux. Also, check Karmasphere Studio.

是在 Eclipse/Linux 上调试/开发 Hadoop 作业的博客。另外,请检查Karmasphere Studio

回答by vmorusu

Yes, It's possible to run the MapReduce jobs in Eclipse. There is a lot of difference running MapReduce programs on a cluster and in Eclipse. Bellow are the few points worth noting:

是的,可以在 Eclipse 中运行 MapReduce 作业。在集群上和在 Eclipse 中运行 MapReduce 程序有很多不同。以下是值得注意的几点:

  • In Eclipse, it runs in a special mode called: LocalJobRunner mode i.e. all Hadoop daemons are run in single JVM.
  • In Eclipse, All paths refer to local file paths, and not to HDFS path.
  • 在 Eclipse 中,它以一种特殊的模式运行,称为:LocalJobRunner 模式,即所有 Hadoop 守护进程都在单个 JVM 中运行。
  • 在 Eclipse 中,所有路径都是指本地文件路径,而不是 HDFS 路径。