bash 从 oozie 到 Hue 运行 shell 脚本
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/23628129/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Running shell script from oozie through Hue
提问by Sourabh Potnis
I am invoking a bash shell script using oozie editor in Hue.
我正在使用 Hue 中的 oozie 编辑器调用 bash shell 脚本。
I used the shell action in the workflow and tried below different options in shell command:
我在工作流中使用了 shell 操作，并在 shell 命令中尝试了以下不同的选项：
- Uploaded the shell script using 'choose a file'
- Gave local directory path where shell script is present
- Gave HDFS path where shell script is present
- 使用“选择文件”上传 shell 脚本
- 给出存在 shell 脚本的本地目录路径
- 给出存在 shell 脚本的 HDFS 路径
But all these options gave following error:
但是所有这些选项都给出了以下错误：
Cannot run program "sec_test_oozie.sh" (in directory "/data/hadoop/yarn/local/usercache/user/appcache/application_1399542362142_0086/container_1399542362142_0086_01_000002"): java.io.IOException: error=2, No such file or directory
无法运行程序“sec_test_oozie.sh”（在目录“/data/hadoop/yarn/local/usercache/user/appcache/application_1399542362142_0086/container_1399542362142_0086_01_000002”中）：没有这样的文件或目录，错误=2。
How should I give the shell script execution command? Where the shell script file should be residing?
我应该如何给出shell脚本执行命令？shell 脚本文件应该存放在哪里？
回答by SergioRG
You need add file "sec_test_oozie.sh" in oozie shell step. In add files
您需要在 oozie shell 步骤中添加文件“sec_test_oozie.sh”。在添加文件中
回答by Crypt
I think you are creating the file from windows machine which is adding extra line break characters.You need to convert the shell script file to Unix format.I also faced the same issue.Then I created the file from a Linux system and it started working.The error is misguiding.
我认为您正在从 Windows 机器创建文件，该文件添加了额外的换行符。您需要将 shell 脚本文件转换为 Unix 格式。我也遇到了同样的问题。然后我从 Linux 系统创建了该文件并开始工作.错误是误导。
回答by Ameba Spugnosa
I want to extend the @SergioRG answer. Oozie, at least with Cloudera's Hue interface is very counterintuitive.
我想扩展@SergioRG 的答案。Oozie，至少与 Cloudera 的 Hue 界面是非常违反直觉的。
To run a script file, three conditions should be met:
要运行脚本文件，需要满足三个条件：
- the file is on the HDFS file system, in a folder accessible by Oozie
- the file should be indicated in the shell command field
- the file should be added with any other dependent file in the "Files+" part of the task card.
- 该文件位于 HDFS 文件系统上，位于 Oozie 可访问的文件夹中
- 该文件应在 shell 命令字段中指明
- 该文件应与任务卡的“文件+”部分中的任何其他相关文件一起添加。
I wonder why they didn't add by default the script file you are calling.
我想知道为什么他们没有默认添加您正在调用的脚本文件。
Edit: please also check in advanced options (the gear in the left upper corner) if you need to set the path variable (eg. PATH=/usr/local/bin:/usr/bin).
编辑：如果您需要设置路径变量（例如PATH=/usr/local/bin:/usr/bin），请同时检查高级选项（左上角的齿轮）。
回答by drordk
I encountered the same problem, and the problem was that the script echoed some irrelevant line while the workflow tried to parse it as a property line. Oozie gave a very irrelevant error messageof java.io.IOException: error=2, No such file or directorywhich only added confusion.
我遇到了同样的问题，问题是脚本回显了一些不相关的行，而工作流试图将其解析为属性行。Oozie 给出了一个非常不相关的错误消息，java.io.IOException: error=2, No such file or directory它只会增加混乱。
- You will need to use
<file>to add your script. - If you used
<capture-output/>then you must make sure that your script prints only "key=value" lines, like java properties, otherwise you will get the error you seejava.io.IOException: error=2, No such file or directorywith some path pointing to.../yarn/local/usercache/...
- 您将需要使用
<file>来添加您的脚本。 - 如果您使用，
<capture-output/>那么您必须确保您的脚本仅打印“key=value”行，例如 java 属性，否则您会看到java.io.IOException: error=2, No such file or directory一些指向的错误.../yarn/local/usercache/...
回答by Romain
回答by Muthu
"No such a file or directory" oozie cannot locate the file. Please check the AddPath setting in the command.
“没有这样的文件或目录”oozie 找不到文件。请检查命令中的 AddPath 设置。
In the edit node seciton, get the oozie application hdfs path.
在编辑节点部分，获取oozie应用程序的hdfs路径。
Upload the shell script in hdfs oozie application path.
在 hdfs oozie 应用程序路径中上传 shell 脚本。
In the oozie edit node step, Shell command - specify the shell script name which is uploaded.
在 oozie 编辑节点步骤中，Shell 命令 - 指定上传的 shell 脚本名称。
Below that there would be option to AddPath, then add files, add the shell script which was uploaded in the hdfs path.
下面是AddPath的选项，然后添加文件，添加在hdfs路径中上传的shell脚本。
回答by Kunal Khaire
Removing the #!/bin/bash from my shell script helped me
从我的 shell 脚本中删除 #!/bin/bash 对我有帮助
