java 如何通过Spark submit传递外部参数
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/35783770/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
How to pass external parameters through Spark submit
提问by Satish Karuturi
In my Application, i need to connect to the database so i need to pass IP address and database name when application is submitted.
在我的应用程序中，我需要连接到数据库，因此我需要在提交应用程序时传递 IP 地址和数据库名称。
I submit the application as follows: :
我提交申请如下：
./spark-submit --class class name --master spark://localhost:7077 \
--deploy-mode client /home/hadoop/myjar.jar
回答by zero323
If you check the official documentationyou'll see that spark-submithas following syntax:
如果您查看官方文档，您将看到spark-submit具有以下语法：
./bin/spark-submit \
--class <main-class>
--master <master-url> \
--deploy-mode <deploy-mode> \
--conf <key>=<value> \
... # other options
<application-jar> \
[application-arguments]
You can use either application-argumentsand confto pass required configuration to the main method and SparkConfrespectively.
您可以使用application-argumentsandconf分别将所需的配置传递给 main 方法SparkConf。
回答by Ajit K'sagar
As stated by zero323 you can use the spark-submitcommand from the link
如 zero323 所述，您可以使用链接中的spark-submit命令
./bin/spark-submit \
--class <main-class>
--master <master-url> \
--deploy-mode <deploy-mode> \
--conf <key>=<value> \
... # other options
<application-jar> \
[application-arguments]
Here, confis used to pass the Spark related configs which are required for the application to run like any specific property(executor memory) or if you want to override the default property which is set in Spark-default.conf.
在这里，conf用于传递应用程序像任何特定属性（执行程序内存）一样运行所需的 Spark 相关配置，或者如果您想覆盖Spark-default.conf 中设置的默认属性。
As far as your use case is concerned you want to pass the IP to the application to connect to database then you can use the [application-arguments]which are passed after the JAR.
就您的用例而言，您希望将 IP 传递给应用程序以连接到数据库，然后您可以使用在 JAR 之后传递的[application-arguments]。
When you set up your main as:
当您将主要设置为：
def main(args: Array[String])
Then you can accept anything as an argument given after .jar line.
然后你可以接受任何东西作为 .jar 行之后给出的参数。
Please refer for more details
请参考更多详细信息
