从 Scala 中的配置读取值
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/28966791/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Read values from config in Scala
提问by Stephan Rozinsky
In Scala, if I have the following config:
在 Scala 中，如果我有以下配置：
id = 777
username = stephan
password = DG#%T@RH
The idea is to open a file, transform it into a string, do getLineson it and get the right-hand side value based on the left-hand side key. What would be the nicest code to read constant configuration values into my app?
这个想法是打开一个文件，将其转换为一个字符串，getLines对其进行操作并根据左侧键获取右侧值。将常量配置值读入我的应用程序的最佳代码是什么？
Client usage: val username = config.get("username")
客户端使用： val username = config.get("username")
回答by Ende Neu
Best way would be to use a .conffile and the ConfigFactoryinstead of having to do all the file parsing by yourself:
最好的方法是使用.conf文件，ConfigFactory而不必自己进行所有文件解析：
import java.io.File
import com.typesafe.config.{ Config, ConfigFactory }
// this can be set into the JVM environment variables, you can easily find it on google
val configPath = System.getProperty("config.path")
val config = ConfigFactory.parseFile(new File(configPath + "myFile.conf"))
config.getString("username")
I usually use scalaz Validationfor the parseFileoperation in case the file it's not there, but you can simply use a try/catchif you don't know how to use that.
我通常使用 scalazValidation进行parseFile操作，以防文件不存在，但try/catch如果您不知道如何使用它，则可以简单地使用 a 。
回答by Nikunj Kakadiya
You can configure this values in json file ( I have named it as config.json)as below
您可以在 json 文件中配置此值（我将其命名为 config.json）如下
{
"id": "777",
"username": "stephan",
"password": "DG#%T@RH"
}
Now you can store this json file at hdfs location and read this file from hdfs location using spark in your scala and read your configuration values as below:
现在您可以将此 json 文件存储在 hdfs 位置，并使用 scala 中的 spark 从 hdfs 位置读取此文件，并读取您的配置值，如下所示：
val configData = spark.read.option("multiline",true).json("/tmp/user/config.json")
val id = configData.select("id").collect()(0)
val username = configData.select("username").collect()(0)
val password = configData.select("password").collect()(0)
In the first line of the code you need to use option with parameter of multiline = true as your json file will have each value on new line. if you don't use that you will get error saying _corrupt_record : string
在代码的第一行中，您需要使用参数为 multiline = true 的选项，因为您的 json 文件将在新行中包含每个值。如果你不使用它，你会得到错误说_corrupt_record : string
回答by Aman Sehgal
If your Spark version is less than 2.2 then first convert your JSON file content in to JSON String i.e. convert your file content to single string and load it to HDFS location.
如果您的 Spark 版本低于 2.2，那么首先将您的 JSON 文件内容转换为 JSON 字符串，即将您的文件内容转换为单个字符串并将其加载到 HDFS 位置。
Sample JSON:
示例 JSON：
{
"planet" : "Earth",
"continent" : "Antarctica"
}
Convert to:
转换成：
{ "planet" : "Earth", "continent" : "Antarctica"}
Next, to access data create a data frame:
接下来，要访问数据，请创建一个数据框：
val dataDF = spark.read.format("json").load("<HDFS location>")
val planet = dataDF.select("planet").collect(0).mkString("")
Hope this helps Spark 2.1 and less users.
希望这有助于 Spark 2.1 和更少的用户。
