我如何知道 Scala 中代码的运行时间？
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/37730808/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
How I know the runtime of a code in scala?
提问by David Rebe Garcia
I need to calculate the runtime of a code in scala. The code is.
我需要在 Scala 中计算代码的运行时间。代码是。
val data = sc.textFile("/home/david/Desktop/Datos Entrada/household/household90Parseado.txt")
val parsedData = data.map(s => Vectors.dense(s.split(' ').map(_.toDouble))).cache()
val numClusters = 5
val numIterations = 10
val clusters = KMeans.train(parsedData, numClusters, numIterations)
I need to know the runtime to process this code, the time have to be on seconds.
我需要知道处理此代码的运行时，时间必须以秒为单位。
回答by evan.oman
回答by Shu
Starting from Spark2+we can use spark.time(<command>)(only in scala until now) to get the time taken to execute the action/transformation..
从Spark2+我们可以使用spark.time(<command>)（直到现在仅在 Scala 中）获取执行操作/转换所花费的时间..
Example:
例子：
Finding count of records in a dataframe
查找计数 records in a dataframe
scala> spark.time(
sc.parallelize(Seq("foo","bar")).toDF().count() //create df and count
)
Time taken: 54 ms //total time for the execution
res76: Long = 2 //count of records
回答by fr3ak
You can use scalameter: https://scalameter.github.io/
您可以使用标量：https://scalameter.github.io/
Just put your block of code in the brackets:
只需将您的代码块放在括号中：
val executionTime = measure {
//code goes here
}
You can configure it to warm-up the jvm so the measurements will be more reliable:
您可以将其配置为预热 jvm，以便测量更可靠：
val executionTime = withWarmer(new Warmer.Default) measure {
//code goes here
}
回答by Larsenal
The most basic approach would be to simply record the start time and end time, and do subtraction.
最基本的方法是简单地记录开始时间和结束时间，然后做减法。
val startTimeMillis = System.currentTimeMillis()
/* your code goes here */
val endTimeMillis = System.currentTimeMillis()
val durationSeconds = (endTimeMillis - startTimeMillis) / 1000
回答by Ram Ghadiyaram
Case : Before spark 2.1.0
案例：spark 2.1.0之前
< Spark 2.1.0explicitly you can use this function in your code to measure time in milli seconds
< Spark 2.1.0明确您可以在代码中使用此函数以毫秒为单位测量时间
/**
* Executes some code block and prints to stdout the time taken to execute the block. This is
* available in Scala only and is used primarily for interactive testing and debugging.
*
*/
def time[T](f: => T): T = {
val start = System.nanoTime()
val ret = f
val end = System.nanoTime()
println(s"Time taken: ${(end - start) / 1000 / 1000} ms")
ret
}
Usage :
用法 ：
time {
Seq("1", "2").toDS().count()
}
//Time taken: 3104 ms
Case : After spark 2.1.0
案例：spark 2.1.0之后
>= Spark 2.1.0 There is a built in function given in SparkSession
>= Spark 2.1.0 有一个内置函数 SparkSession
you can use spark.time
您可以使用 spark.time
Usage :
用法 ：
spark.time {
Seq("1", "2").toDS().count()
}
//Time taken: 3104 ms
回答by Sandish Kumar H N
this would be the best way to do calculate time for scala code.
这将是计算 Scala 代码时间的最佳方法。
def time[R](block: => (String, R)): R = {
val t0 = System.currentTimeMillis()
val result = block._2
val t1 = System.currentTimeMillis()
println(block._1 + " took Elapsed time of " + (t1 - t0) + " Millis")
result
}
result = kuduMetrics.time {
("name for metric", your function call or your code)
}
