scala 获取超过 20 行并在 spark-shell 中显示列的完整值

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/37741841/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-10-22 08:22:03  来源:igfitidea点击:

fetch more than 20 rows and display full value of column in spark-shell

scalaapache-sparkpysparkapache-spark-sql

提问by Naresh

I am using CassandraSQLContextfrom spark-shell to query data from Cassandra. So, I want to know two things one how to fetch more than 20 rows using CassandraSQLContextand second how do Id display the full value of column. As you can see below by default it append dots in the string values.

我正在使用CassandraSQLContextfrom spark-shell 从 Cassandra 查询数据。所以,我想知道两件事,一是如何使用 获取超过 20 行CassandraSQLContext,二是如何显示列的完整值。正如您在下面看到的那样,默认情况下它会在字符串值中附加点。

Code :

代码 :

val csc = new CassandraSQLContext(sc)
csc.setKeyspace("KeySpace")
val maxDF = csc.sql("SQL_QUERY" )
maxDF.show

Output:

输出:

+--------------------+--------------------+-----------------+--------------------+
|                  id|               Col2|              Col3|                Col4| 
+--------------------+--------------------+-----------------+--------------------+
|8wzloRMrGpf8Q3bbk...|             Value1|                 X|                  K1|
|AxRfoHDjV1Fk18OqS...|             Value2|                 Y|                  K2|
|FpMVRlaHsEOcHyDgy...|             Value3|                 Z|                  K3|
|HERt8eFLRtKkiZndy...|             Value4|                 U|                  K4|
|nWOcbbbm8ZOjUSNfY...|             Value5|                 V|                  K5|

回答by eliasah

If you want to print the whole value of a column, in scala, you just need to set the argument truncate from the showmethod to false:

如果要打印列的整个值,在scala 中,只需将show方法中的参数 truncate 设置为false

maxDf.show(false)

and if you wish to show more than 20 rows :

如果您希望显示超过 20 行:

// example showing 30 columns of 
// maxDf untruncated
maxDf.show(30, false) 

For pyspark, you'll need to specify the argument name :

对于pyspark,您需要指定参数名称:

maxDF.show(truncate = False)

回答by WoodChopper

You won't get in nice tabular form instead it will be converted to scala object.

您不会得到漂亮的表格形式,而是将其转换为 Scala 对象。

maxDF.take(50)