java Spark - 按数据帧语法分组？
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/38848866/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Spark - Group by HAVING with dataframe syntax?
提问by lte__
What's the syntax for using a groupby-having in Spark without an sql/hiveContext? I know I can do
在没有 sql/hiveContext 的 Spark 中使用 groupby-have 的语法是什么？我知道我能做到
DataFrame df = some_df
df.registreTempTable("df");
df1 = sqlContext.sql("SELECT * FROM df GROUP BY col1 HAVING some stuff")
but how do I do it with a syntax like
但是我如何使用类似的语法来做到这一点
df.select(df.col("*")).groupBy(df.col("col1")).having("some stuff")
This .having()does not seem to exist.
这.having()似乎不存在。
回答by zero323
Yes, it doesn't exist. You express the same logic with aggfollowed by where:
是的，它不存在。你用agg后跟表达相同的逻辑where：
df.groupBy(someExpr).agg(somAgg).where(somePredicate)
回答by Sri_Karthik
Say for example if I want to find products in each category, having fees less than 3200 and their count must not be less than 10:
例如，如果我想在每个类别中查找费用低于 3200 且数量不少于 10 的产品：
- SQL query:
- SQL查询：
sqlContext.sql("select Category,count(*) as
count from hadoopexam where HadoopExamFee<3200
group by Category having count>10")
- DataFrames API
- 数据帧 API
from pyspark.sql.functions import *
df.filter(df.HadoopExamFee<3200)
.groupBy('Category')
.agg(count('Category').alias('count'))
.filter(column('count')>10)
