Python pyspark 行号数据框
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/37390721/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
pyspark row number dataframe
提问by matlabit
I have a dataframe, with columns time,a,b,c,d,val. I would like to create a dataframe, with additional column, that will contain the row number of the row, within each group, where a,b,c,d is a group key.
我有一个数据框，包含时间、a、b、c、d、val 列。我想创建一个带有附加列的数据框，其中包含每个组中行的行号，其中 a、b、c、d 是组键。
I tried with spark sql, by defining a window function, in particular, in sql it will look like this:
我尝试使用 spark sql，通过定义一个窗口函数，特别是在 sql 中它看起来像这样：
select time, a,b,c,d,val, row_number() over(partition by a,b,c,d order by time) as rn from table
group by a,b,c,d,val
I would like to do this on the dataframe itslef, without using sparksql.
我想在数据帧本身上执行此操作，而不使用 sparksql。
Thanks
谢谢
回答by Carlos Vilchez
I don't know the python api too much, but I will give it a try. You can try something like:
我不太了解python api，但我会尝试一下。您可以尝试以下操作：
from pyspark.sql import functions as F
df.withColumn("row_number", F.row_number().over(Window.partitionBy("a","b","c","d").orderBy("time"))).show()
