scala 如何在scala中进行外部连接

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/39905701/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-10-22 08:42:01  来源:igfitidea点击:

How to do OUTER JOIN in scala

scalajoinapache-sparkdataframe

提问by Newbie

I havce two data frames : df1 and df2

我有两个数据框:df1 和 df2

df1

df1

|--- id---|---value---|
|    1    |    23     |
|    2    |    23     |
|    3    |    23     |
|    2    |    25     |
|    5    |    25     |

df2

df2

|-idValue-|---count---|
|    1    |    33     |
|    2    |    23     |
|    3    |    34     |
|    13   |    34     |
|    23   |    34     |

How do I get this ?

我怎么得到这个?

|--- id--------|---value---|---count---|
|    1         |    23     |    33     |
|    2         |    23     |    23     |
|    3         |    23     |    34     |
|    2         |    25     |    23     |
|    5         |    25     |    null   |

I am doing :

我在做 :

 val groupedData =  df1.join(df2, $"id" === $"idValue", "outer") 

But I don't see the last column in the groupedData. Is this correct way of doing ? Or Am I doing any thing wrong ?

但我没有看到 groupedData 中的最后一列。这是正确的做法吗?还是我做错了什么?

回答by KiranM

From your expected output, you need LEFT OUTER JOIN.

根据您的预期输出,您需要 LEFT OUTER JOIN。

val groupedData =  df1.join(df2, $"id" === $"idValue", "left_outer").
       select(df1("id"), df1("count"), df2("count")).
       take(10).foreach(println)