Jak obliczyć sumę i liczyć w jednym groupBy?

Question

podstawie poniższego dataframe:

val client = Seq((1,"A",10),(2,"A",5),(3,"B",56)).toDF("ID","Categ","Amnt") 
+---+-----+----+ 
| ID|Categ|Amnt| 
+---+-----+----+ 
| 1| A| 10| 
| 2| A| 5| 
| 3| B| 56| 
+---+-----+----+ 

chciałbym, aby uzyskać numer ID oraz całkowitą ilość według kategorii:

+-----+-----+---------+ 
|Categ|count|sum(Amnt)| 
+-----+-----+---------+ 
| B| 1|  56| 
| A| 2|  15| 
+-----+-----+---------+ 

jest to możliwe do zrobienia liczbę i sumę bez musząc dołączyć?

client.groupBy("Categ").count 
     .join(client.withColumnRenamed("Categ","cat") 
      .groupBy("cat") 
      .sum("Amnt"), 'Categ === 'cat) 
     .drop("cat") 

Może coś takiego:

client.createOrReplaceTempView("client") 
spark.sql("SELECT Categ count(Categ) sum(Amnt) FROM client GROUP BY Categ").show() 

Answer 1

Daję inny przykład niż twoje

multiple group functions are possible like this. try it accordingly

// In 1.3.x, in order for the grouping column "department" to show up, 
// it must be included explicitly as part of the agg function call. 
df.groupBy("department").agg($"department", max("age"), sum("expense")) 

// In 1.4+, grouping column "department" is included automatically. 
df.groupBy("department").agg(max("age"), sum("expense")) 

Answer 2

Można zrobić agregację jak poniżej na danej tabeli:

client.groupBy("Categ").agg(sum("Amnt"),count("ID")).show() 

+-----+---------+---------+ 
|Categ|sum(Amnt)|count(ID)| 
+-----+---------+---------+ 
| A|  15|  2| 
| B|  56|  1| 
+-----+---------+---------+