
        
        
            
                
                
                    df = spark.createDataFrame([["Alex", 30], ["Bob", 20], ["Cathy", 20]], ["name", "age"])
df.show()
                
            
            +-----+---+
| name|age|
+-----+---+
| Alex| 30|
|  Bob| 20|
|Cathy| 20|
+-----+---+

Sorting a PySpark DataFrame in ascending order by a single column

To sort our PySpark DataFrame by the age column in ascending order:


        
        
            
                
                
                    df.sort("age").show()   # ascending=True
                
            
            +-----+---+
| name|age|
+-----+---+
|Cathy| 20|
|  Bob| 20|
| Alex| 30|
+-----+---+

We could also use sql.functions to refer to the column:


        
        
            
                
                
                    import pyspark.sql.functions as F
df.sort(F.col("age")).show()
                
            
            +-----+---+
| name|age|
+-----+---+
|Cathy| 20|
|  Bob| 20|
| Alex| 30|
+-----+---+

Sorting a PySpark DataFrame in descending order by a single column

To sort a PySpark DataFrame by the age column in descending order:


        
        
            
                
                
                    df.sort("age", ascending=False).show()
                
            
            +-----+---+
| name|age|
+-----+---+
| Alex| 30|
|  Bob| 20|
|Cathy| 20|
+-----+---+

Sorting a PySpark DataFrame by multiple columns

To sort a PySpark DataFrame by the age column first, and then by the name column both in ascending order:


        
        
            
                
                
                    df.sort(["age", "name"]).show()
                
            
            +-----+---+
| name|age|
+-----+---+
|  Bob| 20|
|Cathy| 20|
| Alex| 30|
+-----+---+

Here, Bob and Cathy appear before Alex because their age (20) is smaller. Bob then comes before Cathy because B comes before C.

We can also pass a list of booleans to specify the desired ordering of each column:


        
        
            
                
                
                    df.sort(["age", "name"], ascending=[True, False]).show()
                
            
            +-----+---+
| name|age|
+-----+---+
|Cathy| 20|
|  Bob| 20|
| Alex| 30|
+-----+---+

Here, we are first sorting by age in ascending order, and then by name in descending order.

Published by Isshin Inada

Edited by 0 others

Did you find this page useful?

thumb_up

thumb_down

Comment

Citation

Ask a question or leave a feedback...

Official PySpark Documentation

https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.sql.DataFrame.sort.html

thumb_up

thumb_down

chat_bubble_outline

settings

Enjoy our search

Hit / to insta-search docs and recipes!