
        
        
            
                
                
                    import datetime
df = spark.createDataFrame([["Alex", datetime.date(1995,12,16)], ["Bob", datetime.date(1995,5,9)]], ["name", "birthday"])
df.show()
                
            
            +----+----------+
|name|  birthday|
+----+----------+
|Alex|1995-12-16|
| Bob|1995-05-09|
+----+----------+

Getting the day of the week from datetime values in PySpark DataFrame

To get the day of the week in PySpark DataFrame:


        
        
            
                
                
                    import pyspark.sql.functions as F
df.select(F.dayofweek('birthday').alias('day')).show()
                
            
            +---+
|day|
+---+
|  7|
|  1|
+---+

Here:

7 represents Sunday while 1 represents Monday.
we are using alias(~) to give a label to the Column object returned by dayofweek(~).

Getting the day of week from date strings in PySpark DataFrame

Note that the method still works even if the date is in string form:


        
        
            
                
                
                    df = spark.createDataFrame([["Alex", "1995-12-16"], ["Bob", "1990-05-06"]], ["name", "birthday"])
df.select(F.dayofweek("birthday").alias("day")).show()
                
            
            +---+
|day|
+---+
|  7|
|  1|
+---+

Published by Isshin Inada

Edited by 0 others

Did you find this page useful?

thumb_up

thumb_down

Comment

Citation

Ask a question or leave a feedback...

Official PySpark Documentation

https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.sql.functions.dayofweek.html

thumb_up

thumb_down

chat_bubble_outline

settings

Enjoy our search

Hit / to insta-search docs and recipes!