
        
        
            
                
                
                    import datetime
df = spark.createDataFrame([["Alex", datetime.date(1995,12,16)], ["Bob", datetime.date(1998,5,9)]], ["name", "birthday"])
df.show()
                
            
            +----+----------+
|name|  birthday|
+----+----------+
|Alex|1995-12-16|
| Bob|1998-05-09|
+----+----------+

Extracting the year component from datetime values in PySpark DataFrame

To extract the year component from datetime values:


        
        
            
                
                
                    from pyspark.sql import functions as F
df.select(F.year("birthday").alias("year")).show()
                
            
            +----+
|year|
+----+
|1995|
|1998|
+----+

Here, we are assigning the label "year" to the Column returned by year(~) using alias(~).

Extracting year component from date string in PySpark DataFrame

To extract the year component from date string values:


        
        
            
                
                
                    from pyspark.sql import functions as F
df = spark.createDataFrame([["Alex", "1995-12-16"], ["Bob", "1998-05-06"]], ["name", "birthday"])
df.select(F.year("birthday").alias("year")).show()
                
            
            +----+
|year|
+----+
|1995|
|1998|
+----+

Published by Isshin Inada

Edited by 0 others

Did you find this page useful?

thumb_up

thumb_down

Comment

Citation

Ask a question or leave a feedback...

Official PySpark Documentation

https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.sql.functions.year.html

thumb_up

thumb_down

chat_bubble_outline

settings

Enjoy our search

Hit / to insta-search docs and recipes!