PySpark DataFrame | toPandas method
Start your free 7-days trial now!
toPandas(~) method converts a PySpark DataFrame into a Pandas DataFrame.
Watch out for the following:
All the data from the worker nodes are transferred to the Driver, and so make sure that your Driver has sufficient memory.
Driver must have the Pandas libraries installed.
This method does not take in any parameters.
A Pandas DataFrame.
Consider the following DataFrame:
Converting a PySpark DataFrame into a Pandas DataFrame
To convert this PySpark DataFrame into a Pandas DataFrame:
df.toPandas()name age0 Alex 201 Bob 242 Cathy 22