Python | Generators
Start your free 7-days trial now!
While list comprehension returns a list (stored in memory), generators return a generator object (not stored in memory) that we can iterate over to produce elements of a list as required. Generator expressions look similar to list comprehension, however, use round brackets () rather than square brackets [].
When working with small datasets you may not need to leverage generators as the data will comfortably fit in memory, however, when working with large datasets you will need to leverage generators as not all the data will be able to be stored in memory at once.
Examples
Basic usage
To create a generator object with values 0 to 4:
result = (num for num in range(5))print(result)
<generator object <genexpr> at 0x7f8e5d36e4a0>
To iterate through the generator object and retrieve next element:
next(result)
0
If we run next(result) again we will retrieve the next element in the generator object i.e. 1. We can continue running next(result) until there are no more elements left in the iterable, at which point a StopIteration error will be thrown.
Creating a list using generators
To create a list from a generator object:
result = (num for num in range(5))list(result)
[0, 1, 2, 3, 4]
Conditionals
Similar to list comprehension, we can use conditionals in generator expressions:
odd_nums = (num for num in range(5) if num % 2 == 1)list(odd_nums)
[1, 3]
Refer to list comprehension for more details on conditionals.
Generator function
A generator function uses the keyword yield instead of return for standard functions, which again produces a generator object:
def gen_values(n): """Generate values from 0 to n.""" i = 0 while i < n: yield i i += 1
Let us call gen_values and iterate over the resulting generator object:
gen_obj = gen_values(4)
for element in gen_obj: print(element, end = " ")
Alternatively, we could have used the next() method to retrieve the next item in the iterable:
gen_obj = gen_values(4)print(next(gen_obj))
0
If we run next(result) again we will retrieve the next element in the generator object i.e. 1. We can continue running next(result) until there are no more elements left in the iterable, at which point a StopIteration error will be thrown.
