Learn Python in 7 days

Iterators & Generators

The Course
•
January 11, 2026
•
5 min read

Every for loop you have written runs on the same machinery: an object hands out one value at a time until it says stop. Today you build things that do the handing out.

The iterator protocol

What a for loop really does Python 
numbers = [10, 20, 30]

iterator = iter(numbers)      # the for loop calls this
print(next(iterator))
print(next(iterator))
print(next(iterator))

try:
    print(next(iterator))     # nothing left to hand out
except StopIteration:
    print("StopIteration - this is where a for loop stops")

for x in thing: means: call iter(thing), then next() repeatedly until StopIteration. Anything that supports those two calls can be looped over.

Generators

Writing that protocol by hand is tedious. A function containing yield becomes a generator and does it for you.

yield in action Python 
def countdown(n):
    print("  (starting)")
    while n > 0:
        yield n
        n -= 1
    print("  (finished)")

gen = countdown(3)
print(gen)                # nothing has run yet

print(next(gen))          # runs up to the first yield, then pauses
print(next(gen))
print(next(gen))

for value in countdown(3):
    print("  got", value)

The key idea: yield pauses the function and hands a value out. The next next() resumes exactly where it stopped, with all local variables intact.

Why it matters: memory

A list of a million versus a generator Python 
import sys

def squares_list(n):
    return [i ** 2 for i in range(n)]

def squares_gen(n):
    for i in range(n):
        yield i ** 2

as_list = squares_list(1_000_000)
as_gen = squares_gen(1_000_000)

print("list:     ", sys.getsizeof(as_list), "bytes")
print("generator:", sys.getsizeof(as_gen), "bytes")

print(sum(squares_gen(1_000_000)))

The list holds a million integers at once; the generator holds one. Both produce the same sum. This is why reading a file line by line works on files larger than your memory — file objects are iterators.

Infinite sequences

A generator need never finish, because nothing is computed until asked for.

An endless generator, safely consumed Python 
def fibonacci():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

# Take only what you need:
for index, value in enumerate(fibonacci()):
    if index >= 10:
        break
    print(value, end=" ")
print()

from itertools import islice
print(list(islice(fibonacci(), 10)))

Pipelines

Generators chain together, each stage pulling from the one before. Nothing is buffered in between.

A three-stage pipeline Python 
def read_lines(text):
    for line in text.splitlines():
        yield line

def drop_blanks(lines):
    for line in lines:
        if line.strip():
            yield line

def parse(lines):
    for line in lines:
        name, _, score = line.partition(",")
        yield name.strip(), int(score)

raw = """ada, 95

alan, 88
grace, 91
"""

for name, score in parse(drop_blanks(read_lines(raw))):
    print(f"{name:6} {score}")

Each function does one thing and could be tested alone. Swap the source for a 20-gigabyte file and nothing else changes.

yield from

Delegating to another generator Python 
def letters():
    yield from "abc"

def numbers():
    yield from range(3)

def both():
    yield from letters()
    yield from numbers()

print(list(both()))

itertools

The standard library ships a toolbox of ready-made generators.

A few worth knowing Python 
from itertools import count, cycle, islice, chain, groupby, pairwise

print(list(islice(count(10, 5), 4)))          # 10, 15, 20, 25
print(list(islice(cycle("ab"), 5)))
print(list(chain([1, 2], [3, 4])))
print(list(pairwise([1, 2, 3, 4])))           # consecutive pairs

words = ["apple", "avocado", "banana", "blueberry", "cherry"]
for letter, group in groupby(words, key=lambda w: w[0]):
    print(letter, list(group))
A generator is consumed once

Once you have iterated a generator, it is empty — there is no rewind. If you need the values twice, either collect them into a list or call the generator function again to get a fresh one.

Exhaustion Python 
gen = (n for n in range(3))

print(list(gen))
print(list(gen))      # empty - already consumed
Exercise

A log filter

Write a generator errors_only(lines) that yields only the lines containing ERROR, and another with_numbers(lines) that yields (line_number, line) pairs counting from 1 in the original log. Chain them so the numbers refer to the original file, not the filtered result.

log = """INFO starting up
ERROR disk full
INFO retrying
ERROR timeout
INFO done"""

# Your code here
Show one solution 
log = """INFO starting up
ERROR disk full
INFO retrying
ERROR timeout
INFO done"""

def with_numbers(lines):
    for number, line in enumerate(lines, start=1):
        yield number, line

def errors_only(numbered):
    for number, line in numbered:
        if "ERROR" in line:
            yield number, line

for number, line in errors_only(with_numbers(log.splitlines())):
    print(f"{number}: {line}")

Numbering first, then filtering, is what keeps the line numbers honest — filter first and you would be numbering the errors, not the log.

What you learned

  • for runs on iter() and next(), ending at StopIteration.
  • A function with yield is a generator: it pauses and resumes, keeping its state.
  • Generators use constant memory regardless of how many values they produce.
  • They can be infinite, and chained into pipelines.
  • A generator can only be consumed once.
Last updated on January 11, 2026

Was this article helpful?

Your response is saved on this device.

More lessons