Skip to content

Iterators & Generators Intermediate

🐍 Core Python Track · Level 3
⏱️ ~1 week 📚 Prerequisite: Comprehensions

The Iterator Protocol

The Iterator Protocol — a key concept in Iterators & Generators.

class CountUp:
    """Custom iterator that counts from 1 to max."""
    def __init__(self, max_val):
        self.max = max_val
        self.current = 0

    def __iter__(self):
        return self

    def __next__(self):
        self.current += 1
        if self.current > self.max:
            raise StopIteration
        return self.current

print(list(CountUp(5)))   # [1, 2, 3, 4, 5]

Generators — the easy way

Generators — the easy way, part of Iterators & Generators.

def count_up(max_val):
    current = 1
    while current <= max_val:
        yield current      # pause here, resume on next()
        current += 1

print(list(count_up(5)))   # [1, 2, 3, 4, 5]

yield from

yield from in Iterators & Generators — what it is and when to use it.

def flatten(nested):
    for item in nested:
        if isinstance(item, list):
            yield from flatten(item)   # delegate to sub-generator
        else:
            yield item

print(list(flatten([1, [2, 3], [4, [5, 6]]])))
# [1, 2, 3, 4, 5, 6]

Generator Pipelines

Generator Pipelines in Iterators & Generators — what it is and when to use it.

def read_lines(path):
    with open(path) as f:
        for line in f:
            yield line.strip()

def filter_comments(lines):
    for line in lines:
        if not line.startswith("#"):
            yield line

def to_upper(lines):
    for line in lines:
        yield line.upper()

# Compose the pipeline — lazy, memory-efficient
pipeline = to_upper(filter_comments(read_lines("config.txt")))
for line in pipeline:
    print(line)

itertools highlights

itertools highlights in Iterators & Generators — what it is and when to use it.

from itertools import chain, islice, groupby

# chain — concatenate iterables
print(list(chain([1, 2], [3, 4])))      # [1, 2, 3, 4]

# islice — slice any iterable (even an infinite/large one)
print(list(islice(count_up(100), 5)))   # [1, 2, 3, 4, 5]

# groupby — group CONSECUTIVE equal keys (sort first!)
students = [
    {"name": "Alice", "grade": "A"},
    {"name": "Bob", "grade": "B"},
    {"name": "Charlie", "grade": "A"},
]
data = sorted(students, key=lambda s: s["grade"])
for grade, group in groupby(data, key=lambda s: s["grade"]):
    print(grade, [s["name"] for s in group])
# A ['Alice', 'Charlie']
# B ['Bob']

groupby needs sorted input

itertools.groupby only groups consecutive equal keys. Sort by the same key first, or you'll get fragmented groups.


Practice exercises

  1. Write a generator fibonacci() that yields Fibonacci numbers forever.
  2. Build a 3-stage pipeline: read CSV → filter rows → transform → output.
  3. Implement flatten() that handles arbitrarily nested lists.
  4. Use itertools.groupby to group words by their first letter.

💬 Discussion

Have a question about this topic? Found an error? Share your thoughts below.