Generators

Generators and iterators are two of Python's most powerful features — allowing you to process massive datasets, stream data, write memory-efficient code, build pipelines, and design systems that behave like professional-grade libraries.

Learn Python, JavaScript, Java and more with free interactive lessons, real projects and AI-powered help. Beginner-friendly.

Part of the free Python course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.

What You'll Learn in This Lesson

This lesson will take you from advanced fundamentals → deep internal mechanics → real-world patterns used in production.

An iterator is any object that can give you items one at a time. It follows a simple contract:

A generator is Python's elegant shortcut for creating iterators using the yield keyword.

Instead of writing an entire class, just use yield :

This produces the same behavior as the iterator class — but with 90% less code.

When Python sees yield , something magical happens:

Step 1: Function becomes a generator object (not executed yet!)

Step 4: Next next() resumes from exact pause point

Step 5: Repeat until function ends → StopIteration

Imagine parsing a 5GB log file . Without generators, you'd crash. With generators:

Python itself uses this pattern in its own IO libraries.

Chain generators together like Unix shell pipes ( command1 | command2 | command3 ):

Each step processes data lazily. Nothing loads into memory at once.

yield from lets you delegate iteration to another generator:

This is cleaner and faster than nested loops.

You can send values INTO generators, making them interactive:

Generators can act as coroutines — functions that can pause and receive data:

Now superseded by async def, but still heavily used in internal libraries and advanced scheduling systems.

This pattern merges generators + cleanup logic.

Here's a complete example with detailed comments:

Understanding this gives you total control over how objects behave.

Streaming large CSVs with chunksize parameter

Generators still power internal scheduling logic.

This simulates how Airflow, Spark, and Pandas internally process data.

✔ How to use generators for memory-efficient processing

✔ How advanced frameworks use iterators under the hood

You understand lazy evaluation and how to build memory-efficient pipelines with generators — the same technique used inside Pandas, Airflow, and Spark.

Up next: Advanced Async & Await — write non-blocking concurrent code with asyncio.

Practice quiz

Which two methods define the iterator protocol?

  • __start__ and __stop__
  • __init__ and __call__
  • __iter__ and __next__
  • __get__ and __set__

Answer: __iter__ and __next__. An iterator implements __iter__ (returns itself) and __next__ (returns the next item).

What signals that an iterator has no more items?

  • raise StopIteration
  • return None
  • break
  • yield None

Answer: raise StopIteration. Raising StopIteration tells Python the sequence is exhausted; for-loops catch it to stop.

What keyword turns a function into a generator?

  • return
  • gen
  • async
  • yield

Answer: yield. Using yield in a function makes it a generator that produces values lazily.

What is the key difference between return and yield?

  • They are identical
  • return ends the function permanently; yield pauses it and can resume
  • yield ends the function; return pauses it
  • yield only works in classes

Answer: return ends the function permanently; yield pauses it and can resume. return exits a function forever; yield pauses execution and resumes from that point next time.

What does list(count_up_to(5)) produce for a generator yielding 1..n?

It yields 1 through 5 inclusive, so list() gives [1, 2, 3, 4, 5].

How do you write a generator expression for squares of 0..9?

A generator expression uses parentheses () instead of the brackets [] used by a list comprehension.

What is the main advantage of a generator expression over a list comprehension?

  • It is alphabetical
  • It uses far less memory by streaming items lazily
  • It sorts the data
  • It runs only once and caches

Answer: It uses far less memory by streaming items lazily. Generators yield items one at a time, so they don't store the whole sequence in memory.

What does 'yield from sub' do?

  • Returns sub as a list
  • Stops the generator
  • Sends a value into sub
  • Delegates iteration to the sub-generator/iterable

Answer: Delegates iteration to the sub-generator/iterable. yield from delegates to another iterable, yielding all its items cleanly (great for flattening).

Which method sends a value INTO a running generator?

  • .push(value)
  • .send(value)
  • .give(value)
  • .next(value)

Answer: .send(value). gen.send(value) resumes the generator and the yield expression evaluates to that value.

Which decorator turns a generator into a context manager?

  • @property
  • @staticmethod
  • @contextlib.contextmanager
  • @wraps

Answer: @contextlib.contextmanager. @contextmanager makes a generator a context manager: code before yield runs on entry, after on exit.