Language Integration

Master Python + C/Rust integration using ctypes, cffi, Cython, and PyO3 for maximum performance while maintaining Python's productivity

Learn Python, JavaScript, Java and more with free interactive lessons, real projects and AI-powered help. Beginner-friendly.

Part of the free Python course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.

What You'll Learn

Why Integrate Python with Other Languages?

Python is excellent for productivity, but sometimes you need more. Language integration lets you keep Python's ease of use while accessing native performance and capabilities.

Common Reasons

Keep 95% of your code in Python for maintainability. Move the critical 5% (hot loops, numeric kernels) to native code. This is exactly how NumPy, pandas, and PyTorch work.

CPython's Architecture

Understanding CPython's internals helps you write better extensions.

Key Facts

How Extensions Work

ctypes: Call C Without Compilation

ctypes is Python's built-in FFI (Foreign Function Interface) for calling C libraries.

Workflow

Pros & Cons

cffi: More Ergonomic C Interface

cffi improves on ctypes with C-style declarations and better error handling.

Key Features

When to Use

Choose cffi when you need to wrap complex C APIs with many structs and pointers, or when you want more safety than raw ctypes provides.

Native Python Extensions in C/C++

The Python/C API provides maximum control and performance for extension modules.

Core Components

Use Cases

C extensions are powerful but error-prone. Manual memory management, reference counting, and GIL handling make them challenging. Consider Cython or Rust instead for new projects.

Cython: Python Syntax, C Speed

Cython lets you write Python-like code with optional type annotations that compile to C.

Performance Tips

• Use cdef for C-level variables and functions

• Use typed memory views for arrays: double[:]

• Release the GIL for CPU-intensive work: with nogil:

Scientific computing, numeric algorithms, and data processing. If your bottleneck is a tight numeric loop, Cython is often the fastest path to optimization.

Rust Integration with PyO3

Rust offers C-like performance with memory safety guarantees. PyO3 makes Rust-Python integration seamless.

Why Rust + Python?

PyO3 Features

Building with maturin

Embedding Python

Sometimes you want to embed Python as a scripting language inside a C/Rust application.

Basic Workflow

Data Marshalling Strategies

Efficient data transfer between languages is critical for performance.

Zero-Copy Techniques

Best Practices

• Batch operations - Call once with 1M elements, not 1M times with 1 element

• Use simple types at boundaries - Primitives, strings, byte arrays

• Avoid repeated conversions - Convert once, reuse

• Check array layout - Ensure C-contiguous for efficient access

Converting large NumPy arrays to Python lists loses all performance benefits. Always pass array pointers directly to C/Rust code.

When to Choose Which Integration Path

Decision Matrix

→ Rust + PyO3 (modern) or C++ with pybind11 (legacy)

→ Direct C extension (but consider alternatives first)

Practical Workflow

The best approach is incremental optimization based on profiling.

Recommended Process

Get it working first. Premature optimization wastes time.

Use cProfile, line_profiler, or py-spy to identify hot spots.

Use NumPy, better algorithms, caching. Native code may not be needed.

Only the 5-10% that's truly slow. Keep the rest in Python.

Callers shouldn't know it's implemented in C/Rust.

Test the boundary thoroughly. Native code bugs are harder to debug.

Practice quiz

Which integration tool is built into Python's standard library and needs no compilation step to call an existing C library?

  • Cython
  • PyO3
  • ctypes
  • maturin

Answer: ctypes. ctypes is Python's built-in FFI; it loads a shared library with CDLL() and needs no Python-specific build step.

What is the '95/5 Rule' described in the lesson?

  • Keep 95% of code in Python and move the critical 5% to native code
  • Keep 95% in native code, 5% in Python
  • Optimize 95% of functions before profiling
  • Use 95 native modules per project

Answer: Keep 95% of code in Python and move the critical 5% to native code. Keep 95% in Python for maintainability and move only the critical 5% (hot loops) to native code.

What is the GIL in CPython?

  • A garbage collector
  • A C compiler
  • A memory allocator
  • The Global Interpreter Lock that serializes Python bytecode execution

Answer: The Global Interpreter Lock that serializes Python bytecode execution. The GIL (Global Interpreter Lock) serializes Python bytecode; C extensions can release it for CPU-bound work.

Which tool offers Python-like syntax with optional type annotations that compile to C?

  • ctypes
  • Cython
  • cffi
  • PyO3

Answer: Cython. Cython lets you write Python-like code with optional types that compile to C for speed.

Which language binding emphasizes memory safety with no segfaults or data races?

  • Rust + PyO3
  • C extension via the Python/C API
  • ctypes
  • cffi

Answer: Rust + PyO3. Rust offers memory safety (no segfaults, no data races) and PyO3 makes Rust-Python integration seamless.

Which build tool handles building, packaging, and uploading PyO3 (Rust) extensions to PyPI?

  • setuptools
  • cythonize
  • maturin
  • cibuildwheel

Answer: maturin. maturin builds, packages, and publishes Rust/PyO3 extension wheels.

What is the recommended FIRST step in the practical optimization workflow?

  • Rewrite everything in C immediately
  • Write everything in pure Python and get it working first
  • Add PyO3 bindings
  • Disable the GIL

Answer: Write everything in pure Python and get it working first. Write it in pure Python first; premature optimization wastes time. Profile before optimizing.

What is the most efficient way to pass a large NumPy array to C/Rust code?

  • Convert it to a Python list first
  • Pickle the array
  • Send it one element at a time
  • Pass the raw array pointer directly (zero-copy)

Answer: Pass the raw array pointer directly (zero-copy). Zero-copy: pass the array's data pointer directly. Converting to a Python list destroys performance.

What does ctypes.CDLL('lib.so') do?

  • Compiles a C source file
  • Loads a C shared library so you can call its functions
  • Creates a new Python class
  • Releases the GIL

Answer: Loads a C shared library so you can call its functions. CDLL loads and gives access to a C shared library so its functions can be called from Python.

When import myextension runs for a compiled extension, what does Python look for first?

  • A .py source file
  • A Cargo.toml
  • A shared library (.so, .pyd, .dll)
  • A Dockerfile

Answer: A shared library (.so, .pyd, .dll). Python looks for a shared library (.so/.pyd/.dll), loads it, and calls its PyInit_ function.