Python Protocols

Structural Subtyping

PEP 544↗ introduces the notion of structural subtyping↗ or static duck typing to Python. Here's how it works:

1from typing import Protocol
2
3class OrganicOrganism(Protocol):
4
5    def grows(self) -> None: ...
6
7    def reproduces(self) -> None: ...
8
9    def responds_to_stimuli(self) -> None: ...
10
11    def adapts(self) -> None: ...
12
13
14class Human:
15
16    def grows(self) -> None: pass
17
18    def reproduces(self) -> None: pass
19
20    def responds_to_stimuli(self) -> None: pass
21
22    def adapts(self) -> None: pass
23
24    def conscious(self) -> None: pass
25
26
27H: OrganicOrganism = Human()
28O: Human = OrganicOrganism() 
29

Type checking this with MyPy↗ shows complaints

(.venv) ➜ mypy main.py 
main.py:26: error: Cannot instantiate protocol class "OrganicOrganism"  [misc]
main.py:26: error: Incompatible types in assignment (expression has type "OrganicOrganism", variable has type "Human")  [assignment]
Found 2 errors in 1 file (checked 1 source file)

But when you fix it

H: OrganicOrganism = Human()
# O: Human = OrganicOrganism() 
# no every organism is conscious

You get

(.venv) ➜ mypy main.py 
Success: no issues found in 1 source file

Every human behaves like an organism in some way, but an organism may not behave like a human

How about a plant

class Plant:

    def grows(self) -> None: pass

    def reproduces(self) -> None: pass

    def responds_to_stimuli(self) -> None: pass

    def adapts(self) -> None: pass

P: OrganicOrganism = Plant() #Ok

Even though Human and Plant don't share a common ancestor, they both conform to the OrganicOrganism protocol because they exhibit the required methods. No need for inheritance here.

Ok we know all that, what's special

Let's say you have various classes representing different request or response objects, possibly from different libraries. You aim to consolidate these diverse objects and their methods into a unified class or adapter↗. You might expect these classes to inherit from a common base class. But, by using protocols, you can define a common interface that outlines the expected behavior of response objects with an abstracted one. This enables disparate classes to conform to the same structure without requiring a shared ancestry.

from typing import Optional, Dict, Protocol, runtime_checkable


@runtime_checkable
class RequestProtocol(Protocol):
    def p_cookie(self, cookie_name: str) -> Optional[str]:
        ...
    @property
    def p_all_cookies(self) -> Dict[str, str]:
        ...
    @property
    def p_is_connection_secure(self) -> bool:
        ...

    @property
    def p_base_url(self) -> str:
        ...

You can extend these different Request objects to where they adhere to the protocol.

from libx import Request

class RequestX(Request):
    def p_cookie(self, cookie_name: str) -> Optional[str]:
        return self.cookies.get(cookie_name)

    @property
    def p_cookies(self) -> Dict[str, str]:
        return self.cookies

    @property
    def p_is_connection_secure(self) -> bool:
        return self.url.is_secure

    @property
    def p_base_url(self) -> str:
        return str(self.base_url)

from liby import Request

class RequestY(Request):
    def p_cookie(self, cookie_name: str) -> Optional[str]:
        return self.cookie(cookie_name)

    @property
    def p_all_cookies(self) -> Dict[str, str]:
        return self.cookies

    @property
    def p_is_connection_secure(self) -> bool:
        return self.is_https

    @property
    def p_base_url(self) -> str:
        return self.base_url

As you can see these different libraries have different implementation over how they access different request properties/methods, but with a protocol you can define a unified way that all these different objects adhere to, now for example if you want to create a function that processes a given request you can annotate the request parameter with RequestProtocol, this means that his function will process any class that adheres to this protocol.

def process_request(req: RequestProtocol) -> None:

    # do anything with the shared methods
    process_request(req=RequestX(...)) # OK, no type issues 
    process_request(req=RequestX(...)) # OK, no type issues

Another use case is using Protocol to type function signatures.

class Foo:
    ...

@runtime_checkable
class Bar(Protocol):
    async def __call__(self, foo: Foo) -> None:
        ...

Any function that is asynchronous, accepts an argument called foo of type Foo and returns nothing, adheres to this protocol, a given function might be.

async def bar(foo: Foo): pass

If you would have typed this function otherwise it would have been.

f: Callable[[Foo],Awaitable[None]] = bar

It's much easier to type functions using Protocol instead of:

def f(bar: Callable[[Foo],Awaitable[None]]) -> Any: ...

If you annotate the protocol with @runtime_checkable, you're allowed access to isinstance() and issubclass() functions to check the type at runtime.

def f(bar: Bar) -> Any:
	if not isinstance(bar,Bar): ... # do something

You can also create a function that checks the signature of another function, if it adheres to the protocol or not, and provide helpful error messages over why the given object does not adhere to the protocol.

import inspect 

def check_signature(f: Bar) -> None:
    sig = inspect.signature(f)
    if not (
        "foo" in sig.parameters
        and sig.parameters["foo"].annotation == Foo
    ):
        raise TypeError(
            f"Given function does not adhere to Bar protocol: {f}"
        )