Corran Webster

Observers in Python

2023-01-29

The observer pattern is one of the original patterns identified in Design Patterns. The core idea is that an object can notify other objects when their state changes. Because the formal presentation of patterns tend to come from languages like C++ and Java, the presentation tends to be somewhat verbose and not particularly intuitive to use.

Consider the Python example from Wikipedia:

class Observable:
    def __init__(self):
        self._observers = []

    def register_observer(self, observer):
        self._observers.append(observer)

    def notify_observers(self, *args, **kwargs):
        for obs in self._observers:
            obs.notify(self, *args, **kwargs)


class Observer:
    def __init__(self, observable):
        observable.register_observer(self)

    def notify(self, observable, *args, **kwargs):
        print("Got", args, kwargs, "From", observable)

and how it is used:

subject = Observable()
observer = Observer(subject)
subject.notify_observers("test", kw="python")

But this is not encapsulating the concept of observers being informed of state changes: code needs to manually call notify_observers rather than having it being a consequence of state change. A more complete observable might look more like:

class Observable:
    def __init__(self):
        self._observers = []
        self._state = {}

    ...

    def get_state(self):
        return self._state()

    def set_state(self, **kwargs):
        self._state.update(**kwargs)
        self.notify_observers(**kwargs)

But this is somewhat unpythonic: the state of a Python object is in its attributes (or items for a collection) rather than a separate dictionary, and Python provides hooks for setting an attribute (or items). The other is that the Observer instance could be replaced by a simple function or callable class. A more Pythonic implementation might look more like:

class Observable:
    def __init__(self):
        self._observers = []

    def register_observer(self, observer):
        self._observers.append(observer)

    def _notify_observers(self, name, value):
        for observer in self.observers:
            observer(name, value)

    def __setattr__(self, name, value):
        super().__setattr__(name, value)
        self._notify_observers(name, value)

(or something equivalent using __setitem__ for collections). This permits much more flexible and Pythonic use:

subject = Observable()
subject.register_observer(print)
subject.test = "python"

which for example will print every change to the observable's true state. Of course this isn't sufficient for robust generality: you need to have methods for unregistering and there are concerns around references to observers.

When you look at any Python system which has long-lived state, you tend to find that it implements some sort of observation system.

There are a number of mature libraries that provide this sort of functionality to varying degrees of complexity and additional functionality: Params, Traits, Traitlets (used internally by IPython), and Atom are the ones I'm most familiar with, but there are surely others out there. However most of these libraries also provide things like run-time type-checking, validation and hooks for GUI systems. They also provide the ability to do more fine-grained observation, for example listening for changes to just one attribute or a list of attributes that an observer might care about and ignoring the rest.

Because of this, these libraries have significant conceptual overlap with tools like dataclasses, Pydantic, Beartype (and other run-time type-checkers) which use the new type annotation system in Python, but are not really compatible with them. However these systems don't have any sort of observation system. The other distinction with these systems is that they tend to have a base class (or even a metaclass) that you need to inherit from, unlike dataclasses or Beartype which instead decorate regular classes. This means that they tend to be "all or nothing" and are hard to integrate with systems that do have special base classes.

It would be useful to have an observer library that: