Python Function Attributes Part I - Under the Hood

Function attributes are a relatively little-known corner of the Python language. Ultimately, this is because most of the problems they may solve are arguably better solved by a different approach (as will be shown). However, understanding why they exist may highlight some interesting cornerstones of Python's design to beginners or even intermediate users.

Prior to beginning, I'd like to make it clear that my purpose is not to proselytize, merely inform. Function attributes appears to be a topic that has not been blogged-to-death so I thought they might be worthwhile to discuss at some length.

Function Attributes 101

Attributes in general should be most familiar in the context of custom classes.¹ For the sake of introduction and comparison, consider this example:

class Foo:
    some_attribute = 'spam'
    another_attribute = 'eggs'

foo_instance = Foo()
Foo.yet_another_attribute = 'a dynamic attribute'

print(foo_instance.some_attribute)        # Output: spam
print(foo_instance.yet_another_attribute) # Output: a dynamic attribute

In the example above, some_attribute and another_attribute are attributes of the class Foo. They are class attributes, as opposed to instance attributes, though understanding the distinction is not particularly important for the purpose of this article (IRL is another matter). In line 6, a third class attribute named yet_another_attribute is added; it is called a dynamic attibute because it is added at runtime, i.e. after the class definition.

A Minimal Example

Back to the topic at hand, it is also possible in Python to add attributes to function objects in the same way(s) they can be added to custom classes, such as in the Foo example above. Consider the function shout:

def shout(s):
    print(str(s).upper() + '!')

shout('albatross')

Output:

ALBATROSS!

Attributes can be added to this function both within the function definition and at runtime, as shown below:

def shout(s):
    # Ignore (for now) the potential pitfalls of the function referring 
    # to itself by name.
    shout.some_attribute = 'Function attribute'

    print(str(s).upper() + '!')

shout('burma')
shout.another_attribute = "Another function attribute"
print(shout.some_attribute)
print(shout.another_attribute)

Output:

BURMA!
Function attribute
Another function attribute

Note: The potential problems related to functions referring to themselves by name is discussed in Function Names Are Not Magic elsewhere on this blog.

As the example shows, attributes can be assigned to shout, a function object, just as they were to the custom type Foo in the earlier example, and are accessed in the same object.attribute syntax. This is because class and function attributes are fundamentally the same - the topic of the next section. Additionally, the function of course continues to be callable and work as intended.

Under the Hood

This section delves into what's happening under the hood with regard to function attributes. This information mostly comes from the Standard Type Hierarchy section of the Data Model portion of the docs, which I suggest reading if you want to know more. If magic methods and __dict__ are old news to you, I suggest skipping ahead to the next section.

First-class objects

In Python, class and function attributes are fundamentally the same because classes and functions are fundamentally the same - they are both objects. A basic cirriculum in programming typically makes object synonymous with "instance of a class." However, to delve into the innards of the language, it is crucial to understand that everything is an object in modern Python (i.e. 3.x, and new-style classes in 2.1+).²

Practically speaking, everything being an object means everything is an instance of a class that inherits from object. Integers, booleans, etc. are not primitives as they are in languages such as C, C#, and Java - they are all instances of classes whose top-level parent is object. A function is no exception; calling type on a typical function results in <class 'function'>.

Beyond being objects, essentially everything in Python is a first-class object, which means anything may generally be assigned to, passed as an argument, and returned from a function. For example, even a module in Python may be passed to a function. Though not typically included in the definition, the setting and getting of attributes on objects can be thought of as an extension of this concept in Python, as by default anything that inherits from object has these capabilities, though they can be overridden.

For more about this topic, see this helpful StackOverflow question and its answers.

__dict__ and __setattr__

As stated previously, functions are first-class objects in Python, and implement a lot of the same functionality via so-called "magic methods" as e.g. user-created classes. An illustration:

class C:
    pass

def f():
    pass

print('Intersection of class and function attributes')
for attr in set(dir(f)) & set(dir(C)):
    print(attr)

Output:

Intersection of class and function attributes
__le__
__reduce__
__init__
__sizeof__
__class__
__ge__
__new__
__gt__
__format__
__doc__
__eq__
__repr__
__reduce_ex__
__lt__
__dir__
__dict__
__delattr__
__str__
__setattr__
__module__
__hash__
__getattribute__
__ne__
__subclasshook__

As per the docs, note that dir() is not guaranteed to return every possible attribute on an object. However, the above list serves as an important demonstration for this discussion because relevant attributes are present.

The most relevant attributes to this discussion in the intersection list above are __dict__ and __setattr__. These are the underlying reason that runtime attributes are possible in both functions and custom classes. In code such as...

class Foo:
    pass

s = Foo()
s.bar = 5

...under the hood, the final line results in a call to __setattr__ with eggs and 5 as the arguments; __setattr__ adds the attribute name eggs to the instance's __dict__ attribute, and maps it to the value 5. This is explicitly illustrated in the following example:

class Foo:
    def __setattr__(self, *args):
        print('setattr got:', args)
        super().__setattr__(*args)

f = Foo()

print(f.__dict__)
f.bar = 5
print(f.__dict__)

Output:

{}
setattr got: ('bar', 5)
{'bar': 5}

The __dict__ of a custom class instance contains the current instance attribute dictionary; class attributes would be held in the class object's __dict__. It is important to note that local variables do not appear in an object's __dict__. This, of course, falls in line with proper Python scoping as such variables should not be visible externally.

Functions work in the same manner:

def foo():
    local_variable = True

print(foo.__dict__)
foo.bar = 5
print(foo.__dict__)

Output:

{}
{'bar': 5}

The same process occurred in the function's case: at the line foo.bar = 5, foo.__setattr__ was called which added 'bar' to the function's __dict__ with a value of 5.

Note that local_variable never appears in foo.__dict__ for the scope reasons cited previously. A function's __dict__ will typically be empty; it is only used to hold function attributes. In fact, functions did not have a __dict__ attribute before the introduction of function attributes in PEP 232.³

Conclusion

There are inherently two questions being asked when asking "why" function attributes exist in Python: 1) why they exist from a language design perspective, and 2) why they exist from a pragmatic, use-case perspective. This article has attempted to elucidate much of the answer to the first question. In short, functions, as first-class objects in Python, implement much of the same functionality as other user-defined objects, including the dynamic setting of attributes. Given Python's propensity to empower rather than restrict, it's reasonable that functions, as objects, should have this capability. The second question is a subject of some debate, and is the topic of the future articles in this series.

Part II of this series will focus on providing a more practical example of the use of a function attribute, and introduce alternative means to similar functionality.

Notes