Python Function Attributes Part III - Comparison with Class Wrappers

Previous articles in this series:

The previous articles about the topic of Python function attributes gave an example of how a decorator can be used to add functionality to a given function using function attributes. It then introduced an alternative approach using classes, and no function attributes, to achieve similar functionality.

The purpose of this article is to examine these two approaches in-depth to see if they hold up as "interchangeable", or if perhaps one approach has distinct advantages over the other. At its end, I'll share my view on the status of function attributes.

Comparison of the Class and Decorator Approaches

At a glance, the decorator and class approaches in the previous article appear very similar. If they are essentially identical, the question of why function attributes were included in the language at all could be raised.

But are these approaches functionally (i.e. from the end-user perspective) identical? To examine this question, two edge cases and the problems they create will be discussed, and each of the function attribute/decorator and class wrapper approaches will attempt to solve those problems.

Gotcha #1 - Overwriting an Attribute Name

Function Attributes

Consider the situation where a function is decorated twice, with two decorators that use the same generic attribute name count for different purposes.

Assume the decorators are immutable, i.e. you can't just give the counter attribute a more specific (i.e. better) name, which would be the real solution to this problem if it were possible. Also, acknowledge and then forget the fact that additional decorators could be written to wrap the existing decorators and provide access to the count attributes via different names. Yes, that is a potential solution but it skirts around the purpose of this section.

def count_calls(func):
    """
    Decorates func with added 'count' attribute which accumulates the number
    times func is called.
    """
    def wrapper(*args, **kwargs):
        wrapper.count += 1
        return func(*args, **kwargs)

    wrapper.count = 0
    return wrapper

def count_args(func):
    """
    Decorates func with added 'count' attribute which accumulates the number of
    args sent to func across all calls.
    """
    def wrapper(*args, **kwargs):
        wrapper.count += len(args)
        return func(*args, **kwargs)

    wrapper.count = 0
    return wrapper

@count_args
@count_calls
def foo(*args):
    pass

foo(1)
foo(2, 3, 4)
print(foo.count)

What does the final line print in this situation: 2 (the number of calls) or 4 (the number of arguments)? Due to the order in which decorators are chained, it prints "4." The problem here is the 'count' used by count_calls can no longer be accessed by the user. The accidental overwriting of already-existing attributes is an inherent problem with languages which allow dynamic runtime attributes, but in this case the class approach offers a reprieve.

Class Wrapper

The class approach has an advantage with regard to the attribute name overwriting problem because it carries a reference to the original function in its func attribute. Any function attributes assigned to the function it is wrapping could be accessed with wrapper_instance.func.attribute (or chained further if there were multiple class wrappers similar to the multiple decorator example). This is seen below:

class BaseFuncWrapper:
    def __init__(self, func):
        self.func = func

    def __call__(self, *args, **kwargs):
        return self.func(*args, **kwargs)

class ArgumentCounter(BaseFuncWrapper):
    """
    Wrap a function 'func' and keep a cumulative count of how many args it
    receives in attribute `count`.
    """
    def __init__(self, func):
        self.count = 0
        super().__init__(func)

    def __call__(self, *args, **kwargs):
        self.count += len(args)
        return super().__call__(*args, **kwargs)

class CallCounter(BaseFuncWrapper):
    """
    Wrap a function 'func' and keep a cumulative count of how many times it
    is called in attribute `count`.
    """
    def __init__(self, func):
        self.count = 0
        super().__init__(func)

    def __call__(self, *args, **kwargs):
        self.count += 1
        return super().__call__(*args, **kwargs)

def foo(*args):
    pass

foo = ArgumentCounter(foo)
foo = CallCounter(foo)

foo(1)
foo(2, 3, 4)

print(foo.count)
print(foo.func.count)

Output:

2
4

Unlike in the function attribute version of this task, both count attributes are easily accessible. Of course, the identical names are confusing, and which count is which takes extra effort to determine.

Wrappers via decorators could, of course, each be given e.g. an original_func attribute, but this would be a little more tedious, and not "naturally-occurring" like the class example, which must maintain a reference to the function (though not necessarily a public one).

Incidentally, the chaining of the example above is an apt example of how interchangeable a callable class instance and a function can be. The func attribute of the CallCounter instance is of course not a function, but an instance of ArgumentCounter; however, there is no need for CallCounter to ever be aware of that fact.

Gotcha #2 - Wrapping a Function with Pre-existing Attributes

Function Attributes

There is another case in which things start to smell with function attributes when decorators are involved. Consider what happens when decorating a function that already has a function attribute:

def foo():
    pass

def useless_decorator(func):
    def wrapper(*args, **kwargs):
        return func(*args, **kwargs)

    return wrapper

# Note this attribute is added PRIOR to wrapping the function
foo.some_attribute = 'something'

foo = useless_decorator(foo)
print(foo.some_attribute)

Output (portion of traceback omitted):

line 12, in 
    print(foo.some_attribute)
AttributeError: 'function' object has no attribute 'some_attribute'

The attribute some_attribute is assigned to the original foo function object, NOT the wrapped version of which replaces it. Trying to access it after foo has been wrapped results in an AttributeError. A small change to the decorator can alleviate this issue:

def foo():
    pass

def useless_decorator(func):
    def wrapper(*args, **kwargs):
        return func(*args, **kwargs)

    # This line added.
    wrapper.__dict__ = func.__dict__

    return wrapper

# Note this attribute is added PRIOR to wrapping the function
foo.some_attribute = 'something'

decorated_foo = useless_decorator(foo)
decorated_foo.another_attribute = 'something else'
print(foo.__dict__)
print(decorated_foo.__dict__)

Output:

{'some_attribute': 'something', 'another_attribute': 'something else'}
{'some_attribute': 'something', 'another_attribute': 'something else'}

Notice in this case wrapper.__dict__ now points to the exact same object as foo.__dict__. Pre-existing attributes can now be accessed via decorated_foo.attribute; however, note the overwriting problem described above would remain a potential problem.

Class Wrapper

A class wrapper approach can elegantly get around this issue using the __getattr__ magic method:

def foo():
    pass

class UselessWrapper:
    def __init__(self, func):
        self.func = func

    def __call__(self, *args, **kwargs):
        return self.func(*args, **kwargs)

    def __getattr__(self, name):
        return self.func.__getattribute__(name)

# Note this attribute is added PRIOR to wrapping the function
foo.some_attribute = 'something'

foo = UselessWrapper(foo)
print(foo.some_attribute)
print(foo.__dict__)

foo.another_attribute = 'something else'
print('\n' + foo.another_attribute)
print(foo.__dict__)

Output:

something
{'func': <function foo at 0x0000000003467D90>}

 something else
{'another_attribute': 'something else', 'func': <function foo at 0x0000000003467D90>}

This works due to a distinction between __getattribute__ and __getattr__: the __getattr__ method is only invoked if __getattribute__ failed to find the requested attribute. So, in this example only after some_attribute is not found to be an attribute of the UselessWrapper instance is __getattr__ called, and the attribute looked for within the func attribute (through the usual means of __getattribute__).1 Future attributes, as seen with another_attribute, are assigned to the UselessWrapper instance.

Use of the foo.func attribute would permit the cases of updating an attribute of the original function object or trying to create a duplicate attribute name. This makes the class approach, in my view, the superior approach, at least when taking these edge cases into consideration.

Conclusion

The previous section seems to support the conclusion that not using function attributes, and instead wrapping a function in a class, is likely an ideal approach in many cases where function attributes seem a viable option. Note that this opinion is, in part, what makes up the dissenting opinion section of PEP 232.

I should clarify that I'm not claiming there is no task which function attributes can accomplish that other approaches (even ignoring globals) can not. A discussion linked by PEP 232 provides several examples where they would be useful, if not necessary.

Looking at the previous section, I feel a little conflict with the "There should be one-- and preferably only one --obvious way to do it" piece of the Zen of Python. I don't think any of the implementations are particularly obvious. The class approach can likely be called the most robust, but the callable class pattern might make a casual or even intermediate user scratch his/her head (as would a decorator, possibly). An approach with a global variable explicity updated each time a function is called would be easy to follow for a light user, but that is a smelly solution. I feel this is both a pro and con to the inclusion of function attributes in Python; their inclusion doesn't replace an obvious solution, which is a pro, but they are not the obvious solution by any means, so whether they are redundant seems a valid question (that's the con).

I admit I'm a little torn about this subject, but in the end I agree with the inclusion of function attributes in the language (fifteen years after the fact...). I cite the following as my reasoning:

  1. No potential "code smell" is introduced that didn't already exist due to misuse of global state.

  2. The existence of function attributes falls entirely in line with the "functions are objects" mantra which is important to understanding Python.

  3. Probably most importantly, the language has consistently been soundly on the side of empowering users rather than constraining them. Even if the use-cases for function attributes are minimal, they don't really hurt anything. Python writers could go (and have gone) years without realizing they even exist.

Despite my decision, as has probably been implied several times, I would suggest they be used very sparingly.

In conclusion, I wrote a large amount about a minuscule feature of Python I possibly will never feel the need to use. That being said, I'm a firm believer in taking the time to consider the minutiae in areas about which one is passionate. I won't feel my time was wasted looking into this subject - hopefully even when the myriad ways in which I am wrong are inevitably pointed out.

Notes

  1. __getattr__ is not implemented by default in a custom class, so there is no need to, for example, call super().__getattr__. In fact, this would result in an AttributeError. ↩