Python Reflection and Introspection

Python Reflection and Introspection
Python Reflection and Introspection .For me, these two language features really make Python fun and set it apart from less-dynamic languages. Python supports both introspection and reflection, and in this article, I will explain how they work.

In programming, introspection is the ability to find out information about an object at runtime. Reflection takes this a step further by enabling objects to be modified at runtime.

For me, these two language features really make Python fun and set it apart from less-dynamic languages. Python supports both introspection and reflection, and in this article, I will explain how they work.

Introspection

The most basic information you can introspect at runtime is an object’s type. This can be achieved with the type() function.

>>> type(1)
<class 'int'>
>>> type(1.0)
<class 'float'>
>>> type(int)
<class 'type'>

Here, the interpreter is telling us that the integer 1 is of class int, 1.0 is of class float, and int is of class type.

The return value from the type() function is called a type object. A type object tells us what class the argument is an instance of.

We can confirm this neatly with isinstance():

>>> isinstance(1, int)
True
>>> isinstance(int, type)
True

Type objects support the is operator, so we can write:

>>> type(1) is int
True
>>> type(int) is type
True

However, type() and isinstance() aren’t directly equivalent since isinstance() also considers the base class of an object:

>>> class A:     
    pass
>>> class B(A):
    pass
>>> type(A()) is A
True
>>> type(B()) is A
False
>>> isinstance(B(), A)
True

isinstance.py

So, we may want to use type() and isinstance() together, like this:

if isinstance(obj, A):
    # do something for all children of A
    
    if type(obj) is B:
        # do something specifically for instances of B

Other techniques we can use to find out about an object in Python include:

  • hasattr(obj,'a') — This returns True if obj has an attribute a.
  • id(obj) — This returns the unique ID of an object.
  • dir(obj) — Returns all the attributes and methods of an object in a list.
  • vars(obj) — Returns all the instance variables of an object in a dictionary.
  • callable(obj) — Returns True if obj is callable.

We can also directly access some of this information using attributes that are automatically added to an object on creation. For example:

  • obj.__class__ stores the type object for obj.
  • obj.__class__.__name__ stores the class name for obj.
  • For function objects, obj.__code__ stores a code object with information about the code in the function.

For more information about code objects see the Python docs.

Introspection example

Putting all of this together, we can create a simple introspect() function:

class Test:
    def __init__(self):
        self.x = 1
        self.y = 2
        

def introspect(obj):
  for func in [type, id, dir, vars, callable]:
        print("%s(%s):\t\t%s" % (func.__name__, introspect.__code__.co_varnames[0], func(obj)))
        

introspect(Test())

introspect.py

Notice how we even use introspection within our function to print the name of the function being called (func.__name__) and the name of the introspect’s argument (introspect.__code__.co_varnames[0])!

The output would look like this:

type(obj):              <class '__main__.Test'>                                                                       
id(obj):                139779613404408                                                                               
dir(obj):               ['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__
', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__red
uce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'x', '
y']
vars(obj):              {'y': 2, 'x': 1}                                                                              
callable(obj):          False

Not bad for essentially two lines of code!

Advanced introspection

The tools outlined above are already very powerful. However, should you want to dig deeper into introspection, Python’s inspect module provides further capabilities for introspecting live objects.

Modifying Objects Dynamically

So far, we’ve discussed how to find information about an object at runtime. We’re now going to learn how to dynamically modify or even create new objects and classes!

Firstly, we should know that attributes can be added to a class or object at runtime. So, we can write:

>>> class A:
    pass
>>> A.x = 1
>>> a = A()
>>> a.y = 2
>>> a.y
2
>>> a.x
1

ClassA.py

Since methods are just a special type of attribute, this means we can also add methods at runtime. Let’s modify our class by dynamically adding an __init__ method to it.

>>> def init(self):    # the function and argument can have any name
    self.x = 1
>>> class A:
    pass
>>> setattr(A, '__init__', init)
>>> a = A()
>>> a.x
1

init.py

Notice how we use the [setattr](https://docs.python.org/3/library/functions.html#setattr) function to set the __init__ method of A to init. This allows the name of the attribute we are setting to also be determined dynamically.

We can take this concept a step further by modifying a function’s __code__ attribute. This time by simple assignment:

>>> def test():
    print("Test")
>>> test()
"Test"
>>> test.__code__ = (lambda: print("Hello")).__code__
>>> test()
"Hello"

test.py

This could be used, for example, to create a function that executes only once:

>>> def test():
    test.__code__ = (lambda: None).__code__
    print ("Test")
>>> test()       # First call prints "Test"
"Test"
>>> test()       # Subsequent calls do nothing

test.py

Creating classes at runtime

We are now going to revisit the type() function mentioned earlier and use it to create a new class dynamically. To do this, we call type() with three arguments:

type(name, bases, dict)

Where:

  • name is the name of the class we are creating.
  • bases is a tuple of base classes we inherit from.
  • dict is a dictionary of attribute name, attribute value pairs.

So, in its simplest form, we can write:

>>> A = type('A', (), {'x': 1})
>>> a = A()
>>> a.x
1

However, we can get much more advance that that. For example, to create a fully-fledged class:

>>> exec("def init(self):\n\tprint(self.__class__.__name__ + \" created!\")")
>>> A = type('A', (), {'__init__' : init })
>>> a = A()
"A created!"

Notice that, here, we use Python’s built-in [exec](https://docs.python.org/3/library/functions.html#exec) function to generate our class’s __init__ method from a string. The method definition itself uses self.__class__.__name__ to dynamically get the class name!

A Final Word of Caution

Whilst the language capabilities outlined in this article are powerful, with great power comes great responsibility!

These techniques should be used very sparingly. Excessive use of dynamic programming can make code harder to read. In some cases, it can also introduce security vulnerabilities, especially if dynamic execution involves user input.

Dynamically modifying code at runtime is sometimes referred to as monkey patching. Further details about potential applications and pitfalls can be found on Wikipedia.

Thanks for reading!!

Suggest:

☞ Python Tutorials for Beginners - Learn Python Online

☞ Learn Python in 12 Hours | Python Tutorial For Beginners

☞ Complete Python Tutorial for Beginners (2019)

☞ Python Programming Tutorial | Full Python Course for Beginners 2019

☞ Python Tutorial for Beginners [Full Course] 2019

☞ Introduction to Functional Programming in Python