digital, abstract, binary-1742687.jpg

Functions and Functional Programming in Python (2. Functions under the hood)

In the name of Allah, most gracious and most merciful,

1. Introduction

In this post, I will talk about some of the very important concepts in Python like Python objects, namespaces, how to inspect these namespaces, scopes, name resolution, the order of resolving them, how variables are passed to functions, what the functions return, what are functions’ hidden attributes’, and what else can you do with functions.

2. Everything is an Object in Python

In simple terms, you could think of an object as a box that has three things:

  1. type (int, bool, str, ...). In Python type(object)
  2. id which represents the identity of the object: It is a unique number assigned to the object on creating it. It is the memory address of the object, and thus it is different on each time you run the program except for some objects that has a constant unique id, like integers from -5 to 256.
  3. data: The data that is in the object whether it is a number, string, or even a collection of data.

You could validate this by typing this piece of code in your Python environment or interpreter isinstance(3, object) which results in True. Even None which indicates Nothingness in Python is an object.

False is not None in Python. So False Means something that is False in Python like empty objects. As far as I know there is only one thing that means Nothingness in Python which is None which could be similar or close to Null in C & C++.

Note 1

3. Variables and Namespaces

In Python, a variable name is just a reference to the object in memory. So variable names aren’t objects but they point to objects. Reference means something like a tag for the same object. Another term is namespace (or symbol table) which is associative mapping from variable names to objects. You can think of a namespace as a dictionary where keys are object names and values are the objects themselves. Note that Python always handles objects construction (creation) and destruction.

Let me give you three examples for clarification. Watch out for example 2 because it could introduce bugs in your code if you aren’t aware of this behavior.

##### Example 1 #####
# now var_1 is just a reference to the value 3 in memory
var_1 = 3
# var_2 is just another reference to the same value 3 in memory. So no new object is
 created
var_2 = var_1
# to prove it check do the following
id(var_1) == id(var_2)
>> True

##### Example 2 #####
list_1 = [1, 3, 5]
list_2 = list_1
list_1.append(6)
print(list_2)
>> [1, 3, 5, 6]
# so list_2 and list_1 are tags to the same thing. They are pointing to the same object in memory

##### Example 3 ##### (What if you want list_2 to be a different object, you just want a copy)
list_1 = [1, 3, 5]
list_2 = list_1.copy()
list_1.append(6)
print(list_2)
>> [1, 3, 5]
print(list_1)
>> [1, 3, 5, 6] # now they are different objects
# for further validation
id(list_1) == id(list_2)
>> False

== is used for comparing values and not objects while is is used for comparing identities or objects.

Note 2

4. Inspecting Namespaces

  1. For debugging purposes you could access the local namespace using locals()['variable_name'] to know the value corresponding to the variable_name in the local namespace. Since as I have just mentioned that you could see the namespace as a dictionary.
  2. Similarly you could access global namespace using globals()['variable_name']

locals and globals are related to the idea of scope. Scope is the place where the variable lives. If a variable is defined inside a function then that is its scope. Outside this scope it is undefined. There could be scope within scope within scope if for instance you have function defined within a function within a function. Global scope is a big scope that is accessible from anywhere in the code even from inside functions.

Note 3: Global vs Local

5. Variable or Scope Resolution (Name Resolution)

Name resolution is the process that Python uses to find the object value that is referred to by a variable name. In other words, name resolution is required to determine which scope should the value of a variable come from. In Python, scope resolution follows the LEGB rule. There are four namespaces inside Python.

  • L (Local Namespace): Variables defined inside a function, lambda, and try-except block and not declared as global in any of these are called local variables, and thus they have a new local namespace or symbol table that is created for each function separately on executing it. This local namespace is deleted when the code block or the function finishes execution.
  • E (Enclosed Namespace): When a function is defined inside a function. It has the same lifecycle as the local namespace.
  • G (Global Namespace): It is in the current Python script, or module or by using the global statement to define a variable anywhere in the file even inside a function. Its life cycle ends on quitting the interpreter.
  • B (Built-in Namespace): Names assigned in built-in names like range, open, etc. This namespace is created on starting the interpreter, and it is never deleted.

5.1 global vs local

Related to the idea of scope is using global keyword in Python.

  • Variables created outside any function or class are considered global variables in Python. So they could be accessed for anywhere in the code even inside functions. However, if you created a variable with the same global variable inside a function it will be local to that function, but outside the function it won’t be seen since the global variable will be seen instead. –> Recall the variable resolution I have just talked about.
  • Normally variables created inside functions will be local to that function so that you couldn’t access them from outside the function unless you use the Python keyword global like this global x without assigning a value to it then you can insert a value to x and access it from outside the function like any global variable as seen in the following example.
def some_fun():
  global x
  x = 7

# for letting the code containing global x work
some_fun()

print(x)
>> 7

You could also check the nonlocal variable use in Python from here.

5.2 Resolving Variables Order – “LEGB” Order

Any variable is resolved in the LEGB order starting from L “Local Namespace” if it didn’t find the variable, it goes to E, G, and finally B “Built-in Namespace”. If it reached the B “Built-in Namespace” and didn’t find the variable defined then a NameError is raised. Let us see the code example below to further clarify how things work.

# global variable
a = 3

def resolving_variable_ex_1_fn(b):
    # local variable
    c = 5
    print(f"Local namespace or local symbol table:\n{locals()}")
    print(f"Global variable 'a' value: {globals()['a']}")
    print(f"Resolved value of the 'a' variable for the print function is: {a}")

resolving_variable_ex_1_fn(4)

>> Local namespace or local symbol table:
{'b': 4, 'c': 5}
Global variable 'a' value: 3
Resolved value of the 'a' variable for the print function is: 3

# because there is no 'a' in the local symbol table, so the value of 'a' is
# that of the global variable 'a'

# global variable
a = 3
def resolving_variable_ex_2_fn(b):
    # local variables
    a = 7
    c = 5
    print(f"Local namespace or local symbol table:\n{locals()}")
    print(f"Global variable 'a' value: {globals()['a']}")
    print(f"Resolved value of the 'a' variable for the print function is: {a}")

resolving_variable_ex_2_fn(4)

>> Local namespace or local symbol table:
{'b': 4, 'a': 7, 'c': 5}
Global variable 'a' value: 3
Resolved value of the 'a' variable for the print function is: 7

# because there is both local and global variables with the same name 'a' so
# the one that is resolved is the local variable's value since the rule of LEGB
# is followed, and the value of the 'a' global variable didn't change in both
# cases

6. Pass-By-Object-Reference

When a variable is passed to a function, it is passed as a reference to the variable’s object. Therefore, the variable is just a reference to the object so the object isn’t copied, but the reference to the object is copied. This variable (reference to the object) is copied to the function’s local symbol table. Therefore, changing the variable inside the function changes the original variable. Look at the below example for further clarification.

def append_to_list_1_fn(some_list):
    # append to the same 'some_list' object
    some_list.append(3)

# 'a' is a variable defined outside the function
a = [ ]
# now 'a' is passed by object reference for three consecutive times so we
# didn't change anything about 'a' outside the function but only inside it
append_to_list_1_fn(a)
append_to_list_1_fn(a)
append_to_list_1_fn(a)
print(a)
>> [3, 3, 3]

# however we see that the 'a' has changed, and that is because the 'a' value is
# passed by object reference. So inside the function we were referenceing to it
# will adding 3 to the list, so we actually changed the 'a' value

# but what if we don't want that, then just make a new copy of the 'a' variable
# inside the function so that a new object is created in memory on executing
# the function, and it dies on finishing the function execution, but the
# original object 'a' will not be touched, and that's it

def append_to_list_2_fn(some_list):
    # make a new object in memory
    some_list = some_list.copy()
    # append to that new object
    some_list.append(3)

# new variable 'b'
b = [ ]

append_to_list_2_fn(b)
append_to_list_2_fn(b)
append_to_list_2_fn(b)
print(b)
>> [ ]
# so 'b' is still the same (i.e. it is still empty)

7. Function Returns

All functions in Python must return a value even if that value if None. If more than one value is returned, the function returns a tuple of those values, but if only one value is returned then the function returns that value with its correct type. However, if nothing is returned then the function will return None. Here are some code examples.

def empty_1_fn():
    # nothing is returned
    pass

print(empty_1_fn())
>> None

def empty_2_fn():
    # nothing is returned
    return

print(empty_2_fn())
>> None

def return_one_value_1_fn():
    return 3

print(f"Returned Type is '{type(return_one_value_1_fn())}' with a value of"
      f" '{return_one_value_1_fn()}'.")

>> Returned Type is '<class 'int'>' with a value of '3'.

def return_one_value_2_fn():
    return 'one'

print(f"Returned Type is '{type(return_one_value_2_fn())}' with a value of"
      f" '{return_one_value_2_fn()}'.")

>> Returned Type is '<class 'str'>' with a value of 'one'.

def return_many_values_1_fn():
    return 1, 2, 3

print(f"Returned Type is '{type(return_many_values_1_fn())}' with values of"
      f" '{return_many_values_1_fn()}'.")

>> Returned Type is '<class 'tuple'>' with values of '(1, 2, 3)'.

def return_many_values_2_fn():
    return 1, 2, "three"

print(f"Returned Type is '{type(return_many_values_2_fn())}' with values of"
      f" '{return_many_values_2_fn()'}.")

>> Returned Type is '<class 'tuple'>' with values of '(1, 2, 'three')'.

# so even if the returned values have different types like integer, and string,
# they are returned in a tuple

8. Function Hidden Attributes

Since everything in Python is an object, functions are objects too. There are some attributes or tags associated with the function object that are created when you define a new function object by the def statement so this function object contains all the information you need. Some of these attributes are the function’s name __name__, the function’s documentation or description __doc__, and the function’s code object __code__ which is the actual object that makes up the code that tells Python how to execute this function. Let’s take a look at them in action.

def something_fn(something):
    return something

something_fn(3)
>> 3

print(f"Function's Name: {something_fn.__name__}")
>> Function's Name: something_fn

print(f"Function's Code: {something_fn.__code__}")
>> Function's Code: <code object something_fn at 0x7fcb2caaf780, file "<ipython-input-128-dd997212fc29>", line 1>

print(f"Function's Documentation: {something_fn.__doc__}")
>> Function's Documentation: None

# the documentation is 'None' since no documentation or description is defined
# for the function yet

# let us provide some documentation or description
def something_fn(something):
    """This is some documentation."""
    return something

print(f"Function's Documentation: {something_fn.__doc__}")
>> Function's Documentation: This is some documentation.

# or you can do it directly like this
something_fn.__doc__ = "This is an updated documentation."

print(f"Function's Documentation: {something_fn.__doc__}")
>> Function's Documentation: This is an updated documentation.

# this should give you the function's documentation as well
help(something_fn)

>> Help on function something_fn in module __main__:

something_fn(something)
    This is an updated documentation.

# hidden attributes already exist for built-in functions as well
print(f"The 'len' Function's Name: {len.__name__}")
>> The 'len' Function's Name: len

print(f"The 'len' Function's Documentation: {len.__doc__}")
>> The 'len' Function's Documentation: Return the number of items in a container.

# this should give you the 'len' function's documentation as well
help(len)

>> Help on built-in function len in module builtins:

len(obj, /)
    Return the number of items in a container.

9. What else can you do with functions?

Well, you can assign them to variables, you can return them from functions like returning any values, you can pass them as parameters to other functions, and you can even modify them by using the decorator which I intend to talk about in the future. I will just give you an example of assigning a function to a variable, because then you can deal with functions as dealing with variables by returning them from functions, passing them as parameters to other functions, and so on. Remember that everything is an object in Python.

def add_fn(a, b):
    return a + b

# now I have assigned the function to a variable (i.e. I have an
# object-reference to the function in that variable)
add_var = add_fn

# I can then use the variable normally as if it is the function
add_var(3, 8)
>> 11

# so it is working normally

Finally

Thank you. I hope this post has been beneficial to you. I would appreciate any comments if anyone needed more clarifications or if anyone has seen something wrong in what I have written in order to modify it, and I would also appreciate any possible enhancements or suggestions. We are humans, and errors are expected from us, but we could also minimize those errors by learning from our mistakes and by seeking to improve what we do.

Allah bless our master Muhammad and his family.

References

https://www.udacity.com/course/intermediate-python-nanodegree–nd303

Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments
0
Would love your thoughts, please comment.x
()
x