Python Functions - def, *arg, **kwargs

The def create a function object and assigns it to a name. Its general format is:

def <name>(arg1, arg2,...,argN)
	<statement>

The statement block becomes the function's body. The def header line specifies a function name and the function bodies often contain a return statement:

def <name>(arg1, arg2,...,argN)
	return <value>

The notable thing here is that the function doesn't define a return datatype. Python functions do not specify the datatype of their return value. They don't even specify whether or not they return a value. Actually, however, every Python function returns a value if the function ever executes a return statement, and it will return that value. Otherwise, it will return None.

Everything in Python is a function, all functions return a value even if it is None, and all functions start with def.

Here are brief descriptions:

1. def is an executable code.
   Python functions are written with a new statement, the def. Unlike functions in compiled language def is an executable statement. Our function does not exist until Python reaches and runs the def. Actually, it's legal to nest def statements inside if statement, while loops, and even other defs. In general, def statements are coded in module files and are naturally run to generate functions when a module file is first imported.


2. def creates an object and assigns it to a name.
   When Python reaches and runs a def statement, it generates a new function object and assigns it to the function's name. As with all assignments, the function name becomes a reference to the function object. There's nothing magic about the name of a function. The function object can be assigned to other names, stored in a list, and so son. Functions objects may also have arbitrary user-defined attributes attached to them to record data.


3. return sends a result object back to the caller.
   When a function is called, the caller stops until the function finishes its work and returns control to the caller. Functions that compute a value send it back to the caller with a return statement. The returned value becomes the result of the function call.

The def is an executable statement. When it runs, it creates a new function object and assigns it to a name. Because it's a statement, a def can appear anywhere a statement can even nested in other statements:

# function.py
def func():
    print('func()')
    def func1():
        print('func1')
        def func2():
            print('func2')
        func2()
    func1()

func()

Output should look like this:

$ python function.py
func()
func1
func2

Because function definition happens at runtime, there's nothing special about the function name. What's important is the object to which it refers:

>>> def func():
    print('func()')
    def func1():
        print('func1')
        def func2():
            print('func2')
        func2()
    func1()

>>> othername = func    # Assign function object
>>> othername()         # Call func again
func()
func1
func2
>>> othername2 = func() # Call func one more time
func()
func1
func2

Here, the function was assigned to a different name and called through the new name. Functions are just object. They are recorded explicitly in memory at program execution time. In fact, besides calls, functions allow arbitrary attributes to be attached to record information for later use:

def func():...		# Create function object
func()			# Call object
func.attr = value	# Attach attributes

Here, we typed the definition of a function, times, interactively. It returns the product of its two arguments:

>>> # Create and assign function
>>> def times(x, y):     # Create and assign function
	return x * y     # Body executed when called

>>> times(2, 5)          # Arguments in parentheses
10

When Python reaches and runs this def, it creates a new function object that packages the function's code and assigns the object to the name times. After the def has run, we can call (run) the function as shown above. The times function's body is just a return statement that sends back the result as the value of the call. If we want to use it later we could instead assign it to a variable:

>>> x = times(365,5)
>>> x
1825

Let's call the function again with different objects passed in:

>>> times('blah!', 4)
'blah!blah!blah!blah!'

That was possible because we never declared the types of variables , argument, or return values in Python. We can use times to either multiply numbers or repeat sequences. In other words, what our times function means depends on what we pass into it. This is core idea in Python and it is polymorphism.

Let's make a function that collects items held in common in two strings:

>>> def intersect(seq1, seq2):
	res = []			# Start empty
	for x in seq1:			# Scan seq1
		if x in seq2:		# Common item?
			res.append(x)	# Add to the end
	return res

The algorithm of the function is:
for every item in the first argument, if that item is also in the second argument, append the item to the result.

Now, let's call the intersect function:

>>> s1 = "WORLD"
>>> s2 = "WORDS"
>>> intersect(s1, s2)
['W', 'O', 'R', 'D']

We could have used a single list comprehension expression giving the same result:

>>> 
>>> [x for x in s1 if x in s2]
['W', 'O', 'R', 'D']
>>> 

The function intersect is polymorphic. It works on arbitrary types as long as they support the expected object interface:

>>> 
>>> x = intersect([1, 2, 3],(1, 4))	# Mixed types
>>> x
[1]
>>> 

We passed in different types of objects: a list and a tuple. It's working because we don't have to specify the types of argument ahead of time. For intersect, this means that the first argument should support the for loop and the second has to support the in membership test. If we pass in objects that do not support these interfaces (e.g., numbers), Python will detect mismatch and raise an exception:

>>> 
>>> x = intersect(1, 4)
Traceback (most recent call last):
  File "<pyshell#35>", line 1, in <module>
    x = intersect(1, 4)
  File "<pyshell#22>", line 3, in intersect
    for x in seq1:			# Scan seq1
TypeError: 'int' object is not iterable
>>> 
</pyshell#22></module></pyshell#35>

The variable res inside intersect is what is called a local variable. It is visible only to code inside the function def and that exists only while the function runs. All names assigned inside a function are classified as local variables by default. They appear when the function is called and disappear when the function exits. The return statement at the end of intersect sends back the result object, but the name res goes away.

Here is the keyword example:

>>> def f(a, b, c):
	print(a, b, c)
	
>>> f(1, 2, 3)
1 2 3
>>> f(c=3, b=2, a=1)
1 2 3
>>> f(1, c=3, b=2)
1 2 3

>>> def f(a, b=2, c=3):
	print(a, b, c)

>>> f(1)
1 2 3
>>> f(10, 50)
10 50 3
>>> f(10, 70, 90)
10 70 90
>>> f(10, 200)
10 200 3

Table from Learning Python by Mark Lutz, 2009.

Putting *args and/or **kwargs as the last items in our function definition's argument list allows that function to accept an arbitrary number of anonymous and/or keyword arguments.
Those arguments are called Keyword Arguments. Actually, they are place holders for multiple arguments, and they are useful especially when we need to pass a different number of arguments each time we call the function.

We may want to use *args when we're not sure how many arguments might be passed to our function, i.e. it allows us to pass an arbitrary number of arguments to our function.

>>> def f(*args):
	print(args)
	
>>> f()
()
>>> f(10)
(10,)
>>> f(10, 20, 30)
(10, 20, 30)

When the f() is called, Python collects all the positional arguments into a new tuple and assigns the variable args to that tuple. Since it is a normal tuple object, it can be indexed:

>>> def print_all(*args):
	for x in enumerate(args):
		print x
		
>>> print_all('A','b','b','a')
(0, 'A')
(1, 'b')
(2, 'b')
(3, 'a')

We can pass in lists for the args:

>>> def print_all(*args):
...    for x in enumerate(args):
...       print x
... 
>>> print_all([1,2,3],[4,5,6])
(0, [1, 2, 3])
(1, [4, 5, 6])

The ** is similar but it only works for keyword arguments. In other words, it collects them into a new dictionary. Actually, ** allows us to convert from keywords to dictionaries:

>>> def f(**kwargs):
	print(kwargs)
	
>>> f()
{}
>>> f(a=10, b=20)
{'a': 10, 'b': 20}

The keyword arguments is a special name=value syntax in function calls that specifies passing by name. It is often used to provide configuration options.

>>> def kwargs_function(**kwargs):
	for k,v in kwargs.items():
		print (k,v)

>>> kwargs_function(**{'uno':'one','dos':'two','tres':'three'})
('dos', 'two')
('tres', 'three')
('uno', 'one')
>>>
>>> kwargs_function(dos='two', tres='three', uno='one')
('dos', 'two')
('tres', 'three')
('uno', 'one')

We can use the * or ** when we call a function. In other words, it unpacks a collection of arguments, rather than constructing a collection of arguments. In the following example, we pass five arguments to a function in a tuple and let Python unpack them into individual arguments:

>>> def f(a, b, c, d, e):
	print(a, b, c, d, e)
	
>>> args = (10, 20)
>>> args += (30, 40, 50))
>>> f(*args)
10 20 30 40 50

In the same way, the ** in a function call unpacks a dictionary of key/value pairs into separate keyword arguments:

>>> kwargs = {'a':10, 'b':20, 'c':30}
>>> kwargs['d']=40
>>> kwargs['e']=50
>>> f(**kwargs)
10 20 30 40 50

Also, with various combinations:

>>> f(*(10, 20), **{'d':40, 'e':50, 'c':30})
10 20 30 40 50
>>> f(10, *(20, 30), **{'d':40, 'e':50})
10 20 30 40 50
>>> f(10, c = 30, *(20,), **{'d':40, 'e':50})
10 20 30 40 50
>>> f(10, *(20,30), d=40, e=50)
10 20 30 40 50
>>> f(10, *(20,), c=30, **{'d':40, 'e':50})
10 20 30 40 50

Let's test what we've learned so far:

def fnc(*args, **kwargs):
   print('{} {}'.format(args, kwargs))

print('fnc()')
fnc()
fnc(1,2,3)
fnc(1,2,3,'flask')
fnc(a=1, b=2, c=3)
fnc(a=1, b=2, c=3, d='ansible')
fnc(1, 2, 3, a=1, b=2, c=3)

lst = [1,2,3]
tpl = (4,5,6)
dct = {'a':7, 'b':8, 'c':9}

fnc(*lst, **dct)
fnc(*tpl, **dct)

fnc(1,2,*lst)
fnc(1,2,*tpl)
fnc('jupyter',**dct)
fnc(arg='django',**dct)
fnc(1,2,*tpl,q='bottle',**dct)
print

def fnc2(arg1, arg2, *args, **kwargs):
   print('{} {} {} {}'.format(arg1, arg2, args, kwargs))

print('fnc2()')
#fnc2() # error
fnc2(1,2)
fnc2(1,2,3,'haystack')
fnc2(arg1=1, arg2=2, c=3)
fnc2(arg1=1, arg2=2, c=3, d='Spark')
fnc2(1,2,3, a=1, b=2)
fnc2(*lst, **dct)
fnc2(*tpl, **dct)
fnc2(1,2,*tpl)
fnc2(1,*tpl,d='nltk')
fnc2(1,2,*tpl,d='scikit')

Output:

fnc()
() {}
(1, 2, 3) {}
(1, 2, 3, 'flask') {}
() {'a': 1, 'c': 3, 'b': 2}
() {'a': 1, 'c': 3, 'b': 2, 'd': 'ansible'}
(1, 2, 3) {'a': 1, 'c': 3, 'b': 2}
(1, 2, 3) {'a': 7, 'c': 9, 'b': 8}
(4, 5, 6) {'a': 7, 'c': 9, 'b': 8}
(1, 2, 1, 2, 3) {}
(1, 2, 4, 5, 6) {}
('jupyter',) {'a': 7, 'c': 9, 'b': 8}
() {'a': 7, 'c': 9, 'b': 8, 'arg': 'django'}
(1, 2, 4, 5, 6) {'a': 7, 'q': 'bottle', 'c': 9, 'b': 8}

fnc2()
1 2 () {}
1 2 (3, 'haystack') {}
1 2 () {'c': 3}
1 2 () {'c': 3, 'd': 'Spark'}
1 2 (3,) {'a': 1, 'b': 2}
1 2 (3,) {'a': 7, 'c': 9, 'b': 8}
4 5 (6,) {'a': 7, 'c': 9, 'b': 8}
1 2 (4, 5, 6) {}
1 4 (5, 6) {'d': 'nltk'}
1 2 (4, 5, 6) {'d': 'scikit'}

In the code below, we support any function with any arguments by passing along whatever arguments that were sent in:

>>> def A_function(f, *args, **kwargs):
	return f(*args, **kwargs)

>>> def f(a, b, c, d, e):
	return a*b*c*d*e

>>> print(A_function(f, 10, 20, c=30, d=40, e=50))
12000000

When the code is run, arguments are collected by the A_function.

f(*args, **kwargs)         # newer call syntax: f(*sequence, **dict)
apply(f, args, kwargs)     # deprecated built-in: apply(f, sequence, dict)

The following function accepts any number of positional or keyword arguments:

>>> def echo(*args, **kwargs):
	print(args, kwargs)

>>> echo(10, 20, a=30, b=40)
(10, 20) {'a': 30, 'b': 40}

If we use the apply():

>>> args = (10, 20)
>>> kwargs = {'a':30, 'b':40}

>>> def echo(*args, **kwargs):
	print(args, kwargs)
	
>>> apply(echo, args, kwargs)
((10, 20), {'a': 30, 'b': 40})
>>> echo(*args, **kwargs)
((10, 20), {'a': 30, 'b': 40})