Advanced python CoP

Cool early python features you didn't know about that might be older than you

In [3]:
%%HTML
<img src="https://pbs.twimg.com/media/EUN22kzXYAIxrtJ?format=jpg&name=900x900">

Python 1.x

xD

Printing of recursive dictionaries and lists no longer causes a core dump.

Functional programming: lambda, map, filter, reduce

What does this do?

In [2]:
some_numbers = range(11, 20)
list(map(lambda n: sum(map(int, str(n))), filter(lambda x: x % 2 == 0, some_numbers)))
Out[2]:
[3, 5, 7, 9]

Although map and filter are useful, in most cases list comprehension is more readable

In [5]:
[sum(int(d) for d in str(n)) for n in some_numbers if n % 2 == 0]
Out[5]:
[3, 5, 7, 9]

And the shortest version, because we all love python for brevity

In [6]:
[sum(map(int, str(n))) for n in some_numbers if n % 2 == 0]
Out[6]:
[3, 5, 7, 9]
In [7]:
%%HTML
<blockquote class="reddit-card" data-card-created="1587309184"><a href="https://www.reddit.com/r/ProgrammerHumor/comments/ftupjh/behind_every_oneliner_there_are_the_tears_of_many/">Behind every one-liner there are the tears of many</a> from <a href="http://www.reddit.com/r/ProgrammerHumor">r/ProgrammerHumor</a></blockquote>
<script async src="//embed.redditmedia.com/widgets/platform.js" charset="UTF-8"></script>
Behind every one-liner there are the tears of many from r/ProgrammerHumor

assert with message

In [8]:
if False is not True:
    raise AssertionError("Nope")

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-8-96484b3a7600> in <module>
      1 if False is not True:
----> 2     raise AssertionError("Nope")

AssertionError: Nope
In [9]:
assert False is True, "Nope"

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-9-a113ba9ca585> in <module>
----> 1 assert False is True, "Nope"

AssertionError: Nope

Python 2.0 - October 16, 2000

https://docs.python.org/3/whatsnew/2.0.html

List comprehensions

In [10]:
[i ** 2 for i in range(10) if i % 2 == 0]
Out[10]:
[0, 4, 16, 36, 64]
In [11]:
numbers = []
for i in range(3):
    for j in range(3):
        numbers.append((i, j))
numbers
Out[11]:
[(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2), (2, 0), (2, 1), (2, 2)]
In [12]:
[(i, j) for i in range(3) for j in range(3)]
Out[12]:
[(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2), (2, 0), (2, 1), (2, 2)]
In [13]:
import random

[[random.choice(["O","X"," "]) for _ in range(3)] for _ in range(3)]
Out[13]:
[['O', ' ', 'O'], ['X', 'X', ' '], ['O', ' ', 'O']]

Args and kwargs

In [14]:
def f(*args, **kw):
    # args is a tuple of positional args,
    # kw is a dictionary of keyword args
    print(locals())

a = [1, 2]
b = {"a": 1, "b": 3}
f(*a, **b)
f(1, 2, a=1, b=3)

{'args': (1, 2), 'kw': {'a': 1, 'b': 3}}
{'args': (1, 2), 'kw': {'a': 1, 'b': 3}}

Without kwargs:

In [15]:
import subprocess

subprocess.run("ls", stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
subprocess.run("whoami", stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
subprocess.run("pwd", stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True, check=True)
Out[15]:
CompletedProcess(args='pwd', returncode=0, stdout=b'/home/gjklv8/repos/advanced-python-course/whats_new_00\n', stderr=b'')

with kwargs:

In [16]:
run_kwargs = {"stdout": subprocess.PIPE, "stderr": subprocess.PIPE, "shell": True}
subprocess.run("ls", **run_kwargs)
subprocess.run("whoami", **run_kwargs)
subprocess.run("pwd", **run_kwargs, check=True)
Out[16]:
CompletedProcess(args='pwd', returncode=0, stdout=b'/home/gjklv8/repos/advanced-python-course/whats_new_00\n', stderr=b'')

zip

In [17]:
list(zip("ab","cd"))
Out[17]:
[('a', 'c'), ('b', 'd')]

Very useful when you want to iterate over multiple sequences in one for loop

In [18]:
[a + b for a, b in zip("ab", "cd")]
Out[18]:
['ac', 'bd']
In [20]:
numbers = [1, 2, 3, 4]

# iterate over pairs in the same sequence
[(a, b) for a, b in zip(numbers, numbers[1:])]
Out[20]:
[(1, 2), (2, 3), (3, 4)]

atexit

In [21]:
import atexit

@atexit.register
def goodbye():
    print("You are now leaving the Python sector.")

Python 2.1 - April 17, 2001

Nested scopes

It was already possible to define function inside another function, but the inner didn't have access to enclosing function scope

In Python 2.0, at any given time there are at most three namespaces used to look up variable names: local, module-level, and the built-in namespace

In [ ]:
# module-scope

def f():
    # local scope
    pass

# builtin scope
__builtin__.print
In [ ]:
def f():
    x = 1
    l = lambda: print(x) # NameError in python2.0
    
    def g():
        print(x) # NameError in python2.0
    
    def g2(n):
        if n > 0:
            return n * g2(n-1) # NameError in python2.0
In [23]:
for i in range(3):
    # not a scope
    pass

i # i leaked to the enclosing scope
Out[23]:
2

comprehensions don't leak variables to the enclosing scope

In [24]:
[unique_name for unique_name in range(3)] 

unique_name

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-24-f72037e85152> in <module>
      1 [unique_name for unique_name in range(3)]
      2 
----> 3 unique_name

NameError: name 'unique_name' is not defined
In [6]:
def f():
    a_abc = 2
    b_abc = 3
    c = 4
    print(locals())
    return [locals()[name] for name in locals() if name.endswith("_abc")]

f()

{'a_abc': 2, 'b_abc': 3, 'c': 4}
Out[6]:
[2, 3]
In [31]:
%%HTML
<img src="https://media.makeameme.org/created/what-just-happened-5aaa59.jpg">

To understand why this happened we need to know how the iteration works in python:

In [4]:
l = [1,2,3]
it = iter(l)
while True:
    try:
        i = next(it)
        print(i)
    except StopIteration:
        break

1
2
3
In [32]:
import inspect

class LoudIterable:
    def __iter__(self):
        print(inspect.currentframe())
        return iter([1, 2])

x = [print(inspect.currentframe()) for i in LoudIterable()]

<frame at 0x7efe96388ba0, file '<ipython-input-32-fffd0b1d2f50>', line 5, code __iter__>
<frame at 0x7efe962ee040, file '<ipython-input-32-fffd0b1d2f50>', line 8, code <listcomp>>
<frame at 0x7efe962ee040, file '<ipython-input-32-fffd0b1d2f50>', line 8, code <listcomp>>

Basically, iter is called in a diferent scope than comprehension body

Very useful feature when it comes to generators

In [14]:
l = (print(i) for i in range(3))
In [16]:
non_iterable = 2
x = (i for i in non_iterable)
print("We won't reach this line, even though the iteration haven't started yet")

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-16-5604b753fe89> in <module>
      1 non_iterable = 2
----> 2 x = (i for i in non_iterable)
      3 print("We won't reach this line, even though the iteration haven't started yet")

TypeError: 'int' object is not iterable

Btw, common scoping gotcha:

In [30]:
funcs = [lambda: print(i) for i in range(3)]

for f in funcs:
    f()

2
2
2

Since we're talking about scopes: name resolution rules

When a name is used in a code block, it is resolved using the nearest enclosing scope.

In [21]:
x = 1

def f():
    print(x)

f()
print(x)

1
1
In [22]:
x = 1

def f():
    x = 2
    print(x)

f()
print(x)

2
1
In [24]:
x = 1

def f():
    x += 1
    print(x)

f()
print(x)

---------------------------------------------------------------------------
UnboundLocalError                         Traceback (most recent call last)
<ipython-input-24-100ea2cd66a4> in <module>
      5     print(x)
      6 
----> 7 f()
      8 print(x)

<ipython-input-24-100ea2cd66a4> in f()
      2 
      3 def f():
----> 4     x += 1
      5     print(x)
      6 

UnboundLocalError: local variable 'x' referenced before assignment
In [25]:
x = 1

def f():
    global x
    x = x + 1
    print(x)

f()
print(x)

2
2
In [26]:
x = 0

def outer():
    x = 1

    def f():
        nonlocal x
        x = x + 1
        print(x)

    f()
    print(x)
outer()
print(x)

2
2
0

PEP 230: Warning Framework

In [27]:
import warnings

def deprecated_function():
    warnings.warn("will be removed tomorrow!", category=DeprecationWarning)

deprecated_function()

<ipython-input-27-722835bcd232>:4: DeprecationWarning: will be removed tomorrow!
  warnings.warn("will be removed tomorrow!", category=DeprecationWarning)

PEP 205: Weak References

Python has a reference counting garbage collector

In [ ]:
import sys

class A:
    def hello(self):
        print("hello")

a = A()
sys.getrefcount(a)
In [ ]:
a_ref = a
sys.getrefcount(a)
In [ ]:
del a
sys.getrefcount(a_ref)
In [ ]:
import weakref

a_weakref = weakref.ref(a_ref)
In [ ]:
a_weakref().hello()
In [ ]:
sys.getrefcount(a_ref)
In [ ]:
del a_ref

import gc

gc.collect() # force garbage collection
In [ ]:
a_weakref() is None

PEP 232: Function Attributes

In [ ]:
def f():
    if not hasattr(f, "_call_counter"):
        f._call_counter = 1
    f._call_counter +=1
In [ ]:
f()
In [ ]:
f._call_counter
In [ ]:
f()
In [ ]:
f._call_counter

Modules can now control which names are imported when from module import * is used, by defining an all attribute containing a list of names that will be imported

Python 2.2 - October 14, 2002

TODO

In [ ]: