Tools to Write Better Python

• flake8
• pdb/ipdb
• IPython

flake8

The package flake8 is a suite of packages for code linting/static analysis of Python projects. It's easy to configure and use. There are plugins which can be added for more checks.

flake8 is composed of

• pycodestyle for checking PEP8 style consistency
• pyflakes for static analysis
• mccabe for checking McCabe complexity

What is Linting

The term linting comes from the lint program for checking C source code for suspicious and non-portable constructs. In Python this can be shadowing a built-in name, using the * import, or having a bare exception handler.

Static Analysis

Static analysis refers to checking the program source without importing or running it. Because Python code can potentially have side effect when it's imported, using static analysis is generally preferred.

Example Violations

# %load ../code/ugly.py
import os

start =  1
end = 10

for i in range(start, end):
    print(i)

!flake8 ../code/ugly.py

../code/ugly.py:1:1: F401 'os' imported but unused
../code/ugly.py:3:8: E222 multiple spaces after operator

Finding a Bug

# %load ../code/buggy.py
def hypotenuse(leg1, leg2):
    return (leg1 ** 2 + lge2 ** 2) ** 0.5

!flake8 ../code/buggy.py

../code/buggy.py:2:25: F821 undefined name 'lge2'

Overly Complex Code

# %load ../code/complicated.py
def group_name(animal):
    name = None
    while name is None and not animal.endswith('s'):
        if animal == 'ants':
            return 'colony'
        elif animal == 'bees':
            return 'hive'
        elif animal == 'cats':
            return 'litter'
        elif animal == 'dogs':
            return 'pack'
        animal += 's'
    return name

!flake8 --max-complexity=5 ../code/complicated.py

../code/complicated.py:1:1: C901 'group_name' is too complex (6)

From the original paper 10 is considered to be too complex and that's the default.

Ignoring Errors

# %load ../code/ignore.py
message = 'I know that this line is too long but I don\'t want to break up this sentence because it would be awkward.'  # noqa

!flake8 ../code/ignore.py

Some Available Plugins

• flake8-import-style/flake-isort
• flake8-pytest
• flake8-todo
• flake8-commas
• flake8-debugger

pdb - Python Debugger

pdb is included with Python to provide debugging capabilities. You can set a breakpoint and step through the code

ipdb is a similarly improved interactive debugger which is included with ipython/juptyer.

pdb Usage

Breakpoints are set with pdb using set_trace()

import pdb; pdb.set_trace()

When the program executes it will stop and allow you to interact with the currently running environment.

This doesn't have to be on a single line but you'll often find it that way so it can be easily removed later.

You don't want to leave these in your code because it will stop your code from running.

pdb Commands

When at a breakpoint you can inspect the current namespace as well as step through the program execution using the following commands:

• s - Execute the current line, stop at the first possible occasion.
• n - Continue execution until the next line in the current function is reached or it returns.
• r - Continue execution until the current function returns.
• l - List source code for the current file.
• c - Continue until the next breakpoint.

pdb Demo

We are going to do a little debugging on the Github API code from previous lectures and slowly step through the execution.

Set a break point inside the try block and step through. Examine the returned response and some of the inner workings of requests.

IPython

We touched on IPython/jupyter in previous lectures but I wanted to give more examples of things that IPython can do that the standard interpreter does not.

• Tab completion
• Input/output caching
• Command history
• Shell commands
• 🦄 Magic 🦄

Tab Completion

Within the IPython shell you can tab complete imports, local names, built-ins, well as object properties/methods.

You suffix any object name with ? to get more information about that object.

Input/Output Caching

The In and Out variables noted on the side can be used to examine previous input and output.

_, __, and ___ reference the last three outputs.

Command History

The IPython shell keeps track of previous command history. You can see the input history through the %history command.

You can return the previous command with %rerun.

You can save a set of commands to a file via %save

Shell Commands

Using the ! prefix you can run shell commands inside of IPython. You can also capture the resulting output and store it in Python names.

result = !ls
result

['example1.txt',
 'example2.txt',
 'example.log',
 'example.txt',
 'MA792-002-Python-1.ipynb',
 'MA792-002-Python-2.ipynb',
 'MA792-002-Python-3.ipynb',
 'MA792-002-Python-4.ipynb',
 'MA792-002-Python-5.ipynb',
 'MA792-002-Python-6.ipynb',
 'MA792-002-Python-7.ipynb',
 'MA792-002-Python-8.ipynb',
 'mandel.png']

Magic Commands

Commands which begin with % are called magic commands in IPython. There are many of them so here are my highlights:

• %timeit
• %prun
• %load
• %debug
• %reset

For all availalbe magic commands see: http://ipython.readthedocs.io/en/stable/interactive/magics.html

More Help

• Python Website: https://www.python.org/
• Python Package Index: https://pypi.org/
• Local Python Meetup/User Groups:
  • TriPython - http://trizpug.org/ and https://www.meetup.com/tripython/
  • PyLadies RDU - https://www.meetup.com/pyladies-rdu/