Exceptions and File Handling

  • Understanding Exceptions
  • Handling Exceptions
  • Defining Custom Exceptions
  • Reading/Writing Files
  • Navigating the File System

Understanding Exceptions

Things do not always go as planned. If you don't want your program to just crash you'll need to be able to handle exceptions.

Same Code but Different Style

You have a couple options.

In some cases you can do some preliminary checks to make sure that your code will successfully execute.

Or you can dive in an catch errors as they happen.

Look Before You Leap

In [1]:
a = 2
c = 200
if a != 0:
    c = 200 / a

Easier to Ask Forgiveness than Permission

In [2]:
a = 2
c = 200
try:
    c = 200 / a
except ZeroDivisionError:
    pass

Catching Exceptions

The try keyword starts the block where exceptions are to be handled. The except keyword denotes which exception classes are handled.

You can also define an else block which will only excute if no exceptions were raised. A finally block is also optional and will be excuted regardless of whether there were exceptions or not.

In [3]:
try:
    # Something dangerous
    pass
except IndexError:
    # Handle the error
    pass
else:
    # No problems so do something
    pass
finally:
    # Clean up either way
    pass

Catching Multiple Types of Exceptions

You can handle multiple types of exceptions in one except block.

In [4]:
try:
    # Something dangerous
    pass
except (TypeError, IndexError):
    # Handle either type
    pass

Handling Different Exceptions Differently

You can also define multiple except cases to handle different exception types in different ways.

In [5]:
try:
    # Something dangerous
    pass
except TypeError:
    # Handle type error
    pass
except IndexError:
    # Handle index error
    pass
except:
    # Handle all other types
    pass

Getting Exception Info

In [6]:
try:
    # Something dangerous
    'a' + 1
except TypeError as e:
    print(e)
    print(e.args)
    # Handle type error
must be str, not int
('must be str, not int',)

Built-in Exceptions

Some common exception classes:

  • Exception - Base exception class
  • AttributeError - Attempted to access an object attribute that doesn't exist
  • IOError - I/O related error (file not found, disk full, etc)
  • ImportError - Module import error
  • IndexError - Accessing index outside of list
  • KeyError - Accessing dictionary key that doesn't exist

For a full list see http://docs.python.org/library/exceptions.html

Creating Exceptions

Creating exceptions is as easy as creating a class.

In [7]:
class EveryonePanicException(Exception):
    pass

Creating Exceptions Expanded

As with any class you can also pass additional information into your exceptions.

In [8]:
class EveryonePanicException(Exception):
    
    def __init__(self, reason):
        self.reason = reason

    def __str__(self):
        return 'Everyone panic! %s' % self.reason

Raising Exceptions

The raise keyword is used to raise the specified exception.

In [9]:
raise EveryonePanicException("It's Godzilla!")
---------------------------------------------------------------------------
EveryonePanicException                    Traceback (most recent call last)
<ipython-input-9-8856e3f3e26e> in <module>()
----> 1 raise EveryonePanicException("It's Godzilla!")

EveryonePanicException: Everyone panic! It's Godzilla!

Raising Exceptions Again

If you've caught an exception that you don't intend to handle then you can re-raise the last exception with raise.

In [10]:
try:
    raise EveryonePanicException("It's Godzilla!")
except EveryonePanicException:
    print("There was an exception.")
    raise
There was an exception.
---------------------------------------------------------------------------
EveryonePanicException                    Traceback (most recent call last)
<ipython-input-10-4a848a688b40> in <module>()
      1 try:
----> 2     raise EveryonePanicException("It's Godzilla!")
      3 except EveryonePanicException:
      4     print("There was an exception.")
      5     raise

EveryonePanicException: Everyone panic! It's Godzilla!

Supporting Different Versions

lxml is a high-performance XML parser which supports the same API as the XML parser in the standard library. You can fallback to the standard libary version if it isn't installed/available.

In [11]:
try:
  from lxml import etree
except ImportError:
  try:
    import xml.etree.cElementTree as etree
  except ImportError:
    import xml.etree.ElementTree as etree
    
etree.__file__
Out[11]:
'/usr/lib/python3.6/xml/etree/cElementTree.py'

Opening Files

The built-in open function is used to open files. It takes the filename, mode (optional), and buffer size (optional). This is implemented as stdio fopen() in the underlying C. The mode defaults to 'r' (for reading).

In [12]:
open_file = open('MA792-002-Python-4.ipynb')
contents = open_file.readlines() # Reads entire file
open_file.close()

The File Object

Let's take a look at what methods are on the file type.

In [13]:
dir(open_file)
Out[13]:
['_CHUNK_SIZE',
 '__class__',
 '__del__',
 '__delattr__',
 '__dict__',
 '__dir__',
 '__doc__',
 '__enter__',
 '__eq__',
 '__exit__',
 '__format__',
 '__ge__',
 '__getattribute__',
 '__getstate__',
 '__gt__',
 '__hash__',
 '__init__',
 '__init_subclass__',
 '__iter__',
 '__le__',
 '__lt__',
 '__ne__',
 '__new__',
 '__next__',
 '__reduce__',
 '__reduce_ex__',
 '__repr__',
 '__setattr__',
 '__sizeof__',
 '__str__',
 '__subclasshook__',
 '_checkClosed',
 '_checkReadable',
 '_checkSeekable',
 '_checkWritable',
 '_finalizing',
 'buffer',
 'close',
 'closed',
 'detach',
 'encoding',
 'errors',
 'fileno',
 'flush',
 'isatty',
 'line_buffering',
 'mode',
 'name',
 'newlines',
 'read',
 'readable',
 'readline',
 'readlines',
 'seek',
 'seekable',
 'tell',
 'truncate',
 'writable',
 'write',
 'writelines']

Opening Files Safely

You can avoid having to remember to close the file by opening the file using a with statement:

In [14]:
with open('../code/example.txt') as f:
    # do something with f
    print(f.readlines())
["I'm in a file.\n"]

With Statement

To take a small detour let's talk about the with statement.

The with statement is used to wrap a code block called a context manager. The context manager defines an __enter__ to setup the context and __exit__ to clean up the code execution. The common use case is reusing try/except blocks for opening/closing resources.

In [15]:
# %load ../code/withexample.py
class Example(object):

    def __enter__(self):
        print('Calling Enter')
        return 73

    def __exit__(self, exc_type, exc_value, traceback):
        print('Calling Exit: %s, %s, %s' % (exc_type, exc_value, traceback))
        # This will stop the exception
        # from being propagated
        return True
In [16]:
with Example() as ex:
    print(ex)
Calling Enter
73
Calling Exit: None, None, None
In [17]:
with Example() as ex:
    raise Exception
Calling Enter
Calling Exit: <class 'Exception'>, , <traceback object at 0x7f12ecd8f5c8>

contextlib

The contextlib module in the standard libary provides some utilities for working with and writing your own context managers to reduce some of the boilerplate.

The contextmanager decorator can be used to create a simple context manager from a generator function.

In [18]:
import contextlib
import time

@contextlib.contextmanager
def timer():
    start = time.time()
    yield
    result = time.time() - start
    print('It took {:.02f} seconds'.format(result))
In [19]:
with timer():
    time.sleep(0.25)
It took 0.25 seconds

Reading Files

The file objects have a number of methods for reading content. readlines reads all of the file conents to the EOF character. readline reads a single line including the new line character. You can also read lines in a file using an interator syntax.

In [20]:
with open('../code/example.txt') as f:
    for line in f:
        print(line)
I'm in a file.

A Program Which Outputs Itself

See code/output.py

Writing Files

You can write a single string with write or a list of strings with writelines. Keep in mind that neither method will automatically write new line characters for you.

In [21]:
with open('example1.txt', 'w') as f:
    f.write('Line 1\n')
    f.writelines(['Line 2\n', 'Line 3\n'])
In [22]:
# Appending Files
with open('example2.txt', 'w') as f:
    f.write('First pass.\n')

with open('example2.txt', 'a') as f:
    f.write('This is new.\n')

File Paths

The os module has helper functions for working with file paths. Some handy functions are

  • os.getcwd
  • os.path.abspath
  • os.path.dirname
  • os.path.join
  • os.path.splitext

You can also get the relative file using the module's __file__ attribute.

Directories

The os module also has helper functions for working with directories.

  • os.listdir
  • os.path.walk
  • os.mkdir
  • os.makedirs
  • os.remove
  • os.rmdir
  • os.removedirs

pathlib

The os.path module has many function-based utilities for working with paths and file objects. If you prefer a more object-oriented approach then you can use pathlib instead.

In [23]:
from pathlib import Path

path = Path('/var')
syslog = path / 'log' / 'syslog'
syslog
Out[23]:
PosixPath('/var/log/syslog')
In [24]:
syslog.exists()
Out[24]:
True
In [25]:
syslog.is_dir()
Out[25]:
False
In [26]:
path.is_dir()
Out[26]:
True

Modules Special File Types

  • XML
    • DOM (Document Object Model): xml.dom
    • SAX (Simple API for XML): xml.sax:
  • CSV
  • JSON

Up Next

Functional programming in Python