Tour of the Python Standard Library ---------------------------------------- - random - datetime - urllib/urllib2 - glob - re - logging - timeit - unittest What is the Standard Library? ---------------------------------------- Python has a “batteries included” philosophy meaning the language library ships with a number of robust modules. Today we're going to look a few of them. Being a good Python programmer isn't about being able to write any program in Python. It's also about knowing which things to write and which things are already written, much like being a good mathematician. Modules We've Already Seen/Discussed ---------------------------------------- - math/cmath - sys - os - xml - csv - json - itertools Python 3 Notes ---------------------------------------- The move to Python 3 included some module clean up in terms of both naming and content. For instance ``urllib`` and ``urllib2`` are now one package called ``urllib``. ``ConfigParser`` was renamed to ``configparser`` to match the PEP8 naming conventions. ``random`` ---------------------------------------- The ``random`` module contains classes and functions for generating random data of various types and distributions. .. code-block:: python import random print random.random() # Random float from [0.0, 1.0) print random.randint(1, 10) # Random integer from [1, 10] print random.uniform(3, 5) # Uniform float from [3, 5] # Randomly selected item from the given list/iterable print random.choice(['a', 'b', 'c']) ``datetime`` ---------------------------------------- The ``datetime`` module defines ``datetime``, ``date``, ``time``, and ``timedelta`` types for handling date arithmetic and date comparison. Note all of these types are immutable. .. code-block:: python import datetime today = datetime.date.today() tomorrow = today + datetime.timedelta(days=1) if today < tommorow: print today.isoformat() Date Formatting (dateformat.py) ---------------------------------------- These objects have a ``strftime`` method which allows you to represent the date/time in various formats. All allowable formatting directives can be found here http://docs.python.org/library/datetime.html#strftime-strptime-behavior .. literalinclude:: /code/dateformat.py Date Parsing (dateparse.py) ---------------------------------------- The ``datetime.strptime`` function takes a string representing a date and another string representing the format and returns a ``datetime`` object if it can be found. .. literalinclude:: /code/dateparse.py ``urllib``/``urllib2`` ---------------------------------------- ``urllib`` and ``urllib2`` are modules for accessing data over the internet. There are methods for retriving data as well as building urls and encoding query parameters. Both modules contain a ``urlopen`` function for accessing remote resources but ``urllib.urlopen`` cannot handle HTTPS, server timeouts, or proxies which require authentication. For simple resources you can use either but more complex requests should use ``urllib2``. Twitter Data Example (twitter.py) ---------------------------------------- .. literalinclude:: /code/twitter.py :lines: 1-6 More in the file. ``glob`` ---------------------------------------- The ``glob`` module is used to find filenames matching a given pattern. You can use ``*`` and ``?`` wildcard characters as well as ``[]`` character ranges. The rules used match the Unix shell. .. code-block:: python import glob print glob.glob('../lectures/s*.rst') print glob.glob('../lectures/s??.rst') ``re`` ---------------------------------------- Python's support for regular expressions are contained in the ``re`` module. Regular expresssions (or regex) are used for matching character patterns in strings. While they are very useful and powerful they can also get quite complicated. .. epigraph:: Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems. -- Jamie Zawinski Basic Regex Example (regex.py) ---------------------------------------- .. literalinclude:: /code/regex.py :lines: 5-14 ``logging`` ---------------------------------------- ``print`` statements are great but sometimes you need something a little bit more robust. The ``logging`` module contains support for logging programs via console output, file output (including file rotation), socket ouput, email output via SMTP, and HTTP output. You can configure multiple loggers for a program each with different formats, outputs, and logging levels. Log Configuration Example (log.conf) ---------------------------------------- .. literalinclude:: /code/log.conf :lines: 1-8 Log Usage Example (log.py) ---------------------------------------- .. literalinclude:: /code/log.py :lines: 1-10 ``timeit/cProfile`` ---------------------------------------- ``timeit`` is a module for timing/profiling small pieces of python code. ``cProfile`` is a more robust module for profiling which is combined with ``pstats`` to configure the profiler output statistics. ``profile`` is a pure-Python module with the same API as ``cProfile`` introduces additional overhead compared to ``cProfile`` which is written as a C extension. Note there is a bug/license issue which excludes ``pstats`` from the default Python install on Ubuntu. You'll need to install the ``python-profiler`` package. See https://bugs.launchpad.net/ubuntu/+source/python-defaults/+bug/123755 Simple Profile Example (timeme.py) ---------------------------------------- .. literalinclude:: /code/timeme.py Larger Profile Example (profileme.py) ---------------------------------------- .. literalinclude:: /code/profileme.py :lines: 1-10 ``unittest`` ---------------------------------------- The ``unittest`` module is a testing framework based on Kent Beck's Smalltalk testing framework (`original paper `_). This same style of testing framework, called xUnit, has been written in most every language. Testing Example - Program (program.py) ---------------------------------------- Sample program with some known flaws .. literalinclude:: /code/program.py :lines: 24-28 Testing Example (program_test.py) ---------------------------------------- Test suite to expose those flaws .. literalinclude:: /code/program_test.py :lines: 1-4,64-65 There are More Modules ---------------------------------------- Be sure to check the documenation on the Python website at http://docs.python.org/library/ Up Next ---------------------------------------- Common third-party Python libraries .. header:: MA792K Spring 2011 Lecture 6 .. footer:: © Mark Lavin 2011