201. Common Third Party Libraries

  • lxml
  • PIL
  • NumPy
  • SciPy
  • matplotlib

202. Python Packaging

Python packaging is a messy subject that is under going a bit of a make over right now. By packaging I mean a system to manage Python libraries on a given system.

  • Installation/Upgrading/Removal
  • Managing Dependencies
  • Listing Currently Installed Packages
  • Searching for New Packages

203. Packaging Solution - pip/virtualenv

Just use pip to install Python packages http://pip.openplans.org/.

For more complex environments use virtualenv to manage multiple Python environments. http://virtualenv.openplans.org/

204. The Python Package Index (PyPi)

Also (or formerly known) as the Cheeseshop, it is a central repository of Python projects. When you install a package by name via pip this is where it comes from. If you are looking for a particular package this is where you can search to find a suitable one.

See them at http://pypi.python.org/pypi

205. lxml

lxml is a Python binding for the libxml2 and libxslt C libraries. It provides a Pythonic API for handling xml files and it is very fast. Unlike the standard lib xml it supports XPath expressions and XSLT. lxml can also handle HTML.

206. lxml Example (geonames.py)

"""
Parsing xml data from GeoNames
http://www.geonames.org/export/web-services.html
"""

import os
import urllib2

from lxml import etree

207. PIL (Python Image Library)

PIL provides image processing for Python. This includes reading and writing images, type conversions, and image manipulation. Along with this set of operations it provides a clean interface for handling images.

208. PIL Example (mandel.py)

"""
Mandelbrot fractal using PIL (Python)

http://code.activestate.com/recipes/577111-mandelbrot-fractal-using-pil/
"""

from PIL import Image

209. NumPy/SciPy

NumPy is a package for handling array based data. It adds vector and matrix operations on top of standard Python lists.

SciPy is built on top of NumPy. It provides scientific computational tools for statistics, optimization, numerical integration, linear algebra, Fourier transforms, signal processing, image processing, ODE solving, and more.

210. NumPy For Matlab Users

If you use Matlab then I would recommend reading this page on comparing NumPy to Matlab:

http://www.scipy.org/NumPy_for_Matlab_Users

211. Creating NumPy Arrays

ndarray is the basic array type provided by NumPy. There a number of ways of creating them.

import numpy
a = numpy.array([1, 2, 3]) # 1-dimensional
b = numpy.array([[1, 2, 3], [4, 5, 6]]) # 2-dimensional
zero = numpy.zeros((3, 3))
one = numpy.ones((2, 3, 4))
even = numpy.arange(2, 20, 2)
unit = numpy.linspace(0, 1, 10)

212. Using NumPy Arrays

Arithmetic operators on arrays apply elementwise and create a new array with the result.

import numpy
x = numpy.arange(4)
y = numpy.arange(4)
x + y
x*2
x**2
x*y
numpy.dot(x, y)

213. Array Slicing

import numpy
x = numpy.arange(20).reshape(4, 5)
x[1, 1]
x[:, 2]
y = x.reshape(2, 2, 5)
y[:, :, 1]
y[..., 1]

214. SciPy Modules

  • Clustering package (scipy.cluster)
  • Constants (scipy.constants)
  • Fourier transforms (scipy.fftpack)
  • Integration and ODEs (scipy.integrate)
  • Interpolation (scipy.interpolate)
  • Input and output (scipy.io)
  • Linear algebra (scipy.linalg)
  • Maximum entropy models (scipy.maxentropy)
  • Miscellaneous routines (scipy.misc)
  • Multi-dimensional image processing (scipy.ndimage)
  • Orthogonal distance regression (scipy.odr)
  • Optimization and root finding (scipy.optimize)

215. SciPy Modules (cont.)

  • Signal processing (scipy.signal)
  • Sparse matrices (scipy.sparse)
  • Sparse linear algebra (scipy.sparse.linalg)
  • Spatial algorithms and data structures (scipy.spatial)
  • Distance computations (scipy.spatial.distance)
  • Special functions (scipy.special)
  • Statistical functions (scipy.stats)
  • C/C++ integration (scipy.weave)

216. SciPy Cookbook

This page hosts “recipes”, or worked examples of commonly-done tasks. Some are introductory in nature, while others are quite advanced (these are at the bottom of the page). We encourage new users to post recipes even for simple tasks that are not yet represented here. Our goal is an easy learning experience for new users. Some of these recipes may be incorporated into tutorials in the future.

http://www.scipy.org/Cookbook

217. SciPy Regression Example (regress.py)

from scipy import array
from scipy.stats import linregress

from geonames import parse_country_info


def get_country_data(country):

218. Matplotlib

Matplotlib is a 2D plotting library. It makes use of NumPy arrays for large data sets so it plays well with NumPy and SciPy. It also supports using LaTex for figure titles and text.

219. Simple Plotting Example (regressplot.py)

from matplotlib import pylab
from scipy import polyval
from scipy.stats import linregress

from regress import parse_pop_size_data