Packaging Your Programs

Once your program is ready for release, you will probably want to package it properly before distributing it. If it consists of a single .py file, this might not be much of an issue. If you’re dealing with nonprogrammer users, however, even placing a simple Python library in the right place or fiddling with the PYTHONPATH may be more than they want to deal with. Users normally want to simply double-click an installation program, follow some installation wizard, and then have your program ready to run.
    Lately, Python programmers have also become used to a similar convenience, although with a slightly more low-level interface. The Setuptools toolkit, and the older Distutils, for distributing Python packages makes it easy to write install scripts in Python. You can use these scripts to build archive files for distribution, which the programmer (user) can then use for compiling and installing your libraries.
    In this tutorial, I focus on Setuptools, because it is an essential tool in every Python programmer’s toolkit. And Setuptools actually goes beyond the script-based installation of Python libraries. It’s also quite convenient for compiling extensions, and with the extensions py2exe and py2app, you can even build stand-alone Windows and macOS executable programs.

Setuptools Basics 

You can find lots of relevant documentation in the Python Packaging User Guide (packaging.python.org) and on the Setuptools web site (http://setuptools.readthedocs.io). You can use Setuptools to do all manner of useful things by writing a script as simple as the one shown in Listing 18-1. (If you don’t already have Setuptools, you can install it using pip.)
Listing 18-1. Simple Setuptools Setup Script (setup.py)

from setuptools import setup

setup(name='Hello',

         version='1.0',

        description='A simple example',

        author='Magnus Lie Hetland',

        py_modules=['hello'])

You don’t really have to supply all of this information in the setup function (you don’t actually need to supply any arguments at all), and you certainly can supply more (such as author_email or url). The names should be self-explanatory. Save the script in Listing 18-1 as setup.py (this is a universal convention for Distutils setup scripts), and make sure that you have a simple module called hello.py in the same directory.

■ Caution  the setup script will create new files and subdirectories in the current directory when you run it, so you should probably experiment with it in a fresh directory to avoid having old files being overwritten.
Now let’s see how you can put this simple script to use. Execute it as follows:

python setup.py

You should get some output like the following:

usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]

   or: setup.py --help [cmd1 cmd2 ...]

   or: setup.py --help-commands

   or: setup.py cmd --help

error: no commands supplied
As you can see, you can get more information using the --help or --help-commands switch. Try issuing the build command, just to see Setuptools in action.

python setup.py build

You should now see output like the following:
running build running build_py creating build creating build/lib copying hello.py -> build/lib
Setuptools has created a directory called build, with a subdirectory named lib, and placed a copy of hello. py in build/lib. The build subdirectory is a sort of working area where Setuptools assembles a package (and compiles extension libraries, for example). You don’t really need to run the build command when installing, because it will be run automatically, if needed, when you run the install command.

 ■ Note  in this example, the install command will copy the hello.py module to some system-specific directory in your PYTHONPATH. this should not pose a risk, but if you don’t want to clutter your system, you might want to remove it afterward. make a note of the specific location where it is placed, as output by setup. py. You could also use the -n switch to do a dry run. at the time of writing, there is no standard uninstall command (although you can find custom uninstallation implementations online), so you’ll need to uninstall the module by hand.

Speaking of which . . . let’s try to install the module.

python setup.py install

Now you should see quite a bit of output, ending with something like the following:

Installed /path/to/python3.5/site-packages/Hello-1.0-py3.5.egg 

Processing dependencies for Hello==1.0 

Finished processing dependencies for Hello==1.0 byte-compiling

 ■ Note  if you’re running a version of python that you didn’t install yourself and don’t have the proper privileges, you may not be allowed to install the module as shown, because you don’t have write permissions to the correct directory.

This is the standard mechanism used to install Python modules, packages, and extensions. All you need to do is provide the little setup script. As you can see, as part of the installation process, Setuptools has built an egg, a self-contained bundled Python package.
    The sample script uses only the Setuptools directive py_modules. If you want to install entire packages, you can use the directive packages in an equivalent manner (just list the package names). You can set many other options as well (some of which are covered in the section “Compiling Extensions” later in this chapter). These options let you specify such things as what to install and where to install it. And your configuration can be used for more than one thing. The following section shows you how to wrap the modules you specified for installation as an archive file, ready for distribution.

Wrapping Things Up 

Once you’ve written a setup.py script that will let the user install your modules, you can use it yourself to build an archive file, for example. You can also construct a Windows installer, an RPM package, an egg distribution, or a wheel distribution, among other things. (Wheels are intended to replace eggs, eventually.) I’ll just walk you through the .tar.gz example, and you should easily pick up the other formats from the documentation. You build a source archive file the sdist (for “source distribution”) command.

python setup.py sdist

If you run this, you will probably get quite a bit of output, including some warnings. The warnings I get include a complaint about a missing author_email option, a missing README file, and a missing URL. You can safely ignore all of these (although feel free to add an author_email option to your setup.py script, similar to the author option, and a README.txt text file in the current directory). After the warnings you should see output like the following:

creating Hello-1.0/Hello.egg-info 

making hard links in Hello-1.0... 

hard linking hello.py -> Hello-1.0 

hard linking setup.py -> Hello-1.0 

hard linking Hello.egg-info/PKG-INFO -> Hello-1.0/Hello.egg-info 

hard linking Hello.egg-info/SOURCES.txt -> Hello-1.0/Hello.egg-info 

hard linking Hello.egg-info/dependency_links.txt -> Hello-1.0/Hello.egg-info 

hard linking Hello.egg-info/top_level.txt -> Hello-1.0/Hello.egg-info 

Writing Hello-1.0/setup.cfg 

Creating tar archive removing 'Hello-1.0' (and everything under it)

Now, in addition to the build subdirectory, you should have one called dist. Inside it, you will find a gzip’ed tar archive called Hello-1.0.tar.gz. This can be distributed to others, and they can unpack it and install it using the included setup.py script. If you don’t want a .tar.gz file, several other distribution formats are available, and you can set them all through the command-line switch --formats. (As the plural name indicates, you can supply more than one format, separated by commas, to create more archive files in one go.) You get a list of the available formats with the --help-formats switch to the sdist.

Compiling Extensions 

You may agree that compiling these extensions could be a bit cumbersome at times. Luckily, you can use Distutils for this as well. You may want to refer to Chapter 17 for the source code to the program palindrome (in Listing 17-6). Assuming that you have the source file palindrome2.c in the current (empty) directory, the following setup.py script could be used to compile (and install) it:
from setuptools import setup, Extension

setup(name='palindrome',

      version='1.0',

      ext_modules = [

          Extension('palindrome', ['palindrome2.c'])

      ])

If you run the install command with this setup.py script, the palindrome extension module should be compiled automatically before it is installed. As you can see, instead of specifying a list of module names, you give the ext_modules argument a list of Extension instances. The constructor takes a name and a list of related files; this is where you would specify header (.h) files, for example. If you would rather just compile the extension in place (resulting in a file called palindrome.so in the current directory for most UNIX systems), you can use the following command:

python setup.py build_ext --inplace

Now we get to a real juicy bit. If you have SWIG installed (see Chapter 17), you can have Setuptools use it directly! Take a look at the source for the original palindrome.c (without all the wrapping code) in Listing 17-3.  It’s certainly much simpler than the wrapped-up version. Being able to compile it directly as a Python extension, having Distutils use SWIG for you, can be very convenient. It’s all very simple, really—you just add the name of the interface (.i) file (see Listing 17-5) to the list of files in the Extension instance.

from setuptools import setup, Extension

setup(name='palindrome',

      version='1.0',

      ext_modules = [

          Extension('_palindrome', ['palindrome.c',

                                    'palindrome.i'])

      ])

If you run this script using the same command as before (build_ext, possibly with the --inplace switch), you should end up with an .so file again (or some equivalent), but this time without needing to write all the wrapper code yourself. Note that I have named the extension _palindrome, as SWIG will create a wrapper named palindrom.py that imports a C library by this name.

Creating Executable Programs with py2exe 

The py2exe extension to Setuptools (available via pip) allows you to build executable Windows programs (.exe files), which can be useful if you don’t want to burden your users with having to install a Python interpreter separately. The py2exe package can be used to create executables with GUIs . Let’s use a very simple example here:

print('Hello, world!') 

input('Press <enter>')

Again, starting in an empty directory containing only this file, called hello.py, create a setup.py file like this:

from distutils.core import setup 

import py2exe

setup(console=['hello.py'])

You can run this script like this:

python setup.py py2exe

This will create a console application (called hello.exe) along with a couple of other files in the dist subdirectory. You can either run it from the command line or double-click it. For more information about how py2exe works and how you can use it in more advanced ways, visit the py2exe web site (http://www.py2exe.org). If you’re using macOS, you might want to check out py2app (http://pythonhosted.org/py2app), which provides similar functionality for that platform.

REGISTERING YOUR PACKAGE WITH PYPI

if you want others to be able to install your package using pip, you must register it with the python package index, pypi. the standard library documentation describes how this works in detail, but essentially you use the following command:

python setup.py register

at that point you’ll get a menu that lets you log in or register as a new user. once your package is registered, you can use the upload command to upload it to pypi. For example,

python setup.py sdist upload

will upload a source distribution.