Monthly Archives: April 2009

Package installer wish

Mac OS X administrators frequently need to build installer packages to
help deploy and manage software on a network of Macs. The motive for
creating a package is one of

* Packaging software that does not have a dedicated installer. This applies
to all the nice drag-and-drop applications like [Firefox][firefox] and
[Cyberduck][cyberduck].
* Packaging your own site’s software, whether that is as simple as printer
descriptions or as complex as a full-blown application.
* Re-packaging some miserable piece of shit installer that either totally
denies the harsh reality of Apple’s non-cross platform installer formats or
which manages to make such a balls of an install package that you were
better off before they bothered.
[Most everything by Adobe is in this category][adobeinstallers].

The first of those three is very common, and it ought to be easy to
create packages for existing installed applications.
[Apple’s PackageMaker][packagemaker] application provides a nice interface
for creating packages, but it has two drawbacks:

* PackageMaker is not installed by default on Mac OS X (it gets intalled as
part of the Developer Tools).
* Before PackageMaker 3 (part of Xcode 3, which requires Mac OS X 10.5)
there was no *simple* method for quickly packaging an installed application.

What I want is a package creation tool that works on a 10.4 system without
requiring the developer tools and which can be scripted. The `packagemaker`
command-line tool requires an existing `Info.plist` file or `.pmdoc` file
if you want to set a custom default installation directory – not the end
of the world, but tedious.

Useful links
————

* [Iceberg][iceberg] is an excellent graphical tool for building packages, by
Stéphane Sudre who also wrote up the…
* [PackageMaker how-to][pmhowto] which is a very useful introduction to the
details of `.pkg` files. Bit out-of-date these days.
* Man pages for the command-line [packagemaker][man1pm] and for the
[installer][installer] tools.
* Apple’s [software distribution documentation][softdist], which is very
quiet on the subject of custom installer plug-ins. Xcode has a template
project for an installer plugin, and the `InstallerPlugins.framework` headers
have lots of information.
* [Installer-dev mailing list][installerdev], where the people who wrote
the tools and documentation help out a lot.
* [JAMF Composer][composer] which is part of the Casper management tools.
Version 7 is no longer free.

[firefox]: http://www.mozilla.com/firefox/
[cyberduck]: http://cyberduck.ch/
[packagemaker]: http://developer.apple.com/DOCUMENTATION/DeveloperTools/Conceptual/PackageMakerUserGuide/index.html
[adobeinstallers]: http://blogs.adobe.com/OOBE/
[iceberg]: http://s.sudre.free.fr/Software/Iceberg.html
[pmhowto]: http://s.sudre.free.fr/Stuff/PackageMaker_Howto.html
[man1pm]: http://developer.apple.com/DOCUMENTATION/DARWIN/Reference/ManPages/man1/packagemaker.1.html
[installer]: http://developer.apple.com/DOCUMENTATION/Darwin/Reference/ManPages/man8/installer.8.html
[softdist]: http://developer.apple.com/documentation/DeveloperTools/Conceptual/SoftwareDistribution
[installerdev]: http://lists.apple.com/mailman/listinfo/installer-dev
[composer]: http://www.jamfsoftware.com/products/composer.php

Python features for a PHP refugee

These are things that particularly impressed me
when I decided I had had enough of [PHP][php] and I really ought to look at
[the crazy white-space language called Python][python] that was used by
[Plone][plone], [Trac][trac] and [Django][django].

The [Zen of Python][zen] states most of this in 19 lines, for all you
_tl;dr_ types.

Name-spaces and a minimal set of built-ins
——————————————

I like that the set of keywords is small, and that the built-in methods are
not much larger. This leaves you with an unpolluted name-space (and if you
enjoy confusing people you can always override the built-ins).

Explicit versus implicit
————————

Related to name-spaces is the notion that Python is explicit: there is very
little magic in a Python script. Perhaps the closest thing to magic are the
various special methods that define a behaviour, for example the `__getattr__`
/ `__setattr__` / `__delattr__` methods on a class to control attribute access.
But even then Python makes it obvious those methods have special meaning by
using a double-underscore for the method names.

See [the page on the data model][special] in the Python documentation for
a description of these methods and their purpose.

Generators and list comprehensions
———————————-

I never realized how much I missed these until I went back to PHP for a small
web project. So much of my code seems to be looping through lists and
accumulating a result or applying a function to each member of the list. I
don’t think the syntax is particularly obvious, but then I can’t think of a
better way to do it. At first glance generators looked to be the same as
list comprehensions, but eventually I began to understand the difference
between needing a finite list of objects ([list comprehension][listcomp])
and consuming a sequence of objects ([generators][generators]).

Named arguments for methods and functions
—————————————–

Gosh, not having named arguments is painful. As a consumer, named arguments
allow one to forget a function’s precise argument signature, and as a
designer it allows one to provide sensible defaults and flexibility.

Dates and times as a native type
——————————–

Well, not _native_, but readily available.

[Python’s `datetime` module][datetime] provides representations of calendar
dates and times and a bunch of obvious behaviour for comparing two moments.
PHP 5 introduced a proper DateTime class, but I had jumped ship a while
before then – my affection for Python’s date handling is borne of a time
when one had to rely on PEAR for useful date functions. Converting everything
to seconds since the Unix epoch was never fun.

The greatest annoyance in Python’s date implementation is its shrugging
support for timezones – you nearly always need to resort to a third-party
module ([such as pytz][pytz] or [python-dateutil][dateutil]) to handle
timezones without jeopardizing one’s sanity.

Batteries included
——————

It is odd that one _does_ need an additional module to handle timezones
seeing as the Python standard library includes so many useful modules for
common tasks.

Need to work with [CSV files][csv]? Or [command-line arguments][getopt]?
Or [Mac OS X-style .plist][plistlib] files? Or configuration files in
[INI format][ini]? Or [tar archives][tar] (with gzip or bzip2 compression)?

Oh golly so much tedious work has been done for you in the Python standard
library. I suppose this reflects PHP’s emphasis as a scripting language for
the Web versus Python’s use as a general purpose language, but I am very
grateful for that distinction.

[zen]: http://www.python.org/dev/peps/pep-0020/
[special]: http://docs.python.org/reference/datamodel.html#special-method-names
[datetime]: http://docs.python.org/library/datetime.html
[pytz]: http://pytz.sourceforge.net/
[dateutil]: http://labix.org/python-dateutil
[csv]: http://docs.python.org/library/csv.html
[getopt]: http://docs.python.org/library/getopt.html
[plistlib]: http://docs.python.org/library/plistlib.html
[ini]: http://docs.python.org/library/configparser.html
[php]: http://www.php.net/
[plone]: http://plone.org/
[django]: http://www.djangoproject.com/
[trac]: http://trac.edgewall.org/
[python]: http://www.python.org/
[listcomp]: http://docs.python.org/tutorial/datastructures.html#list-comprehensions
[generators]: http://docs.python.org/tutorial/classes.html#generators
[tar]: http://docs.python.org/library/tarfile.html

Q and operator.or_

I’ve finally settled on a nice syntax for `OR`-ing [Django Q objects][q].

For a simple site search feature I needed to search for a term across several
fields in a model. Suppose the model looks like this:

class BlogPost(models.Model):
title = models.CharField(max_length=100)
body = models.TextField()
summary = models.TextField()

And you have a view method that accepts a parameter `q` for searching across
the `title`, `body` and `summary` fields. I want to find objects that contain
the `q` phrase in any of those fields. I need to build a [`QuerySet`][queryset]
with a filter that is the equivalent of

queryset = BlogPost.objects.filter(
Q(title__icontains=q) | Q(body__icontains=q) | Q(summary__icontains=q)
)

That’s not too much of a hassle for this simple example, but in cases where
the fields you are searching are chosen dynamically, or where you just have
an awful lot of fields to search against, I think it is nicer to do it like so:

import operator

search_fields = (‘title’, ‘body’, ‘summary’)
q_objects = [Q(**{field + ‘__icontains’:q}) for field in search_fields]
queryset = BlogPost.objects.filter(reduce(operator.or_, q_objects))

Nice one! The list comprehension gives me a list of `Q` objects generated from
the names in `search_fields`, so it is easy to change the fields to be searched.
And using [`reduce`][reduce] and [`operator.or_`][operator] gives me the
required `OR` filter in one line.

I see for Python 3 `reduce` has been [moved to the `functools` module][functools].

This stuff never used to be that obvious to me. It kind of isn’t even now.

P.S. I promise I am not writing a blog engine at this time, it was just for
the example.

[q]: http://docs.djangoproject.com/en/dev/topics/db/queries/#complex-lookups-with-q-objects
[queryset]: http://docs.djangoproject.com/en/dev/ref/models/querysets/
[operator]: http://docs.python.org/library/operator.html
[reduce]: http://docs.python.org/library/functions.html#reduce
[functools]: http://docs.python.org/library/functools.html