Tag Archives: python

Custom template folders with Flask

3 Replies

Someone was asking on [Flask][flask]’s IRC channel [#pocoo][irc] about sharing templates across more than one app but allowing each app to override the templates (pretty much what [Django’s TEMPLATE_DIRS setting][django] is for). One way of doing this would be to customise the Jinja2 template loader.

Here’s a trivial Flask app that searches for templates first in the default folder (‘templates’ in the same folder as the app) and then in an extra folder.

import flask
import jinja2

app = flask.Flask(__name__)
my_loader = jinja2.ChoiceLoader([
app.jinja_loader,
jinja2.FileSystemLoader(‘/path/to/extra/templates’),
])
app.jinja_loader = my_loader

@app.route(‘/’)
def home():
return flask.render_template(‘home.html’)

if __name__ == “__main__”:
app.run()

The only thing special here is creating a new template loader and then assigning it to [the `jinja_loader` attribute on the Flask application][attr]. [`ChoiceLoader`][choice] will search for a named template in the order of the loaders, stopping on the first match. In this example I re-used the loader that is created by default for an app, which is roughly like `FileSystemLoader(‘/path/to/app/templates’)`. There are [all kinds of other exciting template loaders available][loaders].

I really like the fact that Flask and Bottle’s APIs are so similar. Next I want Flask to include [Bottle’s template wrapping decorator][view] by default (there’s [a recipe in the Flask docs][templated]) and for both of them to re-name it `@template`.

[flask]: http://flask.pocoo.org/
[irc]: http://flask.pocoo.org/community/irc/
[attr]: http://flask.pocoo.org/docs/api/#flask.Flask.jinja_loader
[choice]: http://jinja.pocoo.org/docs/api/#jinja2.ChoiceLoader
[loaders]: http://jinja.pocoo.org/docs/api/#loaders
[django]: https://docs.djangoproject.com/en/dev/ref/templates/api/#loading-templates
[templated]: http://flask.pocoo.org/docs/patterns/viewdecorators/#templating-decorator
[view]: http://bottlepy.org/docs/stable/api.html#bottle.view

Inspecting your routes in Bottle

1 Reply

Marcel Hellkamp [recently added a small feature][3] to Bottle that makes it easy to inspect an application’s routes and determine if a particular route is actually for a mounted sub-application.

([Bottle is a small module written in Python for making websites][4].)

Route objects (items in the `app.routes` list) now have extra information when the route was created by mounting one app on another, in the form of a new key `mountpoint` in `route.config`.

Here’s a trivial app with another app mounted on it:

import bottle

app1 = bottle.Bottle()

@app1.route(‘/’)
def app1_home(): return “Hello World from App1”

app2 = bottle.Bottle()
@app2.route(‘/’)
def app2_home(): return “Hello World from App2”

app1.mount(prefix=’/app2/’, app=app2)

And a utility function that returns a generator of prefixes and routes:

def inspect_routes(app):
for route in app.routes:
if ‘mountpoint’ in route.config:
prefix = route.config[‘mountpoint’][‘prefix’]
subapp = route.config[‘mountpoint’][‘target’]

for prefixes, route in inspect_routes(subapp):
yield [prefix] + prefixes, route
else:
yield [], route

Finally, inspecting all the routes (including mounted sub-apps) for the root Bottle object:

for prefixes, route in inspect_routes(app1):
abs_prefix = ‘/’.join(part for p in prefixes for part in p.split(‘/’))
print abs_prefix, route.rule, route.method, route.callback

This new feature is sure to revolutionise everything.

[1]: http://blog.nturn.net/?p=289
[2]: http://jason.cleanstick.net/post/19943282016/stupid-simple-api-reference-for-bottle-py-web-services
[3]: https://github.com/bottlepy/bottle/commit/9b24401605e0470388a65c80a0964cba2bf64caf
[4]: http://bottlepy.org/

Widths & Heights with xlwt + Python

3 Replies

This article about using xlwt to generate Excel in Python reminded me I needed to see exactly how to set column widths (the xlwt documentation doesn’t cover it).

Let’s create a new Excel workbook and add a sheet:

>>> import xlwt
>>> book = xlwt.Workbook(encoding='utf-8')
>>> sheet = book.add_sheet('sheeeeeet')

We need to get a column in order to set its width. You do that by call col() on the sheet, passing the column’s index as the only argument (or row() for accessing rows):

>>> sheet.col(0)    # First column

>>> sheet.row(2)    # Third row

The index is zero-based. You can fetch a column even if you have not written to any cell in that column (this applies equally to rows).

Columns have a property for setting the width. The value is an integer specifying the size measured in 1/256 of the width of the character ‘0’ as it appears in the sheet’s default font. xlwt creates columns with a default width of 2962, roughly equivalent to 11 characters wide.

>>> first_col = sheet.col(0)
>>> first_col.width = 256 * 20              # 20 characters wide (-ish)
>>> first_col.width
5120

For rows, the height is determined by the style applied to the row or any cell in the row. (In fact rows also have a property called height but it doesn’t do what you want.) To set the height of the row itself, create a new style with a font height:

>>> tall_style = xlwt.easyxf('font:height 720;') # 36pt
>>> first_row = sheet.row(0)
>>> first_row.set_style(tall_style)

Setting the style on the row does not change the style of the cells in that row.

There is no obvious way to set a default width and height for all columns and rows. An instance of xlwt.Worksheet.Worksheet has properties for col_default_width and row_default_height but changing those does not actually change the defaults.

The problem is that new columns are always created with an explicit width, while rows take their height from the style information.

My first attempt at setting defaults set the width on every column and the height on every row. It works, but creates 65,536 unnecessary empty row objects.

A slightly better approach is to set the width on every column and to set the font height on the default style record in the workbook:

import itertools
import xlwt

book = xlwt.Workbook(encoding='utf-8')
sheet = book.add_sheet('sheeeeeet')

col_width = 256 * 20                        # 20 characters wide

try:
    for i in itertools.count():
        sheet.col(i).width = col_width
except ValueError:
    pass

default_book_style = book.default_style
default_book_style.font.height = 20 * 36    # 36pt

book.save('example.xls')

Here I used itertools.count() and wrapped the loop in a try block so I can forget exactly how many columns are permitted. When the loop tries to access a bad index it will throw ValueError and the loop will exit.

You mustn’t replace the default style on an instance of xlwt.Workbook.Workbook, you have to update the property of the existing style (to ensure you are changing the first style record). Unfortunately there is no way to set a default column width (as of xlwt version 0.7.2) so the brute force method of setting every column will have to do – it isn’t so bad since there are only 256 columns.

Talking of widths and heights, have you heard "Widths & Heights" by Magic Arm? Is good.

Free software FTW! Updated filetimes.py

1 Reply

[Two years ago][old] (flippin’ heck it seems like only yesterday) I wrote about converting between Unix timestamps and Windows timestamps using Python. In that post I linked to my very simple implementation of a module that provides converting back and forth between the formats.

A few weeks ago I received an e-mail from Timothy Williams with changes to the my module so that it preserves the fractions of a second in the conversion. How sweet is that?!?!! Exclamation mark question mark exclamation mark cellida diaresis em-dash full stop king of punctuation.

It is fantastic that not only did someone find my code useful but also that they were generous enough to take the time to improve it and give the changes back to me. I love tasty, delicious free software and the people like Tim who make it tastier and more delicious.

So here is the new version of [filetimes.py incorporating Tim’s fixes][ft].

[old]: http://reliablybroken.com/b/2009/09/working-with-active-directory-filetime-values-in-python/
[ft]: http://reliablybroken.com/b/wp-content/filetimes.py

XPath bug in old versions of ElementTree

Leave a reply

I figured out why my XML parsing code works fine using the [pure-Python ElementTree XML parsing module][elementtree] but fails when using [the speedy and memory-optimized cElementTree XML parsing module][celementtree].

[The XPath 1.0 specification][xpath] says `’.’` is short-hand for `’self::node()’`, selecting a node itself.

Parsing an XML document and selecting the context node with ElementTree in Python 2.5:

>>> from xml.etree import ElementTree
>>> ElementTree.VERSION
‘1.2.6’
>>> doc = “BUG”
>>> node1 = ElementTree.fromstring(doc).find(‘./Example’)
>>> node1

>>> node1.find(‘.’)

>>> node1.find(‘.’) == node1
True

See how the result of `node1.find(‘.’)` is the node itself? [As it should be][selfnode].

Parsing an XML document and selecting the context node with cElementTree in Python 2.5:

>>> from xml.etree import cElementTree
>>> doc = “BUG”
>>> node2 = cElementTree.fromstring(doc).find(‘./Example’)
>>> node2

>>> node2.find(‘.’)
>>> node2.find(‘.’) == node2
False

Balls. The result of `node2.find(‘.’)` is `None`.

However! I have a kludgey work-around that works whether you use ElementTree or cElementTree. Use `’./’` instead of `’.’`:

>>> node1.find(‘./’)

>>> node1.find(‘./’) == node1
True
>>> node2.find(‘./’)

>>> node2.find(‘./’) == node2
True

*Kludgey because `’./’` is not a valid XPath expression.*

So we are back on track. Also works for Python 2.6 which has the same version of ElementTree.

Fortunately Python 2.7 got a new version of ElementTree and the bug is fixed:

>>> from xml.etree import ElementTree
>>> ElementTree.VERSION
‘1.3.0’
>>> doc = “BUG”
>>> node3 = ElementTree.fromstring(doc).find(‘./Example’)
>>> node3

>>> node3.find(‘.’)

>>> node3.find(‘.’) == node3
True

However! They also fixed my kludgey work-around:

>>> node3.find(‘./’)
>>> node3.find(‘./’) == node3
False

So I can’t code something that works for all three versions. This is annoying. I was hoping to just replace ElementTree with the C version, makes my code run in one third the time (the XML parts of it run in one tenth the time). And cannot install any compiled modules – the code can only rely on Python 2.5’s standard library.

[celementtree]: http://effbot.org/zone/celementtree.htm
[elementtree]: http://effbot.org/zone/element-index.htm
[xpath]: http://www.w3.org/TR/xpath/
[selfnode]: http://www.w3.org/TR/xpath/#path-abbrev

Styling your Excel data with xlwt

1 Reply

This post is about how to create styles in [Excel spreadsheets][excel] with [the most excellent xlwt][xlwt] for [Python][python]. The documentation for xlwt (version 0.7.2) is a little sketchy on how to use formatting. So here goes…

To apply formatting to a cell you pass an instance of the `xlwt.XFStyle` class as the fourth argument to the `xlwt.Worksheet.write` method. The best way to create an instance is to use the `xlwt.easyxf` helper, which takes a string that specifies the formatting for a cell.

The other thing about using styles is you should only make one instance of each, then pass that same style object every time you want to apply it to a cell.

An example which uses a few styles:

import xlwt

styles = dict(
bold = ‘font: bold 1’,
italic = ‘font: italic 1’,
# Wrap text in the cell
wrap_bold = ‘font: bold 1; align: wrap 1;’,
# White text on a blue background
reversed = ‘pattern: pattern solid, fore_color blue; font: color white;’,
# Light orange checkered background
light_orange_bg = ‘pattern: pattern fine_dots, fore_color white, back_color orange;’,
# Heavy borders
bordered = ‘border: top thick, right thick, bottom thick, left thick;’,
# 16 pt red text
big_red = ‘font: height 320, color red;’,
)

I have no idea what it is based on, but 20 = 1 pt. So 320 = 16 pt text.

book = xlwt.Workbook()
sheet = book.add_sheet(‘Style demo’)

for idx, k in enumerate(sorted(styles)):
style = xlwt.easyxf(styles[k])
sheet.write(idx, 0, k)
sheet.write(idx, 1, styles[k], style)

book.save(‘Example.xls’)

It isn’t included with [the current distribution on the cheese shop][pypi], but there is [a useful Excel spreadsheet demonstrating cell patterns][sheet] in the source repository.

You can find the complete list of possible cell formats by reading [the source for `xlwt.Styles`][styles].

[excel]: http://office.microsoft.com/en-us/excel/
[xlwt]: http://www.python-excel.org/
[python]: http://www.python.org/
[styles]: https://secure.simplistix.co.uk/svn/xlwt/tags/0.7.2/xlwt/Style.py
[pypi]: http://pypi.python.org/pypi/xlwt/0.7.2
[sheet]: https://secure.simplistix.co.uk/svn/xlwt/tags/0.7.2/xlwt/doc/pattern_examples.xls

Class-based views for Bottle

Leave a reply

I’m not convinced this is actually a good idea, but I have an approach for using class-based views as handlers for a route with [Bottle][bottle].

_(If you were mad keen on [Django’s][django] shift to [class-based views][cbv] you might reckon life wouldn’t be complete with a Bottle-based application until you employ classes for views. However Bottle’s use of decorators for tying URLs to views means it is less a natural fit than the same thing in Django.)_

The problem is that you can’t just decorate the method in your class using [`bottle.route`][route] because if you use that decorator on a method in a class you are telling Bottle to use the method before it has been bound to an instance of that class.

So although I wish it did, the following example will not work:

import bottle

class ViewClass(object):
@bottle.route(“/”)
def home_view(self):
return “My home page.”

obj = ViewClass()
bottle.run()

Running that will lead to errors about not enough arguments passed to the view method of your `ViewClass` instance.

Instead you need to register the route right after the object is created. This can be done in [the class’s `__new__` method][new]:

import bottle

class ViewClass(object):
def __new__(cls, *args, **kwargs):
obj = super(ViewClass, cls).__new__(cls, *args, **kwargs)
bottle.route(“/”)(obj.home_view)
return obj

def home_view(self):
return “My home page.”

obj = ViewClass()
bottle.run()

It works. It isn’t that pretty. You could achieve exactly the same thing by explicitly passing the `obj.home_view` method to `bottle.route` _after_ the instance is created. The advantage to doing this in the `__new__` method is it will happen automatically whenever `ViewClass` is instantiated.

And if you go down this path then [you should be aware of threads][threads]. Hey! Nice threads! Also I have a cold.

[bottle]: http://bottle.paws.de/
[cbv]: http://docs.djangoproject.com/en/dev/topics/class-based-views/
[django]: http://www.djangoproject.com/
[route]: http://bottle.paws.de/docs/dev/api.html#routing
[new]: http://docs.python.org/reference/datamodel.html#object.__new__
[threads]: http://bottle.paws.de/docs/dev/tutorial.html#accessing-request-data

Running Django on Mac

5 Replies

These are semi-detailed steps for installing all the bits to host a [Django][django] application on [Mac OS X][macosx]. Tested on 10.5, should work perfectly on 10.6.

Use [MacPorts][macports]: relatively easy to install and the best thing is everything is contained in a directory that you can be confident won’t eff up Apple’s stuff and won’t be effed up by Apple’s stuff.

Install Xcode
————-

You need the compiler and bits that are installed with Xcode. If you can’t find your Mac install discs (Xcode is included with every new Mac but not installed) you can [download it from Apple’s developer website][xcode]. Registration is required but is free.

The current version of Xcode is easy to find, while older versions are available in the downloads section under “Developer Tools”. Xcode version 3.1.4 is the last version that will work for Mac OS X 10.5 systems.

Install MacPorts
—————-

MacPorts have a nice pkg installer. You can also build it from source.

curl -O http://distfiles.macports.org/MacPorts/MacPorts-1.9.1-10.5-Leopard.dmg
hdiutil attach MacPorts-1.9.1-10.5-Leopard.dmg
sudo installer -pkg /Volumes/MacPorts-1.9.1/MacPorts-1.9.1.pkg -target /
hdiutil detach /Volumes/MacPorts-1.9.1

If for some reason MacPorts cannot fetch updates you may need to [pull updates by hand][manualports].

Check your $PATH after installing ports to make sure `/opt/local/bin` is in there. If it isn’t your can do `export PATH=/opt/local/bin:/opt/local/sbin:${PATH}` to fix things, and even add taht line to `~/.profile` so that bash picks it up every time (assuming you haven’t switched your shell).

Install software
—————-

The `port` command is used to manipulate the MacPorts installation. Use it to build and install the various bits we need. This takes a while, especially on old PowerPC machines. Make it more exciting by adding the `–verbose` flag. Exciting!

sudo port install python26
sudo port install apache2
sudo port install mysql5-server
sudo port install mod_python26
sudo port install py26-mysql
sudo port install py26-django
sudo port install py26-south

And if you want to hook Django into a network directory then you almost certainly want to use LDAP.

sudo port install py26-ldap

Cool kids these days say use [mod_wsgi][modwsgi] instead of [mod_python][modpython] for hosting Python applications with Apache, but I am not cool (and on 20 September 2010 I couldn’t persuade mod_wsgi to build from MacPorts on a clean installation).

Configuring and starting MySQL
——————————

*UPDATED: [commenter matea][matea] pointed to [Jason Rowland’s MySQL on Mac][jason] posting that includes steps to secure a default installation, so I’ve updated this section with the appropriate steps.*

I always seem to be the only person who cares about non-English visitors… anyway, so I want to have [MySQL][mysql] use UTF8 for everything. Edit the configuration so it does. As root, create a configuration at `/opt/local/var/db/mysql5/my.cnf` with these lines:

[mysqld]
init-connect = ‘SET NAMES utf8’
character-set-server = utf8
collation-server = utf8_general_ci
skip-networking

[mysql]
default-character-set = utf8

One thing about the line `skip-networking` in the configuration file is that it means MySQL will not listen to __any__ network clients, including connections to `127.0.0.1`. Instead clients should connect to `localhost` or they should specify the path to the socket that MySQL uses for communication. If your MySQL “client” is a Django instance running on the same host then that should not be a problem.

Now initialize the database and start the server. (The use of `-w` in the second line tells launchctl to have the database daemon start at boot. If you don’t want to have MySQL running at boot use `-F` to __force__ start just this one time instead of every time.)

sudo -u mysql /opt/local/bin/mysql_install_db5
sudo launchctl load -w /Library/LaunchDaemons/org.macports.mysql5.plist

And let’s check that the server is up and configured right.

/opt/local/bin/mysql5 -e “SHOW variables LIKE ‘%char%'”

You should see a table showing that the character set for the client and server is set to utf8.

Now run the secure installation script for MySQL. This will ask you to set a password for MySQL’s root account (the administrator) and ask whether to remove the test database and anonymous user access (you should do both):

/opt/local/bin/mysql_secure_installation5

Thaz better.

Configuring Postgresql instead of MySQL
—————————————

If you want to use [Postgres][postgres] instead of MySQL then you need a couple different packages out of ports.

sudo port install postgresql84-server
sudo port install py26-psycopg2

Did you know Apple’s management tools use Postgres? Is true.

Configuring Apache to serve a Django project
——————————————–

Let’s suppose your Django project lives under `/Library/WebServer/example.com/myproj`, and the project’s settings file is `/Library/WebServer/example.com/myproj/settings.py`. Here’s how to configure Apache with mod_python to serve your project.

Create a separate site configuration for Apache in `/Library/WebServer/example.com/site.conf`.


SetHandler python-program
PythonHandler django.core.handlers.modpython
SetEnv DJANGO_SETTINGS_MODULE myproj.settings
PythonOption django.root /
PythonDebug On
PythonPath “[‘/Library/WebServer/example.com’] + sys.path”


Order deny,allow
Allow from all

Of course once everything is hunky dory you will go back and edit the site configuration so that `PythonDebug Off`.

And finally tell Apache to use mod_python and read the site configuration. Edit `/opt/local/apache2/conf/httpd.conf` and add a line at the end of the modules like:

LoadModule python_module modules/mod_python.so

And then a line like:

Include /Library/WebServer/example.com/site.conf

Now fire up Apache:

sudo launchctl load -w /Library/LaunchDaemons/org.macports.apache2.plist

MacPorts has a convenient shortcut for this:

sudo port load apache2

You also want to save Apache a little grief by pre-compiling the Python source files for the project:

/opt/local/bin/python2.6 -m compileall /Library/WebServer/example.com

Hope this helps.

[manualports]: http://reliablybroken.com/b/2010/03/using-macports-behind-a-firewall/
[macosx]: http://www.apple.com/macosx/
[django]: http://www.djangoproject.com
[macports]: http://www.macports.org/
[xcode]: http://developer.apple.com/technology/xcode.html
[mysql]: http://www.mysql.com
[postgres]: http://www.postgresql.org
[modwsgi]: http://docs.djangoproject.com/en/dev/howto/deployment/modwsgi/
[modpython]: http://docs.djangoproject.com/en/dev/howto/deployment/modpython/
[matea]: http://reliablybroken.com/b/2010/09/running-django-on-mac/comment-page-1/#comment-1458
[jason]: http://www.jasonrowland.com/2009/10/install-mysql5-on-snow-leopard-using-macports/

Bottle’s view decorator and default variables

2 Replies

[Bottle][bottle]’s [`@view` decorator][view] provides a simple way to designate a template to render an HTML page. Your view function just has to return a dictionary, and its contents can be accessed from the template using the `'{{ name }}’` syntax.

The `@view` decorator can also take keyword arguments. These are treated as default template variables – if the dictionary returned by your view function doesn’t have a key for one of the keyword arguments then the template will use the value passed into the decorator, like so:

from bottle import view

@view(‘default.html’, author=’David Buxton’)
def home():
return {‘title’: ‘Home page’}

That would render any instance of `'{{ author }}’` as `’David Buxton’`. And then you can have another view function that overrides the keywords by returning a different value in the dictionary:

from bottle import view

@view(‘default.html’, author=’David Buxton’)
def music():
return {‘title’: ‘Thalassocracy’, ‘author’: ‘Frank Black’}

And at that point I wonder what is the advantage of using keyword arguments with `@view`: you have to decorate each function separately, and if you want to override a keyword in your return dictionary then it would be easier not to specify the keyword in the first place.

Thus the real point of using keywords with the `@view` function is only apparent if you curry the `@view` decorator with keywords first so that you can re-use the curried decorator and avoid repeating yourself.

*Someday I will re-write the previous sentence. Until then, sorry.*

Instead of passing a default author each time as in the examples above, let’s make a new `@view` decorator (using Python’s [functools module][functools]) and then use that on each view function:

import functools
from bottle import view

view = functools.partial(view, author=’David Buxton’)

@view(‘default.html’)
def home():
return {‘title’: ‘Home page’}

@view(‘default.html’)
def music():
return {‘title’: ‘Thalassocracy’, ‘author’: ‘Frank Black’}

The new decorator means you get the default keyword arguments wherever you use `@view` while permitting any function to override those defaults in the dictionary it returns.

And if you wanted to get really lazy you could even pass in a template name when wrapping the decorator with `functools.partial`, however you would not be able to use your wrapped decorator to change the template name because it is a positional argument (like what [it explains here in the functools documentation][partial]). You would also have to call the decorator with no arguments like `’@defaultview()’`. So forget I mentioned it.

I’m not saying you are lazy.

[bottle]: http://bottle.paws.de/
[functools]: http://docs.python.org/library/functools.html
[partial]: http://docs.python.org/library/functools.html#functools.partial
[view]: http://bottle.paws.de/docs/0.8/api.html#bottle.view

Django-style routing for Bottle

3 Replies

[Bottle][bottle] provides the [`@route` decorator][route] to associate URL paths with view functions. This is very convenient, but if you are a [Django][django]-reject like me then you may prefer having all your URLs defined in one place, the advantage being it is easy to see at a glance [all the different URLs your application will match][urlconf].

*Updated: I have re-written this post and the example to make it simpler following Marcel Hellkamp’s comments (Marcel is the primary author of Bottle). My original example was needlessly complicated.*

It is possible to have [a Django-style urlpatterns stanza][urlpatterns] with a Bottle app. Here’s how it can work:

from bottle import route

# Assuming your *_page view functions are defined above somewhere
urlpatterns = (
# (path, func, name)
(‘/’, home_page, ‘home’),
(‘/about’, about_page, ‘about’),
(‘/contact’, contact_page, ‘contact’),
)

for path, func, name in urlpatterns:
route(path, name=name)(func)

Here we run through a list where each item is a triple of URL path, view function and a name for the route. For each we simply call the `route` method and then invoke it with the function object. Not as flexible as using the decorator on a function (because the `@route` decorator can take additional keyword arguments) but at least you can have all the routes in one place at the end of the module.

Then again if you have so many routes that you need to keep them in a pretty list you probably aren’t writing the simple application that Bottle was intended for.

(This was tested with Bottle’s 0.8 and 0.9-dev branches.)

[bottle]: http://bottle.paws.de/
[route]: http://bottle.paws.de/docs/0.8/api.html#bottle.route
[django]: http://www.djangoproject.com/
[urlconf]: http://docs.djangoproject.com/en/1.2/topics/http/urls/#example
[urlpatterns]: http://docs.djangoproject.com/en/dev/topics/http/urls/