Testing with Django, Haystack and Whoosh

The problem: you want to test a [Django][django] view for results of a search query, but [Haystack][haystack] will be using your real query index, built from your real database, instead of an index built from your test fixtures.

Turns out you can generalise this for any Haystack back-end by replacing the `haystack.backend` module with the simple back-end.

from myapp.models import MyModel
from django.test import TestCase
import haystack

class SearchViewTests(TestCase):
fixtures = [‘test-data.json’]

def setUp(self):
self._haystack_backend = haystack.backend
haystack.backend = haystack.load_backend(‘simple’)

def tearDown(self):
haystack.backend = self._haystack_backend

def test_search(self):
results = SearchQuerySet().all()
assert results.count() == MyModel.objects.count()

My first attempt at this made changes to the project settings [and did `HAYSTACK_WHOOSH_STORAGE = “ram”`][ram] which works but was complicated because then you have to re-build the index with the fixtures loaded, except the fixtures [don’t get loaded in `TestCase.setUpClass`][setupclass], so the choice was to load the fixtures myself or to re-index for each test. And it was specific to [the Whoosh back-end][whoosh] of course.

(This is for Django 1.4 and Haystack 1.2.7. In my actual project I get to deploy on Python 2.5. Ain’t I lucky? On a fucking PowerMac G5 running OS X Server 10.5 [for fuck sacks][bug].)

[django]: https://www.djangoproject.com/
[whoosh]: http://bitbucket.org/mchaput/whoosh
[haystack]: http://haystacksearch.org/
[setupclass]: http://docs.python.org/2/library/unittest.html#unittest.TestCase.setUpClass
[ram]: https://django-haystack.readthedocs.org/en/v1.2.7/settings.html#haystack-whoosh-storage
[bug]: http://www.youtube.com/watch?v=XZtpAxDEzl8

4 thoughts on “Testing with Django, Haystack and Whoosh

  1. Alex

    So if I understand this correctly, the simple backend queries / updates the database instead of the Solr index. This allows you to run queries / updates that would normally hit Solr without touching it.
    So you can’t really test the querying / updating that Haystack does (and you don’t need to in your app’s unit test) and you receive “real” results based on what is in the database which in the case of django tests, is a temp database created for testing so nothing permanently changes.

    Is everything I said correct?

  2. david Post author

    Hi @Alex,

    Yes, you got it.

    The simple backend doesn’t use Solr (or whatever other proper backend you configure Haystack to use in production); instead it just does plain SQL queries using the Django ORM, which implies Haystack doesn’t actually do updates to its own search indexes when using the simple backend.

    (I’m assuming this is still true: I haven’t checked if current versions of Haystack have changed the behaviour of the simple backend since I wrote this post in December 2012.)

  3. Sébastien Brochet

    Hi,

    As of version 2.x of Haystack, you’re no more able to modify the backend with the “backend” attribute.
    You can however get the same result with this code:


    from haystack.utils import loading
    from haystack import connections
    haystack.connections = loading.ConnectionHandler({ 'default': { 'ENGINE': 'haystack.backends.simple_backend.SimpleEngine' } })

    Small issue is that haystack.backends.simple_backend.SimpleEngine doesn’t support advanced filtering and if you make use of it you need another solution.
    Based on your experimentations with Whoosh backend, I was able to configured it and get the right results when doing filtering:


    HAYSTACK_WHOOSH_STORAGE = 'ram'
    HAYSTACK_SIGNAL_PROCESSOR = 'haystack.signals.RealtimeSignalProcessor'

    from haystack.utils import loading
    from haystack import connections
    haystack.connections = loading.ConnectionHandler({ 'default': { 'ENGINE': 'haystack.backends.whoosh_backend.WhooshEngine' } })

    As I don’t make use of fixtures (yet), it does the job.

  4. Sébastien Brochet

    My bad, the method I’ve described in my previous comment doesn’t work.
    I was using my local file-based Whoosh index created during normal runs.

    It seems also that HAYSTACK_WHOOSH_STORAGE is no more supported in Haystack 2.x

    To make my unittest, I’ve cleared my Whoosh index:


    python manage.py clear_index

    And forced the real-time signal processor:


    HAYSTACK_SIGNAL_PROCESSOR = 'haystack.signals.RealtimeSignalProcessor'

    Now, all my tests are OK :-)

Leave a Reply

Your email address will not be published. Required fields are marked *