Testing with Django, Haystack and Whoosh

The problem: you want to test a Django view for results of a search query, but Haystack will be using your real query index, built from your real database, instead of an index built from your test fixtures.

Turns out you can generalise this for any Haystack back-end by replacing the haystack.backend module with the simple back-end.

from myapp.models import MyModel
from django.test import TestCase
import haystack

class SearchViewTests(TestCase):
    fixtures = ['test-data.json']

    def setUp(self):
        self._haystack_backend = haystack.backend
        haystack.backend = haystack.load_backend('simple')

    def tearDown(self):
        haystack.backend = self._haystack_backend

    def test_search(self):
        results = SearchQuerySet().all()
        assert results.count() == MyModel.objects.count()

My first attempt at this made changes to the project settings and did HAYSTACK_WHOOSH_STORAGE = "ram" which works but was complicated because then you have to re-build the index with the fixtures loaded, except the fixtures don’t get loaded in TestCase.setUpClass, so the choice was to load the fixtures myself or to re-index for each test. And it was specific to the Whoosh back-end of course.

(This is for Django 1.4 and Haystack 1.2.7. In my actual project I get to deploy on Python 2.5. Ain’t I lucky? On a fucking PowerMac G5 running OS X Server 10.5 for fuck sacks.)

5 thoughts on “Testing with Django, Haystack and Whoosh

  1. Alex

    So if I understand this correctly, the simple backend queries / updates the database instead of the Solr index. This allows you to run queries / updates that would normally hit Solr without touching it. So you can’t really test the querying / updating that Haystack does (and you don’t need to in your app’s unit test) and you receive “real” results based on what is in the database which in the case of django tests, is a temp database created for testing so nothing permanently changes.

    Is everything I said correct?

  2. david Post author

    Hi @Alex,

    Yes, you got it.

    The simple backend doesn’t use Solr (or whatever other proper backend you configure Haystack to use in production); instead it just does plain SQL queries using the Django ORM, which implies Haystack doesn’t actually do updates to its own search indexes when using the simple backend.

    (I’m assuming this is still true: I haven’t checked if current versions of Haystack have changed the behaviour of the simple backend since I wrote this post in December 2012.)

  3. S├ębastien Brochet

    Hi,

    As of version 2.x of Haystack, you’re no more able to modify the backend with the “backend” attribute. You can however get the same result with this code:

    from haystack.utils import loading from haystack import connections haystack.connections = loading.ConnectionHandler({ 'default': { 'ENGINE': 'haystack.backends.simple_backend.SimpleEngine' } })

    Small issue is that haystack.backends.simple_backend.SimpleEngine doesn’t support advanced filtering and if you make use of it you need another solution. Based on your experimentations with Whoosh backend, I was able to configured it and get the right results when doing filtering:

    HAYSTACK_WHOOSH_STORAGE = 'ram' HAYSTACK_SIGNAL_PROCESSOR = 'haystack.signals.RealtimeSignalProcessor'

    from haystack.utils import loading from haystack import connections haystack.connections = loading.ConnectionHandler({ 'default': { 'ENGINE': 'haystack.backends.whoosh_backend.WhooshEngine' } })

    As I don’t make use of fixtures (yet), it does the job.

  4. S├ębastien Brochet

    My bad, the method I’ve described in my previous comment doesn’t work. I was using my local file-based Whoosh index created during normal runs.

    It seems also that HAYSTACK_WHOOSH_STORAGE is no more supported in Haystack 2.x

    To make my unittest, I’ve cleared my Whoosh index:

    python manage.py clear_index

    And forced the real-time signal processor:

    HAYSTACK_SIGNAL_PROCESSOR = 'haystack.signals.RealtimeSignalProcessor'

    Now, all my tests are OK :-)

  5. Earnest89

    Reading your website is big pleasure for me, it deserves to go viral, you need some initial traffic only. If you want to know how to get it search for: blackhatworren’s strategies

Leave a Reply

Your email address will not be published. Required fields are marked *