Skip to content

Add unaccent to SearchFilter #9385

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mgaligniana wants to merge 9 commits into
base: main
Choose a base branch
from
+53 −0
Open

Add unaccent to SearchFilter #9385

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
deb6657
Add `unaccent` to `SearchFilter`
mgaligniana Apr 15, 2024
fb3b63a
Setup CI to run tests with Postgres
browniebroke Apr 6, 2026
69967f9
Add test for `unaccent` filter
mgaligniana Apr 7, 2026
a06a08b
Merge branch 'main' into add-unaccent-to-search-filter
browniebroke Apr 7, 2026
dcda6df
Merge branch 'main' into add-unaccent-to-search-filter
auvipy Apr 8, 2026
406adf9
Merge branch 'main' into add-unaccent-to-search-filter
browniebroke May 14, 2026
8a854bc
Add UnaccentExtension
browniebroke Jun 6, 2026
de96fd4
Remove PG extension creation from GHA
browniebroke Jun 6, 2026
0ad7873
Merge branch 'main' into add-unaccent-to-search-filter
browniebroke Jun 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/api-guide/filtering.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -230,6 +230,7 @@ The search behavior may be specified by prefixing field names in `search_fields`
| `$` | `iregex` | Regex search. |
| `@` | `search` | Full-text search (Currently only supported Django's [PostgreSQL backend][postgres-search]). |
| None | `icontains` | Contains search (Default). |
| `&` | `unaccent` | Accent-insensitive search. (Currently only supported Django's [PostgreSQL backend][postgres-lookups]). |

For example:

Expand Down Expand Up @@ -370,3 +371,4 @@ The [djangorestframework-word-filter][django-rest-framework-word-search-filter]
[HStoreField]: https://docs.djangoproject.com/en/stable/ref/contrib/postgres/fields/#hstorefield
[JSONField]: https://docs.djangoproject.com/en/stable/ref/models/fields/#django.db.models.JSONField
[postgres-search]: https://docs.djangoproject.com/en/stable/ref/contrib/postgres/search/
[postgres-lookups]: https://docs.djangoproject.com/en/stable/ref/contrib/postgres/lookups/#unaccent
1 change: 1 addition & 0 deletions rest_framework/filters.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ class SearchFilter(BaseFilterBackend):
'=': 'iexact',
'@': 'search',
'$': 'iregex',
'&': 'unaccent',
Copy link
Copy Markdown
Member

@browniebroke browniebroke Jun 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Turns out this is doing a match on the full string, hence why the tests don't pass... We should probably change this to unaccent__icontains.

I suppose some folks will probably want to have similar istartswith and iexact unaccented variants.

Copy link
Copy Markdown
Member

@browniebroke browniebroke Jun 6, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might be better off using a dedicated class for unaccent searches (not tested, I assume that's how it works):

class UnaccentedSearchFilter(SearchFilter):
    lookup_prefixes = {
        '^': 'unaccent__istartswith',
        '=': 'unaccent__iexact',
        '@': 'unaccent__search',
        '$': 'unaccent__iregex',
    }

And change the base SearchFilter class to make the default lookup easier to override in subclasses:

django-rest-framework/rest_framework/filters.py

Lines 107 to 108 in 156bd5d

# Otherwise, use the field with icontains.
lookup = 'icontains'

Copy link
Copy Markdown
Contributor Author

@mgaligniana mgaligniana Jun 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @browniebroke! I I can try to do an approach to test it, if you'd like!

Copy link
Copy Markdown
Member

@browniebroke browniebroke Jun 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes sure 😊

}
search_title = _('Search')
search_description = _('A search term.')
Expand Down
29 changes: 29 additions & 0 deletions tests/conftest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,15 @@
import contextlib
import os

import dj_database_url
import django
import pytest
from django.apps import apps
from django.contrib.postgres.operations import UnaccentExtension
from django.core import management
from django.core.management.color import no_style
from django.db import connection
from django.db.migrations.state import ProjectState


@pytest.fixture
Expand Down Expand Up @@ -141,3 +144,29 @@ def pytest_collection_modifyitems(config, items):
for item in items:
if 'requires_postgres' in item.keywords:
item.add_marker(skip_postgres)


@contextlib.contextmanager
def _postgres_extension(extension):
"""Helper to enable a PostgreSQL extension in tests."""
with connection.schema_editor(atomic=False) as schema_editor:
extension.database_forwards(
app_label='tests',
schema_editor=schema_editor,
from_state=ProjectState(),
to_state=ProjectState(),
)
yield
extension.database_backwards(
app_label='tests',
schema_editor=schema_editor,
from_state=ProjectState(),
to_state=ProjectState(),
)


@pytest.fixture
def postgres_unaccent(db):
"""Enable the unaccent PostgreSQL extension in tests."""
with _postgres_extension(UnaccentExtension()):
yield
21 changes: 21 additions & 0 deletions tests/test_filters.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -247,6 +247,27 @@ def as_sql(self, compiler, connection):
{'id': 2, 'title': 'zz', 'text': 'bcd'},
]

@pytest.mark.requires_postgres
@pytest.mark.usefixtures('postgres_unaccent')
def test_search_field_with_unaccent(self):
SearchFilterModel.objects.create(title='Jeremy', text='jeremy')
SearchFilterModel.objects.create(title='Jérémy', text='jérémy')
SearchFilterModel.objects.create(title='Jérémie', text='jérémie')
SearchFilterModel.objects.create(title='Jeremie', text='jeremie')

class SearchListView(generics.ListAPIView):
queryset = SearchFilterModel.objects.all()
serializer_class = SearchFilterSerializer
filter_backends = (filters.SearchFilter,)
search_fields = ('&title',)

view = SearchListView.as_view()

request = factory.get('/', {'search': 'Jerem'})
response = view(request)
assert len(response.data) == 4
assert {item['title'] for item in response.data} == {'Jeremy', 'Jérémy', 'Jérémie', 'Jeremie'}

def test_search_field_with_multiple_words(self):
class SearchListView(generics.ListAPIView):
queryset = SearchFilterModel.objects.all()
Expand Down
Loading