I have a text column which contains comma-separated values - inherited from an older database schema.
\nI should refactor this into a many-to-many field (or maybe even a PostgreSQL array field), but I haven't done that yet. And I wanted to be able to filter by those values in the Django admin.
\nSince I'm using PostgreSQL, I decided to figure out how to do this using the PostgreSQL regexp_split_to_array() function.
There are two necessary SQL queries here: one to figure out all of the unique distinct values that are represented across all of those comma-separated lists, and one to filter for rows that include a specific value.
\nHere's what I came up with for the first:
\nselect distinct unnest(\n regexp_split_to_array(my_column, ',\\s*')\n) from my_tableThis uses unnest(), see this TIL.
For filtering down to rows that contain a specific value in their comma-separated list, I figured out this:
\nselect\n *\nfrom\n my_table\nwhere\n array_position(\n regexp_split_to_array(\n my_column, ',\\s*'\n ),\n 'MyValue'\n ) is not nullThat second one, translated into the Django ORM, looks like this:
\nfrom django.contrib.postgres.fields import ArrayField\nfrom django.db.models import F, IntegerField, TextField, Value\nfrom django.db.models.expressions import Func\n\nqueryset.annotate(\n value_array_position=Func(\n Func(\n F(my_column),\n Value(\",\\\\s*\"),\n function=\"regexp_split_to_array\",\n output_field=ArrayField(TextField()),\n ),\n Value(my_value),\n function=\"array_position\",\n output_field=IntegerField()\n )\n).filter(value_array_position__isnull=False)I didn't bother figuring out the ORM equivalent of that first unnest() SQL.
Here's the reusable admin filter factory I came up with using these:
\nfrom django.contrib.admin import SimpleListFilter\nfrom django.contrib.postgres.fields import ArrayField\nfrom django.db import connection\nfrom django.db.models import F, TextField, Value\nfrom django.db.models.expressions import Func\n\n\ndef make_csv_filter(filter_title, filter_parameter_name, table, column):\n class CommaSeparatedValuesFilter(SimpleListFilter):\n title = filter_title\n parameter_name = filter_parameter_name\n\n def lookups(self, request, model_admin):\n sql = \"\"\"\n select distinct unnest(\n regexp_split_to_array({}, ',\\\\s*')\n ) from {}\n \"\"\".format(\n column, table\n )\n with connection.cursor() as cursor:\n cursor.execute(sql)\n values = [r[0] for r in cursor.fetchall() if r[0]]\n return zip(values, values)\n\n def queryset(self, request, queryset):\n value = self.value()\n if not value:\n return queryset\n else:\n return queryset.annotate(\n value_array_position=Func(\n Func(\n F(column),\n Value(\",\\\\s*\"),\n function=\"regexp_split_to_array\",\n output_field=ArrayField(TextField()),\n ),\n Value(value),\n function=\"array_position\",\n output_field=IntegerField()\n )\n ).filter(value_array_position__isnull=False)\n\n return CommaSeparatedValuesFilterThen you use it in a ModelAdmin subclass like this:
@admin.register(Reporter)\nclass ReporterAdmin(admin.ModelAdmin):\n list_filter = (\n make_csv_filter(\n filter_title=\"Roles\",\n filter_parameter_name=\"role\",\n table=\"reporter\",\n column=\"role_names\",\n ),\n )