Selectize filter-by bug

[codematsing]

Hi, Finally figured out the issue as to why Selectize filter-by functionality does not work in custom projects integrating django-formset library.

Note that this issue could not be replicated in the testapp for the investigation stated below.

I hope my explanation is clear. Please feel free to message me if it doesn't make sense

Issue:

• when installing django-formset==1.5 to custom projects, when I try to use Selectize filter-by, the functionality does not work
• when I try to use the testapp to replicate the issue, I could not report bug because Selectize filter-by works in testapp

How I discovered the issue:

• I followed the steps here so that I can investigate why Selectize filter-by is not working
• To my understanding, the flow for filter-by to function is:
  • when app is run with a form that uses selectize widget, django-formset imports DjangoSelectize.js
  • DjangoSelectize.js waits for select value to change, when triggered, will send fetch request for filtering
  • FormView will receive fetch request, will process query, and return new select options back to frontend
• Using chrome browser dev-tools, I tried to understand which js files are triggered
• Running both test app and my custom project side by side, I compared and contrasted staticfiles being loaded in page.
• I found that the testapp DjangoSelectize-*.js filename differ from what is being used in my custom project
• I found that django-formset.js is the one importing what DjangoSelectize-*.js file will be used by the running django app
• I found that django-formset.js imports different DjangoSelectize-*js files when used in testapp and in custom proejct
  • in testapp, it imports: DjangoSelectize-VSQ5POIH.js
  • in custom project, it imports: DjangoSelectize-6XCBY3D6.js
• I tried to find why DjangoSelectize-VSQ5POIH.js is being imported in testapp, event though workdir/static/formset/js/django-formset.js imports DjangoSelectize-6XCBY3D6.js
• I found that testapp uses formset/static/formset/js/django-formset.js

Conclusion and Summary:

• testapp selectize filter by works
• using selectize filter by in custom project do not work
• testapp and custom projects do not have the same django-formset.js files
• testapp uses formset/static/formset/js/django-formset.js which imports DjangoSelectize-VSQ5POIH.js (successfully triggers fetch request)
• custom projects uses similar to that located in workdir/static/formset/js/django-formset.js which imports DjangoSelectize-6XCBY3D6.js (unsuccessful tigger fetch request)

My Workaround:

• right now I have to manually override venv/lib/site-packages/.../formset/static/js/django-formset.js to import DjangoSelectize-VSQ5POIH.js
• it isn't nice. Lol. I have to be mindful of what I did. But works for the time being to make use of filter-by functionality.

Caveats:

• I have no knowledge with the differences for DjangoSelectize-VSQ5POIH.js and DjangoSelectize-6XCBY3D6.js
• I also don't know which file is the most recent js file
• I just understand that django-formset.js only imports one DjangoSelectize-*.js file so I assume that rather than these files work simultaneously, these are mutually exclusive / differ in versions

[codematsing]

Note, I also found that testapp includes incomplete attr in select tag, while it does not happen in custom project

so as an additional workaround I need to put:

my_field = ModelChoiceField(
    ...,
    widget=Selectize(filter_by=..., attrs={"incomplete":True})
)

If I don't put the incomplete tag, queries that have relationship similar to state ||--o{ county filter-by query will not work in custom projects:

# testapp/forms/state.py
class StateForm(forms.Form):
    state = models.ModelChoiceField(
        label="State",
        queryset=State.objects.all(),
        widget=Selectize(
            search_lookup='name__icontains',
        ),
        # initial=2,
    )

    county = models.ModelChoiceField(
        label="County",
        queryset=County.objects.all(),
        widget=Selectize(
            search_lookup=['name__icontains'],
            filter_by={'state': 'state__id'},
        ),
        # initial=70,
    )

Overall, I just feel like the package is not updated with what is running in testapp. I'll try to check this out when I have time. Just sharing my workaround for now

[codematsing]

Upon further investigation, it seems that it's not the DjangoSelectize-*.js causing the issue but the lack of incomplete attribute in the select tag causing the issue.

Therefore, the only needed workaround is to add attrs={"incomplete":True}?

Suggestion

In formset/widgets.py

class IncompleteSelectMixin:
    def build_attrs(self, base_attrs, extra_attrs):
        attrs = super().build_attrs(base_attrs, extra_attrs)
        if isinstance(self.choices, SimpleModelChoiceIterator):
            if self.choices.queryset.count() > self.max_prefetch_choices:
                attrs['incomplete'] = True
            if self.filter_by:
                attrs['filter-by'] = ','.join(self.filter_by.keys())
                attrs['incomplete'] = True #intrdouce this if filter_by is also present in html attrs
        return attrs

Logic:

Filter-by is technically always incomplete due to the filtering mechanism?

[jrief]

I really like your deep investigation into problems. Would be nice if all other users of my libraries would dig that deep using their debugger.

What you describe is weird indeed. However, when importing the right /static/formset/js/django-formset.js this file should trigger to load the right version of DjangoSelectize-XXX.js. This is hard-coded into the file. Might it be that your browser has a cached version of /static/formset/js/django-formset.js and this is causing the problems.

Please also try the monolithic build. There no asynchronous loading takes place.

[codematsing]

Regarding this, I should note that my conclusion for this issue is not particularly the js files but it's really related to the filtering functionality as referenced in my comment above: #162 (comment)

I found that if the filtering field does not exceed the max_prefetch_choices count, the filter function will not work.

Please see replication of this issue here:
codematsing@e60b9f5

For this implementation, please note that I just randomly sampled 200 counties for initial queryset of counties field.
The purpose of the demonstration is to show that if queryset count < max_prefetch_choices filtering by state will not work.

Therefore, for models with objects not yet exceeding 250, filter by will not work

Reproduction Steps

• Run page http://localhost:800/bootstrap/state
• Select a state
• See county field (which has initial queryset < 250) - will not be filtered. will still display all options
• See counties field (which has initial queryset > 250) - will be filtered

Working Solution

class IncompleteSelectMixin:
    def build_attrs(self, base_attrs, extra_attrs):
        attrs = super().build_attrs(base_attrs, extra_attrs)
        if isinstance(self.choices, SimpleModelChoiceIterator):
            if self.choices.queryset.count() > self.max_prefetch_choices:
                attrs['incomplete'] = True
            if self.filter_by:
                attrs['filter-by'] = ','.join(self.filter_by.keys())
                attrs['incomplete'] = True #intrdouce this if filter_by is also present in html attrs
        return attrs