We started getting reports that our data we were reporting was incomplete. We had changed nothing on our side and up to this point it was pretty solid. I have been digging around trying to see what could have caused it. I ran into this (picture attached). It looks like you added new options on your filter to match things and different percentages. Well, unfortunately it “moved” my filter from what it was to one of these options. This filter was on “is not blank” and now it’s on what you can see in the image. This one is going to cause us a lot of issues. Do you know when this was added so I can go back and rerun reports from this time frame?
I will investigate and get back to you soon.
The additional fuzzy matching options were added to Filter in v1.36.0 on 14-Nov-2022.
I tried reproducing your issue here.
- I created a Filter transform with ‘Is empty’ in v1.35.0.
- I read it in to v1.36.0, it showed ‘Is empty’ as expected.
- I read it in to v1.37.1, it showed ‘Is empty’ as expected.
Can you give me the steps to reproduce the problem?
Which transform is this? Filter, If or something else?
I think I found the issue.
We modified the Filter transform so that any .transform file created before v1.36.0 that is read in to v1.36.0 or later is backward compatible. But we missed this fix for the If transform. This means that:
- any .transform created before v1.36.0 with an If clause set to ‘Is not empty’ is read in to v1.36.0 or later as 80% fuzzy match to empty.
- any .transform created before v1.36.0 with an If clause set to ‘Is empty’ is read in to v1.36.0 or later as 90% fuzzy match to empty.
Apologies for the inconvenience. I am testing a fix now.
Yes it was the if function for sure. I went back and redid the analysis manually and it caused us to have 175 errors out of about 5K lines. unfortunately for us, those 175 are pretty big deals. So I guess, for now I’ll go check all “if” filters.
I suggest you also compare against the results from v1.35.0.
I just checked against all data from November. doesnt look like any errors prior to 11/25