Hi There!
I have a strange effect on V2.0.1 Mac. I have a validate transform marking some fields, but happily passing the data forward to a JavaScript transform.
As soon as I mark some validations as error and have the corresponding lines removed, the JavaScript transform and downstream is no longer calculated. The data field of the validation transform looks correct and tells me that 28 rows were removed.
Is this a bug?
best Jochen
If you set Severity to Error then any child items will not be processed. That is by design. You should probably set it to Warning.
Oh, I see. But then the “remove rows with fault” does not really make sense, because after that, the data set would be fine to continue…
You can set it to a warning and remove warning fails. It will then carry on processing the valid rows.
I see,
But for my head it makes no real sense that I can use the validate transform to isolate rows with faults, but I cannot write those excerpt to a file to give it someone to fix those errors. Additionally, that I have the option to exclude faults from the set, but nevertheless cannot continue with the filtered set…
The validate transform already is very mighty. Perhaps you could add an option to stop/allow downstream execution on faults?
Y
You can do that. Below I identify all rows with a non-integer value and can output these to a file. I can also continue processing with rows that didn’t fail.
this works with information and warning level. If you choose the error-level, downstream execution stops.
So use warning (not error) if you want processing to continue.
That’s what I did. But it is counterintuitive and I would have use for the three status levels and so had to go with only two.
My understanding is that the validate transform lets me handle errors by filtering them. Why shouldn’t I continue processing a dataset if faulty lines were eliminated?
Just please make this more helpful and intuitive by making the stop of execution optional.
