I use for repeating (productive) transformation scripts the Stack transformation to ensure that the column of read files get the right order for the next transactions. This works generally fine.
Now I have a problem that sometimes the name of the header alter, e.g. “Description” is changed to “Desc”. Currently I have to check each input file before processing if the column header correct and change it if not. Or I need to check the output if the “Description” field is empty, which is 99% due the name change in the input file. (Desc and Description are just an example, but the variations of pattern are known)
I would need something like: if header name is “Desc” rename it to “Description”.
I cannot use the Rename Col transformation as the column can have a different position in the different input files.
Dynamic renaming is somewhere on the wishlist. But I’m not sure how practical it is. What if there is already a column called ‘Description’? What if there are multiple columns called “Desc”. Also, what if it is called “D” or “Desc.” or “Descrip”?
One of the big things we want to do for v2 is to add powerful validation rules. For example “stop processing and return an error if there isn’t a column called ‘Description’”. That would leave you to do some manual tweaking, but it would at least catch the issue.
thanks for your feed, I’m aware of the challenges you mention. I don’t look for AI even with my natural intelligence I don’t get it all and have to check back.
For the situation I’m facing currently I know the exactly the differences variances and I want to control and change these. so just a stupid codes If header = “some” change it to “something”. If header equal “something” don’t do anything,
I don’t get it all and have to check back.