Using file wildcards in batch mode may not cover what you want to do. In that case you might have to do it in 2 steps, for now. E.g. use the batch feature to aggregate all the data into a big CSV. Then create a second .transform to process the big CSV.
Perhaps you could provide a bit more detail about what you are trying to do.
Hi
Iâve already given the detail in the quote below or do you really need more?
Just making sure I understand correctly. So you need to aggregate all the JSON data files into 1 dataset before you can do the Gather and/or Spread?
I suggest again please take a closer look at the link I provided for a similar âalmost essential within workflow pipelineâ iterator functionality available in ArcGIS modeler.
Itâs really not the same-same as the current ability in EDT to only be able to iterate/batch as a wrapper the whole transform.
I realize being able to iterate within an element of a .transform not the same as iterating the whole .transform. But I want to ensure I understand the problem correctly before I consider any solutions.
Other way round⌠I use Gather and Spread to parse the multiple json files to first one big CSV. Works better than trying to use Stack alone when you have tens of thousands of files with many different field headings.
But also this was just one general example of where a âwithin-transform iteratorâ operator would be usefulâŚ
Having such a capability could potentially be a helpful alternative in any situation where EDT users currently have to resort to the batch mode.
A âwithin-transform iteratorâ also potentially adds a lot of complexity. But I will have a think about it.
Many Thanks.
I suggest maybe also have a look at the functionality of the
Bubble.io equivalent plugin iterator function available here:
Iterator Plugin | Bubble
Or the Mendix loop function here:
https://docs.mendix.com/refguide/loop
Essentially this copy request is asking for a way to either embed or link transforms. Basically the ability to ânest transformsâŚâ
For example, if EDT had a processing step to âAllow jumping to and automatically running another outside transform and then returning the result to continueâ then that would be great.
Idea is to allow the user to be able to establish a âLibrary of useful mini-transformsâ that could then be referenced and re-used.
For example:
I have several âEDT transform branchesâ that I typically use again and again in a lot of different scripts. At the moment I must manually edit and re-create those standard âtransform branchesâ from scratch each time I need them in a new transform script.
It would be great if I could either:
Copy-and-paste the needed âtransform branchâ into the new transform from my existing transforms whenever needed.
Or save that âtransform branchâ as a ready-to-run transform in a library
and then have an EDT operator that could find it in the library and run it and return the result.
Essentially wishing for the capability to be able somehow to package needed EDT mini-branches as a ânew operatorâ that could then be re-used and dropped into a transform wherever needed allowing users to create their own re-usable EDT processing operators.
A âMasterâ EDT Transform could then be created that simply kicks off a series of stored library mini-transforms in the right processing order.
Onwards and upwards 
Being able to copy and paste a group of items from the center pane between different .transform items would certainly be useful and is on the wishlist already. But note that any column-related options in the transforms would be unset, unless you also copied and pasted all the inputs the transforms were connected to. Also there are a lot of other details to work through.
Once we have copy and paste working it should also be possible for a user to build up a library of standard transform sequences. E.g. select a sequence whitespace â case â filter and copy it into the library as âcleanupâ. You can then copy that sequence from the library in a click or two, much as you would add any other transform. You would need to be able to edit/manage this library and possibly move it from one machine to another.
Being able to embed one .transform inside another is an interesting idea that we have thought about a bit. I donât know that it would be practical to pass the data directly from the parent .transform to the embedded child .transform and back. So it might be down to the user to ensure that the input files for the child match the outputs from the parent. Or maybe we could do somethign with aliases, as we do with batch processing. But that is starting to get complicated.
Currently we are concentrating on smoothing off some edges on existing features before adding any major new features.
Support for Applescript and Javascript for Automation would be great. ( I realize you support command line operation and for the most part that can accomplish the same thing - but Applescript or Javascript would be simpler and/or more powerful.)
I will make a note of the request for Applescript/Javascript automation. However this would only work on the Mac version. Iâm not sure what the closest Window equivalent is.
I think the Replace transform is the one I would most like to copy between transforms. I have a conversion that is pretty easily done on an output from an IP address that translates to a site name. It would be great to not have to rebuild it for every transform. Iâm not exactly sure I need it every now and again so using a template hasnât been effective but Iâm going to try to start doing that. just being able to copy all the replace data would be awesome.
Also you might be able to use a Lookup transform, instead of Replace.
was not aware, will start using. and will evaluate the lookup transform. Thank you!
Lookup might work better for you. You can use a separate input file for the lookup values. But note that it requires an exact match, whereas Replace can replace part of a string.
You are right Lookup is very nice and will solve several of my needs, the copy function is also very nice.
I see I never replied! For the record, I love the notes feature, I use it all the time and find it really useful when revisiting old projects. Thank you.
As a feature request or suggestion, it would be fantastic to have a mechanism similar to Excelâs functionality for multi level pivots. For instance, the ability to drag and drop the pivot column (or row) for easy organization, and the option to nest aggregations for a more flexible and intuitive data summarization experience. This could greatly enhance the efficiency and user-friendliness of the tool. Thank you!"
