Snapshot release 15-Apr-2021 (improved memory use, 'Split Col' improvements)

News
1

There is a new snapshot release with some improvements:

  • More efficient with memory, meaning larger files can be loaded. In testing Easy Data Transform can now:

    • Load a 49 cols x 875 thousand rows (593MB) CSV file in 15 seconds.
    • Load a 14 cols x 5 million rows (624MB) CSV file in 27 seconds.
    • Load a 15 cols x 16 million rows (2.0GB) CSV file in 114 seconds.
    • Load a 15 cols x 31 million rows (3.7GB) CSV file in 10 minutes.

    And it is using significantly less memory than v1.15.1.

  • Split Col transform can now split a column into multiple rows.

split-col-example
split-col-example623×853 148 KB

  • Split Col transform now has a Reverse option.

split-col-reverse
split-col-reverse276×600 43.7 KB

  • The Right pane has been modified to avoid nested scrollbars.

Here are the links:

Windows: https://www.easydatatransform.com/downloads/EasyDataTransform__1_15_2_snapshot3.exe
Mac: https://www.easydatatransform.com/downloads/EasyDataTransform__1_15_2_snapshot3.dmg

Please let us know if you have any problems or suggestions.

2

This looks very promising. I have a dataset that is space delimited (firs name, last name, email) but a cell (skills) has multiple entries delimited by a comma.

1 Like
3

Wow, the functionality for the Split Col transform worked flawlessly. I successfully used the Column and Row.

1 Like
4

Glad to hear that it worked ok. You can probably split a column into rows using a sequence of other transforms, but it is a bit painful. Nice to be able to do it in one transform!

5

In the past, I would have to Remove columns, Split columns, Gather the columns, and then re Join the data.

1 Like