Easy Data Transform native binaries for Mac M1 / M2 / ARM

News
1

We have created an experimental version of Easy Data Transform that is a Mac Universal (‘fat’) binary. This means that it contains native version of the software for both older Intel-based Macs and for the newer ARM-based Macs (with M1 or M2 chips). This means that, if you have one of the newer ARM-based Macs, it no longer needs to use the Rosetta2 emulation and should be faster.

Some preliminary tests show the Universal binary:

  • runs about 16% faster on our test ARM Mac than the Intel binary does.
  • runs at the same speed on our test Intel Mac as the Intel binary does.

The only downside is that the download is bigger, increasing from 56MB to 69MB. But that is still positively anorexic compared to many modern applications!

You can download the new Universal binaries here:

https://www.easydatatransform.com/downloads/EasyDataTransform_1_35_1.dmg

The functionality should be identical to v1.35.0.

If you have a Mac, Intel or ARM, please download this new version and reply in this post with:

  • what type of Mac? E.g: iMac, Mini, MacBook Air, MacBook Pro etc
  • what version of macOS? E.g. macOS 12.3.1.
  • did it run ok? if not, any error message?

Any comparisons of run times of v1.35.1 vs v1.35.0 on ARM Macs with realworld datasets would also be very helpful.

image
image1518×552 96.7 KB

2

MacBook Air M2 running macOS 12.6

Runs perfectly - Opens in a flash with no errors - extremely responsive.

1.35.0 to 1.35.1 comparison:

Test 1: Open and process a 98Mb json file:
1.35.0: 3.74 seconds
1.35.1: 2.72 seconds

Test 2: Batch process of 3,541 Json Files and write results to .csv:
1.35.0: 12.809 seconds
1.35.1: 8.089 seconds

:grinning: Fantastic results! Thanks so much for the update!

1 Like
3

Cool. Thanks for the numbers. Significantly faster on the 2nd test, I see. The expected speed up is ~15% on ARM, but it depends on the task and system.

4

MacBook Pro 2019 (Intel)

MacOS 12.6

image

Little decrease of performance, but not really significant

1 Like
5

@Olaf
That might just be ‘noise’, e.g. from other jobs running. You probably need to average a few runs to get a reliable result when the difference is only a few %.

6

Using Mac Studio M1 Max 32GB, Monterey 12.6

EasyDataTransform 1.35.1
Started run at: Fri Sep 30 16:38:13 2022
Processed Input ‘RTD_MPS_2022-07-01-0000_2022-09-29-1100.csv’ (12 cols x 23.4m rows) in 57.528 second(s) [Read], Up to date - has warnings
Processed Transform ‘Filter’ (12 cols x 9.4m rows) in 12.301 second(s) [13,937,220 rows removed], Up to date
Processed Transform ‘Filter’ (12 cols x 4.6m rows) in 4.403 second(s) [4,830,787 rows removed], Up to date
Processed Transform ‘Pivot’ (181 cols x 25.9k rows) in 18.114 second(s) [Pivot table created], Up to date
Processed Transform ‘Summary’ (182 cols x 30 rows) in 1.875 second(s) [Summary created], Up to date
Processed Transform ‘Rename Cols’ (182 cols x 30 rows) in <0.001 second(s) [1 col renamed], Up to date
Processed Transform ‘Filter’ (12 cols x 2.3m rows) in 2.966 second(s) [7,091,736 rows removed], Up to date
Processed Transform ‘Pivot’ (95 cols x 25.9k rows) in 8.832 second(s) [Pivot table created], Up to date
Processed Transform ‘Summary’ (96 cols x 30 rows) in 0.91 second(s) [Summary created], Up to date
Processed Transform ‘Rename Cols’ (96 cols x 30 rows) in <0.001 second(s) [1 col renamed], Up to date
Processed Transform ‘Filter’ (12 cols x 2.5m rows) in 2.753 second(s) [6,949,915 rows removed], Up to date
Processed Transform ‘Pivot’ (98 cols x 25.9k rows) in 9.354 second(s) [Pivot table created], Up to date
Processed Transform ‘Summary’ (99 cols x 30 rows) in 1.023 second(s) [Summary created], Up to date
Processed Transform ‘Rename Cols’ (99 cols x 30 rows) in <0.001 second(s) [1 col renamed], Up to date
Processed Transform ‘Sample’ (12 cols x 10k rows) in 0.201 second(s) [10,000 sampled, 23,363,439 unsampled], Up to date
Ended run at: Fri Sep 30 16:40:16 2022
15 item(s) processed in 123.001 second(s)
Up to date: 15, Needs updating: 0, Error: 0
1 item(s) with warnings

EasyDataTransform 1.35.0
Started run at: Fri Sep 30 16:33:42 2022
Processed Input ‘RTD_MPS_2022-07-01-0000_2022-09-29-1100.csv’ (12 cols x 23.4m rows) in 86.417 second(s) [Read], Up to date - has warnings
Processed Transform ‘Filter’ (12 cols x 9.4m rows) in 18.667 second(s) [13,937,220 rows removed], Up to date
Processed Transform ‘Filter’ (12 cols x 4.6m rows) in 6.316 second(s) [4,830,787 rows removed], Up to date
Processed Transform ‘Pivot’ (181 cols x 25.9k rows) in 24.312 second(s) [Pivot table created], Up to date
Processed Transform ‘Summary’ (182 cols x 30 rows) in 2.727 second(s) [Summary created], Up to date
Processed Transform ‘Rename Cols’ (182 cols x 30 rows) in <0.001 second(s) [1 col renamed], Up to date
Processed Transform ‘Filter’ (12 cols x 2.3m rows) in 4.361 second(s) [7,091,736 rows removed], Up to date
Processed Transform ‘Pivot’ (95 cols x 25.9k rows) in 12.006 second(s) [Pivot table created], Up to date
Processed Transform ‘Summary’ (96 cols x 30 rows) in 1.33 second(s) [Summary created], Up to date
Processed Transform ‘Rename Cols’ (96 cols x 30 rows) in <0.001 second(s) [1 col renamed], Up to date
Processed Transform ‘Filter’ (12 cols x 2.5m rows) in 4.168 second(s) [6,949,915 rows removed], Up to date
Processed Transform ‘Pivot’ (98 cols x 25.9k rows) in 12.85 second(s) [Pivot table created], Up to date
Processed Transform ‘Summary’ (99 cols x 30 rows) in 1.55 second(s) [Summary created], Up to date
Processed Transform ‘Rename Cols’ (99 cols x 30 rows) in <0.001 second(s) [1 col renamed], Up to date
Processed Transform ‘Sample’ (12 cols x 10k rows) in 0.325 second(s) [10,000 sampled, 23,363,439 unsampled], Up to date
Ended run at: Fri Sep 30 16:36:41 2022
15 item(s) processed in 178.494 second(s)
Up to date: 15, Needs updating: 0, Error: 0
1 item(s) with warnings

No problems, just faster!

1 Like
7

You are right, there is some variance:

image

1 Like
8

@bert
I make that ~31% faster.

1 Like