Performance reading/writing tabular file formats

Reading and writing the same 500,000 rows x 14 columns in different formats on a Mac M1 Air using Easy Data Transform v1.43.0:

Format Read (seconds) Write (seconds)
Tab Separated Value (unquoted) 1.203 0.516
Comma Separated Value (quoted) 1.980 0.870
Excel XLSX 30.314 31.302

So tab separated without quoting is the best choice for performance, as long as your data doesn’t contain tabs. And Excel is much slower than either comma or tab delimited.

1 Like