I'm using Ubuntu 18.04 and above, both desktop and server, and looking for terminal tools to query / write to large CSV file efficiently, and also easy to interact (few simple commands or commands that resemble SQL query language).

What would you recommend?


Solution 1:

There a lot of great tools:

  • awk
  • miller https://miller.readthedocs.io/en/latest/
  • xsv https://github.com/BurntSushi/xsv
  • csvtk https://bioinf.shenwei.me/csvtk/
  • VisiData https://www.visidata.org/

Solution 2:

An excellent suit of tools to clean and process csv files is :

https://csvkit.readthedocs.io/en/latest/

The link provides installation guides and tutorials to get you started.