What is the best tool for removing duplicates from a CSV data source?

[Breno_Sampaio_dos_Sa]

I would like to remove duplicate records from a CSV data source, what is the best tool for this?

[thadguidry]

Hi @Breno_Sampaio_dos_Sa and Welcome to OpenRefine !

Depending on your needs, you might look to command line tools such as:

• qsv
• Structs - Polars user guide

or Visual tools you can download or use online and more similar to OpenRefine such as:

• Excel 365 online
• LibreOffice - remove duplicates
• Apache Nifi (DeduplicateRecord)