psm
March 23, 2025, 3:41pm
1
I am observing this error since OpenRefine 3.8.1 for large CSV file with more than 32 columns and around 10K rows (for example any 100K dataset you may download from OpenAlex to reproduce this error).
This error from OpenRefine 3.9.1:
Interestingly, the same dataset is opening in OpenRefine version 3.6.2 without any issue.
How to get rid of this CsvParserSettings error?
--Partha
1 Like
I think we have an issue for this already. The setting is hardcoded but in the issue we said we might make this configurable. The change happened from earlier OpenRefine version that had a dependency change in the CSV library.
@Rory we might prioritize that issue since a few folks have run into this.
Thad Guidry
1 Like
I think @Rory is on a break this week so I'll try to get that done in 3.9.2.
I released 3.9.2 today with an increased limit.
opened 05:59PM - 14 Mar 25 UTC
Type: Feature Request
Status: Pending Review
I would like to mention the issue I faced multiple times while importing a CSV f… ile comparatively bigger in size, whose number of characters per column may exceed the stipulated amount allowed (32768). It doesn't allow me to open a project, and mentions to check the CsvParserSettings. It becomes impossible to work with such CSV dataset. It shows as follows:
Length of parsed input (32769) exceeds the maximum number of characters defined in your parser settings (32768). Parser Configuration: CsvParserSettings:.........
### Proposed solution
Add the option to customise the maximum number of characters per column (more than 32769), according to requirements of the user. To be more memory efficient, it may allow the deduction in number of columns (less than 16384) as well. It must allow the user to adjust according to their needs.
### Alternatives considered
Increase the maximum number of characters per cell to some extent higher than the current one, but the above mentioned customisation is the preferred solution.
1 Like
