# Bug report: IO error while editing: Unexpected character ('\<' (code 60))

**URL:** <https://forum.openrefine.org/t/bug-report-io-error-while-editing-unexpected-character-code-60/809>\
**Category:** Support and Helpdesk\
**Tags:** wikimedia-commons\
**Created:** [June 12, 2023, 4:45pm UTC](https://forum.openrefine.org/t/bug-report-io-error-while-editing-unexpected-character-code-60/809 "2023-06-12T16:45:26Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![olea](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.openrefine.org/olea/32/231_2.png) [@olea](https://forum.openrefine.org/u/olea)\
**Post date:** [June 12, 2023, 4:45pm UTC](https://forum.openrefine.org/t/bug-report-io-error-while-editing-unexpected-character-code-60/809/1 "2023-06-12T16:45:26Z")

</div>

I’m working in a project uploading 46 images to Wikimedia Commons. I was able to to upload 37 but I can’t the other 9. I’m getting this error:

```plaintext
18:13:06.259 [..ting.EditBatchProcessor] IO error while editing: Unexpected character ('<' (code 60)): expected a valid value (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
 at [Source: (okio.RealBufferedSource$inputStream$1); line: 1, column: 2] (150866ms)

```

Using the -v debug or trace flags the message is the same:

```plaintext
18:33:11.233 [..ting.EditBatchProcessor] IO error while editing: Unexpected character ('<' (code 60)): expected a valid value (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
 at [Source: (okio.RealBufferedSource$inputStream$1); line: 1, column: 2] (47807ms)

```

And I don’t really know where to find the offending “\<”, since I’m not using any:

 ![127.0.0.1_3333_project_project=2237704229443 (1)](https://europe1.discourse-cdn.com/flex017/uploads/openrefine/original/1X/96411ac5fac09c8c2ee46c9c96ffbb880d1772f1.jpeg)

All the 46 files has been processed in the same batch with the same commands.

Any tip to diagnose and fix?

---

<div class="post-metadata">

**Author:** ![olea](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.openrefine.org/olea/32/231_2.png) [@olea](https://forum.openrefine.org/u/olea)\
**Post date:** [June 13, 2023, 8:58am UTC](https://forum.openrefine.org/t/bug-report-io-error-while-editing-unexpected-character-code-60/809/2 "2023-06-13T08:58:41Z")

</div>

Not a very popular post, I guess 😃

---

<div class="post-metadata">

**Author:** ![abbe98](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.openrefine.org/abbe98/32/1299_2.png) [@abbe98](https://forum.openrefine.org/u/abbe98)\
**Post date:** [June 13, 2023, 6:32pm UTC](https://forum.openrefine.org/t/bug-report-io-error-while-editing-unexpected-character-code-60/809/3 "2023-06-13T18:32:42Z")

</div>

I’m guessing it’s because of the TIFF file, the ability to upload TIFF files is offered as a courtesy but not recommended on Commons: [COM:TIFF](https://commons.wikimedia.org/w/index.php?title=Commons:TIFF)

We should maybe display a warning about TIFF files in the “issue” tab given how common TIFF failures are.

---

<div class="post-metadata">

**Author:** ![olea](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.openrefine.org/olea/32/231_2.png) [@olea](https://forum.openrefine.org/u/olea)\
**Post date:** [June 13, 2023, 9:51pm UTC](https://forum.openrefine.org/t/bug-report-io-error-while-editing-unexpected-character-code-60/809/4 "2023-06-13T21:51:57Z")

</div>

This is interesting. What I can’t see is an alternative format to TIFF for Commons, being lossless and keeping metadata 🤔

---

<div class="post-metadata">

**Author:** ![olea](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.openrefine.org/olea/32/231_2.png) [@olea](https://forum.openrefine.org/u/olea)\
**Post date:** [June 13, 2023, 10:22pm UTC](https://forum.openrefine.org/t/bug-report-io-error-while-editing-unexpected-character-code-60/809/5 "2023-06-13T22:22:39Z")

</div>

Maybe the problem origin is the Commons size limits for OpenRefine?

![image](https://europe1.discourse-cdn.com/flex017/uploads/openrefine/original/1X/b43ed6d117226bfdb189d65411982cdda3939aad.png)

[https://commons.m.wikimedia.org/wiki/Commons:Maximum\_file\_size](https://commons.m.wikimedia.org/wiki/Commons:Maximum_file_size)

Seems my pending files are really big:  
 ![image](https://europe1.discourse-cdn.com/flex017/uploads/openrefine/original/1X/0859b3af5ac52bc10b05759e30b18e68166efee1.png)

---

<div class="post-metadata">

**Author:** ![thadguidry](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.openrefine.org/thadguidry/32/653_2.png) [@thadguidry](https://forum.openrefine.org/u/thadguidry)\
**Post date:** [June 14, 2023, 12:06am UTC](https://forum.openrefine.org/t/bug-report-io-error-while-editing-unexpected-character-code-60/809/6 "2023-06-14T00:06:55Z")

</div>

@antonin_d Do we not support chunking uploads currently? Seems that wiki page suggests to especially for TIFF files. I don’t see chunking happening here: [Wikidata-Toolkit/wdtk-wikibaseapi/src/main/java/org/wikidata/wdtk/wikibaseapi/ApiConnection.java at f0c31c150e2213adffc4d77ed5ebe5f5448b0f3c · Wikidata/Wikidata-Toolkit · GitHub](https://github.com/Wikidata/Wikidata-Toolkit/blob/f0c31c150e2213adffc4d77ed5ebe5f5448b0f3c/wdtk-wikibaseapi/src/main/java/org/wikidata/wdtk/wikibaseapi/ApiConnection.java#L427)

And where [Chunked Uploading](https://www.mediawiki.org/wiki/API:Upload#Chunked_uploading) is described further by one of the links to the API documentation in that wiki page.

@olea OpenRefine also does not do any TIFF compression (and neither does Wikidata Toolkit currently from what I see), which might help your situation. You might want to compress using any tool with `LZW` or `deflate` as one of the links [Tiff Files](https://commons.m.wikimedia.org/wiki/Help:Server-side_upload#Tiff_files) on the wiki page suggests. Then upload the compressed TIFF files using OpenRefine?

---

<div class="post-metadata">

**Author:** ![thadguidry](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.openrefine.org/thadguidry/32/653_2.png) [@thadguidry](https://forum.openrefine.org/u/thadguidry)\
**Post date:** [June 14, 2023, 12:21am UTC](https://forum.openrefine.org/t/bug-report-io-error-while-editing-unexpected-character-code-60/809/7 "2023-06-14T00:21:50Z")

</div>

@olea The alternatives to TIFF lossless are many actually nowadays. But you’ll have to see what Commons supports exactly or could (if you raised an issue, but I’ve also mentioned my support on a stalled 3 year issue: [⚓ T257719 Add support for AVIF: allow uploading AVIF files to Wikimedia servers](https://phabricator.wikimedia.org/T257719))

[Commons:Project scope/Allowable file types - Wikimedia Commons](https://commons.m.wikimedia.org/wiki/Commons:Project_scope/Allowable_file_types).

A few are:

- [PNG](https://developer.mozilla.org/en-US/docs/Web/Media/Formats/Image_types#png_portable_network_graphics) - lossless compression
- [WebP](https://developer.mozilla.org/en-US/docs/Web/Media/Formats/Image_types#webp_image) - 25% better compression typically than PNG and still retaining lossless ability.
- [AVIF (which uses AV1)](https://en.wikipedia.org/wiki/AVIF) - typically even better lossless compression (up to 50%) than WebP

---

<div class="post-metadata">

**Author:** ![abbe98](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.openrefine.org/abbe98/32/1299_2.png) [@abbe98](https://forum.openrefine.org/u/abbe98)\
**Post date:** [June 14, 2023, 5:04am UTC](https://forum.openrefine.org/t/bug-report-io-error-while-editing-unexpected-character-code-60/809/8 "2023-06-14T05:04:45Z")

</div>

> What I can’t see is an alternative format to TIFF for Commons, being lossless and keeping metadata

@olea WebP should match those requirements I believe.

---

<div class="post-metadata">

**Author:** ![antonin\_d](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.openrefine.org/antonin_d/32/32_2.png) [@antonin\_d](https://forum.openrefine.org/u/antonin_d)\
**Post date:** [June 14, 2023, 11:35am UTC](https://forum.openrefine.org/t/bug-report-io-error-while-editing-unexpected-character-code-60/809/9 "2023-06-14T11:35:08Z")

</div>

Chunked uploading is not supported by OpenRefine yet and so uploads of files greater than 100MB will fail.

The error reporting for the Wikibase extension is unacceptably bad, in my opinion. I have some ideas around how to improve that and want to discuss them soon.

---

<div class="post-metadata">

**Author:** ![antonin\_d](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.openrefine.org/antonin_d/32/32_2.png) [@antonin\_d](https://forum.openrefine.org/u/antonin_d)\
**Post date:** [June 21, 2023, 7:14pm UTC](https://forum.openrefine.org/t/bug-report-io-error-while-editing-unexpected-character-code-60/809/10 "2023-06-21T19:14:54Z")

</div>

@olea here is a preview of the UX I would like to offer in 4.0:

> <https://github.com/OpenRefine/OpenRefine/pull/5944>
>
> Closes #1782.
> 
> This follows the design proposed \[on the forum\](https://forum.o…penrefine.org/t/partial-results-of-long-running-operations/482).
> The Wikibase upload operation creates a new column (or reuses the column, if it already exists) to store the editing errors of edits related to a given row:
> !\[image\](https://github.com/OpenRefine/OpenRefine/assets/309908/744b10c4-bafa-44a8-9d04-768fbf5c5615)
> Runnning the operation also creates a facet which can be used to select the rows based on whether they encountered errors or not:
> !\[image\](https://github.com/OpenRefine/OpenRefine/assets/309908/cf00e670-4ee7-496b-9b87-37b1d22aab49)
> This facet is generated by the GREL expression \`if(isError(value), 'failed edit', 'successful edit')\`.
> 
> For successful edits I think it would be nice to store some value (to distinguish it from no edit being made). I thought about adding a URL to the diff view of the edit (such as https://www.wikidata.org/w/index.php?diff=1920784148&oldid=1919995661).
> 
> This also closes #5166, since it makes it possible to relate the errors to the offending lines better.
> 
> You can try this PR on \[this sample project\](https://github.com/OpenRefine/OpenRefine/files/11822075/Wikibase-editing-error-example.openrefine.tar.gz): uploading that project to Wikidata will not make any edits (the first row fails to create an item, the second one does not make any edit because the data being pushed matches what is already on Wikidata).
> 
> 
> cc @OpenRefine/designers
> 
> This is meant to be a basic implementation that can be improved on (for instance, localization can be improved). I would like to release this in 4.0-alpha2 to gather feedback about the UX. One other question is where to insert the column with the editing results. Currently it is added after all other columns, which might be a bit inconvenient for projects with many columns.

Curious to know what you think of it!

---

<div class="post-metadata">

**Author:** ![olea](https://dub1.discourse-cdn.com/flex017/user_avatar/forum.openrefine.org/olea/32/231_2.png) [@olea](https://forum.openrefine.org/u/olea)\
**Post date:** [June 26, 2023, 7:50pm UTC](https://forum.openrefine.org/t/bug-report-io-error-while-editing-unexpected-character-code-60/809/11 "2023-06-26T19:50:38Z")

</div>

It looks good 👍  
.
