Bug report: IO error while editing: Unexpected character ('<' (code 60))

Support and Helpdesk
1

I’m working in a project uploading 46 images to Wikimedia Commons. I was able to to upload 37 but I can’t the other 9. I’m getting this error:

18:13:06.259 [..ting.EditBatchProcessor] IO error while editing: Unexpected character ('<' (code 60)): expected a valid value (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
 at [Source: (okio.RealBufferedSource$inputStream$1); line: 1, column: 2] (150866ms)

Using the -v debug or trace flags the message is the same:

18:33:11.233 [..ting.EditBatchProcessor] IO error while editing: Unexpected character ('<' (code 60)): expected a valid value (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
 at [Source: (okio.RealBufferedSource$inputStream$1); line: 1, column: 2] (47807ms)

And I don’t really know where to find the offending “<”, since I’m not using any:

127.0.0.1_3333_project_project=2237704229443 (1)
127.0.0.1_3333_project_project=2237704229443 (1)1920×1797 146 KB

All the 46 files has been processed in the same batch with the same commands.

Any tip to diagnose and fix?

2

Not a very popular post, I guess :smiley:

3

I’m guessing it’s because of the TIFF file, the ability to upload TIFF files is offered as a courtesy but not recommended on Commons: COM:TIFF

We should maybe display a warning about TIFF files in the “issue” tab given how common TIFF failures are.

1 Like
4

This is interesting. What I can’t see is an alternative format to TIFF for Commons, being lossless and keeping metadata :thinking:

5

Maybe the problem origin is the Commons size limits for OpenRefine?

image

https://commons.m.wikimedia.org/wiki/Commons:Maximum_file_size

Seems my pending files are really big:
image

6

@antonin_d Do we not support chunking uploads currently? Seems that wiki page suggests to especially for TIFF files. I don’t see chunking happening here: Wikidata-Toolkit/wdtk-wikibaseapi/src/main/java/org/wikidata/wdtk/wikibaseapi/ApiConnection.java at f0c31c150e2213adffc4d77ed5ebe5f5448b0f3c · Wikidata/Wikidata-Toolkit · GitHub

And where Chunked Uploading is described further by one of the links to the API documentation in that wiki page.

@olea OpenRefine also does not do any TIFF compression (and neither does Wikidata Toolkit currently from what I see), which might help your situation. You might want to compress using any tool with LZW or deflate as one of the links Tiff Files on the wiki page suggests. Then upload the compressed TIFF files using OpenRefine?

7

@olea The alternatives to TIFF lossless are many actually nowadays. But you’ll have to see what Commons supports exactly or could (if you raised an issue, but I’ve also mentioned my support on a stalled 3 year issue: :anchor: T257719 Add support for AVIF: allow uploading AVIF files to Wikimedia servers)

Commons:Project scope/Allowable file types - Wikimedia Commons.

A few are:

  • PNG - lossless compression
  • WebP - 25% better compression typically than PNG and still retaining lossless ability.
  • AVIF (which uses AV1) - typically even better lossless compression (up to 50%) than WebP
8

What I can’t see is an alternative format to TIFF for Commons, being lossless and keeping metadata

@olea WebP should match those requirements I believe.

1 Like
9

Chunked uploading is not supported by OpenRefine yet and so uploads of files greater than 100MB will fail.

The error reporting for the Wikibase extension is unacceptably bad, in my opinion. I have some ideas around how to improve that and want to discuss them soon.

1 Like
10

@olea here is a preview of the UX I would like to offer in 4.0:

Curious to know what you think of it!

1 Like
11

It looks good :+1:
.