OR 3.7beta2 or 3.8-20221220.184714 (Java included) do not work

Yann · December 26, 2022, 5:33pm

Hi, I am trying to upload files to Commons, so I tried OR 3.7beta2 and OR 3.8-20221220.184714 (Java included).
OR 3.8-20221220.184714 loads OK, but the “Next” button after selecting files is not accessible. I am stuck there.
OR 3.7beta2 doesn’t even load (Java not found).
Second issue: Firefox is used, although Chrome is my default browser. OR loads in Chrome when copying-paste the URL http://127.0.0.1:3333/ to it.

antonin_d · January 2, 2023, 8:10pm

Hi!
Thanks for reporting this, it is curious.
Could you share:

a screenshot of your OR 3.8 showing the fact that the “Next” button is not accessible
in the same situation, a copy of your browser’s console (right click anywhere in OpenRefine’s interface, click “Inspect”, and then go to the console tab and copy or screenshot everything there)

This should help us locate the problem for OR 3.8 at least.

Yann · January 5, 2023, 5:07pm

Thanks for your answer. Here is screenshot of OR 3.8. Crome on Windows 10.

Yann · January 5, 2023, 5:09pm

JQMIGRATE: Migrate is installed with logging active, version 3.4.0
index-bundle.js:63403 Language 'fr' missing translation. Defaulting to 'en'.
success @ index-bundle.js:63403
index-bundle.js:11135 JQMIGRATE: jQuery.trim is deprecated; use String.prototype.trim
migrateWarn @ index-bundle.js:11135
index-bundle.js:11137 console.trace
migrateWarn @ index-bundle.js:11137
index-bundle.js:11135 JQMIGRATE: jQuery.isFunction() is deprecated
migrateWarn @ index-bundle.js:11135
index-bundle.js:11137 console.trace
migrateWarn @ index-bundle.js:11137
index-bundle.js:11135 JQMIGRATE: jQuery.fn.unbind() is deprecated
migrateWarn @ index-bundle.js:11135
index-bundle.js:11137 console.trace
migrateWarn @ index-bundle.js:11137
index-bundle.js:11135 JQMIGRATE: jQuery.fn.bind() is deprecated
migrateWarn @ index-bundle.js:11135
index-bundle.js:11137 console.trace
migrateWarn @ index-bundle.js:11137
DevTools failed to load source map: Could not load content for chrome-extension://gighmmpiobklfepjocnamgkkbiglidom/browser-polyfill.js.map: System error: net::ERR_FILE_NOT_FOUND
DevTools failed to load source map: Could not load content for http://127.0.0.1:3333/underscore-umd.js.map: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE

Yann · January 5, 2023, 5:15pm

Same with Firefox

Yann · January 5, 2023, 5:17pm

Console with FF:

Le cookie « host » n’a pas de valeur d’attribut « SameSite » appropriée. Bientôt, les cookies dont l’attribut « SameSite » est manquant ou défini avec une valeur invalide seront traités comme « Lax ». Cela signifie que le cookie ne sera plus envoyé dans des contextes tiers. Si votre application dépend de la disponibilité de ce cookie dans de tels contextes, veuillez lui ajouter l’attribut « SameSite=None ». Pour en savoir plus sur l’attribut « SameSite », consultez https://developer.mozilla.org/docs/Web/HTTP/Headers/Set-Cookie/SameSite 127.0.0.1:3333
Cette page utilise la propriété non standard « zoom ». Envisagez d’utiliser calc() dans les valeurs des propriétés pertinentes ou utilisez « transform » avec « transform-origin: 0 0 ». 127.0.0.1:3333
JQMIGRATE: Migrate is installed with logging active, version 3.4.0 index-bundle.js:11103:17
L’utilisation d’XMLHttpRequest de façon synchrone sur le fil d’exécution principal est obsolète à cause de son impact négatif sur la navigation de l’utilisateur final. Consulter http://xhr.spec.whatwg.org/ pour plus d’informations. index-bundle.js:10110:8
Language 'fr' missing translation. Defaulting to 'en'. index-bundle.js:63403:25
JQMIGRATE: jQuery.trim is deprecated; use String.prototype.trim index-bundle.js:11135:12
console.trace() index-bundle.js:11137:13
    migrateWarn http://127.0.0.1:3333/index-bundle.js:11137
    prop http://127.0.0.1:3333/index-bundle.js:11169
    getData http://127.0.0.1:3333/index-bundle.js:38101
    headers http://127.0.0.1:3333/index-bundle.js:36284
    map http://127.0.0.1:3333/index-bundle.js:469
    buildHeaders http://127.0.0.1:3333/index-bundle.js:36250
    setup http://127.0.0.1:3333/index-bundle.js:35979
    tablesorter http://127.0.0.1:3333/index-bundle.js:38327
    each http://127.0.0.1:3333/index-bundle.js:387
    each http://127.0.0.1:3333/index-bundle.js:209
    tablesorter http://127.0.0.1:3333/index-bundle.js:38316
    _renderProjects http://127.0.0.1:3333/index-bundle.js:65109
    success http://127.0.0.1:3333/index-bundle.js:64943
    fire http://127.0.0.1:3333/index-bundle.js:3567
    fireWith http://127.0.0.1:3333/index-bundle.js:3697
    done http://127.0.0.1:3333/index-bundle.js:9909
    callback http://127.0.0.1:3333/index-bundle.js:10170
    send http://127.0.0.1:3333/index-bundle.js:10222
    ajax http://127.0.0.1:3333/index-bundle.js:9803
    <anonyme> http://127.0.0.1:3333/index-bundle.js:11378
    prop http://127.0.0.1:3333/index-bundle.js:11180
    _fetchProjects http://127.0.0.1:3333/index-bundle.js:64938
    _buildTagsAndFetchProjects http://127.0.0.1:3333/index-bundle.js:64914
    OpenProjectUI http://127.0.0.1:3333/index-bundle.js:64883
    renderActionArea http://127.0.0.1:3333/index-bundle.js:64506
    <anonyme> http://127.0.0.1:3333/index-bundle.js:64510
    mightThrow http://127.0.0.1:3333/index-bundle.js:3833
    process http://127.0.0.1:3333/index-bundle.js:3901
JQMIGRATE: jQuery.isFunction() is deprecated index-bundle.js:11135:12
console.trace() index-bundle.js:11137:13
    migrateWarn http://127.0.0.1:3333/index-bundle.js:11137
    prop http://127.0.0.1:3333/index-bundle.js:11169
    buildCache http://127.0.0.1:3333/index-bundle.js:36719
    setup http://127.0.0.1:3333/index-bundle.js:35997
    tablesorter http://127.0.0.1:3333/index-bundle.js:38327
    each http://127.0.0.1:3333/index-bundle.js:387
    each http://127.0.0.1:3333/index-bundle.js:209
    tablesorter http://127.0.0.1:3333/index-bundle.js:38316
    _renderProjects http://127.0.0.1:3333/index-bundle.js:65109
    success http://127.0.0.1:3333/index-bundle.js:64943
    fire http://127.0.0.1:3333/index-bundle.js:3567
    fireWith http://127.0.0.1:3333/index-bundle.js:3697
    done http://127.0.0.1:3333/index-bundle.js:9909
    callback http://127.0.0.1:3333/index-bundle.js:10170
    send http://127.0.0.1:3333/index-bundle.js:10222
    ajax http://127.0.0.1:3333/index-bundle.js:9803
    <anonyme> http://127.0.0.1:3333/index-bundle.js:11378
    prop http://127.0.0.1:3333/index-bundle.js:11180
    _fetchProjects http://127.0.0.1:3333/index-bundle.js:64938
    _buildTagsAndFetchProjects http://127.0.0.1:3333/index-bundle.js:64914
    OpenProjectUI http://127.0.0.1:3333/index-bundle.js:64883
    renderActionArea http://127.0.0.1:3333/index-bundle.js:64506
    <anonyme> http://127.0.0.1:3333/index-bundle.js:64510
    mightThrow http://127.0.0.1:3333/index-bundle.js:3833
    process http://127.0.0.1:3333/index-bundle.js:3901
JQMIGRATE: jQuery.fn.unbind() is deprecated index-bundle.js:11135:12
console.trace() index-bundle.js:11137:13
    migrateWarn http://127.0.0.1:3333/index-bundle.js:11137
    prop http://127.0.0.1:3333/index-bundle.js:11169
    bindEvents http://127.0.0.1:3333/index-bundle.js:36173
    setup http://127.0.0.1:3333/index-bundle.js:35999
    tablesorter http://127.0.0.1:3333/index-bundle.js:38327
    each http://127.0.0.1:3333/index-bundle.js:387
    each http://127.0.0.1:3333/index-bundle.js:209
    tablesorter http://127.0.0.1:3333/index-bundle.js:38316
    _renderProjects http://127.0.0.1:3333/index-bundle.js:65109
    success http://127.0.0.1:3333/index-bundle.js:64943
    fire http://127.0.0.1:3333/index-bundle.js:3567
    fireWith http://127.0.0.1:3333/index-bundle.js:3697
    done http://127.0.0.1:3333/index-bundle.js:9909
    callback http://127.0.0.1:3333/index-bundle.js:10170
    send http://127.0.0.1:3333/index-bundle.js:10222
    ajax http://127.0.0.1:3333/index-bundle.js:9803
    <anonyme> http://127.0.0.1:3333/index-bundle.js:11378
    prop http://127.0.0.1:3333/index-bundle.js:11180
    _fetchProjects http://127.0.0.1:3333/index-bundle.js:64938
    _buildTagsAndFetchProjects http://127.0.0.1:3333/index-bundle.js:64914
    OpenProjectUI http://127.0.0.1:3333/index-bundle.js:64883
    renderActionArea http://127.0.0.1:3333/index-bundle.js:64506
    <anonyme> http://127.0.0.1:3333/index-bundle.js:64510
    mightThrow http://127.0.0.1:3333/index-bundle.js:3833
    process http://127.0.0.1:3333/index-bundle.js:3901
JQMIGRATE: jQuery.fn.bind() is deprecated index-bundle.js:11135:12
console.trace() index-bundle.js:11137:13
    migrateWarn http://127.0.0.1:3333/index-bundle.js:11137
    prop http://127.0.0.1:3333/index-bundle.js:11169
    bindEvents http://127.0.0.1:3333/index-bundle.js:36174
    setup http://127.0.0.1:3333/index-bundle.js:35999
    tablesorter http://127.0.0.1:3333/index-bundle.js:38327
    each http://127.0.0.1:3333/index-bundle.js:387
    each http://127.0.0.1:3333/index-bundle.js:209
    tablesorter http://127.0.0.1:3333/index-bundle.js:38316
    _renderProjects http://127.0.0.1:3333/index-bundle.js:65109
    success http://127.0.0.1:3333/index-bundle.js:64943
    fire http://127.0.0.1:3333/index-bundle.js:3567
    fireWith http://127.0.0.1:3333/index-bundle.js:3697
    done http://127.0.0.1:3333/index-bundle.js:9909
    callback http://127.0.0.1:3333/index-bundle.js:10170
    send http://127.0.0.1:3333/index-bundle.js:10222
    ajax http://127.0.0.1:3333/index-bundle.js:9803
    <anonyme> http://127.0.0.1:3333/index-bundle.js:11378
    prop http://127.0.0.1:3333/index-bundle.js:11180
    _fetchProjects http://127.0.0.1:3333/index-bundle.js:64938
    _buildTagsAndFetchProjects http://127.0.0.1:3333/index-bundle.js:64914
    OpenProjectUI http://127.0.0.1:3333/index-bundle.js:64883
    renderActionArea http://127.0.0.1:3333/index-bundle.js:64506
    <anonyme> http://127.0.0.1:3333/index-bundle.js:64510
    mightThrow http://127.0.0.1:3333/index-bundle.js:3833
    process http://127.0.0.1:3333/index-bundle.js:3901
Erreur dans les liens source : Error: request failed with status 404
URL de la ressource : http://127.0.0.1:3333/index-bundle.js
URL du lien source : underscore-umd.js.map

antonin_d · January 5, 2023, 8:16pm

In your screenshot above, I do not see where a “Next” button is missing. Can you simply click on “Configure parsing options” in the top right corner? That should lead you to the next screen.

Yann · January 5, 2023, 9:42pm

I get an error message saying “Attempt to parse an Excel file but failed. Try to use Excel to re-save as different Excel version or as TSV and upload again.” There is no project created yet, so no Excel file exists.

Then I get “Error: Job not ready.” and back on the first page.

I also tried to create an empty Excel file project, but then I don’t see how to add files into it. Anyway, the documentation says to select files first ( Starting a project | OpenRefine ).

There should be a Next button at the bottom, as shown here: 2.2 Create an OpenRefine project (Google Doc). Documentation given at Commons:OpenRefine - Wikimedia Commons (“Upload files to Wikimedia Commons with OpenRefine (version 3.7 and newer)”).

ostephens · January 5, 2023, 10:54pm

It looks like you are trying to create an OpenRefine project from a PDF file? I’m afraid it is not possible to do this. You need to start from a format that can be read by OpenRefine into an OpenRefine project. Examples of compatible file formats include Excel, OpenOffice, Wikitables, CSV/TSV, JSON, XML.

Perhaps you could explain further what you are trying to do with OpenRefine and we could help give guidance on how to do that?

Yann · January 6, 2023, 4:45pm

No, I am trying to upload some PDF files (around 10,000), and I followed the information provided in the documentation. That’s why I am using version 3.8. Or the documentation is badly written? Looking at the interface, what I did seems logical, although naming the project before selecting the files would be better IMO.

ostephens · January 6, 2023, 5:55pm

Hi @Yann

I'll start by saying I've not used the process of uploading files to WikiCommons, (I use OpenRefine extensively for more general data cleaning processes, and although I do have quite a bit of experience using it to upload data to Wikidata, I've not used the new Wikicommons functionality yet).

As the functionality to work with Wikicommons is new, it maybe that the documentation is still in development - the information I found did seem to suggest this.

However - I hope that I can help get you started.
The instructions I found at OpenRefine 3.7 - How to upload new files to Wikimedia Commons - Google Docs say:

2.1.1 I want to upload files from my own computer / local drive
You will need to locate the files in the Finder / File explorer on your local computer, and copy their absolute file paths.

I'm guessing this is what you are trying to achieve?

If so, the first step is to get a list of the file paths - it's this list (and not the files themselves) that forms the basis of the OpenRefine project you are going to work with. So you need to produce just the text list of files something like:

C:\Documents\myPDFs\xxx-01-7272791-Warren-Minnesota-109.pdf
C:\Documents\myPDFs\xxx-01-8554908-Crosby-NorthDakotea.pdf

Obviously the exact path will depend on where the files are stored on your computer.

The next step described is:

2.2 Create an OpenRefine project
You can now use this list of files to start an OpenRefine project. Any way is good: you can paste the list from clipboard, or import it as a csv file, from a spreadsheet

So using the example above you might do something like:

and after clicking "Next"

Then after clicking "Create project"

The next steps require the addition of columns to the project which are detailed in the document at OpenRefine 3.7 - How to upload new files to Wikimedia Commons - Google Docs

I hope that the information above is enough to get you started, but if you have more questions feel free to ask.

ostephens · January 6, 2023, 6:00pm

Via another issue on this forum I’ve just seen a link to a YouTube video on using OpenRefine to upload files to WikiCommons which might make the process clearer

Yann · January 6, 2023, 11:34pm

OK, this is quite different to what I read in the documentation.

First try with a set of 242 files (among 10,000), and I get a message “Memory usage: 26214392% (268435366/1024MB)” in red. Resource monitor says OR uses more than 4 GB… not so good.

ostephens · January 7, 2023, 10:00am

Hi @Yann

At what point in the process do you see these issues?

Topic		Replies	Views
New items not uploading to Commons (error about not allowing text/html mime type) Support and Helpdesk wikimedia-commons	12	154	April 10, 2024
Problem adding Commons reconciliation service Support and Helpdesk reconciliation , wikimedia-commons	6	443	May 23, 2024
Problem with Wikimedia Commons schema - not showing media options Support and Helpdesk wikimedia-commons	3	418	January 13, 2023
Error on uploading files to Wikimedia commons Support and Helpdesk wikimedia-commons	7	311	December 14, 2022
I can't edit schemas in 3.8.2 Support and Helpdesk	2	30	August 12, 2024

OR 3.7beta2 or 3.8-20221220.184714 (Java included) do not work

Related topics