I’m using the latest version of OpenRefine (3.7.1) on macOS Ventura with the latest version of Opera (97.0.4). Trying to use the cell.cross function like I did numerous times before. But this time I get an error message I am not used to: Error: “java.lang.NullPointerException”. What could it be? There is no syntax error. Names of project and columns have been doubled checked. I’ve tried everything I could think of, alas, to no avail.
Cell.cross not working: getting the following error message: Error: "java.lang.NullPointerException"
1 Like
Could you reply with slightly more of the java stacktrace from your console log so we can see what part of OpenRefine is actually throwing a NullPointerException?
Thanks for replying. Could you point me to the consol log? I quoted the error message directly from the "Add a Column based on this column" window:
The cross function will return an array of rows from the second project because it’s possible that the ID could appear multiple times. This means you have to process the response from cross as an array - using either a control (General Refine Expression Language | OpenRefine) or function that will work with an array (GREL functions | OpenRefine)
I’d usually use the forEach control which would mean using something like:
forEach(cell.cross('corpus_alimentation_EN_complet','ID'),r,r.cells['U5'].value).join("|")
This is going to process each row returned, extract the value from the U5 column, and if there are multiple values join them together into a single value with a separator (the pipe character | here but you can use whatever character/string of characters you like)
If this doesn’t help and you are still seeing errors, please post more information on your issue including:
- version of OpenRefine
- what do you see in the GREL preview if you just use
cell.cross('corpus_alimentation_EN_complet','ID')?
2 Likes
cell.cross(‘corpus_alimentation_EN_complet’,‘ID’)[0].cells.US.value
or
cell.cross(‘corpus_alimentation_EN_complet’,‘ID’)[0].cells[“US”].value
should work. Documentation needs to be fixed.
Thanks for reporting that.
Regards,
Antoine
What I said in the previous comment is not true. Both forms are correct!
@ostephens comment in GH explains it.
Regards,
Antoine
Hi Owen,
Thank you for the helping hand. While it did not solved my issue, I like this answer and I will keep it bookmarked. Actually, there are no duplicate of ID, but we did found a problem, e.i. there where a small number with letters instead of strings of numbers exclusively. It resulted in a problem while using Python, (we were trying to resolve the problem outside of OpenRefine). However, as much as I know, it should not be a problem in OpenRefine.
A friend who offered to help has been able to do the crossing between the tables without problem. For this reason, I am much inclined to believe it is a problem specific to a certain system configuration, a problem of software. This friend works on Windows and he might be using an older version of OpenRefine, while I’m on a Mac with the latest OS and latest version of the software.
I don’t know. I hope either to figure out the problem soon, or simply to see it vanish, as it is a function I use semi-frequently, and I need to use it on another dataset this very week.
Right now I use version 3.7.2.
cell.cross('corpus_alimentation_EN_complet','ID') will return the exact same message of error (Erreur: java.lang.NullPointerException).
Bonjour Julien, @archilecteur,
I'm also on Mac, and I did use the cross() function many time, even participate very closelly in its bonification in v3.3, if I recall correctly, around 2018.
As far as I know, since that time, it ALWAYS worked for me, but of course, I cannot garantee that you system doesn't have a particular condition/situation that makes your setup generate this NPE error in the Java code. Maybe you should make sure that you have a java version not above v17.
I could offer you a remote session and look at your issue directly.
I use AnyDesk mostly for these situation, but the MacOS does have a builtin remote tool that could also be used.
As a side information, I live 1h out of Montréal, and I also work in a professional way with a researcher from the LATICCE, an UQAM « Laboratoire de recherche sur la découvrabilité et les transformations des industries culturelles à l’ère du commerce électronique ».
https://laticce.uqam.ca/
Regards,
Antoine
Same issue here, we tried to install java version 17 like Antoine recommended.
We ended up using sdk cli for sdkman to install java 17.0.9-oracle.
When we run java -version we get
java version "17.0.9" 2023-10-17 LTS
Java(TM) SE Runtime Environment (build 17.0.9+11-LTS-201)
Java HotSpot(TM) 64-Bit Server VM (build 17.0.9+11-LTS-201, mixed mode, sharing)
This did not solve the issue.
We tried to install openrefine again but via brew, brew install openrefine, still the same issue.
We then tried to install openrefine from source, cloning the repo, installing mvn and npm and running ./refine.
Now we still get the same error but it is a bit more elaborate, specifically:
Error: java.lang.NullPointerException: Cannot invoke "com.google.refine.ProjectMetadata.getName()" because the return value of "java.util.Map$Entry.getValue()" is null
We tried older versions as well, specifically version 3.7.6 same issue.
I am using Mac Os version 13.6.2
Any help would be greatly appreciated
I'm bumping this problem because I am having the same issue.
Has anyone created an issue in the bug tracker? If not, please do so
and include as much information as possible. That's where the dev team
organizes its work.
Antoine - what were the results of your interactive debugging session?
Bart - if you want to try the 3.8 Beta1 (please backup your data
first!), you should get a stack trace on the console with more
information about where the error is occurring
Since there have only been a few reports, I suspect this is a data
corruption issue rather than a coding problem, but the code can
probably be made more resilient and/or informative in the face of the
corruption.
Tom
About this issue, since I’m the one who started the tread back then. It didn’t get much attention, and we couldn’t resolve it. I had to find a workaround, using an older computer to do the cell.cross for a couple of months. Then I had to replace my main computer for an unrelated issue, going from a Macbook Pro M1 to a Macbook Pro M2, and the issue was gone.
Not of much help, I know, but to add a bit of information on this issue.
@Antoine, désolé de ne pas t’avoir répondu en novembre. Partant pour une rencontre si l’invitation tient toujours, suis à Montréal. Je tâche de retrouver ton adresse, et je t’écris.
I get the same issue with a Refine release 3.7.9 under ubuntu and `ava version "21.0.2" 2024-01-16 LTS
below the json failing operation
"baseColumnName": "default_code",
"expression": "grel:cells["Identifiants2 (default_code)"].cross("Masterdata","Identifiants2 (default_code)")[0].cells[id].value",
"onError": "set-to-blank",
"newColumnName": "id",
"columnInsertIndex": 1,
"description": "Create column id at index 1 based on column default_code using expression grel:cells["Identifiants2 (default_code)"].cross("Masterdata","Identifiants2 (default_code)")[0].cells[id].value"
Not conclusive. All the same information (project, operation, etc.) was run on my computer and didn't crash. Didn't run it from the code locally, so couln’d see the logs with a stack trace.
Regards,
Antoine
mon courriel si tu veux me rejoindre directement :
antoine@beaubien.qc.ca.
Cordialement, Antoine
I'm having a similar problem applying cell.cross from a script that has always worked for me before and continues to work for other users. It now generates the error "java.lang.NullPointerException" when I apply it or enter it in the expression window.
I've double-checked that the project and field names are correct within the expression, so I don't think it's user error.
I just changed to a new computer so I'm guessing that's related. I was using OpenRefine 3.7.5 on Windows 11 when I encountered the issue today. Checking java -version from the command line showed no versions. I installed OR 3.7.9 (with Java included) but the cross command continued to generate the same error, and trying to find my java version continued to come up empty.
I installed Java manually from Adaptium as recommended in the OR user manual - version 17. Now, the cell.cross still doesn't work, but java -version in the command line now returns: openjdk version "17.0.10" 2024-01-16
OpenJDK Runtime Environment Temurin-17.0.10+7 (build 17.0.10+7)
OpenJDK 64-Bit Server VM Temurin-17.0.10+7 (build 17.0.10+7, mixed mode, sharing)
Here is some of the terminal log after trying to add a column with cell.cross in the expression:
17:02:59.676 [ refine] GET /command/core/get-csrf-token (2ms)
17:02:59.693 [ refine] POST /command/core/log-expression (17ms)
17:02:59.700 [ refine] POST /command/core/add-column (7ms)
17:02:59.741 [ refine] GET /command/core/get-history (41ms)
17:02:59.757 [ refine] GET /command/core/get-project-metadata (16ms)
17:02:59.765 [ refine] GET /command/core/get-models (8ms)
17:02:59.777 [ refine] POST /command/core/get-all-preferences (12ms)
17:02:59.784 [ refine] POST /command/core/get-rows (7ms)
17:02:59.820 [ refine] POST /command/core/compute-facets (36ms)
17:02:59.844 [ refine] GET /command/core/get-preference (24ms)
17:03:06.253 [ refine] GET /command/core/get-history (6409ms)
17:03:06.274 [ refine] POST /command/core/get-rows (21ms)
17:03:06.307 [ refine] POST /command/core/compute-facets (33ms)
17:03:06.333 [ refine] GET /command/core/get-preference (26ms)
Not sure if I installed java incorrectly (I went with the default settings). In any case, I'm wondering how to fix this. Apologies for the long narrative but I'm hoping it helps for diagnosis. Let me know if I can provide more information. Thanks!
I created an issue to track this on the development side
1 Like
I created an issue to track this on the development side
Thank you Martin. If anyone has additional information, especially projects which reproduce the problem or more detailed error messages, please add them to the issue.
Tom
TL;DR These are likely user errors on how to properly use cross() if they are using OpenRefine 3.7 or newer.
Personally, all of my cross() problems were resolved after 3.7
What happens now is that
- if the user doesn't have real values but instead only has
nullvalues in the cells of the column requested
or - if they do not properly provide a String
columnNameas the 3rd argument
then they get in the Expression Preview the Error: java.lang.NullPointerException which actually is unfriendly and should instead say something like "Cross found no matching cell values".
Things actually still work properly since the last fix on cross() and the null handling bug(forgot whom, Antonin or Tom if I recall?)
Here's my tests just now with 3.7.9 with embedded Java on Windows 11
NOTE: Carefully notice in screenshots the expression changes and the grey colored preview output.
My column c has null values in rows 1 and 3, but 1212 in row 2.
Below, Not using proper syntax for 3rd argument (I didn't enclose with double quotes)
Below, Using double quotes around columnName argument and row matches are found, but values not unwound yet with my shortened expression until I ask for them
Below, Testing something out of scope, but just to see if evaluation is actually working well even through cross() functionality, and indeed it does so correctly where I asked to lookup the columnName that is the column b value and I do not get a red syntax parsing error. It said it couldn't find a column named 1212 , Great!
Below, Finally, trying the full correct syntax and asking for the column a values for the first [0] matching row of any cross() column value match. And it works beautifully and correctly as seen with value zzz being previewed correctly.
1 Like