Hi, I’m new to Open Refine and trying to reconcile using reconcile-csv-0.1.2 from Reconcile-csv - join dirty data - Open Knowledge Foundation Labs (I’ve followed a colleagues instruction for how to set that all up)
I’ve added http://localhost:8000/reconcile as a standard service but then I get this response
Guess Types query failed error : java.io.IOException: HTTP error 500 : Server Error for URL /reconcile
Any ideas for how I trouble shoot this?
Welcome @Helen_Close
First thing is to check that reconcile-csv is running OK and accessible
After starting the reconcile-csv running can you visit http://localhost:8000/reconcile in a browser. You should get something like:
{"name":"CSV Reconciliation service","identifierSpace":"http:\/\/localhost:8000\/","schemaSpace":"http:\/\/localhost:8000\/","defaultTypes":[],"view":{"url":"http:\/\/localhost:8000\/view\/{{id}}"},"preview":{"width":500,"url":"http:\/\/localhost:8000\/view\/{{id}}","height":350},"suggest":{"entity":{"service_url":"http:\/\/localhost:8000","service_path":"\/suggest","flyout_service_url":"http:\/\/localhost:8000","flyout_sercice_path":"\/flyout"}}}
If that’s working then I’d look at the reconcile-csv and OpenRefine terminal logs next to see if there is anything in either of those places that gives some more information on what’s wrong. If you can share any information from there (and if you don’t know how to access those then I can try to help if you say if you are on Windows/Mac/Linux)
Thanks for your help.
Reconcile isn’t opening - I’m getting
This site can’t be reached
localhost refused to connect.
Try:
ERR_CONNECTION_REFUSED
FYI I’m on Windows 10 and have tried to open in both firefox and chrome (chrome without any privacy extensions). I also switching of my anti-virus and firewall temporarily.
Hi
I still can’t get the reconciliation to run in open refine but the link is now working (I have no idea why)
Am now getting this {“name”:“CSV Reconciliation service”,“identifierSpace”:“http://localhost:8000/”,“schemaSpace”:“http://localhost:8000/”,“defaultTypes”:,“view”:{“url”:“http://localhost:8000/view/{{id}}”},“preview”:{“width”:500,“url”:“http://localhost:8000/view/{{id}}”,“height”:350},“suggest”:{“entity”:{“service_url”:“http://localhost:8000”,“service_path”:“/suggest”,“flyout_service_url”:“http://localhost:8000”,“flyout_sercice_path”:“/flyout”}}}
Any help with the next steps woudl be appreciated
In the reconciliation data log I get the following
2023-01-16 10:44:02.994:WARN:oejs.AbstractHttpConnection:/reconcile
java.util.concurrent.ExecutionException: java.lang.NullPointerException
at java.util.concurrent.FutureTask.report(Unknown Source)
at java.util.concurrent.FutureTask.get(Unknown Source)
at clojure.core$deref_future.invoke(core.clj:2108)
at clojure.core$future_call$reify__6267.deref(core.clj:6308)
at clojure.core$deref.invoke(core.clj:2128)
at clojure.core$map$fn__4207.invoke(core.clj:2487)
at clojure.lang.LazySeq.sval(LazySeq.java:42)
at clojure.lang.LazySeq.seq(LazySeq.java:67)
at clojure.lang.RT.seq(RT.java:484)
at clojure.core$seq.invoke(core.clj:133)
at clojure.core$zipmap.invoke(core.clj:2713)
at reconcile_csv.core$reconcile_params.invoke(core.clj:130)
at reconcile_csv.core$reconcile.invoke(core.clj:140)
at reconcile_csv.core$fn__2664.invoke(core.clj:225)
at compojure.core$make_route$fn__534.invoke(core.clj:94)
at compojure.core$if_route$fn__522.invoke(core.clj:40)
at compojure.core$if_method$fn__515.invoke(core.clj:25)
at compojure.core$routing$fn__540.invoke(core.clj:107)
at clojure.core$some.invoke(core.clj:2443)
at compojure.core$routing.doInvoke(core.clj:107)
at clojure.lang.RestFn.applyTo(RestFn.java:139)
at clojure.core$apply.invoke(core.clj:619)
at compojure.core$routes$fn__544.invoke(core.clj:112)
at ring.middleware.keyword_params$wrap_keyword_params$fn__1341.invoke(keyword_params.clj:32)
at ring.middleware.nested_params$wrap_nested_params$fn__1383.invoke(nested_params.clj:70)
at ring.middleware.params$wrap_params$fn__205.invoke(params.clj:58)
at ring.adapter.jetty$proxy_handler$fn__81.invoke(jetty.clj:18)
at ring.adapter.jetty.proxy$org.eclipse.jetty.server.handler.AbstractHandler$0.handle(Unknown Source)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:363)
at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:483)
at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:931)
at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:992)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:856)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Unknown Source)
Caused by:
java.lang.NullPointerException
at fuzzy_string.core$bigrams.invoke(core.clj:8)
at clojure.lang.AFn.applyToHelper(AFn.java:161)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.core$apply.invoke(core.clj:617)
at clojure.core$memoize$fn__5049.doInvoke(core.clj:5735)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at fuzzy_string.core$dice.invoke(core.clj:20)
at reconcile_csv.core$score$fuzzy_match__2621.invoke(core.clj:76)
at clojure.core$map$fn__4207.invoke(core.clj:2487)
at clojure.lang.LazySeq.sval(LazySeq.java:42)
at clojure.lang.LazySeq.seq(LazySeq.java:60)
at clojure.lang.RT.seq(RT.java:484)
at clojure.core$seq.invoke(core.clj:133)
at clojure.core.protocols$seq_reduce.invoke(protocols.clj:26)
at clojure.core.protocols$fn__6026.invoke(protocols.clj:53)
at clojure.core.protocols$fn__5979$G__5974__5992.invoke(protocols.clj:13)
at clojure.core$reduce.invoke(core.clj:6175)
at reconcile_csv.core$score.invoke(core.clj:78)
at clojure.lang.AFn.applyToHelper(AFn.java:163)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.core$apply.invoke(core.clj:619)
at clojure.core$partial$fn__4190.doInvoke(core.clj:2396)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$map$fn__4207.invoke(core.clj:2487)
at clojure.lang.LazySeq.sval(LazySeq.java:42)
at clojure.lang.LazySeq.seq(LazySeq.java:60)
at clojure.lang.RT.seq(RT.java:484)
at clojure.core$seq.invoke(core.clj:133)
at clojure.core$sort.invoke(core.clj:2752)
at clojure.core$sort_by.invoke(core.clj:2769)
at clojure.core$sort_by.invoke(core.clj:2767)
at reconcile_csv.core$scores.invoke(core.clj:111)
at reconcile_csv.core$reconcile_param.invoke(core.clj:123)
at clojure.core$pmap$fn__6275$fn__6276.invoke(core.clj:6354)
at clojure.core$binding_conveyor_fn$fn__4107.invoke(core.clj:1836)
at clojure.lang.AFn.call(AFn.java:18)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
When selecting tools, it’s useful to evaluate how current and well supported they are. In the case of this Clojure based reconciliation service, it hasn’t been touched since mid-2015, so unless you’ve got strong Clojure skills, it might not be the best choice. The blog post isn’t dated, which makes it difficult to easily evaluate currency, but if you click through to the Github repository, you can see how (in)active it is.
The stack trace you got looks pretty similar to the one in issue #5, which could mean the problem is as simple as a typo in your column name, but you might want to give some consideration to using something more modern and well supported like csv-reconcile. The author of that tool seems responsive and the repo has zero open issues. Even if they were to disappear, it’d be easier to get help with Python than Clojure.
Tom