Looking for example recipes (JSON representations of operations)

Development & Design
1

Hi all,

As part of my work on improving the usability of the Extract / Apply feature of OpenRefine's history tab, I am looking for examples of operation scripts (in JSON) that people are already using / sharing. I call those "recipes" (we should also have a broader discussion about whether we want to adopt this terminology in the tool itself). I am looking for literally any sort of example:

  • a recipe you saved somewhere because you want to reuse it from time to time, or
  • something that was published in a tutorial / a research article / anything else, or
  • the history of some random OpenRefine project you just did, which isn't necessarily meant to be reusable in other contexts.

Here are some examples I already know of, found by a combination of various methods:

Feel free to point me to more of them!
I will go through this list and evaluate the UX of applying those recipes, checking that they are properly rendered and that the column names used in the scripts are correctly detected.

2

Dunno if it still works, but this one mixed API requests plus some calculations and regexes.
Refine_RCS/rcs exploit refine.txt at master · hpiedcoq/Refine_RCS · GitHub

1 Like
3

@antonin_d I reached out to University of Washington and they'll provide some they use for MARC21 / RDF, etc. by the end of January.

1 Like
4

Thanks a lot both, that's super helpful!

Would any of you be keen to get on a quick call with me, for you to try out the recipe visualization feature and for me to observe your intuitive reactions? I am quite flexible in the coming weeks.

1 Like
5

@antonin_d I would be interested for sure. From the user survey 2024, I can also extract a list of user who indicated using recipe and who are open to do usability testing and review. Let me know if you are interested and I can share the list privately.

6

That would be fantastic! I trust Thad to raise quite a few usability issues for me to weed out, getting feedback from more people afterwards would be ideal :slight_smile:

2 Likes
7

After the testing session with Thad, I'd like to do similar ones with others from the core team before opening to the broader user community. I'll try to arrange a meeting with Martin on this. If anyone else is keen to have an influence on the early prototype let me know. (@ostephens perhaps?)

Note that I am intentionally not posting screenshots or videos of the recipe visualization feature here, so that I can see your intuitive reaction to the visualization without more explanations (as it should be as self-explanatory as possible). If you have already tried it out by running the code in my GitHub fork that's fine, but it's helpful for me to know that :slight_smile:

1 Like
8

Thanks for your work on this, Antonin! Looking at your list of examples made me think how cool it would be to have an "OR-awesome" kind of library of such recipes, ready for reuse or inspiration.

1 Like
9

I am adding GitHub - paldhous/refine-geocoder: JSON to geocode list of addresses in OpenRefine, using HERE and OpenStreetMap Nominatim APIs to the list of example.

10

Hi! I have some older code from a library project that transformed MARC21 data to Wikidata using OpenRefine. I posted the instructions on this Wikidata page in both English and JSON: Wikidata:WikiProject PCC Wikidata Pilot/University of Washington/Workflow, Trainings, and Resources/MARC21 to Wikidata Workflow - Wikidata I was going to clean them up and test them before posting here, but I don't have capacity to do it and this project has been somewhat orphaned. You are welcome to the recipes!

2 Likes