Moving project location to a cloud

I know OR for Mac stores all of the projects in ~/Library/Application Support/OpenRefine but I'm increasingly wanting to access my projects from different machines. The way I manage this with other systems is to put my files into cloud storage (I mostly use Microsoft's OneDrive) but I couldn't see a way to specify that OR use a different project directory. My solution was to move the OpenRefine directory to a cloud-synchronised location on my machine and then create a symbolic link to it in ~/Library/Application Support/ on all the the machines that I wanted to work with. This seems to work fine - I now have OR instances on different machines which are seeing the same set of projects. My questions are: is there a better way to do this, and are there any gotcha's I should be aware of? I assume it's a good idea to keep the OR versions the same on every machine. And I'm also being cautious not to have the same project open on both machines simultaneously and I'm not sure if the project files could get into an ambiguous or corrupted state when being manipulated from more than one OR instance.

1 Like

@jquartel you can change the directory used to store projects with the JVM argument
-Drefine.data_dir

On mac this can be modified in the info.plist file (see Running OpenRefine | OpenRefine)

A few years back I ran OpenRefine in the way you describe with a synchronised project directory between a laptop and desktop (I changed the directory used by OR using the JVM argument rather than linking the default directory). I don't remember all the details now, but my recollection is that it generally worked well, and allowed me to easily work across two computers. I have a feeling that I did occasionally have issues with the file sync from one computer conflicting with the file sync from the other leading me to get an older version of the project synchronised - so I definitely would agree about not having the projects open on both machines at the same time and I think your concern this could lead to problems is justified. I don't know how this works for OneDrive but with Dropbox it was possible to revert files in a folder to a previous point in their history so this offered some protection against files in the workspace directory getting overwritten by the sync.

I stopped this practice when I moved to just using a laptop again rather than laptop+desktop - so my motiviation to stop doing it wasn't driven by any particular issues with the approach but rather just my need to have this setup went away.

So my (vaguely recalled!) experience is that this setup can work but with some care! Its not a wholehearted recommendation - clearly OR isn't designed with this particular setup in mind - but it's a "it seemed to mainly work for me for a couple of years" :slight_smile:

2 Likes

Super, thanks Owen - I'll give the Drefine.data_dir setting a try (and sorry I didn't find it in the documentation on my first search). Having said that, I suspect the sym link approach will withstand new OR version installs without having to set the parameter again, no?
I do think centralised data in the cloud is becoming quite common practice, so maybe we could consider moving these preferences to some other static location (say, ~/Library/Preferences/OpenRefine on Mac) rather than within the installed package...? I did have an occasion where my laptop died and I was able to recover some of the project files from the disk but they wouldn't work when just dropped into the data directory of another machine - I had to carefully work out the original project name (from the data within), rename the project archive, then 'import' each archive (from the app) into the other machine's OR. So I do feel a lot more comfortable with these projects sitting in the cloud and so far it's been working ok - the first opening of OR was very slow, but I think now that it's loaded the files locally it's swimming.

1 Like

I did this recently - I used a dropbox folder. It seemed to be working seamlessly until I acquired a new MacBook and tried to do the same with OpenRefine on the new one. For some reason, NONE of my projects are visible under "Open Projects", even though the directory points to the same place. I've tried re installing OpenRefine a few times and editing the directory file over and over. And no luck. Wondering if anyone else has any experience?