After adding rows and editing a cell, project failing to load

Hi, I'm on openrefine 3.92, Windows 10, Chrome.
I added blank rows to the end of my project. Then I edited a blank cell in one of the new rows, and clicked "apply to all identical cells". This caused the project to just spin and I can no longer even open the project. I can open and edit any other project except this one

I see this message when I try to open the project in question:
[project_utilities] All data files and backup data files failed to load (4ms)

I'm not sure what happened otherwise or how I can open this project.
Thanks for any help!
Jennifer

Hi @Jenyoung,

Sorry to hear that! It sounds like a very serious bug with the operation that adds new rows. Following your description of the issue, I have been able to reproduce it on my end, so I should be able to figure out a fix.
If your project contains valuable data, we might be able to salvage it. To do so:

  1. Try to open the project and note the project id found in the address bar of your browser (the long number at the end of the address, which should look like http://127.0.0.1:3333/project?project=2710973631026)
  2. Go back to OpenRefine's home page, go to the list of projects, and at the bottom click on "Browse workspace directory". This should bring up a directory with all your project data inside. Find the one that starts with the project id noted above
  3. Create an archive of that project and send it to me (antonin@delpeuch.eu)

I'll try to transform that into a valid project archive that you should hopefully be able to re-import on your side.

Many apologies for the trouble!

1 Like

I sent you the archive. I'm relieved it's a bug in a way. I couldn't figure out what I did incorrectly.
Thank you!
Jennifer

In case it's helpful, I couldn't reproduce this using the latest commit on the master branch. It wasn't until I checked out the 3.9 branch that I could reproduce the bug. I'll open an issue on the repository to track the this. Thanks for bringing this to our attention @jenyoung!

1 Like

Here's the GitHub issue to track this: Mass edit of new blank cells corrupts project in 3.9.2 · Issue #7245 · OpenRefine/OpenRefine · GitHub

2 Likes

Hi, I had the issue two days ago, after adding new line. After resort, project crashes. Same error message when trying to start.
Version 3.9.1.

I recently had a similar case, but it was easier to start from scratch instead of trying to fix the broken project or extract the data.

I also started to regularly export my projects before closing OpenRefine.
Maybe I should start automating this :thinking:.

Nevertheless I created a ticket with a feature request to create a general recovery tool or function for project recovery: Recovery tool for broken projects · Issue #7246 · OpenRefine/OpenRefine · GitHub.

I've found the root cause of the issue and developed a fix. Once this fix is released, existing projects should be able to load again.
I've also manually fixed @jenyoung's project for it to load in 3.9.2 already. I can do it for others if needed.

@b2m I'm curious how you'd automate project recovery in general. In this case it's a pretty ad-hoc fix.

Ideally we do have the initial state of the project and the history of the project.

We then should be able replay the history with a fixed version of OpenRefine, while (automatically) fixing/correcting broken action items in the history ad hoc.

Ideally we do have the initial state of the project and the history of the project.

That would make a lot of sense - it's also something that would greatly simplify the migration to a different architecture where operations are lazily computed on top of the original data. But currently, only the "current" state of the project is stored.

We then should be able replay the history with a fixed version of OpenRefine, while (automatically) fixing/correcting broken action items in the history ad hoc.

In this particular case, it wouldn't work, because the serialization of the operation metadata itself is corrupted. Because the operation can't be read from the project, it wouldn't be able to be replayed with that strategy.

1 Like

Hello together,
i have the second crash of the project, after adding new line. Same error message.
... 82 more
02:30:17.953 [ project_utilities] All data files and backup data files failed to load (2ms)
I have attached the json text after trying to backup.
Is there any way to fix this ?
Any help highly appreciated.
Kcim

error-3-9-2.pdf (73,1 KB)

@kcim , version 3.9.3 is available now, I could restore my seemingly corrupted project with this version. Maybe that helps you too?

1 Like

@colognella ; thanks, great to hear this. I was able to restore the corrupted project somehow, but i lost the history. Just in case it happens again :blush: ! Big thanks to the Openrefine Team @Rory :heart_hands:!

1 Like