I'm wondering how to make progress on this.
I have to say those scheduling decisions do depend on deciding on a longer-term roadmap, because some of the features that would be listed there would require breaking changes, of course.
As I have just posted in the thread about extension support, I see the state of affairs there as very dire, so I'd be tempted to argue that for now we should only do breaking changes that are meant to improve extension stability (apart from vulnerability fixing, say), and then once that is stable we could work on new features again. Of course that would completely stall the reproducibility project, which is not exactly ideal to me…
Honestly our situation feels so bad that I am not sure if incremental improvements, even if they are carefully planned, are desirable at all. I have started to think that rewriting OpenRefine from scratch in a different stack is not that crazy of an idea. Over the past years, whenever people would encourage me to rewrite OpenRefine in a different language / framework (which happens regularly), I would politely smile and decline, saying that the satisfaction of maintaining existing software that is useful to many is more important to me than the satisfaction of working with the latest fanciest framework. Well, this problem is really making me reconsider that response.
Sure, rewriting OpenRefine is a lot more work than what is needed to patching the extension system. But if we do that work, we also have the opportunity to improve on a lot of other things:
- migrate out of Python 2.7 to something a bit more recent? There are prospects to do this while remaining with a Java backend (GraalPython, py4j) but there's still quite a lot of uncertainty around that in my opinion
- adopt a stack that is approachable and attractive to more developers, to make the project more sustainable? I think Java and jQuery put off a lot of people.
- by rewriting the UI in a modern stack, we'd likely use an existing widget library, which would give a fresh coat of paint on the UI in the same go
- although I have been trying to split down my new architecture into reviewable chunks, I still don't know how to split the central piece at all, so a rewrite from the ground up would be also be an occasion to introduce this sort of architectural change.
- we have a pretty decent end-to-end test suite that we might be able to keep and adapt if keep an eye on CSS selectors (assuming the UI is still web based, which I think I would support)
Maybe it's just me being too depressed about the state of things and I just need to take proper holidays (which I am actually counting on doing, yay!) but I feel like this sort of thoughts is worth voicing.