I am starting this thread to discuss the upcoming financing option for OpenRefine.
In 2019, OpenRefine began exploring a new sustainability model by seeking grants and corporate sponsorships. Four years later, we secured four grants from three organizations. Thanks to these grants, we have been able to maintain a full-time paid position for one developer (and our release manager), @antonin_d , and support several initiatives, such as
Having a part-time project manager position (myself).
Our current grants will cover our budget until December 31, 2024. We are currently seeking funding opportunities for 2024 and beyond to maintain our current funding levels and paid contributions at the same level.
The advisory committee has identified the following Request for Application as of interest:
NLNET with NGI0 Entrust and NGI0 Core applications, which are due October 1, 2023.
the Chan Zuckerberg Initiative (CZI) Letter of Intent for EOSS 6 is due by October 17, 2023.
We would love to hear your idea on what type of development we should support with the grants. Do you have any specific projects in mind that we should work on? Are there particular features or improvements to OpenRefine that we should focus on? Should we provide more support for our user communities, like training, improved documentation, or support for attending events?
We'd appreciate any thoughts or ideas you have on this!
I think attending one of the major Computational Biology conferences would be a good first step to making other domains aware of OpenRefine's existence. Maybe reaching out to current ISCB president or maybe better someone on the ISCB Education Committee ? to see what they think and which conference on calendar might have best bang-for-buck for OpenRefine.
The understanding is that Computational Biology is ushering more and more AI approaches, but there's a huge need for clean data, transformations, and schema validation prior to training models (not to mention Reconciliation against Linked Data services)
I would like to share some suggestions for the EOSS-6 call for proposals and how we could use the grant to continue investing in day-to-day support for the community.
Firstly, we could improve the support for the trainer community. OpenRefine is currently missing out on important relationships with our trainer community. Trainers not only teach new users but also act as our ambassadors and provide the first level of support within their own community. On the other hand, they are relevant liaisons to escalate bugs and scope feature requests and inform the roadmap. In my recent interview, I found that trainers are shy to reach out to the core community. I think @Sandra work with the Wikimedia community (Open call: train-the-trainer program for OpenRefine-Wikimedia trainers) is a great example of how things can be improved.
Therefore, the grant would be used for the following purposes:
Developing a community to maintain training curricula for bioscience users, which will be the focus of the EOSS-6 grant. This will be similar to what the Carpentries are doing with their OpenRefine lessons for Library Carpentries Lesson , Scocial Science and Ecology
Offering a financial support for trainers to serve as liaisons between their local communities and OpenRefine.
Improve the feedback loop process from trainers and community leaders.
Support a project manager, which could evolve into a community manager role, would support the program administration.
Secondly, the grand would fund a technical position. The person would act as release manager and would help maintain the GitHub repository by triaging tickets, reviewing pull requests, and onboarding new technical contributors. That person could also help implement bug and feature request as reported by the community (including the trainer laison). Currently, I don't see the need to include support for specific new features as we are still in the process of releasing OpenRefine 4.x.
The grant duration is two years, so I propose that we begin on January 1st, 2025. This will ensure continuity from the previous grants, EOSS-5 and EOSS-Diversity, which both end on December 31st, 2024. Let me know if you have any questions or concerns.
I am looking forward to reading your thoughts and feedbacks.
You said “help implement bug and feature requests…”
But later say “ I don't see the need to include support for specific new features…”
Can you clarify if there is some grant limit or alignment issue with working on new features? Or was this a mistake in wording on your part on what the technical position can/cannot do to satisfy grant requirements? Because it looks very open for support needs:
support of open source software projects essential for biomedical research. The goal of the program is to support software maintenance, growth, development, and community engagement for these critical tools.
We should be clear to the community in summarizing the grant particulars of any “work efforts” supported. Then summarize opinions on if some potential planned work efforts have dependencies on prior active work currently in scope for existing grants.
IMO, for OpenRefine's user communities, the priority isn't 'trainer support' but rather reducing the need for extensive in-person training; this would allow trainers and users alike to spend more time actually using the tool doing their actual jobs.
I'm indeed co-ordinating the train-the-trainer program for Wikimedia OpenRefine users. I'm already noticing that time investment for myself and for the course participants is much higher than originally planned and budgeted. At least for the Wikimedia movement, which relies on volunteer work, on usually small and very tightly funded affiliate organizations, on donor money, and which works with usually quite underresourced partners, any cut in necessary training time and money would be extremely welcome.
Although OpenRefine has excellent documentation and tutorials available, my experience is that many/most users barely engage with them, even when pointed at them with great emphasis. And folks who have actually checked online tutorials and documentation still need help, sometimes over the long term. As software goes, OpenRefine requires an unusually high amount of human time and effort in training and mentoring. To alleviate this, it would be an enormous win if OpenRefine would make its interface more intuitive and conducive to self-paced learning. Some time ago @lozanaross brought up the idea of an in-app guided tutorial (similar to what you see in many video games); that would be amazing to have (and in first responses to a current survey for Wikimedia users I see that there is a lot of support for this idea). Removing jargon and generally making OpenRefine's UX more intuitive to use would also be so helpful.
I notice this not just with trainers, but with all users. The forum and other places where OpenRefine is discussed are very tech-oriented and perceived as intimidating. Asking trainers (who usually do training as a small side gig besides a day job) to report back to OpenRefine's community is a very big ask. In that regard, the best type of support would actually fully alleviate trainers' work; the most helpful thing OpenRefine could do for trainers (and users in general) would be to create a helpdesk-like separate infrastructure (manned by a helpdesk person / product manager or community liaison type) which interfaces with all users.
@thadguidry I wanted to share some ideas and get feedback from the community. I am open to changing the approach based on the feedback received. Thank you for sharing your thoughts.
Although our EOSS-1 and EOSS-5 grant applications focus on specific features, in general, EOSS grants do not require us to commit to any specific features, as you pointed out. I apologize for the confusion caused by my poorly worded text. What I meant to convey was that our technical support team would work to implement new features as requested by the community, rather than predetermined by the grant application.
And yes we need to better communicate our community wish list for future work and current efforts.
@Sandra Thank you for sharing your feedback. I agree with your analysis. However, I would like to point out that trainers play an essential role in contextualizing the use of OpenRefine for a particular community, which can be as small as a Library department in a University or as big as the Wikimedia community. Additionally, trainers often act as the first line of support, as they are familiar with the community and can better understand the context of the question being asked.
While I agree we can make the application a more user-friendly interface and provide better in-app guidance, it's crucial to remember that what may be considered intuitive for one group of users may not be the same for another. For instance, some users prioritize reconciliation, while others focus on making API calls and parsing JSON. Meanwhile, there are those who simply use OpenRefine for data exploration, with little need for data transformation.
@Martin My comments are exactly coming from that place. I am a trainer myself. As you mentioned earlier, I am working on establishing a group of trainers in the Wikimedia movement. As you can see in past posts on this forum, I have also established a non-Wikimedia trainer group in the Netherlands. So I interact with trainers all the time. Both groups I've created or am creating are fragile and full of very busy, overstretched, practical folks, who are working within underresourced communities. I hope you will take my remarks seriously as they are coming from continuous engagement with users and trainers. I'm trying to show you our perspective and situation here, which is only meant to help OpenRefine successfully interface with its users, which includes not overstretching us or burning us out.
In that regard, I want to highlight that the proposal risks to place unfair and unfit weight on trainers when it comes to community representation and contextualization. From my experience, trainers often juggle multiple responsibilities alongside their training duties. They tend to be pragmatic and hands-on, rather than networkers, strategic and high-level thinkers. Many/most of them may not desire to grow into such a role and be overwhelmed and become burnt out when pushed to do this. If OpenRefine wants to gain such insights, other approaches may work better, like the old idea of having community ambassadors who actually have strategic insight and are good networkers (this proposal, as I understand, has been ditched, but I think having individual ambassadors with a distinct profile - not a council - still has a lot of merit, outside of governance structures).
With respect to understanding communities and their specific questions, there's an obvious area where OpenRefine has room for improvement. The current user survey (I know it very well, because I coordinated the last edition) falls short in reaching OpenRefine's users comprehensively, and lacks depth in questions around user needs and priorities. The OpenRefine application itself could serve as a powerful instrument if it would occasionally prompt all users to fill in more in-depth surveys about their priorities and the issues they encounter. This will reach OpenRefine users more broadly and deeply than any trainer group ever would. Designing and managing such surveys is specialized and intensive work, and ideally should be compensated to make sure there's consistency and quality. It typically falls into the skillset of a good UX designer, and you are about to hire one, if I'm correct.
At a higher level, I'm concerned about OpenRefine's leadership pushing for more community members to contribute voluntarily. Being able to volunteer is an immense privilege, which many don't have, particularly among OpenRefine's less resourced user groups. Just becoming a volunteer involves a significant learning curve to understand the project's and its ecosystem's operations and how the core team prefers interaction.
Many of OpenRefine's user communities, including Wikimedia, already face severe funding challenges and operate on very limited resources. Given these circumstances, I strongly oppose asking more from these already stretched groups. Please let the strong (and not the weak) shoulders bear the weight more.
In that regard I wholeheartedly support OpenRefine's pursuit of EOSS-style funding and the hiring of a dedicated team, which will then also be tasked to secure ongoing financial support from more stable sources. Please let this team focus on lightening the burden on the community, and finding resources where they actually are. To be clear, I'm very happy that you are going to be hiring a designer and I am also 1000% in favor of hiring the technical position you mention.
If volunteers show up, of course that's a cherry on the cake, and they can contribute in significant ways. But having volunteers as the cornerstone makes the project socially unbalanced (only those with the privilege of time, energy, lucky personal situation, and advanced skill will have the capacity to show up) and makes it unstable (volunteers can leave at any time; staff can be replaced).
I fully agree and I am indeed very aware of that. I have trained and interacted with (non-Wikimedia) OpenRefine users with all those profiles. In all those cases, training the people with these diverse needs is each time individually different, it always comes with a steep learning curve, and lots of preparation for the trainer. In all these cases, it will be helpful for users, and it will vastly alleviate trainers, to have more easy, self-paced and self-guided onboarding and a more intuitive UX. There are shared issues among all these use cases (the general jargon and behavior of the interface) where improvements will make everyone's life better.
On my side, I support doing another funding application, ideally not promising the earth in terms of new features but rather focusing on continued maintenance, usability improvements and community development.
I would not commit to be able to work on such a grant if it is awarded, because I think there is value in leaving space for others and I have recently become more tempted to move on to other projects myself.
The concerns about the forum not being a welcoming place for users should be discussed more in detail, but let's do that in a different thread as it's a separate topic.
I am pleased to inform you that OpenRefine has been invited to present the full application for the EOSS-6 round. The full application is due on December 5th. I will share the final version of the letter of intent shortly.
Thank you all for your valuable feedback and support in making this submission stronger.
I am sharing the letter of intent for the EOSS-6 application. Currently, the document is on a separate branch for the website, as I am working on a page that will detail all the grants OpenRefine has received so far. But I didn't want to delay sharing the letter of intent any further.
