OpenRefine 2024 Barcamp: Strategizing OpenRefine Roadmap

This session (see Barcamp page) was highly productive, and help us strategize how we can continue the work on the different topic discussed during the Barcamp. We proposed using high-level epics or goalposts to effectively manage long-term goals and improve transparency. The group emphasized the importance of collaborative working groups and realistic timelines to align with our volunteer-driven efforts. The session also highlighted the potential benefits of introducing a product owner role to guide and refine our roadmap, fostering a structured yet flexible approach to project development.

How to Rank Requirements

The session began with a discussion on the lack of processes for ranking requirements. @tfmorris mentioned that currently, work gets done when people write grant applications based on user needs and funder interests. We need to remove the arbitrariness from this process. @martin complemented that the need for a roadmap is not just about funding; it's also about providing visibility to users regarding our future priorities. This helps decide whether to collaborate on a feature or develop it independently.

We organized the conversation into two threads. We first discussed how we can better manage high-level, long-term roadmap goals using epics or goal posts and how we can then translate them into our issue tracker. @tfmorris reminded the group of the risk when shifting tooling since it creates barriers when we search for information. @martin pointed out that many epics can't be broken down into issues easily because they require design work first and agreement from the community regarding what needs to be built. @ostephens reminded that long-term roadmaps should not be confused with independent bugs that can be fixed easily. Larger projects require more coordination and planning.

Creating Goal Posts or Epics

@tfmorris started the conversation by suggesting that blog posts should cover important topics. However, blog posts are not collaborative. @martin presented CiviCRM's Make It Happen approach to funding lists goals and shows the required resources. We could adopt a similar model for volunteers, time commitments and grant applications. Bi-yearly or yearly roadmap workshops could refine these goalposts.

The group discussed this proposition and indicated that goal posts are a good way to show how people can donate their time (@ostephens) and even be a starting point to create working groups to draft requirements (@mack-v). This system will also allow the community to vote and prioritize epics, instead of organizing individual issues (@lozanaross). Goal posts help us to pre-curate ideas for a few projects and then submit them for community voting (@mack-v).

Working group activities should be publicly visible by sharing their documents and meeting notes. They can help break big epics into smaller tasks (@mack-v). @ostephens pointed out that splitting epics and curating possible improvements takes significant work. @tfmorris: Before prioritization, we should break down big goals into smaller tasks with clear descriptions and estimate the effort required for each task. This helps relate the popularity of tasks to their priority.

Managing GitHub Issues

@martin asked if there is interest in using GitHub options to vote on issues and then sort them to see how many "+1" each issue gathers (see example with OpenRefine project). Overall, the group was not keen to go that route for the following reasons:

  • It mixes a work tracker and roadmap tool (@ostephens).
  • Many people aren't on GitHub, so their voices aren't heard (@lozanaross).
  • The top issues in the current sorting demonstrate the problem; some are not well-defined or reflect only a subcommunity's wishes (@tfmorris).
  • The issue's longevity also affects its position (@ostephens).

André referenced Sandra's voting process on the AllOurIdeas platform.

@thadguidry reminded the group of existing tools to manage GitHub issues:

  • A good filtering system with tags and categories can help offer filtered views of issues so people can see what they can work on.
  • GitHub Projects: OpenRefine Projects scope specific projects and group issues together in a kanban board. @lozanaross pointed out that GitHub Projects allows "notes" before they become issues. GitLab has better features for roadmaps and timelines.

Managing Timeline

Then the conversation moved on to the need to provide a timeline for when specific features or projects would be completed. @martin pointed out that being mostly a volunteer-run project makes it challenging to commit to specific release dates. @tfmorris added that we should be realistic and account for our resources. Volunteers work on what interests them or what they perceive as important. We could agree on a release schedule and advertise themes. @lozanaross mentioned that planned release dates would help. Include reasonably certain tasks based on funded projects or community commitments.

@ostephens noted that open-source work often aligns with funding availability. A governing body may oversee the product, but actual development depends on goodwill and funders. Transparency about this reality is better than having no plan. Announcing ambitions can steer contributions.

Discussion Regarding a Product Owner Role

The conversation also touched on the role of a product owner who would own the roadmap and help define the requirements. @tfmorris pointed out that product management is a non-trivial role requiring a specific skill set, and it's unclear how to resource it. @lozanaross mentioned that acquiring a product manager and UX designer could be part of the roadmap. Regular meetings about this would be beneficial. For @AtesComp, it is more important to start somewhere, even if it's not perfect.

Following this session, I will create a roadmap page on the website that lists each goal post to make it publicly visible. No timeline will be indicated at this stage. Each goal post will redirect to a discourse conversation (to be created), which will be structured as a project charter with links to relevant documents (forum conversations, Google Docs, GitHub issues, tags, or projects). On discourse, contributors can indicate their interest to volunteer and organize a working group. This can also serve as a reference point when seeking funding.

I will use the following pages as starting point to define the different goal post:

This will relate to the roadmap section of the draft playbook from Requesting Feedback: Documenting OpenRefine Community Handbooks