Drafting the 2024 User Survey

Community
15

When I distributed the final survey for review to a limited group of people, I received feedback challenging its structure and purpose. I believe these concerns are valid, and I want to ensure the survey is useful. My goal is to create an actionable survey for all community members, whether developers, designers, or community managers. Therefore, I am reopening the discussion about the survey design based on the feedback received.

I organized the feedback into four categories.

    1. Why are we doing this survey?
    1. Should we collect more demographics?
    1. Are we looking for current usage vs what the user wants?
    1. Other suggested questions

Initial Survey Design

You can review the initial survey design in the screenshot of the Google Form survey below. I also copy pasted the question in this new Google Doc for those who want to comment in line.

1. Why are we doing this survey?
I personally use the results for the following purposes:

  1. To gain a better understanding of our user demographic when presenting the community to partners or funders.
  2. To understand the general usage of OpenRefine, including which features, extensions, and reconciliation services are being used, in order to see if there are any changes. Over the last 5 surveys, no major shifts have occurred.
  3. To identify individual contributors who may not be vocal on the forum or GitHub. I contacted several of them during my 2023 interviews to recruit them for the advisory committee and invite them to the Barcamp.
  4. The 2022 survey helped to identify feature requests that have been implemented since.

2. Should we collect more demographics?

The survey is light regarding demographic information on age, educational level, and geographic location. If we collect that information, how will we use it?

3. Are we looking for current usage vs what the user wants?
During the initial survey design, part 2 was extended to get into extreme details (and lightly) details to understand how often different features are used and the level of expertise of the user on such features. The feedback I received is that parts 2 to 4 are overwhelming.

  • Are those information valuable to the development or design contributors?
  • If so, how do we make those sections shorter?
  • An alternative will be to anonymously track usage (with the user's consent) directly in OpenRefine. This would be a totally different project but should yield more accurate data.

On the other hand, do we want to ignore current usage to focus on what users need and want? For this, we set up a separate survey in the Allourideas platform to collect feature requests. I set it up as the last question since I am concerned that many users may not complete the survey if we redirect them too early to AllOurIdea. The feature question opens in a new tab and starts a flow of 20 prompts. By the time the participant answers them, they may have moved on, and they may not return to complete the survey. I am open to any suggestion to address this.

To facilitate feedback, I am creating a poll in Discourse, feel free to comment to add details on your vote. If you're reading this from your email, open the thread in your browser to vote.

What should we focus on
  • Current usage using the survey
  • Current usage using an automated collection
  • Feature request using allourideas
0 voters

4. Other suggested questions

I also received the following questions.

  1. How did you learn to use OpenRefine? (Book, web resource, in-person training, etc, etc)
  2. What is your prefered way of learning about OpenRefine?
  3. Is it important to you to have the OpenRefine UI translated into your native language?
  4. What languages would you like to see the OpenRefine interface translated into?
  5. Do you prefer paid professional support or free peer support?
  6. Tell us how you use OpenRefine in your workflows
  7. How often do you need to work with non-Latin based alphabets? What additional capabilities do you need to help with this?
  8. How much memory (RAM) do you allocate to OpenRefine? (Leave blank if you use the default): ____ MB
  9. If you've increased the maximum number of text facet choices, what value did you increase it to? (Leave blank if you use the default)
  10. Thinking specifically about OpenRefine as a tool, what do you see as its biggest weakness?
  11. Thinking about the OpenRefine ecosystem as a whole, what do you see as its biggest weakness? Biggest strength?
  12. What feature would you most like to see implemented/improved in OpenRefine?