Regular Checkups Are Vital To Community Health

People, in general, have a hard time moving outside their comfort zone. Or even thinking about it. We can plan ahead for events that cause us discomfort, but when things are going well--especially when they are going well--we can fall into the trap of complacency.

For example, right now I am dealing with a head cold. It's no big deal, I am still plodding along at work, getting things done. But I did not plan for this, and my productivity levels are correspondingly low this week.

As I recover from this minor illness, the benefits of hindsight run through my brain. Should I have eaten healthier food? Taken vitamins? Washed my hands more? All good ideas that I should have been thinking about last week or whenever I caught this cold. But I likely did not, so here I am, sniffling. There is, of course, a chance I would have caught this cold regardless of any precautions, but I cannot help but wonder what might have been.

Many people do not think about being sick when they are healthy. They tend to be in the moment, enjoying their health, perhaps not even consciously. This is why it can be hard for people to think about getting maintenance exams from their health-care providers.

It should be emphasized that this is not everyone. There are also lots of people who are very good about maintaining their health, putting a priority on preventative care. Personally, I salute such people.

The mindset of preventative care, whether it is firmly entrenched in someone's behavior or not, can overlap into other areas. As you can imagine, open source community health can be one of those areas.

Shifting the Perspective

Community health can be a complex thing to manage even when community leaders are at the top of their game. There is never just one specific thing someone can point to and say "See? My community is healthy." In the past, some would point to number of downloads as a health indicator. But even a bazillion downloads does not reflect that fact that your production and QA processes might be out of sync. Or you have trolls running around unchecked in your project mailing lists.

There are various ways to measure community health, many of them centered around not just results (downloads, release cadence) but also around processes (onboarding barriers to entry, how easily are maintainers discovered and promoted). Project CHAOSS, with which our Open Source and Standards (OSAS) team has been working, has listed many actionable metrics.

Knowing the effective measures of what can be measures is the first step. The next step is, how do you go about actually measuring? If you are a busy community lead with a lot on your plate, how do you take on even more administrative overhead to look at metrics?

One way could be how OSAS went about it: conducting a series of independent audits on various communities to see how they measured up from an "outsiders'" point of view.  This outsiders' perspective on community infrastructure, processes, and presence, was designed to identify areas that could be improved to maintain overall community health.  The audit was thought of as a “welcomeness check” for the project. 

Using predetermined questions (built around Project CHAOSS metrics), an auditor reviewed a project. Aspects such as web page design, presence of clear contribution guidelines, and ease-of-discovery of project software were just a few of the questions asked.

Once data was gathered, a complete report document was created by the auditor, highlighting the discovered strengths and weaknesses of the project and its community. Upon completion, the report was reviewed by the project auditor and the community manager. The review included discussion of perceived issues, resolution of issues, and prioritization of any suggested fixes. This would ensure that the audit did not artificially inflate the priority of issues of which the community lead was already aware.

When an agreement was reached on the final report (which will include action items to fix any discovered issues), an external version of the report could be formally published to the appropriate project if that is what the receiving community lead wished.

By conducting these audits in an independent way, two big goals were accomplished:

• Community leads gained a different perspective on potential trouble spots within their communities that they may not have realized were trouble. (They also gained insights into successes.)
• Specialists within OSAS would have a number of new tasks identified for them to work on improving in the coming months alongside the community leads.

Stepping back and asking for help looking at your community is a great way to find things that, while you are in the middle of the community, you might have missed. As we conduct a post-mortem on the questions and audit process itself, we hope to have a final, public set of questions that any community can modify and use for their own audits. So, watch this space!

Image courtesy of Max Pixel, under CC0 1.0 license.