In Search Of... Software Users

When I was kid, there was a pretty cool documentary show called In Search Of..., which examined various paranormal (Bigfoot, The Bermuda Triangle) and natural (killer bees, hurricanes) phenomena and mysteries. This may seem dull for a pre-teen kid, but it was narrated by Leonard Nimoy, so I was pretty much all in.

One of the things this show (and others like it) does is avoid any real conclusions. Episodes paint a picture of many possibilities, present some counter arguments, then wrap up with a vague innocuous statement like "Is there really a Bigfoot? Only time may tell."

(Yeah, not really big on the mystery solving.)

As entertaining as this show is, it's not really the vague approach community team members want when trying to solve their greatest mystery: "Who is using our software?"

Plain as the Nose?

It may initially seem like a dumb question: why wouldn't you know who your software users are? After all, you know there are downloads, you see people participating on mailing lists and chat channels--what's the problem?

The problem lies in the fact that what you see in your project's community channels is a representative sample at best, and likely not even a good sample at that.

Depending on your community, what you may be seeing on your communication channels (mailing lists, forums, IRC, Slack, StackOverflow, GitHub) are just the really passionate members of your community. There is an old axiom in the journalism community: those who write letters to the editor are either really, really happy about something, or really, really angry. The same notion can be applied to active and vocal community members. I have used some form of Mozilla Firefox since the days when it was known as Netscape Navigator... and to date, I have never posted a single word on any forum or chat channel. I just use it. If it ever breaks, I search online for a solution and consume that solution. None of these things, though, gives Mozilla and Firefox's team any notion of who I am and what are my use cases. This blog, actually, may be the first time they've heard of my use.

Even numbers are a tricky business. Many people in the software industry like the "golden arches" ideal of being able to say "X users served, just like McDonald's and their hamburgers. But downloads are a very hard thing to even track accurately, especially if your project has mirrors involved in the dissemination process. And, even if you think you have some good numbers, my colleague Michael Scherer has made a compelling argument why downloads aren't something you want to, er, hang your hat on.

Less Nails Sticking Out

This wasn't always the case, of course. Before the turn of the century, when all consumer and server software was shiny and new and pretty much broken. Free or commercial, early software that hadn't been born on a mainframe and was being consumed by the general public and a new breed of IT manager was clunky and likely to fly apart in spectacular blue screens of death or little computer icons with x-ed out eyes.

In those wild west days, books, magazines, and an Internet in its public infancy were the places to find solutions, and online communities were intimate affairs where a lot of people would come in for help. Developers were more apt to communicate with the Average User because everyone was too busy figuring out how all this stuff worked to stand on much ceremony.

Buggy software created a lot of nails sticking out, and there was a lot of hammering going on. These days, there are still nails to be hammered, but not as many and (usually) not as potentially catastrophic. Software is easier to install.. you can even install an entire Linux distribution in minutes and not worry about XF86Setup frying your monitor if you chose the wrong refresh rate.

The interoperability and ease of software across all platforms and licenses has, I believe, created less strong user-based communities, because when things are running smoothly, the need to band together decreases. It doesn't disappear completely, of course, but the "Average User" tends to be much quieter than the louder disgruntled this-is-broken-and-I-want-it-fixed-now users and the passionate this-is-the-best-thing-since-sliced-bread advocate.

This leads to the challenge for community members who very much want to know who's using their software: contented users are quiet users, but you want to know what they are getting from your software so they stay contented.

Ask many community participants and leads what their biggest unknown is and they will give you a variant of this same answer:

"Who are my users and where are my gaps against my use cases?," replied Fedora Action and Impact Coordinator Brian Exelbierd. "Specifically, I'd like to figure out how to increase the number of feedback-level contributors who I could then court for greater contribution. I'd also like to figure out how much of the 'strong feelings' that are heard in Fedora are actually felt by the non-vocal members of the community. I worry at times we are trapped by a loud voice who is not representative."

CentOS Community Liaison Rich Bowen agrees. "Who are all of the people that are using the project but are not connecting back with the community in any way? Finding these people, and being able to talk with them about their experiences, would hugely inform everything else that we do."

Seeking Answers

Tracking users in a given community is hard, but not impossible. Right now, the best tools for discovering information about your users include:

• User Surveys
• Registration
• Forum/mailing list analytics

You will not I did not mention more aggressive user tracking, like opt-in reporting from client applications. Historically, apps that "phone home" have been an historical non-starter in the FLOSS community, due to serious privacy concerns. Even the more passive methods can raise privacy issues.

Moving forward, we will investigate each of these methods and try to determine what process might work for your project.

Image courtesy of Edith Soto, under CC BY-SA 2.0 license.