Measuring user experience success with building blocks

First impressions are important in the world of software. More than ever, users expect to be delighted by technology, and visual design plays a huge role in pushing user experiences from "meh" to magic.

Reactions to PatternFly 4’s new visual design language

PatternFly is an open source design system used by Red Hat to maintain visual consistency and usability across the product portfolio. When the PatternFly team started work on PatternFly 4, the next major version of the system, they focused a large part of their effort on evolving the visual language. But how would users respond to the new look and feel?

To get the raw and unfiltered feedback the team needed, Sara Chizari, a UXD user researcher, planned a reaction study with a fun twist and then headed to Red Hat Summit in San Francisco.

The study

In a typical reaction study, you provide participants with a list of cards that can be used to describe a product, like "compelling" or "frustrating." Then, you ask users to choose the cards that resonate with them most.

For the PatternFly study, Sara swapped cards for building blocks. Each color represented a different word.

Image 1: Building blocks

Sara presented Summit participants with a mockup of a product dashboard in the new visual language. Then, she asked them to pick up to five blocks that best represented their reaction to the design. Once participants had their blocks, they could either use them to build something fun or simply stack them up on the board we provided (see image 3).

Image 2: Mockup of a dashboard in the new visual language.

The results

It’s important to note here that, while each block color represented a specific word, participants weren’t able to see the color of any blocks until they made their selections. Also, the objects built by previous participants were randomly scattered across the board. These factors made it difficult for participants to get a sense of the choices others were making, and enabled us to control the bandwagon bias that usually comes with open group studies like this.

At the end of the study, Sara had 487 blocks from more than 100 participants. "I counted the bricks and sorted them by frequency… 80% of the blocks were positive words. This is a very significant result. We got very positive feedback," said Sara.

Image 3: Overview of results.

UXD relies on research and testing like this to make customer-driven design decisions that are based in real user feedback. In this study, the top ten words selected by participants to describe the visual system were all positive. The results of the study helped to reassure the visual design team that they were headed in the right direction and enabled them to move forward with confidence.

Image 4: Reaction words

The blocks?

But why use building blocks instead of simple cards for the study? Color coding. Positive words were represented by bright colors, and negative words by dark colors. As the brightly colored blocks took over the board, the team was able to get an at-a-glance idea of the study results. Also, blocks are just more fun. "At the end of the day, we took apart all of those blocks, and put bricks together by color. We built some buildings with it and called it PatternFly city."

Image 5: PatternFly City.

To learn more about the great work being done on PatternFly 4, join our community and keep up with the PatternFly blog and watch for more user experience posts here on the Red Hat Blog.