Getting to know the user

An effective research study depends on a good plan, at the heart of which is deciding what your research questions are and defining the methodology to get the answers. With discovery research, the research questions are more open ended by design, and in this case we basically wanted to know:

• What were the biggest pain points and how much did they differ from the current hypothesis?

• What was the best way to craft the new system’s flows so they fit into the staff workflows?

• What, if anything, were we missing from our current understanding?

I did some advanced reconnaissance to understand enough detail to be able to skip basic questions and get to the meat of the matter as soon as possible in the interviews.

​

I quickly realized I faced a complex picture, which meant there would be a challenging amount of material to get through in the alotted time, even if I had 45 minutes or so with each participant.

 

• Each department was unique in its operations, with different staffing levels, staff specialties, funding levels, fan base, and so forth

• Colleges offered different selections of sports, as few as one or two and as many as a dozen or more, each with their own — and vastly different — scoring systems

• There was a large range of approaches to scoring and streaming games, depending on funding and local preferences 

• The ecosystem of people involved was layered. Not only was the purchasing decision maker different from the users, but the users were also providing a service to another set of “users,” visitors to the department website

• Departments were members of regions and conferences that had their own information ecosystems, who in turn laddered up to the national level. 

 

This suggested that recruiting for interviews would be a puzzle, getting as much representation as possible to be successful in understanding the detail and making recommendations. We ultimately decided that about two dozen interviews would be sufficient to cover things. We segmented the types of departments by size, school size, sports offered, and the like, and had our participant list. 

 

I also had to decide how to approach the interviews. Even at a 45 minute length, I would be pressed for time to get through all the topics I needed to: a basic understanding of the department’s operations and staff roles, where the participants fit into it, what their particular pain points and goals were, and deep dives into specific workflows.

 

My view is that for most qualitative research, the focus is more on the spread of situations and ideas, so an exhaustively representative sample set is less important that getting most of the range. With even a handful of interviews, it is possible in many cases to establish a basis for understanding what is “mainstream” and what is outside of that. With the two dozen interviews, I could do that and come away with an understanding of a landscape of this complexity.

 

On top of that, I also believe that after a few interviews, if I’m hearing a lot of the same answers, I can shift the topics in future interviews to cover more ground. In this manner I could get all the information I needed within the limitations of the interview parameters.

​

​Interviews were scheduled, and over three weeks I conducted over two dozen 45-60 minute conversations with people from all over the country.