Preparing for research, and presenting findings

Weeknotes #35: Jan 18–22, 2021

I recently had a thought-provoking conversation with a fellow researcher that I need to unpack a bit.

I have two projects on my plate at the moment, and in both cases they seem like “special projects” that aren’t directly tied to a program area committed to action. They’re both somewhat speculative; one to raise the question of whether we should commit resources to improvements, and the other is essentially a technical feasibility study.

Typically in designing a research study, we want to have a sense of the biggest questions we’re trying to answer. What are we trying to learn, and what will we do with the information? Are we committed to action?

Without a sense of who the stakeholders are, and where their interests lie, we run the risk of doing research for research’s sake, or trying to share information that doesn’t align with the needs or expectations of those we’re presenting to.

In the case of one of my projects, we have a lot of data as a result of our study. And different people may be interested in what we learned, but at different levels.

There are quantitative measures: did we hit pre-determined quantifiable goals? And there are qualitative learnings: how did people understand or undertake the tasks presented. And then there are possible recommendations or next steps: what can we do with what we observed?

We can try to be unbiased as researchers, and only gather and present ‘evidence’. But we can’t ignore that different parties have different needs when it comes to consuming information. A content designer may need a different level of detail than an executive, because they are making different decisions.

When we perform research, it should be to help people make decisions. So the trick is when we aren’t clear on who is making the decision, and what decision they’re trying to make.

In the case of the speculative technical feasibility project, I am concerned that we are trying to proactively anticipate questions internal stakeholders may have down the line. But we don’t actually know.

One approach is to try to anticipate ANYTHING THEY MAY WANT TO KNOW. This is my typical approach when I’m unclear or things.. but there are some serious problems with this approach.

• you may arrive at invalid conclusions because of the unknown scope or circumstances
• your incomplete research may serve as an excuse not to research again later (once the scope is actually known)
• your research may never, ever get looked at

So why do I do this? Well, obviously its not a great approach, but I think I tend to err on this side because of fear of the alternative: no research.

I really want us to conduct some research with users of assistive technology because leaning on documented accessibility guidelines doesn’t actually mean your interfaces are usable. Is anyone chomping at the bit asking for this research to get done? Not at the moment, at least not for this project, but I feel like it’s the right thing to do. So I’d rather do it in the hopes of raising awareness of potential issues, than say nothing at all.

But there is the nagging voice in the back of my head asking me whether doing the research is really ‘enough’, if there is no champion to whom I’d be sharing the results. Would the findings be ‘interesting’ but not readily applied?

The other approach is to ask. And keep asking. Why are we doing this? What will we do with what we learn? We live in a world of limited resources, so if we’re not prepared to do anything with what we uncover, should we be spending time uncovering things?

This makes complete sense to me, but I worry that we as researchers may be asking a lot of our stakeholders. I’ve been to more than one research planning session where we ask stakeholders what they plan to do with the findings, and they defer, because they’re not precisely sure what those findings may look like. This may mean we just need to spend time talking about the program goals, rather than the research goals, so we fit our research into helping them achieve their goals, rather than asking them to define our research goals.

I feel a bit like I’m stumbling in the dark, and this conversation helped me understand why. I’m doing work, but I am not quite sure how best to tell the story for impact. Sounds like I have some more clarity to seek!

Research activities have stakeholder inputs (research goals) and outputs (reports, presentations, reports). We may aspire to pure, unsullen research gold within our sphere, but we may need to compromise, or at least modify what we present so that it’s valuable and useful to those for whom its intended.