This is Part 2 of Dr. Liz Kelley’s blog, “Evolution, Part 1: Distribution planning models for the DER age” where she discussed research using a typology taken from Donald Rumsfeld, that of the “known-knowns”, the “known-unknowns” and the “unknown-unknowns”. Different methods lend themselves to disparate situations — for instance, predictive modeling is great in a “known-unknown” context, but may not be so valuable when you are in an “unknown-unknown” situation. In this post, she discusses how, as an anthropologist, she researches the unknown-unknowns.
How do you measure exploratory research?
Picking up where I left off in my last post: How do you research the unknown-unknowns? My short answer: cast a wide net—you don’t yet know what’s relevant and what’s not, so it’s crucial to design your research with a broad scope. This approach doesn’t have to be cost prohibitive or take a long period of time, but it does require keeping the orientation of your research open enough to uncover new opportunities.
The ILLUME team recently completed a project conducting ethnographic interviews with customers focused on understanding how people think about energy in the context of their homes and households. In discussing the project, a colleague asked me “what are the metrics you’re using to determine success?” The question gave me pause. Not because there are no possible metrics (there are), nor because the project was not successful (it was), but because those metrics were not easily quantifiable at the outset.
We were not doing program-based or offerings-based research (we weren’t looking for the known-unknowns). Rather, we wanted to know more broadly how people thought about energy in the context of their homes and households. This project was exploratory, structured around open-ended, in-context research, with the goal of uncovering findings that hadn’t already emerged through other more targeted areas of research. The research sponsor was blinded, so that participants would not respond in the context of their feelings and experiences with the client organization or their programs.
“Our wider net allowed us to absorb information not captured in the answers to questions the program and implementation teams were raising…These additional insights can lead to happier, more satisfied customers because you can design programs and services that better meet people where they are, on their own terms”
What emerged from the research were not answers to specific and discrete questions, but rather new ways of framing and discussing challenges and motivations. Our wider net allowed us to absorb information not captured in the answers to questions the program and implementation teams were raising. If you ask people a set of questions framed in a specific way, you’ll get answers specific to those questions. Often, that is what you want (e.g., in a program evaluation context). In this context, however, we wanted to know what questions we weren’t asking, what topics we weren’t addressing – the unknown-unknowns. These additional insights can lead to happier, more satisfied customers because you can design programs and services that better meet people where they are, on their own terms.
In this project, one of our findings was that when we spoke with people about the lighting in their homes, they told us about how they chose lights to make their home feel a particular way. They told us how they teach their children to turn off the lights when they leave the room – this was important for reducing their bills, but just as important was passing on familial values to the next generation. Training your children what it means to take care of a home and a family begins with the simple act of turning off the lights.
Residential lighting programs often communicate with homeowners about lights – but those messages are often framed around the financial and bill-savings of LED lights or laden with unfamiliar technical terminology, from LED, to CFL, to lumens to kelvins, and so on. This framing centers ideas around new technologies, cost-savings, and energy efficiency, which are topics that may not be central to how people think about lighting without such prompts. Indeed, this is what we found when we showed people images of LED lights – their demeanor often changed, and they made comparisons to other new technologies or changes they had made.
By contrast, when we spoke with people about lighting in their homes outside the explicit context of a conversation about switching to LED bulbs, we learned much more about their households and family lives, about what matters to them. Even for individuals for whom bill savings were of tantamount importance, money wasn’t the only driver.
How to transform exploratory research into action
The challenge for the program and implementation team now is to use these insights to develop communication materials that speak to people in the language they use when they talk about their homes. In turn, program and service offerings can be tailored to the challenges, goals, and delights of specific customer groups. In the example I’ve discussed, the client team has incorporated our findings into its marketing and messaging platforms, changing the ways that they are communicating with customers about the value of LED lighting in ways that residents recognize, identify with, and value. How did they get from insight into action?
At ILLUME, we use different techniques to make sure that exploratory research ties back to program operations. Empathy exercises ignite conversation and help program implementers apply these “extra” insights.
What does this mean for DERs?
Given that lighting is hardly new territory, our research is an interesting example of how using more open-ended research methods can lead to unexpected research findings even in an area that has been well-investigated in the past.
In the context of DERs and transactional energy, there has been even less research on the customer or end-user side (at the grid edge) and little accumulated wisdom about what customers want, need, think, or feel. As EVs, rooftop PV, or other DERs gain traction in a broader segment of the market, the characteristics of early adopters may not usefully predict characteristics of more mainstream adoption.
In these cases, how do you know what to research and what to measure when you don’t know what you don’t know?
As I mentioned before, when you don’t yet know what’s relevant and what’s not, you can’t limit yourself to preconceived notions about what you should be investigating. At ILLUME, we adopt an open, curious stance that allows the unknown-unknowns to be revealed and makes enough room for more viable research options. It’s an approach that anthropologists are well-trained to carry out.
ILLUME’s Anthropologists see our work as identifying the unknown unknowns through open-ended and exploratory research. Ethnography, the method of anthropology, relies on a flexible orientation that demands both an openness to what data is relevant (and what constitutes ‘data’ in any given case), along with a willingness to adjust preconceptions and expectations when faced with contradictory or unexpected data. This stance creates an adaptive research process, where the initial findings shape later research questions. In fact, it is a mark of authenticity in anthropological fieldwork to return from the field with a new (or enhanced/tweaked/adjusted) set of research questions and objectives.
With DERs, you’re largely dealing with the unknown unknown. You don’t know who among non-participants might be potential participants, or what value they would get from a given product or service. Why would they want to sign up for or purchase this? What barriers might be preventing them from engaging with this product or service? And then there are even more unknowns – the questions we aren’t even asking, that we don’t know to ask. With all these unknown-unknowns, you have to begin with exploration.
