Stakeholder Management: Balancing Data Quality, Stakeholder Satisfaction, and Client Education

In discussions throughout the Alignment phase of the project, the EPAM Continuum and NMB teams had discussed extensively the “ideal” number of participants to bring along to an interview. While EPAM Continuum was accustomed to bringing a client along to a given respondent interview for the helpful empathy, context, and skills it could build, the team also sought to limit the number of interview attendees to make respondents more comfortable by having fewer strangers in their home.

For this engagement, a typical interview would have a separate note taker and lead interviewer from EPAM Continuum, a translator/fixer, and an employee of NMB for a total of four people attending each interview. In addition, the research team also needed to account for additional guests for certain interviews—an employee of the Central Bank of Jordan interested in learning about human-centered design, and an employee of USAID who was both interested in human-centered design and also wanted to observe interviews to be able to report back that the EPAM Continuum team had in fact delivered upon their promised process.

This made for challenges to the EPAM Continuum team’s usual process, as they struggled to balance accommodating multiple stakeholders’ representatives that they wanted to attend the interview with the need for intimacy, due to the sensitivity of the conversations that the teams would be having with NMB clients about their finances. (Taylor, 2013) From their past work, the EPAM Continuum team was distinctly aware of the potential decrease in candidness (and therefore quality of ethnographic data) that came with having more strangers in the interview environment. To get around this, the team decided to “over-recruit” and run additional in-context ethnographic interviews around the capital, Amman, (where it would be most convenient for the representatives from both USAID and the Central Bank of Jordan to attend) and be prepared to account for those interviews not yielding the same quality of data as interviews where it was only the core NMB and EPAM Continuum team members in attendance.

Recruiting and Employee Interviews

To recruit respondents for the project’s first round of generative interviews, the NMB/EPAM Continuum team worked directly with bank branch managers in the different cities and towns they planned upon visiting for research, chosen for locations with NMB branches, and to give the research team a diverse sample of different population densities (urban and rural) and geographic regions. Although the team was aware of the potential trade-offs that would come with the respondent knowing that NMB was the bank about which the team was interested in learning, the team decided that it would be too difficult for the research team’s fixer/translator, Shereen Zoumot, to reach out to a respondent independently of NMB and build the trust necessary to get invited into their homes.

To limit the bias that would be inherent if the local NMB branch employee that the respondent personally knew were in the same room during the interview, the team designed the protocol to include the moment where, after making the introduction between the research team members and the respondent, and reminding the respondent to share their honest opinions and answer the team’s questions candidly, the local NMB branch employee who arranged the interview would step out of the room. For the purposes of limiting bias, any additional NMB stakeholders from NMB’s headquarters observing the interview would not be identified as such, instead being identified either as part of EPAM Continuum’s team, or “assisting” the team in some way. The research team felt this method still allowed the respondent to share their candid thoughts about their microloan, their personal finances, and any weaknesses they saw in NMB’s current microloan experience, while also making them feel comfortable enough to speak with the research team after having been introduced by a trusted person in their social network.

Finally, to understand microloans as comprehensively as possible, the team interviewed local branch employees (typically the manager in charge of selecting and recruiting customers for the team) at each of the NMB branches that helped the researchers recruit from their customer base. Through conversations with branch employees across the country, the research team understood how the formal mechanisms of credit (background checks into whether an applicant has had past loans, whether they are currently in debt, etc.) are supplemented by less formal “secondary sources,” such as when branch employees ask multiple NMB customers with strong ties to the bank and who live in the same neighborhood as the microloan applicant various cross-referencing questions (whether they know the applicant, whether the applicant is in good standing within the community, whether they’re a gambler or owe others money, or if the applicant has a background of saving through participation in community savings clubs).

Defining the Social Protocol of an In-home Ethnographic Interview

Despite EPAM Continuum and NMB’s efforts to minimize the number of attendees in an interview, oftentimes the surprise would come from the respondents themselves, for whom there was no prior social protocol for an in-home or at-work interview. From previous work, ethnographic interview respondents often did not know how to prepare to accept a group of four or five strangers into their home, and so it often became a sort of “hosting” experience, with respondents offering drinks or snacks, and sometimes invitations to stay for a meal following an interview. Particularly for interviews in smaller urban and rural settings outside of Amman, the NMB/EPAM Continuum interview often became a “social” event involving the respondent’s fellow employees, friends, relatives, neighbors, and others. In the first such encounter, the team walked into what they were expecting to be an interview with a 40-year-old female microloan customer in the smaller northern city of Irbid, a place with a population of around 500,000 near the border with Syria. After being greeted by her husband, the team was shown into the family room, where the team was joined by the respondent, her husband, a neighbor, and, at various times, the couple’s four children, ranging in age from six- to twelve years old. Sitting in the car afterwards, Zach Hyman, a Design Strategist on the EPAM Continuum team, raised the question, “Do we think this is a problem? Having all those other people in the room [besides the respondent] during the interview?” To which NMB’s Cabezas countered, “There wasn’t much of an option in that case. We are guests in their home, after all—it’d be too rude to ask a visitor to leave.” Zoumot, the team’s fixer/translator agreed, saying: “People here don’t know what to do with this kind of an interview, so they treat it almost like a party, and imagine how you would feel if someone showed up to your party and asked to you to make your friends leave.” EPAM Continuum’s Bianchini said: “What if we have a plan in place for next time, if we feel that there are too many extra people and that the number of people is hurting the conversation quality?” The team agreed to develop a plan in case encountering a similar situation in the future, which the team ended up needing to enact several days later, in the village of Deir’Alla.

After being welcomed into the respondent’s home in Deir’Alla, that of a 48-year-old woman who ran a small convenience store adjacent to her home, the team sat down and began the interview as normal. About 30 minutes in, once word had traveled around the small neighborhood that there were visitors, a neighbor showed up bearing a silver platter brimming with steaming cups of tea ? one for each of the people in the room, plus herself ? and proceeded to join into the conversation, despite not having any prior experience with microloans or interactions with NMB. Hyman and Zoumot struggled to keep the conversation focused upon the firsthand experiences of the NMB customer as her neighbor shared various observations and anecdotes ? stories of the risks of selling things on credit to customers, or the challenges of trying to assess whether a wholesaler was taking advantage of her. Eventually the research team members made eye contact with one another, and set the previously agreed-upon plan into motion; as the lead interviewer (and ostensibly the one with control over the conversation), EPAM Continuum’s Hyman stood up and said, “Ah, I forgot something important in the car, can we take a short break so I can go out and get it?”

After Zoumot translated this, Hyman and NMB’s Cabezas walked out to the car, where Cabezas explained to Taha (the team’s NMB-appointed driver who was familiar with the roads all across Jordan, and who typically waited in the car while the team conducted interviews) that it would be a great help if he could come in and have a friendly, separate conversation with the talkative guest while Cabezas (who was a strong Arabic speaker) wrote down notes. While a slight deviation from protocol, this still managed to create a positive outcome for all parties; the guest would feel like she was having her opinion heard, and Cabezas would help advance the research by asking her questions about money and her financial life. The team’s “bonus” respondent also signed an NDA to make things feel as “official” as possible. An unforeseen positive outcome from Cabezas’ conversation with the bonus respondent was that he and Taha were able to direct and control the flow conversation with her; instead of leaving open the opportunity for her to add commentary as she liked to the research team’s conversation with the NMB client. This way, Cabezas could assemble a meaningful set of observations and takeaways through his structured conversation with the guest.

The team decided to use the spare audio recorder to capture the conversation, so that it would feel as if the separate conversation she was having with Cabezas was no less important than the conversation that the EPAM Continuum team was having with the “intended” respondent. Signing an NDA also meant that both her privacy and the integrity of the research team’s data and methods would be protected. Since the team only brought a limited amount of incentive packages along, the team was unable to give a separate incentive package to the bonus respondent.

Incentives

One element that defined the NMB/EPAM Continuum team’s ethnographic interviews and might have caused them to be interpreted as more of a “social” event was the team’s choice of interview incentive. After the EPAM Continuum team landed in Jordan and met with Zoumot, their fixer/translator, for the first time, they reviewed their intended approach to how interviews would ideally run. When Hyman asked Zoumot what she thought a reasonable amount to pay respondents would be for what would be around a two-hour conversation in their home, her response was “Oh, no — paying them cash would be considered very rude.” The team had encountered what Jan Chipchase covers in detail in The Field Study Handbook, when he begins by sharing how “sometimes non-monetary incentives are a better choice.” (Chipchase, 2017). The research team considered factors such as whether refrigeration would be available for all respondents (deciding to assume it would not be) and what sorts of products and brands would both be most desirable and would confer the most status upon the respondent as their recipient.

In assembling the box of incentives, Zoumot recommended a value of approximately $50 US equivalent as appropriate to award a respondent for their participation in the two-hour interview, and together Zoumot and the EPAM Continuum team wandered the aisles of a grocery store located nearby their neighborhood-based pop-up studio in the Jabal al-Weibdeh neighborhood of Amman. After much deliberation, the team decided to include in the box:

Figure 4. Two incentive packages for the day’s interviews. Image © EPAM Continuum, used with permission.

• almonds
• dried apricots
• three flavors of Lindt dark chocolate bars
• a jar of Nutella
• a jar of Honey
• a box of dried dates
• a bottle of organic olive oil
• a bottle of Vimto (a type of fruit cordial)

Figure 5. Contents of an incentive package. Image © EPAM Continuum, used with permission.

All of the items were non-perishable, did not require refrigeration, and conferred status through an appearance of elevated taste amongst the recipients. There were also items that would appeal to any children in the household.