Benchmarking

The benchmarking done by the anthropologists in the project focused on the existing literature on their topics. They spent longer time on the benchmarking efforts compared to the other two approaches, searching for and reading literature. The benchmarks were used to help them narrow down their focus and decide on what could be interesting to focus on in in their initial interviews. Furthermore the analysis of the collected material was aided by including insights from other anthropologists’ studies of similar topics.

The interaction designer team which did benchmark focused on how other advent fairs and Christmas markets have solved the issues they hypothesized would be interesting, such as way finding and event calendars. The benchmarking was used as inspiration during the ideation, but did not (explicitly) affect their own user research.

There is no built in benchmarking in the Mobile Ethnography app that was used, but the participants did their own benchmarking comparing with their expectations and previous experiences. A reoccurring example was that many users complained about the weather, as the winter weather during the market was the worst conceivable; windy nor any snow, whereas it normally would be snow and no wind worth mentioning.

Scope of study

The anthropologists started their projects by narrowing down their overall focus to specific research questions. Once the scope of the study was set it was not challenged again. The strong delimitation of the subject area however gives the opportunity to explore it in detail and have a deep research approach.

The interaction designers started by formulating a hypothesis on what could be interesting, which were used as starting points. The path set by the hypothesis could be left at any time if the emerging insights pointed to better design opportunities somewhere else. Thus new hypothesizes could emerge at any time. This means that the scope of the interaction designers is open. The openness however comes at the price of depth – the interaction designers’ study does not consider the phenomena studied at the same level of detail as the anthropologists.

The overall scope of the Mobile Ethnography study is set by the organizers of the study, and is manifested in how they brief the participants. However, there is no way of making certain that all participants stay true to the intended scope, they may misinterpret instructions or (unconsciously) delimit the scope further.

What is made the focus of the study?

Whereas the scope of the study relates to width and depth of the study, the focus of the study relates to which aspects are studied within the scope. Focus of the study closely corresponds to the ‘Questions’ heading in Tunstall (2008).

The anthropologists focused on behaviors and motivations, and how what is observed relates to the existing body of knowledge. Being able to describe a small section of human behavior is the goal of the study. In the long run this is a part of what can be called “the anthropological task” – to describe humanity through detailed studies of all aspects of human behavior (see van Veggel, 2005, quote above).

In contrast, the interaction designers focus on findings situations which currently do not live up to expectations (or which can meet the unmet needs of the users). As a part of this the constraints on potential design solutions are also of interest. The goal of the user research is to find design opportunities within the current situation.

In Mobile Ethnography which aspects get studied are once again in the hands of the study participants. The things which happen to happen, where the participants are, and their experience of events, become the focus of each individual’s participation (and data record). The overall focus for this approach thus emerges from the events which are most frequently chosen by the participant group as a whole.

Where and how are data collected?

The approaches on how to collect the data was similar between the anthropologists and interaction designers. Both used non-contextual interviews and contextual observations as their basic methodology. The difference between the two was mainly in the details; the anthropologists used a truly semi-structured approach to the interviews whereas the interaction designers had a structured approach with the possibility to add non-prepared follow-up questions. During the contextual observations the anthropologists did participatory observation whereas the interaction designers choose passive observations. Both did however observe continuously, getting a full view of their participants’ fair visits. One difference, however, was that one of the Interaction Design teams chose to use surveys.

The Mobile Ethnography approach stands in contrast to the other two; it was only done in context and at the participants’ discretion. As the participants added touchpoints only as they saw fit, the data gives ‘snapshots’ of the visit rather than a continuous description.

How is the analysis performed?

The analysis done by the anthropologists was thematically based, taking advantage of the different types of data collected and done without the help of any software. The differences between the various types of data collected are used to build a stronger argument for the conclusions. When the preliminary analysis was finished, the informants were approached again to fill in knowledge gaps.

The interaction designers seemed to view the analysis as a tool to support the synthesis. Both teams searched for patterns in their material through manual sorting and resorting, but mainly used the more tangible interview and survey data, neglecting their observations.

The Mobile Ethnography app mSF and its back-end ServiceFollow provides the project owner with the raw material submitted by the participants, thus making it the project owner’s responsibility to conduct the analysis (according to whatever methodology they choose to use). There is functionality built into ServiceFollow to ‘help’ the project owner with the analysis, but usage is optional.

How is the synthesis performed?

Synthesizing the material is not part of the anthropologists’ usual process; any synthesis which occurs is a by-product of the analysis.

On the other hand, the synthesis is the main focus for the interaction designers. They grouped and ordered their insights, aiding them in creating tangible evidences of their user research, manifested through the two teams’ creation of personas/user profiles. These tangible summaries of the user research are later used as both inspiration for the design work and as a validator of the suggested design ideas.

As with the analysis, myServiceFellow and ServiceFollow do not provides the project owner with a synthesis but offers tools to aid them in performing the synthesis.

What is more important – analysis or synthesis?

As the discussions on analysis and synthesis point towards, there is a difference in what is seen as the most important in the translation of raw data to insights (and reports); the anthropologists put their emphasis on the analysis whereas the interaction designers focus on the synthesis. ServiceFollow is neutral in this regard; having functionality to support both analysis and synthesis, but leaving the decision on how to do it up to the project owner.