The Study

The study approaches these issues by starting simple. Observing an urban site with a view to understanding how people interact with information is problematic; this was the stimulation to regard it as an interface. Cities are certainly crucibles of shared interaction and interminably subject to diverse interpretations from their inhabitants, we sought to peer into this process. As Coyne intimates at our core we are interpreting (hermeneutical) beings. Our whole world is imbued with this imperative to interpret (Coyne 2005). This insight gave license to engage ordinary people in the interpretive work usually ascribed to ethnographers and to entrust this interpretive expertise to the public as a user group. This is the self same capacity that allows such seemingly effortless interactions with common features of a city where people live e.g. traffic lights, signs, paths. The tool was designed to activate this invaluable faculty that we came to characterize as innate interpretive expertise. The interaction logic of ethnographic tools needs to be designed with inclusion as its first principle. This meant producing a tool that had extremely low barriers to access and effectively agnostic to the ability to use or access technology.

We developed an open evaluation tool system based on the following twofold logic; (i) color (ii) positive and negative value. The underlying assumption was that two orders of complexity remain generally interpretable but allow for enough complexity to generate rich responses. We chose the traffic light color value system because it binds onto a universally recognized visual code, is inherently connected to motion and mobility in urban areas and because it was easier to communicate the following 6 values to our research participants:

Figure 1: Heuristic tagging tool

The binary value system (+ or ?) was chosen because it is universally representative of positive and negative values. It is coherent with affirmation and negation in decision-making and again, the widely held connotation of positive and negative allows for simple choices to be made quickly. For example, a Red ? (stopped negatively) would indicate feeling obstructed whereas a Red+ (stopped positively) would mean that someone made a conscious decision to stop.

Each evaluator was briefed loosely to connect 6 key locations in the city surrounding a central starting point. We asked participants to identify the ‘best’ routes between these locations, giving as little direction as possible. We standardized an introduction to using the tool and produced two versions; one reliant on using a GPS enabled smart phone and camera, the other using physical materials. We offered a free choice for which to select and designed the capture tool such that both data sets could be entered into the same space for analysis. We provided each participant with a set of six tags to represent each of possible choices.

The tags were deliberately developed to be reminiscent of place-markers used in digital cartography. This was a deliberate strategy that itself resulted in a key insight to pre-sensitize users who feel less comfortable with technology, whilst drawing upon what they do intuitively. In this sense, enabling participants to harness their expertise in interacting with common urban situations. The emplacement of artificial signifiers in the physical world and vice versa points to an approach to designing situated HCI with inclusivity as a key organizing concept. This became a conceptual maneuver to nudge interaction behavior positively (Thaler and Sunstein 2012). Overlaying iconography from digital environments in physical contexts is a phenomenon that is already underway and itself became a principle insight from the study. The tool was also designed to encourage our participants to employing their evaluative faculties whilst making judgments about what they observed in situ. The tool was designed to elicit both analytic and synthetic insights from participants – a walk aloud method equivalent to think aloud protocols (Van Someren, Barnard, and Sandberg 1994) commonly used in evaluating the usability of digital systems.

The study took the form of a research through design process in three iterative phases (Gaver 2012). We located our study in Lancaster, UK; the university town of Lancaster in the North of England. Lancaster has a vibrant merchant culture and an active city council that seemingly is determined but uncertain of how best to implement and capture the value of digital strategy for infrastructure and services. The University itself is engaged with the city with a growing number of collaborative research projects and also provides the city with a diverse international student population. Lancaster has aspirations to enrich its heritage with sustainable innovation.

The research began with a series of semi-structured ethnographic observations of the site, taking the form of static observations and free walking observation by the research team. This was followed by a series of workshops to gauge requirements and frame a problem space with multiple stakeholders and members of the public. Way-finding was deemed to be a key problem in the site, Lancaster was thought of as holding rich heritage and social capital assets but felt disconnected both socially and spatially.

Throughout we used a card sorting method borrowing from (Moore and Benbasat 1991) and (Nielsen 1995a) to develop a method to thematize insights for feeding back into a design research process. Although not statistically rigorous, this method allowed the researchers to derive conceptual constructs from our observations and we brought some internal validity to the data by spreading the analysis amongst our research team and calling upon a wide variety of interpretations around a common problem from broad user groups.

A pilot study with 7 pair teams engaged in tagging the city, researchers passively observed some of these studies and short debrief interviews were conducted after each session. This allowed the honing of the tool design through user feedback. The first participants were all tracked using a commercial GPS application, to trace their movements. This allowed the development of a study strategy and captured 500 data points and associated semantic data of incidents of attention. Finally, a public research intervention trialed the tool with the general public; 35 pair teams comprised of members of the public, resident and nonresident. Their raw data, necessarily, was more uneven, however the qualitative responses formed a rich perceptual image. The sessions lasted around 90 minutes on average, although a timeframe was deliberately not stipulated. The task instruction urged participants to build the activity into their planned movements and was designed to be similarly replicable for each team.

Participants were briefed to attend to moments that drew their attention and assign one of the six tags provided, marking the time, location and direction. Participants were asked to evaluate which of the tags pertained to the incident. They were instructed to use the tag to point out the issue, using the tag held at arms length and take a point of view picture. The tag was thus in the field of the photograph, giving context. Using an open tagging system allowed for a quick way to capture perceptions and allowed participants to evaluate which incidents were worthy of mention and to ascribe their own meaning to each.

Each photograph was GPS tagged and time stamped. For participants who preferred to use a paper map we asked them to mark the appropriate symbol by hand. We encouraged both groups to make extensive notes and drawings to supplement each tagged incident. Each team tagged on average 60 incidents resulting in circa 2000 separate tagged incidents. This data was assembled into a database for pattern spotting and the data was loaded into a digital model of the city showing location data and user choices. This allowed for a rapid unraveling of the perceptual usability issues in urban contexts, the tool mediated a way to capture shared perceptual images (Lynch 1960b) of research participants of the city in parallel and then assemble these into a shared artifact. Deploying this data into a digital environment was extremely useful for further interpretive work and pattern spotting, drawing on the pattern language work of Alexander, Ishikawa, and Silverstein (1977)

Nielson’s heuristic evaluation matrices show the success of evaluators in finding usability problems. This heuristic evaluation principle relies on parallel evaluations of user environments to collectively identify a broader gamut of usability problems than experts can (Nielsen 1995b).

Discussion of Insights

The outcomes of this study are useful for both urban planners and system engineers but also to a third class of activity that blends the two practices. The ambiguous boundary between information and space afforded by technological mediation suggest a growing sense that place is experienced as a blended space (Turner 1998) where physical and symbolic features are reciprocally projected onto one another. Fixed systems are unlikely to remain useful in public contexts, as users will have flexible interpretations of the worth of a system and how to use it.

It may also be of interest for researchers interested in innovative research methodologies. Although the study was quite specific, the underpinning aim was more general. The tool itself was not designed as a research method (although it proved useful as one).

It was a good fit to repurpose usability heuristics because the underlying premise of heuristics is that with an increase in evaluators engaged in using a system there would be a commensurate increase in the proportion of usability problems that can be identified. All that was required was to frame the type of problems to be identified and then facilitate the capture of incidents flagged up by users whilst interacting with the system. In a traditional HCI context, the user can be observed or the interactions recorded. In the context of a city, a tool was needed that would facilitate this. Determining that this tool should itself be assembled from common heuristics and call upon heuristic capacity, in the true sense, meaning designating or relating to decision-making that is performed through intuition or common sense, was a vital design strategy. Heuristics can be thought of as methods that place participants as far as possible in the mindset of the discoverer (OED). In psychology, heuristics pertains to simple, efficient rules, learned by experience that have been proposed to explain how people make decisions, come to judgments, and solve problems typically when facing complex problems or incomplete information (Sternberg 2011). Heuristics tap into innate cognitive ability; they explain our ability to deal effectively with contingent events, by using situated insight to inform onward progress. This could be characterized as an abductive approach (Peirce 1901). As such, abductive reasoning lends daily decision making the kind of deftness to do the best with the information at hand, which often is incomplete. This resembles the mode of thought a trained ethnographer assumes to understand a context. The professional relationship between design ethnography and user experience design ensured this connection would be commensurate and mutually intelligible.

Taking an abductive approach to decision-making allows for rapid flexible thinking and recognizes the value of intuition (or nous) to work with effectively in contingent situations where no optimal situation exists. Herbert Simon identifies that engagement with complex research contexts requires satisificing as in these situations, single optimum solution exists (Simon 1996). As this research engages with a lived context, it must proceed via satisficing, proceeding by finding the best available interim step towards a goal. Essentially, establishing a continuum of informed guesses, this seems to harken the mode of thinking people rely on whilst navigating an environment. Generally, people are innately very good at this. The resultant strategy meant borrowing from this insight to ensure the methods were fit for purpose in unstable contexts, but also calling upon this faculty in our participants to unravel usability problems in the site. This meant that our epistemological stance was a good fit for what we observe in the field.

The core principles of heuristic evaluation were found to be good match for identifying usability problems in an urban context. This would seemingly work equally well in sites that have little or no digital infrastructure and sites with extensive Smart infrastructure. Having said this, we advise that ethnographers working in Urban HCI contexts begin this work in earnest to determine what smart layers could bring to the civics of a city, rather than refigure a system after the fact. In this way, ethnographers have an enormous part to play in a global Smart City market projected to have cumulative value $1.565 trillion USD by 2020 (Vidyasekar 2013).

Harrison & Dourish give weight to the reasoning that reinforcing the loop between way-finding in physical environments with equivalent digital systems should be indispensible: “We live in a three-dimensional world, the structure of the space around us shapes and guides our actions and interactions. With years of experience, we are all highly skilled at structuring and interpreting space for our individual or interactive purposes” (Harrison and Dourish 1996). By acknowledging this tacit spatial and interpretive expertise that humans bring, developed through the course of their daily lives, our ability to produce meaningful systems is greatly amplified, a view strongly mirrored in the work of (Suchman 1987) and (Orlikowski 2009). This is the same machinery of interaction spoken about by (Sacks 1984) and later (Crabtree, Rouncefield, and Tolmie 2012).

Exclusion will always result from a user’s inability to find utility in a service; this seems to result from a failure to make meaning from an interaction. In lieu of finding validity in an interaction system a person will resort to orientating themselves with reference to physical features, this strong instinct to return to embodied experience is evidence of an innate spatial expertise. We see this as a source of insight for the design of emerging systems. Our tool was ridiculously simple but enormously productive as a mode of thinking through the site as ethnographers, furthermore, it facilitated the participant group to reflect on the usability of the site. As such simple interventions like this can act as enormous resources for learning quickly about a city. Drawing on Lynch’s methodology, our data acted to form an inter-subjective perceptual image of the city. We present the product of our data analysis and what this leads us to understand about design for inclusion in Urban HCI.

We recognize this as a resource for intelligent design of situated HCI. Way-finding was deemed to be a key problem in the site. A shared perceptual image emerged, representing Lancaster as rich in environmental, social and economic assets but a sense of dislocation. An overarching political climate of funding cuts to public services, a national scale transition towards self-organizing public services to replace legacy government funded ones feeds into the megatrend of transitioning traditional built space into spaces of situated information. There is little digital infrastructure presently in Lancaster, but a sense that the existing information environment was not able to adapt or signify the life of the city. Insight about life in the city is held by inhabitants and exchanged somewhat tacitly (Polanyi 1966). Accessing knowledge that lies in networks of people, using co-creative strategies to harness value is an increasingly important role for design ethnographers (Ramaswamy and Ozcan 2014).

We found that for each physical characteristic, there was a counterpart that would tell us something about the role information played in informing decision-making. What we observe is a growing evidence for linguistic and interactional structures born of e-culture finding their way into public space. We see great desire in our participants to feel oriented and connected to the history and social dimension of the city. The engagement pattern in our study was asymmetric, just as engagement in digital infrastructure would likely be. The study gave us a keen sense of how the city is used as an interface and delivered rich insight into quickly deploying effective diagnostic tools in neighborhoods, streets and city centers.

We found a skew in our participation rate towards families with young children, visitors to the city and persons with factors limiting their mobility. The common thread between engaging groups was a sense of disorientation, difficulty in connecting social and spatial experiences and a vested interest raising conviviality through place-making. This is likely because the study was phrased as an activity, a way of engaging with the city and finding more about it, more importantly, these groups unanimously expressed a sense that the city was disconnected and badly provisioned with orienting information and services.

There also seemed to be division amongst social groups in the city, a rupture between different social worlds using the city in parallel but without interaction. This was typified by a group of disenfranchised local people who did not engage with the study, but watched intently throughout. They used the central space outside the town hall to conduct their social interactions with a sense of ownership of the space, ignoring and being ignored with some anxiety by the wider populous.

Engagement – Our initial, more conventional attempts at ethnographic observations and interacting with people proved to be extremely difficult and did not seem to permit ‘participation’ of any kind. In response to this however, a research through design process was useful to rapidly develop tools to engage participants resulting in a more ‘emic’ view, akin to the perceptual images sought after by Lynch. The tool and the activity together in some ways performed as a shared boundary object (Star and Griesemer 1989). The tool was open enough to sustain a degree of interpretive flexibility, becoming a way to express different meanings and agendas through a common framework. The activity in itself stimulated independent dialogue around the theme and begin to raise the issue into public awareness. The assemblage of the people, the tool and the activity itself in its various different configurations allows us to frame interaction within a wider activity system (Engeström, Kajamaa, and Lahtinen 2015) that could be usefully deployed in cities, the challenge remains to develop shared experiences that are more than fleeting moments and become and integral part of the shaping of urban interaction. As such, ongoing, shared ways to contribute to civic planning and contextualizing appropriate Urban HCI would mean establishing and sustaining boundary objects in use (Barrett and Oborn 2010).