BACKGROUND

Nissan, just like many automotive companies (OEMs), is developing autonomous vehicles (AV) and like many other OEMs has invested in a Silicon Valley-based research center where key aspects of the AV’s software systems are being developed. The particular focus for the lab is on autonomous driving for the city. From its establishment in 2013, the director of the research center, Maarten Sierhuis, maintained that a central challenge for autonomous vehicles would be effective interaction with other road users. The demands of city driving require this; urban contexts are interaction rich. A goal of Nissan’s AV development, therefore, is to ensure “socially acceptable” autonomous driving. To this end, he contracted Gitti Jordan, a world-renowned anthropologist, and later built a small group of social scientists to help work out what socially acceptable driving might mean in practice.

In Maarten’s original vision socially acceptable autonomous driving would be driving in which AVs, when interacting with other road users, operate smoothly and in a manner appropriate to the specific interactional context; AVs that behave neither too aggressively nor yield incessantly to other road users, neither impede the normal flow of traffic nor cause undue notice. In short, socially acceptable autonomous driving would mean that Nissan AVs would smoothly integrate into the flow of traffic and handle roadway interactions without disrupting other road users moving down the road. Determining what it would take for AVs to operate in this manner has been one of the key areas of focus for our group.

There could be other interpretations of socially acceptable autonomous driving, of course. One could aim to understand what consumers in different markets think they want from autonomous driving. Indeed surveys suggest the general public is rather divided about whether they truly want and trust AVs. Another interpretation of socially acceptability would be that autonomous driving would bring a wholesale shift to some socially desirable outcome, such as new mobility options for the disabled persons. This certainly is an often touted and highly anticipated benefit of AVs by both OEMs and representatives of disability communities. Our work has yet to focus directly on either of these dimensions. Instead it is positioned in upstream research, at the very inception of the core software architectures and programming necessary for an AV to move about autonomously, ahead of the stage of engineering in which the ideas conceptualized and tested in research are “hardened” or firmed up so that vehicles can go into production and marketing strategies can be formulated. Our work thus precedes the kind of research ethnographers may more familiarly be brought into associated with the developing new car models, business strategies, or marketing plans.

Here we tell the story about how our group has undertaken research to ground the notion of socially acceptable autonomous driving in empirical investigation with a social scientific lens. We thus add to the literature on the impact of social scientific research in high-tech industries (the EPIC archives are filled with instances of this, as are numerous other publications over the proceeding two decades), exploring both some of the successes and challenges we’ve had integrating research findings with technology development.

APPROACH

Excited about the possibility to impact the design of the AV and thus the mobility of the future, we embarked on our quest to help define socially acceptable autonomous driving. To have impact, we felt we needed to have a certain amount of independence in order to avoid being constrained by the starting assumptions of the engineering-driven effort. We could make a contribution to the notion of autonomous driving by doing ethnographically-informed research that we could analyze on our own terms, staying honest to our own discipline before attempting to adapt our findings to the terms deemed most relevant to the engineering team. Doing more than paying lip service to the idea that AVs must learn to behave in socially appropriate ways demands understanding what happens on the roads. And what happens on the road is undeniably social, as a broad mixture of people find their way to various destinations using a variety of transportation options, ultimately, and nearly unavoidably, by means of interacting with other road users to establish a self-organized order of traffic. And yet, whether regarding traffic as a particular form of public social life can be made useful and compelling to those actually building Nissan’s AV technologies remains a tremendous challenge, as we discuss below.

Given the broad remit for our work, we quickly initiated a program to collect a variety of data to study social practices of road use. We collected video data from stationary cameras at different city intersections that captured the interactions between road users. We supplemented this data with intercept interviews to gain participants’ sense of the nature of their interactions. We also captured first person perspective of road users, by conducting “travel-alongs”, interviewing participants on their own transportation experiences, traveling along with them as they drove or walked through local urban contexts, and inviting them to review the video recordings with us and reflect on their experiences. (This phase of research was ably advanced by Logan McLaughlin, a Master’s student at the University of North Texas and who joined us as in intern for the summer of 2015; McLaughlin 2016). We furthered our first person perspective, and diversified our focus through a brief immersive study in Sao Paolo, Brazil. We also collected data recorded by bicyclists riding through Sao Paulo and in Amsterdam.

Methods of interaction analysis have been key to our analytical practices. To date, video analysis has provided an anchor to our analysis, which has also included attention to how people talk about and reflect on their own road use practices. Beyond these more typical ethnographic modes of inquiry to develop a basis for our understanding, we have also designed some concepts and performed preliminary testing of those to advance the design and refine our empirical understanding, which we will describe below.

Naturally, we continue to advance our knowledge and thinking through review of the literature, from social and cultural histories of transportation and mobility, to walking and pedestrian life, to in-car behavior, to the development of autonomous systems and more. We have collaborated with others in aspects of this work, including members of the DesignLab at UC San Diego under the direction of Don Norman. And we have also enjoyed the opportunity to work with classes at both the University of North Texas (see Jordan and Wasson’s 2015 EPIC paper for a description of this work) and San Jose State University with Jan English-Lueck.

DUALING SCIENCES IN THE ENGINEERING OF AUTONOMOUS VEHICLES

What does it mean to bring an understanding of the profoundly social nature of driving and road use into the very foundations of technical development?

That driving is not only a technical but also a social skill is obvious from the moment one progresses from driving in a parking lot to driving on the public roads. Indeed for anyone to manage their movement through time and space, whether in a car, on a bike, or as a pedestrian, is a social act that involves the interpretation of cultural signs and signals, to interacting with others. Even aspects of mobility one might deem purely technical at first—accelerating, slowing down, keeping one’s balance on a bicycle—must be considered profoundly social skills on second viewing.

Take the problem of locating where the AV is. Despite advanced GPS capabilities, this remains a non-trivial technical challenge (see Brown et al. [CITE] for a demonstration that driving with a GPS is also a non-trivial cognitive task). GPS is needed to help determine the precise location of the vehicle, exactly where it is on the road in relation to lane markings and curbs, for instance, but it must also integrate such information with the maps in order to determine precisely where it is. This gets especially tricky when the GPS signal isn’t completely reliable (for instance, next to tall buildings) and the system needs to decide whether to trust the GPS location or the information from its more proximate sensors that track the markings on the road.

And aside from the technical challenges, there are the social considerations of a location. For a car driving down the street, for instance, an area where children are playing by the side of the road changes the sense of location for a driver significantly. We might expect that an AV take such social considerations into account when it drives down the road, yet that necessitates that the AV has a concept of what “playing children” are—that it is able to recognize not only children, but also their behavior as playing—and that it could adjust its driving style dynamically. (The same street without playing children does not require a similar level of caution).

Or take the rules of the road. Some may seem simple and could easily be implemented algorithmically. For instance, an AV can be designed to adhere strictly to the speed limit. Yet we all know that driving the speed limit can be too fast in some cases (the aforementioned street with playing children, e.g.), whereas in others the socially acceptable way of driving is to exceed the speed limit. Similarly, an AV can be programmed to never cross a double yellow line, yet drivers often break this rule to skirt around a turning vehicle (Picture 1) or around a bicyclist on a narrow road, and refraining from doing so could engender frustration for traffic behind the AV.

PICTURE 1. A car crosses a double yellow line to skirt around a car waiting to turn right. The car performed the same maneuver; it is a common practice to break the rules of the road to keep traffic flowing.

Moreover there are many rules of the road that explicitly refer to a driver’s judgment: just consider yield signs and four-way-stop intersections; you must yield when others are present but not when nobody is, and at a stop you must of course stop and go in the order of arrival, but what counts as one’s arrival when cars stop (or keep rolling) at different distances from their lines? As AVs are driven by means of a computer program, they don’t have the capability to use this required “judgment”. How to act appropriately must be pre-specified by its engineers who thus have the Herculean task of defining all possible situations.¹ This is something that is easily done in case of games like Chess or Go where all possible legal moves are well-defined, but nigh impossible when one deals with a real-world environment such as human behavior on public roads.

And this points to one of the key differences between social science research and AI research. While there are vast differences in approach among social scientists, our anthropological and ethnomethodological backgrounds hold in common that we regard human behavior as massively contingent on the situation, in some regards the very opposite of algorithmic. Even a relatively circumscribed form of human behavior, like movement in traffic, is dependent on a host of external circumstances. These include aspects of the physical environment—the road signs, the markings on the street—temporal factors—the time of day, the season, the weather—and social factors—perceptions and meanings of the environment and specific locations (a “neighborhood”), the presence of other road users including bicyclists and pedestrians, their ages and physical ability, the cause or reasons for people being on the road in the first place (for work? for pleasure?) and special events such as a parade or a farmers market. We also recognize that how people move about in traffic is but a small aspect of people’s life, that their presence on the road and movement from place to place is but a fleeting moment (and often a not a very notable one at that) in an on-going set of personal experiences. While such considerations of the contingencies of human behavior are a starting point for our research, for AV software engineers these considerations just aren’t very helpful, as they try to get on with the programming which is dependent on defining just what the AV should do in what situation.