1. Please check and comment entries here.
Table of Contents

    Topic review

    External Human–Machine Interface of AVs

    Submitted by: Marc Wilbrink


    In future urban traffic, it is more likely that automated vehicles (AVs) will operate not in separated traffic spaces but in so-called mixed traffic environments where different types of traffic participants interact. Therefore, AVs must be able to communicate with other traffic participants, e.g., pedestrians as vulnerable road users (VRUs), to solve ambiguous traffic situations. Taking current traffic communication patterns into account, a combination of implicit communication via the driving behavior (e.g., deceleration, position in lane) and explicit communication via an external Human–Machine Interface (eHMI) seems to be a promising approach. The eHMI consists of an external interface connected to the vehicle, which can transmit explicit signals enabling interaction between AVs and other TPs.

    1. Background

    With the introduction of automated vehicles (AVs), today’s mobility will undergo fundamental changes. Nowadays, the human driver still executes the driving task. In future highly and fully AVs (SAE level 4 and 5), the human driver will be more or less decoupled from the vehicle control [1]. Along with this shift of control, the communication with other traffic participants (TPs), previously performed by the human driver, needs to be substituted by AVs [2][3]. Schieben et al. [4] describe today’s dyad of interaction between human drivers and other TPs (e.g., cyclists and pedestrians) shifting to a triad of interaction. Within this triad, the main actors are represented by the onboard user (i.e., the former driver), the AV, and other TPs in the driving environment of the AV. Regardless of the existence of a human driver, AVs need to communicate with other TPs in their surroundings in ambiguous traffic situations, enabling a cooperative interaction among all TPs [4].

    2. Interaction between AVs and Other Traffic Participants

    Initially, AVs will be introduced into mixed traffic environments. This means that AVs will coexist and interact with other TPs, e.g., conventional manually driven vehicles, cyclists, and pedestrians, in the same traffic environment. Therefore, AVs need to communicate with other TPs to interact safely [2][4][5]. Moreover, Stanciu et al. [6] state that adequate communication is highly relevant, especially for VRUs, e.g., pedestrians, to maintain the traffic flow. Generally, humans use different cues for communication and interaction in traffic, which can be categorized into the following categories [4][7]: implicit cues are signals that are not directly sent to a receiver or are sent out unattended by the sender. This means that the content of the message may not be evident at first glance. For example, the braking process is an essential implicit signal that illustrates the intention to prioritize a pedestrian [8]. However, explicit communication is directly sent out to a particular receiver. It can be transmitted via different modalities reflecting the vehicle’s status, e.g., hand gestures or the use of turn indicators [7]. Both means of communication, explicit and implicit, can give information about future intentions and maneuvers of the AV, and therefore they are of high relevance for the understanding and coordination of other TPs [4]. Studies by Dey and Terken [9] and Lee et al. [10] investigated the behavior of pedestrians when interacting with a conventional manually driven vehicle while crossing a street. The results show that the pedestrians used mainly implicit signals to decide if they could cross the street or not. Explicit communication between vehicle drivers and pedestrians was rarely used. The authors concluded that pedestrians tend to call upon additional explicit communication, especially when the vehicle does not behave as expected.

    However, Lundgren et al. [11] state that the communication requirements will differ between pedestrians and AVs. Current research often focuses only on the technical avoidance of collisions for surrounding traffic, e.g., unpredictable stopping maneuvers of an AV, rather than the interaction between AVs and other TPs. Besides the high relevance of collision avoidance to maintain a safe traffic flow, a lack of communication between AVs and other TPs can lead to frustration due to poor understanding or even more severe accidents [4][12]. According to the current number of traffic-related deaths and accidents, more than one-third can be referred to as the interaction between pedestrians and vehicles [10][13]. This number demonstrates the risk potential for (critical) interaction between pedestrians and vehicles. Having this in mind, an understandable and expectation-conforming interaction between the AV and other TPs becomes even more relevant to traffic safety. Therefore, AVs’ future intentions and behaviors need to be transmitted to the surrounding traffic to enable other TPs to derive their actions subsequently [2].

    3. eHMI as a Communication Tool

    AVs could use an eHMI as a communication tool addressing the requirements for smooth and safe interactions with other TPs. According to Schieben et al. [4], the communication content of the signals presented by an eHMI can be divided into four information categories. Firstly, information about the vehicle’s driving status allows other TPs to understand the current automation status of the AV, which is needed to ensure proper mode awareness. Secondly, information about future maneuvers enables TPs to anticipate the AV’s upcoming actions and plan theirs accordingly. The third main information category is the AV’s perception of its environment. This information allows other TPs to verify if the AV has perceived them. The fourth category includes the cooperation capabilities of the AV. This category contains information if an AV can communicate cooperatively.

    Focusing on the interaction between AVs and pedestrians, Bartels and Liers [14] emphasize that pedestrians’ behavioral patterns can vary significantly between and within individuals depending on the situation. Therefore, an eHMI should not only display the vehicle automation level but also transmit information about the AV’s perception and intention [15][16]. Although it seems that implicit signals have a significant impact on pedestrians’ awareness, explicit signals still need to be considered in the design of an eHMI to resolve ambiguity during low speed, e.g., [9][16][17]. Additionally, an AV should communicate through an eHMI, at least in situations where pedestrians are not satisfied with implicit signals from an AV.

    One major problem in the interpretation of existing eHMI research results is the large variety of different eHMI designs tested. Thus far, various eHMI designs have been developed, e.g., projections on the street, light bands, or displays [18][19][20]. Conclusions made in different research studies are always highly affected by the used eHMI design. It is sometimes difficult to form generalized statements regarding the pure effect of the eHMI itself [21]. In the present study, we distinguish between static eHMIs, which only display the current vehicle automation status (VAS), and dynamic eHMIs, which are capable of communicating an AV’s perception and intention additionally.

    4. Effects of eHMIs on the Interaction with Other Traffic Participants

    The use of an eHMI as a communication tool presents a promising approach to support a safe interaction between AVs and pedestrians, e.g., [2][4][22][23][24]. Besides the positive effects, possible negative effects, i.e., miscommunication, need to be considered as well [25][26]. Concerning a safe interaction and efficient traffic flow, it is of interest how the use of an eHMI influences the interaction between AVs and other TPs. Especially for vulnerable road users (VRUs), such as pedestrians and cyclists, an understandable and safe interaction would be essential for traffic safety.

    Generally, when interacting with AVs, many pedestrians see the need for an AV to be equipped with an eHMI as a form of communication [15][16][24]. Habibovic et al. [2] argue that an eHMI is particularly beneficial in situations where negotiation is required, e.g., who moves first. Moreover, the use of an eHMI could improve safety, acceptance, and traffic flow even before it is actually necessary [4]. Previous findings support this assumption by showing a higher sense of safety for pedestrians when interacting with an AV equipped with an eHMI [15][25]. Furthermore, De Clercq et al. [15] state that a safe street-crossing can influence the overall acceptance of the eHMI.

    Focusing on pedestrians’ crossing behaviors in interaction with AVs, the latest research shows controversial findings. Whereas Clamann et al. [16] found no significant differences for the crossing decision when interacting with a dynamic eHMI as opposed to a static eHMI, De Clercq et al. [15] describe that the earlier the dynamic eHMI indicates a deceleration of the AV, the sooner participants feel safe to cross the street. The authors conclude that an eHMI can increase the efficiency of the crossing process.

    Another important factor regarding safe street-crossing for pedestrians is an adequate strategy for gaze checks. Especially in traffic scenarios with multiple TPs, VRUs need to ensure that they have been perceived by all TPs and that a street-crossing is safe. Mitman, Ragland and Zegeer [27] showed that pedestrians at unmarked crossings tend to check the traffic more often in both directions than at ordinary pedestrian crossings. This finding is supported by Knoblauch, Nitzburg and Seifert [28], who show that pedestrians tend to make more gaze checks while crossing the street as opposed to waiting at the curb. With regard to the effects of an eHMI, Kitazaki and Daimon [25] conducted a study in which they investigated the use of an eHMI on pedestrians’ street crossing behaviors by taking the frequency of gaze checks into account. The results indicate that participants who encounter AVs with a dynamic eHMI tend to check the oncoming traffic less than those with a static eHMI. The additional information of the dynamic eHMI regarding the perception and intention of the AV could create a feeling of overtrust regarding the future behavior of the AV and the behavior of other road users. This overtrust might lead to the impression that it is always safe to cross the street when an AV triggers an eHMI, assuming that other road users behave accordingly to the AV. As a result, reduced gaze checks toward the oncoming traffic could lead to critical situations and demonstrate that the presence of an eHMI can also have negative effects on a pedestrian’s street-crossing behavior.

    To sum up, eHMIs can be described as a possible solution showing benevolent effects for communication, hence the interaction between AVs and other TPs [4]. Nevertheless, negative effects, e.g., reduced gaze checks to the traffic environment, can occur and need to be considered to prevent accidents, especially in the interaction with VRUs [10]. A poor eHMI design can lead to overtrust and confusion if a message is misinterpreted or different designs are used to transfer the same message. These ambiguous situations can be potentially dangerous for a safe traffic flow [15]. Therefore, an area of growing research interest is the evaluation of possible negative effects of eHMI designs on VRUs [2][15][16][25][28], which should be performed in an early stage of an AV’s interaction design development.

    The entry is from 10.3390/su13158396


    1. SAE International. Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles; Standard J3016_202104; SAE International: Warrendale, PA, USA, 2021.
    2. Habibovic, A.; Andersson, J.; Lundgren, M.V.; Klingegård, M.; Englund, C.; Larsson, S. External Vehicle Interfaces for Communication with Other Road Users? In Road Vehicle Automation 5; Meyer, G., Beiker, S., Eds.; Springer International Publishing: Cham, Switzerland, 2019; pp. 91–102.
    3. Färber, B. Communication and Communication Problems Between Autonomous Vehicles and Human Drivers. In Autonomous Driving; Maurer, M., Gerdes, J.C., Lenz, B., Winner, H., Eds.; Springer: Berlin/Heidelberg, Germany, 2016; pp. 125–144. ISBN 978-3-662-48845-4.
    4. Schieben, A.; Wilbrink, M.; Kettwich, C.; Madigan, R.; Louw, T.; Merat, N. Designing the interaction of automated vehicles with other traffic participants: Design considerations based on human needs and expectations. Cogn. Tech. Work 2019, 21, 69–85.
    5. Stiller, C.; Burgard, W.; Deml, B.; Eckstein, L.; Flemisch, F. Kooperativ interagierende Automobile. Automatisierungstechnik 2018, 66, 81–99.
    6. Stanciu, S.C.; Eby, D.W.; Molnar, L.J.; St. Louis, R.M.; Zanier, N.; Kostyniuk, L.P. Pedestrians/bicyclists and autonomous vehicles: How will they communicate? Transp. Res. Rec. 2018, 2672, 58–66.
    7. Markkula, G.; Madigan, R.; Nathanael, D.; Portouli, E.; Lee, Y.M.; Dietrich, A.; Billington, J.; Schieben, A.; Merat, N. Defining interactions: A conceptual framework for understanding interactive behaviour in human and automated road traffic. Theor. Issues Ergon. Sci. 2020, 57, 1–24.
    8. Beggiato, M.; Witzlack, C.; Springer, S.; Krems, J. The Right Moment for Braking as Informal Communication Signal between Automated Vehicles and Pedestrians in Crossing Situations. In Advances in Human Aspects of Transportation; Stanton, N.A., Ed.; Springer International Publishing: Cham, Switzerland, 2018; pp. 1072–1081. ISBN 978-3-319-60441-1.
    9. Dey, D.; Terken, J. Pedestrian Interaction with Vehicles: Roles of Explicit and Implicit Communication. In Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. AutomotiveUI’ 17:ACM 9th International Conference on Automotive User Interfaces and Interactive Vehicular Application, Oldenburg, Germany, 24–27 September 2017; Boll, S., Pfleging, B., Donmez, B., Politis, I., Eds.; ACM: New York, NY, USA, 2017; pp. 109–113, ISBN 978-1-4503-5150-8.
    10. Lee, Y.M.; Madigan, R.; Giles, O.; Garach-Morcillo, L.; Markkula, G.; Fox, C.; Camara, F.; Rothmueller, M.; Vendelbo-Larsen, S.A.; Rasmussen, P.H.; et al. Road users rarely use explicit communication when interacting in today’s traffic: Implications for automated vehicles. Cogn. Tech. Work 2020, 5, 145.
    11. Lundgren, V.M.; Habibovic, A.; Andersson, J.; Lagström, T.; Nilsson, M.; Sirkka, A.; Fagerlönn, J.; Fredriksson, R.; Edgren, C.; Krupenia, S. Will there be new communication needs when introducing automated vehicles to the urban context? In Advances in Human Aspects of Transportation: Advances in Intelligent Systems and Computing; Stanton, N., Landry, S., Di Bucchianico, G., Vallicelli, A., Eds.; Springer: Cham, Switzerland, 2017; pp. 485–497.
    12. Brown, B.; Laurier, E. The trouble with autopilots: Assisted and autonomous driving on the social road. In Proceedings of the CHI Conference, Denver, CO, USA, 6–11 May 2017; pp. 416–429.
    13. World Health Organisation. Global Status Report on Road Safety 2018. Available online: https://www.who.int/violence_injury_prevention/road_safety_status/2018/en/ (accessed on 30 June 2021).
    14. Bartels, B.; Liers, H. Bewegungsverhalten Von Fußgängern Im Straßenverkehr-Teil 2; Forschungsvereinigung Automobiltechnik e.V.: Berlin, Germany, 2014.
    15. de Clercq, K.; Dietrich, A.; Nuñez Velasco, J.P.; de Winter, J.; Happee, R. External Human-Machine Interfaces on Automated Vehicles: Effects on Pedestrian Crossing Decisions. Hum. Factors 2019, 61, 1353–1370.
    16. Clamann, M.; Aubert, M.; Cummings, M.L. Evaluation of Vehicle-to-Pedestrian Communication Displays for Autonomous Vehicles. In Proceedings of the 96th Annual Research Board Meeting, Washington, DC, USA, 8–12 January 2017; pp. 6–12.
    17. Dey, D.; Matviienko, A.; Berger, M.; Pfleging, B.; Martens, M.; Terken, J. Communicating the intention of an automated vehicle to pedestrians: The contributions of eHMI and vehicle behavior. IT Inf. Technol. 2020, 63, 123–141.
    18. Kettwich, C.; Dodiya, J.; Wilbrink, M.; Schieben, A.M. Light-based communication of automated vehicles with other traffic participants—A usability study in a Virtual Reality environment. In Proceedings of the 13th International Symposium on Automotive Lightning; Khanh, T.Q., Ed.; Utzverlag GmbH: Munich, Germany, 2019; ISBN 978-3-8316-4817-7.
    19. Bazilinskyy, P.; Dodou, D.; de Winter, J. Survey on eHMI concepts: The effect of text, color, and perspective. Transp. Res. Part F Traffic Psychol. Behav. 2019, 67, 175–194.
    20. Bengler, K.; Rettenmaier, M.; Fritz, N.; Feierle, A. From HMI to HMIs: Towards an HMI Framework for Automated Driving. Information 2020, 11, 61.
    21. Dey, D.; Habibovic, A.; Löcken, A.; Wintersberger, P.; Pfleging, B.; Riener, A.; Martens, M.; Terken, J. Taming the eHMI jungle: A classification taxonomy to guide, compare, and assess the design principles of automated vehicles’ external human-machine interfaces. Transp. Res. Interdiscip. Perspect. 2020, 7, 100174.
    22. Böckle, M.-P.; Brenden, A.P.; Klingegård, M.; Habibovic, A.; Bout, M. SAV2P-Exploring the Impact of an Interface for Shared Automated Vehicles on Pedestrians’ Experience. In Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. AutomotiveUI’ 17:ACM 9th International Conference on Automotive User Interfaces and Interactive Vehicular Application Adjunct, Oldenburg, Germany, 24–27 September 2017; Boll, S., Pfleging, B., Donmez, B., Politis, I., Eds.; ACM: New York, NY, USA, 2017; pp. 136–140.
    23. Fridman, L.; Mehler, B.; Xia, L.; Yang, Y.; Facusse, L.Y.; Reimer, B. To Walk or Not to Walk: Crowdsourced Assessment of External Vehicle-to-Pedestrian Displays. 2017. Available online: http://arxiv.org/pdf/1707.02698v1 (accessed on 30 June 2021).
    24. Merat, N.; Louw, T.; Madigan, R.; Wilbrink, M.; Schieben, A. What externally presented information do VRUs require when interacting with fully Automated Road Transport Systems in shared space? Accid. Anal. Prev. 2018, 118, 244–252.
    25. Kitazaki, S.; Daimon, T. To Make Automated Vehicles Communicative and Sociable on Roads. In Proceedings of the ITS World Congress, Copenhagen, Denmark, 17–21 September 2018.
    26. Lee, J.; Daimon, T.; Kitazaki, S. Negative Effect of External Human-Machine Interfaces in Automated Vehicles on Pedestrian Crossing Behaviour: A Virtual Reality Experiment. In Proceedings of the 21st Congress of the International Ergonomics Association (IEA 2021); Black, N.L., Neumann, W.P., Noy, I., Eds.; Springer International Publishing: Cham, Switzerland, 2021; pp. 718–725. ISBN 978-3-030-74607-0.
    27. Mitman, M.F.; Ragland, D.R.; Zegeer, C.V. Marked-Crosswalk Dilemma. Transp. Res. Rec. 2008, 2073, 86–93.
    28. Knoblauch, R.L.; Nitzburg, M.; Seifert, R.F. Pedestrian Crosswalk Case Studies. Sacramento, California; Richmond, Virginia; Buffalo, New York; Stillwater, Minnesota (FHWA-RD-00-103); Federal Highway Administration: McLean, VA, USA, 2001.