EEG investigation of the spatial sound perception for the automotive industry

Abstract

Interior and exterior vehicle noise comfort have improved significantly in recent years thanks to the increasingly strict pass-by noise regulations and the emergence of quieter electric vehicles. Unfortunately, this noise reduction is preventing the delivery of an appropriate level of informative and emotional feedback to the humans inside and outside the vehicle. The need, therefore, rises for new artificial sounds to provide suitable acoustic feedback for different levels of human-vehicle interaction while maintaining a comfortable acoustic environment. Considering the increasing application of Advanced Driver-Assistance Systems (ADAS), the vehicle soundscape will ever more contribute to vehicle safety by creating continuous situational awareness for surrounding pedestrians and occupants. The design of such enhanced soundscapes requires a data-driven approach to balance the possible conflicts between subjective acoustic perception and objective criteria such as driver response time and quality. To go beyond of classical subjective evaluations, the aim of the present work is to leverage patterns in electroencephalography (EEG) measurements that correlate with aspects of human acoustic perception, exploiting them in the design and selection phases of vehicle sounds. In particular, the research focuses on horizontal sound localization and perception of radial sound motion. Initially, a portable multimodal acquisition infrastructure designed for the investigations of this thesis is presented and the results of its validation in a real-world driving experiment are discussed. The validation consists in the implementation of a Brain-Computer Interface (BCI) allowing to control the infotainment menu of the vehicle via the detection of Event-Related Potentials (ERPs) in the EEG signals, and thus without the need of any physical interventions while driving. It is shown that the infrastructure can be effectively used for the collection of physiological signals along with vehicle telemetry and that it guarantees high signal quality in laboratory as well as in more demanding driving conditions. Considering the BCI itself, a single Convolutional Neural Network (CNN), trained on as few as three EEG electrodes combining the data of ten participants achieved an average classification accuracy of 49% with five unseen test participants. This accuracy is three times greater than the random baseline, which corresponds to 16.6% since the menu was composed of six icons. Notably, the performance of the general model is just 6% lower than the average performance of the individualized models which is 54% and, moreover, it has withstood also totally unseen people. Then, the correlation between EEG signals and the localization of sound sources on the horizontal plane is investigated, initially in a pilot study with five participants, and then in a complete experiment involving thirty participants. The pilot study consisted in an active sound localization task and employed a short static wood-cracking sound played from eight directions. Based on the preliminary results and the refined research questions thereafter, the main experiment implemented a passive sound localization task with no attended directions and adopted a sound made up of a series of four hand claps simulating approaching or receding motion from four directions. Statistical analyses of the brain responses and single-trial classification accuracies achieved by the best machine learning model reveal a moderate correlation between EEG responses and the azimuth of a sound source. Average classification accuracies greater than the random baseline have been achieved in all configurations, with peak discriminability in the “longitudinal vs lateral” localization configuration (front and rear directions against left and right directions), independently from the motion direction. In particular, the average individualized classification accuracy obtained in this binary configuration is 62.7% whereas the leave-one-participant-out performance is 59.1%. Grand average EEG responses associated to this configuration present major differences in the N1 and P2 ERP peak amplitudes located in the frontal and occipital lobes, and frontal lobe alone respectively. Additionally, different amplitudes, latencies, and shapes associated with the N2 peak and other late components have been identified mainly in the parietal lobe. These results are in line with other studies which identified noticeable EEG patterns correlating with the eccentricity of the sound source with respect to the listener. Indeed, eccentricity greatly influences the availability of localization cues as, the more we move from the extreme case of longitudinal sounds to the other extreme of lateral sounds, the more we move from uniquely monoaural cues, which carry little localization information, to more expressive fully binaural cues. It is hypothesized that this macroscopic difference is the origin of the distinct cerebral mechanisms that produce the identified patterns in the EEG signals. A complementary habituation effect correlated with eccentricity has also been identified. It consists in the presence of statistically significant differences between subsequent sounds coming from the same direction and from different directions only in case of lateral sounds, and not in case of longitudinal ones. No late information buildup associated with the perception of the sound direction has been found in the experiment as statistically significant differences have been identified uniquely in the onset ERPs and not in the responses to the following impulses. In contrast, statistically significant amplitude differences in the N1 and P2 components, directly proportional to sound intensity, have been found comparing the grand average EEG responses associated with approaching and receding sounds. As expected, the statistically significant windows of interest correspond to the first and last impulse, i.e., where the difference of intensity is maximum, thus suggesting a relationship with the sound pressure level (SPL) rather than the perception of motion direction. To summarize, the first preparatory result presented in this work demonstrates the reliability of the proposed infrastructure for cognitive assessment of secondary tasks while driving. The purpose is to use it in the future to make objective choices in the design and evaluation phase of vehicle sounds that need to address the tradeoff between subjective perception and quality of the resulting behavioral outcomes. A spatial warning sound is an example of such scenario as it needs to successfully deliver as much information as possible in the shortest time possible, while avoiding to overload or confuse the recipient inside or outside the vehicle which would result in erroneous behavioral outcomes. In this regard, the infrastructure has been used throughout the doctoral work to investigate the EEG responses associated with the perception of two aspects of horizontal sound localization: angle of the sound source and motion direction. Statistically significant population-wide results have been achieved in discriminating the angle of the sound source from single-trial EEG responses, especially when comparing longitudinal and lateral sounds or sounds coming from left and right, while no meaningful EEG patterns correlating with sound motion perception have been identified.