PROJECT TITLE :

Learning Multimodal Representations for Drowsiness Detection

ABSTRACT:

The detection of drowsiness is an essential step toward ensuring safe driving. A significant amount of work has been put into developing an automatic drowsiness detection system by making use of data from pervasive sensors (for example, video and physiology data) and enabling it with machine learning. However, the majority of the existing methods are based on complex wearables (such as an electroencephalogram) or computer vision algorithms (such as an eye state analysis), which makes it difficult to implement the relevant systems in the wild. In addition, the data that are derived from these methodologies are inadequate in nature because there were not enough simulation experiments. In this context, we propose a novel method for the driver drowsiness detection task that is easy to implement and is based on full non-invasive multimodal machine learning analysis. The degree of drowsiness was evaluated through the use of a self-reported questionnaire within the context of pre-designed protocols. To begin, we take into consideration the inclusion of environmental data (such as temperature, humidity, illuminance, and many other factors), which can be regarded as supplementary data for human activity data recorded by accelerometers or actigraphs. Second, we show that the models that were trained using data from everyday life can still be effective when used to make predictions about how a subject will perform in a simulator. This finding may have implications for how data will be collected in the future. In the final step of this research project, we conduct an exhaustive investigation into various machine learning methods, including traditional "shallow" models and more recent "deep" models. According to the findings of our experiments, the proposed methods are capable of achieving an unweighted average recall of 64.6% for drowsiness detection in a scenario that does not involve a subject.