| Nama Pengarang | : | Andreas Triantafyllopoulos; Bjorn W. Schuller; Gökçe ?ymen; Metin Sezgin; Xiangheng He; Zijiang Yang; Panagiotis Tzirakis; Shuo Liu; Silvan Mertes; Elisabeth Andre; Ruibo Fu; Jianhua Tao |
| Nama Majalah/Jurnal | : | Proceedings of the IEEE |
| Volume / Nomor Edisi | : | 111 (No. 10) |
| Nomor Halaman | : | 1355-1381 |
| Abstraksi Narasi | : | Speech is the fundamental mode of human communication, and its synthesis has long been a core priority in human–computer interaction research. In recent years, machines have managed to master the art of generating speech that is understandable by humans. However, the linguistic content of an utterance encompasses only a part of its meaning. Affect, or expressivity, has the capacity to turn speech into a medium capable of conveying intimate thoughts, feelings, and emotions—aspects that are essential for engaging and naturalistic interpersonal communication. While the goal of imparting expressivity to synthesized utterances has so far remained elusive, following recent advances in text-to-speech synthesis, a paradigm shift is well under way in the fields of affective speech synthesis and conversion as well. Deep learning, as the technology that underlies most of the recent advances in artificial intelligence, is spearheading these efforts. In this overview, we outline ongoing trends and summarize state-of-the-art approaches in an attempt to provide a broad overview of this exciting field. |
| Nama Pengarang | : | Temitayo Olugbade; Liang He; Perla Maiolino; Dirk Heylen; Nadia Bianchi-Berthouze |
| Nama Majalah/Jurnal | : | Proceedings of the IEEE |
| Volume / Nomor Edisi | : | 111 (No. 10) |
| Nomor Halaman | : | 1333-1354 |
| Abstraksi Narasi | : | Given the importance of affective touch in human interactions, technology designers are increasingly attempting to bring this modality to the core of interactive technology. Advances in haptics and touch-sensing technology have been critical to fostering interest in this area. In this survey, we review how affective touch is investigated to enhance and support the human experience with or through technology. We explore this question across three different research areas to highlight their epistemology, main findings, and the challenges that persist. First, we review affective touch technology through the human–computer interaction literature to understand how it has been applied to the mediation of human–human interaction and its roles in other human interactions particularly with oneself, augmented objects/media, and affect-aware devices. We further highlight the datasets and methods that have been investigated for automatic detection and interpretation of affective touch in this area. In addition, we discuss the modalities of affective touch expressions in both humans and technology in these interactions. Second, we separately review how affective touch has been explored in human–robot and real-human–virtual-human interactions where the technical challenges encountered and the types of experience aimed at are different. We conclude with a discussion of the gaps and challenges that emerge from the review to steer research in directions that are critical for advancing affective touch technology and recognition systems. In our discussion, we also raise ethical issues that should be considered for responsible innovation in this growing area. |
| Nama Pengarang | : | Dongrui Wu; Bao-Liang Lu; Bin Hu; Zhigang Zeng |
| Nama Majalah/Jurnal | : | Proceedings of the IEEE |
| Volume / Nomor Edisi | : | 111 (No. 10) |
| Nomor Halaman | : | 1314-1332 |
| Abstraksi Narasi | : | A brain–computer interface (BCI) enables a user to communicate directly with a computer using only the central nervous system. An affective BCI (aBCI) monitors and/or regulates the emotional state of the brain, which could facilitate human cognition, communication, decision-making, and health. The last decade has witnessed rapid progress in aBCI research and applications, but there does not exist a comprehensive and up-to-date tutorial on aBCIs. This tutorial fills the gap. It introduces first the basic concepts of BCIs and then, in detail, the individual components in a closed-loop aBCI system, including signal acquisition, signal processing, feature extraction, emotion recognition, and brain stimulation. Next, it describes three representative applications of aBCIs, i.e., cognitive workload recognition, fatigue estimation, and depression diagnosis and treatment. Several challenges and opportunities in aBCI research and applications, including brain signal acquisition, emotion labeling, diversity and size of aBCI datasets, algorithm comparison, negative transfer in emotion recognition, and privacy protection and security of aBCIs, are also explained. |
| Nama Pengarang | : | Yekta Said Can; Bhargavi Mahesh; Elisabeth Andre |
| Nama Majalah/Jurnal | : | Proceedings of the IEEE |
| Volume / Nomor Edisi | : | 111 (No. 10) |
| Nomor Halaman | : | 1287-1313 |
| Abstraksi Narasi | : | An automatic emotion recognition system can serve as a fundamental framework for various applications in daily life from monitoring emotional well-being to improving the quality of life through better emotion regulation. Understanding the process of emotion manifestation becomes crucial for building emotion recognition systems. An emotional experience results in changes not only in interpersonal behavior but also in physiological responses. Physiological signals are one of the most reliable means for recognizing emotions since individuals cannot consciously manipulate them for a long duration. These signals can be captured by medical-grade wearable devices, as well as commercial smart watches and smart bands. With the shift in research direction from laboratory to unrestricted daily life, commercial devices have been employed ubiquitously. However, this shift has introduced several challenges, such as low data quality, dependency on subjective self-reports, unlimited movement-related changes, and artifacts in physiological signals. This tutorial provides an overview of practical aspects of emotion recognition, such as experiment design, properties of different physiological modalities, existing datasets, suitable machine learning algorithms for physiological data, and several applications. It aims to provide the necessary psychological and physiological backgrounds through various emotion theories and the physiological manifestation of emotions, thereby laying a foundation for emotion recognition. Finally, the tutorial discusses open research directions and possible solutions. |
| Nama Pengarang | : | James Z. Wang; Sicheng Zhao; Chenyan Wu; Reginald B. Adams; Michelle G. Newman; Tal Shafir; Rachelle Tsachor |
| Nama Majalah/Jurnal | : | Proceedings of the IEEE |
| Volume / Nomor Edisi | : | 111 (No. 10) |
| Nomor Halaman | : | 1236-1286 |
| Abstraksi Narasi | : | The emergence of artificial emotional intelligence technology is revolutionizing the fields of computers and robotics, allowing for a new level of communication and understanding of human behavior that was once thought impossible. While recent advancements in deep learning have transformed the field of computer vision, automated understanding of evoked or expressed emotions in visual media remains in its infancy. This foundering stems from the absence of a universally accepted definition of “emotion,” coupled with the inherently subjective nature of emotions and their intricate nuances. In this article, we provide a comprehensive, multidisciplinary overview of the field of emotion analysis in visual media, drawing on insights from psychology, engineering, and the arts. We begin by exploring the psychological foundations of emotion and the computational principles that underpin the understanding of emotions from images and videos. We then review the latest research and systems within the field, accentuating the most promising approaches. We also discuss the current technological challenges and limitations of emotion analysis, underscoring the necessity for continued investigation and innovation. We contend that this represents a “Holy Grail” research problem in computing and delineate pivotal directions for future inquiry. Finally, we examine the ethical ramifications of emotion-understanding technologies and contemplate their potential societal impacts. Overall, this article endeavors to equip readers with a deeper understanding of the domain of emotion analysis in visual media and to inspire further research and development in this captivating and rapidly evolving field. |
| Nama Pengarang | : | Guoying Zhao; Xiaobai Li; Yante Li; Matti Pietikainen |
| Nama Majalah/Jurnal | : | Proceedings of the IEEE |
| Volume / Nomor Edisi | : | 111 (No. 10) |
| Nomor Halaman | : | 1215-1235 |
| Abstraksi Narasi | : | Micro-expression (ME) is an involuntary, fleeting, and subtle facial expression. It may occur in high-stake situations when people attempt to conceal or suppress their true feelings. Therefore, MEs can provide essential clues to people’s true feelings and have plenty of potential applications, such as national security, clinical diagnosis, and interrogations. In recent years, ME analysis has gained much attention in various fields due to its practical importance, especially automatic ME analysis in computer vision as MEs are difficult to process by naked eyes. In this survey, we provide a comprehensive review of ME development in the field of computer vision, from the ME studies in psychology and early attempts in computer vision to various computational ME analysis methods and future directions. Four main tasks in ME analysis are specifically discussed, including ME spotting, ME recognition, ME action unit detection, and ME generation in terms of the approaches, advance developments, and challenges. Through this survey, readers can understand MEs in both aspects of psychology and computer vision, and apprehend the future research direction in ME analysis. |
| Nama Pengarang | : | - |
| Nama Majalah/Jurnal | : | Jurnal Filsafat |
| Volume / Nomor Edisi | : | 18 (No. 3) |
| Nomor Halaman | : | 273-294 |
| Abstraksi Narasi | : | In discussing about lecturing (teaching and learning) at the higher education, we are usually too much concerned with the approaches, methods, media of lecturing, but do seldom or even never pay attention to the objectives of lecturing. We are too much occupied with changing from one method of lecturing to another, for example, from active learning method, contextual method, con- structivistic method, student-based curriculum, school-based curri- culum, competence-based curriculum, up to curriculum of lesson unit level. Such a complicated thinking is futile unless people are real- ly concerned with directions and objectives of the lecturing activi- ties. In term of lecturing as a way of thinking, the planned activi- ties, we need previously to find out the direction and the goals of the activities, and then we may draw our attention to the methods in order to reach the targeted goals. The targeted goals of lecturing are making students think clearly and distinctly, making students can find the truth of scientific knowledge, making students become problem finders and problem solvers. As academic activities, the lecturing processes should be held scientifically. In addition to obtaining clear scientific stuffs, the lecturing activities should support the students to find the truth of science. After comprehending all types of truth, we may start to find out the description of scientific truth, as the targeted goal to pursue in the scientific activities. Furthermore, we may try to reveal and to find the truth of scientific knowledge in the lecturing activities. Through finding out the description of the truth during the lecturing activities, we expect to have appropriate orientation and steps of implementing the lecturing activities in order to reach the targeted goals of lecturing. By reaching the goals of lecturing, the students are expected to find some benefits from the lecturing activities. Then the activities will not be considered as burden for the students, but rather as the activities resulting in mental richness, generating enlightment, and increasing students' abilities. |
| Nama Pengarang | : | Jing Yang; Yordan Hristov; Jie Shen; Yiming Lin; Maja Pantic |
| Nama Majalah/Jurnal | : | Proceedings of the IEEE |
| Volume / Nomor Edisi | : | 111 (No. 10) |
| Nomor Halaman | : | 1198-1214 |
| Abstraksi Narasi | : | Facial action unit (AU) detection plays an important role in performing facial behavioral analysis of raw video inputs. Overall, there are three key factors that contribute toward the optimal performance of AU detectors: 1) being able to capture local AU-centered features; 2) exploiting the fact that some AUs co-occur with others; and 3) utilizing appearance changes across frames. We briefly review current techniques addressing each factor and discuss the challenges they meet. Given that very few works consider how to effectively and efficiently merge them all into a single framework that can be trained in an end-to-end manner, we propose facial AU detection with face alignment(AUNet), a simple yet strong baseline for landmark-based AU detection. AUNet implements the abovementioned key factors by: 1) using the intermediate layers of a pretrained face alignment model to act as our AU features’ space; 2) optimized to satisfy a correlation constraint, derived from the AU labels, and 3) temporal constraint, derived from variations in the contents of consecutive frames in the input videos. The proposed model, with its three key components, remains simple in nature and aligns with the primary AU detection task. Experiments on several benchmarks show that it substantially improves the AU detector’s accuracy and achieves new state-of-the-art AU detection results on popular benchmarks: BP4D and DISFA. Code is available at https://github.com/jingyang2017/AU-Net . |
| Nama Pengarang | : | Sicheng Zhao; Xiaopeng Hong; Jufeng Yang; Yanyan Zhao; Guiguang Ding |
| Nama Majalah/Jurnal | : | Proceedings of the IEEE |
| Volume / Nomor Edisi | : | 111 (No. 10) |
| Nomor Halaman | : | 1159-1197 |
| Abstraksi Narasi | : | Emotion and sentiment play a central role in various human activities, such as perception, decision-making, social interaction, and logical reasoning. Developing artificial emotional intelligence (AEI) for machines is becoming a bottleneck in human–computer interaction. The first step of AEI is to recognize the emotion and sentiment that are conveyed in different affective signals. Traditional supervised emotion and sentiment analysis (ESA) methods, especially deep learning-based ones, usually require large-scale labeled training data. However, due to the essential subjectivity, complexity, uncertainty and ambiguity, and subtlety, collecting such annotations is expensive, time-consuming, and difficult in practice. In this article, we introduce label-efficient ESA from the computational perspective. First, we present a hierarchical taxonomy for label-efficient learning based on the availability of sample labels, emotion categories, and data domains during training. Second, for each of the seven paradigms, i.e., unsupervised, semisupervised, weakly supervised, low-shot, incremental, domain-adaptive, and domain-generalizable ESA, we give the definition, summarize existing methods, and present our views on the quantitative and qualitative comparison. Finally, we provide several promising real-world applications, followed by unsolved challenges and potential future directions. |
| Nama Pengarang | : | Chi-Chun Lee; Theodora Chaspari; Emily Mower Provost; Shrikanth S. Narayanan |
| Nama Majalah/Jurnal | : | Proceedings of the IEEE |
| Volume / Nomor Edisi | : | 111 (No. 10) |
| Nomor Halaman | : | 1142-1158 |
| Abstraksi Narasi | : | The substantial growth of Internet-of-Things technology and the ubiquity of smartphone devices has increased the public and industry focus on speech emotion recognition (SER) technologies. Yet, conceptual, technical, and societal challenges restrict the wide adoption of these technologies in various domains, including, healthcare, and education. These challenges are amplified when automated emotion recognition systems are called to function “in-the-wild” due to the inherent complexity and subjectivity of human emotion, the difficulty of obtaining reliable labels at high temporal resolution, and the diverse contextual and environmental factors that confound the expression of emotion in real life. In addition, societal and ethical challenges hamper the wide acceptance and adoption of these technologies, with the public raising questions about user privacy, fairness, and explainability. This article briefly reviews the history of affective speech processing, provides an overview of current state-of-the-art approaches to SER, and discusses algorithmic approaches to render these technologies accessible to all, maximizing their benefits and leading to responsible human-centered computing applications. |