Jusletter IT

Emotional Artificial Intelligence (EAI) is emerging as a mainstream technology. With the increasing use of EAI in different sectors, novel legal and ethical concerns are being raised. This paper focuses on an entertainment and creative content delivery recommendation system application, namely Amazon Alexa’s use of Neural-Text-to-Speech (NTTS) technology to capture and respond to users’ perceived emotions based on their voice. It also enables Alexa to play music based on, among other elements, its perception of users’ emotions. Given the recognised impact ‘music‘ has on emotions and Alexa‘s increasing involvement with big music sector actors, we are particularly concerned about how this domain can enable the manipulation of users. We use this example to highlight problems with the approach to AI regulation taken in the proposed EU Artificial Intelligence Act (AIA).