DESIGN AND IMPLEMENTATION OF A BIO-INSPIRED COGNITIVE SYSTEM FOR SOCIAL ROBOTICS: FROM EMOTION TO CONSCIOUSNESS.

1

REPORT SULLE ATTIVITÀ SVOLTE

REPORT ON THE ACTIVITIES UNDERTAKEN

Corso di dottorato / PhD Program ______in Ingegneria dell’Informazione

Cognome / Family Name __________LORENZO

Nome / Name ___________________COMINELLI

Data nascita / Date of Birth ________30/09/1984

Luogo nascita / Place of Birth _______Livorno (LI)

Data di rilascio del titolo / Date of issue of the certificate _______22/05/2018

Eventuale valutazione (se prevista) / Subsequent evaluation (if applicable) ___________________

Supervisore / Supervisor ___________Prof. Danilo Emilio De Rossi, Prof. Giovanni Vozzi, Eng.

Daniele Mazzei

Commissione di esame finale (Nome, cognome e ente di afferenza) / Final exam board (Name,

Family Name and relevant department) __Prof Krishna Persaud, University of Manchester; Prof

Chiara Vitale, Polytechnic of Turin; Prof Piergiorgio Gentile, Newcastle University; Prof.ssa Arti

Ahluwalia, University of Pisa.

ATTIVITÀ FORMATIVE / FORMATIVE ACTIVITIES

I anno di corso / I course year (anno accademico / academic year 2014/2015) DENOMINAZIONE

DEL CORSO / NAME OF THE COURSE DURATA / DURATI ON (in ore / in hours) SEDE / LOCATION (indicare Università o altra sede di svolgimento / indicate the university or other course location) DOCENTE / PROFESSOR (nome e cognome / name and family name) VOTAZIONE / FINAL MARK GIUDIZIO / EVALUATION LINGUA UTILIZZATA (italiano, inglese, altra lingua / Italian, English or other languages) Corsi del dottorato

frequentati/ Doctorate courses attended "SMART SPACES", 1 - 5 DECEMBER 2014 20 DII DMITRY G. KORZUN, PETROZAVOD SK STATE UNIVERSITY PASSED (GRADE NOT POVIDED) ENG PROGRAMMA PHD PLUS 2015: “CREATIVITÀ, INNOVAZIONE, SPIRITO IMPRENDITORIALE” DELL'UNIVERSITÀ DI PISA 32 COMPUTER SCIENCE DEPARTMENT GUALTIERO FANTONI, MECHANICAL DEPARTMENT , UNIVERSITY OF PISA PASSED (GRADE NOT POVIDED) ENG

2 , “ELECTROACTIVE POLYMER DEVICES FOR BIOMEDICAL AND BIOINSPIRED MECHATRONIC SYSTEMS”, 10, 17, 24 APRIL 2015 8 FACULTY OF ENGINEERING, PISA FEDERICO CARPI, QUEEN MARY UNIVERSITY, LONDON UK PASSED (GRADE NOT POVIDED) ENG

II anno di corso / II course year (anno accademico / academic year 2015/2016) DENOMINAZIONE DEL CORSO / NAME OF THE COURSE DURATA / DURATION (in ore / in hours) SEDE / LOCATION (indicare Università o altra sede di svolgimento / indicate the university or other course location) DOCENTE / PROFESSOR (nome e cognome / name and family name) VOTAZIONE / FINAL MARK GIUDIZIO / EVALUATION LINGUA UTILIZZATA (italiano, inglese, altra lingua / Italian, English or other languages)

III anno di corso / III course year (anno accademico / academic year 2016/2017) DENOMINAZIONE DEL CORSO / NAME OF THE COURSE DURATA / DURATION (in ore / in hours) SEDE / LOCATION (indicare Università o altra sede di svolgimento / indicate the university or other course location) DOCENTE / PROFESSOR (nome e cognome / name and family name) VOTAZIONE / FINAL MARK GIUDIZIO / EVALUATION LINGUA UTILIZZATA (italiano, inglese, altra lingua / Italian, English or other languages) Corsi del Dottorato

frequentati/ Doctorate Courses Attended "INTRODUCTION TO MODELING & SIMULATION OF COMPLEX & MULTI-DISCIPLINARY DYNAMICAL SYSTEMS", 5-9 JUNE 2017 20 FACULTY OF ENGINEERING, PISA LUCA DANIEL, MIT - USA PASSED (GRADE NOT POVIDED) ENG

3 "SYSTEMATIC PRODUCT DEVELOPMENT FOLLOWING THE CDIO APPROACH AND SUPPORTED BY CREATIVITY PROMOTION TOOLS AND BY ADVANCED DESIGN AND MANIFACTURING TECHNOLOGIES" 4 FACULTY OF ENGINEERING, PISA DR. ANDRES DIAZ LANTADA, UNIVERSIDA D POLITECNIC A DE MADRID - PASSED (GRADE NOT POVIDED) ENG "FINITE-ELEMENT MODELING FOR ENGINEERS", 19, 20, 26, 27 OCTOBER 2017 16 DII GIOVANNI PENNELLI - DII PASSED (GRADE NOT POVIDED) ENG "NATO LECTURE SERIES SET-243", 6-7 NOVEMBER 2017 12 DII FABRIZIO BERIZZI PASSED (GRADE NOT POVIDED) ENG Seminari di ricerca/Research Seminars (argomento e ore di svolgimento / topic and number of hours carried out)

13th LASER Summer School on Software Engineering – Speakers: Davide Brugali, University of Bergamo; Rodolphe Gelin, Softbank Robotics; Ashish Kapoor, Microsoft Research; Nenad Medvidovic, University of Southern California; Bertrand Meyer, Politecnico di Milano; Issa Nesnas, NASA Jet Propulsion Laboratory; Hiroshi “Gitchang” Okuno, Waseda University and Kyoto University. Goals: The 13th session of the LASER summer school covers the fascinating topic of software for robotics. Robots are everywhere, with an ever-increasing potential for helping humans achieve their goals. Software is a fundamental part of modern robotics systems; LASER brings together top international experts in the field, who will share their knowledge of robotics software and insights on outstanding research challenges. The school is intended for professionals from the industry (engineers and managers) as well as university researchers, including PhD students. The school’s focus is applied, although theory is welcome to establish solid foundations. The format of the school favors extensive interaction between participants and speakers.

9-17 SEPTEMBER 2017, ELBA ISLAND, ITALY (8 EXT. CREDITS)

PERIODO DI SVOLGIMENTO (da ___ a ____) ENTE OSPITANTE (nome, luogo e eventuale supervisore)

ATTIVITÀ SVOLTE E RISULTATI Esperienze di ricerca presso altre strutture / Research experience in other institutions FROM 01/09/2016 TO 01/12/2016

SPECS LAB, IN UPF (UNIVERISTAT POMPEU FABRA), BARCELONA, SPAIN

SOFTWARE INTEGRATION OF THE EASEL ARCHITECURE IN THE FACE ROBOT,

DEVELOPMENT OF DEMO SCENARIOS TO BE USED IN SCHOOLS AND MUSEUMS FOR EASEL, PREPARATION OF THE EASEL FINAL REVIEW

4

TITOLO DELLA TESI / TITLE OF THE PHD THESIS

“DESIGN AND IMPLEMENTATION OF A BIO-INSPIRED CONGITIVE SYSTEM FOR SOCIAL ROBOTICS:

FROM EMOTIONS TO CONSCIOUSNESS”

RESEARCH PROJECT

The research conducted in these three years of PhD has been aimed at the design, development, and test of control systems for social humanoid robotics. In particular, several software and hardware components have been designed and integrated in the FACE Robot (Facial Automaton for Conveying Emotions) and tested by means of Human-Robot Interaction experiments with human subjects. These systems have been used and constituted a fundamental part in the context of EASEL (Expressive Agents for Symbiotic Education and Learning) European Project, grant n. 611971. The cognitive system, which is the core of the research project, has been continuously upgraded to process data acquired by the new developed perception modules and to control the new created hardware components. Moreover, it has been enriched with a bio-inspired mechanism based on the Damasio’s theory of mind, simulating the way humans perceive emotions and feelings and how they influence the decision making and our beliefs on the social world. The result is SEAI, a Social Emotional Artificial Intelligence for making an artificial agent able to possess an artificial emotional state that is modulated by external events and biases the behavior of the robot.

CONVEGNI, WORKSHOP E SEMINARI IN ITALIA E ALL’ESTERO/ CONFERENCES, WORKSHOPS,

SEMINARS ATTENDED IN ITALY AND ABROAD (indicare denominazione, sede, data e se la

partecipazione ha previsto anche la presentazione di un contributo / please provide the name, the

location, the date and whether the participation included also the presentation of a contribution)

- LIVING MACHINE 2014, MILANO (ITA), 29/08 - 01/09, PAPER PRESENTATION AND DEMO

- AISB CONVENTION 2015, ARTIFICIAL INTELLIGENCE AND SIMULATION OF BEHAVIOR, CANTERBURY, UNIVERSITY OF KENT (EN) 20-24/04, PARTICIPATION ONLY

- LIVING MACHINE 2015, BARCELONA (ES), 27-31/07, PAPER PRESENTATION

- ACHI 2016, ADVANCES IN COMPUTER-HUMAN INTERACTION, VENEZIA (IT), 25-28/04, PAPER PRESENTATION

- LIVING MACHINE 2016, EDIMBURGH (SCO), 18-22/07, PAPER PRESENTATION - TECHNOLOGY HUB 2017, MILANO (IT), 19-23/04, DEMO WITH FACE ROBOT

5

PUBBLICAZIONI (N.B. depositate in ARPI) / PUBLICATIONS (Note: Publications must be stored in

ARPI)

COAUTHORS PUBBLICAZIONE TITOLO DELLA TITLE ESTREMI DELLA PUBBLICAZIONE (rivista, editore, convegno, altro) PUBLICATION DETAILS (journal title, publishing company, conference

title and so on)

ISSN/ISBN

INTERNATIONAL JOURNAL ARTICLE

Lazzeri N., Mazzei D., & De Rossi D. Designing the Mind of a Social Robot MDPI, Applied Sciences SECTION: Social Robotics EISSN 2076-3417 INTERNATIONAL JOURNAL ARTICLE

Mazzei D. & De Rossi D.

SEAI - The Social Emotional Artificial Intelligence based on Damasio’s Theory of Mind Frontiers in Robotics and AI, section Humanoid Robotics EISSN 2296-9144 INTERNATIONAL JOURNAL ARTICLE

Zaraki, A., Pieroni, M., De Rossi, D., Mazzei, D., Garofalo, R., & Dehkordi, M. B. Design and Evaluation of a Unique Social Perception System for Human-Robot Interaction IEEE Transactions on Cognitive and Developmental Systems. Print ISSN: 2379-8920 Online ISSN: 2379-8939 INTERNATIONAL JOURNAL ARTICLE Carbonaro N., Mazzei D., Garofalo R., Tognetti A. & De Rossi D. A Multimodal Perception Framework for Users Emotional State Assessment in Social Robotics

Future Internet ISSN 1999-5903

INTERNATIONAL CONFERENCE PROCEEDINGS

Mazzei, D., Lazzeri, N., Zaraki, A., & De Rossi, D.

I-CLIPS BRAIN - A hybrid cognitive system for social robots Biomimetic and Biohybrid Systems, Springer International Publishing ISBN 10: 3319094343 ISBN 13: 97833190943 42

6 INTERNATIONAL

CONFERENCE PROCEEDINGS

Mazzei, D., Pieroni, M., Zaraki, A., Garofalo, R., & De Rossi, D. Damasio's Somatic Marker for Social Robotics: Preliminary Implementation and Test Biomimetic and Biohybrid Systems, Springer International Publishing ISBN 978-3-319-22979-9 INTERNATIONAL CONFERENCE PROCEEDINGS Mazzei, D., Carbonaro, N., Garofalo, R., Zaraki, A., Tognetti, A., & De Rossi, D. A Preliminary Framework for a Social Robot “Sixth Sense” Biomimetic and Biohybrid Systems, Springer International Publishing ISBN 978-3-319-42417-0 INTERNATIONAL CONFERENCE PROCEEDINGS Reidsma, D., Charisi, V., Davison, D., Wijnen, F., van der Meij, J., Evers, V., Cameron, D., Fernando, S., Moore, R., Prescott, T., Mazzei, D., Pieroni, M., Garofalo, R., De Rossi, D., Vouloutsi, V., Zucca, R., Grechuta, K., Blancas, M. & Verschure, P. The EASEL Project: Towards Educational Human-Robot Symbiotic Interaction. Biomimetic and Biohybrid Systems, Springer International Publishing ISBN 978-3-319-42417-0 INTERNATIONAL CONFERENCE PROCEEDINGS Vouloutsi, V., Blancas, M., Zucca, R., Omedas, P., Reidsma, D., Davison, D., Charisi, V., Wijnen, F., van der Meij, J., Evers, V., Cameron, D., Fernando, S., Moore, R., Prescott, T., Mazzei, D., Pieroni, M., Garofalo, R., De Rossi, D. & Verschure, P. Towards a synthetic tutor assistant: the EASEL project and its architecture Biomimetic and Biohybrid Systems, Springer International Publishing ISBN 978-3-319-42417-0

7

ATTIVITÀ DI TUTORAGGIO E DIDATTICA INTEGRATIVA / TUTORING AND SUPPLEMENTARY

TEACHING ACTIVITIES

• Tutoring numerous Biomedical Engineering Students’ thesis

• Supplementary Teaching Activity in the course “SOCIAL ROBOTICS AND IOT” of the M.Sc. program in Bionics Engineering