A dynamic network model simulating brain plasticity for a robotic bioinspired intentional architecture : implementation and test

(1)

POLITECNICO DI MILANO

Corso di Laurea Specialistica in Ingegneria Informatica Dipartimento di Elettronica e Informazione

A dynamic network model simulating

brain plasticity for a robotic

bioinspired intentional architecture:

implementation and test

AI & R Lab

Laboratorio di Intelligenza Artificiale e Robotica del Politecnico di Milano

Relatore: Prof.ssa Giuseppina Gini

Correlatore: Ing. Flavio Mutti

Correlatore: Prof. Riccardo Manzotti

Tesi di Laurea di:

Alessio Mauro Franchi, matricola 749760

(2)

(3)

A me stesso, per tutto l’impegno profuso nel raggiungimento di questo importante traguardo.

Alla mia famiglia, che ha sempre creduto nelle mie capacit`a e mi ha spinto

(4)

(5)

List of Figures

2.1 Object recognition in cerebral cortex . . . 8

2.2 The thalamus as a cortical map . . . 10

2.3 Rewired ferret: before & after. . . 11

2.4 The bioinspired hexapod robot. . . 12

2.5 The Khepera robot . . . 14

3.1 Human brain areas . . . 16

3.2 fMRI scan example . . . 17

3.3 Subdivision of cortical areas . . . 18

3.4 The amygdala and the thalamus . . . 20

4.1 From brain to IA . . . 26

4.2 Intentional Module schema . . . 27

4.3 Ontogenetic Module schema . . . 29

4.4 Log-Polar transformation examples. . . 31

4.5 State-Action table . . . 32

4.6 K-Means algorithm . . . 42

4.7 The training schema . . . 44

4.8 The input projection . . . 46

4.9 Dynamic network schema . . . 48

5.1 Visual Studio Solution Structure . . . 52

5.2 UML class diagram, first section . . . 53

5.3 UML class diagram, second section . . . 54

5.4 Configuration files . . . 58

5.5 The IDRA main windows . . . 60

5.6 The IDRA binding phase . . . 60

5.7 Network’s and robot’s infos visualization . . . 61

(8)

5.9 The IDRA robot editor window . . . 62

5.10 The IDRA behaviours editor window . . . 63

5.11 The IDRA sensors editor window . . . 63

5.12 The IDRA actuators editor window . . . 64

5.13 The IDRA Viewer window . . . 64

5.14 The IDRA Viewer Form Focus . . . 65

5.15 The IDRA Viewer graphs . . . 65

5.16 The IDRA training setting form . . . 66

5.17 The IDRA Training Viewer . . . 67

5.18 The IDRA Training Form Focus . . . 67

6.1 The NAO robot . . . 70

6.2 Network for first experiment. . . 71

6.3 Gaussian function for head movements. . . 72

6.4 Experiment 1, the first table. . . 73

6.5 Experiment 1, the second table. . . 73

6.6 Experiment 1, first stage . . . 74

6.7 Experiment 1, second stage . . . 75

6.8 Experiment 1, final stage . . . 76

6.9 Network for second experiment. . . 78

6.10 Network fitness values. . . 79

6.11 Experiment 2, early development. . . 81

6.12 Experiment 2, form focus . . . 81

6.13 Experiment 2, the starting network. . . 82

6.14 Experiment 2, first stage . . . 82

6.15 Experiment 2, second stage . . . 83

6.16 Experiment 2, third stage . . . 84

6.17 Experiment 2, the final network. . . 84

7.1 Distributed computing . . . 90

7.2 Recurrent neural network . . . 92

7.3 The NAO robot . . . 93

B.1 The TestICA interface . . . 136

B.2 ICA example . . . 138

C.1 KmeansLib Simulator . . . 143

(9)

Summary

Bio-inspired robotics is a fairly new field where engineering meets biology; both take advantage of this tight collaboration, robotics by studying biolog-ical system for drawing inspiration from nature to develop new models or solve problems in the engineering field, biology for the validation of its hy-pothesis on artificial systems through an accurate methodology or for solving enigmas by observing robots. In this context, a newborn interest focus on human brain, studying in particular its mental capabilities and cognitive de-velopment, earned throughout the whole life; how human beings consolidate these skills is not yet fully understood, but some evidences have shown that two main features are involved; the first is the autonomous generation of new goals and motivations by means of the interaction with the environ-ment, the second refers to an interesting properties of cortical areas, referred as neuroplasticity.

From these considerations an intentional cognitive architecture based on an amygdala-thalamo-cortical model has been realized , aiming at repro-ducing in artificial systems these brain skills; first experimental results have demonstrated good system’s performances in creating new goals and com-plex behaviours starting from simple innate criteria, but have also highlighted some limitations; an important missing aspect is the lack of adaptability of the system to new situations and input stimuli, a fundamental property of human brain known as brain plasticity, which refers to changes in neural pathways and synapses due to modification in behaviour, environment or resulting from bad brain injuries.

The objective of this research is to extend the here proposed model, with the intent of making the architecture more dynamic and malleable, giving it the ability to abstract from raw sensory data thus forming high level con-cepts, and to adapt its structure to environmental changes or internal faults. Three key areas have been involved in this process: the input filtering stage,

(10)

Summary

responsible for the retrieval and filtering of incoming data; the categorization phase, during which interesting stimuli from the environment are collected in a sort of system’s memory, upon which new goals are developed; and finally the internal network, which is the real core of the architecture and represents the implementation of the amygdala-thalamo-cortical model.

(11)

Estratto in lingua italiana

Grazie al grande progresso delle scienze e delle tecnologie a cui stiamo assi-stendo in questi ultimi anni e al crescente interesse generale verso i sistemi autonomi, la robotica e l’intelligenza artificiale hanno fatto importanti passi in avanti; l’uso di nuovi e migliori materiali, gli studi di meccanica, le ricerche nel campo dell’intelligenza artificiale e della biologia hanno permesso di su-perare alcune barriere che fino a poco tempo fa sembravano insormontabili; hanno inoltre portato in questi ultimi anni alla realizzazione di una vasta gamma di robot, che comprende sistemi artificiali di ogni tipo certamente

molto sofisticati, ma anche efficienti e pi`u economici, in grado di portare a

termine in modo completamente autonomo compiti molto complessi.

Esistono comunque tuttora limiti non indifferenti ancora non superati e gli scienziati hanno di recente cominciato a cercarne una soluzione traendo ispirazione dalla natura; nasce cos`ı la robotica bio-ispirata, il punto di incon-tro tra l’ingegneria e la biologia, che si pone l’obiettivo di realizzare nuovi sistemi artificiali che imitino la struttura e il comportamento degli esseri vi-venti. In particolare, le ricerche si stanno negli ultimi anni concentrando sullo

sviluppo delle attivit`a cognitive del cervello nei bambini, in termini di

risor-se concettuali, del trattamento delle informazioni provenienti dall’ambiente esterno e dell’acquisizione del linguaggio naturale, con l’intento di realizzare agenti artificiali intelligenti dotati di capacit`a cognitive.

`

E la robotica dello sviluppo che si occupa di questi studi, nel tentativo

di riprodurre nei sistemi artificiali l’abilit`a degli esseri umani di sviluppare

autonomamente le loro capacit`a mentali e una coscienza dell’ambiente

cir-costante; `e infatti riconosciuto che questa propriet`a del cervello necessaria

agli esseri umani per adattarsi e rispondere in modo appropriato a nuove e sconosciute situazioni, sulla base dell’esperienza costruita nel tempo. Oltre a questa, fondamentale la generazione autonoma di nuovi obiettivi e stimo-li, differenti dagli istinti innati, tramite l’interazione con l’ambiente esterno.

(12)

Estratto in lingua italiana

Come tutti questi compiti complessi siano svolti `e ancora un mistero e non

ne sono ancora state svelate le meccaniche di base, ma ricordiamo che gli studi neuroscientifici sono ancora agli inizi.

Le ricerche svolte per questa tesi mirano alla realizzazione di un sistema artificiale bio-ispirato basato sui processi interni al cervello umano, che mostri

la capacit`a di sviluppare autonomamente nuovi obiettivi interni da

persegui-re, a parte da alcuni istinti di base innati; l’architettura realizzata si propone di affrontare un livello cognitivo intermedio, che `e ci`o che permette agli esseri umani di essere consapevoli dell’ambiente circostante e di interagire con esso per la maggior parte delle operazioni di base dell’organismo. Questa archi-tettura vuole raggiungere il proprio fine tramite l’imitazione della struttura

e delle funzionalit`a di una ben definita porzione del cervello umano; tre aree

in particolare sono state considerate, poich`e si ritiene che giochino un ruolo

chiave nelle capacit`a cognitive del cervello: la corteccia cerebrale, il talamo

e l’amigdala.

Questi tre elementi sono stati riprodotti nell’architettura da diversi

mo-duli; l’elemento chiave `e il Modulo Intenzionale (IM), che rappresenta la

corteccia cerebrale e il talamo; l’IM contiene al suo interno due sotto-moduli

pi`u specifici, quello di Categorizzazione (CM) e quello Ontogenetico (OM);

il primo svolge il ruolo della corteccia cerebrale, mettendo a disposizione del

sistema una sorta di memoria degli stimoli rilevanti e capacit`a di associazione

di base in risposta ad eventi esterni; il secondo invece rappresenta il talamo

dell’architettura ed `e il centro per la generazione di nuovi obiettivi; questo

modulo riceve in ingresso dal CM un vettore che descrive l’attivazione neu-rale della corteccia in risposta allo stato in input e, per mezzo di un sistema di apprendimento Hebbiano, sviluppa nuovi gol e sceglie il prossimo da

per-seguire; come output restituisce un segnale che indica se lo stato corrente `e

conforme ai nuovi obiettivi.

Un numero variabile di Moduli Intenzionali viene quindi raggruppato in una rete, il cuore di tutto il sistema, chiamata Architettura Intenzionale; i Moduli Intenzionali sono disposti nella rete su diversi livelli e comunicano tra loro per mezzo di link vettoriali e scalari, che possono essere sia diretti che

in retroazione; da notare che la struttura gerarchica a livelli di questa rete `e

fondamentale perch`e permette al sistema di astrarre dai dati grezzi mentre

questi attraversano la rete dall’input verso l’output, esattamente come accade all’interno del nostro cervello.

(13)

Mo-Estratto in lingua italiana

dulo Filogenetico Globale, che svolge il ruolo dell’amigdala, nel quale sono contenuti i criteri di bootstrap e alcuni obiettivi di base dell’agente, inseriti a mano durante la fase di sviluppo del software; questo sistema imita quan-to accade in natura, dove gli animali hanno alcuni istinti innati, necessari ad esempio per sopravvivere o per riconoscere la propria madre. Il Modulo Filogenetico ha il compito di valutare quanto l’input attuale soddisfi questi criteri interni, generando di conseguenza un segnale che indica il livello di in-teresse per lo stato corrente, che viene inoltrato direttamente a tutti i Moduli Intenzionali nella rete.

Il funzionamento del sistema `e lineare: il ciclo ha inizio quando un nuovo

input sensoriale `e pronto; queste informazioni sono recuperate dall’agente

e vengono inserite in una struttura dati predefinita, che permette all’archi-tettura di poter maneggiare qualsiasi tipo di dato sensoriale senza alcuna

difficolt`a; questi vengono poi opportunamente filtrati ed inoltrati ai Moduli

Intenzionali del primo livello della rete, rispettando i collegamenti tra filtri e moduli; anche il Modulo Filogenetico Globale riceve l’input corrente filtra-to, sulla base del quale calcola il segnale filogenetico e ne diffonde il valore in tutta l’architettura intenzionale. A questo punto le informazioni ricevute cominciano a fluire nella rete: ogni Modulo Intenzionale riceve in input il segnale filogenetico e un vettore di dati proveniente dai filtri o da altri mo-duli; questi due input sono opportunamente processati e viene prodotto un

output composto da un vettore e uno scalare: il primo `e una sorta di codifica

neuronale dell’input, il secondo indica quanto lo stato attuale dell’ambiente `

e interessante per il modulo stesso.

L’output globale dell’Architettura Intenzionale viene in ultimo passato al

sistema motorio, che `e in grado di decidere e generare il movimento migliore

considerando lo stato dell’architettura e gli obiettivi da perseguire; esatta-mente come si crede avvenga negli animali e negli essere umani, i movimenti sono generati dinamicamente, componendo dei segnali elementari chiama-ti primichiama-tive motorie. Si vuole sottolineare qui che per la scelta del miglior movimento da effettuare non viene impiegato un modulo decisionale, ma `

e il Comportamento Dinamico che, sulla base dei dati ricevuti, genera un

movimento che permetter`a all’agente di raggiunte i propri gol.

Le performance dell’architettura proposta sono state ampiamente veri-ficate per mezzo di alcuni semplici esperimenti; in un primo test, tramite

l’inserito di un singolo istinto nel Modulo Filogenetico, il sistema `e stato

ad-destrato in modo che inizialmente avesse interesse solo per gli oggetti molto V

(14)

colorati, senza specificare nulla riguardo le loro forme. Se nei primi istanti del test il sistema mostrava una forte attrazione per la figura della stella colorata, come d’altronde specificato nei suoi istinti, dopo un periodo di as-sestamento l’architettura ha cominciato a distinguere anche le forme degli oggetti, dimostrando un certe interesse anche per le figure non colorate a forma di stella.

I risultati sperimentali raccolti hanno messo in luce ottime capacit`a

del-l’architettura nel generare nuovi gol e comportamenti, ma hanno anche evi-denziato diversi limiti, attribuili molto probabilmente al fatto che questo

progetto `e solo all’inizio e che gli argomenti trattati sono innovativi; il

mar-gine di miglioramento `e ampio e possono essere introdotte svariate nuove

funzionalità o migliorie; probabilmente però l’aspetto su cui è necessario

la-vorare `e la mancanza di adattabilit`a del sistema alle nuove situazioni, sia

esterne che interne, ed a stimoli sensoriali mai osservati in precedenza; `e

infatti provata l’esistenza nel cervello umano di una importante propriet`a,

la plasticit`a cerebrale, che contribuisce allo sviluppo cognitivo e permette al

nostro encefalo di modificare la propria struttura e la propria funzionalit`a

sulla base dell’interazione con l’ambiente esterno.

Le ricerche svolte per questa tesi si concentrano proprio su questo ulti-mo punto, nel tentativo di sopperire alla mancanza evidenziata; il lavoro ha

previsto tre fasi principali: per prima cosa si `e proceduto ad una completa

re-implementazione e re-ingegnerizzazione del software esistente, nel rispetto delle classiche regole dettate dall’ingegneria del software, partendo dalla de-finizione di un diagramma UML delle classi, che ci aiutasse nella successiva

fase di stesura del codice; a seguire `e stato rivisto il Modulo di

Categorizza-zione, deputato principalmente alla classificazione degli input sensoriali; l’uso di ICA (Analisi delle Componenti Indipendenti) permette di astrarre dal

da-to grezzo, mentre per mezzo dell’algoritmo di clustering K-Means `e possibile

realizzare una codifica neuronale dello stato del sistema; infine, l’ultimo pas-so ha coinvolto l’Architettura Intenzionale, nel tentativo di rendere dinamica

la rete di Moduli Intenzionali, riproducendo cos`ı la plasticit`a cerebrale.

Segue la struttura della tesi.

Nel capitolo 2 `e presentato lo stato dell’arte della robotica bio-ispirata e di altri aspetti chiave riguardanti l’architettura qui presentata, quali il modello

amigdala-talamo-corticale e la plasticit`a cerebrale.

Nel capitolo 3 viene fornita al lettore una breve introduzione generale sul funzionamento del cervello umano, focalizzando l’attenzione principalmente

(15)

sulle tree aree coinvolte nello sviluppo cognitivo, l’amigdala, il talamo e la

corteccia cerebrale. `E poi sviluppata con cura la nozione di plasticit`a

cere-brale, con svariati riferimenti biologici, concetto chiave alla base di questo lavoro.

Nel capitolo 4 `e esposto il problema affrontato da questa ricerca, il

mo-dello realizzato e le soluzioni implementate per la sua risoluzione; nella

pri-ma parte `e descritta nel dettaglio l’Architettura Cognitiva, con tutte le sue

componenti e il loro funzionamento complessivo, partendo dai dati sensoriali grezzi fino alla generazione dinamica dei movimenti; sono quindi presentate

le funzionalit`a aggiuntive implementate: la fase di elaborazione dei dati

sen-soriali in input, gli algoritmi ICA e K-Means per il clustering dei dati e la Rete Dinamica, con i suoi criteri di sviluppo.

Nel capito 5 `e descritto il software IDRA (Architettura Robotica

Intenzio-nale Distribuita), ovvero l’implementazione dell’Architettura Cognitiva de-finita nel capitolo precedente; ne viene innanzitutto mostrato il diagramma UML, per poi passare alla descrizione tecnica del software e dell’interfaccia grafica.

Nel capitolo 6 sono descritte le attivit`a di sperimentazione ed i risultati

ottenuti.

Nel capitolo 7 vengono infine tirate le conclusioni di questo lavoro e sono suggeriti alcuni filoni possibili per ulteriori sviluppi futuri dell’architettura.

(16)

(17)

Chapter 1 Introduction

Even a journey of a thousand miles begins with a single step.

Lao Tzu, Chinese philosopher.

Concurrently with the very quick progress in science and technology we are assisting nowadays, robotics in these last years has grown a lot and taken significant steps forward, both in physical structure and in software. Thanks to the use of new and smarter materials and to studies in mechanics, artificial intelligence and biology, barriers that seemed insurmountable a few years ago now are not any more; recent robots are for sure more complex and have a more efficient physical structure, allowing them to solve for complex tasks, but are at the same time less expensive. There are however important limitations that have not yet been overcome, and scientists are now starting to search for new solutions drawing inspiration from nature; this is the field of bio-inspired robotics, where engineering meets biology with the objective of developing new architectures that imitate real natural systems behaviour [21].

In this context, a recent field of research is focusing on brain development in children in terms of conceptual resources, information processing and lan-guage learning, with the purpose of designing artificial agents with some sort of cognition capabilities [7]; in particular, developmental robotics aims at reproducing in artificial systems the ability of humans in autonomously de-veloping their mental capabilities and their awareness of the environment; these two skills are necessary for humans to adapt and respond properly to new and unknown situations, exploiting the experience built in the time. In

(18)

Chapter1 - Introduction

addition to these, it i fundamental the brain’s capability in generating new motivations and goals, different from innate criteria, by means of the inter-action with the environment; the way human brain accomplishes all of these complex tasks is not yet completely clear but studies in neuroscience are just at the very beginning.

This thesis is a first attempt in creating a bio-inspired intentional arti-ficial system with the ability of autonomously developing new goals during its interaction with the environment, starting from a few hard-coded in-nate criteria; the implemented architecture addresses an intermediate level of cognition, that is what allows humans and mammals to be aware of the surrounding environment. This architecture tries to achieve its objectives through the imitation of a portion of the human brain functionalities; in par-ticular three main areas and their interactions have been considered , that are known to play a fundamental role in cognitive capabilities: the cerebral cortex, the thalamus and the amygdala [14].

These three elements are reproduced in the architecture by different mod-ules; the key element is the Intentional Module (IM), that reproduces the functionalities of the cerebral cortex and thalamus; this module consists of two smaller internal ones, the Categorization Module (CM) and the Ontoge-netic Module (OM); the first plays the role of the brain cortex, providing to the system memory and basic association capabilities in response to external stimuli; the second acts as thalamus and is the responsible for new goals gen-eration; it receives a vector of neural activations from the CM and through an Hebbian learning function develops and selects new objectives.

A variable number of IMs is then collected in a network called Intentional Architecture, the core of our system; IMs in this network are disposed in layers and communicate through vector and scalar links; these connections can be both straightforward o feed-forward. It is important to note that this hierarchical structure is fundamental for the system to be able to abstract from raw data as information flows from network input to the output, just as it happens in our brain.

Finally, beside this network, there is a third module, the Global Phyloge-netic Module (GPM), which contains all of the bootstrap criteria and objec-tives hard-coded during software development; this mechanism imitates what happens in nature, where all animals have some innate instincts, e.g. for sur-viving or recognizing their mother’s face. The GPM performs the function of the amygdala and feeds all the other IMs with a signal much higher as the

(19)

current state satisfy some of the innate criteria.

The operations of the system start when a new input from all the available sensors is ready; raw data are packaged and properly filtered and are then sent to the IMs in the first layer according to the links, and also to the GPM, which spreads its activation value in the network. Now information flows through all the net: each IM receives as input the phylogenetic signal and a vector coming from sensors or from other IMs and processes these received information; at the end it produces an output composed by a vector, that is a high level representation of the state of the system, and a scalar, which states how much the current vector input is good for the architecture.

The final output of the network is finally sent to the Motor System, that selects and generates the correct movement for the actual situation and goals; movements are formed dynamically by a series of elementary components, the motor primitives, imitating the way animals and humans compose their movements [55]. There is not any decision making module for selecting the best action to perform, but it is just the Dynamic Behaviour that, according to input data, generates an action that will lead to the accomplishment of its goals.

This proposed architecture has been tested with some experiments, in order to verify its capabilities; in a first experiment, for example, the system was trained to like high saturated objects, such as red or yellow stars, using a single rule in the Global Phylogenetic Module, but nothing was specified about object’s shapes; while at the beginning the system follows its predeter-mined instinct, after some running time it develops a new goal and started to distinguish also the shapes of the objects, regardless their color.

The collected experimental results have demonstrated good capabilities of the architecture in autonomously generating new goals, but have also showed off some limitations, probably arising from the youth of this project. A lot of improvements and future developments could be made, like code optimization or the addition of new functionalities; but probably the most limiting aspect is the lack of adaptability of the system to new situations and input stimuli, simulating the human brain plasticity, an important property of cerebral cortex that refers to changes in neural pathways and synapses, due to modification in behaviour, environment or resulting from bad injuries [44].

Researches done for this thesis focus precisely on this last aspect, trying to account for the highlighted problem; works has consisted in three main steps; 3

(20)

Chapter1 - Introduction

first a complete re-engineering and re-implementation of the whole system, in respect of all the main rules of software engineering and with the definition of a detailed UML class diagram; second the implementation of Independent Components Analysis (ICA) and K-means algorithm, to obtain abstraction from raw data and for input classification; and last, an improvement of the system focusing on the Intentional Architecture making it dynamic to pro-vide to the architecture a sort of malleability, reproducing cerebral cortex plasticity.

Following is the thesis structure.

In chapter 2 is presented the state of art about bio-inspired robotics and other aspects related to the architecture here presented, such as the amygdala-thalamus-cortex model and the neuroplasticity concept.

In chapter 3 is given to the reader a brief general introduction about the functioning of the human brain, focusing in particular on the cognitive development and the three involved areas, the amygdala, the thalamus and the cerebral cortex; it is then illustrated the notion of brain plasticity, with various biological references, which represents the key concept of this thesis. In chapter 4 are presented an analysis of the problem that this thesis aim to solve and adopted solutions. In the first section is given a general descrip-tion of the Cognitive Architecture, of all its components and of their func-tioning; in the second are then shown the additional features implemented: the input processing, the ICA (Independent Component Analysis) algorithm for clustering and the Dynamic Network with its development criteria.

In chapter 5 is described the IDRA (Intentional Distributed Robotic Ar-chitecture) software, that implements the Cognitive Architecture shown in previous chapter; the UML class diagram is presented and the description of the software follows the subdivision of the Visual Studio solution in different projects, each one representing a significant part of the system.

In chapter 6 are described the experimental activities and are discussed results and system performances.

Finally, in chapter 7 are drawn conclusions of this work and are suggested some possible future developments for this architecture.

In appendix A are listed some of the most important code sections, with a brief description.

In appendix B is shown the library implementing all the necessary func-tions for the ICA execution.

(21)

for the execution of the K-Means clustering algorithm.

(22)

(23)

Chapter 2 State of art

A new approach to robotics and artificial intelligence arises from the obser-vation of natural systems; the key idea is that intelligence emerges primary from cells, bodies and societies, but also from evolution, development, and learning [21]; some examples of this new vision are evolutionary computa-tion, artificial neural network, bio-robotics and swarm robotics; a simple but effective result of this collaboration between robotics and biology can be seen in [23], where flying insects inspired an artificial controller for a robotic heli-copter, while at the same this regulator explained how insects manage to fly safely over a contrasting ground.

Recently, artificial intelligence has been in particular concerned with re-producing the abilities of human brains, thanks to new researches in neuro-science; developmental robotics aims at reproducing the evolution of cogni-tive abilities of the human brain, and represents the intersection of robotics, cognitive and developmental science [35].

2.1 Cognitive development in human brain

There are several studies which have shown the importance of the amygdala, the thalamus and the cortex in the process of cognitive development [30] [51]. The cortex is a thick layer of neural tissue that covers all the other parts of the brain; its structure can be subdivided in different areas, such as the primary visual cortex, posterior parietal cortex, primary motor cortex, etc., and each section receives input from a specific sensory organs; however, sev-eral studies demonstrated that different areas are able to process correctly

(24)

Chapter2 - State of art

different stimuli [14]. Its dimensions and its evolution in time have led many theorists to assign the highest of cognitive capabilities to a specific area, the frontal cortex; some researches on patients with frontal lobe damages have in fact highlighted that PFC injuries or diseases create cognitive deficits, including problems of attention, motor control, short-term memory, tempo-ral and source memory, metamemory and associative learning; there is then agreement that cortex is critical for working memory functions [31]. More-over it seems likely that the cortex, in particular perirhinal cortex, plays an important role not only in memory, but also in object perception and recog-nition [40]; in the service of object identification, perirhinal cortex associates different views of objects with other various attributes, creating thus a uni-fied representation; it also helps in associating objects with other objects and some abstractions, for progressing toward a goal.

Figure 2.1: The cerebral cortex forms high level concepts for associations and recog-nition of objects.

The amygdala is a collection of nuclei found in the anterior portions of the temporal lobes in the brains; it receives projections from frontal cortex, asso-ciation cortex, temporal lobe, olfactory system and other parts of the limbic system. It then sends back its afferents to frontal and prefrontal cortex, brain stem nuclei orbitofrontal cortex, as well as hypothalamus and hippocampus. While other parts of the brain, such as the brainstem, are almost fully func-tional at birth, the amygdala is more plastic and slower to develop; some experimental evidences have shown that it requires social, emotional and

(25)

2.1 - Cognitive development in human brain

cognitive stimulation during the first years of life to grow normally and, if sufficient stimulation is not provided, neurons involved in perceiving social emotional nuances and those forming and remembering emotional attach-ment may atrophy or develop abnormal activities [29]. This part of the brain is a primary source of social behaviours and the lack of a functioning amyg-dala severely limit human range of social responses; more recent findings have confirmed this important role of the amygdala in social behaviours [1]. More-over, through studies on lesions and functional neuroimaging analysis arises that the amygdala is involved also in complex cognitive functions, like the judgement of emotion in facial expression; these deficits expressed at a high behavioural level are most likely the manifestations of a defect in coupling stimuli with their emotional attributes [4]. Finally, it has been demonstrated that the amygdala is involved in memory processing, in particular in the for-mation and storage of memories associated with emotional events, with the important task of memory consolidation [38].

The thalamus is a symmetrical structure found in the brains of verte-brates, including humans, situated between the cerebral cortex and midbrain, both in terms of location and neurological connections. In past literature there were little focus on this brain area, due to the fact that thalamus, in contrast to the cerebral cortex, basal ganglia, and cerebellum known to play roles in cognitive aspects of behaviour, was generally considered a simple groups of relay nodes. Moreover, its structure is quite complex to study, as it is composed by nuclei with irregular boundaries. But, now that many other brain regions have been explored, considering its position in the brain thala-mus functions are starting to seem more attractive, mostly for experiments and testing purpose [50]. There is a large agreement on the hypothesis that thalamus is a sort of miniature map of the cortex, as described in [29]. The thalamus is divided into different nuclei, which do not directly communicate; instead, each nucleus is connected with a specific cortical area, as illustrated in figure 2.2. This structure is common in all mammals, but with different dimensions: in mammals with bigger brain this circuitry is augmented by the addition of an entire cortico-thalamic units. Among cerebral functions attributed to the thalamus there are the relaying of sensory information and motor signals to the cerebral cortex and the regulation of consciousness, sleep and alertness [48]. In its relaying function, the thalamus also filters sensory data and elaborates input on the basis of the emotional state of the subject; this allows to determine how incoming stimuli are relevant, thus absolving a 9

(26)

crucial task for human episodic memory, for consciousness and cognition.

Figure 2.2: Some circuitry of the thalamo-cortical complex seen in a horizontal slice; central lobes represent amygdala nuclei; on the left are represented cortico-thalamic projections, while thalamo-cortical are on the right.

Besides these three brain areas, neuroscientists agree on neuroplasticity being fundamental for a correct cognitive development; plasticity is an intrin-sic property of the human brain, highly related to dendritic arborization as discussed in [32], which enable the nervous system to escape the restrictions of its own genome and to adapt to environmental and physiologic changes; brain plasticity is involved in development and learning, but can also be the cause of some pathologies; in [44] is faced the problem of understanding how the mechanisms of plasticity work, aiming at modulating them to achieve the best behavioural outcome for a given subject. From several works arises that changes in the mapping between behaviour and neural activity take place in response to changes in afferent input or efferent demand, as can be deduced by experiments of ”switched” ferrets [47]. From this kind of works, it has been demonstrated that electrical activity of thalamocortical afferents plays a progressive role in shaping cortex [52]; while at the beginning the presence of this activity is sufficient, the spatiotemporal pattern of input activity is necessary for features that develop later.

(27)

2.2 - Bio-inspired and developmental robotics

Figure 2.3: Rewired ferret experiment: on the left are represented involved brain areas and connection in normal ferret, while on the right is shown the result of the modification.

Another interesting experiment has shown that cortical areas deprived of its input respond at a later time to other adjacent input [39]; studies on adult monkey’s brain before and after the amputation of the third digit has confirmed that before this amputation there were five distinct areas, one for each digit; after some times of rehabilitation, the area corresponding to the amputated digit was invaded by those corresponding to second and fourth digit. From analysis of transcranial magnetic stimulation (TMS) or functional magnetic resonance imaging (fMRI), researchers are able to study cortical plasticity and thus characterize how it varies in time; it is interesting to note that, as stated in [43], the efficiency of brain plasticity gets lower as we get older, with a decrease factor distinct for each individuals, depending on some variables defined by genetic, biological, and environmental factors.

2.2 Bio-inspired and developmental robotics

Bio-inspired robotics is about studying biological systems, looking for the mechanisms that may solve a problem in the engineering field; despite it is an almost newborn area of study, it is very vivid due to the fact that first experiments have shown encouraging results. Examples from literature are numerous and range from modular to humanoid robots, including also studies in swarm robotics; biology and neuroscience have not only influenced physical robot structure, but also sensors, with the arrival of biosensors, bioactuators or biomaterials, and artificial intelligence, involving for example navigation algorithms as in [49].

(28)

2.2.1 Bio-inspired robots

There are several examples in bio-inspired robotics; among the most ad-vanced humanoid robots we can find Petman, a first attempt of Boston Dy-namics, and Asimo, introduced in 2000 by Honda, 130 cm tall and weighs 54 kg; it is able to recognize moving objects, postures, gestures, voices and faces, enabling it to interact efficiently with human begins.

Also the design of non humanoid legged robots draws inspiration from animals; a common problem to deal with is the lack of speed and robustness seen in even the simplest of animals; in [12] is described a small six-legged run-ning robots which tries to imitate small arthropods, in particular cockroaches, which are capable of traversing uneven terrain without appreciably slowing down. From biology have emerged that key factors are a good self-stabilizing posture mechanism, a stabilizing leg function, a ”visco-elastic structure” and an open loop with feedforward controller; this research resulted in a very simple robotic prototype dynamically similar to its inspiration, with a basic feedforward control schema sufficient for straight ahead running over smooth and uneven terrain.

Figure 2.4: The transposition of cockroach locomotion into an hexapod robot; each leg performs a different function: while front legs act as decelerators, hind legs act as accelerators; those in the middle act as both.

(29)

2.2 - Bio-inspired and developmental robotics

Recent researches in neuroscience have led to a new branch of bio-inspired robotics, called motivational-based robotics, also known as epigenetic robotics or developmental robotics [35]; while behaviour-based have fixed goals to be pursued, this new kind of robots must be able to create autonomously its mo-tivations, which they did not possess at design time but are developed by the interaction with the environment, just as happens in mammals and humans during their life timespan. This distinction is well explained in [37]: in nature animals at birth have very limited set of motivations, which is extended on the basis of genetic background and past experience; it is always thus possible to distinguish between phylogenetic and ontogenetic aspects, where the first refers to those goals which are determined before the actual development of an agent, the latter represents those specified and generated after the birth. This distinction is the central key of the amygdala-thalamus-cortex model, the basic concept for the developed motivation-based architecture.

2.2.2 Brain plasticity in robotics

How animals are able to accomplish ontogenetic functions is not yet fully understood, but evidences state that brain plasticity is involved; as proof of this, in [41] is explained an interesting experiment focusing on brain-machine interplay: the brains of sea lamprey larvae were extracted and placed in a recording chamber, maintained at constant temperature. Two stimulation microelectrodes were placed on the right and on the left side of the midline, while two recording glass-electrodes were connected to each side of the brain-stems midline, for motor commands. These brains were thus connected, via a simple encoder/decoder, to a small wheeled robot, a Khepera by K-Team (figure 2.5), which had a set of optical sensors for light intensity sensing; the brain driven robot was stimulated for some time with a light source and was quite impressive to see the small robot starting to respond with movements to light stimuli; the robots showed both ”positive phototaxis” and ”negative

phototaxis”1_{. What happened here is that lamprey’s brain, stimulated with}

a new incoming pattern, was able to modify its structure and learn to

under-1_{Positive phototaxis is a concept defined by Braitenberg in his book: if you take a}

simple two-wheeled robot with just two light sensors, and directly connect the right sensor to the left wheel and vice-versa, then a light shining on one side will cause the wheel on the opposite side to spin faster (spinning is proportional to measured light intensity). As a result, the vehicle will tend to orient itself toward the light and to proceed in the forward direction. Negative phototaxis is the opposite, with direct connections on the same side.

(30)

stand this information; moreover the same occurred for motor stimuli thanks to feedback signals, resulting in the brain driving the wheeled robot.

Figure 2.5: The Khepera robot used by [41] for experiments on brain-machine interplay The most immediate transposition of this brain plasticity into robotics are dynamic neural networks, a sub-field of artificial intelligence dealing with networks able to change autonomously their structure; several interesting works focusing on this topic can be found in literature; in [8] artificial neural networks have been merged with genetic algorithm, which can be used by higher neural network layers to improve performances in image clustering, in terms of classification percentage and confidence; the motivation for this re-search comes from the fact that genetic algorithm performs well in stochastic global search, while ANNs are good in accurate local optimization based on gradient information.

Another possible criterion for the evolution of neural networks could be the removal of useless neurons; in [24] has emerged that, when this method is used in the context of the growing neural gas model, the resulting network is able to closely track non-stationary distributions, where slow changes in this distribution are handled by adaptation of existing units, while rapid ones are followed by removal of useless neurons, followed by the addition of new units in other places; the key parameter of this algorithm is the definition of a local utility function for choosing the neurons to be removed and the location of new nodes.

(31)

Chapter 3 How the brain might work

Human and mammalian brain is a very complex but surprising organ; even if it is the subject of numerous studies, both in neuroscience and psychology, its functioning is not yet fully understood, mainly due to its complexity. The brain is fundamental for every living beings, and performs an incredible number of tasks, ranging from motor control, to regulation of vital functions, and also lets you think, dream, reason and interact with the environment.

Among all of these functionalities, recent studies are dealing in particular on cognitive development, which refers to the process of mental capabilities learning that occurs in human during their life, from childhood to adult-hood, focusing in particular on children’s perceptual skills and information processing progress or language learning [36].

How cognitive processes take place in our brain is not yet completely un-derstood [10], but from recent studies have emerged that three main areas are primarily involved, the cerebral cortex, the amygdala and the thalamus [40] [9]. Further, on the basis of other evidences, it is assumed that au-tonomously goals generation and neural plasticity are necessary for a correct mental development.

This last important property of the brain, referred as neuroplasticity, is what makes it able to adapt to environmental changes or also to modifi-cations within the organism itself; it is strictly related to experience and learning, two factors which affect the internal structure of the brain and also the functionalities of different areas of the cortex. A first interesting study about the development of cortical layers has shown that input electrical ac-tivity and its spatio-temporal pattern has a progressive role in shaping cortex [52]; an experiment on ferrets, where input from the retina has been routed

(32)

Chapter3 - How the brain might work

to the auditory pathway at early stage, has demonstrated some changes in intra-cortical connections and in the behavioural role of rewired cortex, thus confirming the presence of some kind of adaptation of brain to new type of stimuli [47].

3.1 Human brain functionalities

Generally speaking, the human brain is not the biggest in nature if we con-sider its absolute measures, but it becomes so when related to body’s sizes; much of the brain is occupied by the cerebral cortex, especially by the frontal lobes, which are associated with executive functions as self-control, planning or abstract thought [17] [2]; also vision areas are very developed in humans. Cerebral cortex is a sort of layer of neural tissue that covers almost all of the other brain areas and it is folded in order to increase the amount of surface available in the same volume (see image 3.1). Human brain is fundamental for human living and despite it is well protected by the cerebrospinal fluid and by the bones of the skull, it is subject to damages or diseases, as brain stroke, Parkinson’s disease or multiple sclerosis.

Figure 3.1: A view of a section of a human brain, with its most important areas.

Although it has always been much studied, the functioning of the brain has not yet been fully unveiled by scientists because of its complexity; beyond

(33)

3.1 - Human brain functionalities

that, there are also limitations in the techniques used to analyse this organ, that are quite different from those used for other animals. Information about structure and function comes from a variety of experimental methods, e.g. electroencephalography, magnetoencephalography, structural and functional imaging or also from the study of the effects of brain damages, but invasive techniques like internal electrodes or disabling parts of the brain must be avoided. However on the other hand humans are able to respond better than other animals to complex verbal instructions and so harder experiments with clearer results can be obtained; finally it is important to note that language can hardly be studied at all except in humans.

Figure 3.2: An example of functional magnetic resonance imaging, a procedure that measures brain activity by the analysis of blood flow.

Among the brain abilities, are very interesting cognition and goals gen-eration; from childhood through adolescence to adulthood our brain shows a very significant cognitive development, during which we assist to the con-struction of thought processes, remembering and problem solving attitude. Understanding the relation between brain and mind is a great challenge, and it is very difficult to discover how a network of neurons and synapses allow our brain to have thoughts or emotions [11]; these difficulties have led to du-17

(34)

alism, a theory by Descartes which states that the mind exists in parallel to the brain [26]; of course this theory has been strongly criticized, because of some evidences showing that physical manipulations or damages to the brain also affect mind; this close relation between the activity of the two pushed neuroscientists toward materialism, which believes that mental phenomena are reducible to physical phenomena [33].

3.1.1 Cerebral Cortex

The cerebral cortex is constituted by a thick sheet of neural tissue, folded back on itself to increase the surface that can be contained in the same volume. It is the outermost part of the brain and covers most of it; there are many connections with other structures such as the thalamus or the basal ganglia, with which it exchanges a lot of information; it is significant to see, for example, that most of the sensory data are routed to the cerebral cortex via the thalamus. Among all the features of the cerebral cortex and its connections, one of the most interesting is the neuroplasticity, that is the ability to modify itself to adapt to new conditions, new stimuli or to compensate for damages in certain areas of the brain [43] [44].

Figure 3.3: The cerebral cortex is subdivided in three main parts: sensory, motor and association areas.

(35)

3.1 - Human brain functionalities

Composed by left and right hemispheres, the cortex is described by as comprising three parts: sensory, motor, and association areas [54].

Sensory areas receive and process input from the senses; in particular, the parts that receive information from the thalamus is called primary sensory areas. Other important areas are the visual of the occipital lobe, the auditory, in parts of the temporal lobe and insular cortex, and the somatosensory in the parietal lobe. It is interesting to note that the two hemispheres receive data from the opposite side of the body.

Motor areas are located in both hemispheres of the cortex; they are related to the control of voluntary movements and send axons down to motor neurons in the brainstem and spinal cord. Motor areas are divided in primary motor cortex, which executes voluntary movements, and supplementary motor ar-eas, which select voluntary movements. Other zones of the brain accounted for motor functions are the posterior parietal cortex, for guiding movements in space, and dorsolateral prefrontal cortex, delegated to the choice of the correct movement according to higher-order instructions, rules and thoughts. Finally, the last sections consist in the remaining parts of the cortex and they are called association areas; these are meant to receive input from sensory areas and from lower parts of the brain to produce a perceptual experience of the world, supporting language and the complex process that we call perception. Another important function of association areas is the integration of data from different sensors and to relate these information to past experiences.

3.1.2 Amygdala

The amygdala is an almond-shaped groups of nuclei 1, located deep within

the medial temporal lobes of the brain and it is considered part of the limbic system; that is not the only type of structure, and in fact there is also a section of the amygdala with a layered cell organization, much more similar to the cerebral cortex.

The amygdala has numerous connections with other brain regions, and it participates in a lot of the brain functions; for example it is involved in emotional functions and responses, both learned and innate, or in the genesis of many fear responses, such as freezing (immobility), tachycardia,

1_{A nucleus is a brain structure consisting of a relatively compact cluster of neurons.}

(36)

increased respiration and stress-hormone release [45]. It has been observed that damages impairs both acquisition and expression of fear conditioning.

The amygdala is also responsible for the modulation of cognitive func-tions, such as attention, perception, by means of the use of rewards to mo-tivate and reinforce behaviours or the release of hormones and neuromodu-lators in the brain that alter cognitive processing. Finally, it is important in the processing of memory, with a primary role in the formation and storage of memories associated with emotional events; the amygdala is not a storage, but is necessary for memory consolidation [38].

Figure 3.4: A pictures representing positions and dimensions of amygdala and thalamus in the human brain

3.1.3 Thalamus

The thalamus is an oval-shaped structure composed by about fifty nuclei, situated between the cerebral cortex and midbrain, near the center of the brain; this nuclei have a lot of links to and from the cerebral cortex that form the thalamo-cortico-thalamic circuits, closing loops of neural pathways that are believed to have an important role in consciousness. The thalamus

(37)

3.2 - Neuroplasticity

is a part of a more complex structure, composed by the hypothalamus, the epithalamus, the ventral thalamus, and the dorsal thalamus.

Among the functions of the thalamus there are the regulation of con-sciousness and alertness [48]; it also plays an important role in the regulation of the arousal and of the level of awareness and activity, and these hypothe-sis are confirmed by evidences showing that important damages to this brain area may lead to coma. The thalamus also acts as filters for sensory data destined to other parts of the brain, elaborating input according to the emo-tional state of the subject, and this function is crucial for human episodic memory and for some motor system functionalities.

3.2 Neuroplasticity

Neuroplasticity, also known as brain plasticity or simply plasticity, is the abil-ity of the brain to modify its structure and neural pathways to accommodate changes in behaviour, in the environment or resulting from severe injuries [43] [25]. Its functioning is associated with the concept of synaptic pruning, which states that connections among different brain’s areas are continuously added or removed, according to their utility.

Brain plasticity is strictly connected to the acquisition of new knowledge, new skills and to experience, events that actually modify both structure and functional organization of brain; the problem of learning a new skill is interesting: initially one is totally inept, but starts to master the task in a quite short time with practice. A new set of behaviours has been acquired, and some changes in the brain has happened [32]. It is believed in fact that if one repeatedly engage in a particular behaviour, this may alter the brain, thus facilitating the behaviour itself. Recently, also the concept of creativity has been linked to neuroplasticity: the common idea of one part of the brain being responsible for creativity is inaccurate and, according to a new view, the brain is seen much like like a muscle, the more you work it the more efficient it gets.

This concept of neuroplasticity is quite new among neuroscientists, which initially believed that brain, after a first period during childhood, was a rigid and immutable organ; several recent researches have highlighted, however, that this is not true, showing how substantial changes can occur in the

low-est neocortical processing areas. For example, an interesting experiment

conducted by Michael Merzenich, Jon Kaas and Doug Rasmusson has shown 21

(38)

that cortical areas deprived of its input respond at a later time to other adjacent input; in particular they have studied adult monkey’s brain before and after the amputation of the third digit. It has emerged that before the amputation there were five distinct areas, one for each digit; after some times of rehabilitation, the area corresponding to the amputated digit was invaded by those corresponding to second and fourth digit [39].

A more recent research focused on the evolution of plasticity in both the somatosensory and auditory systems, which both have shown similar changes with respect to behaviour; cortical representation of a stimulus, when asso-ciated with a reinforcement, is strengthened and enlarged, becoming two to three times bigger in about one or two days, stopping its growth within a few weeks. Control studies have shown that these changes are not caused solely by sensory experience, but they require some learning about the sensory experience.

Another fundamental consequence of brain plasticity is the process of recovery from brain injuries: it has been shown that brain activity associ-ated with a certain function can be mapped onto a different brain area; as consequence, a good rehabilitation approach after brain injuries is based on goal-directed experiential therapeutic programs, to strongly stimulate the brain to adapt to its new conditions. An interesting clinic case of this brain ability can be found in [19]; in this article is described a patient with a

bi-lateral brain damage, involving an important section of the limbic system2.

Despite the importance of his brain damage, the patient has an average IQ, shows a normal attention level, good executive functioning skills and strong

speech and language abilities; the patient suffers only from working memory3

problems and from anosmia4 _{and ageusia}5_.

Two other cases of brain injury have induced Paul Bach-y-Rita, one of the first neuroscientist to seriously study the idea of neuroplasticity, to define the concept of sensory substitution: stimulating one working sense to take the place of another, working or not; he applied this concept into a tool to treat patients suffering from neurological disorders, the BrainPort, an

2_{The limbic system include various cerebral structures: hippocampus, amygdala,}

an-terior thalamic nuclei, and limbic cortex; it is involved in some various psychological functions, such as behaviour, long term memory and olfaction.

3_{Working memory is defined as a system for information processing that store or retrieve}

data from a collection of short term memories for an active manipulation store item [5].

4_{Anosmia is the inability to perceive odours.} 5_{Ageusia is the loss of taste functions of the tongue.}

(39)

3.2 - Neuroplasticity

external machine that substitutes the lost sense, sending correct input to the brain through the tongue [15]. The first case of this therapy is related to a

patient with her vestibular system6 _{damaged by some antibiotic taken after}

an infection due to surgery; this problem caused her inability in walking or just in standing up; she then decided to undergo the experimental treatment with the BrainPort: a fat strip of tape composed by an array of micro-electrodes was placed on her tongue and connected to an external sensor, hard mounted on an hat she had to wear; the sensor, which replaces her vestibular apparatus, sends spatial coordinates data as pulses to the brain through her tongue; after some training times with this stimulus, the woman was able to get back her normal vestibular functions.

A second similar case involves a middle-aged man, with a strong passion for climbing; he faced mountains all over the world, also reaching the summit of Mount Everest, but what impresses here is that this man is blind, because of retinoschisis that caused him to be visually impaired at birth and com-pletely blind by age thirteen. He too decided to try the treatment with the BrainPort, where this time the sensor was constituted by an head-mounted camera, connected to a computer that translate images into electrical im-pulses, then sent to a tape on his tongue. The conducted tests were quite impressive: in a very little time the man was able to reach out and grab a rolling tennis ball; after about an hour he started to distinguish different shapes, while a few hours later he learned to avoid obstacles.

These two tests have highlighted once again the great ability of the brain in adapting and re-mapping its structure to account for new environmental condition or in response to serious injuries, taking a relative short time to learn new input stimuli, with a surprising efficacy.

6_{The vestibular system contributes to balance and to the spatial orientation in humans}

and in most of the animals.

(40)

(41)

Chapter 4 Modelling the cognitive

development

In the next pages will be presented in detail the problem addressed in this thesis, the methodology and the solutions adopted to solve it. The starting point of this work is the amygdala-thalamus-cortex model, as described in the previous chapter, and the IDRA architecture, an artificial system that aims at reproducing cognitive development abilities of human brain through the imitation of some features of the brain; it represents a cognitive middleware for the goal-generating behaviour.

Early experiments have shown convincing performances and a very in-teresting ability in the autonomous generation of new goals on the basis of the agent’s experience, and of the interaction with the environment. This architecture has proved to be a valid starting point for reaching the objective of realizing an intentional cognitive system, but has also emerged that it is still unripe and that there are a some missing key ingredients.

This thesis is focusing specifically on the lack of neuroplasticity in the architecture, with the objective of making its core structure more plastic and as much general as possible, by the use of a dynamic network, able to adapt to the new developed goals, to new incoming input data and to different environment’s conditions. Three main critical areas to be reviewed have been then highlighted; first the input stage, where some minor changes have been made to make the subsequent sections of the system independent from data format and input type, exactly as our brain does; second the Categorization Module, with the introduction of a novel abstraction algorithm and a new mechanism for the implementation of a neural coding of the input vector; and

(42)

Chapter4 - Modelling the cognitive development

last, the third area involved is the network of Intentional Modules, which has undergone a significant transformation from static to dynamic, thanks to the introduction of update criteria based on the fitness of the architecture.

4.1 Details of the Intentional Architecture

The Intentional Architecture implements the amygdala-thalamus-cortex model and aims at reproducing the cognitive development ability of human brain, allowing the agent to create new goals autonomously starting from some sim-ple innate criteria. Roughly speaking, the architecture consists of a network of modules, the Intentional Modules, plus a global module, the Phylogenetic Module, and has a structure similar to that observed in human brain between the three aforementioned brain areas.

Figure 4.1: Mapping of the Amygdala-Thalamus-Cortex model to Intentional Archi-tecture

Connections between IMs are realized by means of links, which are divided into vector link, from sensors to IMs and from IM to IM, and scalar link, permitted only between two IMs; these links can be both feedforward or feedback.

The network is subdivided in layers, that allow the architecture to ab-stract from raw input data: as information flows from sensors to output through the network the real meaning of data is lost, going from simple sensors reading to some sort of high level concepts.

The input of the Intentional Architecture is composed by various signals coming from the robot sensors, properly processed and filtered; the output is formed by two elements, a state vector and a relevant signal, which are coming directly from the last Intention Module in the network. The structure of the input stage is such that the architecture automatically performs sensor

(43)

4.1 - Details of the Intentional Architecture

integration and a first step in abstraction; just like the functioning of the human brain, input data are wrapped in a predefined data structure and filtered, so that the subsequent stages do not have to worry about number and type of sensors used and all data are treated in the same way. Moreover, when needed, vector links coming from different sources, as for example vision and proprioceptive data, can be easily fused together and sent to a single IM. When the computation is over, the output of the network is then passed to the motor system, which has the task of selecting and generating new movements pursuing actual agent’s goals, new or innate.

4.1.1 The Intentional Module

The basic building block of the Intentional Architecture is the Intentional Module, the key unit for the ability of developing new goals; this module has been developed able to adapt to varying input data, to learn to categorize incoming stimuli and to use acquired categories as the agent’s experience. It is composed by two inner units, the Ontogenetic Module and the Catego-rization Module, which represent respectively the thalamus and the cerebral cortex of human brain. The following figure 4.2 illustrates the Intentional Module structure.

Figure 4.2: An Intentional Module with its sub-modules, internal connections and input/output signals.

(44)

In detail, the Categorization Module acts as the architecture warehouse for memories of positive events or stimuli; it is driven by the Relevant Signal, an internal index of Intentional Module fitness which is higher when one or more goals are satisfied, and it selectively memorizes high level concepts extracted from vector input data. This module also returns as output a vector that represents how much actual state of the world is coherent to all of the previously saved categories; this output is a sort of neural coding of the input, with respect to the categories saved in the Categorization Module. All the details of the functioning of this module will be explained better in section 4.3.

The other unit involved is the Ontogenetic Module, which has a primary role in the development of new objectives; this module, as in human brain, is strictly connected to the Categorization Module and directly uses its output; the module output is computed as the maximum value between the products of input neural activities and an internal weight vector, as in equation (4.1),

where os is the Ontogenetic Signal, wi is the weight associated to neuron i

and si is the value of neuron i.

os = max

i

X

i

(wisi) (4.1)

The computed os value is then normalized between zero and one.

Internal weights this way become fundamental for the final behaviour of the system and so they are updated on every cycle during a learning phase; an hebbian learning function is used:

wi = wi+ η(hssi− wij2) (4.2)

In (4.2) η is the fixed learning rate, while hs is the Hebbian Signal, which

acts as a control factor for learning and could be for example the Relevant Signal itself. Finally a threshold is fixed, so that if a weight gets greater than this threshold its value is set to one.

What is important to underline here is the meaning of the Ontogenetic Signal; its values represents how much the actual system state satisfies one of the new developed goals: the higher the signal is, the more the new goals are met.

(45)

4.1 - Details of the Intentional Architecture

Figure 4.3: The ontogenetic signal is computed as the max value between the products of neural activities and an internal weight vector.

Every Intentional Module in the network admits two required input: a vector coming from sensors or from other IMs, and a scalar, sent from the global Phylogenetic Module; there is also a third optional input, the Relevant Signal of other IMs. The vector input is directly passed to the Categorization Module, which uses it for building its categories; the two scalar input together with the Ontogenetic Signal form an internal signal called Relevant Signal,

computed as the maximum of those signals. In equation (4.3) rs is the

computed Relevant Signal, osis the Ontogenetic Signal, psis the Phylogenetic

Signal and extr∗_s is the optional third scalar input.

rs = max(os, ps, extrs∗) (4.3)

This Relevant Signal is used for driving the acquisition of new categories: when its value is greater than a fixed threshold and the new vectorial input is enough different from recorded stimuli, a new category is sampled; actually this mechanism is quite more complicated and will be explained in detail in section 4.3.

As output each Intentional Module returns two components, a vector and a scalar; the first one comes from the output of the Categorization Module, the second is the Relevant Signal.

4.1.2 The Global Phylogenetic Module

Beside the network of IMs, there is the Global Phylogenetic Module, a unique module which represents the amygdala of the system; it receives all the sen-29

(46)

sor’s readings and, based on a series of rules, returns a scalar value, namely the Phylogenetic Signal; this module is highly connected to Intentional Ar-chitecture, and when a new output is ready its value is spread to all the IMs

Each of the rules inside this module represents an instinctive function and processes only a certain group of input, according to the type associated with it; we could have for example a video instinct stating that the agent likes coloured objects, or an audio one that tells it is afraid of some kind of sound. The trend of the Phylogenetic Signal shows how much the actual state is significant according to these a priori stored criteria and is such that it is low when no instinct is activated and it gets high as soon as one or more criteria are met.

4.1.3 The input of the Intentional Architecture

The input of the Intentional Architecture is composed by data coming from the agent’s sensors; obviously they have different structures, according to the type of sensor used: an RGB image coming from a camera is for example really different from the binary reading of a touch sensor. One of the main feature of the Intentional Architecture is its independence from type and number of sensors used and this is achieved thanks to a stage in which data are properly formatted and packed in a structure identical for all. Right after this stage, data are sent to the filter layer; there exists a lot of possible filters, each one dealing with a particular sensor type (video, audio, tactile, etc.), and their function is to modify input data or extract some kind of meaningful information; an example of a widely used filter is the log-polar filter, which implements the log-polar transform on Cartesian images and reproduces the modality with which human eye sees the environment (see figure 4.4).

Thanks to the high modularity of the system, if necessary new other filters can easily be implemented, with the only foresight to respect dimensions and structure of the input and output data.

4.1.4 Motor system, movement generation and

selec-tion

The output of the Intentional Architecture, composed by the output of the last Intentional Module in the network, when ready is sent to the motor

A dynamic network model simulating brain plasticity for a robotic bioinspired intentional architecture : implementation and test

POLITECNICO DI MILANO