Tesi di Laura Magistrale
“Simultaneous and proportional EMG dexterous
finger control for rehabilitation exoskeleton
hand: a synergistic approach”
Master of Science in Embedded Computing Systems
Academic Year: 2016/2017
Relatore: Proff. Antonio Frisoli Correlatore: Dr. Michele Barsotti Candidato: Luis Peláez Murciego
ABSTRACT
Synergy model based EMG control has been demonstrated as a promising technology for EMG applications such as clinic rehabilitation or prosthetic control. This study proposes a simultaneous and proportional EMG dexterous finger control system based on the combination of both main approaches in myoelectric control: regression and classification. In this study, 4 DoFs are successfully decoded by using the non-negative matrix factorization (NMF) algorithm to estimate the subject-specific synergies. A Bayesian estimator filtering is used as EMG feature extraction. The first experiment is conducted using the publicly available NinaPro database. Then, a second protocol is proposed by using the exoskeleton hand developed on the PERCRO laboratory. The data recorded with the exoskeleton includes the combination of 2, 3 and 4 fingers simultaneously. A synergy-based classifier is then proposed as an extra control effort to successfully remove false activations. The incorporation of the classification stage provides the benefits of the DoF identification without resign to the advantages of a natural and simultaneous proportional control granted by the regression models.
1 – HUMAN BODY ANATOMY ... 5
1.1 – Muscles description ... 5
1.2 – Electromyography principles ... 11
2 – MYOELECTRIC CONTROL ... 15
2.1 – Academic Vs commertial SoA ... 16
2.2 – Pattern recognition ... 18
2.3 – Regression model ... 23
3 – CLINIC REHABILITATION ... 26
3.1 – Robot-assisted rehabilitation ... 26
3.2 – PERCRO exoskeleton hand ... 28
4 – SYNERGY BASED CONTROL ... 30
4.1 – Bayesian estimator ... 32
4.2 – Synergy estimation ... 37
4.3 – Offline control ... 43
5 – ACQUISITION PROTOCOLS ... 46
5.1 – The NinaPro Database ... 47
5.2 – Exoskeleton hand protocol ... 50
6 – SYNEGY BASED CLASSIFIER ... 60
6.1 – Implementation ... 61
7 – RESULTS ... 63
7.1 – NinaPro experiment results ... 63
7.2 – Exoskeleton hand experiment results ... 68
7.3 – Classifier results ... 74
8 – CONCLUSSIONS AND FUTURE WORK ... 78
REFERENCES ... 80
1 - HUMAN BODY ANATOMY
Among all the splendid and unique integral parts that constitute the human body, let us be focus in one particular element who will be fundamental for this study: the body muscles. Since there is already plenty of well-documented resources about human anatomy only the essentials for a complete understanding of this study will be discussed during this report. This chapter includes a briefly mention to muscle types, basic muscle functioning, skeletal muscles description and more detailed hand/forearm anatomy including its kinematics.
1.1 Muscles description
The human muscular system is composed by three major muscle types including smooth muscle, mainly present in the walls of the organs and blood vessels as well as in the eyes or skin; cardiac muscle, an involuntary muscle found in the walls of the heart; and skeletal muscle, mostly of it attached to the bones, it is the responsible for skeletal movements. The last type is composed by striated muscle fibers that can act independently or in neighbor groups under voluntary control through the central nervous system (CNS). Skeletal muscle is the responsible for any physical movement, it will be analyzed deeper in order to explain the basic principles of its behavior in which this study is actually sustained.
Figure 1.1. Human body with visible skeletal muscles. [1]
There are more than 600 muscles in the body, which together account for around 40% of the body weight. The function of skeletal muscle is to contract to move parts of the body closer to the bone that the muscle is attached to. Most skeletal muscles are attached to two bones through tendons. The place on the stationary bone that is connected via tendons to the muscle is called the origin and the place on the moving bone that is connected to the muscle via tendons is called the insertion. Normally the origin is more stable during the contraction and has greater mass than the insertion.
For example, in the biceps brachii the origin is close to the shoulder and the insertion is close to the elbow. The fleshy part of the muscle in between the tendons that does the actual contraction is named the belly.
Rarely the muscles work by themselves to achieve the body movements, more often they work in groups to produce precise movements. Two individuals may use the same muscle group but with different proportion to reproduce the same move. The synergy theory characterized this physiological behavior. This concept is described later in this section. For each movement there is always an antagonist muscle that produces the opposite effect of the agonist muscle in the same bone. A clear example of this concept is when the biceps brachii flexes the arm at the elbow so the triceps brachii extends the arm at the elbow at the same time. They are so-called antagonistic pairs, to contract one, the other relaxes. Antagonistic pairs include abductor-adductor and flexor-extensor. The last pair consist of an extensor muscle which open the angle between two bones, and a flexor muscle which does the opposite by decreasing the angle between two bones.
Although this is a common behavior, not always the muscles work this way. For example, a co-activation occurs when the antagonistic muscle is slightly activated during the action of an agonistic muscle. It happens naturally and it should not be a problem unless it is excessive and disturbs the motion. This effect will be matter of interest for prosthetic control as it will be later commented.
Figure 1.2 Skeletal muscle types. [1]
Muscle fibers run in different directions depending of the muscle (Figure 1.2). There are three main muscle types described according to the fibers direction: fusiform, fibers run parallel to the length of the muscle (biceps brachii); unipennate, fibers run parallel of only one side of the muscle (extensor digitorum); and bipennate, the strongest one among the three due a larger physiological cross-sectional area (PCSA), muscles consist of two rows of oblique muscle fibers (rectus femoris).
It worth noting that the PCSA is the cross-section perpendicular to the muscle fibers and the anatomical cross-sectional area (ACSA) is the cross-section perpendicular to the
muscle longitudinal axis, therefore the PCSA and the ACSA will coincide only for non-pennate muscle, which has the fibers parallel to the longitudinal axis. Since muscle force is determined by the PCSA, then the pennate muscles can pack more fibers in parallel allowing the muscle to produce more force.
It is important to consider the direction of the fibers during the acquisition stage for an accurate data collection. A string of electrodes should be located along the direction of the fibers for proper signal recording.
TABLE I – Forearm muscles
Posterior compartment Anterior compartment
superficial brachioradialis superficial flexor carpi radialis superficial extensor carpi radialis longus superficial palmaris longus superficial extensor carpi radialis brevis superficial flexor carpi ulnaris superficial extensor carpi ulnaris superficial pronator teres
superficial anconeus superficial flexor digitorum superficialis intermediate extensor digitorum deep flexor digitorum profundus intermediate extensor digiti minimi deep flexor pollicis longus deep abductor pollicis longus deep pronator quadratus deep extensor pollicis longus
deep extensor pollicis brevis deep extensor indicis deep supinator
Forearm anatomy
The upper limb is divided into two segments: the arm, which technically extends from the shoulder to the elbow and the forearm, which is the region between the elbow and the wrist. This last segment includes all the muscles related to the hand gestures. The are more than 20 muscles involved in wrist, hand, and finger movements located along the forearm. The forearm contains two long bones, the radius and the ulna. The muscle contained in the forearm includes the flexors and extensors for the digits, a flexor of the elbow (brachioradialis) and pronators and supinators for the wrist functioning. In figure 1.3 it can be seen an illustration of the cross sectional view of the forearm that can be divided into two fascial compartments: posterior (dorsal) and anterior (ventral).
Figure 1.3: Cross sectional view of the forearm (left hand supinated, front view). Anterior compartment (dark grey). Posterior compartment (light grey). Muscles in red color are mainly responsible for finger moves, green color for
thumb move and blue color for wrist movement.
The posterior compartment contains 12 muscle mainly responsible for the extension of the wrist and digits and supination of the hand. The anterior compartment contains principally muscles involved in digit flexion and hand pronation. The muscles contained in both compartments can be themselves divided in two groups in function of its internal location: superficial or deep. A relation of the muscles comprehended in each group are listed in Table I.
Hand anatomy
The human hand has 27 bones, including the 14 phalanges of the fingers (distal, middle and proximal for the four fingers and proximal and distal for the thumb). The skeletal muscles responsible of the hand movement are divided in two groups: extrinsic and intrinsic muscle group. The extrinsic group contains the muscles located in the forearm which includes the flexors and extensors. The intrinsic group is a small group of muscles that are located in the hand itself (fig. 1.5). More specifically, these muscles are the responsible of abduction/opposition of the thumb (abductor pollicis brevis, flexor pollicis brevis and opponens pollicis) and the responsible of the abduction/opposition of the little finger (abductor digiti minimi, flexor digiti minimi brevis and opponens digiti minimi). This peculiarity will limit the possible moves reproduced by a muscular-controlled robotic device in case of the missing limb (i.e. hand amputees will be unable to naturally control the abduction of the thumb or little finger using a muscle-activated prosthetic hand).
Figure 1.5 Intrinsic skeletal muscles of the hand. From abductor digiti minimi on the top (little finger) to the abductor pollicis brevis on the bottom (thumb).
The extrinsic muscle groups are the long flexors and extensors. The fingers have two long flexors, the deep flexor attached to the distal phalange and the superficial flexor attached to the middle phalange. The extensors are located in the back of the forearm and are connected to the dorsum of fingers in a more complex way. Regarding the size, the extensor muscles are larger than the extensors. Nevertheless, extensors are mainly superficial muscles. It means that for an extension move, the electrical signals recorded from the surface of the forearm will be stronger than the signals of a flexing move when a similar amount of force is exerted.
Muscle activation
Muscle fibers are the single contractile unit within the muscle. A muscle can contain up to thousands of fibers, each of them innervated by an axiom terminal of a motor neuron. A single motor neuron may innervate many fibers, but only one motor neuron can innervate the same fiber. This union is called motor unit which is the responsible for the muscle contraction. A brief insight into the physiological process of a contraction is provided following as an introduction of the electromyography (EMG) technique, which it will be later discussed.
Figure 1.6 Action potential of the cell membrane.
Skeletal muscle fiber contains two proteins that are the responsible for the contraction of the fiber. The two proteins (troponin and tropomyosin) reacts to a neurotransmitter released by the motor unit as a result of a voluntary command sent by the brain. The chemical reaction originated by the neurotransmitter causes a depolarization of the cell membrane. During the depolarization the cell suffers a change in the electric charge that goes from resting negative voltage (around −70𝑚𝑉) to a less negative value. If the effect of the depolarization is large enough, the potential difference in the cell membrane became briefly positive, and therefore the cell became positive charged (see fig. 1.6). This shift, called Motor Fiber Action Potential (MFAP), typically last around 2-4 ms. The combination of all the muscles fibers of a single motor unit is called Motor Unit Action Potential (MUAP). The electrical characteristic of the signals depends of the muscle under observance, but the usual range laid between 50𝑢𝑉 and 30𝑚𝑉.
1.2 Electromyography principles
EMG definition
Electromyography (EMG) is the technique that measure the electrical potentials (MUAPS) generated by the muscle cells. This signals can be used for biomedical applications and have been from interest in clinical applications due to the ability to provide an important source of information for clinical diagnosis and neuromuscular disorders.
The development of EMG came from long with Francesco Redi's in 1666. More recent advances have been done in the last century with Marey in 1890 who introduce the term of electromyography after discover it was possible to record electrical activity during voluntary contractions [1]. Over the last decades EMG researches have increased significantly the understanding of this signals thanks to the development of more reliable electrodes and the performance increase in operational amplifiers that allowed them to work in the desired range of microvolts.
Two different techniques have evolved in parallel: intramuscular EMG and surface EMG (sEMG), being the last one the most common and desired approach due the non-invasive method of acquisition. Intramuscular EMG, which uses intramuscular needle electrodes to acquire the muscle signals, is commonly relegated to deep muscles analysis only.
Figure 1.7: Diagram of surface EMG acquisition and decomposition in MUAPs [10].
EMG provides a complex signal composed by trains of MUAPs. The quality of the signal comes determined by the signal-to-noise ratio which is the ratio of the energy of EMG
signal to the energy of the noise signal [2]. The following equation shows a simple model of EMG signal: 𝑥(𝑛) = ℎ(𝑟)𝑒(𝑛 − 𝑟) 234 567 + 𝑤(𝑛) (1.1)
where 𝑥(𝑛) is the modelled EMG signal, 𝑒(𝑛) represents the firing impulses, ℎ(𝑟) represents the MUAP, 𝑤(𝑛) is a zero mean additive white Gaussian noise which is independent of 𝑒(𝑛) that represents the system noise and 𝑁 is the number of motor unit firings.
Noise rejection
One of the main concerns in EMG acquisition is to reject the external noise that affect the recording of the signal [3]. The most common EMG noise sources are described following together with different techniques for diminishing its influence. Some of them have been implemented for this project. In Chapter 5 is provided a detailed description of the EMG acquisition protocol used.
- Electrode noise: Independently of the technique employed (intramuscular of surface EMG), the electrode generates a half-cell potential at the electrode-electrolyte interface which vary in function of the material of the electrode. This potential can be considerable and may disturb the original signal. In order to minimize it all the electrodes employed during the recording should be made of the same material. The electrode-electrolyte interface of silver (Ag) is stabilized by coating the electrodes with a Silver-Chloride (AgCl) layer. Therefore, Ag-AgCl are the most widely used electrodes for surface electromyography [4].
In the case of surface EMG, since the human skin impedance is high, the potential derive from the skin surface is considerably affected. A high electrode-skin impedance may lead to a very reduce signal amplitude and may be cause of noise interferences during the recording phase. To reduce the skin-electrode impedance a conductive paste-coupled electrode may be used instead of dry electrodes. Usually a skin cleaner is also applied before the recording in order to reduce the skin impedance. Also active electrodes may be used instead of passive electrodes [5], with the advantage of avoiding all the skin conditioning for the benefit of the patient. Active electrodes are mounted onto the operational amplifier which provides high input impedances (in the order of 𝐺Ω) and low output impedance (less than 1 Ω).
- Motion artifacts: Motion artifacts causes disturbance in the data. Two main sources of motion artifacts are electrode cable disturbance and mechanical disturbance of the
electrode. If the cable is unshielded the ambient magnetic or electric fields may generate current flowing through it due to its intrinsic capacitance. The magnitude of the noise may be equal or superior to the EMG itself and it will be related to the electrode-skin impedance. Therefore, reducing the electrode-skin impedance as mentioned before will also reduce the cable interferences. Shielded cables are a recommended option to reduce ambient interferences, however, shielded cables should be handle carefully since the movement or twist of the cable it is a source of interference itself. Using active electrodes is an effective solution for this problem [6]
Mechanical disturbance of the electrode occurs when the skin is deformed under the electrode. Preparing the skin with a conductive gel between the electrode and the skin will minimize this effect. The power density of this motion artifact is usually under 20Hz, therefore a pre-processing filtering can be done by implementing a high pass filter with a cut-off frequency of around 10-20 Hz. Furthermore, two differential electrodes lying around an innervation zone may lead to another kind of motion artifact. Different methods have been implemented to identify the innervation zone [7] that can be used to avoid such location during the electrodes placement.
- Power line interference: The order of magnitude of power lines interference may be larger than the EMG itself. The frequency of this interferences corresponds to the AC power supply (60Hz in North America or 50Hz in Europe) and its harmonics. The electrode and the patient are both capacitively coupled with the electric field. It creates a current flow through the electrode impedance that results in a potential at the electrode. This interference potential is sense by the amplifier. With bipolar recording electrodes the common mode voltage caused by the interference current may be attenuated, or in an ideal case, completely removed. The magnetic field can be reduced by keeping the cables as short as possible and twisted around themselves. However, these methods may not completely remove the power line interference and/or its harmonics. In that case a pre-processing online/offline filtering may be necessary.
Muscle synergies
Synergy in Greek means “work together”. This words implies that a synergy always does something (work) and share the activity (together).
The central nervous system may implement a strategy to simplify the control of multiple degrees of freedom of the musculoskeletal system. In the last few years, recent studies have led to the hypothesis that CNS uses a set of muscle synergies to co-activate different muscles by a single neural command [8]. One common approach to detect muscle synergies is to measure the EMG signal from the muscles during certain activity so that a muscular activation pattern can be identified [9]. It has been found that a small number
of synergies could explain a large fraction of the variation in the muscle patterns and that the synergies extracted from the same behavior in different individuals were usually similar [8]. Nevertheless, these studies have been conducted by analyzing coarse behaviors in small animals, such as jumping, walking or swimming [57]. More precise behaviors, like human dexterous finger moves, have not been investigated yet widely to determine whether or not the synergies are commonly shared between subjects. Part of this study is to theorize about the existence of subject-specific synergies related to the finger moves. A deep analysis of EMG muscle synergies extraction algorithms is provided in Chapter 4.
2 - MYOELECTRIC CONTROL
The human hand is a complex organ capable of both gross grasp and fine motor skills. Despite many successful high-level skeletal control techniques, achieving realistic hand motion control remains tedious and challenging [11].
Surface EMG provides muscular information through electrodes placed on the skin. This non-invasive method allows to extract control signals that are directly related with the muscle activity which can be used to manage external devices such as limb prostheses medical rehabilitation robots.
Myoelectric control has been in development since the decade of the 1940s, achieving great level of accuracy in the laboratories, specially during the recent years. However, all clinical prostheses available nowadays are still based on the earliest control strategies. The gap between academic and commercial applications is mainly due to the inability to replicate the laboratory results in an uncontrolled environment. The most advanced algorithms and strategies has still facing the big challenge that represents to simulate the natural daily activities that affect the quality of the results and, at the same time, to provide an intuitive, easy to train, robust real-time control.
In section 2.1 an overview of the state-of-the-art (SoA) in myoelectric control is provided, discussing both commercial and academic systems, and analyzing the challenges, benefits and contras of each approach. Sections 2.2 and 2.3 provides wide description of the two main approaches for myoelectric control: pattern recognition and regression methods respectively. Figure 2.1 illustrate the scheme of myoelectric control SoA. Since the purpose of this study is to discuss a simultaneous finger control system, all the examples mentioned in this report will be always referenced to prosthetic hand/fingers control.
2.1 Academic vs Commercial
More than 60% of the EMG-control prosthesis users stop using it after a period of 6 months due to the complexity of the control system. As it will be shown, reliability has been the main concern for the industry during the designing process of clinical prostheses, but building a robust system has lead to a devices based on a non-intuitive control with a difficult learning curve. The side effect of this robustness systems is the lack of intuitiveness which, in the worst case scenario, will lead the users to abandon the use of the device, as it has been shown by the statistics [55].
Earlier commercial systems are based on an on/off control strategy, which reads the EMG activity of different muscle groups to performance a movement [12]. For one muscle group only one channel is used which consists in a pair of bipolar electrodes placed on the target muscles. A threshold-based control is then implemented to execute an open gesture when the EMG signal is above a certain small value, that is, when a slight contraction is performed by the user; and a close gesture of the prosthetic limb occurs when the EMG signal is above a greater threshold (i.e. when a strong contraction is performed by the user) and hold it for a small period of time (figure 2.1). When the signal is under the lowest threshold the hand must remain in rest position.
Worth noting the incongruence that this control scheme may lead to, due to the unnatural way of control since the user has to contract the same muscles in order to perform a pair of opposite gestures (e.g. in order to open the robotic hand, the user has to flex the wrist for a few seconds). While this is a coarse example, the idea is to expose the frustration that this type of control may cause to an unexperienced user.
The solution in this case is to add a new channel on the opposite muscles to perform the gestures in a more natural way, for example by mapping the wrist flexor and extensor with the open and close gestures respectively. The direct control obtained is proportional to the EMG amplitude and each channel will control a physiological degree of freedom (DoF) of the hand (open/close). However, complexity increases when new gestures are added to the system. Ideally direct proportional control could perform more precise gestures besides from open/close by targeting different muscles and mapping them with the new gestures. However, the existing interferences caused by the activation of neighbor muscles produce crosstalk of the resulting EMG signal, which reduce the sensitivity of the electrodes. Therefore, the number of gestures that may be implemented for this scheme is limited by the number of individual muscle groups that can be independently controllable [13]. An option to avoid crosstalk is to implement intramuscular EMG, since this method inserts the electrode directly into the muscle the interferences will not be significant. As mention before, the invasive approach of this method has limited its implementation on human-machine interfaces in benefit of surface EMG.
Figure 2.2. Basic EMG control approach operating with two thresholds.
Multifunctional devices are difficult to implement following direct proportional control. Another method to increment the number of functions is using a mix of both approaches described above by implementing a multichannel control system with two different thresholds for each channel. This allows each EMG channel to execute different functions of the prostheses with a maximum number of functions per channel being three (inactive, low activation and strong activation). However, the complexity of the control increases and eventually the system will become less intuitive and more difficult to use.
An example of a common commercial application is shown in figure 2.3. In this case the system is controlled by two EMG channels placed on opposite muscles (flexor/extensor). The prostheses can execute a total of 2 DoFs separated in two different modes: grasping and rotating. When the mode grasping is selected two gestures are available, open and close. For the rotating mode the two gestures available are rotate left and rotate right. In order to change the mode a co-activation is performed. As a matter of clarity, a flexor/extensor co-activation may be performed by opening the hand in a quick fashion at maximum extension. Although the robustness of this approach, the number of possible DoFs is limited and the usability will significantly decrease with the number of co-activation needed to switch within the modes.
Figure 2.3: Example of two muscles control. Each muscle is activated by performing different gestures. From left to right: flexion (electrode 1), co-‐activation (both electrodes), extension (electrode 2). Next figure illustrates an example
of this type of control using a robotic hand with two DoFs.
Figure 2.3: Two electrodes controls each DoF of the robotic hand (e.g. wrist left/right and hand open/close). The green color indicates the hand DoF is selected. In this mode, the hand open when electrode 1 is active and close when electrode 2 is active. If the two electrodes are activated at the same time the control switches the DoF. Red color indicates the wrist DoF is selected. When the system is in this mode, electrode 1 activation rotates the wrist to
the right and the activation of electrode 2 to the left.
2.2 Pattern recognition
New pattern recognition control strategies have been developed in order to overcome the limitations of multifunctional direct proportional control. The idea is not new, since the last 60 years, academics and researchers have tested and developed algorithms based on pattern recognition.
This method classifies different muscle activations patterns for different tasks (gestures). It is based on the assumption that the EMG signal produced can be characterized by a set of features. Ideally, the set of features obtained should be differentiable between different
tasks and reproducible among trials of the same task. In order to extract the features, the raw signal is first pass-band filtered and then windowed for further processing. The length of the window is determined by real-time constraints, but it have been shown that a length larger than 200ms affect negatively to the user's performance [14]. The advantages of this algorithms include increasing the number of gestures, however, the gestures only can be executed sequentially, this means two different tasks cannot be performed simultaneously (i.e. only one DoF can active at a time). The robustness of the control scheme it depends of the EMG feature and the training set that must be representative of the full set of inputs using during the testing phase. Pattern recognition algorithms consist essentially in three succeeding steps that includes feature extraction, feature space reduction and classification. A further discuss of each step is provided following.
Feature extraction
Feature extraction is a method to extract the useful information that is hidden in EMG signals. Optimal set of features should satisfy three properties including class separability, robustness and complexity. Deeper analysis comparison between features and scatter plots are common techniques employed to avoid redundancy [15]. The idea is to remove those features which contain similar information between themselves.
The features extracted from the analysis of an EMG signal can be divided into two main groups: time-domain and frequency-domain. The most commonly used feature for each domain are briefly described following.
- Time domain features:
Time domain (TD) features set was popularized by Hudgins et al (1993) [16] which is described following. Despite the fact that a wide variety of feature extraction has been presented in the literature for pattern recognition, most recent studies has shown the Hudgins TD set in combination with the autoregressive (AR) features can achieve high classification accuracy [17, 14]. This set includes the mean absolute value (MAV), the number of zero crossing (ZC), the number of slope sign changes (SSC) and the waveform length (WL). Other popular features such is wavelet transform (WT) has been shown also to provide great accuracy in pattern classification [18]. The statistics of each feature described bellow are computed for each window-time of the input channel and then concatenated forming a feature vector used to represent the myoelectric pattern.
1) Mean absolute value (MAV)
An estimate of the mean absolute value of the EMG signal, in segment i, which is N samples in length it will be defined as:
𝑀𝐴𝑉 = 1 𝑁 |𝑥A| 2 A64 (2.1) 2) Zero crossing (ZC):
A simple measure of frequency information obtained by counting the number of times the EMG signal cross the zero value. A threshold must be included to reduce background noise due to the low voltage fluctuations. Is defined as:
𝑀𝐴𝑉 = [𝑠𝑔𝑛(𝑥A∗ 234 A64 𝑥A34) ∩ |𝑥A∗ 𝑥AG4| ≥ 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑] 𝑠𝑔𝑛 𝑥 = 1, 𝑖𝑓 𝑥 ≥ 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑 0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 (2.2)
3) Slope sign changes (SSC):
Another method to represent frequency information of EMG signals. It counts the number of times the slope of the signal changes the sign. Once again, a threshold is required to avoid background noise. Is defined as:
𝑆𝑆𝐶 = [𝑓(𝑥A − 234 A6S 𝑥A34) ∗ |𝑥A − 𝑥AG4| ≥ 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑] 𝑓 𝑥 = 1,0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒𝑖𝑓 𝑥 ≥ 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑 (2.3) 4) Waveform length (WL):
This feature provides information on the EMG signal complexity. It is the cumulative length of the EMG waveform over the window time. Is defined as:
𝑊𝐿 = 1
𝑁 |𝑥AG4− 𝑥A|
234
A64
(2.4)
5) Root mean square (RMS):
This feature is also popular for the EMG signal analysis. It is modeled as amplitude modulated Gaussian random process whose relates to constant force and non-fatiguing contraction. It is also similar to standard deviation method. The mathematical definition of RMS feature can be expressed as:
𝑅𝑀𝑆 = 1
𝑁 𝑥AS 2
A64
(2.5)
6) Auto-regressive coefficients (AR):
Autoregressive model of order 𝑃 models an EMG sample as a linear combination of 𝑃 previous samples plus a noise correction term 𝑤A. Coefficients of AR model can be used as a feature vector and are found by minimizing the mean square error between the predicted sample and the actual sample value. The calculation is defined as:
𝑥A = 𝑎Y𝑥A3Y+ 𝑤A Z
Y64
(2.6)
where 𝑃 is the order of the AR model. It has been shown how using 4th or 6th [15] order
AR model provides a good performance in pattern classification [14, 19].
- Frequency domain features
Features based on frequency domain does not provide good results for EMG pattern classification. However, they are mostly used to study muscle fatigue and motion unit analysis. A brief description of two mainly used features based on a power spectral density which includes mean frequency and median frequency is given following. 1) Mean frequency (MNF):
Is the average frequency, which is calculated as follows:
𝑀𝑁𝐹 = 𝑓\𝑃\ ] \64 𝑃\ ] \64 (2.7)
Where 𝑓\ is the frequency spectrum at frequency bin 𝑗, 𝑃\is the EMG power spectrum and 𝑀 is the length of the frequency bin.
2) Median frequency (MDF):
Is the frequency at which the spectrum is divided into two equal-amplitude regions. Is calculated as follows: 𝑃\ ]_` \64 = 𝑃\ = ] \6]_` 1 2 𝑃\ ] \64 (2.8)
Feature space reduction
A multichannel approach is usually utilized for EMG recording to capture information from different muscles. High number of channels may cause the feature vector to became very large and computationally inefficient to work with. Therefore, a dimensionality reduction of the feature space is often required. The reduction method hast to be able to select a small set of feature that can best discriminate between classes. Feature set dimensionality can be reduced by using feature projection methods. The two most well known methods are Linear Discriminant Analysis (LDA) and Principal Component Analysis (PCA) [20]. In general terms PCA is a technique used to transform a number of correlated variables into a significant smaller number of uncorrelated variables called principal components. A variation of the classical LDA is the Fuzzy LDA (FLDA), that determines the maximum separation of fuzzy groups in real space. Khushaba el at [21] proposed a new variation of the FLDA method referred as Orthogonal Fuzzy Neighborhood Discriminant Analysis (OFNDA), which provides great classification accuracy using a very small number of features.
Classification
Linear Discriminant Analysis is used also as a classifier. However, linear classifiers perform significantly worse when data is not clearly separable. Due to its computational simplicity LDA is often used for real-time applications [22]. A recent study in finger classification control [19] achieved great accuracy by using a combination of TD-AR feature extraction, OFNDA for feature reduction and LDA for classification.
One of the most common classification methods used in the literature is the supported vector machine (SVM). SVM guarantee a globally optimal separation of classes and achieve nonlinear classification through the use of kernel functions.
Artificial neural networks (ANN) including MLP and more complex networks have been shown more flexible and better suited for classification techniques compared to SVM [23, 24]. The outputs of an ANN can be also represented as proportional or continuous variables rather than on/off as in SVM. Moreover, the low computational load makes them suitable for real-time operations.
2.3 Regression model
As mentioned before, pattern recognition applications have to be controlled sequentially requiring switching techniques such as co-contractions to control multiple number of DoF’s. Advances research has extended the classification into more than one class at a time [25]. However, these approaches still limit the movement since both gestures has to be performance at the same velocity, making it unable to control each move independently if two functions are activated simultaneously. Natural movements can only be achieved by independent and proportional control of the related DoFs.
The main difference to classification is that the regression method does not choose a particular class but provides a continuous output estimated for each DoF. This approach allows to control proportional and simultaneously more than DoF independently. A major challenge of regression training is to obtain the force data for training in the absence of the missing limb. Some research [26] investigated a bilateral training for associating features of the surface EMG recorded from one upper limb to the force produced by the contralateral limb. This method requires only to measure the forces from one limb and has the potential to be implemented in clinical applications for simultaneous and proportional control of multiples DoFs.
Most studies in myoelectric control uses the variance or the lowpass-filtered EMG. In [27] has been demonstrate that the variance is highly non-linear with respect to the wrist angle but is possible to solve this problem by applying simple transformations in the feature space. This allows to use linear methods such as linear regression and mixture of linear experts. A brief description of the linear regression model is provided following
Linear regression model
The variable 𝑌 contains the labeled training data (e.g. force or position). The goal of all the regression methods is to find a mapping 𝑌 = 𝑓(𝑋) , where 𝑌 is an estimation of 𝑌 and the matrix 𝑋 is the set of the EMG features. The mapping function is linear
𝑌 = 𝑊d𝑋 + 𝑤
7 (2.9)
where 𝑊dcontains the weight vectors and 𝑤
7 is the bias that can compensate the offset.
The mean square error solution is obtaining by minimizing the following error function
𝑒𝑟𝑟(𝑤e) =1
2 [
d
𝑦 e 𝑡 − (𝑦 e d𝑥 𝑡 )]S+𝜆
2 𝑤 e d𝑤 e (2.10) The closed form solution is given by
𝑊 = (𝑋𝑋d+ 𝜆𝐼)34𝑋𝑌d (2.11)
Where 𝐼 is the identity matrix and 𝜆 is the regularization constant optimized in a nested cross-validation.
Other non-linear methods such as multilayer perceptron (MLP) or kernel ridge regression are powerful solutions for non-linear problems. MLP produce complex nonlinear mapping using hidden nodes with nonlinear functions such as sigmoidal function at each node. Kernel Ridge Regression is another simple nonlinear regression method. The main difference with the Linear Regression is that the error function is not minimized in the input space of the data. Instead, the data in 𝑋 is mapped through a nonlinear mapping into a kernel space. The problem is solved through the use of kernel functions. It has been shown that predictions using the log-variance as EMG feature provides better results for wrist angles estimation [27].
Muscle synergies are also influential in myoelectric control schemes due to the fact that sEMG directly encode muscle activity. Synergy features extract information from the EMG signal to represent the underlying muscle coordination principles that represent the movements [59]. Nevertheless, the inability of surface EMG to measure the muscle activations consistently has been well documented. As indicated before, physical factors such as muscle depth, skin cleanness, electrodes shift, intensity of muscle contractions and cross-talk from nearby muscles all add variability to the EMG measurement. When recording EMG for synergy extraction, all this effects are magnified.
Synergistic control is gaining more attention during the last years and is haven been demonstrated to provide robust results with a short training phase [28]. The most common method for synergy estimation is the non-negative matrix factorization (NMF). Most recently, NMF has been used for 2DoF and 3DoF simultaneous and proportional control [29]. The purpose of this study is to successfully control up to 4DoF simultaneous and proportionally by using a semi-supervised NMF algorithm fully described in Chapter 4.
3 - CLINIC REHABILITATION
3.1 Robot-assisted rehabilitation
The number of patients with a mobility impairment in a certain part of the body caused by a stroke is increasing in step with the aging of the population. In fact, stroke is one of the main diseases that leads to high disability according to the World Health Organization. Stroke patients who completed a long term rehabilitation training for the upper limb were usually able to recover some of the mobility at the elbow and shoulder joints. However, the recovery for the hand and wrist movements such as opening and close, were considerably limited [30]. Unfortunately, there is a relative shortage of therapist available and therefore it may be difficult for patients to get access to a full proper rehabilitation program. A solution to this problem would come with the possibility for the patient to carry out the rehabilitation exercises by themselves.
Robot-aided training exercises may provide an improve in the rehabilitation performance while reducing the workload of the therapists. They have been proved to be effective in assisting the therapist to provide a safe and intensive rehabilitation programs [31, 58]. Over the last 10 years, numerous robotics devices for hand function with various levels of complexity have been developed [32]. The results of said studies indicate that robot-assisted rehabilitation reduce motor impairment and fairly improve the functionality of the affected hand and arm [33].
The human hand is a complex mechanism that has a wide range of degree-of-freedom. Hand motion-assisted devices are limited in size and rehabilitation techniques are somehow more difficult compared with other limbs. The dexterity of the hand is the result of a complex coordinated response of multiple muscles located on the hand and the forearm. The control signals are generated in the cortex and then delivered to the muscles via the CNS pathways [34]. The electric response of the neural commands can be decoded by using some of the EMG techniques presented in Chapter 2. An exoskeleton training device is specially designed for stroke patients to provide training on the impaired hand by using an exoskeleton hand driven by their own muscle signals. By measuring the EMG signal from the impaired hand, the device detects the patient’s intention and assist the move of the hand in consequence.
Figure 3.1: Bilateral EMG training using an exoskeleton hand.
Different training strategies have been successfully proposed based on both unimanual or bimanual training (figure 3.1). Bimanual training requires the two hands to cooperate to accomplish the aimed function. It has been shown how the simultaneous movement of both limbs helps the neuro-muscular system to regain some stability and improve the use of the impaired limb [35].
Most of the hand-assist robots mentioned in the literature are focus in less dexterous moves of the hand such as grasping patterns and/or wrist rotations. Recently, with the development of more advanced exoskeleton models, the researches start investigating classification and regression methods for finger identification in order to better mimic the human hand dexterity [36, 37]. Most of this studies addressed the classification problem without proportional information (i.e. force exerted or position). The discrete output of the pattern recognition approach limits the clinical application of such approaches. Forces and position can be estimated from EMG signals by using regression methods such as the method described in the next chapter.
An advanced rehabilitation exoskeleton hand designed by PERCRO laboratory (Scuola Superior Sant’Anna, Italy) is presented in the following section. The device allows for individual finger control and is provided with 5 linear force sensors used to measure the force level exerted by each finger. A described protocol EMG acquisition using this hand is provided in Chapter 5.
3.2 The PERCRO hand exoskeleton
Figure 3.2: Subject wearing the PERCRO exoskeleton hand
From the kinematic point of view, the human fingers (with the exception of thumb) has 4 DoF’s: 3 flexion/extension joints and 1 abduction/adduction. Due to the high cost and complex design, there are not practical devices that allows to control each finger joint individually. One alternative to implement single actuator hand exoskeletons is to adopt underactuation by introducing passive DoFs to the mechanism through passive joints. In order to simplify the design and improve wearability during grasping tasks, the proposed exoskeleton provides assistance only to the medial and distant phalanges. This approach is taken by connecting the device to the first and the second phalange and leaving the third phalange free. The linear actuators are fitting on top of the hand to improve portability. For the purpose of this project, the control scheme will consider only one DoF for each finger.
Only normal forces are applied to the user’s finger during operation. This property significantly improves the design and the functionality of the fasteners, which attach the exoskeleton to the fingers. In particular, since the fasteners are crucially used to transmit the tangential shear forces, there is no need to excessively tighten the finger for transmitting either torques or longitudinal forces to the skin. The underactuation approach allows the device to apply stable forces to the finger phalanges during grasping tasks using objects with any shape and size.
Figure 3.3 PERCRO exoskeleton hand.
The design of this hand satisfies the requirements needed to make an exoskeleton suitable for clinic rehabilitation applications such as comfortable and easy wearable, self-adjustability to different size of fingers, effective transmission of forces to the phalanges and grasping objects with generic shapes.
Five linear force sensors are attached to the back of the actuators, allowing to measure the forces exerted by each finger. The exoskeleton hand can work in different modes: backdrived, assisted or force sensing. The first mode allows the motors to imitate the force exerted by the fingers and reproduce the intended move without any interference. The second mode can assist the move by exerting a small increments of force with respect to the force sensed. In the third mode the motors are block, hence no finger moves are possible. This mode is used to measure the maximum force levels that can be exerted by the fingers without moving the phalanges. In the acquisition protocol described in section 5.2 the exoskeleton hand is configured to work in this mode.
Full description of the mechanism and features of the exoskeleton hand can be found in [38].
4-‐ SYNERGY-‐BASED EMG CONTROL
Most of the commercial upper limb prostheses available can control only one DoF at a time, therefore a mechanism to switch between DoFs is required (e.g. muscle coactivation). Despite the good results obtained in pattern recognition algorithms, this control scheme remains sequential and in an on/off fashion (the gesture is totally performed or not performed at all). In order to provide a more natural control to the prosthetic users, is highly desirable to design an intuitive control able to perform simultaneous and proportional movements (i.e. wrist plus hand gestures or different fingers at the same time).
Recently, the muscle synergy concept, explored by nonnegative matrix factorization (NMF) of EMG signals, has been identified as a promising approach for decoding multiple degrees-of-freedom (DoF). Jiang et al [39] presented an online control of 2 DoFs wrist movements (flexion/extension and radial/ulnar) by using a semi-supervised NMF algorithm. However, a significant performance loss was reported when adding a 3th DoF (pronation/supination) to the set of gestures. Additionally, they manifested an increase of performance between the results obtained during the offline experiment compared to the online results. Jiaxin et al [40] proposed a synergy-based control scheme for simultaneously active wrist gestures pronation/supination with open/close hand gesture by using the same approach proposed by Jiang.
Common clinical rehabilitation techniques usually require to move the fingers one-by-one but also to perform different gestures that involve multiple fingers at the same time. Dextrous finger control is also an important feature in prosthetic applications which provides more natural control and execution of the moves.
The object of this study is to successfully decode each finger individually instead of treating the gestures as an entity. Apart from the benefits of providing an intuitive control where the muscle activations are directly related to natural gesture, this approach also make the control more robust against errors. Since each finger is controlled individually, an error in one finger might not necessary compromise the whole gesture and can be corrected before loosing the position (e.g. the gesture of closing the hand involve all the fingers, if there is an error it is more likely to fail one or two fingers instead the 5 of them, therefore the purpose of the gesture is still working and the subject will not feel the frustration as if the gesture would change completely from the desired).
Compared with the wrist movements, finger movements involve more muscles, which adds complexity to the problem. When the muscles actives are close to each other it is expected to suffer a crosstalk that will contaminate the EMG signal and therefore the gesture will be more difficult to identify. Unfortunately, the solution to this problem is
target muscle with intra-muscular sensor, which is normally not a desired option for rehabilitation.
As far as the author's knowledge concern, this is the first publicly available study for a dexterous proportional and simultaneous finger control using muscle synergies, which decodes a total of 4 DoF's (index, middle, ring and little finger).
In this chapter the control method proposed is described in three different sections. Section 4.1 provides a brief insight of the Bayesian filtering theory that is used in this study as EMG feature. The consequent section 4.2 details the NMF semi-supervised algorithm applied to the estimation of synergies. Finally, in the section 4.3 is explained the procedure to obtain the control signals during the offline test. A description of the algorithm proposed is illustrated on figure 4.1.
In Chapter 5 two experimental protocols for data acquisition are described (NinaPro database and exoskeleton protocol) together with the implementation results and the synergy analysis.
Figure 4.1: Block diagram of the algorithm described in section 4.2
One of the main characteristic of this control approach is the short calibration time required. The data acquired for training the algorithm can be recorded for each patient in a single session of less than 10 minutes. Consequently, the possible re-calibration process needed over the time due to the physical changes in the subject musculature would not be a relevant issue anymore.
4.1 Bayesian estimator
Currently, all approaches for myoelectric control are based on feature extraction from the EMG signal and almost all of them use the signal amplitude or power estimation. Signal amplitude is a key feature in classic pattern recognition and essentially the only feature for regression-based methods. The most common methods for amplitude estimation are the mean absolute value (MAV) and the root mean square value (RMS). However, alternatives to EMG amplitude estimation has been proposed, such as Bayesian based estimator, which has been shown to outperform MAV and RMS both in terms of stability and responsiveness [41].
The purpose of filtering is to extract an underlying driving signal that describes the measured surface EMG. The Bayesian estimator provides the filtered signal that best describes the observed EMG signal. This estimator yields results with very low short-time variability but also with the capability of very rapid response to change. The estimate signal approximates isometric joint torque with lower error and higher signal-to-noise ratio than current linear methods [56].
The instantaneous relation between the latent driving signal x and the resulting EMG signal can be described by a conditional probability density 𝑃 𝐸𝑀𝐺 𝑡 𝑥 𝑡 , which is the probability of the underlying neural drive taking a certain value. Sanger et al [41] proposed three models for the conditional probability of the rectified 𝑒𝑚𝑔 𝑡 :
1 - The Poisson model:
𝑃(𝑒𝑚𝑔(𝑡)|𝑥(𝑡)) ≈𝑥(𝑡)k𝑒3l(m)
𝑛! (4.1)
where x is the average rate of events for the whole muscle, and thus represents the unknown driving signal, and n is an integer that is proportional to the magnitude of the rectified EMG signal. He uses the rectified EMG because is commonly used in estimation algorithms.
2 - The Half-Gaussian measurement model:
Is it assumed the EMG signal can be described as amplitude-modulated zero-mean Gaussian noise such as:
𝑃 𝑒𝑚𝑔(𝑡) 𝑥(𝑡) = e
3pqr(m)s Sl(m)s
(2𝜋𝑥(𝑡)S) (4.2)
3 - The exponential model or Laplacian model:
Close observation of the EMG signal suggests that the density may be better approximated by a Laplacian density
𝑃 𝑒𝑚𝑔(𝑡) 𝑥(𝑡) = e
3pqr(m) l(m)
𝑥(𝑡) (4.3)
The goal is to estimate the probability of the underlying driving signal taking a certain value given a single measurement of 𝑒𝑚𝑔 𝑡 . At time t, the function P[emg(t) | x(t)] specifies the likelihood of each possible value of x(t) given that particular measurement. The posterior density can be expressed using the Bayes's rule
𝑃 𝑥 𝑡 𝑒𝑚𝑔 𝑡 = 𝑃 𝑒𝑚𝑔 𝑡 𝑥 𝑡 ∗ 𝑃[𝑥 𝑡 ]
𝑃[𝑒𝑚𝑔 𝑡 ] (4.4) where P[x(t)] is the probability density for x(t) immediately before the measurement of 𝑒𝑚𝑔 𝑡 . In general, the prior P[x(t)] will depend on the entire past history of measurements and is estimated by using a discrete time recursive algorithm where t is an integer, using the Bayes rule
𝑃 𝑥 𝑡 𝑒𝑚𝑔 𝑡 − 1 , …
= 𝑃 𝑒𝑚𝑔 𝑡 𝑥 𝑡 ∗ 𝑃[𝑥(𝑡)|𝑒𝑚𝑔 𝑡 − 1 , … ]
𝐶 (4.5)
where C is a constant chosen so that the density integrates to 1. The estimate can be written as
𝑃 𝑥 𝑡 𝑒𝑚𝑔 𝑡 − 1 , 𝑒𝑚𝑔(𝑡 − 2) … = 𝑝(𝑥, 𝑡 − 1) (4.6)
In order to estimate the conditional density of x at time t the Fokker-Planck equation is used approximated by
𝑝 𝑥, 𝑡 − ≈𝛼𝜕S𝑝 𝑥, 𝑡 − 1
𝜕𝑥S + 𝛽 + 1 − 𝛽 ∗ 𝑝(𝑥, 𝑡 − 1) (4.7)
where 𝑝 𝑥, 𝑡 − indicates the conditional density of x at time t immediately before the measurement of 𝑒𝑚𝑔 𝑡 . Finally, after discretizing x into bins of width epsilon, as soon as a new measurement of 𝑒𝑚𝑔 𝑡 is available, it is possible to calculate
𝑝 𝑥, 𝑡 = 𝑝 𝑒𝑚𝑔 𝑡 𝑥 𝑡 ∗ 𝑝 𝑥, 𝑡 − 1 𝐶 = 𝑃 𝑒𝑚𝑔 𝑡 𝑥 𝑡 ∗ [𝛼𝑝(𝑥 − 𝜀, 𝑡 − 1) + 1 − 2𝛼 ∗ 𝑝 𝑥, 𝑡 − 1 + 𝛼𝑝 𝑥 + 𝜀, 𝑡 − 1 + 𝛽 + 1 − 𝛽 ∗ 𝑝(𝑥, 𝑡 − 1)]/𝐶 (4.8)
where 𝛼 is the diffusion rate and 𝛽 is the Poisson jump rate for the driving signal mode. The maximum a posteriori (MAP) estimate of 𝑥 is found by maximizing 𝑃(𝑥|𝑒𝑚𝑔). It can be calculated at each step as
𝑀𝐴𝑃 𝑥, 𝑡 = 𝑎𝑟𝑔𝑚𝑎𝑥 𝑝(𝑥, 𝑡)
(4.9)
At the first step, 𝑝 𝑥, 0 can be initialize to a constant.
The recursive density propagation algorithm proposed by Sanger et al [41] can be summarized as follows:
1) Initialize 𝒑 𝒙, 𝟎 = 𝟏
2) Forward propagate 𝒑(𝒙, 𝒕−) 3) Measure the rectified EMG signal
4) Calculate the posterior likelihood function 𝑷 𝒙, 𝒕 ≈ 𝑷 𝒆𝒎𝒈 𝒙 ∗ 𝒑(𝒙, 𝒕−) 5) Output the signal estimate 𝑴𝑨𝑷 𝒙, 𝒕 = 𝒂𝒓𝒈𝒎𝒂𝒙 𝒑(𝒙, 𝒕)
6) Divide 𝒑 𝒙, 𝒕 by a constant 𝐶 so that 𝒑 𝒙, 𝒕 𝒅𝒙 = 𝟏 7) Repeat from step 2
The measurement model 𝑃(𝑒𝑚𝑔|𝑥) can be chosen to be one of the three models described earlier. An extended proof of this algorithm can be found in [41].
An improved estimator of the driving signal will have both rapid response and adequate rejection of irrelevant components of the EMG signal. Hoffman et al [42] proposed a variation of the algorithm above using the differential Chapman-Kolmogorov equation based on Markov processes rather than a Fokker-Plank equation. He assumed that both the measured EMG signal and the driving signal x are Markov process (i.e. predictions for the future of the process can be made based solely on its present state).
They choose the point estimation to be the expectation value rather than the maximum value of the posterior distribution, in order to reduce the sudden jumps during fast dynamic contractions. After deciding on a point estimation method the recursive Hoffman's algorithm is described following:
1) Time evolution updating:
𝑝 𝑒𝑚𝑔 𝑡 + 1 𝑥 𝑡 = (𝛼𝑝(𝑒𝑚𝑔 𝑡 + ∆𝑒𝑚𝑔(𝑡) ∆𝑒𝑚𝑔 𝑡 S + 𝑝 𝑒𝑚𝑔 𝑡 + ∆𝑒𝑚𝑔 𝑡 − 2𝑝(𝑒𝑚𝑔(𝑡) ∆𝑒𝑚𝑔 𝑡 S + 𝛽 1 𝑒𝑚𝑔qŒl− 𝑝 𝑒𝑚𝑔 𝑡 + 𝑝(𝑒𝑚𝑔 𝑡 ) (4.10) 2) Observation updating: 𝑝 𝑒𝑚𝑔 𝑡 + 1 𝑥 𝑡 + 1 = 𝑝 𝑥 𝑡 + 1 𝑒𝑚𝑔 𝑡 + 1 ∗ 𝑝(𝑒𝑚𝑔(𝑡 + 1)|𝑥 𝑡 ) 𝑝(𝑥(𝑡 + 1)|𝑥 𝑡 ) (4.11) 3) Point estimation: 𝑒𝑚𝑔 𝑡 + 1 = 𝑒𝑚𝑔 𝑡 + 1 ∗ 𝑝(𝑒𝑚𝑔(𝑡 + 1)|𝑥 𝑡 + 1 ) pqr (4.12)
Point estimation is chosen in this case to be the expectation value rather than the maximum of the posterior distribution as proposed before. The idea is to reduce the frequent large sudden jumps in estimates produced by the dynamic contractions.
An extended proof of this algorithm can be found in [42].
There are three free parameters: 𝛼, 𝛽, and 𝜀, which specifies the expected rate of gradual drift in the signal, the expected rate of sudden shifts in the signal and the bin width for discretization of the estimate 𝑥, with 𝑛 = max 𝑥 /𝜀 respectively. These values are selected empirically by testing on the data set. Figure X shows the effect of the sudden jumps rate 𝛽 parameter on the Bayes estimates.
The modified version of the Sanger's algorithm proposed by Hoffman et al [42] has been used in this study.
Figure 4.2. Comparison of Bayesian estimates using three different values for 𝛽 parameter for the same EMG signal that represents three muscular contractions. Laplacian model has been chosen as measurement model 𝑃 𝑒𝑚𝑔 𝑥 and the expectation value has been used for point estimation. First figure corresponds to 𝛽 = 1347, second figure
corresponds to 𝛽 = 13•7, and in the third figure a value of 𝛽 = 13S•7 is used. Values of 𝛼 and 𝜀 are fixed to 13S7and
150 respectively.
As shown in Figure 4.2, decreasing the parameter value β will result in a smoother signal. However, too small values will cause a delay of the responsiveness of the estimate signal and a loss of signal information. The optimal values of the three parameters α, β and ε are chosen once empirically for the first subject and then fixed for the rest of the experiments.
4.2 Synergies estimation
According to the muscle synergies theory, when a subject’s central nervous system (CNS) controls the body movements, it does not send a single command to multiple muscles individually. Instead, it activates a small set of coordinated movements or a cascade of movement primitives called synergies. Each synergy is responsible for activating a larger set of muscles. This behavior allows the CNS to perform certain tasks by controlling a small number of synergies, instead of controlling individual muscles.
The existence of muscles synergies has been shown with strong evidence in animals [42] and humans [43] by using linear decomposing algorithms such as nonnegative matrix factorization (NMF) [44].
Synergy model
Based on the neurophysiological process of natural movements, Jiang et al [28] proposed a generative model of the surface EMG described following.
The model represents the control information driven by the motor units (MU) as a time varying force functions which represents the intended activations level of all the DoF's of the limb.
For a joint which has N DoF's the force functions can be expressed as: 𝐹 𝑡 = [𝑓4 𝑡 , 𝑓S 𝑡 , … , 𝑓2 𝑡 ]
(4.13) where 𝑓2 𝑡 is the intended activation level of the 𝑛th DoF. The activation function of all the 𝑀 muscles involved on that join is expressed as:
𝑋 𝑡 = [𝑥4 𝑡 , 𝑥S 𝑡 , … , 𝑥] 𝑡 ]
(4.14) where 𝑥] 𝑡 is the activation function of the 𝑚th muscle which represents the neural drive directly applied to the muscle. The activation vector is an instantaneous mixture of 𝐹 𝑡 with 𝑆 such that
𝑋 𝑡 = 𝑆 ∗ 𝐹(𝑡)
where 𝑆 represents the level of which the 𝑚th muscle is actually participating in the activation of the 𝑛th DoF of the limb. This instantaneous mixing process generates the synergies corresponding to that particular joint.
In the particular case of EMG control, 𝑋(𝑡) is a nonlinear mixture of the force functions observed on the EMG surface. Usually only the force functions are available to read, and therefore extracting 𝑆 from (4.15) became an instantaneous nonlinear blind source separation (BBS) problem. However, then the nonlinearity of the problem decreases if assuming the two following conditions are satisfied:
1) The crosstalk between EMG channels is small.
2) The demodulator (i.e. the feature of the EMG signal) is approximately linear with respect to its activation function.
Assuming both conditions are fairly well satisfied, then the problem can be reduced to a linear mixture and so, a semi-supervised source separation algorithm based on the non negative matrix factorization is used to estimate the activation signals 𝐹(𝑡) in (4.15). A more detailed proof of the BBS problem can be found in [28].
NMF algorithm
NMF is a method for dimensionality reduction that respects the non negativity of the input data while constructing a low-dimensional approximation. It has been widely used in pattern recognition applications and data mining [44]. The difference with other matrix factorization methods such as principal component analysis or independent component analysis is that NMF requires both matrices to have nonnegative values, which means that the data can be constructed only using additive elements. That is, the original matrix can be represented by linear combination of only addition without subtraction of the decompound matrix.
Being 𝑋 a nonnegative 𝑁 × 𝑀 data matrix, where the columns are the sample vectors and the rows corresponds to the channels, it can be approximate by NMF as:
𝑋 = 𝑊 ∗ 𝐻
(4.16) where 𝑊 and 𝐻 are two matrix of size 𝑀 × 𝑅 and 𝑅 × 𝑁 respectively.
The factorization result is not exact, 𝑊 ∗ 𝐻 is a lower rank approximation of 𝑋 which can be determined by minimizing the error function between the original data matrix and the
decomposition 𝑊 ∗ 𝐻. Different cost function can be used for the error optimization. The most common cost function is the Euclidean distance which can be written as:
𝐽 = | 𝑋 − 𝑊𝐻 |
(4.17)
The constrains of this minimization problem have to satisfy the non negativity of 𝑊 and 𝐻. Different techniques are using to solve this optimization problem such as gradient method, fixed point iteration method, etc. Lee and Seung [45] proposed an extended version of gradient based update equations:
𝐻 𝑛 + 1 = 𝐻 𝑛 ∗ 𝑊d𝑋
𝑊d𝑊𝐻 (4.18)
𝑊 𝑛 + 1 = 𝑊 𝑛 ∗ 𝑋𝐻d
(𝑊𝐻𝐻d) (4.19)
The proof of this update equations can be found on [45].
In this study, the NMF algorithm is applied to the myoelectric control context and it will have the following construction form:
𝑋]×d = 𝑊]×Z∗ 𝐻Z×d
(4.20) where 𝑋 is an 𝑀×𝑇 matrix (𝑀 sEMG channels, 𝑇 samples) containing the Bayesian estimates of the EMG signal. 𝑊 is an 𝑀×𝑃 matrix of the muscle synergies and 𝐻 is the activation signals matrix 𝑃×𝑇. The value of 𝑃 can be expressed as 𝑃 = 𝐾 ∗ 𝑁, where 𝑁 is the total number of DoF’s considered for the problem and 𝐾 is the number of synergies estimated for each DoF.
In order to obtain the synergy matrix 𝑊]×Z, a semi-supervised algorithm is proposed by Jiang [28], which involves a divide-and-conquer approach that estimates the synergies corresponding to one DoF at a time before combine them all together into the final 𝑊 matrix. The algorithm is described in the section below.
DoF-wise algorithm:
As stated before, the observed EMG surface 𝑋 𝑡 can be expressed by the product of a synergy matrix 𝑊 and the activation signal 𝐻 𝑡 :
