VERIFICATION AND VALIDATION (V&V) ON MILITARY SDR SYSTEMS

(1)

UNIVERSITÁ DIPISA

DOTTORATO DI RICERCA ININGEGNERIA DELL’INFORMAZIONE

V

ERIFICATION AND VALIDATION

(V&V)

ON MILITARY

SDR

SYSTEMS

DOCTORALTHESIS

Author

Fulvio Arreghini

Tutor (s)

Prof. Prof. Marco Luise

Reviewer (s)

Eric Nicollet, Andrea Giovagnola

The Coordinator of the PhD Program

Prof. Marco Luise

Pisa, 09 2016 XXIX

(2)

(3)

Acknowledgements

A

Tthe end of my PhD path I would like to thank, in the first place the Italian Navy,

that gave me the opportunity to enrich my professional skills undergoing the exciting and challenging path of the PhD course, while continuing my service as Navy Officer. I am the first in the Italian Navy to make this experience and I hope I have been able to meet the expectations of the Navy so that in the future additional Navy Officers could do the same.

A grateful acknowledgment then goes to my tutors: Prof. Marco Luise and Captain (Navy) Andrea Manco who have been supporting me during my PhD path. I can un-derstand that for both of them the tutoring task has been not simple: for Prof. Luise I have for sure been a strange kind of PhD student, as I am pretty different for my age and professional experience from the standard and as I had to harmonize my PhD with my duties as Navy Officer in national and international contexts. On the other hand, I probably have been also a strange kind of Navy Officer for Capt. Manco, as, while be-ing under his command , I had to carry out also my duties as PhD student, participatbe-ing to conferences and lessons and preparing papers.

My tutors have had a very positive attitude towards me, allowing me to organize my work with wide autonomy, while constantly monitoring it.

Another acknowledgment goes to my reviewers: Eric Nicollet and Captain (Navy) Andrea Giovagnola. When I asked them to evaluate my thesis, both of them accepted with unexpected enthusiasm. I know that the review job is long and sometimes tedious but it provided me very valuable inputs for this document and also for my future work. I would like to thank also all the industry teams that have been working with me in different projects at the LANCERS lab, allowing the development of several tools described in this thesis.

Similarly, I wold like to thank the entire ESSOR community, made of Nations, In-dustries and the Organisation Conjointe de Cooperation en matiere d’ARmement (OC-CAR). I have been involved in the ESSOR SDR programme since several years and I had the opportunity to follow it very closely during my experience in OCCAR in 2016. I believe that the ESSOR programme has been and still is more than a simple SDR pro-gram: it created a complete ecosystem around the SDR technology and it provided me

(4)

valuable examples of methodologies constituting best practice to be followed. ESSOR in my opinion really has the potential to change the rules of military SDR and this is mainly thanks to the community of committed professionals working in it.

Last but not least, I would like to give a very special thank my wife Stefania, and my sons, William and Gabriel. Having a husband and a father that spends his time between the Navy duties and the PhD activity is not simple for a family, as the time is always a critical resource. All my family has really gone out of its way to put me in the best condition to complete the challenging PhD path. In this context a special thank is devoted to my wife, who, in addition to making all the possible to leave me time to study, has always been a constant support pushing me to do my best.

(5)

Summary

S

DR, since its theoretical definition by J. Mitola, has been really appealing for the

military communications. The possibility to move forward from the legacy radio devices, where all the functionalities were directly embedded in the hardware, towards a new type of radio that allows reconfigurability and flexibility is seen as a

Deus ex machinain the complex and crowded panorama of tactical communications.

The possibility to execute different waveforms on the same platform can potentially prolong the lifecycle of an operational radio for several years. The possibility to port the same waveform to different hardware, on the other hand, has twofold implications: it allows the re-use of the same waveform, maximizing the return on investment for the developers and, in addition, it allows decoupling the radio manufacturer from the software developer, potentially creating a new market for the waveforms development and procurement. Similarly to what has been commonly experienced in the smartphone market, where Apps are developed independently by the vendor of the terminal, in the future, SDR waveforms might be developed by different subjects, even outside Industry (e.g. universities and research centers) and then ported on an existing SDR platform.

The “SDR revolution” is anyway not feasible as long as common development rules for waveforms and common interfaces between the waveform and the underlying hard-ware will be clearly defined. The most relevant contribution in this sense was the def-inition of the Software Communication Architecture (SCA), published and maintained by the Joint Tactical Networking Centre (JTNC). The SCA defined a set of require-ments and rules both for the development of waveforms and for the software definition of processing elements and functions within the host platforms, though leaving to im-plementers the possibility to choose between different hardware and software solution for the implementation of a SCA-compliant SDR solution. SCA, rapidly became a de

factostandard in the military panorama, as it has been widely adopted for the major

military SDR programmes also in Europe, mainly in its version 2.2.2. SCA 2.2.2 was designed to meet the stringent requirement of military communications, in terms of se-curity, timing accuracy, real time behavior and it was not able to spread widely outside this context for several years, mainly because of the overhead it imposes to the appli-cations. Today SCA, in its version 4.1, has been published in the US as an emerging

(6)

standard and its evolution is also supported by the Wireless Innovation Forum. One of the goal of the SCA 4.1 is to define profiles that allows its implementation in devices with limited processing resources, meeting the needs of small form factor devices in the military context and projecting towards applications outside the military. Neverthe-less, some other emerging standards are facing the military SDR market, as possible alternatives to SCA.

From the point of view of a procurement agency (like MoDs are in the context of military communications),SDR offers significant benefits but, at the same time, it raises new issues and challenges. First, the transition to the software implementation of some functions of the radio platform, requires the development of new testing skills inside the Defence panorama. Each service (i.e. Armed Force) in the Italian Defence, developed during the years, its verification and validation (V&V) facilities, holding a proven expe-rience in the testing methodologies for military radios. The transition to SDR requires that radio testing skills are enhanced with software verification and signal processing skills, as many of the function of new radios are not only performed in software but they also offer the possibility to perform inspection on their behavior. The military procurement strategy for SDR is today closely linked to SCA-based architectures. For these architectures new waveforms have been developed. The ability to test these wave-forms, prior to their porting on the final hosting platwave-forms, requires the verification of the compliance of the software modules to the rules of the underlying software archi-tecture (SCA-based). This means that the verification strategy shall comprise the ability to verify that the SCA implementation of a software component is properly performed, assuring future compatibility of the component with the other elements of the waveform and of the platform. After the aforementioned V&V steps, carried out usually during the development of the SDR system or of one of its component (platform, waveform), the complete system is usually delivered to the test facility to undergo a number of tests analogous to that performed on the legacy systems: they span from testing the behavior of the radio over the air, to environmental and electromagnetic compatibility (EMC) testing, to interoperability testing with existing communication systems, where appli-cable. These activities are usually performed through “live fire exercise”, how they are called in military jargon: the system is deployed into an environment that tries to recre-ate as much as possible the operational scenario and tested under realistic conditions, by the operators that will employ it in real operations soon afterwards.

In response to these needs, the Italian MoD has identified the development of a gov-ernmental capability for SDR V&V as a strategic pillar and has consequently funded the establishment of a dedicated laboratory (called LANCERS), based on a cooperation between CSSN ITE, a research and experimentation center of the Italian Navy located in Livorno and CNIT (Consorzio Interuniversitario per le Telecomunicazioni) where experimentation and research on tools and technicques for SDR V&V are performed. The laboratory has developed with the time also some collaboration with industries involved in the SDR market.

This work summarizes the outcomes of the activities carried out in the LANCERS laboratory, regarding methodologies, tools and experimentation oriented to V&V of military SDR.

After providing a brief overview of the major programmes and stakeholders of the military SDR panorama in chapter 1, an analysis of the domains of application of the

(7)

T&E strategies for SDR will be identified in Chapter2. Chapter 3 will provide an in-depth description of the tools and procedures developed at LANCERS lab for military SDR test and evaluation. Different tools, addressing the needs of different phases of SDR development, will be presented, together with the design process that brought to their creation and some results of real applications of these tools. Chapter 4 will present a field testing campaign performed as a necessary complement to the lab test activities listed in chapter 3. Chapter 5 will draw the conclusions and present future work plans for further improving the tools and procedures presented.

For some testing activities the presentation of the results will be limited, due to the fact that the disclosure of information related to some particular SDR product is at the moment of writing this thesis yet subject to restrictions. Provided results will be anyway sufficient to provide the reader with a good understanding of the functionalities and applicability of the presented tools and procedures.

(8)

(9)

Sommario

S

In dalla sua prima definizione teorica elaborata da J. Mitola, la Software defined

Radio (SDR) è stata di notevole interesse per le comunicazioni militari. La pos-sibilità di passare da radio di vecchia generazione (c.d. "legacy") nelle quali le funzionalità erano inscindibili dalla piattaforma a radio nelle quali si possa riconfigu-rare la piattaforma in maniera flessibile, vista come una sorta di Deus ex machina nel complesso panorama delle comunicazioni tattiche. La possibilitaà di eseguire wave-form diverse sulla stessa piattawave-forma può estendere la vita operativa delle piattawave-forme di molti anni. Inoltre, la possibilità di installare (portare) la stessa forma d’onda su piattaforme diverse, ha una duplice implicazione: consente il riutilizzo del codice della forma d’onda, massimizzando il ritorno sugli investimenti per gli sviluppatori e con-sente la separazione delle competenze tra il produttore di radio e lo sviluppatore di software di forme d’onda, creando un potenziale nuovo mercato per il procurement di tali software. A similitudine di quanto avvenuto nel mercato degli smartphone, dove le cosiddette app sono sviluppate in maniera indipendente dal produttore dell’hardware, in futuro, le forme d’onda per SDR potrebbero essere sviluppate persino al di fuori del-l’industria, da università o centri di ricerca e poi portate sulle piattaforme esistenti. La "rivoluzione SDR" non è però fattibile fino a che non verranno definite regole comuni per lo sviluppo delle waveform e interfacce comuni tra queste e le piattaforme radio. In questo senso, il contributo più rilevante è stato la definizione della Software Commu-nication Architecture (SCA) da parte del Joint Tactical Networking Centre (JTNC). La SCA ha definito una serie di requisiti e regole per lo sviluppo di waveform, lasciando comunque gli sviluppatori liberi di scegliere tra diverse soluzioni per la realizzazione di un’architettura SCA-compliant. La SCA è divenuta rapidamente uno standard de fac-to nelle cmomunicazioni militari ed è stata adottata, principlamente nella sua versione 2.2.2 da tutti i maggiori programmi SDR europei. Essendo concepita per gli stringen-ti requisistringen-ti di delle comunicazioni militari, tuttavia, la SCA ha trovato poco riscontro al di fuori del panorama militare, a causa dei vincoli di performance che impone alle piattaforme e all’overhead che richiede di introdurre nello sviluppo del software.

Oggi l’ultima versione pubblicata come "emerging standard" della SCA è la 4.1, la cui evoluzione è supportata anche dal Wireless Innovation Forum. Uno degli obiettivi

(10)

di questa versione è quello di definire dei profili che ne consentano l’implementazione anche su piattaforme dalle limitate capacità di calcolo. In parallelo, nuovi standard per architetture SDR stanno venendo introdotti sul mercato, come possibili alternative alla SCA.

Dal punto di vista del procurement (quindi della Difesa) la SDR offre dei grandi benefici ma allo stesso tempo pone nuovi quesiti e sfide.

Per prima cosa il nuovo paradigma richiede che vengano sviluppate nuove capacità di test all’interno della difesa. Oggi ciascuna Forza Armata ha dei centri di V&V (verifica e validazione) per comunicazioni tattiche, con comprovata esperienza nel test di radio militari. L’introduzione della SDR richiede che le competenze esistenti siano arricchite con altre, spiccatamente orientate alla verifica di softwae e alla valutazione di algoritmi di elaborazione dei segnali.

In secondo luogo, essendo la maggior parte delle nuove SDR militari sviluppate sulla base della SCA, è richiesto alle Forze Armate di acquisire capacità di testing relative alla compliance di waveform e piattaforme a questo standard. La strategia di verifica deve quindi implementare l’abilità di verificare la corretta implementazione di componenti software secondo le regole della SCA.

Successivamente ai passi di verifica menzionati, che avvengono nella fase di svilup-po del sistema SDR o dei suoi singoli comsvilup-ponenti, il sistema completo (radio+waveforms) viene solitamente testato in maniera analoga alle radio di vecchia generazione per de-terminarne il comportamento a radiofrequenza o la rispondenza a requisiti ambientali, come quelli di compatibilitaà elettromagnetica (EMC). Questi test richedono solita-mente delle prove dul campo che replichino fedelsolita-mente scenari di impiego assimilabili a quelli operativi dove il sistema verrà impiegato.

In risposta ai summenzionati bisogni, in Italia la Difesa ha identificato come di in-teresse strategico lo sviluppo di una capacità governativa di V&V di SDR e ha conse-guentemente finanziato la creazione di un laboratorio dedicato (chiamato LANCERS) basato su una cooperazione tra il CSSN ITE, un centro di ricrca e sperimentazione della Marina Militare Italiana situato a Livorno e il CNIT (Consorzio Interuniversitario per le Telecomunicazioni).

All’interno del laboratorio vengono eseguiti sperimentazione e ricerca su tools e metodologie per la V&V di prodotti SDR. Nel tempo, il laboratorio ha instaurato una serie di collaborazioni mirate con le principali industrie del mercato SDR su progetti dedicati.

In questa tesi vengono raccolti i risultati delle attività eseguite al laboratorio LAN-CERS riguardanti lo sviluppo di metodi e strumenti per la verifica e validazione di SDR. Per alcune di queste attività,

Dopo aver introdotto una panoramica dei principali programmi SDR militari e degli attori coinvolti nel panorama SDR nel capitolo 1 i possibili domini di applicazione delle strategie di test and evaluation per SDR militari verranno analizzati nel capitolo 2. Il capitolo 3 fornirà una descrizione dettagliata di strumenti e procedure sviluppati al laboratorio LANCERS per il test di SDR militari. Diversi strumenti, rivolti al testing di varie fasi dello sviluppo di un prodotto SDR verranno rpesentati, corredati della descrizione del processo di sviluppo di ciascun tool, motivandone le scelte progettuali e di risultati di applicazioni reali degli stessi. Il capitolo 4 presenterà i risultati di alcune

(11)

campagne di test sul campo effettuate, come necessario complemento alle attività di test di laboratorio. Il capitolo 5 presenterà le conclusioni e le prospettive future.

Per alcune attività di test presentate in questo documento, potrà essere fornita solo una parte dei risultati, in quanto la diffusione di alcune informazioni legate ai relativi prodotti SDR è ad oggi soggetta a specifiche restrizioni. I risultati forniti saranno ad ogni modo sufficienti a comprendere le funzionalità e l’applicabilità degli strumenti e delle procedure presentate.

(12)

(13)

List of publications

International Journals

1. Arreghini, F., Vitiello, C., Luise, M., Manco, A., Bacci, G., Falzarano, M. (2016). An Approach to T&E of Military SDR Platforms and Waveforms: the LANCERS Lab. Journal of Signal Processing Systems, 83(1), 93-111.ISO 690

International Conferences/Workshops with Peer Review

1. Arreghini, F., Agrone, R., Danielli, P., Pigni, A. (2015, October). Heterogeneous network testbed for tactical communication in shore scenario. In Military Com-munications Conference, MILCOM 2015-2015 IEEE (pp. 483-488). IEEE. 2. Mondini, L., Arreghini, F. (2013). Protezione dei lavoratori dai campi

elettromag-netici: gli obblighi connessi al d. lgs. 81/08 e l’approccio mediante procedura di zoning. Campi elettromagnetici e innovazione tecnologica in ambito difesa, industria e ricerca, 105-108.

3. Arreghini Fulvio, Vitiello Carmine, Luise Marco, Bacci Giacomo, Manco Andrea, F. M. (2014, April). Test and Evaluation of Military SDR Platforms and Wave-forms: Initial Outcomes from the Laboratory funded by the Italian Ministry of Defens. In RTO Symposium IST 123 "Cognitive radio and future networks". 4. Serra Christian, Chedhomme Charles, Arreghini Fulvio et al. ESSOR HDRWF

Secure Coalition Waveform Verification Achievements. Wireless Innovation Fo-rum European Conference, Paris, 11-14 October 2016.

5. Mondini Luciano, Sbrana Carlo, F. A. (2014). Exposure by WLAN in an In-door Environment. Atti del III Convegno Nazionale Interazioni tra Campi Elet-tromagnetici e Biosistemi,Napoli 2-4 luglio 2014http://www.icemb.org/ wp-content/uploads/2011/08/Atti.pdf

(14)

Others

1. Arreghini Fulvio, Chedhomme Charles. Cross Platform versatility of SDR. De-fence Communication Conference, Rome, 20-21 September 2016.

(15)

List of Abbreviations

A

AIS Automatic Identification System. 58

API Application Programming Interface. 7, 29,

43, 59

ASIC Application Specific Integrated Circuit. 36

B

BSI Basic Software Item. 8–10, 69

C

CORBA Common Object Request Broker. XVIII, 4,

24–26, 28–32, 34, 55, 57, 67

COTS commercial off the shelf. 1

D

DoD Department of Defence. 4, 26, 36

DSP Digital Signal Processor. 4, 5, 28, 63

DTD Document Type Descriptor. 58–62, 67

E

EPC Evolved Packet Core. 75–77

ESM Electronic Support Measure. 83, 84

F

FPGA Field Programmable Gate Array. 4, 5, 28,

63 G

GPP General Purpose Processor. 28–33, 63, 67

GPS Global Positioning System. 80

(16)

H

HF High Frequency. 56

HMI Human to Machine Interface. 78, 82

I

IDL Interface Definition Language. 8, 26, 31,

43, 64

ITU International Telecommunication Union.

58 J

JTEL Joint Test and Evaluation Laboratory. 6

JTNC Joint Tacctical Networking Centre. 4, 23,

25–27, 31 K

Kbit/s Kilobit per second. 58

L

LTE Long Term Evolution. XIX, 74–77

M

MHAL Modem Hardware Abstraction Layer. 29

MIB Management Information Base. 78, 79

MoD Ministry of Defence. 71, 85, 86

N

NTE Native Test Environment. 7, 68, 70, 71

O

OCCAR Organisation Conjointe de Cooperation en

matiere d’ARmement. I

OE Operating Environment. 23, 24, 27, 47, 48,

59, 71 P

PESQ Perceptual Evaluation of Speech Quality.

75

PIM Platform Independent Model. 28, 54, 67

PRF Property Descriptor. 25, 62–65

PSM Platform Specific Model. 67

S

(17)

SAD Software Application Descriptor. 25, 57, 60, 61, 64–67

SBW Soldier Broadband Waveform. 75–77, 79

SCA Software Communication Architecture.

XVI, XVIII, 4, 23–35, 37, 40, 43, 46, 48, 54, 58, 64

SCD Software Component Descriptor. 25, 60–

67

SDR Software Defined Radio. XVI, XIX, 1–5,

22–38, 40, 41, 43, 53–55, 57, 59, 67, 70– 73, 75–77, 80–87

SNMP Simple Network Management Protocol. 78

SPD Software Package Descriptor. 25, 44, 45,

59, 60, 62, 63, 65 T

T&E Test and Evaluation. XIX, 22, 23, 36, 40,

41, 53, 70–72 U

UDP User Datagram Protocol. 76

UML Universal Modeling Language. 8, 43

US United States (of America). 6, 19

USRP Universal Software Radio Peripheral. 55

V

V&V Verification and Validation. 14, 54, 59, 64

VHF Ultra High Frequency. 56

VHF Very High Frequency. 56

VoIP Voice over IP. 75

W

WF Waveform. 68

WinnF Wireless Innovation Forum. 25, 26, 32

X

(18)

List of Figures

1.1 SDR adoption forecast (source the wireless innovation forum). . . 2

1.2 a comparison between different SDR markets). . . 2

1.3 overview of the ESSOR methodology for WF development). . . 8

1.4 outcome of the design phase). . . 9

1.5 golden source development). . . 9

1.6 German SVFuA SDR and OBISS interface . . . 11

1.7 German SVFuA SDR porting methodology [1] . . . 12

1.8 NATO framework for waveform sharing . . . 14

1.9 relationship between MOE and MOP [2] . . . 16

1.10 EDA three basket model for SDR certification . . . 18

1.11 Wireless Innovation Forum model for SDR certification . . . 19

2.1 structure of a SCA system . . . 24

2.2 sca architecture layer diagram . . . 25

2.3 platform independent model of SCA . . . 28

2.4 SCA 4.1 layer architecture . . . 29

2.5 ESSOR architecture . . . 30

2.6 ESSOR architecture deployment with Common Object Request Broker (CORBA) connectivity . . . 31

2.7 STRS architecture . . . 33

2.8 FACE model for software development . . . 35

3.1 SCA 4.1 base device. . . 44

3.2 state diagram for releaseobject. . . 47

3.3 state diagram for capacity management. . . 48

3.4 PVSIO, calling a function directly from the shell. . . 51

3.5 PVSIO, calling the test function test03. . . 52

3.6 example of SCA components for tactical waveforms developed. . . 58

3.7 GUI of the SCA tool. . . 61

3.8 definition of SCA ports in the SCA specification. . . 64

(21)

3.10 conceptual structure of ESSOR NTE. . . 70

3.11 methodology for SDR Test and Evaluation (T&E) at LANCERS lab . . 70

4.1 coverage of tactical LTE base station . . . 74

4.2 testbed for SDR interoperability tests . . . 76

4.3 SDR interoperability tests with Long Term Evolution (LTE) network . . 77

4.4 generation of a scenario with the SDR test tool . . . 81

4.5 report for the execution of a scenario . . . 82

4.6 SDR test tool structure . . . 83

(22)

List of Tables

1.1 ESSOR HDR development methodology . . . 7

2.1 ESSOR radio service APIs . . . 32

3.1 parameters defining the state of SCA device . . . 45

(23)

CHAPTER

1

Overview of the problem

1.1

SDR: a new paradigm rising new needs

The SDR concept enabled a deep transformation in the way of conceiving communi-cations. Several definitions have been provided for SDR, representing different point of view of the different stakeholders in the communications panorama. One definition, jointly developed by the Wireless Innovation Forum (WinnF) and IEEE [3] states that SDR is a " Radio in which some or all of the physical layer functions are software defined.

Since its first presentation by J. Mitola [4] SDR has been a very appealing concept for military communications. A market survey performed in 2011 [5] revealed that Virtually all of the tactical radios sold for military communications today utilize SDR technology. In the same study, a forecast for the growth of tactical SDR market was provided. (Figure 1.1)

The same study revealed that also the public safety market is experiencing a transi-tion towards SDR. Military and public safety markets share some requirements for their communication systems that can be fulfilled by SDR (reliability, security, flexibility, interoperability). Anyway, some important differences exist between the two markets: first of all the public safety market is more oriented towards the acquisition of a large number of devices, usually derived from commercial off the shelf (COTS) equipment, maintaining low the cost per unit procured, while the military market is more oriented towards a smaller number of units procured, but with high degree of customization, so that usually ad-hoc solutions are procured. Second, the public safety communications systems usually have to comply with civilian regulations to operate in populated or crit-ical areas (such as airports), where regulatory frameworks are well defined, while the military radios are usually more focused to operations in harsh electromagnetic envi-ronment and to information security, so that different and more extensive test campaign

(24)

Figure 1.1: SDR adoption forecast (source the wireless innovation forum).

are needed. In Figure 1.2, it is possible to see how the growth rate in the adoption of SDR is almost the same in all the considered market, but, in terms of size, the military market is several order of magnitude smaller than the other ones.

Figure 1.2: a comparison between different SDR markets).

An analysis of the trends of commercial and tactical SDR market is reported in [6]. The report indicates that the introduction of SDR into tactical communication market requires a change in the value chain paradigm: in legacy radios, the value of the communication device was mainly determined by the radio hardware, while the software (usually delivered as a firmware inside the radio set) embedded in the platform

(25)

was not considered a separate element of the overall price. In this model, the software of the radio existed only as an appendix of the radio set and not as a self-consistent product. With SDR the opportunity to create a software value chain is now foreseen. In this model, the development of software (i.e. waveforms) to be ported into an SDR platform could be carried out independently from the radio hardware manufacturer and can become a self-consistent activity. The software value chain model provides great opportunities also to Governments. In fact, if a Government funds the development of a new waveform, it is likely to get, at the end of the development program, the ownership of some work products (when they are deliverable in the contract) and some rights on the industrial products developed. That is to say, that when the developed waveform or part of it is sold (stand-alone or embedded in a radio system) to a third party customer, the funding Government is likely to get some revenues in terms of levies. This was not possible with the hardware-centric value chain adopted for legacy radio systems. The transition to the software business model requires that a waveform is treated as a self-consistent product and, for this reason, tested and certified independently by the hosting SDR platform. One important commonality between military and public safety markets is that both are adopting SDR in response to a strong need of interoperability: for years operators in these markets have been using a number of different communications systems, that were not able to exchange data with each other, if not through the use of gateways. This situation resulted in a heavy chaotic and congested environment when multiple services are deployed in a complex operation.

The deep consensus achieved by SDR in the military market is the result of the advantages it brings at different levels.

From the point of view of the warfighter on the ground (tactical level), the possi-bilities offered by the SDR translate into the need of having less devices to carry, to operate and to maintain, simplifying the conduction of modern operations, that heavily rely on communications and situation awareness (SA) systems. For the procurement officer, in charge of procuring and delivering the most effective technological solutions to answer the operational needs (operational level), SDR means a longer life cycle of the radios, as they can be updated with new waveform and functionalities without a compete replacement of the entire device. This implies that SDR leads to savings in the mid-term perspective: even if the initial procurement cost is higher than a tradi-tional (legacy) radio, a SDR can have a longer and wider deployment (it can be deliv-ered to different services/units changing only the waveform application) thus reducing the Total cost of Ownership of the system. Finally, from a Strategic point of view, the theoretical possibility to have new interoperable waveforms, vendor-independent and platform independent, offers the decision makers at high level new options for the planning of multinational operations, that are today the most common deployment situation. Since now, in fact, the deployment of forces in multinational context often required the interconnection of different communication systems, that are not natively interoperable. Sometimes, even different national implementations of communication systems formally compliant to the same standard have proven to be not directly inter-operable. This required the use of complex architectures based on multiple gateways between national systems that clearly introduced bottlenecks and points of failure in the mission network. Thus a multinational interoperable radio system is a strong need of the modern warfighter. The aforementioned technological opportunities mainly rely on

(26)

the concept of portability, that should be part of the SDR paradigm. Portability means that the code governing the radio frequency behavior of the radio (waveform applica-tion, WF) should be transferable to different platforms and, on the other hand, that one single platform can host different waveforms. Here comes the first great challenge of SDR: the software definition of some entities and functions of a radio system alone does not guarantee that the application code using these functions can be easily trans-ferred into different platforms. This limitation was highlighted in one of the first SDR programs run by the US Department of Defence (DoD): the Speakeasy program [7]. When it was presented, the Speakeasy program had the goal of providing interoperable

communications to the different services1 of the US armed forces, through the SDR

paradigm. The program developed a technological demonstrator, able to join the func-tions of ten legacy radio systems into a unique platform. Its architecture was based on the state of the art technology available at that time: the TMS320C40 Digital Signal Processor (DSP) from Texas Instruments, plus some Field Programmable Gate Array (FPGA). The demonstrator was anyway unsuitable for operational use, as it required a dedicated truck, given its dimensions. in developers’ mind, the technological evolu-tion of DSP could allow the speakeasy platform becoming an operaevolu-tional ready device, when they realized that the software part of the radio was developed using a device specific programming language (the C40 assembler). For this reason, the porting of the waveforms to a different DSP would have required a complete rewriting of the code. The SDR paradigm then met its first obstacle: transferring the radio functions to a software implementation is not enough if software is not hardware agnostic. The speakeasy lesson learned made clear that an abstraction layer between the application software and the platform hardware was needed. The research effort in this direction lead to the definition of the SCA, published by the Joint Tacctical Networking Cen-tre (JTNC) [8]. The SCA allowed the formal definition of interfaces and components, providing a complete abstraction of the platform hardware from the waveform develop-ment. The version 2.2.2 of the SCA is today the most widely used SDR architecture for military applications [9]. Outside the military context, anyway, SCA was not widely adopted as expected by JTNC. One of the possible reasons for this fact is the fact that the SCA 2.2.2 specification defines a transport mechanism for the communication be-tween software components based on CORBA middleware [10]. CORBA inevitably brings some overhead and additional constraints for the developer, as reported in [11]. This overhead is a minor drawback for the great flexibility that the CORBA middleware offers for mission critical highly specialized applications like military communications, but has proven to be less favorably perceived for general purpose solutions, especially when the major driver of a project is keeping the hardware simple and cheap. An-other major concern regarding the SCA 2.2.2 architecture is the fact its implementation is challenging when it comes to low-power, low-performance devices, like handhelds. For these reasons, the latest version of the SCA specification, named SCA 4.1 is ad-dressing profiles that do not require the full CORBA use and can be implemented on small-size device. The SCA 4.1 specification is currently published as emerging stan-dard in the DoD Information Technology Stanstan-dards Registry (DISR) [12] aiming to be the future de facto standard for military SDR, as SCA 2.2.2 is today.

1_{service is a a common wording of military jargon to indicate an Armed Force: inside Defence Army, Navy, Air Force, are}

(27)

SCA is today endorsed and monitored by the Wireless Innovation Forum. Other architecture definitions are emerging in the technological panorama, such as FACE, published by the Open Group [13].

The definition of architectures like SCA enables only one of the two requirements for portability. Given a SCA implementation, installed on an SDR platform, several waveforms can be developed on top of this architecture and then the radio platform can host different waveform, giving to the operator the option to instantiate the desired waveform according to the operational needs. Some limitations of this assumptions will be analyzed dealing with major military programs. The other way portability is intended, is the capability to port the same waveform on different platforms. In prin-ciple, the SCA specification is completely platform agnostic and, in its latest versions, addresses platforms based on different combinations of processing elements: General Purpose Processors (GPP), DSP, FPGA. When it comes to the real implementation of the different waveform algorithms (usually written with high level languages such as C/C++) into the target platform, an extensive process of code optimization is performed to ensure the best performances to the final system. This process introduces modifica-tion to the code of the waveform. Finally, depending on the specific brand and family of processing element used in the target platform, a dedicated compiler with slightly dif-ferent behaviors and constraints is used, so the porting of the waveform into the hosting platform could need an additional effort, which can be also very demanding.

SDR offers new possibilities to manufacturers and end users but, on the other hand, poses new challenges. First, only if a common architecture is used to couple the plat-form hardware and the waveplat-form, the goal of portability can be achieved. Second, only if the waveform code is design from the beginning to reduce the burden for the porting on the target platform portability is worthwhile (otherwise the code should be writ-ten again from scratch). Without portability there is no interoperability and the major opportunities of SDR are largely wasted.

For the above reason, not only the definition of common architectures and proce-dures is needed in for the development but also the definition and implementation of common verification and validation procedures is required. The aim of the V&V pro-cess is to ensure the the goal of portability can be reached, verifying from the early stages of the development phase of new products (platforms and waveforms) that com-mon rules are applied, regarding architecture and waveforms. Finally, the V&V process allows technical personnel of the Armed Forces to inspect and verify from the early stage of development of an SDR system, that the technical requirements given to the manufacturer are met. These considerations lead the Italian Ministry of Defence to create a V&V laboratory for SDR located in Livorno and called LANCERS lab, where the problem of SDR V&V and certification is studied and possible solutions are identi-fied. LANCERS lab was created with cooperation between CSSN ITE, a research and experimentation centre of the Italian Navy, and CNIT (Consorzio Nazionale Interuni-versitario per le Telecomunicazioni), an Academic consortium for telecommunications. This allowed the sharing of the military expertise with the Academic skills of the Uni-versity of Pisa. In the LANCERS lab, some tools for SDR V&V have been developed and several experiments have been conducted, also in cooperation with industries of the SDR market. In the following chapters some activities carried out at LANCERS lab to develop new tools and method for SDR V&V will be presented. Prior to describing

(28)

the V&V procedures and tools developed at LANCERS lab, in order to give the reader a more comprehensive overview of the tactical SDR panorama, the following sections will briefly present the main military SDR programmes running today in Europe and United States (of America) (US).

1.2

Overview of main military SDR programmes in Europe

The first structured implementation of SDR for military purposes has been the fol-low up of the already cited Speakeasy programme, called Joint Tactical Radio System (JTRS) [8]. The major outcome of the JTRS program has been the development of tactical radios based on the SCA architecture in its version 2.2.2. Since then, the SCA 2.2.2 has become the de facto standard for the great majority of military SDR pro-grams worldwide. Details of the JTRS program will not be provided in this work, as it is deeply discussed in almost all the SDR-related literature. Instead, some outcomes of the SDR programmes, impacting the European context, will be provided. In the US, tactical SDR programs are managed by a unique entity, the Joint Tactical Network Centre (JTNC) which comprises a dedicated laboratory: Joint Test and Evaluation Lab-oratory (JTEL) [14] for Test ad Evaluation of waveforms and radios. In addition to the SCA specification, several policy papers have been published by the JTNC containing the guidelines for the implementation of the standard. One of the most relevant pa-per is the waveform portability guidelines [15]. This papa-per is a reference adopted by the great part of SDR programmes on how to achieve portability for SDR waveforms. In Europe, tactical SDR are developed as a result of multiple national and multina-tional programmes, so a unified authority in charge of monitoring its evolution and of performing test and evaluation has not yet been identified. This section will briefly summarize the main SDR programs having some impact on European context, to better clarify the European SDR panorama.

1.2.1 The European Secure Software Defined Radio program (ESSOR)

The ESSOR program is the first multinational program for tactical SDR in Europe. It started in 2007 from a cooperation of six Nations (Spain, France, Finland, Italy, Poland and Sweden). The goals of the programs were the definition of a common SDR architecture based on the public part of the SCA 2.2.2 [16] and the development of a common, interoperable wideband tactical waveform called ESSOR High Data Rate (HDR) [17].

The ESSOR program completed the definition of the ESSOR architecture in 2010. The architecture has then been implemented into six different SDR platforms, each one developed within a national SDR program in every nation participating to ESSOR. Since then, the ESSOR Participating States have decided to offer the ESSOR archi-tecture to third party nations interested in implementing it within their national SDR program/equipment [18]. The release of the ESSOR architecture requires a sponsor-ship by one of the Participating States. The release of the ESSOR architecture is an important step towards the construction of an European SDR architecture, as it offers an interoperable and enhanced SCA-based architecture entirely developed in Europe.

The ESSOR architecture provided significant contributions to the SCA specification update, including:

(29)

• extension of the execution environment to DSP and FPGA; • extension of the connectivity based on CORBA to FPGA;

• extension of the definition of the major SCA entities (radio devices, radio ser-vices);

• definition of a complete specification for radio security services.

In addition, the ESSOR lesson learned provided significant contributions to the elabo-ration of the SCA 4.1 specification, allowing the definition of new application profiles allowing the implementation of a SCA-based architecture in small-factor platforms.

Finally the ESSOR community, has recently released to the Wireless Innovation Fo-rum (WinnF) the Transceiver Application Programming Interface (API) of the ESSOR architecture, and the release of the Timing Service API is foreseen in a short time frame. The APIs provided to the WinnF will serve as a basis for the elaboration of SCA related winnf products [19].

The second product developed by the ESSOR program is the High Data Rate (HDR) waveform.The development of the HDR waveform represented a cornerstone in the tactical SDR panorama, not only for the advanced capabilities of the waveform but also for the definition of a structured and common methodology for the design and development in order to ensure the maximum level of portability [20]. In the ESSOR program, in fact, the target platforms were designed and developed by each industry independently, and each platform had different hardware architecture, resulting from a different combination of processing elements (GPPs, DSPs, FPGAs). The internal structure of each platform was known only to the manufacturing industry. To develop a really portable waveform, a platform independent model (PIM) was adopted. Each development step is followed by a corresponding validation step. The development cycle of the ESSOR HDR is summarized in table 1.1

Table 1.1: ESSOR HDR development methodology

development step validation step

Dim C.1 Dim C.3

system design Hi Fidelity simulation

Base WF software development Native Test Environment (NTE) validation

WF porting on national PTF national test bed

interoperability test multinational test bed

The relationships between the different steps in the methodology are depicted in figure 1.3

The development and the validation of the base waveform are linked to the first two steps in table 1.1, while the steps depicted in the third and fourth row are linked to development and verification of the target waveform.

The first phase of the ESSOR methodology is the Base Waveform (BWF) system design and it is composed by three steps:

• Waveform/platform separation: in this step the entire set of requirements for the waveform is decomposed into a layered specification, assigning to each layer the related funcionalities. In this step two entities are identified:

(30)

Figure 1.3: overview of the ESSOR methodology for WF development).

– the waveform functionalities, that will be developed as portable software; – the functional support, defining the SCA entities that will be implemented in

each platform;

• base waveform partitioning: in this step all the functionalities allocated to the dif-ferent layers of the waveform are decomposed in Basic Software Item (BSI). The number and type of BSI obtained from the decomposition is a critical element for the final waveform performances and it is a trade-off between the great flexibil-ity obtained by a high level of granularflexibil-ity and the improvement of performances gained with a more monolithic design;

• base waveform mapping: once the number of BSI to be developed is determined, the programming language(s) to be used for the coding of the BSI are agreed. The choice of multiple programming languages (e.g. C and VHDL) implies that all the BSIs that can be allocated to multiple processing elements, will be developed in multiple instances, with different languages in the base waveform.

At the end of the design phase, the number and type of BSIs for each layer of the waveform, along with the interfaces among different BSIs and different layers are iden-tified. This step is the entry point to obtain a portable waveform: only if all the entities and all the interfaces are correctly identified, the development of the waveform will be successful. In the design phase, platform independent languages are used for the def-inition of each component: Universal Modeling Language (UML) for the components and Interface Definition Language (IDL) for the interfaces. At the end of this phase, the behavior of the waveform is simulated in order to verify that all the BSIs and the layers are properly connected. Figure 1.4 depicts the outcome of the design phase.

The second phase of the waveform design is the design of each BSI. In order to preserve the maximum portability, the BSI is splitted into two parts: the worker, that is the code executing the functional part of the BSI (that is to say, for example, the signal processing) and the container, that is in charge of ensuring the communications among workers and between worker and functional support. The worker implementa-tion is completely independent from the execuimplementa-tion environment (processing elements, connectivity broker used, architecture) and it is the pure algorithmic part of the BSI. The container is instead specific to an execution environment. The container, then, is the software element providing the worker with interfaces with the "external world"

(31)

Figure 1.4: outcome of the design phase).

(platform, other workers, other layers) according to the rules and choices of the under-lying architecture. For this reason, during the development of the ESSOR HDRWF, all the container needed to interface each BSI with the ESSOR architecture have been developed. The collection of the workers compose the so called golden source of the waveform: the golden source is the code containing all the waveform functionalities and totally independent from the execution environment (see figure 1.5) . At the end of

(32)

this step, a complete description of each worker in a platform independent language is available. In this sense the ESSOR methodology is compliant to the portability guide-lines defined in [15] and extends them separating the functional part of the BSI (the golden source) from the platform dependent container, that can be replace according to the connectivity mechanism implemented by the platform.

The next step is the coding of the base waveform. This step comprises writing the code with the language selected during the design phase. Specific coding rules have been defined inside the program to ensure the proper coding of each worker and container.

Once the golden source for a given BSI is fully developed, it is verified in a dedicated test environment developed within the ESSOR program: the Native Test Environment (NTE). The validation of a BSI in the NTE requires a porting operation: a proper con-tainer (Native Concon-tainer) is developed in order to link the BSI worker to the execution environment provided by the NTE. NTE allows the execution of extensive testing for the HDRWF with different granularity, and it can be used, to validate and test generic SDR components, not strictly belonging to HDR. NTE usage will be further explained in chapter 3.

The success of the ESSOR methodology is witnessed by the all the documentation provided by each validation step but also by the successful demonstration given by the ESSOR community towards NATO [21]. This demo was indeed the first documented case of five different SDR platforms interoperating with different target implementa-tion of a waveform jointly developed by six different industries. This demonstraimplementa-tion achieved a great value as it proved that even if the architecture of each national plat-form was known only to the manufacturer, the joint design and development procedure described, lead to the successful porting of BSI written by different industrial stakehold-ers. One of the key of this success was also the decision of the ESSOR participating states to select a unique industrial partner in each nation, that had a proven competence and reliability, called National Champion (NC). National champions involved in the ESSOR programme are Indra for Spain, Thales Communication Systems for France, Finmeccanica (toady Leonardo) for Italy, Bittium for Finland, Radmor for Poland and Saab for Sweden.

The ESSOR program is today entering into a new phase, where the enhancement of some features of the ESSOR HDR waveform and of the ESSOR architecture are foreseen, in addition to field testing of the radio developed during the first phase.

Under the umbrella of the commonly defined ESSOR architecture and HDR wave-form, each nation of the ESSOR community is running its own national SDR program, with the production of SDR platforms implementing the ESSOR architecture and with the development and porting on these platforms of SDR versions of legacy waveforms and with development of new waveforms for national use. This situation represents the perfect integration between the multinational effort of the ESSOR program and the national control granted to each participating State over its strategic choices for the national SDR program.

1.2.2 The German SDR Program: SVFuA

Together with ESSOR, the German SDR program called SVFuA [1] is the major Eu-ropean tactical SDR program. The SVFua program, in fact, lead to the definition of a

(33)

SCA 2.2.2 derived architecture and some SDR waveforms.

The basis of the SVFua program were laid down in 1998, with a cooperation pro-gram between France and Germany for the realization of a technological demonstrator called Multirole Multiband Radio–Advanced Demonstrator Mock-up (MMR-ADM). The outcome of the program was the development of an SDR platform (non SCA com-pliant as the SCA specification was not yet published) and the porting on it of seven tactical waveforms.

In 2007 Germany started a study on the possibility of porting the Wideband Net-working Waveform (WNW) on a SCA compliant architecture then in 2008 the archi-tecture of a SCA 2.2.2 platform compliant to the JTRS API was defined.

The first peculiar characteristic of the SVFuA program is the procurement strategy, that separates the procurement of the transceiver modules, of the main unit of the plat-forms and of the waveform. The main unit of the platform is in fact manufactured by Rhode & Schwartz while the transceiver modules operating in different frequency bands are manufactured by different vendors (Figure 1.6). To ensure a high degree of modularity, a specification for the interface between the main unit and the transceiver has been developed. This specification is called Open Baseband Interface Specifica-tion for SDR (OBISS) and has been proposed for standardizaSpecifica-tion within the Wireless Innovation Forum and NATO.

Figure 1.6: German SVFuA SDR and OBISS interface

The second major outcome of the SvFua program is the definition of a framework for waveform development and simulation and of a development platform that emulates the real SVFuA platform, where new developed waveform can be deployed and tested prior to the final porting (Figure 1.7).

The SvFua approach for waveform portability is then aligned with [15] with a dif-ferent interpretation of the so called "platform Independent model (PIM)". The pro-curement of the waveforms is done for their development on a waveform development environment (WDE) and not on the target platform. The waveform developed in this environment is the base waveform, that is ported on a platform procured from a differ-ent vendor and with a completely differdiffer-ent procuremdiffer-ent channel.

(34)

Figure 1.7: German SVFuA SDR porting methodology [1]

1.2.3 The COALWNW program

The COALWNW (Coalition Wideband Networking Waveform, read as "coalwin") [22] program was launched in 2009 by a cooperation comprising 9 nations, then enlarged to 11 nations. All the nations taking part in the ESSOR program are part of the COAWLNW program.

The initial goal of the COALWNW program was the development of a wideband coalition network with MANET features, intended to be used in multinational opera-tion, to overcome the lack of interoperability. In the phase 1 of the program, a complete set of requirements was defined, starting from the Operational Requirement Document (ORD). The Steering Committee of the program then decided to move from a devel-opment program to a procurement program: as some wideband SDR waveform were already appearing on the global market, COALWNW decided to opt for the procure-ment of a Non Developprocure-mental Item (NDI) best meeting the requireprocure-ment defined in phase 1. Phase 2 of the program then comprises the selection of a wideband waveform and a reference implementation of the waveform to be used for testing and compliance verification purposes. Phase 2 of the program is still ongoing as the steering Commit-tee is still assessing the candidate waveforms with the plan of completing this phase in 2017. Phase 3 of the program will focus on interoperability testing of the target waveforms (ported into national platforms) and compliance testing against the refer-ence implementation.

The importance of the COALWNW program is mainly due to the great number of participating nations. The waveform selected as COALWNW waveform will have an important role in the future tactical communication panorama and, as the great ma-jority of COALWNW nations are also part of NATO, it is very likely that the COAL-WNW waveform will be also adopted by NATO through a Standardization Agreement (STANAG).

(35)

1.2.4 The NATO Narrowband waveform

NATO recognized the lack of interoperability in modern tactical communications as a limiting factor, especially for coalition operation. as reported in [23] Unfortunately, the existing CNR communications equipment used by nations for this, usually in the 30MHz – 108MHz VHF band has not been included in any international standards, and each nation operates using different proprietary equipment. The report identify once again SDR as the enabling technology to reach the desired level of interoperability and, at the same time, points out how the lack of standardization in the SDR domain potentially leads to the risk to lose the opportunity to build a real interoperable communication system for coalition operation.

Based on these consideration, NATO issued in 2008 a first version of the require-ments for a common, set of interoperable waveforms [24]. According to the NATO analysis, the future battlefield needs will require at least two different waveforms:

• a narrowband waveform, using the standard channelization of 25 KHz in the VHF or UHF band, to be used for basic voice and data communications. Ideally this waveform should operate in the VHF band to take advantage of the greater range achievable;

• a Wideband waveform, used as backbone for bulk data transfer and as a transit network between different access networks. The wideband waveform will likely operate in the UHF band, with a limited range.

It is interesting to note that, according to the NATO definition, the bandwidth of the narrowband waveform is clearly defined in 25 KHz, for compatibility with legacy radio systems, while the wideband spectrum occupancy is undefined, giving only its lower bound. This situation is creating in the present an ambiguity that often results in the question "how wide is wideband?". A non -ambiguous definition of wideband waveform shall be agreed in international standardization bodies.

In 2011 [25] NATO defined the requirements for the development of a Narrowband coalition waveform with voice and data capabilities. One of the major requirements for the waveform was that it should be free of intellectual property rights (IPR) for specification, allowing its implementation by any interested party. The definition of the technical specification of the narrowband waveform was carried on by a "coalition of willings" including governmental research centers and Universities from different countries. In this sense the NATO NBWF is the first example of collaborative and open development of a tactical SDR waveform. Several technical solutions adopted in the NBWF have been presented in relevant conferences [26–28] and a reference implementation running on Universal Software Radio Peripheral has been presented by the Royal Belgium Military Academy [29].

NATO also defined the waveform sharing waveform that encompasses the develop-ment of a platform independent model, called again base waveform, which constitutes a working prototype of the waveform, running on a generic reference platform, that will be later used for the waveform porting [30].

The definition of the standard for the NATO NBWF was supervised by the NATO Line of Sight Communications Capability team (LOS COMMS Cat), and it is published in the Standardization Agreement (STANAG) family 5630.

(36)

Figure 1.8: NATO framework for waveform sharing

Currently no real working implementation of tactical radio operating NBWF are recorded but all the necessary definition steps have been carried out so the deployment to the market is expected in a short timeframe.

1.3

Challenges in V&V of military SDR

The overview of the major military SDR programs provided in previous section depicts a very heterogeneous panorama. For the architecture, if it is in fact true that almost all the tactical SDR programs today have implemented their architecture on the common reference of the SCA 2.2.2 it is also true that each program defined different APIs (i.e. JTRS APIs, ESSOR APIs) and defined different solutions for the communications be-tween the different entities of the system (e.g. OBISS for the communication bebe-tween transceivers and base unit, CORBA/non CORBA communication for data exchange between different software modules).

This situation comes from different strategic choices operated in each program and different background in software development for the participating industries. When it comes to Verification and Validation (V&V), some formal activities have to be per-formed to prove that the given SDR product (being it a complete system, a waveform or part of it, a platform) meets the desired set of requirements, also known as "quality expectation of the customer".

Te V&V process for a new system is not just linked to the execution of technical test procedures to verify the functionaities or performances of a product. It is true that the so called Test and Evaluation (T&E) is the process that implements the V&V policy, through the execution of formal methods to prove that the system under test has the desired features, but this is only the final part of the process. The formulation of an effective V&V policy indeed requires several steps.

The starting point of the process is to capture the user requirements. When a new SDR system is to be procured, the user requirements are usually expressed in the form of Military Operational Requirements (MORs). The wording of MORs indicates the operational capability that is desired from the system. At the stage of MORs

(37)

defini-tion it often happens that a requirement is stated in a very abstract manner, because no technical expertise is provided to the working groups. An extensive literature on program management methods for requirements definition is available and even with with a slightly different terminology, the different approaches agree that the definition of the requirements is a crucial step for the success in the development. The set of requirements shall not be expressed simply as a wish list but it has to identify specific features and capabilities that will be required to the system, together with the method-ology that will be applied for the verification of the requirement. For example,in [31] a requirement is defined as quality expectation that includes:

• the key requirement for the product (what is expected);

• the applicable standards or regulations to be satisfied (constraints); • the acceptance criteria with the related tolerances.

It is then clear that in the definition of requirements the V&V process has to be taken into account as the acceptance criteria implicitly define what procedures will be per-formed to verify the requirement fulfillment. A very meaningful formulation of this concept can be found in [32], if a T&E item or requirement is not in the Statement of Work (SOW), it probably will not be in the Request for Proposal (RFP), and if it is not in the RFP, it probably will not be in the contract.If it is not in the contract, do not expect to get it!

In the second step, the translation of the quality expectation into technical features for the system has to be performed. During this phase some architectural and imple-mentation choices are taken for the system and a definition of the corresponding T&E methods has to be done.

The translation has to differentiate between functionalities and performance require-ments and provide a compliance criteria for each of the identify feature. If a feature is identified as a functionality, the outcome of the V&V process shall be the evidence that the system embeds it or not. The result of the technical procedures aimed to verify functionality is then a sort of boolean value providing the "go/no go" for the system. The desired functionalities shall be clearly identified and allocated to the different part of the system (i.e. platform, waveform etc.) so that during the verification on the im-plementation, the component in charge of providing the given functionality is known a priori. The identification and allocation of functionalities to the different components of the systems also generates a possible list of constraints.

For example, let’s say that a functionality has been identified in "Push to Talk voice service" that is to say that the system shall provide the capability of transmitting real time broadcast voice when a button (PTT) is pressed. The functionality will be allo-cated to some components of the waveform (mainly in the physical and MAC layers) that shall be capable of providing quick access to the RF section of the system after the PTT button is pressed. The "quick" access is linked to the "real time" requirement for voice, where the requirement shall specify what is the maximum acceptable delay be-tween the PTT button pressure and the transmission of RF signal. The allocation of the feature then raise some constraints for the hardware of the platform hosting the wave-form: even if the waveform is capable to provide samples of the modulated voice at the physical layer within a specific timeframe, the access to the RF section will depend

(38)

on the performances of the frequency converter (in the transceiver) of the platform. For this reason, the fulfillment of the requirement is possible and demonstrable for the waveform (ie.in simulated environment) but only if the hosting platform meets some performance criteria.

The context is a little bit more complicated when it comes to performances. For a given functionality, given that it has been identified in the system, the system could provide it with different level of performance. For example let’s say that the capability of the SDR system to change the running waveform has been proven. There are several level of performance with which this can be provided, i.e. the instantiation of a new waveform could require the reboot of the device or not.

A possible way to perform a good technical translation and allocation of require-ments is differentiation between Measure of Effectivenes (MOE) and Measure of Per-formance (MOP), according to [2]. Even if a unique definition for MOEs is not yet agreed, They are shown to be “mission” or “purpose” oriented and not concerned

with the internal details of the solutions per se[33]. The MOEs represent the impact

of a given feature of the system on the mission for which it is designed, taking into account the operational context (i.e. scenario) where the system is to be deployed.

On the other hand, MOPs, will provide a quantitative measure, verifiable through technical procedures of the adequateness of the implementation of a desired feature in the system, against its technical specification. Differently from the MOEs, MOPs don’t take into account the operational scenario but address the technical details of the system implementation. A single MOE is build up upon several MOPs and the assessment of a group of MOPs usually require the involvement of several domain experts.

Figure 1.9: relationship between MOE and MOP [2]

To give an example, a MOE could be expressed in the form the radio shall be de-signed to allow a continuous mission up to 24 hours for a dismounted soldier. The related MOPs composing the MOE could for example be related to the battery dura-tion, size and weight for the radio, or the capability of the system to dissipate the heat, when fitted in the soldier equipment and powered up for 24 hours.

The definition of MOEs and MOPs is done during the design phase of the system and their verification is performed during the development in accordance to the V&V strategy. If one MOP, after formal verification, is not within the desired range, the impact on the parent MOE has to be assessed to determine whether it is more effective to change the system design or to accept a deviation from the original MOP [11].

(39)

has also to include field validation of the system. This is still in fact the only possible way to really assess the operational adequateness of the system and to prove its inter-operability with other systems and equipment used by the final users. Field test and interoperability testing require the involvement of experts in the operational usage of systems, with an in-depth knowledge of the doctrine, procedures and constraints of the military operations. Very often, this kind of test require to replicate the real operational scenario conditions.

1.3.1 The problem of standardization and certification

Certification is the set of activities intended to formally prove the compliance of a product to a given standard. Certification is defined as procedure by which a third party gives written assurance that a product, process or service conforms to specified

characteristics [34]. Given this definition, the certification implies the accreditation,

defined as “procedure by which an authoritative body gives formal recognition that a body or person is competent to carry out specific tasks. Certification then requires a number of entities to be involved in the process:

• an agreed standard that defines the requirements to be fulfilled by the system to demonstrate the conformance to specific characteristics;

• an accreditation/certification authority, who is in charge of the formal recognition that the system is compliant to the standard. This can often imply the analysis of reports from test activities but can be also limited to the acknowledgment of technical activities performed by the party requiring certification.

For military SDR certification is one of the most challenging matters. From a tech-nical point of view, the application of the SDR paradigm gives the opportunity to de-couple the waveform from the platform ensuring flexibility and cross-vendor interoper-ability. In reality, anyway, if a formal evidence of compliance to a set of requirements cannot be provided by an independent party (different from manufacturer), there is no confidence that a system developed within a given program (i.e. a platform) can be compatible with a system coming from another program. For this reasons, a certifica-tion strategy is needed to overcome the barriers among different programmes and to ensure the real harmonization of different SDR systems.

The need of certification as a cornerstone of a global SDR strategy brought the US to develop within their SDR programmes a certification facility, the JTNC Test and Eval-uation Laboratory (JTEL) which is in charge of evaluating all the SDR products and ensuring that they are compliant with the SCA requirements. A common policy [15] has been defined for the evaluation of waveforms and some dedicated tools for verifica-tion of SCA compliance has been developed, such as JTAP [35]. The policies and tools adopted in the US JTEL are currently the only mean to obtain the formal recognition of SCA compliance. In the US market this situation results in a very well defined path to obtain certification of new SDR products.One major drawback of this situation is that the certification of SCA compliance today is achievable uniquely through the US facility or through some dedicated agreements with US, as the tools and procedures are not publicly available. This situation deters non-US Government and industries from seeking the SCA certification and implies that the "official" number of documented

VERIFICATION AND VALIDATION (V&amp;V) ON MILITARY SDR SYSTEMS

V