On-board monitoring system for road traf fic safety analysis
SebastianoBattiato,GiovanniMaria Farinella,GiovanniGallo,OliverGiudice*
ImageProcessingLaboratory,Dept.ofMathematicsandComputerScience,UniversityofCatania,Italy
ARTICLE INFO
Articlehistory:
Received31August2017
Receivedinrevisedform20December2017 Accepted25February2018
Availableonlinexxx
Keywords:
On-boardsystems Trafficsafety Stereovision
ABSTRACT
Thispaperpresentsaframeworkforroadtrafficsafetyanalysis.Itisbasedonastereo-visionsystemthat, afterbeinginstalledon-boardofpublictransportationvehicles,collectsdataofwhathappensinfrontofa movingvehicle.ThecollecteddataareanalysedthroughoutaprocessthatacquirerawGPSinformation, video sequencesand stereo-based depth maps to computethe surrogate safety measures. These measuresareobtainedbyexploitingtheTrafficConflictTechniqueinconjunctionwithcomputervision algorithmsandacascadeofclassifiers.Thesafetymeasuresarethenusedforfurtheranalysisinorderto identifydangerouslocationsinwhichaninterventionisneededtoimprovethesafetylevelandprevent accidents. Experiments performed in a real urban environment confirm the effectiveness of the framework.
©2018ElsevierB.V.Allrightsreserved.
1.Introduction
Roadaccidentsareoneoftheleadingcausesofdeath,therefore road safety is a crucial and a delicate matter for national governmentstobesolved.Overthelastyears,governmentshave launchedaseriesofinitiativeswiththeintentiontosignificantly reduce road accidents by acting directly on the three basic interacting elements of the road: the human behaviour, the vehicles security features and the road infrastructure. Human behaviouris the mostdifficulttodeal with, becauseit canbe predictedonlytoalimitedextend.Theimprovementsinvehicle safetyaredependentontheimplementationofnewtechnologies and security features by the manufacturers. Finally the road infrastructure hasa key role in safety influencing both human behaviourandvehiclesperformances.Relevantroadinfrastructure factorsincludethequalityoftheroadillumination,thepresence and the readabilityof signs and road markings as wellas the qualityofthepaving[1].Roadsafetyishencethefinaloutcomeof regulationsandpreventiontechniquesthat,actingontheabove mentioned three basic elements, make roads more secure. A methodtoobjectivelymeasurethesafetylevelofaroadmayhelp tochoosethemosteffectiveintervention.Theresearchforsucha measuremotivatedmanystudies.Allofthesecanbedividedinto twomaincategories:thosebasedontheanalysisofstatisticaldata ofaccidents[1]andthosebasedonroadusersanalysisandinter-
operation, called naturalisticstudies [2]. Mostof the statistical studiesareperformedbyevaluatingcrashdatacollectedovera longperiodoftime.Theyfocusonrigorousstatisticalmodelsand cantellsomethingaboutthelevelof safetyonlyaftera certain numberofeventshasalreadyoccurred.Howeverthenaturalistic studiescanfindthecausesthatcanleadtoanaccidentinorderto preventit.Theactualdifficultyinnaturalisticstudiesistocollect andaccessthedataneededforagoodoutput.Currenttechnologies offertheopportunitytocollectdatacontinuouslywithnetworksof cameras, wireless sensor networks and on-board monitoring systems.Theframeworkpresentedinthispapercanautomatically collect data, while moving on board of a vehicle, and locate dangerousplacesforroadusersinordertosendalarmstoexperts andallowapromptinterventiontoimprovethesafetylevelofthe location.Tothisaim,theframeworkiscomposedofavideo-based on-boardmonitoringsystemthatexploitsthemodeloftheTraffic ConflictTechnique(TCT)tocomputesafetymeasuresinconjunc- tion with computer vision techniques and classification algo- rithms. This framework mounted properly on a public bus transportation system, which runs every day the same roads, cancreateadistributedmovingsensornetworkwhich continu- ously collects data directly on the field. Compared to other solutionsofthestateoftheart,theproposedapproachprovides moreinformationrelativetopre-crashandcrasheventsthanwhat iscurrentlyavailable,it islessexpensive.Moreover itallowsto investigatewithfewsensorsaverylargeterritoryandithassafety riskdetectionwithreal-timecapabilities.Finallyitisnotbasedon theanalysisof thedriver behaviour,but it focuses onlyonthe interaction between road users, making it less intrusive with respecttoprivacyissues.
*Correspondingauthor.
E-mailaddresses:battiato@dmi.unict.it(S.Battiato),gfarinella@dmi.unict.it (G.M.Farinella),gallo@dmi.unict.it(G.Gallo),giudice@dmi.unict.it(O.Giudice).
https://doi.org/10.1016/j.compind.2018.02.014 0166-3615/©2018ElsevierB.V.Allrightsreserved.
ContentslistsavailableatScienceDirect
Computers in Industry
j o u r n a l h o m e p a g e : w w w . e l s e v i e r . c o m / l o c a te / c o m p i n d
Thepaperisorganisedasfollows.Thestateofthearttechniques relatedtotrafficanalysisapproachesarediscussedinSection2.We then introduce the reader to the Traffic Conflict Technique fundamentalsinSection3.Theproposedframeworkisdescribed inSection4.Section5discussestheexperimentalresultsobtained consideringrealdatacollectedinthecityofCatania–Italy.Finally, conclusionanddirectionsforfutureworksaregiveninSection6.
2.Relatedwork
Effortstoreducerisksandimprovesafetyonroadsaremade throughmanystrategiesthattypicallyinvolveprocessesofdriver screeningandselection,drivertraining,vehiclemaintenanceand roadsafety inspections in additiontothestatistical analysisof accidentdata.
Whilethoseapproacheshavehadsomesuccess,theyareslow andveryexpensiveandoftentheyfailtohaveasufficientlybroad and reactive view of the reality in order to swiftly identify problems for road safety and thus to prevent accidents [1].
Naturalistic studies have demonstrated to be more reactive in identifyingproblems for safety [2] and nowadays technologies offertheopportunitybetterresultswithsystemsthatcandetect problemswithsafetyandautomaticallytriggeraninterventionto solveit.Itispossibletodifferentiatebetweenthosesystemsbased onstaticpositionedsensors(singleormultiple)andthosebasedon sensorsmountedonboardofvehicles.
In[3,4]manytechniques werepresented,mostly focusedon driverbehaviouranalysis.Theirstudiesgivetheinspirationfora possiblebroadclassificationofallanalysistechniquesinto:Real- Timetechniques,abletoidentifyasafetyissueatcurrenttimeand non Real-Time ones, able to collect and analyse data from numeroussamples.
Basedonthedevicesexploited,itispossibletofurtherclassify thestateofthearttechniquesinto:fixed-positionsensornetworks andon-boardsystems.
Thereisahugenumberofstudiesthatusenetworksofsensors placedincriticalpositions,specificallytostudythetrafficflowor thevehiclesbehaviour[5–8].Thesestudiesusevarioustechniques tocollectandanalysevideodataexploitingtheTTCmodel[6–8]in ordertoidentifypre-crashsituations.Theyareveryreliableand have a limited cost. However the positioning factor is crucial becausethesesystemsallowyoutomonitoronlyareasthatalready arethoughttobedangerousthereforedonothelptoidentifyhigh- riskareaswhicharenotknown.Inadditionroadsandthetraffic flows constantly change, so fixed-placed solutions need to be constantlyupgradedandre-positionedaswell.
On-boardsystems allowtoknow what happens around the vehiclewhilemovingtogetherwithit.Mostsystemsarebasedon thesensorsofthevehicleitself[9](speed,acceleration,GPS,etc.), whileothers[10]arevideo-basedandcanachievegreataccuracy in identifyingdrivers behaviour. Horreyet al. [11] presented a detailed review of the stateof theart of on-board monitoring systems.However,despitetheyhavegoodaccuracyinidentifying driver'sbehaviourtheirsolutionlacksindetectinglocationswhere roadsafetyiscompromised.
Asregardson-boardsystems,mobile-phonebasedsolutionsare promisingintermsofbeingabletocollectaverybigamountof dataexploitingthewidespreadofmobiledevicesamongpeople.
Amongothers in [12] mobilephone sensors (GPSreceiver and orientationsensor)wereusedtodetecterraticdrivingbehaviour caused by overtaking considering phone as a steering wheel.
Anotherinterestingstudythatusesthethree-axisaccelerometer sensors of smart-phones was developed in [13]. The system presentedbytheauthorsclaimstobeusedtodetectandanalyse variousexternalroadconditionsanddriverbehavioursthatcould be dangerous to the health of the driver, automobile and the
neighbouring public. In [14] the authors proposed a vehicle mountedsystemwhichprovidesasafetyindextoeachdriveron thebasisofhisqualityofdriving.Theandroiddevicewasfixedto the windshieldand capturedthe audio and video signals (10s before and after the event) and in order to detect driving manoeuvreslikeaggressiveturnandsuddenbrakeetc.,thresholds weredefined fortheaccelerometer in each direction.Asimilar studywasdevelopedin[15]wheretheauthorspresentedamobile applicationthatcombinesthesensorsofmobilephonelikeGPS, accelerometer and microphone to detect the driver behaviour, trafficandroadconditions.
While promising mobile-based studies work in extremely unconstrained conditions and with a various quantity of low- quality sensors. Thus the overall quality of collected data is extremely low. Moreover, given the big amount of data, the analysisandinsightextractionshouldbedoneoffline.
Studiesconducted in [16] demonstrated theeffectiveness of usinganon-boardmonitoringsystembasedonstereo-visionto automaticallycomputemeasuresusefultocomputealevelofrisk inadetectedpre-crashevent.
Asdiscussedin[17],theresearchanddevelopmentofon-board monitoring systems should ideally: (a) identify and validate behavioursthatmaybeprecursorstocrashes orinjuries;(b)be basedoncost-effectivewaystomonitorbehaviour;(c)establish managementanddriveracceptanceoftheprogram.
Inthispaperwepresenttheimplementationofaframeworkfor road safety analysis that satisfies all of the above mentioned conditionsandfillsthegapinthefieldofautomaticroadsafety analysiscombininghigh-qualityreal-timevideoandsensordata collectionandanalysistechniquesinamixtureofreal-timeand nonreal-timeanalysis.
3.TheTrafficConflictsTechnique
The ‘Heinrich Triangle’ theory [18] provides a conceptual frameworktoreasonaboutroadaccidents(Fig.1).Itisfoundedon therelationshipthat‘no-injuryaccidents’precede‘minorinjuries’ (i.e.,eventsclosertothebaseofthetriangleprecedeeventsnearer thetop).Moreover the‘Heinrich Triangle’events nearthebase occurmorefrequentlythaneventsnearthetop.
Fig.1.TheHeinrichTriangledescribescrasheventsintermsofinjuryseverityand nearcrashorpotentialcrasheventsintermsofrisk.Eventsatthebaseofthe trianglehappenmorefrequently.
TheTrafficConflictTechnique(TCT)takesofffromthe‘Heinrich Triangle’theory,assumingthattheappropriateTrafficConflict(TC) factorscanbedefinedasmeasuresofnear-crashevents.ATCis definedasanobservablesituationinwhichtwoormoreroadusers approacheachotherinspaceandtimetosuchanextentthatthere isriskofcollisioniftheirbehaviourremainunchanged[19](e.g.a pedestriancrossingtheroadandavehicleapproachingthatdoes notbrakeorchangedirection).
The TCT is based on the measurement of both spatial and temporalvariablesdescribingtheinteractionsbetweentworoad users(e.g.,acarandapedestrian)involvedinacriticaleventfor safety.TheTimeToCollision(TTC)isthemainindicatorusedbythe Traffic Conflict Technique and it is computed for the vehicles (TTCv)andobstacles(TTCo).FormallyTTCisdefinedasfollows [20,16]:
TTC¼D
V ð1Þ
whereDisthedistancebetweenthesubjectandtheconflictpoint andVisthevelocityoftheroadusers(thevehicleorthegeneric obstacle).
Considering theeventdynamicsit ispossibletodifferentiate the Traffic Conflicts into three major classes with related TTC measures(Fig. 2: Pedestrian Conflict, Car Following Conflict and IntersectionConflict.
Incasetheobstacleisapedestriancrossingtheroadwehavea PedestrianConflict.TheTTC measures,related tothisconflict,at
timeiforthetwoinvolvedactors(i.e.,vehicleandpedestrian),are definedasfollows:
TTCiðvÞ¼DziðvÞ
ViðvÞ; ð2Þ
TTCiðpÞ¼DxiðvÞDxiðpÞ
ViðpÞ : ð3Þ
wherebyconsideringa3Dreferencesystemcentredonthevehicle, DziðvÞ represents the distance between the vehicle and the pedestrian along the Z axis and DxiðvÞDxiðpÞ represents the distancebetweenthevehicleandthepedestrianalongtheXaxis (seeFig.2c).LetbeTfithetimerequiredforthevehicletostopat timeidefinedas:
Tfi¼TrþViðvÞ
af ð4Þ
whereTristhereactiontime,ViðvÞisthevehiclevelocityattimei, af is the deceleration during braking. Taking into account the previousconsiderationsitispossibletodifferentiatethefollowing cases,whichdefinetheareasmarkedinFig.2d:
TTCiðvÞ>Tfi:vehiclemaystopbeforeconflictarea,
TTCiðpÞ>TTCiðvÞ:thevehiclepassestheareaofconflictbefore thepedestrianreachesit,
Fig.2.ThethreeclassesofTC:(a)Intersection,(b)CarFollowing,(c)PedestrianCrossing,(d)TemporaltrendofthequantitiesinvolvedintheTTC.Theareawhere TTCo(i)<TTCv(i)<Tf(i)isrelatedtotheconflict.Eachactorhasareferencesystemintegraltoitself.
TTCiðpÞ<TTCiðvÞ<Tfi:PedestrianConflict.
If the conflict exists it is possible to evaluate its duration (TTZduration)asthedifferencebetweenthefinalandinitialtime(if
i0)inwhichTrafficConflictconditionsaresatisfied.
ThesecondtrafficconflictclassistheCarFollowing(Fig.2b)in whichtherearetwovehiclesmovingalongthesamedirection.The conflictoccurswhenthevehicleinfront(v1)ofanothervehicle(v2) makes a rush braking or a fast deceleration. If thedistance to actuallystopforvehiclev1isequalorminortothedistanceofthe vehiclev2infrontofit,thecrashwilloccur.TheTTCmeasureat timeiforthesecondvehiclev2canbedefinedasstatedin[21]:
TTCi¼ Vi2
ðv1Þ
2Viðv2Þaiðv1ÞViðv2Þ 2aiðv2Þþ Dyi
Viðv2Þ ð5Þ
whereDyiisthedistancebetweenthevehiclesalongtheYaxisat time i, Viðv1Þ and aiðv1Þ are respectively the velocity and decelerationof thefrontvehicleat timei, Viðv2Þandaiðv2Þare respectivelythevelocityanddeceleration oftherearvehicleat timei.
IfTTCi<Tfi,withTfidefinedasinEq.(4),thevehicleV2couldnot stopin time, the conflict occurs and it is possible toevaluate TTZduration.
ThethirdtrafficconflictclassistheIntersectionConflict(Fig.2a).
ThisclassisverysimilartothePedestrianConflictoneassuming thattheinvolvedactorsarenowtwovehicles.Thedifferenceisthat inthisclassmuchgreaterspeedsareinvolved.Forthisreasonitis mandatorytotakeintoaccounttheTfiofbothvehiclesfromthis assumptiontheTTCmeasuresofbothvehiclesaredefinedas:
TTCiðv1Þ¼Dyiðv1Þ
Viðv1Þ; ð6Þ
TTCiðv2Þ¼Dxiðv2Þ
Viðv2Þ: ð7Þ
where by considering a 3D reference system centred on the consideredvehiclev1:Dyiðv1Þrepresentsthedistancebetweenthe firstvehicleandthesecondalongtheYaxis,DxiXðv2Þrepresentsthe distancebetweenthesecondvehicleandthefirstalongtheXaxis, Viðv1ÞandViðv2Þrepresentsrespectivelythevelocitiesofthefirst vehicleandthesecondonealongtheirdirectionoftravel.
If both conditions TTCiðv1Þ<Tfiðv1Þ and TTCiðv2Þ<Tfiðv2Þ happen, the conflict occurs and it is possible to evaluate TTZ(duration).
GiventheTTZdurationitispossibletocomputetheseverityofthe conflictas RiskImpact(RI)at timeiaccordingtothefollowing equation:
RIi¼ X
i2TTZduration
V2v
iDTi ð8Þ
where Vvi is the velocity of the vehicle at time i and
DTi¼TfiTTCðviÞ.
Itis alsopossibletoevaluatetheRIofthewholeconflictas follows(accordingto[16]):
RI¼ N1 PN
i RIi
TTZduration ð9Þ
Thediscriminationbetweenthethreeclassesofconflictcanbe doneautomatically.TheCarFollowingcaseiseasilydistinguishable from the others by analysing the magnitude and direction of speedsofbothvehicles.Ontheotherhandtodiscriminatebetween intersectionandpedestrianclassestheTTCmeasuresalonearenot
sufficientandvisualinformationisneeded.Automaticclassifica- tionofTCclasseswillbeintroducedinSection4.5.
4.Frameworkdescription
Theproposedframeworkiscomposedofseveralmodulesthat operate in different phases. For clarity we first describe the hardware components in Section 4.1 whereas the software modulesthatrunontopoftheminthesubsequentsections.
4.1.Hardwarecomponents
The hardware employed in the proposed framework is composedofanumbersensors,processorsandnetworkcompo- nentsthatareconnectedtogetherasdescribedinFig.3.
TheTYZXDeepSeaG3EmbeddedVisionSystem[22](EVS in Fig.3)isanembeddedstereoimageprocessorwhich isusedto computethedepth-mapofthesceneinfrontofthevehicle.The designoftheEVSisbasedonanarchitecturethatimplementsthe Censusstereovisionalgorithm[23].Astheinputpixelsenterthe EVS,theCensustransformiscomputedateachpixelbasedonthe localneighbourhood,resultinginastreamofCensusbitvectors.
For each pixel of one image of the stereo system, a summed HammingdistanceisusedtocomparetheCensusvectorsaround thepixeltothoseatthewindowlocationsintheotherimage.These comparisons are pipelined and occursimultaneously. The best match,shortestsummedHammingdistance,islocatedwithfive bits of sub-pixel precision. The EVS Processor converts the computedpixeldisparitytometricdistancemeasurementsusing the stereo camera calibration parameters and the depth units specified by the user. To have a system able to compute the distanceofanobjectfromthedriverperspective,weneededto takeintoaccounttheworkingrangesandtheHFOVoftheEVS.We used a baseline of 33cm and an83 degrees HFOV lens in our experiments. This configuration has a working distance range between2.5and50m.ThespecificationsoftheEVSusedinour experimentsaredescribedinTable1.
Fig.3showshowtheEVSisconnectedwitharoutertoaGPS module anda notebook. Theroutersolves theEVSdata access problem.AgigabitEthernetWLANroutermanagesanetworkin whichtheEVSisaccessedbyanotebook.Aclientcanaccessthe EVSbyconnectingtoaTCPservicethatsendseveryimageandthe computeddepthdata.Since therequiredstereovideosmustbe geo-referenced we also connected the EVSto a Bluetooth GPS receivermodule.Thechoiceofawirelessdeviceismandatoryto solvethesignalsatelliteproblem,sincewehaveusedacitybusfor ourexperiments.TheshapeofacitybusforcedustoputtheGPS moduleonthevehiclerooftohavethebestsignalquality.Finallya notebook PC connected tothe LAN acts as data collector. The
Fig.3. Theconnectionschemeofallthesensorsthatacquirerawdataduringthe inputphase.
