Agenda Computer Vision @ STMicroelectronics

Computer Vision @ STMicroelectronics

Competencies and Advanced Research Topics

Alessandro Capra (alessandro.capra@st.com) Mirko Guarnera (mirko.guarnera@st.com)

AST – SI – Computer Vision Platform

UniCT – DMI – IPLab

About ST Alessandro Capra

AST – Computer vision Alessandro Capra Active Research Areas for

stage proposals

Mirko Guarnera

Agenda

Time

Speaker Presentation

2

About ST

12 April 2012

3

• A global semiconductor leader

• The largest European semiconductor company

• 2011 revenues of $9.73B⁽¹⁾

• Approx. 50,000 employees worldwide⁽¹⁾

• 12,000 people working in R&D

• 12 manufacturing sites

• Listed on New York Stock Exchange, Euronext Paris and Borsa Italiana, Milano

About ST

(1) Including ST-Ericsson, a 50:50 joint venture with Ericsson 4

An Unwavering Commitment to R&D

Access this map:http://g.co/maps/kg63 5

7 main technology R&D centers 39 design centers – 20,000 patents

12,000 people in technology, design, product and system R&D

Technology R&D centers Design Centers

12 April 2012

People are our Foundation…

36⁺ countries Location Headcount

Asia ~24,000

France ~11,000

Italy ~8,500

Malta & Morocco ~4,500 Rest of Europe ~2,500 United States ~1,500

43%

18%

24%

15% Direct

Manufacturing Indirect Manufacturing Research &

Development Sales, General &

Administrative

…working everyday to increase the quality and experience of life for all

6

Location Overview

Main distances:

• distance from airport 8 km

• distance from port 10 km

ST CATANIA LOCATION OVERVIEW

ST 3SUN

12 April 2012

7

Univ Degree 1402

High school 2480 Others

98

Catania Site: Workforce development

54% 25%

21%

R & D & Designers Manufacturing Product Management&Administration 8

AST – Computer vision

12 April 2012

Advanced System Technology

AST Mission

• To provide ST with leading edge innovation in Security/Imaging- Video/Connectivity, Ultra-Low Power & Healthcare ensuring an early IP access to all ST’s divisions

• To address R&D in new fields to guarantee an early access to new Markets &

Applications

AST Vision

• To Provide the Company with long-term core business and leadership in key markets by:

• identifying fundamental market changes

• ensuring early access to Intellectual Property

• developing system architectures and prototypes

• pioneering and testing new business approaches while sensing potential new markets

10

Computer Vision Platform:

Solutions for Product Innovation ₁₁

12 April 2012

IMAGING

Automotive Car Multimedia

Digital Consumer

Printers

Touch Imaging

Computer Vision Platform – Catania Lab

•People: 14

• Team active since 1999

• Patents: 80+

• Papers: 87+ in conferences, 30 in Journals

• Books: 1 book and several book chapters

•Joint Lab with IPLab – DMI – UniCT

• Active since 2007

• Shared challenging R&D

• PhD sponsorships, contracts

•Collaborations with UniCT, UniMI-Bicocca

12

CFA Sensor technology Wide gamut sensors Pixel Level Processing Auto Focus

Auto exposure

…

CV competencies

12 April 2012

Acquisition

Global Noise Management Color Space Management Demosaicing

Contrast enhancement Sharpening

Video stabilization

….

…..

CV competencies

Acquisition Processing

Channel coding

Lossless compression JBIG

JPEG - JPEG 2000 MPEG4

…..

CV competencies

12 April 2012

Acquisition Processing Storage

Face detection Red Eye Removal Blink Detection Scene Detection Panorama stitching

Key point descriptors for AR Advanced user interfaces

CV competencies

Acquisition Processing Storage

Services &

Post-processing

CV competencies

12 April 2012

Acquisition Processing Storage

Services &

Post-processing

Printers pipelines Projection

Display management 3D Graphics & AR

Rendering

Active Research Areas for stage

proposals

Active Research programs in AST

• Touch Panels

• Multi-touch

• Gesture recognition

• Proximity

• CDVS

• Keypoint descriptors

• Pico-Projection

• Color management

• Multi-sensor fusion

• Image analytics

• Scene detection

• Object detection

• Smart Cities

• Smart Camera applications

19

12 April 2012

Touch Displays

• AST exploits its competences in Image Processing and Analytics for 3D Touch Displays:

• Finger Approaching (direction, versus).

• Gesture Recognition (circle, pinch,..):

• Shape Recognition for ear/cheek proximity detection

• Multi-touch

• Proximity

• Target markets:

• Smartphone/Tablet 3.5” ~ 7”

• Netbook/Tablet 7”~12”

• Notebook/Tablet 10”~15.6”

20

Visual Search

 Capability to initiate a search using an image

Can be implemented in mobile phones

Mobile Visual Search

 Existing applications:

Google Googles

Bing mobile app

Nokia’s Point & Find

Snaptell

Kooaba

Like.com

 Essential functionality to enable Augmented Reality

 Interoperability with 3D Graphics Compression and Khronos OpenGL-ES needed

12 April 2012

21

Compact Descriptors

… …

AR Content Output of Video Analytics

Output of 3D Model &

Position Video Input

Searching Local/Remote DB for Augmented Information

Augmented navigation

Scene Classification: Purpose

•Scene classification (also known as Visual categorization) aims to automatically classify images, distinguishing among different classes, based on information extracted from the images.

23

12 April 2012

Target classes:

{Building, Open Country, Forest}

Building Image

representation

Classification model

Personal-projection

•Human interfaces

24

Final Fantasy 2005

 Augmented reality

 Head up display

Color Management for Accurate Color Reproduction

• Color Management allows mapping source color space information (sRGB, Adobe RGB, … ) into the output device color space such that colour accuracy is preserved.

25

12 April 2012

Color

Management Display

Source Image

Color Managed

Image Source

Gamut

Display Gamut

Smart Cities

Low-cost low-power interconnected smart

video cameras allow additional services

26

THANK YOU

12 April 2012

27