Equipe Intermedia équipe TIPIC labo CNRS SAMOVAR - Bernadette Dorizzi
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Equipe Intermedia équipe TIPIC labo CNRS SAMOVAR Bernadette Dorizzi http://www.telecom-sudparis.eu/eph Bernadette.Dorizzi@it-sudparis.eu Institut MinesTélécom;Télécom SudParis Institut Mines-Télécom
Intermedia une équipe bi-localisée EVRY-Nano-INNOV, Palaiseau ■ Projet Bio-Identité: ● Bernadette Dorizzi ● Mounim El Yacoubi ● Sonia Garcia ● Dijana Petrovska ● Doctorants: Mouna Selmi, Mohamed Ibn Kheder, Nadia Othman, Janio Canuto, Raida Hentati ■ Projet Geste: ● Patrick Horain ● Doctorants: Maher Mkinini, ■ Projet télévigilance ● Jerome Boudy ● Jean louis Baldinger ● Doctorants: Pierrick Mihorat, Toufik Guettari, Mohamed Sehili ● Post doc Paulo Cavalcante pa ge Institut Mines-Télécom 2
Research Context ■ Propose and adapt models from signal and image processing, statistical pattern recognition for various applications related to interactions between Man and Machine ■ Scientific background ● Statistical Pattern recognition ● Machine Learning ● Image processing, signal processing ● Sensors and micro-electronics ● Multi-sensor data fusion ■ Models: ● HMM, particular filtering, PSO, entropy, DTW, Gabor wavelets, ACP, LDA, Appearance and registration models. pa ge Institut Mines-Télécom 3
Intermedia : Interactions for Multimedia 7 permanent staff, PhD students, post-doc ■ Application domains: ● Biometrics (including iris, 2D-3D face, speech, signature) ● Crypto-biometrics (revocable biometrics and crypto- biometric key generation from biometric data) ● Audio Indexation using the speech recognition framework (publicity detection, music,…) ● Avatars personalisation ● Action and Activity recognition ● Home healthcare activity ● Gesture-based communication in networked virtual environments ● Person re-identification in videos ● Video indexing and retrieval through face and silhouette Institut Mines-Télécom
Intermedia 1/… Biometrics Bernadette Dorizzi Mounim El Yacoubi, Sonia Garcia Dijana Petrovska, Intermedia/TELECOM SudParis Authentication of persons from their personal characteristics (physiological, behavioural) Institut Mines-Télécom
3D morphable model From side view to frontal view Dynamic Signatures • Design of models relying on continuous HMMs • Tests on different databases (Philips, BIOMET, MYCT, SVC2004) •Participation to the international evaluation campaigns SVC 2004, BioSecure 2007, BSEC09 Combining Biometrics and Cryptography Iris verification from for Secure Authentication degradated acquisition conditions Differential sensors IR for face and Télécom hand vein verification SudParis Interests •2D and 3D face recognition Multimodal fusion • Scores Fusion with means models, SVM, décision trees • Evaluation Protocols on multi-modal databases Usage Tests and field studies in the context of biometric systems deploiement. pa ge 6
Research problems ■ Development of new algorithms ● On-line signature (patent) ● Iris verification in degraded mode (patent) ● Face Verification in 2D and 3D ● Quality assessment ■ Multimodality : Development and test of score fusion algorithms, independance tests, feature selection and fusion (1 PhD) ■ Biometrics and Security ● Development of new crypto-biometric strategies ■ Coupling sensors/algorithms (collaboration Société NIT, Yang Ni) ● Differential image sensor able to decrease illumination effects ■ Assessment protocols for biometric algorithms and multibiometric algorithms (projet BioSecure) ■ Biometric implementation on embedded systems (PDA, mobile devices) (projets VINSI, SecurePhone, SIC) ● Taking into account degradations linked to mobility, Interest of multibiometry Signature verification on iphone, ipad, android platforms Usage tests and field studies in the framework of deploiement of biometric systems. Institut Mines-Télécom
Principaux résultats de recherche ■ Reconnaissance de visage en 2D avec conditions d’illumination variables par codage de Gabor et projection par LDA ■ Vérification par le visage en mode proche infra-rouge: méthode d’appariement locale basée sur les points de contour ■ Qualité locale mesurée par GMM pour reconnaissance de l’iris en mode dégradé ■ Mesure d’entropie pour qualifier des qualités de signatures manuscrites en-ligne et la texture d’iris ■ Modèles de “Morphing” 2D-3D pour la reconnaissance de visage avec variations de pose, pour l’animation d’avatars 3D ■ Fusion multimodale de scores, d’images et de caractéristiques ■ Sélection de variables par méthodes “Optimisation par Essai de Particules” ■ Sécurité des systèmes biométriques (biométrie révocable et crypto- biométrie) Crypto-biométrie : obtention d’une clefs crypto- biométrique de 147 bits à partir d’images d’iris et de visage, proposition de nouveau protocoles ■ Indexation audio en utilisant la principe de reconnaissance de la parole (détection de publicité, musique,…) excellents résultats sur données YACAST (diffusions radios) et QUAERO ■ Ré-identification dans des videos par approches SURF et représentation parcimonieuse 8 Institut Mines-Télécom SSD 2013
On- line signature : Personal Entropy Measure Nesma Houmani, Sonia Garcia-‐Salice3 ■ We proposed a Personal Entropy Measure computed locally on a set of genuine signatures [1,2,3] ■ We generated with such measure 3 writer categories, coherent in terms of signatures’ visual aspect, complexity and variability. ■ We tested 3 classifiers on each category of users and compare results There are users by far more difficult (a factor 2) to recognize than others: High Personal Entropy category [2,3] ■ We studied in [4] the Robustness of Coordinates, Pen Pressure and Pen inclination angles to Time Variability with Personal Entropy (x,y) is the most robust combination to long-term time variability (as observed by other criteria and performance assessment) . [1] S. Garcia-Salicetti, N. Houmani, B. Dorizzi, "A Client-entropy Measure for On-line Signatures", Proc. of IEEE Biometrics Symposium (BSYM 2008), Tampa, USA, September 2008. [2] N. Houmani, S. Garcia-Salicetti, B. Dorizzi, "A Novel Personal Entropy Measure confronted with Online Signature Verification Systems’ Performance", Proc. Of IEEE Second International Conference on Biometrics: Theory, Applications and Systems (BTAS 2008), Washington, September 2008. [3] S. Garcia-Salicetti, N. Houmani, B. Dorizzi, "A Novel Criterion for Writer Enrolment based on a Time-Normalized Signature Sample Entropy Measure", EURASIP Journal on Advances in Signal Processing , vol. 2009, Article ID 964746, 12 pages, 2009. doi: 10.1155/2009/964746. [4] N. Houmani, S. Garcia-Salicetti, B. Dorizzi, "On assessing the Robustness of Pen Coordinates, Pen Pressure and Pen inclination to Short- term and Long-term Time Variability with Personal Entropy", Proc. Of IEEE Second International Conference on Biometrics: Theory, Applications and Systems (BTAS 2009), Washington, September 2009. Institut Mines-Télécom
Combining Biometrics and Cryptography for Secure Authentication Sanjay Kanade, Dijana Petrovska Problem: Biometric data is not revocable; cannot replace and reissue the template in case of compromise Solution: Use biometrics in combination with password; both must be provided at the same time for the system to work; there is no sequential processing Revocable / Verification Yes/No Enrollment cancelable Password Key regeneration Key template (101010011101010100011 010101010101110101101. .. …) Password Our work: ■ A shuffling scheme is used which makes the biometric enrollment data (template) revocable ■ Different templates can be issued for different applications; So user privacy is preserved ■ Stored data does not reveal information about the biometric enrollment data ■ Reduce variability in biometric data using error correcting codes [2]; improves biometric performance ■ Cryptographic keys having 83-bit entropy in single eye mode and 147-bit entropy in two-eye mode can also be obtained; this is the highest reported entropy in literature 1. Kanade, S.; Camara, D.; Krichen, E.; Petrovska-Delacrétaz, D. & Dorizzi, B. “Three Factor Scheme for Biometric-Based Cryptographic Key Regeneration Using Iris”, The 6th Biometrics Symposium 2008 (BSYM2008), 2008 2. Kanade, S.; Petrovska-Delacrétaz, D. & Dorizzi, B., “Cancelable Iris Biometrics and Using Error Correcting Codes to Reduce Variability in Biometric Data”, IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2009 3. Kanade, S.; Petrovska-Delacrétaz, D. & Dorizzi, B., “Multi-biometrics based cryptographic key regeneration scheme”, IEEE Conference on Biometrics: Theory, applications and systems (BTAS), 2009 Institut Mines-Télécom 10
Gait Recognition Mounim El Yacoubi, Ayet Shaiek, Mohamed Ibn-Kheder, ■ Silhouette Normalization ● Normalizing the varying distance from each gait frame to camera ■ Gait Period Detection ● Analysis of the Density of Foreground Pixels associated with the legs ■ Feature Extraction ● Vector of the Widths (Structural-Dynamic Information) ● Motion Vector between 2 Consecutive Frames (explicit dynamic information) ■ Recognition ● Hidden Markov Models (HMMs) − Explicitly divide the gait sequences into its periods − Robustness toward Gait sequence local variations ● Dynamic selection of the feature type based on: − Kmeans distance − PCA output − Output of a generic HMM Cf. Ayet SHAIEK, “Reconnaissance des personnes à partir de la démarche, “ Master Report, June 2009 Page 11 Institut Mines-Télécom
Person Re-identification System based on SURF Matching (M El Yacoubi, M Ibn Kheder, B Dorizzi) Detection of the Test Video Interest Region Motion and Silhouette Detection Interest Points (SURF) Extraction Majority Vote Interest Rule Points Reference Matching Database of SURFs 12 17 Institut Mines-Télécom SSD 2013
Experimental Results " Re-identification Performance = Correct Classification Rate (CCR) Re-Identification = f(Angle Difference) Re-Identification = f(Angle Pairs) CASIA Dataset-A : 20 persons 6 Different View Angles 2 Sequences for each View Angle 1 sequence as Reference 1 Sequence for Test 13 Institut Mines-Télécom SSD 2013
SURF with SPARSE representation/ Results on PRID-2011 PRID-2011: 749 persons as Reference. 200 persons for Test. In average,100 images per person. Approach Re-identification rate Combination of 19.18% two methods [18] Probabilistic 22% SURF Matching [25] Our approach with 28% sparsity CMC curve 14 Institut Mines-Télécom SSD 2013
Human Activity Recognition in videos Mouna Selmi, Mounim El Yacoubi, Bernadette Dorizzi Modelization: Project Juliette (Feder FP7) • Video = Activities sequences Partners : Telecom SudParis, Aldebaran • Activity = Sequences of elementary actions Robotics, Brain Vision Systems (BVS), Features: spatio-temporal interest points Institut de la vision Optic flow ■ Compagnon Robot at home for restricted Hierachical Classification mobility persons Probabilistic Model: Conditional Random Fields ● Old, visually disabled, … (CRFs) ● Acquisition of video sequences of humanLight integration of features of different natures activities ■ Challenge ● Simultaneous segmentation and Recognition Page 15 Institut Mines-Télécom
Biometric Multimodal Fusion Lorène Allano, R. Raghavendra, Bernadette Dorizzi ■ Score Fusion Scheme, dependency measures, building of efficient virtual databases ■ Feature and Image Fusion schemes, Particle Swarm Optimization feature selection •Lorene Allano, Bernadette Dorizzi , Sonia Garcia-Salicetti, « Tuning Cost and Performance in Multi-Biometric Systems: A Novel and Consistent View of Fusion Strategies based on the Sequential Probability Ratio Test (SPRT)” PRL (2010) •Raghavendra.R, Bernadette Dorizzi, et al.“Designing Efficient Fusion Schemes for Multimodal Biometric System using Face and Palmprint” submitted PR (2009) Institut Mines-Télécom
3D static model for pose transformation (Dijana Petrovska, D. Zhou) Original with landmarks Result Institut Mines-Télécom 17
Avatar personalisation with a 3D model from a 2D image 2 (Dijana Petrovska, D. Zhou) 3D Scans Database for Building the Morphable Model 3DMM Face Space High Resolution (75k vertices) Face Analyzer 3D Photo-Realistic 2D Input Image Representation 18
Iris verification in degraded mode Good quality iris image Constrained acquisition conditions Bad quality iris images Less Constrained acquisition conditions « iris on the move » 19 Institut Mines-Télécom SSD 2013
Global system for iris recognition Segmentation Normalization 0100001100010011 Encoding 1110111111000101 Match ? 20 Institut Mines-Télécom SSD 2013
Recent Results in the bio-identity project ■ PhD Emine Krichen/patent 2007: ● Definition of local quality measure relying on a GMM for estimating “good quality” iris texture ■ Thales collaboration (2008-2013) : 2 CIFRE PhD ● S. Cremer, B. Dorizzi, S. Garcia, and N. Lemperiere. How a local quality measure can help improving iris recognition. BIOSIG, 2012. ● T. Lefevre, B. Dorizzi, S. Garcia, and N. Lemperiere and S. Belardi, Effective Elliptic Fitting For Iris Normalization, Computer Vision and Image Understanding / parametric active contour ● T. Lefevre, B. Dorizzi, S. Garcia, and N. Lemperiere and S. Belardi,New Segmentation Quality Metrics for Iris Recognition, submitted to ICIAR 2013 21 Institut Mines-Télécom SSD 2013
Iris Segmentation via Triplet Markov Tree Collaboration with Wojciech Pieczynski: TSP/CITI ■ Unsupervised eye image segmentation via Triplet Markov Trees as a first processing for subsequent iris segmentation Triplet Markov Tree (Column 3) Improves segmentation quality versus Hidden Markov Tree (Column 2) Two different localizations of the normalization circles for the same eye image: wrong (grey-level pixel based) on the left; good (TMF-based) on the right. Dalila Benboudjema, Nadia Othman,Bernadette Dorizzi, Wojciech Pieczynski, “Challenging eye segmentation using triplet markov spacial Models”, accepted at ICASSP 2013 22 Institut Mines-Télécom SSD 2013
Improving Video-Based Iris Recognition Via Local Quality Weighted Super Resolution Nadia Othman, Nesma Houmani, Bernadette Dorizzi, ICPRAM 2013 ■ Iris recognition at a distance and on the move: Video Uncontrolled acquisition: Less constraints . Loss in quality Eye localization Iris extraction Lack of resolution, blur. Our proposal: Strong occlusions: eyelids, eyelashes, Fuse the frames in the video spots… to get more information from the person 23 Institut Mines-Télécom SSD 2013
Spoofing detection via OCT approaches Projet PARADE; Collaboration Yanneck Gottesman TSP/EPH 2 pending patents, August 2012 Context: fingerprint recognition Actual fingerprint systems: due to the 2D surface acquisition modes, detection of spoofing is very difficult (death finger, false finger, overlay) The quality of the recorded image presents a significant variability due to external parameters (temperature, humidity, pressure) Our proposal: ■ Development of a new type of sensor (3D OCT imaging sub-cutaneous) to test the living character of the biological tissue presented to the sensor ■ Signal processing / 3D volumes to allow biometric recognition and detect attempted identity theft 24 Institut Mines-Télécom SSD 2013
Finger with an overlay ■ Overlay depth < dimensions groove (sillon) Without overlay With overlay Difficult because the overlay depth is inferior to the OCT resolution Detection is possible as we use the phase image 25 Institut Mines-Télécom SSD 2013
Recent Projects ■ BioSecure : Biometric Secure Authentication ● Coordination of the NoE, 2004-2007, 30 partners, 3 M€ ● Framework for test of biometric algorithms ■ Biotyful : BIOmetrics and crypTographY for Fair aUthentication Licensing (2007-2010) ● ANR telecom, ATMEL, FRANCE TELECOM, GET/INT, GREYC ● Cryptobiométrie VIDEO-SURVEILLANCE ■ Vidéo-ID : Identification via face and Iris in video-surveillance ● ANR CSOSG (2008-2011) ■ Kivaou : Face identification in video-surveillance ● ANR CSOSG (2008-2009) ■ Xvision: Special vision sensors for outdoor applications ● System@tic (2008-2010) ■ METHODEO: ● ANR CSOSG (2011-2013) ■ Juliette: FEDER 2010 Institut Mines-Télécom
Recent Projects ■ Nouveau projet ITEA2 PRIBIOSEC ● crypto-biométrie, coordinateur français CASSIDIAN et PME Secure-IC ■ Collaboration avec PME E-Closing ● Sur la signature électronique avec biométrie ■ Projet SurfOnHertz avec YACAST: ● indexation audio ■ Projet avec PW Consultants: Institut Mines-Télécom
Intermedia 2/.. Institut Mines-Télécom
■ « Télévigilance » or remote Healthcare for Patients at Home : ● for elderly persons (vigilance problems) or persons with cardio-vascular pathologies or chronic diseases needing remote healtcare and surveillance ● makes the Patient more secure for his/her health and keeps social link through connected communication services ● also releases Home Hospitalisation load ■ System connected to a remote centralised Healthcare services platform : ● mobile terminal fixed to the patient in a full ambulatory way ● connected to remote Healthcare servers ● alarm automatisation at the patient ’s level and management of Patient’s data at the server level ● merged in the framework of the Smart home with domotic sensors with potential link to assistance robots (Companion type). Institut Mines-Télécom
■ Multi-sensors mobile terminal recording actimetric and vital patient ’s data : ● Simplicity of use and robustness to patient deambulation and movements ● Sensors design towards acceptance (miniaturisation), autonomy (battery) and noise-robustness (low-complex/embedded efficient signal processing) ■ Alarm automatisation based of multimodal fusion ● heterogeneous fusion schemes applied on actimetric and physiological data to control specificity of the distress situations detection of the whole system [Medjahed-2010] ● forward alarms to remote healthcare center with sufficient information for remote emergency diagnosis ● Multimodal localisation with internal domotic sensors for patient’s localisation or ADL [Guettari-2010] Institut Mines-Télécom
pa ge Hamid MEDJAHED 31 31
Televigilance Lab at Telecom SudParis for technical validation Pa ge Institut Mines-Télécom 32
Televigilance terminal (projet QuoVADIS) Ambulatory terminal of televigilance designed at Telecom SudParis and integrated in partnership with ASICA. (Patent J.L. Baldinger) Fall detection, posture, mouvement, robust on-line pulse, with emergency push On-going validation in an hospital context. Institut Mines-Télécom
■ Projet ACI-Ville: MEDIVILLE (2001-03) ■ 3 Projets RNTS et ANR-TecSan: ● TelePat (2004-06): Televigilance médicale ● Tandem (2005-09): Thérapie cognitive et sécurité à domicile ● QuoVadis (2008-10): Sécurité multimodale (capteurs sur la personne, domotiques et robot) ■ 2 Projets de coopération internationale (échanges): ● SAFETI-IE4IL (2007-10) : France Afrique du Sud ● Brancusi (2009-11) : France et Roumanie ■ Projet IST-FP7 CompanionAble: Combination of the Smart home and the Robot-Compagnon (2008-11) Institut Mines-Télécom
Perspectives à moyen-/long-terme s’inscrivant dans les TIC Santé et la stratégie de Telecom SudParis ■ Télévigilance Médicale appelée à faire partie d’applications plus générales comme la domotique et l’assistance robotique déjà appelées à se combiner => RobotCompanion dialoguant avec son maitre (le patient) mais aussi veillant à son confort et sa sécurité grâce à une communication généralisée entre capteurs et modalités de tout type ■ Combinaison/ Fusion/ Adaptation des données/ paramètres domotique/ patient par des approches de type Fusion/ Fouille de données et Context Awareness adaptation => Intelligence Ambiante et Nomade ■ Axes de recherche importants pour TSP et IT sur l’application des TIC, des capteurs et de l’informatique réseau à la Santé => engendre des projets transversaux intra- et inter-écoles faisant appel à des compétences variées et très techniques Institut Mines-Télécom
Intermedia 3/.. Perceiving and rendering users in a 3D interaction P. Horain Institut Mines-Télécom
3D body motion capture by real-time computer vision ■ Motion capture by computer vision ■ Real-time remote virtual rendering MPEG 4 /BAP 3D/2D registration Institut Mines-Télécom
Gesture statistical modeling Conversational gesture generated with the Greta software with varying expressivity Institut Mines-Télécom
Statistical gesture models for 3D motion capture and rendering Rendering motion parameters that cannot be captured WITHOUT gesture model Hands ! WITH gesture model Institut Mines-Télécom
GPUCV: GPU acceleration for Computational Vision → http://picoforge.int-evry.fr/projects/gpucv Institut Mines-Télécom
Paris MobileMii Evry Platform for the Development of Nomadic Saclay Services Institut Mines-Télécom
Multi scale and nomadic Services Joint platfom partnership • CEA LIST • Institut Telecom / Telecom SudParis Supporting partners • Academics, end users, industry, clusters (pôle de compétitivité) MobileMii • A technology research and development Project • An infrastructure including an apartment located on the Saclay Campus Objective and concept : • Provide prevention, comfort and awareness services to people • Provide seamless services in the apartment, on the campus and beyond • Develop hardware and software modules for different applications in relation to the apartment, the building, the campus Applications • Security • Education • Energy • Transportation • Health 42 Institut Mines-Télécom 12
MobileMii: MobileMiiInfrastructure : Infrastructure MobileMii – Saclay: •A 250m2 modular architecture •A showroom •Several technical labs for development and assessments Saclay Campus •Situated inside Nano-INNOV building sensors distributed over the campus 43 Institut Mines-Télécom 12
Functionalities being studied to build services Sensors /Embedded signal People localization Vision processing Gesture recognition Emotion understanding Equipe Biometrics for people Gait Analysis INTERMEDIA identification Data Fusion Equipe Context awareness Navigation Geolocalization • Automatic learning •Ontology Équipe Simbad •Fusion of heterogeneous sensor information Sensorial Man Machine Interface: MMI Equipe Middleware Marge Equipe S3 Communication Environmental •Sensor network and communication between sensors equipments •Management of communication between people • Acquisition and Monitoring •Coverage of mobility communications (Heat, Air conditioning, intrusion, fire) 44 Institut Mines-Télécom 12
Scenarios, Applicative domains ■ Televigilance: surveillance of dependent people at home ● Sensor and data − Actimetry (accelerometer, gyroscope ...) − Monitoring vital signals (heart ...) ● Objectives − Preventing crisis situations (falls, respiratory, cardiac problems) − Detection of crisis situations (falls, respiratory, cardiac problems) − Detection of abnormal behaviors (eg Alzheimer's, stress, lack of food, deshydration) ■ Monitoring the activities of daily living ● Televigilance without embedded sensors (camera, sound ...) ● At home or at work ● Detection of risk situations ■ Security and Global Monitoring of public places ● Detection of abnormal behavior ● Detection of risk situations ● Identification of persons (Biometrics) ■ Intelligent Energy ● In a house − Managing the various sources of energy (production and storage) − Conserving energy (heating) based on the presence of people − Managing peak periods: subscription and especially under-exploitation of global network ● In a public / place to live / working place/ building − User-based energy saving: lights, computers, appliances ■ Home Automation − Home or workplace, monitoring of blinds, lights, temperature … ■ Media Room ● New GUI (via tactile sensors) 45 Institut Mines-Télécom 12
You can also read