Big Data for Bioacoustics & Ethoacoustics of Marine Mammals - Machine Learning & Listening
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Big-Data RU-FR 2018 – UNESCO / Paris
Big Data for Bioacoustics & Ethoacoustics
of Marine Mammals
- Machine Learning & Listening -
Hervé Glotin,
with Dyni team CNRS LIS, SMIoT, Univ Toulon, EADM MADICS
glotin@univ-tln.fr
http://sabiod.org http://glotin.univ-tln.fr http://smiot.univ-tln.fr
A Big data issue
→ Recording 300 kHz SR
stereo
~ 100 Go / day
3 To / month
High velocity low power
sound card
SMIoT univ Toulon CNRS
⬜ Qualilife-Sound : high performance audio extension board
■ Up to 5 synchronous channels with adjustable sampling rate
■ Up to 2 Msps in 2 channels configuration
■ Up to 800 Ksps on 5 channels configuration
■ High performance anti-aliasing filter.
■ Direct HDD USB recording
Experimented Installations
Peru 2016 / Toulon 2018
CHILI CHILI 2017 2018,
Patris et al, LIS STICAMSUD
Corsica 2018, LIS DYNI
Long term Frequency and voicing pattern analysis
In submission
Global statistics at long range Malige et al 2018
Automatic Humpback song classification
Hierarchical Dirichlet
process hidden Markov
model for unsupervised
bioacoustic analysis
M Bartcus, F Chamroukhi, H
Glotin, in Neural Networks
(IJCNN), 2015
It allows
computation of
cultural
distance !
[ Razik Glotin
et al. IEEE
ICDM 2015]
(Haut) Chant de baleine à bosse, sur 12 secondes (enregistré par le groupe à la Réunion). (Bas) Partition construite automatiquement et de manière non supervisée par
l'équipe de Pr. H. Glotin [2]. Ces chants de baleine sont disponibles avec leur « partitions » en ligne sur: http://sabiod.univ-tln.fr/workspace/IHMM_Whale_demo/
suivant la méthode développée dans Bartcus et al. Glotin [2]"
Example of recording in AMAZON (Glotin 2014-2018, DCLDE 2018 Paris Sorbonne)
[0 micro second 50 ]
[0 micro second 50]
Distributed web
collaborative annotation
and deep learning
Active learning, mixture of experts
DYNITAG toolclassification
f(x) Species A: yes/no
~10 sec
J. Schluter, DyniPredictions from Audio
Fully-Convolutional Net
≤ 30 sec for training,
Variant A: Vanilla ConvNet
full recording for testing conv 64@3x3, bn, lrelu
Magnitude Compression
conv 64@3x3, bn, lrelu
max-pool 3x3 2D
Standardization features
conv 128@3x3, bn, lrelu
Fully-Convolutional Net
conv 128@3x3, bn, lrelu
Global Pooling conv 128@3x19, bn, lrelu merge
max-pool 3x5 bands
Softmax conv 1024@9x1, bn, lrelu
conv 1024@1x1, bn, lrelu classify
conv 1500@1x1Ensembling of Predictions
Date: 2014-01-22
Time: 16:00
≤ 30 sec for training,
full recording for testing
Longitude: -78.3951
Latitude: -3.0936
Magnitude Compression
Elevation: 650
Vector Encoding
Standardization
Standardization
Fully-Convolutional Net
Global Pooling Multi-Layer Perceptron
SoftmaxStereophonic, 4 or more hydrophones array : tracking, direction and behavior
Revealing “Megafauna” Vessel-Avoidance Strategies to
better Manage Collision Risk
Material and method
Bombyx with stereo antenna pointed to South to observe the megafauna (credit Dyni).Revealing “Megafauna” Vessel-Avoidance Strategies to
better Manage Collision Risk
Example of monitoring of Pm versus time from stereo Bombyx. Time Delay Of Arrival showing acoustic detections of Pm going from East to West in
5 mn nearby Bombyx the 21/09/2016.Revealing “Megafauna” Vessel-Avoidance Strategies to
better Manage Collision Risk
Example of monitoring of Pm versus time from stereo Bomby Total Pm countings and directions in the 0-15 km range of Bombyx,
Red: from East to West, Blue inverse, Green: unknown, on 76 days of summer 2016 (Glotin et al., Vamos Pelagos 2016)Ethoacoustics with 4 hydro ASV Sphyrna Odyssey, Glotin et al. 2018 meter
or our tracking in 3D in Bahamas (Glotin et al 2011, Canadian acoustics) from passive acoustic recordings demo @ sabiod.org
Ethoacoustics with 6 hydro demo at sabiod.univ-tln.fr/orcalab/
Predictions from STEREO Audio
≤ 30 sec for training,
full recording for testing
Magnitude Compression
Standardization
Fully-Convolutional Net
Global Pooling
SoftmaxPERSPECTIVE : Monitoring Seals of Baykal : proof of concept CNRS-HSE august 2018 withDmitr (master)
Perspectives : Binaural monitoring of fauna of the Baikal
CONCLUSION
We designed for 15 years advanced machine learning and
signal processing for innovative big data ethoacoustics.
We have OPEN solutions to compute DETECTION, tracking and
CLASSIFICATION of cetaceans.
Promote homogeneity, distribution, quality of scientific
observations.References @ http://sabiod.org
- Massive Biosonar recordings and inversion model for new sonar generation - DGA and Amiens region Phd Thesis, Maxence Ferrari, Codir M.
Asch and H. Glotin, 2017-2020
- High Performance Computing for Blue whale monohydrophone localisation, Phd Thesis J. Patris, Codir M. Asch and H. Glotin, 2015-2018
- Semi-supervised Deep learning for bioacoustic monitoring, Phd Thesis, Vincent Roger, Codir F. Chamroukhi and H. Glotin, 2016-2019, with
NORTEK SA
- F. Chamroukhi (dir Glotin), 'Statistical learning of latent data models for complex data analysis' with some applications to Bioacoustics, HDR
thesis, defended in dec 2015 in front to G. McLachlan, C. Ambroise, Y. Benani, Christophe Biernacki...
- M. Bartcus (dir Glotin, codir Chamroukhi) Non Parametric Bayesian Model - with some applications to Bioacoustics, defended in october
2015
- Yann Doh (dir Glotin, codir Adam, co-adv. Razik, Nolibe Cesigma)
A new intra-spectral monohydrophone range estimator and bioacoustic sparse coding for scaled submarine biodiversity.
Defended dec. 2014, jury: Pinquier, Zarader, Gerard (DGA), Pavan, Cristini, Razik, Nolibe, Adam, Glotin.
- Régis Abeille (dir Glotin, coll. Pavan, co-adv. Giraudet),
Automatic Inter-Pulse Interval diarization - Application to scaled whale bio-population in Pelagos sanctuary [.pdf] Defended dec. 2013,
jury: Sueur, Pavan, Adam, Giraudet, Glotin.
PROCEEDINGS in int. conf.
- Franck Malige, Julie Patris, Susannah Buchan, Marie Trone, and Hervé Glotin. Advanced interdisciplinary bioacoustical analyses for cetacean
observatories in chile and peru. In 1st Listening for Aquatic Mammals in Latin America Workshop (LAMLA 1), Natal, Brazil, 2016.
- Julie Patris, Hervé Glotin, Dimitri Komatitsch, Elwin van ‘t Wout, Franck Malige, and Mark Asch. High-performance computing for whale
sound propagation in south american oceans based on accurate numerical techniques. In 1st Listening for Aquatic Mammals in Latin
America Workshop (LAMLA 1), Natal, Brazil, 2016.
- Patris et al. "Congreso internacional de turismo comunitario sustentable : Conservar y Valorar”, 11-12 December 2017 Chañaral de
Aceituno. " Sonidos del Pacíficos : presentación y primeros resultados de un experimento acústico en la caleta Chañaral de Aceituno -
verano 2017"
- Glotin, Pavan, Dugan, Zhao, 'Environmental Acoustic Data Mining', IEEE ICDM 2015, http://sabiod.org/eadm, Atlantic city
- Glotin, Alecu, Big Data Sciences for Bioacoustic Environmental Survey 21 and 22 April 2015, Toulon - http://glotin.univ-tln.fr/ERMITES15
- Chamroukhi, Glotin, Dugan, Clark, Artières, LeCun, et al., Proc. of the second workshop on Machine Learning for bioacoustics -BOOKS / BOOK CHAPTERS
- Patris et al. SPECFEM to monitor bioacoustic sources, in Berkowitz Héloïse & Dumez Hervé [eds] (2017) Racket in the oceans: why underwater noise
matters, how to measure and how to manage it. Paris: Observatory for Responsible Innovation / Palaiseau (France): i3-CRG (CNRS – École polytechniqu
e).
The Racket in the Oceans initiative is open to industry, policy, science and societal stakeholders, and to anybody interested in the problem of
underwater noise.
- Joly, Goeau, Glotin, Spampinato, Bonnet, Vellinga,..., Müller, Lifeclef 2014: multimedia life species identification challenges. In Information Access
Evaluation. Multilinguality, Multimodality, and Interaction (pp. 229-249). Springer International Publishing, 2014.
- Soundscape Semiotics - Localization and Categorization, collected by Glotin, ISBN 978-953-51-1226-6, 208 p., Publisher: InTech, Open Book, 2014.
- Detection Classification Localization of Marine Mammals Using Passive Acoustics: 2003-2013, 10 Years of International Research, collected by Adam
, Samaran, Dirac NGO Ed., ISBN2746661187, 298 p., 2013.
- SAMARAN, GANDILHON, DOH, PACE, CAZAU, LAPLANCHE, LOPATKA, GLOTIN, WHITE, ZARZYCKI, MOTSCH and ADAM,
Inside the sounds emitted by some cetacean species, In DCL MM using PA, Dirac Ed, 2013
- Dufour, Artières, Glotin, Giraudet, Clusterized Mel Filter Cepstral Coefficients and Support Vector Machines for Bird Song Identification, in
Soundscape Semiotics, Localization and Categorization, InTech Open Book, 2013.
JOURNAL ARTICLE
- Unsupervised Bioacoustic Segmentation by Hierarchical Dirichlet Process Hidden Markov Model. Roger V., Chamroukhi, Glotin H. MTAP special issue,
to appear in 2018.
- A Real-Time Streaming and Detection System for Bio-acoustic Ecological Studies after the Fukushima Accident Hill Hiroki Kobayashi, Hiromi Kudo,
Hervé Glotin, Vincent Roger, Marion Poupard, Daisuké Shimotoku, Akio Fujiwara, Kazuhiko Nakamura, Kaoru Saito and Kaoru Sezaki. MTAP special
issue, to appear in 2018.
- Gaël Richard, Tuomas Virtanen, Nobutaka Ono and Juan Pablo Bello, Hervé Glotin edit a special issue of the prestigious IEEE/ACM Transactions on
Audio, Speech and Language Processing. The topic of the issue is scaled sound scene and event analysis for indoor and outdoor environments,
including applications in bio-acoustics, 2017 march
- Trone, M., Glotin, H., Balestriero, R., Bonnett, D.E. (2015). Enhanced feature extraction using the Morlet transform on 1 MHz recordings reveals the
complex nature of Amazon River dolphin (Inia geoffrensis) clicks. Journal of the Acoustical Society of America, 138, 1904.
http://dx.doi.org/10.1121/1.4933985
- Towsey M, Parsons S, Sueur J, Editorial: Ecology and acoustics at a large scale. Ecological Informatics, 21 : 13., 2014.
- Potamitis, Automatic Classification of a Taxon-Rich Community Recorded in the Wild, in PLOS One, 10.1371, 2014
- Pavan G., Favaretto A., Bovelacci B., Scaravelli D., Macchio S., Glotin H., 'Bioacoustics and Ecoacoustics Applied to Environmental Monitoring andmore to see at http://sabiod.org or https://scholar.google.com/citations?user=DqieizcAAAAJ&hl=en Project supported by BRILAM STIC AmSud 17-STIC-01, CNRS SABIOD, INPS Toulon, SMIoT, EADM MADICS CNRS
Contextual analyses
Multimodal data
Accuracy in hydrophones quality, orientation, effort,
Meteorological condition, acoustic masking…
Content is governed by schema.
Complete diagrams at:
http://tethys.sdsu.edu/schema/d
iagrams/
3536 Thetis coll. M. Roch
Examples in int. symp Acoustical Society of America, 2018
Etho-acoustics : High dimensional clustering on Dolphin
Whistles & Evidenced of Anthropic ImpactsAnalyzing
Dolphin Whistles in presence of
[ Poupard, Glotin, Mongolfier 2017 ]
Presentation of the experiment
boats in Martinique
a. Context
Localisation : West coast of Martinique
Species : Pantropical spotted dolphin, Stenella attenuata
Development of “Whales-watching” and tourism
Partners : Aquasearch
b. Objective
Analyse impact of whales watching on communication by Pantropical spotted dolphin : comparing
whistles produced without boat or in the presence of several boats.Materials and methods
On December 1, in 2003 and April 28, 2016 in Martinique with AQUASEARCH
- Hydrophone (H2a-XLR, Aquarian Audio Products)
- Records were realized in continue from animals were coming, until they lived zone.
The environmental data :
- Start and end of the observation, the date, Geographic coordinates
- Number of animals, Behaviors, adults and juveniles
- Number of boats in the area
*Anthropogenic pressureWhistles tracking
Detection processing
Automatic detection
Signal on spectrogram
Binarization Fig 5: Spectrogram of 13 seconds containing signals from Sa, and representation of
whistles with our detector, for each windows
Continuous trajectories ?
Select optimal parameters for the algorithm (windows size…) (DECAV PNPC Glotin et al 2012).Extraction of
Automatic detection
features for
each whistles
Matrix for each recording containing:
16 features (for each whistles) : maximal, minimal
frequencies, duration, velocities of whistles... Dimensionality reduction
t-SNE: Ethoacoustic clusters?
Max freq Min freq duration … … … …
whi 1
Whi 2Evidences of the effect of Anthropic Pressure
on bioacoustic emissions
Whistles depend on activity
Acoustic emissions in
anthropogenic pressure (AP) are
different compared to other
behaviours
See also submitted to PONE-D-18-30047 "Behavioural
responses of humpback whales to food-related chemical
stimuli.". Bertrand Bouchard, Jean-Yves Barnagaud; Hervé
Glotin; Marion Poupard; et al.
Visualizing dolphin Sa whistles in 2-dimensions with t-SNE as a function of
velocity and behaviors (with 4 important features), according to BNP clustering
=> Prevention of anthropogenic pressure by acoustic passive methodScoring : JASA Halkias Glotin
You can also read
