Embedded Algorithms for Motion Detection and Processing - Marco Castellano System On Sensor Team Leader Analog, Mems & Sensors
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Embedded Algorithms for Motion Detection and Processing 28th February 2018 Marco Castellano System On Sensor Team Leader Analog, Mems & Sensors
Embedding Algorithms on Sensors 2
Existing Embedded Standard Solution
Solutions
Algo1..2..3..
Sensor
Sensor General
Purpose
Algo1..2..3..
Microcontroller
Pros - Cons Pros - Cons
Low Power × (Low) Power
Number of dice × Number of dice
1 communication bus × 2 communication bus
× Number of Algos Number of Algos
× Configurability Configurability
Conclusions 3
Innovative Embedded
Solution
Inertial Sensor Two digital configurable
LSM6DSOX embedded modules for fast and
effective implementation of
FSM MLP
motion detection processing have
ALG1-16 ALG17-24 been presented.
• A
Pros - Cons Common application cases
shows 20 to 100 times current
1 Ultra Low Power
consumption reduction respect to
Number of dice
standard solutions.
1 communication bus
Number of Algos
Configurability
Motion Detection Algorithms 4
Inertial algorithms overview
Glance Carry
Fitness
Activity Pos
Face Rec
Up/ Wake
Down Up
Vibration
Tap Monitor
Tap Step
Free
6D Count
Fall
Motion Detection Algorithms 5
Inertial algorithms overview
Glance Carry
Fitness
Activity Pos
Face Rec
Up/ Wake
Down Up
Vibration
Tap Monitor
Tap Step
Free
6D Count
Fall
Two Motions Comparison 6
Face-Down Transition Foot Impact
Medium Medium
Strong Strong
Slow Slow
Facing down the phone determines a sign change on Z Foot impacts determine peaks, which shape, magnitude
component of acceleration (1 g -> -1g) and frequency are heavily variable
Two Motions Comparison 7
Face Down Peak to Peak Occurences Foot Impact Peak to Peak Occurences
Narrow BroadMotion Detection Algorithms 8
Inertial algorithms overview
DEDUCTIVE INDUCTIVE
Glance Carry
Fitness
Activity Pos
Face Rec
Up/ Wake
Down Up
Vibration
Tap Monitor
Tap Step
Free
6D Count
FallInductive or Deductive? 9
Divide et impera
DEDUCTIVE INDUCTIVE
• Based on general hypothesis that • Based on generalization from specific
predict the observation with high observations
precision
• Campaigns of data collecting needed
• Robust and fully validated hypothesis
needed • The algo is dependant on the
population under consideration
(gender, age, provenience...).Motion Detection Algorithms 10
Inertial algorithms overview
Finite State Machine For motion Machine Learning Processing
Detection
Glance Carry
Fitness
Activity Pos
Face Rec
Up/ Wake
Down Up
Vibration
Tap Monitor
Tap Step
Free
6D Count
FallMotion Detection Finite State Machine 11
LSM6DSOX is provided with an interpreter that decodes and executes up to 16
independent Finite State Machines
Each FSM is intended to detect a single specific gesture.
• Wrist Tilt
• Free Fall
• Pick Up
• Wake-Up Easy and Effective Application
• Shake Development
• Glance
• STM Library of gestures available
• Tap
• High level of parametrization
• Motion /Stationary
• Etc…
• STM development GUI
• High level of customizationSingle Finite State Machine Strategy 12
Motion Detection in Three Steps
Sensors States Motion
Data Detection
Accelerometer Conditions Interrupts
Gyroscope Commands Source
External
Parameters Long Counter
SensorFinite State Machine Inputs 13
Select Appropriate Sensor Data
for the Application
Sensors A wide Set of Inputs to be chosen
Data
1. Accelerometer [ax ay az av ]
2. Gyroscope [gx gy gz gv ]
3. Calibrated magnetometer [mx my mz mv ]
4. Filtered signal #1 [fx fy fz fv ] Magnitude Available
Accelerometer
5. Filtered signal #2 [fx fy fz fv ]
6. Filtered signal #3 [fx fy fz fv ]
Gyroscope
7. Filtered signal #4 [fx fy fz fv ]
External 8. Integrated gyroscope [dx dy dz dv ]
SensorFinite State Machine Core (1) 14
Smart Memory Allocation
FIXED DATA
FSM Programs Variables and Instructions States SECTION
A simple program is made by:
• a data section, composed of a fixed part (same size VARIABLE DATA
for all the FSMs), and a variable part (size is specific for SECTION
each FSM)
Fixed data Section
• an instructions section, composed of commands and
conditions (size is specific for each FSM) Variable Data ISTRUCTION
Section SECTION
Instructions
SectionFinite State Machine Core (2) 15
Highly Configurable Core
START
Instructions Set States
Commands are immediately evaluated. RESET NEXT
When a command is executed, the program pointer
is set to next command/condition line. RESET NEXT
Conditions are executed at data input arrival. COMMAND
If RESET is true the program pointer is set to last Conditions PARAMETERS
configured Reset Point;
If NEXT is true the program pointer is set to next line.
Commands RESET NEXT
Parameters are application mask, threshold and
timers Parameters CONTMotion Detected: Status Outputs 16
High Reconfigurability on Interrupts Routing:
Interrupts and Status Motion
Interrupts available for each state machine can Detection
be routed on INT1/INT2 pin
Interrupts can be latched/pulsed.
Source registers available for each state Interrupts
machine keeps axis positive/negative triggering
the interrupt event
Source
Long counter countes an event in a FSM
Long CounterExample: Build a Simple Effective Face Up to
17
Face Down Detection (1)
Application case: rotate phone on the table up to down to silence a call
Start: Phone on the table Application Definition
1) Be still, zeta axis up
Threshold1 2) Start transition
3) Check turning zeta respecting
GNTH1
timer
Threshold2
4) Debounce
- Threshold1 5) Be still, zeta axis down
End: Phone reversed
TimerExample: Build a Simple Effective Face Up to
18
Face Down Detection (2)
Application case: rotate phone on the table up to down to silence a call
Start: Phone on the table
Threshold1
GNTH1
Threshold2
- Threshold1
End: Phone reversed
TimerFinite State Machine Conditions 19
States are defined by means of RESET / NEXT conditions
CONDITION DESCRIPTION CONDITION DESCRIPTION
0x0 NOP None execution - condition on current sample 0xB GRTH1 Any triggered axis > -THRS1
0x1(234) TI1(234) Timeout 1(234) expired 0xC LRTH1 Any triggered axis THRS1 0xD PZC Any triggered axis crossed zero pos. slope
0x7(8) LNTH1(2) Any triggered axis THRS1 0xF MLPCHK MLP check
0xA LLTH1 All triggered axisFinite State Machine Commands 20
Commands Overview
STOP
Commands are immediatly
STHR1(2) evaluated (until a State is found)
CONT
SELTHR1(3) JUMP
SRP CRP
Thresholds
SELMA
UMSKIT
SELMB Mask
SM(ABC) MSKIT
SELMC Timers
MSKITEQ
Execution
STIMER3(4)
CANGLE
Output
STCT0(1) REL
Various
SINMUXMachine Learning Processing 21
An interpreter decodes and executes up to 8 independent decision trees
Each Application is intended to detect user contexts.
• Activity recognition
• Fitness activities
• Motion intensity
• Vibration intensity Easy and Effective Application
• Carry position Development
• Context awareness
• STM Library for context awareness
• False positive rejection
• Fully reconfigurable
• Etc…
• STM development GUI
• Designed to be used with machine
learning toolMachine Learning Approach 22
Define Classes to be
recognized
Self learning approach Collecting multiple Logs
Configure the device and
run the application. Define Features that better
characterized the defined
classes;
• Define Decision Tree using
machine learning toolsMachine Learning Processing Strategy 23
Context Awareness defined in Three Steps
Sensors Computation Decision
Data Block Tree
Accelerometer Filters Connections
Gyroscope Features Meta-classifier
External
Connections Results
SensorMotion Processing Inputs 24
Select Appropriate Sensor Data and Filtering
for the Application
Sensors A wide Set of Inputs to be chosen
Data
1. Accelerometer [ax ay az av ] , [av2]
2. Gyroscope [gx gy gz gv ] , [gv2]
3. External sensor [mx my mz mv ], [mv2]
Accelerometer
Gyroscope
External
SensorMotion Detection Processing(1) 25
Filters: Smart Memory Allocation
Inputs to Computationals Block are Filtered Data Computation
Block
1. High Pass [fx fy fz fv ] , [fv2]
2. Band Pass [fx fy fz fv ] , [fv2] FIXED DATA
3. First Order IIR [fx fy fz fv ] , [fv2] SECTION
4. Second Order IIR [fx fy fz fv ] , [fv2] Filters
VARIABLE DATA
SECTIONMotion Detection Processing(2) 26
Features: Smart Memory Allocation
Filtered Data is Elaborated:
Computation
@TIME WINDOW
Block
@ Trigger (to be configured as feature)
• Mean
• Variance
• Energy FIXED DATA
• Peak SECTION
• Zero crossings Features
• Min
• Max VARIABLE DATA
• Duration Connections SECTION
• Etc.Motion Detection Decision Trees (1) 27
Outputs from Computation Blocks are Inputs to
Decision Trees
Fully Reconfigurable Decision Trees Topology
Decision
Tree Example Trees
Typical node
Start Input Condition
node
Topology
Connections
Connections
define
node
True Path False Path the algorithm
aaaa
behaviourMotion Detection Decision Trees (2) 28
Decision Tree Outputs Can Be Filtered
Up to 8 decision trees, Up to 8 Meta-classifier for 8
Decision Tree Family Results Decision
Trees
• A Meta-classifier is a outliers filter [M,N]
Example: [3,4] meta-classifier
Result
IN a a a b a b b b
Interrupts and
Ca 1 2 3 2 3 2 1 0 Status
Cb 0 0 0 1 0 1 2 3 Meta-classifier When a change
OUT x x a a a a a b of result is
Results detectedYou can also read
