Deep Learning at the Edge - HP Workstations, CGG, HP Labs 3D Print March 2018 - Source: NVIDIA - GTC On-Demand
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Source: NVIDIA Deep Learning at the Edge HP Workstations, CGG, HP Labs 3D Print March 2018
Today’s Presenters
BRUCE BLAHO STEVE DOMINGUEZ HE LUAN DR. JUN ZENG
Fellow Team Lead Research Scientist, APhD2B Principal Investigator
Workstations Chief Technologist Seismic Interpretation Software HP LABS 3D Print HP LABS 3D Print
HP Inc. CGG HP Inc. HP Inc.
Deep Learning Edge Development Platforms HP Z8 Workstation Up to: • 56 CPU cores • 3 TB RAM • 48 TB Storage • 3-7 GPU’s • 9 PCI-e slots • 1125 - 1700 Watt Power Supply • 3 year warranty NVIDIA NVIDIA GPU Cloud Quadro GV100 Containers • 32 GB HBM2 • 5120 CUDA Cores • NVLink • 118.5 TFLOPS
After the presentations…
Machine Learning at the Edge for Seismic Interpretation Workflows Steve Dominguez, CGG GeoSoftware
Machine Learning at the Edge - Talk Outline • Deep Learning Applications in Seismic Interpretation – Recognize plausible applications – Noise Filtering – Object Classification / Image Segregation • On-going R&D Overview – Neural Network Designs – Training Data Sets • R&D Workflow – Powered by HP Z workstations – Machine Learning at the Edge: Scalable R&D workflow for assured success 6 Machine Learning at the Edge; Seismic Interpretation Applications – March 28, 2018
Deep Learning in Seismic 7
Deep Learning Concept Overview
• Computers
– great at repetitive tasks, raw calculations
– Slow and clunky at pattern recognition
• People
– Great at pattern recognition
– Slow and clunky at repetitive calculation
• How would we design software to excel at pattern
recognition?
8 Machine Learning at the Edge; Seismic Interpretation Applications – March 28, 2018
Deep Learning Applications in Seismic Interpretation
• “Hierarchical Learning Model”
– Algorithms model high-level data abstractions using complex structures of non-linear transformations
– Enables software to identify data patterns without being explicitly programmed
• Involve *simple* math circuits in great numbers to model complex problems
• “Train” the network of simple circuits with isolated individual circuit adjustments to optimize output
9 Machine Learning at the Edge; Seismic Interpretation Applications – March 28, 2018
Deep Learning Applications in Seismic Interpretation
• Salt Body Interpretation
Machine Learning at the Edge; Seismic Interpretation Applications – March 28, 2018
10Deep Learning Applications in Seismic Interpretation
• Salt Body Interpretation
Machine Learning at the Edge; Seismic Interpretation Applications – March 28, 2018
11Deep Learning Applications in Seismic Interpretation
• Acquisition Footprint / Noise
Machine Learning at the Edge; Seismic Interpretation Applications – March 28, 2018
12Deep Learning Applications in Seismic Interpretation
• Acquisition Footprint / Noise
13Deep Learning Applications in Seismic Interpretation
• Acquisition Footprint / Noise
14Deep Learning Applications in Seismic Interpretation
• Fault Imaging
Machine Learning at the Edge; Seismic Interpretation Applications – March 28, 2018
15Deep Learning Applications in Seismic Interpretation
• *Pattern Recognition* • Deep Learning approach
– Noise Mitigation – Interpreter Guided ‘teaching’
– Fault Imaging – Developers must work with interpreters to guide the
– Geobody Detection neural network with appropriate teaching data to
facilitate good ‘learned’ behavior
– Well Ties
– Training initially happens in a controlled environment
• InsightEarth state-of-the-art approach (development lab)
– Take mundane repetitive tasks of ‘picking’ and accelerate – Released as ‘commercially viable’ with reasonable
them through computer automation accuracy expectations
– “Interpreter-Guided” automation
• The human must set appropriate parameters based on their – Should include tuning parameters to refine behavior for
geoscience learning and experience to guide the algorithms various common scenarios
Machine Learning at the Edge; Seismic Interpretation Applications – March 28, 2018
16On-Going R&D Overview 17
Deep Learning Applications in Seismic Interpretation
• How much of the ‘middle man’ processes could we cut?
• How much faster could this approach run over the current workflow…?!?
Machine Learning at the Edge; Seismic Interpretation Applications – March 28, 2018
18“NeUral Network Recognition for Faults - NURF” (working
title) ☺
• Automatic Fault Extraction
• Current state-of-art:
• 15x in-memory working copies of data, 12 passes through data
• Machine Learning Approach:
• 1x in-memory active copy, 1 pass through data!
Machine Learning at the Edge; Seismic Interpretation Applications – March 28, 2018
19“Neural Network Noise Negation – N4” (working
title) ☺
• Footprint Removal / Noise Conditioning
• Current state-of-art:
• Algorithm orients to match 1st derivative directional vectors at each point of data
sampling
• Calculates a 2D operator “pane” of medians, takes the mean of the medians and
adjusts the center location value
• GPU accelerated, but the orientation and interpolation is still time consuming…
• Machine Learning Approach:
• 1x in-memory active copy, 1 pass through data
Machine Learning at the Edge; Seismic Interpretation Applications – March 28, 2018
20Neural Network Geobody Identification
(don’t even have a working title yet)
• Salt / Stratigraphic Interpretation
• Workflow is HIGHLY subjective – X passes through data, dependent upon image quality,
attributes, calculations required to “see” the geobodies
• Machine Learning Approach…
• Salt - 1 pass through the data!
• This is the easy one…
• Stratigraphy – in R&D stages determining applicability
• Not entirely convinced you can even “see” these objects without a “domain transformation”
Machine Learning at the Edge; Seismic Interpretation Applications – March 28, 2018
21“Teach Your Children Well”
“Strange bird lost at sea”
• Network has never been trained with any
fish before
• Knows over a 1000 species of birds,
dogs, cats, etc.
• But no fish….
• Good guess, all things considered!
• Imagine what a young child might
have called this thing if they had never
been taught the word ‘fish’ in any
context…
Machine Learning at the Edge; Seismic Interpretation Applications – March 28, 2018
22“Teach Your Children Well”
• Proper training data sets governs behavior!
• Deeper networks can learn more content, more accurately
• Top layer and bottom layer govern input constraints and output behavior
Machine Learning at the Edge; Seismic Interpretation Applications – March 28, 2018
23Next Steps
• Deepen network layers for increased accuracy
– Improved accuracy… more layers can learn more detections with improved results
• Integrate network layers for 3D processing vs. 2D flat imaging
– Interpretation is a 3D process! Why would we want to restrict the available information to 2D slices? No reason the algorithms
can’t function in 3D…
• Improve training data
– Consider edge-stack vs. fault-enhance
– Consider wider variety of structural deformation examples for training: can’t expect it to identify a thrust fault if it’s never seen one
before…
• Abstract output layer for multi-item detections
– Will eventually want to consider geo-body detection of any type
• Convert for ‘heat-map’ output instead of explicit bounding-box output
– Better ‘probability’ model outputs / more accurate object bounding in this manner
Machine Learning at the Edge; Seismic Interpretation Applications – March 28, 2018
24R&D Workflow – Powered by HP Z Workstations Machine Learning at the Edge! Scalable R&D workflow for assured success 25
Recipe for Successful R&D
• Agile Development Process
– … but “Mile-High Agile” is a totally different conference…
– Tangible / measurable milestones, clearly stated and understood objectives
– Why ‘train in the cloud’ or ‘train on the cluster’ before you’re reasonably sure of success??
• Expensive and time consuming!
• Start small, and scale!
– Laptop R&D PC R&D Workstation R&D …
Machine Learning at the Edge; Seismic Interpretation Applications – March 28, 2018
26R&D Path
• Retrained ‘LeNET’ deep learning model for fault detection
– Training conducted attempted with ‘fault / no-fault’ categories
– Training conducted with varying fault degree categories (0%, 25%, 50%, 75%, 100% certainties)
– Done on a laptop during a flight home from GTC
• Trained accuracy then tested on other Gulf area data sets
– Results around 83% accuracy when measured vs. expected behaviors from corresponding available Fault Enhance volumes
• Trained accuracy then tested on different structural regime data; North Sea
– Results around 70% accuracy when measured vs. expected behaviors from corresponding available Fault Enhance volumes
Machine Learning at the Edge; Seismic Interpretation Applications – March 28, 2018
27R&D Path
• Extending other convolutional networks with a deconv end layer into 3D models
– Training conducted arbitrary probability score floating point accuracy
– STILL ON A LAPTOP!
• Scale up into additional data sets
– Wider variety of geology involved
– Still using size-constrained input data sets…
– Scaled exact same dev / training environment up to a Z4, then Z6
Machine Learning at the Edge; Seismic Interpretation Applications – March 28, 2018
28R&D Path
• Grow into full-sized data sets, covering global expanse of Geological Examples
– Conduct initial training on Z8 workstation at full scale / full speed
• Multi GPU acceleration on GV100!
– 80% prediction of success can be measured from preliminary training – first few training epochs provide good indication of
“correct” or “incorrect” direction
• Convergence or divergence seen in learning models
• Could then scale up across multiple Z8 “nodes”, or choose to go “cloud” depending on time, resources, and cost
calculations…
– Study your learning curves vs. time / compute, and decide a) is more time training needed and b) are the cost-benefit
returns present for ‘cloud’ investments
Machine Learning at the Edge; Seismic Interpretation Applications – March 28, 2018
29Apply Deep Learning to HP’s 3D Printing Fusing Science Research He Luan1, Jun Zeng1, Sam Stodder2, Jordi Roca3, David Murphy 1, Thomas Paula4 1HP Labs, 2HP/3D Printing/Fusing Sciences, 3HP/3D Printing/Software, 4HP/CTO GTC 2018 Silicon Valley March 28th 2018
Multi Jet Fusion - MJF Basics
Polymer 3D Printing
Build Method Lamp Energy Melt Polymer Powder
Build Rate (mm/hr) High
Build Dimensions Medium
Typical Materials Nylon, other SC thermoplastics
Material Constraints No thermosets, amorphous
Resolution (Microns) 80 micron typical
Supports none
Post-process Decake, sandblast
Surface Finish Medium
Printer Cost
Material Cost Med, good reuse
MJF Part Power Med-High
Parts/Print Multiple Parts in bed
Finished Parts still in print bed
Printer Companies HP
Additional Voxel-level control
31Voxel thermal history
High amount of
Gamma Crystal
Structures
Slow cool
Temperature (deg C)
Melt Temperature must be held
long enough to completely Fast cool
break down Crystal Structure Higher Crystallinity
hrs
More Brittle
Lower Crystallinity
More Ductile
Powder Melt In Process Cool Post Process Cool
TimeLosses
Energy In
Problem Statement
• Multi Jet Fusion technology relies on precisely projecting thermal energy at
voxel level to ensure end-part quality.
• Problem addressed: create thermal prediction per layer at voxel level to enable
closed-loop voxel thermal control.
• Why difficult:
– energy at voxel level is affected by voxel energy absorption/loss, in-layer
thermal diffusion (spatial) and cross-layer thermal diffusion (tempo).
– Material properties are not only anisotropic but also phase dependent.
– Hi-resolution physical sensing is challenging.
• Our approach: apply deep learning to create a 3-stage deep neutral network
model (DL4FS) that predicts thermal energy at voxel level based on digital print
pipeline outputs.Deep neural network architecture
Current layer contone Current layer heat
detailing & fusing prediction
Architectural Spatial (CNN)
innovation
learn heat map generated by Current layer ultimate
Decouple principal spatial voxel energy map
voxel energy driver and prediction
(fusing/detailing agents).
principal tempo voxel
energy driver, and then Synthesis (CNN)
synthesize both
components as the final Previous layers heat Current layer heat learn the contribution of
prediction. image prediction above two components
and synthesize them.
Spatiotemporal
(Conv-LSTM)
learn the layer heat
transferred from previous
layers simulating heat
transfer.Spatial correlation
local features: spatial correlation, learn possible spatial correlations by extracting multiple feature maps
One convolution layer: learn feature maps.
multiple convolution layer: learn the feature of features, explore deeper and more complex correlations!
shape Part-part Thermal coupling Boundary thermal diffusionHow RNN capture sequential influence? Contone change • CNN is layer independent • RNN could capture the information transferred form previous layer. And this information is spatial. •This is how heat transfers! CNN infers from current layer only, therefore lose important sequential information
Model network structure in Tensorflow
Thermal prediction
layer by layer:
inputs, predictions
vs. ground truth
Data collected with a HP Jet Fusion 3D 4200 printer running at our R&D facilityPrediction error “heat-map” layer by layer
Computing cost: Training & Predictions
Sec. Num.
Training Accuracy Prediction time
Dataset /epoc epoche Hardware
time (hrs) (MSE/MSSIM) (sec.)
h s
Printer #1
Patch level 38.5 9.2K 125.7 2.97/0.87 0.13/layer 2 x Nvidia/M6000 (12GBx2)
(HP/San Diego)
Build-bed level 37 10K 110.4 2.00/0.94 0.05/layer 1 x Tesla/K80 (12GB)
Printer #2
Patch level 22 22.7K 170.3 2.43/0.88 0.12/layer 2 x Nvidia/M6000 (12GBx2)
(HP/Vancouver)
Build-bed level 13.5 11.5K 43.1 2.22/0.91 0.04/layer 1 x Tesla/K80 (12GB)
Per-layer production time is in order of 10 seconds. Current per-layer
prediction time cost shown here gives us reasonable hope that we may be
able to integrate this as a run-time prediction-correction step.Thank you!
MB1@Jabil
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
42And the winner is…
You can also read
