Empirical 3D and 4D structural tree models from TLS data - Pasi Raumonen 3D Tree Models for Forest Dynamics 9th - 10th Jan 2020 Helsinki, Finland

Empirical 3D and 4D structural
 tree models from TLS data
         Pasi Raumonen

3D Tree Models for Forest Dynamics
        9th - 10th Jan 2020
         Helsinki, Finland

Outline

1. Data: Terrestrial laser-scanning

2. Empirical 3D models: Woody structure

3. Empirical 3D models: Leaves

4. Empirical 4D models

Terrestrial laser-scanning

                             3D point cloud

Terrestrial laser-scanning

• Accurate and comprehensive 3D data from tree’s surface
• Makes possible to measure (and model) trees
  • non-destructively
  • safely
  • fast
  • cheaply
• We can have accurate empirical models of trees
  • Branching structure
  • Volumetric and geometric data
  • Leaves

Photogrammetric point clouds

•   Use a camera (even a smart phone camera) to take lot of photos of the
    trees and then produce 3D point cloud using the Structure from Motion
    (SfM) method.

•   Marzulli et al. 2020: “Estimating tree stem diameters and volume from
    smartphone photogrammetric point clouds”. Forestry: An International
    Journal of Forest Research.

•   This conference: Phil Wilkes: “A comparison of terrestrial LiDAR and
    photogrammetry for rapid characterisation of fine scale branch structure”.

2. Empirical 3D models:
    Woody structure

Empirical tree models

•   No right or obvious way to model trees.

    •   No useful parametrisable surface presentations.

•   Model that contains all the essential information of
    the data.

•   Robust way to reconstruct the model.

•   Solution: Building block or geometric primitive
    approach.

    •   Tree modelled as a hierarchical collection of
        cylinders or other primitives fitted to local details.

                                                                 Photo and point cloud data provided by Eric Casella, UK Forest Research Agency

QSM - Quantitative Structure Model

•   Hierarchical collection of cylinders fitted
    to local details of the tree.

•   Compact model containing essential
    topological, geometrical and volumetric
    tree properties.

    •   Branching structure, branching order.

    •   Volumes, lengths, angles, diameters,
        etc.

Geometric primitives

•   Other simple shapes as building
    blocks.

    •   Elliptical and polygonal
        cylinders, cones.

•   Hybrid models with different
    building blocks possible.

•   Cylinder is the most robust choice.

•   Not yet implemented in TreeQSM,        •   Åkerblom et al. 2015: “Analysis
                                               of geometric primitives in
    except the possibility for modelling
                                               quantitative structure models of
    the bottom of the stem with
                                               tree stems”. Remote Sensing
    triangular mesh.

Geometric primitives

•   Simple model whose surface is discontinuous.

•   May not be visually pleasing or realistic-looking.

•   Still topologically and geometrically accurate
    and contains most of the structural information.

•   More complicated modelling often only
    decreases the stability and accuracy without any
    essential or useful new information.

•   Visualisation with nice-looking surfaces and
    textures possible from QSMs.

QSM papers and reconstruction methods
•   Raumonen et al. 2013: “Fast Automatic Precision Tree Models from Terrestrial Laser
    Scanner Data”. Remote Sensing.

•   Calders et al 2015: “Nondestructive estimates of above-ground biomass using
    terrestrial laser scanning”. Methods in Ecology and Evolution.

•   Raumonen et al. 2015: “Massive-scale tree modelling from TLS data”. ISPRS Annals.

•   Disney et al. 2018: “Weighing trees with lasers: advances, challenges and
    opportunities”. Interface Focus.

•   TreeQSM - MATLAB implementation, freely available in GitHub.

•   Hackenberg et al. 2015: “SimpleTree —An Efficient Open Source Tool to Build Tree
    Models from TLS Clouds”. Forests.

•   Trochta et al. 2017: “3D Forest: An application for descriptions of three-dimensional
    forest structures using terrestrial LiDAR”. Plos One.

•   Computree. GIP ECOFOR, ONF, Arts et Métiers Paristech, IGN, INRA and Université
    de Sherbrooke.

TreeQSM: How it works?

Input: point cloud, parameters              —> Branch-segmented point cloud          —>    Cylindrical QSM

         xyz-data                                    Topology, branching structure        Geometry, volumes

          Point cloud data provided by Eric Casella, UK Forest Research Agency

TreeQSM: Basic assumptions
•   Single tree in the point cloud. Can have some points from ground and understory.

•   Only (x,y,z) -data needed, not using intensity, colour, etc.

•   Whole tree is wood. Leaves/noise present in the data may be modelled as part of the wood.

•   “Data-driven”: Only sufficiently visible tree parts can be accurately reconstructed.

•   Cylinder is an acceptable building block.

•   Optional: Branches taper and are smaller in diameter than their parents.

•   Separate stem near the ground clearly visible in the data.

•   No special assumptions about tree species or size other than the above ones.

Structural but not fully architectural

•   TLS-derived QSMs have structure: stem and branches, diameters,
    lengths, volume, etc.

•   But normally the TLS-derived QSMs do not contain full tree architecture.
•   For example, the primitives (cylinders) don’t correspond to yearly growth
    (internodes), shoots, or other functional units, etc.
•   Still, a lot of useful structural information can be accessed from QSMs.
•   This conference: Hans Verbeeck: “Time for a Plant Structural Economics
    Spectrum”.

Challenges and limitations
• Need for automatic tree extraction.
  • Raumonen et al. 2015: “Massive-scale tree modelling from TLS data”. ISPRS
     Annals.
   • Andrew Burt 2017: “treeseg", “New 3D measurements of forest structure”.
     UCL.
   • This conference: Di Wang: “Towards an automated processing chain for 3D
     tree reconstructions from large scale TLS data”.

• Leaves should be separated from the point clouds.
  • Vicari et al. 2018, Methods in Ecology and Evolution.
  • Moorthy et al. 2019, IEEE Transactions On Geoscience And Remote Sensing.
  • Wang et al. 2019, Methods in Ecology and Evolution.
• Occlusion, particularly the visibility of the canopy.
  • More scans, drones, dynamic scanning positioning, mobile scanning.
• Parameters need to be somehow optimise.
  • How to measure the fit of the model against the data?
  • How to decide the best model?

Above-ground volume and biomass
•   Lidar+QSM gives volume + wood density = biomass
     •   Calders et al. (2015). Non-destructive estimates of above-ground biomass using
         terrestrial laser scanning. Methods in Ecology and Evolution.
     •   Raumonen et al. 2015: “Massive-scale tree modelling from TLS data”. ISPRS Annals.
     •   Hackenberg et al. (2015). SimpleTree - an efficient open source tool to build tree
         models from TLS clouds. Forests.
     •   Kunz et al. (2017): Comparison of wood volume estimates of young trees from
         terrestrial laser scan data. iForest.
     •   Gonzalez de Tanago Menaca et al. 2018: ”Estimation of above-ground biomass of
         large tropical trees with Terrestrial LiDAR”. Methods in Ecology and Evolution.

•   Generally under 10% error in biomass

•   Accuracy/error independent of tree size

•   For big trees allometry can give large (30-50%) errors

•   This conference: Eric Casella: “Sensitivity analysis of an automated
    processing chain and uncertainty in the prediction of tree above ground
    biomass from TLS data”.

•   This conference: Alvaro Lau: “Tropical tree biomass equations from
    terrestrial LiDAR”.

Below-ground biomass and structure

•   Stump-root systems of big trees were uprooted
    and cleaned, then scanned.

•   Hybrid-QSM: mesh (stump) and cylinders (roots).

•   Smith et al. (2014): “Root system characterization
    and volume estimation by terrestrial laser
    scanning”. Forests.

    •   Underestimated volume by 4.4%.

Feature spaces from QSMs
• QSMs allow to access myriad (potentially thousands) tree
  features, most manually unmeasurable.

• Increases the dimensionality of LiDAR data.
• Åkerblom et al. 2017: “Automatic tree species recognition
  with quantitative structure models”. Remote Sensing of
  Environment.

Species classification

•   Åkerblom et al. 2017: “Automatic tree
    species recognition with quantitative
    structure models”. Remote Sensing
    of Environment.

    •   3 species, 5 plots, over 1000 trees.

    •   95% recognition accuracy.

Database of QSMs

•   Save QSMs with proper metadata into a database with free access.

•   Make queries to mine the data.

•   Could be useful for validation and generation of many scientific
    hypothesis in ecology and forest research.

•   This conference: Special discussion session hosted by Markku Åkerblom.

•   This conference: Atticus Stovall: “Global Trends In Three-Dimensional Tree
    Structure”.

3. Empirical 3D models: Leaves

Wood-leaf segmentation

•   Woody structure modelling: preferred to scan in leaf-off conditions.

•   Usually the point cloud contains points both from leaves and wood.

•   There is a need for accurate/rough separation of wood and leaf points:
    •   Accurate QSM-reconstruction.
    •   Ecological applications (e.g. total leaf-area).

Wood-leaf segmentation

•   Classification methods:
•   Vicari et al. 2018: “Leaf and wood classification framework for terrestrial LiDAR point clouds”.
    Methods in Ecology and Evolution.
    •   Point-wise classification based on geometric features
    •   Path-analysis
•   Moorthy et al. 2019: “Improved Supervised Learning-Based Approach for Leaf and Wood
    Classification From LiDAR Point Clouds of Forests”. IEEE Transactions On Geoscience And
    Remote Sensing.
    •   Point-wise classification based on geometric features
•   Wang et al. 2019: “LeWoS: A Universal Leaf-wood Classification Method to Facilitate the 3D
    Modelling of Large Tropical Trees Using Terrestrial LiDAR”. Methods in Ecology and Evolution.
    •   Point-wise geometric features
    •   Recursive point cloud segmentation and regularisation

QSMs with leaves
• Hard to measure leaves, particularly individual leaves.
• It might be possible to measure leaf-distributions:
  • location (leaf-area-density)
  • leaf-size
  • leaf-orientation
  • This conference: Van-Tho Nguyen: “Validation of plant
    area density estimated from TLS data by using a voxel
    representation of 3D forests”.

• Distributions supported by QSMs.
• Sample those distributions to populate QSMs with leaves.

•   Åkerblom et al 2018: “Non-Intersecting Leaf Insertion Algorithm for Tree Structure Models”.
    Interface Focus.

•   Matlab code: https://github.com/InverseTampere/qsm-fanni-matlab

4. Empirical 4D models

4D empirical tree models

• Generate a time-series of empirical 3D models:
  • Scan the trees repeatedly for many years.
  • Reconstruct empirical 3D models (QSMs) from each repeated scan.
• This conference:
  • Eric Casella: “Sensing the growth of oak trees from an eight-year TLS survey period”.
  • Kim Calders: “Quantifying forest growth in a free-air CO2 enrichment experiment using
     terrestrial laser scanning”.
  • Miro Demol: “TLS for long-term forest monitoring: experience from the ICOS flux tower
     network”.
  • Eetu Puttonen: “Experiences in monitoring seasonal variation in vegetation with high
     density spatial and temporal terrestrial laser scanning time series”.

How to model tree growth?

•   Biology-based theoretical functional-structural plant models
    (FSPMs) such as Lignum.

•   TLS-derived empirical 3D models can help for the development and
    validation of FSPMs.

Validation/development of FSPMs with empirical
                            tree models

•   Beyer et al. 2017: “Validation of a
    functional-structural tree model using
    terrestrial Lidar data”. Ecological Modelling.
    •   Comparison of the simulated tree crown
        (3D spatial leaf density) to empirical
        crowns obtained from TLS data.
    •   TLS data provides an unprecedented
        degree of information on tree geometry
        compared to traditional forest inventory
        measurements.

Validation/development of FSPMs with
                 empirical tree models

•   Sievänen et al. 2018: “A study of crown development
    mechanisms using a shoot-based tree model and segmented
    terrestrial laser scanning data”. Annals of Botany.
    •   LIGNUM and pseudo-time-series of empirical QSMs.
    •   Different formulations of crown development (flushing of
        buds and length of growth of new internodes) in LIGNUM.
    •   Optimized the parameter values of each formulation to
        observe the best fit of LIGNUM simulations to the
        measured trees.
    •   Metric combined both tree-level characteristics and
        measures of crown shape.

•   Pseudo time series in Sievänen et al. 2018:

•   TLS-derived QSMs (left) and modified
    Lignum models (right).

How to model tree growth?

•   Biology-based theoretical functional-structural plant models (FSPMs).

•   More fully synthetic “4D-geometric” models that flexibly represent intuitive
    aspects of growth, resources, and structure without strict biological rules.

•   Combine the two and add stochastic properties:

•   Stochastic Structure Model - SSM.

Stochastic Structure Model - SSM

•   A model with fixed parameters (deterministic ones and those of probability
    distributions) creates statistically similar trees:

•   Morphological clones approximating the case:
    •   same genotype
    •   similar growth conditions
    •   so the differences are due to random effects.

•   Potapov et al 2016: “Bayes Forest: a data-intensive generator of morphological
    tree clones”, GigaScience.

•   Matlab code: https://github.com/inuritdino/BayesForest

Tuning SSMs Using Empirical Tree Models (QSMs)

Potapov et al 2016: “Bayes Forest: a data-intensive generator of morphological tree clones”. GigaScience.

Clone-generation

Potapov et al.: Data-based stochastic modeling of tree growth and structure formation, Silva Fennica, 2016

Next steps
•   Fully automatic processing chain: Plot level point cloud —> QSMs of individual trees with leaves.
    •   Automatic tree extraction.
    •   Automatic wood-leaf-segmentation.
    •   Automatic QSM reconstruction with leaves.

•   QSM and point cloud quality estimation/grading.

•   Upscaling: from TLS to satellite data – large comprehensively analysed test plots for large-scale
    calibration.

•   Use multi-channel or hyperspectral lidar information in QSM and leaf reconstruction and for 4D models.

•   Large scale repeated TLS measurements and time series of QSMs.
    •   Ecological research.
    •   4D tree model development and validation.

•   Measuring and modelling accurately distributions of leaf area density, size, and orientation.

•   Publicly accessible QSM database with tens of thousands of trees.