High-Resolution Segmentation of Tooth Root Fuzzy Edge Based on Polynomial Curve Fitting with Landmark Detection
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
High-Resolution Segmentation of Tooth Root
Fuzzy Edge Based on Polynomial Curve Fitting
with Landmark Detection
Yunxiang Li1 , Yifan Zhang2 , Yaqi Wang3∗ , Shuai Wang4∗ , Ruizi Peng1 , Kai
Tang1 , Qianni Zhang5 , Jun Wang6 , Qun Jin7 , and Lingling Sun1∗
arXiv:2103.04258v1 [cs.CV] 7 Mar 2021
1
Hangzhou Dianzi University, 12626 Hangzhou, Zhejiang, China, 310018
2
State Key Laboratory of Oral Diseases, National Clinical Research Center for Oral
Diseases, West China Hospital of Stomatology, Sichuan University, Chengdu, Sichuan
3
College of Media Engineering, Communication University of Zhejiang, 92254
Hangzhou, Zhejiang, China
4
Shandong University, Weihai, China
5
Queen Mary University of London, UK
6
Shanghai Jiao Tong University, 12474 Shanghai, China
7
Department of Human Informatics and Cognitive Sciences, Faculty of Human
Sciences, Waseda University, 13148 Shinjuku-ku, Tokyo, Japan
wangyaqi@cuz.edu.cn, shuaiwang@sdu.edu.cn, sunll@hdu.edu.cn
Abstract. As the most economical and routine auxiliary examination
in the diagnosis of root canal treatment, oral X-ray has been widely used
by stomatologists. It is still challenging to segment the tooth root with
a blurry boundary for the traditional image segmentation method. To
this end, we propose a model for high-resolution segmentation based on
polynomial curve fitting with landmark detection (HS-PCL). It is based
on detecting multiple landmarks evenly distributed on the edge of the
tooth root to fit a smooth polynomial curve as the segmentation of the
tooth root, thereby solving the problem of fuzzy edge. In our model,
a maximum number of the shortest distances algorithm (MNSDA) is
proposed to automatically reduce the negative influence of the wrong
landmarks which are detected incorrectly and deviate from the tooth
root on the fitting result. Our numerical experiments demonstrate that
the proposed approach not only reduces Hausdorff95 (HD95) by 33.9%
and Average Surface Distance (ASD) by 42.1% compared with the state-
of-the-art method, but it also achieves excellent results on the minute
quantity of datasets, which greatly improves the feasibility of automatic
root canal therapy evaluation by medical image computing.
Keywords: Fuzzy edge segmentation · Root canal treatment · Land-
mark detection · Polynomial curve fitting · Tiny datasets
1 Introduction
Root canal treatment as a common operation in dentistry has a non-trivial
filling error rate[1,2], which leads to a significant negative impact on the patient
2 Anonymous
outcome[3,4,5]. At present, the evaluation of the root canal treatment results in
the medical field relies on the personal empirical assessment of endodontists[6].
Firstly, due to the differences in the experience of endodontists, the accuracy of
manual examination cannot be guaranteed[7]. Secondly, the manual process will
inevitably suffer from inter-observer variability, not to mention the substantial
time and resources required. With the edge of the tooth root, the results of root
canal treatment can be easily evaluated. So the first and most important step
in evaluating root canal treatment automatically is to segment the edge of the
tooth root by deep learning [8,9,10].
In X-ray images, the boundaries of the teeth are blurry and some tissues
around the teeth have similar intensities to the teeth, which makes the tooth
segmentation more difficult and challenging. To solve that, Zhao et al.[11] pro-
pose a two-stage attention segmentation network for the tooth segmentation
task, following a similar approach to Attention U-Net[12]. Lee et al.[13] propose
a deep learning method using a fine-tuned mask R-CNN[14] algorithm. Koch et
al.[15] apply an FCN based on the U-Net architecture for the task of dental ra-
diographs segmentation. However, their method does not solve the segmentation
problem of fuzzy boundaries and the performance improvement is incremental.
Cheng et al.[16] propose U-Net[17]+DFM to learn a direction field, which char-
acterizes the directional relationship between pixels and implicitly restricts the
shape of the segmentation result. Their method achieves a great improvement
on the fuzzy edge segmentation task, but it is still limited by the accuracy of
U-Net. The evaluation of root canal treatment is based on the relative position
of the head of the pulp canal and the head of the tooth root. Traditional dental
segmentation methods are all aimed to segment the whole tooth, occupying un-
necessary computational resources and parameters in the segmentation of other
parts of the tooth, while the evaluation of root canal treatment only needs the
segmentation results of the head of the tooth root. The difference between the
filling results of root canal therapy is very small, so this task requires high pre-
cision of the segmentation of the head of the root. A large number of current
methods are based on improving U-Net, and their improvement compared with
U-Net is very limited. However, the root head is one of the fuzziest parts of the
tooth boundary, and there is no model that can accurately segment the tooth
root head to meet the precision requirement of root canal treatment evaluation
in the existing segmentation methods.
Contribution: In order to segment the tooth root with the fuzzy edge, we
propose a model of high-resolution segmentation based on polynomial curve fit-
ting with landmark detection (HS-PCL). The contributions of our paper are as
follows: (1) It is the first time to realize the automatic assessment of root canal
treatment by segmenting the tooth root with the landmarks. (2) Our method
segments the tooth root through landmark detection and fits a polynomial curve
as the segmentation result so that it just needs to detect a few landmarks to
achieve segmentation. (3) A maximum number of the shortest distances algo-
rithm (MNSDA) is proposed to automatically reduce the bad influence of the
Segmentation Based on Polynomial Curve Fitting with Landmark 3 wrong landmarks which deviate from the tooth root. (4) In the comparison against five kinds of U-Net networks, the performance of our model is up to 40% better than the state-of-the-art method. In addition, the results obtained based on 1/16 training datasets (only 5 images) are already competitive against most of the methods trained by the complete training datasets. Fig. 1. The framework of our proposed method. It consists of three parts: Automatic Rotation, HRnetV2, Segmentation. 2 Method The architecture of our method is illustrated in Fig.1. First, we preprocess each original X-ray image into a standard image through the automatic rotation mod- ule and detect the landmarks of tooth root edge by HRNetV2[18]. The shape of the tooth root is very similar to the quadratic function curve based on only a few detected landmarks. A polynomial curve can be fit as the segmentation result. In order to reduce the influence of some wrongly detected landmarks on the final polynomial curve fitting result, we propose MNSDA. The proposed pipeline greatly improves the segmentation accuracy and reduces the amount of data required for training. Details of each module will be provided below. 2.1 Automatic Rotation Considering many tooth roots of datasets are inclined to a certain angle, the extra difficulty is introduced to fit a polynomial curve through landmarks as the
4 Anonymous
segmentation result. To solve this problem, before detecting landmarks of tooth
root edge, we segmented the pulp canal by U-Net and detected two special land-
marks, the head and the tail of the pulp canal, and a line is acquired by segment
it. Then we calculate the angle between this line and the vertical direction and
eventually rotate the image by the angle to calibrate the image to a standard
image.
2.2 HRNetV2
HRNet[19] maintains the high-resolution representation by high-resolution con-
volution while enhancing it by parallel low-resolution convolution. The high-
resolution representation is maintained by connecting parallel different resolution
representations and repeating multiscale fusion, and the resulting high-resolution
output representation is informative and accurate in spatial representation. The
modularized block of HRNet is divided into two components: multi-resolution
parallel convolutions (Fig.2. (a)), and multi-resolution fusion (Fig.2. (b)). HR-
NetV2 (Fig.2. (c)) takes both high and low-resolution feature representations as
the output to obtain more spatial representation information.
Fig. 2. (a) Multi-resolution parallel convolution (b) Multi-resolution fusion (c) HR-
NetV2: Concatenate the representations that are from all the resolutions. fusion.
2.3 Segmentation
Polynomial Curve Fitting: The basic process of our fitting function is to take
T
the landmark (x, y) as a pair of observations, and [x1 , x2 , · · ·, xn ] ∈ Rn , y = R
satisfies the following theoretical function.
y = f (x, w) (1)
T
w = [w1 , w2 , · · ·, wn ] are pending parameters. In order to find the optimal
estimated value of the parameter w of the function f (x, w) when it takes the
minimum, the following objective function is solved for the given m sets of
observation landmark (x, y).
m
X 2
L(y, f (x, w)) = [yi − f (xi , wi )] (2)
i=1Segmentation Based on Polynomial Curve Fitting with Landmark 5
Maximum Number of the Shortest Distances Algorithm: After the land-
marks are detected, they are examined so that any deviated landmark is de-
tected. The average distances between all feature landmarks and the next feature
landmark are calculated. When the sum of the distances between a landmark
and its adjacent two landmarks is 5 times longer than the average distance of
all the landmarks, that is, longer than the dlim . This landmark is considered as
a deviated landmark that is far away from the tooth root edge.
18 q
P 2 2
(x[i] − x[i + 1]) + (y[i] − y[i + 1])
1
dlim = 5 × (3)
18
So we propose MNSDA to reduce the impact of landmark detection errors in
polynomial curve fitting. We traverse all possible combinations of 34 → 1 of all
the landmarks to fit a polynomial curve. Then we calculate the distance between
all the landmarks and the fitting curve result and compute the average of the
minimum distance to get the davg . After that we obtain the new sensitive dis-
tance value through the following short-range sensitivity function:
dsen = logdavg (distance + 1) (4)
After that, we again calculate the average of the dsen of all the landmarks for this
fitting polynomial curve. Finally, we find the shortest the short-range sensitive
distance by fitting polynomial curves in all the random combinations and make
it the final polynomial fitting result. Fig.3. shows the comparison before and
after adding MNSDA.
Fig. 3. (a), (c) are the results without MNSDA. (b), (d) are the results with MNSDA.
The polynomial curve fitting results with MNSDA will eliminate the wrong landmarks
and the final fitting results segmented the tooth root better.6 Anonymous
3 Implementation and Experiments
3.1 Datasets
We cooperated with a number of hospitals to obtain consent for the use of oral
X-rays in individual root canal treatment patients. It took several months to
collect the datasets used in this project. There is a large difference between our
method which is based on landmark detection and the mothod based U-Net
which is a mask with the entire tooth root, so we annotated the datasets used
by the two methods. A stomatologist, experienced in oral radiography and root
canal therapy, classifies image of the tooth under treatment and marked both
the edge and the landmarks. In addition, the labeling results were reviewed by
multiple stomatologists to ensure the accuracy of the labeling. When marking
the landmarks of the datasets, we labeled a total of 19 landmarks from left to
right on the tooth root head of each image. There are three treatment results of
root canal treatment, which are respectively called correctfilling (Fig.4. (A1)),
underfilling (Fig.4. (A2)), overfilling (Fig.4. (A3)) in the medical field[20]. The
detailed number of our datasets is shown in Table 1. Our final datasets are 1020
images and all the data is classified correctly. Because it is difficult to label the
segmentation annotation, we only label a part of collected datasets. In order
to test the performance of our model on small training datasets, we randomly
selected images from the original training set to a 1/2 training dataset, a 1/4
training dataset, a 1/8 training dataset, a 1/16 training dataset.
Table 1. Root Canal Treatment Datasets
classification segmentation landmark detection
training testing training testing training testing
overfilling 24 15 24 15 24 15
underfilling 269 24 30 24 30 24
correctfilling 662 26 25 26 25 26
3.2 Implementation Details
The comparison methods include several commonly used networks: U-Net, R2U-
Net, Attention U-Net, Attention R2U-Net, U-Net+DFM, HRNetV2. All of the
models were implemented using PyTorch and trained on one RTX 2080Ti GPU.
During model training, we conducted the same histogram equalization for all
the training datasets. U-Net and its improved versions are set with exactly the
same and appropriate initialization parameters, with the initial learning rate set
to 0.0002 and training for 200 epochs. The initial learning rate of HRNetV2 is
also set to 0.0002, and the training rounds are 70 epochs.Segmentation Based on Polynomial Curve Fitting with Landmark 7
Table 2. Average Surface Distance (ASD) and 95% Hausdorff Distance (HD95) for
tooth root head segmentation
Datasets Methods HD95(mm) ASD(mm)
U-Net 1.044 0.670
Attention U-Net 1.092 0.731
Complete R2U-Net 1.162 0.804
Training Attention R2U-Net 0.928 0.595
Datasets U-Net+DFM 0.7334 0.477
(79 images) HRNetV2 0.835 0.533
HS-PCL
0.490 0.278
(without MNSDA)
HS-PCL 0.485 0.276
1/2 Datasets
HS-PCL 0.585 0.298
(39 images)
1/4 Datasets
HS-PCL 0.693 0.363
(20 images)
1/8 Datasets
HS-PCL 0.727 0.451
(10 images)
1/16 Datasets
HS-PCL 0.935 0.654
(5 images)
3.3 Experimental Results
All methods were evaluated on two metrics, Average Surface Distance (ASD),
and 95% Hausdorff Distance (HD95). Root canal treatment evaluation is deter-
mined by comparing the position of the end of the pulp canal filled with drug and
the head of the tooth root. So the effective position of the segmentation result is
the edge of the tooth root head. Therefore, we only marked the landmarks of the
tooth root head as the datasets trained by our model. We rotated the predicted
results of U-Net and its improved versions as well as the ground truth to the
same rotation angle as our model. Additionally, we cropped the same height on
the left and right edge to ensure the comparison of the predicted results of each
method at the same tooth root position.
Complete Traning Datasets: We drawed the results predicted by the model
with a blue solid line and Ground Truth with an orange dashed line on the vi-
sual results. Fig.4. (A1-A3) shows the segmentation results with clear tooth root
edge. Fig.4. (B1-B3) presents the segmentation results with fuzzy tooth root
edge. Although we can observe that all the methods can reach a good level in
the segmentation of images with the clear edge, U-Net and its improved versions
performed unsatisfactorily for the fuzzy edge segmentation. U-Net+DFM which
is proposed for the fuzzy edge segmentation achieves a much better performance
than the other U-Net algorithms on the tooth root segmentation with the fuzzy
edge. Nevertheless, it is still outperformed by our method.
Our result is shown in Table 2. All our experimental results were calculated
on the same testing datasets with 65 images. We can observe that the proposed8 Anonymous polynomial curve fitting and MNSDA for segmentation both improve our model prediction. Our method significantly leads U-Net, R2U-Net, Attention U-Net, Attention R2U-Net, U-Net+DFM in both HD95 and ASD metrics. Compared with the latest U-Net+DFM, our method HS-PCL reduces ASD by 42.1% and HD95 by 33.9%. Tiny Training Datasets: Our method is designed to be able to work based on tiny training datasets. Due to the guidance of the shape of the polynomial curve and the superior performance of the high-resolution network for landmark de- tection, our final result is still superior compared to the state-of-the-art method even when we reduce the training datasets to 1/2, 1/4, 1/8. When reducing the training datasets to 1/16, our method can still reach a competitive level of the original size used by other models. Fig. 4. The segmentation results of the tooth root. The orange dashed line is the exact position of the root edge, and the blue solid line is the model prediction result. 4 Conclusion We present a model of high-resolution segmentation of tooth root fuzzy edge based on polynomial curve fitting with landmark detection. In addition, MNSDA
Segmentation Based on Polynomial Curve Fitting with Landmark 9
is proposed to reduce the influence of the landmarks of prediction errors on the
polynomial curve fitting results. The proposed model reduces HD95 and ASD by
about 40% compared with the SOTA model and still performs well with the tiny
training datasets (5 images). After the evaluation of professional stomatologists,
the accuracy of our model has met the clinical requirements of automatic root
canal treatment evaluation for the first time.
References
1. J Meirinhos, JNR Martins, B Pereira, A Baruwa, J Gouveia, SA Quaresma, A Mon-
roe, and A Ginjeira. Prevalence of apical periodontitis and its association with pre-
vious root canal treatment, root canal filling length and type of coronal restoration–
a cross-sectional study. International endodontic journal, 53(4):573–584, 2020.
2. Y Boucher, L Matossian, F Rilliard, and P Machtou. Radiographic evaluation of the
prevalence and technical quality of root canal treatment in a french subpopulation.
International endodontic journal, 35(3):229–238, 2002.
3. Louis M Lin, Paul A Rosenberg, and Jarshen Lin. Do procedural errors cause
endodontic treatment failure? The Journal of the American Dental Association,
136(2):187–193, 2005.
4. Carlos Estrela, Roberto Holland, Cyntia Rodrigues de Araújo Estrela, Ana He-
lena Gonçalves Alencar, Manoel Damião Sousa-Neto, and Jesus Djalma Pécora.
Characterization of successful root canal treatment. Brazilian dental journal,
25(1):3–11, 2014.
5. WP Saunders, EM Saunders, J Sadiq, and E Cruickshank. Technical standard of
root canal treatment in an adult scottish sub-population. British dental journal,
182(10):382–386, 1997.
6. JW Field, JL Gutmann, ES Solomon, and H Rakusin. A clinical radiographic
retrospective assessment of the success rate of single-visit root canal treatment.
International Endodontic Journal, 37(1):70–82, 2004.
7. GI Eleftheriadis and TP Lambrianidis. Technical quality of root canal treatment
and detection of iatrogenic errors in an undergraduate dental clinic. International
endodontic journal, 38(10):725–734, 2005.
8. Mohammad Hesam Hesamian, Wenjing Jia, Xiangjian He, and Paul Kennedy. Deep
learning techniques for medical image segmentation: achievements and challenges.
Journal of digital imaging, 32(4):582–596, 2019.
9. Xiaolong Liu, Zhidong Deng, and Yuhan Yang. Recent progress in semantic image
segmentation. Artificial Intelligence Review, 52(2):1089–1106, 2019.
10. Swarnendu Ghosh, Nibaran Das, Ishita Das, and Ujjwal Maulik. Understand-
ing deep learning techniques for image segmentation. ACM Computing Surveys
(CSUR), 52(4):1–35, 2019.
11. Yue Zhao, Pengcheng Li, Chenqiang Gao, Yang Liu, Qiaoyi Chen, Feng Yang, and
Deyu Meng. Tsasnet: Tooth segmentation on dental panoramic x-ray images by
two-stage attention segmentation network. Knowledge-Based Systems, 206:106338,
2020.
12. Ozan Oktay, Jo Schlemper, Loic Le Folgoc, Matthew Lee, Mattias Heinrich, Kazu-
nari Misawa, Kensaku Mori, Steven McDonagh, Nils Y Hammerla, Bernhard Kainz,
et al. Attention u-net: Learning where to look for the pancreas. 2018.
13. Jeong-Hee Lee, Sang-Sun Han, Young Hyun Kim, Chena Lee, and Inhyeok Kim.
Application of a fully deep convolutional neural network to the automation of10 Anonymous
tooth segmentation on panoramic radiographs. Oral surgery, oral medicine, oral
pathology and oral radiology, 129(6):635–642, 2020.
14. Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In
Proceedings of the IEEE international conference on computer vision, pages 2961–
2969, 2017.
15. Thorbjørn Louring Koch, Mathias Perslev, Christian Igel, and Sami Sebastian
Brandt. Accurate segmentation of dental panoramic radiographs with u-nets. In
2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019),
pages 15–19. IEEE, 2019.
16. Feng Cheng, Cheng Chen, Yukang Wang, Heshui Shi, Yukun Cao, Dandan Tu,
Changzheng Zhang, and Yongchao Xu. Learning directional feature maps for car-
diac mri segmentation. In International Conference on Medical Image Computing
and Computer-Assisted Intervention, pages 108–117. Springer, 2020.
17. Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional net-
works for biomedical image segmentation. In International Conference on Medi-
cal image computing and computer-assisted intervention, pages 234–241. Springer,
2015.
18. Jingdong Wang, Ke Sun, Tianheng Cheng, Borui Jiang, Chaorui Deng, Yang Zhao,
Dong Liu, Yadong Mu, Mingkui Tan, Xinggang Wang, et al. Deep high-resolution
representation learning for visual recognition. IEEE transactions on pattern anal-
ysis and machine intelligence, 2020.
19. Ke Sun, Bin Xiao, Dong Liu, and Jingdong Wang. Deep high-resolution repre-
sentation learning for human pose estimation. In Proceedings of the IEEE/CVF
Conference on Computer Vision and Pattern Recognition, pages 5693–5703, 2019.
20. Louis M Lin, Joseph E Skribner, and Peter Gaengler. Factors associated with
endodontic treatment failures. Journal of endodontics, 18(12):625–627, 1992.You can also read
