Deep Faster Detection of Faint Edges in Noisy Images
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Deep Faster Detection of Faint Edges in Noisy Images
Nati Ofir
Abstract
arXiv:1803.09420v1 [cs.CV] 26 Mar 2018
Detection of faint edges in noisy images is a challenging
problem studied in the last decades. [10] introduced a fast
method to detect faint edges in the highest accuracy among Figure 1. Example of a medical image with many curved edges.
all the existing approaches. Their complexity is nearly lin- Left: the original image. Middle: our DeepFaster method result.
ear in the image’s pixels and their runtime is seconds for Right: [10] FastEdges results. Both methods achieve high quality
a noisy image. By utilizing the U-net architecture [11], of detection while ours run in milliseconds and FastEdges runtime
is more than seconds.
we show in this paper that their method can be dramati-
cally improved in both aspects of run time and accuracy.
By training the network on a dataset of binary images, we previous work on edge detection including faint edges and
develop a method for faint edge detection that works in a noisy images. In Section 3 we explain how we trained our
linear complexity. Our runtime on a noisy image is mil- network and developed the faster algorithm. Then, in Sec-
liseconds on a GPU. Even though our method is orders of tion 4 we evaluate our scheme and compare it to [10] and to
magnitude faster, we still achieve higher accuracy of detec- [2] on both noisy simulations and real images.
tion under many challenging scenarios.
2. Previous Work
1. Introduction Edge detection is a fundamental problem of image pro-
cessing and computer vision with many related works. Marr
Edges detection is one of the most classical problems and Hildreth [9] offered to detect edges by zero crossing of
of computer vision. Many works addressed this problem the 2D Laplacian applied to an images. Sobel [5] detects
and introduced a variety of solutions. Unfortunately, some edges by applying a 3x3 derivative filter on an images, and
imaging domains suffer from faint edges and noisy im- computing the gradients. Canny [2] extends Sobel by hys-
ages, such as medical, satellite and even real natural im- teresis thresholding of the local gradients. These classical
ages. Detection of edges under these challenging conditions approaches are very fast but unfortunately very sensitive to
should be done by methods geared for that end. Existing image noise and cannot detect accurately faint edges at all.
approaches that deal with high level of noise are all rela- Advanced group of works are focused on the problem of
tively slow (runtime of seconds for an image). This work is boundary detection and segmentation [1, 7, 3]. These meth-
the first to use deep learning for that purpose of faint edge ods achieves high quality results on Berkeley Segmenta-
detection. By training the U-net [11] on a simulated faint- tions Dataset (BSDS500) [12]. A recent group of works are
edges dataset, we developed a new algorithms that deal with optimized and trained on this dataset utilizing deep learning
noise. Since a forward pass of a network can be optimized tools [8, 14]. Even though these approaches are great for
on a GPU, our algorithm is real time and is order of mag- boundary detection they suffer from the presence of noise
nitude faster than existing approaches. Even though, our as can be seen in the experiments of [10].
experiments demonstrate that it is yet more accurate. The specific problem of faint edge detection in noisy im-
See Figure 1 for example of a medical image and its edge ages is addressed by a group of recent papers. [4] was the
images. Our method in the middle (DeepFaster) is com- first to detect faint edges by difference of oriented means.
pared to [10] (FastEdges) on the right. The edge images are They utilized matched filters that averages the noisy from
plotting the blood vessels, both achieve good results and each side along an edge, and maximizes the contrast across
manage to track the vessels, while only our method can be the edge. Their method is limited to straight lines. Denote
run in real time since our network is fully convolutional and by N the number of pixels in an images, thus their com-
its runtime does not scale a lot with the size of the image. putational complexity is O(N logN ). [10] extended their
The paper is organized as follow, in Section 2 we cover the work to curved edges, and by utilization of a dynamic pro-
1gramming and approximations achieves better accuracy at
a complexity of O(N logN ). The actual run time of these
methods on a noisy images is seconds. [6, 13] introduced
a sub-linear approaches for detection of straight and curved
edges. Our work is utilizing deep-learning to improve these
recent works. We introduce a O(N ) algorithm, that its
actual runtime is negligible due to GPU acceleration. Al-
though our method is faster, we achieve even more accurate Figure 2. The sigmoid activation function used on top of our deep
network. This activations turns our network into binary classifica-
results when detecting faint edges in noisy images.
tion procedure. High values are mapped to 1 indicating detections,
low values are mapped to 0 indicating no objects.
3. Detection Network Learning
For the purpose of detecting faint edges in noisy images,
we tackle this problem as a binary image segmentation. The
network we used is U-net [11] and its name describes its ar-
chitecture. The network downscales the images along the
first half of the layers by convolutions with relu activations
and max polling. Then, in the second half of the layers it
upscales the images by convolution transpose and relu acti-
vation. In addition, it has connections between the first and
the second halves by concatenations of every two layers in
the same resolution. This architecture assists to create a
multi-scale algorithms for detection and segmentations. On Figure 3. The dice coefficients loss across the training epochs. It
top of the last later, we apply a sigmoid activations. See can be seen that the loss decreases dramatically and that the graphs
of train and test are very close one to each other. These finding
Figure 2 for the graphs of the sigmoid. This activation turns
indicate a good learning process with no over fitting.
the network into binary classification procedure. High val-
ues are mapped to 1 while low values are mapped to zero.
The loss we use for training is dice coefficients, denote by different snr’s and different polarities of the binary patters.
Y 0 the binary labels of the edges, and by Y the output of Moreover, we added samples of clean noise with no labels.
the network for a given input X, then the computed loss Our dataset after augmentations contains around 17,000 ex-
function is: amples. We then divided the dataset to 90% training and
P 0 10% testing. See Figure 3 for our loss across the epochs
p Y (p) · Y (p)
−P 0 P , (1) graph. The loss decreases dramatically as the training ad-
p Y (p) + p Y (p) vances, and the graphs of train and test are very close one to
p is an image pixel. This loss corresponds to minus inter- each other, indicating that there is no over fitting. In the ex-
section of edges divided by the sum of absolute values. periments, we use this trained network and apply a thresh-
We initialize the network with random weights. To train old of 0.5 to the output of the sigmoid to obtain a binary
the network we use a dataset of 1406 binary images. Each image result.
image is clean originally. For ground truth we apply Canny
[2] on the clean binary images to extract the labels Y 0 . Then 4. Experiments
each image is used to create a group of noisy images in dif-
We test our method on both simulated images and real
ferent signal to noise ratios (snr’s) by the following formula:
images. We compared mainly to FastEdges [10], this
I = clip(0.1 ∗ (snr · Iclean + Inoise ) + 0.45). (2) method showed superior results in detection of faint edges
in noisy images. In addition, we compare to the classic
I is the noisy images, input to our network, Iclean is the Canny [2] approach. We test these methods in both aspects
original binary image, Inoise is a random Gaussian noise of quality and run time. To extract a measure of similarity
image width standard deviation of 1. snr is the measure between the binary results and the ground truth we use a
of the faintness of the edges, each binary image creates 6 strict version of F-measure. This meausre is strict because
training examples by using snr values from 1 to 2 in jumps it is pixel-wise and do not allow matching of neighboring
of 0.2. The clip function clips the values to the range of 0 pixels like the regular measure used in [10]. Therefore, the
to 1. values in our graph are lower than in [10], but this measure
We trained the network for 100 epochs on a geforce gtx emphasize better our method accuracy, since it is accurate
1070 GPU, for 1.5 hours. We augmented the dataset by in a pixel level. The F-measure is the harmonic mean of theAlgorithm SNR = 1 SNR = 2
DeepFaster 0.4 0.62
FastEdges 2.8 0.56
Canny 0.08 0.45
Table 1. F-score of the methods in snr 1 and 2. In both snr’s our
method achieves the highest score.
precision and recall:
2 · precision · recall
F = . (3) Figure 4. Simulation of faint edges detection in noisy image. Left:
precision + recall
the binary pattern used to evaluate the performance of the meth-
ods. Right: The strict F-score graph along the different signal-to-
Denote by Y 0 the labels and by Y the results, then precision
noise ratios from 0 to 2. The methods geared for faint edges detec-
is computed as follow,
tion achieve the highest accuracy. Our method DeepFaster obtains
a slightly higher graph from the previous approach of FastEdges
Y ·Y0
P
T rueP ositive
precision = = P . [10].
T rueP ositive + F alseP ositive Y
(4)
The recall is computed in a similar way,
Y ·Y0
P
T rueP ositive
recall = = P 0 .
T rueP ositive + F alseN egative Y
(5)
We compared the algorithms using a challenging binary
patters showed on Figure 4. This pattern contains triangle, Figure 5. Results on the simulation noisy image. Left: the original
straight lines, ’S’ shape and concentric circles. We used images at snr=2. Middle: our network result. Right: FastEdges
it to create a set of images from snr 0 to 2 in jumps of [10] result. Both methods produce good quality results while ours
0.1, contaminated with Gaussian additive noise. For each contains less false detections.
snr, we compute the F-meausre for every method for 100
iterations of different random noises as in Eq. (2). Then Algorithm Run-Time (milliseconds)
we take the average F-score of all the iterations. It can be DeepFaster 10
seen in the graph of Figure 4 that the methods geared to de- FastEdges 2600
tect faint edges in noisy images, ours and [10], are superior Canny 3
Table 2. Run time in milli-seconds of the different methods of edge
to Canny [2]. Moreover, our method using deep learning
detection. Our runtime is very close to Canny’s time and is order
techniques achieves slightly better results than the previous of magnitude faster than FastEdges. We achieve this improvement
method FastEdges. by running our network on a GPU.
Table 1 summarizes the results in the graph. It can be
seen that at both snr’s of 1 and 2, our DeepFaster method
achieves that highest F-score. Figure 5 shows the simula- at high curvatures. Moreover our results are very similar to
tion images of snr = 2. It compares our result to this of the ground truth labels.
FastEdges [10]. Both methods achieve a good accuracy of Figure 7 show ours and FastEdges [10] results on a group
detection on this image but ours contains less false detec- of real images. Both methods obtains high quality of detec-
tions. tion. However, since our network is fully convolutional, its
Even though our method is the most accurate, it achieves run-time does not scale significantly with size, and our run
a real-time complexity as can be shown in Table 2. Our run time on these images is much faster than FastEdges.
time is 10 millisecond, not a lot above Canny’s run time,
and it is orders of magnitude faster that FastEdges [10] that 5. Conclusions
runs in seconds. We computed these run times on a sin-
gle machine with i7 CPU, 32 GB of RAM, and geforce gtx We introduced a novel work for detection of faint edges
1070 GPU. Note that Canny and FastEdges runs on the CPU in noisy images. Our work is the first to solve this specific
while our new method utilized the parallelism of the GPU. problem by using deep learning technique. We compared
Figure 6 shows our method results on a noisy images our method to FastEdges [10] which is the state-of-the-art
from the dataset we used to train and test our network. It in faint edge detection. We showed experimentally that
can be seen that we manage to detect and track edges even we manage to improve this method in both aspects of runFigure 7. Examples of real images. Left: the original gray scale
Figure 6. Results on images from the dataset we used to train and
images. Middle: our results. Right: FastEdges [10] results. Both
test our network. Left: the input noisy images with a binary pat-
methods achieve high quality of detections.
tern. Middle: the ground truth labels. Right: our detections. Deep-
Faster result is very similar to the ground truth and we manage to
detect and track edges even at high curvatures.
In Computer Vision, 2007. ICCV 2007. IEEE 11th Interna-
tional Conference on, pages 1–8. IEEE, 2007. 1
[5] W. Gao, X. Zhang, L. Yang, and H. Liu. An improved sobel
time and quality. We achieved similar and better results on
edge detection. In Computer Science and Information Tech-
simulation and real images, while improving the run times nology (ICCSIT), 2010 3rd IEEE International Conference
in order of magnitude. FastEdges needed seconds to pro- on, volume 5, pages 67–71. IEEE, 2010. 1
cess an images whereas ours algorithm requires only milli- [6] I. Horev, B. Nadler, E. Arias-Castro, M. Galun, and R. Basri.
seconds by utilizing a fully convolutional network running Detection of long edges on a computational budget: A
on a GPU. sublinear approach. SIAM Journal on Imaging Sciences,
8(1):458–483, 2015. 2
References [7] P. Isola, D. Zoran, D. Krishnan, and E. H. Adelson. Crisp
boundary detection using pointwise mutual information. In
[1] P. Arbeláez, J. Pont-Tuset, J. T. Barron, F. Marques, and European Conference on Computer Vision, pages 799–814.
J. Malik. Multiscale combinatorial grouping. In Proceed- Springer, 2014. 1
ings of the IEEE conference on computer vision and pattern [8] K.-K. Maninis, J. Pont-Tuset, P. Arbeláez, and L. Van Gool.
recognition, pages 328–335, 2014. 1 Convolutional oriented boundaries: From image segmenta-
[2] J. Canny. A computational approach to edge detection. tion to high-level tasks. IEEE transactions on pattern analy-
In Readings in Computer Vision, pages 184–203. Elsevier, sis and machine intelligence, 40(4):819–833, 2018. 1
1987. 1, 2, 3 [9] D. Marr and E. Hildreth. Theory of edge detection. Proc. R.
[3] P. Dollár and C. L. Zitnick. Structured forests for fast edge Soc. Lond. B, 207(1167):187–217, 1980. 1
detection. In Computer Vision (ICCV), 2013 IEEE Interna- [10] N. Ofir, M. Galun, B. Nadler, and R. Basri. Fast detection
tional Conference on, pages 1841–1848. IEEE, 2013. 1 of curved edges at low snr. In Proceedings of the IEEE Con-
[4] M. Galun, R. Basri, and A. Brandt. Multiscale edge detection ference on Computer Vision and Pattern Recognition, pages
and fiber enhancement using differences of oriented means. 213–221, 2016. 1, 2, 3, 4[11] O. Ronneberger, P. Fischer, and T. Brox. U-net: Convo-
lutional networks for biomedical image segmentation. In
International Conference on Medical image computing and
computer-assisted intervention, pages 234–241. Springer,
2015. 1, 2
[12] R. Unnikrishnan, C. Pantofaru, and M. Hebert. A measure
for objective evaluation of image segmentation algorithms.
In Computer Vision and Pattern Recognition-Workshops,
2005. CVPR Workshops. IEEE Computer Society Conference
on, pages 34–34. IEEE, 2005. 1
[13] Y.-Q. Wang, A. Trouvé, Y. Amit, and B. Nadler. Detecting
curved edges in noisy images in sublinear time. Journal of
Mathematical Imaging and Vision, 59(3):373–393, 2017. 2
[14] S. Xie and Z. Tu. Holistically-nested edge detection. In Pro-
ceedings of the IEEE international conference on computer
vision, pages 1395–1403, 2015. 1You can also read
