UTILIZING THE STRUCTURE OF THE CURVELET TRANSFORM WITH COMPRESSED SENSING
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
U TILIZING THE S TRUCTURE OF THE C URVELET T RANSFORM
WITH C OMPRESSED S ENSING
A P REPRINT
Nicholas Dwork∗ Peder E. Z. Larson
Department of Radiology and Biomedical Imaging Department of Radiology and Biomedical Imaging
University of California in San Francisco University of California in San Francisco
arXiv:2107.11664v1 [eess.IV] 24 Jul 2021
July 27, 2021
A BSTRACT
The discrete curvelet transform decomposes an image into a set of fundamental components that
are distinguished by direction and size as well as a low-frequency representation. The curvelet
representation is approximately sparse; thus, it is a useful sparsifying transformation to be used with
compressed sensing. Although the curvelet transform of a natural image is sparse, the low-frequency
portion is not. This manuscript presents a method to modify the sparsifying transformation to take
advantage of this fact. Instead of relying on sparsity for this low-frequency estimate, the Nyquist-
Shannon theorem specifies a square region to be collected centered on the 0 frequency. A Basis
Pursuit Denoising problem is solved to determine the missing details after modifying the sparisfying
transformation to take advantage of the known fully sampled region. Finally, by taking advantage
of this structure with a redundant dictionary comprised of both the wavelet and curvelet transforms,
additional gains in quality are achieved.
Keywords compressed sensing · curvelet · wavelet · imaging · MRI
1 Introduction
Compressed sensing permits accurate reconstructions of images with fewer samples than required to satisfy the
Nyquist-Shannon theorem. By solving a basis pursuit denoising (BPD) problem, one finds the solution of a corre-
sponding sparse signal recovery problem [1]. This technique has found many uses in inverse problems including
medical imaging [2], holography [3], photography [4], and communications [5, 6].
In these applications, the optimization variable is not often sparse in the standard basis. However, when expressed
in the coordinates of a different (possibly overcomplete) basis, it is sparse. The linear transformation that converts a
vector to the coordinates of this basis is the sparsifying transformation. Common choices include the wavelet transform
[7] and the curvelet transform [8], as well as learned dictionaries [9]. It is often beneficial to have fast algorithms that
express a vector in the basis of the relevant dictionary, and fast algorithms exist for wavelet [10, 11] and (for images)
the two-dimensional curvelet transforms [12]. Thus, we focus on these sparsifying transforms for this manuscript.
In [13], Dwork et al. show that a portion of the wavelet transform of natural images is not sparse, the portion corre-
sponding to the lowest frequency bin. They use this property with Fourier Sampling to specify the size and shape of a
region centered on the 0 frequency that is fully sampled, and to alter the optimization problem to solve for a modified
vector. The resulting problem can be converted into the standard BPD problem. Thus, image reconstruction becomes
a two step process: 1) estimate the low frequency portion of the image, and 2) enhance the image with high frequency
details that are the solution to a BPD problem. The system matrix remains the same, thus all theoretical guarantees of
compressed sensing pertain to the new BPD problem. However, since the sparsity of the optimization variable in the
new problem is higher than that of the original problem, the error of the result is reduced.
In this work, we show that a similar process can be applied when the curvelet transform is used as the sparsifying
transform. That is, we show that although the curvelet transform of natural images is sparse, there is a region of the
∗
www.nicholasdwork.com, nicholas.dwork@ucsf.eduA PREPRINT - J ULY 27, 2021
curvelet transform that is not sparse. And one can take advantage of this structure to alter the reconstruction algorithm
so that the error in the result is reduced1 .
When using the curvelet transform as the sparsifying transform instead of the wavelet transform, an orthogonal trans-
formation has been replaced with wide transformation. Intuitively, one would expect that increasing the redundancy of
the set of vectors would permit an even more sparse representation of an image. Suppose, for example, one wanted to
represent v = wi + ci where wi is an element of the wavelet basis and ci is an element of the curvelet basis [14]. If one
were to express v as a linear combination of vectors from both bases then the number of non-zero linear coefficients
required would be 2. However, if one were to try to represent the same image v using only the wavelet basis or only the
curvelet set, then it would require a larger number of significant linear coefficients. Indeed, after the benefit was shown
empirically [15, 16], the theory was developed to show that accurate reconstruction was possible with a redundant (or
overcomplete) incoherent set of vectors [17, 18].
In this manuscript, we also show how the structures of the wavelet and curvelet transforms can simultaneously be
exploited to alter the sampling pattern and reconstruction algorithm. Again, the theoretical guarantees of compressed
sensing that pertain to reconstruction with a redundant vector set remain. The increased sparsity further reduces the
theoretical bound on the error, and we will demonstrate that the error is reduced in practice with several images.
2 Background
The basis pursuit denoising problem has the following form:
minimize kyk1 subject to (1/2)kAy − βk2 < σ, (1)
y
where, k · kp is the Lp norm, A is the system matrix, β is the data vector, and σ > 0 is a bound on the noise. The theory
of compressed sensing dictates that when A satisfies specific properties (e.g. the Mutual Coherence Conditions [19],
the Restricted Isometry Property [17], or the Restricted Isometry Property in Levels (RIPL) [20]) then the solution to
(1) is the same as if one knew which coefficients were the largest a priori and specifically measured them [1].
With Fourier sensing, for a given sparsifying transformation Ψ, one solves the following optimization problem:
minimize kΨxk1 subject to (1/2)kM F x − bk2 < σ, (2)
x
where F is the unitary Discrete Fourier Transform, M is the sampling mask that identifies those samples that were
collected, and b is the vector of Fourier values. When Ψ is a tight frame matrix, this can be converted into the form of
(1) by letting y = Ψx. To find the coordinates in the sparsifying set, solve
minimize kyk1 subject to (1/2)kM F Ψ∗ y − bk2 < σ, (3)
y
where ∗ represents the adjoint.
Figure 1a shows an image of a dog eating from an ice cream container, and Fig. 1b shows the magnitude of the discrete
Daubechies wavelet transform of order 4 (DDWT-4) coefficients when applied recursively to the lowest frequency bin
4 times. The vast majority of the resulting coefficient appear black, which demonstrates the sparsifying behavior of
the DDWT-4. The upper left corner of the transform is not black; this is a low-pass filter and downsampling of the
original image, and will likely not be sparse for natural images.
Let W represent the orthogonal DDWT-4. In [13], Dwork et al. show that a fully sampled region with size equal to
that of the lowest frequency bin of the DDWT-4 centered on the 0 frequency can generate a low frequency estimate.
This is the same fully sampled region as specified by the two-level sampling scheme in [20]. By subtracting away this
estimate from the optimization variable, the sparsity is increased. The missing details of the image can be estimated
by solving the following problem:
minimize kΨ (x − xL ) k1 subject to (1/2)kM F x − bk2 < σ, (4)
y
where Ψ = W , xL = F ∗ KB ML b, and β = b − M F xL . Here, KB is a Kaiser Bessel window used to reduce
ringing in the blurry estimate, and ML is a mask that specifies the fully sampled low frequency region. Problem (4) is
equivalent to
minimize kyk1 subject to (1/2)kM F Ψ∗ y − βk2 < σ, (5)
y
where y = Ψ x. By letting A = M F Ψ∗, it becomes apparent that this problem has the form of the BPD problem (1).
Once the solution y ? is determined, the reconstructed image is x = xL + Ψ∗ y ? .
1
An early version of this work was presented at the 2021 annual meeting of the Society for Industrial and Applied Mathemati-
cians.
2A PREPRINT - J ULY 27, 2021
Figure 1: (a) Original image, (b) Magnitude of the discrete Daubechies-4 wavelet transform coefficients,
(c) Magnitude of curvelet transform coefficients.
Curvelets offer the same opportunity to increase the sparsity of the optimization variable. Curvelets offer a repre-
sentation of an image using a multi-scale pyramid with several directions available at each position (as opposed to
wavelets, which decompose an image into horizontal and vertical components). By using curvelets as the sparsifying
transform, one hopes to reduce the blocking artifacts common to wavelet representations [21]. Figure 1c shows the
magnitude of the discrete curvelet coefficients. Note that the curvelet is itself a wide transform (it is a frame [22]);
it has more coefficients than there are pixels in the original image. The display of the curvelets includes white-space
to ease interpretation; it was generated using the CurveLab software [23]. Similar to the low frequency bin of the
wavelet transform, the center portion of the curvelet transform is the result of windowing, low-pass filtering, and
downsampling the original image. As previously discussed, this region will not be sparse for most natural images.
3 Methods
In this section, we will discuss compressed sensing with curvelets as the sparsifying transformation. We will then
discuss compressed sensing using the redundant dictionary comprised of wavelet and curvelet vectors. In both cases,
we will utilize the structure of the sparsifying transform to better sample the Fourier domain and increase the sparsity
of the optimization variable. Results of reconstructing images with these algorithms are presented in section 4.
3.1 Curvelets as the Sparsifying Transform
Since sparsity cannot be assumed for low frequency region of the Curvelet transform, we rely on the Nyquist-Shannon
sampling theorem. This dictates a fully sampled region with spacing equal to the inverse of the full size of the image
and with size equal to the center portion of the curvelet transform. The values of the fully sampled region are used
to create a blurry image by applying a low pass window (W0 as specified in [12]) and then performing an inverse
discrete Fourier Transform. Figure 2a shows this blurry estimate for the image of Fig. 1a. Figure 2b shows the result
of subtracting this blurry estimate from the original image, and Fig. 2c shows the discrete Fourier transform of the
difference. When comparing Fig. 1c with Fig. 2c, the increased sparsity of the low frequency region is apparent.
This is generally the case for natural images: after subtracting the blurry estimate from the original image, the sparsity
of the curvelet transform is increased. The details of the image can be estimated by letting Ψ = C (the curvelet
transform) and xL = F ∗ W0 b, and then solving problem (4). Due to the increased sparsity of the optimization
variable, the error in the result is reduced. After solving for y ? , the reconstructed image is the sum of the blurry image
and the detailed image: xL + C ∗ y ? .
3.2 Redundancy Using Wavelets and Curvelets
Let B = [W C] where W is the matrix representation of the wavelet transform and C is the matrix representation of
the Curvelet transform. By using B as the sparsifying transform, the redundancy in the columns of B may offer a
more sparse representation of the image. Note that C is a tight frame matrix, meaning that it satisfies C ∗ C = κI for
some scalar κ. Since W is orthogonal, B is also a tight frame. The redundancy makes a more sparse representation of
the optimization variable available, leading to a reconstruction with a reduced error bound.
As discussed, both wavelets and curvelets have a region that is a low-pass filter and downsampling of the original
image. For wavelets, it is the lowest-frequency bin. And for curvelets, it is the center of the transform. The larger of
these two regions specifies a fully sampled region centered on 0 frequency to be collected that satisfies the Nyquist-
3A PREPRINT - J ULY 27, 2021
Figure 2: (a) Blurry estimate, (b) Result of subtracting the blurry estimate from the original image, (c)
Magnitude of curvelet transform coefficients of b. Note that the low frequency center region in c is much
more sparse than the corresponding region of Fig. 1c.
Shannon sampling theorem. From this fully sampled region, a blurry frequency estimate is created. If the center of
the curvelet is larger than the lowest-frequency bin of the wavelet transform, then the blurry estimate is generated
according to subsection 3.1. Otherwise, the blurry estimate is generated according to [13]. Denote this blurry estimate
as xL . Let Ψ = B. Then the image can be reconstructed by solving (5) for y ? and setting x = xL + B ∗ y ? .
4 Results
All images studied in this manuscript were 256 × 256 pixels squared, and all images were scaled so that pixel values
lied within [0, 1]. For all results presented, the Fast Iterative Shrinkage Threshold Algorithm (FISTA) with line search
[24, 25] run for 100 iterations was used to solve the equivalent Lagrangian form of the problem. For example, to solve
problem (1), the following equivalent optimization problem was solved:
minimize (1/2)kAx − βk22 + λkxk1 ,
where λ > 0 is the regularization variable. The values of λ considered for each problem were
5000, 2000, 1000, 500, 200, 100, 50, 20, 10, 5, 2, 1, 0.5, 0.2, and 0.1. The value of λ that yielded the smallest error
was considered as that of the optimal solution. The error metric reported is relative error, defined as
ktrue − estimatek2
e= .
kestimatek2
Three different sparsifying transformations were tested: DDWT-4, the wrapping version of the discrete curvelet trans-
form, and a concatenation of both of these transformation. For each transformation, two different sampling patters
were tested: a variable density sampling pattern and a variable density sampling pattern with a fully sampled center
region. The sampling patterns used for the results in this manuscript were generated using a separable Laplacian dis-
tribution as shown in Fig. 3. When a fully sampled region was included, the number of variable density samples was
reduced to retain the same total number of samples.
When solving problem (3), we label the result with the sparsifier: curvelets, wavelets, or redundant WavCurv (meaning
both wavelets and curvelets). When solving problem (5), we label the result as structured along with the sparsifier.
Figure 4 shows the reconstruction of the image from Fig. 1 using the curvelet sparsifier with a variable density
sampling pattern, the curvelet sparsifier with a smapling pattern that includes the fully sampled center region, the
structured curvelet sparsifier with the fully sampled center region, and the structured redundant WavCurv sparsifier
with the fully sampled center region. Utilizing the fully sampled center region improves the result significantly. Taking
advantage of the structure reduces the error further still. And the lowest error is achieved by taking advantage of the
structure and the redundant dictionary.
Figure 5 shows the results of structured reconstructions with the curvelet, the wavelet, and the redundant WavCurv
sparsifier on knee data taken from mridata.org [26]. The sampling pattern included the fully sampled center re-
gion. The structured curvelet sparsifier yields a lower error than the structured wavelet sparsifier, and the structured
redundant sparsifier yields the lowest error.
Figure 6 shows the results of the structured WavCurv sparsifier with the fully sampled center region included in the
sampling pattern for different percentages of collected samples. As expected, the quality degrades (the error increases)
as the number of samples increases.
4A PREPRINT - J ULY 27, 2021
Figure 3: Variable density sampling patterns generated as a realization of a separable Laplacian distribution with a
standard deviation of 75 pixels. The top/bottom row shows sampling patterns without/with the fully sampled center
region corresponding to the size of the low-frequency region of the curvelet transform, respectively. From left to right,
the sampling patterns include 10, 000, 12, 000, 15, 000, and 20, 000 point, respectively; this corresponds to 15%,
18%, 23%, and 31% of the total number of samples.
Figure 4: Reconstruction created with a) curvelet sparsifier, b) curvelet sparsifier with fully sampled center region,
c) structured curvelet sparsifier, and d) structure wavelet and curvelet sparsifier. The first row shows the reconstructed
images, the second row shows the difference images. The error for each reconstruction is printed below the second
row; the structured redundant sparsifier using wavelets and curvelets yields the lowest error.
5 Discussion and Conclusion
In this work, we show that the structure of the curvelet transform can be exploited to increase the quality of compressed
sensing image reconstructions. We further show that this structure can be simultaneously exploited in wavelets and
curvelets when using a redundant dictionary comprised of both transforms to improve the quality further still. Results
were shown on optical and magnetic resonance images.
Several neural networks have been proposed to perform compressed sensing reconstruction. Though these networks
can perform the reconstruction quickly, they are unstable [27, 28]. Other works learn the set of dictionary vectors
from a set of data [9]. When performing reconstruction of a general image, this may not be an appropriate solution,
as the dictionary would need to be immense in order to capture all types of details that could be presented. However,
these techniques may be appropriate when confined to a particular image type; e.g., MR images of the brain. For a
specific type of data, the learned dictionary may result in increased sparsity over the redundant dictionary comprised
5A PREPRINT - J ULY 27, 2021
Figure 5: Reconstruction of knee magnetic resonance imaging data from mridata.org with 15% of all samples
using structured compressed sensing with b) the DDWT-4 sparsifier, c) the curvelet sparsifier, and d) the redundant
sparsifier of wavelets and curvelets. The first row shows the reconstructed images, the second row shows the difference
images. The error for each reconstruction is printed below the second row; the structured redundant sparsifier using
wavelets and curvelets yields the lowest error.
Figure 6: Reconstructions created with the structured redundant sparsifier of wavelets and curvelets. The recon-
structed image is shown in the top row and the difference image (magnified by a factor of 4) is shown in the second
row. After the original image, the images are reconstructed with (from left to right) 31%, 23%, 18%, and 15% of the
total number of samples. The relative error is shown below the difference images; the error is increased as the amount
of data is decreased.
of wavelet and curvelet basis vectors. However, there are fast implementations of the wavelet and curvelet transforms
[10, 12] which could be beneficial for several applications. Learned dictionaries do not lend themselves to similarly
fast implementations.
Additional gains in reconstruction quality may be had by either replacing the curvelet sparsifier with wave-atoms
[29, 30] or hexagonal wavelets [31], or by augmenting the redundant dictionary with these overcomplete bases. We
leave these prospects as future work.
6A PREPRINT - J ULY 27, 2021
6 Acknowledgments
ND would like to thank the American Heart Association and the Quantitative Biosciences Institute at UCSF as funding
sources for this work. ND is supported by a Postdoctoral Fellowship of the American Heart Association. ND and PL
have been supported by the National Institute of Health’s Grant number NIH R01 HL136965.
The authors would like to thank Daniel O’Connor for the many useful discussions on optimization and frames.
References
[1] Emmanuel J Candès and Michael B Wakin. An introduction to compressive sampling. Signal Processing Mag-
azine, 25(2):21–30, 2008.
[2] Michael Lustig, David Donoho, and John M Pauly. Sparse MRI: The application of compressed sensing for rapid
MR imaging. Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic
Resonance in Medicine, 58(6):1182–1195, 2007.
[3] David J Brady, Kerkil Choi, Daniel L Marks, Ryoichi Horisaki, and Sehoon Lim. Compressive holography.
Optics express, 17(15):13040–13049, 2009.
[4] Yusuke Oike and Abbas El Gamal. CMOS image sensor with per-column σδ ADC and programmable com-
pressed sensing. IEEE Journal of Solid-State Circuits, 48(1):318–328, 2012.
[5] Hong Huang, Satyajayant Misra, Wei Tang, Hajar Barani, and Hussein Al-Azzawi. Applications of compressed
sensing in communications networks. arXiv preprint arXiv:1305.3002, 2013.
[6] Reza Nasiri Mahalati, Ruo Yu Gu, and Joseph M Kahn. Resolution limits for imaging through multi-mode fiber.
Optics express, 21(2):1656–1668, 2013.
[7] Corey A Baron, Nicholas Dwork, John M Pauly, and Dwight G Nishimura. Rapid compressed sensing recon-
struction of 3D non-cartesian MRI. Magnetic resonance in medicine, 79(5):2685–2692, 2018.
[8] Jianwei Ma. Improved iterative curvelet thresholding for compressed sensing and measurement. IEEE Transac-
tions on Instrumentation and Measurement, 60(1):126–136, 2010.
[9] Honglak Lee, Alexis Battle, Rajat Raina, and Andrew Y Ng. Efficient sparse coding algorithms. In Advances in
neural information processing systems, pages 801–808, 2007.
[10] Stephane G Mallat. A theory for multiresolution signal decomposition: the wavelet representation. In Funda-
mental Papers in Wavelet Theory, pages 494–513. Princeton University Press, 2009.
[11] Gregory Beylkin, Ronald Coifman, and Vladimir Rokhlin. Fast wavelet transforms and numerical algorithms. In
Fundamental Papers in Wavelet Theory, pages 741–783. Princeton University Press, 2009.
[12] Emmanuel Candes, Laurent Demanet, David Donoho, and Lexing Ying. Fast discrete curvelet transforms. mul-
tiscale modeling & simulation, 5(3):861–899, 2006.
[13] Nicholas Dwork, Daniel O’Connor, Corey A Baron, Ethan M.I. Johnson, Adam B. Kerr, John M. Pauly, and
Peder E.Z. Larson. Utilizing the wavelet transform’s structure in compressed sensing. Signal, Image and Video
Processing, pages 1–8, 2021.
[14] Jean-Luc Starck, Emmanuel J Candès, and David L Donoho. The curvelet transform for image denoising.
Transactions on image processing, 11(6):670–684, 2002.
[15] Gabriel Peyre. Best basis compressed sensing. IEEE Transactions on Signal Processing, 58(5):2613–2622, 2010.
[16] Mariya Doneva, Peter Börnert, Holger Eggers, Christian Stehning, Julien Sénégas, and Alfred Mertins. Com-
pressed sensing reconstruction for magnetic resonance parameter mapping. Magnetic Resonance in Medicine,
64(4):1114–1120, 2010.
[17] Emmanuel J Candès, Yonina C Eldar, Deanna Needell, and Paige Randall. Compressed sensing with coherent
and redundant dictionaries. Applied and Computational Harmonic Analysis, 31(1):59–73, 2011.
[18] Holger Rauhut, Karin Schnass, and Pierre Vandergheynst. Compressed sensing and redundant dictionaries.
Transactions on Information Theory, 54(5):2210–2219, 2008.
[19] David L Donoho and Michael Elad. Optimally sparse representation in general (nonorthogonal) dictionaries via
`1 minimization. Proceedings of the National Academy of Sciences, 100(5):2197–2202, 2003.
7A PREPRINT - J ULY 27, 2021
[20] Ben Adcock, Anders C Hansen, Clarice Poon, and Bogdan Roman. Breaking the coherence barrier: A new
theory for compressed sensing. In Forum of Mathematics, Sigma, volume 5. Cambridge University Press, 2017.
[21] AW-C Liew and Hong Yan. Blocking artifacts suppression in block-coded images using overcomplete wavelet
representation. Transactions on Circuits and Systems for Video Technology, 14(4):450–461, 2004.
[22] Ben Adcock and Daan Huybrechs. Frames and numerical approximation. Siam Review, 61(3):443–473, 2019.
[23] Curvelab. http://www.curvelet.org/software.html. Accessed: 2021-07-20.
[24] Amir Beck and Marc Teboulle. A fast iterative shrinkage-thresholding algorithm for linear inverse problems.
Journal on imaging sciences, 2(1):183–202, 2009.
[25] Katya Scheinberg, Donald Goldfarb, and Xi Bai. Fast first-order methods for composite convex optimization
with backtracking. Foundations of Computational Mathematics, 14(3):389–417, 2014.
[26] F Ong, S Amin, S Vasanawala, and M Lustig. Mridata.org: An open archive for sharing MRI raw data. In Proc.
Intl. Soc. Mag. Reson. Med, volume 26, 2018. www.mridata.org.
[27] Nina M Gottschling, Vegard Antun, Ben Adcock, and Anders C Hansen. The troublesome kernel: why deep
learning for inverse problems is typically unstable. arXiv preprint arXiv:2001.01258, 2020.
[28] Vegard Antun, Francesco Renna, Clarice Poon, Ben Adcock, and Anders C Hansen. On instabilities of deep
learning in image reconstruction and the potential costs of AI. Proceedings of the National Academy of Sciences,
117(48):30088–30095, 2020.
[29] Laurent Demanet and Lexing Ying. Curvelets and wave atoms for mirror-extended images. In Wavelets XII,
volume 6701, page 67010J. International Society for Optics and Photonics, 2007.
[30] Laurent Demanet and Lexing Ying. Wave atoms and sparsity of oscillatory patterns. Applied and Computational
Harmonic Analysis, 23(3):368–387, 2007.
[31] Wei Zhu and Ingrid Daubechies. Constructing curvelet-like bases and low-redundancy frames. arXiv preprint
arXiv:1910.06418, 2019.
8You can also read
