Foreword |
|
xvi | |
|
Acknowledgments |
|
xvii | |
|
|
xviii | |
|
|
xxiv | |
|
|
1 | (12) |
|
|
|
|
|
1.1 A Taxonomy of Deep Learning Approaches |
|
|
2 | (1) |
|
1.2 Deep Learning in Remote Sensing |
|
|
3 | (4) |
|
1.3 Deep Learning in Geosciences and Climate |
|
|
7 | (2) |
|
1.4 Book Structure and Roadmap |
|
|
9 | (4) |
|
Part I Deep Learning to Extract Information from Remote Sensing Images |
|
|
13 | (148) |
|
2 Learning Unsupervised Feature Representations of Remote Sensing Data with Sparse Convolutional Networks |
|
|
15 | (9) |
|
|
|
|
|
|
|
|
15 | (2) |
|
2.2 Sparse Unsupervised Convolutional Networks |
|
|
17 | (2) |
|
2.2.1 Sparsity as the Guiding Criterion |
|
|
17 | (1) |
|
|
18 | (1) |
|
|
18 | (1) |
|
|
19 | (3) |
|
2.3.1 Hyperspectral Image Classification |
|
|
19 | (2) |
|
2.3.2 Multisensor Image Fusion |
|
|
21 | (1) |
|
|
22 | (2) |
|
3 Generative Adversarial Networks in the Geosciences |
|
|
24 | (13) |
|
|
|
|
|
|
24 | (1) |
|
3.2 Generative Adversarial Networks |
|
|
25 | (3) |
|
|
25 | (1) |
|
|
26 | (1) |
|
3.2.3 Cycle-consistent GANs |
|
|
27 | (1) |
|
3.3 GANs in Remote Sensing and Geosciences |
|
|
28 | (3) |
|
3.3.1 GANs in Earth Observation |
|
|
28 | (2) |
|
3.3.2 Conditional GANs in Earth Observation |
|
|
30 | (1) |
|
3.3.3 CycleGANs in Earth Observation |
|
|
30 | (1) |
|
3.4 Applications of GANs in Earth Observation |
|
|
31 | (5) |
|
3.4.1 Domain Adaptation Across Satellites |
|
|
31 | (2) |
|
3.4.2 Learning to Emulate Earth Systems from Observations |
|
|
33 | (3) |
|
3.5 Conclusions and Perspectives |
|
|
36 | (1) |
|
4 Deep Self-taught Learning in Remote Sensing |
|
|
37 | (9) |
|
|
|
37 | (1) |
|
4.2 Sparse Representation |
|
|
38 | (2) |
|
4.2.1 Dictionary Learning |
|
|
39 | (1) |
|
4.2.2 Self-taught Learning |
|
|
40 | (1) |
|
4.3 Deep Self-taught Learning |
|
|
40 | (5) |
|
4.3.1 Application Example |
|
|
43 | (1) |
|
4.3.2 Relation to Deep Neural Networks |
|
|
44 | (1) |
|
|
45 | (1) |
|
5 Deep Learning-based Semantic Segmentation in Remote Sensing |
|
|
46 | (21) |
|
|
|
|
|
|
46 | (1) |
|
|
47 | (2) |
|
5.3 Basics on Deep Semantic Segmentation: Computer Vision Models |
|
|
49 | (6) |
|
5.3.1 Architectures for Image Data |
|
|
49 | (3) |
|
5.3.2 Architectures for Point-clouds |
|
|
52 | (3) |
|
|
55 | (11) |
|
5.4.1 Encoding Invariances to Train Smaller Models: The example of Rotation |
|
|
55 | (4) |
|
5.4.2 Processing 3D Point Clouds as a Bundle of Images: SnapNet |
|
|
59 | (3) |
|
5.4.3 Lake Ice Detection from Earth and from Space |
|
|
62 | (4) |
|
|
66 | (1) |
|
6 Object Detection in Remote Sensing |
|
|
67 | (23) |
|
|
|
|
|
|
67 | (5) |
|
6.1.1 Problem Description |
|
|
67 | (2) |
|
6.1.2 Problem Settings of Object Detection |
|
|
69 | (1) |
|
6.1.3 Object Representation in Remote Sensing |
|
|
69 | (1) |
|
|
69 | (1) |
|
6.1.4.1 Precision-Recall Curve |
|
|
70 | (1) |
|
6.1.4.2 Average Precision and Mean Average Precision |
|
|
71 | (1) |
|
|
71 | (1) |
|
6.2 Preliminaries on Object Detection with Deep Models |
|
|
72 | (3) |
|
6.2.1 Two-stage Algorithms |
|
|
72 | (1) |
|
|
72 | (1) |
|
|
73 | (1) |
|
6.2.2 One-stage Algorithms |
|
|
73 | (1) |
|
|
73 | (1) |
|
|
73 | (2) |
|
6.3 Object Detection in Optical RS Images |
|
|
75 | (11) |
|
|
75 | (1) |
|
|
75 | (1) |
|
6.3.1.2 Orientation Variance |
|
|
75 | (1) |
|
6.3.1.3 Oriented Object Detection |
|
|
75 | (1) |
|
6.3.1.4 Detecting in Large-size Images |
|
|
76 | (1) |
|
6.3.2 Datasets and Benchmark |
|
|
77 | (1) |
|
|
77 | (1) |
|
|
77 | (1) |
|
|
77 | (1) |
|
|
77 | (1) |
|
6.3.3 Two Representative Object Detectors in Optical RS Images |
|
|
78 | (1) |
|
|
78 | (4) |
|
|
82 | (4) |
|
6.4 Object Detection in SAR Images |
|
|
86 | (3) |
|
6.4.1 Challenges of Detection in SAR Images |
|
|
86 | (1) |
|
|
86 | (2) |
|
6.4.3 Datasets and Benchmarks |
|
|
88 | (1) |
|
|
89 | (1) |
|
7 Deep Domain Adaptation in Earth Observation |
|
|
90 | (615) |
|
|
|
Bharath Bhushan Damodaran |
|
|
|
|
|
90 | (1) |
|
7.2 Families of Methodologies |
|
|
91 | (2) |
|
|
93 | (11) |
|
7.3.1 Adapting the Inner Representation |
|
|
93 | (4) |
|
7.3.2 Adapting the Inputs Distribution |
|
|
97 | (3) |
|
7.3.3 Using (few, well chosen) Labels from the Target Domain |
|
|
100 | (4) |
|
|
104 | (1) |
|
8 Recurrent Neural Networks and the Temporal Component |
|
|
105 | (1) |
|
|
|
8.1 Recurrent Neural Networks |
|
|
106 | (5) |
|
|
107 | (1) |
|
8.1.1.1 Exploding and Vanishing Gradients |
|
|
107 | (2) |
|
8.1.1.2 Circumventing Exploding and Vanishing Gradients |
|
|
109 | (2) |
|
8.2 Gated Variants of RNNs |
|
|
111 | (3) |
|
8.2.1 Long Short-term Memory Networks |
|
|
111 | (1) |
|
8.2.1.1 The Cell State ct and the Hidden State h1 |
|
|
112 | (1) |
|
8.2.1.2 The Forget Gate ft |
|
|
112 | (1) |
|
8.2.1.3 The Modulation Gate v1 and the Input Gate i1 |
|
|
112 | (1) |
|
8.2.1.4 The Output Gate o1 |
|
|
112 | (1) |
|
8.2.1.5 Training LSTM Networks |
|
|
113 | (1) |
|
8.2.2 Other Gated Variants |
|
|
113 | (1) |
|
8.3 Representative Capabilities of Recurrent Networks |
|
|
114 | (3) |
|
8.3.1 Recurrent Neural Network Topologies |
|
|
114 | (1) |
|
|
115 | (2) |
|
8.4 Application in Earth Sciences |
|
|
117 | (1) |
|
|
118 | (2) |
|
9 Deep Learning for Image Matching and Co-registration |
|
|
120 | (16) |
|
|
|
|
|
|
120 | (3) |
|
|
123 | (3) |
|
9.2.1 Classical Approaches |
|
|
123 | (1) |
|
9.2.2 Deep Learning Techniques for Image Matching |
|
|
124 | (1) |
|
9.2.3 Deep Learning Techniques for Image Registration |
|
|
125 | (1) |
|
9.3 Image Registration with Deep Learning |
|
|
126 | (8) |
|
9.3.1 2D Linear and Deformable Transformer |
|
|
126 | (1) |
|
9.3.2 Network Architectures |
|
|
127 | (1) |
|
9.3.3 Optimization Strategy |
|
|
128 | (1) |
|
9.3.4 Dataset and Implementation Details |
|
|
129 | (1) |
|
9.3.5 Experimental Results |
|
|
129 | (5) |
|
9.4 Conclusion and Future Research |
|
|
134 | (2) |
|
9.4.1 Challenges and Opportunities |
|
|
134 | (1) |
|
9.4.1.1 Dataset with Annotations |
|
|
134 | (1) |
|
9.4.1.2 Dimensionality of Data |
|
|
135 | (1) |
|
9.4.1.3 Multitemporal Datasets |
|
|
135 | (1) |
|
9.4.1.4 Robustness to Changed Areas |
|
|
135 | (1) |
|
10 Multisource Remote Sensing Image Fusion |
|
|
136 | (14) |
|
|
|
|
|
|
|
136 | (1) |
|
|
137 | (6) |
|
10.2.1 Survey of Pansharpening Methods Employing Deep Learning |
|
|
137 | (3) |
|
10.2.2 Experimental Results |
|
|
140 | (1) |
|
10.2.2.1 Experimental Design |
|
|
140 | (1) |
|
10.2.2.2 Visual and Quantitative Comparison in Pansharpening |
|
|
140 | (3) |
|
10.3 Multiband Image Fusion |
|
|
143 | (5) |
|
10.3.1 Supervised Deep Learning-based Approaches |
|
|
143 | (2) |
|
10.3.2 Unsupervised Deep Learning-based Approaches |
|
|
145 | (1) |
|
10.3.3 Experimental Results |
|
|
146 | (1) |
|
10.3.3.1 Comparison Methods and Evaluation Measures |
|
|
146 | (1) |
|
10.3.3.2 Dataset and Experimental Setting |
|
|
146 | (1) |
|
10.3.3.3 Quantitative Comparison and Visual Results |
|
|
147 | (1) |
|
10.4 Conclusion and Outlook |
|
|
148 | (2) |
|
11 Deep Learning for Image Search and Retrieval in Large Remote Sensing Archives |
|
|
150 | (11) |
|
|
|
|
|
150 | (2) |
|
11.2 Deep Learning for RS CBIR |
|
|
152 | (4) |
|
11.3 Scalable RS CBIR Based on Deep Hashing |
|
|
156 | (3) |
|
11.4 Discussion and Conclusion |
|
|
159 | (2) |
|
|
160 | (1) |
|
Part II Making a Difference in the Geosciences With Deep Learning |
|
|
161 | (122) |
|
12 Deep Learning for Detecting Extreme Weather Patterns |
|
|
163 | (23) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
12.1 Scientific Motivation |
|
|
163 | (3) |
|
12.2 Tropical Cyclone and Atmospheric River Classification |
|
|
166 | (4) |
|
|
166 | (1) |
|
12.2.2 Network Architecture |
|
|
167 | (2) |
|
|
169 | (1) |
|
|
170 | (5) |
|
12.3.1 Analytical Approach |
|
|
170 | (1) |
|
|
171 | (1) |
|
|
172 | (2) |
|
|
174 | (1) |
|
12.4 Semi-supervised Classification and Localization of Extreme Events |
|
|
175 | (4) |
|
12.4.1 Applications of Semi-supervised Learning in Climate Modeling |
|
|
175 | (1) |
|
12.4.1.1 Supervised Architecture |
|
|
176 | (1) |
|
12.4.1.2 Semi-supervised Architecture |
|
|
176 | (1) |
|
|
176 | (1) |
|
12.4.2.1 Frame-wise Reconstruction |
|
|
176 | (2) |
|
12.4.2.2 Results and Discussion |
|
|
178 | (1) |
|
12.5 Detecting Atmospheric Rivers and Tropical Cyclones Through Segmentation Methods |
|
|
179 | (5) |
|
|
179 | (1) |
|
12.5.1.1 Segmentation Architecture |
|
|
180 | (1) |
|
12.5.1.2 Climate Dataset and Labels |
|
|
181 | (1) |
|
12.5.2 Architecture Innovations: Weighted Loss and Modified Network |
|
|
181 | (2) |
|
|
183 | (1) |
|
12.6 Challenges and Implications for the Future |
|
|
184 | (1) |
|
|
185 | (1) |
|
13 Spatio-temporal Autoencoders in Weather and Climate Research |
|
|
186 | (18) |
|
|
|
|
|
|
186 | (1) |
|
|
187 | (6) |
|
13.2.1 A Brief History of Autoencoders |
|
|
188 | (1) |
|
13.2.2 Archetypes of Autoencoders |
|
|
189 | (2) |
|
13.2.3 Variational Autoencoders (VAE) |
|
|
191 | (1) |
|
13.2.4 Comparison Between Autoencoders and Classical Methods |
|
|
192 | (1) |
|
|
193 | (10) |
|
13.3.1 Use of the Latent Space |
|
|
193 | (2) |
|
13.3.1.1 Reduction of Dimensionality for the Understanding of the System Dynamics and its Interactions |
|
|
195 | (4) |
|
13.3.1.2 Dimensionality Reduction for Feature Extraction and Prediction |
|
|
199 | (1) |
|
13.3.2 Use of the Decoder |
|
|
199 | (2) |
|
13.3.2.1 As a Random Sample Generator |
|
|
201 | (1) |
|
13.3.2.2 Anomaly Detection |
|
|
201 | (1) |
|
13.3.2.3 Use of a Denoising Autoencoder (DAE) Decoder |
|
|
202 | (1) |
|
13.4 Conclusions and Outlook |
|
|
203 | (1) |
|
14 Deep Learning to Improve Weather Predictions |
|
|
204 | (14) |
|
|
|
|
14.1 Numerical Weather Prediction |
|
|
204 | (3) |
|
14.2 How Will Machine Learning Enhance Weather Predictions? |
|
|
207 | (1) |
|
14.3 Machine Learning Across the Workflow of Weather Prediction |
|
|
208 | (5) |
|
14.4 Challenges for the Application of ML in Weather Forecasts |
|
|
213 | (3) |
|
|
216 | (2) |
|
15 Deep Learning and the Weather Forecasting Problem: Precipitation Nowcasting |
|
|
218 | (22) |
|
|
|
|
|
|
|
|
218 | (2) |
|
|
220 | (1) |
|
|
221 | (2) |
|
|
223 | (10) |
|
|
223 | (2) |
|
|
225 | (1) |
|
15.4.3 Encoder-forecaster Structure |
|
|
226 | (1) |
|
15.4.4 Convolutional LSTM |
|
|
226 | (1) |
|
15.4.5 ConvLSTM with Star-shaped Bridge |
|
|
227 | (1) |
|
|
228 | (1) |
|
15.4.7 Memory in Memory Network |
|
|
229 | (2) |
|
|
231 | (2) |
|
|
233 | (3) |
|
|
234 | (1) |
|
15.5.2 Evaluation Methodology |
|
|
234 | (1) |
|
15.5.3 Evaluated Algorithms |
|
|
235 | (1) |
|
15.5.4 Evaluation Results |
|
|
236 | (1) |
|
|
236 | (4) |
|
|
238 | (1) |
|
|
239 | (1) |
|
16 Deep Learning for High-dimensional Parameter Retrieval |
|
|
240 | (18) |
|
|
|
240 | (2) |
|
16.2 Deep Learning Parameter Retrieval Literature |
|
|
242 | (2) |
|
|
242 | (1) |
|
|
243 | (1) |
|
|
244 | (1) |
|
16.2.4 Global Weather Models |
|
|
244 | (1) |
|
16.3 The Challenge of High-dimensional Problems |
|
|
244 | (6) |
|
16.3.1 Computational Load of CNNs |
|
|
247 | (2) |
|
16.3.2 Mean Square Error or Cross-entropy Optimization? |
|
|
249 | (1) |
|
16.4 Applications and Examples |
|
|
250 | (7) |
|
16.4.1 Utilizing High-Dimensional Spatio-spectral Information with CNNs |
|
|
250 | (3) |
|
16.4.2 The Effect of Loss Functions in Retrieval of Sea Ice Concentrations |
|
|
253 | (4) |
|
|
257 | (1) |
|
17 A Review of Deep Learning for Cryospheric Studies |
|
|
258 | (11) |
|
|
|
258 | (2) |
|
17.2 Deep-learning-based Remote Sensing Studies of the Cryosphere |
|
|
260 | (5) |
|
|
260 | (1) |
|
|
261 | (1) |
|
|
262 | (1) |
|
|
263 | (1) |
|
|
264 | (1) |
|
|
265 | (1) |
|
17.3 Deep-learning-based Modeling of the Cryosphere |
|
|
265 | (1) |
|
17.4 A Summary and Prospect |
|
|
266 | (3) |
|
Appendix: List of Data and Codes |
|
|
267 | (2) |
|
18 Emulating Ecological Memory with Recurrent Neural Networks |
|
|
269 | (14) |
|
|
|
|
18.1 Ecological Memory Effects: Concepts and Relevance |
|
|
269 | (1) |
|
18.2 Data-driven Approaches for Ecological memory Effects |
|
|
270 | (2) |
|
18.2.1 A Brief Overview of Memory Effects |
|
|
270 | (1) |
|
18.2.2 Data-driven Methods for Memory Effects |
|
|
271 | (1) |
|
18.3 Case Study: Emulating a Physical Model Using Recurrent Neural Networks |
|
|
272 | (4) |
|
18.3.1 Physical Model Simulation Data |
|
|
272 | (1) |
|
18.3.2 Experimental Design |
|
|
273 | (1) |
|
18.3.3 RNN Setup and Training |
|
|
274 | (2) |
|
18.4 Results and Discussion |
|
|
276 | (5) |
|
18.4.1 The Predictive Capability Across Scales |
|
|
276 | (3) |
|
18.4.2 Prediction of Seasonal Dynamics |
|
|
279 | (2) |
|
|
281 | (2) |
|
Part III Linking Physics and Deep Learning Models |
|
|
283 | (48) |
|
19 Applications of Deep Learning in Hydrology |
|
|
285 | (13) |
|
|
|
|
285 | (1) |
|
19.2 Deep Learning Applications in Hydrology |
|
|
286 | (10) |
|
19.2.1 Dynamical System Modeling |
|
|
286 | (1) |
|
19.2.1.1 Large-scale Hydrologic Modeling with Big Data |
|
|
286 | (4) |
|
19.2.1.2 Data-limited LSTM Applications |
|
|
290 | (2) |
|
19.2.2 Physics-constrained Hydrologic Machine Learning |
|
|
292 | (1) |
|
19.2.3 Information Retrieval for Hydrology |
|
|
293 | (1) |
|
19.2.4 Physically-informed Machine Learning for Subsurface Flow and Reactive Transport Modeling |
|
|
294 | (2) |
|
19.2.5 Additional Observations |
|
|
296 | (1) |
|
19.3 Current Limitations and Outlook |
|
|
296 | (2) |
|
20 Deep Learning of Unresolved Turbulent Ocean Processes in Climate Models |
|
|
298 | (9) |
|
|
|
|
298 | (1) |
|
20.2 The Parameterization Problem |
|
|
299 | (1) |
|
20.3 Deep Learning Parameterizations of Subgrid Ocean Processes |
|
|
300 | (1) |
|
20.3.1 Why DL for Subgrid Parameterizations? |
|
|
300 | (1) |
|
20.3.2 Recent Advances in DL for Subgrid Parameterizations |
|
|
300 | (1) |
|
20.4 Physics-aware Deep Learning |
|
|
301 | (2) |
|
20.5 Further Challenges ahead for Deep Learning Parameterizations |
|
|
303 | (4) |
|
21 Deep Learning for the Parametrization of Subgrid Processes in Climate Models |
|
|
307 | (8) |
|
|
|
|
|
307 | (2) |
|
21.2 Deep Neural Networks for Moist Convection (Deep Clouds) Parametrization |
|
|
309 | (3) |
|
21.3 Physical Constraints and Generalization |
|
|
312 | (2) |
|
|
314 | (1) |
|
22 Using Deep Learning to Correct Theoretically-derived Models |
|
|
315 | (13) |
|
|
22.1 Experiments with the Lorenz '96 System |
|
|
317 | (7) |
|
22.1.1 The Lorenz '96 Equations and Coarse-scale Models |
|
|
318 | (1) |
|
22.1.1.1 Theoretically-derived Coarse-scale Model |
|
|
318 | (1) |
|
22.1.1.2 Models with ANNs |
|
|
319 | (1) |
|
|
320 | (1) |
|
22.1.2.1 Single-timestep Tendency Prediction Errors |
|
|
320 | (1) |
|
22.1.2.2 Forecast and Climate Prediction Skill |
|
|
321 | (3) |
|
22.1.3 Testing Seamless Prediction |
|
|
324 | (1) |
|
22.2 Discussion and Outlook |
|
|
324 | (3) |
|
22.2.1 Towards Earth System Modeling |
|
|
325 | (1) |
|
22.2.2 Application to Climate Change Studies |
|
|
326 | (1) |
|
|
327 | (1) |
|
|
328 | (3) |
|
|
|
|
Bibliography |
|
331 | (70) |
Index |
|
401 | |