Transfer learning for low frequency extrapolation from shot gathers for FWI applications

oleg.ovcharenko@kaust.edu.sa
O. Ovcharenko, V. Kazei, D. Peter, T. Alkhalifah
Transfer learning for low frequency extrapolation
from shot gathers for FWI applications
June 6th, 2019

oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Team 2
Vladimir Kazei,
Post-doctoral Fellow
Oleg Ovcharenko,
PhD student
Tariq Alkhalifah,
Professor
Daniel Peter,
Assistant Professor
KAUST
Saudi Arabia

Temperature on June 6th

Max: 41°C

Min: 23°C

Average: 32°C

Today in London: ~18°C

Team 3
Vladimir Kazei,
Post-doctoral Fellow
Oleg Ovcharenko,
PhD student
Tariq Alkhalifah,
Professor
Daniel Peter,
Assistant Professor
KAUST
Saudi Arabia

Temperature on June 6th

Max: 41°C

Min: 23°C

Average: 32°C

Today in London: ~18°C

Previous work 4
2017
2018
Today
“Low-Frequency Data Extrapolation Using a Feed-
Forward ANN”,

80th EAGE Annual conference, 2018
“Neural network based low-frequency data
extrapolation”,

SEG FWI Workshop: Where are we getting?
“Transfer learning for low frequency extrapolation
from shot gathers for FWI applications”,

81st EAGE Annual conference, 2019

Limitations of real-world acquisitions 5
Lack of low-frequency data

- Due to instrumental limitations

- Due to noise

Low-frequency data in FWI 6
- Inverts large-scale velocity structures

- Less chance to get stuck in local minima

- Reveals deep model structures / below salt
fHigh
fLow
Multiple
local minima
Smooth
(Kazei et al., 2016)

FWI with different misﬁts 7
(Bozdag 2011, Choi & Alkhalifah 2013, Leeuwen & Herrmann, 2014 …)
Pros:
- Established workflow
- Direct image quality control
Cons:
- Computational costs
- Prone to event mismatching
- Sensitivity hard to control
Kalita et al., 2018

FWI with update conditioning 8
(Alkhalifah, 2015; Kazei, et al., 2016;
Yao et al., 2018; Ovcharenko, et al., 2018 ) etc…
Pros:
- Easily accessible sensitivity
- Direct image quality control
- Can be used together with any misfit
Cons:
- Computational costs
- Prone to event mismatching
Ovcharenko, et al., 2018

9Extrapolation of low-frequency data
Pros:
- Cheaper computations
Cons:
- Not well explored robustness
- Wavefield approximations
Hu et al., 2014; Li & Demanet, 2015, 2016, Ovcharenko et al., 2018)
etc…

10
This work: shot-to-shot extrapolation
Beat tone inversion
(Hu et al., 2014)
Bandwidth extension for atomic events
(Li & Demanet, 2015, 2016)
Deep learning freq domain for CSG – Ovcharenko et al., 2017,
Trace to trace deep learning – Sun & Demanet, 2018
Extrapolation of low-frequency data

Why extrapolation should be possible? 11

12Why extrapolation should be possible?

One trace, one shot, one dataset 13
Accuracy
Computational complexity
Trace-to-Trace
Shot-to-Shot
Data-to-Data
(Ovcharenko et al, 2017, 2018)
(Sun & Demanet, 2018)

14Mono-frequency shot gather

Supervised learning to link data in frequency domain 15
One network is trained to extrapolate for One frequency

ML to learn the phenomena 16
High Low
?
Non-linear function

17
Training dataset

Generators which didn’t work well 18

Generator on 1D proﬁles 19

Realistic random models based on wavelets and style transfer 20
“Style transfer for generation of realistically textured subsurface models”,

Ovcharenko et al., SEG Technical Program Expanded Abstracts 2019

“Realistically Textured Random Velocity Models for Deep Learning Applications”,

Kazei et al., 81st EAGE Conference and Exhibition 2019

Size of training dataset 21
Total training samples

=

random models

x

shots in the model

22
Deep learning model

Selection of network architecture 23
Custom design Ready-to-use design

Selection of network architecture 24
Custom design Ready-to-use design

Learned ﬁlters applied to a different dataset 25

Benchmark of network architectures 26
(Bianco et al., 2018,

"Benchmark Analysis of Representative

Deep Neural Network Architectures”)

Benchmark of network architectures 27
(Bianco et al., 2018,

"Benchmark Analysis of Representative

Deep Neural Network Architectures”)

MobileNet (Howard et al., 2017) 28
NVIDIA Titan V
TensorFlow 1.12.0Python 3.6
Keras 2.2.4
Matlab R2016b
Training for one frequency ~ 5 min

Inference time < 1 sec

Total params ~ 3.9M

29
Examples

30Input and target data
(R x F x 2) (R x 1 x 2)

31Parametrization
(R x F x 3) (R x 1 x 2)

32
Example 1/2. Central part of BP 2004

FWI for the central part of BP 2004 benchmark model 33
64 sources and receivers

32 known frequency in range 3-5 Hz
https://github.com/vkazei/fastFWI
Successive mono-frequency inversions at

0.25 0.55 0.93 2.04 2.66 3.46 4.50 Hz
Acoustic

Extrapolated low-frequency data for 0.25 Hz 34
True Pred Diﬀ
Re
Im
Phase

True Pred Diﬀ
Re
Im
Phase

Initial model 37

Conventional regularized FWI 38

39
FWI

40
What can go wrong with inversion?

41
FWI
Frequency
Extrapolation

42
FWI
Frequency
Extrapolation
Normalization
Training data
Training
Assumption
Implementation

43
FWI
Frequency
Extrapolation
Normalization
Training data
Training Misﬁt
Solver
Initial guess
Formulation
Implementation
Assumption
Implementation

44
FWI
Frequency
Extrapolation
Normalization
Training data
Training Misﬁt
Solver
Initial guess
Formulation
Implementation
Assumption
Implementation
True Pred Diﬀ
Re
Im
Phase

45
FWI
Frequency
Extrapolation
Normalization
Training data
Training Misﬁt
Solver
Initial guess
Formulation
Implementation
Assumption
Implementation
True Pred Diﬀ
Re
Im
0.25 Hz

46
FWI
Frequency
Extrapolation
Normalization
Training data
Training Misﬁt
Solver
Initial guess
Formulation
Implementation
Assumption
Implementation
True Pred Diﬀ
Re
Im
0.25 Hz4.5 Hz

47
It is a rocky road
Training
Misﬁt
Solver

48
FWI
True vs Extrapolated

0.25 Hz 49
0.25Hz
0 5 10 15 20
km
0
2
4
6
km
2
3
4
km/s
0.25Hz
0 5 10 15 20
km
0
2
4
6
km
2
3
4
km/s
True Extrapolated
0.25 0.55 0.93 2.04 2.66 3.46 4.50 Hz

0.55 Hz 50
0.55Hz
0 5 10 15 20
km
0
2
4
6
km
2
3
4
km/s
0.55Hz
0 5 10 15 20
km
0
2
4
6
km
2
3
4
km/s
True Extrapolated
0.25 0.55 0.93 2.04 2.66 3.46 4.50 Hz

0.93 Hz 51
0.93Hz
0 5 10 15 20
km
0
2
4
6
km
2
3
4
km/s
0 5 10 15 20
km
0
2
4
6
km
2
3
4
km/s
True Extrapolated
0.25 0.55 0.93 2.04 2.66 3.46 4.50 Hz

4.5 Hz 52
0 5 10 15 20
km
0
2
4
6
km
2
3
4
km/s
0 5 10 15 20
km
0
2
4
6
km
2
3
4
km/s
True Extrapolated
0.25 0.55 0.93 2.04 2.66 3.46 4.50 Hz

53
Do we overﬁt the data?

54Left part of BP 2004
True Pred Diﬀ
Re
Im
Phase

55Marmousi II
True Pred Diﬀ
Re
Im
Phase
True Pred Diﬀ
Re
Im
Phase

56
Example 2/2. One shot from ﬁeld

Streamer acquisition 57

Single shot 58

Time domain 59

Frequency domain 60

Low-frequency part of spectrum 61

Used range 4 - 10 Hz 62

Regular sampling of the range 63

Raw input data for extrapolation 64

65Normalized input data for extrapolation

Train one net for each frequency 66
13 frequencies to extrapolate for:

0.1000 0.1334 0.1778 0.2371 0.3162 0.4217 0.5623 0.7499 1.0000 1.3335 1.7783 2.3714 3.1623 Hz

Extrapolated low frequency 67
Synthetic Field
A reference sample of low-
frequency data for a random
velocity model

Synthetic Field
Extrapolated low-frequency data
for a real-world shot

Synthetic Field
Extrapolated low-frequency data
for a real-world shot
Expected patterns
in low-frequencies

70
Trained on synthetic, tested on synthetic = OK

71
Trained on synthetic, tested on ﬁeld = ???

Conclusions 72
Wavenumber analysis justiﬁes feasibility of bandwidth extrapolation
Pre-trained networks are applicable with mild retraining
Need a priori constraints on random velocity models
Lowest frequencies are better extrapolated
Regularized full-waveform inversion to tolerate inaccuracies

Acknowledgements 73
Mahesh Kalita (KAUST/ION)

Xiangliang Zhang (KAUST)

Gerhard Pratt (UWO)

Tristan van Leeuven (Utrecht University)

Yunyue Elita Li (NUS)
ovcharenkoo.com ai.vkazei.com

References 74
Alkhalifah, T., 2015. Conditioning the full-waveform inversion gradient to welcome anisotropy. Geophysics, 80(3), pp.R111-R122.
Bozdağ, E., Trampert, J. and Tromp, J., 2011. Misfit functions for full waveform inversion based on instantaneous phase and envelope
measurements. Geophysical Journal International, 185(2), pp.845-870.
Choi, Y. and Alkhalifah, T., 2013. Frequency-domain waveform inversion using the phase derivative. Geophysical Journal International, 195(3), pp.
1904-1916.
Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M. and Adam, H., 2017. Mobilenets: Efficient convolutional
neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861.
Hu*, W., 2014. FWI without low frequency data-beat tone inversion. In SEG Technical Program Expanded Abstracts 2014 (pp. 1116-1120). Society
of Exploration Geophysicists.
Kazei, V., Tessmer, E. and Alkhalifah, T., 2016. Scattering angle-based filtering via extension in velocity. In SEG Technical Program Expanded
Abstracts 2016 (pp. 1157-1162). Society of Exploration Geophysicists.
Kazei, V. and Alkhalifah, T., 2019. Scattering Radiation Pattern Atlas: What anisotropic elastic properties can body waves resolve?. Journal of
Geophysical Research: Solid Earth.
Li, Y.E. and Demanet, L., 2015. Phase and amplitude tracking for seismic event separation. Geophysics, 80(6), pp.WD59-WD72.
Li, Y.E. and Demanet, L., 2016. Full-waveform inversion with extrapolated low-frequency data. Geophysics, 81(6), pp.R339-R348.
Kalita, M., Kazei, V., Choi, Y. and Alkhalifah, T., 2018. Regularized full-waveform inversion for salt bodies. In SEG Technical Program Expanded
Abstracts 2018 (pp. 1043-1047). Society of Exploration Geophysicists.
Ovcharenko, O., Kazei, V., Peter, D. and Alkalifah, T., 2017. Neural network based low-frequency data extrapolation. In 3rd SEG FWI workshop:
What are we getting.
Ovcharenko, O., Kazei, V., Peter, D., Zhang, X. and Alkhalifah, T., 2018, June. Low-Frequency Data Extrapolation Using a Feed-Forward ANN.
In 80th EAGE Conference and Exhibition 2018.
Ovcharenko, O., Kazei, V., Peter, D. and Alkhalifah, T., 2018. Variance-based model interpolation for improved full-waveform inversion in the
presence of salt bodies. Geophysics, 83(5), pp.R541-R551.
Sun, H. and Demanet, L., 2018. Low frequency extrapolation with deep learning. In SEG Technical Program Expanded Abstracts 2018 (pp.
2011-2015). Society of Exploration Geophysicists.
van Leeuwen, T. and Herrmann, F.J., 2014. 3D frequency-domain seismic inversion with controlled sloppiness. SIAM Journal on Scientific
Computing, 36(5), pp.S192-S217.

Conclusions 75
Wavenumber analysis justiﬁes feasibility of bandwidth extrapolation
Pre-trained networks are applicable with mild retraining
Need a priori constraints on random velocity models
Lowest frequencies are better extrapolated
Regularized full-waveform inversion to tolerate inaccuracies

Transfer learning for low frequency extrapolation from shot gathers for FWI applications

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (19)

Semelhante a Transfer learning for low frequency extrapolation from shot gathers for FWI applications

Semelhante a Transfer learning for low frequency extrapolation from shot gathers for FWI applications (20)

Último

Último (20)

Transfer learning for low frequency extrapolation from shot gathers for FWI applications