das verfahren von cholesky ein geod¨atischer beitrag zur ... filedas verfahren von cholesky ......
Post on 28-Aug-2019
221 Views
Preview:
TRANSCRIPT
Schuh GW2014, Berlin, 7. Okt. 2014 – 1
Das Verfahren von Cholesky—
ein geodatischer Beitrag zur numerischen Mathematik
Wolf-Dieter Schuh
Institut fur Geodasie und Geoinformation
Universitat Bonn
Geodatische Woche 2014Session 6: Theoretische Geodasie
Berlin, 7.-9. Oktober 2014
Contents
Choleskyfactorization
Choleskyapplications
Example: GOCEprocessing
Resume
Schuh GW2014, Berlin, 7. Okt. 2014 – 2
Cholesky factorization
Cholesky applications
Example: GOCE processing
Resume
Andre-Louis Cholesky
Choleskyfactorization
A.-L. Cholesky
Cholesky approach
Choleskyfactorization
Cholesky inversion
Choleskyapplications
Example: GOCEprocessing
Resume
Schuh GW2014, Berlin, 7. Okt. 2014 – 3
Andre-Louis Cholesky(1875 - 1918)
Commandant Benoit (1924):
NOTE SUR UNE METHODE DE RESOLUTION DESEQUATIONS NORMALES PROVENANT DE L’APPLICATIONDE LA METHODE DES MOINDRES CARRES A UN SYSTEMED’EQUATIONS LINEAIRES EN NOMBRE INFERIEUR A CELUIDES INCONNUES. — APPLICATION DE LA METHODE ALA RESOLUTION D’UN SYSTEME DEFINI D’EQUATIONSLINEAIRES.Bulletin geodesique, 1924, 2, 67-77
Andre-Louis Cholesky
Choleskyfactorization
A.-L. Cholesky
Cholesky approach
Choleskyfactorization
Cholesky inversion
Choleskyapplications
Example: GOCEprocessing
Resume
Schuh GW2014, Berlin, 7. Okt. 2014 – 4
Brezinski (2006?)
unpublished manuscript:
Cholesky, A. (2. Dec. 1910):
Sur la resolution numerique des
systemes d’ equations lineaires
(On the numerical solution of
systems of linear equations)
Andre-Louis Cholesky
Choleskyfactorization
A.-L. Cholesky
Cholesky approach
Choleskyfactorization
Cholesky inversion
Choleskyapplications
Example: GOCEprocessing
Resume
Schuh GW2014, Berlin, 7. Okt. 2014 – 4
Brezinski (2006?)
unpublished manuscript:
Cholesky, A. (2. Dec. 1910):
Sur la resolution numerique des
systemes d’ equations lineaires
(On the numerical solution of
systems of linear equations)
(Sept. 2014)
Cholesky approach
Choleskyfactorization
A.-L. Cholesky
Cholesky approach
Choleskyfactorization
Cholesky inversion
Choleskyapplications
Example: GOCEprocessing
Resume
Schuh GW2014, Berlin, 7. Okt. 2014 – 5
Nx = n =
RTR x︸︷︷︸z
= n =
RTz = n R x = z
= =
forward substitution backward substitution
⇒ z ⇒ x
Cholesky factorization
Choleskyfactorization
A.-L. Cholesky
Cholesky approach
Choleskyfactorization
Cholesky inversion
Choleskyapplications
Example: GOCEprocessing
Resume
Schuh GW2014, Berlin, 7. Okt. 2014 – 6
N = RT
R
n11 n12 n13
n12 n22 n23
n13 n23 n33
=
r11r12 r22r13 r23 r33
r11 r12 r13r22 r23
r33
rii =
√√√√ nii −
i−1∑
k=1
r2ki , i=1, . . . ,m
rij =
(
nij −i−1∑
k=1
rkirkj
)
/rii ,i=1, . . . ,mj=i+1, . . . ,m
Cholesky (forward) reduction
Cholesky reduction
Choleskyfactorization
A.-L. Cholesky
Cholesky approach
Choleskyfactorization
Cholesky inversion
Choleskyapplications
Example: GOCEprocessing
Resume
Schuh GW2014, Berlin, 7. Okt. 2014 – 7
Equation interpretation/reading:
rii =
√√√√ nii −
i−1∑
k=1
r2ki
Cholesky reduction
Choleskyfactorization
A.-L. Cholesky
Cholesky approach
Choleskyfactorization
Cholesky inversion
Choleskyapplications
Example: GOCEprocessing
Resume
Schuh GW2014, Berlin, 7. Okt. 2014 – 7
Equation interpretation/reading:
rii =
√√√√ nii −
i−1∑
k=1
r2ki
i
i i
i
N R
=
√√√√ −
⟨
,
⟩
Cholesky reduction
Choleskyfactorization
A.-L. Cholesky
Cholesky approach
Choleskyfactorization
Cholesky inversion
Choleskyapplications
Example: GOCEprocessing
Resume
Schuh GW2014, Berlin, 7. Okt. 2014 – 8
Equation interpretation/reading:
rij =
(
nij −i−1∑
k=1
rkirkj
)
/rii
i
j i
i
N R
j
=
(
−⟨
,
⟩)
/
Cholesky inversion
Choleskyfactorization
A.-L. Cholesky
Cholesky approach
Choleskyfactorization
Cholesky inversion
Choleskyapplications
Example: GOCEprocessing
Resume
Schuh GW2014, Berlin, 7. Okt. 2014 – 9
N ⇒ R ⇒ N−1
r11 r12 r13r22 r23
r33
⇒
n(−1)11 n
(−1)12 n
(−1)13
n(−1)12 n
(−1)22 n
(−1)23
n(−1)13 n
(−1)23 n
(−1)33
n(−1)ii =
1
r2ii− 1
rii
m∑
k=i+1
rikn(−1)ik , i=1, . . . ,m
n(−1)ij = −
m∑
k=i+1
rikn(−1)kj ,
i=1, . . . ,mj=i+1, . . . ,m
Cholesky inversion by backward substitution
Cholesky applications
Choleskyfactorization
Choleskyapplications
Example: GOCEprocessing
Resume
Schuh GW2014, Berlin, 7. Okt. 2014 – 10
Cholesky application
Choleskyfactorization
Choleskyapplications
Example: GOCEprocessing
Resume
Schuh GW2014, Berlin, 7. Okt. 2014 – 11
Cholesky factorization
asymmetric
sparse
supernodal
symmetric
regular
Toeplitz cyclic
positive definite
rectangular
singular
infinite
dense
illposed
1D 2D
block
Cholesky application
Choleskyfactorization
Choleskyapplications
Example: GOCEprocessing
Resume
Schuh GW2014, Berlin, 7. Okt. 2014 – 11
Cholesky factorization
asymmetric
sparse
supernodal
symmetric
regular
Toeplitz cyclic
positive definite
rectangular
singular
infinite
dense
illposed
1D 2D
block
Choleskysolver
Cholesky application
Choleskyfactorization
Choleskyapplications
Example: GOCEprocessing
Resume
Schuh GW2014, Berlin, 7. Okt. 2014 – 11
Cholesky factorization
asymmetric
sparse
supernodal
symmetric
regular
Toeplitz cyclic
positive definite
rectangular
singular
infinite
dense
illposed
1D 2D
block
CholeskysolverCholeskyinversion
Cholesky application
Choleskyfactorization
Choleskyapplications
Example: GOCEprocessing
Resume
Schuh GW2014, Berlin, 7. Okt. 2014 – 11
Cholesky factorization
asymmetric
sparse
supernodal
symmetric
regular
Toeplitz cyclic
positive definite
rectangular
singular
infinite
dense
illposed
1D 2D
block
Choleskyparallelsolver
Cholesky application
Choleskyfactorization
Choleskyapplications
Example: GOCEprocessing
Resume
Schuh GW2014, Berlin, 7. Okt. 2014 – 11
Cholesky factorization
asymmetric
sparse
supernodal
symmetric
regular
Toeplitz cyclic
positive definite
rectangular
singular
infinite
dense
illposed
1D 2D
block
Choleskysparsesolver
Cholesky application
Choleskyfactorization
Choleskyapplications
Example: GOCEprocessing
Resume
Schuh GW2014, Berlin, 7. Okt. 2014 – 11
Cholesky factorization
asymmetric
sparse
supernodal
symmetric
regular
Toeplitz cyclic
positive definite
rectangular
singular
infinite
dense
illposed
1D 2D
block
Choleskysparsesolver
Choleskypartialinverse
Cholesky application
Choleskyfactorization
Choleskyapplications
Example: GOCEprocessing
Resume
Schuh GW2014, Berlin, 7. Okt. 2014 – 11
Cholesky factorization
asymmetric
sparse
supernodal
symmetric
regular
Toeplitz cyclic
positive definite
rectangular
singular
infinite
dense
illposed
1D 2D
block
Choleskycomplexsolver
Cholesky application
Choleskyfactorization
Choleskyapplications
Example: GOCEprocessing
Resume
Schuh GW2014, Berlin, 7. Okt. 2014 – 11
Cholesky factorization
asymmetric
sparse
supernodal
symmetric
regular
Toeplitz cyclic
positive definite
rectangular
singular
infinite
dense
illposed
1D 2D
block
partialCholeskysolver
Cholesky application
Choleskyfactorization
Choleskyapplications
Example: GOCEprocessing
Resume
Schuh GW2014, Berlin, 7. Okt. 2014 – 11
Cholesky factorization
asymmetric
sparse
supernodal
symmetric
regular
Toeplitz cyclic
positive definite
rectangular
singular
infinite
dense
illposed
1D 2D
block
partialCholeskysolver
CholeskyMoore-Penrose
inverse
Cholesky application
Choleskyfactorization
Choleskyapplications
Example: GOCEprocessing
Resume
Schuh GW2014, Berlin, 7. Okt. 2014 – 11
Cholesky factorization
asymmetric
sparse
supernodal
symmetric
regular
Toeplitz cyclic
positive definite
rectangular
singular
infinite
dense
illposed
1D 2D
block
⇒decorrelation
Example: GOCE processing
Choleskyfactorization
Choleskyapplications
Example: GOCEprocessing
GOCE-TIM
GMM decorrelation
Cholesky revisited
Toeplitz-Cholesky
Adaptive filter
Resume
Schuh GW2014, Berlin, 7. Okt. 2014 – 12
GOCE-TIM gravity field model
Schuh GW2014, Berlin, 7. Okt. 2014 – 13
GOCE-TIM data segments440 mio observations, eff. 1273 days, 88 segments
Special focus on LOOC (13-Jun-2012 — 21-Oct-2013)4 × 40,578,861 observations, eff. 463 days, 48 segments
LOOC-only models have the same accuracy level as the TIM RL4 model
Gauss-Markov model — complete decorrelation
Choleskyfactorization
Choleskyapplications
Example: GOCEprocessing
GOCE-TIM
GMM decorrelation
Cholesky revisited
Toeplitz-Cholesky
Adaptive filter
Resume
Schuh GW2014, Berlin, 7. Okt. 2014 – 14
input: ℓ, Σ ℓ= Hℓ, Σ= I
model: ℓ+ v = Ax ℓ+v=Ax
principle: vTΣ
−1v vTv
(R
−1)T
= H
Σ = RTR ℓ=(R
−1)T
ℓ
A=(R
−1)T
A
Σ=(R
−1)T
ΣR−1 = I
Gauss-Markov model — complete decorrelation
Choleskyfactorization
Choleskyapplications
Example: GOCEprocessing
GOCE-TIM
GMM decorrelation
Cholesky revisited
Toeplitz-Cholesky
Adaptive filter
Resume
Schuh GW2014, Berlin, 7. Okt. 2014 – 15
ℓ = Hℓ
ℓ = (R−1)Tℓ
RTℓ = ℓ
RT ℓt =
1
rtt
(
ℓt −t−1∑
i=1
ritℓi
)
=⇒ linear - time variant - recursive - causal - filter
Gauss-Markov model — complete decorrelation
Choleskyfactorization
Choleskyapplications
Example: GOCEprocessing
GOCE-TIM
GMM decorrelation
Cholesky revisited
Toeplitz-Cholesky
Adaptive filter
Resume
Schuh GW2014, Berlin, 7. Okt. 2014 – 16
pros and cons: recursive decorrelation filters
+ finite filter — finite time series
+ flexible with respect to data distribution (data gaps)
+ utilization of sparse structures — finite covariance functions
– recursive procedures are time-consuming
– time variant filter - attended memory O{n2}– computational complexity O{n3} 1
3(n3 + 8n)
– parallel implementation is very elaborately
– scalability O{√cores}
Decorrelation — Σ =⇒ H
Schuh GW2014, Berlin, 7. Okt. 2014 – 17
crucial issue: numerical implementation of the factorization Σ =⇒ H
Decomposition of Computation of R orR Decorrelationmatrix
Decorrelation of ℓ
Σ = RTR Forward Cholesky reduction H = (RT )−1 L= (RT )−1L ⇐⇒ R
TL=L
∗
∗ RT
∗ ∗ ∗
L
=
L
Causal recursive filter
Σ =RRT
Backward Cholesky reduc-tion
H =R−1
L=R−1
L ⇐⇒RL= L
∗ ∗ ∗
R ∗
∗
L
=
L
Anti-causal recursive filter
Σ = (R−1)TR−1 Recursive backward edging H = R L= RL
L
=
∗ ∗ ∗
R ∗
∗
L
Anti-causal non-recursive filter
Σ =R−1
(R−1
)T Recursive forward edging H =RT
L=RT
L
L
=
∗
∗ RT
∗ ∗ ∗
L
Causal non-recursive filter
Toeplitz systems — Cholesky reduction
Schuh GW2014, Berlin, 7. Okt. 2014 – 18
Focus: Causal filter via Cholesky reduction of Toeplitz structured covariancematrices resulting from AR(p) process:
St = α1St−1 + α2St−2 + . . .+ αpSt−p + Et; E ∼ WN(0, Iσ2E)
Toeplitz systems — Cholesky reduction
Schuh GW2014, Berlin, 7. Okt. 2014 – 18
Focus: Causal filter via Cholesky reduction of Toeplitz structured covariancematrices resulting from AR(p) process:
St = α1St−1 + α2St−2 + . . .+ αpSt−p + Et; E ∼ WN(0, Iσ2E)
Yule-Walker equation
γ0 γ1 . . . γp . . . γnγ1 γ0 . . . γp−1 . . . γn−1
......
. . .. . .
...γp γp−1 . . . γ0 . . . γp−1
......
. . .. . .
...γnγn−1 . . . γp−1 . . . γ0
1−α1
...−αp
0...0
=
σ2E
0...00...0
Toeplitz systems — Cholesky reduction
Schuh GW2014, Berlin, 7. Okt. 2014 – 18
Focus: Causal filter via Cholesky reduction of Toeplitz structured covariancematrices resulting from AR(p) process:
St = α1St−1 + α2St−2 + . . .+ αpSt−p + Et; E ∼ WN(0, Iσ2E)
Yule-Walker equation
γ0 γ1 . . . γp . . . γnγ1 γ0 . . . γp−1 . . . γn−1
......
. . .. . .
...γp γp−1 . . . γ0 . . . γp−1
......
. . .. . .
...γnγn−1 . . . γp−1 . . . γ0
1−α1
...−αp
0...0
=
σ2E
0...00...0
Causal non-recursive filter(written in reversed order)
γ0 γ1 . . . γjγ1 γ0 . . . γj−1
......
. . ....
γj γj−1 . . . γ0
r(−1)jj r
(−1)jj
r(−1)j−1,jr
(−1)jj
...
r(−1)1j r
(−1)jj
=
10...0
Toeplitz systems — Cholesky reduction
Schuh GW2014, Berlin, 7. Okt. 2014 – 18
Focus: Causal filter via Cholesky reduction of Toeplitz structured covariancematrices resulting from AR(p) process:
St = α1St−1 + α2St−2 + . . .+ αpSt−p + Et; E ∼ WN(0, Iσ2E)
Yule-Walker equation
γ0 γ1 . . . γp . . . γnγ1 γ0 . . . γp−1 . . . γn−1
......
. . .. . .
...γp γp−1 . . . γ0 . . . γp−1
......
. . .. . .
...γnγn−1 . . . γp−1 . . . γ0
1−α1
...−αp
0...0
=
σ2E
0...00...0
Causal non-recursive filter(written in reversed order)
γ0 γ1 . . . γjγ1 γ0 . . . γj−1
......
. . ....
γj γj−1 . . . γ0
r(−1)jj r
(−1)jj
r(−1)j−1,jr
(−1)jj
...
r(−1)1j r
(−1)jj
=
10...0
Resume: Efficient computation of the filter coefficients
j < p : r(−1)ij , i=1, . . . , j derived from recursive forward edging (left box)
j ≥ p : r(−1)jj = 1
σE; r
(−1)j−k,j = −αk
σE, k=1, . . . , p fixed by coefficient comparison
Toeplitz systems — Cholesky reduction
Schuh GW2014, Berlin, 7. Okt. 2014 – 19
Causal non-recursive filters: Rigorous warmup strategy for finite time series
sparse filter withalmost Toeplitz
structure
warmup:computed by Choleskyrecursive forward edging
main part:results direct from theAR coefficients
Adaptive whitening filter
Schuh GW2014, Berlin, 7. Okt. 2014 – 20
Adaptive causal non-recursive whitening filter: warmup and single gap
sparse filter withalmost Toeplitz
structure
main part:results direct from theAR coefficients
warmup:computed by Choleskyrecursive forward edging
data gaps:computed by Choleskyrecursive forward edging
Adaptive whitening filter
Schuh GW2014, Berlin, 7. Okt. 2014 – 21
Adaptive causal non-recursive whitening filter: multiple gaps
20 40 60 80 100 120
20
40
60
80
100
120
sparse filter withalmost Toeplitz
structure
characteristics:#data points : 120#model : AR(7)-process#gaps : 21
42 4559 65 68100108
Resume
Choleskyfactorization
Choleskyapplications
Example: GOCEprocessing
Resume
Schuh GW2014, Berlin, 7. Okt. 2014 – 22
Resume
Choleskyfactorization
Choleskyapplications
Example: GOCEprocessing
Resume
Schuh GW2014, Berlin, 7. Okt. 2014 – 23
Cholesky factorization
asymmetric
sparse
supernodal
symmetric
regular
Toeplitz cyclic
positive definite
rectangular
singular
infinite
dense
illposed
1D 2D
block
Resume
Choleskyfactorization
Choleskyapplications
Example: GOCEprocessing
Resume
Schuh GW2014, Berlin, 7. Okt. 2014 – 23
Cholesky factorization
asymmetric
sparse
supernodal
symmetric
regular
Toeplitz cyclic
positive definite
rectangular
singular
infinite
dense
illposed
1D 2D
block
Thank youfor your
attention !
top related