Polyharmonic spline

From testwiki
Revision as of 20:23, 20 September 2024 by imported>OlliverWithDoubleL (short description)
(diff) ← Older revision | Latest revision (diff) | Newer revision β†’ (diff)
Jump to navigation Jump to search

Template:Short description

In applied mathematics, polyharmonic splines are used for function approximation and data interpolation. They are very useful for interpolating and fitting scattered data in many dimensions. Special cases include thin plate splines[1][2] and natural cubic splines in one dimension.[3]

Definition

A polyharmonic spline is a linear combination of polyharmonic radial basis functions (RBFs) denoted by φ plus a polynomial term: Template:NumBlk

where

Polyharmonic basis functions
  • 𝐱=[x1 x2  xd]T (T denotes matrix transpose, meaning 𝐱 is a column vector) is a real-valued vector of d independent variables,
  • 𝐜i=[ci,1 ci,2  ci,d]T are N vectors of the same size as 𝐱 (often called centers) that the curve or surface must interpolate,
  • 𝐰=[w1 w2  wN]T are the N weights of the RBFs,
  • 𝐯=[v1 v2  vd+1]T are the d+1 weights of the polynomial.

The polynomial with the coefficients 𝐯 improves fitting accuracy for polyharmonic smoothing splines and also improves extrapolation away from the centers 𝐜i. See figure below for comparison of splines with polynomial term and without polynomial term.

The polyharmonic RBFs are of the form:

φ(r)={rkwith k=1,3,5,,rkln(r)with k=2,4,6,r=|𝐱𝐜i|=(𝐱𝐜i)T(𝐱𝐜i).

Other values of the exponent k are not useful (such as φ(r)=r2), because a solution of the interpolation problem might not exist. To avoid problems at r=0 (since log(0)=), the polyharmonic RBFs with the natural logarithm might be implemented as:

φ(r)={rk1ln(rr)for r<1,(this works because 00 is defined)rkln(r)for r1.

or, more simply adding a continuity extension in r=0

φ(r)={0for r<ϵ,(for some very small value of ϵ, e.g. if using floating point numbers in double precisions, ϵ=10200)rkln(r)for rϵ.

The weights wi and vj are determined such that the function interpolates N given points (𝐜i,fi) (for i=1,2,,N) and fulfills the d+1 orthogonality conditions

i=1Nwi=0,i=1Nwi𝐜i=𝟎.

All together, these constraints are equivalent to the symmetric linear system of equations

Template:NumBlk

where

Ai,j=φ(|𝐜i𝐜j|),B=[111𝐜1𝐜2𝐜N]T,𝐟=[f1,f2,,fN]T.

In order for this system of equations to have a unique solution, B must be full rank. B is full rank for very mild conditions on the input data. For example, in two dimensions, three centers forming a non-degenerate triangle ensure that B is full rank, and in three dimensions, four centers forming a non-degenerate tetrahedron ensure that B is full rank. As explained later, the linear transformation resulting from the restriction of the domain of the linear transformation A to the null space of BT is positive definite. This means that if B is full rank, the system of equations (Template:EquationNote) always has a unique solution and it can be solved using a linear solver specialised for symmetric matrices. The computed weights allow evaluation of the spline for any 𝐱ℝd using equation (Template:EquationNote). Many practical details of implementing and using polyharmonic splines are explained in Fasshauer.[4] In Iske[5] polyharmonic splines are treated as special cases of other multiresolution methods in scattered data modelling.

Discussion

The main advantage of polyharmonic spline interpolation is that usually very good interpolation results are obtained for scattered data without performing any "tuning", so automatic interpolation is feasible. This is not the case for other radial basis functions. For example, the Gaussian function ekr2 needs to be tuned, so that k is selected according to the underlying grid of the independent variables. If this grid is non-uniform, a proper selection of k to achieve a good interpolation result is difficult or impossible.

Main disadvantages are:

  • To determine the weights, a dense linear system of equations must be solved. Solving a dense linear system becomes impractical if the dimension N is large, since the memory required is O(N2) and the number of operations required is O(N3).
  • Evaluating the computed polyharmonic spline function at M data points requires O(MN) operations. In many applications (image processing is an example), M is much larger than N, and if both numbers are large, this is not practical.

Fast construction and evaluation methods

One straightforward approach to speeding up model construction and evaluation is to use a subset of k nearest interpolation nodes to build a local model every time we evaluate the spline. As a result, the total time needed for model construction and evaluation at M points changes from O(N3+MN) to O(k3*M). This can yield better timings if k is much less than N. Such an approach is advocated by some software libraries, the most notable being scipy.interpolate.RBFInterpolator. The main drawback is that it introduces small discontinuities in the spline and requires problem-specific tuning: a proper choice of the neighbors count, k. Recently, methods have been developed to overcome the aforementioned difficulties without sacrificing main advantages of polyharmonic splines.

First, a bunch of methods for fast O(logN) evaluation were proposed:

  • Beatson et al.[6] present a method to interpolate polyharmonic splines with r2k1 being a basis function at one point in 3 dimensions or less
  • Cherrie et al. [7] present a method to interpolate polyharmonic splines with r2klogr as a basis function at one point in 4 dimensions or less

Second, an accelerated model construction by applying an iterative solver to an ACBF-preconditioned linear system was proposed by Brown et al.[8] This approach reduces running time from O(N3) to O(N2), and further to O(NlogN) when combined with accelerated evaluation techniques.

The approaches above are often employed by commercial geospatial data analysis libraries and by some open source implementations (e.g. ALGLIB). Sometimes domain decomposition methods are used to improve asymptotic behavior, reducing memory requirements from O(N2) to O(N), thus making polyharmonic splines suitable for datasets with more than 1.000.000 points.

Reason for the name "polyharmonic"

A polyharmonic equation is a partial differential equation of the form Δmf=0 for any natural number m, where Δ is the Laplace operator. For example, the biharmonic equation is Δ2f=0 and the triharmonic equation is Δ3f=0. All the polyharmonic radial basis functions are solutions of a polyharmonic equation (or more accurately, a modified polyharmonic equation with a Dirac delta function on the right hand side instead of 0). For example, the thin plate radial basis function is a solution of the modified 2-dimensional biharmonic equation.[9] Applying the 2D Laplace operator (Δ=xx+yy) to the thin plate radial basis function ftp(x,y)=(x2+y2)logx2+y2 either by hand or using a computer algebra system shows that Δftp=4+4logr. Applying the Laplace operator to Δftp (this is Δ2ftp) yields 0. But 0 is not exactly correct. To see this, replace r2=x2+y2 with ρ2=x2+y2+h2 (where h is some small number tending to 0). The Laplace operator applied to 4logρ yields Δ2ftp=8h2/ρ4. For (x,y)=(0,0), the right hand side of this equation approaches infinity as h approaches 0. For any other (x,y), the right hand side approaches 0 as h approaches 0. This indicates that the right hand side is a Dirac delta function. A computer algebra system will show that

limh08h2/(x2+y2+h2)2dxdy=8π.

So the thin plate radial basis function is a solution of the equation Δ2ftp=8πδ(x,y).

Applying the 3D Laplacian (Δ=xx+yy+zz) to the biharmonic RBF fbi(x,y,z)=x2+y2+z2 yields Δfbi=2/r and applying the 3D Δ2 operator to the triharmonic RBF ftri(x,y,z)=(x2+y2+z2)3/2 yields Δ2ftri=24/r. Letting ρ2=x2+y2+z2+h2 and computing Δ(1/ρ)=3h2/ρ5 again indicates that the right hand side of the PDEs for the biharmonic and triharmonic RBFs are Dirac delta functions. Since

limh03h2/(x2+y2+z2+h2)5/2dxdydz=4π,

the exact PDEs satisfied by the biharmonic and triharmonic RBFs are Δ2fbi=8πδ(x,y,z) and Δ3ftri=96πδ(x,y,z).

Polyharmonic smoothing splines

Polyharmonic splines minimize

Template:NumBlk

where ℬ is some box in ℝd containing a neighborhood of all the centers, λ is some positive constant, and mf is the vector of all mth order partial derivatives of f. For example, in 2D 1f=(fx fy) and 2f=(fxx fxy fyx fyy) and in 3D 2f=(fxx fxy fxz fyx fyy fyz fzx fzy fzz). In 2D |2f|2=fxx2+2fxy2+fyy2, making the integral the simplified thin plate energy functional.

To show that polyharmonic splines minimize equation (Template:EquationNote), the fitting term must be transformed into an integral using the definition of the Dirac delta function:

i=1N(f(𝐜i)fi)2=ℬi=1N(f(𝐱)fi)2δ(𝐱𝐜i)d𝐱.

So equation (Template:EquationNote) can be written as the functional

J[f]=ℬF(𝐱,f,α1f,α2f,,αnf)d𝐱=ℬ[i=1N(f(𝐱)fi)2δ(𝐱𝐜i)+λ|mf|2]d𝐱.

where αi is a multi-index that ranges over all partial derivatives of order m for ℝd. In order to apply the Euler–Lagrange equation for a single function of multiple variables and higher order derivatives, the quantities

Ff=2i=1N(f(𝐱)fi)δ(𝐱xi)

and

i=1nαiF(αif)=2λΔmf

are needed. Inserting these quantities into the Eβˆ’L equation shows that

Template:NumBlk

A weak solution f(𝐱) of (Template:EquationNote) satisfies

Template:NumBlk

for all smooth test functions g that vanish outside of ℬ. A weak solution of equation (Template:EquationNote) will still minimize (Template:EquationNote) while getting rid of the delta function through integration.[10]

Let f be a polyharmonic spline as defined by equation (Template:EquationNote). The following calculations will show that f satisfies (Template:EquationNote). Applying the Δm operator to equation (Template:EquationNote) yields

Δmf=i=1MwiCm,dδ(𝐱𝐜i)

where C2,2=8π, C2,3=8π, and C3,3=96π. So (Template:EquationNote) is equivalent to

Template:NumBlk

The only possible solution to (Template:EquationNote) for all test functions g is

Template:NumBlk

(which implies interpolation if λ=0). Combining the definition of f in equation (Template:EquationNote) with equation (Template:EquationNote) results in almost the same linear system as equation (Template:EquationNote) except that the matrix A is replaced with A+(1)mCm,dλI where I is the N×N identity matrix. For example, for the 3D triharmonic RBFs, A is replaced with A+96πλI.

Explanation of additional constraints

In (Template:EquationNote), the bottom half of the system of equations (BT𝐰=0) is given without explanation. The explanation first requires deriving a simplified form of ℬ|mf|2d𝐱 when ℬ is all of ℝd.

First, require that i=1Nwi=0. This ensures that all derivatives of order m and higher of f(𝐱)=i=1Nwiφ(|𝐱𝐜i|) vanish at infinity. For example, let m=3 and d=3 and φ be the triharmonic RBF. Then φzzy=3y(x2+y2)/(x2+y2+z2)3/2 (considering φ as a mapping from ℝ3 to ℝ). For a given center 𝐏=(P1,P2,P3),

φzzy(𝐱𝐏)=3(yP2)((yP2)2+(xP1)2)((xP1)2+(yP2)2+(zP3)2)3/2.

On a line 𝐱=𝐚+t𝐛 for arbitrary point 𝐚 and unit vector 𝐛,

φzzy(𝐱𝐏)=3(a2+b2tP2)((a2+b2tP2)2+(a1+b1tP1)2)((a1+b1tP1)2+(a2+b2tP2)2+(a3+b3tP3)2)3/2.

Dividing both numerator and denominator of this by t3 shows that limtφzyy(𝐱𝐏)=3b2(b22+b12)/(b12+b22+b32)3/2, a quantity independent of the center 𝐏. So on the given line,

limtfzyy(𝐱)=limti=1Nwiφzyy(𝐱𝐜i)=(i=1Nwi)3b2(b22+b12)/(b12+b22+b32)3/2=0.

It is not quite enough to require that i=1Nwi=0, because in what follows it is necessary for fαgβ to vanish at infinity, where α and β are multi-indices such that |α|+|β|=2m1. For triharmonic φ, wiujφα(𝐱𝐜i)φβ(𝐱𝐝j) (where uj and 𝐝j are the weights and centers of g) is always a sum of total degree 5 polynomials in x, y, and z divided by the square root of a total degree 8 polynomial. Consider the behavior of these terms on the line 𝐱=𝐚+t𝐛 as t approaches infinity. The numerator is a degree 5 polynomial in t. Dividing numerator and denominator by t4 leaves the degree 4 and 5 terms in the numerator and a function of 𝐛 only in the denominator. A degree 5 term divided by t4 is a product of five b coordinates and t. The w=0 (and u=0) constraint makes this vanish everywhere on the line. A degree 4 term divided by t4 is either a product of four b coordinates and an a coordinate or a product of four b coordinates and a single ci or dj coordinate. The w=0 constraint makes the first type of term vanish everywhere on the line. The additional constraints i=1Nwi𝐜i=0 will make the second type of term vanish.

Now define the inner product of two functions f,g:ℝdℝ defined as a linear combination of polyharmonic RBFs φm,d with w=0 and w𝐜=0 as

f,g=ℝd(mf)(mg)d𝐱.

Integration by parts shows that

Template:NumBlk

For example, let m=2 and d=2. Then

Template:NumBlk

Integrating the first term of this by parts once yields

fxxgxxdxdy=fxgxx|dyfxgxxxdxdy=fxgxxxdxdy

since fxgxx vanishes at infinity. Integrating by parts again results in fgxxxxdxdy.

So integrating by parts twice for each term of (Template:EquationNote) yields

f,g=f(gxxxx+2gxxyy+gyyyy)dxdy=f(Δ2g)dxdy.

Since (Δmf)(𝐱)=i=1NwiCm,dδ(𝐱𝐜𝐒), (Template:EquationNote) shows that

f,f=(1)mℝdf(𝐱)i=1Nwi(1)mCm,dδ(𝐱𝐜𝐒)d𝐱=(1)mCm,di=1Nwif(𝐜i)=(1)mCm,di=1Nj=1Nwiwjφ(𝐜i𝐜j)=(1)mCm,d𝐰TA𝐰.

So if w=0 and w𝐜=0,

Template:NumBlk

Now the origin of the constraints BT𝐰=0 can be explained. Here B is a generalization of the B defined above to possibly include monomials up to degree m1. In other words, B=[111𝐜1𝐜2𝐜N𝐜1m1𝐜2m1𝐜Nm1]T where 𝐜ij is a column vector of all degree j monomials of the coordinates of 𝐜i. The top half of (Template:EquationNote) is equivalent to A𝐰+B𝐯𝐟=0. So to obtain a smoothing spline, one should minimize the scalar field F:ℝN+d+1ℝ defined by

F(𝐰,𝐯)=|A𝐰+B𝐯𝐟|2+λC𝐰TA𝐰.

The equations

Fwi=2Ai*(A𝐰+B𝐯𝐟)+2λCAi*𝐰=0for i=1,2,,N

and

Fvi=2Bi*T(A𝐰+B𝐯𝐟)=0for i=1,2,,d+1

(where Ai* denotes row i of A) are equivalent to the two systems of linear equations A(A𝐰+B𝐯𝐟+λC𝐰)=0 and BT(A𝐰+B𝐯𝐟)=0. Since A is invertible, the first system is equivalent to A𝐰+B𝐯𝐟+λC𝐰=0. So the first system implies the second system is equivalent to BT𝐰=0. Just as in the previous smoothing spline coefficient derivation, the top half of (Template:EquationNote) becomes (A+λCI)𝐰+B𝐯=𝐟.

This derivation of the polyharmonic smoothing spline equation system did not assume the constraints necessary to guarantee that ℝd|mf|2d𝐱=CwTAw. But the constraints necessary to guarantee this, w=0 and w𝐜=0, are a subset of BTw=0 which is true for the critical point w of F. So ℝd|mf|2d𝐱=CwTAw is true for the f formed from the solution of the polyharmonic smoothing spline equation system. Because the integral is positive for all w0, the linear transformation resulting from the restriction of the domain of linear transformation A to w such that BTw=0 must be positive definite. This fact enables transforming the polyharmonic smoothing spline equation system to a symmetric positive definite system of equations that can be solved twice as fast using the Cholesky decomposition.[9]

Examples

The next figure shows the interpolation through four points (marked by "circles") using different types of polyharmonic splines. The "curvature" of the interpolated curves grows with the order of the spline and the extrapolation at the left boundary (x < 0) is reasonable. The figure also includes the radial basis functions φ = exp(βˆ’r2) which gives a good interpolation as well. Finally, the figure includes also the non-polyharmonic spline phi = r2 to demonstrate, that this radial basis function is not able to pass through the predefined points (the linear equation has no solution and is solved in a least squares sense).

Interpolation with different polyharmonic splines that shall pass the 4 predefined points marked by a circle (the interpolation with phi = r2 is not useful, since the linear equation system of the interpolation problem has no solution; it is solved in a least squares sense, but then does not pass the centers)

The next figure shows the same interpolation as in the first figure, with the only exception that the points to be interpolated are scaled by a factor of 100 (and the case phi = r2 is no longer included). Since φ = (scaleΒ·r)k = (scalek)Β·rk, the factor (scalek) can be extracted from matrix A of the linear equation system and therefore the solution is not influenced by the scaling. This is different for the logarithmic form of the spline, although the scaling has not much influence. This analysis is reflected in the figure, where the interpolation shows not much differences. Note, for other radial basis functions, such as φ = exp(βˆ’kr2) with k = 1, the interpolation is no longer reasonable and it would be necessary to adapt k.

The same interpolation as in the first figure, but the points to be interpolated are scaled by 100

The next figure shows the same interpolation as in the first figure, with the only exception that the polynomial term of the function is not taken into account (and the case phi = r2 is no longer included). As can be seen from the figure, the extrapolation for x < 0 is no longer as "natural" as in the first figure for some of the basis functions. This indicates, that the polynomial term is useful if extrapolation occurs.

The same interpolation as in the first figure, but without the polynomial term

See also

References

Template:Reflist

Computer Code

  1. ↑ R.L. Harder and R.N. Desmarais: Interpolation using surface splines. Journal of Aircraft, 1972, Issue 2, pp. 189βˆ’191
  2. ↑ J. Duchon: Splines minimizing rotation-invariant semi-norms in Sobolev spaces. Constructive Theory of Functions of Several Variables, W. Schempp and K. Zeller (eds), Springer, Berlin, pp. 85βˆ’100
  3. ↑ Template:Cite book
  4. ↑ G.F. Fasshauer G.F.: Meshfree Approximation Methods with MATLAB. World Scientific Publishing Company, 2007, ISPN-10: 9812706348
  5. ↑ A. Iske: Multiresolution Methods in Scattered Data Modelling, Lecture Notes in Computational Science and Engineering, 2004, Vol. 37, Template:ISBN, Springer-Verlag, Heidelberg.
  6. ↑ R.K. Beatson, M.J.D. Powell, and A.M. Tan: Fast evaluation of polyharmonic splines in three dimensions. IMA Journal of Numerical Analysis, 2007, 27, pp. 427–450.
  7. ↑ Template:Citation
  8. ↑ Template:Citation
  9. ↑ 9.0 9.1 Template:Cite journal
  10. ↑ Template:Cite book