Comparison of Gaussian process software

Template:Short description This is a comparison of statistical analysis software that allows doing inference with Gaussian processes often using approximations.

This article is written from the point of view of Bayesian statistics, which may use a terminology different from the one commonly used in kriging. The next section should clarify the mathematical/computational meaning of the information provided in the table independently of contextual terminology.

Description of columns

This section details the meaning of the columns in the table below.

Solvers

These columns are about the algorithms used to solve the linear system defined by the prior covariance matrix, i.e., the matrix built by evaluating the kernel.

• Exact: whether generic exact algorithms are implemented. These algorithms are usually appropriate only up to some thousands of datapoints.
• Specialized: whether specialized exact algorithms for specific classes of problems are implemented. Supported specialized algorithms may be indicated as:
  • Kronecker: algorithms for separable kernels on grid data.
  • Toeplitz: algorithms for stationary kernels on uniformly spaced data.
  • Semisep.: algorithms for semiseparable covariance matrices.
  • Sparse: algorithms optimized for sparse covariance matrices.
  • Block: algorithms optimized for block diagonal covariance matrices.
  • Markov: algorithms for kernels which represent (or can be formulated as) a Markov process.
• Approximate: whether generic or specialized approximate algorithms are implemented. Supported approximate algorithms may be indicated as:
  • Sparse: algorithms based on choosing a set of "inducing points" in input space, or more in general imposing a sparse structure on the inverse of the covariance matrix.
  • Hierarchical: algorithms which approximate the covariance matrix with a hierarchical matrix.

Input

These columns are about the points on which the Gaussian process is evaluated, i.e. $x$ if the process is $f(x)$.

• ND: whether multidimensional input is supported. If it is, multidimensional output is always possible by adding a dimension to the input, even without direct support.
• Non-real: whether arbitrary non-real input is supported (for example, text or complex numbers).

Output

These columns are about the values yielded by the process, and how they are connected to the data used in the fit.

• Likelihood: whether arbitrary non-Gaussian likelihoods are supported.
• Errors: whether arbitrary non-uniform correlated errors on datapoints are supported for the Gaussian likelihood. Errors may be handled manually by adding a kernel component, this column is about the possibility of manipulating them separately. Partial error support may be indicated as:
  • iid: the datapoints must be independent and identically distributed.
  • Uncorrelated: the datapoints must be independent, but can have different distributions.
  • Stationary: the datapoints can be correlated, but the covariance matrix must be a Toeplitz matrix, in particular this implies that the variances must be uniform.

Hyperparameters

These columns are about finding values of variables which enter somehow in the definition of the specific problem but that can not be inferred by the Gaussian process fit, for example parameters in the formula of the kernel.

• Prior: whether specifying arbitrary hyperpriors on the hyperparameters is supported.
• Posterior: whether estimating the posterior is supported beyond point estimation, possibly in conjunction with other software.

If both the "Prior" and "Posterior" cells contain "Manually", the software provides an interface for computing the marginal likelihood and its gradient w.r.t. hyperparameters, which can be feed into an optimization/sampling algorithm, e.g., gradient descent or Markov chain Monte Carlo.

Linear transformations

These columns are about the possibility of fitting datapoints simultaneously to a process and to linear transformations of it.

• Deriv.: whether it is possible to take an arbitrary number of derivatives up to the maximum allowed by the smoothness of the kernel, for any differentiable kernel. Example partial specifications may be the maximum derivability or implementation only for some kernels. Integrals can be obtained indirectly from derivatives.
• Finite: whether finite arbitrary ${ℝ}^n\to {ℝ}^m$ linear transformations are allowed on the specified datapoints.
• Sum: whether it is possible to sum various kernels and access separately the processes corresponding to each addend. It is a particular case of finite linear transformation but it is listed separately because it is a common feature.

Comparison table

Name	License	Language	Solvers			Input		Output		Hyperparameters		Linear transformations			Name
			Exact	Specialized	Approximate	ND	Template:Nowrap | Non-real	Likelihood	Errors	Prior	Posterior	Deriv.	Finite	Sum
PyMC	Template:Free	Python	Template:Yes	Template:Yes	Template:Yes	Template:Yes	Template:No	Template:Yes	Template:Yes	Template:Yes	Template:Yes	Template:No	Template:Yes	Template:Yes	PyMC
Stan	Template:Free	custom	Template:Yes	Template:No	Template:No	Template:Yes	Template:No	Template:Yes	Template:Yes	Template:Yes	Template:Yes	Template:No	Template:Yes	Template:Yes	Stan
scikit-learn	Template:BSD-lic	Python	Template:Yes	Template:No	Template:No	Template:Yes	Template:Yes	Template:Partial	Template:Partial	Template:Partial	Template:Partial	Template:No	Template:No	Template:No	scikit-learn
fbm	Template:Free	C	Template:Yes	Template:No	Template:No	Template:Yes	Template:No	Template:Partial	Template:Partial	Template:Partial	Template:Yes	Template:No	Template:No	Template:Yes	fbm
GPML	Template:BSD-lic	MATLAB	Template:Yes	Template:No	Template:Yes	Template:Yes	Template:No	Template:Partial	Template:Partial	Template:Partial	Template:Partial	Template:No	Template:No	Template:No	GPML
GPstuff	Template:GPL-lic	MATLAB, R	Template:Yes	Template:Yes	Template:Yes	Template:Yes	Template:No	Template:Partial	Template:Yes	Template:Partial	Template:Yes	Template:Partial	Template:No	Template:Yes	GPstuff
GPy	Template:BSD-lic	Python	Template:Yes	Template:No	Template:Yes	Template:Yes	Template:No	Template:Partial	Template:Partial	Template:Yes	Template:Yes	Template:No	Template:No	Template:No	GPy
GPflow	Template:Free	Python	Template:Yes	Template:No	Template:Yes	Template:Yes	Template:No	Template:Partial	Template:Partial	Template:Yes	Template:Yes	Template:No	Template:No	Template:No	GPflow
GPyTorch	Template:Free	Python	Template:Yes	Template:Yes	Template:Yes	Template:Yes	Template:No	Template:Partial	Template:Partial	Template:Yes	Template:Yes	Template:Partial	Template:Partial	Template:Partial	GPyTorch
GPvecchia	Template:GPL-lic	R	Template:Yes	Template:No	Template:Yes	Template:Yes	Template:No	Template:Partial	Template:Partial	Template:No	Template:No	Template:No	Template:No	Template:No	GPvecchia
pyGPs	Template:BSD-lic	Python	Template:Yes	Template:No	Template:Yes	Template:Yes	Template:Partial	Template:Partial	Template:Partial	Template:Partial	Template:Partial	Template:No	Template:No	Template:No	pyGPs
gptk	Template:BSD-lic	R	Template:Yes	Template:Unknown	Template:Yes	Template:Yes	Template:No	Template:No	Template:No	Template:Partial	Template:Partial	Template:No	Template:No	Template:No	gptk
celerite	Template:Free	Python, Julia, C++	Template:No	Template:YesTemplate:Efn	Template:No	Template:No	Template:No	Template:No	Template:Partial	Template:Partial	Template:Partial	Template:No	Template:No	Template:No	celerite
george	Template:Free	Python, C++	Template:Yes	Template:No	Template:Yes	Template:Yes	Template:No	Template:No	Template:Partial	Template:Partial	Template:Partial	Template:No	Template:No	Template:Partial	george
neural-tangentsTemplate:Efn	Template:Free	Python	Template:Yes	Template:Yes	Template:No	Template:Yes	Template:No	Template:No	Template:No	Template:No	Template:No	Template:No	Template:No	Template:No	neural-tangents
DiceKriging	Template:GPL-lic	R	Template:Yes	Template:No	Template:No	Template:Yes	Template:Unknown	Template:No	Template:Partial	Template:Partial	Template:Partial	Template:No	Template:No	Template:No	DiceKriging
OpenTURNS	Template:LGPL-lic	Python, C++	Template:Yes	Template:No	Template:No	Template:Yes	Template:No	Template:No	Template:Partial	Template:Partial	Template:Partial	Template:No	Template:No	Template:No	OpenTURNS
UQLab	Template:Proprietary	MATLAB	Template:Yes	Template:No	Template:No	Template:Yes	Template:No	Template:No	Template:Yes	Template:No	Template:Partial	Template:No	Template:No	Template:No	UQLab
ooDACE	Template:Proprietary	MATLAB	Template:Yes	Template:No	Template:No	Template:Yes	Template:No	Template:No	Template:Yes	Template:No	Template:Partial	Template:No	Template:No	Template:No	ooDACE
DACE	Template:Proprietary	MATLAB	Template:Yes	Template:No	Template:No	Template:Yes	Template:No	Template:No	Template:No	Template:No	Template:Partial	Template:No	Template:No	Template:No	DACE
GpGp	Template:Free	R	Template:No	Template:No	Template:Yes	Template:Yes	Template:No	Template:No	Template:Partial	Template:Partial	Template:Partial	Template:No	Template:No	Template:No	GpGp
SuperGauss	Template:GPL-lic	R, C++	Template:No	Template:YesTemplate:Efn	Template:No	Template:No	Template:No	Template:No	Template:No	Template:Partial	Template:Partial	Template:No	Template:No	Template:No	SuperGauss
STK	Template:GPL-lic	MATLAB	Template:Yes	Template:No	Template:No	Template:Yes	Template:No	Template:No	Template:Partial	Template:Partial	Template:Partial	Template:No	Template:No	Template:Partial	STK
GSTools	Template:LGPL-lic	Python	Template:Yes	Template:No	Template:No	Template:Yes	Template:No	Template:No	Template:Yes	Template:Yes	Template:Yes	Template:Yes	Template:No	Template:No	GSTools
PyKrige	Template:BSD-lic	Python	Template:Yes	Template:No	Template:No	Template:Partial	Template:No	Template:No	Template:Partial	Template:No	Template:No	Template:No	Template:No	Template:No	PyKrige
GPR	Template:Free	C++	Template:Yes	Template:No	Template:Yes	Template:Yes	Template:No	Template:No	Template:Partial	Template:Partial	Template:Partial	Template:Partial	Template:No	Template:No	GPR
celerite2	Template:Free	Python	Template:No	Template:YesTemplate:Efn	Template:No	Template:No	Template:No	Template:No	Template:Partial	Template:PartialTemplate:Efn	Template:Partial	Template:No	Template:No	Template:Yes	celerite2
SMT	Template:Free	Python	Template:Yes	Template:YesTemplate:Efn	Template:Yes	Template:Yes	Template:Yes	Template:No	Template:Partial	Template:Yes	Template:Yes	Template:Yes	Template:No	Template:No	SMT
GPJax	Template:Free	Python	Template:Yes	Template:No	Template:Yes	Template:Yes	Template:Yes	Template:Partial	Template:No	Template:Yes	Template:Yes	Template:No	Template:No	Template:No	GPJax
Stheno	Template:Free	Python	Template:Yes	Template:Yes	Template:Yes	Template:Yes	Template:No	Template:No	Template:Partial	Template:Partial	Template:Partial	Template:Partial	Template:No	Template:Yes	Stheno
CODES		MATLAB	Template:Yes	Template:YesTemplate:Efn	Template:Yes	Template:Yes	Template:No	Template:No	Template:Partial	Template:Partial	Template:Partial	Template:No	Template:No	Template:No	CODES
Egobox-gp	Template:Free	Rust	Template:Yes	Template:No	Template:Yes	Template:Yes	Template:Yes	Template:Yes	Template:Partial	Template:Yes	Template:Yes	Template:Yes	Template:No	Template:No	Egobox-gp
SAMBO	Template:Free	Python	Template:Yes	Template:No	Template:Yes	Template:Yes	Template:Yes	Template:Yes	Template:Yes	Template:Yes	Template:Yes	Template:No	Template:No	Template:No	SAMBO
Name	License	Language	Exact	Specialized	Approximate	ND	Template:Nowrap | Non-real	Likelihood	Errors	Prior	Posterior	Deriv.	Finite	Sum	Name
			Solvers			Input		Output		Hyperparameters		Linear transformations

External links

• [1] The website hosting C. E. Rasmussen's book Gaussian processes for machine learning; contains a (partially outdated) list of software.