View source for Root mean square deviation

{{Short description|Statistical measure}}
{{StatsTopicTOC}}

The '''root mean square deviation''' ('''RMSD''') or '''root mean square error''' ('''RMSE''') is either one of two closely related and frequently used measures of the differences between true or predicted values on the one hand and observed values or an [[estimator]] on the other.
The [[deviation (statistics)|deviation]] is typically simply a differences of [[scalar (mathematics)|scalars]]; it can also be generalized to the [[vector length]]s of a [[displacement (geometry)|displacement]], as in the [[bioinformatics]] concept of [[root mean square deviation of atomic positions]].

==RMSD of a sample==
The RMSD of a [[Sampling (statistics)|sample]] is the [[quadratic mean]] of the differences between the observed values and predicted ones. These [[Statistical deviation|deviations]] are called ''[[Errors and residuals in statistics|residuals]]'' when the calculations are performed over the data sample that was used for estimation (and are therefore always in reference to an estimate) and are called ''errors'' (or prediction errors) when computed out-of-sample (aka on the full set, referencing a true value rather than an estimate). The RMSD serves to aggregate the magnitudes of the errors in predictions for various data points into a single measure of predictive power. RMSD is a measure of [[Accuracy and precision|accuracy]], to compare forecasting errors of different models for a particular dataset and not between datasets, as it is scale-dependent.<ref>{{cite journal|last1=Hyndman|first1=Rob J.|last2=Koehler|first2=Anne B.|title=Another look at measures of forecast accuracy|journal=International Journal of Forecasting|year=2006|pages=679–688|doi=10.1016/j.ijforecast.2006.03.001|volume=22|issue=4|citeseerx=10.1.1.154.9771|s2cid=15947215 }}</ref>

RMSD is always non-negative, and a value of 0 (almost never achieved in practice) would indicate a perfect fit to the data. In general, a lower RMSD is better than a higher one. However, comparisons across different types of data would be invalid because the measure is dependent on the scale of the numbers used.

RMSD is the square root of the average of squared errors. The effect of each error on RMSD is proportional to the size of the squared error; thus larger errors have a disproportionately large effect on RMSD. Consequently, RMSD is sensitive to [[Outlier|outliers]].<ref name=":0">{{Cite journal|last1=Pontius|first1=Robert|last2=Thontteh|first2=Olufunmilayo|last3=Chen|first3=Hao|date=2008|title=Components of information for multiple resolution comparison between maps that share a real variable|journal=Environmental Ecological Statistics|volume=15|issue=2|pages=111–142|doi=10.1007/s10651-007-0043-y|bibcode=2008EnvES..15..111P |s2cid=21427573 |url=https://commons.clarku.edu/context/faculty_geography/article/1769/viewcontent/GeogFacWorks_Pontius_Components_2006.pdf }}</ref><ref>{{Cite journal|last1=Willmott|first1=Cort|last2=Matsuura|first2=Kenji|date=2006|title=On the use of dimensioned measures of error to evaluate the performance of spatial interpolators|journal=International Journal of Geographical Information Science|volume=20|issue=1 |pages=89–102|doi=10.1080/13658810500286976|bibcode=2006IJGIS..20...89W |s2cid=15407960 }}</ref>

== Formulas ==
===Estimator===
The RMSD of an [[estimator]] <math>\hat{\theta}</math> with respect to an estimated parameter <math>\theta</math> is defined as the square root of the [[mean squared error]]:

:<math>\operatorname{RMSD}(\hat{\theta}) = \sqrt{\operatorname{MSE}(\hat{\theta})} = \sqrt{\operatorname{E}((\hat{\theta}-\theta)^2)}.</math>

For an [[unbiased estimator]], the RMSD is the square root of the [[variance]], known as the [[standard deviation]].

===Samples===
If {{math|''X''<sub>1</sub>, ..., ''X<sub>n</sub>''}} is a sample of a population with true mean value <math>x_0</math>, then the RMSD of the sample is

:<math>\operatorname{RMSD} = \sqrt{\frac{1}{n}\sum_{i=1}^n(X_i-x_0)^2}</math>.

The RMSD of predicted values <math>\hat y_t</math> for times ''t'' of a [[regression analysis|regression's]] [[dependent variable]] <math>y_t,</math> with variables observed over ''T'' times, is computed for ''T'' different predictions as the square root of the mean of the squares of the deviations:

:<math>\operatorname{RMSD}=\sqrt{\frac{\sum_{t=1}^T (y_t - \hat y_t)^2}{T}}.</math>

(For regressions on [[cross-sectional data]], the subscript ''t'' is replaced by ''i'' and ''T'' is replaced by ''n''.)

In some disciplines, the RMSD is used to compare differences between two things that may vary, neither of which is accepted as the "standard".  For example, when measuring the average difference between two time series <math>x_{1,t}</math> and <math>x_{2,t}</math>,
the formula becomes
:<math>\operatorname{RMSD}= \sqrt{\frac{\sum_{t=1}^T (x_{1,t} - x_{2,t})^2}{T}}.</math>

==Normalization==
Normalizing the RMSD facilitates the comparison between datasets or models with different scales. Though there is no consistent means of normalization in the literature, common choices are the mean or the range (defined as the maximum value minus the minimum value) of the measured data:<ref>{{cite web|title=Coastal Inlets Research Program (CIRP) Wiki - Statistics|url=http://cirpwiki.info/wiki/Statistics#Normalization|access-date=4 February 2015}}</ref>
:<math>\mathrm{NRMSD} = \frac{\mathrm{RMSD}}{y_\max -y_\min}</math> or <math> \mathrm{NRMSD} = \frac {\mathrm{RMSD}}{\bar y}  </math>.

This value is commonly referred to as the ''normalized root mean square deviation'' or ''error'' (NRMSD or NRMSE), and often expressed as a percentage, where lower values indicate less residual variance. This is also called [[Coefficient of Variation]] or '''Percent RMS'''.  In many cases, especially for smaller samples, the sample range is likely to be affected by the size of sample which would hamper comparisons.

Another possible method to make the RMSD a more useful comparison measure is to divide the RMSD by the [[interquartile range]] (IQR). When dividing the RMSD with the IQR the normalized value gets less sensitive for extreme values in the target variable.

:<math>\mathrm{RMSDIQR} = \frac{\mathrm{RMSD}}{IQR}</math>  where <math>IQR = Q_3 - Q_1</math>

with <math>Q_1 = \text{CDF}^{-1}(0.25)</math> and <math>Q_3 = \text{CDF}^{-1}(0.75) ,</math> where CDF<sup>−1</sup> is the [[quantile function]].

When normalizing by the mean value of the measurements, the term ''coefficient of variation of the RMSD, CV(RMSD)'' may be used to avoid ambiguity.<ref>{{cite web|title=FAQ: What is the coefficient of variation?|url=https://stats.idre.ucla.edu/other/mult-pkg/faq/general/faq-what-is-the-coefficient-of-variation/|access-date=19 February 2019}}</ref> This is analogous to the [[coefficient of variation]] with the RMSD taking the place of the [[standard deviation]].

:<math> \mathrm{CV(RMSD)} = \frac {\mathrm{RMSD}}{\bar y} .</math>

== Mean absolute error ==
Some researchers{{who|date=December 2023}} have recommended{{where|date=December 2023}} the use of the [[mean absolute error]] (MAE) instead of the root mean square deviation. MAE possesses advantages in interpretability over RMSD. MAE is the average of the absolute values of the errors. MAE is fundamentally easier to understand than the square root of the average of squared errors. Furthermore, each error influences MAE in direct proportion to the absolute value of the error, which is not the case for RMSD.<ref name=":0" />

==Applications==
*In [[meteorology]], to see how effectively a [[mathematics|mathematical]] model predicts the behavior of the [[atmosphere]].
*In [[bioinformatics]], the [[root mean square deviation of atomic positions]] is the measure of the average distance between the atoms of [[protein structural alignment|superimposed]] [[proteins]].
*In [[Drug design#Structure-based|structure based drug design]], the RMSD is a measure of the difference between a crystal conformation of the ligand [[Conformational isomerism|conformation]] and a [[docking (molecular)|docking]] prediction.
*In [[economics]], the RMSD is used to determine whether an economic model fits [[economic indicator]]s. Some experts have argued that RMSD is less reliable than Relative Absolute Error.<ref>{{cite journal|url=http://faculty.weatherhead.case.edu/Fred-Collopy/researchArticles/ErrorMeasures.pdf | title = Error Measures For Generalizing About Forecasting Methods: Empirical Comparisons |last1=Armstrong |first1=J. Scott |last2=Collopy |first2=Fred |journal = International Journal of Forecasting | volume = 8 | pages = 69–80 | year = 1992 | doi=10.1016/0169-2070(92)90008-w | issue=1| citeseerx = 10.1.1.423.508 | s2cid = 11034360 }}</ref>
*In [[experimental psychology]], the RMSD is used to assess how well mathematical or computational models of behavior explain the empirically observed behavior.
*In [[GIS]], the RMSD is one measure used to assess the accuracy of spatial analysis and [[remote sensing]].
*In [[hydrogeology]], RMSD and NRMSD are used to evaluate the calibration of a groundwater model.<ref>{{cite book |title=Applied Groundwater Modeling: Simulation of Flow and Advective Transport |publisher=Academic Press |year=1992 |last=Anderson |first=M.P. |author2=Woessner, W.W.  |edition=2nd}}</ref>
*In [[imaging science]], the RMSD is part of the [[peak signal-to-noise ratio]], a measure used to assess how well a method to reconstruct an image performs relative to the original image.
*In [[computational neuroscience]], the RMSD is used to assess how well a system learns a given model.<ref>[http://www.ocgy.ubc.ca/projects/clim.pred/NN/3.1/model.html Ensemble Neural Network Model<!-- Bot generated title -->]</ref>
*In [[protein nuclear magnetic resonance spectroscopy]], the RMSD is used as a measure to estimate the quality of the obtained bundle of structures.
*Submissions for the [[Netflix Prize]] were judged using the RMSD from the test dataset's undisclosed "true" values.
*In the simulation of energy consumption of buildings, the RMSE and CV(RMSE) are used to calibrate models to measured [[building performance]].<ref>[http://www.bpi.org/Web%20Download/BPI%20Standards/BPI-2400-S-2012_Standard_Practice_for_Standardized_Qualification_of_Whole-House%20Energy%20Savings_9-28-12_sg.pdf ANSI/BPI-2400-S-2012: Standard Practice for Standardized Qualification of Whole-House Energy Savings Predictions by Calibration to Energy Use History]</ref>
*In [[X-ray crystallography]], RMSD (and RMSZ) is used to measure the deviation of the molecular internal coordinates deviate from the restraints library values.
*In control theory, the RMSE is used as a quality measure to evaluate the performance of a [[state observer]].<ref>[https://kalman-filter.com/root-mean-square-error/ https://kalman-filter.com/root-mean-square-error]</ref>
*In [[fluid dynamics]], normalized root mean square deviation (NRMSD), coefficient of variation (CV), and percent RMS are used to quantify the uniformity of flow behavior such as velocity profile, temperature distribution, or gas species concentration.  The value is compared to industry standards to optimize the design of flow and thermal equipment and processes.

==See also==
*[[Root mean square]]
*[[Mean absolute error]]
*[[Average absolute deviation]]
*[[Mean signed deviation]]
*[[Mean squared error]]
*[[Squared deviations from the mean]]
*[[Errors and residuals]]
*[[Coefficient of variation]]

==References==
{{reflist}}
{{Machine learning evaluation metrics}}

[[Category:Point estimation performance]]
[[Category:Statistical deviation and dispersion]]