View source for Aliasing (factorial experiments)

{{Short description|Statistical phenomenon where some effects appear the same}}

{{About|aliasing in statistics, in particular fractional factorial designs|other uses|Aliasing (disambiguation)}}

In the statistical theory of [[factorial experiment]]s, '''aliasing''' is the property of [[fractional factorial design]]s that makes some effects "aliased" with each other – that is, indistinguishable from each other. A primary goal of the theory of such designs is the control of aliasing so that important effects are not aliased with each other.<ref>{{harvtxt|Cheng |2019 |p=5}}</ref>

In a "full" factorial experiment, the number of ''treatment combinations'' or ''cells'' (see [[#Overview|below]]) can be very large.{{refn|group="note"|The number of treatment combinations [[Exponential growth|grows exponentially]] with the number of factors in the experiment.<ref>{{harvtxt|Mukerjee|Wu|2006|pages=1-2}}</ref>}}  This necessitates limiting observations to a ''fraction'' (subset) of the treatment combinations.
Aliasing is an automatic and unavoidable result of observing such a fraction.<ref>{{harvtxt|Kempthorne|1947|page=390}}</ref><ref>{{harvtxt|Dean|Voss|Draguljić|2017|p=495}}</ref>

The aliasing properties of a design are often summarized by giving its
[[#Resolution|''resolution'']]. This measures the degree to which the design avoids aliasing between main effects and important interactions.<ref name="auto1">{{harvtxt|Box|Hunter|1961|p=319}}</ref>

Fractional factorial experiments have long been a basic tool in
agriculture,<ref>{{harvtxt|Jankowski|Budzyński|Załuski|Hulanicki|Dubis|2016}}</ref> food technology,<ref>{{harvtxt|Kempthorne|1947|loc = section 21.7}}</ref><ref>{{harvtxt|Cornell|2006|loc = sections 7.6-7.7}}</ref> industry,<ref>{{harvtxt|Hamada|Wu|1992|loc = examples 1 and 3}}</ref><ref>{{harvtxt|Box|Hunter|Hunter|2005|loc = sections 6.3 and 6.4}}</ref><ref>{{harvtxt|Dean|Voss|Draguljić|2017|loc=chapter 7}}</ref> medicine and public health,<ref>{{harvtxt|Hamada|Wu|1992|loc = example 2}}</ref><ref>{{harvtxt|Nair| Strecher|Fagerlin|Ubel|Resnicow|Murphy|Little|Chakraborty|Zhang|2008}}</ref> and the social and behavioral sciences.<ref>{{harvtxt|Collins|Dziak|Li||2009}}</ref>
They are widely used in exploratory research,<ref>{{harvtxt|Kempthorne|1947|page=390}}</ref> particularly in screening experiments, which have applications in industry, drug design and genetics.<ref>{{harvtxt|Dean|Lewis|2006}}</ref>  In all such cases, a crucial step in designing such an experiment is deciding on the desired aliasing pattern, or at least the desired resolution.

As noted [[#History|below]], the concept of aliasing may have influenced the identification of an analogous phenomenon in signal processing theory.

== Overview ==

Associated with a factorial experiment is a collection of ''effects''.  Each factor determines a ''main effect'', and each set of two or more factors determines an ''interaction effect'' (or simply an ''interaction'') between those factors.  Each effect is defined by a set of relations between ''cell means'', as described below.  In a ''fractional'' factorial design, effects are defined by restricting these relations to the cells in the fraction.  It is when the restricted relations for two different effects turn out to be the same that the effects are said to be aliased.

The presence or absence of a given effect in a given data set is tested by statistical methods, most commonly [[analysis of variance]].  While aliasing has significant implications for estimation and hypothesis testing, it is fundamentally a combinatorial and algebraic phenomenon.  Construction and analysis of fractional designs thus rely heavily on algebraic methods.

The definition of a fractional design is sometimes broadened to allow multiple observations of some or all treatment combinations – a [[Multiset|''multisubset'']] of all treatment combinations.<ref>{{harvtxt|Cheng |2019 |p=117}}</ref> A fraction that is a subset (that is, where treatment combinations are not repeated) is called ''simple''. The theory described below applies to simple fractions.

== Contrasts and effects ==

{| class="wikitable" style="float:right;"
|+ Cell means in a 2 × 3 factorial experiment
| {{diagonal split header|''A''|''B''}}
| 1
| 2
| 3
|-
| 1
| μ<sub>11</sub>
| μ<sub>12</sub>
| μ<sub>13</sub>
|-
| 2
| μ<sub>21</sub>
| μ<sub>22</sub>
| μ<sub>23</sub>
|}

In any design, full or fractional, the [[expected value]] of an observation in a given treatment combination is called a ''cell mean'',<ref>{{harvtxt|Hocking |1985 |page=73}}. Hocking and others use the term "population mean" for expected value.</ref> usually denoted using the Greek letter μ. (The term ''cell'' is borrowed from its use in [[Table (information)|tables of data]].)

A ''[[contrast (statistics)|contrast]] in cell means'' is a [[linear combination]] of cell means in which the coefficients sum to 0.  In the 2 × 3 experiment illustrated here, the expression

: <math> \mu_{11} - \mu_{12} </math>

is a contrast that compares the mean responses of the treatment combinations 11 and 12. (The coefficients here are 1 and –1.)

The effects in a factorial experiment are expressed in terms of contrasts.<ref>{{harvtxt|Hocking |1985 |pages=140–141}}</ref><ref>{{harvtxt|Kuehl|2000|pages=186–187}}</ref>  In the above example, the contrast

: <math> \mu_{11} + \mu_{12} + \mu_{13} - \mu_{21} - \mu_{22} - \mu_{23} </math>

is said to ''belong to the main effect of factor A'' as it contrasts the responses to the "1" level of factor <math>A</math> with those for the "2" level.  The main effect of ''A'' is said to be ''absent'' if this expression equals 0.<ref group="note">The claim that a contrast equals 0 is a statement that must be assessed by a [[statistical hypothesis test]]. See the article on [[Factorial_experiment#Contrasts,_main_effects_and_interactions|factorial experiments]] for further detail.</ref>  Similarly,

: <math> \mu_{11} + \mu_{21} - \mu_{12} - \mu_{22} </math> &nbsp; and &nbsp; 
: <math> \mu_{11} + \mu_{21} - \mu_{13} - \mu_{23} </math>

are contrasts belonging to the main effect of factor ''B''.  On the other hand, the contrasts

: <math> \mu_{11} - \mu_{12} - \mu_{21} + \mu_{22} </math> &nbsp; and &nbsp; 
: <math> \mu_{11} - \mu_{13} - \mu_{21} + \mu_{23} </math>

''belong to the interaction of A and B''; setting them equal to 0 expresses the lack of interaction.{{refn|group="note"|Compare the [[interaction (statistics)#In ANOVA|example]] in the article on interaction.}}  These designations, which extend to arbitrary factorial experiments having three or more factors, depend on the pattern of coefficients, as explained [[factorial experiment#Contrasts, main effects and interactions|elsewhere]].<ref>{{harvtxt|Bose|1947|p=110}}</ref><ref>{{harvtxt|Beder|2022|p=161}}</ref>

Since it is the coefficients of these contrasts that carry the essential information, they are often displayed as [[Row and column vectors|column vectors]].  For the example above, such a table might look like this:<ref>{{harvtxt|Beder|2022|loc=Example 5.21}}</ref>
{{anchor|2x3-ex}}
{| class="wikitable defaultright col1center col2center col3center"
! cell  !!  <math>A</math> !! colspan=2 | <math>B</math> !! colspan=2 | <math>A \times B</math>
|-
| 11 ||  1 || 1 || 0 || 1 || 1
|-
| 12 ||  1 || −1 || 1 || -1 || 0
|-
| 13 ||  1 || 0 || −1 || 0 || −1
|-
| 21 ||  −1 || 1 || 0 || −1 || -1
|-
| 22 ||  −1 || −1 || 1 || 1 || 0
|-
| 23 ||  −1 || 0 || −1 || 0 || 1
|}

The columns of such a table are called ''contrast vectors'': their components add up to 0.  While there are in general many possible choices of columns to represent a given effect, the ''number'' of such columns — the ''degrees of freedom'' of the effect — is fixed and is given by a well-known formula.<ref>{{harvtxt|Kuehl|2000|p=202}}</ref><ref>{{harvtxt|Cheng |2019 |p=78}}</ref>  In the 2 × 3 example above, the degrees of freedom for <math>A, B</math>, and the <math>A\times B</math> interaction are 1, 2 and 2, respectively.

In a fractional factorial experiment, the contrast vectors belonging to a given effect are restricted to the treatment combinations in the fraction.  Thus, in the half-fraction {11, 12, 13} in the 2 × 3 example, the three effects may be represented by the column vectors in the following table:
{{anchor|2x3-frac-ex}}
{| class="wikitable defaultright col1center col2center col3center"
! cell  !!  <math>A</math> !! colspan=2 | <math>B</math> !! colspan=2 | <math>A \times B</math>
|-
| 11 ||  1 || 1 || 0 || 1 || 1
|-
| 12 ||  1 || −1 || 1 || −1 || 0
|-
| 13 ||  1 || 0 || −1 || 0 || −1
|}
The consequence of this truncation — aliasing — is described [[#The 2 × 3 experiment|below]].

== Definitions ==

The factors in the design are allowed to have different numbers of levels, as in a <math>2 \times 3 \times 3</math> factorial experiment (an ''asymmetric'' or ''mixed-level'' experiment).

Fix a fraction of a full factorial design. Let <math>U</math> be a set of contrast vectors representing an effect (in particular, a main effect or interaction) in the full factorial design, and let <math>\widetilde{U}</math> consist of the restrictions of those vectors to the fraction. One says that the effect is
* ''preserved in the fraction'' if <math>\widetilde{U}</math> consists of contrast vectors;
* ''completely lost in the fraction'' if <math>\widetilde{U}</math> consists of ''constant'' vectors, that is, vectors whose components are equal; and
* ''partly lost'' otherwise.

Similarly, let <math>U_1</math> and <math>U_2</math> represent two effects and let <math>\widetilde{U}_1</math> and <math>\widetilde{U}_2</math> be their restrictions to the fraction. The two effects are said to be
*  ''unaliased in the fraction'' if each vector in <math>\widetilde{U}_1</math> is [[Orthogonality (mathematics)|orthogonal]] (perpendicular) to all the vectors in <math>\widetilde{U}_2</math>, and vice versa;
*  ''completely aliased in the fraction'' if each vector in <math>\widetilde{U}_1</math> is a [[linear combination]] of vectors in <math>\widetilde{U}_2</math>, and vice versa;{{refn|group="note"|In a more formal exposition, the sets <math>U</math> and <math>\widetilde{U}</math> are [[vector space]]s, and two effects are completely aliased in the fraction if <math>\widetilde{U}_1 = \widetilde{U}_2.</math><ref>{{harvtxt|Beder|2022|loc=Definition 6.4}}</ref>}} and
* [[#Partial aliasing|''partly aliased'']] otherwise.

Finney<ref>{{harvtxt|Finney|1945|p=293}}</ref> and Bush<ref>{{harvtxt|Bush|1950|page=3}}</ref> introduced the terms "lost" and "preserved" in the sense used here. Despite the relatively long history of this topic, though, its terminology is not entirely standardized. The literature often describes lost effects as "not estimable" in a fraction,<ref>{{harvtxt|Mukerjee|Wu|2006|loc=Theorem 2.4.1}}</ref> although estimation is not the only issue at stake. Rao<ref>{{harvtxt|Rao|1947|p=135}}</ref> referred to preserved effects as "measurable from" the fraction.

=== Resolution ===
The extent of aliasing in a given fractional design is measured by the ''resolution'' of the fraction, a concept first defined by [[George E. P. Box|Box]] and Hunter:<ref name="auto1">{{harvtxt|Box|Hunter|1961|p=319}}</ref>

:  A fractional factorial design is said to have ''resolution'' <math>R</math> if every <math>p</math>-factor effect<ref group="note">A 1-factor effect is the main effect of a single factor. For <math>p \ge 2</math>, a <math>p</math>-factor effect is an interaction between <math>p</math> factors. The 0-factor effect is the effect of the grand mean, described below.</ref> is unaliased with every effect having fewer than <math>R-p</math> factors.

For example, a design has resolution <math>R = 3</math> if main effects are unaliased with each other (taking <math>p = 1)</math>, though it allows main effects to be aliased with two-factor interactions.  This is typically the lowest resolution desired for a fraction.  It is not hard to see that a fraction of resolution <math>R</math> also has resolution <math>R-1, R-2</math>, etc., so one usually speaks of the ''maximum'' resolution of a fraction.

The number <math>p</math> in the definition of resolution is usually understood to be a positive integer, but one may consider the ''effect of the grand mean'' to be the (unique) effect with no factors (i.e., with <math>p = 0</math>).  This effect sometimes appears in [[analysis of variance]] tables.<ref>{{harvtxt|Searle|1987|p=30}}</ref>  It has one degree of freedom, and is represented by a single vector, a column of 1's.<ref>{{harvtxt|Beder|2022|p=165}}</ref>  With this understanding, an effect is
* preserved in a fraction if it is unaliased with the grand mean, and
* completely lost in a fraction if it is completely aliased with the grand mean.
A fraction then has resolution <math>R = 2</math> if all main effects are preserved in the fraction.  If it has resolution <math>R = 3</math> then two-factor interactions are also preserved.

=== Computation ===

The definitions above require some computations with vectors, illustrated in the examples that follow.  For certain fractional designs (the ''regular'' ones), a simple algebraic technique can be used that bypasses these procedures and gives a simple way to determine resolution.  This is discussed [[#Determining the resolution of a regular fraction|below]].

== Examples ==

=== The 2 × 3 experiment ===

The fraction {11, 12, 13} of this experiment was described [[#2x3-frac-ex|above]] along with its restricted vectors.  It is repeated here along with the complementary fraction {21, 22, 23}:

<div style="display: flex; flex-direction: row; gap: 80px;">
  <div style=display:inline-table>
{| class="wikitable defaultright col1center col2center col3center"
! cell  !!  <math>A</math> !! colspan=2 | <math>B</math> !! colspan=2 | <math>A \times B</math>
|-
| 11 ||  1 || 1 || 0 || 1 || 1
|-
| 12 ||  1 || −1 || 1 || −1 || 0
|-
| 13 ||  1 || 0 || −1 || 0 || −1
|}
  </div>
  <div style=display:inline-table>
{| class="wikitable defaultright col1center col2center col3center"
! cell  !!  <math>A</math> !! colspan=2 | <math>B</math> !! colspan=2 | <math>A \times B</math>
|-
| 21 ||  −1 || 1 || 0 || −1 || −1
|-
| 22 ||  −1 || −1 || 1 || 1 || 0
|-
| 23 ||  −1 || 0 || −1 || 0 || 1
|}
  </div>
</div>

In both fractions, the <math>A</math> effect is completely lost (the <math>A</math> column is constant) while the <math>B</math> and interaction effects are preserved (each 3 × 1 column is a contrast vector as its components sum to 0). In addition, the <math>B</math> and interaction effects are completely aliased in each fraction:  In the first fraction, the vectors for <math>B</math> are linear combinations of those for <math>A \times B</math>, viz.,

<math>\begin{bmatrix} 1 \\ -1 \\ 0 \end{bmatrix} = \begin{bmatrix} 1 \\ -1 \\ 0 \end{bmatrix} + 0\begin{bmatrix} 1 \\ 0 \\ -1 \end{bmatrix} </math>
{{spaces|2}}and{{spaces|2}}
<math>\begin{bmatrix} 0 \\ 1 \\ -1 \end{bmatrix} = \begin{bmatrix} 1 \\ 0 \\ -1 \end{bmatrix} -\begin{bmatrix} 1 \\ -1 \\ 0 \end{bmatrix}</math>;

in the reverse direction, the vectors for <math>A \times B</math> can be written similarly in terms of those representing <math>B</math>. The argument in the second fraction is analogous.

These fractions have maximum resolution 1. The fact that the main effect of <math>A</math> is lost makes both of these fractions undesirable in practice.  It turns out that in a 2 × 3 experiment (or in any ''a × b'' experiment in which ''a'' and ''b'' are [[relatively prime]]) there is no fraction that preserves both main effects -- that is, no fraction has resolution 2.

=== The 2 × 2 × 2 (or 2³) experiment ===
This is a "two-level" experiment with factors <math>A, B</math> and <math>C</math>. In such experiments the factor levels are often denoted by 0 and 1, for reasons explained [[#mod 2|below]]. A treatment combination is then denoted by an [[ordered triple]] such as 101 (more formally, (1, 0, 1), denoting the cell in which <math>A</math> and <math>C</math> are at level "1" and <math>B</math> is at level "0"). The following table lists the eight cells of the full 2 × 2 × 2 factorial experiment, along with a contrast vector representing each effect, including a three-factor interaction:

{| class="wikitable defaultright col1center col2center col3center sortable"
! cell !! <math>A</math> !! <math>B</math> !! <math>C</math> !!<math>AB</math>!! <math>AC</math> !! <math>BC</math> !! <math>ABC</math>
|-
| 000   ||    1  ||    1  ||    1  ||     1  ||     1  ||     1  ||      1
|-
| 001   ||    1  ||    1  ||   −1  ||     1  ||    −1  ||    −1  ||     −1
|-
| 010   ||    1  ||   −1  ||    1  ||    −1  ||     1  ||    −1  ||     −1
|-
| 011   ||    1  ||   −1  ||   −1  ||    −1  ||    −1  ||     1  ||      1
|-
| 100   ||   −1  ||    1  ||    1  ||    −1  ||    −1  ||     1  ||     −1
|-
| 101   ||   −1  ||    1  ||   −1  ||    −1  ||     1  ||    −1  ||      1
|-
| 110   ||   −1  ||   −1  ||    1  ||     1  ||    −1  ||    −1  ||      1
|-
| 111   ||   −1  ||   −1  ||   −1  ||     1  ||     1  ||     1  ||     −1
|}

Suppose that only the fraction consisting of the cells 000, 011, 101, and 110 is observed. The original contrast vectors, when restricted to these cells, are now 4 × 1, and can be seen by looking at just those four rows of the table. (Sorting the table on <math>ABC</math> will bring these rows together and make the restricted contrast vectors easier to see. Sorting twice puts them at the top.) The following can be observed concerning these ''restricted'' vectors:
* The <math>ABC</math> column consists just of the constant 1 repeated four times.
* The other columns are contrast vectors, having two 1's and two −1s.
* The columns for <math>C</math> and <math>AB</math> are equal. The same holds for <math>A</math> and <math>BC</math>, and for <math>B</math> and <math>AC</math>.
* All other pairs of columns are orthogonal. For example, the column for <math>A</math> is orthogonal to that for <math>B</math>, for <math>C</math>, for <math>AB</math>, and for <math>AC</math>, as one can see by computing [[dot products]].
Thus
* the <math>ABC</math> interaction is completely lost in the fraction;
* the other effects are preserved in the fraction;
* the effects <math>A</math> and <math>BC</math> are completely aliased with each other, as are <math>B</math> and <math>AC</math>, and <math>C</math> and <math>AB</math>.
* all other pairs of effects are unaliased. For example, <math>A</math> is unaliased with both <math>B</math> and <math>C</math> and with the <math>AB</math> and <math>AC</math> interactions.

Now suppose instead that the complementary fraction {001,010,100,111} is observed. The same effects as before are lost or preserved, and the same pairs of effects as before are mutually unaliased. Moreover, <math>A</math> and <math>BC</math> are still aliased in this fraction since the <math>A</math> and <math>BC</math> vectors are negatives of each other, and similarly for <math>B</math> and <math>AC</math> and for <math>C</math> and <math>AB</math>.  Both of these fractions thus have maximum resolution 3.

== Aliasing in regular fractions ==

The two half-fractions of a <math>2^3</math> factorial experiment described above are of a special kind: Each is the solution set of a linear equation using ''[[modular arithmetic]]''. More exactly:
* The fraction <math>\{000,011,101,110\}</math> is the solution set of the equation <math>t_1 + t_2 + t_3 = 0 \pmod 2 </math>. For example, <math>011</math> is a solution because <math>0 + 1 + 1 = 0 \pmod 2 </math>.
* Similarly, the fraction <math>\{001,010,100,111\}</math> is the solution set to <math>t_1 + t_2 + t_3 = 1 \pmod 2 </math>

Such fractions are said to be ''regular''. This idea applies to fractions of "classical" <math>s^k</math> designs, that is, <math>s^k</math> [[Factorial experiment#Notation|(or "symmetric") factorial designs]] in which the number of levels, <math>s</math>, of each of the <math>k</math> factors is a prime or the power of a prime.
: A fractional factorial design is ''regular'' if it is the solution set of a system of one or more equations of the form
:: <math>a_1t_1 + \cdots + a_kt_k = b,</math>
: where the equation is modulo <math>s</math> if <math>s</math> is prime, and is in the [[finite field]] <math>GF(s)</math> if <math>s</math> is a power of a prime.<ref group="note">The case that <math>s</math> is prime is mentioned separately only for clarity, since the set of integers modulo <math>s</math> is itself a finite field, though often denoted <math>\mathbb{Z}/s</math> rather than <math>GF(s)</math>.</ref>   Such equations are called ''defining equations''<ref>{{harvtxt|Cheng |2019 |p=141}}</ref> of the fraction. When the defining equation or equations are [[System of linear equations#Homogeneous systems|''homogeneous'']], the fraction is said to be ''principal''.

One defining equation yields a fraction of size <math>s^{k-1}</math>, two independent equations a fraction of size <math>s^{k-2},</math> and so on. Such fractions are generally denoted as <math>s^{k-r}</math> designs. The half-fractions described above are <math>2^{3-1}</math> designs. The notation often includes the resolution as a subscript, in Roman numerals;  the above fractions are thus <math>2^{3-1}_{III}</math> designs.

Associated to each expression <math>a_1t_1 + \cdots + a_kt_k</math> is another, namely <math>A_1^{a_1} \cdots A_k^{a_k}</math>, which rewrites the coefficients as exponents. Such expressions are called "''words''", a term borrowed from [[group theory]]. (In a particular example where <math>k</math> is a specific number, the letters <math>A, B, C \ldots</math> are used, rather than <math>A_1, A_2, A_3 \ldots</math>.) These words can be multiplied and raised to powers, where the word <math>I=A_1^0 \cdots A_k^0</math> acts as a multiplicative identity, and they thus form an ''[[abelian group]]'' <math>\mathbb{G}</math>, known as the ''effects group''.<ref name="auto" /> When <math>s</math> is prime, one has <math>W^s = I</math> for every element (word) <math>W \in \mathbb{G}</math>; something similar holds in the prime-power case.
In <math>2^k</math> factorial experiments, each element of <math>\mathbb{G}</math> represents a main effect or interaction.  In <math>s^k</math> experiments with <math>s > 2</math>, each one-letter word represents the main effect of that factor, while longer words represent ''components of interaction''.<ref>{{harvtxt|Montgomery|2013|loc=p.397ff}}</ref><ref>{{harvtxt|Wu|Hamada|2009|loc=Section 6.3}}</ref><ref>{{harvtxt|Beder|2022|p=188}}</ref>  An example below illustrates this with <math>s=3</math>.

To each ''defining'' expression (the left-hand side of a defining equation) corresponds a ''defining word''. The defining words [[Generating set of a group|generate a subgroup]] <math>\mathbb{H}</math> of <math>\mathbb{G}</math> that is variously called the ''alias subgroup'',<ref name="auto">{{harvtxt|Finney|1945|p=293}}</ref> the ''defining contrast subgroup'',<ref>{{harvtxt|Wu|Hamada|2009|p=209}}</ref> or simply the ''defining subgroup'' of the fraction.  Each element of <math>\mathbb{H}</math> is a defining word since it corresponds to a defining equation, as one can show.<ref>{{harvtxt|Beder|2022|p=224}}</ref> The effects represented by the defining words are completely lost in the fraction while all other effects are preserved. If <math>\mathbb{H} = \{I, W_1, \ldots, W_{\ell}\}</math>, say, then the equation{{refn| group="note"| The equalities in this equation are a convention, and stand for a kind of equivalence of group elements.<ref>{{harvtxt|Beder|2022|p=234}}</ref>  In a more formal exposition, they represent the actual equality of spaces <math>\widetilde{U}</math> of restricted vectors, where the identity element <math>I</math> stands for the space of constant vectors.  }}
: <math> I = W_1 = \cdots = W_{\ell} </math>
is called the ''defining relation'' of the fraction.<ref>{{harvtxt|Cheng |2019 |p=140}}</ref><ref>{{harvtxt|Dean|Voss|Draguljić|2017|p=496}}</ref><ref>{{harvtxt|Montgomery |2013|p=322}}</ref><ref>{{harvtxt|Mukerjee|Wu|2006|p=26}}</ref><ref>{{harvtxt|Wu|Hamada|2009|p=207}}</ref> This relation is used to determine the aliasing structure of the fraction:  If a given effect is represented by the word <math>W</math>, then its aliases are computed by multiplying the defining relation by <math>W</math>, viz.,
: <math> W = WW_1 = \cdots = WW_{\ell},</math>
where the products <math>WW_i</math> are then simplified. This relation indicates complete (not partial) aliasing, and W is unaliased with all other effects listed in <math>\mathbb{G}</math>.

===Example 1===
In either of the <math>2^{3-1}</math> fractions described above, the defining word is <math>ABC</math>, since the exponents on these letters are the coefficients of <math>t_1 + t_2 + t_3</math>.  The <math>ABC</math> effect is completely lost in the fraction, and the defining subgroup <math>\mathbb{H}</math> is simply <math>\{I, ABC\}</math>, since squaring does not generate new elements <math>((ABC)^2 = A^2B^2C^2 = I)</math>. The defining relation is thus
: <math> I = ABC</math>,
and multiplying both sides by <math>A</math> gives <math>A = A^2BC</math>;  which simplifies to
: <math>A = BC,</math> 
the alias relation seen earlier. Similarly, <math>B = AC</math> and <math>C = AB</math>.  Note that multiplying both sides of the defining relation by <math>AB, AC</math> and <math>BC</math> does not give any new alias relations.

For comparison, the <math>2^{3-1}</math> fraction with defining equation <math>t_1 + t_2 = 0 \pmod 2 </math> has the defining word <math>AB</math> (i.e., <math>A^1B^1C^0</math>). The effect <math>AB</math> is completely lost, and the defining relation is <math>I=AB</math>. Multiplying this by <math>A</math>, by <math>C</math>, and by <math>AC</math> gives the alias relations <math>A=B</math>, <math>C=ABC</math>, and <math>AC=BC</math> among the six remaining effects. This fraction only has resolution 2 since all effects (except <math>AB</math>) are preserved but two main effects are aliased.  Finally, solving the defining equation <math>t_1 + t_2 = 0 \pmod 2 </math> yields the fraction {000, 001, 110, 111}.  One may verify all of this by sorting the table above on column <math>AB</math>.
{{anchor|mod 2}}
The use of arithmetic modulo 2 explains why the factor levels in such designs are labeled 0 and 1.

===Example 2===

In a 3-level design, factor levels are denoted 0, 1 and 2, and arithmetic is modulo 3.  If there are four factors, say <math>A, B, C</math> and <math>D</math>, the effects group <math>\mathbb{G}</math> will have the relations
: <math>A^3 = B^3 = C^3 = D^3 = I.</math>
From these it follows, for example, that <math>D^4 = D</math> and <math>D^6 = I</math>.
{{anchor|3^(4-1)-ex}}
A defining equation such as <math>t_1 + t_2 + t_3 + 2t^4 = 0 \pmod{3}</math> would produce a regular 1/3-fraction of the 81 (= <math>3^4</math>) treatment combinations, and the corresponding defining word would be <math>ABCD^2</math>. Since its powers are
: <math>(ABCD^2)^2 = A^2B^2C^2D </math> &emsp; and &emsp;<math>(ABCD^2)^3 = I</math>,
the defining subgroup <math>\mathbb{H}</math> would be <math>\{I, ABCD^2, A^2B^2C^2D\}</math>, and so the fraction would have defining relation
: <math> I = ABCD^2 = A^2B^2C^2D.</math>
Multiplying by <math>A</math>, for example, yields the aliases
: <math> A = A^2BCD^2 = B^2C^2D.</math>
For reasons explained elsewhere,<ref>{{harvtxt|Wu|Hamada|2009|p=272}}</ref> though, all powers of a defining word represent the same effect, and the convention is to choose that power whose leading exponent is 1.  Squaring the latter two expressions does the trick<ref>This uses relations such as <math>A^4 = A</math></ref> and gives the alias relations
: <math> A = AB^2C^2D = BCD^2.</math>
Twelve other sets of three aliased effects are given by Wu and Hamada.<ref>{{harvtxt|Wu|Hamada|2009|p=275}}</ref>  Examining all of these reveals that, like <math>A</math>, main effects are unaliased with each other and with two-factor effects, although some two-factor effects are aliased with each other.  This means that this fraction has maximum resolution 4, and so is of type <math>3^{4-1}_{IV}</math>.

The effect <math>BCD^2</math> is one of 4 components of the <math>B \times C \times D</math> interaction, while <math>AB^2C^2D</math> is one of 8 components of the <math>A \times B \times C \times D</math> interaction.  In a 3-level design, each component of interaction carries 2 degrees of freedom.

===Example 3===

A <math>2^{5-2}</math> design (<math>1/4</math> of a <math>2^5</math> design) may be created by solving ''two'' equations in 5 unknowns, say
: <math>\begin{cases}
t_1 + t_2 + t_4 = 1\\
t_1 + t_3 + t_5 = 1
\end{cases}</math>
modulo 2. The fraction has eight treatment combinations, such as 10000, 00110 and 11111, and is displayed in the article on [[fractional factorial design]]s.<ref group="note">That article uses alternate notation for treatment combinations; for example, 10000, 00110 and 11111 are expressed as <math>a, cd</math> and <math>abcde</math>.</ref>  Here the coefficients in the two defining equations give defining words <math>ABD</math> and <math>ACE</math>. Setting <math>I = ABD</math> and multiplying through by <math>D</math> gives the alias relation <math>D = AB</math>. The second defining word similarly gives <math>E = AC</math>. The article uses these two aliases to describe an alternate method of construction of the fraction.

The defining subgroup <math>\mathbb{H}</math> has one more element, namely the product <math>(ABD)(ACE)</math> <math>= BCDE</math>, making use of the fact that <math>A^2 = I</math>. The extra defining word <math>BCDE</math> is known as the ''generalized interaction'' of <math>ABD</math> and <math>ACE</math>,<ref>{{harvtxt|Barnard|1936|p=197}}</ref> and corresponds to the equation <math>t_2 + t_3 + t_4 + t_5 = 0 \pmod2</math>, which is also satisfied by the fraction. With this word included, the full defining relation is
: <math>I = ABD = ACE = BCDE</math>
(these are the four elements of the defining subgroup), from which all the alias relations of this fraction can be derived – for example, multiplying through by <math>D</math> yields 
: <math>D = AB = ACDE = BCE</math>.

Continuing this process yields six more alias sets, each containing four effects.  An examination of these sets reveals that main effects are not aliased with each other, but are aliased with two-factor interactions. This means that this fraction has maximum resolution 3. A quicker way to determine the resolution of a regular fraction is given below.

It is notable that the alias relations of the fraction depend only on the left-hand side of the defining equations, not on their constant terms. For this reason, some authors will restrict attention to principal fractions "[[without loss of generality]]", although the reduction to the principal case often requires verification.{{refn|See, for example,<ref>{{harvtxt|Beder|2022|loc=proof of Proposition 6.6}}</ref>.}}

=== Determining the resolution of a regular fraction ===

The ''length of a word'' in the effects group is defined to be the number of letters in its name, not counting repetition. For example, the length of the word <math>AB^2C</math> is 3.<ref group="note">This differs from the definition used in [[Word (group theory)|group theory]], which counts repetitions.  According to the latter view, the length of <math>AB^2C</math> is 4.</ref>

{{math theorem | The maximum resolution of a regular fractional design is equal to the minimum length of a defining word.<ref>{{harvtxt|Raghavarao|1988|p=278}}</ref><ref>{{harvtxt|Beder|2022|p=238}}. The identity element of the defining subgroup is not a defining word.</ref>}}

Using this result, one immediately gets the resolution of the preceding examples without computing alias relations:
*In the <math>2^{3-1}</math> fraction with defining word <math>ABC</math>, the maximum resolution is 3 (the length of that word), while the fraction with defining word <math>AB</math> has maximum resolution 2.
*The defining words of the <math>3^{4-1}</math> fraction were <math>ABCD^2</math> and <math>A^2B^2C^2D</math>, both of length 4, so that the fraction has maximum resolution 4, as indicated.
*In the <math>2^{5-2}</math> fraction with defining words <math>ABD, ACE</math> and <math>BCDE</math>, the maximum resolution is 3, which is the shortest "wordlength".
:One could also construct a <math>2^{5-2}</math> fraction from the defining words <math>ABCD</math> and <math>BCDE</math>, but the defining subgroup <math>\mathbb{H}</math> will also include <math>AE</math>, their product, and so the fraction will only have resolution 2 (the length of <math>AE</math>). This is true starting with any two words of length 4. Thus resolution 3 is the best one can hope for in a fraction of type <math>2^{5-2}</math>.

As these examples indicate, one must consider ''all'' the elements of the defining subgroup in applying the theorem above.  This theorem is often taken to be a definition of resolution,<ref>{{harvtxt|Cheng|2019|p=147}}</ref><ref>{{harvtxt|Wu|Hamada|2009|p=210}}</ref> but the Box-Hunter definition given [[#Resolution|earlier]] applies to arbitrary fractional designs and so is more general.

== Aliasing in general fractions ==
Nonregular fractions are common, and have certain advantages.  For example, they are not restricted to having size a power of <math>s</math>, where <math>s</math> is a prime or prime power. While some methods have been developed to deal with aliasing in particular nonregular designs, no overall algebraic scheme has emerged.

There is a universal ''combinatorial'' approach, however, going back to Rao.<ref>{{harvtxt|Rao|1947}}</ref><ref>{{harvtxt|Rao|1973|p=354}}</ref> If the treatment combinations of the fraction are written as rows of a table, that table is an ''[[orthogonal array]]''. These rows are often referred to as "runs".  The columns will correspond to the <math>k</math> factors, and the entries of the table will simply be the symbols used for factor levels, and need not be numbers. The number of levels need not be prime or prime-powered, and they may vary from factor to factor, so that the table may be a ''mixed-level'' array. In this section fractional designs are allowed to be mixed-level unless explicitly restricted.

A key parameter of an orthogonal array is its ''strength'', the definition of which is given in the [[Orthogonal array#Definitions|article on orthogonal arrays]]. One may thus refer to the ''strength'' of a fractional design. Two important facts flow immediately from its definition:
*If an array (or fraction) has strength <math>t</math> then it also has strength <math>t'</math> for every <math>t'<t</math>.  The array's ''maximum'' strength is of particular importance.
*In a fixed-level array, all factors having <math>s</math> levels, the number of runs is a multiple of <math>s^t</math>, where <math>t</math> is the strength.  Here <math>s</math> need not be a prime or prime power.

To state the next result, it is convenient to enumerate the factors of the experiment by 1 through <math>k</math>, and to let each nonempty subset <math>I</math> of <math>\{1, \ldots, k\}</math> correspond to a main effect or interaction in the following way:  <math>I = \{i\}</math> corresponds to the main effect of factor
<math>i</math>, <math>I = \{i, j\}</math> corresponds to the interaction of factors <math>i</math> and <math>j</math>, and so on.
{{math theorem | The Fundamental Theorem of Aliasing<ref>{{harvtxt|Beder|2022|loc=Theorem 6.43}}</ref>|  Consider a fraction of strength <math>t \ge 1</math> on <math>k</math> factors.  Let <math>I, J \subseteq \{1, \ldots, k\}</math>.
#If <math>1 \le |I| \le t</math>, then the effect corresponding to <math>I</math> is preserved in the fraction.<ref>{{harvtxt|Bush|1950|page=3}}</ref>
#If <math>|I \cup J| \le t</math> and <math>I \neq J</math>, then the effects corresponding to <math>I</math> and <math>J</math> are unaliased in the fraction.
 }}

'''Example''': Consider a fractional factorial design with factors <math>A, B, C, D, E</math> and maximum strength <math>t=3</math>. Then:
# All effects up to three-factor interactions are preserved in the fraction.
# Main effects are unaliased with each other and with two-factor interactions.
# Two-factor interactions are unaliased with each other ''if'' they share a factor. For example, the <math>A \times B</math> and <math>A \times C</math> interactions are unaliased, but the <math>A \times B</math> and <math>C \times D</math> interactions may be at least partly aliased as the set <math>\{A,B,C,D\}</math> contains 4 elements but the strength of the fraction is only 3.

The Fundamental Theorem has a number of important consequences. In particular, it follows almost immediately that if a fraction has strength <math>t</math> then it has resolution <math>t+1</math>.  With additional assumptions, a stronger conclusion is possible:
{{math theorem | Theorem<ref>{{harvtxt|Beder|2022|loc=Theorem 6.51}}</ref>| If a fraction has ''maximum'' strength <math>t</math> and ''maximum'' resolution <math>R,</math> then <math>R=t+1.</math>  }}

This result replaces the group-theoretic condition (minimum wordlength) in regular fractions with a combinatorial condition (maximum strength) in arbitrary ones.

'''Example.'''  An important class of nonregular two-level designs are [[Plackett-Burman design]]s.  As with all fractions constructed from [[Hadamard matrix|Hadamard matrices]], they have strength 2, and therefore resolution 3.<ref>{{harvtxt|Hedayat|Sloane|Stufken|1999|loc=Theorem 7.5}}</ref>  The smallest such design has 11 factors and 12 runs (treatment combinations), and is displayed in the [[Plackett-Burman design|article]] on such designs.  Since 2 is its maximum strength,<ref group="note">The strength cannot be 3 since 12 is not a multiple of <math>2^3</math>.</ref> 3 is its maximum resolution. Some detail about its aliasing pattern is given in the next section.

== Partial aliasing ==

In regular <math>2^k</math> fractions there is no partial aliasing: Each effect is either preserved or completely lost, and effects are either unaliased or completely aliased. The same holds in regular <math>s^k</math> experiments with <math>s > 2</math> if one considers only main effects and components of interaction. However, a limited form of partial aliasing occurs in the latter.  For example, in the <math>3^{4-1}</math> design described [[#3^(4-1)-ex|above]] the overall <math>A \times B \times C \times D</math> interaction is partly lost since its <math>ABCD^2</math> component is completely lost in the fraction while its other components (such as <math>AB^2C^2D</math>) are preserved.  Similarly, the main effect of <math>A</math> is partly aliased with the <math>B \times C \times D</math> interaction since <math>A</math> is completely aliased with its <math>BCD^2</math> component and unaliased with the others.

In contrast, partial aliasing is uncontrolled and pervasive in nonregular fractions. In the 12-run Plackett-Burman design described in the previous section, for example, with factors labeled <math>A</math> through <math>K</math>, the only ''complete'' aliasing is between "complementary effects" such as <math>A</math> and <math>BCDEFGHIJK</math> or <math>ABCJK</math> and <math>DEFGHI</math>. Here the main effect of factor <math>A</math> is unaliased with the other main effects and with the <math>AB</math> interaction, but it is partly aliased with 45 of the 55 two-factor interactions, 120 of the 165 three-factor interactions, and 150 of the 330 four-factor interactions. This phenomenon is generally described as ''complex aliasing''.<ref>{{harvtxt|Wu|Hamada|2009|loc=Chapter 9}}</ref>  Similarly, 924 effects are preserved in the fraction, 1122 effects are partly lost, and only one (the top-level interaction <math>ABCDEFGHIJK</math>) is completely lost.

== Analysis of variance (ANOVA) ==
Wu and Hamada<ref>{{harvtxt|Wu|Hamada|2009|loc=Section 6.5}}</ref> analyze a data set collected on the <math>3^{4-1}_{IV}</math> fractional design described [[#3^(4-1)-ex|above]]. Significance testing in the [[Analysis of variance#Algorithm|analysis of variance]] (ANOVA) requires that the error sum of squares and the degrees of freedom for error be nonzero. In order to insure this, two design decisions have been made:

{| class="wikitable" style="float:right"
|+ ANOVA in the <math>3^{4-1}_{IV}</math> design
| Source
| df
|-
| <math>A</math>
| 2
|-
| <math>B</math>
| 2
|-
| <math>C</math>
| 2
|-
| <math>D</math>
| 2
|-
| <math>AB=CD^2</math>
| 2
|-
| <math>AB^2</math>
| 2
|-
| <math>AC=BD^2</math>
| 2
|-
| <math>AC^2</math>
| 2
|-
| <math>AD</math>
| 2
|-
| <math>BC=AD^2</math>
| 2
|-
| <math>BC^2</math>
| 2
|-
| <math>BD</math>
| 2
|-
| <math>CD</math>
| 2
|-
| Error
| 54
|-
|Total
| 80
|}

* Interactions of three or four factors have been assumed absent.  This decision is consistent with the ''effect hierarchy principle''.<ref>{{harvtxt|Hamada|Wu|1992|page=132}}</ref>
* Replication (inclusion of repeated observations) is necessary. In this case, three observations were made on each of the 27 treatment combinations in the fraction, for a total of 81 observations.

The accompanying table shows just two columns of an ANOVA table<ref>{{harvtxt|Wu|Hamada|2009|loc=Tables 6.6 and 6.7}}</ref> for this experiment. Only main effects and components of two-factor interactions are listed, including three pairs of aliases.  Aliasing between some two-factor interactions is expected, since the maximum resolution of this design is 4.

This experiment studied two response variables.  In both cases, some aliased interactions were statistically significant. This poses a challenge of interpretation, since without more information or further assumptions it is impossible to determine which interaction is responsible for significance. In some instances there may be a theoretical basis to make this determination.<ref>{{harvtxt|Wu|Hamada|2009|pp=279–280}}</ref>

This example shows one advantage of fractional designs. The full <math>3^4</math> factorial experiment has 81 treatment combinations, but taking one observation on each of these would leave no degrees of freedom for error.  The fractional design also uses 81 observations, but on just 27 treatment combinations, in such a way that one can make inferences on main effects and on (most) two-factor interactions.  This may be sufficient for practical purposes.

== History ==
The first statistical use of the term "aliasing" in print is the 1945 paper by Finney,<ref>{{harvtxt|Finney|1945|p=292}}</ref> which dealt with regular fractions with 2 or 3 levels. The term was imported into [[signal processing]] theory a few years later, possibly influenced by its use in factorial experiments; the history of that usage is described in the article on [[Aliasing#Historical_usage|aliasing in signal processing]].

The 1961 paper in which Box and Hunter introduced the concept of "resolution" dealt with regular two-level designs, but their initial definition<ref name="auto1" /> makes no reference to lengths of defining words and so can be understood rather generally. [[C. R. Rao|Rao]] actually makes implicit use of resolution in his 1947 paper<ref>{{harvtxt|Rao|1947}}</ref> introducing orthogonal arrays, reflected in an important parameter inequality that he develops. He distinguishes effects in full and fractional designs by using symbols <math>[\cdots]</math> and <math>\{\cdots\}</math> (corresponding to <math>U</math> and <math>\widetilde{U}</math>), but makes no mention of aliasing.

The term ''confounded'' is often used as a synonym for ''aliased'', and so one must read the literature carefully. The former term "is generally reserved for the indistinguishability of a treatment contrast and a block contrast",<ref>{{harvtxt|Dean|Voss|Draguljić|2017|p=495}}</ref> that is, for ''[[confounding with blocks]]''. [[Oscar Kempthorne|Kempthorne]] has shown<ref>{{harvtxt|Kempthorne|1947|pp=264–268}}</ref> how confounding with blocks in a <math>k</math>-factor experiment may be viewed as aliasing in a fractional design with <math>k+1</math> factors, but it is unclear whether one can do the reverse.

== See also ==
*The article on [[fractional factorial design]]s discusses examples in two-level experiments.

== Notes ==
{{reflist|group = note}}

== Citations ==
{{reflist}}

== References ==
{{refbegin}}
* {{cite journal|last = Barnard |first = Mildred M. | author-link=Mildred Barnard | title = Enumeration of the confounded arrangements in the <math>2 \times 2 \times 2 \cdots</math> factorial designs | journal = Supplement to the Journal of the Royal Statistical Society | date= 1936 | volume = 3 | pages = 195–202  |doi = 10.2307/2983671 |jstor = 2983671 }}
* {{cite book |last = Beder |first = Jay H. |date = 2022 |title = Linear Models and Design |publisher = [[Springer Publishing|Springer]] |location = Cham, Switzerland |doi=10.1007/978-3-031-08176-7|isbn = 978-3-031-08175-0 |s2cid = 253542415 }}
*{{Cite journal |last=[[Raj Chandra Bose|Bose]] |first=R. C. |date=1947 |title=Mathematical theory of the symmetrical factorial design |journal=Sankhya |volume=8 |pages=107–166}}
*{{cite journal|last1 = Box |first1 = G. E. P. | author-link1=George E. P. Box | last2 = Hunter | first2 = J. S. | title = The <math>2^{k-p}</math> fractional factorial designs | journal = Technometrics | date= 1961 | volume = 3 | pages = 311–351 }}

*{{cite book |last1 = Box |first1 = George E. P. | author-link1=George E. P. Box |last2 = Hunter |first2 = J. S.|last3 = Hunter |first3 = William B.|date = 2005 |title = Statistics for Experimenters: Design, Innovation, and Discovery| edition = 2nd |publisher = [[John Wiley & Sons, Inc.|Wiley]] |location = Hoboken, N.J. |isbn=978-0471-71813-0}}

*{{cite thesis |last=Bush |first=K. A. |date=1950 |title=Orthogonal arrays |degree=PhD |publisher=University of North Carolina, Chapel Hill}}
* {{cite book |last=Cheng |first=Ching-Shui |title = Theory of Factorial Design: Single- and Multi-Stratum Experiments |publisher = CRC Press |location = Boca Raton, Florida |date=2019 |isbn=978-0-367-37898-1}}

*{{cite journal|last1 = Collins |first1 = Linda M. |author-link1=Linda Collins (psychologist) | last2 = Dziak| first2 = John J.  | last3 = Li |first3 =Runze | author-link3=Runze Li  |  title = Design of experiments with multiple independent variables: A resource management perspective on complete and reduced factorial designs. | journal = Psychological Methods | date=2009 |volume = 14 |issue = 3 | pages= 202–224 |issn=1082-989X | doi= 10.1037/a0015826|pmid = 19719358 |pmc = 2796056 }}

*{{cite book |last = Cornell |first = John A.|date = 2006 |title = Experiments with Mixtures: Designs, Models, and the Analysis of Mixture Data |publisher = [[John Wiley & Sons, Inc.]] |location = New York| edition = 3rd |isbn=
0-471-39367-3}}

*{{cite book |last1 = Dean |first1 = Angela |last2 = Lewis |first2 = Susan|date = 2006 |title = Screening: Methods for Experimentation in Industry, Drug Discovery, and Genetics |publisher = [[Springer Publishing|Springer]] |location = Cham, Switzerland |isbn=978-0-387-28013-4}}

* {{cite book |last1 = Dean |first1 = Angela |last2 = Voss |first2 = Daniel |last3 = Draguljić |first3 = Danel |edition = 2nd |date = 2017 |title = Design and Analysis of Experiments|publisher = [[Springer Publishing|Springer]] |location = Cham, Switzerland |isbn=978-3-319-52250-0}}
* {{cite journal|last = Finney |first = D. J. | title = The fractional replication of factorial arrangements | journal = Annals of Eugenics | date= 1945 | volume = 12 | pages = 291–301  |doi = 10.1111/j.1469-1809.1943.tb02333.x }}
* {{cite journal|last1 = Hamada |first1 = M. S. | last2 = Wu | first2 = C. F. J. | author-link2=C. F. Jeff Wu | title = Analysis of designed experiments with complex aliasing | journal = Journal of Quality Technology | date= 1992 | volume = 24 |issue = 3 | pages=130–137 |doi = 10.1080/00224065.1992.11979383 }}
* {{cite book |last1 = Hedayat |first1 = A. S. |last2 = Sloane |first2 = N. J. A.| author-link2=Neil Sloane | last3 = Stufken |first3 = John |date = 1999 |title = Orthogonal Arrays: Theory and Applications |publisher = [[Springer Publishing|Springer]] |location = Cham, Switzerland |isbn = 978-0-387-98766-8}}
* {{cite book|last = Hocking |first = Ronald R. | title = The Analysis of Linear Models | publisher = [[Brooks/Cole]] | date= 1985 |location = Pacific Grove, CA |isbn = 978-0-534-03618-8  }}

*{{cite journal|last1 = Jankowski |first1 = Krzysztof J.| last2 = Budzyński| first2 = Wojciech S. | last3 = Załuski |first3 = Dariusz | last4 =Hulanicki | first4 = Piotr S.| last5 = Dubis |first5 = Bogdan| title = Using a fractional factorial design to evaluate the effect of the intensity of agronomic practices on the yield of different winter oilseed rape morphotypes | journal = Field Crops Research | date=2016 |volume = 188 | pages= 50–61 | doi=10.1016/j.fcr.2016.01.007 | bibcode=2016FCrRe.188...50J |issn=1872-6852}}

* {{cite journal|last = Kempthorne |first = Oscar |author-link = Oscar Kempthorne | title = A simple approach to confounding and fractional replication in factorial experiments | journal= [[Biometrika]] | date= 1947 |volume = 34 |issue = Pt 3-4 |pages = 255–272  |doi = 10.1093/biomet/34.3-4.255 |pmid = 18918693 }}
* {{cite book|last = Kuehl |first = Robert O. | title = Design of Experiments: Statistical Principles of Research Design and Analysis |edition = 2nd | publisher = [[Brooks/Cole]] | date= 2000 |location = Pacific Grove, CA |isbn =978-0-534-36834-0  }}
* {{cite book|last = Montgomery |first = Douglas | title = Design and Analysis of Experiments |edition = 8th | publisher = [[Wiley (publisher)|Wiley]] | date= 2013 |location = New York |isbn =978-1-118-14692-7  }}
* {{cite book |author-link1=Rahul Mukerjee |last1 =Mukerjee |first1 = Rahul | last2 = Wu | first2 = C. F. Jeff | author-link2=C. F. Jeff Wu | title = A Modern Theory of Factorial Designs | publisher = [[Springer Publishing|Springer]] | date= 2006 |location = New York |isbn = 978-0-387-31991-9 }}

*{{cite journal|last1 = Nair |first1 = Vijay| last2 = Strecher| first2 = Victor | last3 = Fagerlin |first3 =Angela | last4 =Ubel | first4 = Peter| last5 = Resnicow |first5 =Kenneth| last6 = Murphy | first6 = Susan| last7 = Little |first7 = Roderick| last8 = Chakrabort| first8 = Bibhas | last9 =Zhang  |first9 =Aijun| title = Screening experiments and the use of fractional factorial designs in behavioral intervention research | journal = American Journal of Public Health | date= 2008 |volume = 98 | issue=8 | pages= 1354–1359 |doi = 10.2105/AJPH.2007.127563| pmid=18556602 | pmc=2446451 |issn=0090-0036}}

* {{cite journal|last = Rao |first = C. R. | title = Factorial experiments derivable from combinatorial arrangements of arrays | journal = Supplement to the Journal of the Royal Statistical Society | date= 1947 | volume = 9 |issue = 1 | pages = 128–139  |doi = 10.2307/2983576 |jstor = 2983576 }}
*{{cite book |last =Rao |first = C. R. | editor-last = Srivastava | editor-first = Jagdish N. | title = A Survey of Combinatorial Theory | publisher = [[Elsevier]] | date= 1973 | chapter = Some combinatorial problems of arrays and applications to design of experiments |pages = 349–360 |location = New York |isbn = 978-0-444-10425-0 }}
* {{cite book |author-link=D. Raghavarao |last1 =Raghavarao |first1 = Damaraju | title = Constructions and Combinatorial Problems in Design of Experiments | publisher = [[Dover Publications|Dover]] | date= 1988 |location = Mineola, New York |isbn = 978-0-486-65685-4 }}
* {{cite book|last = Searle |first = Shayle R. |author-link=Shayle R. Searle | title = Linear Models for Unbalanced Data | publisher = [[Wiley (publisher)|Wiley]] | date= 1987 |location = New York |isbn = 978-0-471-84096-1  }}
* {{cite book |author-link1=C. F. Jeff Wu |last1 =Wu |first1 = C. F. Jeff | last2 = Hamada | first2 = Michael | title = Experiments: Planning, Analysis, and Optimization |edition = 2nd | publisher = [[Wiley (publisher)|Wiley]] | date= 2009 |location = New York |isbn = 978-0-471-69946-0 }}

{{refend}}

[[Category:Design of experiments]]
[[Category:Statistical process control]]