Partial correlation

From Wikipedia, the free encyclopedia

In probability theory and statistics, partial correlation measures the degree of association between two random variables, with the effect of a set of controlling random variables removed.

1 Formal definition
2 Computation
3 Interpretation
- 3.1 Geometrical
- 3.2 As conditional independence test
4 See also
5 References

[edit] Formal definition

Formally, the partial correlation between X and Y given a set of n controlling variables Z = {Z₁, Z₂, …, Z_n}, written ρ_XY·Z, is the correlation between the residuals R_X and R_Y resulting from the linear regression of X with Z and of Y with Z, respectively.

[edit] Computation

[edit] Using linear regression

The obvious way to compute a (sample) partial correlation is to solve the two associated linear regression problems, get the residuals, and calculate the correlation between the residuals. If we write x_i, y_i and z_i to denote i.i.d. samples of some joint probability distribution over X, Y and Z, solving the linear regression problem amounts to finding

$\mathbf{w}_X^* = \arg\min_{\mathbf{w}} \left\{ \sum_{i=1}^N (x_i - \langle\mathbf{w}, \mathbf{z}_i \rangle)^2 \right\}$

$\mathbf{w}_Y^* = \arg\min_{\mathbf{w}} \left\{ \sum_{i=1}^N (y_i - \langle\mathbf{w}, \mathbf{z}_i \rangle)^2 \right\}$

with N being the number of samples and $\langle\mathbf{v},\mathbf{w} \rangle$ the scalar product between the vectors v and w. The residuals are then

$r_{X,i} = x_i - \langle\mathbf{w}_X^*,\mathbf{z}_i \rangle$

$r_{Y,i} = y_i - \langle\mathbf{w}_Y^*,\mathbf{z}_i \rangle$

and the sample partial correlation is

$\hat{\rho}_{XY\cdot\mathbf{Z}}=\frac{N\sum_{i=1}^N r_{X,i}r_{Y,i}-\sum_{i=1}^N r_{X,i}\sum r_{Y,i}} {\sqrt{N\sum_{i=1}^N r_{X,i}^2-\left(\sum_{i=1}^N r_{X,i}\right)^2}~\sqrt{N\sum_{i=1}^N r_{Y,i}^2-\left(\sum_{i=1}^N r_{Y,i}\right)^2}}.$

[edit] Using recursive formula

It can be computationally expensive to solve the linear regression problems. Actually, the nth-order partial correlation (i.e., with |Z| = n) can be easily computed from three (n - 1)th-order partial correlations. The zeroth-order partial correlation ρ_XY·Ø is defined to be the regular correlation coefficient ρ_XY.

It holds, for any $Z_0 \in \mathbf{Z}$ :

$\rho_{XY\cdot \mathbf{Z} } = \frac{\rho_{XY\cdot\mathbf{Z}\setminus\{Z_0\}} - \rho_{XZ_0\cdot\mathbf{Z}\setminus\{Z_0\}}\rho_{YZ_0\cdot\mathbf{Z}\setminus\{Z_0\}}} {\sqrt{1-\rho_{XZ_0\cdot\mathbf{Z}\setminus\{Z_0\}}^2} \sqrt{1-\rho_{YZ_0\cdot\mathbf{Z}\setminus\{Z_0\}}^2}}.$

Naïvely implementing this computation as a recursive algorithm yields an exponential time complexity. However, this computation has the overlapping subproblems property, such that using dynamic programming or simply caching the results of the recursive calls yields a complexity of $\mathcal{O}(n^3)$ .

[edit] Using matrix inversion

Another approach allows to compute in $\mathcal{O}(n^3)$ all partial correlations between any two variables X_i and X_j of a set V of cardinality n given all others, i.e., $\mathbf{V} \setminus \{X_i,X_j\}$ , provided the correlation matrix Ω = (ω_ij), where ω_ij = ρ_{X_iX_j}, is invertible. If we define P = Ω^-1, we have:

$\rho_{X_iX_j\cdot \mathbf{V} \setminus \{X_i,X_j\}} = -\frac{p_{ij}}{\sqrt{p_{ii}p_{jj}}}.$

[edit] Interpretation

Geometrical interpretation of partial correlation

[edit] Geometrical

Let three variables X, Y, Z be chosen from a joint probability distribution over n variables V. Further let v_i, 1 ≤ i ≤ N, be N n-dimensional i.i.d. samples taken from the joint probability distribution over V. We then consider the N-dimensional vectors x (formed by the successive values of X over the samples), y (formed by the values of Y) and z (formed by the values of Z).

It can be shown that the residuals R_X coming from the linear regression of X using Z, if also considered as an N-dimensional vector r_X, have a zero scalar product with the vector z generated by Z. This means that the residuals vector lives on a hyperplane S_z which is perpendicular to z.

The same also applies to the residuals R_Y generating a vector r_Y. The desired partial correlation is then the cosine of the angle φ between the projections r_X and r_Y of x and y, respectively, onto the hyperplane perpendicular to z.^[1]

[edit] As conditional independence test

With the assumption that all involved variables are multivariate Gaussian, the partial correlation ρ_XY·Z is zero if and only if X is conditionally independent from Y given Z.^[2] This property does not hold in the general case.

In order to test if a sample partial correlation $\hat{\rho}_{XY\cdot\mathbf{Z}}$ vanishes, Fisher's z-transform of the partial correlation can be used:

$z(\hat{\rho}_{XY\cdot\mathbf{Z}}) = \frac{1}{2} \ln\left(\frac{1+\hat{\rho}_{XY\cdot\mathbf{Z}}}{1-\hat{\rho}_{XY\cdot\mathbf{Z}}}\right).$

The null hypothesis is $H_0: \hat{\rho}_{XY\cdot\mathbf{Z}} = 0$ , to be tested against the two-tail alternative $H_A: \hat{\rho}_{XY\cdot\mathbf{Z}} \neq 0$ . We reject H₀ with significance level α if:

$\sqrt{N - |\mathbf{Z}| - 3}\cdot z(\hat{\rho}_{XY\cdot\mathbf{Z}}) > \Phi^{-1}(1-\alpha/2),$

where Φ(·) is the cumulative distribution function of a Gaussian distribution with zero mean and unit standard deviation, and N is the sample size.

[edit] See also

[edit] References

^ Rummel, R. J. (1976). Understanding Correlation.
^ Baba, K.; Shibata, R. and Sibuya, M. (2004). "Partial correlation and conditional correlation as measures of conditional independence". Australian and New Zealand Journal of Statistics 46 (4).

Retrieved from "http://en.wikipedia.org../../../p/a/r/Partial_correlation.html"

Categories: Covariance and correlation | Time series analysis