This week, we will start to work on multivariate models, and non-independence. The first idea to discuss non-independence will be to use the concept of exchangeability. A sequence of random variable http://freakonometrics.blog.free.fr/public/perso5/exch-06.gif is said to be exchangeable if for all http://freakonometrics.blog.free.fr/public/perso5/exch-05.gif

http://freakonometrics.blog.free.fr/public/perso5/exch-01.gif

for any permutation http://freakonometrics.blog.free.fr/public/perso5/exch-02.gif of http://freakonometrics.blog.free.fr/public/perso5/exch-03.gif. A standard example is the case where http://freakonometrics.blog.free.fr/public/perso5/exch-07.gif, with

http://freakonometrics.blog.free.fr/public/perso5/exch-08.gif

and

http://freakonometrics.blog.free.fr/public/perso5/exch-09.gif

Since http://freakonometrics.blog.free.fr/public/perso5/exch-19.gif, a necessary condition is that

http://freakonometrics.blog.free.fr/public/perso5/exch-11.gif

i.e.

http://freakonometrics.blog.free.fr/public/perso5/exch-12.gif


Since this inequality should hold for all http://freakonometrics.blog.free.fr/public/perso5/exch-05.gif it comes that necessarily http://freakonometrics.blog.free.fr/public/perso5/exch-13.gif.
de Finetti (1931): Let http://freakonometrics.blog.free.fr/public/perso5/exch-06.gif be a sequence of random variables with values in http://freakonometrics.blog.free.fr/public/perso5/exch-14.gif. http://freakonometrics.blog.free.fr/public/perso5/exch-06.gif is exchangeable if and only if there exists a distribution function http://freakonometrics.blog.free.fr/public/perso5/exch-15.gif on http://freakonometrics.blog.free.fr/public/perso5/exch-16.gif such that

http://freakonometrics.blog.free.fr/public/perso5/credit-04.gif

where http://freakonometrics.blog.free.fr/public/perso5/exch-20.gif. Note that http://freakonometrics.blog.free.fr/public/perso5/exch-15.gif is the distribution function of random variable

http://freakonometrics.blog.free.fr/public/perso5/exch-22.gif

A nice proof of that result can be found in Heath & Sudderth (1995) - see also Schervish (1995), Chow & Teicher (1997) or Durrett (2010) and also probably in several bayesian books because that result has a strong interpretation in bayesian inference (as far as I understood, see e.g. Jaynes (1982)).
From the exchangeability condition, for any permutation http://freakonometrics.blog.free.fr/public/perso5/defi02.gif of http://freakonometrics.blog.free.fr/public/perso5/defi03.gif,

http://freakonometrics.blog.free.fr/public/perso5/defi01b.gif

that can be inverted in

http://freakonometrics.blog.free.fr/public/perso5/defi05.gif

The idea is then to extend the size of the vector http://freakonometrics.blog.free.fr/public/perso5/defi09.gif, i.e. for all http://freakonometrics.blog.free.fr/public/perso5/defi07.gif, define

http://freakonometrics.blog.free.fr/public/perso5/defi10.gif

so that, if we condition on http://freakonometrics.blog.free.fr/public/perso5/defi11.gif,

http://freakonometrics.blog.free.fr/public/perso5/defi08.gif

but since given the sum of components of http://freakonometrics.blog.free.fr/public/perso5/defi11.gif, all possible rearrangements of the ones among the http://freakonometrics.blog.free.fr/public/perso5/GPD11.gif elements are equally likely, we can write

http://freakonometrics.blog.free.fr/public/perso5/defi15.gif

The first idea is to work on the blue term, and to invocate a theorem of approximation of the hypergeometric distribution http://freakonometrics.blog.free.fr/public/perso5/defi17.gif to a binomial distribution http://freakonometrics.blog.free.fr/public/perso5/defi19.gif, when http://freakonometrics.blog.free.fr/public/perso5/defi50.gif becomes large. Then

http://freakonometrics.blog.free.fr/public/perso5/defi16.gif

Let http://freakonometrics.blog.free.fr/public/perso5/defi20.gif and let http://freakonometrics.blog.free.fr/public/perso5/defi21.gif denote the cumulative distribution function of http://freakonometrics.blog.free.fr/public/perso5/defi28.gif.

http://freakonometrics.blog.free.fr/public/perso5/defi33.gif

The idea is then to write the sum as an integral, with respect to that distribution,

http://freakonometrics.blog.free.fr/public/perso5/defi30.gif

The theorem is then obtained since http://freakonometrics.blog.free.fr/public/perso5/defi31.gif, i.e.

http://freakonometrics.blog.free.fr/public/perso5/defi32.gif

In the case of non-binary sequences, there is an extension of the previous result,
Hewitt & Savage (1955): Let http://freakonometrics.blog.free.fr/public/perso5/exch-06.gif be a sequence of random variables with values in http://freakonometrics.blog.free.fr/public/perso5/exch-24.gifhttp://freakonometrics.blog.free.fr/public/perso5/exch-06.gif is exchangeable if and only if there exists a measure http://freakonometrics.blog.free.fr/public/perso5/exch-25.gif on http://freakonometrics.blog.free.fr/public/perso5/exch-26.gif such that

http://freakonometrics.blog.free.fr/public/perso5/exc99.gif

where http://freakonometrics.blog.free.fr/public/perso5/exch-25.gif is the measure associated to the empirical measure

http://freakonometrics.blog.free.fr/public/perso5/exch-29.gif

and

http://freakonometrics.blog.free.fr/public/perso5/exc98.gif

For instance, in the Gaussian case mentioned earlier, if

http://freakonometrics.blog.free.fr/public/perso5/exch-23.gif

then

http://freakonometrics.blog.free.fr/public/perso5/exch-30.gif

where

http://freakonometrics.blog.free.fr/public/perso5/exch-31.gif

i.e. conditionally on http://freakonometrics.blog.free.fr/public/perso5/exch-32.gif, the http://freakonometrics.blog.free.fr/public/perso5/exch-06.gif are conditionally independent, with distribution http://freakonometrics.blog.free.fr/public/perso5/exch-33.gif. The proof can be found in Kingman (1978) and is based on martingale arguments.
Note that in the Gaussian case, http://freakonometrics.blog.free.fr/public/perso5/excccc02.gif where http://freakonometrics.blog.free.fr/public/perso5/exccc03.gif are i.i.d. random variables. To go further on exchangeability and related topics, see Aldous (1985)  (see also here).
This construction can be used in credit risk, to model defaults in an homogeneous portfolio, see e.g. Frey (2001),

Assuming a Beta distribution for the latent factor, we can derive the probability distribution of the sum

http://freakonometrics.blog.free.fr/public/perso5/credit-01.gif

Since

http://freakonometrics.blog.free.fr/public/perso5/exch61.gif

if we assume that - given the latent factor - http://freakonometrics.blog.free.fr/public/perso5/exch67.gif (either the company defaults, or not),

http://freakonometrics.blog.free.fr/public/perso5/exch66.gif

i.e.

http://freakonometrics.blog.free.fr/public/perso5/exch63.gif

Thus, we can derive the (unconditional) distribution of the sum

http://freakonometrics.blog.free.fr/public/perso5/exch60.gif

i.e.

http://freakonometrics.blog.free.fr/public/perso5/exch68.gif

> proba=function(s,a,m,n){
+ b=a/m-a
+ choose(n,s)*integrate(function(t){t^s*(1-t)^(n-s)*
+ dbeta(t,a,b)},lower=0,upper=1)$value
+ }

Based on that function, it is possible to plot the probability distribution over http://freakonometrics.blog.free.fr/public/perso5/credit-5.gif. In the upper corner is plotted the density of the Beta distribution.

> a=2
> m=.2
+ n=10
+ V=rep(NA,n+1)
+ for(i in 0:n){
+ V[i+1]=proba(i,a,m,n)}
> barplot(V,names.arg=0:10)

http://freakonometrics.blog.free.fr/public/perso5/exchangeable-beta.gif

Those two theorems are extremely close,

De Finetti's theorem: a random sequence http://freakonometrics.blog.free.fr/public/perso5/dfhs1.gif of http://freakonometrics.blog.free.fr/public/perso5/dfhs4.gif random variables is exchangeable if and only if http://freakonometrics.blog.free.fr/public/perso5/dfhs2.gif's are conditionnally independent, conditionnally on some random variable http://freakonometrics.blog.free.fr/public/perso5/dfhs3.gif.

Hewitt-Savage's theorem: a random sequence http://freakonometrics.blog.free.fr/public/perso5/dfhs1.gif is exchangeable if and only if http://freakonometrics.blog.free.fr/public/perso5/dfhs2.gif's are conditionnally independent, conditionnally on some sigma-algebra http://freakonometrics.blog.free.fr/public/perso5/dfhs5.gif

Olshen (1974), proposed an interesting discussion about those theorems, see also in the Encyclopedia of Statistical Science,

http://freakonometrics.free.fr/copecran1.png

The subtle difference between those two theorem is also discussed in Freedman (1965)

http://freakonometrics.free.fr/copecran2.png