Conditional Independence in AI

Conditional Independence 

In probability theory a variable A is conditionally independent of a set of variables   given a third set 

if P(A_i | v_i,v_j) = P(A | v_j). 

We use the, notation I(A, v_i | v_j) to state the fact that A is conditionally independent. The notation I(A, v_i | v_j) states that A and v_i are conditionally independent if and only if given knowledge v_j knowledge of whether A occurs does not provide any information on likelihood of v_i occurring and knowledge of where v_i occurs provide no information on likelihood of A.

Example: 

    Let us say that we roll a blue die and a red die. The two results are independent of each other. Now a third condition that the blue result is not a 6 and red result is not a 1 imposed. This is the new information we got, but this has not affected the independence of the results.

    Because, we can't gain any information about the red die by looking at the blue die. So the probabilities for the results are conditionally independent given the additional information.

    If the third condition is like the sum of the two result is even then we can gain a lot about red die by looking at blue die. For instance, i.e., if wee see a 3 on blue die, the red die can only be 1, 3 or 5. So in this case there is no conditional independence.

    It cannot be noticed that the conditional independence is always relative to the condition given.

    If a single variable A_j is conditionally independent of another variable A_j where a set v is given. Then by definition we have,

P(A_i | A_j, v) = (A_i | v)   ------------ 1

From the definition of conditional probability we, have, 

P(A_i, | A_n, v) P(A_i | v) = P(A_i, A_j | v)  ------ 2

From Equations. 1 and 2 we have, 

        P(A1, Aj | v) = P(A, | v) P(A, | v) 

in case of 

    It can be noted that A_i and A_j appears to be symmetrical. That is if A_i is conditionally independent given set v also implies that A_j is conditionally independent of A_j     given v. 

    It also applies to sets v_i and v_j given v so p(v_i, v_j, | v)= P(v_i | v_j ) P(v_j | v). If the set v is empty then v_i and v_j are independent.

Mutually Conditional Independent:

 If each of the variables are conditionally independent of all other variable given a set v then the variables A₁,..... A_k are mutually conditionally independent 

        . 

and as every A_i is conditionally independent of others given a set v, we have, 

        

if v is empty, we have,. 

        

and the variables are unconditionally independent.