Econometric Methods - Uni Trier€¦ · Econometric Methods Revision: Matrix Algebra and Probability Dr. Matthias Op nger Lehrstuhl für Finanzwissenschaft WS 2015/16 Dr. Matthias

Econometric Methods

Dr. Matthias Op�nger

Lehrstuhl für Finanzwissenschaft

WS 2015/16

Dr. Matthias Op�nger Econometric Methods WS 2015/16 1 / 43

Econometric Methods

Revision: Matrix Algebra and Probability

Dr. Matthias Op�nger

Lehrstuhl für Finanzwissenschaft

WS 2015/16


Review of Matrix Algebra

Moving on to ...

1 Review of Matrix AlgebraBasic De�nitionsMatrix OperationsLinear Independence and Rank of a MatrixQuadratic Forms and Positive De�nite MatricesIdempotent MatricesDi�erentiation of Linear and Quadratic Forms

2 Review of ProbabilityRandom variables and Probability DistributionsExpected Values, Mean and VarianceTwo Random VariablesThe Normal DistributionRandom Sampling and the Sample Distribution


Review of Matrix Algebra Basic De�nitions

Basic De�nitions

A =

a11 a12 · · · a1Ka21 a22 · · · a2K...

. . .an1 an2 · · · anK

A matrix is a rectangular array of numbers.

An (n X K ) matrix has n rows and K columns.

n is the row dimension and K is the column dimension.

aij represents the element in the i th row and j th column.

A real number can be interpreted as a (1 X 1) matrix, which is called ascalar.



Basic De�nitions

A (1 X K ) matrix is called a row vector (of dimension K ) and can bewritten as x = (x1, x2, . . . , xK ).

An (n X 1) matrix is called a column vector (of dimension n) and can bewritten as

y =

y1y2...yn

Matrix A could be written in the following form:

A =[a1 a2 · · · aK

]in terms of column vectors aj .



Basic De�nitions

A square matrix has the same number of rows and columns (n = K ).

A symmetric matrix is a square matrix in which aij = aji for all i and j .

A =

1 3 73 5 27 2 1

A diagonal matrix is a square matrix whose o�-diagonal elements arezero, that is, aij = 0 for all i 6= j .

A =

a11 0 · · · 00 a22 · · · 0...

. . .0 0 · · · ann



Basic De�nitions

An (n X n) identity matrix, denoted I, or sometimes In to emphasize itsdimension, is the diagonal matrix with unity (one) in each diagonalposition, and zero elsewhere.

I = In =

1 0 · · · 00 1 · · · 0...

. . .0 0 · · · 1

An (n X K ) zero matrix, denoted 0, is the (n X K ) matrix with zero forall entities. A column vector of zeros will be denoted as o.

A triangular matrix is a square matrix that has only zeros either above orbelow the main diagonal. If the zeros are above the diagonal, the matrix islower triangular.


Review of Matrix Algebra Matrix Operations

Matrix Addition and Subtraction

Two matrices A and B, each having dimension (n X K ) can beadded/subtracted element by element: A + B = [aij + bij ]. More precisely,

A + B =

a11 + b11 a12 + b12 · · · a1K + b1Ka21 + b21 a22 + b22 · · · a2K + b2K

.... . .

an1 + bn1 an2 + bn2 · · · anK + bnK

Properties:

A + 0 = A

0 + A = A

A− A = 0

A + B = B + A

(A + B) + C = A + (B + C)



Scalar Multiplication

Given any real number (scalar) γ, scalar multiplication is de�ned asγA ≡ [γaij ], or

γA =

γa11 γa12 · · · γa1Kγa21 γa22 · · · γa2K...

. . .γan1 γan2 · · · γanK



Transposition

The transpose of a matrix A, denoted A′, is obtained by creating thematrix whose kth row is the kth column of the original matrix. If A is(n X K ), A′ is (K X n).

A =

1 2 35 1 56 4 53 1 4

A′ =

1 5 6 32 1 4 13 5 5 4

Properties:

(A′)′ = A

(αA)′ = αA′ for any scalar α.

(A + B)′ = A′ + B′

If A is symmetric, A = A′.The transpose of a column vector a is a row vector.



Inner Product and Matrix Multiplication

The inner product (or dot product) of a row vector a′ and a columnvector b is a scalar:

a′b = a1b1 + a2b2 + · · ·+ anbn

To multiply matrix A by matrix B to form the product AB, the columndimension of A must equal the row dimension of B.Let A be an (n X K ) matrix and let B (K X m) matrix. The productmatrix,

C = AB

is an (n X m) matrix whose ij th element is is the inner product of row i ofA and column j of B.



Matrix Multiplication

Let

A =

[a11 a12a21 a22

]and B =

[b11 b12 b13b21 b22 b23

]

A cross table might help to multiply the matrices and to determine theproduct matrix C.

B

A C

=

b11 b12 b13b21 b22 b23

a11 a12 c11 c12 c13a21 a22 c21 c22 c23

For instance, to �nd out c23 (cij), we calculate the inner product of the 2nd

(i th) row of matrix A and the 3rd (j th) column of matrix B, that isc23 = a21b13 + a22b23.




B

A C

=

b11 b12 b13b21 b22 b23

a11 a12 c11 c12 c13a21 a22 c21 c22 c23

C =

[c11 c12 c13

c21 c22 c23

]=

[a11b11 + a12b21 a11b12 + a12b22 a11b13 + a12b23

a21b11 + a22b21 a21b12 + a22b22 a21b13 + a22b23

]




IA = AI = A

0A = A0 = 0

AB 6= BA

(α + β)A = αA + βA for scalars α and β

α(A + B) = αA + αB

(αβ)A = α(βA)

α(AB) = (αA)B = A(αB) = (AB)α

(AB)C = A(BC)

A(B + C) = AB + AC

(A + B)C = AC + BC

(A + B)(C + D) = AC + AD + BC + BD

(AB)′ = B′A′

(ABC)′ = C′B′A′



Trace

The trace of a matrix is a very simple operation de�ned only for squarematrices.

For an (n X n) matrix A, the trace of a matrix A, denoted tr(A), is thesum of its diagonal elements. Mathematically,

tr(A) =n∑

i=1

aii

Properties:

tr(In) = n

tr(A′) = tr(A)

tr(A + B) = tr(A) + tr(B)

tr(αA) = αtr(A) for any scalar α

tr(AB) = tr(BA)Dr. Matthias Op�nger Econometric Methods WS 2015/16 15 / 43


Inverse

The notion of a matrix inverse is very important for square matrices.

An (n X n) matrix A has an inverse, denoted A−1, provided thatA−1A = In and AA−1 = In. In this case, A is said to be invertible ornonsingular. Otherwise, it is said to be noninvertible or singular. If aninverse exists, it is unique.

Properties:

(αA)−1 = (1/α)A−1 if α 6= 0 and A is invertible.

(AB)−1 = B−1A−1 if A and B are both n X n and invertible.

(A′)−1 = (A−1)′

(A−1)−1 = A


Review of Matrix Algebra Linear Independence and Rank of a Matrix

Linear Independence

Let {x1, x2, . . . , xr} be a set of n X 1 vectors. These are linearlyindependent vectors if, and only if,

α1x1 + α2x2 + . . .+ αrxr = 0 (*)

implies that α1 = α2 = . . . = αr = 0.

If (*) holds for a set of scalars that are not all zero, then {x1, x2, . . . , xr} islinearly dependent. At least one vector in this set can be written as a linearcombination of the others.


Review of Matrix Algebra Linear Independence and Rank of a Matrix

Rank of a Matrix

(a) Let A be an (n X K ) matrix. The column (row) rank of a matrix isthe maximum number of linearly independent columns (rows) of A.

(b) It can be shown that the column rank of a matrix always equals therow rank of a matrix. Therefore, it is enough to talk about rank of amatrix, denoted by rank(A).

Properties:

rank(A′) = rank(A)

rank(A′A) = rank(AA′) = rank(A)

rank(In) = n

If A is (n X K ), rank(A) ≤ min(n,K ).

An (n X K ) matrix A has a full rank, if rank(A) = min(n,K ).

A square matrix, which has a full rank, is de�ned to be a regular matrix;otherwise it is singular. A regular matrix is invertible.


Review of Matrix Algebra Quadratic Forms and Positive De�nite Matrices

Quadratic Forms

Let A be an (n X n) square matrix. The quadratic form associated withthe matrix A is the real-valued function b′Ab de�ned for all (1 X n)vectors b′ = [b1 b2 . . . bn]:

b′Ab = b′ [Ab] = [b1 b2 . . . bn]

a11b1 + a12b2 + . . .+ a1nbna21b1 + a22b2 + . . .+ a2nbn

...an1b1 + an2b2 + . . .+ annbn

= b1(a11b1 + a12b2 + . . .+ a1nbn)

+ b2(a21b1 + a22b2 + . . .+ a2nbn)

· · ·+ bn(an1b1 + an2b2 + . . .+ annbn)

=n∑

i=1

bi (ai1b1 + ai2b2 + . . .+ ainbn) =n∑

i=1

bi

n∑j=1

aijbj =n∑

i=1

n∑j=1

aijbibj



Positive De�nite and Positive Semi-De�nite Matrices

(a) A square matrix A is said to be positive de�nite if b′Ab > 0 for all(n X 1) vectors b except b = 0

(b) A square matrix A is said to be positive semi-de�nite if b′Ab ≥ 0for all (n X 1) vectors.



Positive De�nite and Positive Semi-De�nite Matrices

Properties:

Let A be an (n X K ) matrix:A′A and AA′ are positive semi-de�nite.Let A be an (n X K ) matrix with rank(A) = K :A′A is always positive de�nite and therefore nonsingular.Let A be a positive de�nite matrix:A−1 exists and is also positive de�nite.Let A be a positive de�nite (n X n) matrix and B an (n X K ) matrixwith rank(B) = K :the (K X K ) matrix B′AB is positive de�nite.Any positive de�nite (K X K ) matrix C has rank(C) = K .For any two regular matrices of the same order:A− B positive de�nite ⇔ B−1 − A−1 positive de�nite.Let A be a positive de�nite matrix. There is at least one regularmatrix B, such that B′B = A−1.


Review of Matrix Algebra Idempotent Matrices

Idempotent Matrices

Let A be an (n X n) symmetric matrix. Then A is said to be anidempotent matrix if, and only if, AA = A.

Properties:

rank(A) = tr(A)

A is positive semi-de�nite.

We can construct idempotent matrices very generally. Let X be an(n X K ) matrix with rank(X) = K . De�ne:

P ≡ X(X′X)−1X′

M ≡ In − X(X′X)−1X′ = In − P

Then P and M are symmetric, idempotent matrices with rank(P) = K andrank(M) = n − K .


Review of Matrix Algebra Di�erentiation of Linear and Quadratic Forms

Di�erentiation of Linear and Quadratic Forms

For a given (n X 1) vector a, consider the linear function de�ned by

f (x) = a′x

for all (n X 1) vectors x. The derivative of f with respect to x is the(1 X n) vector of partial derivatives, which is simply

∂f (x)/∂x = a

For an (n X n) symmetric matrix A, de�ne the quadratic form

g(x) = x′Ax

Then,

∂g(x)/∂x = 2x′A

which is a (1 X n) vector.Dr. Matthias Op�nger Econometric Methods WS 2015/16 23 / 43

Review of Probability

Moving on to ...

1 Review of Matrix AlgebraBasic De�nitionsMatrix OperationsLinear Independence and Rank of a MatrixQuadratic Forms and Positive De�nite MatricesIdempotent MatricesDi�erentiation of Linear and Quadratic Forms

2 Review of ProbabilityRandom variables and Probability DistributionsExpected Values, Mean and VarianceTwo Random VariablesThe Normal DistributionRandom Sampling and the Sample Distribution


Review of Probability Random variables and Probability Distributions

Random variables and Probability Distributions

Most aspects of world around us have an element of randomness, Thetheory of probability provides mathematical tools for quantifying anddescribing this randomness.

The mutually exclusive potential results of a random process are called theoutcomes.

The probability of an outcome is the proportion of the time that theoutcome occurs in the long run.

The set of all possible outcomes is called the sample space. An event is asubset of the sample space, that is, an event is a set of one or moreoutcomes.

A random variable is a numerical summary of a random outcome.

discrete random variable takes on only discrete set of values.

continuous random variable takes on a continuum of possiblevalues.




Example:

Let u1, u2 and u3 be random numbers.

A random experiment: A die is rolled once.

u1 ="The number of dots on the face which turns up"

has six possible outcomes. Each outcome has a probability of 1/6.

u2 ="The sum of the numbers of dots when a die is rolled twice"

has eleven possible outcomes (N=11).

Probability of outcome 2: f (2) = (1/6)(1/6) = 1/36

Probability of outcome 3: f (3) = 2(1/6)(1/6) = 2/36

u3 =Ä real number in the interval [0,1]"




The probability distribution of a discrete random variable is the list of allpossible values of the variable and the probability that each value willoccur. These probabilities sum to 1. Probability is summarized by aprobability density function in the case of continuous random variables.

Abbildung : Probability distribution for a discrete (a) and for a continuous variable (b)

The probability of an event can be computed from the probabilitydistribution. The cumulative probability distribution is the probabilitythat a random variable is less than or equal to a particular value.


Review of Probability Expected Values, Mean and Variance

Expected Values

The expected value of a random variable u, denoted E (u), is the long-runaverage value of the random variable over many repeated trials oroccurrences.

The expected value of a discrete random variable is computed as aweighted average of the possible outcomes of that random variable, wherethe weights are the probabilities of that outcome.

E (u) =N∑i=1

f (ui )ui

The expected value of u is also called the expectation of u or the meanof u and is denoted by µu.


Review of Probability Expected Values, Mean and Variance

The Standard Deviation and Variance

The variance and standard deviation measure the dispersion or theÿpreadöf a probability distribution.

The variance of a random variable u, denoted var(u), is the expectedvalue of the square of the deviation of u from its mean:

var(u) = E [(u − µu)2] =N∑i=1

f (ui )(ui − E (u))2

The standard deviation is the square root of the variance:

sd(u) =√

var(u)


Review of Probability Two Random Variables

Joint and Conditional Distributions

Example: A die is rolled once.

u1 ="The number of dots on the face which turns up"

u4 ="The number of natural numbers by which the number of dots on thedie is divisible"

Random Variable Outcomeu1 1 2 3 4 5 6u4 1 2 2 3 2 4

Tabelle : Outcomes of random variables u1 and u4




The distribution of a random variable (u1) conditional on another randomvariable (u4) taking on a speci�c value is called the conditionaldistribution of u1 given u4, denoted f (u1i |u4j).

f (u1 = 3|u4 = 2) = 1/3

f (u1 = 1|u4 = 2) = 0

The joint probability distribution of two discrete random variables, sayu1 and u4, is the probability that the random variables simultaneously takeon certain values, say i and j ; denoted f (u1i , u4j).

In our example, there are 6 X 4 = 24 possible outcomes for the jointdistribution of u1 and u4.

f (u1i , u4j) = f (u1i |u4j).f (u4j) = f (u4i |u1j).f (u1j)




f (u1i , u4j) = f (u1i |u4j).f (u4j) = f (u4i |u1j).f (u1j)

f (u1i = 3, u4j = 2) = f (u1i = 3|u4j = 2).f (u4j = 2) = 1/3.1/2 = 1/6

u1i1 2 3 4 5 6

1 1/6 0 0 0 0 0u4j 2 0 1/6 1/6 0 1/6 0

3 0 0 0 1/6 0 04 0 0 0 0 0 1/6

Tabelle : Joint distribution of random variables u1 and u4



Independence

Two random variables are independently distributed, or independent, ifknowing the value of one of the variables provides no information about theother .

u1 and u2 are independent if the conditional distribution of u1 given u2equals the marginal distribution of u1:

f (u1|u2) = f (u1)

Example: Two dice are rolled once simultaneously.

u1 ="The number of dots on the face which turns up by the �rst die."

u1 ="The number of dots on the face which turns up by the second die."



Independence

Using the de�nition of joint distribution:

f (u1, u2) = f (u1|u2).f (u2)

gives an alternative expression for independent random variables.

If u1 and u2 are independent, then

f (u1, u2) = f (u1).f (u2)

Example: Two dice are rolled once simultaneously.

f (u1 = 2, u2 = 5) = f (u1).f (u2) = 1/6 ∗ 1/6 = 1/36



Covariance

One measure of the extent to which two random variables move together istheir covariance.

The covariance between u1 and u2 is

cov(u1, u2) = E [(u1 − E (u1))(u2 − E (u2))]

cov(u1, u2) =

N1∑i=1

N2∑j=1

f (u1i , u2j)[(u1i − E (u1))(u2j − E (u2))]

Covariance is a measure of a linear relationship between two randomvariables.

If tend to move in the same (opposite) direction, the covariance is positive(negative).

If u1 and u2 are independent, then the covariance is 0.Dr. Matthias Op�nger Econometric Methods WS 2015/16 35 / 43


Correlation

The units of covariance are, awkwardly, the units of u1 times the units ofu2. This ünits"problem can make numerical values of the covariancedi�cult to interpret.

The correlation is an alternative measure of dependence between tworandom variables that solves the ünits"problem.

The correlation between u1 and u2 is

cor(u1, u2) =cov(u1, u2)√var(u1)var(u2)

=cov(u1, u2)

sd(u1)sd(u2)

The correlation is always between -1 and 1, and is usually represented by aρ.

−1 ≤ ρ ≤ 1



Means, Variances and Covariances of Sums of Random

Variables

Let u1 and u2 be two random variables and let x1 and x2 be constants. Thefollowing facts follow from the de�nition of mean, variance and covariance:Mean:

E (x1) = x1

E (x1.u1) = x1.E (u1)

E (u1 + u2) = E (u1) + E (u2)

E (x1 + x2.u2) = x1 + x2.E (u2)

E (E (u1)) = E (u1)

As a general rule: E (u1.u2) 6= E (u1).E (u2)

Only if the two random variables are uncorrelated or independent:

E (u1.u2) = E (u1).E (u2)




Variables

Let u3 be another random variable. De�ne

u3 = x1.u1 + x2.u2

then, variance of u3 is:

var(u3) = x21 .var(u1) + x22 .var(u2) + 2x1x2cov(u1, u2)

For the special case u1 = 1 (var(u1) = 0 and cov(u1, u2) = 0):

u3 = x1 + x2.u2

then, variance of u3 is:

var(u3) = x22 .var(u2)




Variables

Let u4 be another random variable.

Covariance between u3 and u4:

cov(u3, u4) = cov(x1.u1 + x2.u2, u4) = x1cov(u1, u4) + x2cov(u2, u4)

Covariances are always symmetric:

cov(u3, u4) = cov(u4, u3)


Review of Probability The Normal Distribution

The Normal Distribution

A continuous random variable with a normal distribution has the familiarbell-shaped probability density.A normally distributed random variable u with an expected value E (u) anda variance var(u) can be expressed by

u ∼ N(E (u), var(u))

The standard normal distribution is the normal distribution with anexpected value E (u) = 0 and a variance var(u) = 1, denoted N(0, 1).Random variables that have a N(0, 1) distribution are often denoted by Z .

To compute probabilities for a normal variable with a general mean andvariance, it must be standardized �rst subtracting the mean, then dividingthe result by the standard deviation.

z =u − E (u)

sd(u)


Review of Probability Random Sampling and the Sample Distribution

Random Sampling and the Distribution of the Sample

Average

We are going to deal with various ÿamples of data"during our course.

Random sampling is the act of randomly drawing a sample from a largerpopulation. This has the e�ect of making the sample average itself arandom variable.

Assume, for a random variable x, we have a random sample of Tobservations: x1, x2, . . ., xT .

The sample mean is:

x =1T

T∑t=1

xt



Random Sampling and the Sample Distribution

The sample variance is:

var(x) =1

T − 1

T∑t=1

(xt − x)2 =1

T − 1Sxx

where Sxx =∑T

t=1(xt − x)2.

The sample standard deviation is then:

sd(x) =√

var(x)



Random Sampling and the Sample Distribution

Assume, we have a random sample of T observations for random variablesx and y : (x1, y1), (x2, y2), . . ., (xT , yT ).

The sample covariance of x and y is :

cov(x , y) =1

T − 1

T∑t=1

(xt − x)(yt − y) =1

T − 1Sxy

where Sxy =∑T

t=1(xt − x)(yt − y)

Finally, the sample correlation coe�cient between x and y follows

cor(x , y) =cov(x , y)

sd(x)sd(y)=

Sxy/(T − 1)√Sxx/(T − 1)

√Syy/(T − 1)

=Sxy√

Sxx√

Syy


Documents

Econometric Methods - Uni Trier€¦ · Econometric Methods Revision: Matrix Algebra and Probability Dr. Matthias Op nger Lehrstuhl für Finanzwissenschaft WS 2015/16 Dr. Matthias