vec space def and ex
- set which allows linear combination of vectors to form another vector in the set
basis vecotrs def
- vectors which span vec space
linear combination
linear span
- all possible linear combination
linear transformation
- L(ax+by)=aL(x)+bL(y)
centering operation as idempotent operation
- I-(1/n)ii'
tr(ABC)=tr(BCA)=tr(CAB)
rank(A)=rank(A')
partitioned matrices
- make sure dimension consistency
vec operator
- transform a matrix into vector
In linear algebra, a symmetric (n x n) real matrix M is said to be positive definite if the scalar z'Mz is positive for every non-zero column vector z n real numbers.
symmetric with positive eigenvalues is positive definite
Hessian matrix :second derivative symmetric matrix
symmetric matrix has real eigenvalues and real orthogonal eigenvectors
AC=C$\Lambda$
-
spectral composition symmetric A=C$\Lambda$C'
-
diagonalize symmetric A, CAC' =
$\Lambda$ -
rank, tr, det is the same as A
-
easy to multiply
for non-singular square matrix
-
$A^{-1}$ exist -
det(A) != 0
-
columns independent
-
full rank
===
probability space
-
sample space S
- set of elements
-
power set (
$\sigma$ field)- set of all subset of S
-
probability measure
- P: power set -> [0,1]
independence
- f(x,y) = f(x)·f(y)
no correlation
-
cov(x,y) = 0 or corr(x,y) = 0
-
indep -> var = 0
moment generating func
-
$E(e^{tx})$ -
E(x) = M'(0), E(x^2) = M"(0)
normal distribution
f(x1,x2)=...
===
good estimator
-
unbias
-
efficiency
-
var(theta1) < var(theta2)
-
exist cramer-rao low bound
-
-
sufficiency: statistics can convey same information as parameter
-
asymptotic consistency： convergence in probability
-
asymptotic normality
method of estimation
-
LSE
-
GMM: combine 2nd and 4th moment
-
MLE: choose theta to max L(theta, x) = joint pdf of f(x;theta)
===
parametric statistical steps
- set model x~f(x;theta)
- sample
- estimate
- test