-
Inductive Bias : It is a set of assumptions learner uses to predict results for unseen inputs. Every machine learning algorithm that is trained for generalization purpose has inductive bias. For example, linear regression model assumes that output Y is linearly dependent on input X. This introduces bias in the training called inductive bias. Similarly for SVM, inductive bias is: the classes are separated by large margin. Please refer to website for knowing inductive biases of other machine learning algorithms.
-
Self Organizing Maps :
-
Self organizing maps(SOM) are special type of neural network used for unsupervised feature learning. SOM falls into the category of non-linear dimensionality reduction methods.
-
Unlike normal neural network that apply error-correcting learning algorithm(e.g. backpropagation with gradient descent), SOM apply competitive learning algorithms.
-
It maps multi-dimensional data to 2D grid. Because of competitive learning algorithm, only one neuron gets fired for a given input. If two examples are similar, then the same neuron or nearby neurons are activated. Hence SOM are used for finding structure in the data.
-
SOM learns lower dimensional representation such that the points that are nearby n high-dimensional space are also nearby in SOM. I.e. SOM are topology-preserving algorithm, which makes it suitable for dimensionality reduction tasks.
- Dirichlet Energy/Laplacian energy :
- It is a measure of how variable a function is. It is used as a regularization term in matrix completion problem by Kalofolias et al in their paper on Matrix Completion on Graphs.
- Isotropic Gaussian :
- The Gaussian distribution in which all dimensions are independent and has same variance is called Isotropic Gaussian. Such Gaussian distributions are circular/spherical. The variance matrix
$\Sigma$ of Isotropic Gaussian is given by scalar variance$\sigma^2$ multiplied by identity matrix. Hence,$\Sigma = \sigma^2 I$ .
- Lanczos algorithm:
- It is an algorithm to find the eigenvalues and eigenvectors of nth order linear system with limited number of operations m, such that m << n. This algorithm uses adaptation of power iteration method for finding eigenvectors and eigenvalues.