Eigenfaces inScalaA quick walkthrough and sample implementation of the Eigenfaces algorithm in Scala.
Justin Long
What is Eigenfaces?
Need to do quick and dirty object or facial recognition? Eigenfaces may be for you.
The essence of Eigenfaces
Eigenfaces is the name given to a set of eigenvectors when they are used for facial recognition.
A typical use for calculating Eigenfaces works as such:
1. Obtain a training set of faces and convert to a pixel matrix
2. Compute the mean image (which is an average of pixel intensity across each image).
3. Compute a differential matrix by subtracting the mean from each training image, pixel by pixel
4. Compute covariance matrix of the differential matrix
The essence of Eigenfaces
Your “average face” may look a little uncanny…
4. Calculate eigenvectors from covariance matrix
5. Compute Eigenfaces by multiplying eigenvectors and covariance matrices together, and normalizing them
Putting it together
def computeEigenFaces(pixelMatrix: Array[Array[Double]], meanColumn: Array[Double]): DoubleMatrix2D = {
val diffMatrix = MatrixHelpers.computeDifferenceMatrixPixels(pixelMatrix, meanColumn)
val covarianceMatrix = MatrixHelpers.computeCovarianceMatrix(pixelMatrix, diffMatrix)
val eigenVectors = MatrixHelpers.computeEigenVectors(covarianceMatrix)
computeEigenFaces(eigenVectors, diffMatrix)
}
Multiplying eigenvectors/differential
(0 to (rank-1)).foreach { i =>
var sumSquare = 0.0
(0 to (pixelCount-1)).foreach { j =>
(0 to (imageCount-1)).foreach { k =>
eigenFaces(j)(i) += diffMatrix(j)(k) * eigenVectors.get(i,k)
}
sumSquare += eigenFaces(j)(i) * eigenFaces(j)(i)
}
var norm = Math.sqrt(sumSquare)
(0 to (pixelCount-1)).foreach { j =>
eigenFaces(j)(i) /= norm
}
}
Preprocessing is key
You need to preprocess your images!
- Grayscale: important for calculating proper pixel intensity values
- Normalization: very helpful if your images were taken in different lighting conditions
- Cropping: Very important to focus only on facial features
Without preprocessing, you’re gonna have a bad time.
Potential for Eigenfaces?
Facial recognition is most obvious use.
Eigenfaces and eigenvectors are also useful for other applications of computer vision:
- Subjects that can be read 2-dimensionally, from same angle
- Optical Character Recognition (OCR)
- Image segmentation
- http://www.cs.huji.ac.il/~yweiss/iccv99.pdf
Tinderbox used Eigenfaces
What happened when I got fed up with all the swiping involved in Tinder? I automated it with Eigenfaces (a slightly modified strategy).
- Added a simple step of averaging all faces together for yes/no faces
- The “average” face was used for Eigenfaces selection (k-nearest neighbor between 2 models)
A quick look at Tinderbox
http://crockpotveggies.com/2015/02/09/automating-tinder-with-eigenfaces.html