Machine learning: Supervised methods, SVM and kNN

In supervised learning, a set of input variables, such as blood metabolite or gene expression levels, are used to predict a quantitative response variable like hormone level or a qualitative one such as healthy versus diseased individuals. We have previously discussed several supervised learning algorithms, including logistic regression and random forests, and their typical behaviors with different sample sizes and numbers of predictor variables. This month, we look at two very common supervised methods in the context of machine learning: linear support vector machines (SVM) and k-nearest neighbors (kNN). Both have been successfully applied to challenging pattern-recognition problems in biology and medicine.