Machine learning for package users with R (4): Neural Network

These days almost everybody appears to love a variation of Neural Network (NN) -- Deep Learning. I already argued about how Deep Learning works and what kind of parameters characterizes it in the previous post.

Machine learning for package users with R (3): Support Vector Machine

Actually support vector machine (SVM) is the one that I love the most among various machine learning classifiers... because of its strong generalization and beautiful decision boundary (in high dimensional space). Although there are other better classifier than SVM, every time I can't help trying it on any data. :)

At any rate, I believe SVM is one of the most widely used machine learning classifier all over the world. It's advantage is: 1) high performance but with a long history, 2) a lot of well-supported libraries or packages of SVM, and 3) highly efficient performance on linearly non-separable patterns.

About 1), I have to emphasize SVM has already more than 20 years history. Can you believe SVM first appeared even before the 1st iPod???*1 There are a plenty of accumulated knowledge about its application and possible problems. 2) is the most important I think because now we have LIBSVM, LIBLINEAR, Kernel Lab, Weka or scikit-learn with implementation of SVM. Some of them can be imported to multiple platforms*2. Finally and needless to say, 3) means its great advantage derived from the kernel method.

*1:SVM got to be widely used after Platt's excellent work in 1999, but the 1st iPod was released in 2001

*2:LIBSVM can be used with R as {e1071}, Python, Java, Matlab, C or C++

Machine learning for package users with R (2): Logistic Regression

I think a lot of people love logistic regression because it's pretty light and fast. But we know it's just a linear classifying function -- I mean it's only for linearly separable patterns, not linearly non-separable ones.

It's primary idea is simple: fitting binomial dependent variable with logit function. But its advantage is great even though mainly it works only for linearly separable patterns.