Machine Learning - Part 1: Overview

I plan to start a tutorial series on Machine Learning. It is a topic I am interested in and I guess this will help others too.

The series will cover introductory topics and no machine learning background is assumed. Some basic familiarity with linear algebra is assumed. I'll also cover some programming examples using GNU Octave. If you are not familiar with it, a good tutorial is available here.

Definition

So, what is machine learning? It is a broad area and presents some challenge to come up with a good definition. It'll be easier to go through an example to understand the kind of problems solved by machine learning and then come up with a definition for it.

Consider the game of checkers. It is a board game with relatively simple rules. There were a lot of attempts - as early as in 1950s - to develop algorithms to make computers play checkers.

So how can we teach a computer to play such a game? One simplistic approach is to record every single board position and the corresponding best move the computer should play. However, this will soon prove to be tedious because there are too many board positions and it may not be possible to store all of them in memory. An improved version of this would be to record "patterns", group similar board positions in to one pattern and the program knows the best strategy for each pattern. However, even this approach would prove tiresome in more complex games because there are just too many patterns and it may not be possible for a human to list all of them correctly, in any case it will take a huge effort.

Arthur Lee Samuel was one of the pioneers in artificial intelligence and he found a way to tackle this problem. Instead of explicitly programming the computer to handle every single move in the game he allowed it to "learn" itself. He let the program play thousands of games against itself and understand the hidden patterns/features of the game. The program got better and better and eventually reached a respectable skill level to challenge human players. This approach is closer to how humans learn to play such games. Instead of remembering every single board position, we understand recurring patterns in the game and strategies to handle them. Arthur Samuel defined machine learning as a "Field of study that gives computers the ability to learn without being explicitly programmed".

A more formal definition was given by Tom M. Mitchell: "A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E". Thus the machine learning program becomes better and better in performing tasks as it learns from experiences.

Put it differently, let us say we want to perform some task using a computer. A machine learning program will come up with an algorithm to perform that task. The initial algorithm given by the machine learning program will not perform so well. But as we feed more "experiences" (whatever that means, we'll see more on this later) to the machine learning program it will tune the output algorithm better and better to achieve a higher performance.

Some applications of machine learning are:

• Medical diagnosis
• Fraud detection
• Speech and handwriting recognition
• Recommender systems
• Spam detection
• Autonomous driving

I will discuss some of these applications in detail in future posts.

Hopefully, this post gave you a high level overview of machine learning. I'll explain classification of machine learning algorithms in the next post.