Machine learning

Introduction

We crossed an important boundary in moving from the previous chapters to here. Much of the book so far has focused on systems whose behaviour is specified explicitly: algorithms, protocols, instruction sets, compilers, and data structures. Machine learning adds a different style of computation. The underlying computer is still a deterministic machine following exact instructions, but the behaviour we care about is now described statistically. Instead of reasoning mainly from logic and formal rules, we start reasoning from data, uncertainty, and probability. The cost is that we often lose some interpretability, and we may lose reproducibility unless the data, randomness, and software environment are carefully controlled, but we gain capabilities such as spam filtering, image recognition, and language understanding that are hard to build with hand-written rules alone.

The umbrella term for this new paradigm is machine learning. Instead of specifying the behaviour entirely with hand-written rules, we provide examples as data and let a learning algorithm fit the pattern. Instead of telling the computer how to do what we want, we show it many data samples and let it figure out the patterns itself. The computer learns from data rather than from explicit instructions.

At first this can sound like magic. How can a computer “learn” anything if it’s still just following instructions? The answer is that the instructions are different. Instead of saying “here is exactly how to do the task”, they say “adjust yourself to get better at the task”. What looks like learning is the result of applying that idea at scale.

Machine learning now powers much of the modern web. Recommendations, fraud detection, search ranking, and spam filtering all use it. For software engineers, some understanding of ML is now part of the job. You may need to integrate an ML-powered feature, decide whether a problem is suitable for ML, or debug why a model is behaving unexpectedly.

The next three chapters build on each other. This chapter lays the foundation. We’ll examine what machine learning is and how it differs from traditional programming. We’ll look at several important algorithms and when to use each. We’ll cover training, evaluation, and the pitfalls that trip up practitioners. We’ll also discuss deployment, monitoring, and the ethical concerns that come with real systems. Throughout the chapter, we’ll keep returning to two running examples: a spam filter for classification