Operating systems

Introduction

What is an operating system?

An operating system is the software layer between applications and hardware. It starts the machine, runs programs as processes, manages memory, files and devices, and enforces security boundaries.

That is a lot of jobs for something most of us only notice when it starts updating itself at exactly the wrong time.

In the previous chapter, we learned how a real computer is put together. You may be forgiven for thinking that it still seems very far from how you actually use a computer. That’s because there’s a whole layer sitting between the hardware and you: the operating system (OS).

The OS provides solutions to some tricky problems. How does the processor know where to begin executing code or where to find the next program? How do you change between running programs? How do I stop an evil program locking me out of the entire system? To handle these issues we need hardware and software to work very closely together. The software side of the solution is the OS. It’s the first program that runs on the computer and it acts as an interface between the user and the hardware. It abstracts away many of the details and peculiarities of different hardware components and provides a consistent platform on which other programs can run. It makes the computer easier to use, more secure and more performant.

Not giving user programs direct access to the hardware is a very good thing. Imagine writing a program where you had to handle the cursor yourself. You’d need to check for and identify any attached mice or trackpads, listen for and correctly interpret the hardware signals generated, recognise different types of clicks, gestures, taps and so on. It would be a nightmare! It’s much better for everyone concerned if the OS can take care of all this and simply feed the user program a stream of notifications about where the cursor is and what it is doing.

Modern OSes include Microsoft Windows, Apple’s macOS and GNU/Linux. They are all immensely complex conglomerations of software components that can handle a huge range of tasks. In this chapter, we’ll try to pare things down to the core commonalities. We’ll see how a computer starts up. Then we’ll briefly review some necessary additions to our hardware model. We’ll examine the implications they have on the OS architecture. Then we’ll look at the core abstractions an OS provides to manage the key system components: the processor, memory and storage. In each case, the OS provides a virtual representation of the underlying resource.

Common operating systems

There have been many operating systems over the years. Let’s look at a few of the most popular and how they differ from each other.

Windows is a family of closed source operating systems developed by Microsoft. The very first Windows was not much more than a visual interface on top of the older MS-DOS (Disk Operating System). DOS is interesting as a salutary tale because it was a very simple OS that lacked a lot of what we’d now consider to be standard features. It gave user programs more or less complete control over the hardware and relied on them to behave themselves. This worked as well as you might expect. Later versions of Windows moved away from DOS entirely in a bid to improve its terrible reputation for reliability and security. Windows has traditionally focused on providing an easy to use interface for non-technical users, backwards compatibility and supporting a wide range of hardware.

Unix was an OS developed in the 1970s at Bell Labs that spawned a whole family of derivatives and reimplementations. “Unix” nowadays is more like a description of how to design an OS, formalised in a specification known as POSIX. Strictly speaking Linux refers to an open source reimplementation of the original Unix kernel, dating from the early 1990s. Many of the standard Linux system utilities were provided by a separate project, GNU (a recursive acronym of GNU’s Not Unix), and so you’ll sometimes see the complete OS referred to as GNU/Linux. There are many distributions (or distros) consisting of the Linux kernel packaged with a variety of extensions and user programs to form a complete operating system. Ubuntu, CentOS and Arch are examples of popular Linux distros, each targeting a different use case. Of course, the world has moved on a lot since the 1990s and Linux has continued to develop and add new features that never existed in Unix. Linux is popular with programmers and technically minded users because it offers complete control over the system to those who are prepared to learn how to use it. Being open source means that the user can read and, if they feel up to it, modify the operating system’s code. It’s basically the opposite of Windows. Linux is extremely popular on servers due to its reliability, performance and tweakability.

macOS (previously OS X) is the closed source, proprietary OS that runs on Apple computers. It is built around an open source, Unix-like core called Darwin, which is itself based on a branch of the Unix family known as the Berkeley Software Distribution (BSD). The main distinction between BSD and Linux is that the BSD licence allows it to be used in closed source projects, unlike Linux. On top of this open source foundation, Apple has built a large amount of proprietary, closed source code that makes macOS distinct from Linux. Because they share the same Unix-like fundamentals, Linux and macOS have many similarities. Despite that, programs built to run on Linux will not run on macOS (and vice versa) because the code on top of the common base differs too much. The combination of Unix-like foundation and good user interface makes Apple products popular with developers. Some will tell you that real developers use Linux. Ignore them.

The boot process

To understand why we want an operating system, we’ll start at the very beginning. Let’s imagine that we’ve built a computer like the one described in the architecture chapter. It’s basic but sufficient to run programs and do useful work. How do we actually get it to start running a program? Let’s think about what we’ve learned already. We know that we need to load the program instructions into memory and set the processor’s program counter to the address of the first instruction. The processor will then start fetching and executing our program.

How do we actually get the instructions into memory and how do we tell the processor where to begin? Both of these puzzles can be solved easily if we’re willing to make changes to the hardware. We can write our program to a piece of read-only memory (ROM) and wire it up so that it lives at a fixed memory address. ROM is memory that can only be written once. The benefit is that it doesn’t need electricity to maintain the data. Our program will persist in the ROM even when the machine is powered off. We can then hardcode the ROM start address into the processor’s program counter. Now every time we power on the processor, it will start running our application!

Many simple microcontrollers work like this. Though workable, we have a few problems. For one thing, how do we change the running program? We could load different programs on to different ROMs and just physically switch them out. This is what old games consoles used to do but clearly this won’t work for general computing. A better idea would be to put a special program on the ROM that would let the user select what they wanted to run. It would then load the chosen program into memory and tell the processor where to start executing. What we’ve just invented is a bootloader.

When a computer boots, or starts up, it is hardwired to first execute code stored in a small piece of ROM. Previously, this would have been the BIOS (basic input/output system) but EFI (extensible firmware interface) is more common nowadays. The distinction isn’t terribly important because both perform the same role: work out what hardware the system has and initialise it, find the OS’s code, load it into memory and start executing it.

Every operating system includes a bootloader which is stored on the computer’s disk at the location specified by the BIOS/EFI standard. The bootloader’s sole job is to get the OS up and running. Due to various terribly tedious technical constraints imposed on bootloaders over the years, the modern process actually involves the first bootloader calling a second, more powerful bootloader that then loads and starts the OS. Happily, there’s no need to know all the arcane details as long as you get the general concept.

The boot process begins by the computer powering up. The BIOS/EFI runs and then hands control over to the bootloader, which loads the OS. Finally, the boot process is complete when the OS has finished initialising and is ready to receive user input.

Interrupts: hardware support for software

Under the processing model we developed in the previous chapter, there’s no obvious way to force a program to relinquish control of the processor. Obviously, switching programs is something that it would be convenient to be able to do. There are important security implications too. If I can’t stop a running program, I have no way to stop a program that’s trapped in an infinite loop from freezing the whole system. Similarly, I can’t intercept attempts by an evil program to reformat the hard disk or steal my passwords.

The problem is that our processor gives the running program complete control. The program can use that control to prevent any attempts to transfer control. Frankly, the program can do whatever it likes, up to and including destroying the system. One solution could be to only ever allow the OS to run directly on the processor. The OS would load other programs, analyse their instructions, pass them through to the processor if they looked safe and flag them if they seemed dangerous. This approach could work but would entail a high performance cost. Every user program instruction would need many OS instructions to analyse and verify its safety.

A simple hardware modification, the interrupt, means that user code can run directly on the processor without the OS completely relinquishing control. An interrupt is a hardware signal that causes the processor to stop what it’s currently doing and switch to executing a chunk of code known as an interrupt handler. The processor circuitry is amended to trigger an interrupt on a fixed timer or whenever a program attempts to perform a potentially risky operation. The OS sets up handlers for each interrupt. The handlers will examine the cause of the interrupt and take corrective action if need be.