How your operating system works
20th Mar 2011 | 12:00
It sits there quietly but what does it actually do under the hood?
How your operating system works
When we use a PC, we're usually only concerned with the program we're currently using, whether it's a browser, a word processor or our preferred social networking app. We don't often think about the rather extensive program underlying everything that happens on the PC: its operating system.
We can fire off the name of our chosen OS at the drop of a hat (Windows 7, Chromium or Ubuntu, for example), but could we say just what it is and what it does?
When we think of an operating system at all, it's usually to help define what our PC is - 2.66GHz Core 2 Duo, 4GB RAM, 256GB SSD, Windows 7, for example. It's almost like a physical peripheral; something we could swap for a similar substitute in the future.
And until now, the operating system defined a PC just like the hardware. We've grown up with large, all-encompassing operating systems that provided a wealth of services, but that might be on the cusp of changing.
Let's start by considering what a PC is. It's a collection of hardware: the motherboard, CPU, memory chips, video adaptor, hard disk (or SSD), optical drive, keyboard, mouse, screen and so on. Each of those pieces of hardware is, in essence, interchangeable with others that perform the same functionality, but perhaps faster, more efficiently or more compactly.
With a desktop machine, you can pretty much replace the whole PC a bit at a time by upgrading each component. The modular nature of PCs sometimes seems almost miraculous (it's certainly not the case for cars - I can't just decide to pop a Ford engine into my Audi, for example) and it's only possible because the operating system smooths over any differences between components and hides them from us, usually through the use of specially written drivers.
In the beginning
Let's approach what an operating system does by considering what happens when you boot your PC. Pressing the power switch starts up a small program that's stored in read-only memory - a chip on the PC's motherboard.
This program, known as the BIOS (basic input/output system) on PCs and EFI (extensible firmware interface) on Macs, runs a set of routines intended to identify peripherals and system devices, and initialise their firmware. This is known as the POST (power-on self test) and, as its name suggests, is needed to check the proper functioning of the hardware.
POST routines, if the option is set, can also perform intensive checks like testing all installed memory, but the level of checking is usually curtailed for the sake of speed.
Both the BIOS and EFI are written for the particular motherboards they're found on. They have knowledge of the chipsets that are embedded in the motherboard and require changes in order to run on other systems. They're encoded on EEPROMs soldered onto the motherboard so they can be updated.
The BIOS is archaic - the archetypal legacy system, first implemented in the days of DOS PCs. Back then, you could access the BIOS directly from your applications using a well-known standard API (applications programming interface), circumventing DOS completely. You could almost view it as part of the operating system.
These days, despite the increased capability of modern PCs compared to their DOS forebears, the BIOS is still a 16-bit program running in 1MB of memory (the start point of the program is at address 0xFFFF0, 16 bytes below the 1MB mark).
EFI was designed as the successor to BIOS. It has better support for larger disks and the ability to boot from them, a much faster boot time (since it doesn't use backwards-compatible code to access chipsets and disks), 32-bit or 64-bit code, installable drivers and so on. Unfortunately for PC users, the BIOS maintains its stranglehold on our machines.
One of the final stages of the POST process is to identify and initialise the drives in the system. The BIOS has a table that defines the boot order for disks, and it starts looking through this table for a disk that contains an operating system.
To do this, it looks for the bootstrap loader on each disk in sequence. The BIOS doesn't know about filesystems (the filesystem is only loaded as part of the operating system). Instead, it only knows about sectors, heads and cylinders.
To find the bootstrap loader, the drive must be formatted in a peculiar way. In particular, the first 512-byte sector of the drive (found at cylinder 0, head 0, sector 1) has to be formatted as a master boot record (MBR).
Although small, the MBR has enough code to continue the loading of the installed operating system (each operating system has its own code), and also has a small table of partitions that the drive has been split into.
This table contains information about all the partitions on the drive. Since the MBR only contains enough room for the information on four partitions, that's what we're limited to for PC systems.
On the run
Once the BIOS finds an MBR, it loads it into memory and starts executing it. The MBR code identifies the primary active partition by looking at the partition table, reads the partition's boot loader (known as the volume boot record) from the disk into memory (the partition table stores where the VBR can be found), and starts executing it.
This second boot loader is the one that will load yet more sectors into memory, have more knowledge of the filesystem and continue the loading of the operating system. It will load the actual operating system boot loader (known as 'ntdlr' prior to Windows Vista and 'winload.exe' after that), and this boot loader will switch the CPU to 32-bit mode or, increasingly, to 64-bit mode.
At this point the BIOS is no longer used because Windows uses boot-level drivers in order to access the hardware.
For EFI, the process is largely similar, the difference being that EFI has knowledge of the filesystem and can load the boot loader directly. This is called BootX on a Mac. Once the operating system starts to boot, it initialises the various systems it moderates. These include the CPU, memory, on-board devices, persistence mechanisms, and app and user interfaces.
FIGURE 1:Major components in an operating system
For the CPU, the operating system not only switches it into 32-bit or 64-bit operation, but also virtualises the CPU for security and protection. The OS and its kernel drivers execute at the highest privilege at Ring 0, whereas normal user applications (such as the browser or a word processor) run at the lowest privilege in Ring 3 (there are four privilege or protection rings in modern Intel architecture, of which Windows uses only two).
The main reason for this is security; by ensuring that non-privileged programs can't corrupt the OS, either maliciously or accidentally, the system as a whole is made more stable.
By providing this level of protection, the OS can also abstract away certain physical limitations of the PC (say, the amount of memory) from user applications and then provide services like program swapping and multitasking, making the application easier to write and the overall system more efficient.
Managing apps, files and folders
The operating system also manages how much CPU time each running application has to ensure that it gets work done. Most modern PCs have multiple CPUs, but let's imagine we only have one.
Only one application or process can use that CPU at any one time. To create the illusion of many applications running simultaneously, the OS will switch rapidly back and forth between the current set of programs, giving each one a small timeslice in which to get some work done.
Once the current application has run for a certain amount of time (the time slice), or has been suspended waiting for a resource or some form of user input, the OS will save its current state (register values, memory, current execution point), load the saved state for the next application in line, and start executing it.
After it's completed its timeslice or has suspended, the next process gets a turn using the CPU to get work done. This round-robin scheduling continues as long as the OS is running.
Sometimes an interrupt will occur needing immediate attention, in which case the current process is interrupted, its state is saved and the interrupt is serviced.
Modern 32-bit and 64-bit operating system provide services that manage memory for user applications, including the virtualisation of memory. In essence, the memory layout for a user application looks exactly the same to each one.
To ensure that different programs don't interfere with each other's memory structures, the OS doesn't provide access to physical memory directly. Instead, it maps user-mode memory through mapping tables called descriptor tables to either real memory or to the swap file on disk.
This provides a great deal of flexibility for the operating system: it can move memory blocks around to accommodate other programs' memory requirements without the original program knowing; it can swap out memory blocks to disk if the program isn't being used and it can defer assigning (or committing) memory requests until the memory is written to.
This mapping of memory through descriptor tables also means the OS gives every user application the convenient fiction that it's the only application running on the system. No applications will clash by trying to use the same memory, and an application can't cause another to crash by writing to its memory space.
FIGURE 2:Mapping virtual memory to physical memory and the swap file with a descriptor table
Figure 2 shows two programs, A and B, both with the same view of their memory space. Program A has one block of memory allocated at a particular position in its memory space; in reality it's found somewhere else in physical memory via the descriptor table. Program B has two blocks allocated, one of which is found in the swap file for the system.
Files and folders
Another important virtualised service provided by the OS is the filesystem. Disk hardware works on logical block addresses (LBAs), which are essentially numbers that define the sector number from the beginning of the drive volume.
The disk controller hardware converts the LBA into physical parameters (such as head, track and platter) to find the actual sector. For SSD drives, the disk controller simply converts the LBA into a memory address (although the controllers on SSDs will move data blocks around to even out access to the flash memory unbeknownst to the OS).
The operating system hides these raw LBAs from user programs by imposing a hierarchical filesystem over the disk. The filesystem organises the physical disk sectors (each one usually being 512 bytes in size, although the market is starting to move towards 4kB sectors since most filesystems use that block size as a minimum allocation and granular size for a file) into files and directories.
The filesystem is responsible for maintaining the mapping between files and blocks, which blocks appear in which files (and in which order they appear), which directory a file is found in, and other similar services.
The filesystem virtualisation also means that user programs only need to worry about high-level operations with files and folders: creating new ones, deleting existing ones, adding data to the end of a file, reading and writing to files and enumerating the folder contents.
All the mapping between userfriendly names and LBAs is done by the operating system under the hood. To the user program, a file is just a contiguous set of bytes somewhere on the disk and it doesn't have to work out that the file consists of a block over here, followed by that one over there.
This filesystem abstraction points to another set of services provided by the operating system: the application programming interfaces (also know as APIs).
These are plug-in points that let user programs like browsers and word processors to take advantage of various services exposed by the operating system. These include APIs for memory management, file and folder management, network management, user input (keyboard and mouse), the windowing user interface, multimedia (video and audio) and so on.
In all cases, the API provides a standardised way for user applications to obtain and use resources from the PC, no matter what hardware was actually present. So, for example, a user program doesn't have to know anything about which video adaptor or screen the PC is using in order to display something on it. It merely makes calls to the standard API ('draw a window here of this size') and the adaptor and screen drivers translate those standard requests to calls to the hardware that provide the required result.
That is perhaps the last part of the operating system story. It isn't a monolithic program, written to work with every single piece of hardware out there. It is instead a framework into which hardware-specific drivers are plugged.
These drivers know how to access their particular hardware, can translate between standard function calls and the requirements of the device, and are written to use the operating system's APIs.
First published in PC Plus Issue 305
Liked this? Then check out How motherboards are made: a miracle of modern electronics
Sign up for TechRadar's free Weird Week in Tech newsletter
Get the oddest tech stories of the week, plus the most popular news and reviews delivered straight to your inbox. Sign up at http://www.techradar.com/register