The limited direct execution protocol is a protocol that the OS implements in order to let the processes execute with performance (the direct part) but with a mechanism in order for the OS to retain control (the limited part). Without control a process could take over the machine and access information that it shouldn’t be allowed to access.

The “Direct Execution” part

Ignoring the “limited” part, the “direct execution” of a process consists in running it directly on the CPU. This is done (by the OS) by creating a process entry in a process list, allocating some memory pages for it, loading the program code from disk into memory, locate the entry point (usually main()), jumping to it and executing the code.

The “Limited” part

There are two major problem to this approach:

  1. How can the OS know if the program isn’t doing anything malicious, while still running efficiently?
  2. If the program is running on the CPU and the OS isn’t, how can the OS stop the program from running and switch to another process? This is what the “limited” part of the protocol tries to solve.

Problem 1: Restricted Operations

The first problem is solved by introducing two modes:

  1. User mode: in this mode, the process is restricted in the action it can perform. For example, it cannot issue I/O requests (if it tries, the process will be killed).
  2. Kernel mode: in this mode, the process has no restriction at all. This is the mode the OS is running in. We cannot run the process always in kernel mode, otherwise it may perform malicious operations (erase the whole disk for example). To solve this, most hardware provide the programs to perform a system call^[See Process APi in C]. A system call exposes the program to key pieces of functionality to user programs, like accessing the file system, creating and destroying processes, communicating with other processes etc.

When a program execute a system call, it executed a trap instruction, which raises the process’ privilege to kernel mode, executes the privileged operation, and then returns to user mode via a return-form-trap instruction, which is called by the OS.

In order for the OS to know which code to run when a certain system call is called, the OS sets up a trap table at boot time, which maps system call to the code that has to be executed.

When the trap instruction is called, the system has to remember some information (which are saved on the kernel stack) about the process in order to restore it once the return-from-trap instruction is called.

Problem 2: Switching Between Processes

The second problem is a bit tricky: the fact that the OS is a process itself, if another process is running, the OS cannot in take control of the system in any way. There are two approaches to solve this problem:

  1. The cooperative approach, where the OS trusts the processes to periodically give up the CPU so that the OS can decide to run some other task. The control is transferred for example when a system call is called. The problem here is that if a process is stuck on a infinite loop, then the OS cannot ever run. This was an approach used on older operating systems, and the solution to the last problem is as old as it can be: just reboot the machine.
  2. The non-cooperative approach, where an interrupt is raised every so many milliseconds by a timer interrupt, and a pre-configured (in the boot by the OS) interrupt handler is run, which gives the control back to the OS. If the scheduler now decides to run another process, then the OS executed a piece of code known as context switch, where information about the process is saved on some registers and restored when the return-from-trap instruction is called.

tags: operating-systems