What is memory protection? And what are the techniques to ensure memory protection?
Memory protection is a core security component that must be designed and implemented into an operating system (OS). Moreover, memory protection must be enforced regardless of the programs executing in the computer system to protect system instability, integrity violation, denial of service (DoS) and unauthorized disclosure. Memory protection is mainly employed to prevent an active process from interacting or interfering with an area of memory that was not specifically assigned or allocated to it. The memory protection enables the OS to load multiple programs into main memory at the same time and prevent one program from referencing memory not specifically assigned do it. Furthermore, if the program attempts to reference a memory address it is not permitted to access, the system blocks the access, suspends the program, and transfers control to the OS.
A similar hardware feature that supports memory protection is dual mode operation. Furthermore, the processor can operate in one of two modes namely privileged (kernel) mode and unprivileged (user) mode. The OS runs in privileged mode that grants it permission to set up and control the memory protection subsystem. The privileged mode also permits the OS to execute special privileged instructions that control the processor environment.
Once a program has been loaded into memory and the memory protection is configured to restrict that program’s access to those areas of memory assigned to it by the OS, the OS transfers control to the program and simultaneously transitions to unprivileged (user) mode. The program runs in unprivileged mode that restricts it to accessing only the specific memory area dictated by the OS. If it makes an illegal memory reference, that includes attempting to execute a privileged CPU instruction, or spend the time slice granted to it by the OS, control is reverted back to the OS (running in privileged mode). The OS determines if control has been returned to the OS because of an illegal operation by the user program and decides how to handle that illegal operation. Another important concept related with memory protection is address space layout randomization (ASLR). ASLR seeks to mitigate the risks of predictable memory address location by randomization of the location and thereby minimize security threats.