Memory Speed and Size
A typical computer memory hierarchy comprises several levels, each level having a characteristic size and speed.
- The processor registers can be viewed as the top of the memory hierarchy. A RISC processor will typically have around thirty-two 32-bit registers making a total of 128 bytes, with an access time of a few nanoseconds.
- On-chip cache memory will have a capacity of eight to 32 Kbytes with an access time around ten nanoseconds.
- High-performance desktop systems may have a second-level off-chip cache with a capacity of a few hundred Kbytes and an access time of a few tens of nanoseconds.
- Main memory will be megabytes to tens of megabytes of dynamic RAM with an access time around 100 nanoseconds.
- Backup store, usually on a hard disk, will be hundreds of Mbytes up to a few Gbytes with an access time of a few tens of milliseconds.
Note that the performance difference between the main memory and the backup store is very much greater than the difference between any other adjacent levels, even when there is no secondary cache in the system.
The data which is held in the registers is under the direct control of the compiler or assembler programmer, but the contents of the remaining levels of the hierarchy are usually managed automatically. The caches are effectively invisible to the application program, with blocks or 'pages' of instructions and data migrating up and down the hierarchy under hardware control. Paging between the main memory and the backup store is controlled by the operating system, and remains transparent to the application program. Since the performance difference between the main memory and the backup store is so great, much more sophisticated algorithms are required here to determine when to migrate data between the levels.
An embedded system will not usually have a backing store and will therefore not exploit paging. However, many embedded systems incorporate caches, and ARM CPU chips employ a range of cache organizations.
Fast memory is more expensive per bit than slow memory, so a memory hierarchy also aims to give a performance close to the fastest memory with an average cost per bit approaching that of the slowest memory.