How are cache memories shared in multicore Intel CPUs?

by

In a multiprocessor system or a multicore processor (Intel Quad Core,
Core two Duo etc..) does each cpu core/processor have its own cache
memory (data and program cache)?

  1. Yes. It varies by the exact chip model, but the most common design is for each CPU core to have its own private L1 data and instruction caches.

    On old and/or low-power CPUs, the next level of cache is typically a L2 unified cache is typically shared between all cores. Or on 65nm Core2Quad (which was two core2duo dies in one package), each pair of cores had their own last-level cache and couldn’t communicate as efficiently.

Modern mainstream Intel CPUs (since the first-gen i7 CPUs, Nehalem) use 3 levels of cache.

  • 32kiB split L1i/L1d: private per-core (same as earlier Intel)
  • 256kiB unified L2: private per-core. (1MiB on Skylake-avx512).
  • large unified L3: shared among all cores

Last-level cache is a a large shared L3. It’s physically distributed between cores, with a slice of L3 going with each core on the ring bus that connects the cores. Typically 1.5 to 2.25MB of L3 cache with every core, so a many-core Xeon might have a 36MB L3 cache shared between all its cores. This is why a dual-core chip has 2 to 4 MB of L3, while a quad-core has 6 to 8 MB.

On CPUs other than Skylake-avx512, L3 is inclusive of the per-core private caches so its tags can be used as a snoop filter to avoid broadcasting requests to all cores. i.e. anything cached in a private L1d, L1i, or L2, must also be allocated in L3. See Which cache mapping technique is used in intel core i7 processor?

David Kanter’s Sandybridge write-up has a nice diagram of the memory heirarchy / system architecture, showing the per-core caches and their connection to shared L3, and DDR3 / DMI(chipset) / PCIe connecting to that. (This still applies to Haswell / Skylake-client / Coffee Lake, except with DDR4 in later CPUs).

Can one processor/core access each other’s cache memory, because if
they are allowed to access each other’s cache, then I believe there
might be lesser cache misses, in the scenario that if that particular
processors cache does not have some data but some other second
processors’ cache might have it thus avoiding a read from memory into
cache of first processor? Is this assumption valid and true?

  1. No. Each CPU core’s L1 caches tightly integrate into that core. Multiple cores accessing the same data will each have their own copy of it in their own L1d caches, very close to the load/store execution units.

    The whole point of multiple levels of cache is that a single cache can’t be fast enough for very hot data, but can’t be big enough for less-frequently used data that’s still accessed regularly. Why is the size of L1 cache smaller than that of the L2 cache in most of the processors?

    Going off-core to another core’s caches wouldn’t be faster than just going to L3 in Intel’s current CPUs. Or the required mesh network between cores to make this happen would be prohibitive compared to just building a larger / faster L3 cache.

    The small/fast caches built-in to other cores are there to speed up those cores. Sharing them directly would probably cost more power (and maybe even more transistors / die area) than other ways of increasing cache hit rate. (Power is a bigger limiting factor than transistor count or die area. That’s why modern CPUs can afford to have large private L2 caches).

    Plus you wouldn’t want other cores polluting the small private cache that’s probably caching stuff relevant to this core.

Will there be any problems in allowing any processor to access other
processor’s cache memory?

  1. Yes — there simply aren’t wires connecting the various CPU caches to the other cores. If a core wants to access data in another core’s cache, the only data path through which it can do so is the system bus.

A very important related issue is the cache coherency problem. Consider the following: suppose one CPU core has a particular memory location in its cache, and it writes to that memory location. Then, another core reads that memory location. How do you ensure that the second core sees the updated value? That is the cache coherency problem.

The normal solution is the MESI protocol, or a variation on it. Intel uses MESIF.

More Related Contents:

  1. Why is the loop instruction slow? Couldn’t Intel have implemented it efficiently?
  2. Why is Skylake so much better than Broadwell-E for single-threaded memory throughput?
  3. Why is this SSE code 6 times slower without VZEROUPPER on Skylake?
  4. How are x86 uops scheduled, exactly?
  5. Branch alignment for loops involving micro-coded instructions on Intel SnB-family CPUs
  6. Why is XCHG reg, reg a 3 micro-op instruction on modern Intel architectures?
  7. Are there any modern CPUs where a cached byte store is actually slower than a word store?
  8. 32-byte aligned routine does not fit the uops cache
  9. Non-temporal loads and the hardware prefetcher, do they work together?
  10. Size of store buffers on Intel hardware? What exactly is a store buffer?
  11. Which Intel microarchitecture introduced the ADC reg,0 single-uop special case?
  12. Why can’t my ultraportable laptop CPU maintain peak performance in HPC
  13. Cycles/cost for L1 Cache hit vs. Register on x86?
  14. Return address prediction stack buffer vs stack-stored return address?
  15. Do 128bit cross lane operations in AVX512 give better performance?
  16. What is the best way to set a register to zero in x86 assembly: xor, mov or and?
  17. Enhanced REP MOVSB for memcpy
  18. INC instruction vs ADD 1: Does it matter?
  19. Understanding the impact of lfence on a loop with two long dependency chains, for increasing lengths
  20. What happens after a L2 TLB miss?
  21. What methods can be used to efficiently extend instruction length on modern x86?
  22. Lost Cycles on Intel? An inconsistency between rdtsc and CPU_CLK_UNHALTED.REF_TSC
  23. Is using double faster than float?
  24. Unexpectedly poor and weirdly bimodal performance for store loop on Intel Skylake
  25. simplest tool to measure C program cache hit/miss and cpu time in linux?
  26. Modern x86 cost model
  27. When should we use prefetch?
  28. Relative performance of x86 inc vs. add instruction
  29. Efficient sse shuffle mask generation for left-packing byte elements
  30. How can the rep stosb instruction execute faster than the equivalent loop?

Leave a Comment