Simulations have shown that the amount of data which can be put into a processor cache is insufficient to support more than two processors on a single M bus. One way to get more data is to put however many processor cache chips are required per processor. This is unacceptable because it requires too much board area per processor and also increases the number of M bus loads to the point where bus timing requirements are difficult to meet. A more workable solution is a two-level cache scheme. The first level cache reduces the average latency sufficiently that each processor's performance is not impacted severely. The second level cache reduces the amount of bandwidth required by each processsor so that a single M bus may support approximately 10 processors.

There is quite a variety of two-level schemes possible. The scheme described here is a compromise between complexity of implementation, compatibility with previous implementations and the performance advantages we're after.

The two-level cache scheme consists of three components: the processor cache, the bus cache, and approximately 64KB of local memory. The processor cache and bus cache are both custom chips, while the local memory consists of off-the-shelf memory chips.

The processor caches connected to a processor sit on an internal bus called the MI bus (for M bus, internal) whose protocol is a strict superset of the M bus. The processor cache front end connects to the processor and traffics in reads and writes with virtual addresses; its back end connects to the MI bus and traffics in real addresses using the usual M bus transactions. Processor caches are designed so it is also possible to plug them directly into the M bus and have them work (indeed this is essential since it allows Dragons to be brought up without putting the bus cache in the critical path).

The front end of a bus cache is connected to a MI bus, and its back end to the main M bus. The local memory connects directly to the bus cache from the side or it may connect directly to MI, depending on timing and bus capacitance issues. There is only one bus cache per MI bus and the arbiter for the MI bus sits inside the bus cache.

This section has latency and bandwidth estimates based on Dragonman simulations. We believe that the cache models are correct and that Dragoman executes PrincOps correctly. I do not know of any verification ensuring that the reference stream which connects the Dragoman emulator to the cache model is correct.

Latency: wait cycles / cycle = (ifu ref / cycle * m ref / ifu ref * ifu cache wait cycles / m ref) + (eu ref / cycle * m ref / eu ref * eu cache wait cycles / m ref). I believe these are additive since the m bus serializes eu and ifu misses.

M bus bandwidth: m cycles / cycle = (ifu ref / cycle * m ref / ifu ref * ifu cache m cycles / m ref) + (eu ref / cycle * m ref / eu ref * eu cache m cycles / m ref).

Effective processor power:

We need to get the following two statistics:

(a) How often do processor writes result in processor cache misses? This is needed to verify that the three cycles of latency added when a write misses in a processor cache are not a performance problem.

(b) How often does a processor write into a clean quad in a processor cache? This is needed to verify that the spurious WS generated by a processor cache in communicating to the bus cache that a quad has been written into is not a performance problem.

How is the processor number initialized during reset so that minimal systems can be booted without the baseboard?

Should there be programmable page size? Over what range? Could implement it by running enable and enable' lines down through the low order bits of page so that the cells can drive the page or block match line low.

Why not a standard component implementation?

when the m controller sees its own command it must return bus fault to the processor and release the bus.

how can we have two virtual addresses for a single real address at the same time in one cache?

breakpoint registers.

what other actions cause quads to be removed from caches besides victimization? A flush causes dirty quads to be written back first but the clean quads merely have their valid bits reset. But what order does this occur in? While the flush is in progress is it possible for a quad to be in a processor cache but not in a bus cache? What prevents this?

there is a specific space id which must be sent during readmapsetref or readmapsetrefsetdirty. Write it down in here.

have reschedule be a line which comes out of the processor cache and is invoked by an I/O operation on the M bus.

This section describes the reasoning behind the cache pin assignment shown in Figure 2. The initial assumption is that the control logic is all at the top of the array and that the data buses run across the top of the chip and then narrow down the two sides, the M bus on the right and the P bus on the left, meeting in the middle of the bottom of the chip. The initial pin count constraints are as follows:

This requires a total of 123 pins out of the 144 available. The power pins are the obvious place to use some of the remaining 21 pins. One would like to minimize the internal wiring length from the closest power pads to any point on the chip. The added power pins should also be chosen from those with the lowest inductance in the package. The initial power pin count was chosen simply to use up all the low inductance pins in the package. They were then arbitrarily divided up as follows:

The pad power pins were assigned to the lowest inductance pins. The logic power pins were assigned to the next lowest inductance pins. The arrays in the logic are likely to use quite a bit of current during their precharge and discharge cycles. Lets add 2 pins to each of the logic supplies leaving 17 pins. The M bus has about between 5 and 20 times as much capacitance as the P bus but it runs at half the rate so it should get between 2 and 10 times more pins. Arbitrarily chosing 2 as the multiplier and 8 pins for the M bus means 4 additional pins for the P bus leaving 5 pins uncommited. They will be left uncommited so that if any extra functionality is required in the controller there are spare pins to implement them, e.g. a mechanism for determining when a cache flush or launder process is finished may require a pin or so. The M bus has about 39 pins that all swing at the same time so 39/8 means about every 10 pins there should be a PadVdd/PadGnd pair. They should also be constrained to be on the inner row of pins in the package. For the M bus this means pins 60, 66, 69, 77, 81, 85, 96 and 102 are the obvious choices. The P bus has 37 pins so every 9 pins should have a power or ground. The possible pins in the range of 20 through 53 are 24, 30, 41, 45, and 49. Eliminating 49 leaves 4 pins.