<> <> <> <> <> COMMON M BUS SPECIFICATIONS COMMON M BUS SPECIFICATIONS COMMON M BUS SPECIFICATIONS FOR INTERNAL XEROX USE ONLY FOR INTERNAL XEROX USE ONLY FOR INTERNAL XEROX USE ONLY The Common M Bus Description and Specifications DRAFT of March 22, 1985 ..nearly done.. Release as [Indigo]Documentation>CommonMBus>CommonMBusSpecs.tioga, .press © Copyright 1985 Xerox Corporation. All rights reserved. Abstract: This memo proposes a common M bus for Dragon and C workstation. It is intended to be used both as a convenient source for information about the M bus and as a reference manual for bus specifications. The memo begins with a brief overview of the M bus. It then lists all the bus signals and defines the meaning of each signal group. Next, it uses these signals to describe in detail how the bus operates at both the cycle and transaction levels, and how certain transactions are used to maintain cache consistency. Finally, it gives the detailed timing of the bus transactions currently defined. XEROX Xerox Corporation Palo Alto Research Center 3333 Coyote Hill Road Palo Alto, California 94304 For Internal Xerox Use Only Contents 1. Introduction 2. Bus Signals 3. Bus Operation 4. Bus Timing 1. Introduction The M bus is a high bandwidth synchronous bus that interconnects a multiprocessor system. Five kinds of devices connect to an M bus: caches, that intermediate between processors and the bus, memories, some of which may be multiported, I/O device controllers, a map, responsible for informing the caches about the current virtual-to-physical memory address mapping, an arbiter, that assigns mastership to at most one requesting device. This document is the specification for the M bus interface of caches. It implies restrictions on other devices that connect to the M bus as well, but it is not intended to specify those devices with any completeness. M buses have a brief but interesting history within Xerox. The cache consistency protocol, which many consider to be the key idea that the M bus implements, first occurred to Dashiell of Xerox ED and Thacker of Xerox PARC about 1980. The original intention was that ED and PARC develop a compatible M bus that could be used by PARC's workstation projects and ED's printer projects. Not enough energy was devoted to overcoming the increase of entropy, and by summer of 1984 ED nearly had a working M bus and PARC had a paper design of a rather different M bus. Since then it has become clear that the ED M bus as it exists today (we shall call that the ESS M bus) will not satisfy all of ED's requirements for its planned C workstation line, and so considerable effort is being devoted at ED and PARC to settling on a common design for the second-generation M bus. This document is the first detailed proposal from PARC in that direction. Many of the apparently arbitrary choices in the following sections are dictated by a strong desire that existing ESS caches function as well as possible on the new common M bus. The remainder of the document goes into greater detail about the structure, operation, and timing of the M bus. The next section lists the bus signals in detail and gives the meaning of each signal. Section 3 next uses these signal definitions to explain how the bus operates: it discusses arbitration, lists the set of bus commands currently defined, shows how these commands are used in each of the possible transactions, shows how transactions are used to maintain cache consistency, and introduces housekeeping operations to deal with address space switches and multiport memories. Finally, Section 4 discusses the timing of arbitration and of each transaction in detail. 2. Bus Signals An M bus consists of three independent groups of lines. The first is the clocking group. The second, called the arbitration group, assigns bus mastership. The third, called the transfer group, transports data once mastership has been established. The following description uses these naming conventions: S[a..b) denotes a group of (b-a) lines that encode the bits representing the signal S; most significant bits of the signal are written leftmost, and have the smallest numeric index. A signal represented by a single wire is written unencumbered with the [) notation. All signals are assumed to follow positive logic unless they have an "n" as the second character of the signal name. Some of these lines have pullup resistors to ensure that they remain unasserted during cycles when they are not driven. These resistors do not supply enough current to de-assert a line in one cycle; however they must de-assert the line in a minimum reset period. 2.1 Clocking Group The M bus is timed in cycles of a two-phase clock. The phases are named A and B, and a clock cycle begins at the leading edge of A and ends at the trailing edge of the following B. A signal that conforms to standard M bus timing is driven onto the M bus as soon as possible after the start of phase A, and is sensed from the M bus by latches that are open (enabled) during phase B. This means that the largest portion of a clock cycle is available for data to propagate and stabilize on the M bus. All of the following signals except MPhA and MPhB conform to standard M bus timing. MPhA This is asserted during the "first" phase of a two-phase clock cycle. MPhB This is asserted during the "second" phase of a two-phase clock cycle. MHold The state of this line as sampled at the end of cycle i determines whether the state of the M bus system at the end of cycle i+2 is to be identical to its state at the end of cycle i+1. MnReset This line is held asserted during power-up, and is asserted for at least 128 consecutive cycles whenever it is asserted. The cycle following the cycle in which MnReset is de-asserted is a valid M bus cycle. 2.2 Arbitration Group The arbitration group consists of a pair of request/grant lines for each bus requestor, and one other shared line called MnNewRQ. A single, central arbiter time-multiplexes the transfer group among several contending devices. Each of these devices makes requests to the arbiter using its dedicated request line, and the arbiter grants mastership of the bus using the dedicated grant lines. A device that has been granted the bus is called bus master. The sequence of cycles initiated by a master will typically involve at least one other device on the bus; such a device is called a slave. MnRq[0..16) These lines are used by the caches to request bus mastership. There is a dedicated line from each cache to the bus arbiter. MnGnt[0..16) These lines are used by the arbiter to signal which of the caches is the next M bus grantee. Like the request lines, these lines are also dedicated, one per cache. There is a possibility that they will be replaced by 4 encoded grant lines that supply the number of the cache to which the bus has been granted. Alternatively, these encoded lines may be run in addition to the decoded lines. The encoded lines are needed by the Map processor to figure out which processor is currently bus master. MnNewRq This line is logically unnecessary, but can reduce the variance in M bus latency. This line signals that the current grantee wants to retain his grant, but is willing to relinquish it if there are other requesters at this time. This line has a pullup resistor. 2.3 Transport Group The transfer group consists of approximately 42 wires. MnAdCycle This line is asserted low during cycles when a valid command is being asserted on MCmd. This line may be asserted either by the master of the current transaction or by the slave selected by the first cycle of the current transaction. MTransport[0..36) These thirty-six wires are used to encode three fields: MCmd[0..4), MAddr[0..32), and MData[0..32). Since this adds up to more than thirty-six wires, these fields are time-multiplexed. The somewhat peculiar way in which this multiplexing is done allows the ESS cache to attach to thirty-two of them, MTransport[5..36). MCmd[0..4) = MTransport[4..8) This field is only valid if MnAdCycle is asserted. It encodes the M bus command, which specifies the action during the current bus cycle, and may carry implications for future cycles as well (see transactions, section 3.3). It is asserted by the same device that is asserting MnAdCycle. The commands currently defined are discussed later in Section 3.2. MAddr[0..32) = MTransport[0..4)|MTransport[8..36) This field is only valid if MnAdCycle is asserted. Its interpretation depends on MCmd, which is a disjoint field. This field normally carries an address. MData[0..32) = MTransport[4..36) This field is only valid if MnAdCycle is de-asserted. Its interpretation depends on previous values of MCmd. This field normally carries data. MParity This line carries the parity of MTransport. The parity is odd and driven one cycle later by the device that drove MTransport. MnShared This line indicates that a copy of the addressed memory word is present in a slave cache. This line has a pullup resistor. MnAbort This line prevents memory from responding to a ReadQuad and allows a cache to respond instead. This wire is not logically necessary to the cache consistency protocol, but it allows a performance optimization. This line has a pullup resistor. MnHousekeepingInProgress Following any cycle in which this line is not asserted, every cache that is doing housekeeping asserts it. Following any cycle in which it is asserted, every cache actively de-asserts it. Two unasserted cycles in succession mean that no housekeeping is in progress. This line has a pullup resistor. MnError This line is used to signal a bus error. The only bus error currently defined is bus parity error. This line has a pullup resistor. 3. Bus Operation M bus operation is specified at two levels: cycles and transactions. As explained earlier, a cycle is one complete period of the system-wide two-phase clock. A transaction is a sequence of contiguous cycles initiated by some bus master. During the first cycle of every transaction, MnAdCycle is asserted, MCmd specifies the transaction type, and MAddr specifies an address, whose interpretation varies with transaction type. Subsequent cycles transfer data, perform other manipulations, or signal the end of the transaction. Recall that the device that initiates a transaction is called the bus master, and other devices with which it interacts are called slaves. Bus mastership is granted by the arbiter. Thus overall bus operation is a simple pipelined cyclic process of arbitration followed by transaction execution. The description below first takes up arbitration. It then describes the various M bus commands as well as the set of transactions that use these commands. For completeness, it also discusses how caches maintain consistency of multiple copies of shared data. Finally it discusses a set of housekeeping operations that deal with the mapping problems introduced by moving a processor from one address space to another, and with the consistency problems introduced by multiport memories. The detailed timing of arbitration and of each of the transactions is not taken up here but is left to Section 4. 3.1 Arbitration The signals involved in arbitration are the MnRq-MnGnt pairs running to and from each potential master and the arbiter, and a single line MnNewRq used to increase the fairness of arbitration. A device requests the bus by asserting its request line. If one or more MnRq lines are asserted, and there is no master (grantee) at the moment, the arbiter performs a grant cycle in which it issues the bus by asserting MnGnt for a device that has MnRq asserted. The bus master keeps MnRq asserted while it still wants the bus, and the arbiter in turn keeps that master's MnGnt asserted. If the master is executing a long sequence of cycles that is not required to be atomic with respect to other devices, it may assert MnNewRq to indicate its willingness to give up the bus if there are other requests pending (for correct operation, MnNewRq must be asserted only by the current master). MnNewRq causes the arbiter to do a grant cycle even though the current master still has MnRq asserted. This strategy prevents a master from hogging the bus (unless, for atomicity, it needs to). At the same time, it avoids wasting cycles in the case that no devices are waiting to get the bus. A solution in which the master releases and reacquires the bus, for example, does waste cycles in this case. 3.2 M Bus Transactions There are currently nine master transactions: ReadQuad, WriteQuad, WriteSingle, MapRef, MapRefDirty, IORead, IOWrite, IOReadFlow, and IOWriteFlow. The bus master initiates a master transaction by asserting the appropriate command with an address of some kind. This command and address is interpreted by every device attached to the M bus, and one or more devices decide to act as slaves during the remainder of the transaction. Some master transactions consist of a fixed sequence of cycles: WriteSingle, IORead, and IOWrite. Others are flow-controlled by the MnDV line: ReadQuad and WriteQuad. The remainder are flow-controlled by the slave transaction Done. Flow-controlled master transactions begin with a fixed sequence of cycles, depending on the transaction, wait for their flow-controlling event, and then terminate in a fixed sequence of cycles, again depending on the transaction. Flow-controlled transactions must select exactly one slave. The M bus specification includes means for detecting when this is not so. (Oh yeah? How do we do this? Whatever is easiest for ED should probably prevail.) Transactions that are flow-controlled by Done may contain subtransactions that are not flow-controlled by Done. During those subtransactions, the slave of the main transaction acts as master of the subtransaction. IO devices may respond to any or all of the four IO transactions. M bus commands are carried in the MCmd field and are valid when MnAdCycle is asserted. Twelve of these are currently defined, and the other four are reserved for future use. In the description below, the decimal number in curly brackets after the command name indicates the encoding of the command. (On page 8 of the 21 Jan 84 ESS cache specification, there are (optional?) wait cycles appearing before several transactions. What does this mean?) ReadQuad {0 or 2 (sigh!)} This command is issued by the master during the first cycle of a ReadQuad transaction, which reads a quad into the master from memory or from another cache. A quad is four contiguous 32-bit words aligned in physical address space such that the address of the first word is 0 MOD 4. During this first cycle MAddr is the physical address of one of the words in the quad. The transaction then consists of zero or more wait cycles until the MnDV line signals that the memory system is responding. The addressed word will be asserted on MData by the slave during that cycle. The remaining words of the quad will be asserted on MData by the slave in subsequent cycles in an order determined by their address MOD 4: Addr Transport order 0 => {0, 1, 2, 3} 1 => {1, 0, 3, 2} 2 => {2, 3, 0, 1} 3 => {3, 2, 1, 0} Caches built to this specification will understand both 0 and 2 as encodes for ReadQuad, but will issue only 0. WriteQuad {1 or 3 ((sigh!)} This command is issued by the master during the first cycle of a WriteQuad transaction, which writes a quad from the master to memory and to other caches. In the first cycle MAddr carries the address of the quad which, unlike the address for a ReadQuad transaction, must be 0 MOD 4. In subsequent cycles the master asserts the data of the quad on MData, in order of increasing address. The master releases MnRq for this transaction in the cycle after MnDV is asserted by the slave. Caches built to this specification will understand both 1 and 3 as encodes for WriteQuad, but will issue only 1. WriteSingle {7} This command is used by the master during the first cycle of a WriteSingle transaction, which writes a single word into the physical address space. WriteSingle never writes the word to memory, but only to caches that already hold a copy of the quad containing the word being written. MAddr carries the physical address of the word in the first cycle of the transaction. The data is asserted by the master in the next cycle of the transaction. MapRef {12} This command is used by the master during the first cycle of a virtual-to-physical page translation transaction. Use of this transaction indicates that the master intends to read data in the referenced page of its virtual address space (each master may be operating in a different virtual address space). In that first cycle MAddr[7..32) contains a virtual 512-byte-page number. The final cycle of the transaction is the cycle in which the slave (map) asserts a Done command that contains the results of the translation in MAddr. (Does the ESS cache really require three dead cycles after the Done? Is that because its matcher is tied up?) MapRefDirty {10} This command is used by the master during the first cycle of a virtual-to-physical page translation transaction. Use of this transaction indicates that the master intends to write data in the referenced page of its virtual address space (each master may be operating in a different virtual address space). In that first cycle MAddr[7..32) contains a virtual 512-byte-page number. The final cycle of the transaction is the cycle in which the slave (map) asserts a Done command that contains the results of the translation in MAddr. IORead {4} This command is used by a master during the first cycle of an IORead transaction. MAddr contains the IO register address. The second and third cycles are idle. Any number of devices may generate side effects in response to an IORead, but exactly one slave must assert data on MData during the fourth cycle. The interpretation of an IORead is entirely up to the responding device. Thus it may actually read from a real device or may cause some side effect, or both. IOWrite {5} This command is used by a master during the first cycle of an IOWrite transaction. MAddr contains the IO register address. The master asserts the data on MData in the next cycle. Any number of slaves may respond, but they do so undetected by the IOWrite protocol. IOReadFlow {8} This command is used by a master during the first cycle of an IOReadFlow transaction. MAddr contains the IO register address. The slave signifies readiness by a Done command, and asserts data on MData in the following cycle. IOWriteFlow {9} This command is used by a master during the first cycle of an IOWriteFlow transaction. MAddr contains the IO address. During the second cycle the master asserts data on MData. In some subsequent cycle the slave signifies readiness by a Done command, and the master re-asserts the same data on MData in the following cycle. Done {14} This command is issued by the slave device to flow-control a MapRef, MapRefDirty, IOReadFlow, or IOWriteFlow transaction. The value that it asserts on MAddr depends on the transaction and the slave. If the transaction is MapRef or MapRefDirty, then the slave is the map. MAddr[0..25) contains the number of the 512-byte physical page corresponding to the virtual page that initiated the transaction. MAddr[25..28) carries one of the following orders: NoClear {7} -- no action required ClearRPDirty {6} -- write permission for the current physical page is summarily cancelled in all caches. From this point on, any cache wishing to write to a virtual address lying in this physical page must acquire new permission via MapRefDirty. ClearVPValid {4} -- virtual mappings to the current physical page are summarily cancelled in all caches. From this point on, any cache wishing to reference this physical page must acquire a new virtual-physical linkage via MapRef or MapRefDirty. WriteProtectFault {2} -- a MapRefDirty failed to acquire write permission from the slave. This order is of interest only to the M bus master. PageFault {1} -- the slave failed to locate a valid virtual-to-physical mapping. This order is of interest only to the M bus master. MAddr[28..32) carries a group of four flags from which the master cache should update its mapping information (except if there is a PageFault): MAddr[28] -- reserved MAddr[29] -- write protect ( = NOT dirty) MAddr[30] -- dirty MAddr[31] -- referenced ( = true) (Note: These flags are all redundant. They can be inferred from the transaction and its faults. They are included for backwards compatability with the ESS cache.) If the transaction is IOReadFlow or IOWriteFlow, and the slave is naive (ignorant of mapping), during the Done cycle the field MAddr[25..28) carries the order NoClear. IO data is then transported in the following M bus cycle. If the transaction is IOReadFlow or IOWriteFlow, and the slave is sophisticated (aware of mapping, probably the map itself), then MAddr[0..25) can contain the number of a 512-byte physical page, and MAddr[25..28) can contain the orders NoClear, ClearRPDirty, or ClearVPValid. IO data is then transported in the following M bus cycle (even though that data may be meaningless; the IOReadFlow or IOWriteFlow may have been executed for the side effects of its Done command). In fact, the map will probably contain a "Done reflector" IO register, that will simply save the data sent by the master in an IOWriteFlow transaction and retransmit it in the appropriate fields of the Done command that terminates the transaction. Reserved {6, 11, 13, 15} These commands are reserved for future use. Caches built to this specification will ignore them. 3.3 The Cache Consistency Protocol The cache consistency problem is to prevent multiple copies of a datum in distinct caches from having different values. This problem arises because Dragon caches contain shared data that may be written into. The solution involves the use of the MnShared and MnAbort bus lines and two additional bits shared and owner stored with each quad in a cache. For a given quad, if shared is false then no other cache contains a copy of the quad. However, if shared is true other caches may or may not contain the quad (there is uncertainty because when sharing stops this fact is not discovered immediately). Owner is true if and only if the last write into the quad was into this very cache. A cache initiates a ReadQuad when it gets a miss; it issues a WriteQuad when a quad needs to be removed from the cache to make room for another quad (only quads with owner set need to be written back to memory); and it issues a WriteSingle and sets owner when its processor does a write to a quad that has shared set. When a cache issues a ReadQuad, WriteQuad, or WriteSingle all other caches on the bus match the quad address to determine if they hold a copy of the quad. Any cache that matches asserts MnShared to signal that the quad is shared. During ReadQuads the master and all matching slaves copy the value of MnShared into the shared bit for that quad. Also, when a cache issues a WriteSingle the master copies the value of MnShared into the shared bit. This ensures that sharing is detected as soon as it starts and is eventually detected when it stops. During ReadQuad, WriteQuad, or WriteSingle, all matching quads except the one supplying data to the bus load new values from the bus and clear their owner bits. The owner bit is set each time a processor writes one of the words of the quad. If the quad is shared, the resulting WriteSingle will update all other copies of the quad and cause their owner bits to be cleared. Thus at any time at most one of the caches will have owner set. Owner may not be set at all, of course, if the quad has not been written into since it was read from memory. During a ReadQuad, if the owner bit is set on a matching quad, that cache asserts MnAbort and supplies the data to the M bus itself (since the memory doesn't know the current value). Otherwise MnAbort is not set and the memory supplies the data. 3.4 Housekeeping operations We are left with two problems that Dragons and probably C machines will have to deal with, and that that the basic M bus operations above have dealt with incompletely. The first is context switches between address spaces. When a processor moves from one address space to another, all of the old virtual-to-physical associations in its caches that are not also present in the new address space must be broken. There are two ways to solve this problem with the basic M bus operations. First, the processor can issue a sequence of P bus references that victimizes all data in all its caches. Second, a sequence of M bus Done commands can be generated, in cooperation with a sophisticated slave, to clear VPValid in all caches for all physical pages that might be known to the processor. Both of these solutions are so expensive as to prohibit frequent context switches between address spaces. The second remaining problem is multiport memories. Some memories have two ports, a primary one which attaches to the M bus and a secondary one which does not. Such a memory might appear in a display controller that can do autonomous BitBlt, for instance. When both ports are active, our protocol for maintaining consistency does not work, because the secondary port can do reads and writes without informing the M bus. A partial solution to the second problem is to time-multiplex two-port memories among several access-limited states. At any moment such a memory is in one of three states: either its secondary port is entirely inactive, or else the M bus has no knowledge of it, or else both M bus and secondary port are prevented from writing. This solution is a bit too coarse. We can refine it by allowing each page in such a memory to be in one of the three states independent of the states of other pages. To deal with the first problem we introduce an operation called DeMapping (breaking virtual-to-real association, leaving orphans in the cache). To deal with the second we introduce Laundering (rewriting all dirty quads to memory and leaving them clean) and Flushing (clearing RPValid). (Flushing appears not to be necessary if ReadQuads always refill clean orphans from memory. Laundering and DeMapping suffice in that case.) Collectively we refer to these as housekeeping operations. For efficiency housekeeping operations apply not to single physical pages but rather to sets of physical pages matching a mask of {0, 1, don't care} "trits". Housekeeping operations can be applied to all caches at the same time, or to a single cache. Each cache contains a pair of 32-bit registers that together hold the physical page mask to be used. In addition, each cache has a single housekeeping initiation register located at two I/O addresses: a broadcast address, and a private one. 1 bits stored into appropriate bits of the initiation register cause initiation of DeMap, Launder, and Flush. (If we could use addresses in ESS cache mailbox space, there would be a way of notifying processors attached to ESS caches that these housekeeping operations were being initiated.) This all implies that a cache needs to respond as a slave to its own IOWrites, and probably to its own IOReads as well. It is not necessary for caches to respond to flow-controlled IO. Launder can require enough M bus cycles to transport all dirty quads in every cache back to memory. While a Laundry operation is underway each cache re-arbitrates for the M bus for every dirty quad that it owns and needs to store. Requests from the P bus side demanding M bus service have higher priority than the operation requests. These stipulations together prevent a hiccup from occuring in the system when there is a great deal of dirty data to be laundered. Not only Laundry, but all housekeeping operations, depending on their implementations, may take many cycles to complete. Conceptually each cache maintains a housekeeping in progress bit that indicates that a housekeeping operation has been started in that cache but not yet finished. Whether an operation is in progress in any cache in the system can be known by observing the state of the MnHousekeepingInProgress wire for two successive cycles. If it is not asserted in some cycle i, then any cache with an operation still in progress will assert MnHousekeepingInProgress in cycle i+1. If it is asserted in cycle i, then every cache will de-assert it in cycle i+1. MnHousekeepingInProgress will thus toggle on and off until every cache has finished the last housekeeping operation. Each cache has a means by which MHousekeepingInProgress[i+1] OR MHousekeepingInProgress[i] may be read from its PBus without needing M bus cycles. The alert reader has already noticed that concurrent housekeeping operations could be a nightmare. We avoid concurrency by maintaining, as a software convention, a global housekeeping lock in shared (and always mapped) memory. To do housekeeping a process must acquire the lock, verify that previous housekeeping is finished, initiate the operation, and (if logically necessary) hold the lock until housekeeping is once again finished. 4. Bus Timing This section gives the detailed timing of bus arbitration and of each of the transactions just described. In the timing diagrams, each row corresponds to one bus cycle for non-idle cycles. Idle cycles are represented by a row labelled "w cycles", where w indicates the number of idle cycles. Actions during each cycle are indicated using Mesa like statements. Note that the boolean value TRUE corresponds to the high voltage level and the boolean value FALSE corresponds to the low voltage level. For example a negative logic signal x would be asserted as follows: x _ FALSE. The diagrams do not describe the timing of MParity, since this is always one cycle delayed from the value on MTransport. In each description, the requestor asserts MnRq in cycle 0 and the arbiter asserts MnGnt in cycle 1. An arbitrary number of cycles may occur between cycles labelled 0 and 1. 4.1 Arbitration The timing for arbitration appears in Figure 2. A requestor asserts MnRq during phase A of cycle 0. The arbiter latches the request during the B of cycle 0, and if the bus is free MnGnt gets asserted in the A of cycle 1. The new master can drive the bus at the beginning of cycle 2A. Thus the minimum arbitration delay is two cycles. If the bus is not free, zero or more wait cycles are inserted immediately after cycle 0. The earliest MnRq can be deasserted is during cycle 2+w. The earliest MnGnt is deasserted in response is during cycle 3+w, and the earliest a new device can become master is during cycle 4+w. The earliest MnNewRq may be asserted is cycle 3+w. It must be deasserted in the following cycle. 4.2 ReadQuad Figure 3 shows the timing for ReadQuad. Cycles 0 through 1 are taken up by arbitration (recall that we are assuming the minimum arbitration delay). The transaction begins in cycle 2. The master sets MCmd to ReadQuad, MAddr to RQAddress, and asserts MnAdCycle. Cycles 3 and 4 do not transmit any data. During cycle 3 the master deasserts MnAdCycle and during cycle 4 any matching cache asserts MnShared, and if its owner bit is set, asserts MnAbort. In the following cycle memory deasserts MnShared and the cache, if any, that asserted MnAbort deasserts it. After cycle 4 the slave may insert an arbitrary number of wait cycles, w In cycle 5+w the slave (responding cache or memory) asserts MnDV to signal that data is valid and sends the first word of the quad on MData. During cycle 6+w the slave deasserts MnDV and drives the second data word onto MData. During cycle 7+w the slave drives the third data word onto MData and the master deasserts MnRq. During 8+w the slave drives the fourth data word onto MData and the arbiter deasserts MnGnt. Note that the minimum number of cycles for ReadQuad is 7, assuming zero memory wait. 4.3 WriteQuad Figure 4 shows the timing for WriteQuad. Cycles 0 through 1 are devoted to arbitration as usual. During cycle 2 the master sets MCmd to WriteQuad, MAddr to the address and asserts MnAdCycle. During cycle 3 the master drives the first data word onto MData and deasserts MnAdCycle. During cycle 4 the master drives the second data word onto MData and if there are zero wait states (w=0) the memory asserts MnDV. During cycle 5 the master sends the third data word on MData and if there are zero wait states, deasserts MnRq. The memory deasserts MnDV if there are zero wait states, or asserts MnDV if there is one wait state. In cycle 6 the master drives the fourth word onto MData, and deasserts MnRq if there is one wait state. If there are zero wait states the arbiter deasserts MnGnt. The memory deasserts MnDV if there is one wait state, or asserts MnDV if there are two wait states. In cycle 7, if there are two wait states, the master deasserts MnRq and the memory deasserts MnDV. If there is one wait state the arbiter deasserts MnGnt. If there are three wait states the memory asserts MnDV. Finally, there are an arbitrary number of wait cycles which are terminated by the memory asserting MnDV, followed in the next cycle by the master deasserting MnRq, and in the next cycle the arbiter deasserting MnGnt. Note that the minumum number of cycles for WriteQuad is 5. 4.4 WriteSingle Figure 5 shows the timing for WriteSingle. Cycles 0 through 1 are devoted to arbitration. During cycle 2 the master deasserts MnRq, sets MCmd to WriteSingle, sets MnAdr to the address, and asserts MnAdCycle. During cycle 3 the master sets MData to the data and deasserts MnAdCycle. The arbiter deasserts MnGnt. During cycle 4 any matching slave asserts MnShared. During cycle 5 any matching slaves deassert MnShared. 4.5 IORead Figure 6 shows the timing for IORead. Arbitration takes place during cycles 0 through 1. During cycle 2 the master asserts MnAdCycle, asserts IORead on MCmd, asserts the IO register address on MAddr, and deasserts MnRq. During cycle 3, the master deasserts MnAdCycle, and the single slave asserts data on MData. 4.6 IOWrite Figure 7 shows the timing for IOWrite. Cycles 0 through 1 are for arbitration as usual. During cycle 2 the master asserts MnAdCycle, asserts IOWrite on MCmd, asserts the IO register address on MAddr, and deasserts MnRq. During cycle 3, the master asserts the write data onto MData and deasserts MnAdCycle. 4.7 IOReadFlow Figure 8 shows the timing for IOReadFlow. Cycles 0 through 1 are for arbitration as usual. During cycle 2 the master asserts MnAdCycle, asserts IOReadFlow on MCmd, asserts the IO register address on MAddr. During cycle 3 the master deasserts MnAdCycle. Following cycle 3 there can be w MnAdCycle and MCmd=Done during the same cycle. During cycle 4+w the slave asserts MnAdCycle, sets MCmd to Done, and sets the orders field of MData to NoClear (other values are acceptable if the slave is the map). In cycle 5+w the master (slave?) deasserts MnAdCycle. The slave asserts the data on MData and the master deasserts MnRq. 4.8 IOWriteFlow Figure 9 shows the timing for IOWriteFlow. Cycles 0 through 1 are for arbitration as usual. During cycle 2 the master asserts MnAdCycle, asserts IOWriteFlow on MCmd, asserts the IO register address on MAddr. During cycle 3, the master asserts the write data onto MData and deasserts MnAdCycle. Following cycle 3 there can be w during which the slave does not assert MnAdCycle and MCmd=Done during the same cycle. During cycle 4+w the slave asserts MnAdCycle, sets MCmd to Done, and sets the orders field of MData to NoClear (other values are acceptable if the slave is the map). In cycle 5+w the master (slave?) deasserts MnAdCycle, reasserts the write data, and deasserts MnRq. 4.9 MapRef and MapRefDirty Figure 10 shows the timing for these two transactions. Cycles 0 through 1 are for arbitration as usual. During cycle 2 the master asserts MnAdCycle, asserts MapRef (or MapRefDirty) on MCmd, asserts the vp on MAddr[7..32). During cycle 3, the master deasserts MnAdCycle. Following cycle 3 there can be w MnAdCycle and MCmd=Done during the same cycle. During cycle 4+w the slave asserts MnAdCycle, sets MCmd to Done, and puts rp, orders, and flags onto MData. In cycle 5+w the master (slave?) deasserts MnAdCycle. /indigo/dragon/documentation/commonmbus/DragonCluster.press leftmargin: 2 in, topmargin: 1 in, width: 7.5 in, height: 8 in /indigo/dragon/documentation/commonmbus/BusArb.press leftmargin: 2 in, topmargin: 1 in, width: 7 in, height: 5 in /indigo/dragon/documentation/commonmbus/ReadQuad.press leftmargin: 2 in, topmargin: 1 in, width: 7 in, height: 6.5 in /indigo/dragon/documentation/commonmbus/WriteQuad.press leftmargin: 2 in, topmargin: 1 in, width: 7 in, height: 5.5 in /indigo/dragon/documentation/commonmbus/WriteSingle.press leftmargin: 2 in, topmargin: 1 in, width: 7 in, height: 4 in /indigo/dragon/documentation/commonmbus/IORead.press leftmargin: 2 in, topmargin: 1 in, width: 7 in, height: 5 in /indigo/dragon/documentation/commonmbus/IOWrite.press leftmargin: 2 in, topmargin: 1 in, width: 7 in, height: 5.5 in /indigo/dragon/documentation/commonmbus/IOReadFlow.press leftmargin: 2 in, topmargin: 1 in, width: 7 in, height: 5.5 in /indigo/dragon/documentation/commonmbus/IOWriteFlow.press leftmargin: 2 in, topmargin: 1 in, width: 7 in, height: 5.5 in /indigo/dragon/documentation/commonmbus/MapRef.press leftmargin: 2 in, topmargin: 1 in, width: 7 in, height: 5.5 in