# Copyright (c) 1993 Xerox Corporation.  All rights reserved.
#
# MPNotes.txt
#


Questions:

1. EnableHostSigs/DisableHostSigs must work per lwp!
2. The PrepareForSpawn stuff has to be examined ... maybe it should be
removed altogether, since it's so complex!
3. Leaving the running proc on the run queue is a bad idea if there
are multiple lwp's  -- DONE --

4. RunExclusive will have mp memory model problems?!

Clearly doCheckIO is for all vps?  doCheckTimeout ??? and doCheckChildren is
problematic -- I'm becoming more convinced we shouldn't try to emulate
fork/spawn.

 
Current implementation has:

    sys cr sec nest (per thread)
    deferredAction
      per thread? an accelerator, says there's something
      interesting to do on sys cr sec exit 
    mustCheckSigs
      per thread, indicates the thread needs to get signals
      by calling AcceptPendingSigs (in cr sec) and then
      PCR←ThPvt←HandleAcceptedSigs (outside cr sec).
    PCR←ThGlue←doCollectSigs
      global used to tell the thread switcher that it's time to
      collect host sigs.
    PCR←SigPvtImpl←sigsRecvd[], PCR←SigPvtImpl←postCnt
      the host signals received.  To make this work with multiple lwp's
      we'll need to do mutual exclusion among the lwp host sig handlers
      that are storing into the alternate word of sigsRecvd.  This
      shouldn't be too hard.
    PCR←SigPvtImpl←externalInterrupts
      list of external interrupts that have arrived at the process
      but not yet used by the PPCR external interrupt mechanism.
      Actually it may be okay that there's a global one of these,
      since it's only manipulated from sys critical section.


For the revised system:

Sys critical section:

    - mutual exclusion
    - nestable
    - thread signal handlers aren't invoked
    - no preemption?  This is irrelevant in MP ... but preemption per se
      would require entering a sys critical section, so ...
    - voluntary thread switching?  This could be arranged, since the meta
      lock would be acquired around a switch, and the switchee's nesting
      count would control release of it ...

Lib critical section:

    - mutual exclusion
    - nestable
    - thread signal handlers aren't invoked ?
      Maybe they could be, if unwind protection is done carefully in the
      library shim implementation.  Think about this.
    - preemption is okay.
    - voluntary thread switching is of course okay.

    Does this have anything to do with GC?  I think vd implementation
    takes care of that ...
    Does this have anything to do with errno?  Would be nice if it didn't!
     
n.b. can't do EnterLib while in a sys critical section!

If I ignore the Lib critical section part and just concentrate on sys:

    Implementation:

    - sys cr sec nest count is per-thread

    - gotKicked is per lwp, set if the lwp receives a (host) signal
      while in a sys critical section, maybe other reasons ????

    - PCR←ThGlue←doCollectSigs is global, set if there are any
      unprocessed host sigs to be picked up.
      The host signal handler(s) have to do mutual exclusion (using
      lwp mutex stuff, on solaris) around the PCR←SigPvtImpl←sigsRecvd[]
      repository; the alternating-location strategy currently being used
      will work okay between the thread level and the handlers.

    - mustCheckThreadSigs is per thread, set if there might be signals
      available for the thread.  This is set by code in sys critical
      sections that sends signals; (???not needed for traps, because the
      thread switcher should notice???)



    - thread switch:

      EnterSys()
      if( PCR←ThGlue←doCollectSigs ) {
          PCR←SigPvtImpl←CollectSigs(&myTraps)
          PCR←ThPvtImpl←DistributeSigs(&myTraps)
      }
      
      do {
          gotKicked[thislwp] := false;
          do computation to determine next thread
          do coroutine jump to next thread
          /* when I get back ... */
      } while( gotKicked[thislwp] );
      at some point somebody does exitsys ...




<<<
NOTES FOR REMOVING RUNNING THREAD FROM READY QUEUE:

NOTE: run exclusive is going to have mp memory model problems!

PCR←ThPvt←RemSelfRunQ() is a pretty good indication ... called in 4 places.

Active thread slot data entries are in a circular list headed by PCR←ThPvtImpl←active.
A single field tsd←next puts slotdata entries on either ML/CV or run lists;
the switcher can test whether curr thread is on a list and if not put it
on the appropriate run list.  This means inactive tsd's just go on another
thread queue, and aren't linked together by overloading the active slot data
links.

PCR←ThPvt←MoveSelfRunQ XXX !!!

Do CheckIO, checkTimeout, checkChildren possibly switch?  If so, they'll
break horribly if switcher is called with curr thread on a wait queue?
maybe not -- think about this.  No, I think this is okay.

-- this is DONE, seems okay --
>>>

================================

two actions require rescheduling: blocking, and making runnable.

rescheduling may be appropriate for a different processor from the
one I'm running.

directed yield is still a reasonable concept, and affects *my* processor ...

Suppose there's a GblSched() proc:

    EnterSys()
    for(;;) {
        find next most deserving runnable thread
        if its priority <= that of lowest prio vp break
        n++
    }
    send RESCHED to the n lowest pri vp's


Note: when a signal is sent to a thread, it's either a trap (in which case
the thread is guaranteed to be on the stack below the host sig handler,
and will look for it immediately) or was sent by another thread (in which
case we need to force that thread to RESCHED?!???)

Suppose we say every host signal results in a resched of its processor.
The above GblSched() proc is still useful; we reserve a host sig for RESCHED.

If we have a VM-based virtual dirty implementation, then SEGV may not
involve a resched?  Probably this is viable only if protection state is
shared by all lwp's.

When a host sig arrives:

  - A non-trap signal is put somewhere that the host sigs implementation
    can get at it ...
  - if interrupted thread was in critical section, the sig had better
    not be a trap or we're broken.
  - if interrupted thread was *not* in critical section, we're going
    to switch out of the trap handler.  It must do entersys, arrange
    for the host traps (after translation) to be delivered to the current
    thread, then switch.  Must arrange to have host sigs enabled right
    on threads switched to and from