Representation of links:
use scrollbar thumbnail with arrows? limited real estate in scrollbar, mult locations on a single line difficult to represent.
want picture of document with view of script, also want script-ordered view (which becomes more complex in the presence of branching.
Dan suggested using an entry keyword, with a way of generating something if the user doesn't specify one (for scripting ease): perhaps a creation timestamp? which may be useful anyhow. That leads to the idea of colored script building, analogous to voice editing.
Want to see the script entries in context, if possible => that leads to fisheye view idea.
Instead of a textual representation, use a popup window with frames? Script entry location is mouse-sensitive?
Intermedia uses a select loc1/start link/select loc2/stop link paradigm. Their links have names (descriptors) attached, so that a user can filter links to a desired set. A user issues the Follow command to go from one location to another (unclear what happens if there are multiple links from that spot/is this allowed? Looks like there would be multiple separate link marks, and user could follow whichever one was desired; of course only some of them might be visible at a given time.) From picture in paper, appears that linkage map shows only linkages between documents, rather than finer grain. Any selection can be a source/dest. Links appear to be bidirectional.
This brings up some implementation restrictions: by only allowing a location to be scripted (by an arbitrary number of script entries) or not - desire to avoid changing typesetting is a partial goal. Whether or not each scripted location is visible or only script beginnings is important. The former says that this is an annotation system that also supports sequences of annotations, while the latter says that you can only play a script from its start (choice of single-step vs. continuous, however). If this is an annotation system, then labelled and filtered links/actions become interesting, but the meaning of those things in a script is less clear: what does it mean to view every other action of a script -- the author can no longer rely on what the user has or hasn't seen.
Other ease of use issues:
a "start recording action/stop recording action" interface would be easier than typing text into an action field.
Want to capture "mouse" motions? Drive a Magnifier around the screen? Yow!!! Perhaps these will become special animations for a later implementation/design. This moves toward Windex, ToolDriver, user interface notions. Some similarity to Brad Myers's virtual mouse/UI by example stuff? Eric Bier's Gargoyle scripts? Could capture Inscript? but if the user had any different windows open, there would be trouble!!! If individual applications implemented scripts, or allowed themselves to be driven in that way, would be okay. In a sense, ToolDriver+Brad's virtual mouse is exactly what you want:
CreateVirtualMouse {or CreateMagnifier}
MoveTo[FindViewer[ScriptedDocsDesignNotes.tioga].Save]
Wait[.5 sec]
MouseButtonDown[yellow]
Wait[.1 sec]
MouseButtonUp[all]
If you can't do these things, in what sense are actions at all useful? Real demos need to drive the mouse (!?)
Contributions/distinctive aspects of scripted docs
can layer on top of existing (lengthy) documents
want this to work in an open environment, other windows on screen, etc.
There is a major difference between showing (coarse-grained) links between small hypermedia documents for browsing and showing (fine-grained) links between scripted locations inside larger documents for sequential viewing (it's a much harder visualization problem). A generalized annotation system, with actions, could be implemented under some sort of button in the document.
actions are open-ended
has a variety of audio capabilities
can specify a sequence of actions/commands => toward XDE ToolDriver
Problems/outstanding design decisions:
1. What does a scripted location/annotation look like in a document?
a. Intermedia: a small dot represents a link, apparently a separate dot for each link.
b. If we want annotations to be playable outside a script (which may not make sense -- a previous scripted location may need to have set some variable or opened a file or something), then each one must be separately visible. Certainly if there were such a annotation/button capability, then scripts could make use of them. If annotations aren't playable outside scripts, then Suresh's suggestion to make only the script starting location visible has merit.
c. One of Stephen's goals was to make voice annotations noninvasive to a document's appearance. Thus you can't have more than one on a given character. They can also only be placed on a single character. (When a selection is longer than a single character, TiogaVoice will place the annotation on the last character.) A generalized annotation capability would allow multiple annotations per character, with tags on the annotations so that they can be filtered. (Could easily be added to TiogaVoice through char props.)
2. Are there attributes other than script names for a scripted location?
a. Certainly action annotations, or annotations in general,
3. How does one insert/specify a new entry in a script?
a. Intermedia: select loc1/start link/select loc2/stop link paradigm. A user issues the Follow command to go from one location to another.
b. Current design: each entry has a sequence number. The user edits the sequence number field (and other fields, such as action & timing) in the script tool as desired, selects the desired location, and then clicks AddEntry. The entry is added to the script after that sequence number. This and the above show only the current entry (but can do FindEntry on prev or next to discover info about them).
c. Have scripting actions automatically build up a textual script in an associated viewer. To map from the document to the associated script entries, want script entries to be visible (at least during script writing).
4. How to show a script to a writer? to a reader? (related to 3 above)
a. Intermedia: a map of a hypermedia document is a thumbnail sketch of each portion (they tend to be small), with the links between the portions shown (apparently not to specific places in each portion).
b. Connectivity:
c. Document context:
d. Document locality & sequentiality:
e. All, but not very visual: textual representation, with ways to map from script to document & vice versa.
5. A problem with the design as outlined in the Scripted Docs draft for scripts across multiple docs was how the first doc in a script was identified (seqNum = 1 for that script, chain in whatever direction needed to get there -- a disaster, could be very slow to find!). A much better plan would be to store <script name, unique id, script starting file> in a system-wide (LoganBerry) database. (script name may not be necessary here - can just store that in doc header, and use scriptUID otherwise.) Need to consider what can profitably be stored in LoganBerry dbs: whole scripts (just store script entries in doc)?
Dan suggests that you might store most of scripts in db, so that you can overlay a script on an immutable file version. That is, you use entry uids at scripted locations iff they are there. When they are, the file is editable without destroying the script; otherwise not. From the user's viewpoint, you would like it to be easy to specify which kind you are creating, and you would like to be able to change kinds in the middle.
Dan also had an idea about using scripts for debugging: collect breaks into groups (say for logging), then connect them with a named script so that they can be browsed easily. All of this also suggests that you want to (in addition to other capabilities) be able to visit all scripted locations and perform some action (registered somehow, in tool, ??). In this case, it might be SetBreak, or it might be some editing actions, etc. => scripting as a generalized grouping mechanism.
6. Implementation:
For all of the following, store list of <scriptUID, entryUID> on doc chars as $script property.
a. Storing (relevant) script linking info in doc:
store list [scriptID1, [eid,eid,eid,filename,eid,eid,filename,eid,eid]]
b. Storing script in (LoganBerry) db:
i. If store nothing in document, how do you discover that the document has scripts associated with it?
ii. Accelerator would store list of <scriptUID, entryUID> on doc chars as $scriptEntries property; list of scriptUID (possibly with script name) on doc root as $scriptIndex property.
iii. Store character positions in script in db?
7. Research aspects:
Generalized annotation mechanism
Active locations
initial: items that can be specified to CommandTool. XDE ToolDriver functionality, while backwards, would be useful. Also, the ability to drive a simulated mouse around to point things out could be very useful.
How to specify/load (correct versions of) program pieces
initial: assume programs installed unless they install themselves? Use StatementInterpreter or write standalone programs for real program fragments.
Editing annotated docs
initial: for internal scripts, use char props to allow editing. Copying, or changing/deleting named script entries (altering others repair themselves automatically?), are problems
Annotating public/private docs
initial: scripts stored in db. internal scripts use char props => docs are editable; external scripts use char positions (could use unique string? dumb, I think) => applies to unique version. Both types use unique entry ids
Multiple annotations at a single doc location
initial: allowed, and script/annotation tool is required to sort out which is which in this case. Visualization of overlapping locations is a problem using some sort of square; may be better with some sort of oval.
Generalized scripting mechanism
Multiple docs in a single script (w/ version mgmt)
initial: store doc global names in db w or w/o version, depending on internal/external script
Branching scripts
initial: use label or unique entry id; a bit clumsy
User interaction
initial: could do something in the tool - this one needs more thought
Generalized grouping mechanism
Applying a single action across a group of locations
initial: use script tool to type in desired action?
User interface research
Visualization of annotations/scripts/groups
Architecture to support user interaction
Ease of use
Simplicity/intelligibility of user model
8. User interface ideas:
To handle branching, with multiple annotations/entries at a single doc loc, have an operation that takes a doc loc and opens a window for each annotation/entry that refers to that loc (in the given script only? If not, then how to find public and private scripts? AHA! use LoganBerry public/private merging capability! User will have a private script db specified in user profile and system will have a public one, just as telephone directories.). Then the writer can point to the desired window and say branch to that entry. Need a way to copy script entries from one script to another -- could use the preceding mechanism, by allowing the windows to be editable, so the user can edit in the changed script name (system provides script uid?).