token1, token2: Rope.ROPE;
WHILE
TRUE
DO
ENABLE IO.EndOfStream => EXIT;
token2 ← NIL;
[token: token1] ← IO.GetTokenRope[stream, WhiteSpace];
IF token1 = NIL THEN LOOP;
Things are a bit tricky if the first letter of the token is "-". In this case, the argument could be either a switch (like in Unix, remember?), or it could signify the beginning of a comment. If it's not a comment, make sure that the rope doesn't contain more than one more character, and if it does then chop off the stuff after the first character into a separate token.
length ← Rope.Length[token1];
IF length >= 2
THEN
IF (Rope.Fetch[token1, 0] = '-)
AND (Rope.Fetch[token1, 1] = '-)
THEN EXIT;
IF (Rope.Fetch[token1, 0] = '-)
AND (length > 2)
THEN
BEGIN
token2 ← Rope.Substr[base: token1, start: 2, len: length-2];
token1 ← Rope.Substr[base: token1, start: 0, len: 2];
END;
Generate either one or two Arg objects for the token(s).
cur ← NEW[ArgRec];
cur.rope ← token1;
IF result = NIL THEN result ← cur;
IF prev # NIL THEN prev.next ← cur;
prev ← cur;
IF token2 #
NIL
THEN
BEGIN
cur ← NEW[ArgRec];
cur.rope ← token2;
prev.next ← cur;
prev ← cur;
END;
ENDLOOP;