XEROX Lisp Library Packages 2 4 1 1 TCP/IP 1 4 4 4 This document describes the Interlisp-D implementation of the TCP/IP suite of networking protocols. These protocols were developed under the auspices of the Department of Defense for the purpose of standardizing communication mechanisms within such DoD networks as the ARPAnet. The entire protocol family is documented in a collection of working papers known as RFC's, or Requests for Comments; all RFC's are available online from {SRI-NIC}RFCnnn.TXT through FTP anonymous login. We will refer to appropriate RFC numbers throughout this document as we introduce new protocols. Installation instructions are to be found at the end of this document. TCP/IP PROTOCOL LAYERS The TCP/IP family consists of four principal protocol layers: LINK LAYER The physical link layer, the medium for transferring packets between hosts, is assumed to be any medium capable of transporting packetized data between hosts. Common link layers in this family include the Ethernet and the ARPAnet. The Address Resolution protocol (AR) is supplied to enable hosts to map between internet addresses and link layer medium addresses. For example, the internet layer IP protocol (see below) uses a 32-bit combined unique host and network address; the host address field is of variable size and depends on the pattern encoded in the high order bits of the address. On the other hand, the (10 MB) Ethernet uses a fixed-size 48-bit unique host address. The address resolution protocol, documented in RFC826, allows hosts to dynamically discover the link layer address equivalents of other internet hosts. INTERNET LAYER The internet layer is responsible for routing packets between hosts. Unlike the link layer, the internet layer is capable of moving packets between hosts which are not connected to the same network. The term IP in TCP/IP refers to "Internet Protocol," the name of the protocol which performs this task in the TCP/IP family. IP is documented in RFC791. IP is not assumed to be error-free; packets may be lost or duplicated while moving through the Internet; it is the responsibility of the transport layer (see below) to guarantee perfect delivery should the client require it. IP also depends on an associated protocol called ICMP, the "Internet Control Message Protocol." ICMP is responsible for for handling exceptional conditions which arise between hosts using IP. Such conditions include the inability to deliver packets, errors in packet formats, etc. ICMP is documented in RFC792. TRANSPORT LAYER The transport layer is responsible for assuring error-free, duplicate-free, sequenced delivery of packets between communicating processes. The most common transport layer is TCP, the "Transport Control Protocol." TCP maintains the appearance of a perfect byte stream between processes. TCP is documented in RFC793. An unreliable transport layer called UDPthe "User Datagram Protocol"is also documented. UDP allows for packet exchange between communicating processes, but makes not attempt to guarantee delivery, suppress duplication, etc. Clients of UDP must provide their own error-recovery mechanism if necessary. UDP is documented in RFC768. APPLICATION LAYER A large number of applications currently exist in the TCP/IP family. The most common applications are: FILE TRANSFER Two principal file transfer applications are in use: FTP, based on TCP and documentated in RFC765; and TFTP (the "trivial file transfer protocol"), based on UDP and documented in RFC783. Both are implemented in Interlisp-D, and we shall discuss each at greater length later in this document. VIRTUAL TERMINAL The TELNET protocol, documented in RFC854, specifies the protocol for virtual terminal interaction between a user and a remote system. The Interlisp-D Chat system will use the TELNET protocol to connect to TCP-only hosts. MAIL SMTP (simple mail transfer protocol) enables the delivery of mail between system elements using TCP. It is not currently implemented in Interlisp-D. SMTP is documented in RFC821. INTERLISP-D FILES The files which implement the TCP/IP protocol suite are divided into two classes: those which implement low-level functionality, normally not of interest to general users, and those which implement higher-level functionality for user programs (either application or transport layer protocols). All files reside in the LispLibrary directory. The higher-level functions reside in these files: TCP The TCP layer. Implements TCP streams, based on the buffered TCP device (e.g. BIN runs in microcode). TCPDEBUG Routines to help debug TCP and TCP-based applications. TCPFTP The TCP-based file transfer protocol. Creates a new virtual I/O device, allowing transparent filing operations with TCP-only hosts. TCPCHAT Implements the TELNET protocol for the Chat system. UDP The UDP layer. TFTP Implements the TFTP protocol. Creates a buffered TFTP device to allow efficient bulk transfer between hosts. The low-level functions reside in the following files: LLIP Implements the IP layer. LLICMP Implements ICMP for IP. LLAR Implements AR (Address Resolution) for 3 and 10 MB Ethernet. HTE Implements functionality necessary to parse RFC810-style HOSTS.TXT files. This allows name to address translation within the Interlisp-D host, as there is currently no satisfactory network-accessible service to perform this task. LLICMP, LLAR, and HTE are loaded automatically by LLIP when necessary. LLIP is itself loaded by TCP or UDP when necessary. We now document each of these files: TCP TCP implements the transport control protocol for Interlisp-D. After TCP is loaded, Interlisp-D supports a TCP-stream, capable of bi-directional I/O to a remote system element. The following function are intended for use by applications programs: (TCP.OPEN DST.HOST DST.PORT SRC.PORT MODE ACCESS NOERRORFLG) [Function] Opens a TCP stream to DST.PORT on DST.HOST from SRC.PORT. MODE is either ACTIVE, meaning act as initiator of the connection, or PASSIVE, meaning to wait for a remote system element to initiate the connection. ACCESS is either INPUT, OUTPUT, or APPEND (OUTPUT and APPEND are treated in the same manner). If NOERRORFLG is non-NIL, TCP.OPEN will return NIL if the connection fails; otherwise, TCP.OPEN will call ERROR to signal failure. If TCP.OPEN succeeds, it returns a STREAM open as specified by ACCESS. The generic operations BIN, BOUT, PEEKBIN, BINS, BOUTS, READP, EOFP, OPENP, GETFILEPTR, FORCEOUTPUT, and CLOSEF may be performed on stream opened for suitable access. (TCP.OTHER.STREAM STREAM) [Function] Returns the STREAM open in the other direction with respect to STREAM (e.g., if STREAM is open for INPUT, TCP.OTHER.STREAM returns a STREAM open for OUTPUT, and vica-versa). (TCP.URGENT.EVENT STREAM) [Function] Returns an event upon which a user process may wait for URGENT data to arrive on STREAM. (TCP.URGENTP STREAM) [Function] Returns T is STREAM is currently reading URGENT data. (TCP.STOP) [Function] Disables the TCP protocol, closing all open TCP streams. (\TCP.INIT) [Function] (Re)Initializes the TCP package. \TCP.DEFAULT.RECEIVE.WINDOW [Variable] The default number of bytes allowed outstanding from the remote system. Initially 2000. TCPDEBUG TCPDEBUG implements tracing and test functions used to debug TCP and TCP-based applications. The following functions may be of interest to applications programmers: (TCPTRACE) [Function] Opens a trace window and attaches a menu to the window's top. The menu entries represent state changes or data elements to be traced; each entry is toggle: buttoning once will activate the trace of the particular element, and will gray-over the entry; buttoning a second time will deactivate the tracing and ungray the menu item. The following data elements/transitions may be displayed: Contents A line's worth of packet contents are displayed. The Incoming or Outgoing switch must be on. Incoming Incoming data is displayed. Outgoing Outgoing data is displayed. Checksums Checksums for each TCP segment are displayed. Time The time interval since the last action on the connection is displayed. Transitions State transitions on the TCP state machine are displayed. (PPTCB TCB FILE) [Function] The state of a TCP connection is printed. PPTCB is normally the INFO function for the process which monitors a connection; thus, buttoning INFO in the process status window will cause a window to pop up containing a report on the status of the associated connection. (TCP.ECHOTEST HOST NLINES) [Function] Opens a TCP connection to the TCP echo port on HOST, and sends NLINES of random text. The echo responses are displayed in a window. (TCP.ECHO.SERVER PORT) [Function] Starts a process acting as a TCP echo server on PORT (defaults to the TCP echo port. Other tracing and debugging functions may be found by examining the source file. TCPFTP TCPFTP implements a virtual I/O device which performs Interlisp-D filing operations transparently using the RFC765 FTP protocol. The standard filing operations of reading, writing, renaming, deleting, and directory enumeration are supported by the TCPFTP device. Random access filing, on the other hand, is not supported, as there is no protocol specification for performing random access operations on files. Thus, Interlisp-D operations such as recompiling of parts of files will not work. Once TCPFTP is loaded, filing operations should be transparent to users; no additional initialization need be performed. The following functions are available for debugging or extracting one's self from pathological states: (FTPDEBUG FLG) [Function] If FLG is T, opens a scrolling trace window which displays FTP commands as they are issued. PUPFTP commands will also be displayed in this window (the window is the value of FTPDEBUGLOG). (\TCP.BYE HOST) [Function] Breaks an FTP connection to HOST. TCPCHAT TCPCHAT implements the TELNET protocol for virtual terminal I/O between Interlisp-D and a remote system. Once loaded into Interlisp, the standard Chat system will use TCP TELNET to communicate with hosts which are believed to support the protocol. No user-callable functions reside in this package. UDP UDP implements the user datagram protocol. The following functions are meant to be called by client applications: (UDP.INIT) [Function] Initializes the UDP package. This function is normally called when UDP loaded, and should not need to be called again under normal circumstances. (UDP.STOP) [Function] Disables the UDP package, closing any open UDP sockets. (UDP.OPEN.SOCKET SKT# IFCLASH) [Function] Opens a socket for UDP operations. SKT#, if supplied is a 16-bit number, and will default to a number between 1000 and 65535. IFCLASH specifies what to do if the requested socket is already open, and is handled as in OPENPUPSOCKET and OPENNSOCKET (see IRM). Returns an instance of an IPSOCKET. (UDP.CLOSE.SOCKET IPSOCKET NOERRORFLG) [Function] Closes an open IPSOCKET. If IPSOCKET not an open socket, and NOERRORFLG is NIL, an error will occur; otherwise, NIL is returned if the socket is not active, and T is return if the socket was active. Any remaining packets on the socket's input queue are discarded when this function is called. (UDP.SOCKET.EVENT IPSOCKET) [Function] Returns an event which a process may use to wait for packet arrival on IPSOCKET. (UDP.SOCKET.NUMBER IPSOCKET) [Function] Returns the socket number of IPSOCKET. (UDP.GET IPSOCKET WAIT) [Function] Returns the next packet waiting on IPSOCKET. If no packets are waiting, does one of the following based on the value of WAIT: NIL Returns immediately. T Waits forever for a packet to arrive. A FIXP Waits upto WAIT milliseconds for a packet to arrive, and returns NIL if none arrived during that time. Thus, this function is like GETPUP and GETXIP. (UDP.SETUP UDP DESTHOST DESTSOCKET ID IPSOCKET REQUEUE) [Function] Initializes a fresh packet (as returned from \ALLOCATE.ETHERPACKET). The packet will be send to DESTSOCKET on DESTHOST. ID is a number to be placed in the IP header ID field (zero is fine). REQUEUE specifies what to do with the packet after it is sent; NIL (the default) means no special treatment; FREE means to release the packet and return it to the free packet queue; any instance of a SYSQUEUE will cause the packet to be queued on the tail of the specified queue. UDP.SETUP initializes all IP and UDP fields, and sets the packet up as being a minimum length UDP packet. (UDP.SEND IPSOCKET UDP) [Function] Sends UDP, a UDP-formatted packet out from IPSOCKET. (UDP.EXCHANGE IPSOCKET OUTUDP TIMEOUT) [Function] Sends OUTUDP out from IPSOCKET, and waits TIMEOUT milliseconds for a response; returns NIL if no response came in during the specified interval, or the packet which did come in during that time. Clears the socket's input packet queue before waiting for a packet to arrive. (UDP.APPEND.BYTE UDP BYTE) [Function] Appends BYTE to the UDP data portion of UDP, and increments the UDP and IP length fields by 1. (UDP.APPEND.WORD UDP WORD) [Function] Appends WORD to the UDP data portion of UDP, and increments the UDP and IP length fields by 2. (UDP.APPEND.CELL UDP CELL) [Function] Appends CELL to the UDP data portion of UDP, and increments the UDP and IP length fields by 4. (UDP.APPEND.STRING UDP STRING) [Function] Appends STRING to the UDP data portion of UDP, and increments the UDP and IP length fields by the length STRING. TFTP TFTP implements the trivial file transfer protocol. This protocol is useful for transferring unimportant files rapidly (for example, between workstations and printers). Two user-callable functions exist: (TFTP.PUT FROM TO PARAMETERS) [Function] Sends a file to a TFTP host. FROM may refer to any accessible file; TO must refer to a file accessible via TFTP. No attempt is currently made to translate between Interlisp filename syntax and remote system filename syntax for TO; for example, if TO resides on a Unix host, it would take syntax like: {HOST}/dir/subdir/filename PARAMETERS is currently list of parameters in the same format as used by OPENFILE in MACHINE.DEPENDENT.PARAMETERS; for example: ((EOLCONVENTION 1) (TYPE TEXT)) (TFTP.GET FROM TO PARAMETERS) [Function] Gets a file from a TFTP host. FROM must be a file accessible by TFTP; TO may be any file. The filename syntax caveats for FROM are the same as for TO in TFTP.PUT. PARAMETERS is also as in TFTP.PUT. For users with applications requiring explicit control over the transfer of data using TFTP, the following functions exist: (\TFTP.OPENFILE FILENAME ACCESS RECOG PARAMETERS) [Function] Returns a STREAM to open for ACCESS on FILENAME. PARAMETERS is the usual format; TYPE is the only recognized parameter (BINARY opens a stream in octet format; TEXT [the default] opens a stream in netascii formatsee RFC783). BIN, BOUT, READP, EOFP, etc may be used on this stream. Note that the stream is not RANDACCESSP. (\TFTP.CLOSEFILE STREAM) [Function] Closes the open stream. This is normally useful for streams open for OUTPUT; for INPUT streams, end-of-file will occur eventually. LLIP For users planning implementations on top of IP, the following low-level functions are available. We document here only three categories of functions: IP socket access, IP packet building, and IP packet sending; other functions should not be of interest to IP clients. (\IPINIT) [Function] Reinitializes the IP world; e.g. after some catastrophe. (STOPIP) [Function] Disables IP. (\IP.ADD.PROTOCOL PROTOCOL SOCKETCOMPAREFN NOSOCKETFN INPUTFN) [Function] Defines a new a IP-based protocol. The lowest-level IP functions maintain a list of active protocols, and perform packet-delivery based on the existence of open sockets for protocols of received packet types. PROTOCOL is a protocol number, a number between 1 and 255. The following protocols are defined and should not be disturbed: TCP 6 ICMP 1 UDP 17 SOCKETCOMPAREFN is a function of two arguments, an IP packet which has just been received, and an open IPSOCKET. This function should return NIL if the packet does not belong to the supplied socket, or T if it does. The function will typically be interested in the IPSOCKET field of the IPSOCKET. NOSOCKETFN is a function of one argument, an IP packet which has just been received. Its purpose is to handle received packets for which no socket can be found. If NOSOCKETFN is NIL, the default function, \IP.DEFAULT.NOSOCKETFN, will be used; this function simply returns an ICMP message indicating the socket is unreachable. INPUTFN is a function of two arguments, a received IP packet and an open IPSOCKET. The INPUTFN is supposed to handle reception of packets when their destination socket has been found. If INPUTFN is NIL, the default function, \IP.DEFAULT.INPUTFN, will be supplied; this function does the following: Enqueues the received packet on the IPSQUEUE field of the IPSOCKET if the current queue length (stored in the IPSQUEUELENGTH field) is less than the allocated length (stored in the IPSQUEUEALLOC) field. Increments the IPSQUEUELENGTH field. Notifies the event stored in the IPSEVENT field. \IP.ADD.PROTOCOL returns an IPSOCKET datum, which represents the active protocol; it is not in fact a useful IPSOCKET, and may be safely ignored. (\IP.DELETE.PROTOCOL PROTOCOL) [Function] Deactivates a protocol with protocol number PROTOCOL. Any open sockets are closed. (\IP.OPEN.SOCKET PROTOCOL SOCKET NOERRORFLG SOCKETCOMPAREFN NOSOCKETFN INPUTFN) [Function] Attempts to open an IPSOCKET for protocol PROTOCOL. SOCKET is the identifying information for this socket; this quantity will be EQUAL-compared with other sockets open on PROTOCOL. Should a match be found, an error will occur, unless NOERRORFLG is T, in which case the existing socket will be returned. SOCKETCOMPAREFN, NOSOCKETFN, and INPUTFN may be supplied to override the functions specified when the protocol was defined; they are not normally useful, however. (\IP.CLOSE.SOCKET SOCKET PROTOCOL NOERRORFLG) [Function] Closes a socket open on PROTOCOL. SOCKET is the same quantity passed to \IP.OPEN.SOCKET; it is currently not an instance of an IPSOCKET. If NOERRORFLG is T, an error will not occur if the socket is not found. The following functions are useful for placing bytes into IP packets (as allocated by \ALLOCATE.ETHERPACKET. Note that most applications will probably want to define a blockrecord to overlay the data portion of an IP packet; an example of such a blockrecord is provided at the end of this section. (\IP.APPEND.BYTE IP BYTE INHEADER) [Function] Appends BYTE to the IP data portion of IP, and increments the IP length field by 1. If INHEADER is T, the IPHEADERLENGTH field is appropriately incremented so that the bytes appear to have been appended to the options portion of the IP header. There must not be any data bytes in the data portion of the packet if this function is to work correctly. (\IP.APPEND.WORD IP WORD INHEADER) [Function] Appends WORD to the IP data portion of IP, and increments the IP length field by 2. INHEADER is as in \IP.APPEND.BYTE. (\IP.APPEND.CELL IP CELL INHEADER) [Function] Appends CELL to the IP data portion of IP, and increments the IP length field by 4. INHEADER is as in \IP.APPEND.BYTE. (\IP.APPEND.STRING IP STRING) [Function] Appends STRING to the IP data portion of IP, and increments the IP length field by the length STRING. Finally, we document the functions used to transmit IP packets: (\IP.SETUPIP IP DESTHOST ID SOCKET REQUEUE) [Function] Initializes IP. This function should be called just after IP is obtained from \ALLOCATE.ETHERPACKET; if this is not done, the append functions above will fail. DESTHOST is the 32-bit IP address to which this packet will be sent. ID is an arbitrary 16-bit quantity which will become the IPID field of the packet. SOCKET is the open IPSOCKET from which the packet will be sent. REQUEUE defaults to FREE, and controls the disposition of the packet after transmisson (see the IRM for the documentation of SETUPPUP or FILLINXIP). (\IP.TRANSMIT IP) [Function] Tries to send IP. Performs IP checksum algorithm prior to sending. Returns NIL if successful, else returns a status indication, such as NoRouting or AlreadyQueued. This function is like SENDPUP and SENDXIP, except that no socket argument is required. HTE HTE provides functions for parsing HOSTS.TXT files as documented by RFC810. This file is loaded automatically by LLIP, and is used by \IPINIT to read in the initial file, HOSTS.TXT. The following variable and function may be of interest: HOSTS.TEXT.DIRECTORIES [Variable] The search path for the file HOSTS.TXT. This variable is initialized to NIL, and thus, the search path to be used is by default DIRECTORIES. (\HTE.READ.FILE FILE WANTEDTYPES) [Function] Reads a HOSTS.TXT file. WANTEDTYPES is a list of types, drawn from the set {HOST, NET, GATEWAY}, to be read from the file; types not specified in WANTEDTYPES are ignored. WANTEDTYPES defaults to (HOST). INSTALLING TCP/IP To install TCP/IP, place the files on the Lisp Packages directory (e.g. any directory on the search path LISPUSERSDIRECTORIES). Create a HOSTS.TXT file containing entries for the TCP/IP hosts needed by the user community, and place the file on a directory contained in the DIRECTORIES search path. The following is a sample HOSTS.TXT file: ; DoD Internet Host Table ; 6-Apr-84 ; ; Changes, corrections, comments or questions to ; (HOSTMASTER@SRI-NIC) ; ; The format of this file is documented in RFC 810, "DoD Internet ; Host Table Specification", which is available on-line at SRI-NIC ; as the file ; [SRI-NIC]RFC810.TXT ; It may be retrieved via FTP using username ANONYMOUS with ; any password. ; ; NOTE CAREFULLY: RFC 810 has been slightly revised since the ; original ; version was written. In particular, the version printed in the ; "Internet Protocol Transition Notebook" does not document the ; added "machine type" field (between the host-name and system-name ; fields). ; ; The format for entries is: ; ; NET : NET-ADDR : NETNAME : ; GATEWAY : ADDR, ADDR : NAME : CPUTYPE : OPSYS : PROTOCOLS : ; HOST : ADDR, ALTERNATE-ADDR (if any): HOSTNAME,NICKNAME : CPUTYPE : ; OPSYS : PROTOCOLS : ; ; Where: ;; ADDR = internet address in decimal, e.g., 26.0.0.73 ;; CPUTYPE = machine type (PDP-11/70, VAX-11/780, FOONLY-F3, C/30, ;; etc.) ;; OPSYS = operating system (UNIX, TOPS20, TENEX, ITS, etc.) ;; PROTOCOLS = transport/service (TCP/TELNET,TCP/FTP, etc.) ;; : (colon) = field delimiter ;; :: (2 colons) = null field HOST: 36.9.0.9 : VAX-A! : VAX-11/750 : UNIX : TCP/TELNET,TCP/FTP : HOST: 36.9.0.30 : 1108-A : DANDELION : D : : This file defines two hosts, a VAX-11/750 host called VAX-A!, and an 1108 host called 1108-A. Note that if any host is accessible via another network protocol (e.g. PUP or NS), you may desire to call the host an unambiguous name when it is accessed via TCP; you can do this by giving it such an unambiguous name in the HOSTS.TXT file (in this case, we've appended a "!"). Once the software is installed on the appropriate directory, simply performing a LOAD operation on any of the application files (e.g. TCPFTP, TCPCHAT, TFTP, etc) should load the remaining necessary files and initialize the software. (LIST ((PAGE NIL NIL (0 0 612 792) ((FOLIO NIL (PARALOOKS (QUAD CENTERED) CHARLOOKS (SUPERSCRIPT 0 SIZE 10 FAMILY HELVETICA OVERLINE OFF STRIKEOUT OFF UNDERLINE OFF SLOPE REGULAR WEIGHT MEDIUM)) (282 42 72 36) NIL) (HEADING NIL (HEADINGTYPE BOO) (84 744 444 36) NIL) (TEXT NIL NIL (84 96 456 600) NIL))) (PAGE NIL NIL (0 0 612 792) ((FOLIO NIL (PARALOOKS (QUAD CENTERED) CHARLOOKS (SUPERSCRIPT 0 SIZE 10 FAMILY HELVETICA OVERLINE OFF STRIKEOUT OFF UNDERLINE OFF SLOPE REGULAR WEIGHT MEDIUM)) (282 42 72 36) NIL) (HEADING NIL (HEADINGTYPE BOO) (84 744 444 36) NIL) (TEXT NIL NIL (84 96 456 600) NIL))) (PAGE NIL NIL (0 0 612 792) ((FOLIO NIL (PARALOOKS (QUAD CENTERED) CHARLOOKS (SUPERSCRIPT 0 SIZE 10 FAMILY HELVETICA OVERLINE OFF STRIKEOUT OFF UNDERLINE OFF SLOPE REGULAR WEIGHT MEDIUM)) (282 42 72 36) NIL) (HEADING NIL (HEADINGTYPE BOO) (84 744 444 36) NIL) (TEXT NIL NIL (84 96 456 600) NIL)))))` ` 88 488 PAGEHEADINGBOO HELVETICA HELVETICA HELVETICA  HELVETICA HELVETICA  HELVETICA HELVETICA LOGO HELVETICA HELVETICA    HRULE.GETFN HELVETICA  HRULE.GETFN HELVETICA HRULE.GETFN HRULE.GETFN HELVETICA HRULE.GETFN HELVETICA HRULE.GETFN HRULE.GETFN HRULE.GETFN HELVETICA> Z&:?Nh%V2k@<s7B 1 X v? ?X Q   #9!'Yh%%8MF    / @ 0!Q5    -s8  "X|$  ^ G   #N( X/ + a Ej  "   N 4 4 4  <  #4 v!  $3 |   V.b 9 # u  Q^i(4 , = *q8 ;  z  A! ;+ / , ,  3@   -e>R; "  o  V 1BC(<? B@D >F 8D">= C-[7 z