A Layman's Explanation of High Speed Modems Stan Simmons Business Forms Estimating Systems, Inc. April 4, 1991 High speed modems have 4800 bps or faster data transfer rates. Until recently most high speed modems used proprietary modulation schemes. In other words, one manufacturers' modem would not connect to another manufacturers' modem. Standards exist now that allow the interconnection of different manufacturers modems. MODULATION STANDARDS In the mid 1980's the International Consultative Committee on Telephone and Telegraph (CCITT in French) established the V.32 standard. The V.32 standard describes how modems should "talk" to each other using two-way signaling at 4,800 and 9,600 bps over dial-up telephone lines. Unfortunately the V.32 standard did not provide a method for error control. Since V.32 signaling is more sensitive to noise and echoes on the telephone line than lower-speed protocols are, you need an error control scheme to retain accuracy. In early 1991, the CCITT issued the V.32bis standard. The V.32bis standard adds 7,200, 12,000, and 14,400 bps transfer rates and a faster renegotiation protocol to the V.32 standard. It is important to understand that the V.32 standard primarily describes the electrical signaling scheme used over the telephone wire. Other standards, such as MNP, V.42, and V.42bis, describe actions taking place above the level of electrical signaling. So you can have modems using different combinations of signaling and error correction protocols. ERROR CONTROL STANDARDS Microcom developed its own standard for asynchronous data error control, Microcom Network Protocol (MNP). MNP Class 4 is the most commonly used version of this family of error control. The error checking operates independently of the signaling scheme used by the modem. In 1988, the CCITT issued a hardware-implemented asynchronous error correction standard called V.42, which describes two error correction schemes. The primary protocol is Link Access Procedure for Modems (LAPM). The secondary protocol is functionally equivalent to MNP Class 4. The LAPM method offers slightly better error recovery and reliability than MNP Class 4. While the V.42 and MNP Class 4 protocols help maintain reliability, they do little to improve throughput. Both protocols convert asynchronous data characters to a synchronous data stream, making it a bit oriented protocol instead of character oriented. Most asynchronous characters consist of one start bit, eight data bits, and one stop bit, for a total of ten bits per character. V.42 removes the start and stop "framing" bits, which results in a 20% increase in efficiency. However, in order for the protocol to work, V.42 adds about 12% in overhead back into the transmission. The resulting 8% cushion helps maintain full transfer speed during periods of moderate error correction activity (usually caused by noisy telephone lines.) For all practical purposes, the result of the V.42 link is an error free transmission. Using the 16 bit redundancy check, it will detect every error that is 16 bits or smaller, with 100% probability. As a result, the chances of an error occurring are so small that you can, in practice, ignore them. DATA COMPRESSION STANDARDS The next step in increasing throughput involves data compression. Data compression can be used to provide a modem with an effective data throughput rate that is higher than the modem's bps transmission speed. The amount of this increase in throughput will depend largely on the type of data being transferred. Microcom introduced the MNP Class 5 data compression protocol. Software supporting the MNP Class 5 protocol offers the ability to compress files to half their original size during transmission, thus providing a 100% increase in speed. However, 80-85% increases in speed are more typical. MNP Class 5 requires concurrent error correction using MNP Class 4. In late 1989, the CCITT issued the V.42bis standard, describing how to implement data compression in hardware. V.42bis uses the Lempel-Ziv compression algorithm and offers a 35% greater data compression than MNP Class 5. For 9600 bps modems this means a potential throughput of 38,400 bps. For most file transfers, however, a throughput of 19,200 bps on non- compressed files can be expected. The V.42bis standard adapts to the data flow more quickly than MNP Class 5, turning data compression on and off as required. This gives it an advantage over MNP Class 5 when transmitting previously compressed files, since the MNP Class 5 compression algorithm can cause compressed files to expand, reducing throughput. V.42bis simply passes pre-compressed data through without trying to compress it. V.42bis compression software only works with hardware that uses the V.42 error correction protocol. DATA TRANSFER When using a file transfer protocol to send and receive data, the type of protocol used will have a big effect on the speed gain due to compression. In general, a protocol that uses long data blocks (the longer the better) will transfer files quicker. To take full advantage of MNP or V.42 error correction, you should select the software's no-error- correction option. To make use of the data compression, the modems need to be driven at full capacity. In other words, the data needs to be present at enough volume (file transfers and batch operations) and speed to get maximum compression benefits. For a V.32bis connection with V.42bis compression the serial ports should be set for 38,400 bps. In order for data compression to take place, both the answer and originate modems at each end of the telephone line must have compression and error correction enabled. If one unit does not have data compression enabled, only error correction takes place. Overall, on-the-fly compression with V.42bis on a V.32bis connection is the most desirable and economical mode of operation for most applications. COMPUSERVE At the present time all of CompuServe's 9600 bps modems are US Robotics Dual Standard modems. These modems support the V.32 modulation standard, the V.42 error correction protocol, and V.42bis & MNP level 5 compression protocols. The US Robotics Dual Standard is also upgradeable to V.32bis modulation. CompuServe sets the modems to V.32 mode, and leaves both the MNP level 5 and V.42bis data compression enabled. But, even when using compression, nothing is gained during normal operation because the ports are locked at 9600 bps. If an error occurs during transmission the re-transmitted frames will be compressed. The result is that the throughput will remain close to the maximum port speed even with some phone line noise or other interference. CompuServe's Host-Micro Interface (HMI), used by the CompuServe Information Manager (CIM) and other CompuServe software products, uses B+ protocol full-time as the transport layer, and results in a measure of data compression due to "bit packing" of transmitted data into a smaller number of bits, using a technique similar to V.42. As with V.42, it primarily acts to maintain throughput at a high level by offsetting the protocol overhead, rather than increasing throughput significantly beyond that achieved at the same baud rate without compression or error detection. CIM and other HMI products enjoy continuous error detection and correction as a function of the B+ protocol transport layer, and this error correction, being integral to the HMI, cannot be disabled. As a result, the use of other error correction protocols such as MNP-4 or V.42 "in series" with the software's own error correction may be, in many cases, redundant and unnecessary, and can actually slow down data transfer and/or interfere with flow control. For this reason, it is sometimes suggested that hardware error correction not be activated when using HMI products. CompuServe does not, at this time, permit data transfer rates at the port above 9600 baud. The reasons for this have mainly to do with the "backbone" of CompuServe's network, which handles the overall data traffic for many users simultaneously, and the need to manage the expansion of local nodes and the backbone itself in tandem. CompuServe's dial-up data network currently includes approximately 20,000 "1200 bps equivalents," each representing the load on the network presented by a port operating at 1200 baud. Logically, a port operating at 9600 baud represents 8 "1200bEs" in terms of the demands placed upon the network. Currently, 9600 baud ports represent approximately 3% of the total number of ports, but account for as much as 15% of network load. During the current fiscal year, CompuServe plans to expand the number of 9600 baud ports in the network by a full 200 percent. Overall, the impact on the CompuServe network "backbone" will be an increase in total data traffic by as much as 50 percent. Such an increase requires effective planning and more than a little control over how, when and where the expansion is performed. If CompuServe was to allow the use of data compression to increase the effective data rate at the port, and hence demand on the network, by a factor of as much as 4:1, the effective increase in network load as a result of expansion of the 9600 baud ports could easily jump to 200 percent of the current load. Needless to say, that's not something that can be done with a "flip of a switch." The network "backbone" must be expanded in tandem with the addition of 9600 baud ports; there's much more involved than simply hooking up a 9600 baud modem at the port end. Hardware and software must be replaced, enhanced and reconfigured, and new facilities brought on-line on the host end to deal effectively with the increased amount of data coming into the computer centers. While the primary benefits of data compression are not available to CompuServe users now, they will be available in the not-distant future. For now, CompuServe's primary concern is to make sure that the expansion of the 9600 baud service does not negatively impact other users of the network, while providing maximum benefit from the expansion and the availability of basic 9600 baud service. ELECTRONIC BULLETIN BOARD SYSTEMS Electronic Bulletin Board System's (BBS's) have been around for a number of years now, and their numbers are growing on an almost daily basis. All major cities boast several BBS's, and many smaller cities have at least one BBS. Most BBS's limit each user's time on-line, so it is in your best interest to transfer as much data as possible while you are on-line. High speed modems can allow as much as 16 times the data transfer per unit of time over standard 2400 bps modems. US Robotics, and several other modem manufacturers, often provide high speed modems to established BBS system operators (SysOps) at or below cost. These promotions are usually in the best interest of the manufacturer. The SysOps become familiar with the product and recommend it to the users. If you are connecting to a BBS over long distance a high speed modem the savings on your phone bill can greatly offset the initial cost of high speed data transfer. The best way to save on telephone line charges is to use scripts or front end programs to automate your BBS activities. Many BBS's use a similar format for message and upload/download areas. As BBS's become more standardized perhaps better front end programs for the various BBS's will be written. DISCLAIMER The information in this document is correct to the best of my knowledge. I make no warranty as to accuracy of the information, nor do I accept any responsibility for the use or misuse of it. This document may be freely copied and distributed in any form, as long as it is presented unaltered, in its entirety and not for profit. Copyright (c) 1991, Stan Simmons.