Intrastream Synchronization for Continuous Media Streams: A Survey of Playout Schedulers
---
paper_title: Synchronization of multimedia data for a multimedia news-on-demand application

paper_content:

We present a complete software control architecture for synchronizing multiple data streams generated from distributed media-storing database servers without the use of a global clock. Independent network connections are set up to remote workstations for multimedia presentations. Based on the document scenario and traffic predictions, stream delivery scheduling is performed in a centralized manner. Certain compensation mechanisms at the receiver are also necessary due to the presence of random network delays. A stream synchronization protocol (SSP) allows for synchronization recovery, ensuring a high quality multimedia display at the receiver. SSP uses synchronization quality of service parameters to guarantee the simultaneous delivery of the different types of data streams. In the proposed architecture, a priority-based synchronization control mechanism for MPEG-2 coded data streams is also provided. A performance modeling of the SSP is presented using the DSPN models. Relevant results such as the effect of the SSP, the number of synchronization errors, etc., are obtained.
---
paper_title: Flow synchronization protocol

paper_content:

Presents an adaptive flow synchronization protocol that permits synchronized delivery of data to and from geographically distributed sites. Applications include inter-stream synchronization, synchronized delivery of information in a multisite conference, and synchronization for concurrency control in distributed computations. The contributions of this protocol in the area of flow synchronization are the ability to synchronize over arbitrary topologies, the introduction of an adaptive synchronization delay, the flexibility to maintain multiple synchronization groups, and the use of a modular architecture that permits the application to tailor synchronization calculations to its service requirements. The authors take advantage of network protocols capable of maintaining network clock synchronization in the millisecond range. >
---
paper_title: Multipoint multimedia teleconference system with adaptive synchronization

paper_content:

This paper discusses major issues involved in the design and implementation of a multipoint multimedia conference system, such as system architecture, conference management, session control, and intramedium and intermedia synchronization. In particular, emphasis is given to conference management and adaptive synchronization algorithms. The management of multiparticipants is based upon a distributed architecture for greater flexibility. The proposed synchronization algorithm is adaptive to network changes, eliminates the need for a global clock, and is immune to the clock frequency drift, while its realization is very simple and the involved overhead is minimal. The essence of the algorithm is partitioning the vicinity of the arrival epochs of multimedia objects into three regions and counting arrivals at each region. The function of the synchronizer is to shift the playback clock (PBC) according to the individual counter contents. The ideas proposed are implemented within a teleconference system on the Ethernet/FDDI using the TCP/UDP. Experimental results show that the proposed synchronization algorithm performs well in our network testbed environment.
---
paper_title: The use of network delay estimation for multimedia data retrieval

paper_content:

Multimedia data have specific temporal presentation requirements. For example in video conferencing applications the voice and images of the participants must be delivered and presented synchronously. These requirements can be achieved by scheduling or managing system resources. We present a technique called limited a priori scheduling (LAP) to manage the delivery channel from source to destination for digital multimedia data. By using delay estimation a LAP scheduler can retrieve stored digital media spanning arbitrary networks with unspecified delays. The use of delay estimation also facilitates selective degradation of service in bandwidth and buffer limited situations. Such degradation enables the continuous real-time playout and synchronization of various media arriving from different sources. The performance of the LAP scheduler is described based on implementation and experimentation using Ethernet.
---
paper_title: Analysis and Optimal Design of a Packet-Voice Receiver

paper_content:

The analysis problems arising from a digital packet switched speech network are outlined in a wide statistical environment. The statistical models assumed are discussed with respect to practical applications. Particular attention is devoted to a formal description of the influence of the packet voice receiver on the system behavior in terms of the delay pdf. The optimization of the system performance, in order to obtain the best voice quality, is stated as an optimal control problem, and the analysis results play a central role in the construction of the objective functions. The problem is solved in a particular analytical environment. The outlined analytical results are supported by critical comments on their comparison with practical implementations, allowing the designer a better capability in handling any problems.
---
paper_title: Flow synchronization protocol

paper_content:

Presents an adaptive flow synchronization protocol that permits synchronized delivery of data to and from geographically distributed sites. Applications include inter-stream synchronization, synchronized delivery of information in a multisite conference, and synchronization for concurrency control in distributed computations. The contributions of this protocol in the area of flow synchronization are the ability to synchronize over arbitrary topologies, the introduction of an adaptive synchronization delay, the flexibility to maintain multiple synchronization groups, and the use of a modular architecture that permits the application to tailor synchronization calculations to its service requirements. The authors take advantage of network protocols capable of maintaining network clock synchronization in the millisecond range. >
---
paper_title: Techniques for Packet Voice Synchronization

paper_content:

Packet switching has been proposed as an effective technology for integrating voice and data in a single network. An important aspect of packet-switched voice is the reconstruction of a continuous stream of speech from the set of packets that arrive at the destination terminal, each of which may encounter a different amount of buffering delay in the packet network. The magnitude of the variation in delay may range from a few milliseconds in a local area network to hundreds of milliseconds in a long-haul packet voice and data network. This paper discusses several aspects of the packet voice synchronization problem, and techniques that can be used to address it. These techniques estimate in some way the delay encountered by each packet and use the delay estimate to determine how speech is reconstructed. The delay estimates produced by these techniques can be used in managing the flow of information in the packet network to improve overall performance. Interactions of packet voice synchronization techniques with other network design issues are also discussed.
---
paper_title: QoS impact on user perception and understanding of multimedia video clips

paper_content:

L ABSTIL4CT The widespread and increasing advent of multimedia technologies means that there must be a departure from the viewpoint that users expect a Quality of Service (QoS) which will only satisfy them perceptually. What should be expected of multimedia clips is that the QoS with which they are shown is such that it will enable the users to assimilate and understand the informational content of such clips. In this paper we present experimental resuks Iiiking users’ understanding and perception of multimedia ctips with the presentation QoS. Results show that the quality of video clips can be severely degraded without the user having to perceive any significant 10ss of informational content 1.1
---
paper_title: Dynamic Video Playout Smoothing Method for Multimedia Applications

paper_content:

Multimedia applications including video data require the smoothing of video playout to prevent potential discontinuity. In this paper, we propose a dynamic video playout smoothing method, called the Video Smoother, which dynamically adopts various playout rates in an attempt to compensate for high delay variance of networks. Specifically, if the number of frames in the buffer exceeds a given threshold (TH), the Smoother employs a maximum playout rate. Otherwise, the Smoother uses proportionally reduced rates in an effort to eliminate playout pauses resulting from the emptiness of the playout buffer. To determine THs under various loads, we present an analytic model assuming the Interrupted Poisson Process (IPP) arrival. Based on the analytic results, we establish a paradigm of determining THs and playout rates for achieving different playout qualities under various loads of networks. Finally, to demonstrate the viability of the Video Smoother, we have implemented a prototyping system including a multimedia teleconferencing application and the Video Smoother performing as part of the transport layer. The prototyping results show that the Video Smoother achieves smooth playout incurring only unnoticeable delays.
---
paper_title: Human perception of jitter and media synchronization

paper_content:

Multimedia synchronization comprises both the definition and the establishment of temporal relationships among media types. The presentation of 'in sync' data streams is essential to achieve a natural impression, data that is 'out of sync' is perceived as being somewhat artificial, strange, or even annoying. Therefore, the goal of any multimedia system is to enable an application to present data without no or little synchronization errors. The achievement of this goal requires a detailed knowledge of the synchronization requirements at the user interface. The paper presents the results of a series of experiments about human media perception that may be used as 'quality of service' guidelines. The results show that skews between related data streams may still give the effect that the data is 'in sync' and gives some constraints under which jitter may be tolerated. The author uses the findings to develop a scheme for the processing of nontrivial synchronization skew between more than two data streams.
---
paper_title: Multipoint multimedia teleconference system with adaptive synchronization

paper_content:

This paper discusses major issues involved in the design and implementation of a multipoint multimedia conference system, such as system architecture, conference management, session control, and intramedium and intermedia synchronization. In particular, emphasis is given to conference management and adaptive synchronization algorithms. The management of multiparticipants is based upon a distributed architecture for greater flexibility. The proposed synchronization algorithm is adaptive to network changes, eliminates the need for a global clock, and is immune to the clock frequency drift, while its realization is very simple and the involved overhead is minimal. The essence of the algorithm is partitioning the vicinity of the arrival epochs of multimedia objects into three regions and counting arrivals at each region. The function of the synchronizer is to shift the playback clock (PBC) according to the individual counter contents. The ideas proposed are implemented within a teleconference system on the Ethernet/FDDI using the TCP/UDP. Experimental results show that the proposed synchronization algorithm performs well in our network testbed environment.
---
paper_title: The Concord algorithm for synchronization of networked multimedia streams

paper_content:

Synchronizing different data streams from multiple sources simultaneously at a receiver is one of the basic problems involved in multimedia distributed systems. This requirement stems from the nature of packet based networks which can introduce end-to-end delays that vary both within and across streams. We present a new algorithm called Concord, which provides an integrated solution for these single and multiple stream synchronization problems. It is notable because it defines a single framework to deal with both problems, and operates under the influence of parameters which can be supplied by the application involved. In particular these parameters are used to allow a trade-off between the packet loss rates, total end-to-end delay and skew for each of the streams. For applications like conferencing this is used to reduce delay by determining the minimum buffer delay/size required.
---
paper_title: End-to-end delay analysis of videoconferencing over packet-switched networks

paper_content:

Videoconferencing is an important global application-it enables people around the globe to interact when distance separates them. In order for the participants in a videoconference call to interact naturally, the end-to-end delay should be below human perception; even though an objective and unique figure cannot be set, 100 ms is widely recognized as the desired one-way delay requirement for interaction. Since the global propagation delay can be about 100 ms, the actual end-to-end delay budget available to the system designer (excluding propagation delay) can be no more than 10 ms. We identify the components of the end-to-end delay in various configurations with the objective of understanding how it can be kept below the desired 10-ms bound. We analyze these components step-by-step through six system configurations obtained by combining three generic network architectures with two video encoding schemes. We study the transmission of raw video and variable bit rate (VBR) MPEG video encoding over (1) circuit switching; (2) synchronous packet switching; and (3) asynchronous packet switching. In addition, we show that constant bit rate (CBR) MPEG encoding delivers unacceptable delay-on the order of the group of pictures (GOP) time interval-when maximizing the quality for static scenes. This study aims at showing that having a global common time reference, together with time-driven priority (TDP) and VBR MPEG video encoding, provides adequate end-to-end delay, which is (1) below 10 ms; (2) independent of the network instant load; and (3) independent of the connection rate. The resulting end-to-end delay (excluding propagation delay) can be smaller than the video frame period, which is better than what can be obtained with circuit switching.
---
paper_title: Preserving temporal signature: a way to convey time constrained flows

paper_content:

Addresses the problem of a temporal signature conservation, arising in multimedia communications. Specifically, the authors are interested in the problem of the flow reconstitution and variable end-to-end delay compensation at the receiver end-system. Requirements and algorithm associated with this design are described. The proposed algorithm is quickly understood, it may exploit either the globally synchronized or locally available sender/receiver clocks. The authors also present the simple formal model of the algorithm. This formal model allows to make an elegant and short proof of correctness of the protocol. >
---
paper_title: A Synchronization Scheme for Stored Multimedia Streams

paper_content:

Multimedia streams such as audio and video impose tight temporal constraints due to their continuous nature. Often, different multimedia streams must be played out in a synchronized way. We present a scheme to ensure the continuous and synchronous playout of stored multimedia streams. We propose a protocol for the synchronized playback and we compute the buffer required to achieve both, the continuity within a single substream and the synchronization between related substreams. The scheme is very general because it only makes a single assumption, namely that the jitter is bounded.
---
paper_title: Surveyor: An Infrastructure For Internet Performance Measurements

paper_content:

A digital memory system employing a rectangular array of known MNOS variable threshold insulated gate field effect transistor memory cells is actuated by auxiliary circuits which provide a four-step operating sequence. The memory cells are arranged in word rows in which the gate electrodes of all memory cells in a given row are connected together and in bit columns having common source and common drain connections. The auxiliary circuits provide intermediate gate voltages to a selected row of memory cells in the first step of the operating cycle so as to read the information stored in the memory cells into a register. In the second step of the operating sequence, a large negative gate voltage is applied to the selected row to circumvent the cumulative effect that might arise from a succession of positive WRITE pulses. In the third operating step, the memory cells in the selected row are set to their least negative threshold value by an appropriate "clear" pulse, and in the fourth operating step information is written back into the selected memory cells from the register.
---
paper_title: The Concord algorithm for synchronization of networked multimedia streams

paper_content:

Synchronizing different data streams from multiple sources simultaneously at a receiver is one of the basic problems involved in multimedia distributed systems. This requirement stems from the nature of packet based networks which can introduce end-to-end delays that vary both within and across streams. We present a new algorithm called Concord, which provides an integrated solution for these single and multiple stream synchronization problems. It is notable because it defines a single framework to deal with both problems, and operates under the influence of parameters which can be supplied by the application involved. In particular these parameters are used to allow a trade-off between the packet loss rates, total end-to-end delay and skew for each of the streams. For applications like conferencing this is used to reduce delay by determining the minimum buffer delay/size required.
---
paper_title: Delay Reduction Techniques for Playout Buffering

paper_content:

Receiver synchronization of continuous media streams is required to deal with delay differences and variations resulting from delivery over packet networks such as the Internet. This function is commonly provided using per-stream playout buffers which introduce additional delay in order to produce a playout schedule which meets the synchronization requirements. Packets which arrive after their scheduled playout time are considered late and are discarded. In this paper, we present the Concord algorithm, which provides a delay-sensitive solution for playout buffering. It records historical information and uses it to make short-term predictions about network delay with the aim of not reacting too quickly to short-lived delay variations. This allows an application-controlled tradeoff of packet lateness against buffering delay, suitable for applications which demand low delay but can tolerate or conceal a small amount of late packets. We present a selection of results from an extensive evaluation of Concord using Internet traffic traces. We explore the use of aging techniques to improve the effectiveness of the historical information and hence, the delay predictions. The results show that Concord can produce significant reductions in buffering delay and delay variations at the expense of packet lateness values of less than 1%.
---
paper_title: End-to-end Internet packet dynamics

paper_content:

We discuss findings from a large-scale study of Internet packet dynamics conducted by tracing 20,000 TCP bulk transfers between 35 Internet sites. Because we traced each 100 Kbyte transfer at both the sender and the receiver, the measurements allow us to distinguish between the end-to-end behaviors due to the different directions of the Internet paths, which often exhibit asymmetries. We characterize the prevalence of unusual network events such as out-of-order delivery and packet corruption; discuss a robust receiver-based algorithm for estimating "bottleneck bandwidth" that addresses deficiencies discovered in techniques based on "packet pair"; investigate patterns of packet loss, finding that loss events are not well-modeled as independent and, furthermore, that the distribution of the duration of loss events exhibits infinite variance; and analyze variations in packet transit delays as indicators of congestion periods, finding that congestion periods also span a wide range of time scales.
---
paper_title: Internet time synchronization: The Network Time Protocol

paper_content:

The network time protocol (NTP), which is designed to distribute time information in a large, diverse system, is described. It uses a symmetric architecture in which a distributed subnet of time servers operating in a self-organizing, hierarchical configuration synchronizes local clocks within the subnet and to national time standards via wire, radio, or calibrated atomic clock. The servers can also redistribute time information within a network via local routing algorithms and time daemons. The NTP synchronization system, which has been in regular operation in the Internet for the last several years, is described, along with performance data which show that timekeeping accuracy throughout most portions of the Internet can be ordinarily maintained to within a few milliseconds, even in cases of failure or disruption of clocks, time servers, or networks. >
---
paper_title: End-to-end delay analysis of videoconferencing over packet-switched networks

paper_content:

Videoconferencing is an important global application-it enables people around the globe to interact when distance separates them. In order for the participants in a videoconference call to interact naturally, the end-to-end delay should be below human perception; even though an objective and unique figure cannot be set, 100 ms is widely recognized as the desired one-way delay requirement for interaction. Since the global propagation delay can be about 100 ms, the actual end-to-end delay budget available to the system designer (excluding propagation delay) can be no more than 10 ms. We identify the components of the end-to-end delay in various configurations with the objective of understanding how it can be kept below the desired 10-ms bound. We analyze these components step-by-step through six system configurations obtained by combining three generic network architectures with two video encoding schemes. We study the transmission of raw video and variable bit rate (VBR) MPEG video encoding over (1) circuit switching; (2) synchronous packet switching; and (3) asynchronous packet switching. In addition, we show that constant bit rate (CBR) MPEG encoding delivers unacceptable delay-on the order of the group of pictures (GOP) time interval-when maximizing the quality for static scenes. This study aims at showing that having a global common time reference, together with time-driven priority (TDP) and VBR MPEG video encoding, provides adequate end-to-end delay, which is (1) below 10 ms; (2) independent of the network instant load; and (3) independent of the connection rate. The resulting end-to-end delay (excluding propagation delay) can be smaller than the video frame period, which is better than what can be obtained with circuit switching.
---
paper_title: Synchronization of multimedia data for a multimedia news-on-demand application

paper_content:

We present a complete software control architecture for synchronizing multiple data streams generated from distributed media-storing database servers without the use of a global clock. Independent network connections are set up to remote workstations for multimedia presentations. Based on the document scenario and traffic predictions, stream delivery scheduling is performed in a centralized manner. Certain compensation mechanisms at the receiver are also necessary due to the presence of random network delays. A stream synchronization protocol (SSP) allows for synchronization recovery, ensuring a high quality multimedia display at the receiver. SSP uses synchronization quality of service parameters to guarantee the simultaneous delivery of the different types of data streams. In the proposed architecture, a priority-based synchronization control mechanism for MPEG-2 coded data streams is also provided. A performance modeling of the SSP is presented using the DSPN models. Relevant results such as the effect of the SSP, the number of synchronization errors, etc., are obtained.
---
paper_title: Packet audio playout delay adjustment: performance bounds and algorithms

paper_content:

In packet audio applications, packets are buffered at a receiving site and their playout delayed in order to com- pensate for variable network delays. In this paper, we con- sider the problem of adaptively adjusting the playout delay in order to keep this delay as small as possible, while at the same time avoiding excessive "loss" due to the arrival of packets at the receiver after their playout time has al- ready passed. The contributions of this paper are twofold. First, given a trace of packet audio receptions at a receiver, we present efficient algorithms for computing a bound on the achievable performance of any playout delay adjustment algorithm. More precisely, we compute upper and lower bounds (which are shown to be tight for the range of loss and delay values of interest) on the optimum (minimum) average playout delay for a given number of packet losses (due to late arrivals) at the receiver for that trace. Second, we present a new adaptive delay adjustment algorithm that tracks the network delay of recently received packets and efficiently maintains delay percentile information. This in- formation, together with a "delay spike" detection algorithm based on (but extending) our earlier work, is used to dy- namically adjust talkspurt playout delay. We show that this algorithm outperforms existing delay adjustment algorithms over a number of measured audio delay traces and performs close to the theoretical optimum over a range of parameter values of interest.
---
paper_title: Adaptive playout mechanisms for packetized audio applications in wide-area networks

paper_content:

Recent interest in supporting packet-audio applications over wide area networks has been fueled by the availability of low-cost, toll-quality workstation audio and the demonstration that limited amounts of interactive audio can be supported by today's Internet. In such applications, received audio packets are buffered, and their playout delayed at the destination host in order to compensate for the variable network delays. The authors investigate the performance of four different algorithms for adaptively adjusting the playout delay of audio packets in an interactive packet-audio terminal application, in the face of such varying network delays. They evaluate the playout algorithms using experimentally-obtained delay measurements of audio traffic between several different Internet sites. Their results indicate that an adaptive algorithm which explicitly adjusts to the sharp, spike-like increases in packet delay which were observed in the traces can achieve a lower rate of lost packets for both a given average playout delay and a given maximum buffer size. >
---
paper_title: End-to-end packet delay and loss behavior in the internet

paper_content:

We use the measured round trip delays of small UDP probe packets sent at regular time intervals to analyze the end-to-end packet delay and loss behavior in the Internet. By varying the interval between probe packets, it is possible to study the structure of the Internet load over different time scales. In this paper, the time scales of interest range from a few milliseconds to a few minutes. Our observations agree with results obtained by others using simulation and experimental approaches. For example, our estimates of Internet workload are consistent with the hypothesis of a mix of bulk traffic with larger packet size, and interactive traffic with smaller packet size. We observe compression (or clustering) of the probe packets, rapid fluctuations of queueing delays over small intervals, etc. Our results also show interesting and less expected behavior. For example, we find that the losses of probe packets are essentially random unless the probe traffic uses a large fraction of the available bandwidth. We discuss the implications of these results on the design of control mechanisms for the Internet.
---
paper_title: Design and Experimental Evaluation of an Adaptive Playout Delay Control Mechanism for Packetized Audio for Use over the Internet

paper_content:

We describe the design and the experimental evaluation of a playout delay control mechanism we have developed in order to support unicast, voice-based audio communications over the Internet. The proposed mechanism was designed to dynamically adjust the talkspurt playout delays to the traffic conditions of the underlying network without assuming either the existence of an external mechanism for maintaining an accurate clock synchronization between the sender and the receiver during the audio communication, or a specific distribution of the audio packet transmission delays. Performance figures derived from several experiments are reported that illustrate the adequacy of the proposed mechanism in dynamically adjusting the audio packet playout delay to the network traffic conditions while maintaining a small percentage of packet loss.
---
paper_title: The Concord algorithm for synchronization of networked multimedia streams

paper_content:

Synchronizing different data streams from multiple sources simultaneously at a receiver is one of the basic problems involved in multimedia distributed systems. This requirement stems from the nature of packet based networks which can introduce end-to-end delays that vary both within and across streams. We present a new algorithm called Concord, which provides an integrated solution for these single and multiple stream synchronization problems. It is notable because it defines a single framework to deal with both problems, and operates under the influence of parameters which can be supplied by the application involved. In particular these parameters are used to allow a trade-off between the packet loss rates, total end-to-end delay and skew for each of the streams. For applications like conferencing this is used to reduce delay by determining the minimum buffer delay/size required.
---
paper_title: Delay Reduction Techniques for Playout Buffering

paper_content:

Receiver synchronization of continuous media streams is required to deal with delay differences and variations resulting from delivery over packet networks such as the Internet. This function is commonly provided using per-stream playout buffers which introduce additional delay in order to produce a playout schedule which meets the synchronization requirements. Packets which arrive after their scheduled playout time are considered late and are discarded. In this paper, we present the Concord algorithm, which provides a delay-sensitive solution for playout buffering. It records historical information and uses it to make short-term predictions about network delay with the aim of not reacting too quickly to short-lived delay variations. This allows an application-controlled tradeoff of packet lateness against buffering delay, suitable for applications which demand low delay but can tolerate or conceal a small amount of late packets. We present a selection of results from an extensive evaluation of Concord using Internet traffic traces. We explore the use of aging techniques to improve the effectiveness of the historical information and hence, the delay predictions. The results show that Concord can produce significant reductions in buffering delay and delay variations at the expense of packet lateness values of less than 1%.
---
paper_title: Voice synchronization in packet switching networks

paper_content:

An algorithm for voice synchronization for packet switching networks is presented. The algorithm has been tested both in simulation and on a real network. The algorithm runs on the TRAME packet switching network for both the Vocoder and CELP DoD voice coding standards. Some results of these tests are presented. Some details of the algorithm development and implementation are given as well. >
---
paper_title: Packet audio playout delay adjustment: performance bounds and algorithms

paper_content:

In packet audio applications, packets are buffered at a receiving site and their playout delayed in order to com- pensate for variable network delays. In this paper, we con- sider the problem of adaptively adjusting the playout delay in order to keep this delay as small as possible, while at the same time avoiding excessive "loss" due to the arrival of packets at the receiver after their playout time has al- ready passed. The contributions of this paper are twofold. First, given a trace of packet audio receptions at a receiver, we present efficient algorithms for computing a bound on the achievable performance of any playout delay adjustment algorithm. More precisely, we compute upper and lower bounds (which are shown to be tight for the range of loss and delay values of interest) on the optimum (minimum) average playout delay for a given number of packet losses (due to late arrivals) at the receiver for that trace. Second, we present a new adaptive delay adjustment algorithm that tracks the network delay of recently received packets and efficiently maintains delay percentile information. This in- formation, together with a "delay spike" detection algorithm based on (but extending) our earlier work, is used to dy- namically adjust talkspurt playout delay. We show that this algorithm outperforms existing delay adjustment algorithms over a number of measured audio delay traces and performs close to the theoretical optimum over a range of parameter values of interest.
---
paper_title: Adaptive playout mechanisms for packetized audio applications in wide-area networks

paper_content:

Recent interest in supporting packet-audio applications over wide area networks has been fueled by the availability of low-cost, toll-quality workstation audio and the demonstration that limited amounts of interactive audio can be supported by today's Internet. In such applications, received audio packets are buffered, and their playout delayed at the destination host in order to compensate for the variable network delays. The authors investigate the performance of four different algorithms for adaptively adjusting the playout delay of audio packets in an interactive packet-audio terminal application, in the face of such varying network delays. They evaluate the playout algorithms using experimentally-obtained delay measurements of audio traffic between several different Internet sites. Their results indicate that an adaptive algorithm which explicitly adjusts to the sharp, spike-like increases in packet delay which were observed in the traces can achieve a lower rate of lost packets for both a given average playout delay and a given maximum buffer size. >
---
paper_title: Techniques for Packet Voice Synchronization

paper_content:

Packet switching has been proposed as an effective technology for integrating voice and data in a single network. An important aspect of packet-switched voice is the reconstruction of a continuous stream of speech from the set of packets that arrive at the destination terminal, each of which may encounter a different amount of buffering delay in the packet network. The magnitude of the variation in delay may range from a few milliseconds in a local area network to hundreds of milliseconds in a long-haul packet voice and data network. This paper discusses several aspects of the packet voice synchronization problem, and techniques that can be used to address it. These techniques estimate in some way the delay encountered by each packet and use the delay estimate to determine how speech is reconstructed. The delay estimates produced by these techniques can be used in managing the flow of information in the packet network to improve overall performance. Interactions of packet voice synchronization techniques with other network design issues are also discussed.
---
paper_title: Multipoint multimedia teleconference system with adaptive synchronization

paper_content:

This paper discusses major issues involved in the design and implementation of a multipoint multimedia conference system, such as system architecture, conference management, session control, and intramedium and intermedia synchronization. In particular, emphasis is given to conference management and adaptive synchronization algorithms. The management of multiparticipants is based upon a distributed architecture for greater flexibility. The proposed synchronization algorithm is adaptive to network changes, eliminates the need for a global clock, and is immune to the clock frequency drift, while its realization is very simple and the involved overhead is minimal. The essence of the algorithm is partitioning the vicinity of the arrival epochs of multimedia objects into three regions and counting arrivals at each region. The function of the synchronizer is to shift the playback clock (PBC) according to the individual counter contents. The ideas proposed are implemented within a teleconference system on the Ethernet/FDDI using the TCP/UDP. Experimental results show that the proposed synchronization algorithm performs well in our network testbed environment.
---
paper_title: Dynamic Video Playout Smoothing Method for Multimedia Applications

paper_content:

Multimedia applications including video data require the smoothing of video playout to prevent potential discontinuity. In this paper, we propose a dynamic video playout smoothing method, called the Video Smoother, which dynamically adopts various playout rates in an attempt to compensate for high delay variance of networks. Specifically, if the number of frames in the buffer exceeds a given threshold (TH), the Smoother employs a maximum playout rate. Otherwise, the Smoother uses proportionally reduced rates in an effort to eliminate playout pauses resulting from the emptiness of the playout buffer. To determine THs under various loads, we present an analytic model assuming the Interrupted Poisson Process (IPP) arrival. Based on the analytic results, we establish a paradigm of determining THs and playout rates for achieving different playout qualities under various loads of networks. Finally, to demonstrate the viability of the Video Smoother, we have implemented a prototyping system including a multimedia teleconferencing application and the Video Smoother performing as part of the transport layer. The prototyping results show that the Video Smoother achieves smooth playout incurring only unnoticeable delays.
---
paper_title: Delay and Synchronization Control Middleware to Support Real-Time Multimedia Services over Wireless PCS Networks

paper_content:

This paper presents a discussion of several middleware design issues related to the support of real-time multimedia communications over wireless personal communication services (PCS) networks. Specific interests are given to error recovery and synchronization mechanisms. A hybrid automatic repeat request (ARQ) scheme is employed for error control in the proposed system because it can efficiently adapt to nonstationary wireless channels and yield high throughput and reliability. In particular, delay and delay jitter control related to retransmissions in the error control module are addressed. An adaptive source rate control mechanism is used to handle the fluctuation of the effective channel data rate due to retransmissions. An adaptive synchronization scheme is developed to compensate for long-term delay variation caused by large-scale fading so that synchronization is preserved and end-to-end delay is kept low. Simulation results from the performance evaluation of the system are presented.
---
paper_title: A Synchronization Scheme for Stored Multimedia Streams

paper_content:

Multimedia streams such as audio and video impose tight temporal constraints due to their continuous nature. Often, different multimedia streams must be played out in a synchronized way. We present a scheme to ensure the continuous and synchronous playout of stored multimedia streams. We propose a protocol for the synchronized playback and we compute the buffer required to achieve both, the continuity within a single substream and the synchronization between related substreams. The scheme is very general because it only makes a single assumption, namely that the jitter is bounded.
---
paper_title: Adaptive playout strategies for packet video receivers with finite buffer capacity

paper_content:

Due to random delay variations in current best effort networks, packet video applications rely on end-system buffering and playout adaptation to reduce the effects of disruptions on the required smooth stream presentation. To study the effect of buffering and playout adaptation, we present an analytical model based on the M/G/1 queueing system with finite buffer capacity, and traffic intensity equal to or greater than unity. This model fits well a range of new applications that have limited buffer resources for the reception of incoming frames. We introduce the variance of distortion of playout (VDoP), a new metric that accounts for the overall presentation disruption caused by buffer underflows, intentionally introduced gaps during slowdown periods and data loss from overflows. VDoP is an elegant and fair metric for the estimation of playout quality and will hopefully assist the development of better adaptation algorithms. Furthermore, the effect of finite buffer capacity is examined in relation to stream continuity revealing a system behavior not previously accounted for. The sensitivity of the system to the variance of the arrival process is also examined by means of simulation. Finally, an online algorithm is presented for the exploitation of our study on implemented systems.
---
paper_title: Intelligent video smoother for multimedia communications

paper_content:

Multimedia communications often require intramedia synchronization for video data to prevent potential playout discontinuity resulting from network delay variation (jitter) while still achieving satisfactory playout throughput. In this paper, we propose a neural network (NN) based intravideo synchronization mechanism, called the intelligent video smoother (IVS), operating at the application layer of the receiving end system. The IVS is composed of an NN traffic predictor, an NN window determinator, and a window-based playout smoothing algorithm. The NN traffic predictor employs an on-line-trained back-propagation neural network (BPNN) to periodically predict the characteristics of traffic modeled by a generic interrupted Bernoulli process (IBP) over a future fixed time period. With the predicted traffic characteristics, the NN window determinator determines the corresponding optimal window by means of an off-line-trained BPNN in an effort to achieve a maximum of the playout quality (Q) value. The window-based playout smoothing algorithm then dynamically adopts various playout rates according to the window and the number of packets in the buffer. Finally, we show that via simulation results and live video scenes, compared to two other playout approaches, IVS achieves high-throughput and low-discontinuity playout under a mixture of IBP arrivals.
---
paper_title: Intra- and inter-stream synchronisation for stored multimedia streams

paper_content:

Multimedia streams such as audio and video impose tight temporal constraints due to their continuous nature. Often, different multimedia streams must be presented in a synchronized way. We introduce a scheme for the continuous and synchronous delivery of distributed stored multimedia streams across a communications network. We propose a protocol for the synchronized playback, compute the buffer requirement, and describe the experimental results of our implementation. The scheme is very general and does not require bounded jitter or synchronized clocks and is able to cope with clock drifts and server drop outs.
---
paper_title: An Adaptive Stream Synchronization Protocol

paper_content:

Protocols for synchronizing data streams should be highly adaptive with regard to both changing network conditions as well as to individual user needs. The Adaptive Synchronization Protocol we are going to describe in this paper supports any type of distribution of the stream group to be synchronized. It incorporates buffer level control mechanisms allowing an immediate reaction on overflow or underflow situations. Moreover, the proposed mechanism is flexible enough to support a variety of synchronization policies and allows to switch them dynamically during presentation. Since control messages are only exchanged when the network conditions actually change, the message overhead of the protocol is very low.
---
paper_title: Dynamic Video Playout Smoothing Method for Multimedia Applications

paper_content:

Multimedia applications including video data require the smoothing of video playout to prevent potential discontinuity. In this paper, we propose a dynamic video playout smoothing method, called the Video Smoother, which dynamically adopts various playout rates in an attempt to compensate for high delay variance of networks. Specifically, if the number of frames in the buffer exceeds a given threshold (TH), the Smoother employs a maximum playout rate. Otherwise, the Smoother uses proportionally reduced rates in an effort to eliminate playout pauses resulting from the emptiness of the playout buffer. To determine THs under various loads, we present an analytic model assuming the Interrupted Poisson Process (IPP) arrival. Based on the analytic results, we establish a paradigm of determining THs and playout rates for achieving different playout qualities under various loads of networks. Finally, to demonstrate the viability of the Video Smoother, we have implemented a prototyping system including a multimedia teleconferencing application and the Video Smoother performing as part of the transport layer. The prototyping results show that the Video Smoother achieves smooth playout incurring only unnoticeable delays.
---
paper_title: Adaptive playout strategies for packet video receivers with finite buffer capacity

paper_content:

Due to random delay variations in current best effort networks, packet video applications rely on end-system buffering and playout adaptation to reduce the effects of disruptions on the required smooth stream presentation. To study the effect of buffering and playout adaptation, we present an analytical model based on the M/G/1 queueing system with finite buffer capacity, and traffic intensity equal to or greater than unity. This model fits well a range of new applications that have limited buffer resources for the reception of incoming frames. We introduce the variance of distortion of playout (VDoP), a new metric that accounts for the overall presentation disruption caused by buffer underflows, intentionally introduced gaps during slowdown periods and data loss from overflows. VDoP is an elegant and fair metric for the estimation of playout quality and will hopefully assist the development of better adaptation algorithms. Furthermore, the effect of finite buffer capacity is examined in relation to stream continuity revealing a system behavior not previously accounted for. The sensitivity of the system to the variance of the arrival process is also examined by means of simulation. Finally, an online algorithm is presented for the exploitation of our study on implemented systems.
---
paper_title: Intelligent video smoother for multimedia communications

paper_content:

Multimedia communications often require intramedia synchronization for video data to prevent potential playout discontinuity resulting from network delay variation (jitter) while still achieving satisfactory playout throughput. In this paper, we propose a neural network (NN) based intravideo synchronization mechanism, called the intelligent video smoother (IVS), operating at the application layer of the receiving end system. The IVS is composed of an NN traffic predictor, an NN window determinator, and a window-based playout smoothing algorithm. The NN traffic predictor employs an on-line-trained back-propagation neural network (BPNN) to periodically predict the characteristics of traffic modeled by a generic interrupted Bernoulli process (IBP) over a future fixed time period. With the predicted traffic characteristics, the NN window determinator determines the corresponding optimal window by means of an off-line-trained BPNN in an effort to achieve a maximum of the playout quality (Q) value. The window-based playout smoothing algorithm then dynamically adopts various playout rates according to the window and the number of packets in the buffer. Finally, we show that via simulation results and live video scenes, compared to two other playout approaches, IVS achieves high-throughput and low-discontinuity playout under a mixture of IBP arrivals.
---
paper_title: Intra- and inter-stream synchronisation for stored multimedia streams

paper_content:

Multimedia streams such as audio and video impose tight temporal constraints due to their continuous nature. Often, different multimedia streams must be presented in a synchronized way. We introduce a scheme for the continuous and synchronous delivery of distributed stored multimedia streams across a communications network. We propose a protocol for the synchronized playback, compute the buffer requirement, and describe the experimental results of our implementation. The scheme is very general and does not require bounded jitter or synchronized clocks and is able to cope with clock drifts and server drop outs.
---
paper_title: Dynamic Video Playout Smoothing Method for Multimedia Applications

paper_content:

Multimedia applications including video data require the smoothing of video playout to prevent potential discontinuity. In this paper, we propose a dynamic video playout smoothing method, called the Video Smoother, which dynamically adopts various playout rates in an attempt to compensate for high delay variance of networks. Specifically, if the number of frames in the buffer exceeds a given threshold (TH), the Smoother employs a maximum playout rate. Otherwise, the Smoother uses proportionally reduced rates in an effort to eliminate playout pauses resulting from the emptiness of the playout buffer. To determine THs under various loads, we present an analytic model assuming the Interrupted Poisson Process (IPP) arrival. Based on the analytic results, we establish a paradigm of determining THs and playout rates for achieving different playout qualities under various loads of networks. Finally, to demonstrate the viability of the Video Smoother, we have implemented a prototyping system including a multimedia teleconferencing application and the Video Smoother performing as part of the transport layer. The prototyping results show that the Video Smoother achieves smooth playout incurring only unnoticeable delays.
---
paper_title: A Reliable, Adaptive Network Protocol for Video Transport

paper_content:

We present an adaptive network layer protocol for VBR video transport. It: (1) minimizes the buffer requirement in the network while guaranteeing that packets of VBR encoded video flows will not be lost, and (2) minimizes the end-to-end delay and jitter of frames. To achieve the former objective, we utilize a receiver-oriented adaptive credit-based flow control algorithm, and derive the necessary and sufficient number of buffers that should be reserved for ensuring its reliability. To minimize the end-to-end delay and jitter for VBR encoded video streams, we: (1) present bandwidth estimation techniques which exploit the structure of the video traffic, and (2) define a new fairness criteria for buffer allocation and then present a fair buffer/bandwidth allocation algorithm. We experimentally evaluate this protocol for a wide range of parameters and many network configurations, and demonstrate its adaptability. We also compare the performance of the protocol with numerous other schemes and demonstrate its suitability for video transport.
---
paper_title: Adaptive playout strategies for packet video receivers with finite buffer capacity

paper_content:

Due to random delay variations in current best effort networks, packet video applications rely on end-system buffering and playout adaptation to reduce the effects of disruptions on the required smooth stream presentation. To study the effect of buffering and playout adaptation, we present an analytical model based on the M/G/1 queueing system with finite buffer capacity, and traffic intensity equal to or greater than unity. This model fits well a range of new applications that have limited buffer resources for the reception of incoming frames. We introduce the variance of distortion of playout (VDoP), a new metric that accounts for the overall presentation disruption caused by buffer underflows, intentionally introduced gaps during slowdown periods and data loss from overflows. VDoP is an elegant and fair metric for the estimation of playout quality and will hopefully assist the development of better adaptation algorithms. Furthermore, the effect of finite buffer capacity is examined in relation to stream continuity revealing a system behavior not previously accounted for. The sensitivity of the system to the variance of the arrival process is also examined by means of simulation. Finally, an online algorithm is presented for the exploitation of our study on implemented systems.
---
paper_title: Intelligent video smoother for multimedia communications

paper_content:

Multimedia communications often require intramedia synchronization for video data to prevent potential playout discontinuity resulting from network delay variation (jitter) while still achieving satisfactory playout throughput. In this paper, we propose a neural network (NN) based intravideo synchronization mechanism, called the intelligent video smoother (IVS), operating at the application layer of the receiving end system. The IVS is composed of an NN traffic predictor, an NN window determinator, and a window-based playout smoothing algorithm. The NN traffic predictor employs an on-line-trained back-propagation neural network (BPNN) to periodically predict the characteristics of traffic modeled by a generic interrupted Bernoulli process (IBP) over a future fixed time period. With the predicted traffic characteristics, the NN window determinator determines the corresponding optimal window by means of an off-line-trained BPNN in an effort to achieve a maximum of the playout quality (Q) value. The window-based playout smoothing algorithm then dynamically adopts various playout rates according to the window and the number of packets in the buffer. Finally, we show that via simulation results and live video scenes, compared to two other playout approaches, IVS achieves high-throughput and low-discontinuity playout under a mixture of IBP arrivals.
---
paper_title: Intra- and inter-stream synchronisation for stored multimedia streams

paper_content:

Multimedia streams such as audio and video impose tight temporal constraints due to their continuous nature. Often, different multimedia streams must be presented in a synchronized way. We introduce a scheme for the continuous and synchronous delivery of distributed stored multimedia streams across a communications network. We propose a protocol for the synchronized playback, compute the buffer requirement, and describe the experimental results of our implementation. The scheme is very general and does not require bounded jitter or synchronized clocks and is able to cope with clock drifts and server drop outs.
---
paper_title: An Adaptive Stream Synchronization Protocol

paper_content:

Protocols for synchronizing data streams should be highly adaptive with regard to both changing network conditions as well as to individual user needs. The Adaptive Synchronization Protocol we are going to describe in this paper supports any type of distribution of the stream group to be synchronized. It incorporates buffer level control mechanisms allowing an immediate reaction on overflow or underflow situations. Moreover, the proposed mechanism is flexible enough to support a variety of synchronization policies and allows to switch them dynamically during presentation. Since control messages are only exchanged when the network conditions actually change, the message overhead of the protocol is very low.
---
paper_title: Dynamic Video Playout Smoothing Method for Multimedia Applications

paper_content:

Multimedia applications including video data require the smoothing of video playout to prevent potential discontinuity. In this paper, we propose a dynamic video playout smoothing method, called the Video Smoother, which dynamically adopts various playout rates in an attempt to compensate for high delay variance of networks. Specifically, if the number of frames in the buffer exceeds a given threshold (TH), the Smoother employs a maximum playout rate. Otherwise, the Smoother uses proportionally reduced rates in an effort to eliminate playout pauses resulting from the emptiness of the playout buffer. To determine THs under various loads, we present an analytic model assuming the Interrupted Poisson Process (IPP) arrival. Based on the analytic results, we establish a paradigm of determining THs and playout rates for achieving different playout qualities under various loads of networks. Finally, to demonstrate the viability of the Video Smoother, we have implemented a prototyping system including a multimedia teleconferencing application and the Video Smoother performing as part of the transport layer. The prototyping results show that the Video Smoother achieves smooth playout incurring only unnoticeable delays.
---
paper_title: Human perception of jitter and media synchronization

paper_content:

Multimedia synchronization comprises both the definition and the establishment of temporal relationships among media types. The presentation of 'in sync' data streams is essential to achieve a natural impression, data that is 'out of sync' is perceived as being somewhat artificial, strange, or even annoying. Therefore, the goal of any multimedia system is to enable an application to present data without no or little synchronization errors. The achievement of this goal requires a detailed knowledge of the synchronization requirements at the user interface. The paper presents the results of a series of experiments about human media perception that may be used as 'quality of service' guidelines. The results show that skews between related data streams may still give the effect that the data is 'in sync' and gives some constraints under which jitter may be tolerated. The author uses the findings to develop a scheme for the processing of nontrivial synchronization skew between more than two data streams.
---
paper_title: Adaptive playout strategies for packet video receivers with finite buffer capacity

paper_content:

Due to random delay variations in current best effort networks, packet video applications rely on end-system buffering and playout adaptation to reduce the effects of disruptions on the required smooth stream presentation. To study the effect of buffering and playout adaptation, we present an analytical model based on the M/G/1 queueing system with finite buffer capacity, and traffic intensity equal to or greater than unity. This model fits well a range of new applications that have limited buffer resources for the reception of incoming frames. We introduce the variance of distortion of playout (VDoP), a new metric that accounts for the overall presentation disruption caused by buffer underflows, intentionally introduced gaps during slowdown periods and data loss from overflows. VDoP is an elegant and fair metric for the estimation of playout quality and will hopefully assist the development of better adaptation algorithms. Furthermore, the effect of finite buffer capacity is examined in relation to stream continuity revealing a system behavior not previously accounted for. The sensitivity of the system to the variance of the arrival process is also examined by means of simulation. Finally, an online algorithm is presented for the exploitation of our study on implemented systems.
---
paper_title: vic: A Flexible Framework for Packet Video

paper_content:

The deployment of IP Multicast has fostered the development of a suite of applications, collectively known as the MBone tools, for real-time multimedia conferencing over the Internet. Two of these tools — nv from Xerox PARC and ivs from INRIA — provide video transmission using software-based codecs. We describe a new video tool, vic , that extends the groundbreaking work of nv and ivs with a more flexible system architecture. This flexibility is characterized by network layer independence, support for hardware-based codecs, a conference coordination model, an extensible user interface, and support for diverse compression algorithms. We also propose a novel compression scheme called “Intra-H.261”. Created as a hybrid of the nv and ivs codecs, Intra-H.261 provides a factor of 2–3 improvement in compression gain over the nv encoder (6 dB of PSNR) as well as a substantial improvement in run-time performance over the ivs H.261 coder.
---
paper_title: Intelligent video smoother for multimedia communications

paper_content:

Multimedia communications often require intramedia synchronization for video data to prevent potential playout discontinuity resulting from network delay variation (jitter) while still achieving satisfactory playout throughput. In this paper, we propose a neural network (NN) based intravideo synchronization mechanism, called the intelligent video smoother (IVS), operating at the application layer of the receiving end system. The IVS is composed of an NN traffic predictor, an NN window determinator, and a window-based playout smoothing algorithm. The NN traffic predictor employs an on-line-trained back-propagation neural network (BPNN) to periodically predict the characteristics of traffic modeled by a generic interrupted Bernoulli process (IBP) over a future fixed time period. With the predicted traffic characteristics, the NN window determinator determines the corresponding optimal window by means of an off-line-trained BPNN in an effort to achieve a maximum of the playout quality (Q) value. The window-based playout smoothing algorithm then dynamically adopts various playout rates according to the window and the number of packets in the buffer. Finally, we show that via simulation results and live video scenes, compared to two other playout approaches, IVS achieves high-throughput and low-discontinuity playout under a mixture of IBP arrivals.
---
paper_title: Overcoming Workstation Scheduling Problems in a Real-Time Audio Tool

paper_content:

The recent interest in multimedia conferencing is a result of the incorporation of cheap audio and video hard-ware in today's workstations, and also as a result of the development of a global infrastructure capable of supporting multimedia traffic - the Mbone. Audio quality is impaired by packet loss and variable delay in the network, and by lack of support for real-time applications in today's general purpose workstations. A considerable amount of research effort has focused on solving the network side of the problem by providing packet loss robustness techniques, and network conscious adaptive applications. Effort to solve the operating system induced problems has concentrated on kernel modifications. This paper presents an architecture for a real-time audio media agent that copes with the problems presented by the UNIX operating system at the application level. The mechanism produces a continuous audio signal, despite the variable allocation of processing time a real-time application is given under UNIX. Continuity of audio is ensured during scheduling hiccups by using the buffering capabilities of workstation audio devices drivers. Our solution also tries to restrict the amount of audio stored in the device buffers to a minimum, to reduce the perceived end-to-end delay of the audio signal. A comparison between the method presented here (adaptive cushion algorithm), and that used by all other audio tools shows substantial reductions in both the average end-to-end delay, and the audio sample loss caused by the operating system.
---
paper_title: Packet audio playout delay adjustment: performance bounds and algorithms

paper_content:

In packet audio applications, packets are buffered at a receiving site and their playout delayed in order to com- pensate for variable network delays. In this paper, we con- sider the problem of adaptively adjusting the playout delay in order to keep this delay as small as possible, while at the same time avoiding excessive "loss" due to the arrival of packets at the receiver after their playout time has al- ready passed. The contributions of this paper are twofold. First, given a trace of packet audio receptions at a receiver, we present efficient algorithms for computing a bound on the achievable performance of any playout delay adjustment algorithm. More precisely, we compute upper and lower bounds (which are shown to be tight for the range of loss and delay values of interest) on the optimum (minimum) average playout delay for a given number of packet losses (due to late arrivals) at the receiver for that trace. Second, we present a new adaptive delay adjustment algorithm that tracks the network delay of recently received packets and efficiently maintains delay percentile information. This in- formation, together with a "delay spike" detection algorithm based on (but extending) our earlier work, is used to dy- namically adjust talkspurt playout delay. We show that this algorithm outperforms existing delay adjustment algorithms over a number of measured audio delay traces and performs close to the theoretical optimum over a range of parameter values of interest.
---
paper_title: Adaptive playout mechanisms for packetized audio applications in wide-area networks

paper_content:

Recent interest in supporting packet-audio applications over wide area networks has been fueled by the availability of low-cost, toll-quality workstation audio and the demonstration that limited amounts of interactive audio can be supported by today's Internet. In such applications, received audio packets are buffered, and their playout delayed at the destination host in order to compensate for the variable network delays. The authors investigate the performance of four different algorithms for adaptively adjusting the playout delay of audio packets in an interactive packet-audio terminal application, in the face of such varying network delays. They evaluate the playout algorithms using experimentally-obtained delay measurements of audio traffic between several different Internet sites. Their results indicate that an adaptive algorithm which explicitly adjusts to the sharp, spike-like increases in packet delay which were observed in the traces can achieve a lower rate of lost packets for both a given average playout delay and a given maximum buffer size. >
---
paper_title: Dynamic Video Playout Smoothing Method for Multimedia Applications

paper_content:

Multimedia applications including video data require the smoothing of video playout to prevent potential discontinuity. In this paper, we propose a dynamic video playout smoothing method, called the Video Smoother, which dynamically adopts various playout rates in an attempt to compensate for high delay variance of networks. Specifically, if the number of frames in the buffer exceeds a given threshold (TH), the Smoother employs a maximum playout rate. Otherwise, the Smoother uses proportionally reduced rates in an effort to eliminate playout pauses resulting from the emptiness of the playout buffer. To determine THs under various loads, we present an analytic model assuming the Interrupted Poisson Process (IPP) arrival. Based on the analytic results, we establish a paradigm of determining THs and playout rates for achieving different playout qualities under various loads of networks. Finally, to demonstrate the viability of the Video Smoother, we have implemented a prototyping system including a multimedia teleconferencing application and the Video Smoother performing as part of the transport layer. The prototyping results show that the Video Smoother achieves smooth playout incurring only unnoticeable delays.
---
paper_title: A survey of packet loss recovery techniques for streaming audio

paper_content:

We survey a number of packet loss recovery techniques for streaming audio applications operating using IP multicast. We begin with a discussion of the loss and delay characteristics of an IP multicast channel, and from this show the need for packet loss recovery. Recovery techniques may be divided into two classes: sender- and receiver-based. We compare and contrast several sender-based recovery schemes: forward error correction (both media-specific and media-independent), interleaving, and retransmission. In addition, a number of error concealment schemes are discussed. We conclude with a series of recommendations for repair schemes to be used based on application requirements and network conditions.
---
paper_title: Survey of error recovery techniques for IP-based audio-visual multicast applications

paper_content:

IP-based audio-visual multicast applications are gaining increasing interest since they can be realized using inexpensive network services that offer no guarantees for loss or delay. When using network services that do not guarantee the quality of service (QoS) required by audio-visual applications, recovery from losses due to congestion in the network is a key problem that must be solved. This survey gives an overview of existing transport-layer error control mechanisms and discusses their suitability for use in IP-based networks. Additionally, the impact of IP over ATM on the requirements of error control mechanisms is discussed. Different network scenarios are used to assess the performance of retransmission-based error correction and forward error correction.
---
paper_title: Integrating packet FEC into adaptive voice playout buffer algorithms on the Internet

paper_content:

Transport of real-time voice traffic on the Internet is difficult due to packet loss and jitter. Packet loss is handled primarily through a variety of different forward error correction (FEC) algorithms and local repair at the receiver. Jitter is compensated for by means of adaptive playout buffer algorithms at the receiver. Traditionally, these two mechanisms have been investigated in isolation. In this paper, we show the interactions between adaptive playout buffer algorithms and FEC, and demonstrate the need for coupling. We propose a number of novel playout buffer algorithms which provide this coupling, and demonstrate their effectiveness through simulations based on both network models and real network traces.
---
paper_title: Effects of interaction between error control and media synchronization on application-level performances

paper_content:

We study the interaction between error control and media synchronization. We approach the problem by considering a virtual application data unit (ADU) network with error control within the network boundary and media synchronization outside the network boundary. We introduce the concept of cumulative inter-ADU jitter, which is the running total of the differences between inter-ADU spacings at the receiver and at the sender, to model the ADU departure process from the network observed at the entry of the playback buffer. We investigate how different error control schemes, network and source characteristics influence the process on one hand and how the process affects the choice of playback delay in order to minimize ADU losses on the other hand. Our numerical results, obtained through simulation, verify the analysis and reveals that jitter control within the network is not as important as controlling the bandwidth and the delay, especially in the presence of error control and variable bit rate source.
---
paper_title: Error Control Schemes for Networks: An Overview

paper_content:

In this paper, we investigate the issue of error control in wireless communication networks. We review the alternative error control schemes available for providing reliable end-to-end communication in wireless environments. Through case studies, the performance and tradeoffs of these schemes are shown. Based on the application environments and QoS requirements, the design issues of error control are discussed to achieve the best solution.
---
paper_title: Packet audio playout delay adjustment: performance bounds and algorithms

paper_content:

In packet audio applications, packets are buffered at a receiving site and their playout delayed in order to com- pensate for variable network delays. In this paper, we con- sider the problem of adaptively adjusting the playout delay in order to keep this delay as small as possible, while at the same time avoiding excessive "loss" due to the arrival of packets at the receiver after their playout time has al- ready passed. The contributions of this paper are twofold. First, given a trace of packet audio receptions at a receiver, we present efficient algorithms for computing a bound on the achievable performance of any playout delay adjustment algorithm. More precisely, we compute upper and lower bounds (which are shown to be tight for the range of loss and delay values of interest) on the optimum (minimum) average playout delay for a given number of packet losses (due to late arrivals) at the receiver for that trace. Second, we present a new adaptive delay adjustment algorithm that tracks the network delay of recently received packets and efficiently maintains delay percentile information. This in- formation, together with a "delay spike" detection algorithm based on (but extending) our earlier work, is used to dy- namically adjust talkspurt playout delay. We show that this algorithm outperforms existing delay adjustment algorithms over a number of measured audio delay traces and performs close to the theoretical optimum over a range of parameter values of interest.
---
paper_title: Adaptive playout mechanisms for packetized audio applications in wide-area networks

paper_content:

Recent interest in supporting packet-audio applications over wide area networks has been fueled by the availability of low-cost, toll-quality workstation audio and the demonstration that limited amounts of interactive audio can be supported by today's Internet. In such applications, received audio packets are buffered, and their playout delayed at the destination host in order to compensate for the variable network delays. The authors investigate the performance of four different algorithms for adaptively adjusting the playout delay of audio packets in an interactive packet-audio terminal application, in the face of such varying network delays. They evaluate the playout algorithms using experimentally-obtained delay measurements of audio traffic between several different Internet sites. Their results indicate that an adaptive algorithm which explicitly adjusts to the sharp, spike-like increases in packet delay which were observed in the traces can achieve a lower rate of lost packets for both a given average playout delay and a given maximum buffer size. >
---
paper_title: QoS impact on user perception and understanding of multimedia video clips

paper_content:

L ABSTIL4CT The widespread and increasing advent of multimedia technologies means that there must be a departure from the viewpoint that users expect a Quality of Service (QoS) which will only satisfy them perceptually. What should be expected of multimedia clips is that the QoS with which they are shown is such that it will enable the users to assimilate and understand the informational content of such clips. In this paper we present experimental resuks Iiiking users’ understanding and perception of multimedia ctips with the presentation QoS. Results show that the quality of video clips can be severely degraded without the user having to perceive any significant 10ss of informational content 1.1
---
paper_title: Adaptive FEC-Based Error Control for Interactive Audio in the Internet

paper_content:

Excessive packet loss rates can dramatically decrease the audio quality perceived by users of Internet telephony applications. Recent results suggest that error control schemes using forward error correction (FEC) are good candidates for decreasing the impact of packet loss on audio quality. With FEC schemes, redundant information is transmitted along with the original information so that the lost original data can be recovered at least in part from the redundant information. Clearly, sending additional redundancy increases the probability of recovering lost packets, but it also increases the bandwidth requirements and thus the loss rate of the audio stream. This means that the FEC scheme must be coupled to a rate control scheme. Furthermore, the amount of redundant information used at any given point in time should also depend on the characteristics of the loss process at that time (it would make no sense to send much redundant information when the channel is loss free), on the end to end delay constraints (destination typically have to wait longer to decode the FEC as more FEC information is used), on the quality of the redundant information, etc. However, it is not clear how to choose the ”best” possible redundant information given all the constraints mentioned above. We address this issue in the paper, and illustrate our approach using a FEC scheme for packet audio recently standardized in the IETF. We find that the problem best redundant information can be expressed mathematically as a constrained optimization problem for which we give explicit solutions. We obtain from these solutions a simple algorithm with very interesting features: i) it optimizes a subjective measure of quality (such as the perceived audio quality at a destination) as opposed to a non subjective measure (such as the packet loss rate at a destination), ii) it incorporates the constraints of rate control and playout delay adjustment schemes, and iii) it adapts to varying (and estimated on line with RTCP) loss conditions in the network. We have been using the algorithm, together with a TCP-friendly rate control scheme, for a few months now and we have found it to provide very good audio quality even with high and varying loss rates. We present simulation and experimental results to illustrate its performance. Submitted to IEEE Infocom ’99.
---
paper_title: Effects of interaction between error control and media synchronization on application-level performances

paper_content:

We study the interaction between error control and media synchronization. We approach the problem by considering a virtual application data unit (ADU) network with error control within the network boundary and media synchronization outside the network boundary. We introduce the concept of cumulative inter-ADU jitter, which is the running total of the differences between inter-ADU spacings at the receiver and at the sender, to model the ADU departure process from the network observed at the entry of the playback buffer. We investigate how different error control schemes, network and source characteristics influence the process on one hand and how the process affects the choice of playback delay in order to minimize ADU losses on the other hand. Our numerical results, obtained through simulation, verify the analysis and reveals that jitter control within the network is not as important as controlling the bandwidth and the delay, especially in the presence of error control and variable bit rate source.
---
paper_title: Adaptive playout mechanisms for packetized audio applications in wide-area networks

paper_content:

Recent interest in supporting packet-audio applications over wide area networks has been fueled by the availability of low-cost, toll-quality workstation audio and the demonstration that limited amounts of interactive audio can be supported by today's Internet. In such applications, received audio packets are buffered, and their playout delayed at the destination host in order to compensate for the variable network delays. The authors investigate the performance of four different algorithms for adaptively adjusting the playout delay of audio packets in an interactive packet-audio terminal application, in the face of such varying network delays. They evaluate the playout algorithms using experimentally-obtained delay measurements of audio traffic between several different Internet sites. Their results indicate that an adaptive algorithm which explicitly adjusts to the sharp, spike-like increases in packet delay which were observed in the traces can achieve a lower rate of lost packets for both a given average playout delay and a given maximum buffer size. >
---
paper_title: Delay and Synchronization Control Middleware to Support Real-Time Multimedia Services over Wireless PCS Networks

paper_content:

This paper presents a discussion of several middleware design issues related to the support of real-time multimedia communications over wireless personal communication services (PCS) networks. Specific interests are given to error recovery and synchronization mechanisms. A hybrid automatic repeat request (ARQ) scheme is employed for error control in the proposed system because it can efficiently adapt to nonstationary wireless channels and yield high throughput and reliability. In particular, delay and delay jitter control related to retransmissions in the error control module are addressed. An adaptive source rate control mechanism is used to handle the fluctuation of the effective channel data rate due to retransmissions. An adaptive synchronization scheme is developed to compensate for long-term delay variation caused by large-scale fading so that synchronization is preserved and end-to-end delay is kept low. Simulation results from the performance evaluation of the system are presented.
---
paper_title: Distributed Network Storage Service with Quality-of-Service Guarantees

paper_content:

This paper envisions a distributed network storage service with Quality-of-Service (QoS) guarantees and describes its architecture and key mechanisms. When fully realized, this service architecture would be able to support, in one integrated framework, network storage services ranging from best-effort caching to replication with performance guarantees. Content owners could, through the use of standardized protocols, reserve network storage resources to satisfy their application-specific performance requirements. They would be able to specify either the number and/or placement of the replicas, or higher-level performance goals based on access latency, bandwidth usage or data availability. The network storage provider would then optimally allocate storage resources to meet the service commitments, using leftover capacity for best-effort caching. Content consumers would then retrieve the nearest copy of the data object, be it from a replica, cache, or the original source, in a completely transparent manner. Furthermore, a distributed network storage infrastructure should be integrated with the existing transmission-based QoS framework so that applications can select the optimal combination of storage and transmission resources to satisfy their performance requirements.This work identifies and discusses key research areas and problems that need to be tackled, including those in service specification, resource mapping, admission control, resource reservation and real-time storage management. The paper establishes a QoS framework upon which community discussion on this vision can proceed.
---
paper_title: Proxy-based distribution of streaming video over unicast/multicast connections

paper_content:

Multimedia streaming applications consume a significant amount of server and network bandwidth. In this paper we develop transmission schemes (SBatch, UPatch and MPatch) that exploit proxy-based prefix caching to reduce the transmission cost when part or all of the end-end path from the server to the clients is only unicast capable. We consider the problem of allocating a limited proxy buffer to a set of videos with different sizes and popularities. For a fractional caching policy which allows prefix caching, we develop a technique to analytically determine, for a given transmission scheme, the optimal proxy buffer allocation to each video that minimizes the bandwidth cost for the set. Our evaluations show that even a relatively small prefix cache can result in dramatic cost reductions, and that prefix caching significantly outperform policies that cache only entire objects at the proxy. Also proxy cache allocations that are tuned to the cost characteristics of a particular transmission scheme can significantly outperform transmission-scheme agnostic allocations. We find that carefully designed transmission schemes in conjunction with an optimal prefix caching scheme are the key to significant cost reductions over the case of transmitting a video separately to every client, when the end-end delivery path is either entirely unicast capable, or offers multicast only on the proxy-client path.
---
paper_title: Video staging: a proxy-server-based approach to end-to-end video delivery over wide-area networks

paper_content:

Real-time distribution of stored video over wide-area networks (WANs) is a crucial component of many emerging distributed multimedia applications. The heterogeneity in the underlying network environments is an important factor that must be taken into consideration when designing an end-to-end video delivery system. We present a novel approach to the problem of end-to-end video delivery over WANs using proxy servers situated between local-area networks (LANs) and a backbone WAN. A major objective of our approach is to reduce the backbone WAN bandwidth requirement. Toward this end, we develop an effective video delivery technique called video staging via intelligent utilization of the disk bandwidth and storage space available at proxy servers. Using this video staging technique, only part of a video stream is retrieved directly from the central video server across the backbone WAN whereas the rest of the video stream is delivered to users locally from proxy servers attached to the LANs. In this manner, the WAN bandwidth requirement can be significantly reduced, particularly when a large number of users from the same LAN access the video data. We design several video staging methods and evaluate their effectiveness in trading the disk bandwidth of a proxy server for the backbone WAN bandwidth. We also develop two heuristic algorithms to solve the problem of designing a multiple video staging scheme for a proxy server with a given video access profile of a LAN. Our results demonstrate that the proposed proxy-server-based approach provides an effective and scalable solution to the problem of the end-to-end video delivery over WANs.
---
paper_title: Distributing layered encoded video through caches

paper_content:

The efficient distribution of stored information has become a major concern in the Internet which has increasingly become a vehicle for the transport of stored video. Because of the highly heterogeneous access to the Internet, researchers and engineers have argued for layered encoded video. We investigate delivering layered encoded video using caches. Based on a stochastic knapsack model we develop a model for the layered video caching problem. We propose heuristics to determine which videos and which layers in the videos should be cached. We evaluate the performance of our heuristics through extensive numerical experiments. We also consider two intuitive extensions to the initial problem.
---
paper_title: Proxy prefix caching for multimedia streams

paper_content:

High latency and loss rates in the Internet make it difficult to stream audio and video without introducing a large playback delay. To address these problems, we propose a prefix caching technique whereby a proxy stores the initial frames of popular clips. Upon receiving a request for the stream, the proxy initiates transmission to the client and simultaneously requests the remaining frames from the server. In addition to hiding the delay, throughput, and loss effects of a weaker service model between the server and the proxy, this novel yet simple prefix caching technique aids the proxy in performing workahead smoothing into the client playback buffer. By transmitting large frames in advance of each burst, workahead smoothing substantially reduces the peak and variability of the network resource requirements along the path from the proxy to the client. We describe how to construct a smooth transmission schedule, based on the size of the prefix, smoothing, and playback buffers, without increasing client playback delay. Experiments with MPEG traces show how a few megabytes of buffer space at the proxy can substantially reduce the bandwidth requirements of variable-bit-rate video. Drawing on these results, we present guidelines for allocating buffer space for each stream, and how to effectively share buffer and bandwidth resources among multiple clients and streams.
---

Title: Intrastream Synchronization for Continuous Media Streams: A Survey of Playout Schedulers

Section 1: INTRODUCTION
Description 1: This section introduces the concept of intrastream synchronization, the significance of preserving temporal relationships in continuous media streams, and outlines the key focus areas and structure of the paper.

Section 2: ISSUES OF INTEREST AND OUTLINE
Description 2: This section discusses various metrics used for assessing intrastream synchronization quality, categorizes existing schedulers, and provides an outline of the topics covered in the remaining sections of the paper.

Section 3: MEDIA TYPE AND PLAYOUT ADAPTATION
Description 3: This section elaborates on the applicability of different playout schemes for various media types, distinguishing between continuous and semi-continuous media, and discusses how playout schedulers address the unique characteristics of each type.

Section 4: TIME-ORIENTED PLAYOUT SCHEDULERS
Description 4: This section presents a detailed overview of time-oriented playout schedulers, including methods that use timestamps and clocks to manage media unit presentation times.

Section 5: BUFFER-ORIENTED PLAYOUT SCHEDULERS
Description 5: This section explores buffer-oriented playout schedulers that manage delay jitter by observing buffer occupancy and adjusting the playout point accordingly.

Section 6: DISCUSSION AND IMPLEMENTED SYSTEMS
Description 6: This section summarizes key concepts discussed, addresses the appropriateness of various playout schedulers for different applications, and cites examples of implemented systems.

Section 7: RELATED RESEARCH ISSUES
Description 7: This section briefly covers additional research areas relevant to playout adaptation, such as forward error correction and video caching.

Section 8: CONCLUSIONS
Description 8: This section provides a summary of the survey, highlighting the importance of playout schedulers in maintaining synchronization quality in stream communications and the major contributions of the paper.