Algorithm/Architecture Co-Exploration of Visual Computing on Emergent Platforms: Overview and Future Prospects
---
paper_title: A high-quality spatial-temporal content-adaptive deinterlacing algorithm

paper_content:

This paper introduced a spatial-temporal content- adaptive algorithm, which can precisely select an appropriate interpolation technique for high-quality deinterlacing according to the spectral, edge-oriented and statistical features of local video content. Our algorithm employs a linear-phase statistical- adaptive vertical-temporal filter to deal with generic video scenes and adopts a modified edge-based line-averaging interpolation to efficiently recover moving edges. In addition, annoying flickering artifacts are efficiently suppressed by a flickering detection and a field-averaging filter. As a result, our algorithm outperforms other non-motion compensated methods in terms of objective PSNR and reveals more impressive subjective visual quality.
---
paper_title: A motion-adaptive deinterlacer via hybrid motion detection and edge-pattern recognition

paper_content:

A novel motion-adaptive deinterlacing algorithm with edge-pattern recognition and hybrid motion detection is introduced. The great variety of video contents makes the processing of assorted motion, edges, textures, and the combination of them very difficult with a single algorithm. The edge-pattern recognition algorithm introduced in this paper exhibits the flexibility in processing both textures and edges which need to be separately accomplished by line average and edge-based line average before. Moreover, predicting the neighboring pixels for pattern analysis and interpolation further enhances the adaptability of the edge-pattern recognition unit when motion detection is incorporated. Our hybrid motion detection features accurate detection of fast and slow motion in interlaced video and also the motion with edges. Using only three fields for detection also renders higher temporal correlation for interpolation. The better performance of our deinterlacing algorithm with higher content-adaptability and less memory cost than the state-of-the-art 4-field motion detection algorithms can be seen from the subjective and objective experimental results of the CIF and PAL video sequences.
---
paper_title: Motion Adaptive Deinterlacing via Edge Pattern Recognition

paper_content:

In this paper, a novel edge pattern recognition (EPR) deinterlacing algorithm with successive 4-field enhanced motion detection is introduced. The EPR algorithm surpasses the performance of ELA-based and other conventional methods especially at textural scenes. In addition, the current 4-field enhanced motion detection scheme overcomes conventional motion missing artifacts by gaining good motion detection accuracies and suppression of "motion missing" detection errors efficiently. Furthermore, with the incorporation of our new successive 4-field enhanced motion detection, the interpolation technique of EPR algorithm is capable of flexible adaptation in achieving better performance on textural scenes in generic video sequences.
---
paper_title: Spatial-temporal content-adaptive deinterlacing algorithm

paper_content:

An algorithm, which adapts to an appropriate interpolation technique for high-quality deinterlacing based on the spatial-temporal spectral and edge-oriented features of local video contents, is introduced. This algorithm employs a spectrum-adaptive vertical-temporal filter to deal with generic video scenes, and adopts a texture-adaptive edge-based line-averaging interpolation with an edge-consistency check to interpolate moving edges with high efficiency. This deinterlacing algorithm likewise incorporates an economical recursive stationary-pixel detection scheme with a field-averaging filter to effectively interpolate stationary video scenes. Because of the precise selection of the interpolation technique for versatile video contents, the proposed algorithm provides not only better objective peak signal-to-noise ratio, but also results in more impressive subjective visual quality when compared with other non-motion compensated deinterlacing approaches.
---
paper_title: Rate control algorithm based on intra-picture complexity for H.264/AVC

paper_content:

An efficient rate control algorithm based on the content-adaptive initial quantisation parameter (QP) setting scheme and the peak signal-to-noise ratio (PSNR) variation-limited bit-allocation strategy for low-complexity mobile applications is presented. This algorithm can efficiently measure the residual complexity of intra-pictures without performing the computation-intensive intra-prediction and mode decision in H.264/AVC, based on the structural and statistical features of local textures. This can adaptively set proper initial QP values for versatile video contents. In addition, this bit-allocation strategy can effectively distribute bit-rate budgets based on the monotonic property to enhance overall coding efficiency while maintaining the consistency of visual quality by limiting the variation of quantisation distortion. The experimental results reveal that the proposed algorithm surpasses the conventional rate control approaches in terms of the average PSNR from 0.34 to 0.95 dB. Moreover, this algorithm provides more impressive visual quality and more robust buffer controllability when compared with other algorithms.
---
paper_title: Macroblock-level decoding and deblocking method and its pipeline implementation in H.264 decoder SOC design

paper_content:

This paper presents a macroblock-level (MB-level) decoding and deblocking method for supporting the flexible macroblock ordering (FMO) and arbitrary slice ordering (ASO) bit streams in H.264 decoder and its SOC/ASIC implementation. By searching the slice containing the current macroblock in the bit stream and switching slices correctly, MBs can be decoded in the raster scan order, while the decoding process can immediately begin as long as the slice containing the current MB is available. This architectural modification enables the MB-level decoding and deblocking 3-stage pipeline, and saves about 20% of SDRAM bandwidth. Implementation results showed that the design achieves real-time decoding of 1080HD (1920×1088@30 fps) at a system clock of 166 MHz.
---
paper_title: H.264/AVC baseline profile decoder complexity analysis

paper_content:

We study and analyze the computational complexity of a software-based H.264/AVC (advanced video codec) baseline profile decoder. Our analysis is based on determining the number of basic computational operations required by a decoder to perform the key decoding subfunctions. The frequency of use of each of the required decoding subfunctions is empirically derived using bitstreams generated from two different encoders for a variety of content, resolutions and bit rates. Using the measured frequencies, estimates of the decoder time complexity for various hardware platforms can be determined. A detailed example is provided to assist in deriving time complexity estimates. We compare the resulting estimates to numbers measured for an optimized decoder on the Pentium 3 hardware platform. We then use those numbers to evaluate the dependence of the time complexity of each of the major decoder subfunctions on encoder characteristics, content, resolution and bit rate. Finally, we compare an H.264/AVC-compliant baseline decoder to a decoder that is compliant with the H.263 standard, which is currently dominant in interactive video applications. Both "C" only decoder implementations were compared on a Pentium 3 hardware platform. Our results indicate that an H.264/AVC baseline decoder is approximately 2.5 times more time complex than an H.263 baseline decoder.
---
paper_title: Spatiotemporal video segmentation based on graphical models

paper_content:

This paper proposes a probabilistic framework for spatiotemporal segmentation of video sequences. Motion information, boundary information from intensity segmentation, and spatial connectivity of segmentation are unified in the video segmentation process by means of graphical models. A Bayesian network is presented to model interactions among the motion vector field, the intensity segmentation field, and the video segmentation field. The notion of the Markov random field is used to encourage the formation of continuous regions. Given consecutive frames, the conditional joint probability density of the three fields is maximized in an iterative way. To effectively utilize boundary information from the intensity segmentation, distance transformation is employed in local objective functions. Experimental results show that the method is robust and generates spatiotemporally coherent segmentation results. Moreover, the proposed video segmentation approach can be viewed as the compromise of previous motion based approaches and region merging approaches.
---
paper_title: Video Analysis and Compression on the STI Cell Broadband Engine Processor

paper_content:

With increased concern for physical security, video surveillance is becoming an important business area. Similar camera-based system can also be used in such diverse applications as retail-store shopper motion analysis and casino behavioral policy monitoring. There are two aspects of video surveillance that require significant computing power: image analysis for detecting objects, and video compression for digital storage. The new STI CELL Broadband Engine (CBE) processor is an appealing platform for such applications because it incorporates 8 separate high-speed processing cores with an aggregate performance of 256Gflops. Moreover, this chip is the heart of the new Sony Playstation 3 and can be expected to be relatively inexpensive due to the high volume of production. In this paper we show how object detection and compression can be implemented on the CBE, discuss the difficulties encountered in porting the code, and provide performance results demonstrating significant speed-up.
---
paper_title: Visual pattern matching in motion estimation for object-based very low bit-rate coding using moment-preserving edge detection

paper_content:

This paper proposes an object-based coding method for very low bit-rate channels, using a method based on motion estimation with a block-based moment-preserving edge detector. In most existing object-based coding methods, only the global motion components are transmitted. However, the global motion prediction error is large, even after motion compensation using the discrete cosine transform (DCT), when images contain rapid moving objects and noise. Furthermore, the global motion-compensating method cannot result in small prediction error if the segmented objects consist of subobjects that move along different directions. The technique proposed in this paper involves segmenting moving objects from video sequences and representing objects compactly by visual-pattern approximations of the boundary. A visual pattern is obtained by detecting the line edge from a square block using the moment-preserving edge detector. Since high computational complexity is required for motion estimation using block matching, a fast block-matching method based on the visual patterns is proposed to reduce the burden of the overall coder complexity. Computer simulation results show that the proposed method gives good performance in terms of the subjective quality, the peak signal-to-noise ratio, and the compression ratio.
---
paper_title: Image classification for content-based indexing

paper_content:

Grouping images into (semantically) meaningful categories using low-level visual features is a challenging and important problem in content-based image retrieval. Using binary Bayesian classifiers, we attempt to capture high-level concepts from low-level image features under the constraint that the test image does belong to one of the classes. Specifically, we consider the hierarchical classification of vacation images; at the highest level, images are classified as indoor or outdoor; outdoor images are further classified as city or landscape; finally, a subset of landscape images is classified into sunset, forest, and mountain classes. We demonstrate that a small vector quantizer (whose optimal size is selected using a modified MDL criterion) can be used to model the class-conditional densities of the features, required by the Bayesian methodology. The classifiers have been designed and evaluated on a database of 6931 vacation photographs. Our system achieved a classification accuracy of 90.5% for indoor/outdoor, 95.3% for city/landscape, 96.6% for sunset/forest and mountain, and 96% for forest/mountain classification problems. We further develop a learning method to incrementally train the classifiers as additional data become available. We also show preliminary results for feature reduction using clustering techniques. Our goal is to combine multiple two-class classifiers into a single hierarchical classifier.
---
paper_title: Predictive watershed: A fast watershed algorithm for video segmentation

paper_content:

The watershed transform is a key operator in video segmentation algorithms. However, the computation load of watershed transform is too large for real-time applications. In this paper, a new fast watershed algorithm, named P-watershed, for image sequence segmentation is proposed. By utilizing the temporal coherence property of the video signal, this algorithm updates watersheds instead of searching watersheds in every frame, which can avoid a lot of redundant computation. The watershed process can be accelerated, and the segmentation results are almost the same as those of conventional algorithms. Moreover, an intra-inter watershed scheme (IP-watershed) is also proposed to further improve the results. Experimental results show that this algorithm can save 20%-50% computation without degrading the segmentation results. This algorithm can be combined with any video segmentation algorithm to give more precise segmentation results. An example is also shown by combining a background registration and change-detection-based segmentation algorithm with P-Watershed. This new video segmentation algorithm can give accurate object masks with acceptable computation complexity.
---
paper_title: Region-based representations of image and video: Segmentation tools for multimedia services

paper_content:

This paper discusses region-based representations of image and video that are useful for multimedia services such as those supported by the MPEG-4 and MPEG-7 standards. Classical tools related to the generation of the region-based representations are discussed. After a description of the main processing steps and the corresponding choices in terms of feature spaces, decision spaces, and decision algorithms, the state of the art in segmentation is reviewed. Mainly tools useful in the context of the MPEG-4 and MPEG-7 standards are discussed. The review is structured around the strategies used by the algorithms (transition based or homogeneity based) and the decision spaces (spatial, spatio-temporal, and temporal). The second part of this paper proposes a partition tree representation of images and introduces a processing strategy that involves a similarity estimation step followed by a partition creation step. This strategy tries to find a compromise between what can be done in a systematic and universal way and what has to be application dependent. It is shown in particular how a single partition tree created with an extremely simple similarity feature can support a large number of segmentation applications: spatial segmentation, motion estimation, region-based coding, semantic object extraction, and region-based retrieval.
---
paper_title: Extraction of Perceptual Hue Feature Set for Color Image/Video Segmentation

paper_content:

In this paper, we present a simple but effective algorithm for the extraction of perceptual hue feature set used in color image/video segmentation with emphasis on color textures. Feature extraction, with significant impact on the overall image/video analysis process, plays a critical role in classification-based segmentation. Color textures are accurately characterized by the newly introduced feature set with invariance to illumination, translation, and rotation, which is contributed by the statistical scheme in exploring the distribution of six rudimentary colors and the achromatic component at local positions. The feature set provides characteristic information and enables segmentation that is more meaningful than the recently published works do.
---
paper_title: Multiresolution-Based Texture Adaptive Algorithm for High-Quality Deinterlacing

paper_content:

This paper introduces a texture analysis mechanism utilizing multiresolution technique to reduce false motion detection and hence thoroughly improve the interpolation results for high-quality deinterlacing. Conventional motion-adaptive deinterlacing algorithm selects from inter-field and intra-field interpolations according to motion. Accurate determination of motion information is essential for this purpose. Fine textures, having high local pixel variation, tend to cause false detection of motion. Based on hierarchical wavelet analysis, this algorithm provides much better perceptual visual quality and considerably higher PSNR than other motion adaptive deinterlacers as shown. In addition, a recursive 3-field motion detection algorithm is also proposed to achieve better performance than the traditional 2-field motion detection algorithm with little memory overhead.
---
paper_title: Real-time 3 D computed tomographic reconstruction using commodity graphics hardware

paper_content:

The recent emergence of various types of flat-panel x-ray detectors and C-arm gantries now enables the construction of novel imaging platforms for a wide variety of clinical applications. Many of these applications require interactive 3D image generation, which cannot be satisfied with inexpensive PC-based solutions using the CPU. We present a solution based on commodity graphics hardware (GPUs) to provide these capabilities. While GPUs have been employed for CT reconstruction before, our approach provides significant speedups by exploiting the various built-in hardwired graphics pipeline components for the most expensive CT reconstruction task, backprojection. We show that the timings so achieved are superior to those obtained when using the GPU merely as a multi-processor, without a drop in reconstruction quality. In addition, we also show how the data flow across the graphics pipeline can be optimized, by balancing the load among the pipeline components. The result is a novel streaming CT framework that conceptualizes the reconstruction process as a steady flow of data across a computing pipeline, updating the reconstruction result immediately after the projections have been acquired. Using a single PC equipped with a single high-end commodity graphics board (the Nvidia 8800 GTX), our system is able to process clinically-sized projection data at speeds meeting and exceeding the typical flat-panel detector data production rates, enabling throughput rates of 40–50 projections s−1 for the reconstruction of 5123 volumes.
---
paper_title: Parallel Encoding - Decoding Operation for Multiview Video Coding with High Coding Efficiency

paper_content:

Multiview Video Coding (MVC) standardization is an ongoing effort aiming to extend H.264/AVC by developing novel tools optimized for 3D and multiview video use cases. One of the key identified requirements for the MVC standard is its ability to support parallel processing of different views. The parallel operation is known to be especially important for 3DTV applications, where the display needs to output many views simultaneously to support head-motion parallax. In this paper, we present a novel coding structure that enables parallel encoder/decoder operation for different views, without compromising from the coding efficiency. This is achieved by systematically restricting the reference area of each view, so that encoding and decoding of macroblocks from different views could be efficiently pipelined and parallel operation of separate views becomes possible. As the inter-view prediction is still used, proposed structure achieves up to 0.9 dB gain compared to simulcast, maintaining very similar desirable parallelism characteristics.
---
paper_title: Overview of free viewpoint television

paper_content:

We have been developing ray-based 3D information systems that consist of ray acquisition, ray processing, and ray display. Free viewpoint television (FTV) based on the ray-space method is a typical example. FTV will bring an epoch-making change in the history of television because it enables us to view a distant 3D world freely by changing our viewpoints as if we were there. We constructed a real-time FTV including the complete chain from capturing to display. A new algorithm was developed to generate free viewpoint images. In addition, a new user interface is presented for FTV to make full use of 3D information. FTV is not a pixel-based system but a ray-based system. We are creating ray-based image engineering through the development of FTV.
---
paper_title: Towards systematic exploration of tradeoffs for medical image registration on heterogeneous platforms

paper_content:

For the past decade, improving the performance and accuracy of medical image registration has been a driving force of innovation in medical imaging. The ultimate goal of accurate, robust, real-time image registration will enhance diagnoses of patients and enable new image-guided intervention techniques. With such a computationally intensive and multifaceted problem, improvements have been found in high performance platforms such as graphics processors (GPUs) and general purpose clusters, but there has yet to be a solution fast enough and effective enough to gain widespread clinical use. In this study, we examine the differences in accuracy and speed of implementations of the same image registration algorithm on a general purpose uniprocessor, a GPU, and a cluster of GPUs. We utilize a novel domain specific framework that allows us to simultaneously exploit parallelism on a heterogeneous platform. Using a set of representative images, we examine implementations with speedups of up to two orders of magnitude and accuracy varying from sub-millimeter to 2.6 millimeters of average error.
---
paper_title: Interpolator data compression for MPEG-4 animation

paper_content:

Interpolator representation in key-frame animation is now the most popular method for computer animation. The interpolator data consist of key and key value pairs, where a key is a time stamp and a key value is the corresponding value to the key. In this paper, we propose a set of new technologies to compress the interpolator data. The performance of the proposed technique is compared with the existing MPEG-4 generic compression tool. Throughout the core experiments in MPEG-4, the proposed technique showed its superiority over the existing tool, becoming a part of MPEG-4 standard within the Animation Framework eXtension framework.
---
paper_title: Future graphics architectures

paper_content:

Graphics architectures are in the midst of a major transition. In the past, these were specialized architectures designed to support a single rendering algorithm: the standard Z buffer. Realtime 3D graphics has now advanced to the point where the Z-buffer algorithm has serious shortcomings for generating the next generation of higher-quality visual effects demanded by games and other interactive 3D applications. There is also a desire to use the high computational capability of graphics architectures to support collision detection, approximate physics simulations, scene management, and simple artificial intelligence. In response to these forces, graphics architectures are evolving toward a general-purpose parallel-programming model that will support a variety of image-synthesis algorithms, as well as nongraphics tasks.
---
paper_title: GPU algorithms for radiosity and subsurface scattering

paper_content:

We capitalize on recent advances in modern programmable graphics hardware, originally designed to support advanced local illumination models for shading, to instead perform two different kinds of global illumination models for light transport. We first use the new floating-point texture map formats to find matrix radiosity solutions for light transport in a diffuse environment, and use this example to investigate the differences between GPU and CPU performance on matrix operations. We then examine multiple-scattering subsurface light transport, which can be modeled to resemble a single radiosity gathering step. We use a multiresolution meshed atlas to organize a hierarchy of precomputed subsurface links, and devise a three-pass GPU algorithm to render in real time the subsurface-scattered illumination of an object, with dynamic lighting and viewing.
---
paper_title: An introduction to the MPEG-4 animation framework eXtension

paper_content:

This paper presents the MPEG-4 Animation Framework eXtension (AFX) standard, ISO/IEC 14496-16. Initiated by the MPEG Synthetic/Natural Hybrid Coding group in 2000, MPEG-4 AFX proposes an advanced framework for interactive multimedia applications using both natural and synthetic objects. Following this model, new synthetic objects have been specified, increasing content realism over existing MPEG-4 synthetic objects. The general overview of MPEG-4 AFX is provided on top of the review of MPEG-4 standards to explain the relationship between MPEG-4 and MPEG-4 AFX. Then we give a bird's-eye view of new tools available in this standard.
---
paper_title: REALIZATION AND OPTIMIZATION OF H.264 DECODER FOR DUAL-CORE SOC

paper_content:

A filter for filtering micro-emboli from a patient's blood during an angioplasty procedure is disclosed which comprises a plurality of curved wires connected to a rod between a first connector fixed with respect to the rod and a second connector slidingly mounted on the rod. Two layers of filter material are connected to opposite sides of the wires, and each layer includes perforations which are offset from the perforations in the other layer. When the rod and the wires are disposed within a catheter, the inner wall of the catheter compresses the wires toward the rod and when the rod is extended from the catheter, the wires resume their curved shape and pull the sliding connector along the rod toward the fixed connector.
---
paper_title: Overview of ITRI PAC project - from VLIW DSP processor to multicore computing platform

paper_content:

The Industrial Technology Research Institute (ITRI) PAC (parallel architecture core) project was initiated in 2003. The target is to develop a low-power and high-performance programmable SoC platform for multimedia applications. In the first PAC project phase (2004-2006), a 5-way VLIW DSP (PACDSP) processor has been developed with our patented distributed & ping-pong register file and variable-length VLIW encoding techniques. A dual-core PAC SoC, which is composed of a PACDSP core and an ARM9 core, has also been designed and fabricated in the TSMC 0.13 mum technology to demonstrate its outstanding performance and energy efficiency for multimedia processing such as real-time H.264 codec. This paper summarizes the technical contents of PACDSP, DVFS (dynamic voltage and frequency scaling) -enabled PAC SoC, and the energy-aware multimedia codec. The research directions of our second-phase PAC project (PAC II), including multicore architectures, ESL (electronics system-level) technology, and low-power multimedia framework, are also addressed in this paper.
---
paper_title: Trends in multicore DSP platforms

paper_content:

In the last two years, the embedded DSP market has been swept up by the general increase in interest in multicore that has been driven by companies such as Intel and Sun. One reason for this is that there is now a lot of focus on tooling in academia and also a willingness on the part of users to accept new programming paradigms. This industry-wide effort will have an effect on the way multicore DSPs are programmed and perhaps architected. But it is too early to say in what way this will occur. Programming multicore DSPs remains very challenging. The problem of how to take a piece of sequential code and optimally partition it across multiple cores remains unsolved. Hence, there will naturally be a lot of variations in the approaches taken. Equally important is the issue of debugging and visibility. Developing effective and easy-to-use code development and real-time debug tools is tremendously important as the opportunity for bugs goes up significantly when one starts to deal with both time and space. The markets that DSP plays in have unique features in their desire for low power, low cost, and hard real-time processing, with an emphasis on mathematical computation. How well the multicore research being performed presently in academia will address these concerns remains to be seen.
---
paper_title: Power efficient processor architecture and the cell processor

paper_content:

This paper provides a background and rationale for some of the architecture and design decisions in the cell processor, a processor optimized for compute-intensive and broadband rich media applications, jointly developed by Sony Group, Toshiba, and IBM. The paper discusses some of the challenges microprocessor designers face and provides motivation for performance per transistor as a reasonable first-order metric for design efficiency. Common microarchitectural enhancements relative to this metric are provided. Also alternate architectural choices and some of its limitations are discussed and non-homogeneous SMP as a means to overcome these limitations is proposed.
---
paper_title: Cell Broadband Engine Architecture and its first implementation—A performance view

paper_content:

The Cell Broadband Engine™ (Cell/B.E.) processor is the first implementation of the Cell Broadband Engine Architecture (CBEA), developed jointly by Sony, Toshiba, and IBM. In addition to use of the Cell/B.E. processor in the Sony Computer Entertainment PLAYSTATION® 3 system, there is much interest in using it for workstations, media-rich electronics devices, and video and image processing systems. The Cell/B.E. processor includes one PowerPC® processor element (PPE) and eight synergistic processor elements (SPEs). The CBEA is designed to be well suited for a wide variety of programming models, and it allows for partitioning of work between the PPE and the eight SPEs. In this paper we show that the Cell/B.E. processor can outperform other modern processors by approximately an order of magnitude and by even more in some cases.
---
paper_title: Larrabee: a many-core x86 architecture for visual computing

paper_content:

This paper presents a many-core visual computing architecture code named Larrabee, a new software rendering pipeline, a manycore programming model, and performance analysis for several applications. Larrabee uses multiple in-order x86 CPU cores that are augmented by a wide vector processor unit, as well as some fixed function logic blocks. This provides dramatically higher performance per watt and per unit of area than out-of-order CPUs on highly parallel workloads. It also greatly increases the flexibility and programmability of the architecture as compared to standard GPUs. A coherent on-die 2nd level cache allows efficient inter-processor communication and high-bandwidth local data access by CPU cores. Task scheduling is performed entirely with software in Larrabee, rather than in fixed function logic. The customizable software graphics rendering pipeline for this architecture uses binning in order to reduce required memory bandwidth, minimize lock contention, and increase opportunities for parallelism relative to standard GPUs. The Larrabee native programming model supports a variety of highly parallel applications that use irregular data structures. Performance analysis on those applications demonstrates Larrabee's potential for a broad range of parallel computation.
---
paper_title: Scientific Computing Kernels on the Cell Processor

paper_content:

Scientiﬁc Computing Kernels on the Cell Processor Samuel Williams, John Shalf, Leonid Oliker Shoaib Kamil, Parry Husbands, Katherine Yelick Computational Research Division Lawrence Berkeley National Laboratory Berkeley, CA 94720 { swwilliams,jshalf,loliker,sakamil,pjrhusbands,kayelick } @lbl.gov ABSTRACT The slowing pace of commodity microprocessor performance improvements combined with ever-increasing chip power de- mands has become of utmost concern to computational sci- entists. As a result, the high performance computing com- munity is examining alternative architectures that address the limitations of modern cache-based designs. In this work, we examine the potential of using the recently-released STI Cell processor as a building block for future high-end com- puting systems. Our work contains several novel contribu- tions. First, we introduce a performance model for Cell and apply it to several key scientiﬁc computing kernels: dense matrix multiply, sparse matrix vector multiply, stencil com- putations, and 1D/2D FFTs. The diﬃculty of programming Cell, which requires assembly level intrinsics for the best performance, makes this model useful as an initial step in algorithm design and evaluation. Next, we validate the ac- curacy of our model by comparing results against published hardware results, as well as our own implementations on a 3.2GHz Cell blade. Additionally, we compare Cell per- formance to benchmarks run on leading superscalar (AMD Opteron), VLIW (Intel Itanium2), and vector (Cray X1E) architectures. Our work also explores several diﬀerent map- pings of the kernels and demonstrates a simple and eﬀective programming model for Cell’s unique architecture. Finally, we propose modest microarchitectural modiﬁcations that could signiﬁcantly increase the eﬃciency of double-precision calculations. Overall results demonstrate the tremendous potential of the Cell architecture for scientiﬁc computations in terms of both raw performance and power eﬃciency. INTRODUCTION Over the last decade the HPC community has moved to- wards machines composed of commodity microprocessors as a strategy for tracking the tremendous growth in processor performance in that market. As frequency scaling slows and the power requirements of these mainstream processors con- tinue to grow, the HPC community is looking for alternative architectures that provide high performance on scientiﬁc ap- plications, yet have a healthy market outside the scientiﬁc community. In this work, we examine the potential of the recently-released STI Cell processor as a building block for future high-end computing systems, by investigating perfor- mance across several key scientiﬁc computing kernels: dense matrix multiply, sparse matrix vector multiply, stencil com- putations on regular grids, as well as 1D and 2D FFTs. Cell combines the considerable ﬂoating point resources re- quired for demanding numerical algorithms with a power- eﬃcient software-controlled memory hierarchy. Despite its radical departure from previous mainstream/commodity pro- cessor designs, Cell is particularly compelling because it will be produced at such high volumes that it will be cost- competitive with commodity CPUs. The current implemen- tation of Cell is most often noted for its extremely high per- formance single-precision arithmetic, which is widely consid- ered insuﬃcient for the majority of scientiﬁc applications. Although Cell’s peak double precision performance is still impressive relative to its commodity peers (˜14.6 Gﬂop/s @ 3.2GHz), we explore how modest hardware changes could signiﬁcantly improve performance for computationally in- tensive double precision applications. This paper presents several novel results and expands our previous eﬀorts [37]. We present quantitative performance data for scientiﬁc kernels that compares Cell performance to leading superscalar (AMD Opteron), VLIW (Intel Ita- nium2), and vector (Cray X1E) architectures. We believe this study examines the broadest array of scientiﬁc algo- rithms to date on Cell. We developed both analytical mod- els and lightweight simulators to predict kernel performance that we demonstrated to be accurate when compared against published Cell hardware results, as well as our own imple- mentations on a 3.2GHz Cell blade. Our work also explores the complexity of mapping several important scientiﬁc algo- rithms onto the Cell’s unique architecture in order to lever- age the large number of available functional units and the software-controlled memory. Additionally, we propose mod- est microarchitectural modiﬁcations that would increase the eﬃciency of double-precision arithmetic calculations com- pared with the current Cell implementation. Overall results demonstrate the tremendous potential of the Cell architecture for scientiﬁc computations in terms of both raw performance and power eﬃciency. We exploit Cell’s heterogeneity not in computation, but in control and system support. Thus we conclude that Cell’s heterogeneous multi-core implementation is inherently better suited to the HPC environment than homogeneous commodity multicore
---
paper_title: Multicore system-on-chip architecture for MPEG-4 streaming video

paper_content:

The newly defined MPEG-4 Advanced Simple (AS) profile delivers single-layered streaming video in digital television (DTV) quality in the promising 1-2 Mbit/s range. However, the coding tools involved add significantly to the complexity of the decoding process, raising the need for further hardware acceleration. A programmable multicore system-on-chip (SOC) architecture is presented which targets MPEG-4 AS profile decoding of ITU-R 601 resolution streaming video. Based on a detailed analysis of corresponding bitstream statistics, the implementation of an optimized software video decoder for the proposed architecture is described. Results show that overall performance is sufficient for real-time AS profile decoding of ITU-R 601 resolution video.
---
paper_title: Niagara: a 32-way multithreaded Sparc processor

paper_content:

The Niagara processor implements a thread-rich architecture designed to provide a high-performance solution for commercial server applications. This is an entirely new implementation of the Sparc V9 architectural specification, which exploits large amounts of on-chip parallelism to provide high throughput. The hardware supports 32 threads with a memory subsystem consisting of an on-board crossbar, level-2 cache, and memory controllers for a highly integrated design that exploits the thread-level parallelism inherent to server applications, while targeting low levels of power consumption.
---
paper_title: DVFS Aware Techniques on Parallel Architecture Core (PAC) Platform

paper_content:

Rapid developments of multimedia and communication technologies enrich the applications of portable devices. However, design flexibility and low power are two important criteria for real-time system development. In this paper, a DVFS-aware implementation is introduced to demonstrate intelligent dynamic voltage and frequency scaling (DVFS) technique on dual-core PAC Platform. The power management of DVFS technique is verified with the H.264/AVC decoder example which can save 46% of power consumption.
---
paper_title: An Architecture for Programmable Multi-core IP Accelerated Platform with an Advanced Application of H.264 Codec Implementation

paper_content:

A new integrated programmable platform architecture is presented, with the support of multiple accelerators and extensible processing cores. An advanced application for this architecture is to facilitate the implementation of H.264 baseline profile video codec. The platform architecture employs the novel concept of virtual socket and optimized memory access to increase the efficiency for video encoding. The proposed architecture is mapped on an integrated FPGA device, Annapolis WildCard-II? or WildCard-4?, for verification. According to the evaluation under different configurations, the results show that the overall performance of the architecture, with the integrated accelerators, can sufficiently meet the real-time encoding requirement for H.264 BP at basic levels, and achieve about 2---5.5 and 1---3 dB improvement, in terms of PSNR, as compared with MPEG-2 MP and MPEG-4 SP, respectively. The architecture is highly extensible, and thus can be utilized to benefit the development of multi-standard video codec beyond the description in this paper.
---
paper_title: A Simulation and Exploration Technology for Multimedia-Application-Driven Architectures

paper_content:

The increasing of computational power requirements for DSP and Multimedia application and the needs of easy-to-program development environment has driven recent programmable devices toward Very Long Instruction Word (VLIW) [1] architectures and Hw-Sw co-design environments [2]. VLIW architecture allows generating optimized machine code from high-level languages exploiting Instruction Level Parallelism (ILP) [3]. Furthermore, applications requirements and time to market constraints are growing dramatically moving functionalities toward System on Chip (SoC) direction. This paper presents VLIW-SIM, an Application-Driven Architecture-design approach based on Instruction Set simulation. VLIW architectures and Instruction Set simulation were chosen to fulfill multimedia domain requirements and to implement an efficient Hw-Sw co-design environment. The VLIW-SIM simulation technology is based on pipeline status modeling, Simulation cache and Simulation Oriented Hw description. An effective support for Hw-Sw co-design requires high simulation performance (in terms of Simulated Instruction per Second--SIPS), flexibility (the ability to represent a number of different architectures) and cycle accuracy. There is a strong trade-off between these features: cycle accurate or close to cycle accurate simulation have usually low performance [4, 5]. Good simulation performance can be obtained loosing the simulator flexibility. Moreover SoC simulation requires a further degree of flexibility in simulating different components (core, co-processors, memories, buses). The proposed approach is focused on interpretative (not compiled [6]) re-configurable Instruction Set Simulator (ISS) in order to support both application design and architecture exploration. VLIW-SIM main features are: efficient host resource allocation, Instruction Set and Architecture description Flexibility (Instruction Set Dynamic Generation and Simulation Oriented Hardware Description), Step by step pipeline status tracking, Simulation Speed and Accuracy. Performance of simulation test for three validation case studies (TI TMS320C62x, TI TMS320C64x and ST200) are reported.
---
paper_title: A Platform-Independent Methodology for Performance Estimation of Multimedia Signal Processing Applications

paper_content:

A methodological framework for performance estimation of multimedia signal processing applications on different implementation platforms is presented. The methodology derives a complexity profile which is characteristic for an application, but completely platform-independent. By correlating the complexity profile with platform-specific data, performance estimation results for different platforms are obtained. The methodology is based on a reference software implementation of the targeted application, but is, in constrast to instruction-level profiling-based approaches, fully independent of its optimization degree. The proposed methodology is demonstrated by example of an MPEG-4 Advanced Simple Profile (ASP) video decoder. Performance estimation results are presented for two different platforms, a specialized VLIW media processor and an embedded general-purpose RISC processor, showing a high accuracy of he methodology. The approach can be employed to assist in design decisions in the specification phase of new architectures, in the selection process of a suitable target platform for a multimedia application, or in the optimization stage of a software implementation on a specific platform.
---
paper_title: System-Level Performance Analysis for Designing On-Chip Communication Architectures

paper_content:

This paper presents a novel system-level performance analysis technique to support the design of custom communication architectures for system-on-chip integrated circuits. Our technique fills a gap in existing techniques for system-level performance analysis, which are either too slow to use in an iterative communication architecture design framework (e.g., simulation of the complete system) or are not accurate enough to drive the design of the communication architecture (e.g., techniques that perform a "static" analysis of the system performance). Our technique is based on a hybrid trace-based performance-analysis methodology in which an initial cosimulation of the system is performed with the communication described in an abstract manner (e.g., as events or abstract data transfers). An abstract set of traces are extracted from the initial cosimulation containing necessary and sufficient information about the computations and communications of the system components. The system designer then specifies a communication architecture by: 1) selecting a topology consisting of dedicated as well as shared communication channels (shared buses) interconnected by bridges; 2) mapping the abstract communications to paths in the communication architecture; and 3) customizing the protocol used for each channel. The traces extracted in the initial step are represented as a communication analysis graph (CAG) and an analysis of the CAG provides an estimate of the system performance as well as various statistics about the components and their communication. Experimental results indicate that our performance-analysis technique achieves accuracy comparable to complete system simulation (an average error of 1.88%) while being over two orders of magnitude faster.
---
paper_title: Efficient optimal design space characterization methodologies

paper_content:

One of the primary advantages of a high-level synthesis system is its ability to explore the design space. This paper presents several methodologies for design space exploration that compute all optimal tradeoff points for the combined problem of scheduling, clock-length determination, and module selection. We discuss how each methodology takes advantage of the structure within the design space itself as well as the structure of, and interactions among, each of the three subproblems. (CAD)
---
paper_title: Comparing analytical modeling with simulation for network processors: a case study

paper_content:

Programming network processors remains an art due to the variety of different network processor architectures and due to little support to reason and explore implementations on such architectures. We present a case study of mapping an IPv4 forwarding switch application on the Intel IXP1200 network processor and we compare this implementation with an analytical model of both the application and architecture used to evaluate different design alternatives. Our results not only show that we are able to model the IXP1200 and our application within 15% of the accuracy compared to that of IXP1200 simulation, but also find closely matching trends for different workloads. This shows the clear potential of such analytical techniques for design space exploration.
---
paper_title: An integrated environment for HW/SW co-design based on a CAL specification and HW/SW code generators

paper_content:

This demonstration presents an integrated environment that translates a CAL-based dataflow specification [1] into a heterogeneous implementation, composed by HDL and C codes. The demonstration focuses on the capability of the co-design environment to automatically build an executable heterogeneous system implementation running on a platform composed of a processor and a FPGA from the annotation of the CAL specification. The possibility of direct synthesis from a high level specification is a crucial issue for enabling efficient re-design cycles that include rapid prototyping and validation of performances of the final implementation. The design approach enabled by such integrated environment is particularly suited for development of complex processing systems such as video codecs. As a case study, the demonstration provides the analysis and validation of different software and hardware partitioning of a MPEG-4 Simple Profile decoder.
---
paper_title: A hierarchical simulation framework for application development on system-on-chip architectures

paper_content:

We propose a hierarchical simulation methodology to assist application development on System-on-Chip architectures. Hierarchical simulation involves simulation of a SoC based system at different levels of abstraction. Thus, it enables a system designer to exploit simulation speed vs. accuracy of results trade-offs. Vertical simulation is a special case of hierarchical simulation, where a feedback mechanism between the different simulation levels helps in "interpreting" the results of stand-alone simulations in the system-wide context. The paper presents an approach to perform vertical simulation of a class of applications under a simplified scenario.
---
paper_title: A framework for evaluating design tradeoffs in packet processing architectures

paper_content:

We present an analytical method to evaluate embedded network packet processor architectures, and to explore their design space. Our approach is in contrast to those based on simulation, which tend to be infeasible when the design space is very large. We illustrate the feasibility of our method using a detailed case study.
---
paper_title: YAPI: application modeling for signal processing systems

paper_content:

We present a programming interface called YAPI to model signal processing applications as process networks. The purpose of YAPI is to enable the reuse of signal processing applications and the mapping of signal processing applications onto heterogeneous systems that contain hardware and software components. To this end, YAPI separates the concerns of the application programmer, who determines the functionality of the system, and the system designer, who determines the implementation of the functionality. The proposed model of computation extends the existing model of Kahn process networks with channel selection to support non-deterministic events. We provide an efficient implementation of YAPI in the form of a C++ run-time library to execute the applications on a workstation. Subsequently, the applications are used by the system designer as input for mapping and performance analysis in the design of complex signal processing systems. We evaluate this methodology on the design of a digital video broadcast system-on-chip.
---
paper_title: Algorithm/Architecture Co-Design of 3-D Spatio–Temporal Motion Estimation for Video Coding

paper_content:

This paper presents a new spatio-temporal motion estimation algorithm and its VLSI architecture for video coding based on algorithm and architecture co-design methodology. The algorithm consists of the new strategies of spatio-temporal motion vector prediction, modified one-at-a-time search scheme, and multiple update paths derived from optimization theory. The hardware specification is for high-definition video coding. We applied the ME algorithm to H.264 reference software. Our algorithm surpasses recently published research and achieves close performance to full search. The VLSI implementation proves the low cost feature of our algorithm. The algorithm and architecture co-design concept is highly emphasized in this paper. We provide some quantitative example to show the necessity of algorithm and architecture co-design
---
paper_title: Parallel architecture overview

paper_content:

Abstract An increasing number of parallel computer products are appearing in the market place. Their design motivations and market areas cover a broad spectrum: (i) Transaction Processing Systems, such as Parallel UNIX systems (e.g. SEQUENT Balance), for data processing applications; (ii) Numeric Supercomputers, such as Hypercube systems (e.g. INTEL iPSC), for scientific and engineering applications; (iii) VLSI Architectures, such as parallel microcomputers (e.g. INMOS Transputer), for exploiting very large scales of integration; (iv) High-Level Language Computers, such as Logic machines (e.g. FUJITSU Kabu-Wake), for symbolic computation; and (v) Neurocomputers, such as Connectionist computers (e.g. THINKING MACHINES Connection Machine), for general-purpose pattern matching applications. This survey paper gives an overview of these novel parallel computers and discusses the likely commercial impact of parallel computers.
---
paper_title: VLSI Array processors

paper_content:

High speed signal processing depends critically on parallel processor technology. In most applications, general-purpose parallel computers cannot offer satisfactory real-time processing speed due to severe system overhead. Therefore, for real-time digital signal processing (DSP) systems, special-purpose array processors have become the only appealing alternative. In designing or using such array Processors, most signal processing algorithms share the critical attributes of regularity, recursiveness, and local communication. These properties are effectively exploited in innovative systolic and wavefront array processors. These arrays maximize the strength of very large scale integration (VLSI) in terms of intensive and pipelined computing, and yet circumvent its main limitation on communication. The application domain of such array processors covers a very broad range, including digital filtering, spectrum estimation, adaptive array processing, image/vision processing, and seismic and tomographic signal processing, This article provides a general overview of VLSI array processors and a unified treatment from algorithm, architecture, and application perspectives.
---
paper_title: High-abstraction level complexity analysis and memory architecture simulations of multimedia algorithms

paper_content:

An appropriate complexity analysis stage is the first and fundamental step for any methodology aiming at the implementation of today's (complex) multimedia algorithms. Such a stage may have different final implementation goals such as defining a new architecture dedicated to the specific multimedia standard under study, or defining an optimal instruction set for a selected processor architecture, or to guide the software optimization process in terms of control-flow and data-flow optimization targeting a specific architecture. The complexity of nowadays multimedia standards, in terms of number of lines of codes and cross-relations among processing algorithms that are activated by specific input signals, goes far beyond what the designer can reasonably grasp from the "pencil and paper" analysis of the (software) specifications. Moreover, depending on the implementation goal different measures and metrics are required at different steps of the implementation methodology or design flow. The process of extracting the desired measures needs to be supported by appropriate automatic tools, since code rewriting, at each design stage, may result resource consuming and error prone. This paper reviews the state of the art of complexity analysis methodologies oriented to the design of multimedia systems and presents an integrated tool for automatic analysis capable of producing complexity results based on rich and customizable metrics. The tool is based on a C virtual machine that allows extracting from any C program execution the operations and data-flow information, according to the defined metrics. The tool capabilities include the simulation of virtual memory architectures. This paper shows some examples of complexity analysis results that can be yielded with the tool and presents how the tools can be used at different stages of implementation methodologies.
---
paper_title: Overview of parallel processing approaches to image and video compression

paper_content:

In this paper we present an overview of techniques used to implement various image and video compression algorithms using parallel processing. Approaches used can largely be divided into four areas. The first is the use of special purpose architectures designed specifically for image and video compression. An example of this is the use of an array of DSP chips to implement a version of MPEG1. The second approach is the use of VLSI techniques. These include various chip sets for JPEG and MPEG1. The third approach is algorithm driven, in which the structure of the compression algorithm describes the architecture, e.g. pyramid algorithms. The fourth approach is the implementation of algorithms on high performance parallel computers. Examples of this approach are the use of a massively parallel computer such as the MasPar MP-1 or the use of a coarse-grained machine such as the Intel Touchstone Delta.
---
paper_title: Evaluation of the parallelization potential for efficient multimedia implementations: dynamic evaluation of algorithm critical path

paper_content:

This paper presents a model metrics and a methodology for evaluating the critical path on the data flow execution graph (DFEG) of multimedia algorithms specified as C programs. The paper describes an efficient dynamic critical path evaluation approach that generates no explicit execution graph. Such approach includes two key stages: 1) the instrumentation of the C code and the mapping into a C++ code version and 2) the execution of the C++ code under real input data and finally the actual dynamic evaluation of the critical path. The model metrics and the software analysis methodologies aim at the estimation and at the increase of the upper bound of execution speed and parallelization potential. Both metrics and methodology are particularly tailored for application with complex multimedia algorithms. Critical path analysis and the subsequent algorithmic development stage is a fundamental methodological preliminary step for the efficient definition of architectures when the objective is the implementation of the multimedia algorithm on systems-on-chips or heterogeneous platforms.
---
paper_title: Profiling dataflow programs

paper_content:

As dataflow descriptions of media processing become popular, the techniques for analyzing and profiling the performance of sequential algorithms are no longer applicable. This paper describes some of the basic concepts and techniques for analyzing the computations described by dataflow programs, and illustrates them on an MPEG-4 decoder.
---
paper_title: Algorithm/Architecture Co-Design of 3-D Spatio–Temporal Motion Estimation for Video Coding

paper_content:

This paper presents a new spatio-temporal motion estimation algorithm and its VLSI architecture for video coding based on algorithm and architecture co-design methodology. The algorithm consists of the new strategies of spatio-temporal motion vector prediction, modified one-at-a-time search scheme, and multiple update paths derived from optimization theory. The hardware specification is for high-definition video coding. We applied the ME algorithm to H.264 reference software. Our algorithm surpasses recently published research and achieves close performance to full search. The VLSI implementation proves the low cost feature of our algorithm. The algorithm and architecture co-design concept is highly emphasized in this paper. We provide some quantitative example to show the necessity of algorithm and architecture co-design
---
paper_title: REALIZATION AND OPTIMIZATION OF H.264 DECODER FOR DUAL-CORE SOC

paper_content:

A filter for filtering micro-emboli from a patient's blood during an angioplasty procedure is disclosed which comprises a plurality of curved wires connected to a rod between a first connector fixed with respect to the rod and a second connector slidingly mounted on the rod. Two layers of filter material are connected to opposite sides of the wires, and each layer includes perforations which are offset from the perforations in the other layer. When the rod and the wires are disposed within a catheter, the inner wall of the catheter compresses the wires toward the rod and when the rod is extended from the catheter, the wires resume their curved shape and pull the sliding connector along the rod toward the fixed connector.
---
paper_title: Parallel Encoding - Decoding Operation for Multiview Video Coding with High Coding Efficiency

paper_content:

Multiview Video Coding (MVC) standardization is an ongoing effort aiming to extend H.264/AVC by developing novel tools optimized for 3D and multiview video use cases. One of the key identified requirements for the MVC standard is its ability to support parallel processing of different views. The parallel operation is known to be especially important for 3DTV applications, where the display needs to output many views simultaneously to support head-motion parallax. In this paper, we present a novel coding structure that enables parallel encoder/decoder operation for different views, without compromising from the coding efficiency. This is achieved by systematically restricting the reference area of each view, so that encoding and decoding of macroblocks from different views could be efficiently pipelined and parallel operation of separate views becomes possible. As the inter-view prediction is still used, proposed structure achieves up to 0.9 dB gain compared to simulcast, maintaining very similar desirable parallelism characteristics.
---
paper_title: Parallel Process of Hyper-Space-Based Multiview Video Compression

paper_content:

Multiview video coding (MVC) is a key technology in free-viewpoint television. MVC based on traditional existing codec system has been studied widely, but all of them need powerful computational capacity in processing. Parallel process of MVC can facilitate the efficient implementation of encoder and decoder and has been required as a function by MPEG. In this paper, a parallelization methodology for MVC based on hyper-space theory is presented and tested on the local area multi-computer - message passing interface (LAM-MPI) parallel platform and modified H.264 codec. Experimental results show that the proposed method can speed up processing of multiview video compression and obtain high rate-distortion results.
---
paper_title: Multi-pass algorithm of motion estimation in video encoding for generic GPU

paper_content:

TABLEI Abstract- Theimportance ofvideoencoding hasboomed rapidly since video datacommunication waswidely needed. In this paper, wepropose amulti-pass algorithm toaccelerate the motionestimation (ME),thedominant partinvideoencoding, Tools
---
paper_title: Parallelization of AdaBoost algorithm on multi-core processors

paper_content:

This paper examines and extracts the parallelism in the AdaBoost person detection algorithm on multi-core processors. As multi-core processors become pervasive, effectively executing many threads simultaneously is crucial in harnessing the computation power. Although the application exposes many levels of parallelism, none of them delivers a satisfactory scaling performance on newest multi-core processors due to load imbalance and parallel overhead. This paper demonstrates how to analyze the thread-level parallelism, and how to choose appropriate one to utilize current 4-core and 8-core processors. With careful optimization and parallelization, the AdaBoost person detection algorithm can efficiently utilize the power of multi-core processors, and now it is 7 times faster than the serial version.
---
paper_title: Parallel Scalability of Video Decoders

paper_content:

An important question is whether emerging and future applications exhibit sufficient parallelism, in particular thread-level parallelism, to exploit the large numbers of cores future chip multiprocessors (CMPs) are expected to contain. As a case study we investigate the parallelism available in video decoders, an important application domain now and in the future. Specifically, we analyze the parallel scalability of the H.264 decoding process. First we discuss the data structures and dependencies of H.264 and show what types of parallelism it allows to be exploited. We also show that previously proposed parallelization strategies such as slice-level, frame-level, and intra-frame macroblock (MB) level parallelism, are not sufficiently scalable. Based on the observation that inter-frame dependencies have a limited spatial range we propose a new parallelization strategy, called Dynamic 3D-Wave. It allows certain MBs of consecutive frames to be decoded in parallel. Using this new strategy we analyze the limits to the available MB-level parallelism in H.264. Using real movie sequences we find a maximum MB parallelism ranging from 4000 to 7000. We also perform a case study to assess the practical value and possibilities of a highly parallelized H.264 application. The results show that H.264 exhibits sufficient parallelism to efficiently exploit the capabilities of future manycore CMPs.
---
paper_title: REAL-TIME MUTUAL-INFORMATION-BASED LINEAR REGISTRATION ON THE CELL BROADBAND ENGINE PROCESSOR

paper_content:

Emerging multi-core processors are able to accelerate medical imaging applications by exploiting the parallelism available in their algorithms. We have implemented a mutual-information-based 3D linear registration algorithm on the Cell Broadband Enginetrade (CBE) processor, which has nine processor cores on a chip and has a 4-way SIMD unit for each core. By exploiting the highly parallel architecture and its high memory bandwidth, our implementation with two CBE processors can compute mutual information for about 33 million pixel pairs in a second. This implementation is significantly faster than a conventional one on a traditional microprocessor or even faster than a previously reported custom-hardware implementation. As a result, it can register a pair of 256times256times30 3D images in one second by using a multi-resolution method. This paper describes our implementation with a focus on localized sampling and speculative packing techniques, which reduce the amount of the memory traffic by 82%
---
paper_title: A Multi-core Architecture Based Parallel Framework for H.264/AVC Deblocking Filters

paper_content:

Deblocking filter is one of the most time consuming modules in the H.264/AVC decoder as indicated in many studies. Therefore, accelerating deblocking filter is critical for improving the overall decoding performance. This paper proposes a novel parallel algorithm for H.264/AVC deblocking filter to speed the H.264/AVC decoder up. We exploit pixel-level data parallelism among filtering steps, and observe that results of each filtering step only affect a limited region of pixels. We call this "the limited propagation effect". Based on this observation, the proposed algorithm could partition a frame into multiple independent rectangles with arbitrary granularity. The proposed parallel deblocking filter algorithm requires very little synchronization overhead, and provides good scalability. Experimental results show that applying the proposed parallelization method to a SIMD optimized sequential deblocking filter achieves up to 95.31% and 224.07% speedup on a two-core and four-core processor, respectively. We have also observed a significant speedup for H.264/AVC decoding, 21% and 34% on a two-core and four-core processor, respectively.
---
paper_title: On the efficient algorithm/architecture co-exploration for complex video processing

paper_content:

Targeted for highly sophisticated visual signal processing, we introduce in this paper complexity metrics or measures of algorithms which featuring architectural information are feedback or back annotated in early design stages to facilitate concurrent exploration of both algorithmic and architectural optimizations. With application to 3D spatio-temporal motion estimation for video coding, we have demonstrated significant reduction in design cost while the algorithmic performance still surpasses recent published works and even full search under many circumstances. Moreover, we have also shown the importance and substantiality of this complexity analysis technique in the extraction of features common to various different de-interlacing algorithms adapted for versatile video content, in designing highly efficient reconfigurable video architectures. As such this novel algorithm/architecture co-exploration methodology forms the basis for dataflow models with more accurate software/hardware partitioning resulting in multi-million gate and/or instruction software simulation platforms and fast prototyping hardware platforms for the next generation electronic system level design of SoCpsilas.
---
paper_title: Real-time Visual Tracker by Stream Processing

paper_content:

In this work, we implement a real-time visual tracker that targets the position and 3D pose of objects in video sequences, specifically faces. The use of stream processors for the computations and efficient Sparse-Template-based particle filtering allows us to achieve real-time processing even when tracking multiple objects simultaneously in high-resolution video frames. Stream processing is a relatively new computing paradigm that permits the expression and execution of data-parallel algorithms with great efficiency and minimum effort. Using a GPU (graphics processing unit, a consumer-grade stream processor) and the NVIDIA CUDA? technology, we can achieve performance improvements as large as ten times compared to a similar CPU-only tracker. At the same time, the Stream processing approach opens the door to other computing devices, like the Cell/BE? or other multicore CPUs.
---
paper_title: Mapping of h.264 decoding on a multiprocessor architecture

paper_content:

Due to the increasing significance of development costs in the competitive domain of high-volume consumer electronics, generic solutions are required to enable reuse of the design effort and to increase the potential market volume. As a result from this, Systems-on-Chip (SoCs) contain a growing amount of fully programmable media processing devices as opposed to application-specific systems, which offered the most attractive solutions due to a high performance density. The following motivates this trend. First, SoCs are increasingly dominated by their communication infrastructure and embedded memory, thereby making the cost of the functional units less significant. Moreover, the continuously growing design costs require generic solutions that can be applied over a broad product range. Hence, powerful programmable SoCs are becoming increasingly attractive. However, to enable power-efficient designs, that are also scalable over the advancing VLSI technology, parallelism should be fully exploited. Both task-level and instruction-level parallelism can be provided by means of e.g. a VLIW multiprocessor architecture. ::: ::: To provide the above-mentioned scalability, we propose to partition the data over the processors, instead of traditional functional partitioning. An advantage of this approach is the inherent locality of data, which is extremely important for communication-efficient software implementations. Consequently, a software implementation is discussed, enabling e.g. SD resolution H.264 decoding with a two-processor architecture, whereas High-Definition (HD) decoding can be achieved with an eight-processor system, executing the same software. Experimental results show that the data communication considerably reduces up to 65% directly improving the overall performance. Apart from considerable improvement in memory bandwidth, this novel concept of partitioning offers a natural approach for optimally balancing the load of all processors, thereby further improving the overall speedup.
---
paper_title: Parallelization strategies and performance analysis of media mining applications on multicore processors

paper_content:

This paper studies how to parallelize the emerging media mining workloads on existing small-scale multi-core processors and future large-scale platforms. Media mining is an emerging technology to extract meaningful knowledge from large amounts of multimedia data, aiming at helping end users search, browse, and manage multimedia data. Many of the media mining applications are very complicated and require a huge amount of computing power. The advent of multi-core architectures provides the acceleration opportunity for media mining. However, to efficiently utilize the multi-core processors, we must effectively execute many threads at the same time. In this paper, we present how to explore the multi-core processors to speed up the computation-intensive media mining applications. We first parallelize two media mining applications by extracting the coarse-grained parallelism and evaluate their parallel speedups on a small-scale multi-core system. Our experiment shows that the coarse-grained parallelization achieves good scaling performance, but not perfect. When examining the memory requirements, we find that these coarse-grained parallelized workloads expose high memory demand. Their working set sizes increase almost linearly with the degree of parallelism, and the instantaneous memory bandwidth usage prevents them from perfect scalability on the 8-core machine. To avoid the memory bandwidth bottleneck, we turn to exploit the fine-grained parallelism and evaluate the parallel performance on the 8-core machine and a simulated 64-core processor. Experimental data show that the fine-grained parallelization demonstrates much lower memory requirements than the coarse-grained one, but exhibits significant read-write data sharing behavior. Therefore, the expensive inter-thread communication limits the parallel speedup on the 8-core machine, while excellent speedup is observed on the large-scale processor as fast core-to-core communication is provided via a shared cache. Our study suggests that (1) extracting the coarse-grained parallelism scales well on small-scale platforms, but poorly on large-scale system; (2) exploiting the fine-grained parallelism is suitable to realize the power of large-scale platforms; (3) future many-core chips can provide shared cache and sufficient on-chip interconnect bandwidth to enable efficient inter-core communication for applications with significant amounts of shared data. In short, this work demonstrates proper parallelization techniques are critical to the performance of multi-core processors. We also demonstrate that one of the important factors in parallelization is the performance analysis. The parallelization principles, practice, and performance analysis methodology presented in this paper are also useful for everyone to exploit the thread-level parallelism in their applications.
---
paper_title: An efficient block motion estimation method on CELL BE

paper_content:

In order to take advantage of the byte-type data parallelism in CELL BE's SIMD technique, this paper introduces a new algorithm, which can achieve much higher prediction accuracy and lower computational complexity compared with diamond search (DS) algorithm. Most of the conventional fast motion estimation algorithms just focus on reducing the number of search points. However, less search points is not equal to less computational complexity for CELL BE's architecture. The introduced method uses a large rectangular search pattern including 35 search points at the first search step. Through increasing search points, it gains 4.32% coding efficiency compared with DS. Through SAD reuse and reducing unaligned memory access, it needs 29.5% less computation than DS method on CELL BE. Our implementation of this algorithm on CELL BE shows that we can satisfy the requirement of real-time HD H.264 encoding.
---
paper_title: A 3D Spatio-Temporal Motion Estimation Algorithm for Video Coding

paper_content:

This paper presents a new spatio-temporal motion estimation algorithm for video coding. The algorithm is based on optimization theory and consists of the strategies including 3D spatio-temporal motion vector prediction, modified one-at-a-time search scheme, and multiple update paths. The simulation results indicate our algorithm is better than other recently proposed ones under the same computational budget and is very close to full search. The low-cost feature and regular demand of computational resource make our algorithm suitable for VLSI implementation. The algorithm also makes single chip solution for high-definition coding feasible.
---
paper_title: Multicore system-on-chip architecture for MPEG-4 streaming video

paper_content:

The newly defined MPEG-4 Advanced Simple (AS) profile delivers single-layered streaming video in digital television (DTV) quality in the promising 1-2 Mbit/s range. However, the coding tools involved add significantly to the complexity of the decoding process, raising the need for further hardware acceleration. A programmable multicore system-on-chip (SOC) architecture is presented which targets MPEG-4 AS profile decoding of ITU-R 601 resolution streaming video. Based on a detailed analysis of corresponding bitstream statistics, the implementation of an optimized software video decoder for the proposed architecture is described. Results show that overall performance is sufficient for real-time AS profile decoding of ITU-R 601 resolution video.
---
paper_title: An Architecture for Programmable Multi-core IP Accelerated Platform with an Advanced Application of H.264 Codec Implementation

paper_content:

A new integrated programmable platform architecture is presented, with the support of multiple accelerators and extensible processing cores. An advanced application for this architecture is to facilitate the implementation of H.264 baseline profile video codec. The platform architecture employs the novel concept of virtual socket and optimized memory access to increase the efficiency for video encoding. The proposed architecture is mapped on an integrated FPGA device, Annapolis WildCard-II? or WildCard-4?, for verification. According to the evaluation under different configurations, the results show that the overall performance of the architecture, with the integrated accelerators, can sufficiently meet the real-time encoding requirement for H.264 BP at basic levels, and achieve about 2---5.5 and 1---3 dB improvement, in terms of PSNR, as compared with MPEG-2 MP and MPEG-4 SP, respectively. The architecture is highly extensible, and thus can be utilized to benefit the development of multi-standard video codec beyond the description in this paper.
---
paper_title: Analysis and Parallelization of H.264 decoder on Cell Broadband Engine Architecture

paper_content:

Emerging video coding technology like H.264/AVC achieves high compression efficiency, which enables high quality video at the same or lower bitrate. However, those advanced coding techniques come at the cost of more computational power. Developed with such multimedia applications in mind, the CELL broadband engine (BE) processor was designed as a heterogeneous on-chip multicore processor to meet the required high performance. In this paper, we analyze the computational requirements of H.264 decoder per-module basis and implement parallelized H.264 decoder on the CELL processor based on the profile result. We propose and implement a hybrid partitioning technique that combines both functional and data partitioning to avoid the dependencies imposed by H.264 decoder, and optimize it using SIMD instructions. Through experiments, the parallelized H.264 decoder runs about 3.5 times faster than the single core (PPE only) decoder, by using 1 PPE and 4 SPEs.
---

Title: Algorithm/Architecture Co-Exploration of Visual Computing on Emergent Platforms: Overview and Future Prospects

Section 1: Introduction
Description 1: This section introduces the concept of Algorithm/Architecture Co-Exploration, the challenges of traditional design methodologies, and the increasing demands of video design challenges.

Section 2: Advanced Visual Computing Algorithms for Versatile Applications
Description 2: This section surveys advanced visual computing algorithms for emerging applications in four areas: video coding, video processing, computer vision, and computer graphics.

Section 3: Visual Computing on Multicore and Reconfigurable Architectures
Description 3: This section describes the spectrum of emerging architectures for visual computing, the challenges of mapping and porting algorithms onto these platforms, and the need for algorithm/architecture co-exploration.

Section 4: Algorithm/Architecture Co-Exploration
Description 4: This section outlines different design scenarios (architecture-oriented, algorithm-oriented, and algorithm/architecture design), and discusses the importance of concurrent exploration of both algorithmic and architecture optimizations.

Section 5: Dataflow Models for the Representation and Co-Design of Algorithms and Architectures
Description 5: This section discusses various dataflow representations that provide good models for the co-exploration of algorithms and architecture.

Section 6: Algorithmic Intrinsic Complexity Characterization
Description 6: This section elaborates on intrinsic complexity metrics or measures of algorithms that provide architectural information to facilitate concurrent exploration of both algorithmic and architectural optimizations.

Section 7: Dataflow Modeling and Complexity Characterization for Multicore and Reconfigurable Systems Design
Description 7: This section discusses the complexity measurements and dataflow modeling techniques used for designing and mapping algorithms onto multicore and reconfigurable architectures.

Section 8: Innovative Architectures with Multiple Processors and Reconfigurability for Video Coding and Processing
Description 8: This section surveys how visual computing algorithms are implemented on multicore platforms and how architecture-algorithm co-exploration can be used to map these applications effectively.

Section 9: Conclusion
Description 9: This section summarizes the importance of concurrently optimizing both algorithm and architecture, noting the advantages of advanced multicore platforms and reconfigurable architectures for future visual computing algorithms.