CMOS Image Sensor with On-Chip Image Compression: A Review and Performance Analysis
---
paper_title: A CMOS Image Sensor With On-Chip Image Compression Based on Predictive Boundary Adaptation and Memoryless QTD Algorithm

paper_content:

This paper presents the architecture, algorithm, and VLSI hardware of image acquisition, storage, and compression on a single-chip CMOS image sensor. The image array is based on time domain digital pixel sensor technology equipped with nondestructive storage capability using 8-bit Static-RAM device embedded at the pixel level. The pixel-level memory is used to store the uncompressed illumination data during the integration mode as well as the compressed illumination data obtained after the compression stage. An adaptive quantization scheme based on fast boundary adaptation rule (FBAR) and differential pulse code modulation (DPCM) procedure followed by an online, least storage quadrant tree decomposition (QTD) processing is proposed enabling a robust and compact image compression processor. A prototype chip including 64×64 pixels, read-out and control circuitry as well as an on-chip compression processor was implemented in 0.35 μm CMOS technology with a silicon area of 3.2×3.0 mm2 and an overall power of 17 mW. Simulation and measurements results show compression figures corresponding to 0.6-1 bit-per-pixel (BPP), while maintaining reasonable peak signal-to-noise ratio levels.
---
paper_title: Subband image coding using watershed and watercourse lines of the wavelet transform.

paper_content:

Reports progress in primitive-based image coding using nonorthogonal dyadic wavelets. A 3D isotropic wavelet is used to approximate the difference-of-Gaussians (D-o-G) operator. Convolution of the image with dilated versions of the wavelet produces three band-pass signals that approximate multiscale smoothed second derivatives. An additional convolution of the image with a Gaussian-shaped low-pass wavelet creates a fourth subband signal that preserves low-frequency information not described by the three band-pass signals. The authors show that the original image can be recovered from the watershed and watercourse lines of the three band-pass signals plus the lowpass subband signal. By thresholding the watershed/watercourse representation, subsampling the low-pass subband, and using edge post emphasis, the authors achieve data reduction with little loss of fidelity. Further compression of the watersheds and watercourses is achieved by chain coding their shapes and predictive coding their amplitudes prior to lossless arithmetic coding. Results are presented for grey-level test images at data rates between 0.1 and 0.3 b/pixel.
---
paper_title: A CMOS Image Sensor for Multi-Level Focal Plane Image Decomposition

paper_content:

An alternative image decomposition method that exploits prediction via nearby pixels has been integrated on the CMOS image sensor focal plane. The proposed focal plane decomposition is compared to the 2-D discrete wavelet transform (DWT) decomposition commonly used in state of the art compression schemes such as SPIHT and JPEG2000. The method achieves comparable compression performance with much lower computational complexity and allows image compression to be implemented directly on the sensor focal plane in a completely pixel parallel structure. A CMOS prototype chip has been fabricated and tested. The test results validate the pixel design and demonstrate that lossy prediction based focal plane image compression can be realized inside the sensor pixel array to achieve a high frame rate with much lower data readout volume. The features of the proposed decomposition scheme also benefit real-time, low rate and low power applications.
---
paper_title: Image Compression Based On Best Wavelet Packet Bases

paper_content:

An adaptive subband decomposition technique for efficient signal representation and compression is introduced and tested in a rate-distortion framework. The best wavelet packet bases exhibit a SFFT (short-time fast Fourier transform) subband decomposition at one source instance, a wavelet decomposition at another instance, or any intermediate wavelet packet decomposition at yet other instances to best match the signal's characteristics. For a given subband decomposition, commonly used information measures such as entropy, distortion (mse), and rate (no. of coefficients) are minimized over all subbands decomposition to search the most efficient wavelet packet tree of signal and a method to provide such a wavepacket tree is proposed. Image coding application results using the joint rate-distortion cost measure demonstrated superior performance over the entropy only information cost measure. >
---
paper_title: A CMOS Imager With Focal Plane Compression Using Predictive Coding

paper_content:

This paper presents a CMOS image sensor with focal-plane compression. The design has a column-level architecture and it is based on predictive coding techniques for image decorrelation. The prediction operations are performed in the analog domain to avoid quantization noise and to decrease the area complexity of the circuit. The prediction residuals are quantized and encoded by a joint quantizer/coder circuit. To save area resources, the joint quantizer/coder circuit exploits common circuitry between a single-slope analog-to-digital converter (ADC) and a Golomb-Rice entropy coder. This combination of ADC and encoder allows the integration of the entropy coder at the column level. A prototype chip was fabricated in a 0.35 mum CMOS process. The output of the chip is a compressed bit stream. The test chip occupies a silicon area of 2.60 mm times 5.96 mm which includes an 80 times 44 APS array. Tests of the fabricated chip demonstrate the validity of the design.
---
paper_title: Memory-efficient architecture for JPEG 2000 coprocessor with large tile image

paper_content:

The experimental results show that using a larger tile size to perform JPEG 2000 coding results in better image quality (i.e., greater than or equal to 256 /spl times/ 256 tile image). However, processing large tile images also requires relatively high memory for the hardware implementation. For example, it would require tile memory of 256 K words to support the process of a 512 /spl times/ 512 tile image in the straightforward architecture. To reduce hardware resources, we have proposed the quad code-block (QCB) -based discrete wavelet transform method to reduce the size of tile memory by a factor of 4. In this paper, the remaining 1/4 tile memory can be further reduced through two approaches: the zero-holding extension with slight image degradation and the QCB-block size extension without any image degradation. That is, it only requires 12 K words tile memory to support the process of 512 /spl times/ 512 tile image by using zero-holding extension, and 13.58 K words memory through QCB-block size extension. The low memory requirement makes the on-chip memory practicable.
---
paper_title: A VLSI architecture of JPEG2000 encoder

paper_content:

This paper proposes a VLSI architecture of JPEG2000 encoder, which functionally consists of two parts: discrete wavelet transform (DWT) and embedded block coding with optimized truncation (EBCOT). For DWT, a spatial combinative lifting algorithm (SCLA)-based scheme with both 5/3 reversible and 9/7 irreversible filters is adopted to reduce 50% and 42% multiplication computations, respectively, compared with the conventional lifting-based implementation (LBI). For EBCOT, a dynamic memory control (DMC) strategy of Tier-1 encoding is adopted to reduce 60% scale of the on-chip wavelet coefficient storage and a subband parallel-processing method is employed to speed up the EBCOT context formation (CF) process; an architecture of Tier-2 encoding is presented to reduce the scale of on-chip bitstream buffering from full-tile size down to three-code-block size and considerably eliminate the iterations of the rate-distortion (RD) truncation.
---
paper_title: Compressive Acquisition CMOS Image Sensor: From the Algorithm to Hardware Implementation

paper_content:

In this paper, a new design paradigm referred to as compressive acquisition CMOS image sensors is introduced. The idea consists of compressing the data within each pixel prior to storage, and hence, reducing the size of the memory required for digital pixel sensor. The proposed compression algorithm uses a block-based differential coding scheme in which differential values are captured and quantized online. A time-domain encoding scheme is used in our CMOS image sensor in which the brightest pixel within each block fires first and is selected as the reference pixel. The differential values between subsequent pixels and the reference within each block are calculated and quantized, using a reduced number of bits as their dynamic range is compressed. The proposed scheme enables reduced error accumulation as full precision is used at the start of each block, while also enabling reduced memory requirement, and hence, enabling significant silicon area saving. A mathematical model is derived to analyze the performance of the algorithm. Experimental results on a field-programmable gate-array (FPGA) platform illustrate that the proposed algorithm enables more than 50% memory saving at a peak signal-to-noise ratio level of 30 dB with 1.5 bit per pixel.
---
paper_title: VLSI Architectures for JPEG 2000 EBCOT

paper_content:

EBCOT tier-1, as entropy encoder of JPEG 2000, is a huge time-consuming part (typically more than 50%) and is considered a bottleneck for the entire system. In this paper, EBCOT tier-1 algorithm, including serial coding mode and parallel coding mode, is introduced. The VLSI architectures for EBCOT tier-1 can be divided into two categories: parallel bit-plane coding scheme (ParaBCS), where all bit-planes in a code block are coded in parallel, and serial bit-plane coding scheme (SeriBCS), where all bit-planes in a code-block are coded in serial. These two schemes are compared in terms of system throughput, PSNR performance, power consumption, and memory size. Finally, two case studies (one is based on SeriBCS and the other is based on ParaBCS) are presented.
---
paper_title: MPEG (moving pictures expert group) video compression algorithm: a review

paper_content:

A video coding technique aimed at compression of video and its associated audio at about 1.5 Mbits/s has been developed by the Moving Pictures Expert Group (MPEG-Video). The Moving Pictures Expert Group is part of the ISO-IEC JTC1/SC2/WG11, an organization responsible for the standardization of coded representation of video and audio for information systems. The video compression technique developed by MPEG covers many applications from interactive systems on CD-ROM to delivery of video information over telecommunications networks. The MPEG video compression algorithm relies on two basic techniques: block based motion compensation for the reduction of the temporal redundancy and transform domain based compression for the reduction of spatial redundancy. Motion compensation techniques are applied with both predictive and interpolative techniques. The prediction error signal is further compressed with spatial redundancy reduction (DCT). The quality of the compressed video with the MPEG algorithm at about 1.5 Mbits/s has been compared to that of consumer grade VCR's. The quality, cost and features of the MPEG video algorithm make it directly applicable to personal computers and workstation thus allowing the development of many new applications integrating video, sound, images, text and graphics.
---
paper_title: A CMOS image sensor with analog two-dimensional DCT-based compression circuits for one-chip cameras

paper_content:

This paper presents a CMOS image sensor with on-chip compression using an analog two-dimensional discrete cosine transform (2-D DCT) processor and a variable quantization level analog-to-digital converter (ADC). The analog 2-D DCT processor is essentially suitable for the on-sensor image compression, since the analog image sensor signal can be directly processed. The small and low-power nature of the analog design allows us to achieve low-power, low-cost, one-chip digital video cameras. The 8/spl times/8-point analog 2-D DCT processor is designed with fully differential switched-capacitor circuits to obtain sufficient precision for video compression purposes. An imager array has a dedicated eight-channel parallel readout scheme for direct encoding with the analog 2-D DCT processor. The variable level quantization after the 2-D DCT can be performed by the ADC at the same time. A prototype CMOS image sensor integrating these core circuits for compression is implemented based on triple-metal double-polysilicon 0.35-/spl mu/m CMOS technology. Image encoding using the implemented analog 2-D DCT processor to the image captured by the sensor is successfully performed. The maximum peak signal-to-noise ratio (PSNR) is 36.7 dB.
---
paper_title: JPEG: Still Image Data Compression Standard

paper_content:

Foreword. Acknowledgments. Trademarks. Introduction. Image Concepts and Vocabulary. Aspects of the Human Visual Systems. The Discrete Cosine Transform (DCT). Image Compression Systems. JPEG Modes of Operation. JPEG Syntax and Data Organization. Entropy Coding Concepts. JPEG Binary Arithmetic Coding. JPEG Coding Models. JPEG Huffman Entropy Coding. Arithmetic Coding Statistical. More on Arithmetic Coding. Probability Estimation. Compression Performance. JPEG Enhancements. JPEG Applications and Vendors. Overview of CCITT, ISO, and IEC. History of JPEG. Other Image Compression Standards. Possible Future JPEG Directions. Appendix A. Appendix B. References. Index.
---
paper_title: On sensor image compression

paper_content:

In this paper, we propose a novel image sensor which compresses image signals on the sensor plane. Since an image signal is compressed on the sensor plane by making use of the parallel nature of image signals, the amount of signal read out from the sensor can be significantly reduced. Thus, the potential applications of the proposed sensor are high pixel rate cameras and processing systems which require very high speed imaging or very high resolution imaging. The very high bandwidth is the fundamental limitation to the feasibility of those high pixel rate sensors and processing systems. Conditional replenishment is employed for the compression algorithm. In each pixel, current pixel value is compared to that in the last replenished frame. The value and the address of the pixel are extracted and coded if the magnitude of the difference is greater than a threshold. Analog circuits have been designed for processing in each pixel. A first prototype of a VLSI chip has been fabricated. Some results of experiments obtained by using the first prototype are shown in this paper.
---
paper_title: Focal-Plane Algorithmically-Multiplying CMOS Computational Image Sensor

paper_content:

The CMOS image sensor computes two-dimensional convolution of video frames with a programmable digital kernel of up to 8 times 8 pixels in parallel directly on the focal plane. Three operations, a temporal difference, a multiplication and an accumulation are performed for each pixel readout. A dual-memory pixel stores two video frames. Selective pixel output sampling controlled by binary kernel coefficients implements binary-analog multiplication. Cross-pixel column-parallel bit-level accumulation and frame differencing are implemented by switched-capacitor integrators. Binary-weighted summation and concurrent quantization is performed by a bank of column-parallel multiplying analog-to-digital converters (MADCs). A simple digital adder performs row-wise accumulation during ADC readout. A 128 times 128 active pixel array integrated with a bank of 128 MADCs was fabricated in a 0.35 mum standard CMOS technology. The 4.4 mm times 2.9 mm prototype is experimentally validated in discrete wavelet transform (DWT) video compression and frame differencing.
---
paper_title: On the Complexity of Finite Sequences

paper_content:

A new approach to the problem of evaluating the complexity ("randomness") of finite sequences is presented. The proposed complexity measure is related to the number of steps in a self-delimiting production process by which a given sequence is presumed to be generated. It is further related to the number of distinct substrings and the rate of their occurrence along the sequence. The derived properties of the proposed measure are discussed and motivated in conjunction with other well-established complexity criteria.
---
paper_title: PRISM: A Video Coding Paradigm With Motion Estimation at the Decoder

paper_content:

We describe PRISM, a video coding paradigm based on the principles of lossy distributed compression (also called source coding with side information or Wyner-Ziv coding) from multiuser information theory. PRISM represents a major departure from conventional video coding architectures (e.g., the MPEGx, H.26x families) that are based on motion-compensated predictive coding, with the goal of addressing some of their architectural limitations. PRISM allows for two key architectural enhancements: (1) inbuilt robustness to "drift" between encoder and decoder and (2) the feasibility of a flexible distribution of computational complexity between encoder and decoder. Specifically, PRISM enables transfer of the computationally expensive video encoder motion-search module to the video decoder. Based on this capability, we consider an instance of PRISM corresponding to a near reversal in codec complexities with respect to today's codecs (leading to a novel light encoder and heavy decoder paradigm), in this paper. We present encouraging preliminary results on real-world video sequences, particularly in the realm of transmission losses, where PRISM exhibits the characteristic of rapid recovery, in contrast to contemporary codecs. This renders PRISM as an attractive candidate for wireless video applications.
---
paper_title: The rate-distortion function for source coding with side information at the decoder

paper_content:

Let \{(X_{k}, Y_{k}) \}^{ \infty}_{k=1} be a sequence of independent drawings of a pair of dependent random variables X, Y . Let us say that X takes values in the finite set \cal X . It is desired to encode the sequence \{X_{k}\} in blocks of length n into a binary stream of rate R , which can in turn be decoded as a sequence \{ \hat{X}_{k} \} , where \hat{X}_{k} \in \hat{ \cal X} , the reproduction alphabet. The average distortion level is (1/n) \sum^{n}_{k=1} E[D(X_{k},\hat{X}_{k})] , where D(x,\hat{x}) \geq 0, x \in {\cal X}, \hat{x} \in \hat{ \cal X} , is a preassigned distortion measure. The special assumption made here is that the decoder has access to the side information \{Y_{k}\} . In this paper we determine the quantity R \ast (d) , defined as the infimum ofrates R such that (with \varepsilon > 0 arbitrarily small and with suitably large n )communication is possible in the above setting at an average distortion level (as defined above) not exceeding d + \varepsilon . The main result is that R \ast (d) = \inf [I(X;Z) - I(Y;Z)] , where the infimum is with respect to all auxiliary random variables Z (which take values in a finite set \cal Z ) that satisfy: i) Y,Z conditionally independent given X ; ii) there exists a function f: {\cal Y} \times {\cal Z} \rightarrow \hat{ \cal X} , such that E[D(X,f(Y,Z))] \leq d . Let R_{X | Y}(d) be the rate-distortion function which results when the encoder as well as the decoder has access to the side information \{ Y_{k} \} . In nearly all cases it is shown that when d > 0 then R \ast(d) > R_{X|Y} (d) , so that knowledge of the side information at the encoder permits transmission of the \{X_{k}\} at a given distortion level using a smaller transmission rate. This is in contrast to the situation treated by Slepian and Wolf [5] where, for arbitrarily accurate reproduction of \{X_{k}\} , i.e., d = \varepsilon for any \varepsilon >0 , knowledge of the side information at the encoder does not allow a reduction of the transmission rate.
---
paper_title: Code and parse trees for lossless source encoding

paper_content:

This paper surveys the theoretical literature on fixed-to-variable-length lossless source code trees, called code trees, and on variable-length-to-fixed lossless source code trees, called parse trees. In particular, the following code tree topics are outlined in this survey: characteristics of the Huffman (1952) code tree; Huffman-type coding for infinite source alphabets and universal coding; the Huffman problem subject to a lexicographic constraint, or the Hu-Tucker (1982) problem; the Huffman problem subject to maximum codeword length constraints; code trees which minimize other functions besides average codeword length; coding for unequal cost code symbols, or the Karp problem, and finite state channels; and variants of Huffman coding in which the assignment of 0s and 1s within codewords is significant such as bidirectionality and synchronization. The literature on parse tree topics is less extensive. Treated here are: variants of Tunstall (1968) parsing; dualities between parsing and coding; dual tree coding in which parsing and coding are combined to yield variable-length-to-variable-length codes; and parsing and random number generation. Finally, questions related to counting and representing code and parse trees are also discussed.
---
paper_title: Subband image coding using watershed and watercourse lines of the wavelet transform.

paper_content:

Reports progress in primitive-based image coding using nonorthogonal dyadic wavelets. A 3D isotropic wavelet is used to approximate the difference-of-Gaussians (D-o-G) operator. Convolution of the image with dilated versions of the wavelet produces three band-pass signals that approximate multiscale smoothed second derivatives. An additional convolution of the image with a Gaussian-shaped low-pass wavelet creates a fourth subband signal that preserves low-frequency information not described by the three band-pass signals. The authors show that the original image can be recovered from the watershed and watercourse lines of the three band-pass signals plus the lowpass subband signal. By thresholding the watershed/watercourse representation, subsampling the low-pass subband, and using edge post emphasis, the authors achieve data reduction with little loss of fidelity. Further compression of the watersheds and watercourses is achieved by chain coding their shapes and predictive coding their amplitudes prior to lossless arithmetic coding. Results are presented for grey-level test images at data rates between 0.1 and 0.3 b/pixel.
---
paper_title: Image Compression Based On Best Wavelet Packet Bases

paper_content:

An adaptive subband decomposition technique for efficient signal representation and compression is introduced and tested in a rate-distortion framework. The best wavelet packet bases exhibit a SFFT (short-time fast Fourier transform) subband decomposition at one source instance, a wavelet decomposition at another instance, or any intermediate wavelet packet decomposition at yet other instances to best match the signal's characteristics. For a given subband decomposition, commonly used information measures such as entropy, distortion (mse), and rate (no. of coefficients) are minimized over all subbands decomposition to search the most efficient wavelet packet tree of signal and a method to provide such a wavepacket tree is proposed. Image coding application results using the joint rate-distortion cost measure demonstrated superior performance over the entropy only information cost measure. >
---
paper_title: Wyner-Ziv coding of motion video

paper_content:

In current interframe video compression systems, the encoder performs predictive coding to exploit the similarities of successive frames. The Wyner-Ziv theorem on source coding with side information available only at the decoder suggests that an asymmetric video codec, where individual frames are encoded separately, but decoded conditionally (given temporally adjacent frames) could achieve similar efficiency. We report the first results on a Wyner-Ziv coding scheme for motion video that uses intraframe encoding, but interframe decoding.
---
paper_title: A Universal Algorithm for Sequential Data Compression

paper_content:

A universal algorithm for sequential data compression is presented. Its performance is investigated with respect to a nonprobabilistic model of constrained sources. The compression ratio achieved by the proposed universal code uniformly approaches the lower bounds on the compression ratios attainable by block-to-variable codes and variable-to-block codes designed to match a completely specified source.
---
paper_title: Arithmetic coding revisited

paper_content:

During its long gestation in the 1970s and early 1980s, arithmetic coding was widely regarded more as an academic curiosity than a practical coding technique. One factor that helped it gain the popularity it enjoys today was the publication in 1987 of source code for a multi symbol arithmetic coder in Communications of the ACM. Now (1995), our understanding of arithmetic coding has further matured, and it is timely to review the components of that implementation and summarise the improvements that we and other authors have developed since then. We also describe a novel method for performing the underlying calculation needed for arithmetic coding. Accompanying the paper is a "Mark II" implementation that incorporates the improvements we suggest. The areas examined include: changes to the coding procedure that reduce the number of multiplications and divisions and permit them to be done to low precision; the increased range of probability approximations and alphabet sizes that can be supported using limited precision calculation; data structures for support of arithmetic coding on large alphabets; the interface between the modelling and coding subsystems; the use of enhanced models to allow high performance compression. For each of these areas, we consider how the new implementation differs from the CACM package.
---
paper_title: MPEG-4 video and its potential for future multimedia services

paper_content:

The Moving Picture Experts Group (MPEG) committee, which originated the MPEG-1 and MPEG-2 video and audio compression standards, is currently developing MPEG-4 with wide industry participation. MPEG-4 is targeted for interactive multimedia applications and will become an international standard in 1998. It is expected that MPEG-4 will become the enabling technology for multimedia audio-visual communications as much as MPEG-2 has become the enabling technology for digital television. The purpose of the paper is to discuss the scope and potential of the MPEG-4 standardization activities for networked interactive multimedia applications with particular emphasis on the MPEG-4 video standard.
---
paper_title: Distributed Video Coding

paper_content:

Distributed coding is a new paradigm for video compression, based on Slepian and Wolf's and Wyner and Ziv's information-theoretic results from the 1970s. This paper reviews the recent development of practical distributed video coding schemes. Wyner-Ziv coding, i.e., lossy compression with receiver side information, enables low-complexity video encoding where the bulk of the computation is shifted to the decoder. Since the interframe dependence of the video sequence is exploited only at the decoder, an intraframe encoder can be combined with an interframe decoder. The rate-distortion performance is superior to conventional intraframe coding, but there is still a gap relative to conventional motion-compensated interframe coding. Wyner-Ziv coding is naturally robust against transmission errors and can be used for joint source-channel coding. A Wyner-Ziv MPEG encoder that protects the video waveform rather than the compressed bit stream achieves graceful degradation under deteriorating channel conditions without a layered signal representation.
---
paper_title: JPEG: Still Image Data Compression Standard

paper_content:

Foreword. Acknowledgments. Trademarks. Introduction. Image Concepts and Vocabulary. Aspects of the Human Visual Systems. The Discrete Cosine Transform (DCT). Image Compression Systems. JPEG Modes of Operation. JPEG Syntax and Data Organization. Entropy Coding Concepts. JPEG Binary Arithmetic Coding. JPEG Coding Models. JPEG Huffman Entropy Coding. Arithmetic Coding Statistical. More on Arithmetic Coding. Probability Estimation. Compression Performance. JPEG Enhancements. JPEG Applications and Vendors. Overview of CCITT, ISO, and IEC. History of JPEG. Other Image Compression Standards. Possible Future JPEG Directions. Appendix A. Appendix B. References. Index.
---
paper_title: MPEG-4 very low bit rate video

paper_content:

The MPEG video group is currently developing the so-called MPEG-4 video coding standard, targeted for future interactive multimedia video communications calling for content-based functionalities, universal access in error prone environments and high coding efficiency. Besides the provisions for content-based functionalities the MPEG-4 video standard will assist the efficient storage and transmission of very low bit rate video in error prone environments. This paper outlines the techniques that are currently being investigated by MPEG-4 to provide universal accessibility of video at very low bit rates and discusses the scope of some of the promising techniques under discussion.
---
paper_title: Compression of Individual Sequences via Variable-Rate Coding

paper_content:

Compressibility of individual sequences by the class of generalized finite-state information-lossless encoders is investigated. These encoders can operate in a variable-rate mode as well as a fixed-rate one, and they allow for any finite-state scheme of variable-length-to-variable-length coding. For every individual infinite sequence x a quantity \rho(x) is defined, called the compressibility of x , which is shown to be the asymptotically attainable lower bound on the compression ratio that can be achieved for x by any finite-state encoder. This is demonstrated by means of a constructive coding theorem and its converse that, apart from their asymptotic significance, also provide useful performance criteria for finite and practical data-compression tasks. The proposed concept of compressibility is also shown to play a role analogous to that of entropy in classical information theory where one deals with probabilistic ensembles of sequences rather than with individual sequences. While the definition of \rho(x) allows a different machine for each different sequence to be compressed, the constructive coding theorem leads to a universal algorithm that is asymptotically optimal for all sequences.
---
paper_title: Charge-based prediction circuits for focal plane image compression

paper_content:

This work presents the design of a computational charge-based circuit to be part of a focal plane compression chip. The image compression scheme pursued is predictive coding. The proposed circuit computes the prediction error at every pixel. It carries out the computations by integrating the photocurrents of the pixels in a small neighborhood. The prediction weights for every pixel can be changed by changing the switching timing of the circuit making possible the use of adaptive prediction algorithms. The circuit is compact and can be integrated at the pixel level.
---
paper_title: A CMOS imager with focal plane compression

paper_content:

A focal plane video compression integrated circuit is presented. The design consists of a 128 /spl times/ 128 pixel array and a bank of column-level processors. Each one of the column-level processors performs the tasks of image decorrelation, quantization, and entropy encoding. The chip provides at its output a compressed bit stream. The integration of the quantizer and the entropy encoder at the column level is possible by sharing circuitry between a single-slope analog-to-digital converter and a Golomb-Rice entropy encoder. In addition, the design includes a low-complexity algorithm for the adaptation of the Golomb-Rice coder to the statistics of the video signal. The design has been fully verified through simulations and has been implemented in a 0.35 /spl mu/m CMOS technology. The chip layout occupies an area of 7 /spl times/ 5 mm/sup 2/.
---
paper_title: A CMOS Imager With Focal Plane Compression Using Predictive Coding

paper_content:

This paper presents a CMOS image sensor with focal-plane compression. The design has a column-level architecture and it is based on predictive coding techniques for image decorrelation. The prediction operations are performed in the analog domain to avoid quantization noise and to decrease the area complexity of the circuit. The prediction residuals are quantized and encoded by a joint quantizer/coder circuit. To save area resources, the joint quantizer/coder circuit exploits common circuitry between a single-slope analog-to-digital converter (ADC) and a Golomb-Rice entropy coder. This combination of ADC and encoder allows the integration of the entropy coder at the column level. A prototype chip was fabricated in a 0.35 mum CMOS process. The output of the chip is a compressed bit stream. The test chip occupies a silicon area of 2.60 mm times 5.96 mm which includes an 80 times 44 APS array. Tests of the fabricated chip demonstrate the validity of the design.
---
paper_title: CMOS wavelet compression imager architecture

paper_content:

The CMOS imager architecture implements /spl Delta//spl Sigma/-modulated Haar wavelet image compression on the focal plane in real time. The active pixel array is integrated with a bank of column-parallel first-order incremental over-sampling analog-to-digital converters (ADCs). Each ADC performs column-wise distributed focal-plane sampling and concurrent signed weighted average quantization, realizing a one-dimensional spatial Haar wavelet transform. A digital delay and adder loop performs spatial accumulation over multiple adjacent ADC outputs. This amounts to computing a two-dimensional Haar wavelet transform, with no overhead in time and negligent overhead in area compared to a baseline digital imager architecture. The architecture is experimentally validated on a 0.35 micron CMOS prototype containing a bank of first-order incremental oversampling ADCs computing Haar wavelet transform on an emulated pixel array output. The architecture yields simulated computational throughput of 1.4 GMACS with SVGA imager resolution at 30 frames per second.
---
paper_title: Predictive coding on-sensor compression

paper_content:

This paper presents the design and measurements of a predictive coding on-sensor compression CMOS imager. Predictive coding is employed to decorrelate the image. The prediction operations are performed in the analog domain to avoid quantization noise and to decrease the area complexity of the circuit. The decorrelated image is encoded with a bank of column-parallel entropy encoders. Each encoder is combined with a single-slope analog-to-digital converter (ADC) to reduce area complexity and power consumption. The area savings resulting from such combination allow to integrate an ADC and an entropy encoder at the column level. A prototype chip was fabricated in a 0.35 mum CMOS process. The output of the chip is a compressed bit stream. The test chip occupies a silicon area of 2.60 mm times 5.96 mm which includes an 80 times 44 APS array. Tests of the fabricated chip demonstrate the validity of the design.
---
paper_title: A CMOS imager with pixel prediction for image compression

paper_content:

A predictive approach to on-chip focal plane compression is presented. This approach, unlike the transform coding techniques, is based on pixel prediction. In this technique, the value of each pixel is predicted by considering the values of its closest neighbor pixels. An analog circuit has been designed to accomplish a simple prediction function based on the values of two neighbor pixels. To demonstrate the approach, a prototype chip containing an imager and the prediction circuits has been designed, implemented, and tested.
---
paper_title: Focal-Plane Spatially Oversampling CMOS Image Compression Sensor

paper_content:

Image compression algorithms employ computationally expensive spatial convolutional transforms. The CMOS image sensor performs spatially compressing image quantization on the focal plane yielding digital output at a rate proportional to the mere information rate of the video. A bank of column-parallel first-order incremental DeltaSigma-modulated analog-to-digital converters (ADCs) performs column-wise distributed focal-plane oversampling of up to eight adjacent pixels and concurrent weighted average quantization. Number of samples per pixel and switched-capacitor sampling sequence order set the amplitude and sign of the pixel coefficient, respectively. A simple digital delay and adder loop performs spatial accumulation over up to eight adjacent ADC outputs during readout. This amounts to computing a two-dimensional block matrix transform with up to 8times8-pixel programmable kernel in parallel for all columns. Noise shaping reduces power dissipation below that of a conventional digital imager while the need for a peripheral DSP is eliminated. A 128times128 active pixel array integrated with a bank of 128 DeltaSigma-modulated ADCs was fabricated in a 0.35-mum CMOS technology. The 3.1 mm times 1.9-mm prototype captures 8-bit digital video at 30 frames/s and yields 4 GMACS projected computational throughput when scaled to HDTV 1080i resolution in discrete cosine transform (DCT) compression
---
paper_title: A novel integration of on-sensor wavelet compression for a CMOS imager

paper_content:

A novel integration of image compression and sensing is proposed to enhance the performance of a CMOS image sensor. By integrating a compression function onto the sensor focal plane, the image signal to be read out can be significantly reduced and consequently the pixel rate can be increased. This can be applied to overcome the communication bottleneck for high-resolution image sensing and high frame-rate image sensing or for power- and bandwidth-constrained devices such as cell phones. A modified Haar wavelet transform is implemented as the compression scheme. A simple but efficient computation design is developed to implement the transform on-chip.
---
paper_title: A High-Speed CMOS Image Sensor with On-chip Parallel Image Compression Circuits

paper_content:

This paper presents a high-speed CMOS image sensor with on-chip parallel image compression circuits. The chip consists of a pixel array, an A/D converter array with noise canceling function and an image compression processing element array and buffer memories. The image compression processing element is implemented with a 4times4 point discreate cosine transform(DCT) and a modified zigzag scanner with 4 blocks. A prototype high-speed CMOS image sensor integrating the image compression circuits is implemented based on 1-poly 5-metal 0.25-mum CMOS technology. Image encoding using the implemented parallel image compression circuits to the image captured by the high-speed image sensor is successfully performed at 3,000[frame/s].
---
paper_title: A compact-pixel tri-mode vision sensor

paper_content:

We present a custom image sensor capable of reporting intensity, spatial contrast, and temporal difference images at the pixel level. The smart pixel is composed of only 11 transistors, allowing tight integration of different functionalities in a 16×21 µm2 pixel area. The sensor array is 128×128 pixels, with a fill factor of 42%, and operates at 800 fps and 13M events/s with a power consumption of 1.2mW.
---
paper_title: A compressed digital output CMOS image sensor with analog 2-D DCT processors and ADC/quantizer

paper_content:

Progress in CMOS-based image sensors is creating opportunities for a low-cost, low-power one-chip video camera with digitizing, signal processing and image compression. Such a smart camera head acquires compressed digital moving pictures directly into portable multimedia computers. Video encoders using a moving picture coding standard such as MPEG and H.26x are not always suitable for integration of image encoding on the image sensor, because of the complexity and the power dissipation. On-sensor image compression such as a CCD image sensor for lossless image compression and a CMOS image sensor with pixel-level interframe coding are reported. A one-chip digital camera with on-sensor video compression is shown in the block diagram. The chip contains a 128/spl times/128-pixel sensor, 8-channel parallel read-out circuits, an analog 2-dimensional discrete cosine transform (2D DCT) processor and a variable quantization-level ADC (ADC/Q).
---
paper_title: A CMOS smart image sensor LSI for focal-plane compression

paper_content:

A CMOS smart image sensor LSI with video compression is presented. The proposed on-sensor compression scheme using an analog 2-D DCT processor is particularly useful to achieve low-power one-chip digital camera. A prototype image sensor LSI has been developed using 0.35 /spl mu/m double polysilicon, triple metal CMOS technology. Image coding using the implemented LSI has been demonstrated.
---
paper_title: CMOS image compression sensor with algorithmically-multiplying ADCs

paper_content:

A 128×128 CMOS image compression sensor fabricated in a 0.35µm CMOS process is reported. It computes block-matrix and convolutional image transforms with digital kernels of up to 8×8 pixels directly on the focal plane. A pixel output is sampled only when the corresponding bit of the kernel coefficient is one. Bit-wise accumulation of adjacent pixel outputs in a column is performed by the switched-capacitor accumulator circuit. A column-parallel algorithmic multiplying ADC performs binary-weighted summation by adding the accumulator circuit outputs with cyclic residues of the same binary weight. The signal range is maintained by generating two bits per cycle. The imager performs three computations per pixel readout. Image compression experimental results at 30fps and 8-bit output resolution are presented.
---
paper_title: CMOS Image Sensors with Video Compression

paper_content:

This paper describes CMOS image sensors integrating video compression circuits. The on-sensor compression is particularly useful for the low-power design of moving picture compression hardware, which is demanded especially in the mobile computing and telephony. Recent progress of the CMOS image sensor technology allows us to realise the integration of high-performance image sensor and computational functions on a chip. An example of the CMOS image sensor with compression is described. Prospects of the CMOS image sensors with moving picture compression are discussed.
---
paper_title: A CMOS image sensor with analog two-dimensional DCT-based compression circuits for one-chip cameras

paper_content:

This paper presents a CMOS image sensor with on-chip compression using an analog two-dimensional discrete cosine transform (2-D DCT) processor and a variable quantization level analog-to-digital converter (ADC). The analog 2-D DCT processor is essentially suitable for the on-sensor image compression, since the analog image sensor signal can be directly processed. The small and low-power nature of the analog design allows us to achieve low-power, low-cost, one-chip digital video cameras. The 8/spl times/8-point analog 2-D DCT processor is designed with fully differential switched-capacitor circuits to obtain sufficient precision for video compression purposes. An imager array has a dedicated eight-channel parallel readout scheme for direct encoding with the analog 2-D DCT processor. The variable level quantization after the 2-D DCT can be performed by the ADC at the same time. A prototype CMOS image sensor integrating these core circuits for compression is implemented based on triple-metal double-polysilicon 0.35-/spl mu/m CMOS technology. Image encoding using the implemented analog 2-D DCT processor to the image captured by the sensor is successfully performed. The maximum peak signal-to-noise ratio (PSNR) is 36.7 dB.
---
paper_title: Focal-Plane Algorithmically-Multiplying CMOS Computational Image Sensor

paper_content:

The CMOS image sensor computes two-dimensional convolution of video frames with a programmable digital kernel of up to 8 times 8 pixels in parallel directly on the focal plane. Three operations, a temporal difference, a multiplication and an accumulation are performed for each pixel readout. A dual-memory pixel stores two video frames. Selective pixel output sampling controlled by binary kernel coefficients implements binary-analog multiplication. Cross-pixel column-parallel bit-level accumulation and frame differencing are implemented by switched-capacitor integrators. Binary-weighted summation and concurrent quantization is performed by a bank of column-parallel multiplying analog-to-digital converters (MADCs). A simple digital adder performs row-wise accumulation during ADC readout. A 128 times 128 active pixel array integrated with a bank of 128 MADCs was fabricated in a 0.35 mum standard CMOS technology. The 4.4 mm times 2.9 mm prototype is experimentally validated in discrete wavelet transform (DWT) video compression and frame differencing.
---
paper_title: A compressed digital output CMOS image sensor with analog 2-D DCT processors and ADC/quantizer

paper_content:

Progress in CMOS-based image sensors is creating opportunities for a low-cost, low-power one-chip video camera with digitizing, signal processing and image compression. Such a smart camera head acquires compressed digital moving pictures directly into portable multimedia computers. Video encoders using a moving picture coding standard such as MPEG and H.26x are not always suitable for integration of image encoding on the image sensor, because of the complexity and the power dissipation. On-sensor image compression such as a CCD image sensor for lossless image compression and a CMOS image sensor with pixel-level interframe coding are reported. A one-chip digital camera with on-sensor video compression is shown in the block diagram. The chip contains a 128/spl times/128-pixel sensor, 8-channel parallel read-out circuits, an analog 2-dimensional discrete cosine transform (2D DCT) processor and a variable quantization-level ADC (ADC/Q).
---
paper_title: Transpose memory for video rate JPEG compression on highly parallel single-chip digital CMOS imager

paper_content:

A transpose switch matrix memory (TSMM) is proposed to enable a highly parallel single-chip CMOS sensor/image processor, Xetal, developed at Philips to perform JPEG compression at video rate (30 frames per second, fps) at an image dimension of 640/spl times/480 pixels. The integrated solution consists of 320 processing elements and 80 TSMMs, operates at 16 MHz clock rate and 3.3 V supply voltage, and is designed for fabrication at 0.25 micron technology. The processing system can sustain a maximum throughput of 5.12 billion operations per second consuming an estimated 120 mW providing a processing power efficiency of 7 BOPS/Watt. The Xetal architecture is capable of performing pixel level image processing such as fixed pattern noise (FPN) correction, defective pixel concealment, Bayer pattern filtering, RGB-YUV conversion, auto white balancing, and auto exposure control. The TSMM expands support to block level operations including chrominance subsampling, separable 8/spl times/8 recursive DCT, and ZZ scan required for JPEG.
---
paper_title: A real-time JPEG encoder for 1.3 mega pixel CMOS image sensor SoC

paper_content:

In this paper, we propose a hardware architecture of low-power and real-time JPEG encoder for 1.3 mega pixels CMOS image sensor SoC which can be applied to mobile communication devices. The proposed architecture has an efficient interface scheme with CMOS image sensor and other peripherals for real-time encoding. The JPEG encoder supports the base-line JPEG mode, and processes motion images of which resolution is up to 1280/spl times/960 (CCIR601 YCrCb 4:2:2,15 fps) by real-time processing. The JPEG encoder supports 8 types of resolution, and can serve the 4 levels of image quality through quantization matrix. The proposed JPEG encoder can transfer encoded motion pictures and raw image data from CMOS image sensor to external device through USB 2.0 and a compressed still image is stored at external pseudo SRAM through SRAM interface. And proposed core can communicate parameters of encoding type with other host by I2C. The proposed architecture was implemented with VHDL and verified for the functions with Synopsys and Modelsim. The encoder proposed in this paper was fabricated in process of 0.18u of Hynix semiconductor Inc.
---
paper_title: A CMOS Front-End for a Lossy Image Compression Sensor

paper_content:

A CMOS image sensor has been designed to perform the front-end image decomposition in a prediction-SPIHT image compression scheme. The prediction circuitry based on charge sharing is integrated inside the sensor array to perform 3-level image decomposition. A CMOS test chip has been prototyped and tested. The test results justify the pixel design and demonstrate that lossy prediction based focal plane image compression can be realized inside the sensor pixel array to achieve a high frame rate but with much less data readout volume. Also, the sensor can be used to achieve comparable compression performance with much lower computational complexity compared to 2D discrete wavelet transform (DWT) based image compression.
---
paper_title: A CMOS image sensor for focal plane decomposition

paper_content:

An alternative image decomposition method that exploits prediction via nearby pixels has been integrated on a CMOS sensor focal plane. The proposed focal plane decomposition is compared to the 2D discrete wavelet transform (DWT) decomposition commonly used in state of the art compression schemes such as EBCOT and SPIHT. A technique for focal plane prediction is described that can be used to decompose the image into pyramid subbands as is done in DWT based approaches. Although its simulated compression performance is somewhat lower than that obtained using wavelet decomposition, prediction decomposition has much lower computational complexity and is suitable for implementation directly on the focal plane in a completely pixel parallel structure. When coupled with SPIHT-like encoding, the focal plane prediction technique obtains all the advantages of the DWT based SPIHT compression algorithm, including both high image quality at low bit rates and optimized data ordering for progressive image transmission. A CMOS test chip to perform single layer image decomposition based on pixel level parallel computation has been prototyped. The design and test results for this chip are discussed.
---
paper_title: A CMOS image sensor with focal plane SPIHT image compression

paper_content:

In this paper, a focal plane SPIHT image compression scheme has been integrated on the focal plane of CMOS image sensor. A test chip is prototyped in 0.35 mum technology to verify the design. In our image compression scheme, focal plane prediction replaces the traditional wavelet transform for image decomposition, and the traditional SPIHT coding is modified to reduce computational complexity and to suit the parallel implementation on the sensor focal plane. Simulated compression performance is still good, especially at low bit rates, compared to traditional SPIHT and other compression schemes. Integrating SPIHT compression on the focal plane not only reduces the sensor readout volume, but also provides all the advantages of SPIHT compression including both high image quality and optimized data ordering for progressive image transmission.
---
paper_title: Charge-based prediction circuits for focal plane image compression

paper_content:

This work presents the design of a computational charge-based circuit to be part of a focal plane compression chip. The image compression scheme pursued is predictive coding. The proposed circuit computes the prediction error at every pixel. It carries out the computations by integrating the photocurrents of the pixels in a small neighborhood. The prediction weights for every pixel can be changed by changing the switching timing of the circuit making possible the use of adaptive prediction algorithms. The circuit is compact and can be integrated at the pixel level.
---
paper_title: CMOS Camera With In-Pixel Temporal Change Detection and ADC

paper_content:

An array of 90times90 active pixel sensors (APS) with pixel-level embedded differencing and comparison is presented. The nMOS-only 6T 2C 25 mum times 25 mum pixel provides both analog readout of pixel intensity and a digital flag indicating temporal change at variable thresholds. Computation is performed through a pixel-level capacitively coupled comparator which also functions as analog-to-digital converter. The chip, fabricated in a 0.5 mum 3M2P CMOS, process consumes 4.2 mW of power while operating at 30 fps. Change sensitivity is 2.1% at an illumination of 1.7 W/cm2. Gating of raster-scanned pixel output by change detection typically produces a 20-fold compression in the data stream, depending on image conditions and reconstruction quality set by the change detection threshold.
---
paper_title: A CMOS Image Sensor for Multi-Level Focal Plane Image Decomposition

paper_content:

An alternative image decomposition method that exploits prediction via nearby pixels has been integrated on the CMOS image sensor focal plane. The proposed focal plane decomposition is compared to the 2-D discrete wavelet transform (DWT) decomposition commonly used in state of the art compression schemes such as SPIHT and JPEG2000. The method achieves comparable compression performance with much lower computational complexity and allows image compression to be implemented directly on the sensor focal plane in a completely pixel parallel structure. A CMOS prototype chip has been fabricated and tested. The test results validate the pixel design and demonstrate that lossy prediction based focal plane image compression can be realized inside the sensor pixel array to achieve a high frame rate with much lower data readout volume. The features of the proposed decomposition scheme also benefit real-time, low rate and low power applications.
---
paper_title: Computational image sensor for on sensor compression

paper_content:

We propose a novel integration of image compression and sensing in order to enhance the performance of an image sensor. By integrating a compression function onto the sensor focal plane, the image signal to be read out from the sensor is significantly reduced and the pixel rate of the sensor ran consequently be increased. The potential applications of the proposed sensor are in high pixel-rate imaging, such as high frame-rate image sensing and high-resolution image sensing. The compression scheme we employ is a conditional replenishment, which detects and encodes moving areas. In this paper, we introduce two architectures for on-sensor compression; one is the pixel parallel approach and the other is the column parallel approach. We prototyped a VLSI chip of the proposed sensor based on the pixel parallel architecture. We show the design and describe the results of the experiments obtained by the prototype chip.
---
paper_title: A 128 X 128 120db 30mw asynchronous vision sensor that responds to relative intensity change

paper_content:

A vision sensor responds to temporal contrast with asynchronous output. Each pixel independently and continuously quantizes changes in log intensity. The 128times128-pixel chip has 120dB illumination operating range and consumes 30mW. Pixels respond in <100mus at 1klux scene illumination with <10% contrast-threshold FPN
---
paper_title: A QVGA 143dB dynamic range asynchronous address-event PWM dynamic image sensor with lossless pixel-level video compression

paper_content:

Conventional image/video sensors acquire visual information from a scene in time-quantized fashion at some predetermined frame rate. Each frame carries the information from all pixels, regardless of whether or not this information has changed since the last frame had been acquired, which is usually not long ago. This method obviously leads, depending on the dynamic contents of the scene, to a more or less high degree of redundancy in the image data. Acquisition and handling of these dispensable data consume valuable resources; sophisticated and resource-hungry video compression methods have been developed to deal with these data.
---
paper_title: On sensor image compression

paper_content:

In this paper, we propose a novel image sensor which compresses image signals on the sensor plane. Since an image signal is compressed on the sensor plane by making use of the parallel nature of image signals, the amount of signal read out from the sensor can be significantly reduced. Thus, the potential applications of the proposed sensor are high pixel rate cameras and processing systems which require very high speed imaging or very high resolution imaging. The very high bandwidth is the fundamental limitation to the feasibility of those high pixel rate sensors and processing systems. Conditional replenishment is employed for the compression algorithm. In each pixel, current pixel value is compared to that in the last replenished frame. The value and the address of the pixel are extracted and coded if the magnitude of the difference is greater than a threshold. Analog circuits have been designed for processing in each pixel. A first prototype of a VLSI chip has been fabricated. Some results of experiments obtained by using the first prototype are shown in this paper.
---
paper_title: High speed digital smart image sensor with image compression function

paper_content:

We propose a high speed digital smart image sensor on which A/D conversion circuits and digital processing circuits of image compression are integrated. By on-sensor image compression, the proposed sensor can overcome a communication bottle neck between a sensor and peripherals that is one of the most serious problems for achieving high frame rates. The compression method uses the correlation between two consecutive frames. It has some compression modes by controlling spatial and temporal resolutions. In this paper, the algorithm of on-sensor-compression is explained and some experimental results are shown. finally, we show the design of new digital smart sensor.
---
paper_title: Effects of charge-based computation non-idealities on CMOS image compression sensors

paper_content:

We present a CMOS image sensor that performs focal plane image decomposition based on charge sharing computation circuitry. The effect of parasitic capacitance between capacitor bottom plate and substrate on the computational accuracy is discussed and a new circuit is also proposed to characterize the parasitic effects. The test results demonstrate that prediction based focal plane image compression can be realized inside the sensor array resulting in high compression performance using frame rate pixel parallel computation. This architecture can subsequently be combined with backend level testing encoding to form a complete compression sensor.
---
paper_title: A CMOS Image Sensor With On-Chip Image Compression Based on Predictive Boundary Adaptation and Memoryless QTD Algorithm

paper_content:

This paper presents the architecture, algorithm, and VLSI hardware of image acquisition, storage, and compression on a single-chip CMOS image sensor. The image array is based on time domain digital pixel sensor technology equipped with nondestructive storage capability using 8-bit Static-RAM device embedded at the pixel level. The pixel-level memory is used to store the uncompressed illumination data during the integration mode as well as the compressed illumination data obtained after the compression stage. An adaptive quantization scheme based on fast boundary adaptation rule (FBAR) and differential pulse code modulation (DPCM) procedure followed by an online, least storage quadrant tree decomposition (QTD) processing is proposed enabling a robust and compact image compression processor. A prototype chip including 64×64 pixels, read-out and control circuitry as well as an on-chip compression processor was implemented in 0.35 μm CMOS technology with a silicon area of 3.2×3.0 mm2 and an overall power of 17 mW. Simulation and measurements results show compression figures corresponding to 0.6-1 bit-per-pixel (BPP), while maintaining reasonable peak signal-to-noise ratio levels.
---
paper_title: A digital pixel sensor array with programmable dynamic range

paper_content:

This paper presents a digital pixel sensor (DPS) array employing a time domain analogue-to-digital conversion (ADC) technique featuring adaptive dynamic range and programmable pixel response. The digital pixel comprises a photodiode, a voltage comparator, and an 8-bit static memory. The conversion characteristics of the ADC are determined by an array-based digital control circuit, which linearizes the pixel response, and sets the conversion range. The ADC response is adapted to different lighting conditions by setting a single clock frequency. Dynamic range compression was also experimentally demonstrated. This clearly shows the potential of the proposed technique in overcoming the limited dynamic range typically imposed by the number of bits in a DPS. A 64 /spl times/ 64 pixel array prototype was manufactured in a 0.35-/spl mu/m, five-metal, single poly, CMOS process. Measurement results indicate a 100 dB dynamic range, a 41-s mean dark time and an average current of 1.6 /spl mu/A per DPS.
---
paper_title: PWM digital pixel sensor based on asynchronous self-resetting scheme

paper_content:

In this letter, a pulse-width modulated digital pixel sensor is presented along with its inherent advantages such as low power consumption and wide operating range. The pixel, which comprises an analog processor and an 8-bit memory cell, operates in an asynchronous self-resetting mode. In contrast to most CMOS image sensors, in our approach, the photocurrent signal is encoded as a pulse-width signal, and converted to an 8-bit digital code using a Gray counter. The dynamic range of the pixel can be adapted by simply modulating the clock frequency of the counter. To test the operation of the proposed pixel architecture, an image sensor array has been designed and fabricated in a 0.35-/spl mu/m CMOS technology, where each pixel occupies an area of 45/spl times/45 /spl mu/m/sup 2/. Here, the operation of the sensor is demonstrated through experimental results.
---
paper_title: A CMOS Image Sensor with on Chip Image Compression based on Predictive Boundary Adaptation and QTD Algorithm

paper_content:

This paper presents the architecture, algorithm and VLSI hardware of image acquisition, storage and compression on a single-chip CMOS image sensor. The image array is based on time domain digital pixel sensor technology equipped with nondestructive storage capability using 8-bit Static-RAM device embedded at the pixel level. An adaptive quantization scheme based on Fast Boundary Adaptation Rule (FBAR) and Differential Pulse Code Modulation (DPCM) procedure followed by an online Quadrant Tree Decomposition (QTD) processing is proposed enabling low power, robust and compact image compression processor. A prototype chip including 64 times 64 pixels, read-out and control circuitry as well as the compression processor was implemented in 0.35mum CMOS technology with a silicon area of 3.2 times3.0 mm2. Simulation results show compression figures corresponding to 0.75 Bit-per-Pixel (BPP), while maintaining reasonable PSNR levels.
---
paper_title: A novel DPS integrator for fast CMOS imagers

paper_content:

A novel DPS integrator scheme for fast CMOS imagers is presented, which combines the advantages of analog APS and DPS circuits. The reset-insensitive integrator proposal improves the linearity of the ADC curve, while it allows both low-power consumption for the active blocks and low-voltage operation for the switching devices even at high frame rates. In this sense, a comparative study is presented in 0.18 mum 1-poly 6-metal 1.8 V CMOS technology to demonstrate the advantages of this novel solution.
---
paper_title: Compressive Acquisition CMOS Image Sensor: From the Algorithm to Hardware Implementation

paper_content:

In this paper, a new design paradigm referred to as compressive acquisition CMOS image sensors is introduced. The idea consists of compressing the data within each pixel prior to storage, and hence, reducing the size of the memory required for digital pixel sensor. The proposed compression algorithm uses a block-based differential coding scheme in which differential values are captured and quantized online. A time-domain encoding scheme is used in our CMOS image sensor in which the brightest pixel within each block fires first and is selected as the reference pixel. The differential values between subsequent pixels and the reference within each block are calculated and quantized, using a reduced number of bits as their dynamic range is compressed. The proposed scheme enables reduced error accumulation as full precision is used at the start of each block, while also enabling reduced memory requirement, and hence, enabling significant silicon area saving. A mathematical model is derived to analyze the performance of the algorithm. Experimental results on a field-programmable gate-array (FPGA) platform illustrate that the proposed algorithm enables more than 50% memory saving at a peak signal-to-noise ratio level of 30 dB with 1.5 bit per pixel.
---
paper_title: Architecture of a digital pixel sensor array using 1-bit Hilbert predictive coding

paper_content:

In this paper, the architecture of a digital pixel sensor (DPS) array with an online 1-bit predictive coding algorithm using Hilbert scanning scheme is proposed. The architecture of the sensor array reduces by more than half the silicon area of the DPS by sampling and storing the differential values between the pixel and its prediction, featuring compressed dynamic range and hence requiring limited precision (only 1-bit signed value in the proposed architecture as compared to 8-bit unsigned full precision). Hilbert scanning is used to read-out the pixel's value, hence avoiding discontinuity in the read-out path, which is shown to improve the quality of the reconstructed image. The Hilbert scanning path is all carried out by hardware wire connection without increasing the circuit complexity of the sensor array. Reset pixels are inserted into scanning path to overcome the error accumulation problem inherent in predictive coding. System level simulation results show a PSNR of around 25dB can be reached while using the proposed 1-bit Hilbert predictive coding algorithm. VLSI implementation results illustrate a pixel level implementation featuring a pixel size reduction of 67% with a fill-factor of 40% compared with a standard PWM DPS architecture.
---
paper_title: Digital pixel sensor with on-line spatial and temporal compression scheme

paper_content:

In this paper, the concept of compressive acquisitive image sensor is extended to the temporal domain. Architecture of a compact CMOS image sensor designed based on the design paradigm capture ↦ compress ↦ store is proposed. The image sensor array is divided into quadrants. Spatial compression algorithm is integrated into quadrant-level circuit and carried out on-line before the storage phase. All the quadrants are further classified into background/non-background quadrants by an off-array adaptive judge logic. A reference array is built in the off-array logic to adaptively tracking the movement between frames. Temporal redundance between frames are removed in the readout phase. Experimental results show that the proposed algorithm enables more than 50% memory saving at a PSNR level of 27dB with around 0.5 BPP.
---
paper_title: Compressive acquisition CMOS image sensor using on-line sorting scheme

paper_content:

In this paper, we propose a compact block-based compressive acquisition CMOS image sensors. The overall image is decomposed into blocks. The proposed design reorders the raw captured pixel values within each block, only recording the brightest pixel value and the darkest pixel value for each block using full precision (typically 8-bit). During reconstruction, block models are built to model the value distribution within one block. As in a digital pixel sensor (DPS) array the pixel data are naturally ordered (from the brightest to the darkest pixel), the block-level hardware overload while implementing the proposed scheme is very low. The proposed scheme enables reduced memory requirement, and hence enabling significant silicon area saving. The compression performance and image quality of the proposed algorithm are thoroughly analyzed for different block sizes and a mathematical model is subsequently derived. Simulation result shows that the proposed algorithm enables more than 80% pixel-level memory reduction at a PSNR level around 23dB and using 1BPP.
---
paper_title: A CMOS Image Sensor With On-Chip Image Compression Based on Predictive Boundary Adaptation and Memoryless QTD Algorithm

paper_content:

This paper presents the architecture, algorithm, and VLSI hardware of image acquisition, storage, and compression on a single-chip CMOS image sensor. The image array is based on time domain digital pixel sensor technology equipped with nondestructive storage capability using 8-bit Static-RAM device embedded at the pixel level. The pixel-level memory is used to store the uncompressed illumination data during the integration mode as well as the compressed illumination data obtained after the compression stage. An adaptive quantization scheme based on fast boundary adaptation rule (FBAR) and differential pulse code modulation (DPCM) procedure followed by an online, least storage quadrant tree decomposition (QTD) processing is proposed enabling a robust and compact image compression processor. A prototype chip including 64×64 pixels, read-out and control circuitry as well as an on-chip compression processor was implemented in 0.35 μm CMOS technology with a silicon area of 3.2×3.0 mm2 and an overall power of 17 mW. Simulation and measurements results show compression figures corresponding to 0.6-1 bit-per-pixel (BPP), while maintaining reasonable peak signal-to-noise ratio levels.
---
paper_title: A CMOS Image Sensor for Multi-Level Focal Plane Image Decomposition

paper_content:

An alternative image decomposition method that exploits prediction via nearby pixels has been integrated on the CMOS image sensor focal plane. The proposed focal plane decomposition is compared to the 2-D discrete wavelet transform (DWT) decomposition commonly used in state of the art compression schemes such as SPIHT and JPEG2000. The method achieves comparable compression performance with much lower computational complexity and allows image compression to be implemented directly on the sensor focal plane in a completely pixel parallel structure. A CMOS prototype chip has been fabricated and tested. The test results validate the pixel design and demonstrate that lossy prediction based focal plane image compression can be realized inside the sensor pixel array to achieve a high frame rate with much lower data readout volume. The features of the proposed decomposition scheme also benefit real-time, low rate and low power applications.
---
paper_title: Computational image sensor for on sensor compression

paper_content:

We propose a novel integration of image compression and sensing in order to enhance the performance of an image sensor. By integrating a compression function onto the sensor focal plane, the image signal to be read out from the sensor is significantly reduced and the pixel rate of the sensor ran consequently be increased. The potential applications of the proposed sensor are in high pixel-rate imaging, such as high frame-rate image sensing and high-resolution image sensing. The compression scheme we employ is a conditional replenishment, which detects and encodes moving areas. In this paper, we introduce two architectures for on-sensor compression; one is the pixel parallel approach and the other is the column parallel approach. We prototyped a VLSI chip of the proposed sensor based on the pixel parallel architecture. We show the design and describe the results of the experiments obtained by the prototype chip.
---
paper_title: A 128 X 128 120db 30mw asynchronous vision sensor that responds to relative intensity change

paper_content:

A vision sensor responds to temporal contrast with asynchronous output. Each pixel independently and continuously quantizes changes in log intensity. The 128times128-pixel chip has 120dB illumination operating range and consumes 30mW. Pixels respond in <100mus at 1klux scene illumination with <10% contrast-threshold FPN
---
paper_title: CMOS wavelet compression imager architecture

paper_content:

The CMOS imager architecture implements /spl Delta//spl Sigma/-modulated Haar wavelet image compression on the focal plane in real time. The active pixel array is integrated with a bank of column-parallel first-order incremental over-sampling analog-to-digital converters (ADCs). Each ADC performs column-wise distributed focal-plane sampling and concurrent signed weighted average quantization, realizing a one-dimensional spatial Haar wavelet transform. A digital delay and adder loop performs spatial accumulation over multiple adjacent ADC outputs. This amounts to computing a two-dimensional Haar wavelet transform, with no overhead in time and negligent overhead in area compared to a baseline digital imager architecture. The architecture is experimentally validated on a 0.35 micron CMOS prototype containing a bank of first-order incremental oversampling ADCs computing Haar wavelet transform on an emulated pixel array output. The architecture yields simulated computational throughput of 1.4 GMACS with SVGA imager resolution at 30 frames per second.
---
paper_title: Compressive Acquisition CMOS Image Sensor: From the Algorithm to Hardware Implementation

paper_content:

In this paper, a new design paradigm referred to as compressive acquisition CMOS image sensors is introduced. The idea consists of compressing the data within each pixel prior to storage, and hence, reducing the size of the memory required for digital pixel sensor. The proposed compression algorithm uses a block-based differential coding scheme in which differential values are captured and quantized online. A time-domain encoding scheme is used in our CMOS image sensor in which the brightest pixel within each block fires first and is selected as the reference pixel. The differential values between subsequent pixels and the reference within each block are calculated and quantized, using a reduced number of bits as their dynamic range is compressed. The proposed scheme enables reduced error accumulation as full precision is used at the start of each block, while also enabling reduced memory requirement, and hence, enabling significant silicon area saving. A mathematical model is derived to analyze the performance of the algorithm. Experimental results on a field-programmable gate-array (FPGA) platform illustrate that the proposed algorithm enables more than 50% memory saving at a peak signal-to-noise ratio level of 30 dB with 1.5 bit per pixel.
---
paper_title: A 10 000 fps CMOS Sensor With Massively Parallel Image Processing

paper_content:

A high-speed analog VLSI image acquisition and pre-processing system has been designed and fabricated in a 0.35 ?m standard CMOS process. The chip features a massively parallel architecture enabling the computation of programmable low-level image processing in each pixel. Extraction of spatial gradients and convolutions such as Sobel or Laplacian filters are implemented on the circuit. For this purpose, each 35 ?m × 35 ?m pixel includes a photodiode, an amplifier, two storage capacitors, and an analog arithmetic unit based on a four-quadrant multiplier architecture. The retina provides address-event coded output on three asynchronous buses: one output dedicated to the gradient and the other two to the pixel values. A 64 × 64 pixel proof-of-concept chip was fabricated. A dedicated embedded platform including FPGA and ADCs has also been designed to evaluate the vision chip. Measured results show that the proposed sensor successfully captures raw images up to 10 000 frames per second and runs low-level image processing at a frame rate of 2000 to 5000 frames per second.
---
paper_title: Focal-Plane Spatially Oversampling CMOS Image Compression Sensor

paper_content:

Image compression algorithms employ computationally expensive spatial convolutional transforms. The CMOS image sensor performs spatially compressing image quantization on the focal plane yielding digital output at a rate proportional to the mere information rate of the video. A bank of column-parallel first-order incremental DeltaSigma-modulated analog-to-digital converters (ADCs) performs column-wise distributed focal-plane oversampling of up to eight adjacent pixels and concurrent weighted average quantization. Number of samples per pixel and switched-capacitor sampling sequence order set the amplitude and sign of the pixel coefficient, respectively. A simple digital delay and adder loop performs spatial accumulation over up to eight adjacent ADC outputs during readout. This amounts to computing a two-dimensional block matrix transform with up to 8times8-pixel programmable kernel in parallel for all columns. Noise shaping reduces power dissipation below that of a conventional digital imager while the need for a peripheral DSP is eliminated. A 128times128 active pixel array integrated with a bank of 128 DeltaSigma-modulated ADCs was fabricated in a 0.35-mum CMOS technology. The 3.1 mm times 1.9-mm prototype captures 8-bit digital video at 30 frames/s and yields 4 GMACS projected computational throughput when scaled to HDTV 1080i resolution in discrete cosine transform (DCT) compression
---

Title: CMOS Image Sensor with On-Chip Image Compression: A Review and Performance Analysis

Section 1: Introduction
Description 1: Introduction to the importance and applications of image sensors, the need for image compression, various compression algorithms and standards, and the advantages of integrating image compression with CMOS technology.

Section 2: Still Image-Compression Algorithms
Description 2: Overview of image compression algorithms, including lossy and lossless methods, spatial and transform domain techniques, and standardization efforts.

Section 3: On-Chip Image Compression for CMOS Image Sensors
Description 3: Detailed discussion of predictive coding and wavelet-based image compression integrated with CMOS image sensors, including infrastructure, designs, and effectiveness.

Section 4: Efficiency of the Image-Sensor Array Readout
Description 4: Analysis of different system designs for image-sensor readout efficiency and their impacts on image compression, including MATIA architecture and various block-based readout techniques.

Section 5: High-Performance Compression-Processor Design
Description 5: Exploration of the design and performance of high-efficiency off-array processors for image compression in CMOS sensors.

Section 6: Image Sensors with Focal-Plane Image Compression
Description 6: Investigation of focal-plane image compression techniques for image sensors, including the complexities and advantages of parallel in-array processing.

Section 7: APS Array with Focal-Plane Compression Processor
Description 7: Examination of APS arrays with integrated focal-plane processors for real-time image and video compression and processing, and their architectural designs.

Section 8: DPS Array Integrated Compression Processor
Description 8: Discussion about the latest DPS array image sensors, their integration with adaptive quantization schemes, and their advantages for parallel processing and storage requirements.

Section 9: Compressive Acquisition Image Sensor
Description 9: Introduction and evaluation of the compressive acquisition paradigm, which integrates image compression during the capture phase to reduce on-chip storage and enhance processing efficiency.

Section 10: Performance Analysis and Comparison
Description 10: Comparative analysis of various CMOS image sensors with on-chip compression in terms of resolution, throughput, frame rate, and power consumption, along with challenges associated with these designs.

Section 11: Conclusion
Description 11: Summary of the advantages and future potentials of integrating on-chip image compression with CMOS image sensors, highlighting key takeaways and expectations for future advancements.