Survey of Parallel Computing with MATLAB
---
paper_title: Highly Parallel Computing

paper_content:

Part 1 Foundations: overview - overview and scope of this book, definition and driving forces, questions raised, emerging answers, previous attempts why success now?, conclusions and future directions sample applications - scientific and engineering applications, database systems, artificial intelligence systems, summary technological constraints and opportunities - processor and network technology, memory technology, storage technology computational models and selected algorithms - computational models an operational view, computational models - an analytical view, selected parallel algorithms. Part 2 Parallel software: languages and programming environments - review of the major serial languages, parallel imperative languages and extensions, declarative languages, the programmer's view compilers, other translators - serial compiler essentials, parallelizing compiler essentials, summary and perspective operating systems - operating systems for serial machines, controlling concurrency, classifying operating systems for parallel computers, history of parallel operating systems. Part 3 Parallel architectures: interconnection networks - static connection topologies, dynamic connection topologies SIMD parallel architectures - evolution from von Neumann machines, vector processors, pipelined SIMD vector processors, parallel SIMD designs MIMD parallel architectures - stepping up to MIMD, private memory (message-passing), MIMD designs, shared memory MIMD designs hybrid parallel architectures - VLIW architectures, MSIMD tree machines, MSIMD reconfigurable designs.
---
paper_title: Highly Parallel Computing

paper_content:

Part 1 Foundations: overview - overview and scope of this book, definition and driving forces, questions raised, emerging answers, previous attempts why success now?, conclusions and future directions sample applications - scientific and engineering applications, database systems, artificial intelligence systems, summary technological constraints and opportunities - processor and network technology, memory technology, storage technology computational models and selected algorithms - computational models an operational view, computational models - an analytical view, selected parallel algorithms. Part 2 Parallel software: languages and programming environments - review of the major serial languages, parallel imperative languages and extensions, declarative languages, the programmer's view compilers, other translators - serial compiler essentials, parallelizing compiler essentials, summary and perspective operating systems - operating systems for serial machines, controlling concurrency, classifying operating systems for parallel computers, history of parallel operating systems. Part 3 Parallel architectures: interconnection networks - static connection topologies, dynamic connection topologies SIMD parallel architectures - evolution from von Neumann machines, vector processors, pipelined SIMD vector processors, parallel SIMD designs MIMD parallel architectures - stepping up to MIMD, private memory (message-passing), MIMD designs, shared memory MIMD designs hybrid parallel architectures - VLIW architectures, MSIMD tree machines, MSIMD reconfigurable designs.
---
paper_title: MATLAB®: A Language for Parallel Computing

paper_content:

Parallel computing with the MATLAB® language and environment has received interest from various quarters. The Parallel Computing Toolbox™ and MATLAB® Distributed Computing Server™ from The MathWorks are among several available tools that offer this capability. We explore some of the key features of the parallel MATLAB language that these tools offer. We describe the underlying mechanics as well as the salient design decisions and rationale for certain features in the toolset. The paper concludes by identifying some issues that we must address as the language features evolve.
---
paper_title: Survey of Parallel MATLAB Techniques and Applications to Signal and Image Processing

paper_content:

We present a survey of modern parallel MATLAB techniques. We concentrate on the most promising and well supported techniques with an emphasis in SIP applications. Some of these methods require writing explicit code to perform inter-processor communication while others hide the complexities of communication and computation by using higher level programming interfaces. We cover each approach with special emphasis given to performance and productivity issues.
---
paper_title: MATLAB®: A Language for Parallel Computing

paper_content:

Parallel computing with the MATLAB® language and environment has received interest from various quarters. The Parallel Computing Toolbox™ and MATLAB® Distributed Computing Server™ from The MathWorks are among several available tools that offer this capability. We explore some of the key features of the parallel MATLAB language that these tools offer. We describe the underlying mechanics as well as the salient design decisions and rationale for certain features in the toolset. The paper concludes by identifying some issues that we must address as the language features evolve.
---
paper_title: Survey of Parallel MATLAB Techniques and Applications to Signal and Image Processing

paper_content:

We present a survey of modern parallel MATLAB techniques. We concentrate on the most promising and well supported techniques with an emphasis in SIP applications. Some of these methods require writing explicit code to perform inter-processor communication while others hide the complexities of communication and computation by using higher level programming interfaces. We cover each approach with special emphasis given to performance and productivity issues.
---
paper_title: Enhancements to MatlabMPI: Easier Compilation, Collective Communication, and Profiling

paper_content:

This paper provides a brief overview of several enhancements made to the MatlabMPI suite. MatlabMPI is a pure MATLAB code implementation of the core parts of the MPI specifications. The enhancements provide a more attractive option for HPCMP users to design parallel MATLAB code. Intelligent compiler configuration tools have also been delivered to further isolate MatlabMPI users from the complexities of the UNIX environments on the various HPCMP systems. Users are now able to install and use MatlabMPI with less difficulty, greater flexibility, and increased portability. Collective communication functions were added to MatlabMPI to expand functionality beyond the core implementation. Profiling capabilities, producing TAU (Tuning and Analysis Utility) trace files, are now offered to support parallel code optimization. All of these enhancements have been tested and documented on a variety of HPCMP systems. All material, including commented example code to demonstrate the usefulness of MatlabMPI, is available by contacting the authors.
---
paper_title: MATLAB®: A Language for Parallel Computing

paper_content:

Parallel computing with the MATLAB® language and environment has received interest from various quarters. The Parallel Computing Toolbox™ and MATLAB® Distributed Computing Server™ from The MathWorks are among several available tools that offer this capability. We explore some of the key features of the parallel MATLAB language that these tools offer. We describe the underlying mechanics as well as the salient design decisions and rationale for certain features in the toolset. The paper concludes by identifying some issues that we must address as the language features evolve.
---
paper_title: Survey of Parallel MATLAB Techniques and Applications to Signal and Image Processing

paper_content:

We present a survey of modern parallel MATLAB techniques. We concentrate on the most promising and well supported techniques with an emphasis in SIP applications. Some of these methods require writing explicit code to perform inter-processor communication while others hide the complexities of communication and computation by using higher level programming interfaces. We cover each approach with special emphasis given to performance and productivity issues.
---
paper_title: 'pMATLAB Parallel MATLAB Library'

paper_content:

MATLAB® has emerged as one of the languages most commonly used by scientists and engineers for technical computing, with approximately one million users worldwide. The primary benefits of MATLAB are reduced code development time via high levels of abstractions (e.g. first class multi-dimensional arrays and thousands of built in functions), interpretive, interactive programming, and powerful mathematical graphics. The compute intensive nature of technical computing means that many MATLAB users have codes that can significantly benefit from the increased performance offered by parallel computing. pMatlab provides this capability by implementing parallel global array semantics using standard operator overloading techniques. The core data structure in pMatlab is a distributed numerical array whose distribution onto multiple processors is specified with a "map" construct. Communication operations between distributed arrays are abstracted away from the user and pMatlab transparently supports redistribution between any block-cyclic-overlapped distributions up to four dimensions. pMatlab is built on top of the MatlabMPI communication library and runs on any combination of heterogeneous systems that support MATLAB, which includes Windows, Linux, MacOS X, and SunOS. This paper describes the overall design and architecture of the pMatlab implementation. Performance is validated by implementing the HPC Challenge benchmark suite and comparing pMatlab performance with the equivalent C+MPI codes. These results indicate that pMatlab can often achieve comparable performance to C+MPI, usually at one tenth the code size. Finally, we present implementation data collected from a sample of real pMatlab applications drawn from the approximately one hundred users at MIT Lincoln Laboratory. These data indicate that users are typically able to go from a serial code to an efficient pMatlab code in about 3 hours while changing less than 1% of their code.
---
paper_title: Survey of Parallel MATLAB Techniques and Applications to Signal and Image Processing

paper_content:

We present a survey of modern parallel MATLAB techniques. We concentrate on the most promising and well supported techniques with an emphasis in SIP applications. Some of these methods require writing explicit code to perform inter-processor communication while others hide the complexities of communication and computation by using higher level programming interfaces. We cover each approach with special emphasis given to performance and productivity issues.
---
paper_title: MATLAB®: A Language for Parallel Computing

paper_content:

Parallel computing with the MATLAB® language and environment has received interest from various quarters. The Parallel Computing Toolbox™ and MATLAB® Distributed Computing Server™ from The MathWorks are among several available tools that offer this capability. We explore some of the key features of the parallel MATLAB language that these tools offer. We describe the underlying mechanics as well as the salient design decisions and rationale for certain features in the toolset. The paper concludes by identifying some issues that we must address as the language features evolve.
---
paper_title: 'pMATLAB Parallel MATLAB Library'

paper_content:

MATLAB® has emerged as one of the languages most commonly used by scientists and engineers for technical computing, with approximately one million users worldwide. The primary benefits of MATLAB are reduced code development time via high levels of abstractions (e.g. first class multi-dimensional arrays and thousands of built in functions), interpretive, interactive programming, and powerful mathematical graphics. The compute intensive nature of technical computing means that many MATLAB users have codes that can significantly benefit from the increased performance offered by parallel computing. pMatlab provides this capability by implementing parallel global array semantics using standard operator overloading techniques. The core data structure in pMatlab is a distributed numerical array whose distribution onto multiple processors is specified with a "map" construct. Communication operations between distributed arrays are abstracted away from the user and pMatlab transparently supports redistribution between any block-cyclic-overlapped distributions up to four dimensions. pMatlab is built on top of the MatlabMPI communication library and runs on any combination of heterogeneous systems that support MATLAB, which includes Windows, Linux, MacOS X, and SunOS. This paper describes the overall design and architecture of the pMatlab implementation. Performance is validated by implementing the HPC Challenge benchmark suite and comparing pMatlab performance with the equivalent C+MPI codes. These results indicate that pMatlab can often achieve comparable performance to C+MPI, usually at one tenth the code size. Finally, we present implementation data collected from a sample of real pMatlab applications drawn from the approximately one hundred users at MIT Lincoln Laboratory. These data indicate that users are typically able to go from a serial code to an efficient pMatlab code in about 3 hours while changing less than 1% of their code.
---
paper_title: Survey of Parallel MATLAB Techniques and Applications to Signal and Image Processing

paper_content:

We present a survey of modern parallel MATLAB techniques. We concentrate on the most promising and well supported techniques with an emphasis in SIP applications. Some of these methods require writing explicit code to perform inter-processor communication while others hide the complexities of communication and computation by using higher level programming interfaces. We cover each approach with special emphasis given to performance and productivity issues.
---
paper_title: MATLAB®: A Language for Parallel Computing

paper_content:

Parallel computing with the MATLAB® language and environment has received interest from various quarters. The Parallel Computing Toolbox™ and MATLAB® Distributed Computing Server™ from The MathWorks are among several available tools that offer this capability. We explore some of the key features of the parallel MATLAB language that these tools offer. We describe the underlying mechanics as well as the salient design decisions and rationale for certain features in the toolset. The paper concludes by identifying some issues that we must address as the language features evolve.
---
paper_title: 'pMATLAB Parallel MATLAB Library'

paper_content:

MATLAB® has emerged as one of the languages most commonly used by scientists and engineers for technical computing, with approximately one million users worldwide. The primary benefits of MATLAB are reduced code development time via high levels of abstractions (e.g. first class multi-dimensional arrays and thousands of built in functions), interpretive, interactive programming, and powerful mathematical graphics. The compute intensive nature of technical computing means that many MATLAB users have codes that can significantly benefit from the increased performance offered by parallel computing. pMatlab provides this capability by implementing parallel global array semantics using standard operator overloading techniques. The core data structure in pMatlab is a distributed numerical array whose distribution onto multiple processors is specified with a "map" construct. Communication operations between distributed arrays are abstracted away from the user and pMatlab transparently supports redistribution between any block-cyclic-overlapped distributions up to four dimensions. pMatlab is built on top of the MatlabMPI communication library and runs on any combination of heterogeneous systems that support MATLAB, which includes Windows, Linux, MacOS X, and SunOS. This paper describes the overall design and architecture of the pMatlab implementation. Performance is validated by implementing the HPC Challenge benchmark suite and comparing pMatlab performance with the equivalent C+MPI codes. These results indicate that pMatlab can often achieve comparable performance to C+MPI, usually at one tenth the code size. Finally, we present implementation data collected from a sample of real pMatlab applications drawn from the approximately one hundred users at MIT Lincoln Laboratory. These data indicate that users are typically able to go from a serial code to an efficient pMatlab code in about 3 hours while changing less than 1% of their code.
---

Title: Survey of Parallel Computing with MATLAB

Section 1: INTRODUCTION
Description 1: This section introduces serial computation, parallel computing fundamentals, and the advantages of parallel computing.

Section 2: Computer Memory Architectures
Description 2: This section discusses the types of memory architectures in parallel computer hardware: shared memory, distributed memory, and distributed shared memory. 

Section 3: Matlab (matrix laboratory)
Description 3: This section provides an overview of MATLAB, its capabilities, advantages, and widespread applications in various fields.

Section 4: HISTORY OF PARALLEL COMPUTING WITH MATLAB
Description 4: This section reviews the historical development of parallel computing with MATLAB from 1995 to 2011, exploring various approaches and challenges faced.

Section 5: MatlabMPI
Description 5: This section explains MatlabMPI, its design, advantages, and limitations, including improvements made by Ohio Supercomputer Center.

Section 6: bcMPI
Description 6: This section details bcMPI, an alternative to MatlabMPI developed by Ohio Supercomputer Center, discussing its compatibility and advantages.

Section 7: pMatlab
Description 7: This section introduces pMatlab, its implicit programming approach, and support for global arrays for optimized performance.

Section 8: pMatlab benchmark
Description 8: This section presents benchmark results of pMatlab and comparisons with serial MATLAB and C+MPI.

Section 9: Star-P
Description 9: This section describes Star-P, a set of extensions to MATLAB for simplifying parallel computations.

Section 10: Development of PCT
Description 10: This section traces the development stages of the MATLAB Parallel Computing Toolbox (PCT) from 2006 to mid-2012.

Section 11: CURRENT RESEARCH IN PARALLEL MATLAB
Description 11: This section focuses on new features in MATLAB R2012b, particularly associated with the Parallel Computing Toolbox and GPU support.

Section 12: Graphics Processing Unit (GPU)
Description 12: This section explores the evolution, advantages, and applications of GPUs in parallel computing.

Section 13: Parallel Computing Toolbox in R2012b
Description 13: This section highlights the new features and capabilities of the Parallel Computing Toolbox in MATLAB R2012b and discusses execution of benchmarking problems.

Section 14: Benchmarking A\b on the GPU in R2012b
Description 14: This section demonstrates the performance comparison of matrix left division (\) on CPU and GPU in R2012b.

Section 15: PARALLEL MATLAB FOR NEAR FUTURE
Description 15: This section discusses anticipated future developments in MATLAB parallel computing tools, including an increase in the number of workers and enhanced CUDA compatibility.

Section 16: CONCLUSIONS AND FUTURE WORK
Description 16: This section summarizes the paper, emphasizing the benefits of parallel MATLAB and outlining plans for future work, including the development of a new tool combining previous versions' advantages.