

Rayson Ho raysonlogin at gmail.com
Mon Jan 23 15:50:09 EST 2012

On Mon, Jan 23, 2012 at 11:35 AM, Lux, Jim (337C)
<james.p.lux at jpl.nasa.gov> wrote:
> The "processors in a sea of memory" model has been around for a while
> the chip level: transputers, early hypercubes, etc.)  So this is
> revisiting the architecture at a smaller level of integration.

I remember 12-15 years ago I was reading quite a few papers published
by the Berkeley Intelligent RAM (IRAM) Project:


ship IRAM systems to real customers?? :-D


Open Grid Scheduler / Grid Engine

Scalable Grid Engine Support Program

> One thing about power consumption.. Those memory cells consume so little
> power because most of them  are not being accessed.  They're essentially
> "floating" capacitors. So the power consumption of the same transistor in
> consumption in a memory cell (where the duty factor is 0.001% or
> something).
> And, as always, the challenge is in the software to effectively use the
> distributed computing architecture.  When you think about it, we've had
> almost a century to figure out how to program single instruction stream
> computers of one sort or another, and it was easy, because we are single
> stream (SISD) ourselves.  We can create a simulation of multiple threads
> by timesharing in some sense (in either the human or machine models)
> And we have lots of experience with EP type, or even scatter/gather type
> processes (tilling land, building pyramids, assembly lines) so that model
> of software/hardware architecture can be argued to be a natural outgrowth
> of what humans already do, and have been figuring out how to do for
> millenia.  (did Imhotep use some form of project planning tools?  You bet
> he did)
> However, true parallelism (MIMD) is harder to conceptualize.  Vector and
> matrix math is one area, but I'd argue that it's just the same as EP
> tasks, just at a finer grain. Systolic arrays, vector pipelines, FFT boxes
> from FloatingPointSystems, are all basically ways to use the underlying
> structure of the task, in an easy way (how long til there's a hardware
> implementation of the new faster-than-FFT algorithm published last week?)
> And in all those cases, you have to explicitly make use of the special
> capabilities.  That is, in general, the compiler doesn't recognize it
> (although, modern parallelizing compilers ARE really smart.. So they
> probably do find most of the cases)
> I don't know that we have good conceptual tools to take a complex task and
> break it effectively into multiple disparate component tasks that can
> effectively run in parallel.  It's a hard task for something
> straightforward (e.g. Designing a big system or building a spacecraft),
> and I don't know that any of outputs of current project planning
> techniques (which are entirely manual) can be said to produce
> "generalized" optimum outputs.  They produce *an* output for dividing the
> the output is provably optimum or even workable (an awful lot of projects
> So the problem facing would-be users of new computing architectures (be
> they TOMI, HyperCube, ConnectionMachine, or Beowulf) is like that facing a
> project planner given a big project, and a brand new crew of workers who
> speak a different language, with skill sets totally different than the
> planner is used to.
> This is what the computer user is facing:  There's no compiler or problem
> description technique that will automatically generate a "work plan" to
> use that new architecture. It's all manual, and it's hard, and you're up
> against a brute force "why not just hook 500 people up to that rock and
> drag it" approach.  The people who figure out the new way will certainly
> benefit society, but there's going to be a lot of false starts along the
> way.  And, I'm not particularly sanguine about the process being automated
> (at least in the sense of automatic parallelizing compilers that recognize
> loops and repetitve stuff).  I think that for the next few years
> (decades?) using new architectures is going to rely on skilled humans to
> sitting on my desk for a while.  I wanted to figure out something to do
> focused on stuff like LISP and Prolog).  I started, as I'm sure many
> people do, by taking a multithreaded application I had, and distributing
> the threads to processors.  You pretty quickly realize, though, that it's
> tough to evenly distribute the loads among processors, and you wind up
> with processor 1 waiting for something that processor 2 is doing, which in
> turn is waiting for something that processor 3 is doing, and so forth.  In
> a "shared processor" this isn't a big deal, and is transparent: the
> processor is always working, and aside from deadlocks, there's no
> particular reason why you need to balance load among threads.
> For what it's worth, the task I was doing was comparable to taking
> execution of a Matlab/simulink model and distributing it across multiple
> processors.  You had signals flowing among blocks, etc.  These things are
> computationally intensive (especially if you have loops in the design, so
> you need an iterative solution of some sort) so the idea of putting
> multiple processors to work is attractive.   But the "work" in each block
> in the diagram isn't known a-priori and might vary during the course of
> the simulation, so it's not like you can come up with some sort of
> automatic partitioning algorithm.
>>If you read this PDF from Venray Technologies, which is linked to in the
>>reading it, Venray lost a lot of credibility in my book.
>>On 01/23/2012 08:45 AM, Eugen Leitl wrote:
>>> CPU Startup Combines CPU+DRAM‹And A Whole Bunch Of Crazy
>>> Sunday, January 22, 2012 - by Joel Hruska
>>> The CPU design firm Venray Technology announced a new product design
>>> week that it claims can deliver enormous performance benefits by
>>>earlier this
>>> fall discussing the new TOMI (Thread Optimized Multiprocessor) with
>>> CTO Russell Fish, but while the idea is interesting; its presentation is
>>> marred by crazy conceptualizing and deeply suspect analytics.
>>> The Multicore Problem:
>>> There are three limiting factors, or walls, that limit the scaling of
>>> microprocessors. First, there's the memory wall, defined as the gap
>>> Parallelism) wall, which refers to the difficulty of decoding enough
>>> instructions per clock cycle to keep a core completely busy. Finally,
>>> it consumes.
>>> Attempting to compensate for one wall often risks running afoul of the
>>> discrepancy adds die complexity and draws more power, as does raising
>>> clock speed. Combined, the three walls are a set of fundamental
>>> constraints--improving architectural efficiency and moving to a smaller
>>> process technology may make the room a bit bigger, but they don't
>>>remove the
>>> walls themselves.
>>> TOMI attempts to redefine the problem by building a very different type
>>> microprocessor. The TOMI Borealis is built using the same transistor
>>> structures as conventional DRAM; the chip trades clock speed and
>>>simple. Not
>>> counting the cache, TOMI is a 22,000 transistor design, as compared to
>>> transistors for the original ARM2. The company's early prototypes,
>>>built on
>>> legacy DRAM technology, ran at 500MHz on a 110nm process.
>>> Instead of surrounding a CPU core with a substantial amount of L2 and L3
>>> Borealis core connects eight TOMI cores to a 1Gbit DRAM with a total of
>>> Because they're built using ultra-low-leakage processes and are so
>>> such cores cost very little to build and consume vanishingly small
>>>amounts of
>>> power (Venray claims power consumption is as low as 23mW per core at
>>> It's an interesting idea.
>>> The Bad:
>>> When your CPU has fewer transistors than an architecture that debuted in
>>> prediction, pipelining, or any form of speculative execution. Venray
>>>may have
>>> created a chip with power consumption an order of magnitude lower than
>>> anything ARM builds and more memory bandwidth than Intel's highest-end
>>> but it's an ultra-specialized, ultra-lightweight core that trades 25
>>>years of
>>> flexibility and performance for scads of memory bandwidth.
>>> The last few years have seen a dramatic surge in the number of
>>> many-core architectures being floated as the potential future of
>>> but Venray's approach relies on the manufacturing expertise of
>>>companies who
>>> have no experience in building microprocessors and don't normally serve
>>> foundries. This imposes fundamental restrictions on the CPU's ability to
>>> scale; DRAM is manufactured using a three layer mask rather than the
>>> layers Intel and AMD use for their CPUs. Venray already acknowledges
>>> these conditions imposed substantial limitations on the original TOMI
>>> Of course, there's still a chance that the TOMI uarch could be
>>>effective in
>>> certain bandwidth-hungry scenarios--but that's where the Venray Crazy
>>> goes flying off the track.
>>> The Disingenuous and Crazy
>>> the same systems being compared across three different characteristics.
>>> pointed us to this publicly available document and claimed that the
>>> The "Power consumption" graphs show Oracle's maximum power consumption
>>>for a
>>> system with 10x Xeon E7-8870s, 168 dedicated SQL processors, 5.3TB
>>>(yes, TB)
>>> it's a figure utterly unrelated to the workload shown in the Performance
>>>ten of
>>> them only come out to 1.3kW--Oracle's 17.7kW figure means that the
>>> overwhelming majority of the cabinet's power consumption is driven by
>>> components other than its CPUs.
>>> From here, things rapidly get worse. Fish makes his points about power
>>> by referring to unverified claims that prototype 90nm Tejas chips drew
>>> at 2.8GHz back in 2004. That's like arguing that Ford can't build a
>>> car because the Edsel sucked.
>>> After reading about the technology, you might think Venray was planning
>>>wrong. The
>>> company expects the following to occur as a result of this revolutionary
>>> architecture (organized by least-to-most creepy):
>>>     Computer speech will be so common that devices will talk to other
>>> in the presence of their users.
>>>     Your cell phone camera will recognize the face of anyone it sees
>>>and scan
>>> the computer cloud for backround red flags as well as six degrees of
>>> separation
>>>     Common commands will be reduced to short verbal cues like clicking
>>> tongue or sucking your lips
>>>     Your personal history will be displayed for one and all to
>>> will create search engines to find eligible, prosperous men. Men will
>>> search engines to qualify women. Criminals will find their jobs much
>>> difficult because their history will be immediately known to anyone who
>>> encounters them.
>>>     TOMI Technology will be built on flash memories creating the
>>> build robust communicating structures, and collaborate to perform tasks.
>>>     A disposable diaper company will give away TOMI enabled teddy bears
>>> teach reading and arithmetic. It will be able to identify specific
>>> also diagnose a raspy throat, a cough, or runny nose.
>>> Conclusion:
>>> Fish has spent decades in the microprocessor industry--he invented the
>>> vision of the future is crazy enough to scare mad dogs and Englishmen.
>>> His idea for a CPU architecture is interesting, even underneath the
>>> obfuscation and false representation, but too practically limited to
>>> take off. Google, an enthusiastic and dedicated proponent of energy
>>> still beat wimpy cores, most of the time."
>>>two or
>>> so behind the higher end of current-generation commodity processors,
>>>making a
>>> business case for switching to the wimpy system becomes increasingly
>>>moderation, or
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing

This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

More information about the Beowulf mailing list