Slashdot: News for Nerds


Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Scaling To a Million Cores and Beyond

kdawson posted about 4 years ago | from the can't-get-there-from-here dept.

Supercomputing 206

mattaw writes "In my blog post I describe a system designed to test a route to the potential future of computing. What do we do when we have computers with 1 million cores? What about a billion? How about 100 billion? None of our current programming models or computer architecture models apply to machines of this complexity (and with their corresponding component failure rate and other scaling issues). The current model of coherent memory/identical time/everything can route to everywhere; it just can't scale to machines of this size. So the scientists at the University of Manchester (including Steve Furber, one of the ARM founders) and the University of Southampton turned to the brain for a new model. Our brains just don't work like any computers we currently make. Our brains have a lot more than 1 million processing elements (more like the 100 billion), all of which don't have any precise idea of time (vague ordering of events maybe) nor a shared memory; and not everything routes to everything else. But anyone who argues the brain isn't a pretty spiffy processing system ends up looking pretty silly. In effect, modern computing bears as much relation to biological computing as the ordered world of sudoku does to the statistical chaos of quantum mechanics.

cancel ×


Reminds me of Hillis (1)

Arancaytar (966377) | about 4 years ago | (#32741008)

With his Connection Machine.

Don't remember how many cores that one had...

Re:Reminds me of Hillis (2, Informative)

headhot (137860) | about 4 years ago | (#32741286)

The CM-1 has 65,536 one bit processors. The CM-5 was in Jurassic Park, and some phone companies.

Re:Reminds me of Hillis (2, Insightful)

teazen (876487) | about 4 years ago | (#32741488)

Exactly! New is the new old. A million processors? Pah! Old hat. There has been done lots of interesting research into parallel processing in the past. Read the Connection Machine book [] It's a great read.

Feynman was also involved with the machine at a certain point. There's a great writeup [] on him and it for a quick introduction: '.. It was a complicated device; by comparison, the processors themselves were simple. Connecting a separate communication wire between each pair of processors was impractical since a million processors would require $10^{12]$ wires. Instead, we planned to connect the processors in a 20-dimensional hypercube so that each processor would only need to talk to 20 others directly. ..'

The C-5 looked [] awesome as well. And I'll just keep quiet about all the cool Lisp stuff they did on it.

Re:Reminds me of Hillis (1)

Z00L00K (682162) | about 4 years ago | (#32741648)

And I would probably select Erlang [] as programming language for a massive amount of cores.

Re:Reminds me of Hillis (4, Interesting)

pieterh (196118) | about 4 years ago | (#32741880)

You don't even need Erland, you can use a lightweight message-passing library like ZeroMQ [] that lets you build fast concurrent applications in 20 or so languages. It looks like sockets but implements Actors that connect in various patterns (pubsub, request-reply, butterfly), and works with Ruby, Python, C, C++, Java, Ada, C++, CLisp, Go, Haskell, Perl, and even Erlang. You can even mix components in any language.

You get concurrent apps with no shared state, no shared clock, and components that can come and go at any time, and communicate only by sending each other messages.

In hardware terms it lets you run one thread per core, at full efficiency, with no wait states. In software terms it lets you build at any scale, even to the scale of the human brain, which is basically a message-passing concurrent architecture.

Re:Reminds me of Hillis (0, Troll)

swilver (617741) | about 4 years ago | (#32742094)

And what these researchers don't seem to get is that for application flow and implementing business logic, which is what most programmers are concerned with, there's no need for programming for multiple CPU's.

Everything that really sucks the life out of your CPU/GPU is usually trivial to run in parallel (and the most time critical stuff is doing so already), and it perhaps represent as little as 5% of all code run by these processors. All the other code that most programmers concern themselves with daily does not need a new programming model.

Better be running OSS (0)

Anonymous Coward | about 4 years ago | (#32741018)

This 1-million core machine better be running open source software and not proprietary software. You know who runs proprietary software? Microsoft and Apple. Yeah, they make a lot of money, but they'd make much more money if they gave away their software and sold support licenses.

Re:Better be running OSS (3, Interesting)

jd (1658) | about 4 years ago | (#32741126)

I don't know about this specific project, but Manchester is strongly Open Source. The Manchester Computer Centre developed one of the first Linux distributions (and - at the time - one of the best). The Advanced Processor Technologies group has open-sourced software for developing asynchronous microelectronics and FPGA design software.

Manchester University is highly regarded for pioneering work (they were working on parallel systems in 1971, and developed the first stored-program computer in 1948) and they have never been ashamed to share what they know and do. (Disclaimer: I studied at and worked at UMIST, which was bought by Manchester, and my late father was a senior lecturer/reader of Chemistry at Manchester. I also maintain Freshmeat pages for the BALSA projects at APT.)

Re:Better be running OSS (5, Funny)

capo_dei_capi (1794030) | about 4 years ago | (#32741166)

This 1-million core machine better be running open source software and not proprietary software.

Yeah, especially if their software is licensed on a per-core basis.

1 billion cores (2, Informative)

should_be_linear (779431) | about 4 years ago | (#32741042)

thats about 30 forks(), and there you go.

Re:1 billion cores (0)

Anonymous Coward | about 4 years ago | (#32741148)

...or spaghetti for 29 dining philosophers!

Re:1 billion cores (2, Insightful)

BhaKi (1316335) | about 4 years ago | (#32741174)

Wrong. You need 999999999 forks.

Re:1 billion cores (3, Informative)

TheSpoom (715771) | about 4 years ago | (#32741318)

I'm pretty sure the poster meant to do something like this:

fork(); // etc.

which would make the number of processes increase exponentially every time the forked processes forked again. Not 1, 2, 3, but 1, 2, 4, 8, 16... and 2^30 gets you above 1 billion.

multi core design (4, Insightful)

girlintraining (1395911) | about 4 years ago | (#32741054)

Simply put, there are some computational problems that work well with parallelization. And there are some that no matter how you try to approach it, you come back to a serial-based model. You could have a billion core machine running at 1Ghz get stomped by a single core machine running at 1.7Ghz for certain computational processes. We have yet to find a way computationally or mathematically to make intrinsically serialized problems into parallel ones. If we did, it would probably open up a whole new field of mathematics.

Re:multi core design (4, Interesting)

jd (1658) | about 4 years ago | (#32741090)

You cannot parallelize a serial task, any more than you can have 60 people dig one posthole in one second. On the other hand, there are MANY tasks that are inherently parallel but which are serialized because either the programmers aren't up to the task, the OS isn't up to the task or the CPU isn't up to the task.

(I don't know if kernel threads under Linux will be divided between CPUs in an SMP system, they certainly can't migrate across motherboards in any MOSIX-type project. That limits how parallel the bottlenecks in the program can ever become. And it's one of the best OS' out there.)

Re:multi core design (1, Offtopic)

tuomoks (246421) | about 4 years ago | (#32741270)

Correct, some problems are serial - at least in current world, another topic! But parallelism is not same as threads - you can run any number of tasks, possible since 60's or 70's depends on who you ask. Unfortunately today, the younger generation has to learn the multi-tasking again, all they know (if they know!) is multi-threading, another way to make parallel processing but not the only. There is much, much more in parallel processing - code is not the only issue but one where more "cores" help. Sometimes wonder - non-locking memory management and queue processing are old, old - see some IBM research from 70's, how many has even heard of those? Very useful when several "cores"!

Re:multi core design (3, Informative)

jd (1658) | about 4 years ago | (#32741578)

You've got to be careful when talking about threads. There are four basic models: SISD, SIMD, MISD and MIMD. Of those, only SISD is serial, but if you've two independent SISD tasks, you can run them in parallel. Most modern supercomputers are built on the premise that SIMD is good enough. Not sure where MISD is used, MIMD fell out of favour when vector processors became too expensive but may be revived on more modest CPUs with modern interconnects like Infiniband.

Re:multi core design (5, Interesting)

William Robinson (875390) | about 4 years ago | (#32741760)

Not sure where MISD is used

Back in 1987, when I was part of team that was designing parallel processing machine, with 4 neighboring CPUs sharing common memory (apart from their own local memory, kind of systolic array), we were designing machine suitable to simulate aerodynamics or weather forecasting using diffusion equations. We believed that it was working on MISD model, where different algorithms running in different CPUs utilized same data for analysis, using bus arbitration logic.

Re:multi core design (2, Interesting)

Your.Master (1088569) | about 4 years ago | (#32741294)

Actually, you can to some extent serialize a parallel task, with sufficiently many cores.

For instance, you could just guess at all the intermediate results of halfway through a long sequence of operations, and execute from there, but discard the information if it's wrong. With lots of cores and a good chokepoint, you might be able to gain a 2x speedup a significant percent of the time (for a lower average speedup). 2x, that is, from billions of cores.

Kind of like branch prediction, or a dynamically generated giant lookup table.

It just isn't a very efficient speedup, at all, compared to the gains of even modestly parallelizable tasks.

Re:multi core design (1)

Your.Master (1088569) | about 4 years ago | (#32741304)

Ugh, I meant "parallelize a serial task".

Re:multi core design (1)

gknoy (899301) | about 4 years ago | (#32741448)

you could just guess at all the intermediate results of halfway through a long sequence of operations, and execute from there, but discard the information if it's wrong.

How would you know that the calculations were wrong?

Re:multi core design (1)

Paradigma11 (645246) | about 4 years ago | (#32741486)

you could just guess at all the intermediate results of halfway through a long sequence of operations, and execute from there, but discard the information if it's wrong.

How would you know that the calculations were wrong?

You get the arguments for the next computation and can look the result up in a dictionary/ hash table. (space for time trade)

Re:multi core design (1)

CarpetShark (865376) | about 4 years ago | (#32741620)

have 60 people dig one posthole in one second

Where there's a will, there's a way.

Re:multi core design (1)

roman_mir (125474) | about 4 years ago | (#32741788)

You cannot parallelize a serial task, any more than you can have 60 people dig one posthole in one second.

- the appropriate analogy is 9 women giving birth to one baby 1 month after conception.

Then again, this is slashdot, but it's good for learning new stuffs []

Re:multi core design (1)

Ihlosi (895663) | about 4 years ago | (#32741848)

- the appropriate analogy is 9 women giving birth to one baby 1 month after conception. Then again, this is slashdot, but it's good for learning new stuffs

The woman-month is even more mythical than the man-month in this case.


Re:multi core design (0)

Anonymous Coward | about 4 years ago | (#32741978)

I think your point that "you cannot parallelize a serial task" is exactly what the author is talking about - how do you take a serial problem and make it more efficient through massive parallelization?

Re:multi core design (1)

anarche (1525323) | about 4 years ago | (#32741124)

that sounds like a mathematics phd in the making to me...

Re:multi core design (1)

EWillieL (15339) | about 4 years ago | (#32741274)

There are whole classes of tasks that simply can't be done in parallel.

For example, If you want a baby in three months, you don't impregnate three women and hope for the best. You buy him/her from the Impoverished Nation of the Week.

Re:multi core design (3, Interesting)

Anonymous Coward | about 4 years ago | (#32741350)

If one is willing to use the transistors of a billion core machines to speed up a single problem anyway, some transistors could be better used to accelerate the specific problem in the form of custom circuits. There could be a similar layered structure in the system as the brain uses. The lowest, fastest and task customized levels could be build using hard-wired logic, the layer above it using slowly reconfigurable circuits with very fast switching speeds, the layers above easily reconfigurable circuits with slower switching speeds and the highest level using the normal general purpose logic. A higher level could train and use the services of a lower level just like the brains might do in a case of a phobia, trauma or some psychosomatic condition. New sub fields of computer science and computer engineering of a computer psychologist and a computer psychiatrist might be created out of necessity..

Re:multi core design (1)

chichilalescu (1647065) | about 4 years ago | (#32741940)

mod parent up. i'm not sure wether it's funny, insightful or plain stupid, but it needs to be seen by others.
as a sidenote, our brain is perfectly capable of performing complex numerical computations. the problem is that we don't have a way of consciously accessing those functions. similar problems will arise for any system designed on "levels".

Re:multi core design (2, Interesting)

smallfries (601545) | about 4 years ago | (#32742052)

While turning intrinsically serial problems into a parallel form would certainly open up a new field it is doubtful that it would be a "a whole new field of mathematics" or did you just like the sound of your hyperbole?

On a slightly different note; every time there is any article about parallel architecture on slashdot someone raises the problem of inherently serial tasks. Can you name any? Or more to the point can you name an inherently serial task on a realistically sized data-set that can't be tackled by a single 2Ghz core?

It would seem that we have scaled single-core performance to the point that we don't care about serial tasks any more. All of the interesting performance hogs that we do care about can be parallelised.

Neural Network (0)

Anonymous Coward | about 4 years ago | (#32741062)

Isn't that a neural network, you know the things that have been research for over 40 years and do things great but mostly do one thing well and are very poor at doing things in general

Re:Neural Network (0)

Anonymous Coward | about 4 years ago | (#32742058)

No, it's not.

Bluudy Blogs (3, Informative)

jd (1658) | about 4 years ago | (#32741064)

I've left out links to some projects, by request, but everything can be found on their homepage anyway. Anyways, it is this combination that is important, NOT one component alone.

Re:Bluudy Blogs (1)

mattaw (718560) | about 4 years ago | (#32741398)

Fair points, all of them. Now explicit in the article itself.

Distributed systems (1)

MichaelSmith (789609) | about 4 years ago | (#32741096)

don't have any precise idea of time (vague ordering of events maybe) nor a shared memory; and not everything routes to everything else

Sounds like a large scale distributed system. Maybe somebody should ask google about this.

Re:Distributed systems (1)

CarpetShark (865376) | about 4 years ago | (#32741598)

Boink has half a million computers, many of which probably have more than 2 cores.

Re:Distributed systems (1)

CarpetShark (865376) | about 4 years ago | (#32741612)

Boinc, even.

Re:Distributed systems (1)

MichaelSmith (789609) | about 4 years ago | (#32741678)

Yeah though boinc is pretty simple. Just a flat array of machines which do stuff. Google is closer to the spaghetti like structure of the brain.

Distributed Computing (1)

sfcat (872532) | about 4 years ago | (#32741102)

What you are describing is the problems around distributed systems. What would I do with a billion cores? Run tens of millions of instances of VMWare (x8 or 16 each) and write distributed code that runs on millions of machines. No shared memory, communication channels which are slow compared with computation? Basically, that's the line between distributed systems and non-distributed systems. Not that most distributed systems problems are solved, but this is the model that we would be investigating assuming no major shift in the computational model (turning vs quantum, etc).

Re:Distributed Computing (1)

CarpetShark (865376) | about 4 years ago | (#32741674)

What would I do with a billion cores? Run tens of millions of instances of VMWare (x8 or 16 each) and write distributed code that runs on millions of machines.

Not without very serious disk, memory, and network subsystems. CPU cores are not the only bottleneck in a VM setup.

multi-tasking (1)

vivek7006 (585218) | about 4 years ago | (#32741114)

"Our brains have a lot more than 1 million processing elements and not everything routes to everything else"

Thats why I can watch pr0n and code at the same time!

Problems with this blog. (4, Informative)

Anonymous Coward | about 4 years ago | (#32741140)

The problem posed by the author is somewhat of a straw man argument: "The trouble is once you go to more than a few thousand cores the shared memory - shared time concept falls to bits."

Multiple processors in a single multicore aren't required even today to be in lockstep in time (it is actually very difficult to do this). Yes, locally within each core and privates caches they do maintain a synchronous clock, but cores can run in their own clock domains. So I don't buy the argument about scaling with "shared time".

Secondly, the author states that the "future" of computing should automatically be massively parallel. Clearly they are forgetting about Amdahl's Law ('s_law). If your application is 99.9% parallelizable, the MOST speedup I can expect to achieve is 1000X, forget about millions. High sequential performance (ala out-of-order execution, etc.) will not be going away anytime in the near future simply because they are best equipped to deal with serial regions of an application.

Finally, I was under the impression that they were talking about fitting "millions" of cores onto a single die, until I read the to the end of the post that they are connecting multiple boards via multi-gigabit links. Each chip on a board has about 20 or so cores with privates caches or local store. They talk to other cores on other boards through off-chip links...... SO isn't this just a plain old message passing computer?! What's the novelty here? Am I missing something?

Re:Problems with this blog. (2, Insightful)

jd (1658) | about 4 years ago | (#32741180)

The main problem is that it's horribly hard to pass that many messages around without the overheads of the network exceeding the benefit from the parallelization. If they have found a way to reduce this problem, I'd call that a major novelty.

Re:Problems with this blog. (2, Interesting)

WheelDweller (108946) | about 4 years ago | (#32741966)

I've felt the same way; just _because_ you can create 1,000,000 CPUs connected to a single bus doesn't mean it's the right thing to do. We're still limited to physics, and if they're x86 cores, we're still dealing with all that "address bus is multiplexed" crap that was a problem in 1985. (!)

I have the feeling that, if we're so enamored with multiple cores, they need to be smaller, simpler, and able to communicate amongst each other. This is how (in the real world) multiple CPUs actually map like the human brain.

Just how much memory and refresh cycles ARE there in synapse? (See the point?)

If we can arrange types of CPUs, give them the ability to communicate with every other node, put that 1,000,000 into a net with neural-net style layouts, and THEN we're talking.

Consider the Texas Instruments 1-bit CPU. Only 16 instructions and something like 7 of them are no-ops! All programs take the same amount of time to execute. You can make BANKS of these things, to create 2048-bit computers if you like. Of course, it helps to do this at the core-level, not the 8-pin DIP level. :)

Unfortunately for the traditional integrators, THIS is how we're going to the next level of operation. The PC has little in common with brains. It's a metaphor.

Human brain != computer (2, Insightful)

i-like-burritos (1532531) | about 4 years ago | (#32741142)

The things we use computers for are different from the things we use humans for.

Computers are consistant and predictable. The human brain is not.

We have billions of human brains cheaply available, so let's use those when we want a human brain. And let's use computers when we want computers.

Re:Human brain != computer (0)

Anonymous Coward | about 4 years ago | (#32741178)

We use humans for different things than computers? Really? I think you've missed about the last 50 years of automation...
One of the main things computers are used for is doing what were formerly human tasks.

Re:Human brain != computer (1)

chichilalescu (1647065) | about 4 years ago | (#32741314)

Yes, because only humans were able to do them. But a computer can count much faster and much more accurately than a human, BECAUSE it is consistent and predictable.
In practice the relationship is:
        human brains are very good at generating algorithms to solve specific problems.
        computers are very good at applying the algorithm.

Re:Human brain != computer (1, Insightful)

Anonymous Coward | about 4 years ago | (#32741346)

Then all we have to do is, generate a algorithm (that can run on a computer) that is good enough at generating algorithms to solve specific problems.

Mods, feel free to mod off-topic

Re:Human brain != computer (1)

chichilalescu (1647065) | about 4 years ago | (#32741670)

the best we came up with (as far as I know) is to have an algorithm that mimicks the human brain :)
so it would only be useful for robots going in places where humans can't follow. because, at least for now, we lack the technology of efficiently simulating the human brain. These guys are trying to do just that (I think): rethink the hardware to simulate brains more efficiently.

Re:Human brain != computer (1)

aramosfet (1824288) | about 4 years ago | (#32741340)

Let me correct that for you, When you say "different things" its means things computers cant do, like doing research, figuring out how to build complex things etc.automation is done to things which are routine so that the brain can spend time doing what computers cannot.

Re:Human brain != computer (1)

iammani (1392285) | about 4 years ago | (#32741306)

I would say Brain = Computer; only the Input Output ports/devices are different. Say your task is to find a picture of Megan Fox among a bundle of photos, you could use a brain (along with necessary I/O devices of course) to perform this task. You need to input these photos (along with a few marked snaps of Fox, if the brain does not already know her), through the eyes attached to it and you could ask the brain to output the result using the mouth attached to it.

Re:Human brain != computer (1)

dargaud (518470) | about 4 years ago | (#32741530)

The things we use computers for are different from the things we use humans for.

Yeah, zombies can't eat computers...

Re:Human brain != computer (1)

MemoryDragon (544441) | about 4 years ago | (#32741726)

I would not call computers entirely predictable, there are too many influences which can derail them, for instance an accidental bit flip, hardware design issues etc...

Re:Human brain != computer (1)

wvmarle (1070040) | about 4 years ago | (#32741786)

I think this discussion is mainly relevant considering we are reaching end points hardware-wise. There is only that many transistors one can put in a processor. There is only that many GHz silicon can manage. So we're reaching the end point of the processor as single unit - the logical step is not to increase the one processor, but to take more of those processors and try to let them work together. The human brain may be an interesting model for that.

And of course in the end Intel et. al have to think of the longer term future.

Re:Human brain != computer (1, Interesting)

Xest (935314) | about 4 years ago | (#32741904)

I was going to post about this, but you've already covered part of it, so I'll reply instead.

I take issue with this statement from the summary:

"But anyone who argues the brain isn't a pretty spiffy processing system ends up looking pretty silly."

No, they don't look silly, they look smarter and more knowledgeable about the topic than the idiot who made this comment. The human brain has multiple flaws, and whilst it's excellent for some things, it's terrible for others. The human brain relies on emergence, which is great for solving some problems but not so for others. To give an example, the human brain is great for picking out an object in a scene, but hopeless for calculating with a reasonable degree of accuracy the actual distance to that object- the margin of error for most people is on average going to be quite large.

But also the human brain is prone to mistakes, minor changes to it's working can make it come up with odd results- notice how sometimes something you know inside out you just happen to get wrong? Look at how sometimes people are mid-conversation, talking about something they know in depth and suddenly they forget what they were going to say- this is because processing in the brain has gone completely off track.

The fundamental problem with copying the brain is that it's far from perfect, it doesn't give us the level of preciseness that classic computing generally does. It doesn't guarantee there wont be fluctuations in a task every single time it performs that task.

A brain like computer would certainly be useful, it could help solve some interesting problems, but it's most definitely not perfect, and it's most definitely not the ultimate solution to our computing problems- there are plenty of scenarios where it would be completely useless. This is after all, why we bother to have computers in the first place.

I do not see the brain as any more spiffy than computing technology, sure its amazingly powerful, but there's a lot that current computers can do that the brain can't- serious large scale number crunching for example. Could we ever trust a brain like model to crunch numbers when we know full well the brain could make a mistake giving erroneus results on a few of those numbers where the computer would not?

Re:Human brain != computer (1)

f3r (1653221) | about 4 years ago | (#32741990)

We have billions of human brains cheaply available

I wouldn't say raising a human up to adulthood is cheap...

Re:Human brain != computer (1)

perryizgr8 (1370173) | about 4 years ago | (#32741992)

Computers are consistant and predictable. The human brain is not.

this is because we do not understand the algorithms/software running inside the brain. believe me, to a newbie computer user, a computer is just as unpredictable as a normal person. sometimes computers are unpredictable even to me, especially if they are running m$ software!
i also think that the brain is exactly analogous to a digital computer.

green array chips - 144 cores per die (1, Interesting)

Anonymous Coward | about 4 years ago | (#32741152)

tiny Forth-based computers with up to 144 cores on a chip, and that's in a low tech 180 nanometer process. Each core has a rather fast ALU but just a few hundred(?) bytes of memory. Seems closer to neurons than the thing that guy is making where each core is a 32-bit ARM processor.

link [] .

Damaged Brains (3, Insightful)

b4upoo (166390) | about 4 years ago | (#32741168)

Some folks with severely damaged brains seem to make better human computers than people with healthy brains. Rain Man leaps to mind as well as other savants. It seems that when some parts of the brain are impaired the energy of thought is diverted to narrower functions. Perhaps we need to think of delivery more energy to less cores to make machines that do tasks that normal humans are not so good at doing.

Re: Damaged Brains (1)

imakemusic (1164993) | about 4 years ago | (#32742004)

Isn't that the status quo? I currently have a relatively low-power computer that can calculate Pi to an impressive amount of decimal places in less than a second - considerably better than I can! However it still can't hold a decent conversation.

Re: Damaged Brains (1)

perryizgr8 (1370173) | about 4 years ago | (#32742006)

i think you are confusing a hollywood film with the real life. rain man does not exist out of the movie.

Link with IMAC ExaScience lab? (2, Interesting)

Hedon (192607) | about 4 years ago | (#32741184)

Is it coincidence that earlier this month there was a press release from IMEC regarding the issues of massively scaling up computational power ("exascaling")?
Press blurb can be found here [] .
Killer application would be "space weather prediction".

ANALOG REVIVAL!!!!111!!!11!1!1 (1)

Xanavi (1197431) | about 4 years ago | (#32741188)

always seemed logical the next big step would be a return to analog :P

Distributed Computing (2, Informative)

thomasinx (643997) | about 4 years ago | (#32741190)

The problems with "coherent memory/identical time/everything can route to everywhere" isnt only seen when you get up to a million cores. I've done plenty of work with MPI and pthreads, and depending on how it's organized, a significant portion of these methods start showing inefficiencies when you get into just a few hundred cores.

Since there are already plenty of clusters containing thousands upon thousands of individual processors (which dont use coherent memory..etc), the step to scale up to a million would likely follow the same logical development. There should already be one or two decent CS papers on the topic, since it's basically a problem that's been around since beowulf clusters were popularized (or even before then)

Put BeOS on it.. (1)

moooooooo (411115) | about 4 years ago | (#32741202)

and write an OpenGL app which puts multiple videos on the panes of multiple rotating truncated icosahedrons a la the old famous BeOS rotating OpenGL cube app from years ago.

Last time I run a parallel program... (2, Interesting)

ctrl-alt-canc (977108) | about 4 years ago | (#32741248) seemed to me that Amdahl's law [] was still alive and kicking.

Re:Last time I run a parallel program... (4, Insightful) (1195047) | about 4 years ago | (#32741352)

Given you statement, why would you link to a document entitled Reevaluating Amdahl's Law [] ? Did you even read what you linked to? Here's an excerpt:

Our work to date shows that it is not an insurmountable task to extract very high efficiency from a massively-parallel ensemble, for the reasons presented here. We feel that it is important for the computing research community to overcome the "mental block" against massive parallelism imposed by a misuse of Amdahl's speedup formula; speedup should be measured by scaling the problem to the number of processors, not fixing problem size. We expect to extend our success to a broader range of applications and even larger values for N.

Re:Last time I run a parallel program... (1)

jvonk (315830) | about 4 years ago | (#32741830)

Gustafson's Law giveth, while Amdahl's Law taketh away.

Interesting... []

Re:Last time I run a parallel program... (1)

Burnhard (1031106) | about 4 years ago | (#32741538)

One of the criticisms of Amdahl's Law is that it makes pessimistic assumptions about the amount of program code that must be serial. These assumptions are wholly dependent on the problem domain under consideration of course. To use Google as a case in point, the amount of serial code is closer to 0% and so Amdahl's Law doesn't really apply here. So, to state the obvious, this kind of processing works well if the problem to be solved is naturally parallel in nature (each element can be effectively processed independently).

Transputer, The Next Generation (2, Insightful)

Animats (122034) | about 4 years ago | (#32741252)

This is very similar to the Inmos Transputer [] , a mid-1980s system. It's the same idea: many processors, no shared memory, message passing over fast serial links. The Transputer suffered from a slow development cycle; by the time it shipped, each new part was behind mainstream CPUs.

This new thing has more potential, though. There's enough memory per CPU to get something done. Each Cell processor, with only 256K per CPU, didn't have enough memory to do much on its own. 20 CPUs sharing 1GB gives 50MB per CPU, which has more promise. Each machine is big enough that this can be viewed as a cluster, something that's reasonably well understood. Cell CPUs are too tiny for that; they tend to be used as DSPs processing streaming data.

As usual, the problem will be to figure out how to program it. The original article talks about "neurons" too much. That hasn't historically been a really useful concept in computing.

Dangerous idea (1, Interesting)

Anonymous Coward | about 4 years ago | (#32741258)

To make any computer mimic the design and function of the human brain would invite evolution and sentience. We want tools, not sentient machines.

Re:Dangerous idea (4, Insightful)

ProfessionalCookie (673314) | about 4 years ago | (#32741382)

From a science perspective I'm pretty sure that either computer are already "sentient" or (IMHO, more likely) that we don't really understand what sentience is. At all.

Re:Dangerous idea (1)

Hognoxious (631665) | about 4 years ago | (#32741842)

Butlerian Juhad here we come!

Re:Dangerous idea (2, Interesting)

perryizgr8 (1370173) | about 4 years ago | (#32742018)

on the contrary, i can't wait for sentient machines. at last we will be free of shoddy human programming. no vendor lockins and other such stuff. just tell your computer to write a specific program you desire for a specific purpose, and he will write it for you.

The Brain. (1)

headhot (137860) | about 4 years ago | (#32741308)

Well, when you start thinking about 10^6 or more cores, is pretty obvious that they cannot all be connected to each other and cannot all share memory. At that point your in the realm of neural networking, and are looking at many many serial (and parallel) tasks running in parallel.

When you look at the brain, it has evolved so that different areas have different purposes and techniques for processing data. There are some very highly specialized systems in there for very specific problems.

Steve Furber (1)

Alioth (221270) | about 4 years ago | (#32741332)

I saw Steve Furber talk at Retro Reunited in Huddersfield last year (where he spoke about the past - Acorn, the development of the ARM, the present, and the future in terms of what they were doing with SpiNNaker. Very interesting talk. (I also saw Sophie Wilson, another one of the original ARM developers at Bletchley park a couple of weekends ago, another fascinating talk. She now works for Broadcom designing processors for telecommunications).

Yep, its not like computers (2, Interesting)

Anonymous Coward | about 4 years ago | (#32741342)

The brain isn't like computers at all. The brain is compartmentalized. There are dozens of separate pieces each with its specialty. Its wired to other pieces in specific ways. There is no "Total Information Awareness"(tm) bullshit going on (what 1 million cores would give you). The problem with TIA is that there is too much crap to wade through. Too big a haystack to find the needle you need. What they found when analyzing Berger-Liaw speech recognition systems against other systems is that the Berger-Liaw system kept temporal (time-based) subtleties, in contrast to other speech recognition systems that were simply digital (with the clock/oscillator sampling in a Nyquist format, destroying or failing to capture temporal information). The Berger-Liaw system can best the best human listeners (which is why the US navy got it instead of it becoming an available commercial product). It could act as a 'sonic input device' using only a tiny neural network (20 to 30 nodes) for superhuman input, instead of the digital ones, giving crappy results with 2048 or 4096 nodes. The brain is wired with a lot of 'specialty components' which use a spare number of components to get the job done. Some of the excess appears to be redundant (although I am not a neural-scientist and could be wrong).

.... ends up looking pretty silly (1)

Chrisq (894406) | about 4 years ago | (#32741360)

But anyone who argues the brain isn't a pretty spiffy processing system ends up looking pretty silly.

Wouldn't they have just proved the point then. In their case at least......which would mean that they weren't silly after all ..... so their comment would be silly .... which would prove their point.... [stack overflow]

Obviously... (1)

muckracer (1204794) | about 4 years ago | (#32741368)

> What do we do when we have computers with 1 million cores? What about a billion? How about 100 billion? really awesome screensavers!

Where to start? software (2, Interesting)

CBravo (35450) | about 4 years ago | (#32741372)

My opinion is that you should not require software to be parallelized from the start. You parallelize it during runtime or at compile time.

This makes sense because parallelization does not add anything in functionality (the outcome should not change). My point is: program functionality and configure/compile parallelization afterwards (possibly by power-users). There could be a unique selling point for open source: parallel performance because you can recompile.

Re:Where to start? software (1)

dargaud (518470) | about 4 years ago | (#32741568)

My opinion is that you should not require software to be parallelized from the start. You parallelize it during runtime or at compile time.

It certainly is the easiest for the lazy programmer, but certainly not the best way. Using a language that's inherently parallel like Erlang is certainly (one of) the best ways: you don't need to fight with mutexes, semaphores, p-loops as the semantics of the language takes care of it for you. Neat but you need to rethink many processes.

Re:Where to start? software (1)

CBravo (35450) | about 4 years ago | (#32741616)

Rewriting all software in the world using Erlang will not happen. It is not about a rewrite of the world imho, it is about the possible path, starting where we are now, to a new situation.

Re:Where to start? software (1)

selven (1556643) | about 4 years ago | (#32742082)

So the compiler is supposed to guess which parts of the program can be parallelized and which can't be? What if a programmer, not thinking about parallelization at all, accidentally makes his entire program impossible to break up (eg. by using the same variable for loops throughout the entire code)? To do this, the compiler would have to understand the program's intent, and I think we'll get to 1 million processors long before we get that kind of AI.

melissa haro (-1, Offtopic)

tanikaray24 (1845162) | about 4 years ago | (#32741390)

As soon as i saw the headline i couldn't wait to get to start reading the comments. []

The Internet (5, Interesting)

pmontra (738736) | about 4 years ago | (#32741400)

The Internet is at least in the 1 billion cores range. The way to use many of them for a parallel computation has been demonstrated by Seti@home, Folding@home and even by botnets. They might not be the most efficient implementations when you have full control of the cores but they show the way to go when the availability of the cores and the communication between them is unreliable, when they have different times and different clocks and when they might be preempted to do different tasks.

Bye bye Von-neuman (1)

kikito (971480) | about 4 years ago | (#32741500)

With 100+ cores we should start considering leaving Von-neuman behind.

Separated memory/processing/instructions/registers would stop making sense. We would have to follow another model. I don't know which.

Re:Bye bye Von-neuman (1)

TheTurtlesMoves (1442727) | about 4 years ago | (#32741732)

This is the way for a lot of micro controllers and DSPs. Its not all it crack up to be. Pros and cons and all that.

patent-bustering "bad" processors (1)

colordev (1764040) | about 4 years ago | (#32741516)

For possible future patent trolls and to the all reading professional... remember reading the following as "prior art". (kind of like marking bad HDD sectors but instead marking bad processors broken)

TITLE: System, method and device for reliable computing applications

CLAIM 1: Computer device, system and failure eliminating method for computer devices k n o w n for the computer systems ability to evaluate each processor's ability to operate without significant errors and if such errors would be detected the system could be capable for marking such processor units broken and thereby eliminate the future use of such broken processor units withing the computer system.

Special Comment (1)

Xanavi (1197431) | about 4 years ago | (#32741520)

All the human activity thus far, has failed to even begin to match the human brain made by "random chance", if you believe in that malarkey!

The brain isn't a spiffy processing system. (4, Insightful)

master_p (608214) | about 4 years ago | (#32741534)

The brain does not do arithmetic, it only does pattern matching. That's what most people don't get and that's the obstacle to understanding and realizing AI.

If you ask how can humans can then do math in their brain, the answer is simple: they can't, but a pattern matching system can be trained to do math by learning all the relevant patterns.

If you further ask how humans can do logical inference in their brain, the answer is again simple: they can't, and that's the reason people believe in illogical things. Their answers are the result of pattern matching, just like Google returning the wrong results.

Re:The brain isn't a spiffy processing system. (0)

Anonymous Coward | about 4 years ago | (#32741894)

"The brain does not do arithmetic, it only does pattern matching"

Except you are incorrect, math and pattern matching = the same thing.

Re:The brain isn't a spiffy processing system. (0)

perryizgr8 (1370173) | about 4 years ago | (#32742040)

ok, then how can you catch a ball i throw at you? unless your brain is parallely solving quadratic equations, alongwith estimating the result continuously (in parallel to the main calculation), theres no way any human can catch the ball.

What's wrong with the coherent memory model ? (1)

Arlet (29997) | about 4 years ago | (#32741536)

The current model of coherent memory/identical time/everything can route to everywhere just can't scale to machines of this size

Why not ? Obviously, you can't have a million processors accessing the same variable in memory, but with a layered system of caches, you could keep most processors working in their own local copy. As soon as a processor writes to memory that's also used by another process, extra hardware will keep the memory coherent. This architecture is basically a superset of a message passing architecture (memory coherency signals are equivalent to messages), but much simpler for the CPU. Because the CPU isn't aware of the messages, this allows the coherency hardware to be improved without changing the program in any way.

Erlang (1)

ocularsinister (774024) | about 4 years ago | (#32741592)

Erlang [] is probably a good start.

If our brains have 100 billion processing elements (1)

Ihlosi (895663) | about 4 years ago | (#32741826)

... then our processors also have millions already.

A neuron is a fairly simple processing element, after all. Complexity comes from the sheer number of connections with other neurons that a single neuron can have.

Obligatory (1)

Hognoxious (631665) | about 4 years ago | (#32741852)

In Soviet Russia, a Beowulf cluster of these imagines YOU!!!!!

Not a brain (1)

Yvanhoe (564877) | about 4 years ago | (#32741964)

Our brains have billions of neurons, not billions of cores. These are completely different beasts when it comes to architecture.
Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Create a Slashdot Account