×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Auto-threading Compiler Could Restore Moore's Law Gains

Unknown Lamer posted about a year ago | from the sufficiently-smart-compiler dept.

Programming 404

New submitter Nemo the Magnificent writes "Develop in the Cloud has news about what might be a breakthrough out of Microsoft Research. A team there wrote a paper (PDF), now accepted for publication at OOPSLA, that describes how to teach a compiler to auto-thread a program that was written single-threaded in a conventional language like C#. This is the holy grail to take advantage of multiple cores — to get Moore's Law improvements back on track, after they essentially ran aground in the last decade. (Functional programming, the other great hope, just isn't happening.) About 2004 was when Intel et al. ran into a wall and started packing multiple cores into chips instead of cranking the clock speed. The Microsoft team modified a C# compiler to use the new technique, and claim a 'large project at Microsoft' have written 'several million lines of code' testing out the resulting 'safe parallelism.'" The paper is a good read if you're into compilers and functional programming. The key to operation is adding permissions to reference types allowing you to declare normal references, read-only references to mutable objects, references to globally immutable objects, and references to isolated clusters of objects. With that information, the compiler is able to prove that chunks of code can safely be run in parallel. Unlike many other approaches, it doesn't require that your program be purely functional either.

cancel ×
This is a preview of your comment

No Comment Title Entered

Anonymous Coward 1 minute ago

No Comment Entered

404 comments

Fast First Post (-1)

Anonymous Coward | about a year ago | (#42174531)

Does that mean I can get first post even faster?

Re:Fast First Post (-1)

Anonymous Coward | about a year ago | (#42175411)

fuck you Microsoft, no one likes you just give up and die already.

ok now (0)

Anonymous Coward | about a year ago | (#42174535)

now that this has been described, programmers make it so!

Not this shit again (4, Informative)

Anonymous Coward | about a year ago | (#42174551)

Moore's law has nothing to do with performance, numbnuts. Apparently the real geeks left Slashdot ages ago.

Re:Not this shit again (4, Interesting)

xetovss (17621) | about a year ago | (#42174707)

Exactly, Moore's Law isn't a law, it is a marketing plan. I don't see why so many people get so serious about it. A real law (of science) would be something like the law of gravity where it has a near universal application, whereas Moore's Law is a "law" that describes Intel's marketing plan.

Re:Not this shit again (4, Informative)

Baloroth (2370816) | about a year ago | (#42174871)

Moore's law has nothing to do with performance, numbnuts. Apparently the real geeks left Slashdot ages ago.

Yes and no. Moore's law states that the number of transistors will double every 2 years. The problem is that we are nearing the peak of what is possible with current technology in a single core, hence all the focus on 4,6,8, and even 16 core systems for consumers (always been popular in supercomputers and the like). That means doubling transistor count every 2 years can be done through increasing cores... but there is no point in doing that if programs can only use a few of them (very few consumers now need 4 cores, much less 8 or 16).

So, if you can scale up the ability to use processor cores, then Moore's law can continue to hold for CPU's as we increase processor cores. If you can't, then it won't. It'll have to stop sometime, of course, just not necessarily today.

Re:Not this shit again (4, Insightful)

Jonner (189691) | about a year ago | (#42174893)

Moore's law has nothing to do with performance, numbnuts. Apparently the real geeks left Slashdot ages ago.

Try reading the headline again. Usually, clueless posters have to be reminded to read TFA, but this is ridiculous. As Moore's "law" continues to be mostly true, the added transistors are being used for extra more cores rather than to make one core faster. Most of the cores sit idle most of the time because few programs can use more than one of them at once.

Re:Not this shit again (-1)

Anonymous Coward | about a year ago | (#42174931)

Maybe you need to read?

This is the holy grail to take advantage of multiple cores — to get Moore's Law improvements back on track, after they essentially ran aground in the last decade.

Again, Moore's law doesn't have anything to do with performance. It's about the shrinking of transitors.

Re:Not this shit again (5, Insightful)

Spiridios (2406474) | about a year ago | (#42175127)

Maybe you need to read?

Here's the headline (emphasis mine): Auto-threading Compiler Could Restore Moore's Law Gains

In the past, adding more transistors to a core meant the core worked faster (simplification), so Moore's law indirectly lead to better performance. Now a core is close to as fast/efficient as its going to get, so we throw the extra transistors at new cores (still Moore's law). The problem is, there's only so many single-threaded processes a person can run before there will literally be one core per process. In order to see benefits from the extra transistors again, "gains" in the summary's terminology, then we need a way to take advantage of it (this new compiler technique, functional programming, programmers who can actually grok threads). In the end, if we're not seeing some kind of gain from Moore's law, then the chip manufacturers will stop adding new transistors because no one will pay money for a chip that's just as good as their current chip, and Moore's law will fail.

Re:Not this shit again (0)

Anonymous Coward | about a year ago | (#42175063)

For chrissake... relax. We refer to Moore's law, in almost every case, in the context of chip performance, not actual transistor count.

Moore's law is the observation that over the history of computing hardware, the number of transistors on integrated circuits doubles approximately every two years. The period often quoted as "18 months" is due to Intel executive David House, who predicted that period for a doubling in chip performance (being a combination of the effect of more transistors and their being faster).[1]

Obivously we hit the wall on that, and the summary suggests the net result of this auto-threading would put "work done" back on track with predictions of Moore's law. Pull the stick out.

Great potential (1)

Anonymous Coward | about a year ago | (#42174561)

There is great potential here if this sort of thing can be made to work cleanly and safely. I suspect a lot of programmers (myself included) think better in a single-threaded manner. People are mostly used to dealing with A B C D and, as such, I think we usually write code to execute steps the same way. Having the compiler identify things which could run in parallel and thread work to run as A B&C D would be a huge step forward as long as it doesn't introduce bugs.

Re:Great potential (3, Interesting)

MightyMartian (840721) | about a year ago | (#42174643)

I'm wondering how this works. Does it scan for loops to remake them as event driven processes? Does it splice off multiple function calls and then throw the results into some sort of stack for retrieval? It's a cool idea, but man, for any kind of non-trivial program it must be some monumental piece of work.

Re:Great potential (2, Interesting)

Anonymous Coward | about a year ago | (#42174809)

Any code that doesn't access the same resources can be run simultaneously. Alternately, if no thread is writing to a resource, it can be shared without locking.

Those two observations allow you to determine when code can be run together in different threads. If a method only uses 'final' or 'const' things, then you can run it. If your method only uses one isolated cluster of objects, then it can run simultaneously with another method that only uses a different isolated cluster of objects.

Honestly, any programmer worth $120k could make something more efficient just using threads, if they thought about it. This will get you 20% of the efficiency gains without thinking of it.

Re:Great potential (3, Insightful)

elfprince13 (1521333) | about a year ago | (#42175407)

One problem, and the great attraction of this way of doing things, is that most software companies don't have too many programmers worth $120k. Lots of $50k-$80k, but not so much on the higher end of the pay scale until you get into research divisions at big companies like Microsoft or Google or IBM.

Re:Great potential (5, Insightful)

allcoolnameswheretak (1102727) | about a year ago | (#42174935)

Imagine you have a for-loop that calls the method 'hop' on every object 'bunny' in the list:

for every bunny in list {
        bunny.hop()
}

This is a simple, sequential process - bunny.hop() is called in order, for every bunny in the list, one after the other.
Now suppose you have defined the data in your program in such a way that the compiler knows that method 'bunny.hop()' only ever accesses read-only data, or modifies local data that is unaccessible from anywhere else. The compiler now knows that the order of execution of the bunny hops doesn't really matter, as every call of bunny.hop() is independent from anything else. This frees the compiler to spawn threads or processes to call as many bunny.hop()'s as he likes at the same time, thereby processing through the list alot faster.

Another method, bunny.eat() actually performs write access on a shared object 'carrot' when called. If two bunnies eat the same carrot, the compiler can not perform automatic parallelization, as running two bunny.eat() methods could lead to invalid state (only one piece of carrot remaining, two bunnies eating 1 piece of carrot at the same time, results in -1 pieces of carrot). In this case, the compiler will take care to run two bunnies eating away at the same carrot sequentially. However if there are 2 bunnies eating the green carrot and another 2 bunnies eating the yellow carrot, these are again independent from each other and can again be paralleled.

The requirement to make this possible is to provide the compiler with information on what kind of data something is - is it an isolated hopping? Or a shared carrot? or a global Bunnygod that affects all bunnies?

Re:Great potential (5, Funny)

Boawk (525582) | about a year ago | (#42175121)

If two bunnies eat the same carrot...the compiler will take care to run two bunnies eating away at the same carrot sequentially. However if there are 2 bunnies eating the green carrot and another 2 bunnies eating the yellow carrot

Am I the only one who is totally horny after reading that?

Re:Great potential (0)

Anonymous Coward | about a year ago | (#42175329)

No, you are not. I wish it was implemented in GCC.

Re:Great potential (2, Interesting)

Culture20 (968837) | about a year ago | (#42175237)

Imagine you have a for-loop that calls the method 'hop' on every object 'bunny' in the list:

for every bunny in list {
bunny.hop()
}

And if the original intent is to make the bunnies hop in sequence like performing "the wave", making a new thread for each hop might produce a visual effect of them all hopping at once if the sleep occurred in the hop() function instead of the for loop calling the hops.

Re:Great potential (4, Insightful)

paskie (539112) | about a year ago | (#42175309)

One would assume that "world modification" (monad; in this case, visual I/O) does not fit the restriction "only ever accesses read-only data, or modifies local data that is unaccessible from anywhere else" and as soon as you are showing something to the user, that blocks the auto-threading.

Then again, if you are drawing map tiles, the order you draw them in doesn't matter - so this goes to show that in some situations, there's no way around the programmer actually having to stop, think and declare whether the program's contract allows it to do some user-visible operation in parallel.

Re:Great potential (0)

Anonymous Coward | about a year ago | (#42175443)

Did you ever think to read the paper that's linked in the article? It goes over in some depth how it works.
http://research.microsoft.com/pubs/170528/msr-tr-2012-79.pdf

Re:Great potential (5, Informative)

Desler (1608317) | about a year ago | (#42174667)

Having the compiler identify things which could run in parallel and thread work to run as A B&C D would be a huge step forward as long as it doesn't introduce bugs.

Compilers already try to do this for example with auto-vectorization. The problem is that they are usually quite terrible at it. Even Intel's compiler which is probably the best at it usually misses out on most of the obvious places that should be vectorized. This is why pretty much all code dealing with multimedia content (audio/video/image codecs, games, etc.) still rely on tons of had written SIMD to be optimized to their fullest.

Meh. Not that big a problem. (4, Insightful)

fyngyrz (762201) | about a year ago | (#42175075)

Honestly, you know what this says? This just says some programmers still need to go practice in the threading space.

Threading is not that hard at all for a VERY large class of problems. There are issues -- like, the memory bandwidth will eventually choke the ability of processors to get at nearby memory regions -- and you need some kind of sane plan to deal with cacheing in one core as compared to the next core when you share flags or data coming in during execution from elsewhere -- and you need to figure out when thread overhead starts to chew into your efficiencies -- but really, it's entirely doable, and isn't that difficult a problem space as compared to the actual problems *serious* programmers have to solve. It may just be that your "thing" isn't really all that threadable. Some things aren't. But it should also be very obvious when they aren't.

Then again, some problem spaces lend themselves very well to multiple core approaches -- I just finished a library that slices up images and then applies various separable algorithms to the slices in a variable number of threads (the user of the library can specify.) I wrote it in pure C, used posix threads, no sweat at all, in my 8-core machine, you get just about exactly the gains you'd think (x the number of cores) when the process is CPU cycle intensive, less as the ops are simpler and memory I/O rates rise.

Seriously. If threading seems that hard, you need to go do some studying. It's not the threading. It's you. You really should get on this bus.

Re:Meh. Not that big a problem. (2)

Desler (1608317) | about a year ago | (#42175099)

You seem to have replied to the wrong person as I'm not seeing what your post has to do with mine.

Re:Meh. Not that big a problem. (5, Funny)

rk (6314) | about a year ago | (#42175247)

Maybe it's subtle sarcasm regarding learning threaded programming? People can't identify the right thread in a conversation, but should be expected to write threaded code? :-)

Re:NO potential (-1)

Anonymous Coward | about a year ago | (#42175105)

IBM already tried this approach, I think with Carnegie-Mellon about 7 years ago, and didn't get too far with the idea so more M$ innovation?

MFG, omb

Re:NO potential (1)

Eskarel (565631) | about a year ago | (#42175231)

Innovation doesn't mean "doing something no one has ever thought of before" it means "doing something no one has done before". If Microsoft's technique works where Carnegie-Mellon's didn't then it's innovative, if it doesn't, it probably isn't. There could still be something innovative in the way they tried which could help someone else later, but mostly what we'd be looking for from an innovation point of view is "actually does what it says in the tin".

Nothing new here (4, Informative)

stevel (64802) | about a year ago | (#42175213)

This is nothing new. As in decades-old. Back when I was at DEC in the 1980s we had auto-threading compilers that worked very well on standard application code. Today, Intel has been doing auto-threading in its C++ and Fortran compilers for more than a decade and it is very effective for a large class of applications. Intel's compiler also has features that help you tweak the code for better auto-parallelism, notably the Guided Auto Parallel feature introduced in 2010. Other compiler vendors also have auto-parallelizing compilers.

I've been in the industry for decades, and every couple of years someone comes out with yet another programming language or technique that is supposed to revolutionize application parallelization. This is just another one, and many years behind the rest of the industry from what I can tell.

Re:Great potential (5, Interesting)

Jeremi (14640) | about a year ago | (#42175313)

Compilers already try to do this for example with auto-vectorization. The problem is that they are usually quite terrible at it.

I suspect one reason they are so bad at it is they have to be very conservative in how they optimize, due to the relaxed nature of the C language. For example, if the C optimizer cannot prove beyond a shadow of a doubt that a particular memory location isn't being aliased, it can't make any assumptions about the location's value not changing at any step of the process. In practice, that means no optimizations for you.

Given that, it would seem that the Microsoft approach (using not only the higher-level language C#, but a specially customized version of C#) gives their optimizer much greater latitude. Because the language forces the programmer to annotate his objects with readable/writable/immutable tags (think C's "const" tag, but with teeth), and because the language (presumably) doesn't allow the programmer to do sneaky low-level tricks like casting away const or aliasing pointers or pointer math, the optimizer can safely make assumptions about the code that a C or C++ optimizer could never get away with. That may allow it to be more effective than you might anticipate (or maybe not, we'll see).

Re:Great potential (2)

Kjella (173770) | about a year ago | (#42175275)

There is great potential here if this sort of thing can be made to work cleanly and safely. I suspect a lot of programmers (myself included) think better in a single-threaded manner. People are mostly used to dealing with A B C D and, as such, I think we usually write code to execute steps the same way.

If you've ever done a dish with more than one casserole or making a side dish while the main dish is cooking you've done parallelization in real life, the problem is that expressing it in a computer language is hard. Functional programming works okay if you're doing lots of similar things at the same time, but not if you're doing lots of different - but parallelizable - tasks. I want to call oven.roast( meat ), stove.boil( potatoes ) and workbench.slice( salad ) but I don't particularly care about the order and I know they're side effect free relative to each other, but how do I express that? Either I write them in sequence ABC and they happen in sequence ABC or I have to write threads and schedule them which is honestly overkill - I want it to use the resources efficiently not micromanage the schedule myself.

Particularly if these come in random and unpredictable amounts like the orders at a restaurant I need the computer to work it out itself. It's also very important if you have small resource conflicts like the hot casserole and hot pot roast both requiring mittens to move. I know quite a lot about mutexes and locks and semaphores to make it explicit but now it feels like I'm explicitly describing the opportunities for parallelism not the dependencies preventing it. Imagine if you were a project manager, do you start with every task depending on every other task? No, you start with the tasks then make dependencies where they're needed.

I wonder if you could do something more with "scoped const-ness", for lack of a better term. That is to say a function is non-const in a certain scope and const outside it, and thus can be executed in parallel with anything else outside that scope. To continue the example above, let's say I could define a function oven.raiseTemperature() that's non-const in the oven scope and const in the global scope and a function stove.stirPot() that is the same for the stove. If I then call them the compiler would know it's free to reorder them or run them in parallel, it picks the optimal execution based on the hardware available or even dynamically at run time. Functions that depend on multiple objects would automatically become fences to "sync up", though not necessarily on a global scale. Like a function on both the oven and stove may sync the kitchen, but not the living room.

It might not give you massive parallelization but it'd at least let you parallelize many mixed tasks that I don't feel you manage to do easily in any language today. It certainly sounds like a safer way than starting off a bunch of worker threads and praying you didn't create any deadlocks or live locks or race conditions or whatnot. Much like the global const the compiler should be able to validate that it doesn't in fact have any side effects outside the scope it claims. I'm sure there's lots of reasons why this wouldn't be as useful as I imagine it would be, but it certainly seems easier for those of us used to imperative programming.

curtailing of Moore's Law is not the problem (-1)

Anonymous Coward | about a year ago | (#42174567)

The problem is the bottleneck accessing secondary storage.

I hate when people misuse Moore's law (2, Informative)

rminsk (831757) | about a year ago | (#42174571)

Moore's law is only about the number of transistors on integrated circuits.

Re:I hate when people misuse Moore's law (0)

Anonymous Coward | about a year ago | (#42174597)

it is now about intel cpu naming scheme...

Re:I hate when people misuse Moore's law (4, Interesting)

oodaloop (1229816) | about a year ago | (#42174611)

It turns out that Moore was observing only a small portion of a greater trend. That is, many (or most) things involving technology have a short doubling period, usually between 1 and 2 years. Total information created, processing power per dollar, resolution of brain scanning technology, et al all follow a similar doubling period to what Moore observed with transistors on integrated circuits. It's not that everyone else is misusing Moore's Law, so much as Moore didn't make make the law broad enough.

Re:I hate when people misuse Moore's law (3, Informative)

Anonymous Coward | about a year ago | (#42174671)

You do realize that all the things you listed are a side effect of the doubling of transistor density - and not some separate independent phenomenon?

Re:I hate when people misuse Moore's law (2)

oodaloop (1229816) | about a year ago | (#42174681)

Does that include the doubling of processing power before the transistor was invented?

Re:I hate when people misuse Moore's law (4, Interesting)

loneDreamer (1502073) | about a year ago | (#42175065)

Magnetic Hard drives, for instance is an example of the same curve with no transistors.

Re:I hate when people misuse Moore's law (5, Funny)

Anonymous Coward | about a year ago | (#42174615)

No no, you misunderstand. They're teaching the compiler to be so efficient that it actually starts creating new transistors to make itself smarter.

Make no mistake, this is how the rise of the machines will start.

Re:I hate when people misuse Moore's law (4, Insightful)

timeOday (582209) | about a year ago | (#42174835)

I hate it when all the potentially interesting discussion on a slashdot story is headed off by a pedantic nitpick.

Re:I hate when people misuse Moore's law (0)

Anonymous Coward | about a year ago | (#42174911)

I hate it when all the potentially interesting discussion on a slashdot story is headed off by a pedantic nitprick.

FTFY

How about (0)

Anonymous Coward | about a year ago | (#42174627)

How about using threads when it makes sense, and not when it doesn't? It doesn't matter how many cores, how many threads, and how many gigahertz you have when the computer is doing nothing but blinking a cursor waiting for me to press the keyboard. The vast majority of programs match this situation, and all the magic compiler is going to do is increase the speed of the System Idle Thread.

Re:How about (5, Funny)

chromas (1085949) | about a year ago | (#42174833)

all the magic compiler is going to do is increase the speed of the System Idle Thread.

Have you seen how much processing power that thing consumes? Usually 90% and up. On all cores. It really needs some serious optimization.

Or.. teach devs to use threading as appropriate? (4, Informative)

databeast (19718) | about a year ago | (#42174639)

Or, gawd forbid.. we could teach programmers how to use threading? I am a casual developer, with no formal training beyond writing practical code and reading as much as I can from the keyboards of real developers. I've run into my fair share of "real", "professional" developers who've been tasked to work on my code and thrown their hands up in defeat - not, as I feared, because of the awfulness of my coding style, but "This uses threading, I don't know this!".. Of course, looking at their resumes, a long list of single threaded webapps where the actual threading is handled for them by the webserver itself.. I give up. Even some basic native GUI client development should teach developers simple threading and asynchronous callbacks? alas, yet another talent abandoned in the age of the webapp. Is it any wonder the security issues (my actual realm of supposed 'expertise') in their code often make the architectural issues look trivial in comparison?

An interesting development, and much needed I fear, but yet another layer of abstraction to allow lazy developers to not have to really bother about knowing what their code is actually doing (that's for the poor SoB who has to maintain it is for...)

Re:Or.. teach devs to use threading as appropriate (4, Insightful)

lennier (44736) | about a year ago | (#42174781)

Or, gawd forbid.. we could teach programmers how to use threading?

That's easy: "Don't."

From everything I've read about threading, there's no general way a hand-coded multithreaded program can ever be proven to be correct. Threads introduce all sorts of extremely subtle timing-based logic and security bugs which even the smartest programmers in the world think they can handle but in practice don't. And most programmers are not the smartest programmers in the world (they only think they are).

The correct solution is to switch from C-based imperative languages to pure-functional implicitly parallelised languages, but that's not likely to happen before the heat death of the universe.

Re:Or.. teach devs to use threading as appropriate (0)

HornWumpus (783565) | about a year ago | (#42174837)

No non-trivial program can ever be proven to be correct.

Re:Or.. teach devs to use threading as appropriate (2)

Intropy (2009018) | about a year ago | (#42174923)

This is totally true.

So long as you define trivial programs to be a subset of programs that can be proven correct.

Re:Or.. teach devs to use threading as appropriate (0)

Anonymous Coward | about a year ago | (#42174845)

The researchers are proposing neither solution, and I think they are right.

Explicit threading is hard to get right and the bugs are insane. The typical proposed solution, "use a functional programming language and abandon all state", is like using sarin on ants. Effective, and pretty neurotoxic.

The researchers propose that humans annotate languages with information about intended uses of variables which is not too difficult to infer in the cognitive model of the programmers, and then let the the compiler take care of the coordination of the threads. As it should be.

Re:Or.. teach devs to use threading as appropriate (1)

Desler (1608317) | about a year ago | (#42174867)

The correct solution is to switch from C-based imperative languages to pure-functional implicitly parallelised languages, but that's not likely to happen before the heat death of the universe.

Well, yes. Are you going to be the one to pay the 10s of billions of dollars, if not 100s of billions, to rewrite all software in existence into a functional language?

Re:Or.. teach devs to use threading as appropriate (2)

betterunixthanunix (980855) | about a year ago | (#42174995)

How about we write new code in a functional language, instead of clinging to imperative languages? Put C, C++, and Java in the category they belong in: maintenance and legacy code only.

Re:Or.. teach devs to use threading as appropriate (3, Insightful)

SplashMyBandit (1543257) | about a year ago | (#42175215)

Actually you don't need functional. Java works fine with multi-threading (well, it does for my daily use). In fact, that is the real strength of the garbage collection model of Java, it can track and manage efficient release of resources in a multi-threaded system. Also Java has direct language support for synchronization mechanisms and extensive library support for multi-thread friendly data structures (Doug Leah is a legend!). Lots and lots of multi-threaded programs get written in Java, without needing functional languages, it's just that we we do write them they do work (after we test for various possible thread-related problems), work well, and we don't have to bitch about needing to change paradigms to get things going (eg. needing to change to functional languages when they are not always the best fit for many classes of problem).

Of course you'll hear those still stuck on legacy languages designed without multi-threading in mind whinging. That doesn't mean there isn't a straightforward, easy to learn language and development platform out there that isn't strong on multi-threading. There is - and it is cross-platform and general purpose too.

Re:Or.. teach devs to use threading as appropriate (1)

betterunixthanunix (980855) | about a year ago | (#42175485)

Why Java, though? What does the Java language have that other JVM languages do not? As a language, Java is pretty awful; you are stuck with a single programming paradigm even in cases where that paradigm makes no sense, you are writing enormous amounts of code to express what you are trying to do, and features that programmers in other languages take for granted are just not there (or are very limited). Meanwhile, you have Scala and Clojure on the JVM -- so why not just use these where functional approaches make sense, like for parallel algorithms?

easy to learn language

Having taught Java to novices at one time, I can tell you that this is not true. Here is a very small example of why Java is harder to learn than other languages:

someObj.someMethod(if(x == 2) { 5; } else { 10; });

Yes, they should have used the ternary operator. Why does Java even separate these? Why not just allow conditional statements to appear everything, which is what you get in Clojure (and any other Lisp, and many other programming languages that use reduction semantics)? This sort of rigid syntax makes the task of learning Java needlessly hard, because learning Java means learning how to think in a very different, very unnatural way.

Re:Or.. teach devs to use threading as appropriate (1)

Nerdfest (867930) | about a year ago | (#42174903)

There are plenty of threading frameworks in most languages where you can just define threadable operations. Your job is simply to ensure the task is correct, use the right framework and trust it to the correct degree. As with many things, someone only needs to do it right once.

Re:Or.. teach devs to use threading as appropriate (1)

doublebackslash (702979) | about a year ago | (#42175043)

It is possible to prove a multi-threaded program correct.

First you might start at the memory model, and all the guarantees that one can make based on that (and, by association, all the guarantees in terms of the memory model that locking / atomic calls make), then one can move on to he library structures and routines (objects and methods to put it in another paradigm).

Once you have hard guarantees using primitives and low level constructs it might be easy to construct a state-based proof. One example is Cliff Click's lock free hashtable (don't know ifhe has a formal proof, but nearly so if not)

Correctness in multithreaded environments takes a different form than in a single thread, but it is nothing that cannot be managed. Generally there are only a few "tough locks" to crack and the majority of the speedup can be had. Some problems are harder than others, and some problems don't have great parallelism, but that is just a generalization of programming, really. "Some things are harder than others, but nothing doable is impossible"

Re:Or.. teach devs to use threading as appropriate (1)

timeOday (582209) | about a year ago | (#42174803)

Goto is swell also. Just be sure not to make any mistakes!

Re:Or.. teach devs to use threading as appropriate (1)

Desler (1608317) | about a year ago | (#42174887)

I think the point is that just because something is hard is no reason not to teach it to people. If no one knew every learns these low-level details who is going to be maintaining the compilers?

Re:Or.. teach devs to use threading as appropriate (2)

databeast (19718) | about a year ago | (#42174927)

Bingo.. and as many other folks have pointed out, 90% of of threading is purely to decrease latency and bypass blocking operations - very few applications out there today are heavily dependent on concurrency to achieve any kind of raw horsepower... This does just seem like, if it were to become mainstream, would just become a crutch for developers to ignore threading and blocking I/O issues entirely, because "The compiler will sort that out".. In theory, this isn't necessarily a bad thing. In theory, all things work in practice too.

Re:Or.. teach devs to use threading as appropriate (1)

doublebackslash (702979) | about a year ago | (#42174865)

I wrote up a very long comment that ranted and raved and didn't come close to this comment right here. Some threading problems are hard, really hard. They aren't that common.

Man up and learn. I like it.

Re:Or.. teach devs to use threading as appropriate (2)

betterunixthanunix (980855) | about a year ago | (#42174925)

Or, gawd forbid.. we could teach programmers how to use threading?

No, we want our compilers to do these things for us, because things that compilers can do they usually do better than humans can. Once your programs become large enough and complex enough, compilers outperform humans every time -- it's not about laziness, it is about the limits of human programming ability.

Compilers surpassed humans when it comes to optimization a very long time ago, except for very small programs or very short inner loop bodies.

Re:Or.. teach devs to use threading as appropriate (1, Informative)

Desler (1608317) | about a year ago | (#42175011)

Compilers surpassed humans when it comes to optimization a very long time ago, except for very small programs or very short inner loop bodies.

Hahah, no they didn't. If this were true anything multimedia-related would not still require tons of hand-optimized SIMD code. Even the best of C or C++ compilers are terrible at vectorization of code. Just compile something like x264 with the best C compiler you can find and compare it to the assembly-optimized version for, say, the latest i7 processors. The difference in speed will be quite huge.

Re:Or.. teach devs to use threading as appropriate (0)

Desler (1608317) | about a year ago | (#42175123)

To further add, this is doubly compounded when running such things on, say, ARM processors. Anything multimedia related that doesn't have NEON SIMD or isn't using some form of hardware acceleration will run terribly slow relying only on compiler optimization.

Re:Or.. teach devs to use threading as appropriate (5, Insightful)

betterunixthanunix (980855) | about a year ago | (#42175293)

Let me repeat myself: except for very small programs or very short inner loop bodies. Most of what you are describing is pretty small in terms of the amount of code and its complexity; hand-optimizing assembly code does not scale up.

Even the best of C or C++ compilers are terrible at vectorization of code.

Yeah, and the best humans are terrible at allocating registers -- so bad, in fact, that the best C compilers ignore the register keyword. What do you think is more relevant to the general case: vectorizing multimedia operations, or allocating registers? Compilers are also better than humans at:

  • Finding redundant or unreachable code
  • Finding dead or otherwise unneeded variables
  • Finding algebraic reductions
  • Strength reducing transformations
  • Boolean optimizations
  • Reducing program size

...and numerous other optimizations that, like register allocation, are more generally applicable than SIMD. There has also been work on compiler optimizations that are utterly out of reach for even the most skilled humans, like probabilistic optimizations that have a negligible (with respect to some tunable parameter) probability of being unsound.

To put it another way, look at the Orbitz story, which is over a decade old now:

http://www.paulgraham.com/carl.html [paulgraham.com]

On the one hand, you have hand-tuned assembly language. On the other, you have a program written in Lisp (a high level, compiled language) with some C++ mixed in (for managing memory). Orbitz was able to compete on speed, but more importantly, it was returning better results. It's not that the people who wrote that mainframe assembly language were idiots -- they were taking advantage of all sorts of special hardware features, they knew how to hack their machines better than anyone else -- it is just that the Orbitz algorithm was far too complex for efficient hand-rolled assembly code, at which point compilers are really the only choice. The mainframe guys were busy thinking about how to make use of special machine features in assembly language; the ITA team was busy solving the higher-level problem, and relying on their compiler to generate good assembly language code. This is a particularly telling line:

We disassemble most every Lisp function looking for inefficiencies and have had both CMUCL and Franz enhanced to compile our code better.

[emphasis mine]. They disassembled their code...and then improved their compiler when they saw problems. They did not hand-roll the code, they made the compiler do a better job of generating code. These are not lazy programmers, nor are they programmers who do not know how to use assembly language; they are programmers who understand that they have a tool that is far better at generating assembly language than they are, and that they have more important things to do with their time.

I deal with quite a bit of crypto code in my work. I have seen lots of hand-tuned assembly language, I dealt with code that took advantage of the AESNI instructions to perform very fast encryption. I am well aware that in small, highly specialized functions (like AES), humans are better able to utilize special instructions to improve performance. Those are niche cases, and the techniques used in those cases have very limited applicability (even SSE is fairly limited in its applicability, by comparison with the sort of code programmers write and maintain every day), and the techniques scale very poorly.

Re:Or.. teach devs to use threading as appropriate (1)

doublebackslash (702979) | about a year ago | (#42175135)

That is because of knowledge of processor details, memory details, comple operations, and, well, they [compilers] are better than us. Except that the ability to optimise depends on pure logic, of some form. As the state gets large optimization gets more complex.

Just like quicksort(3) is far faster than bubblesort so too is a highly threadable code faster than non-threadble code. Languages do not, contrary to belief, express intent, the provide a strict set of instructions that the computer MUST respect. In the end a good algorithm with no compiler help will beat optimized "dumb" code in all cases larger than "toy" (say, a few dozen "n" in Big-O notation)

Good thought about opimizing compilers, but it doesn't bear out.

Re:Or.. teach devs to use threading as appropriate (5, Insightful)

betterunixthanunix (980855) | about a year ago | (#42175437)

Just like quicksort(3) is far faster than bubblesort so too is a highly threadable code faster than non-threadble code

First, just to be pedantic, I'll point out that quicksort is as bad as bubblesort in the worst case, to a constant factor (you should have picked heapsort or mergesort). That aside, it is worth noting (and I am somewhat bothered by this when it comes to TFA) that we still do not know if it is even possible to optimize any program by parallelizing it; see the NC-vs-P question:

https://en.wikipedia.org/wiki/P-complete [wikipedia.org]

Multithreading is not a magic bullet, and in all likelihood it is not generally applicable.

Languages do not, contrary to belief, express intent, the provide a strict set of instructions that the computer MUST respect

Wrong on all counts. Imperative languages are a way to convey instructions to a computer; declarative languages do not convey instructions, and programming in a declarative language requires an entirely different mode of thinking (it is closer to asking a question that giving instructions). It is also not strictly necessary for the computer to do exactly what a program expresses; there has been some work on compiler optimizations that have a (tunable) chance of not maintaining soundness.

In the end a good algorithm with no compiler help will beat optimized "dumb" code in all cases larger than "toy" (say, a few dozen "n" in Big-O notation)

If you are convinced of this, try implementing something more complex than the algorithms you see in Knuth's books; say, this:

http://eurocrypt2010rump.cr.yp.to/9854ad3cab48983f7c2c5a2258e27717.pdf [cr.yp.to]

Then ask yourself this: could the constant factors in your implementation be better? At the end of the day, big constant factors will hurt your performance so badly that you might as well have used an asymptotically worse algorithm; indeed, consider fast integer multiplication:

https://en.wikipedia.org/wiki/Sch%C3%B6nhage%E2%80%93Strassen_algorithm [wikipedia.org]

10000 digits are needed before that algorithm actually outperforms the asymptotically worse Toom-Cook family of algorithms. Here is an even more extreme example:

https://en.wikipedia.org/wiki/Coppersmith%E2%80%93Winograd_algorithm [wikipedia.org]

Sure, that's a better matrix multiplication algorithm in the asymptotic sense...but only for matrices that are so large that you cannot even store them on today's computers.

So really, while you are right that asymptotic improvements will always beat constant factor improvements (which is what compilers are mostly going to do for you), you are wrong to ignore the importance of constant factor improvements. In the real world, constant factors matter. In the real world, quicksort and mergesort will use asymptotically worse algorithms below a certain problem size because of constant factors. In the real world, large integer multiplication is done using Karatsuba or Toom-Cook methods, not FFT methods, because of constant factors. In the real world, if you are not using a compiler to optimize your code, your code is going to be slower than it needs to be, even if you spent hours hand-tuning it, unless you are only dealing with toy problems.

Re:Or.. teach devs to use threading as appropriate (1)

Hentes (2461350) | about a year ago | (#42174991)

Most parallel problems tend to fall into a small set of categories, and I see no problem with abstracting them away. There are many libraries already that handle the threading for you.

Re:Or.. teach devs to use threading as appropriate (5, Insightful)

Jonner (189691) | about a year ago | (#42175001)

An interesting development, and much needed I fear, but yet another layer of abstraction to allow lazy developers to not have to really bother about knowing what their code is actually doing (that's for the poor SoB who has to maintain it is for...)

Developing software is all about managing complexity. Abstractions are the primary tool used to do so. They are neither inherently good or bad. A large part of writing good software is finding the appropriate abstractions and eliminating inappropriate ones. If an abstraction allows a program to be written more easily with an acceptible level of performance and correctness, it is an appropriate abstraction.

To know what code is "actually doing" is relative. No programmer knows what his code is doing at the level of gates on the chip. It's rarely necessary or even helpful to know what the code is doing at the level of CPU instructions or microinstructions. This is especially true if the code runs on multiple CPUs.

Re:Or.. teach devs to use threading as appropriate (0)

Anonymous Coward | about a year ago | (#42175017)

And what in the research paper itself would you dislike, then?

In particular, what do you dislike, more than all the "layers of abstraction" a compiler already adds between you and assembly-level management of code and machine state?

As far as I'm concerned, FP / immutability *is* the thing programmers should learn to describe, not the threading details themselves. Describing immutability is more profound than the actual threading details. If you want to execute code out of order in sequence (=multi-threaded) and have a predictable result, it needs to be composed only of commutative operations, and operating only on immutable values (at least for the section that is executed in parallel, rather than in sequence). Threading doesn't allow you to get around that, either way. And people already largely know how to describe operations (it's up to the library developers to decompose as much of the basic ones into commutative ones, though). In my opinion, the main task is to make people work with immutable values, nothing else.
 

Re:Or.. teach devs to use threading as appropriate (0)

Anonymous Coward | about a year ago | (#42175125)

Threading as it is today is inherently non-deterministic, which makes isolating and fixing threading bugs very difficult. It doesn't matter how comfortable you are in writing threaded applications. At some point you -will- introduce thread-related bugs in your application. In fact many threaded apps run for years with threading bugs lying in wait and they aren't triggered until some later change alters conditions just enough for a race condition or other problem to be exposed. Those dormant bugs are a SERIOUS PITA to debug and fix because you look through recent changes in your version control and everything looks innocuous.

Anything that can allow a compiler to prove correctness of concurrent execution is a welcome improvement. I'm all for additional education, but I wouldn't poo-poo this type of advance in compiler technology.

Re:Or.. teach devs to use threading as appropriate (1)

shutdown -p now (807394) | about a year ago | (#42175477)

"Using threading" as in spinning off a thread or two is not hard to teach. However, even when people know to use threads, and, more importantly, know how to use them right (avoiding race conditions etc), they are still not necessarily optimal at determining when to fork, and how many threads to create at each fork point. A compiler doing analysis of the entire code base to see what depends on what can come up with more opportunities to parallelize, some in very non-obvious places. This isn't really any different from many compiler optimizations that you see in today's C++ compilers - some of them are very non-obvious, when you look at the generated assembly.

Anyway, if you want to do "threading for the masses", you need to add another level of abstraction on top - tasks/futures, and chains thereof. Like Apple's GCD, or .NET TPL. It's much easier to reason in terms of forking and joining tasks than it is in terms of threads and synchronization primitives. Also, you can do nifty syntactic sugar on top of tasks (see C# async/await for an example) without hiding all their flexibility.

Parallelizing is easy, performance is hard (4, Insightful)

HuguesT (84078) | about a year ago | (#42174651)

OK, so now we shall have another way to produce parallel programs.

Running things safely in parallel is a very good thing, but it does not by itself guarantee significant improvements in speed. The hard bit is actually to optimize the parallel code [futurechips.org].

Re:Parallelizing is easy, performance is hard (0)

Anonymous Coward | about a year ago | (#42174777)

Quite so; this is presumably sensationalized by someone who's never heard of automated parallelization before.

As far as I can tell, the authors are simply introducing semaphore abstraction, which is pretty much the lowest hanging fruit and still requires very careful consideration by the programmer for significant general purpose improvement over serial. There are a variety of much harder problems (cache blocking, dependency analysis/transformation, contraction, partitioning, ect.) that, while being studied, have seen extremely incremental improvements and are very far from seeing practical application.

Wikipedia has a surprisingly good list of the automated tools in development - http://en.wikipedia.org/wiki/Automatic_parallelization_tool

Re:Parallelizing is easy, performance is hard (1)

godrik (1287354) | about a year ago | (#42175315)

Exactly, I am doing parallel programming and HPC for a leaving. I do not believe in compiler automagic^Wautomatic parallelisation. Typing is only a small parts of the problem. Most algorithms need to be written VERY differently to be able to exploit the parallelism within. Some times, you actually need to have variable concurrency and race condition within your algorithm but you have ways to fix problems whenever they appear.

In brief, I do not think that compilers will be able to do anything more than quite basic stuff. What ever is proposed here do not appear to me to be significantly more powerful than Cilk (now Intel Cilk+).

Re:Parallelizing is easy, performance is hard (0)

Anonymous Coward | about a year ago | (#42175367)

The funny thing is, the compiler is able to do the optimizations you're linking to. If you actually read the academic paper behind the blog post, it show's you how it avoids resource contention by tracking mutability. The strategy is to remove resources to contend with - the compiler only splits to multiple threads when it knows a resource contention would be impossible. From you're link, I'm sure you think the compiler is auto-generating locks & synchronization blocks (which slow down execution) - that's simply not true. The compiler that Microsoft wrote generates lock-less multi-threaded code.

mutable state (4, Informative)

jbolden (176878) | about a year ago | (#42174703)

Mainstream language have mutable state all over the code. Functional programming's big change on state issues is to careful isolate state. The Microsoft approach means that state needs to be tracked carefully so that it could be isolated by the compiler even if it isn't isolated by the code. Which is likely just as much work as isolating state. And the nice thing about isolating state is once you do it you can make use of all sorts of incredibly powerful functional paradigms like like first class functions (closures, partial execution...) and lazyness (infinite data structures, no need to figure out proper order of evaluation..)

The solution to parallelism is functional programming. And no it is not too hard. Excel is a functional programming language that lots of people know that does a great job isolating state.

Re:mutable state (2)

readin (838620) | about a year ago | (#42174953)

I wish I knew more about functional programming than I do. In reading the article I found the concept of a language the limits mutable variables interesting. In using Java I find that making most variables "final" is helpful to me for a number of reasons. I can easily find where the variable got its current value. If I write a line of code that changes the wrong variable it is immediately obvious. It keeps me from re-using variables that shouldn't be re-used (generally a variable should have one meaning and one value). It helps me keep variable scope narrow.

Is this article suggesting that most functional languages make their variables the similar the Java "final" or am I way way off?

Re:mutable state (3, Interesting)

betterunixthanunix (980855) | about a year ago | (#42175113)

In functional algorithms and data structures, everything is immutable (in theory) -- rather than thinking in terms of "final," think in terms of Java's String class. If you want to change one character in a String instance, you must create a new String instance. For a less trivial example, consider how a functional algorithm that removes an element from a list would work (written in Java-like syntax):

List remove_if(List lst, Predicate pred)
{
if(lst == null)
{
return null
}
else if(pred.test(lst.first()))
{
return remove_if(lst.rest());
}
else
{
return new List(lst.first(), remove_if(lst.rest()));
}
}

Notice that this constructs an entirely new list, even if none of the elements in the list pass the test. This may seem like a terrible idea, but let's put it this way: if you have 10 threads that share the list, and one of them wants to remove some nodes, you would have had to have copied the list anyway; the functional approach to remove_if is exactly what you want. Now, consider this function, which only removes the first node to match:

List remove_first(List lst, Predicate pred)
{
if(lst == null)
{
return null;
}
else if(pred.test(lst.first))
{
return lst.rest();
}
else
{
return new List(lst.first(), remove_first(lst.rest()));
}
}

Now you have a situation where lists share nodes -- and again, imagine a situation where 10 threads share the list, and one wants to perform this operation. This is one of the reasons that functional programming is so promising for parallel algorithms: you have fewer situations where explicit mutexes are needed, because you are usually copying things that you intend to modify (or more precisely, you never really modify anything).

Of course, things are more complicated in the real world. Purely functional approaches would obviously be pretty inefficient in a lot of cases, since things would be needlessly copied. Lisp, as an example, has both a non-destructive append as well as a destructive nconc, the latter being intended for use in situations where the original lists will not be used again (and can therefore be modified).

Re:mutable state (1)

segfault_0 (181690) | about a year ago | (#42175245)

The benefits of using functional languages is realized in terms of program safety and a lack of defects -- not necessarily performance. I think we are all aware of the fact that there are plenty of programmers out there who care very little about introducing a few bugs for the sake of speed and looking clever.

But even if you just use immutable data and message passing as IPC under a procedural language you are headed in the right direction. It's really a mindset and even the procedural programming folks don't really have many excuses for not doing what the functional languages make natural.

Auto-Threading (2)

DaneM (810927) | about a year ago | (#42174733)

About time.

Re:Auto-Threading (0)

Anonymous Coward | about a year ago | (#42175025)

Labview has done auto threading for ages. Whenever a blocks inputs are valid, the block executes. The problem, at least with 8.6.1 and previous is it will generally make a copy of whatever the common inputs are so it can execute things in parallel, and, well, that copy often just makes things worse than just executing serially. A smarter compiler, perhaps similar to the one suggested in this, could avoid the copies, sometimes. Of course, the other problem with Labview is C# seems to be way faster, and for a sufficiently large project, I'd rather maintain the C# code.

Still, I remain somewhat of the opinion that a graphical language makes it easier to design multithreaded code in principle...

Great white hope? (1)

Anonymous Coward | about a year ago | (#42174745)

"Great white hope"?

Is this Slashdot or segregation-era Mississippi?

Re:Great white hope? (2)

belg4mit (152620) | about a year ago | (#42174811)

Indeed. silicon is sort of a shiny black color!

OP probably once read or heard the phrase,
misunderstood the context and thought it
would embiggen the post's language.

So... (0)

Anonymous Coward | about a year ago | (#42174767)

I skimmed the paper... it requires you to describe how variables are accessed... is this statically checked? If not, then it is no better than the current paradigm, except the compiler will be introducing races rather than the developer.

Also, there are no performance numbers. How much of the gains of paralleling such small chunks of code are lost through synchronization?

Doubling processors would not restore Moore's law (1)

rroman (2627559) | about a year ago | (#42174885)

Doubling of processor cores doesn't imply that things can be twice as fast. There is actually pretty hard science going on researching what can be done with N processors to speed up problem solving. It turns out, that even with large numbers of processors we are not able to speed up majority of algorithms in any significant way. Maybe in real world, few additional processors could compute some basic stuff in the meantime to speed things up, but I doubt that doubling the number of processors will add significant speed to computers when the number of them is large enough.

Where are the benchmarks? (-1)

Anonymous Coward | about a year ago | (#42174945)

I do not see a single performance benchmark in the extended version. Garbage.

Rubbish (0)

Anonymous Coward | about a year ago | (#42174951)

This is not auto-parallelising. This is adding in hints to the compiler to allow it to parallelise code. That's not automatic, that requires the programmer to do it. And with zero performance analysis in the paper, wtf? Seriously, this is a much harder problem than is being made out to be, and there is not one size fits all way of doing it. So they've found a neat way of doing things, so have other people. They work well in some circumstances, and not in others. Go figure. Paper probably quite interesting, but article complete tripe.

Performance.. (0)

Anonymous Coward | about a year ago | (#42174969)

So, first we create a virtual machine and a new language, called c#. That cost about 50% performance, but that's disregarded as 'irrelevant because desktop apps don't need speed' and 'computers only get faster'.

Then, when halved performance, we go seek ways to split inefficient single core code and split among two or more threads.

So basically, we found a way to have 2 processors do the job of one. Doubling energy usage. I'm impressed.

No, it cannot (0)

gweihir (88907) | about a year ago | (#42175083)

First, many problems do not have significant speed-up when parallelized. No automated tool can change that. Second, parallelizing things has absolutely nothing to do with Moore's law. Maybe look up the definition some time?

Oh! My Sweet Day. (2)

140Mandak262Jamuna (970587) | about a year ago | (#42175151)

All these days of careful coding diligently avoiding static_casts and const_casts ... All those recompilations triggered because I had to touch one header file used by everyone and his brother to satisfy my strict insistence of const correctness... I paid and paid and paid all these days. I avoided mutable data members because Soustroup pontificated in some vague posting in comp.lang.c++ (OMG I even forgot the usenet group name!) that "a well constructed object oriented code should not require mutables".

This is my pay day baby! My code is going to run super fast in multicore machines without any (more) extra work from me! Wooo Hooo!

Please take pity on my and let me enjoy my day dream for a few hours, without rudely inserting reality into my reverie! Thanks slashdotters, I know I can count on you!

Teach programmers (2)

Culture20 (968837) | about a year ago | (#42175187)

The holy grail of parallel processing is teaching programmers how to handle parallel processing (and what domains can benefit and where).

Re:Teach programmers (0)

Anonymous Coward | about a year ago | (#42175253)

Agreed. Oddly enough, I don't see the hordes of people here on /. who complain about how HR/management doesn't think there skills are fresh rushing over to defend this point of view.

functional programming (2)

segfault_0 (181690) | about a year ago | (#42175193)

A compiler may be able to thread your program, but it will be a long time before it understands your intent well enough to do it well. Also I can think of many situations under which such functionality may not even be a good idea at all. I would assume such a system would use a pool of threads to avoid constant thread construction overhead and if you get a few IO-busy actions threaded out in an ignorant fashion I think you will find your program blocking a lot more, and producing a lot less throughput than it should.

Also, the OP blithely stated that functional programming isn't happening -- yet features of the functional paradigm, like anonymous functions, have made their way into nearly every language of consequence and many new languages proudly tout functional programming features (see f#, scala, rust, clojure, and go). Perhaps pure functional programming is moving pretty slow, but it's features are not.

Communicating Sequential Processes (1)

Marxdot (2699183) | about a year ago | (#42175209)

Or we could just pursue CSP so that we have a friendly concurrency model with which to parallelize software that needs it, or just to introduce asynchronicity in a simple way. Shared-memory concurrency is pretty poor, whether or not a human user of the language is responsible for it.

keep moving them goalposts! (0)

Shavano (2541114) | about a year ago | (#42175269)

People who opine that a compiler advancement can get Moore's law back on track. clearly have no idea whatsoever what Moore's "law" states.

Moore's law has absolutely nothing to do with compilers or any other sort of software. It's about transistor count in integrated circuits. Specifically, Moore's "law" observed that transistor count of state of the art ICs had doubled every year since the IC was invented and he predicted that it would continue to do so for at least 10 years.

And he was approximately right (the best one can expect for a pseudo-prophetic utterance). But over time, the rate of increase has fallen below what Moore predicted, with an average doubling time of about 2 years. Now tech roadmaps assume more like 3 years. Every step is harder than the last and eventually we will reach a point of diminishing returns where the investment necessary to double IC complexity won't be seen as worthwhile.

Automatically paralleling compilers (2)

Animats (122034) | about a year ago | (#42175273)

Automatically paralleling compilers aren't new. SGI had one for C ten years ago. Intel has it for C++ and Fortran now. [intel.com] It's been done for Matlab. There's plenty of theory [wikipedia.org].

Outside of the supercomputing community, it's never gone mainstream. Microsoft is at least trying to do it for C#, which is a reasonable language. An interesting point is that this Microsoft approach exploits immutable objects. Immutable objects can be freely shared without locking, so wide use of immutable objects makes automatic extraction of parallelism easier.

I'd looked at doing something similar for Python, to get rid of the Global Interpreter Lock, Python's boat-anchor. Python already makes wide use of immutable objects, but doesn't gain any performance from them. If everything is thread-local, immutable, or synchronized in the Java sense, you don't need global locking. But the politics of the Python community do not favor performance. (Potentially, Python could be up to at least the LISP level of performance, within a factor of 2 of C, if the language was restricted in certain ways.)

Another part of the problem is the pernicious heritage of POSIX/Linux locking primitives. Many programmers think that locking is an library or OS-level problem, not a language level problem. Locking via library primitives means language doesn't know what data is locked by which lock. This makes optimization and race checking very tough.

The political and social problems here are tougher than the technical ones. So the open source community can't solve them. It takes someone with a big hammer, like Microsoft, to do it.

"Who tells whom what to do?" - V. Lenin

Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Sign up for Slashdot Newsletters
Create a Slashdot Account

Loading...