Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Emscripten Compiler Gets Optimizations, Now Self-Hosting

Unknown Lamer posted about 2 years ago | from the because-they-said-we-couldn't-do-it dept.

Programming 60

Emscripten is an LLVM-based compiler from dozens of languages to JavaScript (previously demoed as a repl and used to port Doom to the browser), and some recent changes have made it a bit faster, and allowed it to compile itself. Some highlights include a redundant variable eliminator, parallelization of the optimizier and compiler, and a new relooper. From the developer's weblog: "With all of the emscripten optimization passes now in JavaScript, I then worked on parallelizing that. ... The speedup can be close to linear in the number of cores. ... For the LLVM to JS compiler, I made the emscripten compiler parallel as well: It splits up the LLVM IR into 3 main parts: type data, function data, and globals. The function data part is unsurprisingly by far the largest in all cases I checked (95% or so), and it can in principle be parallelized - so I did that. Like in the optimizer, we use a Python process pool which feeds chunks of function data to multiple JavaScript compiler instances. There is some overhead due to chunking, and the type data and globals phases are not parallelized, but overall this can be a close to linear speedup. ... [On the new relooper] Note that this update makes Emscripten a 'self-hosting compiler' in a sense: one of the major optimization passes must be compiled to JS from C++, using Emscripten itself. Since this is an optimization pass, there is no chicken-and-egg problem: We bootstrap the relooper by first compiling it without optimizations, which works because we don't need to reloop there. We then use that unoptimized build of the relooper (which reloops properly, but slowly since it itself is unoptimized) in Emscripten to compile the relooper once more, generating the final fully-optimized version of the relooper, or 'relooped relooper' if you will."

Sorry! There are no comments related to the filter you selected.

Yo dawg... (-1)

Anonymous Coward | about 2 years ago | (#41962325)

...oh, I can't be bothered.

yes but does it... (1)

Coeurderoy (717228) | about 2 years ago | (#41962387)

run linux in the browser ?

Sincerely Emscripten is way cool...

Re:yes but does it... (3, Interesting)

The MAZZTer (911996) | about 2 years ago | (#41962403)

Probably not emscripten-based, but here you go [] .

Re:yes but does it... (0)

Anonymous Coward | about 2 years ago | (#41965175)

I'm so embiggened that it's emscripten based!

Re:yes but does it... (0)

Anonymous Coward | about 2 years ago | (#41965881)

Nice! And it even runs emacs.

Re:yes but does it... (1)

lister king of smeg (2481612) | about 2 years ago | (#41964215)

now what we need to do is compile virtualbox with it so that we can run any pc operating system in the browser even several of them at a time. this could be a great way to bench mark you browser and its js engine of course it would be hell of a slow but a fun project to attempt

Re:yes but does it... (1)

DrXym (126579) | about 2 years ago | (#41965297)

Browsers don't typically spamming around tight in timer loops making heavy use of array buffers. So while it might be an interesting test it is not reflective of what browsers are doing for most of their lives.

It's neat to see it working and there will be lots of ways it could be applied but really browsers need something like PNaCl - where LLVM bitcode can be compiled or interpreted by the browser with as little overhead as possible in a sandboxed environment.

Re:yes but does it... (1)

Derek Pomery (2028) | about 2 years ago | (#41965601) []

I don't see any limitations there really in how they are applying it. Demos that work on IE, Opera, Firefox, Chrome, Safari on any chip type.

I don't see a need for pnacl.

Even last year azakai was reporting emscripten output was running at 2x to 3x of optimised C.

There's a few things I guess it'd be nice if emscripten had like native 64 bit integers, but there's a Firefox bug on adding that to javascript w/ actual patches, so I imagine it'll get added to all the other browsers eventually.

Re:yes but does it... (1)

DrXym (126579) | about 2 years ago | (#41965675)

Empscripten compiles a low level bitcode into a high level language which ends up being parsed, compiled / interpretted inside the browser as a JS app. It's inherently wasteful.

Aside from the overheads of the above, JS is executed from inside a browser through events. You might have seen a message in a browser along the lines of "Javascript is taking too long to return should I kill it?" if a JS snippet does not exit quickly enough within some preset duration. The browser is essentially frozen while it's waiting for JS to return since with the exception of web workers everything is synchronous. So Emscripten would have to use a timer - set timer for 1 ms, wait for wakeup, do a slice of stuff, save state, set timer for 1 ms, save state, wait for wakeup. etc. It basically has to have a scheduler sitting at the bottom of itself so the browser doesn't lock up which drags down performance and it's inefficient. If it does anything to the screen it must also do it through the DOM which adds its own overheads even if everything is directed to a canvas.

Don't get me wrong I think Empscripten is a technological marvel and will find many uses. At the same time it's horribly inefficient compared to the way it should be done. Providing it was endorsed as an open standard PNaCl or something akin to it which allows the browser to compile it straight into native code (with the appropriate hooks and safeguards to prevent the code from smashing stacks or escaping its sandbox) is the right way.

Re:yes but does it... (1)

DrXym (126579) | about 2 years ago | (#41965679)

I also didn't mention that the time slicing above would also have to simulate threading for C/C++ apps which need it - a bit like the way Java used to do pseudo threading circa 1.0.2. Everything would be running on the one thread though.

Re:yes but does it... (1)

Derek Pomery (2028) | about 2 years ago | (#41966351)

If you checked out the demos, such as bananabread, emscripten is fully capable of doing javascript threading.
That is, any work that does not need DOM access can be done on background threads.
The "freezing" and "time slicing" are not particularly interesting objections. Any application that has to interact with the DOM has the same issue, which is why JavaScript hasn't solved it on the main thread, apart from the perfectly reasonable approach of event based code. And yes, this means using setTimeout and requestAnimationFrame on the main thread. Big deal.

The objection to highlevel language? Really not interesting. Might as well apply that to perl, python, ruby even java has a runtime JIT for its bytecode, and while I don't know much about "portable" NaCL, I imagine it does too.

JavaScript JITs are not a significant portion of the program's overhead. You may be able to do better in NaCL but at the loss of browser support, cross-platform support, integration into the whole universe of existing JavaScript, extensibility by other scripts/addons and even readability (yes, I've read through emscripten generated code when debugging. not high on readability, but certainly higher than assembly).

Re:yes but does it... (1)

DrXym (126579) | about 2 years ago | (#41967521)

PNaCl *is* bitcode - LLVM bitcode with some APIs that it can see for audio, display and minimal interaction with the browser surrounding it. So instead of translating bitcode to JS and compiling / interpreting that again through the constraints of the browser, JS and the DOM, the browser could execute it directly or hand over execution to a plugin, even one running with reduced privileges in its own process space, and only interacting with the browser for repaints. That's the point. Performance would be a LOT better. Proper threading, proper code execution, no time slicing except through the standard kernel scheduler. Aside from any instrumentation / security checks it could potentially execute at full native speed.

Re:yes but does it... (0)

Anonymous Coward | about 2 years ago | (#41966505)

Even last year azakai was reporting emscripten output was running at 2x to 3x of optimised C.

Running actual programs or just stupid synthetic benchmarks like looped method calls? Please provide links to this as it sounds like total bullshit.

Re:yes but does it... (1)

StripedCow (776465) | about 2 years ago | (#41966269)

What we need first is to compile Firefox or Chromium to javascript. That way, we can finally be freed from that annoying non-compliant and otherwise broken IE crap that calls itself a browser.

Chrome Frame (1)

tepples (727027) | about 2 years ago | (#41967647)

Chrome is already available as an IE browser helper object. If the goal is to improve the standards compliance of Internet Explorer, the only advantage of making a browser-within-a-browser in JavaScript is to allow it to run on systems where the user lacks the privilege to install Chrome Frame.

Re:Chrome Frame (1)

StripedCow (776465) | about 2 years ago | (#41967793)

Any serious management team will discard the "Chrome Frame" option as it requires the user to go through extra hassles, and like you said, in some cases where the user does not have the proper privileges it simply does not work. Therefore, this solution is actually a non-solution, unfortunately.

Re:yes but does it... (0)

Anonymous Coward | about 2 years ago | (#41965751)

Imagine a full install of the gentoo distro ported to that :D

lather, rinse, repeat (1)

thereitis (2355426) | about 2 years ago | (#41962559)

How would 'lather, rinse, repeat' look after being relooped?

It is called... (0)

Anonymous Coward | about 2 years ago | (#41962563)


wow (1)

Tumbleweed (3706) | about 2 years ago | (#41962623)

That's empressive.

DOOM (3, Informative)

Dan East (318230) | about 2 years ago | (#41962665)

I followed the link to Doom (and the next link, and the next link...) until I finally got to:

Harvey Anderson
Mozilla Corporation
650 Castro Street, Suite 300
Mountain View, CA 94041
Phone Number: 650-903-0800
Fax: 650-903-0875

Re: DOOM on browser available on Mozilla website - Cease and Desist Notice - DMCA Notice of Copyright Infringement

Dear Mr. Anderson:

In accordance with Mozilla’s copyright infringement policy, this is to notify you of activity occurring on the Mozilla site listed below which infringes on the exclusive intellectual property rights of Id Software LLC, a wholly owned subsidiary of ZeniMax Media Inc. The copyrighted work at issue is Id Software’s proprietary software game DOOM® (“DOOM”). The link below offers an unauthorized derivation or version of Id Software’s DOOM game. []

DOOM is a registered trademark and the game assets are copyrighted material. Use of the mark DOOM and copyrighted assets without our authorization and consent, directly violates our trademark and copyright rights in and to such intellectual property. We hereby demand the immediate removal of all such links from your website and written assurance that you will prevent any further infringement of Id Software’s intellectual property rights. I have a good faith belief that use of the copyrighted materials described above as allegedly infringing is not authorized by the copyright owner, its agent, or the law. I swear, under penalty of perjury, that the information in this notification is accurate and that I am the copyright owner or am authorized to act on behalf of the owner of an exclusive right that is allegedly infringed.


Joshua Gillespie

Associate General Counsel
ZeniMax Media Inc.
1370 Piccard Dr., #120
Rockville, MD 20850
T: 301.948.2200

Re:DOOM (2)

cultiv8 (1660093) | about 2 years ago | (#41962907)

Not surprising, iD Software had DOOM removed from Android market [] a few years ago.

Re:DOOM (1)

TheRealMindChild (743925) | about 2 years ago | (#41963563)

Nothing wrong with that. The code is "free", but the resources (textres, models...) are not

Re:DOOM (0)

Anonymous Coward | about 2 years ago | (#41964561)

well cultiv8 you can cross compile doom for yourself. The problem is when you do such things for the mass market.

ZeniMax CEO is a piece of shit (-1, Troll)

Anonymous Coward | about 2 years ago | (#41963369)

ZeniMax Media Inc [] CEO Robert A. Altman [] is a known criminal, and has been banned from banking in the United States. His wife Lynda Carter [] (of Wonder Woman fame) has been a known alcoholic due to her husband's constant affairs with underage FOBs [] . While Robert has access to a beautiful, well developed woman, his weakness is for flat chested poon tang. Robert's unreleased biography contains an insightful chapter, on his love for tearing open small hairless pussy. Mr. Altman is a horrible piece of human garbage, and TES IV:Oblivion was an equally terrible piece of shit.

Re:ZeniMax CEO is a piece of shit (0)

Anonymous Coward | about 2 years ago | (#41964217)

Why are the ZeniMax shills downmodding the parent post? Why is ZeniMax against revealing the facts of its CEO?

Re:ZeniMax CEO is a piece of shit (0)

Anonymous Coward | about 2 years ago | (#41964671)

Why won't he take a lie detector test!?

Re:ZeniMax CEO is a piece of shit (2)

91degrees (207121) | about 2 years ago | (#41966205)

It could be more that 's it a piece of irrelevant character assassination.

Is sending a C&D regarding Doom on the browser more acceptable if it turns out that it's from his angelic business partner?

Re:ZeniMax CEO is a piece of shit (3, Informative)

IamTheRealMike (537420) | about 2 years ago | (#41965481)

The wikipedia page you reference states explicitly that he was acquitted of all charges in the criminal suit (and is therefore NOT a known criminal), and he agreed to be banned from banking to make a civil suit go away. That suit was never resolved, so we don't really have any idea whether he was guilty or not - he might have agreed on the basis that he was tired of defending himself in court and didn't care much for being a banker anyway. Let's face it, settlements in the face of expensive yet questionable government prosecutions are hardly rare, especially in the USA.

The rest of the wild claims you assert aren't supported by any of the wiki pages either. Maybe it's all true but it's impossible to tell from your post. At any rate, it has nothing to do with the fact that DOOMs artwork and level data is still under copyright. If you want to get pissy at somebody about that, go complain to Carmack - he's the one who decided to open source the code but not the game data. Or not: he does a lot more for open source than most game company CEOs do.

Re:DOOM (0)

Anonymous Coward | about 2 years ago | (#41964227)

That's what FreeDOOM is for.

Re:DOOM (0)

Anonymous Coward | about 2 years ago | (#41965215)

Just tell him to fuck off.

Re:DOOM (1)

ameen.ross (2498000) | about 2 years ago | (#41965459)

The name's


Re:DOOM (1)

LoRdTAW (99712) | about 2 years ago | (#41966895)

The problem is they cant distribute the wad files. Other Doom engines such as prboom, zdoom, jdoom (my favorite), etc. have done just fine for over a decade as they only provide a "doom compatible" engine. The user must provide the wad files themselves even if they intend to play the shareware wad files that are freely available. They should have made or searched for a freely distributable total conversion wad to avoid any problems. I am sure there are plenty to be had off Doom fan sites.

There was a Doom game for the Android some years back but it too was pulled for the same reason, you cant distribute their wad files without permission. The game engine code is GPL so anyone can make a Doom engine on the mobile platform of choice be it Android, iOS, Browser, etc. You can even charge money for it. Just don't call it Doom and distribute a copyrighted wad file.

Re:DOOM (0)

Anonymous Coward | about 2 years ago | (#41968311)

Use DosBox and distribute their original shareware - problem solved.

Take down notice from ZeniMax Media (1)

OrangeTide (124937) | about 2 years ago | (#41963269)

I like how the Doom Port [] now redirects to a take down notice sent by ZeniMax Media Inc's legal counsel.

There are free assests for running on the Doom engine, but I guess not running actual Doom won't get you linked anywhere cool.

Re:Take down notice from ZeniMax Media (0)

Anonymous Coward | about 2 years ago | (#41963495)

I always wonder in cases like this why zero apparent effort is made to work out a free licensing arrangement with the developer so that all trademarks and copyrights are respected and the application stays up.

When I go to look up a project on the Internet and find out it's been yanked by or on behalf of a company it usually makes me think less of the company, unless of course it's a matter of outright piracy of a product. With all the competition in games one wonders if a publisher ought to be thankful for the continued attention to their brand instead of shitting on it?

Re:Take down notice from ZeniMax Media (1)

TheRealMindChild (743925) | about 2 years ago | (#41963683)

The reason is, there is no one at iD that knows who owns the copyright to what on the Doom era games. It would take way too much effort and resources to just figure that out, just to let some indy license it for a loss

Re:Take down notice from ZeniMax Media (2)

vux984 (928602) | about 2 years ago | (#41964553)

So they assert they own the copyright, and issue cease and desist orders based on that assertion, because they don't know who owns the copyright?

Seems one could challenge the cease and desist by requesting they identify precisely what they are claiming ownership of, and proof that they do in fact own it.

If, they can provide that, great, then request a license.

If not, I'm not, then the C&D doesn't hold a lot of water.

That's how I'd tackle a C&D for most abandonware. This is a little different though; ID still sells Doom; its on their website last I checked, its on Steam too; so they clearly feel they have all the authority they need to copy and distribute copies, and negotiation distribution of the Doom WADs that are always at the heart of these C&D orders. If they can still sell them, they can give them away, so the argument that they don't have the rights to just distribute them is bunk.

They'll pay a lawyer to write a C&D letter. ID didn't care, they wouldn't do that, and they don't need to do that. And if they were really worried about who owns what, they could simply let whoever else apparently would lay claim to the assets send their own C&D letters.

Re:Take down notice from ZeniMax Media (1)

tepples (727027) | about 2 years ago | (#41967709)

So they assert they own the copyright, and issue cease and desist orders based on that assertion, because they don't know who owns the copyright?

They know they own the copyright at least in part. They are claiming infringement of copyright in that part, whether or not that part is complete.

Re:Take down notice from ZeniMax Media (0)

Anonymous Coward | about 2 years ago | (#41964241)

Because they aren't hippies, you stupid faggot.

Re:Take down notice from ZeniMax Media (1)

hazah (807503) | about 2 years ago | (#41966211)

Does the stupid hurt?

Re:Take down notice from ZeniMax Media (0)

Anonymous Coward | about 2 years ago | (#41964541)

This slash and burn approach to software copyrights is a total PR disaster for these companies. I really do not understand why they aren't more careful about their reputation.

Re:Take down notice from ZeniMax Media (1)

Lunix Nutcase (1092239) | about 2 years ago | (#41966547)

Yes, total PR disaster. I remember being unable to pick up a newspaper or turn on a news program without hearing the outrage. I'd be walking around on the street or in a store and everyone was complaining about how evil Zenimax was for their takedown notice.

Wait, what?

Re:Take down notice from ZeniMax Media (0)

Anonymous Coward | about 2 years ago | (#41971339)

There will be a reckoning of the likes which you have never seen before. Gamers will rise out of their duct tape covered chairs, dust crumbs of doritos off their lap, and march on to Washington to demand an end to these abusive practices. Possibly stopping at 7-eleven on the way to get a bottle of Code Red.

the only reason we need this... (1)

kenorland (2691677) | about 2 years ago | (#41964181)

is because a bunch of squabbling companies can't agree what kind of runtime to put in the browser. So, we're stuck with JavaScript, .NET, and the JVM. How hard can it be to turn the llvm into a plugin?

Re:the only reason we need this... (1)

Dasuraga (1147871) | about 2 years ago | (#41964531)

n compteting standards

let's add a new one!

n+1 competing standards

Re:the only reason we need this... (1)

kenorland (2691677) | about 2 years ago | (#41965357)

JVM and CLR are not competing standards, they are failed standards.

JavaScript is a standard scripting language, but not a VM.

Right now, the number of competing standards for embedded virtual machine for compiled languages is zero.

Re:the only reason we need this... (1)

Serious Callers Only (1022605) | about 2 years ago | (#41964969)

The big issue is probably sandboxing, so you'd need a VM which was sandboxed like Java or Javascript, and then to deal with the constant security headaches as people find vulnerabilities. If it was a trivial problem someone would have done it by now, but please do go ahead :)

Re:the only reason we need this... (1)

spage (73271) | about 2 years ago | (#41965239)

No, the squabbling companies have agreed on JavaScript and the DOM as the runtime, and agreement on additional Web APIs for audio, game pads, fullscreen mode, orientation, etc. is likely soon. It works, it's spectacularly compatible in the latest browsers, and there's little need for anything more.

LLVM isn't a runtime, NaCl or better PNaCl talking to the Pepper API is a runtime, but it's an underspecified Google-controlled approach that Apple, Mozilla, Microsoft and Opera will never adopt . For all the alleged benefits of running some lower-level bytecode in a VM, there's no credible or likely path to a standard. Meanwhile JavaScript rolls on.

Re:the only reason we need this... (1)

RCL (891376) | about 2 years ago | (#41978303)

Won't client-side runtime "compilation" limit the complexity of software which is going to be written, similarly how GLSL run-time compiled shaders lost (on PC) versus offline-compiled HLSL? Not to mention problems with keeping your source code to yourself.

These are two aspects where bytecode (or native) runtime would be certainly better. I don't see benefits of source level runtime... it's no easier to audit sources security-wise than to audit VM and it can break in so many more ways.

Re:the only reason we need this... (1)

spage (73271) | about 2 years ago | (#41978641)

emscripten is compilation in advance of massive complex C code into JavaScript that truly runs anywhere. In most cases keeping your source code to yourself is unimportant; Facebook and Google ship MBs of JavaScript source to billions of browsers every minute. My impression is Java has more security vulnerabilities than JavaScript, but then again it's able to do more.

Java in the browser did deliver benefits; yet Java in the browser is almost completely dead. Maybe a bytecode or native runtime would have delivered more than the astounding progress in computing, technology, and society that JavaScript in the browser has brought us for 18 years. I doubt it.

Re:the only reason we need this... (1)

RCL (891376) | about 2 years ago | (#41978715)

I don't feel any "astounding progress", in my humble opinion user experience of doing things in browser didn't change much since Java applets day: it's still slow, unresponsive and unreliable interface and it loses with native clients each time there exists one. Perhaps some progress has been achieved throughout the years, but it's unobvious for me how Javascript's source-levelness has had anything to do with that. If Microsoft weren't so Java-averse (and if the rest of industry weren't so Microsoft-averse) we could have had something better.

Source-level runtime looks like a temporary scenario. It implies the need for a compiler on the client, unpredictable and hard to debug behavior of the same code, limits complexity of the program. I realize that Javascript is the standard now, but I don't see any inherent advantage of it, apart of being a "least common denominator" not owned by any one single vendor.

And BTW, I heard that Facebook is thinking about some kind of bytecode on client side.

The obvious is completely missed.. (1)

MnemonicMan (2596371) | about 2 years ago | (#41964545)

On the fly recompiler for different operating systems or processor architectures... Mmm? ;)

Re:The obvious is completely missed.. (1)

MnemonicMan (2596371) | about 2 years ago | (#41964625)

Remember, machine language is just another language except it is one tailored to processors. So, I should have said "and/or" instead of just "or" in the parent post. For operating systems, the APIs can be high-level thunked so you could do something like translate DirectX calls on Windows 7 to an equivalent library on an ARM Linux machine. The program code is recompiled but the libraries the program code calls are native to the destination device. DirectX is also just another "hardware feature." You can thunk DirectX in exactly the same way you would thunk a VGA card. The recompiler doesn't care - all that matters is that the calls from recompiled code go to equivalents, no matter how high or low level those may be.

Qt (2, Interesting)

Anonymous Coward | about 2 years ago | (#41964923)

Somebody's porting Qt to it - looks like it's just the non-gui parts for now, though: []

Would be cool to be able to get Qt apps running in browsers without using the architecture-specific nacl [] .

well (0)

Anonymous Coward | about 2 years ago | (#41965667)

well that was a mindfuck of a summary.

Javascript? Pfah (4, Funny)

david.given (6740) | about 2 years ago | (#41965815)

Anyone can compile C into Javascript. Only I can compile C into... Perl.

See cluecc [] .

(Clue allows C to be compiled, badly, into a variety of scripting languages including Lua, Javascript and Perl as well as Java. Some nutter even contributed a Common Lisp backend. It was an experiment to see whether exploiting certain vaguenesses in the ANSI C spec concerning pointer representation was useful. Unlike Enscripten, Clue doesn't have a big array of bytes representing the C memory; instead pointers are represented as object-offset tuples. It worked really well, but unfortunately nearly all existing code out there doesn't work right on a system where sizeof(int)==sizeof(double)==sizeof(char)==1 and sizeof(void*)==2. Plus, the compiler frontend I was using had a number of major issues. But it works well enough to run benchmarks.)

(And before you ask, yes, compiling C into Perl 5 is a total, utter, complete waste of time.)

Re:Javascript? Pfah (0)

Anonymous Coward | about 2 years ago | (#41966209)

Good for you...

Please allow me to compile myself... (-1)

Anonymous Coward | about 2 years ago | (#41966433)

I'm a compiler of wealth and taste.

Check for New Comments
Slashdot Login

Need an Account?

Forgot your password?