Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Python-to-C++ Compiler

timothy posted more than 8 years ago | from the calibrate-your-scales dept.

181

Mark Dufour writes "Shed Skin is an experimental Python-to-C++ compiler. It accepts pure, but implicitly statically typed, Python programs, and generates optimized C++ code. This means that, in combination with a C++ compiler, it allows for translation of pure Python programs into highly efficient machine language. For a set of 16 non-trivial test programs, measurements show a typical speedup of 2-40 over Psyco, about 12 on average, and 2-220 over CPython, about 45 on average. Shed Skin also outputs annotated source code."

cancel ×

181 comments

Sorry! There are no comments related to the filter you selected.

How to obfuscate C (-1)

Anonymous Coward | more than 8 years ago | (#15541269)

Write it in Python.

Sounds good... (0)

bblazer (757395) | more than 8 years ago | (#15541329)

This is an interesting development. Programmers can now use the simple syntax of python and create faster machine code. THis may make rapid prototyping even more rapid, and allow programmers with little or no C++ experience create code that will run faster and will not require someone to install Python to run something. I will have to explore it more, but it will be intriguing to see how they handle things like pointers and structs that are not in python.

Re:Sounds good... (3, Insightful)

B'Trey (111263) | more than 8 years ago | (#15541464)

I will have to explore it more, but it will be intriguing to see how they handle things like pointers and structs that are not in python.

Uh, why would they have to? This goes from Python to C++, not vice versa. If there are no pointers or structs in the Python code, why would they have to handle them? Certainly, it's quite possible that some Python variable types will be converted to pointers or structs in the output code, but that's orthagonal to the issue of Python not having them natively.

If you were trying to go from C++ to Python, then you'd have to convert C++ pointers and structs to some sort of Python data type, and your comment would make sense. As it is, I'm not sure what you were trying to say.

Re:Sounds good... (1)

bblazer (757395) | more than 8 years ago | (#15542420)

I do understand that Python does not have structs or pointers. But pointers are fundamental to C++. I was very unclear in my first post. I was wondering if they would be converting certain types to pointers and I was curious how that was going to happen.

Re:Sounds good... (1)

j35ter (895427) | more than 8 years ago | (#15542742)

If there are no pointers or structs in the Python code, why would they have to handle them?
whenever you pass a object to a method, you actualy pass a pointer who references this object(struct). Python just helps you not to think (too much) about it :)

Re:Sounds good... (2, Insightful)

masklinn (823351) | more than 8 years ago | (#15541576)

it will be intriguing to see how they handle things like pointers and structs that are not in python.

Why would one ever need to do that? The goal is not to write C++ in Python, it's to compile Python to machine code via an intermediate Python -> C++ compilation.

Re:Sounds good... (1)

Frizzle Fry (149026) | more than 8 years ago | (#15541940)

THis may make rapid prototyping even more rapid

If you are just making a prototype, why is squeezing out extra perf so important? Prototyping is the sort of situation where you should be fine with just using straight python.

not terribly useful quite yet (4, Insightful)

Surt (22457) | more than 8 years ago | (#15541340)

Until he addresses mixed types in n-tuples, this won't be useful for very many people.

Underlying technology (1, Informative)

Anonymous Coward | more than 8 years ago | (#15541527)

PyPy [codespeak.net]

Re:Underlying technology (1)

ZephyrXero (750822) | more than 8 years ago | (#15542232)

"Underlying technology: PyPy [codespeak.net] "

I could see this Python > C++ > Machine code project being something PyPy was built on top of, but not the other way around? Please explain...

Re:not terribly useful quite yet (2, Insightful)

goombah99 (560566) | more than 8 years ago | (#15541871)

But he's on the right track. Python allows dynamic typing but nearly all of ones programs do not take advantage of it. Recognizing that is key to making it go fast I think. It would be nice to have a filter you could run over python that would find all the type ambiguous points and let you insert some sort of compiler hinting.

I could envision it working like this. Instead of statically declaring all your variable types in every function, you instead simply declare that whatever tpyes are being used, they are always the same every time this function gets called. All the compiler then has to do is to find one instance of calling that function or one instance of the use of the arguments within the subrotuine in which the type is unambiguous to reverse engineer the types without having to be told. It could then flag the few cases where it can't resolve the type and either handle those in a slower dynamically typed fashion, or allow you to hint the types it was confused by.

Re:not terribly useful quite yet (1)

Surt (22457) | more than 8 years ago | (#15541984)

I could well even imagine that using RTTI in C++ he could actually just split out whatever logic expected different types. Certainly there's some work there, but it seems plausible to me.

Why not just use pure C++? (0)

Anonymous Coward | more than 8 years ago | (#15541351)

Why not just use pure C++?

Re:Why not just use pure C++? (2, Insightful)

bzerodi (731405) | more than 8 years ago | (#15541377)

Why not pure assembler ?

Re:Why not just use pure C++? (0)

Anonymous Coward | more than 8 years ago | (#15541477)

Why not pure 1's and 0's?

Re:Why not just use pure C++? (1)

tiocsti (160794) | more than 8 years ago | (#15541521)

Portability. Besides, why not leverage the decent optimization that already exists in the gnu compiler collection? I guess for me, a better question might be, why not a python frontend to gcc.

Re:Why not just use pure C++? (1)

masklinn (823351) | more than 8 years ago | (#15541643)

That's pretty much what he's doing, ShedSkin is a Python to C++ compiler, then you need to compile the C++ code ShedSkin yields to machine code, you can do that with gcc.

The goal (for the author) at the moment is to get a fairly complete Python to C++ compiler (ShedSkin is already very good if you're mostly doing simple operations such as crunching numbers, but if your program is really complex or uses libraries then you're out of luck)

Re:Why not just use pure C++? (1)

tiocsti (160794) | more than 8 years ago | (#15541839)

I meant frontend in a specific gcc sense (as in, something that parses the language and outputs rtl which can be consumed by gcc backends), not as a generic intermediate step that will eventually be compiled via gcc (or whatever), as that's generally true of most things that output c or c++ code these days, yeah?

Re:Why not just use pure C++? (1)

boggartlaura (780968) | more than 8 years ago | (#15541389)

Well, personally, I like Python more. I'm not against C++, but I find Python works better and is easier to use for what I want to do.

Re:Why not just use pure C++? (-1, Flamebait)

Anonymous Coward | more than 8 years ago | (#15541431)

Because Python is WAY faster and easier to write, and it's highly optimized C++ under the hood, far more optimized than (all due respect) you're likely to be able write in one try. For example, if you know both languages, write a C++ program to read a large text file (1G+)into memory, do something to it, and write it back out. Then do it in Python. Do a time comparison of the execution and of the time it took you to write each.

After four hours of tweaking, our expert C++ programmer was finally able to write something that beat our ten lines of Python code that took under five minutes to write. And it didn't beat it by much, whereas the first pass at a C++ version was an order of magnitude slower.

Re:Why not just use pure C++? (3, Interesting)

b17bmbr (608864) | more than 8 years ago | (#15541942)

After four hours of tweaking, our expert C++ programmer was finally able to write something that beat our ten lines of Python code that took under five minutes to write. And it didn't beat it by much, whereas the first pass at a C++ version was an order of magnitude slower.

Which is why languages like python were written in the first place. They pretty much just make the underlying C calls anyways, but do so in a way that handles buffer overflows, pointers, etc., that pretty much make C/C++ so troublesome, hazardous, and hard to learn. I like java (alot really), but nothing beats a good scirpting language, like perl or python, to handle tasks like text manipulation. Python is especially good at using libraries, such as the imaging library, which are written in C anyways. How much faster can you get calling a C library from C than from python? I honestly don't know, but I can't imagine it's that much more. But when you add in speed of development, safety, and even portability, it's powerful.

Python's OOP is also a feature that makes it far more attractive than perl for me. Perl does OOP, but it's not as clean as python's, and I don't think it supports all the OOP features either. Doing GUI's is not the strength of any scripting language, but it depends on what you need to do. You can write a native frontend and embed python into a C or even a java application.

Re:Why not just use pure C++? (2, Informative)

ZephyrXero (750822) | more than 8 years ago | (#15542313)

"Doing GUI's is not the strength of any scripting language..."

This is why projects like pyGTK [pygtk.org] exist ;)

Re:Why not just use pure C++? (1)

baadger (764884) | more than 8 years ago | (#15543301)

Programmatically creating GUI's? Ewww, why not just take another leap and go with the python bindings for libglade?

Re:Why not just use pure C++? (1)

Schraegstrichpunkt (931443) | more than 8 years ago | (#15543382)

I think the Python Glade bindings require the GTK bindings.

Re:Why not just use pure C++? (1)

radtea (464814) | more than 8 years ago | (#15543520)

I like java (alot really), but nothing beats a good scirpting language, like perl or python, to handle tasks like text manipulation.

You say that like you think Java isn't a scripting language, but an analysis of language features, like anonymous inner classes (which encourage on-the-fly, non-designed extensions to applications) clearly shows that it is more appropriate to scripting than an applications development, particularly if you care about run-time performance (yeah, I know, with the right JVM and stuff Java can go fast--so prove to me that my customers will be running that JVM and that it will support all the language features I need for the specific application I'm building--"can go fast" != "does go fast".)

Re:Why not just use pure C++? (2, Interesting)

joss (1346) | more than 8 years ago | (#15542351)

Sorry, but without more details it would seem to me that
your "expert" C++ guy wasn't an expert. Can you describe the
problem a little better.. if what you say is true, I as
a long term C++ programmer would consider switching, but
I've looked at python, and I simply don't believe you.

I'll grant that C++ is a nightmare for beginners with more pitfalls
than an indiana jones movie, but once you know them, writing
poorly performing code is unlikely.

Re:Why not just use pure C++? (1)

Bill Dog (726542) | more than 8 years ago | (#15542634)

They could be experts in C++, and not be already familiar with the highly unusual case of manipulating insanely large text files efficiently. Like me.

Re:Why not just use pure C++? (1)

19thNervousBreakdown (768619) | more than 8 years ago | (#15542527)

Why would you even try to compete with file I/O? Disk access is slow enough that an interpreted language is going to have no problem keeping up with a compiled one. As for the 4 hours, yeah, there's some stuff that Python just does better, but I'm inclined to say they just didn't know what they were doing, mostly because they even attempted competing on the I/O front.

Stupid comparison (3, Insightful)

ardor (673957) | more than 8 years ago | (#15542750)

As another poster already said, file I/O is a bottleneck regardless of ANY language. So, try something different. Real-time h264 decoding for example.

Re:Stupid comparison (0)

Anonymous Coward | more than 8 years ago | (#15543927)

Agreed. The only possible order of magnitude performance increase I can even think of by writing in C/C++ would possibly be to use mmap rather than stdio/sfio/iostreams or read/write, and I wouldn't be surprised if a few minutes searching didn't turn up something to do that in python.

IO intensive tasks are probably the absolute worst language shoot out benchmark you could choose

Re:Stupid comparison (1)

Tim Browse (9263) | more than 8 years ago | (#15543997)

Or, to put it another way:

"More computing sins are committed in the name of efficiency (without necessarily achieving it) than for any other single reason - including blind stupidity."

Re:Why not just use pure C++? (4, Informative)

SigmoidCurve (188795) | more than 8 years ago | (#15541656)

bzerodi's point, made with Zen-like simplicity, is that language choice should be made to minimize programmer time, not machine time. I am at least a factor of ten more productive with Python than with C or C++. I am also far more confident in the correctness of what I write per line of Python than with what I write per line of C/C++.

Yes, I have have wasted some time staring at the shell waiting and waiting for it to return from some complicated Python routine. I know that compiled C would faster, and hand-rolled assembler would be faster still. But I say to myself: hey, I wrote this code in a single afternoon, how many weeks of hair-pulling would it take to re-engineer this - and make it bug-free - in C? When I put it that way, I don't mind waiting the extra minutes for Python to do my dirty work.

As a previous poster mentioned, the ability to handle tuples of mixed-types is critical. I look forward to seeing great things from Shed Skin in the future.

Re:Why not just use pure C++? (1)

mrchaotica (681592) | more than 8 years ago | (#15542532)

Not to mention that you can speed up execution time by throwing more hardware at it. If you try that with programmer time you just end up with a bruised programmer, and nobody wants that!

Re:Why not just use pure C++? (1)

dancpsu (822623) | more than 8 years ago | (#15542044)

The programming language can encourage the programmer to write an application faster. People have pointed this out before. But also, a programming language can encourage better programming principles. C++ makes it difficult to use complex data structures, while scripting languages like Perl and Python make it a breeze. Python also has the advantage of making object oriented programming simple, whereas the convoluted structure of C++ struct-based objects discourages programmers from taking advantage of them.

Re:Why not just use pure C++? (1)

mypalmike (454265) | more than 8 years ago | (#15542625)

C++ makes it difficult to use complex data structures, while scripting languages like Perl and Python make it a breeze.

Complex data structures in Perl? Such pain I wish to never endure.

Re:Why not just use pure C++? (1)

Bill Dog (726542) | more than 8 years ago | (#15542755)

C++ makes it difficult to use complex data structures...

It does? I've always managed, somehow. If you're referring to things like structures of pointers to structures, smart pointer techniques have been around for ages that manage cleanup. Maybe not as trivial as a scripting language, but not difficult.

Python also has the advantage of making object oriented programming simple, whereas the convoluted structure of C++ struct-based objects discourages programmers from taking advantage of them.

Nice talking out your ass there. Ya, C++ programmers everywhere refuse to use struct-based objects (i.e. C++'s classes) and OOP because they're so discouraging.

Re:Why not just use pure C++? (0)

Anonymous Coward | more than 8 years ago | (#15542846)

ShedSkin shows that it's possible to write fast programs with a really simple syntax.

Re:Why not just use pure C++? (0)

Anonymous Coward | more than 8 years ago | (#15544009)

C++ makes it difficult to use complex data structures

Now that compilers have actually caught up to the standard this is flat out false, the STL makes working with data structures at least as easy in C++ as it is in perl for standard cases, and easier for complicated cases. Many problems with perl "TMTOWTDI" become evident with complex data structures, you can turn on all the strict checking and warnings you want - but complex perl structures are very prone to doing the wrong thing silently.

2-40 what? (0, Redundant)

Rorian (88503) | more than 8 years ago | (#15541372)

2x to 40x speedup? 2% to 40%? 2 to 40 seconds?

Standardised units are your friend.

Re:2-40 what? (1)

chills42 (750137) | more than 8 years ago | (#15541449)

2 times as fast up to 40 times as fast is the Standard interpretation...

Re:2-40 what? (0, Redundant)

Rorian (88503) | more than 8 years ago | (#15541494)

I'm sure it is, but I hate it when people don't specify :(

Re:2-40 what? (2, Insightful)

Schraegstrichpunkt (931443) | more than 8 years ago | (#15543406)

"Times faster" is a unitless quantity.

Ewwwww (2, Funny)

$RANDOMLUSER (804576) | more than 8 years ago | (#15541375)

As a UNIX admin, I was saddled with one of these kinds of things years ago, a DEC-BASIC to C compiler for UNIX. The output code quality was incredibly bad: machine generated variable and function names, bizarro nested struct/union/struct data structures, 400-line functions peppered with calls to 1-line functions. Completely unreadable. Thank $DEITY that project died quickly.

Re:Ewwwww (5, Insightful)

Anonymovs Coward (724746) | more than 8 years ago | (#15541434)

Completely unreadable.

I think you're not supposed to read it. You're only supposed to feed it to your C++ compiler. f2c produced unreadable output too, but nobody read the output; at one time it was the only free fortran option on linux.

Re:Ewwwww (1, Funny)

Virak (897071) | more than 8 years ago | (#15541489)

Which is why I suggest you use brainfuck for all your coding needs. The generated code will make just as much sense as the original, if not more.

Re:Ewwwww (0)

Anonymous Coward | more than 8 years ago | (#15542096)

Ever try to read C-to-machien language compiler output? Good lord!

Re:Ewwwww (2, Insightful)

IamTheRealMike (537420) | more than 8 years ago | (#15542406)

If you actually tried ShedSkin you'd find the C++ it produces is very similar to what a human might produce, and is actually quite easily readable. But then - why would you want to anyway? It's an intermediate form useful to pass to an optimising C++ compiler, not as something to read.

Re:Ewwwww (3, Insightful)

Tim Browse (9263) | more than 8 years ago | (#15543944)

Yeah, whenever I look at the output of my optimising compiler, it's really hard to understand too. It's all in assembler, for a start.

Plus, the quality of C code generated by CFront was rubbish - unreadable.

Same with the Modula-3 compiler I tried. You couldn't work out what was going on in the resulting C code without a load of work.

Can you see where I'm going with this?

Static Typing? (0)

Anonymous Coward | more than 8 years ago | (#15541378)

If you're only allowed to use static typing, doesn't that defeat the purpose of coding in python in the first place?

Re:Static Typing? (1)

goofyheadedpunk (807517) | more than 8 years ago | (#15541479)

No, not really. A large number of people, including myself, just use python as a nicer C. Futzing with pointers and other such things can be ingnored while making a prototype and, after finishing the prototype, the bits that need to be faster can then be rewritten.

I recently wrote a largish simulation in python for a Biology course. The goal was to watch how a species spread over a planet given other competing species, natural disasters and the like. It took four in deep hack mode to write the whole thing, all of it implicitely typed due to the equations at the base of the simulation. Implicite static typing is used a lot in large applications. So much so that in fact, if I recall, python 2.5 is supposed to have optional type declaration.

Re:Static Typing? (1)

pthisis (27352) | more than 8 years ago | (#15542238)

So much so that in fact, if I recall, python 2.5 is supposed to have optional type declaration.

No, it doesn't. AFAIK all the Type-SIG and other groups looking at it decided against it and the issue is dead.

Re:Static Typing? (2, Interesting)

Surt (22457) | more than 8 years ago | (#15541548)

Well, it's not quite as bad as it sounds. He's seemingly only really forbidding incompatible mixed types in the same variable, a usage that isn't exactly extremely common.

A more significant roadblock, IMO, is that he can't handle mixed types in 3+-tuples, which is very common.

Re:Static Typing? (1)

masklinn (823351) | more than 8 years ago | (#15541687)

  1. Not necessarily, some things (such as reusing a name to bind widly different objects) is frowned upon and bad style anyway (since it makes the code much less readable and maintainable)
  2. The compiler is still in a very early phase of it's development

Re:Static Typing? (3, Interesting)

MBCook (132727) | more than 8 years ago | (#15542240)

I love Python, but I hate the dynamic typing. It can be handy at times, but 99% of the time you make a variable to hold one kind of thing. Having the static typing would both improve performance (because the interpreter knew what you were up to) but would also eliminate bugs (because it would complain when I tried to set a double to "And now press...").

I'd love to see Python get optional static typing.

Re:Static Typing? (1)

mad.frog (525085) | more than 8 years ago | (#15542460)

I'd love to see Python get *required* static typing.

But then, I guess it wouldn't be Python....

Re:Static Typing? (0)

Anonymous Coward | more than 8 years ago | (#15543137)

I'd love to see Python get optional static typing.

Well, to each his own. But if that is the only thing you miss about python help is available [python.org] .

Yeah, but that's not what we need. (2, Insightful)

stonecypher (118140) | more than 8 years ago | (#15541390)

See, it's all well and good to compile python to speed it up. The problem is, people are now saying that they can write efficient code in python just because it magically translates to C++, and because this translator is faster than other python compilers.

This won't be meaningful until a converted python script is compared to efficient code written natively in C++ in the first place.

Re:Yeah, but that's not what we need. (3, Insightful)

Anonymovs Coward (724746) | more than 8 years ago | (#15541568)

I don't see your point. Some of us use python. It takes me a fraction the time to do something in python than to do it in any other language. I'm not interested in writing native C++ code because it's hypothetically faster (it's not faster if I count coding time). But I am interested in a good python-to-C++ translator. Why wouldn't any python user be?

Re:Yeah, but that's not what we need. (4, Insightful)

advocate_one (662832) | more than 8 years ago | (#15541857)

But I am interested in a good python-to-C++ translator. Why wouldn't any python user be?

no, I'd be far more interested in a good compiler to compile that python straight to machine code...

Re:Yeah, but that's not what we need. (2, Insightful)

Abcd1234 (188840) | more than 8 years ago | (#15542152)

Why? If you can convert Python to reasonably optimized C++, then you can leverage the C++ compiler to do all the machine-level optimizations, rather than reinventing yet another wheel.

Re:Yeah, but that's not what we need. (1)

advocate_one (662832) | more than 8 years ago | (#15542573)

it gives you an extra area for weird bugs to creep in... get the Python right and go straight to machine code with a trusted compiler.

Re:Yeah, but that's not what we need. (2, Insightful)

styrotech (136124) | more than 8 years ago | (#15543273)

it gives you an extra area for weird bugs to creep in... get the Python right and go straight to machine code with a trusted compiler.

Is that the same way the method of using layers of multiple simple tools that all do one thing really well is more buggy that just using one larger general purpose monolithic app?

A cross platform Python to machine code compiler would presumably need to reinvent a whole lot of difficult platform specific stuff that has already been solved by C++ compilers.

Re:Yeah, but that's not what we need. (3, Insightful)

mrchaotica (681592) | more than 8 years ago | (#15543090)

...and that's why it shouldn't be a Python to C++ translator; it should be a GCC frontend instead (i.e., translating to GCC's internal representation).

Re:Yeah, but that's not what we need. (1)

Abcd1234 (188840) | more than 8 years ago | (#15543862)

Which has already been covered elsewhere. The GCC IL isn't suitable for a dynamic language like Python.

Re:Yeah, but that's not what we need. (2)

menace3society (768451) | more than 8 years ago | (#15543378)

I'd prefer a python-to-Common-Lisp compiler, but only because I hate running out of stack space for recursive algorithms.

Re:Yeah, but that's not what we need. (1)

masklinn (823351) | more than 8 years ago | (#15541753)

The problem is, people are now saying that they can write efficient code in python just because it magically translates to C++

No they aren't, if only because ShedSkin doesn't handle modules yet and Python without modules is not really useful.

and because this translator is faster than other python compilers.

Last time I checked, it was the only Python compiler... (CPython is an interpreter, PyPy is also an interpreter, I'm pretty sure Vyper was also an interpreter [written in O'Caml before it got trashed though], JPython and IronPython can hardly be called compilers)

Re:Yeah, but that's not what we need. (2, Informative)

pthisis (27352) | more than 8 years ago | (#15542171)

Last time I checked, it was the only Python compiler... (CPython is an interpreter, PyPy is also an interpreter

Neither CPython nor PyPy is a strict interpreter, both of them compile source to byte-code and then act as a virtual machine to run that byte-code. PyPy also does some work on compiling to native code on the fly, depending on which version you're using (Armin Rigo's is the most sophisticated on the JIT/native code front, but it's far from stable).

Re:Yeah, but that's not what we need. (1)

masklinn (823351) | more than 8 years ago | (#15542282)

They're still not compilers. And CPython + Psyco is not a compiler either.

Re:Yeah, but that's not what we need. (1)

Schraegstrichpunkt (931443) | more than 8 years ago | (#15543436)

Neither CPython nor PyPy is a strict interpreter, both of them compile source to byte-code and then act as a virtual machine to run that byte-code.

I think that falls under the standard definition of "interpreter".

Even ancient versions of MS BASIC were bytecode-oriented. They had to be to fit any decent-sized program into the limited RAM available.

Re:Yeah, but that's not what we need. (1)

try_anything (880404) | more than 8 years ago | (#15541958)

Yep, every new tool is just an opportunity for idiots to screw up in new ways.

What a gloomy way of looking at the world.

Very nice, but... (1, Interesting)

MikTheUser (761482) | more than 8 years ago | (#15541404)

Since Python is about the only language I know very well, I find this fascinating. But it also reminds me of the .NET development suite, where the way you write your code in any language doesn't matter, since it all become one machine code. So if you think you can do something more memory efficient in C# than in VB.NET - well, no.
So the bottom line is, the quality will depend on the quality of the converter, and that's not so cool. Adding layers between code and machine code is not the best way IMHO.

Re:Very nice, but... (1)

jallen02 (124384) | more than 8 years ago | (#15541657)

Except that in .NET it all becomes MSIL, not Machine Code.

Jeremy

Re:Very nice, but... (2, Insightful)

rjshields (719665) | more than 8 years ago | (#15542621)

MSIL is machine code for a virtual machine rather than a physical one. This distinction makes no difference to the point the GP was making.

Re:Very nice, but... (2, Informative)

cnettel (836611) | more than 8 years ago | (#15542355)

Well, C# has unsafe arrays, while VB.NET only exposes them quite indirectly through the marshalling API. Some other language implementations also uses some dose of reflection/late binding to implement certain features. You can sometimes avoid use features, but this will sometimes result in code that is "non-idiomatic" in that language. I like the .NET framework, but it's no panacea for a language-agnostic future.

Native code (3, Insightful)

Roy van Rijn (919696) | more than 8 years ago | (#15541419)

This is a good step to make Python run a bit faster, but I don't think it'll really make a huge difference.

The best way to get some speed and still keep the nice Python functions and layout is just to export the most heavily used functions to native code (C/C++).
I don't know if its possible to take the C++ output and optimize it seperatly, that way you will have a good start to make native code though.

In short: Better, fast and easy, but not the best (if you can write native code)

Re:Native code (1)

Surt (22457) | more than 8 years ago | (#15541678)

I think this sort of tool serves a different purpose. If you have an evolving program, one that needs some speed, but also needs rapid development, then hopefully what this allows is that you do not need to write the heavily used functions in c++, but instead can translate and compile each version of your python program as part of your tool chain. Your strategy makes more sense as your project reaches maturity and stability, whereas this sort of technique is more effective for situations where performance is important, but there is also still rapid development going on.

Re:Native code (0)

Anonymous Coward | more than 8 years ago | (#15541846)

The best way to get some speed and still keep the nice Python functions and layout is just to export the most heavily used functions to native code (C/C++).

Well, no, the best way is to use a better algorithm.

Very interesting... (3, Informative)

FuzzyDaddy (584528) | more than 8 years ago | (#15541460)

This is a very interesting development, both from the practical promise and just 'cause it's cool. However, as a python programmer myself, it's not yet in a usable form. Much of the efficiency of programming in python is the standard libraries (in particular Tkinter for user interfaces), and the non-standard libraries (for example, the serial port library). This project does not yet support these.

Among python programmers, I'm curious - how many use psyco (another python performance enhancement tool) for their projects? I fiddled with it a while ago (it didn't work because of a C module that it didn't like), but never had a compelling reason to go back to it. Performance optimization has never been important enough for my applications to merit the effort.

Re:Very interesting... (2, Interesting)

tcopeland (32225) | more than 8 years ago | (#15541594)

> However, as a python programmer myself, it's not yet in a usable form

Yup. Along the same lines, Ruby has a related project by Ryan Davis, Ruby2C [rubyforge.org] . It's useful for small localized speedups, but you wouldn't want to try to write your entire app in it.

Re:Very interesting... (1)

masklinn (823351) | more than 8 years ago | (#15541813)

A friend used it once because he was generating Fractals in python and Psycho significantly sped up his script (77% gain with his first algorithm [run time dropped from 197s to 46.52s], 98% with his second algorithm [runtime dropped from 154s to 13.4s])

Re:Very interesting... (3, Interesting)

zhiwenchong (155773) | more than 8 years ago | (#15541863)

It's all a matter of magnitude.

I use Psyco in my work. My app is a code generator that processes multiple models and transforms them into optimization code. Psyco reduced the time it took for process 1 model from 20 seconds to 2 seconds. It doesn't sound like much, but when you have to do it for lots of models, the speedup suddenly becomes quite substantial.

Re:Very interesting... (1)

Just Some Guy (3352) | more than 8 years ago | (#15542038)

I'm curious - how many use psyco (another python performance enhancement tool) for their projects?

<aol>me too!</aol> Much of my code spends the majority of its time waiting for database queries to finish, ImageMagick to finish doing its stuff, files to copy, etc. Psyco doesn't do a thing for that stuff. On the other hand, a small amount of my code is pretty CPU intensive - not enough to break out of pure Python into Pyrex or anything else, but enough to want a performance upgrade. For those modules, psyco is the clear choice since I can point it at pre-existing code and get a nice boost for free.

Lots of cross-language compiling... (1)

tcopeland (32225) | more than 8 years ago | (#15541481)

...kind of reminds me of the Google Web Toolkit [google.com] which is more or less a Java to Javascript/HTML compiler. It's not an optimization thing like ShedSkin, instead it lets folks use the Java skills they already have to write better web apps. I wonder what they use to parse the Java code? I don't see any mention of JavaCC [java.net] on their site, or ANTLR either for that matter...

I'm confused... (4, Interesting)

advocate_one (662832) | more than 8 years ago | (#15541823)

surely the best way to speed it up is to compile it straight to object code... c++ has to be compiled and just adds an intermediate step which will make things harder to debug...

Re:I'm confused... (2, Interesting)

Dasher42 (514179) | more than 8 years ago | (#15543393)

I think that the best example of what you're saying would be the Java compiler in the gcc suite. That separate front-end, back-end approach of gcc is terribly helpful.

And yet, if you're going to compile Python, I'd want the translation into source code. If it's worth rewriting in C++, it's worth tuning, especially if you can improve the usage of type-safe code.

If they can do this... (1)

Ant P. (974313) | more than 8 years ago | (#15541900)

...why not make it into a GCC frontend so Python can be compiled directly?

Re:If they can do this... (1)

sploxx (622853) | more than 8 years ago | (#15541943)

...why not make it into a GCC frontend so Python can be compiled directly?
Probably because it is fairly hard to translate a dynamically typed language into the RTL GCC uses for its backends.

I think they had a lot of problems even 'only' with C++ and all its powerful constructs like templates etc.

Re:If they can do this... (1)

Overzeetop (214511) | more than 8 years ago | (#15541972)

We must not be programmers; I though the same thing. The only advantage is that you could go in and tweak the C++. But if you can read a machine compiled code well enough to make "corrections", wouldn't you just code it that way to begin with?

On second thought, it could be to allow compilation on different platforms. Write once and precompile from Python, then compile for each system you prefer, using the system-specific compiler to optimize for the processor architecture. Of course, I'm just guessing. Hell, the closed I've gotten to Python is a /. recommended intro in a three ring binder on my nightstand. I use it when I have trouble sleeping - it's cheaper than prescription meds.

Re:If they can do this... (5, Insightful)

rpwoodbu (82958) | more than 8 years ago | (#15542682)

It is worth mentioning that one of the the original implementations of C++ (if not the very first) was "cfront", a C++-to-C converter. I see this as a much easier way to get a new language implemented quickly, as you can take advantage of the common functionalities already implemented in the target language of the converter. Although Python is not a new language, using it as a compiled language is new, and thus I believe it is comparable to being a new language for this argument. C++ and Python have a lot in common, which makes C++ a very suitable target language for a Python-to-[compiled_language] converter.

If this converter proves to be successful, I believe that a GCC frontend will be written eventually. There are probably potential optimizations that would be difficult or impossible to implement any other way.

Some may think that the dynamic nature of Python may preclude its inclusion in GCC. Technically, all that would need to be done is to have a runtime to handle dynamic things, similar to how Objective-C (for which there is GCC support) has a runtime to handle message passing and late binding. However, a large portion of the potential efficiency of a compiled version of the language would be lost to these dynamic capabilities; luckily, a compiler can detect when things are implicitly static (in fact, this converter is limited to implicitly static constructs), and optimise them to be truly static at compile-time.

File as NBNC (Nice But No Cigar) (4, Insightful)

suitepotato (863945) | more than 8 years ago | (#15541953)

Why? Read the linked page? Says it all. Violates most any Python code of any complexity out there. So if it doesn't convert Python code from the real world, what is it for? Making Python coders learn enough about C++ to remember the limitations and write/rewrite Python code to use it?

What the Python C/C++ interested people REALLY need is a book written by a group of Python AND C/C++ masters which teaches the two simultaneously showing complimentary methods of doing any given thing working from beginner to advanced and I DON'T mean "How to turn your n00b Python code into C/C++ hotness" sort of viewpoint. I mean both taught simultaneously in synch showing how they can interchange and compliment.

Software tricks for converting? Ultimately worse than not having them because it leads to horrible obfuscation because we don't know exactly what is going on when 13,412 lines of Python is turned into C++ because WE DIDN'T WRITE IT AND WE NEVER LEARNED C/C++. "Say Mike, that's great but you're the company code cowboy and you don't do C++ natively and I sure as hell don't read it being management so exactly what happens if this needs to be fixed? We've gone from importing open source code you couldn't read to writing our own open source code you can't read."

Re:File as NBNC (Nice But No Cigar) (2, Informative)

hubritc (770594) | more than 8 years ago | (#15542798)

The way this will be used by pythonistas is not to convert 13,412 lines of code blindly in to C++. Rather, it provides a pythonic way of getting some speed benefit for those parts of the program that need it and that code will also be accessable to C++ programs as an added benefit.

Re:File as NBNC (Nice But No Cigar) (1)

Schraegstrichpunkt (931443) | more than 8 years ago | (#15543473)

Why? Read the linked page? Says it all. Violates most any Python code of any complexity out there. So if it doesn't convert Python code from the real world, what is it for? Making Python coders learn enough about C++ to remember the limitations and write/rewrite Python code to use it?

And when Linux was still at 1.x, it should have been dismissed by business because it didn't support SMP.

The software is barely written. Have patience.

C++front? (0)

ClosedSource (238333) | more than 8 years ago | (#15542447)

Somebody had to say it.

Human language analogy (0)

Anonymous Coward | more than 8 years ago | (#15542678)

Writing a program in C++ and compiling it with a C++ compiler is like writing poetry in German and using babelfish to convert it to English.

Writing a program in Python, then converting it to C++, then compiling with a C++ compiler is like writing poetry in French, then using babelfish to convert it to German, then using babelfish again to convert it to English.

If it's not obvious, then the fewer levels of translation you have, the better your output will be: "The vodka is good, but the meat is terrible."

Re:Human language analogy (0)

Anonymous Coward | more than 8 years ago | (#15542902)

Reading your post is like trying to translate to English something written in stupid.

C++ Proto-typing (1)

oblivion95 (803698) | more than 8 years ago | (#15543533)

Python is by far the best language for proto-typing code intended to be written in C++ later. ShedSkin facilitates this process.

Not enough people proto-type their code, which is why hardly anybody talks about how to do it.

Speaking from experience... (0)

Anonymous Coward | more than 8 years ago | (#15543568)

Writing a interpreted language to compiled language converter is a fairly simple step that can yield an impressive speedup with very little effort. However, doing so makes you dependent on a separate compiler, and it can also force you to make unwise decisions when you attempt to coerce dynamic language features into a static language. Clearly the "Shed Skin" compiler is in its early stages, and its author is currently favoring the removal of Python features in order to accomplish his goal. That's a shame, because it need not be so. Hopefully his later attempts will support the full range of Python syntax and semantics.

A couple years I wrote a specification for a dynamically typed language with first order functions. Over a period of 3-4 months, I wrote an the interpreter in C++, and then as an experiment I made a bytecode-to-C++ converter. That first prototype was about 3x faster than the already blazing fast interpreter, but I didn't want to have to depend on gcc, so my next step was to write a bytecode-to-x86 converter. That allowed the intepreter to compile down to raw machine code in the same pass as the bytecode verification. With the JIT in place, its performance compared favorably to an unoptimized C compiler, even for tight loops and recursive function call benchmarks like the Ackermann function. Keep in mind that we're talking about dynamically resolved types as loop counters and first class function objects that carry writable closure state from their parent functions, so this is quite a bit more impressive than it might seem at first glance.

At the time, my little unoptimized JIT compiler would have easily made the top 10 on the language shootout page. With the benchmarks normalized to show C, C++ and Ocaml as 1.0, my simple JIT compiler was about a 4-5, considerably faster than my interpreter which ran at about 30-40. Meanwhile, all the other interpreters including perl, python, ruby, lua came in at about 60-200 (*note: my first attempt at the interpreter ran in the 150 range -- about the same as perl).

My experience convinced me that it should be fairly easy (read: only a couple months of work) to speed up most interpeted languages by at least a factor of 10. For some of the dogs out there, you could probably make it closer to a 40x speedup. Granted, it would take a few days to a week to write the machine code driver for each new platform; however, the benefit of not having to run a separate (slow) compilation phase would outweigh that negative, IMHO.

Unfortunately though, some languages constructs are just inherently slow. For example, perl and python both treat their objects as string-based lookup tables, so any code that uses that functionality will be limited by the access speed of the lookup table -- potentially close to O(1) in the average case but no better than O(lgN + length of string) in the worst case. As another example, perl uses a list construction for its function arguments instead of a vector. List allocations require significantly more memory than arrays, and their indexed lookup time is O(N) instead of O(1).

Finally, as the "Shed Skin" compiler author realized (and dutifilly noted that his implementation ignored), there are serious performance penalties that come with allowing container objects to hold dynamic types. These penalties really can't be worked around unless you add optional static typing or simply ignore the feature. I prefer optional static typing, because dynamic typing is way too valuable of a tool to remove from a language.

Python as prototyping language (3, Interesting)

radtea (464814) | more than 8 years ago | (#15543569)


Python is a terrific prototyping language (and lots of other things besides.) As a C++ coder I've been using it for prototyping stuff that will eventually be integrated into a larger application and therefore MUST be translated to C++. So what I'd like to see is a tool (written in Perl, just for the fun of having a linguistic threesome) that just does a light gloss on Python syntax to get me most of the way to human-readable C++. That would be far more useful (to me) than thsi thing, which sounds more like f2c, whose output could case brain damage in humans and cancer in rats, or possibly the other way around.

Buisness Proposition (1)

JimXugle (921609) | more than 8 years ago | (#15543819)

1) Download Bittorrent
2) Download Shed Skin
3) ???
4) Profit!

Ever heard of cfront? (1)

Ritchie70 (860516) | more than 8 years ago | (#15543967)

I'm amused that the same mechanism that was originally used to implement C++ (a precompiler that, in that case, generated C code) is now being used with C++ as the "low-level" language with a readily-available compiler.

Surely some of you remember cfront? Generated some truly bloated, completely unreadable stuff, but humans weren't supposed to read its output - cc was.
Load More Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>