×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Van Rossum: Python Not Too Slow

Soulskill posted more than 2 years ago | from the prefers-terms-like-stately-or-majestic dept.

Python 510

snydeq writes "Python creator Guido van Rossum discusses the prospects and criticisms of Python, noting that critics of Python performance should supplement with C/C++ rather than re-engineering Python apps into a faster language. 'At some point, you end up with one little piece of your system, as a whole, where you end up spending all your time. If you write that just as a sort of simple-minded Python loop, at some point you will see that that is the bottleneck in your system. It is usually much more effective to take that one piece and replace that one function or module with a little bit of code you wrote in C or C++ rather than rewriting your entire system in a faster language, because for most of what you're doing, the speed of the language is irrelevant.'"

cancel ×
This is a preview of your comment

No Comment Title Entered

Anonymous Coward 1 minute ago

No Comment Entered

510 comments

007087 (5, Insightful)

Anonymous Coward | more than 2 years ago | (#39382529)

Title is kinda silly.. as the basic referenced statement is that in some cases python _is_ too slow but that one can work around that using hacks (or a language agnostic component oriented architecture).

As for:

You said that if you trust your compiler to find all the bugs in your program, you've not been doing software development for very long.

It’s not about finding all the bugs, or even many of them. It’s about another layer where a potential bug can be caught. Runtime bugs are the worst kind as they can sit dormant for a while if in a rarely traveled branch. The more checking that can be done at the compile level, the better (imo).

Personally my biggest complaint about python wasn’t on the list: A lot of the (common) libraries out there are poorly documented, inconsistent, buggy, or incomplete.

As a Gentoo user, the python 2/3 thing is also especially annoying. Obviously this isn’t really python’s fault.. but it still gives me a bad taste about python.

That said, this was a great article.. short, to the point, and the answers were pretty good!

Re:007087 (0, Flamebait)

stanlyb (1839382) | more than 2 years ago | (#39382623)

And just as a comment, if you are smart enough to pinpoint this critical little piece of code, which you could write in C/C++......then your skills would be pretty much enough to write the same program on C/C++, and i dare to say, even with less skill set. So, what was the point again of learning Python!!!

Re:007087 (2, Insightful)

MightyMartian (840721) | more than 2 years ago | (#39382721)

No fucking kidding. "Python isn't slow, especially when you rewrite the Python function causing the problem in C..." WTF do these people come from?

Re:007087 (-1, Troll)

pe1rxq (141710) | more than 2 years ago | (#39382837)

You are surprised????? This is the guy that thought whitespace should be part of the language!!!!!

Re:007087 (2)

madprof (4723) | more than 2 years ago | (#39383231)

Yeah but on actually using the language this isn't an issue. It certainly works for me.

Re:007087 (5, Insightful)

Anonymous Coward | more than 2 years ago | (#39382839)

This thread is idiocy. The point is, you can write the code in python several times in the same amount of time it takes to write it in C or C++. So if you then spend a fraction of that to optimize it, you write a very small portions in C/C++, you still wind up WAAAAY ahead.

These people come from a place where they understand the value of coders all that it entails. A better question is, where do you come from that you don't have to deal with the reality of programming in the real world?

Re:007087 (-1)

MightyMartian (840721) | more than 2 years ago | (#39382941)

Yes, in the real world you have to sometimes dip your toes in a lower level to optimize. But I'm not deriding that notion, I'm deriding the apologetic for Python issues.

Re:007087 (5, Insightful)

Anonymous Coward | more than 2 years ago | (#39383149)

Also most people don't realize these were the same arguements given back in the 80s for writing stuff in assembly instead of C, and given that people have moved on to C++ since with the same things being said 'If it's too slow in C++ then optimize that routine in C', I would say this explanation is correct... ASSUMING python offers good enough profiling capabilities to pinpoint hotspots in the code so that you CAN optimize them away.

That said as the GGGP stated, gentoo's usage of python in a somewhat slow and half assed manner is frustrating (but if you compare it to ANY rpm based distro it's still faster than heck. Be it a 200 mhz or 3ghz processor. Most of the speed issues seem to be pertaining to sleep states rather than actual code anyways).

Re:007087 (2, Insightful)

jedidiah (1196) | more than 2 years ago | (#39383395)

People still optimize to assembler. That idea never went away. So pointing out that we've "moved on" to C++ or Java doesn't really mean this kind of issue has gone away.

Some stuff still remains performance sensitive enough that people will go to the trouble of optimizing it at a lower level.

Re:007087 (5, Insightful)

lgw (121541) | more than 2 years ago | (#39383385)

The point is, you can write the code in python several times in the same amount of time it takes to write it in C or C++. So if you then spend a fraction of that to optimize it, you write a very small portions in C/C++, you still wind up WAAAAY ahead.

I have to disagree with that, a bit. C and old-style C++ has a lot of boilerplate allocate/release stuff going on, and yes that really slows you down making sure you got it right. But modern C++ RAII-style auto-everything code isn't like that. I find modern-style C++ and Java equivalent in the time it takes to solve the actual problem.

Python is sometimes faster to write in. If you're doing a lot of parsing, for example, it's great. If the total size of your Python codebase remains small, that really helps. But once you start writing very formal Python where the type of every argument is declared in comments, and error handling being done with exacting precision and logging, and so on, you might as well be writing in C++ or Java.

It's really not so much the language that makes adding code to a large code base slow! It's the need to obsess over geting the details of your thought process recorded in the code once you pass the point that any one person could possibly understand the whole codebase, or it's old enough that the original authors have all left.

Wrting code for small projects is what's (potentially) fast - and Python is great for realizing that potential.

Re:007087 (2, Insightful)

Anonymous Coward | more than 2 years ago | (#39382877)

Both of you guys are goddamned idiots. The fucking language is designed to seamlessly use libraries written in C. Can you not really in your infinite arrogant wisdom not see how a ridiculously easy language like Python would be something to use to write all of an application save for that one little spot that needs just a little more performance? Of course, I know what you're thinking, "fuck that, I'm teh l33t; why would I use a language that uses 1/10 the code to do the job when I can just do it The Hard Way(TM)." Fucking morons.

Re:007087 (0, Troll)

MightyMartian (840721) | more than 2 years ago | (#39382915)

As the GP pointed out, if you're skilled enough to write optimized code in C/C++, why fuck around with Python at all?

Re:007087 (5, Insightful)

nedlohs (1335013) | more than 2 years ago | (#39383021)

Because it's significantly faster in terms of development time, or in terms of cost (having the less skilled, cheaper programmers work on it).

It's always been a case of write the critical stuff in C/C++. That's why languages like python and perl make doing so so simple.

Re:007087 (4, Informative)

buchner.johannes (1139593) | more than 2 years ago | (#39383027)

As the GP pointed out, if you're skilled enough to write optimized code in C/C++, why fuck around with Python at all?

Because we don't want to spend our time thinking about pointers and how to iterate over things? Because functional programming is actually really nice? Because in Python, you can download some data from the web, analyse it using a machine learning algorithm, plot the results, and install another package on the fly, combining 4 independent packages, and many ideas, in just 50 lines of code.

ctypes is really easy to use and to interface with C or Fortran. I use it a lot, namely for the 1% of the code that takes 99% of the time. The rest is nice OOP and functional.

Garbage Language For Shit Coders Like You (-1, Troll)

Anonymous Coward | more than 2 years ago | (#39383197)

At least we now know where all the idiots who clung to previous garbage languages like Perl have ended up...

Re:Garbage Language For Shit Coders Like You (1)

madprof (4723) | more than 2 years ago | (#39383259)

Ah, that is a really good troll. :) I am actually quite impressed. :)

Re:007087 (1)

Teckla (630646) | more than 2 years ago | (#39383331)

Because we don't want to spend our time thinking about pointers and how to iterate over things?

Because the only alternative to Python with higher performance is C or C++, and all of those non-existent alternatives have all the same pitfalls as C and C++?

*sigh*

Re:007087 (1)

RoccamOccam (953524) | more than 2 years ago | (#39383039)

Because it takes quite a bit more time to do it without Python? I'm not sure why that point is not coming through.

Re:007087 (3, Insightful)

Grishnakh (216268) | more than 2 years ago | (#39383061)

It's not about skills, it's about developer time. It generally takes longer to write code in C or C++ that to write something to do the same thing in a higher-level language like Python (C much more so than C++ I would think, especially if you're doing anything involving strings; C really sucks for that stuff; C++ is much better, esp. with a good library like Qt).

Another factor is that you might be working on a larger project, and not everyone is all that skilled. So you can put the Python monkeys to work on some parts, and put the C or C++ people to work on the performance-critical sections, and merge the two together.

Re:007087 (3, Informative)

Anonymous Coward | more than 2 years ago | (#39383195)

... because given two equally talented developers, the one working in python will literally run circles around the guy working in C/C++... and in the real world developer time = money.

The fact is that interfacing with C libraries in Python is already trivial. Furthermore modern tools like Cython make it EVEN EASIER!

So you take your code, profile it, decorate the most time-intensive portions to be compatible with Cython (trivial for most applications) -or- interface it with your C library. This way, you get your code up and running in a fraction of the developer time, with near identical performance to the C implementation.

Re:007087 (3, Informative)

vgerclover (1186893) | more than 2 years ago | (#39383255)

As most things in life do, code usually follows the Pareto distribution: 80% of the time is spend in 20% of the code. If Python is fast enough for, let's say 90% of your code, and you are much more productive in Python than C, then writing most all the code in Python first, and replacing the bits and pieces that are too slow for you with C functions, is much more efficient use of your time than writing everything in C.

Also consider that sometimes you have to go in fishing expeditions for the correct algorithm to do what you are doing. Doing so in Python, with the speedup in iterative design that that carries, and even if that once you find the most efficient algorithm for your problem you implement it in C, you will have had spend the same time as before, but knowing all the ways you can't do it, and have arrived to an at least nearly optimal solution.

Most of the time you don't need that much speed. When you do, you have to have the right algorithm and the right language. I also put forth that Python has a lot of modules that are already written in C, so you take advantage of existing optimized code that you don't have to write.

Re:007087 (1)

Anonymous Coward | more than 2 years ago | (#39382795)

I can see the argument being made, though I've made it more with java/c++. It's not about skill set, it's about tool set/existing infrastructure and so on. If a system is primarily built on Java and uses an extensive set of frameworks / libraries.. it really doesn't make sense to re-write the whole thing because there is _one_ section where java is the worst possible language for the job.

That said I always consider this type of approach a last result.. and in general, I would vastly prefer to write the whole thing in c++ if I anticipated performance being an issue .. especially on a new project.

Re:007087 (1)

errandum (2014454) | more than 2 years ago | (#39382803)

But python brings some things to the table that other languages don't have, like some of those libraries that, as you said, are not well documented, but most of the time do what you want and you don't have to worry about memory management, pointers, etc. while having access to dictionaries out of the box.

I'm all for writing the program in whatever language you feel more comfortable on and just do the heavy duty stuff in c++ daemons or something like that.

Re:007087 (1)

Anonymous Coward | more than 2 years ago | (#39382815)

And just as a comment, if you are smart enough to pinpoint this critical little piece of code, which you could write in C/C++......then your skills would be pretty much enough to write the same program on C/C++, and i dare to say, even with less skill set. So, what was the point again of learning Python!!!

Not necessarily. There are quite a lot of use cases where you may want to write a relatively small inner data-processing loop over a bunch of arrays in C or C++, but you don't want to have to implement all of the GUI, networking protocols, I/O, configuration management etc. in C or C++. In order to turn a proof of concept into a usable tool you can take a lot of time developing the peripheral stuff, but the critical requirement for that sort of code is flexibility rather than raw performance. That is a place where Python might be ideal to minimize development time and cost. Python sets up and loads all of the data and parameters in an appropriate way and then a C function carries out the grunt work before Python formats and displays the results.

Re:007087 (5, Insightful)

Anonymous Coward | more than 2 years ago | (#39382847)

Python, apart from the inconsistent standard of the libraries that the GP mentioned, is brilliant for the 'glue code' that holds small to medium sized programs together. Ruby may have the edge for organising larger programs.

Rewriting a relatively small Python program in C++, one that does a fair bit of file handling, string chopping about etc, can make it into a relatively large program. Doing it in C is asking for pointer problems. I've been programming in C from before C++ existed, C++ since then and Python for about 5 years.

I am quite capable of writing the glue code successfully in any of the languages, but to do it efficiently I'd choose to do exactly as TFA says - write the main bulk of the code in Python( (or Ruby, or some other higher level language), then measure it to find the real slow bits (you wouldn't optimise until you have measured where the problems REALLY are, would you?) and rewrite just those bits in a lower level, faster language.

Of course, once you see where the speed issue lies, it's just as likely you can find an algorithmic improvement that fixes it 'enough' anyway...

Re:007087 (4, Interesting)

sjames (1099) | more than 2 years ago | (#39383003)

I have been coding in C for quite a while. Everything from simple user programs through kernel code, drivers, and firmware. I find that python is quite liberating. It allows me to accomplish a lot more faster and have it tend to just work. I do perform the analysis and occasionally re-implement a performance critical function in C, but I find that it's surprising how little that actually improves most situations, not because of a deficiency in Python, but because it wasn't actually all that slow.

Yes, I could implement the cool handling of various data types Python has in C, but doing so will produce something that looks a LOT like a poorly tested and less optimum implementation of Python.

Re:007087 (0)

Anonymous Coward | more than 2 years ago | (#39383063)

Not about skills. About time.

Re:007087 (0)

Anonymous Coward | more than 2 years ago | (#39383133)

True, if you are skilled enough to rewrite that piece of critical code in C/C++, so why not do the entire code in C/C++? The idea behind this philosophy (IMO) is to reduce coding time, as opposed to having to spend time writing everything in C/C++, which depending on your requirements may be daunting, or simply a drag, then you can simply write everything in Python, then while testing if you see anything that is running too slow, and there is no native way of speeding it up, then just identify the problematic code, and rewrite just that piece in C/C++ and be done with it. Simply put, Python is language that is powerful, and allows you to get things done quickly, and it should be taken as a bonus that you can branch out to another language, and exploit its speed/functionality.

Re:007087 (5, Insightful)

Pieroxy (222434) | more than 2 years ago | (#39382707)

And to add to that:

Python Not Too Slow

True. It's not too slow. It's just not fast enough.

Re:007087 (0)

Anonymous Coward | more than 2 years ago | (#39383191)

Obviously this isn’t really python’s fault

Except for the decision to make an incompatible change in the language, but...

Re:007087 (5, Insightful)

tomhath (637240) | more than 2 years ago | (#39383229)

as the basic referenced statement is that in some cases python _is_ too slow but that one can work around that using hacks

You completely missed what he said, which is "use the best tool for the job".

Most apps can be written much more quickly in Python than C/C++. If they perform adequately you're done. If there are slow spots, use an extension (it's not a hack) to optimize that tiny part of the app. This has been SOP since compiled languages were first invented; we used to write that last 5% of the app in Assembler. But obviously using C/C++ is more productive than Assembler and usually fast enough, just as using Python is more productive than C/C++ and usually fast enough.

Coincidentally (1)

Anonymous Coward | more than 2 years ago | (#39383233)

"a bad taste about python"

Coincidentally, "it tastes like python" ['det smakar pyton'] is a Swedish idiom for something that tastes really bad.

Yeah, right (0, Flamebait)

stanlyb (1839382) | more than 2 years ago | (#39382551)

Like what, calculator? Or Spreadsheet? Or gmail.com? Now i see why Google does not have any C/C++ developers

Re:Yeah, right (1)

WilCompute (1155437) | more than 2 years ago | (#39382955)

Wait, Google doesn't have any C/C++ developers?! But, how do they contribute to projects like LLVM and Linux. How did they create the Native Client for Chrome?

There abilities just went up by major points in my book!

O_o

Van Rossum: Python Not Too Slow (-1)

Anonymous Coward | more than 2 years ago | (#39382567)

because you can never have too much "slow!"

Language Philosophies (1)

Anonymous Coward | more than 2 years ago | (#39382595)

How much of Python is just straight-up "slow", and how much comes from the fact that Python devs prioritize readable code over efficient code? In C, you usually try to squeeze as much memory out of each line as possible, but Python is designed to be human-readable first and machine-executable second.

Re:Language Philosophies (1)

timothyb89 (1259272) | more than 2 years ago | (#39382813)

Strictly speaking, the language itself shouldn't have any effect on how fast it executes, it's the implementation that really matters. Some languages might be more difficult to parse but in the end it's what the interpreter does with it that really matters. The whole sentiment that "fast code equals C/C++" is a little fishy to begin with, modern interpreted languages compile down to machine code via JIT anyway and often don't have a significant performance decrease compared to the same code in C/C++. Not that I'm against the notion completely, as native code (and specifically native code modules embedded in other languages) has its benefits, but it shouldn't be used as an excuse for a slow interpreter.

So.. (-1)

Anonymous Coward | more than 2 years ago | (#39382603)

About time for a new project leader I'd say. I mean this really doesn't help his case after the 3.0 fiasco...

Agreed. (5, Insightful)

Anonymous Coward | more than 2 years ago | (#39382611)

Now that that is settled we can get back to the real problem with python: Type errors.

Re:Agreed. (0)

Anrego (830717) | more than 2 years ago | (#39382661)

That was covered in the article as well (kinda) .. with the same kind of "just do more testing" type answers you hear from the PHP guys.

Personally I like my languages strongly typed, with as much idiot proofing and compile time checking as I can get and still have a usable language. Not as a substitute for testing/QA as the article would imply.. but as an additional layer.

Re:Agreed. (0, Flamebait)

thetoadwarrior (1268702) | more than 2 years ago | (#39382855)

As someone who's worked with Python for years both in personal projects and professionally, I've never had an issue with that. Maybe python attracts more intelligent programmers than the dime-a-dozen .Net / Java programmers being pumped out of university.

Re:Agreed. (1)

Wattos (2268108) | more than 2 years ago | (#39382981)

As someone who's worked with Python for years both in personal projects and professionally, I've never had an issue with that. Maybe python attracts more intelligent programmers than the dime-a-dozen .Net / Java programmers being pumped out of university.

just as javascript and php does ^^

Re:Agreed. (0)

Anonymous Coward | more than 2 years ago | (#39383097)

same thing C coders said about C++

Re:Agreed. (0)

Anonymous Coward | more than 2 years ago | (#39382917)

Python IS strongly typed. It isn't statically typed.

Re:Agreed. (1)

Teckla (630646) | more than 2 years ago | (#39383155)

That was covered in the article as well (kinda) .. with the same kind of "just do more testing" type answers you hear from the PHP guys.

Personally I like my languages strongly typed, with as much idiot proofing and compile time checking as I can get and still have a usable language. Not as a substitute for testing/QA as the article would imply.. but as an additional layer.

Well said. Developers should think of static typing and strong typing as testing built right into the code -- testing your boss can't make you skip writing in order to meet a deadline, because if you have type errors, your code will not even compile.

It's like they always say... (0)

Anonymous Coward | more than 2 years ago | (#39382647)

Two languages are better than one!


...Right?

Re:It's like they always say... (0)

NatasRevol (731260) | more than 2 years ago | (#39383163)

Two ladies are better than one!

3 weeks a month. Then things get slow. Just like Python.

Logically Logical Logic (5, Funny)

l0ungeb0y (442022) | more than 2 years ago | (#39382657)

"It is usually much more effective to take that one piece and replace that one function or module with a little bit of code you wrote in C or C++ rather than rewriting your entire system in a faster language"

Ahh -- yes, I see, so I should write my Apps in Python, except where they need to be rewritten in C/C++ because that will run faster than when written in Python, but Python is not slow when you rewrite portions -- so don't rewrite in a faster language because Pyton is fast enough.

Alrighty then.

Re:Logically Logical Logic (5, Insightful)

Cogita (1119237) | more than 2 years ago | (#39383033)

Ahh -- yes, I see, so I should write my Apps in Python, except where they need to be rewritten in C/C++ because that will run faster than when written in Python, but Python is not slow when you rewrite portions -- so don't rewrite in a faster language because Pyton is fast enough.

Alrighty then.

Essentially yes, that's it exactly. It's a lot simpler to write a 5000 lines of python and 300 lines of C than it is to write 20,000+ lines of C. Plus Python manages most of the memory management for you so you have less chance of memory leaks. I would argue that the reduction in bugs memory bugs and more maintainable code would justify saying that one should use two languages in this case. It's not a matter of which is better overall, it's that python is easier to read, whereas C is faster. Use both where their benefits are most powerful.

Re:Logically Logical Logic (5, Informative)

Frnknstn (663642) | more than 2 years ago | (#39383075)

Yes, that is correct. You should write your apps in Python.

Your libraries, you should write in Python first, because it is also a great prototyping language. If they work fine (which they will in most cases) you have saved yourself a bunch of time. If they are too slow, you have saved yourself a bunch of time by fixing algorithmic bugs in a flexible language like Python. It is now trivial to convert it to bug-free C or C++.

Python: Not Much Worse Than Ruby (5, Funny)

busyqth (2566075) | more than 2 years ago | (#39382663)

I'm waiting for the article:

Van Rossum: Python Not Much Worse Than Ruby
"Python creator Guido van Rossum discusses the prospects and criticisms of Python, noting that critics of Python should supplement with Ruby rather than re-engineering Python apps into a better language."

Re:Python: Not Much Worse Than Ruby (1)

Anonymous Coward | more than 2 years ago | (#39382841)

Supplementing with Ruby will make your program slower, not faster.

Re:Python: Not Much Worse Than Ruby (2, Insightful)

Anonymous Coward | more than 2 years ago | (#39382849)

That's the problem with Python. Ruby is expressive and object-oriented, C is fast. Python is neither.

Re:Python: Not Much Worse Than Ruby (2)

MattBD (1157291) | more than 2 years ago | (#39383049)

It's faster than Ruby, and it most definitely is object oriented. Expressive is rather more subjective, but I've used it to some extent and I've found it very expressive.

Kinda digging Python (4, Insightful)

XxtraLarGe (551297) | more than 2 years ago | (#39382695)

I'm signed up for the CS101 course @ Udacity, and I was surprised they were using Python for the course. It does seem a bit weird using whitespace for blocks, especially when you're used to writing stuff like
if(a > 0) { return a + 1; } else { return a -1; }
for the simple stuff. I do really like things like being able to return multiple values from a procedure, etc., but Python seems more useful for rapid prototyping rather than anything else.

Re:Kinda digging Python (0)

Anonymous Coward | more than 2 years ago | (#39382809)

return a+1 if a>0 else a-1

Re:Kinda digging Python (1)

XxtraLarGe (551297) | more than 2 years ago | (#39382913)

return a+1 if a>0 else a-1

Just starting out, so they didn't teach us that syntax. We've been shown:

if (a > 0):
return a + 1
else:
return a - 1


Pretend that there are 4 spaces before the 2nd & 4th lines, since Slashcode doesn't recognize non-breaking spaces.

Re:Kinda digging Python (0)

buchner.johannes (1139593) | more than 2 years ago | (#39383169)

return a+1 if a>0 else a-1

Just starting out, so they didn't teach us that syntax. We've been shown:

if a > 0:
        return a + 1
else:
        return a - 1
Pretend that there are 4 spaces before the 2nd & 4th lines, since Slashcode doesn't recognize non-breaking spaces.

You don't need the brackets. I'm a bit unsure about the "return a+1 if a > 0 else a-1" syntax. It's a bit harder to read. numpy.where is probably the right thing for the job if you have more than one a for which you want to calculate.

Re:Kinda digging Python (0)

Anonymous Coward | more than 2 years ago | (#39382833)

You can still write in-line in that manner, it's just not recommended. I try to avoid it for anything for complicated than one or two nested comprehensions.

Re:Kinda digging Python (1)

AuMatar (183847) | more than 2 years ago | (#39382919)

Being able to return multiple values is nice (although you can do that in C++ with a Pair template object, it isn't frequently done). Just remember there's a lot of pitfalls. For example, if you use the frequently used for loop replacement

for x in range(len(somearray)):

you're actually doing this in C

int size = length(somearray);
int array = new int[size];
for (i = 0; i size; i++) {array[i] = i;}
for (i =0; i size; i++){ j = array[i]; //do loop body on j}

If size is big, you're hosed on memory and time.

Now try and do the whitespace issue in a large company with people who aren't used to Python. It's cost me at least a month of my working life. I once spent 2 weeks debugging a 100K program where the sole author left, it turns out on 1 line out of 100K he tabbed instead of spaced.

Re:Kinda digging Python (1)

Anonymous Coward | more than 2 years ago | (#39383213)

yes, obviously you should use:

for x in xrange(len(somearray)):

or more likely you are better off with:

for element in somearray:

if you don't need to know the index of the element.

Re:Kinda digging Python (1)

AuMatar (183847) | more than 2 years ago | (#39383315)

The second I definitely agree with (and if you don't need to know the index). Or even

index = 0
for i in somearray: //do loop
    index++

If you don't need to know the index, the python style for loop is awesome. But there's a very common need to do things N times even when not dealing with arrays. Not having a traditional for loop is kind of silly.

As for xrange- I think there's a large percentage of python users who don't know xrange exists. It also has some fancy behavior behind the scenes- it takes constant memory but takes more time than a simple list iteration. The right answer in python is to use a while loop.

Re:Kinda digging Python (1)

Anonymous Coward | more than 2 years ago | (#39382945)

return a>0 ? a+1 : a-1;

Kinda what they made it for.

Re:Kinda digging Python (1)

vgerclover (1186893) | more than 2 years ago | (#39382993)

Specially considering that you can rewrite that as return a + (a > 0) - (a <= 0) or return a + 1 if a > 0 else a - 1 or a multiplicity of other options :)

Re:Kinda digging Python (0)

Anonymous Coward | more than 2 years ago | (#39383239)

Specially considering that you can rewrite that as return a + (a > 0) - (a 0 else a - 1 or a multiplicity of other options :)

I like the readability of this. A++++++++++++++

/S

Re:Kinda digging Python (1)

Terrasque (796014) | more than 2 years ago | (#39383399)

if(a > 0) { return a + 1; } else { return a -1; }

Here's yet an alternative way to write it in python :o)

return a > 0 and a + 1 or a - 1

>>> for a in range(-1,3):
... print a, a > 0 and a + 1 or a - 1
...
-1 -2
0 -1
1 2
2 3

Not the big problem (0)

Anonymous Coward | more than 2 years ago | (#39382753)

Python is slow, but that's not its biggest problem. It's biggest problem is stability. My biggest complaint with Python apps is that they tend to crash a lot. More so than any other apps on my systems. In fact, most of the apps I've seen crash in the past couple of years have been Python utilities. Every once in a while I'll consider learning Python, but then I ask myself if I really want to develop slow and crashtastic programs and the answer is no.

usually? (5, Insightful)

v1 (525388) | more than 2 years ago | (#39382811)

It is usually much more effective to take that one piece and replace that one function or module with a little bit of code you wrote in C or C++ rather than rewriting your entire system in a faster language, because for most of what you're doing, the speed of the language is irrelevant.'"

I have a lot of experience in code optimization, and I would dispute this generalization. "often" is a lot more realistic than "usually". The most common thing I see is where one particular segment of an operation is coded by someone that doesn't understand their O's and is doing something like multilevel lookup loops instead of a hash table. Fundamental mistakes in algorithm choice are the biggest "HERE is the biggest problem" issues I find.

Once you're past the stupid implementation mistakes, it goes just slightly in favor of "it's a little bit of everything" land. Something running significantly slower in one language than another often boils down to the coder not understanding how to make things scale in the chosen language. I can make C move slower than BASIC if I want to. Sometimes it's just knowing how the compiler is going to react to your structures. Little things like "roll up the loops when coding in VB" can produce an order or two of magnitude in speed improvement, and if you don't realize this you may think you're comparing identical implementations when you're not. "this language sucks!" often translates into "I don't know how to do it so it runs fast!"

My last project was reduced from 23 hrs per run to 21 minutes by a small but complex change in implementation. From there, getting it down to 4 minutes required a LOT of little changes all over the place, to nickel-and-dime it down. I'll trade you my "guy that knows how to recode it in C" for your "guy that knows how to code, and REALLY knows his compiler" any day.

Re:usually? (1)

Billly Gates (198444) | more than 2 years ago | (#39383065)

But the other guy in India did it for 1/5th the price!

Think about the shareholders, managers, and the CEO? Not not to mention that big bonus for the person who is so smart for going cheap.

Who cares about quality. How does making it better reduce the share price or pay for the CEOs kids college tuition?

Re:usually? (2)

jmerlin (1010641) | more than 2 years ago | (#39383221)

Sometimes, sometimes the language you're working in really is the bottleneck. I've written a log parser in JavaScript that runs entirely in the browser and can process >200MB of text logs with full aggregation per second. That's an enormous amount of work (it's actually bottlenecked by the disk). V8 is enormously fast when used correctly. Later, for fun, I wrote an application that deciphered simple strings (
A large chunk of that is the raw speed of C. Another (larger) chunk is because strings in JS are immutable and the algorithm is almost entirely string manipulation (fixed length, per-character manipulations). Combine language limitation + raw speed = wtf improvement.

Re:usually? (2)

jmerlin (1010641) | more than 2 years ago | (#39383245)

Wow /. really butchered that one. The strings are encrypted with a fixed-length repeating pattern of XORs. The brute-force (try every pattern) then match against a word dictionary took 8 hours. Modifying the algorithm to be smarter and to look for the pattern rather than accepting/rejecting the result lowered that to 10 minutes. But the original algorithm in C finishes in 5 seconds.

Misleading article with fluf (0, Insightful)

Anonymous Coward | more than 2 years ago | (#39382827)

Actually the only problem with python is the last question and honest Guido answer: python doesn't scale. That's where it is slow, you have these multicore machines and you can't fully use them. Having to go to C/C++ is a shame. Even cython which can speed up a lot python code is limited by the global lock of death. So yeah, people migrating to python-like languages without this drawback.

Personally (5, Informative)

ciascu (1345429) | more than 2 years ago | (#39382835)

As someone simulating fluid-structure interaction with a number of constituent models and a lot of finite element (i.e. big matrix problems; using FEniCS - fenicsproject.org), using Python makes my overall quite-long algorithm much easier to flick through. Invaluable for debugging the theory as well as the implementation. FEniCS' Python interface ties into the standard C/C++ libraries using SWIG and, in simple cases, saves me working in C++. Very clear, well-written C++ is great for this application but I find it takes considerably longer to write than clear Python.

When I hit a more intricate problem, I realized I was going to have to solve a series of FE matrices by hand (with PETSc, written in C). It turned out to be pretty straightforward to pick up SWIG, write a short module in C and a Python interface. Done! Particularly useful as I believe getting FEniCS and petsc4py to play well is tricky.

So, I'd agree - having written a C++ version of my (simpler) problem and a Python/C version of the complicated one, the latter was definitely easier, and all the rate-limiting stuff is in C anyhow.

Doubt it would be true for every situation but +1 from an FE perspective.

Purists! (3, Insightful)

macraig (621737) | more than 2 years ago | (#39382903)

To bend a cliche: Purists! Can't live with 'em, can't live without 'em!

Seriously, we live in an era when programmers are no longer bound to the use of a single language for an entire project, as was the pragmatic case once upon a distant time. Why not just use the language for each module which best suits the need? If performance outweighs simplicity of code management, then use the better performing language for that module. No language is perfectly suited for all goals, so own your chosen criteria and don't 'blame' a language creator for having different criteria.

Duh... (0)

Anonymous Coward | more than 2 years ago | (#39382939)

This is the case with all modern (java, c#, ruby) languages, you find the "simple" algorithm X isn't performing, find out the the language you are using doesn't optimize for that structurized implementation. Find out there is some memory optimized algo in C (c++) and decide to use it and then it is a measure of how well your crap integrates with the real machine code crap.

Metaphor (3, Insightful)

Ukab the Great (87152) | more than 2 years ago | (#39382947)

Arguing about whose interpreted scripting language is slower is like arguing about whose rich delicious cheesecake is less fattening. When you eat the cheesecake, you accept the tradeoff of tastiness for five minutes off your total lifespan.

Re:Metaphor (0)

Anonymous Coward | more than 2 years ago | (#39383177)

can you express that as something I understand better, like cars? TIA

world needs a better high performance language (1)

Anonymous Coward | more than 2 years ago | (#39382951)

As far as high performance languages go, we have:

FORTRAN: King of the performance hill, but so annoying to use that nobody really does outside some scientific circles.

C++: This is what everybody uses to write high performance applications, but it's a mess of special cases and annoying syntax and megabyte-long error messages from deeply nested templates.

We need a modern language, with things like functions as first class objects and introspection, but with the performance and "to-the-metal" nature of C++ when you care about designing for optimal cache efficiency and so on.

Re:world needs a better high performance language (0)

Anonymous Coward | more than 2 years ago | (#39383015)

FORTRAN is as you put it "king of the performance hill" largely because of its different aliasing assumptions compared to C++. Compilers can do more agressive scheduling and other optimizations if they can assume lack of aliasing, which (generally speaking) you can't in C++ and many other languages.

There are other minor reasons, but that one aliasing issue is a big reason why FORTRAN rules the performance roost after 50+ years. That being said, I never use it because it lacks modern language features. C++ at least *tries* a little bit, and is a good balance between non-sucky performance and a language that's not stuck in the 1950's.

Python's problem (4, Informative)

spiffmastercow (1001386) | more than 2 years ago | (#39382957)

The problem with Python isn't the speed -- he's right about optimizing with bits of C. The problem is the GIL. Without good multithreading support, I have to give up on Python for a large number of application domains.

Re:Python's problem (2)

Hentes (2461350) | more than 2 years ago | (#39383243)

Multithreading is only a problem in CPython. Other interpreters like Jython, IronPython or PyPy all have facilities for multithreading.

Isn't this what all the scripters do? (2)

istartedi (132515) | more than 2 years ago | (#39382973)

Don't most users of these scripting languages (the good ones anyway) profile and write the speed-critical sections in C or C++ anyway? That's not Python specific. It's not even specific to scripting languages. It's the same thing that C programmers do when they use inline assembly. It's like this all the way down the line. You start with rapid development at a higher level, then profile and optimize what you need.

It's still too slow, despite what he says. (4, Informative)

Animats (122034) | more than 2 years ago | (#39383017)

Says the guy whose whole life is tied up in the language, and whose project, at Google, to speed it up, crashed and burned. [wikipedia.org]

Python is slow because von Rossum refuses to cut loose the boat-anchor of "anything can change anything at any time". The straightforward implementation of Python, CPython, boxes all numbers (everything is a CObject, including an int or a float) and looks up functions, attributes, and such in a dictionary for every reference. And only one thread is allowed to run at a time. This allows one thread to dynamically patch the objects and code of another thread. Which is cool, but useless. 99.99+% of the time, there's no need for a dynamic lookup. Most program dynamism is shortly after program startup - once things are running, they don't change much. If, sometime shortly after startup, the program said "OK, done with self-modification", at which point a JIT compiler did its thing, the language would be much faster. But no. That's "un-Pythonic".

PyPy, the newer Python implementation, uses two interpreters and a JIT compiler to try to handle the dynamism with less overhead. They're making progress, but they need a very complex implementation to do it, and they're many years behind schedule.

Python, as a language, is very usable. But it's too slow for volume production. That's not inherent in the basic language. Python could remain declaration-free if there were just a few more restrictions on unexpected dynamism. By this is meant ways the program can change itself that aren't obvious from looking at the source code. For example, if a module or class could only be modified from outside itself if it contained explicit self-modification code (like a relevant "setattr" call) most modules and classes could be nailed down as fixed, "slotted" objects at compile time. The other big win is using enough type inference to decide if a variable can always be represented as a machine type (int, float, char, bool, etc.). That's a huge performance win.

Claiming that the "slow parts" should be rewritten in C is a cop-out. It makes the program more fragile, since C code can break Python's memory safety. Except for number-crunching, or glue code for existing libraries, it's seldom done.

(I have a Python program running right now which will run for over a week, parsing the street address of every business in the US into a standard format. The parser is complex enough that rewriting it in C would be a big job. There's no "inner loop".)

Re:It's still too slow, despite what he says. (1)

Hatta (162192) | more than 2 years ago | (#39383183)

(I have a Python program running right now which will run for over a week, parsing the street address of every business in the US into a standard format. The parser is complex enough that rewriting it in C would be a big job. There's no "inner loop".)

This would be a good application for Perl. Perl handles regexps about 4 times faster than Python.

Static vs. Dynamic Typing (5, Insightful)

Teckla (630646) | more than 2 years ago | (#39383045)

I was a little bit disappointed by Guido's response regarding static vs. dynamic typing:

InfoWorld: You talked about the arguments for and against dynamic typing. You said that if you trust your compiler to find all the bugs in your program, you've not been doing software development for very long. So you're satisfied with Python being dynamic?

Van Rossum: Absolutely. The basic philosophy of the language is not going to change. I don't see Python suddenly growing a static subdivision or small features in that direction.

Proponents of static typing do not claim that compilers, combined with languages that use static typing, will find all the bugs in your program. This is nothing more than Infoworld erecting a straw man and Guido knocking it down.

However, static typing does make a huge number of potential errors stick out like a sore thumb (the compiler will refuse to compile the code, and will emit appropriate error messages).

Some people (rightfully) argue that dynamic typing makes for shorter, prettier, easier code.

Some of us believe the primary concern should be correctness, and that shorter, prettier, easier code are secondary concerns -- almost always. People should think about this every time their computer crashes, or an application crashes, or something is acting up and needs to be rebooted, or they get a virus through no fault of their own, or their data gets corrupted.

Will users be thinking, "Gosh, this sucks, but I'm sure glad the programmer used a dynamic language, because it made it easier on him (the programmer)."? No, they'll be thinking, "Damn buggy programs! I just lost X (hours,minutes,seconds) of work, and now I'm frustrated!" Programming languages are a means to an end, not an end in itself. Don't be a self centered developer: the fruits of your labor are for users, not so you can write the code equivalent of poetry.

Not to mention, statically typed languages allow for easy refactoring possibilities that make it possible to fix all sorts of serious issues, including architectural ones, with reasonable effort expended. Dynamic languages, while they have made some progress in the area of refactoring, are really in the dark ages here.

I know dynamically typed programming languages are the hotness right now, and I'm sure my opinion will be hammered relentlessly, but I do ask that if you disagree, don't mod me down, but instead, bring forth a reasonable argument for a different position. This should not be a popularity contest, where the loser is not heard, no matter what side the loser is on.

Re:Static vs. Dynamic Typing (2)

TheVoice900 (467327) | more than 2 years ago | (#39383375)

As someone who's been working on a fairly large scale application suite in Python for the last 4 years I would say that dynamic typing is both a blessing and a curse during development. It certainly makes writing and reusing code easier to an extent, but it can make some things more difficult especially when you have code that deals with objects of several different types and there's the potential to get things confused.

However, I could probably count the number of times I've seen a part of our system fail in production due to a type error on one hand. The type errors usually crop up during development or in the test suite. They're more of a pain when developing than anything.

GuidoVR is wrong and this is why (1)

eminencja (1368047) | more than 2 years ago | (#39383055)

I have similar problems with Perl -- it is too slow -- and it is impossible to fix it by re-writing parts of the system in C.

Here's the catch: you manipulate large data structures (e.g. large data sets read from a database). There is a bunch of functions for processing the data. Even if you re-write some of them in C they will still have to manipulate the very same stuff -- for example: a number in Perl is a scalar; a scalar has a string representation, a numeric representation, a reference count, etc. If you manipulate it in C you still have to take care of all this crap. So what's the point in writing that in C? Perl interpreter is also written in C. (Perl C API is way harder to use than Python's but that's irrelevant.)

So, it's like saying: if your program (in whatever language is too slow), run it through the profiler, find the routine that uses 90% of the CPU time and optimize it. Anyone who has used a profiler will know that this not always that easy.

This why we are stuck with Perl (1)

eminencja (1368047) | more than 2 years ago | (#39383119)

My friend from Belgium (who speaks Dutch as the first language) said once: "oh, so the guy who created python is from the Netherlands? Well, I don't wanna touch it then".

Donald Knuth (3, Informative)

JohnWiney (656829) | more than 2 years ago | (#39383121)

Donald Knuth made this point in 1971, in his Empircal Study of Fortran Programs - virtually none of most programs has any significant effect on performance.

Re:Donald Knuth (1)

Animats (122034) | more than 2 years ago | (#39383323)

Donald Knuth made this point in 1971, in his Empircal Study of Fortran Programs - virtually none of most programs has any significant effect on performance.

That's much less true than it used to be. In batch number-crunching programs (which is what people wrote in FORTRAN), there are typically inner loops that use most of the CPU time. However, a browser, a window system, a parser, an OS, or a GUI application usually has a much flatter profile. Then the speed of the language matters.

Jon Louis Bentley ``Writing Efficient Programs" (1)

DutchUncle (826473) | more than 2 years ago | (#39383143)

Not that he was the first to say these things, but that book says them so *well*.

I hit a will with the garbage collection (1)

Balial (39889) | more than 2 years ago | (#39383237)

I use python a lot to process large string logs (hudreds of megs or a couple of gigs). The problem is it's all super quick until the working set in the garbage collector gets too big and you fall off a performance cliff. I dunno what they're doing in there, but you easily go from a minute or two run time to half an hour to an hour because of the paging.

I'm not familiar enough with other garbage collected languages and such workloads to know if this is inherent or just a problem with the Python GC. Either way, I think it's fair to say that Python is too slow under such circumstances. I'd like to see it fixed, though, rather than abandon it :)

And so it died out... (0)

Anonymous Coward | more than 2 years ago | (#39383297)

Those scripting languages guys are so stubborn when it comes to recognizing mistakes. That's not only that python guy, it's the perl and ruby community, too.

Instead of building pragmatically and fast vms and new superior copying generational garbage collectors, they say "use C or C++".
While i have to say that building a binding for your own code is not that hard, distributing them is terror. For example, it takes really much effort to get SDL for perl running on the system. And don't get me started on SDLx.
While SDL is a must-have, you don't want to have more depedencies in binaries as needed. So in some cases, you just which a bytecode interpreter
  which is a bit faster. And I have those moments, daily.

And there is a lot room for improvement due to that Euphoria benchmark:
http://www.rapideuphoria.com/bench.txt

At least, they should get up to pike or lua. And the fact, that a one-man project (Gambas) can beat all those languages in performance doesn't make it better.

I hate to say, but PHP seems to learn from their mistakes. But Python, Perl and Ruby? No way. They can be happy, if they don't extinct like Tcl did in ten year.

Monty Python: Not dead, er, too slow yet (1)

UnknownSoldier (67820) | more than 2 years ago | (#39383343)

For some reason the title makes me think of Monty Python's Holy Grail, the "Not dead yet" line.

Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Sign up for Slashdot Newsletters
Create a Slashdot Account

Loading...