Beta

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Beginning Python Visualization

samzenpus posted more than 5 years ago | from the read-all-about-it dept.

Programming 46

aceydacey writes "Sometimes a picture is worth a thousand words. Beginning Python Visualization: Creating Visual Transformation Scripts, published in February 2009 by Apress, shows how Python and its related tools can be used to easily and effectively turn raw data into visual representations that communicate effectively. The author is Shai Vaingast, a professional engineer and engineering manager who needed to train scientists and engineers to do this kind of programming work. He was looking for a tutorial and reference work, and unable to find a suitable text, wound up writing his first book. He writes in the easy and clear style of someone comfortable and engaged with the subject matter." Keep reading for the rest of aceydacey's review.The book uses several very specific examples that illustrate general principles.

The first example is using GPS data. By using Python one can extract data from GPS receivers and enter it into the computer and manipulate it to do what one wants including creating graphs and charts. In this section he shows how to use CSV, comma separated values, as a most useful file format. He shows show to extract data from real world GPS devices and import it via serial ports and the PySerial module. It would be easy for the reader to duplicate and extend this project.

The heart of the book is coverage of useful examples utilizing MatPlotLib, NumPy and SciPy. These related tools are easy to use and fully integrated with Python. MatPlotLib is for plotting data and graphs, including interactive graphs and image files. NumPy is a powerful math library comparable to commercial tools like MatLab, and SciPy extends NumPy to for the sciences. Examples are numerous and include signal analysis using Fourier transforms.

There is also a section on Image Processing using PIL, the Python Imaging Library. This is used for relatively simple image cropping and sizing and also for bit by bit image processing. Interpolation and curve fitting are also well covered. For anyone wanting an introduction to graphical analysis of statistical data, this would be an excellent resource.

The author is obviously a professional in this field. He has a knack for good organizational style and a pragmatic approach to the work. In the book he says "Most of the time, research is organized chaos. The emphasis, however, should be on organized, not chaos." A real value I got from the book is a better understanding of data files, format, and organization as well as methods and guidelines for selecting file formats and storing and organizing data to enable fast and efficient data processing. It is obvious that this book was written by a practicing engineer.

The theme of the book is that Python can be an all purpose environment for data manipulation and visualization, using nothing but free and open source tools that are easily integrated and scriptable without using multiple programming languages. The book should be an invaluable tool for scientists and engineers but it is also easily accessible to anyone interested in math and data analysis. There is no need for an advanced math background. While, as a matter of full disclosure, I have undergraduate degrees in Math and Physics, I feel the book should be easily accessible to anyone with a solid high school math background who is seriously interested in the subject. The book contains a short introductory tutorial on the basics of Python so anyone familiar with programming in any language should be fine.

The book is an easy read from front to back, and I am sure it will also be a good reference resource for the future. The writing style is very clear and unforced and I found surprisingly few errors. While the Python world has a surplus of introductory and general books, books covering this kind of specific domain are especially welcome, and we could use more on other topics by competent authors.

At 363 pages the book is a surprisingly fast read. Its methodology is to use specific, short code examples to make all the key points. Most of the code samples are well selected, short and written in clear, concise Python. This is not the kind of book that overwhelms you with massive amounts of code. Either the book was well edited or else it was written by an exceptionally lucid thinker, or both.

So, if you want to learn how to process, organize, and visualize data from various sources using the Python language, I recommend this book to you. I have also posted a podcast of an interview with the author at Python411

You can purchase Beginning Python Visualization: Crafting Visual Transformation Scripts from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

cancel ×

46 comments

Sorry! There are no comments related to the filter you selected.

really? (0, Troll)

cool_story_bro (1522525) | more than 5 years ago | (#28112637)

are book reviews news?

Re:really? (5, Informative)

eldavojohn (898314) | more than 5 years ago | (#28112879)

are book reviews news?

If you use the classic index, click here [slashdot.org] and select the no smoking column for book reviews. If you use the dynamic index, click here [slashdot.org] . In this view, find Exclusions under the dynamic index and click it. In the box for other terms, enter "book review" with the quotes and save it the page. Upon a page refresh of your home page, you should notice no more reviews of books.

Re:really? (1)

jddj (1085169) | more than 5 years ago | (#28113647)

Thank you so much! I've been wondering for weeks if there was a way to turn that index crap off - it's pretty profoundly broken on every platform I use, takes forever to load, makes the page disappear out from under me every few minutes - awful!

PUBES (-1, Troll)

Anonymous Coward | more than 5 years ago | (#28112647)

PUBES

Hmm (1)

No2Gates (239823) | more than 5 years ago | (#28112665)

Is this like "visualizing whirled peas"?

Getting up and running (5, Informative)

GlobalEcho (26240) | more than 5 years ago | (#28112773)

A great way to set yourself up with a Python-based analysis environment is a tool amalgamation called Python(x,y) [pythonxy.com] . I don't use it myself since my setup predates the project, but it has all the major tools I know and love.

And for those of you who want statistical and graphing capabilities lacking there, know that you can communicate back and forth with an instance of GNU R, using rpy2 [sourceforge.net] .

Re:Getting up and running (2, Informative)

Anonymous Coward | more than 5 years ago | (#28113599)

All of these libraries are also include in Enthought's python distribution called EPD [enthought.com] . While it is a commercial product, it is free for commercial use and there is also a 30-day free trial for anyone. EPD is available on platforms Python(x,y) doesn't cover.

Re:Getting up and running (-1, Offtopic)

zqtvlyj (1563585) | more than 5 years ago | (#28114061)

Without number limited, ãFree Live HQ streamã' ,100%free and no virus live stream online,Barcelona VS Manchester United live!!go go go!!!ole ole ole! http://www.nowgoal.com/17.shtml [nowgoal.com]

Re:Getting up and running (-1, Offtopic)

zqtvlyj (1563585) | more than 5 years ago | (#28114069)

Without number limited, Free Live HQ stream ,100%free and no virus live stream online,Barcelona VS Manchester United live!!go go go!!!ole ole ole! http://www.nowgoal.com/17.shtml [nowgoal.com]

Re:Getting up and running (5, Informative)

Anonymous Coward | more than 5 years ago | (#28114519)

Sage (http://www.sagemath.org) is also includes numpy, scipy, matplotlib, R, and many, many other packages. All of these are easily usable from a command line interface as well as an easy web interface (see http://www.sagenb.org)

Mayavi should also be mentioned when talking about python and visualization: http://code.enthought.com/projects/mayavi/

You can use Mayavi and Sage together as well. Here's a link to a public Sage worksheet illustrating how to use Mayavi to generate a 3d VRML interactive image embedded in the page: http://nb.hpfem.org/home/pub/16/ (install a X3D viewer like FreeWRL to see the last interactive image). Sage also does 3d graphing with JMOL.

DVD (5, Funny)

valnar (914809) | more than 5 years ago | (#28112951)

My favorite way to visualize Python is the DVD set. The Holy Grail is still my favorite, although Life of Brian is a close second.

Re:DVD (1)

MyLongNickName (822545) | more than 5 years ago | (#28113423)

I can quote it for you now and make you R-O-T-F-L-O-L

Re:DVD (1, Funny)

Anonymous Coward | more than 5 years ago | (#28113817)

STFU, or I shall say NI to you again.

import tk? (2, Informative)

zukinux (1094199) | more than 5 years ago | (#28112983)

most of the people think that python GUI ends up with
import tk
and that's not correct at all, if you're among these people, you should read this book... good luck!

Beginning Python Visualization (-1, Offtopic)

Aahzimandious (1424393) | more than 5 years ago | (#28113107)

Graham Chapman, Eric Idle, Terry Gilliam, Terry Jones, John Cleese, Michael Palin ruined the word "Python" for me. God bless them. :)

I really was wishing more for a book that was going to help me visualize Python in its core essence, to become One with that which is Python.

Oooooommmmmmm... Relax. Breath in. Breath out. Think Python. Be Python. Do Python!

What? Well, okay... cool.

"Beethoven's gone, but his music lives on, and Mozart don't go shopping no more. You'll never meet Liszt or Brahms again, and Elgar doesn't answer the door...."

"I had to get up in the morning at ten o'clock at night, half an hour before I went to bed, eat a lump of freezing cold poison, work twenty-nine hours a day down mill, and pay mill owner for letting us work there, and when we got home, our Dad would murder us in cold blood, each night, and dance about on our graves singing "Hallelujah." But you try and tell the young people of today that... and they won't believe you."

:)

Processing / "Visualizing Data" (3, Interesting)

weston (16146) | more than 5 years ago | (#28113151)

Ben Fry's Visualizing Data [amazon.com] using the Processing [processing.org] environment seems like it's addressing a similar topic... wonder if Vaingast didn't know about it or if he found something unsuitable about it...

Re:Processing / "Visualizing Data" (2, Informative)

kramulous (977841) | more than 5 years ago | (#28114655)

There is also The Visualization Handbook [amazon.com] which covers all the different classifications of data and the appropriate techniques to visualise. Lots of theory and algorithms under the hood.

Re:Processing / "Visualizing Data" (3, Interesting)

Anonymous Coward | more than 5 years ago | (#28115809)

In our lab (@ Georgia Tech), we developed an processing port to python that uses pygame + OpenGL as the base.

We haven't open-sourced it yet, but do think people would find that interesting?

Re:Processing / "Visualizing Data" (1, Interesting)

the_brobdingnagian (917699) | more than 5 years ago | (#28116817)

A processing like environment in Python would be awsome!

Re:Processing / "Visualizing Data" (1)

chartreuse (16508) | more than 5 years ago | (#28117159)

Hell yes! Please do.

Re:Processing / "Visualizing Data" (3, Interesting)

karmac0ma (1111641) | more than 5 years ago | (#28119501)

There's already something very close to Processing using Python, BSD licensed, even though it's OSX only: Nodebox (http://www.nodebox.net/ [nodebox.net] ).

On the other hand, there's a GPL fork/port of this project that works on GNU/Linux and Windows -- Shoebot (http://www.tinkerhouse.net/shoebot [tinkerhouse.net] ) (Disclosure: i'm one of the authors)

Nodebox uses Cocoa as its graphics back-end, while Shoebot uses Cairo (http://www.cairographics.org/ [cairographics.org] ); our plans for it are to implement other graphics back-ends including OpenGL (via Pyglet or Pygame), so seeing the parent open-source their Python/OpenGL project would be a godsend to us. Hey, there's common goals and all, so no reason to keep it closed, right? ;-)

(do pardon me if the links come out wrong, it's my first post over here -- bonjour to all)

Re:Processing / "Visualizing Data" (0)

Anonymous Coward | more than 5 years ago | (#28116195)

I think the key is that he was looking to use Python. It looks like that book is using Java.

Re:Processing / "Visualizing Data" (0)

Anonymous Coward | more than 5 years ago | (#28116545)

If it's about Processing, it's quite pretty obvious it's using "Java" (actually, it is a superset of an old Java version).

Python to visualize in a virtual world (3, Interesting)

rexping (1505799) | more than 5 years ago | (#28113187)

realXtend open source virtual world platform can be programmed in Python, I made a test to visualize email inbox as a terrain [maxping.org] .

Viz without interaction? (3, Interesting)

Bejitt (537518) | more than 5 years ago | (#28113455)

I've used Python extensively to do (Information) Visualizations, and found the language to be very handy for getting started quickly. I used NumPy a bit, but I mainly offloaded all the heavy display work to OpenGL. I'm surprised that I don't see anything about using Python to develop interactive Visualization techniques, which should be a top priority in 'most' visualizations. (some can do without, but some are built around interaction) Perhaps that would be a good topic for continuing this series =)

Review FAIL (1)

Bazman (4849) | more than 5 years ago | (#28113663)

"Sometimes a picture is worth a thousand words". And then we get a review with no pictures. Just links to other stuff. And a link to an audio file - sorry, I mean 'PodCast' - which also won't have any pictures, or text I can search, or examples I can cut n paste.

It's like a review of Playboy without at least one carefully cropped picture, coupled with a taped interview with one of the bunnies.

I like matplotlib, scipy but ... (1)

Falstius (963333) | more than 5 years ago | (#28113697)

I like matplotlib and scipy but I have lots of problems with memory not being freed when making and closing a lot of plots. It might be something I'm doing though, so I'm curious if the book discusses this.

Re:I like matplotlib, scipy but ... (1)

Lockjaw (41852) | more than 5 years ago | (#28202669)

You don't mention ipython, but I had a very similar problem making and closing plots when using ipython, but not when using straight python for the interactive session (or, obviously, in scripts). The problem is that ipython caches things it really shouldn't, like closed plots. Try setting the ipython "-cs" option to 0 and see if things improve.

Fail already (2, Interesting)

Hognoxious (631665) | more than 5 years ago | (#28113731)

In this section he shows how to use CSV, comma separated values, as a most useful file format.

Useful? Not in an international environment. English CSV uses commas as the name implies, but export a .csv from French excel and it uses semicolons.

I hate CSVs. Mr noob author, may I introduce my good friend Mr tab?

Re:Fail already (2, Informative)

shaiay (21101) | more than 5 years ago | (#28113895)

in the python csv module you can specify the delimiter (so it can be semicolons, or even tabs). I'm sure you, being so not n00b (what's the opposite of n00b anyway?), could write up a python function to quickly determine the delimiter used in a file. csv files are also nice in that they allow to surround text in quotes, thus allowing to include the delimiter as data in the field (i.e. "hello,",goodbye would give you two fields)

Disclaimer: I know the author

Re:Fail already (1)

omuls are tasty (1321759) | more than 5 years ago | (#28116873)

(what's the opposite of n00b anyway?)

u can use l33t, 1337, 0r 3v3n 001011011111, but tehn again, how would u kn0w, suxxor?

the opposite of n00b (1)

wiredog (43288) | more than 5 years ago | (#28121525)

1337

Re:Fail already (1)

maxume (22995) | more than 5 years ago | (#28113929)

The book is about Python so it probably isn't that out of place that there is reasonable support for csv files included in the standard install:

http://docs.python.org/library/csv.html [python.org]

No support for Unicode (yet) would probably be the biggest caveat. I've had good luck with it, but I haven't exercised it in particularly interesting ways.

Re:Fail already (1)

TopherC (412335) | more than 5 years ago | (#28114707)

I was guessing that although there is nothing even remotely new about a .csv file, our reviewer mentions this as an example of good style. You could write data out in any format you invent for each project, and with Python you might be particularly tempted to use some pickle (Python's object serialization) format. But using CSV whenever it makes any sense seems like it would give you the most long-term flexibility since so very many programs can read these files, even text editors. The format is never going away because it's too simple.

The separator you use is a non-issue since almost any program that reads these in can understand most typical separators. Excel might be a rare exception, I couldn't see an obvious way to specify that but I'm sure someone has a plugin that works. Tabs are okay too, and gnuplot would like those, but I'd still call it a CSV file just to be boring.

Re:Fail already (2, Funny)

HornWumpus (783565) | more than 5 years ago | (#28116651)

Your thinking has you almost to understanding why XML was invented.

The one, only true serial file format that will rule them all.

Learn to use an XML tool set and never deal with reading or writing data issues again.

Why is XML better then CSV (I remember asking this)? It's hierarchical right in the spec.

Sure you can make a hierarchical CSV file. But yours will be special. Not so with XML.

Re:Fail already (1)

kramulous (977841) | more than 5 years ago | (#28120227)

Fair point. But what about csv files that are > 2TB each in size that describe integers, floats and unsigned shorts? XML will increase this size by what factor? An order of magnitude?

For small files ... absolutely.

Commas Suck (2, Informative)

whitefox (16740) | more than 5 years ago | (#28119491)

Yes, I know commas are part of the CSV acronym but seriously they suck. A lot of the data I've worked with over the years (names, addresses, etc) was usually bound to have some records with embedded commas in the field values thus generating import exceptions.

I learned early on to use tabs for any CSV exports I had to create. And Excel has no problem interpreting tab-delimited files.

Re:Fail already (1)

kraut (2788) | more than 5 years ago | (#28120733)

if semicolons instead of commas in a CSV file are the biggest problem you've run across with CSVs, you haven't lived.

Now get off my lawn.

CairoPlot (2, Informative)

Anonymous Coward | more than 5 years ago | (#28113761)

There's a fairly new project called Cairoplot [launchpad.net] that creates charts and plots using Python and Cairo. It's not meant to do number-crunching like some of the other tools mentioned, but it creates great-looking charts.

(Disclaimer: I've done a little bit of development on Cairoplot, trying to get it more LaTeX-friendly)

3D Visualization (3, Interesting)

raguirre (986049) | more than 5 years ago | (#28114271)

Does it say anything about 3D Visualization of data? a very handy tool is VPython [vpython.org] .

Re:3D Visualization (1)

codesmythe (1539843) | more than 5 years ago | (#28114525)

Interesting to learn of. You might also like blender - http://blender.org/ [blender.org] Blender's modelling, animation, and game engine can be driven using python.

Great Book (1)

Murdoch5 (1563847) | more than 5 years ago | (#28118811)

This book is awesome, it helped me really get into python. As a Computer Eng student I really wanted to get into Python but never had a good outlet, finally thanks to this book I've been able to go in head first and feel this will really help me to get going.

Google Earth (1)

Viadd (173388) | more than 5 years ago | (#28119093)

If you have geographic data, Google Earth is a great output viewer. It is easy to use Python to write a KML [google.com] file.

matplotlib (1)

Ben1220 (1503265) | more than 5 years ago | (#28119817)

Yes, matplotlib is pretty powerful, but they don't have a version for python 2.6 yet. You need to install it yourself :( Lazy developers, 2.6 has been out for ages...

matplotlib (1)

clint999 (1277046) | more than 5 years ago | (#28139361)

in the python csv module you can specify the delimiter (so it can be semicolons, or even tabs). I'm sure you, being so not n00b (what's the opposite of n00b anyway?), could write up a python function to quickly determine the delimiter used in a file. csv files are also nice in that they allow to surround text in quotes, thus allowing to include the delimiter as data in the field (i.e. "hello,",goodbye would give you two fields) Disclaimer: I know the author

Check for New Comments
Slashdot Login

Need an Account?

Forgot your password?
or Connect with...

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>