Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 


Forgot your password?
Close
typodupeerror
×
Australia

Australian Bureau of Statistics Doesn't Like Direct Downloads of Census Data 136

Posted by Unknown Lamer from the sort-of-open-data dept.
Bismillah writes "The ABS has released the census data for the country under a Creative Commons license, but instead of making it easy to get, they've put in Javascript to obfuscate file paths and more. All commented in the source code of course." At first glance, it's an attempt to get people to pay $250 for a DVD with the data instead.
This discussion has been archived. No new comments can be posted.

Australian Bureau of Statistics Doesn't Like Direct Downloads of Census Data More

Australian Bureau of Statistics Doesn't Like Direct Downloads of Census Data

Comments Filter:

  • Bit torrent (Score:5, Insightful)

    by DiSKiLLeR ( 17651 ) on Wednesday April 17, 2013 @12:11AM (#43469265) Homepage Journal

    Sounds like an excellent use for Bit Torrent? I assume someone will download the whole dataset and make a torrent out of it before long....

  • Link to torrent (Score:5, Informative)

    by sdreader ( 2893571 ) on Wednesday April 17, 2013 @12:12AM (#43469269)

    Link to the torrent of the census data from the article:
    http://blog.angrygoats.net/2013/04/12/2011-australian-census-release-3/ [angrygoats.net]

    Since the data is available for free (obfuscated or not) and was released under a CC license, technically this should all be considered legal, right? Not that it should be necessary of course.

    • Re: (Score:3)

      by mjwx ( 966435 )

      Link to the torrent of the census data from the article:
      http://blog.angrygoats.net/2013/04/12/2011-australian-census-release-3/ [angrygoats.net]

      Since the data is available for free (obfuscated or not) and was released under a CC license, technically this should all be considered legal, right? Not that it should be necessary of course.

      The obfuscation is probably because hosting and bandwidth are not cheap in Oz and some inventive public servant (stop snickering, they do exist, there aren't many of them but they do exist) came up with a way to reduce the bandwidth bill. With the current emphasis on public service spending and impending election, this wouldn't surprise me.

      Either that or some hopeless public servant coder has no idea what they've done.

      Could be either case really, I've seen both.

      • There may be inventive public servants, but I highly doubt they are inventive enough to make a stupid obfuscated download system just so that some guy would bittorrrent it, and thereby save the government a small amount of money on bandwidth. I mean really.

        • Re:Link to torrent (Score:4, Interesting)

          by mjwx ( 966435 ) on Wednesday April 17, 2013 @02:13AM (#43469769)

          There may be inventive public servants, but I highly doubt they are inventive enough to make a stupid obfuscated download system just so that some guy would bittorrrent it, and thereby save the government a small amount of money on bandwidth. I mean really.

          You've never worked in the APS have you. The fewer people you have to serve, the better your balance sheet looks. If someone else can do it, why not.

        • Re:Link to torrent (Score:4, Informative)

          by GerryHattrick ( 1037764 ) on Wednesday April 17, 2013 @02:42AM (#43469863)
          Usual reason for doing this with official data is to avoid sensation-seekers 'hotlinking' to specific data without noting the disclaimers, statistical cautions, changes of basis etc. which moderate any interpretation.

          • Re:Link to torrent (Score:4, Insightful)

            by SgtChaireBourne ( 457691 ) on Wednesday April 17, 2013 @09:40AM (#43471715) Homepage
            There may be a usual reason to ensure disclaimers, etc are read, but javascript is definitely not the way to go. You can very easily require a specific http referrer URL by configuring Apache to require it for a file or directory. Or you can simply have a plain old README or LICENSE file included in the tarball. Javascript just hurts usability and makes things over complicated and broken.

      • Re: (Score:1)

        by Anonymous Coward

        some hopeless public servant coder has no idea what they've done.

        Hopeless? No idea? Put yourself in their shoes. Here you've got some CC licensed data. Manager tells you he wants to dissuade people from downloading it, charging 250 pop for the data on DVD instead. You just *know* that this is a waste of time, because the first getting the DVD is gonna be disgruntled and will legally put the stuff on bittorrent anyway. So technically, you're just wasting everyones time: Yours, your managers, and the download

  • I saw the title text and thought the census data was being provided through bittorrent. A few games including the popular World of Warcraft distribute their updates through the protocol, seeing it adopted in other areas to reduce the bandwidth costs seems like a good idea.

  • Excellence in Government (Score:4, Interesting)

    by edibobb ( 113989 ) on Wednesday April 17, 2013 @12:30AM (#43469361) Homepage
    I think that is really funny! They declare the data free and then make an inept attempt to force people to pay for it. It's almost as bad as copyrighting public laws.

    • Funny, isn't it? Laws are the only thing corporations invest a lot of money in that they don't try to copyright or patent.

    • Re:Excellence in Government (Score:4, Interesting)

      by Anonymous Coward on Wednesday April 17, 2013 @03:04AM (#43469927)

      It's almost as bad as copyrighting public laws.

      I'm not sure if you're joking here but the Australian government actually DOES copyright legal documents. For example to comply with telephone wiring regulations requires access to a document released by "Standards Australia" which costs about $200 last I checked. I don't doubt that the document was developed using public funds. I'm sure this shit happens a lot more than people realise.

  • Damn Streisand Effect (Score:5, Funny)

    by istartedi ( 132515 ) on Wednesday April 17, 2013 @12:32AM (#43469365) Journal

    Thanks a lot Slashdot. Now I have a sudden urge to know precisely how many married couples with the husbands between the ages of 30 and 32 inclusive have children in Queensland, and what the genders of and ages of the children are.

  • If you're wondering why the data are licensed... (Score:4, Informative)

    by Anonymous Coward on Wednesday April 17, 2013 @12:33AM (#43469367)
    Remember that in many countries works of the national government are not automatically in the public domain like they are in the US. In Commonwealth realms this is called Crown Copyright.

    https://en.wikipedia.org/wiki/Crown_copyright/ [wikipedia.org]

    • Re: (Score:1)

      by Anonymous Coward

      No, no, you're right, the US government is always transparent and forthcoming with information

      (yes it was sarcasm)

    • In Commonwealth realms this is called Crown Copyright.

      Did someone say apply the CC licence and not specify which?

    • Re: (Score:2)

      by Nugoo ( 1794744 )
      How is it that census data is creative enough to merit a copyright?

  • You Know What They're Up To? (Score:5, Interesting)

    by Greyfox ( 87712 ) on Wednesday April 17, 2013 @12:39AM (#43469387) Homepage Journal
    This follows on my "best method to get tech support from a computer person." You don't ask "How do you...?" You assert, loudly, within hearing range of the computer person "This is the absolute beset way to do it!" and provide a woefully incorrect method of getting to the result you're trying to achieve. One of these methods will have the computer person falling over himself to help you. Guess which one. Have I mentioned that I'm Evil lately?

    Anyway, they're pulling the same thing here. They want someone to gather up their data and present it in a nice package for free. The best way to do that is to drop an ineptly-presented steaming pile of crap on the internets. There'll probably be 15 open source projects to slice and dice it on github by the weekend, and it didn't cost the Australian government a dime! It's brilliant!

    • Re: (Score:3)

      by martas ( 1439879 )
      Yes, because it takes a lot of technical know-how to create a zip file and making it available as a torrent... /s
    • Comment removed based on user account deletion
    • An alternative is to sadly declare that some simple task you're experiencing resistance over simply cannot be done.

  • It's free (Score:1)

    by Anonymous Coward

    Magnet link for the lazy:

    magnet:?xt=urn:btih:EE2DEAA27287952089AE257EC8B009E382598239&dn=2011%20Datapacks%20BCP_IP_TSP_PEP_ECP_WPP_Release%203.tar.xz&tr=udp%3a%2f%2ftracker.publicbt.com%3a80%2fannounce&tr=udp%3a%2f%2ftracker.openbittorrent.com%3a80%2fannounce

    or
    http://mgnet.me/DTyE

    or torrent: http://grond.angrygoats.net/torrent/2011%20Datapacks%20BCP_IP_TSP_PEP_ECP_WPP_Release%203.tar.xz.torrent

  • No, Br'er Rabbit, don't tell the world we are hiding our data, they might get copies and make sure the whole world has access to it, no, don't do that Br'er Rabbit.

  • to conspiracy that which can be explained by incompetence. The real goal could of had nothing to do with "hiding" the data.

    • The real goal could of had nothing to do with "hiding" the data.

      If you could read, you might have seen that phrase spelled "could've", which is a contraction of the phrase "could have". Instead, you heard it spoken out loud and parsed it incorrectly as "could of". What the !@#$ does "could of" even mean?!?

      You're welcome.

  • Crikey, they're a bunch of naughty little critters for doing that!
  • From TFA:

    "The ABS is constantly looking at ways it can simplify the website and enhance the user experience," iTnews was told via email.

    Stop hosting it on Lotus Domino servers and you won't have to worry about how many people download the damned data.

    • Re: (Score:2)

      by c0lo ( 1497653 )

      From TFA:

      "The ABS is constantly looking at ways it can simplify the website and enhance the user experience," iTnews was told via email.

      Stop hosting it on Lotus Domino servers and you won't have to worry about how many people download the damned data.

      U crazy? After millions paid for the Lotus servers and zillions in staff training (or... was it train stuffing? in the context, the results would be the same), you want the IT dept head to... well, lose her/his head?

  • What is the point of putting a creative commons license on data that is not copyrightable. Anyone can take the data and do anything they want with it and there is nothing anyone can do about it. If it were otherwise, no one would be able to broadcast the temperature without permission from the weather office. How well would that system work?

    • What is the point of putting a creative commons license on data that is not copyrightable. Anyone can take the data and do anything they want with it and there is nothing anyone can do about it. If it were otherwise, no one would be able to broadcast the temperature without permission from the weather office. How well would that system work?

      You can't copyright facts, but there are copyright-style laws covering a collection of facts organised into a database. That said, creative commons probably isn't the right licence for the same reason it wasn't the right licence for open street map (who have now migrated to a different permissive licence designed for databases of facts).

      • What is the point of putting a creative commons license on data that is not copyrightable.

        You can't copyright facts, but there are copyright-style laws covering a collection of facts organised into a database.

        Or everybody could just understand that US copyright law does not apply world-wide and that, in many more countries than not, facts and collations of facts are often copyrightable.

    • Re: (Score:1)

      by Anonymous Coward

      Creative Commons is in fact a copyright licence and has restrictions. Public Domain would have been better.

  • I can see the conversation that happened.. (Score:3)

    by gregmac ( 629064 ) on Wednesday April 17, 2013 @02:32AM (#43469837) Homepage

    From the code:

    // Also, generate a random number, which we append to the URL, to make it appear as if a complex
    //key is required. This is a pathetic attempt to discourage someone from downloading the ZIPs
    //directly (ie. without having to login), if they deduce the URL pattern.

    Translation:

    Coder: "Here's the census web application."
    PHB: "Great. But wait..I can just type in these other names and download them really easily! People will hack us and we'll be out possibly a COUPLE THOUSAND DOLLARS! "
    Coder: "It is Creative Commons data, so of course we added no protection. Changing that now will be a massive rewrite and take months."
    PHB: "So let's add some random numbers to the end so it looks really complex and people can't guess how to get in."
    Coder: "But they still will eventually see the links because they do actually have to download it, so this is not really doing anything."
    PHB: "Psh, no one is smart enough to figure that out. I read about this GUID things and they're really hard to guess. It will work. This is your job today."
    Coder "..Ok, fine. I'll do it exactly the way you asked."

    • ya the code snippet provided in parent post indicates to me that this was something forced upon a level-headed coder by some moronic middle manager.

      • Re: (Score:2)

        by Thud457 ( 234763 )

        that this was something forced upon a level-headed coder by some moronic middle manager.

        That describes 99.5% of all software written since the time of Noah.

    • Re: (Score:2)

      by PRMan ( 959735 )

      Coder: (And then I'll put it in the comments so that everyone can see what idiots we are)

      Yeah, nice try. But the coder actually thinks he's being really clever and doesn't realize all his Javascript comments are available for the world to read because he's actually an idiot (but he's a coder working for a government institution, so that's pretty much a given). No conspiracy here. They probably don't even realize what it means that it's under a CC license.

      • Re: (Score:2)

        by tqk ( 413719 )

        But the coder actually thinks he's being really clever and doesn't realize all his Javascript comments are available for the world to read ...

        More likely he knows exactly what he's doing, meaning he's telling all the world what a blithering moron of a manager told him to do today. There are times when diplomacy is contra-indicated and the potential downside (blithering moron manager finding out about it) is very small. I'd say blithering moron manager painted himself into this corner.

      • Hey, PRMan, allow me to introduce you to my friend Passive A. G. Gressive. [wikipedia.org]

        The actual thought process behind the comments would have been more like:

        Coder: (And then I'll put it in the comments so that everyone can see what idiots my bosses are)

        I have news for you: The geek community laboring in bondage to governmental PHBs lives for the opportunity to secretly sabotage their masters' moronic agendas while looking like the perfect collaborationist stooges to everyone who can't read code. A nerd underground,

  • .... in 3... 2... 1...

  • Why on earth did they waste time and money obfusticating something that is licensed on the creative commons. All someone has to do is either buy the DVD or reverse engineer the site once and they can put it up on their own website
    • It's because bureaucrats everywhere have a visceral belief that THEY own the data and it should never be released to the public without the maximum of foot-dragging, time-wasting and hoop-jumping.
  • I gather this is data being published by a government agency. As all agencies are funded by taxpayers, all records -- with exceptions for security and privacy -- should already be open to the public. Creative Commons seems inappropriate here; the correct notice should be "Public Domain", or is Aussie law different in this respect from US law?

    • I gather this is data being published by a government agency. As all agencies are funded by taxpayers, all records -- with exceptions for security and privacy -- should already be open to the public. Creative Commons seems inappropriate here; the correct notice should be "Public Domain", or is Aussie law different in this respect from US law?

      US law is actually the one out of step with the rest of the world - in the vast majority of countries, government records are under some form of copyright, not PD.

  • Was certain I had read those comments before. Yep [thedailywtf.com]
  • The Australian government is excellent at selling to the public products that the public have paid to produce. In the US (at least in theory) products of the US government are in the public domain and not eligible for copyright. I know, there are tons of things the US government produces that are exempt from this. I know this is a simplistic view, but Australia should not be selling things to people that they have already bought with their tax dollars.

Slashdot Top Deals

One man's constant is another man's variable. -- A.J. Perlis

Close