Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

US Monitoring Database Reaches Limit, Quits Tracking Felons and Parolees

timothy posted about 4 years ago | from the paid-by-the-row dept.

Bug 270

An anonymous reader writes "Thousands of US sex offenders, prisoners on parole and other convicts were left unmonitored after an electronic tagging system shut down because of data overload. BI Incorporated, which runs the system, reached its data threshold — more than two billion records — on Tuesday. This left authorities across 49 states unaware of offenders' movement for about 12 hours." As the astonished submitter asks, "2 billion records?"

cancel ×

270 comments

Sorry! There are no comments related to the filter you selected.

Now.. (4, Insightful)

Anonymous Coward | about 4 years ago | (#33844000)

They just need to upgrade it so they can track the other 4 billion properly.

Damn sick criminals! ALL OF THEM.

Re:Now.. (-1, Troll)

Anonymous Coward | about 4 years ago | (#33844032)

WARNING! WARNING! WARNING!

This person is a pedophile. Please report him to your local police.

Re:Now.. (4, Insightful)

hairyfeet (841228) | about 4 years ago | (#33844552)

To quote some lyrics from one of my favorite bands "Ain't it funny how the school doors closed, round the time that the factory doors closed, round the time that 100,000 jails cells opened up to greet you, like the reaper". Considering in this country one can be busted for a sex offense for pissing on a bush, sexting pics of your own body to your GF/BF if you are under 18, or even words on a page or drawings in a comic book, the fact that we allow private contractors to do these jobs (thus giving an even greater incentive for bribery and worse laws) just makes me sick.

Re:Now.. (3, Insightful)

h4rm0ny (722443) | about 4 years ago | (#33844092)

I'm getting this scene in my mind like Austen Powers, where Senator McCarthy is unfrozen and keeps rubbing his hands with glee saying "We'll track one million US citizens." His NSA assistant coughs politely. "Uh, [i]billion[/i], sir".

Re:Now.. (2, Insightful)

arivanov (12034) | about 4 years ago | (#33844230)

The joy of using centralised versus distributed systems. We live in the age of the network. Load balancing data sets across multiple databases, machines and systems and merging them whenever they are needed is trivial. Designing for such load balancing is trivial as well.

Anyone designing a system that piles up everything on a single box gets whatever christmas they deserve. By the way, considering the name of the company I am not surprised. It says everything that there is to be said about their design methodology...

Not that most of government contracts in the UK or USA are any different. They have been taken over the BI crowd and competence in design has been replaced by competence in explaining how it is not your fault that a f*ck up has occured. PRINCE, ISO, TOGAF, ZAMAN all have this as their primary function and they are now the only requirement towards jobs in this area. It is quite scary - you look at an advert for an architect and see these listed _WITHOUT_ any technical knowledge domain whatsoever...

Re:Now.. (4, Insightful)

gmack (197796) | about 4 years ago | (#33844350)

I really doubt space was the actual problem because TFA says "BI Incorporated, which runs the system, reached its data threshold - more than two billion records - on Tuesday. " The max value of a signed 32 bit int is 2 147 483 647. It is much more likely that someone set an index value on the database to int years ago and then forgot about it.

Re:Now.. (3, Informative)

autocracy (192714) | about 4 years ago | (#33844576)

Yeah, we've all seen that happen before [slashdot.org] .

This wouldn't have happened with a NoSQL database. (-1, Flamebait)

Anonymous Coward | about 4 years ago | (#33844742)

Incidents like this are actually impossible when you're using a NoSQL database. By their very design and implementation, NoSQL databases use sharding and distributed computing to ensure that data is inconsistent, randomly corrupted or lost, and sometimes not even actually stored in the first place. After all, you can't hit storage limits when you're not even storing any data.

Re:Now.. (2, Funny)

Sulphur (1548251) | about 4 years ago | (#33844250)

HTTP/1.1 200 OK Date: Sat, 09 Oct 2010 10:31:23 GMT Server: Apache/1.3.41 (Unix) mod_perl/1.31-rc4 Connection: close Transfer-Encoding: chunked Content-Type: text/html; charset=iso-8859-1
OK
The server encountered an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator, admin@fbi.gov and inform them of the time the error occurred, and anything you might have done that may have caused the error.

Re:Now.. (1, Insightful)

modmans2ndcoming (929661) | about 4 years ago | (#33844790)

Are people really this dumb?

A record does not equate to a single individual. They are tracking movements so in the movement table, I am sure they have a lot of records for each person they are tracking, not to mention the other tables that contain multiple entries for people

Visual Basic? (0, Troll)

Anonymous Coward | about 4 years ago | (#33844004)

Visual Basic for the FAIL, signed int? They really should know better.

Re:Visual Basic? (1, Redundant)

CarpetShark (865376) | about 4 years ago | (#33844030)

It's not Visual Basic's fault, they just had too much data. 640K ought to be enough for anyone.

Re:Visual Basic? (1, Flamebait)

odies (1869886) | about 4 years ago | (#33844034)

Yeah, because only Visual Basic has signed int. The fail is on your side.

Well no wonder (5, Funny)

Fry-kun (619632) | about 4 years ago | (#33844006)

MS Access can't possibly handle 2 billion records, no matter how much hardware you throw at it.

Re:Well no wonder (3, Funny)

sakdoctor (1087155) | about 4 years ago | (#33844086)

65536 Excel rows should be enough for anyone

Re:Well no wonder (0)

fluor2 (242824) | about 4 years ago | (#33844160)

FYI, that limit is no longer present in Excel 2010.

Re:Well no wonder (2, Insightful)

realityimpaired (1668397) | about 4 years ago | (#33844194)

Wasn't present in 2007, either. But have you ever tried loading a spreadsheet with more than 30,000 records, let alone one with more than 100,000 records?

Hope you have enough RAM, and that nothing else is open on your system....

Re:Well no wonder (1)

Threni (635302) | about 4 years ago | (#33844204)

Yeah, it's really handy sometimes to select a bunch of records from a database, then paste it into a spreadsheet, where it's much more natural to do number crunching on it than doing it in a select statement. It's trivial to tweak the spreadsheet so people can click on columns to sort/group by different columns. 65000 is a stupid limit when you're looking, for example, sales in a chain of shops on a given day.

Oh dear oh dear oh dear (1, Informative)

Kupfernigk (1190345) | about 4 years ago | (#33844234)

I think you really need to read a primer on data mining. The attitude you describe is all too common and brings pain in its wake. If you find you want more than 65000 rows in a spreadsheet, your problem is almost certainly inappropriate for a spreadsheet solution. Even Access is vastly better, and if you (a) don't have even basic data mining and (b) the data is going out to PHBs, Filemaker is your friend.

Also, chances are, if you think any typical business data set is best represented by a spreadsheet, you are probably not qualified to make the call.

Re:Oh dear oh dear oh dear (5, Insightful)

Ironhandx (1762146) | about 4 years ago | (#33844292)

And you are clearly completely unaware of the accounting world.

I have yet to meet an accountant that knows much of anything about access or any other database system. On the other hand the majority of them have complained about the 65000 line limit in excel.

They ALL do this. You're telling thousands of accountants to change how they do things, and honestly, not for the better. They know how to use excel and know how to make things balance with excel.

A large portion of them took accounting because it was supposed to make them a lot of money, these people don't even use 1/10th of the functionality provided in excel, lets not try to make them learn another entirely different software skill set, ok?

Even if you're currently working in IT and are like "Oh, no, our accountants have access to all this stuff in our system and they would never do that". Trust me, they do. It all ends up in an excel sheet somewhere eventually.

Re:Oh dear oh dear oh dear (3, Insightful)

schon (31600) | about 4 years ago | (#33844856)

And you are clearly completely unaware of the contracting world.

I have yet to meet a contractor that knows much of anything about screwdrivers or any other tool than a hammer. On the other hand the majority of them have complained about how hard it is to drive screws with the hammer.

They ALL do this. You're telling thousands of contractors to change how they do things, and honestly, not for the better. They know how to use a hammer and know how to drive nails.

A large portion of them took contracting because it was supposed to make them a lot of money, these people don't even use 1/10th of the functionality provided by a hammer, lets not try to make them learn another entirely different tool skill set, ok?

Even if you're currently working in contracting supply and are like "Oh, no, our contractors have access to all this stuff and they would never do that". Trust me, they do. It all ends up pounded by a hammer somewhere eventually.

Re:Well no wonder (0)

Anonymous Coward | about 4 years ago | (#33844272)

That's why they made the 64-bit version of 2010.

Re:Well no wonder (1)

gblfxt (931709) | about 4 years ago | (#33844258)

you son, have neither dealt with the US government, nor ms access.

Re:Well no wonder (1)

Eskarel (565631) | about 4 years ago | (#33844344)

Nor should it, if you've got 2 billion records use something designed for that kind of load.

Re:Well no wonder (3, Interesting)

maxume (22995) | about 4 years ago | (#33844452)

Yes, that was the joke. See, GP poster is implying that even though the system should have been using something designed for the load, since it is a government contract, they used Access.

Quickbase would be my guess (3, Interesting)

Sycraft-fu (314770) | about 4 years ago | (#33844388)

It seems to be the crap database of choice these days, especially for consulting companies. Friend of mine got a job not long ago as a consultant for a consultant. Yes really, he consults for a consulting firm. Not like he is someone they hire out, he is a consultant they hire to work on jobs they've been hired to work on. The thing that got him the job was his Quickbase experience. This company loves them some Quickbase for some reason. However they are always bashing in to limits it has. Had they used MSSQL or Oracle they'd be fine, but they didn't. So a major thing he does is work around those limits in various creative ways. Retarded, but that's what they want and they'll pay for it.

Re:Well no wonder (2, Interesting)

assertation (1255714) | about 4 years ago | (#33844484)

If you haven't, rent a copy of the documentary "Hacking Democracy".

Diebold chose to use MS Access as the backend for voting machines

Thereby solving the problem... (5, Funny)

Adambomb (118938) | about 4 years ago | (#33844010)

BI increased its data storage capacity to avoid a repeat of the problem.

ONCE AND FOR ALL.

Re:Thereby solving the problem... (1)

siddesu (698447) | about 4 years ago | (#33844114)

Yes, I heard that after a long and challenging, but well-planned and spotlessly executed migration, the system now works on a future-proof fully 32-bit operating system, capable of accessing more than 640k of RAM.

two billion locations perhaps? (0)

Anonymous Coward | about 4 years ago | (#33844014)

Is it that hard to think of having multiple rows per offender perhaps?

Re:two billion locations perhaps? (1)

cgenman (325138) | about 4 years ago | (#33844352)

2 billion offenders tracked should be fine, as there are only about 300 million people in the US. But 2 billion locations? Someone needs a real database. Or a chron job to archive these puppies.

Re:two billion locations perhaps? (0)

Anonymous Coward | about 4 years ago | (#33844474)

2 billion offenders tracked should be fine, as there are only about 300 million people in the US.

They are tracking dangerous criminals, not honest Americans!

That's why we, the 300 million honest Americans, need to protect ourselves from the billions of terrorists trying to destroy us.

I think more than 2147483648 records (2, Insightful)

Anonymous Coward | about 4 years ago | (#33844018)

Seems like it took them a few hours to change the key column from unsigned +/- 2^31 to signed 0-2^32-1

Re:I think more than 2147483648 records (1)

biryokumaru (822262) | about 4 years ago | (#33844616)

Thus solving the problem once and for all. [youtube.com]

How many? 2 billion? (0)

Anonymous Coward | about 4 years ago | (#33844022)

2 billion records ought to be enough for anybody.

Re:How many? 2 billion? (1)

AHuxley (892839) | about 4 years ago | (#33844082)

MS did the sales math on a napkin over a yummy lunch.
The fine print showed the need for much more cash as the system expanded onto more cores and the database grew.
MS does not just 'gift' that kind of power away to anyone without deep long term paid up rental deals.

2 billion... (4, Interesting)

onion2k (203094) | about 4 years ago | (#33844026)

Assuming that's a normal "US" billion, and assuming it's a journal of historical data going back a few years, I don't think it's unreasonable to think there could be information in there on a couple of hundred thousand people each of whom has been track for an average of at least 6 months. So, approximately and with some guesses, that's around 55 [wolframalpha.com] records per prisoner per day. 1 update every 30 minutes? That sounds about right, maybe a little on the low side if anything.

What is surprising is that they were running some sort of database process that maxxed out at 2 billion records, and that it just stopped once it hit that limit rather than failing over to a backup process. But then, this is a government IT contract, so maybe it's not too surprising.

Re:2 billion... (0)

Anonymous Coward | about 4 years ago | (#33844044)

It has nothing to do with prisoners.

Re:2 billion... (1)

eugene2k (1213062) | about 4 years ago | (#33844052)

The article says there were around 300 people tracked.

Re:2 billion... (0)

Anonymous Coward | about 4 years ago | (#33844076)

The article says there were around 300 people tracked.

That was just in the one state.

Re:2 billion... (1)

John Hasler (414242) | about 4 years ago | (#33844656)

16,000. 300 was the number in the state of Wisconsin.

Re:2 billion... (1)

barrylb (149470) | about 4 years ago | (#33844070)

If their system could not handle more than 2 billion records then there is no point failing over to a backup system which would probably also have the same limitation.

Seems more likely to me that the id of their tracking record table was a 32 bit signed integer which maxes out at 2,147,483,647 and when they say they "increased its data storage capacity" they just changed it to a larger data type.

Re:2 billion... (1)

jimicus (737525) | about 4 years ago | (#33844088)

Seems more likely to me that the id of their tracking record table was a 32 bit signed integer which maxes out at 2,147,483,647 and when they say they "increased its data storage capacity" they just changed it to a larger data type.

Which just goes to show how well designed it was. Exactly how often do they need to track a negative number of people?

Re:2 billion... (1)

AHuxley (892839) | about 4 years ago | (#33844098)

Hmm number of people .. post death squad? Might be of interest in some parts of the world.

Re:2 billion... (0)

Anonymous Coward | about 4 years ago | (#33844180)

In the United States of America, for example.

Re:2 billion... (0)

Anonymous Coward | about 4 years ago | (#33844802)

My my my, you are in a black mood this morning. It'll never happen here; too many guns in a population who grew up hearing those stirring heroic tales of Valley Forge, Tippacanoe and the Alamo. Too many TV shows where the hero was a hero because he went down swinging. No, if it happens here it will be much more subtle. Like vote fraud or economic subservience or a manufactured external enemy.

Re:2 billion... (2, Insightful)

anss123 (985305) | about 4 years ago | (#33844150)

Which just goes to show how well designed it was. Exactly how often do they need to track a negative number of people?

I know that in some programming languages, like java, you have to jump through hoops to get unsigned values. For all we know that database was fine, but the server frontend trunctuated values down to signed ints.

Re:2 billion... (2, Interesting)

Teancum (67324) | about 4 years ago | (#33844304)

Still, the complaint about how intelligently the software architecture was put together is seriously put into question as those who designed the system really didn't think through how long their software would last or what kinds of records were being put into the system. I understand how IPv4 had unanticipated problems with billions of computers on a network originally designed to handle merely hundreds and when v4 came out it was still in the mere thousands of computers being connected. In this design, it sounds like it was almost by design going to eat up a whole bunch of records.

Besides, people have been bitching about IPv4 running out for decades and have anticipated the problem by introducing IPv6 quite some time ago. Any competent software engineer should have seen something like this coming years ago, so when I see something like "running out of space" I can only assert either:

  • The software developers on the software were incredibly incompetent and deserve to be fired.
  • The management of the company involved doesn't know jack about what it is that they are doing, likely hiring the developers on a short-term contract or they fired the competent engineers somehow along the way.

Either way, it certainly doesn't inspire confidence in this company, and they certainly seem to be in way over their head here. If you hire a bunch of developers from Waziristan because they low bid on the development contract, you get what you pay for. This certainly isn't going to be the only problem with the software coming from this company as rookie mistakes like this are likely to be the tip of the iceberg.

Yes, this is a rookie mistake I would expect out of a freshman CS student, not somebody trying to sell a supposed professional service.

Re:2 billion... (1)

Dodgy G33za (1669772) | about 4 years ago | (#33844384)

Tracking Record ID != number of people It is a unique id of the tracking record. One to many relationship with an offender. Possible many to many if there was a gathering of priests at a given location.

Re:2 billion... (4, Interesting)

Statecraftsman (718862) | about 4 years ago | (#33844130)

If you track 16000 people and store their location once per second, you'll only need 1.55 days to reach 2^31 records. Once per minute only gives you 90 days. Once every 10 minutes, less than 3 years... I wonder if anyone is on the user end of this system that can comment.

Re:2 billion... (5, Informative)

bertok (226922) | about 4 years ago | (#33844238)

it just stopped once it hit that limit rather than failing over to a backup process.

"just over 2 billion" is almost certainly 2^31 (2 147 483 648), or the maximum number representable by a signed 32-bit integer. People usually think of "over 4 billion" (2^32) as the integer limit, but that's for unsigned integers only, which are rarely used, especially in databases. I'm willing to bet that they used an "int" as a primary key in one of their tables, and simply overflowed the maximum possible value.

This kind of bug has impacted lots of systems in the past. If it happens, there's no "fail over" that could possibly save the system. The replica would have the same data, and hence the same issue, and would have failed as well. The usual fix is to extend the key type to 64-bits or longer (e.g.: GUIDs), but for a 2 billion row table, that's going to take hours at best, probably days.

Most database systems do not provide a warning when the keys start to approach large values, so it's easy to miss.

Lemming database design (2, Interesting)

Kupfernigk (1190345) | about 4 years ago | (#33844254)

We consistently see this in monitoring systems designed by other companies. In our own monitoring systems we make extensive use of sparse appends (i.e. data only gets added to when there is a significant change, and we maintain a timestamp for the last update for each entity being monitored so we know that monitoring is actually taking place.) Of course this puts a lot more up front effort into actual system design.

There seems to have been a period, roughly when hard drive capacity was rising more rapidly than application demands for data, when nobody cared too much. Before that, backing store was limited and we had to worry about data size. Now, application data sets are growing enormous even for quite trivial applications, and we need to worry about keeping data storage in bounds again.

Re:Lemming database design (2, Interesting)

MichaelSmith (789609) | about 4 years ago | (#33844336)

And maybe they don't need every GPS position in the database. It could be there just to cover for legal requirements in which case they could append it to a binary file and open a new file every day. Compress the old files with bz2 and archive all data more than a year old.

Re:2 billion... (1)

complete loony (663508) | about 4 years ago | (#33844326)

Probably a signed 32bit primary key....

Overflow? (0)

Anonymous Coward | about 4 years ago | (#33844050)

Sounds like an int overflow ;)

about 16000 (4, Interesting)

roman_mir (125474) | about 4 years ago | (#33844072)

Prisons and other corrections agencies were blocked from getting notifications on about 16,000 people, BI Incorporated spokesman Jock Waldo said on Wednesday.

- interesting number. Anyway, it's not about the number of people in the database, it's about some number of records associated with each person presenting their location, so probably GPS coordinates taken at some time intervals.

Also note that they are still logging the data, they just can't read it, so it's an application for displaying the coordinates that is failing. Quite possible that the actual problem is in filtering the data, maybe they are just trying to view data for an entire time period per person rather than looking at latest records, something like: 'last month only'. But this is, in the words of infamous W, 'speculaaation'.

Re:about 16000 (0)

Anonymous Coward | about 4 years ago | (#33844100)

If it's not displaying the coordinates, they need to check is there a green light on the monitor. I work in IT, I know how these things work!

Re:about 16000 (0)

Anonymous Coward | about 4 years ago | (#33844106)

Probably cast the record number as (int) ...

Re:about 16000 (1)

Kjella (173770) | about 4 years ago | (#33844278)

So on a wild guess there's a function "int recordId()" that'll return -1 on problems (network failure, database connection failure, whatever). Sounds like a completely reasonable design except it's time to move to int64, 2^63 records ought to be enough for everyone. Though with the US prison population, who knows...

1 in 31 US Citizens in custody or parole (4, Informative)

mykos (1627575) | about 4 years ago | (#33844096)

"According to the U.S. Bureau of Justice Statistics (BJS): "In 2008, over 7.3 million people were on probation, in jail or prison, or on parole at year-end — 3.2% of all U.S. adult residents or 1 in every 31 adults."

This doesn't make me feel safe.

Re:1 in 31 US Citizens in custody or parole (5, Informative)

GMThomas (1115405) | about 4 years ago | (#33844146)

Right? You shouldn't feel safe. Not because of the "criminals" but because of the reason why there are so many "criminals." Have a joint on you? You're a criminal. Do you know how many people are in jail because of simple drug-related offenses? Be afraid. http://www.whitehousedrugpolicy.gov/publications/factsht/crime/index.html [whitehousedrugpolicy.gov] Look at that. 25% of federal inmates are in there for drug possession. I bet you a good amount of these people wouldn't rob you at gunpoint. Good luck, America!

Re:1 in 31 US Citizens in custody or parole (0)

Anonymous Coward | about 4 years ago | (#33844264)

I'm absolutely sure the DA never settles for an easy drug conviction when they catch someone who's committed a "real crime" but is hard to prosecute.

Re:1 in 31 US Citizens in custody or parole (1)

TheLink (130905) | about 4 years ago | (#33844318)

Or assumes has committed a "real crime" but is hard to prosecute?

Re:1 in 31 US Citizens in custody or parole (2, Informative)

jfengel (409917) | about 4 years ago | (#33844432)

Not that I'm disagreeing with your point, but I think you're misreading that page. That 25% figure is for people who were high at the time of the offense. (I assume you're looking at table 2).

Re:1 in 31 US Citizens in custody or parole (0)

Anonymous Coward | about 4 years ago | (#33844512)

Don't forget that being high is enough to be an offense.

Re:1 in 31 US Citizens in custody or parole (1)

mbone (558574) | about 4 years ago | (#33844462)

The entire reason the Federal penitentiary system was created and the first Federal penitentiary (in Atlanta) was built was to handle drug criminals. Before that, except for the military, there basically weren't any Federal prisoners.

Re:1 in 31 US Citizens in custody or parole (1, Insightful)

Anonymous Coward | about 4 years ago | (#33844542)

I bet you a good amount of these people wouldn't rob you at gunpoint.

At least, not BEFORE they did time...

Re:1 in 31 US Citizens in custody or parole (0)

Anonymous Coward | about 4 years ago | (#33844726)

Isn't the easy solution just not to smoke joints? I'm not snobbish enough to feel entitled to a good time even if it means I break the law. The problem is, this sort of thinking is the extreme exception, not the rule. There are billions of ways to have a good time legally.

Re:1 in 31 US Citizens in custody or parole (0)

Anonymous Coward | about 4 years ago | (#33844782)

Federal inmates? Drugs? You don't become a federal inmate by a cop pulling you over and finding a joint. That makes you an inmate of the State of whatever if you're extremely unlucky. You become a federal inmate by having a serious amount of drugs, usually confiscated in some sort of raid. Drugs is also a nebulous term and could refer to other less harmless substances.

Re:1 in 31 US Citizens in custody or parole (0, Offtopic)

Dexter Herbivore (1322345) | about 4 years ago | (#33844788)

You're posting in the wrong thread, the thread you're looking for is here. [slashdot.org]

Re:1 in 31 US Citizens in custody or parole (2, Funny)

ieatcookies (1490517) | about 4 years ago | (#33844196)

No no it's fine, they're all being monitored. Oh wait....

Re:1 in 31 US Citizens in custody or parole (0)

w00tsauce (1482311) | about 4 years ago | (#33844228)

^ made me lol

Re:1 in 31 US Citizens in custody or parole (0)

Anonymous Coward | about 4 years ago | (#33844708)

Just cap your circle of relationships at 30 and you'll be fine.

32 bit signed integer (0, Redundant)

DFurno2003 (739807) | about 4 years ago | (#33844116)

this have anything to do with the max value of a 32 bit signed integer? just popped into my head...

How uplifting (1)

cacba (1831766) | about 4 years ago | (#33844132)

Is it odd that I feel secure in their incompetence?

CSV To The Rescue! (5, Funny)

WidgetGuy (1233314) | about 4 years ago | (#33844134)

The actual data was only about 500K. The rest was XML markup.

Re:CSV To The Rescue! (1)

GarryFre (886347) | about 4 years ago | (#33844156)

Lawl!! I love it! that's great!

Datatype Limit? (0)

Anonymous Coward | about 4 years ago | (#33844154)

2 billion sounds like the records are numbered with signed long's as they have a range of -2 147 483 648 to 2 147 483 647. It would explain why the system just fell over rather than slowing grinding to a halt, anyway.

Re:Datatype Limit? (1)

Sulphur (1548251) | about 4 years ago | (#33844244)

Your Honor, my client is numbered as -2147483648. Since this is obviously incorrect I move for dismissal.

32 bit signed integer strikes again (4, Interesting)

Co0Ps (1539395) | about 4 years ago | (#33844166)

2 billion? That's awkwardly close to 2147483647... This is why your ID field should be BIGINT and not INT.... They where probably logging coordinates etc.

Re:32 bit signed integer strikes again (2, Insightful)

HyperQuantum (1032422) | about 4 years ago | (#33844312)

2 billion? That's awkwardly close to 2147483647... This is why your ID field should be BIGINT and not INT....

And I see no reason why someone would use a signed integer for an ID field. You're wasting half of the type's range (assuming negative ID's are not used).

Re:32 bit signed integer strikes again (1, Informative)

Anonymous Coward | about 4 years ago | (#33844386)

sorry, but Microsoft SQL Server doesn't support unsigned integers.

Re:32 bit signed integer strikes again (0)

Anonymous Coward | about 4 years ago | (#33844470)

+1 Unfuckingbelievable

2 billion (-1)

Anonymous Coward | about 4 years ago | (#33844178)

Somehow, I suspect it's exactly 2147483648 records... ;)

oceans (0)

Anonymous Coward | about 4 years ago | (#33844202)

obviously, this is false! the system must have crashed "accidental" when some criminals were in the middle of a heist!

Data loss is not guaranteed (2, Interesting)

Ebbesen (166619) | about 4 years ago | (#33844210)

I'm not sure any data has been lost. Say they have a table with the following columns:

id (auto increment)
felonid
gps
timestamp ...

If the 2 billion number is simply id that has run over, there's still enough data in the database to recreate the felons whereabouts using the gps and timestamp columns. Might be a problem in the system pulling data (based on id), but probably no data has been lost.

Re:Data loss is not guaranteed (2, Informative)

Grygus (1143095) | about 4 years ago | (#33844248)

You're right. From the press release: '“Importantly, the monitoring system continued to operate and gather information, but transmissions were delayed until the system was restored. Offender activity logged while the server was being worked on was effectively processed at 7:25 p.m. MT when the system was restored. Alerts that may have occurred during this period were transmitted to our customers at that time."'

Database wrong type? (1)

hcs_$reboot (1536101) | about 4 years ago | (#33844214)

Looks like the ID type of the row in the database was a signed integer, overflowed from ~2 billions.
In this case the problem would be a simple database structure design problem, using the wrong type (with heavy consequences).

On MySQL instead of int, using bigint would have allowed up to the theoretical value of 2^63-1 records.

Re:Database wrong type? (1)

TheLink (130905) | about 4 years ago | (#33844300)

BTW if you ever need to load billions of records (restore or upgrade) into a DB be prepared to wait for hours. Hope the boss/customer doesn't expect it'll be done in just an hour or so ;).

This guy took 45 minutes to insert 40 million rows:
http://www.justincarmony.com/blog/2009/01/12/mysql-40-million-rows-myisam-innodb/ [justincarmony.com]

This guy probably did better:
http://www.dslreports.com/shownews/Mysql-and-a-billion-rows-using-innodb-87890 [dslreports.com]

At a prev work place they were using an older version of MySQL and the DB guy had to resort to switching from innodb to myisam just to load in a multiGB DB (he was restoring from a backup). Not good to lose transactions just because of this limitation, but given they picked MySQL, I doubt they cared that much about data integrity (the company did lose data or have it corrupted because of MySQL more than once, but hey the company survived it ;) ).

Re:Database wrong type? (0)

Anonymous Coward | about 4 years ago | (#33844846)

Don't worry, Innodb is now a gateway drug to Oracle. It can handle a few more rows than can MySQL.

Lindsay Lohan celebrated the news (0)

Anonymous Coward | about 4 years ago | (#33844276)

With her traditional keg stand.

Slashdot had this problem (5, Informative)

Chris Snook (872473) | about 4 years ago | (#33844284)

Anyone remember when Slashdot hit 16,777,215 comments, and overflowed MEDIUMINT? The ALTER TABLE statement that fixed it took hours to run. I shudder to think how long it'll take to fix this, even with the problem diagnosed.

Re:Slashdot had this problem (0)

Anonymous Coward | about 4 years ago | (#33844320)

Only a few minutes on modern hardware. Mostly just depends on the hdd.

Hmmmm (3, Interesting)

luis_a_espinal (1810296) | about 4 years ago | (#33844314)

Sharding? Partitioning? But most importantly, using 64bit int types (or bigger) rather than 32-bit ints for primary indexes? I mean, what the hell they were using to store that data anyways? A Visicalc spreadsheet running on a TRS-80?

censorship not just for 3rd world megalomaniacs (-1, Offtopic)

Anonymous Coward | about 4 years ago | (#33844348)

just add immeasurable amounts of MISinformation, & there you have IT? that's US? thou shalt not... oh forget it. fake weather (censored?), fake money, fake god(s), what's next? seeing as we (have been told) came from monkeys, the only possible clue we would have to anything being out of order, we would get from the weather.

the search continues;
google.com/search?hl=en&source=hp&q=weather+manipulation

google.com/search?hl=en&source=hp&q=bush+cheney+wolfowitz+rumsfeld+wmd+oil+freemason+blair+obama+weather+authors

meanwhile (as it may take a while longer to finish wrecking this place); the corepirate nazi illuminati (remember, we came from monkeys, & 'they' believe they DIDN'T, & continue to demand that we learn to live on less/nothing while they continue to consume/waste/destroy immeasurable amounts of stuf/life, & feast on nubile virgins with their friend morgion) is always hunting that patch of red on almost everyones' neck. if they cannot find yours (greed, fear ego etc...) then you can go starve. that's their (slippery/slimy) 'platform' now. see also: http://en.wikipedia.org/wiki/Antisocial_personality_disorder

never a better time to consult with/trust in our creators. the lights are coming up rapidly all over now. see you there?

greed, fear & ego (in any order) are unprecedented evile's primary weapons. those, along with deception & coercion, helps most of us remain (unwittingly?) dependent on its' life0cidal hired goons' agenda. most of our dwindling resources are being squandered on the 'wars', & continuation of the billionerrors stock markup FraUD/pyramid schemes. nobody ever mentions the real long term costs of those debacles in both life & any notion of prosperity for us, or our children. not to mention the abuse of the consciences of those of us who still have one, & the terminal damage to our atmosphere (see also: manufactured 'weather', hot etc...). see you on the other side of it? the lights are coming up all over now. the fairytail is winding down now. let your conscience be your guide. you can be more helpful than you might have imagined. we now have some choices. meanwhile; don't forget to get a little more oxygen on your brain, & look up in the sky from time to time, starting early in the day. there's lots going on up there.

"The current rate of extinction is around 10 to 100 times the usual background level, and has been elevated above the background level since the Pleistocene. The current extinction rate is more rapid than in any other extinction event in earth history, and 50% of species could be extinct by the end of this century. While the role of humans is unclear in the longer-term extinction pattern, it is clear that factors such as deforestation, habitat destruction, hunting, the introduction of non-native species, pollution and climate change have reduced biodiversity profoundly.' (wiki)

"I think the bottom line is, what kind of a world do you want to leave for your children," Andrew Smith, a professor in the Arizona State University School of Life Sciences, said in a telephone interview. "How impoverished we would be if we lost 25 percent of the world's mammals," said Smith, one of more than 100 co-authors of the report. "Within our lifetime hundreds of species could be lost as a result of our own actions, a frightening sign of what is happening to the ecosystems where they live," added Julia Marton-Lefevre, IUCN director general. "We must now set clear targets for the future to reverse this trend to ensure that our enduring legacy is not to wipe out many of our closest relatives."--

"The wealth of the universe is for me. Every thing is explicable and practical for me .... I am defeated all the time; yet to victory I am born." --emerson

no need to confuse 'religion' with being a spiritual being. our soul purpose here is to care for one another. failing that, we're simply passing through (excess baggage) being distracted/consumed by the guaranteed to fail illusionary trappings of man'kind'. & recently (about 10,000 years ago) it was determined that hoarding & excess by a few, resulted in negative consequences for all.

consult with/trust in your creators. providing more than enough of everything for everyone (without any distracting/spiritdead personal gain motives), whilst badtolling unprecedented evile, using an unlimited supply of newclear power, since/until forever. see you there?

"If my people, which are called by my name, shall humble themselves, and pray, and seek my face, and turn from their wicked ways; then will I hear from heaven, and will forgive their sin, and will heal their land." )one does not need to agree whois in charge to grasp the notion that there may be some assistance available to us(

boeing, boeing, gone.

Maybe the answer isn't better software (5, Insightful)

assertation (1255714) | about 4 years ago | (#33844498)

Maybe the answer isn't better software, but fewer criminals to fill up the database with.

I keep seeing articles here and there how the U.S. has more people imprisoned than China. A large chunk of the prison population are inmates convicted of drug crimes and a large portion of that set of people were convicted on marijuana laws.

I don't smoke, but as a tax payer I would rather see the government make marijuana into a tax revenue generator instead of a huge expense to paid for with taxes.
 

Re:Maybe the answer isn't better software (4, Insightful)

WinstonWolfIT (1550079) | about 4 years ago | (#33844562)

The Chinese use a simpler, more lethal solution to prison overcrowding.

Re:Maybe the answer isn't better software (1)

TheScreenIsnt (939701) | about 4 years ago | (#33844592)

Our jails and prisons are also swelling because of the slow death (through years of budget cuts) of our community mental health system. A scholarly article on the matter [psychiatryonline.org]

Re:Maybe the answer isn't better software (0)

Anonymous Coward | about 4 years ago | (#33844814)

As long as marijuana is illegal, then people caught with it are criminals. They aren't innocent because you don't agree with the law.

Where's Waldo? (1)

hcdejong (561314) | about 4 years ago | (#33844540)

We don't know!

(check the name of the BI Inc spokesman)

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?