Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

What Developers Can Learn From Healthcare.gov

Soulskill posted 1 year,18 days | from the prepare-for-overwhelming-demand-or-prepare-for-whining dept.

Programming 267

An anonymous reader writes "Soured by his attempt to acquire a quote from healthcare.gov, James Turner compiled a short list of things developers can learn from the experience: 'The first highly visible component of the Affordable Health Care Act launched this week, in the form of the healthcare.gov site. Theoretically, it allows citizens, who live in any of the states that have chosen not to implement their own portal, to get quotes and sign up for coverage. I say theoretically because I've been trying to get a quote out of it since it launched on Tuesday, and I'm still trying. Every time I think I've gotten past the last glitch, a new one shows up further down the line. While it's easy to write it off as yet another example of how the government (under any administration) seems to be incapable of delivering large software projects, there are some specific lessons that developers can take away. 1) Load testing is your friend.'"

Sorry! There are no comments related to the filter you selected.

What can they learn (0, Funny)

Anonymous Coward | 1 year,18 days | (#45040229)

Bribing the procurement officer gets you juicy gov tenders ... yes muthafuka! dats true!

Re:What can they learn? (0)

sycodon (149926) | 1 year,18 days | (#45040459)

Not one god damned fucking thing.

Re:What can they learn? (1, Informative)

slick7 (1703596) | 1 year,18 days | (#45040949)

Not one god damned fucking thing.

Not true, I learned that the portal is as useful as a politician. Considering the failure to balance the budget, reining in of these arrogant bastards who declared war on the American people. Over time will one understand the uselessness of these politicians and their insurance industry written healthcare policies. These CONgressMEN are as bankrupt as the nation they supposedly lead.

Re:What can they learn? (5, Insightful)

Albanach (527650) | 1 year,18 days | (#45041003)

Considering the failure to balance the budget

Why would you want to do this? If you had an income that fluctuated each year, would you not save in the good years so you could maintain a reasonable quality of lifestyle in the barren years? Or would you downsize your house and sell your car every other year as your income fluctuated.

Balancing the budget is not the challenge. The real challenge is finding a government that can save when the going is good, and convincing the US electorate of the need for a rainy-day fund, rather than giving it all back and more in tax breaks.

Real demand or Right-Wing DDOS? (0, Flamebait)

Anonymous Coward | 1 year,18 days | (#45040243)

Let's have our great media investigate if this is poor planning...or good planning if once the initial load gets through then they didn't overspend on equipment they don't need.

Or if there is a secret effort by the people who want this to fail to hire botnets and hackers to DDOS it... I wouldn't put it past them.

Re:Real demand or Right-Wing DDOS? (3, Interesting)

ackthpt (218170) | 1 year,18 days | (#45040271)

Let's have our great media investigate if this is poor planning...or good planning if once the initial load gets through then they didn't overspend on equipment they don't need.

Or if there is a secret effort by the people who want this to fail to hire botnets and hackers to DDOS it... I wouldn't put it past them.

Would be something to see a considerable amount of traffic going out from Newscorp ip addresses into the healthcare.gov servers.

nothing unusual, aside a few million malformed packets...

Re:Real demand or Right-Wing DDOS? (2)

ScottCooperDotNet (929575) | 1 year,18 days | (#45040347)

Let's have our great media investigate if this is poor planning...or good planning if once the initial load gets through then they didn't overspend on equipment they don't need.

Or if there is a secret effort by the people who want this to fail to hire botnets and hackers to DDOS it... I wouldn't put it past them.

Would be something to see a considerable amount of traffic going out from Newscorp ip addresses into the healthcare.gov servers.

nothing unusual, aside a few million malformed packets...

That would be an even more stupid idea than Newscorp buying MySpace.

Project Managers can learn giving only minimal time for QA, at the very end of the project, with no time allotted for corrections is bad practice.

Re:Real demand or Right-Wing DDOS? (2)

ackthpt (218170) | 1 year,18 days | (#45040435)

Let's have our great media investigate if this is poor planning...or good planning if once the initial load gets through then they didn't overspend on equipment they don't need.

Or if there is a secret effort by the people who want this to fail to hire botnets and hackers to DDOS it... I wouldn't put it past them.

Would be something to see a considerable amount of traffic going out from Newscorp ip addresses into the healthcare.gov servers.

nothing unusual, aside a few million malformed packets...

That would be an even more stupid idea than Newscorp buying MySpace.

Project Managers can learn giving only minimal time for QA, at the very end of the project, with no time allotted for corrections is bad practice.

"Are we meeting with some network engineers, tech writers and systems analysts?"

"No, we are meeting with a bunch of appointees who know next to nothing about the guts of the project.

"Great... we may as well watch cartoons."

Re:Real demand or Right-Wing DDOS? (5, Insightful)

lq_x_pl (822011) | 1 year,18 days | (#45040377)

Never attribute to malice that which is adequately explained by stupidity.
I'd have a hard time believing that the servers have been this consistently overwhelmed with traffic. A more likely explanation is that a poorly designed system was patched together from components hastily built from a thousand different vendors. The web-app equivalent of a diesel engine held together with duct-tape and baling wire was then rolled out without any real testing.
The only time, "Good enough for government work," has ever escaped my lips was when I was confronted with a marginally functional mess of spaghetti code.

Re:Real demand or Right-Wing DDOS? (4, Insightful)

hawguy (1600213) | 1 year,18 days | (#45040833)

Never attribute to malice that which is adequately explained by stupidity.

I'd have a hard time believing that the servers have been this consistently overwhelmed with traffic. A more likely explanation is that a poorly designed system was patched together from components hastily built from a thousand different vendors. The web-app equivalent of a diesel engine held together with duct-tape and baling wire was then rolled out without any real testing.

The only time, "Good enough for government work," has ever escaped my lips was when I was confronted with a marginally functional mess of spaghetti code.

You needn't source from multiple vendors to get a system that falls apart under load - single vendor solutions are also susceptible to such problems.. Even if you specify load testing in the contract, that doesn't mean that their load test had any relation to actual real-world load. Of course, the hard part is predidcting what load to expect, especially with a system that has a potential audience of 100+ million people.

Re:Real demand or Right-Wing DDOS? (0)

Anonymous Coward | 1 year,18 days | (#45040555)

They are surely looking for any excuse possible for their pathetic failures here. If there was a DDOS, then they would surely let us know.

Reminds me of vendor systems I deal with (2)

ackthpt (218170) | 1 year,18 days | (#45040247)

No accountability of the contractors, no accountability of those who were to oversee the contractors and no accountability of the people who were to oversee those overseeing the contractors.

and I was ønce bitten by a møøse nø realli!

Re:Reminds me of vendor systems I deal with (2, Interesting)

Anonymous Coward | 1 year,18 days | (#45040273)

working in the trenches on one of the state projects, it's clear that the main problem was the inability for the state overseers to make up their mind on the most basic of concepts. This cost us huge amounts of time and resources.

Re:Reminds me of vendor systems I deal with (0)

Anonymous Coward | 1 year,18 days | (#45040311)

Some states have been ready for months.
Didn't I read on /. that the first testing of healthcare.gov was scheduled for September 30?

Re:Reminds me of vendor systems I deal with (5, Informative)

ackthpt (218170) | 1 year,18 days | (#45040349)

I went through the site and found it responsive. Possibly the time of day and my western timezone had something to say about it, but had no issues.

Even CNN looks bad when something major happens and everyone hits them at once, despite humming along for months without any issues.

Re:Reminds me of vendor systems I deal with (4, Insightful)

Albanach (527650) | 1 year,18 days | (#45041035)

I'm not sure load testing alone would be the solution. For a site like this, I see little point in making the expenditure to handle all the day 0 traffic.

Rather they should have load tested to find out how many users they could safely serve. Then they should have simply restricted the number of active connections. Other users should have seen a static holding page. That way, everyone that gets through gets a good experience.

By adopting this approach, you can save money. And, given the publicity available pre launch, they could easily have explained how this would work so as to manage expectations. After the first few week or so, they would likely be able to manage the traffic comfortably.

Re:Reminds me of vendor systems I deal with (5, Informative)

Mr. Freeman (933986) | 1 year,18 days | (#45040915)

Having worked in government offices, I can tell you this is the real problem.

Because there are so many laws about making the government use contractors instead of hiring employees (because private sector is allegedly so much more efficient), damn near everything has to be contracted out. Then the contractors fail to deliver, they go over budget and come in way behind schedule. The government has no choice but to pay them and accept their useless work, again, due to more laws about "helping the private sector".

There's no way to fire a contractor or even to hold them to their original contract. They agreed to do something for a certain price? Too bad, they're going to sue the government and use those biased laws in order to deliver less than half of what they promised at more than 3 times the price they quoted and agreed to.

How is it even still up? (3, Insightful)

SuperKendall (25149) | 1 year,18 days | (#45040257)

Nothing shows up the sheer arbitrariness of a government shutdown than some sites like Healthcare.gov being up, and others being forced to shut down at extra expense when they could have just been left running (and the servers that are there just to tell you the site is shut down are still consuming power and bandwidth).

Can't 0wn a powered-off server (4, Informative)

tepples (727027) | 1 year,18 days | (#45040381)

I thought the consensus from the last story about the shutdown was that the web sites were closed because a server that's turned off is less likely to get 0wn3d without anyone there to fix it.

Re:Can't 0wn a powered-off server (1)

Derek Pomery (2028) | 1 year,18 days | (#45040999)

That would explain a closed website.
It does nothing to explain websites that were left on and serving a "shutdown" page, in some cases, using a redirect such that the actual page loads before sending you to the block page.

It is more directly comparable to Wikipedia's SOPA protest in function.

https://en.wikipedia.org/wiki/Washington_Monument_Syndrome [wikipedia.org] has been brought up a few times.

Re:How is it even still up? (3, Informative)

Anonymous Coward | 1 year,18 days | (#45040385)

It's up because they had a separately authorized source of funds.

Remember we haven't hit the debt limit yet, we hit the government budget limit.

Re:How is it even still up? (2)

Sarten-X (1102295) | 1 year,18 days | (#45040657)

Also remember that we hit a time limit. October 1st is just the start of the fiscal year, and the shutdown is just waiting for direction on how the next year is going to run.

Re:How is it even still up? (1, Troll)

nomadic (141991) | 1 year,18 days | (#45040393)

Nothing arbitrary about it; if the other side is trying to do an end-run around the democratic process to shut down a government program they don't like, why would you shut down that program for them?

Re:How is it even still up? (1, Troll)

cold fjord (826450) | 1 year,18 days | (#45040423)

It's not arbitrary, it's calculated to cause pain as a political move.

Obama Forcing Shut Down of Parks the Feds Don't Even Fund [breitbart.com]

Private Air Show Stopped Due to Government Shutdown [breitbart.com]

Obama Illegally Furloughing Civilian Defense Employees at STRATCOM [breitbart.com]

PRUDEN: The cheap tricks of the game [washingtontimes.com]

Monuments and memorials remained open during previous shutdown [dailycaller.com]

Republicans press Obama to back FEMA funding bill as storm nears [foxnews.com]

There is plenty more.

The Senate isn't being useful, but at least they did vote on something today [washingtonpost.com] : "... the Senate also unanimously approved a measure deeming next week as National Chess Week."

Re:How is it even still up? (2, Flamebait)

Ralph Wiggam (22354) | 1 year,18 days | (#45040505)

I wouldn't believe breitbart.com if it told me that the sky was blue.

Re:How is it even still up? (1)

cold fjord (826450) | 1 year,18 days | (#45040615)

Your disbelief is a different issue than the reports being wrong. But, I can understand your stance. Some people prefer to only drink from the approved water carriers, especially if news from other sources might cause uncomfortable facts and thoughts to creep into one’s mind. Rest easy with your approved news, citizen.

Re:How is it even still up? (1)

Anonymous Coward | 1 year,18 days | (#45040685)

Either that or the fact that breitbart has a terrible history of axe-grinding, making stuff up and being just plain wrong. But that is ok. You are free to believe the hit job against them is a massive conspiracy between the socialist commi-nazis and the Friends of Hamas.

Re:How is it even still up? (1)

Obfuscant (592200) | 1 year,18 days | (#45040673)

"Liberals refuse to accept things they've see with their own eyes when it would mean they agree with a conservative. News at 11."

Re:How is it even still up? (2)

hurfy (735314) | 1 year,18 days | (#45040767)

lol, no idea what the site was so had to check. Private airshow...was alternative to canceled military one....AT A MILITARY BASE... yup, totally not related to gov, eh? ;p

Good to know there are still sites with 'better'headlines than us ;)

Re:How is it even still up? (0)

Anonymous Coward | 1 year,18 days | (#45040787)

If you bother to check you'll find that the military isn't shut down. To the extent that there are military shutdowns, it is calculated to cause pain, not to comply with budget issues.

Re:How is it even still up? (0)

Anonymous Coward | 1 year,18 days | (#45040511)

Yay, Breitbart. The pinnacle of journalistic integrity.

Re:How is it even still up? (-1)

Anonymous Coward | 1 year,18 days | (#45040535)

Better than something like NPR, the well known bastion of liberal propaganda. Which, despite being a federal institution, is - you guessed it - still up.

Re:How is it even still up? (0)

Anonymous Coward | 1 year,18 days | (#45040623)

NPR is not a federal institution, you dope.

Re: How is it even still up? (0)

Anonymous Coward | 1 year,18 days | (#45040741)

Not even close

Re:How is it even still up? (2)

theshowmecanuck (703852) | 1 year,18 days | (#45040815)

I find that NPR reporting is probably the most neutral of all the broadcast news. Now before you start freaking out about commy liberal pinkos, I will say yes, there are individual commentators that have a liberal slant. But these are more like newspaper columnists. We know their perspective and can factor that in. But as far as the actual news reporting, it seems to be pretty factual. Strangely enough, I find that a lot of Fox news website stories have pretty decent reporting too; but then again I have avoided political stories there. But their TV broadcasts are total right wing shilling shite. Come to think of it, most of the print/website stories from most news orgs are decent. It is the broadcast stuff where essentially non-journalist commentators get involved where you find the various leanings.

Re:How is it even still up? (1)

linuxguy (98493) | 1 year,18 days | (#45040589)

Please resists the urge to cite breitbart dailycaller or foxnews as your sources. These places do not have any credibility in many people's eyes.

Re:How is it even still up? (3, Informative)

hondo77 (324058) | 1 year,18 days | (#45040611)

Conservatives are outraged that their government shutdown caused some things to actually shut down. Film at 11.

Re:How is it even still up? (0, Troll)

cold fjord (826450) | 1 year,18 days | (#45040631)

Progressives and Democrats claim outrage that the government is shut down, but continue to resist efforts to open it - take targeted actions to magnify the pain. Film at 11.

Re:How is it even still up? (4, Insightful)

the eric conspiracy (20178) | 1 year,18 days | (#45040707)

All that needs to happen is for Boehner to bring the Senate bill to the floor of the House and BOOM the government will reopen because there are enough moderate Republicans + Democrats to pass it.

The idea that the Democrats are forcing the Government to close is ludicrous.

Re:How is it even still up? (1, Insightful)

Obfuscant (592200) | 1 year,18 days | (#45040773)

All that needs to happen is for Boehner to bring the Senate bill to the floor of the House and BOOM the government will reopen because there are enough moderate Republicans + Democrats to pass it.

All that needed to happen for it not to happen at all is for the Senate Democrats to jump the party line and approve the continuing resolution the House had already passed.

The idea that the Democrats are forcing the Government to close is ludicrous.

They're the ones who control the Senate and decided to force a conference committee which they knew wasn't going to accept their version. They're also the party of the current President, who is refusing to negotiate. From here: [go.com] :

But Democratic leaders Rep. Nancy Pelosi, D-Calif., and Sen. Harry Reid, D-Nev., said the president reiterated that they would hold firm in their position.

So, no, the Democrats are not the innocent party here. They'd rather see a shutdown than a delay in funding ACA which doesn't prevent the exchanges from opening anyway.

Re:How is it even still up? (2)

hondo77 (324058) | 1 year,18 days | (#45040919)

Wrong. The Senate voted on the House's bill and rejected it. The House refuses to vote on the Senate's bill (because Boehner knows it will pass). It really is that simple.

Re:How is it even still up? (2)

Obfuscant (592200) | 1 year,18 days | (#45040979)

Wrong. The Senate voted on the House's bill and rejected it.

How am I wrong when that's what I said? The Democrats in the Senate toed the party line and refused to pass the House continuing resolution, substituting their own, forcing the matter to a conference committee that they knew wouldn't accept their version. Had the Senate Democrats opted to avoid the shutdown, all they had to do was pass the House bill intact. It really was that simple.

Re: How is it even still up? (0)

Anonymous Coward | 1 year,18 days | (#45040751)

And _who_ is the leader of the house who had not even presented the actual budget for a vote? That's what I thought...

Re: How is it even still up? (0)

Anonymous Coward | 1 year,18 days | (#45040831)

That would be Senator Harry Reid (D-NV), leader of the Senate that didn't produce a budget for more than 1,400 days. It should be an annual event.

1,400 Days Later, Still No Senate Budget [senate.gov]

Senator Reid still hasn't designated a conference committee to work with the House on the current budget matters.

Is that who you thought?

Re:How is it even still up? (2)

Sarten-X (1102295) | 1 year,18 days | (#45040735)

Alright, I'll bite. One of those headlines is preposterous enough to warrant clicking... I wonder how exactly one pulls off a "private air" anything, what with the FAA, airports, and various safety groups all being government bodies...

Okay, that wasn't too bad, just run-of-the-mill ignorance. It's private aircraft flying from a Marine base, and with no budget the military can't legally authorize the expense of opening the base and running the show. As expected, the article makes a big deal about a wholly-expected consequence. Maybe someone will point this out in the comments...

OH DEAR GOD GET ME BACK TO SLASHDOT!

My fellow Slashdotters, you folks are jackasses sometimes. So am I, I'm sure... but I thank you all heartily for at least being intelligent asshats.

Re:How is it even still up? (1)

cold fjord (826450) | 1 year,18 days | (#45040749)

The military isn't shut down.

Re:How is it even still up? (1)

Obfuscant (592200) | 1 year,18 days | (#45040829)

I wonder how exactly one pulls off a "private air" anything, what with the FAA, airports, and various safety groups all being government bodies...

Use of public airspace does not make an airshow produced by private individuals a government function. Obtaining the necessary FAA waivers and TFRs and NOTAMs for an airshow does not make a private airshow a government function. There are private, state, city, and county run airports all over the place. And EAA is a private organization that regularly holds one of the largest aviation events [airventure.org] in the world.

Re:How is it even still up? (4, Informative)

ackthpt (218170) | 1 year,18 days | (#45040447)

Nothing shows up the sheer arbitrariness of a government shutdown than some sites like Healthcare.gov being up, and others being forced to shut down at extra expense when they could have just been left running (and the servers that are there just to tell you the site is shut down are still consuming power and bandwidth).

One more time, because some people clearly haven't read it or heard it: The Affordable Healthcare Act is not affected because it was fully funded. The budget Continuing Resolution is for things which are not already funded.

Re:How is it even still up? (1)

WaffleMonster (969671) | 1 year,18 days | (#45041027)

Nothing shows up the sheer arbitrariness of a government shutdown than some sites like Healthcare.gov being up, and others being forced to shut down at extra expense when they could have just been left running (and the servers that are there just to tell you the site is shut down are still consuming power and bandwidth).

Apparently nearly every government agency under the sun has taken to sabotaging their sites to I assume make a statement about how much not getting paid sucks.

While I understand it is still childish and offensive to taxpayers. I would respect an agency if their site and servers were actually shut down or if they left a message saying sorry content may not be up to date... very few I know anything about are actually doing that.

Blame Canada? (4, Interesting)

cold fjord (826450) | 1 year,18 days | (#45040281)

Canadian firm hired to build troubled Obamacare exchanges [washingtonexaminer.com]

A Canadian tech firm that has provided service to that country's single-payer health care system is behind the glitch-ridden United States national health care exchange site healthcare.gov.

CGI Federal is a subsidiary of Montreal-based CGI Group. With offices in Fairfax, Va., the subsidiary has been a darling of the Obama administration, which since 2009 has bestowed it with $1.4 billion in federal contracts, according to USAspending.gov.

The "CGI" in the parent company's name stands for "Conseillers en Gestion et Informatique" in French, which roughly translates to "Information Systems and Management Consultants." However, the firm offers another translation: "Consultants to Government and Industry."

The company is deeply embedded in Canada’s single-payer system. CGI has provided IT services to the Canadian Ministries of Health in Alberta, British Columbia, New Brunswick, Quebec and Saskatchewan, as well as to the national health provider, Health Canada, according to CGI's Canadian website.

Re:Blame Canada? (1)

Stoutlimb (143245) | 1 year,18 days | (#45040537)

It's a Quebec company... they're fucked. It's like hiring a "European" company which just happens to be run out of Sicily. They're so stupidly corrupt there, that I can honestly say they deserved it.

Re:Blame Canada? (2)

quantaman (517394) | 1 year,18 days | (#45040935)

According to the article the project has been behind schedule for a while:

Earlier this year the U.S. Government Accountability Office criticized the pace of development and testing for Healthcare.gov.'s IT system and noted that it was missing important milestone deadlines.

This is worrying as it suggests this isn't the case of a few glitches and poor load testing, the project might simply not be done.

In defence of CGI (since I'm Canadian and will reflexively look for excuses for my cultural brethren) it's not uncommon for software projects to miss launch dates, they just seem to be in the unfortunate position of having to launch anyways. The other excuse is that their requirements exploded when 34 states refused to join ObamaCare and had to be handled by the Federal exchange.

main quote (2)

phantomfive (622387) | 1 year,18 days | (#45040291)

Here's the quote from the article that I consider key:

The biggest takeaway though, is that the way that the federal government bids out software is fundamentally broken. There are clearly companies in the industry who understand exactly the kind of problems that healthcare.gov needed to address. Intuit’s online TurboTax is much more complicated than the sign-up process for healthcare, and it works under heavy load. Amazon and Google both handle crushing loads gracefully as well. Why can’t the government draw on this kind of expertise when designing a site as critical to the public as healthcare.gov, rather than farming it out to the lowest bidder?

Although it's not entirely right.....government contracts are more complicated than 'going to the lowest bidder.'

Re:main quote (2)

g01d4 (888748) | 1 year,18 days | (#45040399)

While the contracts may be more complicated you've got to wonder whether the right incentives are built in. Perhaps the gov't could have tied payment (or penalty) to certain post delivery metrics such as average time to sign up. What are the incentives that make e.g. Amazon, Google and Facebook software deliver a better user experience and how can they be incorporated into the contract?

Re:main quote (2)

Sarten-X (1102295) | 1 year,18 days | (#45040967)

Government contracts typically come with a large list of requirements (on the order of 500 pages), almost entirely written by a committee with no idea what they're actually looking for. They'll require silly things like "must weigh over 1750 pounds" or "[a Windows XP system] must be accessible via VT-100 terminal", or my personal favorite, "all components [including electronics] must be manufactured in the United States or France".

I'm told, though I haven't seen it myself, that the requirements aren't actually all enforced, but instead provide an escape mechanism the government can pull out when it wants to add features that weren't in the original plan. They'll ask for the new feature, and if the contractor refuses, they'll void the contract and claim it's because of all the missing required features. They do usually also append funding for the new features, but it's

With so many requirements, of course a few key details like stress testing get missed. The political folks writing the specs don't always think about such trivialities. That means it's more difficult to get funding for such testing, and you certainly shouldn't expect government help.

The problem with load testing is that it's as much an exercise in testing resources as in the application's efficiency. When it comes time to simulate a million users' load on the server, Amazon or Google could just spin up a few thousand virtual machines on their spare capacity, and simulate a few thousand users on each one. Smaller companies have to make do with what they have - probably a few old servers running a few hundred simulations. Combined with the forced feature creep and short-but-we-can-extend-it-and-blame-you deadlines, I'm always a little surprised when a government website runs at all.

Re:main quote (2)

Obfuscant (592200) | 1 year,18 days | (#45040651)

Intuit's online TurboTax is much more complicated ..

Correct me if I'm wrong, but isn't it Intuit's TurboTax that scribbled data into some of the first 63 sectors of the user's hard drive as a primitive means of DRM? Yes [slashdot.org] , I did remember correctly. They're also the company that runs my credit union's web presence and have arbitrarily decided what characters a valid email address can contain -- in violation of the RFC. Certainly, let's have Intuit do the website for people who need health insurance and must buy it or face penalties.

Re:main quote (1)

hondo77 (324058) | 1 year,18 days | (#45040661)

Intuit's online TurboTax is much more complicated than the sign-up process for healthcare, and it works under heavy load.

To be fair, TurboTax didn't always work well under heavy load. It has evolved over the years so now it works just fine. Something to keep in mind.

Re:main quote (1)

DaHat (247651) | 1 year,18 days | (#45040989)

To be fair, TurboTax didn't always work well under heavy load. It has evolved over the years so now it works just fine. Something to keep in mind.

So... because it took a while for TurboTax to reach the level of stability they have today... we should just accept the feds incompetence in this area?

I would think what should be kept in mind is those who built & run healthcare.gov seemingly never bothered to reach out to companies & organizations which run massive data systems that can handle heavy load without falling over.

Clearly no one from the HHS ever called up a systems admin or developer at the NSA and asked "How do we make sure we can to the cloud^H^H^H^H^Hworld?" ... Let alone someone at Apple, Amazon, Facebook, Google, Microsoft, Rackspace, Twitter or any number of other companies who have massive online presences and do not fail as quickly & easily like healthcare.gov... doubly so with the 3 years of lead-time the feds had on this project.

Crazy requirement - usernames with numbers??? (2)

SuperKendall (25149) | 1 year,18 days | (#45040299)

From the list, one of the items casually mentions that usernames require numbers. What? I've never heard of a requirement like that from any other consumer system, ever.. they may suggest it (like YourName024 when a prior user has already used YourName) but do not require it.

If they worry about uniqueness, just use email addresses as logins.

Re:Crazy requirement - usernames with numbers??? (0)

Anonymous Coward | 1 year,18 days | (#45040339)

Golly, I hope 420BlazeItFaggot is still available.

When you leave your ISP (2)

tepples (727027) | 1 year,18 days | (#45040389)

If they worry about uniqueness, just use email addresses as logins.

That's exploitable when you leave your ISP, someone else claims your username at that ISP, and your old ISP-provided e-mail address now points to another person.

Re: When you leave your ISP (1)

corychristison (951993) | 1 year,18 days | (#45040795)

Who the hell uses the email address provided by their ISP?

Personally, I use paid e-mail hosting services for my family's e-mail needs, at a domain that I own.

Re:Crazy requirement - usernames with numbers??? (2)

BoberFett (127537) | 1 year,18 days | (#45040493)

I've seen that requirement from banks, and a gym of all places.

two things you can learn from it (1)

stenvar (2789879) | 1 year,18 days | (#45040305)

If you can get it, get a government contract to implement some huge IT system; you can have cost overruns up the wazoo, miss your deadlines, and create unusable interfaces; there will likely be few consequences, the customers can't run away from you, and the pockets of the government are infinitely deep to cover whatever you want.

If you can't get in on such a boondoggle as a vendor, vote against any kind of politician who promises to solve problems with some huge, government-paid IT system; they rarely are cost-effective solutions or very usable.

Your friend. (1)

jamesl (106902) | 1 year,18 days | (#45040309)

"Load testing is your friend."

In this case, any testing at all would have been friends to both developers and customers.

What developers can learn: (0)

Anonymous Coward | 1 year,18 days | (#45040315)

BAHAHAHAHAHAHAHAHAHAHA

From what I've seen, the only thing that hasn't gone tits up was the UI.

Poor execution will spoil even the best designs. Since this was contracted out to a hundred different lowest-bidders, I'm sure integration issues have compounded a laundry list of poor choices.

web developers, not developers (0, Flamebait)

Anonymous Coward | 1 year,18 days | (#45040327)

A lot of developers don't work on bunk ass ghetto web shit

"Launched" is such an optimistic word... (4, Funny)

cirby (2599) | 1 year,18 days | (#45040361)

"Launch" suggests that it actually, you know, worked.

When a quarter million people hit a game company's servers and only half of them get to play, it's a disaster of unrivaled proportions.

When millions of people hit billions of dollars in government investment and a few thousand of them actually get the site to work at all, it's a "learning experience."

Re:"Launched" is such an optimistic word... (1)

cold fjord (826450) | 1 year,18 days | (#45040515)

s/launched/lurched/g

Or, if you prefer: a car analogy [youtube.com] (You can either jump to 1:07, or watch the whole thing.)

Re:"Launched" is such an optimistic word... (0)

Anonymous Coward | 1 year,18 days | (#45040541)

When millions of people hit billions of dollars in government investment and a few thousand of them actually get the site to work at all, it's a "learning experience."

the article itself says the federal government has a broken system when awarding contracts, what makes the article funny is suggesting the feds contract out to the cheapest companies that are inexperienced in dealing with a heavily traffic sites.

"" TurboTax which is far more complicated then healthcare.gov has no problem under heavy loads, Amazon, Google which receive crushing loads work without any hangups""

Considering Amazon, Google get away with questionable practices from government regulations, these companies would probably do it for the same price and the site would work far better. You are going to still have issues but they would be minor and fixed within minutes or hours. healthcare.gov was thought of more as a poorly built video game from its UI, and UX, then an actual functioning site.

Health care from the same gov't as the NSA (1)

Anonymous Coward | 1 year,18 days | (#45040403)

Yep, let's turn over even more money and power to the same government that gives us the NSA, TSA, Patriot Act, and who knows what else that hasn't been leaked.

Yeah, that governement is going to help us, right?

No worse/better than private business. (4, Insightful)

Above (100351) | 1 year,18 days | (#45040431)

GTA V? Sim City? Final Fantasy? Battlefield?

Turns out millions of users who start using something on the same day often don't follow the expected and tested for behavior.

Anyone who launches a service like this should expect to spend the first week in triage mode, and the first month making adjustments. I'd like to say proper planning would mean that never occurs, but the only way to insure that would be to spend 10x what is really needed. People would hate the government even worse if they did that.

This is not news, yet. It will be news in a month if it is still fubared.

Re:No worse/better than private business. (1)

alostpacket (1972110) | 1 year,18 days | (#45040539)

but the only way to insure that would be

... to log on to website and buy insurance :)

Re:No worse/better than private business. (1)

BoberFett (127537) | 1 year,18 days | (#45040553)

That's what scalable architectures and cloud services are designed for. Expand and contract as necessary.

State Sites Also (1)

jasnw (1913892) | 1 year,18 days | (#45040445)

The Washington State's exchange website, for which the state paid $54 million to Delloite LLC, hasn't been a rollicking success either. I'm trying to wrap my head around why it costs $54 million to set up a pretty straight-forward website (costs evidently do not include hardware, just people/time/software). I believe that cost was over half what the state received from the feds to set up the exchange. Details here (such as they are) [bizjournals.com] .

planned? (0)

Anonymous Coward | 1 year,18 days | (#45040453)

NPR this morning says Obamacare was rolled out as planned. They planned this?

The basic problem (0)

Anonymous Coward | 1 year,18 days | (#45040463)

Was trusting the government to set it up. Health insurance should be sold like car, or auto, or any other insurance. Market based pricing should be the rule. Portable no matter where you live in the US.

Asking the feds to create a market was sheer hubris. They are not capable of creating a budget, much less balancing it. Why would we believe they could accomplish something on this scale?

Re:The basic problem (3, Funny)

nomadic (141991) | 1 year,18 days | (#45040691)

"Why would we believe they could accomplish something on this scale?"

Because they are the only ones who actually have successfully created healthcare systems on that scale, specifically medicare, medicaid, and the VA system.

How about management? (1)

Sarusa (104047) | 1 year,18 days | (#45040489)

When your boss says you're going to launch on October 3 no matter what, you get whatever you've got.

I've occasionally (thankfully not often) had to turn out things I'm not proud of for customers who have no idea how to schedule and won't hear otherwise. Stuff like the front end/back end error handling is high up the chopping block.

Re:How about management? (1)

DaHat (247651) | 1 year,18 days | (#45041017)

I guess my boss & I are better at planning than 'your boss'... as we'll work together to scope out what sort of work is realistic in a given time frame... do that, and then re-evaluate... and do this multiple times during a dev/release cycle.

By the end not everything we wanted may be in the box, but it works.

Though not being able to get this whole mess right after 3 years another sign of their poor planning.

You meant, "What Project Managers can learn..." (3, Insightful)

quietwalker (969769) | 1 year,18 days | (#45040565)

I've got a personal gripe about folks who think that 'developer' is code for 'guy who's expected to do everything in the project'. Outside of small projects, that's not how it should work in a healthy software development lifecycle.

Developers architect and write code, and some of the topics covered in that short editorial are relevant; use of AJAX necessitates good error handling on the front end, and synchronization of client and server side validations. Sure, they may have a broad skillset besides and understand databases, and graphical design, and so on, but there's no guarantee they're the ones meant to provide those skills.

For example, QA encompasses an incredibly large set of skills, familiarity with a wide range of products, and to be fair, seems to attract folks with a different life philosophy than those who identify themselves as developers. To talk about load testing - which itself is not a simple unit test to be added to a build - as a developer's responsibility, and ignore the vast, separate set of specialized knowledge and experience required to pull it off is ignorance. To include UX and UI design, and say these too are in the developers purview is equally misguided. (in fact, most developers are really, really bad at UI/UX, for some reason)

Not that a developer couldn't do those things, or will automatically lack the knowledge or skills, but those are separate roles and separate disciplines.

So, tell a project manager that they should make sure the QA team does load testing, and tell the project manager that the UI/UX team needs to provide descriptive error messages when validation fails, and so on. Very little of this is important to someone who's currently wearing the 'developer' hat.

very interesting situation (4, Interesting)

spiffmastercow (1001386) | 1 year,18 days | (#45040573)

The devs are in a pretty interesting situation that you don't see too often.. They're tasked with developing an application that generally can anticipate a low load level, except for one (and only one) extreme peak load. Do you develop for the general case, or the (very important) exception? Remember that the difference between these two options would make a difference in the basic structure of the app. Do you use a traditional RDBMS (perfect for the low load case), or some sort of no-SQL system (possibly necessary for the peak load case)? Remember that you can't leverage any commercial cloud resources either -- these are government records, and there are laws saying they'll have to be housed on government computers.

Uh duh. (2)

RJFerret (1279530) | 1 year,18 days | (#45040585)

Odd, in my state it worked fine...no, wait a minute, it's only Oct. 4th, who in their right mind with technical savvy or experience would access such a new product in the first week of it's availability?

I live in one of the most population dense states. My current health insurance is paid up through the end of the month. I won't be accessing the exchange for three weeks yet because everything in the article is obvious, but even if implemented within the time constraints to the best of their ability, will still probably have issues in the first few days.

Duh.

Re:Uh duh. (1)

Obfuscant (592200) | 1 year,18 days | (#45040731)

My current health insurance is paid up through the end of the month. I won't be accessing the exchange for three weeks yet ...

You better send of a couple more payments to your current insurance company. ACA coverage through the exchanges doesn't start until 1 Jan 2014.

I always wonder whyObama is having so much trouble (0)

Anonymous Coward | 1 year,18 days | (#45040587)

Given that the IT portion of his re-election campaign went so well. Then I'm reminded Google and Facebook pretty much showed up and offered to do it for him. I wish they did that on this given that this is arguably more important that that.

Stupid design (3, Informative)

seyfarth (323827) | 1 year,18 days | (#45040603)

I didn't make it very deep into the web site. I was mainly interested in reviewing the rates for my county. What a surprise that there was a list with all the states's counties together! I was expecting to fill in my zip code possibly or enter the state and county to get a list of available policies. The resulting table was large enough to generate bandwidth problems. One stupid error in design could saturate their network! A good design would be easier on the users, the network and the servers. Now sometimes you have to trade server time and convenience for user time and convenience, but this was apparently not thought through. Surely someone in the government must realize that good design works better than bad design. If a web site is to be used by millions, it obviously needs a good design.

architecture (3, Interesting)

worldthinker (536300) | 1 year,18 days | (#45040635)

Did a little sleuthing and discovered they're using an F5 load balancer in front of it (at least my state exchange is). I'm rather shocked that they chose a classical client/server architecture and not say, a cloud architecture for this. This could have been written on Google's cloud or Amazon's or OpenStack even and probably done a much better job of handling this load.

I would surmise that HIPPA requirements may have made cloud architecture problematic.

California's site is running well (1)

mspohr (589790) | 1 year,18 days | (#45040637)

I checked out California's exchange web site and it's running fast. No problem registering or logging in and it's well designed to let you see your options.
California is a state which is completely in control of the Democrats. We don't have the tea baggers who are trying to destroy government here. (I think there are a few but we ignore them and they are powerless.) Government (and most other things) work better here.

load balancing and queuing (1)

shuz (706678) | 1 year,18 days | (#45040639)

It is worth noting that a raspberry pie computer could handle the work load of all the requests for healthcare.gov with correct load balancing and queuing. For PR you would need to set some expectation such as estimated wait time to get into the system, however your customer base would at least know that the system is working and that they just need to wait their turn due to the high demand. It is incorrect for most systems to be architected to assume everyone who accesses your system gets helped right away. The only exception to that would be an emergency response system where peoples well being is at stake. Look at many classic support call lines for major companies. Though they often have certain shortcuts based on how much you pay them, they have queuing systems that means regardless of how many millions of dollars a second you are losing, they may not be able to help you for x period of time.

It is reasonable that healthcare.gov could not complete transactions with each person visiting immediately. It is not reasonable for a lack of a system advising users of either estimated wait time or at minimum notification that they have a place in line and that they will be helped at some point in the future with no further requirement of action on their part. Any developer or software/system architect creating a transactional system big or small would be wise to first code-in this mechanism. It will save them headaches, and maybe a weekend, at some point in the future.

Re:load balancing and queuing (1)

shuz (706678) | 1 year,18 days | (#45040737)

Admittedly Raspberry Pi as an example is a bit extreme for this workload. But for fun, think about this. 3byte session token is ~16.7 million 4 billion if you go 4byte. The Pi has 512MB of memory. 16.7 million bytes is about 50MB. So lets say you load embedded linux, a small web server, and support tools hmm 32MB. Think you web developers out there could write a website in perl, c, or c++ with only 430 MB of memory? You couldn't get too crazy with images, but I think someone out there could do it.

But what about session data? You can architect the system to server only one person at a time. Hopefully the profile on each person on the healthcare.gov website is not so large that you couldn't sneak it somewhere into your 430MB website.

Re:load balancing and queuing (1)

Obfuscant (592200) | 1 year,18 days | (#45040899)

16.7 million bytes is about 50MB.

I vote we hire you as the government website author. You've proven an ability to inflate simple numbers by a factor of three, which is about a factor of six less than current contractors usually do.

You can architect the system to server only one person at a time.

So if there are just 1 million people who need to sign up for insurance and they take ten minutes each to review the material and decide, that means you'd have all of them "servered" in just 19 years. The CT website that had 100,000 visitors in the first day would have had all of them dealt with by sometime in 2015.

Have Patience (3, Insightful)

BenSchuarmer (922752) | 1 year,18 days | (#45040645)

If a web site is rushed into place on October 1st but there's no reason to sign up until January 1st, wait several weeks before you try use it.

It's not slashdot. There's no advantage to getting FIRST POST!!!

Re:Have Patience (0)

Anonymous Coward | 1 year,18 days | (#45040683)

It's not slashdot. There's no advantage to getting FIRST POST!!!

The IRS isn't going to fine you if you don't sign up for CowboyNeal Care.

Re:Have Patience (1)

Obfuscant (592200) | 1 year,18 days | (#45040939)

If a web site is rushed into place on October 1st but there's no reason to sign up until January 1st,

Well, if you want to avoid a fine for not having insurance and you want to use the exchange, you need to have signed up by December 15. That would be a good reason not to wait until Jan 1.

Re:Have Patience (1)

Pretzalzz (577309) | 1 year,18 days | (#45040981)

You need to sign up by December 15th to get coverage January 1st. But your general point still stands.

Oregon in the same boat (1)

linuxguy (98493) | 1 year,18 days | (#45040647)

Oregon paid millions to Oracle for their own solution. It was a disaster. It did not work for me as I kept getting errors. And Oregon actually opted for a simple solution where you could not actually sign up for a plan online. You only received information about available plans. 3 years and millions of dollars later, they could not make that work reliably. As a developer I am baffled.

Re:Oregon in the same boat (1)

Anonymous Coward | 1 year,18 days | (#45040861)

Oracle won't be punished and they'll advertise their Oregon connection to get more contacts. Why waste money making something that works when you don't have to?

Re:Oregon in the same boat (1)

Obfuscant (592200) | 1 year,18 days | (#45041019)

And Oregon actually opted for a simple solution where you could not actually sign up for a plan online.

No, that's not what we opted for. The exchange is supposed to allow people to sign up, but since the site wasn't completed and couldn't provide information on the prices based on income, they disabled the ability to sign up and have made it "coming soon".

what i learned from the health exchange (0)

Anonymous Coward | 1 year,18 days | (#45040953)

is that the federal exchange is also funneling all data it collects to dhs (fbi/cia/etc) for reasons unspecified..... which is totally not needed for the exchange web site(s) to do their job.. any identity or eligibility verification that is needed can be done without giving a single bit of data to dhs or any of its agencies or departments.

Why it wasn't easy to handle the number of users.. (4, Interesting)

JWSmythe (446288) | 1 year,18 days | (#45040961)

    Everyone goes on the assumption that scale is "just make it bigger". I'd like to add some of my own notes on why this launch was doomed from the start.

    I used to work for an adult internet company who had massive traffic. We were serving millions of people daily before 2000. We would exceed 10M daily viewers about once a week. That fluctuated by rather consistent calendar influences, like the day of the week, part of the month, and part of the year. Sept 11, 2011 dropped 3/4 of our traffic for almost exactly 2 hours. So we knew how long huge news event would impact us.

    To handle 10M customers without a hiccup, we had to consider a lot of things. We didn't do much dynamic content. That's a killer. There were some elements that had to be dynamic, such as the voting/polling systems, message forums, etc. Otherwise, we had to try to keep the pages (html and images) as light as possible.

    The hardest abused system we had was user authentication and authorization. We only had a few million users that hit it, but there were thousands of hackers (and script kiddies) that wanted to try to get something for nothing. Come on, it was cheap porn, just pay for it. We could easily see over 10M auth requests per hour. In time, we fine tuned the system, and outright blocked abusive users at the firewall.

    The advantage we had was, when I was first in control over the IT work, we'd only see about 1M/day, so we had the luxury of growing it out. We'd watch for the problematic parts, and fix them. What works on your test bed where 10,000 users try it, even if they try hard, it doesn't mean you can put it on 100 servers and expect it to work for 1M users.

    healthcare.gov has some other severe disadvantages. From what I understand, they are hitting the SSA database. I don't know if that's an online query to the SSA, or if they're provided a static file to import periodically. I'd assume all kinds of government organizations have put their 2 cents in too. What are they checking identity against? Drivers licenses, SS cards, voter ID, green cards? That means they could be hitting 151+ more databases run by other organizations. Does DHS get the information? Is it fed back to them when a users accesses? Are the checked against law enforcement databases? Only those directly involved in the development will know. You can disregard anything in the privacy statements. You're not going to see a friendly note in the FAQ "If you're a wanted felon, information will be transmitted to the law enforcement organization looking for you." That kind of defeats the purpose.

    Depending on load testing never replicates what real users will do. Real users do weird things, just because they can. No amount of planning and testing will give you everything. There is always a lot of reactive work to be done. Shit, everyone reads the FAQ 14 times before logging in? They 20% of the people go through the login screens, back out to the 2nd page, and try again?

    I'm stuck on the same non-functional healthcare.gov site as everyone else is. I signed up. I never got an email confirmation or email address verification.

    My girlfriend got the verification and signed up again. I was able to present my user:pass and it did seem to say it was valid, but stayed there until I was thrown the overloaded message. Later, it said my user:pass was invalid. Is it really invalid?

    I tried to do the username and password recovery. Neither sent me anything, so I assumed my account wasn't made. When signing up again, it said my combination of email, username, and real name was not unique. Ok, so I'm at least partly there.

    I signed up again with a different username. This time I received the email verification, and clicking it did say I was confirmed to be a user. I still can't get in. It says my user:pass is wrong. Is there something really wrong, or is it still totally broken? I don't know.

    All I do know is, their admins are probably still going nuts trying to react to the volume of users. Associate database admins are going nuts with the interactions that may or may not be happening. They have my sympathy. This is probably day 4 of not getting any sleep.

    The better approach would have been if they let people enroll online over the last few months. Traffic would have ramped up quickly, but it would have been less than 10M people on the first day. 10M is the number I read somewhere that they got on the first day of operation. Allowing a slower ramp-up, even if the authenticated area only showed some pretty stats on the number of carriers, doctors, registered users, would have been something for them to prepare themselves with. Now is too late to hope for a graceful start.

    I'm not going to bother with it much right now.. They'll get it worked out. I won't even bother calling the support number (800-813-2596) to see if there's a problem. They're already getting hammered. As long as I can hope to have insurance on the Jan 1, I'll be happy. I was laid off a couple weeks ago. At best, if i was hired Monday morning, the soonest I'd see employer sponsored healthcare is Jan 7. Knowing the amount I've had to pay in the past, if the estimated amounts talked about are accurate, I'll be saving a fortune, and actually get something for it. In the past, I used to not pay for insurance, because it was cheaper for me to pay for care at the cash price.

Doctors and hospitals have been happy to raise prices so much that it's impossible without having an awful lot of disposable cash. I was in an accident about a year ago. Because I *might* take narcotic pain killers in the future, they had me take a drug test. The test was $1200. Insurance covered $600. I could get a plane ticket, fly to San Diego, go down to Tijuana and buy up a year's supply for less than just the drug test. I also don't want to be labeled as an international drug smuggler, just because American medical prices are outrageous. Felony, or pain? It's a really tough decision.

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?