Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

London Stock Exchange Price Errors 'Emerged At Linux Launch'

timothy posted more than 3 years ago | from the hiccough-hiccough-hiccough dept.

Businesses 168

DMandPenfold writes "Within the first 20 seconds of the London Stock Exchange's new matching engine going live on Monday, price data vendors began displaying incorrect prices, blank prices and wrong trading volumes, according to Computerworld UK sources. Thomson Reuters, Interactive Data and Netbuilder are among the largest data vendors, providing share prices to traders, that have been displaying pricing problems on some stocks throughout the week. Even the LSE's own data vendor, ProQuote, experienced problems. Concerns are being raised that there could be mistakenly setup connections or incorrect software interfaces at some of the large data vendors. Alternatively, there may be a data caching issue at the LSE that means data going out is not properly synchronised between different systems."

cancel ×

168 comments

Sorry! There are no comments related to the filter you selected.

Testing? (2)

Tubal-Cain (1289912) | more than 3 years ago | (#35252370)

Within the first 20 seconds of the London Stock Exchange's new matching engine going live on Monday, price data vendors began displaying incorrect prices, blank prices and wrong trading volumes, according to Computerworld UK sources.

Surely they would have run test data before letting it go live. Maybe even feed it the actual data and simply not publish its results.

Re:Testing? (5, Informative)

PNutts (199112) | more than 3 years ago | (#35252394)

Surely they would have run test data before letting it go live. Maybe even feed it the actual data and simply not publish its results.

From TFA: "Meanwhile, the private view at the exchange is that the 15 month testing window should have been plenty of time for the vendors to interface perfectly with it."

Re:Testing? (5, Informative)

cronius (813431) | more than 3 years ago | (#35252528)

From this article a couple of days earlier http://www.computerworlduk.com/news/it-business/3261625/frustrations-mount-over-london-stock-exchange-data-interface-problems/ [computerworlduk.com] :

Publicly, the LSE and the data vendors say they are working together as engineers scramble to fix the price data problems, even though some statements to clients suggest elements of blame.

But behind the scenes, sources close to the several of the parties, including the exchange, told Computerworld UK they were immensely frustrated at the reputational impact of the problems.

The LSE is taking the position that its data feeds are working correctly. Industry sources said the exchange had placed a great deal of emphasis on the launch, which was largely providing successful high-speed trading, and that it had allocated sufficient time - 15 months - for the vendors to be fully prepared for the new system.

The LSE statement appeared to place some blame with the vendors. "Unfortunately a couple of market data vendors have experienced some specific issues aligning to the new Millennium Exchange platform and we are actively working with them to help resolve their issues," it said.

"All other trading customers and vendors are successfully trading on the new platform, benefiting from Millennium Exchange's superior functionality and speed."

Emphasize mine. Looks to me like the exchange is getting the flack for a couple of amateurish (or saboteurish?) vendors.

Re:Testing? (0)

Yvanhoe (564877) | more than 3 years ago | (#35252922)

Emphasize mine. Looks to me like the exchange is getting the flack for a couple of amateurish (or saboteurish?) vendors.

So you mean they asked their vendors to change a communication protocol and never tested before day 1 ? Honestly, that not how you test such a development. Sure, one can blame vendors for not correctly implementing a spec, but this is regular, expectable error in the software world. The real mistake is the lack of a test with the various sources.

Re:Testing? (1)

$pace6host (865145) | more than 3 years ago | (#35253292)

Emphasize mine. Looks to me like the exchange is getting the flack for a couple of amateurish (or saboteurish?) vendors.

So you mean they asked their vendors to change a communication protocol and never tested before day 1 ? Honestly, that not how you test such a development. Sure, one can blame vendors for not correctly implementing a spec, but this is regular, expectable error in the software world. The real mistake is the lack of a test with the various sources.

Huh? Did you skip the earlier sentence?

The LSE is taking the position that its data feeds are working correctly. Industry sources said the exchange had placed a great deal of emphasis on the launch, which was largely providing successful high-speed trading, and that it had allocated sufficient time - 15 months - for the vendors to be fully prepared for the new system.

I think he means it was available for test for 15 months. I don't think the LSE can force the vendors to use the new protocol, but giving them 15 months to do so seems fairly reasonable. On the other hand, it could be that the quantity of testing was fine, but the quality (as in how accurate with respect to important things like scaling) was poor. You can test for years, but if your testbed doesn't look anything like the real environment, what are you really testing?

Re:Testing? (0)

Anonymous Coward | more than 3 years ago | (#35253994)

Emphasize mine. Looks to me like the exchange is getting the flack for a couple of amateurish (or saboteurish?) vendors.

So you mean they asked their vendors to change a communication protocol and never tested before day 1 ? Honestly, that not how you test such a development. Sure, one can blame vendors for not correctly implementing a spec, but this is regular, expectable error in the software world. The real mistake is the lack of a test with the various sources.

Come on, it was a well-established Microsoft version of an industry standard, so what could go wrong?

Re:Testing? (4, Insightful)

CaymanIslandCarpedie (868408) | more than 3 years ago | (#35253000)

I think you can take off the tinfoil hat on the sabotage thoughts. For one LSE's own quote providing solution is experiencing the same issue. So unless they have some diabolical plan to sabotage themselves, I think that fact should rule that out. Also, this is effecting large players. These are billion dollar companies whose sole propose is to provide accurate data and provide it fast. Without that they simply don't exist. You'd have to live in a very conspiracy theory driven existence to think they'd all really all throw their own companies and livelihoods away in some dark plan to get the LSE or Linux.

Now would all these large players all have made the exact same errors in their interfaces so they are seeing these same issues while the smaller players seem to have got it right? Sure. But the simplest explanation, of course is they are all experiencing the same issue because of some upstream problem with scaling to their large volumes. Those actually involved seem to point to some caching issue. Is that true? No clue, but they are certainly in a better position to offer analysis of the issue than you or I. That doesn't mean Linux is bad for christ's sake (so you don't have to go on some crazy conspiracy hunt to explain it away). If it is true (and we certainly don't know that it is, though the information currently available may point that way), you know what? There is software which just happens to be running on Linux, which may have an issue. No big deal. Happens every day on every platform. Not some big black eye on Linux if some software that happens to run on Linux has issues and no need to start jumping to conspiracies to explain it away.

Re:Testing? (1)

smallfries (601545) | more than 3 years ago | (#35253402)

Also, this is effecting large players

Really? I hadn't heard about that aspect of the story. So previously there were only small players, but the effect of this bug is causing large players to be created. Gosh, I hope they are not also affected by the problem.

Re:Testing? (0)

Anonymous Coward | more than 3 years ago | (#35253906)

Arrr, go to it, ye grammarr nazzi!!! My affection for your effections are never affected!!

Should have kept Windows. (-1)

Anonymous Coward | more than 3 years ago | (#35252636)

Go ahead, downvote, I know you will.

Re:Should have kept Windows. (1, Insightful)

Ferzerp (83619) | more than 3 years ago | (#35253674)

If this has been Windows story, there would be much frothing at the mouth and blame on Microsoft. Since it is Linux, the blame magically lies with the implementation.

Hyporcrite much, Slashdot?

Re:Testing? (0)

Anonymous Coward | more than 3 years ago | (#35252688)

They seem to have forgotten that vendors are fucking idiots.

Re:Testing? (2, Insightful)

Anonymous Coward | more than 3 years ago | (#35252840)

First off, you can have 15 YEARS To test, if you are understaffed and underskilled you will discover jack squat.

Second, THIS news comes out three days after Slashdot wrote what amounts to a slam piece on .Net's failures to handle this business. Hmm, what happened here? These problems are noted in the title of the article itself as emerging after Linux/C++ solutions were brought to bear.

Lets face facts, the core of EVERY issue (the failures in the .Net system that led to its replacement, the failures we're witnessing now, the failure at switch, the failure to test properly, notice the trend?) isn't the language/platform in question but the general incompetence with which the issue is being handled.

Re:Testing? (0)

Cronock (1709244) | more than 3 years ago | (#35252404)

At very least you'd expect a large scale dry-run with simulated data that should bring these errors out before becoming live.

Re:Testing? (2)

Davorama (11731) | more than 3 years ago | (#35252484)

Indeed, but please stop calling me Shirly.

If only they had thought of that (4, Funny)

raftpeople (844215) | more than 3 years ago | (#35252488)

Surely they would have run test data before letting it go live. Maybe even feed it the actual data and simply not publish its results.

Quote from the project manager:
"Dang. If only we had read Tubal-Cain's post before going live. Who would have thought to run test data through this darn thing?"

Re:Testing? (3, Funny)

Anonymous Coward | more than 3 years ago | (#35252532)

When Apple released a phone they hadn't tested and it turned out "holding it a certain way" prevented calls, it suddenly changed to "holding it wrong", thus blaming the user. No scope for that here?

Re:Testing? (1)

johnsnails (1715452) | more than 3 years ago | (#35253186)

why isint this modded funny? I found it funny... or at least informative.

Linux fails... AGAIN (-1, Flamebait)

Anonymous Coward | more than 3 years ago | (#35252390)

Once again we witness how Linux destroys companies, One after another (Novell, SCO, Nokia, etc) each of them adopted Linux and then fell victim to the unreliability, unmarketability and anti-business attitude of the linux zealots.

Who's next?

Re:Linux fails... AGAIN (2)

Nuno Sa (1095047) | more than 3 years ago | (#35252398)

Google, Amazon, IBM, Redhat, ... Take your pick :-)

Re:Linux fails... AGAIN (5, Informative)

TooMuchToDo (882796) | more than 3 years ago | (#35252406)

Don't forget the US DoD, The US Navy Submarine Fleet, the FAA, and Cisco:

http://www.focus.com/fyi/information-technology/50-places-linux-running-you-might-not-expect/ [focus.com]

I don't see IBM running Watson on Windows 2008 Server because, well, you know....they want it to work.

Re:Linux fails... AGAIN (1, Interesting)

Nuno Sa (1095047) | more than 3 years ago | (#35252448)

Btw, they are _upgrading_ to Linux, because the previous system (Windows and .net) failed several times, required MW of energy and was slow.

This new _upgraded_ system is many orders of magnitude faster.

Unfortunately, it appears, it wasn't properly tested... WTH? I'd expect for the system to run only 100x faster, instead of 1000x faster, during the first few weeks. Or maybe that some API from 1990 stopped working. Wrong data is just too much. Maybe the story isn't telling all the facts? ...Hmmm... Maybe the affected clients are running Windows?!!!!!!

Re:Linux fails... AGAIN (5, Interesting)

MichaelSmith (789609) | more than 3 years ago | (#35252486)

I work on large scale air traffic control systems which run Linux and I don't envy the LSE in their task. Most of our interfaces are relatively simple and go out to organisations with a good history of validating interfaces. This trading system seems to have to interface to a lot of little offices around the place running various implementations. Its no surprise some of the interfaces weren't tested to the point where they are known to work 100%, though they may be 100% correct.

Re:Linux fails... AGAIN (1)

Hylandr (813770) | more than 3 years ago | (#35252982)

Sure wish I had mod points, this man knows how to think.

- Dan.

Re:Linux fails... AGAIN (1)

Smallpond (221300) | more than 3 years ago | (#35253934)

I work on large scale air traffic control systems which run Linux and I don't envy the LSE in their task. Most of our interfaces are relatively simple and go out to organisations with a good history of validating interfaces. This trading system seems to have to interface to a lot of little offices around the place running various implementations. Its no surprise some of the interfaces weren't tested to the point where they are known to work 100%, though they may be 100% correct.

According to the article, the little organizations are running fine -- it's some of the big company systems that are having problems. My guess would be that their systems are so large and complex that they haven't been able to do sufficient testing, and that they weren't able to make the changes that they needed to make because the systems are all in use 24 x 7.

Its like the story of why a heart surgeon is paid more than a car mechanic. Try fixing a car engine while its running.

Re:Linux fails... AGAIN (-1)

Anonymous Coward | more than 3 years ago | (#35252612)

Right, but did those systems fail because of Windows itself? No, you stupid fucking cocksucker, and you don’t have any hard data to prove the root cause. You’re just another troll squirting out your favor of bullshit and calling it fact.

Re:Linux fails... AGAIN (-1, Flamebait)

Nuno Sa (1095047) | more than 3 years ago | (#35252776)

Those *are* facts. The new system is better in every metric. The old system failed several times. Once it crashed and stayed down for the better part of the day.

The cause of the error in unknown... As always.

Anyway, there's no need for the "root cause". It failed. It was slow. Somebody had to eat a frog and scrap the whole system. And in the 90s they said nobody ever got fired for buying MS. Good old days. Too bad for some it's 2011!!

And you don't have to be bitter, MS' stock may hold a few months more with the Nokia buyout. :-)

Re:Linux fails... AGAIN (1)

Desler (1608317) | more than 3 years ago | (#35253568)

Those *are* facts. The new system is better in every metric. The old system failed several times. Once it crashed and stayed down for the better part of the day.

And yet you cite not a single source with your supposed metrics or stories showing it failed "several times". There is a single downtime that one can find any mention of. Care to provide actually evidence of your claims now instead of just asserting they are facts?

Re:Linux fails... AGAIN (3, Funny)

Anonymous Coward | more than 3 years ago | (#35252422)

The amusing part of this troll is that it would be modded up to 5, instantly, if it were attacking Windows. The day when I see a pro-linux comment modded "troll" is the day that I take anything posted on Slashdot seriously. Until then, it's just self-important, self-congratulatory, mindless groupthink.

Re:Linux fails... AGAIN (-1)

Anonymous Coward | more than 3 years ago | (#35252516)

Yeah the LinuxFags will mod us down but nobody ever got fired for buying Microsoft. What are you going to do in this case, ring up RMS and have him come in without shoes and have his Tinea-ridden toes treading all over your office space? At least Ballmer can afford shoes and a tie.

Re:Linux fails... AGAIN (0)

Anonymous Coward | more than 3 years ago | (#35252752)

albeit, shoes and a tie soaked with sweat as he leaps about like a trained monkey desperately trying to feign relevance.

Re:Linux fails... AGAIN (0)

Anonymous Coward | more than 3 years ago | (#35252548)

I wouldn't go that far, but I would like any Linux fanboys out there to remember this the next time they read a story about a Microsoft based system having problems. No OS is flawless.

Re:Linux fails... AGAIN (2)

perryizgr8 (1370173) | more than 3 years ago | (#35252552)

using a solid os won't protect you from human stupidity. i thought that was a given. perhaps letting microsoft setup everything would be better for stupid people like lse. more expensive but atleast no stupid errors.
meanwhile, intelligent people continue using linux to their advantage.

Off (3, Funny)

Anonymous Coward | more than 3 years ago | (#35252402)

Turn it off and on again.

Re:Off (2)

93 Escort Wagon (326346) | more than 3 years ago | (#35252742)

Turn it off and on again.

Jiggle the cord!

Re:Off (1)

mindwhip (894744) | more than 3 years ago | (#35253222)

defrag the disks!

Re:Off (0)

Anonymous Coward | more than 3 years ago | (#35253980)

As a last resort they called MS support for help, who suggested they reload the OS from their last restore point.

either sympathy or accusation (5, Insightful)

DreadPirateShawn (1246208) | more than 3 years ago | (#35252414)

My heart goes out to the devs "working long hours and night shifts" to suss this out.

That being said, the line that catches my eye most is: "The fact the majority of smaller vendors were fine demonstrated that those having trouble had made mistakes." In my experience, that means one of two things:

1) The devs configuring the system didn't properly account for the sheer scale of the stress on their systems.
2) The smaller vendors took the change more seriously, and being smaller and more flexible, successfully updated their systems to interact properly with the new systems.

Or, of course, both.

Re:either sympathy or accusation (4, Interesting)

micheas (231635) | more than 3 years ago | (#35252464)

Or the big vendors were doing excessive cost cutting to maintain profits during the recession.

As long as this is just a one weekend nightmare, the execs that ordered the QA not done are looking like great business monkeys. they took a risk that cut costs and managed to defect the blame to the exchange when it blew up, and it looks like nothing bad will happen to the big vendors, unless one of the small vendors gets a really unethical sales monkey that steals a large chunk of business from a couple of the big vendors.

Re:either sympathy or accusation (0)

Anonymous Coward | more than 3 years ago | (#35252472)

Crazy. When this was a .NET based app it was the OS and the platform choice and all Microsoft's fault for sucking, but when it's Linux it's automatically (insert excuse here). Don't get me wrong - I don't blame Linux at all, but nor was I partisan fool enough to blame Windows or .NET for a poor implementation of a trading tool.

Re:either sympathy or accusation (3, Interesting)

jmorris42 (1458) | more than 3 years ago | (#35252500)

These are teething issues. And it apparently worked for the smaller traders/vendors so it looks more like a problem at the trader's end in that they didn't get their crap straight. But when changing out a large system like that it isn't that incredible there were problems not caught in testing.

> I don't blame Linux at all, but nor was I partisan fool enough to blame Windows or .NET for a poor implementation of a trading tool.

Of course Linux isn't to blame, unless they turn up kernel or other low level library problems, that system is a mass of spanking new code running atop linux. But we already know Linux can handle the transactional load reliability requirements and real time needs of a stock exchange since several are already doing it. The question is whether the guys working for the London Exchange have good enough code-fu.

But apparently they blamed the performance of .NET/Windows or the cost benefit of .NET/Windows otherwise they wouldn't have embarked on an expensive rip and replace operation in the first place. Personally I'm shocked .NET ever managed such a task in the first place, even poorly.

Re:either sympathy or accusation (0)

Anonymous Coward | more than 3 years ago | (#35252564)

I happen to know that the TR system is .NET/C++, so their problems might be related to known issues of TCP/IP message passing from Unix/BSD stack to a Win stack. Let's just say you can get the most mysterious error messages in .NET. Of this, I have first hand experience in my own projects.

Re:either sympathy or accusation (2)

jgrahn (181062) | more than 3 years ago | (#35252624)

I happen to know that the TR system is .NET/C++, so their problems might be related to known issues of TCP/IP message passing from Unix/BSD stack to a Win stack. Let's just say you can get the most mysterious error messages in .NET. Of this, I have first hand experience in my own projects.

I really doubt the Linux IP stack (which isn't BSD-based) has problems talking to the Windows one. And even if it did, it couldn't have nothing to do with .NET (unless .NET comes with its own IP implementation).

What *are* these known issues, according to you?

Re:either sympathy or accusation (3, Insightful)

IainCartwright (733397) | more than 3 years ago | (#35252750)

I can well believe you have had problems in your projects if you think that there are "known issues of TCP/IP message passing from Unix/BSD stack to a Win stack".

Re:either sympathy or accusation (1)

micheas (231635) | more than 3 years ago | (#35252526)

It is not at all clear that there was/is a problem with the new system.

What is clear is that if you unfortunate enough to have signed up to 500 quid per workstation to Thomson Reuters per month you got garbage quotes from Thomson Reuters.

Personally if I was one of the people that received garbage quotes / no quotes at those price levels I would be looking for a new vendor and talking to a solicitor.

Re:either sympathy or accusation (1)

Anonymous Coward | more than 3 years ago | (#35253506)

Unethical? Why couldn't they steal lots of business from the big guys by showing their customers that they handled the transition properly while the big guy botched it? Why would unethical sales be needed to steal business from an apparently incompetent incumbent?

Re:either sympathy or accusation (1)

fatp (1171151) | more than 3 years ago | (#35252578)

Maybe the smaller vendors have fewer customer so there is smaller chance to detect the problem.

Also, it said "the majority of smaller vendors", not all smaller vendors were fine.

Re:either sympathy or accusation (5, Interesting)

Anonymous Coward | more than 3 years ago | (#35253334)

As I work for one of the vendors cited in the article, I can assure you that the reason is freaking simple: the systems that consume the feeds to move them into the trader's desktop are absolute mess of epic proportions riddled with mind boggling wtfs. Missing data, blank values, wrong numbers are common on those systems (root cause are mainly race conditions everywhere, and assumptions about missing data), assembled from random left and right acquisitions. There is no way to really know how they will behave under stress, and corporate culture will not allow managers to actually fix the issues (extremely costly, and corporate bonuses are not aligned to quality).

In fact, it is much cheaper for everyone to just pretend this is a one time occurrence, to fire a couple of middle managers and continue frantically fixing bugs in the mess until it sorta works.

Re:either sympathy or accusation (0)

Anonymous Coward | more than 3 years ago | (#35254014)

Indeed. That's a typical culture within the industry as a whole. It was fun getting the data right on all 8 of the markets one of my previous employers provided. LSE was one of them (gotta wonder if they are one of the ones having problems or not...) and it's not easy trying to ensure accuracy- especially since at least one of the feeds could literally choke an OC-3 at peak trading volumes.

Lots of money is at stake with this thing and the upper management doesn't want to typically own up to it being their screwup.

Re:either sympathy or accusation (1)

Evildonald (983517) | more than 3 years ago | (#35253420)

Those poor devs, who are earning 500-800 pounds a day. My heart goes out to them too.

so... (0)

Anonymous Coward | more than 3 years ago | (#35252416)

Did they reboot it 3 times? I've heard that works.

Oh wait this isn't windows 2008.

Re:so... (1)

darth dickinson (169021) | more than 3 years ago | (#35252508)

No they had to recompile the kernel with the --WORK_NOW_DAMN_YOU flag enabled, but not until after they had downloaded the latest kernel source and repackaged it for their current distribution-of-the-month. After working out all the dependencies, and *their* dependencies, of course.

Re:so... (1)

cynyr (703126) | more than 3 years ago | (#35253794)

you repackage it?!!!??

you just extract it(tbh anywhere you like, but convention says /usr/src/linux-$VERSION-$PATCH/) and then "cp ../$OLDKERNEL/.config ./ ; make oldconfig; make ; make modules_install; mount /boot; cp $ARCH/boot/bzImage /boot; $EDIT_BOOTLOADER_CONFIG; shutdown -r now"

On most binary distros you need some sort of dev tools, that changes based on distro, but should be simple.

P.S. Has Ubuntu fixed the deps on the kernel sources to include the build-utils yet? cause it's not like all i want to do when i install the sources is look at them.

Odd.. (-1)

Anonymous Coward | more than 3 years ago | (#35252444)

I expected to come in hear reading how this was proof Linux sucks and how it's not ready for mission critical apps. I was just _sure_ that would be the tenor of the comments, as that was the tenor previously when a different system was used and had issues. Odd, I guess I just guessed wrong.

Hmmm.. So .Net was bad ? Bot a good start on Linux (0)

Anonymous Coward | more than 3 years ago | (#35252514)

Hmmm.. So .Net was bad ? Bot a good start on Linux

Binux? (0)

raftpeople (844215) | more than 3 years ago | (#35252542)

botNet wab bab? Bot a bood bart on Binux

Re:Binux? (0)

Anonymous Coward | more than 3 years ago | (#35253212)

That was one of the funniest things I've read on Slashdot, yet I have no idea why it made me laugh so hard.

Anyone actually surprised? (0)

Anonymous Coward | more than 3 years ago | (#35252518)

You get what you pay for with Linux.

If they had gone with MS for the upgrade it would have been bulletproof, as they have a reputation to keep up.

Re:Anyone actually surprised? (0)

Anonymous Coward | more than 3 years ago | (#35252570)

from TFA:

The new system replaced Microsoft .Net-based trading software that was written in C# by Accenture, and ran on Windows Server and SQL Server. That system experienced a devastating eight hour outage in 2007 during heavy trading. The problem was a major factor in the system being scrapped, alongside its ongoing two millisecond latency. The exact cause of the outage was never disclosed.

Plenty of vendors aren't having problems w/ the new system.. wonder why.

Quote Services (5, Informative)

Anonymous Coward | more than 3 years ago | (#35252536)

Article is a a bit misleading....
Reuters, Netbuilder et al are quote services that run externally and fetch quotes (price/volume) from the LSE and are *not* part of what would be considered the main trading platform. The problem isnt the actual OS (Linux) or tradiing platform, which has been heralded as a big step forward from the MS solution, but rather how the resulting trade data that is pushed from the exchange to the 3rd party sites and then ultimately to the end users.

Re:Quote Services (2, Informative)

Anonymous Coward | more than 3 years ago | (#35252562)

Article is a a bit misleading....
Reuters, Netbuilder et al are quote services that run externally and fetch quotes (price/volume) from the LSE and are *not* part of what would be considered the main trading platform. The problem isnt the actual OS (Linux) or tradiing platform, which has been heralded as a big step forward from the MS solution, but rather how the resulting trade data that is pushed from the exchange to the 3rd party sites and then ultimately to the end users.

BTW... these protocols while well documented are complicted. To get an idea, check out either the NYSE ArcaEX Direct API [nyse.com] or the NYSE Arca FIX specs [nyse.com]

If someone could point out the LSE equivalent, I'd be interested in seeing the differences

Re:Quote Services (5, Interesting)

Anonymous Coward | more than 3 years ago | (#35252740)

Hi,

The NYSE Arca FIX docs are not quite relevant to this - as what we're talking about here is the market data API.

The FIX protocol (although it does support market data transactions), is typically not the protocol of choice for market data. Typically it'll be non standard provider specific protocols which are optimised for delivering partial updates of information in a very bandwidth efficient manner.

In the parent post someone mentions that Reuters, Netbuilder etc fetch quotes - which is also not entirely correct either. They accept push feeds from the exchange, which contain tick updates to the information about the instruments traded on that exchange. The generation of these ticks is triggered when a match occurs on the order book - which may (or may not) involve a change in price data. They then typically augment this information with other data - such as other identifiers, or derived information such as previous days close value / percentage etc.

It is with these derived fields that the ongoing issues seem to lie, although for a certain period of time, the exchange was actually pushing out zero values for the bid/offer fields. As you can imagine, these plays merry hell with systems that attempt to drive automatic execution based on current price.

I am informed by Netbuilder that they will have pushed a final fix to live within a subset of their servers - but it will probably be Tuesday before this is available across their whole system. This is over a week since the original identification of the issue, not good!

And I thought my bug was bad (2)

JazzXP (770338) | more than 3 years ago | (#35252560)

I thought I'd stuffed up big time when I hooked all of our users (around 1000 users at the time) up to test market prices rather than live data (Australian Stock Exchange) over market opening one morning (our product is aimed at day traders too). My stuff up seems minor in comparison to this.

What is your biggest mistake? Monetarily... (1, Offtopic)

Colin Smith (2679) | more than 3 years ago | (#35252834)

I of course don't make mistakes, but what do you think would happen if for example you wiped 8.5 billion dollars off of the value of your company?

http://finance.yahoo.com/q/bc?s=NOK+Basic+Chart&t=3m [yahoo.com]

Do we have a world record here Mr. Elop?
 

Re:What is your biggest mistake? Monetarily... (2)

WalkingBear (555474) | more than 3 years ago | (#35252944)

The drop in stock price after the announcement of the Nokia Microsoft venture was sharp for a single day. It's also one of the highest volume days in the last 5 years.

However, the drop is insignificant compared to the drop from late 2007 to now. Market Cap at Nokia is 1/4 what it was. Stock prices have dropped from mid 40's / share to ~ 10/share.

Besides, the 8.5B drop in market cap isn't nearly as important to the bottom line as the billions in cash coming in from Microsoft as part of this deal.

Data (1)

gravel junkie (1987790) | more than 3 years ago | (#35252582)

Data is always seems to be the least respected item in the IT stack (TBV). Yet its the lifeblood. Referential integrity, blah. At your peril.

The offending code (1, Funny)

billcopc (196330) | more than 3 years ago | (#35252584)

The offending code was traced back to :

        $stockprice = power( 2, rand()*10 + 0.0001 );

Data vendors quickly addressed the issue by cutting the LSE out entirely, and rolling their own rand() function.

Fucking stock markets... .what a joke!

Why Am I Not Surprised (1)

oakwine (1709682) | more than 3 years ago | (#35252652)

On the other hand, a moon mission interfaced a host of complex systems including the lives of the astronauts. Difference is the level of integration of QA into the project.

If it was a Windows implementation... (0)

Anonymous Coward | more than 3 years ago | (#35252692)

...We'd have heard a million nerds orgasming on Slashdot, reminding us how much Windows sucks.

Re:If it was a Windows implementation... (0)

Anonymous Coward | more than 3 years ago | (#35252766)

...We'd have heard a million nerds orgasming on Slashdot, reminding us how much Windows sucks.

They did last year. 8 hour outage on the master trading system rather than just the quotes one, oh dear...

Re:If it was a Windows implementation... (1)

t2t10 (1909766) | more than 3 years ago | (#35252960)

Let me clear that up for you: system crashes, outages, BSODs -- fault of the operating system; bad data, bad XML, bad UI -- fault of the application programmers and application platform.

Most likely, this was Java programmers screwing up.

Effect on stockbrokers (5, Informative)

Anonymous Coward | more than 3 years ago | (#35252716)

As someone on the receiving end of this problem (as head of Development for one of the largest UK stockbrokers), it has been immensly frustrating trying to get an admission of blame from anyone involved in this issue. Our clients are complaining to use because elements of the data we're displaying on our site are just so out of whack they are laughable. In the end we've just had to remove those elements of data from our website, and they remain removed until the fix is confirmed working on Monday. This is after several attempts to fix the issues during the week.

It's also causing serious issues for things like our black box trading engine which is used to drive automatic execution of client orders when the price moves into range for execution. The data being bad means these systems just have to be turned off, and clients are being advised to simply delete their orders.

This is, frankly, a pretty appalling situation as the service to our clients is being impacted pretty severely.

For an exchange that is attempting to position itself as the true 'global' exchange, this is doing an immense amount of damage to it's reputation.

Re:Effect on stockbrokers (0)

Anonymous Coward | more than 3 years ago | (#35252760)

As someone on the receiving end of this problem (as head of Development for one of the largest UK stockbrokers), it has been immensly frustrating trying to get an admission of blame from anyone involved in this issue.

What is more important, an admission of blame or a fix? Ultimately you won't get either until the cause is understood..

Our clients are complaining to use because elements of the data we're displaying on our site are just so out of whack they are laughable. In the end we've just had to remove those elements of data from our website, and they remain removed until the fix is confirmed working on Monday. This is after several attempts to fix the issues during the week.

But what are you doing differently than the (as I understand, majority of) people who are not experiencing such issues?

Also, as Head of Development of a house experiencing such issues, wtf are you doing on slashdot?

Re:Effect on stockbrokers (0)

Anonymous Coward | more than 3 years ago | (#35253734)

Also, as Head of Development of a house experiencing such issues, wtf are you doing on slashdot?

As Head of Development, he has plenty of underpaid developers doing the real work.

Re:Effect on stockbrokers (postmortem) (3, Interesting)

realxmp (518717) | more than 3 years ago | (#35252794)

As with most major issues there's bound to be a big ol' postmortem on this. As head of Dev you've probably got a unique insight into this, I'm curious as to your perspective on this, what you think the cause of failure might be? More strategic or more technical? Poor interface specification? Inability to handle queries under full load? From TFA there was supposedly 15 months of testing. So I'm curious as to why it failed, whether the testing simply wasn't realistic and/or thorough enough, or it was something that just wouldn't come up except on the live system?

Re:Effect on stockbrokers (postmortem) (0)

Anonymous Coward | more than 3 years ago | (#35253324)

From TFA there was supposedly 15 months of testing. So I'm curious as to why it failed, whether the testing simply wasn't realistic and/or thorough enough, or it was something that just wouldn't come up except on the live system?

Well, he had 15 months to test it, but spent 14 of them on /. instead of actually testing the push feed.

the real stupidity shown here (0)

sxpert (139117) | more than 3 years ago | (#35252746)

is the concept of "high speed trading",
This horsecrap has to stop, the stock market has to be dismantled

Re:the real stupidity shown here (0)

Anonymous Coward | more than 3 years ago | (#35253076)

> This horsecrap has to stop

If you can't bring yourself to say "bullshit" then don't fuck around with pseudo-swearing.

LOL (0, Interesting)

Anonymous Coward | more than 3 years ago | (#35252786)

I am curious which part of the Linux (kernel) they'll be going to blame and how Slashdot will turn it into FUD with adverts around it... xD

I mean (1)

Dunbal (464142) | more than 3 years ago | (#35252788)

It's only a stock exchange. Why the fuck should they bother testing it first, right? This just goes to underline that QA does not exist in the software world.

Re:I mean (1)

rrossman2 (844318) | more than 3 years ago | (#35253338)

They did. They had 15 months of/to test. It's right in the article

Re:I mean (1)

Dunbal (464142) | more than 3 years ago | (#35253806)

We have a different definition of testing. Testing != compile it and see if it runs. Testing != hiring some minimum wage loser to tell you exactly what you want to hear.

Re:I mean (1)

Zoxed (676559) | more than 3 years ago | (#35253484)

> This just goes to underline that QA does not exist in the software world.

My 24 years in the software industry tell me that QA is available: the customer just has to be prepared to pay for it !

Re:I mean (1)

jo_ham (604554) | more than 3 years ago | (#35253676)

It's only a news discussion site. Why the fuck should they bother reading the article before commenting on it? This just goes to underline that reading comprehension is non existent in the slashdot world, yet still they believe they have all the facts and can pass judgement.

what does that have to do with Linux? (3, Insightful)

t2t10 (1909766) | more than 3 years ago | (#35252940)

If it didn't crash and didn't drop its network connections, Linux was doing its job.

If the application software had bugs, then the application software developers are to blame.

Re:what does that have to do with Linux? (2, Insightful)

$pace6host (865145) | more than 3 years ago | (#35253354)

If it didn't crash and didn't drop its network connections, Linux was doing its job.

If the application software had bugs, then the application software developers are to blame.

Even if the OS itself isn't at fault, don't think that someone with an axe to grind won't blame it. You'll just hear something like "It's Too Hard to develop on Linux. A platform with more / better tools would have made it easier to develop and comprehensively test." I don't buy that either (regardless of what price you quote me! ;) ), but I'm sure someone will be saying it. Especially someone in marketing.

Re:what does that have to do with Linux? (0)

Anonymous Coward | more than 3 years ago | (#35253878)

...don't think that someone with an axe to grind won't blame it.

You mean like the Linux zealots and FOSS advocates did with the Windows based platform?

Nothing to do with Linux (2)

Wowsers (1151731) | more than 3 years ago | (#35252970)

An attempt to rubbish Linux.

After 20 seconds they should have realised that they should have tested more, and fired the programmers for allowing such mistakes. This is nothing to do with Linux.

You'd think the LSE would have learnt from their last computer system rollout which also had massive problems. No quality control = management problem.

Re:Nothing to do with Linux (0)

ScentCone (795499) | more than 3 years ago | (#35253456)

This is nothing to do with Linux.

Yes, but can you imagine if the new system had a part of the MS stack running it? Slashdot would be full of people casually blaming that, instead of the app/dev side of things. You know it's true.

Re:Nothing to do with Linux (1)

cynyr (703126) | more than 3 years ago | (#35253820)

Well, is it was caused by bugs in .Net's XML tools? sure MS's fault. Remember that linux is just a kernel, and the rest of the GNU software may or may not be running on this beast and most of the important software(the push server thingy) is likely to be custom in house stuff.

Re:Nothing to do with Linux (1)

JonySuede (1908576) | more than 3 years ago | (#35253788)

The lack of testing, in a corporate setting, is not the programmers fault, it is a management failure and as such the managers should be fired.

SOUNDS ABOUT RIGHT !! UNQUALIFIED CODERS !! (-1)

Anonymous Coward | more than 3 years ago | (#35253034)

You just can't find qualified programmers for Linux. Sure, you can find many out of work in russia but that's a worse plan. The question then is, how much will it cost to re-train real programmers to work on its "free" operating system?

Look closely at the vendors having issues. (-1)

Anonymous Coward | more than 3 years ago | (#35253068)

I bet many of them are 100% windows shops with a significant investment in Microsoft.

Would it really be so far fetched that MS supporters would try to embarrass a massive publicized Linux rollout?

Re:Look closely at the vendors having issues. (0)

Anonymous Coward | more than 3 years ago | (#35253350)

Windows is not ready till it breaks the LSE. Tim S. PS: Already rated a post in this thread so posting anom.

Re:Look closely at the vendors having issues. (1)

Jasonjk74 (1104789) | more than 3 years ago | (#35253414)

Aaaaannnd.... there it is! I was about to write a post saying that I wonder how someone would try to turn this into an apropos-of-nothing, Microsoft-bashing event.

Re:Look closely at the vendors having issues. (1)

$pace6host (865145) | more than 3 years ago | (#35253418)

Well, eventually the root cause(s) will be found - and if the faults lies within their own systems, who will be embarrassed? It would seem to be an extremely poor strategy in the long term to do something like that. I suspect that it will turn out to be run-of-the-mill variety bugs in code, or in configuration, that managed to slip through on both sides, and were not caught previously perhaps as a result of testing with insufficient load. It may be that the larger vendors, having more complicated networks and more volume, were just the most vulnerable to having bad configurations. This just points out how important it is to test, test, test, test, test, test some more, and then test again - but most importantly, test realistically, with realistic configurations and realistic loads. Especially there is something valuable depending on the result, like lives, or the financial well being of millions of trusting pensioners.

Good luck to the developers and integrators who will undoubtedly be working 16-hour days until this is fixed.

M$- Novell - SuSE - FUD (0)

Anonymous Coward | more than 3 years ago | (#35253344)

Troll summary is troll. (2)

GNUALMAFUERTE (697061) | more than 3 years ago | (#35253450)

That quoted phrase 'emerged at linux launch' is NOWHERE in the article. The article says "London Stock Exchange price data failures ‘emerged immediately at Millennium launch’". This is obviously an application issue, why is the summary sort of blaming the operating system?

LSE is a buerocratic mess (2)

alexmin (938677) | more than 3 years ago | (#35253912)

No surprise here for anyone working with them. Their marked data specs have hundreds of pages while Chi-X EU have about 20. Chi-X EU quoting and trading volume is far greater. Coincidence? I say no - people voting with their feet and going from LSE to Chi-X EU (biggest trading platform in EU now), BATS EU, and other MTFs. Maybe the only thing keeping LSE still afloat is a fact that FTSE 100 index is computed using LSE prices. Once that gone, potentially as soon as MIFID II comes out, LSE is going to circle in the crapper.

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>