Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

OKCupid Experiments on Users Too

Unknown Lamer posted about 2 months ago | from the statistics-are-only-skin-deep dept.

Stats 161

With recent news that Facebook altered users' feeds as part of a psychology experiment, OKCupid has jumped in and noted that they too have altered their algorithms and experimented with their users (some unintentional) and "if you use the Internet, you’re the subject of hundreds of experiments at any given time, on every site. That’s how websites work." Findings include that removing pictures from profiles resulted in deeper conversations, but as soon as the pictures returned appearance took over; personality ratings are highly correlated with appearance ratings (profiles with attractive pictures and no other information still scored as having a great personality); and that suggesting a bad match is a good match causes people to converse nearly as much as ideal matches would.

cancel ×

161 comments

Sorry! There are no comments related to the filter you selected.

Shallow people will be shallow (0)

Anonymous Coward | about 2 months ago | (#47553873)

Stunning.

Re:Shallow people will be shallow (4, Insightful)

viperidaenz (2515578) | about 2 months ago | (#47554383)

or perhaps all people are shallow.

Re:Shallow people will be shallow (4, Funny)

ColdWetDog (752185) | about 2 months ago | (#47554691)

"A walk through the ocean of most souls would scarcely get your feet wet".
          - Deteriorata

Flash panic (5, Insightful)

LordLucless (582312) | about 2 months ago | (#47553877)

World discovers A/B testing
Freaks out
Until the next reality tv show comes on

Re:Flash panic (5, Insightful)

stranger_to_himself (1132241) | about 2 months ago | (#47554135)

World discovers A/B testing
Freaks out
Until the next reality tv show comes on

When we (academics) do experiments on people however trivial we usually have to go through ethical clearance, get informed consent etc. I think its skipping that part that people are uncomfortable about. Of course that happens every day in the business world (and even did before computer scientists rediscovered basic experiments and called it A/B testing), but in some of these cases it does start to look like an academic psychology experiment. Perhaps use of OK Cupid implies consent to be experimented on but I doubt that consent is collected in a transparent way.

Your grocery store experiments on you ... (4, Insightful)

perpenso (1613749) | about 2 months ago | (#47554339)

That probably depends upon whether you consider the terms of use of the online service, grocery store loyalty card, casino player's card, etc to be transparent. Those terms of use that no one reads.

There is also consent by action. The casino does A/B testing by offering some a $40 steak dinner plus $40 in chips while it offers others $80 in chips. You clicked on the advertisement/offer, or you opened the envelope that arrived in your postal mail, etc.

Similarly the coupons a grocery store offers you are often part of an experiment. Hell, changing the items on the isle end caps are sometimes part of an experiment.

My marketing processor thought that grocery store loyalty cards were the greatest invention ever in the history of marketing. The data collected and opportunity for experiments enormous.

Re:Your grocery store experiments on you ... (0)

Anonymous Coward | about 2 months ago | (#47554925)

They didn't publish the results in a widely-read publicly-available academic journal, though (and neither did OKCupid). Facebook did. People who do academic peer-reviewed research are (and IMO should be) held to higher standards. One of those standards involves informed consent.

Re:Flash panic (1)

DerekLyons (302214) | about 2 months ago | (#47554353)

When we (academics) do experiments on people however trivial we usually have to go through ethical clearance, get informed consent etc. I think its skipping that part that people are uncomfortable about.

The operative word being "usually", which implies there exist cases where you don't. The discomfort come from people not grasping the existence of the "usually", and that businesses are not academics and product testing is not held to the same standard.

Re:Flash panic (1)

bondsbw (888959) | about 2 months ago | (#47554581)

When we (academics) do experiments on people however trivial we usually have to go through ethical clearance, get informed consent etc.

Academic experiments have external results, publishing findings as scientific research. Business experiments have internal results, data mining with the goal of increasing profits (via providing better value to the consumer, at least in capitalist theory).

Well, at least, I can hope the results stay internal to the business. As with data mining in general, that's not always the case. But perhaps this becoming a mainstream topic will end with a framework on which to judge companies that release "experiment" data about their customers.

Re:Flash panic (2)

pipedwho (1174327) | about 2 months ago | (#47555043)

The problem with most 'commercial experimentation' is that it isn't about getting better value for the consumer, but about how to to best convince the consumer to pay more for something, or buy something, that they otherwise would not have.

Loyalty cards are a way for a business to encourage a customer to return whether or not it is really in their best interest. Phone contracts, transaction 'fees' and 'licensing' are other ways to get people coming back for more of a beating. If you make the fine print and pricing structure too complicated to understand, while offering all sorts of shiny bling in the big print, marketers have found that they can significantly increase sales. Auto bank account debits are great in that the consumer starts to forget that they are continually paying for something, and may take a few extra months (and therefore payments) to cancel a service that they are no longer using - especially when you make it difficult to do.

Factory rebates are another example of sneaky marketing. They make it hard to claim the rebate, in some cases always 'losing' your first application, or finding something incorrect or incomplete in their overly complicated request form. In the end they pay out less than 1 in 5 rebates because most people give up trying to claim that $100. However, when buying the product the consumer factored in the rebate and probably avoided a more suitable competitor with a better more 'expensive' product.

All these techniques would have been arrived at by experimenting on consumers. It is simply about a business trying to get as close to the threshold of pain as possible to maximise profit. Too far and they go backwards - which is why they experiment on a small sample of their market before any large scale roll out.

Re:Flash panic (1)

Bing Tsher E (943915) | about 2 months ago | (#47555289)

That's weird, because here where I live 'Loyalty Cards' are just the only means by which you would ever want to frequent the store. Kroeger (groceries) Walgreen and CVS (both drugstores) all have loyalty card programs. Especially with Kroeger, the prices if you don't use the card are such that you'd just never go into the store with those prices. It doesn't really 'pull' you to Kroeger over any other store chain.

Re: Flash panic (1)

bondsbw (888959) | about 2 months ago | (#47555441)

Specifically regarding getting you to purchase something you wouldn't have, well, I don't see that as bad. You buy stuff because that stuff is worth more to you than the money is sitting in your wallet. That is what every honest transaction is...an exchange that favors both parties.

Shady practices like making it difficult to redeem a coupon are a different story, and frankly should be illegal.

Re:Flash panic (3)

taustin (171655) | about 2 months ago | (#47554597)

It's hard to imagine how anyone could find this to be scientific experimentation, rather than some random crap done in hopes of finding a way to sell more advertising.

Re:Flash panic (0)

Solandri (704621) | about 2 months ago | (#47554913)

When we (academics) do experiments on people however trivial we usually have to go through ethical clearance, get informed consent etc. I think its skipping that part that people are uncomfortable about.

You do realize that you yourself conduct such "experiments" on your friends every day? While making conversation in the lunch room you ask, "Hey, anyone wanna see Planet of the Apes tonight?" That elicits a lukewarm response, so you then ask "Well what about How to Train your Dragon?" You get a lot of interest in that one, so next time you ask about watching movies you're more likely to make suggestions where they can bring along their kids.

I think the dividing line between when you need to get informed consent is when the experiment begins to make people do things they wouldn't have done anyway. Tweaking how people get paired up for dates is fine if they were looking for a date anyway. Forcing them to go on a date when they weren't planning to would require informed consent (and probably compensation).

Re:Flash panic (2)

mattwarden (699984) | about 2 months ago | (#47554929)

Do academic demographers get "informed consent" before processing census data? What about crime statistics? Network security incidents?

Showing a page with and without images and then processing access_log is not the same as monitoring someone's eating habits and stress levels for a week. Just because you call something "an experiment" (a) doesn't mean it is one, and (b) doesn't mean it's the same as all other "experiments".

At least it isn't reddit or Hacker News tyranny. (-1)

Anonymous Coward | about 2 months ago | (#47554141)

I find it really weird that people can get so worked up over A/B testing, but they don't think twice about the blatant censorship that we see daily at reddit or Hacker News. All in all, being exposed to a different version of a website is pretty minor compared to the numerous people who have their freedom of expression absolutely crushed over at reddit and Hacker News. Just look at how many legit comments got unjustly downmodded [ycombinator.com] in some discussion today about Mozilla's new CEO.

I'd expect to see that kind of oppression in a hellhole like Uganda, or perhaps a former Soviet republic. Even if they aren't obliged to encourage free expression, American-based and American-centric websites like reddit and Hacker News should at least not enable and not encourage community-based censorship. Censorship is a form of tyranny, and tyranny has no place in America!

I know, I know, you're going to say, "But what about /.?!" Well, yes, /. does engage in some mild censorship, but at least it's nowhere near as bad as what goes on at reddit or Hacker News. It's totally easy to view -1 downmodded comments here, and they aren't shown in a color that's like totally impossible to read.

I would so much rather be subjected to A/B testing than I would be to censorship. A/B testing is kinda justifiable. Censorship and tyranny is not.

Re:At least it isn't reddit or Hacker News tyranny (2)

cavreader (1903280) | about 2 months ago | (#47554379)

It's obvious you do not have a clue about what real "censorship" is. So a website rejects posts that do not meet their basic and usually very low standards you agree to when posting there, BFD. On the other hand under real censorship the site would not even exist in the first place and if you tried to start one in some countries you would have state security knocking on your door.

Re:At least it isn't reddit or Hacker News tyranny (0)

Anonymous Coward | about 2 months ago | (#47554421)

On the contrary, what the GP has described is indeed real censorship. Censorship isn't just about completely preventing the dissemination of ideas, like you mistakenly think it is. Censorship is about allowing some expression, but carefully controlling it, like the GP described. That way a certain message or theme is delivered in a way that appears to be "organic", but in reality it is all very tightly controlled behind the scenes. The secretly manipulative type of censorship described by the GP is by far a more effective and dangerous type of censorship, due to its deceptive nature.

Re: At least it isn't reddit or Hacker News tyrann (0)

Anonymous Coward | about 2 months ago | (#47554573)

This person was complaining about the color choice of downmodded comments. I'm sure lots of censored writers would love to have had their works distributed by their critics for free in a slightly disagreeable font color.

These sites may not be objective and free platforms for expression, but I don't think anyone should expect them to be. They are private, commercial enterprises. If they actually were the best places for free expression, there would be a problem, because it would be in the hands of private entities. Instead, they are forums for discussions which are moderated by groups with clearly stated rules and agendas. Imperfect and biased, yes, but also doing pretty much exactly what they portray as their agenda transparently, which isn't so bad.

Re: At least it isn't reddit or Hacker News tyrann (0)

Anonymous Coward | about 2 months ago | (#47554599)

White or extraordinarily light grey text on a white background is censorship. It doesn't matter if it's on a screen or on paper. Making it selectively difficult to read certain content is censorship, plain and simple. That can't be denied; it's just a fact!

Re: At least it isn't reddit or Hacker News tyran (0)

Anonymous Coward | about 2 months ago | (#47554949)

I won't argue about dictionary definitions. I'll just say that if you have ever run a website, you know how bad the spammer, troll, and flamebait problems will always be. Personally, I can read those downmoddes colors, but they are an extremely minor way of dealing with the problem of maintaining a decent online conversation among anonymous people.

Re:At least it isn't reddit or Hacker News tyranny (1)

cavreader (1903280) | about 2 months ago | (#47555109)

If someone is secretly manipulating or shaping information to push a preferred outcome it first needs to be secret to have any true effects. Without the secrecy you are free to evaluate the posted information with the knowledge that someone is trying to influence your opinion by excluding certain pieces of information or posts in this particular case. If you recognize this pattern you are free to go to another source for information. Unfortunately there are far to many news outlets or websites pushing their own agendas and partisan editorial lines instead of facts. A lot of folks can not recognize fact from opinion and tend to gravitate towards sources that publish information that validates their pre-determined opinion while ignoring any information that contradicts their stated opinion. You have the far right and far left and everything in between supposedly reporting on or describing the same thing but the information they publish turns out looking like the people providing the information all live in their own little universe. Web forums are notorious echo chambers where facts tend to get in the way. "Winning" the argument comes before facts. Most popular news sources and web sources are becoming adept at using "lies of omission" to shape their stories. This allows them to state that everything they published was factually correct which in a sense would be true but the information omitted could have put a whole different slant on the argument.

Re:At least it isn't reddit or Hacker News tyranny (2)

OhPlz (168413) | about 2 months ago | (#47554723)

Do you think the donation information was leaked accidentally? If you do, you probably think that the IRS also accidentally lost months worth of email that may have contained evidence pertaining to that agency's targeting of conservative groups during an election season. It won't be state security knocking on your door, but the feds are still going to arrange for someone to come after you.

Re:At least it isn't reddit or Hacker News tyranny (0)

Anonymous Coward | about 2 months ago | (#47555029)

"Real censorship?" Do you own a fucking dictionary? Scale of action != Action moron!

OKC did something else, though.. (0)

Anonymous Coward | about 2 months ago | (#47554201)

OKC was doing very different experiments, though. For one, they tried to find out things like questions highly correlated with whether or not you put out on the first date. So it's a bit more than just trying variations of the site to see what gets people to buy more and more of a "how can we rob people of their privacy further" like Facebook and their attitude towards wearing people down on privacy settings by changing them constantly and setting you back to insensible defaults.

Oh, and OKC had the whole thing where they talked about why it's not in your interests to pay to be able to contact people on a dating site (which they removed as soon as they changed their business model...).

Anyhow, my point was that they're a bunch of wankers and they can't be trusted.

The bottom line (0)

Anonymous Coward | about 2 months ago | (#47554289)

People are shallow and have no idea what they want.

I postulate that every person reading this agrees that it is true of most everyone but themselves.

Even after reading that, most readers will believe that they predicted that statement precisely because they are not in that group.

It's a lie!

A/B Testing (4, Insightful)

Mr Z (6791) | about 2 months ago | (#47553879)

Isn't that what A/B testing is all about?

Re:A/B Testing (0)

mozumder (178398) | about 2 months ago | (#47554111)

Yes, it is what it's about, but the narcissistic libertarian princesses that infest this site haven't figured out that other people have power and control over them.

These narcissistic libertarian princesses made the mistake of thinking they are the ultimate power.

Sorry narcissistic libertarian princesses, but you're not in control over your lives - other people have control over you.

Maybe in your next life, you'll learn to gain power in order to maintain some self control?

Meanwhile, we will continue to A/B test the shit out of you and manipulate you to your heart's content.

Re:A/B Testing (2)

Mister Liberty (769145) | about 2 months ago | (#47554185)

My manipulation's as good as yours.

Re:A/B Testing (1)

Bite The Pillow (3087109) | about 2 months ago | (#47554315)

People being manipulated, even if the manipulation is only demonstrated by the experiment, without knowing they are being A/B tested, is not what A/B testing is about.

Knowingly participating in an A/B test is kinda part of A/B testing. Is this lens better, or worse? Which of these televisions side by side looks better to you?

You are looking for love, your soulmate, or someone who will put up with your desire to have cough medicine inserted into your rectum by someone dressed as a Teletubby.

You don't find it because you are accustomed to the way people on dating sites work, and these people are not behaving your way. Cough syrup inserting Teletubbies usually post pictures, and no one fitting the description has. Or vice versa, I don't know.

Or, I have been treating this person who did not post a picture differently, even though they did.

This is much more about how people interact with people who post pictures, and misrepresenting people as picture posters or non picture posters. The misrepresentation is NOT part of A/B testing. Knowing that you don't know is part of an A/B test. Not knowing that you don't know is not.

So no, the answer to your question is no.

Re:A/B Testing (1)

Bite The Pillow (3087109) | about 2 months ago | (#47554415)

Fuck me, I made the point using the wrong experiment. Otherwise, the argument still stands. People trust that the number is as correct as the website can be. Given that it doesn't know whether you like Teletubbies putting cough syrup in your ass.

Still, not A/B testing in any but the most ignorant sense of words.

Re:A/B Testing (1)

Bing Tsher E (943915) | about 2 months ago | (#47555299)

Fuck me,

*cough* I only had the Actimates Teletubbie long enough to dissect it and investigate the interesting LED array. You should try on Craigs list, I suppose.

Re:A/B Testing (1)

Anubis IV (1279820) | about 2 months ago | (#47554501)

A/B testing, as a concept, is fine. The issue here is that A was "truth" and B was "deception", and that's something you shouldn't be A/B testing (at least not without getting ethics waivers signed). Facebook provided feeds that were not representative of what was actually going on and OKCupid flipped bad matches to good matches, both of which compromised their relevant services by misleading users or misrepresenting information. You can't do stuff like that in most (all?) ethical systems, and it may even open them up to legal trouble, since they're knowingly providing something other than the promised service.

At the very least, their doing so runs contrary to the categorical imperative [wikipedia.org] , so for any deontological ethicists out there, it should seem pretty apparent that they were out of line. And if you subscribe to more consequentialist ethical thinking, such as utilitarianism (either the Act or Rule variety), it's trivial to point out that the users were going to obviously be worse off in several of these cases and that happiness was not maximized, nor would it be if everyone was misleading their users like this.

Again, A/B testing is a great tool, but it needs to be used ethically.

people are shallow (1)

asmkm22 (1902712) | about 2 months ago | (#47553881)

All they did was discover that everyone on a dating site places physical attraction (based on a photo) above everything else by a wide margin. Reported "compatibility" and profile data are largely irrelevant. Basically, Hot or Not should be as effective for online dating as eHarmony.

Re:people are shallow (0)

Anonymous Coward | about 2 months ago | (#47553893)

I have no idea what you are talking about but I want to hack their site and take their money.

Re:people are shallow (0)

Anonymous Coward | about 2 months ago | (#47553923)

RTFA. They did the opposite. Controlling the other variables and artificially raising the "compatibility score" increased the number of messages. People are still sheep, just of a different color in this case.

Re:people are shallow (1)

Bite The Pillow (3087109) | about 2 months ago | (#47554355)

They did not control for people thinking they were similar, and looking deeper than they would have otherwise. They made hypotheses and passed them off as conclusions.

Your conclusion is therefore based only on confirmation bias, not on fact.

Re:people are shallow (2, Funny)

Anonymous Coward | about 2 months ago | (#47554217)

"People are shallow and place too much value on appearances," said the fat, ugly nerd. "They should be like me, and value what's in peoples' souls. For instance, I can tell Jennifer Lawrence and Scarlett Johansson are perfect for me, because they have such beautiful souls."

Re:people are shallow (1)

Falos (2905315) | about 2 months ago | (#47554535)

Either you're lamenting in monologue or you're trying to shoehorn in some kind of distance to keep yourself safely apart and hopefully polar from a stereotype you're assembling ad hoc. Need to stay distinct to protect that fragile ego.

Happens during vilifying too.

Re:people are shallow (0)

Anonymous Coward | about 2 months ago | (#47554933)

What, you think that ugly people don't have the same instinctual lust for attractive people that everyone else does? Or perhaps that ugly people are automatically more self-aware and hence are more likely to resist their own desire for attractive people?

Sure, plenty of ugly people eventually realize that they aren't attractive to the hotties, and many of them eventually find a way to settle for what they can have. But that doesn't mean that none of them had to go through a process of rejection in order to get there. And it especially doesn't mean that no ugly people ever indulged in the hypocritical insistence that attractive people should love them for their inner beauty even though they loved the attractive people for their outer beauty.

I am sorry that ugly people are ugly; but I don't presume that their ugliness is automatically counter-balanced by enlightenment.
 

Re:people are shallow (1)

Bing Tsher E (943915) | about 2 months ago | (#47555311)

Possibly they're just lampooning, though, which renders your socio-psych judgment into something kinda silly. And which points to the inquiry: what provoked your rather strong response?

Enough already! (0)

Anonymous Coward | about 2 months ago | (#47553885)

Every site has trolls/flamebaiters that abuse the moderation system to unfairly give some users good and others a bad review, thus making them proud/happy respectively cranky/unhappy and watching the resulting flamewars with glee.

This has been happening since 1991 or so.

Just calling it 'an experiment' doesn't make it more 'newsy'

Re:Enough already! (1)

Opportunist (166417) | about 2 months ago | (#47555173)

Yeah, the good ol' times. When it was other users that trolled you, not the page owners themselves...

So does Slashdot. (0)

Anonymous Coward | about 2 months ago | (#47553889)

Beta is an experiment designed to induce uncontrollable rage in a susceptible populace as part of a plan to create an unstoppable army of Reavers.

Re:So does Slashdot. (0)

Anonymous Coward | about 2 months ago | (#47553955)

"If they take the ship, they'll rape us to death, eat our flesh, and sew our skins into their clothing – and if we're very, very lucky, they'll do it in that order."

what? (0, Troll)

geekoid (135745) | about 2 months ago | (#47553915)

That’s how websites work.".

No. It's what some unethical douche bags do. it has nothing to do with how websites work, asshole.

Re:what? (2)

nospam007 (722110) | about 2 months ago | (#47553927)

"No. It's what some unethical douche bags do."

There are ethical douche-bags?

Re: what? (2, Funny)

Anonymous Coward | about 2 months ago | (#47554101)

Yes, they're divorce attorneys.

Re:what? (1)

Bing Tsher E (943915) | about 2 months ago | (#47554163)

There are ethical douche-bags?

Believe it or not, there are FDA approved douche-bags which are produced by ethical companies, and sold by ethical retailers.

Re:what? (1)

geekoid (135745) | about 2 months ago | (#47554287)

Since douching is harmful and not needed, I would disagree with your statement.

Re:what? (1)

geekoid (135745) | about 2 months ago | (#47554277)

Yes.
I know a physicist who is ethical, but when he speaks outside his expertise, he becomes a douche. Really irritating.

Re:what? (0)

Anonymous Coward | about 2 months ago | (#47553941)

A/B testing is how websites work.

Re:what? (0)

Anonymous Coward | about 2 months ago | (#47554017)

No, websites work with HTTP and shit; A/B testing is how *marketing* works.

Re:what? (1)

thieh (3654731) | about 2 months ago | (#47553999)

Thsi is how the internet works, not just web sites

Re:what? (0)

Anonymous Coward | about 2 months ago | (#47554015)

No, it is how REALITY works, not even just websites.

Your entire life is a big A/B test from every second you are awake, whether you want to think it is or not.
Your life is a toy.

Re:what? (5, Insightful)

vux984 (928602) | about 2 months ago | (#47554033)

No. It's what some unethical douche bags do. it has nothing to do with how websites work, asshole.

Anyone who has ever:
a) taken any metrics about there site
a) altered their website in any way
b) measured whether or not it made any difference

Change the font? Rewriting the sales pitch? Moving the photo to the left? Changing the checkout sequence? Showing more or fewer related products? Added bitcoin as a payment option? Offered a discount? Let you checkout without registering? Adjusted your online advertising budget or changed the keywords you were paying for or targeted a new demographic or region...

Do any or all of those one at a time, checking whether sales increased or not... congrats you effectively "experimented" on your users.

Whether or not it is insidious or unethical doesn't depend on "did you or did you not experiment" it depends on what EXACTLY you've been doing.

Me, I've noticed that people tend to click on articles that are finite lists of things. Hypothetically take an article called "Retirement Savings Strategies Everyone should know" gets fewer clicks than "7 retirement savings strategies everyone should know".

The only change is the addition of the number 7.

The internet has gradually been replaced by "X Y's" articles, because it gets more clicks, as this has become increasingly "discovered" by people "experimenting" on users with different headline styles.

The only upside is that I can safely ignore any "news" site with more than 1 article that starts with a count in the title, as containing nothing more than processed brain diarrhea.

Re:what? (1)

geekoid (135745) | about 2 months ago | (#47554323)

"congrats you effectively "experimented" on your users."

That's not experimenting. generic research on overall performance is not the same as selecting a sample of users, and conducting tests on them specifically to change their response.

You're definition is so loose it's useless.
Now, I need to experiment ans see if I can get home from work.

Re:what? (0)

Anonymous Coward | about 2 months ago | (#47554401)

There's definitely a difference between what Slashdot does with Beta. I even noticed Dictionary.com doing Beta on some page loads. It's annoying. I don't like either Beta site. But thankfully they LABEL IT BETA.

What Facebook and OKCupid are doing is doing it without labeling it BETA.

Label it BETA, probably ethical.
Don't label it, not so much.

And while labeling it BETA can let people know something is up, I don't see the experiment being changed that much different.

Really, pictures not loading in profile? Isn't that like breaking the site to see how people respond?

Re:what? (1)

vux984 (928602) | about 2 months ago | (#47554507)

selecting a sample of users, and conducting tests on them specifically to change their response.

How does changing something about your website to get them to spend more money not qualify as "selecting a sample of users, conduncting tests specifically to change their response"?

So what if the 'sample of users' is everyone, and the A/B test occurs over the same users in two non-overlapping timeframes? If I make the changes to my regionalized .CA website to test the impact on "Canada" before making it to the global site? Does that qualify? Because pretty much all sites do that sort of thing too.

You're definition is so loose it's useless.

That is PRECISELY my point. Getting in a huff about "experimenting on users" is absurd, because the definitions in play ARE uselessly broad. What did OKcupid or facebook do, SPECIFICALLY, that crossed a line that any other website wouldn't do to increase whatever metric they were looking at.

Because, to my view, they haven't done anything different from any other site, at all. So this all really is much ado about nothing.

Re:what? (1)

jxander (2605655) | about 2 months ago | (#47554587)

I see your point, but I can't equate a change of font or layout with lying about compatibility on a dating site.

Especially without any kind of warning to the users in question. Double especially when it's a potentially-paid service (a quuick google search says they OKCupid has both free and paid options... though I've never used the site myself)

On the flip side, if people knew that such shenanigans were afoot, we probably wouldn't get any decent results. Still, it seems like there should at least be a "we are altering our algorithms regularly to try and optimize the compatibility... blah blah blah" stated fairly clearly when you sign up.

Re:what? (0)

Anonymous Coward | about 2 months ago | (#47554899)

I have an account there and this doesn't really change anything. One of the selling points of the site in the past was the fact that they were doing this and publishing the results. So, you'd know what types of profiles were working for other people.

Re:what? (0)

Anonymous Coward | about 2 months ago | (#47555085)

I disagree with your overall assessment, at least in terms of Facebook. (In fairness and honesty I didn't read what OKCupid did for their experiment in detail. On the surface it's not even close to Facebook).

Facebook manipulated content for a select set of users in an effort to evoke psychological reaction and measure responses. That experiment does not match any of your criteria, at least in terms of the generic changing a font. Further, Facebook's experiment was not geared at the perception of "Facebook". Their experiment looks nearly identical to DARPA funded social trolling experiments and documentation leaked about what numerous 3 letter agencies and military organizations were doing for psychological manipulation.

Re:what? (0)

Anonymous Coward | about 2 months ago | (#47555159)

I'll leave this one to Randall Munroe: xkcd. [xkcd.com]

Is it better to try 2 things and make decisions based on the results, or consider 2 things, and make the decision based on a guess? I don't see how screwing with people based on your own guesses is better than doing exactly the same based on a comparison of how the options perform. Its not like hiding profile images is "unethical". In fact, given the context, it sounds like showing them might have been a bad idea in the first place.

OKC started as a science project (4, Insightful)

slaker (53818) | about 2 months ago | (#47553973)

The people who run OKC were a bunch of statistics nerds. It runs (ran, anyway) on a custom web server that performs a lot of real time analysis. Their blog is chock full of incredibly detailed information about their users. This shouldn't be news to anyone who has even the slightest clue as to how OKCupid actually works.

Re: OKC started as a science project (4, Informative)

Anonymous Coward | about 2 months ago | (#47554005)

OKCupid was sold to the owners of match.com a long time ago. When that happened the best blog post that they wrote (Why you shouldn't pay for online dating) was taken down, and the blog itself hasn't been updated (not counting the entry this articles about) since April of 2011.

Re: OKC started as a science project (5, Informative)

Anonymous Coward | about 2 months ago | (#47554121)

Here you go:
https://web.archive.org/web/20... [archive.org]
Yeah the blog title sais everything: Communist Inc, which doesn't want to make money, gets swallowed by Monster money Corp, which makes money.

"Everybody is doing it so everybody is doing it" (0)

Anonymous Coward | about 2 months ago | (#47553981)

Shyeah, uh, no.

SlashAd (0)

egr (932620) | about 2 months ago | (#47553989)

Is that some sort of advertisement?

I see no other reason to "jump in" with that kind of information.

Plane loo makes shit fall into the cloud (-1)

Anonymous Coward | about 2 months ago | (#47553995)

and because of passenger poo in the cloud, all cloud user experiments are shit!!!!

OKC's match algos suck (4, Insightful)

thisisauniqueid (825395) | about 2 months ago | (#47554063)

Findings include that ... suggesting a bad match is a good match causes people to converse nearly as much as ideal matches would.

All this means is that OKC's match algorithms suck: there's only a weak correlation between match scores and real-world compatibility (like with every other dating site).

Re:OKC's match algos suck (1)

penguinoid (724646) | about 2 months ago | (#47554165)

Findings include that ... suggesting a bad match is a good match causes people to converse nearly as much as ideal matches would.

All this means is that OKC's match algorithms suck: there's only a weak correlation between match scores and real-world compatibility (like with every other dating site).

No, it means that:
1) People trust OKCupid's rating system enough to try harder when it suggests a good match
2) OKCupid has to take into account their stated match rating, not just length of conversation, when trying to use conversation length as data to improve their algorithm.

Re:OKC's match algos suck (2)

penguinoid (724646) | about 2 months ago | (#47554205)

edit: 1) Users of OKCupid trust OKCupid's rating system enough to try harder when it suggests a good match

Re:OKC's match algos suck (1)

Impy the Impiuos Imp (442658) | about 2 months ago | (#47554173)

It's called the "tyrrany of dimensions". The more variables you have, the more data points you need exponentially to derive meaningful partitioning analysis from it, regardless of how clever your distance algorithms are.

And they have hundreds of questions when a dozen would be about all the entire population of Earth could support.

Re:OKC's match algos suck (1)

Too Much Noise (755847) | about 2 months ago | (#47555433)

It's called the "tyrrany of dimensions". The more variables you have, the more data points you need exponentially to derive meaningful partitioning analysis from it, regardless of how clever your distance algorithms are.

Indeed, but only if you insist on carrying along in your analysis all the irrelevant and correlated dimensions.

And they have hundreds of questions when a dozen would be about all the entire population of Earth could support.

So do surveys, for significantly smaller sample sizes. I wouldn't be surprised if a non-trivial percentage of those questions are intentionally redundant - you know, to check *ahem* consistency, improve accuracy, etc. If, say, you have 100 questions grouped into 10 categories with 10q/cat, you have just dropped the dimensionality significantly while at the same time having more confidence in your data. A rule of thumb in surveys is don't trust the user^W^W^W^W *ahem* trust, but verify.

Re:OKC's match algos suck (1)

wvmarle (1070040) | about 2 months ago | (#47555409)

Well, of course. They're digging too deep in that. In the real world, I believe that there is no such thing as "the one" or "the perfect match". Maybe it feels like it, but that's in part thanks to the "pink glasses" effect of being in love and because both parties tend to adopt to one another, especially when a relationship lasts long (years, decades).

People probably can form lasting romantic relationships with a large number of other people, after the following basic matches are followed (assuming heterosexual relations but some will apply for homosexual relations too):

  • Geographic proximity.
  • Speaking the same language, or at least share a second language.
  • Similar age, preferably the female 1-5 years younger than the male.
  • Similar educational level, or the male having higher education than the female.
  • Similar political/religious views (left/right wing, Muslim/Christian/Buddhist/etc).

The above are true for the vast majority of heterosexual relationships. Another major factor in partner choice is also the availability of the person, as in, that s/he is not in another relationship already. The fact that someone is active on sites like OKCupid fulfils that requirement. Coincidence plays a great role as well: whether you meet a person now (when he's single and looking) or in half year (when he's just got a girlfriend). Whether you meet the person at all. He may be a perfect match for you on all fronts, yet unobtainable due to living 1,000 km away.

Duh (0)

Anonymous Coward | about 2 months ago | (#47554073)

Any website based on an algorithm run by even remotely responsible people is going to use A/B testing. You're always testing changes on your users by the simple fact that they're using your site.

If you're not running a hold-back experiment and measuring the impact of changes, you'll never know which changes are actually good for your users.

Ok Cupid.... (1)

bobbied (2522392) | about 2 months ago | (#47554087)

That was STUPID!

Conduct your little experiment if you have to, just keep your mouth shut about it.... At least until you have notified ALL your users that such experiments *might* be taking place (Or if you intend to issue refunds from the resulting class action suit.)

Re:Ok Cupid.... (1)

penguinoid (724646) | about 2 months ago | (#47554221)

As opposed to just randomly matching people so as to not have to learn things about people in general nor their users in particular?

Re:Ok Cupid.... (1)

joe_frisch (1366229) | about 2 months ago | (#47554437)

The particularly stupid part was messing with their match algorithm. If they imply that their algorithm has any value, then their users will feel at least ripped off (since the algorithm doesn't seem to work well), and possibly angry because they were given incorrect information .

Blocking pictures was visible to users and I don't have any problem with that .

This is different from what Facebook did (5, Interesting)

nikhilhs (1292298) | about 2 months ago | (#47554177)

FB's experiment was to see if they could alter the mood of their user. OKC tried to see if they could get more conversations going. Intent matters. OKC's is fairly harmless. FB's experiment could have a ripple effect and cause negative consequences.

Huge differences (1)

jimmyswimmy (749153) | about 2 months ago | (#47554199)

There's a huge difference between A/B testing, designed to optimize your website with the direct intent to improve sales, and performing experiments on how different news feeds affect your users' moods. A/B testing typically comprises changes in button size and color, website layout, font variations, etc; should we lead with the price, or with the benefits, or with something else? On the other hand, what FB and OKC are doing - admitting to, and proudly! - amounts to wholesale experimentation on their users, with undisclosed intent - perhaps to make the users come back more frequently for another hit.

This seems akin to me to cigarette companies manipulating the nicotine content of their products. That didn't go over well when it was finally disclosed.

You can't just tell people you "might" experiment with them, they have to know and understand that they are part of an experiment. They don't have to understand the goal, they just need to know what they are part of, and they have to consent to that experimentation. One could argue that A/B testing should submit to the same level of scrutiny as other psychological experiments, but I think people generally understand and accept corporations' profit incentive. We don't accept the idea that a company might wish to screw around with our mood or set us up on a date when they know it won't work out.

Marketing (4, Insightful)

Bo'Bob'O (95398) | about 2 months ago | (#47554253)

The fact is that the experiment they Facebook conducted was mild to what other corporations do every day under the umbrella of "marketing".

They use control groups and try every trick they can to manipulate your mood, feelings, impressions of their products. They carefully script interactions to take advantage of your feelings and social norms. Also take the recent example in the past few weeks of the scripts that Verizon's 'account retention' departments use to try and wedge people into keeping their account longer. Those weren't just thrown together, those were made with careful research and years of experiments on customers and focus groups.

The only difference with what Facebook did and the rest do is that they shared their results with everyone. Was Facebook Unethical manipulating people the way they did? I think so, and I'm only less interested in the service after that scandal, but what they got them in trouble was sharing it with the rest of the world in a way that might have also done some honest good. Now they will learn from their mistakes, keep it to themselves, and use that research purely to manipulate people for higher profit and no one will say a thing.

Re:Marketing (1)

DNS-and-BIND (461968) | about 2 months ago | (#47554821)

There are three professions where being untruthful is the key to success: Lawyers, salespeople, and marketing. All three are hired to portray their client in the most favorable light possible, and the very best ones lie through their teeth. The worst of these three are the marketers because they have legions of psychologists and scientists trying to figure out the best way to lie to people.

Mozilla (0, Flamebait)

Anonymous Coward | about 2 months ago | (#47554303)

OKCupid.. the intolerable twats that managed to get a Mozilla CEO fired because of his mainstream beliefs. Why is anything they do on the front page?

Re:Mozilla (0, Flamebait)

Anonymous Coward | about 2 months ago | (#47554637)

Nice down-modding. Liberalism folks, agree or be silenced. No wonder west coast employers are so desperate for tech workers. Who would want to deal with that?

Re:Mozilla (0)

Anonymous Coward | about 2 months ago | (#47555145)

Hahaha, disregard my rants, I suck cocks!!!!

Dating sites simply lie. (2, Interesting)

Kris_J (10111) | about 2 months ago | (#47554527)

Try this the next time you want to try an online dating site: Create two profiles, a "real" one and a fake perfect match to your real profile and see how long it takes for the site to claim that your fake perfect match has attempted to contact you and for only $4.95 you can sign on to the paid service and reply.

Re:Dating sites simply lie. (1)

Ksevio (865461) | about 2 months ago | (#47554663)

Funny you should mention that since this very blog had a length article discussing that topic. It's gone now that match.com bought them out, but you can still find it in archives.

Re:Dating sites simply lie. (1)

Cammi (1956130) | about 2 months ago | (#47554777)

You forgot to link the blog ....

Re:Dating sites simply lie. (1)

Ksevio (865461) | about 2 months ago | (#47554909)

The blog is in the summary...

Re:Dating sites simply lie. (0)

Anonymous Coward | about 2 months ago | (#47554943)

I think this is the one the GP meant to refer to https://web.archive.org/web/20100725135309/http://blog.okcupid.com/index.php/why-you-should-never-pay-for-online-dating

Anyways, that's why I don't pay for a subscription, I'd be more than happy to pay them a couple hundred dollars after I've gotten married, but paying up front is idiotic. There's no incentive to improve the service so that people are anything more than just barely happy with their prospects. And to keep the subscribers strung along for as long as possible.

Re:Dating sites simply lie. (1)

Anonymous Coward | about 2 months ago | (#47554729)

That's not actually how okcupid works. You aren't charged for talking to your perfect match. You are charged if you go over the limit of 100 messages and don't want to delete any. And, you are charged if you want to remove advertisements. If you use adblocker their ads actually ask you to turn it off for their site because they get most of their revenue that way.

Retarded Idiots (not redundant) (0)

Anonymous Coward | about 2 months ago | (#47554557)

World produces enough stress without retards playing God because they can f#ck with people on their lame websites.

This is outrageous (1)

mysidia (191772) | about 2 months ago | (#47554689)

I think Firefox should boycott the site.... display a message about it being possibly malicious/dangerous to all users attempting to visit OKCupid, showing a link to the article as a warning message in bright red... (Just kidding <EG>).

Re:This is outrageous (1)

Bing Tsher E (943915) | about 2 months ago | (#47555339)

The problem with that is, the folks who remain now at Firefox bought into the same sub-ethical slime ethos. They were basically part of the same chorus on that earlier issue as these fine folks at OKCupid. Those who would enjoy the irony are gone or at least thoroughly stifled within the present organization.

I've wondered what OKCupid thinks when... (0)

Anonymous Coward | about 2 months ago | (#47554745)

I've wondered what OKCupid thinks when I click on a profile that's a 10% match and 50% enemy. Sometimes I do that just out of curiosity. It's like watching a train wreck or something. Usually they're pretty accurate when it's that severe--I'm the moderately spiritual non-religious logical person who has only had a few partners and doesn't like body art. They'll be moon-beamy people who are serious about their astrology and are having a hard time keeping track of their partners and piercings. Sometimes though, these people are not that different. Other times I'll get a good solid 85% match, and I'll be like... WTF. This person has a kennel in their house. I couldn't live with that. Last time I looked, I think their system *does* account for "deal breakers", but maybe that one isn't in my profile.

Anyway, I think you get what you deserve if you take a site like that too seriously. I know one couple (through another online forum) that got married because of OKCupid. There are probably a lot more that just had bad dates.

"That’s how websites work." (1)

Cammi (1956130) | about 2 months ago | (#47554771)

"That’s how websites work." Whoa so OKCupid was retarded enough to hire someone who have NEVER, EVER been on the internet? Some heads need to roll ...

business is all about experimentation (1)

mattwarden (699984) | about 2 months ago | (#47554947)

very few successful businesses are doing what they were originally founded to do. business is all about experimentation. you tweak and reset and change and reset again until you see the numbers going in the right direction at the desired speed. unsuccessful businesses usually do the same thing, too; they just don't ever find a combination that works.

see also: "Getting to Plan B: Breaking Through to a Better Business Model" http://www.amazon.com/Getting-... [amazon.com]

"To succeed, you must change the plan in real time as the inevitable challenges arise. In fact, studies show that entrepreneurs who stick slavishly to their Plan A stand a greater chance of failing-and that many successful businesses barely resemble their founders' original idea. ... Testing those assumptions and unearthing why the plan might not work."

What next? (1)

AlCapwn (1536173) | about 2 months ago | (#47555135)

OkCupid will start pairing everyone with the worst possible match. Once and for all, we'll be able to prove that opposites attract!

Experimentations or shady practices? (1)

ruir (2709173) | about 2 months ago | (#47555369)

I have a completely different opinion of online dating services at all. When you are trying them in the trial period, which of trial is a waste of time, as just it lets you browse the public profiles and receive messages, you are most certain to receive one or two messages, often in english, no matter what your mother tongue, of someone VERY INTERESTED in meeting you, just to make sure you sign up for the service. Do those people think we are dumb?
Load More Comments
Slashdot Login

Need an Account?

Forgot your password?