Statistics Losing Ground To CS, Losing Image Among Students
Unknown Lamer posted about three weeks ago | from the big-bad-data dept.
115
theodp (442580) writes Unless some things change, UC Davis Prof. Norman Matloff worries that the Statistician could be added to the endangered species list. "The American Statistical Association (ASA) leadership, and many in Statistics academia," writes Matloff, "have been undergoing a period of angst the last few years, They worry that the field of Statistics is headed for a future of reduced national influence and importance, with the feeling that: [1] The field is to a large extent being usurped by other disciplines, notably Computer Science (CS). [2] Efforts to make the field attractive to students have largely been unsuccessful."
Matloff, who has a foot in both the Statistics and CS camps, but says, "The problem is not that CS people are doing Statistics, but rather that they are doing it poorly. Generally the quality of CS work in Stat is weak. It is not a problem of quality of the researchers themselves; indeed, many of them are very highly talented. Instead, there are a number of systemic reasons for this, structural problems with the CS research 'business model'." So, can Statistics be made more attractive to students? "Here is something that actually can be fixed reasonably simply," suggests no-fan-of-TI-83-pocket-calculators-as-a-computational-vehicle Matloff. "If I had my druthers, I would simply ban AP Stat, and actually, I am one of those people who would do away with the entire AP program. Obviously, there are too many deeply entrenched interests for this to happen, but one thing that can be done for AP Stat is to switch its computational vehicle to R."
Well (1)
Anonymous Coward | about three weeks ago | (#47764169)
My margin of error is pretty high so things never really seem to turn out how I expect them to turn out.
As a statisticians (3, Interesting)
Anonymous Coward | about three weeks ago | (#47764197)
As a statisticians, you should know better that you don't make your point with a succession of anecdotes as
- A few years ago, for instance, I attended a talk by a machine learning specialist who had just earned her PhD at one of the very top CS Departments. in the world. She had taken a Bayesian approach to the problem she worked on, and I asked her why she had chosen that specific prior distribution. She couldn’t answer – she had just blindly used what her thesis adviser had given her–and moreover, she was baffled as to why anyone would want to know why that prior was chosen.
- But there is no substitute for precise thinking, and in my experience, many (nominally) successful CS researchers in Stat do not have a solid understanding of the
fundamentals underlying the problems they work on. For example, a recent paper in a top CS conference incorrectly stated that the logistic classification model cannot handle non-monotonic relations
Re:As a statisticians (3, Funny)
BorisSkratchunkov (642046) | about three weeks ago | (#47764215)
Re:As a statisticians (4, Insightful)
Anonymous Coward | about three weeks ago | (#47764295)
Machine learning is an example in the article. This is a blatant attack on all CS students, researchers and professors.
Let’s consider the CS issue first. Recently a number of new terms have arisen, such as data science, Big Data, and analytics, and the popularity of the term machine learning has grown rapidly.
He seems to not really know CS. Statistics and probability are a tool to CS since the very inception. This is no news.
Re:As a statisticians (0)
Anonymous Coward | about three weeks ago | (#47764335)
In just one journal (Journal of Machine Learning Research [jmlr.org] ), there is hundreds of papers [jmlr.org] . 2 anecdotes is insignificant. This guy have no shame.
Re:As a statisticians (1)
Anonymous Coward | about three weeks ago | (#47764587)
According to you, he's attacking himself: He is a CS professor (with a PhD in statistics) in a CS department, who thinks that computer science needs to make better use of the tools of statistics and probability.
Re:As a statisticians (0)
Anonymous Coward | about three weeks ago | (#47764675)
Let’s consider the CS issue first. Recently a number of new terms have arisen, such as data science, Big Data, and analytics, and the popularity of the term machine learning has grown rapidly.
He seems to not really know CS. Statistics and probability are a tool to CS since the very inception. This is no news.
You're responding to a point he never made. 'Data science', 'Big Data', and 'analytics' *are* terms which were recently added to the IT trade lexicon, and machine learning *is* growing rapidly in popularity. So he's correct on the points that you quoted.
Re:As a statisticians (0)
Anonymous Coward | about three weeks ago | (#47764933)
Wrong :
- The term "data science" (originally used interchangeably with "datalogy") has existed for over thirty years and was used initially as a substitute for computer science by Peter Naur in 1960. In 1974, Naur published Concise Survey of Computer Methods, which freely used the term data science in its survey of the contemporary data processing methods that are used in a wide range of applications. [wikipedia.org]
- big data history [forbes.com] . Words coined at least 15 years old probably more. History, at least back 70 years
- analytics I won't discuss about, so generic terminoly, so old concept.
Re:As a statisticians (2, Informative)
93 Escort Wagon (326346) | about three weeks ago | (#47765041)
You're arguing against a point he didn't make. He didn't say those terms were recently created - he stated they recently were added to the "IT trade lexicon", which is true.
10-15 years ago you didn't hear people bandying about the terms "big data" or "data science".
Re: As a statisticians (0)
Anonymous Coward | about three weeks ago | (#47765133)
As a scientists forced to become a SAN and cluster admin, it's because how data is generated changed.
Film was replaced by digital phosphor and fluoro digital imaging systems. Sequencing progressed from tens of bases to a hundred Gbases per experiment. Cameras migrated from Kodak slides to high bit depth digital movies. I'm a small lab admin. We generate about 2TB of new data to process every day. Scientist can now generate data in volumes they aren't equipped to handle. It was a good time to be in science.
Re: As a statisticians (1)
bill_mcgonigle (4333) | about three weeks ago | (#47766781)
what's funny is that CS nerds and stats nerds work very hard together to enable hard drive firmwares that permit the very dense and cheap storage that scientists and statisticians need. Not to mention the broad applicability of coding theory to every other discipline. TFA might have a point on the margins but by and large he's trolling academia (which is working to bring attention to his issue).
Re:As a statisticians (0)
Anonymous Coward | about three weeks ago | (#47765223)
All my apologies, I did not noticed CS was now IT trade.
Let’s consider the CS issue first. Recently a number of new terms have arisen, such as data science, Big Data, and analytics, and the popularity of the term machine learning has grown rapidly.
For sure, my argument was cleary a straw man by misrepresenting CS as not IT Trade and confusing buzz words with science.
Re:As a statisticians (3, Insightful)
u38cg (607297) | about three weeks ago | (#47765031)
Re:As a statisticians (0)
Anonymous Coward | about three weeks ago | (#47766237)
>Anyone doing "statistics" who doesn't understand the concept of a prior is just pretending to do statistics.
Please explain.
Sure, we could lose 50% of our statisticians (5, Funny)
NotDrWho (3543773) | about three weeks ago | (#47764199)
But there's only a 25% chance of that.
Re:Sure, we could lose 50% of our statisticians (0)
Anonymous Coward | about three weeks ago | (#47764359)
So it's either a coin toss or a crap shoot. The CS students should let /dev/urandom decide.
Re:Sure, we could lose 50% of our statisticians (1)
Austerity Empowers (669817) | about three weeks ago | (#47765951)
I question whether the distribution of /dev/urandom provides the distribution necessary to ensure the desired outcome.
Re:Sure, we could lose 50% of our statisticians (1)
Anonymous Coward | about three weeks ago | (#47764409)
but that applies 100% of the time.
Re:Sure, we could lose 50% of our statisticians (0)
Anonymous Coward | about three weeks ago | (#47765415)
110%, because Coach Smith always told us that's what we needed to give to win.
Re:Sure, we could lose 50% of our statisticians (1)
HornWumpus (783565) | about three weeks ago | (#47768911)
Whenever someone asks for 110% I give it to them. 25% mon-thurs and 10% on friday.
Re:Sure, we could lose 50% of our statisticians (1)
mysidia (191772) | about three weeks ago | (#47767043)
But there's only a 25% chance of that.
And that's just the average probability, with the actual probability for a given sample in terms of percentage points having a standard deviation of + / - 25 percentage points.
Agreed (0)
Anonymous Coward | about three weeks ago | (#47764213)
Yeah, I'd definitely switch AP Stat's computational vehicle to R. I might even take it straight to S or T if it could be done at reasonable cost.
Re:Agreed (2)
Bigbutt (65939) | about three weeks ago | (#47764369)
But what about Q?
[John]
Re:Agreed (4, Funny)
Chris Mattern (191822) | about three weeks ago | (#47764777)
We'll have to ask M.
Re:Agreed (1)
Hotawa Hawk-eye (976755) | about three weeks ago | (#47764977)
Hmm ... John de Lancie in the next Bond film as the gadgeteer for the CIA, with whom MI6 partners on a mission of importance to both agencies? I'd see that.
Statistics as standalone field (4, Insightful)
sinij (911942) | about three weeks ago | (#47764227)
Correctly applying all of these require subject matter expertise. You need to understand what you analyzing. As a result pure statistician is not very useful - generic analysis can be performed by software, in-depth analysis requires specific knowledge.
This is not unlike complaining that assembly coding is dying. Well, yes, we now have less need to code everything that way because we have better tools.
Re:Statistics as standalone field (4, Insightful)
Anonymous Coward | about three weeks ago | (#47764405)
Correctly applying all of these require subject matter expertise. You need to understand what you analyzing. As a result pure statistician is not very useful - generic analysis can be performed by software, in-depth analysis requires specific knowledge.
From my experience, statisticians tend to be far more successful acquiring subject matter expertise than people in other fields have in using proper statistical procedures for their problems.
It's like saying mathematicians are not useful because calculators. It's simply not true, and while software can perform generic analysis, it is only quite a tiny part of doing a statistical problem correctly. What we have now are coders who think that computers can set up and interpret their problems correctly, and thus we have an increase in bad results.
Re:Statistics as standalone field (1)
sinij (911942) | about three weeks ago | (#47764683)
Re:Statistics as standalone field (0)
Anonymous Coward | about three weeks ago | (#47764435)
Who will complain against some old people dying [wikipedia.org] furthermore french ? Anyway I never noticed they were coding.
Re:Statistics as standalone field (4, Insightful)
wisnoskij (1206448) | about three weeks ago | (#47764569)
Re:Statistics as standalone field (3, Insightful)
sinij (911942) | about three weeks ago | (#47764613)
Re:Statistics as standalone field (1)
Anonymous Coward | about three weeks ago | (#47764767)
Amen! Again anecdotal, but my experience is that junior statisticians, i.e. recent graduates, are for more likely to be able to perform the statistics you need in business, health research, etc than a statistician with postgraduate qualifications. I suppose it's a case of a little knowledge going a long way. The junior guy has more than enough knowledge to deal with 80% of what most people might need to do. It's rare that business or the health sciences need anything more complex than an ANOVA to make their decisions, assuming they've captured their data correctly. Ask your local stats professor a basic question: how do I tell if my new business process is effective? or How do I tell if my experimental treatment affects fur growth on rats? are rarely answered simply and in context. A t-test is probably sufficient for both cases, and a junior statistician will blurt that out first. The professor will pontificate and not give you a straight answer.
I had a simple case where I had two datasets produced by different software algorithms running on the same raw data. No algorithm was particularly reliable, and I wanted to combine their reliabilities, i.e. combine the p-values. Fischer's method does this just fine, and yes my data matched the assumptions. Could one qualified statistician tell me how to do this? NO! It was a an undergraduate who had just learnt about it who told me about it, and showed me a textbook to look it up in.
Re:Statistics as standalone field (1)
HornWumpus (783565) | about three weeks ago | (#47768957)
What you say is generally true of professors. Not just statisticians.
But it's reasonable. The professors assume you already know the simple answer and give the somewhat complete one.
Re:Statistics as standalone field (1)
radtea (464814) | about three weeks ago | (#47769923)
If your view that "everyone is complete shit at statistics", that should include statisticians.
This has been my experience as well. I would go so far as to say that statisticians understand probability less well than most working experimental scientists. They are overly-enamoured of abstract models and rarely dig down to the raw probability distributions underneath, which is what working scientists actually care about.
Re:Statistics as standalone field (1)
JeffSh (71237) | about three weeks ago | (#47765205)
it seems to me that statisticians are CS overlapping with practical insight.
When a report is prepared, it takes consideration to define all of the inputs and modifiers that lead to a successful statistical analysis. Without this hard-to-define inputs, i can see how and why a CS-only based approach to stats fails.
Re:Statistics as standalone field (1)
disposable60 (735022) | about three weeks ago | (#47765537)
But I thought Econ was the Dismal Science.
Re:Statistics as standalone field (1)
Anonymous Coward | about three weeks ago | (#47766033)
I want to learn statistics, but it's so ... big. There just aren't enough hours in the day as a working software-industry professional to do something as big and complex as learning statistics. After a day of coding and debugging, my brain is fried. Opening a probability book just doesn't work. I guess you need to learn it while you're still in school, and your job is studying. I took a statistics class in college, but can't remember anything about it.
Besides, "statistics" is an umbrella term for probability, combinatorics, regression, and so on. Each field could take years to master.
Statistics as standalone field (3, Informative)
aaaaaaargh! (1150173) | about three weeks ago | (#47765013)
Quite the opposite is the case. Unless we are talking about experiments with terrabytes of data most software packages are complete overkill anyway, you could make your statistics with a pocket calculator instead. The problem is the conceptual work. Most institutes and individual scientists would be much better off if they employed a well-trained full-time statistician. Provided they were interested in correct and robust results rather than getting one more pilot study published as soon as possible (which will in turn be based on an insignificantly small non-random sample using an inadequate model).
Re:Statistics as standalone field (1)
Rich0 (548339) | about three weeks ago | (#47766783)
I think you hit the nail on the head. Nobody cares about getting it right, they just care about getting it accepted. People know enough statistics to be dangerous.
But the same is true of almost any field. Unless you work in some kind of skunk works team, how many of your coworkers REALLY have a good grasp of the fundamentals in whatever profession you work in? Do you think the average CS major has any idea what an opcode is, or how to implement a binary adder (just in terms of theoretical gates, let alone actual circuitry)? How many cooks really appreciate how the presence and properties of water affects the temperature at which food cooks?
I'm sure even among experts, the average statistician is, well, average.
Re:Statistics as standalone field (0)
Anonymous Coward | about three weeks ago | (#47765319)
Don't forget that Statistics isn't one of those "Hard" math fields. A properly taught statistics course should be about 50% math and 50% ethics. This turns a lot of CS types off since they usually don't like social science.
A properly taught statistics course will also turn you in to a cynical skeptic. You'll basically never believe anything told to you by any authority every again.
Re:Statistics as standalone field (1)
Capt.Albatross (1301561) | about three weeks ago | (#47766121)
As a result [a] pure statistician is not very useful - generic analysis can be performed by software, in-depth analysis requires specific knowledge.
In-depth analysis requires a real understanding of statistics as well as of the domain. CS knowledge, at least as commonly taught, is not a substitute for for the statistics requirement.
This is not unlike complaining that assembly coding is dying. Well, yes, we now have less need to code everything that way because we have better tools.
This is not a valid analogy. HLLs automated some of the rote, mechanical aspects of implementing algorithms. They do not automate away the need for a higher-level understanding of what you are doing.
Re:Statistics as standalone field (0)
Anonymous Coward | about three weeks ago | (#47769073)
It sounds to me like you're so bad at statistics that you don't know how bad at statistics you are.
Is the benefit worth the cost? (0)
Anonymous Coward | about three weeks ago | (#47764237)
If there is a decline in quality it is probably because the quality wasn't needed and people don't want to pay for things they don't need.
Students don't want to waste their time learning something they aren't going to use when they go to work and in most cases bad or even incorrect statistics will be just as useful.
In the same way most engineering doesn't require much math but rather just an understanding of how things work. Why you have that understanding the math doesn't get much more complicated than division.
It would be great if the students knew things better, but they are limited by time and money so they will focus on the things that gives most reward to the time put in. If Prof. Matloff is worried he might want to consider teaching the students statistics for free.
Re: Is the benefit worth the cost? (0)
Anonymous Coward | about three weeks ago | (#47764421)
> In the same way most engineering doesn't require much math but rather just an understanding of how things work. Why you have that understanding the math doesn't get much more complicated than division.
Engineering stops at division? You must be joking. Engineering is more than applying d=rt everywhere, idiot.
Re: Is the benefit worth the cost? (0)
Anonymous Coward | about three weeks ago | (#47765649)
NASCAR Engineering...
Ask a math major what stats major is (-1)
Anonymous Coward | about three weeks ago | (#47764253)
Re:Ask a math major what stats major is (1)
Salem Lowe (3800579) | about three weeks ago | (#47764491)
Re:Ask a math major what stats major is (0)
Anonymous Coward | about three weeks ago | (#47764597)
Statistics has always had difficulty with usurpers (5, Insightful)
BorisSkratchunkov (642046) | about three weeks ago | (#47764273)
What is concerning is how many statistical tools, each with their own set of assumptions, have blossomed up within the past few decades. There are so many stats now that stats can no longer be an ancillary to other disciplines- it needs to be given its own space and statisticians need to be given respect for their unique expertise. There is simply too much knowledge in that domain for those in more theory-driven fields to be able to claim both expertise in the conceptual models of their fields and statistics.
Re:Statistics has always had difficulty with usurp (0)
Anonymous Coward | about three weeks ago | (#47764367)
Most notably psychology, economics, mathematics and beer brewing. In fact, most of the developments in stats have come about as a result of a need arising in a different discipline. Stats is inherently an applied discipline, so this is not unusual.
That can be said for most math.
Pure mathematicians tend to only deal with what is already known. If you start to look at the greatest advances in math and the mathematicians behind them you'll see that they most of the time came from another field and needed new math to solve their problem. Necessity is the mother of invention.
Re: Statistics has always had difficulty with usur (0)
Anonymous Coward | about three weeks ago | (#47764717)
IIRC isn't that how calculus came about?
Correlation != Causation (0)
Anonymous Coward | about three weeks ago | (#47764293)
Any time you see a summary on Slashdot with the word "statistics" repeated a few times, that observation has to apply somewhere.
Statistical Practitioners need to Modernize (5, Interesting)
wispoftow (653759) | about three weeks ago | (#47764319)
I am a researcher in medical informatics, and statistics is a huge part of my job, though I am not a classically-trained statistician.
First, I would like to offer a stark contrast between two types of statisician: 1) statisticians of the old mold who are wedded to SAS and related tools and 2) research statisticians who employ modern methods such as Bayesian statistics and rather advanced calculus. The former tend to mold all problems into what is available in the canon of SAS routines, while the latter are capable of creating custom models that suit the problem at hand.
Then, there is a new breed of scientist -- the data scientist -- who tends to use black-box machine learning methods and the classical techniques, as programs such as SAS and R have "democratized" the field. I agree with the common gripe of many traditionally-trained statisticians who object that these "data scientist" tend not to understand the statistical background of these computer codes. In fact, it is easy to download R onto one's computer and start firing data through, with little regard for the merits of the model or its results. (Not all data scientists are like this, but I'm simply stating a general observation.)
Another problem with statistics is that it can be very confusing, understanding just what things like p-values mean. After a first course in statistics, it leaves many with a bad taste -- either being terribly confusing, or rather boring. In my opinion, this is because of traditional (frequentist) statistics, which have their origins from luminaries such as Fisher and Pearson.
The "action" today is in Bayesian statistics. This formulation allows for statistical concepts to be expressed is ways that (I believe) most people can understand. But executing Bayesian statistics mandates that one understand the underlying formulation of models; in general, they are not black-box methods. Furthermore, they can be quite computationally-expensive for large data.
Statistics is suffering from perceptions of being a button-pushing, boring profession. As has happened in many other fields (e.g. computational chemistry and CFD), computer programs have democratized the field so that those who have not had years of dedicated study and training can execute statistical models. In my experience, this can be a good thing, or a very bad thing. Another issue is that there is a significant build-up of half a century of code and protocols in both industry (think big business analysis) and government agencies (think FDA).
But modern statistics is actually a hot field. Provided that one understands the background, and is willing to go the extra mile to write custom code, the rewards are endless.
Re:Statistical Practitioners need to Modernize (1)
fropenn (1116699) | about three weeks ago | (#47765609)
Re:Statistical Practitioners need to Modernize (0)
Anonymous Coward | about three weeks ago | (#47767417)
Statistics is mostly thought of as an insurance company job, or working for the gov. on things such as unemployment, death rates, etc.
Re:Statistical Practitioners need to Modernize (5, Informative)
Anonymous Coward | about three weeks ago | (#47765681)
It's a funny coincidence this appeared on Slashdot, as I was just reading about this issue and discussing it with my colleagues.
I'm a statistics researcher in an applied field (university academic research) that suffers its own image problem, and my impression is that what we're witnessing in many STEM areas are problems with stereotyping in science, and marketing fads. I'm not sure that I disagree with what you're saying, but I think that there's another stereotype operating as well that cuts at the field of statistics in a second direction.
As you point out, there are the sort of applied consulting statisticians who are probably getting increased competition from "data scientists."
On the other side of the issue, though, you have complaints about theory-focused statisticians who really don't understand how to implement their developments computationally, who are also getting increased competition from "data scientists." This has been mentioned in a number of blog posts in various places, and I see as much more as the driver of "data science" as a banner than competition with consulting statisticians. E.g., CS individuals who feel they can do Hadoop and so forth, and who have had enough stats training, probably in undergrad, that they feel like they can just sort of usurp the statistics from the statisticians. They see the theory as irrelevant or something.
The problem as I see it is that individuals who identify as "data scientists" don't really understand that the theory has to come from somewhere, and they fail to appreciate the issues that come up when dealing with uncertainty. It's like everyone in the field has some undergraduate-engineering-student level understanding of statistics, and don't have to deal with thorny data collection designs, complex inferences, or replicability of findings. The sort of scenario that's motivated "data science" is essentially this: a extremely large dataset involving relatively simple classification or prediction questions about observational data where there's really no scrutiny about generalizability or the meaning of the results. This problem scenario is why they got involved instead of a statistician in the first place: because the bottleneck was the size of the dataset, not the analysis scenario.
All of the attempts to distinguish "data science" from statistics it seems to me are based on stereotypes or misunderstandings about statistics, as you point out, or on extremely short-sighted perspectives on science and math. Computational statistics has been a core part of statistics for decades (there are journals devoted to the topic), and you can find peer-reviewed articles on all sorts of computational problems in statistics (e.g., the use of GPUs in estimation problems, how to approach optimization with distributed processors, etc.). The idea that statistics is all theory, and that statisticians don't understand computational issues is naive or has a very stereotyped view of statistics (or I pity their experiences in high school and college--it sounds like they got a poor education in statistics).
This isn't to pooh-pooh the contributions of CS--it's critical. But I hate the banner of "data science"--not only is the term stupid and redundant (how can you have science without data? What other kind of science is there?), it's based on ignorant stereotypes about statistics as a field.
To me, this speaks to a longer term problem in CS, which is CS essentially discovering what's been going on in other fields and reinventing the wheel over and over again. I don't see this necessarily in CS academic departments, but I do see it where there's some interface with the business world. It's coming up now with statistics, it's come up before with social sciences and economics, it's come up with AI and neuroscience, it's come up with genomics, it comes up over and over again. It speaks to a sort of arrogance or autism in the field's culture, where they act as if their unawareness of a phenomenon means that no one has ever researched it before.
Ughh... think about statistics as the mathematics of uncertainty, and see how far you get with deemphasizing that. Damn, I hate society sometimes. I need a walk.
Re:Statistical Practitioners need to Modernize (1)
Anonymous Coward | about three weeks ago | (#47766893)
Would mod parent up if I could.
I think part of the issue is that people in software are often expected to understand and implement (code) difficult concepts in fields well outside of their domain of knowledge, simply because they are experts at coding. For example, an academic in bioinformatics explained to me that he spent months correcting code in bioinformatics software that made the most basic of errors in genetics, such as reading a strand of DNA backward. I've seen similar issues in government, where the developers either fail to understand the policies they are coding around, or willfully ignore them because they "know a better way". I wouldn't expect a programmer's understanding of some of the more difficult concepts in statistics to be much better. Part of the problem is that domain knowledge can be difficult to pick up without bungling it; another part is the hubris of some in software development. Full disclosure: this is coming from a former programmer turned scientist.
AP? (1)
Anonymous Coward | about three weeks ago | (#47764331)
What the fuck does "AP" mean?
I'm dabbling on the "AP Central" website and other but they all talk about AP courses, how to get a course labelled AP, "AP is your time well spent" but never a definition of what AP is. It's ridiculous to use such a two-letter initialism and hide its meaning like it's a secret thing for "consumers" who buy higher end education in the US.
Re:AP? (2)
LateArthurDent (1403947) | about three weeks ago | (#47764579)
It stands for "Advanced Placement." They're college-level high school courses. At the end of the year, you take the advanced placement exam, and depending on your scores and the college you attend, you can get college credits for them.
I think getting rid of an AP is a stupendously short-sighted idea. Having students take more advanced courses earlier is a great idea. If there's reason to believe the courses aren't actually as demanding as their college equivalent (and I don't think there is, based on my experience taking AP Calculus in high school and looking at what people taking Calculus in college were seeing. We covered the same material, and if anything my high school class covered more), then you can make an argument for the tests more challenging / add to the requirements of those courses. Getting rid of it is just an attempt to waste students' time and extract more money from them by forcing them to take more university courses.
Re:AP? (2, Interesting)
Anonymous Coward | about three weeks ago | (#47765417)
Getting rid of it is just an attempt to waste students' time and extract more money from them by forcing them to take more university courses.
I suspect his complaint is that in high school, AP Statistics is taught by math teachers. In college, classes are taught by professors who specialize in statistics. This goes along with his general complaint that people in other disciplines don't take the time to really understand how statistics work. Of course, the same problem exists in college statistics courses. You can take a one semester survey course or the two semester theory course. He'd prefer that everyone took the two semester course and that it was rigorously graded.
He may be right about AP Statistics though. Taking statistics in high school means that most people will have forgotten it by the time they get to advanced courses that use statistical methods. This leads to students learning statistics from the professors in those advanced courses (who are not focused on statistical rigor). Statistics is a sophomore/junior level class, where most other AP classes substitute for freshman classes.
I would tend to agree with you about the other AP classes though. There's no such thing as a "calculus professor" -- calculus is taught by a mathematics professor who is likely interested in something very different. It doesn't make much difference whether it is taught in a small high school class or a large college lecture.
Re:AP? (-1)
Anonymous Coward | about three weeks ago | (#47765979)
I wonder about, instead of giving college credit, if someone is bright enough to pass the courses in HS here in the US, why not fund their college?
A friend's two kids from Germany have their whole college career funded by the Fatherland.
A colleague of mine from China had his degree and foreign visa paid for by his country.
A former classmate from Chile had his degree paid for by his government.
Here in the US, public universities pay for room and board for foreign students anyway... why not US citizens?
Re:AP? (-1)
Anonymous Coward | about three weeks ago | (#47766395)
We had that in the US... right up until the 1980s. Once the baby boomers had all graduated and got into the workforce, they took a look at the tax rates that had paid for their college, said fuck that, and pulled the ladder up behind them. Kids nowadays either have to get in debt up to their eyeballs, or hope mommy and daddy are loaded.
The Greatest generation and every one preceding it understood the societal importance of paying things forward, and leaving your children and grandchildren more opportunities than you had. The Boomers went and turned that around. The final joke is going to be on them, though. When it comes time to retire, who are you going to sell your house and your stocks to? It won't be to the kids with $40k in student loan debt, especially when they can't even get jobs thanks to all our shit getting outsourced. Which, again, was thanks to the Boomers wanting cheap shit and damn the consequences.
All ran off to quant finance (0)
Anonymous Coward | about three weeks ago | (#47764341)
This is not a new phenomenon. Most stats grads get big checks from banks...
1984 machine (0)
Anonymous Coward | about three weeks ago | (#47764383)
You can make money from writing your own statistical software, but it's not as easy to make money as an analyst I think.
In the end, we can all get jobs sweeping the floors of the AI farms.
Because lies and statistics are interchangable (0)
Anonymous Coward | about three weeks ago | (#47764407)
If you get a degree in law, business management, or marketing you can now put statistics as an academic minor on your resume.
CS is innocent... (2)
Buchenskjoll (762354) | about three weeks ago | (#47764467)
If They're so Smart, Then why is this Problem? (1)
dcw3 (649211) | about three weeks ago | (#47764501)
Efforts to make the field attractive to students have largely been unsuccessful."
You would think they would know which efforts work and which don't. I'm only being a bit sarcastic with the Subject line, but seriously they should be able to figure out what does and doesn't work.
This explains why statistics invades CS (0)
Anonymous Coward | about three weeks ago | (#47764507)
While statisticians complain, I have to face the quite opposite situation of having my research field swamped by people doing statistical methods for everything. And in some cases this is not reasonable. Of course, many system designers dream of just throwing pieces of code together and still being able to prove that it works (with a certain probability), but getting meaningful probabilities still requires that certain independence hypotheses are respected, and in computers true stochastic independence is difficult to obtain...
Demographic of the Current Statistician (1)
Anonymous Coward | about three weeks ago | (#47764509)
I don't dare publish this unless I am anonymous, but I must state this observation:
We are always on the lookout for new statisticians in our medical group. About 95% of our applicants are Chinese females! I had asked one of our (Chinese) scientists about this, and he said that this is because of the proliferation of MS in statistics programs that are amenable to spouses who were interested in a profession that could be attained (and makes good bacon) while their husbands were working on advanced degrees in other fields.
There are eternal complaints about how some fields are full of old white men, and this tends to squeeze out others who do not fit this profile. As a result, those fields tend to lack innovation in thought and culture.
I believe it's possible that statistics is suffering a similar issue. Perhaps the field has become homogenized, and people just aren't interested in overcoming these cultural barriers. No one wants to be the minority, e.g. no one wants to be the lone woman in the field of a bunch of IT dudes. The opposite side of the same coin is that no one wants to be the lone white man in a room full of Chinese women. (Let's keep it clean.)
Every field needs diversity, which in turn fosters content and innovation. Statistics is no exception, and the field really needs to do something about this. I think their hands are tied, as fixing this problem would be so politically incorrect, that no one would dare.
Re:Demographic of the Current Statistician (1)
ColdWetDog (752185) | about three weeks ago | (#47765169)
The opposite side of the same coin is that no one wants to be the lone white man in a room full of Chinese women.
What is the pretest probability of this being true?
Odor of Corruption (1)
drinkypoo (153816) | about three weeks ago | (#47764641)
Statistics is a dirty word today, even though modern science depends upon it. The public most commonly encounters them when they are lied about.
Re:Odor of Corruption (1)
Xtifr (1323) | about three weeks ago | (#47766513)
Indeed, which is why, when I talk to kids about math in school, one of the things I like to point out is that while statistics are, in general, rather boring, it's really important to learn enough to have at least a chance of recognizing when they're being used to lie to you. This argument gets through to a suprising number of them.
Of course it's shrinking (2)
dywolf (2673597) | about three weeks ago | (#47764647)
As far as the general public is concerned:
When it's convenient, people use numbers, real or made up, in order to disprove the other sides point and prove their own...
When it's not convenient, all statistics become questionable ("ya, but msot statistics are made up") in order to disprove the other sides point and prove their own...
The reality of the numbers don't matter. People just don't care about actual objective facts, they just want to back up their preconcieved notions to spread their stupidity. It's just like how Americans approach science in general really.
Hard to do right, easy to not notice you're wrong (4, Insightful)
DoofusOfDeath (636671) | about three weeks ago | (#47764673)
I'm not very trained in statistics, but I've read more than my fair share of academic computer science papers over the years.
Even with my limited training in statistics, I've known enough to be appalled by the errant statistical reasoning used. Or even not used. I.e., "We don't know how many times to run a program to get a 'valid' average running time, so we ran it three times. Here's the average: ..." The authors seemingly aren't just ignorant of how to get the answer; they often seem to have not thought through what questions they're trying to answer in the first place with their measurements and resulting statistics.
I think a few problems come into play here:
Despite CS majors thinking we're so smart about mathematical issues, I think this might be one area where that confidence is delusional. I suspect most psychology majors who paid attention in their Experimental Design courses are more capable in the appropriate mathematics than are most CS majors.
Re:Hard to do right, easy to not notice you're wro (0)
Anonymous Coward | about three weeks ago | (#47765791)
How can you compare a kid running a program three times to obtain a mean to the calculus required for even the most trivial statistical problems?
Re:Hard to do right, easy to not notice you're wro (1)
Rich0 (548339) | about three weeks ago | (#47768127)
How can you compare a kid running a program three times to obtain a mean to the calculus required for even the most trivial statistical problems?
That was his whole point. The fact that many people think that the one is a substitute for the other indicates that there is a big problem in CS.
That said, trivial statistical problems usually don't require calculus to solve. I fully appreciate that commonly used statistical functions are rooted in calculus, and you need to understand it at some level to apply them properly. However, the mechanics of solving the problems usually do not require solving integrals/etc. To use a car analogy - I can use a speedometer without knowing what a derivative is.
Statistics? (1)
Rambo Tribble (1273454) | about three weeks ago | (#47764701)
Not surprised (0)
BCW2 (168187) | about three weeks ago | (#47764711)
No wonder there are lies, damn lies, and statistics!
Re:Not surprised (3, Insightful)
u38cg (607297) | about three weeks ago | (#47765121)
You missed the point of the lesson. The point was that you didn't have enough data to demonstrate that your model was valid. That's all.
Re:Not surprised (3, Insightful)
ColdWetDog (752185) | about three weeks ago | (#47765183)
Take one set of data and produce two diametrically opposed answers and have them both correct? Sounds like rumor, gossip, and BS to me, not science.
No wonder there are lies, damn lies, and statistics!
Somebody missed the lecture on assumptions.
There are three types of lies... (1)
Xiver (13712) | about three weeks ago | (#47764991)
http://en.wikipedia.org/wiki/L... [wikipedia.org]
Actually there are five types of lies (0)
Anonymous Coward | about three weeks ago | (#47766069)
In rank order:
lies damn lies statistics benchmarks {hated corporation} benchmarks
Difficult to determine what TFA is about (1)
dtjohnson (102237) | about three weeks ago | (#47765099)
So? (0)
Anonymous Coward | about three weeks ago | (#47765269)
So who cares?
When the number of people doing statistics gets low enough, salaries will go up and more people will start studying them.
There's also H1Bs to solve the problem.
The number of Liberal Arts majors is declining as well; if that happens where will Starbucks get there baristas?
Lies (0)
Anonymous Coward | about three weeks ago | (#47765299)
There four types of lies:
Lies, Damned Lies, Statistics and CS Simulations.
Stats in c++ STL (0)
Anonymous Coward | about three weeks ago | (#47765551)
Participate in the c++ meetings and code up STL stats facilities make gets into next c++ standard.
Stats is hard, let's go coding (1)
RWerp (798951) | about three weeks ago | (#47766231)
Finding work in statistics (1)
tomhath (637240) | about three weeks ago | (#47766451)
I work with a couple of very good statisticians. What they do is a mystery to me, but one thing I can say for sure - a good programmer or DBA will find work much more easily than a good statistician. In large part because PHBs have no clue why they need someone with more than two semesters of probability in almost every application.
Another problem with students going into statistics in the US is that virtually all of the instructors don't speak very good English. To this day I want to say things like "probabirity", "rotatation about the ashes", and the one that confused everyone in the class - "ashama" (eventually translated to axiom).
Re:Finding work in statistics (1)
Zero__Kelvin (151819) | about three weeks ago | (#47766859)
Did you know that, statistically speaking, 100% of people who "don't speak very good English." don't speak English very well?
Re:Finding work in statistics (0)
Anonymous Coward | about three weeks ago | (#47767125)
"Good" in front of "English" is an adjective and perfectly acceptable. If it were behind, and thus an adverb on "speak", then you'd have a problem.
Re:Finding work in statistics (1)
Zero__Kelvin (151819) | about three weeks ago | (#47767969)
I'm going to do the unthinkable and use a programming analogy circa 2014 on Slashdot:
if (x=1) { printf("God, like your English, is good! (though I don't program very well)"); }
Get it? It's semantically correct, but it doesn't "say" what you think it does
Have you ever seen "happy" statistics? (0)
Anonymous Coward | about three weeks ago | (#47766457)
No, neither have I. They are always dreary and depressing. Nobody wants to be a statistician.
Consider being a subset of SC (1)
Karmashock (2415832) | about three weeks ago | (#47766727)
Statistics as a subset of CS isn't unreasonable given that nearly all statistics will be calculated by software.
Re:Consider being a subset of SC (1)
the eric conspiracy (20178) | about three weeks ago | (#47769005)
That's complete horseshit (along with this article). It's like saying math is a subset of CS because nearly all maths will be calculated by CS.
Stats is orthogonal to CS. You don't need one to do the other.
Having both though can give you a skill set that's quite useful.
Re:Consider being a subset of SC (1)
Karmashock (2415832) | about three weeks ago | (#47769421)
That's fine. Just throwing out an idea. Perhaps keep statistics as it is... but add a CS statistics course so that the CS students get a better background in it.
Not trying to upset apple carts here. Just trying to find reasonable solutions. :)
Who is this guy? (0)
Anonymous Coward | about three weeks ago | (#47767289)
Looking at his publications page, he hasn't published anything of note in years: http://heather.cs.ucdavis.edu/matloff/public_html/vita.html
Reading the blog post, it seems that he is woefully ignorant of the state of modern machine learning research.
Never mind the actual arguments he makes (which others addressed), I'm not really sure this guy has much authority on this matter, either in CS *or* in statistics.
I apply stats everyday (0)
Anonymous Coward | about three weeks ago | (#47767479)
The only stats I need to understand and to apply is what is the probability of me winning the powerball lottery? how many tickets can I afford to buy per attempt? and knowing that there is a high probability that gov will show up to collect a high amount of the winnings....
Statistics is awesome (1)
HalfFlat (121672) | about three weeks ago | (#47767989)
It's a shame it has such a reputation for being boring, and it is a shame that it seems to be rarely taught in an engaging way.
Statistics is the first artificial intelligence. It formalises what we know when we 'know'. It is fundamental.
It's also fairly hard to do right. But many worthwhile things are hard.
worms eye view (1)
micahraleigh (2600457) | about three weeks ago | (#47767999)
The stats people I know are really, really rich. And there are a lot of them.
That's in Raleigh.
Better Long-Term Prospects than IT (1)
Tablizer (95088) | about three weeks ago | (#47768025)
If you like the field of statistics it seems a better long-term bet than IT. The "laws" of math are not going to change in 40 years, where-as in IT the languages, GUI's, frameworks, and Paradigm Fad of the Day will change...several times. Plus it won't give you Carpel Tunnel (unless you can't trick a grunt into data entry). You are expected to know the domain (industry) such that outsourcing is not as likely either.
Software may pay more in the short term, but career-wise, stats seems more stable.