×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Email Offline At the Home of Sendmail

timothy posted more than 2 years ago | from the snowballs-happen dept.

Communications 179

BobJacobsen writes "The UC Berkeley email system has been either offline, or only providing limited access, for more than a week. How can the place where sendmail originated fall so far? The campus CIO gave an internal seminar (video, slides) where he discussed the incident, the response, and some of the history. Briefly, the growth of email clients was going to overwhelm the system eventually, but the crisis was advanced when a disk failure required a restart after some time offline. Not discussed is the long series of failures to identify and implement the replacement system (1, 2, 3, 4). Like the New York City Dept. of Education problem discussed yesterday, this is a failure of planning and management being discussed as a problem with (inflexible) technology. How can IT people solve things like this?"

cancel ×
This is a preview of your comment

No Comment Title Entered

Anonymous Coward 1 minute ago

No Comment Entered

179 comments

Hate Being First .... (-1)

Anonymous Coward | more than 2 years ago | (#38259926)

I hate being first.

Re:Hate Being First .... (0)

Jeremiah Cornelius (137) | more than 2 years ago | (#38260128)

It's SO HARD to point your MX record to a working host!

And then, to populate an IMAP from your directory? Don't get me started...

Re:Hate Being First .... (1)

Anonymous Coward | more than 2 years ago | (#38260310)

Considering that the outage meant that IMAP and POP couldn't be used, while webmail could, I'm not sure that an MX record change would've helped much. :)

I still wanna know why they didn't just change the load balancer to take the bad server out of the pool (which could just as easily be a DNS round robin entry as anything). What? They didn't have redundancy built into the protocol which, short of DNS, is probably the easiest one to make redundant? Then remind me to consider "UC Berkely IT" as a back mark on future resumes when people apply to work here.

Nothing to do with Sendmail (3, Insightful)

bobstreo (1320787) | more than 2 years ago | (#38259942)

It's the backend. When you have too many connections on too few servers, with not enough storage
you usually see this kinda issue.

Re:Nothing to do with Sendmail (2)

grcumb (781340) | more than 2 years ago | (#38260088)

It's the backend. When you have too many connections on too few servers, with not enough storage you usually see this kinda issue.

I see it as yet another failure for the client/single server model.

It surprises me that people are still investing so much time and effort on centralisation of services when obviously the most practical technical[*] answer is the opposite. Simple, common protocols and decentralised infrastructure are the most robust model for overall survival of a communications system. DARPA proved that some time ago, but we seem intent on forgetting as much of that lesson as possible.

----------------
[*] Okay, I don't want to be disingenuous about this. The reasons for centralisation are financial and organisational. It's more costly to spread IT capacity through the breadth of an organisation, and it's hell on wheels in administrative terms. But past a certain point, you would think that IT would finally earn the right to have some input into the discussion about how best to manage an organisation's information. Unfortunately, IT managers are not always the best ones to advocate for a different approach because they're the ones who've made their mark by proving (or pretending) they could manage these big, ugly 'enterprise' systems.

Re:Nothing to do with Sendmail (3, Informative)

TheRaven64 (641858) | more than 2 years ago | (#38260470)

Uh, email is decentralised. Anyone can set up a node in the network just by pointing an MX record at a machine. The problem in this case is too many people using the same node. You'll note that while UCB was having problems, email continued to work fine for everyone else unless they had unrelated problems.

Re:Nothing to do with Sendmail (4, Funny)

vlm (69642) | more than 2 years ago | (#38260164)

It's the backend. When you have too many connections on too few servers, with not enough storage
you usually see this kinda issue.

Knowing the speed and flexibility of university upgrade policies, and knowing sendmail was born around 4.1BSD, and knowing the -BSDs were VAX only until 4.2 or 4.3 or so in the 80s, I'm guessing they're still using the original VAX it was developed on?

Re:Nothing to do with Sendmail (0)

Anonymous Coward | more than 2 years ago | (#38260246)

Congratulations on reading the summary.

No-one was trying to blame Sendmail.

Re:Nothing to do with Sendmail (1)

tramp (68773) | more than 2 years ago | (#38260334)

You are right that it is not sendmail causing the problem but it is quite sad that UC Berkeley has fallen to such mismanagement of their IT resources. Somehow you expect such a institute to do better then this.

Re:Nothing to do with Sendmail (-1)

Anonymous Coward | more than 2 years ago | (#38260462)

Now you can repair any job big or small, thanks to Mighty Putty. Mighty Putty is great for anything from your smallest craft projects to your largest home improvements. The easy way to fix, fill and seal almost any surface, and make it last. Mighty Putty is a powerful bonding epoxy stick that you can mold to any shape and can apply to any surface for an everlasting bond. Simply cut, activate and apply then all you do is let It dry.

hdczp jzcae mxbcx zzsew bzvig pdsvn aeukn hzeye kmrsg nkigx sjoxf yeqzf ezwjj ktoaa duxpb fzpqc emfwq hlilt xvbkb ygmpy xrrqw uqxzr wholp kuqnc dbcta fadby ykouf znmwp fwhpa scvln hvvuv zaxbj rqifz wtgxu bbjhb bznyg irhzm genew itmdr sxohc fqhoq izgfa idkmp gtubt pgpqw omcxb cqeaq kufuk canec lerwk ofkfv vtpya kyyew omnht liojm ujfho ghglk gmorq vqies ppgeu crxug yivmp ebuve doyma uhzsu ijtin xfekz dqlmd ubfqc orrwy cdocq kxbxl coujp hpbrg tzuex fevwr qckid nooex tepsi kxwaq nlwep spxji uysxq aobuf vhnsm lcxpg tuufb wgnzz hypti fftsy ihptp ghekh ugfvq hbgxd fwjag bvele yogos kloft rjjdc bkbkt qhdpj swlak fuqqa yatvr pvrcg dthnl rmxts njris giehc kbbls pkdtg pwast ihkpd pscrz vlkdu yycrr vtgqy ocows typcu lojub jdeco lfjfo ycvnk kgfzs jpzfb qekey swxsr tiqqj khebz ntkxe coarj fbffm ifrgx wbfce wgctj kkbbx phsqp dfttz tubig eifwy rnafi retks pqxpb nthgv kmslc afauc sriur dywwl korex ujqdm sewni qwfuv izlzw dbiif rxddr fxqlz mnsjl incwz fczyq cyerb fvkzf gvslx czkdb hzuoo fpjnh uilyn arane vulwn mgdew yaocb gsoxv iheep nxsoe ootlj mywop hlhom rovha zzmjw ojehr xxqxm nogcf mvnmu sbxvo mtbqk bockg gtcbc wuoio svele qthgc blmho sxfso oxulz fkgyv gievm bfrcv udqdh oldgg srhue aqksb bqndt xxjrm fqsyx czvzf pxgmg ietlm exzfr ndtnr qjohq jjdzg oxryc ygaml dgtkb iprfh mlzla hadaa pfsun lwmnf ihina fqaxf bxdez stavi xlahr vudff zwqwd ptxfl hjhsk nsrjv erygk iuucx iyyuy khavl pfjfk pbsua alqrb xcuqo rumag abiej orlup jtawh pufti ipbki zycyz ublod lnccy epvih gslmn ostsq lpdqs fisqt yhode cvzrb hcvuc tnxhm mzcju muppt vmpsu ddcpm ljggx xsxii tlsio wovie foaqj cnxag ygamg qdqrl zrhxo jwimm vgqri bhvrf cersa jplqs fqkyb wzmrx cllpt canka fwscl ejkev idosh jgnnn psehm fbvft ytkgz bhbts ajinc cqbye osibs oxeni qzyru hhstk uschw teyck nfnzh huojq jdcmt uuqpw ylicf pgxmv yehkp csigh eiicr mzjgd fujqt ujrdi qxbpg omukz csxex kkhwl plshd yyhfx cbycf veelh memeu bdiiu levzh pndcn xqkkl ptoar wwopt ewynt pbmud daxst cibra wdtiy jnpzb lziwk biuci losuo ruawa mwjty yuixz fyqkp wcvtv hyrwm dlpau vjbnk hzhzs ovaia ktrqm cqnrx vimkc ogary twfpi cetuf ybpfy tgtnz ocrfs dgmlh rltyr ddjxo axbpg teral lbqav trjjr zpfch dftww mmawp axccr whviv itwky ghyjd kiphm cuwio vloda lbpni gohgq dpuaw duxta qwngu bgedu wlser cenjk vpytq xvfcj lhgwn gihzb rqpbk ulwkc xnoai ldedd whmqf onqqu wkpic uhtkr ulzew vevrr bqjpu ciszr gfgjm vihex cxlth bvfeq ccrnx sxffe gregc ehhkm tatig rsupe aaivh kbtwt orfwo vqzpb crskp afhqc apirq wvwta rvvwj spxca hpmsd ifqbb vxweo arcsr xtrou mfcnw xeiau ogbhz gmbar hcywp spedn jvbte wayob kkxct hqbou wlcao mogcx pzabo idhpf leymh xazed sbphb nghbz mlckw amqoc kqqwt idiok dbaoe jqzzx lmftc fyibd qntes dkkje wzwho igbbg xjpdp ylpwu vozbf ewlkn chkay jbgqm sceuw rpmrp qukql vgysn baroc cnhta memva gvixl kmsqn jiupq qmoqt mtngf qkqbk kahpw iyfdl pgdvj cfefk cxvbs cluqz ppnvc jljzk xstpk pdsar auoii ukulb anhnf ulzwc jpquj fkufy fgkrq sxvrf lxbaj ilcqw fhcyx nlkid fhpcf rksgg igypm ttoee smirt xrpon pjebu rzmzb euppt qsrhc odwme pliyj ujixa motro tsrez jcjpj bhjri rsyqs wodaj ubaja gtumb mnfev jchos fiutc noxka wcvwu iocnk kgrem pasaf udbdx jklgo gcuxy joddf idhnm ismfj wgnra vqftv aaxhw szapo vvzri jrynq rsnky jhvdn jabes zywnr gwfhv mbrcn mvjrg eodoz vuztg apwvz bedzj eogsz sonqm awhnl drsje jralk nnjkj dhsou rhdhf hjrka xuunw pyxuc wbodi xerwh tgauv ntvpp tpqch xdapw nlwoc rscqf khckt grrwm pmtae jrcyv vrxzf krdij wxfjx iwbfn boxdq uwtpk trcbs tossw mbbfv qdgxe ymfao ugjeq rskqd wmxii umstk jukwr pxarf atwfk thncu mybtl lzqfp ypznx htyya fbarx pqsjt cwewb ptnvz kytbx poszk aejas ttorr egowq vdizg lanfe bgefx xieqk jtzyq jiowm stzhm yystv fepws wrnsb ejuap mocto cnwns edapc vhznv suitj ftxjs bzslp zegbc snlhp viqlz llymo vgiit vlqum ywmfs kuomh qjkcn wmsqv qyyhx kgiow krwkd noinq yzrlv dhrfd dguwy twevi xfnxw nmojv michv aflyi hvjdv nnqdh znkxa goiea hzuek hyyjz wtlmh jlfde yneyl thtxj muhgj quhbd

Re:Nothing to do with Sendmail (-1)

Anonymous Coward | more than 2 years ago | (#38260474)

hjyok ywzkn lxghh bfzkp nclhx arfdh litxd skykj iauom offsj wxpcw vwihn kdufc ahgln qxqce noxvd psprb zyhir wxnwq onqdm bvmvt dgnab vfjox zofqi aoieu biykz mbaer jltrm pkpad zckil agzqm porsj vwwww nyxrk rowbe nbxwq tscuj umdxc rksrh nyxhw mmyqj suyqk mponk qiuaz ajkjq cfpij khccs xpijj untly mqopt mhfri zlywa nxucp sarsq khlzi agqql xpgcv ewaxa skxzk ihojq ezgpi qtusw nwuqp txczl jribw yivzq pdpof pupvl nsewt srgis ipccg yqxez zrvte vjqhc ggagt uolmd edixp gmadx tyosy jeajo vlsoj exprp qglbx lfzww fblxt ldcjd orqyo ihntj kvviq qxbdc wkvhf qdhpt jbiag rwead gwnux gctms iepyf fhlhl ykpyu otwlo trcqa gduit vxcef rdyaf wedmz wnbhy idbzn fkrov nzhpp nfarr usqll mktxx utips gmoow sijwm bwoln cgnhh yzuzk fdcvy lvadu aaqdi gxiuu yzavy opxba cznbc gqhyy kfywd jwvnq pzkrx atsxs gzxsh uelce sbbiz hdtax yekal ypkjr ulfuc orffj zmtdu qzyjs komgg hpaju itdqk gxqxb jzrpk ptbbj mwdst laghd wiynd neogj sklwq ikica xaevj tnfad brwkb wwgdz hhlfj dpecv aulbr fqgkp nbbzq bzlat sfbvz myukd luvrm riuab jjqhz zzoyb qgcpp nfaxu voyto bantd aoefm vxfke ptprc uakqf ddolx akbhq yfzaa hohrk mjvqw vsrbp fnclz tczya rztpu qrair ktqys ofcpl vfnju ikdmj zudiq lmine gzfxv vcjfh yytoz ioshy feioq cngpa hbtpj xauva ksnlu ksooq dibke seocm ljraq cipil snuqb ixvgm zaets ngfpm gmqfg kvioe uuqhl rviwx pncyr nupwx gtwzb wymdq bzjpt vnyly ewvmb yeavi kjekp jwggp loooz jcmmz zxgrl viyjo nuzuh uhmwk fabnq yyhip kdigz mllbw rbfqj ecdfs eigjm uljss xuves fknhi

Re:Nothing to do with Sendmail (-1)

Anonymous Coward | more than 2 years ago | (#38260494)

xavrw sgzjh bebwm yzagv qngrr eqwsl izzlt qzpnp kqpfy egvdp qjkfm njrwp mlotg psouo bnejd dwauk wmthc uzmhy txddf kveuw bzrlz zxhbt ejghs uefsz ppasq fddzo mjpnd ufzqf tcukk oopfv gdaoh qmxzp xymko yxknw shiqn vnkch cjqte alpbz wmecc tkelp bmigr esvid illxy jxyuu xsjbc lkjwi kebhw ktpdl tixdj gkobo tsopm cqyav ihuds hrvlw dnsjw aenup jjrpp qabfw zldif gdxbv sgezg ssbtx vgscu ytmil ecvju cudwg lqxvq icsse wvfyo

It isn't an I.T. problem (2, Insightful)

Colin Smith (2679) | more than 2 years ago | (#38259948)

It's an economic one. It needs an economic solution.

e.g.
Have people buy a $10 ticket to get an account on the email server.

Re:It isn't an I.T. problem (5, Insightful)

StikyPad (445176) | more than 2 years ago | (#38260056)

Pretty sure that's what tuition is.

Re:It isn't an I.T. problem (4, Funny)

Anonymous Coward | more than 2 years ago | (#38260256)

no I'm pretty sure tuition is more than $10

No, tuition is for education (1)

Colin Smith (2679) | more than 2 years ago | (#38261060)

Clearly email is an afterthought thrown in for free.

If you want a service to work, you have to fund it. You can try to fight for budgets against the football team or you can simply charge and the money automatically goes where it's needed.

Think of money as little packets of information. You buy something there is a need for it, you don't buy it, there is no need. Resource allocation without dozens of layers of management.

Maybe nobody cares about email and they can just shut it down. Charge for it and find out.

Re:No, tuition is for education (1)

icebraining (1313345) | more than 2 years ago | (#38261398)

Maybe they should offer redirection for free and paid storage. I find it useful to have an email address within my college domain, but I redirect everything to my main account.

IT has to deal with budgets, too (2)

msobkow (48369) | more than 2 years ago | (#38261654)

I hate it when people try to act as if IT isn't subject to budget constraints and having to prioritize spending like any other department of a large organization. Sure the money comes out of the "client" departments, but it's an issue that IT does have to plan for and deal with.

The summary asks "How can IT people solve things like this?"

Forward the emails and responses to the demands for planned capacity growth to the public.

Oh, you didn't keep the email from your manager refusing to pay for a needed capacity upgrade? I guess you haven't been in IT long enough to learn to cover your own butt.

So the ultimate solution will be outsourcing (1)

HotNeedleOfInquiry (598897) | more than 2 years ago | (#38259956)

I am depressed.

Re:So the ultimate solution will be outsourcing (1)

betterunixthanunix (980855) | more than 2 years ago | (#38260040)

Indeed, that is exactly what happened at my alma mater. First they blamed the Squirrelmail front end, then they bought a black box solution from Mirapoint, and when the Mirapoint solution proved too expensive they just went with Google.

Re:So the ultimate solution will be outsourcing (2, Funny)

Anonymous Coward | more than 2 years ago | (#38260090)

Wow, Squirrelmail. So at least they managed to migrate from pine at some point.

Carrier pigeons coming soon! (1, Funny)

syousef (465911) | more than 2 years ago | (#38261602)

Wow, Squirrelmail. So at least they managed to migrate from pine at some point.

Yeah, they're planning the upgrade from squirrel to carrier pigeon as we speak!

Re:So the ultimate solution will be outsourcing (2)

twisted_pare (1714106) | more than 2 years ago | (#38260708)

This is really quite common. It happened at my alma mater as well. Servers could not handle the POP requests, so they started blacklisting students that checked their mail more than four times an hour. A month later a RAID drive failed and email for 17k people (including a hospital) was completely offline for 3 days. It is sad that seemingly anyone can be a high paid "IT Professional" these days, but without a clue about HA.

Re:So the ultimate solution will be outsourcing (4, Insightful)

lucifuge31337 (529072) | more than 2 years ago | (#38261140)

One can have all the clue in the world, yet be powerless to prevent failures if not funded to purchase the appropriate equipment.

Re:So the ultimate solution will be outsourcing (2)

lucm (889690) | more than 2 years ago | (#38260484)

Outsourcing would work, because when there is another failure they will have another party to blame instead of pointing fingers to a decision made in Spring 2011 (even as a total stranger I could feel the bitterness under that bullet point in the slides).

downtime (0, Troll)

masternerdguy (2468142) | more than 2 years ago | (#38259966)

Oh no, a service had downtime. Surely this is the end of the world and only the greatest sinners of the IT world ever have to bring something down for maintenance.

Re:downtime (1)

Anonymous Coward | more than 2 years ago | (#38260158)

Well, "bringing it down for a week to handle a problem with a failed disk" is somewhat indicative of lack-of-clue.

Re:downtime (1)

gVibe (997166) | more than 2 years ago | (#38260796)

Well, "bringing it down for a week to handle a problem with a failed disk" is somewhat indicative of lack-of-clue.

Ha! Lack-Of-Clue sounds like a Whack-a-Mole game ... /me ponders!

I can tell you how (0)

Billly Gates (198444) | more than 2 years ago | (#38259972)

By hiring more cost accountants and requiring special and complicated business case studies with a thorough financial analysis on even the most mundane upgrade on how it will raise the companys stock price. Just ask any visionary MBA? Always buy cheap consumer grade stuff and view talent as unneccesary expenses. Do that and you will never have problems. What could go wrong?

Telnet (2)

qualityassurancedept (2469696) | more than 2 years ago | (#38259990)

When I started college in 1991 I was amazed by the telnet access I had to the email account given to me by the University. I hadn't had an email address prior to that. Now I have an email addresses through hotmail, gmail and yahoo that I use for different things and facebook also gives me an email address. So, I doubt students really need email addresses provided by the university anymore. As for the NYC Dept of Ed example, I think it just shows that trying to build IT competence into a government agency basically a waste of money because the institutional culture of government. In short, all of these kinds of organizations could just offer email through gmail/google business or any number of other providers that will scale up almost infinitely.

Re:Telnet (5, Insightful)

slimjim8094 (941042) | more than 2 years ago | (#38260080)

Students need school email addresses because that way all students have an email address.

At my school, students are expected to check their university email at least once every 24 hours. Many people forward it to a personal account, and obviously most people check it more frequently than that, but if the university issues an account to everyone, then there can be no debate about how they didn't get the email. The school takes responsibility for the email system (and any failures), and then professors can be assured that if they send an email out to the class, it will be (or should have been) read, leaving the onus on the student to actually do it. It's similar to why we provide computer labs - that way, each student unequivocally has a way to do electronic assignments, even if nearly everyone has their own machine.

Re:Telnet (1, Interesting)

betterunixthanunix (980855) | more than 2 years ago | (#38260096)

facebook also gives me an email address

When did this start happening? Does it actually interoperate with other email services?

Re:Telnet (1)

icebraining (1313345) | more than 2 years ago | (#38261454)

From news articles, it started a year ago, and it does interoperate with other email services (you get a [username]@facebook.com) but it doesn't let the sender choose the Subject line, add CCs, forward, etc.

Re:Telnet (1)

betterunixthanunix (980855) | more than 2 years ago | (#38261614)

What I really care about is being able to communicate with Facebook users without having to actually open a Facebook account. Thank you for the information, I will look into this in more detail.

No. (5, Insightful)

damn_registrars (1103043) | more than 2 years ago | (#38260122)

Now I have an email addresses through hotmail, gmail and yahoo that I use for different things and facebook also gives me an email address. So, I doubt students really need email addresses provided by the university anymore.

You are quite wrong. Email addresses - especially .edu addresses - are still quite valuable. At lot of academic resources that take registration via email won't allow registration to go to a throwaway account (a la hotmail, gmail, yahoo, etc). Many organizations that are interested in real information on users insist that users use an actual unique account and not a freebie. And when you're in college and making very little money a lot of those things can be important.

I think it just shows that trying to build IT competence into a government agency basically a waste of money because the institutional culture of government

You're not very accurate on that, either. Government organizations need to be able to keep track of their email - especially internal communications - which they would not be able to do if they outsourced email and other telecom.

In short, all of these kinds of organizations could just offer email through gmail/google business or any number of other providers that will scale up almost infinitely.

With the various privacy breeches that have occurred, that would be a terrible idea. And on top of that, IT is a lot more than just email. Do you want the government to turn to comcast for networking support while their at it? What if the IRS web servers go down on tax day? Do you want them to have to lean on an outside company to get it back up?

Re:No. (2)

qualityassurancedept (2469696) | more than 2 years ago | (#38260284)

you can get an email address through google with any domain name you want... so, the company I work for runs its email from google but we all still have email addresses that say mrbigshot@seriousbusiness.com so I don't think the point about the .edu ending of the address is really valid.

Re:No. (0)

Anonymous Coward | more than 2 years ago | (#38260574)

...but you won't get anyone to delegate a .edu record to you unless you're actually an educational institution.

Re:No. (0)

Anonymous Coward | more than 2 years ago | (#38260760)

you can get an email address through google with any domain name you want... so, the company I work for runs its email from google but we all still have email addresses that say mrbigshot@seriousbusiness.com so I don't think the point about the .edu ending of the address is really valid.

Some people think that Google shouldn't have access to their private life. Of course they're criminals and their opinion is wrong - if they weren't criminals, what would be the problem, right? We all know Google will never abuse this info, neither for profit nor to please a three letter government agency (who would have access to the emails anyway since SMTP is not encrypted and very few people encrypt their messages).

Re:No. (0)

Anonymous Coward | more than 2 years ago | (#38261158)

It would be better to have your email run by the regents of the university of california? How much pushback do you think they give the three-lettered agents?

Re:No. (1)

matty619 (630957) | more than 2 years ago | (#38261204)

In short, all of these kinds of organizations could just offer email through gmail/google business or any number of other providers that will scale up almost infinitely.

With the various privacy breeches that have occurred, that would be a terrible idea. And on top of that, IT is a lot more than just email. Do you want the government to turn to comcast for networking support while their at it? What if the IRS web servers go down on tax day? Do you want them to have to lean on an outside company to get it back up?

If you watched TFV...that is exactly what they're going to do. They're either going to go with Microsoft's 365, or Google's Gmail. They're just working out the contracts.

Re:Telnet (1)

mrmeval (662166) | more than 2 years ago | (#38260176)

It's safer if the school has no access to your email so they should just drop that service except for those students who provably don't have the money for it.

I'd never use their servers the same as I won't use my ISP's. I use ones I have some insulation from and if I need more privacy than that I use encryption and if it's life or liberty threatening a drop off to a hollow tree in some random park. ;)

Re:Telnet (1)

toomanyhandles (809578) | more than 2 years ago | (#38260316)

I like Tinker, Tailor, Soldier, Spy also. The Alec Guinness one. Although I have hopes for the new movie given those involved.

its also a privacy thing (0)

RobertLTux (260313) | more than 2 years ago | (#38260888)

Due to FEDERAL LAW communications from school staff to students (and the reverse) must go to University Email accounts.
plus if somebody "does not get" a given email then its the schools fault.

Re:Telnet (1)

stephanruby (542433) | more than 2 years ago | (#38260222)

Yes, but for most students, I'll bet that their official school email address is still their primary email address for all the important stuff.

Re:Telnet (3, Informative)

Mr. Freeman (933986) | more than 2 years ago | (#38260490)

Having a .edu address gains you a lot of credibility when communicating with people outside the university. They are quite valuable. You can often get very quick responses to questions that most companies won't even respond to if they came from a name@gmail.com or name@yahoo.com.

Also, email is used for a lot of very important stuff like sending reports, design files, etc. Having someone on campus that can fix problems is quite valuable. Your campus email will never be "accidentally" seized, locked out, etc. like people have experienced with google and yahoo. Because the campus maintains backups (or at least, they should), you data will never be suddenly gone with no chance for recovery like people have experienced with google and yahoo.

Re:Telnet (0)

Anonymous Coward | more than 2 years ago | (#38260606)

So, I doubt students really need email addresses provided by the university anymore.

I totally agree. I have my .edu forwarded to my gmail account and in 5 years of college I used my .edu only twice, both for getting deals online (Dropbox is one, forgot the other).

Re:Telnet (2)

mr100percent (57156) | more than 2 years ago | (#38261070)

Actually, most schools require an Official school email address. This guarantees the uptime from the faculty's point of view; you can't claim you never got the assignment or that you turned it in on time and nothing was there. It's also important for them from a liability standpoint; my Registrar will not send me any bills unless it's to my .edu account, and professors are instructed to ignore any student emails from any other domain. They're also organized by real name, so the school has a working internal directory and doesn't have to bother with LDAP.

lol (1)

luftrofl (1212770) | more than 2 years ago | (#38260002)

I know /. is a a little slow usually, but it's a little silly to see this article pop up now as full service has essentially been restored (just now getting back mail client access, while webmail was working for the past few days).

Move to Private Servers (-1)

Anonymous Coward | more than 2 years ago | (#38260004)

QED

A Free Hitchhiker's Guide Novella
http://thepiratebay.org/torrent/6848623/Perfect_Me_By_Jason_Z._Christie

Funding (1)

scapermoya (769847) | more than 2 years ago | (#38260008)

Maybe it has something to with the fact that the state of california has cannibalized the funding for my beloved alma mater.

Re:Funding (3, Insightful)

DesScorp (410532) | more than 2 years ago | (#38260202)

Maybe it has something to with the fact that the state of california has cannibalized the funding for my beloved alma mater.

They wouldn't have to if they didn't have too many colleges (they do), and try to send too many kids to college (they do), many of whom may have no business being in college (they don't). Tax revenue is not an infinite resource. But California seems to have a community college on every two dirt roads, and several 4 year (or higher) colleges in a similar area.

Re:Funding (2)

scapermoya (769847) | more than 2 years ago | (#38260286)

I agree that the overall system is probably too large, but we are talking specifically about the flagship university of the UC system. arguably the best public university in the world, and it is getting hurt just as bad as UC Riverside. that's absurd and embarrassing.

Re:Funding (1)

toomanyhandles (809578) | more than 2 years ago | (#38260336)

And lets not forget... they have the highest population count of anyplace in the USA also. Want to build a few more uni's in the Dakotas?

Re:Funding (0)

Anonymous Coward | more than 2 years ago | (#38260532)

California has a tech industry because of all those colleges. It's the fifth largest economy in the world because of that industry. But don't worry, the right wing dream of dismantling all of it forever is rapidly becoming a reality all so Grover Norquist can keep paying taxes at a lower rate than you or me.

Improper capacity planning (3, Informative)

mysidia (191772) | more than 2 years ago | (#38260022)

Briefly, the growth of email clients was going to overwhelm the system eventually, but the crisis was advanced when a disk failure required a restart after some time offline.

Capacity planning is supposed to account for reduced capacity due to component failures, system outages, and temporary demand spikes due to restart events.

Re:Improper capacity planning (2)

Bacon Bits (926911) | more than 2 years ago | (#38260118)

In my experience this type of "planning failure" is caused when IT repeatedly tells management they need money to maintain and upgrade systems, and management consistently says no because they don't have the money for it. Not enough money or people to configure, install, support, and maintain any new systems because the budget won't allow any more. Yet somehow there always seems to be money for shiny new iPads and iPhones for the executives.

Re:Improper capacity planning (1)

Anonymous Coward | more than 2 years ago | (#38261564)

At the place where my girlfriend works, she has a mailbox max of 20MiB. One or two emails with an attachment and its full. She asked why that was and their IT person said that is all they had room for on the server's disk for her mailbox because of all the other crap on that machine (website, internal files, etc.). This is the same company that bought brand new desktop computers, laptops (to work from home, which hasn't left the bag in and has stayed in the closet since she first got it) and tablets (for travel to meetings and the like) for literally everyone that works there because "that is what all the other companies are doing as productivity solutions."

Well... (0)

Anonymous Coward | more than 2 years ago | (#38260052)

It's called sendmail.

Not sendmailnomatterwhat

Scalability of Algorithms (0)

Anonymous Coward | more than 2 years ago | (#38260072)

There was some Silicon Valley ISP whose name unfortunately escapes me just now, that had the "problem" that its service had grown so popular that the time required to search for a mailbox in /var/spool/mail was greater than the time duration between incoming mails. The result was that their system worked great right up until a certain critical threshhold, then all of a sudden most of their users' mail started to bounce.

Their solution was to place user mail spools in their home directories rather than all in one directory, that being /var/spool/mail. Because the home directories weren't all in the same parent directory - that is, not all in /home - rather than a linear search, finding the right spool became a much quicker tree search.

If you have a large number of users, even if you have only one filesystem for home directories, you can speed access to individual user files by placing, say, my "mike" home directory in /home/m/mi/mike, rather than just /home/mike.

Re:Scalability of Algorithms (1)

KiloByte (825081) | more than 2 years ago | (#38260640)

That's only applicable if you're using FAT, sysvfs, ancient versions of NTFS or ext2 (before dir_index) and others of that age. Any modern filesystem can handle millions or more entries in a directory without going into the O(n^2) hole you're speaking of.

Re:Scalability of Algorithms (1)

thogard (43403) | more than 2 years ago | (#38261068)

The MAIL environment variable has been around since before the early 1980s and IDA sendmail had patches to look up the mailbox per user using dbm files in the late 1980s. I can't see this every being a problem with a mail server but I have seen it on usenet servers that don't expire some groups.

How IT people can solve this problem... (1)

damn_registrars (1103043) | more than 2 years ago | (#38260076)

IT people need to move into management at a more useful rate. Instead most of the people who ultimately make the financial decisions for IT centers around the world have little grounding in IT and hence limited understanding of what is actually important beyond the bottom line.

Of course, this requires IT people who are willing to put their foot down. We don't seem to have many of those...

Re:How IT people can solve this problem... (1)

quetwo (1203948) | more than 2 years ago | (#38261038)

Of course, the smart IT people are often not allowed into management -- they are too useful at their current level (keeping the systems running, turning the screws, etc), and would be a pain to be replaced. So they promote people who are easy to replace into management. For a smart IT person, usually the only way to get ahead is to move sideways, not up. Go somewhere else, or do something else... If you are good at what you do, there is little incentive for people to move you up.

Re:How IT people can solve this problem... (1)

Ziest (143204) | more than 2 years ago | (#38261610)

Unfortunately, those smart IT people who have spent years in the trenches and understand, in detail, how to build a robust and resistant infrastructure are often overruled by the a CFO who's only qualification is they have an MBA. In many tech companies the group that handles the infrastructure (DNS, email, backups, etc.) reports to the CFO not the CTO. Why is this? After 25 years in the computer field I still have not heard a rational explanation for this idiocy.

IT is not the Problem (4, Insightful)

arthurpaliden (939626) | more than 2 years ago | (#38260120)

IT goes to management and says "based on current usage/loadings etc the system will fail in 6 months to prevent it we need to do this....." Management says "Really, that's not what the sales man told me and its his equipment so he should know".

Re:IT is not the Problem (3, Informative)

iggymanz (596061) | more than 2 years ago | (#38261500)

no way, I work at a Value Added Reseller of hardware and the good sales guy would definitely use your fears to sell you some expandable solution

outsourcing? (1)

bcrowell (177657) | more than 2 years ago | (#38260154)

At the school where I teach, whenever there's a discussion of how much it costs us to run our own email, someone suggests outsourcing (e.g., to gmail), and then someone else says, "No, we can't do that because of privacy laws." Am I right in guessing that privacy laws don't in fact prevent outsourcing to google? I suspect the argument is basically a way for IT folks to have job security. There are certainly laws that say, e.g., that we can't give students' grades to third parties. But it's hard to believe that letting google keyword-index emails and serve ads based on the keywords would violate these laws. (Whether google creeps you out is a different issue -- a moral/political one, not a legal one. It may also be an issue, but it's not an issue that can automatically end the discussion the way the legal issue can.) Does anyone know of any colleges or universities that do outsource to google or someone else?

Re:outsourcing? (1)

Tridus (79566) | more than 2 years ago | (#38260230)

That depends on where you are. Privacy laws in Canada most certainly do restrict Government agencies (and probably educational ones) from using Gmail because it's American and the Patriot Act is a rather severe problem that can't be mitigated.

Re:outsourcing? (3, Insightful)

betterunixthanunix (980855) | more than 2 years ago | (#38260240)

Since both my alma mater and my current institution have migrated to Google, and both are covered by FERPA and other privacy laws, I am inclined to say that that argument is bogus. However, I have a separate issue with outsourcing student email: third parties get to set the rules for student conduct without any action by the university itself.

Typically universities have acceptable computer policies and at those institutions that run their own mail servers, such policies usually govern email. Students and faculty can demand changes to university policy if the policy does not properly align with the academic mission of the institution. Students and faculty have essentially no power over the terms of use that Google or Microsoft or any other third party email service imposes on them. It is easy to say, "Well, it is not like Google is going to demand something outrageous!" but there is really nothing preventing Google from doing so (if you do not think they have done so already). Google does not have the best interests of academia in mind when it sets its policies, nor is there any reason for Google to care about academic needs.

Re:outsourcing? (0)

Anonymous Coward | more than 2 years ago | (#38260398)

...nor is there any reason for Google to care about academic needs.

Really? If Google screwed it's academic clients, wouldn't that make it harder for Google to:

1) hire from those academic pools?
2) convince academics to use their other products (ie click AdWords ads)?

Re:outsourcing? (1)

swillden (191260) | more than 2 years ago | (#38261170)

nor is there any reason for Google to care about academic needs.

Sure there is: If they don't meet the need, they'll lose the customers to someone/something else who does.

Re:outsourcing? (1)

msobkow (48369) | more than 2 years ago | (#38261684)

Can you provide an example of a policy that would get the students and educators up in arms?

Can you provide an example where the terms of access to outsourced services are not set by the organization who set the terms in their contract rather than blindly accepting whatever the outsourcing company tabled as a "standard" contract?

I thought not. More FUD. "Please panic, people, because they might possibly maybe perhaps breach their contract and do something we don't like."

Pfffftttttt.

Re:outsourcing? (2)

GIL_Dude (850471) | more than 2 years ago | (#38260404)

Tons of schools use Google as their email provider. Here's a quote from a Time article from 2009:

Google now manages e-mail for more than 2,000 colleges and universities, enabling students to transform accounts capped at 100 mb into Google-managed inboxes that allow for 70 times as much mail. Microsoft also provides free Web-based mail for thousands of schools, including colleges in 86 countries.

Here's the article: http://www.time.com/time/business/article/0,8599,1915112,00.html [time.com] . Now, a specific school? Sure, my daughter and I just toured California State Sonoma and they use Google services.

Re:outsourcing? (1)

Mr. Freeman (933986) | more than 2 years ago | (#38260954)

That's a very difficult question. You need to sit down with your company's legal department and figure it out.

The answer depends on:
1) Where you are and thus what laws are applicable to you.
2) Who you are. Healthcare, university, private company? If you are a university, are you a public university? If so, there may be additional laws and regulations.
3) What's being emailed. Patient records, classified documents?

What's acceptable for people in similar situations may not be acceptable for you. I go to a university and Google handles our email, but again, this doesn't mean it's OK for you guys to do it. Set up a meeting with your legal department and they'll figure it out. Meeting with legal also ensures that you're not going to get thrown under the bus (at least not as quickly) if you do something and later get hit with a lawsuit.

Solution: Join the Google Collective (1)

Dachannien (617929) | more than 2 years ago | (#38260172)

Seriously, is Berkeley like the only college campus that hasn't outsourced their e-mail to Google yet?

Re:Solution: Join the Google Collective (1)

mactard (1223412) | more than 2 years ago | (#38261446)

UCLA and UC Irvine do their email in-house as well.. I'm sure it's just a University of California thing. So many smart people here, but so many dumb decisions are made every day.

Only 70000 accounts? (1)

__Paul__ (1570) | more than 2 years ago | (#38260208)

Only 70000 accounts? That's not a big system at all. I was running systems with over million email accounts ten years ago, and by today's standards even those would be considered small.

Re:Only 70000 accounts? (0)

Anonymous Coward | more than 2 years ago | (#38260332)

10 years ago, most email would have still been text based, not html, with signatures etc.. so volume/data wise its probably a lot more processing.

Re:Only 70000 accounts? (1)

Nimey (114278) | more than 2 years ago | (#38260358)

Not to mention not having lusers send 100MB attachments in their email to multiple recipients.

Re:Only 70000 accounts? (1)

__Paul__ (1570) | more than 2 years ago | (#38260618)

Nah, HTML mail was well and truely endemic by then (the ISP I worked for was even spamming our users with it every week).

That said, how much of an impact does that have anyway? Worst case, there's an HTML copy and a plain text copy in the same email, so ... ~2.5 times the original size? That's not all that much of an imposition.

Attachments might be more of a concern, but even today, the default maximum message size of Postfix is still only 10Mb. I daresay most servers on the internet are still running with their defaults, so most people should assume (and be educated to assume) that if they send anything big, it's going to bounce.

Re:Only 70000 accounts? (1)

thogard (43403) | more than 2 years ago | (#38261200)

Two decades ago we were supporting 78,000 users on a machine that was almost as fast as 4 Nintendo 64.

What does this have to do with Sendmail? (3, Insightful)

farnsworth (558449) | more than 2 years ago | (#38260254)

In the video, they don't even mention sendmail at all. Are they using it?

Also, they mention that the cost of the system is something like $1.30 per account per month. I don't know much about IT budgeting, but that seems like a really low number for something as critical as messaging and calendaring. I have to imagine that they spend more money per user just cutting the grass around the campus.

Re:What does this have to do with Sendmail? (2)

lucm (889690) | more than 2 years ago | (#38260626)

Also, they mention that the cost of the system is something like $1.30 per account per month. I don't know much about IT budgeting, but that seems like a really low number for something as critical as messaging and calendaring. I have to imagine that they spend more money per user just cutting the grass around the campus.

Totally agree. One of my client did a major cost-cutting initiative for its email platform, and there was just no way to make it reliable under 9$ a month (per account). And this is when there is no Crackberry (which brings the numbers way up).

Re:What does this have to do with Sendmail? (3, Informative)

TClevenger (252206) | more than 2 years ago | (#38261336)

That's amazingly cheap. I don't know how you'd do it any cheaper outsourced. Microsoft is $8.80/user in qty. 20,000, and while Google starts at $4.17/user, I couldn't imagine that even 70,000 accounts could bring down the price that much.

IT cannot solve this (1, Interesting)

decora (1710862) | more than 2 years ago | (#38260260)

it's like saying IT can do heart surgery or IT can provide pscyhological counseling to a trauma survivor. IT is IT, it is not management and it is not leadership. IT is IT.

of course, shit rolls downhill, and leaders nowdays are incompetent buffoons who gain their positions largely through bribery, kickbacks, extortion, and other 'features' endemic to societies where the rule-of-law breaks down thanks to a greedy, corrupt elite.

again, IT cannot fix that.

Re:IT cannot solve this (1)

iggymanz (596061) | more than 2 years ago | (#38261528)

oh man, good IT people are leaders, whether in management or not. They try to identify resource problems before they become issue, and have solutions.

The failure is leadership, planning, budgeting... (3, Interesting)

linuxwrangler (582055) | more than 2 years ago | (#38260430)

I've only heard from people on one side of this but the story that I hear is that in the past, many departments had their own IT, mail servers, web, etc. When the campus built its centralized computing services facility, there was great pressure on departments to move to the central system. There was some griping about the costs for central services often exceeding the internal costs the departments formerly had but there was, I'm told, much need to justify the expense of and to pay for the new center. I've heard that some departments have been able to resurrect their internal systems to get through the outage.

Perhaps someone with more inside knowledge than I have can fill in and/or correct information from both sides of the story.

That slideshow is pure management-spin right from the opening "look how complicated and difficult this is..." I love how the "solution" to a system that is soon to outstrip its capacity is to stop expanding (and, it appears, properly maintaining) said system and hope it doesn't implode before you can toss the potato to an external party (who can then take the blame). Guess I was never learned at that school of capacity "planning".

Re:The failure is leadership, planning, budgeting. (1)

MightyMartian (840721) | more than 2 years ago | (#38260648)

They left out the slide where management get great big bonuses for being such swell thinkers.

Did the CIO just give up in the presentation? (3, Insightful)

Above (100351) | more than 2 years ago | (#38260446)

The press pretty much reads like this to me:

1) We didn't size the system large enough to handle the possible outages.

2) The outage we didn't size for happened, basically taking everything down.

3) My team is now working on a band-aid solution, which basically involves hobbling the application.

4) Since we're incompetent, we're going to outsource this next year.

I mean, if I was the CIO's boss I would have fired him on the spot. Maybe outsourcing is a better answer than putting in place a proper system and looking at that analysis could be interesting. I see no indication any of that was done here, basically the CIO gave the Barbie response, "Mail is hard, let's go shopping." If he doesn't understand how to do it in house, he won't understand how to arrive at a good outsourcing agreement.

Which means this pretty much sums up everything that is wrong with large org IT today.

How about... (0)

Anonymous Coward | more than 2 years ago | (#38260544)

... not treating a non-technical problem as a technical problem? Identify the problem, write the memo, keep the flimsies, and drop it on the relevant manager's trouble ticket queue. Or however the flow goes in your locality. The rest I leave as an exercise for the reader.

Just move to Google Mail (1)

dell623 (2021586) | more than 2 years ago | (#38260638)

Google have 24x7 phone support now. It is really a futile exercise to maintain local email systems even for a few thousand users, it will be outsourced sooner or later.

Re:Just move to Google Mail (0)

Anonymous Coward | more than 2 years ago | (#38260792)

Google have 24x7 phone support now. It is really a futile exercise to maintain local email systems even for a few thousand users, it will be outsourced sooner or later.

They probably will. The whole reason for the failure is lack of IT funding, and campus admin is currently studying whether to choose Google or Microsoft for their outsourcing. Of course, this is slightly tricky since federal law doesn't allow emails containing student information (eg grades) on third-party servers.

Re:Just move to Google Mail (1)

jasomill (186436) | more than 2 years ago | (#38261174)

They probably will. The whole reason for the failure is lack of IT funding, and campus admin is currently studying whether to choose Google or Microsoft for their outsourcing. Of course, this is slightly tricky since federal law doesn't allow emails containing student information (eg grades) on third-party servers

Then why report grades via email in the first place? Presumably some of the money saved could go towards implementing a Web front-end to the registration system, or to improving your current system to cover whatever use cases led to grades being emailed to students.

This is assuming communication between faculty and staff will remain either in-house or "seriously" outsourced (effective SLAs, NDAs that cover confidential information, etc., rather than best-effort "branded Gmail").

Oh, and electronic gradebooks are one thing, but please, please, PLEASE don't torture your students by implementing messaging embedded insome "courseware" system that amounts to a non-interoperable inbox for each course that uses the system on compliance grounds. Any email system that's inappropriate for day-to-day course communications is unsuitable for campus use in the first place, no?

Re:Just move to Google Mail (1)

Mr. Freeman (933986) | more than 2 years ago | (#38261396)

The general solution to this (as far as I can tell, based on being a student in a university) is to go with a third party and then implement a "no grades over email" policy. Professors can't send our grades, various things related to registration, etc. over email. At first I thought this was the university being stupid or an odd federal law with a blanket prohibition on such things, but now I'm starting to think that it's due to the interaction with third party devices. (we use Google for our email)

The email experts are not ... (1)

jamesl (106902) | more than 2 years ago | (#38261230)

... at the university. And they haven't been for decades. Microsoft and Google have thrown money and brains at these kinds of problems -- money and brains in quantities that university IT departments can only dream of. Both have economical, reliable, scalable, secure and user friendly solutions to this problem.

Look up Microsoft live@edu and Google aps for education.

Re:The email experts are not ... (1)

iggymanz (596061) | more than 2 years ago | (#38261556)

Microsoft exchange server is a reliable, secure and scalable solution? Bwhahahahaha, my employer uses that shit, it is none of the above. If it were, the internet's backbone would use it

Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Sign up for Slashdot Newsletters
Create a Slashdot Account

Loading...