Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

A Proper Environment for Web Development?

Cliff posted more than 7 years ago | from the a-sane-and-careful-way-to-code dept.

Programming 66

umdenken wonders: "I'd like to know how others on Slashdot do their server-side web programming. We have dozens of Perl CGI scripts, and are currently doing development by editing these production scripts in place on the web server (!). Our sysadmins have finally installed an SVN client on the server (Solaris), and have offered to create a new virtual host that we can use as the development server. What are some of the practices you use for organizing this kind of set up?"

Sorry! There are no comments related to the filter you selected.

Er, uh (-1, Troll)

baldass_newbie (136609) | more than 7 years ago | (#17382196)

You mean like a test server?
We are talking basic code control, right? I mean, this seems so fucking obvious I feel like I'm missing something.

Hey slashdot, can you help me figure out what question Captain Obvious is asking?

Re:Er, uh (2, Informative)

FooAtWFU (699187) | more than 7 years ago | (#17382388)

I'm sure this sort of problem is trivially "obvious" if you've been in the IT industry for a decade or so, but if you're not already in this sort of environment and someone sort of dumps the problem in your lap, what are you supposed to do? How to construct and manage a source control system for a decent-sized environment isn't usually the sort of thing you will find with your average B.S. in computer science. I certainly know that I wouldn't know how to go about doing much more than the basics (give me svn, don't edit things straight from production, umm, two machines to work on would be nice, and... uhh... I dunno, what next?). It's the sort of thing that I'd expect to learn about either on the job (using one that someone else with more IT knowledge set up for me), or from some sort of technical articles (which?), or both. Perhaps someone can recommend an online guide on how to structure a sophisticated system for source code, system configurations, and such... or maybe some books...

Re:Er, uh (0, Troll)

frank_adrian314159 (469671) | more than 7 years ago | (#17383120)

...but if you're not already in this sort of environment and someone sort of dumps the problem in your lap, what are you supposed to do?

Tell your boss he should have hired someone experienced, rather than some "wet behind the ears" kid? :-)

What is training? (0, Offtopic)

tepples (727027) | more than 7 years ago | (#17383420)

Tell your boss he should have hired someone experienced, rather than some "wet behind the ears" kid? :-)

How does "some 'wet behind the ears' kid" become "someone experienced"?

Re:What is training? (1)

Surt (22457) | more than 7 years ago | (#17383922)

By working with and doing the work assigned to him by the someone experienced, receiving assistance on the parts he doesn't understand as needed?

Re:What is training? (1)

Clover_Kicker (20761) | more than 7 years ago | (#17383962)

How does "some 'wet behind the ears' kid" become "someone experienced"?
Option 1 - work with experienced people for a few years so they can tell him which mistakes to avoid (and why).

Option 2 - work by himself for a few years, making a few thousand (highly educational) mistakes along the way.

Re:Er, uh (1)

arb (452787) | more than 7 years ago | (#17382542)

He's currently doing dev directly on the production server and is wondering if there is a better way. I can't help but wonder at the irony of asking Slashdot how to do proper development! ;^)

Re:Er, uh (3, Insightful)

ePhil_One (634771) | more than 7 years ago | (#17382714)

He's currently doing dev directly on the production server and is wondering if there is a better way.

His COMPANY was doing dev directly on the server, he has implemented SVN and a Test environment, and is wondering what the Best Practices might be. He knows there's a better way, and rather brilliantly knows he doesn't have all the answers.

For the record just a dev and prod environment isn't enough, ideally you would have multiple dev environments (individual playgrounds plus common test areas, two QA environments (New releases and current release for bugfix testing), and possibly even a User Acceptance Testing area. There should be no code updates a release is migrated through the environments, all environmental variables get read from the environment...

Of course, every dev environment will be different, with different needs, release cycles, etc...

Re:Er, uh (1)

fyrie (604735) | more than 7 years ago | (#17383484)

Agreed. Development and production environment are not enough. Here are the environments we have at my shop: * Production - The live application. * QA - Whatever needs to be tested, but the hardware and software configuration mirrors production (os, patches etc are the same as production). * Development - A daily, weekly, whatever-serves-your-development-pace controlled environment. Developers do not have control over this environment. This environment should also mirror QA and Production as far as OS and patches etc... * Sandbox - developers have control over this environment, but there should be some sort of mechanism to track what changes they make to the server etc... This environment serves the purpose of testing and messing with configurations and is not to be considered stable. Ideally, the base mirrors the hardware and software of production, but the purpose of this environment is to try different settings and configurations. * Workstation - Localhost webserver If you have a web farm, the picture becomes more complicated. In this case I would recommend having at least a small web farm in QA.

Re:Er, uh (1)

umdenken (729008) | more than 7 years ago | (#17384366)

"His COMPANY was doing dev directly on the server, he has implemented SVN and a Test environment, and is wondering what the Best Practices might be."

Exactly!

Re:Er, uh (1)

Doctor Memory (6336) | more than 7 years ago | (#17398056)

For the record just a dev and prod environment isn't enough
Actually, it could be. It sounds like this is a pretty small shop, so it's not unreasonable that developers develop something in dev, then have their account managers (or other responsible-as-in-accountable persons) review the changes, then roll them into production. Lots of web projects are easily broken down into discrete non-interdependent pieces that can be developed and rolled out separately. This is certainly true for the Perl and PHP shops I've worked in, much less so for Java (since you often have to deploy an entire war file to roll out new features).

ideally you would have multiple dev environments (individual playgrounds plus common test areas, two QA environments (New releases and current release for bugfix testing), and possibly even a User Acceptance Testing area.
If you're running a large web site or creating a significant web-based application, this would be appropriate. The progression I usually see in "mature" shops is: development -> integration -> UAT -> QA -> production. Lots of places place UAT after QA, but if you're doing proper regression testing, then you waste a lot of time testing in QA every time the users decide they want the layout changed, or certain fields moved to another form.

In response to the original question, I'd suggest that each developer have a copy of the production system on their machine (which should be running Solaris using the same versions of Perl and Apache and whatever else you use), and there should be a separate test/QA environment (could probably be set up under its own zone on the production box). You'll also need to define processes for common tasks (release a new module, release a new version of an existing module, retire a module, roll back a module to a previous version, release a new version of the whole system, etc.) and document them. Then develop a process for keeping the documentation up to date. Wikis are good for this.

Easy: (1)

Rosco P. Coltrane (209368) | more than 7 years ago | (#17382212)

- Several web browsers (IE, FF, Opera...)
- Emacs of vi (depending on which side of the flame war you fight for)
- a W3 validation tool

Re:Easy: (-1, Flamebait)

Anonymous Coward | more than 7 years ago | (#17383584)

I use emacs because I fight for the correct side of the war :)

Re:Easy: (0)

Anonymous Coward | more than 7 years ago | (#17387510)

Also, you should live in the early nineties, where you can still make the big bucks.

Virtualisation (4, Informative)

mccalli (323026) | more than 7 years ago | (#17382222)

Even with a virtual host, you'll still be hitting the production hardware. Sounds like an ideal use for virtualisation here - mimic as much of the production environment as possible (OS versions, web server versions, application container versions etc.) and have a go with that.

Subversion is definitely a stride forward - well worth using and getting used to, it's good that you have a client there. You should be able to fix your config scripts such they they recognise the environment (prod, dev, uat) and can be deployed directly from a tag in svn. A tag of course, not the trunk. Given the constraints it sounds like you're up against, I would definitely be looking to virtualise at least three environments - one dev, one system test, one UAT. You may have multiple dev virtual machines depending on your needs.

Cheers,
Ian

Re:Virtualisation (3, Informative)

mikemulvaney (24879) | more than 7 years ago | (#17382422)

Don't use virtual hosts at all. If you can't get another machine, then at least run another copy of the web server on a different port. Then your production address would be something like http://server/whatever [server] and the development would be http://serverport/whatever [serverport] .

You are going to have to restart your web server in development more often than you would like to in production. You don't want to bounce production every time you change a module, right?

Re:Virtualisation (1)

mccalli (323026) | more than 7 years ago | (#17382458)

Don't use virtual hosts at all.

Agreed, and I should have clarified - by virtualisation I mean full machines, such as VMWare instances. Not Apache virtual hosts. Ideally I'd get separate hardware for UAT and dev/system test but it sounds like the constraints the submitter is under won't allow for that. That's why I'm suggesting falling back on virtual machines as a good second base.

Cheers,
Ian

Re:Virtualisation (-1, Redundant)

Anonymous Coward | more than 7 years ago | (#17382684)

Don't use virtual hosts at all. If you can't get another machine, then at least run another copy of the web server on a different port. Then your production address would be something like http://server/whatever [server] and the development would be http://serverport/whatever [serverport] .

Huh, browsers default to port 80 for HTTP, so I think that you mean "http://server:port/whatever".

Re:Virtualisation (1)

JahDread (457290) | more than 7 years ago | (#17382558)

A tag of course, not the trunk. Thats an interesting point to me... Anyone care to explain why? Aren't tags forever static? Seems like my websites are constantly changing, so why would I deploy from a tag instead of a branch or trunk? Just curious.

Re:Virtualisation (2, Informative)

mccalli (323026) | more than 7 years ago | (#17382590)

It's to gain defined release states, with defined roll-back points. The trunk will contain simply what's being worked on at any given moment - it may well not be in a deployable state (probably isn't). A branch...well, sort of. You'd create the tag to get your defined release point, the branch off that tag for any bug-fix releases prior to merging that branch back to the trunk. Any bug-fix releases made from that branch would also get their own tag.

Cheers,
Ian

Re:Virtualisation (1)

Bitsy Boffin (110334) | more than 7 years ago | (#17384054)

You do some development, once it's at a point where you want to release, tag it, switch production to the tag (svn switch), if something wasn't quite right (Murphy's Law) svn switch back to the last tag you used.

Separate your environments (5, Interesting)

KermodeBear (738243) | more than 7 years ago | (#17382330)

In my experience, I have found that it is best to have four environments: Production, Pre-Production, Testing, and Development. Every environment is isolated and has the same hardware and software.

Developers do their development in one environment (if possible, each developer should be isolated), and when their code is written it goes to a testing server where it can be hammered by your QA/Testing team.

When it passes QA, it goes into Pre-Production, which is what your Production environment should look like when you push your changes live. Any kind of external integration can test against this environment, since it is as close to your production environment as possible.

Then, you have your production environment where everything is live.

It is VITAL that each environment is set up the exact same way, or as close as possible, to every other environment. Each one should have its own separate hardware running identical software versions - unless you are testing software upgrades, in which case you FREEZE THE CODE, update Development, then QA, the Pre-Production, then Production (testing everything, everywhere), and THEN resume writing your code again. It is incredibly frustrating for a developer when code works on servers A, B, and D, but breaks on C and E due to non-matching hardware and software.

It is also important that your developers and qa team have access to fresh, live data if at all possible. It is easier to develop when you have real data to develop with; Plan on updating your QA and development data once a week.

If you have a set up like this, then development, testing, and deployment will be very smooth. It can be a bit of a pain to set up at first, especially if you are not used to the idea, but once you have something like this I promise that you will never go back.

Other people may recommend different set ups, but the basic idea is the same. Keep the developers separate from the testers, keep everyone separate from production. The pre-production just makes deployment easier - push code that passes QA to pre-production, then when the time comes, just rsync the files over.

Re:Separate your environments (5, Insightful)

Phillup (317168) | more than 7 years ago | (#17382430)

It is VITAL that each environment is set up the exact same way, or as close as possible, to every other environment. Each one should have its own separate hardware running identical software versions - unless you are testing software upgrades, in which case you FREEZE THE CODE, update Development, then QA, the Pre-Production, then Production (testing everything, everywhere), and THEN resume writing your code again. It is incredibly frustrating for a developer when code works on servers A, B, and D, but breaks on C and E due to non-matching hardware and software.

I like for the pre-production and production environments to be as close as possible.

But...

I like the dev and testing environments to be different from each other... and from the production environment.

I've found that doing this helps me shake out some "dependencies" that I may not have thought about.

Taking care of those "dependencies" helps me write code that is easier to move to another environment if the customer wants to upgrade their systems.

YMMV

Re:Separate your environments (2, Insightful)

Fulcrum of Evil (560260) | more than 7 years ago | (#17383312)

I like the dev and testing environments to be different from each other... and from the production environment.

Simple solution that mostly works: run a lot of the dev stuff on your desktop. Depending on your environment, that can isolate developers from each other and free up a test server for 'more stable' code.

Re:Separate your environments (1)

poot_rootbeer (188613) | more than 7 years ago | (#17387092)

I like the dev and testing environments to be different from each other... and from the production environment.
I've found that doing this helps me shake out some "dependencies" that I may not have thought about.


That's okay, but the problem is that those "dependendies" aren't being revealed until the product breaks in production. And by then the damage has already been done.

I can see merit to having a different configuration for the QA/test environment than for the development environment, but you need to be as certain as you can that there will be no surprises when the product is deployed to the production environment, and that means having a pre-production environment ('staging' if you have multiple tiers, 'development' if you have only one) that mirrors the production environment to the greatest extent possible.

Re:Separate your environments (1)

Phillup (317168) | more than 7 years ago | (#17388302)

That's okay, but the problem is that those "dependendies" aren't being revealed until the product breaks in production. And by then the damage has already been done.

Actually, they break in pre-production. That is why it is supposed to be exactly like the production system...

Re:Separate your environments (1)

tonigonenstein (912347) | more than 7 years ago | (#17383112)

The parent is spot on, although maintaining completely separate hardware systems is probably too expensive for simple projects. First, I can give you two very important advices:
  • do not allow developers to make modifications directly to the live server. Nothing should be able to avoid going through testing.
  • do not manipulate the files on the server directly. At work I use the following system: each site is kept in subversion and a custom script (with a simple web frontend) synchronise the web server with a particular revision of the repository. This provides three benefits:
    • integrity. What is put on the server is exactly what was tested.
    • logging. You know which revision was live at any given time. Helpful for problem reports.
    • rollback. If you have a problem with the last version, you can instantly roll back to a previous working revision while you investigate the problem.
To provide separate environments, giving each developer a XAMPP installation (I prefer a linux VM) is sufficient. When they are done they can commit their changes.
Testing can checkout the code and test it in a similar environment. When it is done, the approved revision is pushed on the server using a technique similar to the one mentionned above.
In this setup the dev and test systems are not 100% identical to the web server but this doesn't usually matter. If you have the same versions of php/mysql it is enough.

Re:Separate your environments (1)

Steepe (114037) | more than 7 years ago | (#17383488)

Decent. The way we do it is...

First off, our developers are either Java (resin engine) or .net
Pretty much the only perl that gets written is by me and the other sysadmin. We do still have one ancient PHP app, I'd LOVE to nuke it, but development resources to rewrite it .....

Java developers run Linux kickstarted and set up to our environment (build system, installs required packages, sysadmins control the yum repository) they run dev on their own boxes, with their own java engines and apache.

Test mirrors production, only to a smaller scale (4 app servers in test, 12 in prod, 2 web servers in test, 8 in prod) beyond that they are exactly the same. same load balancers, etc. but they DON"T touch the same hardware EVER. I don't want a runaway test app taking out my disk space on the NAS for production, or whatever.

Production apps start out on one resin engine for a few weeks until they get some production use and we are sure they are not going to drop a money app. Then we move them to a production resin engine. .net stuff and asp we are just getting into. basically its the same, test mirrors production, only smaller.. and they develop on their workstations. Beyond that its the same. dev, test, production.

We use CVS currently for versioning, One of the tasks on my plate are to move us to subversion with https access etc. got it mostly working, just linking it to AD to do.

Re:Separate your environments (1)

ePhil_One (634771) | more than 7 years ago | (#17387982)

It is also important that your developers and qa team have access to fresh, live data if at all possible. It is easier to develop when you have real data to develop with; Plan on updating your QA and development data once a week.

In some environments this is a serious security risk, since production data can contain SSN's, medical or other "private" data, etc. Use at your own risk.

Re:Separate your environments (1)

Doctor Memory (6336) | more than 7 years ago | (#17398498)

Plan on updating your QA and development data once a week.
It's better if you simply give your developers the ability to update their development databases at will (perhaps by restoring from a weekly snapshot). I know several times I've set up some complex (and sometimes 'illegal', constraint-wise) data relationships to ferret out a particularly evil bug, and it can take time to recreate those. Plus, in a dynamic development environment, updating the database can require updating your code base as well (tables get added/merged/split/converted to views), which can be a non-trivial task. It can be frustrating to put your work on hold for half a day while you go chasing dependencies and merging your code with the head, only to have to turn around and do it all again a couple of hours later when you get your code working and want to merge it back.

Re:Separate your environments (1)

jetlagQ (611731) | more than 7 years ago | (#17398808)

All good practices. I would add that with subversion you can branch the code for any significant work and deploy from the branches to the dev machines. I say machines because in a shop with lots of concurrent development (messy but needed if you have tight financial and regulatory deadlines) you will want a different branch and dev machine for each concurrent effort. then as projects finish up they can be merged into release codelines and then tested on the test machine(s) from those codelines/branches. as each release concludes it can then be merged into the trunk. anyway that was our practice at bank of america when i was there and it worked pretty well.

Re:Separate your environments (1)

sparklingdreams (1046114) | more than 7 years ago | (#17490876)

Thank you, Someone who feels development should be equally painless on like us developers as on end users. Yeah this is like my dream set-up. And yeah its worth the money & time to setup those environments!

My perspective... (3, Informative)

Yakman (22964) | more than 7 years ago | (#17382332)

I hope you're not doing anything important. Although chances are if you're using Perl CGI it's probably not.

Most web development environments I've been exposed to have a development, UAT (User Acceptance Testing) and production environment. Alternatively your development environment can be local and you can stick a "system testing" environment in between dev and UAT. Your UAT environment should mirror production, and before you apply any changes should have whatever code is currently in production.

You do your development, and if it's being done locally you integrate your changes with everyone else's in system test and do your automated testing and so forth. Once the developers/testers are happy a release is packaged up and deployed to UAT. You should probably run your automated tests again here for a sanity check.

The end users (business or whoever) do their testing in UAT, and if they're happy (and this is important) you take the same package you applied to UAT and apply it in production.

In some environments the developers aren't the same people with access or rights to apply changes in produciton, so you've got different groups performing different roles, but you get the idea.

Disclaimer: this is just my experiences of corporate web development, your mileage may vary, but I believe this sort of setup is pretty common (with differences here and there).

Re:My perspective... (1)

Phillup (317168) | more than 7 years ago | (#17382460)

Although chances are if you're using Perl CGI it's probably not.

Unless you meant "instead of mod_perl"... I call troll.

Hell, I call troll anyway...

Let the language wars begin!

;-)

Re:My perspective... (1)

grcumb (781340) | more than 7 years ago | (#17382504)

I hope you're not doing anything important. Although chances are if you're using Perl CGI it's probably not.

Oh puh-lease. Perl is used for important things all the time. I mean, Slashdot is written in Perl!

Okay, seriously: It's not clear whether you're referring to Perl or CGI as inappropriate for serious work. I agree that CGI deserves that status, but not Perl.

Re:My perspective... (1)

Yakman (22964) | more than 7 years ago | (#17382926)

I assumed when the OP said "Perl CGI" that it was legacy sort of CGI scripts like those form mailers and things from back in the early 90s. I don't do Perl, so I don't know what the state of affairs is as far as Perl web development goes, that was just the first thing that came to mind.

I was thinking as compared to say an MVC Java application where they're editing their business objects directly on the server.

Re:My perspective... (0)

Anonymous Coward | more than 7 years ago | (#17384842)

I don't do Perl, so I don't know what the state of affairs is as far as Perl web development goes, that was just the first thing that came to mind.

Wait let me get this straight: you don't know shit from shinola yet you offer an opinion on what's good and bad?

This one is easy... (0)

Anonymous Coward | more than 7 years ago | (#17382360)

Mac OS X [blogspot.com] , duh!

Buggie production server... (2, Interesting)

creimer (824291) | more than 7 years ago | (#17382402)

If you find a bug on the test server but can't reproduce it on the production, don't assumed that it's not there. My boss made that mistake since he couldn't reproduced the bug for a new tool on the test server and he approved the release candidate. The art department was consistently crashing the production server the next morning by using the new tool as described in my bug report. The production server was off line for three days as the programmers had to do a thorough code rewrite that cost the company $250K. A few months later, they declined to renew my contract. Go figure.

Process with Tomcat (2, Insightful)

wmshub (25291) | more than 7 years ago | (#17382466)

I use JSP, which makes it a little bit easier to use "good practices" since you can't just edit the files on the web server.

Instead I build up a .war file (with ant), which as a side effect also verifies that all source is committed into subversion, then it tags it all so I can always see exactly what went into the current release on the web server. Then I install the .war on the test server. Test. Then remove the current webapp from the release server. Then, as quickly as I can, install the .war that was tested. (There should be a way to tell tomcat to install a new webapp *on top of* an old one, then undeploy the old one, removing this "gap" where no webapp is present...anybody know how to do this?)

There are drawbacks though. Once in a while there's an emergency and I'm asked, "can't you just edit this file and get it fixed in 30 seconds?" and I have to explain that even if I skip the "test" period, the build/upload/install takes about 20 minutes (it's a big webapp, the build script runs some image generation apps that take 5-10 minutes minutes).

Re:Process with Tomcat (2, Informative)

gsandie (753576) | more than 7 years ago | (#17382698)

(There should be a way to tell tomcat to install a new webapp *on top of* an old one, then undeploy the old one, removing this "gap" where no webapp is present...anybody know how to do this?).
We use JBoss which lets you hot deploy the new wars in thie way. JBoss comes with Tomcat built in, might be worth checking it out. :)

Non-obvious Tomcat advantage (0)

Anonymous Coward | more than 7 years ago | (#17382810)

Our school's JSP pages seem to have a significant advantage over our static pages when it comes to search engine results. Example: Our department teaches electronics. The school's search engine finds a policy document (jsp) that contains the word 'electronic' only once before it finds our department's home page which is linked to by many pages and has many occurences of the word 'electronics'. (Plone also seems to claim that pages produced by it fare better on searches.)

If anyone can explain why the JSP pages do better with search engines I would be grateful. If what I have observed is valid, then it would make a lot of sense to use Tomcat.

Re:Process with Tomcat (1)

poot_rootbeer (188613) | more than 7 years ago | (#17387122)

I use JSP, which makes it a little bit easier to use "good practices" since you can't just edit the files on the web server.

That's a feature/flaw of your servlet container configuration, not of JSP itself. Websphere, for example, can allow JSP templates to be edited and recompiled on-the-fly.

Re:Process with Tomcat (1)

angel'o'sphere (80593) | more than 7 years ago | (#17392694)

Ofc you can just edit the fiels on the web server, also with tomcat.
But it makes no sense if hey are deployed via a .war file and not as individual files.

JSP as part of J2EE defines that it is/must be possible to edit JSP files and let them automatically recompiled.

angel'o'sphere

Minimize complexity (5, Insightful)

riceboy50 (631755) | more than 7 years ago | (#17382536)

In my opinion, there is a point at which complexity outstrips any gains from separation. Just keep it simple; dev, staging, production.

Re:Minimize complexity (0)

Anonymous Coward | more than 7 years ago | (#17382608)

Sounds almost like debian's model.

One setup that works (4, Informative)

suv4x4 (956391) | more than 7 years ago | (#17382594)

I'll tell you how our entire setup works, and hope it helps:

A. Development Box
B. Test Server
C. Production Server

A. Development Box

Every developer has Apache (or IIS respectively) and PHP/SQL on his box. People without experience can just install one of the premade packages that exist (like XAMPP or whatever its name was). This setup is isolated from the outside and responds only to 127.0.0.1 and the virtual domains. Each virtual host in Apache is a separate project.

Server Side Developers work in Eclipse PHP IDE with SubClipse, designers/client side work in Dreamweaver/Photoshop with SVN4DW & TortoiseSVN.

B. Test Server

This is used for few purposes: devs can checkout a revision and run it there on a "real" server to test, QA (well we have no dedicated QA.. it's a small team) can test on this server too.

If everything is ok it goes to...:

C. Production Server ...

That's it, it works really well though, everyone has his own server that can run files right of his PC, and this helps a lot in quick development. Showing to clients is as easy as checking out a revision on the test server.

Re:One setup that works (1)

szembek (948327) | more than 7 years ago | (#17388472)

Yup, this is the solution.

Re:One setup that works (0)

Anonymous Coward | more than 7 years ago | (#17394070)

Wow, I'm sure glad that comment had score of 2, or I might have missed it! The insight!! The magic!! Oh thank you szembek, thank you!

Disaster Recovery? (0)

Anonymous Coward | more than 7 years ago | (#17404074)

What happens if your production site becomes a hole in the ground? You need geographic diversity for your new production location.

Honestly, the problem statement doesn't provide enough information to design an environment strategy. I do this for a living. Simple questions need to be answered by the customer first.
- How long can the production service be down for software failure and/or hardware failure and/or disaster?
- How many customers will be impacted during that outage?
- What are the maintenance periods allowed?
- How many transactions (web, db, per interface)?
- How many developers, testers, UAT users, and production users?
- Can backups be performed while online?
- What existing infrastructure do you have to leverage?
- What is the current application stack and development toolset?
- What existing enterprise licenses should we take advantage of?
- Are there any bandwidth limits we need to be aware of?

These just get us started.
OTOH, this is slashdot, so just do what the guy above says and cross your fingers. It will probably be just fine.

Re:One setup that works (1)

achacha (139424) | more than 7 years ago | (#17429624)

You missed a critical part: production should be 2 identical environments. One is currently running and the other is the fallback/deployment.

Lets say you have 1 machine in production (not likely but makes it easy). Machine A is serving all the traffic. Machine B is out of the "loop" via load balancer. You deploy your staging code to machine B and run some smoke tests. Once all is go with Machine B, the load balancer flips or slowly migrates traffic from A to B (can do all or stage it by percentage of users, this depends on the volume of your site, very big sites usually do 25/50/100% or 10/25/100% or 25/50/75/100% whatever fits your size). Now A is offline and contains the last release and ready for fallback until your next deployment, which buys you enough time to flip to A if there is something critical in B that affects business. I worked at a place where there were multiple fallback just in case (and when billions of dollars are on the line, you need that kind of fallback).

In real world, Machine A is a cluster of machines maybe 100 maybe a 1000, depends on your business demands and if you are running 1000 machines, having another 1000 as a backup is not am issue since you can borrow from the fallback pool if anything fails in production and things fail (drives, memory, network cards, power supplies, whatever).

SVN is nice from teams Staging -> Prod is dead on.

The only thing I would add is keeping a machine for each older release (up to maybe 5) so that you can do regression to see when a certain bug was introduced. At one place I worked we had 10 machines that worked like a circular queue of all past releases and if you get a bug in the current staging build, you can go back a in releases to see when something was introduced. This depends on the business you are in and frequency of the releases, you may only need a few regression machines.

Another point is to deploy things to a VM image (like VMWare) and allow developers to make copies of it (assuming you have the licensing for the OS) and test things in that environment. VMWare is a useful way to keep older releases around without devoting hardware to it. If you are on linux then you have no licensing issues and can make VM copies as needed (you can download an image from VMWare site they have links to a bunch of linux distros).

Email me if you have more questions, I have done this stuff with shops running on 10 machines and ones with over 5000 machines and 1000 developers, so I may have some useful hints.

Seperate environments, deploy from source control (1)

OnyxRaven (9906) | more than 7 years ago | (#17382654)

Dev / Testing / Staging-Acceptance-PreProduction / Production

Each developer should get their own virtualhost on the dev server that can mimic production (apache virtualhosts if possible, or use vmware), that they can upload and self-test to. Use wildcard dns for hostnames like username.dev.company.com.

Depending on how many paralell things are tested, you might want more testing systems. Testing boxes should be mirrors of production software-wise (or at least as close as you feel comfortable).

Staging/Preproduction might be optional if you're a small shop or don't need acceptance testing - this should be a more stable place to do system tests outside of the possibly buggy testing environ. again, should mimic production, possibly closer.

production - obvious.

If you can do it, do deployments from SVN via labels or tags (not sure which it uses) like RELEASE_2006-12-27_NOTE (and something like RELEASE_LIVE which moves), so you can track revisions which are on production. Instead of copying code up, you use svn to sync code down, probably through some kickoff script. this occurs to each environment. Take everyone's access away from production, and probably staging - testing you might give select people access. Only the people in charge of the machines themselves, and whoever's in charge of releasing code should have access. Use automation with secure keys and stuff to do deployments.

automated tests happen in testing on a machine that can automatically sync itself to the latest checked in code.

Potentially, all of the above (cept for probably production) could easily be virtual machines.

PXE and Kickstart (2, Interesting)

adamy (78406) | more than 7 years ago | (#17382672)

Big Fan of PXE and kickstart for deploying development machines and Yum/Apt for deploying code.

1. Use PXE as a way to get an install running as quickly as possible. Do a minimal install to get the machine up and on the network.
2. Use Yum, Apt or Yast according to your distro to install everything required to support your development and application stack.
3. Your code should be done as a package (RPM/Deb). Yes, even for web stuff. If you customize your Apache install, that should be a package, too.
4. Use Subversion. CVS makes me sad, because it isn't Subversion.
5. Have a build environment that pulls your code out of subversion and builds it for QA. A package that is built here is what will be pushed live.

The bottom line is make it as easy as possible to reproduce your environment.

And back everything up over the network to a server in another state. Prefereablle, two different servers, at least one on another coast.

Local PHP (1)

Teifion (1022083) | more than 7 years ago | (#17382910)

I run a PHP server on my Mac and use that to test my pages. None of my normal pages need anything else. When I do have to use live data I ensure that what I am doing is non-critical, if it is I'll duplicate the data I need and run it in a seperate area.

Blah blah blah (-1, Troll)

Anonymous Coward | more than 7 years ago | (#17382932)

Cue Slashdotters listing their pet development tools. The "proper" solution is one that works for you and you shouldn't have to ask.

I use a popular text editor and RCS, developing on linux and testing locally before deploying to a shared staging server with dev and stable vhosts. The project managers are responsible for pushing our apps to the production servers from there. It's rocket science.

how we do it (4, Informative)

Anonymous Coward | more than 7 years ago | (#17383088)

This is Unix/OSX-centric, and I've been doing this for years and showing others the way:

* My code always has a standard layout (bin, conf, src, lib, and so on). No exceptions, because you never know when that little script will become a big app (this happens to me at least once a year).

* Use good coding practices: unit tests, continuous integration, whatever

* The code is checked into CVS/Subversion/Darcs, whatever. Use branches and so forth intelligently (dev on the trunk, release branches which are bugfix only, whatever). Make it "obvious" where the latest stable code always lives, so that someone besides yourself can deploy it.

* I have a script which will deploy the app to any server with rsync (excluding CVS, config, test, and dev files). There is also a flag that will "pull" the files from the server, in case of an emergency fix that was done directly on the server.

* There's a "config" directory with all system-dependent configs. No passwords or other stuff is hard-wired into the app.

* As others have described, you have your various "dev" machines, a "staging" server (identical to production but non-critical), and a "production" server. NEVER WORK ON THE PRODUCTION SERVER EXCEPT IN EMERGENCIES. Also, resist the temptation to install extra stuff on the staging server like some do (MRTG or Nagios or whatever). The staging server must be identical to the production in all respects.

If you do this properly, all you have to do to work is the following:

1) go to your laptop or other dev environment and check out the code.
2) review the REQUIREMENTS file for any packages you might need to install
3) adjust the app config files appropriately
3) code, test, checkin, code, test, checkin, .....

Then when you're ready to update the code on the server, first sync to the staging server and test as needed (user tests, whatever). Virtual hosts can work this way too (I've done it this way, for instance "foo.com" is the app and "dev.foo.com" is the staging version, on the same machine).

Once everything is working, push to the production servers (my script will also restart anything that's needed .. by the way, use daemontools to supervise your daemons, anything else like crappy pidfile-based startup scripts are unpredictable). Ideally someone OTHER THAN YOURSELF should be able to mindlessly follow some simple instructions to accomplish this step. Design your procedures with this in mind.

I also agree with the poster above who said it's good to have different OS or environments on your dev machines, to catch any hidden assumptions. I dev on Linux and OSX and push to FreeBSD usually. Use conditional code as appropriate, and sparingly.

Ideally, you can take a blank server, install the OS, install the REQUIREMENTs, push the app, config the app, run a setup script, and go. No undocumented requirements. No weird "procedures". Just "push button install".

You should also get into the habit of making apps "learn" as much about their environment as possible. For instance, in my Ruby, Perl, PHP code, I use the __FILE__ variable or equivalent to determine the install dir, that way I don't have to configure it. A common library sets up all the necessary paths based on that.

Write your code to be flexible and backwards-compatible. For instance, if you need to move some data files around or change database fields, write code that detects the old version and does the update at startup time. A little extra work, but oh-so-automatic.

Once you get this working, you'll never want to work any other way. Being able to check out and deploy your code ANYWHERE in just a few steps is a very powerful feeling. Heck, just being able to check out in a different directory on your dev machine is useful. Having separate release branches are awesome when the client reports a bug but you've already started the big changes for 2.0 .. OR when you get a SECOND client who will run an older version of the code and you have to maintain both. *Assume* your code will be popular and successful, think big from the start .. no throwaway scripts allowed, do things right from the get-go.

And if you're inheriting a system that doesn't work this way, refactor it until it's like this. Just make small changes regularly. I inherited an UTTERLY CRAPTACULAR PHP system and over the course of a year, hammered into shape while doing the work the client requested. Now everything is laid out beautifully, there's a test framework, it can be deployed with one command, all dependencies are documented, etc.

Good luck!

Dont forget the essentials (0)

Anonymous Coward | more than 7 years ago | (#17383274)

Besides the *obvious* stuff everyone else is telling you, don't forget:

Caffeine
Hot Pockets
Xena tapes

Essential to keep you sharp!

Where I work (1)

complete loony (663508) | more than 7 years ago | (#17383610)

The process has evolved slowly over time to cope with different numbers of clients / developers / environments. We have somewhere around 15-20 client production environments to support at different versions.

Developers are expected to document how to test their change before they commit.

Every week we build from trunk and deploy to an Integration Test environment.

Developers are then expected to re-test their change to ensure it's all in there and playing nice with any other changes.

The build is then deployed to a System Test environment where our testing team re-tests all the changes and does some other regression / system testing.

Now, most changes are bug fixes or small features for a specific client at a specific version, so we also maintain at least one environment per client internally, and all changes are patched into the client code base as well by the developers. A patch for the client environment along with deployment instructions is also produced, most of this process is automated.

The client patch would then be deployed to the internal client environment, and be re-tested by the client support team, who keep track of the whole process.

At the client site, the fix would be deployed to their UAT environment, and the fix retested again by the client.

Once they've signed off on the fix, it would be deployed to their other 2 environments. Production support, which is their sandbox for mucking around with the application, or attempting to re-produce bugs. And it that worked, then to production.

For a major client upgrade we would create at least another internal environment, and a client UAT environment. And perhaps a separate dedicated dev environment as well. Sometimes the dev and internal client environments will share the same database.

All this is fairly cumbersome, but we bend over backwards to make sure we don't deploy any change to a client they were not expecting, and without both us and them testing it.

Ghetto (1)

lorcha (464930) | more than 7 years ago | (#17383882)

That was the most ghetto setup I've heard of in a long time.

For Apache and perl CGI, why can't every developer have his own Apache instance on his own development workstation? Use Subversion, and away you go.

You have a simple architecture. I don't see why you need to be more complex than this.

How does this change with Database sites (1)

white_owl (134394) | more than 7 years ago | (#17384468)

I am working on a Drupal site. Fortunately, I am doing the Web programming so the revision control stuff works for me. It seems that for the content creators it is a bit harder. In the development environment there is going to be a bunch of cruft that you will not want in production, and the production site never sleeps. So what are the best practices for keeping track of new stuff, dumping it out of several tables and then merging it back into the production site.

Perl (0)

Anonymous Coward | more than 7 years ago | (#17385324)

yer like 2.5 generations behind thar buddy. get on the boat with Java and Ruby.

start with this:
sudo find / -name .pl --exec rm -f

Solaris 10 zones (1)

Kymermosst (33885) | more than 7 years ago | (#17385338)

by editing these production scripts in place on the web server (!).

Yeah. That exclamation point does not begin to describe what a Bad Idea this is.

Here's what you should do first, this is similar to what we are doing with one of our setups (note, I am thinking cheaply here - this won't be the "best practice" but it will be good enough): Get two more servers. Put Solaris 10 on both of them.

Call the first one your development/integration machine and create a bunch of Solaris 10 zones on it, one for each developer plus a few for integration and release candidate functional testing. Each zone should have a copy of the software used in production. If you have a lot of developers, then you'll need more machines. A good mid-level server will easily host 15-20 zones if they aren't all running a C compiler constantly - as long as you put a lot of memory into the machine. Create a master zone that can be restored onto any of the developer zones (in case they want to start clean).

Call the second one your QA/Staging server. Create a zone for staging releases as the final step before you put them into production, and use the root zone for QA. Aside from the staging zone, everything should be configured as close as possible to your production setup.

Ideally, you'd want a separate QA and staging environment, with QA exactly replicating the production setup so, but the above is a good compromise if you can't afford that much hardware.

Finally, implement some change control procedures. Putting code into production is a risky thing to do. I don't know what business you are in, but you want to have a specific process and documentation for putting code into production. Trust me, it's worth it, especially when it all breaks and somebody higher up the food chain says "what changed?" followed by "can we go back to the previous release?"

Emacs (1)

scottsk (781208) | more than 7 years ago | (#17386118)

I have used GNU Emacs since I did my first web page in '93 or '94 or whenever. Other tools have come and gone. Emacs works more or less the same on all plaforms. You can use the same muscle-memory keystrokes to edit any file. It hasn't changed since the 70s (you can boot up ITS in an emulator and Emacs' keystrokes are basically the same ones). It supports every new language that's come along like Java and PHP. It isn't going to get bought out, the next version isn't going to be ruined by "improvements", turn into abandonware, etc. It will be ported to whatever the successor to Linux will be. It will support whatever the next hot language is (is Ruby still hot, or is it passe yet?). It's very difficult to make a case for not using Emacs to a professional programmer whose livelihood is on the line and results matter. Fads come and go (is Eclipse still hot, or is it passe yet?) but Emacs is where I go when I need to get work done.

How do you configure SVN for automatic checkout? (1)

sjmac (7414) | more than 7 years ago | (#17386212)

I have a related problem:

I work in a small dev team that produces a specialized server that exposes some of its functionality via a web-UI. The dev-team is fantastic, and does everything right (and still uses CVS!)

Outside of my team we have some people in a pre-sales role, and part of their job is putting together quick customer-specific demos online for potential customers. It drives me nuts that if I have to go in and fix an issue, then my changes will be overwritten if the presales team decide to tweak their local files and reupload.

Presales are always working under time pressure, and are very resistent to any change that doesn't make their job easier, so I need to tread carefully in how I try to force them to use version control for their demo work.

I thought that I might switch off direct access via scp/sftp, and set up a version control system as the primary method of publishing. In CVS, I can do that by adding a line like this to loginfo:

^websites/site1_com/html (date; cat; (sleep 2; cd /home/me/site1.com_html; cvs -q update -d) &) >> /dev/null 2>&1 #$CVSROOT/CVSROOT/updatelog 2>&1
That checks out files to the live site as soon as they are commited to CVS.

Can I do that in SVN too?

Re:How do you configure SVN for automatic checkout (1)

FinestLittleSpace (719663) | more than 7 years ago | (#17386458)

Google using shell scripting with SVN 'hooks'

The Proper Environment for Web Design (1)

fretlessjazz (975926) | more than 7 years ago | (#17399044)

Under an umbrella on a beach in the cayman islands.
Check for New Comments
Slashdot Login

Need an Account?

Forgot your password?