Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Documentation As a Bug-Finding Tool

Soulskill posted more than 2 years ago | from the //-this-is-where-i-forgot-a-parenthesis dept.

Bug 188

New submitter Sekrimo writes "This article discusses an interesting advantage to writing documentation. While the author acknowledges that developers often write documentation so that others may better understand their code, he claims documenting can also be a useful way to find bugs before they ever become an issue. Taking the time to write this documentation helps to ensure that you've thought through every aspect of your program fully, and cleared up any issues that may arise."

cancel ×

188 comments

Sorry! There are no comments related to the filter you selected.

Common knowledge? (4, Insightful)

O('_')O_Bush (1162487) | more than 2 years ago | (#39691211)

I thought everyone knew that documentation describes what you intended code to do, rather than what it actually does.

Re:Common knowledge? (4, Insightful)

jgrahn (181062) | more than 2 years ago | (#39691233)

I thought everyone knew that documentation describes what you intended code to do, rather than what it actually does.

The documentation tries to document what your intentions are, just like the code tries to implement them. The code can fail to do its job, and of course the documentation can too!

Re:Common knowledge? (-1)

Anonymous Coward | more than 2 years ago | (#39691593)

So, I took a shit today, and when I had a look at my masterpiece, in my amazement I discovered an amazing foot and a half coil. Yes that's right, an unbroken coil of shit at least a foot and a half long. What do you think?

Re:Common knowledge? (4, Interesting)

TheRaven64 (641858) | more than 2 years ago | (#39691949)

There's a very old saying:

If the code and documentation disagree, then they're both wrong

The documentation itself is probably not the important bit. The thing that a lot of programmers seem to do wrong is getting the correct ordering of the thinking and coding steps mixed up. Writing documentation first means that you have to do the thinking before the coding, and that eliminates a whole load of problems.

Re:Common knowledge? (2, Funny)

Anonymous Coward | more than 2 years ago | (#39692043)

Kudos to you if you can refactor the implementation in your head before you write it!

Re:Common knowledge? (1)

sourcerror (1718066) | more than 2 years ago | (#39692147)

This might work if you're writing a new webshop. If you're doing something innovative it won't work.

Theory doesn't always work in practice. (4, Insightful)

Barbara, not Barbie (721478) | more than 2 years ago | (#39692029)

The theory was, write the documentation, then code to the documentation.

In practice, that isn't sufficient to reduce bugs significantly, for several reasons.

1. As you develop something, you find that "getting from A to B" sometimes requires going via D instead of C;
2. Other times, you realize that the documentation doesn't completely capture the requirements, and you need to visit both C and D, and maybe Z
3. Still other times, you realize that A is entirely superfluous.
4. "Can you add/change this feature?"

The initial specification should never be too specific about implementation details - that's a mistake that too many people fall for, going with the illusion that they actually can nail down every detail of a non-trivial problem and just throw the spec at "code monkeys." They don't understand that a specification should only say what, not how. Writing "code documentation" before writing the code is writing the "how".

So they can end up with something that meets that spec, but doesn't work either as intended or just flat-out is wrong.

Unfortunately, code, then document (when and if you get around to it) is the reality because, unlike theory, reality is messy.

Re:Theory doesn't always work in practice. (4, Insightful)

St.Creed (853824) | more than 2 years ago | (#39692145)

You need to distinguish between functional and technical specs.

Functional specs are very usefull (if done even halfway right). Technical specs are a waste of time unless you assume by default that you're dealing with incompetents, in which case you're better off saving yourself time, money and aggravation and hire a better developer.

So I do write a lot of functional specs. Even now, in an agile environment, with HUGE time pressure and multi-million penalties for delivering late - just finishing up my final 40 pages (90 pages total, in this 4 week sprint). Why? Because not doing so will make the project much later. Good architecture (system design) specs will make the project about 20% more likely to deliver on time (citation if I can find it again) even if the programmers don't follow the guidelines (interesting, right?). This matches with my experience: if you write decent designs, you are more likely to find the pitfalls before they can bite you in the ass. If you cover all the bases and make sure the business has provided for all scenarios before you get there, your project will run smoother.

So docs may not prevent all the bugs. But it does prevent a large number of nasty stuff before it gets to the stage where it turns into a defect.

Re:Theory doesn't always work in practice. (5, Funny)

ThirdPrize (938147) | more than 2 years ago | (#39692167)

Wrong. First you write the documentation, then you write the tests and then, if you have time, you write the code.

Re:Common knowledge? (5, Insightful)

icebike (68054) | more than 2 years ago | (#39691275)

I thought everyone knew that documentation describes what you intended code to do, rather than what it actually does.

Just as often, while writing documentation on code I just wrote, I've thrown up my hands, and thought "this is so ridiculous and embarrassing that I can't be associated with it", and I went back and re-wrote it to do it the right way. The act of documenting it revealed you left too much undone, or too many situations un-handled.

Any time your documentation reads like you are describing a game of twister [wikipedia.org] you just know the code can't be worth documenting, or even keeping.

But as for finding bugs, I don't know. You may document exactly what you intended, and thought the code did, but still be wrong because of some corner case. Documenting it isn't likely to reveal all of these situations any better than the code itself. A

Re:Common knowledge? (4, Interesting)

ZackSchil (560462) | more than 2 years ago | (#39691319)

Just as often, while writing documentation on code I just wrote, I've thrown up my hands, and thought "this is so ridiculous and embarrassing that I can't be associated with it", and I went back and re-wrote it to do it the right way. The act of documenting it revealed you left too much undone, or too many situations un-handled.

This is by far more common for me as well. I'll find myself describing some really brittle setup process or some common operation that seems to take a thousand steps, get embarrassed over how bad it is, and then go fix the code. More than once, I've written sophisticated automatic parameter selection code because it was easier than finishing the documentation I had started explaining to the user how to set up parameters properly.

Re:Common knowledge? (1)

olau (314197) | more than 2 years ago | (#39692041)

Agreed. That's why, if you writing a library, it's a really good idea you write some documentation before others start depending on the API.

Re:Common knowledge? (1)

julesh (229690) | more than 2 years ago | (#39691951)

But as for finding bugs, I don't know. You may document exactly what you intended, and thought the code did, but still be wrong because of some corner case. Documenting it isn't likely to reveal all of these situations any better than the code itself.

Handling corner cases is accomplished by identifying what they are and specifically implementing code to handle them when necessary. In order to identify them, we must think about what the code is doing. As programmers, there are three times we do this:

- When writing the code
- When testing the code
- When writing the documentation

It happens that we tend to pick up different corner cases in each case. This happens because different corner cases require different ways of thinking to identify them, and while performing these three activities we tend to be thinking in different ways (logically, spatially, and verbally, as a rough approximation). So, yes, I think coding and documenting is likely to reveal corner cases better than coding alone, particularly if we leave a few days between coding and documenting (thus allowing us to forget how we were thinking about the code at the time we wrote it, and get more into the verbal method of thinking about the problem).

Re:Common knowledge? (2)

St.Creed (853824) | more than 2 years ago | (#39692151)

Interesting sidestep: consider Literate Programming (http://en.wikipedia.org/wiki/Literate_programming). Donald E. Knuth advocates the approach of document, AND code, then compile the documented code. I have always been fond of this and try to use this in all my programming.

Just comment preconditions, postconditions and mock up the pseudo-code, then extend it.

I'd absolutely love it if Eclipse and Visual Studio supported this. However, most people don't know Literate programming so I doubt it's going to be in there anytime soon.

Re:Common knowledge? (5, Insightful)

O('_')O_Bush (1162487) | more than 2 years ago | (#39691281)

Article summarised: "Thinking about what you write produces better results than not."

Re:Common knowledge? (2, Insightful)

Anonymous Coward | more than 2 years ago | (#39691343)

Tell that to the PHB that wants the code to ship YESTERDAY--bugs or no bugs!

To me, proper programming is and exercise in minimalism--get the most work you can out of the simplest and least least amount of code.

Then comments/documentation isn't so critical.

But that takes too long and time is money to the average PHB. So you get apps that barely work and must be 'updated' to work better and to be an income stream for the company that put said software out.

But when you have people DELIBERATELY writing needless, convoluted code in an effort to keep their job because they are the only one who can upgrade it--you have the situation we have to day which is addressed humorously by the The International Obfuscated C Code Contest [ioccc.org]

Re:Common knowledge? (1)

DaneM (810927) | more than 2 years ago | (#39691379)

Article summarised: "Thinking about what you write produces better results than not."

Someone mod up parent. :-)

Re:Common knowledge? (1)

Beryllium Sphere(tm) (193358) | more than 2 years ago | (#39692323)

Can we moderate this above +5? There's no need for any other comments after that one.

Re:Common knowledge? (1)

Anonymous Coward | more than 2 years ago | (#39691467)

Some won't follow the coding plan and won't write documentation. We have four software departments and they just walked out 30 people out of 50 across all of them for failure to document and other issues. Six were managers. This happened about 3 months ago. It took them less than two months to fill those positions.

Re:Common knowledge? (0)

Anonymous Coward | more than 2 years ago | (#39691569)

Sounds like good business at work, assuming the new hirees have proven they'll do a better job documenting?

Re:Common knowledge? (3, Insightful)

AuMatar (183847) | more than 2 years ago | (#39691683)

If you really filled 30 positions in 2 months, your problem is likely in hiring shitty programmers. A most companies I've worked at, we made offers to 10-20% of the people who interviewed. Unless you're doing 5+ candidates a day, and all offers are accepting, you're murdering that rate. Some of that may be better offers or more efficiency, but it sounds like you're hiring a lot of mediocre people to fill seats.

Re:Common knowledge? (0)

Anonymous Coward | more than 2 years ago | (#39691551)

The problem is more often maintaining good documentation. Sometimes code and documentation disagree but code is actually right because documentation was not updated when changes were implemented.

Having tests as documentation is much better: you should have them anyway and if you change what the code does you are forced to update the tests (and confirm the new behavior).

Better than customer service (2)

kawabago (551139) | more than 2 years ago | (#39691213)

as a bug finding tool. Lots of unhappy customers that way.

False (0)

Anonymous Coward | more than 2 years ago | (#39691217)

"Taking the time to write this documentation helps to ensure that you've thought through every aspect of your program fully, and cleared up any issues that may arise."

Not true.

Re:False (4, Insightful)

icebike (68054) | more than 2 years ago | (#39691295)

For some methods of documentation this is very true. For some programmers that care about their work its very true.

But if you don't care about your code, you probably won't care about the documentation. In this case, I agree its False.

If you know the documentation you just wrote is a bag of lies but you turn it in anyway, because you know that the PHB won't understand it and couldn't check if it was true if his life depended on it, then you might as well junk the code you just wrote as well. Chance are it will break the minute you walk out the door.

Re:False (1)

David Gerard (12369) | more than 2 years ago | (#39691773)

I must point out that I will never apply that link in practice ... having had a friend end up working somewhere I'd worked four years previously and discover my name in the documentation. Always leave the documentation in the form you'd like to have found before you did whatever it was.

Re:False (1)

St.Creed (853824) | more than 2 years ago | (#39692163)

I once got hired back to a place I worked for 3 years before that, and had to answer questions about the code I designed when I worked there before... fortunately I always document at least the basics diligently but it still made for some sweaty moments. So yeah, it could be your friend. But it could also be you after 3 or more years :)

Re:False (1)

expatriot (903070) | more than 2 years ago | (#39691927)

In the article someone commented on the difference between business needs and developer needs. This is why I favour doc generating systems like doxygen.

Part of the problem might be brain organisation, some people think very spatially and words get in the way. The code and fix as fast as you can does work if you are good, lucky, or fast. If so you probably do not want to doc every iteration.

Indeed (2)

bazald (886779) | more than 2 years ago | (#39691219)

Explaining your work is a great way to demonstrate that you actually understand it. As the article illustrates, perhaps the most critical part is going back and verifying that the code matches the explanation.

Re:Indeed (2)

paimin (656338) | more than 2 years ago | (#39691247)

This is news? Wow, the human race is going downhill.

Re:Indeed (1)

Jens Egon (947467) | more than 2 years ago | (#39691447)

Stuff that matters, surely. Certainly not news.

Re:Indeed (3, Informative)

grcumb (781340) | more than 2 years ago | (#39691293)

Explaining your work is a great way to demonstrate that you actually understand it.

My standard development process is:

  1. Document the method;
  2. Write the tests for the method;
  3. Write the code for the method;
  4. Make corrections to each until everything works;
  5. Move on to the next method.

My rationale is precisely that: I'm not really sure I know what I'm doing until I've described it, then figured out how my idea might fail.

Forgive my ignorance, but doesn't everyone do this?

Re:Indeed (3, Interesting)

purpledinoz (573045) | more than 2 years ago | (#39691473)

I think most people to this:

Write method.
Write unit tests.
Fix until it works.
Write documentation.

I generally find that code documentation is a lot better if the document first approach is used. In the above approach, the method is done, and the documentation becomes a chore.

Re:Indeed (1)

OddJobBob (1965628) | more than 2 years ago | (#39691781)

The process I have seen followed by the most senior developer in my last company was more like:
Write code
Don't write tests
Don't put a label on the trunk
Commit hundreds of files to the trunk
Write specification
Write requirements
Run out of budget
Trunk now abandoned for other projects

He did work "at a higher level to all of you" so what did we to know!

Actually, I think most people do this: (2)

petes_PoV (912422) | more than 2 years ago | (#39692107)

Sit around trying to make sense of the requirement
Write some code that they think does what the requester really meant
Spend most of their time fighting the code management system and getting it to build cleanly
Pull an all-nighter just before the deadline so it doesn't crash when fed correct input
Toss it "over the wall" to the integration team
Refuse to answer any questions about it as they're not "too busy" on the next project

Have a nice feeling of satisfaction that they never have to do support on old code, as none of it ever gets into a production system
Get promoted frequently for meeting their targets on productivity and delivery times

Re:Indeed (2)

Jah-Wren Ryel (80510) | more than 2 years ago | (#39691543)

Forgive my ignorance, but doesn't everyone do this?

Yep, pretty much every one does do that.
Where they tend to differ is in their particular definitions of "do."

Re:Indeed (0)

Anonymous Coward | more than 2 years ago | (#39691677)

I don't. I'm a pretty intuitive sort of programmer -- probably not ideal, but it's who I am -- and it is very difficult for me to write tests or documentation before I write code. My intuition works well on the level of code, but not on the level of tests or documentation. For me test writing usually involves me thinking about the major use cases of a particular (mostly finished) piece of code, and also what I could do to break it. I implement tests based on what I come up with. Code documentation I sometimes add as a I go, sometimes add after I'm finished (generally simple code gets on the fly documentation complex code does not). User level/ API documentation gets written when everything else is done.
Again, I don't say that this is the ideal process, but, for me, it does work reasonably well -- and much better than the method you describe.

Re:Indeed (0)

Anonymous Coward | more than 2 years ago | (#39691765)

I am more of a holistic type of programmer/designer. I think about what I have to make for quite a long time, scribble down some schematics, think about it more. I mostly design all my stuff in the shower in the mornings. You could say I am doing test-driven development in my head.

Then, when I thought of how to make the program as simple as possible with many ways to expand on it later, I start programming it immediately. When I am done with writing some functions I start to see if I can make the implementation of each function better. Possibly I split the function up in separate functions until my code becomes mostly self documenting, and I can share the most amount of code. I write my function headers after this process since during it everything is still very much in flux.

Most of the times I do not put comments inside the function, except when I write low-level code, especially with wait-free code it is important that people don't mess with the order and type of intrinsics in those function, even if it looks like one can optimize it.

I read a great comment on slashdot recently: "in code comments: don't explain what you are doing, explain why you are doing it."

I only make unit tests if a function has failed, in an attempt to debug the code. Then when I have fixed the code, I actually have a regression test as well. This way the number of unit tests grows and it will cover actual real world situations. On the other hand I don't often write code that needs to be perfect in one go either, I rather have it working quick than be perfect :-)

At home right now I have a large project where I am reinventing literate programming, the Knuth version went way to far in it. I have not gotten too far yet, I write my documentation and some code in straight HTML. Then I used attributes of tags to tell the code generator what to do with it. For example I have tables with constants in it, which are converted into constants/enumerations code files for languages like C and Python. Or I describe a database table which are converted into DDL SQL. HTML is great for this, simple to parse (I use XHTML), nicely nested, expandable with attribute. This way documentation and code keeps in sync, like the reverse of Doxygen, I call it Noxygen.

Re:Indeed (0)

Anonymous Coward | more than 2 years ago | (#39692031)

Forgive my ignorance, but doesn't everyone do this?

No, the boss says "All You Have To Do Is..." and "Git 'er Dun!", and "By the way, we also need this other stuff", and you hack something out, run a few hasty tests, then dump it into production.

The users complain it's crap, but try and get a decent budget for a decent effort. Especially since they've been told that Offshore Corporation X can do it in half the time and 1/4 the price.

Re:Indeed (1)

boaworm (180781) | more than 2 years ago | (#39691935)

Explaining your work is a great way to demonstrate that you actually understand it. As the article illustrates, perhaps the most critical part is going back and verifying that the code matches the explanation.

Explaining your work by writing it down is fine, but if noone reads what you have written, it isn't as useful anymore. Hence, it is not the documentation part but rather the reviewing part that helps. Hence, what really does the job is code reviewing, rather than documentation. If you document it before reviewing, thats even better.

The key thing here is that the person who wrote the code isn't likely to find bugs in it, as that person "knows" how the code works. The important thing is to make someone else look through it, or at least have the author explain it to someone else. Most coders will for sure have experienced the as-soon-as-i-explained-it-to-my-college-i-found-the-bug-phenomena.

I'd believe it... (1)

tecker (793737) | more than 2 years ago | (#39691241)

I have caught a number of problems documenting my code. When you describe what it is supposed to do and you realize that it really doesn't do that then you can correct as such. I would say I have found more incorrect behavior rather then show stopping bugs. However if we had shipped the product with the code the way it was we would have probably called it a bug so it is probably the same either way.

IS0 9000? (4, Insightful)

jayveekay (735967) | more than 2 years ago | (#39691263)

Say what you do.
Do what you say.

The documentation becomes an error check on the code and vice-versa. If the 2 disagree, something needs to be fixed.

Re:IS0 9000? (1)

Coryoth (254751) | more than 2 years ago | (#39692149)

If you are trying to save time you can always use a DbC system for some of your "documentation" of what the function is intended to do and have that become an actual error check on your code. You can even use the contracts to automatically generate unit tests for you. It's also harder for documentation to fall out of sync with code since it is part of the testing and flags an error if it isn't kept up to date.

For fucks sake (-1)

Anonymous Coward | more than 2 years ago | (#39691273)

Thinking about your work improves it. Thank god someone pointed that out!

Dickhead.

Plato? (0)

Anonymous Coward | more than 2 years ago | (#39691277)

What TFA suggests is not new. But it astounds me how close this is to what Plato was saying 2500 years ago.

This (1)

Anonymous Coward | more than 2 years ago | (#39691299)

We use a commercial mail processing application.
The documentation stated that the feature works 'this' way.
Which was to route mail to and from third party applications.
It only partially did this, it wouldn't route it back from the application. Understand they they also integrate a optional licensed virus processing feature.
We contacted the company, and they said it worked 'that' way. 'That' way was far less usefull or functional.
We pointed to the documentation and they corrected the problem.

Reality Sucks (0)

Anonymous Coward | more than 2 years ago | (#39691305)

In my utopian world, the code is well documented and thought out. In the mutli-million dollar/SLOC world, it is a mountain dew/provigil fueled race to the finish line and payday. As schedule, cost and requirements slide in the middle of a project, the documentation is often left in the dust while we scramble to successfully compile our deliverable.

For debug, depending on the system requirements, the test software has even less documentation than the original software.

Documentation and Planning (0)

Anonymous Coward | more than 2 years ago | (#39691307)

Please don't confuse documentation with actual planning. If you think documenting your code is helping to prevent bugs then you're most likely not doing your planning properly. There's a reason we do DFDs, function models and interaction, behaviour and structure diagrams. Use a better development methodology.

I'd fire that guy if he was on my team (0)

Anonymous Coward | more than 2 years ago | (#39691315)

Writing documentation after writing the code shows that he doesn't exercise TDD, either.

Re:I'd fire that guy if he was on my team (1)

julesh (229690) | more than 2 years ago | (#39691955)

Writing documentation after writing the code shows that he doesn't exercise TDD, either.

He could be writing *extra* documentation. Not everyone is happy with the idea that tests are documentation.

Documentation IS NOT a Bug-Finding Tool (0)

Anonymous Coward | more than 2 years ago | (#39691323)

The only reason you find bugs when writing documentation is because you are creating a duplicate set of logic in a different format which should match your source code logic. You would get the same exact result if you wrote the same application in another language, say C# for instance. It's amazing the bugs for find when you verify your logic in multiple formats.

Someone teach this guy about circular buffers STAT (0)

Anonymous Coward | more than 2 years ago | (#39691335)

FTFA:

public void doA(int num) {
        for(int i = 0; i < array.length - 1; i++) {
                array[i] = array[i+1];
        }
        array[array.length - 1] = num;
}
public float getB() {
        float total = 0;
        for(int num : array) {
                total += num;
        }
        return total / array.length;
}

There's no excuse for copying array.length - 1 items on every update. Someone teach this guy about circular buffers, STAT. Then maybe he'll have something worthwhile to blog about.

Re:Someone teach this guy about circular buffers S (1)

commlinx (1068272) | more than 2 years ago | (#39691367)

Plus he mentions concurrency and the confusion it may cause for other developers re-using the code, but doesn't address the obvious divide by zero that will occur if getB is called first even in a single-threaded application.

Re:Someone teach this guy about circular buffers S (2)

donscarletti (569232) | more than 2 years ago | (#39692171)

Crap coders tend to focus on incidental things that they feel will improve the quality of their code, rather than addressing the issues directly.

Wouldn't hire him.. (0)

Anonymous Coward | more than 2 years ago | (#39691339)

His examples seem to be based on having documentation help poor design. Knowing what you feed to your methods is a basis for programming. If you can't trust that (external interface), be ready to catch the exceptions.

He also forgot the bit about testing; if you're writing stuff up without a design, try at least to write some testcases so that you can verify classes to work individually. Rather than piling them up like a card house..

Good code (2)

pwnyxpress (2597273) | more than 2 years ago | (#39691349)

If your code is good, it should be self documenting. Everyone knows what the variables 'foo' and 'bar' do.

So, someone rediscovered Literate Programming? (3, Informative)

Elf Sternberg (13087) | more than 2 years ago | (#39691351)

Oh, come on, Literate Programming has been around for 30 years! Knuth made exactly this argument in his 1984 essay entitled, surprisingly enough, "Literate Programming!" Wikipedia asserts in it "Literate Programming" entry: "According to Knuth, literate programming provides for higher-quality programs, since it forces programmers to explicitly state the thoughts behind the program, making poorly thought-out design decisions more obvious. Knuth also claims that literate programming provides a first-rate documentation system, which is not an add-on, but is grown naturally in the process of exposition of one's thoughts during a program creation. The resulting documentation allows authors to restart their own thought processes at any later time, and allows other programmers to understand the construction of the program more easily."

Congratulations to Slashdot for posting about some kid rediscovering an ancient technology by a revered master of the craft. What's next? "Snot-nosed highschooler discovers GOTO is a bad idea?"

Re:So, someone rediscovered Literate Programming? (1)

julesh (229690) | more than 2 years ago | (#39691969)

Actually, he's talking about something entirely different to literate programming. LP advocates writing a single document that contains both documentation and code, whereas what this guy's actually advocating is basically finding a reason to take a second pass at looking at the code, which is what happens when you document it separately. You'd get the same benefit from, for example, having a policy of self-reviewing code after completion, or in TDD the refactor phase of the red-green-refactor cycle.

Re:So, someone rediscovered Literate Programming? (1)

St.Creed (853824) | more than 2 years ago | (#39692177)

You're right. Knuth is still much more advanced than this writer.

Peer Review (0)

Anonymous Coward | more than 2 years ago | (#39691375)

Peer reviewing all code before it goes into the main repository also does the same thing. And the upside is you don't need to waste any time writing documentation!

I must be in bizarro world.. (0)

Anonymous Coward | more than 2 years ago | (#39691397)

My life must just abnormally suck, I guess. I have no time for documentation. I barely have time to get code that mostly works done before I have to move on to some hot bugix, or I have to move on to deliver some feature the customer needs 2 weeks ago.

I always longingly read people talking about test driven development, or code contracts, or good code documentation. I mean I love all that shit. Sure would be nice if I could do it in the real world.

Either I have an extraordinarily shitty job or some people just have dream jobs where they have time to write this mystical "quality code" with documentation, tests, etc...

Either way, if you have time for that shit I fucking hate you.

Re:I must be in bizarro world.. (0)

Anonymous Coward | more than 2 years ago | (#39691597)

Change job. Go to some small company where there is less than 12 people total. If you like what you see when you first speak with owner and other employees, this tends to be a good job. I had been in the same situation, now I've changed jobs and it's better pay, work atmosphere and more interesting projects.

Re:I must be in bizarro world.. (2)

St.Creed (853824) | more than 2 years ago | (#39692209)

I understand, but at least do this:
- keep a change log. Even when I'm under extreme pressure I write down at least what the customer wants, and what I'm going to do about it. What did I change where.
- keep a decision log. Every time you have to interpret a request or design spec in a certain way, write it down.

Over time, these will keep you afloat.

This is why I comment code last. (1)

Anonymous Coward | more than 2 years ago | (#39691407)

This is why I got into the habit of not commenting code as I was writing it. I always saved the documentation as the last step before getting it ready for code reviews or committing it to version control. By documenting last you don't waste time writing documentation that may end up getting deleted because you decided on a different solution, doesn't get out of sync with what the code is actually doing, and it serves as a forced code review for yourself.

Documemtation Priority (4, Insightful)

PolygamousRanchKid (1290638) | more than 2 years ago | (#39691413)

Manager, at the beginning of a project: "Forget the documentation! Just get it to run!"

Manager, at the end of a project: "Where's the documentation! You were lazy, and didn't write any!"

Documentation is at the ass-end of a project. The Manager's Manager wants to see something running. He doesn't accept paper as a currency. So documentation will always get low priority. And that ass-end will be hanging out and swinging in the breeze.

Someone could do a scientific study that proves that documentation cures cancer.

It will still get low priority in a software development project.

Re:Documemtation Priority (1)

Patch86 (1465427) | more than 2 years ago | (#39691893)

There are two types of manager (and indeed two types of company) out there. The kind you describe, where everything needs to be done yesterday, damn the protocol, is one kind. Ones where everyone sticks dogmatically to bureaucracy and obsesses with "project gateways" is the other.

When you're stuck with the former, the common reaction is "for god's sake, if they won't let me follow procedure properly the code won't be any good at all!". The reaction to the latter is usually "for god's sake, if they obsess over the paperwork so much I'm never going to actually code it at all!".

I guess you just make the best of the hand you're dealt with, and don't fantasise about which way would be best. As an analyst, I always prefer to do the paperwork properly; but most coders out there would probably feel the opposite.

Re:Documemtation Priority (1)

wrook (134116) | more than 2 years ago | (#39691897)

It's easy to simply dismiss this as poor management (and it is), but solving the problem is often a bit trickier than replacing the manager. The biggest problem is that a really large segment of the software industry does not know how to connect business goals with software development activities. It's not just the managers who are clueless, but also the programmers. Many programmers do not consider business goals further than, "If I do things *right* then it will be better in the long run" (for various descriptions of the words "right" and "better").

What I've tried to do in the past is to encourage my manager to worry exclusively about business goals while I worry exclusively about maximizing through put. It's important to stress to the manager that you want to achieve maximum through put, not just maximum velocity. If you are running a marathon, it does you no good at all to break the world record in the 100 meter dash. You need to achieve the highest *average* speed possible over the entire course. This is done by intentionally slowing down and paying attention to things other than that when running a short distance. It's important to illustrate that the world record holder in the 100 meter dash would probably finish a marathon several times slower than even an average marathon runner.

Having said that, there really are times when you have to sprint. It does you no good at all to get tremendous throughput and finish on Friday when the company runs out of money on the Monday before. But just like a sprinter, after we sprint, we have to stop and recover. Sprinting continuously leads to death.

What I tend to do is to be very respectful when my manager asks me to sprint. I tell him that we will be happy to sprint for him. I tell him how far we can sprint and how fast we are likely to be able to run. I tell him the cost in terms of productivity (i.e., after this sprint, we have to stop and spend at least an equivalent time recovering). I let him choose, because that's his area. If he says we have to do something that I think is impossible, I agree to try my best. But I ask him to find another business plan, because this one will almost certainly lead to failure. It's his call and after he makes it, I will never say another word about it.

Finally, if my manager tells me that he thinks we can go faster with another method, I will certainly listen to his ideas. If I disagree, I will tell him that I don't think it's a good idea. Getting the highest productivity that meets the business goals is my job. If he does not have confidence in my ability to do that job, then he should certainly find someone else for the job.

Once we have an understanding in place, I've never had any trouble introducing any development practice I want. But my ass is on the line if we don't hit the business goals. And that's the way it should be.

Re:Documemtation Priority (1)

St.Creed (853824) | more than 2 years ago | (#39692227)

Good practice. I use it too and it works great for me as well.

Re:Documemtation Priority (1)

gl4ss (559668) | more than 2 years ago | (#39692183)

in reality, in modern changing by half year update platforms.. you first write it to see if it can be written. then you ship it.

it's sad. but wtf, in which gigs can you actually document what you're going to do beforehand? where you'd have the specs at the start of the project - "this is what we want and we checked it can be done".

I suppose with financial backends etc db stuff you'd know that.

do you know what kind of "for end user" documentation I really hate though? the lying kind. press here for more information and then you don't get more information.

Side note... (1)

DaneM (810927) | more than 2 years ago | (#39691417)

"As an added benefit, documenting your code makes it possible for people who didn't write the program to use it."

Yes, this seems obvious, but it's not. It's a Zen koan. Really. It's a well-known fact that koans are more fun to write and think about than documentation.

Dance with the pinch pine blossom [wikipedia.org] .

Knuth (4, Interesting)

phantomfive (622387) | more than 2 years ago | (#39691437)

I believe this is the method Knuth recommended, and formed the beginnings of the idea of literate programming [wikipedia.org] .

Re:Knuth (1)

wrook (134116) | more than 2 years ago | (#39691797)

I've always wanted to write "tests" in a literate way. For me tests are a way to document the behaviour that I expect and various assumptions along the way. If the behaviour changes, or if I do something that disregards the assumptions I made, I want the tests to fail.

I feel that I *should* be able to organize this in a literate way. If I want, I can even write human language documentation in the same place. The testing framework provides the same purpose that Web (or its equivalent) would -- it allows me to organize my discussion by design or by behaviour. The added benefit is that it is executable and so tells me if the production code is meeting my expectations. It also has the benefit of providing examples of how I expect the code to be used.

The problem that I tend to run into is that I have too much coupling between the implementation and the tests. As I refactor code, the discussion I make with the tests drifts and I end up having to refactor the tests considerably more than the production code. To combat this, I have made a practice of separating my unit tests from my larger scale behavioural tests, but I still end up with a lot of churn whenever I'm refactoring. Because of this churn, it's quite easy to lose track of where I'm discussing what issue and my ability to use the tests as literate documentation slowly disintegrates.

I've never actually done literate programming in the proper sense, so I wonder if people who do run into the same issues. I suppose it is possible they just try to avoid refactoring, which would be unfortunate.

Re:Knuth (1)

St.Creed (853824) | more than 2 years ago | (#39692245)

The idea is interesting but the tight coupling makes it difficult to check behaviour, unless you test the output on screen. Perhaps your tests are too granular for that?

Documentation as a static code analysis (1)

Anonymous Coward | more than 2 years ago | (#39691491)

I believe the opinion of the OP is that documentation forms a powerful technique for static code analysis. This I think is likely, although there are better tools such as peer review, and static code analysis tools.

I have to ask "at what moment in time did you write the documentation?". The purpose for the question is to point out that code changes across time. There are complex links between the documentation and code. These links are either correct, or in error. If they are in error we could categorise them as "bugs". True they are not bugs in the traditional sense: they are not the sort of bugs that can be detected and prevented with tests, or that affect the execution of the program, but they do affect the cognitive integrity of the code, and influence the developer.

Let us be charitable and assume that no bugs were present at the first release. Then changes are made. Fortunately for the developer, IDEs and refactoring tools exist that allow the developer to identity areas of the code that likely to be impacted. In a scenario in which code reviews are mandatory, other peoples intellect are present to help reduce the chance of consequential changes. The developers test suite is present to help prevent regression errors.

What about the documentation?

The links between the documentation and code are complex, and difficult to understand in their entirety: the nature of plain text is that facts are repeated all over the place. There is no automated way to check this integrity. Even in a world in which you review all the code and documentation, you have little or no tool support (unless you use a requirements management tool, which are expensive in developer time) to help you. Effectively you are adding a manual testing stage to each release.

So lets look at the costs:
In my experience documentation adds around 50% to the cost of coding. The release cycle time deteriorates, and code is released into the user community much slower, which results in a poorer quality product. More defects are introduced.

Lets look at the wins:
It is clear that the documentation for an API makes a massive difference to the usability of the API. I have yet to be convinced that documentation of the code enhances that maintainability of it. The OP is correct I feel in his assertion that documentation forces a peer review.

Personal conclusion:
Document your APIs as thoroughly as you can. Spend the effort you would have spent on documentation on more effective peer reviews, and more ruthless code committing practices that ensure that the code is more maintainable.

Re:Documentation as a static code analysis (1)

rollingcalf (605357) | more than 2 years ago | (#39692087)

"It is clear that the documentation for an API makes a massive difference to the usability of the API. I have yet to be convinced that documentation of the code enhances that maintainability of it."

Rather than enhancing the maintainability per se, the documentation helps with letting you know what needs to be maintained. In particular, a block of code may look perfectly error-free without looking at the documentation, but you only realize something is wrong when the documentation doesn't match what the code is actually doing. That then triggers further investigation to determine whether the documentation or the code is wrong or both, whereas without the documentation you might have glossed over that block of code because it looked logically correct by itself.

How to do documentation (1)

Anonymous Coward | more than 2 years ago | (#39691507)

Most programmers complain about documenting because they've never been told the correct way to do it. Well here it is motherfuckers:

    1) Requirements need to be written and have a unique id. Functional reqs = what is needed, non-functional = constraints on how things shall be done.
    2) Design document = plan for fulfilling each functional req, each design configuration item has a unique id and references the id of the req it is fulfilling.
    3) Architecture document = plan for fulfilling each non-functional req, each architecture item has a unique id and references the id of the req it is fulfilling.
    4) In-code comments = id reference to the design/architecture/requirement configuration item the code is implementing.

Re:How to do documentation (1)

davide marney (231845) | more than 2 years ago | (#39692021)

This is actually a pretty good list, but why are you posting as AC? AC posters have no Karma. You should get credit when you post insightful things like this.

Re:How to do documentation (1)

St.Creed (853824) | more than 2 years ago | (#39692281)

I'm not sure that Architecture is just about the non-functional requirements. In my opinion, the Architecture doc is about ALL the constraints on the solution from an implementation point of view, not just the technical constraints.

Your list does make sense though. Perhaps I should reconsider my opinion.

Only proprietary software needs documentation (-1)

Anonymous Coward | more than 2 years ago | (#39691513)

Software using open standards for configuration and data uses open standards that have been documented already.

Everything else is nitpicking by incompetent people which do not belong in the audience anyway.

Re:Only proprietary software needs documentation (1)

Zontar The Mindless (9002) | more than 2 years ago | (#39691697)

Everything else is nitpicking by incompetent people which do not belong in the audience anyway.

I'm sorry, you seem to have written incompetent people where you really meant to say,

Devs who need to make changes but who came on board after the original code was written;

Support techs who have to support and debug code they themselves didn't write;

QA folks who are responsible for making sure a release does what we claim it does before it gets out in the wild;

Managers who like to have some assurance that when they tick the little box labelled Task Complete , the task they've just signed off on is actually complete and correct;

Customers who are trying to understand why the software does [foo] when they expected it to do [bar];

Community members who'd like to contribute patches;

Tech writers who have to write end-user documentation.

Re:Only proprietary software needs documentation (0)

Anonymous Coward | more than 2 years ago | (#39691901)

No need to be sorry.

If you look closely you'll see that I have also written: Software using open standards for configuration and data uses open standards that have been documented already.

Devs can read that documentation.

Support techs can read that documentation.

QA folks can read that documentation.

Managers instruct Devs to change realities to resemble their incompetent believes.

Customers will never read documentation but, instead, contact/bother QA folks, Support and Devs.

Community members can read that documentation and contact Devs.

Tech writers write what they want (see Slashdot summaries).

Welcome to reality. Now mod it down as much as you want.

Re:Only proprietary software needs documentation (1)

julesh (229690) | more than 2 years ago | (#39691995)

Reality? The reality is that a design document written before coding starts is likely to never be accurate enough to perform the kind of annotation you're talking about, because as soon as coders sit down with it to actually implement stuff, they'll realise the design missed some crucial point of logic about how the application should work. And as soon as the code is demonstrated to the customer, the customer will point out misunderstandings about the design. And as soon as you start changing requirements your entire cross-referencing scheme starts taking more time than actually implementing the software does. The developers begin to hate it. The managers start to hate the developers because they can't stick to an apparently simple scheme. QA wonders why the software that has been produced appears to be different to the software they expected to be testing (because the coders had to update the spec to fix logical ommissions, but nobody forwarded the revised spec to QA), and starts filing bugs against correct behaviour. Before long the entire system falls apart and people are bickering about whose fault it is.

Been there, done that, and it's the last time I ever work for an FTSE250-listed corporation.

CS Research From the 1950s Onward Called (1)

jfz (917930) | more than 2 years ago | (#39691525)

Isn't this precisely what specifications written in formal logic was intended to solve? http://en.wikipedia.org/wiki/Formal_verification [wikipedia.org]

Cool (1)

DeBaas (470886) | more than 2 years ago | (#39691563)

Cool, the software we have to implement hardly has any documentation, so this must mean hardly any bugs ;-) Knew there had to be a reason why the documentation is hardly existing...

This article from 1996 never gets old (4, Interesting)

sn00ker (172521) | more than 2 years ago | (#39691573)

Titled They Write the Right Stuff [fastcompany.com] it looks at the coding practices at the company that wrote the control software for the space shuttles. If you want to know about documentation as a bug-finding tool, this is pretty much the holy grail.

Consider these stats : the last three versions of the program -- each 420,000 lines long-had just one error each. The last 11 versions of this software had a total of 17 errors. Commercial programs of equivalent complexity would have 5,000 errors.
...
Take the upgrade of the software to permit the shuttle to navigate with Global Positioning Satellites, a change that involves just 1.5% of the program, or 6,366 lines of code. The specs for that one change run 2,500 pages, a volume thicker than a phone book.

Maybe it is the gaming industry (1)

Paxinum (1204260) | more than 2 years ago | (#39691617)

Part of what got me into computers and programming was computer games. I started with just messing around with config files for games, and proceeded to creating character editor programs for Diablo II. Gaming is a male-dominated area (see how many female pro-gamers there are in Starcraft, for example, and compareto the number of men that are pro-gamers). Somehow, the gaming scene is very much oriented towards men. The question is, is there a biological reason why females are not as much into gaming, or is it a society problem? Are females raised in an environment, where men are considered more childish, and therefore it is considered ok to be a gamer, while females are supposed to be mature? In my opinion, learning to do programming is about letting your inner child loose, and explore unknown terrirory. Boys are encouraged to play when they are young, while girls are supposed to be well-behaved and well-developed. Certainly, this has some advantages for girls, as it seems like interaction with other people are encouraged more, and women are more common in such occupations, while boys seems to be allowed a wider range of games, where more "childish" aspects are allowed.

The term "documentation" is subjective (5, Interesting)

ciaran.mchale (1018214) | more than 2 years ago | (#39691631)

There is an old joke: "The definition of promiscuous is somebody who has more sex than you do". From reading TFA and some of the comments on slashdot, I get the feeling that the definition of documentation is equally subjective and self-serving for developers. Some developers think that writing documentation means adding comments to code. Others feel it involves writing Javadoc/Doxygen-style comments at the start of every class and method, and then generating HTML from that. Yet others feel that documentation hasn't been written unless it involves an architectural description.

When I am working on my own open-source projects, I feel that documentation isn't complete until I have written a few hundred pages of text that aim to be cover most/all of the following: (1) API reference guide, (2) programming tutorial, (3) user guide, (4) architectural guide, and (5) suggestions for "future work" that I hope other people will volunteer to do. Yes, I recognise that I am a bit extreme in the amount of effort I put into writing documentation. However, it does enable me to elaborate on the thesis of TFA: attempting to write such a comprehensive amount of documentation often highlights not just coding bugs, but also architectural flaws. This causes me to work in an iterative manner. I implement a first draft version of the code. Then I start documenting it, and when I encounter a part of the software that is difficult to explain, I realise that I need to re-architect the code base a bit. So I do that, and then get back to writing documentation, which causes me to notice another difficult-to-explain issue with the code. Working in this manner is slow, and I suspect it wouldn't work in a business with time-to-market pressures, but I find it gives excellent results in my own, non-time-pressured open-source projects. I touched on this issue [config4star.org] in the documentation for one of my open-source projects.

Re:The term "documentation" is subjective (1)

olau (314197) | more than 2 years ago | (#39692071)

Are you sure you're really receiver-focused when you write all that stuff? Most people don't want to read that much text to use, say a configuration parser. If it takes people 10 hours to dig through your documentation and 1 hour to actually write the code, you're probably not doing it right. Sometimes less is more.

Reference documentation is a bit different because people to some extent can just go directly to what they need. But in my experience, most people just want something they can copy-paste into their project and get started with.

One problem with producing too much stuff yourself is that you tend to build up your own little bubble where everything makes sense. In theory. :)

Re:The term "documentation" is subjective (0)

Anonymous Coward | more than 2 years ago | (#39692093)

Not at all. We wright pretty damn good code. It's not quite space shuttle quality, but it's the same order of magnitude. We use the term "requirement traceability" instead of "documentation" since we do the documentation first. However, if you read RTCA DO-175C, you'll see that there's a good way to write life safety critical code. We don't start with a programming language; picking that it later. We start with a definition of what the system will do.Then we describe the requirements of the system. Identifying edge cases is interesting, when you have mechanical sensors that can fail in creative ways, but it must be done. Then we decompose those requirements down to the module of code level. It's a waste of time if you're writing another Angry Birds knockoff. However, if your code is the flight controls for an angry bird, well, it's important. That being said, though, writing code is remarkably fast when you know what the hell you want to do and how it will play with the rest of the program. Yes, even our fucking idiot retarded Northrup Grumman can get it right. Most of our code maintenance now is loosening up parameters so that it will accept more risks, now that we've got numbers to support that.

Found some bugs (1)

roguegramma (982660) | more than 2 years ago | (#39691715)

I found some bugs documenting my own code lately, but I think that only works if you write the documentation a month or so after you wrote the code.

Documentation requires a human reader (0)

Anonymous Coward | more than 2 years ago | (#39691735)

Good code should require little low level documentation to describe it and of you spend your time writing tests surely they document the intent of your code and can be tested by a computer.
Personally I would prefer to see high level design drawings/sketches, tests and code I can read.

Not really sure ... (1)

thoughtspace (1444717) | more than 2 years ago | (#39691763)

All in favor of the idea; but many boundary conditions are dependent on the implementation. If the documentation goes down that far into the implementation level - well ... it becomes code.

Aehm, why is this news? (1)

gweihir (88907) | more than 2 years ago | (#39691825)

I was taught that in my first year CS studies, when we were required not only to write programs, but to clearly document the algorithms used. Are there really people that write software and do not know this? Well, I guess there must be. Supports my claim that the only real problem the human race has is too many idiots.

Re:Aehm, why is this news? (1)

St.Creed (853824) | more than 2 years ago | (#39692307)

Ratio of CS educated programmers to random people picked off street and cleaned up: 1 to 100.

Yeah I'm making that up, but in the course of my career I've worked with hundreds of developers and I think I've only encountered 1 or 2 CS graduates in my work. I meet more of them socially than I've ever seen in the workplace doing real work (not managing the IT dept.).

So 1:100 might be an optimistic estimate.

there are 10 types of people (0)

Anonymous Coward | more than 2 years ago | (#39691845)

there are 10 types of people. those who read binary and those who don't.

why bother with documentation if you could read binary?
why bother with binary if you can read code that can be compiled to binary?
why bother with code if you can read documentation that can be compiled to code?

so why isn't there a code that is human readable and can be compiled to code? because its too damm hard for humans to do it. and if humans cannot compile code from documentation well, why would you invest in a process the yields no extra power, just extra place to introduce bugs?

documentation will not help, if it is not fused into the code.
you need to ponder before you go into coding? go talk to pinky.

Pick descrisptive variable names (2)

Required Snark (1702878) | more than 2 years ago | (#39691857)

This is another good documentation tool, and a way of avoiding bugs. It is surprisingly hard to do.

If you can't think of a good reasonably short and descriptive name then you don't understand the concepts as well as you should.

I only use variable names like i,j,k for loops. I use x1,x2,y1,y2 and similar names only for numeric values. This is applicable when I am implanting math algorithms. If I have a lot of similar variables differing by their last digit and I'm not doing equations, I know I am writing code that I won't be able to read later.

I tend to declare one variable per line, and describe what I am using it for as a comment. If I have a lot of variables I split them into groups, which I separate by blank lines.

I try and avoid reusing intermediate variable names, unless they are in different lexical scopes. It is fine to have similar name inside loops that doing similar work, but make sure that you are not confusing concepts when reusing variable names thie way.

I have been working on algorithms, and have stopped and spent an hour or more thinking about what to call the variables. I do this when I get confused. It always pays off. When you have a good descriptive name and you see it in it's use context, you can actually see the mistakes before you make them.

Re:Pick descrisptive variable names (2)

davide marney (231845) | more than 2 years ago | (#39692061)

Even better:

1. When naming entities, pick the SAME names that the business people use when they talk about their domain. Do they call that thing you're working on an "XYZ Thingy"? Then that's the name you should give that entity in the code, formatted according to your conventions for names. Why? Because then your code aligns with the requirements, models, and documentation, no translation needed.

2. When naming methods, use GENERIC names for generic actions, or build names from combining generic action verbs with entity names, sort of the way the German language builds compound words. The entire web, for example, runs on a handful of method names, and the top three (HEAD, GET, POST) account for 99% at that.

This is the domain-driven design way to do it.

Another Anecdotal Datapoint (1)

mattpalmer1086 (707360) | more than 2 years ago | (#39692101)

I'm currently writing up API documentation for a large code branch which was never properly documented (and wasn't written by me), but now needs merging into trunk. I've found several serious bugs in the code as a result, all from trying to explain to the client how to use the API. These bugs were actually blindingly obvious when the behaviour of the code had to be explained.

I've also found some horrible design issues, where various settings the code allowed were contradictory or meaningless, or one setting overrode the behaviour of another, unless something else had been configured, in which case... you get the idea. As soon as you try to explain it, the awkwardness makes the design problems incredibly obvious - and in fact a much better way of doing it is also obvious. I can almost picture the evolutionary process by which this code was written, with each developer adding a new feature without regard to how the whole thing hung together.

So documenting after the fact can definitely detect bugs and design weaknesses - although I don't think this is an ideal way of doing so. Having said that, I'm not sure forcing the developers to document their design beforehand would have helped either, as a lot of developers simply regard documentation as a necessary evil (assuming it is enforced), and will simply write down whatever it is they intend to code, no matter how awkward the result.

I guess you have to have a mentality which loves elegance at all levels, not just in the specific lines of code that are written but how the system as a whole functions. Unfortunately (and surprisingly to me), many developers don't seem to care about the bigger picture, even when they have a deep appreciation of clever or elegant code.

If you can't explain it, you don't understand it (1)

time961 (618278) | more than 2 years ago | (#39692267)

Attributed to Albert Einstein, the full quotation sometimes reads “If you can't explain it simply, you don't understand it well enough yourself". He was a smart dude.

The most useful type of documentation is about intent and goals: WHY does the software have these interfaces, what are they supposed to accomplish, what is the overall model of operation, etc. That, I think, is where the best bugs are found. If the model isn't complete, then the code won't be. And that kind of documentation isn't bulky or repetitive--it has a very high return on effort. It's also useful to have documentation that explains particularly clever or complicated implementations.

Less useful is documentation that can easily be inferred from the code. Writing API documentation by hand is tedious and unproductive. If the API and its parameters use well-chosen names there's not a lot to add--and using something like Doxygen makes it easy to include a few hints where they are needed. But Doxygen isn't the place to explain the architecture or system model--that should be thought out first, not stapled into API comments here and there.

Many times I've found bugs by explaining to someone how a system is supposed to work. Doesn't have to be someone who knows much about it, occasionally it's even been my dog. High-level documentation is just another way to exercise that technique, with the advantage that the explanation itself can be reused.
Load More Comments
Slashdot Login

Need an Account?

Forgot your password?