iabervon's Journal: Peer-to-peer review journals

The current situation in academia is silly. People write papers, which may be of variable quality. In order to weed out bad ones, and, more importantly, fix ones that have problems, they are reviewed by anonymous other people in the same field, who point out problems and suggest changes. Then they are published in journals, which are read by people in the field. After this, the journals end up with the publication rights. This made sense when only journals could easily publish things, so neither the author nor the peers would have much use for the publication rights anyway. But now it is easy to publish things yourself, so the main work the journals do that the authors cannot do easily is arranging the peers.

I would like to set up a P2P system for this task. An author would submit a paper to the system, which would randomly chose reviewers, who would then get the paper and a psuedonym for giving responses. Reviewers, either anonymous or not, could sign statements about the paper which include system-wide reasons the paper is worthwhile (e.g., "This paper is correct", "This paper is inspirational", "This paper should be considered"). Once the anonymous review period had ended (either the deadline passes or all of the reviews have been submitted for a final version), the paper is published if it meets the publication criteria; it has attached the system-defined signed statements which demonstrate it as worthy of publication, and thus appear on people's lists of new papers. People could also add statements saying what they think of the paper, so that other people could essentially search for the papers a given author likes.

It would be easy to have a web-of-trust mechanism for the creation of accounts which can publish papers, can be chosen as reviewers, and can make statements in their own name. The anonymizing protocol doesn't have to be particularly robust, since the authors tend to not want to know who the reviewers are, and could probably figure it out by analyzing the style if they wanted to. The protocol would presumably avoid giving a paper to its own author, or someone from the same organization.

It seems like it would be possible to do all this with a relatively simple web+P2P app. I may sit down and write something of the sort at some point.