I think the proper model is to automatically create a link to every paper on arxiv, either at the beginning when the site goes live, updated daily with new submissions, or on a case by case basis, as they are referenced by others, whose "question" is simply the arxiv paper title, the question body is simply a link to the arxiv page or to the pdf directly, and whose "answers" are simply refereeing comments regarding the paper's accuracy. This is extremely useful for two reasons:
1. It is value added--- it immediately distinguishes this site from stackexchange, in that refereeing is not Q/A. This is important, as a clone of existing sites is not original enough to get widespread attention and acceptance, you need something which adds new value.
2. It explains the "free speech" aspects of political moderation--- you need to be free to criticize when you are refereeing, and to retract wrong statements when you make a mistake. Scientific refereeing is extremely hostile (when work is wrong, or duplicative of earlier literature).
3. It will draw in EVERYBODY. I mean EVERYBODY. The moment you referee a Maldacena paper, Maldacena will be here within a matter of days. If you make a comment on an 't Hooft paper, 't Hooft will be here in days. You saw this on stackexchange. It is coercive, it forces the whole physics community to pay attention immediately.
Because of this coercive aspect, there is zero chance that this site will be ignored, or wither or die. It will be important, simply because if too few physicists are contributing, all you have to do is go to the refereeing section and criticize some of their mistakes in the published literature, or point out insufficient attribution to earlier work, and boy, will then come. If they have referee reports lying around, or if they have some things that they remember wanting to say about this or that paper, then they will do so at the appropriate location.
The paper-questions themselves can spin off questions related to the paper, by adding related questions, so as to clarify the content. When the paper is old, not freely available, then the question body can and should include a complete summary of the contents of the paper, edited wiki-style, by various people, but mostly belonging to one person. So, for example, if you want to cite a result of Newton in the "Principia", or talk about this work, you can add it to the refereeing section, with a summary of the relevant result, working out all the calculations. The same for an old Polyakov paper, or something from the 1970s by Wilson, or Jackiw or someone like this.
In this case, I think it is important to be able to criticize summaries for their accuracy somewhere, so that a good summary emerges. If the questions are edited community style, with honest comments, this shouldn't be hard, as the work is not original, and all you need to do is stay faithful to the original. But there are always simplifications possible, it is usually possible to summarize old literature very well with newer notations and methods that were developed later. This adds important forward-references to future work to old literature, something which you don't get from following the references in the literature. This is also value added, and it is better than a reverse-citation index, because it is curated by humans, and points to the relevant clarifications immediately.
In doing this, it is perhaps useful to add a bot to the site which makes a question automatically for every arxiv paper every day, and adds them to the "referee" section.
For the referee section, I think a good guideline is that the upvoting/downvoting should be in two parts:
1. Originality: this is the originality of the paper, how original is it?
2. Significance: this is the importance of the paper--- how important is it?
Then the answers can point out various mistakes in the paper, and these can be upvoted/downvoted according to accuracy, as usual. This allows the site to substitute for a journal with very little modification to the software--- simply automatic addition of papers.
But I am going to say up front that the site will get VERY heavy traffic the moment you do this, because every single academic will worry about comments about their paper. You should try to avoid people writing nice things on these pages, as these will be the authors, or their colleagues patting each other on the back. The comments should be for explanations, reworking of results, pointing out errors, pointing out missing citations, the usual negative refereeing.
This is an atomic bomb onthe academic community, because they can't stop you from doing it, and they can't help participating when it happens.
Q&A (4871)
