I would like to ask folks to really take into consideration what it takes to get the physics community over here. In order to do this, you want to unobtrusively add value to something that they already do every day.
Nearly every professional physicist will, at some point in the day, browse through the new articles on arxiv, either that day, or the previous days, to find the new articles which are relevant and interesting. One easy way to add value for them is to provide them with an arxiv feed in the form of a list of articles with other people's comments already attached to these articles.
To do this, you simply need to link each arxiv paper to a question, and have a page that simply displays arxiv through a simple html filter, the filter simply adds a fourth option to the bottom (where you click for a pdf), it will open the commentary page here. This filter is dead easy to write, so long as there is a systematic method of setting up a page for every paper quickly, the moment it is put up on arxiv.
Then you can suggest to folks to brows arxiv through your filter, and then they can easily get to your discussion, with one click. This will add value for academics, as they will be able to easily comment on a paper they just read, to warn others, or to upvote it if it looks interesting and correct. This will provide instant feedback.
In order to do this, it will help to do the following: change "question" to "submission" everywhere, and change "answer" to "response". This is simply semantics, but it changes the model from "question/answer" to "submission/response", and this psychologically makes people think more about their "submission" before "submitting" it.
I really think this is the only way to produce a successful site, so I hope that the big cheeses here will take this into consideration. A stackexchange clone will not have any more success than stackexchange, while a method for refereeing arxiv, that is relatively transparent and easy to use (just as a front-end for browsing arxiv) will take over the world.
To make an arxiv front-end useful, it is also nice to add features, like displaying the reputation of authors on each paper (authors will gain reputation automatically through votes on the site, even if they are not members of the site, and with enough time, these rep-numbers will become useful), displaying the total votes the paper has gotten so far, and so on, and the number of "responses" to the paper.
Then the responses can contain links to other arxiv papers which rebut the paper, or simply to original text on the site itself. The reputation on the site will then become a very close proxy for the actual reputation that scientists use when evaluating work.
Under these conditions, I think the site will be used by every working physicist (and mathematician), and it will blow stackexchange, mathoverflow, and all the rest out of the water. It is also under these circumstances that one can make a revenue stream from academic advertising, because you can suggest to folks to look at a paper in their feed, and you can target experts in the field, so long as they are willing to take a look.
I believe this model can be a great success, but it requires looking beyond the stackexchange model. Who's interested in putting all the journals permanently out of business?
Q&A (4871)
