Will all users (above a minimum critical reputation) be allowed to vote in review section ?

Question

I am not sure if this issue has been raised before so I am asking this question.  Suppose I've earned my reputation mainly by answering/asking questions related to statistical mechanics. Then should I be allowed to take part in reviewing of a quantum gravity paper ? 

Comments

I think Q2A allows for category-specific priviledges. Anyway, this should not be a problem, if we have originality and accuracy scores.  

---

Yes, you should. This is exactly what open internet review is about. ANYONE can comment on a paper, if the comment is ridiculous, you need to trust the community to downvote it.

Restricting comments to topic-established experts is what everyone else is doing, and why they all fail. In my opinion, if this site doesn't do this, will succeed tremendously. There is a huge need for a person who has no credentials (but has knowledge) to be able to comment on any paper on any topic. If the comment is stupid, don't worry, someone will soon let them know.

---

This is among the things why I enjoy reading TRF so much: Lumo is not afraid of blatantly pointing out errors and misunderstanding in everybody's work. He never gets overimpressed honors like Nobel prizes etc but knows that even highly estimated people can and do make mistakes, get things wrong, etc from time to time :-)

Answer 1

Absolutely! Anyone can comment on anything, as long as they exceed some small nominal reputation.

Restricting comments to experts, that's the huge mistake all the other sites are making. This is the opportunity for an open-review site to succeed big. This is why I am optimistic about the future of this site, because it is dedicated to high quality physics, and to openness and no restrictions using authority or power.

Just because a person studies quantum mechanics does not mean that they can't see the mistake in a fluid dynamic paper immediately. It does not mean that they aren't capable of seeing that another author's attempt to claim that a superconductivity paper is derivative is simply an attempt to pad the resume. The explanations are allowed to be lengthy, and if you have someone who makes a mistake, you can explain the mistake, and there is no need for an authority mechanism at all.

In the refereeing section, you informally encourage people to vote sincerely. That means, vote only when they think they understand what's right and what's wrong pretty well. Even still, they will make mistakes, but sincere mistakes are easy to fix, you simply point out the errors, and the community self-adjusts.

It is important to TRUST online communities, they have a good idea of what's going on, much more so than any group of experts. Frankly, the main reason is that people who don't know anything just sit on the sidelines and don't vote. But the other reason is that there are people who are capable of commenting from outside the power structure of a given field, who can demolish low quality work, but inside the field, the experts don't dare, simply because the low quality work comes from a person with power.

The way to deal with incompetent voting (only if it becomes a problem) is simply to request that the donwvotes/upvotes be specific in their claims, so that a downvote on accuracy comes with a specific mistake pointed out (usually by upvoting someone else's review which points out the mistake), and that an upvote with a sincere appreciation of the utility and scope of the paper, usually by upvoting some extension, or by pointing out some important derivative work, or simply by aesthetic appreciation of a masterful physical insight. The people who vote by definition are always the people who care, and people who care are people who have an idea what the paper is about.

It is completely wrong, and contrary to the principles of openness which make websites like this valuable to require any sort of expertise barrier, other than a moderately low rep threshhold just for getting rid of spammers and charlatans.

Comments

-1 I disagree for the same reasons as I mentioned in my answer. 

---

I think that we dont have to be afraid of agressive pompous overreaching dilletants becoming too powerful or even dominant on PhysicsOverflow, even if everybody is allowed to vote everywhere ... ;-)

1. As Ron mentions too, the reasonable and knowledgeable people that should be naturaly attracted to the site, in the majority of cases are simply wise enough and know better than to negatively interfer with questions about topics that are outside their domain of knowledge and interest. Already on MathOverflow I got the impression that people voluntarily refrain from voting on questions in tags they are not knowledgeable about and have now interest in (not even speaking about vorint to close/reopen), simply because they themself would feel that such things would be inappropriate (or even dishonest) to do, and the serious character of the community forbids such a behavior too. So it should not be needed to enforce the honest and reasonable behavior by the system.
2. If a dimwit comes to the site to wrongly attack papers/authors/topics for invalid reason and explicitely points them out for example in comments and answers to the review questions, I suspect the community would be quick enough to give him the appropriate feedback by comments and votes, such that he will not be able to do much harm.
3. A slight problem could be coward rascals who come here to troll-vote without leaving corresponding comments and answers. But in this case I think too that the community would be able to note it of blatantly fishy and suspicious things are going on at some point, such that a moderator/administrator could look into it and deal appropriately with the troll(s) if needed.

Answer 2

I think that we dont have to be afraid of agressive pompous overreaching dilletants becoming too powerful or even dominant on PhysicsOverflow, even if everybody is allowed to vote everywhere.

1. As Ron mentions too, the reasonable and knowledgeable people that should be naturaly attracted to the site, in the majority of cases are simply wise enough and know better than to negatively interfer with questions about topics that are outside their domain of knowledge and interest. Already on MathOverflow I got the impression that people voluntarily refrain from voting on questions in tags they are not knowledgeable about and have now interest in (not even speaking about vorint to close/reopen), simply because they themself would feel that such things would be inappropriate (or even dishonest) to do, and the serious character of the community forbids such a behavior too. So it should not be needed to enforce the honest and reasonable behavior by the system.
2. If a dimwit comes to the site to wrongly attack papers/authors/topics for invalid reason and explicitely points them out for example in comments and answers to the review questions, I suspect the community would be quick enough to give him the appropriate feedback by comments and votes, such that he will not be able to do much harm.
3. A slight problem could be coward rascals who come here to troll-vote without leaving corresponding comments and answers. But in this case I think too that the community would be able to note it of blatantly fishy and suspicious things are going on at some point, such that a moderator/administrator could look into it and deal appropriately with the troll(s) if needed.

So, if everything goes well and as expected, there is in my opinion no need to restrict voting in the Reviews section (or elsewhere) apart from the generall preconditions we agreed on for the voting privilege.

Comments

I think that open review system is prone to serious mistakes and even if its "overall successful" it may do harm to some good research work. Suppose there are 1000 people on the site from 10 major research fields, with 100 researchers from each field. Now consider a paper to be reviewed from any given field X. We can't expect all the other 900 people to not to wrongly interfere in the review process. Even if 50 people vote without leaving comment it may lead to a  big review errors.

---

Hm, I seriously hope that we will be able to create on PhysicsOverflow an atmosphere that discourages such things to happen, and people who are neither interested nor knowledgeable about a field, will leave papers (and questions) about these topics alone (conversely to what happens in other places in the interened I do not need to name explicitely ...). 

It works on MathOverflow (as it did and Theoretical Physics SE too).

Of course, this needs the community to be wary and suspicious to suppress any upcoming nonsense right at the beginning. I always highly enjoy it when observing the MathOverflowers clearly telling pompous "101 rep users" who are trying to patronize that community, to kindly mind their own business :-).
 

---

BTW I have just recently heared again about now accepted research (sometimes of famous people) having been rejected first, when the author tried to publish it in a journal. Maybe opening the reviewing process to more than the 2-3 by the editor "invited" reviewers can help avoiding such errors ...?

Answer 3

i) It is very necessary to make sure that in the review process of a paper from a given field, only the experts of that field are allowed to take part; For otherwise the review scores will have no meaning; I mean, think about (say) a string theorist's paper being reviewed by a group of people most of whom work in fluid dynamics, condensed matter or some very different field ? If they wrognly give a accuracy/originality score of say -20/ -40 then it would only discourage the author. 

ii) If its not possible to allow category specific priviledges, and all users can take part in review process then (in my opinion) it would be better to simply give up the idea of a review section. Or may be in this case we can replace it with an "improvement" section in which researchers and PhD scholars can post their work and can request for any suggestions for improving the paper (by pointing out any mistakes in calculations, or suggesting some more calculations etc) before being sent to a peer reviewed journal. 

Comments

Downvoted, because this is absolutely 100% wrong. It is important to allow the review to extend to ANYONE with a point that is legitimate, no matter where it comes from. The "experts" in a field have a political order, and this political order is used to squelch contributions which are useful and important. It is also used to prevent criticism of papers from big shots, which sometimes are nonsense.

The review section is, for me, the most interesting thing, because it can succeed big, and it can take over the world. Open review is not a failure, simply it has never existed before. Just make it exist, and it should work. It worked well enough for a while even on Wikipedia, which lacked a voting mechanism.

The scores will not be meaningless, we already know this from stackexchange experience. But the method of commenting and voting needs to be carefully adjusted so that poltical processes are excluded. The easiest way to introduce politics is to require some sort of expertise for contributions. It is a HUGE mistake, and this is the mistake all the other science 2.0 sites are making. This is what allows this site an advantage. It is our only advantage so far, we don't have numbers, or visibility, or developer strength. We only have committment to openness.