Should votes in the reviews section *always* be accompanied by reviews?

Question

The “open peer review” system (Review Section) makes it possible to give any score without any review. A score without review is a nonsense. It should not be just a "voting system", it should be a review system at least.

Comments

This is designed to allow people who do not have patience to write a review the ability to up/down vote. It is somewhat political, but it is mitigated by the reviews themselves, which can correct misperceptions. Most people who don't understand anything don't vote. Your stuff is an exception, because you are challenging well understood things that many people already know as second nature.

---

Nobody is obliged to give a developed review, but no review at all serves to nothing. The author cannot improve his article if necessary. Also, I remember you saying that my Russian friends might come massively and raise my score to unreasonable values. This did not happen to my papers, but such a situation should also be avoided with imposing an obligation to leave at least a short review. Then the author will share his shame with the reviewers, if they are evil.

---

We discussed it in the abstract, I didn't say that about your papers. The issue is whether you need to coerce a review to have a vote. It is perhaps desirable to lock voting until after the first review is published, and then have every vote (up or down) be linked permanently with a upvote on a review, or with a new review you write yourself. I considered this, as it is a way to make incentive for review voting and for writing reviews, as otherwise you can just politically vote. But there is no protection against your Russian friends, sites like this rely on honest, well-informed voting. The surprise is that it works so well under normal circumstances, not that it can be abused on occasion. I don't think your papers were treated unfairly, just you.

---

@RonMaimon:

I don't think your papers were treated unfairly

Yes, they were downvoted without being reviewed, and it is unfair. Moreover Arnold wrote: No attention is given to the question whether such an alternative would produce results in accordance with experiment." I never gave a pretext for such a conclusion, frankly. I am a physicist, you know.

Nowadays everybody is busy, has no time to read carefully, think over carefully, and write sensible reviews. Most papers in the Review section are without any review. Professionals have their own networks, they do not come on PO. We must do something to attract them here.

---

If no changes are implemented, we may safely call it an "Open no review system".

I insist that each vote must be done after the corresponding review, one vote per review, by the reviewer solely. Votes without reviews must be canceled.

---

OK, Dilaton, they are physicists and I will edit my comment. But I am also a physicist. What do you think of the Arnold's phrase I mentioned?

As well, what do you think of my main proposal?

---

@VladimirKalitvianki nobody said you are not a physicist.

Concerning the proposal I agree with Ron this time. Optimally, each vote in the reviews section would be accompanied by a (partial) review, comment at least, or correspond to a vote on a pre-existing review or comment.

But I think this should not be politically or technically enforced.

---

If so, then it may be an "Open garbage system".

Note, Ron wrote bad reviews and reconsidered them later on. He was wrong scientifically. Those who joined him by downvoting the reviewed papers, reinforced his wrongness. So every vote must be done by the corresponding reviewer solely and after giving his review. Otherwise it is not a review system. You must get it.

---

@VladimirKalitvianski: The review wasn't bad because the paper was ultimately correct, the paper is incorrect on general principle, and the review identified the general principle that allows you to see this (the vacuum in interacting and free theories are unitarily inequivalent).

The review was bad because it pretended to find a worse specific mistake than the one actually made, and then heckled the author for making a terrible elementary mistake that he did not make. Because the general principle of the review was valid, it had propaganda value--- casual readers with some familiarity with quantum fields, but no familiarity with the specific paper, could come to believe that the criticism was valid. This made it more dangerous, because it was partly right and mostly wrong.

The paper introduced free field operators over the classical solutions, and I skimmed the construction, noticed that the bound is proved by having the vacuum fixed when interactions are turned on, and knew that this was impossible in a non-free theory. So I figured that H must be free and wrote as if Dynin multiplied the classical energy by the number operator for the mode to get the quantum energy. This is an extremely stupid thing to do, so the review also went on to heckle Dynin for doing this stupid thing.

But this stupid thing is manifestly not what Dynin did, as his Hamiltonian is quartic in the creation and annihilation operators, it is not free. What really happens here is that the products of fields are not local products of field operators, they are defined to be normal ordered in the creation and annihilation operators, so that the interactions preserve the noninteracting vacuum.  This is still a fatal mistake, just not the idiotic mistake that the review pretended to find.

Further, the paper then had the gall to mock Dynin for making the stupider of the two mistakes, and since the overall theme was cogent, wrong kinematics, this gave propaganda weight to the review for even an expert but casual reader who didn't read Dynin in depth.

Still, the voting mechanism on the site worked, more or less. The paper got a reasonable review from Arnold and the mistake in my review got criticism from Arnold. My review was at net 0 votes, the paper was downvoted. The reason it was horrible was because it ended up being net propaganda instead of mathematics.

---

If so, then it may be an "Open garbage system".

@Vladimir Kalitvianksi until now, we happily could avoid collecting garbarge in the reviews section. Calling the reviews section of PhysicsOverflow an "Open garbage system" is very insulting to Polarkernel, who has put large effort, time, and love into developping that feature. So please dont do that.

All of us (me, Ron, Dimension10, Arnold, etc ) made the experience that feature requests we thought were a good idea, did not find the support of the community and did not get implemented. This might be a bit disappointing at the first moment. But starting to throw around insults or leaving disparaging, sarcastic, and scornful comments out of frustration, does not do anything about it and helps nobody (including yourself). It only makes the atmosphere unfriendly and the poster of such comments and remarks looking bad.

If we finally succeed in achieving that academics and professional physicists accept PhysicsOverflow and start to make use of the site, there will be more activity in the reviews section too.

---

@Dilaton: You avoid the point and pretend that I speak of something else. You are not so stupid to miss the point of my post, so I am figuring out that I will never obtain any response to my request. OK, let it be.

Answer 1

I understand you concerns VK, however a system when in every upvote or downvote NEEDS to be explained is not a viable one. Not every one had patience or often even in depth understanding to write a review.

I tend to up vote Papers if I think the general direction is a good one, for instance I up voted both the Amplituhedron papers on this site.  I am far from understanding it deeply enough to review it.

I think the current system is fine. However, I would like it if the voting was not so easily swung. It is due to the strange multiplicative formula that is being used I think. Though, that is not something I would worry about.

Comments

What is significance of your vote for others, if you may vote without understanding the paper? How can such a system be fine or correspond to "high quality site"?

Voting in the Review section must be different from voting some comments, etc. It is voting with responsibility..

---

@VladimirKalitvianski The general direction proposed in the paper is an interesting one, as simple as that. Of course the point of the review system is not check who understood the paper and who did not.

I don't want to discuss this topic any further.

---

@VladimirKalitvianski I usually vote on originality, and sometimes on accuracy - for voting on originality, you usually just need to skim through the paper and know what's new in it, and for voting on accuracy, you usually need to read it and have a more or less clear understanding of it. On the contrary, you need to have valuable insights about the paper to review it, and this is difficult.

The standards of understanding needed for voting and reviewing are totally different, but this doesn't mean that votes do not require understanding. There might be a few voters here and there who vote without understanding, but the situation of this is mostly symmetrical, and largely overshadowed by meaningful and well-thought-out votes.

Answer 2

I completely agree with @Prathyush. To vote on originality, one just needs some knowledge of existing literature, and as to whether the paper takes it in a good direction. To vote on accuracy, one often needs relatively more knowledge, some intution regarding the derivations and claims made in the paper. On the contrary, a review needs insights on the paper that are significant enough to produce a review. You don't need these insights to vote, for which a general idea of the paper is enough.

Comments

I see and I do not mind if your vote is accompanied with a short comment:

"I skimmed this paper and I like its general idea. +1 for this and +1 for that from me. Bob"

Then the value of your votes is comprehensible.

---

@VladimirKalitvianski: That usually happens automatically without enforcement. In your case, much of the discussion in the comments and chat duplicate what those comments would say, so if there is a vote without comment, it probably overlaps a previously stated answer or comment, either in reviews or in chat. It's good to avoid duplication like the plague, otherwise the same things can be rehashed a thousand times without any significant gain.

---

I give up.