additional counter in reviews

Question

At present, the papers in the review section are listed in a somewhat random order. 

It would be nice (and perhaps necessary when the whole arxiv is imported) to have for each submission a counter where people can vote affirmatively for 

''I'd like to have this paper reviewed''

The reviews could the be sorted (by default or upon request) by the number of votes of this kind received (ties sorted by time of submission). This would give potential reviewers a guide for which papers need reviews most urgently.

Answer 1

@ArnoldNeumair @RonMaimon @Dilaton @dimension10

I have enabled a feature that enables additional sort orders for all categories (including reviews). The start point is the categories page (tab at the same line as our logo). Select there the desired category ('Reviews' for the request related to this answer) and you get as a result page with a sub-navigation bar. There you may select the desired sort order.

Q2A defines a sort order 'Hot'. Hotness is determined by the attributes set in "Relative importance for question hotness", at the bottom of Lists tab in Admin Panel (also discussed here). These attributes are

– Question is new
– Question has a new answer
– Question has many answers
– Question has many up votes
– Question has many views

Maybe your new counter could be constructed from this hotness.

By the way, this feature resolves also the feature request here.

Comments

This is just great! And very unexpected too, as I thought this would be extremely difficult to implement (the Q2A team listed it as a priority for Q2A v1.9, when the latest stable version is just 1.6.3).

---

Is this just me, or does sorting with respect to "most votes" only work for the Reviews category, whereas for the other (sub) categories it does nothing at all? If it is not just me, could it be related to the different typ of question (submissions) we have for the Reviews section? How is "most votes" defined for the conventional questions anyway, is it the net score?

---

@Dilaton As always, a very small bug (in this case a missing comma) corrupted the result. It is corrected now. "Most votes" is defined as most net votes for questions and most score for submissions.
 

---

Thanks Polarkernel :-)

And I second Dimension10, it is a very pleasant surprise that the sorting business already works now !

---

would it be better if such feature is made manifest, i.e. usable without going to "categories"? 

Answer 2

I use the "views" as a guide of this kind, the number of views of the page is a reasonable indicator of interest. I think it works well for this purpose, and it doesn't require intervention by the user.

One possible reason to avoid this is that it can lead to a political popularity contest, of either a good or bad kind, where either very good papers are totally ignored, because very few people are competent to understand the significance, and not so good papers with a big mistake are urgently asking for review, because of the mistake, so look very popular (when they aren't, they are just wrong, and people want to see the review). I am not sure how this will impact the process, it is difficult to judge in advance.

Comments

Then it should at least be ensured that the items can be sorted by several criteria, including the number of views as one of them.

---

Actually, the number of views is highest for those papers having already a review (and a controversial discussion), since that attracts people to the page. So views measure something different.

---

Use the views on papers without an upvoted review, that should work. We can't sort by this criterion yet, but we should have various sorting methods including this for sure, we just haven't gotten around to it yet.

---

The problem with the present submissions is that views are purely by titles (and guessing what is behind them); thus views just indicate curiosity about what the entry might contain, not interest. Then being on the page, there are neither authors nor abstracts (unless someone bothered already to place them there by hand); thus to find out whether a paper is worth something one has to click on the link to actually see the paper. So not even the number of clicks on the link (assuming they could be recorded) really serves to indicate real interest. 

I cannot see what is wrong in having people indicate interest in a third counter. Reviewers can review anyway whatever they like to review, and a paper I don't find it interesting will not be reviewed by me even if it gets 100 upvotes for interest and has 10000 views.

But at least I want to review only papers where I expect people here are really interested in. Reviewing is time intensive; I don't want to waste my time on reviewing papers that I'd find interesting but that meet no real interest.

---

Hm, I always thought that "preexisting" interest in or popularity of a paper with the PO community before it gets a review here is not that important...

I see the reviews section as a place where by reviewing them on PhysicsOverflow, good papers can get highlighted and earn the (missing?) attention they deserve, whereas wrong things and nonsense can be debunked by a negative review.

The interest in the papers will (partly) come with the reviews they obtain. so that it should IMHO be enough to start with, when the reviewer thinks he has something interesting or important (positive or negative) to say about a paper.

Also, reviewing papers about not yet sufficiently loved on-topics, may broaden the range of PhysicsOverflow by draging in new users who are interested in them.

---

But if the main purpose of the review section is good papers can get highlighted and earn the (missing?) attention they deserve, whereas wrong things and nonsense can be debunked by a negative review - what then is the point of pre-importing all arXiv articles rather than only papers that are actually reviewed? 

---

We learned that massively importing papers from the ArXiv could be a bit risky, the ArXiv admins might not approve it ...

So the import of papers should rather be driven by members of PhysicsOverflow than massive and automated ...

Answer 3

I second this. In fact just yesterday I tried to figure out if submission can be ordered in scores. 

Comments

I am really worried about such a counter becoming a political thing. Perhaps it can be made to disappear once a paper gets a significantly upvoted review? Like a rep bounty, which people can contribute by checking the box? I am really worried about non-scientific fluff criteria being added to published papers, this is well intentioned, but this can have the effect of unfairly neglecting some work over other work.

Remember that reviews is in total beta, it's being tested out and kicked around, and it's not at all in its final state. The final state should look a lot like a more streamlined version of arxiv, with reviews attached, with hierarchically organized categories, with automatic mutual links for references, with perhaps even hard-linking of voting so that upvoting originality on one submission automatically upvotes another, and so on, with lots of features that are not sorted out or discussed yet, they depend on what the beta looks like.

The current view system is just to test out the concept, to see if it works. But we do need a way to order the submissions by various criteria, that's for sure, but that's work for polarkernel, and I don't know if he's so gung-ho on it.

---

We have also beta feature requests of making the lists of questions, users, etc sortable concerning different criteria such as votes, activity, age, and so on... With these feature requests realised, the list of submissions will hopefully be sortable (by score for example) too, es they are technically just a special kind of question.

I absolutely dont see at all what should be "political" about the list of submissions being sortable with respect to scores for example. Sortability of questions (including submissions) will greatly improve the user experience of the site, which is very useful.

Improving the user experience is important for attracting and retaining people (and keep them happy), and we should not decline useful feature requests for strange "politica/politicsl" reasons as it is done on other interactive networks of internet sites too often ...

---

This is a subset of the currently most upvoted public beta feature request. @RonMaimon it's just an additonal (more useful) sorting criteria of submissions, by votes.

---

I agree with sorting by votes. The proposal I disagree with is for a new votable counter measuring "interest in paper".

---

@RonMaimon Interest in paper is just calculated from originality and accuracy, not a new counter. By "additonal counter", I think that Arnold meant "additional sort".

---

By a new counter, he meant a votable third criterion called "interest", in addition to originality and accuracy. This is used by journals to determine the importance of papers, what to include in high-impact journals, vs. what to send to second tier journals. It is a total bugaboo of peer review, it is completely political and it leads to politically biased ignoring of important work. It is also totally unnecessary on the internet, as there is no space limit unlike a paper journal. It is the analog of "notability" on Wikipedia. I left it out on purpose.

On the other hand, sorting unreviewed papers by views is useful, we should add that feature.