Now that we are in the public beta, we should re-look at this idea as promised.
Having 4 pieces of statistics (votes, answers, views, answer quality), takes up too much space.
I think that instead of completely getting rid of the number of answers, we could display the answer quality in place of the number of answers, but still display the number of answers as the title text, which appears when one hovers upon the answer quality.
Is everyone (who supports the idea proposed in the question) fine with this? If so, we can then proceed to take a proper vote of whether we should "leave it as it is" or "make the change", and then if the result is "make the change", we should list it under the public beta feature requests.