I still think that it's not the right place for this type of questions. Nevertheless, the topic itself is interesting, and I'll also have a shot at it. Since I'm neither a philosopher of science, nor an historian (and there are probably very few such people on this site, one of the reasons this question might not be suitable), I'll focus on my own restricted field, statistical physics.
1) There are many. For example, a satisfactory rigorous derivation of Boltzmann equation, the best result to this day remaining the celebrated theorem of Lanford proved in the late 1970s. In equilibrium statistical mechanics, one of the major open problems is the proof that the two-dimensional $O(N)$ models have exponentially decaying correlations at all temperatures when $N>2$ (there is supposedly a close relationship between such models and four-dimensional gauge models, and this problem might shed light on the issue of asymptotic freedom in QCD, see this paper for a critical discussion of these issues). Of course, there are many others, such as trying to understand why naive real-space renormalization (say, decimation) of lattice spin systems provides reasonably accurate results (even though such transformations are known to be generally ill-defined mathematically); but it seems to me that it's unlikely to happen, which does not mean that the philosophy of the renormalization group cannot find uses in mathematical physics (it already has led to several profound results).
2) Well, one major example was Onsager's rigorous computation of the free energy of the 2d Ising model, which showed that all approximation schemes used by physicists at that time were giving completely wrong predictions. Rigorous results can also lead to (i) new approaches to old problems (this is the case recently with SLE), (ii) new results that were not known to physicists (this is the case with, e.g., the results of Johansson and others on growth models), (iii) a much better understanding of some complicated phenomena (e.g., the equilibrium properties of fixed magnetization Ising models), (iv) settling controversies in the physics literature (a famous example was the problem of determining the lower critical dimension of the random-field Ising model, which was hotly debated in the 1980s, and was rigorously settled by Bricmont and Kupiainen).
3) None that I know of. Although, one might say that the "paradoxes" raised against Boltzmann's theory by Zermelo and Loschmidt were both of mathematical nature (and thus criticized the apparent lack of of rigour of Boltzmann's approach), and did delay the acceptance of his ideas.
4) Not sure about this point. Certainly the numerous conjectures originating from physics, in particular striking predictions, provide both motivation, and sometimes some degree of insight to the mathematicians... But I am not sure that's what you're asking for.
5) There are many papers discussing such issues, e.g.:
and references therein.
This post has been migrated from (A51.SE)
Q&A (4871)
