$O_l(x)$ is a renormalized product of field operators at $x$. Note that $\phi(x)^2$ doesn't make sense as $\phi(x)$ is only a distribution, but after appropriate renormalization (by adding ''infinite'' linear and constant counterterms - i.e., adding finite counterterms in the cutoff theory that change with the cutoff in such a way that the limit exists when the cutoff is removed) there is a renormalized operator for every power product (and product of derivatives). These are the $O_l(x)$. On the other hand, the $A_l(x)$ are arbitrary local fields, typically those that enter the Lagrangian.
Saying that $O_l(x)$ transforms like $\psi_l$ just expresses that $O_l(x)$ transforms according to an irreducible representation of the Lorentz group. This is done for simplicity only, and $\psi$ refers to the general irreducible fields discussed in Chapter 6.
The italic $O_l$ in (6.4.4) come from interaction terms in (6.4.1) via transition to the Heisenberg picture, and have nothing per se to do with the curly $O_l$ in Chapter 10, though they may be taken as examples of the latter. But apparently we are using different editions of Weinberg's book as the numbers you give are not consistent with mine.