Let's first focus on the spin-1 case. In the context of QFT, these tensors should be understood as the coefficient functions in front of the creation and annihilation operators, that is
\[v^\mu(x)=\sum\limits_\sigma\int\frac{d^3p}{\sqrt{2p^0}}[\varepsilon^\mu(\mathbf{p},\sigma)a(\mathbf{p},\sigma)e^{ip\cdot x}+\varepsilon^{\mu*}(\mathbf{p},\sigma)a^\dagger(\mathbf{p},\sigma)e^{-ip\cdot x}]\cdots(1), \]
where I've written a charge-neutral field for simplicity(and without loss of generality when we are only concerned with transformation properties). The question is what the form of $\varepsilon^\mu(\mathbf{p},\sigma)$ is, so that $v^\mu (x)$ really transforms like a vector field. Let me explain what it means.
Following Wigner's classification scheme, $a^\dagger(\mathbf{p},\sigma)$'s create one-particle states that transform unitarily and irreducibly under Poincare transformations, and in particular the subgroup $SO(3)$. Our $\sigma$ labels the degrees of freedoms of the representation restricted to the subgroup $SO(3)$, and we can choose it to label the angular momenta in the $z$ direction, that is, the values of $J_z$. Let's denote $U(\Lambda, a)$ a representation of the Poincare group , where $(\Lambda, a)$ is any element of the Poincare group. Once we fix a specific representation, the transformation rules on the creation(annihilation) operators are fixed, i.e., the exact form of $U(\Lambda, a)a^\dagger (\mathbf{p},\sigma)U^{-1}(\Lambda, a)$ is fixed, but I won't write it down explicitly since it'll only distract us from the logical development of this answer.
Now, $v^\mu(x)$ must transform like a vector field, that is,
\[U(\Lambda, a)v^\mu(x)U(\Lambda, a)=\Lambda^\mu_{\ \ \nu} v^\nu(\Lambda x+a).\]
The above equation, together with the transformation rules of creation/annihilation operators, put a severe constraint on the form of $\varepsilon^\mu$. In fact, $\varepsilon^\mu$ is uniquely determined up to an equivalence. I'll refer you to Weinberg Chap 5 for mathematical details, but here is the result:
\[ \varepsilon^\mu(\mathbf{p},\sigma)=L^\mu_{\ \ \nu}(\mathbf{p})\varepsilon^\nu(0,\sigma)\cdots(2),\]
\[\sum\limits_\bar{\sigma}\varepsilon^\mu(0,\bar{\sigma})\mathbf{J}_{\bar{\sigma}\sigma}=\mathbf{\mathcal{J}}^\mu_{\ \ \ \nu}\varepsilon^\nu(0,\sigma)\cdots(3)\]
where $L^\mu_{\ \ \nu}(\mathbf{p})$ is the boost that boosts a particle at rest to the same particle with momentum $\mathbf{p}$; $\mathbf{J}$'s are the three rotational generators(angular momenta operators) of the representation $U(\Lambda, a)$(technically it also has to be restricted to the space that carries only the discrete index $\sigma$), which are the irreducible spin-1 matrices by assumption(hence $J$'s are $3\times 3$ matrices); $\mathbf{\mathcal{J}}$'s are the three rotational generators of $(\Lambda, a)$(hence $\mathcal{J}$'s are $4 \times 4$). It's clear from equation (2) that if we know the form of $\varepsilon^\nu(0,\sigma)$ we will know all of $\varepsilon^\nu(\mathbf{p},\sigma)$. Now I claim (3) uniquely determines the form of $\varepsilon^\mu(0,\sigma)$(up to equivalence). You can solve it by brute force, but here's a perhaps smarter argument:
The form of equation (3) makes Schur's lemma for Lie algebra readily applicable(think of $\varepsilon^\mu(0,\sigma)$ as a $4 \times 3$ matrix carrying indices $\mu$ and $\sigma$), which claims $\mathbf{\mathcal{J}}$ must have an irreducible block that is equivalent to $\mathbf{J}$, and the blocks of $\mathbf{\mathcal{J}}$ that are not equivalent to $\mathbf{J}$ can only be connected to $\mathbf{J}$ by a zero matrix; in other words, $\varepsilon^\mu(0,\sigma)$ must be zero at the entries that correspond to the blocks that are not equivalent to $\mathbf{J}$, and can only have non-zero entries otherwise.
We know very well that $\mathbf{\mathcal{J}}$ has two irreducible blocks: the 1-dimensional one that acts on the 0th component(rotations leave energy invariant), and the 3-dimensional one that acts on the 3-vector components(rotations transform 3-vectors to 3-vectors). Only the latter is equivalent to the spin-1 representation of $\mathbf{J}$, hence by the previous reasoning, $\varepsilon^\mu(0,\sigma)$ can only have nonzero entries in the 3-vector part. (In fact, the arguments we have developed by now can be used to show that a vector field can only describe spin-1 or spin-0 particles.)
We adopt the most commonly used matrix representation of $\mathbf{J}$ and $\mathbf{\mathcal{J}}$:
\[J_z=\begin{bmatrix} 1&0&0\\0&0&0\\0&0&-1\end{bmatrix}, \]
\[J_+=J_x+iJ_y=\begin{bmatrix} 0&\sqrt{2}&0\\0&0&\sqrt{2}\\0&0&0\end{bmatrix}, \]
\[J_-=J_x-iJ_y=\begin{bmatrix} 0&0&0\\ \sqrt{2}&0&0\\0&\sqrt{2}&0\end{bmatrix}, \]
and $(\mathcal{J}_k)^i_j=-i\epsilon_{ijk}$, and we only write the 3-vector indices for $\mathcal{J}$ since we just argued 3-vector indices are the only relevant part. These are all we need: to get $\varepsilon^\mu(0,0)=(0, 0, 0, 1)$ just specialize equation (3) to the case
\[\sum\limits_\bar{\sigma}\varepsilon^\mu(0,\bar{\sigma})(J_z)_{\bar{\sigma}0}=(\mathcal{J}_z)^\mu_{\ \ \ \nu}\varepsilon^\nu(0,0),\]
and to get $\varepsilon^\mu(0,\pm 1)=\frac{1}{\sqrt{2}}(0, \mp 1, -i, 0)$, just specialize to the cases
\[\sum\limits_\bar{\sigma}\varepsilon^\mu(0,\bar{\sigma})(J_\pm)_{\bar{\sigma}\ \pm1}=(\mathcal{J}_\pm)^\mu_{\ \ \ \nu}\varepsilon^\nu(0,\pm 1).\]
You said you wanted the form of the polarization tensors for a particle traveling along the z-axis with a momentum $p$, and now it's clear you just need to apply our equation (2), with a boost matrix $L^{\mu}_{\ \ \nu}$ boosting in the z direction, and you easily get
\[\varepsilon^\mu(p,0)=(\frac{p}{m}, 0, 0, -\frac{E}{m}),\]
\[\varepsilon^\mu(p,\pm 1)=\frac{1}{\sqrt{2}}(0, \mp 1, -i, 0),\]
which is exactly the result you wanted with indices raised.
Spin-2 story is the same, but now that we have the spin-1 results at hand, we don't have to start from scratch but just need to construct the spin-2 irreducible part of the tensor product of spin-1 tensors, and the math is covered in any QM textbook under the title of "Addition of angular momenta", and this is where Clebsch-Gordan coefficients come in. Fermion story is again more or less the same, but you need to start the procedure all over again since it's a spinor not a tensor.
1. For a comprehensive explanation see Weinberg Chap 5, but note Weinberg used 1,2,3,0 ordering of the 4-vector indices, where here OP and I used 0,1,2,3.
2. The first half of this answer is really showing how the polarization tensors, which are finite-dimensional, are related to the spin of particles that live in an infinite-dimensional Hilbert space.