1. To begin with, let me just mention a matter of notation and definitions. You said that gauge theories can be described by \(\mathcal G\) bundles, said that \(\mathcal G\) represents the base manifold (identified with the spacetime) but then said that \(\mathcal G\) is the symmetry group that "describes the gauge degrees of freedom." This notation that you introduced in the first paragraph introduces a confusion between the base manifold (physically: spacetime) and the Lie group of the p-bundle where the gauge theory lives (physically: the gauge symmetry group). So, lets just refresh our memory a little bit: A gauge theory is defined on a p-bundle \((\mathcal B,\mathcal M, \Pi, G)\), where \(\mathcal B\) is the total space (mathematicians denotes by \(E\)), \(\mathcal M\) is the base space or spacetime, \(\Pi\) the bundle projection and \(G\) the Lie group. Recall that the fibers \(\Pi^{-1}(p) \forall p\in \mathcal M\) are acted (on the right usually) upon the Lie group \(G\) and are moreover identified with that Lie group itself. This is a p-bundle. (Observe that associated to the Lie group \(G\) we have the Lie algebra \(\mathcal G\), which is finally the notation that you introduced above.)
2. The potential of our gauge theory is just a 1-form on the p-bundle \(\mathcal B\) taking values in the Lie algebra \(\mathcal G\), namely, \(\omega \in \mathcal G \otimes \bigwedge^1\mathcal B\). Math-people call this a connection on the p-bundle. In physics we actually want to push the potential from the p-bundle to where physics is really going on, namely, the spacetime \(\mathcal M\). It is in doing so that we need to introduce a local gauge, mathematically, a local cross-section \(\sigma: U \subset \mathcal M \longrightarrow \mathcal B\). Then we push the potential as \((\omega)_{\sigma,U}=\sigma^*(\omega) \in \mathcal G \otimes \bigwedge^1U\). Observe that it still takes values on the Lie algebra \(\mathcal G\), but it now lives on a neighborhood of the spacetime, \(U \subset \mathcal M\). So \((\omega)_{\sigma,U}\) is a local potential in spacetime.
3. To understand the importance of the twisting of the fibers (by the action of the Lie group), we consider what happens when we choose different gauges. Above I have introduced a gauge \(\sigma, U\). Now lets introduce another one, \(\mu:V\subset \mathcal M\longrightarrow \mathcal B\) and push the potential again to the spacetime as \((\omega)_{\mu,V}=\mu^* \omega \). If the Lie group is a matrix group, as is usually the case in physics, the transition between these gauges is effected by the transition function \(g_{UV}:U\cap V \subset \mathcal M\longrightarrow G\), and the potential transformation between these local trivializations is given by \((\omega)_{\mu,V}=g_{UV}^{-1}dg_{UV}+g_{UV}^{-1}(\omega)_{\sigma, U}g_{UV}\). [Observe that: (i) the transition looks like the transformation of the Christoffel symbols of GR, (ii) it is very non-linear when the matrix group is non-commutative.]
Consider now the case of electromagnetism. The Lie group is just \(G=U(1)\), it is commutative, so \(g_{UV}^{-1}(\omega)_{\sigma, U}g_{UV} = (\omega)_{\sigma, U}\) and we easily prove from \(d^2=0\) and Leibniz rule that \(g_{UV}^{-1}dg_{UV} = 0\). Thus \((\omega)_{\mu,V}=(\omega)_{\sigma, U}\). So the family local potentials \((\omega)_{\sigma,U}\) generated by each local section \(\{\sigma, U\}\) of our p-bundle pieces together to define a unique potential in the whole spacetime. It is nothing but the electromagnetic potential 1-form: \(A=i\omega \). (Moreover, observe that the covariant derivative associated to our potetial reduces to just the exterior differential, so the curvature 2-form is \(F=dA=iD\omega\), that is, the EM field strength!) This fact depends heavily on the commutativity of the gauge group of EM. For nucleon gauge theories, like Y-M \(SU(N)(N\geq 2)\), you are only going to be able to piece the potential together on the whole spacetime manifold and define a simple gauge potential like \(A\in \bigwedge^1M\) if you have a global cross-section \(\mathcal M \longrightarrow \mathcal B\). The transition function of these potential derived from non-Abelian gauge groups is very non-linear, and this is one of the advantages of using the geometric language of p-bundles, because it is explicitly gauge invariant. There is no need to push the potential to spacetime and the field strength we define by using the covariant derivative.
4. Therefore, I think that the lesson from the above comparison of EM and Y-M is that the physical aspect of the gauge group is contained not only in the fact that it describes the twisting of the fibers, but moreover, is the lack of the group that describe the twisting to be commutative.
5. Above you have used the same notation \(\mathcal G\) for the base manifold (spacetime) and the gauge group. As I have explained, this is not the case for gauge theories, these (spacetime and Lie group) being different manifolds of the same object, namely, the p-bundle. But this raises a good opportunity to discuss the differences between the Kaluza-Klein and gauge theoretical approaches. In KK, one or more of the spacetime dimensions is really identified with one of the gauge degrees of freedom. In the original KK unification of G+EM, the 5th dimension was really identified with \(S^1 \equiv U(1)\), and the components of the metric tensor of the spacetime along this 5th dimension (for example, \(G_{5\mu}\forall \mu=1,...,5\)) entered in the higher-dimensional Einstein equation \(\mathbb G_{ab} = T_{ab}\) as a dynamical variable (here \(G_{ab}\) is the metric tensor for KK spacetime and \(\mathbb G_{ab}\) its Einstein tensor). In the gauge theory approach, this do not happen since the gauge group \(U(1)\) is separated from the spacetime manifold, and the metric tensor do not have components along it. So the gauge theory approach avoids the introduction of extra degrees of freedom of the metric.
6. Just a final issue, a diversion lets say. You mentioned above the possibility of introducing a gauge theory on a space like Mobius band. Recall that the Mobius is non-orientable. How would you introduce a spin structure on it in order to describe fermionic fields?
To be precise, if \(\mathcal M\) is a spacetime, let \(F(\mathcal M) \) be the frame bundle whose gauge group is the Lorentz group \(L=O(1,3)\). The Lorentz group have four components, but depending on the topology of \(\mathcal M\), the frame bundle may have less than four; it would be a non-orientable spacetime. A physical spacetime being orientable, we choose one component of the frame-bundle, say \(F_0(\mathcal M)\), which fixes both the future-direction and a screw sense of spatial rotation. A spin structure on this frame bundle is the p-bundle \(S(\mathcal M)\longrightarrow \mathcal M\) with \(SL(2,\mathbb C)\) as gauge group, and a bundle morphism \(f:F_0(\mathcal M) \longrightarrow S(\mathcal M)\). There is then a theorem by Geroch which says that if the frame bundle is trivial then a spin structure exists. The converse is true when \(\mathcal M\) is non-compact, namely, a spin structure exists if the frame bundle is trivial. But the frame bundle of the Mobius is non-trivial, but it still have interesting gauge theories defined on it because Mobius being non-compact avoids Geroch theorem!
Applications: https://arxiv.org/pdf/hep-th/9710057.pdf