At the non-rigorous/intuitive level, OP's observations are spot on. To facilitate such thinking, physicists often use DeWitt's condensed notation, where a field ϕα(x)
is written as ϕi, while pretending that i=(α,x) is an index of a local coordinate ϕi for some differential manifold.
The problem is that the space of all field configurations is typically an infinite-dimensional space, while ordinary differential geometry is usually only discussed in the context of finite-dimensional manifolds.
Thus strictly speaking, one would have to master/study/develop an infinite-dimensional mathematical version of differential geometry to make OP's picture precise/rigorous.
This post imported from StackExchange Physics at 2014-03-28 17:12 (UCT), posted by SE-user Qmechanic