This is a very general argument: On a finite lattice with finitely many degrees of freedom, the Hamiltonian becomes a finite-dimensional matrix. The spectrum of a finite matrix is always a set of isolated points, and we can apply Cauchy's formula to the resolvent to obtain the projector onto the lowest eigenspace. If the Hamiltonian $H(g)$ is analytic in $g$, then the resolvent and hence the projector are also analytic in $g$.

So, strictly speaking, non-analyticity of the ground state can only be seen in the limit of infinitely many degrees of freedom. But it is certainly possible for a sequence of analytic functions (one for each system size) to converge pointwise to a function that is no longer analytic (the infinite system limit). (Example: $\lim_{n\to\infty} x^n$ on the interval $[0,1]$).