There are plenty of things to use against Gingrich, to the point where it's hard to predict which ones will actually hurt him. I was completely taken by surprise when a serial adulterer was lambasted for committing the cardinal sin of keeping a charge account at a jewelry store.
As far as inflammatory rhetoric goes, Gingrich doesn't seem to be backing down from it. At one of the debates, after moderator Charlie Rose gave Gingrich a chance to back down, Gingrich instead insisted that Chris Dodd and Barney Frank belonged in jail, citing no particular crimes (eg, Frank was "close to" lobbyists).