In that initial statement, he singled out neonazis and white nationalists. He didn't exclude KKK or white supremacists until later, after his absurd comment about there being "fine people" among them blew up in his face. But his comment was:
Excuse me. You had some very bad people in that group. But you also had people that were very fine people on both sides. You had people in that group, excuse me, excuse me, I saw the same pictures as you did. You had people in that group that were there to protest the taking down of, to them, a very, very important statue and the renaming of a park, from Robert E. Lee to another name. George Washington was a slave-owner. Was George Washington a slave-owner? So will George Washington now lose his status — are we going to take down — excuse me. Are we going to take down statues of George Washington? How 'bout Thomas Jefferson? What do you think of Thomas Jefferson? You like him? Ok, good. Are we going to take down the statue because he was a major slave-owner? Now we’re going to take down his statue. So you know what, it’s fine. You’re changing history, you’re changing culture. And you had people, and I’m not talking about the neo-Nazis or the white nationalists because they should be condemned totally. But you had many people in that group other than neo-Nazis and white nationalists, ok? And the press has treated them absolutely unfairly.
The only other people from what I saw were KKK and White Supremacists. So who else was there that he was talking about? You can't say either.