Taking them down and moving them to memorial sites is fine, but not destroying them. That does not erase history. History is history, like it or not.
History is a thing of books, records and discussion but statues and memorials are there to honor the subject. The men who started and led the confederacy never deserved honor.
You are projecting your (today's) values instead of just appreciating history. I don't want to defend the confederacy, but they were honorable men fighting for their "questionable" cause. Robert E. Lee and Stonewall Jackson and many of the confederate generals were great generals of the day and deserve their statues, albeit in civil war museums and war cemeteries and battle scenes.
Its not like honoring Hitler or Pol Pot or Stalin or Saddam or other mass murderers. They were fighting for their economic well being based on many years and generations of precedence. You should ask, "how could such good men support such a twisted system? " The same way you boil a frog, slowly over time.