There were a whole lot of people that fought for justice to end slavery. Not all were on the front lines. "Owens" and "Brown" are a portion of my family ancestry. These people did not want violence but were raised in the violent atmosphere of the times. Pioneers were tough people. Others that we have researched in my husband's family heritage were working on the political end of the spectrum and one in particular worked with Abraham Lincoln. Their children actually had sleep overs together when they were at their homes in the Midwest. It would be a poor judgment call on your part to be limited in your own mind towards people because of the color of their skin but you are free to do that if you like. Slavery was ended not because of the violence brought forth by John Brown and those who were with him but because it was time for people to become more compassionate towards one another and a majority felt passionate about it. Most of those were people who believed that all men (humans) were created equal. My grandparents attended and were married in a church that was mainly black people. It was a spiritual thing with grandpa not something of the typical judgmental carnal flesh. He loved the spirit of the people he went to church with.
That sounds like a Hallmark card description of how slavery was ended but in reality it nothing to do with compassion. It was a means to an end. You mentioned Lincoln. Did you realize he would have gladly made Black people slaves forever? Of course there were white abolitionists like there are cool white people today. The problem is that their numbers were few and far between and a lot of them still believed whites were superior.
Go back through the notes of history. The banksters wanted to have people enslaved. It is no different today than it was yesterday. They did not care if people were white or black or brown. They wanted control of all of the resources and labor is a consideration in what they consider commodities.
You have ethnic groups and little factions all over the world that believe that they are superior when in fact they are not. You are not going to change some people so there will always be dickheads in some areas.