Refuse what, and in what manner?
Refuse to embrace Islam. Let's say they don't get violent, just make public proclamations that they do not accept Islam.
That's their prerogative.
"It would be naive to assume that a call is raised to free the whole of humankind throughout the earth, and it is confined to preaching and exposition. Indeed, it strives through preaching and exposition when there is freedom of communication and when people are free from all these influences, as "There is no compulsion in religion"; but when the above- mentioned obstacles and practical difficulties are put in its way, it has no recourse but to remove them by force so that when it is addressed to peoples' hearts and minds they are free to accept or reject it with an open mind." - Sayyid Qutb