It is not about refusing business but doing business that benefits the community as a whole.
You can't do business with a business that has been shut down because a thin skinned customer sued them under the law because he or she had his or her feelings hurt by the religious consciences of the proprietor. Tell me, you speak of business and community... yet this kind of behavior, (i.e. suing a religious cake maker for not catering a gay wedding, and subsequently having them eviscerated from the marketplace) only harms the community, does it not? The glue that should bind business and community should be tolerance, on both sides of the spectrum, but neither side is willing to show it.
Nobody wants to meet in the middle, simply because they are too busy prejudging each other.
Yes, actually that's true. You are harming the entire community by eliminate jobs and services, when you drive someone out of business because they have a religious belief you don't like.
But... I for one, will just open another business. You shut that down, and I'll open another. Some you will drive out, and they'll go somewhere else that is more tolerant, and benefit that community.
However, what you will not do, is change our faith. You are not going to change what we believe, or us worshiping our G-d. That's never going away.
Prejudging? We're not prejudging anything. Homosexuality is a sin in our belief. That's what G-d said, and that's what we believe. There is nothing prejudging about it.