Stolen opportunities and services from the natives.
We've viewed it differently up here....well....not initially...but we had a town and area that was dying...the town had lost more than 50% of its population due to their paper mill shutting down, then one small mom and pop closed after another, and a school closed, kids left were bussed to another town miles away, and the people abandoned their town and homes and land. They tried to get another business to buy the mill and retrofit it in to another manufacturing business but that failed too.
I moved to Maine a few years after this happened and live maybe 100 miles North of there....(our sister town lost their paper mill a few years ago too)
To make a long story short, a group of black Muslim refugees started moving in to the dead town and region and rented and bought the empty homes for dirt cheap in this distressed community, and they bought the surrounding farm land and began to farm...I think initially to feed themselves but later developed in to the farming business and sold the food to local grocery stores... Then they bought up some of the empty mom and pop stores that left and opened businesses and shops.... Schools in the town that schooled kids from that town, got overwhelmed with costs of schooling and other problems with the huge influx of these foreigners....and the State had to come in and help initially with money....
Now Maine was 98% white and Judeo Christians....so this situation got heated for just a handful of Mainers making a lot of noise...but....that settled down and other Mainers set those loud Mainers in line and the community in the whole region, began to thrive again...the 50% left got their town services and health clinic back, their schools back, etc...
The population increase of these refugees rescued and saved the town and community.
I can understand how this may not work as well in a community that is already crowded and tons of services are used already on citizens etc...
but, imo, if they are allowed to stay, eventually after the short struggle with the influx, prosperity will come with added sales in the community, and workers hands available for businesses to expand.