There is nothing wrong with killing a regulation if the cost to business exceeds the value of the regulation to the rest of society. The sensible way is to assemble as non-partisan a group as possible and revise the regulations. Wholesale eliminations of regulations is like throwing out the baby with bath water.
What if we simply made the members of the boards of directors of every corporation personally and criminally liable for the environmental mistakes their corporation makes?
Do you reckon they might make sure the toxic waste gets handled properly if they faced jail time instead of a "cost of doing business" corporate fine if it doesn't?
All that would leave to regulate is the amounts of pollution that are acceptable - as long as those laws are applied evenly and fairly in the marketplace, no business would be placed at an advantage or disadvantage compared to their competition.
The concept of fairness through simplicity does not just apply to the tax code. Regulations could be simple too, as long as fucking something up carried the possibility of jail time for those at the helm of the organization that fucked up.