More often than not, regulations are a vehicle for government to play favorites. Lobbyists push for regulations that favor companies they're paid by to keep out competition that finds it harder to comply. So yeah, I typically support deregulation. There are instances where it makes sense, but the current regulatory environment for most avenues of life is very much excessive. Regulations are rarely a matter of safety.
Your view of taxation is very off. You actually have it backwards. Any tax you impose on the rich or a company is paid by the consumer, ultimately. If you increase property taxes, renters pay more. If you increase income tax on a business, prices go up.
When it comes to businesses that evade most or all taxation, that's actually evidence for why the tax code should be simplified. It's much harder to create or exploit tax loopholes when taxation is simple and easily collected. It's why the simplest way to tax is consumption. Sales taxes are hard to avoid and loopholes don't typically exist. You only pay in proportion to what you buy.
Property taxes are also hard to evade, but you have to be careful how far you take that. You don't want rents to get too high.