Of course it is fair
The wealthy benefit so much more from our society
If you have to differentiate between two (or more) groups and set individual standards for each group ... That is not fair.
You can attempt to justify it ... Your justifications may be well-intended ... But, it will never amount to fairness ...
.