What does it having to hold up in court have to do with anything if I have to quarter these people in my home without them having a warrant? You do realize the court has upheld that the government can literally arrest you indefinitely, throw you in a cage, indefinitely, without the slightest hint of due process?
It's the very essence of a protected right. Bear in mind, rights will be violated, by police and others. Our laws cannot alter behavior of those willing to violate them.
However, whether or not you have the right (it being protected in a court of law) is whether it was violated is determined in court.
For example, your spouse may consent to a search. But you do not have to, and the police have no right to enter, unless they see something illegal in plain sight, have cause to believe evidence might be destroyed or lives put at risk, or they arrest you and do a search for safety / evidence preservation reasons ... or get a warrant, describing the places to be searched and items to be found (no fishing trip, without cause)
A roommate is a bit different. They can consent to a search of your home, but only common areas and not your private space, such as your bedroom. For that, police need your consent, which you have a right to not give them, sans a warrant.
Does that help?