He gets the basic rights of anyone accused of a crime -- a speedy public trial before a jury of his peers, presumption of innocence, the right to appeal -- because that is what the Constitution demands.
Deporting him to a foreign country where he is not guilty of a crime would hardly serve justice. If he's to be tried, convicted, and punished, that has to happen in the place where he is accused of committing the crime, which in this case is the U.S. And there are rules that apply to that, because we believe in due process of law, and are not (yet) a fascist dictatorship.