No one does. That's because he circumvented judicial oversight. That's the whole point of checks and balances - so at least one branch knows what the other is doing, even if the people themsevles don't know, it makes it far less likely for a government to have too much power if one branch is checked by the other.
I would think you would be happy to limit possible prosecutions to terrorist activities.
Wrong.
Its called a slippery slope. And you have yet to say why we shouldn't do the same thing for regular criminal suspects, as according to your line of reasoning, only those actually committing crimes would have anything to worry about.