Really. Glance at a map once in a while.
I have. In 1942, we had a lot of ocean, and the US only had five aircraft carriers to defend it all. After Coral Sea and the loss of USS Lexington, and the Saratoga was in drydock because of a torpedo hit. That just left Enterprise, Yorktown and Hornet to defend Midway. The Wasp was off doing, something, I guess.
So imagine what would have happened if at Midway, instead of four Japanese Carriers going to the bottom, it was three American carriers.... We'd have been well and screwed. We'd have lost Hawaii and nothing but water between that and California.
Again, today we know the Japanese were already stretched to their limits, but the people at the time didn't know that. Axis victory was a real possibility, and the last thing you wanted was 100,000 potential collaborators in a war zone, because that would have been stupid.