Again, during the fog of war, particularly with Muslims who do not fight conventionally, it is entirely reasonable for Israel to believe that it might be a literal false flag. They didn't want to run the risk of being wrong, so they did what any other nation would do which was fire on them.
In WWII, German soldiers proficient in English would slip inside our lines and create havoc. That's why the soldiers had code words they could speak to one another to make sure they weren't the enemy. And those code words changed all the time, sometimes daily.
No such code existed between the US and Israel, so there was no way to verify if the USS Liberty was who they said they were, and since they were in an active warzone that they shouldn't have been in, it is entirely reasonable to assume they are an enemy vessel.