If you believe in evolution, you could just as easily ask where pre-humans got their sense or morality. For example, chimpanzees absolutely have their own chimp culture and chimp morality. There are all sorts of social rules among chimps involving grooming and mating. If this is not morality it is something very like it. According to chimp morality, it's absolutely moral to kill or at least chase off all members of other chimp tribes wherever you meet them. Peaceful coexistence is not a thing in chimp morality.
Of course no one knows if the first humans ascribed to chimp morality, or if they were more peaceful. But while morality is an individual thing, it changes and evolves over time. Humans don't really do cannibalism or slavery anymore. They used to be considered quite moral and even necessary. Now they are highly immoral.
I don't see where you need a government for morality to evolve. Not saying it can't play a role, but I'd say morality controls more how humans govern themselves than government controlling what is considered moral. I'd even say the same for religion. Many Jewish dietary restrictions can be justified for health reasons without bringing God into the mix.