You can match the voting rolls and signatures
Which still doesn't match up to a ballot.
After the signatures are matched and the envelopes are opened, the ballot is separated from the envelope, forever. There is no longer any way to match a ballot to an envelope. That's how a non-Stalinist regime does things, in order to keep everyone's vote a secret.
So, say you've done your audit. It will find nothing wrong, but say it did. How would you know that the cheating wasn't done by Republicans? After all, given the Republican history of vote fraud, that would be the logical assumption.