It's a tough call. I'd be inclined to agree that it's infringement of your rights, but children are susceptible to many teachings of things that are wrong being ok. I was always told the job of the parent was to monitor and correct these misteachings. I will admit it would be easier to raise my future children if the country were just a crackerjack version of my personal morality, but alas I don't see that happening.
That they are susceptible to being taught things that are wrong are okay doesn't justify teaching them more.
And that parent thing is a cop-out. Parents do the best they can, but cannot hope to compete with society and peer pressure.
Beyond any of that, however, is the simple difference that a homocidal sociopath, by definition, infringes on rights. A homosexual, by definition, doesn't.