You make a rational point; thank you for that.
The problem is that pot is both very dangerous and very addictive, and the more it's available legally, the more it can seep down - and has seeped down - into underage use.
No one under 25 should use it except in very specialized/supervised circumstances, and it's generally not great for over 25s either, but at least not as dangerous.
So while it isn't equivalent to the drugs it's been grouped with in many ways - as you say - its danger to the young/vulnerable if rescheduled is great.
That's my sole concern.
If strict liability applied to its distribution, and/or a truly safe strain were to be developed which doesn't induce psychosis with prescribed use, I would no longer give two shits about how its addicts/pimps behave.