Cyclists do pay income tax, property tax, etc which all pay for roads. Cycles do not operate on highways, which I'm pretty sure gas taxes would hardly cover maintaining.
Sure cyclists can duck responsibility, but so can drunk drivers, pedestrians, and anyone associated with the police force.
Downtown, especially in Toronto, the issue is that there is too much real estate allowed to single occupant cars with insufficient bike infrastructure. This slows down cyclists and transit users who can use the space more efficiently. As much as I love cars ( we're on a car forum!), motorists do not cover the cost of infrastructure. A mild inconvenience to those sitting on their asses in a climate controlled controlled box doesn't seem reason enough to ban cyclists on the streets.
Re the 1m rule meet: the whole point of the gap is to allow the cyclist some space to swerve, be it to avoid a chipmunk or pothole