Congrats on the decision airb! Cycling is a great way to have fun & stay in shape.
As long as you buy from a real bike shop, and stay away from department stores or Canadian Tire, you should find lots of decent bikes to choose from. For good value for your money, Specialized and Giant are a couple of brands that spring to mind, but there's others, too.
Aside from the gear you've mentioned, you'll want some gloves and bike shorts as well. The shorts don't have to be spandex, but real bike shorts with a proper chamois (i.e. crotch padding

) will make cycling much more enjoyable. Stiff-soled bike shoes are also nice, but not absolutely necessary to get started.