Can a bike save you money?

As a cycling novice, I'm trying to find out if it's really such a good idea in practice and if it'll save me any money.

In an effort to save money and get some free exercise, I started cycling a few weeks ago and so far, despite a few hairy moments, it's working out pretty well.