I've seen guys cart home some serious inventory for extremely cheap on Craigslist.
A few companies make starter kits at a very affordable price. You usually get a vice and materials along with instruction to tie a few patterns. I started out with a $50 kit from Gander MT. a few years ago and now it's out of control.
Hareline Dubbin, Rumpf and Umpqua make some kits as well. The more you spend on the kit the better the vise,materials and quantity of materials will be.