Amazon - Great prices generally, but sometimes need to order more than needed. Being online you can't touch the item, so it's harder to know if it's what you want or the quality you want.
JOANN Fabric & Craft Stores - You can get all your supplies here and see them in person. Can be overwhelming, but also fun.
Michaels - They have prepackaged fabric, and most of the other supplies needed.
Your local sewing store: Mine is Crimson Tate which also has online sales.
Etsy: So many beautiful fabrics and unique supplies. Beware, you can fall into an Etsy hole looking at pretty things for hours.
Other: Walmart, Hobby Lobby, Dollar Store.
In a Kit: You can get many types of kits cheaply, but cheapy can also mean bad quality. Also watch out for tiny spools of thread. They are not what we need for our sewing machine, but they do work for hand mending. This Eversewn Kit is great I like this kit because I know the qualities of the supplies. It does have some things you don’t need, but are nice.