Guinea pigs can eat a lot of veggies. If you also get veggies, you may be able to share some. They still should have pellets, but really don't need more than 1/8 of a cup a day, so even a small bag can last quite a while. They also need vitamin C in their diet, which can come from veggies, supplements or pellets.
One way to save money is to buy hay in bulk. A bale of hay might cost $10, but should last close to a year (depends on how much they eat). This can drastically lower hays costs, especially when a small bag can cost $5-10 and only last a few days.
Bedding is another area where costs can be cut. Many people use fleece and towels. It may cost a bit more upfront, but will last a long time and just needs to be washed when dirty. I do recommend having a few sets for the cage so you can just put in a clean set while you wash the dirty stuff.
Make sure you do a lot of research before getting a guinea pig. They are social animals, so do best in same sex pairs or trios. This does mean you need a bigger cage, so look into a C&C cage. It really should not cost more than a pet store cage and can offer a ton more space.