Yes, you need to fuse the power wire going to the relay. Relays are actually cheap, it is the fuel pump you are protecting with the fuses and those are expensive. I would use a relay for each pump.
You can pick up the relays at Radio Shack and most likely any autoparts store. I normally get...