I would probably add a fuse somewhere along the AC path if I were building this setup, even if there is a general fault in the wiring where something eventually falls on it and shorts it, it's good to fuse it in addition to protecting the relay rating. Whatever type of fuse is used, it has to be rated for the 120volt AC (or 250) along with the proper current handling. I am not too familiar with automotive fuses, being a 12volt system, are those fuses rated for household AC voltages? If not, have to maybe buy an inline fuse holder (anywhere, radio shack even) that can accept a "normal" AC fuse and has wires you would solder in line with the AC hot side.
So with that application of using the fuse, I'd wire it physically close to the source AC lines before they go to feed the relay contacts rather than fusing the downstream load network after the relay contacts switch the AC. That way if there is a short or overload on the main AC source wiring, the fuse will do a better job in protecting both the relay/load network as well as any general AC wiring faults that show up on the path towards the relay circuit. If you put the fuse somewhere after the relay and before the load, and somehow a short develops on the AC source before it gets to the relay, there is no fuse there to protect the short (the relay and its load and the fuse would be all shorted out of the equation).
As for what fuse current rating, I think this application is light enough that just guessing on a value like 8A would be fine, because we dont expect the load to come close to the relay rating and having a fuse with more than the load but less than the contacts would allow the load to have extra start up current spikes without nuissance fuse blowing and still stay within the contact ratings. So as close to 10A as possible I'd use.