By "silver handles" I'm assuming that you're talking about the stainless steel ones.
If you look at the exploded view of the P360 assembly, they tell you to use the .500" bushing.
If you are using a stainless handle, the bushing is not required. It's only for the stock P360 handle.
I also found that the spring that came in the P360 was just as strong or stronger than the loose spring.