I cringe when I hear stories like this! At the moment we are lucky, the neighbors on either side of us are cool, and we each have decent sized yards, so there is a buffer. We have no one behind us, just woods. The issue is that the house directly across the street for us is up for sale, and I am worried that some jack-ass is going to move in. Every time I see someone looking at the house, I want to run outside blast some death-metal, let my dogs run wild, and swear alot, just to scare them off, but my wife won't let me.
No one in my neighborhood is overly friendly, which is perfect. We wave, say hi, chat every once in a while and that's about it. I hear stories from my co-workers about neighbors that stop over all the time, unannounced, and stay for hours. That would drive me insane. Our next house will definitely have even more land, 5 acres minimum.
Oh, and on topic, I would NOT let the painters work on your property. As others have mentioned, you are potentially opening yourself up to alot of liability.