There have been a few times when I stopped getting notifications, and I assumed that it was I replied without checking the box again. Now every time I reply to anything, I check the "notify" box - it's become instinctive. Not sure what the exact rules are, though.