Depends on what you mean by "electronic" and how you do it. Digital logic hardware? I poke and prod at that stuff all the time. I mean, it's mostly 5/3.3/2.5/1.8/1.2V from pretty current limited supplies. Not much that can go wrong there safety-wise. Worst thing you may do is damage the device by shorting something to a bad place, but that happens during development. Debugging new hardware designs pretty much implies that you're going to be poking it while it's on to take measurements and tweak settings (pots, switches, etc.).
Now, "electrical" hardware that runs right off the AC line? Yes, be very careful and avoid doing things with it on wherever possible. If you've got an inherently limited isolation transformer, things aren't quite so dangerous, but you definitely have to be careful and know what parts of the device are at what potential relative to others. The isolation transformer at least means that nothing is "hot" relative to earth ground (unless you re-introduce earth ground - again, know what's going on). If you have to run something with a real "hot chassis" on a bench, that requires the utmost caution of course. Unfortunately, it's sometimes required. I just did it. And yes, I was very respectful of what was going on. Also, I used a GFCI. It at least prevents some classes of faults from being quite as dangerous.
How much you can avoid working on things while they're on really depends on the nature of your work. If you're mostly doing machine-level integration of commodity parts, you can probably get away with almost never poking things while they're on. However, troubleshooting new and unknown/unproven designs often requires it. Of course, the unknown/unproven aspect just makes it that much more dangerous, so you ALWAYS must watch out. Even those dinky 5V/1A supplies I mentioned above can start a (small) fire in the right circumstances or even kill you outright if you manage to break the skin on both sides of your heart (e.g. by poking a finger on opposite hands).