Oh, the load times were real. And they sucked (though were never anything like 5 minutes).
I simply mean that consumers didn't choose CDs over cartridges, or load times over the lack thereof (not directly or consciously anyway). Consumers chose the console with the best library. You can, of course, rail on PS1 games as being nothing but FMV fests, but this is pretty far from the truth. Obviously this problem did exist, but not exclusively. PS1 gave us Tony Hawk, Parappa the Rapper, Metal Gear Solid, Resident Evil, Tekken (which is a ---smurfing--- retarded game, but other people seem to love it

), Twisted Metal, Wipeout, Fear Effect, NFL Gameday, Dance Dance Revolution, Spyro, Crash Bandicoot (another retarded game, but what are you gonna do?). Tenchu, Final Fantasy VII and Tactics, Medal of Honor, Castlevania: Symphony of the Night, Syphon filter, Grandia, Ape Escape, Silent Hill, Gran Turismo.
Seriously, I could go on. This list isn't anything like complete. Are any of these games as good as Zelda: TOoT or Mario 64? No way. But there were a lot of great games for PS1. Significantly more than N64. You can make a (probably spurious) argument that N64 had a higher ratio of good games to bad games. This is almost certainly not true, but the only thing that would really give you if it were true is a better chance of picking a good game from the rental store when you are blindfolded. If N64 only had 10 games total in its lifetime and 7 of them were great they'd have a 7/10 ratio, but I'd still rather the system that had 1000 games made for it with only a 2/10 ratio of great vs. bad games since that would give me 200 great games to choose from and I can easily ignore the other 800. PS1 had a much more diverse library that appealed to a broader demographic and they simply had more good games.
It is the better library that consumers chose. They would have chose the console that used punch cards as a storage medium if that was the one that had the most great games. Playstation was simply the console with the most great games. This isn't because the PS1 was a better console. It was clearly inferior to the N64 in virtually every respect. But Nintendo made developing for the Playstation a no-brainer for third parties so that's what they did.
By the way, you can pretty much copy and paste everything I just said but substitute N64 with Gamecube and PS1 with PS2. The scenario played itself out again. Gamecube's hardware is superior in almost every way. Gamecube's controller is the greatest gamepad ever made, while the PS2 is still using a souped up SNES controller with horribly
placed tacked on analog sticks. Gamecube had the Nintendo exclusives which were, as usual, the greatest games of the generation and among the greatest games of all time. But there were just a few of them and PS2, once again, had a far bigger library and, more to the point, had a far bigger selection of great games.