For what it's worth, solid state media, thumb drives, SD cars, etc, do have transfer rates. Levels would still need to be loaded into RAM, etc. I don't know enough about the transfer rate to know if it is so fast as to be negligible for videogame purposes, but I know that some of these mod cartdridges for the Nintendo DS suggest that you only buy the highest quality, fastest microSD cards because otherwise you will experience slowdown in your games. I know that when I transfer large files from a flash drive to a PC and vice versa, I do have to sit there and wait for it to read from and write to the drive. It's not instantaneous.
Nevertheless, even if the load times stayed exactly the same, and I don't think they would, flash media like SD cards still seem like the best option. I'm not sure how I feel about purley downloadable games. I mean, what if I want to bring my copy of Resident Evil 8 to play on my friend's PS4. Do we really want to lock all our games to a specific machine the way the Virtual Console does with NES games?