The MP3 player is fairly barebones. It can read subfolders, so my music is organized by folder. It reads and displays ID3 tags when a file is selected or played. When playing, the bottons control RW, FF, Pause, next track, etc. There's no visualizer, support for playlists, or any other WMP/WinAmp/iTunes type features.
I'm expecting that a future release of the player software will offer far more in the way of features.
There's a "hold" position on the power switch. Slide the power switch down into "hold" and the PSPs buttons are all locked out. The idea is that you start playing an MP3, turn on "hold" and then slide the PSP into your pockets secure in the knowledge that an accidental button press will not ruin your musical nirvanic state. The screen will automatically turn off after x-minutes.
I have a 512MB SanDisk Pro Duo card that holds a small set of music files. I think it's the Apple 512MB iPOD Shuffle that cites 120 tracks and/or 10 album capacity. I'd suggest that spring for the 1gig stick until the 2gig sticks drop in price. The new high performance "gaming" sticks appear to be of little extra value on the PSP. The regular Pro Duo cards will do you fine for MP3 playback.
PSP plays MP3 and ATRAC but I've never bothered with anything but 128/256/192kbps MP3 and I've been happy enough given the quality of the included headphones. It won't play Apple's AAC format or MSFT's WMA.
I cannot comment on battery life. I plug it in often enough that I have yet to test it. I'll test that during travel later this month, but I'll ruin the test by playing games along the way.