Well, first, get a pandora (tool) battery
Dealextreme sells excellent stuff according to forumers
DealExtreme: $11.71 Hybrid 2-in-1 Digitally Activated Pandora PSP 2000 Unbricker Service + Normal 1200mAh Battery
This is a digitally activated Pandora, restore it with square in the pandora menu or something (I forgot, do the option where it says "Restore pandora to original battery" or "Restore IDstorage keys" or whatever.) You do not need TimeMachine for this one. $11.71.
DealExtreme: $9.50 PSP Unbricker Pandora Battery for PSP 1000/2000 (1800mAh/3.6V)
This is a non-digitally activated battery. You will need TimeMachine to boot this one - it's just one extra step anyways, not that hard

there are tutorials for it everywhere!
$9.50
DealExtreme: $17.33 Pandora PSP Unbricker Service Battery + Memory Stick Duo 2GB Combo
This is the above option with a 2GB memory card
$17.33
DealExtreme: $22.28 Pandora PSP Unbricker Service Battery + Memory Stick Duo 4GB Combo
This is the 2nd option with a 4GB memory card
$22.28
Then, get a MMS
» Rain’s UltraLite MMS Maker v1.1 Released PSP Hacks · PSP Homebrew, PSP Downloads, PSP Mods
Then, downgrade! n.n
Choose the "4.01" option.
After that, do a network update (it's patched, so it does hacked firmwares only!), and upgrade to 5.01 M33-3. Install TimeMachine 0.1 if you want to - that makes your pandora battery usable as a normal battery.
» TimeMachine v0.1 Released PSP Hacks · PSP Homebrew, PSP Downloads, PSP Mods
Also, check out
PSP Hacks · PSP Homebrew, PSP Downloads, PSP Mods for new updates on homebrew and such.
And customizepsp.co.uk for custom themes (Current version of cxmb is 3.3, it works with 5.01m33)
Oh, and if you have a Slim, get the Tool battery for the slim. Bewarned there are unhackable PSP's. What model is your PSP? If it's PSP-3000 or PSP-2000 TA088-V3 we have a problem. But if it came with less than 3.x you are okay.
PM me for my email/skype for further info.
Another copy-pasted answer by fishyfish