I am a Capcom fanboy and the PS2 has done plenty to feed my need. Buccaneer mentioned the Megaman collections. I have the original Megaman collection, but have heard good things about the Megaman X collection. Additionally, the Capcom Classics collections are about $20 a pop (there should be another one coming out in November) with a bunch of side-scrollers, platformers, and other genres to screw around with. For every one you don't like, there is one you'll love.
Volume 1 includes 1942, 1943, 1943 Kai, Street Fighter II, Street Fighter II Hyper Fighting, Street Fighter II Champion Edition, Ghosts’ N Goblins, Ghouls’ N Ghosts, Super Ghouls’ N Ghosts, Son Son, Exed Exes, Pirate Ship Higemaru, Vulgus, Commando, Mercs, Gun Smoke, Section Z, Bionic Commando, Forgotten Worlds, Legendary Wings, Trojan, and Final Fight.
Volume 2 includes 1941, Avengers, Black Tiger, Block Block, Captain Commando, Eco Fighters, Knights of the Round, King of Dragons, Last Duel, Mega Twins, Magic Sword, Quiz and Dragons, Side Arms, The Speed Rumbler, Street Fighter, Super Street Fighter II Turbo, Strider, Three Wonders, Tiger Road, and Varth.
If you like Ghouls n' Ghosts, the Maximo series is basically a 3D rendition of it. It's a fairly difficult game, but it's an older game and you can get it pretty cheap (somewhere between $7 and $10).
Finally, there is a Sega Genesis collection coming out that also looks good and should also retail for $20. The list for this collection includes Alex Kidd in the Enchanted Castle, Altered Beast, Bonanza Bros., Columns, Comix Zone, Decap Attack starring Chuck D. Head, Ecco the Dolphin, Ecco II: The Tides of Time, Ecco Jr., Kid Chameleon, Flicky, Gain Ground, Golden Axe, Golden Axe II, Golden Axe III, Phantasy Star II, Phantasy Star III: Generations of Doom, Phantasy Star IV: The End of the Millenium, Ristar, Shadow Dancer: The Secret of Shinobi, Shinobi III: Return of the Ninja Master, Sonic the Hedgehog, Sonic the Hedgehog 2, Super Thunder Blade, Sword of Vermilion, Vectorman, Vectorman 2, and Virtua Fighter 2.
All these games are cheap and have decent replay value. Good luck with your selection.