Before you start work inside your computer eliminate the hazards.
The first performance upgrade you should consider is more memory. It can double or triple the speed of bargain computers that only have 128 MB of memory. You need at least 256 MB for Windows XP. Before you upgrade anything else to get more speed you should have at least 512 MB. Windows 98 will run fine on half those amounts. Read more about memory selection at ExtremeTech, particularly if you're considering a new computer
Go to Crucial if you want to buy memory online. They're the online arm of Micron, who makes memory for most of the top computer brands. Their website makes it easy to find exactly the right memory for your computer. Read more about selecting memory at the "Memory" page.
Upgrading your CPU is more ambitious than it used to be. It was easy to upgrade computers built before 1998 but not the ones sold today. A worthwhile upgrade today would be expensive and difficult to do. It's usually better and cheaper to buy a new computer than to go down the CPU upgrade route.
Upgrading the CPU alone won't enable an upgrade to Windows XP either. The XP upgrade costs $100: you might as well put that, plus the cost of upgrade components toward a new computer. Depending on what brand of computer you have, you might be able to replace the whole motherboard and then upgrade XP. That might not be cost effective either.
A bigger hard drive (disk) won't make your computer run perceptively faster. That's just an urban myth. If you simply need more room, consider adding a second hard drive.
Adding a second hard drive is easier than installing a bigger primary drive. It adds wonderful flexibility, adds capacity and redundancy for backups, and can even speed up your computer. You'd have to transfer Windows and all your files if you just get a bigger primary drive, and that's not a simple task.
You might consider an external (USB) hard drive as an alternative -- particularly if you have a newer computer that has USB-2 ports. USB-1 might make it too slow for your purposes. If you have more than one computer, an external drive is a big advantage. You only need one. You can then use it to shuttle all your files between computers, say from Happy Trails to home and back.
More on hard drives.
Add a CD-R/RW burner: They're great for backups -- CDs are the most durable medium you can use. (I'm not fond of RW myself -- you might as well just use the plain "R" type, as cheap as they are now.) Make music CD's, photo albums, etc., with it too. CD "burners" are a great upgrade for your computer.
Prices for DVD burners are now low enough to be interesting too. They offer much more capacity and should be more durable.
http://pcin.net/help/articles/harddrive.shtml