The different versions have different features unlocked. They're all the same OS though and each has the exact same performance.
Ultimate is more expensive and comes with extra features which you probably don't need (such as bitlocker, remote desktop and others features which are intended for business and IT pro usage).
Here is the list of the major differences in features:
1. The ultimate edition has BitLocker while the home premium edition does not.
2. The ultimate edition has AppLocker while the home premium edition does not.
3. The ultimate edition is capable of booting from a virtual hard drive while the home premium edition is not
4. The ultimate edition has the BranchCache feature while the home premium edition does not.
5. Additional enhancements to VDI and MUI languages are present in the ultimate edition while not in the home premium edition. (allows you to install windows in multiple languages)
All in all, you are probably fine with home premium but if you can get ultimate for the same price you can consider that as well. (take the below upgrade path and the costs in consideration however)
Take note that you can't downgrade to a lower edition of a newer windows edition; for example you can't install Windows 8.1 over Windows 7 Ultimate so you will have to get Windows 8.1 Pro to upgrade over Windows 7 Ultimate.
If you would go with windows 7 home premium, you can upgrade to Windows 8.1 or Windows 8.1 pro.
In other words: you can only upgrade to a version which has the same features or more features, and you can't downgrade to a version with less features.
So if you want to upgrade 7 ultimate, you will have to buy the Pro edition for Windows 8.1 which is more expensive than the normal 8.1 edition.
@CSB
Yes the 64 bit version is the way to go, 32 bit is starting to get outdated and is already deprecated for Windows Server OS's.
64bit has some small benefits over 32 bit, such as extra security features build in to the kernel and a bigger address space which allows you to utilize more ram (memory).
In a few cases it can be faster than 32bit as well, although this is mostly with video encoding and database applications.