Your outlook contacts are stored within your .pst and not on the Exchange server, so unless your company backs up its desktops (doubtful but possible) deleting them from your Outlook contacts should do the trick. You might want to casually ask your IT guy whether any documents stored on your desktop are backed up, and if so for how long. He'll probably tell you no and that you need to make sure that all your important documents are stored on your personal share on a file server. That's the answer you want. But... (always one of those), if you have emailed people from work then the email messages themselves have the email address contained within them (but no other personal info you might have stored like real name, address, phone, etc.). You could also ask your IT guy whether they archive the data on the Exchange servers. My company does because we are required to by the FERC, yours may not. If they do you can ask for how long. We are required to keep them for a few years but jettison them as soon as we can (again for legal reasons).
If you want to erase your IE history click on tools - Internet options - Clear History. That should do it. On this same tab you can delete files and cookies as well. You can also delete all your bookmarks if you want.
Don't use a magnet on your hard drive. You'll cause a lot of trouble for your poor, over worked IT support guy. Just delete all your stuff and move along to your next job. No sense burning unnecessary bridges.
fwiw, I doubt that anybody at my work would be the least bit interested in who my friends and family are and what Internet sites I've visited. YMMV.
Cheers.