you should be fine just deleting them and also emptying your trash in the e-mail system.
Depends on who is doing the search.
Are you afraid of Uncle Ernie hopping on your computer and seeing your steamy love letters, or Uncle Sam digging through your files for conspiracy charges?