Shop around a lot; we did and found some great variance in rates. The Lending Tree and Ditech were the worst; we got less than six quotes between the two, and the rates and terms were awful. I even called them about their "thousands of lenders" and they said that was the best they could do even with perfect credit. Since 1996, we refinanced three times ($79,000 was the original amount) and got better deals each time. Our last refinance was in 2003 and we got 4.75% with no fees or points. It didn't ever cost us a fee or points to refinance. We overpaid a lot towards our house and will have it paid off in the early summer. Paying down the mortgage early and fast saved us a lot of money in interest over the years. Our house has dropped in value over the last few years from $225,000 to $200,000, but paying it off fast still means you aren't paying as much interest as you could have.
Our local Sunday paper has a business section that publishes rates. That's a good place to start. Credit Unions are a good bet as well. See what banks can do. Ask lots of questions, shop around, and don't feel pressure to sign right now.
