I was driving yesterday and within the 3-4 miles my right front brake was smoking. After cooling it with water and letting it set for a little I attempted to drive it home, it was still far too hot to drive so I left it in a parking lot overnight. When I picked it up and drove it the three or so miles home today it seemed fine, although I was very light on the brakes the whole way.

Is there anything I should be checking out before I replace the caliper?

 

Caution: do not apply water to the hot brake components. This can greatly weaken and damage the metal.
I recommend checking the sliding pins on the calipers; make sure they are properly lubricated. If they are not, apply grease to the sliding pin and reinstall. Here is a demonstration video to help:
Hope this helps.