Pulled both sides apart today to see if I could tell what was rubbing - and no luck. I'm really at a loss what could be causing the rubbing noise..
The rotors came right off and went back on without any issues. It can't be the e-brake shoes because they retract in quite a bit. I thought maybe it could be the lip of the backing plate that the shoes and springs mount to so I tapped the edge in with a hammer - not sure if it really moved but a lot of rust particles were freed up.. cleaned it all with brake parts cleaner again.
Put it all back and the sound is still there, if not slightly worse. I'm thinking whatever is rubbing will wear down eventually and then I won't hear it.
All I can think of is that the rotors I purchased (Wagner) are ever so slightly different that they come in contact somewhere. I should note that while these rotors have the hole for the inspection plug, they do not have holes to thread the two phillips screws into the hub.