My RX 350 has almost 150k miles and recently I started getting a faint rubbing noise coming from the rear end. I have been trying to isolate it to one side or another and cannot. The noise comes and goes and sound like something is rubbing against the tire but visually I could not find anything. The weird thing is that I cannot isolate it to either side! Any ideas? Anything like an axle or differential that fail at 150k miles?

 

My advice is to take it to a Reputable Lexus/Toyota repair shop and get it checked out, it could be a number of things causing the noise.
It could be an Axle Bearing, a Carrier Bearing in the Differential, a Brake Caliper hanging up, Brake Pads dragging, a Rusty and Bent Brake Backing Plate, a Wheel Opening Liner, a Loose Wheel or Axle Nut and More.
Sometimes it's difficult to pinpoint the location of the noise, and if it's something serious, You need to get it fixed ASAP!
With 150k miles, it could be any of those things, Good Luck in finding the Noise.

 

Rubber, or metal sound?
First check,
Caliper dust covers (inner)?
I had to cut/rip the caliper covers off as mine had rusted and were rubbing against the tire and rim.

If not, then yeah, what cfh13427 said.