So both my side mirrors are doing different wrong things. So my options are to buy 2 $600 Lexus mirrors or buy two $160 Ebay mirrors, but either way, they won't color match the car and will need to be painted.
Left mirror doesn't tilt down any more, not a major thing, but I like the feature.
Right mirror does tilt down, but that's only after I wrench it back into place from folding itself backwards every time I start the car. It folds properly when I turn it off.

Is it possible I could swap out the guts from the new mirrors into my color matched covers? Would it be very complicated?

 

Sounds like you need to take the mirror apart and take a look at the mechanism.
https://www.clublexus.com/forums/ls-...with-pics.html
There are a variety of gears in there that are probably gummed up, especially if someone has been squirting lube in there.
If you use the wrong lube on plastic gears it causes all kinds of problems.
You may also need to replace the Mirror adjustment button as dirty/corroded contacts in there can cause weird behaviour.