I just had my rear view mirror repaired for the same problem (peeling, dull, streaky mirror) at radar-mirror. Couldn't be happier. Looks like a brand new mirror and I can finally see behind me!
Took me a while to figure out how to take the mirror off, but it is not hard at all!
1. Just remove the hex bold from under the mirror. It is exposed so you can just take a 3/16 or so size hex key or a tiny flat head screwdriver.
2. you would need to remove the 'nose' and the cover on the roof panel. On the '06 GS430, the panel you remove is the most forward one, in front of where your sunglass holder drop down is. It pops right of, just do it gently and carefully. I did it with just my fingers. You can put your fingers in the groove facing the windshield and work your way to the sides and back. It is held by clips to the sides, with two big white clips on the left and right. Once you pop the panel, the 'nose' or middle piece should come with it. The nose is to hide the wires that run all the way to the roof panel.
3. Once you remove that, you can slide the mirror off of its support. It's kind of like a hook support, kind of what you would use to hang your towels in your bathroom. To slide it off, just slide the mirror upwards. The support itself is attached to the windshield. By this time your mirror will be dangling by the wire (for power, compass etc.)
4. Slide the clip from the wire from the connector in the roof assembly. You would then need to carefully wiggle the clip out of the opening to fully get your mirror. At first it may seem impossible and you may seem tempted to cut the wire. DON'T. you can push up on the cloth/roofliner piece to push the connector through the seemingly small opening and carefully wiggle it out. Trust me, it will work.
To reassemble, just follow the reverse. First, I hooked the mirror back on the windshield support by sliding it down and hold it in place. Next I pushed the clip back into the small opening and connected i back to the connector in the roof assembly. Then I put the cover and nose back on and slowly push back until everything clicks back in place.
* You do NOT need to remove the wire clip on the mirror. You will break it. Some of you may not even have it anymore. If you break it, no biggie. All it is is basically a big plastic wire tie.
* You do NOT need to remove the bottom cover on the mirror assembly (where the hex bolt goes into). This is where the circuits are.
I hope this made some kind of sense...