You're in luck. You CAN upgrade your factory stereo with anything you want in your vehicle. I just recently replaced my factory head unit, speakers and amplifier in my 2004 RX330 with Navigation without losing any functions in my dash. It's not for the novice but with time and the correct tools it can be done. I will do a full DIY with pictures soon but here's some areas of interest.
1- Contact Beatsonic to get the new trim panel and correct harness for your vehicle. Most likely you need the SLA-82 but in my case I used the MVA-82. The harness you use is based on year, ML or no ML, and Nav. or no Nav. Tip, beatsonic stocks other harnesses not shown on their U.S. site.
2- Select whatever double-din head unit you want. I'm using a Clarion VX409.
3- I suggest changing your speakers. I used Infinty Reference Series 6x9's in the front and Infinity Reference Series 6.5's in the rear. I cut out the factory speakers and retained the factory "basket" and simply dropped in the Infinity's and soldered the connections behind the speaker. I also installed Stinger Roadkill in the doors.
4- I went with 2 12" Infinity Reference Series subs. This might or might not be your thing but it allows me to take the heavier bass out of the doors. The sub is not permanent so I can take it out in a minute.
5- The mono and 4-channel Alpine amps are mounted under the upper compartment in the hatch where your spare tire jack is etc. I removed the bottom tray so the amps could get better air flow.
6- I also installed a 7-band Clarion EQ which is hidden from view.
Again, a much more comprehensive DIY is coming soon!
Words can't describe how much better music sounds now. It took two weekends to complete but was well worth it! I was in the same boat as you trying to find out how to upgrade everything. I found a few threads where people changed some things but never how or who made the modifications. An installer maybe? Anyway, I always messed around with sound systems from high school to my mid 20's and was glad to see in my mid 30's I can still do it!
Here's some pics of my system. Sorry for the quality but it got dark on me. Enjoy!