On my 95 ES300, when I shift the car into reverse my light surrounding the R on my console nor the rear lights don't come on unless I wiggle the gearshift til I find the spot and then once I let go of the shift they go out immediately. Fuses and bulbs are good.

Problem #2: I noticed recently that when I press on my brake peddle or wind the windows up and down while driving at night my headlights go dim. Is this a sign that my alternator is on its way out?

 

I have that problem on my 98 too, do a search on how to and clean your neutral safety switch (shift indicator switch). Lights dimming? Check battery connectors and test battery, your alternator is probably fine.