No matter what bulbs I put in my 1998 Lexus LS400, the passenger side map light doesn't work, could it be something like a fuse or even one of the wires connected to the light falling off? I just ordered a set of BA9 led bulbs for it since I haven't tried led yet and I figured the lower current draw would help

 

If Driver's side working, then possibly a contact issue with the alternate action pushbutton switch.

Locate a voltmeter to assist you.

 

The driver's side does work, that makes sense. How would I go about fixing this? taking the button off and trying to clean it internally?

 

map lights rely on a ground to work properly. try pushing up the housing while the switch is on to see if the light illuminates. if it does it's a grounding issue which uses the housing as a ground (more specifically a bolt holding the housing to the roof).
does the door courtesy light turn on too?

 

On my '91, I had a similar problem. From testing with a voltmeter, I found the issue to be with the rivet that goes through the housing. It is supposed to hold things together and also serve as a conductor. But I think the surface corrosion (barely noticeable visually) over time degraded this electrical contact. I think I fixed it by drilling a hole and putting a screw through there to replace the rivet. Fresh metal and tight contact fixed the circuit.