Anyone using Google Voice (GV) in their Lexus?

I've set up GV so that—while driving—I can verbally tell Siri to call someone or to send someone a text. That part works well.

However, when I'm speaking with someone on the phone via GV (doesn't matter if I initiated the call or if it was an incoming call to my GV number), the person on the other end hears me very faintly. I can hear them perfectly.

Has anyone had this issue and, if so, how did you resolve it (assuming it is resolvable)?