Ok, so here is the update. I went to a Lexus dealership and they found all of the struts to be leaking. So pretty much that was the main problem, had nothing to do with the wheels (as I have suspected). Changing the struts and ball joints made a tremendous difference in the ride quality. I can now ride freely on the 20" rims w/o any worries, regardless if there are bumps present. Very smooth ride, I am finally enjoying the Sc430.