vicsis250

I have been experiencing a stiff brake pedal for the past 5 days every morning after the car has been parked all night and sometimes after it's been driven a few miles this only happens upon start up not or never while driving. I own a 2008 IS250 with 77,000 miles on it I'm second owner. I learned from my SA that I must have accidentally pressed on pedal before exiting the car after turn off. But I know for a fact that I didn't touch the pedal at all offer turn off and it's still doing it. Does any experts have any idea what's going on most recently it's been doing everyday before it would do it once in a while. Brake booster? Sensor? What do you guys think.