It makes the car more unstable.

Less forgiving of driver error.

Not very pleasant at the end of a 2 hour drive, for example, when your concentration is likely to be degraded.

 

We are not saying that more camber doesn't help in certain circumstances, but on a road car it's a balance of the suspension geometry, spring weights, damping, vehicle weight, and many other variables.
On the LS that balance is weighted towards comfort and all-round ability.
On an M3 that balance is weighted towards cornering and steering feel, but an M3 has very stiff suspension and is not what I would call comfortable.
On a track day car or a "sports" car that is accepted but on a limo where comfort and quiet is the goal, you really don't want that.

As for the weight, the 2004 (E46) M3 is 500+lbs lighter than the 2004 LS430. Later model M3's got heavier probably because of safety structures.

The E46 has a factory rear camber of -2° where the LS has -1°.
Neither are adjustable without fitting different camber arms.