So my new to me '04 LS has had the trunk lid replaced by the previous owner, and currently it's very finicky for me to actually get it to open.
I've observed the following symptoms (Just as a heads up, the valet key lock has been unlocked, so that isn't the issue here):

First off, the trunk release latch in the cabin does nothing no matter if the ignition is off or on. I can hear the sound of an electric motor as I pull on the latch when the car is turned on, but the trunk itself doesn't budge at all.
Secondly, pressing the trunk unlock button on my key fob (which seems to be an aftermarket replacement) also doesn't unlock the trunk. If I press and hold it for 2 seconds when the car's unlocked I can hear a beep coming from the car, but the trunk still doesn't budge.
Thirdly, pressing the physical button on the trunk itself (under the lip) doesn't work when the smart key system is off.

However, the physical button on the trunk does work when I enable the smart key system, unlock the car, and then turn the ignition to the "on" position (where the dashboard lights up); at this point I just need to hold the button on the trunk and lift the tailgate manually. This has been my only method of getting the trunk to open so far, and it clearly isn't supposed to work like this.
As this seems like a rather peculiar problem, I wasn't able to find someone with a similar issue and solution on the internet, thus I'm posting here again hoping that some knowledgeable users can maybe help me out. Since this is a replacement trunk lid, the mechanical master key isn't going to turn the cylinder. What might be the cause of the issue here and what do I have to replace? I live in a country where there's just not a lot of Lexus dealers around me in close vicinity, so the best options are to source parts online and do a DIY repair or hit up a locksmith or repair shop and ave them check things out.