Thank you everyone for chiming in. After testing several configurations out, my tentative conclusion is that it is a bug of some sort. It must not happen to everyone because if it were the case I would have heard of it. There hasn't been a mass outcry of vmdk eating 2~3 times the size of physical disk space when Windows 11 VM is concerned.
1. It seems like encryption does have something to do with it. I tried creating Windows 10 VM (no encryption required) instead of Windows 11 (encryption required), and I had no issue creating a pre-allocated 280GB of vmdk in a 500GB disk.
2. I was able to create a 100GB pre-allocated vmdk for Windows 11 on a 500GB physical disk. So obviously VMware thinks that is good enough. From there, however, instead of installing Windows 11 on the 100GB blank vmdk, I went into the VM setting and attempted to expand the size by 100GB (total 200GB). And VMware completed that task without complaining.
Encouraged, I attempted to add another 100GB for total 300GB which is about what I originally planned. And what do you know, that works, too! So VMWare wouldn't let me create a 280GB vmdk for once, complaining that I need total 850GB of blank disk space, but if I did it in 3 (wasteful) steps, it was none the wiser.
I am not sure what to make out of this but at least now I know a workaround.