Is it normal that the networking service is masked?
Nope.
From what I'm reading online seems Debian 11 should have the systemd-networkd services masked.
Which is not the case for me.
Could be it's conflicting with netplan? or netplan is using it?
Incorrect. Look at /etc/netplan/10-openmediavault-default.yaml and you will see that netplan is using networkd for its backend. netplan doesn't actually configure networking itself. It can work with network-manager as well. OMV just chooses to use networkd. What is the output of: dpkg -l | grep dhcp