I've got Nextcloud working perfectly on OMV 5. Now granted, I'm using duckdns/letsencrypt. I had a lot of trouble trying to set up my own domain. I'll figure that out when I have more time to give to it (thankfully it was only a few bucks..lol).
As it stands however, setting it up with my duckdns domain and letsencrypt was no problem.
I also had some trouble. But in the end it was the domain provider who made it super difficult to configure the cname.