Hmm, think I know what it is now! Found other people having issues with TimeMachine-backups on Big Sur if Sophos was installed, I do not have that, but on my work-machine I have "Crowdstrike Falcon".
Updated one of my private machines to Big Sur, and Time Machine works just fine on it!
So, it must be an issue with "Crowdstrike Falcon" on the work-machine! Will open a ticket with them!
Sorry for the "red herring"!