As you describe it it seems you need a pretty beefy x86-64 NAS. I think calibre is the clincher.
But I don't do what you describe. So I have several tiny ARM32 NAS. And run calibre on my laptop.
Calibre is pretty simple.
I've got Calibre-web (keeps me from having to download the Calibre app.. I can just read in any browser).. Doesn't seem to be overly taxing on resources, so I'm not sure how calibre would be difficult for a Pi to manage... especially if you end up transcoding with Plex or Emby.