Migrating from FreeNAS to Open Media Vault while also importing FreeNAS ZFS to ZFS on Linux - Roadmap Advice sought please

  • Hi crashtest,


    That is exactly what i did.


    You looked at the picture before i made the changes.


    Below is after the changes you suggested.


    Did not check any box under User / Group Permission.


    Read/Writ/Execute selected for Others.


    Recursive check box selected.


    Replace check box not selected.


    But still getting the same error.


  • OK there are two possibilities from here. (The second is pretty much guaranteed to fix this.)


    First:

    Do you have OMV-Extras installed? If not, the easiest way to install it is on the CLI.


    SSH in as root:

    wget -O - https://github.com/OpenMediaVault-Plugin-Developers/packages/raw/master/install | bash


    When this is complete, you'll see System, OMV-Extras in the GUI. Click on OMV-Extras, and the Settings Tab and enable the Testing and Extras Repos.
    ______________________________________________

    In the GUI, look under System, Plugins. Scroll down and look for openmediavault-resetperms.
    Install it.

    This will add a third tab to Access Rights Mangement, Shared Folders named Reset Permissions.

    Click on the Tab.

    Shared Folder: Use the drop down to select backups.

    Permissions : Set to Everyone Read/Write

    Clear ACL: ON (green)


    Then click Reset Permissions.

    (I would set the shared folder back to None and save it. This sets the default condition of the tab.)

    See if that fixes it.

  • Thanks crashtest,


    I actually had already installed OMV-Extras and Rest Permissions.


    I think i tested on one of the Pools as well.


    Just in this case i had not ran it on the pool on which i was trying to modify files. My Bad :-)


    After your suggestion i again ran on all pools and things look to be perfect now.


    I am able to read and write to all pools.


    Thanks for you patience and support. :-)


    OMV has an excellent community. I think i would stick to OMV moving forward. :-)

  • Some closing notes:

    1. Those command lines that I posted earlier set the pool and datasets to a very close approximation of posix permissions and ACL's, but those ZFS attributes are "inherited". (I think they should be defaults for ZOL, and they may now be the defaults, but I don't know for sure. I haven't checked the features of the latest ZOL versions.)

    However, note that the attributes of the existing files and folders in the pool and child filesystems may not have these attributes. That means you may have mixed attributes in the files and folders in the pool and filesystems, going forward. (I don't know how to change them recursively.) This may have no effect on you at all. If that's the case, great.

    If there's some weird permission issue in the future, with the old data, the only way to recursively change file and folder attributes is to copy the old data into a new filesystem (like Backups2), then copy it back into the original Backups filesystem. When the copy out is done, the new attributes will be inherited when the now "new" files are copies back. Keep that in mind if you have weird permissions issues with the old files and folders.

    2. You might give thought to installing the Proxmox Kernel. The reason is, ZFS modules are preinstalled in the Proxmox kernel which might save you from some issues in the future when upgrading.


    3. With or without the Proxmox Kernel, do not update the kernel or ZFS packages without first backing up your boot drive. (Trust me on that.)

    4. Here's some more attributes to consider:
    Sub in your pool name for ZFS1


    zfs set compression=lz4 ZFS1

    Saves space on uncompressed files. Near zero CPU impact.

    zpool set autoreplace=on ZFS1

    If you have a dead or dying drive in a zmirror, RAIDZ1, etc., and it's physically replaced, the above will auto integrate the new drive into the pool.


    5. If you're interesting in automating local snapshots, take a look at this doc.

  • Thanks for the valuable inputs crashtest.


    If i see any issues with permissions moving forward i would consider your suggestion on copying back the files to and fro from another storage.


    I might even consider getting rid of ZFS all together. Since I don't use ZFS RAID anyway. I just use rsync to copy important files on another disk based on a daily schedule.


    I am already using Proxmox Kernel. Installed that before installing ZFS.


    Would keep your suggestion of not updating the kernal and ZFS packages before i have a backup.


    Was looking at the below video to backup the OS USB Drive. Lets me know what you think about this process:



    issued the command "zfs set compression=lz4 ZFS1" on the pools that contained Data. Would there be any use issuing this command on the pool that contains only multimedia files, like movies and songs etc. Since i don't think they would be compressed.


    Never used snapshots on FreeNAS either. I replicate the data on this NAS to another NAS periodically to have an offline backup of things.

  • Never used snapshots on FreeNAS either. I replicate the data on this NAS to another NAS periodically to have an offline backup of things.

    An off line backup is good, but it gives you only one choice when restoring files. That is the exact state files were in, as of the last backup. If they're corrupted or contain a virus, and get backed up, it's over. They're permanently lost.

    Snapshots give you versioned backup. If you read the beginning of the ZFS auto-snap document it gives a little abstract on what snapshots do. Basically, if you change a file, the original is still maintained, in it's original state, in the prior snapshot. That gives you "version" protection in that you can return to any previous state the file was in for a configurable time. After the time period lapses, older versions of a file or folder are "purged", on rotation - first in, first out.

    Since I have some turnover in my Video folder, I only save 1 month of snapshots. When it comes to pictures and music files, which are relatively static, I maintain snapshots up to one year. Snapshots give you the option to return to a prior state, say, before a virus infection or before ransomware encrypted your files. Snapshots provide complete immunity from malware.

    issued the command "zfs set compression=lz4 ZFS1" on the pools that contained Data. Would there be any use issuing this command on the pool that contains only multimedia files, like movies and songs etc. Since i don't think they would be compressed.

    There won't be an issue but, if they're already compressed (most media files are) a second compression of an already compressed file won't save much space. In any case, it won't hurt anything and might save a bit of space with new uncompressed files.

    I might even consider getting rid of ZFS all together. Since I don't use ZFS RAID anyway. I just use rsync to copy important files on another disk based on a daily schedule.

    I use ZFS in a zmirror (the equivalent of RAID 1) for bitrot protection which I think is important. With a single disk, say on a backup device, I might consider EXT4 or XFS. (But again; snapshots for versioned files/backup, even if they're on the same disk, would make it worth running ZFS on a single volume. Complete malware immunity, especially if it's accidently/ backed up in an automated process is worth considering.)

    I'm not sure how you're using rsync to backup, but there's a method of creating a backup server that doesn't require getting on the CLI. Remote Mount.

    Was looking at the below video to backup the OS USB Drive. Lets me know what you think about this process:

    This subject is covered in the User Guide -> OS Backup, with enough detail so that beginners can stay on track and understand the reasons "why" they should backup. There's no easier way to maintain OS backup than booting from a USB thumbdrive. (And there's the added bonus of saving a SATA port for an additional hard drive if needed.) Cloning thumbdrives is dirt simple. It doesn't require knowledge of 3rd party app's or CLI operations.

  • Thanks crashtest,


    You have a very solid point on using snapshots for backup and restore. I didn't think of it from that angle. Would enable snapshots right away.


    Enabled compression on the Multimedia pool as well. Maybe a little bit of space can be saved there. Lets see.


    I was still getting permission errors. So decided to go as per your suggestion and recreate all pools and copy data again. Took me some time to copy over 10TB of Data to and fro between internal disks while recreating each ZFS pool. But it was still much much faster as compared to copying data over the network.


    Best regards,


    Adeel Akram

  • I am current having a challenge configuring rSync though. Appears that the source needs to be a shared folder.


    In my case i have a folder hierarchy. For Example i created a shared folder named data and this folder has multiple folders like office data, personal data etc. I want to use rSync for one of these sub folders not the main folder for each pool. Is that possible in any way?


    Best regards,


    Adeel Akram

  • I want to use rSync for one of these sub folders not the main folder for each pool. Is that possible in any way?

    Yep. You can run an rsync command line as a scheduled task, with no shared folder required if the copy is local. An example of how to do that is -> here. That documents a local drive-to-drive copy. All you have to do is drill farther down to the sub-dir of your choice.

    As an example:
    rsync -av --delete /ZFS1/ /srv/dev-disk-by-label-DATA1/ (This is in use and it copies a ZFS pool to an external drive.)


    The above could be:
    rsync -av --delete /ZFS1/pictures/ /srv/dev-disk-by-label-DATA1/pictures/ (This would copy a child filesystem in the pool to another drive/folder.)


    A piece of advice: Leave the --delete switch out of the your command line until two copies are confirmed.

  • Hi crashtest,


    Found a way to browse the subfolders from the GUI as well.


    But even after recreating all Pools and copying data back and forth. I still cant seem to get rid of the permission issues.


    When i try to add the source subfolder i get the below error>


    Failed to execute command 'export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin; export LANG=C.UTF-8; setfacl --modify group:users:--x /Backups/Backups 2>&1' with exit code '1': setfacl: /Backups/Backups: Operation not supported

  • Ok, I think the option in the GUI is to create a folder not share an existing one.


    So i guess the rsync command through a scheduled task is the only option.


    Thanks for the suggestion.


    For the snapshot. Do i need to install the zfs-auto-autosnapshot for automated scheduled snapshots to make auto snapshots work or can this also be done through a scheduled task?

  • Please google about ZFSsend and ZFSReceive

  • Do i need to install the zfs-auto-autosnapshot for automated scheduled snapshots to make auto snapshots work or can this also be done through a scheduled task?

    No. zfs-auto-snapshot is something you'll have to install on the command line. If you haven't used it yet, you'll need to install PuTTY.

    The installation itself does nothing more than set up a series of scripts and cron jobs that utilize. This is both the up side and downside of auto-snapshot. It's simple and has no software dependencies, with next to nothing to break or go wrong and there's no software packages to be upgraded, ever. (And it will be this way into the foreseeable future.) On the other hand, zfs-auto-snapshot lacks the features of a more sophisticated package like ZnapZend. As raulfg3 suggested, ZnapZend can utilize the commands zfs send and zfs receive.)

    (BTW: I gave thought to writing up a How-To for ZnapZend but it has package dependencies and there were dependency issues with Debian 10, around the time that OMV5 came out. Sometimes, simple is better than sophisticated, especially where new users AND the command line are involved.)

    Once set up, zfs-auto-snapshot runs in the background. There's a doc for it -> here. Read through it and try to get an understanding of it first. Otherwise, the document walks through the setup, and it's pretty straight forward. The document goes to great lengths to make it understandable and easy to configure.

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!