Assistance Needed for Seamless ZFS Pool Transition in OMV

    • New

    Hello OMV Community,


    I'm encountering an issue with a consistently degrading ZFS pool located at /srv/Wharf/. This pool, built with SSDs, is used primarily for user home directories and Docker storage. To address this, I have created a new replacement pool at /srv/fast/, also built with SSDs.


    Currently, I'm using the following command to copy data from the old pool to the new one:


    Code
    rsync -aq /srv/Wharf/ /srv/fast/ &


    However, I'm unsure how to seamlessly transition the active mount from /srv/Wharf/ to /srv/fast/ without causing significant downtime. I suspect one potential approach could involve:


    1. Stopping Docker (e.g., using docker kill $(docker ps -q)).
    2. Unmounting the degraded ZFS pool at /srv/Wharf/ (I'm uncertain of the correct procedure for unmounting a ZFS pool).
    3. Changing the mount point of /srv/fast/ to **/srv/Wharf/** (again, not sure how to modify the mount point for a ZFS pool).
    4. Changing the mount point of /srv/Wharf/ to /srv/WharfOLD/ (again, not sure how to modify the mount point for a ZFS pool).
    5. Rebooting the system.

    I would greatly appreciate any guidance or best practices on how to perform this transition smoothly. Any help on the proper commands or configuration changes needed to achieve this would be very valuable.


    Thank you in advance for your support!

    ajaja

    Version7.1.0-2 (Sandworm)
    ProcessorAMD EPYC 7302P 16-Core Processor :evil:
    KernelLinux 6.1.15-1-pve
    HardwareDell R7515
    • New

    Please let me know if this is valid direction:
    From ChatGPT:

    Below is a recommended approach for achieving a seamless ZFS pool transition in OMV. The goal is to minimize downtime and ensure that your services (especially Docker) have a smooth transition from your degraded pool (/srv/Wharf) to your new pool (/srv/fast). This procedure assumes you have a maintenance window available:


    ---


    Recommended Transition Procedure


    1. Full Data Synchronization:

    - Before you begin the transition, ensure that all data from the old pool is completely synchronized to the new pool. You mentioned using:

    Code
    rsync -aq /srv/Wharf/ /srv/fast/ &

    - Verify that all data is in sync and perform additional runs if necessary.


    2. Prepare for Downtime:

    - Notify users that the system will be offline briefly.

    - Stop all services that write to or read from the pool. For Docker, you can stop the service entirely rather than individually killing containers:

    Code
    systemctl stop docker

    - Additionally, if other services depend on /srv/Wharf, stop or temporarily disable them.

    Code
    3. Unmount the Old ZFS Pool:

    - Since ZFS pools don’t unmount in the traditional Linux sense, you’ll want to export the pool so that it’s no longer mounted. Ensure no processes are accessing it, then run:

    Code
    zpool export <old_pool_name>

    - Alternatively, if you want to keep the old pool online for fallback, you can change its mount point to a temporary location:

    Code
    zfs set mountpoint=/mnt/old_wharf <old_pool_name>


    4. Update the Mount Point on the New Pool:

    - Set the new pool’s mount point to the location formerly used by the old pool. Assuming you want the new pool to be accessible at /srv/Wharf:

    Code
    zfs set mountpoint=/srv/Wharf <new_pool_name>

    - Verify the new setting with:

    Code
    zfs get mountpoint <new_pool_name>


    5. Verify and Test:

    - Check that the new pool is mounted correctly at /srv/Wharf:

    Code
    mount | grep Wharf

    - Verify that all your data is accessible.

    - If you’ve changed the mount point of the old pool rather than exporting it, double-check that you’re not encountering conflicts.


    6. Restart Services:

    - Once you’ve verified that the new pool is mounted at /srv/Wharf and all data is present, restart Docker and any other dependent services:

    Code
    systemctl start docker

    - Monitor your logs to ensure that services are accessing the new pool correctly.


    7. Final Clean-Up:

    - After confirming a successful transition and stable operation, you may choose to permanently export the old pool or repurpose it as needed.


    ---


    Additional Tips


    Backups:

    Before making any changes, ensure you have a complete backup of critical data in case you need to revert.


    Testing:

    If possible, test this transition procedure in a lab or staging environment to work out any issues specific to your configuration.


    Documentation:

    Review any OMV or ZFS documentation regarding changing mount points and exporting pools to ensure you follow best practices for your version.


    Monitoring:

    After restarting services, monitor system performance and logs closely to catch any unexpected issues early.


    ---


    Summary


    By fully synchronizing your data, stopping services, reassigning mount points via ZFS commands, and then restarting your services, you can achieve a relatively seamless transition from your degraded pool to your new pool with minimal downtime. This method avoids the need for a full reboot and minimizes the risk of data inconsistency during the switch.

    Version7.1.0-2 (Sandworm)
    ProcessorAMD EPYC 7302P 16-Core Processor :evil:
    KernelLinux 6.1.15-1-pve
    HardwareDell R7515
    • New

    depending on the type of content and file size, rsync can be painfully slow.


    from my limited knowledge zfs send/recieve it the recommended way of transfering data between pools. i have recently discovered bzfs https://github.com/whoschek/bzfs which looks like a wrapper around zfs send/receive. I havent tried it myself but the time will come soon, as i also possibly will need to recreate my zfs pool

    SuperMicro CSE-825, X11SSH-F, Xeon E3-1240v6, 64 GB ECC RAM, LSI 9211-8i HBA controller, 6x HGST 12 TB ZFS

    Powered by Proxmox VE

    • New

    The rsync just completed. I will run it one more time, then shut down

    docker and run the final time before proceeding.


    Version7.1.0-2 (Sandworm)
    ProcessorAMD EPYC 7302P 16-Core Processor :evil:
    KernelLinux 6.1.15-1-pve
    HardwareDell R7515
    • New

    All is well.
    ChatGPT for the win.
    All Docker containers seem to be working as expected.

    Code
    root@omv:~# zfs list 
NAME     USED  AVAIL   REFER  MOUNTPOINT
Zenith   785G  1013G   785G   /srv/oldWharf
fast     787G  1.03T   787G   /srv/Wharf
root@omv:~# zpool list
NAME    SIZE   ALLOC  FREE   FRAG  CAP  DEDUP  HEALTH
Zenith  1.81T  785G   1.05T  17%   42%  1.00x  DEGRADED
fast    2.78T  1.15T  1.63T  0%    41%  1.00x  ONLINE
    Version7.1.0-2 (Sandworm)
    ProcessorAMD EPYC 7302P 16-Core Processor :evil:
    KernelLinux 6.1.15-1-pve
    HardwareDell R7515

  • ajaja

    Added the Label resolved
    • New

    ajaja Not a win for ChatGPT at all. Using rsync in this case is the wrong tool for both speed and the need to unmount stuff because otherwise what you are trying to copy is changing. The correct tool is "zfs replication", a combination of zfs snapshot and zfs send and zfs receive.


    A simple example of replicating data from one pool to another:


    Clean up snapshots as required:


Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!

Register Yourself Login