Replacing 1TB HDDs with larger HDDs in RAIDZ1 (ZFS)

  • Hello! I'm kind of a beginner in Linux and hardware in general, but I've figured out just enough to put together a small NAS (which I'm using as a home/family media server) using a Raspberry Pi 5 and a Radxa Penta SATA HAT (which I gather is a fairly common setup). I'm using Jellyfin, installed directly on the Raspberry Pi, to manage and stream everything, and despite some wonky file permission issues at the outset, it's running fairly well so far.


    My initial storage setup is fairly simple: I used the ZFS plugin to incorporate 4x 1TB HDDs into a single storage pool using RAIDz1. Based on my initial research, I figured that this setup had just enough redundancy and flexibility that, if I were able to afford one or more larger HDDs in future, I could swap out the individual 1TB HDDs for larger ones one at a time, resilvering / rebuilding the filesystem each time and hopefully expanding the available storage with each replacement. (I didn't foresee that this might cause excessive wear and tear on the HDDs, something I've learned since.)


    Of course, that time has come much sooner than expected. I've found a pretty good deal on a pair of 4TB HDDs with the same form factor (2.5"), and, sure enough, my existing setup is filling up fast, so I feel it's best to get ahead of the game and expand my storage now. My ideal course of action would be to remove two of the existing 1TB HDDs and replace them with the 4TB HDDs, giving me about 10TB of actual space (more like 7.5TB with the RAIDz1, I know). My questions are:


    1. What problems can I expect to run into if I try to swap out a 1TB HDD with a 4TB one? Will I actually expand my storage capacity, or will it be limited by the relatively small size of the existing HDDs?


    2. If I try to swap them out one at a time, would the repeated resilvering process wear out the disks?


    3. If yes to #2, what other course of action can I take? (see below!)


    4. On the software end, is there a way to accomplish all of this in OMV directly, or is this stuff best done using the command line?


    Another approach: the SATA HAT does come with an eSATA port for attaching a fifth drive, hence its name. Theoretically, this could allow me to: (1) attach one of the new 4TB drives to the existing setup, (2) expand the storage pool with the new drive in OMV, (3) remove one of the 1TB drives and attach the 4TB one in its place, (4) restart the system and make sure everything is working properly, (5) repeat with the second 4TB drive. I'm thinking that this might cut down on the amount of resilvering that would need to happen otherwise, thus keeping the disks healthier for longer.


    Any thoughts from the community? Keep in mind that I'm still kind of a beginner, and I'm working on a budget that doesn't really allow for a lot of shiny new gear. Thanks!

  • crashtest

    Approved the thread.
    • New
    • Official Post

    1. To expand the capacity of an existing Raidz1 pool, you need to replace all the hard drives in that pool one by one. If you only replace one (or two), the capacity will remain the same.


    2. Yes.


    3. Point 1 invalidates your proposal.


    Perhaps you could consider using the 4TB hard drives as multimedia storage without RAID. If, in addition to multimedia, you also have sensitive documents on the NAS and want them in a RAID array, you could configure a mirrored RAID with two 1TB drives. The two 4TB drives would then be used for multimedia storage, formatted in EXT4. MergerFS can make these two drives appear as a single drive. You can even combine the ZFS pool within the MergerFS pool to have a single data folder with two subfolders: documents and multimedia.


    This would give you 8TB of storage for multimedia in EXT4 and 1TB of storage for documents in ZFS.


    If you don't have sensitive documents or data, you might not need RAID. In that case, a MergerFS pool would give you 10TB if you include the two 4TB hard drives and two 1TB drives. You could add another 1TB drive to the fifth port if you want, giving you 11TB.


    To do all this starting from your current setup, the easiest way would be to connect a 4TB hard drive to the fifth port, format it as EXT4, and copy all your data there. Then, disassemble the ZFS pool and configure it as needed. You can add or remove file systems from a MergerFS pool without any data loss.

  • Okay, this makes sense! I'm not really storing sensitive documents at the moment, I'm just using RAID mostly because I'm dealing with second-hand drives and I'd prefer to have some peace of mind in the unlikely event that something fails before its time (which I understand is unlikely if I'm not doing frequent, intensive R/W operations, but still - I have a very cautious approach to storage in general).


    That said - because nothing that's currently on the drives is irreplaceable, I could certainly see myself moving away from the current RAID configuration, and instead putting together a setup where I use the two 4TB drives for video, plus two of the 1TB drives for music (which has significantly smaller filesizes). I think I'd still like to use a basic RAID array (or, preferably, ZFS pool) for each group - primarily for the increased read speed, and secondarily because I prefer the flexibility of not having to mess around with multiple file paths and determine what fits where if the drives fill up. How does the following plan sound?


    1. attach 4TB drive to fifth SATA port, make it its own *basic* ZFS pool, transfer all current data

    2. shut down system, remove two of the 1TB drives, attach *both* 4TB drives in their place

    3. add the second 4TB drive to a basic ZFS pool with the first, totalling 8TB of space

    4. create a new, blank ZFS pool with the remaining two 1TB drives, totalling 2TB (for music)

    5. optional: combine both pools into a single data folder with two subfolders


    Thinking about step 3, though: would this require data to be redistributed across both drives, effectively resilvering and potentially wearing out the disks? (If it helps at all, this would only be done once; if I need to expand again, I'll save up for a pair of larger drives and use the same process described here.)


    Also, if I end up storing documents and other sensitive data outside my main computer, I'll likely just save up for parts for a second NAS, and configure it with a mirrored RAID array of some kind. My ultimate goal is to divest from proprietary cloud storage entirely, while still maintaining a bit of flexibility for collaborating, sharing files, and working on-the-go.

    • New
    • Official Post

    I think there's something being overlooked. If you don't need parity, you don't need ZFS. Formatting in EXT4 is sufficient.


    In any case, if you plan to store sensitive data on your NAS in the future, keep in mind that RAID is not a backup solution. https://www.raidisnotabackup.com/

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!