Posts by auanasgheps

    INTRODUCTION

    If you like to spin-down your hard drives but you’re having trouble in doing so, you’re not alone.


    Not all hard drives like hdparm, the utility used by Debian and OpenMediaVault. There’s an alternative called hd-idle which does the job but introduces another issue: it will eventually fail if you use standby/hibernation on your server, plus other inconveniences.


    Hd-idle has been abandoned many years ago and was left in a buggy state, but we finally have a solution: a re-implementation of hd-idle with this bug fixed, along with many other improvements.


    CREDITS

    This guide would not exist without _Michael_ and his work in the original thread about hd-idle.


    Thanks to adelolmo on Github for rewriting and keeping hd-idle alive.

    INSTALLATION

    The new hd-idle can be installed in two ways: via an external repository or manually. I recommend the repository approach because it’s easier and will keep the program updated when you’ll run omv-update.


    REPOSITORY INSTALL

    Follow the instructions posted by the developer to add his repository:


    Code
    sudo apt-get install apt-transport-https
    wget -O - http://adelolmo.github.io/andoni.delolmo@gmail.com.gpg.key | sudo apt-key add -
    echo "deb http://adelolmo.github.io/$(lsb_release -cs) $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/adelolmo.github.io.list


    Update apt with apt-get update, then you're ready to install hd-idle with apt-get install hd-idle. You don’t have to worry about the old and buggy version that is still in Debian’s repo, apt will install the new version.


    Note: If you manually installed hd-idle and now you want to use the repo, just uninstall the package and start over. The config file should not be removed, but always make a copy before uninstalling.

    MANUAL INSTALL

    Download the latest package, linked here wget https://github.com/adelolmo/hd-idle/releases/download/v1.12/hd-idle_1.12_amd64.deb

    Install the package dpkg -i hd-idle_1.12_amd64.deb



    CONFIGURATION OPTIONS

    Before you begin, disable any spin-down in the OMV Web interface. Browse to Storage > Disks, edit each disks and ensure

    • Spindown time is set to Disabled
    • Advanced Power Management is set between 128 and 255. Lower values may interfere with hd-idle.


    The configuration file is placed in /etc/default/hd-idle. Edit this file with your favourite file editor.


    There's a lot to be said about hd-idle but this guide will only cover the basics. Check out the full documentation here.


    To actually enable hd-idle, change START_HD_IDLE=false to START_HD_IDLE=true.

    The default (commented) configuration is very simple:

    Code
    HD_IDLE_OPTS="-i 180 -l /var/log/hd-idle.log”
    • -i 180 sets the spin-down time in seconds
    • -l /var/log/hd-idle.log sets the log location, which is optional

    When using this configuration, all drives will be put in spin-down after 3 minutes of inactivity. It’s a bit aggressive: I would rather use a more conventional amount of 20 minutes (1200 seconds) to avoid stressing your drives.


    You may still want to use a short spin-down time if you have a very very very inactive drive such as a SnapRAID parity drive that is woken up only once per day for the sync job. If this is the case, you can further customize the configuration by using disk UUID or label. Do not use device name (eg:/dev/sda1) because they can change and things can get messy.


    Code
    HD_IDLE_OPTS="-i 0 -a /dev/disk/by-label/HDD1 -i 180 -a /dev/disk/by-label/HDD2 -i 1200 -a /dev/disk/by-label/HDD3 -i 1200 -l /var/log/hd-idle.log"

    In this example I've set the spindown time for HDD1 to 3 minutes, and other drives to 20.

    Trick: use the parameter -i 0 at the beginning (like above) to disable spin-down of not explicitly specified HDDs.


    LOGROTATE

    After writing this guide, I suggested to integrate logrotate: has been added in version 1.14.

    Just enable logs in your hd-idle config.


    What follows is the original section for archival purposes.


    SERVICE SETUP

    • Run this command to enable the new hd-idle at boot systemctl enable hd-idle
    • Run this command to start hd-idle right away systemctl start hd-idle


    If you get the error service is masked when trying to configure hd-idle, run the following commands

    Code
    systemctl unmask hd-idle
    systemctl start hd-idle
    systemctl enable hd-idle


    CONCLUSION

    You're ready to finally enjoy hard disk spin down! (done properly)

    Please note the full hd-idle documentation is available here. The new developer has done a great job explaining configuration options and log entries.


    If you need help with hd-idle, please write a post in the original discussion since you can't reply to this guide.

    Can you please share such a script?

    Is it also possible to have the script that does display all the file changes just save the changes to a log file and send out an email saying that it ran and with a short summary? This way emails are readable and if there are problems then one can refer back to the full log.

    That's what my script does!

    Please send me a PM so I don't forget (quite busy during these days)

    If you have a backup you should not worry too much.

    The drive has reallocated sectors and this is potentially dangerous. The drive could survive for some time, but it's still better to replace it.


    You could also consider HDSentinel for Linux. I've set it to send me an email every Sunday. It produces a nice report and exitates the life of the drive.

    It's just another safety measure but it only runs when invoked so it's still good to keep OMV SMART monitoring enabled.

    When I shut down my server, sometimes the monitoring service triggers reporting one of the file systems can't be read. This is expected, because the file system has been unmounted for shutdown, but this email should not be sent.


    This leads to heartbreaking moments, because this email is only sent when the system is back on.



    I found a couple of environment variables but I don't know if they fit the case and how they should be configured:

    OMV_MONIT_CHECK_INTERVAL

    OMV_MONIT_DELAY_SECONDS

    But clearly the UPS failed to manage this surge properly.

    I'm no UPS expert, but is it "healthy"?

    Blackouts are an easy task, but spikes from events like thunderstorms or network issues are unpredictable, although an UPS should manage it!


    Ill probably upload at least the last backup to Gsuite, unfortunately my 5 Mbps upload is quite poor and it takes time.

    You could use borbackup method instead of dd/fsarchiver. The first archive would be big as normal, subsequent backups only being with changed files.

    However I do not recommend this because Borg is not made to backup filesystems but only user files. I tried to restore borgbackups in test VM and did not work well at all.


    If you're using dd, switch to fsarchiver: only backs up the actual used space and compresses it. My fsarchiver backup is 1.5GB, down from 3.5GB. It would take 45 mins with your connection to upload it, maybe you can shedule it in the night.


    Otherwise run a SMB/NFS share from a USB stick connected to your router and upload it there. Rclone supports everything.

    I had a similar experience in the past when I was a kid and didn't had a UPS.

    Spikes like these can break your entire system and you're probably aware of this, but sometimes I've seen corrupted partitions/mbr during similar events.


    This is an event I'm accounting for, that's why I installed rclone and copy my backups to Google Drive every day after they're created.


    Here's my own-written procedure about. if you need more in-depth info let me know.


    lol I actually forgot I made this lonely post.


    I figured out that Rsync is not the ideal candidate.

    I played a lot with Borgbackuup which is perfect for this purpose, then wrote extensive guide here.


    The main pre-requisite is setting up SSH key authentication, then everything else is covered in the guide. I've done a lot of testing and Borg is solid and reliable, will go "in production" during the Christmas holidays.

    In pratica nessuna, l'ho installato di default e basta, non ho installato plugin, o altro.

    Purtroppo non so darti una risposta precisa.

    Considera che il mio sistema è installato da febbraio e occupa (stabilmente) 2.5 GB circa di spazio occupato.

    Certo, qualcosa potrebbe andare in errore e inondarti di log, ma è difficile. Di solito lo spazio si esaurisce installando Docker in maniera sbagliata.


    Ti consiglio a questo punto di rifare l'installazione appuntando ogni configurazione che fai, così sarà più difficile trovare l'errore.


    Ricorda che il sistema non fa nulla "da solo", non è Windows :P


    Login con ssh sulla console e inserisci df -h e enter. Copia e inserisci nel forum il risultato di df -h, così possiamo vedere dove son i dati.

    Giusto, ottima idea. Non ci avevo pensato!

    Quali applicazioni o container hai installato? È colpa di una di quelle se si esaurisce lo spazio.

    I would recommend this Docker container which has a GUI for youtube-dl, the best video downloader out, not only from Youtube.


    This GUI has customizable options and you can select MP3 output. I think is the best option and you don't have to rely on shady 3rd party services, all running from OMV!!


    EDIT: Interesting, I didn't know that the downloader plugin (which is the same software) was still supported as today.

    Ciao ragazzi.Un saluto dalla Toscana.Ho installato OMV su un Orange PI H5,una scheda simile al Raspberry,ma mi piacerebbe implementare il transmissio,per fare in modo che mi scriva direttamente neli dischi,e il PiHole.Attualemnte li ho in separata installazione,ma il Transmission mi da un po' di problemi.

    Qualche dritta?

    Grazie

    Sì, Transmission devi installarlo con Docker.
    Segui questa guida

    Yup making a live system clone with borgupdates is too advanced for me. For some reason it doesn't work their documentation says this should be the command

    I guess we should stick with fsarchiver, both of us confirmed works very well. Borg isn't really made for this purpose.

    My goal was to save on bandwidth - I upload OMV backups every night to Google Drive using Rclone, but it's not a big deal.


    FYI I did check out the source code and omv-backup plugin with option borg has the root hard coded in it. It will always back up only / directory. It will not care even if someone manually sets root dir form the plugin web interface.

    Thanks for checking it out, makes sense: Borg backs up files, so it's correct to hard code the / directory. Other methods do backup the partition instead.

    Borg/the plugin backs up the root file system, which you can specify if not detected automatically.

    I also believe the logic that picks up this file system is built in the plugin itself, in fact fsarchiver works fine.


    To restore, you need to mount the new empty partition and restore the borg backup: when this is done, you'll see system files and folders if was successful.


    Try to mount the backup done by the plugin when the system is running and check if folders of the root fs are there.


    At the moment I will stick with fsarchiver because I've already spent too much time on trying this scenario, but let me know if you find out more.


    The VM case from Borg docs does not apply here, I'm using a VM just to test this scenario without actually crashing my real server :)


    Fsarchiver is great in this case because it is actually made to backup a file system to a compressed archive.

    Borg is designed for normal file backup and I love it (I just wrote a guide about the borgbackup plugin) so probably is a bit borderline for this usecase.

    It should but grub is tricky to restore with just dd. I would boot the debian install iso and repair the install. It allows you to fix grub.

    Thanks.

    I discovered that

    • grubparts does not restore correctly if I replace the virtual disk with an identical one
    • grubparts restores correctly if I just format the disk/override partition table.

    I don't know if this could be limited to the VM environment.


    So I took the illetterate approach:

    - format the new disk to ext4

    - restore the backup

    - run Debian ISO and reinstall Grub


    It worked when using fsarchiver method but not borg. With the latter, Debian installer fails with a generic error:

    Code
    Unable to install GRUB in /dev/sda
    Executing `grub-install /dev/sda` failed.
    This is a fatal error.

    Applies to all partitions.


    That's strange. I'll keep looking into it.