Possible limitations of OMV when using command line ?

  • Hi all,


    I am a long time user of Qnap and Synology NAS. But I am fed up with two things, that both Vendors "suffer" from:


    1)

    when I go to the command line and I change a configuration on the command line (because it is not available via the web frontend and I know how to do it on the command line) chances are high, that my changes won't survive the next boot. Mostly because the whole firmware file system is rebuilt from scratch at every boot.


    2)

    several standard tools (like e.g. samba, rsync, etc) are modified with proprietary company specific extensions. This has led to the situation, that features, I expect to work, don't work (at least not as expected) or that I run into errors, that nobody else in the internet has (except for other users of that vendor). I have lost way too many hours of my lifetime chasing such phantoms.


    Now I am investigating OMV. So far I have the impression, that OMV is using just the plain unmodified tools, that everybody also uses. So I will not run into the situation, that there are missing features (in e.g. samba or rsync), or that I run into errors, nobody else will know. Can somebody please confirm this?


    My impression is, that for OMV the file system with Debian and OMV on top is not recreated from scratch, so I have better chances, that any changes will survive a reboot, but I fear, that I may still run into the situation, that changes I make using the command line (just because, the web frontend doesn't provide that functionality) will disappear after the next reboot. At least i came across some threads here in the forum, where changes via command line were clearly discouraged.


    Is there anybody who can tell me, which changes via command line will be overwritten / deleted by OMV? (and which ones are save to do)


    Many thanks, quack

    Qnap TS-853A

    Qnap TS-451+

    Qnap TS-259 Pro+

  • The basic rule of thumb is that anything that is configured in the OMV interface should not be messed with in the cli. If you mess with a setting via the cli that omv controls, you can break omv or the setting will be overwritten the next time a chage is applied in the onv gui.


    Some thing's like samba have "extra options" fields that you can use for customizations, but there are also some customizations that can be done via omv environment variables. If the environment variable exists, it's best to use them instead of the extra options fields if you want the customization to apply everywhere.


    Other than that, the developers can speak in more depth about the "under the hood" workings if there are more questions.

  • The basic rule of thumb is that anything that is configured in the OMV interface should not be messed with in the cli.

    That sounds obvious. And if things can be customized via web UI I would do so. It is rather about things the web UI doesn't provide means for...

    Or maybe if there is at least some way I can find out, if changes are safe to be made or not.

    Qnap TS-853A

    Qnap TS-451+

    Qnap TS-259 Pro+

    • Official Post

    And if things can be customized via web UI I would do so. It is rather about things the web UI doesn't provide means for...

    Or maybe if there is at least some way I can find out, if changes are safe to be made or not.

    OMV uses configuration management which means omv code is source of truth regardless of changes you have made. If you see a managed by OMV header, then you cannot change anything in that file without having it overwritten at some point.


    Many OMV plugins or core settings have extra options where you can set your extra options. Samba is one of those. But if you want to have full control of your configs, you should use something like webmin instead of OMV (not together).

    omv 7.5.0-1 sandworm | 64 bit | 6.11 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.15 | compose 7.3.2 | cputemp 7.0.2 | mergerfs 7.0.5 | scripts 7.0.9


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    Edited once, last by ryecoaaron ().

  • Well you only mentioned samba and rsync and as I said, samba has customization capabilities. rsync is rsync. it has a gui page, but you can also do it via cli if you wish. You can make custom scripts and run them however you see fit, or schedule them (or normal cli commands like rsync) with the task scheduler (basically cron with a gui).


    Coming from proprietary solutions, I think you will be surprised with the flexibility of OMV, just don't expect an app store like those other ones. Docker is basically your app store. However, without more direct questions I have no idea on what you are trying to find information on.


    As an example of what I mean by flexibility, my system runs docker with 50+ containers, kvm with several qemu VM's and several lxc's. Some of those things are internet exposed through a reverse proxy. I have custom scripts made to do automated rotating backups for all the dockers, vm's, OS, and important data the way I like to do it, even though there are backup options in the plugins. I use portainer to manage my dockers, even though there is a docker compose plugin because I find it suits my needs better. Most native debian cli tools can be added if required, but be wary of ones that would cause updates to php or python. The omv interface uses nginx, so you can't install apache natively, but you can use a docker for it or another nginx to allow you host pages or do custom reverse proxy setups (there are also nginx-proxy-manager, SWAG, traefik, haproxy, etc. dockers that are prebuilt reverse proxies). Things like webmin and olivetin can be installed natively as the web servers they use are "micro servers", not nginx and they use non-standard ports so there is no interference.


    Once again, this is my setup as an example. It's the best I can tell you without the afore mentioned "more direct questions"



    ***edit***

    just a point to extend what Aaron said about webmin. You can install it (I have it installed strictly for the easy file manager), but it will not allow you to circumvent the omv configuration management he mentioned. OMV is still the "final word" on configs. Installing it without OMV would give you full unrestricted control the same as if you were using a GUI-less server.

    • Official Post

    just a point to extend what Aaron said about webmin.

    I meant webmin without OMV. Definitely not together.

    omv 7.5.0-1 sandworm | 64 bit | 6.11 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.15 | compose 7.3.2 | cputemp 7.0.2 | mergerfs 7.0.5 | scripts 7.0.9


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • If you see a managed by OMV header, then you cannot change anything in that file without having it overwritten at some point.

    Thanks, that was heplful.



    BernH

    After some sleep, I realized that my answer yesterday night was rather impolite. Please accept my apologies.


    Then thanks for your (long) answer and the description of your system.

    That there is no app shop and that I have to use docker is no drawback for me. I am already either using docker on Syno and Qnap or I use their equivalent of apt-get.


    Well you only mentioned samba and rsync


    I only mentioned those two, because I run into problems with them, since they were modified by Qnap / Syno and that I spent hours chasing errors, that only existed because they did those modifications (GRRR). So that example was not so much related to configuration but rather to lost life time and as the reason, why I want to move away from Qnap / Syno.


    I can give you a few examples, where I had to make workarounds, like using scripts to modify config files after the NAS was up and running.


    1) NUT:

    Qnap and Syno both use NUT to control UPS, but they have different users / names / passwords etc. So out of the box they are not able to communicate and thus can't use the same UPS. It is however possible to customize the configuration files in a way, that they are able to communicate. But this is gone after a reboot.


    2) rsnapshot / rsync:

    I use rsnapshot for my backups. Since rsnapshot uses rsync, I sometimes have to configure the rsync server (e.g. adding modules - as they call them), so that rsnapshot can do remote backups. This is gone after a reboot.


    Another example, that I might come across with OMV is managing BTRFS:

    As far as I was able to see, BTRFS is supported by OMV, but only to format one (or more) HDD in single mode or RAID1. If I want to do do something more (eg. adding a disk and enlarging the pool or simply replacing a failed disk) then I will need to go to the command line. That's not a problem. I either know how to do it, or I can read the man pages etc. to get the needed knowledge. The question is rather: will I mess up my NAS / OMV or will the changes get lost.


    Or yet another example with disks:

    Synology has a very special Raid (SHR). Using this, you can have a Raid5 consisting of disks with different size without wasting any space.

    To my own surprise this is not some proprietary modification, but just a creative usage of partitions, raid (md) and logical volumes (lvm). So this "Sliced Hybrid Raid" can be put together just using the mentioned standard tools. (I can explain this in detail, if you are interested)


    I know how to do this (and I even tried it with OMV in a virtual machine once, where it worked), but since it is always said, that partitions are not supported in OMV, I am at least cautious...


    And to maybe get the view back to reality:

    I am completely happy to use the WebUI for those 80% of functionality, that I need most of the time. I wouldn't like to do this via command line. I know, that I could do, but I am not that familiar with Linux, that I known all those commands for Systems Administration without looking them up. So I am really happy to use a WebUI. I really am.

    But every now and then I come across something, that is not covered by the WebUI. And I just want to be able to do those small things via command line without losing them after the next boot.


    I am confident, that I am able to configure NUT on OMV in a way that will allow communication with either the Syno or Qnap (who ever will be the master). I have not yet looked into the possibilities of the rsync server in OMV. But as I said, these are just examples from the other systems.


    With OMV I might run into other topics that I can't even think of now. Before running into the problems with NUT I would have always expected that every NAS is able to communicate with every other NAS (at long as both use NUT). Therefore I have a hard time to come up with topics that have relevance for OMV. But they might have to do with communication between the different Systems.


    But knowing, that OMV writes a special header into those files, will give me a heads up, when I see it, and I can then ask here in the forum.

    Qnap TS-853A

    Qnap TS-451+

    Qnap TS-259 Pro+

  • After some sleep, I realized that my answer yesterday night was rather impolite. Please accept my apologies.

    No worries. No offense taken.


    Regarding your other points:

    I don't use NUT, so I can't comment

    Any rsync stuff I do is not using an rsync server

    I have custom scripts I run that do a similar job to rnasnapshot using rsync, so once again I can't really coment.

    I don't use BTRFS. I have a mdadm raid 5 and a raid 1 using xfs, all with the same size drives so I am not trying to mix and match. If you are trying that, the best route with OMV is to use mergerfs with snapraid on top, designating the largest drive as the parity drive. Most prople doing it are probably using ext4.

  • I do have two questions, that I was not able to find out during my experiments with the virtual machine:


    How do I know, when / if a hard disk fails?

    I would have expected some indication on the WebUI, that I can't miss.

    But I didn't see anything obvious...


    The other question is about backing up the system / installation.

    I will have the system / installation on an external USB stick. I know, that there is a plugin, that is meant to backup that USB stick to another one, but I couldn't figure out (yet), which of the many plugins to use for that.

    Qnap TS-853A

    Qnap TS-451+

    Qnap TS-259 Pro+

  • Disk failure would probably be best looked after using the monitoring and notifications settings. SMART monitoring can warn you before a disk completely fails.. This needs to be enabled per disk. There is SMART widget you can put on the dashboard. There are also notifications that can be configured to send you an email alert when something fails, and of course in the case of a drive, the SMART widget should also reflect the failed status.


    If you are installing on a USB stick, make sure you install the flash memory plugin. It will redirect the logging to RAM and only flush it to disk occasionally. This will greatly reduce writes to the stick and increase the lifespan of a USB stick OS drive from months to years. As for backups with a USB stick install, I would recommend keeping a clone of the usb stick that you freshen up periodically just in case that USB goes read only as they have a finite number of write cycles. Clonezilla or dd can look after the cloning.


    As for the actual data backups, I personally use fsarchiver to backup my ssd os drive, as it can back up a live filesystem, but as with a lot of my backups, this is a custom script I run to do things the way I like to do them. You can however use the backup plugin if you want to use the built in tools. This plugin can use fsarchiver, dd, borg, or rsync to backup the drive.


    You should also think about installing omv-regen. This is a brilliant little cli application written by Chente, that will gather and backup all the settings that you need to restore all your configurations on a freshly installed system, and it can even update the system before it does the settings dump. This is great for ultimate disaster recovery or even just to check the configs and replicate a section if you happen to "nuke a bunch of settings and can't remember or figure out what they were before.


    My backup solutions is to use fsarchiver doing rotating full filesystem backups, and omv-regen doing the rotating settings dumps as an extra safety.

  • omv-regen really sounds nice. Haven't read about it yet. Thanks for the hint. Where do I get it? apt-get?


    I will definitely install the flash memory plugin. This is already on my list.

    And yes, I want to keep a clone of my installation on a second USB stick. I have read here in the forum, that the recommended way is, to have a second stick in a second USB port, so you can just change boot order, in case the old USB stick fails. This second USB stick must be regularly (e.g. once a week) updated with a clone. There seems to exist a plugin specifically designed for it, but I wasn't yet able to identify this. I will look into fsarchiver. Maybe its this.


    As to the notifications: During my experiments I didn't activate neither monitoring nor notifications. Didn't make any sense for me in an experimental virtual machine. So yes, this might be the reason, why I didn't see anything...

    Qnap TS-853A

    Qnap TS-451+

    Qnap TS-259 Pro+

  • hat the recommended way is, to have a second stick in a second USB port, so you can just change boot order, in case the old USB stick fails. This second USB stick must be regularly (e.g. once a week) updated with a clone.

    No.

    You can't have 2= OS drives connected at the same time.

  • Why not? As long as I don't tell the system to boot from it, it should just be a second drive, that happens to contain the same data files.

    There would not even be the need to mount it, since cloning works directly on the block device.


    BTW: I currently have a very similar situation. I have a USB stick attached to my NAS with a Debian system on it and at the same time I have another Debian installation on one of the installed hard drives. I could use Grub to choose where I boot from, but currently I have two independent instances of Grub, each one bootstrapping the system on its own device, and I switch using the BIOS.


    Which problem will there be with OMV, having two drives with a system on it?

    Qnap TS-853A

    Qnap TS-451+

    Qnap TS-259 Pro+

    Edited once, last by Quacksalber ().

  • There would not even be the need to mount it, since cloning works directly on the block device.

    Your BIOS will see 2x boot drives with the exact same UUID.

    It doesn't matter if it's mounted or not.

  • The best thing to do to clone the drives would be to boot into clonezilla and do a disk to disk clone (or if you don't want to keep 2 around at least do a disk to image that can be written back to another usb stick) or to use dd to do a disk to disk clone.


    omv-regen can be found in the guides section of the forum:

    forum.openmediavault.org/index…&postID=377038#post377038
  • Quacksalber


    Since you edited your post, I need to add more info to mine:

    You CAN'T have 2 clones attached at the same time on ANY system.

    Same goes if you clone DATA disks.


    If they are boot systems (the OS drive of OMV, for eg), your BIOS won't care if it's a Windows, Linux, Android, whatever... boot drive.

    The BIOS will only try to launch a bootable UEFI/MBR drive and it will do it on the 1st one it finds. (Not always the same)

    Since it is a 1:1 copy of another, you have no idea which one you are booting. Same goes with the system.


    What is even worst is, you can end with the system booting from one drive and mounting the rootfs of the other, because it has the exact same UUID. You will never have a clue that you booted with the USB stick on port1 and the system was mounted with the rootfs of port2.


    Bottom line on the idea of cloning the USB stick is:
    Either
    Power down, clone to another, use the "another"


    OR

    Make a dd-full to img of the OS drive

    Flash to "another"

    Use "another"


    Nowhere, you'll see to have BOTH connected at the same time

  • So this is specific to cloned drives...


    Ok, that topic of the UUID id new to me. Does this UUID belong to the hardware, the partition, the file system? Just asking to find background information easier.


    And thanks for the heads up. Looks like I did understand something completely wrong.

    Qnap TS-853A

    Qnap TS-451+

    Qnap TS-259 Pro+

  • Does this UUID belong to the hardware, the partition, the file system?

    glossary - What is a UUID? - Stack Overflow


    The UUID in Linux operating system explained | Linux Today

    Quote

    It's an identification number that will uniquely identify something. The idea being that id number will be universally unique. Thus, no two things should have the same uuid. In fact, if you were to generate 10 trillion uuids, there would be something along the lines of a .00000006 chance of two uuids being the same.

    linux what is this UUID thing at DuckDuckGo


    It can be a FS created, a shared folder, a pool, etc...

  • seems my question wasn't specific enough.

    I know what a UUID ist - all that stuff with unique identifier and small probability regarding a duplicate.

    I also know that UUIDs are used all over the place. I just wasn't aware of the fact, that they are also used somewhere in partitions / file systems, why they are used there. Which consequences it has, etc.


    Will do some searching.

    Qnap TS-853A

    Qnap TS-451+

    Qnap TS-259 Pro+

  • Debian/omv gives each filesystem a uuid and the mounts are based on uuid because the device identifiers are dynamic (ie. a drive identified as /dev/sda on this boot might be /dev/sdb on the next one depending on the order the os registers them). Duplicate uuid, can therefore “confuse” the os as soma mentioned.

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!