Setting up a 'bricked' OMV 4 installation as OMV 6 (looking for tips)

I am running a OMV 4 installation on a HP Microserver for years and it works pretty well for my needs (few users).

Here the situation:

• HP Mircoserver with 4 Disk Slots
  • 1x 250 GB for system
  • 2x 3TB in Raid1
  • 1x 3TB for internal backup (e.g. 2 times a week)
  • 2x external USB3 disks for offsite backup (each Friday with 1 disk, so each disk gets a backup every 2 weeks)

My problem: Some time ago an update killed my GUI and even with the help of the forum I didn't manage to get it to work again. And from there on things went downwards....

I can still access the shared folders and everything, so the NAS is in use as a file server, but since then I had do do backups via "FreeFileSync" through my computer to the external hard disks (which is slower) and as I am not very good at Linux the backup from Raid1 to the internal backup disk never happened again. Yes, I know, I could probably trigger the backup via command line somehow, but - you know...

OK, so currently I am making my backups from everything to different external disks to make sure I have everything, before I "kill" the current installation.

However, ideally I would also be able to copy everything from the raid the the internal backup disk, but I am not sure how I can do this safely. I assume I will have to figure out to rsync this somehow.

To make things worse: I could in theory access the shares on this backup disk, but somehow I can't find the correct password and Windows won't let me access these shares without the password. Tried all my passwords that I usually use, to no avail. Is there any way to find this password via command line? Or change it (but wouldn't want to risk this, if necessary, because currently I can access the other shares on the raid and currently still need to work with this). But once I "kill" the installation, I could also try changing the password and try to make a copy, if anything else doesn't work. But I think it is probably easier to syncronize the raid to the internal backup disk directly. This should also help me to quickly setup the system again. Any tips how I can get the raid to the internal backup disk easily? (I am not very firm in linux commands, but I can follow instructions... ).

Now to the new system:

I would like to get some tips from the experts on this, so that I setup my new system well. Here a few thoughts that I have:

• I have a couple of notebook HDs lying around and could think of using one of these for the system (not sure how I will put it in the server yet, but I have read that some people such a 5th disk in the HP Microserver for the system and the 4 normal slots for data). Not that I will need much more space in the near feature. I am getting towards the end of my 3TB disks (am at around 2.5 TB), but I have projects on there that are over 10 years old and I can start to delete those now I think. So I am planning with the old 3TB disks for now, but might buy bigger for any that needs replacement.
• Raid1: at the time i thought it would be a good idea to have the availability of the two disks in Raid1 and then have the internal backup disk for local backup (plus the external usb drives for offsite backup). But to be honest, I am not very sure if the Raid1 is such a good idea. First of all, I was not able to access the data on the disks when they are not in Raid. I guess this is normal, but it makes me a little unconfortable. What if the server fails and I can't setup a new raid. Or what if I can't get a new disk quickly? I think I'd rather have the two disks separate and a syncronization via rsync or something every hour. Because current project are also always on the local computer, so even if the main disk fails and the data hasn't been synced to the second disk yet, it is no big deal, the data should still be on the local computer until the end of the project. And by the end of the project the files should also already be on various other disks (internal backup disk and external backup disks).
  • So what are your recommendations for the OMV6 setup: Raid1 or not? I am currently leaning towards no raid because I feel more comfortable getting data back from the disk without raid.
  • So was thinking the following setup:
    • Disk 1: System (might swap that to a notebook HD that I put into the HP Microserver, but not sure if I want to go down this route)
    • Disk 2: Main data disk where users save and access projects
    • Disk 3: "Mirrored" data disk (I don't know what a good frequency would be, but as we might need to go down the Windows 365 route soon, the main collaboration will be via cloud anyway, and the NAS will be actually just a local backup of what happens in the cloud. So I am currently thinking about something like hourly or daily might actually be enough for this "mirrored" disk.
    • Disk 4: This would be like my current internal backup disk and depending on the sync frequency of Disk 2 to Disk 3 I could imagine daily or twice weekly.
    • (Disk 5: should I go down the route and install OMV on an additional disk, I would have one of the main slots free and could add another 3.5 disk to the server)

So overall, I would like to hear your opinion and how you would set this up and why. I understand that use cases can be vastly different and your approach might not be the right one for me, but I would like to hear some opinions and understand the reasoning behind it, so I can see if I could have a better setup. Mine worked well to the point when the GUI went away and I wasn't able to get it back. But even then, the system did still what it was supposed to be, but I couldn't make any changes anymore, etc. which was a pain and I want to be able to administer the server again.

Ok, I was doing some research on how to access my shares via CLI and if I can somehow find out how to sync the files from the raid to the internal backup disk.

Found this post here (old but seems to be valid): Noob Q: access my shared files via shell/ssh?

So I went

cd /media/
ls

and got two shares plus cdrom (doesn't exist in my server) and usb and usb0.

bbaa055f-7fd0-49bf-b63a-6d75bc3665f7 cba71ddf-13bb-4a45-9af4-77d17f6f36bc cdrom usb usb0

Now I went into the shares and bbaa055f-7fd0-49bf-b63a-6d75bc3665f7 seems to be the share with the data and cba71ddf-13bb-4a45-9af4-77d17f6f36bc seems to be the system disk? Because it only includes this:

php5-pam_1.0.3-2_armhf.deb php5-pam_1.0.3-2_armhf.deb.1

But I should actually have the raid1 and the internal backup disk. Or does the raid1 not show up in /media/?

OK, with df -h I actually found that I have this:

/dev/sda1 is the disk with the system.

/dev/sdd1 is most probably the internal backup disk. You can see that I am about 900 GB behind in internal backups compared to the raid1

/dev/md0 must be the raid system.

So the question is how do I get my files stored in the folders pdb_admin and pdb_research from md0 to sdd1? Will need to do some research.

@KM0201

Thank you for your feedback. So for Raid 1 you'd go the same direction that I thought. I think it makes things simpler and also easier to trouble shoot if something goes wrong. And as described, syncs a couple of times per day should be more than enough. Once we go for collaboration via cloud then it could be even less frequent I guess.

I was thinking about reusing the HDs because I rescued them from notebooks that had other issues and were discarded. It is not necessarily about costs, but these hard drives should just work fine and I don't like to throw them away.

Zoki: Thank you for your reply.

Not sure what generaltion is. *lol*

Mine is first or second generation I think. The old one.

Afaik mine does not have "hot swap bays". You can easily switch disks out, but they are not hot swappable.

As I don't really need a fourth data disk in the server, I might not go down that route at the end. I think this would require a little bit more time and when I kill the thing, it should be running again in a couple of days. Ideally I can do this over a rainy weekend.

But then, it would be nice to have a 4th data disk, or at least the option to put one in, in case we need it.

Yeah, that is a fairly new one. Mine is this one:

I think I have seen people running an additional HDD in a casae in the DVD-Slot. Need to do some research if there is an additional sata connector available.

I am not sure about running the server via the internal usb slot. I have read that the USB sticks die quickly. Might have changed with newer versions of OMV though, haven't looked into this for a long time.

Thanks! Might look into this and see what type of stick is required and if it is easy to do and setup.

I had the system running without much "care", especially as I couldn't access the backend anyway. I should have done this quite some time ago, but now I am motivated to really get this running again.

I like the idea of the USB-Stick and you probably need very little space for the system anyway.

I have a couple of USB-sticks lying around as well, but I am not sure if they meet the requirements.

Will do some research in the forum and see how this all works. Because it sounds easy to switch a USB-Stick, just open the front door and the USB port is there.

Hi Agricola, thank you for the reply. I will look into this USB approach.

I think I have a couple of sticks lying around. Some of them are company merchandise, but with 4GB I don't think it is worth giving them out to anyone. Might be too small for this though, as you mentioned 32G and 16G.

Will need to do some research. I am still trying to get the data from the raid1 copied to the internal backup disk, as this would be so much quicker later to set everything up, by not using an external disk, but rather an internal disk to copy data.

However, it seems that only the raid or only the backup disk show up in media (see screenshots above).

Maybe the internal backup disk isn't mounted? Or is the Raid 1 generally not shown in media?

Zitat

Not sure when the mount point changed. If you recently mounted it, have a look in /srv instead of /media

I haven't used the internal backup disk for probably well over a year, as I couldn't access it. There is a good to fair chance that it isn't even mounted.

Here you can see that there are only 2 disks mounted in /media/:

The first one with 134 G is the system disk.

The 1.4TB one is my internal backup disk

The 2.3T is the Raid1 with the main data. There is a difference of 900-1000GB between the two disks, because I can't do internal backups as the web interface doesn't work for quite some time now. In theory both should have the same data, more or less.

Before I 'kill' the system and set it up new, I wanted to syncronize the data between the raid1 and the internal backup disk. I have also backed it up on external USB disks, but I would feel more comfortable if I had the interal backup disk up to date as well. And copying from the internal disk should be so much quicker than from an external USB-disk via the network.

Yeah, that was what I was thinking. I am just not very good at Linux, hahahaha. I'll have a look at your links (thanks Macom and Zoki).

Now, any idea how to address the raid and how to address the internal backup disk? As in media only one of the two seems to shows up.

There should be 3 different media show up: the system disk, the raid1 and the internal backup disk. But I only have two:

The internal backup disk might not be mounted. Maybe I mounted it manually through the web interface. It has been so long now. I will need to do some research on how to mount it via console.

But then, with df -h it shows everything. Not firm enough in Linux to know if it is mounted then or not. Also, how will I know which of these strange codes belong to which "drive"?

On the aspect of installing on USB: I just watched this video, not sure if it is very good as he recommends to install the ISO on the USB-drive by burning a CD/DVD first (I don't even have a CD/DVD drive anymore, hahahahaha).

But he basically says that USB2 is too slow and from what I can see, my HP Microsever N36L only has USB2 ports, not sure if it is worth installing via USB if it is only USB2? Might be too slow for running the system from it?

ooooooh, I might have been stupid.

It is probably the system drive that doesn't show up in /media/, right?

I saw only 2 drives and as the system runs and the raid1 runs my conclusion was that the internal backup disk must be missing. That the system disk might not show in /media/ never occored to me.

Now one question: How do you know that the raid is /media/bbaa... and the internal backup disk is /media/cba7... ? Can you see that from the weird code? To me it says nothing at all.

Zitat

Please post text in code box, not screen shots.

Oh, sorry, will do from here on! Thank you for your help so far.

So here is what I get when checking what is on the /cba7... drive. Seems to be the internal backup disk. Yeahhh!!!! It is there.

root@pdb-matrix:/# ls -la /media/cba71ddf-13bb-4a45-9af4-77d17f6f36bc/
total 60
drwxr-xr-x   7 root root   4096 Jan 18  2018 .
drwxr-xr-x   6 root root   4096 Jan 19  2018 ..
-rw-------   1 root root   9216 Feb  8  2018 aquota.group
-rw-------   1 root root   7168 Feb  8  2018 aquota.user
drwx------   2 root root  16384 Jan 28  2017 lost+found
drwxrwxrwx  15 root root   4096 Jan 18  2018 omvbackup
drwxrwxr-x+ 21 root users  4096 Jan 15  2018 pdb_intbak_admin
drwxrwxr-x+ 10 root users  4096 Jul 12  2015 pdb_intbak_backup
drwxrwsr-x+  6 root users  4096 Jan 15  2018 pdb_intbak_research

And here is the raid1:

root@pdb-matrix:/# ls -la /media/bbaa055f-7fd0-49bf-b63a-6d75bc3665f7
total 52
drwxr-xr-x  6 root root   4096 Jan 15  2018 .
drwxr-xr-x  6 root root   4096 Jan 19  2018 ..
-rw-------  1 root root   6144 Feb  8  2018 aquota.group
-rw-------  1 root root   7168 Feb  8  2018 aquota.user
drwx------  2 root root  16384 Jul 12  2015 lost+found
drwxrwxr-x 26 root users  4096 May 13 10:26 pdb_admin
drwxrwxr-x 10 root users  4096 Jul 12  2015 pdb_backup
drwxrwsr-x  6 root users  4096 May 13 10:40 pdb_research

This is great. Now I need to rsync the folder pdb_admin on bbaa... to pdb_intbak_admin on cba7...

Will give this a try. Can't do much more than destroying it anyway, hahahaha. If I can't do it, then I will have to go via the external USB-Backup drive anyway.

Thank you so much for the help so far. Your advice gave me some hope that I can actually manage to do this!

Will have some look at this, but I will try something like this:

rsync -av --dry.run /media/bbaa055f-7fd0-49bf-b63a-6d75bc3665f7/pdb_admin/ /media/cba71ddf-13bb-4a45-9af4-77d17f6f36bc/pdb_intbak_admin/