Posts by snoopy913

snoopy913 · Nov 6th 2022

I basicly got exact what I mentioned above, the Adaptec 7 Series Card worked out oft the box and has a HBA mode, so I just use it as an HBA.

I also switched to Snapraid and MergerFS as my array is only media.

I would also love the idea of a HW List, I just searched for Debian compatibility.

As a case I got the Node 804, which is now full.

Later I will switch to m.2 SSDs and larger 2.5" drives beside the media array.

snoopy913 · Oct 22nd 2022

I hope to stay with 8 drives at least for the next 3-4 years.

The PSU should be fine, as the toshiba HDDs use also the 12V line.

One issue is that I want to use the HW transcoding feature of Plex, which is best working with intel Quick Sync and that means I need a intel core CPU... I didn't find any fitting server grade m/board and therefore also ECC Support is hardly available, thats why I would stay with consumer tech.

I had a look on the HBAs, but didn't find any good deal, other then the Adaptec 71605 which is available secondhand for 80€ in Germany.

I know.it is not generally recommendex, but they are mentioned to have debian support and did not yet find any hints to issues.

I will of course update if it is working.

snoopy913 · Oct 21st 2022

thanks, that goesin the same direction as I already thought.

I am currently thinking about this setup:

Motherboard: ASRock B660M Phantom Gaming 4
Or: MSI MAG B660M MORTAR, would habe DDR5, 6x SATA and 2,5GBaseT
CPU: Intel Core i5-12400
RAM: 2x16GB
Expansion Card: Adaptec RAID 71605

I would use the Card without RAID and stay with the SW RAID, if there is no significant benefit.

Is there any improvement to the HW possible?

snoopy913 · Oct 21st 2022

Hi all,

I thougth I have a simple problem, that I just need more SATA connectors, but now I think I will need to build a new NAS...

Current Setup:

Case: Node 804
- basically enough HDD space and cooling currently.
CPU / Motherboard: ASRock J5040-ITX
- Great CPU with enough power, mostly utilized at 20-30%
- Motherboard is more the issue, as it has to less SATA ports and only PCIe 2.0 x1
RAM 16GB
- is mostly utilized at 30-50%, so it's fine for now
PSU: 400W be quiet System Power 9 CM
- until now it managed all drives, but I really don't know how much it is utilised

Expansion Card: InLine 76617F 8 Port PCIe 2.0 x1
- Currently with 4 HDDs and it takes a lot of time to "initialise" the RAID on startup or when the HDDs are woken up from standby
- When in normal use, the decreased throughput is currently not an issue
- The Card has 2 Marvell 88SE9215 but only 1 (4 SATAs) is recognised when checking lspci
2,5" SSD 256GB with OMV and some config paths for the docker container
- far from to less space
HDDs: 6x14TB Toshiba MG07/MG08 SATA in RAID5 (is getting to small) and a 4TB WD with Dokuments
- upgrade will come with 2 additional 14TB drives

Currently running Services:

OMV with SMB shares
Docker
- Plex Media Server with mainly 4K content, currently ~50TB
- vaultwarden, syncthing, nextcloud, homeassistant (but not very used very much), heimdall, glances
- 4-5 other selfhosting solutions, but not very CPU/RAM intensive
- If I upgrade, I would plan with a lot of buffer

My possibilities to get more storage:

Change to larger HDDs: with 18TB I would get <24TB more space, but the bottleneck of the PCIe Card remains, also my HDDs are still fine
SATA Port Multiplier: one of the worst solutions I could use
USB SATA Hub: maybe not as bad as the Port multiplier. No idea how this would work with the RAID
New Mainboard, CPU, RAM
- As I use the Plex Server, I would go for the Intel 12th / 13th Gen with iGPU, they would have AV1 transcoding support.
- And the Mainboard should have as many SATA Ports as possible
- Mainboard Requirements: Sockel 1700, mATX, best 8 SATA, 2+ PCIe extensions
- I found this one: Biostar B660MXC PRO but nowhere to buy it
- Or I go for supermicro, but there I have to go for other CPUs i think
I could also separate the storage and self-hosting stuff
- there are supermicro boards with Atom CPUs and many SATAs
- This would maybe double the energy I need, and I need everything twice

Does anyone have other ideas how I could increase my SATA ports? Or have a create upgrade recommendation, or any idea idea would be very helpful.

Thanks in advance.

snoopy913 · Oct 17th 2022

Thank you very much, everything worked now

snoopy913 · Oct 17th 2022

great it worked, I did it without the spare disk, as I want to add it to RAID later, together with to others, so I have to resize it only once.

Code

root@omv-j5040:/srv# mdadm --assemble --force --verbose /dev/md0 /dev/sd[bdegh]
mdadm: looking for devices for /dev/md0
mdadm: /dev/sdb is identified as a member of /dev/md0, slot 2.
mdadm: /dev/sdd is identified as a member of /dev/md0, slot 0.
mdadm: /dev/sde is identified as a member of /dev/md0, slot 1.
mdadm: /dev/sdg is identified as a member of /dev/md0, slot 4.
mdadm: /dev/sdh is identified as a member of /dev/md0, slot 3.
mdadm: forcing event count in /dev/sdd(0) from 60673 up to 60679
mdadm: forcing event count in /dev/sde(1) from 60673 up to 60679
mdadm: forcing event count in /dev/sdb(2) from 60673 up to 60679
mdadm: added /dev/sde to /dev/md0 as 1
mdadm: added /dev/sdb to /dev/md0 as 2
mdadm: added /dev/sdh to /dev/md0 as 3
mdadm: added /dev/sdg to /dev/md0 as 4
mdadm: added /dev/sdd to /dev/md0 as 0
mdadm: /dev/md0 has been started with 5 drives.

Display More

I cannot select it in the GUI to mount it, is there a recommended way to do it?

snoopy913 · Oct 17th 2022

This is the output:

Code

root@omv-j5040:/home/Simon# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : inactive sde[1](S) sdg[3](S) sdf[5](S) sdb[2](S) sdh[4](S) sdd[0](S)
      82033508352 blocks super 1.2

unused devices: <none>

currently, a sixth disk is plugged in which I tried to add to the RAID but with an error message, not is added as a spare:

Code

root@omv-j5040:/home/Simon# mdadm --examine /dev/sdf
/dev/sdf:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x1
     Array UUID : 180db90e:5dc436bc:d1266620:e9385686
           Name : omv-j5040:RAID5  (local to host omv-j5040)
  Creation Time : Wed Sep  8 14:51:27 2021
     Raid Level : raid5
   Raid Devices : 5

 Avail Dev Size : 27344502784 (13038.88 GiB 14000.39 GB)
     Array Size : 54689005568 (52155.50 GiB 56001.54 GB)
    Data Offset : 262144 sectors
   Super Offset : 8 sectors
   Unused Space : before=262064 sectors, after=0 sectors
          State : clean
    Device UUID : bf43fdd7:f836f512:70066978:9881a123

Internal Bitmap : 8 sectors from superblock
    Update Time : Mon Oct 17 14:17:11 2022
  Bad Block Log : 512 entries available at offset 64 sectors
       Checksum : 3cbaebf4 - correct
         Events : 60679

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : spare
   Array State : ...AA ('A' == active, '.' == missing, 'R' == replacing)

Display More

This is the output when I stop and assemble it again:

Code

root@omv-j5040:/home/Simon# mdadm --stop /dev/md0
mdadm: stopped /dev/md0
root@omv-j5040:/home/Simon# mdadm --assemble --scan -v
mdadm: looking for devices for /dev/md0
mdadm: /dev/sdd is busy - skipping
mdadm: /dev/sdh is busy - skipping
mdadm: /dev/sdg is busy - skipping
mdadm: /dev/sdf is busy - skipping
mdadm: /dev/sde is busy - skipping
mdadm: No super block found on /dev/sdc1 (Expected magic a92b4efc, got 000005d3)
mdadm: no RAID superblock on /dev/sdc1
mdadm: No super block found on /dev/sdc (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdc
mdadm: No super block found on /dev/sda3 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sda3
mdadm: No super block found on /dev/sda2 (Expected magic a92b4efc, got 0000041d)
mdadm: no RAID superblock on /dev/sda2
mdadm: No super block found on /dev/sda1 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sda1
mdadm: No super block found on /dev/sda (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sda
mdadm: /dev/sdb is busy - skipping

Display More

I suspect I have to force it, or can I see why these disks are busy?

snoopy913 · Oct 17th 2022

Yes, I remember, the output was confirming it is inactive and also marked them as spare I think, but I couldn't activate it earlier.

I am currently in a walk with my dog, will post the exact output later. I will also try to explicitly stop it and then start again.

snoopy913 · Oct 17th 2022

Hi together,

I have been using a RAID5 with 5 disks without issues for over a year now, but today a moved everything to a new case to get more space and increase the RAID to maybe 8 disks later.

After I moved everything I had several issues with disks not beeing recognised, but this is an issue with the SATA expansion card mith 2 Marvell 88SE9215 where only 1 is usable.

Now all my disks from the RAID are recognised and in good condition in the GUI but the SW RAID is not shown on obviously then the file system on the RAID is marked as missing.

Then I have checked mdadm in the terminal, the output shows that it is defined as a raid0 instead of raid5, but the raid devices are the correct ones

Code

root@omv-j5040:/proc# mdadm --detail /dev/md0
/dev/md0:
Version : 1.2
Raid Level : raid0
Total Devices : 5
Persistence : Superblock is persistent


State : inactive
Working Devices : 5


Name : omv-j5040:RAID5  (local to host omv-j5040)
UUID : 180db90e:5dc436bc:d1266620:e9385686
Events : 60673


Number   Major   Minor   RaidDevice


-       8       64        -        /dev/sde
-       8       80        -        /dev/sdf
-       8       48        -        /dev/sdd
-       8       16        -        /dev/sdb
-       8       96        -        /dev/sdg

Display More

When using mdadm --examine on each disk, I see two different pictures, two disks have newer update time with 3 disks missing, and 3 disks have older update time, but when all disks where still active.

Older superblock of a disk

Code

root@omv-j5040:/proc# mdadm --examine /dev/sde
/dev/sde:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x1
     Array UUID : 180db90e:5dc436bc:d1266620:e9385686
           Name : omv-j5040:RAID5  (local to host omv-j5040)
  Creation Time : Wed Sep  8 14:51:27 2021
     Raid Level : raid5
   Raid Devices : 5

 Avail Dev Size : 27344502784 (13038.88 GiB 14000.39 GB)
     Array Size : 54689005568 (52155.50 GiB 56001.54 GB)
    Data Offset : 262144 sectors
   Super Offset : 8 sectors
   Unused Space : before=262064 sectors, after=0 sectors
          State : clean
    Device UUID : 7811787d:544074f5:c515d20c:847a67ef

Internal Bitmap : 8 sectors from superblock
    Update Time : Mon Oct 17 14:11:45 2022
  Bad Block Log : 512 entries available at offset 64 sectors
       Checksum : 24e3c9fe - correct
         Events : 60673

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 1
   Array State : AAAAA ('A' == active, '.' == missing, 'R' == replacing)

Display More

Newer superblock of a disk

Code

root@omv-j5040:/proc# mdadm --examine /dev/sdf
/dev/sdf:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x1
     Array UUID : 180db90e:5dc436bc:d1266620:e9385686
           Name : omv-j5040:RAID5  (local to host omv-j5040)
  Creation Time : Wed Sep  8 14:51:27 2021
     Raid Level : raid5
   Raid Devices : 5

 Avail Dev Size : 27344502784 (13038.88 GiB 14000.39 GB)
     Array Size : 54689005568 (52155.50 GiB 56001.54 GB)
    Data Offset : 262144 sectors
   Super Offset : 8 sectors
   Unused Space : before=262064 sectors, after=0 sectors
          State : clean
    Device UUID : caa0a466:43d02013:6f374468:be1c50be

Internal Bitmap : 8 sectors from superblock
    Update Time : Mon Oct 17 14:17:11 2022
  Bad Block Log : 512 entries available at offset 64 sectors
       Checksum : 5617ae6e - correct
         Events : 60679

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 3
   Array State : ...AA ('A' == active, '.' == missing, 'R' == replacing)

Display More

mdadm.conf:

Code

root@omv-j5040:/proc# cat /etc/mdadm/mdadm.conf
# This file is auto-generated by openmediavault (https://www.openmediavault.org)
# WARNING: Do not edit this file, your changes will get lost.

# mdadm.conf
#
# Please refer to mdadm.conf(5) for information about this file.
#

# by default, scan all partitions (/proc/partitions) for MD superblocks.
# alternatively, specify devices to scan, using wildcards if desired.
# Note, if no DEVICE line is present, then "DEVICE partitions" is assumed.
# To avoid the auto-assembly of RAID devices a pattern that CAN'T match is
# used if no RAID devices are configured.
DEVICE partitions

# auto-create devices with Debian standard permissions
CREATE owner=root group=disk mode=0660 auto=yes

# automatically tag new arrays as belonging to the local system
HOMEHOST <system>
# instruct the monitoring daemon where to send mail alerts
MAILADDR notify@muerwald.de
MAILFROM root

# definitions of existing MD arrays
ARRAY /dev/md0 metadata=1.2 name=omv-j5040:RAID5 UUID=180db90e:5dc436bc:d1266620:e9385686

Display More

I tried mdadm --assemble /dev/md0, it takes some time without output, and the details are still the same.

I very much believe there is an easy fix for that, as it looks its just that the informations on the disks are not consistent. But I am carefull, as I don't want anything to be lost (I don't have an backup, because it is nearly 50TB, but the files are also not "unique" but it would take a lot of time to get them back)

Does anyone know what I could try to fix the RAID5?

Thank you very much in advance.

Posts by snoopy913

Upgrade recommendation for more SATA ports

Upgrade recommendation for more SATA ports

Upgrade recommendation for more SATA ports

Upgrade recommendation for more SATA ports

[SOLVED] RAID5 is not recognised anymore after reboot

[SOLVED] RAID5 is not recognised anymore after reboot

[SOLVED] RAID5 is not recognised anymore after reboot

[SOLVED] RAID5 is not recognised anymore after reboot

[SOLVED] RAID5 is not recognised anymore after reboot