RAID5 Array Built in OMV 7.4.17, on Beelink ME Mini, Corrupted Very Quickly after I Built It

I also ordered a Beelink ME mini, it is scheduled to arrive in about 1.5 weeks. I intend to run raid-5 on it with omv with 4x 2 tb Lexar nm790 drives. Did you get in touch with beelink support?

Seems a bug in kernel of truenas for the ASM2824 controller.

Good to know I might run in to problems, once mine has arrived I will definately test it before filling it with actual data.

This is worrying, as I have just purchased and received my ME Mini to replace my CM3588 NAS that was using OMV 7 with next cloud server. This setup killed my 4x WD-SN770 2Tb NVME drives with all 4 devices being put in to read only mode (configured for Raid5). Got them replaced via SanDisk warranty so looked for a different device and settled on the ME mini. I hope this is not a recurring theme with a NVME NAS system?

Mine will arrive tomorrow, I intent do run a newer kernel on it than the debian original one. I have tried this in my hyper-V test OMV and that seemed to work ok.

if that works fine with the ME and my raid set stays ok I will make a new topic on what I did and how.

I am going to try with 5 nvme's in it, it should arrive tomorrow, and hopefully this weekend there's time to tinker with it. The NM790 nvme drives use way less power:

Lexar NM790 2 TB Review

The Lexar NM790 2 TB offers fantastic performance at outstanding pricing. The 2 TB model sells for just $110, which makes it the most affordable high-end SSD…

www.techpowerup.com

That's mainly why I choose them. But you are right, the power unit feels a bit on the light side. I also indend to remove the wireless board to save some power as I won't use that anyways.

This thread would suggest the unit has power problems: https://forums.truenas.com/t/u…in-truenas-scale/47306/69

Quote from ryecoaaron

The Lexar sticks are dram-less. This means their performance will be even worse in raid 5 since they have to use system ram for cache. Will be interesting to see your power consumption and performance numbers.

I'll see what happens when it is here. Mind you, all nvme's have 1 pci lane except for slot 6 which has 2, so the nvme's will never run on max performance anyways, but this setup will hopefully use less power and has more performance than my old atom based qnap.

Thanks for the suggestion, I'll have a look at mergerfs

Ok, it seems it is like I expected, there's a bug in the default kernel that is shipped with debian (6.12.38).

See this: https://lkml.iu.edu/hypermail/linux/kernel/2201.0/00333.html

What I did is following:

sudo curl -fsSL https://pkgs.zabbly.com/key.asc -o /etc/apt/keyrings/zabbly.asc

vi /etc/apt/sources.list.d/zabbly-kernel-stable.sources

Put this in the zabbly-kernel-stable.sources

Enabled: yes

Types: deb

URIs: https://pkgs.zabbly.com/kernel/stable

Suites: bookworm

Components: main

Architectures: amd64

Signed-By: /etc/apt/keyrings/zabbly.asc

apt update

apt install linux-zabbly

inspiration came from this website:https://fostips.com/install-li…15-in-debian-12-bookworm/

Be sure to have the kernel plug-in installed so you can check if the zabbly kernel gets selected as bootable kernel. My Beelink ME Mini has been rock solid for more than 48 hrs, and I have written 2.5 Tb to my Raid-5 set (In the end I opted for raid-5 as I am familiar with md raids)

My Beelink ME mini runs with kernel 6.15-10-zabbly (just updated to 6.15.10, I started off with 6.15.09, and updates come as regular updates.

Temps of the nvme's eventhough it was 31 C in the room not above 56 C when read/writes were done, overall I am happy with the little device and it is soo much more responsive compared to my qnap.

Quote from rdjmsy53

1. No matter how many times I tried, I could never get 3, 4 or 5 of those, brand new, Crucial 4TB NVMe SSDs to live in a RAIDz1 array

This is a very common problem. I don't have a real solution, but I would point out:

1. The vast majority of complaints I see are about failures with Crucial NVME drives.

2. The voltage for the 3.3V rail in BIOS is excessively high, like 4.2V.

3. People claim that while they see a problem with TrueNAS Scale, they don't with TrueNAS core. I think the latter uses BSD not Linux.

4. The BeeLink Mini ME only has 12 GB of RAM which is insufficient to run ZFS, which would include RAIDz1. btrFs might be a better choice, which is easier on the RAM.

Quote from chente

I have no idea about the rest, but this statement is incorrect. ZFS can run on 2GB RAM systems without problems, just like the other file systems.

It will run on 2GB, but not well. A rule of thumb is 1GB RAM for every 1TB of disk. So if you don't have that much NVME storage, then it would indeed be possible to use ZFS on the Mini ME. There is also some ZFS tweaks to make it use less RAM, but those are above my pay grade.

Quote from n4mwd

It will run on 2GB, but not well. A rule of thumb is 1GB RAM for every 1TB of disk. So if you don't have that much NVME storage, then it would indeed be possible to use ZFS on the Mini ME. There is also some ZFS tweaks to make it use less RAM, but those are above my pay grade.

Broad statements about how ZFS uses RAM which repeat common misconceptions are bound to draw criticism. This thread is worth reading by anyone who wants to improve their understanding of the role of the ZFS ARC: https://discourse.practicalzfs…zfs-linux-in-general/2420

Quote from n4mwd

This is a very common problem. I don't have a real solution, but I would point out:

1. The vast majority of complaints I see are about failures with Crucial NVME drives.

2. The voltage for the 3.3V rail in BIOS is excessively high, like 4.2V.

3. People claim that while they see a problem with TrueNAS Scale, they don't with TrueNAS core. I think the latter uses BSD not Linux.

4. The BeeLink Mini ME only has 12 GB of RAM which is insufficient to run ZFS, which would include RAIDz1. btrFs might be a better choice, which is easier on the RAM.

Display More

I wouldn't be too quick to point the figure at Crucial nvmes. Beelink themselves are selling these units with pre-installed Crucial 2TB p3 plus. The number of complaints about Crucial nvmes may simply reflect their popularity, not any inherent problem. What seems most important is that the sum of power draw of the nmves remains within the units overall power envelope.

There's the additional question of whether some linux kernels - stable debian - are handling power control on the ASM chip correctly or not. wiz101 elected to use a zabbly kernel and has a stable nmve RAID ( MD RAID?). We'll never know if, as chente suggested, the installation of a 6.14 proxmox kernel would have led to a stable Beelink system.

This thread has other tips for those who cannot use other kernels re: bios version, kernel boot params and various bios settings: https://forums.truenas.com/t/u…in-truenas-scale/47306/83

Krisbee

Correct, mine runs MD Raid.

What I read on the ASM chip was that it had something to do with the link speed switching between two values so the datalink layer would not get active on some of the ports. This has been addressed in newer kernels. I did not test it with the proxmox kernels, I could not find any details if this was fixed and with what kernel version so my guess was to take the highest kernel version I could easily get working correctly with OMV and then test if it would stay stable.

Off course you have to make sure the total power usage has to stay within the 45 Watts. I had also removed the wireless board in order to save power (plus unnecessary wifi+bluetooth on a nas device imho). Compared with my qnap nas the load on my ups has dropped with 19.5 W and that's without the disks spinning, so overall I am happy with the result.