Missing nvme filesystem

    Hello,


    Something strange is happening to a nvme ssd Crucial CT500P3PSSD8. Currently it is showing as missing device on the file system under storage section. It happened out of the blue one day. Odd part is, I can still access the data fine on the ssd through the /dev/

    Code
    savanna@omv:/$ ls -la /srv/ 
drwxr-xr-x  9   root root    4096 Jan  2 21:14 .
drwxr-xr-x  20  root root    4096 Jan 19 13:14 ..
drwxrwxrwx  3   root root    4096 Jan 27 19:21 dev-disk-by-uuid-9e1b0...27d2


    but I cannot see the nvme on the /dev/nvme0... meaning???


    and I can also see it on the pci

    Code
    savanna@omv:/$ ls -la /srv/
...
06:00.0 Non-Volatile memory controller [0108]: Micron Technology Inc 2550 NVMe SSD (DRAM-less) [1344:5416] (rev 01)
...


    Looking into some errors I see the below.


    Any guidance? Google shows that is a power saving problem?

    Not sure what to try? Any advise? - Thank you in advance.

  • crashtest

    Approved the thread.

  • savanna

    Added the Label OMV 7.x

  • macom

    Approved the thread.

    So my motherboard is a cwwk 305N and according to this post from servethehome, nvme port on the board gets unstable when ASPM settings are enabled in BIOS. Need to try and report back.


    *edit:

    There seems to be a current issue a lot of people are experiencing with nvme ssd. Here, in a post from proxmox and from a bug report it is being discussed ASPM on the nvme pci port could solve the issue.

    So, after looking into the BIOS of my board, I see the ASPM for the NVMe port is already disabled:

    Code
    BIOS > Chipset > PCH-IO COnfiguration > PCI Express Configuration > PCI Express Root Port 10 > Disabled


    So I had to disable the ASPM all together for al the PCIe ports (see attached) as it was set as 'Auto'.

    Code
    BIOS > Chipset > PCH-IO COnfiguration > PCI Express Configuration > DMI Link ASPM Control > Disabled

    After a reboot, the NVMe is now being detected (not always but now 50% of the time) and mounted properly


    is this only related to my motherboard? Perhaps is a kernel thing?

    I have tried to set a 'delay' before not recognizing Port10 but no success.


    I noticed that when NVMe is not recognized on OMV, it is also not recognized on BIOS.


    I appreciate a little guidance from the team.

    I have that same board and also the same Crucial nvme and have no errors on boot:

    I don't remember which config I've done on the BIOS and can only check it next week when I return home.


    Did you made any changes to the disk on power management GUI?

    Code
    USER@HOST:~ $ sudo dmesg | grep nvme
[    4.307395] BTRFS: device fsid 425c5c15-b46e-4893-bbbe-24b743c25f1f devid 1 transid 346380 /dev/nvme1n1 scanned by btrfs (265)
[    4.307532] BTRFS: device fsid 425c5c15-b46e-4893-bbbe-24b743c25f1f devid 2 transid 346380 /dev/nvme0n1 scanned by btrfs (265)
[    7.955102] BTRFS info (device nvme1n1): first mount of filesystem 425c5c15-b46e-4893-bbbe-24b743c25f1f
[    7.955116] BTRFS info (device nvme1n1): using crc32c (crc32c-intel) checksum algorithm
[    7.955125] BTRFS info (device nvme1n1): enabling ssd optimizations
[    7.955128] BTRFS info (device nvme1n1): disk space caching is enabled
[46866.381921] BTRFS info (device nvme1n1): device stats zeroed by btrfs (134757)


    Thank you for your replay,


    Checking, Power management on nvme is Disabled.


    As a little bit of update, after disabling ASMP all together form BIOS is kidda stable. I have restarted at least 5 times this week and nvme has been recognized every time.


    Looking forward to see your changes on BIOS.

    Did you update BIOS at any point? - I haven been following this board in the last 6 months.

    Quote from savanna

    Did you update BIOS at any point?

    Not yet.


    I saw what seems an updated BIOS for the n305 board but the site is in Chinese and the Web Translator doesn't work straigth so I'm holding on try anything on it, until I know more:

    AlderLake-N四网NAS_N100-I3-N305_2024-11-12更新 | 畅网官方网盘




    Will give more info later.

    Quote from savanna

    Looking forward to see your changes on BIOS.

    Did you update BIOS at any point? - I haven been following this board in the last 6 months.

    So, finally had the time to mess with the board and things went (somewhat) sideways.


    Before flashing the BIOS, checked the settings on the DMI Link ASPM Control and it was showing Auto


    I flashed the new BIOS ( the one mentioned on post #6) and all went OK and BIOS was updated from "Project Version: CW-ALDN-NAS-V10" to "Project Version: CW-ALDN-NAS B01"


    Rechecked the setting and no changes happened.


    Then all hell broke loose:

    Server went on a bootloop to the BIOS and it didn't boot the USB stick with the OS even if it did recognize it on the BIOS boot order.

    Forcing it to boot the USB stick would just go back to bios.


    Thought that maybe I f*k up by using the BIOS file that wasn't meant to this board so, flashed back the previous BIOS file with the old version.

    SSDD (same sh*t, different day) and still no OS boot with the USB stick (with my OMV/Debian OS)


    Remembered to try with a Debian ISO on a fresh USB stick and it booted imediately.

    So, the bootloader from the USB stick (or something on the board) was fracked.


    So, flashed again the new BIOS file, loaded the defaults on the new BIOS and tried again the Debian ISO (choose Live amd64)

    After it booted to the Live system, I plugged my OMV USB stick (on another port) and just fixed the bootloader by following the instructions shown here:

    GrubEFIReinstall - Debian Wiki


    Just had to use the variables that aplied to my system (check the correct sdX that was my USB stick) and presto, system was booting again with no issues.


    Regarding the BIOS itself, there's a huge amount of settings that I never cared to dwelve into.

    I've just set the "default values" and use it like that.

    With the previous version, it worked without any issues so, I just left it like that.


    Now, will have to see how it goes with this version.

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!

Register Yourself Login