Information about ZFS cache and hitrate

  • Hi there,


    Over the last couple days I have been slowly transitioning from btrfs to zfs using the plugin provided by omv-extras. This all works great after some kinks got ironed out on my end. Having had some experience with TrueNAS Scale, I have some pretty basic knowledge about zfs and its features. So my question is: is there a way to know how big my zfs cache is in memory? If so, can it be placed on the dashboard? And lastly: can I see the cache hitrate and put that on the dash?


    Best regards,

    Emiel

  • is there a way to know how big my zfs cache is in memory?

    Are you referring to arc cache?

    If so, can it be placed on the dashboard? And lastly: can I see the cache hitrate and put that on the dash?

    I guess I could create a dashboard widget to show this if you are referring to arc.


    # grep -E "^size|^hits" /proc/spl/kstat/zfs/arcstats

    hits 4 18005878

    size 4 65875680736

    omv 6.0.41-1 Shaitan | 64 bit | 5.19 proxmox kernel | plugins :: omvextrasorg 6.1.1 | kvm 6.1.23 | mergerfs 6.3.3 | zfs 6.0.11
    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.
    Please don't PM for support... Too many PMs!

  • Yeah I was referring to ARC cache. A dashboard widget would be nice, especially since it's relatively important statistic for ZFS.


    Also, I might be wrong here, but isn't hitrate a percentage instead of a count? In TrueNAS it's implemented as a percentage.

  • especially since it's relatively important statistic for ZFS.

    Never looked at it and yet I use zfs : )


    Yeah I was referring to ARC cache. A dashboard widget would be nice,

    I will see what I can do. Shouldn't be too hard.

    omv 6.0.41-1 Shaitan | 64 bit | 5.19 proxmox kernel | plugins :: omvextrasorg 6.1.1 | kvm 6.1.23 | mergerfs 6.3.3 | zfs 6.0.11
    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.
    Please don't PM for support... Too many PMs!

  • I looked it up on my own install and I misspoke. It's called a hit ratio, not a hit rate. My bad. A hit ration is much more indicative of whether the ARC cache is adequately sized.

    This is the only ratio I see:


    # grep -i ratio /proc/spl/kstat/zfs/arcstats

    l2_data_to_meta_ratio 4 0

    omv 6.0.41-1 Shaitan | 64 bit | 5.19 proxmox kernel | plugins :: omvextrasorg 6.1.1 | kvm 6.1.23 | mergerfs 6.3.3 | zfs 6.0.11
    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.
    Please don't PM for support... Too many PMs!

  • Judging by the naming this is a hit ratio of L2 cache.


    This is how Truenas (freenas) reports on hit ratio:


    Note: this is on my backup NAS which mirrors my OMV nas.

    I don't really need the graphs or anything, but just a current hit ratio of ARC cache would make a good addition to the plugin.

  • just a current hit ratio of ARC cache would make a good addition to the plugin.

    How is it calculated?

    omv 6.0.41-1 Shaitan | 64 bit | 5.19 proxmox kernel | plugins :: omvextrasorg 6.1.1 | kvm 6.1.23 | mergerfs 6.3.3 | zfs 6.0.11
    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.
    Please don't PM for support... Too many PMs!

  • A hit ratio, as I know it, is a ratio of how many data requests are fetched from cache instead of from the pool. So if 3/4 of the requested data can be retrieved from cache, the ratio is 75%.


    I came across a python script that basically gets you all data you will ever need from a zfs pool (github). This will also calculate a hitpercentage the same way truenas does it. The script adds the cache hits and misses, and then calculates a percentage of hits. This is the hit ratio.


    These are the relevant lines:

    By the way: I'd like to reiterate that I don't want or need a whole summary as what this script can provide, but a hit ratio would be nice :)

  • Ah, that is simple enough.

    omv 6.0.41-1 Shaitan | 64 bit | 5.19 proxmox kernel | plugins :: omvextrasorg 6.1.1 | kvm 6.1.23 | mergerfs 6.3.3 | zfs 6.0.11
    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.
    Please don't PM for support... Too many PMs!

  • Well, I have had some problems with instability which I am quite sure are not related to the hard drives. Think of stuff like my array being reported as 'missing' while btrfs itself reports no such problems and the drives being mounted just fine. Also, I like the stability and focus of ZFS more. A bunch of stuff related to raid not being fully stable yet just kind of rub me the wrong way.

  • Case in point: my array is reported as missing this very moment, though nothing else besides the OMV dashboard is reporting any problems. Yesterday the array was shown as being a-okay, no errors, no s.m.a.r.t errors, btrfs scrub showed no errors. I reboot and bam, array is 'missing' while it's functioning just fine and btrfs fi show [mountpoint] shows both drives mounted and all docker containers that store their data on that array function just fine.

  • While no percentage, I think this graphically shows the ratio.


    omv 6.0.41-1 Shaitan | 64 bit | 5.19 proxmox kernel | plugins :: omvextrasorg 6.1.1 | kvm 6.1.23 | mergerfs 6.3.3 | zfs 6.0.11
    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.
    Please don't PM for support... Too many PMs!

  • Yeah that's already a pretty good representation of the data. Couldn't you maybe make it so both misses and hits are in the same arc? That way it's more 'like' a percentage. Right now one of the two arcs will always be full and that's not that useful.


    By the way, as a first year CS student I do feel I'm missing something here: why not just computing a simple percentage? Is there a particular reason why one would not do that in this case? Asking so I can learn from this :)

  • Couldn't you maybe make it so both misses and hits are in the same arc?

    They are in the same arc, no? Look at the load widget. This is supposed to do the same thing.

    Right now one of the two arcs will always be full and that's not that useful.

    Hits will always be full (unless misses are more than hits) and misses will always be a portion of that max. Not sure what I am missing here.

    why not just computing a simple percentage?

    The code does - https://github.com/OpenMediaVa…/engined/rpc/zfs.inc#L100

    Is there a particular reason why one would not do that in this case?

    Just showing a percentage is not very dashboard-ish. I thought the chart looked better but this is my first addition for the zfs dashboard widgets. I can add another widget for those who have to see a precise number.

    omv 6.0.41-1 Shaitan | 64 bit | 5.19 proxmox kernel | plugins :: omvextrasorg 6.1.1 | kvm 6.1.23 | mergerfs 6.3.3 | zfs 6.0.11
    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.
    Please don't PM for support... Too many PMs!

  • They are in the same arc, no? Look at the load widget. This is supposed to do the same thing.

    Well not quite. You have the bigger arc representing the hits, and then a smaller arc representing the misses. My suggestion would be that it would maybe be a bit more intuitive to have both hits and misses in the same arc, much like how ram usage is displayed. This way when for example 3/4 are hits, then you have an arc that's (for example) 3/4 green while the last quarter of the arc is red. In my humble opinion that would be somewhat more easy to read than a full arc and another half full arc.


    By the way, I would very much like to contribute to the project. Can we maybe move it to DMs? I would like to know how you have your dev env set up :)

  • My suggestion would be that it would maybe be a bit more intuitive to have both hits and misses in the same arc, much like how ram usage is displayed. This way when for example 3/4 are hits, then you have an arc that's (for example) 3/4 green while the last quarter of the arc is red. In my humble opinion that would be somewhat more easy to read than a full arc and another half full arc.

    I see what you mean but I didn't go with that because my test box has almost no misses. So, you don't see any misses and don't get any concept that they are there visually. I'll just make another dashboard widget for that style.


    By the way, I would very much like to contribute to the project. Can we maybe move it to DMs? I would like to know how you have your dev env set u

    Sure.

    omv 6.0.41-1 Shaitan | 64 bit | 5.19 proxmox kernel | plugins :: omvextrasorg 6.1.1 | kvm 6.1.23 | mergerfs 6.3.3 | zfs 6.0.11
    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.
    Please don't PM for support... Too many PMs!

  • I still prefer the two arcs to one.

    omv 6.0.41-1 Shaitan | 64 bit | 5.19 proxmox kernel | plugins :: omvextrasorg 6.1.1 | kvm 6.1.23 | mergerfs 6.3.3 | zfs 6.0.11
    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.
    Please don't PM for support... Too many PMs!

  • I mean, that means that you have enough ram to have a high hitratio, but for example a friend of mine who built a nas with TrueNAS sometimes gets way lower hitratios when he boots his minecraft server. His arc will schrink a lot which causes his hitratio to tank. If it looks like that, that means all is well.


    But anyway, I'm not an expert. My personal preference goes to the percentage display, but then again, that's personal. I like both and I think you did great👍😃

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!