frequent disk IO

  • I installed IOTOP and see that collectd is causing disk IO every 5 seconds. Looking at the config for collectd I dont see where I can adjust it? I would prefer it to update MUCH less frequently ie: every minute or 15 minutes!

  • The frequent disk writing is a general Debian problem, thats why it is not recommended to use flash drives for OS Installation except SSDs.


    Greetings
    David

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • Hi David, Since the stock omv build wont let you use raid, I installed squeeze and setup raid 5 partitions before installing omv. So I am not worried about any flash/ssd problems but I would like to slow down the disk updates as they really are not wanted.


    You mention this is generally just a debian issue? With iotop I really only see collectd thrashing every 5 seconds.


    Would there be a problem if I just remove collectd?

  • If you dig through the forum you might find some more information about this, as it has been discussed multiple times now.


    I don't understand what you mean that OMV could not handle RAID? OMV can handle 0,1,5,6,10 and many more i guess, because debian itself can handle it.


    Greetings
    David

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • From the man page of collectd.conf.

    Quote

    Interval Seconds
    Configures the interval in which to query the read plugins. Obviously smaller values lead to a higher system load produced by collectd, while higher values lead to more coarse statistics. Warning: You should set this once and then never touch it again. If you do, you will have to delete all your RRD files or know some serious RRDtool magic! (Assuming you're using the RRDtool or RRDCacheD plugin.)


    The default is supposed to be ten seconds. However by changing the interval than all of the graphs that OMV creates will be trashed and will apparently have to be recreated. How? Good question. Simply delete as the man page states and hope they are recreated. I don't know. Do so at your own risk.


    If collectd were to be stopped/removed there may also be other errors, from the cron jobs trying to create the OMV graphs, to anything else OMV uses collectd for.


    Maybe this would be an idea to add to the install routine of OMV. What interval a user would want, or ask what type of drive OMV is being installed on. SSD or HD, system drive. I would not expect this change though anytime soon as I see it as a nice to have feature. Also because Volker has his hands full trying to get 0.5 OMV out as well as working on 0.6


    If collectd is the process doing the majority of disk IO, then stopping/removing it may save a lot of wear and tear on all media, particularly flash media. If it's only use is to create the graphs then they would be lost. Though some of us like our graphs and being able to see how our OMV box is performing.


    Or is there a different process that could replace it?

  • Davidh2k - I have the system installed on a raid volume which is something supposedly not supported by omv's installer. I did read the forums searching for detail on that and collectd. It was suggested the only way to get the OS on raid was to install squeeze as its install process allowed for raid. I didnt find anything regarding the collection interval so thats why I posted.


    DaveVM - I will look closer. I didnt see the refresh interval coded in any of the collectd conf files so assumed it was defaulting.


    ---


    Thanks for the help.


    I noticed that puppy linux just released a new version and is using f2fs - http://en.wikipedia.org/wiki/F2FS
    It would be interesting to see if omv could be built to run on that so it could be run from a usb stick.

  • If your system drive dies the raid created under OMV should be ok. You do not have to create a raid during install. Even if you are using a raid controller you can do it post install. If your system disk dies you just have to unplug the data (raid) drives and reinstall OMV on a new system drive. Then you simply turn off the machine, reconnect the data drives, start and once in the web-gui you can mount the raid again.

  • I think you missunderstood, he installed OMV directly to a raid, I guess raid 1 or 0...


    Greetings
    David

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • Yeah, it was not clear and I did not read everything. So I read everything and now it is still not clear. :shock:


    Quote from "earbiter"

    I installed squeeze and setup raid 5 partitions before installing omv

  • LOL, okay, I just read his last post... maybe he will clarify it for us ;)


    Greetings
    David

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • Sorry if I wasnt clear. See bottom for what appears to resolve this for me.


    I have 3 2tb drives so I setup six raid 5 partitions:


    1-3 are 20g for os installs
    4 swap
    5-6 are 1.8tb for data for omv shares


    I installed omv on #1 and linux mint 15 on #2 although the only way I can get lm15 to boot is to use the super grub2 boot cd and manually find/boot it!



    >! UPDATE !<


    I was able to eliminate a lot of the disk IO by changing my root / mount in fstab by adding commit=60 as well as adding an 'Interval 30' entry to the /etc/collectd/collectd.conf although I imagine that has trashed my stats as I havent deleted the rrd files yet..


    I still see rsyslog and flush IO as well as some jbd2. But not every 5-10 seconds now. I see the disk light flash but IOTop doesnt report anything. Strange. I may up these values and see if that makes any difference.

  • Well... I would not recommend your setup, as it has a reason that OMV splits the OS from any data disks.


    Get a small notebook HDD or SSD for the OS, way better than your setup.


    Greetings
    David

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • I wouldve used a spare drive to host OMV -BUT- I built my system with a asrock e350m1 and it only has 4 sata of which all are in use. Guess I could lose the dvd but find it useful for htpc. With this setup I can reboot into a full desktop (linux mint 15). I plan to install either vortexbox, xbmcbuntu, or mythbuntu on the other slice so I can reboot between them. Although I guess I could try do it all virtually as omv supports virtualbox instances! Although I dont know how the performance would be.


    I have run software raid on linux for over a decade on mission critical systems with NO problems. So I dont see why omv cant run on raid5? I am doing it now. Just had some hoops to jump through to make it work.

  • Well, get the idea out of your head that you can virtualize your HTPC (for the media part).


    I would suggest you to get an external DVD drive, or you could hookup one disk to your eSata port that is on your I/O Panel.


    Besides, there are many, really low level power usage media center solutions out there like the raspberry pi (more for the geeks under us) or the intel nuc (which i just discovered myself) where even the cheapest celeron version is able to play untouched blurays, or you could get some other media players like the mede8er or wd live.


    Greetings
    David

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • You could also try a sata port multiplier like this or this. Note they do not use a PCI or other port on the motherboard.

  • At least don't use any flash storage except a SSD...


    Greetings
    David

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • I've found https://collectd.org/wiki/inde…Inside_the_RRDtool_plugin and modified the rrdtool config in OMV 0.5 to


    Code
    LoadPlugin rrdtool
    <Plugin rrdtool>
    DataDir "/var/lib/collectd/rrd"
    CacheTimeout 120
    CacheFlush 900
    WritesPerSecond 50 <========== new
    </Plugin>


    The omv-mkgraph command sends now a FLUSH command before creating the graphs, but i'm not sure if this will finally reduce disk I/O. Any tip how to improve collectd is welcome. But i will not use RRDCacheD plugin.

  • Quote from "votdev"

    Any tip how to improve collectd is welcome.


    Don't know if this will help.


    About the only thing I can come up with is changing the Interval at which collectd gathers information. Another link to man pages I found.


    Code
    LoadPlugin rrdtool
    <Plugin rrdtool>
    DataDir "/var/lib/collectd/rrd"
    CacheTimeout 120
    CacheFlush 900
    WritesPerSecond 50 <========== new
    Interval 15 ---<<< Proposed & could be higher >>>---
    </Plugin>


    Where I came up with the supposed default of ten seconds I can not recall.


    What would be a good interval? Again I do not know. For those who like moment by moment data graphs as they are now, then the default is good, for those who are not so concerned a bigger interval would be good. I would set up several different VM's of OMV with different interval values of say 10, 15, 20, 30, 45 and run the VM's for about 30 days and then compare the graphs to come up with what may be acceptable.


    For current users of an installed OMV version changing the interval on an active system will trash the OMV graph data; from the collectd man;

    Quote

    Interval Seconds
    Configures the interval in which to query the read plugins. Obviously smaller values lead to a higher system load produced by collectd, while higher values lead to more coarse statistics.


    Warning: You should set this once and then never touch it again. If you do, you will have to delete all your RRD files or know some serious RRDtool magic! (Assuming you're using the RRDtool or RRDCacheD plugin.)


    So do not change the interval on an active system unless you want to restart ALL of your graphs.

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!