"Chap" - My 15TB+ NAS

  • ## 2013/05/06 17:11 Thread started
    ## 2013/05/07 00:23 Screenshots added (Mounting the first four Drives, one bad Drive, w/ old Controller)
    ## 2013/05/08 03:31 Added Plugins, purpose, future plans etc.
    ## 2013/06/XX WoL is now running


    Hey Guys,


    i thought today is the Day I will start my own Thread here.


    As a little Background, i've been using Linux for over 3 Years now at home for my NAS and used OMV since Version .2.
    So iam aware what it means to setup SMB Shares by hand etc. etc.


    Specs Today:
    Core i5 650 @ 3,2GHz
    4GB RAM
    500GB System Drive
    RAID Controller: (LSI) 3ware 9650SE-16ML (upgraded 04/2013 from 9650SE-8LPML)
    5 x WD RED 3TB as a Hardware RAID 5 - 10,91TiB net.


    The Story behind


    So, Some Months ago i had moved from an old Desktop Case which held a 4x2TB RAID5 to the Xigmatek Asgard 381.


    This Case provides easy acces to all my Drives and also the place for two 120mm Fans in front of all the Harddrives attached to it.


    After moving to the new Case i got some problems regarding an unusable Array because one Drive Failed and another one got disconnected - I thank Josef from the LSI support here, thanks to him i could recover all my Data.


    My other Problem was that my Disk Usage was already over 5TB so i had to expand my Storage.
    I was looking at the Prices for the WD RED 3TB some weeks and waited for the point where i could order 4 Drives for under 600 Euros. So it was close before Easter when i ordered the 4 Drives.
    Unfortunately one Drive got damaged at shipping and the Dealer refused to resend a new Drive right away.
    At this Point i didn't wanted to wait over the long Holidays to get a new Drive, so i went to my local Dealer an picked up one "replacement" Drive.
    With these 4 Drives i then build a new RAID 5 and filled it with my Data.


    A few Days later, the replaced Drive from the online Dealer came, and i thought what should i do with it?
    Well after some discussion with a friend he told me into putting the Drive into the Array (i wanted to stick with 4 Drive Arrays before) and use the OCE (Online Capacity Expansion) Feature of my RAID Controller.
    Now I already had about 5TB of Data on my Disks and thought how long the "Migration" would take, and just little over 5 Days later i had an Array with 10,91TiB ;).
    All i had todo now was to expand the XFS Filesystem (i used a gparted live cd for it) to use the new Space available.


    Penis-o-Meter


    Here are some Benchmarks of the Array:


    Write:

    Code
    # dd if=/dev/zero of=tempfile bs=1MB count=102400
    10737418240 Bytes (11 GB) kopiert, 19,9063 s, 539 MB/s
    107374182400 Bytes (107 GB) kopiert, 224,111 s, 479 MB/s


    Read:

    Code
    # dd if=tempfile of=/dev/zero bs=1MB count=102400
    10737418240 Bytes (11 GB) kopiert, 22,4036 s, 479 MB/s
    107374182400 Bytes (107 GB) kopiert, 220,974 s, 486 MB/s


    hdparm:

    Code
    /dev/sdb:
    Timing O_DIRECT cached reads: 1168 MB in 2.00 seconds = 583.57 MB/sec
    Timing O_DIRECT disk reads: 1640 MB in 3.00 seconds = 546.61 MB/sec
    
    
    /dev/sdb:
    Timing cached reads: 10732 MB in 2.00 seconds = 5368.82 MB/sec
    Timing buffered disk reads: 1646 MB in 3.00 seconds = 548.62 MB/sec



    I'll add some more Screenshots later.


    Greetings
    David

    Bilder

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

    3 Mal editiert, zuletzt von davidh2k ()

  • Iam using the following Plugins:


    (forced, HTTPS)
    Autoshutdown (no ips configured, just using the port watchdog, shutdown
    MySQL
    OpenVPN
    (Ajaxexplorer - doesn't work, so i don't use it^^) - deleted 2013/07/02
    DLNA (only for some testing purposes, i use an XBMC DB for my media)
    Plex - i wanted to test it out how it compares to DLNA and XBMC - added 2013/06/XX


    and via xrdp:
    jdownloader2 (not using it yet)
    linuxdcpp (sharing files on LAN Partys)
    handbrake - encoding some fraps videos to h.264 (i now know that my cpu cooler goes up to 100% in case my cpu hits 85° :roll: ) - added 2013/07/04


    I have configured a MySQL DB for my XBMC, iam using it mostly for my raspberry pi so it is optimized for it (i use nfs shares for the pi) therefore i lack the ability to use it on my Windows PC...
    As you can see in my sig, i am sharing Folders with Movies and TV Series' and "TV Series 2013", where the directorys are handled via Softlinks:



    To add the Softlinks i have a little Script: (if you use it, remember to use "" for Series with Spaces)


    add.sh:

    Bash
    #!/bin/sh
    # Script zum hinzufuegen von Softlinks
    
    
    cd /media/d3c21c66-af4b-41d4-b098-462e83fa641d/Serien/
    ln -s "$@" /media/d3c21c66-af4b-41d4-b098-462e83fa641d/ASerien/$@
    echo "Softlinkuer Serie $@ erfolgreich angelegt!"


    To look at my HDD temperatures i need to look it up manually via smartmontools, i have a little script for it too:


    smart3ware.sh:


    The output looks like: (its running every 3 minutes via cron)

    Code
    2013-05-07_02:21:02 |  0:(30°) 1:(29°) 2:(29°) 3:(29°) 4:(32°)
    2013-05-08_03:03:01 |  0:(22°) 1:(22°) 2:(22°) 3:(22°) 4:(22°)
    2013-05-08_03:06:01 |  0:(24°) 1:(23°) 2:(24°) 3:(24°) 4:(24°)


    Penis-o-Meter 2 2013/07/04


    I'm now storing about

    Zitat

    [*]ca. 280 Movies in both 1080p (The big part of them, 2,3TB+) and 720p Quality


    and over

    Zitat

    [*]4TiB of TV Series :twisted:
    [*]57 TV Series
    [*]35 Seasons in 1080p
    [*]96 Seasons in 720p
    [*]2400 Episodes in HD (720p and 1080i/p)



    Future Plans:
    using chap as a download station
    replacing the system drive with a (128/256gb-) SSD for better performance/virtualbox usage.
    bonding the 2 NICs together (but i think for real speed i need a better switch by then)
    adding more wd red 3tb as needed, maybe if some time has gone and the 4 and 5tb versions that are coming by the end of the year are pricely attractive changing to them with a second array.




    *reserved*

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

    6 Mal editiert, zuletzt von davidh2k ()

  • *reserved* Pics, more are coming in the next Days.





    2013/05/20: Finally here are some more Pictures of the actual state:








    3ware Webinterface:


    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

    2 Mal editiert, zuletzt von davidh2k ()

  • Very nice build.


    <EDIT 07May13> Even better with the pictures!

  • Zitat von "DaveVM"

    Very nice build.


    Thanks :)


    I added some more information, maybe i get to take some pictures of the actual state tomorrow.


    Greetings
    David

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • Zitat von "knumsi"

    Nice build!


    What is the power consumption of this PC?
    How do you wake the OMV NAS up, when you need it?
    What kind of Bonding do you use for your NICs?



    - i don't know yet, but i'll check it right away
    - I put my finger on the power button (no kidding, it stands right under my desktop/tv - last time i checked WOL didn't work, i have to look deeper for this wakeup-method)
    - i don't use it yet, but i would go after that tutorial: http://louwrentius.com/blog/20…-how-to-get-beyond-1-gbs/


    Greetings
    David


    Edith says: i checked the power consumption, it is up to 160watts on boot and then falls down to 113watts. It goes up to 145-150watts on Raid write.

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • Thanks for your answer.


    In my opinion the power consumption is too high. Did you try powertop to lower it?


    For WOL you could try this: http://phpbb.openmediavault.org/viewtopic.php?f=13&t=1882 For the hardware support of WOL think of enabling it in the BIOS, plugin the WOL cord or assure the PCI port of the mainboard supports it =)


    Which switch will you use to support bonding with round robin and link aggregation? The time (ca. 5 yrs ago) i was trying bonding those switches were too expensive.


    knumsi

  • i used powertop, but it now hangs at ~138watts, it seems the sata optimization doesn't worked out the way it should ...



    Greetings
    David


    Edith: after a reboot he is back to 117watts idle.
    Edith2 says:


    Was dein WOL Script angeht, so kann ich dies nicht nutzen da ich die gemietete Kabel-Fritz!Box von kabeldeutschland habe, aussedem waere das gar nicht meine intention, dass der server automatisch startet sobald meine rechner eingeschaltet werden.


    As a Switch i would get simply one of these:
    http://geizhals.de/?cat=switchgi&xf=658_managed#xf_top
    all seem to be managed switches, starting as low as 50 euros for an 8port switch, 105 euros for 24port hp pro curve or 112 euros for a good 16port TP-Link.

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • Thanks for your answer.


    Perhaps powertop works better on the newer wheezy kernel, as soon omv is released with it.


    The prices for the switches supporting Link Aggregation are ok. I am looking forward to your performance tests ;)

  • I just updated the "I'm now storing about..." Part.


    knumsi, sry, but that won't happen anytime soon, cause of some personal stuff that comes first :-x


    Hopefully i can get the new Pictures up tomorrow or so...


    Greetings
    David


    PS: After downloading this much stuff in the past two weeks, iam getting agressive at my jdownloader....
    PPS: Did i told you the funny part of the last two weeks? My Online Dealer send me a second replacement drive, i have now a 6th wd red 3tb lie around here, for which i, in fact, did no pay any cent... iam wondering if they may send me another one... :lol:

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • I just added a bunch of new pics.
    Enjoy it ;)


    Greetings
    David

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • For WOL I used to use the client on DD-WRT. But more and more I am using my handi for everything. There are great apps for android that do WOL, Wol Wake on LAN WAN by Brobble. Fing is another great app to scan your network.

  • I have a "rent" Fritz!box from cable germany, i simply cannot use anything besides the integrated WOL button.
    Oh and btw.: i use an iPhone :D


    Greetings
    David

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • Thanks tekkbebe, you pushed me to look into it and now my WOL is working fine :) works even after halt.


    Greetings
    David

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • I updated the first Posts with some additional Information.
    Iam now running a Plex Media Server for testing purposes and am using my server to encode some Videos to h264 with Handbrake. ( I now know that my bios setting for my cpu cooler works. If the CPU reaches 85° it starts to spin to 100% (runs on 25% normally, as the boxed cooler is really the loudest part on this setup!))


    knumsi: What we forgot to think about is the power consumption of the dedicated graphics card. Even in idle i think i will use at least ~30 watts.


    Also the sixt WD Red is still laying around here... never heard from the vendor again.


    The 3rd and the 5th drive are the most of time the hottest ones. 36° Celsius was the highest temperature they encoutered. Ignore Disk 6 and 7, they were previously attached to the controller and i added them to the watch script


    Greetings
    David

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • Is funny about the Red drive. hehe


    I've been getting more into Plex lately. Starting to find faults and solutions. I think next up is to redo the naming of the few tv series that I have. I think Plex is great if you take time to understand how it works.


    If your cpu is getting so hot maybe you should try one of the Corsair water coolers. Do the Red disks normally run this hot?


    I got ajaxexplorer working a while back with a virtualhost in OMV. Was not to enthused by it.

  • Ah i forgot to mention: I already renamed my ENTIRE TV Episodes to the format Plex expects. (about 2-3k Files..., My Renamer asks me now every time if i want to donate, because i already renamed 4840 files with it)


    Erm.... i could post you full log so you scroll through it and see temps for yourself.
    http://88.198.26.218/omv/pall.txt
    Knock yourself out.


    My CPU just run that because a have the cpu (boxed) cooler locked to 25% (it does not go slower that that, it is set to 12.5 percent but has a minimum rpm) and i had running some h264 encodes. But thanks for the watercooler suggestion. Maybe this keeps the server even quieter.


    For the Quiet part i have to say that the WD Red are really quiet! The 2TB Samsung/Seagate Eco Green are noticeably louder!


    Greetings
    David

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • The drives seem ok. Not consistently that hot. I think 85 celsius is pretty hot for the cpu. You must be doing it to keep it quiet. I have my machine shutdown if cpu reaches 65 celsius. I would change this. Can't be good to let it get this hot. If noise is what you worried about, and not price, ask Ryeco about the noise on the Corsairs. Otherwise check out a different cooler if you are going to run your fan at 25%.

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!