My AMD E350 NAS

  • It must be about time I show my build.
    I tried to build a energy efficient nas with enough disks to hold all our data. It should replace our existing QNAP TS409pro which has served us well for more than 7 years.


    The primary goals of the new nas is to serve files through SMB to our laptops, NFS shares to the mythtv backend server, the mailserver and the ownCloud server. It should also service mediafiles to the different media apliances ind the household.
    I'm more comfortable with raid 6 than with raid 5 so at least hold 6 disk in a raid 6 configuration was mandatory.
    Budget was a bit tight so it had to be fairly cheap. Last but not least it had to be small and quiet..


    I ended up with:
    Mobo: Asus E35M1-i with 6 x Sata III ports
    Ram: 2 x 4Mbyte Kingston
    Disks: 6 x WDRED 3TB (initial Seagate ST31500346AS)
    Chassis: Fractal design Node 304
    Power: BeQuiet 350W
    Pci card for 2 x Sata III ports and 4 x USB 3


    The Fractal design chassis is smart with rubber grommets damping vibrations from the big disks. Two fans in front of the disks and a big fan in the back. Standard configurtion only has room for 6 x 3.5" disks. So where to mount the system SSD. To begin with I had the system SSD just glued to the bottom of the chassis. Later I discovered some intrusions on the top railings that could hold a piece af aluminium above the mobo. So I made a small ekstra mount piece for the system SSD. There is room for three 2.5" disks next to each other.
    With the new homemade disk "tray" I have room for 6 x 3.5” and 3 x 2.5”.


    I installed Squeeze headless with backports, Tuned it for performance (swappiness low, elevator deadline and transparent hugepages etc). Then installed OMV.


    Parallel to building the new main OMV nas, I build a intermediate nas. The purpose was to hold all data from the old QNAP because I had to reuse the disks from the QNAP in the new nas. The intermediate nas was build from a Via C7 mini pc with two 1TB sata disks – OMV and no raid.
    I rsync'ed all data from the QNAP to intermediate nas, moved the disks from the QNAP to the newly build main nas – rsync'ed from intermediate to the new main nas. Everything worked as expected thanks to OMV.


    Besides basic nas functionallity i'm running Subsonic, Solarpower monitor and a minecraft server on the main nas.


    After the initial build the project evolved. Why not try to run iRedMail and ownCloud servers on Wheezy as a KVM guest on the main nas? So I did, and made a Howto: http://forums.openmediavault.org/viewtopic.php?f=13&t=2417


    During the first month after build one of my disks (Seagate ST31500346AS) failed. SMART data told me that three of the disks had rising number of reallocations. The Seagates was 5 years old so they where at the end of their operating life. I had to buy new disks – farewell initial budget.


    After reading in this forum I decided on buying 6 x WDRED's 3TB. I bought from 3 different dealers in the hope to get them from different batches so they won't fail at the same time.


    As they arrived I changed them one by one syncing the raid – hoping that the Seagates would survive the extra strain syncing raid up with new disks 5 times. Making daily rsync's to the intermediate nas at the same time. Fortunately everything work ok – no data was lost even though i had another disk failing during the process – glad I was running raid 6 and not raid 5.


    The new Nas performs like (with KVM guests running):

    Code
    root@LHJ-OMV:/media/8b8dba5d-6565-48d6-9544-6fcf6a0c7800/jk# dd if=/dev/zero of=speedtest bs=512k count=30720 
    30720+0 blokke ind 
    30720+0 blokke ud 
    16106127360 byte (16 GB) kopieret, 74,2848 s, 217 MB/s 
    root@LHJ-OMV:/media/8b8dba5d-6565-48d6-9544-6fcf6a0c7800/jk# dd of=/dev/zero if=speedtest bs=512k count=30720 
    30720+0 blokke ind 
    30720+0 blokke ud 
    16106127360 byte (16 GB) kopieret, 36,8988 s, 436 MB/s


    Iperf meassures throughput of 996Mb/sec through the onboard Realtek 8111E Gbit nic.



    The intermediate nas now has the role as a weekly backup unit. It has rsync jobs defined that every night does a rsync pull on every change of the important parts of the main nas. Once every week I turn it on just before going to sleep. Next morning it is turned of.
    Besides this I run a quarterly backup to external usb disks stored off site.


    Next step is to make the main nas more energy efficient. This involves turning of the GPU parts of the AMD E350. To this I need to install AMDs native Catalyst drivers. Unfortunately I installed the main nas as a headless unit. I have tried to install a minimal X environment from backports but can't seem to get it to work. I might wait til OMV 0.6 so I can reinstall on Wheezy with X.


    I read about undervolting thus saving energy. I'm not experienced in this and I'm hesitating a bit. Do I undervolt in the BIOS or in software? How about the stability of the nas when undervolting the CPU? Any feedback on this would be nice.


    In the meantime I guess I must live with an extra costs on the electricity bill.

  • Well i had to a be a bit creative to get room for everything. I wonder why Fractal design doesn't make this extra tray for the chassis now they have made the intrusions on the top railings. There is plenty of room and it doesn't disturb the airflow.


    Any good information on how to save energy without the risk of halting system for a undervolting noop?

  • Wow, A lot of stuff in that small box. How are your drive temps?


    Glad you got the raid rebuilt before the drives died.


    Peel and stick on velcro strips work in a pinch as well to attach things.


    If you get the autoshutdown plugin working for your build it will help save on the electric bill.

  • The Fractal design chassis has 2 fans frontmountet blowing air in to the chassis - right in front of the 6 disk. A big fan is removing air in the back above the mobo. You can se the big fan running on my picture with the homemade tray.


    This picture shows the general layout. The 6 x 3.5" disks are hanging in pairs in the white disk"trays". In front of these trays - behind the front panel - there are 2 fans blowing air into the chassis. Power supply is mountet under the 6 disks and mobo is in the back under the big fan.


    It works - Disk temp is between 29 and 35 degrees celsius - as I remember. When I come home from work I will check with SMART data from OMV GUI and post.


    It is a bit difficult for me to use the autoshutdown as I run the family's mailserver in a KVM guest on the nas. So I need to have it on 24/7.
    I can see a lot of postings regarding undervolting but I have no way to do it in software under squeeze. So I thought that I could do it in the bios - question is how much undervolting is safe.


    The other part is turning off the GPU. It demands that i have X installed - I must look at this soon or hope that OMV 0.6 is out soon.


    BTW: shifting from the Seagates to the WDREDs lowered disk temp by 5 degrees on average.

  • My recent omv build is an asrock e50m1 with 8g ram, 3 2g HD(seagate), dvd, 300w PSU in a coolermaster elite 120.


    I put it behind a killawatt and it uses between 50-60w. I will have to dig out my old buffalo nas and see how much it pulls for comparison.


    The drives show 32c. Waiting for wheezy for better hardware detection to monitor the cpu and chipset temps.


    Im curious as to how much energy you think you would be saving by undervolting?

  • I have checked the drive temps - they are:


    This time of day the disks are accessed all the time so this is the drive temp on full load


    earbiter many driverproblems disapear when using kernel 3.2xx from Squeeze backports repo though not all. I think that we are many that look forward to 0.6 on Wheezy


    I need to close down to move my consumption measurer. Right now it is meassuring as small ups with a D2700dc server running mythtv-backend, My nas and a small netgear switch.
    Everything at full power/utilisation draws 73W lowest consumption is 38W. What are you using for govenor? Check by:

    Code
    cpufreq-info


    I will move the Kill-a-watt tomorrow morning to meassure the nas alone.

  • Your temps look good. It is good to know the temp difference between the WD Reds & the Seagates. Less heat generally means less power used.


    I also have an AMD E350 board, mine is a Gigabyte, though it is not used as a server. Used it to rebuild an HP slimline.


    The power use items.


    You mentioned using 'cpufreq-info' to see if the CPU throttles down, mine does. I think there is a config file to set how aggressive the throttling is. See this article.


    The power supply should be an '80 Gold' or Platinum. Though I don't know if there are any 350W platinums out there. And the pay back time may be months to a year or more, based on the savings.


    I don't know if turning off things in the BIOS that are not used like serial and printer ports will save power. Maybe, maybe not.


    Depending on budget, maybe build a micro mail server, from something like a Pi or Beagle Bone. That way the Node 304 box could have the autoshutdown plugin. Payback though from electric savings will be longer.


    All of this may be moot once a Kill-A-Watt is hooked up and how much is actually used.

  • Yes I have thought about using one of my RPi's or my CubieBoard for the mailserver but my reason for buying a AMD E350 based mobo for the nas was that I would run mail and ownCloud etc on the nas board and remove several of my many small servers from the setup.
    I have moved services from 4 small servers to the nas - either as processes on the nas Squeeze it self or in KVM guest vm's running on the nas. The primary premise for this move was that the AMD E350 mobo would use less energy than my old QNAP nas and the 4 small servers together.


    My setup is default using the conservative govenor. I runs the CPU (both cores) at 800Mhz 95% of the time so the energy use of the CPU part of the E350 APU is scaling correct.


    Right now this doesn't seem so - i'm using about the same energy as before. I hope that with OMV 0.6 on Wheezy energy use will be lower all together. Is there a schedule for when OMV 0.6 is ready for release?


    Next time I install OMV (0.6) I will install it with X (LXDE) and the proprietary Catalyst driver. As I understand it this will give me the option of turning of the GPU.


    Any input on what undervolting is safe to do?


    BTW: my nas is using 53W running at full load. I'll check idle energy use tomorrow morning at 6 am as this should be when disks are spun down.

    • Offizieller Beitrag
    Zitat von "jensk"

    Is there a scheule for when OMV 0.6 is ready for release?


    No.


    53 watts is very good for a system with six hard drives. Each one of your drives can pull 6 watts and the fans in the system are pulling a few more watts. So, the motherboard is using less than 15 watts.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • You might be right maybe i'm demanding to much in respect to energy efficiency. 15watts for a mobo including a PCI Sata card is really quiet efficient. Thank you for setting this in the right perspective for me.
    Maybe iI should be satisfied with this and stop chasing more energy efficiency.

    • Offizieller Beitrag

    I think you would lose quite a bit of performance chasing that energy too :)

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • 53W used is good. I agree about not underclocking as I don't think the savings would be seen.

    I used this site to find the cost of power for my state. And I used this site to find the cost of use. The amount is in USD. So for me the cost is reasonable, but I live in the land of cheap electricity.


    After reviewing Global Electricity Price Comparison I understand why you are so concerned. Especially since your electric costs are more than four times mine. It would cost me about $3.75 a month where it costs you more than $16.50. Ouch!


    I would look at building a micro NAS with a Pi/Cubieboard, maybe in a case something like this. The Pi's OS can be put on an HDD. And if 2 (one for OS, one for data) 2.5 HDD's were used the Pi and drives should fit inside. I'm playing with one myself, but time is in short supply. Take a look at http://forums.openmediavault.org/viewtopic.php?f=25&t=2651 for a possible Pi mail server program.


    I now understand why Pi's are so popular for home server use in Europe. Maybe you could use one of the other NAS boxes you have to build a Pi into.


    After seeing your costs I would look at underclocking. Be sure to back up though, as I have no idea what it will do for system reliability. Take a look at more efficient power supplies as payback will be much faster.


    Good luck!

  • Zitat von "PhantomSens"

    [...]
    After reviewing Global Electricity Price Comparison I understand why you are so concerned. Especially since your electric costs are more than four times mine. It would cost me about $3.75 a month where it costs you more than $16.50. Ouch!
    [...]


    WTF? :o :o :o Electricity is so cheap overseas?


    Greetings
    David

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • Ya! And I complain about my electric bill! This put it into perspective for me. It also explains why we here in the US don't seem to understand why others are so concerned about the power used by their servers.

  • That is realy cheap electricity you have.


    Electricity here is so expensive that last year I installed 4000w solarpanels on the roof of the house. Allmost all light is now LED based. I keep telling all the kids to remember to turn of light when they leave the room etc. This leaves our computers, game consoles and other entertainment equipment as some of the biggest consumers.


    Saving on light, new low energy freeser and such has cut electricity consumption by 25%. The solar panels can supply approx. 60-70%. So I have 30-40% of total consumptons that I buy from the utility company at 2.10 danish kroner = 0.28€ = 0,38$ pr. kilowatt hour incl taxes and VAT.


    Maybe I should look at building a Pi nas.


    BTW I read that the WDRED's should have apm implemented. Should I set the spin down parameters and such in OMV our should I leave this to the buildin apm features of the WDREDs?

  • You have done a lot to save. I understand about saving even more. Good luck!


    Sorry I don't know enough about the drive question though.

  • The WD reds are designed to run 24/7. I cannot even tell if they spin down if left alone but I rather prevent the spindown because I grab smart data every 3 mins.


    Greetings
    David

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • After two days of meassure on the energy consumptions:


    Max utilisation = 53 watts
    Min utilisation = 32 watts


    Both is with KVM guests running.


    CPUgovenor is "conservative" and the lowest CPU speed (800MHz) is chosen for more than 95% of running time.

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!