Posts by ptruman

    TUTORIAL : Installing OMV & converting O/S to RAID1


    Postby mr-pete (now ptruman) » Fri Mar 30, 2012 4:00 pm


    After much fiddling, I now have the OMV O/S partitions on a software RAID1. I did originally do this with dmraid and my servers BIOS "fakeraid", but it was a bit problematic, so I moved it to s/w RAID with mdadm. My data is then on 2 other drives, also in RAID1.


    I did this *ideally* I wanted the O/S to be resilient, as :


    a) I don't have a "proper" H/W RAID controller
    b) My machine only support FakeRAID (BIOS RAID) and OMV doesn't get on it with it
    c) My OMV server runs more than just OMV, so ideally it needs to be recoverable ASAP


    The method I've gone with should (I hope) let my system boot if either of the RAID1 O/S pair fails (and is removed), or, failing that, can be recovered via a rescue boot from the ISO, and a swift dose of mdadm and grub-mkconfig


    And this is how I did it...


    Prerequisitites
    A server :)
    At least 4 drives - 2 for the O/S and 2 for data (each pair should be of identical size!)
    At least 2 hours spare to get the O/S done (based on my O/S drives being SATA 1.5Gbps 80GB drives)
    A CD-ROM drive (I used a PATA IDE CDROM connected via a USB adaptor)
    A CD-ROM/USB stick with the OMV boot ISO on it


    Disclaimer 1 : The majority of these instructions came from this page (http://www.pinguin-systeme.net…system-to-a-raid-1-system) on converting a running system to RAID1 - but I had to tailor it somewhat. Hopefully this also saves you searching :)
    Disclaimer 2 : If you break your system/data/life - it's not my fault - these instructions worked for me. They may not for you/your system!
    Disclaimer 3 : I am by no means a Unix/Linux expert/practitioner, but I got this going. Again, if you're not a guru, I hope this helps.


    Step 1)
    Starting with a powered DOWN server,
    Remove ALL HDDs from the computer
    Insert ONE HDD that you want as part of the intended RAID1 O/S drive into the server, on the first SATA port
    NB: If you're going to use a USB boot device for the ISO, connect it now...


    Step 2)
    Power up the server - insert the boot CD if you're going to use it
    Ensure the server boots from CD (amend your BIOS/boot order if necessary)
    Allow the server to run a default/basic (hands free) installation (unless you need to play with expert mode)
    Remove the CD/USB stick when prompted, and let the server reboot


    Assuming all went well with the default install, OMV should boot up, on a single disk - now you can start upgrading :)


    Step 3)
    Login to the server as root, or sudo bash to a root prompt
    Now is the time to save yourself time later, especially if you're working with one monitor and/or no KVM!


    Open the OMV Web GUI and go to the SSH section. Enable it, and click Ok.
    If you setup a root login during the install process, tick "Permit root login" and click Ok.
    If you did NOT setup a root login (i.e. you opted to use sudo), then go back to a command prompt on the server and run


    Code
    usermod -a -G ssh <youruser>



    Your user can now login via ssh and you can avoid using the console, unless you really screw things up later :)


    Step 4)
    Re-insert the OMV ISO CD/USB stick


    Code
    shutdown -h now


    To shutdown the server


    Now insert the 2nd disk you want as part of the intended RAID1 O/S drive into the server, on the second SATA port and boot up and get to a root prompt.


    Then, you need to exactly clone the partition table from /dev/sda to /dev/sdb, by running


    Code
    sfdisk -d /dev/sda | sfdisk --force /dev/sdb


    The --force switch is key, otherwise you will find your partitions likely don't exactly match, and you'll get stuck later. I did, and it cost me an hour... :(


    You should then run


    Code
    fdisk -l /dev/sda
    fdisk -l /dev/sdb


    You should output from both two drives, and (this is key) all the start, end and block columns should match. If they don't, you didn't use --force above, or something has gone wrong - go no further until you've sorted that out.


    NB: In the fdisk -l output, you should see three partitions (/dev/sda1, /dev/sda2 and/dev/sda5), listed as Ext4, Extended and Swap. Again, if you don't have those three listed, and they are not of those types, something went wrong in the installation partitioning - go no further until you've sorted that out.


    Step 5)
    Assuming the partitions cloned successfully, you can setup the initial RAID on the 2nd drive, by typing :


    Code
    mdadm --create /dev/md0 --level=1 --raid-disks=2 missing /dev/sdb1
    mdadm --create /dev/md1 --level=1 --raid-disks=2 missing /dev/sdb5


    The above creates a software RAID1 array on the second drive (you'll see why in a minute), using partitions 1 (ext4) and 5 (swap).


    Now we need to format both new partitions, which is achieved by typing:


    Code
    mkfs.ext4 -j /dev/md0
    mkswap /dev/md1


    Now we have to copy all the data from the first drive to the second, to get things ready, so type


    Code
    mkdir /mnt/md0
    mount /dev/md0 /mnt/md0
    cp -axu / /mnt/md0


    The above will mount the RAID array on the second drive and copy everything across.


    Now things get funky :)


    Step 6)
    Now, backup your /etc/fstab & GRUB2 configuration files (as at present your system can still boot if necessary, via /dev/sda1)


    Code
    cp /etc/fstab /etc/fstab.orig
    cp /boot/grub/grub.cfg /boot/grub/grub.cfg.orig


    Now, you need to get the unique ID names for your drive partitions, by running :


    Code
    blkid



    Copy the output from that and save it somewhere handy so you can use it cut & paste.
    You are looking for the line that starts /dev/md0 - and you need the full UUID string between the quote " marks (not including the " marks)


    Edit the /etc/fstab file by typing :


    Code
    nano /etc/fstab


    Look under the following line...
    # <file system> <mount point> <type> <options> <dump> <pass>


    ...to find the root filesystem lineThis will look something like
    /dev/sda1 / ext4 errors=remount,ro 0 1


    OR
    UUID=<UUID string> / ext4 errors=remount,ro 0 1



    Replace EITHER /dev/sda1 OR the existing <UUID string> with the one you copied from the blkid output earlier.
    Press CTRL-X to quit nano, and press Y when it asks if you want to save the file, then press RETURN to confirm the save.


    You have now amended the mount configuration to mount the root filesystem from the newly defined (and still incomplete) RAID array.
    Now we need to get GRUB2 to boot off the second SATA disk.


    Edit the /boot/grub/grub.cfg file by typing :


    Code
    nano /boot/grub/grub.cfg



    Scroll down in the file until you find the line
    ### BEGIN /etc/grub.d/10_linux ###



    Find the FIRST set of two lines that start
    search --no-floppy --fs-uuid --set
    linux /boot/vmlinuz-2.6.32-5-amd64 root=UUID=


    And replace the UUID at the end with the UUID of /dev/md0 you copied from the blkid output earlier.
    Press CTRL-X to quit nano, and press Y when it asks if you want to save the file, then press RETURN to confirm the save.


    Now reboot your system and check all is well!


    Code
    shutdown -r now



    Step 7)
    At this point you should have a booting system (now loading the root filesystem from the second disk), although your RAID is still not setup.
    However, as your root filesystem is now on the second disk, you are free to add the first disk into the array (as it's not locked by the O/S)


    First of all, the main partitions on both drives must be changed from ext4 & swap to "Linux raid autodetect"
    To do this, login, and get a root prompt.
    Then update the first disk by typing :


    Code
    fdisk /dev/sda


    Then enter the following keystrokes :


    Code
    t
    1
    fd
    t
    5
    fd
    w


    Don't worry if you see any errors about needing to reboot first.
    Then update the second disk by typing :


    Code
    fdisk /dev/sdb


    Then enter the following keystrokes :


    Code
    t
    1
    fd
    t
    5
    fd
    w


    Again, don't worry if you see any errors about needing to reboot first.
    NB: We only need to update the partitions numbered 1 & 5 - don't worry about partition 2.


    Now force the partitions to commit and be re-read by typing :


    Code
    partprobe



    Verify the new partition types have set by running :


    Code
    fdisk /dev/sda
    fdisk /dev/sdb


    Check the types now read (in order, on both disks)
    Linux raid autodetect
    Extended
    Linux raid autodetect



    Step 8)
    Now it's (finally) time to add the first disk into the array. Remember you're now loading the root filesystem from the second disk, so it's not locked - so enter :


    Code
    swapoff -a
    mdadm /dev/md126 -a /dev/sda1
    mdadm /dev/md127 -a /dev/sda5


    The above disable swap usage so you can add partition 5, and then adds partitions 1 & 5 to the array.
    We're now referencing /dev/md126 & /dev/127 as the system rebooted and changed the references from 0 & 1. If the above doesn't work for you, run blkid again, and find the two bottom entries, which should read ext4 & swap. The ext4 partition is the lower number (i.e. /dev/md0 OR /dev/md126) and the swap partition is the other.


    Hopefully now the RAID is assembling. You can speed this up and watch it by typing :


    Code
    echo 50000 >/proc/sys/dev/raid/speed_limit_min
    watch cat /proc/mdstat


    NB: It took me 30 mins to get to this stage, and took my system a further 30 mins to fully complete the RAID assembly.


    Step 9)
    Once it's finished, we can add some future redundancy by ensuring the system can fully boot from the second disk if the first fails - by typing


    Code
    grub-install /dev/sdb



    Now - put the OMV ISO CD in the drive, and reboot


    Code
    shutdown -r now



    When the boot menu appears, choose Rescue mode - select your language and let it do it's thing.
    When you get to choose a partition, you need to select Assemble Raid and press RETURN
    You should then see /dev/md0 as an option - select that and run a shell in it.


    Now, we ensure GRUB2 is on the first disk AND the RAID by typing


    Code
    grub-install /dev/sda
    grub-install /dev/md0


    Then we update the GRUB2 config to boot out of the new RAID partition entirely - but first back it up!


    Code
    cp /boot/grub/grub.cfg /boot/grub/grub.cfg.stage2
    grub-mkconfig >> /boot/grub/grub.cfg


    Then, type exit and reboot the system.


    If all has gone well, your OMV will now be booting from a RAID1 partition. You can then shutdown, add your other drives and format/RAID those as necessary from the GUI.


    IMPORTANT NOTE :
    Your system still boots from the first SATA disk - where the grub config is read and told to load out of the /dev/md0 partition
    If you later come to do an apt-get dist-upgrade you MAY find (if you don't read all the prompts you get) that your system fails to boot when you reboot - as the mdadm.conf may have been amended by OMV, and you're stuck at a nice (initramfs) prompt.


    If that's the case, don't worry - you can temporarily boot your system by typing


    Code
    mdadm --assemble -scan
    CTRL-D


    Your system will assemble your RAID and boot, but you have to do this every time.


    If this happens, let me know and I'll post my solution :)
    (which was a minor edit of /etc/mdadm.conf and adding two symbolic links in /dev)
    mr-pete

    I had EXACTLY this question the other day, after a good few years of using DynDNS, I found my subdomain had vanished, and then realised I'd used an old email address.....and then that I had to login every 30 days.


    I started was looking at a variety of Dynamic providers - and being a user of OpenDNS, I use their DNSOMatic (http://www.dnsomatic.com) service to keep my network in line with OpenDNS - and DNSOMatic also updated my DynDNS for me - so I had a look at their list of services (http://www.dnsomatic.com/wiki/supportedservices) to see what I could find....


    HOWEVER, whilst some were free, some required 30 day logins, or just didn't seem right, or didn't have the subdomain name I wanted, or a domain I liked :)


    So, as I own quite a few domains, I thought laterally...


    I decided to just pick ANY subdomain - in the end, I plumped for a subdomain off of anydns.com (from freedns.com) - I just need a non-expiring free dynamic record - name unimportant....


    I then setup a CNAME from one of my domains to point to the A record for the dynamic anydns.com subdomain.
    Then I setup DNSOMatic.com to update the anydns.com record for me.


    And it works :)


    So I've got (a) the subdomain I wanted and (b) my own domain - which just point to the dynamic IP which is updated via my router updating DNSOMatic.


    Job done :)


    If you have a domain but no control over the DNS itself, consider hosting with domainmonster.com - they will host your domains for free if you transfer to them for renewals - and give you full DNS control :)

    Lo all,


    Mostly "because it was there", I decided to play with CUPS.


    I've got a Brother DCP-770CW printer (about 4-5 years old), connected to my LAN via CAT5 - and my Windows PCs print directly to it. I'd also seen the Google Cloud Print (Beta) stuff (http://www.google.com/cloudprint/learn/) and spied a set of CUPS wrappers for my printer.


    1) Login to OMV
    2) Choose "Plugins"
    3) Find and highlight "OpenMediaVault CUPS"
    4) Click install
    5) Wait :)


    That got CUPs installed, but printer drivers were still missing. The Brother website was as useful as a chocolate fireguard, as they only supply x86 drivers, and my N40L is x64....


    So I had to :


    6) apt-get install csh
    7) Manually find & install (via dpkg) brother-lpr-drivers-common
    8) Manually find & install (via dpkg) brother-cups-wrapper-common
    9) Manually find & install (via dpkg) brother-lpr-drivers-extra
    10) Manually find & install (via dpkg) brother-cups-wrapper-extra


    So that's CUPs with the relevant drivers, but still no installed printer...


    11) Then navigate to http://openmediavault:631/admin (login/password is the OMV admin one btw - that slipped me by!)
    12) Click "Find New Printers"


    For some reason mine was shown twice, so I just used the first one....


    NOW you have a printer - and can choose to share it via Samba (I chose not to share mine as it's already mapped manually)
    You may also need to add the printer to the OMV CUPs plugin page once it's in CUPs


    Now for Google Cloud Print....


    13) Visit http://dev.shyd.de/2012/01/rem…ebian-google-cloud-print/
    14) Follow instructions 1 - 3, i.e. :


    apt-get install python-pip python-cups
    pip install daemon
    pip install cloudprint


    15) Follow the instructions to cut/paste the /etc/init.d/cloudprint file (and edit "myuser" to be "lp"
    16) You'll also need to mkdir /var/spool/lpd and chown/chgrp the dir to be owned by "lp")


    17) Follow the instructions to add the service and start it, and you'll probably get told you need to add python-daemon with a "-d" to get the daemon to work. Except it's already installed?


    18) Run


    pip install python-daemon --upgrade
    apt-get install python-daemon --upgrade


    19) You'll also need to get another package installed, to fix an error. I can't find it on it's own but running :


    pip install distribute


    ...fixed the problem.


    20) Now run /etc/init.d/cloudprint start


    and you should be asked for your Google username/password....I can strongly recommend setting up 2 factor auth and application passwords, so you can disable the Cloudprint without changing all your Google passwords. But that's just me :0


    21) Then go back to the Google Cloud print page (https://www.google.com/cloudprint?hl=en-GB#jobs) and you should see your printer :)


    I've since installed a Google Android cloudprint app and can print to the house whilst on the move :)

    Lo all :)


    Would it be possible to "hide" any default plugins which are disabled?
    For example, "NFS" - I don't use it, but it takes up menu space....


    Just a thought :)


    Also, any news on a "howto" for plugin creation? (rather than reverse engineering?) :)

    Quote from "tekkbebe"

    If you have questions on Suid you might want to contact this forum member "ptruman". He is nice guy and will probably give you some advice.


    Thankyou tekkbebe :)


    I have not installed SquidGard, as I had some issues getting it to behave.
    What I have done is install Squid3 (I believe I built from source for some reason - probably to play with SSL_REWRITE, but can't recall/check from my present location!).


    I did (for a while) have all port 80/443 routes directed (from my router) back to OMV/Squid to ensure http traffic was force-proxied and nothing on my LAN could get out in an unauthorised fashion. However I had issues with that, particularly with SSL behaving properly when redirected, so it just runs as a "normal" squid.


    I'm happy to share my configs (which are mostly default, full RAM, no cache to disk, some ACLs)


    The only filtering I use is a combination of manual (but could be cron'd) adblocker updates via ACL, and OpenDNS (which would be free for an education establishment). That blocks most of the ads which annoy me, and prevents 90% of undesirable sites.


    I *have* enabled router based redirection of all TCP/UDP port 53 (DNS) traffic, and linked up OMV to use DNSCrypt (but you could just use normal DNS) so that all LAN originated queries go via OpenDNS, and no-one can get around my specified blocking that way.


    I've also used the OMV DNSMasq configs to use DHCP with option 252, which advertises a proxy auto-conf URL (served from the OMV website plugin!) for all browsers on the LAN. If you do the above with a firewall to only allow your proxy onto the web, you'll be most of the way there (all my devices can get onto to the web, but by default they'll use my advertised proxy and can only use the one DNS source).

    Lo all,


    I have 2 x 80GB SATA drives in a RAID1 pair running the OMV O/S.
    I then have 2 x 500GB SATA drives in another RAID1 pair running the data share.


    If I want to upgrade the 2 x 500 GB drives to something larger (1, 1.5 or 2TB drives) how much grief is that? Has someone done a HOW TO on removing one drive from a RAID pair and replacing another for rebuild? And once the rebuild is done on both disks, can you expand the RAID to use the new space?


    Enquiring minds etc... :)

    Here is a good one for you all :)


    My LAN is 192.168.1.* - and let's say my OMV server is 192.168.1.100


    I have OpenVPN installed on OMV, set to use 10.8.0.0
    I also has Asterisk installed, and LAN to LAN and LAN to internet VoIP is working fine.


    What I'm having problems with is VoIP traffic from OUTSIDE the LAN, when connected via VPN....(i.e. my Android phone)


    I can VPN in perfectly fine, and get an IP address like 10.8.0.6
    I can also connect to the Asterisk/OMV server, using 192.168.1.100 or 10.8.0.1 (which is the default OpenVPN IP for OMV when running).


    However, audio only lasts about 30 seconds, if that, before dying. I've amended keep-alives, and tried SIP proxying.


    I'm suspecting a route issue - as I can see this:


    Code
    root@MEDIAVAULT:/root # route
    Kernel IP routing table
    Destination Gateway Genmask Flags Metric Ref Use Iface
    10.8.0.2 * 255.255.255.255 UH 0 0 0 tun0
    10.8.0.0 10.8.0.2 255.255.255.0 UG 0 0 0 tun0
    192.168.1.0 * 255.255.255.0 U 0 0 0 eth0
    default 192.168.1.1 0.0.0.0 UG 0 0 0 eth0
    root@MEDIAVAULT:/root #


    Note above that the route seems to be for 10.8.0.2, or that the default route for 10.8.0.0 is 10.8.0.2 - NOT 10.8.0.1


    If I ping 10.8.0.2 I get nothing. If I ping 10.8.0.1 I get a response.


    If I run ifconfig on tun0 I get :


    Code
    root@MEDIAVAULT:/root # ifconfig tun0
    tun0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
    inet addr:10.8.0.1 :10.8.0.2 Mask:255.255.255.255
    UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1
    RX packets:371789 errors:0 dropped:0 overruns:0 frame:0
    TX packets:547723 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:100
    RX bytes:42990649 (40.9 MiB) TX bytes:543931143 (518.7 MiB)


    So the P-t-P IP is 10.8.0.2 - is that right? Do I need to add another route in to be safe?
    I can ping the VPN client when connected (and vice versa) - but something is being odd....(and it may well just be asterisk/latency...)

    Lo there


    Due to my stupidity, I have been fidding with /etc/samba/smb.conf and didn't back it up.
    Can someone post their smb.conf FIRST SECTION, i.e. the lines from (including):


    #======================= Global Settings =======================


    to (including) :


    #======================= Share Definitions =======================


    As something odd is going on with samba streaming for me that wasn't before... (stuttering)

    Righty ho, here is my mod on this, which works with sudo (if you didn't opt for an OMV root account)...


    Follow the apt-get instructions in the OP, and install any other packages you are prompted for (I was asked for one).


    If you (like me) used a DVD+RW instead of a plain CDR, you may find your device is /dev/sr0
    If not, modify the following to suit your setup...


    I modified my launch_abcde.sh to read :


    Code
    echo "Starting abcde..." > /tmp/abc.log
    sudo -u halevt /usr/bin/abcde -c /path/to/my/.abcde.conf -N -d /dev/sr0
    echo "Starting wind down..." >> /tmp/abc.log
    sudo -u halevt /bin/chmod -R a+w /path/to/my/OUTPUTDIR/*
    echo "Ending..." >> /tmp/abc.log


    The above just tells me wether or not a rip finished. Anything that barfs will stop "Ending..." appearing.


    I then ran visudo and added the following lines :


    Code
    halevt ALL=(ALL) NOPASSWD:/usr/bin/abcde -c /path/to/my/.abcde.conf -N -d /dev/sr0
    halevt ALL=(ALL) NOPASSWD:/bin/chmod -R a+w /path/to/my/OUTPUTDIR/*


    The above lets the halevt user (created when halevt is installed) run the sudo commands to call abcde AND chmod


    The chmod instruction is key (for me), as my rips are all stored in a NAS directory, accessed by me via Samba/CIFS. If I don't chmod the files, I can't move them from Windows, which is a PITA....


    I also had to grant the halevt user access to the cdrom group, otherwise when sudo ran abcde it couldn't read the disk.


    To do this :


    Code
    usermod -a -G cdrom,plugdev halevt


    You then need to ensure your .abcdef.conf file is public, otherwise halevt can't read it.


    That should sort you. It all works spotlessly for me.

    Erm, something isn't right with your install then.
    I have three options, and always have had.


    Unless it's something odd to do with a fresh .3 install, rather than .2->.3 which is what I did.


    Maybe log a bug on the actual plugin source pages :)

    Dammit, stupid forum ate my post!


    I am mr_pete on the old forum - I really should ask Volker to change my login name :)


    Simply change the "All Network Traffic" dropdown to "Local network" - and everything intended for your home LAN will go over the VPN - everything else, won't. Simples.

    Heh, I am mr_pete on the old forum. Should probably request Volker rename my account :)


    As you can see in your pic, you've selected "All Network Traffic" - so when OpenVPN starts, it sets up a route with high priority for all traffic, and shunts it over the OpenVPN interface.


    Technically this is good, as otherwise you can get "leaky" traffic (i.e. you try to visit http://www.gaming.com and your DNS request might go to your corporate LAN, or your web traffic via your work proxy - getting you (or your traffic) in trouble).


    If you look in /etc/openvpn/omv.conf file, you will probably see


    push "redirect-gateway def1 bypass-dhcp"


    or something similar - that is forcing all your traffic over the VPN, as intended.
    Change the "All Network Traffic" to "Local Network" or "Server Only" and ONLY your VPN LAN (i.e. home network) traffic will go over the VPN. Everything else will go "as it was".


    Obviously you need to ensure your VPN Client (Tunnelblick) doesn't clash with the server config :)

    Is the move to Wheezy going to happen via apt-get dist-upgrade? Or is this a re-install affair?
    (I've still not moved from Omnius to Fedaykin yet, primarily as I can't see clearly if all the plugins are migrated cleanly)