Copying or moving files from one Storage disk to another

    • OMV 3.x
    • Copying or moving files from one Storage disk to another

      If I want to copy or move files from one storage device to another, in order to be able to unmount a disk without losing the data, I would like to know if the files move directly from one disk to the other, or the data has to pass up to the router first, and then back down the connection to the second disk? Really, I guess what i want to know is how efficient and fast is it to move those files?

      If I move all the data off one disk and unmount it, then can I mount it again with a different name? Any tricks to doing this?

      Thanks,
      Michael
    • Depends on how you do it. If you do it over smb shares, then yes. If you do it local.. no

      Easiest way to do what you're asking is either..

      1. Lean to use the command line "cp" tool (copy).

      2. Easier.. especially if you're bit UNcomfortable with command line... Set up a simple rsync job. Your data you want to copy is on "Drive A", create a shared folder on "Drive B" to copy said data to. Then create an rsync job using "drive a" as a source, and "drive b" as the destination
      Air Conditioners are a lot like PC's... They work great until you open Windows.

      The post was edited 2 times, last by KM0201: Clarity ().

    • Thanks, I am not very adept at using Linux. Both of the drives are shared over my network via Windows. I have SMB1 implemented in Windows. I thought I could just select the files there and copy them to the other hard drive.

      When you say to use rsynch, I don't know if this is done through the OMV web control panel, or if I have to do it by accessing the root folder in OMV.
    • Right. Okay, do I restart the machine with a live CD or USB with a Linux OS and use the tools on the disk, or is this something that I can do within the OMV OS? As I said, I am not very familiar with using Linux. I thought I could do this from Windows, copying files from one disk/folder to another,...but I don't hear anyone confirming that I should be able to do that.

      Edit: Here is what I am trying to understand--as a User with 30 years of experience with windows, who has only occasionally used Linux tools. it seems to me I should be able to do this the way I would normally--using Windows networking and the web interface. Within a Networking window I would copy or move the files to a different hard drive in the NAS (I'm not using Raid, each drive has it's own file system). Then I would wipe the drive in the web interface under Physical disks, and proceed to set it up again with a new file system and mount it with a new name. Then I could move the files back to the new drive setup.

      I'm not sure if there is some reason I cannot do it this way, or if those who are giving me advice are just so used to using Linux, that the suggestions I am being given are all based on Linux tools.

      The post was edited 1 time, last by MPPurcell ().

    • If you copy "remotely", for instance from a Windows box, then the files have to travel back and forth on the network. That can be slow. But it should work perfectly fine.

      If you login to your NAS using SSH then you can copy "locally", by using cp, rsync or Midnight Commander. That can be much faster.
      OMV 4: 9 x Odroid HC2 + 1 x Odroid HC1 + 1 x Raspberry Pi 4
    • Okay, thanks. Where do I find instructions on how to log in to my NAS using SSH? I suppose I could also restart the NAS with a CD with a linux distribution which has Midnight Commander on it, and move the files that way. Could I also rename the file system that way, and not have to wipe the drive in the NAS web interface?

      I am using the NAS from Windows normally, and all the drives are shared there, so I guess I could do the moving of files from the Windows shares. I get that it might be slow. Maybe also more prone to error....
    • Can anyone help me with choosing between these three choices?

      1. Log into my NAS using SSH--Where do I find instructions how to do that?
      2. Simply restart the NAS with a Linux live CD and use a tool like Midnight Commander to rename two of my storage disks (ext4 file systems) and copy files from one drive to the other.
      3. Use Windows, which is most familiar to me, to move the files from one shared drive to the other; then go into the web interface of the NAS, wipe the drive, and remount it with a new file system name.

      Are there any issues with any of these working (I know Windows might be slow, it's just the most familiar to me.)
    • MPPurcell wrote:

      Log into my NAS using SSH--Where do I find instructions how to do that?
      Google? I recommend putty. Put the server's hostname or ip address in the hostname box. It will ask for a username and password. Either use root with the password you specified at install time or a user you create from the OMV web interface that is in the ssh group.

      MPPurcell wrote:

      Simply restart the NAS with a Linux live CD and use a tool like Midnight Commander to rename two of my storage disks (ext4 file systems) and copy files from one drive to the other.
      You would have to mount them both and then copy files. Might as well do step 1 since it would be the same commands after mounting.

      MPPurcell wrote:

      Use Windows, which is most familiar to me, to move the files from one shared drive to the other; then go into the web interface of the NAS, wipe the drive, and remount it with a new file system name.
      This is obviously the one you want to use and you already know it is slow. If you are ok with that, use this method. It is more than just wiping the drive though. You will have to remove the shared folder(s) for that drive from all services, then delete the shared folders for that drive, then unmount it, then wipe it.
      omv 5.0.14 usul | 64 bit | 5.0 proxmox kernel | omvextrasorg 5.1.5
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • Okay, I am ready to say "Uncle". I was able to move all but one file (had too many characters in the folder string for Windows), but the transfer was really, really, really, REALLY, REALLY SLOOOOW! I am now Googling Putty, and I'll see if I can figure how to do the other drive that way. This was the smaller data drive.
    • MPPurcell wrote:

      Okay, I am ready to say "Uncle". I was able to move all but one file (had too many characters in the folder string for Windows), but the transfer was really, really, really, REALLY, REALLY SLOOOOW! I am now Googling Putty, and I'll see if I can figure how to do the other drive that way. This was the smaller data drive.

      That's what everyone told you, that it was going to be slow.. because essentially what you would be doing is..

      Nas Share 1 > Computer> Nas Share 2... It's going to travel over your network twice, just to end up on the same machine... of course it's going to be slow.

      You're really making a mountain out of a mole hill on this.... While learning to use Putty and the cp command is certainly fine... rsync is built right into the omv webUI and will literally accomplish what you're asking in a couple of clicks... (Edit: I just realized in my previous post I put "if you're a bit COMFORTABLE with command line"... that should be "if you're a bit UNCOMFORTABLE with command line."... Sometimes auto-correct isn't near as smart as it thinks it is.)
      Air Conditioners are a lot like PC's... They work great until you open Windows.

      The post was edited 1 time, last by KM0201 ().

    • It seems the forum server was down for part of the weekend, so I wasn't able to get back here till last night.

      I apologize both for this being long, and that it may sound critical of some. I tried very hard to explain that I have lots of experience with computers and Windows, but little with linux or OMV; also, I hoped for reference to where to look for what was being suggested. I tried to follow the suggestions I could understand, but ran into roadblocks.

      I'd like to review some previous posts, and why I think many well-intended comments really were not so helpful to me. It might help the commenters to understand what someone like me really needs to know. If you want to cut to the chase and avoid reading most of this, you could look at the last 6 or 7 paragraphs.

      Essentially, my basic questions were about moving files, and "...If I move all the data off one disk and unmount it, then can I mount it again with a different name?..."

      I appreciated most of KM0201' responses--however, since I don't know the territory, many of the general pointers were just too general (applies also to others.) For example:

      "Depends on how you do it. If you do it over smb shares, then yes. If you do it local.. no

      "Easiest way to do what you're asking is either..

      "1. Learn to use the command line "cp" tool (copy).

      "2. Easier.. especially if you're bit UNcomfortable with command line... Set up a simple rsync job. Your data you want to copy is on "Drive A", create a shared folder on "Drive B" to copy said data to. Then create an rsync job using "drive a" as a source, and "drive b" as the destination."

      These are great general pointers, but since I'm not at all familiar with the tools, I am sort of swimming here. I replied:
      "I am not very adept at using Linux.... When you say to use rsynch, I don't know if this is done through the OMV web control panel, or if I have to do it by accessing the root folder in OMV." Fortunately, KM0201 clarified this in his latest post.

      gderf wrote: "Midnight Commander could also be used for this." Since I have only slight familiarity with MC, I sort of got the idea, but I was still swimming about what to do.

      I asked:
      "...do I restart the machine with a live CD or USB with a Linux OS and use the tools on the disk, or is this something that I can do within the OMV OS?..."

      Adobe was helpful in answering my first question, but I only learned from experience just how slow this would be:
      "...If you copy "remotely", for instance from a Windows box, then the files have to travel back and forth on the network. That can be slow. But it should work perfectly fine." [Not sure I would say, "fine".]

      Adobe added: "...If you login to your NAS using SSH then you can copy "locally", by using cp, rsync or Midnight Commander. That can be much faster."

      Okay, this sounds helpful, but there is no direction how to start or set up any of these options. I have never used any of them, and have only a basic familiarity with the OMV interface.

      So I asked:
      "...Where do I find instructions on how to log in to my NAS using SSH? I suppose I could also restart the NAS with a CD with a linux distribution which has Midnight Commander on it, and move the files that way. Could I also rename the file system that way, and not have to wipe the drive in the NAS web interface?"


      ryecoaaron wrote:
      "...I recommend putty. Put the server's hostname or ip address in the hostname box. It will ask for a username and password. Either use root with the password you specified at install time or a user you create from the OMV web interface that is in the ssh group."

      Okay, fair enough. I looked up Putty, read about it, and downloaded and installed it. However, once I got to the NAS (via IP address), my login and password would not work. I tried both the root, and the web interface passwords. I don't know why it would not let me in. As to creating another User from the web interface in the SSH group, these are the settings I found under SSH:

      SSH settings in web interface:
      Enable [Enabled]
      Permit root login [Enabled]
      Password authentication [Enabled]
      Public key authentication [Enabled]
      TCP forwarding [Not Enabled]
      Compression [Not Enabled]
      Extra options [Blank]

      So, I found no setting for creating a special User in the SSH group.

      In the latest post, KM0201 wrote:
      "...You're really making a mountain out of a mole hill on this...." [Okay, okay, I can see why it seems that way to you, but how much experience do you have using these tools? (Linux and OMV?) It makes a big difference without have that base, how I can even understand what you are referring to!] "...While learning to use Putty and the cp command is certainly fine... rsync is built right into the omv webUI and will literally accomplish what you're asking in a couple of clicks..."

      Okay, maybe we have finally come around to what could have made this a lot easier for me. Thank you, KM0201! I see now that there is an option to enable Rsync, and there is something called Jobs, and also Server. Where to I find instruction what to do with these--and how to move files between disks? I take it this would all happen within the server, and be much faster than using Windows networking.

      Also, I see under File Systems that the Labels of the File Systems are there, but I see no way to change the label, and the "Unmount" option is greyed out. So I suppose I could go to Physical Disks after moving the files, Wipe the disk, then remount it with the new name I want. This looks like it could all be done within the web interface. Is this correct? (If so, this is pretty much what I was going for in the first place.)
    • Okay, in trying to use Putty, I found a thread at

      Default SSH password "openmediavault" not work (OMV_3_0_88_RaspberryPi_2_3_4.9.41.img)

      with a post by KM0201 (#7) on how to create a User for sign in to SSH. Without further direction, I assumed this would all be done under SSH. I see now that I need to first create a new user under User. (A step that wasn't mentioned above.) Thanks to KM0201 for the clarity here.

      1. Create a user under the "users" section of the webUI (remember usernames are case sensitive)
      2. Choose your shell (I prefer /bin/bash)
      2. In the user creation window, click the Groups tab
      3. Make sure "SSH" is checked
      4. Save and apply settings for the new user
      5. SSH the server as your new username and enter the password when prompted
      6. Once logged in su -
      7. Enter root password
      8. You should now have a root prompt.
      9. passwd
      10. Enter and confirm new root password


      However, I confess to being confused by items #6-#10. I don't understand why once singed in to SSH, why I would need to mess with the root password, and why create a new root password? Do I use this in future to sign in to the text interface if I start the NAS and want to do something with keyboard and monitor?
    • I never set a password for root.

      Instead I create a user and add that user to groups ssh and sudo. Then I can use that user over SSH and run commands as root by preceding them with "sudo". I've gotten used to that from using Ubuntu a lot.

      sudo apt update
      sudo apt upgrade
      sudo apt install mc

      Sometimes I run mc (Midnight Commander) with root privileges. Don't tell anyone....

      sudo mc

      Regarding rsync. It is a crazy powerful tool. But easy to use. Make sure you you know what you want to copy and where to. Source and destination. That you know the paths. Then just run:

      rsync -a source destination

      After a while you have a copy of source at the destination, complete with all subfolders and files. Read up on rsync. It can do a lot. Also copy over SSH between servers.

      Another very cool tool is the FISH filesystem. It was "invented" for mc. It allows mc to remote execute commands over the network and transfer files between servers over SSH. It makes it possible for me to run mc with one pane in my Linux laptop and one pane in one of my HC2. Over SSH. And I can freely copy files between the panes. Not quite as fast as over NFS or using rsync. But a great way to copy over sets of configuration and see what I'm doing.

      Sometimes you do very big jobs using the command line or mc. If you close the terminal window the job terminates. To avoid that you can use the screen utility. You start screen and then start your job. You can then close the terminal window and the job continues executing. While I write this I have a rsync job running in a screen. It copies three folders from one server to another. Around 9 TB. Should be finished in the morning. I can connect using SSH and reconnect to the running job with the command screen -r to see if it is finished.

      Everything I mentioned above is documented to death on the internet. Tutorials galore. You need to learn how to find and use that documentation.
      OMV 4: 9 x Odroid HC2 + 1 x Odroid HC1 + 1 x Raspberry Pi 4

      The post was edited 2 times, last by Adoby ().

    • MPPurcell wrote:

      Okay, in trying to use Putty, I found a thread at

      Default SSH password "openmediavault" not work (OMV_3_0_88_RaspberryPi_2_3_4.9.41.img)

      with a post by KM0201 (#7) on how to create a User for sign in to SSH. Without further direction, I assumed this would all be done under SSH. I see now that I need to first create a new user under User. (A step that wasn't mentioned above.) Thanks to KM0201 for the clarity here.

      1. Create a user under the "users" section of the webUI (remember usernames are case sensitive)
      2. Choose your shell (I prefer /bin/bash)
      2. In the user creation window, click the Groups tab
      3. Make sure "SSH" is checked
      4. Save and apply settings for the new user
      5. SSH the server as your new username and enter the password when prompted
      6. Once logged in su -
      7. Enter root password
      8. You should now have a root prompt.
      9. passwd
      10. Enter and confirm new root password

      However, I confess to being confused by items #6-#10. I don't understand why once singed in to SSH, why I would need to mess with the root password, and why create a new root password? Do I use this in future to sign in to the text interface if I start the NAS and want to do something with keyboard and monitor?

      To put it simply, you don't. I didn''t read the post I made, but if I suggested that it was most likely because I don't like to SSH as root due to security concerns. It's infinitely more secure to log in as a user, and then change to root (this was partly how Heartbleed and it's variants could potentially cause such damage). I always disable root SSH access, and generally recommend it if anyone asks me. Leaving the root password as the default is just as bad, which is probably why I suggested to change it.

      That said, for what you're wanting to do, there really is no really is no reason to be root just to copy/move files.
      Air Conditioners are a lot like PC's... They work great until you open Windows.

      The post was edited 2 times, last by KM0201 ().

    • Okay, since I found your explanation (and your clarification here) I may try to use Putty again, to at least understand how it works, but my preference as stated at the end of my longer post, is to see if I can use rsynch, as you suggested. I will try to focus on that and see what I run into.
    • Sorry for the delay in getting back to this--I've been occupied with other things.

      @macom -- Thanks for your link. Yes, I saw and used your file extensively in learning about and setting up OMV. The PDF I had is slightly smaller, so there may be some changes/additions in the version you linked. I took another look at it.

      UPDATE TO THIS THREAD: I struggled with the many suggestions I've been given previously re: copying files within the NAS. I was about to give up on most of them, then I found a reference on OMV and rsync that gave me enough info to be able to use it. I think having worked almost exclusively with GUI interfaces for 25 years made it difficult for me to follow what to do with many of the tools suggested, even with a lot of Googling. Not that I don't believe command line can be useful--I was actually quite adept with DOS some 25-30 years ago. But on Unix/Linux, there is just too much for this old dog to learn now, I guess--it may seem easy to those who know the territory, but that's not a universal. I even tried restarting the machine with an old Live DVD of Ubuntu (11), that had graphical file moving tools. Unfortunately, although this works fine with Windows volumes, in OMV, Ubuntu would not let me mount the DATA drives. The OMV OS drive--no problem, but there must be some proprietary limitation that won't allow the data drives to mount. This would have been a nice, easy graphical interface for me to use to move files within the OMV--but no dice!

      The article I found most useful in setting up rsync on OMV is this one, which almost made me ready to spring for a QNAP machine:

      ridwankhan.com/building-an-ope…figuring-omv-ee15322602be

      Note the Intro to the section on this issue of moving files:

      "Moving files on your NAS

      The other thing that threw me for a loop using OMV was file transfers on the NAS itself. On my QNAP machine, the system included a GUI file browser, similar to Windows Explorer or Finder, and it was relatively easy to move files. OMV — at least out of the box — doesn’t include a GUI file browser.

      Therefore, realtalk — it can be harder to move files on your NAS than it would be on your desktop computer or a QNAP system. That said, once you get the basic idea, it’s not too hard, and OMV’s rsync is a powerful tool."

      ...So, it would be nice if OMV could some day provide a GUI for moving files between the storage disks. I was finally able to set it up and copy some files. I am now stuck about the best way to delete files and shared folders on the disk I want to re-Label. I have already asked that question in a second post,--hoping I will get a reply there:

      Once again, changing File System label--"Unmount" Grayed Out