if you are just doing these syncs from time to time, plug the drive in, mount it on /mnt, and rsync the files to the destination. The mount and umount commands aren't hard.
I’ll learn it, thanks!
Posts by andyvs
-
-
Most people have gigabit ethernet which should comfortably do 100 MB/s. If you are connecting a usb hard drive (not ssd or nvme), it most likely can't sustain more than 180 MB/s. So, you aren't saving a ton of time by connecting it to the mac.
Really? This isn't an OMV problem. The exfat filesystem standard is maintained by Microsoft. Are you going to have them make exfat posix compatible and then implement the changes in the Linux kernel?
First of all, I have to say how awesome is the fact that you guys put up a system like this together and offer it to the community for free. So I’m not in a million years implying that you’re under any obligation whatsoever to do anything…just to be clear…
That said, I’ve been struggling with many little things like this because I want to make it work and for the most part I’ve been successful so far, even learning so little Linux as I do, which is not easy after the initial level even for a smart guy like me (and I know I am because my mother and my wife always told me so, which should count as irrefutable evidence… 😉).
E.g. I have a 2.5gbe connection. I can transmit files at about 45 mb/s back and forth. Thunderbolt connection, which I have been using (but insist in disconnect every time I turn the computer down, again, after stopping this behavior for a while …) get’s me to the 150 mb/s range…not near the advertised speed of at least 2000 m/s for reasons some of you guys explained to me in another post. But much better.
Yesterday It took me I think an hour to send 1tb of photos from the Mac to the server. This server has a 20gb USB port, that I assume (have yet to be able to test it, already ordered a suitable case to test with some ext4 formatted NVME) would get me over 1000 mb/s as there would be no middle men…
About been easy to implement it’s obviously out of my knowledge limits, I just assumed so because when I connect an exFAT drive to my Linux laptop it just works, but I can be completely wrong on this of course.
In the end, I understand that OMV, unlike other systems, is made also for the Linux-humble of us, so everything we can have from the UI is very useful and desirable, as it can save us from so much pain! 😊. And those who don’t need it can just look the other way…but I suspect there a lot of us out there!
🤙🏻
-
It still isn't a native Linux filesystem and doesn't support everything that a native linux filesystem does.
The whole point of a NAS is to not have to connect the drive to a system other than the NAS.
In a perfect world, yes. But in real life most of us do not have high speed ethernet, and when you have a terabyte or more of files to transport from a Mac to the server or vice-versa, that simple thing would be SO much easier. And besides, that shouldn´t be that difficult to implement, and would make life so much easier for a much larger audience...
-
It’s fully implemented in Linux for a long time…having it at least for reading external drives would make our lives so much easier for dealing with large amount of files from other systems like Mac…
-
Update. I discovered that if I shutdown the system (not reboot) and after turning it on again I disconnect the cable and connect it again on the MacBook, the network interface raises from the dead and all works again.
I´m sure it has a deeper meaning, but I have no idea what...any ideas?
Shutdown the server every time I reboot the Mac is not something I wanna do...
-
So, I installed a Thunderbolt card on my server running OMV, created a new ETH interface with a fixed IP adress, connected to my MacBook, and it just worked, making me super happy. I was able to transfer file 5x faster then through the ethernet 2.5gbe connection. Worked for hours. Then I rebooted my Mac…and the thunderbolt interface on the server vanished…when I look at the thunderbolt0 interface configuration, the device has disappeared...
I´ve been trying a lot of things I found online. One important hint I got is that maybe I had to re-authorize the interface. But it is authorized.
Checked with modprobe´s lsmod and both thunderbolt-net and thunderbolt modules are loaded.
BOLTCL shows:
boltctl info 19408780-00e3-286c-ffff-ffffffffffff
● GIGABYTE GC-MAPLE RIDGE
├─ type: host
├─ name: GC-MAPLE RIDGE
├─ vendor: GIGABYTE
├─ uuid: 19408780-00e3-286c-ffff-ffffffffffff
├─ dbus path: /org/freedesktop/bolt/devices/19408780_00e3_286c_ffff_ffffffffffff
├─ generation: USB4
├─ status: authorized
│ ├─ domain: 19408780-00e3-286c-ffff-ffffffffffff
│ ├─ parent: (null)
│ ├─ syspath: /sys/devices/pci0000:00/0000:00:1c.4/0000:06:00.0/0000:07:00.0/0000:08:00.0/domain0/0-0
│ └─ authflags: none
├─ authorized: Fri 09 Aug 2024 08:23:45 PM UTC
├─ connected: Fri 09 Aug 2024 08:23:45 PM UTC
└─ stored: no
Found this and created this file…
Those users who just want to connect any device without any sort of manual work can add following line to /etc/udev/rules.d/99-local.rules:
ACTION=="add", SUBSYSTEM=="thunderbolt", ATTR{authorized}=="0", ATTR{authorized}="1"
This will authorize all devices automatically when they appear. However, keep in mind that this bypasses the security levels and makes the system vulnerable to DMA attacks.
The card has two thunderbolt ports, so I tried changing the port and rebooting the server. It worked. A new TB interface showed, I edited the configuration to give the interface a new IP, it connected to my Mac, I was able to mount the SMB shares, but after a minute or so the shares disconnected and I was back to square zero.
One interesting thing is that when I’m looking at the dashboard, if I disconnect the cable from my Mac and connect it again, I see the spinning wheel for a moment, like it’s trying to load something, but nothing happens…
Any ideas? What and how do I have to load? Am I missing something?
-
Update on this: bought a thunderbolt card, installed, installed the Thunderbolt packages using the CLI, created a new Ethernet connection in OMV, connected a cable between the server and the MacBook…voilá! Just worked, and I’m getting 5x times the speed I was getting over 2.5gbe… couldn’t be happier…🤗 For editing photos and vídeo, no need to spend on 10gbe infrastructure. Perfect for my use case…
-
Display More
Not necessarily.
As an example, we have a large 1/2 Petabyte SAN implementing 4 spinning disc chassis totaling 64 Exos HDD's with a 24 SSD drive cache tier on top of it for increased bandwidth. It has Linux, Windows, and Mac's clients connected.
One of those clients is an M1 Ultra Mac Pro. It has a 100Gbps connection to our SAN via an Atto Fast Frame card. We were not seeing great speeds from the Atto card (around 1500MBps or 12,000Mbps), so as a test we swapped to an Atto thunderlink thunderbolt connected box with a 25Gbps connection and are getting 1200MBps or 9600Mbps. 1/4 the maximum connection bandwidth, but 3/4 of the speed reached on the 100Gbps connection.
Linux clients with the same 100Gbps connection can get 2 to 3 times the speed and Windows clients with the same 100Gbps connection are about 3/4 the speed.
This is all iscsi connection to minimize hardware overhead. None of it can saturate the 100Gbps connections. Linux might be able to saturate the 25Gbps connection, but nothing else can. At best it's more like about 40% of the connection bandwidth on other OS's.
iperf tests show much higher "normal" numbers through the network switches, and the math says that the storage chassis should be able to keep up to the bandwidth, but in this case the bottleneck is in the way the OS's are handling the connection.
There is more than just the bandwidth math to consider
😳
-
The point is, it doesn't matter how fast your connection is if there is something in the mix that can't keep up. It will only be as fast as the slowest link in the chain.
Totally agreed. But all that taken into consideration, upgrading my current setup from connecting my Mac to the OMV’s NVME storage with a 2.5gbe to 10gbe will upgrade the speed considerably. Doing so with a 40gbs thunderbolt connection will enhance it even more, to a potencial 4x, real world speed to be seen…and without the cost of a new switch…and the NAS sits right beside my Mac…
-
There is no reason you can't use thunderbolt. It just isn't configurable from the web interface. I would need hardware to try but it might not be hard to add to OMV.
As for the speed, most people don't have hardware that can keep up with faster than 10Gbe networking.
I´ll try to get a hand on such an add on board, if I do I'll report back. As for hardware, machines with thunderbolt connections are actually more prevalent then with 10gbe I think...most modern laptops are coming with it...
-
Never tried since I don't have two systems with thunderbolt.
Theoretically it would be the ideal way to connect a NAS for photo/video editing, since it would give maximum speed, 4x times the 10gbe NICs (using NVMEs on the NAS) while saving the need for 10gbe switches...would greatly enhance the use for OMV for these uses...some mainstream brands do work with thunderbolt, like QNAP, so I guess it´s doable...
-
Look for anything that works on Linux. I currently have a Solarflare SFC9020 but previously had Mellanox. At work. we use emulex primarily but also qlogic and intel.
Fine, so I'll look for Intel, there are plenty...now, something I can't find out about...will a direct connection between the OMV NAS and a Mac using thunderbolt interfaces work?
-
If you want to build something more expandable, I´d suggest looking for a motherboard with better connectivity like 10gbe on board, at least 2 pci express slots and more then one NVME slots, as well as as many data ports as possible...there are many NAS cases out there...a processor with video capabilities will save you a pci slot... I´d love if thunderbolt worked with OMV for direct connecting a Mac, but it doesn't apparently...
-
Hi all...anything that I should be known about 10gbe NICs, specially controllers that will or not work in OMV? I´ve seen Intel, Marvell...
Thanks!
-
You left the vm on the "default" network.
See the networking section here:
Now that I read it all I figured it out...it was the OMV network, not the VM...changed and all works...thanks!
-
You left the vm on the "default" network.
See the networking section here:
But it’s on Bridge…I’ll check the documentation anyways…
-
Hi all. Installed two VMs using KVM, one Windows and one PopOS. Both were setup using bridge network as can be seen in the image. None can see the local network…
Any ideas on how to solve this?
Thanks!
-
I suggest creating a new Raid (if you really need a Raid and you also have a backup), copy the data to the new Raid, modify the relative paths of the shared folders and undo the old Raid. Or clone the Raid on a standalone system or booting from a live USB.
On “if you really need a raid”…this is a good point…I do have an external backup disk connected to the OMV server, that backup everything every night using the Rsync plugin…that said, I’ve used RAID arrays for years with the rationale that I’m protecting from disk failures between two backups and reducing downtime in such fatalities…I’m using only two disks so redundancy is all I’m getting, no speed gains… so yes, sometimes I question if RAID is a necessity or just a habit…something most people do…gotta reflect on that…
-
My current configuration includes one NVME for system and virtual machines, one NVME for media and two 3tb in RAID 1. Everything backed up to an external drive.
About to replace the RAID with 8TB Hds. After reading many posts here, still in doubt about the best way and came up with this idea, does it sound right?
I have 4 sata ports, and the Raid drives use only two. I would connect the new drives to the other 2 ports, build a new array, then use the clone plugin to clone the old array to the new one. Remove the old disks, reboot and voila! Everything would be as it was with the new capacity…too good to be true?
One thing I’m not sure is how the system would handle the different disk’s UIDDs…or would the clone process also include that?
-
Display More
Hmmm - I did a quick test:
- On MacBook: connected smb-share on my omv-system containing a backup-copy of my Obsidian-Vault
- On MacBook: With Obsidian I opened this vault w/o any error message
So, at least on macOS it works to open a vault via smb-connected share.
But I cannot say if its working with iOS devices ...
Exactly, works perfectly on the MacBook, but not working on iThings, where I actually take most of my notes is a dealbreaker…
