Posts by merill

    Hi friends.


    I have a standard setup with OMV users/groups, OMV shares and CIFS shares. Each group has access to its share and everything goes fine. As easy and direct as it sounds.


    Now, I have been tasked to allow a given user (say Pete), NOT belonging to a given group (say testgroup), to access some of the folders inside that group's share (testshare). The user MUST NOT see anything else apart from those folders (say folder1, folder2) and the files inside them, being able to read and write them too.


    Well, this looks like a task for the ACL part of the GUI as I need to give diferentiated rights so some user. So I created a new group (extragroup), made the user (Pete) belong to it and gave the new group ACL RW rights to the folders. But even though the parent folder ('testshare') had RX rights for 'other' I was not able to see them from a windows session.


    So we have '/testshare/folder1' and /testshare/folder2' as full paths
    testshare rights are root->rwx, testgroup->rwx, other->r-x
    folder1 rights are root->rwx, testgroup->rwx, other->r-x, extended ACL extragroup->rwx
    folder2 rights are root->rwx, testgroup->rwx, other->r-x, extended ACL extragroup->rwx


    With this setup I am not able to see both folder1 and folder2 inside testshare. I expected to see testshare and, when expanded, both folders inside iit but I am not able to see testshare either. If I add the new group 'extragroup' to the original share via the 'privileges' button (not the ACL one) then I am able to see everything inside no matter what the POSIX rights are configured (I explicitly chowned nothing to other: other->---).


    Access Based Enum is active system-wide so even though all groups have r-x rights to all shares in POSIX they do not show at windows until they are explicitely given rights in the OMV share thus making them appear in the valid users' option of the smb.conf file.


    After losing lot of hair with this approach and not understanding why SAMBA did not show testshare as there were folders inside it where the user had enough rights to see them (perhaps because the @"extragroup" group does not show up in the 'valid users' list) I took a different approach. I created a new share for Pete, say testshare2 and gave Pete's group (extragroup) RW rights to it. This immediatly showed the new share in Pete's windows file manager as expected. Then I created soft links (via ssh session) to the original share folders (ln -s ../testshare/folder1 folder1, etc.). I expected that as the 'extragroup' group had RW rights ACL for /testshare/folderN, I could access the files indirectly as I had enough rights to.


    Well, this is not true. I get an error when I try to open those linked folders.


    Do anyone knows why? Can someone, please, give some light here?


    Thanks in advance for your help.


    Regards.

    I have read your post about building a plugin and it is quite helpful. Unfortunately it is not mentioned anywhere how you can survive to the inevitable mistakes that one makes when coding.


    Perhaps this is the not right place to ask such a dumb novice question but I am sure you have already resolved this time ago. Sorry if thi is not the right place to do this. Please tell me where I must go if this is the case.


    So, how do you debug the plugin? I mean, how dou you 'print_r' or whatever the value of a variable or object? May be it is evident but I have not been able to figure it out so far.



    I am extending the borg plugin. One of the things that I am now with is executing a call to an external program and getting (supposedly) a result back in JSON format. Then I decode it with


    $output = json_decode($output,true);


    Now I expect to have an array with the values returned in $output. The same call by hand from the shell return this JSON result:


    root@openmediavault:~# borg info --json xxxxx@xxxxxxxxxx.repo.borgbase.com:repo
    {
    "cache": {
    "path": "/root/.cache/borg/bla-bla-bla",
    "stats": {
    "total_chunks": 27260,
    "total_csize": 9056317707,
    "total_size": 10897350512,
    "total_unique_chunks": 2921,
    "unique_csize": 975903007,
    "unique_size": 1180942311
    }
    },
    "encryption": {
    "keyfile": "/root/.config/borg/keys/xxxxx_borgbase_com__repo",
    "mode": "keyfile"
    },
    "repository": {
    "id": "e7b91xxxxxx",
    "last_modified": "2019-11-09T03:38:40.000000",
    "location": "ssh://xxxxx@xxxxxxx.repo.borgbase.com/./repo"
    },
    "security_dir": "/root/.config/borg/security/exxxxxxxxxxxxxxxx"
    }



    Now I need to check if I have something returned because I am noy getting anything when I use $output["cache"]["stats"]["total_csize"] to send the data to the front-end. If I use a constant of rand() function I get the value so it is not the renderer itself so it must be the content of $output variable.


    Something is going bad but I do not know what is so I need a way to show the contents of the variables to be able to fix it.


    Any ideas?


    Thanks in advance.

    When enumerating the repos, a second call to borgbackup info /path/to/repo --json

    Oh. I understand now. I didn't catch that you were refering to the borg call. Yes receiving the reply in JSON format will make parsing it a lot easier. The info button may also be the best option depending of the mean time that a call to borg to get it will take (not the same for a local or a remote repo)


    The performance of this idea would degrade quickly the larger the number of archives in a repo and the larger the number of repos

    Completely agree. There is no special reason to select this mechanism (a grid or table) except it is the traditional way that backup front-ends present the history of finished jobs. Anyway reading any of them just to fill the drop-down will take time too so I also do not see any gain of having them in that kind of control versus a collapsable grid/panel/table. The expected performace should be the same.


    I am not proposing to present all the panes for all the repos expanded. The idea is to only, initially, show the list of repos (that we have in the local config file) the same that it is done in the repos tab and read, fill and present the list of archives of a given repo only when requested by the user by means of the expand icon (much like the dropdown).


    We probably we are speaking about the same. In fact this could be marged with the repos tab but having its own tab makes things a bit more ordered. We already have it in the listing button. The only thing with this ist is that it is difficult to read. The proposed archives tab is a merge of the repos+list button with some fancy view.


    And as for code changes, I don't want to touch the 4.x plugin

    I understand. I do not know if porting to next version is too hard or not. Taking into account that I am not able neither to display a single constant in the panel I doubt that I could be of any help with that but I will keep trying a bit more to make this work in 4.x if only for not saying that I surrendered too early. Though, for now, OMV is goaling me out by far. :(

    Hello back ryecoaaron. Thanks for your reply and sorry for this being a looong one.


    Quote from ryecoaaron

    So, forgive me when I am touchy on this one.

    You are plenty forgiven. Believe me :-)


    Quote from ryeccoaaron

    When originally creating this plugin (had no feedback or help), everything seemed fine since I had very small samples. From a code perspective, it would make sense (to me) to have another mount dialog on the repos tab that would you to select an archive and shared folder.

    And you are quite right. It had a lot of sense and more if you had no data where to get an idea about the working of the tool from. I did not even kow about this tool a few months ago but having the chance to have an efficient backup storage at the price that the Hetzner Storage Boxes are (2TB for less than 10€/month) made me focus on this. I have to admit that it may not be the best backup tool that may be around but it is available from that cloud storage provider so.... And I think that if we are able to take this tool to the next step it may be quite useful for many others.


    Regarding the option to mount a given archive I think that option should be in its own tab, one that shows a listing of the existing backups (much the same as the one you have right now in the repos tab) where the user can see all the archives that he/she already has in the repo with it's stats (size, compressed size and deduped size), can select one and mount/umount it. As I am doing it now (by hand) I mount the repo in a given mount point (that could be requested in the mount dialog or configured in the jobs tab near he job definition. More on this later)


    That is probably easier to do now with the --json flag. I don't think that existed a couple of years ago. It would be nice to put those in the repo grid but it maybe too slow. I will have to try it I guess.

    I swear I have tried all that I have imagined to show this data without success. And it loos like it should be not difficult. I am adding


    objectv->add('reposize', 'integer', 100); (and similar lines for the other fields that have been defined in the corresponding file just below the 'mounted' field)


    in the getRepoList funcion inside the foreach loop. This should be enough to push the data into the field but I get columns with '0' inside or with '--' if I use a render function.


    i do not know if the --json flag could make any difference as I even do not know where to use it. I even have not found a way to debug (show variable's contents) nor in the client area nor in the console log area. And have not found any info about how to achieve this either, so my debugging capabilities are very limited rigt now. How do you do it, man?


    You are right that getting the actual data from the repo using borg info may be slow. If it could be an asynchronous call that would be great. If not it would have to be decided if it is better to know how many space you are using out of your repo even if you have to wait some time or not. Prehaps an 'info' button giving this data would be enough if it takes too long to get that data on-line.



    Just to warn you that even though I am porting it, I didn't want to rewrite the whole thing

    Fully understandable. I have managed to make the changes for the Archives tab (and ALL related elements as the structure in the config.xml file, etc.) to be named Jobs. Now ALL references to archive (unless those that make sense like getArchiveListing) are changed to Job (or Jobs depending the number). If you want I can send you all the files (OMV 4.x) that make this setup.


    I have creted an skeleton for the Archive tab too and started to change the repo code to insert (as told before) the repo stats but I am stuck on both tasks.


    The Archive tab as I foresee it should have a list of the repos (the same as the repo tab but with only the name and encryption fields) and, for each repo, a collapsable listing of all the archives that it holds (this could be very similar to what the List button offer now). Thus you get a complete view of your saved data in a single pane. Selecting one of the archives in one of the repos will enable the mount button and you can mount it in a given mount point. It could be fine if we could add a delete button too to drop a ginven archive from the repo.


    Now, after mounting by hand an archive of a repo, I soft-link (ln -s) the inner folder of the mounted archive (borg has the bad use to create a full tree folder structure with the name of the device and many other things before getting to the saved data. You cannot give that to the user because he/she does not know anything about how your disk device is named and alike. So I soft link this to a share that I have created for this, that I then publish to the user that have to get access to the backup. Now he/she has an entry in his windows explorer with the data where it can be restored the needed files. Easy for them, difficult for us as system admins. The old story of our lives...


    So in short, your points are correct but they may need some fancy touch just to take some of the wierdness of the tool out of the way :-)


    And thanks for honouring me with your reply and interest. :thumbup:


    Kind regards.

    I vote for this too. I have a remote windows server that I am offering to the local users as NFS shares and without this plugin it would be a lot more complicated to do this.


    Another use is to connect to a remote repository to be used as target bor backups so it is quite useful.

    Hi Ryecoaaron. Thanks for your reply.


    I am not complaining about your job. At all. Full the contrary I am quite grateful that you did it (even when you are not using it).


    I also know that the specific naming used does not affect the way the plugin works. It is only that, though an achive can be a grouping of other archives, in this case what we are configuring at the 'Archives' tab are not archives themselves but the jobs to create the archives. Probably semantics only that would not go any further if we could handle the archives themselves and their contents in a more granular way somewhere else.


    Mounting a full repository can be a mess. You may have tens or hundreds of archives inside a given repo and you end up with a mess of folders at the mount point that you cannot point the final user to. This is why I suggest to have a way to mount a single archive and, for that task, an archive tab would be fine.


    As for the more features, that is why I told that I was trying with the code. But I am an absolute nerd both regarding the framework used (I use a different one) and the object model developed in OMV (I have read the docs I have been able to find but even that no way man. Probably I'm getting old...).


    That is why I humbly request your help. I am not requesting that anyone else (like you that even does not use it) take all the task over his or her shoulders but, at least, help me with it until I get enough knowledge to go alone. I am starting with the simplest action that I can imagine that is to get the full, compressed and deduped size of the repo and present it in the repos screen and I am unable to do even this easy task (I am NOT making the call to borg right know just filling up an array and using it later but the numbers in the array do not get their way to the screen so far :-( ) so I really need your help and I humbly request it if you can, and want, to.


    And, in the meantime, as you are the one who are porting it, it would be fine if we could agree over the final naming and deployment now that it is being ported to OMV5 to build a better plugin for this service. I am actually using it and it would be a great releaf if we could handle the individual archives from the plugin and not from the command line.


    Anyway thanks again for your time and hard work.

    Have you managed to configure your borg repo so far?


    If still not you may want to know that there is an issue with the plugin and all borg repos that connect via ssh. The first time you connect to the repo they send back an ECDSA request for verification that the plugin does not handle and does not expect that make all fail.


    If this is your case, bypassing this is as simple as first creating the repo and then, before setting it up with the plugin (in fact issuing an init command to the repo), making any kind of ssh connection to the repo from the ssh console of the OMV host just to receive the verification request that you must accept by hand. It does not mind if the ssh command fails or not. You get the remote host validated in your OMV host and now you can register your repo without any further issues.


    As told I do not know if you have managed to resolve your issue but just in case here you have what I had to do to be able to register mine. Hope this help.


    Regards.

    Hi all folks.


    Looking desperately for a mechanism to save the data handled by OMV in Hetzner Storage Boxes (tried everything I could think of including rsync scripts but as no SSH commands no easy way to handle retention policies) I ended installing and using BorgBackup plugin that, at least, is doing the job.


    But the plugin falls short under my point of view. Yes, we can configure repos and launch backup tasks, but we have no other kind of information about the repos. For example we do not know how much space we are using in the repos, nor the compressed and deduped size they are taking. And lot of stuff must be done from the command line (mounting a given archive not the full repo for example).


    Regarding the actual 'Archives' tab, I think this shouls be named Jobs or Tasks because it defines the jobs (or taks) to get done not the archives that are created. The actual archives are listed when you click the 'List' button not when you go the the archives tab.


    And there should exist a new 'Archives' tab that listed all the archives that we have in each repo with its original, compressed and deduped size (at the very least) as we get from the info borg query. This listing should be in a collapsable panel hanging over the corresponding repo entry (the repo they belong to). And it is here where we should have the 'mount' command to mount a given archive and not the whole repo (the full mount still can be useful nonetheless) .


    So I tried to achieve something of this by myself. I managed to rename the 'Archives' tab (and all the related code) and have a functional 'Jobs' tab. This was easy as you only have to go to all the related files that someone else's has already worked out (and they are only in two places) so nothing to be proud of.


    Now comes the difficult part that is to build a whole new tab with the actual archives info. And this is where I am stuck. I do not know enough ExtJS nor OMV programming model to know how to make this collapsable listing nor to make the borg calls nor to fill up the data structures to hold the data to be presented. Mostly this should be what the 'List' button does for a repo but presented in its own tab.


    So I request your help. I have seen that you folks are quite helpful and supportive so may be that it could be possible to improve this plugin and make it truly useful to make it easy for an admin to publish an archive contents to a SMB share without needing to go SSH.


    The attached image shows some of the former concepts I have been talking about.


    And if this is not possible or of any interest for anyone thanks anyway for your time and for having developed such a good product for all of us.



    Miguel.


    borg_plugin.png