openmediavault-s3: Minio have removed support for existing files

1st Official Post

    I set up openmediavault-s3 today and configured it up to use an existing shared folder.


    I then visited the Minio admin page and started configuring access.


    Very few of the folders actually appeared in the output of aws s3 ls and I wasn't able to find a way to force Minio to scan my files.


    This is because the Minio developers have removed support for existing files since openmediavault-s3 was implemented.


    Deprecation of the MinIO gateway
    MinIO is deprecating the gateway and will be completely removed in six months. This should not come as a surprise, we began informing the community in 2020 and…
    blog.min.io

    Feature request: bring back the legacy FS backend · Issue #15455 · minio/minio
    Is your feature request related to a problem? Please describe. Minio should support serving a simple filesystem directory over the S3 API and via the console.…
    github.com

    Mount existing folders with files to minio · Discussion #15496 · minio/minio
    Hello, i run docker-compose and share volumes with files to minio contanier. Then when i connect to aws, my bucket is empty. How mount existing files to minio?
    github.com


    The changes mean that openmediavault-s3 is only a one way link between Minio's S3-compatible API and the shared folder, whereas before it was a two way link.


    The changes will also make it awkward if any of the minio metadata gets corrupted and you need to reinstall minio. You then need to re-upload all your data.


    I can see why the Minio developers have made this change, but it definitely impacts the usefulness for self-hosters with a single filesystem.

    • Official Post

    I think you are confused what the minio gateway does. If you were using minio for s3 object storage, you won't see any change. The minio gateway provided access to minio storage via other protocols like nfs. The gateway deprecation has been coming for months.


    The shared folder you select in the plugin is where minio stores its files.

    omv 6.3.4-1 Shaitan | 64 bit | 6.1 proxmox kernel

    plugins :: omvextrasorg 6.1.1 | kvm 6.2.9 | compose 6.6.1 | cputemp 6.1.3 | mergerfs 6.3.5 | zfs 6.0.12


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    I might be, but if you take a look at this link: https://github.com/minio/minio/issues/15455, in minio RELEASE.2022-05-26T05-48-41Z and before, it looks like if you created files in the filesystem (the OMV shared folder), you then see those files via the S3-compatible API.


    Now, this no longer works (tested today). The only way to add files, and see them in the S3-compatible API is to upload them via the S3-compatible API.

    • Official Post
    Quote from johncant

    I might be, but if you take a look at this link: https://github.com/minio/minio/issues/15455, in minio RELEASE.2022-05-26T05-48-41Z and before, it looks like if you created files in the filesystem (the OMV shared folder), you then see those files via the S3-compatible API.


    Now, this no longer works (tested today). The only way to add files, and see them in the S3-compatible API is to upload them via the S3-compatible API.

    You are right. That was a feature of the gateway. I didn't know it existed and I didn't read your initial post as what you were doing. As a minio admin at work, I don't think files should be able to be injected into a bucket by adding them to the filesystem. It makes sense to me that an S3 server would only be able to add files via S3. But there isn't anything OMV can do about this since minio removed it. I am working on adding s3fs support to the remotemount plugin that would effectively add this functionality back.

    omv 6.3.4-1 Shaitan | 64 bit | 6.1 proxmox kernel

    plugins :: omvextrasorg 6.1.1 | kvm 6.2.9 | compose 6.6.1 | cputemp 6.1.3 | mergerfs 6.3.5 | zfs 6.0.12


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    Quote from votdev

    È ancora possibile utilizzare il plug-in, ma non è più possibile accedere ai file esistenti del file system tramite S3. La cartella condivisa ora viene utilizzata solo per archiviare i metadati MinIO.

    Then there is another problem, because I activated the service through the plugin, but the page is unreachable

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!

Register Yourself Login