Nextcloud AIO with Nginx Proxy Manager and fail2ban: I keep getting banned by fail2ban

Zitat von DonkeeeyKong

Hello,

I have Nexctloud AIO set up with Nginx Proxy Manager and fail2ban. Big thanks to chente and BernH for the awesome guides (Nextcloud AIO and NPM with fail2ban). I did everything according to the guides. The setup was quite easy and it's running very nicely.

If I understand BernH 's instructions and the regex file correctly, his guide sets a fail2ban filter, that scans for all HTTP status codes from the 3xx and 4xx range in the npm log files.

Before I get to my actual problem, I have one question: Why is it necessary to include the 3xx codes?

One example: Apparently the (preinstalled) Nextcloud richdocuments-app checks for custom fonts every few minutes or so - and most of the time receives a 304 code ( [12/Mar/2024:23:48:52 +0100] - 304 304 - GET https mynextcloud.xxx "/apps/richdocuments/settings/fonts.json" [Client 123.456.789.0] [Length 0] [Gzip -] [Sent-to 192.168.178.15] "COOLWSD HTTP Agent 23.05.9.2" "-") -> in the last 24 hours alone there are more than 300 events like that in the npm log - all registering my IP with fail2ban.

Aren't 3xx HTTP codes just redirections and not errors? (I'm a complete noob, so this is a genuine question. )

Anyway: Since everything went so smoothely, I started to use nextcloud and imported all my contacts to the contacts app.

That's when the problems started: I got banned by fail2ban every few minutes. It took me some time and research, but I think I found the problem:

Apparently there's an open bug, that the contacts app requests profile photos for every contact each time the app is used. And it gets a 404 code for every contact that has no photo set. This gets me banned instantly, when opening contacts in nextcloud. I also get banned without using the web interface, because apparently contact syncing (as I use it with the Gnome desktop and iOS) also generates 404 errors, just not as many as the web interface.

For crowdsec there's a whitelist for this problem and for a similar bug of the file browser.

I would like to stay with nextcloud-aio, since I very much like the approach. Is there any option to make my configuration work with fail2ban?

There's regex recommendations in the official documentation and there is a nextcloud-aio fail2ban community container. This would apparently solve the problem for now. (I guess.) But I still want to use fail2ban with npm, because I plan on setting up other containers that might need to be published to the internet...

Would crowdsec be an easier option? Does it work with my configuration?

Any helpful advice is much appreciated!

Alles anzeigen

Redirect codes are are not catching incorrect login attempts, but are include because it is possible, although not likely, that hacker like activity can use redirections to your server. This just protects against that kind of thing by only allowing direct intentional connections. Feel free to remove the 300 filters if you so desire.

Zitat von DonkeeeyKong

Hello, thanks for the answer and the explanation!

I just stumbled across this question when I was trying to find out why fail2ban keeps banning me every few minutes. The 300 filters are not the problem, I was just wondering why they were there.

For now I have to solve the bigger problem of making fail2ban and npm work with nextcloud... Right now in order to be able to use the web interface of nextcloud I have to stop the fail2ban container before accessing the web interface...

I can't help you too much with the nextcloud docker stuff, as I run nextcloud in an LXC not a docker, aside from saying that you need to make sure your nextcloud config.php is configured correctly and the NPM host is configured correctly. Here are the settings I use if it helps:

NPM:

Advanced - Custom NGINX config

rewrite ^/\.well-known/carddav https://$server_name/remote.php/dav/ redirect;

rewrite ^/\.well-known/caldav https://$server_name/remote.php/dav/ redirect;

4 Custom locations pointing you you nextcloud

/.well-known/caldav

/.well-known/carddav

/.well-known/webfinger

/

nextcloud config.php must include something like this. I have 2 trusted proxy addresses listed, the first being the actual ip of the server and the second being the internal KVM "gateway" ip for use completely "within the box" since I using the KVM plugin to run the LXC, allowing me to have NPM direct to a secondary static ip nic that I have set up in the LXC sitting on that internal KVM network

  'trusted_domains' => 
  array (
    0 => '<ip of server if you wish to connect by ip as a "backup"',
    1 => '<web address of server>',
  ),
  'trusted_proxies' => 
  array (
    0 => '192.168.0.241', 
    1 => '192.168.122.1',
  ),
  'overwrite.cli.url' => 'https://<web address of server>',

Zitat von DonkeeeyKong

Phew! Thanks for your help. That looks a lot more complicated than what I have done so far. I don't think I understand what's happening there or that I am able to change that to work with my setup.

I think I might try switching to crowdsec instead of fail2ban...

The recommendations for fail2ban in the official nextcloud documentation don't seem to cover the usecase of using more than just nextcloud with one npm instance. My understanding of all of this is limited though.

chente : Maybe it is advisable to add a comment to your guide that using BernH 's guide with your nextcloude-aio guide (that's what I did) will lead to constantly being banned by fail2ban. Since you linked it in your npm post one could assume they work well together, if there's no comment on that. It's just that BernH's fail2ban configuration from that guide doesn't work with nextcloud...

Alles anzeigen

It’s not that the fail2ban guide doesn’t work with nextcloud, it has to do with making sure nextcloud is not triggering errors in the NPM logs. The settings I have outlined in the NPM-fail2ban guide are simply using fail2ban to watch the NPM logs and ban failed attempts.

From what I understand, Nextcloud by design has some very strict security related configurations that are fine when deployed for direct internet access, but need to be compensated for when using a reverse proxy.

My Nextcloud is running as an lxc with the settings I have outlined above, works fine and is not triggering 300 or 400 errors, but once again, I am not running it as a docker container. Running as an lxc it behaves like a separate computer/server but as a docker it is different and I don’t know what the container is doing differently. The extra NPM configs and Nextcloud config.php settings above are designed to not trigger those errors by allowing proxy access from the ip of NPM and allowing NPM to pass the “problematic” portions of Nextcloud through without triggering the errors.

Zitat von DonkeeeyKong

Sorry. What I meant was: The two guides don't work together out of the box. Getting the nextcloud errors out of the NPM logs would be great!

I see. Running the nextcloud-aio container via docker is the official recommended form of using Nextcloud though. It seems strange, that this causes errors... I thought the 400 errors came from this bug and not from bad configuration.

I will try to copy what you did using my docker setup.

I'm not sure how to set up the custom locations:

What do I put in the Forward Hostname and Port fields? The same settings as on the details tab? (In my case <IP of my server>:11000) Or something else?

This was almost right already. Trusted proxies included only 127.0.0.1 though. I added my server's IP. I hope that's enough. I'm not sure which IP docker uses internally.

Thanks a lot for your help and patience! I guess it seems possible for me to get this working after all.

Alles anzeigen

If you actually look at the nextcloud install guides, manual/bare metal install and using a vm are also recommended. The vm, however, is one made by HanssonIT (yes I used to run this one too). Originally there was only a manual install, then they came out with the VM and finally the docker, but that whole process took several years. I have been using nextcloud since it started, (before it started really as I used to run owncloud), so I have seen all of those deployment steps and have tried them all, but opted to make my own VM (using the manual install guide in a VM as I didn't like the way the HanssonIT VM was built, and ultimately settled on using lxc as the vm when OMV KVM supported it, but not before I tried the original docker release that they came out with. I had problems using nextxtcloud talk in the original docker, so went back to the lxc route (using a docker postgres database and docker redis).

I also prefer a more manual install with nextcloud, as I am more in control regarding updates and I'm not forced into the additional stuff that HanssonIT builds in, and the lxc has about the same performance as docker, while I found the HanssonIT vm to be sluggish in comparison.

As for your NPM custom locations, all you are missing is the ip address or host name and port of the docker (use the same that you use on the main page of the virtual host)

The trusted proxies section of the nextcloud config.php has to have an entry in it to allow access from the Ip address that NPM is running on. In My setup, since I have KVM installed on my OMV system and I'm running nextcloud as an lxc, it behaves as a completely independent computer and even has it's own physical network port, so I have a LAN address for nextcloud that is different than my OMV address, I can also use 192.168.122.1 as the proxy address because KVM sets up the 192.168.122.0/24 "default"/nat network for it's own use, which operates completely inside the system. I have a second virtual NIC in the VM that is attached to this network with a static ip.

As I am not running NC in docker and it has been many years since my initial look at the docker, I am not exactly sure how to treat the proxies section for docker, but my gut reaction says if the NC docker is set up on a macvlan, it should get a lan IP, but I'm not sure if it will work like that by itself because you will be hairpinning network traffic on the same physical connections that NPM uses. Setting up a bridge for your OMV LAN should get around this as it makes the network port behave kind of like a network switch. The alternative would be to possibly set it up with NPM and NC on the same docker network so they can address each other by container name instead of ip address. This is a feature of docker networks so if using ip addresses does not work this may be a way to get around the problem. If you have KVM installed then the 192.168.122.0/24 network also becomes available for "inside the box" connections with vm's (as I am doing with my lcx).

I personally just use that second physical NIC (not configured in OMV but still plugged int to the LAN) for my VM's as it acts like a network switch for the them but does not cause OMV to have 2 or more ip addresses since it's not configured. This helps keep some vm traffic away from my OMV/docker traffic that (NPM). I have around 50 dockers and 4 to 6 vm's running so splitting the bandwidth is not a bad thing and actually have 3 NIC ports that I have configured for different uses to separate that traffic.

The cuda drivers were nvidia-cuda-toolkit if I recall correctly.

Zitat von DonkeeeyKong

You're right. Still, the Docker image is recommended if you want to use a reverse proxy.

I tried setting up the custom locations that way. No matter how I do it: As soon as I add a custom location, the proxy host status in npm changes to 'offline'.

Here is what I tried:

- adding the custom locations and putting the IP of my server in the forward IP field.

- adding the custom locations and putting the IP of my server plus /remote/dav in the forward IP field ( for caldav and carddav - as recommended here)

- adding the custom locations and putting the IP of my server plus /remote.php/dav in the forward IP field ( for caldav and carddav)

- trying port 80 instead of 11000 with all combinations

- any combination of scheme http/https in the custom locations and in the details page

- changing the advanced configuration from $server_name to $server and trying other configurations from here:

Code

location = /.well-known/carddav {  return 301 $scheme://$host/remote.php/dav; }
location = /.well-known/caldav {  return 301 $scheme://$host/remote.php/dav; }
location ^~ /.well-known { return 301 $scheme://$host/index.php$uri; }

and

Code

    location = /.well-known/carddav {
      return 301 $scheme://$host:$server_port/remote.php/dav;
    }
    location = /.well-known/caldav {
      return 301 $scheme://$host:$server_port/remote.php/dav;
    }

and others from other sources I don't recall anymore.

Regarding the trusted proxies section I tried the following:

First it looked like this:

Code

  array (
    0 => '127.0.0.1',
    1 => '::1',
  ),

- I added the IP of my server (2 => '192.168.xxx.xx',)

- then I tried to find out the internal Docker IP of NPM and added that

- then I changed npm's compose file to use the nextcloud-aio network, changed the hostname in npm to nextcloud-aio-apache and added jc21-npm as a trusted proxy. I could access and use Nextcloud via my domain but I still couldn't add any custom locations without the proxy host going offline and I still got 404 errors in the log when opening the contacts section in the Nextcloud web GUI.

When I undo all these changes and type https://mynextcloudaddress.net/.well-known/carddav in my browser I get a page that says This is the WebDAV interface. It can only be accessed by WebDAV clients such as the Nextcloud desktop sync client.

I came to think: Doesn't that mean that the redirection is already working? Syncing my iPhone calendar with my nextcloud calendar works as well. So does syncing my contacts and syncing with GNOME Calendar and GNOME Contacts. So I'm wondering if this is actually my problem or if the proxy is already set up fine 'out of the box'.

The only problem I have is that the contacts app is generating 404 errors for all contacts that have no foto set up. As this seems to be an open bug I'm wondering if changing anything in the npm configuration would stop theses errors from showing up in the logs.

After opening the contacts page in nextcloud my log file looks like this:

Code

[22/Mar/2024:17:26:13 +0100] - 404 404 - GET https mynextcloudaddress.net "/remote.php/dav/addressbooks/users/admin/kontakte-1/9CB5A33A-7966-4286-BBE2-44DDCB574A3D.vcf?photo" [Client 31.xx.xx.11] [Length 0] [Gzip -] [Sent-to nextcloud-aio-apache] "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:123.0) Gecko/20100101 Firefox/123.0" "-"
[22/Mar/2024:17:26:13 +0100] - 404 404 - GET https mynextcloudaddress.net "/remote.php/dav/addressbooks/users/admin/kontakte-1/DEA49D70-C871-44D7-9D44-D7C15D54831E.vcf?photo" [Client 31.xx.xx.11] [Length 0] [Gzip -] [Sent-to nextcloud-aio-apache] "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:123.0) Gecko/20100101 Firefox/123.0" "-"
[22/Mar/2024:17:26:13 +0100] - 404 404 - GET https mynextcloudaddress.net "/remote.php/dav/addressbooks/users/admin/kontakte-1/BD0C613F-46B9-4449-AFC9-6E5988FA9AA7.vcf?photo" [Client 31.xx.xx.11] [Length 0] [Gzip -] [Sent-to nextcloud-aio-apache] "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:123.0) Gecko/20100101 Firefox/123.0" "-"
[22/Mar/2024:17:26:13 +0100] - 404 404 - GET https mynextcloudaddress.net "/remote.php/dav/addressbooks/users/admin/kontakte-1/44B00D57-6FCA-4E72-9EA7-9DBA4062E35E.vcf?photo" [Client 31.xx.xx.11] [Length 0] [Gzip -] [Sent-to nextcloud-aio-apache] "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:123.0) Gecko/20100101 Firefox/123.0" "-"
[22/Mar/2024:17:26:13 +0100] - 404 404 - GET https mynextcloudaddress.net "/remote.php/dav/addressbooks/users/admin/kontakte-1/948D302A-F419-4125-9E68-3D6165FB8C01.vcf?photo" [Client 31.xx.xx.11] [Length 0] [Gzip -] [Sent-to nextcloud-aio-apache] "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:123.0) Gecko/20100101 Firefox/123.0" "-"
[22/Mar/2024:17:26:51 +0100] - 304 304 - GET https mynextcloudaddress.net "/apps/richdocuments/settings/fonts.json" [Client 31.xx.xx.11] [Length 0] [Gzip -] [Sent-to nextcloud-aio-apache] "COOLWSD HTTP Agent 23.05.9.3" "-"

Would this change if I set up npm differently? Syncing contacts and calendar with my iPhone and with desktop applications works very well.

Alles anzeigen

I don't use the contacts in nextcloud so have never encountered that error, but it suggests that the addressbook is also using a dav connection that is being blocked requiring another properly formatted rewrite directive for it to pass the proxy. I am by no means an expert on nextcloud and reverse proxying, I have only shared with you the configuration that works for the way I configure my system.

Everything I have posted has been found via searching for an answer to a problem.

The custom location settings were put in place to quiet errors I saw in the nextcloud admin overview.

Regarding your statement about docker being being recommended for reverse proxy, that is not really true. It is recommended for a "simpler" setup because you don't need to manage multiple servers, but any server, be it a docker service, a vm or a bare metal install can be reverse proxied if you configure the reverse proxy correctly for your install.

The settings in the nextcloud config.php tell nextcloud to allow reverse proxy configurations from a given reverse proxy server, allow connections from a given ip or domain name and specify the server address, nothing more, nothing less. Without them nextcloud will refuse the connection if it does not conform to those settings.

The reverse proxy settings are about allowing nextcloud through the reverse proxy without triggering an error or changing information that nextcloud expects to see in it's data stream (meaning don't overwrite/change information that needs to be present on both the server and the client for a successful connection).

A lot of your problems, as far as I can tell, is that you are not understanding how to configure those two components and don't understand how docker works within your server, and there is nothing wrong with that. We all have to learn these things. However, I can't really tell you exactly how to configure NPM and nextcloud docker because I don't run nextcloud in docker so have never had to figure out that combination.

I can tell you that if nextcloud and npm are on the same docker network, they should be able to be able to address each other by container name instead of ip address (this is a feature of docker designed to get around dynamic docker internal ip addresses). An alternative option could possibly be to put nextcloud on a macvlan so that it gets a LAN ip address, then if it is static, you can use ip addresses to communicate between them, just like if it is a VM or bare metal install with a direct LAN connection, but using lan ip addresses to communicate between docker containers that are not on a mac vlan does not always work, depending on the containers in question, and I know that nextcloud is a bit of a special case so this could be part of your issue.

You may be better off trying/using the nextcloud swag guide if you don't understand the under the hood coding changes needed to make it work, as swag has prebuilt nextcloud configs, all designed around having the containers on the same docker network. However swag is a bit harder to use for things that they don't have configs for as there is no GUI to set up your proxy hosts, so you would have to do some config file editing to get those things working.

I’m glad you are on the road to a solution that works for you.

Nextcloud logs its own errors and doesn’t throw failed logins back to NPM so no it will not catch them. My config has a fail2ban running in my Nextcloud vm that is tailored for Nextcloud. I am not trying to monitor the Nextcloud logs with the same fail2ban because it is essentially on a different computer.

If I get a chance the weekend I will see if the contacts app does the same thing to my config, but normal WebDAV stuff isn’t triggering errors.