Nginx as reverse proxy on CentOS 9 Stream – a problematic combination

Posted on by

In my last post, I described that I had chosen CentOS 9 Stream as the operating system for my new home server installation, and I described that this caused some issues. The first and by far the largest issue I encountered was with my reverse proxy setup.

Nginx has always been my first choice for a reverse proxy from when I built my very first Linux server at home. We’re talking 2010 now, even before Nginx as a company was founded, and my choice appears to have been a good one, as Nginx is one of the most used pieces of software for this task. On my new server installation, it therefore made sense to stay with Nginx and use my old configuration files. For this blog everything worked great and this blog was the first site to be back up. But then I encountered some really weird issues. The first one was that I couldn’t get my reverse proxy setup for my HCL Connections installation working. Connections and Apache itself were working fine, so the problem was really with the Nginx reverse proxy. My config file backup had been pretty old, so I was suspecting a configuration error, but I couldn’t find any.

The second issue was with Domino where mail through iNotes would work fine, but mail through Verse wouldn’t load. Again, I could verify that the problem was not with HCL Verse. Also, my Home assistant container wouldn’t work with my Nginx config.
The error in the log was cryptic: SSL_read() failed (SSL: error:0A000126:SSL routines::unexpected eof while reading) while reading upstream, client [..]. Google gave me 274 results for that string, but I had to look hard for hints on how to actually solve it.

The cause seems to be a new “feature” in OpenSSL 3.0, which brings me to one of the big changes in CentOS 9 Stream compared to CentOS 8 (Stream). CentOS 8 uses OpenSSL 1.1.1k, while CentOS 9 Stream comes with OpenSSL 3.0.1 14. As this page explains, OpenSSL 3 introduced a new protection against truncation attacks, which causes problems with quite a lot of “non-compliant” servers. The OpenSSL team seems to have added a “legacy” parameter, which could be set, but I couldn’t find much documentation about it.

My next hint was this discussion on the OpenSSL issue tracker on GitHub (and here), where a developer of Nginx participated. This pointed me to a forum thread on the Nginx forum, where they discussed this problem and wondered if this was a problem of Nginx or a problem of OpenSSL. The outcome seemed to be that the Nginx developers felt that the OpenSSL guys would be too arrogant to fix it on their side, so the Nginx developers should probably do something. This thread, however, was from March 2020. That’s over 2 years ago. That can’t be the problem now, can it?

But what if the version of Nginx that Red Hat delivered with CentOS Stream 9 was an older version? That wouldn’t be a first. So, how can I test with a new version of Nginx to rule out this possibility? Where can I find the latest version of Nginx? Of course, on the Docker Hub as the official Nginx container! So, I stopped Nginx on my server. Created an Nginx container, which I pointed to the configuration files of my Nginx installation on my server and let it run with the –network=host setting. This provided me with a drop-in replacement of Nginx, but instead of version 1.20.1, I was now running 1.23.1. You probably guessed where this post was going. At this point, suddenly Connections was working through my reverse proxy, Verse worked, etc.

Of course a container doesn’t just come with a new version of Nginx. It comes with its own version of all libraries including OpenSSL. The Nginx container uses OpenSSL 1.1.1n, so no OpenSSL 3.0. I therefore don’t know if this Nginx container works because it’s a newer version of Nginx or because it’s an older version of OpenSSL. That will need more research.

So the lesson to take away from this post: Beware when you want to use Nginx as a reverse proxy on CentOS 9 Stream server (and I suspect on more RHEL 9 derivatives) as with the out-of-the-box configuration of OpenSSL and Nginx, you might run into problems. I suspect that there are other solutions to the one I chose, but using an Nginx container instead of the Nginx that’s shipped with CentOS 9 Stream, is one possible solution.