tunneling secure websocket connections with apache

I’ve got it working.

Scenario

-------------       ----------------       ----------
| Browser   |<----->| Apache httpd |<----->| Tomcat |
|           |  SSL  |    2.4.9     |  SSL  | 7.0.52 |
-------------       ----------------       ----------

Browser WebSocket through Apache httpd, reverse proxying to the web app in Tomcat. All SSL front-to-back.

Here’s the configuration for each piece:

Browser Client

Note the trailing “https://stackoverflow.com/” in the url: wss://host/app/ws/. It was necessary to match the correct wss ProxyPass directive (shown further down in the Apache config section) and preventing a 301 redirect to https://host/app/ws. That is, it was redirecting using the https scheme and not the wss scheme for the back-end.

Test Page

<!doctype html>
<body>

<script type="text/javascript">
    var connection = new WebSocket("wss://host/app/ws/");

    connection.onopen = function () {
        console.log("connected");
    };

    connection.onclose = function () {
        console.log("onclose");
    };

    connection.onerror = function (error) {
        console.log(error);
    };
</script>

</body>
</html>

Apache httpd

I am using Apache httpd 2.4.9, which out of the box provides mod_proxy_wstunnel. However, the mod_proxy_wstunnel.so provided does not support SSL when using wss:// scheme. It ends up trying to connect to the back-end (Tomcat) in plaintext, which fails the SSL handshake. See bug here. So, you have to patch mod_proxy_wstunnel.c yourself by following the suggested correction in the bug report. It’s an easy 3 line change.

Suggested correction,
314a315
>     int is_ssl = 0;
320a322
>         is_ssl = 1;
344c346
<     backend->is_ssl = 0;
---
>     backend->is_ssl = is_ssl;

Then rebuild the module and replace in your new mod_proxy_wstunnel.so with the old one.

Building Apache httpd

Here’s the (2.4.9) command I used to build in the modules I wanted. You might not need them all.

./configure --prefix=/usr/local/apache --with-included-apr --enable-alias=shared
--enable-authz_host=shared --enable-authz_user=shared 
--enable-deflate=shared --enable-negotiation=shared 
--enable-proxy=shared --enable-ssl=shared --enable-reqtimeout=shared
--enable-status=shared --enable-auth_basic=shared
--enable-dir=shared --enable-authn_file=shared
--enable-autoindex=shared --enable-env=shared --enable-php5=shared
--enable-authz_default=shared --enable-cgi=shared
--enable-setenvif=shared --enable-authz_groupfile=shared
--enable-mime=shared --enable-proxy_http=shared
--enable-proxy_wstunnel=shared

Note the very last switch: --enable-proxy_wstunnel=shared At first, I was incorrectly using --enable-proxy-wstunnel=shared, which seemed to build fine, but ultimately wouldn’t work when I used the resultant .so file. See the difference? You want to make sure to use an underscore in "proxy_wstunnel" rather than a dash.

Apache httpd config

httpd.conf

...
LoadModule proxy_module modules/mod_proxy.so
...
LoadModule proxy_wstunnel_module modules/mod_proxy_wstunnel.so
...
LoadModule ssl_module modules/mod_ssl.so
...
Include conf/extra/httpd-ssl.conf
...
LogLevel debug
ProxyRequests off

# Note, this is the preferred ProxyPass configuration, and *should* be equivalent
# to the same inline version below, but it does NOT WORK!
#<Location /app/ws/>
#        ProxyPass wss://localhost:8443/app/ws
#        ProxyPassReverse wss://localhost:8443/app/ws
#</Location>
#<Location /app/>
#        ProxyPass https://localhost:8443/app/
#        ProxyPassReverse https://localhost:8443/app/
#</Location>

# NOTE: Pay strict attention to the slashes "https://stackoverflow.com/" or lack thereof!
# WebSocket url endpoint
ProxyPass /app/ws/ wss://localhost:8443/app/ws
ProxyPassReverse /app/ws/ wss://localhost:8443/app/ws

# Everything else
ProxyPass /app/ https://localhost:8443/app/
ProxyPassReverse /app/ https://localhost:8443/app/

If you didn’t see my note in the above config, here it is again: Pay strict attention to the slashes “https://stackoverflow.com/” or lack thereof!

Also, if you are seeing debug log statements in your apache log that says a wss connection was made then closed, it is possible that you have mod_reqtimeout enabled as I did, so make sure it not loaded:

#LoadModule reqtimeout_module modules/mod_reqtimeout.so

Tomcat

Assuming your HTTP connector is setup correct, there’s not much to configure in tomcat. Though to aid in debugging, I found it useful to create a $CATALINA_HOME/bin/setenv.sh that looked like this:

setenv.sh

CATALINA_OPTS=$CATALINA_OPTS" -Djavax.net.debug=all -Djavax.net.debug=ssl:handshake:verbose"

This allowed me to see if the mod_proxy_wstunnel.so that I modified was working or not for wss://. When it wasn’t working, my catalina.out log file would show:

javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
http-nio-8443-exec-1, SEND TLSv1 ALERT:  fatal, description = internal_error
http-nio-8443-exec-1, WRITE: TLSv1 Alert, length = 2
http-nio-8443-exec-1, called closeOutbound()
http-nio-8443-exec-1, closeOutboundInternal()

Final Thoughts

Though I am using Apache httpd 2.4.9, I’ve seen where backports of mod_proxy_wstunnel can be applied to versions 2.2.x. Hopefully my notes above can be applied to those older versions.

Leave a Comment