Case Study: Loadbalancing whitin the Amazon Cloud

Initial Situation

Duden provides Internet-based correction solutions at www.duden.de. The corrections are performed by multiple instances of the Duden Enterprise Servers that run at the Amazon cloud. So far loadbalancing occurred at the level of the HTTP protocol, which distributed the load onto individual nodes. A scaling of the server farm was responsible for the launch of additional nodes during heavy load or for the shutdown of nodes in times of decreasing usage.
The existing loadbalancing did not support sessions based on Web Services. However, sticky sessions using the SOAP protocol were necessary to run more applications at the server farm.

Requirements

To run all applications within the cloud a loadbalancing solution was needed that would meet the following requirements:

All requests of a session must be forwarded to the same server node (sticky sessions).
Depending on the load, the list of servers has to be extended or shortened.
The ability to work within the Amazon Cloud.
The balancing should scale up to hundreds of thousands requests per hour.
Extraction of the session id out of an XML element inside the SOAP body.
SSL encryption.

Membrane Service Proxy

When searching for a suitable loadbalancer among others the Membrane Service Proxy was investigated. At that time Membrane did not support SOAP sessions, but Membrane was easily extensible by Java and the source code was freely available.

Realization

The decision was made in favor of an extension for SOAP-based sessions for Membrane Service Proxy. To implement the extension an open source sponsorship contract was signed with predic8, the manufacturer of Membrane Service Proxy. The implemented functions for Duden were published under the ASF open source license and thus are available to the general public. Duden benefits from bug fixes and the experience of other users. Since the extension was integrated into the Membrane distribution, Duden can upgrade to new Membrane versions without performing a migration project.
The extension consists of the following components:

An extractor that reads the session id from an XML element.
A REST based interface for adding and removing nodes from the cluster.
A Web frontend to manage clusters.

Experience

"Due to the extensibility of Membrane, we were able to implement the load balancing exactly as required. Since over a year Membrane Service Proxy is working fast and reliably. The machine running the software is barely on load. We would do it with Membrane Service Proxy any time again."

Heinz-Detlev Koch, Speech Technology Development at Duden