Understanding the challenges in scaling WebSocket servers
In my previous article, I wrote about designing and constructing a WebSocket server in a microservice structure. Though the implementation works high quality for a single occasion of a WebSocket server, we are going to begin dealing with points once we attempt to scale up the variety of WebSocket server cases (aka horizontal scaling). This text seems to be into the design issues for scaling the WebSocket server utilizing a publish-subscribe messaging sample.
First, let’s attempt to perceive why we want horizontal scaling. As our consumer base grows, the load on the server grows. And when the load grows, a single server won’t be able to offer excessive efficiency for all of the customers. Therefore, it’s essential to offer the aptitude to extend/lower the variety of servers every time essential to fulfill the consumer’s demand in addition to to avoid wasting sources as a part of our design issues.
Horizontal scaling refers to including extra machines to your infrastructure to deal with the excessive demand on the server. In our microservice context, scaling horizontally is identical as deploying extra cases of the microservice. A load balancer will then be required to distribute the visitors among the many a number of microservice cases, as proven under:
With this, I hope you higher perceive why we want horizontal scaling in our infrastructure. So let’s transfer on to study the design issues for scaling WebSocket servers in a microservice structure.
Beforehand, we carried out the WebSocket server utilizing Spring Boot, Stomp, and Redis Pub/Sub. Communication between the net software (frontend) and WebSocket server is through WebSocket, whereas communication between the microservices (backend) and WebSocket server is through API and publish-subscribe messaging sample. For extra info, confer with the previous article.
The earlier design works completely high quality in a setup the place we solely have a single occasion of every microservices. Nonetheless, having a single occasion will not be sensible in a manufacturing atmosphere. Usually, we are going to deploy our microservices with a number of replicas (or cases) for prime availability in a manufacturing atmosphere. Due to this fact, once we attempt to horizontally scale the variety of WebSocket servers (microservice) or backend microservices, we are going to discover the next issues.
Challenge #1: Message loss because of the load balancer
In our earlier article, we added APIs for backend microservices to ship messages to the WebSocket server for unidirectional real-time communication. As proven under, a load balancer helps to deal with visitors redirection when scaling the variety of WebSocket servers.
Within the above setup, an occasion of the net software (frontend) establishes a WebSocket connection to the WebSocket server (occasion B). When the backend server tries to ship messages to the net software, the load balancer redirects the API request to the WebSocket server (occasion A). Since WebSocket server (occasion A) doesn’t have a WebSocket connection to that exact occasion of the net software, the message can be misplaced.
Resolution for Challenge #1: Broadcast messages utilizing Pub/Sub
Notice: This resolution is drastically impressed by Amr Saleh, who wrote about Building Scalable Facebook-like Notification using Server-Sent Events and Redis. Do test that out!
To resolve the primary challenge, we are able to introduce a broadcast channel utilizing the publish-subscribe messaging sample the place all messages acquired from the backend microservices can be broadcasted to all WebSocket server cases as proven within the diagram above. This ensures that every one net software cases (frontend) will obtain that message through WebSocket from the WebSocket server.
Challenge #2: Duplicate message processing on account of a number of backend subscribers to a single matter
In our earlier article, we used Redis Pub/Sub to deal with bidirectional real-time communication between the WebSocket server (microservice) and backend microservices. After we scale up the variety of WebSocket servers and backend microservices, you’ll discover that every one subscribers to Redis Pub/Sub will obtain the messages as proven under.
Let’s take a look at the message movement in every course in bidirectional real-time communication.
- Message Circulate: Microservices to net software (no duplicated processing) → It’s essential for all cases of the WebSocket server to obtain the messages as every net browser establishes a WebSocket reference to solely a single WebSocket server occasion. Therefore, when messages movement from the backend microservices to the net software (backend → WebSocket server → frontend), just one occasion of the net software will obtain the message, which is the proper conduct.
- Message Circulate: Internet software to microservices (duplicated processing) → When messages are flowing from the net software to the backend microservices (frontend → WebSocket server → backend), we’d anticipate just one occasion of the backend microservices to course of the message. Nonetheless, all backend microservices (as subscribers) will obtain the message, ensuing within the message being processed a number of occasions, which is inaccurate conduct.
Resolution for Challenge #2: Pub/Sub with client teams
To resolve the second challenge, we are going to make use of the idea of Shopper Teams (launched by Kafka), the place just one subscriber receives the message for processing. This ensures that there can be no duplicated message processing as just one backend microservice occasion will obtain the message.
As Redis Pub/Sub in my earlier article implementation doesn’t assist the buyer group idea, we are able to both use Redis Streams, Google Pub/Sub, RabbitMQ, or Apache Kafka to implement the publish-subscribe messaging sample with client teams. I cannot go into particulars on which is healthier to your implementation as this isn’t the intent of this text.
To wrap issues up, we’ve run by way of the design issues on find out how to scale the WebSocket server in a microservice structure horizontally. Primarily, we’re utilizing publish-subscribe messaging patterns to make sure that there is no such thing as a message loss or duplicated message processing within the technique of real-time communication between the net software (frontend) and microservices (backend).
That’s it! I hope you realized one thing new from this text. This text solely covers the design issues for scaling the WebSocket server. Keep tuned for the following one, the place I’ll elaborate extra on how one can implement this design utilizing Redis Pub/Sub, Redis Streams, and Spring Boot.
Should you like this text, please observe me for extra :).
Thanks for studying till the tip. Comfortable studying!