Why Using HTTP Client Has a Bug That You Must Avoid | by Dwen | May, 2022

Sharing of occasional issues encountered in utilizing HTTP Consumer in Java

Photograph by Andyone on Unsplash

As software program builders, we all know what number of issues are buried in a system that appears regular.

Right now, I’ll share with you an issue attributable to improper use of HTTP Consumer encountered within the precise growth course of.

The writer traced during the looks of the issue and at last discovered the basis explanation for the issue: HTTP Consumer.

So it is a actual expertise sharing.

Virtually each system has the perform of asynchronously calling third-party providers. The accountable system implements a message queue based mostly on blocking queues to name third-party providers.

To make sure idempotency, queues are consumed sequentially. This results in an issue, as soon as one of many messages is blocked, the next messages can’t be consumed. When the queue is full, messages can’t be added to the queue both.

It appears: In extraordinarily occasional situations, the message queue is blocked for greater than ten minutes. What the hell is that this?

The next is step-by-step troubleshooting of the issue.

The very first thing that involves thoughts is whether or not there’s a drawback with the underlying mechanism of the message queue implementation. For instance, the message queue implements steady information entry by producers and shoppers by whereas(true) polling + sleep.

Because the message is blocked, is it as a result of the sleep is overdone? After occupied with it, even when the CPU performs fragmentation processing, it is not going to sleep so it is not going to be woken up. It is usually unlikely that the thread is interrupted. As a result of, whether it is interrupted, your entire queue will cling up, and it’ll return to regular after no delay.

I’ve been confused right here for a very long time, and I’ve learn the supply code of the message queue implementation for a very long time, however there isn’t a breakthrough. So, I mentioned it with a good friend, and a sentence jogged my memory: it is probably not an issue of sleep, however an issue of producers or shoppers.

So I fastidiously scoured the log and located that it was true: the producer misplaced information to the queue as soon as, and didn’t lose information for a very long time; the patron nonetheless had consumption logs a couple of minutes after the producer misplaced information to the queue. Clearly, the producer is blocked by the patron.

Preliminary conclusion: The patron consumes too lengthy, inflicting the queue to be full, and the producer is blocked when including information to the queue.

Empirical guess: There are HTTP requests in shoppers, and HTTP requests might maintain connections for a very long time with out releasing them.

When the evaluation locates the reason for the HTTP request, it’s simple to resolve. First, I analyzed the log and located that there was certainly an HTTP request. The request parameters have been printed earlier than the request, however the print of the returned end result was by no means seen.

The monitoring log lastly noticed that the log of the returned end result was printed after quarter-hour, and the log content material was irregular for the opposite occasion’s service.

Wanting on the code once more, it’s discovered that the HTTP request is applied based mostly on the HTTP Consumer, and the one that wrote this code didn’t set the timeout interval. As a way to preserve the enterprise desensitized, I discovered the same code:

Just like the above code, the request parameters are set, however the timeout for HttpPost is just not specified. The seemingly regular code hides an enormous pit, and the result’s that the HTTP request has been ready.

The state of affairs encountered by the writer is just not unhealthy, the timeout time set by the opposite occasion is quarter-hour, and the result’s returned. If the opposite occasion doesn’t set the timeout time, it might wait on a regular basis. When the enterprise quantity is comparatively giant, it is going to result in disasters!

Discovering the issue is usually the toughest, and when the issue is discovered, it’s a lot simpler to resolve. Totally different variations of HTTP Consumer have other ways of setting the timeout interval. That is one other main disadvantage of HTTP Consumer. The API model modifications an excessive amount of.

Configuration for model 4.3:

httpClient.getParams().setParameter(CoreConnectionPNames.CONNECTION_TIMEOUT,10000);httpClient.getParams().setParameter(CoreConnectionPNames.SO_TIMEOUT,10000);

Configuration after model 4.3:

RequestConfig requestConfig = RequestConfig.customized().setSocketTimeout(10000).setConnectTimeout(10000).construct();httpGet.setConfig(requestConfig);

Amongst them, setConnectTimeout is the connection timeout time in milliseconds.

setSocketTimeout is the timeout for requesting information acquisition, in milliseconds. If an interface is accessed and information can’t be returned throughout the specified time, the decision will probably be deserted straight.

For using different variations, it is suggested to check with the related API directions.

Thanks for studying this text. In case you discover any errors on this article, please let me know.

More Posts