Apache HTTP Client - https://hc.apache.org/httpcomponents-client-4.5.x/current/tutorial/html/connmgmt.html

apache_http_client.md

Source

• https://hc.apache.org/httpcomponents-client-4.5.x/current/tutorial/html/connmgmt.html

Reference

• https://stackoverflow.com/questions/44821561/how-to-set-proxy-host-on-httpclient-request-in-java

Table of Contents

• 2.3.3. Pooling connection manager
• 2.4. Multithreaded request execution
• 2.5. Connection eviction policy
• 2.6. Connection keep alive strategy
• 2.7.1. Secure socket layering
• 2.7.4. Hostname verification
• 2.8. HttpClient proxy configuration

2.3.3. Pooling connection manager

• Managing a pool of client connections and is able to service connection requests from multiple execution threads.
• Connections are pooled on a per route basis.
• A request for a route for which the manager already has a persistent connection available in the pool will be serviced by leasing a connection from the pool rather than creating a brand new connection.

2.4. Multithreaded request execution

• HttpClient can be used to execute multiple requests simultaneously using multiple threads of execution.
• If all connections for a given route have already been leased, a request for a connection will block until a connection is released back to the pool. One can ensure the connection manager does not block indefinitely in the connection request operation by setting http.conn-manager.timeout to a positive value.
• While HttpClient instances are thread safe and can be shared between multiple threads of execution, it is highly recommended that each thread maintains its own dedicated instance of HttpContext.

2.5. Connection eviction policy

• If the connection gets closed on the server side, the client side connection is unable to detect the change in the connection state (and react appropriately by closing the socket on its end).
• HttpClient tries to mitigate the problem by testing whether the connection is stale, that is no longer valid because it was closed on the server side, prior to using the connection for executing an HTTP request.
• The stale connection check is not 100% reliable.
• The only feasible solution that does not involve a one thread per socket model for idle connections, but is a dedicated monitor thread used to evict connections that are considered expired due to a long period of inactivity.
• The monitor thread can periodically call ClientConnectionManager#closeExpiredConnections() method to close all expired connections and evict closed connections from the pool.
• It can also optionally call ClientConnectionManager#closeIdleConnections() method to close all connections that have been idle over a given period of time.

public static class IdleConnectionMonitorThread extends Thread {
    
    private final HttpClientConnectionManager connMgr;
    private volatile boolean shutdown;
    
    public IdleConnectionMonitorThread(HttpClientConnectionManager connMgr) {
        super();
        this.connMgr = connMgr;
    }

    @Override
    public void run() {
        try {
            while (!shutdown) {
                synchronized (this) {
                    wait(5000);
                    // Close expired connections
                    connMgr.closeExpiredConnections();
                    // Optionally, close connections
                    // that have been idle longer than 30 sec
                    connMgr.closeIdleConnections(30, TimeUnit.SECONDS);
                }
            }
        } catch (InterruptedException ex) {
            // terminate
        }
    }
    
    public void shutdown() {
        shutdown = true;
        synchronized (this) {
            notifyAll();
        }
    }
    
}

2.6. Connection keep alive strategy

• If the Keep-Alive header is not present in the response, HttpClient assumes the connection can be kept alive indefinitely.
• However, many HTTP servers in general use are configured to drop persistent connections after a certain period of inactivity in order to conserve system resources, quite often without informing the client.
• In case the default strategy turns out to be too optimistic, one may want to provide a custom keep-alive strategy.

ConnectionKeepAliveStrategy myStrategy = new ConnectionKeepAliveStrategy() {

    public long getKeepAliveDuration(HttpResponse response, HttpContext context) {
        // Honor 'keep-alive' header
        HeaderElementIterator it = new BasicHeaderElementIterator(
                response.headerIterator(HTTP.CONN_KEEP_ALIVE));
        while (it.hasNext()) {
            HeaderElement he = it.nextElement();
            String param = he.getName();
            String value = he.getValue();
            if (value != null && param.equalsIgnoreCase("timeout")) {
                try {
                    return Long.parseLong(value) * 1000;
                } catch(NumberFormatException ignore) {
                }
            }
        }
        HttpHost target = (HttpHost) context.getAttribute(
                HttpClientContext.HTTP_TARGET_HOST);
        if ("www.naughty-server.com".equalsIgnoreCase(target.getHostName())) {
            // Keep alive for 5 seconds only
            return 5 * 1000;
        } else {
            // otherwise keep alive for 30 seconds
            return 30 * 1000;
        }
    }

};
CloseableHttpClient client = HttpClients.custom()
        .setKeepAliveStrategy(myStrategy)
        .build();

2.7.1. Secure socket layering

• HttpClient ships with SSLSocketFactory that implements SSL/TLS layering.
• Please note HttpClient does not use any custom encryption functionality.
• It is fully reliant on standard Java Cryptography (JCE) and Secure Sockets (JSEE) extensions.

2.7.4. Hostname verification

• Important: hostname verification should not be confused with SSL trust verification.

  • DefaultHostnameVerifier: The default implementation used by HttpClient is expected to be compliant with RFC 2818. The hostname must match any of alternative names specified by the certificate, or in case no alternative names are given the most specific CN of the certificate subject. A wildcard can occur in the CN, and in any of the subject-alts.
  • NoopHostnameVerifier: This hostname verifier essentially turns hostname verification off. It accepts any SSL session as valid and matching the target host.

SSLContext sslContext = SSLContexts.createSystemDefault();
SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(
        sslContext,
        NoopHostnameVerifier.INSTANCE);

• As of version 4.4 HttpClient uses the public suffix list kindly maintained by Mozilla Foundation to make sure that wildcards in SSL certificates cannot be misused to apply to multiple domains with a common top-level domain.
• HttpClient ships with a copy of the list retrieved at the time of the release.
• The latest revision of the list can found at https://publicsuffix.org/list/.
• It is highly adviseable to make a local copy of the list and download the list no more than once per day from its original location.

2.8. HttpClient proxy configuration

• Even though HttpClient is aware of complex routing schemes and proxy chaining, it supports only simple direct or one hop proxy connections out of the box.
• The simplest way to tell HttpClient to connect to the target host via a proxy is by setting the default proxy parameter:

HttpHost proxy = new HttpHost("someproxy", 8080);
DefaultProxyRoutePlanner routePlanner = new DefaultProxyRoutePlanner(proxy);
CloseableHttpClient httpclient = HttpClients.custom()
        .setRoutePlanner(routePlanner)
        .build();

// One can also instruct HttpClient to use the standard JRE proxy selector to obtain proxy information:
SystemDefaultRoutePlanner routePlanner = new SystemDefaultRoutePlanner(
        ProxySelector.getDefault());
CloseableHttpClient httpclient = HttpClients.custom()
        .setRoutePlanner(routePlanner)
        .build();

• proxy request example:

/* https://stackoverflow.com/questions/44821561/how-to-set-proxy-host-on-httpclient-request-in-java */

HttpClient client = new HttpClient();

HostConfiguration config = client.getHostConfiguration();
config.setProxy("someProxyURL", "someProxyPort");

Credentials credentials = new UsernamePasswordCredentials("username", "password");
AuthScope authScope = new AuthScope("someProxyURL", "someProxyPort");
client.getState().setProxyCredentials(authScope, credentials);

EntityEnclosingMethod method = new PostMethod(url);
method.setRequestEntity(new StringRequestEntity(requestXML, "text/xml", "utf-8"));