Skip to content

Instantly share code, notes, and snippets.

@MehdiFal
Last active November 7, 2018 08:46
Show Gist options
  • Save MehdiFal/b6fc37350e102e2fd7c75a83da2517eb to your computer and use it in GitHub Desktop.
Save MehdiFal/b6fc37350e102e2fd7c75a83da2517eb to your computer and use it in GitHub Desktop.
import java.io.IOException;
import java.nio.charset.Charset;
import okhttp3.Interceptor;
import okhttp3.OkHttpClient;
import okhttp3.Request;
import okhttp3.Response;
import okhttp3.ResponseBody;
import okio.Buffer;
import okio.BufferedSource;
public class TestChunks {
private final static OkHttpClient Client = client ();
public static void main (String[] args) throws IOException {
Request Request = new Request.Builder ()
.url ("http://localhost:8100/hello")
.get ()
.build ();
Client.newCall (Request).execute ();
}
private static OkHttpClient client () {
OkHttpClient.Builder clientBuilder = new OkHttpClient.Builder ();
clientBuilder.addNetworkInterceptor (newNetworkInterceptor ());
return clientBuilder.build ();
}
private static Interceptor newNetworkInterceptor () {
return chain -> {
Response originalResponse = chain.proceed (chain.request ());
ResponseBody responseBody = originalResponse.body ();
BufferedSource source = responseBody.source ();
int i = 0;
Buffer buffer = new Buffer (); // we manually create our own buffer
// source exhausted means no more bytes to consume
while (!source.exhausted ()) {
System.out.println ("\n__________________________________________________________________\n");
System.out.println ("Chunk " + i + ": \n");
// I'm gonna read the whole buffer, this way I'm sure I get the real chunks as they come
long readBytes = source.read (buffer, Long.MAX_VALUE);
System.out.println ("Read: " + readBytes + " bytes");
System.out.println ("Content: \n" + buffer.readString (Charset.forName("UTF-8")));
System.out.println ("\n__________________________________________________________________\n");
i ++;
}
return null; // just for convenience as we don't care
};
}
}
/*
My Output looks like the following,
notice the 6th chunk got split into 2 smaller chunks, which gives us 11 chunks instead of 10
__________________________________________________________________
Chunk 0:
Read: 1279 bytes
Content:
------------ 0-Start ------------
Full content of the Chunk 0 of the Data
------------ 0-End ------------
__________________________________________________________________
__________________________________________________________________
Chunk 1:
Read: 1279 bytes
Content:
------------ 1-Start ------------
Full content of the Chunk 1 of the Data
------------ 1-End ------------
__________________________________________________________________
__________________________________________________________________
Chunk 2:
Read: 1279 bytes
Content:
------------ 2-Start ------------
Full content of the Chunk 2 of the Data
------------ 2-End ------------
__________________________________________________________________
__________________________________________________________________
Chunk 3:
Read: 1279 bytes
Content:
------------ 3-Start ------------
Full content of the Chunk 3 of the Data
------------ 3-End ------------
__________________________________________________________________
__________________________________________________________________
Chunk 4:
Read: 1279 bytes
Content:
------------ 4-Start ------------
Full content of the Chunk 4 of the Data
------------ 4-End ------------
__________________________________________________________________
__________________________________________________________________
Chunk 5:
Read: 1279 bytes
Content:
------------ 5-Start ------------
Full content of the Chunk 5 of the Data
------------ 5-End ------------
__________________________________________________________________
__________________________________________________________________
Chunk 6:
Read: 471 bytes
Content:
------------ 6-Start ------------
First part of content of the Chunk 6 of the Data
__________________________________________________________________
__________________________________________________________________
Chunk 7:
Read: 808 bytes
Content:
Second part of content of the Chunk 6 of the Data
------------ 6-End ------------
__________________________________________________________________
__________________________________________________________________
Chunk 8:
Read: 1279 bytes
Content:
------------ 7-Start ------------
Full content of the Chunk 7 of the Data
------------ 7-End ------------
__________________________________________________________________
__________________________________________________________________
Chunk 9:
Read: 1279 bytes
Content:
------------ 8-Start ------------
Full content of the Chunk 8 of the Data
------------ 8-End ------------
__________________________________________________________________
__________________________________________________________________
Chunk 10:
Read: 1279 bytes
Content:
------------ 9-Start ------------
Full content of the Chunk 9 of the Data
------------ 9-End ------------
__________________________________________________________________
*/
import org.apache.commons.io.IOUtils;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.Charset;
import okhttp3.mockwebserver.MockResponse;
import okhttp3.mockwebserver.MockWebServer;
public class Main {
private final static String BaseUrl = "/hello";
public static void main (String[] args) throws IOException {
// Full size of data 12790 bytes
InputStream is = new FileInputStream ("/Users/mehdi/Desktop/DataSets/data.txt");
String data = IOUtils.toString (is, Charset.forName ("UTF-8"));
is.close ();
MockWebServer server = new MockWebServer ();
MockResponse response = new MockResponse ();
response.setChunkedBody (data, 1279); // I split the data.txt file into 12 blocks of 1279b each
server.enqueue (response);
server.start (8100);
server.url (BaseUrl);
}
}
/*
My Data Looks like the following (removed the content):
------------ 0-Start ------------
1029b of data is comprised from the first character of the separator above to the \n of separator below are 1029 bytes.
------------ 0-End ------------
------------ 1-Start ------------
1029b of data is comprised from the first character of the separator above to the \n of separator below are 1029 bytes.
------------ 1-End ------------
------------ 2-Start ------------
1029b of data is comprised from the first character of the separator above to the \n of separator below are 1029 bytes.
------------ 2-End ------------
------------ 3-Start ------------
1029b of data is comprised from the first character of the separator above to the \n of separator below are 1029 bytes.
------------ 3-End ------------
------------ 4-Start ------------
1029b of data is comprised from the first character of the separator above to the \n of separator below are 1029 bytes.
------------ 4-End ------------
------------ 5-Start ------------
1029b of data is comprised from the first character of the separator above to the \n of separator below are 1029 bytes.
------------ 5-End ------------
------------ 6-Start ------------
1029b of data is comprised from the first character of the separator above to the \n of separator below are 1029 bytes.
------------ 6-End ------------
------------ 7-Start ------------
1029b of data is comprised from the first character of the separator above to the \n of separator below are 1029 bytes.
------------ 7-End ------------
------------ 8-Start ------------
1029b of data is comprised from the first character of the separator above to the \n of separator below are 1029 bytes.
------------ 8-End ------------
------------ 9-Start ------------
1029b of data is comprised from the first character of the separator above to the \n of separator below are 1029 bytes.
------------ 9-End ------------
*/
@WarrenFaith
Copy link

WarrenFaith commented Nov 7, 2018

The kotlinified interceptor:

class ChunkInterceptor : Interceptor {
    override fun intercept(chain: Interceptor.Chain): Response {
        val originalResponse = chain.proceed(chain.request())
        val responseBody = originalResponse.body()
        val source = responseBody!!.source()
        val buffer = Buffer() // we manually create our own buffer
        val charset = Charset.forName("UTF-8")

        while (!source.exhausted()) {
            // I'm gonna read the whole buffer, this way I'm sure I get the real chunks as they come
            source.read(buffer, Long.MAX_VALUE)
            println("Chunk: ${buffer.readString(charset)}")
        }
        return originalResponse
    }
}

Most important: This needs to be registered as a networkInterceptor, not a normal interceptor... I missed that part the first time :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment