"Premature end of chunk coded message body" and TruncatedChunkException when calling HttpResponse.disconnect()

[joaomagalhaes]

Environment details

Using:
com.google.apis:google-api-services-androidpublisher:v3-rev142-1.25.0
com.google.http-client:google-http-client:1.39.2

2. OS type and version: OS Big Sur (11.2.3)
3. Java version: OpenJDK Runtime Environment Corretto-11.0.10.9.1 (build 11.0.10+9-LTS)
4. google-http-client version(s): 1.39.1 and 1.39.2

Steps to reproduce

Call com.google.api.client.http.HttpResponse.disconnect() after com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed()

Code example

final AndroidPublisher.Inappproducts inAppProductsApi = this.publisherApiClient.inappproducts();
final AndroidPublisher.Inappproducts.List list = inAppProductsApi.list(this.packageName);

final HttpResponse httpResponse = list.executeUnparsed();
final int statusCode = httpResponse.getStatusCode();
httpResponse.disconnect();

Stack trace

http-outgoing-0 >> GET /androidpublisher/v3/applications/{package_name}/inappproducts HTTP/1.1
http-outgoing-0 >> User-Agent: Google-API-Java-Client Google-HTTP-Java-Client/1.39.2 (gzip)
headers|http-outgoing-0 >> Host: www.googleapis.com
headers|http-outgoing-0 >> Connection: Keep-Alive

headers|http-outgoing-0 << HTTP/1.1 200 OK
headers|http-outgoing-0 << Content-Type: application/json; charset=UTF-8
headers|http-outgoing-0 << Content-Encoding: gzip
headers|http-outgoing-0 << Transfer-Encoding: chunked

MainClientExec|Connection can be kept alive indefinitely
MainClientExec|Cancelling request execution
DefaultManagedHttpClientConnection|http-outgoing-0: Shutdown connection
MainClientExec|Connection discarded
PoolingHttpClientConnectionManager|Connection released: [id: 0][route: {s}->https://www.googleapis.com:443][total available: 0; route allocated: 0 of 20; total allocated: 0 of 200]

org.apache.http.TruncatedChunkException: Truncated chunk (expected size: 974; actual size: 595)
	at org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:197) ~[httpcore-4.4.14.jar:4.4.14]
	at org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:210) ~[httpcore-4.4.14.jar:4.4.14]
	at org.apache.http.impl.io.ChunkedInputStream.close(ChunkedInputStream.java:312) ~[httpcore-4.4.14.jar:4.4.14]
	at org.apache.http.impl.execchain.ResponseEntityProxy.streamClosed(ResponseEntityProxy.java:142) ~[httpclient-4.5.13.jar:4.5.13]
	at org.apache.http.conn.EofSensorInputStream.checkClose(EofSensorInputStream.java:228) ~[httpclient-4.5.13.jar:4.5.13]
	at org.apache.http.conn.EofSensorInputStream.close(EofSensorInputStream.java:172) ~[httpclient-4.5.13.jar:4.5.13]
	at org.apache.http.client.entity.LazyDecompressingInputStream.close(LazyDecompressingInputStream.java:97) ~[httpclient-4.5.13.jar:4.5.13]
	at com.google.api.client.http.HttpResponse.ignore(HttpResponse.java:427) ~[google-http-client-1.39.2.jar:1.39.2]
	at com.google.api.client.http.HttpResponse.disconnect(HttpResponse.java:441) ~[google-http-client-1.39.2.jar:1.39.2]

Any additional information below

Does not happen on 1.39.0. The change that cause the issue was f84ed59

[benketteridge]

This may very well also be causing Quarkus builds to fail to push to Google container registries.
Note the user-agent: "Google-HTTP-Java-Client/1.39.2"

curl -v --compressed -X POST -H 'Accept: ' -H 'Accept-Encoding: gzip' -H 'Authorization: <Not Logged>' 
-H 'User-Agent: jib 0.13.1 Quarkus Google-HTTP-Java-Client/1.39.2 (gzip)' -- 'https://eu.gcr.io/v2/[REDACTED]/[REDACTED]/blobs/uploads/'

HTTP/1.1 202 Accepted
Docker-Distribution-API-Version: registry/2.0
Docker-Upload-UUID: [REDACTED]
Location: https://eu.gcr.io/v2/[REDACTED]/[REDACTED]/blobs/uploads/[REDACTED]
Content-Type: application/json
Date: Wed, 14 Jul 2021 07:16:51 GMT
Server: Docker Registry
Cache-Control: private
X-XSS-Protection: 0
X-Frame-Options: SAMEORIGIN
Transfer-Encoding: chunked

I/O error for image [eu.gcr.io/[REDACTED]/[REDACTED]]:
    org.apache.http.ConnectionClosedException
    Premature end of chunk coded message body: closing chunk expected

Using 1.38.1 is successful with no other changes.

[chanseokoh]

google-http-client version(s): 1.39.1 and 1.39.2
Does not happen on 1.39.0.

Exactly. A lot of Jib users are hitting this issue.

[chanseokoh]

Alright, anyone can easily reproduce this with https://gcr.io. (See GoogleContainerTools/jib#3409 (comment) if you think you can't reproduce.)

    new ApacheHttpTransport().createRequestFactory()
        .buildGetRequest(new GenericUrl("https://gcr.io")).execute().disconnect();

Caused by: org.apache.http.TruncatedChunkException: Truncated chunk (expected size: 2,724; actual size: 229)
    at org.apache.http.impl.io.ChunkedInputStream.read (ChunkedInputStream.java:197)
    at org.apache.http.impl.io.ChunkedInputStream.read (ChunkedInputStream.java:210)
    at org.apache.http.impl.io.ChunkedInputStream.close (ChunkedInputStream.java:312)
    at org.apache.http.impl.execchain.ResponseEntityProxy.streamClosed (ResponseEntityProxy.java:142)
    at org.apache.http.conn.EofSensorInputStream.checkClose (EofSensorInputStream.java:228)
    at org.apache.http.conn.EofSensorInputStream.close (EofSensorInputStream.java:172)
    at org.apache.http.client.entity.LazyDecompressingInputStream.close (LazyDecompressingInputStream.java:97)
    at com.google.api.client.http.HttpResponse.ignore (HttpResponse.java:427)
    at com.google.api.client.http.HttpResponse.disconnect (HttpResponse.java:441)
    at Main.main (Main.java:242)

@chingor13 @rawler I think this is a critical bug, and #1303 should be reverted.