Repository navigation
Upload Large File via LargeFileUploadTask not working #1806
Description
Activity
The Mistake is at this point .byDriveItemId("root/fileName")
Try .byDriveItemId("root:/fileName:")
Then you will avoid this cliff, but you will not get any further with upload, because it always returns a ReturnCode 400 with the message "com.microsoft.kiota.ApiException: generalException". I have tried a lot, e.g. omitting the name and size in DriveItemUploadableProperties, but it didn't help. The first chunk is uploaded to the server and then the error code 400 appears.I always get the return code 400 when uploading.
I have tried it with different DriveItemUploadableProperties and maxslicesice of 327680 but unfortunately without success.
DriveItemUploadableProperties:{ "item": { "@microsoft.graph.conflictBehavior": "replace" } }or
{ "item": { "@microsoft.graph.conflictBehavior": "replace" }, "deferCommit": true }or
{ "item":{ "description":"test", "fileSize":8589114, "name":"test.log", "@microsoft.graph.conflictBehavior":"replace" } }If I use the method to upload small files, it works well
RequestInformation requestInformation = graphClient.drives().byDriveId(drive.getId()).items() .byDriveItemId("root:/" + destination + ":").content().toPutRequestInformation(inputstream); URI uriIncludesConflictBehavior = new URI( requestInformation.getUri().toString() + "?@microsoft.graph.conflictBehavior=rename"); requestInformation.setUri(uriIncludesConflictBehavior); graphClient.getRequestAdapter().sendPrimitive(requestInformation, null, InputStream.class);Hi vectorius (@araneolus)
Also with byDriveItemId("root:/fileName:") not wokring
Thanks,
ItayWhich error do you got ? Item already exists, or a DateTimeException ?
Hi vectorius (@araneolus)
Always datetime errorYou have to use the com.microsoft.kiota Version 1.0.2.
The cause you can find at #1785At the console I see following Debug Message, before I receive the 400 error code
14 Feb. 2024 13:38:33.573- Http2 [] DEBUG okhttp3.internal.http2.Http2Writer 258 frameHeader - >> 0x00000003 16384 DATA 14 Feb. 2024 13:38:33.599- Http2 [] DEBUG okhttp3.internal.http2.Http2Writer 258 frameHeader - >> 0x00000003 16384 DATA 14 Feb. 2024 13:38:33.625- Http2 [] DEBUG okhttp3.internal.http2.Http2Writer 258 frameHeader - >> 0x00000003 16384 DATA 14 Feb. 2024 13:38:33.651- Http2 [] DEBUG okhttp3.internal.http2.Http2Writer 258 frameHeader - >> 0x00000003 3898 DATA END_STREAM 14 Feb. 2024 13:38:33.656- Http2 [OkHttp xx-my.sharepoint.com] DEBUG okhttp3.internal.http2.Http2Reader 112 nextFrame - << 0x00000003 4 WINDOW_UPDATE 14 Feb. 2024 13:38:33.656- Http2 [OkHttp xx-my.sharepoint.com] DEBUG okhttp3.internal.http2.Http2Reader 112 nextFrame - << 0x00000000 4 WINDOW_UPDATE 14 Feb. 2024 13:38:33.661- Http2 [OkHttp xx-my.sharepoint.com] DEBUG okhttp3.internal.http2.Http2Reader 112 nextFrame - << 0x00000003 1149 HEADERS END_HEADERS 14 Feb. 2024 13:38:33.663- Http2 [OkHttp xx-my.sharepoint.com] DEBUG okhttp3.internal.http2.Http2Reader 112 nextFrame - << 0x00000003 174 DATA 14 Feb. 2024 13:38:33.663- Http2 [OkHttpxx-my.sharepoint.com] DEBUG okhttp3.internal.http2.Http2Reader 112 nextFrame - << 0x00000003 0 DATA END_STREAMHi vectorius (@araneolus)
Now I am getting
Exception in thread "main" com.microsoft.graph.models.odataerrors.ODataError: Invalid request
at com.microsoft.graph.models.odataerrors.ODataError.createFromDiscriminatorValue(ODataError.java:36)
at com.microsoft.kiota.serialization.JsonParseNode.getObjectValue(JsonParseNode.java:210)
at com.microsoft.kiota.http.OkHttpRequestAdapter.lambda$throwIfFailedResponse$0(OkHttpRequestAdapter.java:672)
at com.microsoft.kiota.ApiExceptionBuilder.(ApiExceptionBuilder.java:26)
at com.microsoft.kiota.http.OkHttpRequestAdapter.throwIfFailedResponse(OkHttpRequestAdapter.java:671)
at com.microsoft.kiota.http.OkHttpRequestAdapter.send(OkHttpRequestAdapter.java:279)
at com.microsoft.graph.drives.item.items.item.createuploadsession.CreateUploadSessionRequestBuilder.post(CreateUploadSessionRequestBuilder.java:60)
at com.microsoft.graph.drives.item.items.item.createuploadsession.CreateUploadSessionRequestBuilder.post(CreateUploadSessionRequestBuilder.java:45)
at com.bmc.cm.aft.utils.SPOUtilsTest.main(SPOUtilsTest.java:116)Hi vectorius (@araneolus)
I added the updated graph core to 3.1.2 and now I am getting new error:
Exception in thread "main" com.microsoft.kiota.ApiException: generalException
at com.microsoft.kiota.ApiExceptionBuilder.withMessage(ApiExceptionBuilder.java:45)
at com.microsoft.graph.core.requests.upload.UploadResponseHandler.handleResponse(UploadResponseHandler.java:61)
at com.microsoft.graph.core.requests.upload.UploadSliceRequestBuilder.put(UploadSliceRequestBuilder.java:69)
at com.microsoft.graph.core.tasks.LargeFileUploadTask.uploadSlice(LargeFileUploadTask.java:207)
at com.microsoft.graph.core.tasks.LargeFileUploadTask.upload(LargeFileUploadTask.java:131)
at com.microsoft.graph.core.tasks.LargeFileUploadTask.upload(LargeFileUploadTask.java:111)
at com.bmc.cm.aft.utils.SPOUtilsTest.main(SPOUtilsTest.java:138)You are now as far along as I am. Welcome to the club :-)
This is the structure of the 400 Response
url=https://xx-my.sharepoint.com/personal/hugo_musterman_xxx_de/_api/v2.0/drives/.../items/.../uploadSession?guid=%27...%27&overwrite=True&rename=False&dc=0&tempauth=...}28 remaining items
Hi Vincent Biret (@baywet)
I tried latest kiota 1.1.1
and I am gettng this error:
Exception in thread "main" java.lang.IndexOutOfBoundsException
at java.base/java.io.FileInputStream.readBytes(Native Method)
at java.base/java.io.FileInputStream.read(FileInputStream.java:279)
at com.microsoft.graph.core.tasks.LargeFileUploadTask.chunkInputStream(LargeFileUploadTask.java:280)
at com.microsoft.graph.core.tasks.LargeFileUploadTask.uploadSlice(LargeFileUploadTask.java:204)
at com.microsoft.graph.core.tasks.LargeFileUploadTask.upload(LargeFileUploadTask.java:131)
at com.microsoft.graph.core.tasks.LargeFileUploadTask.upload(LargeFileUploadTask.java:111)
at com.bmc.cm.aft.utils.SPOUtilsTest.main(SPOUtilsTest.java:285)Seems that when working with chunks the chunkInputStream in LargeFileUploadTask always set the begin instead of 0 every time to store to the buffer increase it every slice so the begin can be greater than the buffer.
begin should be zero.
private byte[] chunkInputStream(InputStream stream, int begin, int length) throws IOException {
byte[] buffer = new byte[length];
int lengthAssert = stream.read(buffer, begin, length);
assert lengthAssert == length;
return buffer;
}If I modified the begin to be zero the upload succeeded.
But when I try to make the part 3 MB sometimes it's failed with GeneralException
Exception in thread "main" com.microsoft.kiota.ApiException: generalException
at com.microsoft.kiota.ApiExceptionBuilder.withMessage(ApiExceptionBuilder.java:45)
at com.microsoft.graph.core.requests.upload.UploadResponseHandler.handleResponse(UploadResponseHandler.java:62)
at com.microsoft.graph.core.requests.upload.UploadSessionRequestBuilder.get(UploadSessionRequestBuilder.java:52)
at com.microsoft.graph.core.tasks.LargeFileUploadTask.updateSessionStatus(LargeFileUploadTask.java:197)
at com.microsoft.graph.core.tasks.LargeFileUploadTask.upload(LargeFileUploadTask.java:140)
at com.microsoft.graph.core.tasks.LargeFileUploadTask.upload(LargeFileUploadTask.java:111)
at com.bmc.cm.aft.utils.SPOUtilsTest.main(SPOUtilsTest.java:285)Thanks,
ItayThanks everyone for trying the new version and reporting the results, it helps!
We were not expecting 1.1.0 or 1.1.1 to solve this issue, right now most of the code changes to address this issue are in place with microsoft/kiota-java#1119. We're performing thorough integration tests to make sure all scenarios and different service behaviours are covered by the fix.
Thanks everyone for your patience!Reacted by Guillaume ToisonHi Vincent Biret (@baywet),
I have the same issue as ihudedi:If I modified the begin to be zero the upload succeeded. But when I try to make the part 3 MB sometimes it's failed with GeneralException Exception in thread "main" com.microsoft.kiota.ApiException: generalException at com.microsoft.kiota.ApiExceptionBuilder.withMessage(ApiExceptionBuilder.java:45) at com.microsoft.graph.core.requests.upload.UploadResponseHandler.handleResponse(UploadResponseHandler.java:62) at com.microsoft.graph.core.requests.upload.UploadSessionRequestBuilder.get(UploadSessionRequestBuilder.java:52) at com.microsoft.graph.core.tasks.LargeFileUploadTask.updateSessionStatus(LargeFileUploadTask.java:197) at com.microsoft.graph.core.tasks.LargeFileUploadTask.upload(LargeFileUploadTask.java:140) at com.microsoft.graph.core.tasks.LargeFileUploadTask.upload(LargeFileUploadTask.java:111) at com.bmc.cm.aft.utils.SPOUtilsTest.main(SPOUtilsTest.java:285)
in my case it happens for files that are fully uploaded in one slice, it looks that sometimes (not always) while trying to updateSesssionStatus, the uploadSession is not there any more. The body of the response is:
{"error":{"code":"itemNotFound","message":"The upload session was not found"}}
I'm on v.6.5.1.
There is also an open ticket for this issue at
microsoftgraph/msgraph-sdk-java-core#1531Hi Vincent Biret (@baywet) Fernando F Costa (@ffcdf)
I saw in microsoftgraph/msgraph-sdk-java-core#1531
that you are changing the type for the upload from InputStream to FileInputStream.
I don't upload from local and I need that that stream will be inputstream because I can't upload large file not from local.
Thanks,
ItayHi ihudedi.
The first idea was actually this.
However, following Philip Gichuhi (@Ndiritu)'s suggestion, a new commit and a new pull request were made that resolved the error without significant changes. The correction basically consists of changing the line:
int lengthAssert = stream.read(buffer, begin, length);for:
int lengthAssert = stream.read(buffer);I believe that the next version of msgraph-sdk-java-core will already bring this correction.
Hi Petr Hollay (@petrhollayms)
Any updates?My example is from the document. Why do I sometimes succeed when uploading files:
Uploaded 13538 bytes of 13538 total bytes
Upload complete
Item ID: 015CS7XKADN44E66JPPRBZYOIE7IMTPOTK
Sometimes, although there may be error messages, OneDrive has already been uploaded:
Uploaded 13538 bytes of 13538 total bytes
Exception in thread "main" com.microsoft.kiota.ApiException: generalException
at com.microsoft.kiota.ApiExceptionBuilder.withMessage(ApiExceptionBuilder.java:45)
at com.microsoft.graph.core.requests.upload.UploadResponseHandler.handleResponse(UploadResponseHandler.java:62)
at com.microsoft.graph.core.requests.upload.UploadSessionRequestBuilder.get(UploadSessionRequestBuilder.java:52)
at com.microsoft.graph.core.tasks.LargeFileUploadTask.updateSessionStatus(LargeFileUploadTask.java:197)
at com.microsoft.graph.core.tasks.LargeFileUploadTask.upload(LargeFileUploadTask.java:140)
at net.tjeerd.onedrive.OneDriveDemo.main(OneDriveDemo.java:102)May I ask what the reason is? I spent a lot of time but there was no result
Hi @lllllllhz
I have the same issue.
Seems like there is an issue when trying to updateSessionStatus.
You can for now ignore the error if the file was uploaded completely.
I am also waiting for a fix for long time.
Thanks,
Itayihudedi
Thanks for your replyihudedi, another cause of the exception after the upload is specifying the wrong expected return type on successful completion of the upload task.
For OneDrive uploads, the return type should be a
DriveItemobject after the last chunk of the file is uploaded successfully rather than aDriveItemUploadablePropertiesobject. The wrong return type would cause deserialization of the final payload to fail & a now completed/non-existent uploadSession gets updated throwing the exception.I realize this may have been guided by our Upgrade Guide code sample. There's a fix to point to the docs for an accurate sample.
Hi Philip Gichuhi (@Ndiritu)
I encountered the same issue when using DriveItem instead of DriveItemUploadableProperties
// Set body of the upload session request
DriveItemUploadableProperties driveItemUploadableProperties = new DriveItemUploadableProperties();
Map<String, Object> additionalData = new HashMap<>();
additionalData.put("@microsoft.graph.conflictBehavior", "replace");
driveItemUploadableProperties.setAdditionalData(additionalData);CreateUploadSessionPostRequestBody uploadSessionPostRequestBody = new CreateUploadSessionPostRequestBody(); uploadSessionPostRequestBody.setItem(driveItemUploadableProperties); // Create an upload session UploadSession uploadSession = graphServiceClient.drives() .byDriveId("b!8Y-KT6ZRA0qPRfGPGztHMmxPUHODkTZHgWOrFidiD475p3XU86ktQZsM9dIWmnXx") .items() .byDriveItemId("root:/tools.jar:") .createUploadSession()/*.withUrl("https://graph.microsoft.com/v1.0/sites/MSFT22/ItayLibrary/bambi")*/.post(uploadSessionPostRequestBody); if (null == uploadSession) { throw new SPOException("SPOWrapper::uploadObjectInParallel: Could not create upload session"); } String originalFilePath = "tools.jar"; FileInputStream inputStream = new FileInputStream("C:\\tools.jar"); // Create the large file upload task LargeFileUploadTask<DriveItem> uploadTask = new LargeFileUploadTask(graphServiceClient.getRequestAdapter(), uploadSession, inputStream, 18265082, 5 * Constants.MB, DriveItem::createFromDiscriminatorValue); // Create a callback used by the upload provider IProgressCallback callback = (current, max) -> System.out.println( String.format("Uploaded %d bytes of %d total bytes", current, max)); // Do the upload try { UploadResult<DriveItem> uploadResult = uploadTask.upload(); if (uploadResult == null || !uploadResult.isUploadSuccessful()) { throw new SPOException(String.format("SPOWrapper::uploadObjectInParallel: filePath= '%s' wasn't uploaded successfully", originalFilePath)); } else { logger.debug(String.format("SPOWrapper::uploadObjectInParallel: filePath= '%s' uploaded successfully", originalFilePath)); } } catch (Exception e){ e.printStackTrace(); }The error of general exception is always appeared when the size has chunks ( file size bigger than the chunk size)
Same issue occured when I used the DriveItemUploadableProperties class.
Seems it didn't solve the general exceptioncom.microsoft.kiota.ApiException: generalException
at com.microsoft.kiota.ApiExceptionBuilder.withMessage(ApiExceptionBuilder.java:45)
at com.microsoft.graph.core.requests.upload.UploadResponseHandler.handleResponse(UploadResponseHandler.java:62)
at com.microsoft.graph.core.requests.upload.UploadSessionRequestBuilder.get(UploadSessionRequestBuilder.java:52)
at com.microsoft.graph.core.tasks.LargeFileUploadTask.updateSessionStatus(LargeFileUploadTask.java:197)
at com.microsoft.graph.core.tasks.LargeFileUploadTask.upload(LargeFileUploadTask.java:140)
at com.microsoft.graph.core.tasks.LargeFileUploadTask.upload(LargeFileUploadTask.java:111)
at com.bmc.cm.aft.utils.SPOUtilsTest.main(SPOUtilsTest.java:190)When I rerun the code for the same file exists the error doesn't exists for DriveItem and DriveItemUploadableProperties
Thanks,
Itay
Hi,
I use your example and it's failed
I am failing when trying to create UploadSessin
I use in byDriveItemId also your example ("root/fileName" and it's not working
Thanks,
Itay