Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save zplume/9f4c1a658517802701deff3473f23a60 to your computer and use it in GitHub Desktop.
Save zplume/9f4c1a658517802701deff3473f23a60 to your computer and use it in GitHub Desktop.

Moving Files with SharePoint Online REST APIs in Microsoft Flow

About

This is an overview of how to move files with Microsoft Flow using the SharePoint - Send an HTTP request to SharePoint action and SharePoint Online REST APIs.

These SharePoint Online REST APIs can be used outside of Flow, but you will need to handle authentication and provide an X-RequestDigest header with any POST requests (as with any SP REST POST request).

In the case of Flow's SharePoint - Send an HTTP request to SharePoint action, authentication is handled for you and you do not need to provide the X-RequestDigest header.

Why

This is something I came across while building a two-stage approval workflow with Flow, which had the following requirements:

  • Allow users to submit a document for approval (using Flow's SharePoint - For a selected item trigger)
  • When triggered, the Flow should create a copy of the document in a Stage 1 Approval folder (in the same library) and start an approval process on the approval copy of the document via the Approvals - Start an approval action
    • If approved at stage 1, move the approval copy of the document to a Stage 2 Approval folder (in the same library) and start another approval process
    • If rejected, move the approval copy of the document to a Rejected folder
  • If approved at stage 2, move the approval copy of the document to an Approved folder

While Flow already had a SharePoint - Copy file action, there wasn't (at least when I wrote this) a SharePoint - Move file action.

At this point I started looking at using the SharePoint - Send an HTTP request to SharePoint action (the most useful Flow SharePoint action?) with SharePoint Online REST APIs to move files.

Several samples online (including Microsoft documentation) suggest the following API call (or some variation) to move a file:

https://tenant.sharepoint.com/_api/web/getFileByServerRelativeUrl('/Shared Documents/FileName.docx')/moveTo(newurl='/Shared Documents/FolderName/FileName.docx',flags=1)

In the example above, flags=1 means overwrite an existing file at newurl if there is one, rather than returning an error.

This was fine in early development, but with more testing I found the following issues:

  • Long file names / paths can quickly exceed URL limits causing the API request to fail
  • ' characters in file/folder names are problematic

I spent some time trying to work around these issues, but was asking myself why I was battling with sending POST data in the request URL that should really (in my humble opinion) be included in the POST body instead.

And then I started thinking about how SharePoint Online's modern document library UI has a 'Move To' button...

Inspecting SharePoint API traffic

If you're curious about how SharePoint Online's modern UI uses SharePoint REST APIs, you can inspect the Network traffic sent from the SharePoint page using either the Network tab of your browser developer tools, or a debugging proxy like Telerik's excellent (and free) Fiddler.

This can be a useful technique for discovering as-yet undocumented APIs or how to use APIs that are documented but not explained clearly enough with examples.

In this post I'll focus on using the Network tab of Chrome's developer tools, but the same principals should apply to other browser development tools.

To only show API requests, you'll want to filter traffic shown in the Network tab by clicking on the XHR heading. You can additionally filter by a text value in the 'Filter' text box.

SP.MoveCopyUtil in the modern UI

If you navigate to a SharePoint Online document library that's configured to show the modern UI and (while inspecting browser network traffic) use the 'Move To' button to move a file to a different folder in the same library, you'll see the following REST request:

https://tenant.sharepoint.com/_api/SP.MoveCopyUtil.MoveFileByPath()

This endpoint accepts source and destination URLs in the request body rather than the request URL, which means it doesn't have the drawbacks of the getFileByServerRelativeUrl('/from/')/moveTo(newUrl='/to/') approach. 🎉

If you try and move a file to a folder that already contains a file with that name, the API request will return an error (HTTP 400) and you'll see the prompt A file with this name already exists with an option to replace the existing one. If you click Replace, the request is sent again with an additional parameter:

https://tenant.sharepoint.com/_api/SP.MoveCopyUtil.MoveFileByPath(overwrite=@a1)?@a1=true

If you inspect the request body (in Chrome this is via Headers > Request payload), you'll see something like the following JSON:

{
    "srcPath": {
        "__metadata": {
            "type": "SP.ResourcePath"
        },
        "DecodedUrl": "https://tenant.sharepoint.com/Shared Documents/FileName.docx"
    },
    "destPath": {
        "__metadata": {
            "type": "SP.ResourcePath"
        },
        "DecodedUrl": "https://tenant.sharepoint.com/Shared Documents/FolderName/FileName.docx"
    }
}

As you can see, the DecodedUrl values are absolute URLs that include the source and destination file names.

Recreating the SP.MoveCopyUtil.MoveFileByPath request

You can recreate this request in Flow using the Data Operations - Compose and SharePoint - Send an HTTP request to SharePoint actions, as follows (using the example JSON above as a starting point for the Data Operations - Compose action's Inputs value):

Moving a SharePoint file with Flow

Creating the request body in a separate Data Operations - Compose action makes debugging the Flow easier if something goes wrong with the request.

When configuring the SharePoint - Send an HTTP request to SharePoint action, you should specify the following values:

Properties

Property Value
Site Address The SharePoint site URL
Method POST
Uri _api/SP.MoveCopyUtil.MoveFileByPath(overwrite=@a1)?@a1=true
Headers See Headers table below
Body The Output of the Data Operations - Compose action

Note: swap out the Uri value for _api/SP.MoveCopyUtil.MoveFileByPath() if you don't want to overwrite an existing file in the destination.

Headers

Name Value
Accept application/json; odata=nometadata
Content-Type application/json; odata=verbose

Is that it?

Well, almost. If you only want to move files from one location in a document library to another location in the same document library, that's pretty much the end of the story.

But what about...

Copying files

If you click the Copy To button in the modern library UI while inspecting browser network traffic, you'll see the following request (or a variation thereof):

https://tenant.sharepoint.com/_api/SP.MoveCopyUtil.CopyFileByPath()

As with SP.MoveCopyUtil.MoveFileByPath, you can recreate this request in Flow with the SharePoint - Send an HTTP request to SharePoint action to copy files.

Metadata and version history

Move To - in the same library

When moving files from one location in a library to another location in the same library, the list item associated with the source and destination file is the same, meaning that version history and field values are automatically preserved, as you are only updating the file location within the library.

Move To - in another library, site or site collection

If you use the above approach to move a file from one library to another library, or even to another site or site collection, the list item associated with the source and destination file are not the same (think of it as a 'copy' followed by a 'delete' of the original file) and version history is not maintained.

In this case field values will be replicated from source to destination (if possible).

However, although it's possible to move files across site collections with SP.MoveCopyUtil.MoveFileByPath, if you move a file from one site collection to another via the modern UI's Move To button while looking at your browser's network traffic, you'll notice that it switches to a different set of APIs...

CreateCopyJobs and GetCopyJobProgress

_api/site/CreateCopyJobs
_api/site/GetCopyJobProgress

I've experimented a little with recreating the CreateCopyJobs and GetCopyProgress APIs in Flow, but these probably deserve their own post. So far I have noticed the following from inspecting network traffic while moving files from one site collection to another via the Move To button and recreating the calls from Flow:

  • The CreateCopyJobs API will return an error if fields from the source list are missing in the destination list, unless an AllowSchemaMismatch parameter in the request body is specified as true. This is matched by the UI prompting you of the same (missing fields), with a button to proceed anyway, which then resends the request with the AllowSchemaMismatch parameter set to true.
  • Version history is copied when using CreateCopyJobs with the IgnoreVersionHistory parameter set to false. If the parameter is set to true, the destination file will be version 1.0.
  • If the destination file already exists when the IsMoveMode parameter is set to true, the copy job will fail (see the Logs when inspecting the output of GetCopyJobProgress). There doesn't appear to be an 'overwrite' parameter in the CopyMigrationOptions class to allow overwrite on a move. This also happens with the UI Move To button - the following error message is displayed: A file or folder with this name already exists. Please rename the file or folder and try again.

API references for CreateCopyJobs and GetCopyJobsProgress

Hopefully someone finds the above useful 🙂

Updates

Date Update
22nd June 2018 Updated to reference 'SharePoint Online' rather than 'SharePoint', as the post content has only been tested in that context
22nd June 2018 Updated info on CreateCopyJobs and GetCopyJobProgress based on further testing
@rdorrani
Copy link

rdorrani commented Aug 6, 2018

Hi,

Tried to Move file across site collections using _api/SP.MoveCopyUtil.MoveFileByPath - get 403 access denied although have all the permissions required

Tried to Move files across site collections using /_api/site/CreateCopyJobs - works only for text and image files. Word, excel, powerpnt throw the following error :
The object of type ListItem and URL '---' that was configured as part of the Export Settings does not exist. CallStack -- at Microsoft.SharePoint.Deployment.SPExport.AddExportObjectToTempTable(SqlCommand command, Guid siteId, Guid objectId, String url, SPDeploymentObjectType objectType, SPIncludeDescendants includeDescendants, Boolean isLink, Boolean isSelected)\r\n at Microsoft.SharePoint.Deployment.SPExport.SeedExportObjectsTable()\r\n at Microsoft.SharePoint.Deployment.SPExport.CalculateObjectsToExport()\r\n at Microsoft.SharePoint.Deployment.SPExport.Run()

Got this to work - i was using link to item for the file path which was embeddeing the open in client experience url - once i updated the path - the copyjobprogress worked well

@zplume
Copy link
Author

zplume commented Aug 12, 2018

@rdorrani - glad you got it working!

@anhhnguyen206
Copy link

Hi,

I couldn't get the overwrite property to work - I believe it's working fine last week but today even with the flag overwrite=true it fails and says 'The destination file already exists'.

Addtionally, i also don't see the replace option that we see if the file exists in SharePoint online anymore

@zplume
Copy link
Author

zplume commented Aug 15, 2018

Hi @anhhnguyen206, I still get the prompt to 'Replace' an existing file when using the Move To option in the SPO UI (if the destination file already exists).

The 'Replace' button is hidden behind a 'sync' icon that appears in the document library command bar, with a red circle containing a white X on top of it. Clicking the icon reveals the option to 'Keep both' or 'Replace'.

When I click 'Replace' this successfully overwrites the destination file. The _api request that is sent by the SPO UI is the same request format documented in this post.

At a guess, perhaps the library settings or permission configuration in your scenario is preventing overwrite, but I'm not sure specifically what could cause that.

@zplume
Copy link
Author

zplume commented Aug 15, 2018

Do you have versioning enabled in your destination document library? Is the destination library the same as the source library?

@zplume
Copy link
Author

zplume commented Aug 15, 2018

Is the destination file that you're trying to overwrite checked-out to someone else?

@mmohamedbasith
Copy link

Are CreateCopyJobs and GetCopyJobProgress offically released by microsoft as Api? Is it safe to use it?

@amonjohnson
Copy link

Got this to work - i was using link to item for the file path which was embeddeing the open in client experience url - once i updated the path - the copyjobprogress worked well

Hi, can you explain this a bit more please? I'm getting the same error, can you share an example of how you changed the path? Thanks in advance.

@zplume
Copy link
Author

zplume commented May 17, 2019

Hi @amonjohnson, have you seen the follow-up blog post I wrote on the CreateCopyJobs API?
https://gist.github.com/zplume/21248c3a8a5f840a366722442cf9ee97

As mentioned in that post, there is a newer Flow 'Copy File' action that you can use instead of calling the CreateCopyJobs API (assuming you are working with Flow), but if you need to use REST for some reason the approach is detailed in the above post.

Both posts are based on looking at the network requests that SharePoint Online sends when you use the Move/Copy UI buttons in a document library (via the Chrome developer tools) and replicating those requests. If you're having issues I'd suggest comparing the content of your requests with those sent by SharePoint to figure out the discrepancy.

@amonjohnson
Copy link

Hi @amonjohnson, have you seen the follow-up blog post I wrote on the CreateCopyJobs API?
https://gist.github.com/zplume/21248c3a8a5f840a366722442cf9ee97

As mentioned in that post, there is a newer Flow 'Copy File' action that you can use instead of calling the CreateCopyJobs API (assuming you are working with Flow), but if you need to use REST for some reason the approach is detailed in the above post.

Both posts are based on looking at the network requests that SharePoint Online sends when you use the Move/Copy UI buttons in a document library (via the Chrome developer tools) and replicating those requests. If you're having issues I'd suggest comparing the content of your requests with those sent by SharePoint to figure out the discrepancy.

Thank you zplume, I really appreciate your response and will check out the other post.
To better explain my scenario, I'm trying to:

  1. Select a file in Library 1
  2. Use a Flow to Copy that file to Library 2, and also dynamically create a duplicate of the original file directory into Library 2.
    Example: /sites/dessertSite/cakes/best-kinds/chocolate.pdf
    Using FLow, I'd like to be able to copy to Library 2.
    /sites/dessertSite/desserts/best-kinds/chocolate.pdf

If I hardcode the Destination directory it works great. But when I use a variable for the Folder Path, it seems like maybe the trailing slash is causing the folder creation to fail.

@amonjohnson
Copy link

Hi @amonjohnson, have you seen the follow-up blog post I wrote on the CreateCopyJobs API?
https://gist.github.com/zplume/21248c3a8a5f840a366722442cf9ee97
As mentioned in that post, there is a newer Flow 'Copy File' action that you can use instead of calling the CreateCopyJobs API (assuming you are working with Flow), but if you need to use REST for some reason the approach is detailed in the above post.
Both posts are based on looking at the network requests that SharePoint Online sends when you use the Move/Copy UI buttons in a document library (via the Chrome developer tools) and replicating those requests. If you're having issues I'd suggest comparing the content of your requests with those sent by SharePoint to figure out the discrepancy.

Thank you zplume, I really appreciate your response and will check out the other post.
To better explain my scenario, I'm trying to:

  1. Select a file in Library 1
  2. Use a Flow to Copy that file to Library 2, and also dynamically create a duplicate of the original file directory into Library 2.
    Example: /sites/dessertSite/cakes/best-kinds/chocolate.pdf
    Using FLow, I'd like to be able to copy to Library 2.
    /sites/dessertSite/desserts/best-kinds/chocolate.pdf

If I hardcode the Destination directory it works great. But when I use a variable for the Folder Path, it seems like maybe the trailing slash is causing the folder creation to fail.

@zplume Here's the situation I'm speaking about, the Yellow highlight is where I think the issue is. The FolderPath variable is adding a trailing slash. If I hardcode the folder as "NewFolder", then it works fine.
Thanks for reading.

image

@amonjohnson
Copy link

Hi @zplume,
I was able to resolve the issue by doing a "Create File" action, using the meta data from the original file, instead of a "Copy File" action.
Thanks for all of your assistance.

@zplume
Copy link
Author

zplume commented May 17, 2019

@amonjohnson - in my testing (looking at the Flow run history) the Folder path property of the Get file properties action output includes both the folder and the parent library name ("HT" in my test) and I'm assuming you don't want the library URL in your destination.

"HT/SourceFolder/"

image

If you add a Get folder metadata using path action and pass in the Folder path returned from Get file properties, you can retrieve the folder Name without the library URL.

image

I was able to use the Name property returned from Get folder metadata using path in the Destination Folder input of the Copy file action to copy a file to that folder in the destination library (assuming the folder already exists there).

image

You'd need to do a bit more work (perhaps with expressions) for handling nested folders in the source location.

@DTGitRepository
Copy link

Hi @zplume,

We have a similar setup - slight difference in that we use a field to manage publishing with a scheduled flow that looks for items to move to 'Publish' folders (Staging [pending approval] -> Published to Dpmt OR Published to All)

So for the 'Published to' folders we have it set with unique permissions for the relevant groups.
A problem that has been raised is where the item has been shared initially then gets moved. Apparently when moved (the flow uses SP.MoveCopyUtil.MoveFileByPath) the file retains the unique permissions.

Any ideas on how to merge or inherit the permissions from the destination folder?
Ideally looking for some parameter or payload for the API request but I appreciate I may need to add an additional step before/after the move.

@kruismanp
Copy link

Hi,

Does flow have restictions moving large files (1GB + )?

@SirLac
Copy link

SirLac commented Jan 14, 2020

Hi,
does anyone know why it is not possible to move a file from a Document Library to another Document Library? I mean without the workaround of copying original file, pasting it in the destination location and, finally, deleting the original one, in a way to save, for example, history version and others metadata or shared urls.

Thanks in advance.

@zplume
Copy link
Author

zplume commented Jan 15, 2020

Hi @DTGitRepository, I just saw your message - I appreciate this is a bit late, but if you're still looking for a way to reset permissions for a file/item in SharePoint via Flow, give me a shout on Twitter: https://twitter.com/lsspdev

@zplume
Copy link
Author

zplume commented Jan 15, 2020

Hi @kruismanp, SharePoint itself should be able to handle moving large files I think (e.g. via the SharePoint document library UI).

I haven't done any testing around various APIs / Flow actions to compare their handling of large files - if you look into this yourself, I'd be interested to know your findings!

@zplume
Copy link
Author

zplume commented Jan 15, 2020

Hi @SirLac, have you seen my followup post on CreateCopyJobs?
https://gist.github.com/zplume/21248c3a8a5f840a366722442cf9ee97

The API is a little more complex to use but did copy version history in my testing (see post linked above for details).
It might be worth testing the SharePoint 'Copy file' and 'Move file' actions in Flow to see if either of these replicates the version history in the destination item.

@johnnyshield
Copy link

johnnyshield commented Apr 30, 2021

Hi @zplume, great post - thanks for sharing.
If anyone is wondering how to move a file between folders and "Keep Both" files, which renames the moved file by appending a 1 to the filename, the following worked for me:

Compose step

{
    "srcPath": {
        "__metadata": {
            "type": "SP.ResourcePath"
        },
        "DecodedUrl": "Server/Lib/SourceFolder/Doc.doc"
    },
    "destPath": {
        "__metadata": {
            "type": "SP.ResourcePath"
        },
        "DecodedUrl": "Server/Lib/DestFolder/Doc.doc"
    },
    "options": {
        "__metadata": {
            "type": "SP.MoveCopyOptions"
        },
        "KeepBoth": true,
        "ResetAuthorAndCreatedOnCopy": false,
        "ShouldBypassSharedLocks": true
    }
}

HTTP request step
Uri is just _api/SP.MoveCopyUtil.MoveFileByPath()

image

@skmuth
Copy link

skmuth commented Mar 12, 2023

Hi there

Im struggling with the last steps to move an item list to another folder... Within my flow, the HTTP function takes a along time to process (came up as bad gateway)... Is there any reason why?

The list items which I am hoping to move will have attachments which will be no more than 25 - 30MB.

image

Your assistance in this matter is highly appreciated!

Finally got it to work.....

@amitpaple
Copy link

Hi @zplume, Firstly Thanks for sharing this wonderful blog, but just wanted to check if there is a way to move or copy Multiple Folder's/File's from one folder to another folder ?

@insuganhau
Copy link

I tried using this example to move files from Sharepoint Online to Sharepoint On-prem, but without success.
Do you know how to achieve it with Power Automate?

@viplavmulka
Copy link

The parameter exportObjectUris does not exist in method CopyFileByPath

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment