Skip to content

Instantly share code, notes, and snippets.

@stokito

stokito/webdav_curl.md

Last active April 16, 2024 13:51
Show Gist options
  • Star 3 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save stokito/cf82ce965718ce87f36b78f7501d7940 to your computer and use it in GitHub Desktop.
Save stokito/cf82ce965718ce87f36b78f7501d7940 to your computer and use it in GitHub Desktop.
Download ZIP
WebDAV with curl for scripts and command line
Raw
webdav_curl.md

Assuming the following:

  • Webdav share URL: http://example.com/dav/
  • Username: user
  • Password: pass

curl options:

  • -u username:password use HTTP Basic authorization with the folowing username and password
  • -X GET send a request with GET method. You can use any other methods here.
  • -H 'Header: value' add a header. Some examples:
    • -H 'Authorization: Bearer <token>' add Authorization header with access token. Some APIs use it instead of Basic auth
    • -H 'Depth: 1' add a WebDAV Depth header used by a PROPFIND method.
    • -H 'Overwrite: T' used in COPY or MOVE methods to say that you really wish to overwrite an existing file.
    • -H 'Destination: http://example.com/dav/new.txt
  • -s silent output, use it in scripts to hide progress and other human messages
  • -v verbose mode: useful for debugging. Will show req/resp headers
  • -i show reponse headers. You may need it to get Content-Type or Content-Lenght

To list files in a directory

curl -u user:pass -X PROPFIND -H 'Depth: 1' http://example.com/dav/

The resulting dir listing will look in XML like:

<?xml version="1.0" encoding="UTF-8"?>
<D:multistatus xmlns:D="DAV:">
    <D:response>
        <D:href>/dav/Some%20Folder/</D:href>
        <D:propstat>
            <D:prop>
                <D:resourcetype>
                    <D:collection/>
                </D:resourcetype>
                <D:getlastmodified>Wed, 08 Mar 2023 12:45:54 GMT</D:getlastmodified>
                <D:supportedlock>
                    <D:lockentry>
                        <D:lockscope>
                            <D:exclusive/>
                        </D:lockscope>
                        <D:locktype>
                            <D:write/>
                        </D:locktype>
                    </D:lockentry>
                </D:supportedlock>
            </D:prop>
            <D:status>HTTP/1.1 200 OK</D:status>
        </D:propstat>
    </D:response>

    <D:response>
        <D:href>/dav/README.txt</D:href>
        <D:propstat>
            <D:prop>
                <D:resourcetype></D:resourcetype>
                <D:getcontentlength>64</D:getcontentlength>
                <D:getcontenttype>text/plain</D:getcontenttype>
                <D:getetag>"174a620a4078606440"</D:getetag>
                <D:getlastmodified>Wed, 08 Mar 2023 12:45:54 GMT</D:getlastmodified>
                <D:supportedlock>
                    <D:lockentry>
                        <D:lockscope>
                            <D:exclusive/>
                        </D:lockscope>
                        <D:locktype>
                            <D:write/>
                        </D:locktype>
                    </D:lockentry>
                </D:supportedlock>
            </D:prop>
            <D:status>HTTP/1.1 200 OK</D:status>
        </D:propstat>
    </D:response>
</D:multistatus>

So here we have a folder Some Folder but it's href e.g. path of URL is /dav/Some%20Folder/ and space was URL escaped to %20. Please note that MS IIS may return an absolute URL in the <href> tag. We know that this is a folder because it has <D:resourcetype> <D:collection/> </D:resourcetype>.

Next we have a README.txt file with href /dav/README.txt. It has <D:resourcetype></D:resourcetype> e.g. empty. It has a size Content-Length: 64 and this is a plain text file with Content-Type: text/plain.

In order to process the resulting XML you can install yq programm:

sudo snap install yq

Then you can redirect curl output into it to pretty format:

curl -s -u user:pass -X PROPFIND -H 'Depth: 1' http://example.com/dav/ | yq  --input-format xml --output-format xml

There are many other options to parse or format the XML https://stackoverflow.com/a/16090892/1049542

You can specify properties that you wish to retrieve:

curl -u user:pass -i -X PROPFIND http://example.com/dav/ -H 'Depth: 1' -H 'Content-Type: application/xml' -d @- << EOF
<?xml version="1.0" encoding="utf-8" ?>
<D:propfind xmlns:D="DAV:">
  <D:prop>
    <D:creationdate/>
    <D:displayname/>
    <D:getcontentlength/>
    <D:getcontenttype/>
    <D:getetag/>
    <D:getlastmodified/>
    <D:resourcetype/>
  </D:prop>
</D:propfind>
EOF

Get response code to check that operation was sucessful:

curl -X DELETE 'http://example.com/dav/file.txt' -sw '%{http_code}'

Upload a file into a directory:

curl -u user:pass -T ./file.txt http://example.com/dav/

Upload a file into a directory but specify the file name:

curl -u user:pass -T ./file.txt http://example.com/dav/new_file.txt

Create a folder

curl -u user:pass -X MKCOL 'http://example.com/dav/new_folder'

Delete a folder

curl -u user:pass -X DELETE 'http://example.com/dav/new_folder'

Delete a file

curl -u user:pass -X DELETE 'http://example.com/dav/somefile.txt'

Upload all files in a folder with shell script:

for file in /folder/path/*
do
  curl -u user:pass -T ${file} http://example.com/dav/
done

Renaming/Move

curl -X MOVE -H 'Overwrite: F' -H 'Destination: http://example.com/dav/new.txt' 'http://example.com/dav/old.txt'

Renaming/Move with overwrite of existing file:

curl -X MOVE -H 'Overwrite: T' -H 'Destination: http://example.com/dav/new.txt' 'http://example.com/dav/old.txt'

Get options supported by webdav server (note that auth is not needed)

curl -i -X OPTIONS http://example.com/dav/

Headers in output should looks like:

DAV: 1,2,3
Allow: PROPFIND, DELETE, MKCOL, PUT, MOVE, COPY, PROPPATCH, LOCK, UNLOCK, OPTIONS, GET, HEAD, POST
Access-Control-Allow-Origin: https://diffuse.sh
Access-Control-Allow-Credentials: true
Access-Control-Allow-Methods: OPTIONS, HEAD, GET, DELETE, PUT, PATCH, MKCOL, MOVE, COPY, LOCK, UNLOCK, PROPFIND, PROPPATCH
Access-Control-Expose-Headers: Content-Range, Date, Etag, Last-Modified, MS-Author-Via, DAV
Access-Control-Allow-Headers: Accept, Accept-Encoding, Accept-Language, Authorization, Accept-Range, Cache-Control, Content-Type, Connection, DNT, Range, Referer, TE, Depth, If, If-Modified-Since, If-Match, Destination, Overwrite
Access-Control-Max-Age: 600

Here

  • DAV: 1,2,3 shows compliance class 3 e.g. it support locks.
  • Allow shows which operations are supported. In this case all .
  • Access-Control-* are CORS headers if configured. In this case the site https://diffuse.sh can have an access to the webdav server.

Other terminal programs

  • rclone - a sync/backup tool. Supports WebDAV as a backup target and can itself act as a WebDAV server.
  • cadaver - a command-line interactive FTP-like WebDAV client
  • wget same a curl can be used to download or upload files. E.g. wget -q --user=admin --password=pass --auth-no-challenge --method=PROPFIND --header='Depth: 1' -O - http://example.com/dav/
  • https://pypi.org/project/webdavclient3/ Python library. Often it's much easier to just use it instead of bash.

See also

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment