authentication – Downloading files from third-party public group using REST API


A SharePoint short URL has been provided from third party:
https://{tenant_name}.sharepoint.com/:f:/s/{site_name}/{tokenized_path_element}?e={short_token}

Accessing it via browser gives me an office365 download page (no password required – as intended). I can download all available files via the browser, but I would like to do it programmatically in a script – either bash with curl CLI or Python. The web-interface mentions “Private group” in the top-left corner.

I am not familiar with either SharePoint nor its REST API. I have attempted a manual interpretation of the XML given by accessing https://{tenant_name}.sharepoint.com/sites/{site_name}/_api/Web/Lists. This did not get me much further.

There is also the question of authentication which confuses me. The site allows for anonymous access but still needs authentication.

The HTTP response headers includes a new Location:
https://{tenant_name}.sharepoint.com/sites/{site_name}/Shared%20Documents/Forms/AllItems.aspx={query}

where {query} is:
id=%2Fsites%2F{site_name}%2FShared%20Documents%2F{sub_path}$p=true$originalPath={long_token}

As expected perhaps, accessing this directly, gives me a login form with tenant logo (which I cannot use).

Experiments with curl on Ubuntu 18.04

#!/usr/bin/env bash
url="https://{tenant_name}.sharepoint.com/:f:/s/{site_name}/{tokenized_path_element}?e={short_token}"
cookie=$(curl -ks -c - -o /dev/null $url)
url_effective=$(echo "${cookie}" | curl -kLIs -b - -o /dev/null -w %{url_effective} $url)
output=$(echo "${cookie}" | curl -kLs -b - $url_effective)
echo $output

Above script uses the ability to get an authentication cookie, cache it in memory, and apply it again on the final (effective) URL. Not sure if this is a working solution? Above script spits out some 100kB of HTML5. I have loosely compared this source and what I get when visiting $url in the browser (FireFox). They are similar, but the curl-retrieved version contains no file links – it seems.

Do I need an extra layer of authentication?