Skip to main content

Transferring data with rclone

Rclone can be configure via a GUI tool however here we cover the command line setup to connect to and sync files from Microsoft OneDrive, which all LSE members have access to.

First you will need to install rclone on your desktop or use our remote desktop service. This set is needed to authorize your connection.

Firstly add rclone to your environment with module add rclone

running rclone config will walk you through an interactive setup process:

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> n
name> myonedrive
Type of storage to configure.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
[snip]
XX / Microsoft OneDrive
   \ "onedrive"
[snip]
Storage> onedrive
Microsoft App Client Id
Leave blank normally.
Enter a string value. Press Enter for the default ("").
client_id>
Microsoft App Client Secret
Leave blank normally.
Enter a string value. Press Enter for the default ("").
client_secret>
Edit advanced config? (y/n)
y) Yes
n) No
y/n> n
Remote config
Use web browser to automatically authenticate rclone with remote?
 * Say Y if the machine running rclone has a web browser you can use
 * Say N if running rclone on a (remote) machine without web browser access
If not sure try Y. If Y failed, try N.
y) Yes
n) No
y/n> n
Option config_token.
For this to work, you will need rclone available on a machine that has
a web browser available.
For more help and alternate methods see: https://rclone.org/remote_setup/
Execute the following on the machine with the web browser (same rclone
version recommended):
	rclone authorize "onedrive"
Then paste the result.
Enter a value.

Now we run rclone authorize "onedrive" either on the remote desktop or your local machine. This will open a web browser and log you into Microsoft service. Switch back to your terminal and you will have a code to paste into rclone on Fabian.

Choose a number from below, or type in an existing value
 1 / OneDrive Personal or Business
   \ "onedrive"
 2 / Sharepoint site
   \ "sharepoint"
 3 / Type in driveID
   \ "driveid"
 4 / Type in SiteID
   \ "siteid"
 5 / Search a Sharepoint site
   \ "search"
config_type> 1

Option config_driveid.
Select drive you want to use
Choose a number from below, or type in your own string value.
Press Enter for the default (b!5ijNWH_a-series-of-letters-and-digits_R3jVAnuc).
 1 / OneDrive (business)
   \ (b!5ijNWH_a-series-of-letters-and-digits_R3jVAnuc)
config_driveid> 1

Drive OK?

Found drive "root" of type "business"
URL: https://lsecloud-my.sharepoint.com/personal/email_address_lse_ac_uk/Documents

y) Yes (default)
n) No
y/n>
--------------------
[remote]
type = onedrive
token = {"access_token":"youraccesstoken","token_type":"Bearer","refresh_token":"yourrefreshtoken","expiry":"2018-08-26T22:39:52.486512262+08:00"}
drive_id = b!a-series-of-letters-and-digits
drive_type = business
--------------------
y) Yes this is OK
e) Edit this remote
d) Delete this remote
y/e/d> y

Keep this "myonedrive" remote?
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote

y/e/d>

Current remotes:

Name                 Type
====                 ====
myonedrive           onedrive

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q

Once configured you can then use rclone like this,

List directories/folders in top level of your OneDrive

rclone lsd myonedrive:

List all the files in your OneDrive

rclone ls myonedrive:

To copy a local directory to an OneDrive directory called backup

rclone copy source myonedrive:backup

To sync a local directory called code with an OneDrive directory called code

rclone sync code myonedrive:code

You can use the same approach with Sharepoint Site including those that hold the files of a teams site.

Documentation for other cloud storage providers is available on the rclone website.

If the version of rclone you are using has changed please follow the official documentation.