Logging in via SSH and transferring data via SCP/SFTP/rsync
Whilst there are a number of ways to get a terminal on the ondemand website (including using jupyterlab, which probably has the best terminal) sometime it is useful to directly ssh or use ssh from your desktop tools.
The information below will configure your machine's ssh configuration to be able to connect to the service.
Uploading data
For small files Ondemand, the remote desktop and Jupyter Lab all offer file transfer options in their toolbars.
When you have a large number of files or large file (greater than 1G - one gigabyte) you will need to use other methods
If your data is on your local machine or a drive attached to it then using an SFTP client, such as cyberduck, can copy files to and from our storage.
If your data is in another cloud service, such as OneDrive/Sharepoint;g-drive;dropbox then rclone can be used to sync the data or mount it to a server. Please note that tools that attempt to sync files such as the one drive client or dropbox can not be used as they are designed to work on a single computer rather than a cluster of servers.
SSH Access and SFTP clients
We need to send us one or two pieces of information to allow you to connect to the service
- SSH Keypair, preferrably in ed25519 format though RSA and ecdsa are also supported, and send us the public half only which will be in a file with the extension .pub
- and, if you are not at the LSE campus, you will also need to know your public IP address which can be found at https://checkip.amazonaws.com/.
We are developing an application to allow you to manage keys yourself and that will detect your public IP address. Until that is ready please send us your details.
If you already have an SSH Keypair please send us the public half of that. If not then follow the next section to generate one.
- Mac or Linux
- Windows
Open a terminal and use it to run the following commands.
ssh-keygen -t ed25519
Open a Windows PowerShell and use it to run the following commands.
ssh-keygen -t ed25519
When creating your key you can protect it with a passphrase (password) of your choice. You will either have to use that passphrase when you use the key or configure your machine to store and use your passphrase for you.
Once we have let you know that your access is in place you need to configure your computer to access the service.
We are going to add some lines of config to your $HOME/.ssh/config file. If you know how to do that already simply copy the config sections from below, otherwise follow these steps.
The lines of config you need can be found in the ondemand application SSH Config you may need to initialise the app the first time you use it.
We can now test that we have access. Running the command ssh fab should give
you a terminal session on the login server (type exit to leave it). Configuring
your SFTP client to connect to server fabfiles and port 22619 should show
you your files and allow you to transfer files in both directions.
Users with lots of files to sync should investigate the rsync or rclone tools
Connection details
| Name | Value |
|---|---|
| Hostname | fab or fabfiles for file transfer via scp or sftp |
| Port | 22619 |
| Username | yor LSE useranme e.g. surnamei1 |
| Password | leave blank **do not enter your lse username |
| SSH Private Key | The Private key corresponding to the .pub file you have sent us |
Advanced configuration
Configure your machine to store and use your passphrase
- Mac
You can configure your mac to store your ssh key's passphrase in our Keychain
add the following section to your .ssh/config file
Host *
AddKeysToAgent yes
UseKeychain yes
run ssh-add -K once with your passphrase