The ISAAC Legacy (formerly ACF) cluster provides several ways for transferring files to/from the NFS home directories, Lustre project directories, and Lustre scratch directories. Data Transfer Nodes (DTNs) furnish this capability and are implemented with Globus and Secure Shell services. At the time of this writing, there are four DTNs available to ISAAC Legacy users. Table 1.1 shows these nodes.
|Data Transfer Node||Hostname (for SCP)||Globus endpoint|
|datamover1||datamover1.acf.utk.edu||UTK ISAAC Legacy 1|
|datamover2||datamover2.acf.utk.edu||UTK ISAAC Legacy 2|
|datamover3||datamover3.acf.utk.edu||UTK ISAAC Legacy 3|
|datamover4||datamover4.acf.utk.edu||UTK ISAAC Legacy 4|
Although these servers previously used an InCommon Credential, all four datamover DTNs are now running Globus Connect Server version 5.4 which uses UTK Central Authentication Service and no longer uses certificates.
To login and obtain a command line access to these DTNs, use an SSH client to login to the Hostname listed in the table above. More information on SSH usage can be found in the Access and Login document. Replace the hostname of the login node with the hostname of the DTN to which you wish to connect, then authenticate with your UT NetID, password, and Duo TFA.
The ISAAC Legacy cluster supports several data transfer protocols including SCP, SFTP, GSISCP, and Globus. SCP and SFTP are both SSH utilities available for transferring files but perform slower than Globus. At the time of this writing, Globus offers the fastest data transfers on the Open Enclave. Still, SCP and SFTP are useful for small transfers. GSISCP is used for unattended data transfers.
SCP and SFTP are both SSH commands available for transferring files to/from ISAAC Legacy using these SSH client commands. Note SSH transfers are slower than Globus transfers. From our performance testing, Globus offers the fastest data transfers to/from ISAAC Legacy DTNs. Still, SCP and SFTP can be useful for quick, small transfers. For larger file transfers, we recommend you use Globus.
SCP and SFTP are available to Linux and MacOS systems by default. Windows 10 users with the most recent updates can use these utilities within Command Prompt or PowerShell. Windows 7 and 8 users must use a third-party utility to use SCP and SFTP. For more information on SSH in Windows, see the Access and Login document. For Windows 7 and 8 users, the third-party utilities FileZilla and WinSCP are reviewed later in this document.
The general syntax of SCP is given below. In general, SCP is useful when transferring a file on your system to a DTN in ISAAC Legacy. The <source> argument is the pathname of the file on your system that you wish to copy. The <destination> (in this case, datamover1) argument is the hostname of the datamover you wish to use. Additionally, the <directory> argument specifies the absolute pathname within the destination to place the file.
scp <source> <NetID>@datamover1.acf.utk.edu:<directory>
If you wanted to copy a file from your system and place it on the ACF, you could use
scp ~/<filename> <NetID>@acf-login.acf.utk.edu:~/Documents
For SFTP, you specify the hostname of the system to which you intend to connect. For example, to securely transfer files between your local system and a DTN in ISAAC Legacy, use the syntax below in a terminal on your local system. Ensure that you enter SFTP from the directory that contains the file(s) you wish to copy to a DTN in ISAAC Legacy. You can use the pwd command to determine your current directory before entering SFTP.
Once you authenticate with your UT NetID, password, and Duo TFA, you will enter SFTP’s interactive mode. Use the
put <file> command to upload a file to the Open Enclave. For example, to upload a file named JobScript.sh to DTN from your local machine, use
put JobScript.sh. This syntax assumes that the JobScript.sh file is in the directory from which you entered SFTP.
To retrieve files from DTN, use the
get <file> command. To download a file named ResearchResults.txt from the DTN to your local machine, use
get ResearchResults.txt. SFTP will place the file in the directory from which you entered the utility. To change directories on the DTN, use the
cd <directory> command. Use the
lcd <directory> command to change the directory on your local system. Once you are done with SFTP, use the
exit commands to exit it. Other commands are available with the SFTP utility. Type
help within SFTP to read more about them.
The Globus web interface allows you to conveniently perform data transfers to and from ISAAC Legacy DTNs. All four datamovers have transitioned to Globus Connect Server 5 and use regular UTK Central Authentication Service (CAS) for authentication. InCommon credentials are no longer be used.
To access the Globus interface in your browser, navigate to the Globus website. Login using the existing organizational login option. Verify that the University of Tennessee is selected, then select “Continue.” Authenticate with your UT NetID, password, and Duo TFA. You will then see the interface depicted in Figure 3.1.
Before you can initiate file transfers between your local machine and the DTN, you must configure endpoints. One endpoint will reference your local system while the other will reference one of the datamover DTNs. Further instructions on these endpoints will be provided below.
To configure the endpoints in the Globus interface, select the “Endpoints” tab on the left-side of the page. You will then see a page similar to Figure 3.2. At the top-right of the page, select “Create new endpoint.” On the endpoint type selection page, choose “Globus Connect Personal.”
On the next page, name the endpoint. The name you choose is unimportant; however, it should be something memorable. After you name the endpoint, generate a setup key for the Globus Connect Personal client software. The option to generate the key is listed under Step 2 in Figure 3.3. Copy this key. Finally, download and install the Globus Connect Personal client software. When prompted, enter the setup key you copied to configure your local machine as an endpoint. Refer to Figure 3.3 for a screenshot of the endpoint creation page.
Once you configure your local machine as a Globus endpoint, return to the “File Manager” tab on the left-side of the page. Make sure you select the double panels option in the top-right of the page (Figure 3.9 highlights this option). This will display your local machine’s filesystem in addition to the datamover’s. Once both panels are displayed, click on “Collection” in the left panel. Type the name of your endpoint in the search bar or find it under “My Collections.”
After your endpoint has been selected, you will return to the File Manager. In the right panel, click on “Collection.” Search for one of the two datamovers. The endpoint names of these DTNs are given below.
Once both endpoints are configured, you can transfer data between the two. You can select individual files and directories for these transfers. When you select the data you wish to transfer, press the “Start” button below the endpoint from which you will transfer data. Additionally, you can navigate throughout the filesystem hierarchy in either endpoint using the Globus interface. Other options are available for your transfers, but they are usually unnecessary for most transfers. Figure 3.4 shows what the Globus interface should look like when both endpoints are selected.
FileZilla will work with file transfers to the Open Enclave. Please only use the DTNs listed in Table 1.1 at the beginning of this document.
To use the FileZilla client with your NetID, password, and Duo TFA, follow these steps.
WinSCP can perform file transfers to and from the DTNs. Please use the DTNs listed in Table 1.1 at the beginning of this document.
To use the WinSCP client with your NetID, password, and Duo TFA, follow these steps.