Skip to content Skip to main navigation Report an accessibility issue
High Performance & Scientific Computing

Data Transfer on ISAAC-NG



INTRODUCTION

The ISAAC-NG computing cluster provides several ways for users to transfer files to/from various file system locations on the ISAAC-NG computing cluster, such as: NFS home directories, NFS project directories, Lustre project directories, and Lustre scratch directories. DTNs (Data Transfer Nodes) furnish this capability.

At the time of this writing, there is one Data Transfer Node (DTN) available to ISAAC-NG users to utilize for the purpose of data transfer. Table 1.1 shows this node.

Data Transfer Node
dtn1.isaac.utk.edu
Table 1.1: ISAAC-NG Data Transfer Node (DTN)

This Data Transfer Node is setup for NetID authentication, Duo multi-factor authentication (MFA), and authentication through an InCommon Credential. Using these means, ISAAC-NG users can login to this node and perform data transfer functions (e.g., moving project data to or from ISAAC-NG directories, moving data to a project’s backup storage medium, moving project data from a project’s data storage to an ISAAC-NG directory for data processing, etc.).

To connect to this DTN, users may use ssh (Secure Shell) in a command-line terminal. More information on how users can use the ssh protocol to access ISAAC-NG computational resources can be found on the Access and Login ISAAC-NG webpage. When using ssh, users may simply replace the hostname of an ISAAC-NG login node (login.isaac.utk.edu) with the hostname of the DTN to which they wish to connect (dtn1.isaac.utk.edu), then authenticate with their UT NetID, their password, and a Duo multi-factor authentication code (MFA).

ISAAC-NG supports several data transfer protocols including SCP, SFTP, GSISCP, and Globus. SCP and SFTP are both ssh utilities available for transferring files but tend to perform slower than Globus. At the time of this writing, Globus offers the fastest data transfers on ISAAC-NG. Still, SCP and SFTP are useful methods for users who are performing small transfers. GSISCP is used for unattended data transfers.

Stuff

GLOBUS WEB-BASED TRANSFERS

The Globus web interface allows you to conveniently perform data transfers to and from ISAAC-NG resources. At the time of this writing, Globus is the fastest and most efficient data transfer method available on the ISAAC-NG computing cluster. Before users can use Globus, each user must create an InCommon Credential and associate it with their account. To perform this task, please consult the User Portal documentation.

Before continuing on in reading and/or using this documentation, each user should associate their InCommon Credential with their account, and then continue to read through this document.

Users should also review the official Globus documentation for more information on how to use the Globus tool.

Credential information is updated on the ISAAC-NG DTN each hour. If you are unable to use Globus, please wait approximately an hour for the ISAAC-NG DTN to obtain your information. Generally, if you see your InCommon credential appear in the User Portal, you should be able to use Globus.

stuff

Using the Globus Web Interface

To access the Globus interface in your browser, navigate to the Globus website (app.globus.org). Once on the official Globus website, users should login using the existing organizational login option. Verify that the University of Tennessee is selected, then select “Continue.” Authenticate with your UT NetID, password, and Duo MFA code.

After authenticating, users will then see the interface depicted in Figure 3.1. If you experience issues logging in, verify that your InCommon credential was configured per the steps given in the Configuring your InCommon Credential section of the User Portal documentation (please see the above section).

FIGURE 3.1 – INITIAL GLOBUS INTERFACE

Before users can initiate file transfers between a user’s local machine and ISAAC-NG, users must configure endpoints. One endpoint will reference the user’s local system while the other endpoint will reference the ISAAC DTN (dtn1.isaac.utk.edu). Further instructions regarding these endpoints will be provided below.

To configure the necessary endpoints in the Globus interface, select the “Endpoints” tab on the left-hand side of the page. You will then see a page similar to Figure 3.2. At the top-right of the page, select “Create new endpoint.” On the endpoint type selection page, choose “Globus Connect Personal.”

FIGURE 3.2 – GLOBUS ENDPOINT MENU

On the next page, users must name the endpoint that will be on the user’s local machine. The name users choose is unimportant; however, it should be something memorable (e.g., JaneSmithLocalEndpoint, etc.). After a user names the endpoint, the user should generate a setup key for the Globus Connect Personal client software using the option to generate the key, which is shown under Step 2 in Figure 3.3. Users should copy the start up key that they generate (for use later in the setup process). Finally, users should download and install the Globus Connect Personal client software from the official Globus Personal Connect website’s “Downloads” section (www.globus.org). When prompted, users should enter the setup key that they copied earlier, to configure their local machine as an endpoint. Refer to Figure 3.3 for a screenshot of the endpoint creation page.

FIGURE 3.3 – GLOBUS ENDPOINT CREATION MENU

Once a user has configured their local machine as a Globus endpoint, the user should return to the “File Manager” tab on the left side of the page. Users should make sure to select the double panels option in the top-right of the page (Figure 3.4 highlights this option). This will display the user’s local machine’s filesystem in addition to the ISAAC-NG DTN’s filesystem.

Once both panels are displayed and users can see their local machine’s filesystem and the ISAAC-NG DTN’s filesystem, the user should click on “Collection” in the left panel. The users should then type the name of the desired endpoint into the search bar, or find it under “My Collections.”

After the desired endpoint has been selected, the user should return to the File Manager page. In the right panel, click on “Collection,” and then search for the Data Transfer Node (DTN) associated with ISAAC-NG, which is named:

  • UTK ISAAC-NG DTN1
FIGURE 3.4 – GLOBUS FILE TRANSFER INTERFACE

Once both endpoints are configured (for the local machine endpoint and the ISAAC-NG DTN endpoint), a user can transfer data between the two endpoints. Figure 3.4 shows what the Globus interface should look like when both endpoints are selected. Users can select individual files and directories for to transfers. When users are finished selecting the data they wish to transfer, the user should press the “Start” button below the endpoint from which the user will transfer data.

Additionally, the user can navigate throughout the filesystem hierarchy in either endpoint using the Globus interface. Other options are available for user data transfers to or from ISAAC-NG, but these other options are usually unnecessary to accomplish most transfers.

Stuff

Unattended Transfers with gsissh

Unattended data transfers are possible on ISAAC-NG with gsissh and its associated tools. Users may find gsissh transfers useful in situations where massive amounts of data must be transferred overnight, or in situations where a job requires input or output data without user intervention. To use gsissh, users should follow the steps in the User Portal document to create and associate an InCommon credential with their user account, and then navigate to the CILogon website and select the University of Tennessee as their identity provider.

After the user authenticates to the UT CAS, the user should follow the following steps to supply a password for their credential and upload that credential to ISAAC-NG.

  1. Expand the “Create Password-Protected Certificate” menu.
  2. Enter a password for your new InCommon credential. It is critical that you remember this password. Please record and store this password in a secure location for future reference.
  3. Download your credential. Click download, then save the credential to your local system. This will act as a backup. .
  4. Right-click on the link to “Download Your Certificate” and copy the URL to your clipboard.
  5. Use ssh to connect to the ISAAC-NG DTN: dtn1.isaac.utk.edu
  6. Make a new directory in your home directory with the name .globus. In the command line, use mkdir ~/.globus to create the new directory. Execute cd .globus to enter the new directory that you just created.
  7. On ISAAC-NG, type wget <copied-url> where <copied-url> is the download link to your InCommon Credential. You should be able to paste the link with the key combination Ctrl + V on Windows and Linux systems or Command + V on MacOS systems. Verify that you are in the .globus directory by typing the “pwd” command into the command line before you execute the “wget” command. Figure 4.3 (below) shows the output of a successful wget download.
    [user-x@acf-login8 .globus]$ wget https://polo1.cilogon.org/pkcs12/36E8DB8887F08ACC5/usercred.p12 --2020-01-16 16:08:58-- https://polo1.cilogon.org/pkcs12/36E8DB8887F08ACC5/usercred.p12 Resolving polo1.cilogon.org (polo1.cilogon.org)... 141.142.149.19 Connecting to polo1.cilogon.org (polo1.cilogon.org)|141.142.149.19|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 2837 (2.8K) [application/x-pkcs12] Saving to: ‘usercred.p12’ 100%[==========================================>] 2,837 --.-K/s in 0s 2020-01-16 16:08:58 (156 MB/s) - ‘usercred.p12’ saved [2837/2837]
    Figure 4.3 – Using wget to Retrieve an InCommon Credential
  8. Execute chmod 600 usercred.p12 to change the permissions of the InCommon Credential file.
  9. Execute ls -l on the .globus directory to verify that the usercred.p12 file has read and write permissions for the user owner. Figure 4.4 (below) shows the permissions that should apply to the credential file.
    [user-x@acf-login8 .globus]$ ls -l total 4 -rw-------. 1 user-x testgrp 2837 Jan 16 16:08 usercred.p12
    Figure 4.4 – Permissions to Set on the usercred.p12 File

10. After you have successfully downloaded and modified the usercred.p12 file, execute grid-proxy-init to use the InCommon Credential for authentication. Enter the password you set in the first step of the credential configuration process. Figure 4.5 (below) shows what should appear if the certificate initialization was successful.

[user-x@acf-login8 ~]$ grid-proxy-init
Enter GRID pass phrase for this identity:
Your identity: /DC=org/DC=cilogon/C=US/O=University of Tennessee/CN=User X A12345678
Creating proxy ...................... Done
Your proxy is valid until: Fri Jan 17 04:30:39 2020

Figure 4.5 – Using grid-proxy-init to Initialize a Credential

In some cases, you may require the credential to be valid for an extended period. Execute grid-proxy-init -valid hh:mm to specify the amount of time the credential should remain valid.

For example, to make the credential last for three days, execute grid-proxy-init -valid 72:00. If successful, the output of the certificate initialization will show the date when the certificate will expire. In Figure 4.5, the certificate expires twelve hours from the time it was initialized. This is the default expiration time amount.

Once your credential is initialized, gsiscp will work without prompting you for a password or Duo MFA. You will receive a password prompt, but gsissh will automatically override the prompt to allow the transfer. The syntax to use gsiscp to transfer data is: gsiscp <source> <destination:directory>.

For instance, to transfer a gzipped tar archive named MedJobResults.tar.gz in your ISAAC-NG home directory to a remote system, execute gsiscp ~/MedJobResults.tar.gz remotestorage.local:~/Documents. To transfer a zip archive named fluid_dynamics_results.zip from your home directory on the ISAAC-NG to a remote endpoint, execute gsiscp ~/fluid_dynamics_results.zip remote_dtn1:~/<the destination file path>

Stuff

Using FileZilla to Transfer Files

FileZilla will work with file transfers to ISAAC-NG. Please only use the ISAAC-NG DTN, dtn1.isaac.utk.edu.

To use the FileZilla client with your NetID, password, and Duo MFA, follow these steps.

  1. Open the FileZilla client after downloading the client from the official filezilla-project.org website.
  2. Select File, then Site Manager.

    Figure 5.1 – FileZilla’s Site Manager Option
  3. Select “New Site,” then provide the required information when prompted. For the host, select the ISAAC-NG DTN, dtn1.isaac.utk.edu. For protocol, select SFTP – SSH File Transfer Protocol. For Logon Type, select Interactive. For User, type your UT NetID. Finally, rename the entry under sites from “New Site” to something more memorable, such as the name of the ISAAC-NG DTN. Refer to Figures 5.2 and 5.3 to identify where to find these options.

    (Above) Figure 5.2 – New Site in FileZilla (Below) Figure 5.3 – FileZilla Site Options
  4. Select Transfer Settings, then check the box for Limit the number of simultaneous connections. Make sure the value beneath this checkbox is 1.
  5. Select “Connect” in the Site Manager window.
  6. When prompted, enter your password.
    Figure 5.4 – FileZilla Password Prompt
  7. When prompted, type a “1” to send a Duo Push to your mobile device, then authenticate with Duo MFA. Upon successful authentication, you will be logged in to the ISAAC-NG DTN through FileZilla.

    Figure 5.5 – FileZilla Duo Prompt

Stuff

Using WinSCP to Transfer Files

WinSCP can perform file transfers to and from ISAAC-NG. Please use the ISAAC-NG DTN: dtn1.isaac.utk.edu.

To use the WinSCP client with your NetID, your password, and a Duo MFA code, please follow these steps.

  1. After downloading WinSCP from the official website (winscp.net), open WinSCP, then click on “New Site.”
  2. Provide the hostname of the ISAAC-NG DTN, dtn1.isaac.utk.edu, for “Host name,” your UT NetID for “User name,” and your password. Leave the port number as 22.

    Figure 6.1 – WinSCP New Site Creation
  3. When warned about an unknown server, select “Yes.

    Figure 6.2 – Initial WinSCP Key Warning
  4. The authentication banner will appear. Select “Continue.”

    Figure 6.3– WinSCP Authentication Banner
  5. When prompted, type “1” to receive a Duo Push on your mobile device. Authenticate with Duo. You will then be logged in.

    Figure 6.4 – Duo Prompt in WinSCP
  6. Once you authenticate, you will get the WinSCP application screen. On the left side of the screen, you see your local machine. On the right side of the screen, you see the remote system into which you are logged.

stuff

Preparing Data for Tansfer

Before users initiate any data transfers from the Open Enclave to another storage resource, they should consider preparing the data they wish to transfer by archiving and compressing it.

When a user archives data, several files and directories can be added to the same location.

When a user compresses data, they can reduce the data’s total size.

Both methods reduce the total amount of data that must be sent across the network and makes it easier for users to organize the data they wish to transfer.

At the time of this writing, the tar and zip utilities are the best methods for data archiving and compression for ISAAC-NG users who use machines running Linux, MacOS, and Windows.

Stuff

Using the tar Utility

The tar (tape archiver) utility uses simple command syntax and allows large amounts of data to be aggregated into the same archive. Linux, MacOS, and updated Windows 10 systems can use tar. Older Windows systems will be limited to the zip utility.

To create a tar archive, execute tar czvf <archive-name> <dir-to-archive>. Replace the <archive-name> argument with the name of the new archive you wish to create. Be sure to follow the name with the .tar.gz extension, as in “my_archive.tar.gz.” Replace the <dir-to-archive> argument with the name of the directory into which you wish to place the new archive. If the directory you intend to archive is not within your working directory, YOU MUST specify the relative or absolute path to the archive. By default, tar will recursively place the directory and its contents into the new archive. Figure 2.1 shows the successful creation of a tar archive.

[user@acf-login5 ~]$ tar czvf new_archive.tar.gz Documents
Documents/
Documents/IntroUnix.pdf
Documents/JobSubData.zip
Documents/MATLAB/
Documents/Scripts.zip
Documents/PyLists.py

Figure 2.1 – Creating a tar Archive

After the archive is created, execute ls -l to verify that the archive exists. You can view its contents with the tar tvf <archive-name> command.

You may then transfer the archive using one of the data transfer methods described in this document. In general, Globus is the best method. Please refer to the Globus section to learn how to configure it for your system. On the remote system, execute tar xvf <archive-name> to extract the contents of the archive. The files will be extracted into your ISAAC-NG working directory.

stuff

Using the zip Utility

When working with older Windows systems, the zip utility should be used to archive and compress your data from ISAAC-NG.

To create a zip archive on ISAAC-NG, execute zip -r <archive-name>.zip <dir-to-archive>. 

Be sure that the directory that you wish to archive is in your working directory. Otherwise, you MUST specify the relative or absolute path to the directory you wish to archive.

Replace the <archive-name> argument with the name of the new zip archive. You may or may not choose to include the .zip file extension in the new archive’s name at this time; if you do not include the .zip file extension in the new archive’s name, the zip utility will add it to the new archive’s name automatically. Replace the <dir-to-archive> argument with the directory you wish to place in the zip archive. The -r option ensures that the directory and its contents are archived and compressed. Figure 2.2 shows the successful creation of a zip archive.

[user@acf-login5 ~]$ zip -r Documents Documents
adding: Documents/ (stored 0%)
adding: Documents/IntroUnix.pdf (deflated 4%)
adding: Documents/MATLAB/ (stored 0%)
adding: Documents/PyLists.py (deflated 61%)

Figure 2.2 – Creating a zip Archive

After the zip archive has been created, execute ls -l in the directory from which you created the archive to ensure the archive exists. The new archive that has been created will appear with the name you gave to the new archive followed by the .zip extension.

With the zip archive created and verified, transfer the archive to your system using one of the data transfer methods described in this document. In most cases, Globus is the most convenient method. Please refer to the Globus section to learn how to configure it for use on your system.

Once you transfer the zip archive to your system, open the File Explorer and navigate to the directory in which you placed the archive. Right-click on the archive and select the “Extract All…” option in the submenu. Figure 2.3 shows where to locate this option. Specify the directory into which the contents should be extracted, then select “Extract.” You may then open the archive and peruse its contents.

This image has an empty alt attribute; its file name is Unzip-Archive-Windows-1.png
Figure 2.3 – Extracting the Contents of a zip Archive in Windows