Open OnDemand on ISAAC Secure Enclave
Introduction
OpenOnDemand is a web-based portal/interface that provides access to the cluster’s HPC resources. You can manage files, jobs, and access graphical interfaces all through Open OnDemand. This document describes how to access and use Open OnDemand and its related services in the Secure Enclave.
Access and Login
Please follow the below steps to successfully log in to the web based graphical interface of ISAAC Secure Enclave cluster.
- Open a browser of your choice and access the Citrix Secure Enclave website

- After clicking the Login button, the window below will appear. Authenticate with DUO push or request a passcode.

- Once logged in, click on the APPS menu and Open Chrome Secure Enclave.

- Once the Chrome web browser is opened, type
sip-login2-se.utk.tennessee.edu
in the address bar. You will be prompted to enter the username and password as shown in Figure 7.4. below.

- Enter the same username and password as you used to login to Citrix. Press Enter or click on Sign in button.
Upon successful authentication, you will see the Open OnDemand dashboard. The top dark gray ribbon of the dashboard contains several menus that grant access to files, jobs, and your current interactive sessions as shown in Figure 7.5.

File Explorer
The “Files” menu in the top-left of the Open OnDemand dashboard gives you access to your files on the ISAAC Secure Enclave HPSC cluster from within a browser. Initially, the File Explorer shows your home directory and all its contents. Figure 4.6 shows the File Explorer interface.

Job Management
Under the Jobs drop-down menu next to Files menu in the OOD dashboard (Figure 7.5), you have two options: Active Jobs and Job Composer. Both options provide advanced job management from within a web browser.
Job Monitoring
In the Active Jobs section, you will see an organized list of your queued and running batch jobs on the cluster. Figure 4.9 shows the Active Jobs interface. You can filter through jobs to see particular entries, in addition to sorting by the attributes of your choice. To do this, click on the arrows next to the attributes until the list is sorted as you desire.

When a job is queued or running on the cluster, you can view detailed information on the job from within the Active Jobs interface. Figure 4.2 shows the information you will see on queued and running jobs.
To access this view on a queued or running job, click the right-facing arrow on the far-left of the job’s row. You will then see statistics and information about the selected job. Of note is the Output Location of the job, which Figure 4.10 highlights. Each job you create and start through OpenOnDemand is placed within your /home/ondemand directory, so when you wish to retrieve the job’s data, remember to navigate to your ondemand directory. You can specify another output location when you create the job through OpenOnDemand or edit the job script to supply another location.

Job Creation
In the Job Composer section, you can create new batch jobs to run on the cluster based upon several criteria. Figure 4.11 shows the Job Composer interface. When you initially open the Job Composer, the system will briefly explain each option available to you. Once you click through these prompts, you may then click “New Job,” then select the template, path, or job you wish to add. Note that if you close the initial information menu, you will not need to click through each prompt.

When you add a new job from the default template, the Job Composer will create a Simple Sequential Job. In addition to the job, it will create a job script and a job directory. Figure 4.11 shows the Job Composer interface when a job is created with the default template. To modify the job’s options, select Job Options. You can specify the job’s name, cluster, script, and account. To edit the job script, highlight the job and select the Edit Files option in the Job Composer interface. Select the job script, then select Edit. You may also rename the job script if necessary. Additional files can also be uploaded or created in the job’s directory from the File Explorer.
Jobs may also be created with other templates. These can be the default templates or templates that you build within the Job Composer and can be found under the dropdown menu +New Job as shown in Figure 4.11.
In addition to template-based job creation, you may also create jobs based upon a specified path or an existing job. With a specified path, you create a job based upon a directory that already exists. The path will either be to your home or project directory. Once you specify the path, you then modify the typical attributes of the job. With a job created based upon an existing one, select an existing job in the queue, navigate to New Job, then select “From Selected Job.”
Beyond job creation, the Job Composer allows you to submit, stop, and delete your existing jobs. Like the Active Jobs menu, you can filter and sort the available jobs.
noVNC
noVNC allows users to interact with the cluster in a graphical desktop environment from within a web browser. noVNC is particularly useful when you require a graphical X11 application.
To launch a noVNC session, log in to OpenOnDemand. In the dashboard, select “Interactive Apps” from the top-left of the window. In the dropdown menu, select “SIP Desktop.” This will open the session submission form depicted in Figures 7.15.

In this form, specify the resources your application(s) require. All the available options map to sbatch options for interactive jobs. To learn the meaning behind these options, please review the Interactive Jobs section of the Running Jobs document. For most situations, a single node with one core is sufficient. If your application requires GPUs, please specify campus-gpu as the partition. Please be aware that the number of cores per node you request directly influences the memory your job receives. Therefore, make sure to request enough cores for your job so that your job does not run out of memory leading to the termination of the job.
It is also important to note that you can request node-exclusivity for your session. With node-exclusivity, no other session will run on the node your session receives. In cases where the application you will use in the session is resource-intensive, setting this option to “Yes” may be beneficial. Otherwise, leave this option as “No.”
Submit the noVNC session to the scheduler by clicking on the Launch button in Figure 7.15. For most single-node, single-core jobs, the session should begin within five minutes; however, this is dependent on the current resource utilization of the cluster. If you requested multiple nodes, it could take longer for the session to start. When the session is ready, access it by clicking the Launch SIP Desktop in New Tab button. Figure 4.13 highlights this button.

You will see a standard Linux XFCE desktop environment when you access the session. The applications menu contains links to several useful packages in the top-left of the GUI. You can open your home directory in the bottom middle of the GUI with the file cabinet icon. The black icon with a greater-than (>) symbol opens a terminal. From this terminal, you can perform any task you would get from a standard terminal, such as accessing files, loading modules, and running applications.
If you wish to manually terminate the noVNC session, click on your username in the top-right of the GUI and select “Log Out.” Then, delete the session in OpenOnDemand. You will receive an email confirmation of the job’s deletion.
Jupyter Notebook
Launching Jupyter Notebook
Jupyter Notebook is a web-based application that enables you to intuitively create and manipulate data on the cluster. It features a fully-functional Python 3 IDE (integrated development environment), a text editor, and a file manager. The sections below outline how to access and set up Jupyter Notebooks on the cluster.
To access Jupyter Notebook, navigate to Interactive Apps on the OpenOnDemand dashboard. Select Jupyter Notebook from the dropdown menu. The Jupyter Notebook session submission form will appear as it does in Figure 7.17.

The Jupyter notebook in Open OnDemand is configured with Anacond3. We have provided two different versions of Anaconda. In this form, specify one of Anaconda versions to use the associated Jupyter Notebook, the account under which the job should run, the partition, the number of hours the job will run, and the number of cores your job requires. If you require the use of GPUs in your Jupyter Notebook session, select the campus-gpu partition. Please review the Running Jobs document for more information on these fields.
When you specify the necessary options in the form and select Launch. It will not take for the system long to deploy the Jupyter Notebook session, but this is dependent on the resources you requested. When the session is ready, you will see the similar interface that appears in Figure 7.18. The “Connect to” option will open a new Jupyter Notebook tab in your browser. Figure 7.18 highlights this option.
