Skip to content Skip to main navigation Report an accessibility issue
High Performance & Scientific Computing

Node and Storage Investments



Overview

Investment criteria and standard node configurations are reviewed annually at the beginning of each fiscal year by OIT. This page has been updated for FY21.

Faculty and other University researchers may purchase compute nodes to be placed in the ISAAC HPC cluster using sponsored research funds, startup funds, or other funds for a private condo. These private condo nodes are exclusive to investors and the project(s) to which they belong and can participate in the responsible node sharing program. If you wish to investigate purchasing nodes for a research project or are ready to purchase nodes for use in a private condo the cluster, please submit an HPSC Service Request (ticket) at this link. A staff member will contact you to discuss details regarding the purchase and once you want to commit to a purchase you will work with the Associate CIO for Service Level and Capacity Management to develop a service level agreement (SLA) that will apply to the node(s). For FY22, the standard SLA period for a compute or GPU node investment will be three years. Other periods will be considered on a case-by-case basis including a proposal performance period.

Note that costs below are estimates and final costs will be determined based on equipment configuration usually in the form of a vendor quote or an OIT Service Level Agreement document corresponding to the funding source. Federal research R accounts can only be charged the direct equipment cost since our facility is not a cost center. A vendor quote would be provided for inclusion in a proposal. After an award, the vendor quote would be requested again as usually vendor quotes are only good for 30 days.

Intel, AMD, and Dell have come out with new generation technology for FY22 that Dell calls their 15th generation technology based on Intel Xeon Ice Lake processors and AMD Milan processors. We can still get last year’s technology but at some point in time we will stop placing orders for Intel Cascade Lake processors. We no longer will order AMD Rome processor based technology. The compute node costs below are an estimated one-time costs for the three year period.

Note all prices below are estimates, pricing change approximately monthly, and Dell advised us in July 2021 that due to the industry-wide shortage of chips, glass, and other parts, lead times are increasing.

Standard Compute Node – Dell 14th Generation Technology

With the 14th generation standard compute node while still available, investors will obtain a powerful compute node that can handle significant HPC workloads. The pertinent technical specifications are listed below.

  • Dell PowerEdge R640
  • 2x 24-Core Intel Xeon Gold 6248R Processors
  • 192GB DDR4 2933MT/s RAM
  • 480GB SATA SSD for operating system
  • HDR Infiniband and Ethernet network interface cards

The estimated cost for a this 14th generation standard compute node is now just over $10,000. Modifications can be made to the above configuration on a case-by-case basis.

Standard Compute Node – Dell 15th Generation Technology

With the 15th generation standard compute node, investors will obtain a powerful compute node based on Intel Xeon Ice Lake processors or AMD Milan processors that can handle significant HPC workloads. The pertinent technical specifications are listed below.

  • Dell PowerEdge R650
  • 2x 28-Core Intel Xeon Gold 6348 Processors, 56 cores total
  • 256GB DDR4-3200 RAM
  • 480GB SATA SSD for operating system
  • HDR Infiniband and Ethernet network interface cards

The estimated cost for a standard Intel processor compute node is approximately $12,700. Modifications can be made to the above configuration on a case-by-case basis.

Information for a Dell PowerEdge R6525 with AMD Milan processors will be available soon.

Standard GPU Node – Dell 14th Generation Technology

With the standard 14th generation GPU node while still available, investors will obtain a compute node that can handle standard HPC workloads and has significant GPU performance for GPU-intensive applications. The pertinent technical specifications are listed below.

The estimated cost for a single standard GPU node is $23,000.00. Modifications can be made to this configuration on a case-by-case basis.

Standard GPU Node – Dell 15th Generation Technology

With the standard GPU node, investors will obtain a compute node that can handle standard HPC workloads and has significant GPU performance for GPU-intensive applications. The pertinent technical specifications are listed below.

The estimated cost for a 15th generation GPU node with two T4 GPUs is ~$15,020 and with two A100 GPUs is ~$28,000. Modifications can be made to this configuration on a case-by-case basis.

Information for a Dell PowerEdge R7525 with AMD Milan processors and either T4 or A100 GPUs will be available soon.

Storage Investments

Unpurged Lustre project storage space (/lustre/haven/proj/…) is available as well as purged Lustre scratch space (/lustre/haven/user/…). Project storage spaces have quotas enabled and Lustre scratch does not empose quotas. University of Tennessee projects receive 1 terabyte (TB) of Lustre project space for no additional cost to the research project. Any project storage beyond 1TB for University of Tennessee sponsored projects and other funded projects should be purchased. External organizations and University faculty leading sponsored projects or other funded projects can purchase storage resources on the high-performance Lustre storage system for their research projects. The FY21 cost per terabyte was $59 and the FY22 cost per terabyte is being calculated and will be available soon. Also, Lustre file systems have a finite number of files that the entire file system can contain. Any project or researcher that uses more that 10 million files will also be charged $100 per year for every 1 million files over the first 10 million.

Please note that the previously listed storage costs only includes the storage equipment, administration, and maintenance of the file system and does not include backups. Backups of Lustre storage are the responsibility of the end users. On a case-by-case basis arrangements can be made for limited amounts of some type of backups, such as, as a duplicate copy or snapshot and should be discussed and included into any investment Service Level Agreement. For more information on the available file systems, please refer to the File Systems document.