#

Odyssey 3, The Next Generation

Summary

Over the next several months, Research Computing will be making a number of improvements to the Odyssey compute environment. This third generation of high performance computing will include increases in raw compute power, a new CentOS 7-based application stack, and the introduction of the Singularity container system. We are targeting December 2017 for the final roll out of these changes. All of these changes are designed to provide a solid foundation for the amazing research being done on Odyssey.

These improvements will impact all Odyssey Slurm compute and login nodes. Virtual machines used for web applications, hybrid nodes, and storage will not be impacted.

Over the next 6 months, you can follow the progress outlined below in the Status section.

Prior to deployment of the new system, RC will host seminars to explain how to use the new application stack and Singularity containers and provide help with migrating existing tools.

Status

  • Initial communication, July 2017. Completed.
  • Base CentOS 7 image
  • New build environment available
  • Core compiler and libraries built
  • Module migration list published
  • CentOS 7 modules deployed with opt-in
  • CentOS 7 re-imaging
  • CentOS 7 modules default
  • New Intel Xeon Broadwell cores added
  • Singlularity containers enabled

Project details

15,000 new Intel Xeon Broadwell cores

The general compute will be refreshed with nearly 15,000 new Intel Xeon Broadwell cores. Our benchmarking shows that they are 3-4 times faster than the current AMD Abu Dhabi cores. Seriously. Very fast.

CentOS 7 migration

The operating system will be updated from CentOS 6.8 to the latest release of CentOS 7.

There are several advantages to the more modern OS:

  • Support to launch modern container at runtime (i.e. Singularity )
  • Support for new technologies such as Xeon Phi, GPU's, 3DXpoint Memory, etc.
  • Performance and power improvements
  • Security enhancements
  • Support for newer code bases

Module system

Much of the software on Odyssey is delivered via a LMOD module system that enables software built for CentOS 6.8. In many cases, this software can be migrated to CentOS 7 without any changes. However, some code and libraries, especially those that depend heavily on glibc, can be sensitive to architecture or system packages and will likely need to be recompiled. To support this transition, the modulepaths of the current system will be split into CentOS 6 and CentOS 7 branches with the default modules transitioning to CentOS 7 at a specified time.

"legacy" modules will no longer be available on Odyssey. These modules typically start with prefixes like "hpc/", "math/", or "centos6/". In many cases, these modules are more than 7 years old, based on CentOS 5, and, in the case of MPI-based code, do not work properly with our current environment. The new module system already has several thousand builds of more than 600 applications and so should be able to accommodate any computation required.

We will be consolidating some of the modules during this migration. Particularly for some compiler and MPI-dependent libraries, there are dozens of builds only a handful of which are really used (see NetCDF). We will be using our module load logging data to determine whether applications or libraries are actually used and they will be given priority.

Migration of some legacy applications can be very difficult and time-consuming. RC staff will be available at Office Hours or by appointment to assist where possible.

In addition we will be moving to a new release pattern for modules where updated versions of common modules (such as python, MPI, gcc) will be released on a regular schedule, and older modules will be deprecated. This will help us stay up with the latest software available and reduce the number of redundant builds. More details on this new release pattern will be forthcoming after the upgrade to CentOS 7 is complete.

Check this page for updates

While this project progresses, we will be sending out periodic communications. These communications will be available from this page, so check below for new postings.


Items for Page: There are no posts that match the selection criteria.