Software Installation
The research computing support group can provide installations of the vast majority of software needed for your research activities.
A large number of applications and libraries are already installed on our development/testing servers and clusters and we will make a reasonable effort to install any additional software necessary for your research activities.
Note that we only perform software installations and other systems management tasks on our own equipment. If you are managing your own computers, we can advise you, but cannot provide material support.
We perform most open source software installations using package managers, specifically FreeBSD ports and pkgsrc packages. These ports and packages are made available to anyone in the world, making it trivial to install the same software on computers running the same operating systems that we use (see below).
We often install new software on FreeBSD development servers for testing purposes first, using the FreeBSD ports system. The FreeBSD ports collection is very large, so even if the software you need isn’t already in the collection, most or all of the software it depends on generally is. This enables us to provide a test installation with minimal delay to you using minimal man-hours on our end.
If, during testing, you determine that the software will be useful on the large HPC clusters, we’ll then convert the port and any necessary dependencies to pkgsrc, which can be used to install on our CentOS systems as well as OS X and many other Unix compatible platforms.
During testing, we may ask that you create a small test case of your own to verify that the functionality you need works properly. We will then use your test case to verify installations on the cluster. This helps avoid wasting your time, our time, and cluster resources on software debugging.
Researchers are also welcome to build and install software in their home directories if they prefer, but be aware that in most cases this will not be necessary.
Operating Systems
CentOS
CentOS (Community Enterprise Operating System) is a free and open source GNU/Linux-based operating system derived from and virtually identical to Redhat Enterprise Linux (RHEL). CentOS is widely used in data centers throughout the world and is by far the most popular operating system for HPC clusters, including our large faculty research clusters.
Some key facts:
- RHEL and CentOS are the most widely supported Unix-compatible operating systems among commercial scientific software vendors. While common commercial applications such as Matlab may be supported on other GNU/Linux distributions, many lesser-known applications are not.
- Enterprise GNU/Linux distributions are more stable and reliable than non-enterprise distributions.
- RHEL and CentOS use older kernels and libraries than non-enterprise GNU/Linux systems for the sake of stability and long-term binary compatibility. As a result, the native package manager (Yum) is not well-suited for deploying the latest open source software. We therefore use the cross-platform pkgsrc package manager for most of our open source installations. The pkgsrc collection is much larger than the RHEL/CentOS repository, and contains more recent versions of most packages.
FreeBSD
FreeBSD is an extremely powerful, fast and reliable platform on which we can easily install the latest open source software. It allows us to very quickly and easily deploy much of the software needed by researchers. We run FreeBSD on some of our development and testing servers, our educational cluster (Peregrine), and our HTCondor grid (Meadows).
Some key facts:
- The FreeBSD ports collection (FreeBSD’s native package manager) offers more than 33,000 software packages ready to install via one simple command, including many scientific programs and libraries, as well as development tools such a GCC and Clang compilers, all popular language interpreters (Lua, Perl, Octave, Python, etc), text editors (eclipse, emacs, nano, vim, etc.), etc. For this reason, we commonly use FreeBSD as a test platform for software that UWM researchers want to evaluate. This minimizes their waiting time to try the software and our time invested in software installations that may or may not be used long-term.
- Most FreeBSD ports can be easily converted to pkgsrc packages for deployment on CentOS and many other Unix-like platforms, using automated tools to do most of the work. If a researcher’s evaluation of a package on FreeBSD determines that it will be useful for large-scale HPC use, we can, in most cases, quickly convert it to pkgsrc and deploy it on our large CentOS clusters.
- FreeBSD is used in some of the most demanding environments on the Internet, such as Netflix, WhatsApp, and Yahoo servers. Many advanced products such as Juniper network switches, pfSense gateways, and Isilon, NetApps, FreeNAS and Panasas storage systems, are built on FreeBSD. ( See https://en.wikipedia.org/wiki/List_of_products_based_on_FreeBSD )
- Most commercial software vendors do not support running their products on FreeBSD at this time. ( The same is true for most non-enterprise Linux distributions. ) However, FreeBSD has a RHEL/CentOS compatibility system that allows it to run closed-source Linux applications such as Matlab, Flash plugin, Oracle, etc. Packages from the CentOS Yum repository can be installed on FreeBSD relatively easily. If you rely mainly on the latest open source software and need to run only a few commercial applications, FreeBSD may be a good option. Users who rely more heavily on commercial software than on the latest open source are advised to run RHEL or CentOS and use pkgsrc to deploy the latest open source software.
Researchers managing their own computers will benefit from using one of these two operating systems, as this will allow them to leverage more of our work and knowledge. For example, we develop packages for the FreeBSD ports and pkgsrc package managers and test them extensively on our CentOS and FreeBSD servers and clusters. These packages can then be used to quickly and easily install the same software on any CentOS or FreeBSD system.
For other platforms such as Cygwin, Mac OS X, and Ubuntu, etc. we are happy to offer advice, but cannot provide material support. Pkgsrc can also be used to deploy software on Mac OS X, although we do not currently support this directly.
