Motivation
Combinations of multiple software packages developed by different groups are essential for large-scale computational science, where the capabilities needed for modeling, simulation, and analysis are broader than any single team has resources to address. The often tedious trial-and-error process of obtaining, configuring, and installing any single tool may arguably be manageable. However, from the perspective of an end-user application scientist, handling each tool’s installation idiosyncrasies can easily become overwhelming when dealing with several packages in combination. Worse, such problems are compounded by the need for consistency among packages to be used within the same application in terms of compiler, compiler versions and options, and common third-party packages such as BLAS and HDF5.
Spack
A goal of xSDK is to simplify the process of working with various xSDK packages in combination. Spack is a package manager for high-performance computing. We thus provide an xSDK Spack metapackage that includes many xSDK numerical libraries (AMReX, ButterflyPACK, Data Transfer Toolkit (DTK), deal.ii, Ginkgo, HeFFTe*, Hypre, libEnsemble, MAGMA, MFEM, Omega_h, PETSc/TAO, PHIST, PLASMA, preCICE, PUMI, SLATE*, SLEPc, STRUMPACK, SUNDIALS, SuperLU_dist, Tasmanian, and Trilinos ) as well as some commonly needed external packages.
Spack allows the xSDK packages to be deployed with a single command. The user can optionally choose compilers, MPI implementation, and build options. We are working toward support of combinatorial test dashboards for all xSDK packages.
*Denotes a package released as part of the xSDK for the first time in version 0.6.0.
xSDK 0.6.0 release components
Final release date: November 14, 2020
The xSDK spack metapackage (version 0.6.0) includes tested release versions of the following xSDK packages:
- AMReX
- ButterflyPACK
- Data Transfer Toolkit (DTK)
- deal.ii
- Ginkgo
- HeFFTe
- hypre
- libEnsemble
- MAGMA
- MFEM
- Omega_h
- PETSc/TAO
- PHIST
- PLASMA
- preCICE
- PUMI
- SLATE
- SLEPc
- STRUMPACK
- SUNDIALS
- SuperLU
- Tasmanian
- Trilinos
- Alquimia
- PFLOTRAN
Users who prefer to work with just one xSDK package rather than several can use Spack to install only that particular package. Alternatively, they can bypass Spack and instead obtain the package directly from its website and install according to instructions there.
Packages using CUDA
The following packages can be installed with CUDA feature enabled
- PETSc
- SUNDIALS
- MAGMA
- STRUMPACK
- Tasmanian
- Ginkgo
- HeFFTe
- SLATE
Open source packages installed with xSDK packages
Users can view a tree of dependencies for any Spack package, including xSDK, by running ‘spack spec xsdk’. This is an example of the dependency tree for the xSDK metapackage as of version 0.6.0:
xsdk@0.6.0%gcc@7.4.0+butterflypack+cuda+datatransferkit+dealii~debug+ginkgo+heffte+libensemble+omega-h+phist+precice+slate+strumpack+trilinos arch=linux-centos7-skylake_avx512 ^alquimia@xsdk-0.6.0%gcc@7.4.0~ipo+shared build_type=RelWithDebInfo arch=linux-centos7-skylake_avx512 ^cmake@3.18.4%gcc@7.4.0~doc+ncurses+openssl+ownlibs~qt patches=bf695e3febb222da2ed94b3beea600650e4318975da90e4a71d6f31a6d5d8c3d arch=linux-centos7-skylake_avx512 ^ncurses@6.2%gcc@7.4.0~symlinks+termlib arch=linux-centos7-skylake_avx512 ^pkgconf@1.7.3%gcc@7.4.0 arch=linux-centos7-skylake_avx512 ^openssl@1.1.1h%gcc@7.4.0+systemcerts arch=linux-centos7-skylake_avx512 ^perl@5.32.0%gcc@7.4.0+cpanm+shared+threads arch=linux-centos7-skylake_avx512 ^berkeley-db@18.1.40%gcc@7.4.0 arch=linux-centos7-skylake_avx512 ^gdbm@1.18.1%gcc@7.4.0 arch=linux-centos7-skylake_avx512 ^readline@8.0%gcc@7.4.0 arch=linux-centos7-skylake_avx512 ^zlib@1.2.11%gcc@7.4.0+optimize+pic+shared arch=linux-centos7-skylake_avx512 ^hdf5@1.10.7%gcc@7.4.0~cxx~debug+fortran+hl~java+mpi+pic+shared~szip~threadsafe api=none arch=linux-centos7-skylake_avx512 ^numactl@2.0.14%gcc@7.4.0 patches=4e1d78cbbb85de625bad28705e748856033eaafab92a66dffd383a3d7e00cc94 arch=linux-centos7-skylake_avx512 ^autoconf@2.69%gcc@7.4.0 arch=linux-centos7-skylake_avx512 ^m4@1.4.18%gcc@7.4.0+sigsegv patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00,fc9b61654a3ba1a8d6cd78ce087e7c96366c290bc8d2c299f09828d793b853c8 arch=linux-centos7-skylake_avx512 ^libsigsegv@2.12%gcc@7.4.0 arch=linux-centos7-skylake_avx512 ^automake@1.16.2%gcc@7.4.0 arch=linux-centos7-skylake_avx512 ^libtool@2.4.6%gcc@7.4.0 arch=linux-centos7-skylake_avx512 ^openmpi@3.1.6%gcc@7.4.0~atomics+cuda~cxx~cxx_exceptions+gpfs~java~legacylaunchers~lustre~memchecker~pmi~singularity~sqlite3+static~thread_multiple+vt+wrapper-rpath fabrics=none schedulers=none arch=linux-centos7-skylake_avx512 ^hwloc@1.11.11%gcc@7.4.0~cairo+cuda~gl~libudev+libxml2~netloc~nvml+pci+shared arch=linux-centos7-skylake_avx512 ^cuda@10.2.89%gcc@7.4.0 arch=linux-centos7-skylake_avx512 ^libxml2@2.9.10%gcc@7.4.0~python arch=linux-centos7-skylake_avx512 ^libiconv@1.16%gcc@7.4.0 arch=linux-centos7-skylake_avx512 ^xz@5.2.5%gcc@7.4.0~pic arch=linux-centos7-skylake_avx512 ^libpciaccess@0.16%gcc@7.4.0 arch=linux-centos7-skylake_avx512 ^util-macros@1.19.1%gcc@7.4.0 arch=linux-centos7-skylake_avx512 ^pflotran@xsdk-0.6.0%gcc@7.4.0 arch=linux-centos7-skylake_avx512 ^petsc@3.14.1%gcc@7.4.0~X~batch~cgns~complex+cuda~debug+double~exodusii~fftw~giflib+hdf5+hypre~int64~jpeg~knl~libpng~libyaml~memkind+metis~moab~mpfr+mpi~mumps~p4est~random123~saws+shared~suite-sparse+superlu-dist+trilinos~valgrind clanguage=C patches=e04bd7afe3d4aa40debf6600b46b64de82d9e62a774d8ef63c922601fec7fc90 arch=linux-centos7-skylake_avx512 ^diffutils@3.7%gcc@7.4.0 arch=linux-centos7-skylake_avx512 ^hypre@2.20.0%gcc@7.4.0~complex~debug~int64~internal-superlu~mixedint+mpi~openmp+shared+superlu-dist patches=6e3336b1d62155f6350dfe42b0f9ea25d4fa0af60c7e540959139deb93a26059 arch=linux-centos7-skylake_avx512 ^openblas@0.3.5%gcc@7.4.0~consistent_fpcsr~ilp64+pic+shared threads=none arch=linux-centos7-skylake_avx512 ^superlu-dist@6.4.0%gcc@7.4.0~int64~ipo~openmp+shared build_type=RelWithDebInfo arch=linux-centos7-skylake_avx512 ^metis@5.1.0%gcc@7.4.0~gdb~int64~real64+shared build_type=Release patches=4991da938c1d3a1d3dea78e49bbebecba00273f98df2a656e38b83d55b281da1,b1225da886605ea558db7ac08dd8054742ea5afe5ed61ad4d0fe7a495b1270d2 arch=linux-centos7-skylake_avx512 ^parmetis@4.0.3%gcc@7.4.0~gdb~int64~ipo+shared build_type=RelWithDebInfo patches=4f892531eb0a807eb1b82e683a416d3e35154a455274cf9b162fb02054d11a5b,50ed2081bc939269689789942067c58b3e522c269269a430d5d34c00edbc5870,704b84f7c7444d4372cb59cca6e1209df4ef3b033bc4ee3cf50f369bce972a9d arch=linux-centos7-skylake_avx512 ^python@3.8.6%gcc@7.4.0+bz2+ctypes+dbm~debug+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tix~tkinter~ucs4+uuid+zlib patches=0d98e93189bc278fbc37a50ed7f183bd8aaf249a8e1670a465f0db6bb4f8cf87 arch=linux-centos7-skylake_avx512 ^bzip2@1.0.8%gcc@7.4.0+shared arch=linux-centos7-skylake_avx512 ^expat@2.2.10%gcc@7.4.0+libbsd arch=linux-centos7-skylake_avx512 ^libbsd@0.10.0%gcc@7.4.0 arch=linux-centos7-skylake_avx512 ^gettext@0.21%gcc@7.4.0+bzip2+curses+git~libunistring+libxml2+tar+xz arch=linux-centos7-skylake_avx512 ^tar@1.32%gcc@7.4.0 arch=linux-centos7-skylake_avx512 ^libffi@3.3%gcc@7.4.0 patches=26f26c6f29a7ce9bf370ad3ab2610f99365b4bdd7b82e7c31df41a3370d685c0 arch=linux-centos7-skylake_avx512 ^libuuid@1.0.3%gcc@7.4.0 arch=linux-centos7-skylake_avx512 ^sqlite@3.33.0%gcc@7.4.0+column_metadata+fts~functions~rtree arch=linux-centos7-skylake_avx512 ^trilinos@13.0.1%gcc@7.4.0~adios2~alloptpkgs+amesos+amesos2+anasazi+aztec+belos+boost~cgns~chaco~complex~cuda~debug~dtk+epetra+epetraext~exodus+explicit_template_instantiation~float+fortran+glm+gtest+hdf5~hwloc+hypre+ifpack+ifpack2~intrepid+intrepid2~ipo~isorropia+kokkos+matio~mesquite+metis~minitensor+ml+mpi+muelu~mumps+netcdf+nox~openmp~phalanx~piro~pnetcdf~python~rol~rythmos+sacado+shards+shared~shylu~stk~stratimikos~strumpack~suite-sparse~superlu+superlu-dist~teko~tempus+teuchos+tpetra~wrapper~x11~xsdkflags~zlib+zoltan+zoltan2 build_type=RelWithDebInfo cuda_arch=none cxxstd=11 gotype=int arch=linux-centos7-skylake_avx512 ^boost@1.74.0%gcc@7.4.0+atomic+chrono~clanglibcpp~container~context~coroutine+date_time~debug+exception~fiber+filesystem+graph~icu+iostreams+locale+log+math~mpi+multithreaded~numpy~pic+program_options~python+random+regex+serialization+shared+signals~singlethreaded+system~taggedlayout+test+thread+timer~versionedlayout+wave cxxstd=98 visibility=hidden arch=linux-centos7-skylake_avx512 ^glm@0.9.7.1%gcc@7.4.0~ipo build_type=RelWithDebInfo arch=linux-centos7-skylake_avx512 ^matio@1.5.17%gcc@7.4.0+hdf5+shared+zlib arch=linux-centos7-skylake_avx512 ^netcdf-c@4.7.4%gcc@7.4.0~dap~hdf4~jna+mpi~parallel-netcdf+pic+shared arch=linux-centos7-skylake_avx512 ^amrex@20.10%gcc@7.4.0~amrdata~cuda~eb~fortran~hdf5~hypre~ipo+linear_solvers+mpi~openmp~particles~petsc~shared~sundials build_type=Release dimensions=3 precision=double arch=linux-centos7-skylake_avx512 ^butterflypack@1.2.1%gcc@7.4.0~ipo+shared build_type=RelWithDebInfo arch=linux-centos7-skylake_avx512 ^arpack-ng@3.7.0%gcc@7.4.0+mpi+shared arch=linux-centos7-skylake_avx512 ^netlib-scalapack@2.1.0%gcc@7.4.0~ipo~pic+shared build_type=Release patches=1c9ce5fee1451a08c2de3cc87f446aeda0b818ebbce4ad0d980ddf2f2a0b2dc4,f2baedde688ffe4c20943c334f580eb298e04d6f35c86b90a1f4e8cb7ae344a2 arch=linux-centos7-skylake_avx512 ^datatransferkit@3.1-rc2%gcc@7.4.0~ipo~openmp+serial+shared build_type=RelWithDebInfo arch=linux-centos7-skylake_avx512 ^dealii@9.2.0%gcc@7.4.0~adol-c+arpack~assimp~cuda~doc+examples~ginkgo~gmsh+gsl+hdf5~int64~ipo+metis+mpi+muparser+nanoflann~netcdf+oce~optflags+p4est+petsc~python+scalapack+slepc~sundials~symengine+threads+trilinos build_type=DebugRelease cuda_arch=none cxxstd=default arch=linux-centos7-skylake_avx512 ^gsl@2.5%gcc@7.4.0~external-cblas arch=linux-centos7-skylake_avx512 ^intel-tbb@2020.3%gcc@7.4.0+shared+tm cxxstd=default patches=62ba015ebd1819c45bef47411540b789b493e31ca668c4ff4cb2afcbc306b476,ce1fb16fb932ce86a82ca87cf0431d1a8c83652af9f552b264213b2ff2945d73 arch=linux-centos7-skylake_avx512 ^muparser@2.2.6.1%gcc@7.4.0 arch=linux-centos7-skylake_avx512 ^nanoflann@1.2.3%gcc@7.4.0~ipo build_type=RelWithDebInfo arch=linux-centos7-skylake_avx512 ^ninja@1.10.1%gcc@7.4.0 arch=linux-centos7-skylake_avx512 ^oce@0.18.3%gcc@7.4.0~X11+tbb arch=linux-centos7-skylake_avx512 ^p4est@2.2%gcc@7.4.0+mpi~openmp arch=linux-centos7-skylake_avx512 ^slepc@3.14.0%gcc@7.4.0+arpack~blopex arch=linux-centos7-skylake_avx512 ^suite-sparse@5.7.2%gcc@7.4.0~cuda~openmp+pic~tbb arch=linux-centos7-skylake_avx512 ^ginkgo@1.3.0%gcc@7.4.0+cuda~develtools~full_optimizations~hip~ipo~openmp+shared build_type=Release cuda_arch=none arch=linux-centos7-skylake_avx512 ^heffte@2.0.0%gcc@7.4.0+cuda+fftw~fortran~ipo+magma~mkl~python+shared build_type=RelWithDebInfo patches=c520da92eb86d41ba6da8a3ec86995ef07489170e93f59c3a7e8908d6336561c arch=linux-centos7-skylake_avx512 ^fftw@3.3.8%gcc@7.4.0+mpi~openmp~pfft_patches precision=double,float arch=linux-centos7-skylake_avx512 ^magma@2.5.4%gcc@7.4.0+cuda+fortran~ipo+shared build_type=RelWithDebInfo cuda_arch=60 arch=linux-centos7-skylake_avx512 ^mfem@4.2.0%gcc@7.4.0~conduit~cuda~debug+examples~gnutls~gslib~lapack~libceed~libunwind+metis+miniapps~mpfr+mpi~netcdf~occa~openmp+petsc~pumi~raja~shared+static~strumpack~suite-sparse+sundials+superlu-dist~threadsafe~umpire+zlib cuda_arch=sm_60 timer=auto arch=linux-centos7-skylake_avx512 ^sundials@5.5.0%gcc@7.4.0+ARKODE+CVODE+CVODES+IDA+IDAS+KINSOL+cuda+examples-c~examples-cuda~examples-cxx~examples-f2003+examples-f77~examples-f90+examples-install~f2003~fcmix+generic-math+hypre~int64~ipo~klu~lapack+mpi~openmp+petsc~pthread~raja+shared+static+superlu-dist~superlu-mt+trilinos build_type=RelWithDebInfo patches=c5baf23d5eceab347fcad44e19585948b3ee69ed8c3694930eb99a181b823210 precision=double arch=linux-centos7-skylake_avx512 ^omega-h@9.32.5%gcc@7.4.0~examples~ipo+mpi+optimize+shared+symbols~throw+trilinos~warnings+zlib build_type=RelWithDebInfo arch=linux-centos7-skylake_avx512 ^phist@1.9.3%gcc@7.4.0~fortran~host~int64~ipo+mpi~openmp~parmetis~scamac+shared~trilinos build_type=RelWithDebInfo kernel_lib=tpetra outlev=2 arch=linux-centos7-skylake_avx512 ^plasma@20.9.20%gcc@7.4.0~ipo~lua+shared build_type=RelWithDebInfo arch=linux-centos7-skylake_avx512 ^precice@2.1.1%gcc@7.4.0~ipo+mpi+petsc~python+shared build_type=RelWithDebInfo arch=linux-centos7-skylake_avx512 ^eigen@3.3.8%gcc@7.4.0~ipo build_type=RelWithDebInfo patches=62590e9b33a8f72b608a72b87147a306e7cb20766ea53c6b8e0a183fa6cb7635 arch=linux-centos7-skylake_avx512 ^pumi@2.2.5%gcc@7.4.0~fortran~int64~ipo~shared+simmodsuite_version_check~zoltan build_type=RelWithDebInfo simmodsuite=none arch=linux-centos7-skylake_avx512 ^py-libensemble@0.7.1%gcc@7.4.0~mpi~nlopt+petsc4py~scipy arch=linux-centos7-skylake_avx512 ^py-numpy@1.19.4%gcc@7.4.0+blas+lapack arch=linux-centos7-skylake_avx512 ^py-cython@0.29.21%gcc@7.4.0 arch=linux-centos7-skylake_avx512 ^py-setuptools@50.3.2%gcc@7.4.0 arch=linux-centos7-skylake_avx512 ^py-petsc4py@3.14.0%gcc@7.4.0+mpi arch=linux-centos7-skylake_avx512 ^py-mpi4py@3.0.3%gcc@7.4.0 arch=linux-centos7-skylake_avx512 ^py-psutil@5.7.2%gcc@7.4.0 arch=linux-centos7-skylake_avx512 ^slate@2020.10.00%gcc@7.4.0+cuda~ipo+mpi+openmp+shared build_type=RelWithDebInfo arch=linux-centos7-skylake_avx512 ^blaspp@2020.10.02%gcc@7.4.0+cuda~ipo+openmp+shared build_type=RelWithDebInfo cuda_arch=none arch=linux-centos7-skylake_avx512 ^lapackpp@2020.10.02%gcc@7.4.0~ipo+shared build_type=RelWithDebInfo arch=linux-centos7-skylake_avx512 ^strumpack@5.0.0%gcc@7.4.0~build_dev_tests~build_tests+butterflypack+c_interface~count_flops+cuda~ipo+mpi+openmp+parmetis~scotch~shared~task_timers+zfp build_type=RelWithDebInfo cuda_arch=none arch=linux-centos7-skylake_avx512 ^zfp@0.5.5%gcc@7.4.0~aligned~fasthash~ipo~profile+shared~strided~twoway bsws=64 build_type=RelWithDebInfo arch=linux-centos7-skylake_avx512 ^tasmanian@7.3%gcc@7.4.0+blas+cuda~fortran~ipo+magma+mpi+openmp~python+xsdkflags build_type=Release arch=linux-centos7-skylake_avx512
The following open source HPC packages are installed by default with the xSDK. Note that you may override these and use a preinstalled version if you wish.