Ansible Roles

Documentation for NSP ansible roles.

codee

Description

Installs Codee and optionally generates a module file.

Variables

`NSP_CODEE_version`

Type:: str
Required:: true
Description:: The version to install.

`NSP_CODEE_license_file`

Type:: str
Required:: true
Description:: This variable specifies which license file codee should use. License files should be placed in "{{ [NSP_install_root, 'codee/licenses'] | path_join }}".

`NSP_CODEE_create_module`

Type:: bool
Required:: false
Default:: true
Description:: Toggles module generation.

cuda

Description

Installs CUDA and optionally generates a module file.

Variables

`NSP_CUDA_version`

Type:: str
Required:: true
Description:: The version to install.

`NSP_CUDA_driver`

Type:: str
Required:: true
Description:: The Nvidia driver version specified in the download URL.

`NSP_CUDA_architecture`

Type:: str
Required:: false
Default:: "linux"
Description:: The CUDA architecture specified in the download URL.

`NSP_CUDA_create_module`

Type:: bool
Required:: false
Default:: true
Description:: Toggles module generation.

files

Description

Deploys files and directories to specified locations. All of the source files should be stored in your home directory under a folder called files.

Variables

`NSP_FILES_inventory`

Type:: list
Required:: false
Default:: []
Description:: Mapping of file/directories and their target locations. Each entry is a dictionary that takes two mandatory keys src and dest and one optional key file_permissions. file_permissions overrides the value of NSP_file_permissions.

NSP_FILES_inventory:
  - src: <some file or dir in `playbook_dir`/files>
    dest: <destination on machine>
    file_permissions: <unix permission string>

gcc

Description

Installs GCC and optionally generates a module file.

Variables

`NSP_GCC_version`

Type:: str
Required:: true
Description:: The version to install.

`NSP_GCC_enabled_languages`

Type:: list
Required:: false
Default:: ["c", "c++", "fortran"]
Description:: The languages that the build will support.

`NSP_GCC_create_module`

Type:: bool
Required:: false
Default:: true
Description:: Toggles module generation.

`NSP_GCC_clear_source`

Type:: bool
Required:: false
Default:: true
Description:: Clear previous build source files.

init

Description

The init role initializes the system init scripts and adds your content to them. To add contents, create a directory called init in the directory that contains your playbook. In the init directory create the files profile.j2 and cshrc.j2 with your desired content.

Note

The init directory and accompanying files are completely optional. Without them, blank init script will be created.

Variables

`NSP_INIT_wipe`

Type:: bool
Required:: false
Default:: false
Description:: Wipe and recreate the init scripts on every run.

llvm

Description

Installs LLVM and optionally generates a module file.

Variables

`NSP_LLVM_version`

Type:: str
Required:: true
Description:: The version to install.

`NSP_LLVM_projects`

Type:: list
Required:: false
Default:: ["clang", "lld", "openmp", "compiler-rt"]
Description:: LLVM projects to build.

`NSP_LLVM_runtimes`

Type:: list
Required:: false
Default:: ["libcxx", "libcxxabi", "libunwind"]
Description:: Library runtimes to build.

`NSP_LLVM_create_module`

Type:: bool
Required:: false
Default:: true
Description:: Toggles module generation.

`NSP_LLVM_clear_source`

Type:: bool
Required:: false
Default:: true
Description:: Clear previous build source files.

lmod

Description

The lmod role configures and under certain circumstances installs lmod. The main part of the configuration is a custom lmod hook that can be configured to look for modules in spack projections and add paths to MODULEPATH when appropriate. It also supports having custom lmod rc.lua, admin.list, and lmodrc.lua files. These files should be placed in a directory named lmod that resides with your system playbook.

Note

The lmod directory and accompanying files are completely optional. If no template files are provided the lmod role will skip generating them.

Variables

`NSP_LMOD_install_type`

Type:: str
Required:: true
Allowed Values:: ["internal", "external"]
Description:: This toggles the lmod role installing its own lmod or using an external installation.

`NSP_LMOD_version`

Type:: str
Required:: false
Description:: The version to install (only applies if NSP_LMOD_install_type = internal).

`NSP_LMOD_enable_tcl`

Type:: bool
Required:: false
Default:: false
Description:: Build support for tcl module files (only applies if NSP_LMOD_install_type = internal).

`NSP_LMOD_default_modules`

Type:: list
Required:: false
Default:: []
Description:: A list of modules to load in the DefApps module.

Note

If you are installing lmod with NSP then the DefApps module will be set to load automatically. If you are using an external lmod installation you will have to configure loading DefApps outside of NSP.

`NSP_LMOD_default_module_paths`

Type:: list
Required:: false
Default:: []
Description:: List of paths to add to MODULEPATH (only applies if NSP_LMOD_install_type = internal).

Note

The path in NSP_module_root will automatically be added. So, there is no need to add it to NSP_LMOD_default_module_paths.

`NSP_LMOD_spack_modules`

Type:: path
Required:: false
Default:: "{{ [NSP_install_root, 'spack/modules'] | path_join }}"
Description:: Path to the spack generated module files.

`NSP_LMOD_hierarchy`

Type:: dict
Required:: false
Default:: {}
Description:: This variable holds a list of components for module projections.

For example:

NSP_LMOD_hierarchy:
  compiler:
    members: ['gcc', 'llvm']
    paths:
      - {path: '|compiler.name|-|compiler.version|', weight: 20}
      - {path: '|mpi.name|-|mpi.version|/|compiler.name|-|compiler.version|', weight: 30}
    level: 0
  mpi:
    members: ['openmpi', 'mpich']
    paths:
      - {path: '|mpi.name|-|mpi.version|/|compiler.name|-|compiler.version|', weight: 30}
    level: 1

Notice the two components named compiler and mpi. Each component has a list of members. These members are the names of modules that belong to that component. In the module files for these members, they should share the same family("...") so that they are mutually exclusive. For each component we also define a list of paths. These paths template the paths that the hook should add/remove from MODULEPATH. Take for example the first path defined for compiler. If gcc/12.3.0 were loaded then |compiler.name|-|compiler.version| would become gcc-12.3.0. These paths are combined with NSP_LMOD_spack_modules to create the final path to add to MODULEPATH. If NSP_LMOD_enable_spack_compiler_projections is set then additional parts are added to the path to accommodate spack’s automatic additions for compiler and version. Each path also has an associated weight. The last item in each component is level which sets the order that the different component members are printed in the lmod header for module avail.

`NSP_LMOD_nv_mappings`

Type:: dict
Required:: false
Default:: {}
Description:: Converts module names to their package names in spack.

For example:

NSP_LMOD_nv_mappings:
    gcc-native/12.3.0: {name: 'gcc', version: '%s'} # maps to gcc/12.3.0
    # %s substitutes in the value from the system module

Sometimes, spack uses a different name for a package than the system does. For example, on Cray systems we now have gcc-native but spack just uses gcc. By default the hook will not pick up on the difference. In order to fix this problem you can define an entry here that will remap a module name to the spack equivalent.

`NSP_LMOD_path_names`

Type:: dict
Required:: false
Default:: []
Description:: Defines lmod section header names for paths that meet the specified regexes.

For example:

NSP_LMOD_path_names:
  /opt/cray: "[ Cray Programming Environment ]"

All modules that reside somewhere under /opt/cray will appear in the [ Cray Programming Environment ] section when using module avail.

`NSP_LMOD_enable_spack_compiler_projections`

Type:: bool
Required:: false
Default:: true
Description:: Toggles that addition of paths in the hook that spack’s automatic additions for compiler and version.

Caution

When in doubt, leave this variable with its default value.

`NSP_LMOD_enable_logging`

Type:: bool
Required:: false
Default:: false
Description:: Enable logging of module loads.

`NSP_LMOD_logging_url`

Type:: str
Required:: false
Default:: "http://localhost:8080/"
Description:: API URL to send module loads to.

miniforge3

Description

The miniforge3 role installs miniforge3 with the specified packages in the base environment. And optionally generates a module file.

Variables

`NSP_MINIFORGE3_version`

Type:: str
Required:: true
Description:: The version to install.

`NSP_MINIFORGE3_revision`

Type:: str
Required:: false
Default:: "0"
Description:: The version revision to install.

`NSP_MINIFORGE3_base_packages`

Type:: list
Required:: false
Default:: []
Description:: A list of packages to install in the base environment.

`NSP_MINIFORGE3_clean_install`

Type:: bool
Required:: false
Default:: true
Description:: Delete installer and re-download.

`NSP_MINIFORGE3_create_module`

Type:: bool
Required:: false
Default:: true
Description:: Toggles module generation.

nsp

Description

This is a meta role that sets up needed infrastructure for the other NSP roles.

Variables

`NSP_system_name`

Type:: str
Required:: true
Description:: The system’s name.

`NSP_help_email`

Type:: str
Required:: true
Description:: Help email.

`NSP_site_name`

Type:: str
Required:: true
Description:: Name of HPC site.

`NSP_install_root`

Type:: path
Required:: false
Default:: "{{ ['/sw', NSP_system_name] | path_join }}"
Description:: The path prefix to NSP installed software.

`NSP_module_root`

Type:: path
Required:: false
Default:: "{{ [NSP_install_root, 'modules'] | path_join }}"
Description:: The path prefix to NSP software modules.

`NSP_max_threads`

Type:: int
Required:: false
Default:: "{{ [((ansible_processor_nproc * 0.75) | int | abs), 16] | min }}"
Description:: Number of threads to use when building software.

`NSP_scratch_directory`

Type:: path
Required:: false
Default:: "{{ ['/tmp', ansible_user_id] | path_join }}"
Description:: Scratch directory to perform builds in.

`NSP_keep_scratch_clear`

Type:: bool
Required:: false
Default:: false
Description:: Toggle retention of scratch directory contents after builds.

`NSP_user`

Type:: str
Required:: false
Default:: "{{ ansible_user_uid }}"
Description:: The unix user for software installations.

`NSP_group`

Type:: str
Required:: false
Default:: "{{ ansible_user_gid }}"
Description:: The unix group for software installation.

`NSP_file_permissions`

Type:: str
Required:: false
Default:: "644"
Description:: The unix permissions for files (non-executables).

`NSP_executable_permissions`

Type:: str
Required:: false
Default:: "755"
Description:: The unix permissions for executables (including folders).

`NSP_architecture`

Type:: str
Required:: false
Default:: "{{ ansible_architecture }}"
Description:: Hardware architecture to target.

oneapi

Description

Installs Intel’s OneAPI and optionally generates a module file.

Variables

`NSP_ONEAPI_version`

Type:: str
Required:: true
Description:: The version to install.

`NSP_ONEAPI_base_toolkit_url`

Type:: str
Required:: true
Description:: The download URL for the base toolkit installer.

`NSP_ONEAPI_base_toolkit_components`

Type:: list
Required:: false
Default:: ["intel.oneapi.lin.dpcpp-ct", "intel.oneapi.lin.dpcpp_dbg", "intel.oneapi.lin.dpl", "intel.oneapi.lin.tbb.devel", "intel.oneapi.lin.dpl", "intel.oneapi.lin.dal.devel", "intel.oneapi.lin.ipp.devel", "intel.oneapi.lin.ippcp.devel", "intel.oneapi.lin.mkl.devel", "intel.oneapi.lin.advisor", "intel.oneapi.lin.vtune"]
Description:: The base toolkit components to install.

`NSP_ONEAPI_hpc_toolkit_url`

Type:: str
Required:: true
Description:: The download URL for the HPC toolkit installer.

`NSP_ONEAPI_hpc_toolkit_components`

Type:: list
Required:: false
Default:: ["intel.oneapi.lin.ifort-compiler"]
Description:: The HPC toolkit components to install.

`NSP_ONEAPI_create_module`

Type:: bool
Required:: false
Default:: true
Description:: Toggles module generation.

spack

Description

The spack role templates spack environments for deployment.

If you have any spack extentions they should be placed in the playbook directory under spack/extentions. Extensions will be deployed in "{{ [NSP_SPACK_config_directory, 'extensions'] | path_join }}" . All Spack patch files should go in the playbook directory under spack/patches.

Variables

`NSP_SPACK_config_directory`

Type:: path
Required:: false
Default:: "{{ [NSP_install_root, 'spack'] | path_join }}"
Description:: The target directory for spack configuration files.

`NSP_SPACK_repo`

Type:: str
Required:: false
Default:: "https://github.com/spack/spack.git"
Description:: The Spack repo to clone.

`NSP_SPACK_versions`

Type:: dict
Required:: true
Description:: Defines available spack versions for the environments.

NSP_SPACK_versions:
  v0.23.1:
    git_reference: 2bfcc69
    patch: v0.23.1.patch

`NSP_SPACK_environments`

Type:: dict
Required:: true
Description:: Defines spack environments.

For example:

NSP_SPACK_environments:
  example_env:
    spack_version: v0.23.1
    extensions:
      - spack-olcf
    specific_templates:
      - packages
    shared_templates:
      - mirrors

For each environment you should create a directory with the same name in the playbook directory under spack/environments.

spack_version should reference a spack version defined in NSP_SPACK_versions and extensions should be a list of extensions that can be found in the playbook directory under spack/extensions.

We like to split our spack configuration for each environment into multiple files. specific_templates are templates that are specific to each environment. They should be places under spack/environments/<environment_name>. shared_templates are used by all of the environments and should be placed under spack/environments. There is a third catagory of templates that should not go into your configuration but is important to know about. We call them nsp_templates these are templates that are shared by all environments but instead of being in your playbook directory they are provided by the spack role itself.

Ansible Roles

codee

Description

Variables

NSP_CODEE_version

NSP_CODEE_license_file

NSP_CODEE_create_module

cuda

Description

Variables

NSP_CUDA_version

NSP_CUDA_driver

NSP_CUDA_architecture

NSP_CUDA_create_module

files

Description

Variables

NSP_FILES_inventory

gcc

Description

Variables

NSP_GCC_version

NSP_GCC_enabled_languages

NSP_GCC_create_module

NSP_GCC_clear_source

init

Description

Variables

NSP_INIT_wipe

llvm

Description

Variables

NSP_LLVM_version

NSP_LLVM_projects

NSP_LLVM_runtimes

NSP_LLVM_create_module

NSP_LLVM_clear_source

lmod

Description

Variables

NSP_LMOD_install_type

NSP_LMOD_version

NSP_LMOD_enable_tcl

NSP_LMOD_default_modules

NSP_LMOD_default_module_paths

NSP_LMOD_spack_modules

NSP_LMOD_hierarchy

NSP_LMOD_nv_mappings

NSP_LMOD_path_names

NSP_LMOD_enable_spack_compiler_projections

NSP_LMOD_enable_logging

NSP_LMOD_logging_url

miniforge3

Description

Variables

NSP_MINIFORGE3_version

NSP_MINIFORGE3_revision

NSP_MINIFORGE3_base_packages

NSP_MINIFORGE3_clean_install

NSP_MINIFORGE3_create_module

nsp

Description

Variables

NSP_system_name

NSP_help_email

NSP_site_name

NSP_install_root

NSP_module_root

NSP_max_threads

NSP_scratch_directory

NSP_keep_scratch_clear

NSP_user

NSP_group

NSP_file_permissions

NSP_executable_permissions

NSP_architecture

oneapi

Description

Variables

NSP_ONEAPI_version

`NSP_CODEE_version`

`NSP_CODEE_license_file`

`NSP_CODEE_create_module`

`NSP_CUDA_version`

`NSP_CUDA_driver`

`NSP_CUDA_architecture`

`NSP_CUDA_create_module`

`NSP_FILES_inventory`

`NSP_GCC_version`

`NSP_GCC_enabled_languages`

`NSP_GCC_create_module`

`NSP_GCC_clear_source`

`NSP_INIT_wipe`

`NSP_LLVM_version`

`NSP_LLVM_projects`

`NSP_LLVM_runtimes`

`NSP_LLVM_create_module`

`NSP_LLVM_clear_source`

`NSP_LMOD_install_type`

`NSP_LMOD_version`

`NSP_LMOD_enable_tcl`

`NSP_LMOD_default_modules`

`NSP_LMOD_default_module_paths`

`NSP_LMOD_spack_modules`

`NSP_LMOD_hierarchy`

`NSP_LMOD_nv_mappings`

`NSP_LMOD_path_names`

`NSP_LMOD_enable_spack_compiler_projections`

`NSP_LMOD_enable_logging`

`NSP_LMOD_logging_url`

`NSP_MINIFORGE3_version`

`NSP_MINIFORGE3_revision`

`NSP_MINIFORGE3_base_packages`

`NSP_MINIFORGE3_clean_install`

`NSP_MINIFORGE3_create_module`

`NSP_system_name`

`NSP_help_email`

`NSP_site_name`

`NSP_install_root`

`NSP_module_root`

`NSP_max_threads`

`NSP_scratch_directory`

`NSP_keep_scratch_clear`

`NSP_user`

`NSP_group`

`NSP_file_permissions`

`NSP_executable_permissions`

`NSP_architecture`

`NSP_ONEAPI_version`

`NSP_ONEAPI_base_toolkit_url`

`NSP_ONEAPI_base_toolkit_components`

`NSP_ONEAPI_hpc_toolkit_url`

`NSP_ONEAPI_hpc_toolkit_components`

`NSP_ONEAPI_create_module`

`NSP_SPACK_config_directory`

`NSP_SPACK_repo`

`NSP_SPACK_versions`

`NSP_SPACK_environments`