diff --git a/containers/containerization.md b/containers/containerization.md index f6b25c8a..dc0a5338 100644 --- a/containers/containerization.md +++ b/containers/containerization.md @@ -19,8 +19,10 @@ While running containers on HPC platforms does not guarantee the same performanc Containers, from a broader viewpoint, are similar to virtual machines. They attempt to isolate an application and its dependencies into a self-contained "box". The main difference is in their implementation or architecture/design.

- virtual machines - containers + virtual machines +

+

+ containers

Figure 1: Virtual Machines (left) structure comparison with Containers (right).

@@ -178,6 +180,7 @@ Singularity, in some ways, is similar to Docker but the primary difference is th In 2021, Singularity development forked into two organization-led efforts: Singularity (with Sylabs) and Apptainer ([part of the Linux Foundation](https://apptainer.org/news/community-announcement-20211130)). Basically, Sylabs' fork from Singularity in 2021 kept the name Singularity (confusing) and Apptainer started with a copy of Singularity ([kept as a snapshot](https://github.com/apptainer/singularity)) and has since branched from there. The versions of Singularity avaialble on NAS are still under Sylabs as [Community Editions](https://github.com/sylabs/singularity). Here are some basic SingularityCE workflow commands: + - `singularity pull` - Can specify either docker (DockerHub) images or library ([Sylabs' Container Library](https://cloud.sylabs.io/library) similar to DockerHub) or shub ([Singularity Hub](https://singularityhub.com/) through Standford University) - `singularity build` - Can build to a SIF (Singularity Image File) or to a `--sandbox dir_name` directory on the filesystem. Albeit confusing, you can also build a SIF file from a sandbox directory. - `singularity run` - This will need bind mounted directories added to it unless environment variables have been set appropriately. Also, this command can be replaced with a `./` if using a SIF image that comes from a definition file with the `%runscript` section used. @@ -191,12 +194,11 @@ Singularity, as opposed to Docker, has defined sections within their definition - `--fakeroot` will need to be used whenever building from a definition file on NAS. This simulates an elevated user priviledge within the container environment. - When building to a sandbox (similar to a folder on the filesystem), sometimes the Lustre filesystem has [issues removing directories](https://github.com/apptainer/singularity/issues/5450) within the sandbox. Waiting a day will allow the system to let you remove them. - The `pwd` or `cwd` will have [different values](https://stackoverflow.com/a/73153806) inside the definition file and when running an image/sandbox. This also correlates to different environment bindings that occur when running singularity images. It is recommended to run with the `-e` option so as to not rely upon external environments or filesystems. Refer to the image below for a visual of some of the bindings that occur automatically in Singularity. -
- +

+file directory structure within a container Mapping of file content and directories from a host system into a Singularity container. - -

+

- Unlike Docker, Singularity does not have a `images` command to show the cache of images built. Instead, you can see them via `singularity cache list -v` ([link](https://stackoverflow.com/a/68543409)) and clear the cache by `singularity cache clean`. Also, a `sif` image file can act as a standalone Singularity executable image. - __Note:__ The Singularity cache for NAS systems needs to be changed to a `$NOBACKUP` location as it defaults to the user's `$HOME` directory. @@ -232,6 +234,7 @@ Once you have the GAMERA code (in a folder named `kaiju`), you should have a dir _Note:_ At the time of this writing, the `MAGE_CCMC_0.75.0` tag is the latest release. Here are the steps in order to build the model code: + 1. Modify the `modules_nas` file to contain any modifications you have to the path to your `kaiju` folder (last line of this file). 2. Source this `modules_nas` file in order to set your Singularity cache directory (very important) and other necessary environment variables for the container. - This loads one of the currently available two singularity environment modules (version 3.11.0). diff --git a/containers/screenshot.png b/containers/screenshot.png new file mode 100644 index 00000000..fbf48851 --- /dev/null +++ b/containers/screenshot.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d2ecbfb513e8bebb548a550d62ddec074a61cf3af99f1ec4c05a08291eda0f3e +size 41764