HTS Bioinf - Release and deployment of variantcalling system
Scope
This document describes how to update the softwares under the variantcalling system on TSD and NSC. This assumes that a software package has been created and is available for deployment. The procedure for developing, testing and documenting pipeline software is described in HTS Bioinf - Process for updates to production pipelines.
Responsibility
See the role descriptions in the procedure HTS Bioinf - Bioinformatics Group roles Release: Release coordinator variant calling Deployment: On duty production duty responsible
Note that "Endringskontroll for bioinformatiske endringer" should be filled in and approved by a manager prior to deploying any major or minor versions (does not apply to hotfixes).
Platforms
All production paths (starting with {production}/
) are given relative to the base path for the system in question:
System | Directory | Location |
---|---|---|
TSD | production | /ess/p22/data/durable |
NSC | production | /boston/diag/ |
As the system can run on different production platforms, the setup can vary slightly.
The relevant servers are gdx-login
, gdx-executor
and gdx-webui
on NSC and p22-hpc-01
, p22-hpc-02
and p22-hpc-03
on TSD.
1. Copy and archive the deployable package to the production platforms
Transfer the package, its checksum and any Singularity images to TSD using the data transfer API. The files will end up in /ess/p22/data/durable/file-import
.
Make copies of the transferred files in /ess/p22/data/durable/file-export
in order to make them accessible from NSC later.
For TSD
- Log into
p22-hpc-01
(orp22-hpc-03
if using the backup server). - Move the package and its checksum to the
{production}/sw/archive
directory. - Move the Singularity images to their respective directories in
/ess/p22/data/durable/production/sw/variantcalling
.
For NSC
- Log into
sleipnir
. - Use
tacl
to download the files from TSD and place them in/boston/diag/transfer
. - Back on
gdx-login
, move the package and its checksum to{production}/sw/archive
. - Always on
gdx-login
, move the Singularity images to/boston/diag/singularity
.
2. Stop production processes
Refer to section "Finish production" for instructions on how to stop executor
and webui
on TSD and NSC.
3. Extract the package in production and staging
AvailableENVIRONMENT_NAME
: production
, staging
Available SYSTEM_NAME
*: vcpipe
, vcpipe-utilities
, site-packages
*: site-packages
are python modules required in the vcpipe
Verify that the version of the deployed package matches the one intended:
Note
The deployment must always be done on both NSC and TSD, production and staging
4. Start production processes
Refer to section "Start production" for instructions on how to start executor
and webui
on TSD and NSC as service user.