HTS Bioinf - Release and deployment of tsd-import
Scope
This procedure explains how to properly make new releases of the tsd-import package, and how to deploy the software into production environment.
Responsibility
Responsible person: A bioinformatician responsible for pipeline release.
Platforms
The software is located on different platforms, see the table below:
Host | Directory | Location | Servers |
---|---|---|---|
TSD | production | /ess/p22/data/durable/production |
p22-cluster-sync or p22-submit-dev |
NSC | transfer | /boston/diag/transfer |
sleipnir and beta |
1. Tag the repositories for the new release
Merge all changes into master
and tag the new release accordingly (from HEAD
of master
). Version name is vX.X.X, and should follow semantic versioning. The tag MUST end with -rel
:
Immediately push the tag to origin:
2. Export the repository
Open the pipelines view of Gitlab CI https://gitlab.com/DPIPE/labautomation/tsd-import and find a pipeline execution that was run on the relevant release tag.
Start the job release-artifact
.
Once finished download the tar
archive from the web page.
Transfer to TSD and NSC.
3. Deploy
1. Deploy on NSC
As lims-exporter-api
is run on beta and nsc-exporter
on sleipnir, to simplify the deployment process, we deploy on NSC only when those have been changed.
Note! The services running on beta and sleipnir must be stopped and started on their respective server. The removal and unpacking of the deploy package is done on beta.
Copy and archive the deployable package
Copy the exported tarball into {transfer}/sw/archive
directory on beta. This is in order to ensure that there always is an archived version of the software before deployment.
Stop nsc-exporter and lims-exporter-api
Refer to Execution and monitoring of pipeline section "Finish production" for instructions on how to stop nsc-exporter
and lims-exporter-api
.
Extract the package
Log into the server: ssh beta
Remove the old version:
rm -rf {transfer}/sw/tsd-import
```
Extract the package and verify that the version is as expected:
```bash
cd {transfer}/sw/
tar -xvf archive/<package-file>
cat tsd-import/version
Start nsc-exporter and lims-exporter-api
Refer to Execution and monitoring of pipeline section "Start production" for instructions on how to stop nsc-exporter
and lims-exporter-api
by {serviceuser}.
2. Deploy on TSD
Since the filelock-exporter
is only running on TSD, to simplify the deployment process, we go through this step only when filelock-exporter
has been changed.
Copy and archive the deployable packages
Copy the exported tarball into {production}/sw/archive
directories on TSD.
Stop filelock-exporter-api
Refer to Execution and monitoring of pipeline section "Finish production" for instructions on how to stop filelock-exporter-api
.
Extract the package
Remove the current module:
rm -rf {production}/sw/tsd-import
```
Extract the package and confirm that the version is correct:
On `cluster`:
```bash
cd {production}/sw
./slurm-wrapper-untar-tsd-import.sh <package-file>
# Wait until the job has finished on the cluster.
cat tsd-import/version
On durable
:
Note! The <package-file>
above must be located inside the sw/archive
directory, but do not include the directory name when referring to the package file.
Start filelock-exporter-api
Refer to Execution and monitoring of pipeline section "Start production" for instructions on how to stop filelock-exporter-api
by {serviceuser}.
Background
The tsd-import
package consists of three different programs, two of which run on the NSC side, and one running on the
TSD side:
- HTS Bioinf - Release and deployment of tsd-import
- Scope
- Responsibility
- Platforms
- 1. Tag the repositories for the new release
- 2. Export the repository
- 3. Deploy
- Background
Its sole purpose is to package samples from NSC and upload them safely into production area on TSD.
lims-exporter-api
The lims-exporter-api
script monitors the lims exporter queue in Clarity for samples and analyses ready to be exported. It
creates sample and analysis packages in the format expected in our automation system.
nsc-exporter
This script assists the user with uploading new samples/analyses
and backup pipeline results to S3 API endpoint on TSD in an
automatic fashion.
filelock-exporter-api
This script watches the S3 API endpoint on TSD for incoming samples and backup pipeline results that are not already
copied to cluster
or durable
or durable3
. When new samples or new backup pipeline results are detected, they are copied to
production area or durable
. There are checks for data integration. If a check has passed, the samples/analyses/ella-incoming
are marked with a READY
file for pickup by the automation system. If nsc-exporter
was run on NSC side, filelock-exporter-api
must be run on TSD side.