Skip to content

HTS Bioinf - Export and deploy gene panels

Scope

This procedure explains how to export gene panel files from the computer holding the gene panel database, eventually comparing them to extant production instances along the way.

Responsibility

A bioinformatician from GDx is responsible for performing these tasks.

The gene panel builder service

The gene panel builder service runs on https://gpb.allel.es.

Export

SSH into gpb.allel.es. [we will assume the build date to be stored in variable DATE=$(date '+%Y-%m-%d')]:

ssh gpbuilder@gpb.allele.es
# make sure that the gene panels repository is up to date with the remote
git -C genepanel-store checkout '<latest-release-tag>'
# enter the gene panel builder directory and fire up a gene panel builder Docker container
cd genepanel-builder
docker exec -it gpbuilder bash

# *only* when running a periodic release of all panels, update all external data:
gpb update all-external

# export the gene panel files (explicit panel-Id(s) aren't needed when updating _all_ panels)
gpb export panels -o /exported-panels/${DATE} --compare-to /current-panels [<panel-Id(s)>]

# If you are running a periodic release, create whole exome panels for each unit in (EGG, EHG, EKG)
# Note: The version should receive a major bump for every update (e.g. 1.0.0 -> 2.0.0), so fetch the
# latest release of the gene panel store for the latest version.
for unit in EGG EHG EKG ; do
  gpb export genes --coding-only -o /exported-panels/${DATE}
Eksom${unit} <version>
done

exit

Copy the diff report to the designated area on the hospital network and notify ELL about the update.

Draft an "Endringskontroll" document.

Release

Once the "Endringskontroll" is approved, the gene panels repository must be updated and the updated files transferred to the production infrastructures (NSC and TSD).

The gene panel files repository is located at gitlab.com/alleles/genepanel-store.

The procedure at a glance:

  • Export the gene panels
  • Update the 'genepanel-store' repository with the exported gene panel files;
  • Tag the repository state and push the tag to remote;
  • Transfer the archive to NSC and TSD;
  • Import the panels in ELLA;
  • Notify everyone that the gene panels have been updated.

The procedure in detail:

SSH into gpb.allel.es.

ssh gpbuilder@gpb.allele.es
# make sure that the gene panels repository is up to date with the remote
git -C genepanel-store checkout '<latest-release-tag>'
# enter the gene panel builder directory and fire up a gene panel builder Docker container
cd genepanel-builder
docker exec -it gpbuilder bash

# export the gene panel files (suppress drafts)
gpb export panels -o /exported-panels/${DATE} --no-draft

exit

# copy the new gene panel files into the repository:
rsync -avz exported-panels/${DATE}/ ../genepanel-store/panels/

Change into the gene panel store repository, tag a new release and push it to remote:

cd ../genepanel-store
# make sure that all existing panels with an incoming version update are removed
git add -A
git commit -m "Build ${DATE}"
git tag -a -m "Tag Build ${DATE}" ${DATE}-rel
git push; git push origin ${DATE}-rel
# Once the pipeline for the tag is complete (<5 minutes; https://gitlab.com/alleles/genepanel-store/-/pipelines), run this:
wget https://genepanel-store.fra1.digitaloceanspaces.com/${DATE}-rel/genepanel-store-${DATE}-rel.tar.gz

Upload the archive to TSD first and copy the gene panels files to the production directory.

  • Log onto TSD, extract the gene panels files and sync them with the local ones:
#TSD
mkdir /tmp/${DATE}-rel
tar xvzf /ess/p22/data/durable/file-import/p22-member-group/genepanel-store-${DATE}-rel.tar.gz \
  --directory /tmp/${DATE}-rel
rsync -avz /tmp/${DATE}-rel/ /ess/p22/cluster/production/genepanel-store/
  • Thereafter, to get the data to NSC, move the repository archive to the export directory:
mv /tsd/p22/data/durable/file-import/p22-member-group/genepanel-store-${DATE}-rel.tar.gz \
  /tsd/p22/data/durable/file-export/

Download the repository archive from sleipnir, on the NSC network:

Follow the same procedure as on TSD to extract the gene panels files and sync them with the local ones.

Back on TSD, remember to remove the archive file from the export directory.

Import the gene panels in ELLA (see (../Production-routines/ELLA-production/#Adding new gene panels)).

Notify all units about the gene panels update.

Fill in the update date in "Endringskontroll".