Skip to content

DPIPE Documentation

Software administration

DPIPE Documentation

Home
Organization
Organization
- Bioinformatics group roles
- Overall process of work organisation
Procedures
Procedures
- Onboarding
  Onboarding
- Production
  Production
  - Production routines
    Production routines
    
    Execution and monitoring of production pipelines
    
    Scheduled recurring jobs
    
    Running annoservice
    
    ELLA daily operations
    
    ELLA core production and maintenance
    
    Using service user
    
    Use of GDx operational channel in Teams
    
    User accounts management
    
    VardeDB manual procedures
  - Helpdesk routines
    Helpdesk routines
    
    GDx-bioinf Service Level Agreement
    
    GDx-bioinf helpdesk user page
    
    GDx-bioinf helpdesk requests
    
    GDx-bioinf helpdesk process
  - Annotation and reference data updates
    Annotation and reference data updates
    
    Update of external annotation data
    
    Update of in-house annotation data
    
    Update of CoNVaDING background data
    
    Generating IGV tracks for CNV annotations
  - Releases and production system updates
    Releases and production system updates
    
    Updates to production pipelines
    
    Release and deployment of TSD import
    
    Release and deployment of vcpipe
    
    Release and deployment of anno system
    
    Using ELLA track config manager
    
    Using the vcpipe staging environment
  - Capture regions and gene panels configuration
    Capture regions and gene panels configuration
    
    Create region files for new capture kits
    
    Design and build gene panels
    
    Export and deploy gene panels
    
    Update external reference data for the gene panel builder
- Development
  Development
Specifications and repo documentation
Specifications and repo documentation
- Systems and infrastructure
  Systems and infrastructure
  - Systems overview
  - Infrastructure
  - Storage and security of sensitive data
  - File system usage and locations
  - Software administration Software administration
    Table of contents
    
    Continuous monitoring of software updates
    
    Validation of changes to third-party software
- Production pipelines and automation
  Production pipelines and automation
  - Maestro library for pipeline automation
  - Steps implementention of Maestro
  - Demultiplexing and QC of raw seq data (NSC)
  - Basepipe pipeline
  - Triopipe pipeline
  - CNV exome targeted
  - TSD import (tsd-import)
    TSD import (tsd-import)
    
    TSD import and LIMS integration (tsd-import)
    
    tsd-import release notes
- Variant calling
  Variant calling
  - Variant calling pipeline (vcpipe)
    Variant calling pipeline (vcpipe)
    
    Variant calling pipeline (vcpipe)
    
    vcpipe architecture
    
    vcpipe automation: executor
    
    vcpipe webUI
    
    vcpipe pipelines
    
    vcpipe test datasets
    
    vcpipe reference datasets
    
    vcpipe release notes
  - CNV calling in targeted gene panels (CoNVaDING)
  - Noninvasive prenatal testing (NIPT)
    Noninvasive prenatal testing (NIPT)
    
    NIPT
    
    NIPT hardware and IT infrastructure
    
    NIPT repository
  - Preimplementation Genetic Testing (PGT)
    Preimplementation Genetic Testing (PGT)
    
    PGT repository
- Annotation and post-processing
  Annotation and post-processing
  - Anno and anno-targets
    Anno and anno-targets
    
    Anno with anno-targets
    
    Annopipe
    
    Anno official documentation
  - Post-processing and filtering of structural variants and CNVs
    Post-processing and filtering of structural variants and CNVs
    
    ELLA CNV module backend
    
    CNV filtering
    
    Post-processing of structural variants (sv-pack)
    
    sv-pack release notes
  - SNV filtering upstream of ELLA
- Interpretation
  Interpretation
  - ELLA: clincical variant interpretation
    ELLA: clincical variant interpretation
    
    ELLA official documentation
    
    ELLA official home page
    
    ELLA CNV module frontend
  - Gene panel builder (GPB)
    Gene panel builder (GPB)
    
    Gene panel builder
    
    User manual
    
    Release notes
    
    Background
    Background
    
    About gene panels
    
    Technical implementation
    
    Set up for production
    Set up for production
    
    Deploying to production
    
    Authentication and autorization
    
    Command-line interface
    
    Coverage and segmental duplications
    
    Set up for development
    Set up for development
    
    Development
    
    Demo deployment
- Data sharing
Documentation
Documentation
- About our documentation
- docs-builder (for docs webpages)
- docs-store (for docs not in other repos)
- Procedures for updating eHåndbok and docs
  Procedures for updating eHåndbok and docs
  - How to update eHåndbok (EHB)
  - How to update docs webpages

HTS Bioinf - Software administration

This document specifies the responsibilities for administration (forvaltningsansvar) of software used in the diagnostic pipeline and for IT systems used as part of development and testing of the pipeline.

The bioinformatics group has two roles with responsibilities for third-party software.

Pipeline software responsible : Responsible for third party bioinformatics software directly used in the diagnostic pipelines, e.g.:
- Dragen, GATK/Picard, BWA, Ensembl VEP
- Nextflow.io
Supporting software responsible: Responsible for all other software, e.g.:
- The OS of our development servers.
- Gitlab : source code repository with extra features such as code reviews and running tests.
- Docker : system for running virtual servers used in testing on the above servers
- Jira : issue tracking and software management, hosted externally at ousamg.atlassian.net
- The programming languages (Python, Perl, JavaScript, shell scripts) and core language libraries.

Continuous monitoring of software updates

The responsibility includes identifying when third-party software or external IT systems should be updated and make sure that the update doesn't introduce errors. The responsible persons should be member of relevant mailing lists and/or read news/homepage of the software covered by his/her role. Any changes relevant for security or correctness or that introduce new features useful for our use should be considered. If critical updates (e.g. serious bugs that can affect analysis quality) are released, then the bioinformatics group should be notified and the update be applied as soon as possible.

Validation of changes to third-party software

To reduce the risk of introducing defects into production, we:

run a set of automated tests in our test environment after each change
run the pipeline using a reference sample and compare the result to a reference
aim to have separate deploys for upgrades to third-part software and for our own code changes

Separate deploys are sometimes not possible when software upgrade also requires changes to our own code or configuration. For more info on validation see "HTS Bioinf - Process for updates to production pipelines".

Since any changes in the supporting software doesn't affect the diagnostic results, the changes don't need a formal validation. Also the changes can often be reverted.