Skip to content

NIPT repository

This repository contains in-house scripts for the NIPT pipeline. Scripts are divided into following modules: * adhoc_batching
* lot_instrument_tracing * duplicate_handling * fcs_reports * swl_integration * genologics * genologics_sql

Each module(except for the helper modules) has an executor and is run continuously through cron jobs(./exe/cron.job)

adhoc_batching

Officially, two options for VeriSeq NIPT batching are provided: 1. Using Clarity
Manually select the samples to be run in Clarity and start the workflow and complete Step 1 of the Workflow. Then the Sample Sheet for VeriSeq is generated automatically. 2. Do ad hoc batching (without Clarity)
Directly load a rack of "random" samples to the robot, then either use the default Sample Sheet (with default metadata, for example, a singleton or twin, basic analysis or genome wide etc.) or edit the Sample Sheet manually in VeriSeq Workflow Manager.

We however want to be able to load a rack of "random" samples as ad hoc batching does, but still want Clarity to generate the Sample Sheet automatically (the metadata is already in Clarity). The adhoc_batching module makes this possible by automatically picking up the "random" samples loaded on the robot in Clarity, starting the workflow and completing Step 1 to let Clarity generate the Sample Sheet automatically.

lot_instrument_tracing

We want to automatically register the reagents and kits used for lab prep. This module parse the reagent report and register the info in Clarity.

duplicate_handling

VeriSeq NIPT Integration for Clarity doesn't allow the same sample to be registered in Clarity multiple times. But lab doesn't want to screen the duplicated samples before importing them into Clarity. For samples that failed the initial analysis, lab wants to include them in a new project (for better tracing). So duplication need to be handled inside Clarity to make sure the VeriSeq NIPT Integration doesn't crash.

fcs_report

We want to also register the sequencing reagent kits as well as instruments, for example NextSeq, PCR machine etc. used for each NIPT run. To make this happen, one additional step (Step 2) has been added to the official NIPT workflow supplied by Illumina. This additional step must be completed manually first before samples can reach the Step 3 queue of the NIPT workflow. VeriSeq NIPT Integration service monitors new batch initiation reports. If a batch initiation report is found, VeriSeq NIPT Integration service checks if the samples are already in the queue of Step 3, and if samples don't reach Step 3 queue within 5 tries, the workflow is aborted. This causes problems for the NIPT Workflow because lab can not complete Step 2 in time. This module solves the problem by feeding the batch initiation report to the VeriSeq NIPT Integration only after lab has completed Step 2.

swl_integration

This module parses the original NIPT report produced by VeriSeq, does interpretation following pre-defined rules and generates an interpreted NIPT report. The interpreted NIPT report is the final product which is copied over to OUS network. Once the interpreted NIPT report is on OUS network, the SwissLab NIPT automation parses the report, imports parsed data into SwissLab and triggers automatic answering out if there are no findings (otherwise, leaves the sample for lab doctors to process manually).

genologics

Helper module for using Clarity API.

genologics_sql

Helper module for accessing Clarity database.

Common problems and solutions

🔴 Sample sheet isn't generated

How to find out:

  • Lab informs bioinformaticians when Sample Sheet for Pooling isn't generated automatically.

🟢 Steps to solve:

Log into ous-lims sever ( ssh username@ous-lims.sequencing.uio.no) and check the "adhoc_batching" module log at /opt/gls/clarity/customextensions/nipt-clarity/logs/adhoc_batching/. Normally, the error message in the log should clearly describe the problem. The most common error message is: 98765432101 not in "Generate Batch & Sample Sheet" queue.

Scenario A -- The error message in the log clearly indicates a non-existing ID

Check the "Generate Batch & Sample Sheet" queue of "VeriSeq NIPT v1.0_EHD" protocol; confirm the erroneous sample isn't in the queue. Inform lab to fix this. Two common reasons: 1. The sample has not been imported into Clarity yet. \ In this case, the Lab should import the missing sample into Clarity and assign it to the NIPT workflow. The Sample Sheet is generated shortly after the sample is assigned to the workflow. 2. The sample is in Clarity, but the barcode scanned by the Hamilton robot does not match the sample name in Clarity. Once the barcode is scanned, it is impossible to change the barcode, so the sample ID must be changed in Clarity for the workflow to proceed. * Lab gives you the project name, for example, Diag-NIPT300, the scanned code (the wrong sample name), for example 10000000 and the correct sample name in Clarity, for example, 20000000. * Perform the following actions between 05 and 55 minutes of any hour. This is for preventing the duplication deletion script -- which runs at every whole hour (00 minutes) -- from deleting the modified sample in case the modified sample name conflicts with an existing sample that has already gone through the workflow. * In Clarity, DASHBOARD -> Projects -> Search for the project -> Click NN In-progress samples -> Click the Derived Sample matching the correct sample name (for example 20000000) -> Click the Action 1 Sample button -> Click Rename container -> type in the scanned code (for example 10000000) -> Click Continue -> You see "Rename container running" message; wait until "Rename container complete" message is shown or the "Rename container running" message disappears. * On the same page, keep the same sample selected, click the Action 1 Sample again -> Click Rename single entity -> type in the scanned code (for example 10000000) again -> Click Continue -> wait until Rename single entity complete message is shown * Wait around 5 minutes and confirm that the Sample Sheet is generated in /mnt/nipt-lab/SampleSheets/, for example Diag-NIPT300-2024-01-03.txt. * Open the Sample Sheet, confirm that the scanned code (for example 10000000) is present and the correct name (for example 20000000) is not. * Inform Lab that the Sample Sheet is ready to use. * Wait until the sequencing and the NIPT analysis are finished and the interpreted NIPT report is generated (this takes around 30 hours). * Open the interpreted NIPT report, for example, /mnt/veriseq-server-output/Diag-NIPT300-2024-01-03/Diag-NIPT300-2024-01-03_.*_nipt_report_.*.tab, and confirm that the scanned code (for example 10000000) is present and the correct name (for example 20000000) is not. * Confirm that the NIPT workflow Step 3 "AUTOMATED - Data Analysis (VeriSeq NIPT v1.0)_EHD" has completed.

Scenario B -- There are no clear error messages in the log

Check if the in-house cron jobs or the VeriSeq NIPT Integration service have stopped (the steps are described below).

🔴 The in-house automation cron jobs stopped

How to find out:

  • There is a new batch initiation report in /mnt/veriseq-server-output/<project>/ProcessLogs folder
  • In the log of "adhoc_batching" module, the new batch initiation report isn't captured (the log stopped updating). Then the cron jobs may have stopped.

🟢 Steps to solve:

  1. ssh to ous-lims.sequencing.uio.no
  2. Run sudo -u glsai -s
  3. Run crontab /opt/gls/clarity/customextensions/nipt-clarity/nipt/exe/cron.job to restart all in-house cron jobs.

🔴 The VeriSeq NIPT Integration service stopped

How to find out:

  • A new batch initiation report or a NIPT report has been copied over to /opt/gls/clarity/customextensions/nipt-clarity/fcs_report folder, but the third step of the "VeriSeq NIPT v1.0_EHD" protocol isn't started or completing.
  • Run sudo -u glsai -s
  • Run cd /opt/gls/clarity/extensions/veriseq-nipt/FileCaptureService
  • The VeriSeqNIPTLauncher.log file isn't updating.

🟢 Steps to solve:

  1. Go back to your own user(exit sudo glsai user).
  2. Run sudo /etc/init.d/veriseq-nipt_service stop
  3. Run sudo /etc/init.d/veriseq-nipt_service start to restart the service.

🔴 Remove NTC samples from workflow

When a project is stopped, all samples can be removed from workflow as normal, except for the NTC samples. NTC samples must finish the normal workflow as follow: 1. Add all the NTC samples to be removed in the "AUTOMATED - Data Analysis (VeriSeq NIPT v1.0)_EHD" queue -> Click VIEW ICEBUCKET button -> Click BEGIN WORK button. 2. On the Step Setup screen, click the "Upload file" button and upload the dummy_batch_initiation_report.tab in this repository. 3. Keep Continue until Finish step.