Skip to content

NIPT repository

This repository contains in-house scripts for the NIPT pipeline.

Contents

Table of contents

Modules

Scripts are organized into modules. Each module, except for the helper modules, includes an executor and is executed continuously through cron jobs (exe/cron.job).

adhoc_batching

Officially, there are two methods for VeriSeq NIPT batching:

  1. Using Clarity: Manually select samples in Clarity, start the workflow, and complete Step 1. The Sample Sheet for VeriSeq is then generated automatically.
  2. Ad hoc batching (without Clarity): Load a rack of "random" samples directly onto the robot. You can either use the default Sample Sheet with default metadata (for example singleton or twin, basic analysis or genome-wide) or manually edit the Sample Sheet in VeriSeq Workflow Manager.

Our goal is to load a rack of "random" samples like in ad hoc batching, but still have Clarity automatically generate the Sample Sheet using existing metadata. The adhoc_batching module enables this by automatically detecting the "random" samples loaded on the robot in Clarity, starting the workflow, and completing Step 1, allowing Clarity to generate the Sample Sheet automatically.

lot_instrument_tracing

We aim to automate the registration of reagents and kits used during lab preparation. This module parses the reagent report and records the information in Clarity.

duplicate_handling

Importing the same sample multiple times (in different projects) crashes the VeriSeq NIPT Integration for Clarity. However, the lab prefers to import all samples that haven't yet finished pipeline into Clarity without checking if a sample has already been registered in Clarity.

Moreover, if a sample failed the pipeline first time, the lab would include it in a new project for better history tracking. This also causes duplicates in Clarity.

Therefore, it's crucial to handle duplicates within Clarity to prevent the VeriSeq NIPT Integration from crashing.

fcs_report

To register sequencing reagent kits and instruments (for example, NextSeq, PCR machine) used in each NIPT run, an additional manual step ("Step 2") has been added to the official NIPT workflow by Illumina. This step must be completed before samples can proceed to Step 3 of the NIPT workflow. The VeriSeq NIPT Integration service monitors for new batch initiation reports. If a report is found, the service checks if the samples have arrived at Step 3. If they haven't arrived at Step 3 within 5 attempts, the workflow is aborted. This creates issues because the lab often can't complete "Step 2" in time. This module addresses the issue by ensuring the batch initiation report is only sent to the VeriSeq NIPT Integration after "Step 2" is completed.

swl_integration

This module processes the raw NIPT reports generated by VeriSeq and create interpreted reports ready to be imported into SwissLab. It interprets the raw nipt report based on predefined rules. The interpreted NIPT report is then transferred to the OUS network to be processed by the SwissLab NIPT automation service which import data into SwissLab and automatically completes the process if no findings. Otherwise, the sample is left for lab doctors to review manually.

genologics

Helper module for using Clarity API.

genologics_sql

Helper module for accessing Clarity database.

Common problems and solutions

Case 1 Sample sheet isn't generated

How to find out case 1

Lab informs bioinformatician inns when Sample Sheet for Pooling isn't generated automatically.

Steps to solve case 1

Log into ous-lims sever ( ssh username@ous-lims.sequencing.uio.no) and check the "adhoc_batching" module log at /opt/gls/clarity/customextensions/nipt-clarity/logs/adhoc_batching/. Normally, the error message in the log should clearly describe the problem. The most common error message is: 98765432101 not in "Generate Batch & Sample Sheet" queue.

Scenario A - The error message in the log clearly indicates a non-existing ID

Check the "Generate Batch & Sample Sheet" queue of the "VeriSeq NIPT v1.0_EHD" protocol to ensure the erroneous sample isn't present. If it's missing, inform the lab to address the issue. There are two common reasons for this problem:

  1. The sample hasn't yet been imported into Clarity. In this situation, the lab should import the missing sample into Clarity and assign it to the NIPT workflow. The Sample Sheet will be generated shortly after the sample is imported into Clarity and assigned to the NIPT workflow.
  2. The sample exists in Clarity, but the barcode scanned by the Hamilton robot doesn't match the sample name in Clarity. Once the barcode is scanned, it can't be changed, so the sample ID must be updated in Clarity to allow the workflow to continue.
    • Lab gives you the project name and the scanned barcode (the wrong sample name) and the correct sample name in Clarity, for example, project Diag-NIPT300, 11111111 was scanned, and the corresponding sample name is 22222222 in Clarity. Since that the scanned barcode can't be changed, we need to continue with the wrong name by changing 22222222 to 11111111 in Clarity.
    • Perform the following actions between 05 and 55 minutes of any hour. This is for preventing the duplication deletion script -- which runs at every whole hour (00 minutes) -- from deleting the modified sample in case the modified sample name conflicts with an existing sample that has already gone through the workflow.
      • In Clarity, DASHBOARD -> Projects -> Search for the project -> Click NN In-progress samples -> Click the Derived Sample matching the correct sample name (for example 20000000) -> Click the Action 1 Sample button -> Click Rename container -> type in the scanned code (for example 10000000) -> Click Continue -> You see "Rename container running" message; wait until "Rename container complete" message is shown or the "Rename container running" message disappears.
      • On the same page, keep the same sample selected, click the Action 1 Sample again -> Click Rename single entity -> type in the scanned code (for example 10000000) again -> Click Continue -> wait until Rename single entity complete message is shown
      • Wait around 5 minutes and confirm that the Sample Sheet is generated in /mnt/nipt-lab/SampleSheets/, for example Diag-NIPT300-2024-01-03.txt.
      • Open the Sample Sheet, confirm that the scanned code (for example 10000000) is present and the correct name (for example 20000000) is not.
      • Inform Lab that the Sample Sheet is ready to use.
    • Wait until the sequencing and the NIPT analysis are finished and the interpreted NIPT report is generated (this takes around 30 hours).
    • Open the interpreted NIPT report, for example, /mnt/veriseq-server-output/Diag-NIPT300-2024-01-03/Diag-NIPT300-2024-01-03_.*_nipt_report_.*.tab, and confirm that the scanned code (for example 10000000) is present and the correct name (for example 20000000) is not.
    • Confirm that the NIPT workflow Step 3 "AUTOMATED - Data Analysis (VeriSeq NIPT v1.0)_EHD" has completed.
Scenario B - There are no clear error messages in the log

Check if the in-house cron jobs or the VeriSeq NIPT Integration service have stopped (the steps are described below).

Case 2 The in-house automation cron jobs stopped

How to find out case 2
  • There is a new batch initiation report in /mnt/veriseq-server-output/<project>/ProcessLogs folder
  • In the log of "adhoc_batching" module, the new batch initiation report isn't captured (the log stopped updating). Then the cron jobs may have stopped.
Steps to solve case 2
  1. ssh to ous-lims.sequencing.uio.no
  2. Run sudo -u glsai -s
  3. Run crontab /opt/gls/clarity/customextensions/nipt-clarity/nipt/exe/cron.job to restart all in-house cron jobs.

Case 3 The VeriSeq NIPT Integration service stopped

How to find out case 3
  • A new batch initiation report or a NIPT report has been copied over to /opt/gls/clarity/customextensions/nipt-clarity/fcs_report folder, but the third step of the "VeriSeq NIPT v1.0_EHD" protocol isn't started or completing.
  • Run sudo -u glsai -s
  • Run cd /opt/gls/clarity/extensions/veriseq-nipt/FileCaptureService
  • The VeriSeqNIPTLauncher.log file isn't updating.
Steps to solve case 3
  1. Go back to your own user(exit sudo glsai user).
  2. Run sudo /etc/init.d/veriseq-nipt_service stop
  3. Run sudo /etc/init.d/veriseq-nipt_service start to restart the service.

Remove NTC samples from workflow

When a project is stopped, all samples can be removed from workflow as normal, except for the NTC samples. NTC samples must finish the normal workflow as follow:

  1. Add all the NTC samples to be removed in the "AUTOMATED - Data Analysis (VeriSeq NIPT v1.0)_EHD" queue -> Click VIEW ICEBUCKET button -> Click BEGIN WORK button.
  2. On the Step Setup screen, click the "Upload file" button and upload the dummy_batch_initiation_report.tab in this repository.
  3. Keep Continue until Finish step.