Skip to content

HTS Bioinf - ELLA daily operations

Scope

ELLA: software for interpretation of genetic variants.

This procedure describes tasks related to managing ELLA in production:

  • Handling import and deletion of analyses
  • Managing gene panels
  • Managing users

Responsibility

Responsible person: A bioinformatician on production duty. ELLA core developers can also assist with the tasks described here, to offload work.

ELLA setup

Location of the application

The source code and other files related to ELLA are located at /ess/p22/data/durable/production/ella. The directories and scripts mentioned later in this document are relative to this unless given with absolute path.

Instances

ELLA runs on the TSD VM p22-podman-02. There are two instances of ELLA:

Instance URL Description
prod http://p22-podman-02:8080 Production. Should be up at all times.
staging http://p22-podman-03:8081 Used for debugging and testing new features and releases.

Processes of each instance

Each instance of ELLA contains several processes:

  • apiv1: API v1 for ELLA
  • analysis-watcher: Program for watching ella-<instance>/data/analyses/incoming for new analyses, and importing them
  • frontend: Serving bundled frontend code (app.js)
  • nginx: Proxy for redirecting requests to frontend or apiv1 containers - the only service with an exposed port
  • polling: Program for communicating with anno/anno-targets

Status of an instance can be monitored with task <instance>:status. Logs can be inspected in ella-<instance>/logs. Logs for backend, polling, and watcher are rotated daily.

ELLA's command line interface (CLI)

ELLA's CLI (ella-cli) can be accessed through task <instance>:cli. Run ella-cli --help to see available options. See also official documentation.

Production routines

Importing analyses to ELLA

New analyses are automatically imported into ELLA when delivered from:

  1. The bioinformatic pipeline.
  2. Reanalysis from anno/anno-targets (ordered through ELLA import).

New analyses are delivered to ella-<instance>/data/analyses/incoming and imported by ELLA's analysis-watcher.

White-/blacklists

Each instance of ELLA has two associated files that determine which analyses should be imported:

  • ops/<instance>-watcher-whitelist.txt
  • ops/<instance>-watcher-blacklist.txt

To allow an analysis / project to be imported, add a regex to the whitelist file. To block an analysis / project from being imported, add a regex to the blacklist file.

Troubleshooting missing analyses

The watcher checks for new analyses (and updates to the whitelist / blacklist files) every thirty seconds. Check the watcher logs (ella-<instance>/logs/watcher.log) to see if an import succeeds, or if not, why it does not succeed.

Common causes of analyses not being imported include:

  • Analysis already in database
  • Analysis already in ella-<instance>/data/analyses/imported
  • Analysis not matching whitelist or matching blacklist
  • Gene panel not in database

The presence of an analysis in imported but not in the database suggests either that the analysis has been deleted from the database or that the database has not been updated. The latter should never be the case for the prod-instance.

The presence of an analysis in imported but not in the UI suggests that the corresponding gene panel is not tied to the user group under which the user is currently signed in. See below for possible solutions to this.

Deleting analyses

Warning

Deleting an analysis will remove all work performed on that analysis. This action cannot be undone.

To delete an imported analysis:

  1. Search for and open the analysis in ELLA. Note down the analysis ID from the URL (ending with /workflows/analyses/{analysis ID}/). Double check that the ID is correct.

  2. Open ELLA's CLI and run:

    ella-cli delete analysis {analysis ID}

    Enter the reason after the prompt, e.g. "Requested by {username} on 190425 by email".

  3. When you are asked for confirmation, double check that the analysis is the correct one. Type "y" and Enter to confirm.

Adding new gene panels

To add new gene panels, import them into the database and add them to the relevant user groups.

I. Import gene panels into the database

Open ELLA's CLI and run for each gene panel:

ella-cli deposit genepanel [--replace] --folder {path to gene panel folder}

NOTE: use the --replace flag when updating already existing gene panels.

II. Add new gene panels to user groups

To update the relevant user groups:

  1. Enter the directory ella-<instance>/data/fixtures and edit the file create_usergroups.py. Add the gene panels to the relevant groups. Please add them in alphabetical order.

  2. Run:

    python create_usergroups.py > usergroups-new.json

  3. Perform a diff on the old one like this diff usergroups.json usergroups-new.json and make sure everything looks fine.

  4. Run:

    mv usergroups-new.json usergroups.json

  5. Go back to ella-cli in \<instance> and run

    ella-cli users add_groups /path/to/ella-<instance>data/fixtures/usergroups.json

  6. Confirm in ELLA that the gene panels are available for the corresponding user groups.

  7. Exit ELLA's CLI and commit the files to Git (provide reason in commit message):

    git add create_usergroups.py usergroups.json
git commit -m "<your-message>"

Changes to users

Reset password

If a user forgets his/her password:

  1. Reset the user's password using ELLA's CLI:

    ella-cli users reset_password <username>

  2. Send the new password in a personal email, including a note that the user needs to change the password before logging in again ("Change password" on ELLA login page).

Add or edit user

To add a new or edit an existing user (including changing the associated user group):

  1. Open ella-<instance>/data/fixtures/users.json and add or change the user with the corresponding user group. For OUS users, the OUS user name (same as in OUS emails) should be used.

  2. Commit the changes to Git (provide reason in commit message):

    git add users.json
git commit -m "your-message"

  3. Open ELLA's CLI and run:

    ella-cli users add_many /path/to/ella-<instance>/data/fixtures/users.json

    This should print out the changes, including newly created user(s) and their temporary passwords.

  4. Provide the user name and password to each new user in a personal email, along with instructions on how to proceed. The e-mail should contain something along the lines of:

    Du har blitt lagt til som [user group] bruker i ELLA. Velkommen!

Merk at innlogging i TSD via VMware (med brukernavn «p22-…» og passord fra TSD) må være i orden før du kan åpne ELLA. Se prosedyre «Bruk av TSD og ELLA» i eHåndbok (http://ehandbok.ous-hf.no/document/133297) for detaljer, vedlegget «TSD_brukerguide_...» har alt du trenger for å komme i gang.

Brukernavn i ELLA:
[user name]

Midlertidig passord:
[temp password]

NB: Passordet må endres før første innlogging, klikk «Change password» på innlogginssiden i ELLA.

Change your own user group (devs only)

Sometimes you may need to change your own user group to be able to test that the configuration is correct, investigate missing samples, find analysis IDs for deletion etc.

To change user group, you can use the following script:

task <instance>:change_usergroup -- {username} {usergroup}

Note

Changes using this method are not persisted when updating regular users and should only be used for developers/bioinformaticians. To change the user group for a regular user, see above.

Tip

The ELL user group has access to all analyses, and will work for most scenarios.

Deactivate user

Removing existing users is not possible by design, due to historical tracking of changes in the system. A user can, however, be locked in ELLA's CLI by running:

ella-cli users lock <username>

Change comment templates

Comment field templates in ELLA are defined in fixtures/usergroups.json. To edit, use the helper script fixtures/create_usergroups.py. In the script, templates have this stucture:

TEMPLATE_<SHORTNAME> = {
    "name": "<Name in dropdown, defined by users>",
    "comment_fields": ["<comment field name in ELLA>"],
    "template": "<html template>"
}

Each user group's specific set of templates are defined under <user group>_CONFIG_BASE/"comment_templates".

Notes

  • Keep html as simple as possible, in line with what's in other templates
  • Any " has to be escaped, e.g. <span style=\"color:red\">Important</span>
  • For text colors, pick one of the colors already in use (don't add additional)

Procedure

  1. Test in staging first!

  2. Edit fixtures/create_usergroups.py directly.

    For existing templates, edit corresponding under "template"

    To add new template, add new name TEMPLATE_<SHORTNAME> with config as well as a new entry in the <user group>_CONFIG_BASE/"comment_templates" list.

  3. Update json:

    python3 ./create_usergroups.py > usergroups.json

  4. Add and commmit changes to both files (check with git diff --cached before commit)

  5. Update user groups in ELLA:

    ella-cli users add_groups /ess/p22/data/durable/production/ella/ella-prod/data/fixtures/usergroups.json

  6. Remember to sync create_usergroups.py and usergroups.json in prod and staging

Other relevant documents