HTS Bioinf - ELLA daily operations
Scope
ELLA: software for interpretation of genetic variants.
This procedure describes tasks related to managing ELLA in production:
- Handling import and deletion of analyses
- Managing gene panels
- Managing users
Responsibility
Responsible person: A bioinformatician on production duty. ELLA core developers can also assist with the tasks described here, to offload work.
ELLA setup
Location of the application
The source code and other files related to ELLA are located at /ess/p22/data/durable/production/ella
. The directories and scripts mentioned later in this document are relative to this unless given with absolute path.
Instances
ELLA runs on the TSD VM p22-podman-02
. There are two instances of ELLA:
Instance | URL | Description |
---|---|---|
prod | http://p22-podman-02:8080 | Production. Should be up at all times. |
staging | http://p22-podman-03:8081 | Used for debugging and testing new features and releases. |
Processes of each instance
Each instance of ELLA contains several processes:
- apiv1: API v1 for ELLA
- analysis-watcher: Program for watching
ella-<instance>/data/analyses/incoming
for new analyses, and importing them - frontend: Serving bundled frontend code (app.js)
- nginx: Proxy for redirecting requests to frontend or apiv1 containers - the only service with an exposed port
- polling: Program for communicating with
anno/anno-targets
Status of an instance can be monitored with task <instance>:status
. Logs can be inspected in ella-<instance>/logs
. Logs for backend, polling, and watcher are rotated daily.
ELLA's command line interface (CLI)
ELLA's CLI (ella-cli
) can be accessed through task <instance>:cli
. Run ella-cli --help
to see available options. See also official documentation.
Production routines
Importing analyses to ELLA
New analyses are automatically imported into ELLA when delivered from:
- The bioinformatic pipeline.
- Reanalysis from
anno/anno-targets
(ordered through ELLA import).
New analyses are delivered to ella-<instance>/data/analyses/incoming
and imported by ELLA's analysis-watcher
.
White-/blacklists
Each instance of ELLA has two associated files that determine which analyses should be imported:
ops/<instance>-watcher-whitelist.txt
ops/<instance>-watcher-blacklist.txt
To allow an analysis / project to be imported, add a regex to the whitelist file. To block an analysis / project from being imported, add a regex to the blacklist file.
Troubleshooting missing analyses
The watcher checks for new analyses (and updates to the whitelist / blacklist files) every thirty seconds. Check the watcher logs (ella-<instance>/logs/watcher.log
) to see if an import succeeds, or if not, why it does not succeed.
Common causes of analyses not being imported include:
- Analysis already in database
- Analysis already in
ella-<instance>/data/analyses/imported
- Analysis not matching whitelist or matching blacklist
- Gene panel not in database
The presence of an analysis in imported
but not in the database suggests either that the analysis has been deleted from the database or that the database has not been updated. The latter should never be the case for the prod-instance.
The presence of an analysis in imported
but not in the UI suggests that the corresponding gene panel is not tied to the user group under which the user is currently signed in. See below for possible solutions to this.
Deleting analyses
Warning
Deleting an analysis will remove all work performed on that analysis. This action cannot be undone.
To delete an imported analysis:
-
Search for and open the analysis in ELLA. Note down the analysis ID from the URL (ending with
/workflows/analyses/{analysis ID}/
). Double check that the ID is correct. -
Open ELLA's CLI and run:
Enter the reason after the prompt, e.g. "Requested by {username} on 190425 by email".
-
When you are asked for confirmation, double check that the analysis is the correct one. Type "y" and Enter to confirm.
Adding new gene panels
To add new gene panels, import them into the database and add them to the relevant user groups.
I. Import gene panels into the database
Open ELLA's CLI and run for each gene panel:
NOTE: use the
--replace
flag when updating already existing gene panels.
II. Add new gene panels to user groups
To update the relevant user groups:
-
Enter the directory
ella-<instance>/data/fixtures
and edit the filecreate_usergroups.py
. Add the gene panels to the relevant groups. Please add them in alphabetical order. -
Run:
-
Perform a diff on the old one like this
diff usergroups.json usergroups-new.json
and make sure everything looks fine. -
Run:
-
Go back to
ella-cli
in \<instance> and run -
Confirm in ELLA that the gene panels are available for the corresponding user groups.
-
Commit the files to Git (provide reason in commit message):
Changes to users
Reset password
If a user forgets his/her password:
-
Reset the user's password using ELLA's CLI:
-
Send the new password in a personal email, including a note that the user needs to change the password before logging in again ("Change password" on ELLA login page).
Add or edit user
To add a new or edit an existing user (including changing the associated user group):
-
Open
ella-<instance>/data/fixtures/users.json
and add or change the user with the corresponding user group. For OUS users, the OUS user name (same as in OUS emails) should be used. -
Commit the changes to Git (provide reason in commit message):
-
Open ELLA's CLI and run:
This should print out the changes, including newly created user(s) and their temporary passwords.
-
Provide the user name and password to each new user in a personal email, along with instructions on how to proceed. The e-mail should contain something along the lines of:
Du har blitt lagt til som [user group] bruker i ELLA. Velkommen! Merk at innlogging i TSD via VMware (med brukernavn «p22-…» og passord fra TSD) må være i orden før du kan åpne ELLA. Se prosedyre «Bruk av TSD og ELLA» i eHåndbok (http://ehandbok.ous-hf.no/document/133297) for detaljer, vedlegget «TSD_brukerguide_...» har alt du trenger for å komme i gang. Brukernavn i ELLA: [user name] Midlertidig passord: [temp password] NB: Passordet må endres før første innlogging, klikk «Change password» på innlogginssiden i ELLA.
Change your own user group (devs only)
Sometimes you may need to change your own user group to be able to test that the configuration is correct, investigate missing samples, find analysis IDs for deletion etc.
To change user group, you can use the following script:
Note
Changes using this method are not persisted when updating regular users and should only be used for developers/bioinformaticians. To change the user group for a regular user, see above.
Tip
The ELL user group has access to all analyses, and will work for most scenarios.
Deactivate user
Removing existing users is not possible by design, due to historical tracking of changes in the system. A user can, however, be locked in ELLA's CLI by running: