HTS Bioinf - ELLA production
Scope
ELLA: software for interpretation of genetic variants.
This procedure describes tasks related to managing ELLA in production:
- Handling import and deletion of analyses
- Managing gene panels
- Managing users
Responsibility
Responsible person: A bioinformatician on production duty. ELLA core developers can also assist with the tasks described here, to offload work.
ELLA setup
Location of the application
The source code and other files related to ELLA are located at /ess/p22/data/durable/production/ella
. The directories and scripts mentioned later in this document are relative to this unless given with absolute path.
Instances
ELLA runs on the TSD VM p22-ella-01
. There are four instances of ELLA:
Instance | URL | Supervisor URL | Description |
---|---|---|---|
prod | http://p22-ella-01:5000 | http://p22-ella-01:9007 | Production. Should be up at all times. |
staging | http://p22-ella-01:7000 | http://p22-ella-01:9008 | Used to test new features and releases. |
test | http://p22-ella-01:7001 | http://p22-ella-01:9009 | Used for various (user) testing purposes.* |
validation | http://p22-ella-01:7002 | http://p22-ella-01:9010 | Used to validate new analysis methods. |
* User testing purposes could be training of new users, testing new filters, new features etc. |
Supervisors
Each instance of ELLA is controlled by a Supervisor process accessible at http://p22-ella-01:9000 (superdupervisor) running locally on the VM. Each instance corresponds to a Singularity container that can be started/stopped from the superdupervisor. Inside each Singularity container instance is another Supervisor process controlling the programs ELLA is running. These programs are:
- api: The ELLA application
- polling: Program for communicating with
anno/anno-targets
- watcher: Program for watching
ella-<instance>/data/analyses/incoming
for new analyses, and importing them.
ELLA's command line interface (CLI)
ELLA's CLI (ella-cli
) can be accessed through ./ops/<instance>-cli.sh
. Run ella-cli --help
to see available options. See also official documentation.
Production routines
Importing analyses to ELLA
New analyses are automatically imported into ELLA when delivered from:
- The bioinformatic pipeline.
- Reanalysis from
anno/anno-targets
(ordered through ELLA import).
New analyses are delivered to ella-<instance>/data/analyses/incoming
. The import is done by the ELLA program analysis-watcher
.
White-/blacklists
Each instance has two associated files that determine which analyses should be imported:
ops/<instance>-watcher-whitelist.txt
ops/<instance>-watcher-blacklist.txt
To allow an analysis / project to be imported, add a regex to the whitelist file. To block an analysis / project from being imported, add a regex to the blacklist file.
Troubleshooting missing analyses
The watcher checks for new analyses (and updates to the whitelist / blacklist files) every thirty seconds. Check the watcher logs from the Supervisor UI to see if an import succeeds, or if not, why it does not succeed.
Common causes of analyses not being imported include:
- Not matching whitelist or matching blacklist
- Gene panel not in database
- Analysis already in database
- Analysis already in
ella-<instance>/data/analyses/imported
The presence of an analysis in imported
but not in the database suggests either that the analysis has been deleted from the database or that the database has not been updated. The latter should never be the case for the prod-instance.
The presence of an analysis in imported
but not in the UI suggests that the corresponding gene panel is not tied to the user group under which the user is currently signed in. See below for possible solutions to this.
Deleting analyses
Warning
Deleting an analysis will remove all work performed on that analysis. This action cannot be undone. Should the need arise to access it, the application logs contain information on the work performed on all analyses, including deleted ones.
To delete an imported analysis:
-
Search for and open the analysis in ELLA. Note down the analysis ID from the URL (ending with
/workflows/analyses/{analysis ID}/
). Double check that the ID is correct. -
Open ELLA's CLI and run:
Enter the reason after the prompt, e.g. "Requested by {username} on 190425 by email".
-
When you are asked for confirmation, double check that the analysis name is correct. Type "y" and Enter to confirm.
Adding new gene panels
To add new gene panels, they must be first imported into the database and then added to the relevant user groups.
I. Import gene panels into the database
-
Copy the relevant files into
ella-<instance>/data/fixtures/genepanels
-
Add them to the local Git repository (there is one in the
fixtures
directory). -
Open ELLA's CLI and run for each gene panel:
Warning
Make sure there is no
/
at the end of the gene panel folder name.
II. Add new gene panels to user groups
After you have added all the gene panels you want, you need to update the relevant user groups:
-
Enter the directory
ella-<instance>/data/fixtures
and edit the filecreate_usergroups.py
. Add the gene panels to the relevant groups. Please add them in alphabetical order. -
Run:
-
Perform a diff on the old one like this
diff usergroups-new.json usergroups.json
and make sure everything looks fine. -
Run:
-
Commit the files to Git (provide reason in commit message):
-
Go back to
ella-cli
in \<instance> and run -
Confirm in ELLA that the gene panels are available for the corresponding user groups.
Changes to users
Reset password
If a user forgets his/her password:
-
Reset the user's password using ELLA's CLI:
-
Send the new password in a personal email, including a note that the user needs to change the password before logging in again ("Change password" on ELLA login page).
Add or edit user
To add a new or edit an existing user (including changing the associated user group):
-
Open
ella-<instance>/data/fixtures/users.json
and add or change the user with the corresponding user group. For OUS users, the OUS user name (same as in OUS emails) should be used. -
Commit the changes to Git (provide reason in commit message):
-
Open ELLA's CLI and run:
This should print out the changes, including newly created user(s) and their temporary passwords.
-
Provide the user name and password to each new user in a personal email, along with instructyions on how to proceed. The e-mail should contain something along the lines of:
Har lagt deg til som bruker i ELLA, brukergruppe [user group]. Velkommen! Merk at innlogging i TSD via VMware (med brukernavn «p22-…» og passord fra TSD) må være i orden før du kan åpne ELLA. Se prosedyre «Bruk av TSD og ELLA» i eHåndbok (http://ehandbok.ous-hf.no/document/133297) for detaljer, vedlegget «TSD_brukerguide_...» har alt du trenger for å komme i gang. Brukernavn i ELLA: [user name] Midlertidig passord: [temp password] NB: Passordet må endres før første innlogging, klikk «Change password» på innlogginssiden i ELLA.
Change your own user group (devs only)
Sometimes you may need to change your own user group to be able to test that the configuration is correct or investigate missing samples, find analysis IDs for deletion etc.
To change user group, you can use the following script:
Note
Changes using this method are not persisted when updating regular users and should only be used for developers. To change the user group for a regular user, see above.
Tip
The ELL user group has access to all analyses, and will work for most scenarios.
Deactivate user
Removing existing users is not possible, due to historical tracking of changes in the system. A user can, however, be deactivated in ELLA's CLI by running: