The stp command line tool

stp allows you to log in and interact with multiple ST4SD instances, simplifying many of the day-to-day tasks that virtual experiment developers perform. This includes downloading, pushing, importing, creating, updating, and syntax checking PVEPs.

To interact with ST4SD instances using stp, we need to log in first. The simplest method is to use the code snippet provided by the Registry UI. If you’re already logged into your OpenShift cluster, you can copy and paste the following command into your terminal to open the ST4SD Registry UI automatically:

Here you will:

The ST4SD Registry allows users to easily access and explore Parameterised Virtual Experiment Packages (PVEPs) through a user-friendly web interface. Through the interface, users can browse available experiments, view their details, logs, and the results of their runs. Additionally, users can download experiments by scrolling to the bottom of the page and clicking the “Download JSON” button. There may be instances, however, where they want to perform these actions programmatically. With stp this is trivial: let us imagine we want to download the band-gap-dft-gamess-us experiment that is available on our public registry.

We can simply copy the link and run:

For the experiment JSON to be downloaded to our currently active directory.

We need to add experiments to our Registry instance before we can view or run them. To do this, we can either create a local PVEP definition from scratch or download it from another instance (e.g., by using the stp package download command). Once we have the definition, we can use stp to push it to our registry.

Assuming our local file is called nanopore-geometry-experiment.json and is in our current directory, we can run:

And stp will add it to our Registry instance.

In the previous sections, we learned how to download PVEPs and add them to our Registry instance using the download and push commands. These commands are versatile and can be used in various scenarios. However, if our goal is simply to download a PVEP from an external instance and add it to our own, stp offers another command, import, that can accomplish this task efficiently.

Copy the link to the experiment you want to import and run:

To automatically add it to your Registry instance.

Parameterised Virtual Experiment Packages (PVEPs) are parameterisations of our experiments that need to be created before they can be added the Registry to be run or shared. stp can help getting started with this by generating a valid, high-level PVEP that the user can later add details to.

As we need a virtual experiment to test this feature, we will use the nanopore-geometry-experiment experiment as our starting point. Let us start by cloning it and cd-ing into its directory:

To create the PVEP we then need to run:

The resulting PVEP can be pushed to the Registry using the stp package push command.

During the development phase of our virtual experiments, we will regularly commit changes to the git repository.

If we are following the set of best practices we have defined, the commit ID will be included in our PVEP, ensuring ST4SD executes the exact version of our experiment, improving results reproducibility. Provided is an example of how this might look like in your PVEP:

Additionally, we might want to give a specific experiment version an easy-to-remember name by means of a tag. stp makes updating the commit ID and metadata tags of our PVEP as painless as possible by providing the update-definition command. In the nanopore-geometry-experiment folder from the previous example we can run:

The PVEP definition may now look similar to the following:

When we manually edit our PVEPs, we risk making mistakes. Fortunately, stp includes a syntax checker that can help us identify and fix our errors. Let’s intentionally introduce a mistake into our nanopore-geometry-experiment PVEP, which we obtained earlier in this tutorial, by adding a nonexistent field to the metadata section.

Here’s how the updated section now looks:

stp will point us to the error when we run stp package test, as such:

With stp it is possible to log in to multiple ST4SD instances - which we refer to as contexts - and quickly switch between them.

To see what the currently active context - meaning the ST4SD instance that stp will run commands against - is, you can use the stp context show command. By default, it will only print the name of the context, but an option is provided to also print out the URL of the ST4SD instance for the context.

To see what instances you have logged in to, you can use the stp context list command. By default, the results will be printed as a table but this can be disabled with the --simple flag.

In case you need your commands to target a different stp context from the one currently active (as shown by stp context show), you can use the stp context activate command.

If you created a context without specifying the --context-name flag, you might have ended up with a context whose name is the full URL of the instance. You can change this with stp context rename.

If you no longer need one of the existing contexts, you can remove it with stp context delete.

stp offers some additional commands for more advanced useDebugValue, such as interacting with stack-wide settings.

Not all images are available publicly, some may be private and require authentication. When importing or pushing a package with stp package import or stp package push, stp will check for you if an image pull secret is available for the registries referenced by your PVEP and emit a warning if that isn’t the case. To see what image pull secrets are available to use in ST4SD, you can use the stp stack pull-secrets list command.

If you are working with a private image registry, you will need to ensure ST4SD has credentials to pull the images. To do so, you can use the stp stack pull-secrets add command: set a name for the secret with the --name flag, pass one or more registries URLs with the --registry flag, and the username with --username. For the password/access token you have two options: either passing it directly via the -t flag, or by writing it in a file and using the --token-file flag (this will prevent leaking the credential in your shell history).