The National Institute for Computational Sciences

Open OnDemand User Guide

  Open OnDemand


OpenOnDemand is a web-based portal that provides access to the cluster’s HPC resources. You can manage files, jobs, and access graphical interfaces all through OpenOnDemand. This document describes how to access and use OpenOnDemand and its related services.

Access and Login

To access Open OnDemand, open a web browser and type in the address bar. This will bring up the login page for Open OnDemand. Use your UT NetID and password to login. Figure 2.1 shows the login page for OpenOnDemand.

OpenOnDemand login
Figure 2.1 - OpenOnDemand Login Screen

After you successfully login, Duo TFA will come up. Select Duo Push or Passcode to log in. The authentication method will be sent to your mobile device. Figure 2.2 shows the Duo TFA options that will appear.

Duo authentication options
Figure 2.2 - Duo Authentication Options

Upon successful authentication, you will see the Open OnDemand dashboard. The top-left of the dashboard contains several menus that grant access to files, jobs, and your current interactive sessions.

File Explorer

The “Files” menu in the top-left of the OpenOnDemand dashboard gives you access to your files on the cluster from within a browser. Initially, the File Explorer shows your home directory. Figure 3.1 shows the File Explorer interface.

File Explorer
Figure 3.1 - File Explorer Interface

File Explorer Navigation

To navigate through directories, double-click on the directory you wish to enter. To return to your former directory, double-click on the two dots. If you are unsure where you are located within the directory tree, look above the directory listing pane. You will see several names and letters separated by forward slashes (/). This indicates the absolute pathname of your current location. In Figure 3.1, the current directory of the user is his home directory located at /nics/b/home/.

In the left pane of the File Explorer, you see the structure of your home directory. If at any time you wish to return to the top-level of your home directory, click on “Home Directory.” You may also navigate to directories within your home directory using this pane.

To quickly navigate to a specific directory, select the option labeled “Go To…” It has a dossier icon to the left of its label. When it opens, type the absolute pathname of the directory to which you wish to go. For example, to navigate to your Lustre directory, type /lustre/haven/user/<username> where <username> is your NICS username. The file explorer will then place you in the specified directory and display its contents. Figure 3.2 highlights the Go To option and the current directory location.

File Explorer Navigation
Figure 3.2 - Current Location and Go To Locations in File Explorer

File Explorer Views

OpenOnDemand’s file explorer can show permissions, ownership, and hidden files. In the top-right of the file explorer interface, check the boxes for “Show Dotfiles” and “Show Owner/Mode” to enable these views. Figure 3.2 shows the directory listing pane with the Show Owner/Mode option enabled. Permissions and ownership cannot be modified from within this interface. Hidden files are prefaced with a dot (.) and are usually user configuration files.

To view a specific file, select the file in the directory listing pane. Click on the option labelled “View.” It has the icon of an eye to the left of the label. The File Explorer will show the plaintext contents of the file.

Creating, Modifying, and Deleting Files

To create a new file in the File Explorer, select the option labelled “New File.” Its icon is a sheet of paper with a plus (+) symbol in the bottom-right. When you click on the option, the File Explorer will prompt you to name the file. After you name the file and confirm its creation, it will be placed within your current directory.

To modify the file, select the file in the directory listing pane, then select the option labelled “Edit.” Its icon is a canvas with a paintbrush on top of it. The OpenOnDemand editor will open. You can then type the information you wish to put in the file. The editor features several customization options, such as the editor’s theme and text mode. Figure 3.3 shows the OpenOnDemand file editor. When you finish editing the file, click “Save” in the top-left of the editor. Figure 3.3 highlights the Save option. When you return to the File Explorer and view the file, you can view the changes you made.

File Editor
Figure 3.3 - OpenOnDemand File Editor

The File Explorer also allows you to rename existing files and directories. To rename a file or directory, select the item you wish to rename in the directory listing pane. Click the option labelled “Rename.” The File Explorer will prompt you for the new name, and once you confirm it, your changes will take immediate effect.

Files and directories can be copied and pasted within the File Explorer. To copy a file or directory, select the item in the directory listing pane, then select the option labelled “Copy.” Once the item has been copied to your clipboard, navigate to the location to which you intend to copy the file, then select the option labelled “Paste.” The copied item will then appear in your current directory.

If you wish to delete a file or directory, select the item in the directory listing pane, then select the “Delete” option on the right-side of the File Explorer. You will be prompted to confirm the deletion. Once confirmed, the item will be permanently deleted.

Downloading and Uploading Files

Files can be both downloaded from and uploaded to the cluster through the File Explorer. Only small files should be transferred between the cluster and your local system through the File Explorer. For larger file transfers, please use Globus. More information on how to configure and use Globus can be found in the Data Transfer document.

To download a file from the cluster to your local system, select the file in the directory listing pane. If you wish to download multiple files, hold down the Ctrl key on Windows and Linux systems or the Command key on MacOS, then select each file in the directory listing pane. The background of the files will become purple to indicate they are selected. Figure 3.4 shows how the files will appear when they are selected.

Multiple File Selection
Figure 3.4 - Multiple Files Selected in File Explorer

Once all the files you wish to download are selected, click the option labelled “Download.” It is indicated with a down arrow. After the download completes, the file(s) will appear in your “Downloads” folder unless you have specified another location for downloads. Be aware that most browsers block attempts to download multiple files unless you explicitly allow the transfer.

To upload a file to the cluster from your local system, select the “Upload” option in the top-right of the File Explorer interface. The Explorer will prompt you to choose a file. Select “Choose Files” when the dialog box appears. You may then select the file(s) you want to upload to the cluster from your local system. In addition to this method, you may also drag and drop files from your local system into the File Explorer. These items will then appear in the directory listing pane.

Job Management

In the “Jobs” menu, you have two options: Active Jobs and Job Composer. Both options provide advanced job management from within a web browser.

Job Monitoring

In the Active Jobs section, you will see an organized list of your queued and running batch jobs on the cluster. Figure 4.1 shows the Active Jobs interface. You can filter through jobs to see particular entries, in addition to sorting by the attributes of your choice. To do this, click on the arrows next to the attributes until the list is sorted as you desire.

Active Jobs Interface
Figure 4.1 - Active Jobs Interface

When a job is queued or running on the cluster, you can view detailed information on the job from within the Active Jobs interface. Figure 4.2 shows the information you will see on queued and running jobs.

To access this view on a queued or running job, click the right-facing arrow on the far-left of the job’s row. You will then see statistics and information about the selected job. Of note is the “Output Location” of the job, which Figure 4.2 highlights. Each job you create and start through OpenOnDemand is placed within your /home/ondemand directory, so when you wish to retrieve the job’s data, remember to navigate to your ondemand directory. You can specify another output location when you create the job through OpenOnDemand or edit the job script to supply another location.

Detailed Job View
Figure 4.2 - Detailed View of a Running Job

Job Creation

In the Job Composer section, you can create new batch jobs to run on the cluster based upon several criteria. Figure 4.3 shows the Job Composer interface. When you initially open the Job Composer, the system will briefly explain each option available to you. Once you click through these prompts, you may then click “New Job,” then select the template, path, or job you wish to add. Note that if you close the initial information menu, you will not need to click through each prompt.

Job Composer
Figure 4.3 - Job Composer Interface

When you add a new job from the default template, the Job Composer will create a Simple Sequential Job. In addition to the job, it will create a job script and a job directory. Figure 4.4 shows the Job Composer interface when a job is created with the default template. To modify the job’s options, select Job Options. You can specify the job’s name, cluster, script, and account. To edit the job script, highlight the job and select the Edit Files option in the Job Composer interface. Select the job script, then select Edit. You may also rename the job script if necessary. Additional files can also be uploaded or created in the job’s directory from the File Explorer.

Default Job Template
Figure 4.4 - Selecting the Default Template for a New Job

Jobs may also be created with other templates. These can be the default templates or templates that you build within the Job Composer. Figure 3.5 shows the Templates menu.

Job Template Creation
Figure 4.5 - Templates Menu within the Job Composer

In addition to template-based job creation, you may also create jobs based upon a specified path or an existing job. With a specified path, you create a job based upon a directory that already exists. The path will either be to your home or project directory. Once you specify the path, you then modify the typical attributes of the job. With a job created based upon an existing one, select an existing job in the queue, navigate to New Job, then select “From Selected Job.”

Beyond job creation, the Job Composer allows you to submit, stop, and delete your existing jobs. Like the Active Jobs menu, you can filter and sort the available jobs.


noVNC allows users to interact with the cluster in a desktop environment from a web browser. In situations where you require a graphical user interface (GUI), you should choose this option for your interactive session. Be aware that while most browsers support noVNC, outdated or uncommon browsers may not function. Please try to use Chrome, Firefox, Edge, or Safari when running noVNC interactive sessions on the cluster.

To launch a noVNC interactive session, login to OpenOnDemand. In the dashboard, select “Interactive Apps” from the top-left of the window. In the dropdown menu, select “ACF Desktop.” That will open the session submission form depicted in Figure 5.1.

noVNC Session Creation
Figure 5.1 - ACF Desktop Session Submission Form

Like an interactive batch job, you specify an account to which the session should be charged, the amount of time you require the session, and the number of nodes the session should use. The Feature field is equivalent to the -l feature option in PBS, which allows you to use a special feature of a given node. The Queue field, equivalent to the -q option in PBS, specifies a particular QoS (Quality of Service) attribute. Please refer to the Running Jobs document for specific features and QoS attributes you can apply to your interactive sessions.

At the bottom of the form, you can check the box to receive an email when your session starts. It should not take the system long to provide an interactive session, though this depends on the options you specified in the submission form.

Be aware that only the number of hours and number of nodes must be specified for the job to run. The Account, Feature, and Queue fields are optional. When they are left blank, the system will use its default settings.

Once you specify the relevant information for your interactive session, select “Launch.” The session will be submitted to the system, and when it is ready, you can connect and receive a noVNC desktop environment by selecting the “Launch noVNC in New Tab” option. You can also share a View-Only link of the session with others if they need to watch your activity. Figure 5.2 shows what will appear when the noVNC interactive session is ready and highlights the “Launch” option.

Connect to noVNC
Figure 5.2 - Active noVNC Interactive Session

When you launch the session, a desktop environment will open in a new browser tab. Remember that you only have the session for the duration you specified in the “Number of hours” field when you created it. After your time expires, the session will be terminated. You may also delete the session manually if you no longer need it. If you manually terminate the session, you will receive email confirmation of the job’s deletion.

Jupyter Notebook

Launching Jupyter Notebook

Jupyter Notebook is a web-based application that enables you to intuitively create and manipulate data on the cluster. It features a fully-functional Python 2 and Python 3 IDE (integrated development environment), a text editor, and a file manager. The sections below outline how to access and set up Jupyter Notebooks on the cluster.

To access Jupyter Notebook, navigate to “Interactive Apps” on the OpenOnDemand dashboard. Select “Jupyter Notebook” from the dropdown menu. The Jupyter Notebook session submission form will appear as it does in Figures 6.1 and 6.2.

Jupyter Session Creation (1)
Figure 6.1 - Jupyter Notebook Session Submission Form (Part 1)

Jupyter Session Creation (2)
Figure 6.2 - Jupyter Notebook Session Submission Form (Part 2)

In this form, specify the Python version to use with Jupyter Notebook, the account under which the job should run, the QoS that applies to the job, any special features or partitions, the number of hours the job will run, and the number of cores your job requires. In most cases, all you should specify is the Python version, account, number of hours, and number of cores for the job. If you require the use of GPUs in your Jupyter Notebook session, select the "Beacon" partition and the "Beacon GPU" feature. Please review the Running Jobs document for more information on these fields.

When you specify the necessary options in the form, select “Launch.” It will not take the system long to deploy the Jupyter Notebook session, but this is dependent on the resources you requested. When the session is ready, you will see the same interface that appears in Figure 6.3. The “Connect to” option will open a new Jupyter Notebook tab in your browser. Figure 6.2 highlights this option.

Connect to Jupyter
Figure 6.3 - Jupyter Notebook Ready to Launch

Working with Jupyter Notebook

The initial Jupyter interface appears as it does in Figure 6.4. Like the OpenOnDemand File Explorer, you can manipulate your files and directories from within the Jupyter Notebook session. Files and directories can be created, edited, deleted, or moved. In addition, you can upload files to the cluster from your local system through the Jupyter Notebook session.

Jupyter File Explorer
Figure 6.4 - Jupyter File Explorer

Under the “New” menu on the right-side of the interface, the dropdown menu depicted in Figure 6.5 appears. From here, you can create a Python 2 Notebook session, a text file, a folder (directory), or a terminal.

Jupyter New Menu
Figure 6.5 - New Item Menu

The first option, the Python 2 Notebook session, creates a Python 2 IDE in which you can write and execute Python 2 code interactively. Figure 6.6 shows what this session looks like. There are multiple options available to use, though they are beyond the scope of this document. Please consult the official Jupyter Notebook documentation for detailed information on all the options available to you within this IDE. You can rename the session in the upper-left of the interface by selecting the default name, Untitled, and then entering the name you wish to assign. In Figure 6.6, the user assigned the name “Py2-Test.”

Jupyter Python IDE
Figure 6.6 - Jupyter Notebook Python 2 IDE

As you work within the IDE, Jupyter saves your work in a .ipynb file within your home directory. Its name will be the same as the name you assign within the IDE. You can view the file from the “Files” section within Jupyter. Additionally, if the session is running, the icon for the file will be green and will state that it is running next to the file’s timestamp.

The second option, text file, opens the Jupyter text editor. Figure 6.7 shows the initial text editor interface. Within this editor you can write code for multiple languages or simply use plaintext. Several different programming languages are supported by the editor. To see them all, click on the “Language” menu as shown in Figure 6.8. When you select a language, the editor will perform syntax highlighting and other functions to assist you as you code.

Jupyter Editor
Figure 6.7 - Jupyter Text Editor

Jupyter Editor Languages
Figure 6.8 - Jupyter Text Editor Languages

Once you finish working in the text editor, select File, then Save. Be sure to rename the file by clicking on the default title, untitled.txt, and changing it to a more descriptive name. The saved file will appear in your home directory by whatever name you selected.

The third option, folder, creates a new directory in your current one. The fourth option, terminal, opens a ssh session in another browser tab. Select this option to perform certain tasks that the graphical interface does not allow, such as long listings of a directory or file searches.

To see the running Jupyter processes, return to the Jupyter Files menu by clicking on the Jupyter icon in the top-left of the interface. From there, select the “Running” menu. If you have any terminals or notebooks active, they will appear here. You can shut these processes down if you no longer need them or have completed your work. Figure 6.9 shows an example of the Running menu.

Jupyter's Running Menu
Figure 6.9 - Jupyter's Running Menu

At the time of this writing, the “Clusters” menu seen in Figure 6.9 next to the Running menu has no functionality.

When you finish your work within Jupyter, select the “Logout” option in the top-right of the window. Close the Jupyter Notebook tab, then return to your interactive sessions in OpenOnDemand. If you have time remaining on your job, please delete it. Do note that if you exceed your allotted time in the session, it will be terminated. Also be aware that when you manually delete the job for a Jupyter Notebook session, you will receive an email that confirms its deletion.

Using Anaconda Environments in Jupyter Notebook

When an Anaconda environment is loaded into a Jupyter Notebook session, all the packages maintained in that environment are available to the notebook. Minor modifications can enable this functionality. If you do not have an Anaconda environment and wish to create one, please consult our Anaconda User Guide. This section assumes you already have a working Anaconda environment that you wish to use with Jupyter Notebook.

To load an Anaconda enviroment into a Jupyter Notebook session, follow these steps.

  1. If it is not already loaded, load the relevant Anaconda module. If your packages use Python 2, load Anaconda 2; if they use Python 3, load Anaconda 3.
  2. Activate your Anaconda environment.
  3. Install the ipykernel package in the Anaconda environment. Figure 6.10 shows the command to use for the installation. Replace <env-name> with the name of your Anaconda environment.
  4. conda install ipykernel
    Figure 6.10 - Installing the ipykernel Package into an Anaconda Environment

  5. Verify that ipykernel exists in the environment by executing the conda list command.
  6. Execute the command presented in Figure 6.11 to add the Anaconda environment to Jupyter. If you are using Python 2, replace python3 with pythonReplace <env-name> with the name of your Anaconda environment.
  7. python3 -m ipykernel install --user --name=<env-name>
    Figure 6.11 - Adding an Anaconda Environment to Jupyter Notebook

  8. Launch OpenOnDemand. Navigate to "Interactive Apps," then select "Jupyter Notebook."
  9. Specify the correct job parameters. Ensure that the correct Python version is selected for your environment. Submit the job to the scheduler.
  10. Launch the job once it is ready. In the top-right corner of Jupyter Notebook, select the "New" dropdown menu. Your Anaconda environment should appear. Figure 6.12 shows where to look for the Anaconda environment. In the figure, the Anaconda environment available to Jupyter Notebook is named "fithic-env."
  11. Figure 6.12 - Locating Anaconda Environments in Jupyter Notebook

  12. Select the environment. You will be placed in a Jupyter Notebook session with your Anaconda environment available.

Two additional commands are helpful when using Anaconda environments with Jupyter Notebook. The first command outputs a list of the Anaconda environments available to Jupyter Notebook. Figure 6.13 shows this command. You can only execute this command after the relevant Anaconda module is loaded.

jupyter kernelspec list
Figure 6.13 - Command to Output a List of Available Environments

The second command enables you to remove an Anaconda environment from the list of environments available to Jupyter Notebook. Figure 6.14 depicts how to use this command. Replace <env-name> with the name of your Anaconda environment.

jupyter kernelspec remove <env-name>
Figure 6.14 - Command to Remove an Environment from Jupyter

RStudio Sessions

RStudio is a web-based IDE (integrated development environment) for the R statistical language on the cluster. Like Jupyter, it provides additional functionality beyond a standard interactive console. The specifics of how to access and manipulate RStudio are the focus of this section. Please note that the cluster's implementation of RStudio runs from within a container, which means it cannot be modified or updated. If you require additional packages within RStudio, or if you need a different version of RStudio, consider building a custom Anaconda environment to use those packages and versions. For more information, please consult the Anaconda User Guide. Be aware that OpenOnDemand runs RStudio version 3.5.3 at the time of this writing.

To access RStudio, open the OpenOnDemand dashboard. In the top-left, select “Interactive Apps.” In the dropdown menu, select “RStudio Server.” After you click that option, the submission form in Figure 7.1 will appear.

RStudio Session Creation
Figure 7.1 - RStudio Session Submission Form

You can specify any of the options in the form to whatever you require; however, only the number of hours and the number of nodes are necessary. The other options will use the default values for those fields if you leave them blank. For more information on the Account, Queue, QoS, and Feature fields, please refer to the Running Jobs document.

After you specify the options for your session, select Launch. The session will be placed into the queue. The start time for the RStudio session depends on the options you specified in the form. When the session is ready, the interface will appear as it does in Figure 7.2. Select the “Connect to RStudio Server” option to begin the session. Figure 7.2 highlights this option.

Connect to RStudio
Figure 7.2 - RStudio Ready to Launch

A new browser tab will open with the RStudio Server session. When it initially starts, you will see an interface with three panes: a console pane, an environment pane, and a file explorer pane.

The console pane, depicted in Figure 7.3, is where you run R code. At the top of the window, there are several menus with different options. It is not the purpose of this document to describe these menus in detail. Instead, please consult the official RStudio documentation for more information.

RStudio Console Pane
Figure 7.3 - RStudio Console Pane

In addition to the menus, there are several buttons that allow you to create, open, and save files and projects, in addition to options for menu layouts and addins. Below these buttons is the console itself. You can input R code into the console, press Enter (Return), and see the output of that code.

The environment pane contains information that pertains to the IDE itself, such as what packages are enabled, what dataset is in use, and your command history. Figure 7.4 shows the environment pane with the Environment tab selected.

RStudio Environment Pane
Figure 7.4 - RStudio Environment Pane

In the file explorer pane, you can view and modify your files on the cluster, view additional packages, read documentation, and more. Figure 7.5 shows the file explorer pane. In the Files tab of the file explorer pane, you can create new folders, upload, delete, and rename files, in addition to several other options. To navigate through directories in the file explorer pane, click on the directory you wish to enter. To return to the previous directory, click the double dots (..). To return to your home directory, select “Home,” which has the icon of a house next to it.

RStudio File Explorer Pane
Figure 7.5 - RStudio File Explorer Pane

Whenever you type the “help()” command, the RStudio Server documentation will appear in the “Help” tab of the file explorer pane. You can view all relevant documentation for RStudio from this tab, if necessary. To expand the view so it is easier to read, minimize the environment pane by clicking the horizontal bar in the top-right of the pane. This will expand the file explorer pane, making it easier to read documentation. After you finish your RStudio Server session, select the red IO button in the top-right of the interface and confirm the shutdown. You will then be disconnected from the RStudio Server session. Be sure to also delete the job if any time remains on it. You will receive an email that confirms this deletion.

Known Issues

If you encounter a problem with OpenOnDemand and its services, please review the official Known Issues document before submitting a ticket. If the issue is not present in the official documentation or the proposed workarounds fail, then please submit a ticket to

Return to Top

Last Updated: 04 / 02 / 2020