FASRC allows the use of AI tools on the Cannon and FASSE FASRC clusters. Our users are free to install AI tools they need to execute their AI-based workflow under their cluster profile and provide it with data. However, we urge our users to be aware of the guidelines that the University has put forward for the use of such tools at Harvard. We do ask that for all HUIT-supported LLMs, our users register apps and access API keys that are under Harvard agreement prior to accessing those LLMs on the cluster.  

Additionally, if you are using AI agents for your work, you need to know how to use them responsibly on the cluster without compromising your data security or your user profile on the cluster. The AI Agents on the FASRC Clusters policy page walks you through the main points you need to be aware of in order to use AI agents responsibly on the cluster while maintaining the data integrity and data privacy of your work.

Note: Sometimes Chrome does not work for HUIT websites, especially if you are *not* on the University VPN. In that case, you can access those websites using Firefox.

Best Practices for AI workflows on FASRC Clusters

In order to execute an AI/ML workflow successfully on the cluster, it is important to understand:

• How to safely install and launch a tool/software on the cluster that’s needed for your workflow
• How to request resources for it properly
• What are some of the pitfalls to be mindful of

This section provides a landing area for all the relevant documentation that will help you implement your AI/ML workflow on the cluster correctly. Before executing your AI workflow on the FASRC clusters, make sure to follow the safety guidelines mentioned above, and FASRC guidelines for launching an AI tool such as OpenAI’s ChatGPT.

How to install and utilize AI/ML tools or platforms? 

To successfully execute your AI/ML workflow on the cluster, you may need a variety of tools or platforms. The list below captures various tools and platforms that are currently available on FASRC clusters. Based on your needs, you can go through the relevant documentation to get guidance on how to install and execute these tools on the cluster, or utilize these platforms for your work.

* Anthropic

* HeavyAI

* Knime

* OpenAI

* Python Package Installation

* PyTorch

* Tensorflow

* VSCode

* Cursor

* AI Extensions

How to request resources for AI workflows?

This will typically require you to request GPUs and utilize them efficiently. GPU Computing on FASRC Clusters walks you through:

• The resource allocation process for GPUs for a batch and an interactive job,
• How to load and work with CUDA modules,
• Example codes for GPU computing on the cluster
• How to monitor your job’s performance on a GPU node.

In addition to that, one must be mindful of their job’s efficiency. A detailed description of what it means to run a job efficiently on the cluster is provided on Job Efficiency and Optimization Best Practices.    

Lookout For:

In addition to following the best practices mentioned above, one must be aware of some of the pitfalls associated with executing AI workflows on the cluster. For example:

1. Are you using an AI tool or an extension via a code editor, such as VSCode or Cursor, on the login node?
2. Have you inadvertently given permission to the AI tool to delete files or folders?
3. Are you aware of prompt injection attacks while using an LLM-based application?
4. What steps have you taken to secure your data against data leakage?
5. Are you aware of your data security level?

Training

We regularly hold training sessions to support AI-based workflows on the cluster.  Please refer to our Training Calendar for upcoming sessions, and Training Material to access AI/ML computing documentation and presentation slide decks &/or video recordings of our previous training sessions.

• All users of the FASRC clusters and associated resources are bound by Harvard University guidelines on AI use.
• Using personal accounts or API keys for work on the FASRC cluster is not in accordance with Harvard policy and guidelines.
• Never run agents with elevated privileges.
• You, the FASRC account holder, are responsible for the actions of your agents, These processes are running with your privileges.
• Like you, your agent should behave responsibly on the cluster following customer and responsibilities and acceptable use guidelines as well as any data use requirements places on you, your lab, your PI, or the data being accessed by an agent.
• You, the FASRC account holder, are responsible for your agent’s access to data that is in any way restricted and for its actions on any file systems resulting in data loss or data leak.
• Limit access. Run agents in a sandbox or container where possible. If not possible, consider whether this action increases the chances of unintended data loss, leaks, or other un-wanted outcomes.
• You, the FASRC account holder, are responsible for reviewing any AI and agentic skills before use.
• You, the FASRC account holder, are responsible for monitoring your agents, their API usage, cluster usage, and any associated costs they may be incurring on behalf of you, your lab, or the university.
  • Monitor your agent sessions actively.
  • Limit the number of agents on login nodes.
  • Don’t run multiple agents without monitoring..
  • Terminate processes that behave unexpectedly.
  • If agents become disruptive, we may introduce automations to moderate their activities so that other users are not affected.

This document covers some, but not all, concerns and contingencies around the use of AI agents on the FASRC clusters.

If you have concerns or questions, please contact FASRC or consider coming to Office Hours.

Data Integrity and Data Privacy

You are responsible for the actions of your agents, therefore you are responsible for any breach of a DUA or other instrument which restricts access to data should that data be leaked, copied, removed, etc. by your agent. If in doubt, consult your PI when running agents in FASSE or on any data which has a data use agreement or other restrictions.

For questions about data use agreements you may be bound to, consult with your PI, OVPR or contact FASRC Research Data Management.

Related Links:

• Generative AI Guidelines | Harvard University Information Technology
• Generative AI @ Harvard
• Cluster Customs and Responsibilities
• PI Responsibilities at FASRC
• Research Data Management and Policy Compliance Resources – OVPR
• FASRC Cluster Storage Policy
• FASRC Data Ownership and Access Policy
• AI Extensions on FASRC Clusters
• Containers
• OpenAI
• FASRC Guidelines for OpenAI Key and Harvard Agreement
• AI Tool Comparison | Harvard University Information Technology

AI extensions are tools that are typically used to integrate generative AI capabilities (or models) directly into interfaces, such as browsers, integrated development environments (IDEs) or interactive computing platforms like code editors, Jupyter notebook, JupyterLab, etc. They are used to enhance productivity while utilizing an AI model of your choice. For example, browser-based AI extensions, such as Sider, Harpa AI, can be used to summarize content, automate tasks through sidebars, or translate texts particularly for Chrome. On the other hand, IDE-based AI extensions, such as Jupyter-AI, GitHub Copilot, JetBrains AI Assistant can be used for accelerating full-stack software development, pair-programming, and analysis, to name a few, while enabling a powerful way to explore generative AI models in your applications.      

Here, we describe the method to install some of the IDE-based AI extensions on FASRC clusters, how to use them with a few of our in-built applications while ensuring that the AI model being utilized is being accessed in a safe manner and abides by Harvard’s guidelines for the use of generative AI (Gen AI) tools at Harvard.

Jupyter-AI Extension

The Jupyter-AI extension is an open-source tool designed to integrate generative AI models directly into the JupyterLab and Jupyter Notebook interfaces. Once the extension is installed, the `%ai` magic turns the notebook into a reproducible Gen AI playground and can work anywhere the IPython kernel is available.

Install Jupyter-AI Extension

In order to install jupyter-ai extension, one needs to follow the instructions for installing Python packages on FASRC clusters along with those for jupyter-ai. Following are the combined steps to illustrate the installation procedure, starting from requesting a compute node on the cluster, and creating a vanilla mamba environment with a specific version of Python. This is followed by activating the environment and installing the minimal jupyter-ai extension (reason explained below) using `pip`. Notice, `ipykernel` is needed so that the environment can be loaded as a Python kernel in a Jupyter notebook or JupyterLab downstream.

* salloc --partition test --nodes=1 --cpus-per-task=2 --mem=8GB --time=02:00:00 
* module load python 
* mamba create -y -p /n/holylabs/<PI_lab>/Lab/jupyter-ai-liteenv python=3.13 
* mamba activate /n/holylabs/<PI_lab>/Lab/jupyter-ai-liteenv 
* pip install ipykernel 'jupyter-ai==3.0.0b9' 'jupyter-ai-magics==3.0.0b7' boto3  
* mamba deactivate

As suggested in our Python package installation document, we recommend users create the environment to install the jupyter-ai package in a location other than their `$HOME`, preferably on the `holylabs` location of your lab folder. Our tests have shown that by doing that, enabling the extension in the notebook downstream is nearly 3-15 times faster (depending on the host and the interface used – Jupyter notebook vs JupyterLab) than creating an environment in `$HOME` to install the extension.

In the installation steps above, we have illustrated the minimal installation method for the jupyter-ai extension (also see Jupyter AI docs). This way ensures that Bedrock-based models, such as Anthropic, can be accessed via HUIT-provided AWS Bedrock, which is recommended for Harvard users. This also includes installing additional dependencies needed to run the available Anthropic models, e.g. boto3 that is needed to provide added support for AWS Bedrock models. OpenAI models, however, are installed by default. You can execute the `mamba list` command to see the packages installed in the environment after activating it.

API key For AI model

After the installation of the extension is complete and prior to utilizing it on the cluster, you need to ensure that the API key of the Gen AI model, you are intending to use, has been obtained and has been associated with Harvard’s agreement with the corresponding company.

OpenAI

The details to create an OpenAI app and obtain an API key that is associated with the Harvard agreement are provided as Option #2 (and Option #3 for limited free usage by FAS) of FASRC OpenAI guidelines. This procedure is also applicable to other API products available on HUIT API portal that can be registered as an app. See HUIT’s AI API service offerings for more details on this topic. 

Bedrock Products

For Gen AI models available via AWS Bedrock, such as Anthropic, you need to follow these instructions to set up Anthropic access that is secured and compliant. These instructions are similar to the ones provided in OpenAI with the exception that you would need to configure your shell environment appropriately to use Harvard’s API endpoint. More details regarding requesting access to Bedrock API, examples to set up client tools to connect to AI API(s) via Harvard’s gateway, and how to use those APIs to access the corresponding LLMs can be found on HUIT AI services – AWS Bedrock. 

Jupyter-AI extension using OOD

Once the jupyter-ai extension is installed (minimal or the full version), you can start using it via our Jupyter notebook/JupyterLab OOD app. If you are new to OOD, then refer to our OOD apps Jupyter notebook section to learn how to launch it and load the jupyter-ai mamba environment, you created above, as a kernel. Below, we link to a sample Jupyter notebook that you can run on OOD to see how to use the jupyter-ai extension with LLMs provided by OpenAI and Anthropic while ensuring that the models are being accessed securely.

Examples

See FASRC User Codes repo for AI Tools to access a sample notebook showing how to access an available provider’s LLM securely and leverage the jupyter-ai extension to write a simple Python code using that LLM. It also shows how to interact with an LLM without the jupyter-ai extension.

KNIME is an open-source data analytics, reporting, and integration platform that is meant to perform various aspects of machine-learning & data mining through its modular data pipelining concept. The platform offers a way to integrate various tasks ranging from developing analytic models to deploying them and sharing insights with your team. The KNIME Analytics Platform offers users 300+ connectors to multiple data sources and integrations to all popular machine learning libraries.

The software’s key capabilities include Data Access & Transformation, Data Analytics, Visualization & Reporting, Statistics & Machine Learning, Generative AI, Collaboration, Governance, Data Apps, Automation, AI Agents. 

Given KNIME’s wide scale use and applicability, we have converted it into a system-wide module that can be loaded from anywhere on any of the FASRC clusters, Cannon or FASSE. Additionally, we have packaged it as an app that can be launched using the cluster web interface, Open on Demand (OOD).

KNIME as a module

Knime is available as a module on the FASRC clusters. In order to know more about the module including the versions available and how to load one of them, execute from a terminal on the cluster: module spider knime

This would pull up the information on the versions of KNIME software that are available to load. For example, for a user jharvard on a compute node, the module spider command would produce the following output:

[jharvard@holy8a26602 ~]$ module spider knime/
knime:
Description:
An open-source data analytics, reporting, and integration platform meant to perform various aspects of machine-learning & data mining through its modular data pipelining concept.

Versions:
knime/5.4.3-fasrc01
knime/5.4.4-fasrc01

For detailed information about a specific "knime" package (including how to load the modules) use the module's full name.

Note that names that have a trailing (E) are extensions provided by other modules.

For example:
$ module spider knime/5.4.4-fasrc01

To load a specific module, one can execute: module load knime/5.4.3-fasrc01

Or, to load the default & typically the latest module, one can run: module load knime command. This would result in, e.g.:

[jharvard@holy8a26602 ~]$ module load knime
[jharvard@holy8a26602 ~]$ module list
Currently Loaded Modules:
  1) knime/5.4.4-fasrc01

Once the knime module is loaded, one can launch the GUI by running the knime executable on the terminal provided you ssh into the cluster using X11 forwarding, preferably with the -Y option, and that XQuartz (MacOS) or  MobaXterm (Windows) is installed on your local device that is being used to login to the cluster. For example:

ssh -Y jharvard@login.rc.fas.harvard.edu


[jharvard@holylogin05 ~]$ salloc -p test --x11 --time=2:00:00 --mem=4g
[jharvard@holy8a26602 ~]$ module load knime
[jharvard@holy8a26602 ~]$ knime

One can ignore the following libGL errors and should expect to see a GUI appear as shown in the screen shot below.
libGL error: No matching fbConfigs or visuals found
libGL error: failed to load driver: swrast

Note: While you can launch KNIME directly on the cluster using X11 forwarding, it is laggy and doesn’t render itself well to faster executions that might be needed for certain KNIME workflows. To avoid issues associated with X11 forwarding, we recommend launching KNIME using OOD.

Both these modules are also available to use via the Knime OOD app, as explained below.

KNIME on OOD

KNIME can be run from Open OnDemand (OOD, formerly known as VDI) by choosing it from the Interactive Apps menu, and specifying your resource needs. Hit Launch, wait for the session to start, and click the “Launch Knime” button.

• Cannon OOD
• FASSE OOD

You can also launch KNIME from the Remote Desktop app on OOD.

Pre-installed Extensions

Both KNIME modules come with the following pre-installed extensions:

1. For GIS: Geospatial Analytics Extension for KNIME
2. For Programming:
   KNIME Python Integration
   KNIME Interactive R Statistics Integration
3. For Machine Learning:
   KNIME H2O Machine Learning Integration
   KNIME XGBoost Integration
   KNIME Machine Learning Interpretability Extension
4. For OpenAI, Hugging Face, and other LLMs: KNIME AI Extension
5. For AI Assistant Coding: KNIME AI Assistant (Labs)
6. For Google Drive Integration: KNIME Google Connectors

Note: New extensions cannot be installed by the users on the fly as modules don’t come with write permissions.

KNIME Tutorial

The link here takes you to the KNIME tutorial that has been prepared by Lingbo Liu from Harvard’s Center for Geographic Analysis (CGA). This tutorial is best executed by launching the Knime app on OOD.

See OpenAI website and documentation.

Security

Please, carefully read Harvard’s AI guidelines and Generative AI tool comparision.  See our FASRC Guidelines for OpenAI Key and Harvard Agreement.

You can only use openAI and other genAI on your own is if it is not Harvard work related and/or public, non-sensitive data (data security level 1).

For data security levels 2 and 3, you need to work with your school to discuss your needs. It is your responsibility to make sure you get setup with the appropriate contractual coverage and environment — esp. to avoid having the model learn from your input and leak sensitive information.

Installation

You can install OpenAI in a conda/mamba environment:

[jharvard@boslogin01 ~]$ salloc --partition test --time 01:00:00 --mem-per-cpu 4G -c 2
[jharvard@holy8a24301 ~]$ module load python/3.10.12-fasrc01
[jharvard@holy8a24301 ~]$ export PYTHONNOUSERSITE=yes
[jharvard@holy8a24301 ~]$ mamba create -n openai_env openai

Run OpenAI Model

You will need to provide an OpenAI key. You can generate one from
https://platform.openai.com/api-keys.

# Request an interactive job
[jharvard@boslogin01 ~]$ salloc --partition test --time 01:00:00 --mem-per-cpu 4G -c 2
# Source conda environment
[jharvard@holy8a24301 ~]$ source activate openai_env
# replace my_key with the key that you generated on OpenAI's website
[jharvard@holy8a24301 ~]$ export OPENAI_API_KEY='my_key'
# set SSL_CERT_FILE with system's certificate
(openai_env) [jharvard@holy8a24301 ~]$ export SSL_CERT_FILE='/etc/pki/tls/certs/ca-bundle.crt'
# run OpenAI example
(openai_env) [jharvard@holy8a24301 ~]$ python openai-test.py

Note: OpenAI uses the python package httpx. You must set the variable SSL_CERT_FILE to use the system’s certificate. If you do not set SSL_CERT_FILE OpenAI will give this error:

ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1006)

Examples

See User Codes on OpenAI including an example on OpenAI Whisper.

Resources

• OpenAI ChatGPT Edu is an advanced version of the ChatGPT tool with enhanced security and privacy protections.
• OpenAI ChatGPT Edu for FAS: ChatGPT Edu to members of the FAS for use in your Harvard work.

See Claude on the website and documentation.

Security

Please, carefully read Harvard’s AI guidelines and Generative AI tool comparison.

You can only use Anthropic tools and other GenAI models on non-sensitive data (data security level 1) public data on Cannon.

For data security levels 2 and 3, you need to work with your school to discuss your needs. It is your responsibility to make sure you get setup with the appropriate contractual coverage and environment — esp. to avoid having the model learn from your input and leak sensitive information.

Installation

You can install Claude software development kit (SDK) for Python in a conda/mamba environment. Here is a quick installation script to get you started:

#!/bin/bash
salloc --partition=test --time=02:00:00 --mem=8G --cpus-per-task=2
module load python
export PYTHONNOUSERSITE=yes
mamba create --name claude_env python -y
source activate claude_env
pip install anthropic
conda deactivate

Run Anthropic Model

You will need to provide an Anthropic API key. You can generate one from their API page. Also, see their quickstart guide. If you want to generate an API key for your lab, then you can follow the instructions here to set that up and add the billing information to your account prior to running Claude on the cluster.

Examples

See FASRC User Codes repo for example Claude scripts.

See also Anthropic’s own Anthropic Cookbooks.

Resources

Anthropic Quickstarts: A collection of projects designed to help developers quickly get started with building applications using the Anthropic API. Each quickstart provides a foundation that you can easily build upon and customize for your specific needs.

Anthropic Official Documentation: Comprehensive guide to using Claude, including setup, API usage, and troubleshooting.

Claude AI API Access: Portal to access the Claude API, set up API keys, and manage your integrations.

Claude’s Capabilities and Model Family: Learn about Claude’s different models such as Sonnet and Haiku, tailored for various performance needs.

PyTorch, developed by Facebook’s AI Research lab, is an open-source machine learning library that offers a flexible platform for building deep learning models. It features a Python front end and integrates seamlessly with Python libraries like NumPy, SciPy, and Cython to extend its functionality. Unique for its use of dynamic computational graphs, unlike TensorFlow’s static graphs, PyTorch allows for greater flexibility in model design. This is particularly advantageous for research applications involving novel architectures.

The library supports GPU acceleration, enhancing performance significantly, which is vital for tackling high-level research tasks in areas such as climate change modeling, DNA sequence analysis, and AI research that involve large datasets and complex architectures. Automatic differentiation in PyTorch is handled through a tape-based system at both the functional and neural network layers, offering both speed and flexibility as a deep learning framework.

These instructions are intended to help you install PyTorch on the FASRC cluster.

PyTorch with CUDA 12.9 in a conda/mamba environment

These instructions set up a conda/mamba virtual environment with PyTorch version 2.2.1 and CUDA version 12.1, where the cuda-toolkit is installed directly in the conda environment.

Start an interactive job requesting GPUs, e.g., (Note: you will want to start a session on the same type of hardware as what you will run on)

salloc -p gpu -t 0-06:00 --mem=8000 --gres=gpu:1

Load required software modules, e.g.,

module load python

Create a conda environment, e.g.,

mamba create -n pt2.9.1_cuda12.9 pip wheel -y

Activate the new conda/mamba virtual environment:

mamba activate pt2.9.1_cuda12.9

Install PyTorch with pip

pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cu129

Install additional Python packages, if needed, e.g.,

mamba install -c conda-forge numpy scipy pandas matplotlib seaborn h5py jupyterlab jupyterlab-spellchecker scikit-learn -y

PyTorch with CUDA 11.8 from a software module

These instructions set up a conda/mamba environment with PyTorch version 2.2.0 and CUDA version 11.8, where CUDA is loaded as a software module, cuda/11.8.0-fasrc01

# Start an interactive job on a GPU node (target the architecture where you plan to run), e.g.,
salloc -p gpu -t 0-06:00 --mem=8000 --gres=gpu:1

# Load the required modules, e.g.,
module load python 
module load cuda/11.8.0-fasrc01 # CUDA version 11.8.0

# Create a conda/mamba environment and activate it, e.g.,
mamba create -n pt2.2.0_cuda11.8 python=3.10 pip wheel -y
mamba activate pt2.2.0_cuda11.8

# Install PyTorch
mamba install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia -y

# Install additional packages, e.g.,
mamba install pandas scikit-learn matplotlib seaborn jupyterlab -y

(pt2.9.1_cuda12.9) [username@holygpu7c26103 ~]$ pip install torch_geometric

If you are running PyTorch on GPU with multi-instance GPU (MIG) mode on (e.g. gpu_test partition), see PyTorch on MIG mode

You can run the following tests to ensure that PyTorch was installed properly and can find the GPU card. Example output of PyTorch checks:

(pt2.9.1_cuda12.9) [jharvard@holygpu8a12102 ~]$ python -c "import torch;print(torch.__version__)"
2.9.1+cu129
(pt2.9.1_cuda12.9) [jharvard@holygpu8a12102 ~]$ python -c 'import torch;print(torch.cuda.is_available())'
True
(pt2.9.1_cuda12.9) [jharvard@holygpu8a12102 ~]$ python -c 'import torch;print(torch.cuda.device_count())'
1
(pt2.9.1_cuda12.9) [jharvard@holygpu8a12102 ~]$ python -c 'import torch;print(torch.cuda.current_device())'
0
(pt2.9.1_cuda12.9) [jharvard@holygpu8a12102 ~]$ python -c 'import torch;print(torch.cuda.device(0))'
<torch.cuda.device object at 0x146606f70ec0>
(pt2.9.1_cuda12.9) [jharvard@holygpu8a12102 ~]$ python -c 'import torch;print(torch.cuda.get_device_name(0))'
NVIDIA H200

For an interactive session to work with the GPUs you can use following:

salloc -p gpu -t 0-06:00 --mem=8000 --gres=gpu:1

Load required software modules and source your PyTorch conda environment.

[jharvard@holygpu8a12102 ~]$ module load python
[jharvard@holygpu8a12102 ~]$ source activate pt2.9.1_cuda12.9
(pt2.9.1_cuda12.9) [jharvard@holygpu8a12102 ~]$

Test PyTorch interactively:

(pt2.9.1_cuda12.9) [jharvard@holygpu8a12102 pytorch]$ python check_gpu.py 
Using device: cuda

NVIDIA H200
Memory Usage:
Allocated: 0.0 GB
Reserved: 0.0 GB

tensor([[ 0.1388, -1.1285, 0.1722, -0.0027]], device='cuda:0')

check_gpu.py: checks if GPUs are available and if available sets up the device to use them.

An example batch-job submission script is included below:

#!/bin/bash
#SBATCH -c 1
#SBATCH -N 1
#SBATCH -t 0-00:30
#SBATCH -p gpu
#SBATCH --gres=gpu:1
#SBATCH --mem=4G
#SBATCH -o pytorch_%j.out 
#SBATCH -e pytorch_%j.err 

# Load software modules and source conda environment
module load python
source activate pt2.9.1_cuda12.9

# Run program
srun -c 1 --gres=gpu:1 python check_gpu.py

If you name the above batch-job submission script run.sbatch, for instance, the job is submitted with:

sbatch run.sbatch

PyTorch on MIG Mode

Note: Currently only the gpu_test partition has MIG mode enabled.

# Get GPU card name
nvidia-smi -L

# Set CUDA_VISIBLE_DEVICES with the MIG instance
export CUDA_VISIBLE_DEVICES=MIG-5b36b802-0ab0-5f37-af2d-ac23f40ef62d

Or automate the process with:

export CUDA_VISIBLE_DEVICES=$(nvidia-smi -L | awk '/MIG/ {gsub(/[()]/,"");print $NF}')

Best Practices

PyTorch and Jupyter Notebook on Open OnDemand

To use PyTorch in Jupyter Notebook on Open OnDemand/VDI, install ipykernel and ipywidgets:

(pt2.9.1_cuda12.9) [jharvard@holygpu8a12102 ~]$ mamba install ipykernel ipywidgets

Pull a PyTorch Singularity Container

Alternatively, you can pull and use a PyTorch singularity container:

singularity pull docker://pytorch/pytorch:2.1.0-cuda12.1-cudnn8-runtime

To install other versions, refer to the PyTorch compatibility chart:

Examples

For example scripts covering installation, and use cases, see our User Codes > AI > PyTorch repo.

External Resources:

• Various PyTorch/CUDA version compatibility chart.
• PyTorch Official Documentation – Comprehensive resource for all functionalities including tutorials and API reference.
• PyTorch Tutorials – Practical tutorials covering basic to advanced topics, specifically tailored for deep learning and high-performance computing tasks.
• PyTorch Discussion Forums – A community forum for discussing specific issues, sharing solutions, and collaborating on projects.
• Introduction to Distributed Deep Learning – A detailed guide on implementing distributed deep learning models in PyTorch.
• Efficient PyTorch – An article by PyTorch developers on best practices for optimizing deep learning models for production.

Python packages on the cluster are primarily managed with Mamba.  Direct use of pip, outside of a virtual environment, is discouraged on the FASRC clusters.

Mamba is a package manager that is a drop-in replacement for Conda, and is generally faster and better at resolving dependencies:

• Speed: Mamba is written in C++, which makes it faster than Conda. Mamba uses parallel processing and efficient code to install packages faster.
• Compatibility: Mamba is fully compatible with Conda, so it can use the same commands, packages, and environment configurations.
• Cross-platform support: Mamba works on Mac, Linux and Windows.
• Dependency resolution: Mamba is better at resolving dependencies than Conda.
• Environment creation: Mamba is faster at creating environments, especially large ones.
• Package repository: Mamba uses Mambaforge ( aka conda-forge ), the most up to date packages available.

Important:
repo.anaconda.com, which hosts the Anaconda Distribution / Miniconda default channels, is blocked at the network level on the FASRC cluster due to licensing restrictions. Use the FASRC python environment module, which is built with Miniforge3, and has conda-forge set as its default channel. 

Mamba is a drop-in replacement for Conda and uses the same commands and configuration options as conda. You can swap almost all commands between conda & mamba.  By default, mamba uses conda-forge, the free Mambaforge package repository.  ( In this doc, we will generally only refer to mamba.)

Usage

mamba is available on the FASRC cluster as a software module either as Mambaforge/Miniforge or as python/3* which is aliased to mamba. One can access this by loading either of the following modules:

$ module load python

To see Python’s version

$ python --version

Environments

You can create a virtual environment with mamba in the same way as with conda. However, it is important to start an interactive session prior to creating an environment and installing desired packages in the following manner:

$ salloc --partition test --nodes=1 --cpus-per-task=2 --mem=4GB --time=0-02:00:00
$ module load python

By default, Python packages are installed in your home directory, in ~/.conda/envs. If you would like to locate the packages elsewhere, like a Lab shared directory, then specify the absolute file path.

export CONDA_PKGS_DIRS=/<FILEPATH>/conda/pkgs
export CONDA_ENVS_PATH=/<FILEPATH>/conda/envs

Create an environment using mamba

$ mamba create -n <ENV_NAME>

Alternatively, you can create an environment and install packages at the same time. This ensures package dependencies are met and could also speed up your setup time significantly. The general syntax to create an environment and install packages is:

$ mamba create -n <ENV_NAME> <PACKAGES>

For example,

$ mamba create -n python_env1 python={PYTHON_VERS} pip wheel

$ source activate python_env1

$ source deactivate

To list packages inside the environment:

$ mamba list

To install new packages in the environment (optional: -y is to proceed with installation):

$ mamba install -y <PACKAGE>

For example, to install numpy:

$ mamba install -y numpy

To install a package from a specific channel, add --channel (or -c) argument:

$ mamba install --channel <CHANNEL-NAME> -y <PACKAGE>

For example, to install the package boltons from the conda-forge channel:

$ mamba install --channel conda-forge boltons

To uninstall packages:

$ mamba uninstall <PACKAGE>

To delete an environment:

$ conda remove -n <ENV_NAME> --all -y

For additional features, please refer to the Mamba documentation.

Pip Installs

Avoid using pip outside of a mamba environment on any FASRC cluster. If you run pip install outside of a mamba environment, the installed packages will be placed in your $HOME/.local directory, which can lead to package conflicts and may cause some packages to fail to install or load correctly via mamba.

For example, if your environment name is python_env1:

$ module load python
$ source activate python_env1
$ pip install <package_name>

Best Practices

Use mamba environment in Jupyter Notebooks

If you would like to use a mamba environment as a kernel in a Jupyter Notebook on Open OnDemand (Cannon OOD or FASSE OOD), you have to install packages, ipykernel (for Python; for other languages, install the corresponding conda package for the supported Jupyter kernel). This packages will allow Jupyter to detect mamba environments that you created from the command line.

For example, if your environment name is python_env1:

$ module load python
$ source activate python_env1
$ mamba install ipykernel

1. Open a Jupyter Notebook (a .ipynb file)
2. On the top menu, click Kernel -> Change kernel -> select the conda environment

Mamba environments in a desired location

With mamba, use the -p or --prefix option to specify writing environment files to a desired location, such as the holylabs location.  Don’t use your home directory as it has very low performance due to filesystem latency.  Using a lab share location, you can also share your conda environment with other people on the cluster.  Keep in mind, you will need to make the destination directory, and specify the python version to use.  For example:

$ mamba create --prefix /n/holylabs/{YOUR_LAB}/Lab/<envname> python={PYTHON_VERS}

$ source activate /n/holylabs/{YOUR_LAB}/Lab/<envname>

To delete an environment at that desired location: $ conda remove -p /n/holylabs/{YOUR_LAB}/Lab/<envname> --all -y

Troubleshooting

Interactive vs. batch jobs

If your code works in an interactive job, but fails in a slurm batch job,

1. You are submitting your jobs from within a mamba environment.
   Solution 1: Deactivate your environment with the command mamba deactivate and submit the job or
   Solution 2: Open another terminal and submit the job from outside the environment.

2. Check if your ~/.bashrc or ~/.bash_profile files have a section of conda initialize or a source activate command. The conda initialize section is known to create issues on the FASRC clusters.
   Solution: Delete the section between the two conda initialize statements. If you have source activate in those files, delete it or comment it out.
   For more information on ~/.bashrc files, see https://docs.rc.fas.harvard.edu/kb/editing-your-bashrc/

Jupyter Notebook or JupyterLab on Open OnDemand/VDI problems

See Jupyter Notebook or JupyterLab on Open OnDemand/VDI troubleshooting section.

Unable to install packages

If you are not able to install packages or the package installation is taking a significantly long time, check your ~/.condarc file. As stated in Conda docs, this is an optional runtime configuration file. One can use this file to configure conda/mamba to search from specific channels for package installation.

We recommend users not have this file or keep it empty. This allows users to install packages in their conda/mamba environments using the defaults provided by the open-source distribution, Miniforge , that we have made available to our users via our newer Python modules.

If, for any reason, ~/.condarc exists in your cluster profile then check its contents. If the default channel is showing up as conda , edit it to conda-forge so that your ~/.condarc uses this open-source channel for package installation.

Similarly, if  you had created an environment a long time ago using the Anaconda distribution and it is no longer working, then it is best to create a new environment using the open-source distribution as described above while ensuring that ~/.condarc, if exists, is pointing to conda-forge as its default channel.

For example, if you created a conda environment using one of our older Python modules, say Anaconda2/2019.10-fasrc01, you can see that conda is configured to use repo.anaconda.com for package installation.

$ module load Anaconda2 
$ conda info 
... 
channel URLs : 
https://repo.anaconda.com/pkgs/main/linux-64 
https://repo.anaconda.com/pkgs/main/noarch 
https://repo.anaconda.com/pkgs/r/linux-64 
https://repo.anaconda.com/pkgs/r/noarch 
...

In order to change this configuration, you can execute the conda config command to ensure that conda now points to conda-forge. This would also create a .condarc file in your $HOME, if it already doesn’t exist:

$ conda config --add default_channels https://conda.anaconda.org/conda-forge/ 

$ cat ~/.condarc 
default_channels: - https://conda.anaconda.org/conda-forge/ 

$ conda info 
... 
channel URLs : 
https://conda.anaconda.org/conda-forge/linux-64 
https://conda.anaconda.org/conda-forge/noarch
...

However, we are asking our users to be aware of and follow the Guidelines for the use of Generative AI tools at Harvard [sometimes Chrome might not work for these HUIT websites, especially if you are “not” on university VPN. In that case, you can access these websites using Firefox].

OpenAI on the cluster

Following is a set of guidelines that we have put together for interested parties to ensure safe usage of OpenAI tools on the cluster and the data that is provided to them. 

There are two ways to get started on the process of getting your OpenAI account attached with Harvard’s enterprise agreement. 

Option 1 (needs a credit card) – Best executed by the PI

1. Create a lab/PI/school-based OpenAI account using an email address and password of your choosing that can be easily distributed to other members of the group.
2. This will generate an OpenAI API key that would need to be stored safely. The key will be required by other members of the account to install and use OpenAI on the cluster.
3. After creating the account, go to your profile and click on Settings.
4. Look for Organization ID on that page.
5. Copy that ID and send an email using the template below to ithelp@harvard.edu or generativeAI@fas.harvard.edu to get this associated with the enterprise agreement that Harvard has with OpenAI. It could take up to a week for the association to take place. 
6. This would ensure that whatever data is provided to OpenAI stays within that agreement and is not made public by the company.
7. Attach a credit card with this account that will be used for billing. This could be the PI or division’s C-card. 
8. Once the newly created OpenAI account has been associated with Harvard’s agreement and a credit card has been attached to it, the PI or the manager of this OpenAI account can now add members/students by going to the Settings page and inviting new members to the group.
9. The OpenAI API key will also have to be shared with new members.
10. Use this API key for the software tool/library that uses the OpenAI API.
11. The member will also need to be made aware of the data classification level that is attached to a certain GAI tool and its subsequent use on the cluster. 

Note: FASRC has tested using the OpenAI key for installing and using the corresponding software with Option# 1.

Email Template

Subject: Request to Associate OpenAI Organization ID with Harvard Enterprise Agreement

Dear HUIT GenAI Support Team,

I am requesting assistance in associating my OpenAI Organization ID with Harvard University’s Enterprise Agreement (EA) with OpenAI. 

As a Harvard affiliate, I want to utilize the OpenAI APIs, which offer increased API rate limits and the ability to use level 3 data, under the terms of the EA established by Harvard.

Organization ID:

• My Organization ID is [Your Organization ID Here]. You can find this ID in the OpenAI API portal under Settings -> Organization (platform.openai.com).

I understand that once my Organization ID is submitted, the association with the Enterprise Agreement could take up to a week to be confirmed by OpenAI. Additionally, I acknowledge that API-related billing will be charged to the credit card on record with my OpenAI account, as OpenAI does not support PO/invoice billing.

Regards,

<Name>

Option 2 (HUIT recommended – needs 33-digit billing code) – Best executed by the PI

Individual Account:

1. Go to Harvard’s API portal.
2. Click on either of the two options: 
   1. AI Services – OpenAI Direct API
   2. Or AI Services – OpenAI via Azure
3. Follow the instructions given on the corresponding page. 
4. For example, for either of the options, one will have to fill out the HUIT billing form for new customers located on HarvardKey – Harvard University Authentication Service to obtain a customer account number followed by registering the app using the API portal’s Guides – Register an App | prod-common-portal
5. Once the app is registered, you should be able to receive the corresponding API key (not the OpenAI key) from HUIT. 
6. The API key is already associated with the enterprise agreement that Harvard has with OpenAI, so no need to send HUIT an email to get it associated as mentioned in Option# 1.
7. Use this API key for the software tool/library that uses the OpenAI API.

Team Account:

This feature on the API portal allows developer teams to “own” an API consumer app, instead of individuals. See Guides – Create a Team | prod-common-portal. 

Following are the steps a PI can take to create a team and add members to it so that they can access the API key associated with the “team” account. The owner can manage the team (add new people as time goes on, or remove them).  Each member of the team can log into the portal to access the API key for their app. 

Please be sure to enter the email addresses carefully in the team setup, and they should be the email addresses associated with their HarvardKey.

1. Create the team and list each developer.
2. Register the app and select the desired team as the app owner.
3. In the app registration, select the APIs you want access to.

Note: FASRC has not verified the use of API key for installing the OpenAI software on the cluster with Option# 2.

Option 3 (free, FAS and HUIT only)

The OpenAI “Community Developer” API provides “$10/month in OpenAI API credits for students, faculty, staff, and researchers in FAS and Harvard College.”

1. Go to Harvard’s API portal and Sign In.
2. Register an app that uses the OpenAI for Community Developers API (must be signed in to follow link) 
   1. Click on <user>@<harvard-email> in the top right corner of the page
   2.  Select “Apps”
   3. Click “+ New App”
   4.  Fill in an App Name
   5. Enable “AI Services – OpenAI API for Community Developers”
   6. Save
3. Copy the resulting API Key
4. Use this API key with your application/library that requires access to the OpenAI API.

Reminder: All our users are allowed to work with data classified as Level 2 on Cannon and Level 3 on FASSE. The member will also need to be made aware of the data classification level that is attached to a certain GAI tool and its subsequent use on the cluster. The university has guidelines on what sort of data can and cannot be exposed to third party systems. Please see: Guidelines for Using ChatGPT and other Generative AI tools at Harvard.

Resources:

1. AI @ The FAS 
2. Generative AI @ Harvard 
3. Generative Artificial Intelligence (AI) Guidelines | Harvard University Information Technology  
4. Comparison of Generative AI Tools at Harvard
5. HUIT AI Services – OpenAI Direct API
6. Guides – Create a Team | prod-common-portal

Important:
We encourage all our users to utilize the Open On Demand (OOD) web interface of the cluster to launch VS Code when remote development work is not required. The instructions to launch VS Code using the Remote Desktop App are given here. 

Prerequisites

1. Recent version of VS Code installed for your local machine
2. Remote Explorer and Remote SSH extensions installed, if not already present by default

FASRC Recommendation

Based on our internal evaluation of the three approaches, mentioned below, and the interaction with our user community, to launch VSCode on a compute node, we recommend our users utilize Approach I: Remote – Tunnel via batch job over the other two. The Remote – Tunnel via batch job approach submits a batch job to the scheduler on the cluster, thereby providing resilience toward network glitches that could disrupt VSCode session on a compute node if launched using Approach II or III.

Note: We limit our users to a maximum of 5 login sessions, so be aware of the number of VSCode instances you spawn on the cluster.

Approach I: Remote – Tunnel via batch job

Note: The method described here and in Approach II will launch a single VS Code session at a time for a user on the cluster. The Remote – Tunnel approaches do not support concurrent sessions on the cluster for a user. 

In order to establish a remote tunnel between your local machine and that of the cluster, as an sbatch job, execute the following steps.

1. Copy the vscode.job script:

   #!/bin/bash
   #SBATCH -p test         # partition. Remember to change to a desired partition
   #SBATCH --mem=8g        # memory in GB
   #SBATCH --time=04:00:00 # time in HH:MM:SS
   #SBATCH -c 1            # number of cores
   
   set -o errexit -o nounset -o pipefail
   MY_SCRATCH=$(TMPDIR=/scratch mktemp -d)
   echo $MY_SCRATCH
   
   #Obtain the tarball and untar it in $MY_SCRATCH location to obtain the
   #executable, code, and run it using the provider of your choice
   curl -Lk 'https://code.visualstudio.com/sha/download?build=stable&os=cli-alpine-x64' | tar -C $MY_SCRATCH -xzf -
   
   VSCODE_CLI_DISABLE_KEYCHAIN_ENCRYPT=1 $MY_SCRATCH/code tunnel user login --provider github
   #VSCODE_CLI_DISABLE_KEYCHAIN_ENCRYPT=1 $MY_SCRATCH/code tunnel user login --provider microsoft
   
   #Accept the license terms & launch the tunnel
   $MY_SCRATCH/code tunnel --accept-server-license-terms --name cannontunnel

   vscode.job script uses the github provider authentication. Harvard users are recommended to use the github authentication provider with a personal GitHub account. The microsoft authentication provider does not work for Harvard Microsoft accounts, as HUIT has phased out Microsoft device code authentication for Harvard Microsoft accounts as of July 2026.  You may use a personal Microsoft account to authenticate.

2. Submit the job from a private location (somewhere that only you have access to, for example your $HOME directory) from which others can’t see the log file

   $ sbatch vscode.job

3. Look at the end of the output file

   $ tail -f slurm-32579761.out
   /scratch/tmp.LOipZlrD5y
   % Total % Received % Xferd Average Speed Time Time Time Current
   Dload Upload Total Spent Left Speed
   100 162 100 162 0 0 150 0 0:00:01 0:00:01 --:--:-- 150
   100 11.6M 100 11.6M 0 0 7874k 0 0:00:01 0:00:01 --:--:-- 7874k
   To grant access to the server, please log into https://github.com/login/device and use code XXXX-1234

4. Got to Github login page and login with your own Github account. Enter the 8-digit code that is in the slurm-jobID.out file.
5. Now, you have two options
   1. Use a web client by opening vscode.dev link from the slurm-jobID.out file (see the last line with the link vscode.dev):

      $ tail -f slurm-30642242.out
      /scratch/tmp.LOipZlrD5y
      % Total % Received % Xferd Average Speed Time Time Time Current
      Dload Upload Total Spent Left Speed
      100 162 100 162 0 0 150 0 0:00:01 0:00:01 --:--:-- 150
      100 11.6M 100 11.6M 0 0 7874k 0 0:00:01 0:00:01 --:--:-- 7874k
      To grant access to the server, please log into https://github.com/login/device and use code XXXX-1234
      *
      * Visual Studio Code Server
      *
      * By using the software, you agree to
      * the Visual Studio Code Server License Terms (https://aka.ms/vscode-server-license) and
      * the Microsoft Privacy Statement (https://privacy.microsoft.com/en-US/privacystatement).
      *
      
      Visual Studio Code Tunnel v1.127.0
      
      ➜ Tunnel: cannontunnel
      ➜ Open: https://vscode.dev/tunnel/cannontunnel/n/home/jharvard/

   2. Use vscode local client — see below

Using vscode local client (option #2)

1. In your local vscode (in your own laptop/desktop), add the Remote Tunnel extension (ms-vscode.remote-server)
   1. On the local VSCode, install Remote Tunnel extension
   2. Click on VS Code Account menu (the person icon on the bottom left corner), choose “Turn on Remote Tunnel Access”
2. Connect to the cluster:
   1. Click on the bottom left corner
   2. Options will appear on the top text bar (if they don’t appear, start typing Connect to Tunnel)
   3. Select “Connect to Tunnel…”
      
3. Then choose the authentication method that you used in vscode.job, microsoft or github
   
4. Click on the Remote Explorer icon and pull up the Remote Tunnel drop-down menu
5. Click on cannontunnel to get connected to the remote machine either in the same VS Code window (indicated by ->) or a new one (icon besides ->).
   Prior to clicking, make sure you see: Remote -> Tunnels -> cannontunnel running
6. Finally, when you get vscode connected, you can also open a terminal on vscode that will be running on the compute node where your submitted job is running.

Enjoy your work using your local VSCode on the compute node.

Approach II: Remote – Tunnel interactive

In order to establish a remote tunnel between your local machine and that of the cluster, as an interactive job, execute the following steps. Remember to replace <username> with your FASRC username.

1. ssh <username>@login.rc.fas.harvard.edu

2. curl -Lk 'https://code.visualstudio.com/sha/download?build=stable&os=cli-alpine-x64' --output vscode_cli.tar.gz

3. tar -xzf vscode_cli.tar.gz

4. An executable, code, will be generated in your current working directory. Either keep it in your $HOME or move it to your LABS folder, e.g.
   1. mv code /n/holylabs/rc_admin/Everyone/
5. Add the path to your ~/.bashrc so that the executable is always available to you regardless of the node you are on, e.g.,
   1. export PATH=/n/holylabs/rc_admin/Everyone:$PATH
6. Save ~/.bashrc, and on the terminal prompt, execute the command: source ~/.bashrc
7. Go to a compute node, e.g.: salloc -p gpu_test --gpus 1 --mem 10000 -t 0-01:00
8. Execute the command: code tunnel
9. Follow the instructions on the screen and log in using either your Github or Microsoft account, e.g.: Github Account
10. To grant access to the server, open the URL https://github.com/login/device and copy-paste the code given on the screen
11. Name the machine, e.g.: cannoncompute
12. Open the link that appears in your local browser and follow the authentication process as mentioned in steps# 3 & 4 of https://code.visualstudio.com/docs/remote/tunnels#_using-the-code-cli
13. Once the authentication is complete, you can either open the link that appears on the screen on your local browser and run VS Code from there or launch it locally as mentioned below.
14. On the local VSCode, install Remote Tunnel extension
15. Click on VS Code Account menu, choose “Turn on Remote Tunnel Access”
16. Click on cannoncompute to get connected to the remote machine either in the same VS Code window (indicated by ->) or a new one (icon besides ->). Prior to clicking, make sure you see:
    Remote -> Tunnels -> cannoncompute running

Note: Every time you access a compute node, the executable, code, will be in your path. However, you will have to repeat step#10 before executing step#16 above in order to start a fresh tunnel. 

Approach III: Remote – SSH

In order to connect remotely to the cluster using VS Code, you need to edit the SSH configuration file on your local machine.

• For Mac OS and Linux users, the file is located at ~/.ssh/config. If it’s not there, then create a file with that name.
• For Windows users, the file is located at C:\Users\<username>\.ssh\config. Here, <username> refers to your local username on the machine. Same as above, if the file is not present, then create one.

There are two ways to get connected to the cluster remotely:

1. Connect to the login node using VS Code.
   Important: This connection must be used for writing &/or editing your code only. Please do not use this connection to run Jupyter notebook or any other script directly on the login node.
2. Connect to the compute node using VS Code.
   Important: This connection can be used for running notebooks and scripts directly on the compute node. Avoid using this connection for writing &/or editing your code as this is a non-compute work, which can be carried out from the login node. 

SSH configuration file

Login Node

Adding the following to your SSH configuration file will let you connect to the login node of the cluster only with the Single Sign-On option enabled. The name of the Host here is chosen to be cannon but you can name it to whatever you like, e.g., login or something else. In what follows, replace <username> with your FASRC username.

For Mac:

Host cannon
User <username>
HostName login.rc.fas.harvard.edu
ControlMaster auto
ControlPath ~/.ssh/%r@%h:%p

For Windows:

The SSH ControlMaster option for single sign-on is not supported for Windows. Hence, Windows users can only establish a connection to the login node by either disabling the ControlMaster option or not having that at all in the SSH configuration file, as shown below:

Host cannon
User <username>
HostName login.rc.fas.harvard.edu
ControlMaster no
ControlPath none

or

Host cannon
User <username>
HostName login.rc.fas.harvard.edu

Compute Node

In order to connect to the compute node of the cluster directly, execute the following two steps on your local machine:

Note: Establishing a remote SSH connection to a compute node via VSCode works only for Mac OS. For Windows users, this option is not supported and we recommend they utilize the Remote-Tunnel Approaches I or II for launching VSCode on a compute node.

1. Generate a pair of public and private SSH keys for your local machine, if you have not done so previously, and add the public key to the login node of the cluster:
   In the ~/.ssh folder of your local machine, see if id_ed25519.pub is present. If not, then generate private and public keys using the command:

   ssh-keygen -t ed25519 -b 4096

   Then submit the public key to the cluster using the following command:

   ssh-copy-id -i ~/.ssh/id_ed25519.pub <username>@login.rc.fas.harvard.edu

   This will append your local public key to ~/.ssh/authorized_keys in your home directory ($HOME) on the cluster so that your local machine is recognized.

2. Add the following to your local ~/.ssh/config file by replacing <username> with your FASRC username. Make sure that the portion for connecting to the login node from above is also present in your SSH configuration file. You can edit the name of the Host to whatever you like or keep it as compute. There are two ProxyCommand examples shown here to demonstrate how the ProxyCommand can be used to launch a job on a compute node of  the cluster with a desired configuration of resources through the salloc command. Uncommenting the first one will launch a job on the gpu_test partition of the Cannon cluster whereas uncommenting the second one will launch it on the test partition.

Host compute
UserKnownHostsFile=/dev/null
ForwardAgent yes
StrictHostKeyChecking no
LogLevel ERROR
# substitute your username here
User <username>
RequestTTY yes
# Uncomment the command below to get a GPU node on the gpu_test partition. Comment out the 2nd ProxyCommand
#ProxyCommand ssh -q cannon "salloc --immediate=180 --job-name=vscode --partition gpu_test --gres=gpu:1 --time=0-01:00 --mem=4GB --quiet /bin/bash -c 'echo $SLURM_JOBID > ~/vscode-job-id; nc \$SLURM_NODELIST 22'"

# Uncomment the command below to get a non-GPU node on the test partition. Comment out the 1st ProxyCommand
ProxyCommand ssh -q cannon "salloc --immediate=180 --job-name=vscode --partition test --time=0-01:00 --mem=4GB --quiet /bin/bash -c 'echo $SLURM_JOBID > ~/vscode-job-id; nc \$SLURM_NODELIST 22'"

Note: Remember to change the Slurm directives, such as --mem, --time, --partition, etc., in the salloc command based on your workflow and how you are planning to use the VSCode session on the cluster. For example, if the program you are trying to run needs more memory, then it is best to request that much amount of memory using the --mem flag in the salloc command prior to launching the VSCode session on the cluster otherwise it could result in Out Of Memory error.  

Important: Make sure to pass the name of Host being used for the login node to the ProxyCommand for connecting to a compute node. For example, here, we have named the Host as cannon for connecting to the login node. The same name, cannon is then being passed to the Proxycommand to establish connection to a compute node via ssh. Passing any other name to Proxycommand ssh -q would result in a connection not being established error.

Once the necessary changes have been made to the SSH configuration file, open VS Code on your local machine and click on the Remote Explorer icon on the bottom left panel. You will see two options listed under SSH – cannon and compute (or whatever name you chose for the Host in your SSH configuration file).

Connect using VS Code

Login Node

Click on the cannon option and select whether you would like to continue in the same window (indicated by ->) or open a new one (icon next to ->). Once selected, enter your 2FA credentials on the VS Code’s search bar when prompted. For the login node, a successful connection would look like the following.

Compute Node

In order to establish a successful connection to Cannon’s compute node, we need to be mindful that VS Code requires two connections to open a remote window (see the section “Connecting to systems that dynamically assign machines per connection” in VS Code’s Remote Development Tips and Tricks). Hence, there are two ways to achieve that.

Option 1

First, open a connection to cannon in a new window on VS Code by entering your FASRC credentials and then open another connection to compute/vscode on VS Code either as a new window or continue in the current window. You will not have to enter your credentials again to get connected to the compute node since the master connection is already enabled through the cannon connection that you initiated earlier on VS Code.

Option 2

If you don’t want to open a new connection to cannon, then open a terminal on your local machine and type the following command, as mentioned in our Single Sign-on document, and enter your FASRC credentials to establish the master connection first.

ssh -CX -o ServerAliveInterval=30 -fN cannon

Then open VS Code and directly click on compute/vscode to get connected to the compute node. Once a successful connection is established, you should be able to run your notebook or any other script directly on the compute node using VS Code.

Note: If you have a stale SSH connection to cannon running in the background, it could pose potential problems. The session could be killed in the following manner.

$ ssh -O check cannon
Master running (pid=#)
$ ssh -O exit cannon
Exit request sent.
$ ssh -O check cannon
Control socket connect(<path-to-connection>): No such file or directory

VSCode For FASSE

One cannot use Remote SSH to establish connection between your local device and a FASSE compute node via VSCode as salloc is not permitted on FASSE due to security reasons. However, one can use Remote SSH to connect to FASSE’s login node using VSCode.

For establishing a connection to the compute node on FASSE, launching a Remote Tunnel using Approach I is your only option. However, prior to doing that, make sure you enable access to the internet, by setting a network proxy, to launch Remote Tunnel on your browser, since you will be connected to the FASSE VPN realm (@fasse). Additionally, once the tunnel is established, remember to install extensions as needed prior to starting your work using VSCode.

In order to launch a remote tunnel on a FASSE compute node using VSCode on your local device, you would also need to add the same network proxy to VSCode Settings on your local device, as shown below.

Note: Remember to unset the network proxy from VSCode, on your local device, when not using FASSE VPN as that will block your other non-FASSE VSCode remote sessions from your local device.

Add Folders to Workspace on VSCode Explorer

Once you are able to successfully launch a VSCode session on the cluster, using one of the approaches mentioned above, you might need to access various folders on the cluster to execute your workflow. One can do that using the Explorer feature of VSCode. However, on the VSCode remote instance, when you click on Explorer  -> Open Folder, it will open $HOME, by default, as shown below.

In order to add another folder to your workspace, especially housed in locations such as netscratch, holylabs, holylfs, etc., do the following:

1. Type: >add on VSCode Search-Welcome bar and choose Workspaces: Add Folders to Workspace... . See below:
2. If you would like to add your folder on netscratch or holylabs or some such location, first open a terminal on the remote instance of VSCode and type that path. Copy the entire path and then paste it on the Search-Welcome bar. See below:Do not start typing the path in the Search-Welcome bar, make sure to copy-paste the full path otherwise VS Code may hang while attempting to list all the subdirectories of that location, e.g., /n/netscratch.
   
   
3. Click ok to add that folder to your workspace
4. On the remote instance, you will be prompted to answer whether you trust the authors of the files in this folder and then Reload.Go ahead and click yes (if you truly trust them), and let the session reload.
5. Now you will be able to see your folder listed under Explorer as Untitled (Workspace)
6. This folder would be available to you in your workspace for as long as the current session is active. For new sessions, repeat steps #1-4 to add desired folder(s) to your workspace

 Best Practices

• Maximum of 5 login sessions are allowed per user at a time. Be aware of the number of VS Code instances you spawn on the cluster 
• Login node session 
  • Use for writing &/or editing your code only 
  • Do not use it to run Jupyter notebook, R, Matlab, or any other script
• Compute node session
  • Use for running notebooks & scripts 
  • Avoid using for writing &/or editing your code as this is a non-compute work
• For interactive sessions, better to be on VPN to get stable connection
• Remember to close jobs that are launched through interactive or sbatch VSCode sessions as follows:
  • click on the icon next to Launchpad on VSCode GUI
  • click on “Close Remote Connection”
• For Remote Tunnel session, if VS Code work is complete, in addition to above, execute the following to ensure that the Slurm job is also cancelled:
  • squeue -u <username>
  • scancel <JOBID>

Troubleshooting VSCode Connection Issues

1. Make sure that you are on VPN to get a stable connection.
2. Match the name being used in the SSH command to what was declared under “Host” in your SSH config file for the login node.
3. Make sure that the --mem flag has been used in the ProxyCommand in your SSH config file and that enough memory is being allocated to your job. If you already have it, then try increasing it to see if that works for you.
4. Open a terminal and try connecting to a login and compute node (if on Mac) by typing: ssh <Host> (replace Host with the corresponding names used for login and compute nodes). If you get connected, then your SSH configuration file is set properly.
5. Consider commenting out conda initialization statements in ~/.bashrc to avoid dealing with plausible issues caused due to initialization.
6. ​Delete the bin folder from .vscode-server or .vscode &/or removing Cache, CachedData, CachedExtensionsVSIXs, Code Cache, etc. folders. You can find these on Cannon on $HOME/.vscode/data &/or $HOME/.vscode-server/data/.
7. Check your $HOME quota and remove plausible culprits, such as ~/.cache. See instructions to clear disk space on Home directory full.
8. Make sure that there are no lingering SSO connections. See the Note at the end of Approach III – Remote SSH section.
9. Failed to parse remote port: Try removing .lockfiles.
10. Try implementing Approach I – Remote Tunnel via batch job to see if you are able to launch a Remote Tunnel as an sbatch job in the background to ensure that your work is not getting disrupted by any network glitches.
11. If you continue to have problems, consider coming to our office hours to troubleshoot this live

Dev Containers for custom software environments

VS Code supports opening a source code repository in a container. The container software environment is defined in a JSON file that adheres to the Development Containers (dev containers) specification (https://containers.dev/). The same git repository can be used to develop and run code in the same software environment on the FASRC cluster, a GitHub Codespace, or your laptop.

Using Dev Containers on the FASRC Cluster

It is recommended to (1) use VS Code in an OOD Remote Desktop session, or (2) use Approach I: Remote – Tunnel via batch job with a VS Code local client (the Visual Studio Code Dev Container extension is currently not supported in the web-browser-based VS Code (https://vscode.dev) with a remote tunnel (microsoft/vscode-remote-release issue #9059)).

Setup

1. Install the Visual Studio Code Dev Containers extension
2. Configure the extension to use Podman on the FASRC cluster:
   Change the following settings in Code > Settings > Extensions > Dev Containers:
   1. Dev > Containers: Docker Path – change “docker” to “podman”
   2. Dev > Containers: Docker Socket Path – change “/var/run/docker.sock” to “/tmp/podman-run-<uid>/podman/podman.sock”, replacing “<uid>” with your FAS RC user ID. Your FASRC user ID can be determined by running the command “id -u” on the FASRC cluster:
      [jharvard@holylogin05 ~]$ id -u
21442NOTE: if using a local VS Code (not OOD Remote Desktop), you will need to revert these changes in the future if you use dev containers via a local Docker installation

Launch

See the VS Code dev container quick start for how to open a repository in a dev container.

Known Limitations

The following are known limitations using VS Code Dev Containers on the FASRC cluster with Podman:

• The docker-in-docker dev container feature is currently unsupported
• The docker-outside-of-docker dev container feature requires a workaround to use podman on the host instead of a Docker daemon:
  1. In a VS Code terminal (on the FASRC cluster), issue the following command to start an API service that listens on /tmp/containers-user-$(id -u)/podman/podman.sock on the compute node:nohup podman system service -t 0 >/dev/null 2>&1 &  2. In the relevant devcontainer.json file, ensure the following settings are listed (see here for a complete example devcontainer.json):
"remoteUser": "root",
"mounts": ["source=/tmp/podman-run-1234/podman/podman.sock,target=/var/run/docker-host.sock,type=bind"], // replace "1234" with your UID
"remoteEnv": {
"DOCKER_BUILDKIT": "0"
}
"workspaceFolder": "${localWorkspaceFolder}",
"workspaceMount": "source=${localWorkspaceFolder},target=${localWorkspaceFolder},type=bind"
3. In the VS Code command palette, select “Dev Containers: Rebuild and Reopen in Container”