Skip to main content

Open Research Cloud VMs

note

NDIP's core services no longer run on ORC VMs. They moved to the mickey and speedy Kubernetes clusters. ORC still hosts the old prod VM (now the VM proxy), the docs server, the ingress-proxy/Kafka VM, the Rucio VM for galaxy.ornl.gov, the monitoring cluster, the galaxy.ornl.gov instance and its Pulsar VMs. See the infrastructure overview.

Open Research Cloud (ORC) operates on the RedHat OpenStack Platform. If you have access to the NDIP project, you can view its resources on the dashboard.

note

To request access, submit a ticket here. Use
Security Level: Moderate and Project Name: NDIP.

note

Do not use the dashboard for deployments. It is only meant for validation and troubleshooting.

Infrastructure Management​

Deployments and updates are managed via Terraform and Ansible using a monorepo.

With this repository, we manage:

  • Security groups
  • Shared and local storage
  • Network ports (IPs)
  • Virtual machines

Additionally, shared storage is mounted using an Ansible playbook.

Configuration​

Current configuration​

resources*.yaml files describes all cloud resources currently deployed. To add or remove a VM or modify configurations, these are the primary files you will need to update. We currently have the main YAML file and two separate files to configure NDIPs and Pulsars.

Available parameters​

For a complete list of possible parameters, please refer to the example file.

Default values​

Additionally, there is a file with default settings that are applied across all instances.

Deployment​

To deploy (apply) the current configuration, we use GitLab CI/CD jobs. The deployment process involves several stages:

  1. Build (optional): This stage builds the Docker container with all necessary dependencies installed.
  2. Init: Initializes Terraform, with the Terraform state stored in GitLab.
  3. Validate: Validates the Terraform configuration.
  4. Plan: Generates or updates the Terraform plan, outlining the intended changes in the job output.
  5. Apply: Applies the Terraform plan, resulting in updates to the cloud resources.

The first three stages are accessible for every GitLab branch, allowing for modifications, validation, and previewing the potential impact.

Once everything looks good, you can create a Merge Request (MR), and when it is merged, the apply stage will become available in the main branch.

Legacy VM deployments​

We still have a couple of resources deployed from legacy repositories. These are:

Please see documentation (when available) or code in those repositories for more information.