/tag/storage
Service Status
Project Storage Data Migration The storage hardware serving the Research Project Storage file system has been experiencing intermittent disruptions since October 2023. In response, we acquired and installed new, upgraded hardware offering faster, more reliable access and capacity. However, because an Active File Management (AFM) connection between the old and new systems was used to facilitate automatic file transfers, the new system’s performance is being negatively impacted by the old hardware and causing accessibility issues for some users.
To mitigate these issues, Research Computing engineers are switching to an alternate backend migration process on February 26, 2024, at 9:00 a.m.
Using UVA’s High-Performance Computing Systems
Afton is the University of Virginia’s newest High-Performance Computing system. The Afton supercomputer is comprised of 300 compute node each with 96 compute cores based on the AMD EPYC 9454 architecture for a total of 28,800 cores. The increase in core count is augmented by a significant increase in memory per node compared to Rivanna. Each Afton node boasts a minimum of 750 Gigabytes of memory, with some supporting up to 1.5 Terabytes of RAM memory. The large amount of memory per node allows researchers to efficiently work with the ever-expanding datasets we are seeing across diverse research disciplines. The Afton and Rivanna systems provide access to 55 nodes with NVIDIA general purpose GPU accelerators (RTX2080, RTX3090, A6000, V100, A40, and A100), including an NVIDIA BasePOD.
Globus Data Transfer
Globus is a simple, reliable, and fast way to access and move your research data between systems. Globus allows you to transfer data to and from systems such as:
Laptops & personal workstations Rivanna/Afton HPC clusters High-Security Research Standard Storage Lab / departmental storage Tape archives Cloud storage Off-campus resources (ACCESS, National Labs) Globus can help you share research data with colleagues and co-investigators, or to move data back and forth between a lab workstation and Rivanna/Afton or your personal computer.
Are your data stored at a different institution? At a supercomputing facility? All you need is your institution’s login credentials.
Drive Mapping on Windows or Mac OSX
Research Standard and Research Project storage can be mapped to your Windows or Mac computer as a remote drive. If you are off Grounds you must be running a VPN, such as the UVA Anywhere or the More Secure VPN from ITS. We recommend the More Secure VPN if that is available to you.
Windows Open a File Explorer page. In the left column, right-click on This PC. In the drop-down box that appears, look for the Map Network Drive option. If you do not see this option, click on Show more Options, and then click on Map network drive….
Mission
Research Computing empowers UVA researchers to achieve more with cutting-edge computational resources. Our support team strives to create innovative solutions for researchers who need help solving complex optimization, parallelization, workflow, and data analysis issues. We build and maintain the University’s best computing platforms while educating the next generation of researchers on the power of advanced computing.
Commercial Data Sharing and Archiving Solutions
Box® Non-sensitive cloud storage UVA Box is a cloud-based storage and collaboration service that gives eligible members of the University community the ability to access, store, and share up to 1 TB of public / internal use University files securely—anywhere, anytime, on any device. Read more DropBox®/Sookasa® Highly Sensitive Data (PHI/PII) storage If you plan on storing highly sensitive data such as PHI or PII, UVA Health System offers a secure encrypted storage for Health System affiliated researchers, students, and staff. “DropBox Sookasa” is a free cloud-based service hosted on Dropbox that can be accessed over the internet on any device.
Rivanna Storage
There are a variety of options for storing large-scale research data at UVa. Public and Internal Use data storage systems can be accessed from the Rivanna and Afton high performance computing systems.
Storage Directories Name Quota Price Data Protection Accessible from Best Practices /home 200GB Free 1-week snapshots Rivanna/Afton /home is best used as a working directory when using Rivanna/Afton interactively. Slurm jobs run against /home will be slower than those run against /scratch. The /home directory is a personal storage space that is not shareable with other users. /scratch 10TB Free No backups, Data removed 90 days after last file access time Rivanna/Afton /scratch is a high performance parallel filesystem that is suitable for large scale computational work.
Research Standard Storage
Overview The Research Standard Storage file system provides users with a solution for research data storage and sharing. Public, internal use, and sensitive research data can be stored in Research Standard storage, and UVA Information Security provides details about data sensitivity classifications. Members in the same group have access to a shared directory created by the team lead or PI. Group membership can be defined and managed through Grouper (requires VPN connection). Research Standard storage is mounted on the HPC cluster and can also be accessed on a personal computer with an SMB mount, allowing for point-and-click file manipulation.
As of July, 2024, Each PI with a Research Computing account will have up to 10 TB of Research Standard Storage at no charge.
Integrated Sensor Networks in Arctic Alaska
Understanding the Changing Natural-Built Landscape in an Arctic Community Navigating the New Arctic (NNA) is one of The National Science Foundation’s 10 Big Ideas. NNA projects address convergence scientific challenges in the rapidly changing Arctic. Arctic research is needed to inform the economy, security and resilience of the Nation, the larger region and the globe. NNA empowers new research partnerships from local to international scales, diversifies the next generation of Arctic researchers, enhances efforts in formal and informal education, and integrates the co-production of knowledge where appropriate. This award fulfills part of that aim by addressing interactions among social systems, natural environment, and built environment in the following NNA focus areas: Arctic Residents, Data and Observation, Education, and Resilient Infrastructure.
Cloud Storage Solutions
Amazon Web Services Tiered object storage Amazon S3 and Glacier offer cloud-based, affordable, unlimited capacity for storage from anywhere. Advanced features include scalability, lifecycle management, encryption, and sharing. S3 is ideal for static files that need to be retrieved from any location (PDFs, images, video, etc.). Glacier is archival storage, perfect for grant compliance that requires data retention. How RC can help: Lower pricing - UVA has an Internet2 discount available for educational use. Contact us to create an account for you or your research project. Cost estimates - Cloud storage is not free. Consideration should be made to the size of your files and how often they will be retrieved.
ITS `home1` directories unavailable on Rivanna
ITS has completed phase 1 of its network reconfiguration and ITS NAS storage volumes have been remounted on Rivanna. RC managed Research Standard and Research Project storage are also fully available. ITS home1 directories, including /nv/t* mounts, remain unavailable on Rivanna. Users will still be able to mount ITS home1 storage to their local workstations and transfer their data via Globus and the UVA Main DTN. We regret the inconvenience.
Pricing
Below is a schedule of prices for Research Computing resources.
High Performance Computing Allocations Type SU Limits Cost SU Expiration Standard None Free 12 months Purchased None $0.01 Never Instructional 100,000 Free 2 weeks after last training session A service unit (SU) resembles usage of a trackable hardware resource for a specified amount of time. In its simplest form 1 SU = 1 core hour, but the SU charge rate can vary based on the specific hardware used. Resources like GPUs and memory may incur additional SU charges. About Allocations
Storage Name Security Cost Research Project Standard $70/TB per year Research Standard Standard $45/TB per year (Each PI with an RC account will be granted up to 10 TB of Research Standard Storage at no charge1) High-Security Research Standard High $45/TB/year Storage Details Request Storage
Transfer Files Using Amazon S3
Setup You will need to install and configure the awscli package in order to access objects in S3.
Install the AWS CLI The AWS CLI is available through the pip/pip3 installer:
If you have administrator privileges type
pip install awscli Otherwise use
pip install –user awscli The project is open source, so you can also download the source at https://github.com/aws/aws-cli
UVA HPC Users have two options:
Load the awscli module:
module load awscli If you need a different version, install it in your user directory:
pip install –user awscli==1.19.29 Authenticate the CLI to Amazon Web Services Once the aws package is installed, you will need to configure it:
ACCORD: Jupyter Lab
Back to Overview
Jupyter Lab allows for interactive, notebook-based analysis of data. A good choice for pulling quick results or refining your code in numerous languages including Python, R, Julia, bash, and others.
Learn more about Jupyter Lab
ACCORD: RStudio
Back to Overview
RStudio is the standard IDE for research using the R programming language.
Learn more about RStudio
ACCORD: Theia IDE
Back to Overview
Theia Python is a rich IDE that allows researchers to manage their files and data, write code with an intelligent editor, and execute code within a terminal session.
Learn more about the Theia Python IDE
How To Tips for Storage
Map your Research Standard or Research Project Storage to your Desktop Use Globus from the Command Line Work with files in Amazon S3
Globus Command-Line Interface
Install the Globus CLI The Globus CLI is available through the pip installer:
If you have administrator privileges type
pip install globus-cli Otherwise use
pip install –user globus-cli The project is open source, so you can also download the source at https://github.com/globus/globus-cli
If you would like to use the CLI from UVA HPC, please follow these directions below.
Authenticate using the Globus CLI Log in against your institutional authentication provider. In the case of UVA, we use NetBadge for signing in:
globus login This will open a page in your web browser where you select your institution and proceed to log in:
Storage FAQs
Accidental File Deletions Why Lease Storage Leased Storage Options File Transfer with Globus Permission Denied Error with Globus Help! I deleted my files accidentally! What can I do? For your home storage, the directory is /home/.snapshots . Snapshots are created once per day. Find the date you wish to find the snapshot for and navigate to your computing id. For GPFS Research Project (leased) storage, the directory is /gpfs/gpfs0/project/.snapshots. Neither Research Standard (leased) nor scratch storage is backed up in any way. Why should I lease storage? Leasing storage from Research Computing means that you do not have to run your own data server or backup system.
Frequently Asked Questions
General Do you have a general computing question?
Read our FAQ› Rivanna and Afton High Performance Computing Platforms
Read our FAQ › Ivy Secure Data Computing Platform
Read our FAQ › Storage Research Data Storage & Transfer
Read our FAQ ›
How To
General General tips and tricks for computational research.
General HowTos › Rivanna and Afton High Performance Computing platforms
HPC HowTos › Ivy Secure Data Computing Platform
Ivy HowTos › Storage Research Data Storage & Transfer
Storage HowTos ›
Research Data Storage
There are a variety of options for storing research data at UVA. Public and internal use data storage systems can be accessed from the Rivanna and Afton high performance computing systems. Sensitive and highly sensitive data can be stored and accessed within the Ivy secure computing environment. University Information Security provides an overview of the data sensitivity classifications.
All UVA faculty are eligible to serve as PI and request access to RC services (e.g. storage & HPC allocations, virtual machines, microservices) for their research group. Postdocs and staff are encouraged to use an allocation provided by a faculty sponsor, although they may request their own allocation pending departmental or RC approval.
Data Transfer
Data transfer Public & Moderately Sensitive Data Transfer Secure Copy (scp) scp uses secure shell (SSH) protocol to transfer files between your local machine and a remote host. scp can be used with the following syntax:
scp [source] [destination]
scp SourceFile mst3k@login.hpc.virginia.edu:/scratch/mst3k
scp SourceFile mst3k@login.hpc.virginia.edu:/project/Grouper_group_name
Detailed instructions and examples for using scp are listed here.
Secure File Transfer Protocol (sftp) sftp is a network protocol for secure file management. Instructions and examples for using sftp are located here.
Graphical File-Transfer Applications Filezilla and Cyberduck, and MobaXterm are examples of open source SFTP client software for file management through an interactive graphical user interface.
Research Project Storage
Overview The Research Project Storage file system provides users with a collaborative space for data storage and sharing. Public, internal use, and sensitive research data can be stored in Research Project storage, and UVA Information Security provides details about data sensitivity classifications. Members in the same group have access to a shared directory created by the team lead or PI. Group membership can be defined and managed through Grouper (requires VPN connection). /project storage is mounted on the HPC cluster and runs on a new scale-out NAS file system.
If you are not a researcher, UVA ITS offers Value storage for long-term storage of large scale data.
Slurm Job Manager
SLURM Would you like to take an interactive SLURM quiz? y/N |
Overview UVA HPC is a multi-user, managed environment. It is divided into login nodes (also called frontends), which are directly accessible by users, and compute nodes, which must be accessed through the resource manager. Users prepare their computational workloads, called jobs, on the login nodes and submit them to the job controller, a component of the resource manager that runs on login nodes and is responsible for scheduling jobs and monitoring the status of the compute nodes.
We use Slurm, an open-source tool that manages jobs for Linux clusters.
Highly Sensitive Data Storage - Ivy
Overview Residing within the High Security Zone (HSZ), the Ivy secure computing environment is designed to fit your highly sensitive research computing needs and meets HIPAA, FERPA, CUI and ITAR compliance standards. Within the HSZ, researchers can store their highly sensitive research data in High-Security Research Standard Storage.
Ivy Central Storage Ivy Central Storage (ICS) was an HSD parking zone and central storage pool with a capacity greater than 1PB. This storage space was available for researchers with highly sensitive data and could be mounted on an Ivy Virtual Machine.
As of 10/15/24, ICS will be upgraded to High-Security Research Standard Storage.
User Guides
High Performance Computing Standard and high security HPC to run your code, generally written in R, Python or shell scripts.
