The National Institute for Computational Sciences

Data Transfer

  Data Transfer

NICS provides several ways for transferring files to/from all of our HPC resources, that can benefit from the high speed network connections with the XSEDE network and Internet2.

GridFTP

The GridFTP protocol is the preferred method to transfer files. All XSEDE resources have GridFTP clients installed. Check for local availability at other locations. This method can be used on the command line (see our dedicated GridFTP page) or through the Globus web interface (see Globus below).

SCP, SFTP, rsync

For smaller data transfers, you can use the standard transfer utilities, SCP, SFTP and rsync, which are usually available on all Linux/Unix machines. There are command line and graphical clients available. These utilities are ill suited for transferring large amounts of data. More information on these utilities can be found on the transfer utilities page.

BBCP

This is another tool to securely and quickly copy data from source to target. The BBCP utility is capable of breaking up your transfer into multiple simultaneously transferring streams, thereby transferring data faster than single-streaming utilities such as SCP and SFTP. Check for local availability, but if it is not available the source code can be obtained from its homepage. Several examples about how to use it, can be found at its dedicated BBCP page.

Globus

NICS users can use the web Globus tool to perform data transfers to/from NICS supported resources. The visual interface makes it quite easy to move, back up or restore relevant data. To get you started, visit the Globus website and consult the Quick Start guide.

Globus endpoints to access NICS resources

XSEDE endpoints (i.e. using XSEDE authentication):

  • xsede#darter

One of the latest features of Globus is Globus Connect Personal. Globus Connect Personal turns your personal computer into a Globus endpoint so you can share and transfer files to/from a local machine—campus server, desktop computer or laptop.


Transfer files between local computer and XSEDE resources