Research Repository: Depositing Large Datasets

To archive and make your large dataset (50GB+) publicly available, please submit the "Large Dataset Deposit" form to initiate the process.

This process is necessary because large datasets require separate storage and access. Smaller datasets can be uploaded directly to the KAUST Repository through the Dataset Submission Form .

After submission of the large dataset deposit form, you will receive instructions for transferring the files to the library. 

After the files are received, a DOI will be registered for the dataset that redirects to a landing page describing the dataset on the KAUST Repository. This landing page will also contain instructions mentioned below for users to retrieve the dataset files via Globus. 

Changes to the metadata or files for an existing dataset should be communicated directly to the repository team at repository@kaust.edu.sa .

Most of the files in the KAUST Repository can be retrieved through the browser, but to download the files from some large datasets, you will need to access these large files through Globus.

What is Globus?

​Globus is a platform developed and operated as a not-for-profit service by the University of Chicago that enables the secure transfer of files of all sizes.

How to Log In and Download Files with Globus?

On KAUST Repository page click on "Link to Globus download directory" to begin the download process.

 

You will be redirected to the Globus login page, you can use your organizational login, Google, ORCID, or Globus ID, all allow you to create new account if you don’t already have one.

 

 

Download files from Globus to your device

Once you have set up a Globus account, a page titled "File Manager | Globus" should appear:

The main location of the data deposit on Globus will appear in the Collection field. The "Path" field represents a unique pathway to the specific files for download. There is no need to alter anything in these two fields, since all the files for download should appear in the list below.

Click on the second "Panels" icon in the top-right corner. Alternatively, you can click on the icon with the tooltip "Transfer or Sync to…" on the right-hand icon menu in the single-panel view. In either case, a page with two "Collection" panels should appear:

 

Specify your device as an endpoint in Globus

You will need to establish a personal endpoint on your computer so that you can transfer data to it. To do this, you will need to download and install Globus Connect Personal, which will connect to the Globus Web App.

You can find detailed installation instructions for Mac, Windows and Linux operating systems at https://www.globus.org/globus-connect-personal. On the instructions for Mac and Windows, the Configuration section will tell you how to change the folder that Globus can access. (In particular, you may need to set the destination folder as writable for transfers to be successful.)

Download files from Globus to your device

After selecting your Collection endpoint, select the files you would like to download from the collection in the first (left-most) panel. You may either click "select all" above the list of files or select only the files you would like to download.

In the second (right-hand) panel, select the location where you would like the files to save on your computer. If you do not select a location, Globus defaults to whatever location you previously set up in Globus Connect Personal.

 

When endpoints for both sides are set, the "Start" button below the list of  files will turn darker blue and will be clickable. Click on this "Start" button to download your files.

 

A message in a green bar (reading "Transfer request submitted successfully. Task id:...") should appear above the file window, indicating a successful transfer request. Because Globus is expected to handle large files that take some time to download, the platform is set up to pause whenever your computer is no longer connected to the Internet. It will resume automatically when your computer is reconnected.

Once the files have been completely downloaded, they will appear in the destination folder on your computer. You can always check the status of tasks by clicking "Activity" in the left-hand navigation on the Globus page where you set up your transfer. You will receive an email, to the email address provided when you signed in, when the transfer has been completed.

More Information

For further help please refer to the "how to guides" on the Globus Docs website, including videos. You can also contact us at repository@kaust.edu.sa .