BioTech Computing (BTC)
We compute life sciences

Data Transfer Instruction

TEST

The UW-Madison Biotechnology Center is currently open and following the university’s COVID-19 Response regarding campus operations.

Data Transfer Instructions

Note: this process is undergoing changes. Links may not work currently.

Beginning January 2021 all deliverable data will be deleted from our systems 60 days after it is made available to download.

Overview

UWBC provides multiple options to download deliverable data:

  1. UW-Madison PIs don’t need to download data at all. We can deliver it to your UW-Madison ResearchDrive.
  2. The UWBC Facility Files download page is available to all UWBC clients. It provides good performance and an easy to use interface for downloading your data. If you download multiple files at once the page will create a zip archive. If you primarily download smaller files (Sanger sequencing, mass spec, etc.) this may still be the most convenient option since you don’t need any special client software. For large or frequent downloads UW-Madison clients should consider the other options.
  3. Globus Connect provides the best performance, better than SFTP or the web portal. It also provides the best user experience, with email notifications of your download status and automatic retries of failed transfers. UW-Madison clients can log in using their NetIDs. Globus is also available for external users whose institution is a member of the InCommon Federation (most larger universities are), or a Google or ORCiD login can be used. Globus requires some setup, details are below on this page.
  4. BRC account holders can connect to brchome.biotech.wisc.edu via SSH or SFTP to access their home directories.
  5. SFTP access is available to all UW-Madison clients with a UW NetID. Performance is slightly better than the web portal. The main advantages over the web portal are the capability to download multiple files without creating a zip archive and accessibility through command-line clients. Details are below on this page.

Data access for deliverable data is based on your lab membership. Download directories are created for each PI. The PI, plus every member of the PI’s lab who is set up in our system will have access to those directories.

Web Portal download:

Your UWBC deliverable data can be downloaded at the UWBC Facility Files download page. Log in using your UW NetID (for UW-Madison users) or email address and password (for external users).

SFTP download:

There are two options for SFTP downloads:

  1. Any UW-Madison user in our system can log into our server download.biotech.wisc.edu using UW NetID credentials. This will give you access to all deliverable data that has been created for you by UWBC facilities. Only SFTP connections are allowed, SSH to this server will not work. At this time only UW-Madison users with UW NetIDs are supported for this type of SFTP transfer. Uploads to this server are not allowed.
  2. BRC account users can log in using their BRC account credentials. This will give access to the same deliverable data, plus the contents of your BRC home directory. This uses a different server, brchome.biotech.wisc.edu, which is reserved for data transfer purposes only. You can’t use the brchome server to analyze your data, just to download it. There are no data analysis applications installed.

For security reasons, we only allow SFTP transfers from UW-Madison networks. Remote users must use the web portal or connect to the campus network using the DoIT VPN service. Contact the DoIT help desk or your local IT support if you need help with the VPN service.

SFTP instructions for Windows:

  1. Download and install WinSCP.
  2. When you start the application you will get a dialog called WinSCP Login.
  3. Enter download.biotech.wisc.edu or brchome.biotech.wisc.edu under Host name (see the difference between the servers above).
  4. Enter your BRC account username and password.
  5. Make sure SFTP is selected under Protocol.
  6. Click Login.
  7. If you get a message saying “The server’s host key was not found in the cache…” click Yes.

If you need more assistance see this page: Guide to setting up WinSCP (Windows File Transfer)

SFTP instructions for Macs:

  1. Download and install CyberDuck.
  2. Select Open Connection under the File menu.
  3. Select SFTP from the drop down menu.
  4. Enter download.biotech.wisc.edu or brchome.biotech.wisc.edu under Server (see the difference between the servers above).
  5. Enter your BRC account username and password.
  6. Click Connect.
  7. If you get a message that says “Unknown host key” click Allow.

Use the Help menu in the application if you need further assistance.

SFTP instructions for Linux:

CentOS 7 and Ubuntu 16 include newer versions of the command line SFTP client that support autocomplete and recursive downloads (the SFTP client is part of openssh). There are also GUI clients available for Gnome, KDE, etc.

Globus Connect – we have a new server, these instructions have changed!

Please follow these instructions to download data from the UWBC Globus Connect servers. Globus Connect is a very fast, convenient method to transfer large amounts of data. Many educational institution users can log in using their home institution credentials. Globus also allows logins using Google or ORCiD accounts. UW-Madison users can log in with their NetIDs.

Regardless of the data transfer method, you need a certain level of network connectivity on your end. If the data to be transferred is over one TB we HIGHLY recommend you download it from a computer with a gigabit connection or faster. Work with your network support staff to get that if possible. A TB of data over a 100 Mb connection will take roughly 30 hours to download. Gigabit will reduce that by a factor of 10.

Please follow the steps below. UW-Madison users can skip to step 6.

  1. If you have been given a password for our web download page, log in here and fill out the form. You can skip the rest of this page.
  2. Otherwise we need your educational institution login, a Google login or an ORCiD iD. It should look like username@institution.edu, username@gmail.com or wwww-xxxx-yyyy-zzzz@orcid.org.Most .com email addresses other than gmail.com and google.com are not valid Globus logins – unless Google is the email provider for your company. Outlook.com email addresses will NOT work.
  3. Verify that this login works with Globus by logging into https://globus.org. Choose your home institution from the list on the page. If you find it, log in. If your home institution isn’t listed select Google or ORCiD.
  4. Once you have successfully logged in, email your user ID to us. It may be in the form of an email address like username@institution.edu. It may be the same as your email address, or it may not. If Google was your choice your login will typically be username@gmail.com. If you use ORCiD your login will look like wwww-xxxx-yyyy-zzzz@orcid.org.
  5. We will set up your access and you will get an email with a link sent to the email address you provided. When you open the link you will be able to access your data via the web, one file at a time.
  6. Install Globus Connect Personal. Your transfers will be FASTER and your experience will be MUCH BETTER.
  7. Once Globus Connect is installed, you need to make a collection on your computer:
    1. Click on the Globus icon (task bar on Windows or menu bar on a Mac) then Preferences.
    2. Use the Access tab to set up your local storage to be available as a collection in Globus. YOU MUST check the Writeable box!
    3. Then you can use Globus to transfer files from our collection to your collection. Open our collection on one side of the file transfer window and yours on the other side.
    4. To return to Globus click on the Globus icon and select Web: Transfer Files.

Choose use your existing organizational login and select your home institution. To find the UWBC endpoint that contains your files:

  • If you have a UW-Madison NetID, search in Globus Connect for UW-Madison Biotechnology Center – Internal Users
  • If you are from another institution, search in Globus Connect for UW-Madison Biotechnology Center – External Users.

NOTES:

  • Please use a WIRED connection to transfer large amounts of data.
  • Make sure you have enough hard drive space to hold the data.