How do I download an entire raw data directory?

In order to recursively download an entire raw data directory from rawdata.oceanobservatories.org, you can use the following script:

wget -r -np -e robots=off URL

Notes:

• -r signifies that wget should recursively download data in any subdirectories it finds.
• -np prevents files from parent directories from being downloaded.
• -e robots=off tells wget to ignore the robots.txt file. If this command is left out, the robots.txt file tells wget that it does not like web crawlers and this will prevent wget from working.

Here is a complete example: 

wget -r -np -e robots=off https://rawdata.oceanobservatories.org/files/CE01ISSP/D00001/

While this is a simple example, there are many more options you can try when using wget. Check out the wget section in this article for more guidance.

— Last revised on November 16, 2017 —