An invaluable tool that I use a lot is wget. wget is a free terminal utility for downloading files from the Internet.
I do not have a reliable Internet connection and when I do get online, I am usually on a metered network. Not being able to connect to the Internet when I want to makes accessing online resources difficult. To solve this problem I use wget to download websites to my computer. I will show you how to do this yourself in this post.
The command to download a site is wget url
Here is an example using the Python Documentation Website wget http://docs.python.org/
This will download everything on the python docs website’s first page. Adding an “r” option to the wget command will cause it to download the entire site.
$ wget --recursive http://docs.python.org/
Other options you can make use of are:
- –domains : don’t follow links outside the domain in url
- –convert-links: convert links so that the work offline
- –no-clobber: don’t overwrite any existing file. Use this option to resume an interrupted download
wget can download files over FTP, HTTPS and even retrieve files through proxies.