The Walter Darby Bannard Archive Returns
Post #1684 • April 22, 2014, 11:34 AM • 1 Comment
After this disaster I wondered if I could get it back. Fortunately some of the modernists in my life are rather technical people, as I suppose I am. After a few hours of research, one especially helpful suggestion from a programmer in Baltimore, Greg Smith, after I bellyached about this on Facebook, and a few more hours of research and tweaking, I was able to scrape the html out of the Wayback Machine with this command in a shell:
wget --mirror -p --convert-links -r -e robots=off -np https://web.archive.org/2013/http://wdbannard.org/protracted/url/at/archive.org
wget is program that retrieves resources over HTTP and the like. --mirror turns on recursion to infinite depth and time-stamping. -p tells wget to retrieve everything the page needs to display properly, such as stylesheets. --convert-links waits until the files are downloaded and then changes the links so that they display properly on the local machine. I shouldn't have used -r—it turns recursion on and was redundant with --mirror. -e allows you to override settings in wget, in this case the one that honors robots.txt instructions to not scrape a site. Since this is the case at archive.org and I had an emergency, I asked Wayback to pardon my manners and went ahead and scraped it. -np stops recursion from going back up above the parent directory, and thereby scraping the entire Internet, basically attempting to produce a bad version of Google on your local machine until you run out of memory. The last field is the URL of the WDBA at archive.org. (Kind of. You get the idea.)
John has done a wonderful thing for contemporary art history and deserves accolades galore. Thank you, John.
Is this your first time reading from the archive? There are some jewels in there. I especially recommend The War Against the Good in Art from 1972, Craft and Art Envy from 1986, and Artbroken: What Art Is and How We Stopped Making It, but I spent many edifying hours putting this together and I recommend the material therein on the highest terms. I am also quite relieved to see it back online.