
Laden Sie das gesamte Wayback Machine-Archiv für eine bestimmte URL herunter.
Waybackpack ist ein Befehlszeilenwerkzeug, mit dem Sie das gesamte Wayback-Machine-Archiv für eine bestimmte URL herunterladen können.
Zum Beispiel, um jede Kopie der Startseite des Arbeitsministeriums durch das Jahr 1996 herunterzuladen (das zufällig das erste Jahr ist, in dem die Seite archiviert wurde), führen Sie Folgendes aus:
waybackpack http://www.dol.gov/ -d ~/Downloads/dol-wayback --to-date 1996
Ergebnis:
~/Downloads/dol-wayback/
├── 19961102145216
│ └── www.dol.gov
│ └── index.html
├── 19961103063843
│ └── www.dol.gov
│ └── index.html
├── 19961222171647
│ └── www.dol.gov
│ └── index.html
└── 19961223193614
└── www.dol.gov
└── index.html
Oder, um nur die URLs aller archivierten Schnappschüsse auszugeben:
waybackpack http://www.dol.gov/ --list
pip install waybackpack
usage: waybackpack [-h] [--version] (-d DIR | --list) [--raw] [--root ROOT]
[--from-date FROM_DATE] [--to-date TO_DATE]
[--user-agent USER_AGENT] [--follow-redirects]
[--uniques-only] [--collapse COLLAPSE] [--ignore-errors]
[--max-retries MAX_RETRIES] [--no-clobber] [--quiet]
[--progress] [--delay DELAY] [--delay-retry DELAY_RETRY]
url
positional arguments:
url The URL of the resource you want to download.
options:
-h, --help show this help message and exit
--version show program's version number and exit
-d DIR, --dir DIR Directory to save the files. Will create this
directory if it doesn't already exist.
--list Instead of downloading the files, only print the list
of snapshots.
--raw Fetch file in its original state, without any
processing by the Wayback Machine or waybackpack.
--root ROOT The root URL from which to serve snapshotted
resources. Default: 'https://web.archive.org'
--from-date FROM_DATE
Timestamp-string indicating the earliest snapshot to
download. Should take the format YYYYMMDDhhss, though
you can omit as many of the trailing digits as you
like. E.g., '201501' is valid.
--to-date TO_DATE Timestamp-string indicating the latest snapshot to
download. Should take the format YYYYMMDDhhss, though
you can omit as many of the trailing digits as you
like. E.g., '201604' is valid.
--user-agent USER_AGENT
The User-Agent header to send along with your requests
to the Wayback Machine. If possible, please include
the phrase 'waybackpack' and your email address. That
way, if you're battering their servers, they know who
to contact. Default: 'waybackpack'.
--follow-redirects Follow redirects.
--uniques-only Download only the first version of duplicate files.
--collapse COLLAPSE An archive.org `collapse` parameter. E.g., --collapse
timestamp:6 collapses results to at most one per
month. See here: https://github.com/internetarchive/wa
yback/blob/master/wayback-cdx-
server/README.md#collapsing
--ignore-errors Don't crash on non-HTTP errors e.g., the requests
library's ChunkedEncodingError. Instead, log error and
continue. See here:
https://github.com/jsvine/waybackpack/issues/19
--max-retries MAX_RETRIES
How many times to try accessing content with 4XX or
5XX status code before skipping?
--no-clobber If a file is already present (and >0 filesize), don't
download it again.
--quiet Don't log progress to stderr.
--progress Print a progress bar. Mutes the default logging.
Requires `tqdm` to be installed.
--delay DELAY Sleep X seconds between each fetch.
--delay-retry DELAY_RETRY
Sleep X seconds between each post-error retry.
Waybackpack ist in reinem Python geschrieben, hängt nur von requests ab und sollte überall dort funktionieren, wo Python funktioniert. Erfordert Python 3.3+.
Vielen Dank an die folgenden Benutzer für das Melden von Fehlern, das Korrigieren von Tippfehlern und das Vorschlagen nützlicher Funktionen: