Archiving things online, 101

LACUNA

Administrator
The subject of this forum (general weirdness, crime, debauchery) is prone to things going down, often quite speedily. Hence, it is extremely important that original or unique things to the case be archived. Posting something relevant, anywhere, ESPECIALLY if it is a case that has the eye of the media, means it is VERY LIKELY TO BE DELETED. Even if not, link rot is common and links go down all the time. Hence: ARCHIVE EVERYTHING. If you need help it is fine to post about it first.

Internet archiving sites
An online archive hosted somewhere is more subject to censorship than one you have, but it is more accessible to others and not subject to your house burning down or something.

Wayback Machine / Internet Archive
Very useful and has many features. However, it takes things down upon request and does not archive some "offensive" sites. Archives can be wiped with little notice. If anything is especially significant, mirror it to other archive sites.

There is also a media section of the Internet Archive that is not a website archive. Some media is uploaded by archive.org itself and some is by users. User uploads are often taken down but often not because it is poorly monitored. This is a good place to post things like videos, images, that were archived, though not for permanent archiving.

There are browser extensions to make using it easier, just click and it archives: Chrome, Firefox

Archive.today
Aka archive.is, archive.ph, archive.md, it has many URLs it shuffles by demand.
In theory, you can download the archive in its original form from the download button. However, a lot of the time this does not work, for reasons I cannot discern. Usefully, you can transfer Wayback captures to archive.today with similar functionality

There are browser extensions to make using it easier, just click and it archives: Chrome, Firefox

Ghostarchive
Decent. Better at archiving videos than Internet Archive.

Megalodon.jp
In Japanese, but has the unique ability to capture some JavaScript-heavy or paywalled things the others do not.

Video archiving
yt-dlp
A very complete and useful video downloader used by a command line. Can download almost anything anywhere with ease. If you don't want to use it with a command line, I have heard this frontend is good: Parabolic

My command for downloading original uploads (where the source is the original copy) is:

yt-dlp --write-info-json --write-description --add-metadata --embed-metadata --cookies-from-browser [whatever your browser is, chrome, firefox, etc) --write-thumbnail --embed-thumbnail --write-subs --embed-subs --sub-langs all --write-playlist-metafiles --write-comments -o "%(title)s/%(title)s [%(id)s].%(ext)s"

For reuploads, it is:

yt-dlp --write-description --write-subs --embed-subs --sub-langs all --cookies-from-browser [whatever your browser is] -o "%(title)s/%(title)s [%(id)s].%(ext)s"

--cookies-from-browser makes it so, if your account is logged in on that browser, it will view it in-account, bypassing age verification. If the title is too long (this happens with twitter), replace the end with -o %(title).200B.%(ext)s

Other video downloading sites
There are a lot of garbage YouTube downloaders out there. They are all running yt-dlp on the backend and they are all TRASH. I guess they're useful if you're on a mobile device.

Image archiving
Gallery-dl
Another command line tool, very effectively downloads all the images for things like Instagram or any gallery upload site.

Downloading full websites
These give local copies.

HHTrack
Simple, very easy website downloader with a gui. Does not give archival grade copies but gives useful, accessible ones.

Wget
Command line tool to download anything, anywhere. Far more complicated but far more powerful. You can configure it to give archival grade copies.

Other guides
This is only a basic guide for use on this forum, so if one wants to go beyond this you will have to look elsewhere

Archiveteam Wiki
Much more in depth guides on archiving a wide variety of things.

Kiwi Farms - Archival Tools
More in-depth guide on how to archive/convert videos for archiving.

What Next?
Where do you upload the things? Mirroring to the Internet Archive media section is a good short term solution for things such as videos, or less moderated video sites. If they are links or images, link here.
 
Last edited:
Back
Top