Okay, I hopefully finally fixed an issue with ArchiveToday, enforced archiving after the first archive was done. It's so weird, it seems they have some protections in place that you can only archive a website once, unless you use a web browser to do it.
I'm now archiving at 7:00 UTC, 11:00 UTC, 14:00 UTC, 17:00 UTC, 20:00 UTC, 1:00 UTC as well as 4:00 UTC.
WeAreTheNews-anon, how often do you contact 8kun for fresh data? Once every hour?
I could try to set my jobs close to yours.
On every job run, I'm now getting all JSON data for updated threads, going through these I'm archiving WEBMs + MP4s as fast as possible.
Plus I'm archiving threads when they reach 751 replies or the thread is in the lower 75% of the catalog.
I went through automatic uploading to anonfiles.com, but that's a problem because let's say a thread has 5 mp4s, and each is 15 megs, that's unlikely but whatever. The problem is now that my upload speed is really really limited, 100 megs takes ages and I looked up my ISP, only the very expensive contracts offer a higher upload speed. I guess that's intentionally so that people are unable to share movies and games.
Anyway so in the worst case my whole internet connection could fill up, and there would be nothing to do about it. I think I could at least still upload JSON data, but I'm not sure if that makes sense. JSON data is now archived on 2 archive websites.