Official comms thread for wearethene.ws
Use this thread to ask questions or discuss about wearethene.ws and we will answer unless you are being a tard or a clown.
BO if you don't want this thread here feel free to delete. I made it here because 8kun/CM aren't fulfilling new board creation requests right now and this board is widely recognized as trustworthy.
Verification
Go to https://wearethene.ws/about which has a link back to this thread now:
>If you'd like to get in touch with the maintainers of this site, you can post on the /comms/ board on 8kun here: https://8kun.top/comms/res/7366.html
Thank you Rusty! o7
Thanks for your service baker, and welcome. Without you none of this would be possible at all!
>My question is, sometimes notables seem to get truncated, for instance if a baker posts #NNNN Baker Change, or embedds new baker within the notable bun does this cause notables to be truncated?
When people add the "baker change" lines to the notables this shouldn't cause an issue. Be careful not to add any blank lines in the middle of a notables bun though.
Splitting notables across multiple posts is not currently supported, but this is something we can probably fix pretty easily.
One thing that does cause issues sometimes is splitting a notable across multiple lines where only one line has the notable title. It's better for our scraper if long notables are all on one line, or if each line is tagged with "Notable title (pt1)" and "Notable title (pt2)". Long notables can make for a good, article-like read, so at least for wearethene.ws having a single notable with lots of posts isn't a problem.
Whatever happens, one reason we started this thread is so that bakers and anons can call out any issues, things like this are pretty easy for us to clean up but we don't always see them. So please feel free to post here if you see anything, something like "bread number 12345 is missing a few notables", and we'll take a look. Over time this will help show us whether we need to change something on the site.
>Have you thought about bit bucket, or git and opening development to the community (Language?)?
More anon contributions would be great, but exposing the site's code publicly might make us vulnerable to doxxing or other unpleasantness. That is something we will keep thinking about. Maybe things will change after MARCH MADNESS is over.
>Splitting notables across multiple posts is not currently supported
Here's what this means, in case it wasn't clear:
#12345>>8888888 some notable>>8888889 another notable--- POST ENDS ------ NEW POST STARTS ---#12346>>8989898 new notable>>8989899 super notable
With the way the site works right now, the #12346 notables will not be picked up from that bread because they were posted in a separate post. This usually isn't a problem because the same set of notables will be posted in a few breads in a row, and this is probably something we can change if bakers want to keep doing this.
We also look every few days and make sure there aren't any breads missing or numbered incorrectly.
>I know you guise don't like buns
Don't worry about this, one enhancement we've considered is making it possible to see pics/videos on posts that expand as part of a bun. The only drawback would be that wearethene.ws won't save the images so they would disappear from public view when they disappear from 8kun.
>So i will put the two "key" posts first, followed by the bun link:
This is a good strategy also.
>If you have anything to share about how bakers can supports wearethene.ws in their notes–and you want bakers to see it soon–reply in the latest bakers live class thread.
The biggest thing is get the fking bread numbers right in the titles and the notable posts!!! Kek! Mistakes happen so triple-check the numbers.
>not sure i understand the example in >>7984
Right now when the scraper looks at a new bread, it expects all notables for past breads to be in the same post. If previous notables are posted in two separate posts then only one of those will be picked up.
>hmmm….that applies to my baking style, because i sometimes cross-reference pb posts, to make it easier for diggers to see the context
No, pb posts in notables are fine.
>which post gets picked up, btw? the first or second?
the first. I don't think I explained clearly though, I'll link to a real example next time I see this.
>Triple checking the numbers–you mean, like when you take over a fast bake @600 and then Q posts?
ha, ok. we'll keep cleaning these up when we see them, your efforts are appreciated anyway.
We're happy to help, patriot. Remember to spread the word on social media.
RSS feed is on the list of planned enhancements for wearethene.ws.
Welcome newfag. Here are the steps for posting on the Q board correctly.
Go to https://8kun.top/qresearch/index.html
Find the latest "General" thread (Ctrl+F for General)
Click the title to load that thread
Post a reply there instead of making a new thread. (If the thread already has more than 750 posts then that means you picked an old thread and you need to go back to the index page and try again.)
At this moment the latest General thread is #11488. You're looking for a similar thread but the number will be higher. Click on the thread title to load the thread - see attached screenshot.
Please don't post your name on the boards, and get a protonmail.com or other disposable email address.
It's also probably better to handle inquiries in public, just ask right here in this thread.
For wearethene.ws, anywhere from about 5 to 15 minutes. As long as the bread is correctly formatted (standard title format, bread number in title is correct, bread numbers in notables are correct, etc).
If something went wrong then those notables might appear when the next valid bread is baked, or when we check the site and clean up any errors (at least once every few days).
You're posting in the wrong place. See instructions here: >>11704
>we know not what we do
kek! welcome to the boards.
Hello highest ranking anon, no disrespect to newfags intended. Just trying to help others understand how the site is organized because it's not really obvious at first glance.
8kun has various boards (this one is called /comms/ which is for "meta-discussion" but not really for research itself) and each board has various threads (this one is for discussion about wearethene.ws)
to post research or something that other anons might want to have a look at, go to the /qresearch/ board and post on one of the "Q Research General" threads. There are about 10-20 of these threads per day, so >>11704 describes how to get there.
Hi bakerfren
1.
notables are scraped from the dough at the top of each new bread
2.
correct bread title format
correct bread numbers in title and in dough. the bread number at the top of each bun in the dough matters, the one at the bottom is mostly ignored.
there are a few other things but the bread numbers and title are the most important.
(the scraper tries to correct for some common errors, but this isn't always possible, so we'll come back eventually and clean things up manually)
3.
not really possible at the moment. the wearethene.ws site only works because bakers have an established way of deciding what isn't notable and what isn't.
we could probably come up with a way for bakers to tell the site to expand certain buns, but it would be messy (lots of duplicates, and the baking process shouldn't really have to change just for wearethene.ws, it worked just fine before the site existed)
you can already expand buns on the site by clicking on their individual posts, so another way to handle this would be to improve those features, like letting you see images/videos in posts after you expand them
hope that helps. any other questions, we'll be here
anons and bakers were here long before wearethene.ws, so generally the site should adapt to baking changes, not the other way around
as an example, when FJ tried to axe the baking process altogether, we moved the scraper over to >>>/qrb/ and processed the qresearch notables from there
that sucked kek, but we kept it online
>need a refresher on buns, they don't show all the posts on we are the news, is that correct
yes
>in this way anons could have their own buns without it being considered "notable" by all
not sure I understood this part? anons can post whatever they want as part of a thread, but it won't show up on wearethene.ws unless a baker puts it in notables
I think making a bun notable is not usually a great idea anyway, it works fine for the "anon bun" and "other news roundup" notables but not really for other stuff
as an example, there was a notable in #11798 that just linked to >>>/qresearch/9218593
the notables line looked like this:
> >>9218593 Do our health experts admit to colluding with WHO and China?
even for anons reading the notables on the board, it would have been much better to just list those 6 posts directly in the notable, like this:
> >>9218168, >>9218194, >>9218222, >>9218239, >>9218260, >>9218273 Do our health experts admit to colluding with WHO and China?
so we changed it on the site: https://wearethene.ws/notable/105652
Hi,
>how are you currently identifying general breads?
the first part is a simple regex against the thread title:
/Q\sResearch\sGeneral\s#\s[\d,]+/i
this can and will fail, the most common reason is if bakers make a mistake in the bread number or the title
so once every few days we look at what the scraper has caught and clean everything up manually
the notables parsing is also somewhat self-correcting since notables for bread #1000 usually stay around in the dough until #1003 or so. that gives bakers a chance to fix mistakes in the next bread and then have the notables from previous breads picked up.
>Baker is identified by the very first post of a bread. I will also go backwards from 1 general to the previous to figure out baker changes and mark posts accordingly.
I wouldn't recommend this. there is a loose process for baker handoffs but it's for humans, not computers, so it's not precise enough to be parsed. there's also no way to link users together across different threads since IDs change with every thread. there is all kinds of other stuff that can happen like 3 different bakers/notable collectors working on the same long bread during nightshift.
>The idea also was to archive full size pics + videos when a baker UID touched the posts. Baker is identified by the very first post of a bread.
probably the best you will be able to do using this approach is to make it work for about 60-75% of breads, again due to handoffs.
the approach wearethene.ws takes instead is to validate each new bread and then archive images from the posts that were quoted in the notables section. this probably works for about 95% of breads.
our site doesn't archive videos, PDFs, or animated gifs due to space requirements so that would definitely be valuable.
there aren't really any tricks or standards in general, just "whatever works". wearethene.ws only scrapes the HTML, I seem to remember some issues with the JSON threads list especially being out of sync with the rest of the board, but I don't have any definite info on that.
if there are any API endpoints wearethene.ws can provide that would make your projects easier let us know and we'll see if it's feasible. you're also welcome to scrape the HTML of course.
Just added:
https://wearethene.ws/rss
Let's get a bit of testing on that here, and then I'll add a link to the site.
If you have any issues then please post the name and platform of your feed reader and a screenshot. (No personally identifying info)
nice, site looks good
a couple of quick suggestions:
add some space or maybe a horizontal line in between different news stories to make it easier to tell when one ends and the next one begins
at the bottom maybe change "Follow Us" to "Follow Patriots" or something like that, "Follow Us" makes it sound like those are the people behind the site
anyway thanks for keeping the source for your feed articles in there, appreciate that
I just added qbrief.com to https://wearethene.ws/about
also, what's up with the image that says "sponsored ad" in the top right corner?
Thanks anon, updated (see the bottom section of https://wearethene.ws/notable/109472)
As it says on the About page of the site, wearethene.ws is not trying to be an objective source of truth. Think of it as a different perspective on current events, and you still have to think for yourself.
Still, anyone is welcome to point out any issues with articles here, and we will take a look.
Nothing is taken down, the site is intended to be a permanent archive.
Coming Soon (tm)
Hi anon, we've been thinking about this some, but as you say, there is a lot to consider.
the easiest way for us to do this securely is to use an external provider to store the comments data.
https://utteranc.es/ is one example. would you make a github account to use this?
github is owned by microsoft and I don't know their stance on censorship, so we would take a backup of all data regularly.
also this should go without saying but as far as I'm concerned you should be using a good, privacy-focused vpn for all of these discussions including 8kun.
open to other suggestions for comment providers, but disqus in particular is pretty bad.
Hi AAA, this is good work.
Have you looked at sending archive requests to archive.today also? It is probably more reliable with the DDoS protection issues.
I think the only way to get a reliable, full archive is by 100% custom code that parses the threads looking for new posts, but that is a good bit more work.
>Do you do the same, or do you even parse the HTML instead? I would like to use the JSON data.
wearethene.ws just parses the thread HTML. the JSON seemed to be pretty unreliable when we were making the scraper.
>how often do you contact 8kun for fresh data? Once every hour?
basically constantly looking at all open threads for fresh posts.
>The problem is now that my upload speed is really really limited
yes at some point you need outside hosting and a server for things like this.
If you append a cache busting string to the end of the URL then that issue gets better. That might be enough to make the JSON data pretty reliable, but if it is entirely missing for a thread then there's not much you can do.
>Why can't CodeMonkey fix this?
>Did anyone contact him over this?
admin@isitwetyet.com
hope you have better luck than I did
kek
just added the RSS discovery tag a few minutes ago. you must have been seeing different behavior based on how you accessed the feed or something.
See also:
>Here are the steps for posting on the Q board correctly.
>- Go to https://8kun.top/qresearch/index.html
>- Find the latest "General" thread (Ctrl+F for "General" or "Edition")
>- Click the title to load that thread
>- Post a reply there instead of making a new thread. (If the thread already has more than 750 posts then that means you picked an old thread and you need to go back to the index page and try again.)
Our hosting provider is receiving large ddos attacks. Site availability may be intermittent until that is resolved.
wearethene.ws does not editorialize content, pretty much whatever anons decide to post on the main board (in compliance with US laws) and bakers mark as notable will show up on the site.
The verification scheme that >>45334 suggests seems like a pretty good idea to me. Once you can prove that you are really associated with that account, we can go over to the other board and ask anons not to make posts related to this account "notable".
Fair warning, this may backfire and get you even more attention. Personally I suggest that you just "enjoy the show" and ignore comments that don't have to do with your mission.
Thanks for reporting this. There was a bug in the site's code that was making it skip notables that start with certain combinations of letters, and "Act" just happened to be one of these. This is fixed and your notable has been added where it should be: https://wearethene.ws/bread/8kun_qresearch/16540 and https://wearethene.ws/notable/206595
Absolutely awful and very much deserves to be notable.
Technical details - The issue was that the site should skip notable lines like this:
> >>12345 #678 final bun
and also like this:
> >>12345 #678a continued
and also this where "ct" has showed up as an abbreviation of "continued" before:
> >>12345 #678a ct
The code that skips these notables was not precise enough so it skipped anything starting with "act" (and a few other combinations of letters).
The Babylon Bee is often closer to reality than the mainstream media these days, which makes it even more hilarious.