Anonymous ID: e463c2 June 29, 2018, 7:36 a.m. No.1957045   🗄️.is 🔗kun   >>7139

>>1956925

Sweet.

Here's some background.

 

Solr will work if I can get over the index hurdle.

I've already got Solr working on my Q posts page. It's HELLA fast. My issue is in creating an index on all the breads. It does take a minute to create and so I need to have it prebuilt. Solr runs in a java/js env native. I'll play around with it again just to see if I can get a js worker thread to create the big bread index and then save that off so that the breadsearch can use it.

 

Check the Solr implementation I have running at qanon.news/Q

 

I just reworked it to function with a worker thread this week so I think that I'm nearly there.

Anonymous ID: e463c2 June 29, 2018, 8:06 a.m. No.1957322   🗄️.is 🔗kun   >>7365

>>1957139

Agree. The scraper is a C# dll and so has no knowledge of javascript or I'd do it there. A way of notifying js that there's a new bread would be magical - otherwise I think you are right, it has to be a timed thing. That wouldn't be too hard since I've got the workerthread habbening already. Add in some timeout code code and we're all set.

 

Here's the 2 big issues I've got to overcome:

1) Generate a prebuilt index for Solr - this will need to be done no matter what search engine we decide to use. I tried a straight text search in the API and it's useless. Takes forever.

2) Create an API/means of retrieving a list of posts from the API. I've got this for single breads/posts, so theoretically it wouldn't be hard to add in a list so all results can be returned at once. Earlier tests proved this would not work since everything ultimately is encoded in the URL which WILL cause URL length issues. Possibly doing it in js? Seems like it would be slow as fuck. The way I'm doing it in the Q post search is by dumping the allQPosts.json on the page and then using jquery to parse that. Solr then can filter that to provide the actual results - not feasible with the bigbreadsearch.

 

Keep brainstorming anon. There are no wrong answers.