guix992

The hard part is already solved, you don't even have to crawl the web to build the index. There is already a periodically refreshed index of the web that you can download: commoncrawl.org

Now someone just needs to configure, Apache Lucene as a proper docker image that can consume this index.

Thanks for your reply - it was a pretty insightful rebuttal.

Do you think in the future it might be possible for say Flutter to:

1. Refactor flutter into a layout engine and gui widgets

2. Provide the layout engine as an interface on all desktop OSes. Something like a standardized syscall like interface.

3. Anyone can implement a new GUI toolkit to this layout engine interface.

4. For desktop OSes, the number of styling options is less than full blown CSS capabilities(in HTML dom) and these options get compiled out, making execution of native apps GUI slightly faster than DOM apps. Basically, desktop GUI styling is a subset of DOM CSS styling.

5. We use a JSX like templating language to specify UI that either gets compiled to HTML or generates native GUI widgets.

Not sure all of my thoughts are 100% fleshed out but does this sound feasible in future?

I don't believe that is a good enough limitation in this day and age of docker and kubernetes.

You take the search engine docker image and keep scaling the number of nodes that are running it in the cluster till your search result time is fast enough for you.

What's been a shame is that there is still no open source Search Engine despite this being a "solved" problem. Like not even something like there is a docker image that you throw at your cluster that gets you faster and faster search results. That's the real shame.

We should've commidified the core of search engine by now with programmatic and API access as commonplace and yet here we are where search engine software is still dominated by proprietary services.

I don't understand why we have to keep reinventing GUI toolkits and layout engines over and over again.

Why can't this domain be smarter like Compiler world and build pluggable components? So something like, a react-engine that renders the recently changed display node. The react-engine is offered as an API/library layer, it could be run in browser where it works with DOM, it could be run in a desktop where it works with GUI nodes of native GUI toolkit.

So now going from each OS to another, you don't have to learn gazillion different ways to specify the layout. The type of nodes displayed on your GUI app changes from each environment to another. So you get to have the same programming techniques and you can get a web-app or a desktop app.

Does it make sense to have a common rendering engine library that is common to both browsers and GUI desktops?

So something like a react rendering engine but the type of nodes change when you use it in different environments:

1. Using it on desktop for GUI: The nodes are like the native GUI elements for that OS

2. Using it on the web: the nodes are whatever HTML DOM nodes that have to be rendered