UnHiddenMar 17, 2024

We're building a search engine to compete with DuckDuckGo. No JS, no WASM, no spying. Just a statically generated results page.

https://lemmy.world/post/13224202

We're building a search engine to compete with DuckDuckGo. No JS, no WASM, no spying. Just a statically generated results page. - Lemmy.World

We’re (a group of friends) building a search engine from scratch to compete with DuckDuckGo. It still needs a name and logo. Here’s some pictures (results not cherrypicked): https://imgur.com/a/eVeQKWB [https://imgur.com/a/eVeQKWB] Unique traits: - Written in pure Rust backend, HTML and CSS only on frontend - no JavaScript, PHP, SQL, etc… - Has a custom database, schema, engine, indexer, parser, and spider - Extensively themeable with CSS - theme submissions welcome - Only two crates used - TOML and Rocket (plus Rust’s standard library) - Homegrown index - not based on Google, Bing, Yandex, Baidu, or anything else - Pages are statically generated - super fast load times - If an onion link is available, an “Onion” button appears to the left of the clearnet URL - Easy to audit - No: JavaScript, WASM, etc… requests can be audited with F12 network tab - Works over Tor with strictest settings (official Tor hidden service address at the bottom of this post) - Allows for modifiers: hacker -news +youtube removes all results containing hacker news and only includes results that contain the word “youtube” - Optional tracker removal from results - on by default h No censorship - results are what they are (exception: underage material) - No ads in results - if we do ever have ads, they’ll be purely text in the bottom right corner, away from results, no media - Everything runs in memory, no user queries saved. - Would make Richard Stallman smile :) THIS IS A PRE-ALPHA PRODUCT, it will get much MUCH better over the coming months. The dataset in the temporary hidden service linked below does not do our algorithm justice, its there to prove our concept. Please don’t judge the technology until beta. Onion URL (hosted on my laptop since so many people asked for the link): ht6wt7cs7nbzn53tpcnliig6zrqyfuimoght2pkuyafz5lognv4uvmqd.onion

Show threadExtremeDullardMar 17, 2024

I applaud your efforts and I admire your idealism.

Unfortunately, the minute you get the bill from your internet provider, you’ll need to find a way to pay for it, and your good intentions will instantly dissolve in the murky realities of modern corporate surveillance capitalism.

But at least while you haven’t gotten your first bill, it’s refreshing to watch your enthusiasm.

Show threadsugar_in_your_teaMar 17, 2024

pay for it

I wonder what a distributed search engine would look like. Basically, the index would be sharded across user computers, and queries would hit some representative sample of that index. This means:

  • hosting costs are very low - just need a way to proxy requests to the network
  • search times should improve as more people use the service
  • no risk of the service logging anything - individual nodes don’t need to know who requested the data, just who to send the response to

My biggest concern is how to build the index, but if OP is willing to share that, I might start hacking on a distributed version.

Show threadgrueMar 17, 2024
Don’t start new; contribute to what already exists: en.wikipedia.org/wiki/YaCy
YaCy - Wikipedia

Show threadsugar_in_your_tea

Awesome! That’s pretty much exactly what I’m looking for, though I’m interested to see how easy it is limit certain peers to certain functions. Not everyone has resources to crawl and index pages, but a lot of people can store the index.

I’m interested in having client-side web storage, so you can participate in the network by just having the search page open (opt-in of course).

I’m honestly not actively working on it, but if OP provides the database and/or crawler, I’ll do some research on feasibility.

Mar 17, 2024 at 5:09pmWeb