I think as the community grows, more search engines will start including us!
You must log in or register to comment.
I made a crude lemmy scanner, but as I’m using the “http api” I’m a bit stuck at servers and instances. Almost 1800 servers and IIRC (I’m on my phone) some 40.000 communities. Yay!
I know the official api docs, but I don’t know how to really use it, just send a json to some server? Would love to have an easy example and I’ll go from there (family, work, art, and other gets in the way).
The goal would be to figure out how to make a search engine without storing the whole lemmyverse locally.
FOSS obviously.
Sounds interesting, what’s the size of lemmy stored locally with only text?
Should be less than a TB.
I don’t know as the (cURL http) API I’m using is confusing for messages so I only have the servers & communities.
If I get it up and running I’ll tell :-D
Kagi has a “Fediverse forums” lens now as well.
Here’s the link for anyone who wants a quick check - https://search.brave.com/search?q=mastodon+android+clients
Odd that it’s an obscure Basque instance though. Wonder why it shows that instead of something like world or ml.
Search engines currently struggle with the concept of federated posts. My guess is that instead of finding a post’s home instance, they accept the first mirror of that post and discard copies from all other instances through deduplication.
The search engine bots are absolutely powerhouse-obnoxious in how many requests they make, and there a ton of them, and Lemmy’s not real optimized to cope with the load – most big instances block all bot traffic simply as a matter of server survival as a result of that. So I would expect not to see any of them in search results any time in the near future.
why not use vivaldi if you have to use a chromium-based browser
It’s firefox (well, librewolf) + vivaldi is closed-source.
Fuck you spez!
yeah of all the names to choose…
It’s a little disingenuous so say that “Vivaldi is closed source” and leave it at that. The vast majority of their browser is built on open-source code:
https://vivaldi.com/blog/technology/why-isnt-vivaldi-browser-open-source/
Only a small portion (~5%) of their bespoke UI code is closed. The vast majority of their source is published.
Obviously whether the small portion of the code they withhold is important to you is a matter of your own to decide, but I feel this was important to clarify.
Well that part is essentially just chromium so that’s why ‘vivaldi’ (as in what makes slightly different being its ui) is essentially closed source. But yeah that distinction is there.
Why Vivaldi?
Because it’s fucking awesome. Here’s why I used* to use it:
- Built in email client
- Built in RSS reader.
- Themes and customisable UI (vertical tabs, bottom tags, tab previews, tab stacking)
- Built in ad blocker
- Sidebar with Mastodon, RSS, mail, translate, etc
*I use Floorp now cos I wanna support Mozilla/Firefox by upping the browser share numbers but Vivaldi is the best Chromium-based browser by far.
And people bash brave for being bloated.
Horses for courses.
Why care which browser they use?
This is exactly what comes to mind when some fedi people want to block all the search engines.
Good answers from real people are what drove everyone to start appending “Reddit” to the end of searches. The rest of the results are SEO trash.
I’d like to see a lot of that traffic move to lemmy and other fedi stuff and I think letting indexers do their thing is positive.
.eus.
.eus is the top-level domain for the Basque language. The abbreviation eus comes from the Basque endonym euskara, meaning “Basque language”.
TIL
it would probablyhelop if the search prompt is not written by a dyslexic… shakes head
dude you write verbs like ‘shakes head’ in your comments.
that’s not a verb, that’s an emote. Considering you chose the dumbest possible username for lemmy, I am not surprised you wouldn’t understand that.
Also: There’s a substantial difference to you searching for “anroid” clients and making a post based on those search results.
Sorry, unlike you, I have a life. Can’t argue over this stupid shit.
Irony that you could drape over yourself and go to sleep with
My excuse being the lemmy android app message editor being bugged, getting out of sync when deleting characters.
I can’t wait until we can start appending “lemmy” to our search strings in order to find useful results.
Is there a “user reviews” community?
Kagi does it already.
!Lemmy [stuff]
Gets you stuff from all over Lemmy
yeah but its, like, 5 bucks a month /:
That’s why all the top results on every search are actual results, instead of a bunch of ads.
Totally worth it.I’m struggling with kagi because I do notike my searches tied to an account/profile (a paid CC and name and address etc.)
I like kagi but this is making me want to cancel my membership. Thoughts?
They don’t keep a search history. Your searches aren’t tied to anything, because they aren’t even saved. They don’t have any reason to save them since they aren’t selling any targeted ads.
It’s always the same story: are confident are you that this is true?
My confidence is inversely proportional to any evidence you have to the contrary.
This was my understanding as well. It’s the point of paying. However , it is just “trust me bro” right? It’s not like they’ve been audited for not storing user data? Even just basic logging too could hold user info. Honestly no idea with paid search engines, I know so little.
Even an audit is really just someone else saying “trust me bro”. You have some level of trust.
What incentive would they have to betray your trust?
deleted by creator
SearXNG has a the same functionality as well for free:
You can self-host it or use one of the publicly available instances.
big fediverse
I wasn’t able to find an option to use Lemmy search an several public instances. Do you know an instance that supports this?
I am self hosting my version so that is where my screenshot came from.
If I were looking for a public instance to use that has this functionality, I’d look to some of the European hosts as they are less likely to use Reddit as their de facto platform relative to Americans
Does self-hosting with those extensions require a lot of system resources?
Not in my experience. I am running a docker container and it is one of +10 containers running on my server which is basically a laptop from a few years ago. So far no issues.
That’s great! I had no idea you could specifically search for lemmy content in search engines (albeit niche).
I’m seeing Lemmy posts on Google results as well.
I see Lemmy on Google too. It surprised me at first but I like that it get indexed.
Fuck Brave and fucks its owner Brendan Eich.
Don’t promote that despicable company in any way.
Wait so the same guy owns brave is now firefoxes CEO?
No, the opposite
no. the old ceo owns brave.
And new CEO fired more devs. Again.
nuh uh
While I agree, but i tried Brave Search for a few weeks, and it seemed pretty good.
I wouldnt use Brave Browser, because Vivaldi is there (if you need Chromium.) But for a search engine which is free? DuckDuckGo maybe?
P.S. Kagi search is pretty good if you are willing to pay.
DDG is so bad in results though it makes not using a search engine seem like a solid alternative. I remember that research from a few months ago where they compared Google, Bing and DDG to do statistical analysis of if and how much Google’s search results have been getting worse in recent years.
Result:
- Google did get worse.
- However, Bing and DDG got worser, faster.
The real conclusion was that SEO spam has found ways to optimize that is no longer easy to exclude for search providers. Hence all search is getting shit pretty quickly.
A lot of people are complaining about Kagi using Brave as a backend but the alternatives aren’t much better. Both Google and Microsoft are BDS for example.
Why what’s up with them? Asking cuz I don’t know.
Tl;dr Brave’s ceo (founder?) has donated to legislation to abolish gay marriage in California.
Brave adds referral links to cryptocurrency urls.
And it is apparently collecting money for YouTubers and not paying them (I did not fully read that article myself)
has donated to legislation to abolish gay marriage in California.
What an absolute fuck-head
Tl;dr Brave’s ceo (founder?) has donated to legislation to abolish gay marriage in California.
Aight, found this. Apparently the twat donated 1000 dollars to some proposal in California that would ban gay marriage.
Brave adds referral links to cryptocurrency urls.
You know? Honestly, I don’t care about that. They’re free and open source. They have to make SOME money somehow. Referrals cost the seller rather than the customer. While this does count as an advertisement, it is still very low profile. So uk…I don’t really mind that.
Didn’t they also click ads when you clicked a link, invisibly in the background?
that’s pretty much adnauseum except it doesn’t prentend
I was reading another comment, apparently they were swapping their referral link in place of your own.
WAIT, WHAT?!!! That’s like cartoon level conning!
And I agree with you, if they were simply adding their referral link when you did not include one, nbd as long as that is going to support the free browser development. But if they were actually replacing your referral link with their own, that is some Mr. Burns shit for sure.
Brave adds referral links to cryptocurrency urls.
How does it even work?
Before displaying a page it searches the content and injects a referrer tag to the end of URLs that match certain things. This is done instantly.
E.g. If there’s a Binance URL on the page it’ll add something like
?ref=bravebrowserBinance will pick up that tag and give them money for any sign ups. Brave do this for any crypto services they have affiliate accounts with.
As a piece of software, nothing. It’s an open source browser, and has an added bonus of having many privacy settings on by default. Not even firefox can say the same, it comes with telemetry, pocket and whatnot out of the box.
But there are some fair criticisms about the company and its administration. For example, there was an incident years ago when you signed on a crypto exchange, it would swap the sign on link for their own referral link. They claimed this was an error and quickly patched it, but I don’t buy it.
You’ll quickly notice that a lot of people on lemmy passionately hate brave. So expect a strong bias and, as a result, truths but overblown, half truths and misinformation. Don’t ignore what they say but double check them.
Read the links of the comment you are replying to.
that’s not reasonable. provide an argument with supporting links, not links without an argument
