Reporting malicious sites that pop up in many queries

Working on it

Temporary fix
Not a problem
Analyzing
Need more info
Solved
Working on it

in DuckDuckGo Forum • 1 year ago

Hi,

I've noticed that there is a malicious site that shows up in most search results and it's always the first result - the site is called cooga (dot) net.

It takes part of your query and creates a fake site for it, like yourquery-hq-13141(dot)cooga(dot)net.

Other sites that do this or a variant of this are f60s(dot)com, downarchive(dot)com, rapiddigger(dot)com, softmann(dot)com, rapidsharegfx(dot)com, esoft(dot)in...and these are the ones that popped up after a few queries, in the first two pages of results. But there are hundreds (if not thousands) of them.

These trash sites make up for the 90% of the results, in some cases. It is not acceptable. You guys are doing a great job with DDG and it's great for experienced users but it's still far from allowing newbies to browse the results safely. You ought to blacklist those sites and develop a valid algorithm for filtering the trash.

Replies(7)

Guest

Re: Reporting malicious sites that pop up in many queries

1 year ago

Found other sites that fit my previous description: fullrshare(dot)com, nexusddl(dot)com, downturk(dot)info, dailydown(dot)net, wz(dot)cz, ideamarketers(dot)com, tkshare(dot)com, bee(dot)pl

I have NOT visited all those sites, however due to their irrelevancy to the queries and to their titles, descriptions and URLs I can safely say that they are indeed malicious. All those sites also try to lure the users with warez or porn related keywords, such as torrent, rapidshare, download, ebook, pdf, mp3 and many others. The queries I submitted were not related to warez or pornography, another reason because those sites shouldn't be in the results.

Comment

Guest

Re: Reporting malicious sites that pop up in many queries

1 year ago

I've made some other queries for research purposes.

Searching for something with mediafire or rapidshare or megaupload at the end brings out those results instantly, you won't even have to go through many pages of results to stumble upon them.

Example:

http://duckduckgo.com/?q=tolkien+mediafire

Look at the first page of results...it's already full of malicious sites!

And I present to you the Holy Grail of trash results:

http://duckduckgo.com/?q=faulkner+rapidshare

Not even a single result is relevant and/or not malicious!

http://duckduckgo.com/?q=puzo+megaupload

Look at that!

And so on. I used the names of highly regarded writers just as examples but I bet those results will show up with any query of that kind.

There is a pattern to those sites:

They promise downloads
They often are Spanish, Czech, Russian or Polish
They employ fake queries
They often make references to pornography using words that aren't pornographic and can slip off filters (such as "18" or "Russian Teen", "Blonde"...all terms that popped up during my non-pornographic queries)

I've reported this behavior in the past (http://duck.co/topic/too-many-trash-sites-in-the-results-and-no-flagging-blocking-report-option) but I'm repeating myself for clarity's sake and because I care about DDG. As I said before, you don't have to be looking for downloads to have those sites in your results. I also provided you with many examples of malicious sites, I hope it'll be helpful.

Comment

yegg13

Re: Reporting malicious sites that pop up in many queries

1 year ago

All these sites are now banned. I also added them to the training set so when the next crawl starts it should do a better job of identifying them, though that will be a few months to take effect (and work through the crawl and spam index).

In the mean time, I can ban more sites if you want to keep identifying them. If it would be easier, I can create a github repository to help with this process.

Another possibility that I have been exploring is to automatically blacklist the sites with low ratings from WOT (that also have high confidence scores). If you turn on the WOT icons and then click on individual ones you can see the scores. I wonder what you think about that idea.

Comment

Guest

Re: Reporting malicious sites that pop up in many queries

1 year ago

(ugh, I got my post 404'd! Trying to rewrite it as accurately as possible!)

They are both great ideas.

WOT should take care of a lot of thing in the short run (the first two-three pages of results, those that usually contain the most relevant results), doubling the effectiveness of most queries or even tripling it. The biggest problem DDG has is the trash results; once you remove them or at least the most intrusive part of them, the results will more likely be presented to the users in the first or second page.

And I'll be more than glad to contribute if you decide to open a repository for reporting bad sites. I will help as much as I can and so will other DDG users, I think. I'm doing all my research with the filter on but I suppose the situation would be even worse with it turned off. That's why it's important to get other DDG users to contribute...I guess you're already aware that DDG has a bit of a cult status among lots of technically competent people, you'll probably have a lot of people reporting. But I'd personally start the hand-picking after you implement WOT, otherwise all our reports will be made useless by it (since all the sites I reported probably already have a low rating). But in any case, I'll be more than glad to help.

Thanks for the fast answer!

rmardo

Re: Reporting malicious sites that pop up in many queries

1 year ago

is there anyway DDG could tap on the repository of avast! for malicious links? on unsuspecting links that appears in search results, avast! always prevented me from going to the site as it is malicious/malware. more effectively than WOT ratings

perhaps a partnership with avast!?

yegg13

Re: Reporting malicious sites that pop up in many queries

1 year ago

Thx--will look into avast.

Mike Dohe..

Re: Reporting malicious sites that pop up in many queries

1 year ago

You should consider matching domains against the Wikimedia Foundation's Meta-Wiki's spam blacklist. Their team of volunteers do an excellent job of monitoring spamming on WMF wikis, and the list of blacklisted regexes is of high quality. You can find info and ask for help via http://meta.wikimedia.org/wiki/WM:SPAM

Comment

Top Reply

Response title

Attachments

New to this Portal?