DuckDuckGo gets its results from over one hundred sources, including DuckDuckBot
(our own crawler), crowd-sourced sites
(like Wikipedia, which are stored in our own index), Yahoo! (through BOSS), Yandex, Yelp
, and Bing
. For any given search, there is usually a vertical search engine out there that does a better job at answering it than a general search engine. Our long-term goal is to get you information from that best source, ideally in instant answer form.
You will notice that the results of our sources and our results are often very different. That's because we have an intelligence layer on top that attempts to improve upon results provided by external sources. Not only do we try to pick the best source and use it in the best manner, but we also do other things like remove spam, re-rank, provide instant answers and add official sites on top, among other things.
While our indexes are getting bigger, we do not expect to be wholly independent from third-parties. Bing and Google each spend hundreds of millions of dollars a year crawling and indexing the deep Web. It costs so much that even big companies like Yahoo and Ask are giving up general crawling and indexing. Therefore, it seems silly to compete on crawling and, besides, we do not have the money to do so. Instead, we've focused on building a better search engine by concentrating on what we think are long-term value-adds -- having way more instant answers, way less spam, real privacy and a better overall search experience.