18 Nov

DuckDuckHack Q&A for contributors

We recently had our first Q&A session for contributors to DuckDuckGo, done over Slack (invites available). Below are the questions you asked and the answers provided by Zaahir. Follow @DuckDuckHack on Twitter to hear about future developer-related events.

I've been trying some Goodies (with test queries from github.com/duckduckgo/zeroclickinfo-goodies/blob/master/t/Game2048.t). Why aren't some of them working?

That particular Instant Answer (IA) isn’t live yet. It’s live on beta.duckduckgo.com though. It’s still a work in progress and we’ll be posting a task soon with the required updates to get it live. All live IAs can be seen here: duck.co/ia/ and all IAs being planned/developed can be seen in our developer pipeline. Anyone is free to try out those IAs on the beta server.

Are beta IAs auto-deployed from the repo?

Not quite — we manually deploy them to the beta server. New cheat sheet pull requests (PRs) are auto-deployed but others need to be manually verified before they get deployed. Our goal is to enable the community to be able to deploy PRs to Beta so they can test their own stuff, and share it with the community. We’ll provide training to devs, like our Community Leaders, to do so. They need to be able to ensure the code is safe to run.

I want to know more about Community Leaders. When is a contributor good enough to apply to become a Community Leader?

You can apply whenever you’d like to be a community leader — what we look for is someone who has been active and has done a good job. Someone we know has a solid grasp on Goodies and/or Spice (where most of the action is), who we think is capable of helping others and reviewing code. Eventually you can earn a commit bit, which gives you the ability to merge PRs as well.

How is it with DuckDuckGo in terms of languages other than English? I'm asking both in terms of developing Instant Answers and in a broader meaning. Are other markets prioritized?

That’s a great question. Right now, you may notice that all our Instant Answers have English triggers and provide English results. We’ve recently opened up some other wiki data for popular languages so users can see results from, for example, Japanese Wikipedia. Internationalizing our Instant Answers is definitely on our roadmap and it’s something we plan to do, but for now we’re asking that developers stick with English.

Can we check travis ci status from Codio before sending a PR? Is their any command to run the tests?

Yup, run duckpan test or prove -Ilib t/ from the root of the repository. ​duckpan test is just an alias for the latter. Also, if travis fails, you can always click through to see what failed. The travis test logs are all public.

What’s the basic difference between Spice and Goodies (without cheat sheets)? What I know is Spice is used for APIs and HTTP requests only.

Yup, that’s the difference. Before, only Spice were able to use JavaScript and as a result, templates. Now, Goodies can also use templates (from the Perl backend) and they can also load a JS file too, making them just about as powerful as Spice. Really, Spice should only be used when you need to use an API. If you don’t need one then a Goodie will work, for both a pure Perl or Perl + JS Instant Answer, like the in-progress 2048 game. We’re currently exploring Pure JS Spice and Goodies which means you’ll be able to ditch the Perl for things that don’t need it.

Why do Instant Answer info boxes (the i icon) show the developer name in some cases instead of their username? For example the Vim cheat sheet shows the full name and in others it shows the username.

It’s because of the metadata. In this case, it was an old Goodie where we imported the developer from the Perl code and we set both the username and display name. In all the new IAs we just grab the username from the pull request. Right now that’s set on a per IA basis though.

Is there any way to install the dependencies needed for developing IAs on my local machine? I'd prefer that over a Codio box.

There are a couple of alternative methods — DuckPAN locally or virtual machine. We have instructions for both methods.

Why Perl?

Our use of Perl gets quite a reaction sometimes! Gabriel, the founder, knew Perl so that's what he used when he created DuckDuckGo. It seems to have done well with scaling as search traffic has grown. There's a help page about our architecture here: duck.co/help/company/architecture

What is the long term vision? And is any structural change in the pipeline?

Long term vision? To provide an Instant Answer for every query.

Structural changes — we plan to launch JS-only Spice + Goodies and are currently investigating them.

Thanks. Do you think that would bring more people to collaborate?

We’re hoping so! The Perl component tends to scare some people off. Also, in many cases we simply don’t need the Perl so we might as well simplify things to a single file that handles everything.

Is the code that handles the JS and integrates with the existing code going to be open source and open to contributions as well? And are IAs (Instant Answers) the only way the community can contribute?

Parts of the JS-only stuff will be open source in the github.com/duckduckgo/duckduckgo repo — that’s the library that defines Instant Answers. Devs are welcome to contribute to IAs, DuckPAN, the duckduckgo/duckduckgo repo and our community platform. Our mobile apps are also open-source on GitHub (Android and iOS).

The templates and template helpers are technically open source, but we don’t have a way for devs to actually test that code locally (i.e. DuckPAN needs to load it from somewhere to use it). To be honest we’re not exactly soliciting devs to contribute to those, because they’re somewhat less “fun” to work with, and we haven’t really provided a great way for devs to test changes yet, but we’re slowly pushing more stuff out into the open.

If you have any great ideas we’re open to suggestions. For example one of our meetup organizers expressed interest in building a cheat sheet tool that would help less technical users create cheat sheets from a GUI.

Thanks for the answer! Keep the good work :)

Can we use an API that's not free to make an Instant Answer?

If there’s a free tier that has a reasonable limit of hundreds per day (or more) that's usually fine. ​If it’s entirely paid, we probably won’t use it, however I always suggest that developers reach out to the API maintainer and ask if they’d be willing to give us free access because we would showcase their info and brand name, including a link back to their site (ideally) above any ads or link results. This means they get top priority on our page, and that creates brand awareness. Many API’s have been willing to give us free access because of those reasons. If they’re not willing to budge, it’s best to look for alternatives. If alternatives don’t exist, or they’re considerably worse, then we can talk but so far we haven’t run into that.

Is there documentation on the $req object?

Not at the moment but there's an internal task to document it. It’s not something devs need often, but the code for it is open source so you could see how it get’s created: github.com/duckduckgo/duckduckgo/blob/master/lib/DDG/Request.pm

You could also p($req) and see the objects internals when you run duckpan query or duckpan server

Thanks for the questions!

1 Tweet

This blog has been archived

Thank you for reading and contributing lively discussion to our blog! Read more posts about online privacy on our new blog at spreadprivacy.com.

Good work for this ! I really enjoy to being there with you guys ! :)

posted by Halival • 2 years and 1 month ago Link