Privacy Problem: Address Bar Hide Query

<hidden> anonymous
Created: 3 years and 11 months ago
Under Settings - Privacy, DDG offers

Address Bar - Hide Query
Whether search queries will appear in your address bar (GET vs POST requests).

POST queries are an important privacy enabler because UK law demands that ISPs keep a record of the URLs that people visit. Any search query typed into ddg and fetched using GET is permanently logged and can be searched by police and security services by negotiating with the ISP, without letting the user know.

When the search term is transferred using POST over https, the ISP has no access to the search term - they can't read it without cracking the https - so the search term remains private. So hiding the search term from the address bar is a major privacy enhancement, in the UK at least.

Unfortunately, DDG gives the game away. I used Chrome - view - developer tools - network to look at the network traffic when I typed "hello" into the ddg home page. DDG does indeed send the query over POST.

Code:
Remote Address:46.51.197.88:443
Request URL:https://duckduckgo.com/
Request Method:POST
Status Code:200 OK
Request Headers
:host:duckduckgo.com
:method:POST
:path:/
:scheme:https
:version:HTTP/1.1
accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
accept-encoding:gzip,deflate,sdch
accept-language:en-US,en;q=0.8
cache-control:max-age=0
content-length:7
content-type:application/x-www-form-urlencoded
cookie:g=p
origin:https://duckduckgo.com
referer:https://duckduckgo.com/
user-agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2005.1 Safari/537.36
Form Dataview sourceview URL encoded
q:hello


However, before that, DDG sends data with every keystroke, with a GET:

Code:
Remote Address:46.51.197.88:443
Request URL:https://duckduckgo.com/ac/?callback=jQuery110209023988840635866_1400658636352&q=hello&_=1400658636358
Request Method:GET
Status Code:200 OK
Request Headers
:host:duckduckgo.com
:method:GET
:path:/ac/?callback=jQuery110209023988840635866_1400658636352&q=hello&_=1400658636358
:scheme:https
:version:HTTP/1.1
accept:*/*
accept-encoding:gzip,deflate,sdch
accept-language:en-US,en;q=0.8
cookie:g=p
referer:https://duckduckgo.com/
user-agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2005.1 Safari/537.36
Query String Parametersview sourceview URL encoded
callback:jQuery110209023988840635866_1400658636352
q:hello
_:1400658636358


And that callback=jQuery110209023988840635866_1400658636352&q=hello contains the search term and must by law be retained by the ISP and must be provided to authorities on legal request.

At the very least, DDG should point out that the hide-request feature does not provide secure searches. Preferably, all data, including individual keystrokes, should be sent by POST, not GET.

This forum has been archived

Thank you all for the many comments, questions and suggestions. Particular thanks go to user x.15a2 for constantly monitoring, replying and helping so many users here. To continue these discussions, please head over to the DuckDuckGo subreddit.


rocketpenguin23
That can be fixed by going to settings > Auto-suggest and toggle to Off.
However, I do think this is a flaw and should be fixed.
posted by rocketpenguin23 3 years and 11 months ago Link
anonymous
Turning off auto-suggest does stop the per-keystroke GET operations. The search term is passed to DDG via POST, securely. But then DDG returns a page which causes all sorts of GET fetches from my browser containing the search term. Here they are:

Code:
https://duckduckgo.com/ (POST query)
...
https://duckduckgo.com/g1105.js
https://duckduckgo.com/d.js?q=hello&t=D&l=us-en&p=1&s=0
https://duckduckgo.com/y.js?s=1&q=hello&l=us-en
https://duckduckgo.com/y.js?x=1&q=hello&l=us-en&safe=1
...
https://duckduckgo.com/m.js?q=hello&t=D&cb=ddg_spice_amazon
https://duckduckgo.com/js/spice/dictionary/definition/hello
https://icons.duckduckgo.com/ip/www.hellomagazine.com.ico
https://icons.duckduckgo.com/ip/hello.com.ico
...
https://duckduckgo.com/js/spice/dictionary/hyphenation/hello
https://duckduckgo.com/js/spice/dictionary/pronunciation/hello

...


The bottom ones there are icons for websites returned in the search results - giving away my search results. I have to use settings - result - Site Icons - Off too. But when I do that, I still get:

Code:
https://duckduckgo.com/d.js?q=hello&t=D&l=us-en&p=1&s=0
https://duckduckgo.com/y.js?s=1&q=hello&l=us-en
https://duckduckgo.com/y.js?x=1&q=hello&l=us-en&safe=1
https://duckduckgo.com/js/spice/dictionary/definition/hello


That looks to me to be a serious security issue. Has nobody at DDG has considered the effect all these extra features will have on data privacy? It's all right saying DDG doesn't track us, but what of everyone else?
posted by <hidden> • 3 years and 11 months ago Link
anonymous
By contrast, typing "hello" into the Startpage search engine - which isn't as good, because it doesn't have !g - sends the query and gets the result by secure POST by default, and the search term definitely does not appear in any of the URLs used to fetch the page decoration.

It's not so pretty as DDG, but I think DDG has really missed a trick with the security here.
posted by <hidden> • 3 years and 11 months ago Link
anonymous
Well, this is disconcerting. I really would like one of the DDG representatives active on this forum to address this. This is a big deal for people in the UK.
posted by <hidden> • 3 years and 11 months ago Link
anonymous
Seriously, could we get a "yes, that does seem to be a problem. We'll look into this and let you know" or something, at least? If my ISP can track everything I search for, what exactly did I gain by leaving Google and adopting DDG? Nothing much, it seems. If this isn't resolved I'm going to StartPage.
posted by <hidden> • 3 years and 11 months ago Link
yegg
I think there is some misunderstanding between GET and POST. As long as you are over an https (encrypted) connection than all headers are encrypted and cannot be seen by an ISP or anyone in between you and us.

https://stackoverflow.com/questions/1876...
posted by yegg Staff3 years and 11 months ago Link
Moollaza
To further clarify what @yegg is saying, because the HTTPS GET requests are encrypted, it means your ISP cannot see the URL you've requested. They will only see a request has been made to the hostname, "duckduckgo.com".

http://stackoverflow.com/questions/88581...
posted by Moollaza Staff3 years and 11 months ago Link
anonymous
Thanks for the responses, and for clarifying this issue. I'd be curious to see if the person who originally posted this topic is satisfied with the level of security you've described, as they seem to know more about this issue than I do.
posted by <hidden> • 3 years and 11 months ago Link
anonymous
My apologies to all; this is the OP, and I have been on an island with no internet access (!) for nearly three weeks. I was indeed mistaken. https: does encrypt the query string. Thanks for pointing this out. Whether this is a complete solution, I'll leave for the future.
posted by <hidden> • 3 years and 10 months ago Link