1
0
mirror of https://github.com/DNSCrypt/dnscrypt-proxy.git synced 2024-12-26 00:02:34 +01:00
Commit Graph

1087 Commits

Author SHA1 Message Date
Frank Denis
0f7bd23b8a Simplify 2019-11-08 11:28:41 +01:00
William Elwood
4324a09fc9 Fix failing tests on Windows
To simulate failures opening a cache file, fixtures are written without the read permission bits.
Since Unix permission bits have no meaning on Windows, a slightly more complicated solution is required to achieve the same permissions.
Thankfully, there's a library to abstract that already.
2019-11-08 10:17:12 +01:00
William Elwood
77a4a3da90 Reduce indentation and long lines in test 2019-11-08 10:17:12 +01:00
William Elwood
0aea5f81ef Raise log level of a prefetch failure
This way it matches with the "loading from URL" info message and users with that log level aren't left with the false impression that it loaded fine when it really didn't.
2019-11-08 10:17:12 +01:00
William Elwood
5ed7b7c24f Reduce the chances of corrupting the cache
Write both parts of the cache to their temp files before renaming
Now only the 2nd rename failing can leave the cache in a bad state.
2019-11-08 10:17:12 +01:00
William Elwood
b6d11b4351 Parse source URLs sooner
URLs only need to be parsed once, after that they are always available to the download func.
2019-11-08 10:17:12 +01:00
William Elwood
bf28325b61 Enable tests for expired cache
If the cache is expired but then all downloads fail, the cache should be used.
2019-11-08 10:17:12 +01:00
William Elwood
38019866ca Move download loop, fix unnecessary cache reads
Previously, an expired cache would be read before trying each URL until a download completed.
By moving the download loop, the cache can be read once outside the loop.
2019-11-08 10:17:12 +01:00
William Elwood
f6f1a75884 Improve logging by keeping a Source's configured name on the struct 2019-11-08 10:17:12 +01:00
William Elwood
b697283309 Minor cleanup, mostly in tests 2019-11-08 10:17:12 +01:00
William Elwood
c0e34d1a9e Verify signature immediately after reading from cache or URL
This allows a large number of tests to be enabled and pass now that the behaviour is expected.
The main fix here is that a download with an invalid signature will always fall back on using a properly signed cache, no matter how old it is.
Additionally, downloads will never be written to the cache unless they are properly signed (both at startup and prefetching).
2019-11-08 10:17:12 +01:00
William Elwood
53d5b0f3cd Remove URLToPrefetch struct 2019-11-08 10:17:12 +01:00
William Elwood
a83ecf626b Move when from URLToPrefetch struct to refresh on Source struct 2019-11-08 10:17:12 +01:00
William Elwood
b29c70551e Refactor writing download to cache 2019-11-08 10:17:12 +01:00
William Elwood
082a4a5e01 Clarify how refresh_delay is intended to be used internally
Cache TTL is how old the cache can be at startup before trying to download an update immediately.
Prefetch delay is how long the prefetcher should wait between successful downloads of a source.
Previously, the refresh_delay configuration was used at startup as both cache TTL and prefetch delay, with subsequent prefetches using a hard-coded delay.
As discussed, refresh_delay is now only used for cache TTL, prefetch delay always uses the hard-coded delay.
2019-11-08 10:17:12 +01:00
William Elwood
190700e5ba Move cacheFile from URLToPrefetch to Source struct 2019-11-08 10:17:12 +01:00
William Elwood
0991749b19 Allow source URLs to contain query parameters
Previously when constructing the signature URL, the `.minisig` suffix was blindly appended to the string version of the source URL.
Now we take the parsed source URL, deep copy it (saves us parsing it twice), and append the `.minisig` suffix to the path component of the URL.
2019-11-08 10:17:12 +01:00
William Elwood
b2ecc45133 Treat each list and signature pair as a single unit
When a list fails to download, there's no point trying to download the signature.
Code duplication moved to where it's easier to refactor away.
Enabled a few more tests.
2019-11-08 10:17:12 +01:00
William Elwood
1e225dbb67 Alter source tests to cover entire prefetch algorithm and make it pass 2019-11-08 10:17:12 +01:00
William Elwood
7e73a26a2f Move most of the prefetching code into sources.go
The proxy shouldn't need to know how prefetching works, just that it needs to do it occasionally. Now the prefetching algorithm can be refactored without having to touch the proxy code.
2019-11-08 10:17:12 +01:00
William Elwood
78f2dead79 Move prefetch URLs onto Source struct
This is mostly in preparation for further refactoring, but does reduce the number of return values from `NewSource()` too.
2019-11-08 10:17:12 +01:00
William Elwood
4a792026eb Refactor cache reading to reduce number of return values 2019-11-08 10:17:12 +01:00
William Elwood
fe34d07b68 Refactor away some unnecessary type shuffling
Signatures in particular were read in from both cache and url as `[]byte`, converted to `string`, then back to `[]byte` to pass through to minisign.
Lists themselves will be converted to `string` by the parsing code anyway.
2019-11-08 10:17:12 +01:00
William Elwood
ad92be5b9c Refactor saving downloads to cache
Moved writing to happen immediately after the download to reduce duplicated code and number of return values from the download function.
2019-11-08 10:17:12 +01:00
William Elwood
4c156784c8 Refactor calculation of update delay when reading cache
Set the default delay once at the top instead of before every early return.
2019-11-08 10:17:12 +01:00
William Elwood
e818eeb800 Refactor reading a URL's content to own function
No longer shadows `url` package with variable of the same name.
2019-11-08 10:17:12 +01:00
William Elwood
d851c9eeb6 Refactor signature verification to own function
The cache is no longer destroyed whenever any signature verification fails.
The public key is stored on the Source struct for future use.
2019-11-08 10:17:12 +01:00
William Elwood
da0d7fe841 Fix various timing inconsistencies
When comparing times in tests, it's necessary to control the `now` value to ensure slow test runs don't fail incorrectly.
Both cache and download code had been using refreshDelay to set the next prefetch delay, which by default meant the 1st prefetch was 3 days after startup - this has now been corrected to match the 1 day expectation.
Enabling some of the cache tests revealed some other incorrect failures in the test that were also fixed.
2019-11-08 10:17:12 +01:00
William Elwood
03dea47130 Remove dead code paths
These paths were unreachable because XTransport.Get already checks the same conditions.
2019-11-08 10:17:12 +01:00
William Elwood
af0629856c Add unit tests for sources.go
Tests cover most of the cache and download related code paths and specify the expected result of various starting states and external failure modes.
Where the current code's behaviour doesn't match a test's expectations, the test is disabled and annotated with a TODO until it can be fixed.
Added dependency on `github.com/powerman/check` and ran `go mod vendor`.
2019-11-08 10:17:12 +01:00
William Elwood
503bfb877b go mod tidy
```console
$ go mod tidy -v
(snip)
unused github.com/agl/ed25519
```
Also add base .gitattributes file to normalize line endings in the repository across differing developer environments.
2019-11-08 10:17:12 +01:00
Frank Denis
e6a4a4ffda Update deps 2019-11-05 01:32:38 +01:00
Frank Denis
b76db70a6c Revert "cleanup: estimators: simplify blindAdjust"
This reverts commit c699e7bec4.
2019-11-05 01:16:22 +01:00
Frank Denis
17a675021e No one ever completes // TODO things 2019-11-05 01:10:57 +01:00
Frank Denis
da3f30871f Revert "fix: proxy: Trigger query logging plugins using defer"
This reverts commit fc9509a8c8.
2019-11-05 00:54:03 +01:00
Frank Denis
14862c2fc7 defer is slow and not worth it here 2019-11-05 00:37:46 +01:00
Frank Denis
316c5ca6b1 Don't return immediately on non-Windows system if netprobe_timout is -1
Fixes #1016
2019-11-04 17:14:31 +01:00
Frank Denis
9852a289f8 Increase the default cache size and minimum TTL 2019-11-03 17:31:41 +01:00
Frank Denis
e0c37f92fc Add a comment about why DoH addresses from stamps don't expire 2019-11-03 00:33:17 +01:00
Frank Denis
0f332c644d Set a minimum TTL when caching resolver IPs
Comcast having a 30 sec TTL is silly
2019-11-02 02:01:03 +01:00
Frank Denis
63ed3b4fef Update comment 2019-11-02 01:52:51 +01:00
Frank Denis
a84a789a8a Keep resolving if needed 2019-11-02 01:50:35 +01:00
Frank Denis
d932d5fdfc Inverse test 2019-11-02 01:20:28 +01:00
Frank Denis
6032c3b79b Add a grace TTL for expired cached IPs
And some comments to make the code more readable
2019-11-01 23:19:07 +01:00
Frank Denis
0dc69eacd5 resolveHostWithCache -> resolveWithCache 2019-11-01 23:10:36 +01:00
Frank Denis
b30904f20b lowercase 2019-11-01 23:06:42 +01:00
Frank Denis
8d191cdcf1 Rename CheckResolver to IsIPAndPort for clarity 2019-11-01 23:05:17 +01:00
Frank Denis
3cef651b07 Rename resolveHost() to resolveHostWithCache() for clarity
(but to be honest, I don't understand anything to that code any more)
2019-11-01 23:00:39 +01:00
Frank Denis
e028f4d483 Don't delete cached server IP addresses
If we can't update an entry, keep the previous one.
2019-11-01 22:55:06 +01:00
Frank Denis
3db3de0a91 Use SystemResolverTTL as a minimum timeout for cached resolver IPs 2019-11-01 21:47:13 +01:00