Commit Graph

1119 Commits

Author SHA1 Message Date
William Elwood 1e225dbb67 Alter source tests to cover entire prefetch algorithm and make it pass 2019-11-08 10:17:12 +01:00
William Elwood 7e73a26a2f Move most of the prefetching code into sources.go
The proxy shouldn't need to know how prefetching works, just that it needs to do it occasionally. Now the prefetching algorithm can be refactored without having to touch the proxy code.
2019-11-08 10:17:12 +01:00
William Elwood 78f2dead79 Move prefetch URLs onto Source struct
This is mostly in preparation for further refactoring, but does reduce the number of return values from `NewSource()` too.
2019-11-08 10:17:12 +01:00
William Elwood 4a792026eb Refactor cache reading to reduce number of return values 2019-11-08 10:17:12 +01:00
William Elwood fe34d07b68 Refactor away some unnecessary type shuffling
Signatures in particular were read in from both cache and url as `[]byte`, converted to `string`, then back to `[]byte` to pass through to minisign.
Lists themselves will be converted to `string` by the parsing code anyway.
2019-11-08 10:17:12 +01:00
William Elwood ad92be5b9c Refactor saving downloads to cache
Moved writing to happen immediately after the download to reduce duplicated code and number of return values from the download function.
2019-11-08 10:17:12 +01:00
William Elwood 4c156784c8 Refactor calculation of update delay when reading cache
Set the default delay once at the top instead of before every early return.
2019-11-08 10:17:12 +01:00
William Elwood e818eeb800 Refactor reading a URL's content to own function
No longer shadows `url` package with variable of the same name.
2019-11-08 10:17:12 +01:00
William Elwood d851c9eeb6 Refactor signature verification to own function
The cache is no longer destroyed whenever any signature verification fails.
The public key is stored on the Source struct for future use.
2019-11-08 10:17:12 +01:00
William Elwood da0d7fe841 Fix various timing inconsistencies
When comparing times in tests, it's necessary to control the `now` value to ensure slow test runs don't fail incorrectly.
Both cache and download code had been using refreshDelay to set the next prefetch delay, which by default meant the 1st prefetch was 3 days after startup - this has now been corrected to match the 1 day expectation.
Enabling some of the cache tests revealed some other incorrect failures in the test that were also fixed.
2019-11-08 10:17:12 +01:00
William Elwood 03dea47130 Remove dead code paths
These paths were unreachable because XTransport.Get already checks the same conditions.
2019-11-08 10:17:12 +01:00
William Elwood af0629856c Add unit tests for sources.go
Tests cover most of the cache and download related code paths and specify the expected result of various starting states and external failure modes.
Where the current code's behaviour doesn't match a test's expectations, the test is disabled and annotated with a TODO until it can be fixed.
Added dependency on `github.com/powerman/check` and ran `go mod vendor`.
2019-11-08 10:17:12 +01:00
William Elwood 503bfb877b go mod tidy
```console
$ go mod tidy -v
(snip)
unused github.com/agl/ed25519
```
Also add base .gitattributes file to normalize line endings in the repository across differing developer environments.
2019-11-08 10:17:12 +01:00
Frank Denis e6a4a4ffda Update deps 2019-11-05 01:32:38 +01:00
Frank Denis b76db70a6c Revert "cleanup: estimators: simplify blindAdjust"
This reverts commit c699e7bec4.
2019-11-05 01:16:22 +01:00
Frank Denis 17a675021e No one ever completes // TODO things 2019-11-05 01:10:57 +01:00
Frank Denis da3f30871f Revert "fix: proxy: Trigger query logging plugins using defer"
This reverts commit fc9509a8c8.
2019-11-05 00:54:03 +01:00
Frank Denis 14862c2fc7 defer is slow and not worth it here 2019-11-05 00:37:46 +01:00
Frank Denis 316c5ca6b1 Don't return immediately on non-Windows system if netprobe_timout is -1
Fixes #1016
2019-11-04 17:14:31 +01:00
Frank Denis 9852a289f8 Increase the default cache size and minimum TTL 2019-11-03 17:31:41 +01:00
Frank Denis e0c37f92fc Add a comment about why DoH addresses from stamps don't expire 2019-11-03 00:33:17 +01:00
Frank Denis 0f332c644d Set a minimum TTL when caching resolver IPs
Comcast having a 30 sec TTL is silly
2019-11-02 02:01:03 +01:00
Frank Denis 63ed3b4fef Update comment 2019-11-02 01:52:51 +01:00
Frank Denis a84a789a8a Keep resolving if needed 2019-11-02 01:50:35 +01:00
Frank Denis d932d5fdfc Inverse test 2019-11-02 01:20:28 +01:00
Frank Denis 6032c3b79b Add a grace TTL for expired cached IPs
And some comments to make the code more readable
2019-11-01 23:19:07 +01:00
Frank Denis 0dc69eacd5 resolveHostWithCache -> resolveWithCache 2019-11-01 23:10:36 +01:00
Frank Denis b30904f20b lowercase 2019-11-01 23:06:42 +01:00
Frank Denis 8d191cdcf1 Rename CheckResolver to IsIPAndPort for clarity 2019-11-01 23:05:17 +01:00
Frank Denis 3cef651b07 Rename resolveHost() to resolveHostWithCache() for clarity
(but to be honest, I don't understand anything to that code any more)
2019-11-01 23:00:39 +01:00
Frank Denis e028f4d483 Don't delete cached server IP addresses
If we can't update an entry, keep the previous one.
2019-11-01 22:55:06 +01:00
Frank Denis 3db3de0a91 Use SystemResolverTTL as a minimum timeout for cached resolver IPs 2019-11-01 21:47:13 +01:00
Alison Winters 97e4c44223 remove err return values that are never set 2019-11-01 17:13:14 +01:00
Alison Winters 36808cdec7 remove unused patternType return 2019-11-01 17:13:14 +01:00
Alison Winters a0d9412a25 run goimports 2019-11-01 17:13:14 +01:00
Will Elwood cd675913b2
Fix copy-paste oversight
I think these variables are always both nil or both not nil, but maybe in the future they might not be.
2019-11-01 09:07:57 +00:00
Alison Winters 816acb9d8d move ConfigLoad into AppMain 2019-10-31 18:55:44 +01:00
Alison Winters 2f7e057996 move flags parsing into main() 2019-10-31 18:55:44 +01:00
Frank Denis 116f985b96 Bump 2019-10-31 18:12:13 +01:00
Frank Denis c5bda9e2ae Update deps 2019-10-31 18:00:44 +01:00
Frank Denis 59c3d5121d Add brackets around cached IPv6 IP addresses
Fixes #1005
2019-10-31 17:55:54 +01:00
Eric Lagergren 1c9924e055 check error that was being erroneously shadowed 2019-10-31 17:55:26 +01:00
Frank Denis 971b08bcec No more beta 2019-10-31 17:54:04 +01:00
Frank Denis 06b0976786 Remove the serverInfo rwlock; just use the global serversInfo rwlock 2019-10-31 17:50:56 +01:00
Frank Denis 3a68f90c37 Back to 2.0.29 beta 3 (ceed905196) 2019-10-31 17:50:19 +01:00
Frank Denis fb1fc14317 Revert "refactoring of pull 980"
This reverts commit 6fa420a8e0.
2019-10-31 17:36:59 +01:00
Frank Denis 7636a78d74 Revert "avoid LoadConfig twice"
This reverts commit 8f2c438c70.
2019-10-31 17:36:32 +01:00
Frank Denis 828af28658 Revert "move ConfigLoad and InitPluginsGlobals to appMain"
This reverts commit b67d19ffb3.
2019-10-31 17:36:28 +01:00
Frank Denis 393195066e Up 2019-10-31 16:50:38 +01:00
Frank Denis ac0425835b Update ChangeLog 2019-10-31 16:42:36 +01:00