* ConfigFile change to allowlist and blocklist
* revised names and warnings
* consistent file naming in kebab case, and generic use of blocklist and allowlist in cmoments for clarity
* update ci files
* impose maximum delay and document
* live update of servers
* update for source prefixes
* fixup test
* stop registerServers being called twice at startup
* prevent double registration at startup
* tidy function signature for loadSource
Co-authored-by: Ian Bashford <ianbashford@gmail.com>
Thanks to @lifenjoiner for testing! Windows 10 behaves even more unexpectedly.
After it parses the "ip:port" string as a hostname, it attempts to upgrade from
http to https by appending `:443` and parsing that new URL again.
This seems to happen concurrently with the doomed DNS lookup and we see the
error from whichever fails first.
Untested attempt to fix unit tests that fail on Windows 10 build 1909.
From the test output mentioned in #1332, it looks like this version of Windows
doesn't report an "invalid port" error when asked to connect to an invalid port,
instead it treats the port as part of the host name and attempts a DNS lookup.
Naturally, this fails because the colon character is not valid in a host name.
This change simply makes this inexplicable error an expected result since the
outcome is the same and we can't fix Windows.
Previously when the cache was written to disk, the modification time was unspecified.
At the next prefetch, it was possible for the cache to be expiring very soon (on the order of milliseconds) but still deemed valid.
Now the modification time is explicitly set to when the prefetch run began to make this situation much less likely.
To simulate failures opening a cache file, fixtures are written without the read permission bits.
Since Unix permission bits have no meaning on Windows, a slightly more complicated solution is required to achieve the same permissions.
Thankfully, there's a library to abstract that already.
This allows a large number of tests to be enabled and pass now that the behaviour is expected.
The main fix here is that a download with an invalid signature will always fall back on using a properly signed cache, no matter how old it is.
Additionally, downloads will never be written to the cache unless they are properly signed (both at startup and prefetching).
Cache TTL is how old the cache can be at startup before trying to download an update immediately.
Prefetch delay is how long the prefetcher should wait between successful downloads of a source.
Previously, the refresh_delay configuration was used at startup as both cache TTL and prefetch delay, with subsequent prefetches using a hard-coded delay.
As discussed, refresh_delay is now only used for cache TTL, prefetch delay always uses the hard-coded delay.
When a list fails to download, there's no point trying to download the signature.
Code duplication moved to where it's easier to refactor away.
Enabled a few more tests.
The proxy shouldn't need to know how prefetching works, just that it needs to do it occasionally. Now the prefetching algorithm can be refactored without having to touch the proxy code.
Signatures in particular were read in from both cache and url as `[]byte`, converted to `string`, then back to `[]byte` to pass through to minisign.
Lists themselves will be converted to `string` by the parsing code anyway.
When comparing times in tests, it's necessary to control the `now` value to ensure slow test runs don't fail incorrectly.
Both cache and download code had been using refreshDelay to set the next prefetch delay, which by default meant the 1st prefetch was 3 days after startup - this has now been corrected to match the 1 day expectation.
Enabling some of the cache tests revealed some other incorrect failures in the test that were also fixed.
Tests cover most of the cache and download related code paths and specify the expected result of various starting states and external failure modes.
Where the current code's behaviour doesn't match a test's expectations, the test is disabled and annotated with a TODO until it can be fixed.
Added dependency on `github.com/powerman/check` and ran `go mod vendor`.
Frank Denis
lifenjoiner
Ian Bashford
William Elwood
Frank Denis
Alison Winters