Git has been nagging me about these files that I haven't touched being changed.
The solution was to run `git add --renormalize .`.
Apparently this needs to be done after setting up `.gitattributes`.
According to `git ls-files --eol windows/`, the .bat files were previously stored in the index with CRLF endings and will now be stored with LF endings (with .gitattributes dictating that checked out copies will use CRLF).
Previously when the cache was written to disk, the modification time was unspecified.
At the next prefetch, it was possible for the cache to be expiring very soon (on the order of milliseconds) but still deemed valid.
Now the modification time is explicitly set to when the prefetch run began to make this situation much less likely.
To simulate failures opening a cache file, fixtures are written without the read permission bits.
Since Unix permission bits have no meaning on Windows, a slightly more complicated solution is required to achieve the same permissions.
Thankfully, there's a library to abstract that already.
This way it matches with the "loading from URL" info message and users with that log level aren't left with the false impression that it loaded fine when it really didn't.
Previously, an expired cache would be read before trying each URL until a download completed.
By moving the download loop, the cache can be read once outside the loop.
This allows a large number of tests to be enabled and pass now that the behaviour is expected.
The main fix here is that a download with an invalid signature will always fall back on using a properly signed cache, no matter how old it is.
Additionally, downloads will never be written to the cache unless they are properly signed (both at startup and prefetching).
Cache TTL is how old the cache can be at startup before trying to download an update immediately.
Prefetch delay is how long the prefetcher should wait between successful downloads of a source.
Previously, the refresh_delay configuration was used at startup as both cache TTL and prefetch delay, with subsequent prefetches using a hard-coded delay.
As discussed, refresh_delay is now only used for cache TTL, prefetch delay always uses the hard-coded delay.
Previously when constructing the signature URL, the `.minisig` suffix was blindly appended to the string version of the source URL.
Now we take the parsed source URL, deep copy it (saves us parsing it twice), and append the `.minisig` suffix to the path component of the URL.
When a list fails to download, there's no point trying to download the signature.
Code duplication moved to where it's easier to refactor away.
Enabled a few more tests.
The proxy shouldn't need to know how prefetching works, just that it needs to do it occasionally. Now the prefetching algorithm can be refactored without having to touch the proxy code.
Signatures in particular were read in from both cache and url as `[]byte`, converted to `string`, then back to `[]byte` to pass through to minisign.
Lists themselves will be converted to `string` by the parsing code anyway.
When comparing times in tests, it's necessary to control the `now` value to ensure slow test runs don't fail incorrectly.
Both cache and download code had been using refreshDelay to set the next prefetch delay, which by default meant the 1st prefetch was 3 days after startup - this has now been corrected to match the 1 day expectation.
Enabling some of the cache tests revealed some other incorrect failures in the test that were also fixed.
Frank Denis
William Elwood
Will Elwood
Alison Winters