4a792026eb
Refactor cache reading to reduce number of return values
2019-11-08 10:17:12 +01:00
fe34d07b68
Refactor away some unnecessary type shuffling
...
Signatures in particular were read in from both cache and url as `[]byte`, converted to `string`, then back to `[]byte` to pass through to minisign.
Lists themselves will be converted to `string` by the parsing code anyway.
2019-11-08 10:17:12 +01:00
ad92be5b9c
Refactor saving downloads to cache
...
Moved writing to happen immediately after the download to reduce duplicated code and number of return values from the download function.
2019-11-08 10:17:12 +01:00
4c156784c8
Refactor calculation of update delay when reading cache
...
Set the default delay once at the top instead of before every early return.
2019-11-08 10:17:12 +01:00
e818eeb800
Refactor reading a URL's content to own function
...
No longer shadows `url` package with variable of the same name.
2019-11-08 10:17:12 +01:00
d851c9eeb6
Refactor signature verification to own function
...
The cache is no longer destroyed whenever any signature verification fails.
The public key is stored on the Source struct for future use.
2019-11-08 10:17:12 +01:00
da0d7fe841
Fix various timing inconsistencies
...
When comparing times in tests, it's necessary to control the `now` value to ensure slow test runs don't fail incorrectly.
Both cache and download code had been using refreshDelay to set the next prefetch delay, which by default meant the 1st prefetch was 3 days after startup - this has now been corrected to match the 1 day expectation.
Enabling some of the cache tests revealed some other incorrect failures in the test that were also fixed.
2019-11-08 10:17:12 +01:00
03dea47130
Remove dead code paths
...
These paths were unreachable because XTransport.Get already checks the same conditions.
2019-11-08 10:17:12 +01:00
3a68f90c37
Back to 2.0.29 beta 3 ( ceed905196)
2019-10-31 17:50:19 +01:00
6680faf665
make sure tcp/udp Conn are closed on stop signal
2019-10-25 12:56:34 +02:00
99e56a400f
when parsing a source, try to process all of the servers to ensure maximum connectivity
...
- a corrupt source (no names/descriptions) will still abort parsing immediately
- a duplicate, missing or invalid server stamp will not abort parsing
- these errors are logged as warnings, and returned as a single error at the end
2019-10-06 13:56:53 +02:00
d27171f62b
Have fetchFromCache support a TTL parameter
...
Partially
fixes #854
2019-06-13 11:24:15 +02:00
b624f8ef58
Accept sdns: scheme without a namespace
2019-03-03 18:20:39 +01:00
887f6ac299
Log invalid stamps
2018-07-09 15:47:59 +02:00
f20da6b611
Remove support for legacy CSV files
2018-04-18 19:06:50 +02:00
3d67c81697
Deps update
2018-04-18 18:58:39 +02:00
f63dc17f90
stamps -> dnsstamps
2018-04-18 18:47:10 +02:00
eb5f391fa6
Split stamps into package
2018-04-18 18:36:47 +02:00
adb0c94a61
April 1st is already over in some time zones :)
...
This reverts commit dac52ab42a
2018-04-01 16:35:32 +02:00
dac52ab42a
Completely remove support for the DNSCrypt protocol
2018-04-01 04:04:12 +02:00
ede564ccf7
Support multiple URLs for a given source
...
Fixes #265
2018-03-28 13:36:19 +02:00
7ed4ce17d7
Move things down for clarity
2018-03-28 13:00:06 +02:00
75f3c6403b
Print absolute paths when file caches cannot be written
2018-03-05 11:58:31 +01:00
0fde986c63
Nits
2018-02-19 19:31:34 +01:00
782c86b785
Fallback to cached content if we don't have a working connection
2018-02-19 19:24:51 +01:00
6baafe3232
Only rewrite a cache file if it doesn't already come from the cache
...
Fixes #150
2018-02-19 15:36:48 +01:00
43f3e64bd9
DoH: fallback to GET on servers that don't support POST
2018-02-05 11:30:10 +01:00
9d69811de9
Add limits to HTTP requests
2018-02-04 11:33:04 +01:00
d7ec318945
Accept sources without an URL; use v2 format by default for remote sources
2018-01-31 14:24:21 +01:00
148d900a72
Legacy format: store server descriptions
2018-01-31 09:45:10 +01:00
f8a6e56026
-list -json now prints the list of available servers as JSON
...
Can be useful for GUIs, especially since this includes the description
2018-01-31 09:42:56 +01:00
8f72a82b13
Don't forget to prepend prefixes to names in v2 lists
2018-01-30 19:47:29 +01:00
ecaf18f614
Use a fallback resolver if the local DNS configuration doesn't work
...
This should fix all chicken-and-egg issues
2018-01-30 15:47:39 +01:00
e16155e22a
DoHstamps
2018-01-26 20:38:31 +01:00
3bbecdcde7
up
2018-01-26 20:16:45 +01:00
79193e6ee3
Add support for V2 source format -- Goodbye, CSV.
2018-01-25 15:02:18 +01:00
1e0e01e8e1
NXLOG: a new output plugin to log suspicious queries
2018-01-20 16:59:40 +01:00
88414e1448
Print stamps; require an env variable for debug level
2018-01-20 13:56:26 +01:00
b9e89d2278
megacheck
2018-01-20 01:00:19 +01:00
1c27d6c230
Improved error handling
2018-01-20 00:30:33 +01:00
7fbb4c5428
Improve the prefetcher; run a dedicated goroutine
2018-01-19 23:43:45 +01:00
41d5de6e8d
Scheduling
2018-01-19 00:06:18 +01:00
43e5689387
Schedule a prefetch if we got a set or its signature from a backup cache
...
This is not pretty, and has to be rewritten for the next beta
2018-01-18 23:54:53 +01:00
1b5e36432e
Remove cache files if we stored corrupted data
2018-01-18 23:33:30 +01:00
f745eb578a
Check HTTP error codes
2018-01-18 23:31:14 +01:00
a85d012a2b
Prefetch previously unreachable sources URLs after a server is reachable
...
Partial fix for #4
Pave the way for regular, background updates as well
2018-01-18 23:19:14 +01:00
c4bd6eb9f0
Make the distinction between a usable cache and a hot cache
...
A hot cache is still fresh. A usable cache exists, and can act as a
backup solution is we can't fetch a list from a remote server.
2018-01-18 22:23:40 +01:00
7e86477a7d
Make megacheck happier
2018-01-18 14:28:05 +01:00
41a9bf5bf3
Add require_nolog and require_dnssec filters
2018-01-18 13:01:16 +01:00
96dadc7aca
Forwarding plugin
2018-01-17 09:44:03 +01:00
170e2e816e
Implement blocking, fully compatible with rules from version 1
2018-01-17 02:40:47 +01:00
004fbef395
Fix source cache
2018-01-16 00:37:04 +01:00
b945e23101
Use time.Since()
2018-01-14 23:53:17 +01:00
4fef1a705c
Fix source cache
2018-01-14 00:20:22 +01:00
13e30ade2b
Skip empty lines in the CSV file
2018-01-13 23:53:33 +01:00
a361aa52f3
Preliminary support for remote sources
2018-01-13 23:52:44 +01:00
William Elwood
Frank Denis
Vladimir Bauer
Alison Winters
gdm85