cotugno/AdGuardHome
2
Fork 0
mirror of https://github.com/AdguardTeam/AdGuardHome.git synced 2024-11-15 09:58:42 -07:00
Code Issues Packages Projects Releases Wiki Activity

all: sync with master; upd chlog

Browse Source
This commit is contained in:
Ainar Garipov 2023-03-09 15:39:35 +03:00
parent 4f928be393
commit a21558f418
98 changed files with 2687 additions and 24734 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.github/workflows/build.yml vendored
View File

@ -1,7 +1,7 @@
'name': 'build'
'env':
'GO_VERSION': '1.19.6'
'GO_VERSION': '1.19.7'
'NODE_VERSION': '14'
'on':

2
.github/workflows/lint.yml vendored
View File

@ -1,7 +1,7 @@
'name': 'lint'
'env':
'GO_VERSION': '1.19.6'
'GO_VERSION': '1.19.7'
'on':
'push':

84
CHANGELOG.md
View File

@ -14,20 +14,90 @@ and this project adheres to
<!--
## [v0.108.0] - TBA
## [v0.107.26] - 2023-03-09 (APPROX.)
## [v0.107.27] - 2023-03-29 (APPROX.)
See also the [v0.107.26 GitHub milestone][ms-v0.107.26].
See also the [v0.107.27 GitHub milestone][ms-v0.107.27].
[ms-v0.107.26]: https://github.com/AdguardTeam/AdGuardHome/milestone/62?closed=1
[ms-v0.107.27]: https://github.com/AdguardTeam/AdGuardHome/milestone/63?closed=1
NOTE: Add new changes BELOW THIS COMMENT.
-->
## [v0.107.26] - 2023-03-09
See also the [v0.107.26 GitHub milestone][ms-v0.107.26].
### Security
- Go version has been updated to prevent the possibility of exploiting the
CVE-2023-24532 Go vulnerability fixed in [Go 1.19.7][go-1.19.7].
### Added
- The ability to set custom IP for EDNS Client Subnet by using the new
`dns.edns_client_subnet.use_custom` and `dns.edns_client_subnet.custom_ip`
fields ([#1472]). The UI changes are coming in the upcoming releases.
- The ability to use `dnstype` rules in the disallowed domains list ([#5468]).
This allows dropping requests based on their question types.
### Changed
#### Configuration Changes
In this release, the schema version has changed from 16 to 17.
- Property `edns_client_subnet`, which in schema versions 16 and earlier used
to be a part of the `dns` object, is now part of the `dns.edns_client_subnet`
object:
```yaml
# BEFORE:
'dns':
# …
'edns_client_subnet': false
# AFTER:
'dns':
# …
'edns_client_subnet':
'enabled': false
'use_custom': false
'custom_ip': ''
```
To rollback this change, move the value of `dns.edns_client_subnet.enabled`
into the `dns.edns_client_subnet`, remove the fields
`dns.edns_client_subnet.enabled`, `dns.edns_client_subnet.use_custom`,
`dns.edns_client_subnet.custom_ip`, and change the `schema_version` back to
`16`.
### Fixed
- Obsolete value of the Interface MTU DHCP option is now omitted ([#5281]).
- Various dark theme bugs ([#5439], [#5441], [#5442], [#5515]).
- Automatic update on MIPS64 and little-endian 32-bit MIPS architectures
([#5270], [#5373]).
- Requirements to domain names in domain-specific upstream configurations have
been relaxed to meet those from [RFC 3696][rfc3696] ([#4884]).
- Failing service installation via script on FreeBSD ([#5431]).
[#1472]: https://github.com/AdguardTeam/AdGuardHome/issues/1472
[#4884]: https://github.com/AdguardTeam/AdGuardHome/issues/4884
[#5270]: https://github.com/AdguardTeam/AdGuardHome/issues/5270
[#5281]: https://github.com/AdguardTeam/AdGuardHome/issues/5281
[#5373]: https://github.com/AdguardTeam/AdGuardHome/issues/5373
[#5431]: https://github.com/AdguardTeam/AdGuardHome/issues/5431
[#5439]: https://github.com/AdguardTeam/AdGuardHome/issues/5439
[#5441]: https://github.com/AdguardTeam/AdGuardHome/issues/5441
[#5442]: https://github.com/AdguardTeam/AdGuardHome/issues/5442
[#5468]: https://github.com/AdguardTeam/AdGuardHome/issues/5468
[#5515]: https://github.com/AdguardTeam/AdGuardHome/issues/5515
[go-1.19.7]: https://groups.google.com/g/golang-announce/c/3-TpUx48iQY
[ms-v0.107.26]: https://github.com/AdguardTeam/AdGuardHome/milestone/62?closed=1
[rfc3696]: https://datatracker.ietf.org/doc/html/rfc3696
<!--
NOTE: Add new changes ABOVE THIS COMMENT.
@ -109,6 +179,7 @@ In this release, the schema version has changed from 14 to 16.
'file_enabled': true
'interval': '2160h'
'size_memory': 1000
'ignored': []
```
To rollback this change, rename and move properties back into the `dns`
@ -1689,11 +1760,12 @@ See also the [v0.104.2 GitHub milestone][ms-v0.104.2].
<!--
[Unreleased]: https://github.com/AdguardTeam/AdGuardHome/compare/v0.107.26...HEAD
[v0.107.26]: https://github.com/AdguardTeam/AdGuardHome/compare/v0.107.25...v0.107.26
[Unreleased]: https://github.com/AdguardTeam/AdGuardHome/compare/v0.107.27...HEAD
[v0.107.27]: https://github.com/AdguardTeam/AdGuardHome/compare/v0.107.26...v0.107.27
-->
[Unreleased]: https://github.com/AdguardTeam/AdGuardHome/compare/v0.107.25...HEAD
[Unreleased]: https://github.com/AdguardTeam/AdGuardHome/compare/v0.107.26...HEAD
[v0.107.26]: https://github.com/AdguardTeam/AdGuardHome/compare/v0.107.25...v0.107.26
[v0.107.25]: https://github.com/AdguardTeam/AdGuardHome/compare/v0.107.24...v0.107.25
[v0.107.24]: https://github.com/AdguardTeam/AdGuardHome/compare/v0.107.23...v0.107.24
[v0.107.23]: https://github.com/AdguardTeam/AdGuardHome/compare/v0.107.22...v0.107.23

30
Makefile
View File

@ -4,17 +4,26 @@
# See https://pubs.opengroup.org/onlinepubs/9699919799/utilities/make.html.
.POSIX:
# This comment is used to simplify checking local copies of the
# Makefile. Bump this number every time a significant change is made to
# this Makefile.
#
# AdGuard-Project-Version: 2
# Don't name these macros "GO" etc., because GNU Make apparently makes
# them exported environment variables with the literal value of
# "${GO:-go}" and so on, which is not what we need. Use a dot in the
# name to make sure that users don't have an environment variable with
# the same name.
#
# See https://unix.stackexchange.com/q/646255/105635.
GO.MACRO = $${GO:-go}
VERBOSE.MACRO = $${VERBOSE:-0}
CHANNEL = development
CLIENT_DIR = client
COMMIT = $$( git rev-parse --short HEAD )
DIST_DIR = dist
# Don't name this macro "GO", because GNU Make apparenly makes it an
# exported environment variable with the literal value of "${GO:-go}",
# which is not what we need. Use a dot in the name to make sure that
# users don't have an environment variable with the same name.
#
# See https://unix.stackexchange.com/q/646255/105635.
GO.MACRO = $${GO:-go}
GOPROXY = https://goproxy.cn|https://proxy.golang.org|direct
GOSUMDB = sum.golang.google.cn
GPG_KEY = devteam@adguard.com
@ -25,7 +34,6 @@ NPM_INSTALL_FLAGS = $(NPM_FLAGS) --quiet --no-progress --ignore-engines\
--ignore-optional --ignore-platform --ignore-scripts
RACE = 0
SIGN = 1
VERBOSE = 0
VERSION = v0.0.0
YARN = yarn
@ -56,13 +64,13 @@ ENV = env\
PATH="$${PWD}/bin:$$( "$(GO.MACRO)" env GOPATH )/bin:$${PATH}"\
RACE='$(RACE)'\
SIGN='$(SIGN)'\
VERBOSE='$(VERBOSE)'\
VERBOSE="$(VERBOSE.MACRO)"\
VERSION='$(VERSION)'\
# Keep the line above blank.
# Keep this target first, so that a naked make invocation triggers
# a full build.
# Keep this target first, so that a naked make invocation triggers a
# full build.
build: deps quick-build
quick-build: js-build go-build

6
bamboo-specs/release.yaml
View File

@ -7,7 +7,7 @@
# Make sure to sync any changes with the branch overrides below.
'variables':
'channel': 'edge'
'dockerGo': 'adguard/golang-ubuntu:6.1'
'dockerGo': 'adguard/golang-ubuntu:6.2'
'stages':
- 'Build frontend':
@ -331,7 +331,7 @@
# need to build a few of these.
'variables':
'channel': 'beta'
'dockerGo': 'adguard/golang-ubuntu:6.1'
'dockerGo': 'adguard/golang-ubuntu:6.2'
# release-vX.Y.Z branches are the branches from which the actual final release
# is built.
- '^release-v[0-9]+\.[0-9]+\.[0-9]+':
@ -346,4 +346,4 @@
# are the ones that actually get released.
'variables':
'channel': 'release'
'dockerGo': 'adguard/golang-ubuntu:6.1'
'dockerGo': 'adguard/golang-ubuntu:6.2'

2
bamboo-specs/test.yaml
View File

@ -5,7 +5,7 @@
'key': 'AHBRTSPECS'
'name': 'AdGuard Home - Build and run tests'
'variables':
'dockerGo': 'adguard/golang-ubuntu:6.1'
'dockerGo': 'adguard/golang-ubuntu:6.2'
'stages':
- 'Tests':

2
client/src/__locales/fi.json
View File

@ -297,7 +297,7 @@
"blocking_mode_refused": "REFUSED: Vastaa REFUSED-koodilla",
"blocking_mode_nxdomain": "NXDOMAIN: Vastaa NXDOMAIN-koodilla",
"blocking_mode_null_ip": "Tyhjä IP: Vastaa IP-nollaosoitteella (0.0.0.0 korvaa A; :: korvaa AAAA)",
"blocking_mode_custom_ip": "Mukautettu IP: Vastaa itse määritetyllä IP-osoitteella",
"blocking_mode_custom_ip": "Mukautettu IP: Vastaa manuaalisesti määritetyllä IP-osoitteella",
"theme_auto": "Automaattinen",
"theme_light": "Vaalea",
"theme_dark": "Tumma",

1
client/src/__locales/no.json
View File

@ -281,6 +281,7 @@
"blocking_mode_nxdomain": "NXDOMAIN: Svar med NXDOMAIN-koden",
"blocking_mode_null_ip": "Null IP: Svar med en 0-IP-adresse (0.0.0.0 for A; :: for AAAA)",
"blocking_mode_custom_ip": "Tilpasset IP: Svar med en manuelt valgt IP-adresse",
"theme_auto": "Auto",
"upstream_dns_client_desc": "Hvis dette feltet holdes tomt, vil AdGuard Home bruke tjenerne som er satt opp i <0>DNS-innstillingene</0>.",
"tracker_source": "Sporerkilde",
"source_label": "Kilde",

2
client/src/__locales/tr.json
View File

@ -449,7 +449,7 @@
"access_disallowed_title": "İzin verilmeyen istemciler",
"access_disallowed_desc": "CIDR'lerin, IP adreslerinin veya <a>İstemci Kimliklerin</a> listesi. Bu listede girişler varsa, AdGuard Home bu istemcilerden gelen istekleri keser. İzin verilen istemcilerde girişler varsa, bu alan yok sayılır.",
"access_blocked_title": "İzin verilmeyen alan adları",
"access_blocked_desc": "Bu işlem filtrelerle ilgili değildir. AdGuard Home, bu alan adlarından gelen DNS sorgularını yanıtsız bırakır ve bu sorgular sorgu günlüğünde görünmez. Tam alan adlarını, joker karakterleri veya URL filtre kurallarını belirtebilirsiniz, ör. \"example.org\", \"*.example.org\" veya \"||example.org^\".",
"access_blocked_desc": "Bu işlem filtrelerle ilgili değildir. AdGuard Home, bu alan adlarından gelen DNS sorgularını yanıtsız bırakır ve bu sorgular sorgu günlüğünde görünmez. Tam alan adlarını, joker karakterleri veya URL filtre kurallarını belirtebilirsiniz, örn. \"example.org\", \"*.example.org\" veya \"||example.org^\".",
"access_settings_saved": "Erişim ayarları başarıyla kaydedildi!",
"updates_checked": "AdGuard Home'un yeni bir sürümü mevcut",
"updates_version_equal": "AdGuard Home yazılımı güncel durumda",

2
client/src/__locales/zh-cn.json
View File

@ -172,7 +172,7 @@
"list_url_table_header": "清单网址",
"rules_count_table_header": "规则数",
"last_time_updated_table_header": "上次更新时间",
"actions_table_header": "活跃状态",
"actions_table_header": "操作",
"request_table_header": "请求",
"edit_table_action": "编辑",
"delete_table_action": "删除",

10
client/src/components/App/index.css
View File

@ -30,6 +30,11 @@
--loading-bg: rgba(255, 255, 255, 0.48);
--font-family-monospace: Monaco, Menlo, "Ubuntu Mono", Consolas, source-code-pro, monospace;
--font-size-disable-autozoom: 1rem;
--alert-message-color: #24426c;
--alert-message-border: #cbdbf2;
--alert-message-bg: #dae5f5;
--checkbox-bg: #e2e2e2;
--radio-bg: #ffffff;
}
[data-theme="dark"] {
@ -59,6 +64,11 @@
--detailed-info-color: #fff;
--gray300: #f3f3f3;
--loading-bg: #131313;
--alert-message-color: #e6e6e6;
--alert-message-border: #363648;
--alert-message-bg: #363648;
--checkbox-bg: #a4a4a4;
--radio-bg: #a4a4a4;
}
body {

2
client/src/components/Dashboard/index.js
View File

@ -60,7 +60,7 @@ const Dashboard = ({
title={t('refresh_btn')}
onClick={() => getAllStats()}
>
<svg className="icons">
<svg className="icons icon12">
<use xlinkHref="#refresh" />
</svg>
</button>;

4
client/src/components/Filters/Table.js
View File

@ -100,7 +100,7 @@ class Table extends Component {
})
}
>
<svg className="icons">
<svg className="icons icon12">
<use xlinkHref="#edit" />
</svg>
</button>
@ -110,7 +110,7 @@ class Table extends Component {
onClick={() => handleDelete(url)}
title={t('delete_table_action')}
>
<svg className="icons">
<svg className="icons icon12">
<use xlinkHref="#delete" />
</svg>
</button>

2
client/src/components/Logs/Cells/ClientCell.js
View File

@ -162,7 +162,7 @@ const ClientCell = ({
{content && (
<button className={buttonArrowClass} disabled={processingRules}>
<IconTooltip
className="h-100"
className="icon24"
tooltipClass="button-action--arrow-option-container"
xlinkHref="chevron-down"
triggerClass="button-action--icon"

1
client/src/components/Logs/Filters/Form.js
View File

@ -129,7 +129,6 @@ const Form = (props) => {
const onInputClear = async () => {
setIsLoading(true);
setDebouncedSearch(DEFAULT_LOGS_FILTER[FORM_NAMES.search]);
change(FORM_NAMES.search, DEFAULT_LOGS_FILTER[FORM_NAMES.search]);
setIsLoading(false);
};

19
client/src/components/Logs/Logs.css
View File

@ -106,6 +106,16 @@
max-height: 100% !important;
}
.icon24 {
width: 24px;
height: 24px;
}
.icon12 {
width: 12px;
height: 12px;
}
.cursor--pointer {
cursor: pointer;
}
@ -130,6 +140,10 @@
background-color: transparent !important;
}
[data-theme="dark"] .form-control--transparent option {
background-color: var(--card-bgcolor);
}
.input-group-search {
background-color: transparent;
position: relative;
@ -307,7 +321,6 @@
height: 100%;
display: flex;
justify-content: center;
align-items: center;
}
.button-action:active {
@ -400,6 +413,10 @@
background-color: var(--logs__row--blue-bgcolor);
}
[data-theme="dark"] .logs__row--blue .logs__text--link {
color: var(--white);
}
.logs__row--green {
background-color: var(--green-pale);
}

4
client/src/components/Settings/Clients/ClientsTable/ClientsTable.js
View File

@ -290,7 +290,7 @@ const ClientsTable = ({
disabled={processingUpdating}
title={t('edit_table_action')}
>
<svg className="icons">
<svg className="icons icon12">
<use xlinkHref="#edit" />
</svg>
</button>
@ -301,7 +301,7 @@ const ClientsTable = ({
disabled={processingDeleting}
title={t('delete_table_action')}
>
<svg className="icons">
<svg className="icons icon12">
<use xlinkHref="#delete" />
</svg>
</button>

6
client/src/components/Settings/Clients/Service.css
View File

@ -54,6 +54,12 @@
color: #495057;
}
.service__icon svg {
width: 20px;
height: 20px;
fill: #495057;
}
.service--global .service__icon {
display: none;
}

2
client/src/components/ui/Checkbox.css
View File

@ -48,7 +48,7 @@
height: 20px;
min-width: 20px;
margin-right: 10px;
background-color: #e2e2e2;
background-color: var(--checkbox-bg);
background-repeat: no-repeat;
background-position: center center;
background-size: 12px 10px;

10
client/src/components/ui/Icons.js
View File

@ -86,10 +86,10 @@ const Icons = () => (
d="m19.4 15a1.65 1.65 0 0 0 .33 1.82l.06.06a2 2 0 0 1 0 2.83 2 2 0 0 1 -2.83 0l-.06-.06a1.65 1.65 0 0 0 -1.82-.33 1.65 1.65 0 0 0 -1 1.51v.17a2 2 0 0 1 -2 2 2 2 0 0 1 -2-2v-.09a1.65 1.65 0 0 0 -1.08-1.51 1.65 1.65 0 0 0 -1.82.33l-.06.06a2 2 0 0 1 -2.83 0 2 2 0 0 1 0-2.83l.06-.06a1.65 1.65 0 0 0 .33-1.82 1.65 1.65 0 0 0 -1.51-1h-.17a2 2 0 0 1 -2-2 2 2 0 0 1 2-2h.09a1.65 1.65 0 0 0 1.51-1.08 1.65 1.65 0 0 0 -.33-1.82l-.06-.06a2 2 0 0 1 0-2.83 2 2 0 0 1 2.83 0l.06.06a1.65 1.65 0 0 0 1.82.33h.08a1.65 1.65 0 0 0 1-1.51v-.17a2 2 0 0 1 2-2 2 2 0 0 1 2 2v.09a1.65 1.65 0 0 0 1 1.51 1.65 1.65 0 0 0 1.82-.33l.06-.06a2 2 0 0 1 2.83 0 2 2 0 0 1 0 2.83l-.06.06a1.65 1.65 0 0 0 -.33 1.82v.08a1.65 1.65 0 0 0 1.51 1h.17a2 2 0 0 1 2 2 2 2 0 0 1 -2 2h-.09a1.65 1.65 0 0 0 -1.51 1z" />
</symbol>
<symbol id="refresh" viewBox="0 0 24 24" stroke="currentColor" fill="none"
strokeLinecap="round" strokeLinejoin="round" strokeWidth="2">
<path d="M23 4v6h-6M1 20v-6h6" />
<path d="M3.51 9a9 9 0 0 1 14.85-3.36L23 10M1 14l4.64 4.36A9 9 0 0 0 20.49 15" />
<symbol id="refresh" viewBox="0 0 24 24" stroke="currentColor" fill="none" strokeLinecap="round" strokeLinejoin="round" strokeWidth="2">
<polyline points="23 4 23 10 17 10"></polyline>
<polyline points="1 20 1 14 7 14"></polyline>
<path d="M3.51 9a9 9 0 0 1 14.85-3.36L23 10M1 14l4.64 4.36A9 9 0 0 0 20.49 15"></path>
</symbol>
<symbol id="dns_privacy" viewBox="0 0 30 30" stroke="none" fill="currentColor"
@ -198,7 +198,7 @@ const Icons = () => (
</svg>
</symbol>
<symbol id="chevron-down" viewBox="0 0 24 24">
<symbol id="chevron-down" width="24" height="24" viewBox="0 0 24 24">
<g fill="none" fillRule="evenodd">
<path d="M0 0h24v24H0z" fill="#878787" fillOpacity=".01" />
<path stroke="currentColor" strokeWidth="1.5" strokeLinecap="round"

14
client/src/components/ui/Tabler.css
View File

@ -5047,9 +5047,9 @@ tbody.collapse.show {
}
.alert-primary {
color: #24426c;
background-color: #dae5f5;
border-color: #cbdbf2;
color: var(--alert-message-color);
background-color: var(--alert-message-bg);
border-color: var(--alert-message-border);
}
.alert-primary hr {
@ -5089,9 +5089,9 @@ tbody.collapse.show {
}
.alert-info {
color: #24587e;
background-color: #daeefc;
border-color: #cbe7fb;
color: var(--alert-message-color);
background-color: var(--alert-message-bg);
border-color: var(--alert-message-border);
}
.alert-info hr {
@ -14317,7 +14317,7 @@ textarea[cols] {
.custom-control-label:before {
border: 1px solid rgba(0, 40, 100, 0.12);
background-color: #fff;
background-color: var(--radio-bg);
background-size: 0.5rem;
}

2
client/src/components/ui/Version.js
View File

@ -39,7 +39,7 @@ const Version = () => {
disabled={processingVersion}
title={t('check_updates_now')}
>
<svg className="icons">
<svg className="icons icon12">
<use xlinkHref="#refresh" />
</svg>
</button>}

4
client/src/helpers/filters/filters.js
View File

@ -101,7 +101,7 @@ export default {
"source": "https://adguardteam.github.io/HostlistsRegistry/assets/filter_13.txt"
},
"POL_polish_filters_for_pi_hole": {
"name": "POL: Polish filters for Pi hole",
"name": "POL: Polish filters for Pi-hole",
"categoryId": "regional",
"homepage": "https://www.certyficate.it/",
"source": "https://adguardteam.github.io/HostlistsRegistry/assets/filter_14.txt"
@ -235,7 +235,7 @@ export default {
"urlhaus_filter_online": {
"name": "Malicious URL Blocklist (URLHaus)",
"categoryId": "security",
"homepage": "https://gitlab.com/malware-filter/urlhaus-filter",
"homepage": "https://urlhaus.abuse.ch/",
"source": "https://adguardteam.github.io/HostlistsRegistry/assets/filter_11.txt"
},
"windowsspyblocker_hosts_spy_rules": {

175
client/src/helpers/trackers/adguard.json
View File

@ -1,175 +0,0 @@
{
"timeUpdated": "2021-12-15",
"categories": {
"0": "audio_video_player",
"1": "comments",
"2": "customer_interaction",
"3": "pornvertising",
"4": "advertising",
"5": "essential",
"6": "site_analytics",
"7": "social_media",
"8": "misc",
"9": "cdn",
"10": "hosting",
"11": "unknown",
"12": "extensions",
"13": "email",
"14": "consent",
"15": "telemetry",
"101": "mobile_analytics"
},
"trackers": {
"akamai_technologies": {
"name": "Akamai Technologies",
"categoryId": 9,
"url": "https://www.akamai.com/",
"companyId": "akamai"
},
"apple": {
"name": "Apple",
"categoryId": 8,
"url": "https://www.apple.com/",
"companyId": "apple"
},
"apple_ads": {
"name": "Apple Search Ads",
"categoryId": 4,
"url": "https://searchads.apple.com/",
"companyId": "apple"
},
"facebook_audience": {
"name": "Facebook Audience Network",
"categoryId": 4,
"url": "https://www.facebook.com/business/products/audience-network",
"companyId": "facebook"
},
"crashlytics": {
"name": "Crashlytics",
"categoryId": 101,
"url": "https://crashlytics.com/",
"companyId": null
},
"flurry": {
"name": "Flurry",
"categoryId": 101,
"url": "http://www.flurry.com/",
"companyId": "verizon"
},
"hockeyapp": {
"name": "HockeyApp",
"categoryId": 101,
"url": "https://hockeyapp.net/",
"companyId": null
},
"firebase": {
"name": "Firebase",
"categoryId": 101,
"url": "https://firebase.google.com/",
"companyId": "google"
},
"appsflyer": {
"name": "AppsFlyer",
"categoryId": 101,
"url": "https://www.appsflyer.com/",
"companyId": "appsflyer"
},
"yandex_appmetrica": {
"name": "Yandex AppMetrica",
"categoryId": 101,
"url": "https://appmetrica.yandex.com/",
"companyId": "yandex"
},
"adjust": {
"name": "Adjust",
"categoryId": 101,
"url": "https://www.adjust.com/",
"companyId": "adjust"
},
"branch": {
"name": "Branch.io",
"categoryId": 101,
"url": "https://branch.io/",
"companyId": "branch_metrics_inc"
},
"markmonitor": {
"name": "MarkMonitor",
"categoryId": 4,
"url": "https://www.markmonitor.com/",
"companyId": "markmonitor"
},
"appcenter": {
"name": "Microsoft App Center",
"categoryId": 5,
"url": "https://appcenter.ms/",
"companyId": null
},
"unity_ads": {
"name": "Unity Ads",
"categoryId": 4,
"url": "https://unity.com/solutions/mobile-business/monetize-your-game",
"companyId": null
},
"azure": {
"name": "Microsoft Azure",
"categoryId": 10,
"url": "https://azure.microsoft.com/",
"companyId": "microsoft"
},
"button": {
"name": "Button",
"categoryId": 4,
"url": "https://www.usebutton.com/",
"companyId": null
},
"netflix": {
"name": "Netflix",
"categoryId": 8,
"url": "https://www.netflix.com/",
"companyId": null
},
"mail.ru_banner": {
"name": "Mail.Ru Banner Network",
"categoryId": 4,
"url": "http://mail.ru/",
"companyId": "vk"
},
"mail.ru_counter": {
"name": "Mail.Ru Counter",
"categoryId": 2,
"url": "http://mail.ru/",
"companyId": "vk"
},
"mail.ru_group": {
"name": "Mail.Ru Group",
"categoryId": 7,
"url": "http://mail.ru/",
"companyId": "vk"
}
},
"trackerDomains": {
"akadns.net": "akamai_technologies",
"akamaiedge.net": "akamai_technologies",
"apple.com": "apple",
"apple.news": "apple",
"apple-dns.net": "apple",
"aaplimg.com": "apple",
"icloud.com": "apple",
"mzstatic.com": "apple",
"iadsdk.apple.com": "apple_ads",
"graph.facebook.com": "facebook_audience",
"crashlytics.com": "crashlytics",
"flurry.com": "flurry",
"hockeyapp.net": "hockeyapp",
"app-measurement.com": "firebase",
"appsflyer.com": "appsflyer",
"appmetrica.yandex.com": "yandex_appmetrica",
"adjust.com": "adjust",
"mobileapptracking.com": "branch",
"edgecastcdn.net": "markmonitor",
"appcenter.ms": "appcenter",
"unityads.unity3d.com": "unity_ads",
"azure.com": "azure",
"bttn.io": "button"
}
}

610
client/src/helpers/trackers/trackers.json
View File

@ -1,5 +1,5 @@
{
"timeUpdated": "2023-02-09T12:31:34.007Z",
"timeUpdated": "2023-03-08T00:09:48.692Z",
"categories": {
"0": "audio_video_player",
"1": "comments",
@ -6772,55 +6772,64 @@
"name": "Facebook",
"categoryId": 4,
"url": "https://www.facebook.com",
"companyId": "facebook"
"companyId": "meta",
"source": "AdGuard"
},
"facebook_beacon": {
"name": "Facebook Beacon",
"categoryId": 7,
"url": "http://www.facebook.com/beacon/faq.php",
"companyId": "facebook"
"companyId": "meta",
"source": "AdGuard"
},
"facebook_cdn": {
"name": "Facebook CDN",
"categoryId": 9,
"url": "https://www.facebook.com",
"companyId": "facebook"
"companyId": "meta",
"source": "AdGuard"
},
"facebook_connect": {
"name": "Facebook Connect",
"categoryId": 6,
"url": "https://developers.facebook.com/connect.php",
"companyId": "facebook"
"companyId": "meta",
"source": "AdGuard"
},
"facebook_conversion_tracking": {
"name": "Facebook Conversion Tracking",
"categoryId": 4,
"url": "http://www.facebook.com/",
"companyId": "facebook"
"companyId": "meta",
"source": "AdGuard"
},
"facebook_custom_audience": {
"name": "Facebook Custom Audience",
"categoryId": 4,
"url": "https://www.facebook.com",
"companyId": "facebook"
"companyId": "meta",
"source": "AdGuard"
},
"facebook_graph": {
"name": "Facebook Social Graph",
"categoryId": 7,
"url": "https://developers.facebook.com/docs/reference/api/",
"companyId": "facebook"
"companyId": "meta",
"source": "AdGuard"
},
"facebook_impressions": {
"name": "Facebook Impressions",
"categoryId": 4,
"url": "https://www.facebook.com/",
"companyId": "facebook"
"companyId": "meta",
"source": "AdGuard"
},
"facebook_social_plugins": {
"name": "Facebook Social Plugins",
"categoryId": 7,
"url": "https://developers.facebook.com/plugins",
"companyId": "facebook"
"companyId": "meta",
"source": "AdGuard"
},
"facetz.dca": {
"name": "Facetz.DCA",
@ -8909,7 +8918,8 @@
"name": "Instagram",
"categoryId": 8,
"url": "https://www.facebook.com/",
"companyId": "facebook"
"companyId": "meta",
"source": "AdGuard"
},
"instant_check_mate": {
"name": "Instant Check Mate",
@ -19215,11 +19225,39 @@
"url": "http://www.zypmedia.com/",
"companyId": "zypmedia"
},
"slack": {
"name": "Slack",
"adguard_dns": {
"name": "AdGuard DNS",
"categoryId": 8,
"url": "https://www.slack.com/",
"companyId": "salesforce",
"url": "https://adguard-dns.io/",
"companyId": "adguard",
"source": "AdGuard"
},
"adguard_vpn": {
"name": "AdGuard VPN",
"categoryId": 8,
"url": "https://adguard-vpn.com/",
"companyId": "adguard",
"source": "AdGuard"
},
"appcenter": {
"name": "Microsoft App Center",
"categoryId": 5,
"url": "https://appcenter.ms/",
"companyId": null,
"source": "AdGuard"
},
"alibaba_cloud": {
"name": "Alibaba Cloud",
"categoryId": 10,
"url": "https://www.alibabacloud.com/",
"companyId": "alibaba",
"source": "AdGuard"
},
"alibaba_ucbrowser": {
"name": "UC Browser",
"categoryId": 8,
"url": "https://ucweb.com/",
"companyId": "alibaba",
"source": "AdGuard"
},
"apple": {
@ -19236,53 +19274,25 @@
"companyId": "apple",
"source": "AdGuard"
},
"facebook_audience": {
"name": "Facebook Audience Network",
"categoryId": 4,
"url": "https://www.facebook.com/business/products/audience-network",
"companyId": "facebook",
"azure": {
"name": "Microsoft Azure",
"categoryId": 10,
"url": "https://azure.microsoft.com/",
"companyId": "microsoft",
"source": "AdGuard"
},
"crashlytics": {
"name": "Crashlytics",
"categoryId": 101,
"url": "https://crashlytics.com/",
"companyId": null,
"source": "AdGuard"
},
"showrss": {
"name": "showRSS",
"azure_blob_storage": {
"name": "Azure Blob Storage",
"categoryId": 8,
"url": "https://showrss.info/",
"companyId": "showrss",
"url": "https://azure.microsoft.com/en-us/products/storage/blobs",
"companyId": "microsoft",
"source": "AdGuard"
},
"hockeyapp": {
"name": "HockeyApp",
"categoryId": 101,
"url": "https://hockeyapp.net/",
"companyId": null,
"source": "AdGuard"
},
"google_trust_services": {
"name": "Google Trust Services",
"categoryId": 5,
"url": "https://pki.goog/",
"companyId": "google",
"source": "AdGuard"
},
"firebase": {
"name": "Firebase",
"categoryId": 101,
"url": "https://firebase.google.com/",
"companyId": "google",
"source": "AdGuard"
},
"yandex_appmetrica": {
"name": "Yandex AppMetrica",
"categoryId": 101,
"url": "https://appmetrica.yandex.com/",
"companyId": "yandex",
"bitwarden": {
"name": "Bitwarden",
"categoryId": 8,
"url": "https://bitwarden.com/",
"companyId": "bitwarden",
"source": "AdGuard"
},
"branch": {
@ -19292,18 +19302,25 @@
"companyId": "branch_metrics_inc",
"source": "AdGuard"
},
"qualcomm": {
"name": "Qualcomm",
"categoryId": 8,
"url": "https://www.qualcomm.com/",
"companyId": "qualcomm",
"button": {
"name": "Button",
"categoryId": 4,
"url": "https://www.usebutton.com/",
"companyId": null,
"source": "AdGuard"
},
"solaredge": {
"name": "SolarEdge Technologies, Inc.",
"categoryId": 8,
"url": "https://www.solaredge.com/",
"companyId": "solaredge",
"crashlytics": {
"name": "Crashlytics",
"categoryId": 101,
"url": "https://crashlytics.com/",
"companyId": null,
"source": "AdGuard"
},
"edgio": {
"name": "Edgio",
"categoryId": 9,
"url": "https://edg.io/",
"companyId": "edgio",
"source": "AdGuard"
},
"element": {
@ -19313,39 +19330,46 @@
"companyId": "element",
"source": "AdGuard"
},
"outlook": {
"name": "Microsoft Outlook",
"facebook_audience": {
"name": "Facebook Audience Network",
"categoryId": 4,
"url": "https://www.facebook.com/business/products/audience-network",
"companyId": "meta",
"source": "AdGuard"
},
"firebase": {
"name": "Firebase",
"categoryId": 101,
"url": "https://firebase.google.com/",
"companyId": "google",
"source": "AdGuard"
},
"gmail": {
"name": "Gmail",
"categoryId": 13,
"url": "https://outlook.live.com/",
"companyId": "microsoft",
"url": "https://mail.google.com/",
"companyId": "google",
"source": "AdGuard"
},
"appcenter": {
"name": "Microsoft App Center",
"google_trust_services": {
"name": "Google Trust Services",
"categoryId": 5,
"url": "https://appcenter.ms/",
"url": "https://pki.goog/",
"companyId": "google",
"source": "AdGuard"
},
"hockeyapp": {
"name": "HockeyApp",
"categoryId": 101,
"url": "https://hockeyapp.net/",
"companyId": null,
"source": "AdGuard"
},
"unity_ads": {
"name": "Unity Ads",
"categoryId": 4,
"url": "https://unity.com/solutions/mobile-business/monetize-your-game",
"companyId": null,
"source": "AdGuard"
},
"azure": {
"name": "Microsoft Azure",
"categoryId": 10,
"url": "https://azure.microsoft.com/",
"companyId": "microsoft",
"source": "AdGuard"
},
"button": {
"name": "Button",
"categoryId": 4,
"url": "https://www.usebutton.com/",
"companyId": null,
"kik": {
"name": "Kik",
"categoryId": 7,
"url": "https://kik.com/",
"companyId": "kik",
"source": "AdGuard"
},
"lets_encrypt": {
@ -19355,11 +19379,11 @@
"companyId": "lets_encrypt",
"source": "AdGuard"
},
"plex": {
"name": "Plex",
"categoryId": 0,
"url": "https://www.plex.tv/",
"companyId": "plex",
"lgtv": {
"name": "LG TV",
"categoryId": 8,
"url": "https://www.lg.com/",
"companyId": "lgcorp",
"source": "AdGuard"
},
"matrix": {
@ -19368,6 +19392,195 @@
"url": "https://matrix.org/",
"companyId": "matrix",
"source": "AdGuard"
},
"medialab": {
"name": "MediaLab.AI Inc.",
"categoryId": 8,
"url": "https://medialab.la/",
"companyId": "medialab",
"source": "AdGuard"
},
"meganz": {
"name": "Mega Ltd.",
"categoryId": 8,
"url": "https://mega.io/",
"companyId": "meganz",
"source": "AdGuard"
},
"msedge": {
"name": "Microsoft Edge",
"categoryId": 8,
"url": "https://www.microsoft.com/en-us/edge",
"companyId": "microsoft",
"source": "AdGuard"
},
"mozilla": {
"name": "Mozilla Foundation",
"categoryId": 8,
"url": "https://www.mozilla.org/",
"companyId": "mozilla",
"source": "AdGuard"
},
"nab": {
"name": "National Australia Bank",
"categoryId": 8,
"url": "https://www.nab.com.au/",
"companyId": "nab",
"source": "AdGuard"
},
"notion": {
"name": "Notion",
"categoryId": 8,
"url": "https://www.notion.so/",
"companyId": "notion",
"source": "AdGuard"
},
"ntppool": {
"name": "Network Time Protocol",
"categoryId": 5,
"url": "https://ntp.org/",
"companyId": "ntppool",
"source": "AdGuard"
},
"oppo": {
"name": "OPPO",
"categoryId": 101,
"url": "https://www.oppo.com/",
"companyId": "oppo",
"source": "AdGuard"
},
"outlook": {
"name": "Microsoft Outlook",
"categoryId": 13,
"url": "https://outlook.live.com/",
"companyId": "microsoft",
"source": "AdGuard"
},
"oztam": {
"name": "OzTAM",
"categoryId": 8,
"url": "https://oztam.com.au/",
"companyId": "oztam",
"source": "AdGuard"
},
"plex": {
"name": "Plex",
"categoryId": 0,
"url": "https://www.plex.tv/",
"companyId": "plex",
"source": "AdGuard"
},
"qualcomm": {
"name": "Qualcomm",
"categoryId": 8,
"url": "https://www.qualcomm.com/",
"companyId": "qualcomm",
"source": "AdGuard"
},
"recaptcha": {
"name": "reCAPTCHA",
"categoryId": 8,
"url": "https://www.google.com/recaptcha/about/",
"companyId": "google",
"source": "AdGuard"
},
"sectigo": {
"name": "Sectigo Limited",
"categoryId": 5,
"url": "https://www.solaredge.com/",
"companyId": "sectigo",
"source": "AdGuard"
},
"showrss": {
"name": "showRSS",
"categoryId": 8,
"url": "https://showrss.info/",
"companyId": "showrss",
"source": "AdGuard"
},
"similarweb": {
"name": "SimilarWeb",
"categoryId": 6,
"url": "https://www.similarweb.com/",
"companyId": "similarweb",
"source": "AdGuard"
},
"slack": {
"name": "Slack",
"categoryId": 8,
"url": "https://www.slack.com/",
"companyId": "salesforce",
"source": "AdGuard"
},
"solaredge": {
"name": "SolarEdge Technologies, Inc.",
"categoryId": 8,
"url": "https://www.solaredge.com/",
"companyId": "solaredge",
"source": "AdGuard"
},
"telstra": {
"name": "Telstra",
"categoryId": 8,
"url": "https://www.telstra.com.au/",
"companyId": "telstra",
"source": "AdGuard"
},
"unity_ads": {
"name": "Unity Ads",
"categoryId": 4,
"url": "https://unity.com/solutions/mobile-business/monetize-your-game",
"companyId": null,
"source": "AdGuard"
},
"vscode": {
"name": "Visual Studio Code",
"categoryId": 8,
"url": "https://code.visualstudio.com/",
"companyId": "microsoft",
"source": "AdGuard"
},
"whatsapp": {
"name": "WhatsApp",
"categoryId": 8,
"url": "https://www.whatsapp.com/",
"companyId": "meta",
"source": "AdGuard"
},
"windows_maps": {
"name": "Windows Maps",
"categoryId": 8,
"url": "https://www.microsoft.com/store/apps/9wzdncrdtbvb",
"companyId": "microsoft",
"source": "AdGuard"
},
"windows_notifications": {
"name": "The Windows Push Notification Services",
"categoryId": 8,
"url": "https://learn.microsoft.com/en-us/windows/apps/design/shell/tiles-and-notifications/windows-push-notification-services--wns--overview",
"companyId": "microsoft",
"source": "AdGuard"
},
"windows_time": {
"name": "Windows Time Service",
"categoryId": 8,
"url": "https://learn.microsoft.com/en-us/windows-server/networking/windows-time-service/how-the-windows-time-service-works",
"companyId": "microsoft",
"source": "AdGuard"
},
"windowsupdate": {
"name": "Windows Update",
"categoryId": 9,
"url": "https://support.microsoft.com/en-us/windows/windows-update-faq-8a903416-6f45-0718-f5c7-375e92dddeb2",
"companyId": "microsoft",
"source": "AdGuard"
},
"yandex_appmetrica": {
"name": "Yandex AppMetrica",
"categoryId": 101,
"url": "https://appmetrica.yandex.com/",
"companyId": "yandex",
"source": "AdGuard"
}
},
"trackerDomains": {
@ -23698,53 +23911,190 @@
"zwaar.net": "zwaar",
"zwaar.org": "zwaar",
"extend.tv": "zypmedia",
"slack.com": "slack",
"slackb.com": "slack",
"slack-edge.com": "slack",
"slack-imgs.com": "slack",
"adtidy.org": "adguard",
"agrd.io": "adguard",
"adguard.app": "adguard",
"adguard.io": "adguard",
"adguard.org": "adguard",
"adguard-dns.com": "adguard",
"adguard-dns.io": "adguard",
"adguard-vpn.com": "adguard",
"adguardvpn.com": "adguard",
"adguard-vpn.online": "adguard",
"nflximg.com": "netflix",
"element.io": "element",
"riot.im": "element",
"adguard-dns.com": "adguard_dns",
"adguard-dns.io": "adguard_dns",
"adguardvpn.com": "adguard_vpn",
"adguard-vpn.com": "adguard_vpn",
"adguard-vpn.online": "adguard_vpn",
"adjust.net.in": "adjust",
"adj.st": "adjust",
"adjust.io": "adjust",
"adjust.world": "adjust",
"apptrace.com": "adjust",
"akadns.net": "akamai_technologies",
"akamaiedge.net": "akamai_technologies",
"akaquill.net": "akamai_technologies",
"aliapp.org": "alibaba.com",
"alibabachengdun.com": "alibaba.com",
"alibabausercontent.com": "alibaba.com",
"aliexpress.com": "alibaba.com",
"alikunlun.com": "alibaba.com",
"aliyuncs.com": "alibaba.com",
"alibabacloud.com": "alibaba_cloud",
"alibabadns.com": "alibaba_cloud",
"aliyun.com": "alibaba_cloud",
"ucweb.com": "alibaba_ucbrowser",
"alipayobjects.com": "alipay.com",
"amazoncrl.com": "amazon",
"aamazoncognito.com": "amazon",
"amazonbrowserapp.es": "amazon",
"amazonbrowserapp.co.uk": "amazon",
"amazon.sa": "amazon",
"amazon.nl": "amazon",
"amazon.in": "amazon",
"amazon.com.mx": "amazon",
"amazon.com.au": "amazon",
"amazon-corp.com": "amazon",
"a2z.com": "amazon",
"amazontrust.com": "amazon_cdn",
"associates-amazon.com": "amazon_cdn",
"amazonpay.in": "amazon_payments",
"amazonvideo.com": "amazon_video",
"taobao.com": "taobao",
"appcenter.ms": "appcenter",
"iadsdk.apple.com": "apple_ads",
"me.com": "apple",
"apple.news": "apple",
"apple-dns.net": "apple",
"aaplimg.com": "apple",
"icloud.com": "apple",
"itunes.com": "apple",
"icloud-content.com": "apple",
"mzstatic.com": "apple",
"matrix.org": "matrix",
"l-msedge.net": "microsoft",
"iadsdk.apple.com": "apple_ads",
"showrss.info": "showrss",
"solaredge.com": "solaredge",
"crashlytics.com": "crashlytics",
"flurry.com": "flurry",
"hockeyapp.net": "hockeyapp",
"app-measurement.com": "firebase",
"appmetrica.yandex.com": "yandex_appmetrica",
"letsencrypt.org": "lets_encrypt",
"lencr.org": "lets_encrypt",
"mobileapptracking.com": "branch",
"plex.tv": "plex",
"plex.direct": "plex",
"edgecastcdn.net": "markmonitor",
"appcenter.ms": "appcenter",
"unityads.unity3d.com": "unity_ads",
"cdn-apple.com": "apple",
"apple-mapkit.com": "apple",
"icons.axm-usercontent-apple.com": "apple",
"apple-cloudkit.com": "apple",
"apzones.com": "apple",
"apple-livephotoskit.com": "apple",
"safebrowsing.apple": "apple",
"safebrowsing.g.applimg.com": "apple",
"applvn.com": "applovin",
"applovin.com": "applovin",
"blob.core.windows.net": "azure_blob_storage",
"azure.com": "azure",
"trafficmanager.net": "azure",
"hotmail.com": "outlook",
"bitwarden.com": "bitwarden",
"mobileapptracking.com": "branch",
"bttn.io": "button",
"cloudflare-dns.com": "cloudflare",
"crashlytics.com": "crashlytics",
"phicdn.net": "digicert_trust_seal",
"alphacdn.net": "edgio",
"edg.io": "edgio",
"edgecast.com": "edgio",
"edgecastcdn.net": "edgio",
"edgecastdns.net": "edgio",
"sigmacdn.net": "edgio",
"element.io": "element",
"riot.im": "element",
"app-measurement.com": "firebase",
"flipboard.com": "flipboard",
"flurry.com": "flurry",
"ghcr.io": "github",
"github.dev": "github",
"gmail.com": "gmail",
"googlehosted.com": "google_appspot",
"gvt1.com": "google_servers",
"gvt2.com": "google_servers",
"gvt3.com": "google_servers",
"pki.goog": "google_trust_services",
"hockeyapp.net": "hockeyapp",
"kik.com": "kik",
"apikik.com": "kik",
"kik-live.com": "kik",
"slatic.net": "lazada",
"lencr.org": "lets_encrypt",
"letsencrypt.org": "lets_encrypt",
"lgsmartad.com": "lgtv",
"lgtvcommon.com": "lgtv",
"lgtvsdp.com": "lgtv",
"lge.com": "lgtv",
"lg.com": "lgtv",
"markmonitor.com": "markmonitor",
"matrix.org": "matrix",
"medialab.la": "medialab",
"media-lab.ai": "medialab",
"mega.co.nz": "meganz",
"mega.io": "meganz",
"mega.nz": "meganz",
"e-msedge.net": "msedge",
"l-msedge.net": "msedge",
"firefox.com": "mozilla",
"mozaws.net": "mozilla",
"mozgcp.net": "mozilla",
"mozilla.com": "mozilla",
"mozilla.net": "mozilla",
"mozilla.org": "mozilla",
"nflximg.com": "netflix",
"nab.com": "nab",
"nab.com.au": "nab",
"nab.net": "nab",
"nabgroup.com": "nab",
"national.com.au": "nab",
"nationalaustraliabank.com.au": "nab",
"nationalbank.com.au": "nab",
"notion.so": "notion",
"ntp.org": "ntppool",
"ntppool.org": "ntppool",
"oppomobile.com": "oppo",
"heytapmobi.com": "oppo",
"heytapmobile.com": "oppo",
"heytapdl.com": "oppo",
"allawnos.com": "oppo",
"allawntech.com": "oppo",
"hotmail.com": "outlook",
"outlook.com": "outlook",
"oztam.com.au": "oztam",
"plex.tv": "plex",
"plex.direct": "plex",
"xtracloud.net": "qualcomm",
"qualcomm.com": "qualcomm"
"qualcomm.com": "qualcomm",
"recaptcha.net": "recaptcha",
"sectigo.com": "sectigo",
"showrss.info": "showrss",
"similarweb.io": "similarweb",
"similarweb.com": "similarweb",
"slack.com": "slack",
"slackb.com": "slack",
"slack-edge.com": "slack",
"slack-imgs.com": "slack",
"addlive.io": "snap",
"feelinsonice.com": "snap",
"sc-cdn.net": "snap",
"sc-corp.net": "snap",
"sc-gw.com": "snap",
"sc-jpl.com": "snap",
"sc-prod.net": "snap",
"snap-dev.net": "snap",
"snapads.com": "snap",
"snapkit.com": "snap",
"solaredge.com": "solaredge",
"telstra.com.au": "telstra",
"telstra.com": "telstra",
"usertrust.com": "trustlogo",
"unityads.unity3d.com": "unity_ads",
"exp-tas.com": "vscode",
"vscode-unpkg.net": "vscode",
"v0cdn.net": "vscode",
"vscode-cdn.net": "vscode",
"whatsapp.net": "whatsapp",
"whatsapp.com": "whatsapp",
"maps.windows.com": "windows_maps",
"client.wns.windows.com": "windows_notifications",
"time.windows.com": "windows_time",
"windowsupdate.com": "windowsupdate",
"ya.ru": "yandex",
"yandex.by": "yandex",
"yandex.com": "yandex",
"yandex.com.tr": "yandex",
"yandex.fr": "yandex",
"yandex.kz": "yandex",
"appmetrica.yandex.com": "yandex_appmetrica"
}
}

23534
client/src/helpers/trackers/whotracksme.json
View File

File diff suppressed because it is too large Load Diff

12
go.mod
View File

@ -4,8 +4,8 @@ go 1.19
require (
// TODO(a.garipov): Use v0.48.0 when it's released.
github.com/AdguardTeam/dnsproxy v0.47.1-0.20230207130636-533058b17239
github.com/AdguardTeam/golibs v0.11.4
github.com/AdguardTeam/dnsproxy v0.48.0
github.com/AdguardTeam/golibs v0.12.0
github.com/AdguardTeam/urlfilter v0.16.1
github.com/NYTimes/gziphandler v1.1.1
github.com/ameshkov/dnscrypt/v2 v2.2.5
@ -51,16 +51,16 @@ require (
github.com/josharian/native v1.1.0 // indirect
github.com/mdlayher/packet v1.1.1 // indirect
github.com/mdlayher/socket v0.4.0 // indirect
github.com/onsi/ginkgo/v2 v2.8.1 // indirect
github.com/onsi/ginkgo/v2 v2.8.3 // indirect
github.com/patrickmn/go-cache v2.1.0+incompatible // indirect
github.com/pierrec/lz4/v4 v4.1.17 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/quic-go/qpack v0.4.0 // indirect
github.com/quic-go/qtls-go1-18 v0.2.0 // indirect
github.com/quic-go/qtls-go1-19 v0.2.0 // indirect
github.com/quic-go/qtls-go1-20 v0.1.0 // indirect
github.com/u-root/uio v0.0.0-20230215032506-9aa6f7e2d72c // indirect
github.com/quic-go/qtls-go1-19 v0.2.1 // indirect
github.com/quic-go/qtls-go1-20 v0.1.1 // indirect
github.com/u-root/uio v0.0.0-20230220225925-ffce2a382923 // indirect
golang.org/x/mod v0.8.0 // indirect
golang.org/x/sync v0.1.0 // indirect
golang.org/x/text v0.7.0 // indirect

26
go.sum
View File

@ -1,9 +1,9 @@
github.com/AdguardTeam/dnsproxy v0.47.1-0.20230207130636-533058b17239 h1:n1oOiywOvdeqWLto809bK1rK1EPDkpaSfT/r1OiCVaQ=
github.com/AdguardTeam/dnsproxy v0.47.1-0.20230207130636-533058b17239/go.mod h1:+Sdi5ISrjDFbeCsKNqzcC1Ag7pJ5Hh9y+UBNb3dfqJ4=
github.com/AdguardTeam/dnsproxy v0.48.0 h1:sGViYy2pV0cEp2zCsxPjFd9rlgD0+yELpIeLkBxHAoI=
github.com/AdguardTeam/dnsproxy v0.48.0/go.mod h1:9OHoeaVod+moWwrLjHF95RQnFWGi/6B1tfKsxWc/yGE=
github.com/AdguardTeam/golibs v0.4.0/go.mod h1:skKsDKIBB7kkFflLJBpfGX+G8QFTx0WKUzB6TIgtUj4=
github.com/AdguardTeam/golibs v0.10.4/go.mod h1:rSfQRGHIdgfxriDDNgNJ7HmE5zRoURq8R+VdR81Zuzw=
github.com/AdguardTeam/golibs v0.11.4 h1:IltyvxwCTN+xxJF5sh6VadF8Zfbf8elgCm9dgijSVzM=
github.com/AdguardTeam/golibs v0.11.4/go.mod h1:87bN2x4VsTritptE3XZg9l8T6gznWsIxHBcQ1DeRIXA=
github.com/AdguardTeam/golibs v0.12.0 h1:z4Q3Mz0pHJ2Zag4B0RBaIXEUue1TPOKkbRiYkwC4r7I=
github.com/AdguardTeam/golibs v0.12.0/go.mod h1:87bN2x4VsTritptE3XZg9l8T6gznWsIxHBcQ1DeRIXA=
github.com/AdguardTeam/gomitmproxy v0.2.0/go.mod h1:Qdv0Mktnzer5zpdpi5rAwixNJzW2FN91LjKJCkVbYGU=
github.com/AdguardTeam/urlfilter v0.16.1 h1:ZPi0rjqo8cQf2FVdzo6cqumNoHZx2KPXj2yZa1A5BBw=
github.com/AdguardTeam/urlfilter v0.16.1/go.mod h1:46YZDOV1+qtdRDuhZKVPSSp7JWWes0KayqHrKAFBdEI=
@ -108,9 +108,9 @@ github.com/miekg/dns v1.1.50 h1:DQUfb9uc6smULcREF09Uc+/Gd46YWqJd5DbpPE9xkcA=
github.com/miekg/dns v1.1.50/go.mod h1:e3IlAVfNqAllflbibAZEWOXOQ+Ynzk/dDozDxY7XnME=
github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e h1:fD57ERR4JtEqsWbfPhv4DMiApHyliiK5xCTNVSPiaAs=
github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno=
github.com/onsi/ginkgo/v2 v2.8.1 h1:xFTEVwOFa1D/Ty24Ws1npBWkDYEV9BqZrsDxVrVkrrU=
github.com/onsi/ginkgo/v2 v2.8.1/go.mod h1:N1/NbDngAFcSLdyZ+/aYTYGSlq9qMCS/cNKGJjy+csc=
github.com/onsi/gomega v1.26.0 h1:03cDLK28U6hWvCAns6NeydX3zIm4SF3ci69ulidS32Q=
github.com/onsi/ginkgo/v2 v2.8.3 h1:RpbK1G8nWPNaCVFBWsOGnEQQGgASi6b8fxcWBvDYjxQ=
github.com/onsi/ginkgo/v2 v2.8.3/go.mod h1:6OaUA8BCi0aZfmzYT/q9AacwTzDpNbxILUT+TlBq6MY=
github.com/onsi/gomega v1.27.0 h1:QLidEla4bXUuZVFa4KX6JHCsuGgbi85LC/pCHrt/O08=
github.com/patrickmn/go-cache v2.1.0+incompatible h1:HRMgzkcYKYpi3C8ajMPV8OFXaaRUnok+kx1WdO15EQc=
github.com/patrickmn/go-cache v2.1.0+incompatible/go.mod h1:3Qf8kWWT7OJRJbdiICTKqZju1ZixQ/KpMGzzAfe6+WQ=
github.com/pierrec/lz4/v4 v4.1.14/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4=
@ -125,10 +125,10 @@ github.com/quic-go/qpack v0.4.0 h1:Cr9BXA1sQS2SmDUWjSofMPNKmvF6IiIfDRmgU0w1ZCo=
github.com/quic-go/qpack v0.4.0/go.mod h1:UZVnYIfi5GRk+zI9UMaCPsmZ2xKJP7XBUvVyT1Knj9A=
github.com/quic-go/qtls-go1-18 v0.2.0 h1:5ViXqBZ90wpUcZS0ge79rf029yx0dYB0McyPJwqqj7U=
github.com/quic-go/qtls-go1-18 v0.2.0/go.mod h1:moGulGHK7o6O8lSPSZNoOwcLvJKJ85vVNc7oJFD65bc=
github.com/quic-go/qtls-go1-19 v0.2.0 h1:Cvn2WdhyViFUHoOqK52i51k4nDX8EwIh5VJiVM4nttk=
github.com/quic-go/qtls-go1-19 v0.2.0/go.mod h1:ySOI96ew8lnoKPtSqx2BlI5wCpUVPT05RMAlajtnyOI=
github.com/quic-go/qtls-go1-20 v0.1.0 h1:d1PK3ErFy9t7zxKsG3NXBJXZjp/kMLoIb3y/kV54oAI=
github.com/quic-go/qtls-go1-20 v0.1.0/go.mod h1:JKtK6mjbAVcUTN/9jZpvLbGxvdWIKS8uT7EiStoU1SM=
github.com/quic-go/qtls-go1-19 v0.2.1 h1:aJcKNMkH5ASEJB9FXNeZCyTEIHU1J7MmHyz1Q1TSG1A=
github.com/quic-go/qtls-go1-19 v0.2.1/go.mod h1:ySOI96ew8lnoKPtSqx2BlI5wCpUVPT05RMAlajtnyOI=
github.com/quic-go/qtls-go1-20 v0.1.1 h1:KbChDlg82d3IHqaj2bn6GfKRj84Per2VGf5XV3wSwQk=
github.com/quic-go/qtls-go1-20 v0.1.1/go.mod h1:JKtK6mjbAVcUTN/9jZpvLbGxvdWIKS8uT7EiStoU1SM=
github.com/quic-go/quic-go v0.32.0 h1:lY02md31s1JgPiiyfqJijpu/UX/Iun304FI3yUqX7tA=
github.com/quic-go/quic-go v0.32.0/go.mod h1:/fCsKANhQIeD5l76c2JFU+07gVE3KaA0FP+0zMWwfwo=
github.com/shirou/gopsutil/v3 v3.21.8 h1:nKct+uP0TV8DjjNiHanKf8SAuub+GNsbrOtM9Nl9biA=
@ -155,8 +155,8 @@ github.com/tklauser/go-sysconf v0.3.9/go.mod h1:11DU/5sG7UexIrp/O6g35hrWzu0JxlwQ
github.com/tklauser/numcpus v0.3.0 h1:ILuRUQBtssgnxw0XXIjKUC56fgnOrFoQQ/4+DeU2biQ=
github.com/tklauser/numcpus v0.3.0/go.mod h1:yFGUr7TUHQRAhyqBcEg0Ge34zDBAsIvJJcyE6boqnA8=
github.com/u-root/uio v0.0.0-20221213070652-c3537552635f/go.mod h1:IogEAUBXDEwX7oR/BMmCctShYs80ql4hF0ySdzGxf7E=
github.com/u-root/uio v0.0.0-20230215032506-9aa6f7e2d72c h1:PHoGTnweZP+KIg/8Zc6+iOesrIF5yHkpb4GBDxHm7yE=
github.com/u-root/uio v0.0.0-20230215032506-9aa6f7e2d72c/go.mod h1:eLL9Nub3yfAho7qB0MzZizFhTU2QkLeoVsWdHtDW264=
github.com/u-root/uio v0.0.0-20230220225925-ffce2a382923 h1:tHNk7XK9GkmKUR6Gh8gVBKXc2MVSZ4G/NnWLtzw4gNA=
github.com/u-root/uio v0.0.0-20230220225925-ffce2a382923/go.mod h1:eLL9Nub3yfAho7qB0MzZizFhTU2QkLeoVsWdHtDW264=
github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
go.etcd.io/bbolt v1.3.7 h1:j+zJOnnEjF/kyHlDDgGnVL/AIqIJPq8UoB2GSNfkUfQ=
go.etcd.io/bbolt v1.3.7/go.mod h1:N9Mkw9X8x5fupy0IKsmuqVtoGDyxsaDlbk4Rd05IAQw=

2
internal/aghnet/arpdb_bsd.go
View File

@ -67,7 +67,7 @@ func parseArpA(sc *bufio.Scanner, lenHint int) (ns []Neighbor) {
}
host := fields[0]
err = netutil.ValidateDomainName(host)
err = netutil.ValidateHostname(host)
if err != nil {
log.Debug("arpdb: parsing arp output: host: %s", err)
} else {

2
internal/aghnet/arpdb_linux.go
View File

@ -198,7 +198,7 @@ func parseArpA(sc *bufio.Scanner, lenHint int) (ns []Neighbor) {
}
host := fields[0]
if verr := netutil.ValidateDomainName(host); verr != nil {
if verr := netutil.ValidateHostname(host); verr != nil {
log.Debug("arpdb: parsing arp output: host: %s", verr)
} else {
n.Name = host

2
internal/aghnet/hostscontainer.go
View File

@ -343,7 +343,7 @@ func (hp *hostsParser) parseLine(line string) (ip netip.Addr, hosts []string) {
// See https://github.com/AdguardTeam/AdGuardHome/issues/3946.
//
// TODO(e.burkov): Investigate if hosts may contain DNS-SD domains.
err = netutil.ValidateDomainName(f)
err = netutil.ValidateHostname(f)
if err != nil {
log.Error("%s: host %q is invalid, ignoring", hostsContainerPref, f)

6
internal/dhcpd/config.go
View File

@ -45,8 +45,10 @@ type DHCPServer interface {
AddStaticLease(l *Lease) (err error)
// RemoveStaticLease - remove a static lease
RemoveStaticLease(l *Lease) (err error)
// FindMACbyIP - find a MAC address by IP address in the currently active DHCP leases
FindMACbyIP(ip net.IP) net.HardwareAddr
// FindMACbyIP returns a MAC address by the IP address of its lease, if
// there is one.
FindMACbyIP(ip netip.Addr) (mac net.HardwareAddr)
// WriteDiskConfig4 - copy disk configuration
WriteDiskConfig4(c *V4ServerConf)

23
internal/dhcpd/dhcpd.go
View File

@ -5,6 +5,7 @@ import (
"encoding/json"
"fmt"
"net"
"net/netip"
"path/filepath"
"time"
@ -42,6 +43,10 @@ type Lease struct {
Hostname string `json:"hostname"`
HWAddr net.HardwareAddr `json:"mac"`
// IP is the IP address leased to the client.
//
// TODO(a.garipov): Migrate leases.db and use netip.Addr.
IP net.IP `json:"ip"`
}
@ -160,7 +165,7 @@ type Interface interface {
Leases(flags GetLeasesFlags) (leases []*Lease)
SetOnLeaseChanged(onLeaseChanged OnLeaseChangedT)
FindMACbyIP(ip net.IP) (mac net.HardwareAddr)
FindMACbyIP(ip netip.Addr) (mac net.HardwareAddr)
WriteDiskConfig(c *ServerConfig)
}
@ -174,7 +179,7 @@ type MockInterface struct {
OnEnabled func() (ok bool)
OnLeases func(flags GetLeasesFlags) (leases []*Lease)
OnSetOnLeaseChanged func(f OnLeaseChangedT)
OnFindMACbyIP func(ip net.IP) (mac net.HardwareAddr)
OnFindMACbyIP func(ip netip.Addr) (mac net.HardwareAddr)
OnWriteDiskConfig func(c *ServerConfig)
}
@ -195,8 +200,10 @@ func (s *MockInterface) Leases(flags GetLeasesFlags) (ls []*Lease) { return s.On
// SetOnLeaseChanged implements the Interface for *MockInterface.
func (s *MockInterface) SetOnLeaseChanged(f OnLeaseChangedT) { s.OnSetOnLeaseChanged(f) }
// FindMACbyIP implements the Interface for *MockInterface.
func (s *MockInterface) FindMACbyIP(ip net.IP) (mac net.HardwareAddr) { return s.OnFindMACbyIP(ip) }
// FindMACbyIP implements the [Interface] for *MockInterface.
func (s *MockInterface) FindMACbyIP(ip netip.Addr) (mac net.HardwareAddr) {
return s.OnFindMACbyIP(ip)
}
// WriteDiskConfig implements the Interface for *MockInterface.
func (s *MockInterface) WriteDiskConfig(c *ServerConfig) { s.OnWriteDiskConfig(c) }
@ -375,11 +382,13 @@ func (s *server) Leases(flags GetLeasesFlags) (leases []*Lease) {
return append(s.srv4.GetLeases(flags), s.srv6.GetLeases(flags)...)
}
// FindMACbyIP - find a MAC address by IP address in the currently active DHCP leases
func (s *server) FindMACbyIP(ip net.IP) net.HardwareAddr {
if ip.To4() != nil {
// FindMACbyIP returns a MAC address by the IP address of its lease, if there is
// one.
func (s *server) FindMACbyIP(ip netip.Addr) (mac net.HardwareAddr) {
if ip.Is4() {
return s.srv4.FindMACbyIP(ip)
}
return s.srv6.FindMACbyIP(ip)
}

13
internal/dhcpd/options_unix.go
View File

@ -263,15 +263,12 @@ func (s *v4Server) prepareOptions() {
// IP-Layer Per Interface
// Since nearly all networks in the Internet currently support an MTU of
// 576 or greater, we strongly recommend the use of 576 for datagrams
// sent to non-local networks.
// Don't set the Interface MTU because client may choose the value on
// their own since it's listed in the [Host Requirements RFC]. It also
// seems the values listed there sometimes appear obsolete, see
// https://github.com/AdguardTeam/AdGuardHome/issues/5281.
//
// See https://datatracker.ietf.org/doc/html/rfc1122#section-3.3.3.
dhcpv4.Option{
Code: dhcpv4.OptionInterfaceMTU,
Value: dhcpv4.Uint16(576),
},
// [Host Requirements RFC]: https://datatracker.ietf.org/doc/html/rfc1122#section-3.3.3.
// Set the All Subnets Are Local Option to false since commonly the
// connected hosts aren't expected to be multihomed.

7
internal/dhcpd/v46_windows.go
View File

@ -4,7 +4,10 @@ package dhcpd
// 'u-root/u-root' package, a dependency of 'insomniacslk/dhcp' package, doesn't build on Windows
import "net"
import (
"net"
"net/netip"
)
type winServer struct{}
@ -16,7 +19,7 @@ func (winServer) GetLeases(_ GetLeasesFlags) (leases []*Lease) { return nil }
func (winServer) getLeasesRef() []*Lease { return nil }
func (winServer) AddStaticLease(_ *Lease) (err error) { return nil }
func (winServer) RemoveStaticLease(_ *Lease) (err error) { return nil }
func (winServer) FindMACbyIP(_ net.IP) (mac net.HardwareAddr) { return nil }
func (winServer) FindMACbyIP(_ netip.Addr) (mac net.HardwareAddr) { return nil }
func (winServer) WriteDiskConfig4(_ *V4ServerConf) {}
func (winServer) WriteDiskConfig6(_ *V6ServerConf) {}
func (winServer) Start() (err error) { return nil }

14
internal/dhcpd/v4_unix.go
View File

@ -108,7 +108,7 @@ func (s *v4Server) validHostnameForClient(cliHostname string, ip net.IP) (hostna
hostname = aghnet.GenerateHostname(ip)
}
err = netutil.ValidateDomainName(hostname)
err = netutil.ValidateHostname(hostname)
if err != nil {
log.Info("dhcpv4: %s", err)
hostname = ""
@ -200,20 +200,20 @@ func (s *v4Server) GetLeases(flags GetLeasesFlags) (leases []*Lease) {
return leases
}
// FindMACbyIP - find a MAC address by IP address in the currently active DHCP leases
func (s *v4Server) FindMACbyIP(ip net.IP) net.HardwareAddr {
// FindMACbyIP implements the [Interface] for *v4Server.
func (s *v4Server) FindMACbyIP(ip netip.Addr) (mac net.HardwareAddr) {
now := time.Now()
s.leasesLock.Lock()
defer s.leasesLock.Unlock()
ip4 := ip.To4()
if ip4 == nil {
if !ip.Is4() {
return nil
}
netIP := ip.AsSlice()
for _, l := range s.leases {
if l.IP.Equal(ip4) {
if l.IP.Equal(netIP) {
if l.Expiry.After(now) || l.IsStatic() {
return l.HWAddr
}
@ -372,7 +372,7 @@ func (s *v4Server) AddStaticLease(l *Lease) (err error) {
return err
}
err = netutil.ValidateDomainName(hostname)
err = netutil.ValidateHostname(hostname)
if err != nil {
return fmt.Errorf("validating hostname: %w", err)
}

4
internal/dhcpd/v4_unix_test.go
View File

@ -251,8 +251,8 @@ func TestV4Server_AddRemove_static(t *testing.T) {
},
name: "bad_hostname",
wantErrMsg: `dhcpv4: adding static lease: validating hostname: ` +
`bad domain name "bad-lbl-.local": ` +
`bad domain name label "bad-lbl-": bad domain name label rune '-'`,
`bad hostname "bad-lbl-.local": ` +
`bad hostname label "bad-lbl-": bad hostname label rune '-'`,
}}
for _, tc := range testCases {

18
internal/dhcpd/v6_unix.go
View File

@ -6,6 +6,7 @@ import (
"bytes"
"fmt"
"net"
"net/netip"
"sync"
"time"
@ -107,21 +108,26 @@ func (s *v6Server) getLeasesRef() []*Lease {
return s.leases
}
// FindMACbyIP - find a MAC address by IP address in the currently active DHCP leases
func (s *v6Server) FindMACbyIP(ip net.IP) net.HardwareAddr {
now := time.Now().Unix()
// FindMACbyIP implements the [Interface] for *v6Server.
func (s *v6Server) FindMACbyIP(ip netip.Addr) (mac net.HardwareAddr) {
now := time.Now()
s.leasesLock.Lock()
defer s.leasesLock.Unlock()
if !ip.Is6() {
return nil
}
netIP := ip.AsSlice()
for _, l := range s.leases {
if l.IP.Equal(ip) {
unix := l.Expiry.Unix()
if unix > now || unix == leaseExpireStatic {
if l.IP.Equal(netIP) {
if l.Expiry.After(now) || l.IsStatic() {
return l.HWAddr
}
}
}
return nil
}

9
internal/dnsforward/access.go
View File

@ -13,6 +13,7 @@ import (
"github.com/AdguardTeam/golibs/stringutil"
"github.com/AdguardTeam/urlfilter"
"github.com/AdguardTeam/urlfilter/filterlist"
"github.com/AdguardTeam/urlfilter/rules"
)
// unit is a convenient alias for struct{}
@ -127,8 +128,12 @@ func (a *accessManager) isBlockedClientID(id string) (ok bool) {
}
// isBlockedHost returns true if host should be blocked.
func (a *accessManager) isBlockedHost(host string) (ok bool) {
_, ok = a.blockedHostsEng.Match(strings.ToLower(host))
func (a *accessManager) isBlockedHost(host string, qt rules.RRType) (ok bool) {
_, ok = a.blockedHostsEng.MatchRequest(&urlfilter.DNSRequest{
Hostname: host,
ClientIP: "0.0.0.0",
DNSType: qt,
})
return ok
}

55
internal/dnsforward/access_test.go
View File

@ -4,6 +4,8 @@ import (
"net/netip"
"testing"
"github.com/AdguardTeam/urlfilter/rules"
"github.com/miekg/dns"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
@ -28,54 +30,75 @@ func TestIsBlockedHost(t *testing.T) {
"host1",
"*.host.com",
"||host3.com^",
"||*^$dnstype=HTTPS",
})
require.NoError(t, err)
testCases := []struct {
want assert.BoolAssertionFunc
name string
host string
want bool
qt rules.RRType
}{{
want: assert.True,
name: "plain_match",
host: "host1",
want: true,
qt: dns.TypeA,
}, {
want: assert.False,
name: "plain_mismatch",
host: "host2",
want: false,
qt: dns.TypeA,
}, {
want: assert.True,
name: "subdomain_match_short",
host: "asdf.host.com",
want: true,
qt: dns.TypeA,
}, {
want: assert.True,
name: "subdomain_match_long",
host: "qwer.asdf.host.com",
want: true,
qt: dns.TypeA,
}, {
want: assert.False,
name: "subdomain_mismatch_no_lead",
host: "host.com",
want: false,
qt: dns.TypeA,
}, {
want: assert.False,
name: "subdomain_mismatch_bad_asterisk",
host: "asdf.zhost.com",
want: false,
qt: dns.TypeA,
}, {
want: assert.True,
name: "rule_match_simple",
host: "host3.com",
want: true,
qt: dns.TypeA,
}, {
want: assert.True,
name: "rule_match_complex",
host: "asdf.host3.com",
want: true,
qt: dns.TypeA,
}, {
want: assert.False,
name: "rule_mismatch",
host: ".host3.com",
want: false,
qt: dns.TypeA,
}, {
want: assert.True,
name: "by_qtype",
host: "site-with-https-record.example",
qt: dns.TypeHTTPS,
}, {
want: assert.False,
name: "by_qtype_other",
host: "site-with-https-record.example",
qt: dns.TypeA,
}}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
assert.Equal(t, tc.want, a.isBlockedHost(tc.host))
tc.want(t, a.isBlockedHost(tc.host, tc.qt))
})
}
}
@ -93,29 +116,29 @@ func TestIsBlockedIP(t *testing.T) {
require.NoError(t, err)
testCases := []struct {
ip netip.Addr
name string
wantRule string
ip netip.Addr
wantBlocked bool
}{{
ip: netip.MustParseAddr("1.2.3.4"),
name: "match_ip",
wantRule: "1.2.3.4",
ip: netip.MustParseAddr("1.2.3.4"),
wantBlocked: true,
}, {
ip: netip.MustParseAddr("5.6.7.100"),
name: "match_cidr",
wantRule: "5.6.7.8/24",
ip: netip.MustParseAddr("5.6.7.100"),
wantBlocked: true,
}, {
ip: netip.MustParseAddr("9.2.3.4"),
name: "no_match_ip",
wantRule: "",
ip: netip.MustParseAddr("9.2.3.4"),
wantBlocked: false,
}, {
ip: netip.MustParseAddr("9.6.7.100"),
name: "no_match_cidr",
wantRule: "",
ip: netip.MustParseAddr("9.6.7.100"),
wantBlocked: false,
}}

2
internal/dnsforward/clientid.go
View File

@ -14,7 +14,7 @@ import (
// ValidateClientID returns an error if id is not a valid ClientID.
func ValidateClientID(id string) (err error) {
err = netutil.ValidateDomainNameLabel(id)
err = netutil.ValidateHostnameLabel(id)
if err != nil {
// Replace the domain name label wrapper with our own.
return fmt.Errorf("invalid clientid %q: %w", id, errors.Unwrap(err))

6
internal/dnsforward/clientid_test.go
View File

@ -119,7 +119,7 @@ func TestServer_clientIDFromDNSContext(t *testing.T) {
cliSrvName: "!!!.example.com",
wantClientID: "",
wantErrMsg: `clientid check: invalid clientid "!!!": ` +
`bad domain name label rune '!'`,
`bad hostname label rune '!'`,
inclHTTPTLS: false,
strictSNI: true,
}, {
@ -131,7 +131,7 @@ func TestServer_clientIDFromDNSContext(t *testing.T) {
wantClientID: "",
wantErrMsg: `clientid check: invalid clientid "abcdefghijklmno` +
`pqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz0123456789": ` +
`domain name label is too long: got 72, max 63`,
`hostname label is too long: got 72, max 63`,
inclHTTPTLS: false,
strictSNI: true,
}, {
@ -330,7 +330,7 @@ func TestClientIDFromDNSContextHTTPS(t *testing.T) {
path: "/dns-query/!!!",
cliSrvName: "example.com",
wantClientID: "",
wantErrMsg: `clientid check: invalid clientid "!!!": bad domain name label rune '!'`,
wantErrMsg: `clientid check: invalid clientid "!!!": bad hostname label rune '!'`,
}, {
name: "both_ids",
path: "/dns-query/right",

164
internal/dnsforward/config.go
View File

@ -7,7 +7,6 @@ import (
"net"
"net/netip"
"os"
"sort"
"strings"
"time"
@ -23,6 +22,7 @@ import (
"github.com/AdguardTeam/golibs/stringutil"
"github.com/AdguardTeam/golibs/timeutil"
"github.com/ameshkov/dnscrypt/v2"
"golang.org/x/exp/slices"
)
// BlockingMode is an enum of all allowed blocking modes.
@ -53,7 +53,6 @@ const (
// The zero FilteringConfig is empty and ready for use.
type FilteringConfig struct {
// Callbacks for other modules
// --
// FilterHandler is an optional additional filtering callback.
FilterHandler func(clientAddr net.IP, clientID string, settings *filtering.Settings) `yaml:"-"`
@ -64,50 +63,82 @@ type FilteringConfig struct {
GetCustomUpstreamByClient func(id string) (conf *proxy.UpstreamConfig, err error) `yaml:"-"`
// Protection configuration
// --
ProtectionEnabled bool `yaml:"protection_enabled"` // whether or not use any of filtering features
BlockingMode BlockingMode `yaml:"blocking_mode"` // mode how to answer filtered requests
BlockingIPv4 net.IP `yaml:"blocking_ipv4"` // IP address to be returned for a blocked A request
BlockingIPv6 net.IP `yaml:"blocking_ipv6"` // IP address to be returned for a blocked AAAA request
BlockedResponseTTL uint32 `yaml:"blocked_response_ttl"` // if 0, then default is used (3600)
// ProtectionEnabled defines whether or not use any of filtering features.
ProtectionEnabled bool `yaml:"protection_enabled"`
// IP (or domain name) which is used to respond to DNS requests blocked by parental control or safe-browsing
// BlockingMode defines the way how blocked responses are constructed.
BlockingMode BlockingMode `yaml:"blocking_mode"`
// BlockingIPv4 is the IP address to be returned for a blocked A request.
BlockingIPv4 net.IP `yaml:"blocking_ipv4"`
// BlockingIPv6 is the IP address to be returned for a blocked AAAA
// request.
BlockingIPv6 net.IP `yaml:"blocking_ipv6"`
// BlockedResponseTTL is the time-to-live value for blocked responses. If
// 0, then default value is used (3600).
BlockedResponseTTL uint32 `yaml:"blocked_response_ttl"`
// ParentalBlockHost is the IP (or domain name) which is used to respond to
// DNS requests blocked by parental control.
ParentalBlockHost string `yaml:"parental_block_host"`
// SafeBrowsingBlockHost is the IP (or domain name) which is used to
// respond to DNS requests blocked by safe-browsing.
SafeBrowsingBlockHost string `yaml:"safebrowsing_block_host"`
// Anti-DNS amplification
// --
Ratelimit uint32 `yaml:"ratelimit"` // max number of requests per second from a given IP (0 to disable)
RatelimitWhitelist []string `yaml:"ratelimit_whitelist"` // a list of whitelisted client IP addresses
RefuseAny bool `yaml:"refuse_any"` // if true, refuse ANY requests
// Ratelimit is the maximum number of requests per second from a given IP
// (0 to disable).
Ratelimit uint32 `yaml:"ratelimit"`
// RatelimitWhitelist is the list of whitelisted client IP addresses.
RatelimitWhitelist []string `yaml:"ratelimit_whitelist"`
// RefuseAny, if true, refuse ANY requests.
RefuseAny bool `yaml:"refuse_any"`
// Upstream DNS servers configuration
// --
// UpstreamDNS is the list of upstream DNS servers.
UpstreamDNS []string `yaml:"upstream_dns"`
// UpstreamDNSFileName, if set, points to the file which contains upstream
// DNS servers.
UpstreamDNSFileName string `yaml:"upstream_dns_file"`
BootstrapDNS []string `yaml:"bootstrap_dns"` // a list of bootstrap DNS for DoH and DoT (plain DNS only)
AllServers bool `yaml:"all_servers"` // if true, parallel queries to all configured upstream servers are enabled
FastestAddr bool `yaml:"fastest_addr"` // use Fastest Address algorithm
// BootstrapDNS is the list of bootstrap DNS servers for DoH and DoT
// resolvers (plain DNS only).
BootstrapDNS []string `yaml:"bootstrap_dns"`
// AllServers, if true, parallel queries to all configured upstream servers
// are enabled.
AllServers bool `yaml:"all_servers"`
// FastestAddr, if true, use Fastest Address algorithm.
FastestAddr bool `yaml:"fastest_addr"`
// FastestTimeout replaces the default timeout for dialing IP addresses
// when FastestAddr is true.
FastestTimeout timeutil.Duration `yaml:"fastest_timeout"`
// Access settings
// --
// AllowedClients is the slice of IP addresses, CIDR networks, and ClientIDs
// of allowed clients. If not empty, only these clients are allowed, and
// [FilteringConfig.DisallowedClients] are ignored.
// AllowedClients is the slice of IP addresses, CIDR networks, and
// ClientIDs of allowed clients. If not empty, only these clients are
// allowed, and [FilteringConfig.DisallowedClients] are ignored.
AllowedClients []string `yaml:"allowed_clients"`
// DisallowedClients is the slice of IP addresses, CIDR networks, and
// ClientIDs of disallowed clients.
DisallowedClients []string `yaml:"disallowed_clients"`
BlockedHosts []string `yaml:"blocked_hosts"` // hosts that should be blocked
// BlockedHosts is the list of hosts that should be blocked.
BlockedHosts []string `yaml:"blocked_hosts"`
// TrustedProxies is the list of IP addresses and CIDR networks to detect
// proxy servers addresses the DoH requests from which should be handled.
// The value of nil or an empty slice for this field makes Proxy not trust
@ -115,26 +146,46 @@ type FilteringConfig struct {
TrustedProxies []string `yaml:"trusted_proxies"`
// DNS cache settings
// --
CacheSize uint32 `yaml:"cache_size"` // DNS cache size (in bytes)
CacheMinTTL uint32 `yaml:"cache_ttl_min"` // override TTL value (minimum) received from upstream server
CacheMaxTTL uint32 `yaml:"cache_ttl_max"` // override TTL value (maximum) received from upstream server
// CacheSize is the DNS cache size (in bytes).
CacheSize uint32 `yaml:"cache_size"`
// CacheMinTTL is the override TTL value (minimum) received from upstream
// server.
CacheMinTTL uint32 `yaml:"cache_ttl_min"`
// CacheMaxTTL is the override TTL value (maximum) received from upstream
// server.
CacheMaxTTL uint32 `yaml:"cache_ttl_max"`
// CacheOptimistic defines if optimistic cache mechanism should be used.
CacheOptimistic bool `yaml:"cache_optimistic"`
// Other settings
// --
BogusNXDomain []string `yaml:"bogus_nxdomain"` // transform responses with these IP addresses to NXDOMAIN
AAAADisabled bool `yaml:"aaaa_disabled"` // Respond with an empty answer to all AAAA requests
EnableDNSSEC bool `yaml:"enable_dnssec"` // Set AD flag in outcoming DNS request
EnableEDNSClientSubnet bool `yaml:"edns_client_subnet"` // Enable EDNS Client Subnet option
MaxGoroutines uint32 `yaml:"max_goroutines"` // Max. number of parallel goroutines for processing incoming requests
HandleDDR bool `yaml:"handle_ddr"` // Handle DDR requests
// BogusNXDomain is the list of IP addresses, responses with them will be
// transformed to NXDOMAIN.
BogusNXDomain []string `yaml:"bogus_nxdomain"`
// IpsetList is the ipset configuration that allows AdGuard Home to add
// IP addresses of the specified domain names to an ipset list. Syntax:
// AAAADisabled, if true, respond with an empty answer to all AAAA
// requests.
AAAADisabled bool `yaml:"aaaa_disabled"`
// EnableDNSSEC, if true, set AD flag in outcoming DNS request.
EnableDNSSEC bool `yaml:"enable_dnssec"`
// EDNSClientSubnet is the settings list for EDNS Client Subnet.
EDNSClientSubnet *EDNSClientSubnet `yaml:"edns_client_subnet"`
// MaxGoroutines is the max number of parallel goroutines for processing
// incoming requests.
MaxGoroutines uint32 `yaml:"max_goroutines"`
// HandleDDR, if true, handle DDR requests
HandleDDR bool `yaml:"handle_ddr"`
// IpsetList is the ipset configuration that allows AdGuard Home to add IP
// addresses of the specified domain names to an ipset list. Syntax:
//
// DOMAIN[,DOMAIN].../IPSET_NAME
//
@ -146,6 +197,18 @@ type FilteringConfig struct {
IpsetListFileName string `yaml:"ipset_file"`
}
// EDNSClientSubnet is the settings list for EDNS Client Subnet.
type EDNSClientSubnet struct {
// CustomIP for EDNS Client Subnet.
CustomIP string `yaml:"custom_ip"`
// Enabled defines if EDNS Client Subnet is enabled.
Enabled bool `yaml:"enabled"`
// UseCustom defines if CustomIP should be used.
UseCustom bool `yaml:"use_custom"`
}
// TLSConfig is the TLS configuration for HTTPS, DNS-over-HTTPS, and DNS-over-TLS
type TLSConfig struct {
cert tls.Certificate
@ -270,12 +333,24 @@ func (s *Server) createProxyConfig() (conf proxy.Config, err error) {
UpstreamConfig: srvConf.UpstreamConfig,
BeforeRequestHandler: s.beforeRequestHandler,
RequestHandler: s.handleDNSRequest,
EnableEDNSClientSubnet: srvConf.EnableEDNSClientSubnet,
EnableEDNSClientSubnet: srvConf.EDNSClientSubnet.Enabled,
MaxGoroutines: int(srvConf.MaxGoroutines),
UseDNS64: srvConf.UseDNS64,
DNS64Prefs: srvConf.DNS64Prefixes,
}
if srvConf.EDNSClientSubnet.UseCustom {
// TODO(s.chzhen): Add wrapper around netip.Addr.
var ip net.IP
ip, err = netutil.ParseIP(srvConf.EDNSClientSubnet.CustomIP)
if err != nil {
return conf, fmt.Errorf("edns: %w", err)
}
// TODO(s.chzhen): Use netip.Addr instead of net.IP inside dnsproxy.
conf.EDNSAddr = ip
}
if srvConf.CacheSize != 0 {
conf.CacheEnabled = true
conf.CacheSizeBytes = int(srvConf.CacheSize)
@ -510,7 +585,7 @@ func (s *Server) prepareTLS(proxyConfig *proxy.Config) (err error) {
if len(cert.DNSNames) != 0 {
s.conf.dnsNames = cert.DNSNames
log.Debug("dnsforward: using certificate's SAN as DNS names: %v", cert.DNSNames)
sort.Strings(s.conf.dnsNames)
slices.Sort(s.conf.dnsNames)
} else {
s.conf.dnsNames = append(s.conf.dnsNames, cert.Subject.CommonName)
log.Debug("dnsforward: using certificate's CN as DNS name: %s", cert.Subject.CommonName)
@ -526,16 +601,6 @@ func (s *Server) prepareTLS(proxyConfig *proxy.Config) (err error) {
return nil
}
// isInSorted returns true if s is in the sorted slice strs.
func isInSorted(strs []string, s string) (ok bool) {
i := sort.SearchStrings(strs, s)
if i == len(strs) || strs[i] != s {
return false
}
return true
}
// isWildcard returns true if host is a wildcard hostname.
func isWildcard(host string) (ok bool) {
return len(host) >= 2 && host[0] == '*' && host[1] == '.'
@ -550,11 +615,12 @@ func matchesDomainWildcard(host, pat string) (ok bool) {
// anyNameMatches returns true if sni, the client's SNI value, matches any of
// the DNS names and patterns from certificate. dnsNames must be sorted.
func anyNameMatches(dnsNames []string, sni string) (ok bool) {
if netutil.ValidateDomainName(sni) != nil {
// Check sni is either a valid hostname or a valid IP address.
if netutil.ValidateHostname(sni) != nil && net.ParseIP(sni) == nil {
return false
}
if isInSorted(dnsNames, sni) {
if _, ok = slices.BinarySearch(dnsNames, sni); ok {
return true
}

8
internal/dnsforward/config_test.go
View File

@ -1,15 +1,15 @@
package dnsforward
import (
"sort"
"testing"
"github.com/stretchr/testify/assert"
"golang.org/x/exp/slices"
)
func TestAnyNameMatches(t *testing.T) {
dnsNames := []string{"host1", "*.host2", "1.2.3.4"}
sort.Strings(dnsNames)
slices.Sort(dnsNames)
testCases := []struct {
name string
@ -31,6 +31,10 @@ func TestAnyNameMatches(t *testing.T) {
name: "match",
dnsName: "1.2.3.4",
want: true,
}, {
name: "mismatch_bad_ip",
dnsName: "1.2.3.256",
want: false,
}, {
name: "mismatch",
dnsName: "host2",

4
internal/dnsforward/dns.go
View File

@ -230,7 +230,7 @@ func (s *Server) onDHCPLeaseChanged(flags int) {
for _, l := range ll {
// TODO(a.garipov): Remove this after we're finished with the client
// hostname validations in the DHCP server code.
err := netutil.ValidateDomainName(l.Hostname)
err := netutil.ValidateHostname(l.Hostname)
if err != nil {
log.Debug("dnsforward: skipping invalid hostname %q from dhcp: %s", l.Hostname, err)
@ -468,7 +468,7 @@ func (s *Server) processRestrictLocal(dctx *dnsContext) (rc resultCode) {
return resultCodeError
}
log.Debug("dnsforward: request is for a service domain")
log.Debug("dnsforward: request is not for arpa domain")
return resultCodeSuccess
}

19
internal/dnsforward/dns64_test.go
View File

@ -273,18 +273,25 @@ func TestServer_HandleDNSRequest_dns64(t *testing.T) {
return resp, nil
})
s := createTestServer(t, &filtering.Config{}, ServerConfig{
UDPListenAddrs: []*net.UDPAddr{{}},
TCPListenAddrs: []*net.TCPAddr{{}},
UseDNS64: true,
}, localUps)
client := &dns.Client{
Net: "tcp",
Timeout: 1 * time.Second,
}
for _, tc := range testCases {
// TODO(e.burkov): It seems [proxy.Proxy] isn't intended to be reused
// right after stop, due to a data race in [proxy.Proxy.Init] method
// when setting an OOB size. As a temporary workaround, recreate the
// whole server for each test case.
s := createTestServer(t, &filtering.Config{}, ServerConfig{
UDPListenAddrs: []*net.UDPAddr{{}},
TCPListenAddrs: []*net.TCPAddr{{}},
UseDNS64: true,
FilteringConfig: FilteringConfig{
EDNSClientSubnet: &EDNSClientSubnet{Enabled: false},
},
}, localUps)
t.Run(tc.name, func(t *testing.T) {
s.conf.UpstreamConfig.Upstreams = []upstream.Upstream{newUps(tc.upsAns)}
startDeferStop(t, s)

8
internal/dnsforward/dns_test.go
View File

@ -467,6 +467,11 @@ func TestServer_ProcessRestrictLocal(t *testing.T) {
s := createTestServer(t, &filtering.Config{}, ServerConfig{
UDPListenAddrs: []*net.UDPAddr{{}},
TCPListenAddrs: []*net.TCPAddr{{}},
// TODO(s.chzhen): Add tests where EDNSClientSubnet.Enabled is true.
// Improve FilteringConfig declaration for tests.
FilteringConfig: FilteringConfig{
EDNSClientSubnet: &EDNSClientSubnet{Enabled: false},
},
}, ups)
s.conf.UpstreamConfig.Upstreams = []upstream.Upstream{ups}
startDeferStop(t, s)
@ -539,6 +544,9 @@ func TestServer_ProcessLocalPTR_usingResolvers(t *testing.T) {
ServerConfig{
UDPListenAddrs: []*net.UDPAddr{{}},
TCPListenAddrs: []*net.TCPAddr{{}},
FilteringConfig: FilteringConfig{
EDNSClientSubnet: &EDNSClientSubnet{Enabled: false},
},
},
aghtest.NewUpstreamMock(func(req *dns.Msg) (resp *dns.Msg, err error) {
return aghalg.Coalesce(

61
internal/dnsforward/dnsforward_test.go
View File

@ -11,6 +11,7 @@ import (
"fmt"
"math/big"
"net"
"net/netip"
"sync"
"sync/atomic"
"testing"
@ -155,6 +156,9 @@ func createTestTLS(t *testing.T, tlsConf TLSConfig) (s *Server, certPem []byte)
s = createTestServer(t, &filtering.Config{}, ServerConfig{
UDPListenAddrs: []*net.UDPAddr{{}},
TCPListenAddrs: []*net.TCPAddr{{}},
FilteringConfig: FilteringConfig{
EDNSClientSubnet: &EDNSClientSubnet{Enabled: false},
},
}, nil)
tlsConf.CertificateChainData, tlsConf.PrivateKeyData = certPem, keyPem
@ -266,6 +270,9 @@ func TestServer(t *testing.T) {
s := createTestServer(t, &filtering.Config{}, ServerConfig{
UDPListenAddrs: []*net.UDPAddr{{}},
TCPListenAddrs: []*net.TCPAddr{{}},
FilteringConfig: FilteringConfig{
EDNSClientSubnet: &EDNSClientSubnet{Enabled: false},
},
}, nil)
s.conf.UpstreamConfig.Upstreams = []upstream.Upstream{newGoogleUpstream()}
startDeferStop(t, s)
@ -305,6 +312,7 @@ func TestServer_timeout(t *testing.T) {
UpstreamTimeout: timeout,
FilteringConfig: FilteringConfig{
BlockingMode: BlockingModeDefault,
EDNSClientSubnet: &EDNSClientSubnet{Enabled: false},
},
}
@ -322,6 +330,9 @@ func TestServer_timeout(t *testing.T) {
require.NoError(t, err)
s.conf.FilteringConfig.BlockingMode = BlockingModeDefault
s.conf.FilteringConfig.EDNSClientSubnet = &EDNSClientSubnet{
Enabled: false,
}
err = s.Prepare(&s.conf)
require.NoError(t, err)
@ -333,6 +344,9 @@ func TestServerWithProtectionDisabled(t *testing.T) {
s := createTestServer(t, &filtering.Config{}, ServerConfig{
UDPListenAddrs: []*net.UDPAddr{{}},
TCPListenAddrs: []*net.TCPAddr{{}},
FilteringConfig: FilteringConfig{
EDNSClientSubnet: &EDNSClientSubnet{Enabled: false},
},
}, nil)
s.conf.UpstreamConfig.Upstreams = []upstream.Upstream{newGoogleUpstream()}
startDeferStop(t, s)
@ -437,6 +451,9 @@ func TestSafeSearch(t *testing.T) {
TCPListenAddrs: []*net.TCPAddr{{}},
FilteringConfig: FilteringConfig{
ProtectionEnabled: true,
EDNSClientSubnet: &EDNSClientSubnet{
Enabled: false,
},
},
}
s := createTestServer(t, filterConf, forwardConf, nil)
@ -492,6 +509,11 @@ func TestInvalidRequest(t *testing.T) {
s := createTestServer(t, &filtering.Config{}, ServerConfig{
UDPListenAddrs: []*net.UDPAddr{{}},
TCPListenAddrs: []*net.TCPAddr{{}},
FilteringConfig: FilteringConfig{
EDNSClientSubnet: &EDNSClientSubnet{
Enabled: false,
},
},
}, nil)
startDeferStop(t, s)
@ -518,6 +540,9 @@ func TestBlockedRequest(t *testing.T) {
FilteringConfig: FilteringConfig{
ProtectionEnabled: true,
BlockingMode: BlockingModeDefault,
EDNSClientSubnet: &EDNSClientSubnet{
Enabled: false,
},
},
}
s := createTestServer(t, &filtering.Config{}, forwardConf, nil)
@ -543,6 +568,9 @@ func TestServerCustomClientUpstream(t *testing.T) {
TCPListenAddrs: []*net.TCPAddr{{}},
FilteringConfig: FilteringConfig{
ProtectionEnabled: true,
EDNSClientSubnet: &EDNSClientSubnet{
Enabled: false,
},
},
}
s := createTestServer(t, &filtering.Config{}, forwardConf, nil)
@ -591,6 +619,11 @@ func TestBlockCNAMEProtectionEnabled(t *testing.T) {
s := createTestServer(t, &filtering.Config{}, ServerConfig{
UDPListenAddrs: []*net.UDPAddr{{}},
TCPListenAddrs: []*net.TCPAddr{{}},
FilteringConfig: FilteringConfig{
EDNSClientSubnet: &EDNSClientSubnet{
Enabled: false,
},
},
}, nil)
testUpstm := &aghtest.Upstream{
CName: testCNAMEs,
@ -621,6 +654,9 @@ func TestBlockCNAME(t *testing.T) {
FilteringConfig: FilteringConfig{
ProtectionEnabled: true,
BlockingMode: BlockingModeDefault,
EDNSClientSubnet: &EDNSClientSubnet{
Enabled: false,
},
},
}
s := createTestServer(t, &filtering.Config{}, forwardConf, nil)
@ -690,6 +726,9 @@ func TestClientRulesForCNAMEMatching(t *testing.T) {
FilterHandler: func(_ net.IP, _ string, settings *filtering.Settings) {
settings.FilteringEnabled = false
},
EDNSClientSubnet: &EDNSClientSubnet{
Enabled: false,
},
},
}
s := createTestServer(t, &filtering.Config{}, forwardConf, nil)
@ -731,6 +770,9 @@ func TestNullBlockedRequest(t *testing.T) {
FilteringConfig: FilteringConfig{
ProtectionEnabled: true,
BlockingMode: BlockingModeNullIP,
EDNSClientSubnet: &EDNSClientSubnet{
Enabled: false,
},
},
}
s := createTestServer(t, &filtering.Config{}, forwardConf, nil)
@ -783,6 +825,9 @@ func TestBlockedCustomIP(t *testing.T) {
BlockingMode: BlockingModeCustomIP,
BlockingIPv4: nil,
UpstreamDNS: []string{"8.8.8.8:53", "8.8.4.4:53"},
EDNSClientSubnet: &EDNSClientSubnet{
Enabled: false,
},
},
}
@ -831,6 +876,9 @@ func TestBlockedByHosts(t *testing.T) {
FilteringConfig: FilteringConfig{
ProtectionEnabled: true,
BlockingMode: BlockingModeDefault,
EDNSClientSubnet: &EDNSClientSubnet{
Enabled: false,
},
},
}
@ -864,6 +912,9 @@ func TestBlockedBySafeBrowsing(t *testing.T) {
FilteringConfig: FilteringConfig{
SafeBrowsingBlockHost: ans4.String(),
ProtectionEnabled: true,
EDNSClientSubnet: &EDNSClientSubnet{
Enabled: false,
},
},
}
s := createTestServer(t, filterConf, forwardConf, nil)
@ -918,6 +969,9 @@ func TestRewrite(t *testing.T) {
ProtectionEnabled: true,
BlockingMode: BlockingModeDefault,
UpstreamDNS: []string{"8.8.8.8:53"},
EDNSClientSubnet: &EDNSClientSubnet{
Enabled: false,
},
},
}))
@ -1009,7 +1063,7 @@ var testDHCP = &dhcpd.MockInterface{
}}
},
OnSetOnLeaseChanged: func(olct dhcpd.OnLeaseChangedT) {},
OnFindMACbyIP: func(ip net.IP) (mac net.HardwareAddr) { panic("not implemented") },
OnFindMACbyIP: func(ip netip.Addr) (mac net.HardwareAddr) { panic("not implemented") },
OnWriteDiskConfig: func(c *dhcpd.ServerConfig) { panic("not implemented") },
}
@ -1032,6 +1086,7 @@ func TestPTRResponseFromDHCPLeases(t *testing.T) {
s.conf.UpstreamDNS = []string{"127.0.0.1:53"}
s.conf.FilteringConfig.ProtectionEnabled = true
s.conf.FilteringConfig.BlockingMode = BlockingModeDefault
s.conf.FilteringConfig.EDNSClientSubnet = &EDNSClientSubnet{Enabled: false}
err = s.Prepare(&s.conf)
require.NoError(t, err)
@ -1107,6 +1162,7 @@ func TestPTRResponseFromHosts(t *testing.T) {
s.conf.TCPListenAddrs = []*net.TCPAddr{{}}
s.conf.UpstreamDNS = []string{"127.0.0.1:53"}
s.conf.FilteringConfig.BlockingMode = BlockingModeDefault
s.conf.FilteringConfig.EDNSClientSubnet = &EDNSClientSubnet{Enabled: false}
err = s.Prepare(&s.conf)
require.NoError(t, err)
@ -1171,7 +1227,8 @@ func TestNewServer(t *testing.T) {
LocalDomain: "!!!",
},
wantErrMsg: `local domain: bad domain name "!!!": ` +
`bad domain name label "!!!": bad domain name label rune '!'`,
`bad top-level domain name label "!!!": ` +
`bad top-level domain name label rune '!'`,
}}
for _, tc := range testCases {

8
internal/dnsforward/filter.go
View File

@ -31,9 +31,11 @@ func (s *Server) beforeRequestHandler(
}
if len(pctx.Req.Question) == 1 {
host := strings.TrimSuffix(pctx.Req.Question[0].Name, ".")
if s.access.isBlockedHost(host) {
log.Debug("host %s is in access blocklist", host)
q := pctx.Req.Question[0]
qt := q.Qtype
host := strings.TrimSuffix(q.Name, ".")
if s.access.isBlockedHost(host, qt) {
log.Debug("request %s %s is in access blocklist", dns.Type(qt), host)
return s.preBlockedResponse(pctx)
}

3
internal/dnsforward/filter_test.go
View File

@ -29,6 +29,9 @@ func TestHandleDNSRequest_filterDNSResponse(t *testing.T) {
FilteringConfig: FilteringConfig{
ProtectionEnabled: true,
BlockingMode: BlockingModeDefault,
EDNSClientSubnet: &EDNSClientSubnet{
Enabled: false,
},
},
}
filters := []filtering.Filter{{

4
internal/dnsforward/http.go
View File

@ -57,7 +57,7 @@ func (s *Server) getDNSConfig() (c *jsonDNSConfig) {
blockingIPv4 := s.conf.BlockingIPv4
blockingIPv6 := s.conf.BlockingIPv6
ratelimit := s.conf.Ratelimit
enableEDNSClientSubnet := s.conf.EnableEDNSClientSubnet
enableEDNSClientSubnet := s.conf.EDNSClientSubnet.Enabled
enableDNSSEC := s.conf.EnableDNSSEC
aaaaDisabled := s.conf.AAAADisabled
cacheSize := s.conf.CacheSize
@ -280,7 +280,7 @@ func (s *Server) setConfigRestartable(dc *jsonDNSConfig) (shouldRestart bool) {
setIfNotNil(&s.conf.LocalPTRResolvers, dc.LocalPTRUpstreams),
setIfNotNil(&s.conf.UpstreamDNSFileName, dc.UpstreamsFile),
setIfNotNil(&s.conf.BootstrapDNS, dc.Bootstraps),
setIfNotNil(&s.conf.EnableEDNSClientSubnet, dc.EDNSCSEnabled),
setIfNotNil(&s.conf.EDNSClientSubnet.Enabled, dc.EDNSCSEnabled),
setIfNotNil(&s.conf.CacheSize, dc.CacheSize),
setIfNotNil(&s.conf.CacheMinTTL, dc.CacheMinTTL),
setIfNotNil(&s.conf.CacheMaxTTL, dc.CacheMaxTTL),

13
internal/dnsforward/http_test.go
View File

@ -69,6 +69,7 @@ func TestDNSForwardHTTP_handleGetConfig(t *testing.T) {
ProtectionEnabled: true,
BlockingMode: BlockingModeDefault,
UpstreamDNS: []string{"8.8.8.8:53", "8.8.4.4:53"},
EDNSClientSubnet: &EDNSClientSubnet{Enabled: false},
},
ConfigModified: func() {},
}
@ -144,6 +145,7 @@ func TestDNSForwardHTTP_handleSetConfig(t *testing.T) {
ProtectionEnabled: true,
BlockingMode: BlockingModeDefault,
UpstreamDNS: []string{"8.8.8.8:53", "8.8.4.4:53"},
EDNSClientSubnet: &EDNSClientSubnet{Enabled: false},
},
ConfigModified: func() {},
}
@ -227,7 +229,10 @@ func TestDNSForwardHTTP_handleSetConfig(t *testing.T) {
require.True(t, ok)
t.Run(tc.name, func(t *testing.T) {
t.Cleanup(func() { s.conf = defaultConf })
t.Cleanup(func() {
s.conf = defaultConf
s.conf.FilteringConfig.EDNSClientSubnet.Enabled = false
})
rBody := io.NopCloser(bytes.NewReader(caseData.Req))
var r *http.Request
@ -337,7 +342,8 @@ func TestValidateUpstreams(t *testing.T) {
}, {
name: "bad_domain",
wantErr: `bad upstream for domain "[/!/]8.8.8.8": domain at index 0: ` +
`bad domain name "!": bad domain name label "!": bad domain name label rune '!'`,
`bad domain name "!": bad top-level domain name label "!": ` +
`bad top-level domain name label rune '!'`,
set: []string{"[/!/]8.8.8.8"},
}}
@ -442,6 +448,9 @@ func TestServer_handleTestUpstreaDNS(t *testing.T) {
UDPListenAddrs: []*net.UDPAddr{{}},
TCPListenAddrs: []*net.TCPAddr{{}},
UpstreamTimeout: upsTimeout,
FilteringConfig: FilteringConfig{
EDNSClientSubnet: &EDNSClientSubnet{Enabled: false},
},
}, nil)
startDeferStop(t, srv)

38
internal/filtering/blocked_test.go
View File

@ -1,38 +0,0 @@
//go:build ignore
// +build ignore
package filtering
import (
"fmt"
"sort"
"testing"
)
// This is a simple tool that takes a list of services and prints them to the output.
// It is supposed to be used to update:
// client/src/helpers/constants.js
// client/src/components/ui/Icons.js
//
// Usage:
// 1. go run ./internal/filtering/blocked_test.go
// 2. Use the output to replace `SERVICES` array in "client/src/helpers/constants.js".
// 3. You'll need to enter services names manually.
// 4. Don't forget to add missing icons to "client/src/components/ui/Icons.js".
//
// TODO(ameshkov): Rework generator: have a JSON file with all the metadata we need
// then use this JSON file to generate JS and Go code
func TestGenServicesArray(t *testing.T) {
services := make([]svc, len(serviceRulesArray))
copy(services, serviceRulesArray)
sort.Slice(services, func(i, j int) bool {
return services[i].name < services[j].name
})
fmt.Println("export const SERVICES = [")
for _, s := range services {
fmt.Printf(" {\n id: '%s',\n name: '%s',\n },\n", s.name, s.name)
}
fmt.Println("];")
}

67
internal/filtering/filtering.go
View File

@ -420,11 +420,11 @@ type ResultRule struct {
// Result contains the result of a request check.
//
// All fields transitively have omitempty tags so that the query log
// doesn't become too large.
// All fields transitively have omitempty tags so that the query log doesn't
// become too large.
//
// TODO(a.garipov): Clarify relationships between fields. Perhaps
// replace with a sum type or an interface?
// TODO(a.garipov): Clarify relationships between fields. Perhaps replace with
// a sum type or an interface?
type Result struct {
// DNSRewriteResult is the $dnsrewrite filter rule result.
DNSRewriteResult *DNSRewriteResult `json:",omitempty"`
@ -813,17 +813,18 @@ func (d *DNSFilter) matchHostProcessDNSResult(
return res
}
if dnsres.HostRulesV4 != nil || dnsres.HostRulesV6 != nil {
// Question type doesn't match the host rules. Return the first matched
// host rule, but without an IP address.
var matchedRules []rules.Rule
if dnsres.HostRulesV4 != nil {
matchedRules = []rules.Rule{dnsres.HostRulesV4[0]}
} else if dnsres.HostRulesV6 != nil {
matchedRules = []rules.Rule{dnsres.HostRulesV6[0]}
return hostResultForOtherQType(dnsres)
}
return makeResult(matchedRules, FilteredBlockList)
// hostResultForOtherQType returns a result based on the host rules in dnsres,
// if any. dnsres.HostRulesV4 take precedence over dnsres.HostRulesV6.
func hostResultForOtherQType(dnsres *urlfilter.DNSResult) (res Result) {
if len(dnsres.HostRulesV4) != 0 {
return makeResult([]rules.Rule{dnsres.HostRulesV4[0]}, FilteredBlockList)
}
if len(dnsres.HostRulesV6) != 0 {
return makeResult([]rules.Rule{dnsres.HostRulesV6[0]}, FilteredBlockList)
}
return Result{}
@ -840,7 +841,7 @@ func (d *DNSFilter) matchHost(
return Result{}, nil
}
ureq := &urlfilter.DNSRequest{
ufReq := &urlfilter.DNSRequest{
Hostname: host,
SortedClientTags: setts.ClientTags,
// TODO(e.burkov): Wait for urlfilter update to pass net.IP.
@ -857,7 +858,7 @@ func (d *DNSFilter) matchHost(
defer d.engineLock.RUnlock()
if setts.ProtectionEnabled && d.filteringEngineAllow != nil {
dnsres, ok := d.filteringEngineAllow.MatchRequest(ureq)
dnsres, ok := d.filteringEngineAllow.MatchRequest(ufReq)
if ok {
return d.matchHostProcessAllowList(host, dnsres)
}
@ -867,17 +868,13 @@ func (d *DNSFilter) matchHost(
return Result{}, nil
}
dnsres, ok := d.filteringEngine.MatchRequest(ureq)
dnsres, matchedEngine := d.filteringEngine.MatchRequest(ufReq)
// Check DNS rewrites first, because the API there is a bit awkward.
if dnsr := dnsres.DNSRewrites(); len(dnsr) > 0 {
res = d.processDNSRewrites(dnsr)
if res.Reason == RewrittenRule && res.CanonName == host {
// A rewrite of a host to itself. Go on and try matching other
// things.
} else {
return res, nil
}
} else if !ok {
dnsRWRes := d.processDNSResultRewrites(dnsres, host)
if dnsRWRes.Reason != NotFilteredNotFound {
return dnsRWRes, nil
} else if !matchedEngine {
return Result{}, nil
}
@ -899,6 +896,26 @@ func (d *DNSFilter) matchHost(
return res, nil
}
// processDNSResultRewrites returns an empty Result if there are no dnsrewrite
// rules in dnsres. Otherwise, it returns the processed Result.
func (d *DNSFilter) processDNSResultRewrites(
dnsres *urlfilter.DNSResult,
host string,
) (dnsRWRes Result) {
dnsr := dnsres.DNSRewrites()
if len(dnsr) == 0 {
return Result{}
}
res := d.processDNSRewrites(dnsr)
if res.Reason == RewrittenRule && res.CanonName == host {
// A rewrite of a host to itself. Go on and try matching other things.
return Result{}
}
return res
}
// makeResult returns a properly constructed Result.
func makeResult(matchedRules []rules.Rule, reason Reason) (res Result) {
resRules := make([]*ResultRule, len(matchedRules))

43
internal/filtering/rewrites.go
View File

@ -1,11 +1,8 @@
// DNS Rewrites
package filtering
import (
"fmt"
"net"
"sort"
"strings"
"github.com/AdguardTeam/golibs/errors"
@ -14,6 +11,8 @@ import (
"golang.org/x/exp/slices"
)
// Legacy DNS rewrites
// LegacyRewrite is a single legacy DNS rewrite record.
//
// Instances of *LegacyRewrite must never be nil.
@ -123,38 +122,24 @@ func matchDomainWildcard(host, wildcard string) (ok bool) {
return isWildcard(wildcard) && strings.HasSuffix(host, wildcard[1:])
}
// rewritesSorted is a slice of legacy rewrites for sorting.
// legacyRewriteSortsBefore sorts rewirtes according to the following priority:
//
// The sorting priority:
//
// 1. A and AAAA > CNAME
// 2. wildcard > exact
// 3. lower level wildcard > higher level wildcard
//
// TODO(a.garipov): Replace with slices.Sort.
type rewritesSorted []*LegacyRewrite
// Len implements the sort.Interface interface for rewritesSorted.
func (a rewritesSorted) Len() (l int) { return len(a) }
// Swap implements the sort.Interface interface for rewritesSorted.
func (a rewritesSorted) Swap(i, j int) { a[i], a[j] = a[j], a[i] }
// Less implements the sort.Interface interface for rewritesSorted.
func (a rewritesSorted) Less(i, j int) (less bool) {
ith, jth := a[i], a[j]
if ith.Type == dns.TypeCNAME && jth.Type != dns.TypeCNAME {
// 1. A and AAAA > CNAME;
// 2. wildcard > exact;
// 3. lower level wildcard > higher level wildcard;
func legacyRewriteSortsBefore(a, b *LegacyRewrite) (sortsBefore bool) {
if a.Type == dns.TypeCNAME && b.Type != dns.TypeCNAME {
return true
} else if ith.Type != dns.TypeCNAME && jth.Type == dns.TypeCNAME {
} else if a.Type != dns.TypeCNAME && b.Type == dns.TypeCNAME {
return false
}
if iw, jw := isWildcard(ith.Domain), isWildcard(jth.Domain); iw != jw {
return jw
if aIsWld, bIsWld := isWildcard(a.Domain), isWildcard(b.Domain); aIsWld != bIsWld {
return bIsWld
}
// Both are either wildcards or not.
return len(ith.Domain) > len(jth.Domain)
// Both are either wildcards or both aren't.
return len(a.Domain) > len(b.Domain)
}
// prepareRewrites normalizes and validates all legacy DNS rewrites.
@ -196,7 +181,7 @@ func findRewrites(
return nil, matched
}
sort.Sort(rewritesSorted(rewrites))
slices.SortFunc(rewrites, legacyRewriteSortsBefore)
for i, r := range rewrites {
if isWildcard(r.Domain) {

6
internal/filtering/safebrowsing.go
View File

@ -8,7 +8,6 @@ import (
"fmt"
"net"
"net/http"
"sort"
"strings"
"sync"
"time"
@ -19,6 +18,7 @@ import (
"github.com/AdguardTeam/golibs/log"
"github.com/AdguardTeam/golibs/stringutil"
"github.com/miekg/dns"
"golang.org/x/exp/slices"
"golang.org/x/net/publicsuffix"
)
@ -241,8 +241,8 @@ func (c *sbCtx) processTXT(resp *dns.Msg) (bool, [][]byte) {
}
func (c *sbCtx) storeCache(hashes [][]byte) {
sort.Slice(hashes, func(a, b int) bool {
return bytes.Compare(hashes[a], hashes[b]) == -1
slices.SortFunc(hashes, func(a, b []byte) (sortsBefore bool) {
return bytes.Compare(a, b) == -1
})
var curData []byte

29
internal/filtering/safesearch.go
View File

@ -13,8 +13,37 @@ import (
"github.com/AdguardTeam/AdGuardHome/internal/aghhttp"
"github.com/AdguardTeam/golibs/cache"
"github.com/AdguardTeam/golibs/log"
"github.com/AdguardTeam/urlfilter/rules"
)
// SafeSearch interface describes a service for search engines hosts rewrites.
type SafeSearch interface {
// SearchHost returns a replacement address for the search engine host.
SearchHost(host string, qtype uint16) (res *rules.DNSRewrite)
// CheckHost checks host with safe search engine.
CheckHost(host string, qtype uint16) (res Result, err error)
}
// SafeSearchConfig is a struct with safe search related settings.
type SafeSearchConfig struct {
// CustomResolver is the resolver used by safe search.
CustomResolver Resolver `yaml:"-"`
// Enabled indicates if safe search is enabled entirely.
Enabled bool `yaml:"enabled" json:"enabled"`
// Services flags. Each flag indicates if the corresponding service is
// enabled or disabled.
Bing bool `yaml:"bing" json:"bing"`
DuckDuckGo bool `yaml:"duckduckgo" json:"duckduckgo"`
Google bool `yaml:"google" json:"google"`
Pixabay bool `yaml:"pixabay" json:"pixabay"`
Yandex bool `yaml:"yandex" json:"yandex"`
YouTube bool `yaml:"youtube" json:"youtube"`
}
/*
expire byte[4]
res Result

34
internal/filtering/safesearch/rules.go Normal file
View File

@ -0,0 +1,34 @@
package safesearch
import _ "embed"
//go:embed rules/bing.txt
var bing string
//go:embed rules/google.txt
var google string
//go:embed rules/pixabay.txt
var pixabay string
//go:embed rules/duckduckgo.txt
var duckduckgo string
//go:embed rules/yandex.txt
var yandex string
//go:embed rules/youtube.txt
var youtube string
// safeSearchRules is a map with rules texts grouped by search providers.
// Source rules downloaded from:
// https://adguardteam.github.io/HostlistsRegistry/assets/engines_safe_search.txt,
// https://adguardteam.github.io/HostlistsRegistry/assets/youtube_safe_search.txt.
var safeSearchRules = map[Service]string{
Bing: bing,
DuckDuckGo: duckduckgo,
Google: google,
Pixabay: pixabay,
Yandex: yandex,
YouTube: youtube,
}

1
internal/filtering/safesearch/rules/bing.txt Normal file
View File

@ -0,0 +1 @@
|www.bing.com^$dnsrewrite=NOERROR;CNAME;strict.bing.com

3
internal/filtering/safesearch/rules/duckduckgo.txt Normal file
View File

@ -0,0 +1,3 @@
|duckduckgo.com^$dnsrewrite=NOERROR;CNAME;safe.duckduckgo.com
|start.duckduckgo.com^$dnsrewrite=NOERROR;CNAME;safe.duckduckgo.com
|www.duckduckgo.com^$dnsrewrite=NOERROR;CNAME;safe.duckduckgo.com

191
internal/filtering/safesearch/rules/google.txt Normal file
View File

@ -0,0 +1,191 @@
|www.google.ad^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.ae^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.al^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.am^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.as^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.at^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.az^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.ba^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.be^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.bf^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.bg^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.bi^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.bj^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.bs^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.bt^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.by^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.ca^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.cat^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.cd^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.cf^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.cg^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.ch^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.ci^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.cl^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.cm^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.cn^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.co^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.co.ao^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.co.bw^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.co.ck^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.co.cr^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.co.id^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.co.il^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.co.in^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.co.jp^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.co.ke^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.co.kr^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.co.ls^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.co.ma^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.co.mz^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.co.nz^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.co.th^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.co.tz^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.co.ug^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.co.uk^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.co.uz^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.co.ve^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.co.vi^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.af^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.ag^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.ai^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.ar^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.au^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.bd^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.bh^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.bn^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.bo^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.br^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.bz^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.co^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.cu^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.cy^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.do^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.ec^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.eg^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.et^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.fj^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.gh^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.gi^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.gt^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.hk^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.jm^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.kh^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.kw^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.lb^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.ly^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.mm^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.mt^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.mx^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.my^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.na^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.nf^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.ng^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.ni^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.np^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.om^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.pa^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.pe^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.pg^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.ph^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.pk^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.pr^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.py^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.qa^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.sa^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.sb^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.sg^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.sl^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.sv^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.tj^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.tr^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.tw^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.ua^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.uy^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.vc^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com.vn^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.com^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.cv^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.cz^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.de^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.dj^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.dk^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.dm^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.dz^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.ee^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.es^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.fi^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.fm^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.fr^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.ga^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.ge^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.gg^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.gl^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.gm^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.gp^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.gr^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.gy^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.hn^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.hr^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.ht^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.hu^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.ie^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.im^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.iq^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.is^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.it^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.je^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.jo^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.kg^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.ki^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.kz^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.la^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.li^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.lk^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.lt^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.lu^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.lv^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.md^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.me^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.mg^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.mk^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.ml^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.mn^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.ms^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.mu^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.mv^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.mw^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.ne^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.nl^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.no^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.nr^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.nu^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.pl^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.pn^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.ps^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.pt^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.ro^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.rs^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.ru^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.rw^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.sc^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.se^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.sh^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.si^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.sk^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.sm^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.sn^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.so^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.sr^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.st^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.td^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.tg^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.tk^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.tl^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.tm^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.tn^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.to^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.tt^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.vg^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.vu^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com
|www.google.ws^$dnsrewrite=NOERROR;CNAME;forcesafesearch.google.com

1
internal/filtering/safesearch/rules/pixabay.txt Normal file
View File

@ -0,0 +1 @@
|pixabay.com^$dnsrewrite=NOERROR;CNAME;safesearch.pixabay.com

52
internal/filtering/safesearch/rules/yandex.txt Normal file
View File

@ -0,0 +1,52 @@
|www.xn--d1acpjx3f.xn--p1ai^$dnsrewrite=NOERROR;A;213.180.193.56
|www.ya.ru^$dnsrewrite=NOERROR;A;213.180.193.56
|www.yandex.az^$dnsrewrite=NOERROR;A;213.180.193.56
|www.yandex.by^$dnsrewrite=NOERROR;A;213.180.193.56
|www.yandex.co.il^$dnsrewrite=NOERROR;A;213.180.193.56
|www.yandex.com.am^$dnsrewrite=NOERROR;A;213.180.193.56
|www.yandex.com.ge^$dnsrewrite=NOERROR;A;213.180.193.56
|www.yandex.com.ru^$dnsrewrite=NOERROR;A;213.180.193.56
|www.yandex.com.tr^$dnsrewrite=NOERROR;A;213.180.193.56
|www.yandex.com^$dnsrewrite=NOERROR;A;213.180.193.56
|www.yandex.de^$dnsrewrite=NOERROR;A;213.180.193.56
|www.yandex.ee^$dnsrewrite=NOERROR;A;213.180.193.56
|www.yandex.eu^$dnsrewrite=NOERROR;A;213.180.193.56
|www.yandex.fi^$dnsrewrite=NOERROR;A;213.180.193.56
|www.yandex.fr^$dnsrewrite=NOERROR;A;213.180.193.56
|www.yandex.kz^$dnsrewrite=NOERROR;A;213.180.193.56
|www.yandex.lt^$dnsrewrite=NOERROR;A;213.180.193.56
|www.yandex.lv^$dnsrewrite=NOERROR;A;213.180.193.56
|www.yandex.md^$dnsrewrite=NOERROR;A;213.180.193.56
|www.yandex.net^$dnsrewrite=NOERROR;A;213.180.193.56
|www.yandex.org^$dnsrewrite=NOERROR;A;213.180.193.56
|www.yandex.pl^$dnsrewrite=NOERROR;A;213.180.193.56
|www.yandex.ru^$dnsrewrite=NOERROR;A;213.180.193.56
|www.yandex.tj^$dnsrewrite=NOERROR;A;213.180.193.56
|www.yandex.tm^$dnsrewrite=NOERROR;A;213.180.193.56
|www.yandex.uz^$dnsrewrite=NOERROR;A;213.180.193.56
|xn--d1acpjx3f.xn--p1ai^$dnsrewrite=NOERROR;A;213.180.193.56
|ya.ru^$dnsrewrite=NOERROR;A;213.180.193.56
|yandex.az^$dnsrewrite=NOERROR;A;213.180.193.56
|yandex.by^$dnsrewrite=NOERROR;A;213.180.193.56
|yandex.co.il^$dnsrewrite=NOERROR;A;213.180.193.56
|yandex.com.am^$dnsrewrite=NOERROR;A;213.180.193.56
|yandex.com.ge^$dnsrewrite=NOERROR;A;213.180.193.56
|yandex.com.ru^$dnsrewrite=NOERROR;A;213.180.193.56
|yandex.com.tr^$dnsrewrite=NOERROR;A;213.180.193.56
|yandex.com^$dnsrewrite=NOERROR;A;213.180.193.56
|yandex.de^$dnsrewrite=NOERROR;A;213.180.193.56
|yandex.ee^$dnsrewrite=NOERROR;A;213.180.193.56
|yandex.eu^$dnsrewrite=NOERROR;A;213.180.193.56
|yandex.fi^$dnsrewrite=NOERROR;A;213.180.193.56
|yandex.fr^$dnsrewrite=NOERROR;A;213.180.193.56
|yandex.kz^$dnsrewrite=NOERROR;A;213.180.193.56
|yandex.lt^$dnsrewrite=NOERROR;A;213.180.193.56
|yandex.lv^$dnsrewrite=NOERROR;A;213.180.193.56
|yandex.md^$dnsrewrite=NOERROR;A;213.180.193.56
|yandex.net^$dnsrewrite=NOERROR;A;213.180.193.56
|yandex.org^$dnsrewrite=NOERROR;A;213.180.193.56
|yandex.pl^$dnsrewrite=NOERROR;A;213.180.193.56
|yandex.ru^$dnsrewrite=NOERROR;A;213.180.193.56
|yandex.tj^$dnsrewrite=NOERROR;A;213.180.193.56
|yandex.tm^$dnsrewrite=NOERROR;A;213.180.193.56
|yandex.uz^$dnsrewrite=NOERROR;A;213.180.193.56

5
internal/filtering/safesearch/rules/youtube.txt Normal file
View File

@ -0,0 +1,5 @@
|www.youtube.com^$dnsrewrite=NOERROR;CNAME;restrictmoderate.youtube.com
|m.youtube.com^$dnsrewrite=NOERROR;CNAME;restrictmoderate.youtube.com
|youtubei.googleapis.com^$dnsrewrite=NOERROR;CNAME;restrictmoderate.youtube.com
|youtube.googleapis.com^$dnsrewrite=NOERROR;CNAME;restrictmoderate.youtube.com
|www.youtube-nocookie.com^$dnsrewrite=NOERROR;CNAME;restrictmoderate.youtube.com

269
internal/filtering/safesearch/safesearch.go Normal file
View File

@ -0,0 +1,269 @@
// Package safesearch implements safesearch host matching.
package safesearch
import (
"bytes"
"context"
"encoding/binary"
"encoding/gob"
"fmt"
"net"
"strings"
"time"
"github.com/AdguardTeam/AdGuardHome/internal/filtering"
"github.com/AdguardTeam/golibs/cache"
"github.com/AdguardTeam/golibs/log"
"github.com/AdguardTeam/urlfilter"
"github.com/AdguardTeam/urlfilter/filterlist"
"github.com/AdguardTeam/urlfilter/rules"
"github.com/miekg/dns"
)
// Service is a enum with service names used as search providers.
type Service string
// Service enum members.
const (
Bing Service = "bing"
DuckDuckGo Service = "duckduckgo"
Google Service = "google"
Pixabay Service = "pixabay"
Yandex Service = "yandex"
YouTube Service = "youtube"
)
// isServiceProtected returns true if the service safe search is active.
func isServiceProtected(s filtering.SafeSearchConfig, service Service) (ok bool) {
switch service {
case Bing:
return s.Bing
case DuckDuckGo:
return s.DuckDuckGo
case Google:
return s.Google
case Pixabay:
return s.Pixabay
case Yandex:
return s.Yandex
case YouTube:
return s.YouTube
default:
panic(fmt.Errorf("safesearch: invalid sources: not found service %q", service))
}
}
// DefaultSafeSearch is the default safesearch struct.
type DefaultSafeSearch struct {
engine *urlfilter.DNSEngine
safeSearchCache cache.Cache
resolver filtering.Resolver
cacheTime time.Duration
}
// NewDefaultSafeSearch returns new safesearch struct. CacheTime is an element
// TTL (in minutes).
func NewDefaultSafeSearch(
conf filtering.SafeSearchConfig,
cacheSize uint,
cacheTime time.Duration,
) (ss *DefaultSafeSearch, err error) {
engine, err := newEngine(filtering.SafeSearchListID, conf)
if err != nil {
return nil, err
}
var resolver filtering.Resolver = net.DefaultResolver
if conf.CustomResolver != nil {
resolver = conf.CustomResolver
}
return &DefaultSafeSearch{
engine: engine,
safeSearchCache: cache.New(cache.Config{
EnableLRU: true,
MaxSize: cacheSize,
}),
cacheTime: cacheTime,
resolver: resolver,
}, nil
}
// newEngine creates new engine for provided safe search configuration.
func newEngine(listID int, conf filtering.SafeSearchConfig) (engine *urlfilter.DNSEngine, err error) {
var sb strings.Builder
for service, serviceRules := range safeSearchRules {
if isServiceProtected(conf, service) {
sb.WriteString(serviceRules)
}
}
strList := &filterlist.StringRuleList{
ID: listID,
RulesText: sb.String(),
IgnoreCosmetic: true,
}
rs, err := filterlist.NewRuleStorage([]filterlist.RuleList{strList})
if err != nil {
return nil, fmt.Errorf("creating rule storage: %w", err)
}
engine = urlfilter.NewDNSEngine(rs)
log.Info("safesearch: filter %d: reset %d rules", listID, engine.RulesCount)
return engine, nil
}
// type check
var _ filtering.SafeSearch = (*DefaultSafeSearch)(nil)
// SearchHost implements the [filtering.SafeSearch] interface for *DefaultSafeSearch.
func (ss *DefaultSafeSearch) SearchHost(host string, qtype uint16) (res *rules.DNSRewrite) {
r, _ := ss.engine.MatchRequest(&urlfilter.DNSRequest{
Hostname: strings.ToLower(host),
DNSType: qtype,
})
rewritesRules := r.DNSRewrites()
if len(rewritesRules) > 0 {
return rewritesRules[0].DNSRewrite
}
return nil
}
// CheckHost implements the [filtering.SafeSearch] interface for
// *DefaultSafeSearch.
func (ss *DefaultSafeSearch) CheckHost(
host string,
qtype uint16,
) (res filtering.Result, err error) {
if log.GetLevel() >= log.DEBUG {
timer := log.StartTimer()
defer timer.LogElapsed("safesearch: lookup for %s", host)
}
// Check cache. Return cached result if it was found
cachedValue, isFound := ss.getCachedResult(host)
if isFound {
log.Debug("safesearch: found in cache: %s", host)
return cachedValue, nil
}
rewrite := ss.SearchHost(host, qtype)
if rewrite == nil {
return filtering.Result{}, nil
}
dRes, err := ss.newResult(rewrite, qtype)
if err != nil {
log.Debug("safesearch: failed to lookup addresses for %s: %s", host, err)
return filtering.Result{}, err
}
if dRes != nil {
res = *dRes
ss.setCacheResult(host, res)
return res, nil
}
return filtering.Result{}, fmt.Errorf("no ipv4 addresses in safe search response for %s", host)
}
// newResult creates Result object from rewrite rule.
func (ss *DefaultSafeSearch) newResult(
rewrite *rules.DNSRewrite,
qtype uint16,
) (res *filtering.Result, err error) {
res = &filtering.Result{
Rules: []*filtering.ResultRule{{
FilterListID: filtering.SafeSearchListID,
}},
Reason: filtering.FilteredSafeSearch,
IsFiltered: true,
}
if rewrite.RRType == qtype && (qtype == dns.TypeA || qtype == dns.TypeAAAA) {
ip, ok := rewrite.Value.(net.IP)
if !ok || ip == nil {
return nil, nil
}
res.Rules[0].IP = ip
return res, nil
}
if rewrite.NewCNAME == "" {
return nil, nil
}
ips, err := ss.resolver.LookupIP(context.Background(), "ip", rewrite.NewCNAME)
if err != nil {
return nil, err
}
for _, ip := range ips {
if ip = ip.To4(); ip == nil {
continue
}
res.Rules[0].IP = ip
return res, nil
}
return nil, nil
}
// setCacheResult stores data in cache for host.
func (ss *DefaultSafeSearch) setCacheResult(host string, res filtering.Result) {
expire := uint32(time.Now().Add(ss.cacheTime).Unix())
exp := make([]byte, 4)
binary.BigEndian.PutUint32(exp, expire)
buf := bytes.NewBuffer(exp)
err := gob.NewEncoder(buf).Encode(res)
if err != nil {
log.Error("safesearch: cache encoding: %s", err)
return
}
val := buf.Bytes()
_ = ss.safeSearchCache.Set([]byte(host), val)
log.Debug("safesearch: stored in cache: %s (%d bytes)", host, len(val))
}
// getCachedResult returns stored data from cache for host.
func (ss *DefaultSafeSearch) getCachedResult(host string) (res filtering.Result, ok bool) {
res = filtering.Result{}
data := ss.safeSearchCache.Get([]byte(host))
if data == nil {
return res, false
}
exp := binary.BigEndian.Uint32(data[:4])
if exp <= uint32(time.Now().Unix()) {
ss.safeSearchCache.Del([]byte(host))
return res, false
}
buf := bytes.NewBuffer(data[4:])
err := gob.NewDecoder(buf).Decode(&res)
if err != nil {
log.Debug("safesearch: cache decoding: %s", err)
return filtering.Result{}, false
}
return res, true
}

202
internal/filtering/safesearch/safesearch_test.go Normal file
View File

@ -0,0 +1,202 @@
package safesearch
import (
"context"
"net"
"testing"
"time"
"github.com/AdguardTeam/AdGuardHome/internal/aghtest"
"github.com/AdguardTeam/AdGuardHome/internal/filtering"
"github.com/AdguardTeam/urlfilter/rules"
"github.com/miekg/dns"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
const (
safeSearchCacheSize = 5000
cacheTime = 30 * time.Minute
)
var defaultSafeSearchConf = filtering.SafeSearchConfig{
Enabled: true,
Bing: true,
DuckDuckGo: true,
Google: true,
Pixabay: true,
Yandex: true,
YouTube: true,
}
var yandexIP = net.IPv4(213, 180, 193, 56)
func newForTest(t testing.TB, ssConf filtering.SafeSearchConfig) (ss *DefaultSafeSearch) {
ss, err := NewDefaultSafeSearch(ssConf, safeSearchCacheSize, cacheTime)
require.NoError(t, err)
return ss
}
func TestSafeSearch(t *testing.T) {
ss := newForTest(t, defaultSafeSearchConf)
val := ss.SearchHost("www.google.com", dns.TypeA)
assert.Equal(t, &rules.DNSRewrite{NewCNAME: "forcesafesearch.google.com"}, val)
}
func TestCheckHostSafeSearchYandex(t *testing.T) {
ss := newForTest(t, defaultSafeSearchConf)
// Check host for each domain.
for _, host := range []string{
"yandex.ru",
"yAndeX.ru",
"YANdex.COM",
"yandex.by",
"yandex.kz",
"www.yandex.com",
} {
res, err := ss.CheckHost(host, dns.TypeA)
require.NoError(t, err)
assert.True(t, res.IsFiltered)
require.Len(t, res.Rules, 1)
assert.Equal(t, yandexIP, res.Rules[0].IP)
assert.EqualValues(t, filtering.SafeSearchListID, res.Rules[0].FilterListID)
}
}
func TestCheckHostSafeSearchGoogle(t *testing.T) {
resolver := &aghtest.TestResolver{}
ip, _ := resolver.HostToIPs("forcesafesearch.google.com")
ss := newForTest(t, defaultSafeSearchConf)
ss.resolver = resolver
// Check host for each domain.
for _, host := range []string{
"www.google.com",
"www.google.im",
"www.google.co.in",
"www.google.iq",
"www.google.is",
"www.google.it",
"www.google.je",
} {
t.Run(host, func(t *testing.T) {
res, err := ss.CheckHost(host, dns.TypeA)
require.NoError(t, err)
assert.True(t, res.IsFiltered)
require.Len(t, res.Rules, 1)
assert.Equal(t, ip, res.Rules[0].IP)
assert.EqualValues(t, filtering.SafeSearchListID, res.Rules[0].FilterListID)
})
}
}
func TestSafeSearchCacheYandex(t *testing.T) {
const domain = "yandex.ru"
ss := newForTest(t, filtering.SafeSearchConfig{Enabled: false})
// Check host with disabled safesearch.
res, err := ss.CheckHost(domain, dns.TypeA)
require.NoError(t, err)
assert.False(t, res.IsFiltered)
assert.Empty(t, res.Rules)
ss = newForTest(t, defaultSafeSearchConf)
res, err = ss.CheckHost(domain, dns.TypeA)
require.NoError(t, err)
// For yandex we already know valid IP.
require.Len(t, res.Rules, 1)
assert.Equal(t, res.Rules[0].IP, yandexIP)
// Check cache.
cachedValue, isFound := ss.getCachedResult(domain)
require.True(t, isFound)
require.Len(t, cachedValue.Rules, 1)
assert.Equal(t, cachedValue.Rules[0].IP, yandexIP)
}
func TestSafeSearchCacheGoogle(t *testing.T) {
const domain = "www.google.ru"
ss := newForTest(t, filtering.SafeSearchConfig{Enabled: false})
res, err := ss.CheckHost(domain, dns.TypeA)
require.NoError(t, err)
assert.False(t, res.IsFiltered)
assert.Empty(t, res.Rules)
resolver := &aghtest.TestResolver{}
ss = newForTest(t, defaultSafeSearchConf)
ss.resolver = resolver
// Lookup for safesearch domain.
rewrite := ss.SearchHost(domain, dns.TypeA)
ips, err := resolver.LookupIP(context.Background(), "ip", rewrite.NewCNAME)
require.NoError(t, err)
var foundIP net.IP
for _, ip := range ips {
if ip.To4() != nil {
foundIP = ip
break
}
}
res, err = ss.CheckHost(domain, dns.TypeA)
require.NoError(t, err)
require.Len(t, res.Rules, 1)
assert.True(t, res.Rules[0].IP.Equal(foundIP))
// Check cache.
cachedValue, isFound := ss.getCachedResult(domain)
require.True(t, isFound)
require.Len(t, cachedValue.Rules, 1)
assert.True(t, cachedValue.Rules[0].IP.Equal(foundIP))
}
const googleHost = "www.google.com"
var dnsRewriteSink *rules.DNSRewrite
func BenchmarkSafeSearch(b *testing.B) {
ss := newForTest(b, defaultSafeSearchConf)
for n := 0; n < b.N; n++ {
dnsRewriteSink = ss.SearchHost(googleHost, dns.TypeA)
}
assert.Equal(b, "forcesafesearch.google.com", dnsRewriteSink.NewCNAME)
}
var dnsRewriteParallelSink *rules.DNSRewrite
func BenchmarkSafeSearch_parallel(b *testing.B) {
ss := newForTest(b, defaultSafeSearchConf)
b.RunParallel(func(pb *testing.PB) {
for pb.Next() {
dnsRewriteParallelSink = ss.SearchHost(googleHost, dns.TypeA)
}
})
assert.Equal(b, "forcesafesearch.google.com", dnsRewriteParallelSink.NewCNAME)
}

53
internal/filtering/servicelist.go
View File

@ -71,6 +71,7 @@ var blockedServices = []blockedService{{
"||amazon.jp^",
"||amazon.nl^",
"||amazon.red^",
"||amazon.se^",
"||amazon.sg^",
"||amazon^",
"||amazonalexavoxcon.com^",
@ -1171,6 +1172,16 @@ var blockedServices = []blockedService{{
"||zuckerberg.com^",
"||zuckerberg.net^",
},
}, {
ID: "gog",
Name: "GOG",
IconSVG: []byte("<svg xmlns=\"http://www.w3.org/2000/svg\" fill=\"currentColor\" viewBox=\"0 0 34 34\"><path d=\"M31 31H3a3 3 0 0 1-3-3V3A3 3 0 0 1 3 0H31a3 3 0 0 1 3 3V28A3 3 0 0 1 31 31ZM4 24.5A1.5 1.5 0 0 0 5.5 26H11V24H6.5a.5.5 0 0 1-.5-.5v-3a.5.5 0 0 1 .5-.5H11V18H5.5A1.5 1.5 0 0 0 4 19.5Zm8-18A1.5 1.5 0 0 0 10.5 5h-5A1.5 1.5 0 0 0 4 6.5v5A1.5 1.5 0 0 0 5.5 13H9V11H6.5a.5.5 0 0 1-.5-.5v-3A.5.5 0 0 1 6.5 7h3a.5.5 0 0 1 .5.5v6a.5.5 0 0 1-.5.5H4v2h6.5A1.5 1.5 0 0 0 12 14.5Zm0 13v5A1.5 1.5 0 0 0 13.5 26h5A1.5 1.5 0 0 0 20 24.5v-5A1.5 1.5 0 0 0 18.5 18h-5A1.5 1.5 0 0 0 12 19.5Zm9-13A1.5 1.5 0 0 0 19.5 5h-5A1.5 1.5 0 0 0 13 6.5v5A1.5 1.5 0 0 0 14.5 13h5A1.5 1.5 0 0 0 21 11.5Zm9 0A1.5 1.5 0 0 0 28.5 5h-5A1.5 1.5 0 0 0 22 6.5v5A1.5 1.5 0 0 0 23.5 13H27V11H24.5a.5.5 0 0 1-.5-.5v-3a.5.5 0 0 1 .5-.5h3a.5.5 0 0 1 .5.5v6a.5.5 0 0 1-.5.5H22v2h6.5A1.5 1.5 0 0 0 30 14.5ZM30 18H22.5A1.5 1.5 0 0 0 21 19.5V26h2V20.5a.5.5 0 0 1 .5-.5h1v6h2V20H28v6h2ZM18.5 11h-3a.5.5 0 0 1-.5-.5v-3a.5.5 0 0 1 .5-.5h3a.5.5 0 0 1 .5.5v3A.5.5 0 0 1 18.5 11Zm-4 9h3a.5.5 0 0 1 .5.5v3a.5.5 0 0 1-.5.5h-3a.5.5 0 0 1-.5-.5v-3A.5.5 0 0 1 14.5 20Z\" /></svg>"),
Rules: []string{
"||gog-cdn-lumen.secure2.footprint.net^",
"||gog-statics.com^",
"||gog.com^",
"||gogalaxy.com^",
},
}, {
ID: "hulu",
Name: "Hulu",
@ -1280,6 +1291,14 @@ var blockedServices = []blockedService{{
"||iq.com^",
"||iqiyi.com^",
},
}, {
ID: "kakaotalk",
Name: "KakaoTalk",
IconSVG: []byte("<svg xmlns=\"http://www.w3.org/2000/svg\" fill=\"currentColor\" viewBox=\"0 0 24 24\" ><path d=\"M22.125 0H1.875C.839 0 0 .84 0 1.875v20.25C0 23.161.84 24 1.875 24h20.25C23.161 24 24 23.16 24 22.125V1.875C24 .839 23.16 0 22.125 0zM12 18.75c-.591 0-1.17-.041-1.732-.12-.562.396-3.813 2.679-4.12 2.722 0 0-.125.049-.232-.014s-.088-.229-.088-.229c.032-.22.843-3.018.992-3.533-2.745-1.36-4.57-3.769-4.57-6.513 0-4.246 4.365-7.688 9.75-7.688s9.75 3.442 9.75 7.688c0 4.245-4.365 7.687-9.75 7.687zM8.05 9.867h-.878v3.342c0 .296-.252.537-.563.537s-.562-.24-.562-.537V9.867h-.878a.552.552 0 0 1 0-1.101h2.88a.552.552 0 0 1 0 1.101zm10.987 2.957a.558.558 0 0 1 .109.417.559.559 0 0 1-.219.37.557.557 0 0 1-.338.114.558.558 0 0 1-.45-.224l-1.319-1.747-.195.195v1.227a.564.564 0 0 1-.562.563.563.563 0 0 1-.563-.563V9.328a.563.563 0 0 1 1.125 0v1.21l1.57-1.57a.437.437 0 0 1 .311-.126c.14 0 .282.061.388.167a.555.555 0 0 1 .165.356.438.438 0 0 1-.124.343l-1.282 1.281 1.385 1.835zm-8.35-3.502c-.095-.27-.383-.548-.75-.556-.366.008-.654.286-.749.555l-1.345 3.541c-.171.53-.022.728.133.8a.857.857 0 0 0 .357.077c.235 0 .414-.095.468-.248l.279-.73h1.715l.279.73c.054.153.233.248.468.248a.86.86 0 0 0 .357-.078c.155-.071.304-.268.133-.8l-1.345-3.54zm-1.311 2.443.562-1.596.561 1.596H9.376zm5.905 1.383a.528.528 0 0 1-.539.516h-1.804a.528.528 0 0 1-.54-.516v-3.82c0-.31.258-.562.575-.562s.574.252.574.562v3.305h1.195c.297 0 .54.231.54.515z\"/></svg>"),
Rules: []string{
"||kakao.com^",
"||kgslb.com^",
},
}, {
ID: "leagueoflegends",
Name: "League of Legends",
@ -1291,6 +1310,24 @@ var blockedServices = []blockedService{{
"||lolstatic.com^",
"||lolusercontent.com^",
},
}, {
ID: "line",
Name: "LINE",
IconSVG: []byte("<svg xmlns=\"http://www.w3.org/2000/svg\" fill=\"currentColor\" viewBox=\"0 0 50 50\"><path d=\"M 9 4 C 6.24 4 4 6.24 4 9 L 4 41 C 4 43.76 6.24 46 9 46 L 41 46 C 43.76 46 46 43.76 46 41 L 46 9 C 46 6.24 43.76 4 41 4 L 9 4 z M 25 11 C 33.27 11 40 16.359219 40 22.949219 C 40 25.579219 38.959297 27.960781 36.779297 30.300781 C 35.209297 32.080781 32.660547 34.040156 30.310547 35.660156 C 27.960547 37.260156 25.8 38.519609 25 38.849609 C 24.68 38.979609 24.44 39.039062 24.25 39.039062 C 23.59 39.039062 23.649219 38.340781 23.699219 38.050781 C 23.739219 37.830781 23.919922 36.789063 23.919922 36.789062 C 23.969922 36.419063 24.019141 35.830937 23.869141 35.460938 C 23.699141 35.050938 23.029062 34.840234 22.539062 34.740234 C 15.339063 33.800234 10 28.849219 10 22.949219 C 10 16.359219 16.73 11 25 11 z M 23.992188 18.998047 C 23.488379 19.007393 23 19.391875 23 20 L 23 26 C 23 26.552 23.448 27 24 27 C 24.552 27 25 26.552 25 26 L 25 23.121094 L 27.185547 26.580078 C 27.751547 27.372078 29 26.973 29 26 L 29 20 C 29 19.448 28.552 19 28 19 C 27.448 19 27 19.448 27 20 L 27 23 L 24.814453 19.419922 C 24.602203 19.122922 24.294473 18.992439 23.992188 18.998047 z M 15 19 C 14.448 19 14 19.448 14 20 L 14 26 C 14 26.552 14.448 27 15 27 L 18 27 C 18.552 27 19 26.552 19 26 C 19 25.448 18.552 25 18 25 L 16 25 L 16 20 C 16 19.448 15.552 19 15 19 z M 21 19 C 20.448 19 20 19.448 20 20 L 20 26 C 20 26.552 20.448 27 21 27 C 21.552 27 22 26.552 22 26 L 22 20 C 22 19.448 21.552 19 21 19 z M 31 19 C 30.448 19 30 19.448 30 20 L 30 26 C 30 26.552 30.448 27 31 27 L 34 27 C 34.552 27 35 26.552 35 26 C 35 25.448 34.552 25 34 25 L 32 25 L 32 24 L 34 24 C 34.553 24 35 23.552 35 23 C 35 22.448 34.553 22 34 22 L 32 22 L 32 21 L 34 21 C 34.552 21 35 20.552 35 20 C 35 19.448 34.552 19 34 19 L 31 19 z\"/></svg>"),
Rules: []string{
"||gcld-line.com^",
"||lin.ee^",
"||line-apps-beta.com^",
"||line-apps-rc.com^",
"||line-apps.com^",
"||line-cdn.net^",
"||line-scdn.net^",
"||line.me^",
"||line.naver.jp^",
"||linecorp.com^",
"||linemyshop.com^",
"||lineshoppingseller.com^",
},
}, {
ID: "mail_ru",
Name: "Mail.ru",
@ -1308,13 +1345,13 @@ var blockedServices = []blockedService{{
"||aus.social^",
"||awscommunity.social^",
"||cyberplace.social^",
"||defcon.social^",
"||det.social^",
"||fosstodon.org^",
"||glasgow.social^",
"||h4.io^",
"||hachyderm.io^",
"||hessen.social^",
"||home.social^",
"||hostux.social^",
"||ieji.de^",
"||indieweb.social^",
@ -1333,6 +1370,7 @@ var blockedServices = []blockedService{{
"||mastodon.au^",
"||mastodon.bida.im^",
"||mastodon.com.tr^",
"||mastodon.eus^",
"||mastodon.green^",
"||mastodon.ie^",
"||mastodon.iriseden.eu^",
@ -1360,10 +1398,8 @@ var blockedServices = []blockedService{{
"||mindly.social^",
"||mstdn.ca^",
"||mstdn.jp^",
"||mstdn.party^",
"||mstdn.social^",
"||muenchen.social^",
"||nerdculture.de^",
"||newsie.social^",
"||noc.social^",
"||norden.social^",
@ -1382,6 +1418,7 @@ var blockedServices = []blockedService{{
"||social.anoxinon.de^",
"||social.cologne^",
"||social.dev-wiki.de^",
"||social.linux.pizza^",
"||social.politicaconciencia.org^",
"||social.vivaldi.net^",
"||sself.co^",
@ -1398,11 +1435,11 @@ var blockedServices = []blockedService{{
"||toot.wales^",
"||troet.cafe^",
"||twingyeo.kr^",
"||uiuxdev.social^",
"||union.place^",
"||universeodon.com^",
"||urbanists.social^",
"||vocalodon.net^",
"||wien.rocks^",
"||wxw.moe^",
},
}, {
@ -1599,6 +1636,14 @@ var blockedServices = []blockedService{{
"||snapchat.com^",
"||snapkit.co",
},
}, {
ID: "soundcloud",
Name: "SoundCloud",
IconSVG: []byte("<svg xmlns=\"http://www.w3.org/2000/svg\" fill=\"currentColor\" viewBox=\"0 0 24 24\" fill-rule=\"evenodd\" clip-rule=\"evenodd\"><path d=\"M19 17.75c2.07 0 3.75-1.68 3.75-3.75 0-2.07-1.68-3.75-3.75-3.75-.173 0-.344.012-.511.035-.73-2.337-2.913-4.035-5.489-4.035-.818 0-1.596.171-2.301.48-.273.119-.449.389-.449.687l0 9.583c0 .414.336.75.75.75l8 0zM7.25 8l0 9c0 .414.336.75.75.75.414 0 .75-.336.75-.75l0-9c0-.414-.336-.75-.75-.75-.414 0-.75.336-.75.75zM4.25 10l0 7c0 .414.336.75.75.75.414 0 .75-.336.75-.75l0-7c0-.414-.336-.75-.75-.75-.414 0-.75.336-.75.75zM1.25 12l0 5c0 .414.336.75.75.75.414 0 .75-.336.75-.75l0-5c0-.414-.336-.75-.75-.75-.414 0-.75.336-.75.75z\"/></svg>"),
Rules: []string{
"||sndcdn.com^",
"||soundcloud.com^",
},
}, {
ID: "spotify",
Name: "Spotify",

102
internal/home/client.go Normal file
View File

@ -0,0 +1,102 @@
package home
import (
"encoding"
"fmt"
"github.com/AdguardTeam/dnsproxy/proxy"
)
// Client contains information about persistent clients.
type Client struct {
// upstreamConfig is the custom upstream config for this client. If
// it's nil, it has not been initialized yet. If it's non-nil and
// empty, there are no valid upstreams. If it's non-nil and non-empty,
// these upstream must be used.
upstreamConfig *proxy.UpstreamConfig
Name string
IDs []string
Tags []string
BlockedServices []string
Upstreams []string
UseOwnSettings bool
FilteringEnabled bool
SafeSearchEnabled bool
SafeBrowsingEnabled bool
ParentalEnabled bool
UseOwnBlockedServices bool
}
// closeUpstreams closes the client-specific upstream config of c if any.
func (c *Client) closeUpstreams() (err error) {
if c.upstreamConfig != nil {
err = c.upstreamConfig.Close()
if err != nil {
return fmt.Errorf("closing upstreams of client %q: %w", c.Name, err)
}
}
return nil
}
// clientSource represents the source from which the information about the
// client has been obtained.
type clientSource uint
// Clients information sources. The order determines the priority.
const (
ClientSourceNone clientSource = iota
ClientSourceWHOIS
ClientSourceARP
ClientSourceRDNS
ClientSourceDHCP
ClientSourceHostsFile
ClientSourcePersistent
)
// type check
var _ fmt.Stringer = clientSource(0)
// String returns a human-readable name of cs.
func (cs clientSource) String() (s string) {
switch cs {
case ClientSourceWHOIS:
return "WHOIS"
case ClientSourceARP:
return "ARP"
case ClientSourceRDNS:
return "rDNS"
case ClientSourceDHCP:
return "DHCP"
case ClientSourceHostsFile:
return "etc/hosts"
default:
return ""
}
}
// type check
var _ encoding.TextMarshaler = clientSource(0)
// MarshalText implements encoding.TextMarshaler for the clientSource.
func (cs clientSource) MarshalText() (text []byte, err error) {
return []byte(cs.String()), nil
}
// RuntimeClient is a client information about which has been obtained using the
// source described in the Source field.
type RuntimeClient struct {
WHOISInfo *RuntimeClientWHOISInfo
Host string
Source clientSource
}
// RuntimeClientWHOISInfo is the filtered WHOIS data for a runtime client.
type RuntimeClientWHOISInfo struct {
City string `json:"city,omitempty"`
Country string `json:"country,omitempty"`
Orgname string `json:"orgname,omitempty"`
}

298
internal/home/clients.go
View File

@ -2,11 +2,9 @@ package home
import (
"bytes"
"encoding"
"fmt"
"net"
"net/netip"
"sort"
"strings"
"sync"
"time"
@ -26,122 +24,16 @@ import (
"golang.org/x/exp/slices"
)
const clientsUpdatePeriod = 10 * time.Minute
var webHandlersRegistered = false
// Client contains information about persistent clients.
type Client struct {
// upstreamConfig is the custom upstream config for this client. If
// it's nil, it has not been initialized yet. If it's non-nil and
// empty, there are no valid upstreams. If it's non-nil and non-empty,
// these upstream must be used.
upstreamConfig *proxy.UpstreamConfig
Name string
IDs []string
Tags []string
BlockedServices []string
Upstreams []string
UseOwnSettings bool
FilteringEnabled bool
SafeSearchEnabled bool
SafeBrowsingEnabled bool
ParentalEnabled bool
UseOwnBlockedServices bool
}
// closeUpstreams closes the client-specific upstream config of c if any.
func (c *Client) closeUpstreams() (err error) {
if c.upstreamConfig != nil {
err = c.upstreamConfig.Close()
if err != nil {
return fmt.Errorf("closing upstreams of client %q: %w", c.Name, err)
}
}
return nil
}
type clientSource uint
// Clients information sources. The order determines the priority.
const (
ClientSourceNone clientSource = iota
ClientSourceWHOIS
ClientSourceARP
ClientSourceRDNS
ClientSourceDHCP
ClientSourceHostsFile
ClientSourcePersistent
)
// type check
var _ fmt.Stringer = clientSource(0)
// String returns a human-readable name of cs.
func (cs clientSource) String() (s string) {
switch cs {
case ClientSourceWHOIS:
return "WHOIS"
case ClientSourceARP:
return "ARP"
case ClientSourceRDNS:
return "rDNS"
case ClientSourceDHCP:
return "DHCP"
case ClientSourceHostsFile:
return "etc/hosts"
default:
return ""
}
}
// type check
var _ encoding.TextMarshaler = clientSource(0)
// MarshalText implements encoding.TextMarshaler for the clientSource.
func (cs clientSource) MarshalText() (text []byte, err error) {
return []byte(cs.String()), nil
}
// clientSourceConf is used to configure where the runtime clients will be
// obtained from.
type clientSourcesConf struct {
WHOIS bool `yaml:"whois"`
ARP bool `yaml:"arp"`
RDNS bool `yaml:"rdns"`
DHCP bool `yaml:"dhcp"`
HostsFile bool `yaml:"hosts"`
}
// RuntimeClient information
type RuntimeClient struct {
WHOISInfo *RuntimeClientWHOISInfo
Host string
Source clientSource
}
// RuntimeClientWHOISInfo is the filtered WHOIS data for a runtime client.
type RuntimeClientWHOISInfo struct {
City string `json:"city,omitempty"`
Country string `json:"country,omitempty"`
Orgname string `json:"orgname,omitempty"`
}
// clientsContainer is the storage of all runtime and persistent clients.
type clientsContainer struct {
// TODO(a.garipov): Perhaps use a number of separate indices for
// different types (string, netip.Addr, and so on).
// TODO(a.garipov): Perhaps use a number of separate indices for different
// types (string, netip.Addr, and so on).
list map[string]*Client // name -> client
idIndex map[string]*Client // ID -> client
// ipToRC is the IP address to *RuntimeClient map.
ipToRC map[netip.Addr]*RuntimeClient
lock sync.Mutex
allTags *stringutil.Set
// dhcpServer is used for looking up clients IP addresses by MAC addresses
@ -157,7 +49,16 @@ type clientsContainer struct {
// arpdb stores the neighbors retrieved from ARP.
arpdb aghnet.ARPDB
testing bool // if TRUE, this object is used for internal tests
// lock protects all fields.
//
// TODO(a.garipov): Use a pointer and describe which fields are protected in
// more detail.
lock sync.Mutex
// testing is a flag that disables some features for internal tests.
//
// TODO(a.garipov): Awful. Remove.
testing bool
}
// Init initializes clients container
@ -203,24 +104,34 @@ func (clients *clientsContainer) handleHostsUpdates() {
}
}
// Start - start the module
// webHandlersRegistered prevents a [clientsContainer] from regisering its web
// handlers more than once.
//
// TODO(a.garipov): Refactor HTTP handler registration logic.
var webHandlersRegistered = false
// Start starts the clients container.
func (clients *clientsContainer) Start() {
if !clients.testing {
if clients.testing {
return
}
if !webHandlersRegistered {
webHandlersRegistered = true
clients.registerWebHandlers()
}
go clients.periodicUpdate()
}
}
// Reload reloads runtime clients.
func (clients *clientsContainer) Reload() {
// reloadARP reloads runtime clients from ARP, if configured.
func (clients *clientsContainer) reloadARP() {
if clients.arpdb != nil {
clients.addFromSystemARP()
}
}
// clientObject is the YAML representation of a persistent client.
type clientObject struct {
Name string `yaml:"name"`
@ -271,7 +182,7 @@ func (clients *clientsContainer) addFromConfig(objects []*clientObject) {
}
}
sort.Strings(cli.Tags)
slices.Sort(cli.Tags)
_, err := clients.Add(cli)
if err != nil {
@ -311,17 +222,22 @@ func (clients *clientsContainer) forConfig() (objs []*clientObject) {
// above loop can generate different orderings when writing to the config
// file: this produces lots of diffs in config files, so sort objects by
// name before writing.
sort.Slice(objs, func(i, j int) bool { return objs[i].Name < objs[j].Name })
slices.SortStableFunc(objs, func(a, b *clientObject) (sortsBefore bool) {
return a.Name < b.Name
})
return objs
}
// arpClientsUpdatePeriod defines how often ARP clients are updated.
const arpClientsUpdatePeriod = 10 * time.Minute
func (clients *clientsContainer) periodicUpdate() {
defer log.OnPanic("clients container")
for {
clients.Reload()
time.Sleep(clientsUpdatePeriod)
clients.reloadARP()
time.Sleep(arpClientsUpdatePeriod)
}
}
@ -484,7 +400,8 @@ func (clients *clientsContainer) findUpstreams(
return conf, nil
}
// findLocked searches for a client by its ID. For internal use only.
// findLocked searches for a client by its ID. clients.lock is expected to be
// locked.
func (clients *clientsContainer) findLocked(id string) (c *Client, ok bool) {
c, ok = clients.idIndex[id]
if ok {
@ -498,13 +415,13 @@ func (clients *clientsContainer) findLocked(id string) (c *Client, ok bool) {
for _, c = range clients.list {
for _, id := range c.IDs {
var n netip.Prefix
n, err = netip.ParsePrefix(id)
var subnet netip.Prefix
subnet, err = netip.ParsePrefix(id)
if err != nil {
continue
}
if n.Contains(ip) {
if subnet.Contains(ip) {
return c, true
}
}
@ -514,20 +431,25 @@ func (clients *clientsContainer) findLocked(id string) (c *Client, ok bool) {
return nil, false
}
macFound := clients.dhcpServer.FindMACbyIP(ip.AsSlice())
if macFound == nil {
return clients.findDHCP(ip)
}
// findDHCP searches for a client by its MAC, if the DHCP server is active and
// there is such client. clients.lock is expected to be locked.
func (clients *clientsContainer) findDHCP(ip netip.Addr) (c *Client, ok bool) {
foundMAC := clients.dhcpServer.FindMACbyIP(ip)
if foundMAC == nil {
return nil, false
}
for _, c = range clients.list {
for _, id := range c.IDs {
var mac net.HardwareAddr
mac, err = net.ParseMAC(id)
mac, err := net.ParseMAC(id)
if err != nil {
continue
}
if bytes.Equal(mac, macFound) {
if bytes.Equal(mac, foundMAC) {
return c, true
}
}
@ -564,24 +486,13 @@ func (clients *clientsContainer) check(c *Client) (err error) {
}
for i, id := range c.IDs {
// Normalize structured data.
var (
ip netip.Addr
n netip.Prefix
mac net.HardwareAddr
)
if ip, err = netip.ParseAddr(id); err == nil {
c.IDs[i] = ip.String()
} else if n, err = netip.ParsePrefix(id); err == nil {
c.IDs[i] = n.String()
} else if mac, err = net.ParseMAC(id); err == nil {
c.IDs[i] = mac.String()
} else if err = dnsforward.ValidateClientID(id); err == nil {
c.IDs[i] = strings.ToLower(id)
} else {
return fmt.Errorf("invalid clientid at index %d: %q", i, id)
var norm string
norm, err = normalizeClientIdentifier(id)
if err != nil {
return fmt.Errorf("client at index %d: %w", i, err)
}
c.IDs[i] = norm
}
for _, t := range c.Tags {
@ -590,7 +501,7 @@ func (clients *clientsContainer) check(c *Client) (err error) {
}
}
sort.Strings(c.Tags)
slices.Sort(c.Tags)
err = dnsforward.ValidateUpstreams(c.Upstreams)
if err != nil {
@ -600,6 +511,35 @@ func (clients *clientsContainer) check(c *Client) (err error) {
return nil
}
// normalizeClientIdentifier returns a normalized version of idStr. If idStr
// cannot be normalized, it returns an error.
func normalizeClientIdentifier(idStr string) (norm string, err error) {
if idStr == "" {
return "", errors.Error("clientid is empty")
}
var ip netip.Addr
if ip, err = netip.ParseAddr(idStr); err == nil {
return ip.String(), nil
}
var subnet netip.Prefix
if subnet, err = netip.ParsePrefix(idStr); err == nil {
return subnet.String(), nil
}
var mac net.HardwareAddr
if mac, err = net.ParseMAC(idStr); err == nil {
return mac.String(), nil
}
if err = dnsforward.ValidateClientID(idStr); err == nil {
return strings.ToLower(idStr), nil
}
return "", fmt.Errorf("bad client identifier %q", idStr)
}
// Add adds a new client object. ok is false if such client already exists or
// if an error occurred.
func (clients *clientsContainer) Add(c *Client) (ok bool, err error) {
@ -665,21 +605,6 @@ func (clients *clientsContainer) Del(name string) (ok bool) {
return true
}
// equalStringSlices returns true if the slices are equal.
func equalStringSlices(a, b []string) (ok bool) {
if len(a) != len(b) {
return false
}
for i := range a {
if a[i] != b[i] {
return false
}
}
return true
}
// Update updates a client by its name.
func (clients *clientsContainer) Update(name string, c *Client) (err error) {
err = clients.check(c)
@ -703,22 +628,11 @@ func (clients *clientsContainer) Update(name string, c *Client) (err error) {
}
}
// Second, check the IP index.
if !equalStringSlices(prev.IDs, c.IDs) {
for _, id := range c.IDs {
c2, ok2 := clients.idIndex[id]
if ok2 && c2 != prev {
return fmt.Errorf("another client uses the same id (%q): %q", id, c2.Name)
}
}
// Update ID index.
for _, id := range prev.IDs {
delete(clients.idIndex, id)
}
for _, id := range c.IDs {
clients.idIndex[id] = prev
}
// Second, update the ID index.
err = clients.updateIDIndex(prev, c.IDs)
if err != nil {
// Don't wrap the error, because it's informative enough as is.
return err
}
// Update name index.
@ -738,6 +652,32 @@ func (clients *clientsContainer) Update(name string, c *Client) (err error) {
return nil
}
// updateIDIndex updates the ID index data for cli using the information from
// newIDs.
func (clients *clientsContainer) updateIDIndex(cli *Client, newIDs []string) (err error) {
if slices.Equal(cli.IDs, newIDs) {
return nil
}
for _, id := range newIDs {
existing, ok := clients.idIndex[id]
if ok && existing != cli {
return fmt.Errorf("id %q is used by client with name %q", id, existing.Name)
}
}
// Update the IDs in the index.
for _, id := range cli.IDs {
delete(clients.idIndex, id)
}
for _, id := range newIDs {
clients.idIndex[id] = cli
}
return nil
}
// setWHOISInfo sets the WHOIS information for a client.
func (clients *clientsContainer) setWHOISInfo(ip netip.Addr, wi *RuntimeClientWHOISInfo) {
clients.lock.Lock()

26
internal/home/config.go
View File

@ -6,7 +6,6 @@ import (
"net/netip"
"os"
"path/filepath"
"sort"
"sync"
"github.com/AdguardTeam/AdGuardHome/internal/aghalg"
@ -21,6 +20,7 @@ import (
"github.com/AdguardTeam/golibs/log"
"github.com/AdguardTeam/golibs/timeutil"
"github.com/google/renameio/maybe"
"golang.org/x/exp/slices"
yaml "gopkg.in/yaml.v3"
)
@ -75,11 +75,21 @@ type osConfig struct {
type clientsConfig struct {
// Sources defines the set of sources to fetch the runtime clients from.
Sources *clientSourcesConf `yaml:"runtime_sources"`
Sources *clientSourcesConfig `yaml:"runtime_sources"`
// Persistent are the configured clients.
Persistent []*clientObject `yaml:"persistent"`
}
// clientSourceConfig is used to configure where the runtime clients will be
// obtained from.
type clientSourcesConfig struct {
WHOIS bool `yaml:"whois"`
ARP bool `yaml:"arp"`
RDNS bool `yaml:"rdns"`
DHCP bool `yaml:"dhcp"`
HostsFile bool `yaml:"hosts"`
}
// configuration is loaded from YAML
// field ordering is important -- yaml fields will mirror ordering from here
type configuration struct {
@ -275,6 +285,12 @@ var config = &configuration{
TrustedProxies: []string{"127.0.0.0/8", "::1/128"},
CacheSize: 4 * 1024 * 1024,
EDNSClientSubnet: &dnsforward.EDNSClientSubnet{
CustomIP: "",
Enabled: false,
UseCustom: false,
},
// set default maximum concurrent queries to 300
// we introduced a default limit due to this:
// https://github.com/AdguardTeam/AdGuardHome/issues/2015#issuecomment-674041912
@ -336,7 +352,7 @@ var config = &configuration{
},
},
Clients: &clientsConfig{
Sources: &clientSourcesConf{
Sources: &clientSourcesConfig{
WHOIS: true,
ARP: true,
RDNS: true,
@ -490,7 +506,7 @@ func (c *configuration) write() (err error) {
config.Stats.Interval = statsConf.LimitDays
config.Stats.Enabled = statsConf.Enabled
config.Stats.Ignored = statsConf.Ignored.Values()
sort.Strings(config.Stats.Ignored)
slices.Sort(config.Stats.Ignored)
}
if Context.queryLog != nil {
@ -502,7 +518,7 @@ func (c *configuration) write() (err error) {
config.QueryLog.Interval = timeutil.Duration{Duration: dc.RotationIvl}
config.QueryLog.MemSize = dc.MemSize
config.QueryLog.Ignored = dc.Ignored.Values()
sort.Strings(config.QueryLog.Ignored)
slices.Sort(config.Stats.Ignored)
}
if Context.filters != nil {

2
internal/home/controlupdate.go
View File

@ -98,7 +98,7 @@ func requestVersionInfo(resp *versionResponse, recheck bool) (err error) {
if err != nil {
vcu := Context.updater.VersionCheckURL()
return fmt.Errorf("getting version info from %s: %s", vcu, err)
return fmt.Errorf("getting version info from %s: %w", vcu, err)
}
return nil

2
internal/home/home.go
View File

@ -125,7 +125,7 @@ func Main(clientBuildFS fs.FS) {
log.Info("Received signal %q", sig)
switch sig {
case syscall.SIGHUP:
Context.clients.Reload()
Context.clients.reloadARP()
Context.tls.reload()
default:
cleanup(context.Background())

48
internal/home/upgrade.go
View File

@ -22,7 +22,7 @@ import (
)
// currentSchemaVersion is the current schema version.
const currentSchemaVersion = 16
const currentSchemaVersion = 17
// These aliases are provided for convenience.
type (
@ -89,6 +89,7 @@ func upgradeConfigSchema(oldVersion int, diskConf yobj) (err error) {
upgradeSchema13to14,
upgradeSchema14to15,
upgradeSchema15to16,
upgradeSchema16to17,
}
n := 0
@ -792,7 +793,7 @@ func upgradeSchema13to14(diskConf yobj) (err error) {
diskConf["clients"] = yobj{
"persistent": clientsVal,
"runtime_sources": &clientSourcesConf{
"runtime_sources": &clientSourcesConfig{
WHOIS: true,
ARP: true,
RDNS: rdnsSrc,
@ -892,19 +893,56 @@ func upgradeSchema15to16(diskConf yobj) (err error) {
"ignored": []any{},
}
k := "statistics_interval"
v, has := dns[k]
const field = "statistics_interval"
v, has := dns[field]
if has {
stats["enabled"] = v != 0
stats["interval"] = v
}
delete(dns, k)
delete(dns, field)
diskConf["statistics"] = stats
return nil
}
// upgradeSchema16to17 performs the following changes:
//
// # BEFORE:
// 'dns':
// 'edns_client_subnet': false
//
// # AFTER:
// 'dns':
// 'edns_client_subnet':
// 'enabled': false
// 'use_custom': false
// 'custom_ip': ""
func upgradeSchema16to17(diskConf yobj) (err error) {
log.Printf("Upgrade yaml: 16 to 17")
diskConf["schema_version"] = 17
dnsVal, ok := diskConf["dns"]
if !ok {
return nil
}
dns, ok := dnsVal.(yobj)
if !ok {
return fmt.Errorf("unexpected type of dns: %T", dnsVal)
}
const field = "edns_client_subnet"
dns[field] = map[string]any{
"enabled": dns[field] == true,
"use_custom": false,
"custom_ip": "",
}
return nil
}
// TODO(a.garipov): Replace with log.Output when we port it to our logging
// package.
func funcName() string {

67
internal/home/upgrade_test.go
View File

@ -579,7 +579,7 @@ func TestUpgradeSchema13to14(t *testing.T) {
// The clients field will be added anyway.
"clients": yobj{
"persistent": yarr{},
"runtime_sources": &clientSourcesConf{
"runtime_sources": &clientSourcesConfig{
WHOIS: true,
ARP: true,
RDNS: false,
@ -597,7 +597,7 @@ func TestUpgradeSchema13to14(t *testing.T) {
"schema_version": newSchemaVer,
"clients": yobj{
"persistent": []*clientObject{testClient},
"runtime_sources": &clientSourcesConf{
"runtime_sources": &clientSourcesConfig{
WHOIS: true,
ARP: true,
RDNS: false,
@ -618,7 +618,7 @@ func TestUpgradeSchema13to14(t *testing.T) {
"schema_version": newSchemaVer,
"clients": yobj{
"persistent": []*clientObject{testClient},
"runtime_sources": &clientSourcesConf{
"runtime_sources": &clientSourcesConfig{
WHOIS: true,
ARP: true,
RDNS: true,
@ -747,3 +747,64 @@ func TestUpgradeSchema15to16(t *testing.T) {
})
}
}
func TestUpgradeSchema16to17(t *testing.T) {
const newSchemaVer = 17
defaultWantObj := yobj{
"dns": map[string]any{
"edns_client_subnet": map[string]any{
"enabled": false,
"use_custom": false,
"custom_ip": "",
},
},
"schema_version": newSchemaVer,
}
testCases := []struct {
in yobj
want yobj
name string
}{{
in: yobj{
"dns": map[string]any{
"edns_client_subnet": false,
},
},
want: defaultWantObj,
name: "basic",
}, {
in: yobj{
"dns": map[string]any{},
},
want: defaultWantObj,
name: "default_values",
}, {
in: yobj{
"dns": map[string]any{
"edns_client_subnet": true,
},
},
want: yobj{
"dns": map[string]any{
"edns_client_subnet": map[string]any{
"enabled": true,
"use_custom": false,
"custom_ip": "",
},
},
"schema_version": newSchemaVer,
},
name: "is_true",
}}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
err := upgradeSchema16to17(tc.in)
require.NoError(t, err)
assert.Equal(t, tc.want, tc.in)
})
}
}

67
internal/querylog/decode.go
View File

@ -247,55 +247,20 @@ var resultHandlers = map[string]logEntryHandler{
}
func decodeResultRuleKey(key string, i int, dec *json.Decoder, ent *logEntry) {
var vToken json.Token
switch key {
case "FilterListID":
vToken, err := dec.Token()
if err != nil {
if err != io.EOF {
log.Debug("decodeResultRuleKey %s err: %s", key, err)
}
return
}
if len(ent.Result.Rules) < i+1 {
ent.Result.Rules = append(ent.Result.Rules, &filtering.ResultRule{})
}
ent.Result.Rules, vToken = decodeVTokenAndAddRule(key, i, dec, ent.Result.Rules)
if n, ok := vToken.(json.Number); ok {
ent.Result.Rules[i].FilterListID, _ = n.Int64()
}
case "IP":
vToken, err := dec.Token()
if err != nil {
if err != io.EOF {
log.Debug("decodeResultRuleKey %s err: %s", key, err)
}
return
}
if len(ent.Result.Rules) < i+1 {
ent.Result.Rules = append(ent.Result.Rules, &filtering.ResultRule{})
}
ent.Result.Rules, vToken = decodeVTokenAndAddRule(key, i, dec, ent.Result.Rules)
if ipStr, ok := vToken.(string); ok {
ent.Result.Rules[i].IP = net.ParseIP(ipStr)
}
case "Text":
vToken, err := dec.Token()
if err != nil {
if err != io.EOF {
log.Debug("decodeResultRuleKey %s err: %s", key, err)
}
return
}
if len(ent.Result.Rules) < i+1 {
ent.Result.Rules = append(ent.Result.Rules, &filtering.ResultRule{})
}
ent.Result.Rules, vToken = decodeVTokenAndAddRule(key, i, dec, ent.Result.Rules)
if s, ok := vToken.(string); ok {
ent.Result.Rules[i].Text = s
}
@ -304,6 +269,30 @@ func decodeResultRuleKey(key string, i int, dec *json.Decoder, ent *logEntry) {
}
}
func decodeVTokenAndAddRule(
key string,
i int,
dec *json.Decoder,
rules []*filtering.ResultRule,
) (newRules []*filtering.ResultRule, vToken json.Token) {
newRules = rules
vToken, err := dec.Token()
if err != nil {
if err != io.EOF {
log.Debug("decodeResultRuleKey %s err: %s", key, err)
}
return newRules, nil
}
if len(rules) < i+1 {
newRules = append(newRules, &filtering.ResultRule{})
}
return newRules, vToken
}
func decodeResultRules(dec *json.Decoder, ent *logEntry) {
for {
delimToken, err := dec.Token()

3
internal/querylog/http.go
View File

@ -54,10 +54,7 @@ func (l *queryLog) handleQueryLog(w http.ResponseWriter, r *http.Request) {
return
}
// search for the log entries
entries, oldest := l.search(params)
// convert log entries to JSON
data := l.entriesToJSON(entries, oldest)
_ = aghhttp.WriteJSONResponse(w, r, data)

58
internal/querylog/json.go
View File

@ -1,7 +1,6 @@
package querylog
import (
"fmt"
"strconv"
"strings"
"time"
@ -117,7 +116,7 @@ func (l *queryLog) setMsgData(entry *logEntry, jsonEntry jobject) {
// it from there as well.
jsonEntry["answer_dnssec"] = entry.AuthenticatedData || msg.AuthenticatedData
if a := answerToMap(msg); a != nil {
if a := answerToJSON(msg); a != nil {
jsonEntry["answer"] = a
}
}
@ -136,7 +135,7 @@ func (l *queryLog) setOrigAns(entry *logEntry, jsonEntry jobject) {
return
}
if a := answerToMap(orig); a != nil {
if a := answerToJSON(orig); a != nil {
jsonEntry["original_answer"] = a
}
}
@ -159,55 +158,24 @@ type dnsAnswer struct {
TTL uint32 `json:"ttl"`
}
func answerToMap(a *dns.Msg) (answers []*dnsAnswer) {
if a == nil || len(a.Answer) == 0 {
// answerToJSON converts the answer records of msg, if any, to their JSON form.
func answerToJSON(msg *dns.Msg) (answers []*dnsAnswer) {
if msg == nil || len(msg.Answer) == 0 {
return nil
}
answers = make([]*dnsAnswer, 0, len(a.Answer))
for _, k := range a.Answer {
header := k.Header()
answer := &dnsAnswer{
answers = make([]*dnsAnswer, 0, len(msg.Answer))
for _, rr := range msg.Answer {
header := rr.Header()
a := &dnsAnswer{
Type: dns.TypeToString[header.Rrtype],
// Remove the header string from the answer value since it's mostly
// unnecessary in the log.
Value: strings.TrimPrefix(rr.String(), header.String()),
TTL: header.Ttl,
}
// Some special treatment for some well-known types.
//
// TODO(a.garipov): Consider just calling String() for everyone
// instead.
switch v := k.(type) {
case nil:
// Probably unlikely, but go on.
case *dns.A:
answer.Value = v.A.String()
case *dns.AAAA:
answer.Value = v.AAAA.String()
case *dns.MX:
answer.Value = fmt.Sprintf("%v %v", v.Preference, v.Mx)
case *dns.CNAME:
answer.Value = v.Target
case *dns.NS:
answer.Value = v.Ns
case *dns.SPF:
answer.Value = strings.Join(v.Txt, "\n")
case *dns.TXT:
answer.Value = strings.Join(v.Txt, "\n")
case *dns.PTR:
answer.Value = v.Ptr
case *dns.SOA:
answer.Value = fmt.Sprintf("%v %v %v %v %v %v %v", v.Ns, v.Mbox, v.Serial, v.Refresh, v.Retry, v.Expire, v.Minttl)
case *dns.CAA:
answer.Value = fmt.Sprintf("%v %v \"%v\"", v.Flag, v.Tag, v.Value)
case *dns.HINFO:
answer.Value = fmt.Sprintf("\"%v\" \"%v\"", v.Cpu, v.Os)
case *dns.RRSIG:
answer.Value = fmt.Sprintf("%v %v %v %v %v %v %v %v %v", dns.TypeToString[v.TypeCovered], v.Algorithm, v.Labels, v.OrigTtl, v.Expiration, v.Inception, v.KeyTag, v.SignerName, v.Signature)
default:
answer.Value = v.String()
}
answers = append(answers, answer)
answers = append(answers, a)
}
return answers

10
internal/querylog/qlog.go
View File

@ -31,7 +31,8 @@ type queryLog struct {
// bufferLock protects buffer.
bufferLock sync.RWMutex
// buffer contains recent log entries.
// buffer contains recent log entries. The entries in this buffer must not
// be modified.
buffer []*logEntry
fileFlushLock sync.Mutex // synchronize a file-flushing goroutine and main thread
@ -100,6 +101,13 @@ type logEntry struct {
AuthenticatedData bool `json:"AD,omitempty"`
}
// shallowClone returns a shallow clone of e.
func (e *logEntry) shallowClone() (clone *logEntry) {
cloneVal := *e
return &cloneVal
}
func (l *queryLog) Start() {
if l.conf.HTTPRegister != nil {
l.initWeb()

72
internal/querylog/qlog_test.go
View File

@ -2,11 +2,8 @@ package querylog
import (
"fmt"
"math/rand"
"net"
"sort"
"testing"
"time"
"github.com/AdguardTeam/AdGuardHome/internal/filtering"
"github.com/AdguardTeam/dnsproxy/proxyutil"
@ -352,72 +349,3 @@ func assertLogEntry(t *testing.T, entry *logEntry, host string, answer, client n
ip := proxyutil.IPFromRR(msg.Answer[0]).To16()
assert.Equal(t, answer, ip)
}
func testEntries() (entries []*logEntry) {
rsrc := rand.NewSource(time.Now().UnixNano())
rgen := rand.New(rsrc)
entries = make([]*logEntry, 1000)
for i := range entries {
min := rgen.Intn(60)
sec := rgen.Intn(60)
entries[i] = &logEntry{
Time: time.Date(2020, 1, 1, 0, min, sec, 0, time.UTC),
}
}
return entries
}
// logEntriesByTimeDesc is a wrapper over []*logEntry for sorting.
//
// NOTE(a.garipov): Weirdly enough, on my machine this gets consistently
// outperformed by sort.Slice, see the benchmark below. I'm leaving this
// implementation here, in tests, in case we want to make sure it outperforms on
// most machines, but for now this is unused in the actual code.
type logEntriesByTimeDesc []*logEntry
// Len implements the sort.Interface interface for logEntriesByTimeDesc.
func (les logEntriesByTimeDesc) Len() (n int) { return len(les) }
// Less implements the sort.Interface interface for logEntriesByTimeDesc.
func (les logEntriesByTimeDesc) Less(i, j int) (less bool) {
return les[i].Time.After(les[j].Time)
}
// Swap implements the sort.Interface interface for logEntriesByTimeDesc.
func (les logEntriesByTimeDesc) Swap(i, j int) { les[i], les[j] = les[j], les[i] }
func BenchmarkLogEntry_sort(b *testing.B) {
b.Run("methods", func(b *testing.B) {
for i := 0; i < b.N; i++ {
b.StopTimer()
entries := testEntries()
b.StartTimer()
sort.Stable(logEntriesByTimeDesc(entries))
}
})
b.Run("reflect", func(b *testing.B) {
for i := 0; i < b.N; i++ {
b.StopTimer()
entries := testEntries()
b.StartTimer()
sort.SliceStable(entries, func(i, j int) (less bool) {
return entries[i].Time.After(entries[j].Time)
})
}
})
}
func TestLogEntriesByTime_sort(t *testing.T) {
entries := testEntries()
sort.Sort(logEntriesByTimeDesc(entries))
for i := range entries[1:] {
assert.False(t, entries[i+1].Time.After(entries[i].Time),
"%s %s", entries[i+1].Time, entries[i].Time)
}
}

12
internal/querylog/search.go
View File

@ -2,10 +2,10 @@ package querylog
import (
"io"
"sort"
"time"
"github.com/AdguardTeam/golibs/log"
"golang.org/x/exp/slices"
)
// client finds the client info, if any, by its ClientID and IP address,
@ -52,7 +52,9 @@ func (l *queryLog) searchMemory(params *searchParams, cache clientCache) (entrie
// Go through the buffer in the reverse order, from newer to older.
var err error
for i := len(l.buffer) - 1; i >= 0; i-- {
e := l.buffer[i]
// A shallow clone is enough, since the only thing that this loop
// modifies is the client field.
e := l.buffer[i].shallowClone()
e.client, err = l.client(e.ClientID, e.IP.String(), cache)
if err != nil {
@ -98,8 +100,8 @@ func (l *queryLog) search(params *searchParams) (entries []*logEntry, oldest tim
// weird on the frontend.
//
// See https://github.com/AdguardTeam/AdGuardHome/issues/2293.
sort.SliceStable(entries, func(i, j int) (less bool) {
return entries[i].Time.After(entries[j].Time)
slices.SortStableFunc(entries, func(a, b *logEntry) (sortsBefore bool) {
return a.Time.After(b.Time)
})
if params.offset > 0 {
@ -130,7 +132,7 @@ func (l *queryLog) search(params *searchParams) (entries []*logEntry, oldest tim
// searchFiles looks up log records from all log files. It optionally uses the
// client cache, if provided. searchFiles does not scan more than
// maxFileScanEntries so callers may need to call it several times to get all
// results. oldset and total are the time of the oldest processed entry and the
// results. oldest and total are the time of the oldest processed entry and the
// total number of processed entries, including discarded ones, correspondingly.
func (l *queryLog) searchFiles(
params *searchParams,

85
internal/querylog/searchcriterion.go
View File

@ -1,6 +1,7 @@
package querylog
import (
"fmt"
"strings"
"github.com/AdguardTeam/AdGuardHome/internal/filtering"
@ -118,7 +119,7 @@ func (c *searchCriterion) match(entry *logEntry) bool {
case ctTerm:
return c.ctDomainOrClientCase(entry)
case ctFilteringStatus:
return c.ctFilteringStatusCase(entry.Result)
return c.ctFilteringStatusCase(entry.Result.Reason, entry.Result.IsFiltered)
}
return false
@ -141,54 +142,70 @@ func (c *searchCriterion) ctDomainOrClientCase(e *logEntry) bool {
return ctDomainOrClientCaseNonStrict(c.value, c.asciiVal, clientID, name, host, ip)
}
func (c *searchCriterion) ctFilteringStatusCase(res filtering.Result) bool {
// ctFilteringStatusCase returns true if the result matches the value.
func (c *searchCriterion) ctFilteringStatusCase(
reason filtering.Reason,
isFiltered bool,
) (matched bool) {
switch c.value {
case filteringStatusAll:
return true
case filteringStatusFiltered:
return res.IsFiltered ||
res.Reason.In(
filtering.NotFilteredAllowList,
filtering.Rewritten,
filtering.RewrittenAutoHosts,
filtering.RewrittenRule,
)
case filteringStatusBlocked:
return res.IsFiltered &&
res.Reason.In(filtering.FilteredBlockList, filtering.FilteredBlockedService)
case filteringStatusBlockedService:
return res.IsFiltered && res.Reason == filtering.FilteredBlockedService
case filteringStatusBlockedParental:
return res.IsFiltered && res.Reason == filtering.FilteredParental
case filteringStatusBlockedSafebrowsing:
return res.IsFiltered && res.Reason == filtering.FilteredSafeBrowsing
case
filteringStatusBlocked,
filteringStatusBlockedParental,
filteringStatusBlockedSafebrowsing,
filteringStatusBlockedService,
filteringStatusFiltered,
filteringStatusSafeSearch:
return isFiltered && c.isFilteredWithReason(reason)
case filteringStatusWhitelisted:
return res.Reason == filtering.NotFilteredAllowList
return reason == filtering.NotFilteredAllowList
case filteringStatusRewritten:
return res.Reason.In(
return reason.In(
filtering.Rewritten,
filtering.RewrittenAutoHosts,
filtering.RewrittenRule,
)
case filteringStatusSafeSearch:
return res.IsFiltered && res.Reason == filtering.FilteredSafeSearch
case filteringStatusProcessed:
return !res.Reason.In(
return !reason.In(
filtering.FilteredBlockList,
filtering.FilteredBlockedService,
filtering.NotFilteredAllowList,
)
default:
return false
}
}
// isFilteredWithReason returns true if reason matches the criterion value.
// c.value must be one of:
//
// - filteringStatusBlocked
// - filteringStatusBlockedParental
// - filteringStatusBlockedSafebrowsing
// - filteringStatusBlockedService
// - filteringStatusFiltered
// - filteringStatusSafeSearch
func (c *searchCriterion) isFilteredWithReason(reason filtering.Reason) (matched bool) {
switch c.value {
case filteringStatusBlocked:
return reason.In(filtering.FilteredBlockList, filtering.FilteredBlockedService)
case filteringStatusBlockedParental:
return reason == filtering.FilteredParental
case filteringStatusBlockedSafebrowsing:
return reason == filtering.FilteredSafeBrowsing
case filteringStatusBlockedService:
return reason == filtering.FilteredBlockedService
case filteringStatusFiltered:
return reason.In(
filtering.NotFilteredAllowList,
filtering.Rewritten,
filtering.RewrittenAutoHosts,
filtering.RewrittenRule,
)
case filteringStatusSafeSearch:
return reason == filtering.FilteredSafeSearch
default:
panic(fmt.Errorf("unexpected value %q", c.value))
}
}

6
internal/stats/unit.go
View File

@ -5,13 +5,13 @@ import (
"encoding/binary"
"encoding/gob"
"fmt"
"sort"
"time"
"github.com/AdguardTeam/golibs/errors"
"github.com/AdguardTeam/golibs/log"
"github.com/AdguardTeam/golibs/stringutil"
"go.etcd.io/bbolt"
"golang.org/x/exp/slices"
)
// TODO(a.garipov): Rewrite all of this. Add proper error handling and
@ -180,8 +180,8 @@ func convertMapToSlice(m map[string]uint64, max int) (s []countPair) {
s = append(s, countPair{Name: k, Count: v})
}
sort.Slice(s, func(i, j int) bool {
return s[j].Count < s[i].Count
slices.SortFunc(s, func(a, b countPair) (sortsBefore bool) {
return a.Count > b.Count
})
if max > len(s) {
max = len(s)

6
internal/tools/go.mod
View File

@ -9,9 +9,9 @@ require (
github.com/kisielk/errcheck v1.6.3
github.com/kyoh86/looppointer v0.2.1
github.com/securego/gosec/v2 v2.15.0
golang.org/x/tools v0.6.0
golang.org/x/vuln v0.0.0-20230213165600-1a019b0c7f30
honnef.co/go/tools v0.4.1
golang.org/x/tools v0.6.1-0.20230217175706-3102dad5faf9
golang.org/x/vuln v0.0.0-20230217204342-b91abcc5ae3c
honnef.co/go/tools v0.4.2
mvdan.cc/gofumpt v0.4.0
mvdan.cc/unparam v0.0.0-20230125043941-70a0ce6e7b95
)

12
internal/tools/go.sum
View File

@ -97,10 +97,10 @@ golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtn
golang.org/x/tools v0.0.0-20201007032633-0806396f153e/go.mod h1:z6u4i615ZeAfBE4XtMziQW1fSVJXACjjbWkB/mvPzlU=
golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.1.10/go.mod h1:Uh6Zz+xoGYZom868N8YTex3t7RhtHDBrE8Gzo9bV56E=
golang.org/x/tools v0.6.0 h1:BOw41kyTf3PuCW1pVQf8+Cyg8pMlkYB1oo9iJ6D/lKM=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/vuln v0.0.0-20230213165600-1a019b0c7f30 h1:Q4B8LhSjZGto/+P5REBb4N51ec2H4efRqjBIeJ6nv/Y=
golang.org/x/vuln v0.0.0-20230213165600-1a019b0c7f30/go.mod h1:cBP4HMKv0X+x96j8IJWCKk0eqpakBmmHjKGSSC0NaYE=
golang.org/x/tools v0.6.1-0.20230217175706-3102dad5faf9 h1:IuFp2CklNBim6OdHXn/1P4VoeKt5pA2jcDKWlboqtlQ=
golang.org/x/tools v0.6.1-0.20230217175706-3102dad5faf9/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/vuln v0.0.0-20230217204342-b91abcc5ae3c h1:7/jJkMpaKZMxdyOQ7IP7aPbJQaDk4cOUxtXtWHQ1cSk=
golang.org/x/vuln v0.0.0-20230217204342-b91abcc5ae3c/go.mod h1:LTLnfk/dpXDNKsX6aCg/cI4LyCVnTyrQhgV/yLJuly0=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
@ -109,8 +109,8 @@ gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
honnef.co/go/tools v0.4.1 h1:HPeloSr0mLOEMOkhT9Au5aeki44kvP6ka3v1xIsM6Zo=
honnef.co/go/tools v0.4.1/go.mod h1:36ZgoUOrqOk1GxwHhyryEkq8FQWkUO2xGuSMhUCcdvA=
honnef.co/go/tools v0.4.2 h1:6qXr+R5w+ktL5UkwEbPp+fEvfyoMPche6GkOpGHZcLc=
honnef.co/go/tools v0.4.2/go.mod h1:36ZgoUOrqOk1GxwHhyryEkq8FQWkUO2xGuSMhUCcdvA=
mvdan.cc/gofumpt v0.4.0 h1:JVf4NN1mIpHogBj7ABpgOyZc65/UUOkKQFkoURsz4MM=
mvdan.cc/gofumpt v0.4.0/go.mod h1:PljLOHDeZqgS8opHRKLzp2It2VBuSdteAgqUfzMTxlQ=
mvdan.cc/unparam v0.0.0-20230125043941-70a0ce6e7b95 h1:n/xhncJPSt0YzfOhnyn41XxUdrWQNgmLBG72FE27Fqw=

1
internal/tools/tools.go
View File

@ -1,5 +1,4 @@
//go:build tools
// +build tools
package tools

54
internal/updater/check.go
View File

@ -10,6 +10,9 @@ import (
"github.com/AdguardTeam/AdGuardHome/internal/aghalg"
"github.com/AdguardTeam/AdGuardHome/internal/aghio"
"github.com/AdguardTeam/golibs/errors"
"github.com/AdguardTeam/golibs/log"
"golang.org/x/exp/maps"
"golang.org/x/exp/slices"
)
// TODO(a.garipov): Make configurable.
@ -81,9 +84,9 @@ func (u *Updater) parseVersionResponse(data []byte) (VersionInfo, error) {
return info, fmt.Errorf("version.json: %w", err)
}
for _, v := range versionJSON {
for k, v := range versionJSON {
if v == "" {
return info, fmt.Errorf("version.json: invalid data")
return info, fmt.Errorf("version.json: bad data: value for key %q is empty", k)
}
}
@ -91,9 +94,9 @@ func (u *Updater) parseVersionResponse(data []byte) (VersionInfo, error) {
info.Announcement = versionJSON["announcement"]
info.AnnouncementURL = versionJSON["announcement_url"]
packageURL, ok := u.downloadURL(versionJSON)
if !ok {
return info, fmt.Errorf("version.json: packageURL not found")
packageURL, key, found := u.downloadURL(versionJSON)
if !found {
return info, fmt.Errorf("version.json: no package URL: key %q not found in object", key)
}
info.CanAutoUpdate = aghalg.BoolToNullBool(info.NewVersion != u.version)
@ -104,25 +107,40 @@ func (u *Updater) parseVersionResponse(data []byte) (VersionInfo, error) {
return info, nil
}
// downloadURL returns the download URL for current build.
func (u *Updater) downloadURL(json map[string]string) (string, bool) {
var key string
// downloadURL returns the download URL for current build as well as its key in
// versionObj. If the key is not found, it additionally prints an informative
// log message.
func (u *Updater) downloadURL(versionObj map[string]string) (dlURL, key string, ok bool) {
if u.goarch == "arm" && u.goarm != "" {
key = fmt.Sprintf("download_%s_%sv%s", u.goos, u.goarch, u.goarm)
} else if u.goarch == "mips" && u.gomips != "" {
} else if isMIPS(u.goarch) && u.gomips != "" {
key = fmt.Sprintf("download_%s_%s_%s", u.goos, u.goarch, u.gomips)
}
val, ok := json[key]
if !ok {
} else {
key = fmt.Sprintf("download_%s_%s", u.goos, u.goarch)
val, ok = json[key]
}
if !ok {
return "", false
dlURL, ok = versionObj[key]
if ok {
return dlURL, key, true
}
return val, true
keys := maps.Keys(versionObj)
slices.Sort(keys)
log.Error("updater: key %q not found; got keys %q", key, keys)
return "", key, false
}
// isMIPS returns true if arch is any MIPS architecture.
func isMIPS(arch string) (ok bool) {
switch arch {
case
"mips",
"mips64",
"mips64le",
"mipsle":
return true
default:
return false
}
}

4
internal/updater/updater.go
View File

@ -272,7 +272,7 @@ func (u *Updater) backup(firstRun bool) (err error) {
wd := u.workDir
err = copySupportingFiles(u.unpackedFiles, wd, u.backupDir)
if err != nil {
return fmt.Errorf("copySupportingFiles(%s, %s) failed: %s", wd, u.backupDir, err)
return fmt.Errorf("copySupportingFiles(%s, %s) failed: %w", wd, u.backupDir, err)
}
return nil
@ -283,7 +283,7 @@ func (u *Updater) backup(firstRun bool) (err error) {
func (u *Updater) replace() error {
err := copySupportingFiles(u.unpackedFiles, u.updateDir, u.workDir)
if err != nil {
return fmt.Errorf("copySupportingFiles(%s, %s) failed: %s", u.updateDir, u.workDir, err)
return fmt.Errorf("copySupportingFiles(%s, %s) failed: %w", u.updateDir, u.workDir, err)
}
log.Debug("updater: renaming: %s to %s", u.currentExeName, u.backupExeName)

14
scripts/hooks/pre-commit
View File

@ -2,10 +2,18 @@
set -e -f -u
# Only show interactive prompts if there is a terminal attached. This
# should work on all of our supported Unix systems.
# This comment is used to simplify checking local copies of the script.
# Bump this number every time a significant change is made to this
# script.
#
# AdGuard-Project-Version: 1
# Only show interactive prompts if there a terminal is attached to
# stdout. While this technically doesn't guarantee that reading from
# /dev/tty works, this should work reasonably well on all of our
# supported development systems and in most terminal emulators.
is_tty='0'
if [ -e /dev/tty ]
if [ -t '1' ]
then
is_tty='1'
fi

22
scripts/install.sh
View File

@ -33,6 +33,19 @@ usage() {
exit 2
}
# Function maybe_sudo runs passed command with root privileges if use_sudo isn't
# equal to 0.
#
# TODO(e.burkov): Use everywhere the sudo_cmd isn't quoted.
maybe_sudo() {
if [ "$use_sudo" -eq 0 ]
then
"$@"
else
"$sudo_cmd" "$@"
fi
}
# Function is_command checks if the command exists on the machine.
is_command() {
command -v "$1" >/dev/null 2>&1
@ -554,7 +567,14 @@ handle_existing() {
# Function install_service tries to install AGH as service.
install_service() {
if ( cd "$agh_dir" && ./AdGuardHome -s install )
# Installing the service as root is required at least on FreeBSD.
use_sudo='0'
if [ "$os" = 'freebsd' ]
then
use_sudo='1'
fi
if ( cd "$agh_dir" && maybe_sudo ./AdGuardHome -s install )
then
return 0
fi

2
scripts/make/Dockerfile
View File

@ -1,6 +1,6 @@
# A docker file for scripts/make/build-docker.sh.
FROM alpine:3.16
FROM alpine:3.17
ARG BUILD_DATE
ARG VERSION

10
scripts/make/build-release.sh
View File

@ -390,6 +390,16 @@ echo "{
\"selfupdate_min_version\": \"0.0\",
" >> "$version_json"
# Add the MIPS* object keys without the "softfloat" part to mitigate the
# consequences of #5373.
#
# TODO(a.garipov): Remove this around fall 2023.
echo "
\"download_linux_mips64\": \"${version_download_url}/AdGuardHome_linux_mips64_softfloat.tar.gz\",
\"download_linux_mips64le\": \"${version_download_url}/AdGuardHome_linux_mips64le_softfloat.tar.gz\",
\"download_linux_mipsle\": \"${version_download_url}/AdGuardHome_linux_mipsle_softfloat.tar.gz\",
" >> "$version_json"
# Same as with checksums above, don't use ls, because files matching one of the
# patterns may be absent.
ar_files="$( find "./${dist}/" ! -name "${dist}" -prune \( -name '*.tar.gz' -o -name '*.zip' \) )"

32
scripts/make/go-build.sh
View File

@ -6,6 +6,11 @@
# only has superficial knowledge of the POSIX shell language and alike.
# Experienced readers may find it overly verbose.
# This comment is used to simplify checking local copies of the script. Bump
# this number every time a significant change is made to this script.
#
# AdGuard-Project-Version: 1
# The default verbosity level is 0. Show every command that is run and every
# package that is processed if the caller requested verbosity level greater than
# 0. Also show subcommands if the requested verbosity level is greater than 1.
@ -111,16 +116,31 @@ readonly o_flags
# must be enabled.
if [ "${RACE:-0}" -eq '0' ]
then
cgo_enabled='0'
CGO_ENABLED='0'
race_flags='--race=0'
else
cgo_enabled='1'
CGO_ENABLED='1'
race_flags='--race=1'
fi
readonly cgo_enabled race_flags
readonly CGO_ENABLED race_flags
export CGO_ENABLED
CGO_ENABLED="$cgo_enabled"
GO111MODULE='on'
export CGO_ENABLED GO111MODULE
export GO111MODULE
"$go" build --ldflags "$ldflags" "$race_flags" --trimpath "$o_flags" "$v_flags" "$x_flags"
tags_flags='--tags='
readonly tags_flags
if [ "$verbose" -gt '0' ]
then
"$go" env
fi
"$go" build\
--ldflags "$ldflags"\
"$race_flags"\
"$tags_flags"\
--trimpath\
"$o_flags"\
"$v_flags"\
"$x_flags"

15
scripts/make/go-deps.sh
View File

@ -1,5 +1,10 @@
#!/bin/sh
# This comment is used to simplify checking local copies of the script. Bump
# this number every time a significant change is made to this script.
#
# AdGuard-Project-Version: 1
verbose="${VERBOSE:-0}"
readonly verbose
@ -7,14 +12,14 @@ if [ "$verbose" -gt '1' ]
then
env
set -x
x_flags='-x'
x_flags='-x=1'
elif [ "$verbose" -gt '0' ]
then
set -x
x_flags=''
x_flags='-x=0'
else
set +x
x_flags=''
x_flags='-x=0'
fi
readonly x_flags
@ -23,6 +28,4 @@ set -e -f -u
go="${GO:-go}"
readonly go
# Don't use quotes with flag variables because we want an empty space if those
# aren't set.
"$go" mod download $x_flags
"$go" mod download "$x_flags"

153
scripts/make/go-lint.sh
View File

@ -1,8 +1,13 @@
#!/bin/sh
verbose="${VERBOSE:-0}"
# This comment is used to simplify checking local copies of the script. Bump
# this number every time a significant change is made to this script.
#
# AdGuard-Project-Version: 3
verbose="${VERBOSE:-0}"
readonly verbose
# Set verbosity.
if [ "$verbose" -gt '0' ]
then
set -x
@ -16,34 +21,12 @@ else
set -e
fi
# We don't need glob expansions and we want to see errors about unset variables.
set -f -u
# Deferred Helpers
not_found_msg='
looks like a binary not found error.
make sure you have installed the linter binaries using:
$ make go-tools
'
readonly not_found_msg
# TODO(a.garipov): Put it into a separate script and source it both here and in
# txt-lint.sh?
not_found() {
if [ "$?" -eq '127' ]
then
# Code 127 is the exit status a shell uses when a command or
# a file is not found, according to the Bash Hackers wiki.
#
# See https://wiki.bash-hackers.org/dict/terms/exit_status.
echo "$not_found_msg" 1>&2
fi
}
trap not_found EXIT
# Source the common helpers, including not_found and run_linter.
. ./scripts/make/helper.sh
@ -52,7 +35,7 @@ trap not_found EXIT
go_version="$( "${GO:-go}" version )"
readonly go_version
go_min_version='go1.19.6'
go_min_version='go1.19.7'
go_version_msg="
warning: your go version (${go_version}) is different from the recommended minimal one (${go_min_version}).
if you have the version installed, please set the GO environment variable.
@ -74,7 +57,7 @@ esac
# Simple Analyzers
# Simple analyzers
# blocklist_imports is a simple check against unwanted packages. The following
# packages are banned:
@ -91,6 +74,8 @@ esac
#
# See https://github.com/golang/go/issues/45200.
#
# * Package sort is replaced by golang.org/x/exp/slices.
#
# * Package unsafe is… unsafe.
#
# * Package golang.org/x/net/context has been moved into stdlib.
@ -101,6 +86,7 @@ blocklist_imports() {
-e '[[:space:]]"io/ioutil"$'\
-e '[[:space:]]"log"$'\
-e '[[:space:]]"reflect"$'\
-e '[[:space:]]"sort"$'\
-e '[[:space:]]"unsafe"$'\
-e '[[:space:]]"golang.org/x/net/context"$'\
-n\
@ -157,96 +143,65 @@ underscores() {
# Helpers
# exit_on_output exits with a nonzero exit code if there is anything in the
# command's combined output.
exit_on_output() (
set +e
if [ "$VERBOSE" -lt '2' ]
then
set +x
fi
cmd="$1"
shift
output="$( "$cmd" "$@" 2>&1 )"
exitcode="$?"
if [ "$exitcode" -ne '0' ]
then
echo "'$cmd' failed with code $exitcode"
fi
if [ "$output" != '' ]
then
if [ "$*" != '' ]
then
echo "combined output of linter '$cmd $*':"
else
echo "combined output of linter '$cmd':"
fi
echo "$output"
if [ "$exitcode" -eq '0' ]
then
exitcode='1'
fi
fi
return "$exitcode"
)
# Checks
exit_on_output blocklist_imports
run_linter -e blocklist_imports
exit_on_output method_const
run_linter -e method_const
exit_on_output underscores
run_linter -e underscores
exit_on_output gofumpt --extra -e -l .
run_linter -e gofumpt --extra -e -l .
# TODO(a.garipov): golint is deprecated, and seems to cause more and more
# issues with each release. Find a suitable replacement.
#
# golint --set_exit_status ./...
# TODO(a.garipov): golint is deprecated, find a suitable replacement.
"$GO" vet ./...
run_linter "$GO" vet ./...
govulncheck ./...
run_linter govulncheck ./...
# Apply more lax standards to the code we haven't properly refactored yet.
gocyclo --over 17 ./internal/querylog/
gocyclo --over 13 ./internal/dhcpd ./internal/filtering/ ./internal/home/
run_linter gocyclo --over 14 ./internal/querylog/
run_linter gocyclo --over 13\
./internal/dhcpd\
./internal/home/\
;
# Apply stricter standards to new or somewhat refactored code.
gocyclo --over 10 ./internal/aghio/ ./internal/aghnet/ ./internal/aghos/\
./internal/aghtest/ ./internal/dnsforward/ ./internal/filtering/rewrite/\
./internal/stats/ ./internal/tools/ ./internal/updater/\
./internal/version/ ./scripts/blocked-services/ ./scripts/vetted-filters/\
./main.go
# Apply the normal standards to new or somewhat refactored code.
run_linter gocyclo --over 10\
./internal/aghio/\
./internal/aghnet/\
./internal/aghos/\
./internal/aghtest/\
./internal/dnsforward/\
./internal/filtering/\
./internal/stats/\
./internal/tools/\
./internal/updater/\
./internal/version/\
./scripts/blocked-services/\
./scripts/vetted-filters/\
./main.go\
;
ineffassign ./...
run_linter ineffassign ./...
unparam ./...
run_linter unparam ./...
git ls-files -- '*.go' '*.mod' '*.sh' 'Makefile' | xargs misspell --error
git ls-files -- 'Makefile' '*.go' '*.mod' '*.sh' '*.yaml' '*.yml'\
| xargs misspell --error
looppointer ./...
run_linter looppointer ./...
nilness ./...
run_linter nilness ./...
exit_on_output shadow --strict ./...
# TODO(a.garipov): Add fieldalignment?
run_linter -e shadow --strict ./...
# TODO(a.garipov): Enable in v0.108.0.
# gosec --quiet ./...
# run_linter gosec --quiet ./...
# TODO(a.garipov): Enable --blank?
errcheck --asserts ./...
run_linter errcheck --asserts ./...
staticcheck ./...
run_linter staticcheck ./...

16
scripts/make/go-test.sh
View File

@ -1,5 +1,10 @@
#!/bin/sh
# This comment is used to simplify checking local copies of the script. Bump
# this number every time a significant change is made to this script.
#
# AdGuard-Project-Version: 1
verbose="${VERBOSE:-0}"
readonly verbose
@ -43,5 +48,12 @@ shuffle_flags='--shuffle=on'
timeout_flags="${TIMEOUT_FLAGS:---timeout=90s}"
readonly count_flags cover_flags shuffle_flags timeout_flags
"$go" test "$count_flags" "$cover_flags" "$race_flags" "$shuffle_flags" "$timeout_flags"\
"$x_flags" "$v_flags" ./...
"$go" test\
"$count_flags"\
"$cover_flags"\
"$shuffle_flags"\
"$race_flags"\
"$timeout_flags"\
"$x_flags"\
"$v_flags"\
./...

41
scripts/make/go-tools.sh
View File

@ -1,22 +1,27 @@
#!/bin/sh
# This comment is used to simplify checking local copies of the script. Bump
# this number every time a significant change is made to this script.
#
# AdGuard-Project-Version: 2
verbose="${VERBOSE:-0}"
readonly verbose
if [ "$verbose" -gt '1' ]
then
set -x
v_flags='-v'
x_flags='-x'
v_flags='-v=1'
x_flags='-x=1'
elif [ "$verbose" -gt '0' ]
then
set -x
v_flags='-v'
x_flags=''
v_flags='-v=1'
x_flags='-x=0'
else
set +x
v_flags=''
x_flags=''
v_flags='-v=0'
x_flags='-x=0'
fi
readonly v_flags x_flags
@ -27,6 +32,25 @@ readonly go
# TODO(a.garipov): Add goconst?
# Remove only the actual binaries in the bin/ directory, as developers may add
# their own scripts there. Most commonly, a script named “go” for tools that
# call the go binary and need a particular version.
rm -f\
bin/errcheck\
bin/fieldalignment\
bin/gocyclo\
bin/gofumpt\
bin/gosec\
bin/govulncheck\
bin/ineffassign\
bin/looppointer\
bin/misspell\
bin/nilness\
bin/shadow\
bin/staticcheck\
bin/unparam\
;
# Reset GOARCH and GOOS to make sure we install the tools for the native
# architecture even when we're cross-compiling the main binary, and also to
# prevent the "cannot install cross-compiled binaries when GOBIN is set" error.
@ -37,14 +61,15 @@ env\
GOWORK='off'\
"$go" install\
--modfile=./internal/tools/go.mod\
$v_flags\
$x_flags\
"$v_flags"\
"$x_flags"\
github.com/fzipp/gocyclo/cmd/gocyclo\
github.com/golangci/misspell/cmd/misspell\
github.com/gordonklaus/ineffassign\
github.com/kisielk/errcheck\
github.com/kyoh86/looppointer/cmd/looppointer\
github.com/securego/gosec/v2/cmd/gosec\
golang.org/x/tools/go/analysis/passes/fieldalignment/cmd/fieldalignment\
golang.org/x/tools/go/analysis/passes/nilness/cmd/nilness\
golang.org/x/tools/go/analysis/passes/shadow/cmd/shadow\
golang.org/x/vuln/cmd/govulncheck\

84
scripts/make/helper.sh Normal file
View File

@ -0,0 +1,84 @@
#!/bin/sh
# Common script helpers
#
# This file contains common script helpers. It should be sourced in scripts
# right after the initial environment processing.
# This comment is used to simplify checking local copies of the script. Bump
# this number every time a remarkable change is made to this script.
#
# AdGuard-Project-Version: 2
# Deferred helpers
not_found_msg='
looks like a binary not found error.
make sure you have installed the linter binaries using:
$ make go-tools
'
readonly not_found_msg
not_found() {
if [ "$?" -eq '127' ]
then
# Code 127 is the exit status a shell uses when a command or a file is
# not found, according to the Bash Hackers wiki.
#
# See https://wiki.bash-hackers.org/dict/terms/exit_status.
echo "$not_found_msg" 1>&2
fi
}
trap not_found EXIT
# Helpers
# run_linter runs the given linter with two additions:
#
# 1. If the first argument is "-e", run_linter exits with a nonzero exit code
# if there is anything in the command's combined output.
#
# 2. In any case, run_linter adds the program's name to its combined output.
run_linter() (
set +e
if [ "$VERBOSE" -lt '2' ]
then
set +x
fi
cmd="${1:?run_linter: provide a command}"
shift
exit_on_output='0'
if [ "$cmd" = '-e' ]
then
exit_on_output='1'
cmd="${1:?run_linter: provide a command}"
shift
fi
readonly cmd
output="$( "$cmd" "$@" )"
exitcode="$?"
readonly output
if [ "$output" != '' ]
then
echo "$output" | sed -e "s/^/${cmd}: /"
if [ "$exitcode" -eq '0' ] && [ "$exit_on_output" -eq '1' ]
then
exitcode='1'
fi
fi
return "$exitcode"
)

36
scripts/make/txt-lint.sh
View File

@ -1,9 +1,13 @@
#!/bin/sh
# This comment is used to simplify checking local copies of the script. Bump
# this number every time a remarkable change is made to this script.
#
# AdGuard-Project-Version: 2
verbose="${VERBOSE:-0}"
readonly verbose
# Set verbosity.
if [ "$verbose" -gt '0' ]
then
set -x
@ -20,31 +24,9 @@ fi
# We don't need glob expansions and we want to see errors about unset variables.
set -f -u
# Deferred Helpers
not_found_msg='
looks like a binary not found error.
make sure you have installed the linter binaries using:
$ make go-tools
'
readonly not_found_msg
# TODO(a.garipov): Put it into a separate script and source it both here and in
# go-lint.sh?
not_found() {
if [ "$?" -eq '127' ]
then
# Code 127 is the exit status a shell uses when a command or
# a file is not found, according to the Bash Hackers wiki.
#
# See https://wiki.bash-hackers.org/dict/terms/exit_status.
echo "$not_found_msg" 1>&2
fi
}
trap not_found EXIT
# Source the common helpers, including not_found.
. ./scripts/make/helper.sh
git ls-files -- '*.md' '*.yaml' '*.yml' 'client/src/__locales/en.json'\
| xargs misspell --error
| xargs misspell --error\
| sed -e 's/^/misspell: /'