English
Storage, sync, and lists
Where things are stored, sync between browsers, and subscribing to lists. Part of Experiments and decisions. Newest notes go at the top of each section.
Room in sync, and what a file can restore (2026-11-03)
- Source: the whole-codebase review of 2026-11-03. Each fix has a unit test that failed before it.
- Room in sync: the personal list only moved to this device when its own save failed, so a list that nearly filled sync made the next save of settings, tag choices, or subscriptions fail instead, and some callers (the keyboard shortcut) dropped that failure unseen. A save now counts what the other sync items take and keeps 8 KB free (
SYNC_HEADROOM_BYTES, one item's limit) before writing; otherwise the list stays on this device, as when it's too big. The next save that fits moves it back. That's about 8% fewer sites in sync than before. - What a file can restore: subscriptions from a backup or sync file only needed a string
idandurl, so a file could add a list Settings would refuse (anhttp:,file:, orjavascript:address) or mark any list as built in. They now need an https address,builtinonly stays onbuiltin:ids, and a name is cut to one short line, as a subscribe link's is. Fields a newer version adds are kept, as with settings. Tag choices under ids no tag can have are dropped, which also stops a__proto__key replacing the object's prototype. - Smaller: Firefox's consent check read whether
permissions.getAll()had adata_collectionkey, which Firefox gives whether or not anything was consented to; it now looks forbrowsingActivityin it. A Brave Goggle link whosegoggles_idisn't a web address now stays the Goggle link, instead of turning into ajavascript:link in the subscribe offer, where the page's security policy blocked it anyway.
Review fixes: allow, hosts files, user spaces, and the first install (2026-11-03)
- Source: a whole-codebase review. Each bug was reproduced with a unit test that failed before the fix.
- A list's
allowdid nothing: subscribed lists only read discard and boost, so a uBlacklist ruleset's@unblock rule didn't beat its own block, and anallowdidn't override other lists asdocs/list-format.mdsays. A list's allow now keeps the result at normal against every list's hide, lens, and moves. Your own ranking and tag choices still come first. A lens's "leaves it out" reason is dropped from Why once an allow holds (tests/matcher.test.ts). - Hosts files didn't load: format detection didn't count
0.0.0.0 hostlines as domains, and any#comment made a list a uBlacklist ruleset. So a hosts file, or a domain list with a comment at the top, failed with "no usable rules".#comments and---no longer count as uBlacklist on their own: they decide only when nothing else does, which keeps a ruleset of regular expressions alone working (tests/listformat.test.ts). /user/spaces:reddit.com/user/…andyoutube.com/user/…were never offered in the result menu, because the list of the sites' own pages (NOT_SPACES) hasuserin it. That list now only applies to a one-part space like GitHub's/users(tests/domain.test.ts).- First install wrote to sync:
migrateLegacysaved the default personal list on every new install. On a second computer, sync can bring the real list afteronInstalled, and that write could compete with it. As with subscriptions, no stored list now means the default; only a legacy block list is migrated (tests/storage.test.ts).
Reading a backup or sync file's settings (2026-10-24)
- Before:
readBackup(utils/backup.ts) checked each tag choice but took settings on trust. A file with"engines": null(hand-edited, damaged, or written by a faulty server) stopped Anubis on every search page, since search pages readengines[id]. - Now: each setting is kept only if it's a value Settings could have saved, else the default (
READ_SETTING, typed over every key ofSettings, so a new setting fails the type-check until it has a reader). - Rejected: keeping only the settings this version knows. Today a setting added by a newer version on another browser survives a sync through an older one. Dropping unknown keys would make the older browser's merge take it away again, so they stay as they are.
Suggest changes as a form in Settings (2026-10-20)
- Labels on screen (2026-10-26): the site field had no visible label and
go.devas its placeholder. Beside "Tag it “Official docs”", the row read as a suggestion already filled in, and people didn't know what to type. The site and the change are now labelled above their fields, like Why, and the placeholder isexample.com, which no list would name. - Before: "suggest changes" was a lower-case link in a list's small print, and it opened the tracker's issues page, so people had to find the template and write the instruction themselves. Beside it, the side column's external-link icon opened the list's raw file, which few people need.
- Now: a Suggest changes button takes the file link's place among the list's buttons. It opens a short form under the list: the site, what should change (one of the list's tags, Take it off the list, or Something else), and why. The form opens a filled-in issue in a new tab, with the
$site=…,tag=…instruction to paste in (changeRequestUrlinutils/subscriptions.ts), laid out like the "Suggest a site" template. - Still opt-in: only lists with
! issues:get the button, as before. A list that never asked for suggestions still doesn't get them. - First try at 320px: the button in the side column squeezed the description into a few characters per line, and the form under the description sat beneath the buttons. Below 760px, the buttons now go on their own line under the description, as a tag row's controls do, and the form is a row of its own across the list (
e2e/run.mjs suggest-changes).
Review findings on list parsing and subscriptions (2026-10-10)
- Source: a whole-codebase review, split across six draft pull requests (#129 to #134, closed unmerged). Only the list-parsing one produced findings worth keeping; the rest held automated code-health comments and nothing else. The three below were checked on 2026-10-24: two were real and are fixed, and one was left as it is.
- Fixed (2026-10-24):
nestedRepeat(utils/listformat.ts) only found a repeated group that contains a quantifier, so/(a|aa)+$/passed the slow-pattern check from "Slow patterns" below, and took 43 ms on 30 a's, about 1.6 times as long with each one more. A repeated group with choices now passes only if they're plain words none of which starts another ((?:com|org)+), since then at most one can match in any place. Every bundled and directory list still parses. - Confirmed, left as it is (2026-10-24): an exact-host uBlacklist rule imported into the personal list (
importIntoPersonal,utils/importers.ts) becomes a site entry, which also matches subdomains, so the import widens the rule. The only faithful import is a pattern line (|https://example.com/$discard), which Settings → Sites can't show or change, so the import keeps the site entry. Subscribing to the ruleset keeps the exact rule. - Fixed (2026-10-24):
refreshStale(utils/subscriptions.ts) had no error handling per subscription, so one list whose copy couldn't be saved (local storage full) stopped the lists after it from updating. It now goes on to the next. - Already known:
migrateDefaultListsswitches every default list back on, not only new ones. That was the project owner's choice (see "Existing installs"). - Smaller: the import counts a site once per rule that covers it; the regex-escaping pattern is written out twice in
utils/listformat.ts; turning a list off drops its compiled cache, so turning it back on parses it again. - Code-health scores: CodeScene flagged long, branching functions across the content script, options page, and list parser (
main.passat complexity 61,parseListat 36,importIntoPersonalat 20). Not acted on; the project has no complexity threshold.
Whose tag is it: lists' tags on your sites (2026-10-08)
- Raised by the owner: people see list tags beside their own in the tag picker, add them to sites, and expect their additions to be overwritten or go out of date. They never were overwritten: a list's update can't touch the personal list, and renaming a list's tag is saved as a tag choice. But they did go stale: tagging a site with a list's tag stored only the id, so when the list dropped or renamed the tag, or you unsubscribed, your sites kept a bare id with a made-up colour.
- Considered and rejected: tag repositories. Subscribable sets of tag definitions that lists would depend on, so a list reuses a tag you already have rather than duplicating it. Tags are already shared by id, which does that de-duplication, and a list of only
! tag:lines already works as such a set. Dependencies would add fetch order, versions, a list that breaks when what it depends on goes, and a second thing to trust, and still wouldn't keep your own tagging from going stale. - Shipped: tagging a site with a list's tag copies the list's
! tag:line into the personal list (tagSite), and untagging the last of your sites takes the copy out again. A subscribed list's definition now wins over the personal list's (collectTags); before, the personal list's won. The copy only names the tag once no list defines it. The background script fills in copies for sites tagged before, after an install or update (copyListTags). Settings → Tags groups by who defines the tag: Your tags holds tags no list gives sites, and a list's tag you use stays under Tags from lists ("Marks 3 sites of yours and 908 from …"). Before, a tag you'd defined that a list also used sat under Your tags with "Also used by". The popup and the result menu split the picker into Your tags and Tags from lists. - Usability check (ux-heuristics): first pass 6/10. In the result menu a tag a list gave the site looked exactly like one you'd pressed, though only yours responds, and which list gave it was only in a tooltip; "From lists" over buttons you can press suggested editing the list; the picker and Settings used different names. Fixed: the menu marks your tags bold and underlined as the popup does, a list's tag is plain with "from" and the list's name after it, the lists' group says "Adding one changes only your list.", both use Settings' names, and the Settings hint lost a third of its words.
list-format.mdasks list authors to keep ids and reuse common ones, and lists the ids Anubis's lists use. - Changed for some: a tag you made whose id a list also defines now shows the list's name and colour. Rename it in Settings → Tags to keep yours; that's saved as a tag choice, which wins over both.
FOSS from awesome-selfhosted (2026-10-08)
- Built: a weekly source,
lists/sources/self-hosted-foss.anubis, on by default. awesome-selfhosted-data keeps one YAML file per program insoftware/; each program'swebsite_urlbecomes a rule labelling it FOSS, the FOSS tools list's tag, with no boost. 1,006 rules from 1,353 entries. The files are flat enough to read without a YAML library (yamlFields). - Corrected: the earlier note counted 1,018 programs with their own domain, but that included proprietary ones. The data lists them in the same folder, marked
⊘ Proprietary, and others are under non-commercial or source-available licences (91 entries in all). Only programs whose every licence is in itslicenses.yml, its list of free licences, are kept, so a licence new to the data stays out until awesome-selfhosted calls it free. - Paths: a program's page on a company's site keeps its whole path (
/software/confluence^), since the first segment alone (/software^) would have labelled all of Atlassian's products; a version or language ends it (0xerr0r.github.io/blocky/latest/→/blocky^). Paths now end in^, so/docs^matches with or without the trailing slash; DevDocs's rules changed the same way. A first try also read#in DevDocs's#{release}as a fragment and made/v^rules fordate-fns.organdsequelize.org;tests/sources.test.tscovers it now. - Left out: 252 websites on GitHub, GitLab, Codeberg, and other shared hosts.
- Licence: CC BY-SA 3.0; the list carries it, credits the commit, and says it is shared under the same licence.
- Overlap: none with the FOSS tools list's two sites, and one later would do no harm: neither list moves results, and a tag shows once.
Official docs from DevDocs (2026-10-08)
- Changed the same day, by the owner's choice: the DevDocs list no longer skips the bundled list's sites. The 36 bundled entries it covers were removed from
official-docs.anubisinstead (24 remain), as the rule for sources says. Cost: an install that had chosen its own lists before the DevDocs list existed isn't subscribed to it, and loses those 36 sites until it subscribes.tests/sources.test.tsnow fails if the bundled list keeps a site DevDocs has, so a weekly update that adds one says to remove it there. - e2e: the harness now answers this repository's list addresses from the checkout. Before, a run downloaded them from
main: the "Official docs" chip on MDN results came frommain's copy, so the check passed or failed with whatevermainheld. Playwright 1.63 routed the background script's downloads withoutPW_EXPERIMENTAL_SERVICE_WORKER_NETWORK_EVENTS; the cached copies were the checkout's. - Built: a weekly source,
lists/sources/devdocs.anubis, on by default. DevDocs's scrapers (lib/docs/scrapers/*.rbinfreeCodeCamp/devdocs) each set abase_url, the address their docs are read from; each becomes a rule tagging Official docs withboost=1, the bundled list's tag and nudge.devdocs.io/docs.jsonwould be simpler, but it can't be reached from cloud sessions to test, and it doesn't carrybase_url, only each project's homepage, which is often not its docs. - Reading an address: a docs host (
docs.,developer.,api., and similar, or*.readthedocs.io) is tagged whole; so is a host whose path starts with a version or language (/en/,/3.13/,#{self.release}). Otherwise only the first path segment is (/docs^$site=prettier.io,/kb^$site=mariadb.com). A first try read any two-letter segment as a language, which made all ofmariadb.comdocs. Rules a parent site's rule covers are dropped, 38 of them, mostly versioned hosts such asv10.angular.ioand18.react.dev. Left out: 23 addresses on GitHub, onlocalhost, or with a host worked out at run time. - Overlap with the bundled list: 36 of
official-docs.anubis's 60 rules are sites DevDocs has, and boosts from two lists add up. The rule for sources is that their data replaces hand-made entries, butmigrateDefaultListssubscribes existing installs to default lists only once, so an install that had chosen its own lists would have lost those sites without gaining DevDocs's list. Instead the DevDocs list leaves out any site the bundled list has, or a part or parent of one (docs.djangoproject.comfordjangoproject.com). That leaves 176 new rules, andtests/sources.test.tschecks the two never overlap. - Licence: DevDocs's code is MPL-2.0; the list carries it and credits the commit. Reports go to DevDocs, as for the other sources.
- Not checked: the job's first run on GitHub, and how many DevDocs sites turn up in real searches.
More lists, and sites on shared hosts (2026-10-08)
A look at other projects whose data or tools could help Anubis. Counts are from each project's default branch on 2026-10-08, and each list was read with parseList without errors.
- Bug found and fixed: "how much of the site" knew country endings (
co.uk) but not hosts that give each person a site, sosomeone.github.ioofferedgithub.ioand chose it by default; one tap hid every GitHub Pages site.domainChoices(utils/domain.ts) now stops at the person's site for 26 such hosts from the Public Suffix List's private section, and still offers a wider domain that already has a ranking of yours so it can be undone (tests/domain.test.ts).substack.com,wordpress.com,tumblr.com,medium.com, anditch.ioaren't in the Public Suffix List, so they stay one site each. - Tried and not used:
tldts, a library built on the whole Public Suffix List, is 130 KB minified and 47 KB gzipped, more than the whole 92 KB content script. Its ICANN-only build (86 KB) still saysgithub.io. The list itself has 6,949 ICANN rules and 3,387 private ones. If the short list misses too much, the next step is a generated table read from storage, not the library. - Added to the directory (not on by default): alvi-se's AI content farms (GPL-3.0, 2,741 rules, 5 sites shared with the HUGE AI Blocklist; added by hand, many Italian), agsimmons's AI Content Blocklist (CC0, 59 rules), popcar2's Bad Website Blocklist (CC0, 398 rules, 18 shared with HUGE; every site has an issue saying why), and quenhus's dev-filter (Unlicense, 710 rules). Its Stack Overflow part has 535 sites against the 139 in arosh's list, which share 79. It was last changed on 2025-04-24, but copy sites don't age as fast as AI farms.
- Left out: FMHY's unsafe-sites list (GPL-3.0, 283 sites; it also has a reason per site in
filterlists-reasons.json), since its sites are mostly clones from piracy communities, far from most searches; its "Plus" version also flags well-known brands. wdmpa's content-farm list and franga2000's fake AliExpress sites (no licence, last changed 2024-01 and 2023-07). OnlyHuman (no licence file). StopModReposts (no longer maintained). - Licences: GPL-3.0 lists are only linked from the directory, not copied into
lists/sources/, so no licence question arises. Copying one would need its licence carried in the list, as UT1's would. - Worth a weekly-job source (in
ROADMAP.md): DevDocs's scrapers each name a project's homepage (links.home) and docs address (base_url); 212 homepage hosts, of whichofficial-docs.anubishas 22. awesome-selfhosted-data (CC BY-SA 3.0) has 1,353 entries withwebsite_url,source_code_url, andlicenses; 1,018 are on their own domains, and the other 311 are on GitHub, GitLab, Codeberg, and similar, which the shared-host fix above makes taggable one project at a time. - Checked and not worth it: Stack Exchange's
/sitesAPI, sincediscussions.anubisalready hasstackexchange.comand the sites on their own domains. ClearURLs's redirect rules, sinceresolveUrl(entrypoints/content/results.ts) already unwraps the redirects its engines use: Bing's, Yahoo's, DuckDuckGo's, and Google's.@ghostery/adblocker, which targets network filters, not search results. Terms of Service; Didn't Read, with no official bulk download. - Already done before this look: Google's
udm=14(the "Always open the Web tab" switch), Dependabot, and store releases throughwxt submit.
Reading long lists on search pages (2026-10-08)
- Measured first: a made-up 41,000-rule Anubis list (
$site=…,tag=…, the shape the weekly lists have) took 140 to 260 ms to parse and compile in Node. Parsing was nearly all of it: aRuleobject, its options, and a URL for every site. - Tried, and kept: subscribed lists are read with their plain
$site=lines only filed under their site (parseList(text, true),deferred); a line is parsed when a result from its site turns up (siteRulesinutils/matcher.ts), and Settings, which goes through whole lists, parses them all (allSites). The same list now loads in 60 to 75 ms, and weighing 30 results stays under 1 ms. What makes a line plain is strict (PLAIN_SITE_LINE): onesite=, nothing a comment could hide, and a domain a URL would read back unchanged (fastDomain); anything else is read in full as before. A line using a tag the list never defines is read in full too, since the made-up tag comes from the rules that use it. - Checked:
tests/deferred.test.tscompares a deferred read with a full one, site by site and whole, for every bundled and generated list and a list of awkward lines. BreakingfastDomain's check for numeric addresses, or the undefined-tag check, fails it. - Not tried yet: compiling in the background script, or keeping a compiled form. Messages and
storage.localcan't carry aRegExpor aMap, so either needs a form of its own. What's left per page is reading the list texts from storage and the filing pass itself.
Settings and tag choices in sync (2026-10-08)
Checked that settings and tag choices follow the user both ways. Through a WebDAV server, two browsers that each change different settings (an engine, a clean-up kind, how hidden results look) and different tag choices end up with all of them, and a tag set back to its default stays set back (tests/webdav.test.ts). Through browser sync, a tag's ranking and the on/off switch arriving from another computer change an open search page straight away (the e2e sync part). Nothing needed fixing. Known limit: all settings are one sync item, so when two computers change different settings before either syncs, browser sync keeps only the later change. WebDAV merges them key by key. Not checkable: a muted tag's chip, since the chips' shadow root is closed to the harness.
Lists made from other projects (2026-10-08)
Asked for: use outside datasets (UT1, Block List Project, Indie Wiki Buddy, Kagi Small Web, the HUGE AI Blocklist, Wikipedia's perennial sources, the Iffy Index, Wikidata) to tag sites, on by default but easy to turn off by source and by tag, credited, and kept current.
- Existing installs (2026-10-08): the plan was to offer new default lists once in Settings rather than subscribe anyone silently. The project owner chose to skip the offer, since next to no one had installed Anubis yet: on update,
migrateDefaultLists(utils/subscriptions.ts) subscribes an install that has saved its subscriptions to every default list and switches each on, once (sync:defaultListsAdded). Its other lists stay. An install that never changed its lists already follows the defaults. - Built: a weekly job (
.github/workflows/sources.yml, modelled on the engine sync) converts each source into a list inlists/sources/and opens a pull request with the diff. Browsers download the lists from GitHub like any subscription, so the data stays current between releases. Turning off by source is unsubscribing; by tag, Settings → Tags. A list that would lose more than a fifth of its rules isn't rewritten and the run fails, since that's likelier an upstream mistake or format change than a clean-up. Rules are sorted and the file only rewritten when rules change, so a new upstream commit alone makes no pull request. - HUGE AI Blocklist (CC0): its main list becomes AI-generated, its "nuclear" list (sites mixing authentic and AI work) Some AI content, and its "AI content farms/spam" section also AI slop, the tag FOSS tools already defines (same definition, so they merge). The data says how much of a site is AI, not how good it is, so "AI slop" otherwise stays a judgement people or lists make by hand. A site in both lists keeps AI-generated. A trailing
/in a path rule becomes^, so/@name/*also matches the channel page/@name; uBlacklist's own rule wouldn't. Left out: 92 regular expressions (Pinterest accounts across its many domains) and one host with a wildcard. The list only labels. - Indie Wiki Buddy (MIT, licence text carried in the header): 846 entries. Fandom, Fextralife, and Neoseeker wikis get Independent wiki elsewhere and
downrank=3, independent wikis Independent wiki andboost=3. Each rule's comment names the other side ("Minecraft Wiki, the official wiki, is at minecraft.wiki."). The menu shows a matched rule without its comment, so the destination isn't named on the page: the owner chose to use the data only, not to rebuild Indie Wiki Buddy's redirects and banners. Known gap: a whole-host rule ($site=x.fandom.com) also covers that host's other languages (/de/), which may not have moved. - No
! issues:on these lists. It's what offers "Suggest it to…", which sends Anubis rules; the sources never asked for those. "Wrong? Report it to…" still reaches them through! homepage:, the same as for other third-party lists, and the list's name carries the source's ("AI content (HUGE AI Blocklist)") so a report makes sense there. - Default lists that aren't bundled:
defaultSubscriptionsmarked every default as built-in with abuiltin:id, so a default list without a bundled copy would have read as "the copy bundled with Anubis". It now uses the address's id for those, and the welcome page falls back to the directory's description until the first download (which runs on install). - Measured: Kagi Small Web's 41,657 feeds give 41,137 hosts; as a 1.7 MB list, parsing and compiling took 170 to 230 ms in Node, which a search page pays on every load. That's why Kagi and UT1's big categories wait for lists compiled once (see
ROADMAP.md). - Licences checked: HUGE AI Blocklist CC0 1.0, Indie Wiki Buddy MIT, Kagi Small Web MIT, Block List Project Unlicense, UT1 CC BY-SA 4.0 (via its GitHub mirror's README). The UT1 site,
query.wikidata.org, andiffy.newscouldn't be reached from the cloud session. - Decided against: shipping all of Wikidata's official websites (well over a million, and many on shared hosts or dead), UT1's broad categories as chips on every result, credibility ratings by default, and the Block List Project. Details in
ROADMAP.md. - Not yet checked: the workflow's first run on GitHub, and how Settings shows a default list before its first download.
FOSS tools default list (2026-10-04)
- Added: a separate bundled, default list labels LibreSpeed and cobalt.tools with the neutral FOSS tag, and makes AI slop available for manual tagging. Kept it separate from Reference rather than broadening that list's meaning; the rules do not change ranking or hide results.
- Compatibility: default subscriptions remain a fallback only when none are stored. Existing users are not automatically resubscribed and can add FOSS tools from Settings → Lists → More lists.
- Not checked: the live sites' source and licence pages. The list reflects the requested classification and does not claim anything about each hosted service's privacy practices.
Passphrase changes that race or lose their answer (2026-09-29)
- Found: a review of the passphrase change turned up three gaps. It ran in the Settings page, so the lock that keeps syncs from overlapping (which only works within one script) didn't hold back the background script's syncs; a sync's last attempt writes without
If-Matchand could put back a file under the old passphrase. When the server saved the new file but the answer was lost, the old passphrase stayed saved and the next sync failed. And a server disconnected during the change was connected again when it finished. - Fixed: Settings asks the background script to change the passphrase (
change-passphrase), where it queues behind syncs. When the save gets no answer or a server error, Anubis reads the file back and keeps the new passphrase if it opens the file; when the server can't be reached, it keeps the old one with anunconfirmedstatus telling the user what to do. The new passphrase is saved only while the same connection is still there. - Checked: unit tests for a lost answer, the server going away after the save, and a disconnect during the change each failed before the fix. The
webdave2e part still changes the passphrase through Settings.
WebDAV passphrase changes and recovery (2026-09-29)
- Built: Settings can change an established encryption passphrase. Anubis reads and decrypts the current encrypted file, encrypts the same data with the new passphrase, and replaces it only with a strong
If-MatchETag. A missing/weak ETag, concurrent edit, wrong current key, or failed write leaves the saved passphrase unchanged. Other browsers must get the new passphrase before their next sync. - Rejected: running a normal merge sync before rotating the key. That could write changes using the old passphrase before a later rotation failure; the rotation now changes only the encryption on the exact server copy it read.
- Test correction: the encrypted envelope is format 2, but its decrypted backup payload remains format 1. The first browser assertion checked the payload as though it were the envelope; it now checks both layers separately.
- Recovery: added a specific concurrent-change status with a retry instruction. Existing status messages already explain how to recover from denied permission, login, folder, network, unreadable-file, wrong-key, and plaintext-downgrade errors; the sync guide now explains updating other browsers after rotation.
- Ranking explanations: the result menu already identified each list and its action, so no second explanation system was needed. It now also shows the exact matched list rule and line number; the matcher already retained both for report links.
- Checked: unit tests cover successful rotation, wrong current passphrase, a concurrent ETag change, server rejection, and matcher reasons. The browser check now requires a matched rule in the result menu. The Settings and result-menu documentation screenshots are regenerated from the mocks.
End-to-end WebDAV encryption (2026-09-29)
- Built: new connections encrypt the sync file by default with AES-256-GCM; PBKDF2-SHA-256 derives the key from a user-chosen passphrase. A random salt and nonce are stored with the ciphertext. The passphrase stays in that browser's local extension storage, never browser sync or the WebDAV request.
- Why this design: Web Crypto provides AES-GCM in both target browsers, so no dependency or downloaded code is needed. GCM provides confidentiality and detects modified ciphertext in one operation; a fresh random nonce is generated for every encryption, and a random salt makes each derived key unique. PBKDF2-SHA-256 is available in the same API and slows offline guessing of the passphrase. This is not a substitute for a strong passphrase or protection for a compromised browser profile.
- Migration and safeguards: existing plaintext connections can opt in without losing their merge base; their next successful write replaces the file with ciphertext. A browser without the passphrase, or with the wrong one, cannot overwrite an encrypted file. Once encryption is established, Anubis rejects a plaintext downgrade. An encrypted file can still be deleted or replayed by its server; this is not an availability or rollback defence.
- Recovery trade-off: losing the passphrase makes the server copy unreadable; data still present in each browser can be exported separately. The passphrase is available to Anubis in local extension storage so background syncing can work, so it does not protect a compromised browser profile. Plain backup exports are not encrypted by WebDAV encryption.
- Checked: WebDAV unit coverage verifies ciphertext has no readable list content or passphrase, cross-browser decryption, plaintext migration, missing/wrong passphrase non-overwrite, and downgrade rejection. Updated the connect screen, privacy policy, sync guide, store notes, and generated screenshots.
- Layout iteration: the first generated screenshot showed the long passphrase hint expanding the form's
max-contentlabel column and pushing inputs offscreen. Grouping the passphrase fields into their own grid fixed it; the refreshed image shows the complete form.
Reporting mistakes to lists
Asked for: a way to report a wrong result, like SponsorBlock's feedback loop but without a backend. Lists already live in Git repositories, so a report can be an issue there.
- Shipped: under Why in the result menu, "Wrong? Report it to list" for each list that weighed the result. It opens a pre-filled issue: the result's address without its query or fragment (which can carry session details), what the list does with it, the rules that matched with their line numbers, and a "What should change:" prompt. The issue is in English whatever the interface language, since it's for the list's maintainers. Line numbers come from the copy Anubis has, which can be older than the repository's, so the rules' text goes in too.
- Where it goes: the list's
! issues:, else the repository its! homepage:or its own address points into (raw.githubusercontent.com/<owner>/<repo>/…, GitLab's…/-/raw/…, Codeberg's…/raw/…). Gists and other hosts have no tracker to derive, so they get no link. None of the seven third-party lists in the directory sets! issues:or! homepage:(checked 2026-09-29), so without deriving it the feature would reach only Anubis's own lists. - Suggestions stay opt-in. "Suggest it to…" still needs
! issues:: a report of a mistake is an ordinary bug report for any repository, but unsolicited additions aren't. Suggestions now skip lists that already weigh the result, since for those the report is the right link; before, such a list could be offered a suggestion. - Considered: a "Report" link on each line under Why. Rejected because one list can give several reasons for one result; one link per list, named, also says where the report goes.
- Fixed later: a suggestion's example result dropped only the query (
split('?')), so an address with just a#fragmentkept it. Both issues now build the address the same way (plainAddressinutils/subscriptions.ts), without query or fragment. - The report and suggestion lines are the first of the menu's text in
messages.json.tListinutils/i18n.tsputs the links in place of$1, joined withIntl.ListFormat("A or B"). - Unverified: what GitHub does when a repository has issues turned off (it should show the repository), or has issue forms with blank issues turned off (it may show the template chooser and drop the pre-filled text).
Storage
Syncing between browsers (2026-09-29): optional, through a WebDAV server the user connects in Settings → Sync; browser sync stays the default and runs either way. (Firefox for Android, which doesn't sync add-on data, gets sync this way too.)
- Why WebDAV. uBlacklist offers Google Drive, Dropbox, OneDrive, and WebDAV. Each of the first three needs the project to register an app with the service and keep it, the
identitypermission for the sign-in window, and for Google a review; together with its sync loop that's about 1,900 lines, not counting settings screens and tests. WebDAV needs none of that and no new permission: connecting asks for the one host through the optionalhttps://*/*that list subscriptions already use. The cost is on the user, who needs an account with a service that offers it (Koofr and InfiniCLOUD have free plans, Nextcloud works), and usually an app password. Dropbox is the lighter follow-up if people ask for something they already have. - One file, the backup format.
anubis-sync.jsonholds what Export backup saves, so Restore backup reads it too. It's read and written whole: small, and one file means one ETag to guard. - Merging, not newest-wins. uBlacklist keeps the newer copy of each part. Here each sync merges three ways against what both sides had at the last one (
local:webdavBase), so a site hidden in Firefox and one tagged in Chrome both survive, and a ranking changed on one side combines with a tag added on the other. The personal list merges line by line, keyed by what a line is about (a site, a tag definition, a! name:header, or the line itself), so moved lines still line up; lines only the other side has go after the line they follow there. 6,000 new sites merge in well under a second. When both sides changed the same thing, the browser syncing now wins. - The first sync uses what a fresh install has as its base, with the server winning: an untouched browser takes everything (without bringing back the default
fandom.comthe other browser removed), and one that already had its own sites keeps them. A test that took the local copy as the base still passed the first case; the second, a browser with its own sites, is what tells them apart. - Saving in between. The write carries
If-Matchwith the ETag just read; a server that got another browser's save in between refuses it (412) and the sync merges again, up to three times, the last without the condition. A weak ETag (W/…) never matches underIf-Match, so it isn't sent. A change made here while the server answered starts the round again too. - When. A few seconds after a change here (skipped when everything still matches the base, so the sync's own writes don't start another), on startup and after an update, and when a search page asks, at most every 5 minutes. That's the moment changes matter, and it needs no
alarmspermission. - Consent. Firefox counts sending anything off the browser as data transmission. The list is a set of sites,
browsingActivityin its terms, declared as optional and requested with the host when the user presses Connect. Settings might also count astechnicalAndInteraction; not declared, since that one shows a switch at install for a feature most people won't use. The login stays inlocal:storage, never in sync. - Verified: merge and sync logic in unit tests with two browsers and a fake server (
tests/webdav.test.ts), including another browser's save in between and a file Anubis can't read (left alone). In Chromium (node e2e/run.mjs webdav), against a mock server with no CORS headers: Connect, the file created withIf-None-Match: *, a site added on the server arriving on Sync now, and a settings change here reaching the server a few seconds later. - Consent regression (2026-09-29): the first permission request used whether consent was already granted to decide whether to ask for it, so a fresh Firefox 140 connection omitted the data prompt. Background sync also treated a missing consent grant as an unsupported browser and fell back to checking only the host; if checking
data_collectionthrew, it made the same fallback. Either path could send the list without consent. Requests now requirebrowsingActivitybefore any transfer. Anubis now requires Firefox 142 or later in its manifest; tests ensure a missing permission or failed consent check cannot transfer data. - Still unverified: real servers (Koofr, InfiniCLOUD, Nextcloud, where signed-in cookies are one known trap, so requests send none) and Firefox's actual consent prompt in a real browser.
- Found on the way: a list from a host other than GitHub, arriving through sync or a backup, couldn't download in the other browser, and Update now didn't ask for the host. It now asks, from the click.
- Why WebDAV. uBlacklist offers Google Drive, Dropbox, OneDrive, and WebDAV. Each of the first three needs the project to register an app with the service and keep it, the
Sync (2026-09-29): browser sync already carried everything but downloaded lists, in Firefox and Chrome alike, with nothing to set up in Anubis. Three weak points, now handled:
- Size. Sync holds about 100 KB, so a list of about 3,300 sites was the most that synced, and importing Brave's Hacker News Goggle (6,238 sites, 174 KB as text) left the list on one computer. The list is now compressed before it's split into chunks (
deflate-rawthrough the browser'sCompressionStream, then base64, as uBlacklist does with LZString): that Goggle takes 57 KB, about 3× less, so some 10,000 sites fit. Where compression isn't available (before Chrome 103 or Firefox 113) the list is saved as plain text, as before. Settings → Sync (then under Share and back up) shows the room used. - Lists read short. Chrome syncs each chunk on its own, so another computer could read the new count of chunks while some still held old text, or chunks from two computers mixed, and an edit made then saved the broken list. The meta item now carries a checksum of the whole text; a list that doesn't match it isn't used, and this computer's last good copy (
local:personalCopy) stands in. An edit waits 1.5 seconds for the rest to arrive before saving on top of the copy. Watching the chunks as well as the meta item re-reads the list when late chunks land. The copy also saves unpacking the chunks on every search page. Checked in Chromium (node e2e/run.mjs sync): a change from the result menu is saved compressed with its checksum, and a count of chunks arriving before the chunks keeps the old list on the page until they land. - Nothing said how to turn it on. Firefox only syncs add-ons' data with Add-ons ticked in its sync settings, about every 10 minutes, and Firefox for Android doesn't sync it at all. Settings and the guide's new page (Syncing between computers) now say so.
- Still unverified: syncing between two real computers (the sandbox had one browser and no account), and
CompressionStreamin Firefox content scripts, where streams have had trouble with Xray wrappers in the past. If it fails there, an edit from a search page saves plain text, and the background script, which reads the list whenever it arrives from sync, keeps the copy that search pages read up to date. - Not done here: syncing between Firefox and Chrome, which browser sync can't do. See the next entry.
- Size. Sync holds about 100 KB, so a list of about 3,300 sites was the most that synced, and importing Brave's Hacker News Goggle (6,238 sites, 174 KB as text) left the list on one computer. The list is now compressed before it's split into chunks (
The personal list is text in the list format (so it can be published as-is), stored in
storage.syncsplit into chunks of at most 7 KB, since sync allows 8 KB per item and about 100 KB in total. If it outgrows sync it falls back to local storage and settings say so.Bug found: the first chunking split on newlines and re-joined with
\n, which would have corrupted a single line longer than a chunk. Chunks are now cut by encoded size and joined with nothing; a test round-trips 3,000 lines with non-ASCII text.Default subscriptions are not written to storage on install. On a second device, sync data can arrive after
onInstalled, and writing defaults would overwrite the user's synced subscriptions. Instead, "no stored subscriptions" means "the defaults".The old
sync:blockedSitesarray is migrated into the personal list on install and on first read.Overlapping writes (2026-09-29): every change reads the stored value, changes it and writes it back. Only personal list edits were queued, so two quick changes on one page could read the same value and one would be lost: two engine switches, two clean-up switches, or two lists subscribed from More lists at once. Settings, tag choices, subscriptions, and the downloaded lists now each have a queue per page (
writeQueueinutils/storage.ts), and changes that depend on the current value (one engine among many) are made inside it.Not done: one queue across pages. Content scripts can't share a lock with extension pages (
navigator.locksbelongs to the search page's origin there), so it would mean sending every change to the background script as data, a rewrite of every edit. Losing a change that way takes two changes in two windows within a few milliseconds of each other.Slow patterns (2026-09-29): a list's
/regex/rules run on every result of every search, in the page, and a pattern like/(a+)+$/backtracks for seconds on the wrong address. JavaScript can't stop a running regex, so they're checked when the list is read: a group that repeats something and is repeated itself (nestedRepeatinutils/listformat.ts) or a pattern over 1,000 characters is skipped as a list error. The check goes by structure, so it also turns down a few safe patterns like([a-z]+\.)*. None of the seven third-party lists in the directory has a repeated group at all (the Huge AI Blocklist's 92 regexes are all like/pinterest.+\/name/, under 100 characters), so none lost a rule."Update all" could be ignored (2026-09-29): the background runs one update at a time, and "Update all" joined a routine check already running (which only updates lists that are due) instead of updating every list. It now runs straight after that check.
Import took half a minute (2026-09-29): importing a big list rewrote the whole personal list once per site, so Brave's Hacker News Goggle (6,238 sites) froze Settings for 33 seconds. Sites are now gathered and written in one pass (
setSitesinutils/personal.ts): 42 ms, with byte-identical results for every list in the directory and the test imports.
Subscriptions and permissions
Verified end to end: pressing Subscribe on "Stack Overflow copies" in the settings downloaded the real file from
raw.githubusercontent.cominside the extension, with no host permission. (In the development sandbox Chromium had to be told to trust the network proxy's CA;e2e/run.mjsdoes that by public key whenPROXY_CA_CERTis set, rather than turning certificate checks off.)raw.githubusercontent.comandgist.githubusercontent.comsendAccess-Control-Allow-Origin: *, so lists there download with no host permission at all. Other hosts ask for permission to that one host when you subscribe (optional_host_permissionson Chrome,optional_permissionson Firefox MV2, which WXT doesn't convert automatically).Updates run when the browser starts and when a search page loads, at most every 30 minutes, instead of using the
alarmspermission.Real lists parsed while testing: Brave's Hacker News Goggle (6,238 rules in 14 ms), Tech blogs, Rust, Copycats, and No Pinterest Goggles; laylavish's AI blocklist in uBlacklist format (1,673 rules; one line skipped for a TLD wildcard, which isn't a valid match pattern); arosh's Stack Overflow and GitHub copy lists. A README fetched by mistake parses to zero rules and is rejected.
Subscribe links (2026-09-29)
Subscribe on the lists directory, and links list authors share, lead to the guide's subscribe?url=…&name=… page, the same shape as uBlacklist's. Anubis opens Settings → Lists with a "Subscribe to …?" panel there; nothing is added until you press Subscribe, since anyone can make a link. A list from the directory shows the directory's name and description, whatever name the link gives.
- How the link reaches the extension. uBlacklist redirects its subscribe page to its options page with a
declarativeNetRequestrule. That needs host access to its site, which it asks for with a separate "enable subscription links" button (so the first link someone follows does nothing), and the options page listed inweb_accessible_resources, which lets any page frame it. Anubis uses a content script that matches only the subscribe page: no new permission, no web-accessible page, and it works from the first click. The cost is one more site in the install prompt (bishop-v.github.io), explained in the privacy page and the store notes. - Where settings open. Turning the subscribe page's own tab into settings (
tabs.update) would leave the subscribe page in its history, so Back would open settings again, and Chrome has no way to replace the entry (Firefox'sloadReplace). Settings open in a new tab beside it instead, and the subscribe page goes back to where the link was, or closes if it was opened in a tab of its own. Back or Forward onto it does nothing (its navigation type isback_forward); the page itself explains and links to itself. - VitePress's router. It follows same-site links without loading a page, and a content script only runs on a page load, so a plain link on the directory would never reach Anubis. Its router leaves links with a
targetalone (seen in its source), so the directory's carrytarget="_self"; the check against the built site below confirms it. - Checked in Chromium:
node e2e/run.mjs subscribe-link(mock directory, subscribe page, and list; the directory tab goes back, a new tab closes, Forward doesn't reopen settings, a second link says you already subscribe), and the built docs site served at its real address, with and without the extension. Not yet run in Firefox, nor on the live site, which isn't published until GitHub Pages is on.
Importing from other tools
- Shipped: Settings → Share and back up → Import sites takes uBlacklist rules, a HOHSER JSON export, a Goggle, or a plain domain list and merges whole-site entries into your list. HOHSER's
FULL_HIDEbecomes Hide,PARTIAL_HIDEbecomes Lower, and its three highlight colours becomehighlight-1..3tags set to highlight. Rules that need URL patterns or regular expressions can't be expressed as whole-site entries, so they're counted and left out, with a pointer to subscribe to the original list instead. - Bug found: the confirmation vanished. Saving triggers a debounced re-render of the section, which ran after the message had already been shown once. Messages now stay for a few seconds across re-renders (
entrypoints/options/flash.ts); the Lists section had the same problem.