English
Engines
Keeping engine definitions current, and engine bugs found while testing. Part of Experiments and decisions. Newest notes go at the top of each section.
Engine fixes from the repository (2026-11-03)
- Why: when an engine changed its page, Anubis stayed broken for everyone until a release passed store review, often days. uBlacklist avoids that by downloading its SERPINFO definitions.
- Built:
engines/fixes.jsononmain, downloaded by the background script at most every six hours (an hour after a failure, keeping the last good copy), and put over the bundled definition by each search page (fixedEngineForinutils/enginefixes.ts). A fix names one engine and a range of Anubis versions (from,until), and can only replace selectors, the ⚖ button's offsets, and a card kind's selectors by its place. Everything else in the file is dropped when it's read, and a selector the page can't parse keeps the bundled one. A phone layout's own values come before a desktop fix's, as they come before the bundled ones;mobilein a fix changes them.npm run release:prepstops while a fix still applies to the version being released, so each fix is folded intoutils/engines.tsand ended there. - Checked: unit tests for reading, versions, applying, downloading, and the committed file (
tests/enginefixes.test.ts); thefixese2e part, inchecks, serves a Yahoo page with its titles inh4, which the bundled definition misses, and finds every result once a fix saysh4, while a fix that doesn't parse leaves the bundled one working. It fails on the previous build. - Not done: what is code today.
isResultsPageand a card'swhenare functions, andmatchesis in the manifest, so they still need a release; so does a new engine. A failed download keeps the last fixes, and an install that never downloaded uses none. - Weighed: fixes are data, read field by field, so they fit Chrome's and Firefox's rules on remote code, as SERPINFO does. The file's trust is the repository's: whoever can merge into
maincan already ship a release, but a fix skips store review, so it should stay small and reviewed. The request goes to raw.githubusercontent.com, which subscribed lists already use, so no permission was added; the privacy policy lists it.
Keeping platform and engine maintenance current (2026-09-29)
- Firefox minimum: AMO lint warned that Firefox for Android 140 predates support for the built-in data-collection manifest key. Raised
gecko.strict_min_versionto 142, the first Android version with that key; deliberately did not addgecko_android, which would advertise untested phone support. The generated Firefox manifest now declares 142.0 andweb-ext lintreports zero warnings. - Action runtimes: CI reported the pinned GitHub Actions running on the removed Node 20 runtime. Replaced the checkout and setup-node pins across CI, docs, releases, and the SERPINFO watch with immutable SHAs for releases whose metadata specifies Node 24.
- Engine selector ownership: moved the Google, DuckDuckGo, and Brave cleanup selectors into each
EngineDef. This removes the parallel engine-id registry and keeps a selector next to the result layout it describes. SERPINFO remains a weekly review signal, not a selector generator: its model differs from Anubis's structural detection, boundaries, and cleanup rules.
Keeping engine definitions in sync (2026-09-29)
- Starting point: the weekly workflow already watched eight uBlacklist
serpinfo/*.ymlfiles, but its file-to-engine mapping lived only inwatch-engines.mjs. It could silently fall behind when an engine was added, and fetching the 50 most recent engine issues could miss a previously reported commit. - Upstream check: GitHub's Contents API listed the current
serpinfo/files, includinggoogle.yml,duckduckgo.yml,bing.yml,brave.yml,startpage.yml,ecosia.yml,kagi.yml,yandex.yml, andyahoo-japan.yml. The GitHub HTML directory view could not be retrieved in this environment; the API endpoint worked.yahoo-japan.ymlis not a match for Anubis's supported Yahoo engine, and there is no Mojeek file, so those remain explicitly unwatched rather than guessed. - Shipped experiment: move the mapping into
.github/engine-watch.json; test that every engine inutils/engines.tsis either watched exactly once or has a stated reason it is not. The watcher also fails if a mapped upstream path disappears. Add local-repository tests for recent upstream changes, deduplication, unrelated files, old commits, and missing mapped files. Remove the issue-watch's silent failure and raise its pagination limit. - Ran against upstream: cloned
ublacklist/builtin, collected the existing engine-issue bodies withgh, and ran the watcher without creating an issue. It found two Google commits from 2026-09-27: desktop video/gotolinks now fall back to the displayed domain, and image links can fall back todata-lpage. Anubis already uses the displayedcitefallback for opaque links on supported web results; image-search pages are intentionally excluded, so neither change needed a selector change here. - Rejected: generating Anubis selectors directly from SERPINFO. The upstream rules describe a different matching model and don't cover Anubis's structural boundaries, page cleanup, or pagination; generated selectors could appear current while breaking real pages. The weekly issue remains a signal for human review and live-page confirmation.
- From an issue to a pull request (2026-09-29): an issue only linked to upstream commits, so each one meant reading uBlacklist's history by hand, and nothing in the repository recorded which version of its rules Anubis was last compared with. Anubis now keeps a copy of the eight watched files in
upstream/serpinfo/(MIT, licence alongside), and the weekly workflow updates the copy on a branch and opens a pull request. The review is the diff;source.jsonrecords the upstream commit, so the next sync lists only the commits since. Still not generated selectors (rejected above): the copy is data for review and tests, and the build doesn't read it. - Checking addresses against the copy: a test reads every address in each copied file's
matchesand checks the engine's ownmatchescovers it. Run against uBlacklist atb2f1acc(2026-09-27), it found Bing'swww2andwww4hosts and 14 Yandex country domains (yandex.az,.by,.co.il,.com.am,.com.ge,.ee,.eu,.kz,.lt,.lv,.md,.tj,.tm,.uz) that Anubis didn't run on; they were added. Google's country list and DuckDuckGo's hosts already matched. A sync pull request now fails CI when uBlacklist starts matching a new address, which is the one kind of upstream change a test can judge by itself. Adding hosts widens the content script's matches, which is fine before the first store release; after it, Chrome asks users to approve new sites, so weigh each addition then. - CI on a workflow's pull request: pull requests opened with
GITHUB_TOKENdon't start other workflows, so the required check would never run. The sync startsci.ymlon the branch withworkflow_dispatch, which is allowed, and the check reports on the branch's commit. Not verified on GitHub yet: the first scheduled run will show whether the protected branch accepts that check. - uBlacklist as a second opinion on the mocks: every search-page check runs against mocks in
e2e/fixtures.mjs, written without access to the real engines, so nothing said whether a mock still resembled the engine. uBlacklist's result selectors are kept current against the real pages, sotests/mock-realism.test.tsparses each mock (with linkedom, no browser) and counts the elements uBlacklist's web-result selector finds. On 2026-09-29 all six (Google desktop, its awkward layout, its phone layout, DuckDuckGo, Bing, and Brave) found exactly one per result, which is the first independent evidence that the mocks use the real class names. The test also checks each selector still appears in the copied rules, so when the weekly sync brings a new one, the pull request names the mock to update. It checks result containers only: uBlacklist's rules say nothing about Anubis's headings, boundaries, clean-up, or paging.
Engine watch
- Upstream changes of 2026-10-06 to 2026-10-17 (DuckDuckGo, Brave), applied from the diff, not checked on live pages. Brave:
safe.search.brave.comis now an address of the engine, and Discussions (.discussions-item > .accordion-item) are weighed on the web tab. Brave's Enrichment Cards are left out, since their link is the card itself and a card's link is looked for inside it. DuckDuckGo: image titles are theh3, and the web list, HTML, and Lite pages leave out ads by the rules' selectors, beside theadsselector. - A weekly workflow opened an issue when uBlacklist changed its rules for an engine Anubis supports (replaced by a pull request on 2026-09-29, below). The development sandbox can't reach GitHub's API, so the script reads upstream history with
git logover a blobless clone and only files the issue throughgh. Run against real history, it reported four Google changes in 45 days; commits already mentioned in anenginesissue are skipped, so the ten-day window can overlap safely.
Bugs found while testing
- Ecosia: "refused to send page 2", checked on the live site (2026-10-15). Anubis's request for page 2 (Firefox,
content.fetch) got a 403 with Ecosia's own "Confirm you're not a robot" page, though its headers and cookies (Cloudflare's clearance included) matched a consolefetchthat got page 2 with status 200. Whatever Ecosia judges by isn't in the headers, and Anubis doesn't try to get past a robot check. The hidden frame tried after it was blocked by Ecosia'sframe-ancestors, so it can never work there. Ecosia now hasframes: falseand skips the frame, and a 403 reads "Ecosia wants to check you're not a robot before it sends page 2." with Open page 2. Thecheckspart has an Ecosia page whose request gets a 403 and whose page load would get results: before the change a frame was tried and the summary said "refused". Confirmed on the live site (2026-10-16): the summary names the check and Open page 2 leads to it. The wiki's Load more results page now has a Robot checks section. - Bing: AI answers took the sponsored carousel with them (reported 2026-10-15, with a console readout). The top carousel is
li.b_PolePAContainerholding.pa_carousel_mlo, in#b_poleat the top ofmain, labelled "Sponsored" in plain text. Clean-up widens a block it finds to the largest container that holds no results, so an AI answer sharing a container with a carousel took both, and the owner wanted them on separate switches. Now ads are removed exactly by the engine'sadsselector (now with Bing's carousel classes), a "Sponsored" label inside such an ad is left to the selector, and no other block grows to take in an ad it doesn't hold (holdsOtherAdsincleanup.ts). Thebingmock'spoleoption has one carousel in#b_poleand one beside the AI answer; thecheckspart fails without the change (the AI answer's removal took the carousel beside it). - Bing: results blank until a reload (reported 2026-10-15, screenshot). The summary and every ⚖ button were there, but each result was empty space, all exactly 247px tall: the placeholder height of a result the browser skips drawing because it takes it to be off-screen (
content-visibility: autowithcontain-intrinsic-size). Results Anubis has reordered stayed skipped.page.cssnow draws every weighed result (content-visibility: visible). Thebingmock'sskippedpage gives its results that style; thecheckspart fails without the change. The mechanism is inferred from the uniform heights, and the mock checks the style rather than the blank page, which Chromium doesn't reproduce, so confirm on the live page. A settle delay for Bing like Ecosia's was tried and dropped: it broke the phone checks' timing and wasn't shown to be needed. - Ecosia: site icons missing in results from Load more results (reported 2026-10-15, with the icon's address from the page's resource timing). Ecosia's script fetches each site's icon (
api.ecosia.org/v2/favicons/fetch?domain=<origin>&fz=s&theme=dark) and puts it in as ablob:image; the server-rendered page only has a globe in afavicon__container--fallbackholder, which is what imported results kept.iconinEngineDefnow has Anubis draw that image for each imported result (the theme from the page's background), and put the placeholder back if it doesn't load. Theecosiamock has the holder and thecheckspart fails without the change (0 of 3 results got an icon). The request is a plain image, like the one Ecosia's page makes for its own results. Bing's sponsored carousel is still unfound: its "Sponsored" label is probably inside a custom element (acf-badge), which neither Anubis's text search nor a console search sees, so it needs matching by its "See …" heading or its container. - Ecosia: the button sat between the site's name and its address, not level with the ⋮ (reported 2026-10-15, screenshot). A result's header is two lines (name, then address with the ⋮ at its end), and the button was centred on both. It now centres on the engine's small menu button when there is one in the lines above the title (
menuSpaninui.ts), and on the lines' text otherwise; this applies on every engine that takes this path (Bing's and Google's ⋮ too). Theecosiamock has the two-line header and a ⋮; thecheckspart fails without the change (12px off). - Ecosia stuck on grey placeholders with no side panel, and Bing's sponsored carousels (reported 2026-10-15, screenshots). Ecosia builds its page in the browser; with Anubis on it stayed on placeholders (the "AI answer that never loads"), had no side panel or icons, and loaded normally with Anubis off for Ecosia. This is the blank first search again, whose fix only revealed results: an element added before the page has finished building breaks it.
settle: 1500on the engine leaves the page alone until 1.5 seconds afterload. Theecosiamock records whether anything of Anubis's was on the page when its script built it; thecheckspart fails without the setting. A guess at the cause that the owner's report fits: confirm on the live site. Bing's "See …" carousels carry a "Sponsored" label: the Ads switch now also finds blocks by that label (headingsof theadskind inutils/cleanup.ts), plus.b_adblocks, and the summary goes above them whether the switch is on or off. Thebingmock'ssponsoredpage has a guessed structure; send the carousel's markup if it isn't found. The ⚖ button rests at 0.7 opacity instead of 0.35, after the owner found it hard to see. - Yahoo: buttons still in two columns after the column fix (reported 2026-10-15, with a console readout). Four results beside the right-hand panel stayed at the box's edge (
right: 2px) while the rest sat in the gutter (-28px): the gutter test found an unclassedDIVover the spot for those four only, probably the side column's wrapper. Once any button is in the gutter, every button now joins it without the test (forceinintoGutter). Thebingmock'sthumbpage puts something over the gutter beside a picture result and a plain one after it; thecheckspart fails without the change (buttons at 804 and 772). The Yahoo mock doesn't reproduce it. Also: the summary goes above an engine's ads (adBlocksinindex.ts), shown or removed. - Yahoo: one result's button sat 30px right of the rest (reported 2026-10-15, screenshot). A result whose title won't wrap (
white-space: nowrap) stretches its box, and the button sits at its own box's edge. Every button is now placed against the column's edge, the middle of the results' right edges (columnEdgeinui.ts), whether at the edge or in the gutter beside it. Theyahoomock'slongTitlestretches one result; thecheckspart fails without the change (buttons at 730 and 1162). Also: Yahoo's ads are found by theyModType: adin theirdata-ygatoo, so an ad between the results leaves the results and goes with the Ads switch. - Bing: Load more results gave page 1 again from a page reached by a redirect (reported 2026-10-15). The Next link on such a page carries
rdr=1&rdrig=…, which Bing reads as "already redirected" and answers with the same results, so the summary said Bing sent the same results again. The address is now built withoutrdrandrdrig(dropinMoreResults). Thebingmock fails without it. Inferred from the address the owner sent, not yet confirmed on the live page. - Yahoo: ads were weighed as results (reported 2026-10-15). The top ads are an
ol.searchCenterTopAdsholding a product carousel, an "Ads related to" row, and text ads with anh3like a result's, so they got ⚖ buttons, counted in the summary, and sat 30px left of the organic buttons.adsinEngineDefnow takes them out of the results, and a new Ads switch in Settings → Remove panels removes them. Only.searchCenterTopAdsis confirmed; the middle and bottom names are guesses. Theyahoomock carries the ads markup as the page showed it and a guess at the organic results. - Bing: the button went left of a picture (reported 2026-10-15). A result with a thumbnail (the Yahoo result with a photo) had its button left of the picture while the others sat in the column past the results' edge: the free-space test for that spot failed for that one result (something else was beside it), and the button fell back to the picture's left. When the others are already in the column, a result with a picture now joins them. The
bingmock'sthumboption puts pictures in two results and something in the gutter beside the second; thecheckspart fails without the change. - Reported from live pages (2026-10-15), fixed against mocks only: Mojeek had no Load more results (it was never set up: pages are
s=11,s=21…, from memory of its pager, so confirm in a browser). Bing's results from Load more results had empty site icons: Bing draws each icon from a.rms_iacplaceholder (data-src) that its own script turns into an image, which never runs for imported results;sanitizeindeeper.tsnow builds the image (thebingmock has the placeholder, and thecheckspart fails without the fix). Ecosia once showed a blank results area on the first search of a session, fine after a reload, with the ⚖ buttons and tags in place: results stay invisible until Ecosia's script addsmainline__result-wrapper--visible, soloadedClassis now added to every result on each pass, not just imported ones. That is a guess at the cause with no mock, so check it on the live site. Not done: Yahoo's ⚖ buttons are not lined up (no screenshot yet), and no Yahoo or Mojeek mock exists. - Yandex: the weigh button stayed beside the ⋮ menu, and there was no Load more results. The menu is a plain
button(aria-label="Actions", 16×24), but the finder only accepted one within 64px of the result box's right edge, and Yandex's box is wider than its card. It now takes the right-most small button in the box's right half. Pages arep=1,p=2… (0-based, none on page 1), so Load more results uses the page-number mode. Unverified on a live page, and Yandex may answer a script with a robot check. The result menu opens to the right of the result's card (popOut: '.Organic'), level with its top, as Yandex's own menu does; the weigh button's colour is read from the menu icon's circles (it only looked at paths). Yandex AI's answer (li[data-ilefbgq-name="neuro_answer"], a.FuturisSearchcard) is removed by clean-up with the other AI answers. - Yandex: no summary until something was weighed, and the weigh button sat over the result's corner. The summary only showed when Anubis had changed or offered something, so a Yandex page with nothing weighed had none (and no Load more results to offer); it now shows whenever there are results, saying it left them as they were. Yandex's results have a ⋮ menu like DuckDuckGo's, so the engine now uses
underMenu. Unverified on a live page. - Ecosia: results from Load more results were invisible. The frame retry got the page through, but Ecosia hides each result (
visibility: hidden) until its own script addsmainline__result-wrapper--visibleas it scrolls into view, which never runs for results Anubis adds. The engine'sloadedClassoption adds that class on arrival. - Ecosia: "refused to send page 2". Fetching the next page from a script gets a 403 with a short robot-check page (4.6 kB, no results), while the same page loads normally in the browser. Load more results used to stop at any refusal, assuming a frame wouldn't be let through either; it now tries the hidden frame after a 403 (a real page load), and still stops at once on 429 or 503. Unverified on a live page.
- Ecosia: Lower and Pin did nothing, the summary sat below the definitions panel and jumped after Load more results. The result was taken to be
article.result, but each article is alone in a.mainline__result-wrapper, so there was nothing to reorder, and loaded articles went into the last wrapper, which then looked like the results list. The result is now the wrapper, and.mainline__contentis the area. Unverified on a live page after the change. - Startpage: no summary on page 2, and no Load more results. Page 1 is a bare
/sp/search(a post), so a page-number parameter or Next link can't find page 2; its pager is one post form per page with a hiddenpagefield. Added aformkind of Load more results that posts the next page's form. The pager lists every page including the one shown, then a Next form that repeats the following page's number (1,2,3,4,5,2on page 1); a first version guessed the current page from gaps in the numbers, picked page 1's own form, and got page 1 back ("sent the same results again"). Now the Next form's repeated number gives the page. Posting the forms from the console on a live page returned new results for pages 2 to 5. Fetched results then came in with the light theme's colours: the fetched page's generated class names differ from the live page's (css-1bggj8vagainstcss-1ubyvt6for the same title), and the colours are rules on those classes. The engine'srestyleoption gives each fetched element the generated classes of the same element (matched by its stable class names and place) in a result already on the page. The summary only shows when Anubis did something, so a page 2 with nothing hidden has none. Also set the engine's boundary tosection#mainso the summary sits above Startpage's ad notice (its class names are generated). - Show on one hidden result reverted when the mouse moved. The button set
data-anubis-revealon the result directly. Engines rewrite parts of the page on hover, that runs another pass, and the pass reset the attribute from the page-wide "Show hidden" state. Results shown one at a time are now remembered by URL until the next search. The e2erevealpart clicks Show, changes the page and moves the mouse; it failed on the previous build. - The popup got no numbers from the page in Chrome. The content script replied to messages by returning a promise, which Firefox accepts and Chrome ignores, so the popup's "This page" section never filled in Chrome. Both scripts now reply with
sendResponse. Found because the e2e harness asked the page for its stats and gotundefined. - Testing closed shadow roots. Page scripts and Playwright locators can't reach into closed shadow roots, so e2e clicks buttons there through the DevTools protocol (
DOM.getDocumentwithpierce: true, then the button's box).
Google in Firefox: flipped tags, missing buttons, misplaced summary
Follow-up: after the fixes below, the summary still sometimes appeared below the first results. The anchor was the first result in the list holding most results, and Google nests some results one level deeper than the rest: a first result with sitelinks, or a group of results. The first result then wasn't in that list, so the summary went below it; with six sitelinks, the sitelinks became that list and the summary went inside the first result. A grouped Google mock (six sitelinks under the first result, two results in a group) reproduces this on the previous build.
- Summary anchor, again. The results area is now the list holding most results, widened to the engine's boundary (Google's
#rso). The summary goes above the first result anywhere in that area, as a direct child of the smallest element holding every result in it. - Sitelinks. Each sitelink has its own
h3, so each was weighed as a result, with its own tags and weigh button, and it cut the first result's container short (its parent held a second heading), leaving the snippet and sitelinks outside it. A heading that links to the same site, and shows no address of its own, now counts as part of the result above it. Grouped results each show an address, so they stay separate. - To confirm live: that Google's sitelinks still use
h3inside a link and show no<cite>, and that#rsostill holds the results.
Reported from real use: on Google in Firefox the "Reference" tag and "Hidden" read upside down, the weigh button appeared on only the first two results, and on page 2 the summary sat under the first result with no weigh buttons at all. Google couldn't be loaded from the development sandbox, so the fixes target every mechanism that produces these symptoms, and a hostile Google mock (google(…, hostile = true) in e2e/fixtures.mjs) reproduces all three on the previous build and passes on the new one.
- Flipped text. Some layouts flip a wrapper with a transform and flip their own children back, which leaves anything else inside the wrapper upside down. The tag row was inserted inside the title link's wrapper. It now climbs out of wrappers that hold only the title, steps outside any transformed wrapper, and, as a last resort, adds up its ancestors' transforms and applies the inverse so its text reads upright.
- Page CSS reaching Anubis's elements. A shadow root protects its contents, not the host element, and the page's stylesheets can still match the host (
… > :last-childand the like), hiding or transforming it. Every host now pins display, visibility, opacity, position, and all transform properties inline with!important, which beats any page rule. - Nesting depth. Finding a result's container gave up after 8 levels, a limit carried over from the original script. Google already nests a result about 8 deep, so a few more wrappers left Anubis on an inner block: the weigh button, hiding, and reranking all acted on part of a result, and the summary landed inside the first result. The limit is now 30; the real stops are the results boundary and a parent holding a second result.
- Summary anchor. The summary went before the first heading in page order, which can belong to something else. It now goes before the first result in the list that holds most of the results (revised in the follow-up above).
- The weigh button now rests at low opacity instead of being invisible until hover, as the original block icon was always visible.