Live pages

What was checked on real search pages, and what is still unverified. Part of Experiments and decisions. Newest notes go at the top of each section.

Still unverified against live pages

The development sandbox could not reach any search engine, so everything on search pages was tested against mock pages (see e2e/) shaped like each engine's markup as described by uBlacklist's maintained SERPINFO definitions, checked on 2026-09-28. Before a release, load the extension and check each engine by hand:

Later live check (2026-09-29): with Anubis loaded in a clean headless Chromium profile, Bing returned usable results and Anubis marked 10 results and rendered its summary. Google returned a CAPTCHA; DuckDuckGo showed its protection/landing page, its HTML and Lite pages returned 403, Brave returned 429 with a CAPTCHA, Startpage returned an access-denied page, Ecosia returned 403, Yahoo navigation failed, Kagi returned a page with no recognized results, Yandex asked whether the visitor was a robot, and Mojeek returned 403. A mobile-user-agent Google request also returned a CAPTCHA. A second Bing query returned usable results but did not show the AI-answer or video panels under investigation. These blocks and missing panels are not confirmations of the corresponding live layouts.

The full mock-page e2e run succeeded with Nix-provided Chromium. The first attempt used branded Chrome, which timed out waiting for the extension worker; the first Chromium attempt evaluated the worker before its extension API was available. The shared harness now waits for storage readiness before seeding, and the rerun completed all parts. The e2e suite exercises the reported Brave/Bing/Google panel mocks, but it does not replace the live checks above.

Engine	How results are found	Confidence
Google	Structural (h3), <cite> fallback for /goto links	Structural approach verified live on main before this work; #pnnext for Load more results is long-standing. Automated Chromium gets a CAPTCHA, so the 2026-09-29 check below couldn't reach it
DuckDuckGo	Structural (h2)	Verified live, including #more-results for Load more results (2026-09-29)
DuckDuckGo HTML / Lite	Selectors from uBlacklist	Verified live, including its redirect links (2026-09-29)
Bing	Selectors from uBlacklist, /ck/a redirects decoded	Results and redirects verified live; a.sb_pagN is right, but the fetched page can be a CAPTCHA (2026-09-29)
Brave	Selectors from uBlacklist	Verified live, including offset for Load more results (2026-09-29)
Startpage	Selectors from uBlacklist	Verified live (2026-09-29)
Ecosia	Selectors from uBlacklist	Results verified live; p is right, but the fetched page is blocked (2026-09-29)
Yahoo	Structural, not covered by uBlacklist	Results and /RU= decoding verified live; a.next is right, but the fetched page is blocked (2026-09-29)
Kagi, Yandex, Mojeek	Selectors from uBlacklist (Mojeek structural)	Unverified: each showed a human check to automated Chromium

Checked on live pages on 2026-09-29, with the Chrome build loaded in automated (headless) Chromium, clean-up fully on, and a test list that hid, lowered, raised, pinned, and tagged common sites. Engines treat automated browsers with more suspicion than a person's, so the blocks below may not all happen in everyday use.

• Results and redirects: every engine that loaded found all its results, put the button on each, and applied the test list correctly, which means redirect links resolved to the real site: DuckDuckGo HTML and Lite, Bing's /ck/a, Yahoo's /RU=, and Startpage, Ecosia, and Brave's direct links.
• Load more results worked on DuckDuckGo (11 results to 26, by pressing #more-results) and Brave (20 to 40, offset=1, which matches Brave's own Next link).
• Load more results failed without saying why on Bing, Ecosia, and Yahoo. Their next-page selectors and parameters are right (Bing's Next link is a.sb_pagN with first=11, Ecosia's is p=1, Yahoo's is a.next), but the page Anubis fetches came back as a bot check: Bing's Turnstile page ("One last step", status 200, so it parses as a page with no results), Ecosia's Cloudflare firewall (403, and navigating to page 2 directly was challenged too), and Yahoo redirecting to a _bv/v.gif beacon (500). A 200 challenge page looks the same as the last page of results, so the button just disappears. To do: recognise an empty or challenged page and say so in the summary ("Bing asked to confirm you're not a robot"), and check these three engines again in an everyday browser.
• DuckDuckGo loses your settings on its no-AI version. DuckDuckGo stores settings as cookies for duckduckgo.com only: changing the region set l=uk-en on duckduckgo.com, and noai.duckduckgo.com went back to the region its location suggests. So "AI answers" quietly reset region, theme, and the rest. Resolved (2026-09-29): Anubis no longer sends DuckDuckGo searches there; see "DuckDuckGo stays put" under Clean up pages.
• DuckDuckGo's Wikipedia panel (li[data-layout="about"], in the results list) counts as a result, linked to Wikipedia. Hiding wikipedia.org hides it. That seems reasonable, but it's a panel, not a result.
• Brave's AI answer is #llm-snippet inside #mixed-top, not #summarizer (which wasn't on the page). It has no heading; clean-up removed it through its disclaimer, "AI-generated answer. Please verify critical facts.", which starts with the AI-generated answer marker.
• Bing's AI answer is not removed. It is li.b_ans.b_top at the top of #b_results, with the answer in .cht_container and a .cht_disclaimer element. Its only label is aria-label="AI Overview", which clean-up doesn't read, and its visible h2 is the answer's first sentence. It didn't appear for every query. Done (2026-09-30): both; see above.
• Bing's video panel is not removed. It's li.b_ans.b_vidAns (#serpvidans) with the heading "Videos of how to bake sourdough bread", which the whole-text match for "Videos" misses. Done (2026-09-30): it's a prefix now. Bing's related searches were removed.
• Yahoo: its video and image panels and related searches were removed. Ecosia: related searches removed. DuckDuckGo: the image row and related searches removed.

DuckDuckGo's own "hide this site" was simulated in the mocks (the result collapses into a notice); Anubis removes its UI from the collapsed result. The real feature's markup is unknown.

Clean-up on Google, confirmed from a live results page on 2026-09-29 (the troubleshooting snippet's output: tags, classes, and roles only):

• Everything in the results list sits in a wrapper: #rso > div > …. Panels are div.ULSxyf > div.MjjYud > …, and "People also ask" and the video panel both start with div.A6K0A[data-rpos].

• The AI Overview's label is div.Fzsovc[role=heading], not a heading element, inside div.YzCcne and #m-x-content. Removed correctly (the whole div.ULSxyf) after the label fix.

• "People also ask" and "Videos" labels are span.mgAbYb[role=heading]. The video panel's header row is div.UjLRDc, inside div.vtSz8d with the videos.

• AI Mode appears twice: a tab (div[role=listitem] in div[role=list] in [role=navigation]) and a button[role=link] beside the search box in form#tsf[role=search]. Both are removed with AI answers (the button alone, never the form).

• Tabs named "Videos" and "Short videos" are links in that navigation row and are rightly left alone.

• A second search showed the AI Overview outside the results column altogether: #rcnt > div.bzXtMb.M8OgIe > …, above #center_col. Only the selector from community filter lists (.M8OgIe) caught it, since its label never reaches the column. The images panel is span.mgAbYb[role=heading] inside div.Lv2Cle[data-count], in the usual div.ULSxyf > div.MjjYud > div.A6K0A[data-rpos].

• The summary went above the first result, so a panel before it (the images panel) pushed it down. It's now the first thing in the results area (#rso on Google), above panels. Some results carry a thumbnail in their top-right corner, where the ⇅ button sat on top of it; the button now moves left of any picture there. Both are in the videos=google mock; the previous build fails them.

Still to check for clean-up:

• Brave's Videos, Discussions, and Related queries panels, and Bing's "People also search for" box after going back to the results, with this build (see "Reported from live pages" below). The troubleshooting snippet now covers their headings.

• DuckDuckGo: the AI answer ([data-testid="duckassist-answer-content"] in the results list's first item, or the wikinlp module) and the Duck.ai tab, links, and search-box button (ia=chat links, [data-testid="aichat-button"], [data-ssg-id="ai-searchbox-chat-submit"]). These come from EasyList's AI list, not a live page: this environment's network policy blocks DuckDuckGo.

• Google: that "AI Overview", "Videos", "People also ask", "Top stories" and "Related searches" are still headings (h1–h4 or role="heading") at the top of their blocks, and that the blocks sit in #rso, #botstuff or the role="main" column. That the AI Mode tab is a link with that exact text in a role="navigation" or role="list" element. That choosing All from the Web tab lands on a /search URL without udm, so Anubis leaves it alone.

• The selectors [data-attrid="AIOverview"] and .related-question-pair (Google) come from community filter lists. #summarizer (Brave) wasn't on the page on 2026-09-29; see above. DuckDuckGo's settings on noai. and Bing's and Brave's AI answers were checked on 2026-09-29, above.

Google's phone layout (for Firefox for Android) is modelled on uBlacklist's "Web (mobile)" rules only. To check on a phone: that titles are role="heading" elements with aria-level="3" inside the result's link, that the address is in .ob9lvb, that top stories cards carry data-news-cluster-id, and how the phone layout loads more results.

Reported from live pages: Bing's AI answer and the DuckDuckGo button (2026-09-30)

Reported with screenshots from everyday use, with every Remove panels switch on.

• Bing's AI answer stayed, and the summary sat under it. On the screenshot the answer ("Comprehensions in Python", with "See more") spans the top of the page, with a video, "Related concepts", and "Community support" beside it and follow-up searches under it; the summary came after all of it, so the answer is outside #b_results (the summary goes at the top of the results list). The live check of 2026-09-29 had found the answer in .cht_container with a .cht_disclaimer and no label but aria-label="AI Overview". Clean-up now reads aria-label on blocks (not on controls, navigation, or the page's header) as a label, Bing has .cht_container, .cht_disclaimer as AI selectors, and "AI-generated content may be incorrect", "Generated by AI" and "Copilot uses AI" are markers. The block goes up to where it meets the results, so the video and links beside the answer go with it, and the summary moves above where it was. The copilot Bing mock has that layout; its markup is a guess, and the previous build left all of it and put the summary under it. Unconfirmed on the live page.
• Bing's "Related searches based on your browsing" and the related searches at the bottom stayed. Only the whole heading "Related searches" was known. "Related searches for " and "Related searches based on " are now prefixes, and so is "Videos of " (Bing's video panel, from the live check of 2026-09-29). The mock has the bottom heading as "Related searches for" with the query in bold, and the browsing one beside the results; both were left before. Unconfirmed on the live page: that these are the headings' words.
• The DuckDuckGo button, beside or under the menu by result. The owner asked for it always under the ⋯ menu, and less obvious: DuckDuckGo cuts long addresses off before the menu, so the spot under it is free. It's now centred under the menu on every result, and faint (like on other engines) until the result is hovered. The pages part checks all three DuckDuckGo mocks.

Reported from live pages: the summary, forum results, and more panels (2026-09-29)

Reported with screenshots from everyday use. This environment's network policy blocks every search engine, so each fix is modelled in a mock (forum=1 on Google, inline=1 on Bing, panels=1 on Brave, with checks in the grouped, cleanup, pages and popover e2e parts); the previous build fails each check. The first fixes were guesses from screenshots; the troubleshooting snippets (panel headings, and the structure around Google's result titles, both now in the guide) then gave the live structure for Brave, Bing, and Google, and the mocks follow it. Not yet confirmed in a browser: that this build removes Brave's three panels and Bing's box, and finds Google's /goto results without an address.

• Summary under Google's AI Overview. The summary went at the top of #rso, but the AI Overview can sit above the results column (#rcnt > div.M8OgIe, above), so the summary came after it, and "Show hidden" pushed it further down. It now goes just before an AI answer that comes before the results area, whether it's showing or removed, and is inset with padding to line up with the results. If the page lays it out anywhere but above the results (a grid, a row that isn't wrapped), it goes back to the top of the results for that page.

• Still under it, on a live page (2026-09-29): the fallback was what people saw. With the AI Overview above the results column (#rcnt > div.bzXtMb.M8OgIe.dRpWwb, found by the .M8OgIe selector), the summary went below the Overview, at the top of the results. The page's structure showed the Overview found; #rcnt lays out its children as a grid (display: grid, the Overview at grid-column: 1 / -1, #center_col at 2 / span 12, rows automatic), so the summary placed before the Overview was put in the first free cell, a narrow one on a row of its own, and failed the "above the results" check. Now, when the place before the AI answer fails, the summary goes at the top of the answer itself: the answer's own element, or its first child that isn't laid out as a row or grid. If that fails too, or the answer is removed, it goes back to the top of the results. The aigrid=1 Google mock lays #rcnt out the same way; the previous build puts the summary below the Overview there, and the rest of the e2e run is unchanged.

• Reddit results on Google had no ⇅ button and no tags. Forum results show the forum's name ("Reddit · r/learnpython") and "20+ comments · 2 years ago" where the address would be. With Google's opaque /goto links (uBlacklist switched to reading the address on 2026-08-28), the address was all Anubis had, so the result was dropped. Now it also tries the link's ping address, another direct link in the result, and then a known site's name in the result's link (never its snippet, which can name any site). Confirmed on a live page (2026-09-29, the result-structure snippet): results link to /goto?url=…; a Reddit thread and a "LinkedIn · Fandom / 34.4K+ followers" profile were both missed that way, so the names now cover social sites too (LinkedIn, Facebook, Instagram, X, YouTube, TikTok, Pinterest). Google rewrites a link to /url?…&url=… once it's clicked, which Anubis could always read; that's why one missed result showed a /url link. The first result's sitelinks (h3s in table.SwU7oc) were /goto with no address too, so they weren't recognised as sitelinks and stayed outside the result; an unreadable link with no address beside it now counts as part of the result above it.

• Google's page navigation went with "People also search for". Both sit in one block at the bottom of the page, and clean-up took the whole block (it showed inside the dashed outline). The Next link, the More results button, and any navigation of page numbers are now never removed, and the climb to a block stops below the element that holds them.

• Bing's "People also search for" stayed. Seen on a live page (2026-09-29): it's div#inline_rs.b_hide, hidden, in an li.b_ans of its own in #b_results, which clean-up already removed. It shows under a result you went to and came back from, so it's inferred that Bing moves it into that result's li.b_algo, where clean-up never looked. Related searches and "People also ask" inside a result now go, as the largest block around the label that holds neither the result's title nor text of its own (the snippet); a label that is or holds a link (a sitelink) doesn't count. The inline Bing mock moves the box that way; the previous build leaves it.

• DuckDuckGo's ⇅ button looked out of place beside the result's own ⋯ menu: a different size, a little higher, and 36px in from the corner by a fixed offset. It now sits 4px left of the ⋯, centred on it and at its size and shape, measured each pass (besideMenu on the engine's button). The ⋯ is found by position, as the right-most small button in the result's top-right corner, so its class names don't matter. Only the size, shape, and place were first borrowed; the project owner later asked for unpinned icons to match the ⋯ in light and dark mode. Hover stays gold, and the pin stays gold at rest.

• Auto theme didn't match between the popup and the result menu (Firefox, dark browser, light Google and Bing). On Auto, the popup follows the browser's light or dark, and everything on a search page followed the page's own background. With Light or Dark chosen they matched. Now the result menu, a card over the page like the popup, follows the browser on Auto too; the summary, tags, and hidden-result lines still follow the page, since they have no background of their own. A content script can't always see what the popup sees (Firefox's "Website appearance" can differ from its theme), so the popup, settings, and Firefox's background page record it in local:colorScheme; Chrome's service worker can't, but there pages see the same.

• Brave: "Related queries" wasn't a known heading; it is now, under Related searches. Its "Discussions" panel had no switch; Discussions is a new kind (also Google's "Discussions and forums"), off by default. Confirmed on a live page (2026-09-29): all three panels are div.snippet in section#mixed-main, with the title in a span.desktop-heading-h4 inside a <header>, and clean-up ignored text in any <header> to leave the page's own alone. Only the page's header is ignored now: one outside main, article or section, or holding a form. The tabs (one named "Videos") are links in nav.tabs, still left alone. A first guess, that the Videos title was a link to Brave's Videos tab, was wrong, and the code for it was taken out.

• Tags now add up (asked for, 2026-09-29). Before, when a result's rule carried several tags you'd set to Raise or Lower, only the strongest choice counted and Lower beat Raise, so three Raise tags moved a result as far as one, and two Raise tags and a Lower moved it down; and the same tag from two lists counted twice. Now each tag counts once and they're weighed together, five places per Raise or Lower: three raises move it 15 places, two raises and a lower 5, a raise and a lower cancel out. A Hide tag still hides, your own ranking still decides, and a rule with a chosen tag still leaves its own boost to the choice. "Average" in the request was read as "on balance": a true average would make three raises count the same as one. The result menu's Why gets a line for your tag settings with each tag's part and the total. No cap: a pin (1000) still beats any number of tags, and on a page of ten results 15 places is the top anyway.

• "Raised" but not at the top (DuckDuckGo). docs.python.org showed "Raised" in fourth place. That's by design: the Official docs list raises by 1, one place, while your own Raise moves five places and Pin goes to the top, and the DuckDuckGo mock confirms a +1 moves one place. But the tag says "Raised" for any raise, so it reads as broken. Not changed yet; options are a bigger raise in the Official docs list, or a tag that says how far ("Raised 1 place").

• Two summaries (2026-09-29): sometimes a Google page showed the summary twice, one above the other, while Anubis was being developed with npm run dev, which reloads the extension on every save. When the extension reloads or updates with the page open, Firefox runs the new copy in the same page and the old copy's summary, tags and buttons stay behind; Chrome instead leaves the old copy running in open tabs. Now the new copy clears the old one's elements and attributes when it starts and keeps only its own summary, and an old copy that's still running stops and clears up when the extension goes away (WXT's ctx.onInvalidated). The leftover=1 Google mock carries an old summary; the previous build shows two, this one one, and node e2e/run.mjs checks asserts it.

• Summary wider than the results on DuckDuckGo (2026-09-29): reported from a live page, where the summary ran across the side panel as well as the results. Not checked against the live markup; the likely cause is that the results area Anubis finds (the closest ol, main or web-vertical around the results) also holds the side panel. The summary now also lines up with the list of results whenever that's narrower than where it sits, with padding as before. The wide=1 DuckDuckGo mock puts the results in a div inside a main with a side panel; the previous build draws the summary across both, this one only across the results, and node e2e/run.mjs checks asserts it.

• No space above the summary (2026-09-29): at the top of Google's AI Overview, the summary sat right under the tab row, since the page's own spacing comes after it. Outside the results area it now gets a 16px top margin.

• The ⚖ button over a result's address (2026-09-29): reported on a live DuckDuckGo page, where a long address ran under the button. The button now checks whether it covers any of the result's text. If it does, it moves under the engine's menu button when there is one, otherwise just past the result's right edge when the window has room, otherwise down that edge. The wide=1 DuckDuckGo mock shows whole addresses. On the previous build the button covers text there and on the Bing mock, and node e2e/run.mjs checks now asserts that it covers none on the DuckDuckGo, Google, Bing, and Brave mocks.

• Summary buttons on a wrapped sentence (2026-09-29): when the summary's sentence wrapped, its buttons and the mark were centred on the whole block. They now line up with the first line.

• The ⚖ button above the result (2026-09-29): on live Brave results, which start with padding, the button sat at the engine's fixed top in the space above the site's name. It now centres on the result's first row: the lines above the title (the site's name and address), or else the title's first line. Google's title link also holds the name and address, so the title there is its heading. The checks assert that no button's centre sits above a result's topmost text; the previous build fails that on the Brave mock.

• Summary over Brave's cards (2026-09-29): Brave's results are cards with padding, so the summary's mark sat out past the results' text. The summary is now inset as far as the first visible result's title is from the results' edge, when that's 48px or less. The Brave mock's cards show it; the previous build fails the check.

• Brave's "Find elsewhere" (2026-09-29): a row between Brave's results with buttons that repeat the search on Google, Bing, and Mojeek. It's now its own clean-up kind, Other search engines, found by its label like the others. The row's markup in the Brave mock is a guess from a screenshot; unverified on a live page.

• The ⚖ button beside thumbnails (2026-09-29): on live Brave, a result with a thumbnail in its corner had its button moved left of the thumbnail, so the page's buttons zig-zagged. Now, when a thumbnail is beside the result's first row and the space just past the result's right edge is free (nothing but the result's ancestors there, and inside the window), the button goes there, and every other result's button follows it into that column. Where that space isn't free, it still goes left of the thumbnail. The Brave panels=1 mock's first result has a thumbnail; the previous build puts its buttons at two positions, this one at one.

• Outlines of shown-again panels (2026-09-29): with Show hidden on live Brave, the dashed outlines around removed panels sat 6px outside them, so neighbouring panels' outlines overlapped and stood off from Brave's rounded cards. The outline now sits 1px inside the block's edge, following its corners.

• DuckDuckGo's side panel as a result (2026-09-29): on a live page, the knowledge panel beside the results (a site's infobox) got a ⚖ button and tags: its heading is an h2 with a link, like a result's. Results are now only headings in [data-testid="web-vertical"] li > article, the same place uBlacklist's rules look. The wide=1 mock's side panel has such a heading, and the checks assert it isn't a result.

• DuckDuckGo's button colour (2026-09-29): beside the ⋯ menu, the ⚖ button now takes that button's colour and full opacity at rest, so the pair match in light and dark; it still turns gold on hover.

• The ⚖ button over DuckDuckGo's open ⋯ menu (2026-09-29): hiding the button while a menu was open, found by aria-expanded or a visible role="menu", didn't work live. The owner's console readout showed the open menu is a role="menu" layer inside the result's article, absolutely placed at z-index: 1, while the button sat at 5. Beside a menu the button now sits at z-index: 0, under it. The pages e2e part adds such a layer and checks it covers the button. The tab checks also reseed the personal list, since earlier parts change it and CI found nothing hidden.

• DuckDuckGo's ⚖ button missing (2026-09-29): reported from a live page, where results showed the ⋯ menu with no ⚖ beside it. The hiding above had stayed in: a pass that ran while a menu was open (or still fading out) set the button to visibility: hidden, and closing a menu changes styles rather than adding or removing nodes, so no pass ran to show it again. Since the button already sits under an open menu, the hiding is gone. The pages e2e part opens a menu layer, lets a pass run, closes it by style, and checks the button shows; it failed on the previous build. Not confirmed on the live page, which this environment can't reach.

• DuckDuckGo's Videos tab (2026-09-29): on a live Videos tab, the summary squeezed into a cell of the video grid and cards got ⚖ buttons laid out for web results. First the tabs were skipped (iax in the address); then, following uBlacklist, which has rules for them, the Images, Videos, and News tabs are supported: their cards are found by uBlacklist's selectors (cards in utils/engines.ts), hidden and tagged but not reranked, since reranking's flex column would break the grid. Hidden cards are removed rather than collapsed, the button sits on each card's corner, and the summary goes before a grid instead of inside it. The Videos and Images mocks are built from uBlacklist's selectors; unverified on a live page.