OpenClaw SearXNG: Self-Hosted Private Search for Your Agent

Q: How do I install SearXNG for use with OpenClaw?

The fastest path is Docker: run docker pull searxng/searxng, then docker run -d -p 8080:8080 searxng/searxng. For production, use docker-compose with a custom settings.yml. Once running, verify it works at http://localhost:8080, then point OpenClaw to it with searxngUrl: http://localhost:8080 in your gateway search config.

Q: Which search engines should I enable in SearXNG for best OpenClaw results?

Enable Google, Bing, and DuckDuckGo as your primary general web engines. Add Wikipedia for factual queries, GitHub for code-related searches, and a news engine like Bing News for current events. Disable engines that frequently fail or return spam — SearXNG degrades gracefully when engines are unreachable, but dead engines slow aggregate response time.

Q: Is SearXNG slower than Tavily for OpenClaw agents?

Yes, generally. SearXNG aggregates results from multiple engines in parallel, which takes 1–3 seconds typically vs under 1 second for Tavily basic search. Advanced Tavily search is comparable. The speed difference matters less if your agent handles async search well — the agent can reason while waiting. For time-sensitive applications, Tavily's managed infrastructure has the edge.

Q: Can SearXNG handle the search load from a busy OpenClaw agent?

Yes, for most deployments. A single SearXNG instance handles tens of searches per minute without breaking a sweat. Issues arise if underlying search engines start blocking your IP due to high volume — Google in particular rate-limits aggressive scrapers. Use rotating proxies or limit search rate if you expect very high search volume from your agent.

Key Takeaways

SearXNG is a self-hosted metasearch engine — it aggregates Google, Bing, DuckDuckGo and more without tracking your queries
Docker install takes under 5 minutes; docker-compose with settings.yml for production configuration
Connect to OpenClaw with searxngUrl in gateway search config — no API keys needed
Enable Google, Bing, DuckDuckGo, Wikipedia as your core engines for best coverage
Run SearXNG behind a reverse proxy in production to prevent public exposure of your instance

Privacy matters. So does cost control. SearXNG gives your OpenClaw agent web search that sends zero queries to external API providers — no Tavily account, no Perplexity bill, no logs at a third-party service. You run the search infrastructure. Your agent queries it. Everything stays in your stack.

Why SearXNG Is Worth the Self-Hosting Overhead

SearXNG is a privacy-respecting metasearch engine that queries multiple search providers simultaneously and aggregates the results. When OpenClaw queries SearXNG, SearXNG fans out to Google, Bing, DuckDuckGo, and whatever other engines you've configured, then returns deduplicated, ranked results.

The privacy benefit: neither Google nor Bing sees that your agent is making the search. They see SearXNG's server IP, not your agent's. Your users' queries don't build a profile at a commercial search company.

The cost benefit: once SearXNG is running, there are no per-search costs. An agent doing 500 searches per day pays nothing beyond server compute. At Tavily pricing, 500 searches per day is 15,000 per month — real money on a paid plan.

The tradeoff is latency and result quality. SearXNG aggregates results instead of processing them for AI consumption. Results come back as raw titles, URLs, and snippets — your agent has to work harder to extract meaning. And aggregation takes 1–3 seconds, noticeably slower than Tavily's sub-second basic searches.

💡

Use SearXNG + result summarization

The raw result quality gap closes significantly when you instruct your agent to fetch and summarize the top 2–3 result pages rather than using snippets alone. Combine SearXNG for discovery with selective content fetching for depth. Costs you an extra LLM call but dramatically improves answer quality.

Installing SearXNG with Docker

Docker is the fastest path to a running SearXNG instance. If you don't have Docker, install it first — the official Docker docs cover this for every platform.

Quick start — single container, good for testing:

docker pull searxng/searxng
docker run -d \
  --name searxng \
  -p 8080:8080 \
  -e BASE_URL="http://localhost:8080/" \
  searxng/searxng

Verify it's running: open http://localhost:8080 in a browser. You should see the SearXNG search interface. Run a test search to confirm results are coming back.

For production, use docker-compose with a persistent config volume:

version: '3.8'
services:
  searxng:
    image: searxng/searxng:latest
    container_name: searxng
    ports:
      - "8080:8080"
    volumes:
      - ./searxng-config:/etc/searxng
    environment:
      - BASE_URL=https://search.yourdomain.com/
      - INSTANCE_NAME=OpenClaw Search
    restart: unless-stopped

Place your settings.yml in ./searxng-config/ to customize engine configuration. SearXNG generates a default settings file on first run — copy it out of the container, customize, and mount it back in.

Connecting OpenClaw to SearXNG

Once SearXNG is running, point OpenClaw to it in your gateway config:

search:
  provider: searxng
  searxngUrl: "http://localhost:8080"    # Or your SearXNG URL
  maxResults: 5
  language: "en"
  timeRange: ""                          # "day" | "week" | "month" | "year" | ""
  engines: ""                            # Leave empty to use SearXNG defaults
  maxSearchesPerConversation: 15
  timeout: 8000                          # ms — SearXNG can be slower than APIs

The timeout setting is important. SearXNG aggregates multiple engines, and some are slower than others. The default OpenClaw timeout of 5000ms is too tight — you'll get partial results or timeouts during slow engine responses. Set it to 8000–10000ms for reliable results.

⚠️

Never expose SearXNG publicly without auth

An open SearXNG instance on a public IP will be discovered by bots and used as a search proxy, burning your server resources and potentially getting your IP blocked by search engines. Run it behind a reverse proxy (Nginx, Caddy, Traefik) with basic auth or restrict access to your OpenClaw server's IP at the firewall level.

Choosing the Right Search Engines

SearXNG supports over 100 search engines. Most are off by default. Enabling too many degrades performance; enabling too few reduces coverage. Here's a tested configuration that balances speed and quality:

In your settings.yml:

engines:
  - name: google
    engine: google
    shortcut: g
    enabled: true
    weight: 2.0

  - name: bing
    engine: bing
    shortcut: b
    enabled: true
    weight: 1.5

  - name: duckduckgo
    engine: duckduckgo
    shortcut: d
    enabled: true
    weight: 1.5

  - name: wikipedia
    engine: wikipedia
    shortcut: w
    enabled: true
    categories: general
    weight: 1.0

  - name: github
    engine: github
    shortcut: gh
    enabled: true
    categories: it
    weight: 1.0

The weight controls how SearXNG ranks results from each engine in the merged output. Google at 2.0 means Google results rank above equal-quality Bing results. Adjust based on which engines you trust most for your use case.

For specialized agents: add a news engine (Bing News, The Guardian) for news-heavy agents, Stack Overflow and GitHub for developer agents, arXiv for research agents. Disable engines that consistently return spam or have high failure rates in your region.

Production Configuration

A production SearXNG deployment for OpenClaw needs three things beyond the default Docker setup: rate limiting, reverse proxy authentication, and health monitoring.

Rate limiting protects against runaway agent loops consuming all your search capacity. SearXNG doesn't rate-limit by default — add Nginx rate limiting in your reverse proxy config or set maxSearchesPerConversation conservatively in OpenClaw.

Reverse proxy: run Nginx or Caddy in front of SearXNG. This gives you TLS termination, basic auth, and connection limiting. OpenClaw connects to https://search.internal.yourdomain.com instead of http://localhost:8080.

Health monitoring: add a health check endpoint to your monitoring stack. SearXNG exposes a /healthz endpoint. If it goes down, your agent silently fails to search rather than throwing an error — you want to know before users do.

Common Mistakes

Exposing SearXNG on a public IP without authentication. Already covered — don't do it. Bots will find it within hours and abuse it.

Leaving the default timeout too short. The default 5-second timeout causes 10–20% of SearXNG searches to time out partially during peak load. Increase to 8–10 seconds and you'll see consistent results.

Enabling every available engine. More engines don't mean better results — they mean slower aggregation and more failure points. Start with 4–5 quality engines and add more only if you have a specific need.

Not mounting a persistent config volume. Without a persistent volume, every container restart wipes your SearXNG settings and regenerates defaults. Your carefully tuned engine configuration disappears. Always mount a config volume and keep the settings file in version control.

Frequently Asked Questions

What is SearXNG and why use it with OpenClaw?

SearXNG is an open-source, self-hosted metasearch engine that aggregates results from Google, Bing, DuckDuckGo, and dozens of others without tracking queries. Connected to OpenClaw, it provides real-time web search without external API dependencies, per-search costs, or privacy concerns. It's the zero-ongoing-cost alternative to Tavily for OpenClaw web search.

How do I install SearXNG for use with OpenClaw?

Run docker run -d -p 8080:8080 -e BASE_URL="http://localhost:8080/" searxng/searxng for a quick test instance. For production, use docker-compose with a mounted settings.yml config volume. Once running, point OpenClaw to it with searxngUrl: http://localhost:8080 in gateway search config.

Does SearXNG need to be on the same server as OpenClaw?

No. SearXNG can run on any server accessible to OpenClaw over the network. For privacy, same-server or LAN deployment is ideal. For distributed setups, run SearXNG behind a reverse proxy with authentication to prevent public access to your search instance.

Which search engines should I enable in SearXNG for best OpenClaw results?

Enable Google, Bing, and DuckDuckGo as primary general web engines, plus Wikipedia for factual queries and GitHub for code-related searches. Disable engines that frequently fail or return spam — dead engines slow aggregate response time without adding value.

Is SearXNG slower than Tavily for OpenClaw agents?

Yes, generally — 1–3 seconds vs under 1 second for Tavily basic search. The speed gap matters less if your agent handles async search well. For time-sensitive applications where latency is critical, Tavily's managed infrastructure has a clear edge. SearXNG wins on cost and privacy.

Can SearXNG handle the search load from a busy OpenClaw agent?

Yes for most deployments. Issues arise if underlying engines start blocking your IP due to volume — Google in particular rate-limits aggressive scrapers. Use maxSearchesPerConversation in OpenClaw to cap usage, and consider rotating proxies at the SearXNG level if you need very high search volume.

R. Nakamura

Developer Advocate

R. Nakamura has deployed and maintained self-hosted search infrastructure for AI agent systems, with hands-on experience running SearXNG at scale for production OpenClaw deployments. Focuses on open-source, privacy-preserving tooling for autonomous agent stacks.