Hub (Go)Overview

Hub

hivecfm-hub is the Go service that sits next to the Next.jsNext.jsReact framework used by HiveCFM Core. Handles routing, server rendering, and API routes in one bundle. app and does the work that is either CPU-bound, long-running, or concurrency-heavy. Think embeddings, sentiment classification, webhook dispatch, and semantic search. It is a separate process, deployed as its own container, but it shares the same Postgres instance with core.

Why Go?

The web app is already TypeScriptTypeScriptJavaScript with a static type system. Every HiveCFM Node service, the frontend, and the dev hub are written in it. — why a second language?

  • Predictable concurrency. Go’s goroutines + channels give us a worker pool model that is hard to match in Node’s event loop. When the Hub processes 500 embedding jobs in parallel, it does so cheaply.
  • Low memory per request. A Go binary with a worker pool holds state in kilobytes, not the hundreds of megabytes a Node worker can balloon to under load.
  • Fewer surprises under GC pressure. Go’s GC is tuned for short-lived allocations. Embedding batches and HTTP retry loops are a natural fit.

The split is not religious: trivial APIs stay in TypeScriptTypeScriptJavaScript with a static type system. Every HiveCFM Node service, the frontend, and the dev hub are written in it.; anything that looks like a background worker pool lives in the Hub.

Ports

  • 8090 — primary HTTP API (semantic search, webhooks, admin).
  • 8080 — optional secondary listener for internal-only management endpoints in some deploys; if your deploy does not set INTERNAL_PORT, 8090 is the only port you need.

Both bind to 127.0.0.1 in Docker ComposeDocker ComposeThe YAML file and CLI that spin up Postgres, Redis, MinIO, MailHog locally with one command.; production puts them behind the same ingress as core.

Layout

hivecfm-hub/
├── cmd/
│   ├── api/                  # HTTP + worker entrypoint (main.go)
│   ├── backfill-embeddings/  # One-shot: embed historical rows
│   └── backfill-sentiment/   # One-shot: classify historical rows
├── internal/
│   ├── config/               # Env loading
│   ├── models/               # Domain types
│   ├── observability/        # OTel metrics + tracing
│   ├── repository/           # Postgres access (pgx)
│   ├── service/              # Business logic (SearchService, WebhooksService, ...)
│   └── workers/              # River job workers
├── pkg/
│   └── database/             # Shared Postgres pool helpers
├── migrations/               # goose-managed SQL migrations
├── openapi.yaml              # API contract, mirrored into the dev hub
└── compose.yml               # Standalone docker-compose for Hub-only dev

Entry point

hivecfm-hub/cmd/api/main.go is the binary’s main. In one page it:

  1. Loads config from env (internal/config).
  2. Sets up slog structured logging.
  3. Opens a pgx pool — database.NewPostgresPool(...) — with pgvector type registration on connect.
  4. Runs migrations in-process (runMigrations — both the goose-managed Hub migrations and the River queue’s own tables).
  5. Constructs the App (wires services, workers, HTTP routes).
  6. Calls app.Run(ctx) which starts the River worker pool and the HTTP server concurrently.
  7. On SIGINT/SIGTERM, app.Shutdown(ctx) drains jobs and closes the pool.

A typical Go service shape: no frameworks, no reflection-based DI, explicit wiring.

The River queue, not Kafka. HiveCFM’s async backbone is RiverRiverThe Go background-job queue Hub uses. Jobs are rows in Postgres, so there is no separate broker to run., which stores jobs in Postgres tables. There is no message broker. See Workers.

What the Hub owns

  • Async jobs. Feedback embedding, sentiment analysis, webhook dispatch — see Workers.
  • Semantic search. The SearchService in internal/service/search_service.go exposes nearest-neighbour lookup with cursor pagination. See Search.
  • Webhook outbox. Storing, retrying, and signing outgoing webhooks.
  • Backfills. One-shot CLIs under cmd/backfill-* to re-process historical data.

It does not own user-facing pages, auth, billing, or anything transactional. Writes that must be synchronous-from-the-user’s-POV stay in hivecfm-core.

Calling the Hub from core

The web app calls the HubHubThe Go service that owns background processing, integrations, and the admin API. Sibling to Core. over HTTP for synchronous reads (e.g. /search for a UI that wants results immediately). For async work, core enqueues a RiverRiverThe Go background-job queue Hub uses. Jobs are rows in Postgres, so there is no separate broker to run. job by inserting a row into the shared river_job table — the Hub picks it up on its nextNext.jsReact framework used by HiveCFM Core. Handles routing, server rendering, and API routes in one bundle. poll. There is no RPC framework.

Migration story

  • Core’s migrations live in hivecfm-core/packages/database/migrations/ (managed by Prisma).
  • Hub’s migrations live in hivecfm-hub/migrations/ (managed by goose).
  • Both run against the same Postgres instance, but each owns a disjoint set of tables. Core does not ALTER Hub tables and vice versa.

Read next

pgvector and SearchWorkers