🌐 Overlay Services

Custom HTTP server for interacting with Overlay Services on the Bitcoin SV blockchain.

CI / CD	Quality & Security	Docs & Meta	Community

🗂️ Table of Contents

• Features
• Installation
• Documentation
• Examples & Tests
• Benchmarks
• Code Standards
• AI Compliance
• Maintainers
• Contributing
• License

✨ Features

• Standalone HTTP Server Operates as a self-contained server with customizable configuration and overlay engine layers.

• OpenAPI Integration Supports OpenAPI specifications with an interactive Swagger UI for exploring and testing endpoints.

• Flexible Configuration Formats Allows importing and exporting configuration using common formats such as .env, .yaml, and .json.

• Real-Time Observability Provides basic real-time observability and performance monitoring out of the box.

Middleware & Built-in Components

• Request Tracing Attaches a unique request ID to every incoming request for consistent traceability across logs and systems.

• Idempotency Support Enables safe request retries by ensuring idempotent behavior for designated endpoints.

• CORS Handling Manages cross-origin resource sharing (CORS) to support web applications securely.

• Panic Recovery Catches and logs panics during request handling, with optional stack trace support.

• Structured Request Logging Logs HTTP requests using a customizable format, including method, path, status, and errors.

• Health Check Endpoint Exposes an endpoint for health and readiness checks, suitable for orchestration tools.

• Performance Profiling Integrates pprof profiling tools under the /api/v1 path for runtime diagnostics.

• Request Body Limits Enforces size limits on application/octet-stream payloads to protect against abuse.

• Bearer Token Authorization Validates Bearer tokens found in the Authorization header of incoming HTTP requests and enforces authorization based on OpenAPI security scopes.

📦 Installation

go-overlay-services requires a supported release of Go.

Running as a Standalone Server

1. Clone the repository

   git clone https://github.com/bsv-blockchain/go-overlay-services.git
   cd go-overlay-services

2. Create a configuration file

   cp app-config.example.yaml app-config.yaml

   Edit app-config.yaml to customize your server settings (port, tokens, etc.)

3. Run the server

   go run examples/srv/main.go -config app-config.yaml

4. Optional: Build a binary

   go build -o overlay-server examples/srv/main.go
   ./overlay-server -config app-config.yaml

The server will start on http://localhost:3000 by default (or the port specified in your config).

Using as a Library

To use go-overlay-services as a library in your own Go application:

go get -u github.com/bsv-blockchain/go-overlay-services

See the examples directory for code examples showing how to:

• examples/srv - Run a server with configuration file
• examples/custom - Embed the server in your own application
• examples/config - Generate configuration files programmatically

📚 Documentation

Supported API Endpoints

HTTP Method	Endpoint	Description	Protection
POST	/api/v1/admin/startGASPSync	Starts GASP synchronization	Admin only
POST	/api/v1/admin/syncAdvertisements	Synchronizes advertisements	Admin only
GET	/api/v1/getDocumentationForLookupServiceProvider	Retrieves documentation for Lookup Service Providers	Public
GET	/api/v1/getDocumentationForTopicManager	Retrieves documentation for Topic Managers	Public
GET	/api/v1/listLookupServiceProviders	Lists all Lookup Service Providers	Public
GET	/api/v1/listTopicManagers	Lists all Topic Managers	Public
POST	/api/v1/lookup	Submits a lookup question	Public
POST	/api/v1/requestForeignGASPNode	Requests a foreign GASP node	Public
POST	/api/v1/requestSyncResponse	Requests a synchronization response	Public
POST	/api/v1/submit	Submits a transaction	Public
POST	/api/v1/arc-ingest	Ingests a Merkle proof	ARC callback token

Configuration

The server configuration is encapsulated in the Config struct with the following fields:

Field	Type	Description	Default Value
AppName	string	Name of the application shown in server metadata.	"Overlay API v0.0.0"
Port	int	TCP port number on which the server listens.	3000
Addr	string	Network address the server binds to.	"localhost"
ServerHeader	string	Value sent in the Server HTTP response header.	"Overlay API"
AdminBearerToken	string	Bearer token required for authentication on admin-only routes.	Random UUID generated by default
OctetStreamLimit	int64	Maximum allowed size in bytes for requests with Content-Type: application/octet-stream.	1GB (1,073,741,824 bytes)
ConnectionReadTimeout	time.Duration	Maximum duration to keep an open connection before forcefully closing it.	10 seconds
ARCAPIKey	string	API key for ARC service integration.	Empty string
ARCCallbackToken	string	Token for authenticating ARC callback requests.	Random UUID generated by default

Default Configuration

A default configuration, DefaultConfig, is provided for local development and testing, with sensible defaults for all fields.

Server Options

The HTTP server supports flexible setup via functional options (ServerOption), allowing customization during server creation:

Option	Description
WithMiddleware(fiber.Handler)	Adds a Fiber middleware handler to the server's middleware stack.
WithEngine(engine.OverlayEngineProvider)	Sets the overlay engine provider that handles business logic in the server.
WithAdminBearerToken(string)	Overrides the default admin bearer token securing admin routes.
WithOctetStreamLimit(int64)	Sets a custom limit on octet-stream request body sizes to control memory usage.
WithARCCallbackToken(string)	Sets the ARC callback token used to authenticate ARC callback requests on the HTTP server.
WithARCAPIKey(string)	Sets the ARC API key used for ARC service integration.
WithConfig(Config)	Applies a full configuration struct to initialize the Fiber app with specified settings.

Development Build Commands

Get the MAGE-X build tool for development:

go install github.com/mrz1836/mage-x/cmd/magex@latest

View all build commands

magex help

Development Task Automation

This project uses a dedicated Taskfile.yml powered by the task CLI to automate common workflows. This centralizes critical operations such as testing, code generation, API documentation bundling, and code linting into a single, easy-to-use interface.

Formalizing these processes ensures:

• Consistency across developer environments
• Automation of chained commands and validations
• Efficiency by reducing manual complexity
• Reproducibility in CI/CD and local setups
• Maintainability with centralized workflow updates

Available Tasks

• execute-unit-tests Runs all unit tests with fail-fast, vet checks, and disables caching for fresh results.

• oapi-codegen Generates HTTP server code and models from the OpenAPI spec to keep the API and code in sync.

• swagger-doc-gen Bundles the OpenAPI spec into a single YAML file, ready for validation and documentation tools.

• swagger-ui-up Bundles, validates, and starts Swagger UI with Docker Compose for interactive API exploration.

• swagger-ui-down Stops Swagger UI services and cleans up containers.

• swagger-cleanup Removes generated Swagger files and stops any running Swagger UI containers.

• execute-linters Runs Go linters and applies automatic fixes to maintain code quality.

Repository Features

• Continuous Integration on Autopilot with GitHub Actions – every push is built, tested, and reported in minutes.
• Pull‑Request Flow That Merges Itself thanks to auto‑merge and hands‑free Dependabot auto‑merge.
• One‑Command Builds powered by battle‑tested MAGE-X targets for linting, testing, releases, and more.
• First‑Class Dependency Management using native Go Modules.
• Uniform Code Style via gofumpt plus zero‑noise linting with golangci‑lint.
• Confidence‑Boosting Tests with testify, the Go race detector, crystal‑clear HTML coverage snapshots, and automatic uploads to Codecov.
• Hands‑Free Releases delivered by GoReleaser whenever you create a new Tag.
• Relentless Dependency & Vulnerability Scans via Dependabot, Nancy and govulncheck.
• Security Posture by Default with CodeQL, OpenSSF Scorecard and secret‑leak detection via gitleaks.
• Automatic Syndication to pkg.go.dev on every release for instant godoc visibility.
• Polished Community Experience using rich templates for Issues & PRs.
• All the Right Meta Files (LICENSE, CONTRIBUTING.md, CODE_OF_CONDUCT.md, SUPPORT.md, SECURITY.md) pre‑filled and ready.
• Code Ownership clarified through a CODEOWNERS file, keeping reviews fast and focused.
• Zero‑Noise Dev Environments with tuned editor settings (.editorconfig) plus curated ignore files for VS Code, Docker, and Git.
• Label Sync Magic: your repo labels stay in lock‑step with .github/labels.yml.
• Friendly First PR Workflow – newcomers get a warm welcome thanks to a dedicated workflow.
• Standards‑Compliant Docs adhering to the standard‑readme spec.
• Instant Cloud Workspaces via Gitpod – spin up a fully configured dev environment with automatic linting and tests.
• Out‑of‑the‑Box VS Code Happiness with a preconfigured Go workspace and .vscode folder with all the right settings.
• Optional Release Broadcasts to your community via Slack, Discord, or Twitter – plug in your webhook.
• AI Compliance Playbook – machine‑readable guidelines (AGENTS.md, CLAUDE.md, .cursorrules, sweep.yaml) keep ChatGPT, Claude, Cursor & Sweep aligned with your repo's rules.
• Go-Pre-commit System - High-performance Go-native pre-commit hooks with 17x faster execution—run the same formatting, linting, and tests before every commit, just like CI.
• Zero Python Dependencies - Pure Go implementation with environment-based configuration via .env.base.
• DevContainers for Instant Onboarding – Launch a ready-to-code environment in seconds with VS Code DevContainers and the included .devcontainer.json config.

Library Deployment

This project uses goreleaser for streamlined binary and library deployment to GitHub. To get started, install it via:

brew install goreleaser

The release process is defined in the .goreleaser.yml configuration file.

Then create and push a new Git tag using:

magex version:bump push=true bump=patch branch=master

This process ensures consistent, repeatable releases with properly versioned artifacts and citation metadata.

Pre-commit Hooks

Set up the Go-Pre-commit System to run the same formatting, linting, and tests defined in AGENTS.md before every commit:

go install github.com/mrz1836/go-pre-commit/cmd/go-pre-commit@latest
go-pre-commit install

The system is configured via .env.base and can be customized using also using .env.custom and provides 17x faster execution than traditional Python-based pre-commit hooks. See the complete documentation for details.

GitHub Workflows

🎛️ The Workflow Control Center

All GitHub Actions workflows in this repository are powered by a single configuration files – your one-stop shop for tweaking CI/CD behavior without touching a single YAML file! 🎯

Configuration Files:

• .env.base – Default configuration that works for most Go projects
• .env.custom – Optional project-specific overrides

This magical file controls everything from:

• ⚙️ Go version matrix (test on multiple versions or just one)
• 🏃 Runner selection (Ubuntu or macOS, your wallet decides)
• 🔬 Feature toggles (coverage, fuzzing, linting, race detection, benchmarks)
• 🛡️ Security tool versions (gitleaks, nancy, govulncheck)
• 🤖 Auto-merge behaviors (how aggressive should the bots be?)
• 🏷️ PR management rules (size labels, auto-assignment, welcome messages)

Workflow Name	Description
auto-merge-on-approval.yml	Automatically merges PRs after approval and all required checks, following strict rules.
codeql-analysis.yml	Analyzes code for security vulnerabilities using GitHub CodeQL.
dependabot-auto-merge.yml	Automatically merges Dependabot PRs that meet all requirements.
fortress.yml	Runs the GoFortress security and testing workflow, including linting, testing, releasing, and vulnerability checks.
pull-request-management.yml	Labels PRs by branch prefix, assigns a default user if none is assigned, and welcomes new contributors with a comment.
scorecard.yml	Runs OpenSSF Scorecard to assess supply chain security.
stale.yml	Warns about (and optionally closes) inactive issues and PRs on a schedule or manual trigger.
sync-labels.yml	Keeps GitHub labels in sync with the declarative manifest at .github/labels.yml.

Updating Dependencies

To update all dependencies (Go modules, linters, and related tools), run:

magex deps:update

This command ensures all dependencies are brought up to date in a single step, including Go modules and any tools managed by MAGE-X. It is the recommended way to keep your development environment and CI in sync with the latest versions.

🧪 Examples & Tests

All unit tests and examples run via GitHub Actions and use Go version 1.24.x. View the configuration file.

Run all tests (fast):

magex test

Run all tests with race detector (slower):

magex test:race

⚡ Benchmarks

Run the Go benchmarks:

magex bench

🛠️ Code Standards

Read more about this Go project's code standards.

🤖 AI Compliance

This project documents expectations for AI assistants using a few dedicated files:

• AGENTS.md — canonical rules for coding style, workflows, and pull requests used by Codex.
• CLAUDE.md — quick checklist for the Claude agent.
• .cursorrules — machine-readable subset of the policies for Cursor and similar tools.
• sweep.yaml — rules for Sweep, a tool for code review and pull request management.

Edit AGENTS.md first when adjusting these policies, and keep the other files in sync within the same pull request.

👥 Maintainers

MrZ	Siggi

🤝 Contributing

View the contributing guidelines and please follow the code of conduct.

How can I help?

All kinds of contributions are welcome 🙌! The most basic way to show your support is to star 🌟 the project, or to raise issues 💬.

📝 License