Karl 🎹

"Because the name 'Jenkins' was already taken. The greatest composer ever."

Karl is a lightweight, event-driven CI/CD automation tool designed to bridge the gap between webhooks and infrastructure. It listens for deployment events (specifically from Woodpecker CI), manages secrets securely using KeePass, and automates Docker Compose stack updates.

🧭 Project background

The previous workflow relied on maintaining multiple docker-compose files and uploading them manually or via IDE to several servers over SSH. Karl is a response to my personal CI/CD requirements, focused on improving repeatability, auditability, and secret handling while staying within realistic operational constraints.

Instead of adopting Kubernetes with GitOps tooling (e.g., Argo CD) or alternatives like Watchtower, Portainer, doco-cd, or Ansible, I intentionally optimized for a Docker Compose–based environment:

• Staying with Docker Compose: I run a non-trivial Compose setup (e.g., Nextcloud AIO, 10+ Compose files, ~20 services) and do not currently have the capacity to migrate everything to Kubernetes.
• Git as the source of truth: I wanted deployments to be driven by Git (pull-based, traceable changes) rather than ad-hoc SSH uploads.
• Better secret management: Multiple services share the same credentials (for example, a shared mail “no-reply” account). I wanted to eliminate secrets stored directly in YAML while keeping reuse manageable.
• Why not SOPS: SOPS did not fit the model I needed—I wanted a hardware-agnostic approach using stable secret identifiers rather than content-derived hashes/encrypted blobs.
• No additional secrets service: I preferred not to operate a dedicated secrets backend. A KeePass database can live alongside the configuration repository, with the unlock key treated as the only high-value secret (e.g., provided via Docker Secrets).
• Trade-offs: Using python-on-whales increased the container image size, which is an acceptable but noted downside.
• Future direction: Split the system into two components: a Woodpecker CI plugin that performs most of the orchestration/decision logic, and a minimal companion agent executed over SSH responsible for git pull and Docker Compose operations.

🚀 Features

• Event-Driven Architecture: Built on a reactive event bus for asynchronous task processing.
• Secure Secret Management: Integrates with KeePass (.kdbx) files to securely inject secrets into configuration templates.
• Docker Compose Automation: Automatically reloads services when relevant files are updated in the repository.
• Template Engine: Supports .mo template files for dynamic configuration rendering.
• FastAPI Powered: High-performance API with built-in OpenAPI documentation.
• Dependency Injection: Clean and testable code using the injectable framework.

🛠 Tech Stack

• Core: Python 3.12+
• Web Framework: FastAPI
• DI Container: injectable
• Task Orchestration: python-on-whales (Docker Compose wrapper)
• Secrets: pykeepass
• VCS: GitPython
• Event Bus: bubus

📋 Prerequisites

• Docker and Docker Compose installed on the host
• YAMLs repository in proper format:

.
|- compose/
|  |- some-service-1/
|  |  `- docker-compose.yaml
|  `- some-service-2/
|     `- docker-compose.yaml
|- config/
|  `- put-your-kdbx-here.kdbx
`- files/
   |- some-service-1/
   |  |- service-1-file-1
   |  `- service-1-file-2
   `- some-service-2/
      |- service-2-file-1
      `- service-2-file-2

⚙️ Configuration

Karl reads configuration from config/config.yaml (if present) and allows overriding any value via environment variables.

• Environment variable prefix: KARL_
• Nested keys delimiter: __ (double underscore)

Configuration reference (YAML → env var)

YAML key	Env var	Type	Default	Description
logging.level	KARL_LOGGING__LEVEL	str	"INFO"	Logging level
logging.path	KARL_LOGGING__PATH	Path	null	Log file path
app.host	KARL_APP__HOST	str	"127.0.0.1"	Bind host
app.port	KARL_APP__PORT	int	8081	Bind port
app.reload	KARL_APP__RELOAD	bool	false	Enable auto-reload (dev)
git.path	KARL_GIT__PATH	Path	"/opt/repo/sample"	Local repository path
git.url	KARL_GIT__URL	str	null	Git remote URL
git.branch	KARL_GIT__BRANCH	str	"master"	Branch to use
git.remote	KARL_GIT__REMOTE	str	"origin"	Remote name
kp.file	KARL_KP__FILE	str	"database.kdbx"	KeePass DB file path
kp.secret	KARL_KP__SECRET	Path	"/run/secrets/kp_secret"	Master password source (file path or string)