Changelog¶
All notable changes to this project are documented here. Format follows Keep a Changelog.
[Unreleased]¶
Added¶
pip-auditdependency-vulnerability scanning in CI, for both the root package andclient/(each as a gating step — a real fail, not just a report).- A dedicated CI job for
client/'s own lint/type-check/test suite. It previously ran locally only — 19 tests were never exercised in CI. - Documented two previously-unwritten limitations in
SECURITY.md,docs/architecture.md, anddocs/api-reference.md: everyAPI_KEYSentry shares one trust domain with no per-key data isolation, andPOST /workflows/{id}/run/POST /agents/routehave no idempotency-key support (a client-side retry re-runs/re-dispatches). Neither is getting built right now — seedocs/design-decisions.mdfor why.
Fixed¶
RedisRoutingStore's Redis connection was never closed on shutdown — it's created via the sameget_routing_store()the supervisor uses, but was never bound toapp.statenor included in the shutdown-close loop inapp/api/main.py. Now bound and closed like every other store.- Tool-execution retry only applied to
/messages(the FSM pipeline); workflow tool-kind steps calledToolRegistry.execute()directly with no retry and no Prometheus metrics. Retry (tenacity) and metrics now live insideToolRegistry.execute()itself, so every caller gets identical behavior. - Error responses were inconsistent — middleware (auth, rate limiting) returned
{"error": ..., "detail": ...}, everything else fell through to FastAPI's default{"detail": ...}. A globalHTTPExceptionhandler now gives every error response the same shape, additively (existingdetail-only consumers are unaffected). This also surfaced and fixed a separate bug:RequestValidationError(raised byQuery(ge=...)constraints, distinct fromHTTPException) needed its own handler, and one specific validation path — a rawValueErrorfrom a Pydanticmodel_validator— embeds the original exception object in the error'sctx, which isn't plain-JSON-safe; the handler now runs errors throughjsonable_encoderlike FastAPI's own default handler does, instead of 500ing on that case. GET /trace/accepted anylimit/offsetint, including negative values, silently clampinglimitto 200 instead of rejecting out-of-range input. Now validated declaratively (Query(ge=1, le=200)/Query(ge=0)), consistent withGET /workflowsandGET /agents/routes.Dockerfilepinned bothFROM python:3.12-slimlines to a specific patch version + digest (3.12.13-slim@sha256:...) instead of the floating3.12-slimtag — builds on different days could previously pull different underlying images silently.- The evaluation harness's ground truth (
Evaluator._expected_intent) expected a"human_escalation"classification intent thatDeterministicClassifierProvidernever produces — escalation is aVerifierAgentoutcome, decided after classification, not a classification category. This made one specific fake ticket template mismatch on every single run, silently cappingclassification_accuracybelow 100% regardless of classifier quality. Removed the bogus branch; added tests that assert real scoring correctness (a wrong classification/tool count must score0.0) instead of the previous>= 0.0check, which passed unconditionally. - Added regression tests confirming a plain, non-
AgentOpsErrorexception (e.g.RuntimeError) raised from inside a dispatched agent is caught and recorded as a failed subtask/step, not propagated — theexcept Exceptionhandling already existed in bothSupervisorAgent._dispatch()andAsyncWorkflowExecutor._execute_step(), but was previously unverified for non-domain exception types.
The three orchestration systems built in 0.x (FSM pipeline, workflow engine, agent bus) are now one substrate instead of three silos, and the gaps a production deployment would actually hit — untested Redis persistence, single-key auth, a rate limiter that only works on one replica, no client — are closed.
Added¶
PipelineAgentwraps the FSM pipeline as aRoutableAgent, registered assupport_pipeline— reachable from a workflow agent-step or a supervisor subtask, not justPOST /messages.WorkflowStep.kind:"tool"(unchanged) or"agent", dispatching through the sameAgentBusthe supervisor uses. Both kinds produce the sameStepResultshape, so output-chaining ($step.<id>.<key>) works across either.- Multi-key API auth (
API_KEYS, comma-separated), replacing the single staticAPI_KEY. Every configured key is checked withhmac.compare_digest; auth success/failure now logs. RedisRateLimiter(RATE_LIMIT_BACKEND=redis) — a shared fixed-window counter, so every replica behind the same host agrees on the limit instead of each enforcing its own.- Background workflow execution:
POST /workflows/{id}/runwith"background": truereturns aPENDINGexecution immediately and runs it via an in-processasyncio.create_task; pollGET .../runs/{execution_id}for the result. agentops-client— an installable Python client (sync + async) andagentopsCLI covering every endpoint, underclient/.- Real Redis CI: a
redis:7-alpineservice container in GitHub Actions, plus test coverage forRedisTraceStore,RedisWorkflowStore, andRedisRoutingStorethat previously only constructed these classes without exercising them. CONTRIBUTING.md, issue templates, PR template,examples/.TRUST_PROXY_HEADERSsetting (defaultfalse): gates whether the rate limiter and auth-rejection logging honorX-Forwarded-For. When enabled, the last hop is used instead of the first.mkdocs.ymland a fulldocs/site (mkdocs-material) covering architecture, API reference, observability, deployment, load testing, and design decisions.locustfile.pyload-testing/messages,/workflows/{id}/run, and/agents/route.project.urlsinpyproject.toml(homepage, repository, issues, changelog).
Changed¶
README.mdanddocs/architecture.mdrewritten to document the workflow engine and agent bus, which existed in code since 0.x but were never documented.- Redis client
close()calls updated toaclose()(redis-py 5+ deprecation).
Removed¶
types-redisdev dependency — redis-py ships its own inline types (PEP 561) as of the version this project depends on; the external stub package was stale and shadowing them.- Dead synchronous accessor methods on
TraceStore(put_sync/get_sync/etc.) left over from the pre-async runtime, unused since the async migration.
Fixed¶
- A test-isolation bug where
Settings()constructed without an explicitredis_urlwould silently inheritREDIS_URLfrom the environment, causing intermittent failures once CI started setting that variable for the Redis-backed tests. - Rate-limit and auth-log client-IP resolution no longer trusts
X-Forwarded-Forby default — the header is client-supplied, and trusting it unconditionally let a caller reset their own rate-limit bucket by sending a fresh spoofed value on every request. /docs,/redoc, and/openapi.jsonnow require auth whenAUTH_ENABLED=true(previously excluded alongside/health//metricswith no stated rationale).POST /agents/routeandPOST /workflows/{id}/runno longer leak raw internal exception text (str(exc)) in the500response body; the exception is logged server-side and a generic message is returned instead.docker-compose.ymlnow hardcodesTRACE_BACKEND/RATE_LIMIT_BACKENDtoredisinstead of${VAR:-redis}. Docker Compose substitutes those from the host's.envfile, and.env.example— which the README/DEPLOYMENT.md quickstart tells you tocpbeforedocker compose up— sets both tomemory, so traces and rate limits silently did not survive an app-container restart as documented. Verified end to end after the fix.- Broken relative links in
SECURITY.mdandDEPLOYMENT.mdthat only resolved correctly on GitHub's blob-URL view, not when included into the mkdocs site.
[0.5.0] — Goal decomposition and routing history¶
DecompositionProvider: deterministic keyword-based and OpenAI-backed goal decomposition, soPOST /agents/routecan generate subtasks from a plain-language goal instead of requiring them explicit.- Routing history persistence (
RoutingStoreProtocol, in-memory and Redis) andGET /agents/routes,GET /agents/routes/{id}. - Workflow step output chaining:
$step.<id>.<key>template resolution against upstreamStepResult.output.
[0.4.0] — Multi-agent bus¶
RoutableAgentABC,AgentRegistry,AgentBus— an async in-process message bus for request/reply dispatch between named agents.SupervisorAgent: concurrent subtask dispatch viaasyncio.gather, with per-subtask success/failure isolation and an aggregatecompleted/partial/failedstatus.- Built-in reference agents (
echo,summary) andGET /agents,POST /agents/route.
[0.3.0] — Workflow engine¶
WorkflowDefinition/WorkflowStep: a validated DAG (duplicate-ID, dangling-dependency, and cycle checks via Kahn's algorithm) at creation time, not at run time.AsyncWorkflowExecutor: topological-layer execution, each layer run concurrently viaasyncio.gather, checkpointed to the store after every layer.- In-memory and Redis-backed workflow stores; full CRUD + run API.
- OpenTelemetry integration: opt-in OTLP export with a no-op tracer shim when disabled, spans per pipeline stage.
[0.2.0] — Async runtime, ops hardening¶
- Migrated the FSM runtime from synchronous to
async def, so a slow LLM call no longer blocks every other in-flight request; concurrent tool execution viaasyncio.gather. APIKeyMiddleware(single static key) and sliding-windowRateLimitMiddleware.- Multi-stage Dockerfile (non-root user, healthcheck) and GitHub Actions CI (lint, type check, test, Docker smoke test).
POST /evaluation— offline evaluation harness against synthetic ticket data.
[0.1.0] — Initial FSM pipeline¶
- Core FSM runtime:
RECEIVED → CLASSIFIED → PLANNING → TOOL_EXECUTION → VERIFYING → COMPLETED, withVALID_TRANSITIONSenforcement. ClassifierAgent,PlannerAgent,VerifierAgent,ResponseGeneratorAgent, each behind a provider ABC with a deterministic default implementation.ToolRegistryand two example tools (CheckOrderStatusTool,RefundPolicyTool).TraceStore(in-memory, LRU-capped), Prometheus metrics, structured JSON logging withtrace_idcorrelation.POST /messages,POST /messages/batch,GET /trace/{trace_id},GET /trace/,GET /tools.