Development Guide¶

ABI is shipped as one Python distribution, abi-agent. This page is a map of the codebase and the main extension points. For the order in which changes should be made and checked, see development_workflow.md.

Source Tree¶

src/abi/
  agent/              ABIAgentInterface, JSON envelopes, agent context export
  agent_integrations.py  Claude Code, OpenCode, and Codex integration installer/doctor
  figures/            FigureEngine (7 renderers), FigureSpec — generic figure system
  report/             write_full_report, write_plugin_report, write_methods,
                      citations, limitations, html — generic report system
  workflow/           ResourceManifest, workflow validation, figure_specs loading
  plugins/            Built-in analysis-type plugins
    metagenomic_plasmid/   Self-contained plugin package (engine in _engine/, 64 tools, 90-node DAG)
    easymetagenome.py     Shotgun metagenomics adapter (10 tools, 25-node DAG)
    viral_viwrap.py       Managed external CLI adapter (1 tool, 7-node DAG)
    rnaseq_expression.py  Bulk RNA-seq (5 tools, 5-node DAG)
    wgs_bacteria.py        Bacterial WGS (5 tools)
    amplicon_16s.py        16S microbiome (10 tools)
    metatranscriptomics.py Metatranscriptomics (3 tools)
  autoplasm/          Backward-compatible re-export shim → plugins/metagenomic_plasmid/_engine/
  sciplot/            Publication-grade Matplotlib figure compiler — FigureSpec → Validate →
                      Render → Export → Lint → Provenance. Pydantic schema, 15 plot types,
                      3 themes, lint rules, SHA-256 provenance.
  dag_planner.py      UniversalDAG — declarative plan generation from pipeline_dag.yaml
  tsv_mapping.py      Declarative TSV column mapper — YAML-driven output parsing, 3 source types
  _shared.py          Shared utilities: _read_tsv, _display_command, _plan_dict, _common_overrides
  provenance.py       RunLogger, PipelineProgressRecorder, TSV provenance writers
  tools.py            ToolRegistry, ToolSkill, GenericCommandSkill, SafeFormatDict, RunResult
  schemas.py          Canonical types: SampleInput, ExecutionPlan, PlanStep, SampleContext
  executor.py         GenericABIExecutor — step iteration, tool invocation, contract enforcement.
                      Supports sample-level parallel execution via ThreadPoolExecutor
                      (config.execution.parallel + config.execution.workers).
  dag.py              DAG inference engine — L1 (literature) / L2 (path) / L3 (validation)
  contracts/          WorkflowSpec, step contract enforcement, checksum chaining, assertion eval
  permissions.py      read_only / planning_write / execution levels
  diagnostics.py      Error taxonomy + DiagnosticHint + classify_exception
  interfaces.py       ABIPlugin, ABIDryRunPlugin, ABIInitializablePlugin protocols
  json_utils.py       JSON file/payload loading with ABIJSONError wrapping
  timeouts.py         Timeout parsing: parse_timeout_seconds, timeout_from_env_or_value
  resources.py        Resource discovery + auto-install: check_resources, setup_resources,
                      ResourceSpec with install_post hooks (e.g. makeblastdb)
  tables.py           StandardTableManager
  tool_descriptors.py Unified tool descriptor SSOT (3 format families, 7+ LLM providers)
  jobs/               HTTP Job Service (service, client, force-kill support)
  runtimes/           local, Nextflow, Snakemake, HPC runtimes
  exporters/          Nextflow DSL2 and Snakemake exporters
  mcp/                Optional MCP stdio server (exposed via ``abi-mcp``)
  skills/             Agent skill files → installed via ``abi install-skills``
  cli.py              Typer CLI (abi, abi-mcp, autoplasm, abi-sciplot entry points)

The abi.autoplasm package is a backward-compatible re-export shim that proxies to abi.plugins.metagenomic_plasmid._engine. Internal code should import from abi.plugins.metagenomic_plasmid._engine for the plasmid engine or from the ABI core modules for shared infrastructure.

Public SDK¶

Module	Purpose
`abi.interfaces`	`ABIPlugin`, `ABIDryRunPlugin`, `ABIInitializablePlugin` protocol classes
`abi.schemas`	`SampleInput`, `SampleContext`, `PlanStep`, `ExecutionPlan`
`abi.tools`	`ToolRegistry`, `ToolSkill`, `GenericCommandSkill`, `RunResult`
`abi.provenance`	`RunLogger`, `PipelineProgressRecorder`, TSV provenance writers
`abi.contracts.step_contract`	`ContractViolationError`, `validate_output_contract`, `evaluate_assertions`, checksum chaining
`abi.contracts`	`WorkflowSpec`, `WorkflowStepSpec`, `load_workflow_spec` — L1/L2/L3 workflow validation
`abi.dag`	`infer_dag`, `ABIDAG`, `StepBinding` — DAG inference with literature + path + validation layers
`abi.dag_planner`	`UniversalDAG`, `build_plan_from_dag`, `PathTemplateContext` — declarative plan generation, shared by all 7 plugins
`abi.tsv_mapping`	`TSVMapper`, `generate_rows` — YAML-driven TSV/JSON/log parsing with 3 source types
`abi.sciplot`	`FigureSpec`, `render_figure`, `validate_spec`, `lint_figure` — publication-grade Matplotlib figure compiler with 15 plot types
`abi.errors`	`ABIError`, `ConfigError`, `SampleSheetError`, `ToolError`
`abi.diagnostics`	Error taxonomy + `DiagnosticHint` + `classify_exception`
`abi.json_utils`	JSON file/payload loading with `ABIJSONError`
`abi.timeouts`	`parse_timeout_seconds`, `timeout_from_env_or_value`
`abi.tool_descriptors`	`ABI_AGENT_TOOLS`, `TOOL_ALIASES`, `export_openai_compatible`, `export_anthropic`, `export_gemini`, `PROVIDER_PROFILES`
`abi.testing`	`assert_plugin_contract`

Local Setup¶

pip install -e ".[dev]"

Useful checks:

ruff check src/ tests/
ruff format --check src/ tests/
mypy src/abi/ --ignore-missing-imports
pytest tests/ -v --tb=short

mypy is intentionally scoped to src/abi/; the bundled pipeline is covered by runtime tests and ruff first, with stricter typing left for later hardening.

Agent Integration Development¶

Treat integrations/<platform>/abi/ as the source of truth for platform-native assets. abi agent install copies those assets into the target project or user configuration, while abi agent doctor validates the installed skill, MCP configuration, command availability, and safe-server initialization without changing files.

When changing an integration:

Keep Claude Code, OpenCode, and Codex behavior aligned unless a platform has a documented constraint.
Run abi agent install <platform> --scope project --project-dir <tmp-dir> and the matching abi agent doctor command for all three platforms.
Run the native validator when the installed client exposes one: currently claude plugin validate for Claude Code. Codex releases without a local plugin validate subcommand rely on the manifest and asset regression tests.
Run python -m build, install the wheel with its [mcp] extra, and repeat install/doctor with the command from the clean wheel environment. Source-tree success alone does not prove the assets or MCP runtime were packaged correctly.
Keep .claude-plugin/plugin.json and .codex-plugin/plugin.json versions equal to project.version; scripts/check_release_identity.py enforces this.

integrations/ is a package and Docker build input. It must remain in the sdist, wheel, and every Docker /app context. Changes under it should receive a recommended manual Docker workflow validation before a container release; they do not trigger an automatic image build.

Runtime Contract Enforcement¶

The generic executor enforces the step-level contract embedded in each PlanStep.params["_contract"]. The DAG-driven planner copies this block from pipeline_dag.yaml, so the DAG remains the source of truth for outputs and runtime assertions.

Execution-time contract handling follows this order:

Verify upstream input checksums against provenance/checksums.json.
Run the external tool.
Resolve actual output files from output_dir when planned paths are abstract.
Validate output contracts and record output checksums.
Evaluate assertions against the resolved outputs.

Output validation supports file/directory existence, min_size, extensions, directory contains, directory/file min_files, FASTA min_contigs, JSON required_keys, and dotted JSON schema constraints.

Two executor details are intentional and should be preserved:

output_dir itself is not pre-created. Some assemblers and workflow tools fail if their output directory already exists. The executor only creates its parent directory and any unrelated file-output parents.
Actual-output resolution is deterministic and read-pair aware. If a tool writes S1_R1.clean.fastq.gz and S1_R2.clean.fastq.gz while the plan holds abstract paths such as S1.fastp.clean_read1, contract checks use the real R1/R2 files.

Regression coverage lives in tests/unit/test_executor.py and tests/unit/test_step_contract.py.

Runtime Assets¶

Small source assets are tracked:

config/
envs/ — generated from environments.yaml via scripts/emit_env_yamls.py
skills/ (inside src/abi/skills/ — bundled with the package, installed via abi install-skills)
plugins/
integrations/ — platform-native Claude Code, OpenCode, and Codex bundles
examples/
scripts/

Large or generated runtime state is ignored:

.mamba/
resources/
results/
log/
Nextflow and Snakemake work directories

Tool execution resolves environments via abi.config.resolved_mamba_root() with this priority:

Explicit CLI/API root
ABI_MAMBA_ROOT and MAMBA_ROOT_PREFIX
AUTOPLASM_MAMBA_ROOT (legacy compatibility)
Populated repository-compatible roots
Global Micromamba/Mamba/Conda metadata
Linux user data root

The env_name for each tool is resolved at runtime from environments.yaml. The current manifest declares 21 Conda environments and 99 tool-to-environment assignments; code and documentation should derive these inventories from the manifest rather than duplicating them.

Parallel Execution¶

GenericABIExecutor supports sample-level parallel execution via ThreadPoolExecutor when config.execution.parallel is set to true:

execution:
  parallel: true
  workers: 8

Samples run concurrently; steps within each sample remain serial respecting DAG topological order. Thread safety is maintained via threading.Lock for StandardTableManager, PipelineProgressRecorder, and RunLogger.

Agent Interfaces¶

ABIAgentInterface is the transport-neutral boundary. Keep CLI JSON (--output-json), MCP (abi-mcp), OpenAI descriptors (abi export-openai-tools), Skills (abi install-skills), abi dispatch, and Job Service behavior aligned with it.

Plugins join this interface through discovery and the shared analysis_type argument; they do not add transport-specific lifecycle tools.

See the Plugin Development Guide for the required Agent context, safety, result, and test contracts.

Execution must remain gated: abi run, abi_run, and Job Service execution submissions should return confirmation_required unless explicit confirmation is passed.

Agent-Facing Commands¶

Command	Purpose
`abi list-types --output-json`	Discover installed plugins
`abi query --type <plugin> --what stages`	Lightweight pipeline metadata (~50ms)
`abi export-agent-context --type <plugin>`	Machine-readable operating context
`abi doctor-agent --type <plugin>`	Human-readable operating guide
`abi check-resources --type <plugin>`	Check resource/database availability
`abi setup-resources --type <plugin> --confirm`	Auto-install/setup resources
`abi install-skills`	Install SKILL.md files to `~/.claude/skills/abi/`
`abi agent install <platform>`	Install a Claude Code, OpenCode, or Codex integration
`abi agent doctor <platform>`	Read-only validation of an installed integration
`abi export-openai-tools --type <plugin>`	OpenAI function-calling descriptors
`abi-mcp`	Start MCP stdio server

Python Agent API¶

import abi
abi.get_agent_guide()          # returns compact operating guide (str)
abi.list_plugins_summary()     # returns list[dict] of (analysis_type, name, description)

Known Gaps¶

MCP SDK >=2.0.0 — src/abi/mcp/server.py imports FastMCP from mcp.server.fastmcp, which was renamed to MCPServer and moved to mcp.server.mcpserver in MCP Python SDK v2.0.0 (2026-07-28 release). The dependency is pinned to mcp>=1.28,<2 in pyproject.toml to avoid the breaking change. Update the import and unpin the upper bound once the codebase is migrated to the v2 API.