Working notes from the design session that produced the Modules v2 split docs.
Covers the Artifacts API schema evolution, Execution Plans, filter model,
Action type, and staged changes to collections. Content has been absorbed into
hack/designs/modules-v2/ on the modules-v2 branch of dagger/dagger.
See: https://github.com/dagger/dagger/tree/modules-v2/hack/designs/modules-v2
The workspace-artifacts design was written as one integrated system covering
collections, addresses, verbs, plans, and provenance. Collections has since
spun out as a standalone primitive on the collections branch, with its own
design, implementation, and CLI integration. This report captures what that
spin-out changes for the artifacts design, and documents emerging design
directions from subsequent analysis.
Related documents:
- do-we-need-artifact-addresses.md
- workspace-artifacts.md (baseline design)
- collections.md (standalone collections design,
on
collectionsbranch)
The artifacts design should reference these, not redefine them.
Collection interface. The standalone design settled on keys/get with
@collection on the type. The engine synthesizes a public projection with
keys, list, get, subset, batch. The artifacts doc's +items/
+lookup/+key interface no longer exists.
Batch shadowing. Any function on the collection type beyond keys/get
is re-homed under batch. When the ModTree walks a collection node, batch
functions with the same name as item-level functions overwrite them. This is
the mechanism for "collection-level handlers pre-empt naive per-item
expansion." No planner rules needed.
Subset algebra. subset(keys) with formal laws. Selectors for
collection-backed items map directly to subset operations.
See do-we-need-artifact-addresses.md for the full analysis.
Typed collection filters collapse the need for a separate artifact address concept. The design direction is:
- No artifact addresses. No
(TYPE, cross-cutting key value)identity layer. No blessed key type list for artifact promotion. No artifact registry. - Artifacts are collection items. An artifact is a concrete thing in the workspace view — with a key, coordinates, and inspectable fields. The noun quality comes from the Artifact type in the API, not from a separate identity layer.
- Blessed scalar types still exist but their role is "parsing and rendering semantics for filter values." WorkspacePath values are resolved relative to the client's working directory. HTTPAddress values render in absolute form. This lives in the filter plumbing, not in an identity layer.
The artifacts design rests on three pillars. Collections is the foundation, already spun out and partially implemented.
Keyed sets, batch, subset, typed filters, dagger list. Already spun out to
its own branch. The foundation everything else builds on. See
collections.md.
Verbs compile to an inspectable Plan before execution. Three phases: selection, compilation, execution. The ModTree today collapses all three into one walk-and-execute pass. Plan separates them. This enables:
- inspecting what will run before running it (critical for
ship) - cross-artifact ordering (
check(B)beforecheck(A)when A references B) - cross-verb dependencies (
checkbeforeship) - test splitting (
--split 3divides a subset into worker buckets)
New verbs (ship, up) are handled as plan nodes as they arrive — they're
developed in separate PRs but their execution model is defined here.
Replaces CheckGroup. Transition path: CheckGroup → CheckGroup + collections (done) → Execution Plans (future).
Workspace-origin metadata from workspace.directory() / workspace.file()
reads. Path and git filtering at query time. The Workspace taint rule for
stored Workspace fields. The "verb methods can't take Workspace" rule forces
early materialization for precise provenance. Entirely orthogonal to
collections and plans.
The CLI is a generic client — no per-workspace codegen. The API is introspection-driven: the CLI discovers dimensions dynamically, generates flags, and constructs filter chains.
"""Entry point to the artifact system."""
extend type Workspace {
"""A filterable view of all artifacts in this workspace."""
artifacts: Artifacts!
}
"""
A scoped, filterable view over workspace artifacts.
Chainable: every filter returns a narrowed Artifacts.
"""
type Artifacts {
"""Narrow by a dimension, optionally to specific values."""
filterBy(dimension: String!, values: [String!]): Artifacts!
"""Narrow to artifacts reachable by check handlers."""
filterCheck: Artifacts!
"""Narrow to artifacts reachable by generate handlers."""
filterGenerate: Artifacts!
"""Narrow to artifacts reachable by ship handlers."""
filterShip: Artifacts!
"""Narrow to artifacts reachable by up handlers."""
filterUp: Artifacts!
"""Filterable dimensions available in the current scope. Static."""
dimensions: [ArtifactDimension!]!
"""Artifacts matching the current filters."""
items: [Artifact!]!
"""Union of available function names across all in-scope artifacts. Static."""
actions: [String!]!
"""Create an action targeting all in-scope artifacts."""
action(name: String!): Action!
"""Compile a check execution plan for the current scope."""
check: Plan!
"""Compile a generate execution plan for the current scope."""
generate: Plan!
"""Compile a ship execution plan for the current scope."""
ship: Plan!
"""Compile an up execution plan for the current scope."""
up: Plan!
}
"""A filterable axis of the artifact graph."""
type ArtifactDimension {
"""Filter name as used in CLI flags. Example: "go-module"."""
name: String!
"""Type of this dimension's keys. Determines parsing and rendering."""
keyType: TypeDef!
}
"""One artifact in the workspace."""
type Artifact {
"""This artifact's key within its immediate collection."""
key: String!
"""Ancestor collection coordinates tracing the path from root."""
ancestors: [ArtifactCoordinate!]!
"""Fields on the underlying object, for inspection."""
fields: [FieldValue!]!
"""Available function names on the underlying object."""
actions: [String!]!
"""Create an action targeting this single artifact."""
action(name: String!): Action!
}
"""A position along one dimension of the artifact graph."""
type ArtifactCoordinate {
"""Which dimension. Example: "go-module"."""
dimension: String!
"""The key value, rendered per its key type. Example: "./cmd/api"."""
value: String!
}
"""One field on an artifact's underlying object."""
type FieldValue {
"""Field name."""
name: String!
"""Field type."""
typeDef: TypeDef!
"""Raw value as JSON."""
json: JSON!
"""Human-readable rendering."""
display: String!
}
"""
A callable action: one or more artifacts + a function.
Actions are the building blocks of execution plans.
"""
type Action {
"""The artifacts this action targets."""
artifacts: [Artifact!]!
"""The function to call."""
name: String!
"""Type definition of the function, for introspection."""
function: Function
"""Actions that must complete before this one runs."""
after: [ActionID!]!
"""Execute this action."""
run: Void
}
"""
A compiled execution plan — a DAG of actions.
Each action is a function call on one or more artifacts.
Edges are "after" dependencies between actions.
Parallel execution is implicit: actions with no pending "after"
dependencies run concurrently.
NOTE FOR IMPLEMENTERS: Each action is backed by a DAGQL call chain
under the hood. The Action/Artifact API is a clean projection over
engine-internal DAGQL structures. Use existing engine-internal call
chain representations rather than building parallel ones.
"""
type Plan {
"""All actions in this plan."""
nodes: [Action!]!
"""
Execute the plan. Returns void on success, error on failure.
All other outputs (check results, deployment URLs, generated files)
are side effects observed through telemetry, TUI, or the filesystem.
"""
run: Void
}One filter model across all commands: --<dimension>=<value>, repeatable, no
comma-separated values. Named by item type (singular). Each flag points to
dagger list for discovery:
$ dagger check --help
--module=<name> Filter by module (see: dagger list modules)
--go-module=<name> Filter by go module (see: dagger list go-modules)
--go-test=<name> Filter by go test (see: dagger list go-tests)
The CLI generates flags from Artifacts.dimensions, parses user input into
filterBy chains, and calls verb methods or items on the result.
dagger list → workspace.artifacts.dimensions
dagger list modules → workspace.artifacts.filterBy("module").items
dagger list go-modules → workspace.artifacts.filterBy("go-module").items
dagger list go-tests → workspace.artifacts.filterBy("go-test").items
dagger list go-tests \
--go-module=./cmd/api → workspace.artifacts
.filterBy("go-test")
.filterBy("go-module", ["./cmd/api"])
.items
dagger check --module=go → workspace.artifacts
.filterBy("module", ["go"])
.check.run
dagger check --module=go \
--plan → workspace.artifacts
.filterBy("module", ["go"])
.check.nodes (display the plan)
dagger check --help → workspace.artifacts.filterCheck.dimensions
dagger check \
--go-test=Foo \
--go-test=Bar → workspace.artifacts
.filterBy("go-test", ["Foo", "Bar"])
.check.run
- Static (schema-only, no runtime cost): dimensions, verb-scoped dimensions, action names, filter flag generation
- Dynamic (reads collection keys): items, item counts, plan compilation, plan execution
- Every chained
filterBy/filterCheck/ etc. is AND - Multiple values within one
filterBycall is OR (list of values) filterBy("X")(no values) → select all items of dimension XfilterBy("X", ["a"]).filterBy("X", ["b"])→ intersection (AND) — empty if no overlapfilterCheck/filterShip/ etc. → narrow to verb-reachable paths
An Action bridges artifacts and functions:
Artifact.action("check")→ action on one artifactArtifacts.action("check")→ action on all in-scope artifacts (batch)Action.after→ DAG edges to other actionsAction.run→ execute just this action
Actions are the building blocks of Plans. A Plan is a DAG of Actions with
"after" edges. The engine compiles verb invocations (Artifacts.check) into
Plans by creating Actions with appropriate ordering.
Example — three tests, batched:
workspace.artifacts
.filterBy("go-test", ["TestFoo", "TestBar", "TestBaz"])
.action("check")
# → one Action targeting 3 artifacts
Example — three tests, per-item:
a1 = workspace.artifacts.filterBy("go-test", ["TestFoo"]).action("check")
a2 = workspace.artifacts.filterBy("go-test", ["TestBar"]).action("check")
a3 = workspace.artifacts.filterBy("go-test", ["TestBaz"]).action("check")
# → three Actions, each targeting 1 artifact
Example — DAG with ordering:
lint = artifacts.filterBy("go-module", ["./cmd/api"]).action("lint")
test = artifacts.filterBy("go-module", ["./cmd/api"]).action("test")
# test.after = [lint.id]
deploy = artifacts.filterBy("netlify-site", ["./docs"]).action("deploy")
# deploy.after = [test.id]
Rendered as a visual DAG:
lint(./cmd/api) ──▶ test(./cmd/api) ──▶ deploy(./docs)
Key design decisions:
- Plan = DAG of Actions. Each action is (artifacts, function) with "after" edges. Parallel is implicit — actions with no pending dependencies run concurrently.
- Always compiled.
dagger checkalways compiles a Plan, then executes it.--planstops before execution and displays the plan. - Engine compiles, CLI displays. The engine owns plan compilation
(
Artifacts.check → Plan). The CLI decides whether to callrunor display the plan nodes. - Plans materialize all implicit config. Workspace defaults, filter results, batch-vs-item decisions — all collapsed into concrete Actions. The plan is the "fully resolved" view. No more "what will this actually do?"
- No mini-VM. Plans are finite DAGs. No loops, conditionals, variables. "Query plan, not bytecode VM."
- The dang test. If a plan step can't be rendered as a readable public API action, it's the wrong layer. Every Action is backed by a DAGQL call chain that the user could understand and run themselves.
- Eager binding. Plan DAGs are built bottom-up: leaf actions first, then dependents referencing their IDs. All references are resolved at plan construction time via DAGQL's native ID system.
- Run returns void.
Plan.runandAction.runreturn void on success, error on failure. Outputs are side effects (telemetry, TUI, filesystem). Structured per-action results for CI are a known extension point, deferred.
Proposed modifications to the collections branch before or at merge time.
-
Rename collection filter flags from collection type to item type.
--go-modules→--go-module,--go-tests→--go-test, etc. Repeatable:--go-module=foo --go-module=bar. -
Forbid comma-separated values in collection filters. Each value gets its own flag instance.
--go-module=foo,baris an error (or treated as a literal key containing a comma). -
Add
dagger listas a verb-independent exploration surface:dagger list modules— catch-all, lists workspace modulesdagger list— lists available dimensions with key typesdagger list <collection>— lists items with ancestor columns- Uses the same filter flags as
check/generate
-
Evaluate retiring
--list-*flags. Ifdagger listships with collections, the--list-go-modules/--list-go-testsflags ondagger checkmay be redundant. -
Replace boolean type filters with
--module=filter.--go=true|falsebecomes--module=go. Same unified valued-filter model.
- Exact rules for automatic column disambiguation when non-collection fields create ambiguous paths.
- How cross-artifact reference ordering interacts with the filter system.
- Whether schema-path notation (
go:lint) should be deprecated in favor of typed filters. - Exact transition path from CheckGroup to Plan.
--mod(global module loader) vs--module(artifact dimension filter) — distinct long forms, but worth cleaning up--modas a true global flag now that workspace-plumbing moved module loading to the engine.- Structured per-action results for CI/programmatic consumers.
- Whether Action needs
withAfter(actions: [ActionID!]): Action!for building custom plans, or if dependencies are always set by the engine.
Active design exploration. Artifacts API and Plan schema are defined. Next: provenance, and implementation planning.