ARCHIVE: Workspace Artifacts Tracking

tracking-final.md

ARCHIVE: Workspace Artifacts Tracking

Running tracking doc for the original workspace-artifacts design session. Superseded by the Modules v2 design docs.

See: https://github.com/dagger/dagger/tree/modules-v2/hack/designs/modules-v2

---

Workspace Artifacts Tracking

Working note for active design discussion around workspace-artifacts.md.

This file is intentionally conversational. The main proposal should stay crisp; this file keeps the theme tree, checkpoints, and burst log that explain how we got there.

Theme Tree

1. Addressing model
2. Artifact scope
3. Discovery semantics
4. Function participation in discovery
5. Origin model
6. Relationship graph
7. CLI views

Current Checkpoints

Theme 1: Addressing Model

Current direction:

• Keep dagger call as the procedural escape hatch.
• Separate artifact addressing from object traversal.
• Artifact addresses are typed and collection-derived.
• Object traversal remains structural and follows rooted objects, fields, and collection hops.
• Treat workspace provenance filtering as separate from identity.

Still open:

• exact rooted schema projection if we pursue multiple constructors as a separate track
• exact CLI split between artifact and object targeting
• exact artifact URI syntax
• exact structural traversal syntax

Theme 2: Artifact Scope

Current direction:

• Treat artifact semantically as an addressed object surfaced by a collection.
• Top-level rooted singletons are object-graph roots, not artifacts.
• Artifacts may still live deep in the object graph if a nested collection addresses them.
• Verbs and artifacts are decoupled:
  • artifacts may have verbs or not
  • verbs do not define artifact status

Still open:

• whether the word "artifact" remains the best user-facing term
• whether collection objects themselves ever deserve artifact-like UX

Theme 3/4: Discovery and Function Participation

Current direction:

• Multiple rooted constructors are now a separate parallel track, not a prerequisite for artifact addressing.
• Collections are orthogonal to rooting.
• The current rooted module object is enough to expose the object graph for artifact discovery.
• Collections discovered anywhere in that graph can surface addressable artifacts.
• Default artifact enumeration projects addressed collection items out of the walked graph.
• Ordinary functions are not automatic discovery edges.
• Collections can also carry verb handlers that pre-empt naive item-by-item expansion.

Still open:

• exact root schema projection if we pursue the multiple-constructor track
• how much deep graph inspection should be surfaced in CLI

Theme 5: Origin Model

Current direction:

• workspace provenance is the main v1 origin/filter model
• provenance comes from workspace API reads
• Workspace taints objects with root-path provenance at /

Still open:

• whether v1 needs a broader origin concept for non-workspace-backed resources

Theme 6: Relationship Graph

Current direction:

• field references are useful as object-to-object structure
• they should not automatically be treated as dependency/supply-chain semantics

Still open:

• whether and how to surface graph/debug/inspect UX

Theme 7: CLI Views

Current direction:

• default UX should stay small:
  • find rooted thing
  • filter by provenance
  • run verb
• deeper graph traversal should be explicit power-user mode

Still open:

• exact dagger call additive syntax for object/artifact targeting
• exact artifact list rendering rules

Discussion Bursts

Burst 1

Topic:

• Why artifacts matter beyond filtered checks.

Themes:

• 2, 5, 6, 7

Outcome:

• Artifacts are about surfacing a module's dynamic project model, not just replacing check trees.

Burst 2

Topic:

• Dogfood reality check.

Themes:

• 3, 4, 5

Outcome:

• Existing dogfood mixes precise materialized inputs with lazy workspace reads; strict construction-only provenance is not current reality.

Burst 3

Topic:

• Provenance, Workspace taint, and verb constraints.

Themes:

• 4, 5

Outcome:

• Provenance comes from workspace API origins; storing Workspace is allowed but makes the object root at /; verbs must not take Workspace.

Burst 4

Topic:

• CLI display shape.

Themes:

• 5, 7

Outcome:

• Provenance-first display is stronger than inventing fake shared object fields like SOURCE.

Burst 5

Topic:

• Function-path versus object selection.

Themes:

• 1, 3, 4

Outcome:

• Keep dagger call as the procedural path; object/artifact addressing should become the primary user-facing model for the new system.

Burst 6

Topic:

• +key and workspace-path-like natural lookup.

Themes:

• 1

Outcome:

• +key is a natural lookup key, not true canonical identity.

Burst 7

Topic:

• Field/function distinction as discovery boundary.

Themes:

• 2, 3, 4

Outcome:

• Strong intermediate direction was "roots + fields only", with functions as leaves.

Burst 8

Topic:

• Dynamic checks precedent.

Themes:

• 1, 3, 7

Outcome:

• Old dynamic checks already needed natural addressing for list items; this strongly informed the later collection model.

Burst 9

Topic:

• Workspace-bounded artifacts versus remote inventories.

Themes:

• 2, 3, 5

Outcome:

• Remote inventories are a poor fit for eager whole-graph discovery; workspace/project-bounded models are a better v1 target.

Burst 10

Topic:

• Canonical discovery rule versus current implementation.

Themes:

• 3

Outcome:

• State the design semantically, not in terms of current DAGQL plumbing.

Burst 11

Topic:

• Discovery roots versus user-facing root artifacts.

Themes:

• 2, 3, 7

Outcome:

• Root objects seed discovery, but not every root should appear by default in artifact UX.

Burst 12

Topic:

• Artifact as a higher-level role over objects.

Themes:

• 2, 7

Outcome:

• Important reframing: artifact may be to object what verb is to function.

Burst 13

Topic:

• Locator families.

Themes:

• 1, 7

Outcome:

• We initially split exact object locators, graph traversal locators, and value/scalar locators.

Burst 14

Topic:

• Multipart natural keys.

Themes:

• 1

Outcome:

• Natural identity may be composite even if v1 keeps lookup simple.

Burst 15

Topic:

• URI as uniqueness canary.

Themes:

• 1

Outcome:

• The key test is whether identity can be canonicalized at all, not whether it looks pretty.

Burst 16

Topic:

• Collection key versus module-level lookup.

Themes:

• 1, 3

Outcome:

• Naming an item inside a collection is not automatically the same as giving it first-class module lookup.

Burst 17

Topic:

• Reverse lookup.

Themes:

• 1, 7

Outcome:

• CLI listing needs a way to print the best canonical address for a discovered object.

Burst 18

Topic:

• First-class collections.

Themes:

• 1, 2, 3, 7

Outcome:

• Collections became the key bridge:
  • +items
  • +lookup
  • +key
  • uniqueness at collection scope

Burst 19

Topic:

• Canonical locator versus display alias.

Themes:

• 1, 7

Outcome:

• Path-first display can coexist with a stronger canonical locator underneath.

Burst 20

Topic:

• Opaque URI-like locators.

Themes:

• 1, 7

Outcome:

• URI-like forms fit opaque/external identities better than workspace-native ones.

Burst 21

Topic:

• How root collections get registered.

Themes:

• 1, 3

Outcome:

• Rooting collections should piggyback on a generalized root-constructor model rather than special-casing fields on one main root object.

Burst 22

Topic:

• Backward compatibility of generalized constructors.

Themes:

• 1, 3, 7

Outcome:

• The model can stay additive if extra roots are explicit opt-ins.

Burst 23

Topic:

• Canonical namespaced schema as a superset of flat Query.

Themes:

• 1, 3, 7

Outcome:

• The canonical model likely needs a better root schema than "module namespace flattened onto Query".

Burst 24

Topic:

• Incremental flat-Query alternative.

Themes:

• 1, 3, 7

Outcome:

• Flat extra root functions are viable as a migration step but not a strong long-term foundation.

Burst 25

Topic:

• new-prefixed compat constructors.

Themes:

• 1, 3, 7

Outcome:

• newFooSite() is a plausible compatibility projection, not a canonical model.

Burst 26

Topic:

• Decision on multiple rooted constructors.

Themes:

• 3, 7

Outcome:

• Multiple rooted constructors are required; schema projection must evolve; collections remain orthogonal.

Burst 27

Topic:

• Object-wrapper root model.

Themes:

• 1, 3, 7

Outcome:

• Grouping root acquisition by object type wrapper looked cleaner than separate new and load trees.

Burst 28

Topic:

• Collections remove the need for generic natural-key root acquisition.

Themes:

• 1, 3, 7

Outcome:

• new(...) and load(...) remain root acquisition modes; natural-key lookup belongs on the collection.

Burst 29

Topic:

• Constructors versus provisioning.

Themes:

• 1, 3, 7

Outcome:

• Constructors should create values/specs/handles, not provision real resources.

Burst 30

Topic:

• Multi-constructors and lower object graph.

Themes:

• 1, 3, 7

Outcome:

• Multi-constructors change the root set, not the basic traversal semantics.

Burst 31

Topic:

• Literal DAGQL retrieval versus user-facing address.

Themes:

• 1, 7

Outcome:

• Artifact-facing addresses should not expose all lower-level DAGQL boilerplate.

Burst 32

Topic:

• Backward-compatible invocation, rooted discovery, duplicate root paths.

Themes:

• 1, 3, 7

Outcome:

• dagger call should remain additive; a rooted collection and a sibling field re-exposing it is usually a smell.

Burst 33

Topic:

• Typed locators and shorthand paths.

Themes:

• 1, 7

Outcome:

• Bare ./docs should be shorthand or display alias; exact targeting needs a typed locator.

Burst 34

Topic:

• If artifacts need unique user-facing addresses, not every walked descendant can be one.

Themes:

• 1, 2, 3, 7

Outcome:

• This forced the object-versus-artifact split to become explicit.

Burst 35

Topic:

• Artifact status comes from rooted access, not incidental discovery path.

Themes:

• 1, 2, 3

Outcome:

• A deep object can still be an artifact, but only by virtue of a separate canonical rooted access path.

Burst 36

Topic:

• Verb-bearing object is not automatically an artifact.

Themes:

• 2, 3, 7

Outcome:

• Verbs belong to the object layer; artifact status belongs to the rooted address layer.

Burst 37

Topic:

• Provenance path filtering versus workspace-path keys.

Themes:

• 1, 5, 7

Outcome:

• These overlap a lot but must stay conceptually distinct:
  • key = identity
  • provenance = filter

Burst 38

Topic:

• Collapse artifact and graph identity.

Themes:

• 1, 3, 5, 7

Outcome:

• Strong simplification:
  • one rooted identity model
  • optional traversal for deeper objects
  • provenance filtering as the only truly separate path concept

Burst 39

Topic:

• Root endpoints and singleton roots.

Themes:

• 1, 3, 7

Outcome:

• The latest direction moved away from "root endpoint" naming and toward a cleaner rule: the scheme should name a type, and the suffix should be that type's canonical rooted address.