feat(presets): Composition strategies (prepend, append, wrap) for templates, commands, and scripts (#2133)

* fix: rebase onto upstream/main, resolve conflicts with PR #2189

upstream/main merged PR #2189 (wrap-only strategy) which overlaps with
our comprehensive composition strategies (prepend/append/wrap). Resolved
conflicts keeping our implementation as source of truth:

- README: keep our future considerations (composition is now fully
  implemented, not a future item)
- presets.py: keep our composition architecture (_reconcile_composed_commands,
  collect_all_layers, resolve_content) while preserving #2189's
  _substitute_core_template which is used by agents.py for skill
  generation
- tests: keep both test sets (our composition tests + #2189's wrap
  tests), removed TestReplayWrapsForCommand and
  TestInstallRemoveWrapLifecycle which test the superseded
  _replay_wraps_for_command API; our composition tests cover equivalent
  scenarios
- Restored missing _unregister_commands call in remove() that was lost
  during #2189 merge

* fix: re-create skill directory in _reconcile_skills after removal

After _unregister_skills removes a skill directory, _register_skills
skips writing because the dir no longer passes the is_dir() check.
Fix by ensuring the skill subdirectory exists before calling
_register_skills so the next winning preset's content gets registered.

Fixes the Claude E2E failure where removing a top-priority override
preset left skill-based agents without any SKILL.md file.

* fix: address twenty-third round of Copilot PR review feedback

- Protect reconciliation in remove(): wrap _reconcile_composed_commands
  and _reconcile_skills in try/except so failures emit a warning instead
  of leaving the project in an inconsistent state
- Protect reconciliation in install(): same pattern for post-install
  reconciliation so partial installs don't lack cleanup
- Inherit scripts/agent_scripts from base frontmatter: when composing
  commands, merge scripts and agent_scripts keys from the base command's
  frontmatter into the top layer's frontmatter if missing, preventing
  composed commands from losing required script references
- Add tier-5 bundled core fallback to collect_all_layers(): check the
  bundled core_pack (wheel) or repo-root templates (source checkout) when
  .specify/templates/ doesn't contain the core file, matching resolve()'s
  tier-5 fallback so composition can always find a base layer

* fix: address twenty-fourth round of Copilot PR review feedback

- Use yaml.safe_load for frontmatter parsing in resolve_content instead
  of CommandRegistrar.parse_frontmatter which uses naive find('---',3);
  strip strategy key from final frontmatter to prevent leaking internal
  composition directives into rendered agent command files
- Filter _reconcile_skills to specific commands: use _FilteredManifest
  wrapper so only the commands being reconciled get their skills updated,
  preventing accidental overwrites of other commands' skills that may be
  owned by higher-priority presets

* fix: address twenty-fifth round of Copilot PR review feedback

- Support legacy command-frontmatter strategy: when preset.yml doesn't
  declare a strategy, check the command file's YAML frontmatter for
  strategy: wrap as a fallback so legacy wrap presets participate in
  composition and multi-preset chaining
- Guard skill dir creation in _reconcile_skills: only re-create the
  skill directory if the skill was previously managed (listed in some
  preset's registered_skills), avoiding creation of new skill dirs
  that _register_skills would normally skip

* fix: add explanatory comment to empty except in legacy frontmatter parsing

* fix: address twenty-sixth round of Copilot PR review feedback

- Unregister stale commands when composition fails: when resolve_content
  returns None during reconciliation (base layer removed), unregister
  the command from non-skill agents and emit a warning
- Load extension aliases during reconciliation: _register_command_from_path
  now checks extension.yml for aliases when the winning layer is an
  extension, so alias files are restored after preset removal
- Use line-based fence detection for legacy frontmatter strategy fallback:
  scan for --- on its own line instead of split('---',2) to avoid
  mis-parsing YAML values containing ---

* fix: address twenty-seventh round of Copilot PR review feedback

- Handle non-preset winners in _reconcile_skills: when the winning
  layer is core/extension/project-override, restore skills via
  _unregister_skills so skill-based agents stay consistent with the
  priority stack
- Update base_frontmatter_text on replace layers: when a higher-priority
  replace layer occurs during composition, update both top and base
  frontmatter so scripts/agent_scripts inheritance reflects the
  effective base beneath the top composed layer

* fix: address twenty-eighth round of Copilot PR review feedback

- Parse only interior lines in _parse_fm_yaml: use lines[1:-1] instead
  of filtering all --- lines, preventing corruption when YAML values
  contain a line that is exactly ---
- Omit empty frontmatter: skip re-rendering when top_fm is empty dict
  to avoid emitting ---/{}/--- for intentionally empty frontmatter
- Update scaffold wrap comment: mention both {CORE_TEMPLATE} and
  $CORE_SCRIPT placeholders for templates/commands vs scripts
- Clarify shell composition scope in ARCHITECTURE.md: note that bash/PS1
  resolve_template_content only handles templates; command/script
  composition is handled by the Python resolver

* fix: address twenty-ninth round of Copilot PR review feedback

- Fix TestCollectAllLayers docstring: reference collect_all_layers()
- Add default/unknown strategy handling in bash/PS1 composition: error
  on unrecognized strategy values instead of silently skipping
- Fix comment: .composed/ is a persistent dir, not temporary
- Fix comment: legacy fallback checks all valid strategies, not just wrap
- Cache PresetRegistry in _reconcile_skills: build presets_by_priority
  once instead of constructing registry per-command

* fix: address thirtieth round of Copilot PR review feedback

- Guard legacy frontmatter fallback: only check command file frontmatter
  for strategy when the manifest entry doesn't explicitly include the
  strategy key, preventing override of manifest-declared strategies
- Document rollback limitation: note that mid-registration failures may
  leave orphaned agent command files since partial progress isn't
  captured by the local vars

* fix: handle project override skills and extension context in reconciliation

* fix: add comment to empty except in extension registration fallback

* fix: filter extension commands in reconciliation and fix type annotation

* fix: filter extension commands from post-install reconciliation

Apply the same extension-installed check used in _register_commands to
the reconciliation command list, preventing reconciliation from
registering commands for extensions that are not installed.

* fix: skip convention fallback for explicit file paths and add stem fallback to tier-5

When a preset manifest provides an explicit file path that does not
exist, skip the convention-based fallback to avoid masking typos.
Also add speckit.<stem> to <stem>.md fallback in tier-5 bundled/source
core lookup for consistency with tier-4.

* fix: scan past non-replace layers to find base in resolve_content

The base-finding scan now skips non-replace layers below a replace
layer instead of stopping at the first non-replace. This fixes the
case where a low-priority append/prepend layer sits below a replace
that should serve as the base for composition.

* fix: add context_note to non-skill agent registration for extensions

Add context_note parameter to register_commands_for_non_skill_agents
and pass extension name/id during reconciliation so rendered command
files preserve the extension-specific context markers.

* fix: Optional type, rollback safety, and override skill restoration

- Fix context_note type to Optional[str]
- Wrap shutil.rmtree in try/except during install rollback
- Separate override-backed skills from core/extension in _reconcile_skills

* fix: align bash/PS1 base-finding with Python resolver

Rewrite bash and PowerShell composition loops to find the effective
base replace layer first (scanning bottom-up, skipping non-replace
layers below it), then compose only from the base upward. This
prevents evaluation of irrelevant lower layers (e.g. a wrap with
no placeholder below a replace) and matches resolve_content behavior.

* fix: PS1 no-python warning, integration hook for override skills, alias cleanup

- Warn when no Python 3 found in PS1 and presets use composition strategies
- Apply post_process_skill_content integration hook when restoring
  override-backed skills so agent-specific flags are preserved
- Unregister command aliases alongside primary name when composition
  fails to prevent orphaned alias files

* fix: include aliases in removed_cmd_names during preset removal

Read aliases from preset manifest before deleting pack_dir so alias
command files are included in unregistration and reconciliation.

* fix: add comment to empty except in alias extraction during removal

* fix: scan top-down for effective base in all resolvers

Change base-finding to scan from highest priority downward to find the
nearest replace layer, then compose only layers above it. Prevents
evaluation of irrelevant lower layers (e.g. a wrap without placeholder
below a higher-priority replace) across Python, bash, and PowerShell.

* fix: align CLI composition chain display with top-down base-finding

Show only contributing layers (base and above) in preset resolve
output, matching resolve_content top-down semantics. Layers below
the effective base are omitted since they do not contribute.

* fix: guard corrupted registry entries and make manifest authoritative

- Add isinstance(meta, dict) guard in bash registry parsing so corrupted
  entries are skipped instead of breaking priority ordering
- Only use convention-based file lookup when the manifest does not list
  the requested template, making preset.yml authoritative and preventing
  stray on-disk files from creating unintended layers

* fix: align resolve() with manifest file paths and match extension context_note

- Update resolve() preset tier to consult manifest file paths before
  convention-based lookup, matching collect_all_layers behavior
- Use exact extension context_note format matching extensions.CommandRegistrar
- Update test to declare template in manifest (authoritative manifest)

* revert: restore resolve() convention-based behavior for backwards compatibility

resolve() is the existing public API used by shell scripts and other
callers. Changing it to manifest-authoritative breaks backward compat
for presets that rely on convention-based file lookup. Only the new
collect_all_layers/resolve_content path uses manifest-authoritative
logic.

* fix: only pre-compose when this preset is the top composing layer

Skip composition in _register_commands when a higher-priority replace
layer already wins for the command. Register the raw file instead and
let reconciliation write the correct final content.

* fix: deduplicate PyYAML warnings and use self.registry in reconciliation

- Emit PyYAML-missing warning once per function call in bash/PS1 instead
  of per-preset to avoid spamming stderr
- Use self.registry.list_by_priority() in reconciliation methods instead
  of constructing new PresetRegistry instances to avoid redundant I/O
  and potential consistency issues

* fix: document strategy handling consistency between layers and registrar

Composed output already strips strategy from frontmatter (resolve_content
pops it). Raw file registration preserves legacy frontmatter strategy
for backward compat; reconciliation corrects the final state.

* fix: correct stale comments for alias tracking and base-finding algorithm

* security: validate manifest file paths in bash/PowerShell resolvers

Reject absolute paths and parent directory traversal (..) in the
manifest-declared file field before joining with the preset directory.
Matches the Python-side validation in PresetManifest._validate().

---------

Co-authored-by: Manfred Riem <15701806+mnriem@users.noreply.github.com>