Skip to content

End-to-end validation of v2 extract/install across multiple AI tools #82

Description

@troylar

Description

Validate the v2 extract → install pipeline end-to-end with real projects and real AI tools. Create integration tests and a manual test matrix that exercises extraction from real-world project configs and installation into multiple tool formats.

Context

v2 (0.12.0) shipped with AI-powered extraction and installation, but testing has been primarily unit-level with mocked LLM responses. We need to verify:

  1. Extraction produces valid practice declarations from real project configs
  2. Installation correctly translates practices to each tool's native format
  3. AI adaptation produces sensible merges with existing rules
  4. File-copy fallback (--no-ai) works correctly for all tools
  5. Round-trip fidelity: extract from tool A, install to tool B, content is meaningful

Acceptance Criteria

  • Integration test suite that runs extract → install for at least 3 tool pairs (e.g., Claude→Cursor, Cursor→Kiro, Claude→Copilot)
  • Test with real project configs (use devsync's own .claude/rules/ as source)
  • Verify file placement, extensions, and content format for each target tool
  • Test --no-ai mode produces valid output for all tools
  • Test v1 backward compatibility (ai-config-kit-package.yaml packages still install)
  • Document a manual test matrix for pre-release validation
  • Tests pass: invoke test-unit && invoke test-integration

Test Matrix

Source Target Mode Status
Claude Code Cursor AI
Claude Code Kiro AI
Claude Code Copilot AI
Claude Code Cursor --no-ai
Cursor Claude Code AI
v1 package Claude Code file-copy
v1 package Cursor file-copy

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions