SmartPages AI Readiness Plan

Executive Summary

This document outlines a technical evaluation and implementation plan for making SmartPages "AI Ready" based on the requirements in the AI Readiness Evaluation document. After deep analysis of the codebase, I've identified the current state, gaps, and a phased approach to achieve AI-native content creation and transformation.

Recommendation: Incremental Refactor - The existing architecture has solid foundations that can be extended for AI-first workflows without a full platform migration.

Current State Analysis

Architecture Overview

┌─────────────────────────────────────────────────────────────────────┐
│                          SmartPages Architecture                     │
├─────────────────────────────────────────────────────────────────────┤
│  Frontend (client/pagedesigner)                                      │
│  ├── PageDesigner.tsx - Main component                               │
│  ├── store/slices/page_design.ts - Redux state (bands, sections)    │
│  ├── components/blocks/* - Block UI components                       │
│  └── types.ts - TypeScript definitions                               │
├─────────────────────────────────────────────────────────────────────┤
│  Backend (common/page_designs)                                       │
│  ├── page_design_helpers.rb - Core parsing/serialization            │
│  ├── bands/* - Band type implementations                            │
│  ├── smartpage/external_api/* - Public API for blocks               │
│  ├── smartpage/internal_api/* - Internal block representation       │
│  └── ai_response_formatter.rb - AI lesson generation formatter      │
├─────────────────────────────────────────────────────────────────────┤
│  Data Model                                                          │
│  ├── PageDesign { settings, bands[], sections[] }                   │
│  ├── Band { kind, settings { id, name, layout, values } }           │
│  └── Section { settings { band_count, title } }                     │
└─────────────────────────────────────────────────────────────────────┘

Current Block Types

Block Name	Internal Name	Key Capabilities
Header	`page_heading_block_1`	Text heading with level (h1-h4)
Banner	`page_banner_block_1`	Image with text overlay
Image	`page_image_block_1`	Single image display
Text	`text_block_1`	Rich HTML text content
Rich Text	`rich_text_block_1`	Advanced HTML editing
Content	`content_block_1`	Embedded items (specific, spot, recommended)
Divider	`page_divider_block_1`	Visual separator
Spacer	`page_spacer_block_1`	Vertical spacing
Navigation	`page_button_group_block_1`	Button links
Spots	`spots_block_1`	Spot listings
People	`people_block_1`	User cards

Existing AI Infrastructure

AI Response Formatter (ai_response_formatter.rb)
- Already converts LLM HTML output to valid page designs
- Creates bands, sections, questions programmatically
- Used by AI lesson generation
GenerateSmartPageJob (generate_smartpage_job.rb)
- Full pipeline for AI-generated lessons from source content
- Proves Level 1 (Generate) is achievable
External API (smartpage/external_api/)
- Simplified block representations for programmatic access
- Supports create and update operations
AI Text Transform (in progress)
- Hook/component for text refinement (shorten, elaborate, professional)
- Streaming support planned

AI Readiness Gap Analysis

Level 1: Generate ✅ Achievable with Current Architecture

Current State:

✅ Can create valid page designs programmatically (proven by lesson generation)
✅ Band templates provide default block configurations
✅ External API allows simplified block creation
⚠️ No semantic annotation for AI-generated blocks
⚠️ Limited style/branding intelligence

Key Code Path:

# common/page_designs/ai_response_formatter.rb
def create_text_content_band(content)
  text_hash = @band_templates_hash["text_block_1"].deep_dup
  PageDesignHelpers.set_value_at_path(text_hash, ["settings", "id"], Bands.generate_band_id)
  PageDesignHelpers.set_value_at_path(text_hash, ["settings", "html"], content)
  text_hash
end

Level 2: Understand ⚠️ Significant Gaps

Current State:

❌ No explicit semantic structure - blocks are a flat array
❌ Block relationships are implicit (only via layout.flow)
❌ No block grouping concept (e.g., Header + Content = logical unit)
❌ No page intent/purpose classification
⚠️ Embedded content requires deep reference resolution

Current Data Model:

// Flat band array with no semantic grouping
interface PageDesign {
  settings: { kind: string };
  bands: Block[];        // Flat array
  sections: Section[];   // Only titles, band counts
}

interface Block {
  kind: string;
  settings: {
    id: string;
    name: string;
    layout: { flow: 'new-row' | 'new-column' | 'continue-column' };
    // ... no semantic metadata
  };
}

Level 3: Modify ⚠️ Partial Support

Current State:

✅ Stable addressable IDs (settings.id per block)
✅ Block-level CRUD operations work
✅ External API supports merging with existing blocks
❌ No "what to preserve vs change" markers
❌ No multi-page update support
❌ No conflict resolution for concurrent edits

Implementation Plan

Phase 1: Semantic Layer Foundation (Effort: Medium)

Add semantic metadata to the data model without breaking existing functionality.

1.1 Block Semantic Metadata Schema

New fields to add to block settings:

interface BlockSemanticMetadata {
  // Semantic purpose of this block
  purpose?: 'title' | 'introduction' | 'body' | 'conclusion' | 'cta' | 
            'navigation' | 'reference' | 'visual' | 'spacing';
  
  // Logical grouping with other blocks
  group?: {
    id: string;           // Group identifier
    role: 'primary' | 'supporting' | 'decoration';
  };
  
  // AI modification hints
  ai?: {
    generated?: boolean;          // Was this AI-generated?
    locked?: boolean;             // Should AI avoid modifying this?
    preserveStyle?: boolean;      // Keep styling during updates
    lastModified?: string;        // ISO timestamp
    sourcePrompt?: string;        // Original generation prompt
  };
  
  // Brand/style classification
  style?: {
    category?: 'header' | 'body' | 'footer' | 'sidebar';
    brandElement?: boolean;       // Part of brand identity
  };
}

Implementation:

# common/page_designs/smartpage/types.rb - Add new types
module Smartpage
  module Types
    BlockPurpose = Types::String.optional.enum(
      "title", "introduction", "body", "conclusion", 
      "cta", "navigation", "reference", "visual", "spacing"
    )
    
    GroupRole = Types::String.optional.enum("primary", "supporting", "decoration")
  end
end

1.2 Page Intent Classification

Add page-level semantic metadata:

interface PageSemanticMetadata {
  intent?: 'informational' | 'tutorial' | 'sales' | 'onboarding' | 
           'reference' | 'landing' | 'interactive';
  
  audience?: string[];        // Target audience tags
  
  structure?: {
    hasHeader: boolean;
    hasFooter: boolean;
    sectionCount: number;
    primaryContentType: 'text' | 'media' | 'interactive' | 'mixed';
  };
}

1.3 Section Grouping Enhancement

interface EnhancedSection {
  settings: {
    band_count: number;
    title?: string;
    // New fields
    purpose?: 'introduction' | 'main' | 'conclusion' | 'appendix';
    blockGroups?: BlockGroup[];  // Logical groupings within section
  };
}

interface BlockGroup {
  id: string;
  bandIndexes: number[];      // Which bands belong to this group
  type: 'hero' | 'feature' | 'testimonial' | 'faq' | 'pricing' | 'contact' | 'custom';
  label?: string;             // Human-readable label
}

Phase 2: AI Understanding Layer (Effort: Medium-High)

Build services to analyze and understand existing pages.

2.1 Page Analyzer Service

# common/page_designs/ai/page_analyzer.rb
module AI
  class PageAnalyzer
    def initialize(user, page_design)
      @user = user
      @page_design = page_design
    end
    
    # Analyze page and return semantic representation
    def analyze
      {
        intent: classify_intent,
        structure: analyze_structure,
        block_groups: identify_block_groups,
        embedded_content: resolve_content_references,
        brand_elements: identify_brand_elements
      }
    end
    
    private
    
    def classify_intent
      # Use LLM to classify page intent based on content
      text_content = extract_all_text
      # Call AI service for classification
    end
    
    def identify_block_groups
      # Analyze layout flow to identify logical groupings
      groups = []
      current_group = []
      
      @page_design.bands.each_with_index do |band, idx|
        if band.settings.dig("layout", "flow") == "new-row" && current_group.any?
          groups << analyze_group(current_group)
          current_group = []
        end
        current_group << { band: band, index: idx }
      end
      
      groups << analyze_group(current_group) if current_group.any?
      groups
    end
    
    def resolve_content_references
      # Deep resolve all embedded content references
      refs = PageDesignHelpers.get_references(@page_design)
      refs.map do |ref_type, ref_id|
        resolve_reference(ref_type, ref_id)
      end
    end
  end
end

2.2 Block Relationship Analyzer

# common/page_designs/ai/block_relationship_analyzer.rb
module AI
  class BlockRelationshipAnalyzer
    COMMON_PATTERNS = {
      hero: { blocks: ['banner', 'header'], pattern: 'banner followed by header' },
      feature_card: { blocks: ['image', 'header', 'text'], pattern: 'image with header and text' },
      cta_section: { blocks: ['text', 'navigation'], pattern: 'text with navigation buttons' },
      content_preview: { blocks: ['header', 'content'], pattern: 'header with content block' }
    }.freeze
    
    def identify_patterns(bands)
      patterns = []
      window_size = 3
      
      bands.each_cons(window_size).with_index do |window, start_idx|
        COMMON_PATTERNS.each do |pattern_name, pattern_def|
          if matches_pattern?(window, pattern_def)
            patterns << {
              type: pattern_name,
              band_indexes: (start_idx...(start_idx + window.size)).to_a,
              confidence: calculate_confidence(window, pattern_def)
            }
          end
        end
      end
      
      patterns
    end
  end
end

2.3 Content Context Resolver

# common/page_designs/ai/content_context_resolver.rb
module AI
  class ContentContextResolver
    def resolve_page_context(page_design, user)
      context = {
        text_content: [],
        embedded_items: [],
        referenced_entities: []
      }
      
      page_design.bands.each do |band|
        case band.kind
        when Band::Kind::TEXT
          context[:text_content] << extract_text_with_structure(band)
        when Band::Kind::CONTENT
          context[:embedded_items].concat(resolve_content_items(band, user))
        end
      end
      
      context
    end
    
    private
    
    def resolve_content_items(band, user)
      source = band.settings.dig("source") || {}
      items = []
      
      case source["kind"]
      when "items"
        item_ids = source["items"]&.map { |i| i["id"] }
        items = EntityFetch.items(item_ids, treat_nil_as_missing: true)
      when "spot"
        spot_id = source.dig("spot", "spot_id")
        items = ItemQueries.for_spot(user.domain_id, spot_id, limit: 10)
      end
      
      items.map do |item|
        {
          id: item.id,
          title: item.title,
          description: item.description,
          content_type: item.content_kind,
          summary: get_item_summary(item)
        }
      end
    end
  end
end

Phase 3: AI Modification Layer (Effort: High)

Enable safe, intelligent page transformations.

3.1 Smart Update Service

# common/page_designs/ai/smart_update_service.rb
module AI
  class SmartUpdateService
    def initialize(user, page_design)
      @user = user
      @page_design = page_design
      @analyzer = PageAnalyzer.new(user, page_design)
    end
    
    # Apply AI modifications while preserving important elements
    def apply_modifications(instructions)
      analysis = @analyzer.analyze
      
      # Identify what should be preserved
      preserved_elements = identify_preserved_elements(analysis, instructions)
      
      # Generate modifications
      modifications = generate_modifications(instructions, analysis)
      
      # Apply modifications safely
      apply_safe_modifications(modifications, preserved_elements)
    end
    
    private
    
    def identify_preserved_elements(analysis, instructions)
      preserved = []
      
      @page_design.bands.each_with_index do |band, idx|
        should_preserve = 
          band.settings.dig("ai", "locked") ||
          band.settings.dig("style", "brandElement") ||
          analysis[:brand_elements].include?(idx)
        
        preserved << idx if should_preserve
      end
      
      preserved
    end
    
    def apply_safe_modifications(modifications, preserved)
      new_bands = @page_design.bands.deep_dup
      
      modifications.each do |mod|
        next if preserved.include?(mod[:band_index])
        
        case mod[:type]
        when :update_text
          apply_text_update(new_bands, mod)
        when :update_style
          apply_style_update(new_bands, mod)
        when :restructure
          apply_restructure(new_bands, mod)
        when :add_block
          new_bands.insert(mod[:position], mod[:block])
        when :remove_block
          new_bands.delete_at(mod[:band_index])
        end
      end
      
      # Recalculate sections and validate
      recalculate_sections(new_bands)
    end
  end
end

3.2 Diff and Merge Service

# common/page_designs/ai/diff_merge_service.rb
module AI
  class DiffMergeService
    def diff(original_design, modified_design)
      diffs = []
      
      # Compare bands
      original_design.bands.each_with_index do |orig_band, idx|
        mod_band = modified_design.bands[idx]
        
        if mod_band.nil?
          diffs << { type: :deleted, index: idx, band: orig_band }
        elsif band_changed?(orig_band, mod_band)
          diffs << { type: :modified, index: idx, original: orig_band, modified: mod_band }
        end
      end
      
      # Check for additions
      if modified_design.bands.length > original_design.bands.length
        modified_design.bands[original_design.bands.length..-1].each_with_index do |band, idx|
          diffs << { type: :added, index: original_design.bands.length + idx, band: band }
        end
      end
      
      diffs
    end
    
    def merge(base_design, ai_changes, user_changes)
      # Three-way merge with conflict resolution
      # Prioritize user changes over AI changes for conflicts
    end
  end
end

3.3 Batch Update Service (for bulk operations)

# common/page_designs/ai/batch_update_service.rb
module AI
  class BatchUpdateService
    def find_and_update(search_criteria, update_instructions, user)
      # Find matching pages
      matching_items = find_matching_pages(search_criteria, user)
      
      # Generate update plan
      update_plan = matching_items.map do |item|
        {
          item_id: item.id,
          current_design: item.page_design,
          proposed_changes: analyze_changes(item.page_design, update_instructions)
        }
      end
      
      # Return plan for review (don't auto-apply)
      update_plan
    end
    
    def apply_batch_updates(update_plan, user)
      results = []
      
      update_plan.each do |update|
        begin
          result = apply_single_update(update, user)
          results << { item_id: update[:item_id], status: :success, result: result }
        rescue => e
          results << { item_id: update[:item_id], status: :error, error: e.message }
        end
      end
      
      results
    end
  end
end

Phase 4: API Layer for AI Integration (Effort: Medium)

Create clean APIs for AI services to interact with SmartPages.

4.1 AI-Focused REST API

# api/controllers/smartpage_ai.rb
Api.controllers :smartpage_ai do
  
  # Analyze a page's semantic structure
  post :analyze, map: "/v1/smartpage/ai/analyze" do
    user = current_account
    halt 403 unless user&.feature_enabled?("smartpage_ai_features")
    
    input = JSON.parse(request.body.read)
    item = EntityFetch.item(input["item_id"])
    halt 404, "Item not found" unless item&.is_smartpage?
    halt 403 unless item.is_authorized?(user, :view)
    
    analyzer = AI::PageAnalyzer.new(user, item.page_design)
    json(analyzer.analyze)
  end
  
  # Generate a new page from instructions
  post :generate, map: "/v1/smartpage/ai/generate" do
    user = current_account
    halt 403 unless user&.feature_enabled?("smartpage_ai_features")
    
    input = JSON.parse(request.body.read)
    
    generator = AI::PageGenerator.new(user)
    page_design = generator.generate(
      instructions: input["instructions"],
      template_id: input["template_id"],
      context: input["context"]
    )
    
    json(page_design)
  end
  
  # Apply AI modifications to existing page
  post :modify, map: "/v1/smartpage/ai/modify" do
    user = current_account
    halt 403 unless user&.feature_enabled?("smartpage_ai_features")
    
    input = JSON.parse(request.body.read)
    item = EntityFetch.item(input["item_id"])
    halt 404, "Item not found" unless item&.is_smartpage?
    halt 403 unless item.is_authorized?(user, :update)
    
    service = AI::SmartUpdateService.new(user, item.page_design)
    result = service.apply_modifications(input["instructions"])
    
    json(result)
  end
  
  # Preview changes without applying
  post :preview_changes, map: "/v1/smartpage/ai/preview" do
    user = current_account
    halt 403 unless user&.feature_enabled?("smartpage_ai_features")
    
    input = JSON.parse(request.body.read)
    item = EntityFetch.item(input["item_id"])
    halt 404 unless item&.is_smartpage?
    
    service = AI::SmartUpdateService.new(user, item.page_design)
    preview = service.preview_modifications(input["instructions"])
    
    json({
      original: item.page_design,
      proposed: preview[:modified_design],
      diff: preview[:diff]
    })
  end
  
  # Batch search for pages matching criteria
  post :batch_search, map: "/v1/smartpage/ai/batch/search" do
    user = current_account
    halt 403 unless user&.feature_enabled?("smartpage_ai_batch")
    
    input = JSON.parse(request.body.read)
    
    service = AI::BatchUpdateService.new
    results = service.find_and_update(input["search"], input["instructions"], user)
    
    json(results)
  end
end

4.2 AI Page Schema for LLM Consumption

Create a simplified schema that LLMs can understand and generate:

// AI-friendly page schema for LLM prompts
interface AIPageSchema {
  title: string;
  intent: 'informational' | 'tutorial' | 'sales' | 'landing';
  
  sections: AISection[];
  
  style?: {
    primaryColor?: string;
    accentColor?: string;
    fontFamily?: string;
  };
}

interface AISection {
  title?: string;
  purpose: 'introduction' | 'main' | 'conclusion' | 'cta';
  
  blocks: AIBlock[];
}

interface AIBlock {
  type: 'header' | 'text' | 'image' | 'content' | 'navigation' | 'divider' | 'spacer';
  
  // Type-specific content
  content?: string;           // For text/header
  imageUrl?: string;          // For image
  items?: string[];           // For content (item IDs)
  buttons?: AIButton[];       // For navigation
  
  // Layout hints
  layout?: 'full-width' | 'half' | 'third';
}

interface AIButton {
  label: string;
  url: string;
  style?: 'primary' | 'secondary' | 'outline';
}

Phase 5: Frontend AI Integration (Effort: Medium)

5.1 AI Command Panel Component

// client/pagedesigner/components/AICommandPanel/AICommandPanel.tsx
interface AICommandPanelProps {
  onCommand: (command: AICommand) => void;
}

type AICommand = 
  | { type: 'refine_text'; blockId: string; preset: 'shorten' | 'elaborate' | 'professional' }
  | { type: 'generate_block'; prompt: string; position: number }
  | { type: 'restructure'; instructions: string }
  | { type: 'update_style'; style: Partial<PageStyle> }
  | { type: 'custom'; prompt: string };

const AICommandPanel = ({ onCommand }: AICommandPanelProps) => {
  // Command input with natural language processing
  // Quick action buttons for common operations
  // Preview of changes before applying
};

5.2 AI-Aware Block Selection

// client/pagedesigner/store/slices/ai_state.ts
interface AIState {
  analyzing: boolean;
  analysis: PageAnalysis | null;
  selectedBlockGroups: string[];
  pendingModifications: AIModification[];
  previewMode: boolean;
}

const aiSlice = createSlice({
  name: 'ai',
  initialState,
  reducers: {
    setAnalysis: (state, action) => { ... },
    selectBlockGroup: (state, action) => { ... },
    addPendingModification: (state, action) => { ... },
    applyModifications: (state) => { ... },
    discardModifications: (state) => { ... },
  }
});

Migration Strategy: The Bridge from Current to Future State

The Core Insight: No Migration Required

The key realization is that AI can infer semantic structure at runtime without requiring stored metadata. The semantic metadata proposed above is an optimization, not a requirement.

┌─────────────────────────────────────────────────────────────────────┐
│                    Two-Mode Operation                                │
├─────────────────────────────────────────────────────────────────────┤
│                                                                      │
│   Existing Page (no metadata)          Page with metadata            │
│   ┌─────────────────────┐              ┌─────────────────────┐      │
│   │ bands: [...]        │              │ bands: [...]        │      │
│   │ sections: [...]     │              │ sections: [...]     │      │
│   │ (no semantic data)  │              │ semantic: {...}     │      │
│   └─────────────────────┘              └─────────────────────┘      │
│            │                                    │                    │
│            ▼                                    ▼                    │
│   ┌─────────────────────┐              ┌─────────────────────┐      │
│   │ Runtime Analysis    │              │ Use Cached Analysis │      │
│   │ (AI inference)      │              │ (fast path)         │      │
│   └─────────────────────┘              └─────────────────────┘      │
│            │                                    │                    │
│            └──────────────┬─────────────────────┘                   │
│                           ▼                                          │
│                 ┌─────────────────────┐                             │
│                 │ Unified AI Features │                             │
│                 └─────────────────────┘                             │
└─────────────────────────────────────────────────────────────────────┘

Strategy 1: Runtime Inference Layer

Create an analyzer that works on ANY page, regardless of whether it has metadata:

# common/page_designs/ai/semantic_analyzer.rb
module AI
  class SemanticAnalyzer
    def initialize(page_design)
      @page_design = page_design
    end
    
    def analyze
      # Check if we have cached/stored analysis
      return @page_design.settings["semantic"] if has_stored_analysis?
      
      # Otherwise, infer it from the structure
      infer_semantic_structure
    end
    
    private
    
    def infer_semantic_structure
      {
        intent: infer_intent,
        block_groups: infer_block_groups,
        structure: analyze_structure,
        brand_elements: detect_brand_elements
      }
    end
    
    def infer_intent
      text_content = extract_all_text
      has_questions = @page_design.bands.any? { |b| b.kind == "Dynamic" && b.settings.dig("name")&.include?("question") }
      has_navigation = @page_design.bands.any? { |b| b.settings["name"] == "page_button_group_block_1" }
      
      return "lesson" if has_questions
      return "landing" if has_navigation && @page_design.bands.first&.kind == "Banner"
      "informational" # default
    end
    
    def infer_block_groups
      groups = []
      current_group = { blocks: [], start_index: 0 }
      
      @page_design.bands.each_with_index do |band, idx|
        flow = band.settings.dig("layout", "flow")
        
        if flow == "new-row" && current_group[:blocks].any? && is_group_boundary?(band, current_group)
          current_group[:type] = classify_group(current_group[:blocks])
          groups << current_group
          current_group = { blocks: [], start_index: idx }
        end
        
        current_group[:blocks] << { band: band, index: idx }
      end
      
      if current_group[:blocks].any?
        current_group[:type] = classify_group(current_group[:blocks])
        groups << current_group
      end
      
      groups
    end
    
    def classify_group(blocks)
      block_types = blocks.map { |b| b[:band].settings["name"] }
      
      return "hero" if block_types.include?("page_banner_block_1")
      return "content_section" if block_types.include?("content_block_1") && block_types.include?("page_heading_block_1")
      return "cta" if block_types.include?("page_button_group_block_1")
      return "text_content" if block_types.all? { |t| t == "text_block_1" }
      "mixed"
    end
  end
end

Strategy 2: Separate Semantic Cache (No Schema Migration)

Instead of modifying the PageDesign schema, store semantic analysis in a separate collection:

# New collection: page_design_semantic_cache
# This avoids any migration of existing page_design documents

class PageDesignSemanticCache
  include Mongoid::Document
  
  field :page_design_id, type: BSON::ObjectId
  field :analysis, type: Hash
  field :analyzed_at, type: Time
  field :page_design_hash, type: String  # For cache invalidation
  
  index({ page_design_id: 1 }, unique: true)
  
  def self.get_or_analyze(page_design, user)
    cached = find_by(page_design_id: page_design.id)
    
    # Check if cache is still valid
    if cached && cached.page_design_hash == compute_hash(page_design)
      return cached.analysis
    end
    
    # Analyze and cache
    analyzer = AI::SemanticAnalyzer.new(page_design)
    analysis = analyzer.analyze
    
    upsert(
      { page_design_id: page_design.id },
      { analysis: analysis, analyzed_at: Time.now, page_design_hash: compute_hash(page_design) }
    )
    
    analysis
  end
  
  def self.compute_hash(page_design)
    content = {
      band_count: page_design.bands.length,
      band_types: page_design.bands.map { |b| b.kind },
      section_count: page_design.sections.length
    }
    Digest::MD5.hexdigest(content.to_json)
  end
end

Strategy 3: Progressive Enrichment

When AI analyzes a page, optionally persist the analysis for faster future access:

module AI
  class SemanticAnalyzer
    def analyze(persist: false)
      result = has_stored_analysis? ? stored_analysis : infer_semantic_structure
      
      # Optionally store for future use
      persist_analysis_async(result) if persist && !has_stored_analysis?
      
      result
    end
  end
end

Strategy 4: Optional User-Managed Metadata

The only fields that go INTO the page_design document are user-controlled:

Field	Storage Location	When Added
`ai.locked`	In page_design (block settings)	User explicitly locks a block
`ai.generated`	In page_design (block settings)	AI creates a new block
`style.brandElement`	In page_design (block settings)	User marks as brand element
Semantic analysis	Separate cache collection	On-demand, auto-invalidated

The User Experience

From the user's perspective, AI features "just work" on any page:

User clicks "AI: Restructure this page"
           │
           ▼
┌──────────────────────────────────────┐
│ 1. Load page_design (existing)       │
│ 2. Check semantic cache              │
│    - Hit? Use cached analysis        │
│    - Miss? Run inference (1-2 sec)   │
│ 3. AI generates modifications        │
│ 4. Show preview to user              │
│ 5. User confirms → Save changes      │
│ 6. Cache analysis for next time      │
└──────────────────────────────────────┘

Background Pre-warming (Optional)

For performance, pre-analyze frequently accessed pages:

class SemanticAnalysisJob
  def self.enqueue_high_value_pages(domain_id)
    high_value_items = ItemQueries.smartpages_by_engagement(domain_id, limit: 1000)
    
    high_value_items.each do |item|
      next if PageDesignSemanticCache.fresh?(item.page_design.id)
      perform_async(item.page_design.id)
    end
  end
end

Migration Summary

Concern	Solution
Existing pages	Work as-is, no migration needed
AI features on old pages	Runtime inference (slightly slower first time)
AI features on new pages	Same runtime inference, with optional metadata
Performance	Separate cache collection, auto-invalidated on change
Schema changes	Optional fields only, added on-demand
Rollback safety	If AI features disabled, pages still work

Key principle: Treat semantic analysis as a computed property, not stored data.

Implementation Priority & Effort

Phase	Description	Effort	Priority	Dependencies
1.1	Block Semantic Metadata	Medium	P0	None
1.2	Page Intent Classification	Low	P0	1.1
1.3	Section Grouping Enhancement	Medium	P1	1.1
2.1	Page Analyzer Service	Medium	P0	1.x
2.2	Block Relationship Analyzer	Medium	P1	2.1
2.3	Content Context Resolver	High	P1	2.1
3.1	Smart Update Service	High	P0	2.x
3.2	Diff and Merge Service	Medium	P1	3.1
3.3	Batch Update Service	High	P2	3.x
4.1	AI REST API	Medium	P0	2.x, 3.x
4.2	AI Page Schema	Low	P1	4.1
5.1	AI Command Panel	Medium	P1	4.x
5.2	AI-Aware Block Selection	Medium	P2	5.1

Effort Legend:

Low: < 1 week
Medium: 1-2 weeks
High: 3-4 weeks

Key Technical Decisions

1. Backward Compatibility

All new semantic metadata fields are optional. Existing pages will continue to work without modification. AI features can be progressively enabled.

2. Data Migration Strategy

No forced migration of existing pages
Semantic metadata populated on-demand when AI features are used
Background job to pre-analyze high-value pages

3. AI Service Architecture

AI analysis runs server-side (Ruby + LLM calls)
Results cached for performance
Frontend receives structured analysis, not raw LLM output

4. Conflict Resolution

User changes always take precedence over AI changes
Three-way merge for concurrent editing
Preview/confirm workflow for all AI modifications

Risk Assessment

Risk	Likelihood	Impact	Mitigation
Performance degradation from analysis	Medium	Medium	Caching, async processing
AI makes destructive changes	Low	High	Preview mode, undo support, locked blocks
Schema migration complexity	Medium	Medium	Optional fields, gradual rollout
LLM output inconsistency	High	Medium	Structured prompts, validation layer
User confusion with AI features	Medium	Low	Progressive disclosure, clear UI

Success Metrics

Level 1 (Generate)

AI can generate valid pages from natural language prompts
Generated pages pass all validation rules
Brand styles are correctly applied

Level 2 (Understand)

AI can accurately classify page intent (>90% accuracy)
Block groupings match human perception (>85% accuracy)
Embedded content is fully resolved and summarized

Level 3 (Modify)

AI modifications don't break page structure
Locked/brand elements are preserved
Batch updates complete without errors (<1% failure rate)

Next Steps

Immediate (This Sprint)
- Review and approve schema additions (Phase 1.1)
- Create feature flags for AI features
- Prototype Page Analyzer service
Short Term (Next 2 Sprints)
- Implement semantic metadata schema
- Build Page Analyzer MVP
- Create AI REST API endpoints
Medium Term (Next Quarter)
- Complete Level 2 (Understand) capabilities
- Build Smart Update Service
- Frontend AI command integration
Long Term
- Batch update capabilities
- Multi-page consistency checking
- Brand guidelines enforcement

Block Schema Discovery for AI

The Problem

For AI to generate valid blocks, it needs to know:

What block types exist
What settings each block accepts
Valid values for each setting (enums, ranges)
Default values
Human-readable labels for UI

Current Schema Sources

Backend (Ruby Dry::Struct) - Source of truth for validation:

# common/page_designs/smartpage/internal_api/blocks/header_block.rb
attribute :values do
  attribute :header_type, Types::String.default("text").enum("text", "text_bg", "text_divider")
  attribute :text?, Types::String.optional
  attribute :heading_level?, Types::String.default("h2").enum("h1", "h2", "h3", "h4")
  attribute :text_align?, Types::String.default("center").enum("left", "center", "right")
  # ...
end

Frontend (TypeScript Valibot) - Source of truth for UI:

// client/pagedesigner/components/blocks/Header/types.ts
export const headerSettingsSchema = v.object({
  header_type: v.optional(enum_(HeaderType), HeaderType.Text),
  heading_level: v.optional(enum_(HeadingLevelType), HeadingLevelType.H2),
  text_align: v.optional(enum_(TextAlign), TextAlign.Center),
  text: v.optional(v.string(), ''),
  // ...
});

UI Options - Human-readable labels:

// client/pagedesigner/components/blocks/Header/options.ts
export const headerTypeOptions = [
  { label: 'Text', value: HeaderType.Text },
  { label: 'Text with Background', value: HeaderType.TextBg },
  { label: 'Text with Divider', value: HeaderType.TextDivider },
];

Solution: Auto-Generated AI Schema Registry

Generate a unified JSON Schema from the TypeScript sources that AI can consume:

// client/pagedesigner/ai/schema-registry.ts
import * as v from 'valibot';
import { toJsonSchema } from '@valibot/to-json-schema';
import { headerSettingsSchema } from '~/pagedesigner/components/blocks/Header/types';
import { headerTypeOptions, headerLevelOptions } from '~/pagedesigner/components/blocks/Header/options';
import { bannerSchema } from '~/pagedesigner/components/blocks/Banner/types';
// ... import all block schemas

export interface AIBlockSchema {
  name: string;
  displayName: string;
  description: string;
  jsonSchema: object;
  options: Record<string, { label: string; value: string; icon?: string }[]>;
  defaults: Record<string, any>;
  examples: object[];
}

export const AI_BLOCK_SCHEMAS: Record<string, AIBlockSchema> = {
  'page_heading_block_1': {
    name: 'page_heading_block_1',
    displayName: 'Header',
    description: 'A heading block with customizable style, level, and alignment',
    jsonSchema: toJsonSchema(headerSettingsSchema),
    options: {
      header_type: headerTypeOptions,
      heading_level: headerLevelOptions,
      text_align: textAlignOptions,
    },
    defaults: v.getDefaults(headerSettingsSchema),
    examples: [
      { header_type: 'text', text: 'Welcome', heading_level: 'h1', text_align: 'center' },
      { header_type: 'text_bg', text: 'Features', heading_level: 'h2', background_color: '#007bff' },
    ]
  },
  
  'page_banner_block_1': {
    name: 'page_banner_block_1',
    displayName: 'Banner',
    description: 'A hero banner with image background, text overlay, and optional CTA button',
    jsonSchema: toJsonSchema(bannerSchema),
    options: { /* ... */ },
    defaults: { type: 'overlaybanner', title_size: 'h1' },
    examples: [
      { 
        type: 'overlaybanner', 
        title: 'Welcome to Our Platform',
        description: 'Get started today',
        bg_image: 'https://example.com/hero.jpg',
        cta_text: 'Learn More',
        cta_link: '/about'
      },
    ]
  },
  
  // ... all other blocks
};

// Generate LLM-friendly prompt context
export function generateBlockSchemaPrompt(): string {
  let prompt = '# Available SmartPage Blocks\n\n';
  
  for (const [name, schema] of Object.entries(AI_BLOCK_SCHEMAS)) {
    prompt += `## ${schema.displayName} (${name})\n`;
    prompt += `${schema.description}\n\n`;
    prompt += `### Settings:\n`;
    prompt += '```json\n' + JSON.stringify(schema.jsonSchema, null, 2) + '\n```\n\n';
    prompt += `### Example:\n`;
    prompt += '```json\n' + JSON.stringify(schema.examples[0], null, 2) + '\n```\n\n';
  }
  
  return prompt;
}

Backend Schema Validation

The backend validates AI-generated blocks against Dry::Struct:

# common/page_designs/ai/block_validator.rb
module AI
  class BlockValidator
    BLOCK_SCHEMAS = {
      'page_heading_block_1' => Smartpage::InternalApi::Blocks::HeaderBlock,
      'page_banner_block_1' => Smartpage::InternalApi::Blocks::BannerBlock,
      'text_block_1' => Smartpage::InternalApi::Blocks::TextBlock,
      'content_block_1' => Smartpage::InternalApi::Blocks::ContentBlock,
      # ...
    }.freeze
    
    def validate(block_data)
      block_name = block_data.dig('settings', 'name')
      schema_class = BLOCK_SCHEMAS[block_name]
      
      return { valid: false, error: "Unknown block type: #{block_name}" } unless schema_class
      
      begin
        schema_class.new(block_data)
        { valid: true }
      rescue Dry::Struct::Error => e
        { valid: false, error: e.message, details: extract_validation_errors(e) }
      end
    end
    
    def coerce_and_fix(block_data)
      # Attempt to fix common AI mistakes
      block_name = block_data.dig('settings', 'name')
      schema_class = BLOCK_SCHEMAS[block_name]
      
      # Apply defaults for missing optional fields
      # Convert string numbers to integers where needed
      # Normalize color values
      
      schema_class.new(block_data).to_h
    end
  end
end

Runtime Schema API

Expose schemas via API for AI services:

# api/controllers/smartpage_ai.rb
get :block_schemas, map: "/v1/smartpage/ai/schemas" do
  user = current_account
  halt 403 unless user&.feature_enabled?("smartpage_ai_features")
  
  schemas = AI_BLOCK_SCHEMAS.transform_values do |schema|
    {
      name: schema[:name],
      display_name: schema[:display_name],
      description: schema[:description],
      settings: schema[:json_schema],
      defaults: schema[:defaults],
      examples: schema[:examples]
    }
  end
  
  json(schemas)
end

get :block_schema, map: "/v1/smartpage/ai/schemas/:block_name" do
  user = current_account
  halt 403 unless user&.feature_enabled?("smartpage_ai_features")
  
  schema = AI_BLOCK_SCHEMAS[params[:block_name]]
  halt 404, "Block not found" unless schema
  
  json(schema)
end

LLM Prompt Engineering

When generating pages, include schema context:

# common/page_designs/ai/page_generator.rb
module AI
  class PageGenerator
    def generate(instructions:, context: {})
      prompt = build_prompt(instructions, context)
      
      response = AI::Chat.new.call(prompt)
      
      # Parse and validate response
      page_design = parse_response(response)
      validate_and_fix_blocks(page_design)
      
      page_design
    end
    
    private
    
    def build_prompt(instructions, context)
      <<~PROMPT
        You are a SmartPage generator. Create a valid page design based on the user's instructions.
        
        #{generate_block_schema_prompt}
        
        ## Page Structure
        A page consists of:
        - settings: { kind: "Page" }
        - sections: Array of { settings: { band_count: N, title: "Section Name" } }
        - bands: Array of blocks (each block has kind, settings with id, name, layout, values)
        
        ## Layout Rules
        - layout.flow can be: "new-row" (starts new row), "new-column" (new column in row), "continue-column" (stack in column)
        - Maximum 3 columns per row
        - Each block needs a unique id (use UUID format)
        
        ## User Instructions
        #{instructions}
        
        ## Additional Context
        #{context.to_json}
        
        Generate a valid JSON page design:
      PROMPT
    end
  end
end

Schema Sync Strategy

Keep frontend and backend schemas in sync:

┌─────────────────────────────────────────────────────────────────┐
│                    Schema Source of Truth                        │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│   TypeScript (Valibot)              Ruby (Dry::Struct)          │
│   ┌─────────────────┐               ┌─────────────────┐         │
│   │ types.ts        │ ◄──────────── │ *_block.rb      │         │
│   │ options.ts      │    Manual     │                 │         │
│   └────────┬────────┘    Sync       └────────┬────────┘         │
│            │                                  │                  │
│            ▼                                  ▼                  │
│   ┌─────────────────┐               ┌─────────────────┐         │
│   │ JSON Schema     │               │ Validation      │         │
│   │ (generated)     │               │ (runtime)       │         │
│   └────────┬────────┘               └────────┬────────┘         │
│            │                                  │                  │
│            └──────────────┬──────────────────┘                  │
│                           ▼                                      │
│                  ┌─────────────────┐                            │
│                  │ AI Schema API   │                            │
│                  │ /ai/schemas     │                            │
│                  └─────────────────┘                            │
└─────────────────────────────────────────────────────────────────┘

Future: Generated Types

Consider generating both from a single source:

# schemas/blocks/header.yaml (Single source of truth)
name: page_heading_block_1
displayName: Header
description: A heading block with customizable style
settings:
  header_type:
    type: enum
    values: [text, text_bg, text_divider]
    default: text
    labels:
      text: "Text"
      text_bg: "Text with Background"
      text_divider: "Text with Divider"
  heading_level:
    type: enum
    values: [h1, h2, h3, h4]
    default: h2
  text:
    type: string
    optional: true
  text_align:
    type: enum
    values: [left, center, right]
    default: center

Then generate TypeScript and Ruby from this YAML:

scripts/generate-block-types.ts → TypeScript types
scripts/generate-block-types.rb → Ruby Dry::Struct

This ensures schemas never drift apart.

Appendix: Existing Code References

Key Files for AI Integration

Backend:
├── common/page_designs/ai_response_formatter.rb      # Existing AI formatter
├── common/page_designs/page_design_helpers.rb        # Core parsing logic
├── common/page_designs/smartpage/external_api/       # Public API layer
├── common/page_designs/band_template_helper.rb       # Block templates
├── common/jobs/items/generate_smartpage_job.rb       # AI generation job
└── api/controllers/page_designs.rb                   # REST endpoints

Frontend:
├── client/pagedesigner/store/slices/page_design.ts   # Redux state
├── client/pagedesigner/types.ts                      # TypeScript types
├── client/pagedesigner/components/blocks/            # Block components
└── client/pagedesigner/README.FE_API.md              # Frontend API docs

Sample Page Design JSON Structure

{
  "settings": { "kind": "Page" },
  "sections": [
    { "settings": { "band_count": 3, "title": "Introduction" } },
    { "settings": { "band_count": 5, "title": "Main Content" } }
  ],
  "bands": [
    {
      "kind": "Text",
      "settings": {
        "id": "abc123",
        "name": "page_heading_block_1",
        "html": "<h1>Welcome</h1>",
        "layout": { "flow": "new-row" }
      }
    }
  ]
}

Document created: Feb 6, 2026 Author: AI Analysis based on codebase review

azizpunjani/SMARTPAGES_AI_READINESS_PLAN.md

SmartPages AI Readiness Plan

Executive Summary

Current State Analysis

Architecture Overview

Current Block Types

Existing AI Infrastructure

AI Readiness Gap Analysis

Level 1: Generate ✅ Achievable with Current Architecture

Level 2: Understand ⚠️ Significant Gaps

Level 3: Modify ⚠️ Partial Support

Implementation Plan

Phase 1: Semantic Layer Foundation (Effort: Medium)

1.1 Block Semantic Metadata Schema

1.2 Page Intent Classification

1.3 Section Grouping Enhancement

Phase 2: AI Understanding Layer (Effort: Medium-High)

2.1 Page Analyzer Service

2.2 Block Relationship Analyzer

2.3 Content Context Resolver

Phase 3: AI Modification Layer (Effort: High)

3.1 Smart Update Service

3.2 Diff and Merge Service

3.3 Batch Update Service (for bulk operations)

Phase 4: API Layer for AI Integration (Effort: Medium)

4.1 AI-Focused REST API

4.2 AI Page Schema for LLM Consumption

Phase 5: Frontend AI Integration (Effort: Medium)

5.1 AI Command Panel Component

5.2 AI-Aware Block Selection

Migration Strategy: The Bridge from Current to Future State

The Core Insight: No Migration Required

Strategy 1: Runtime Inference Layer

Strategy 2: Separate Semantic Cache (No Schema Migration)

Strategy 3: Progressive Enrichment

Strategy 4: Optional User-Managed Metadata

The User Experience

Background Pre-warming (Optional)

Migration Summary

Implementation Priority & Effort

Key Technical Decisions

1. Backward Compatibility

2. Data Migration Strategy

3. AI Service Architecture

4. Conflict Resolution

Risk Assessment

Success Metrics

Level 1 (Generate)

Level 2 (Understand)

Level 3 (Modify)

Next Steps

Block Schema Discovery for AI

The Problem

Current Schema Sources

Solution: Auto-Generated AI Schema Registry

Backend Schema Validation

Runtime Schema API

LLM Prompt Engineering

Schema Sync Strategy

Future: Generated Types

Appendix: Existing Code References

Key Files for AI Integration

Sample Page Design JSON Structure