alumni_lookup

Sub-Phase 9.4: Onboarding Metrics & Analytics

Champion Portal Development Sub-Phase 9.4

Status: ✅ Complete
Completed: January 2026
Estimated Effort: 1 week
Prerequisites: Sub-Phases 9.1, 9.2 Complete

Related Documents:

README.md — Phase 9 Overview

9.1-onboarding-flow.md — Onboarding flow

9.2-seeded-questions.md — Seeded questions system

Completion Summary

Phase 9.4 delivered comprehensive onboarding analytics:

What Was Implemented

Database & Model:

Migration: db/migrate/*_create_cp_onboarding_events.rb
Model: app/models/cp/onboarding_event.rb with all event types and recording class methods

Services (OnboardingInsights Module):

app/services/onboarding_insights/base_service.rb — Caching, period filtering
app/services/onboarding_insights/onboarding_metrics_service.rb — Signup funnel, wizard completion metrics
app/services/onboarding_insights/seeded_question_metrics_service.rb — Community suggestion accept/decline metrics

Controller & Views:

app/controllers/champions/insights_controller.rb — onboarding and seeded_questions actions
app/views/champions/insights/onboarding.html.erb — Signup funnel and wizard metrics dashboard
app/views/champions/insights/seeded_questions.html.erb — Community suggestion metrics dashboard

Event Instrumentation (5 controllers):

cp/registrations_controller.rb — signup_email_submitted
cp/confirmations_controller.rb — signup_email_verified, signup_password_created
cp/omniauth_callbacks_controller.rb — signup_oauth_started, signup_oauth_completed
cp/profile_wizard_controller.rb — wizard_started, step_viewed, step_completed, step_skipped, wizard_completed
cp/community_suggestions_controller.rb — community_accepted, community_declined

Navigation:

Sidebar links in Champions Admin (portal_admin+ required)
Routes: /champions/insights/onboarding, /champions/insights/seeded_questions

Test Results

Service Tests: 27 runs, 98 assertions, 0 failures
Controller Tests: 18 runs, 38 assertions, 0 failures
Full Suite: 2897 runs, 7417 assertions, 0 failures

Files Created/Modified

File	Purpose
`db/migrate/*_create_cp_onboarding_events.rb`	Events table migration
`app/models/cp/onboarding_event.rb`	Event model with recording methods
`app/services/onboarding_insights/base_service.rb`	Base service with caching
`app/services/onboarding_insights/onboarding_metrics_service.rb`	Onboarding funnel metrics
`app/services/onboarding_insights/seeded_question_metrics_service.rb`	Seeded question metrics
`app/controllers/champions/insights_controller.rb`	Admin dashboard controller
`app/views/champions/insights/onboarding.html.erb`	Onboarding metrics view
`app/views/champions/insights/seeded_questions.html.erb`	Seeded questions view
`test/services/onboarding_insights/onboarding_metrics_service_test.rb`	Service tests (432 lines)
`test/services/onboarding_insights/seeded_question_metrics_service_test.rb`	Service tests (341 lines)
`test/controllers/champions/insights_controller_test.rb`	Controller tests (158 lines)

Overview
Key Metrics
Data Model
Instrumentation
Admin Dashboard
Implementation Plan
Testing Requirements

1. Overview

Sub-Phase 9.4 provides metrics and analytics to measure onboarding effectiveness and identify improvement opportunities. This enables data-driven refinement of the onboarding experience.

Goals

Measure funnel performance — Track where Champions drop off during onboarding
Track early engagement — Monitor first-week activity patterns
Evaluate seeded questions — Identify which questions drive most engagement
Surface actionable insights — Dashboard for Engagement Team

Scope

In Scope	Out of Scope
Wizard funnel metrics	Real-time dashboards
First-week engagement tracking	Predictive analytics
Seeded question performance	ML-based recommendations
Admin metrics dashboard	Champion-facing analytics
Activity event instrumentation	Third-party analytics integration

2. Key Metrics

Goal: Understand where potential Champions drop off before even starting the wizard

IMPORTANT: The onboarding journey starts at signup, not the wizard. We need to capture:

Email → Verify email → Create password (email flow)

OAuth button click → OAuth completion (OAuth flow)

Metric	Definition	Target
Email Submission Rate	Emails submitted / Signup page views	60%
Email Verification Rate	Verified / Submitted	70%
Password Creation Rate	Passwords created / Verified	95%
OAuth Start Rate	OAuth clicks / Signup page views	40%
OAuth Completion Rate	OAuth completed / OAuth started	85%
Overall Signup Completion	Account created / Signup page views	65%
Email Verification Delay	Average time from submit to verify	< 10 min

Pre-Wizard Drop-off Points:

Stage	Expected Drop-off	Notes
Signup page view → Email submit	40%	Page bounce, changed mind
Email submit → Verify click	30%	Email not received, forgot
Verify click → Password create	5%	Rare - committed at this point
OAuth start → OAuth complete	15%	Google errors, cancelled

2.1 Onboarding Funnel

Goal: 80% wizard completion rate

Metric	Definition	Target
Wizard Start Rate	Champions who start wizard / Total new signups	95%
Step Completion Rate	Champions completing each step / Started	Varies
Wizard Completion Rate	Champions completing all steps / Started	80%
Drop-off by Step	Where Champions abandon wizard	N/A
Time to Complete	Average time from start to finish	< 10 min

Step-by-step targets:

Step	Target Completion
help_find_you	98%
confirm_education	95%
location	95%
champion_role	92%
profession	90%
photo	85%
bio	85%
affinities	82%
join_communities (NEW)	80%

2.2 Community Joining

Goal: Average 2+ communities joined during onboarding

Metric	Definition	Target
Avg Communities Joined	Total joins / Champions completing wizard	2.5+
Suggestion Accept Rate	Accepted / Shown	40%
Browse-to-Join Rate	Champions who browse and join additional	30%
Join by Community Type	Distribution across types	Balanced

2.3 First-Week Engagement

Goal: 60% engagement in first week

Metric	Definition	Target
First-Week Active Rate	Champions with any activity / Completed onboarding	60%
Discussion Participation	Created post or comment	30%
Reaction Given	Any reaction in first week	40%
Return Visits	Unique days active in first week	3+
Time to First Post	Average days from signup to first post	< 5 days

2.4 Seeded Question Performance

Goal: Seeded questions generate 2x engagement vs organic

Metric	Definition	Target
Reply Rate	Replies per exposure	0.5+
Reaction Rate	Reactions per exposure	1.5+
New Champion Reply Rate	% first-week Champions who reply	15%
Question Effectiveness Score	Composite of above metrics	Benchmark

3. Data Model

3.1 Onboarding Events Table

Track granular wizard progression:

# Migration
create_table :cp_onboarding_events do |t|
  t.references :cp_champion, null: false, foreign_key: true, index: true
  t.string :event_type, null: false
  t.string :step
  t.jsonb :metadata, default: {}
  t.string :session_id
  t.timestamps
end

add_index :cp_onboarding_events, :event_type
add_index :cp_onboarding_events, [:cp_champion_id, :event_type]
add_index :cp_onboarding_events, :created_at

Event Types:

Event	Description	Metadata
`signup_email_submitted`	Email entered on signup form	`{}`
`signup_email_verified`	Clicked email verification link	`{ minutes_to_verify: 2.5 }`
`signup_password_created`	Password set (email flow)	`{ minutes_from_verify: 1.2 }`
`signup_oauth_started`	Clicked OAuth button (Google)	`{ provider: "google" }`
`signup_oauth_completed`	OAuth flow completed	`{ provider: "google" }`
`wizard_started`	Champion begins wizard	`{}`
`step_viewed`	Champion views a step	`{ step: "photo" }`
`step_completed`	Champion completes a step	`{ step: "photo", duration_seconds: 45 }`
`step_skipped`	Champion skips optional step	`{ step: "bio" }`
`wizard_completed`	Champion finishes all steps	`{ duration_minutes: 8.5 }`
`wizard_abandoned`	Champion leaves mid-wizard	`{ last_step: "profession" }`
`community_suggested`	Suggestion shown	`{ community_id: 123, type: "major" }`
`community_accepted`	Suggestion accepted	`{ community_id: 123, type: "major" }`
`community_declined`	Suggestion declined	`{ community_id: 123, type: "major" }`
`community_browsed`	Champion opens browse	`{}`
`community_joined_browse`	Joined from browse	`{ community_id: 456 }`

3.2 Model

# app/models/cp/onboarding_event.rb
module Cp
  class OnboardingEvent < ApplicationRecord
    belongs_to :champion, class_name: 'Cp::Champion', foreign_key: 'cp_champion_id'

    EVENT_TYPES = %w[
      signup_email_submitted
      signup_email_verified
      signup_password_created
      signup_oauth_started
      signup_oauth_completed
      wizard_started
      step_viewed
      step_completed
      step_skipped
      wizard_completed
      wizard_abandoned
      community_suggested
      community_accepted
      community_declined
      community_browsed
      community_joined_browse
    ].freeze

    validates :event_type, presence: true, inclusion: { in: EVENT_TYPES }

    # Scopes
    scope :signup_events, -> { where(event_type: %w[signup_email_submitted signup_email_verified signup_password_created signup_oauth_started signup_oauth_completed]) }
    scope :wizard_events, -> { where(event_type: %w[wizard_started step_viewed step_completed step_skipped wizard_completed wizard_abandoned]) }
    scope :community_events, -> { where(event_type: %w[community_suggested community_accepted community_declined community_browsed community_joined_browse]) }
    scope :in_period, ->(start_date, end_date) { where(created_at: start_date..end_date) }
  end
end

3.3 Metrics Service

# app/services/cp/onboarding_metrics_service.rb
module Cp
  class OnboardingMetricsService
    def initialize(start_date: 30.days.ago, end_date: Time.current)
      @start_date = start_date
      @end_date = end_date
    end

    def signup_funnel_metrics
      {
        emails_submitted: signup_event_count('signup_email_submitted'),
        emails_verified: signup_event_count('signup_email_verified'),
        passwords_created: signup_event_count('signup_password_created'),
        oauth_started: signup_event_count('signup_oauth_started'),
        oauth_completed: signup_event_count('signup_oauth_completed'),
        email_verification_rate: email_verification_rate,
        oauth_completion_rate: oauth_completion_rate,
        avg_verification_delay_minutes: avg_verification_delay_minutes
      }
    end

    def funnel_metrics
      {
        wizard_starts: wizard_starts_count,
        wizard_completions: wizard_completions_count,
        completion_rate: completion_rate,
        step_completion_rates: step_completion_rates,
        avg_duration_minutes: avg_duration_minutes,
        drop_off_by_step: drop_off_by_step
      }
    end

    def community_metrics
      {
        avg_communities_joined: avg_communities_joined,
        suggestion_accept_rate: suggestion_accept_rate,
        browse_to_join_rate: browse_to_join_rate,
        joins_by_type: joins_by_type
      }
    end

    def engagement_metrics
      {
        first_week_active_rate: first_week_active_rate,
        discussion_participation_rate: discussion_participation_rate,
        avg_return_visits: avg_return_visits,
        time_to_first_post: time_to_first_post
      }
    end

    private

    # Signup funnel helpers (pre-wizard)
    def signup_event_count(event_type)
      OnboardingEvent.signup_events
        .where(event_type: event_type)
        .in_period(@start_date, @end_date)
        .select(:cp_champion_id)
        .distinct
        .count
    end

    def email_verification_rate
      submitted = signup_event_count('signup_email_submitted')
      return 0 if submitted.zero?
      (signup_event_count('signup_email_verified').to_f / submitted * 100).round(1)
    end

    def oauth_completion_rate
      started = signup_event_count('signup_oauth_started')
      return 0 if started.zero?
      (signup_event_count('signup_oauth_completed').to_f / started * 100).round(1)
    end

    def avg_verification_delay_minutes
      # Find Champions who have both submitted and verified events
      verified_events = OnboardingEvent.signup_events
        .where(event_type: 'signup_email_verified')
        .in_period(@start_date, @end_date)
      
      return 0 if verified_events.empty?

      delays = verified_events.map do |verified|
        submitted = OnboardingEvent.signup_events
          .where(cp_champion_id: verified.cp_champion_id, event_type: 'signup_email_submitted')
          .where('created_at < ?', verified.created_at)
          .order(created_at: :desc)
          .first
        
        next nil unless submitted
        (verified.created_at - submitted.created_at) / 60.0 # minutes
      end.compact

      return 0 if delays.empty?
      (delays.sum / delays.count).round(1)
    end

    # Wizard funnel helpers
    def wizard_starts_count
      OnboardingEvent.wizard_events
        .where(event_type: 'wizard_started')
        .in_period(@start_date, @end_date)
        .select(:cp_champion_id)
        .distinct
        .count
    end

    def wizard_completions_count
      OnboardingEvent.wizard_events
        .where(event_type: 'wizard_completed')
        .in_period(@start_date, @end_date)
        .select(:cp_champion_id)
        .distinct
        .count
    end

    def completion_rate
      return 0 if wizard_starts_count.zero?
      (wizard_completions_count.to_f / wizard_starts_count * 100).round(1)
    end

    def step_completion_rates
      steps = ProfileWizardController::STEPS
      steps.each_with_object({}) do |step, rates|
        completed = OnboardingEvent.where(event_type: 'step_completed')
          .where("metadata->>'step' = ?", step)
          .in_period(@start_date, @end_date)
          .select(:cp_champion_id)
          .distinct
          .count
        rates[step] = (completed.to_f / wizard_starts_count * 100).round(1) if wizard_starts_count > 0
      end
    end

    def drop_off_by_step
      # Calculate where Champions abandoned
      abandoned = OnboardingEvent.where(event_type: 'wizard_abandoned')
        .in_period(@start_date, @end_date)
        .group("metadata->>'last_step'")
        .count
      abandoned.transform_keys { |k| k || 'unknown' }
    end

    def avg_duration_minutes
      events = OnboardingEvent.where(event_type: 'wizard_completed')
        .in_period(@start_date, @end_date)
        .where("metadata->>'duration_minutes' IS NOT NULL")
      
      return 0 if events.empty?
      events.average("(metadata->>'duration_minutes')::float").to_f.round(1)
    end

    def avg_communities_joined
      completed_ids = OnboardingEvent.where(event_type: 'wizard_completed')
        .in_period(@start_date, @end_date)
        .pluck(:cp_champion_id)
      
      return 0 if completed_ids.empty?
      
      total_joins = ChampionCommunity.where(champion_id: completed_ids).count
      (total_joins.to_f / completed_ids.count).round(1)
    end

    def suggestion_accept_rate
      suggested = OnboardingEvent.community_events
        .where(event_type: 'community_suggested')
        .in_period(@start_date, @end_date)
        .count
      
      accepted = OnboardingEvent.community_events
        .where(event_type: 'community_accepted')
        .in_period(@start_date, @end_date)
        .count
      
      return 0 if suggested.zero?
      (accepted.to_f / suggested * 100).round(1)
    end

    def first_week_active_rate
      # Champions who completed onboarding in period
      completed = Champion.joins(:onboarding_events)
        .where(cp_onboarding_events: { event_type: 'wizard_completed' })
        .where(cp_onboarding_events: { created_at: @start_date..@end_date })
        .distinct
      
      active = completed.joins(:activity_events)
        .where(cp_activity_events: { created_at: @start_date..@end_date })
        .distinct
      
      return 0 if completed.count.zero?
      (active.count.to_f / completed.count * 100).round(1)
    end

    # ... additional private methods
  end
end

3.4 Seeded Question Metrics Service

# app/services/cp/seeded_question_metrics_service.rb
module Cp
  class SeededQuestionMetricsService
    def initialize(question = nil, start_date: 30.days.ago, end_date: Time.current)
      @question = question
      @start_date = start_date
      @end_date = end_date
    end

    def question_performance
      return all_questions_performance unless @question
      single_question_performance(@question)
    end

    def top_performing_questions(limit: 10)
      SeededQuestion.active
        .joins(exposures: :board_post)
        .select('cp_seeded_questions.*, 
                 COUNT(DISTINCT cp_seeded_question_exposures.id) as exposure_count,
                 COUNT(DISTINCT cp_board_comments.id) as reply_count,
                 COUNT(DISTINCT cp_board_reactions.id) as reaction_count')
        .left_joins(exposures: { board_post: [:comments, :reactions] })
        .group('cp_seeded_questions.id')
        .order('reply_count DESC, reaction_count DESC')
        .limit(limit)
    end

    def effectiveness_score(question)
      exposures = question.exposures.count
      return 0 if exposures.zero?

      replies = BoardComment.joins(:post)
        .where(cp_board_posts: { seeded_question_exposure_id: question.exposure_ids })
        .count
      
      reactions = BoardReaction.joins(:post)
        .where(cp_board_posts: { seeded_question_exposure_id: question.exposure_ids })
        .count
      
      # Weighted score: replies worth more than reactions
      ((replies * 3 + reactions) / exposures.to_f * 100).round(1)
    end

    private

    def single_question_performance(question)
      exposures = question.exposures.in_period(@start_date, @end_date)
      posts = BoardPost.where(seeded_question_exposure_id: exposures.pluck(:id))
      
      {
        exposure_count: exposures.count,
        reply_count: BoardComment.where(post_id: posts.pluck(:id)).count,
        reaction_count: BoardReaction.where(post_id: posts.pluck(:id)).count,
        avg_replies_per_exposure: avg_replies_per_exposure(exposures, posts),
        avg_reactions_per_exposure: avg_reactions_per_exposure(exposures, posts),
        effectiveness_score: effectiveness_score(question)
      }
    end
  end
end

4. Instrumentation

4.1 Wizard Instrumentation

Add tracking to ProfileWizardController:

# app/controllers/cp/profile_wizard_controller.rb
class Cp::ProfileWizardController < Cp::BaseController
  
  def show
    track_step_viewed
    # existing code...
  end

  def update
    if process_step_update
      track_step_completed
      # existing code...
    end
  end

  private

  def track_wizard_started
    return if wizard_started?
    
    Cp::OnboardingEvent.create!(
      champion: current_cp_champion,
      event_type: 'wizard_started',
      session_id: session.id
    )
  end

  def track_step_viewed
    Cp::OnboardingEvent.create!(
      champion: current_cp_champion,
      event_type: 'step_viewed',
      step: current_step,
      session_id: session.id
    )
  end

  def track_step_completed
    Cp::OnboardingEvent.create!(
      champion: current_cp_champion,
      event_type: 'step_completed',
      step: current_step,
      metadata: { duration_seconds: step_duration },
      session_id: session.id
    )
  end

  def track_wizard_completed
    Cp::OnboardingEvent.create!(
      champion: current_cp_champion,
      event_type: 'wizard_completed',
      metadata: { duration_minutes: wizard_duration_minutes },
      session_id: session.id
    )
  end

  def wizard_started?
    Cp::OnboardingEvent.exists?(
      cp_champion_id: current_cp_champion.id,
      event_type: 'wizard_started'
    )
  end

  def step_duration
    # Calculate time since step was viewed
    last_view = Cp::OnboardingEvent
      .where(cp_champion_id: current_cp_champion.id, event_type: 'step_viewed', step: current_step)
      .order(created_at: :desc)
      .first
    
    return nil unless last_view
    (Time.current - last_view.created_at).to_i
  end
end

4.2 Community Joining Instrumentation

Add tracking to CommunitySuggestion:

# app/models/cp/community_suggestion.rb
module Cp
  class CommunitySuggestion < ApplicationRecord
    after_create :track_suggested

    def accept!
      track_accepted
      # existing accept logic
    end

    def decline!
      track_declined
      # existing decline logic
    end

    private

    def track_suggested
      OnboardingEvent.create!(
        champion: champion,
        event_type: 'community_suggested',
        metadata: { community_id: community_id, type: community.community_type }
      )
    end

    def track_accepted
      OnboardingEvent.create!(
        champion: champion,
        event_type: 'community_accepted',
        metadata: { community_id: community_id, type: community.community_type }
      )
    end

    def track_declined
      OnboardingEvent.create!(
        champion: champion,
        event_type: 'community_declined',
        metadata: { community_id: community_id, type: community.community_type }
      )
    end
  end
end

5. Admin Dashboard

5.1 Dashboard Location

Path: /champions/insights/onboarding

Extend existing Champion Insights with onboarding tab.

5.2 Dashboard Layout

┌─────────────────────────────────────────────────────────────────────────────┐
│ Champion Insights                                                           │
├─────────────────────────────────────────────────────────────────────────────┤
│ [Overview] [Engagement] [Onboarding] [Seeded Questions]                     │
│                                                                             │
│ Date Range: [Last 30 Days ▼]                [Export CSV]                    │
│                                                                             │
│ ═══════════════════════════════════════════════════════════════════════════ │
│                                                                             │
│ WIZARD FUNNEL                                                               │
│                                                                             │
│ ┌─────────────────────────────────────────────────────────────────────────┐ │
│ │                                                                         │ │
│ │  Started   → Step 1  → Step 2  → ... → Complete                        │ │
│ │   100%        98%       95%              80%                            │ │
│ │                                                                         │ │
│ │  [==============================================================]      │ │
│ │  [============================================================]        │ │
│ │  [=========================================================]           │ │
│ │  [=============================================]                        │ │
│ │                                                                         │ │
│ └─────────────────────────────────────────────────────────────────────────┘ │
│                                                                             │
│ ┌──────────────────┐  ┌──────────────────┐  ┌──────────────────┐           │
│ │     127          │  │     102          │  │     80.3%        │           │
│ │   Started        │  │   Completed      │  │ Completion Rate  │           │
│ │   ↑ 12% vs prev  │  │   ↑ 15% vs prev  │  │   ↑ 2.1% vs prev │           │
│ └──────────────────┘  └──────────────────┘  └──────────────────┘           │
│                                                                             │
│ ─────────────────────────────────────────────────────────────────────────── │
│                                                                             │
│ DROP-OFF ANALYSIS                                                           │
│                                                                             │
│ Step              Completion    Drop-off    Notes                           │
│ ─────────────────────────────────────────────────────────────────────────── │
│ help_find_you     98.0%         2.0%                                        │
│ confirm_education 95.3%         2.7%                                        │
│ location          94.8%         0.5%        ⚠️ Below target                 │
│ champion_role     92.1%         2.7%                                        │
│ profession        89.8%         2.3%                                        │
│ photo             85.2%         4.6%        ⚠️ High drop-off               │
│ bio               83.1%         2.1%                                        │
│ affinities        81.4%         1.7%                                        │
│ join_communities  80.3%         1.1%                                        │
│                                                                             │
│ ═══════════════════════════════════════════════════════════════════════════ │
│                                                                             │
│ COMMUNITY JOINING                                                           │
│                                                                             │
│ ┌──────────────────┐  ┌──────────────────┐  ┌──────────────────┐           │
│ │     2.7          │  │     42.3%        │  │     31.2%        │           │
│ │  Avg Joined      │  │ Accept Rate      │  │ Browse-to-Join   │           │
│ └──────────────────┘  └──────────────────┘  └──────────────────┘           │
│                                                                             │
│ Joins by Type:                                                              │
│ District ████████████████████████ 45%                                       │
│ College  ████████████ 22%                                                   │
│ Major    ██████████ 18%                                                     │
│ Affinity ████████ 15%                                                       │
│                                                                             │
│ ═══════════════════════════════════════════════════════════════════════════ │
│                                                                             │
│ FIRST-WEEK ENGAGEMENT                                                       │
│                                                                             │
│ ┌──────────────────┐  ┌──────────────────┐  ┌──────────────────┐           │
│ │     61.2%        │  │     3.2          │  │     4.1 days     │           │
│ │ First-Week Active│  │ Avg Return Visits│  │ Time to 1st Post │           │
│ │   ↑ 5% vs prev   │  │   ↑ 0.4 vs prev  │  │   ↓ 0.8 vs prev  │           │
│ └──────────────────┘  └──────────────────┘  └──────────────────┘           │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

5.3 Seeded Questions Tab

┌─────────────────────────────────────────────────────────────────────────────┐
│ Champion Insights                                                           │
├─────────────────────────────────────────────────────────────────────────────┤
│ [Overview] [Engagement] [Onboarding] [Seeded Questions]                     │
│                                                                             │
│ Date Range: [Last 30 Days ▼]                [Export CSV]                    │
│                                                                             │
│ ═══════════════════════════════════════════════════════════════════════════ │
│                                                                             │
│ OVERALL PERFORMANCE                                                         │
│                                                                             │
│ ┌──────────────────┐  ┌──────────────────┐  ┌──────────────────┐           │
│ │     156          │  │     78           │  │     312          │           │
│ │   Exposures      │  │   Replies        │  │   Reactions      │           │
│ └──────────────────┘  └──────────────────┘  └──────────────────┘           │
│                                                                             │
│ Seeded vs Organic:                                                          │
│ ┌─────────────────────────────────────────────────────────────────────────┐ │
│ │ Seeded Questions:   0.50 replies/post  |  2.0 reactions/post           │ │
│ │ Organic Posts:      0.23 replies/post  |  0.8 reactions/post           │ │
│ │                                                                         │ │
│ │ Seeded questions generate 2.2x more engagement! ✓                       │ │
│ └─────────────────────────────────────────────────────────────────────────┘ │
│                                                                             │
│ ─────────────────────────────────────────────────────────────────────────── │
│                                                                             │
│ TOP PERFORMING QUESTIONS                                                    │
│                                                                             │
│ #  Question                           Exposures  Replies  Reactions  Score  │
│ ─────────────────────────────────────────────────────────────────────────── │
│ 1  "What's your favorite Belmont...   45         23       89         182   │
│ 2  "What brought you to {communi...   32         18       62         148   │
│ 3  "If you could relive one mome...   28         12       45         101   │
│ 4  "What advice would you give t...   25         10       38          88   │
│ 5  "How has being a Bruin shaped...   22          8       31          73   │
│                                                                             │
│ ─────────────────────────────────────────────────────────────────────────── │
│                                                                             │
│ PERFORMANCE BY TARGET TYPE                                                  │
│                                                                             │
│ Target Type        Avg Replies   Avg Reactions   Effectiveness             │
│ ─────────────────────────────────────────────────────────────────────────── │
│ All Communities    0.48          1.8             ████████████ High         │
│ District           0.52          2.1             █████████████ High        │
│ Affinity Subtype   0.61          2.4             ██████████████ Highest    │
│ College            0.41          1.5             ██████████ Medium         │
│ Major              0.38          1.2             ████████ Medium           │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

6. Implementation Plan

Day 1: Data Model

Files to create:

db/migrate/XXXXXX_create_cp_onboarding_events.rb
app/models/cp/onboarding_event.rb

Day 2: Instrumentation

Files to modify:

app/controllers/cp/profile_wizard_controller.rb
app/models/cp/community_suggestion.rb

Day 3: Metrics Services

Files to create:

app/services/cp/onboarding_metrics_service.rb
app/services/cp/seeded_question_metrics_service.rb

Day 4: Dashboard Views

Files to create:

app/views/champions/insights/_onboarding.html.erb
app/views/champions/insights/_seeded_questions.html.erb

Files to modify:

app/controllers/champions/insights_controller.rb

Day 5: Testing & Polish

Service tests
Controller tests
Dashboard visual testing
Edge case handling

7. Testing Requirements

Model Tests

# test/models/cp/onboarding_event_test.rb
class Cp::OnboardingEventTest < ActiveSupport::TestCase
  test "validates event_type presence" do
    event = Cp::OnboardingEvent.new(champion: cp_champions(:sarah_champion))
    assert_not event.valid?
    assert_includes event.errors[:event_type], "can't be blank"
  end

  test "validates event_type inclusion" do
    event = Cp::OnboardingEvent.new(
      champion: cp_champions(:sarah_champion),
      event_type: 'invalid_type'
    )
    assert_not event.valid?
  end

  test "wizard_events scope returns wizard-related events" do
    champion = cp_champions(:sarah_champion)
    
    wizard_event = Cp::OnboardingEvent.create!(champion: champion, event_type: 'wizard_started')
    community_event = Cp::OnboardingEvent.create!(champion: champion, event_type: 'community_suggested')
    
    wizard_events = Cp::OnboardingEvent.wizard_events
    
    assert_includes wizard_events, wizard_event
    assert_not_includes wizard_events, community_event
  end
end

Service Tests

# test/services/cp/onboarding_metrics_service_test.rb
class Cp::OnboardingMetricsServiceTest < ActiveSupport::TestCase
  setup do
    @champion = cp_champions(:sarah_champion)
  end

  test "completion_rate calculates correctly" do
    # Create 10 started events
    10.times do |i|
      champion = Cp::Champion.create!(email: "test#{i}@example.com", status: :email_verified)
      Cp::OnboardingEvent.create!(champion: champion, event_type: 'wizard_started')
    end
    
    # Create 8 completed events
    Cp::OnboardingEvent.where(event_type: 'wizard_started').limit(8).each do |event|
      Cp::OnboardingEvent.create!(champion: event.champion, event_type: 'wizard_completed')
    end
    
    service = Cp::OnboardingMetricsService.new
    assert_equal 80.0, service.funnel_metrics[:completion_rate]
  end

  test "step_completion_rates tracks each step" do
    Cp::OnboardingEvent.create!(champion: @champion, event_type: 'wizard_started')
    Cp::OnboardingEvent.create!(champion: @champion, event_type: 'step_completed', step: 'location')
    
    service = Cp::OnboardingMetricsService.new
    rates = service.funnel_metrics[:step_completion_rates]
    
    assert_equal 100.0, rates['location']
  end
end

Controller Tests

# test/controllers/champions/insights_controller_test.rb
class Champions::InsightsControllerTest < ActionDispatch::IntegrationTest
  setup do
    @admin = users(:admin)
  end

  test "onboarding tab displays funnel metrics" do
    sign_in @admin
    get champions_insights_url(tab: 'onboarding')
    
    assert_response :success
    assert_select "h2", /Wizard Funnel/
  end

  test "seeded questions tab displays performance data" do
    sign_in @admin
    get champions_insights_url(tab: 'seeded_questions')
    
    assert_response :success
    assert_select "h2", /Top Performing Questions/
  end
end

What Was Implemented

To be updated after implementation

Deferred Items

Item	Reason	Future Phase
Real-time dashboard updates	Complexity	Post-MVP
Cohort analysis	Complexity	Post-MVP
A/B test results display	Depends on A/B framework	Post-MVP
Automated alerts	Nice-to-have	Post-MVP
Export to external analytics	Integration complexity	Post-MVP

Document created: January 2026

alumni_lookup

Sub-Phase 9.4: Onboarding Metrics & Analytics

Completion Summary

What Was Implemented

Test Results

Files Created/Modified

Table of Contents

1. Overview

Goals

Scope

2. Key Metrics

2.0 Pre-Wizard Signup Funnel

2.1 Onboarding Funnel

2.2 Community Joining

2.3 First-Week Engagement

2.4 Seeded Question Performance

3. Data Model

3.1 Onboarding Events Table

3.2 Model

3.3 Metrics Service

3.4 Seeded Question Metrics Service

4. Instrumentation

4.1 Wizard Instrumentation

4.2 Community Joining Instrumentation

5. Admin Dashboard

5.1 Dashboard Location

5.2 Dashboard Layout

5.3 Seeded Questions Tab

6. Implementation Plan

Day 1: Data Model

Day 2: Instrumentation

Day 3: Metrics Services

Day 4: Dashboard Views

Day 5: Testing & Polish

7. Testing Requirements

Model Tests

Service Tests

Controller Tests

What Was Implemented

Deferred Items