andresresiri/meseo-staging
1
0
Fork 0
Code Issues Pull Requests Actions 28 Packages Projects Releases Wiki Activity
Files
c880e8a0f8a875c4394912e1a59a952e05d9fe89
meseo-staging/docs/deep-audit-inventory.md
T
Andres Resiri 70daeb214a
CI / Quality gate (push) Has been cancelled
Details
Initial project commit for Coolify staging
2026-08-03 10:45:41 -04:00

65 KiB
Raw Blame History

meSEO Deep Audit Inventory

Date: 2026-06-10 Commit: 07e37ef (HEAD, main) Author: Claude Code -- audit-only session Purpose: Onboarding reference for new dev team. Grounds all claims in actual file paths, commit hashes, and code. Where something cannot be confirmed from code, it is marked UNCONFIRMED.

Table of Contents

  1. Session and Phase History
  2. Sidebar Pages
  3. Core Platform Architecture
  4. Audit Flows
  5. AI Strategist
  6. JSX Dashboards
  7. Guardrails
  8. Customer-Facing Surfaces
  9. Schema and Crons
  10. Risk Register
  11. Patent Map

1. Session and Phase History

Pre-git-history phases (reconstructed from code evidence)

The visible git history begins at 2026-05-25 (phase 30B). The code itself, however, contains evidence of a complete multi-phase build predating that commit. The following phases are reconstructed from what exists in the codebase today.

Phase 1-3: Foundation

Evidence: prisma/schema.prisma multi-tenant models; src/lib/auth/, src/middleware.ts, Clerk provider in src/app/layout.tsx.

  • Multi-tenant architecture: Organization, Brand, User, OrgMembership, BrandMembership
  • Clerk authentication: userId sync, organization context, BrandMembership role enforcement
  • Brand model: domain, industry, locations[], primaryServices[], secondaryServices[], plan, billingCycle, planExpiresAt, healthScore, isDemo
  • Organization + subscription management
  • Onboarding flow: src/app/onboarding/page.tsx
  • Settings: src/app/settings/page.tsx (tabs: profile, integrations, notifications, white-label)
  • Brands Hub: multi-brand switcher, src/app/brands-hub/page.tsx
  • Pricing page: src/app/pricing/page.tsx

Phase 4-6: Data Layer + Integrations

Evidence: src/lib/services/ga4-sync.ts, src/app/api/gsc/backfill/route.ts, src/lib/integrations/nango.ts, prisma/schema.prisma BrandIntegration model.

Nango OAuth provider (src/lib/integrations/nango.ts):

  • Universal OAuth for GSC, GA4, GBP, Bing
  • getValidAccessToken(brandId, integrationId) -- refreshes if expiring
  • getCredentials(brandId, integrationId) -- returns stored credentials
  • Credentials stored encrypted in BrandIntegration.credentialsEnc (AES-256-GCM)
  • Token expiry tracked in BrandIntegration.tokenExpiresAt

GSC backfill (src/app/api/gsc/backfill/route.ts):

  • POST /api/gsc/backfill -- backfills 16 months of historical data
  • Dimensions: ["date", "page"] and ["date", "query"]
  • Pagination: 25,000 rows per batch; handles multi-batch responses
  • GscPage upsert: brandId, pageUrl, clicks, impressions, ctr, position, date
  • GscQuery upsert: brandId, query, clicks, impressions, ctr, position, date
  • Pre-deletes date range to prevent duplicate rows
  • Verifies connection via totals call before backfilling

GA4 sync (src/lib/services/ga4-sync.ts):

  • storeGa4Data(brandId, rows, startDate, endDate) -- upserts daily snapshots
  • Safeguard: detects inflated key events (>3x sessions) and skips conversion writes
  • MetricSnapshot upsert: { brandId, date, source: "ga4", sessions, users, pageviews, conversions, bounceRate }
  • getGa4Summary(brandId, days) -- summary query for dashboard/AI avatar

MetricSnapshot + ChannelMetric (schema):

  • MetricSnapshot: daily per-brand per-source (ga4|gsc|site_tag|daily|manual|seed)
  • ChannelMetric: daily per-channel breakdown (organic|ai_search|social|gbp|direct)

Integrations page (src/app/integrations/page.tsx):

  • Connect/disconnect: GSC, GA4, GBP, Bing
  • Status indicators per integration
  • API: /api/integrations/connect, /api/integrations/disconnect

Phase 7-9: Core Audit Runner

Evidence: src/lib/audit/audit-runner.ts (2038 lines), 26 dimension aggregator files under src/lib/audit/aggregators/, 7 synthesizer files under src/lib/audit/synthesizers/, src/lib/audit/types.ts (900+ lines).

audit-runner.ts -- main orchestration:

  • 7-phase pipeline: (1) brand context, (2) data fetch, (3) external APIs (DataForSEO + Firecrawl), (4) dimension aggregation, (5) LLM synthesis, (6) output assembly, (7) DB write
  • Cost tracking constants: COST_PER_CRAWL_PAGE = 0.0001, COST_PER_SONNET_CALL = 0.05
  • Cost ceilings: $4.00 first-party, $2.00 third-party -- hard abort if exceeded
  • Degraded mode: triggers when pages.length === 0, uses Site Tag telemetry + GSC data
  • Prompt caching: cache_control: { type: "ephemeral" } on narrative synthesis input
  • External calls: DataForSEO (/on_page/task_post, /backlinks/summary/live, /serp/google/organic/live/advanced), Firecrawl (5-10 key pages, markdown + metadata), Anthropic

AuditInput type (src/lib/audit/types.ts): 40+ optional input fields including: brandId, domain, businessName, industry, locations[], primaryServices[], targetAudience, pages[], gscData, ga4Data, siteTagData, competitors[], competitorGaps[], backlinks, aiVisibility, trackedKeywords[], conversionData, adData, signalData, userEngagementData, existingAudit, analysisConfig

SeoSection type (src/lib/audit/types.ts): 40+ optional output fields including: overallScore, headline, summary, keyFindings[], priorityActions[], competitiveLandscape, organicSearch, technicalHealth, conversionGap, eeat, aiSearchVerification, aiEnginePersonality, voiceSearch, pageSpeed, backlinks, multiTouch, revenueAttribution, executiveBrief, strategicRoadmap, implementationAppendix, industryContext, dimensionScores

DimensionScores: technicalSeo, contentMetadata, aiVisibility, conversionOptimization, userEngagement

26 dimension aggregators (in src/lib/audit/aggregators/): Each returns a typed object fed into the LLM context. Input = slices of AuditInput. Output = dimension-specific analysis.

Aggregator Domain
backlinks.ts Domain authority, link quality, toxicity scoring
brand-authority.ts Trust signals, E-E-A-T surface area
competitor-deep-dive.ts Per-competitor detailed breakdown
competitor-landscape.ts Market positioning, threat levels
competitor-visual.ts Side-by-side metric comparison
content-metadata.ts Title/meta/H1 coverage and quality
conversion-heatmap.ts Conversion-weighted session data
conversion-opportunity.ts Revenue gap vs. competitors
ecommerce.ts Product listing, checkout funnel signals
eeat.ts Expertise-Authority-Trust signals
executive-brief.ts Score + business impact narrative
industry-benchmark.ts Brand vs. vertical median KPIs
key-findings.ts Top 5-7 prioritized findings assembly
live-visual-annotations.ts Real-time behavioral overlay signals
lost-opportunity.ts Revenue impact of ranking gaps
multi-touch-ai.ts Multi-touch attribution across channels
organic-search.ts Click/impression/position from GSC
page-speed.ts Core Web Vitals (LCP/CLS/INP/TTFB/FCP)
page-strategic-pass.ts Per-page content/technical recommendations
predictive-conversion.ts ML-based conversion probability estimates
priority-actions.ts Ranked action item assembly
revenue-attribution.ts Per-channel revenue attribution
schema-opportunity.ts Schema markup gap analysis
signal-center.ts Rank/traffic/competitor signal aggregation
snippet-capture.ts Snippet-by-snippet capture tactics
strategic-roadmap.ts 90-day implementation roadmap

7 synthesizer files (in src/lib/audit/synthesizers/): Each receives aggregated dimension data and returns narrative + structured output.

Synthesizer Model Timeout Notes
seo-synthesizer.ts Sonnet 4.6 (narrative) + Haiku 4.5 (structured) 280s + 45s in Promise.all Primary: 14K tokens. 14 industry lenses. Vendor confidentiality enforced (service names stripped).
real-user-technical.ts -- -- Surfaces non-SEO issues from RealUserMetric
ecommerce.ts -- -- E-commerce product/checkout funnel lens
multilingual.ts -- -- hreflang and regional content gaps
compliance.ts -- -- HIPAA/GDPR/legal compliance signals
b2b.ts -- -- B2B SEO lens
industry-classifier.ts -- -- Derives industry string[] from domain/content/integrations (commit 3fc9729)

Industry lenses (14 categories): healthcare-medical, b2b-saas, ecommerce, local-services, professional-services, finance, legal, real-estate, nonprofit, hospitality, higher-ed, b2b, dentist, other. industryContext persisted to seoSection JSONB (commit ce67953).

Phase 10-12: AI + Public Audit

Evidence: src/app/api/public-audit/route.ts, src/lib/services/ai-visibility-checker.ts (250+ lines), src/app/ai-strategist/page.tsx, src/lib/ai-strategist/artifacts.ts.

Public audit flow (src/app/api/public-audit/route.ts):

  • No auth required
  • Lead capture: firstName, lastName, email, companyName, phoneNumber, websiteUrl
  • Rate limit: 3 req/IP/hour
  • OTP email verification -- returning leads (matched by email) bypass OTP
  • Same synthesizer pipeline as first-party audit
  • LockedFirstPartyTeaser.tsx shows upgrade CTA to anonymous users
  • Audit.auditToken stores the shareable audit URL

AI visibility checker (src/lib/services/ai-visibility-checker.ts):

  • Uses Anthropic API with web_search_20250305 tool
  • 7 AI sources checked: ChatGPT, Perplexity, Gemini, Claude, Copilot, Phind, You.com
  • checkAiVisibility(brandId, query) -- returns AiCheckResult { mentioned, position, context, sentiment, sourceUrls }
  • Results stored in AiVisibilityCheck + aggregate to AiVisibilitySnapshot
  • Weekly cron at 0 0 * * 1

AI Strategist (src/app/ai-strategist/page.tsx, 1342 lines):

  • Conversational interface
  • Context: brand profile + GSC + Site Tag + AnalysisJob.result + paid media + LLMO context
  • Artifacts: <artifact> tags rendered in iframe sandbox via ArtifactRenderer
  • Models: claude-sonnet-4-6, 12K max output tokens, 120s maxDuration
  • Every call logged to AiInteraction (userId, brandId, feature, guardrailFlags[])

Phase 13-15: Content + Execution + Competitive

Evidence: src/app/keyword-research/page.tsx, src/app/rank-tracker/page.tsx, src/app/api/cron/rank-check/route.ts, src/app/backlinks/page.tsx, src/app/competitors/page.tsx, src/app/api/content-briefs/generate/route.ts, src/app/execution-hub/page.tsx.

Keyword research (src/app/keyword-research/page.tsx):

  • Deep-link: ?keyword=X from Topic Map / Execution Hub
  • AI analysis output (Claude): summary, topic clusters, quick wins, gap analysis, question keywords, avoid keywords, content plan
  • Each cluster: name, keywords[], contentStrategy, priority, estimatedTrafficPotential
  • Search history: GET /api/keyword-research/analyze?brandId=...
  • maxDuration: analyze 120s, search 60s

Rank tracker (src/app/rank-tracker/page.tsx):

  • GET /api/cron/rank-check (daily, 300s maxDuration)
  • DataForSEO /serp live checks per TrackedKeyword
  • Cost cap: soft $5 USD/day, hard 100 keywords/brand
  • Summary cards: total tracked, avg position, top-3/10/20/not-ranking counts
  • Distribution chart with click-to-filter, movers panel (improved/declined)
  • TrackedKeyword: keyword, location, device, targetUrl, tags[], isActive
  • KeywordRanking: per-day position + previousPosition + checkedAt

Backlinks (src/app/backlinks/page.tsx):

  • Backlink: sourceDomain, sourceUrl, targetUrl, anchorText, linkType, domainAuthority, relevanceScore, toxicityScore, qualityTier, toxicityReasons[], status (active|lost|disavowed)
  • BacklinkSnapshot: historical totals + avgDA + toxicCount
  • Features: quality distribution chart, toxic panel, gap analysis (competitor backlinks we lack), trend chart (monthly new/lost), ref domain list with DA ranking
  • DataForSEO backlinks API: daily backlink-sync cron

Competitors (src/app/competitors/page.tsx):

  • Competitor: domain, name, type, isActive, lastCrawledAt, detectedCms
  • CompetitorSnapshot: date, pageCount, newPages, movement, rankChange, aiMentions, trafficEstimate
  • Features: discovery + manual add, sitemap monitoring, ranking comparison, AI strategic analysis (Claude), change timeline, content gaps
  • CompetitorDiscoveryJob: async discovery with progress tracking

Content briefs (src/app/api/content-briefs/generate/route.ts):

  • POST /api/content-briefs/generate
  • Types: "seo" (keyword-optimized editorial, H1/H2 structure, FAQ, meta) and "llmo" (Question-Answer-Explanation-Insight cadence for AI citation)
  • Model: callLlm() from @/lib/ai-avatar/llm-client
  • Humanization toggle, tone options (professional|conversational|persuasive)
  • ContentBrief: type, status (draft|review|approved|published), primaryKeyword, sections (JSON), humanizationMode, contentScore

Execution Hub (src/app/execution-hub/page.tsx, 73KB):

  • Task: title, category (seo|llmo|gbp|schema|content|technical|web-dev), status (todo|in_progress|completed), priority, source, sourceId, pipelineStage, briefId, assignedTo, dueDate
  • Views: Calendar, Pipeline board
  • Tasks linked to audit/signal sources via sourceId

Pre-git-history continuation: supporting systems

Email reports (src/lib/services/email-report-generator.ts):

  • Sections (configurable per schedule): GSC, GA4, Site Tag, Technical Audit, Rank Tracker, Conversions, Page Lifecycle, UX Signals
  • Frequency: daily|weekly|biweekly|monthly
  • AI enrichment: Claude analysis of snapshot data -> summary, wins, concerns, recommendation (JSON)
  • Timezone support from User.timezone (fallback: America/New_York)
  • EmailReportSchedule: brandId, frequency, sections[], time, lastSentAt
  • SentEmailReport: scheduleId, recipientEmail, sentAt, contentHash

Alert system (src/lib/services/signals.ts): 14 signal generators, all returning Signal[]:

  • generateTechnicalAuditSignals, generatePerformanceAuditSignals, generateDeadPageSignals
  • generateConversionSignals, generateExecutionSignals, generateAiVisibilitySignals
  • generateSiteTagSignals, generatePageLifecycleSignals, generateRankTrackingSignals
  • generateCompetitorSignals, generateDetectedSignals, generateAiRecommendationSignals
  • generateDataFreshnessSignals, generateAiRecommendationSignals

Signal shape:

{
  id, title, description,
  category: "issue"|"win"|"opportunity"|"revenue"|"ai"|"execution",
  priority: "critical"|"high"|"medium"|"low",
  status: "new"|"viewed"|"actioned",
  timestamp, relatedPage?, relatedChannel?,
  source, issueIds?, suggestedAction?, competitor?,
  revenueImpact?: "high"|"medium"|"low"
}

Dismissal TTL: 14 days (synthetic signals re-surface after this period). Revenue impact derivation: rules-based on category + priority + content keywords.

CRM integration (src/lib/services/crm.ts):

  • Supported CRMs: HubSpot, Salesforce, HighLevel, manual
  • CrmDeal: stages (lead|qualified|proposal|negotiation|closed_won|closed_lost), status (open|won|lost), value, closeDate, contactName, contactEmail, externalId
  • getDeals(brandId, range?, filters?) -- with status/stage/source filtering
  • getPipelineSummary(brandId, range?) -- pipeline value, win rate, by-source breakdown
  • upsertDeal() -- create/update via externalId (sync with external CRM)
  • CrmDeal links to ConversionEvent via conversion relation
  • RevenueAttribution tracks conversion -> deal attribution

Page lifecycle (src/lib/services/page-lifecycle.ts, src/app/page-launch-tracker/page.tsx):

  • resolveSiteFromTag(siteId, pageUrl) -- validates TrackedSite + domain allowlist
  • toPathKey(url) -- normalizes to canonical /path (strips protocol/host/query/hash)
  • buildFullUrl(brandDomain, path) -- reconstructs absolute URLs
  • PageRecord: url, currentStatus, firstSeenAt, lastSeenAt, redirect chains
  • PageStatusChange: status transitions with timestamps
  • RedirectRecord: source, target, statusCode
  • PageDailyMetric: GSC-joined per-URL daily clicks/impressions
  • Page launch detection: joins Site Tag first-seen dates with GSC baselines; tracks post-launch deltas

Admin overview (src/app/api/admin/overview/route.ts):

  • Aggregates: total orgs/brands/users, active brands (7-day), integration counts (GSC/GA4/GBP/Site Tags)
  • AI usage: today/week/month + costUsd
  • Event coverage: 13 categories (Core, Forms, Phone, Booking, Video, E-commerce, UX Signals, Performance, Social, Content, Competitor, Chat, Scheduling)
  • Data sources: TrackedEvent count by eventType, SiteEvent, RealUserMetric
  • In-memory 60s TTL cache

Dashboard (src/app/dashboard/page.tsx):

  • Module-scope Map (60s TTL) + localStorage (12h TTL) for cache
  • 3-phase loading: Phase -1 = localStorage fallback (instant), Phase 0 = server snapshot (/api/dashboard/summary), Phase 1 = live fetch with 20s AbortController timeout
  • 9 customizable sections: KPI Cards, Paid Media, Technical Issues, Execution Hub, Recommendations, AI Visibility, Competitors, Search Performance (lazy-loaded), Quick Actions
  • Channel toggle: GA4 split All Traffic vs. Organic Search
  • API: /api/dashboard, /api/dashboard/summary, /api/dashboard/search-performance, /api/conversion-intelligence/attribution

public/t.js (client tracking tag, 162.8 KB minified):

  • 74 tracked event types across 13 categories (full list in Section 8)
  • Session stitching: meseo_sid + meseo_sts cookies; meseo_xsid/meseo_xsts for cross-subdomain
  • Client session ID: random string, persisted in cookies + memory
  • SPA support: page_view on route changes (history API + URL polling)
  • Event batching: buffered, flushed on timeout or batch-full
  • Collection endpoint: POST /api/collect

Collect route (src/app/api/collect/route.ts):

  • Batch path: { site_id, visitor_id?, events: [...] } (up to 200 events)
  • Single-event path (legacy): full validation + rate check
  • Domain allowlist enforcement (TrackedSite.allowedDomains)
  • createMany for batch inserts; single create for legacy path
  • Side effects (when SITE_TAG_SIDE_EFFECTS_ENABLED=true): platform detection, integration detection, external conversion initiation
  • Request size limit: 64 KB

Git log overview

106 commits on main from 2026-05-25 to 2026-06-10. Commit-message phases (19, 19A.x, 19.5x, 20A) do not map 1:1 to roadmap numbering (13-27). Reconciliation is below.

Phase groups (git history)

Phase 16 (docs/validation) -- 2026-05-26

  • cadef8e docs(phase-16): add validation and auth architecture reference
  • d9417b6 docs(phase-16): replace em dash substitutes
  • Files: docs/architecture/phase-16-validation.md, docs/architecture/CONVENTIONS.md
  • Shipped: Documentation only. No code changes.

Phase 17 (Voice Search Eligibility) -- 2026-05-26

  • 705d2b3 feat(phase-17): voice search eligibility audit backend
  • 14c639f feat(phase-17): voice search eligibility dashboard component
  • 4711328 feat(phase-17): add Voice Search Eligibility section to PDF audit report
  • Shipped: Full voice search eligibility pipeline -- schema, synthesizer output, dashboard component, PDF section.

Phase 19 / 19A.x (Technical Audit + Site Tag Performance) -- 2026-05-27 to 2026-05-28

25+ commits. Sub-milestones:

Commit Date What shipped
545a6c7 05-27 Phase 19A: technical audit page scaffold (mock data)
88b8dde 05-27 Wire technical audit data layer lib
e976da2 05-27 Wire audit trigger buttons
91c41ce 05-27 Wire technical audit page to real seoSection data
e2b1a64 05-27 Hide site tag banner when installed
715e470 05-27 Wire technical audit to background refresh + notifications
3d1c64e 05-27 Add SiteTagDailyRollup cron pre-aggregation
27553b7 05-27 Remove analytics block from site-tag polling route
f909229 05-27 t.js error dedupe + rate limit + counter-based capture
7593eb8 05-27 Redis caching layer for analytics (5-min TTL)
959a662 05-28 Phase 19A.20: loading states, tier label clarification
5488ffd 05-28 Phase 19.5: multilingual, compliance, ecommerce, B2B synthesizers
08420cc 05-28 Phase 19A.21: graceful stats degradation + audit page cache
3b7228f 05-28 Fix 19A.22: wire SiteTagDailyRollup fast path into getTrafficSummary
0572928 05-28 Fix 19A.23: register form_filled, defensive tier guard
51135cf 05-28 Phase 19A.25: expand SiteTagDailyRollup with conversion/event columns
1db278e 05-28 Fix 19A.26: use real siteTagInstalled on audit page
003e7d6 05-28 Fix 19A.26.1: raise first-party audit maxDuration to 800s
8b1b5a8 05-28 Fix 19A.26.2: fix third site-tag banner false-positive
d5e5396 05-28 Fix 19A.27: base maturity banner on rollup active-days
321fdb2 05-28 Phase 19A.28: universal siteTagInstalled in brand context
4dfb003 05-28 Fix 19A.28 followup: suppress banner during brand switch
dd72106 05-28 Fix 19A.28 switch: close banner flash during brand switch
8c761f9 05-28 Run crawl-independent audit sections in degraded mode
1f1fe87 05-28 Raise DFS polling window 240s -> 300s
4f96111 05-28 Fix 19A.29: rollup-ize confidence route + Redis cache + hard 30s timeout

Phase 20A.x (White-Label) -- 2026-05-28

Commit What shipped
6bf61ee Add WhiteLabelConfig + CustomDomain + DomainStatus schema
151f654 White-label settings tab + API route
c961a65 Active-org isolation in white-label API + getEventTrends rollup fast path

Market Position Block 1 and 2 -- 2026-05-28

Commit What shipped
cdb15a9 feat(market-position-block1): 6 new Prisma models (ProbeRun, VisibilityObservation, CompetitorPanel, QueryCorpusEntry, CalibrationRun, MarketPositionSnapshot) + 2 enums
c735d74 feat(market-position-block2): canonicalization + corpus seeding
c018c80 fix(market-position-block2): conversion types, N-scan perf, stop-words

Phase 30B (Platform Detection + Plugin Version) -- 2026-05-25 to 2026-05-29

Commit What shipped
1bad836 Detect + store site platform from page_view events
b2abf55 Platform confidence as strings, tighten signals, fix regexes
55fad07 Plugin version extraction via version() on ANALYTICS_PLUGINS
8f5df21 Gate bounded-async side effects behind SITE_TAG_SIDE_EFFECTS_ENABLED

Phase 32 + 33 (Dedup + Conversion Tier Fixes) -- 2026-05-25

Commit What shipped
f22100e Fix visitorId mismatch Path A vs Path B
8abc2a0 Propagate synthetic flag from TrackedEvent to SiteConversion
b4a44b6 Promote visitor_id to top-level collect body field
2946d61 Fix conversion tier dropdown revert from polling
6282295 Wire canonicalConversionType into BrandConversionConfig override

Session: Ecommerce + Shopify -- 2026-05-29 to 2026-06-02

Commit What shipped
1c25751 orderId column, ingest extraction, API route, UI tab
6c4e8e0 Shopify Web Pixel purchase ingest + lineItems column
9eece0a Extend event buffer flush 10s -> 30s, flush-on-full safety valve
4601e07 Consolidate plugin_detected to single plugins[] event per page
dad4b60 Exclude ingest beacon paths from middleware matcher
7b6e608 1%-sampled drop-reason logging on /event
f677a5f Per-session dedup for plugin_detected, phone_number_visible, call_tracking_detected

Session: Site Audit Phase 1 -- 2026-06-04

Commit What shipped
791d921 SiteAuditRun, SiteAuditIssue, SiteAuditPage schema + renderTier on Brand
62610b6 Crawler service, API routes, cron cleanup
d207a05 Default crawl_delay 200ms, expose zero as opt-in
b0881e5 Replace BrandMembership filter with verifyBrandAccess() on all user-facing routes
b62fdae Use waitUntil to keep Lambda alive after response
3ce9c07 Enforce 60s minimum before accepting DFS "finished"
f4b4582 Resolve canonical host before DFS submit; guard 0-page runs
9ad1005 Cheap pre-reject in /event, sweep stuck runs

Session: Connection-Pool + Cron Fixes -- 2026-06-05

Commit What shipped
accf550 Remove recomputeSessionScore from ingest hot path (stops 25P03 leaks)
5990d96 Attach submissionId token to form_submit
2d1c1b9 Unified phone_call dedup across both write paths
c376913 Restore precompute-dashboard to every 30 min, add budget guard
6a69290 Schedule site-audit-cleanup cron, per-tick cap + debounce
636ac66 Stand up Inngest infrastructure (client, serve route, ping function)

Session: Site Tag 504 Fixes -- 2026-06-08 (this session)

Commit What shipped
543f271 Route 28d analytics onto SiteTagDailyRollup; bound all multi-day raw scans
d6dfed4 Bound activePagesCount to today to stay under withDeadline
6c8cdae Add ?debugTiming=1 instrumentation to GET /api/site-tag
07e37ef Bound all 9 getReconciliationSummary queries to 48h window

Phase numbering reconciliation

  • Phases actually committed: 16, 17, 19/19A, 19.5, 20A, 30B, 32, 33
  • Phases reconstructed from code (pre-git): 1-15 (see above)
  • Phases not in git or code: 18, 21-29 (not built or labelled differently)
  • Market Position is labelled "block1/block2"
  • Site Audit is labelled "Phase 1" internally with no roadmap number

2. Sidebar Pages

Page Route File Wiring
Dashboard /dashboard src/app/dashboard/page.tsx Full
Audit /audit src/app/audit/page.tsx Full
Technical Audit /technical-audit src/app/technical-audit/page.tsx Full
AI Strategist /ai-strategist src/app/ai-strategist/page.tsx Full
Site Tag /site-tag src/app/site-tag/page.tsx Full
Site Tag Analytics /site-tag-analytics src/app/site-tag-analytics/page.tsx Full
Conversion Intelligence /conversion-intelligence src/app/conversion-intelligence/page.tsx Full
Rank Tracker /rank-tracker src/app/rank-tracker/page.tsx Full
Keyword Research /keyword-research src/app/keyword-research/page.tsx Full
Backlinks /backlinks src/app/backlinks/page.tsx Full
Site Performance Audit /site-performance-audit src/app/site-performance-audit/page.tsx Full
Competitors /competitors src/app/competitors/page.tsx Full
Industry Intel /industry-intel src/app/industry-intel/page.tsx Full
AI Visibility /ai-visibility src/app/ai-visibility/page.tsx Full
Signals /signals src/app/signals/page.tsx Full
Execution Hub /execution-hub src/app/execution-hub/page.tsx Full
Page Launch Tracker /page-launch-tracker src/app/page-launch-tracker/page.tsx Full
Content Brief /content-brief src/app/content-brief/page.tsx Full
LLMO Content Brief /llmo-content-brief src/app/llmo-content-brief/page.tsx Full
Schema Generator /schema-generator src/app/schema-generator/page.tsx Partial (UI-driven, not auto-populated)
Dead Pages /dead-pages src/app/dead-pages/page.tsx Full
Broken Links /broken-links src/app/broken-links/page.tsx Full
Report Studio /report-studio src/app/report-studio/page.tsx Full
Content Audit /content-audit src/app/content-audit/page.tsx Full
Recommendations /recommendations src/app/recommendations/page.tsx Full
Writing Assistant /writing-assistant src/app/writing-assistant/page.tsx Full
Email Reports /email-reports src/app/email-reports/page.tsx Full
Notifications /notifications src/app/notifications/page.tsx Full
Integrations /integrations src/app/integrations/page.tsx Full
Settings /settings src/app/settings/page.tsx Full
Brands Hub /brands-hub src/app/brands-hub/page.tsx Full
Client Portal Setup /client-portal-setup src/app/client-portal-setup/page.tsx Full
Local GEO /local-geo src/app/local-geo/page.tsx Stubbed -- "Coming Soon"
Paid Media /paid-media src/app/paid-media/page.tsx Full
Onboarding /onboarding src/app/onboarding/page.tsx Full
Pricing /pricing src/app/pricing/page.tsx Full
Admin /admin src/app/admin/page.tsx Full (admin-only)

Key page details

Dashboard (/dashboard)

  • Data sources: MetricSnapshot, SiteTagDailyRollup (fast path), ConversionEvent, CompetitorSnapshot, ChannelMetric
  • API calls: /api/dashboard/summary, /api/dashboard/search-performance, /api/conversion-intelligence/attribution
  • Cache strategy: 60s TTL module-scope Map + 12h TTL localStorage
  • Range controls: 1d | 7d | 28d | 90d | month | custom

Site Tag (/site-tag)

  • Data sources: TrackedSite, TrackedEvent (recent 15), SiteTagDailyRollup (fast path), SourceEvent, CanonicalEvent
  • API: GET /api/site-tag?brandId=xxx (polls every 30s)
  • Tabs: Setup/Snippet, Domain Status, Custom Events, Plugins Detected, Reconciliation
  • Wiring notes: 07e37ef bounded reconciliation to 48h; 543f271 uses rollup fast path for today counts; d6dfed4 bounded activePagesCount to today; 6c8cdae added ?debugTiming=1 block-level timing

Site Tag Analytics (/site-tag-analytics)

  • Data sources: SiteTagDailyRollup (primary), TrackedEvent (live fallback), HeatmapEvent, RealUserMetric, SiteConversion
  • API: GET /api/site-tag/analytics?brandId=xxx&range=30d
  • Tabs: Traffic, Conversions, UX Signals, Heatmap, Ecommerce (conditional), AI Attribution, Video
  • Deadline: 20s HANDLER_DEADLINE_MS; returns partial: true on timeout

AI Visibility (/ai-visibility)

  • Data sources: AiVisibilitySnapshot, AiVisibilityCheck, AiVisibilityQuery
  • API: GET /api/ai-visibility?summary=true&range=30d, /api/ai-visibility/queries, /check
  • Feature gate: ai_visibility flag -- Professional+ plan
  • Cron: Weekly 0 0 * * 1 via /api/cron/ai-visibility (maxDuration 300s)

Execution Hub (/execution-hub)

  • Data sources: Task (source, sourceId link to audit/signal/dead-page)
  • API: Task CRUD via /api/tasks
  • Categories: seo|llmo|gbp|schema|content|technical|web-dev
  • Views: Calendar, Pipeline board

Page Launch Tracker (/page-launch-tracker)

  • Data sources: PageRecord, PageLaunch, PageLaunchSnapshot, GSC click/impression join
  • API: /api/page-launches
  • Features: lifecycle from first detection (sitemap/GSC/tag), inbound redirect tracking, daily delta chart

Local GEO (/local-geo)

  • Status: COMING SOON -- intentionally stubbed
  • GbpLocation, GbpReview, GbpMetric models and daily gbp-sync cron are active
  • Page deliberately disabled to avoid empty-state before end-to-end wiring

Gaps and unknowns: /topic-map wiring status UNCONFIRMED (POC visualization). issue-detail wiring UNCONFIRMED.

3. Core Platform Architecture

Multi-tenant model

Organization (1) -> Brand (N) -> TrackedSite (1)
Organization (1) -> OrgMembership (N) -> User
Brand (1) -> BrandMembership (N) -> User

Auth: Clerk JWT. verifyBrandAccess() (commit b0881e5) replaces raw BrandMembership filter on all user-facing routes.

Integration layer

Nango (src/lib/integrations/nango.ts):

  • Supports: GSC, GA4, GBP, Bing
  • getValidAccessToken(brandId, integration) -- auto-refresh
  • getCredentials(brandId, integration) -- stored in BrandIntegration.credentialsEnc (AES-256-GCM)
  • BrandIntegration.tokenExpiresAt tracked; marks expired on 2+ refresh failures
  • Nango token refresh cron: hourly /api/cron/nango-sync

Connection pool:

  • DATABASE_URL query params: connection_limit=20, pool_timeout=30
  • Neon pooler endpoint for multiplexing
  • recomputeSessionScore removed from ingest hot path (accf550) -- was causing 25P03 idle-in-transaction leaks

Redis cache

File: src/lib/cache/redis-cache.ts

  • Thin wrapper over redis npm client; resolves REDIS_URL env var
  • Fails open (silently disabled) if REDIS_URL not set
  • Used by: analytics endpoints (5-min TTL), confidence route (30s timeout), pulse heartbeat
  • Commit: 7593eb8 (analytics), 4f96111 (confidence), 2d38e9e (pulse)

Inngest

Status: Infrastructure only. No production functions beyond ping.

  • src/lib/inngest/functions/ping.ts: event test/ping
  • Serve route registered, client configured
  • Commit: 636ac66

prisma-safety-check

File: scripts/prisma-safety-check.mjs

  • Blocks prisma db push if diff contains: DROP COLUMN, DROP TABLE, DROP CONSTRAINT, DROP INDEX, ALTER COLUMN ... DROP
  • Protects the ~22GB SiteEvent table indexes (commit ecd5a48)
  • Override: ALLOW_SCHEMA_DROPS=true
  • Self-test: scripts/test-safety-check.mjs

4. Audit Flows

First-party technical audit (authenticated brand)

Entry: Brand member clicks "Run Audit" on /technical-audit.

Route chain:

  1. POST /api/technical-audit/run (maxDuration 800s)
    • Auth: verifyBrandAccess() (commit b0881e5)
    • Creates TechnicalAudit row (status: pending)
    • Calls runTechnicalAudit() inside waitUntil() so Lambda stays alive after 200 (commit b62fdae)
  2. GET /api/technical-audit/[auditId]/status -- polled every 5s
    • Returns { status, pagesScanned, errorCount, warningCount, passedCount }
  3. GET /api/technical-audit/history -- historical runs

DataForSEO integration:

  • Task submission: submitCrawl(domain) inside site-audit-runner.ts -- POST to DFS /v3/on_page/task_post
  • Payload: url, max_crawl_pages, crawl_delay (default 200ms, commit d207a05), store_raw_html: false
  • Minimum 60s before accepting DFS "finished" (commit 3ce9c07)
  • Polling window: 300s (raised from 240s in 1f1fe87)
  • Results: fetchAllCrawledPages(), fetchRedirectChains(), fetchDuplicateTitles(), fetchDuplicateDescriptions()
  • On DFS unconfigured: limited-mode banner (commit 4028dd2)

Synthesizer pipeline (inside runSiteAudit()):

  • seo-synthesizer.ts: Sonnet 4.6 (14K tokens, 280s) + Haiku 4.5 (1.5K tokens, 45s) in Promise.all
  • real-user-technical.ts, ecommerce.ts, multilingual.ts, compliance.ts, b2b.ts
  • industry-classifier.ts: returns string[] for multi-industry support

Crawl-independent sections (run in degraded mode when DFS crawl fails, commit 8c761f9): SERP competitor analysis, GSC performance, Site Tag signals, AI visibility.

Cron cleanup: /api/cron/site-audit-cleanup -- re-kicks stuck >15 min, fails >2 hours (commit 6a69290).

Third-party / public audit

Entry: Anonymous visitor at public audit URL.

Route chain:

  1. POST /api/public-audit/... -- no auth, validates domain, issues OTP
  2. Lead capture: firstName, lastName, email, companyName, phoneNumber, websiteUrl
  3. Rate limit: 3 req/IP/hour
  4. Returning leads (by email) bypass OTP
  5. Same synthesizer pipeline as first-party (degraded crawl-independent sections)
  6. maxDuration 800s
  7. LockedFirstPartyTeaser.tsx CTA for anonymous users

Audit runner internals (pre-git, audit-runner.ts)

7-phase orchestration (src/lib/audit/audit-runner.ts, 2038 lines):

  1. Brand context assembly (BrandProfile, integrations, plan)
  2. Data fetch (GSC/GA4/Site Tag/competitors in parallel)
  3. External APIs: DataForSEO SERP + backlinks + on-page, Firecrawl 5-10 key pages (markdown + metadata)
  4. 26 dimension aggregators (run in parallel batches)
  5. LLM synthesis (Sonnet 4.6 narrative + Haiku 4.5 structured)
  6. Output assembly (SeoSection object, 40+ fields)
  7. DB write (AnalysisJob upsert, score history)

Cost controls:

  • COST_PER_CRAWL_PAGE = 0.0001, COST_PER_SONNET_CALL = 0.05
  • Hard abort at $4.00 (first-party), $2.00 (third-party)
  • Prompt caching: cache_control: { type: "ephemeral" } on synthesis input

Degraded mode: pages.length === 0 triggers Site Tag telemetry fallback.

5. AI Strategist

Page: src/app/ai-strategist/page.tsx (1342 lines) API: POST /api/ai-strategist/chat (maxDuration 120s)

Context sources

Assembled by getEnrichmentContext() and buildEnrichmentPrompt():

  • Brand profile: name, industry, locations[], primaryServices[]
  • DashboardData: SEO score, AI visibility, crawl health, monthly traffic
  • GSC metrics: searches, impressions, CTR, avg position
  • Site Tag metrics: sessions, conversions, top pages (SiteTagDailyRollup or live)
  • AnalysisJob.result: full JSON grounding for recommendations
  • Paid media: ad spend, ROAS (if AdPlatformIntegration configured)
  • LLMO context: AI visibility snapshot if feature enabled

Models + logging

  • claude-sonnet-4-6, 12K max output tokens (ARTIFACT_MAX_TOKENS)
  • Every call logged to AiInteraction (userId, brandId, feature, userPrompt 4KB cap, aiResponse 16KB cap, guardrailFlags[], flagSeverity)
  • trackEvent() -> PlatformEvent
  • Admin surface: /admin/ai-monitor

Artifacts

  • <artifact> tags in response extracted by parseArtifacts() from src/lib/ai-strategist/artifacts.ts
  • Rendered in iframe sandbox via ArtifactRenderer -- supports JSX dashboards, markdown, structured reports
  • Session persistence: StrategistSession, StrategistMessage

Gaps and unknowns: Token budget enforcement beyond max_tokens UNCONFIRMED. PII guardrail detection logic in guardrailFlags was not read from code.

6. JSX Dashboards

Audit dashboard components (46 files)

Location: src/components/audit/dashboard/

Each receives a slice of the Analysis object from AnalysisJob.result.

Component Renders
AIConversionGapChart.tsx AI-driven conversion gap analysis
AiEnginePersonalitySection.tsx LLM engine personality for brand
AiSearchVerificationSection.tsx AI search mention rates
AiSnippetEligibilitySection.tsx Featured snippet capture readiness
AiSourceAttributionComparisonSection.tsx First-party vs third-party attribution
AuditHero.tsx Report hero with brand + score
AuditTOC.tsx Table of contents with section links
BacklinkProfileSection.tsx Backlink quality, toxicity, domain authority
BehavioralActionBridgeSection.tsx Site Tag behavioral signals -> actions
BilingualContentSection.tsx Multi-language content gaps
BrandAuthorityScoreSection.tsx Domain authority + trust signals
CompetitiveLandscapeSection.tsx Top competitors, overlap, threat level
CompetitorDeepDiveSection.tsx Per-competitor detailed analysis
CompetitorVisualComparisonSection.tsx Side-by-side competitor metrics
ConversionHeatMapSection.tsx Conversion-weighted heatmap overlays
EeatAuditSection.tsx E-E-A-T signals
ExecutiveBriefSection.tsx Executive summary prose
ExecutiveSummaryCard.tsx Summary card with score breakdown
ImplementationAppendixSection.tsx Implementation roadmap table
IndustryBenchmarkSection.tsx Brand vs industry median
KeyFindingsSection.tsx Top 5-7 prioritized findings
LockedFirstPartyTeaser.tsx Upgrade CTA for anonymous audits
LostOpportunityCalculatorSection.tsx Revenue impact of ranking gaps
LiveVisualAnnotationsSection.tsx Real-time behavioral annotations
MethodologySection.tsx Audit methodology explanation
MultiTouchAiPathSection.tsx Multi-touch attribution AI path
OrganicSearchSection.tsx Organic traffic, clicks, impressions
PageSpeedInsightsDeepDiveSection.tsx Core Web Vitals deep dive
PageStrategicPassSection.tsx Per-page strategic recommendations
PredictiveConversionScoringSection.tsx ML-based conversion probability
PriorityActionsSection.tsx Ranked action items
QuickWinsGrid.tsx Fast-ROI action grid
RevenueAttributionSection.tsx Revenue per channel
SchemaOpportunityForecastSection.tsx Schema markup gap analysis
ScoreBreakdown.tsx Score sub-component breakdown
ScoreProjection.tsx Score trend projection
SignalCenterSection.tsx Signal aggregation
SnippetCaptureStrategySection.tsx Snippet capture tactics
StrategicRoadmapSection.tsx 90-day implementation roadmap
TechnicalHealthSection.tsx Technical crawl health summary
ThirdVsFirstPartyComparison.tsx GA4/GSC vs Site Tag comparison
VoiceSearchEligibilitySection.tsx Voice search schema + NLP readiness
index.ts Barrel export

Other component directories

  • src/components/ai-strategist/artifact-renderer.tsx -- iframe sandbox renderer
  • src/components/execution-hub/calendar-view/ -- task calendar
  • src/components/execution-hub/pipeline-board/ -- kanban board
  • src/lib/audit/pdf/ -- @react-pdf/renderer: AuditPdf.tsx, ExecutiveAuditPdf.tsx, pages/, styles.ts

7. Guardrails

Deadline wrappers

withDeadline (src/app/api/site-tag/analytics/route.ts lines 46-51):

async function withDeadline<T>(promise: Promise<T>, ms: number): Promise<Raced<T>> {
  return Promise.race([
    promise.then((data) => ({ winner: "ok", data })),
    new Promise<Raced<T>>((resolve) => setTimeout(() => resolve({ winner: "deadline" }), ms)),
  ]);
}
  • HANDLER_DEADLINE_MS = 20_000 (20s)
  • On deadline: returns { partial: true, analytics: null } with eventsToday: 0 from spread defaults

maxDuration settings:

Route maxDuration
/api/site-audit/run 800s
/api/technical-audit/run 800s
/api/cron/site-tag-rollup 300s
/api/cron/ai-visibility 300s
/api/cron/rank-check 300s
/api/cron/ai-citation-check 300s
/api/ai-strategist/chat 120s
/api/keyword-research/analyze 120s
/api/keyword-research/search 60s
/api/performance-audit/run 60s

waitUntil usage: /api/site-audit/run keeps Lambda alive after 200 (commit b62fdae).

Rate limits

Site Tag ingest (src/lib/services/site-tag-cache.ts):

  • 100 events/min per siteId
  • In-memory token bucket, resets every 60s
  • Returns 200 on drop (not 429) to prevent client console flood

t.js error rate limit (commit f909229):

  • Counter-based per-session dedup for JS errors

Public audit (src/app/api/public-audit/route.ts):

  • 3 req/IP/hour

Rollup fast paths

SiteTagDailyRollup fast path (src/lib/services/site-tag-analytics.ts):

  • _getTrafficSummaryUncached, _getConversionSummaryUncached, getOutboundClickSummary
  • Guard: range === "30d" || range === "28d" || range === "7d" (extended to 28d in commit 543f271)
  • Threshold: rollupRows.length >= expectedDays - 1
  • Turns 1M+ row TrackedEvent group-by into a 30-row indexed lookup on (brandId, date)

Rollup cron (/api/cron/site-tag-rollup, hourly 0 * * * *, maxDuration 300s):

  • Force-recomputes today + yesterday on every run
  • Backfills last 30 days if row missing or conversionsByType IS NULL
  • UTC midnight date key: new Date(Date.UTC(y, m, d))

Today rollup fast path (setup page) (src/app/api/site-tag/route.ts):

  • siteTagDailyRollup.findFirst for eventsToday, eventBreakdown, sessionsToday
  • Falls back to live TrackedEvent only if row absent or eventCountsByType is null

activePagesCount bound (commit d6dfed4):

  • COUNT(DISTINCT CASE ...) bounded to timestamp >= todayStart (was 7 days)
  • CASE expression cannot use an index regardless of window; the bound limits rows scanned

Dedup and dedup resolvers

submissionId token (commit 5990d96):

  • sessionStorage-keyed token attached to form_submit
  • 30-second fixed-from-first window prevents re-fire duplicates

Rapid-fire dedup query (/api/collect/route.ts):

  • WHERE brandId = ? AND sessionId = ? AND pageUrl = ? AND conversionType = ? AND timestamp >= NOW() - 30s
  • Index: @@index([brandId, sessionId, pageUrl, conversionType, timestamp])

Phone call dedup (commit 2d1c1b9):

  • Unified across Path A (/api/collect) and Path B (/api/site-tag/conversion)

Multi-source dedup (CanonicalEvent system) (src/lib/services/site-tag-reconciliation.ts):

  • ingestSourceEvent(): matches existing CanonicalEvent within +-5 min by eventType + pageUrl
  • reconcileTrackedEvents(): fire-and-forget, max 500 TrackedEvents per call, NO date filter (R1)
  • getReconciliationSummary(): all 9 queries bounded to 48h (commit 07e37ef)

Deterministic dedup clustering (src/lib/conversions/dedup.ts):

  • SHA1-based computeDedupKey() for form/phone events
  • Survivor selection: method priority -> touchpoint richness -> earliest timestamp
  • isDuplicateOf pointer written; never hard-delete

Cron dedup (/api/cron/dedup-conversions, daily):

  • Processes unresolved conversions (isDuplicateOf IS NULL)
  • Backfills classifiedChannel after dedup

Connection pool protection

  • DATABASE_URL: connection_limit=20, pool_timeout=30
  • SITE_TAG_SIDE_EFFECTS_ENABLED flag (src/app/api/collect/route.ts line 43): gates all bounded-async side effects in ingest hot path (commit 8f5df21)

Middleware exclusions

Ingest beacon paths excluded from Next.js middleware matcher (commit dad4b60) to eliminate per-request middleware overhead.

8. Customer-Facing Surfaces

Actions a customer can run

Surface Action Route
Audit page Trigger full site audit POST /api/site-audit/run
Technical audit Run Lighthouse + schema crawl POST /api/technical-audit/run
AI Strategist Send message POST /api/ai-strategist/chat
AI Strategist Load/save session StrategistSession CRUD
Site Tag Install snippet GET /api/site-tag/script/[name]
Site Tag Add/remove domain PATCH /api/site-tag action=add_domain / remove_domain
Site Tag Toggle active PATCH /api/site-tag action=toggle_status
Site Tag Regenerate key PATCH /api/site-tag action=regenerate_key
Conversion config Set per-type tier POST /api/brands/[id]/conversion-config
Keyword Research Search keywords POST /api/keyword-research/search
Keyword Research Analyze gap POST /api/keyword-research/analyze
Keyword Research Send to tracker POST /api/keyword-research/send-to-tracker
Content Brief Generate brief POST /api/content-briefs/generate
Schema Generator Generate JSON-LD POST /api/schema/generate-for-brief
Execution Hub Create/update/complete task POST/PATCH /api/tasks
Integrations Connect/disconnect /api/integrations/connect, /disconnect
Email Reports Create/edit schedule POST /api/email-reports
Alert Rules Create/edit alert Alert CRUD endpoints
White-label Configure portal PUT /api/white-label
Client Portal Invite client user POST /api/client-portal/users/invite
AI Visibility Add/manage queries /api/ai-visibility/queries CRUD
AI Visibility Trigger check POST /api/ai-visibility/check
Competitors Discover / add /api/competitors/discover
Page Launch Tracker Mark as launched POST /api/page-launches
GSC Backfill history POST /api/gsc/backfill

Downloadable reports

Report Format(s) Route
Full audit report PDF (comprehensive 40-50pp + executive 6pp) GET /api/admin/audits/[id]/pdf?mode=comprehensive
Report Studio custom PDF, DOCX, Markdown, JSON POST /api/report-studio/export
Report Studio PPTX PPTX POST /api/report-studio/export/pptx
Compliance export JSON GET /api/admin/compliance/events (admin-only)
Platform metrics CSV/JSON GET /api/admin/export/platform-metrics

Site Tag event types (74 total)

Category Events
Core page_view, session_start
Forms form_start, form_submit
Phone click_to_call, sms_click, phone_number_visible, call_tracking_detected
Booking booking_submit
Video video_play
E-commerce purchase (Shopify Web Pixel + standard)
Engagement outbound_click, file_download, pdf_opened, content_engagement
Competitors competitor_visit, competitor_referral, competitive_journey
UX Signals rage_click, dead_click, scroll_bounce (via SiteEvent)
Performance web_vitals, slow_interaction, js_error, resource_error
Custom plugin_detected, custom_event

9. Schema and Crons

Prisma models (129 models, 268 indexes)

Core: Users and Orgs

  • User -- Clerk-synced, email, timezone, emailPrefs (JSON)
  • Organization -- name, slug, plan
  • OrgMembership -- userId + orgId unique
  • BrandMembership -- per-brand role (owner|admin|member|viewer)

Brands

  • Brand -- domain, industry, locations[], primaryServices[], renderTier, isDemo, healthScore
  • BrandIntegration -- integrationId, credentialsEnc (AES-256-GCM), tokenExpiresAt, status
  • BrandConversionConfig -- per-type isCounted + tierOverride + displayLabel
  • BrandProfile -- logoUrl, brandColors[], brandVoice, extractedAt
  • BrandAsset -- filename, url, type, fileSize, dimensions

Analytics + Metrics

  • MetricSnapshot -- daily per-brand per-source (ga4|gsc|site_tag|daily|manual|seed)
  • ChannelMetric -- daily per-channel (organic|ai_search|social|gbp|direct)
  • ConversionEvent -- type, channel, value, landingPage

Site Tag (Real-Time)

  • TrackedSite -- siteId (ms_ prefix), trackingKey (secret), allowedDomains[], customEvents (JSON), detectedPlatform (JSON: name/version/confidence/category/hosting)
  • TrackedSession -- sessionId (unique), device, browser, country, city, channelGroup, predictedConversionScore
  • TrackedEvent -- eventType, pageUrl, timestamp, synthetic, metadata (JSON)
    • Indexes: (trackedSiteId, eventType, timestamp), (trackedSiteId, pageUrl, timestamp), (trackedSiteId, timestamp), (timestamp)
  • SiteConversion -- conversionType, visitorId, sessionId, pageUrl, formProvider, phoneNumber, touchpoints (JSON), conversionValue, orderId, lineItems (JSON), isDuplicateOf, invalidatedReason, classifiedChannel, submissionId
    • Indexes: (brandId, timestamp), (brandId, isDuplicateOf), (brandId, sessionId, pageUrl, conversionType, timestamp)
  • HeatmapEvent -- visitorId, sessionId, pageUrl, eventType, maxScrollPercent, sectionTimes (JSON), pageWidth/Height
  • RealUserMetric -- lcp, cls, inp, ttfb, fcp, device, connection
  • SiteEvent -- eventType (rage_click|dead_click|hesitation|exit_intent|js_error|outbound_click|...), eventData (JSON)
    • Indexes: (trackedSiteId, eventType, timestamp), (sessionId, timestamp) (commit ecd5a48)
  • SiteTagDailyRollup -- brandId, date, pageViews, sessions, conversions, topPages (JSON), byReferrer (JSON), conversionsByType (JSON), conversionsByTier (JSON), eventCountsByType (JSON), outboundClickCount
    • Unique: (brandId, date)
  • PageRecord, PageDailyMetric, NavSnapshot, PageSEOSnapshot

Conversions + Dedup

  • CanonicalEvent -- eventType, pageUrl, timestamp, confidence (0-100), status (verified|inferred|unmatched)
  • SourceEvent -- source (site_tag|ga4|gtm|gsc|crm), isDuplicate, rawData (JSON)

Audit + Analysis

  • Audit -- auditToken, sections (JSON), completedAt
  • AuditScoreHistory -- healthScore, errorCount, warningCount
  • SiteAuditRun -- status, taskId (DataForSEO), pagesProcessed, crawlError
  • SiteAuditPage -- url, title, h1, wordCount, statusCode, crawlHealth
  • SiteAuditIssue -- type, severity (error|warning|notice), affectedUrls (JSON), evidence (JSON), status
  • TechnicalAudit, TechnicalIssue, PerformanceAudit, PerformancePageScore

AI Visibility + Market Position

  • AiVisibilityCheck, AiVisibilityQuery, AiVisibilitySnapshot
  • ProbeRun -- vertical, locale, engine, status (enum ProbeRunStatus: PENDING|RUNNING|COMPLETED|FAILED), sampleCount
  • VisibilityObservation -- probeRunId, query, engine, brandKey, mentioned, position, cited, sentiment, sourceUrls[], sampleIndex
  • CompetitorPanel -- vertical, locale, members[]; unique (vertical, locale)
  • QueryCorpusEntry -- vertical, locale, query, source (enum QuerySource: FRAGMENT_SEEDED|CATEGORY_COVERAGE|CURATED), conversionWeight
  • CalibrationRun -- vertical, version, coefficients (JSON), fittedAt, sampleSize
  • MarketPositionSnapshot -- brandId, vertical, locale, calibratedScore, conversionWeightedSov, rawSov, rankInPanel, panelSize, divergence, stabilityBand, computedAt

Execution + Content

  • Task -- title, category, status, priority, source, sourceId, dueDate, pipelineStage, briefId, assignedTo
  • ContentBrief -- type (seo|llmo), status, primaryKeyword, sections (JSON), humanizationMode, contentScore
  • SchemaMarkup -- type, markup (JSON-LD), appliedPages (JSON)

Page Lifecycle

  • PageLaunch, PageLaunchSnapshot -- baseline vs current GSC metrics
  • DeadPage, DeadPageSource -- 404 detection + redirect suggestions

Competitors

  • Competitor, CompetitorSnapshot, CompetitorRanking, CompetitorPage, CompetitorDiscoveryJob

Backlinks

  • Backlink -- sourceDomain, anchorText, linkType, toxicityScore, qualityTier, status (active|lost|disavowed)
  • BacklinkSnapshot -- totals, avgDA, toxicCount

GSC + GBP

  • GscQuery, GscPage -- daily click/impression/position
  • GbpLocation, GbpReview, GbpMetric

Rank Tracking

  • TrackedKeyword -- keyword, location, device, isActive
  • KeywordRanking -- position, previousPosition, serpFeatures[], source

CRM + Revenue

  • CrmDeal, RevenueAttribution, UserJourney

Email + Notifications

  • EmailReportSchedule, SentEmailReport, AlertRule, Notification

White-Label + Portal

  • WhiteLabelConfig -- orgId (unique), customDomain, logoUrl, primaryColor, removeMeseoBranding
  • CustomDomain -- hostname (unique), status (enum DomainStatus: PENDING|VERIFIED|FAILED), verificationToken
  • ClientPortal -- brandId (unique), allowedPages[], hideSourceBranding
  • ClientUser -- email, role, status, accessToken

AI Strategist

  • AnalysisJob -- type, status, prompt, result (JSON), shareToken, clientVersion (JSON), hiddenSections (JSON)
  • StrategistSession, StrategistMessage

Logging + Telemetry

  • AiUsageLog -- provider, model, feature, inputTokens, outputTokens, estimatedCost
  • AiInteraction -- userId, brandId, feature, userPrompt (4KB), aiResponse (16KB), guardrailFlags[], flagSeverity
  • PlatformEvent, PlatformLog, ComplianceEvent

Predictions + Other

  • Prediction, DashboardSnapshot, IndustryBenchmark, AdCampaignData, AdClickEvent, AdPlatformIntegration

Cron jobs (34 routes under /api/cron/)

Route Schedule maxDuration Key models Notes
site-tag-rollup Hourly 0 * * * * 300s TrackedEvent, SiteTagDailyRollup Pre-aggregates last 30 days. Force-recomputes today+yesterday. Commit 3d1c64e.
site-audit-cleanup Every 5 min -- SiteAuditRun Re-kicks stuck >15 min; fails >2 hours. Commit 6a69290.
rank-check Daily 300s TrackedKeyword, KeywordRanking DataForSEO live SERP. Soft cap $5/day, hard cap 100 keywords/brand.
ai-visibility Weekly 0 0 * * 1 300s AiVisibilityQuery, AiVisibilitySnapshot 7-source mention checks.
signal-detection Hourly -- MetricSnapshot, ConversionEvent Anomaly detection -> AlertRule -> Notification.
dedup-conversions Daily -- SiteConversion Deterministic cluster dedup + classifiedChannel backfill.
ai-citation-check Weekly 300s AiCitation Citation verification.
backlink-sync Daily -- Backlink, BacklinkSnapshot DataForSEO new/lost + toxicity scoring.
gbp-sync Daily -- GbpLocation, GbpReview, GbpMetric Google Business Profile sync.
email-reports Per schedule (nextSendAt) -- EmailReportSchedule, SentEmailReport Assemble + send scheduled reports.
industry-intel Daily -- IndustryBriefing External event correlation.
competitor-crawl Weekly -- Competitor, CompetitorSnapshot Sitemap monitoring, threat scoring.
paid-media-rebuild Daily -- AdCampaignData Ad spend + conversion reconciliation.
track-outcomes Daily -- Task, RecommendationOutcome ROI scoring for completed recommendations.
ai-strategy-recommendations Weekly -- AnalysisJob Re-generate AI recs for all brands.
data-retention Daily -- TrackedEvent, SiteConversion, SiteEvent Delete rows older than brand.dataRetentionDays (default 395 days).
nango-sync Hourly -- BrandIntegration (Nango) OAuth token refresh.
refresh-tokens Daily -- BrandIntegration (Google) Google OAuth refresh.
check-alerts Every 5 min -- AlertRule, Notification Evaluate rules, fire notifications.
compute-benchmarks Weekly -- IndustryBenchmark Platform-wide industry median aggregation.
brand-health Daily -- Brand (healthScore) Synthetic health score + email on decline.
platform-snapshot 1st of month -- PlatformSnapshot Monthly MAU/DAU/churn/feature-adoption archive.
cleanup-stale-audits Daily -- Audit, TechnicalAudit Delete stale >90 days.
precompute-dashboard Every 30 min -- MetricSnapshot Pre-aggregate KPIs with budget guard. Commit c376913.
integration-sync Hourly -- BrandIntegration, GSC/GA4/GBP Sync search data from connected integrations.
signals-competitor-regional Hourly -- Competitor, CompetitorSnapshot Regional threat detection.
signals-site-tag-ai Hourly -- SiteConversion, TrackedSession AI signal detection from behavioral data.

Gaps and unknowns: Schedule values are from comments in each route file; vercel.json was not read. competitor-discovery-queue and brand-sitemap-crawl exist but their schedules and enabled status were not confirmed.

10. Risk Register

Items not yet fully converted to rollup/bounded paths.

Active risks

R1 -- reconcileTrackedEvents is unbounded (fire-and-forget, pool risk)

  • File: src/lib/services/site-tag-reconciliation.ts lines 152-188
  • Fetches up to 500 TrackedEvent rows with NO date filter. For a high-volume brand, the oldest 500 rows are weeks/months old; each unmatched event fires 1 findFirst + 1-2 create = up to ~1500 serial round-trips.
  • Status: Fire-and-forget (void) so it does not block HTTP response, but holds DB connections.
  • Fix: Add timestamp: { gte: window48h } to the findMany.

R2 -- SourceEvent lacks a composite (trackedSiteId, timestamp) index

  • File: prisma/schema.prisma
  • SourceEvent only has @@index([trackedSiteId, source]) and @@index([trackedSiteId, isDuplicate]). The 48h bound (07e37ef) filters on timestamp but without a (trackedSiteId, timestamp) index, Postgres must post-filter.
  • Risk level: Medium (48h bound still dramatically reduces scan; not as fast as it could be).
  • Fix: Add @@index([trackedSiteId, timestamp]) to SourceEvent.

R3 -- hourlySample findMany on TrackedEvent (setup page)

  • File: src/app/api/site-tag/route.ts
  • prisma.trackedEvent.findMany({ where: { trackedSiteId, timestamp >= todayStart }, take: 1000 }). Runs on every 30s poll.
  • (trackedSiteId, timestamp) index exists so this should be fast, but for very high volume brands the today-bounded scan still reads a large index range before the take: 1000 stops it.
  • Risk level: Low. Monitor via ?debugTiming=1.

R4 -- customEventsList fallback: 90-day TrackedEvent.groupBy

  • File: src/app/api/site-tag/route.ts
  • If SiteTagDailyRollup has no rows (new brand, cron lag), falls back to trackedEvent.groupBy over 90 days.
  • Risk level: Low for established brands. High for new brands if cron is delayed.

R5 -- domainStatus per-domain count queries

  • File: src/app/api/site-tag/route.ts
  • For each allowed domain: 2 parallel queries (count + findFirst). With startsWith (sargable on (trackedSiteId, pageUrl, timestamp) index) and Promise.all, this is acceptable for 1-3 domains.
  • Risk level: Low for 1-3 domains. Medium for brands with 10+ allowed domains.

R6 -- Live TrackedEvent aggregate fallback when rollup missing

  • File: src/lib/services/site-tag-analytics.ts
  • If rollupRows.length < expectedDays - 1, falls back to full-range TrackedEvent group-by scans.
  • Risk level: Low for steady-state brands. Medium for brands recovering from cron failures.

R7 -- getEventTrends always runs live (intentional revert)

  • Commit: 40990aa revert(getEventTrends): remove rollup early-return, always run live query
  • 30-day TrackedEvent.groupBy per call.
  • Risk level: Medium -- depends on call frequency and brand volume.

R8 -- getSiteTagInstalledAt and getSiteTagMaturity internals not confirmed

  • Called in Promise.all at end of setup page handler (outside withDeadline).
  • Risk level: UNCONFIRMED.

R9 -- site-tag-rollup cron processes ALL brands serially

  • File: src/app/api/cron/site-tag-rollup/route.ts
  • Outer loop: for site of sites, inner loop: for date of datesToProcess. 9 DB round-trips per iteration. At 100 brands x 30 days = 2700 iterations x 9 = 24,300 DB calls within 300s maxDuration.
  • Risk level: Medium at scale. Would fail silently (partial rollup) if maxDuration hit.

Resolved risks (this session and prior)

  • activePagesCount 7-day unbounded CASE expression -- resolved d6dfed4
  • getReconciliationSummary 9 unbounded full-table scans -- resolved 07e37ef
  • recomputeSessionScore in ingest hot path -- resolved accf550

11. Patent Map

Site Tag ID system

Status: WORKING IMPLEMENTATION

  • public/t.js + source public/t.source.js -- served from /api/site-tag/script/[name]
  • Session stitching: TrackedSession with predictedConversionScore ML field
  • Dedup: submissionId token, rapid-fire query, SHA1 clustering, isDuplicateOf pointer
  • Attribution: touchpoints JSON, firstTouchChannel, lastTouchChannel, classifiedChannel
  • Platform detection: detectedPlatform JSON on TrackedSite (name, version, confidence, category, hosting)
  • Plugin detection: plugin_detected events with plugins[] payload

Multi-source dedup + CanonicalEvent

Status: WORKING IMPLEMENTATION

  • CanonicalEvent + SourceEvent models; 5-source reconciliation (site_tag|ga4|gtm|gsc|crm)
  • ingestSourceEvent(): +-5 min match window, confidence boosting (75 + sourceCount*8, max 100)
  • computeDedupKey(): SHA1 hash, deterministic survivor selection
  • Daily dedup-conversions cron for backfill
  • Note: reconcileTrackedEvents still has unbounded oldest-first scan (R1)

Conversion funnel (multi-tier)

Status: WORKING IMPLEMENTATION

  • CONVERSION_TIERS in src/lib/conversions/tiers.ts: 41 conversion types across 3 tiers (completed|intent|signal)
  • BrandConversionConfig: per-type override (isCounted, tierOverride, displayLabel)
  • SiteConversion: full funnel data including touchpoints, value, orderId, lineItems (JSON)
  • Configure Conversions modal in Site Tag Analytics UI

AI Strategist

Status: WORKING IMPLEMENTATION

  • POST /api/ai-strategist/chat (120s, claude-sonnet-4-6)
  • Context: brand profile + GSC + Site Tag + AnalysisJob.result + paid media + LLMO
  • Artifact renderer: JSX dashboard generation in iframe sandbox
  • Session persistence + guardrail logging

AI Visibility / LLMO attribution

Status: WORKING IMPLEMENTATION (base), PARTIAL (market position calibration)

  • AiVisibilityCheck, AiVisibilityQuery, AiVisibilitySnapshot -- fully wired
  • Weekly cron, 7-source visibility checker using Anthropic web_search_20250305 tool
  • Market Position Block 1+2: schema + corpus seeding implemented (cdb15a9, c735d74)
  • Calibration runner producing MarketPositionSnapshot.calibratedScore: schema-only, no runner service confirmed

Heatmap

Status: WORKING IMPLEMENTATION

  • HeatmapEvent model, POST /api/site-tag/heatmap, GET /api/site-tag/analytics/heatmap
  • UI tab on Site Tag Analytics

Execution Hub

Status: WORKING IMPLEMENTATION

  • Task model: 7 categories, pipeline stage, briefId, assignedTo
  • Calendar view + pipeline board
  • AI explanations per task

Audit Runner (7-phase pipeline)

Status: WORKING IMPLEMENTATION

  • 2038-line orchestration, 26 dimension aggregators, 7 synthesizers
  • $4/$2 cost ceilings; prompt caching; degraded mode
  • 14-industry lens, vendor confidentiality enforced

Report Studio

Status: WORKING IMPLEMENTATION

  • Section reordering, custom assembly
  • PDF, DOCX, Markdown, JSON, PPTX exports
  • @react-pdf/renderer for audit PDFs (comprehensive + executive modes)

Multi-tenant / White-label

Status: WORKING IMPLEMENTATION

  • WhiteLabelConfig: per-org logo, colors, domain, removeMeseoBranding
  • CustomDomain: PENDING|VERIFIED|FAILED with Vercel domain integration
  • ClientPortal: allowedPages[], hideSourceBranding, per-user roles

Local GEO

Status: SCHEMA + CRON READY, UI STUBBED

  • GbpLocation, GbpReview, GbpMetric + daily gbp-sync cron active
  • /local-geo intentionally shows "Coming Soon"
  • Local ranking and AI search citation at location level: ABSENT

Gaps and unknowns:

  • Market Position calibration runner: Block 3+ (models + seeding exist, runner not found)
  • PageLaunchSnapshot data population mechanism (GSC direct vs MetricSnapshot): UNCONFIRMED
  • Inngest: infrastructure only, no production functions beyond ping
  • Voice search synthesizer call path not fully traced from audit-runner.ts

End of inventory. All code paths confirmed via file reads or agent exploration unless marked UNCONFIRMED or ABSENT. Commit hashes reference dmazzei-star/meseo main branch as of 2026-06-10 HEAD 07e37ef.

Reference in New Issue View Git Blame Copy Permalink