Data Flow

This document traces every major data path through the Lens system — from user action to rendered pixels and back. Understanding these flows is essential for debugging and extending the application.

1. Load Cycle: Opening a Lens

When a user opens a lens, the entire view is assembled server-side and delivered as a single payload.

User clicks lens card (LensMenu)
  |
  v
LensView.tsx mounts
  |
  v
api.getLensView(lensId)
  |
  v
@forge/bridge invoke('getLensView', { lensId })
  |
  +---------- crosses Forge iframe boundary ----------+
  |                                                    |
  v                                                    v
Resolver: getLensViewHandler                    (25s timeout)
  |
  |  1. Permission check
  |     a. Fetch lens row from `lenses` table
  |     b. Compare context.accountId with owner_account_id
  |     c. If not owner: query `lens_permissions` for matching grants
  |     d. Reject if no 'view' or higher permission
  |
  |  2. Load hierarchy tree
  |     a. SELECT * FROM hierarchy_nodes WHERE lens_id = ?
  |        ORDER BY depth ASC, position ASC
  |     b. Returns nodes in breadth-first-like SQL order
  |
  |  3. Enrich from issue cache
  |     a. Collect all jira_issue_id values from issue nodes
  |     b. Batch SELECT from issue_cache (chunked for large sets)
  |     c. Attach issueData to each node
  |
  |  4. Filter by Jira BROWSE permission
  |     a. Build JQL: "key IN (KEY-1, KEY-2, ...)"
  |     b. Execute as api.asUser() -- only returns issues user can see
  |     c. Remove nodes for issues not in the result set
  |     d. Cascade: hidden parent --> hidden children
  |     e. Cache permission results in KVS (30-min TTL)
  |     f. Fail-open: if Jira API unreachable, assume all visible
  |
  |  5. Assemble response
  |     a. Load views for this lens
  |     b. Load sync agents
  |     c. Build response: { rows, columns, views, totalCount, syncAgents }
  |
  v
Response returned to frontend (< 5 MB)
  |
  v
LensView receives response
  |
  |  1. buildFlatRows(nodes)
  |     a. Depth-first tree walk (NOT the SQL breadth-first order)
  |     b. Produces flat array matching AG Grid's expected row order
  |     c. Each row has: nodeId, depth, parent_id, issueData, etc.
  |
  |  2. Set state: setNodes(flatRows)
  |
  |  3. buildColumnDefs(view.columns)
  |     a. Maps column config to AG Grid ColDef objects
  |     b. Registers cell renderers, editors, formatters
  |
  v
AG Grid renders the hierarchy
  (tree data, expand/collapse, inline editors registered)

Key points:

The frontend makes exactly ONE resolver call to display an entire lens.
Zero Jira API calls are needed to render cached data.
The entire permission check, data assembly, and filtering happens server-side.
Response payload is bounded by the 5 MB Forge limit.

2. Edit Cycle: Inline Editing a Cell

When a user edits a cell in the grid, the change flows to Jira and back through the cache.

User double-clicks a cell
  |
  v
AG Grid activates cell editor (one of 24 editor types)
  |
  v
User modifies value, presses Enter (or clicks away)
  |
  v
onCellEdit callback fires in HierarchyGrid
  |
  |  1. Optimistic update
  |     a. Update node in local state immediately
  |     b. User sees new value with no delay
  |
  v
api.updateField(lensId, issueKey, field, value)
  |
  +---------- crosses Forge iframe boundary ----------+
  |
  v
Resolver: updateFieldHandler
  |
  |  1. Permission check (edit or higher required)
  |
  |  2. Write to Jira
  |     a. api.asUser().requestJira(PUT /rest/api/3/issue/{issueKey})
  |     b. Costs 2 API points
  |     c. Jira validates the change (workflow rules, permissions, etc.)
  |
  |  3. Update issue cache
  |     a. Upsert the changed field in issue_cache table
  |     b. Cache is now consistent with Jira
  |
  |  4. Publish Realtime event (if connected clients)
  |     a. channel: `lens:{lensId}`
  |     b. event: { type: 'field_updated', issueKey, field, value }
  |
  v
Return { success: true } to frontend
  |
  v
Frontend confirms optimistic update
  (or reverts if the write failed)

Field-specific behaviors (8 core editors):

Field	Editor Component	Jira API Endpoint	Notes
Summary	`TextEditor`	PUT `/issue/{key}`	Standard text edit
Status	`StatusDropdown`	POST `/issue/{key}/transitions`	Must use transitions API, not direct field edit
Priority	`PriorityDropdown`	PUT `/issue/{key}`	Dropdown of valid priorities
Assignee	`AssigneePicker`	PUT `/issue/{key}`	User search via `searchAssignableUsers`
Sprint	`SprintDropdown`	PUT `/issue/{key}`	Active + future sprints only
Due Date	`DatePicker`	PUT `/issue/{key}`	Calendar date picker
Story Points	`NumberInput`	PUT `/issue/{key}`	Numeric input
Labels	`LabelsEditor`	PUT `/issue/{key}`	Multi-value label input

There are 16 additional editors for custom field types (components, versions, groups, users, cascading selects, URLs, issue links, flagged toggle, reporter, datetime, and more). See Inline Editing Guide for the complete reference.

Error handling:

If the Jira write fails (permissions, workflow rules, network error), the frontend reverts the cell to its previous value.
A toast notification informs the user of the failure.
The optimistic update is only persisted if the backend confirms success.

3. Event-Driven Cache Sync

The issue cache is kept fresh by Jira product events. These events are free — they do not consume API rate limit points.

Someone edits an issue in Jira (or another app does)
  |
  v
Jira fires product event (free, no API points)
  avi:jira:created:issue    (trigger key: ic)
  avi:jira:updated:issue    (trigger key: iu, ignoreSelf: true)
  avi:jira:deleted:issue    (trigger key: id)
  avi:jira:created:issuelink (trigger key: ilc)
  avi:jira:deleted:issuelink (trigger key: ild)
  |
  v
issue-handler.ts receives the event
  |
  |  For created/updated:
  |  1. Extract issue key + changed field names from event payload
  |  2. Fetch full issue from Jira API (2 API points)
  |  3. Parse all fields including custom fields
  |  4. Upsert into issue_cache (ON DUPLICATE KEY UPDATE)
  |  5. Check which lenses reference this issue's project
  |  6. Trigger auto-sync for affected lenses (Layer 2 of auto-sync)
  |     - Pushes to generator-queue for background execution
  |  7. Publish Forge Realtime event to affected lens channels
  |
  |  For deleted:
  |  1. Delete from issue_cache
  |  2. Delete corresponding hierarchy_nodes rows
  |  3. Publish Realtime event
  |
  v
Connected frontends receive Realtime event
  |
  v
Frontend: re-fetch or incremental patch
  (depending on change type)

Auto-sync three-layer model:

Layer	Trigger	Mechanism	Latency
Layer 1	User opens a lens	Per-project staleness gate checks if cache is stale	On-demand
Layer 2	Issue event fires	Handler pushes re-sync for lenses with matching project generators	3-10 seconds
Layer 3	Timer	Frontend count-check safety net every 60 seconds	Up to 60 seconds

Sprint events (avi:jira:started/closed/updated:sprint) trigger sprint-handler.ts, which updates sprint fields on all cached issues belonging to that sprint.

4. Generator (Sync Agent) Execution Flow

Sync agents are the automated hierarchy builders. They use JQL to find issues and populate the hierarchy.

Trigger (manual "Run Sync Agents" or auto-sync)
  |
  v
api.executeSyncAgents(lensId)
  |
  v
Resolver: executeSyncAgentsHandler
  |
  |  1. Load all generators for this lens
  |  2. Create a job record (status: 'running')
  |  3. Push to generator-queue (async consumer)
  |  4. Return job ID immediately to frontend
  |
  v
Frontend: begins polling getJobStatus(jobId) every 3 seconds
  |
  v
Async Consumer: sync-agent-executor.ts (15-min timeout)
  |
  |  For each generator (ordered by position):
  |
  |  [JQL Insert Generator]
  |  1. Execute JQL query via Jira API (paginated, 100 per page)
  |     - Cost: 1 point per page + 1 point per result
  |     - Example: 200 results = 201 points (2 pages + 200 issues)
  |  2. For each matched issue:
  |     a. Upsert into issue_cache
  |     b. Create hierarchy_node if not already in lens
  |     c. Attach under generator's parent node
  |  3. Track API points consumed
  |
  |  [Child Extend Generator]
  |  1. For each issue node in the parent scope:
  |     a. Fetch child issues from Jira API (2 pts per parent)
  |     b. Upsert children into issue_cache
  |     c. Create child hierarchy_nodes
  |  2. Repeat for configured depth (1-3 levels)
  |
  |  [Other generator types: extend_linked, jpd_links, etc.]
  |
  |  After all generators complete:
  |  1. Update job status to 'completed' (or 'error')
  |  2. Store DiffAccumulator results in KVS
  |  3. Publish Realtime event: { type: 'sync_completed', lensId }
  |
  v
Frontend: getJobStatus returns 'completed'
  |
  |  1. Fetch getLensDiff for incremental update
  |  2. Apply diff: add new rows, remove deleted rows
  |  3. New rows get 2-second blue highlight animation
  |  4. Toast summarizes changes: "Added 12 issues, removed 3"
  |  5. If diff is too large: fall back to full reload
  |
  v
Grid updated incrementally (no full page reload)

Rate limit budgeting during generator execution:

Each API call’s point cost is tracked via KVS counter
If budget drops below threshold, execution pauses or degrades
Budget status is surfaced to the user via getBudgetStatus resolver

5. Row Ordering Invariant (Critical)

This is a critical invariant that has caused major bugs when violated. Any component that renders content aligned with grid rows must use the same ordering.

The Problem

The backend returns hierarchy nodes in SQL order: ORDER BY depth ASC, position ASC. This produces a breadth-first-like ordering:

SQL order (breadth-first-like):
  Epic-1 (depth 0, position 100)
  Epic-2 (depth 0, position 200)
  Story-1 (depth 1, position 100, parent: Epic-1)
  Story-2 (depth 1, position 200, parent: Epic-1)
  Story-3 (depth 1, position 100, parent: Epic-2)

The grid uses buildFlatRows() which performs a depth-first tree walk:

Grid order (depth-first):
  Epic-1 (depth 0)
    Story-1 (depth 1, parent: Epic-1)
    Story-2 (depth 1, parent: Epic-1)
  Epic-2 (depth 0)
    Story-3 (depth 1, parent: Epic-2)

The Bug (Fixed 2026-03-02)

The Gantt chart’s extractGanttItems was preserving SQL order, causing timeline bars to render at wrong Y positions. 495 out of 497 items were misaligned — bars appeared to be “missing” but actually existed at wrong row indices.

The Rule

Every component that renders rows alongside the grid MUST use the same depth-first tree-walk ordering, not the raw SQL order. The Gantt’s extractGanttItems now performs this reordering internally.

This applies to:

Gantt/Timeline bars (row Y position)
Any future side panel that aligns with grid rows
Export functions that need row-aligned output
Print layouts

6. Permission Flow

Permission checks happen at two levels: lens-level ACL and Jira issue-level visibility.

Lens-Level Permissions

Request arrives at resolver
  |
  v
withLensPermission(requiredLevel) guard
  |
  |  1. Extract accountId from Forge context
  |  2. Load lens row from `lenses` table
  |  3. If accountId === owner_account_id --> Control (pass)
  |  4. Check if user is Jira admin --> Control (pass)
  |  5. Query lens_permissions:
  |     WHERE lens_id = ?
  |     AND (
  |       (grantee_type = 'user' AND grantee_id = accountId)
  |       OR (grantee_type = 'group' AND grantee_id IN userGroups)
  |       OR (grantee_type = 'role' AND grantee_id IN userRoles)
  |       OR (grantee_type = 'everyone')
  |     )
  |  6. Take highest permission from all matching grants
  |  7. Compare: userLevel >= requiredLevel
  |
  v
Pass: continue to handler  |  Fail: return { error: 'forbidden' }

Permission hierarchy: view < edit < control

Issue-Level Visibility (BROWSE Check)

getLensView assembles row set
  |
  v
Collect all issue keys from hierarchy nodes
  |
  v
Batch JQL check: "key IN (KEY-1, KEY-2, ...)" as api.asUser()
  |
  |  Jira returns only issues the user can BROWSE
  |  Missing keys = user cannot see those issues
  |
  v
Filter: remove nodes for issues not in result set
  |
  |  Cascading: if a parent is hidden, all children are hidden too
  |
  v
Cache results in KVS (30-min TTL per project per user)
  |
  v
Return filtered row set to frontend

Fail-open strategy: If the Jira API is unreachable (network error, rate limit exhaustion), the system assumes all issues are visible. This prevents a Jira outage from rendering Lens completely unusable. The tradeoff is accepted because:

Forge SQL data isolation means cross-tenant leakage is impossible
A user who has lens-level view permission has already been granted access by the lens owner
The alternative (fail-closed) would make the app unusable during any Jira API disruption

7. Realtime Update Flow

Forge Realtime enables live UI updates when data changes outside the current user’s session.

Backend event (sync completion, issue update, etc.)
  |
  v
realtime.ts: publish to channel `lens:{lensId}`
  |
  v
Forge Realtime infrastructure
  |
  v
All connected frontends subscribed to that channel
  |
  v
useRealtime hook receives event
  |
  |  Event types:
  |  - hierarchy_changed  --> smartRefresh() or full loadData()
  |  - field_updated      --> patch single node in state
  |  - sync_completed     --> fetch diff, apply incrementally
  |  - lens_modified      --> auto-switch views, refresh state
  |
  v
UI updates without page refresh

Realtime auth: The frontend requests a channel token via getRealtimeToken resolver. Tokens are scoped to specific channels and have limited TTL.

8. Import Flow

Lens supports multiple import sources, all following a preview-then-execute pattern.

User opens ImportHub (/import)
  |
  v
Selects import source:
  - CSV/Excel file
  - Competitor Cloud (direct API)
  - Competitor DC (extractor JSON)
  - Foundation DC (extractor JSON)
  - Bulk Add by Issue Key
  |
  v
Preview step:
  1. Parse/validate input
  2. Show mapped columns, hierarchy structure, issue count
  3. User confirms or adjusts mapping
  |
  v
Execute step:
  1. Push to import-queue (async consumer, 15-min timeout)
  2. Frontend polls import status
  3. Consumer creates lens, hierarchy nodes, populates cache
  4. Status updates: { progress: 45, message: "Importing issues..." }
  |
  v
Import complete:
  1. Navigate to new lens
  2. Toast: "Imported 150 issues into 'Q1 Roadmap'"