# AckNexa product guide

Reviewed 2026-11-15.

## Meet AckNexa

A practical introduction to the product, its concepts, and the people who use it.

### From a monitoring signal to a coordinated response

AckNexa is an incident management and on-call platform. It receives monitoring events, normalizes and deduplicates alerts, routes them to owned services, finds the current responder, and runs escalation policies. Responders acknowledge and resolve incidents through the Web application or the iOS and Android apps, with a shared history of what happened.

Around the response, AckNexa can keep customers informed on status pages, send updates to stakeholders, open Jira or Halo tickets, route hotline calls to whoever is on call, and record a postmortem with action items.

The core runs on customer-controlled infrastructure. You do not need a vendor-hosted AckNexa control plane, a public cloud account, or an external notification API to ingest incidents, coordinate response, and use the Web UI.

> **Know the release status**
> Release 1.0.0-rc.1 is a release candidate for pilot deployments, not yet a general-availability release. Before general availability it still needs a real-world pilot, an external penetration test, SCIM validation against a live identity-provider tenant, store-distributed mobile apps and validation against a real CUCM cluster. This guide does not claim an availability SLA or compliance certification. Read [Capabilities & limitations](/docs/capabilities/).

### Choose your path

| Your role | Start with | Then explore |
| --- | --- | --- |
| Responder | [Respond to an incident](/docs/incident-response/) | [On-call schedules](/docs/on-call/), [Mobile access](/docs/mobile/) and [Postmortems](/docs/postmortems/) |
| Organization administrator | [First organization](/docs/getting-started/) | [Users & access](/docs/users-access/), [Services & teams](/docs/services-teams/) and [Status pages](/docs/status-pages/) |
| Platform operator | [System requirements](/docs/system-requirements/), then [Deployment](/docs/deployment/) | [Security](/docs/security/), [Observability](/docs/observability/), and [Backup & recovery](/docs/backup-recovery/) |
| Integration developer | [Connect monitoring](/docs/integrations/) | [Routing & deduplication](/docs/routing/) and [API guide](/docs/api/) |

### The product model

| Concept | Purpose |
| --- | --- |
| Organization | The tenant boundary for users, configuration, alerts, incidents, and audit records. |
| Team | A group of people responsible for one or more services. |
| Service | An operational system with an owning team, escalation policy, and response context. |
| Integration | An authenticated inbound connection from a monitoring source to AckNexa. |
| Alert | The normalized state of a monitoring condition across its occurrences. |
| Incident | The actionable response record, with ownership, state, timeline, and notification history. |
| Schedule and policy | A schedule answers who is on call; an escalation policy answers who to contact and when. |

### What this public site is

This site is product documentation, not a hosted AckNexa tenant, customer portal, or live status page. AckNexa status pages are served by your own installation. This site does not require a login and does not connect to an installation. Examples use fictional data. Your deployment administrator supplies the actual application address and access.

To reach the AckNexa team, write to hello@acknexa.com.

---

## Create your first organization

A complete path from a fresh installation to a test incident acknowledged by a real responder.

### Before you begin

- Ask your platform operator for the application URL and first-run setup token. The token must be supplied by the operator; there is no public self-registration or universal production password.
- First-run setup creates only the first organization of a deployment. The operator creates further organizations with the bootstrap command in the backend image; see [Users and access](/docs/users-access/#several-organizations-on-one-deployment).
- Confirm PostgreSQL migrations and the API, workers, and Web application are running. See [Deployment](/docs/deployment/).
- Choose an organization name and slug, an IANA timezone such as Europe/London, and an initial administrator.
- Identify at least one responder and a notification channel that works in your network. Internal SMTP is an option for isolated installations.

### Create the administrator

1. Open the Web application. An empty installation shows first-run setup only when the operator has enabled it with a setup token.
2. Enter the setup token, organization details, administrator email/display name, timezone, and a strong password. Passwords must be 12–72 bytes.
3. Create the organization, then sign in using its slug, email, and password. The organization administrator sees configuration navigation.
4. Have the operator retire the setup token after initialization. It is not an ongoing user invitation or tenant registration mechanism.

### Build the response chain

1. In Users, create active responders and set their roles, timezone, contact details, and notification preferences.
2. In Teams, create the owning team and assign its members.
3. In On Call, create a primary schedule with a rotation that covers the current time. Check the selected responder.
4. In Escalation Policies, create a policy with at least one valid target, delay, and available channel.
5. In Services, create an active service, assign its team and policy, and add a runbook URL if available.
6. Configure a Notification Provider or ask the operator to supply a deployment-level provider. Send a provider test to the intended responder.
7. Create an Integration for the service. Copy its webhook secret immediately and store it securely; it cannot be read back as plaintext.

### Prove the whole workflow

1. Use the integration's test action with a representative payload, or send a real authenticated webhook from your monitoring source.
2. Follow the test's processing state. Inspect the created incident, matched routing information, on-call responder, escalation history, and notification result.
3. Have the actual responder acknowledge the incident. Provider acceptance is not the same as a human acknowledgement.
4. Complete the guided onboarding flow after its prerequisites are satisfied, then resolve the test incident with a clear reason.
5. Repeat using your real monitoring source and selected notification channels before enabling production alert traffic.

> **Do not stop at HTTP 202**
> 202 means the raw event was durably accepted, not that routing, notification delivery, or acknowledgement has completed. Check the incident and delivery history.

### If setup or admin navigation is missing

Setup is not available after an organization exists. Admin navigation requires organization_admin; a team_admin is not an organization administrator. After a role change, sign out and back in. If the UI is old, rebuild and recreate the relevant Web/API images: restarting a container does not rebuild it. See [Troubleshooting](/docs/troubleshooting/).

---

## Find your way around

Understand the Web workspace and what each area is for.

### Everyday response

| Area | Use it to |
| --- | --- |
| Overview | Review the current incident picture and operational context, and schedule or end maintenance windows. |
| Incidents | Open a response record, inspect its timeline, acknowledge, resolve, add notes, publish status updates, open a ticket, or take permitted response actions. |
| Alerts | Inspect normalized monitoring conditions and repeated occurrences. |
| On Call | Inspect coverage and current responders; organization admins can configure schedules. |
| My Shifts | See your upcoming shifts, ask for or offer cover, and create a calendar feed. |
| Postmortems | See which incidents owe a review and write or read postmortems. |

### Configuration

| Area | Use it to |
| --- | --- |
| Services / Teams | Understand and manage ownership, runbooks, escalation relationships, stakeholders and chat channels. |
| Escalation Policies / Routing Rules | Decide who is reached and which service owns an alert. |
| Integrations / Heartbeats / Webhooks | Connect monitoring sources, page on silent jobs, and send incident events to your own systems. |
| Stakeholders / Status Pages | Keep people who are not paged informed. |
| Call Routing / Ticketing | Connect a hotline number, and open Jira or Halo tickets from incidents. |
| Notification Providers | Configure the organization's delivery channels and run provider tests. |

### Administration

| Area | Use it to |
| --- | --- |
| Reports | Review incident, acknowledgement and paging trends per service, team and person. |
| Users / API Keys / RBAC | Manage people, automation tokens and role-based access. |
| Identity | Configure single sign-on, directory sign-in, group mappings and SCIM provisioning. |
| Audit | Review, export or stream organization-scoped configuration and response actions. |
| Delivery Health | Inspect processing failures, quarantined events, and notification problems. |
| Setup / Settings | Inspect guided onboarding progress, and manage organization settings such as timezone, default severity and postmortem expectations. |

### Refresh, permissions, and stale state

The backend is authoritative. An acknowledgement is successful only when the API confirms it. Concurrent actions may return a conflict; refresh the incident and inspect its new version before retrying. Hidden controls do not grant access, and a visible URL cannot bypass backend authorization.

> **Current UI boundaries**
> Organization-wide configuration screens are restricted to organization administrators. Team admins can manage some items for their own teams, such as heartbeats, stakeholders, chat channels and maintenance windows. Some lists and previews are bounded; do not treat a short UI preview as an exhaustive reporting or coverage audit. Use approved operational checks for release acceptance.

---

## Respond to an incident

Triage, acknowledge, escalate, and resolve without losing the response history.

### Triage with context

1. Open the incident from the incident list or notification link. Confirm the service, severity, affected resource, environment, and source timestamps.
2. Review the owning team, current on-call responder, runbook, related alerts, and matched routing rule. A repeated alert is not necessarily a new incident.
3. Inspect the timeline and notification history before acting. Another responder may already have acknowledged or resolved it.

### Incident states

| State | Meaning | Next action |
| --- | --- | --- |
| TRIGGERED | An active response is needed and the escalation policy may be running. | Acknowledge if taking ownership; investigate or escalate as needed. |
| ACKNOWLEDGED | A responder has accepted responsibility. This does not mean the system has recovered. | Investigate, coordinate the fix, and resolve when appropriate. |
| RESOLVED | The incident lifecycle has been closed manually or by applicable source recovery. | Review the history; a later alert lifecycle can create another incident. |

### Acknowledge

Select Acknowledge when you are taking responsibility. AckNexa records the actor and time and cancels pending escalation jobs and unsent notifications in the same transaction. A notification request already in flight can still complete; a late message does not necessarily mean the ACK failed.

> **Conflicting actions**
> If the incident changed after you opened it, the API can return 409. Refresh, confirm the latest state and owner, and act again only if still necessary. Do not repeatedly submit stale versions.

### Escalate and resolve

Use the permitted escalation or assignment controls when more help is needed. Check that the destination is an active responder or valid policy target. Changes are visible in the incident history.

Resolve only when the response can be closed, and provide a meaningful resolution reason. A valid source recovery can also resolve the lifecycle. Older, out-of-order source data must not replace newer state. Manual resolution does not make a still-firing monitor healthy.

### Notes, updates and tickets

- Add notes to record what you checked. Notes stay internal unless you tick Send to stakeholders, which posts the note as an update to everyone following the service or team. See [Status pages and stakeholders](/docs/status-pages/).
- Publish a customer-facing update from the incident's Status page update panel when the service is listed on a status page.
- Open a Jira or Halo ticket with Create ticket when your organization has a ticketing connection. See [Ticketing](/docs/ticketing/).
- After resolution, start the review from the incident. See [Postmortems](/docs/postmortems/).

### Severity and ownership

AckNexa normalizes severity to P1 through P5, with P1 highest priority. Agree on a severity mapping for your organization and integrations. Business criticality describes the service; it is not a substitute for incident severity. Acknowledge and resolve rights depend on your role, not on merely receiving a notification.

### A good handoff

- Link to the incident instead of copying credentials or private payloads into chat.
- Explain symptoms, checks already performed, the relevant runbook, and the next expected action.
- Confirm that the receiving responder has actually acknowledged. Provider acceptance or a ringing phone is not an acknowledgement.
- Keep sensitive information out of alert summaries when it is not needed for response.

---

## Alerts and event history

Understand raw events, normalized alerts, occurrences, and incident lifecycles.

### Three different records

| Record | What it captures |
| --- | --- |
| Raw event | The authenticated payload accepted from an integration, plus receipt and processing information. |
| Normalized alert and occurrence | The current monitoring condition and the history of observations that contributed to it. |
| Incident | The response lifecycle, including state changes, escalation, notification attempts, and responder actions. |

### Repeated and out-of-order events

A stable integration idempotency key prevents transport retries from creating a second raw event. Fingerprinting separately identifies repeated observations of the same condition. These are different layers of duplicate protection.

Source timestamps determine precedence; equal-timestamp recovery wins over firing. A persisted receipt timestamp is used when the source omits time information. Preserve stable identity fields and consistent timestamps across firing and recovery events.

### Reading processing status

- Accepted: the raw payload and outbox record are committed, but processing may still be pending.
- Processed: normalization and incident handling completed for that raw event.
- Quarantined: processing could not safely complete; inspect Delivery Health and the source payload.
- Recovery: the source reports the condition is no longer firing; verify its identity and timestamps match the intended lifecycle.

### Protect the event history

Do not delete raw events or rewrite database state to fix a live incident. Diagnose the source/parser/configuration issue and use the audited retry path where appropriate. Changes to integration identity fields with existing history may require a new integration. See [Routing & deduplication](/docs/routing/).

---

## On-call schedules

Set up rotations, priority layers, and temporary coverage overrides.

### Build a schedule

1. An organization administrator opens On Call and creates an active schedule for the owning team.
2. Set an IANA timezone and select primary or secondary tier.
3. Add a rotation layer with start time, optional end time, rotation interval, and ordered active participants.
4. Use distinct layer priorities when multiple layers overlap. Higher-priority active layers take precedence.
5. Save and inspect the current on-call result. Then connect the schedule to an escalation policy and verify a test incident.

### Rotation intervals and local time

Intervals are durations in seconds, not a promise of a fixed local-clock handoff across daylight-saving changes. Explicitly check the intended handoff times, timezone, end date, and participant order. Choose intervals that fit your team's operating model and test DST boundaries if applicable.

### Temporary overrides

1. Choose the schedule and create an override for a specific active user, time window, and reason.
2. Review the displayed boundaries in the intended timezone. Check that the override covers the full shift.
3. Verify the effective on-call responder during the override. Remove an obsolete override through the supported action rather than changing historical event data.

### Avoid coverage gaps

- Keep the team, schedule, and participants active.
- Rotation and override participants are paged only while they are an acting member (responder, team admin or organization admin) of the schedule's team. Organization viewers are never paged. If a schedule shows nobody on call, check team membership first.
- Ensure at least one rotation is effective now and at future handoff boundaries.
- Review primary and secondary coverage independently.
- Do not rely on sampled UI previews to prove every minute of future coverage. Continuous coverage certification is not yet an exhaustive timeline feature.
- Re-test after deactivating a user or changing a schedule referenced by policies.

---

## Escalation policies

Define who to reach, which channels to use, and how response progresses.

### Create a policy

1. Open Escalation Policies as an organization administrator and create an active policy.
2. Add ordered steps with a delay, target type, and one or more channels.
3. Choose a user, team, schedule, primary on-call, secondary on-call, or team-admin target as supported by the form.
4. Configure repeats deliberately. Then assign the policy to an active service.
5. Send a test event and check that the effective responder, timing, and channels match your intent.

### Choose realistic targets

A policy can be syntactically valid but operationally useless if the referenced schedule has no coverage, the target is inactive, or the responder lacks a usable destination. Channel preferences and provider configuration are separate from selecting a channel in a policy.

> **Private-only installations**
> Select internal SMTP or the customer-managed voice gateway when internet access is unavailable. Enabling APNs, FCM or a SaaS channel in a policy does not make that provider reachable.

### Durable execution

Escalation jobs live in PostgreSQL and are claimed by workers. Worker restarts do not erase pending jobs. Acknowledgement or resolution cancels pending escalation work. Notification attempts are tracked separately, with bounded retries and visible failure outcomes.

### Pause paging with a maintenance window

Schedule a maintenance window from the Overview while a deployment or planned work runs. A window covers one service, or every service when an organization administrator creates it; team admins can schedule windows for services their team owns. A window lasts at most 14 days and must start within 90 days.

Incidents created during the window are recorded but not paged: they start acknowledged and snoozed until the window ends, and anything still open when it closes is paged then. Ending a window early releases its held incidents at once. Status pages that list an affected service show it as under maintenance while the window runs.

### Delivery guarantees and limits

Provider calls cannot participate in a database transaction. Notification attempts are at least once: a crash after a provider accepted a message but before success was recorded can cause another attempt. Stable identifiers help, but exactly-once handset delivery is not guaranteed. Provider acceptance is never proof of human response.

---

## Web and mobile access

Choose access methods that fit your network and understand mobile notification dependencies.

### Web access is a core workflow

Responders can sign in, review incidents, acknowledge, and resolve through the customer-hosted Web application. Use the deployment's HTTPS origin and trusted certificate chain. Browser-based response does not require APNs, FCM, or a public cloud service.

### iOS and Android apps

AckNexa has native iOS and Android apps. Both list incidents with live updates, acknowledge and resolve through the API, show your on-call shifts and cover requests, your team and your notification history, and open the incident when you tap a page. An action is shown as successful only after the API confirms it. Session tokens are stored per server: in the iOS Keychain, and encrypted with the Android Keystore.

On Android, P1 pages use a separate alarm channel that can show a full-screen notification when the phone allows it; other pages use a high-importance channel.

### How the apps are distributed

For release 1.0.0-rc.1 neither app is published in the App Store or Google Play. The operator builds them from source: the iOS app with Xcode and their own Apple developer account (TestFlight can distribute it to a pilot group), and the Android app with Android Studio or Gradle, their own Firebase project and their own signing key. Signing, device trust and real-device testing are deployment responsibilities.

### How pages reach a phone

| Path | What the server needs | Connectivity |
| --- | --- | --- |
| iOS, self-built app | The customer's own APNs key (`APNS_*` settings or Apple push under Notification Providers). | Outbound to Apple; the phone also needs Apple connectivity. |
| Android, self-built app | The customer's own Firebase service account for FCM HTTP v1 (`FCM_SERVICE_ACCOUNT_FILE`, optional `FCM_PROJECT_ID`, or the organization's FCM settings). Pages are high-priority data messages. | Outbound to Google; the phone also needs Google connectivity. |
| Push relay | A relay credential (`PUSH_RELAY_URL`, `PUSH_RELAY_CREDENTIAL`) for the store-distribution model, where the relay operator holds the APNs key or FCM account. | Outbound HTTPS from the worker to the relay only. |

Through the relay, the worker sends only an incident reference: the notification and incident IDs, the incident number and severity, the device token, and the server's address and name. No alert title, service name or body text leaves the customer network; the app fetches the details from the server itself. Each relay credential is pinned to the deployment's own server address or addresses, and the relay refuses a push that names any other origin, so one installation cannot label pushes as another. An organization uses either the relay or its own APNs key, not both. The relay model is intended for store-distributed apps, which are not published for this release.

### Several servers on one phone

Each app keeps one signed-in account per server address. A push names the server it came from, and tapping it switches to the matching account; a push from a server the phone is not signed in to says so and pre-fills the sign-in screen.

> **Organizations that share a server**
> Two organizations on the same deployment share one server address, so a phone can be signed in to only one of them at a time: signing in to the second replaces the first. Sign out before switching organization; otherwise the previous organization's push registration stays active on the server.

### Inside a private network

- Devices need a route to the private API, directly or through the customer's approved VPN.
- Private DNS and the private CA must work on the device, not just on servers.
- APNs and FCM require connectivity to Apple and Google. An isolated device cannot receive pushes merely because the API is reachable.
- A lost network connection during ACK is not a successful response. Reconnect and refresh to verify the server state.

### Keep a fallback channel

> **Push is one channel among several**
> Policies and personal rules should always include a channel that does not depend on Apple, Google or the relay, such as internal email or voice. WhatsApp is not implemented in this release. Use the Web workflow where mobile app capabilities are unavailable.

---

## Status pages and stakeholder updates

Keep customers and internal stakeholders informed without paging them.

### What a status page shows

A status page is a curated, customer-facing view of selected services. Only the services an administrator lists appear, under the public names they choose, and an incident appears only once a responder publishes a status update from it. The internal incident title and number never appear on the page.

The Web application serves each page at /status/SLUG without a login. A page is public, or private with a secret link (`?key=…`) that is shown once when you save it and can be rotated. To use a separate hostname such as status.example.com, point it at the Web application and rewrite / to /status/SLUG in your reverse proxy.

### Create a status page

1. Open Status Pages under Configure as an organization administrator and create a page with a name and URL slug.
2. Tick the services to list and give each the name the public should see, for example Checkout rather than an internal service name.
3. Choose public or private. For a private page, copy the secret link when it is shown.
4. Share the URL and check it from the network your audience uses.

### Publish an update during an incident

Open the incident, expand Status page update, pick a state (Investigating, Identified, Monitoring or Resolved) and write a sentence for customers. Optionally set a public title of up to 120 characters; it names the incident on the page and in subscriber emails, and later updates keep it unless you change it. Without one, the service's public name is used.

- The update appears on every status page that lists the incident's service.
- The service shows as Major outage for P1 and P2 incidents, and Degraded performance for lower severities.
- The incident timeline records the update, and every subscriber of those pages is emailed.
- If an incident with published updates is resolved without a final update, the worker publishes "This incident has been resolved." Resolved incidents stay on the page for 14 days.
- Maintenance windows that cover a listed service appear under Scheduled maintenance and show the service as under maintenance while they run.

Publishing needs the responder, team admin or organization admin role, the same as acknowledging.

### Email subscribers

Each page has an email subscribe form. A subscriber receives one email per published update with the page link and a one-click unsubscribe link. Subscribing twice to the same page changes nothing, and the API accepts at most 200 new subscriptions per page per hour. Emails are plain text and deliberately minimal.

### What status pages need

| Requirement | Why |
| --- | --- |
| PUBLIC_BASE_URL set to the Web application's public origin | Builds the page URL shown to administrators and in emails. |
| PUBLIC_API_URL set to the API's public origin | Builds the unsubscribe link in emails. |
| The Web application reachable by your audience | The page is served by the Web application; the visitor's browser never talks to the API. |
| An email provider | Organization SMTP or the deployment SMTP settings. Without one, updates still appear on the page and subscriber deliveries are retried, then marked failed. |

### Stakeholders

Responders get paged; stakeholders get told. A stakeholder follows a service or a team and receives an informational email, and a push for app users, when incidents open, are acknowledged, get an update or resolve. Stakeholders are never part of an escalation policy: organization viewers and plain email addresses outside AckNexa both work.

1. Edit a team or service and open Stakeholders. Team admins manage the lists for their own teams.
2. Add a person in AckNexa or type an email address, and pick the moments they hear about: opened, acknowledged, updates posted, resolved.
3. For app users, choose whether to push as well.

A stakeholder who is an AckNexa user can open the incidents they follow in the Web application and the apps, read-only unless their team role allows acting. Deliveries are retried up to six times, and the Stakeholders panel shows how many went out in the last 30 days per person.

### Post a stakeholder update

In the incident drawer, tick Send to stakeholders before posting a note. The note becomes an update on the timeline and is emailed to everyone following the service or its team with updates enabled. Ordinary notes stay internal. Outbound webhooks carry the same update event, so another system can mirror it.

---

## Postmortems

Write one blameless review per incident, prefilled from its timeline, and track the action items.

### One review per incident

A postmortem records what happened, why, how it was detected and resolved, and the action items that stop it recurring. Postmortems, under Respond, lists what is due, in review and published. Everyone who can see an incident can read its review; responders, team admins and organization admins can write one. Resolved incidents offer Start postmortem or Open postmortem.

### Automatic drafts and reminders

The “Postmortem expected from” setting on the Organization page under Settings sets the severity threshold. The default is P2, so every resolved P1 and P2 incident is listed as waiting for a review until one exists; `none` turns the reminder off.

- Every minute, the worker creates a draft for incidents resolved in the last 24 hours that meet the threshold and have no review yet.
- The owner is the incident's assignee, otherwise whoever acknowledged it, otherwise nobody. Service accounts are never owners.
- The due date is the resolution time plus Postmortem due after (days): default 5, from 1 to 60.
- When a review is past due, the owner is emailed through the organization's email provider, or the owning team's team admins when there is no owner. The reminder repeats every 3 days until the review is published.

### Write the review

The editor opens with a draft prefilled from the incident: a title, when and how it was detected, time to acknowledge, time to resolve, and the full timeline beside the form with notifications filtered out. The sections are Summary, Impact, Root cause, Detection, Resolution, What went well, What went badly and Lessons learned. Each action item has a text, an optional owner, a due date and a done box.

### Review statuses

| Status | Meaning |
| --- | --- |
| draft | Being written. |
| review | Ready for the team to read and comment before publishing. |
| published | Final. Requires a summary and a root cause. |

Each status change is recorded on the incident timeline and in the audit log. Export Markdown downloads the review with its timeline for a wiki or a ticket.

---

## Users and access

Manage organization roles, team membership, authentication, and responder contact details.

### Organization roles

| Role | Scope and expectations |
| --- | --- |
| organization_admin | Organization-wide administration, provider configuration, audit and delivery-health access, plus incident response. |
| team_admin | Incident response and selected team-scoped API actions. This role does not unlock all organization-admin configuration screens. |
| responder | Read operational context and take permitted incident response actions. |
| viewer | Read access without incident acknowledgement/resolution or administrative mutations. |

### Create and maintain a user

1. In Users, create the person with their email, display name, timezone, role, and active state.
2. Set an initial password if password login will be used, and share it through an approved secure process. There is no email invitation workflow in this release; with single sign-on, directory sign-in or SCIM, the identity provider can create accounts instead.
3. Add team memberships and configure available contact details and notification preferences.
4. Test sign-in and delivery as the intended user. After changing a role, sign out and back in to refresh the navigation.

### Deactivation and least privilege

Use the lowest sufficient role. Before deactivating a responder, update schedules and escalation targets to preserve coverage. Inactive users cannot continue normal authentication. Organization administrators should review team membership, active users, and provider access regularly.

### Sign-in methods

| Method | Configured in | Suits |
| --- | --- | --- |
| Local password | Users (an administrator sets the password) | Break-glass administrators and small teams. |
| Single sign-on (OpenID Connect) | Identity, or deployment-wide environment variables | Keycloak, Microsoft Entra ID, Okta, Authentik, Google Workspace and other OpenID Connect providers. |
| Directory sign-in (LDAP / Active Directory) | Identity | Sites with a directory but no OpenID Connect provider. |

The methods can be mixed per organization, and password sign-in keeps working while SSO or LDAP is enabled. Sign-in uses the organization slug. With Create accounts automatically, a first directory sign-in creates the user with the configured default role; otherwise an administrator adds the account first. Group mappings tie directory groups to teams, team roles and organization roles, and are applied at each sign-in. An SSO account is pinned to its issuer and subject on first sign-in, and later mismatches are refused.

Changes in the directory reach AckNexa at the next sign-in, and a person disabled in the directory keeps an existing session until it expires. Deactivate the user in AckNexa as well, or use SCIM, when a departure is immediate.

> **Identity administration**
> OIDC unlinking/rebinding is not managed by the Web admin UI. Organization administrators can point SSO or LDAP at any host, so treat that role as an identity administrator. See [Security & identity](/docs/security/).

### SCIM provisioning

SCIM 2.0 lets Okta, Microsoft Entra ID or Keycloak create, update and deactivate AckNexa users and keep group memberships current, so a departure stops paging at once instead of at the next sign-in. SCIM provisions accounts and groups; people still sign in through SSO, and SCIM-created users have no password.

1. Open Provisioning (SCIM) under Identity, enable provisioning and choose the role for new SCIM users: team admin, responder or viewer (default responder).
2. Generate the SCIM token and copy it; it is shown once and only its hash is stored. Rotate replaces it immediately and Revoke removes it.
3. Configure the identity provider with the base URL PUBLIC_API_URL/scim/v2 and the bearer token, mapping the user's email address to userName.
4. Assign test users and groups, then check the last change and last error shown on the Identity page.

Deactivation never deletes a user: it marks the account inactive, revokes every session, disables the person's mobile device tokens and stops them being paged, in one transaction. AckNexa refuses to deactivate the last active organization administrator. SCIM groups use the same group mappings as SSO and LDAP, so use one source of truth for groups. Bulk operations, sorting, ETags and password changes are not supported.

> **Validation status**
> SCIM has been tested with SCIM test clients, not yet against a live Okta or Microsoft Entra ID tenant. Rehearse provisioning and deprovisioning with your own tenant before relying on it.

### Break-glass accounts

Keep at least one break-glass account: a local organization administrator with a password, marked Break-glass account in Users. Only organization administrators can set the flag, from a signed-in session, and only on an active organization administrator with a local password.

- SCIM can read the account but every write to it is refused with 403 and audited, including the deactivation an identity provider sends on unassignment.
- Group mappings never change its role or team memberships.
- An SSO or LDAP sign-in with its email is refused and audited; it signs in with its password only.
- While the flag is set, it cannot be demoted or deactivated from the Users page, and API keys cannot edit it.
- Identity warns when SCIM is on without an active break-glass account, and when one has not signed in for 90 days. Sign in with it now and then so you know the password still works.

### Organization boundary

Tenant-scoped API queries and database relationships use the organization boundary. A valid account in one organization does not grant access to another organization's resources. The current read model is not a promise of per-team concealment of every incident inside a shared organization.

### Several organizations on one deployment

One deployment can host several organizations; users, teams, services, integrations, incidents and device registrations are all scoped to their organization. First-run setup creates only the first organization. The operator creates each further organization, with its first administrator, using the bootstrap command in the backend image; it also creates a default team, escalation policy, service, schedule and four integrations, and prints their webhook secrets once. There is no API or Web screen for adding an organization, and no organization delete.

> **Known limitations on a shared deployment**
> Incident numbers come from one sequence shared by all organizations of a deployment, so each organization sees gaps in its numbering and can infer how many incidents the others have; no incident content is exposed. The mobile apps keep one account per server address, so a phone can be signed in to only one organization of a deployment at a time.

---

## Services and teams

Model ownership so a monitoring event can reach the right response policy.

### Create a team

1. Open Teams as an organization administrator. Supply a clear name, stable slug, description, and active state.
2. Add users with appropriate team memberships. A team's administration role is separate from organization-wide administration.
3. Check that people referenced by schedules and policies remain active.

### Create a service

1. Open Services and define the service name, slug, environment, description, and business criticality.
2. Assign its owning team and an active escalation policy.
3. Add a runbook URL and useful metadata. Prefer customer-controlled runbooks for isolated environments.
4. Configure severity mappings when source conventions differ from your P1–P5 response model.
5. Connect an integration, then verify that an incoming alert appears under the intended service.

### Configuration is connected

Changing a team or service can affect the integrations, routing rules, schedules, and policies that reference it. Inspect active dependencies before deactivation. Do not remove historical database records to make a configuration form accept a change.

### Stakeholders, chat and alert sources

Edit a team or service to add [stakeholders](/docs/status-pages/#stakeholders), a [chat channel](/docs/notifications/#team-chat-channels) and, for services, filtered alert sources and severity mappings. A service's chat channel overrides its team's. Team admins manage stakeholders and chat channels for their own teams.

### Useful service context

- A runbook should explain how to diagnose the monitored condition, what actions are safe, and when to escalate.
- Keep metadata descriptive rather than storing secrets in it.
- Use environment names consistently in monitoring payloads, rules, and service records.
- Business criticality describes importance; normalized alert severity determines response priority.

---

## Connect your monitoring

Connect Alertmanager, Icinga, Alerta, PagerDuty-compatible senders or a generic authenticated webhook, and page on silent jobs.

### Supported inbound sources

| Source | Integration kind | Operator responsibility |
| --- | --- | --- |
| Prometheus Alertmanager | alertmanager | Point a webhook_configs receiver at the integration URL; firing and resolved alerts are both handled. Grafana unified alerting, Mimir, Thanos Ruler, Cortex and vmalert send the same payload shape. |
| Icinga 2 | icinga | A notification command posts host, service, state, type, output, severity and timestamp; recovery clears. Provide stable condition identity and consistent state/recovery timestamps. |
| Alerta | alerta | Use the Alerta webhook plugin payload; closed or resolved status clears. Validate source severity/status mappings. |
| PagerDuty Events API v2 senders | pagerduty | Point the sender's Events API URL at your AckNexa API path /v2/enqueue and use the integration secret as the routing key. trigger opens or updates, resolve clears, and acknowledge is accepted and ignored. |
| Custom monitoring (Zabbix, Nagios, Checkmk, scripts) | generic | Send the canonical-style JSON fields shown below. |

Email intake and SNMP traps are not available yet.

### Create and secure an integration

1. Open Integrations and choose the source kind, owning service, name, enabled state, and authentication mode.
2. Copy the webhook path and newly issued secret. The stored secret is hashed, so plaintext recovery is not available later.
3. Configure the sender's exact deployment API URL. Keep secrets out of URLs, logs, screenshots, and shared shell history.
4. Use token/HMAC mode or HMAC-only mode as required by policy.
5. Send a representative test, inspect processing status and routing, then verify acknowledgement.

### Generic payload

```json
{
  "alert_name": "DatabaseUnavailable",
  "resource": "db-01",
  "environment": "production",
  "severity": "critical",
  "status": "firing",
  "summary": "Database health check failed",
  "started_at": "2026-09-21T09:00:00Z",
  "labels": { "team": "platform" }
}
```

Use your actual current source timestamp; the example is illustrative. Send recovery with the same stable identity and appropriate source lifecycle timestamps. Test the exact payload shape for your integration kind.

### Authenticated request

```sh
curl --request POST "https://api.example.internal/integrations/webhook/INTEGRATION_ID" \
  --header "Content-Type: application/json" \
  --header "X-AckNexa-Key: YOUR_WEBHOOK_SECRET" \
  --header "Idempotency-Key: YOUR_STABLE_EVENT_ID" \
  --data-binary @event.json
```

> **Protect the secret**
> The command is a protocol illustration. In automation, inject secrets through your approved secret mechanism rather than copying credentials into history or source control. A retry of the same delivery must reuse its idempotency key.

### Signature v2 (recommended)

Sign TIMESTAMP + '.' + IDEMPOTENCY_KEY + '.' + RAW_BODY with HMAC-SHA256 using the webhook secret itself as the key, where TIMESTAMP is the X-AckNexa-Timestamp header exactly as sent and IDEMPOTENCY_KEY is the Idempotency-Key header (empty if absent). Send X-AckNexa-Signature: v2=HEX_SIGNATURE. The server stores the secret encrypted with the deployment configuration key, so database or backup access alone cannot forge requests, and replaying a request under a new idempotency key fails. Choose 'Require signature v2' on the integration to refuse tokens and v1 signatures; rotate older secrets once first.

### HMAC mode (v1, legacy)

Sign RFC3339_TIMESTAMP + '.' + RAW_BODY with HMAC-SHA256 using SHA256(webhook_secret) as the binary HMAC key. Send X-AckNexa-Timestamp and X-AckNexa-Signature: sha256=HEX_SIGNATURE. The timestamp must be within five minutes. Sign the exact bytes transmitted; JSON reformatting after signing changes the signature. When a signature is supplied, it must validate even if a token is also supplied.

### Rotation and retirement

Rotate the integration secret through the supported action, update the sender, and verify a successful event. Do not assume a grace period for the previous secret. Archive an integration when retiring it; retain its history. Identity-affecting changes on an integration with history can require creating a new integration.

### Heartbeat monitors

A heartbeat turns silence into a page: a backup that never ran, a cron job that hung, a pipeline that stopped.

1. Open Heartbeats under Configure and create a heartbeat with its service, the job's interval, a grace period and the severity to raise.
2. Copy the ping URL, which is shown once, and call it at the end of the job. GET, POST and HEAD all count.
3. If no ping arrives within the interval plus grace, an incident opens on the service and its escalation policy pages the team. The next ping resolves it.

```sh
curl -fsS -m 10 "https://api.example.internal/heartbeat/YOUR_HEARTBEAT_TOKEN" >/dev/null
```

Heartbeat incidents use the normal alert pipeline, so routing rules, maintenance windows, deduplication and auto-resolution apply. Pausing a monitor stops the clock; rotating the token invalidates the old URL immediately.

### Outbound webhooks

Webhooks, under Configure, sends selected incident events (created, acknowledged, escalated, assigned, note, update, snoozed, resolved) to your own HTTPS endpoint, for the whole organization, one team or one service. The host must be in the deployment's NOTIFICATION_WEBHOOK_HOSTS allowlist, and private addresses also need NOTIFICATION_PRIVATE_CIDRS. Each request carries X-AckNexa-Signature, an HMAC-SHA256 over the timestamp and body with the secret shown once at creation; verify it and reject old timestamps. Deliveries retry with backoff, and a retry after a timeout can repeat a delivery your endpoint already processed.

---

## Routing and deduplication

Keep event transport retries, condition identity, and service ownership distinct.

### How routing is selected

An integration supplies the default service. Enabled routing rules are evaluated in position order, with stable tie-breaking. The first matching rule selects a service; if no rule matches, the integration default remains. Rules may be scoped to an integration or apply more broadly within the organization.

### Rule conditions

Conditions are exact string matches, not arbitrary expressions. All supplied conditions must match. Supported condition fields include source, alert_name, environment, resource, raw source severity, labels.KEY, and annotations.KEY. The severity condition uses the source's severity value; normalized P1–P5 mapping is a separate step.

```json
{
  "environment": "production",
  "labels.team": "platform"
}
```

Put specific rules before broad fallback rules. Use the integration test and incident routing context to verify the matched rule and destination.

### Stable identity across recovery

Routing binds source alert identity to the selected service. Editing a rule does not automatically move an already-bound condition and strand its recovery. Test a new alert identity when validating a changed rule, and plan migrations of existing integrations deliberately.

### Two kinds of deduplication

| Mechanism | What stays stable | What it prevents |
| --- | --- | --- |
| Transport idempotency | Integration plus idempotency key | A sender retry creating a second accepted raw delivery. |
| Alert fingerprint | Configured condition identity fields | Repeated observations creating unrelated alert identities. |
| Incident lifecycle key | Alert identity and lifecycle start | Reopening the same lifecycle as duplicate incidents after repeated processing. |

### Severity mapping

Source normalization establishes a severity. Service mappings can override it, and integration mappings take precedence over service mappings for matching source values. The organization default applies when source severity is absent. Validate both routing conditions and final P1–P5 severity with realistic payloads.

### Common mistakes

- Putting a changing summary, random request ID, or ingestion timestamp in the fingerprint.
- Using different resource/environment identity on firing and recovery.
- Expecting exact-match conditions to behave like regexes.
- Testing a changed rule only with an existing pinned alert identity.
- Treating a duplicate-accepted response as evidence that the notification reached a human.

---

## Notification channels

Configure delivery without making public providers an architectural dependency.

### Three layers to check

1. Provider: the organization or deployment must have a usable adapter and credentials.
2. Responder: the recipient must be active and have valid destinations and preferences.
3. Policy: the escalation step must select the intended target and channel.

### Connectivity and availability

| Channel | Private-only operation | Current support |
| --- | --- | --- |
| Web response | Yes | Core incident access, ACK, resolution, and audit. |
| Email | Yes, with internal SMTP | Implemented; STARTTLS and certificate verification required. Messages are plain text with a short subject and body. |
| Customer voice gateway (paging calls) | Yes, for internal extensions | Implemented HTTPS adapter; a customer bridge to CUCM/SIP/PBX is required. |
| Hotline through CUCM (live call routing) | Yes on the PBX side; responders' mobiles are reached over the PSTN | Bundled Asterisk gateway on a SIP trunk. See [Live call routing](/docs/call-routing/). |
| Private webhook relay | Yes, with operator allowlists | Compatible webhook payload, approved HTTPS hostname and private CIDR. |
| Team chat: Mattermost | Yes, for on-prem Mattermost with an approved host and private CIDR | Implemented incoming-webhook posts per team or service, with Acknowledge links. |
| Team chat: Slack / Microsoft Teams | No | Implemented incoming-webhook posts; approved outbound endpoints required. |
| APNs (iOS app) | No | Optional implemented adapter, with the customer's own key or the push relay; Apple connectivity required. |
| FCM (Android app) | No | Optional implemented FCM HTTP v1 adapter, with the customer's own Firebase project or the push relay; Google connectivity required. |
| Personal Slack / Teams webhooks | No | Optional implemented webhooks; approved outbound endpoints required. |
| Twilio SMS / voice | No | Optional implemented adapters; provider API and relevant callback connectivity required. |
| WhatsApp | Not available | Not implemented; never a core dependency. |

See [Web and mobile access](/docs/mobile/) for the APNs, FCM and push-relay options.

### Configure and test

Organization administrators can save supported provider settings and run an audited provider test. Saved secret fields are redacted. Leaving a secret field blank during an edit preserves the existing secret; it is not a delete operation. Organization-level configuration overrides deployment defaults for that organization.

Disabled, unreadable, or invalid organization configuration fails closed rather than silently falling back to a different account. Correct the override before relying on a deployment-level adapter.

### Team chat channels

A team, or a single service that overrides its team, can post incident events to a Slack, Mattermost or Microsoft Teams channel: opened, paged or escalated, acknowledged and resolved. Messages carry Open and Acknowledge links. The Acknowledge link opens AckNexa, signs the responder in if needed and acknowledges as that person, so the audit trail stays honest without a bot token or inbound chat app.

Edit the team or service, open Chat channel, pick the kind, paste the incoming-webhook URL, choose the events and send a test. Slack's hooks.slack.com is always allowed; a Mattermost or Teams host must be listed in NOTIFICATION_WEBHOOK_HOSTS. Delivery is asynchronous with retries, and posting failures never delay paging. Webhook URLs are encrypted at rest when ACKNEXA_CONFIG_KEY is set.

### Internal SMTP

Use the customer's SMTP server with mandatory STARTTLS and a trusted certificate. For tenant-configured private destinations, the operator must allow the exact DNS hostname and its private CIDR. Loopback, link-local, and metadata endpoints remain blocked. Private webhook relays use the same operator-controlled private-network guard. See [Security & identity](/docs/security/).

### Delivery evidence

A provider-accepted result means the provider accepted the request. It does not prove inbox arrival, handset presentation, a completed call, or human acknowledgement. Inspect individual attempts and the incident timeline, and test with real recipients. Unknown outcomes can occur after a lease expires or a response is lost; duplicate delivery remains possible.

### Phone and SIP destinations

Use a destination supported by the selected adapter. A SIP address sent through Twilio still depends on Twilio and does not become an internal PBX call. For private telephony, use the [customer voice-gateway contract](/docs/voice-gateway/).

---

## Live call routing

Give callers a hotline number that rings whoever is on call and opens an incident for every call.

### How a call flows

1. The caller hears the route's greeting and, if the route has a menu, picks a service with a digit. No digit within 8 seconds selects the default service.
2. AckNexa opens an incident on the chosen service with the route's severity. Its escalation policy starts as for any alert, so other channels page in parallel with the phone ringing.
3. AckNexa rings, in order, whoever is on call for the service's owning team by schedule tier, then the team's admins. Only people with a phone number on their profile are rung, each for the route's ring timeout (default 25 seconds).
4. Before the call is bridged, the responder hears a short whisper and must press a key, so a personal voicemail box cannot take the call.
5. Answered: the incident is acknowledged by that responder. Nobody answered: the caller can leave a voicemail if the route allows it, the recording link is added to the incident, and the incident stays open and escalating.

Recent calls, on the Call Routing page under Configure, lists every call, who was tried and the outcome. Routes and the call log are for organization administrators.

### Choose how the number connects

| Provider | How it works | Connectivity |
| --- | --- | --- |
| Twilio number | Twilio sends the call to the route's inbound URL and AckNexa drives the greeting, menu, ringing and voicemail. Requests are verified with Twilio's signature. | Twilio must reach PROVIDER_CALLBACK_BASE_URL, the API's public HTTPS origin. Twilio charges per minute and stores voicemail recordings. |
| My own PBX (gateway API) | Your PBX asks AckNexa who to ring and reports how the call ended, authenticated with the route's bearer token. | The PBX must reach the API; no internet access is needed. |
| Cisco CUCM | The bundled gateway takes the hotline over a SIP trunk from CUCM and uses the gateway API for you. | SIP and RTP between CUCM and the gateway; responders' mobiles are called back out through CUCM. |

### Connect your own PBX

Create the route with provider My own PBX. AckNexa shows a bearer token (`cr_…`) once; rotate it from the route card. For each call, the PBX first asks who should ring:

```http
POST /v1/voice/route/ROUTE_ID
Authorization: Bearer YOUR_ROUTE_TOKEN
Content-Type: application/json

{"from": "+13035550199", "digit": "1", "provider_call_id": "pbx-call-id"}
```

The response names the service and incident, the ring timeout, whether voicemail is on, and the ordered list of people with their phone numbers. The incident already exists at that point. When the call ends, the PBX posts the result to /v1/voice/route/ROUTE_ID/result with status connected, missed or voicemail; connected acknowledges the incident for the user who answered, while missed and voicemail leave it open and note the outcome. A PBX that plays the menu itself can read the route configuration with GET on the same path.

> **Treat the route token like a password**
> Responders' phone numbers are returned to the PBX. Restrict who can reach the API with the token, and rotate it if it may have leaked.

### Cisco CUCM gateway

The gateway has two parts: the acknexa-voice-asterisk image, an Asterisk with one SIP trunk to CUCM that accepts SIP only from the CUCM addresses you list and has no phones, registrations or management interfaces; and the voicegateway command in the backend image, which asks AckNexa who to ring, tells Asterisk what to do and reports the result. Deploy it with the deploy/onprem/voice-gateway.compose.yml overlay or the Helm voiceGateway values. Plain CUCM is enough; no UCCX, CUBE scripting or IVR licence is involved.

- CUCM 12.5, 14 or 15, with administrator access.
- A host or Kubernetes node with a fixed IP that CUCM can reach on SIP 5060 (UDP and TCP), plus the RTP range (UDP 10000-10099 by default) from phones, PSTN gateways and media resources.
- PSTN access from CUCM. Responders are called on the numbers on their AckNexa profile, in international format.
- G.711 between the gateway and the phones or PSTN gateways; the gateway does not transcode G.729.
- A call route with provider My own PBX, its ID and its token.

Run voicegateway check after deployment to confirm the route. If AckNexa cannot be reached, the caller hears that the on-call team could not be reached and can be put through to a fallback number (VOICEGW_FALLBACK_NUMBER). Calls in progress are dropped when the gateway restarts.

> **Validation status**
> The CUCM gateway has been verified only against a CUCM stand-in built on Asterisk, not against a real CUCM cluster. Validate it on your cluster, dial plan and PSTN gateways before relying on it.

### Call routing is not voice paging

Call routing answers inbound hotline calls. Voice notifications from escalation policies are separate: they use Twilio or the [internal voice gateway](/docs/voice-gateway/) contract, which still needs a customer-supplied bridge to CUCM, SIP or the PBX.

---

## Ticketing: Jira and Halo

Open tickets from incidents, mirror the response as comments, and optionally resolve incidents when tickets close.

### What it does

AckNexa opens a ticket for an incident in Jira Cloud, Jira Server / Data Center, or Halo (HaloITSM / HaloPSA), links it from the incident, mirrors acknowledgements, notes, stakeholder and status page updates, reassignment and the resolution as comments, and can move the ticket to its resolved state. There is one ticket per incident per tool, never one per alert.

### Connect a tool

| Tool | URL | Credentials |
| --- | --- | --- |
| Jira Cloud | https://SITE.atlassian.net | The integration account's email and an API token. |
| Jira Server / Data Center | Your Jira base URL | A personal access token for the integration account, or username and password. |
| Halo | Your Halo instance URL | An API application with the Client ID and Secret (Services) login type and permission to create tickets and actions. |

Open Ticketing under Configure as an organization administrator and add a connection. Credentials are write-only and stored encrypted with ACKNEXA_CONFIG_KEY. Test connection signs in and loads the projects, issue types, ticket types, priorities and statuses the account can see, so rules are picked from lists. The account needs permission to create issues or tickets, add comments and, for the resolve action, transition them.

### Rules per team

Add one rule per team; a rule on a single service overrides its team's. A rule chooses the connection, the Jira project and issue type or the Halo ticket type, optional Jira components and labels or Halo team, agent, client and category, a priority per severity, the transition or status to apply on resolve, and whether to mirror comments.

It also chooses when to open the ticket: only when a responder asks, when the incident opens, or when it is acknowledged, from a minimum severity. Teams without a rule get no automatic tickets, but anyone who can acknowledge an incident can still open one with Create ticket, using the connection's defaults.

### Ticket closure flowing back

By default, closing a ticket changes nothing in AckNexa. Tick Sync ticket closure on the connection to get a webhook URL, shown once, and configure that URL in Jira (an Issue updated webhook) or Halo (a ticket status-change webhook). Then tick Resolve the incident when the ticket is closed on the rule (default off).

When a linked ticket reaches a closed state, the incident timeline always records it, and if the rule opted in, the system resolves the incident with the ticket as the reason, cancelling escalation and pending notifications. The webhook is idempotent and does not loop with the outbound sync. It only affects incidents that have a ticket on that connection.

### Security and network

- The webhook token is a random value stored only as a hash; a wrong token, a disabled webhook and an unknown connection all get the same 401. Senders that can set headers can send the token in X-AckNexa-Token or a bearer header instead of the path; the query string is never accepted.
- Because the path form is part of the URL, keep reverse-proxy access logs for /v1/ticketing/inbound/ off or redacted.
- AckNexa needs outbound HTTPS from the API and worker to the tool. TICKETING_HOSTS limits the hosts a connection may use, and TICKETING_ALLOW_HTTP=true permits http:// for an internal Jira on a trusted network.
- The tool needs to reach the API only if Sync ticket closure is on.
- Failed creation and comments are retried with backoff; credential and configuration errors stop at once and are reported on the incident timeline.

---

## Audit and delivery health

Understand configuration history, audit export and streaming, quarantine, and safe retry.

### Audit records

The Audit area shows organization-scoped administrative and response actions, including the actor, action, target, request context, and time where recorded. Provider configuration is redacted rather than exposed as plaintext. Normal runtime access cannot edit audit rows; the database also has an immutability trigger.

> **Operational boundary**
> Append-only application controls are not a claim of regulatory certification or tamper-proof storage against a database superuser. Protect database administration, backups, logs, and retention policy separately.

### Export the audit log

Export, on the Audit page, downloads a date range as JSON Lines or CSV, at most 366 days per download; export longer periods in parts. Rows arrive in time order and are streamed, so a failed read aborts the transfer instead of producing a silently truncated file. CSV cells that a spreadsheet could evaluate as a formula are prefixed with a quote; prefer JSON Lines for machines. Exports are themselves audited.

Passwords, tokens, secrets, credentials and private keys never appear in an export or a stream: values under keys whose names contain password, secret, token, credential, private_key or webhook_url are replaced with [redacted].

### Stream to a SIEM

Each organization can have one stream, under Audit stream on the Audit page. It delivers every record written after the stream is saved; use Export for earlier history.

| Transport | Requirements |
| --- | --- |
| HTTPS | Signed JSON batches of up to 500 events to an https:// URL on port 443 whose host is in NOTIFICATION_WEBHOOK_HOSTS. Verify X-AckNexa-Signature (HMAC-SHA256 over the timestamp and raw body, with the secret shown once) and reject old timestamps. |
| Syslog | RFC 5424 messages over TLS 1.2 or newer with a verified certificate, default port 6514, to a host listed in AUDIT_SYSLOG_HOSTS. Syslog streams are not signed; TLS is the protection. |

Delivery is at least once: the cursor moves only after the collector accepts a batch, failures are retried with backoff of up to 10 minutes, and one failing batch holds back the ones after it. De-duplicate on the event id. The stream shows the last error and the number of events waiting, and a test action sends one synthetic event without moving the cursor. Private collector addresses also need NOTIFICATION_PRIVATE_CIDRS.

### Delivery Health

Organization administrators use Delivery Health to inspect pending/unpublished events, processing failures, overdue or failed escalation work, exhausted notifications, and quarantined raw events. This view helps diagnose the tenant's processing path; it is not a replacement for independent infrastructure monitoring.

### Retry a quarantined event

1. Inspect the reported failure and relevant source payload using authorized operational access.
2. Correct the integration, parser, or configuration problem. Do not change the historical payload just to make the record disappear.
3. Use the supported dead-letter retry action. It writes an audit record and queues the original event.
4. Check processing and incident history. If the underlying issue remains, the event may be quarantined again.

### Interpret failures carefully

- No notification record: inspect routing, active policy targets, schedule coverage, and worker progress.
- Failed notification: check destination, provider configuration, credentials, TLS and network reachability.
- Unknown attempt: the provider may have accepted the request; do not assume no delivery occurred.
- Delayed backlog: inspect database capacity, worker health and broker state before changing application data.

### Retention

Workers prune operational data hourly: expired sessions and login state, outbox rows older than RETENTION_OUTBOX_DAYS (default 7), payloads of processed raw events older than RETENTION_RAW_PAYLOAD_DAYS (default 30; 0 keeps them), and finished delivery-queue rows older than RETENTION_DELIVERY_DAYS (default 30). Pending deliveries are never pruned. Raw event rows, incidents, timelines, notifications and audit events are kept; archive or purge those under your own records policy.

Reports offers basic incident and paging charts; it is not a full analytics suite, and the audit export is not a turnkey compliance product. UI lists are bounded; preserve authoritative history through supported backups.

---

## Deploy on your infrastructure

Choose Docker Compose or production Helm without requiring a public cloud.

### Deployment models

| Model | Use | Boundary |
| --- | --- | --- |
| Development Compose | Local development and evaluation | Not a production readiness claim. |
| On-prem Compose | Small customer-controlled installation | Single host; persistent local state, TLS ingress, optional local observability. |
| Kubernetes / Helm | Production topology on customer infrastructure | Customer-provisioned state services, ingress, secrets, monitoring and recovery. |

Check [System requirements](/docs/system-requirements/) for the versions, services, ports, clients and sizing defaults each model expects.

### Core components

The API contains the integration gateway. Workers contain the escalation and notification engines. PostgreSQL is authoritative; NATS JetStream transports committed outbox events. Redis is included in the on-prem Compose profile but is not required for incident correctness. The Web UI talks to a server-side API proxy.

### Prepare the environment

1. Obtain the approved source/release artifacts through your distribution channel. No public repository or license terms are asserted by this guide. Release images (backend, Web and the voice-gateway Asterisk image) are multi-architecture (linux/amd64 and linux/arm64), signed, with SBOM attestations; every binary reports its release with --version.
2. Provision private DNS, trusted certificates, stable time synchronization, storage, and backup destinations.
3. Build or mirror approved backend, Web, PostgreSQL, NATS, Redis, ingress and selected observability images. Use immutable digests where appropriate.
4. Prepare secrets locally, including the database and broker credentials, encryption key, and first-run setup token.
5. Run migrations before serving application traffic, then verify the [first organization workflow](/docs/getting-started/). Both deployment profiles run the [paging upgrade check](/docs/upgrades/#the-upgrade-check-gate) before migrating.

### Small-install Compose

```sh
cp deploy/onprem/.env.example /secure/path/acknexa.env
bash scripts/onprem_init.sh /secure/path/acknexa-config

# Configure approved images, paths, private DNS and TLS files first.
docker compose --env-file /secure/path/acknexa.env \
  -f deploy/onprem/compose.yml --profile observability up -d --wait
```

Run these commands from the product repository. Initialization refuses to overwrite an existing configuration directory. Supply ca.crt, tls.crt, and tls.key from customer PKI, update ingress hostnames, and set the exact HTTPS Web origin. The profile preloads images with pull_policy: never and does not seed demo data.

> **Small-install security boundary**
> The supplied small-install profile uses a single host, a private Docker bridge for state traffic, and a schema-owner database login. It is not the HA/least-privilege production topology. Harden independently or use the production Helm deployment model.

### Production Helm

```sh
helm upgrade --install acknexa ./deploy/helm/acknexa \
  --namespace acknexa \
  -f deploy/helm/acknexa/values-onprem.yaml \
  -f /secure/path/site-values.yaml \
  --atomic --wait --timeout 15m
```

Pre-create the namespace, private-registry credentials, customer CA ConfigMap, state-service Secrets, application encryption key, and ingress TLS Secrets. Supply actual image digests and ingress/resource/topology values. PostgreSQL uses verified TLS and separate runtime/migration identities. NATS uses authenticated TLS seeds and a replicated file-backed stream.

The chart deploys stateless application components, not the customer's entire HA platform. Database operators, NATS clusters, ingress controllers and independent monitoring are customer-managed. A private VIP, hardware appliance or internal load balancer can replace a cloud load balancer.

### Private-network and disconnected operation

Mirror images and build dependencies before disconnecting. Runtime isolation does not automatically provide an offline build cache. Private-registry CA trust must be installed on container nodes; the application's CA mount cannot fix image pulls. Build the Web image with the private API URL so its integration instructions display the correct address.

The on-prem Compose backend network is internal by default. Reach customer LAN services through an approved network attachment or a customer-only firewall allowlist. The Helm overlay supplies restricted egress; use an enforcing CNI and add explicit rules for dependencies outside the namespace. Do not assume NetworkPolicy alone covers every host or ingress path.

### Configuration overlays

- Internal OIDC: deploy/onprem/oidc.compose.yml.
- SMTP authentication: deploy/onprem/smtp-auth.compose.yml.
- Customer voice gateway for paging calls: deploy/onprem/voice.compose.yml.
- CUCM gateway for [live call routing](/docs/call-routing/): deploy/onprem/voice-gateway.compose.yml, or the Helm voiceGateway values. Include the overlay in every later Compose command, upgrades too.
- Android push through FCM: deploy/onprem/fcm.compose.yml.
- Helm adapters are opt-in via oidc.enabled and notifications.*.enabled (including notifications.fcm and notifications.pushRelay; the push relay cannot be combined with APNs or FCM). Disabled adapters do not require their cloud credentials.
- Set PROVIDER_CALLBACK_BASE_URL (Helm config.providerCallbackBaseURL) when Twilio, call routing or ticketing inbound webhooks are used.

---

## System requirements

Components and versions, platforms, the services you provide, ports, clients and sizing defaults for release 1.0.0-rc.1.

### What this chapter covers

This chapter lists what release 1.0.0-rc.1 needs: the components and the versions its deployment files use, where it can be deployed, the services you provide, the ports and outbound connections, the clients, and the sizing defaults in the deployment files. Where the release states no minimum version or size, this chapter says it is not specified.

> **Versions in use, not a support matrix**
> The versions below are the ones pinned in the supplied Compose files, the on-prem image example and the release's continuous integration. No separate compatibility matrix is published for this release. Validate any other version in your own environment first.

### Runtime components

| Component | Version in this release | Notes |
| --- | --- | --- |
| Backend image | 1.0.0-rc.1 | One image for the API (which contains the integration gateway), the workers, database migrations, the upgrade check, the bootstrap command and the CUCM gateway service. Go binaries on Alpine Linux 3.24, run as a non-root user. |
| Web image | 1.0.0-rc.1 | The Web UI and its server-side API proxy. The image carries its own Node.js runtime, so nothing is installed on the host. |
| Voice-gateway Asterisk image | 1.0.0-rc.1, optional | Only for [live call routing](/docs/call-routing/) through Cisco CUCM. |
| PostgreSQL | 16 | The system of record. The supplied Compose files, the on-prem image example and continuous integration all use PostgreSQL 16; no other major version is stated as tested. The first migration creates the `pgcrypto` extension if it is missing, so the migration identity must be allowed to create it, or install it beforehand. |
| NATS with JetStream | 2.10 | Durable transport for committed outbox events. Compose runs one node with one stream replica. Production expects your own cluster of three or five nodes with TLS, authentication, file storage and at least three stream replicas; the chart sets three by default. |
| Redis | 7.4 in the on-prem example, optional | Included in the on-prem Compose profile for customer cache workloads. The application does not use it, and the Helm chart needs no Redis endpoint. |
| Ingress | NGINX `stable-alpine` in the on-prem example | Terminates TLS for the Web and API host names. With Helm, use your own ingress controller; the chart's default ingress class is `nginx`. |
| Prometheus, Grafana, OpenTelemetry Collector | Prometheus 3.5.0 in the on-prem example; Grafana and the Collector are not pinned | The optional local observability profile. Use your approved versions. |

Release images are built for linux/amd64 and linux/arm64; see [Prepare the environment](/docs/deployment/#prepare-the-environment) for signing, SBOMs and image mirroring.

### Deployment targets

| Target | What it needs | Minimum version |
| --- | --- | --- |
| Small install with Docker Compose | One container host with Docker Engine and the Compose v2 `docker compose` command, plus Bash and OpenSSL. The backup script also needs Python 3 and `sha256sum` or `shasum`. | Not specified for Docker Engine or Compose. |
| Production on Kubernetes with Helm | A cluster with an ingress controller, plus your own PostgreSQL and NATS. The on-prem overlay's egress policy needs a CNI that enforces NetworkPolicy. ServiceMonitors need the Prometheus Operator CRDs, and External Secrets is optional. | Helm 3 or later, because the chart uses chart API version v2. Kubernetes: not specified; the chart declares no minimum version. |
| Hosts and nodes | linux/amd64 or linux/arm64 container hosts. | Operating system and kernel versions: not specified. |

### Build toolchains

These are needed only where you build images or the mobile apps, not on the hosts that run AckNexa.

| What you build | Toolchain used by this release |
| --- | --- |
| Backend image | Go 1.27.4 in the Dockerfile's build stage; the Go module requires Go 1.26 or newer. The build cross-compiles for amd64 and arm64. |
| Web image | Node.js 24 in the build stage. The resulting image carries its own Node.js runtime. |
| iOS app | A Mac with Xcode 16 or newer, XcodeGen, and an Apple Developer Program membership for push notifications and TestFlight. |
| Android app | Android Studio (Ladybug 2024.2 or newer) or the included Gradle 8.9 wrapper, JDK 17 or 21, and Android SDK Platform 35. Direct push needs your own Firebase project. |

### Services you provide

Every deployment needs these:

| Service | Notes |
| --- | --- |
| TLS ingress or load balancer | Two host names, one for the Web UI and one for the API, with certificates from your PKI. Compose includes NGINX with your `tls.crt` and `tls.key`. With Helm, use your ingress controller and an internal VIP, appliance or load balancer; no cloud load balancer is needed. |
| DNS and time synchronization | Private DNS for both host names. Keep clocks synchronized: signed webhooks with a timestamp more than five minutes off are rejected, and voice-gateway callbacks are signed with timestamps. |
| PostgreSQL and NATS for Helm | Your highly available PostgreSQL with verified TLS and separate runtime and migration identities, and an authenticated TLS NATS cluster. The chart deploys only the stateless AckNexa components. Compose runs both on the same host. |
| Secret storage | Protected files for Compose; Kubernetes Secrets or an internal secret manager for Helm. This includes `ACKNEXA_CONFIG_KEY`, which needs its own protected backup. |
| Backup storage | Encrypted, customer-controlled storage for database backups and keys. For production, PostgreSQL WAL archiving and point-in-time recovery tooling. |
| Independent monitoring (recommended) | Prometheus and Alertmanager in a separate failure domain. The optional Compose observability profile runs on the same host it watches. |

Only for the features you enable:

| Service | Notes |
| --- | --- |
| Private CA | Mount your CA bundle for the backend (`SSL_CERT_FILE`) and the Web runtime (`NODE_EXTRA_CA_CERTS`), and install it on browsers and phones. |
| SMTP server | Email notifications, stakeholder and status-page subscriber email, and postmortem reminders. STARTTLS and certificate verification are required. |
| Identity provider | OpenID Connect single sign-on, LDAP or Active Directory over `ldaps://` or StartTLS, and SCIM 2.0 provisioning. Keep a local break-glass administrator. |
| Mobile push | Your own APNs key for a self-built iOS app, your own Firebase service account for a self-built Android app, or a push-relay credential. The relay cannot be combined with your own APNs or FCM settings. |
| Twilio | SMS, voice calls and a Twilio hotline number. Twilio must reach `PROVIDER_CALLBACK_BASE_URL` over HTTPS. |
| Telephony | An HTTPS bridge to your PBX for voice paging (`VOICE_GATEWAY_URL`), or Cisco CUCM with a SIP trunk to the bundled gateway for live call routing. |
| Jira or Halo | Tickets per incident. Inbound ticket webhooks also use `PROVIDER_CALLBACK_BASE_URL`. |
| SIEM | An HTTPS collector, or a syslog receiver over TLS, for audit streaming. |

### Ports

The ports AckNexa components listen on inside their containers, as set by the supplied deployment files.

| Component | Port inside the container | Exposure |
| --- | --- | --- |
| Ingress (Compose NGINX) | 443, HTTPS | Published on `ACKNEXA_BIND_ADDR` and `ACKNEXA_HTTPS_PORT`, by default loopback port 8443. Serves the Web and API host names. |
| API | 8080, HTTP | Behind the ingress. Also serves `/health/live`, `/health/ready` and `/metrics`. The Helm API ingress publishes only `/v1`, `/v2`, `/scim`, `/integrations`, `/calendar`, `/heartbeat` and `/health`. |
| Workers | 9090, HTTP | Health and metrics only; not published. |
| Web | 3000, HTTP | Behind the ingress. |
| PostgreSQL and NATS in Compose | 5432; 4222 for clients and 8222 for monitoring | On the internal backend network only; not published. |
| Grafana (observability profile) | 3000 | Published on loopback port 3300 by default (`GRAFANA_PORT`). |
| CUCM gateway: Asterisk (optional) | SIP 5060 over UDP and TCP; RTP over UDP 10000 to 10099 | Published on the host in Compose, and bound with host networking by default in Helm. Allow them only from CUCM and your media endpoints. Calls to CUCM go to its port 5060 over TCP by default. |
| CUCM gateway service (optional) | 4573 for FastAGI from Asterisk; 8090 for voicemail recordings | Internal. Compose serves voicemail links through the ingress on port 8444. |

### Outbound connections

The core needs none of these: each is needed only when you enable the matching feature. Private destinations configured by an organization also need `NOTIFICATION_PRIVATE_CIDRS`; see [Notification egress](/docs/security/#notification-egress).

| Feature | Destination | Protocol |
| --- | --- | --- |
| APNs with your own key | `api.push.apple.com`, and `api.sandbox.push.apple.com` for development builds | HTTPS |
| FCM with your own service account | `fcm.googleapis.com` and `oauth2.googleapis.com` | HTTPS |
| Push relay | The relay URL you are given (`PUSH_RELAY_URL`) | HTTPS |
| Twilio | `api.twilio.com`; Twilio calls back to `PROVIDER_CALLBACK_BASE_URL` | HTTPS |
| Slack | `hooks.slack.com` and `hooks.slack-gov.com`, always allowed | HTTPS on port 443 |
| Microsoft Teams, Mattermost, outbound webhooks, HTTPS SIEM collectors | Hosts you list in `NOTIFICATION_WEBHOOK_HOSTS` | HTTPS on port 443 |
| Syslog SIEM | Hosts you list in `AUDIT_SYSLOG_HOSTS` | Syslog over TLS |
| Email | Your SMTP server (`SMTP_ADDR`); servers configured by an organization must be in `NOTIFICATION_SMTP_HOSTS` | SMTP with STARTTLS |
| Jira and Halo | The configured base URLs, limited by `TICKETING_HOSTS` when set | HTTPS; HTTP only with `TICKETING_ALLOW_HTTP=true` |
| Identity provider | Your OIDC issuer and LDAP servers | HTTPS; `ldaps://` or StartTLS |
| Voice bridge to your PBX | `VOICE_GATEWAY_URL` | HTTPS |

### Clients

| Client | Requirement |
| --- | --- |
| Web browser | Not specified: no list of supported browsers is published for this release. The browser must reach the Web host name over HTTPS and trust its certificate chain. |
| iOS app | iOS 17.0 or later. For this release the operator builds the app; it is not in the App Store. |
| Android app | Android 8.0 (API level 26) or later; it targets Android 15 (API level 35). For this release the operator builds the app; it is not in Google Play. |
| Phones on a private network | A route to the API host name, directly or over your VPN, with private DNS and your CA on the device. Push also needs Apple or Google connectivity. |

See [Web and mobile access](/docs/mobile/) for how the apps are distributed and how pages reach a phone.

### Sizing

This release has no certified sizing guide. The figures below are the defaults in the supplied deployment files, not a capacity statement. Validate CPU, memory, storage and connection limits against your own load, as described in [Capacity and deployment evidence](/docs/architecture/#capacity-and-deployment-evidence).

| Helm workload | Default replicas | Requests per container | Limits per container |
| --- | --- | --- | --- |
| API | 3 | 500m CPU, 512 MiB | 2 CPU, 2 GiB |
| Workers | 3 | 500m CPU, 512 MiB | 2 CPU, 2 GiB |
| Web | 3 | 200m CPU, 256 MiB | 1 CPU, 1 GiB |
| Upgrade-check and migration Jobs | One run per install or upgrade | 100m CPU, 128 MiB | 1 CPU, 512 MiB |
| CUCM gateway (optional) | 1 | Asterisk 100m CPU, 128 MiB; gateway 50m CPU, 64 MiB | Asterisk 1 CPU, 512 MiB; gateway 500m CPU, 256 MiB |

- Together, the default API, worker and Web pods request 3.6 CPU and 3.75 GiB of memory, with limits of 15 CPU and 15 GiB. PostgreSQL, NATS, ingress and monitoring come on top and are not sized by the chart.
- Each API pod opens up to 20 database connections and each worker pod up to 60 (`config.apiDatabaseMaxConnections` and `config.workerDatabaseMaxConnections`): up to 240 at the default replica counts, plus the migration Job. Allow for this in PostgreSQL's connection limit or a connection pooler; a pooler must pass `statement_timeout` through.
- With the chart defaults, each worker pod runs 24 escalation workers, 4 notification workers per channel and 4 raw-event workers.
- The on-prem Compose file sets no CPU or memory limits. The API and the worker each use the built-in pool of 20 database connections, NATS keeps one stream replica, and the optional Prometheus keeps 30 days of metrics.
- Disk sizes for PostgreSQL, NATS and backups are not specified. They grow with your event, incident and audit volume.

### Storage and backups

- PostgreSQL is the system of record: configuration, raw events, incidents, escalation jobs, notification attempts, the outbox and audit.
- Workers prune operational data hourly. By default, outbox rows go after 7 days, payloads of processed raw events after 30 days, and finished delivery rows after 30 days. Raw-event rows, incidents, timelines, notifications and audit events are kept until you archive or purge them; see [Retention](/docs/audit/#retention).
- NATS needs persistent file storage, but its stream is not authoritative: when the stream is empty, the workers recover every unprocessed raw event from PostgreSQL.
- Compose also keeps volumes for Redis, which is not authoritative, and, when their profile or overlay is enabled, for Prometheus, Grafana and voicemail recordings. Voicemail recordings are kept for 30 days by default.
- Keep `ACKNEXA_CONFIG_KEY` in an independently protected copy. Without it, saved provider credentials cannot be decrypted.

Small installs back up with `scripts/onprem_backup.sh`, which by default takes a consistent database snapshot while paging continues, and restore into a new, isolated project. Production uses your PostgreSQL tooling with continuous WAL archiving and point-in-time recovery to encrypted, customer-controlled storage; cloud object storage is not required. See [Backups and disaster recovery](/docs/backup-recovery/).

---

## Security and identity

Understand trust boundaries, customer PKI, secret handling, and operator responsibilities.

### Trust model

AckNexa enforces role checks and organization scope in the API, with tenant-aware database relationships as a second boundary. Sessions and API credentials are represented by hashes at rest; passwords use bcrypt. The Web proxy uses an HttpOnly session cookie and checks the configured browser origin for mutations.

> **No certification claim**
> These controls do not establish SOC 2, ISO 27001, HIPAA or another compliance status. An external penetration test has not yet been performed for release 1.0.0-rc.1. No independent certification, penetration-test warranty, or production SLA is represented by this website.

### TLS and private CA

- Terminate browser/integration traffic at the customer's TLS ingress with certificates covering the exact private DNS names.
- Mount the customer trust bundle for Go with SSL_CERT_FILE and for the Web runtime with NODE_EXTRA_CA_CERTS.
- Production PostgreSQL URLs require sslmode=verify-full and the appropriate sslrootcert path.
- Use authenticated TLS NATS seed URLs and certificates matching those hostnames.
- Keep verification enabled. Do not use TLS verification bypasses as a private-CA workaround. Restart workloads after relevant trust/secret changes.

### Secrets management

Use customer-controlled secret files, Kubernetes Secrets, or an internal secret manager. External Secrets is optional and need not connect to a public service. Protect Kubernetes Secrets with appropriate RBAC and encryption at rest; protect Compose secret files and their parent directory on the host.

| Setting | Purpose |
| --- | --- |
| ACKNEXA_CONFIG_KEY | Stable base64-encoded 32-byte key for encrypted organization provider configuration. |
| ACKNEXA_SETUP_TOKEN | Operator-issued first-run setup authorization. |
| DATABASE_URL / NATS_URL | Customer state-service credentials and endpoints. |
| SMTP_PASSWORD / OIDC_CLIENT_SECRET | Optional internal service authentication. |
| VOICE_GATEWAY_TOKEN | Optional customer voice-bridge bearer token. |
| FCM_SERVICE_ACCOUNT_FILE / PUSH_RELAY_CREDENTIAL | Optional Firebase service account for Android push, and optional push-relay credential. |

Supported sensitive settings accept a corresponding _FILE form, including the settings above and selected APNs/Twilio secrets. Set either the value or the file, not both. Files are loaded at startup; rotation requires a restart. Keep ACKNEXA_CONFIG_KEY in an independently protected recovery copy.

### Single sign-on and directory sign-in

Organization administrators configure SSO (OpenID Connect, for example Keycloak, Microsoft Entra ID or Okta) and LDAP / Active Directory sign-in under Identity, with optional automatic account creation and group mappings. Unverified emails are refused. LDAP uses ldaps:// or StartTLS, never a clear-text password, and accepts a pasted private CA certificate. Client secrets and bind passwords are sealed with ACKNEXA_CONFIG_KEY and never returned by the API. Organization-level SSO needs PUBLIC_API_URL and PUBLIC_BASE_URL so callback URLs are known.

Operators can instead configure one issuer for the whole deployment with the OIDC_* environment variables. That client must put the organization slug in the acknexa_organization claim (or the claim named by OIDC_ORGANIZATION_CLAIM), because one client serves every organization. Control who can change those claims in the IdP.

SSO is optional, not a dependency on a public identity service. Keep a local [break-glass administrator](/docs/users-access/#break-glass-accounts). Identity rebinding requires an approved operator procedure; the Web UI does not yet manage it.

### Notification egress

Tenant-configured SMTP and HTTPS webhooks require operator-approved DNS hosts. Private destinations additionally require explicit NOTIFICATION_PRIVATE_CIDRS. DNS answers are checked and the validated address is dialed; loopback, link-local/metadata destinations and redirects remain restricted. Environment-configured operator endpoints are a separate trusted boundary and need firewall policy. Ticketing connections are limited to TICKETING_HOSTS, and audit syslog streams to AUDIT_SYSLOG_HOSTS.

### Shared responsibility

- Operate hardened container hosts, ingress, database/broker authentication, network policies and encrypted backups.
- Keep sensitive tokens and customer payloads out of logs, tickets and public documentation.
- Review access when people change teams or leave the organization.
- Do not expose metrics or database ports as unauthenticated public services.
- Report a suspected vulnerability in AckNexa to security@acknexa.com, never in a public issue. Include the affected version or image digest, a description and impact, and reproduction steps, but no customer data. No PGP key is published yet; if the details need encryption, say so in a short first email. For anything else, write to hello@acknexa.com.

---

## Architecture and reliability

Understand what is durable, what is asynchronous, and what recovery can and cannot guarantee.

### The event path

**Figure.** AckNexa components and the systems they connect to. Dark boxes are AckNexa components, light boxes are your systems or providers, and dashed boxes are optional.

Diagram of the AckNexa components. Monitoring sources (Prometheus Alertmanager, Icinga, Alerta, generic webhooks and heartbeat pings) send authenticated HTTPS requests to the API, which contains the integration gateway. The API commits each raw event and its outbox row to PostgreSQL, the system of record, in one transaction. Workers publish committed outbox events to NATS JetStream and consume them to normalize, deduplicate and route alerts. Workers also run escalation, notifications, status pages, stakeholder updates, chat channels, outbound webhooks, ticketing, audit streaming, heartbeats, postmortems, retention and reconciliation. They deliver through SMTP email, APNs, FCM or the push relay, Twilio SMS and voice, a voice bridge to your PBX, Slack, Teams and Mattermost, signed webhooks, Jira and Halo, and a SIEM. Responders are paged and act in the Web UI or the iOS and Android apps, which talk to the API. An identity provider (OIDC, LDAP, SCIM) and the CUCM gateway for live call routing are optional and connect to the API.

[Open the diagram](/assets/architecture.svg)

1. A monitoring source sends an authenticated event to the integration gateway in the API.
2. PostgreSQL commits the raw payload and outbox event together before the API returns accepted.
3. An outbox publisher sends committed work to NATS JetStream; a consumer normalizes, routes and deduplicates it.
4. Incident state and escalation jobs are persisted in PostgreSQL.
5. Workers claim due escalation jobs and create notification work; provider attempts and results are tracked separately.
6. Responders view the same authoritative state through the Web/API and acknowledge or resolve the incident.

### What each dependency owns

| Component | Responsibility |
| --- | --- |
| PostgreSQL | Authoritative configuration, raw events, incidents, jobs, notification history and audit. |
| NATS JetStream | Durable transport for asynchronous committed work; not a replacement for the database. |
| Redis | Available in the on-prem profile; no current core correctness dependency. |
| Web/API | Authenticated interaction, ingress validation and authoritative state changes. |
| Workers | Processing, escalation, notification dispatch and reconciliation. |

### Concurrency and ordering

State transitions lock the incident and validate the requested version. Jobs and notification attempts use leases and durable state to support restart/recovery. Source timestamps prevent older alert state from replacing newer observations. Delivery is at least once with database idempotency; this is not a blanket exactly-once delivery promise.

### When infrastructure fails

- Database unavailable: state changes cannot safely commit; readiness fails and clients should retry appropriately.
- NATS unavailable: accepted outbox records remain in PostgreSQL. Broker supervision is separate from SQL workers; reconciliation can recover aged unprocessed raw events.
- Worker restart: pending work remains durable or leases expire for recovery.
- Provider outage: affected notification attempts fail/retry visibly; incident access and response remain available if core dependencies are healthy.
- Whole stream lost: published outbox records are not automatically all replayed. Restore/replay must be coordinated and tested.

### Capacity and deployment evidence

> **Validate your operating envelope**
> Local correctness tests are not a production throughput, latency, high-availability, or disaster-recovery certification. Benchmark the actual infrastructure and providers against agreed targets. Never advertise a capacity figure solely from a development machine.

---

## Observability and health

Monitor the response platform independently of the platform's own notification path.

### Signals to collect

- API request errors and latency; health/readiness failures.
- Unprocessed raw events, oldest backlog age, outbox publication delay and quarantined events.
- Escalation delay, overdue jobs and worker-loop progress.
- Notification attempts, failures, unknown outcomes and provider acceptance latency.
- PostgreSQL connections, replication/WAL health, storage and backup freshness.
- NATS stream replica health, pending messages and durable consumer progress.

### Local observability stack

The on-prem Compose observability profile provides Prometheus, Grafana with a provisioned dashboard, and an OTel Collector configuration. Grafana reporting and update checks are disabled in that profile. Keep private monitoring UIs behind customer access controls.

### OpenTelemetry boundary

AckNexa exposes Prometheus metrics and structured JSON logs. The included collector accepts internal OTLP metrics and exposes them for local Prometheus. A customer Collector can also scrape the application's metrics. Native application OTLP distributed traces are not implemented; do not assume trace correlation exists because a collector is deployed.

Keep metrics off public ingress. The Helm API ingress publishes only the API path prefixes, not /metrics; Prometheus scrapes the in-cluster API Service. The API's /metrics can require a bearer token (METRICS_TOKEN), and the bundled Compose nginx answers /metrics on the API host with 404.

### Health checks are not delivery checks

Readiness proves a limited dependency check, not that every provider, schedule or escalation path works. Combine probes with queue/progress metrics and periodic authenticated end-to-end tests using dedicated internal recipients.

### Independent alerting

Monitor AckNexa from another customer-controlled failure domain, and route critical platform alerts through a channel that does not rely only on AckNexa. A self-hosted Alertmanager and an independent dead-man check can run entirely on the private network. A dashboard on the same failed host cannot page about its own disappearance.

---

## Backups and disaster recovery

Protect authoritative state, queue recovery, configuration, and the keys needed to read it.

### What to protect

- PostgreSQL: configuration, incidents, raw events, jobs, notification attempts, outbox and audit.
- NATS stream state is optional: the workers recover every unprocessed raw event from PostgreSQL when the stream is empty. Use supported snapshots or a quiesced backup if you also want the queue files.
- ACKNEXA_CONFIG_KEY and other recovery secrets, protected independently from the database copy.
- TLS materials, IdP configuration/backups, deployment values, image digests and approved release artifacts.
- Customer-created monitoring dashboards and alert configuration. Redis is not authoritative for the current core.

### Small-install backup

```sh
bash scripts/onprem_backup.sh \
  /secure/path/acknexa.env \
  /secure/backups/acknexa-NEW
```

Use a new absolute destination. By default (ACKNEXA_BACKUP_MODE=live) the script takes a consistent database snapshot while ingress, API and workers keep paging, and saves the protected configuration and a checksum; NATS is not copied. ACKNEXA_BACKUP_MODE=quiesced instead pauses the Web, API, workers and NATS, also copies the NATS files, and restarts only previously running services; paging is paused while it runs. Supply the same optional Compose overlays through ACKNEXA_COMPOSE_OVERRIDES when applicable.

> **Backups contain secrets**
> The backup is not automatically an encrypted vault. Store it on encrypted customer-controlled media, restrict access, keep an independent copy, and define retention and verification procedures.

### Restore safely

1. Verify backup checksums and image/database/broker version compatibility.
2. Provision a new isolated target, not the live source. Restore protected configuration and the encryption key.
3. Restore the PostgreSQL dump into an empty database. The small-install restore script then starts with an empty NATS stream and the workers recover unprocessed raw events from PostgreSQL. If you restore NATS data or snapshots, use the supported, stopped-target procedure; do not merge files into a live stream.
4. Start matching application versions without test seeding. Restored pending escalations resume paging as soon as workers start, so point the target at test recipients unless this is a real cut-over. Inspect tenant configuration, audit history, pending work and incident state.
5. Use dedicated test recipients to verify delivery and ACK before switching traffic. Measure actual recovery time and data loss against agreed targets.

### Production recovery

Use continuous WAL/PITR and encrypted backups to customer-controlled NAS, another site, or an internal object store as appropriate. Cloud S3 is not required. Exercise primary loss, a host/failure-domain loss, broker quorum loss, and restoration of encrypted provider configuration.

### Do not replay blindly

A lost stream may include work whose outbox row was already marked published. Resetting every publication flag can create unwanted replay. Recovery must identify unprocessed work, respect durable idempotency, and account for ambiguous provider acceptance. Backup success alone is not a disaster-recovery pass.

---

## Upgrades and release acceptance

Rebuild deliberately, migrate safely, and validate before routing production traffic.

### Plan a release

1. Read the CHANGELOG upgrade notes for every release you skip. A historical migration may require a coordinated stop rather than a rolling deployment.
2. Run the paging upgrade check with the new image against the production database, and fix its findings.
3. Take and verify a recoverable backup, including encryption keys and state-service recovery information. Do not start an upgrade while a database dump is running: migrations wait behind it, and paging queries then queue behind the migrations.
4. Build or preload approved images and record immutable digests. Rebuilding and restarting are separate actions.
5. Run migrations with the schema-owner identity. Keep the runtime credential separate in production.
6. Roll out matching API, worker and Web images, then verify readiness and the end-to-end incident workflow.
7. Observe queue age, errors, provider outcomes, ACK latency and audit continuity before declaring success.

### The upgrade check gate

Release 1.0.0-rc.1 changes who may be paged: rotation and override participants are paged only while they are an acting member of the schedule's team, and organization viewers are never paged. Anyone else is skipped afterwards, which can leave a schedule with nobody on call. The read-only upgradecheck command reports this before you migrate.

```sh
docker run --rm --entrypoint upgradecheck IMAGE_OF_THE_NEW_RELEASE \
  paging --database-url "$DATABASE_URL" --format text
```

| Exit code | Meaning |
| --- | --- |
| 0 | Nothing changes who gets paged. |
| 3 | Findings: people who will stop being paged, or schedules that lose coverage. Fix them and rerun. |
| 1 | Error, such as a connection failure or a database too old for a direct upgrade. |

Helm runs the check as a pre-install/pre-upgrade hook Job before the migration Job, and findings fail the release. Compose runs an upgradecheck service that must succeed before migrate, API and worker start. On a fresh database, and once that migration has been applied, the gate passes without checking. After reviewing findings, continue with upgradeCheck.allowFindings=true (Helm) or ACKNEXA_ALLOW_UPGRADE_FINDINGS=true (Compose).

### Why restart did not update the UI

Restarting a container starts the same image again. Rebuild or pull the intended approved image, then recreate the relevant service. Make sure the browser is using the correct installation and port; a local test stack and a default development stack can coexist with different images and databases.

### Helm and migration hooks

The production chart runs the upgrade check and then a pre-install/pre-upgrade migration hook. Index builds in this release's migrations block writes to the affected tables while they run, the largest being on audit_events; rehearse on a restored copy to know how long they take. Required database Secrets, CA ConfigMaps and pull credentials must exist before these hooks run. Do not assume an External Secrets resource created in the same release has synchronized in time for first install.

### Rollback boundaries

An image rollback does not automatically reverse a database migration. Only use a previous image if it is compatible with the current schema. Do not run destructive down migrations as an automatic rollback step. Follow an explicitly tested recovery plan if schema/data restoration is required.

> **Do not roll back below 1.0.0-rc.1 after upgrading**
> After migration 000044, older images send Android FCM tokens to APNs and then deactivate them. After migration 000039, images from before the API-key actor fix fail API-key writes. Fix forward, roll back only to an image of this release, or restore the pre-upgrade backup into a new environment.

### Production acceptance

- Complete the real monitoring-source to responder ACK workflow for every enabled channel.
- Prove operation with internet egress denied for a private-only deployment.
- Test backup restore, private-CA rotation, failover and a representative load profile.
- Verify tenant isolation, least-privilege access, secret handling and audit preservation.
- Record PASS, FAIL or NOT TESTED rather than treating manifests or local mocks as production evidence.

---

## API and automation

Use the REST API deliberately, with tenant-scoped credentials, idempotency, and version-aware actions.

### API base and specification

Use your deployment's API origin, not the public documentation site's URL. Paths below are relative to that origin. The OpenAPI document is available as an [API specification download](/downloads/openapi.yaml). It describes implemented contracts but is not a promise that every current administration field is exhaustively modeled.

### Authentication

Password sign-in posts organization, email and password to /v1/auth/login and returns a bearer token with its expiry. Authenticated calls use Authorization: Bearer TOKEN. Store tokens securely, observe expiry/revocation, and do not put them in query strings. The Web UI uses a server-side cookie proxy rather than exposing its bearer token to browser storage.

```json
{
  "organization": "your-organization-slug",
  "email": "responder@example.internal",
  "password": "YOUR_PASSWORD"
}
```

### API keys and service accounts

For scripts, CI and other systems, use an API key instead of a person's session. Organization administrators manage keys under API Keys in Administer, and each person can manage their own personal keys. Send the key as Authorization: Bearer YOUR_API_KEY.

| Kind | Acts as | Role |
| --- | --- | --- |
| Personal | The person who created it. | Never higher than the owner's current role. |
| Organization | A dedicated service account named after the key, so its actions are attributed to the key by name. | Exactly the role chosen when the key was created. |

- The token is shown once at creation; only its hash is stored. Expiry is optional, from 1 to 730 days.
- An organization can have at most 50 active organization keys, and each person at most 10 active personal keys.
- A service account cannot sign in, and is left out of people pickers, on-call candidates, call routing, reports and postmortem owners. Revoking the key deactivates it.
- Whatever its role, a key cannot create API keys, generate a SCIM token, change notification rules or contact details, create a calendar feed link or register a push device; those need a signed-in session.
- Creation and revocation are audited with the key name and prefix, never the token. A revoked or expired key, or one whose owner is deactivated, gets 401.

### Core endpoints

| Method and path | Purpose |
| --- | --- |
| GET /v1/me | Inspect the authenticated principal. |
| GET /v1/incidents | List organization incidents; status filtering is available. |
| GET /v1/incidents/{id} | Read incident, timeline, and operational context. |
| POST /v1/incidents/{id}/acknowledge | Acknowledge using the current version. |
| POST /v1/incidents/{id}/resolve | Resolve with the current version and reason. |
| GET /v1/alerts | Inspect normalized alerts. |
| GET /v1/on-call | Inspect effective on-call information. |
| GET /v1/events | Authenticated server-sent incident events. |
| POST /v1/incidents/{id}/notes | Add a note; notify_stakeholders posts it as a stakeholder update. |
| POST /v1/incidents/{id}/status-updates | Publish a status page update. |
| POST /v1/incidents/{id}/ticket | Open a ticket on a ticketing connection. |
| GET / PUT /v1/incidents/{id}/postmortem | Read or write the incident's postmortem. |
| GET / POST /v1/maintenance | List or schedule maintenance windows. |
| POST /integrations/webhook/{id} | Accept a separately authenticated monitoring payload. |
| POST /v2/enqueue | Accept a PagerDuty Events API v2 compatible event. |
| GET, POST or HEAD /heartbeat/{token} | Record a heartbeat ping. |
| /scim/v2 | SCIM 2.0 provisioning, authenticated with the organization's SCIM token. |

### Version-aware response

```json
{ "version": 7 }
```

The example is an acknowledge request. Read the actual current incident version; do not hardcode it. Resolution additionally accepts a reason. If a concurrent change causes 409, refresh and reassess before resubmitting.

### Administration endpoints

The /v1/admin routes require organization_admin. Resource areas include organization, users, teams, services, schedules, policies, integrations, routing-rules, providers, audit, api-keys, status-pages, call-routes, ticketing, webhooks and scim. Integration tests, provider tests and onboarding completion have separate actions. Do not infer a supported HTTP verb merely from a resource name.

| Action | Endpoint |
| --- | --- |
| Create integration | POST /v1/admin/integrations |
| Rotate webhook secret | POST /v1/admin/integrations/{id}/rotate |
| Send integration test | POST /v1/admin/integrations/{id}/test |
| Save provider | PUT /v1/admin/providers/{channel} |
| Test provider | POST /v1/admin/providers/{channel}/test |
| Create API key | POST /v1/admin/api-keys |
| Export audit log | GET /v1/admin/audit/export?from=&to=&format= |
| Configure audit stream | PUT /v1/admin/audit/sink |
| Delivery health | GET /v1/operations |
| Retry dead letter | POST /v1/operations/dead-letters/{id}/retry |

### Error handling

| Response | Interpretation |
| --- | --- |
| 202 | Webhook accepted durably, not necessarily processed/delivered. |
| 400 | Invalid input; inspect the documented request shape. |
| 401 / 403 | Invalid authentication or insufficient authorization. |
| 404 | Resource unavailable in this authorized scope or path not found. |
| 409 | State/version or configuration conflict; refresh and investigate. |
| 429 | Throttled; honor Retry-After when provided. |
| 503 | A required service or processing dependency is unavailable. |

### Automation rules

- Reuse an idempotency key only for retries of the same integration event.
- Use bounded backoff and distinguish a failed response from an ambiguous committed request.
- Do not log bearer tokens, webhook keys, provider configuration, or raw customer payloads by default.
- Some listing endpoints are bounded, not exhaustive paginated exports. Validate coverage before using them for reporting.
- Test API clients against the exact deployed release; the schema download is a reference, not a hosted API service.

---

## Internal voice gateway

Connect a customer-operated telephony bridge without depending on third-party cloud calling.

### What is implemented

AckNexa can send voice notifications (paging calls from escalation policies) to an operator-configured HTTPS gateway and accept signed, sequenced callbacks. The gateway translates this contract into the customer's CUCM, SIP or PBX operations. For these paging calls no SIP stack or CUCM connector is bundled; the bridge and site-specific call testing remain customer responsibilities.

Inbound hotline calls are a different feature: [live call routing](/docs/call-routing/) includes a ready-made CUCM gateway.

### Configure the adapter

- Set VOICE_GATEWAY_URL to the private HTTPS endpoint and VOICE_GATEWAY_TOKEN to a random bearer token of at least 32 characters; _FILE is supported.
- Configure the API and workers consistently. In Helm, enable notifications.voiceGateway and provide its URL and Secret.
- Install the private CA, allow only the required network path, and restrict the gateway's permitted dial destinations.
- Ensure organization-level voice provider settings do not override the deployment gateway unintentionally.

### Outbound notification

AckNexa POSTs a JSON Message containing ID, Channel, Destination, Title, Body, IncidentID, Severity, Service, Environment and Status. The request uses Authorization: Bearer TOKEN and Idempotency-Key: NOTIFICATION_UUID. The gateway may return its call ID in X-Request-ID.

The gateway must authenticate, validate destinations, and persist idempotency before returning 2xx. Return success only for durable acceptance. Do not redirect: redirects are not followed. A successful HTTP call is not evidence that a phone rang or a person responded.

### Callback and acknowledgement

```json
{
  "event_id": "unique-gateway-event",
  "notification_id": "NOTIFICATION_UUID",
  "state": "answered",
  "sequence": 3,
  "digit": "1"
}
```

POST callbacks to /v1/voice/callback. Supported states are queued, dialing, ringing, answered, no_answer, busy, failed and completed. Only explicit digit 1 with answered invokes acknowledgement; answering alone does not ACK the incident.

Use X-AckNexa-Timestamp with an RFC3339 timestamp and X-AckNexa-Voice-Signature with the hex HMAC-SHA256 of timestamp + '.' + raw body. The HMAC key is SHA256 of the per-organization callback secret. The signature has no sha256= prefix on this voice endpoint. Clock skew must stay within five minutes.

### Credential provisioning and replay

A trusted operator provisions the callback key hash in voice_gateway_credentials for the organization. Use a separate secret from the outbound bearer token. Callback event IDs must be unique and sequence values increase per notification. Duplicate event IDs and stale sequences are handled without regressing call state; the responder must still be authorized.

### Acceptance test

1. Send a notification to a dedicated internal test extension and verify gateway idempotency under retry.
2. Observe queued/ringing/answered callbacks, then test digit 1 and verify the incident ACK and cancelled pending escalation work.
3. Exercise no-answer, busy, duplicate/stale callbacks, wrong signatures, expired timestamps and revoked responder access.
4. Record actual provider/call evidence before claiming CUCM/PBX interoperability.

---

## Troubleshooting

Follow the signal through authentication, ingestion, routing, escalation and delivery.

### I cannot see admin options

1. Confirm you are using the intended application URL and installation.
2. Check that your organization role is organization_admin, not only team_admin.
3. Sign out and back in after a role change, and refresh the browser.
4. Verify the deployed Web/API image versions. Rebuild and recreate services if code changed; restart alone retains the old image.

### Login or setup fails

- Verify the organization slug, email and password; check whether the user is active.
- If first-run setup is absent, check whether an organization already exists or the setup token is missing.
- For origin errors, match ACKNEXA_WEB_ORIGIN exactly to the browser scheme, hostname and port.
- For private HTTPS, install the right CA on servers and browsers; do not disable certificate verification.
- For OIDC, verify issuer discovery, redirect URI, claims, pre-provisioned user and issuer/subject binding.

### Webhook accepted but no incident

1. Record the returned raw event ID and whether the response said duplicate.
2. Inspect integration event history and Delivery Health for processing/quarantine.
3. Confirm the intended integration is enabled, the service is active, and a rule has not routed the alert elsewhere.
4. Check workers, database, broker and backlog age. Remember that 202 is acceptance, not completion.
5. If quarantined, correct the underlying issue and use the audited retry action.

### Incident exists but nobody was reached

1. Check the service's active escalation policy and its targets.
2. Inspect the effective on-call responder and any override at the incident time.
3. Verify active recipient, contact destination and channel preferences.
4. Inspect notification attempts for configuration, authentication, TLS, DNS or provider errors.
5. Test the actual selected provider. Confirm that an organization override is not disabling or replacing the deployment adapter.
6. For private-only networks, remove unavailable cloud channels from the operational policy.

### ACK failed or another notification arrived

A 409 indicates state changed; refresh and inspect before acting again. A network failure is not proof that an action failed to commit, so re-read the incident. A provider request authorized before ACK can still finish afterward; inspect the attempt timestamps rather than assuming escalation restarted.

### Backup or upgrade problem

Stop and preserve evidence. Do not delete volumes, reset outbox flags, edit audit rows, or force a down migration to make the error disappear. Check exact image/schema compatibility and use the [backup and recovery procedure](/docs/backup-recovery/) on a separate target.

If an upgrade stopped at the paging check, read its findings with docker compose logs upgradecheck or kubectl logs on the acknexa-upgradecheck Job, fix the team memberships it reports, and run the upgrade again. See [the upgrade check gate](/docs/upgrades/#the-upgrade-check-gate).

### Prepare a useful support report

- State the deployment model, image versions, affected time window and sanitized request/event IDs.
- Describe expected and observed behavior, including HTTP status and whether it is reproducible.
- Include relevant redacted logs and health/backlog observations.
- Never attach passwords, provider keys, session tokens, private images or raw customer payloads to a public issue.

Your deployment operator is the first point of contact for your installation. To reach the AckNexa team, write to hello@acknexa.com. No support entitlement or response time is implied.

---

## Capabilities and limitations

An honest boundary between implemented features, optional integrations, and validation still required.

### Release status

> **Release 1.0.0-rc.1: a release candidate for pilots**
> 1.0.0-rc.1 is the first release candidate for 1.0.0. It is intended for pilot deployments on customer infrastructure and is not yet a general-availability release. The deployment guides describe the supported architecture and the acceptance work each site still needs.

What has been checked for this release:

- Continuous integration passes: Go unit tests and PostgreSQL/NATS integration tests with the race detector, govulncheck, npm audit, a gitleaks secret scan, the Web build and UI tests, deployment contract tests and the container image builds. Release builds run the full suite first.
- An upgrade check runs before migrations in both Helm and Compose and stops an upgrade that would silently stop paging people. See [the upgrade check gate](/docs/upgrades/#the-upgrade-check-gate).

What remains before general availability:

- A real-world pilot deployment.
- An external penetration test.
- SCIM validation against a live Okta or Microsoft Entra ID tenant; so far it has been tested with SCIM test clients only.
- Store distribution of the mobile apps; for this release the operator builds the Android and iOS apps from source, and neither is in the App Store or Google Play.
- Validation of the CUCM gateway against a real CUCM cluster; so far it has been tested against an Asterisk-based CUCM stand-in.

### Implemented core

- Authenticated Alertmanager, Icinga, Alerta, PagerDuty Events API v2 compatible and generic event ingestion, plus heartbeat monitors.
- Normalization, severity mapping, routing history, duplicate protection and incident lifecycles.
- Teams, services, users, schedule layers/overrides, escalation policies and maintenance windows.
- Web incident response, acknowledgement, resolution, notes, audit and Delivery Health.
- Status pages with public titles and email subscribers, stakeholder updates, and postmortems with action items.
- Organization administration, provider configuration/testing, API keys and service accounts, reports and guided onboarding.
- Local passwords, SSO (OpenID Connect), LDAP / Active Directory sign-in, SCIM 2.0 provisioning and break-glass accounts.
- Audit export (JSON Lines or CSV) and audit streaming to HTTPS collectors or TLS syslog.
- Several organizations per deployment.
- Self-hosted Compose and Helm deployment paths, customer CA and private registry configuration, file-backed secrets and the upgrade check.

### Optional adapters

| Capability | Boundary |
| --- | --- |
| Email / APNs / FCM / Slack / Microsoft Teams / Mattermost / Twilio SMS and voice | Implemented adapters; real delivery and site configuration must be verified. |
| Push relay | Implemented relay with credentials pinned to each deployment's server address; only an incident reference leaves the site. |
| Internal voice (paging calls) | Implemented HTTPS adapter and signed callback contract; customer PBX bridge required. |
| Live call routing | Twilio numbers, a generic PBX gateway API, and a bundled CUCM gateway (Asterisk image plus the voicegateway service on a SIP trunk). |
| Ticketing | Jira Cloud, Jira Server / Data Center and Halo, with comment mirroring and optional ticket-closure sync. |
| iOS and Android apps | Repository clients built from source by the operator; signing, distribution, network and real-device acceptance are operator work. |
| Prometheus / Grafana / OTel | Self-hosted metrics/log integration and deployment examples; native app distributed tracing is not implemented. |

### Known limitations

- No WhatsApp adapter, email intake or SNMP trap intake.
- Additional organizations are created by the operator with the bootstrap command; there is no API or Web screen to add one, and no organization delete.
- Incident numbers are shared across all organizations of a deployment, so each organization sees gaps in its numbering.
- The mobile apps keep one account per server address, so a phone can be signed in to only one organization of a deployment at a time.
- Notification and subscriber emails are minimal plain-text messages; there are no branded templates.
- No Web-based OIDC identity unlinking/rebinding.
- No exhaustive minute-by-minute future on-call coverage timeline.
- No full organization-admin configuration screens for every team-admin workflow.
- No unbounded UI exports, complete analytics/reporting product, or turnkey retention/compliance suite.

### Verification is not certification

CI runs the backend race/integration tests, the Web UI tests and the deployment contract tests, and database backup/restore has been exercised. AckNexa also comes with an end-to-end lab with real Prometheus, Alertmanager and Icinga 2, the mobile apps and a CUCM stand-in, for operators to repeat these checks. None of this certifies production failover, point-in-time recovery targets, capacity, private registry interoperability, real SMTP/IdP/PBX installations, or end-user mobile delivery at your site.

> **Production gate**
> Treat production HA/DR, representative load, independent monitoring, private-CA rotation and real-provider acceptance as site-specific release gates. This site does not advertise a production SLA, customer count, benchmark guarantee or compliance badge.

### Commercial and legal information

AckNexa is self-hosted on your infrastructure; it is not offered as a hosted subscription, and it is not open source. Pricing, licence terms and support are arranged directly: write to hello@acknexa.com.

---

## Frequently asked questions

Straight answers about hosting, notifications, privacy, and everyday use.

### Does AckNexa require a public cloud?

No production-critical core workflow requires a vendor-hosted AckNexa control plane or public cloud API. Customer PostgreSQL, NATS, Web/API/workers, ingress, secrets, monitoring and backups can run on premises.

### Can it run without internet access?

The core can operate in an isolated private network once images, configuration and dependencies are provisioned. Use internal notification channels and private identity/DNS/PKI. APNs, FCM, the push relay and SaaS providers still require their own external connectivity.

### Is this website a hosted application?

No. It is a separately deployable public product site and guide. Your administrator provides the URL and credentials for your own AckNexa installation.

### Is Redis mandatory?

Redis is supplied in the on-prem profile for customer-controlled use, but the current core does not depend on a cache for incident correctness. PostgreSQL remains authoritative.

### Does provider accepted mean the responder was notified?

It proves only the provider accepted the request. Delivery to a device or inbox and an explicit human acknowledgement are separate outcomes. Inspect attempts and verify with real recipients.

### Can I connect Cisco CUCM directly?

For hotline calls, yes: [live call routing](/docs/call-routing/) includes a gateway (an Asterisk image plus the voicegateway service) that takes calls over a SIP trunk from CUCM, with no UCCX, CUBE scripting or IVR licence. It has been tested against a CUCM stand-in, not a real CUCM cluster, so validate it on yours. Paging calls from escalation policies still go through Twilio or a customer-operated HTTPS voice bridge; see [Internal voice gateway](/docs/voice-gateway/).

### Is there an Android app?

Yes. A native Android app (Kotlin, Jetpack Compose, push through Firebase Cloud Messaging) is available alongside the iOS app. For this release the operator builds both from source; neither is in an app store. See [Web and mobile access](/docs/mobile/).

### Can one deployment serve several organizations?

Yes. Data and access are scoped per organization. The operator creates each organization after the first with the bootstrap command. Incident numbers are shared across the organizations of one deployment, and a phone can be signed in to only one organization per server address. See [Users and access](/docs/users-access/#several-organizations-on-one-deployment).

### Is the platform production-certified?

No. Release 1.0.0-rc.1 is a release candidate for pilot deployments, and the existence of these pages or manifests is not certification. Production HA/DR, scale and real-provider gates require measured acceptance on the intended infrastructure. Read [Capabilities & limitations](/docs/capabilities/).

### Does documentation search send my query elsewhere?

No. This website searches a local static index in your browser. No analytics or external search service is included. Your hosting operator may independently retain normal HTTP access logs.

### How do I contact the AckNexa team?

Write to hello@acknexa.com. Report a suspected vulnerability to security@acknexa.com instead, never in a public issue. For access, accounts and incidents in your own installation, your deployment administrator remains the first contact.

---

## Glossary

The vocabulary used in AckNexa and throughout this product guide.

### Product terms

| Term | Meaning |
| --- | --- |
| Acknowledgement / ACK | An authorized responder's explicit acceptance of responsibility for an incident. |
| Alert occurrence | A recorded observation contributing to a normalized alert's history. |
| API key | An automation token that acts as its owner or as a dedicated service account, within a fixed role. |
| Break-glass account | A local-password organization administrator that SCIM, SSO and LDAP never change, kept for when the identity provider fails. |
| Dead letter / quarantine | Work preserved after processing could not safely complete, for diagnosis and controlled retry. |
| Escalation | Progression through configured targets and channels while a response is still needed. |
| Fingerprint | A stable representation of alert condition identity. |
| Heartbeat | A monitor that opens an incident when an expected ping does not arrive in time. |
| Idempotency | Safe handling of repeat delivery/processing without duplicating the intended state change. |
| Integration | An authenticated inbound monitoring connection with an owning service and configuration. |
| JetStream | NATS persistence and consumer capabilities used for asynchronous event transport. |
| Lease | A bounded claim on work that another worker can recover after expiry. |
| Maintenance window | A period during which new incidents for a service are recorded but not paged. |
| On-call override | A temporary assignment that changes the effective responder during a time window. |
| Outbox | Database records of asynchronous work committed with the state change that produced it. |
| Postmortem | The blameless review of an incident, with root cause and action items. |
| Private CA | A customer-managed certificate authority trusted by internal clients and services. |
| Provider acceptance | A provider's acceptance of a notification request, not proof of human delivery or ACK. |
| Push relay | A service holding the mobile push credentials that forwards an incident reference, not incident content, to a phone. |
| Reconciliation | Background repair of durable work that was not completed through its expected path. |
| RPO / RTO | Agreed recovery-point/data-loss and recovery-time objectives, verified through drills. |
| SCIM | The standard protocol an identity provider uses to create, update and deactivate users and groups. |
| Stakeholder | Someone who follows a service or team for informational updates and is never paged. |
| Status page | A curated, customer-facing view of selected services and published incident updates. |
| Tenant | An organization boundary for data and authorization. |