iRonShore Technologies / Lighthouse docs Version 0.37.0

Setting up integrations

Step by step: what to make in Microsoft Teams, Slack, Jira and ServiceNow, what to paste into Lighthouse, how to test it, and what the usual errors mean. The same text is in the console under Help: Integrations and webhooks, Microsoft Teams setup, Slack setup, Jira setup and ServiceNow setup. What each integration receives, and the secret names, are in server.md, Integrations; webhooks and log forwarding are in server.md, Webhooks and Log forwarding.

All of them are on Settings, Integrations (under Settings in the rail, in Connections), for administrators and people with Look after Connections. What each one receives is chosen on Settings, Notifications (in People).

Every integration, the same five steps

  1. Make the receiving end in the other service: a workflow in a Teams channel, an app with an incoming webhook in Slack, an account with an API token in Jira, or an integration user in ServiceNow. The sections below say exactly what to click.
  2. In Lighthouse, open Settings, Integrations, press Add integration and pick the service. Paste what you copied, then press Save.
  3. Press Send test. Lighthouse posts a Lighthouse test message (or opens a test ticket you can close) straight away and says what the service answered.
  4. Open Settings, Notifications. The Integrations table has a row for each one: On, Expiring certificates, Failed renewals and Site changes. All four start ticked.
  5. Set the public address under Settings, Application if it is empty, so every post carries an Open Lighthouse link.
Integration You make there You paste into Lighthouse It gets
Microsoft Teams A workflow from the template Send webhook alerts to a channel Its webhook URL A card per alert run, plus notices
Slack A Slack app with an incoming webhook for the channel Its webhook URL A message per alert run, plus notices
Jira An account that may create issues in a project, and its API token Site address, project key, issue type, account email, API token One issue per certificate, failing job or site change
ServiceNow An integration user with the itil role Instance address, user, password, assignment group One incident per certificate, failing job or site change

What goes where. Expiring certificates: Teams and Slack get one post per alert run, listing every certificate that crossed a threshold; Jira and ServiceNow get one ticket per certificate, at the first threshold it crosses, never a second one at the next threshold. Failed renewals: one notice when a renewal or installation job starts failing, not again until it has worked once. Site changes: what Change watch finds after a scan. Teams and Slack also get notices that never become tickets: requests waiting for approval, recovered keys, Administrator or Key recovery given, planned installs, and the weekly team summary for teams without an email address. A Pipeline's Tell step can name any of the four, whatever its switches say. The words are changed under Settings, Notifications, Message wording.

Custom notifications

Three things shape what reaches a Teams or Slack channel (and Jira and ServiceNow):

  • Which kinds: the switches for each integration on Settings, Notifications (On, Expiring certificates, Failed renewals, Site changes). When expiry alerts start is Default alert thresholds on Settings, Application, or a Certificate expiring Pipeline's own thresholds on Automation, Pipelines. A Pipeline's Tell step reaches them when it tells The integrations that take expiry alerts.
  • The words: Settings, Notifications, Message wording. Pick a message (expiry, weak certificate, Change watch, renewal due, failed renewal, weekly team summary) and change its subject and text with placeholders such as {name}, {when}, {owners} and {link}; Preview shows the result. Teams and Slack get the same words as email; a Teams expiry card keeps its list of certificates under your subject.
  • Which certificates, and when: a Pipeline with a Tell step naming Teams or Slack, such as "Certificate expiring, tag:pci: Tell Slack" or "Install failed: Tell Teams". It goes whatever the integration's switches say. See Pipelines in Running the server.

Lighthouse has one Teams channel and one Slack channel, so every team's notices go to the same channel; a team's own alerts go to the team's email address. The card and message layout is fixed. For a layout of your own, or another channel, add a webhook for the events you want and let your own workflow (Power Automate, Slack Workflow Builder or a script) post what you like.

Before you start. Lighthouse posts from the server; nobody's browser is involved. The server that runs the jobs sends alerts and notices, and Send test goes from the server you are using, so each needs outbound HTTPS to the service: the Teams workflow's host (*.environment.api.powerplatform.com, or *.logic.azure.com for older workflows), hooks.slack.com, your Jira site and your ServiceNow instance. If the server reaches the internet through a proxy, set HTTPS_PROXY (and NO_PROXY for your own names) as system environment variables and restart IIS; .NET reads them before the signed-in user's proxy settings, which a service account does not have. A service on your own network, such as Jira Data Center, needs a certificate this server trusts.

Secrets. Webhook URLs, API tokens and passwords are kept in the server's secret store and never shown again; the field then says saved; type to replace. Leave it empty to keep it. A webhook URL is as good as a password: anyone who has it can post in the channel. On a new server, type them in again. Remove deletes the integration and its secret. To read one from CyberArk, Delinea, Vault or Azure Key Vault instead, point it there on Settings, Password managers (docs/password-managers.md).

When a post fails, the error names the service and what it answered, never the URL or a password, and Teams and Slack errors say what the answer usually means. Alerts that could not be sent are in the Keeper's Log as alert-failed and in the Server Log.

Microsoft Teams

Lighthouse posts cards to a Teams channel through a workflow (the Workflows app in Teams, part of Power Automate). Microsoft has retired the old Incoming Webhook connector, so a webhook.office.com URL no longer works; make a workflow instead.

You need: a standard channel (the workflow posts as Flow bot, which cannot post in private channels), Workflows allowed in your Microsoft 365 tenant, and an account to own the workflow. The workflow stops when its owner's account is disabled, so use a shared or service account, or add a co-owner. No premium Power Automate license is needed.

  1. In Teams, go to the channel, press ••• (More options) next to its name and choose Workflows. Or open the Workflows app and create from a template.
  2. Pick the template Send webhook alerts to a channel. Not from specific people or from people in an org: those want callers to sign in with Microsoft, which Lighthouse does not do, and refuse its posts with HTTP 401.
  3. Name it, such as Lighthouse alerts, check it is signed in as the owning account, and press Next.
  4. Choose the Team and Channel and press Save (Add workflow).
  5. Copy the webhook URL it shows, all of it. It ends in a long sig= value, which is the secret. To find it later, open the workflow in Workflows (Manage workflows) or Power Automate: it is on the first step, When a Teams webhook request is received.
  6. In Lighthouse, Settings, Integrations, Add integration, Microsoft Teams. Paste it into Workflow webhook URL and press Save, then Send test.
  7. A card titled Lighthouse test message appears in the channel within a minute. Choose what Teams gets under Settings, Notifications.

What it posts: an Adaptive Card (version 1.4, in a message with attachments). Expiry alerts list each certificate with when it expires and where it was seen; other notices are a title and text. Both have an Open Lighthouse button when the public address is set.

You see What to do
Send test says it was posted, but nothing shows Teams took the post and the workflow failed afterwards. Open the workflow's run history in Workflows or Power Automate and see which step failed: usually the owner's sign-in expired (fix its connection), the channel is private, or the channel or team was deleted.
HTTP 400 The workflow could not read the card. Make it again from Send webhook alerts to a channel; a workflow built by hand must post the card from attachments.
HTTP 401 or 403 The URL was cut short, or the workflow needs callers to sign in. Copy the whole URL again from a workflow made with Send webhook alerts to a channel.
HTTP 404 The workflow was deleted or turned off. Turn it on, or make a new one and paste its URL.
HTTP 429 Microsoft is holding back posts to the workflow for a while. Wait a few minutes and send again.
Could not reach Teams This server cannot make outbound HTTPS connections to the workflow's host. Open the firewall or set a proxy.
The webhook must be an https URL What you pasted is not the whole URL. Copy it again.

Slack

Lighthouse posts to one Slack channel through an incoming webhook on a small Slack app of your own. The app only posts; it cannot read anything in Slack.

You need: a Slack account allowed to create apps in the workspace (some workspaces send new apps to an administrator for approval first), and the channel. For a private channel, you must be in it.

  1. Open api.slack.com/apps, press Create New App, choose From scratch, name it, such as Lighthouse, pick the workspace and press Create App. The name and an icon, under Basic Information, are what the posts show.
  2. Under Features, open Incoming Webhooks and turn Activate Incoming Webhooks on.
  3. Press Add New Webhook to Workspace, pick the channel and press Allow. If your workspace needs approval, this waits for a Slack administrator.
  4. Under Webhook URLs for Your Workspace, press Copy. The URL looks like https://hooks.slack.com/services/T…/B…/….
  5. In Lighthouse, Settings, Integrations, Add integration, Slack. Paste it into Incoming webhook URL and press Save, then Send test.
  6. Lighthouse test message appears in the channel. Choose what Slack gets under Settings, Notifications.

What it posts: a message with the title in bold, the text, and an Open Lighthouse link when the public address is set. Slack finds webhook URLs leaked in public places and turns them off, so keep it out of scripts, tickets and chats.

Slack answers What to do
no_service, no_active_hooks, invalid_token The webhook was removed, turned off, revoked by Slack or not copied whole. Add a new one under the app's Incoming Webhooks and paste its URL.
channel_not_found The channel was deleted, or the app was taken out of it. Add a new webhook for the channel.
channel_is_archived Unarchive the channel, or add a webhook for another one.
action_prohibited, posting_to_general_channel_denied A Slack workspace administrator does not let the app post there. Ask them, or pick another channel.
team_disabled The Slack workspace is no longer active.
Could not reach Slack This server cannot make outbound HTTPS connections to hooks.slack.com. Open the firewall or set a proxy.

Jira

Lighthouse opens a Jira issue for each certificate that needs attention, each renewal job that starts failing, and each site change, with the label lighthouse. It uses REST API version 2 and works with Jira Cloud and Jira Data Center (or Server 8.14 and later).

You need:

  • A project, such as PKI, and an issue type in it, Task unless you choose another.
  • An account Lighthouse signs in as, best one of its own, with Browse projects and Create issues in that project (the project's permission scheme, or a role that has them).
  • The project's create screen for that issue type must have Summary, Description and Labels, and no other required field without a default. Lighthouse fills in only those three.

Jira Cloud: make an API token, signed in as the account Lighthouse uses:

  1. Open id.atlassian.com, Security, API tokens.
  2. Press Create API token, not Create API token with scopes: a token with scopes works only through api.atlassian.com, which Lighthouse does not use, and gets HTTP 401.
  3. Name it, such as Lighthouse, pick when it expires (1 to 365 days; a year by default), press Create and Copy to clipboard. It is not shown again. Put a reminder in your calendar: when it expires, issues stop opening and Send test says HTTP 401.

Jira Data Center: make a personal access token, signed in as the account Lighthouse uses: your picture, Profile, Personal Access Tokens, Create token. Name it, choose whether it expires, press Create and copy it.

In Lighthouse, Settings, Integrations, Add integration, Jira:

Field What to put
Site address What you open Jira at: https://yourcompany.atlassian.net, or your Data Center address with its path, such as https://jira.example.com/jira
Project key The letters in front of the project's issue numbers: PKI for PKI-123
Issue type Exactly as the project shows it, in the site's language. Empty means Task
Account email Jira Cloud: the email of the account the token belongs to (sent with the token as basic authentication). Data Center: leave it empty, so the token is sent as a personal access token (Bearer)
API token The token you copied

Press Save, then Send test: an issue Lighthouse test message opens in the project. Close it. Choose what Jira gets under Settings, Notifications. A Jira automation rule on the label lighthouse can assign the issues or set their component.

Jira answers What to do
HTTP 401 The token expired or was revoked, the email is not the token's account, or the email is empty on Jira Cloud (or filled in on Data Center). Make a new token, or fix the email.
HTTP 403 The account may not create issues in the project. On Data Center, too many failed sign-ins ask for a CAPTCHA: sign in once as that account in a browser.
HTTP 404 The site address is wrong. Data Center needs its path, such as /jira.
issuetype: Specify a valid issue type The project has no issue type by that name. Type it as the project shows it.
project: Specify a valid project ID or key The key is wrong, or the account cannot see the project.
labels: Field 'labels' cannot be set Add Labels to the create screen for that issue type (the same for Description).
… is required The project wants a field Lighthouse does not fill in. Make it optional, give it a default, or use another issue type.
Could not reach Jira Open outbound HTTPS from this server to the site. For Data Center, the server must also trust Jira's certificate.

ServiceNow

Lighthouse opens a ServiceNow incident for each certificate that needs attention, each renewal job that starts failing, and each site change, through the Table API (POST /api/now/table/incident). Each incident has Correlation display Lighthouse and Correlation ID lighthouse: and the certificate's SHA-256 thumbprint (lighthouse:job-<id> for a failing job), urgency 1 when something has expired or failed (2 otherwise) and impact 2.

In ServiceNow, as an administrator, make a user for Lighthouse:

  1. All, User Administration, Users, press New. User ID such as lighthouse.integration, a name such as Lighthouse, tick Web service access only (it can use the API but never sign in to the web pages) and Active. Press Submit.
  2. Open the user again, press Set Password, make a password, copy it and save it. Make sure Password needs reset is not ticked, or every API call is refused.
  3. Under Roles, press Edit and add itil, which may create incidents. Save.
  4. To send incidents to a group, note its Name exactly as User Administration, Groups shows it.

In Lighthouse, Settings, Integrations, Add integration, ServiceNow: Instance address (https://yourcompany.service-now.com), User (the User ID), Password, and optionally Assignment group (empty leaves incidents unassigned). Press Save, then Send test: an incident Lighthouse test message opens. Close or cancel it. Choose what ServiceNow gets under Settings, Notifications. To list Lighthouse's incidents, filter Incidents on Correlation display is Lighthouse.

ServiceNow answers What to do
HTTP 401 Wrong user or password, the user is not active or is locked out, or Password needs reset is ticked.
HTTP 403 The user lacks the itil role, an access control rule refuses it, or your instance only takes calls from listed addresses (IP Address Access Control): add this server's outbound address.
HTTP 400 or Data Policy Exception Your instance requires a field Lighthouse does not fill in, such as Caller or Category. Give it a default for this user, or relax the data policy for web services.
Incidents have no assignment group The name does not match a group exactly. Copy it from User Administration, Groups.
Could not reach ServiceNow Open outbound HTTPS from this server to the instance.

Webhooks and log forwarding

Webhooks post signed JSON to a service of your own (Add webhook on Settings, Integrations). The receiver checks Lighthouse-Signature, which is sha256= and the hex HMAC-SHA256 of Lighthouse-Timestamp, a dot and the body, made with the signing secret. In Python:

import hashlib, hmac, time

def from_lighthouse(secret: str, headers, body: bytes) -> bool:
    stamp = headers["Lighthouse-Timestamp"]
    if abs(time.time() - int(stamp)) > 300:
        return False
    mac = hmac.new(secret.encode(), stamp.encode() + b"." + body, hashlib.sha256).hexdigest()
    return hmac.compare_digest("sha256=" + mac, headers["Lighthouse-Signature"])

Answer with any 2xx status, quickly, and do the work afterwards. Use Lighthouse-Delivery to skip a post you already have, since a retry sends the same one again. The events and retries are in server.md, Webhooks.

Log forwarding (Add log destination on Settings, Integrations) needs a receiving end:

  • Splunk Enterprise: Settings, Data inputs, HTTP Event Collector. Under Global Settings, set All Tokens to Enabled (and Enable SSL on), then New Token: name it Lighthouse, pick the index it may write to, and copy the token value into HEC token. The URL is https://<splunk server>:8088. Splunk Cloud gives HEC addresses of its own, such as https://http-inputs-<stack>.splunkcloud.com.
  • Syslog: have the receiver listen on TCP, or better TLS on 6514, and accept RFC 5424 messages. UDP can lose messages without anyone knowing.
  • Elastic: make an API key in Kibana (Stack Management, API keys) that may write to the index, then put Authorization in Header for the token and ApiKey, a space and the encoded key in Token. Datadog: DD-API-KEY in Header for the token, an API key from Organization Settings, API Keys in Token, and your Datadog site's logs intake address as the URL.

The rest is in server.md, Log forwarding.