Event Streams

Event Streams Overview

Event Streams enable SGNL to receive events that comply with the Security Event Token (SET) Specification. This includes Shared Signals Framework events like CAEP and RISC.

SGNL’s event streams are a special type of System of Record, and you can find our standards-based templates right next to other SoRs that you may be using in SGNL’s System of Record Catalog. Similar to all other SoRs, Event Stream SoR’s enabling the modeling of different entities that are then stored in the graph, enabling you to create CAEP Hub Actions when you receive an event.

Things to know about Event Streams

While Event Stream SoRs like our CAEP and RISC templates share similarities with other SoRs in terms of how you model entities/attributes and how data is propagated to the graph, this is where many of the similarities end.

A few important things to know:

  • SGNL Event Stream SoRs are used primarily to process Security Event Tokens (SETs) that are designed to communicate security-state across systems
  • The current version of Event Streams support only push delivery of SETs to SGNL
  • The JTI of a SET is expected to be unique, SGNL will reject SETs that provide duplicate JTIs on the same stream
  • Events received from a SET/SSF Transmitter are required to have an id attribute in SGNL, this should be the jti field in the SET, they’re also required to have a subject attribute in SGNL that should correspond to one of the Subject identifiers in the JWT
  • Events are designed to communicate security state that is not expected to be long-lived, given that – SGNL stores the last 30 days of events received in the SGNL graph, and stores a maximum of the 50 most recent events per subject
  • To successfully receive an event in SGNL, you must have either configured Authentication or Signing verification in SGNL, and have applied that consistently when pushing the SET to SGNL

Creating your first Event Stream

Creating an Event Stream is fairly straightforward, with the steps below you can get started with your first set of events, and quickly expand to other event sources.

  1. Login to SGNL with an account that has at least SoR Administrator Permissions

  2. From the Dashboard, Add a new System of Record

  3. Using the SGNL Catalog, select the CAEP Events SoR

    Event Stream - CAEP

  4. Give the Event Stream a name – if you don’t have an existing SSF Transmitter, you can use SGNL’s CAEP Hub or caep.dev for this purpose and name this stream accordingly, save it after you’re done

  5. Head over to Settings, as you’ll need to enable either Authentication or Signing on this Event Stream source before you can use it – SGNL’s well-known endpoint for CAEP Hub is available by default at https://config.sgnlapis.cloud/metadata/v1/.well-known/ssf-configuration for the SGNL SaaS Note: If you’re not yet sure how you’re going to use this event stream, it might be easier at this juncture to enable Authentication and not Signing, and simply use Postman or cURL to send your first few messages, if doing this, just generate a new token and store it somewhere safely for now

  6. Take note of your Push Event API, that’s where you and/or your partners will send SETs to

  7. Head over to the Events section of the Event Stream and verify all the events you will want to receive are there – at this point it’s most likely interesting to look at the configuration of one of the events and verify that the JSONPath Syntax that is in use to parse attributes out of the SET reflects the schema of your Security Event Tokens – if you aren’t sure, just leave them as-is for now

  8. You now want to create a relationship or two between events objects in your graph – you’ll commonly want to create your first relationship from the subject of the token to some authoritative source for that value, i.e. if you are sending an email as the sub_id, you might want to create a relationship between the Event’s subject attribute and the mail attribute in your IdP or Email Provider SoR

  9. Once everything looks good, return to the Events page and enable one of the Events, Session Revoked is a good start, also enable the Event Stream from the top right

  10. You’re ready to start receiving events – you can now proceed to configure your SSF/SET Transmitter to send (POST) events, or use CAEP Hub or caep.dev to test – if you’re looking for some quick testing steps, follow along below

Testing your first Event Stream

So you have a new Event Stream configured and looking to test it out? CAEP Hub and caep.dev are the best possible test beds for this purpose, but we’ll go through the fastest possible way below.

This guide assumes that you’ve completed creating your first Event Stream. What you’ll need is:

  • The POST URL for your Event Stream
  • An Authentication Token for your Event Stream, with Authentication enabled (and Signing disabled)
  • A mechanism to make REST requests such as cURL or Postman
  1. First, get a SET Payload that makes sense for your use-case, this guide assumes you’ve enabled the Session Revoked event in the SGNL Event Stream, a sample is below:
{
 "aud": "https://sgnl.ai",
 "events": {
   "https://schemas.openid.net/secevent/caep/event-type/session-revoked": {
     "event_timestamp": 1729743269,
     "subject": {
       "email": "alice@wholesalechips.co",
       "format": "email"
     }
   }
 },
 "iat": 1729743269,
 "iss": "https://ssf.wholesalechips.co/",
 "jti": "ABJmMDgxZGMtZmVjYy11NjA1LWFjY2QtNzAiM2UEZmYzZTBh",
 "sub_id": {
   "email": "alice@wholesalechips.co",
   "format": "email"
 }
}

Note: Ensure you haven’t used the jti previously for this event stream, SGNL will reject duplicate JTI’s

  1. Next, you’ll want to Base64 encode that payload and then pre-pend eyJhbGciOiJub25lIiwidHlwIjoiSldUIn0. and append a period . – the result should look a little something like:
eyJhbGciOiJub25lIiwidHlwIjoiSldUIn0.ewogICJhdWQiOiAiaHR0cHM6Ly9zZ25sLmFpIiwKICAiZXZlbnRzIjogewogICAgImh0dHBzOi8vc2NoZW1hcy5vcGVuaWQubmV0L3NlY2V2ZW50L2NhZXAvZXZlbnQtdHlwZS9zZXNzaW9uLXJldm9rZWQiOiB7CiAgICAgICJldmVudF90aW1lc3RhbXAiOiAxNzI5NzQzMjY5LAogICAgICAic3ViamVjdCI6IHsKICAgICAgICAiZW1haWwiOiAiYWxpY2VAd2hvbGVzYWxlY2hpcHMuY28iLAogICAgICAgICJmb3JtYXQiOiAiZW1haWwiCiAgICAgIH0KICAgIH0KICB9LAogICJpYXQiOiAxNzI5NzQzMjY5LAogICJpc3MiOiAiaHR0cHM6Ly9zc2Yud2hvbGVzYWxlY2hpcHMuY28vIiwKICAianRpIjogIkFCSm1NRGd4WkdNdFptVmpZeTExTmpBMUxXRmpZMlF0TnpBaU0yVUVabVl6WlRCaCIsCiAgInN1Yl9pZCI6IHsKICAgICJlbWFpbCI6ICJhbGljZUB3aG9sZXNhbGVjaGlwcy5jbyIsCiAgICAiZm9ybWF0IjogImVtYWlsIgogIH0KfQ.
  1. You’ll now want to send that to your Event Stream, you’ll need to update the URL, Token, and Data accordingly, with cURL that request looks somewhat like:
curl --location 'https://events.sgnlapis.cloud/events/ssf/v1/11111111-2222-3333-4444-555555555555' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer eyJ.. (Your Auth Token)' \
--data 'eyJhbGciOiJub25lIiwidHlwIjoiSldUIn0.ewogICJhdWQiOiAiaHR0cHM6Ly9zZ25sLmFpIiwKICAiZXZlbnRzIjogewogICAgImh0dHBzOi8vc2NoZW1hcy5vcGVuaWQubmV0L3NlY2V2ZW50L2NhZXAvZXZlbnQtdHlwZS9zZXNzaW9uLXJldm9rZWQiOiB7CiAgICAgICJldmVudF90aW1lc3RhbXAiOiAxNzI5NzQzMjY5LAogICAgICAic3ViamVjdCI6IHsKICAgICAgICAiZW1haWwiOiAiYWxpY2VAd2hvbGVzYWxlY2hpcHMuY28iLAogICAgICAgICJmb3JtYXQiOiAiZW1haWwiCiAgICAgIH0KICAgIH0KICB9LAogICJpYXQiOiAxNzI5NzQzMjY5LAogICJpc3MiOiAiaHR0cHM6Ly9zc2Yud2hvbGVzYWxlY2hpcHMuY28vIiwKICAianRpIjogIkFCSm1NRGd4WkdNdFptVmpZeTExTmpBMUxXRmpZMlF0TnpBaU0yVUVabVl6WlRCaCIsCiAgInN1Yl9pZCI6IHsKICAgICJlbWFpbCI6ICJhbGljZUB3aG9sZXNhbGVjaGlwcy5jbyIsCiAgICAiZm9ybWF0IjogImVtYWlsIgogIH0KfQ.'
  1. SGNL should send back a 202 Accepted response as a result
  2. If you go to Data Lens, you should see that object in your Graph, as well as the Raw and Parsed JWT as fields in Data Lens

Troubleshooting Event Streams

There are a range of troubleshooting tools within SGNL to help troubleshoot issues you might see with Event Streams in SGNL

HTTP Response Codes

SGNL’s Event Stream APIs will provide standards-confirming response codes and error codes - in many cases, these responses and messages will enable you or your partners to debug the cause of the issue without further investigation. A successful Event Push is accepted with a 202 status code.

SGNL Logs

It really starts with SGNL Logs and we have a number of logs that are interesting to Administrators in SGNL, by heading over to Logs and Event Streams, you’ll have a number of log channels.

You can start with Event Received Logs, this will give you a full output of the Event that was received, inclusive of the whole base64 payload. You can decode this JWT in order to understand the values that were present.

You can verify that not only was the event received by SGNLs APIs, but it was also stored from the Event Stored logs that list out the JTI for each event that has been successfully stored.

The Event Failed logs, these logs are designed to provide a comprehensive mechanism for debugging issues with Event Streams. Some common problems that cause event reciept to fail

  • Issue: The Event being sent is not enabled in the Event Stream
    • Resolution: Enable the Event in the Event Stream
  • Issue: The Event Stream itself is not enabled
    • Resolution: Enable the Event Stream
  • Issue: The Event being sent is not configured in the Event Stream
    • Resolution: From the Event Received Log, you can quickly grab the encoded SET and use that as a guide to create a new entity in the Event Stream that matches this event type
  • Issue: Signing Validation is failing
    • Resolution: Verify the Event is actually being signed by decoding the SET, if it is signed, verify that the KID in the Header of the token matches a KID in the JWKS that’s configured
  • Issue: Token is being received, but not visible in the SGNL graph
    • Resolution: Check to ensure that your attribute configuration is correct, remember that fields like iat and exp are of type DateTime (not Strings), and need to be configured as such