Creating and Configuring an Entity Push System of Record

Early Access Feature: Entity Push Systems of Record are currently available as an Early Access feature. This feature enables real-time updates to the SGNL graph through SCIM Security Event Tokens (SETs), providing real-time graph updates for use in policy evaluation and CAEP Hub.

Entity Push Systems of Record enable objects in the SGNL Graph to be updated using SCIM Security Event Tokens (SETs) delivered via REST commands against the SGNL Identity Fabric APIs. Unlike traditional Entity Poll Systems of Record that periodically synchronize data, Entity Push accepts real-time event-driven updates that are permanently persisted in the SGNL Graph as fully-fledged objects from upstream Systems of Record.

Entity Push is particularly valuable for organizations requiring real-time graph updates for use in policy evaluation and CAEP Hub. When identity events occur in source systems, they can be pushed to SGNL to update graph state and trigger downstream actions, enabling near-real-time access orchestration workflows.

Important: While Entity Push provides real-time updates, there should still be an authoritative system for the data being pushed. Entity Push facilitates rapid updates to the SGNL graph, but the source system remains the system of record for the underlying data.

Prerequisites

  • Understanding of SCIM specifications and Security Event Tokens (SETs)
  • Systems capable of generating and transmitting SCIM SETs according to the SCIM Events specification
  • Network connectivity for event push delivery to SGNL endpoints
  • Understanding of JWT (JSON Web Token) structure and signing concepts
  • Administrative access to configure Entity Push Systems of Record and authentication

Understanding SCIM Security Event Tokens

SCIM Security Event Tokens follow the SCIM Events specification and are delivered as JSON Web Tokens (JWTs) containing standardized event payloads. Each SET includes standard JWT claims (issuer, audience, issued at, JWT ID) plus SCIM-specific event claims that provide identity context.

Supported SCIM Event Types

Entity Push supports the following SCIM event operations:

Create Operations: urn:ietf:params:SCIM:event:prov:create:full - Creates new entities in the SGNL graph with complete resource representations

Update Operations: urn:ietf:params:SCIM:event:prov:put:full - Updates existing entities with complete resource representations, replacing all attributes

Delete Operations: urn:ietf:params:SCIM:event:prov:delete - Removes entities from the SGNL graph

Note: SGNL supports only ‘full’ operations.

SCIM SET Structure

Each SCIM SET contains several key components:

Standard JWT Claims: Include iss (issuer), aud (audience), iat (issued at), and jti (JWT ID) that provide token metadata and enable proper token verification.

Subject Identification: The sub_id claim identifies the subject of the event using SCIM format specifications, including format type, resource URI, external ID, and internal ID for proper entity identification.

Event Payload: The events claim contains the specific SCIM event with its operation type and complete resource data, including all attributes needed for the entity operation.

Here’s an example of a SCIM SET for creating a user:

{
  "jti": "4d3559ec67504aaba65d40b0363faad8",
  "iat": 1458496404,
  "iss": "https://scim.example.com",
  "aud": ["https://events.sgnlapis.cloud/events/scim/v1/2819c223-7f76-453a-919d-413861904646"],
  "sub_id": {
    "format": "scim",
    "uri": "/Users/44f6142df96bd6ab61e7521d9",
    "externalId": "jdoe",
    "id": "100"
  },
  "events": {
    "urn:ietf:params:SCIM:event:prov:create:full": {
      "data": {
        "schemas": ["urn:ietf:params:scim:schemas:core:2.0:User"],
        "emails": [
          {
            "type": "work",
            "value": "jdoe@example.com"
          }
        ],
        "userName": "jdoe@example.com",
        "name": {
          "givenName": "John",
          "familyName": "Doe"
        },
        "enabled": false
      }
    }
  }
}

Authentication and Security

Entity Push Systems of Record support two verification methods that provide different levels of security assurance:

Bearer Token Authentication

SGNL generates authentication tokens that event sources must include in the Authorization header when delivering events. This method provides authentication for event sources and is managed through SGNL’s token generation system.

JWT Signature Verification

Event sources can cryptographically sign SCIM SETs using JSON Web Signature (JWS) with keys distributed through JWKS endpoints or well-known configuration endpoints. This method provides cryptographic verification of event authenticity and integrity.

Security Best Practices: For production deployments, enable JWT signature verification to ensure event authenticity. Bearer token authentication alone provides basic access control but does not verify event integrity. Consider implementing both methods for maximum security assurance.

Configuring SGNL

Creating an Entity Push System of Record

  1. Access SGNL Console: Login to the SGNL Console with appropriate administrative privileges
  2. Navigate to Identity Data Fabric: From the left menu, select “Identity Data Fabric”
  3. Add New System of Record: Click “Add System of Record” or “Add” to open the SoR catalog
  4. Create Custom SoR: Click “Create Custom SoR” to open the template editor for Entity Push configuration

Entity Push Template Configuration

Entity Push Systems of Record use familiar SoR templates that define the entity structure and event processing configuration. Use the following template as a starting point.

Note At General Availability, SCIM SET Templates will be published in the SGNL SoR Catalog.

displayName: Entity Push SoR
icon: PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4KPCEtLSBHZW5lcmF0b3I6IEFkb2JlIElsbHVzdHJhdG9yIDI2...
description: Entity Push
hostname: sgnl.ai
defaultApiCallFrequency: SECONDLY
defaultApiCallMinInterval: 1
pushType: "SCIM"
deliveryMethod: "entityPush"
adapterConfig: e30=
entities:
  User:
    displayName: Users
    externalId: Users
    pushExternalId: Users
    description: ""
    pagesOrderedById: false
    entityAlias: 62864663-0a74-4c6f-9dc8-81d2ae29bdcd
    attributes:
      - name: userName
        externalId: userName
        pushExternalId: $.events.*.data.userName
        description: ""
        type: String
        indexed: true
        uniqueId: false
        attributeAlias: 1b51557d-41d8-4d46-b35b-5dbe928a8718
        list: false
      - name: id
        externalId: id
        pushExternalId: $.sub_id.id
        description: ""
        type: String
        indexed: true
        uniqueId: true
        attributeAlias: 34c283e3-2796-4094-8dcb-ae2f4a2f9a46
        list: false
      - name: enabled
        externalId: enabled
        pushExternalId: $.events.*.data.enabled
        description: ""
        type: Bool
        indexed: false
        uniqueId: false
        attributeAlias: cf9e4484-daf5-4ab8-9dbf-2dd0e3defb17
        list: false
  Email:
    parent: Users
    displayName: emails
    externalId: emails
    pushExternalId: "$.events.*.data.emails.*"
    description: Entity representing user emails
    pageSize: 100
    pagesOrderedById: false
    attributes:
      - name: type
        externalId: type
        pushExternalId: type
        description: ""
        type: String
        indexed: false
        uniqueId: false
        attributeAlias: 3771570d-dee5-42d5-86ac-1ea1fee270fb
        list: false
      - name: value
        externalId: value
        pushExternalId: value
        description: ""
        type: String
        indexed: true
        uniqueId: true
        attributeAlias: 56a0142d-170f-4f62-8573-0ad33439b65d
        list: false
    entityAlias: 4ffecf21-6cc3-4b03-8cb5-b395af82ebf8

Key Template Configuration Elements

Delivery Method: The template is configured with deliveryMethod: "entityPush" and pushType: "SCIM", indicating that it receives SCIM events through push delivery rather than polling for data.

Push External IDs: Each attribute includes a pushExternalId field that uses JSON Path expressions to extract values from incoming SCIM SETs. For example:

  • $.sub_id.id extracts the entity identifier from the subject claim
  • $.events.*.data.userName extracts the username from the event data
  • $.events.*.data.emails.* extracts email objects for child entities

Entity Structure: The template defines a parent-child relationship between Users and Emails, allowing complex SCIM attributes to be modeled as child entities in the SGNL graph.

System Configuration

After pasting the template, configure the system-level settings:

Display Name: Enter a descriptive name for your Entity Push System of Record

Description: Provide a description explaining the purpose and scope of this Entity Push integration

Authentication Configuration: Configure authentication credentials if your Entity Push implementation requires specific authentication beyond the push endpoint security

Event Endpoint Configuration

  1. Save Configuration: Click “Continue” to save your Entity Push System of Record configuration
  2. Obtain Push Endpoint: SGNL will provide a specific endpoint URL where event sources should deliver SCIM SETs. The endpoint follows the format:
    https://events.sgnlapis.cloud/events/scim/v1/{datasource_id}
    
  3. Generate Authentication Token: If using bearer token authentication, generate a new authentication token from the system settings
  4. Configure Signature Verification: If using JWT signature verification, configure the JWKS endpoint or well-known configuration URL for public key distribution

Testing Entity Push Integration

Creating Test SCIM SETs

Use JWT.io to create properly formatted SCIM SETs for testing:

  1. Navigate to JWT.io: Go to https://jwt.io/ and select “JWT Encoder”

  2. Configure Header: In the Header section, enter:

    {
      "alg": "HS256",
      "typ": "secevent+jwt"
    }
    
  3. Create Payload: In the Payload section, construct your SCIM SET:

    {
      "iss": "https://scim.example.com",
      "iat": 1743996545,
      "jti": "9999999999999",
      "aud": "sgnl.ai",
      "sub_id": {
        "format": "scim",
        "uri": "/Users/44f6142df96bd6ab61e7521d9",
        "externalId": "jdoe",
        "id": "100"
      },
      "events": {
        "urn:ietf:params:SCIM:event:prov:create:full": {
          "data": {
            "schemas": [
              "urn:ietf:params:scim:schemas:core:2.0:User"
            ],
            "emails": [
              {
                "type": "work",
                "value": "jdoe@example.com"
              }
            ],
            "userName": "jdoe@example.com",
            "enabled": false
          }
        }
      }
    }
    
  4. Configure Signing: In the “Verify Signature” section, enter a secret key:

    a-string-secret-at-least-256-bits-long
    
  5. Generate Token: The encoded JWT will appear on the right side of the interface

Testing Event Delivery

  1. Prepare Test Request: Use a tool like Postman or curl to send the SCIM SET to your Entity Push endpoint

  2. Configure Headers: Set the following headers:

    • Content-Type: application/secevent+jwt
    • Authorization: Bearer {your-token} (if using bearer authentication)
  3. Send Event: POST the encoded JWT to your Entity Push endpoint:

    curl -X POST \
      https://events.sgnlapis.cloud/events/scim/v1/{your-datasource-id} \
      -H 'Content-Type: application/secevent+jwt' \
      -H 'Authorization: Bearer {your-token}' \
      -d '{encoded-jwt-token}'
    
  4. Verify Response: A successful request should return a 202 Accepted response, indicating that SGNL has accepted the event for processing

Testing Different Event Types

Test various SCIM event operations to ensure comprehensive functionality:

Update Event: Change the event type to test entity updates:

"events": {
  "urn:ietf:params:SCIM:event:prov:put:full": {
    "data": {
      // Updated entity data
    }
  }
}

Delete Event: Test entity deletion:

"events": {
  "urn:ietf:params:SCIM:event:prov:delete": {}
}

Monitoring and Verification

Event Processing Logs

Entity Push provides comprehensive event logging through Logs → Event Streams in the SGNL console:

Event Receipt Monitoring: View real-time logs showing SCIM SETs being received from your event sources, including request headers, authentication status, and processing timestamps.

Event Acceptance Verification: Confirm that events are being accepted and stored successfully in the SGNL graph, with detailed success indicators and processing metadata.

Failed Event Analysis: Identify events that failed processing with detailed failure reasons, including authentication failures, signature verification problems, malformed SET structures, or attribute extraction issues.

Processing Details: Examine complete event processing context including raw SET payloads, extracted attributes, entity operations performed, and any related graph updates.

Authentication Auditing: Monitor authentication attempts and failures, providing visibility into potential security issues or configuration problems with event sources.

Using DataLens for Verification

Use DataLens to explore your Entity Push data:

  1. Query Recent Entities: Search for recently created or updated entities to verify that SCIM SETs are being processed correctly
  2. Validate Attribute Extraction: Confirm that attributes are being correctly extracted from SET payloads using the configured JSON Path expressions
  3. Verify Entity Relationships: Check that parent-child relationships (like Users and Emails) are being created properly
  4. Test Entity Operations: Confirm that create, update, and delete operations are reflected accurately in the graph

Event Processing Verification

  1. Send Test Events: Generate test SCIM SETs from your event sources to verify end-to-end event processing
  2. Monitor Graph Updates: Use SGNL’s interface to confirm that entities are being created, updated, or deleted as expected
  3. Verify Authentication: Confirm that events are properly authenticated using your configured bearer tokens or JWT signature verification
  4. Check Attribute Mapping: Verify that entity attributes are being correctly populated from the SET payloads according to your JSON Path configurations

Troubleshooting

Event Delivery Issues

Events Not Received: If SGNL is not receiving events, verify that event sources are configured with the correct SGNL endpoint URL and that network connectivity exists. Check firewall rules, proxy configurations, and DNS resolution between event sources and SGNL endpoints.

Authentication Failures: If events are being rejected due to authentication errors, verify that bearer tokens are valid and properly included in the Authorization header, or that JWT signature verification is configured correctly with accessible JWKS endpoints containing the appropriate public keys.

Event Format Errors: If events are being rejected due to format issues, verify that event sources are generating properly formatted SCIM SETs according to the SCIM Events specification. Use JWT.io to validate SET structure and claims.

JWT and Authentication Issues

Token Verification Problems: If JWT signature verification is failing, verify that JWKS endpoints are accessible from SGNL and contain the correct public keys. Check that key IDs in event signatures match keys available in the JWKS endpoint and that signing algorithms are supported.

Bearer Token Issues: If bearer token authentication is failing, verify that tokens are current, properly formatted, and included in the correct Authorization header format. Check token expiration and regenerate if necessary.

Signature Validation Errors: If signature validation fails for correctly formatted events, verify that event sources are using supported signing algorithms and that private keys used for signing correspond to public keys available through the configured key distribution mechanism.

Event Processing Challenges

Attribute Extraction Issues: If expected entity attributes are not being populated, verify that SET payloads contain the expected claims and that JSON Path expressions in pushExternalId fields correctly navigate the SET structure. Use the JSON Path guide for expression syntax.

Entity Relationship Problems: If parent-child entity relationships are not being created properly, verify that child entity pushExternalId expressions correctly extract array elements and that parent entity references are properly configured.

Missing or Duplicate Entities: If entities are not being created or are being duplicated, verify that the uniqueId attribute is properly configured and that sub_id values in SETs correctly identify entities. Check that event sources are generating consistent entity identifiers.

Rate Limiting and Performance

Event Rate Limits: Entity Push supports high-volume event processing but includes reasonable rate limits. If events are being throttled, review your event delivery patterns and consider implementing client-side rate limiting or batching strategies.

Processing Delays: While Entity Push is designed for real-time processing, complex entity structures or high event volumes may introduce processing delays. Monitor event logs for processing times and consider simplifying entity structures if delays are problematic.

Graph Update Performance: Large numbers of rapid entity updates may impact SGNL graph query performance. Consider the balance between real-time updates and system performance when designing your Entity Push implementation.

Integration with SGNL Policies

Once your Entity Push integration is successfully configured and receiving events, you can leverage real-time entity updates in dynamic SGNL policies that respond immediately to identity and access changes.

Real-Time Identity Updates: Use Entity Push events to create policies that immediately respond to identity changes such as user account modifications, group membership updates, or attribute changes that affect access decisions.

Event-Driven Access Control: Leverage the immediate nature of Entity Push to create policies that adjust access based on real-time events from identity providers, HR systems, or security tools, ensuring that access changes take effect immediately rather than waiting for scheduled synchronization.

Cross-System Coordination: Use Entity Push to coordinate access changes across multiple systems in real-time. When an event occurs in one system, Entity Push can immediately update the SGNL graph, triggering policy evaluations and actions in related systems.

Audit and Compliance: Leverage the comprehensive event logging and graph update tracking to create audit trails that demonstrate real-time access control responses and compliance with security policies requiring immediate enforcement.

Action Triggering: Use Entity Push events to trigger SGNL Actions that can propagate access changes to downstream systems, creating a comprehensive real-time access orchestration workflow.

For comprehensive guidance on creating policies with Entity Push data, refer to the SGNL Policy Management documentation. For advanced entity modeling and relationship configuration, review the Entities and Relationships guide.