By
At SevenPico we are helping businesses modernize their information systems to move at the speed of thought. Or maybe even better, how about at the speed of nobody-even-has-to-think about it? We call it “Push Analytics”, powered by Event-Driven Architecture.
A client recently asked for help with an intriguing problem: “Our quarterly reports show we have waste at the roll-up level. We know that it is possible to identify the waste in time to head off the waste. And we know our business intelligence tools can answer questions about waste, but even though the organization has been worried about waste for the last two quarters, nobody has yet actually asked the actionable question at the right granularity in the Business Intelligence system.”
“What we really want is a system that constantly asks the actionable question at the transaction level, and sends us a timely notification so we can steer the business.”
This is a job for Event-Driven Architecture!
Push Analytics applies a known analytical question to all transactional business events as they occur, giving real-time (seconds scale) notification and near-real-time (minutes scale) data availability for analytics reporting. On one side of the coin, you can immediately use the known question as a key performance indicator in dashboards and automations to steer business activity; on the flip side, you can use that signal to ask completely new questions informed by up-to-the-minute data. Push Analytics is a great entry point to start transitioning a legacy system to a modern serverless architecture with real-time insights and lower operational costs. A Push Analytics observation layer can deploy in complete isolation from legacy systems, with a small integration footprint that is easy for legacy maintainers to consume.
It is an excellent initial project to begin a transition to Event-Driven Architecture by immediately delivering new business value for the investment. In this article, I’ll walk you through how we do it.
Overview
The broad strokes of what you get in a Push Analytics project are:
- 🟡 A Way to Publish Events
- 🟢 A Way to Model Your Business Question
- A Way to Assemble Records from Events
- A Way to Visualize Record Lifecycle
- A Way to React to Record Lifecycle Events
- 🟢 A Way to Answer Your Business Question
- A Way to Calculate Signals from Record Lifecycle Events
- 🔴 A Way to Notify of Signals
- 🔵 A Way to Steer in Real-Time (seconds scale)
- A Way to See Records History
- A Way to See Signals History
- 🟣 A Way to Report in Near-Real-Time (minutes scale)
- ⚪️ A Way to Document the System Automatically
🟡 1 - A Way to Publish Events - Events API
We want a smooth transition from maintaining the legacy system to modernizing into this Push Analytics solution. To do that, we need it to be very very easy for the legacy maintainers to start sending Events into the new system. We achieve that ease of use with a REST API that can collect Events from any authorized software. Regardless of what part of your legacy system an Event comes from, if you can update code to send a REST API request, you can publish an Event to Push Analytics.
The REST API has a strongly typed schema for Event payloads. It validates all requests to publish an event. This keeps the Event-Driven Architecture protected from corruption because only design-compliant Events are allowed through the door. This makes things easy for legacy maintainers because their integration is obviously broken with error responses if they fat-finger anything or use the API wrong.
You may be thinking that the strongly typed schema will introduce a lot of friction to development time. To keep things moving smooth-and-fast, SevenPico implements the REST API generated from design-first authored OpenAPI Schema files. (We currently recommend Stoplight.io’s authoring tools) This approach means that about as quickly as your Technical Product Manager understands the shape of an event, the auto-validated REST endpoint to accept that event is deployed to production.
🟢 2 - A Way to Model the Business Question - Core Domain Service
We want to cleanly model the business question at hand in isolation so the business logic can be small and without tightly coupled dependency on any part of the legacy system. Our Core Domain service will depend on Events from the legacy system, but that’s the nice loose coupling we are looking for in modern systems.
Each Domain Service is an implementation of a Domain Model. In the initial Push Analytics project to start your transition, the Domain Model is the aforementioned “constantly asked actionable question at the transaction level”. Whatever your particular business question is, we can model it. Domain Models are made up of what we call the “Domain Primitives” necessary to express the business logic of the domain:
- Events - all the Events this Service subscribes to and publishes
- Entities & Value Objects - data that this Service keeps track of, usually originating from Events
- Records - represent a business process unit (often a transaction) as a composition of:
- State Machine - the possible states of a Record and logic for transitions between states
- Entities - data with identity so it can be queried directly
- Value Objects - data without identity which belongs to the Record
- Queries - API endpoints for querying data from the service
- Commands - actually, our Push Analytics Domain won’t use commands. These are API endpoints that create or update a Record by producing an Event to the Event Bus
The Domain Model is expressed in the Business Logic of the Service, which orchestrates how Events assemble a Record and transition its State. The Service’s dedicated database persists the Records, which will each contain their respective Entities and Value Objects which originated from the source Events. The model is made available to the outside world on the Service’s API, using Queries and Commands.
SevenPico will facilitate a number of activities to help you express your unique business processes and information in a Domain Model. We offer Event Storming, Domain Storytelling, and Domain Type Modeling workshops to help create a shared understanding before we begin building.
When it comes time for building, the Business Logic of the Service fall into three components:
- Domain Events Processor - calculate Record state
- Domain State Monitor - visualize Record lifecycle
- Domain State Change Processor - publish Record lifecycle Events to Event Bus
🟢 A Way to Assemble Records from Events - Domain Events Processor
The Domain Events Processor is the primary brain of the micro-service. It subscribes to relevant Events on the Event Bus, and as each one arrives, it processes the event to update the state of one or more Domain Records. It maintains Records in two DynamoDB tables. The State database holds the current State of a Record and its related Entities. The History database holds a time series of all Record States over time.
Some Events may very simply trigger the creation or update of a new Record. Other Events may have more complex relationships in the Domain. For example, several Records may share a relationship to the same Entity. When an Event arrives describing an update to that Entity, the Domain Events Processor must query all Records related to that Entity, and then update the Entity in every one of those Records. Corrective policies may be applied periodically to correct for drift. Event-Driven Choreography ensures the Service’s view of the world will be Eventually Consistent, and always available without coupled dependency.
To make the system more maintainable, the Domain Events Processor logic must handle Events arriving out-of-order and more than once. Those are technical discussions for another day, but very important.
Assembling Records from Events is what makes Push Analytics extremely powerful. By composing building blocks as they arrive from disparate sources, this approach allows for real-time querying of what otherwise would only be possible in Business Intelligence Analytics systems. This small investment in measuring what matters can deliver huge boosts in agility and efficiency.
🟢 A Way to Visualize Record Lifecycle - Domain State Monitor
Although we modeled the Records’ State Machine before implementing the Domain Events Processor, that clear picture may not be readily available to our future selves or other operators and maintainers on our team. Documentation can get lost or out of date. We want a way to be sure that everyone clearly understands the Record’s states and their relationships. What if the system self-documented that state machine? What if it documented the lifecycle not just in general, but with the state path for every single Record in the system?
The Domain State Monitor visualizes the entire state machine and the path taken by each Record, including other attributes along the way. It achieves this by leveraging AWS Step Functions to maintain a shadow of everything that is happening in the Domain Events Processor.
This tooling is outstanding at visualizing and tracing business process state machines that span multiple microservices or softwares, which can be quite challenging otherwise. By linking to execution visualizations from dashboards, operations will have understandable tracing at their fingertips.
🟢 A Way to React to Record Lifecycle Events - Domain State Change Processor
We not only want this Domain Service to know what is happening to Records. We also want the rest of the Event-Driven Architecture to be able to subscribe to Record Lifecycle Events. We need to publish them to the Event Bus.
The Domain State Change Processor evaluates every update to Records in the State Database. Every time there is a meaningful change, it publishes an Event representing that change to the Event Bus. Anything in the present or future can subscribe to those events to begin automating event-driven behavior. For example, the Signals Service will react to these events to evaluate if the latest transition is a meaningful Signal that should be Notified.
🟢 3 - A Way to Answer the Business Question - Signals Service
The whole point of this project is to answer a business question. Something like “which instances (Records) of pending business processes are likely to have waste that we can avoid if we take timely action?” Now is the time to answer this question. Every single time an event happens.
The Signals Service listens for Lifecycle Events for Records. These contain before and after values in their payload. As each one arrives, a Per-Domain Signal Generator calculates whether something interesting happened by comparing before and after against the Threshold business logic (state, value, or time). Events that breach the Threshold trigger Active Signals. Every Active Signal triggers a Notification to people.
The Service maintains the current State and History of Signals in respective DynamoDB tables and an AppSync GraphQL API so consumers like dashboards can display up-to-the-second visualizations of Signals. And it’s not just Domain X that has its signals here. This service can support an arbitrary number of different Signals originating from any number of Domain Events. All it takes is a new Per-Domain Signal Generator to add support for a business question in a new Domain in the future.
🔴 4 - A Way to Notify of Signals - Notification Generator
Whenever a Record in the system has an Active Signal, we want to Notify our people immediately, so they can take action to improve the business outcome.
Notification Generator applies a template to incoming Signal Events to publish a custom notification to AWS ChatBot, which forwards messages to chat channels in Slack or Teams. Optionally, Notification Generator can also publish notifications to email or text message.
🔵 5 - A Way to Steer in Real-Time - Real-Time Dashboards
When your people receive Notifications of Signals, they likely will want to go somewhere to see the consequences of their actions. TPush Analytics hosts an Operations Dashboard for all Signals where operations stakeholders can view all the Records in need of action and monitor Signals after action in a tight feedback loop. That seconds-scale feedback loop only applies to the known-question implemented by the Domain, however, and is only appropriate for tactical transactional use cases.
These Dashboards can be implemented in one of a variety of dashboard clients such as AWS CloudWatch Dashboard, Grafana, Retool, or a custom front-end. In all cases, the Dashboard is powered by the AppSync APIs we implemented in the Services.
Queries of the Domain API are commonly useful for rendering dashboard tables of the current state of many Records, usually showing several attributes at once. Signals API is great for heatmaps. Both APIs readily support line charts to display time-bucket history for line charts or histograms.
🟣 6 - A Way to Report in Near-Real-Time - Data Warehouse Analytics
Strategic people on your team will want to ask novel questions the data. For Analytics use cases, Push Analytics automatically replicates Records and Signals DynamoDB Tables for State and History into a Data Warehouse using Redshift Serverless database. These replications happen continuously at every update of the DynamoDB Tables, and propagate for query availability within minutes.
The Data Warehouse can back your choice of Business Intelligence client, such as Tableau.
⚪️ 7 - A Way to Document the System Automatically
Lastly, a key to the transition to Event-Based Architecture is the discoverability of the Events APIs. To that end, Push Analytics includes hosting of generated documentation sites that are always in sync with the implementation. These include:
- REST API documentation generated from OpenAPI Schema
- Events Schema documentation generated from EventBridge Schema Registry
- GraphQL API documentation generated from GraphQL Schemas
With these at your teams’ fingertips, legacy integrators will know exactly how to publish Events; product managers will know what Events are available for the next Push Analytics question; Service implementers will know how model the next Domain from Events; dashboard implementers will know exactly how to query the chart they need; and everybody will clearly see which Domains in the system are producing and consuming which Events.
Summary
At the end of the project you have two outstanding outcomes:
- The immediate business value of your first Push Analytics question with real-time awareness of actionable, steerable business efficiency opportunities.
- An serverless Event-Driven Architecture deployed in Infrastructure as Code, with repeatable highly-extensible patterns you can leverage as you add additional Domains/Services in future projects.
Additionally, your brand new Event-Driven Architecture is already checking the boxes for these important objectives:
- Adding Events to the system is easy
- Each Domain in the system is conceivable by both maintainers and operators
- Future “Push Analytics” business questions can rapidly and consistently build on Signals Service
- Operations stakeholders are notified immediately when they have an opportunity to make a corrective move for a win
- Operations stakeholders can view real-time dashboards to steer their behavior
- Analytics stakeholders can access data within minutes of its occurrence
- Documentation at your teams’ fingertips is always current because it is generated from the implementation
Want to learn more?
Contact us for a free consultation on how we can help you modernize your systems.
