Navigating the journey to serverless event-driven architecture

Thank you for coming to this session today um uh in person and here and also people watching on the stream in the other hotels and also even people in the future watching this online.

Um today, this talk is called navigating the journey of a servius adventure of an architecture. We have a jam packed session. So I'm just gonna get dive straight into this.

First of all, I have to introduce myself. My name is Dave Boyne. I'm a Developer Advocate here at AWS and I primarily focus on serve and avent driven architectures. And before joining AWS, I spent about 15 years working in enterprises and start ups and even took some time out to do open source, full time building avent driven tooling.

And today it's my job here to navigate you through this journey of an avenger of an architecture using services technology. And this is what we're going to be speaking about today.

As as I mentioned, we have a packed agenda. Um so first of all is where do we get started when we build a event driven solutions where you think you should get started and where you probably should start next we're in the middle of the sec of this talk. We're gonna be talking about integration patterns, talking about messaging patterns and talk about common pitfalls that you'll come across and how do you avoid them? And then finally talking about operational maintenance.

There's three years, five years, 10 years into the future, something we often overlooked as we build a event driven solutions, the resources to the whole deck, the deck itself, everything i speak about you can find on that qr code, this qr code will be shown again at the end of the d uh presentation. So don't worry if you don't get it now.

And I just ask one thing of you all watching uh and also watching on the stream is just thinking about your use cases, your business and requirements and thinking about how can a venture of an architectures or how can the topics we speak about today help you?

So I want to start off by saying this is that the adoption of a venture of an architecture is growing, right? And you see this everywhere, you see it. If you're in the community that the word avenger of an architecture keeps coming up. And it's interesting because a venture of an architecture isn't anything new, right? You can go back, you can find white papers actually going back 40 years um around event driven solutions.

So why is it that we have this adoption of ed a growing? It's because a venture of architectures are becoming more accessible to us. Ok. And here are some of the aws services that are making this possible.

So hands up people are using aws lambda in the room. Of course, I'd say about three quarters of the room. So aws la lambda, the server compute being able to do compute based on different run times all triggered by events.

Now, if we match this to api gateway as an example, that event could be a http event right next door to lambda, we have step functions, any step functions users here. Cool. So I'd say about a third of the room here. So step functions being able to orchestrate business logic, being able to do orchestration, being able to visually see what is going on again, triggered by events.

Next door to that, we have amazon eventbridge, eventbridge users in here. Yeah, of course. So I'd say half the room. So amazon eventbridge is fully managed service event bus with routing capabilities, transform transformation, going to different ads targets again with events.

And then dynamo db, as we listen to dynamo db streams, we can listen to inserts updates and deletes and actually react to these things.

Now, when we talk about a venture of architectures, the question is, well, what are a venture in architectures if we started at the top? What would we see? And when you start to dive deeper into this, you'll come across three main words and terminology that comes up, you'll come across producers, you'll come across consumers and maybe some kind of channel and some kind of broker in the middle here.

Now at a high level, yes, this is event driven architectures, this is the core of event architectures. But what happens when we start to build with a event driven solutions. And in fact, the adoption of a event driven architecture is actually a journey that you will go on. You don't one day wake up and say i have a producer and i have a consumer and now i'm event driven. In fact, it's multiple patterns, it's multiple things. And again, it's this journey that we tend to go on.

Now, if we were to visualize this journey, i'm a very visual person. Uh this is what i would think it would look like at the top here, we have these peaks, these promises that then architecture promises to give us decoupled architecture, this business agility that we can get. And also this evolutionary architecture that we are promised and it is there, we can get it. It is there for the taking. But often you'll find common pitfalls as you go along building this event architecture, you'll think about unclear boundaries and leaking information, lack of standards, lack of governance, lack of discoverability, understanding, async versus sync behaviors. So that's what we're going to be speaking about in this talk, going through this journey.

And the question is when we start building event architectures is where do we actually start? Where should we start? And equally why do we even care about events in the first place?

Now, there's a good analogy here. I like to tell is events, tell a story. Ok. And tables tend to tell us the state right now example would be um where did i, when did i wake up this morning? So events, when did i wake up? What did i have for breakfast? Have i done a talk before this? How did i get here? Did i get a taxi or did i walk here? All this information just about me about today. These are events now equally tables on the other side, we're talking about state.

So if you were to query me, which sounds super weird. Do not do that. But if you were to query me, i'm right here talking to people about event architectures and people on the stream. So again, practical example, let's talk about pizzas, uh pineapple on pizza, any pineapple on pizza fans? Yes, look at this. I found my family.

Um so again, so pineapp ordering app. So i open up, open up the app and i get the state of all the pizzas, right? This is the state. This is information tables. Now, as i'm ordering the app, i'm adding pineapple on, i'm removing, i'm i'm checking out, i'm not checking out. All this information are events.

Every organization i've ever worked in are interested in events. And what you'll find out is that modern applications tend to have both events and state. Again, when we look into the benefits of event architectures, we see this, this loosely coupled systems, this agility extensibility that you're promised before joining a w ss i was building, i was in a start up building event driven solutions on aws, in fact, and it was an agility that i've never seen over 15 years ever in the in the uh team that i was leading.

So the big question is many people have is where should we start when we build event driven solutions? And uh i've distilled it into two main places here. Number one is we just implement, we can take a shortcut across this mountain, right? We can start implementing number two is we understand behavior and i want us to focus on this first, the implementation stage.

What happens if we start to implement? So here we have value over time as you build your event architectures we start to implement. And what you'll notice typically what happens is we get this bump in initial value, right? We maybe we're using eventbridge as an example of fully managed service and we have that bump in initial value that's producer in this consumer.

Now as time goes on, we add more producers and we have more consumers. But what you'll notice as time goes on in your event architectures is that value tends to plateau over time now, why is that?

Well, let's map the complexity over time and see what happens. So what you'll notice with event architectures is that complexity can grow over time. We have the initial producers that initial consumers and then assumptions can be made. We have a lack of standards, lack of governance, lack of discoverability as an example, even unclear domains.

What goes inside our events? Let's have a look exactly how this looks. So here's an architecture that i'll be referencing throughout this talk. Simple one, we have an order service and we have downstream consumers, we have some kind of broker some kind of event bus in the middle.

We as the order service team have been tasked to raise an event. The order created event, an order has been created. So what we decide to do as a team is lean into aws here to help us. So we we're using uh a eventbridge here is an example for those who not uh recognize the payload.

Let's step into that. So we have the detail type, the order has been created, right? This is a mutable fact mutable information. The order has been created. That's fine. Next, we have the source. Now we're using the source as the domain, we are the order service. So what we're gonna do is have uh my app to orders here as an example.

Now down here is the most important part of your envelope of your message is the detail object in this example, right? This is entirely up to us. Now, what we decide to do is add the order id and the user id inside this event. Fine. That's great. So the the downstream consumers are happy, they got an event, they got some information, everyone's happy.

And three months down the line, we are tasked to do the order canceled event. So we've done this before. So let's do it again. We create the order canceled event and what we do in here we had a reason why was the order canceled because it was no longer needed.

Now, as a consumer of this event, this may or not be interesting. It could be interesting cos we could send a text message or an email to say that the order has been canceled, it's no longer needed. Fine. The order has been canceled. tt w does anyone know what type tw means? Put your hands up when you cancel an order, no one in the room? Right? And i don't know what it means. I kind of made it up. But what i mean by that is it's so easy for us to add implementation details of our domains inside of our events as a consumer of this event that type tw means nothing and is useless.

So we move on, we have the user object, this is fine. But this time actually, we've got an object previously, we had the user id in there, but we have the id in here. That might be fine. But we have first name and last name. Is this sensitive information. Are we regulated? Can we share this information? Don't know.

We continue, consumers are happy. We're moving on with our life. We're building this event architecture. We have all updated event that comes along and we continue this journey. In fact, more domains arise and we have more events and more more uh events coming into our event bus and equally down here on the blue blue team, they're actually sending commands with events. Please send an invoice email. So now we're getting into this weird territory of using events as commands. And typically what tends to happen is you're doing this system is unclear, domain language is arising unclear languages.

And i'll tell you a story. I used to work in the life insurance start up actually building on a w bs leading a team, right? And we, we had this idea that a policy, the customer has a policy, a life insurance policy. So we modeled this policy in our development team as something a customer had, right? We'd raise events, so we'd store all this information.

Now when we integrate with the insurer, the insurer turns around and says no, no, no, no. The customer, we own the policy, the user so happens to be part of that. So we have some weird terminologies going on and then the underwriter came along and the person paying the bills actually says hang on no, no, no policy is something completely different.

So we end up in this weird and wonderful world where in fact, we end up with this, which is like this big ball of mud. This is terminology from domain driven design here, but we have unclear domains, unclear structure, unclear intent and boundaries and information leaking in our events.

This quote is from alberto, he's the author of events storming and he says it's the developer's misunderstanding, not the domain experts, knowledge that gets released into production. Ok? I want to focus on that misunderstanding that understanding part because if we took a step back, right, and not do implementation first, what happens if we actually understand the domain first understand the behavior first or a rev event driven solution?

So what do we mean by this? While we understanding the behavior of the system by doing that, we're collaborating with domain experts, we collaborate, collaborating with product owners, collaborating with even customers, maybe cto s or ceo s or whoever else that might be by collaborating, we can actually identify events within our organization. Ok? And when we identify events, we can identify boundaries that that domain information. And we can also identify language, common language that is used in our organization.

So one way to do this is to use event storming and has anyone used event storming before? Ok. It's about 10, 5 people. So events to is a great method a way to actually start with a new system or an existing system to understand the behavior of the system. Ok? So you get posted notes, you get domain experts in the room and we start to map events over a timeline. This happens, then this happens and this happens.

Once we identify events, we can identify boundaries and aggregates, right? It's a very good way to visually represent, excuse me, um a system. So here's an example of what i mean, this is a car parking system uh that i did last year. And for example, you drive up in the car and you press a button and you get a ticket. Now we can model this using event storming. As you see up here. We have the ticket machine, that's the system we're representing here in pink and we have a command. Please create me a ticket commands can be rejected and we're gonna be talking about that in a minute

But then equally as the system does stuff, we raise events, we group these in boundaries and we have a time line across the bottom. Now, the great thing about events stormy is that i can guarantee no one's probably seen this slide in this room or, or watching on the stream. But i can guarantee a lot of people seeing this can understand the flow of the system, understand where the events are and we can discuss the events and the payloads in this particular way. So event storming is a great way to actually visually see what's happening, understanding that domain. Because once you understand this, you can then go on to implementing.

Ok. And when we start to implement, what do we mean and what should we know what should we think about? And the important part of of event architecture is really is understanding different types of messages. Ok. So the first one is command, we have commands queries and also events, your architecture will have probably all of these. So a command, for example, can represent an intent to perform an action. A command can be rejected. Ok, please order that pizza. No, we have no pizzas left. This is a command, it's rejected. Once we order that pizza as an example, uh we can, the order service here can raise an event to notify everyone in the system. Something has happened that the order has been created.

Now a top tip, i would say if you're raising events also is to raise events without consumers because there's the whole point about that event driven architectures is we want that extensibility. So when you think you should raise a domain or business event, it it might be a good time to do that. And equally, you also have queries querying for information. Get me the pizzas, for example on the app.

Now as you're going through your journey, a service event architecture journey, you're gonna be coming across various different messaging patterns ok, you're gonna be coming across point to point communication, publish, subscribe, you'll be coming across event streaming equally. You'll be probably looking at how integration patterns can help you avoid these common mistakes and nuances as you navigate. And also there will be coupling traps that you'll fall into. I can probably almost guarantee that and equally to how domain driven design itself can actually help us build our event solutions. And this is what we're gonna be talking about.

Now, let's dive in to some of these in a bit deeper. So the first thing is this, we are tasked, we are the order service and we are tasked to integrate with this new fulfillment service. We get an api request. That's great and we can call this thing directly fine. That's one way of doing it. But when we start to think about these architectural tradeoffs, start to think about does the order service know too much? Maybe, maybe not all about processing at the same rate. What happens if the order service is taking on a sale? For example, black friday is an example. We got all this traffic and we're bombarding the uh fulfillment service with this, the the these uh these events, this this information. So what happens about failures? Well, if the fulfillment service fails, the order service fails and then the user that made that request might also get a bad experience.

So again, what do we do? What do we look at. To help here, we can introduce some kind of point to point channels. We can introduce cues as an example. So what we can do here is put some information on a queue, we can put these messages on a queue. Now, why would we want to consider this is because the order service no longer needs to know about the downstream consumer about that fulfillment service. We are decoupled and then equally we're processing uh at a desired rate because the fulfillment service here can pull messages off of the queue. And that is the key part, also isolated failures.

Ok? So if the fulfillment service fails, we can set up things like dead letter queues to actually catch the messages rather than letting them drop. And in this way, the user probably won't get that bad experience because these, these errors are handled in the back end. So again, what we're talking about here is point to point communication and messaging. So this decreases the coupling. But also that consumers pull messages from the queue to process them. As consumers pull these messages to process them, we can also scale the consumers. We can have multiple consumers pulling messages off and a visualization. I like to think about this is when i go christmas shopping, right?

He is, i'm gonna leave re invent, go home back to the uk and go christmas shopping for my kids. And as i as i'm pulling stuff into the queue into the uh into the basket. I see a queue, right? A shopping queue and brits is any other brits in here? Yes, we love a queue, right? We love queuing. We see people lining up, we join them. So as i'm standing there queuing up, um always, always there's like one or two cashiers and there's about half an hour, wait time in this queue. So why am i talking about? This is because if you're lucky enough, more cashiers will come along and process that queue faster. That's exactly what we're talking about. We're talking about queuing here and multiple receivers pulling information off.

Now, if we link this back to a service event architectures and link this back to your journey, you'll probably be using something like amazon sqs, which is one of the first services we actually released and it's a fully managed uh message queue. Is anyone using sqs here? Cool a lot of people. So scales to almost infinitely and supports things like dead letter queues. So we can um and also fifo queues. So we can process things in order if that's a requirement for you. So point to point messaging will be part of this journey.

So what's next? What other things will you lean into this time? We are the order service team and we have been tasked to notify downstream consumers. Something has happened. Now again, we can do this, we can integrate, integrate with them and just notify them. Now again, the tradeoffs that we're thinking here does the order service know too much? Probably. Yes. What about failures? If that fulfillment service failed? That middle one and the top one succeeded? What's the state of the architecture? What's that? What's the state of the data now? And equally down here, we've got duplication of effort as we integrate things over time.

Now, of course, what we can do here is lean into pub sub publish, subscribe patterns, actually notifying downstream consumers that something has happened on the left, the producer has no idea who is consuming, they just release and push a an event on to the event broker. Now here what we're talking about, of course, as i mentioned is pub sub. So it's a push model and with many brokers, we can actually do things like routing or enriching as an example. So as you can see here, half the receivers might be interested in blue t-shirts as an example. And half may be interested in purple as another example. So publish, subscribe is another pattern that you probably use. And to help with that as we lean into serverless architecture is to help with. this was amazon eventbridge is a fantastic way of doing this lowering that barrier to a event driven architectures. We we have over 28 different aws targets that you can reach and the fastest, one of the fastest growing features of eventbridge actually is api destinations.

So what we can also do is reach out to third parties with our events and notify them of things happening with api destinations. And equally as you go on into this journey, you'll come across events streaming. So this is thinking about unbounded data, thinking about click stream information as an example or likes on a video streaming application as an example by itself, this is just loads of events, loads of information coming in, not that valuable could be not that valuable to our organization.

So what do we, what can we do here as an example? So streaming use cases might look like something like this. This is kafka data streaming platform, right? Open source apache kafka. So we get this click stream information and we get consumer this consumer aggregates this information. And in downstream, we create a new topic, this topic so happens to be a domain event something we are interested in a discrete event. What we can do here actually is use uh um eventbridge um connector here which was released this year to connect this directly into aws, connect this directly into our event bus where we can go out. So this is kind of both of be bo uh best of both worlds where we have streaming and we also have discrete events. Equally.

Talking about event streaming is we have this which is change data capture, right? As you go on your along your journey, you'll come across these listening to changes in a database super powerful deletes, inserts, updates, dynamo db streams, handles this and can give you this information. So you'll probably lean into some of these tools and some of these messaging patterns as you go along your journey. So we have point to point we have pub sub we have streaming and event storming is a way to kind of visualize these events.

Now, as you go along this architecture, there are common questions and common pitfalls you will probably fall into. And let's have a look at some of these. Now, the first one is this very important one. What goes inside our events, right on the left here, we have uh we are raising the order placed event and the consumer comes along and says, thank you very much. This information is a bit sparse but we, we'll get what we want, this leads and can lead to this, which called the, i like to think of it as an accidental coupling with callbacks.

Ok. I remember when i was working for this uh insurance company, i remember coming in one day with uh with my tea in my hands because yeah, i love tea and uh coming in one day into the office and the engineer comes running up to me and says, dave, dave, we've gone viral, right? And this was it fantastic. Cos this is a start up that i was working at dave. We've gone viral. You should see our graphs. Ok. So as i'm walking through the office, i see the graphs. I see these ticks going up in the top, right. And i'm feeling good about myself and as i kind of sit with him when we're diving deeper into everything else. In fact, we haven't gone viral. All that's happened is we had more consumers of our events. Why has this happened? Because the consumers come along and say, thank you very much. I'm gonna now go and fetch the information i want. Where does it get that from? Maybe the producer, maybe somewhere else. It's an interesting pattern because by design, we are decoupled because we're using events. But the schema of the event itself has led to this coupling.

So what options do we have here? Well, we can use one of the enterprise integration patterns called the content en richer pattern. So this allows us to do is have the sender still use sparse events, sparse schemers. Now you might want to do this because the more, the more things you have in your schema, the more uh things you've got to manage in version as an example. So you can still keep that uh small. And the consumer over here is saying, please give me more information.

So what can we do? We can introduce something in the middle called the enrich. So what the enriched does is get this information enrich it and then pass that on to the downstream consumer. And let's have a look at how we may do this in a surplus example.

So is anyone using eventbridge pipes? One or two people? I think. So, eventbridge pipes, we spoke about point to point communication. Earlier, eventbridge pipes is a point to point um service. The example being here on the left, we have the source and on the right, we have the target. Very popular patterns would be things like dynamo db streams into directly into eventbridge or step functions. Other popular patterns could be kinesis streams, streams of information into other streams. Again, managed for us with pipes.

Now there are two cool features about pipes which i absolutely love. Number one of them is filtering. So what we can do is filter information. We are only interested in this particular information in this stream or dynamo db stream or sqsq, we can filter it and then get that on to downstream consumers. We only pay for the ones that are filtered and not the others. And then equally here we can enrich and transform.

Ok. So we can enrich using lambda, we can enrich using express workflows with step functions or api gateways. We can even enrich using api destinations, going out to third parties and enriching information before it reaches downstream consumers. So a great way here, another way of just managing messages between producer and consumer and enriching them.

So the next question is this sensitive data inside of events. So earlier, we had the first name and last name inside our event, maybe we're regulated and we can't do this. So here we're placing the order placed event and the consumer turns around and says this information is sensitive.

So what can we do about this? Well, we could do something like encryption patterns, right? In fact, we can get the whole object, this whole detail object which is remember, this is entirely up to us. We can encrypt the whole thing and give that to downstream consumers

This is a valid pattern. This could work. But the thing is about doing this is we lose the ability to do eventbridge filtering as an example, remember going from blue to purple. So what can we do here?

Well, we can choose which properties we want to encrypt. So as you can see here, the customer's name, address and credit card are all encrypted. And we leave the rest unencrypted. This allows us to get the best of both worlds.

So let's have a look at an example, a service example of how we could do this. So here on the left, we have a producer, we have a shared key. We're using AWS KMS here to do the encryption. So we have a key and then what we decide to do as a producer is encrypt some properties of our event.

Now we give this to EventBridge and this encrypted data is encrypted at rest. We give this to EventBridge and we create a rule on the bus to a downstream consumer. The downstream consumer gets the information that they need and they use the shared key to decrypt that information.

Now as downstream consumers come and go, because that's the whole point of event architectures, they will come and go. You can only decrypt it if you have the shared key. Of course, in this example, the producer may own that key and may choose who can decrypt that.

So again, some tradeoffs to consider here, some things to think about. But there are ways in which we can also encrypt messages as you go on your journey of your event architectures.

The next problem that you may face is just processing large messages in your event architecture - PDF documents or huge JSON files that exceed service message limits (e.g. EventBridge having 256 kilobytes). So again, how can we get around this? What patterns are out there to help us?

And one of the patterns you'll come across is this, which is the claim check pattern. This is another enterprise integration pattern. So what happens here - an example, we have Lambda as the producer doing some compute and then we decide to raise an event. This time before it raises the event we store, we store information inside of S3, we store that payload inside of S3 and we have that key back to the bucket, that reference and we can push this onto the bus.

Now, the consumer, in this example it would be Step Functions, before it does anything goes to get that information from S3. Examples here could be a pre-signed URL from S3 with a TTL of two hours or two days if you wanted, or maybe just the path directly to the folder or file in the S3 bucket. But again, these are things and tradeoffs to consider because now the consumer needs access to the shared bucket. We need to create a bucket to do this. So again, things to think about. But these are patterns that can help you along your journey.

The next thing I want to talk about actually is one thing I'm super excited about, which is integration time between systems. I can't tell you the amount of times I've integrated with APIs with different services over the years, integrating A with B, using different APIs and credentials and managing all this myself.

But what happens if we actually lean into streaming, lean into events to handle this communication for us? This is exactly what we can start to do. We can actually have our organization, our AWS account, as an example - we can have a broker which other organizations can actually use and put messages into.

As you can see here, we've got Organization One on the left producing messages that Organization Two can consume. And again, back and forth using asynchronous processing.

Here is a quote from James Urquhart, he's the author of a book called Flow Architectures and it's a book I recommend, it talks about the future of events streaming between cross-organizational structures. And I watched a podcast of his recently and he says this:

"Lowering the cost of integration creates an explosion of new integrations and experimentation."

And I thought James has hit the nail on the head here with event-driven architectures. For me, this is what event-driven architectures is all about - it's about lowering that cost of integration so we can explore innovation within our organization.

Okay, and I want to show you an example of what I mean. Has anyone seen the Serverless Video app around re:Invent this week? A few people. So Serverless Video is an application that I spent the last couple of months building with the Serverless DA team here at AWS. It's a video streaming application built entirely on serverless architectures and it's open source. You can dive into this yourself if you're interested.

What this allows you to do is actually broadcast from your phone to anyone watching. People watching can click on emojis and we process that back to the users. Once they've finished the broadcast, we do MP4 processing using Step Functions, EventBridge, Rekognition, using Bedrock and all sorts of things.

And this is what that architecture would look like and does look like. So here on the left, we use Amplify to host the front end application and then we have APIs into the application.

Now we spoke earlier about commands and queries. So we have commands coming in into the architecture and queries - please give me all the videos that have been broadcasted at re:Invent, please start a new broadcast - these are commands of the system.

We have an event bus that we're using, we have microservices, we're using EventBridge to communicate between these services, having that decoupled architecture. We're actually listening to first party events - so AWS services themselves release events into your default event bus, super powerful.

So an example would be we used Amazon IVS, which is the same broadcasting technology that Twitch uses. So we use that and IVS are now publishing events into our event bus that we can consume - when did the broadcast start, when did it end, is it healthy?

Now, the reason why I want to talk about this is the plugins, the extensions that we built. We wanted to go to other Amazonians at AWS and say we're launching this thing at re:Invent, we want you to build plugins - you have an event and you have an MP4 file, what can you do? Document in the event within two days.

Our first plugin came through using Bedrock as an example to generate titles, to generate descriptions and tags - we get the transcription of the video and we chuck it to the model to create those titles and tags.

Other examples were next plugins coming - doing like moderation on the video using Rekognition on that MP4 file and then telling us if the video is valid or not. And we had leaderboard plugins and rollup plugins and all sorts of things.

This is all again open source, you can dive into this. But what I'm trying to say here is that we lowered the barrier to entry with using events, using documentation and schemas. And we had this innovation within this app and this is where event-driven architectures can help.

Here's another example where you see bidirectional events between organizations that we can do using EventBridge and using Step Functions. An example - one - so this week we launched Stripe integrations directly into EventBridge. EventBridge supports over 40 different SaaS integrations that you can connect to.

What this means is organizations can fire events directly into your event bus - there is no code there we have to manage, we can listen to these events, when the events come in we can create rules and targets.

Here is an example - we have a Step Functions workflow - let's talk an example - when a Stripe event comes in we want to award customers if they have purchased 10 products over the last year. We can orchestrate this workflow easily within Step Functions.

Now how do we go back out to Stripe? Well, interestingly, Step Functions released a new feature this week which is HTTP APIs integrations directly with Step Functions. So what we can actually do is talk directly to Stripe again using Step Functions.

So what we've achieved here is events coming in from the left and that also going out to the right - again that cross-organizational sharing of information.

So I'm super, super passionate about this because I think as we go into event architectures in the future there's going to be much more of this stuff happening about streams of information and sharing events and connecting to streams and events through organizational structures.

The next thing you'll see here as you build your event architectures, you're going on your journey, you have loads of events, loads of consumers and you have this - we mentioned a bit of this earlier about coupling through contracts.

This is often overlooked. We are raising the order placed event - thank you, we will consume it as it is. By doing this, by consuming a message or event as it is, you are a conformist by default. What do I mean by this?

So an example - this is bounded context mapping - so now we're leading into domain driven design and understanding how practices there can help us build event driven solutions.

So a conformist is something where we have an event, we consume an event as it is. An example of this would be the order service - the order has been placed - we are the fulfillment service - thank you very much, we have all this information, we consume it.

But in fact we have this, this model, this order model just all over the codebase because we consume it as it is. The fulfillment service - this may or may not be a problem. But an example would be what happens if the order service decides to change the schema, change the payload - then this fulfillment service would just break.

So how do we, what other patterns are there out there to help us mitigate against that risk? What we could do is something like anti-corruption layers - we can say, well, thank you for the event, we are the fulfillment service, the first thing we're going to do is change it into a model, into a language that we understand as a service. We have that protection around our service - anti-corruption layers.

A different way of handling this, equally the opposite - so we can push back onto the producer and say no, no, we agreed a contract, we agreed a public contract in this organization - you are going to do the mapping and we, we can fully understand that. Different patterns depending on different circumstances.

Now the interesting thing here is with these bounded context mappings is actually understanding the level of communication that is required for each one. An example here would be the conformist requires a very high level of communication because if your producer is changing contracts and they may break schemas, they've got to talk to the consumers - there is this communication that is constantly required.

Now again this might make sense. And we're going to speak about this in a second. But equally on the other side of that, the least communication would be open host service - we have that initial communication, we agreed on a standard - but now once the standard has been established, it's less communication that is required.

So again, thinking about these - and I highly recommend there are loads more by the way - this is all open source - highly recommend having a look at the context map cheat sheet - just being aware of how am I consuming my events in my event architectures, how am I going to be coupled by this contract and how is that going to affect our service?

And here's Vlad, he's the author of Learning Domain Driven Design, a fantastic book if you're interested in that. And he's writing a new book at the moment called Balancing Coupling and Cohesion in Software Design. And I watched one of his talks this year in Berlin. And I remember seeing this, which he says:

"The cost of change is proportional to the distance between components."

And this is super important when we're thinking about - and we bring this back to event driven architectures - thinking about the distance of components.

So we're talking about here is private and public information. Now I'll give you an example - we are the order service, everyone in this room is the order service, it's very successful, lots of people, we decide to have an AWS account for this service, we're using EventBridge as an example to communicate between maybe little microservices inside of this service that we have now.

Because we are the order service we can raise private domain events using this technology. What do I mean by this? Well if I add "Type:W" on the event that we discussed earlier, this may not be a problem because everyone in this service understands the implementation details of that service - private information.

This is completely different to public events, events we use to communicate outside the boundaries. Examples - order service talking to payment domains here - that "Type:W" no longer makes sense, that implementation detail no longer makes sense. We're using the same technology under the hood, we may be using SQS, EventBridge as an example.

It's all around contracts, it's all about domains, it's all about boundaries of information.

And again, understanding that that coupling understanding that distance again can really help you as you build event architectures as you go on that journey. Again, just thinking about these nuances and the trade offs that you will be making.

And here I just want to finish this section by saying that event architectures, you know we spoke earlier about consumers and producers and channels, what does it really mean? And to me, it means that event architecture really is a collection of patterns. Ok? And the more patterns that you you, you explore more, more it will help you as you go along your journey.

And in fact, domain driven design can really complement your event architectures. So if you're not looking into domain driven design, I recommend having a look and understanding how that can help you as you build your service ed a apps.

And finally, the last section of this talk, I want to talk about this, which is operational operation and maintenance. So think about. So we've got, we've got queues, we've got uh sqs, we've got eventbridge using events storing, we've got patterns, we've got contracts, we've got all these things going on. But how do we make sure we have this excess operational maintenance in three years, time, five years, time, 10 years time.

So back to this again. So we have, as we build a vendor of in architectures, we have this high agility, low complexity that we're promised. In fact, when you start out, you will probably experience this. And as time goes on, if you're really unlucky, you might end up down in number three where you have high complexity and that low agility as time goes on. And I started to ask myself, why is this? Why does this happen within the community as we build event driven solutions. And there's this terminology that keeps coming up over and over again, which is it hands up if you've ever seen this before or a few people that producers should not know about consumers. Ok. You were told this when you build a venture of in architectures that producers should not know about consumers. We're decoupled by nature.

Here's our example of adventure of an architecture app again, back to this. As time goes on, we add more consumers. This is the whole point. We've got this evolutionary architecture. Consumers are coming, consumers are going and as time goes on, in fact, we have more aws accounts. Ok? We have more producers, more event buses, more consumers and this thing grows over time. This is the whole essence of event architectures as it allows us to do this.

Now as you do this common questions will start to come up. Example one here, what events do we have? So new, new, new people join the company. People have left, people have done this before. What events do we have? And if you can find these events, the next question is, well, what's the payload of the event? Ok. I'm tasked to uh do a new email service as an example. I know you're creating an event but where can i find it? What does it look like? Equally, i am tasked to change an event but who's consuming this? Am i about to break an entire system. I don't know.

It's interesting if we take a step back here that producers, yes, producers should not know about consumers. That is true. Uh technical level producers are decoupled away from the, from the uh consumers. But that does not mean we can skip governance standards, discoverability. Ok? And if we talk about this, there's three sections of this, this, of this maintenance. Really one is standards and discoverability and then communication. So let's dive into more of these in detail.

So first of all, we have standards. So thinking about your events, you're on your journey, you've got 50 events, 100 events, a thousands of events, every event is different. Yes, that's fine. But the question is what standards could we put inside our events? Ok. Examples being what about correlation id between all your events in your organization might be useful all about a custom time stamp in your events. And i recommend doing this with eventbridge. By the way, cos eventbridge owned the time stamp in that payload. So custom timestamps or maybe you could put the domain of, of uh of the event in there or maybe some documentation url s whatever it is, what is the st standard of your payloads uh can really be useful.

An example could be the orders domain. We know that by using the metadata inside the event and what we can do is create an eventbridge rule one day and say actually i'm interested in listening or seeing some of these events within this domain. Now, i can do that because we start to have standards.

The next thing here too is about thinking about event, first thinking. Ok. So i'll give you an example. When we, when we build api s within systems within our architectures, we treat api s, we nurture them, we love them. We've been building api s as a community for many, many years, we have specifications for api s. Uh we, we discuss like rest and we discuss verbs and versioning and all sorts of things. Introducing api s in companies actually can be quite difficult when we introduce events. It's a different story, right? And that story tends to be we have requirements and we design it and then we implement it now event, first thinking what we really need to start doing is more of this is yes, we can identify, yes, we can define and implement, but you've gotta make sure you go back because your events will evolve, your business evolves, your architecture evolves, your schemers will evolve and you have to make sure they're kind of still relevant. And how do you do that? And i've got a talk here. There's a whole section on this. I got a talk here if you're interested uh uh a 40 minute section to dive deeper into that whole topic. But then first thinking treating events as first class citizens is, is really important.

The next one we have here is discoverability. Ok. How do we find events? What's in the structure, what's in the payloads? So here we have schema registry. So i'm a super super geek, by the way, when it comes to schemers, i think they're super important, often overlooked. And eventbridge actually have an awesome feature which is called the schema discovery. So if you're firing events into eventbridge, what you can do is turn on schema discovery on your event bus and eventbridge will manage and version the schemers for you. What it'll do in fact is give you jason draft four versions or open uh open api um versions of your events. And what you can do then is download code bindings. Uh you can download java or.net or typescript code bindings that you can use within your code again to match these events, an awesome feature of the service that might be helpful.

Next thing i wanna shout out here is ac nkp. I has anyone heard of this? Awesome. Yeah, so i'd say about 10 people here. So i used to work on this full time actually before joining aws, i was part of the team here. Um ac nkp i is a specification open source specification that allows you to document your advent driven architectures. Ok. Documentation of advent driven architectures is really um it, there's a huge gap at the moment around around this area. What ay nkp i allows you to do is actually document channels, document uh producers and consumers, it's not tied to any technology. It has many different bindings. C fa nssqs as an example. So you can use ayn kp i to actually do this.

Now, the great thing about a and kp i specifications in general is once you have a standard, you start to build tools and toing around that. So they have hundreds of different tools. And i recommend if you, if you're looking at ways of discovering and documenting, i really recommend checking this project out.

And then finally, on this section, i just wanted to shout out um probably a shameless plug here. Uh but event catalog, has anyone heard of event catalog? Also a few people. So this is a project that i created a couple of years ago uh because i saw this huge gap in discoverability around event driven architectures. And the only reason i'm sharing this with you today is because it's actually uh growing in growth and really helping people document their event architectures, you can use this gives you a static website that you can run anywhere and it allows you to visualize and 3d, visualize your producers, your consumers and all it is powered by mark down and, and md x components. So again, just another way to get people in your organization access to events, consumers producers. So again, if you're interested in that, it has plug in support for a and kp i and eventbridge, and again, you could write your own plugins there. So again, just something there that might be able to help you out.

And then finally, here we have communication, ok. It's actually embracing the ubiquitous language, embracing this language that we have within our domains. As, as you can see here, we have the order domain and the invoice domain, both of them have an orders model within them. But this orders model would be completely different between domains and that's ok. So it's ok to embrace that different languages that we can use within domains that ubiquitous language and understanding that can really help you again, build your event architectures, understand your boundaries, understand the language in which you use within your domains and again, mapping that back to events and how bounded context mappings here can help.

And with all of that, hopefully, this helps you stay away from that number three as time goes on three years or five years. And just keep in that number one, that event architectures can give you and provide you again. I've seen this firsthand, i've seen many customers have high agility and that low complexity as they build a ven driven solutions. But it's really up to you to manage these nuances and things as time goes on.

So with that, i just want to summarize this talk we spoke about at the start that advent driven architecture really is a journey that we go on. We don't one day wake up and say we got a producer and we're, we've got a consumer and now we're event driven. So in fact, there's much more to it than that. When we start to visualize that advent driven architecture, you'll see these, these com these uh pros at the top here, which is like business agility, decoupled architectures, evolutionary architectures that they can provide. But then equally down here, we have the common pitfalls. Um and there are ways in which we can avoid these.

We spoke about initial design. Why do we care about events? While events tell a story, organizations are very interested in events. We spoke about behavior first rather than implementing first and how event storming itself can help you paint that picture uh within your organizations.

Then we talked about implementation. We spoke about commands versus events. We looked at messaging pattern streams point to point, we looked at um pub sub patterns on aws to help you. And there are also integration patterns like enriching on contacts, mapping with domain driven design. Again, can help you avoid some of these pitfalls that people often see.

And then finally, we spoke about operational and maintenance. Ok. Spoke about standards. What goes inside your events? What do you want to standardize within your organization? Discoverability? How are people gonna find your events? What's inside the payload? If you think about that now, that can help your future selves, communication and again, reducing that complexity, move away from that number three into that number one and it is possible, but you need to start thinking about that today.

In this talk, we talk about several services that can help you do this. We spoke about sqs is fully managed message queue. We spoke about amazon venture bridge doing pub sub patterns using filtering or transformations over 28 different targets. Also api destinations, a huge feature uh that i love uh amazon dynamo db streams listening to changes within our database change data capture patterns. How can that help us? And equally here, we've got vent bridge pipes as point to points managed service doing enriching and filtering. Could that help you on your journey if you're interested in service, more service learning, we have this uh free service badge that you can take uh or take this back to your colleagues. Uh so you can do this online in your own time to dive deeper into some service stuff there.

And then finally, i just wanted to share this resource with you at the end. Like i said, i would at the start, everything i spoke about is in this resource pack, the deck, the slides are in there. Uh and uh if you want to dive deeper, you can to the side of it. I've created ed a visuals. Has anyone seen this before? Cool, nice few people, ed, a visual. So i spent a year just building avent driven architecture, visuals. If you have five minutes, you've got five minutes to learn something about avent driven solutions. So these are all free all online in that resource pack, you can download them all, all offline if you want to read them. But again, people seem to love visual ways of learning so you can do that.

And with that, i just wanna say thank you so much uh for turning up today and watching on the stream. If you enjoyed the session, if you've got any questions, feel free, i'll be in the hallway track there and if you like this session, please rate it at five. And thank you very much.