Posts

Circuit Breakers to handle failures in serverless applications

Image
Introduction When you are developing micro-services, you might find that there might be some dependencies which could fail or be temporary unavailable. Examples of these dependencies are calls to other services, data stores, etc. These dependencies might be unavailable for multiple reasons such as maintenance, throttling, etc.  The idea behind the circuit breaker pattern is wrapping the dependency, so if there is a consistent failure, we temporary stop trying to call the dependency and instead we can have define an alternative fallback function. Normally a circuit breaker can be in three different states: Closed : we call the dependency as usual. If there are failures, we track them and we open the circuit if a defined threshold is reached. Open : instead of calling the dependency, we call the fallback function. If we don't define a fallback function, it will fail early, without calling the dependency. This can prevent overloading an unhealthy dependent service or data store. While

AWS assuming role + MFA with pulumi (and other applications)

Image
Introduction In AWS, managing your IAM in a centralized account is a good practice, so then you can assume a role to access to other accounts without need of duplicating users. You can find more info about this approach in this article . In addition, for security reasons, it is highly recommended to enable MFA (Multi-factor Authentication), so you need a device to generate a temporary code to get access. In the image below it is showed how assume role with MFA works, but you also have an article about MFA in this link . This approach is really useful and most of the configuration can be automated with pulumi (or other providers like terraform or CloudFormation). And once it is configured, you can switch your role and get access to other accounts with a few clicks.  However, when it comes to external software such us pulumi, getting access assuming a role and using MFA might not be too straightforward. In this article we will see how this can be configured and how we can easily refresh

Performance improvement for EventSourcing/CQRS with Snapshots

Image
Introduction to Event Sourcing and CQRS The fundamental idea of Event Sourcing is ensuring that every change to the state of an application is captured in an event object, and that these event objects are themselves stored in the sequence they were applied for the same lifetime as the application state itself. (For more information about Event Sourcing I recommend this post written by Martin Fowler ). Based on my experience working with Event Sourcing, having the sequence of events allows you to perform awesome functionalities, like time travelling or replaying your events into multiple storage technologies. However, it can be tricky for some scenarios. To explain this better, I need to introduce Event Sourcing working together with Command Query Responsibility Segregation (CQRS). If you are not familiar with CQRS, the idea behind it is splitting commands (write operations) from queries (read operations), so they can work independently. (more info about CQRS in this post ). CQRS and Ev

Building Micro-Frontends with Single-Spa

Image
Introduction Micro-services architecture is becoming trendy since it is a suitable option many projects, specially those that are complex and we want to be able spilt our business logic into different services that can be built and deliver independently. In spite of the fact that micro-services architectures are more complex, they increase the resilience of your system and increase the time-to-market delivery, which can be crucial for many companies. But it is not all about services and backend. With the introduction of single page applications (SPAs), we can build fully-functional user interfaces that can work independently in the browser. And here we should ask ourselves the same questions as for backend applications: Is my SPA becoming a huge monolith? Would I like to deliver functional modules independently? If you using micro-services, is there a relation between my services and your UI? Do we want to have functional teams that can deliver new features (frontend+backend) independe

Managing snapshots for Amazon ElasticSearch with Dotnet Core Lambdas

Image
Introduction It is awesome to have some useful services like ElasticSearch managed by AWS , so you don't have to care about patching, monitoring, etc.  When it comes to backup management for indexes, Amazon ElasticSearch includes automated snapshots. Automated snapshots are only for cluster recovery. You can use them to restore your domain in the event of red cluster status or other data loss. Amazon ES stores automated snapshots in a preconfigured Amazon S3 bucket at no additional charge. In particular, if you are using a higher version than 5.3, Amazon ElasticSearch takes hourly automated snapshots and retains up to 336 of them for 14 days. However, this policy might not be enough if you need to keep your snapshots for a longer 15 days, or you need to take a snapshot to create a new cluster in another region, subnet, etc. If you have different requirements to manage your snapshots, you will have to work with manual snapshots. For this part, AWS does not offer a good managed exper

How to Setup EventStoreDB on AWS EC2 with Pulumi IaC

Image
Introduction  EventStoreDB (ESDB) is an industrial-strength database technology used as the central data store for event-sourced systems. It is available open-source to run locally on most platforms or as SaaS through Event Store Cloud. Currently their SaaS version is under development, so if you want to run it on your cloud, you will have to set it up. One option is docker, since ESDB is available as docker image on Docker Hub . However, as Greg Young explains in this issue , it might be not the best option when it comes to performance, due to the extra virtualization layer. For the same reason, we don't usually use docker images for other database engines like SQL Server, MongoDB, etc. Then, if you want to run ESDB on cloud, in particular AWS cloud, EC2 + EBS is the most reasonable option.  In this post, I would like to share how you can configure ESDB on EC2+EBS easily with Pulumi, which will allow you to automate this process following the best practices. Creating EC2 with Pulu