Demystifying Dapr — State Management Building Block for Distributed Apps
Today microservices have became a de-facto approach for developing new distributed applications or enhancing existing applications. But building microservices based distributed applications comes with its own set of challenges. These challenges comes from many different micro-applications, which solves specific problems in the solution. The most common problems are:
- Secure communication
- Service discovery
- State management
- Decoupling of services
- Observability
These common problems are well known and standard solutions are available in industry, such as use of mutual TLS, certificate management, vault and fine-grained access control can be used to solve the problem of secure communications. Similarly pub/sub solutions like service bus or apache kafka implement a better decoupling of the services. We will discuss about each problem and the possible solutions in detail further in our discussion, but we implement these solutions in our application as a part of infrastructure layer or cross-cutting concerns. Now since the problems are identified and the solutions are available, lets look at what is Dapr and why to use it in distributed apps.
What is Dapr??
Dapr is an open-source distributed application runtime that provides us with APIs in programming language-agnostic way. These APIs abstracts away the complexity of common problems we encounter when building distributed microservice applications.
Dapr provides solutions in the form of building blocks and each building block provides API that can be called from any application through HTTP endpoint. Using Dapr makes it very easy to implement common patterns and capabilities needed to build the cross-cutting concerns for a distributed application and have a robust industry‑proven implementation so that application becomes stable and robust itself.
Documentation for Dapr is available at https://dapr.io/
Dapr is a Cloud Native Computing Foundation (popularly known as CNCF) incubating project, CNCF hosts critical components of Cloud’s native software stacks including K8s and Prometheus. Various vendors such as Google cloud, AWS, Azure often contribute to support specific implementation if it comes to the cloud based hosting solutions.
Dapr Building Blocks
As mentioned before Dapr is a collection of building blocks, and each of these building blocks implements a capability that is commonly needed while building a distributed application. We can check the current list of Dapr APIs at Dapr.io
As we can see multiple building blocks are available, we will go through them one by one in the series of discussion. The building block we will discuss in this article is State Management. The good thing about Dapr building block is that they are optional, so if we are already using something say Pub/Sub or secret management we can keep using it.
Before moving ahead its important to understand how these building blocks are implemented. Dapr does not take the traditional approach of implementing as a component or library and load into Microservice process but it runs along side Microservices as a sidecar which is a separate process. There is a Dapr sidecar for each for the microservices and when the application wants to make use of any of the Dapr building blocks it does so by making a straightforward network request to its sidecar. Refer to below diagram for better understanding.
There are several benefits if this approach, sidecar can be configured to automatically encrypt the traffic between Microservice using mTLS, Dapr can enforce access control policies also Dapr can capture the traces and metrics with sidecar. Since all this implementation is done inside the sidecar, it keeps all the Microservice code straightforward. Also Dapr has been extensively optimized to keep any performance impact to a minimum as it make use of gRPC for the communication between sidecars.
State Management Building Block
The Dapr state management building block is designed for the situations where microservices need to store state as key‑value pairs. It’s extremely common for services in a distributed application to have some state that need to store, and often need to share that state between multiple instances of the same service running on different machines. And this means that we need a shared store that all of the instances can access. Dapr state management building building block helps us to manage data which is in the form of key-value pair and generally stored in cache like Redis.
Benefits of Dapr
We can directly use Redis or any other store for state management but Dapr has its own benefits. Biggest benefit of using Dapr is to have ability of swapping backend stores. Say today we are using Redis as backend store but tomorrow we swap to use MongoDB or some other store. This ability will help us using may be different stores for different environment. It also provide capability of using multiple stores based on our requirement such as in ecommerce application as we may need to use Redis for storing cart information where as user profile is stored in MongoDB. In order to use multiple stores all we need to do is issue HTTP request to Dapr sidecar and set the state.
By default the state is isolated in microservices, but we can easily configure Dapr shared state store that can share the state between multiple applications. Dapr state management building block offers other useful capabilities such as control over time to live, concurrency control, querying, bulk operation and automatic encryption.
Dapr State Management Building Block in Action
Here in our example we will discuss how to store product list as key-value pair using Dapr state management component. Before that we will setup Dapr on our machine. First we will install Dapr CLI from Dapr official website. Here at this link we will get the installation instruction for all the platforms.
I am using windows machine so will use following command:
Once we install it through command prompt when we run “dapr -v” and can see the dapr version.
Since I have already installed dapr on my machine I can see Runtime Version along with CLI version. If you are installing it for the first time you need to install runtime as well. Instruction are available here on Dapr official website. One thing we need to note that installing Dapr locally does expect that we’ve got Docker already installed on your machine. If we don’t have Docker installed, then I recommend downloading Docker Desktop for Windows or Mac. Next we need to run “dapr init” command. It download some container images.
Once installed we can run “docker ps” command to see the containers. There’s a Redis container, which is used for the state store and pub‑sub messaging Dapr building blocks; there’s a Zipkin container, which is used for observability; and there’s also a dapr placement container, which is there in case you want to use the actor building block. Its not necessary that we use these self hosted containers, we are free to define our Dapr component that point to the services that are not running as docker container.
Dapr Component Setup
Another thing that happened when we ran dapr init is that it created global components definition .dapr folder in user profile. Inside this there is config.yaml which include Dapr configurations for tracing.
Also there is components folder, and if we look inside that folder, we can see that there are two YAML files, and these files define the state and pub‑sub Dapr building blocks. These default components make it even easier for us to get started with local development. And both of these components make use of that Dapr Redis container that got created for us with the dapr init command.
Now if we look into the statestore.yaml file it says its a yaml component its name is statestore. We can have multiple statestore and give them different names. Also when the Dapr is installed in self-hosted mode Redis container was automatically created and the component is ready to use this Redis container. This Redis container is exposed on port number: 6379. We have already verified it by running “docker ps” command above. statestore.yaml configuration file contains other settings as well and to find the settings and their corresponding values we can refer to Dapr documentation here.
Out of all these state stores available to us we will currently consider redis for this discussion. so lets open the documentation for redis and here we can see the complete list of configuration settings for redis:
Based on our preference we can select any statestore other than redis and we can keep multiple statestores as well.
Statestores have different certification levels i.e. Alpha, Beta and Stable. Please refer to stable version for production.
Run Dapr Locally
To run Dapr locally we need to execute “dapr run” command using Dapr http port switch so that Dapr should listen on port 3500. There are other arguments which we can pass which we will discuss later. When we run the command and check the logs we can see Dapr sidecar is up and running.
Next we will send a http request to port 3500 and see if its working fine or not. For that I will use Postman. We create a below post request url: http://localhost:3500/v1.0/state/statestore to store the state in Dapr and pass [{“key”: “name”, “value”: “gagan”}] as body. On sending the request we will get 204 created response.
Next we will test whether the response for that we execute a Get request (http://localhost:3500/v1.0/state/statestore/name ) to statestore and we will receive the value of key: “”name in return:
This shows Dapr is running successfully on local machine. Before moving ahead lets understand the statestore url i.e. http://localhost:3500/v1.0/state/statestore/
By default Dapr sidecar runs on 3500 port. “/state/” part of the url means we are using state management building block. here “v1.0” means we are using version 1 of state management building block. statestore part of it is the name of statestore as defined in YAML file. We can give our own name to state store as we will see in the demo application. Now we have 2 options here either we can directly use these endpoints and make direct http call to use Dapr or Dapr provides a SDK which which will take care of calling these APIs and we can focus on business logic.
Application Setup for Dapr
Here we will discuss about how to setup a .NET 6 application to use Dapr as sidecar. The code for dapr state management sample application can be found at this github link at tekspry/dapr-state-management-sample-app .
Now lets discuss about the application. For this demo we are considering simple 3 layered application. Presentation layer is build using React App. Service layer consist of .Net 6 web api with Dapr running as sidecar and Redis store to persist the product information to Redis.
Web Application
Currently web application shows the product list which is populated from product service. It also has capability of adding a new product to the list.
Since this discussion is about Dapr so we will not get into the depth of React app, we can refer to this link for more details.
Product Service
It is .Net 6 based web API which is configured to load sample date from Redis store. Here is the structure of code which we have used in the project:
ecom.product folder contains the product service and ecom.spa contains the react app.
ecom.product further contains 5 folders. Product service is created based on clean architecture. WebAPI code is written in ecom.ProductService folder rest of them are class libraries.
Dapr folder within ecom.product contains various building blocks which can be used with Dapr.
Only one which matters now is statestore.yml which we have discussed earlier. “type: state.redis” states that it used redis as backend store and its name is “shoppingcache”
We have created a PowerShell script “start-product-app.ps1” to run ecom.ProductService in self-hosted mode. It uses “dapr run” to start Dapr along with other arguments used to configure Dapr sidecar.
Here “ app-id product” is the unique id given to the service along with which this Dapr will run as sidecar.
With “app-port 5016” we tell the Dapr sidecar what port our microservice is listening on. For thsi application its 5016.
“dapr-http-port 3501" defines the port on which Dapr sidecar is listening to.
“component-path ../dapr/components” it defined where are Dapr components are defined, as discussed above they reside in dapr folder.
“dotnet run” last one is to start the product service in self hosted mode.
Configuring Dapr State Management Building Block
To configure Dapr in our project we will need to add the reference of nuget package “Dapr.AspNetCore” . We will add this package to “ecom.product.domain.csproj” project file. At the time of writing this blog we are using 1.7 version.
Next we will discuss about the changes we need to add in “Program.cs” file where all the startup code for the web api resides. Here we need to add “builder.Services.AddControllers().AddDapr();” It will register the Dapr client into the IOC.
Here we have defined a controller which contains 3 endpoints related to product service.
“GetAll()” — will get all the available products
“GetById()” — will get the product based on id
“Add()” — will add a new product to the product list.
Loading Sample Data to Dapr
Lets go to “ecom.product.database” to look at the code to get the product details from Redis using Dapr. Inside the database project we have “ProductRepository” class which is inherited by “IProductRepository” interface.
We need to look at the couple of things in the constructor of this class, first we have added the reference to “DaprClient” and next we have added the reference to cache store name which is “shoppingcache” in our case.
In the constructor “LoadSampleData” method id called to load the sample data to DAPR store when the application is loaded.
As we can see it loads 3 products by default to a product list and then call “SaveProductListToCacheStore” method to store these product list items in redis via Dapr client.
“SaveProductListToCacheStore” method simply creates a key “productlist” for redis store and then call “daprClient.SaveStateAsync();” method to save the information in redis.
Fetching Product List using Dapr Client
This will provide sample data at the time when the application is bootstrapped. Now when the controller ask for lists of product, “GetProducts()” method of “ProductRepository” is called.
Here we have called a method “GetStateAsync()” provided by Dapr client to get all products available in “productlist” key in redis.
Now to run the Product service, we will execute start-product-app.ps1.
When we execute the command we can see a message “You’re up and running! Both Dapr and your app logs will appear here.” which says Dapr is up and running. Also, we can see there is a text in blue color, its basically signify logs from .NET 6 web api. Color is used to differentiate between Dapr and .NET 6 logs.
Now lets open the API with Swagger, we have used a “swashbuckle.AspnetCore” nuget package to implement swagger.
As the logs above suggest that our API is up and running at port 5016 so lets open “http://localhost:5016/swagger/index.html” url.
Wow, application is up and running.
Now lets try to get the products from our API. We will execute the HTTPGET /product endpoint and on execution we will get following result. As we can see request is successful and we have received all 3 product details from Redis via Dapr.
Now lets see how frontend app works. We will not get into the depth of frontend project, but just for the reference we can see within ProductHooks, we have created a hook, called “useFetchProducts” and as we can see we are making a get call to /product endpoint using axios.
Here baseuri is configured in config.ts file as “http://localhost:5016”, which is nothing but endpoint of our API.
Now lets run the web application using “npm start” command. Application is successfully up and running at port 3000.
Adding New Product to Redis via Dapr
Now lets test the another operation of adding new product to Redis via Dapr client. We have a method “CreateProduct()” in ProductRepository. This method will create a new product in Redis for “productlist” key.
Now when the post endpoint /product is called in the controller, it will further call “CreateProduct” in ProductRepository which will further add the new product in Redis and return the product id.
Now lets test this endpoint from the frontend. Inside productHooks.ts, we have created a hook called “useAddProduct” which calls the post endpoint /product using axios to add the product and on successfull response it navigates back to product list page.
So to try this functionality, lets click on Add button on product list page:
Form is routed to new page (http://localhost:3000/product/add):
We will add the required details and try to add Camera as a new product to the list. After filling the details in the form we will click on Submit button.
It will add the the product to the Redis and navigate back to product list page where we can see Camera is now added to the list.
With this we come to the end of this discussion. We can refer to https://docs.dapr.io/developing-applications/sdks/dotnet/dotnet-client/ link to explore other methods supported by Dapr SDK.
You can download the codebase from this github link
Next we will discuss about other building blocks such as pub/sub, secrets etc. supported by Dapr and how to run the Dapr in Kubernetes cluster and used with various cloud vendors.
Disclaimer: This is a personal blog. Any views or opinions represented in this blog are personal and belong solely to the blog owner and do not represent those of people, institutions or organizations that the owner may or may not be associated with in professional or personal capacity, unless explicitly stated.
