How to Detect Performance Issues in Kubernetes with Built-in Monitors

In Kubernetes environments, monitoring critical workload metrics often involves writing promil queries, setting up alert rules and connecting all of them to an external system which can be timeconuming especially during an event where every minute counts and is hard to scale at the same time. That's why we have built monitors in randoli. Hi, I'm Kunal, DEL engineer at Randoli. And in this particular video, I'm going to quickly walk you through how monitors in Randoli helps solve this by providing built-in metric checks for workloads out of the box, which can be configured in minutes without you having to write any queries or configure complex rules yourself. So, in the end, this will help you to track performance issues reliably with far less operational overhead. Let me show you how this works. In order to create a monitor, head over to the monitors page on the dashboard and click on add monitor. Let's start by looking at what are all the configuration options available for you to create a new monitor. The first is the display name for the monitor. This name will be visible to you once a new monitor alert is triggered in the UI. Now, we recommend choosing a descriptive name here, which will make it easier for you to identify alerts later, especially when you're managing multiple monitors. Next option is the mode where you can choose between built-in and advanced. In the built-in mode, Randoli offers a set of built-in monitor types that covers common Kubernetes and application level metrics and patterns. Now, these are preconfigured with promql and evaluation logic which is maintained by randoli. So you don't need to write or manage anything manually. You can also choose the advanced mode where you can define your own custom conditions using promql like syntax for any specialized use cases that you may have. Now we strongly recommend starting with the built-in monitors as they cover most monitoring needs out of the box and are optimized for a fast setup and consistent alerting across workloads. You can choose the severity from these particular options. And this is a way for you to categorize your alerts based on how you want to prioritize them. Now, for the built-in monitors, you have this list of preconfigured metrics that you can track for a particular workload. So, we have the JVM and pod memory usage metrics. You'll find all the HTTP related metrics for both inbound and outbound operations. Similarly, you can track the Kafka related metrics as well for both the consumer and the producer. You also have the ability to track the database query metrics and the list goes on. Now we can choose any type of metrics that we wish to track. For example, HTTP inbound latency. And here you have the ability to define the specific operation name essentially the name of the span which you want to track. You can also select the percentile of the latency all the way from P50 to P99. We can define a threshold value which will essentially be the trigger point of this particular monitor alert. And here you have the latency value unit which is in milliseconds. You also have the ability to scope the monitor using the specific workload. Now we strongly recommend and follow a one-on-one mapping between a monitor and a particular workload. That means we recommend to create a single monitor type for a specific workload only. If you wish to track multiple metrics for a particular workload, we recommend creating separate monitors for all of them. And lastly, you can define the cluster where you wish to apply the monitor detection either by using the cluster tag or you can select a cluster by using the cluster label as well. Let me show you a quick example of an existing monitor. All right. So here's a monitor that we have configured to track the HTTP latency of the order service workload. As you can see in the workload labels, we have selected the order service as a monitor type. We have selected the HTTP inbound latency which is one of the built-in monitors. For this particular example, we haven't specified any operation name. That means it will track all the operations for that particular workload. We are tracking the P95 latency. That means we are alerting when 95% of the requests take longer than the defined threshold. Speaking of the threshold, in this particular case, we have set the threshold as 250 milliseconds. And if the inbound latency of the order service exceeds this particular threshold, we'll get a monitor alert. Lastly, we have also selected the EKS demo cluster where the order service is present. Now, once the monitor is active, the randomly agent will continuously evaluate the detection criteria that you have set and as soon as the threshold is breached by that particular workload, an alert will be surfaced directly to the workload's timeline. Let me show you how that looks. We can navigate to the overview page of the simulation order service using the unified timeline on the overview page. You can quickly view the different triggered events of this particular monitor. As you can see right here, so we have got one and two. We can also scroll down a bit to view our specific monitor alert which was triggered. You can click on the drop-down which will give us some more information. So the alert tells us that the P95 inbound latency type has reached the value of 396.67 milliseconds. It shows the display name of the monitor that we had configured. It will show the value of the threshold that we had set and also the specific value which caused the alert to be triggered. We can also see a few more information for example the name of the specific pod the name of the service which is order service in this case and the specific span or the operation which caused this particular alert. Now this kind of contextual information immediately gives you visibility into the performance regressions of this particular workload. So you are able to pinpoint exactly which particular pod or an operation has caused this and take the necessary actions before it impacts the users. Now by default all the monitor alerts are visible to you on the randoly UI but you also have the ability to route these alerts to any external channel such as Slack or email using the alert groups so that your teams can get notified in real time even if they're not looking at the dashboard. I would highly recommend to check out the documentation to know more about how you can route these monitor alerts to your specific external channels. Overall with monitors in Randoli you have the ability to track these critical workload metrics and get the relevant contextual alerts without the overhead. So in the end your teams are able to catch the issues early and respond to them faster and more efficiently. Thanks for watching.