Skip to toolbar Log Out®Integration With Data Monitoring – Datadog® & Datadog Integration

Datadog provides monitoring for servers, applications, and services. With Datadog, enterprise customers are able to monitor their application workloads and get visibility into Kubernetes clusters of any scale. integration with Datadog aggregates metrics and events and provides resource prediction/recommendation for Kubernetes deployments and application-aware acceleration and optimization for Kafka. The following diagram illustrates the integration workflow.



Datadog Agent posts workload metrices to Datadog Services


Data-Adapter queries workload metrics from Datadog Services


Data-Adapter posts the Prediction/Recommendation created by to Datadog Services


Datadog Cluster Agent gets Prediction/Recommendation from Datadog Services


WPA applies Recommendation to applications


Datadog Dashboard displays workload metrics and Prediction/Recommendation by

Using Kafka as an example, the Datadog Agent monitors and collects many different types of Kafka metrics, including resources (e.g., CPU, memory) utilization and Kafka workload and performance metrics (e.g., message queue offset/length, consumer lags). With integration of ProphetStor, Kafka message production/consumption rate is continuously collected from Datadog by’s data-adapter and analyzed by the AI engine of uses advanced machine learning algorithms to predict the Kafka message production rate. Based on the prediction of message production rate, works with WPA to automatically scale Kafka consumer replicas to handle the increased or decreased workload.

Read More