Skip to main content

Home

Kafka endpoints
/kafka/brokers?cluster_id={cluster_name}&start_time={timestamp}&end_time={timestamp}

Gets a list of Kafka brokers active on a given cluster during a given timeframe.

Request
GET http://unravel-host:3000/api/v1/kafka/brokers?cluster_id={cluster_name}&start_time={timestamp}&end_time={timestamp}
Path parameters

None.

Query parameters

Required parameters are highlighted.

Name

Type

Description

cluster_id

string

Cluster name.

start_time

string

Start time. Format: YYYY-MM-DDTHH:MM:SS.NNNZ or Unix epoch time.

end_time

string

End time. Format: YYYY-MM-DDTHH:MM:SS.NNNZ or Unix epoch time.

Response body

The response body contains an array of metrics per broker.

Name

Type

Description

broker_name

array

Array of broker metrics.

Examples

Request:

curl -X GET "http://myserver.com:3000/api/v1/kafka/brokers?cluster_id=QAHDP26B&end_time=2019-06-03T16:08:52%2B05:30&start_time=2019-06-03T15:08:52%2B05:30" -H  "accept: application/json" -H  "Authorization: JWT token"

Response:

{"QAHDP26B": {
    "ActiveControllerCount-Value":1,
    "OfflinePartitionsCount-Value":43,
    "BytesInPerSec_newTopic1-OneMinuteRate":8.893181625e-314,
    "BytesOutPerSec___consumer_offsets-OneMinuteRate":5549.999999999995,
    "MessagesInPerSec___consumer_offsets-OneMinuteRate":49.99999999999996,
    "TotalFetchRequestsPerSec___consumer_offsets-OneMinuteRate":514.6005947658281,
    "UnderReplicatedPartitions-Value":0,
    }
}
/kafka/{cluster_name}/kpi?broker={broker_name}?start_time={timestamp}&end_time={timestamp}&interval={minutes}&prefix={metrics}

Gets KPIs for a given Kafka broker.

Request
GET http://unravel-host:3000/api/v1/kafka/{cluster_name}/kpi?broker={broker_name}?start_time={timestamp}&end_time={timestamp}&interval={minutes}&prefix={metrics}
Path parameters

Name

Description

cluuster_name

Cluster name.

Query parameters

Required parameters are highlighted.

Name

Type

Description

broker

string

Broker name.

start_time

string

Start time. Format: YYYY-MM-DDTHH:MM:SS.NNNZ or Unix epoch time.

end_time

string

End time. Format: YYYY-MM-DDTHH:MM:SS.NNNZ or Unix epoch time.

interval

string

Interval size, in minutes.

Valid values are 1m, 5m, 10m, 30m.

prefix

string

List of metrics to graph. Separate each metric with +.

Valid values are:

BytesInPerSec
BytesOutPerSec
MessagesInPerSec
TotalFetchRequestsPerSec
UnderReplicatedPartitions
ActiveControllerCount
RequestHandlerAvgIdlePercent-OneMinuteRate
PartitionCount-Value
LeaderCount-Value
OfflinePartitionsCount
TotalTimeMs_Fetch-99thPercentile
RequestsPerSec_Fetch-OneMinuteRate
RequestsPerSec_Produce-OneMinuteRate
PurgatorySize_Produce-Value
PurgatorySize_Fetch-Value
Response body

The response is one array per broker, with the following KPIs:

Name

Type

Description

ts

string

Timestamp.

avg_vd

string

Examples

Request:

curl -X GET "http://myserver.com:3000/api/v1/kafka/QAHDP26B/kpi?broker=QAHDP26B.kafka1
&end_time=2019-06-03T16:56:09%2B05:30&interval=1m&prefix=BytesInPerSec&start_time=2019-06-03T15:56:09%2B05:30" -H  "accept: application/json" -H  "Authorization: JWT token"

Response:

{"QAHDP26B.kafka2":[{"ts":1559559780000,"avg_vd":"504.710000000000"}]}
/kafka/{cluster_name}/kpi_clusters?start_time={timestamp}&end_time={timestamp}&interval={minutes}&prefix={metrics}

Gets KPIs for a given Kafka cluster.

Request
GET http://unravel-host:3000/api/v1/kafka/{cluster_name}/kpi_clusters?start_time={timestamp}&end_time={timestamp}&interval={interval}&prefix={metrics}
Path parameters

Name

Description

cluster_name

Cluster name.

Query parameters

Required parameters are highlighted.

Name

Type

Description

start_time

string

Start time. Format: YYYY-MM-DDTHH:MM:SS.NNNZ or Unix epoch time.

end_time

string

End time. Format: YYYY-MM-DDTHH:MM:SS.NNNZ or Unix epoch time.

interval

string

Interval size, in minutes.

Valid values are 1m, 5m, 10m, 30m.

prefix

string

Metric to graph.

Valid values are:

BytesInPerSec
BytesOutPerSec
MessagesInPerSec
TotalFetchRequestsPerSec
UnderReplicatedPartitions
ActiveControllerCount
RequestHandlerAvgIdlePercent-OneMinuteRate
PartitionCount-Value
LeaderCount-Value
OfflinePartitionsCount
TotalTimeMs_Fetch-99thPercentile
RequestsPerSec_Fetch-OneMinuteRate
RequestsPerSec_Produce-OneMinuteRate
PurgatorySize_Produce-Value
PurgatorySize_Fetch-Value
Response body

The response is one array per cluster, with the following KPIs:

Name

Type

Description

Ts

string

Timestamp.

avg_vd

string

Examples

Request:

curl -X GET "http://myserver.com:3000/api/v1/kafka/QAHDP26B/kpi_clusters?end_time=2019-06-03T16:56:09%2B05:30&interval=1m&prefix=BytesInPerSec&start_time=2019-06-03T15:56:09%2B05:30" -H  "accept: application/json" -H  "Authorization: JWT token"

Response:

{ "QAHDP26B": [ { "Ts":1559557560000, "avg_vd":"0.000000000000" } ] }
/kafka/{cluster_name}/kpi_topics?start_time={timestamp}&end_time={timestamp}&interval={minutes}&prefix={metrics}

Gets KPIs for the Kafka topics within a given timespan.

Request
GET http://unravel-host:3000/api/v1/kafka/{cluster_name}/kpi_topics?start_time={timestamp}&end_time={timestamp}&interval={minutes}&prefix={metrics+topic_OneMinuteRate}
Path parameters

Name

Description

cluster_name

Cluster name.

Query parameters

Required parameters are highlighted.

Name

Type

Description

start_time

string

Start time. Format: YYYY-MM-DDTHH:MM:SS.NNNZ or Unix epoch time.

end_time

string

End time. Format: YYYY-MM-DDTHH:MM:SS.NNNZ or Unix epoch time.

interval

string

Size of interval in minutes .

prefix

string

List of topics names to graph. Separate each metric with +.

Response body

The response body contains one array per topic, with the following fields.

Name

Type

Description

ts

string

Timestamp.

avg_vd

string

Examples

Request:

curl -X GET "http://myserver.com:3000/api/v1/kafka/QAHDP26B/kpi_topics?end_time=2019-06-04T10:21:55%2B05:30&interval=1m&prefix=BytesInPerSec___consumer_offsets-OneMinuteRate&start_time=2019-06-04T09:21:55%2B05:30" -H  "accept: application/json" -H  "Authorization: JWT token"

Response:

{"__consumer_offsets":[{"ts":1559620320000,"avg_vd":"4310.999986671694"}]}
/kafka/clusters?gte={timestamp}&lt={timestamp}

Gets Kafka cluster metrics.

Request
GET http://unravel-host:3000/api/v1/kafka/clusters?gte={timestamp}&lt={timestamp}
Path parameters

None.

Query parameters

Name

Type

Description

gte

string

Start time. Format: YYYY-MM-DDTHH:MM:SS.NNNZ or Unix epoch time. Required.

lt

string

End time. Format: YYYY-MM-DDTHH:MM:SS.NNNZ or Unix epoch time. Required.

Response body

The response body is a JSON structure identified by cluster name containing cluster metrics.

Name

Type

Description

ActiveControllerCount-Value

integer

OfflinePartitionsCount-Value

integer

BytesInPerSec_newTopic1-OneMinuteRate

float

BytesOutPerSec___consumer_offsets-OneMinuteRate

float

MessagesInPerSec___consumer_offsets-OneMinuteRate

float

TotalFetchRequestsPerSec___consumer_offsets-OneMinuteRate

float

UnderReplicatedPartitions-Value

integer

Examples

Request:

curl -X GET "http://myserver.com:3000/api/v1/kafka/clusters?gte=2019-06-02T14:14:18%2B05:30&lt=2019-06-03T14:14:18%2B05:30" -H  "accept: application/json" -H  "Authorization: JWT token"

Response:

{
"QAHDP26B": {    
    "ActiveControllerCount-Value":1,
    "OfflinePartitionsCount-Value":43,
    "BytesInPerSec_newTopic1-OneMinuteRate":8.893181625e-314,
    "BytesOutPerSec___consumer_offsets-OneMinuteRate":5549.999999999995,
    "MessagesInPerSec___consumer_offsets-OneMinuteRate":49.99999999999996,
    "TotalFetchRequestsPerSec___consumer_offsets-OneMinuteRate":514.6005947658281,
    "UnderReplicatedPartitions-Value":0,
    }
}
/kafka/topics/partions/ts?cluster_id={cluster_name}?topic_id={topic_name}&start_time={timestamp}&end_time={timestamp}&interval={minutes}&metric_pattern={metric}

Gets details about a given partition.

Request
GET http://unravel-host:3000/api/v1/kafka/topics/partions/ts?cluster_id={cluster_name}?topic_id={topic_name}&start_time={timestamp}&end_time={timestamp}&interval={minutes}&metric_pattern={metric}
Path parameters

None.

Query parameters

Name

Type

Description

cluster_id

string

Cluster name. Required.

topic_id

string

Topic name. Required.

start_time

string

Start time. Format: YYYY-MM-DDTHH:MM:SS.NNNZ or Unix epoch time. Required.

end_time

string

End time. Format: YYYY-MM-DDTHH:MM:SS.NNNZ or Unix epoch time. Required.

interval

string

Size of interval in minutes . Required.

metric_pattern

string

List of metrics to graph. Separate each metric with +. Required.

Valid values are:

Offset: LogEndOffset___consumer_offsets_partition-number-Value
Consumer Lag: CgLag_*_topic___consumer_offsets_partition-number

where partition-number corresponds to the partition number (0-any number).

Response body

The response body contains details about the given partition.

Name

Type

Description

key_as_string

string

Date in UTC time.

Key

string

Date in Unix epoch time.

Doc_count

string

Document count.

Avg_vl

string

Array of key average value pairs.

Examples

Request:

curl -X GET "http://myserver.com:3000/api/v1/kafka/topics/partions/ts?cluster_id=QAHDP26B&end_time=2019-06-04T12:24:49%2B05:30&interval=1m&metric_pattern=CgLag_*_topic___consumer_offsets_5&start_time=2019-06-04T11:24:49%2B05:30&topic_id=__consumer_offsets" -H  "accept: application/json" -H  "Authorization: JWT token"

Response:

{
"key_as_string":"2019-06-04T05:55:00.000Z",
"Key":1559627700000,
"Doc_count":3,
"Avg_vl":{"value":2016}
}
In this section: