目录
输入和输出集成概述
此插件从 RabbitMQ 服务器读取指标,提供对消息传递系统性能和状态的重要见解。
Telegraf 的 SQL 插件使用简单的表架构和动态列生成,将收集的指标发送到 SQL 数据库。当配置为 ClickHouse 时,它会调整 DSN 格式和类型转换设置,以确保无缝数据集成。
集成详情
RabbitMQ
Telegraf 的 RabbitMQ 插件允许用户通过 RabbitMQ 管理插件从 RabbitMQ 服务器收集指标。此功能对于监控 RabbitMQ 实例的性能和健康状况至关重要,RabbitMQ 实例广泛用于各种应用程序中的消息队列和处理。该插件提供对关键 RabbitMQ 指标的全面深入了解,包括消息速率、队列深度和节点健康状况统计信息,从而使运营商能够保持其消息传递基础设施的最佳性能和稳健性。此外,它还支持 secret-stores 以安全地管理敏感凭据,从而使与现有系统的集成更加顺畅。配置选项允许灵活地指定要监控的节点、队列和交换机,为不同的部署场景提供有价值的适应性。
Clickhouse
Telegraf 的 SQL 插件旨在通过基于传入指标动态创建表和列,将指标数据写入 SQL 数据库。当配置为 ClickHouse 时,它使用 clickhouse-go v1.5.4 驱动程序,该驱动程序采用独特的 DSN 格式和一组专门的类型转换规则,将 Telegraf 的数据类型直接映射到 ClickHouse 的原生类型。这种方法确保了在高吞吐量环境中的最佳存储和检索性能,使其非常适合实时分析和大规模数据仓库。动态模式创建和精确的类型映射实现了详细的时序数据日志记录,这对于监控现代分布式系统至关重要。
配置
RabbitMQ
[[inputs.rabbitmq]]
## Management Plugin url. (default: http://localhost:15672)
# url = "http://localhost:15672"
## Tag added to rabbitmq_overview series; deprecated: use tags
# name = "rmq-server-1"
## Credentials
# username = "guest"
# password = "guest"
## Optional TLS Config
# tls_ca = "/etc/telegraf/ca.pem"
# tls_cert = "/etc/telegraf/cert.pem"
# tls_key = "/etc/telegraf/key.pem"
## Use TLS but skip chain & host verification
# insecure_skip_verify = false
## Optional request timeouts
## ResponseHeaderTimeout, if non-zero, specifies the amount of time to wait
## for a server's response headers after fully writing the request.
# header_timeout = "3s"
##
## client_timeout specifies a time limit for requests made by this client.
## Includes connection time, any redirects, and reading the response body.
# client_timeout = "4s"
## A list of nodes to gather as the rabbitmq_node measurement. If not
## specified, metrics for all nodes are gathered.
# nodes = ["rabbit@node1", "rabbit@node2"]
## A list of queues to gather as the rabbitmq_queue measurement. If not
## specified, metrics for all queues are gathered.
## Deprecated in 1.6: Use queue_name_include instead.
# queues = ["telegraf"]
## A list of exchanges to gather as the rabbitmq_exchange measurement. If not
## specified, metrics for all exchanges are gathered.
# exchanges = ["telegraf"]
## Metrics to include and exclude. Globs accepted.
## Note that an empty array for both will include all metrics
## Currently the following metrics are supported: "exchange", "federation", "node", "overview", "queue"
# metric_include = []
# metric_exclude = []
## Queues to include and exclude. Globs accepted.
## Note that an empty array for both will include all queues
# queue_name_include = []
# queue_name_exclude = []
## Federation upstreams to include and exclude specified as an array of glob
## pattern strings. Federation links can also be limited by the queue and
## exchange filters.
# federation_upstream_include = []
# federation_upstream_exclude = []
Clickhouse
[[outputs.sql]]
## Database driver
## Valid options include mssql, mysql, pgx, sqlite, snowflake, clickhouse
driver = "clickhouse"
## Data source name
## For ClickHouse, the DSN follows the clickhouse-go v1.5.4 format.
## Example DSN: "tcp://localhost:9000?debug=true"
data_source_name = "tcp://localhost:9000?debug=true"
## Timestamp column name
timestamp_column = "timestamp"
## Table creation template
## Available template variables:
## {TABLE} - table name as a quoted identifier
## {TABLELITERAL} - table name as a quoted string literal
## {COLUMNS} - column definitions (list of quoted identifiers and types)
table_template = "CREATE TABLE {TABLE} ({COLUMNS})"
## Table existence check template
## Available template variables:
## {TABLE} - table name as a quoted identifier
table_exists_template = "SELECT 1 FROM {TABLE} LIMIT 1"
## Initialization SQL (optional)
init_sql = ""
## Maximum amount of time a connection may be idle. "0s" means connections are never closed due to idle time.
connection_max_idle_time = "0s"
## Maximum amount of time a connection may be reused. "0s" means connections are never closed due to age.
connection_max_lifetime = "0s"
## Maximum number of connections in the idle connection pool. 0 means unlimited.
connection_max_idle = 2
## Maximum number of open connections to the database. 0 means unlimited.
connection_max_open = 0
## Metric type to SQL type conversion for ClickHouse.
## The conversion maps Telegraf metric types to ClickHouse native data types.
[outputs.sql.convert]
conversion_style = "literal"
integer = "Int64"
text = "String"
timestamp = "DateTime"
defaultvalue = "String"
unsigned = "UInt64"
bool = "UInt8"
real = "Float64"
输入和输出集成示例
RabbitMQ
-
监控队列性能指标:使用 RabbitMQ 插件来跟踪队列性能随时间的变化。这包括设置监控仪表板,以可视化关键队列指标,例如消息速率、消费者数量和消息传递速率。借助此信息,团队可以通过分析趋势并就扩展或优化其 RabbitMQ 配置做出数据驱动的决策,从而主动解决任何瓶颈或性能问题。
-
系统健康状况警报:将 RabbitMQ 插件与警报系统集成,以将 RabbitMQ 实例中潜在的问题通知给运营团队。例如,如果未确认消息的数量达到临界阈值,或者队列变得不堪重负,则可以触发警报,从而可以立即进行调查并迅速采取补救措施以维护消息流的健康状况。
-
分析消息处理指标:使用该插件收集有关消息处理性能的详细指标,例如消息发布、确认和重新传递的速率。通过分析这些指标,团队可以评估其消息消费者应用程序的效率,并在必要时调整配置或代码,从而提高整体系统吞吐量和弹性。
-
跨系统数据集成:利用 RabbitMQ 插件收集的指标来集成 RabbitMQ 和其他系统或服务之间的数据流。例如,使用收集的指标来驱动自动化工作流程或分析管道,这些工作流程或分析管道利用在 RabbitMQ 中处理的消息,从而使组织能够优化工作流程并提高其生态系统中的数据敏捷性。
Clickhouse
-
大容量数据的实时分析:使用该插件将来自大型系统的流式指标馈送到 ClickHouse 中。此设置支持超快的查询性能和近乎实时的分析,非常适合监控高流量应用程序。
-
时序数据仓库:将插件与 ClickHouse 集成,以创建强大的时序数据仓库。此用例允许组织存储详细的历史指标,并执行复杂的查询以进行趋势分析和容量规划。
-
分布式环境中的可扩展监控:利用该插件在 ClickHouse 中为每种指标类型动态创建表,从而更容易管理和查询来自大量分布式系统的数据,而无需事先定义模式。
-
针对物联网部署的优化存储:部署该插件以将来自物联网传感器的数据摄取到 ClickHouse 中。其高效的模式创建和原生类型映射有助于处理海量数据,从而实现实时监控和预测性维护。
反馈
感谢您成为我们社区的一份子!如果您有任何一般性反馈或在这些页面上发现了任何错误,我们欢迎并鼓励您提出意见。请在 InfluxDB 社区 Slack 中提交您的反馈。