gNMI 和 Elasticsearch 集成

强大的性能和简单的集成,由 InfluxData 构建的开源数据连接器 Telegraf 提供支持。

info

对于大规模实时查询,这不是推荐的配置。为了获得查询和压缩优化、高速摄取和高可用性,您可能需要考虑 gNMI 和 InfluxDB

50 亿 +

Telegraf 下载量

#1

时间序列数据库
来源:DB Engines

10 亿 +

InfluxDB 下载量

2,800+

贡献者

目录

强大的性能,无限的扩展能力

收集、组织和处理海量高速数据。当您将任何数据视为时间序列数据时,它都会更有价值。借助 InfluxDB,这个排名第一的时间序列平台旨在与 Telegraf 一起扩展。

查看入门方法

输入和输出集成概览

gNMI(gRPC 网络管理接口)输入插件使用 gNMI Subscribe 方法从网络设备收集遥测数据。它支持 TLS 以实现安全身份验证和数据传输。

Telegraf Elasticsearch 插件无缝地将指标发送到 Elasticsearch 服务器。该插件处理模板创建和动态索引管理,并支持各种 Elasticsearch 特有功能,以确保数据格式正确,以便存储和检索。

集成详情

gNMI

此输入插件与供应商无关,可以与任何支持 gNMI 规范的平台一起使用。它基于 gNMI Subscribe 方法使用遥测数据,从而可以实时监控网络设备。

Elasticsearch

此插件将指标写入 Elasticsearch,Elasticsearch 是一种分布式 RESTful 搜索和分析引擎,能够以近乎实时的速度存储大量数据。它旨在处理 Elasticsearch 5.x 到 7.x 版本,并利用其动态模板功能来正确管理数据类型映射。该插件支持高级功能,例如模板管理、动态索引命名以及与 OpenSearch 的集成。它还允许配置身份验证和 Elasticsearch 节点的运行状况监控。

配置

gNMI


[[inputs.gnmi]]
  ## Address and port of the gNMI GRPC server
  addresses = ["10.49.234.114:57777"]

  ## define credentials
  username = "cisco"
  password = "cisco"

  ## gNMI encoding requested (one of: "proto", "json", "json_ietf", "bytes")
  # encoding = "proto"

  ## redial in case of failures after
  # redial = "10s"

  ## gRPC Keepalive settings
  ## See https://pkg.go.dev/google.golang.org/grpc/keepalive
  ## The client will ping the server to see if the transport is still alive if it has
  ## not see any activity for the given time.
  ## If not set, none of the keep-alive setting (including those below) will be applied.
  ## If set and set below 10 seconds, the gRPC library will apply a minimum value of 10s will be used instead.
  # keepalive_time = ""

  ## Timeout for seeing any activity after the keep-alive probe was
  ## sent. If no activity is seen the connection is closed.
  # keepalive_timeout = ""

  ## gRPC Maximum Message Size
  # max_msg_size = "4MB"

  ## Enable to get the canonical path as field-name
  # canonical_field_names = false

  ## Remove leading slashes and dots in field-name
  # trim_field_names = false

  ## Guess the path-tag if an update does not contain a prefix-path
  ## Supported values are
  ##   none         -- do not add a 'path' tag
  ##   common path  -- use the common path elements of all fields in an update
  ##   subscription -- use the subscription path
  # path_guessing_strategy = "none"

  ## Prefix tags from path keys with the path element
  # prefix_tag_key_with_path = false

  ## Optional client-side TLS to authenticate the device
  ## Set to true/false to enforce TLS being enabled/disabled. If not set,
  ## enable TLS only if any of the other options are specified.
  # tls_enable =
  ## Trusted root certificates for server
  # tls_ca = "/path/to/cafile"
  ## Used for TLS client certificate authentication
  # tls_cert = "/path/to/certfile"
  ## Used for TLS client certificate authentication
  # tls_key = "/path/to/keyfile"
  ## Password for the key file if it is encrypted
  # tls_key_pwd = ""
  ## Send the specified TLS server name via SNI
  # tls_server_name = "kubernetes.example.com"
  ## Minimal TLS version to accept by the client
  # tls_min_version = "TLS12"
  ## List of ciphers to accept, by default all secure ciphers will be accepted
  ## See https://pkg.go.dev/crypto/tls#pkg-constants for supported values.
  ## Use "all", "secure" and "insecure" to add all support ciphers, secure
  ## suites or insecure suites respectively.
  # tls_cipher_suites = ["secure"]
  ## Renegotiation method, "never", "once" or "freely"
  # tls_renegotiation_method = "never"
  ## Use TLS but skip chain & host verification
  # insecure_skip_verify = false

  ## gNMI subscription prefix (optional, can usually be left empty)
  ## See: https://github.com/openconfig/reference/blob/master/rpc/gnmi/gnmi-specification.md#222-paths
  # origin = ""
  # prefix = ""
  # target = ""

  ## Vendor specific options
  ## This defines what vendor specific options to load.
  ## * Juniper Header Extension (juniper_header): some sensors are directly managed by
  ##   Linecard, which adds the Juniper GNMI Header Extension. Enabling this
  ##   allows the decoding of the Extension header if present. Currently this knob
  ##   adds component, component_id & sub_component_id as additional tags
  # vendor_specific = []

  ## YANG model paths for decoding IETF JSON payloads
  ## Model files are loaded recursively from the given directories. Disabled if
  ## no models are specified.
  # yang_model_paths = []

  ## Define additional aliases to map encoding paths to measurement names
  # [inputs.gnmi.aliases]
  #   ifcounters = "openconfig:/interfaces/interface/state/counters"

  [[inputs.gnmi.subscription]]
    ## Name of the measurement that will be emitted
    name = "ifcounters"

    ## Origin and path of the subscription
    ## See: https://github.com/openconfig/reference/blob/master/rpc/gnmi/gnmi-specification.md#222-paths
    ##
    ## origin usually refers to a (YANG) data model implemented by the device
    ## and path to a specific substructure inside it that should be subscribed
    ## to (similar to an XPath). YANG models can be found e.g. here:
    ## https://github.com/YangModels/yang/tree/master/vendor/cisco/xr
    origin = "openconfig-interfaces"
    path = "/interfaces/interface/state/counters"

    ## Subscription mode ("target_defined", "sample", "on_change") and interval
    subscription_mode = "sample"
    sample_interval = "10s"

    ## Suppress redundant transmissions when measured values are unchanged
    # suppress_redundant = false

    ## If suppression is enabled, send updates at least every X seconds anyway
    # heartbeat_interval = "60s"

Elasticsearch


[[outputs.elasticsearch]]
  ## The full HTTP endpoint URL for your Elasticsearch instance
  ## Multiple urls can be specified as part of the same cluster,
  ## this means that only ONE of the urls will be written to each interval
  urls = [ "http://node1.es.example.com:9200" ] # required.
  ## Elasticsearch client timeout, defaults to "5s" if not set.
  timeout = "5s"
  ## Set to true to ask Elasticsearch a list of all cluster nodes,
  ## thus it is not necessary to list all nodes in the urls config option
  enable_sniffer = false
  ## Set to true to enable gzip compression
  enable_gzip = false
  ## Set the interval to check if the Elasticsearch nodes are available
  ## Setting to "0s" will disable the health check (not recommended in production)
  health_check_interval = "10s"
  ## Set the timeout for periodic health checks.
  # health_check_timeout = "1s"
  ## HTTP basic authentication details.
  ## HTTP basic authentication details
  # username = "telegraf"
  # password = "mypassword"
  ## HTTP bearer token authentication details
  # auth_bearer_token = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9"

  ## Index Config
  ## The target index for metrics (Elasticsearch will create if it not exists).
  ## You can use the date specifiers below to create indexes per time frame.
  ## The metric timestamp will be used to decide the destination index name
  # %Y - year (2016)
  # %y - last two digits of year (00..99)
  # %m - month (01..12)
  # %d - day of month (e.g., 01)
  # %H - hour (00..23)
  # %V - week of the year (ISO week) (01..53)
  ## Additionally, you can specify a tag name using the notation {{tag_name}}
  ## which will be used as part of the index name. If the tag does not exist,
  ## the default tag value will be used.
  # index_name = "telegraf-{{host}}-%Y.%m.%d"
  # default_tag_value = "none"
  index_name = "telegraf-%Y.%m.%d" # required.

  ## Optional Index Config
  ## Set to true if Telegraf should use the "create" OpType while indexing
  # use_optype_create = false

  ## Optional TLS Config
  # tls_ca = "/etc/telegraf/ca.pem"
  # tls_cert = "/etc/telegraf/cert.pem"
  # tls_key = "/etc/telegraf/key.pem"
  ## Use TLS but skip chain & host verification
  # insecure_skip_verify = false

  ## Template Config
  ## Set to true if you want telegraf to manage its index template.
  ## If enabled it will create a recommended index template for telegraf indexes
  manage_template = true
  ## The template name used for telegraf indexes
  template_name = "telegraf"
  ## Set to true if you want telegraf to overwrite an existing template
  overwrite_template = false
  ## If set to true a unique ID hash will be sent as sha256(concat(timestamp,measurement,series-hash)) string
  ## it will enable data resend and update metric points avoiding duplicated metrics with different id's
  force_document_id = false

  ## Specifies the handling of NaN and Inf values.
  ## This option can have the following values:
  ##    none    -- do not modify field-values (default); will produce an error if NaNs or infs are encountered
  ##    drop    -- drop fields containing NaNs or infs
  ##    replace -- replace with the value in "float_replacement_value" (default: 0.0)
  ##               NaNs and inf will be replaced with the given number, -inf with the negative of that number
  # float_handling = "none"
  # float_replacement_value = 0.0

  ## Pipeline Config
  ## To use a ingest pipeline, set this to the name of the pipeline you want to use.
  # use_pipeline = "my_pipeline"
  ## Additionally, you can specify a tag name using the notation {{tag_name}}
  ## which will be used as part of the pipeline name. If the tag does not exist,
  ## the default pipeline will be used as the pipeline. If no default pipeline is set,
  ## no pipeline is used for the metric.
  # use_pipeline = "{{es_pipeline}}"
  # default_pipeline = "my_pipeline"
  #
  # Custom HTTP headers
  # To pass custom HTTP headers please define it in a given below section
  # [outputs.elasticsearch.headers]
  #    "X-Custom-Header" = "custom-value"

  ## Template Index Settings
  ## Overrides the template settings.index section with any provided options.
  ## Defaults provided here in the config
  # template_index_settings = {
  #   refresh_interval = "10s",
  #   mapping.total_fields.limit = 5000,
  #   auto_expand_replicas = "0-1",
  #   codec = "best_compression"
  # }

输入和输出集成示例

gNMI

  1. 监控 Cisco 设备:使用 gNMI 插件从 Cisco IOS XR、NX-OS 或 IOS XE 设备收集遥测数据以进行性能监控。

  2. 实时网络洞察:借助 gNMI 插件,网络管理员可以深入了解实时指标,例如接口统计信息和 CPU 使用率。

  3. 安全数据收集:配置具有 TLS 设置的 gNMI 插件,以确保在从设备收集敏感遥测数据时进行安全通信。

  4. 灵活的数据处理:使用订阅选项自定义您想要根据特定需求或要求收集的遥测数据。

  5. 错误处理:该插件包括故障排除选项,用于处理常见问题,例如缺少指标名称或 TLS 握手失败。

Elasticsearch

  1. 基于时间的索引:使用此插件将指标存储在 Elasticsearch 中,以根据收集时间为每个指标编制索引。例如,CPU 指标可以存储在名为 telegraf-2023.01.01 的每日索引中,从而可以轻松进行基于时间的查询和保留策略。

  2. 动态模板管理:利用模板管理功能自动创建针对您的指标定制的自定义模板。这使您可以定义如何索引和分析不同的字段,而无需手动配置 Elasticsearch,从而确保用于查询的最佳数据结构。

  3. OpenSearch 兼容性:如果您正在使用 AWS OpenSearch,则可以通过激活兼容模式来配置此插件以实现无缝协作,从而确保您现有的 Elasticsearch 客户端保持功能正常并与较新的集群设置兼容。

反馈

感谢您成为我们社区的一份子!如果您有任何一般性反馈或在这些页面上发现任何错误,我们欢迎并鼓励您提出意见。请在 InfluxDB 社区 Slack 中提交您的反馈。

强大的性能,无限的扩展能力

收集、组织和处理海量高速数据。当您将任何数据视为时间序列数据时,它都会更有价值。借助 InfluxDB,这个排名第一的时间序列平台旨在与 Telegraf 一起扩展。

查看入门方法

相关集成

HTTP 和 InfluxDB 集成

HTTP 插件从一个或多个 HTTP(S) 端点收集指标。它支持各种身份验证方法和数据格式的配置选项。

查看集成

Kafka 和 InfluxDB 集成

此插件从 Kafka 读取消息,并允许根据这些消息创建指标。它支持各种配置,包括不同的 Kafka 设置和消息处理选项。

查看集成

Kinesis 和 InfluxDB 集成

Kinesis 插件允许从 AWS Kinesis 流中读取指标。它支持多种输入数据格式,并提供带有 DynamoDB 的检查点功能,以实现可靠的消息处理。

查看集成