社区亮点:如何构建InfluxDB模板

导航至

InfluxDB社区成员Ignacio Van Droogenbroeck本文由InfluxAce Ignacio Van Droogenbroeck撰写。

Ignacio是一位来自乌拉圭的技术市场营销工程师。他大约十年前开始写博客,并撰写有关IT基础设施、云、Docker、Linux和可观察性的文章。

在过去几个月里,我一直在积极为InfluxDB 2.0构建InfluxDB模板,为专门用于指标收集和存储的时序数据库InfluxDB做出贡献。让我们一起来构建一个用于监控PostgreSQL的InfluxDB模板!????????

在本教程中,我将向您展示如何创建一个适用于InfluxDB 2.0的InfluxDB模板。“InfluxDB模板”不仅包括仪表板,还包括其他元素,如“标签”、“存储桶”(保存信息的地方)和Telegraf配置。

您可以将这篇教程作为创建其他InfluxDB模板和为此项目做出贡献的指南。让我们开始吧!

构建InfluxDB模板的要求

创建InfluxDB模板的第一步之一是查看所有可用的Telegraf输入插件。利用这个列表作为跳板,我可以缩小InfluxDB模板候选的范围。在这个场合,我选择构建一个针对PostgreSQL的InfluxDB模板。您可以在GitHub上找到PostgreSQL数据库的Telegraf插件。

构建InfluxDB模板的另一个要求是拥有一个运行的InfluxDB 2.0实例。例如,我有一个专用的Docker容器来实验和构建新的InfluxDB模板。

设置InfluxDB 2.0和Telegraf

创建InfluxDB模板的主要原因之一是了解软件如何公开度量指标。在这种情况下,我深入研究PostgreSQL文档。它很容易理解,并且很容易开始监控数据库。如您从下面的Telegraf输入中看到的,它非常容易理解其工作原理。

接下来,我在InfluxDB中创建一个“存储桶”来存储这些PostgreSQL指标。我只是打开我的实例并导航到“加载数据/存储桶”,创建一个名为‘postgres’的存储桶。我还为该资源设置了相同的标签(postgres)。

以下是使用Postgres插件的Telegraf配置。您可以从InfluxDB GUI获取输出配置。

[[outputs.influxdb_v2]]
  ## The URLs of the InfluxDB cluster nodes.
  ##
  ## Multiple URLs can be specified for a single cluster, only ONE of the
  ## urls will be written to each interval.
  ## urls exp: http://127.0.0.1:9999
  urls = ["https://127.0.0.1:9999"]

  ## Token for authentication.
  token = "$INFLUX_TOKEN"

  ## Organization is the name of the organization you wish to write to; must exist.
  organization = "$INFLUX_ORG"

  ## Destination bucket to write into.
  bucket = "postgres"

  [agent]
  interval = "1m"

 [[inputs.postgresql]]
  address = "postgres://postgres:mysecretpassword@localhost:5432"
  ignored_databases = ["template0", "template1"]

在运行使用此配置文件的Telegraf之前,我们需要传递一些数据作为变量——在这种情况下,是令牌组织。您可以使用以下示例这样做

export INFLUX_TOKEN='your-token'

export INFLUX_ORG='your-organization'

完成这些后,我们就可以运行Telegraf了! ????

使用InfluxDB监控PostgreSQL

让我们通过以“开启”调试模式运行Telegraf来开始监控我们的数据库,以了解正在发生的事情。

telegraf --config psql.conf --debug

结果是类似这样的

2020-06-12T14:38:35Z I! Starting Telegraf 1.14.3
2020-06-12T14:38:35Z I! Loaded inputs: postgresql
2020-06-12T14:38:35Z I! Loaded aggregators: 
2020-06-12T14:38:35Z I! Loaded processors: 
2020-06-12T14:38:35Z I! Loaded outputs: influxdb_v2
2020-06-12T14:38:35Z I! Tags enabled: host=thelab
2020-06-12T14:38:35Z I! [agent] Config: Interval:1m0s, Quiet:false, Hostname:"thelab", Flush Interval:10s
2020-06-12T14:38:35Z D! [agent] Initializing plugins
2020-06-12T14:38:35Z D! [agent] Connecting outputs
2020-06-12T14:38:35Z D! [agent] Attempting connection to [outputs.influxdb_v2]
2020-06-12T14:38:35Z D! [agent] Successfully connected to outputs.influxdb_v2
2020-06-12T14:38:35Z D! [agent] Starting service inputs
2020-06-12T14:38:50Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2020-06-12T14:39:00Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2020-06-12T14:39:10Z D! [outputs.influxdb_v2] Wrote batch of 2 metrics in 145.099172ms

如您所见,Telegraf正在运行,并开始将指标发送到InfluxDB。几分钟后,您可以去InfluxDB GUI,特别是数据探索器,选择存储桶“postgres”,并看到度量值和字段。

成功了——一切都在顺利进行,数据正在被收集。 ????????

可视化数据

我们准备好创建仪表板并消费数据了。首先,我转到仪表板并点击创建仪表板。之后,我命名并分配一个标签。最好使用我们之前使用的相同名称——在这个例子中,是postgres 标签。

接下来,我开始在InfluxDB中现有的数据上操作。我还看了看其他创建的仪表板。例如,我检查Grafana仪表板并从中获得灵感。完成的仪表板应看起来像这样

InfluxAce Ignacio Van Droogenbroeck shows how to build an InfluxDB Template

这一步非常重要,因为我们需要创建一个对社区有帮助的仪表板。如果您创建了一个顺序随机的“随机”数据的图形,仪表板将不会被使用。

现在,让我们进入有趣的部分,将这个模板打包起来发送吧!

打包 InfluxDB 模板

当我说到打包或导出模板时,不仅是指仪表板,还包括Bucket、Telegraf配置和标签。

现在我们已经在InfluxDB中有仪表板、标签和Bucket,但我们的Telegraf配置在单独的文件中。我们需要将它们合并到一个非常棒的YAML文件中。

因为我是在Docker容器中运行InfluxDB,所以我需要运行下面的导出命令。(记住:在这个过程中我们分配给资源的标签非常有意义。)

docker exec -it 4d410b0f82ba influx pkg export all --filter labelName=postgres -f postgres.yml -o $INFLUX_ORG -t $INFLUX_TOKEN

接下来,我需要将文件从我的容器中拉到我的机器上,开始编辑并合并Telegraf配置

docker cp 4d410b0f82ba:/postgres.yml

我刚刚导出的文件看起来像这样(这是其中的一部分)

apiVersion: influxdata.com/v2alpha1
kind: Label
metadata:
    name: flamboyant-dubinsky-332001
spec:
    color: '#F95F53'
    name: postgres
---
apiVersion: influxdata.com/v2alpha1
kind: Bucket
metadata:
    name: vivid-heisenberg-732003
spec:
    associations:
      - kind: Label
        name: flamboyant-dubinsky-332001
    name: postgres
---
apiVersion: influxdata.com/v2alpha1
kind: Dashboard
metadata:
    name: jovial-montalcini-f32001
spec:
    associations:
      - kind: Label
        name: flamboyant-dubinsky-332001
    charts:
      - colors:
          - hex: '#00C9FF'
            name: laser
            type: min
          - hex: '#9394FF'
            name: comet
            type: max
            value: 100
        decimalPlaces: 2
        height: 3
        kind: Gauge
        name: Current IOwait
        queries:
          - query: |-
                from(bucket: "postgres")
                  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
                  |> filter(fn: (r) => r["_measurement"] == "cpu")
                  |> filter(fn: (r) => r["_field"] == "usage_iowait")

在文件的末尾,我们将添加经过一些调整的Telegraf配置。记住,这是一个YAML文件,所以缩进非常重要。

Telegraf配置部分看起来像这样

apiVersion: influxdata.com/v2alpha1
kind: Telegraf
metadata:
    name: postgres-config
spec:
    config: |

        [[outputs.influxdb_v2]]
          ## The URLs of the InfluxDB cluster nodes.
          ##
          ## Multiple URLs can be specified for a single cluster, only ONE of the
          ## urls will be written to each interval.
          ## urls exp: http://127.0.0.1:9999
          urls = ["$INFLUX_HOST"]
        
          ## Token for authentication.
          token = "$INFLUX_TOKEN"
        
          ## Organization is the name of the organization you wish to write to; must exist.
          organization = "$INFLUX_ORG"
        
          ## Destination bucket to write into.
          bucket = "postgres"
              
          [agent]
          interval = "1m"
          
        [[inputs.postgresql]]
          # address = "postgres://postgres:mysecretpassword@localhost:5432"
          address = "$PSQL_STRING_CONNECTION"
          ignored_databases = ["template0", "template1"]
        
        [[inputs.cpu]]
          ## Whether to report per-cpu stats or not
          percpu = true
          ## Whether to report total system cpu stats or not
          totalcpu = true
          ## If true, collect raw CPU time metrics.
          collect_cpu_time = false
          ## If true, compute and report the sum of all non-idle CPU states.
          report_active = false
        
        [[inputs.disk]]
          ## By default stats will be gathered for all mount points.
          ## Set mount_points will restrict the stats to only the specified mount points.
          # mount_points = ["/"]
        
          ## Ignore mount points by filesystem type.
          ignore_fs = ["tmpfs", "devtmpfs", "devfs", "iso9660", "overlay", "aufs", "squashfs"]
        
        # Read metrics about disk IO by device
        [[inputs.diskio]]
          ## By default, telegraf will gather stats for all devices including
          ## disk partitions.
          ## Setting devices will restrict the stats to the specified devices.
          # devices = ["sda", "sdb"]
          ## Uncomment the following line if you need disk serial numbers.
          # skip_serial_number = false
          #
          ## On systems which support it, device metadata can be added in the form of
          ## tags.
          ## Currently only Linux is supported via udev properties. You can view
          ## available properties for a device by running:
          ## 'udevadm info -q property -n /dev/sda'
          ## Note: Most, but not all, udev properties can be accessed this way. Properties
          ## that are currently inaccessible include DEVTYPE, DEVNAME, and DEVPATH.
          # device_tags = ["ID_FS_TYPE", "ID_FS_USAGE"]
          #
          ## Using the same metadata source as device_tags, you can also customize the
          ## name of the device via templates.
          ## The 'name_templates' parameter is a list of templates to try and apply to
          ## the device. The template may contain variables in the form of '$PROPERTY' or
          ## '${PROPERTY}'. The first template which does not contain any variables not
          ## present for the device is used as the device name tag.
          ## The typical use case is for LVM volumes, to get the VG/LV name instead of
          ## the near-meaningless DM-0 name.
          # name_templates = ["$ID_FS_LABEL","$DM_VG_NAME/$DM_LV_NAME"]
        
        # Read metrics about memory usage
        [[inputs.mem]]
          # no configuration

注意:我将PostgreSQL字符串连接转换为变量。凭证最初是硬编码在文件中的,但现在可以通过环境变量设置。

$ export PSQL_STRING_CONNECTION=postgres://postgres:mysecretpassword@localhost:5432

保存文件后,我们可以测试导入过程是否顺利,以及InfluxDB模板是否准备好发送。首先,我将新的YAML文件复制到容器中。

docker cp postgres.yml 4d410b0f82ba:/

然后我运行导入过程

docker exec -it 4d410b0f82ba influx pkg -f postgres.yml -o $INFLUX_ORG -t $INFLUX_TOKEN

一切设置好后,终端的响应应该像这样

LABELS    +add | -remove | unchanged
+-----+----------------------------+------------------+---------------+---------+-------------+
| +/- |        PACKAGE NAME        |        ID        | RESOURCE NAME |  COLOR  | DESCRIPTION |
+-----+----------------------------+------------------+---------------+---------+-------------+
|     | flamboyant-dubinsky-332001 | 05d639e9629a9000 | postgres      | #F95F53 |             |
+-----+----------------------------+------------------+---------------+---------+-------------+
|                                                                        TOTAL  |      1      |
+-----+----------------------------+------------------+---------------+---------+-------------+

BUCKETS    +add | -remove | unchanged
+-----+-------------------------+------------------+---------------+------------------+-------------+
| +/- |      PACKAGE NAME       |        ID        | RESOURCE NAME | RETENTION PERIOD | DESCRIPTION |
+-----+-------------------------+------------------+---------------+------------------+-------------+
|     | vivid-heisenberg-732003 | 05d639b5fdf32000 | postgres      | 0s               |             |
+-----+-------------------------+------------------+---------------+------------------+-------------+
|                                                                         TOTAL       |      1      |
+-----+-------------------------+------------------+---------------+------------------+-------------+

DASHBOARDS    +add | -remove | unchanged
+-----+--------------------------+----+---------------+-------------+------------+
| +/- |       PACKAGE NAME       | ID | RESOURCE NAME | DESCRIPTION | NUM CHARTS |
+-----+--------------------------+----+---------------+-------------+------------+
| +   | jovial-montalcini-f32001 |    | Postgres      |             | 17         |
+-----+--------------------------+----+---------------+-------------+------------+
|                                                          TOTAL    |     1      |
+-----+--------------------------+----+---------------+-------------+------------+

TELEGRAF CONFIGURATIONS    +add | -remove | unchanged
+-----+-----------------+----+-----------------+-------------+
| +/- |  PACKAGE NAME   | ID |  RESOURCE NAME  | DESCRIPTION |
+-----+-----------------+----+-----------------+-------------+
| +   | postgres-config |    | postgres-config |             |
+-----+-----------------+----+-----------------+-------------+
|                                   TOTAL      |      1      |
+-----+-----------------+----+-----------------+-------------+

LABEL ASSOCIATIONS    +add | -remove | unchanged
+-----+---------------+--------------------------+---------------+------------------+----------------------------+------------+------------------+
| +/- | RESOURCE TYPE |  RESOURCE PACKAGE NAME   | RESOURCE NAME |   RESOURCE ID    |     LABEL PACKAGE NAME     | LABEL NAME |     LABEL ID     |
+-----+---------------+--------------------------+---------------+------------------+----------------------------+------------+------------------+
|     | buckets       | vivid-heisenberg-732003  | postgres      | 05d639b5fdf32000 | flamboyant-dubinsky-332001 | postgres   | 05d639e9629a9000 |
+-----+---------------+--------------------------+---------------+------------------+----------------------------+------------+------------------+
+-----+---------------+--------------------------+---------------+------------------+----------------------------+------------+------------------+
| +   | dashboards    | jovial-montalcini-f32001 | Postgres      |                  | flamboyant-dubinsky-332001 | postgres   | 05d639e9629a9000 |
+-----+---------------+--------------------------+---------------+------------------+----------------------------+------------+------------------+
|                                                                                                                    TOTAL    |        2         |
+-----+---------------+--------------------------+---------------+------------------+----------------------------+------------+------------------+

如你所见,导入过程进行了一些检查并识别了我们的模板——我们的PostgreSQL InfluxDB模板准备好了!???

准备好构建你自己的InfluxDB模板了吗?

我发现构建InfluxDB模板很有趣且令人兴奋,我很乐意分享制作一个模板的过程。我希望这篇文章对您有帮助——我迫不及待地想看看社区能构建出什么!

想为InfluxDB模板做出贡献?请查看这个GitHub仓库并阅读贡献指南。点击这里了解已经开发的其他InfluxDB模板。

如果您有任何问题,我很乐意帮助。在InfluxDB社区Slack工作空间@Ignacio Van Droogenbroeck找到我。务必查看#Templates频道——来打个招呼!或者如果你在使用这个仪表板,请告诉我Twitter

其他资源