社区亮点:如何构建InfluxDB模板
作者:社区 / 产品, 用例, 开发者
2020年11月13日
导航至
本文由InfluxAce Ignacio Van Droogenbroeck撰写。
Ignacio是一位来自乌拉圭的技术市场营销工程师。他大约十年前开始写博客,并撰写有关IT基础设施、云、Docker、Linux和可观察性的文章。
在过去几个月里,我一直在积极为InfluxDB 2.0构建InfluxDB模板,为专门用于指标收集和存储的时序数据库InfluxDB做出贡献。让我们一起来构建一个用于监控PostgreSQL的InfluxDB模板!????????
在本教程中,我将向您展示如何创建一个适用于InfluxDB 2.0的InfluxDB模板。“InfluxDB模板”不仅包括仪表板,还包括其他元素,如“标签”、“存储桶”(保存信息的地方)和Telegraf配置。
您可以将这篇教程作为创建其他InfluxDB模板和为此项目做出贡献的指南。让我们开始吧!
构建InfluxDB模板的要求
创建InfluxDB模板的第一步之一是查看所有可用的Telegraf输入插件。利用这个列表作为跳板,我可以缩小InfluxDB模板候选的范围。在这个场合,我选择构建一个针对PostgreSQL的InfluxDB模板。您可以在GitHub上找到PostgreSQL数据库的Telegraf插件。
构建InfluxDB模板的另一个要求是拥有一个运行的InfluxDB 2.0实例。例如,我有一个专用的Docker容器来实验和构建新的InfluxDB模板。
设置InfluxDB 2.0和Telegraf
创建InfluxDB模板的主要原因之一是了解软件如何公开度量指标。在这种情况下,我深入研究PostgreSQL文档。它很容易理解,并且很容易开始监控数据库。如您从下面的Telegraf输入中看到的,它非常容易理解其工作原理。
接下来,我在InfluxDB中创建一个“存储桶”来存储这些PostgreSQL指标。我只是打开我的实例并导航到“加载数据/存储桶”,创建一个名为‘postgres’的存储桶。我还为该资源设置了相同的标签(postgres)。
以下是使用Postgres插件的Telegraf配置。您可以从InfluxDB GUI获取输出配置。
[[outputs.influxdb_v2]]
## The URLs of the InfluxDB cluster nodes.
##
## Multiple URLs can be specified for a single cluster, only ONE of the
## urls will be written to each interval.
## urls exp: http://127.0.0.1:9999
urls = ["https://127.0.0.1:9999"]
## Token for authentication.
token = "$INFLUX_TOKEN"
## Organization is the name of the organization you wish to write to; must exist.
organization = "$INFLUX_ORG"
## Destination bucket to write into.
bucket = "postgres"
[agent]
interval = "1m"
[[inputs.postgresql]]
address = "postgres://postgres:mysecretpassword@localhost:5432"
ignored_databases = ["template0", "template1"]
在运行使用此配置文件的Telegraf之前,我们需要传递一些数据作为变量——在这种情况下,是令牌和组织。您可以使用以下示例这样做
export INFLUX_TOKEN='your-token'
export INFLUX_ORG='your-organization'
完成这些后,我们就可以运行Telegraf了! ????
使用InfluxDB监控PostgreSQL
让我们通过以“开启”调试模式运行Telegraf来开始监控我们的数据库,以了解正在发生的事情。
telegraf --config psql.conf --debug
结果是类似这样的
2020-06-12T14:38:35Z I! Starting Telegraf 1.14.3
2020-06-12T14:38:35Z I! Loaded inputs: postgresql
2020-06-12T14:38:35Z I! Loaded aggregators:
2020-06-12T14:38:35Z I! Loaded processors:
2020-06-12T14:38:35Z I! Loaded outputs: influxdb_v2
2020-06-12T14:38:35Z I! Tags enabled: host=thelab
2020-06-12T14:38:35Z I! [agent] Config: Interval:1m0s, Quiet:false, Hostname:"thelab", Flush Interval:10s
2020-06-12T14:38:35Z D! [agent] Initializing plugins
2020-06-12T14:38:35Z D! [agent] Connecting outputs
2020-06-12T14:38:35Z D! [agent] Attempting connection to [outputs.influxdb_v2]
2020-06-12T14:38:35Z D! [agent] Successfully connected to outputs.influxdb_v2
2020-06-12T14:38:35Z D! [agent] Starting service inputs
2020-06-12T14:38:50Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2020-06-12T14:39:00Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2020-06-12T14:39:10Z D! [outputs.influxdb_v2] Wrote batch of 2 metrics in 145.099172ms
如您所见,Telegraf正在运行,并开始将指标发送到InfluxDB。几分钟后,您可以去InfluxDB GUI,特别是数据探索器,选择存储桶“postgres”,并看到度量值和字段。
成功了——一切都在顺利进行,数据正在被收集。 ????????
可视化数据
我们准备好创建仪表板并消费数据了。首先,我转到仪表板并点击创建仪表板。之后,我命名并分配一个标签。最好使用我们之前使用的相同名称——在这个例子中,是postgres
标签。
接下来,我开始在InfluxDB中现有的数据上操作。我还看了看其他创建的仪表板。例如,我检查Grafana仪表板并从中获得灵感。完成的仪表板应看起来像这样
这一步非常重要,因为我们需要创建一个对社区有帮助的仪表板。如果您创建了一个顺序随机的“随机”数据的图形,仪表板将不会被使用。
现在,让我们进入有趣的部分,将这个模板打包起来发送吧!
打包 InfluxDB 模板
当我说到打包或导出模板时,不仅是指仪表板,还包括Bucket、Telegraf配置和标签。
现在我们已经在InfluxDB中有仪表板、标签和Bucket,但我们的Telegraf配置在单独的文件中。我们需要将它们合并到一个非常棒的YAML文件中。
因为我是在Docker容器中运行InfluxDB,所以我需要运行下面的导出命令。(记住:在这个过程中我们分配给资源的标签非常有意义。)
docker exec -it 4d410b0f82ba influx pkg export all --filter labelName=postgres -f postgres.yml -o $INFLUX_ORG -t $INFLUX_TOKEN
接下来,我需要将文件从我的容器中拉到我的机器上,开始编辑并合并Telegraf配置
docker cp 4d410b0f82ba:/postgres.yml
我刚刚导出的文件看起来像这样(这是其中的一部分)
apiVersion: influxdata.com/v2alpha1
kind: Label
metadata:
name: flamboyant-dubinsky-332001
spec:
color: '#F95F53'
name: postgres
---
apiVersion: influxdata.com/v2alpha1
kind: Bucket
metadata:
name: vivid-heisenberg-732003
spec:
associations:
- kind: Label
name: flamboyant-dubinsky-332001
name: postgres
---
apiVersion: influxdata.com/v2alpha1
kind: Dashboard
metadata:
name: jovial-montalcini-f32001
spec:
associations:
- kind: Label
name: flamboyant-dubinsky-332001
charts:
- colors:
- hex: '#00C9FF'
name: laser
type: min
- hex: '#9394FF'
name: comet
type: max
value: 100
decimalPlaces: 2
height: 3
kind: Gauge
name: Current IOwait
queries:
- query: |-
from(bucket: "postgres")
|> range(start: v.timeRangeStart, stop: v.timeRangeStop)
|> filter(fn: (r) => r["_measurement"] == "cpu")
|> filter(fn: (r) => r["_field"] == "usage_iowait")
在文件的末尾,我们将添加经过一些调整的Telegraf配置。记住,这是一个YAML文件,所以缩进非常重要。
Telegraf配置部分看起来像这样
apiVersion: influxdata.com/v2alpha1
kind: Telegraf
metadata:
name: postgres-config
spec:
config: |
[[outputs.influxdb_v2]]
## The URLs of the InfluxDB cluster nodes.
##
## Multiple URLs can be specified for a single cluster, only ONE of the
## urls will be written to each interval.
## urls exp: http://127.0.0.1:9999
urls = ["$INFLUX_HOST"]
## Token for authentication.
token = "$INFLUX_TOKEN"
## Organization is the name of the organization you wish to write to; must exist.
organization = "$INFLUX_ORG"
## Destination bucket to write into.
bucket = "postgres"
[agent]
interval = "1m"
[[inputs.postgresql]]
# address = "postgres://postgres:mysecretpassword@localhost:5432"
address = "$PSQL_STRING_CONNECTION"
ignored_databases = ["template0", "template1"]
[[inputs.cpu]]
## Whether to report per-cpu stats or not
percpu = true
## Whether to report total system cpu stats or not
totalcpu = true
## If true, collect raw CPU time metrics.
collect_cpu_time = false
## If true, compute and report the sum of all non-idle CPU states.
report_active = false
[[inputs.disk]]
## By default stats will be gathered for all mount points.
## Set mount_points will restrict the stats to only the specified mount points.
# mount_points = ["/"]
## Ignore mount points by filesystem type.
ignore_fs = ["tmpfs", "devtmpfs", "devfs", "iso9660", "overlay", "aufs", "squashfs"]
# Read metrics about disk IO by device
[[inputs.diskio]]
## By default, telegraf will gather stats for all devices including
## disk partitions.
## Setting devices will restrict the stats to the specified devices.
# devices = ["sda", "sdb"]
## Uncomment the following line if you need disk serial numbers.
# skip_serial_number = false
#
## On systems which support it, device metadata can be added in the form of
## tags.
## Currently only Linux is supported via udev properties. You can view
## available properties for a device by running:
## 'udevadm info -q property -n /dev/sda'
## Note: Most, but not all, udev properties can be accessed this way. Properties
## that are currently inaccessible include DEVTYPE, DEVNAME, and DEVPATH.
# device_tags = ["ID_FS_TYPE", "ID_FS_USAGE"]
#
## Using the same metadata source as device_tags, you can also customize the
## name of the device via templates.
## The 'name_templates' parameter is a list of templates to try and apply to
## the device. The template may contain variables in the form of '$PROPERTY' or
## '${PROPERTY}'. The first template which does not contain any variables not
## present for the device is used as the device name tag.
## The typical use case is for LVM volumes, to get the VG/LV name instead of
## the near-meaningless DM-0 name.
# name_templates = ["$ID_FS_LABEL","$DM_VG_NAME/$DM_LV_NAME"]
# Read metrics about memory usage
[[inputs.mem]]
# no configuration
注意:我将PostgreSQL字符串连接转换为变量。凭证最初是硬编码在文件中的,但现在可以通过环境变量设置。
$ export PSQL_STRING_CONNECTION=postgres://postgres:mysecretpassword@localhost:5432
保存文件后,我们可以测试导入过程是否顺利,以及InfluxDB模板是否准备好发送。首先,我将新的YAML文件复制到容器中。
docker cp postgres.yml 4d410b0f82ba:/
然后我运行导入过程
docker exec -it 4d410b0f82ba influx pkg -f postgres.yml -o $INFLUX_ORG -t $INFLUX_TOKEN
一切设置好后,终端的响应应该像这样
LABELS +add | -remove | unchanged
+-----+----------------------------+------------------+---------------+---------+-------------+
| +/- | PACKAGE NAME | ID | RESOURCE NAME | COLOR | DESCRIPTION |
+-----+----------------------------+------------------+---------------+---------+-------------+
| | flamboyant-dubinsky-332001 | 05d639e9629a9000 | postgres | #F95F53 | |
+-----+----------------------------+------------------+---------------+---------+-------------+
| TOTAL | 1 |
+-----+----------------------------+------------------+---------------+---------+-------------+
BUCKETS +add | -remove | unchanged
+-----+-------------------------+------------------+---------------+------------------+-------------+
| +/- | PACKAGE NAME | ID | RESOURCE NAME | RETENTION PERIOD | DESCRIPTION |
+-----+-------------------------+------------------+---------------+------------------+-------------+
| | vivid-heisenberg-732003 | 05d639b5fdf32000 | postgres | 0s | |
+-----+-------------------------+------------------+---------------+------------------+-------------+
| TOTAL | 1 |
+-----+-------------------------+------------------+---------------+------------------+-------------+
DASHBOARDS +add | -remove | unchanged
+-----+--------------------------+----+---------------+-------------+------------+
| +/- | PACKAGE NAME | ID | RESOURCE NAME | DESCRIPTION | NUM CHARTS |
+-----+--------------------------+----+---------------+-------------+------------+
| + | jovial-montalcini-f32001 | | Postgres | | 17 |
+-----+--------------------------+----+---------------+-------------+------------+
| TOTAL | 1 |
+-----+--------------------------+----+---------------+-------------+------------+
TELEGRAF CONFIGURATIONS +add | -remove | unchanged
+-----+-----------------+----+-----------------+-------------+
| +/- | PACKAGE NAME | ID | RESOURCE NAME | DESCRIPTION |
+-----+-----------------+----+-----------------+-------------+
| + | postgres-config | | postgres-config | |
+-----+-----------------+----+-----------------+-------------+
| TOTAL | 1 |
+-----+-----------------+----+-----------------+-------------+
LABEL ASSOCIATIONS +add | -remove | unchanged
+-----+---------------+--------------------------+---------------+------------------+----------------------------+------------+------------------+
| +/- | RESOURCE TYPE | RESOURCE PACKAGE NAME | RESOURCE NAME | RESOURCE ID | LABEL PACKAGE NAME | LABEL NAME | LABEL ID |
+-----+---------------+--------------------------+---------------+------------------+----------------------------+------------+------------------+
| | buckets | vivid-heisenberg-732003 | postgres | 05d639b5fdf32000 | flamboyant-dubinsky-332001 | postgres | 05d639e9629a9000 |
+-----+---------------+--------------------------+---------------+------------------+----------------------------+------------+------------------+
+-----+---------------+--------------------------+---------------+------------------+----------------------------+------------+------------------+
| + | dashboards | jovial-montalcini-f32001 | Postgres | | flamboyant-dubinsky-332001 | postgres | 05d639e9629a9000 |
+-----+---------------+--------------------------+---------------+------------------+----------------------------+------------+------------------+
| TOTAL | 2 |
+-----+---------------+--------------------------+---------------+------------------+----------------------------+------------+------------------+
如你所见,导入过程进行了一些检查并识别了我们的模板——我们的PostgreSQL InfluxDB模板准备好了!???
准备好构建你自己的InfluxDB模板了吗?
我发现构建InfluxDB模板很有趣且令人兴奋,我很乐意分享制作一个模板的过程。我希望这篇文章对您有帮助——我迫不及待地想看看社区能构建出什么!
想为InfluxDB模板做出贡献?请查看这个GitHub仓库并阅读贡献指南。点击这里了解已经开发的其他InfluxDB模板。
如果您有任何问题,我很乐意帮助。在InfluxDB社区Slack工作空间@Ignacio Van Droogenbroeck找到我。务必查看#Templates频道——来打个招呼!或者如果你在使用这个仪表板,请告诉我Twitter。
其他资源
- 如何为InfluxDB启动Docker实例(西班牙语)
- 如何使用InfluxDB 2.0 Beta监控Linux(西班牙语)
- 使用Telegraf和InfluxDB在南美洲追踪COVID-19数据
- 还没准备好尝试InfluxDB 2.0?尝试InfluxDB v1.8、Chronograf和Kapacitor - TICK堆栈(西班牙语)