Prometheus 监控k8s的Service和ingress

作者: root007 分类: kubernetes,prometheus 发布时间: 2019-03-07 09:25

首先需要安装 blackbox-exporter
Blackbox Exporter是Prometheus社区提供的官方黑盒监控解决方案,其允许用户通过:HTTP、HTTPS、DNS、TCP以及ICMP的方式对网络进行探测

k8s部署 blackbox-exporter :

piVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: prometheus-blackbox-exporter
  namespace: monitoring
spec:
  selector:
    matchLabels:
      app: prometheus-blackbox-exporter
  replicas: 1
  template:
    metadata:
      labels:
        app: prometheus-blackbox-exporter
    spec:
      restartPolicy: Always
      containers:
      - name: prometheus-blackbox-exporter
        image: prom/blackbox-exporter:v0.12.0
        imagePullPolicy: IfNotPresent
        ports:
        - name: blackbox-port
          containerPort: 9115
        readinessProbe:
          tcpSocket:
            port: 9115
          initialDelaySeconds: 5
          timeoutSeconds: 5
        resources:
          requests:
            memory: 50Mi
            cpu: 100m
          limits:
            memory: 60Mi
            cpu: 200m
        volumeMounts:
        - name: config
          mountPath: /etc/blackbox_exporter
        args:
        - --config.file=/etc/blackbox_exporter/blackbox.yml
        - --log.level=debug
        - --web.listen-address=:9115
      volumes:
      - name: config
        configMap:
          name: prometheus-blackbox-exporter
      nodeSelector:
        prometheus: "core"
      tolerations:
      - key: "node-role.kubernetes.io/master"
        effect: "NoSchedule"
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: prometheus-blackbox-exporter
  name: blackbox-exporter
  namespace: monitoring
  annotations:
    prometheus.io/scrape: 'true'
spec:
  type: NodePort
  selector:
    app: prometheus-blackbox-exporter
  ports:
  - name: blackbox
    port: 9115
    targetPort: 9115
    protocol: TCP
---
apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    app: prometheus-blackbox-exporter
  name: prometheus-blackbox-exporter
  namespace: monitoring
data:
  blackbox.yml: |-
    modules:
      http_2xx:
        prober: http
        timeout: 30s
        http:
          valid_http_versions: ["HTTP/1.1", "HTTP/2"]
          valid_status_codes: [200,302,301,401,404]
          method: GET
          preferred_ip_protocol: "ip4"

prometheus配置新增Service和ingress的自动发现job

- job_name: 'kubernetes-services'

  metrics_path: /probe
  params:
    module: [http_2xx]

  kubernetes_sd_configs:
  - role: service

  relabel_configs:
  # Example relabel to probe only some services that have "example.io/should_be_probed = true" annotation
  #  - source_labels: [__meta_kubernetes_service_annotation_example_io_should_be_probed]
  #    action: keep
  #    regex: true
  - source_labels: [__address__]
    target_label: __param_target
  - target_label: __address__
    replacement: blackbox-exporter:9115
  - source_labels: [__param_target]
    target_label: instance
  - action: labelmap
    regex: __meta_kubernetes_service_label_(.+)
  - source_labels: [__meta_kubernetes_namespace]
    target_label: kubernetes_namespace
  - source_labels: [__meta_kubernetes_service_name]
    target_label: kubernetes_name

# Example scrape config for probing ingresses via the Blackbox Exporter.
#
# The relabeling allows the actual ingress scrape endpoint to be configured
# for all or only some services.
- job_name: 'kubernetes-ingresses'

  metrics_path: /probe
  params:
    module: [http_2xx]

  kubernetes_sd_configs:
  - role: ingress

  relabel_configs:
  # Example relabel to probe only some ingresses that have "example.io/should_be_probed = true" annotation
  #  - source_labels: [__meta_kubernetes_ingress_annotation_example_io_should_be_probed]
  #    action: keep
  #    regex: true
  - source_labels: [__meta_kubernetes_ingress_scheme,__address__,__meta_kubernetes_ingress_path]
    regex: (.+);(.+);(.+)
    replacement: ${1}://${2}${3}
    target_label: __param_target
  - target_label: __address__
    replacement: blackbox-exporter:9115
  - source_labels: [__param_target]
    target_label: instance
  - action: labelmap
    regex: __meta_kubernetes_ingress_label_(.+)
  - source_labels: [__meta_kubernetes_namespace]
    target_label: kubernetes_namespace
  - source_labels: [__meta_kubernetes_ingress_name]
    target_label: kubernetes_nam
service-alerts
groups:
- name: service-alerts
  rules:
  - alert: ServiceCheckFailed
    expr: probe_success{job="kubernetes-services"} < 1
    for: 1m
    labels:
      severity: critical
    annotations:
      description: '{{$labels.kubernetes_name}}  in {{$labels.kubernetes_namespace}}
        failed health check'
      summary: '{{$labels.kubernetes_name | toUpper }} Health Check Failing. in Kubernetes
        Namespace {{$labels.kubernetes_namespace | toUpper}}'

发表评论

邮箱地址不会被公开。 必填项已用*标注


  • http://ediconcept.odns.fr/maillotdefootpascher.html
  • http://ediconcept.odns.fr/maillotsfootpascher.html
  • http://ediconcept.odns.fr/maillotsfootpascher2022.html
  • https://sergiofajardo.co/maillotdefootpascher.html
  • https://sergiofajardo.co/maillotsfootballpascher.html
  • https://sergiofajardo.co/maillotsfootpascherfrance.html
  • https://allphonetoys.com/maillotdefootpascher.html
  • https://allphonetoys.com/nouveaumaillotdefootpascher.html
  • https://allphonetoys.com/nouveaumaillotsdefootball.html
  • http://samosystem.nichost.ru/maillotdefootpascher.html
  • http://samosystem.nichost.ru/nouveaumodèlemaillotdefoot.html
  • http://samosystem.nichost.ru/pascherfemmesfootballjersey.html
  • http://w96397on.beget.tech/maillotdefootpascher.html
  • http://w96397on.beget.tech/pascherfootballt-shirts.html
  • http://w96397on.beget.tech/paschermaillotsdefootpromo.html
  • http://wesleywang.acsite.org/maillotdefootpascher.html
  • https://wesleywang.acsite.org/repliquemaillotsdefootballlivraisongratuite.html
  • https://wesleywang.acsite.org/retrovintagesoccerjerseys.html
  • https://www.rnkhabri.com/maillotdefootpascher.html
  • https://www.rnkhabri.com/personnaliséenfantssoccerjersey.html
  • https://www.rnkhabri.com/personnaliséfemmessoccervêtements.html
  • https://www.oujdacity.net/maillotdefootpascher.html
  • https://www.oujdacity.net/sitedemaillotdefootpascher.html
  • https://www.oujdacity.net/survetementdefoot.html
  • https://infendo.com/maillotdefootpascher.html
  • https://infendo.com/survetementdefootpascher.html
  • https://infendo.com/uniformedefootballdesjeunes.html
  • https://planitagency.com/maillotdefootpascher.html
  • https://planitagency.com/ventemaillotdefootpascherthailande2022.html
  • https://planitagency.com/vesteequipedefoot.html
  • http://immergut.com.br
  • https://immobiliarelerosemarine.com
  • https://immobilienfotografie24.de
  • http://immobilier-lemaroc.com
  • https://imobilideranca.com.br
  • https://impactdigital.space
  • https://imperibiza.com
  • https://impexworldltd.com
  • https://implant.gen.tr
  • https://impread.com
  • http://improveyourdance.com
  • http://improvisamos.com
  • http://impulsarh.com.ar
  • http://inamakethat.com
  • http://inanna.be
  • https://incomementorbox.com
  • https://incomerepairpro.com
  • https://indiatvnewss.com
  • https://indoranews.com
  • https://infendo.com
  • https://infinitebankingagenttraining.com
  • https://infinnium.com
  • http://influenceyourway.com
  • http://infocarbrokers.net
  • http://infohealth-usa.com
  • https://informatic-crew.fr
  • https://informatiotech.com
  • https://informeblumenau.com
  • https://infos-grossesse.fr
  • https://ingatlanmagus.com