Kubernetes Deployment - Failed Provisioning - Config Not found

Hello,
i am trying to deploy chirpstack/chirpstack:4 to a k8s cluster, but i am stuck on the provisioning of the pod.

The problem is on the startup command –config /etc/chirpstack, i can’t figure out why the application can’t find the folder while it is mounted correctly.

configMap.yml

apiVersion: v1
kind: ConfigMap
metadata:
  name: chirpstack
  labels:
    app: chirpstack
data:
  chirpstack.toml: |-
    # Logging.
    [logging]

      # Log level.
      #
      # Options are: trace, debug, info, warn error.
      level="info"


    # PostgreSQL configuration.
    [postgresql]

      # PostgreSQL DSN.
      #
      # Format example: postgres://<USERNAME>:<PASSWORD>@<HOSTNAME>/<DATABASE>?sslmode=<SSLMODE>.
      #
      # SSL mode options:
      #  * disable - no SSL
      #  * require - Always SSL (skip verification)
      #  * verify-ca - Always SSL (verify that the certificate presented by the server was signed by a trusted CA)
      #  * verify-full - Always SSL (verify that the certification presented by the server was signed by a trusted CA and the server host name matches the one in the certificate)
      dsn="postgres://$POSTGRESQL_USERNAME:$POSTGRESQL_PASSWORD@$POSTGRESQL_HOST/chirpstack?sslmode=require&sslcert=/etc/pgsql-certificate/client-cert.pem&sslkey=/etc/pgsql-certificate/client-key.pem"

      # Max open connections.
      #
      # This sets the max. number of open connections that are allowed in the
      # PostgreSQL connection pool.
      max_open_connections=10

      # Min idle connections.
      #
      # This sets the min. number of idle connections in the PostgreSQL connection
      # pool (0 = equal to max_open_connections).
      min_idle_connections=0


    # Redis configuration.
    [redis]

      # Server address or addresses.
      #
      # Set multiple addresses when connecting to a cluster.
      servers=[
        "redis://$REDIS_HOST/",
      ]

      # TLS enabled.
      tls_enabled=false

      # Redis Cluster.
      #
      # Set this to true when the provided URLs are pointing to a Redis Cluster
      # instance.
      cluster=false


    # Network related configuration.
    [network]

      # Network identifier (NetID, 3 bytes) encoded as HEX (e.g. 010203).
      net_id="000000"

      # Enabled regions.
      #
      # Multiple regions can be enabled simultaneously. Each region must match
      # the 'name' parameter of the region configuration in '[[regions]]'.
      # enabled_regions=[
      #   "as923",
      #   "as923_2",
      #   "as923_3",
      #   "as923_4",
      #   "au915_0",
      #   "cn470_10",
      #   "cn779",
      #   "eu433",
      #   "eu868",
      #   "in865",
      #   "ism2400",
      #   "kr920",
      #   "ru864",
      #   "us915_0",
      #   "us915_1",
      # ]


    # API interface configuration.
    [api]

      # interface:port to bind the API interface to.
      bind="0.0.0.0:8080"

      # Secret.
      #
      # This secret is used for generating login and API tokens, make sure this
      # is never exposed. Changing this secret will invalidate all login and API
      # tokens. The following command can be used to generate a random secret:
      #   openssl rand -base64 32
      secret="$API_SECRET"


    [integration]
      enabled=["mqtt"]

      [integration.mqtt]
        server="mqtts://$MQTT_BROKER_HOST:8883/"
        json=true
        
        # Connect with the given username (optional)
        username="$MQTT_BROKER_USERNAME"

        # Connect with the given password (optional)
        password="$MQTT_BROKER_PASSWORD"

deployment.yml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: chirpstack
  labels:
    app: chirpstack
spec:
  replicas: 1
  strategy:
    rollingUpdate:
      maxSurge: 0
      maxUnavailable: 1
    type: RollingUpdate
  selector:
    matchLabels:
      app: chirpstack
  template:
    metadata:
      labels:
        app: chirpstack
    spec:
      containers:
        - name: chirpstack
          image: chirpstack/chirpstack:4
          command: ["--config /etc/chirpstack"]
          env:
            - name: API_SECRET
              valueFrom:
                secretKeyRef:
                  name: chirpstack-secrets
                  key: api-secret
                  optional: false
            - name: MQTT_BROKER_HOST
              value: SECRET
            - name: MQTT_BROKER_USERNAME
              value: SECRET
            - name: MQTT_BROKER_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: chirpstack-secrets
                  key: mqtt-password
                  optional: false
            - name: REDIS_HOST
              value: SECRET
            - name: POSTGRESQL_HOST
              value: SECRET
            - name: POSTGRESQL_USERNAME
              value: SECRET
            - name: POSTGRESQL_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: chirpstack-secrets
                  key: pgsql-password
                  optional: false
          readinessProbe: null
          ports:
            - name: ui-port
              containerPort: 8080
          resources:
            limits:
              cpu: 500m
              memory: 500Mi
            requests:
              cpu: 250m
              memory: 250Mi
          volumeMounts:
            - name: chirpstack
              mountPath: /etc/chirpstack
              readOnly: true
            - name: pgsql-ssl
              mountPath: /etc/pgsql-certificate
              readOnly: true
      volumes:
        - name: chirpstack
          configMap:
            name: chirpstack
        - name: pgsql-ssl
          secret:
            secretName: pgsql-ssl-certificate
            items:
              - key: client-cert
                path: client-cert.pem
              - key: client-key
                path: client-key.pem
              - key: server-ca
                path: server-ca.pem
            optional: false

kubectl describe pod xyz

...
    Mounts:
      /etc/chirpstack from chirpstack (ro)
      /etc/pgsql-certificate from pgsql-ssl (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-bkm29 (ro)
...
Volumes:
  chirpstack:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      chirpstack
    Optional:  false
  pgsql-ssl:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  pgsql-ssl-certificate
    Optional:    false
...
Events:
  Type     Reason                             Age                From                                   Message
  ----     ------                             ----               ----                                   -------
  Normal   LoadBalancerNegNotReady            17s                neg-readiness-reflector                Waiting for pod to become healthy in at least one of the NEG(s): [k8s1-4bc42899-default-chirpstack-443-8762ef1f]
  Normal   Scheduled                          17s                gke.io/optimize-utilization-scheduler  Successfully assigned default/chirpstack-c6d5d4d44-ttcf2 to gk3-gizero-autopilot-nap-6ygnnhug-b7a506a1-8mb7
  Normal   Pulled                             13s (x2 over 15s)  kubelet                                Container image "chirpstack/chirpstack:4" already present on machine
  Normal   Created                            13s (x2 over 15s)  kubelet                                Created container chirpstack
  Warning  Failed                             11s (x2 over 14s)  kubelet                                Error: failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "--config /etc/chirpstack": stat --config /etc/chirpstack: no such file or directory: unknown
  Warning  BackOff                            10s (x2 over 11s)  kubelet                                Back-off restarting failed container
  Normal   LoadBalancerNegWithoutHealthCheck  8s                 neg-readiness-reflector                Pod is in NEG "Key{\"k8s1-4bc42899-default-chirpstack-443-8762ef1f\", zone: \"europe-west8-a\"}". NEG is not attached to any BackendService with health checking. Marking condition "cloud.google.com/load-balancer-neg-ready" to True.

If i change the image name to nginx and try cd /etc/chirpstack the volume is correctly mounted.
Is there a problem with the image?
Thanks.

1 Like

Hello, any update? @brocaar

Bump, problem still unsolved

Up? No one can help?
I am using the same approach as this repository charts/chirpstack/templates/application-server at main · liangyuanpeng/charts · GitHub

I’m sorry, but I can not provide any support. This question is too specific to your Kubernetes setup. For a client I am doing something similar (mounting the configuration within the container using Kubernetes) and all works fine. So all I can say is that what you are trying to do is possible, but there is probably a config error somewhere.

The error was really, really stupid.

Before:

      containers:
        - name: chirpstack
          image: chirpstack/chirpstack:4
          command: "--config /chirpstack-config"

After:

      containers:
        - name: chirpstack
          image: chirpstack/chirpstack:4
          command: ["/usr/bin/chirpstack"]
          args: ["--config", "/chirpstack-config"]

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.