How send all events to an external MQTT Broker using Integration

Hello,
I would like to send all chirpstack events on an external MQTT broker, using the integrations. The same that you see in the dashboard in json format when you click on a deveui or a gateway (frames).

I would like to read the topics with the events of the devices, the gateways and the network server.

I configured the chirpstack.toml file as follows


...
[integration]
enabled = ["mqtt"]

# MQTT integration configuration.
[integration.mqtt]
  # Event topic template.
  event_topic="application/{{application_id}}/device/{{dev_eui}}/event/{{event}}"
  #Command topic.
  #
  # This is the topic on which the MQTT subscribes for receiving (enqueue) commands.
  command_topic="application/{{application_id}}/device/{{dev_eui}}/command/{{command}}"

  gateway_event_topic="gateway/{{gateway_id}}/event/{{event}}"
  gateway_state_topic="gateway/{{gateway_id}}/state/{{state}}"

  # Use JSON encoding instead of Protobuf (binary).
  json = true


  server = "tcp://0.0.0.0:1884/"
  client_id = "mqtt_integration"
  username = ""
  password = ""
  # QoS MQTT (0, 1, 2)
  qos=0
  clean_session=false

But I only get one topic: /application/…

How can I correctly configure the chirpstack.toml file?
Thank you !!

PS: port 1884 is correct because two brokers running on same host.

Here’s the documentation for the MQTT integration: MQTT - ChirpStack open-source LoRaWAN® Network Server documentation. It mentions this format:

application/APPLICATION_ID/device/DEV_EUI/event/EVENT

There is no mention of the topics that you cannot get to work (gateway_event_topic and gateway_state_topic).

The MQTT integration you configured in [integration.mqtt], does not send the gateway messages themselves. The messages to/from the gateway bridge (starting with your region prefix) are still encrypted and thus no use to external applications. Once Chirpstack receives these region_prefix/gateway/… messages it checks whether they are meant for your network and if so, decrypts them and posts them as application/… to where your MQTT integration is. So it’s expected behaviour not to see the gateway messages.

I can’t think of a reason you would want the encrypted messages but if you do I believe you can configure another server in your gateway bridge configuration to get those gateway frames to your additional MQTT broker.

Alternatively, you could use MQTT bridging to accomplish all of this. If you just want your secondary broker to be a mirror of Chirpstacks, the best solution might be to leave the MQTT integration pointed at Chirpstacks broker and configure a bridge in your mosquitto.conf to bridge all topics to your extra broker.

Thank you for the hint !

The request arises from the need to monitor all LoRaWAN radio traffic.

Other LNS allow you to see which other devices are making Join requests regardless of whether or not they belong to a specific network.

Lo stesso per i messaggi di traffico da dispositivi collegati ad altre reti che si trovano nella stessa area dei gateway gestiti dalla LNS.

In the end, the request arises from the need to be able to monitor how much traffic there is and what kind.

Thank you for your answer.

Hi,

Normally ChirpStack LNS only publish LoRa messages from registered devices to the MQTT topic application/APPLICATION_ID/device/DEV_EUI/event/EVENT. I made a small customization to the Chirpstack server code and now it can also publish up-events of the unregistered devices under Unknown device (dev EUI 00000000000000000). Please, see the image below.

This is just experimental code, but it seems possible to implement this kind of functionality quite easily.


I added following changes in chirpstack\src\uplink\data.rs file.

About at the line 265 added a new function call: self.handle_unknown_device(dev_addr).await?

    async fn get_device_for_phy_payload(&mut self) -> Result<(), Error> {
        trace!("Getting device for PhyPayload");

        let dev_addr = if let lrwn::Payload::MACPayload(pl) = &self.phy_payload.payload {
            pl.fhdr.devaddr
        } else {
            return Err(Error::Anyhow(anyhow!("No MacPayload in PhyPayload")));
        };

        match device::get_for_phypayload_and_incr_f_cnt_up(
...
            Err(e) => match e {
                StorageError::NotFound(s) => {
                    info!(dev_addr = %s, "No device-session exists for dev_addr");
                    // Start a new process flow to tracking location of the unknown devices.
                    self.handle_unknown_device(dev_addr).await?;
                    return Err(Error::Abort);
                }
...

        Ok(())
    }


And a new function handle_unknown_device() at the end of the file.

    // GLSensor-tracker - Unkown devices app - unknown device
    async fn handle_unknown_device(&mut self, dev_addr: lrwn::DevAddr) -> Result<(), Error> {

        use crate::uplink::data::fields::DeviceSession;

        const UNKNOWN_DEVICE_EUI: &str = "0000000000000000";
        const UNKNOWN_APPLICATION_NAME: &str = "Unknown devices";
        const UNKNOWN_DEVICE_NAME: &str = "Unknown device";

        let unknown_device_eui: EUI64 =  EUI64::from_str(UNKNOWN_DEVICE_EUI).unwrap_or_default();
        let (dev, app, t, dp) = get_all_device_data(unknown_device_eui).await?;

        if app.name == UNKNOWN_APPLICATION_NAME && dev.name == UNKNOWN_DEVICE_NAME {
            // OK, continue process
        } else {
            return Ok(());
        }   

        info!("Create uplink event for the unknown device (addr: {}) of the unknown devices application", dev_addr );

        if let lrwn::Payload::MACPayload(pl) = &self.phy_payload.payload {
            self.f_cnt_up_full = pl.fhdr.f_cnt;
        } else {
            return Err(anyhow!("Expected MacPayload"));
        };

        // Create a dummy session for the unknown device
        let dev_session = internal::DeviceSession {
            region_config_id: dp.region_config_id.as_ref().map_or("",|v| v).to_string(),
            dev_addr: dev_addr.to_vec(),
            s_nwk_s_int_key: vec![
                0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
                0x00, 0x00, 0x00, 0x00,
            ],
            f_nwk_s_int_key: vec![
                0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
                0x00, 0x00, 0x00, 0x00,
            ],
            nwk_s_enc_key: vec![
                0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
                0x00, 0x00, 0x00, 0x00,
            ],
            f_cnt_up: 1,
            skip_f_cnt_check: true,
            ..Default::default()
        };

        let mut device = dev;
        device.name = format!("{} - {}", "Unknown", dev_addr);
        device.dev_addr = Some(dev_addr);
        device.device_session = Some(DeviceSession::new(dev_session));

        self.device = Some(device);
        self.tenant = Some(t);
        self.application = Some(app);
        self.device_profile = Some(dp);

        self.set_device_info()?;
        
        self.set_device_gateway_rx_info()?;
        self.log_uplink_frame_set().await?;
        self.set_uplink_data_rate().await?;
        self.log_uplink_meta().await?;

        if !self._is_roaming() {
            self.save_device_gateway_rx_info().await?;
        }        
        
        self.append_meta_data_to_uplink_history()?;
        self.send_uplink_event().await?;
        self.update_device().await?;
        self.save_metrics().await?;
            
        Ok(())
    }

1 Like

Hi LouneCode, really great :slight_smile: :slight_smile: :slight_smile:!!
May I ask you send your solution to GitHub Chirpstack repo with the proposal to be integrated into the next release?

Would it be possible to also see the ‘deveui’ of an unknown device?

Thank you.

cocoa

“Would it be possible to also see the ‘deveui’ of an unknown device?”

The Dev EUI is only known by the LNS where the device is registered. So in this case it is not possible to know the EUI of a device that is registered in some other LNS. There is no devEUI information in the LoraWAN packet header.

https://www.rfwireless-world.com/images/LoRa-message-formats.webp

As you can see, the frame header contains a DevAddr attribute, which can be used to identify an unknown LoRaWAN device. For example, unknown device information:

  • Device name: Unknown - e00329db
  • Device EUI: 0000000000000000
  • Device address: e00329db

This is just an idea on how to find a simple solution to collect information about unknown devices. Maybe somebody could review this idea before do any PR actions… :thinking:

By the way, here is very interesting idea about LoRaWAN network maping:
LoRaWAN Network Mapping

https://miro.medium.com/v2/resize:fit:1100/format:webp/1*2lpK9tKP6UY9boIu8G1byg.png