Chirpstack on AWS EKS

Dear all,
I’ve successful installed Chirpstack application on a K8s cluster AWS EKS.
I’ve hosted on my cluster also the gateway bridge, and the service correctly exposes the UDP port 1700 using a Network Load Balancer.
All works correctly, when a device try to join I see this frames and it is activated successfully:

on the gateway bridge I could see these UDP messages:

08:10:10.835556 IP ip-192-10-46-58.eu-west-1.compute.internal.37525 > chirpstack-gateway-bridge-deployment-68bbb98666-xddkv.1700: UDP, length 204
E…B@.’…
.:.
/…V.{“rxpk”:[{“tmst”:27940891,“chan”:1,“rfch”:0,“freq”:868.300000,“stat”:1,“modu”:“LORA”,“datr”:“SF7BW125”,“codr”:“4/5”,“lsnr”:9.2,“rssi”:-29,“size”:23,“data”:“APjgA9B+1bNwGzsE/    v9YF6iudU8Rex8=”}]}
08:10:10.835880 IP chirpstack-gateway-bridge-deployment-68bbb98666-xddkv.1700 > ip-192-10-46-58.eu-west-1.compute.internal.37525: UDP, length 4
E… .,@…t.
/…
.:…H…
08:10:11.060621 IP chirpstack-gateway-bridge-deployment-68bbb98666-xddkv.1700 > ip-192-10-24-49.eu-west-1.compute.internal.37524: UDP, length 192
E…@…
/…
.1…{“txpk”:{“imme”:false,“rfch”:0,“powe”:14,“ant”:0,“brd”:0,“tmst”:32940891,“freq”:868.3,“modu”:“LORA”,“datr”:“SF7BW125”,“codr”:“4/5”,“ipol”:true,“size”:17,“data”:“IAfry30XG6QdI+z    EjfSwWFM=”}}
08:10:11.138983 IP ip-192-10-24-49.eu-west-1.compute.internal.37524 > chirpstack-gateway-bridge-deployment-68bbb98666-xddkv.1700: UDP, length 12
E…(.q@.%…1.
.1.
/…C …V.
08:10:15.529064 IP ip-192-10-24-49.eu-west-1.compute.internal.37524 > chirpstack-gateway-bridge-deployment-68bbb98666-xddkv.1700: UDP, length 12
E…(.y@.%…).
.1.
/…F…V.
08:10:15.529190 IP chirpstack-gateway-bridge-deployment-68bbb98666-xddkv.1700 > ip-192-10-24-49.eu-west-1.compute.internal.37524: UDP, length 4
E… .D@…e.
/…
.1…?..F.    

I’ve deployed the same identical configuration to another K8s cluster on AWS EKS and I’ve changed the gateway configuration to point out the NLB deployed by the service installed on the new cluster.
In this case, unfortunately, randomly two different situations could come:

  • No frames from gateway
  • Only Join Requests received, but not Join Accept

In the first case I can’t see any frame and on UDP side I see this kind of messages only:

09:24:34.679091 IP ip-10-0-18-141.eu-west-1.compute.internal.21305 > chirpstack-gateway-bridge-deployment-6bf6659756-wckxd.1700: UDP, length 12
E..(C7@.%...
...
..OS9....._..........V.
09:24:34.679233 IP chirpstack-gateway-bridge-deployment-6bf6659756-wckxd.1700 > ip-10-0-18-141.eu-west-1.compute.internal.21305: UDP, length 4
E.. ..@.....
..O
.....S9..>.....

In the second scenario I see these frames:

and on UDP side I see messages starting with rxpk but not those starting with txpk.

What could be wrong on the second cluster? My impression is that for some reason the gateway bridge is unable to send DOWNLINK messages to the gateway, probably due to some network issue, but I’m unable to understand what is the problem.
Only difference I see is that the working cluster has nodes with public IPs, while the other cluster has private nodes with an Internet Gateway. Could be this the cause? What is the correct configuration to deploy the gateway bridge on a K8s cluster?

Thank you very much in advance,
bests
Roberto

UPDATE
UDP packets on gateway bridge on the not working cluster:

10:26:34.056575 IP ip-10-0-29-8.eu-west-1.compute.internal.16066 > chirpstack-gateway-bridge-deployment-6bf6659756-hfvkw.1700: UDP, length 485
E....X@.&...
...
...>......b..Z.......V.{"rxpk":[{"tmst":115689916,"chan":5,"rfch":1,"freq":867.500000,"stat":-1,"modu":"LORA","datr":"SF7BW125","codr":"4/6","lsnr":-10.8,"rssi":-98,"size":231,"data":"f/GsiqiSqJo5qQ/4QmxFkNoMC5TtU86q10doQHpRMuBA45KWRUO/eKNjCS0bIFAiEDO1wV0KPc9JqLbP3D6Mi0Vf6Wdc/r/Q1tjPDGOYC2pZrDO2tfdIdEycL00BIq0809wngCG1DXSN5IJGZanelpNNFjf2YbbVUteVnwh6tzMmtP    MHiL75X4t1DebpL3eDmXTjeFmR6TewHBVcCSIi7uKgF4iUkBA518VSvdlbuUYs8+sgRnFOqfEdmVGw3f+Sk8bZ1dSvjLomls0UZWYJvtHxg/S1oppUoesc6oxmG4FwJ9hROMnE"}]}
10:26:34.056740 IP chirpstack-gateway-bridge-deployment-6bf6659756-hfvkw.1700 > ip-10-0-29-8.eu-west-1.compute.internal.16066: UDP, length 4
E.. |.@....c
...
.....>...G...Z.

Hey Roberto,

I’m facing the same problem. Were you able to solve it eventually? if so, how did you manage it?

Best,

Alan