You need to ensure that UDP hole-punching can work. This means that all firewalls & load-balancers (the Nginx included) must keep the “session” alive. I remember that Nginx needs to have settings to tell it that the session hasn’t ended, but I cannot remember the exact attributes. Otherwise, the responses cannot be routed back.
The GWMP protocol involves the LNS normally replying to every heartbeat message, so this can be used to keep the session alive. If the GW will send heartbeat every 10s, then the session must be alive for longer than that in order for downlinks to also work.