Multiple device profiles in a single application

Hello there,
I have an issue with some of my LoRaWan devices. I have setup two device profiles,
Name Region MAC version Revision OTAA Class-B Class-C
Profile A: EU868 LoRaWAN 1.0.2 A yes no no
Profile B: EU868 LoRaWAN 1.0.2 B yes no no

I have one device of type A and one device of type B. When I add the devices to two applications, one for each profile type, everything is running fine. When I add the devices to the same application, the join procedure fails for both of them. I can see that for each device join events are received in the Events tab but the join is never accepted. For both devices the correct device profile is selected.
With the join events I can see, that while the DevEUI stays the same, the DevAddr is changing from join request to join request. Is this an expected behaviour? As far as I understood, the DevAddr is part of the join accept message, meaning that the request was handled somehow, but no join accept frame is sent.

Are there limitations on which devices (or which / how many profiles) can be added to an application?
Having two applications running is not that much of an issue, but I would prefer to have a single application.

Thanks in advance for any help on this.