This was already the case before group video calls and, yes, IPs can be used as identifiers. I don't think this can be prevented on a technical level[0] – you will always need to trust the server here. Signal does have a good track record, though: https://signal.org/bigbrother/

[0]: Unless you do p2p routing of video calls – with the known downsides regarding traffic, performance and NAT. But even then you would probably still need a server as rendez-vous point.