Disturbingly similar to my employer.

I don’t oppose calling it the Arabian Gulf. I also don’t really give a shit.

The second option. With the first option you’ll end up in situations where you have spare compute/network resource that isn’t being utilised because all the remaining ones in the current batch of 100 are being handled by other threads / worker processes.

Write a scraper using python and selenium or something. You may have to manually log in as part of it

Not a DevOps eng so this might not be the most elegant answer but you could write an Ansible playbook to edit a hosts file on the target server and trigger that after deployment, within your CI/CD pipeline.

In an ideal situation, though, all your servers connect to a centralised DNS server (usually your gateway) where those domain names are managed.