Sunday, December 17, 2023

Worker Pool Scaling on Azure Batch

Azure Batch supports high-performance batch processing and I have been using it for a few months to test with heavy data workloads. The architecture excellence of using Batch service comes when how they are scaled-up when in need. You can set it up to autoscale, but I prefer manual if the workloads are low priority and less frequent.

In my experience it takes 2-10 minutes to perform a scale up operation within Azure Batch. Scale up performs a set of tasks to allocate an already warmed up machine from a pool of Azure resources within the region.