Worker Pool Scaling on Azure Batch

Azure Batch supports high-performance batch processing and I have been using it for a few months to test with heavy data workloads. The architecture excellence of using Batch service comes when how they are scaled-up when in need. You can set it up to autoscale, but I prefer manual if the workloads are low priority and less frequent.

In my experience it takes 2-10 minutes to perform a scale up operation within Azure Batch. Scale up performs a set of tasks to allocate an already warmed up machine from a pool of Azure resources within the region. 

Deploying Batch Application Packages

Azure Batch Service allows deploying and executing batch applications as parallel tasks. Scalability of the underlying compute nodes support parallel processing and efficiency on resource consumption. In this article we will discuss the lifecycle of applications referenced in Azure Batch service. 

You can access the Applications within a Batch account via the Azure Portal. 

Azure Batch for High Performance Batch Processes?

Batch processing often demands two constraints. Time and Power. You need high-demand processing for a limited duration and if the processing cannot be powerful it can consume a significant time or incur timeouts. Usually the high-end processing can be costly for many small or medium scale organisations to have on-premise.

What if there is a PaaS/ IaaS service that allows you to perform powerful batch processes while paying for only the consumption? What if you can scale-up and scale-down the processing power relative to the demand. 

Azure Web App APIs exposed to APIM

It is true we are living in a modern era where many tasks are automated. But I do not personally backoff from a great portal feature on Azure that can help make your life easy. Today I am going to show you how simplified is creating an APIM API referencing an API App deployed on the Azure Web App.

Finding this feature on an Azure Web App is very easy. You can just navigate through the service options and then can find API Management as a section.

Leveraging variables in Azure DevOps pipelines

Azure DevOps pipelines use variables to store and manage data that can be used throughout the CI/CD process. These variables can hold values like configuration settings, environment-specific information, and secrets. Azure DevOps provides different types of variables, including predefined system variables and custom variables that you can define. Here's an overview of Azure DevOps pipeline variables:

System Variables: 

These are predefined variables provided by Azure DevOps. They include information about the pipeline, build agent, and more. Some common system variables include: 

Introduction to Microsoft Cost Management

Source: Azure Product Page 

 Microsoft Cost Management is a powerful tool Microsoft offers to help organisations manage their cloud costs on the Azure platform. When many applications within the organisations get migrated, keeping track of cloud costs is becoming increasingly important to ensure they remain within budget.

Microsoft Cost Management offers a range of features designed to help businesses keep their cloud spending under control. Here are some of the key features:

• Cost analysis: Provides a detailed breakdown of cloud spending, allowing businesses to see exactly where their money is being spent. This helps identify areas where costs can be reduced.

What are the VM Classes in Azure?

 Azure Virtual Machines offer flexible deployment options, including customizable virtual machine sizes and multiple pricing tiers.

As a consumer, you have multple classes to choose your VM workload from. Lets take a closer look at the different Azure VM classes available and how they can be used to meet specific needs.

• General-purpose VMs

General-purpose VMs are designed for a wide range of workloads and applications. These VMs offer a balance of CPU, memory, and temporary storage, making them ideal for applications that require moderate-to-high performance, such as web servers, small databases, and development and testing environments. 

Azure Load Testing goes GA

Azure Load Testing allows you to conduct your workloads hosted on cloud.  While there are definite room for new features, it has been announced as Generally Available.  Creating an Azure Load Test instance on Azure is very easy. Hence, I will start with a created instance. 

In the quotas section you can see the quotas associated with the test instance. It also explains how the quotas are applied at subscription level. 

Submit .Net class library to Azure Artifacts Feed

 Azure Artifacts feed provides great means to share the common reusable components across your organization. In the following code base I will display a template for Azure DevOps build pipeline. 

trigger:

- 'main'

pool:

vmImage: ubuntu-latest

demands:

- npm

variables:

buildConfiguration: 'Release'

dotNetFramework: 'net7.0'

targetRuntime: 'linux-x64'

moduleName: 'ModuleName'

entityTypeName: 'EntityType'

// Once you create a feed, that ID will be added here.

vstsFeedId: '<A Guid>'

major: '1'

minor: '0'

revision: $[counter(variables['minor'], 1)]

nugetVersion: '$(major).$(minor).$(revision)'

steps:

- task: NuGetAuthenticate@1

inputs:

forceReinstallCredentialProvider: true

// Optional: Run unit tests

- task: DotNetCoreCLI@2

displayName: 'Run unit tests - $(buildConfiguration)'

inputs:

command: 'test'

arguments: '--framework $(dotNetFramework) --configuration $(buildConfiguration)'

publishTestResults: true

projects: '**/*.Tests.csproj'

// Does packaging. Nuget version is automatically increased to next revision.

- task: NuGetCommand@2

displayName: 'Nuget Pack'

inputs:

command: 'pack'

packagesToPack: '**/$(moduleName).$(entityTypeName).csproj'

versionEnvVar: 'nugetVersion'

versioningScheme: 'byEnvVar'

packDestination: '$(Build.ArtifactStagingDirectory)'

- task: PublishPipelineArtifact@1

inputs:

targetPath: '$(Build.ArtifactStagingDirectory)'

publishLocation: 'pipeline'

- task: NuGetCommand@2

displayName: 'Nuget Push'

inputs:

command: 'push'

packagesToPush: '$(Build.ArtifactStagingDirectory)/**/*.nupkg;!$(Build.ArtifactStagingDirectory)/**/*.symbols.nupkg'

nuGetFeedType: 'internal'

publishVstsFeed: '$(vstsFeedId)'

Setting up App Config values with Bicep

There are often times where we need to setup a master configuration instance for the config values that might be referenced within pipelines. In such situations we can setup the initial configs such as environment name initially. 

First of all we will initialise a set of key value names. 

param keyValueNames array = [

  'appName$dev'

  'environmentType$dev'

  'Acr_Name$dev'

  'region$dev-${region}'

]