Using Batching in PnP PowerShell

On By sean mcavinueIn Uncategorized

Large Tasks Are Time Consuming

A few weeks ago I wrote about how you can update metadata on SharePoint Online items using PnP PowerShell. When I wrote about updating metadata I mentioned that this is very useful in a migration scenario. Unfortunatly, migration scenarios also often come with a tight timeframe to complete the updates.

While the examples I gave in the previous article are absolutly valid and a great way to update items in SharePoint programmatically, there are ways we can optimize the number of requests we send and speed up the update process. In this article, I’ll look at how we can enhance the previous example by adding batching to our script.

What is Batching?

Batching is when we combine multiple requests into a single (or at least fewer) requests that we send together. For example, take the below code:

foreach($item in $items){
Set-PnPListItem -List $DocumentLibrary -Identity $item.id -Values $Values
 }

In the above example, we loop through an array of items and update each one with specific values. The values we are updating don’t really matter for this example, what matters is that we are running each one individually.

There’s nothing wrong with this approach and it keeps things simple. If we wanted to try optimize this code, we could look to implement batching.

Why Use Batching?

So why would we use batching? Taking the above example, if we had 100 items in our array, we are making a number of requests for each item, depending on what exactly is being updated (Full detail is available in the documentation). If we implement batching, we drastically reduce the number of requests we send, and by extension, drastically reduce the time it takes to run.

What Cmdlets Support Batching?

At the moment, as described in the documentation linked above, there aren’t too many cmdlets that support batching but I’ve no doubt this list will grow. The current list at the time of writing is:

The big ones for me are the PnPListItem cmdlets as they allow you to delete, update and create new items in SharePoint – these are key activities, particularly during migrations.

How To Add Batching To Your Code

Now for the fun part, how do we add batching to our code? The first step is to create a new batch object with the New-PnPBatch cmdlet as shown below:

$Batch = New-PnPBatch

With the batch created, we add the -Batch parameter to our cmdlets and passing our batch variable which adds them to our batch. I’ve updated the previous example to add this parameter:

foreach($item in $items){
Set-PnPListItem -List $DocumentLibrary -Identity $item.id -Values $Values -Batch $Batch
 }

Now when we look through the items, the cmdlets will be added to the batch. We then run the batch using Invoke-PnPBatch as shown below:

Invoke-PnPBatch -Batch $Batch

This runs the batch. If there are more than 100 items in the batch, it will be automatically broken into groups of 100 to run. Note that we run our batch outside of the loop so it only runs once.

Putting all of that together, our full code looks like this:

$Batch = New-PnPBatch
foreach($item in $items){
Set-PnPListItem -List $DocumentLibrary -Identity $item.id -Values $Values -Batch $Batch
 }
Invoke-PnPBatch -Batch $Batch

Summary

While in most cases, if you are performing small updates or building automations, you might not need batching. When it comes to large scale activities though, it can save a lot of time if you structure your script correctly.

The concept of batching isn’t unique to PnP, it’s also available in the Microsoft Graph API. If you need to build larger, more demanding code – it’s worth giving batching a try!

Leave a comment