Ugly truths about installing an SPFx package through a provisioning pipeline

My current project requires us to ad-hoc provision SharePoint site collections based on data entered into Dynamics 365, and these sites need to be provisioned with a specific structure. Achieving this structure is only possible using PnP Provisioning and the PnP Modern Search Web Parts package.

The version of the Modern Search web parts package we use is not configured for global deployment (this has changed in the newest package) and thus needs to be installed in each site collection separately. This is where we encountered some interesting behavior!

To summarize, these were our needs:

1. Create the site collection
2. Activate the PnP Modern Search Web Parts
3. Create the pages and put the web parts on them

Let's investigate the different possible approaches (and their positive/negative points) to achieve step 2, shall we?
All of them assume that the package already exists in the tenant app catalog.

Activate through the PnP Provisioning schema

This was our first attempt, and it is as easy as adding the following code to the provisioning template you are already applying to the site collection:

      <pnp:ApplicationLifecycleManagement>
        <pnp:Apps>
          <pnp:App AppId="{AppPackageId:PnP - Search Web Parts}" Action="Install" />
        </pnp:Apps>
      </pnp:ApplicationLifecycleManagement>

In the background this XML translates to the SharePoint Online Application Lifecycle API's.

Ugly truth

The ALM API's just trigger the installation process, the "install" call is actually asynchronous and finishes before the app is actually installed. To work around this, the PnP code goes into a loop and uses a different API call to check install status every 5 seconds for 5 minutes.

In our provisioning scenario's we noticed this process taking up to 9 minutes in some cases, much longer than the defined 5 minutes. The PnP Provisioning Engine then just bugs out, displaying an error message:

App Install timeout hit, could not determine installed state

This made our provisioning pipeline rather unreliable, resulting in broken site collections and a not so happy customer.

Activate through a Site Design

Second attempt uses a Site Design that contains a Site Script that installs the SPFx app. This Site Design was to be applied first to the site collection and then the PnP Provisioning Engine would create the pages with the web parts.
The JSON code for the Site Script is also very straightforward:

    {
      "verb": "installSolution",
      "id": "11a1daac-ada7-4f2a-85c1-36cfe8b58645",
      "name": "PnP - Search Web Parts"
    }

Ugly truth

We noticed very fast that this approach probably uses the same SharePoint Online Application Lifecycle API's in the background. We saw the Site Design action finishing but the app wasn't actually installed.
This time the PnP Provisioning engine didn't bug out, but all pages were created empty: since the web parts weren't available, they are just ignored during the provisioning process.

As a workaround, we tried using a Durable Function to go check the installation state of the application, before continuing with the PnP Provisioning Engine. This turned out to potentially slow down the whole provisioning process with minutes of waiting.

Once again our provisioning pipeline turned out to be very unreliable, resulting in broken site collections and unhappy customers. Our workaround also made it very slow in some cases, which was unacceptable.

Activate by switching to global deployment

The last thing we did, and are still doing, is make an update to the code for the web parts and switch to global deployment. Similarly to the previous methods, this is also a rather simple JSON code change to the package-solution.json file of the PnP Modern Search Web Parts:

"skipFeatureDeployment": true,

Now, when we used the PnP Provisioning Engine to create the pages, everything worked out correctly. The speed of the provisioning improved drastically since we weren't waiting for an app to finish installation anymore, and it also became a lot more consistent in timing due to this change.

All site collections turn out as we intended, speeds are acceptable and the customer is happy!

Ugly truth

This approach results in all site collections in your tenant having your web parts show up in the toolbox. While this is not a big deal (it might even be an asset) for the PnP Modern Search Web Parts, this might not be the preferred behavior for all your apps.

Second potential hurdle might be that you need access to the source code to make this change. It's not a problem in your own code, or with open source code, but it might be challenging with some third party packages that do not provide any source code.