When you want to connect from an Azure DevOps pipeline to outside systems (SharePoint, Azure, etc.) you need a Service Connection. Service connections contain the endpoint for connection and the authentication information.
Some Service Connection types (like Azure Resource Manager) allow you to "bring your own Service Principal": you create the Service Principal yourself in Azure, and you fill in the information in the wizard in Azure DevOps. I prefer this over an automatic creation through Azure DevOps, for several reasons:
- Each new Service Connection creates a new Service Principal
- Service Principals don't get deleted when you delete the Service Connection
- Service Principals get created with a weird name that doesn't really explain where it is used
- The creation experience requires the logged-in user to have permissions to create Service Principals in Azure AD
- The creation experience requires the logged-in user to have permissions to assign permissions to the Service Principal in the subscription
These limitations are quite difficult to overcome in some organizations. So I got a bit annoyed this week when I needed to work with a Docker Registry connection to Azure Container Services, as it seemingly does not support bringing your own Service Principal. First of all you need to figure out that you need Docker Registry in the New service connection experience.
You'll get three choices:
- Docker Hub: for connections to your registry hosted on https://hub.docker.com
- Others: you can provide the URL to your own (self hosted?) Docker registry
- Azure Container Registry: for connections to your private container registry within Azure
I needed a connection to my Azure Container Registry, so naturally, I picked the obvious choice: Azure Container Registry. It automatically loads all Azure subscriptions you can access, and it loads the registries from that subscription. When you finish the connection, it will create it's own Service Principal and no way of changing that behaviour.
I dove into the documentation but it is nearly non-existent, and doesn't bring me closer to a solution. This is not different from the other Service Connections, none have much documentation. Probably the experience is deemed to be self explanatory...
After a good Google session, I finally found this deep in the comments of a GitHub issue promising better support for Service Principal reuse:
Ignore this one, which claims to work but still creates a new Service Principal anyways. It might have worked some day but at this point doesn't work at all.
EUREKA! This is just what I needed!
Bring your own Service Principal, UI experience
While this experience isn't clear from the UI, it is actually very simple to bring your own Service Principal for an Azure Container Registry connection.
Form the creation experience, you select Others as the type:
- Docker Registry: the URL to your Azure Container Registry, eg. https://contose.azurecr.io
- Docker ID: the client id of your Service Principal in Azure
- Docker Password: the secret you created for your Service Principal in Azure
Give the connection a name, decide on Grant access permission to all pipelines and click Save. Tadoum, there is your connection with your own Service Principal. The UI is horrible, but it works!
Bring your own Service Principal, REST API experience
You can do the same thing through REST API. The UI method is simple, so it might be a stretch but I'm a developer and I love API's 😅. So here it goes!
First of all you need a Personal Access Token and choose your scopes. Full will work but if you'd want it scoped down a bit, make sure you at least have Read, query, & manage underneath Service Connections.
Transform that access token in base64 encoding, and use it in the header of your API request to Azure DevOps: Authorization: Basic {your base64 encoded PAT}.
Create Service Connection
Your API request has to go to https://dev.azure.com/{organization}/{project}, or if you are still on the old URL: https://{organization}.visualstudio.com/{project}.
POST https://dev.azure.com/{organization}/{project}/_apis/serviceendpoint/endpoints?api-version=5.1-preview.2 HTTP/1.1
Authorization: Basic {your base 64 encoded PAT}
{
"authorization": {
"scheme":"UsernamePassword",
"parameters": {
"registry":"https://contoso.azurecr.io",
"username":"{client id}",
"password":"{secret}",
"email":""
}
},
"data": {
"registrytype":"Others"
},
"description":"",
"name":"Contoso Container Registry",
"type":"dockerregistry",
"url":"https://contoso.azurecr.io",
"isShared":false,
"owner":"library"
}Make sure you replace the URL to your Azure Container Registry instance, and put in the correct {client id} and {secret}.
Authorize the Service Connection
In the UI you have the Grant access permission to all pipelines. This allows all pipelines to use this connection. When you don't check this, you'll need to approve the usage of the connection in each pipeline once on the first run.
Setting this through API is possible, but cannot be in the same call as the creation of the Service Connection. You'll need to call another API endpoint:
PATCH https://dev.azure.com/{organization}/{project}/_apis/build/authorizedresources?api-version=5.1-preview.1 HTTP/1.1
Authorization: Basic {your base 64 encoded PAT}
[{
"authorized": true,
"id": "{Service Connection ID}",
"name": "Contoso Container Registry",
"type": "endpoint"
}]Don't forget to put in the correct Service Connection ID, this gets returned when you create the Service Connection with the previous REST call, and update the name of the endpoint too.
Tadoum, now you can use the DevOps REST API's as well!
So while it seemed impossible first, there are multiple ways to bring your own Service Principal for an Azure Container Registry service connection in Azure DevOps. The experience could be a bit smoother but at least our Azure Active Directory will not be littered with random Service Principals! 😃