For a while now I have made good use of the Trace functionality in the API Management (APIM) Test Client. If you haven’t, I would highly advise having a look. The Trace functionality allows us to unveil (debug) the complexity and inner workings of our reverse proxy APIs (their routing / hierarchical policies / alternate backends / caching / etc.). With the Portal this is fairly trivial to do, you simply:

1. Navigate to your API Management instance.
2. Select one of your APIs that you would like to Trace.
3. Select the Test tab.
4. Select one of the Operations to send a request to.
5. Select Trace.

By enabling tracing on the API you are now able to inspect each part of the request:

• Inbound | The original request received from the caller and the policies applied.
• Backend | The requests API Management sent to the backend and the response it received.
• Outbound | The policies applied to the response before sending back to the caller.
• On error | The errors that occurred during processing of the request and any policies applied to those errors.

⚠️ Important

Be careful when enabling tracing as it can expose sensitive information in the trace data.

Problem Space

Although the APIM Test client has certainly proven useful, I have wondered if there are any other methods of enabling and obtaining the trace for a given API. More importantly, brining the development experience back to something that we are use to such as Visual Studio Code, Postman, etc.

Just over a week ago, I had a conversation with the APIM product group, of which they mentioned that the service has a REST API that can be used to enable trace functionality. After a bit of digging, I found the documentation around this and thought I would share my findings and implementation with you.

⚠️ Important

Before we get started, please bare in mind:

• The API Management REST API version 2023-05-01-preview or later is required.
• The Identity used to call the REST API must be assigned the Contributor or higher role on the API Management instance.

Enabling tracing on an API through the REST API is a four step process:

1. We need to obtain trace credentials using the List Debug Credentials API.
2. Enable Tracing on your API by adding the Trace Credential token as a header (Apim-Debug-Authorization) to your request.
3. Given that the Trace Credential token was valid, we need to retrieve the Trace ID from the response header Apim-Trace-Id.

   Invalid Cases Include:

   • Token expired | The response will include a Apim-Debug-Authorization-Expired header with information about expiration date.

   • Token obtained for wrong API | The response will include a Apim-Debug-Authorization-WrongAPI header with an error message.

4. We need to retrieve the trace log by calling the List Trace API using the Trace ID from the previous step.

In both steps 1 and 4, you will need to provide a valid Authorization Bearer token where the identity is assigned Contributor or higher on the APIM instance.

Given that this is not the simplicity that we know and love in the Test Client, making and orchestrating these various Rest calls can and most certainly will take time when conducted singularly. What we need is tooling or extensions to such tooling that will enable us to orchestrate these various calls so that we can obtain the trace logs with repeatable ease.

My Solution

I required a solution that would allow me to orchestrate the various calls with repeatable ease whilst also keeping me close to the tooling that I know and love. My solution is the REST Client extension in VS Code. Using this extension I can:

1. Orchestrate the various Rest calls required.

2. Use variables to make as much of the calls required somewhat generic.

3. The extension provides me the ability to obtain an AAD (Entra ID) token.

   By default if a token has already been obtained, the extension will reuse the previous token for the specified directory from an in-memory cache. Expired tokens are refreshed automatically. (The token cache can be manually cleared, or is by default cleared upon closure of VS Code).

The implementation that I have come to is as follows:

# ***************************************************************************************************************
# Enable tracing for an APIM API
# ------------------------------
#
# ⚠️Note:
#    The following steps require API Management REST API version 2023-05-01-preview or later. 
#    You must be assigned the Contributor or higher role on the API Management instance to call the REST API.
#
#  Steps:
#  1. Obtain trace credentials by calling the List debug credentials API.
#  2. Call the API with the trace credentials.
#  3. Retrieve trace logs.
#
#  Author: Andrew Wilson
# ***************************************************************************************************************

# -----------------
# Custom Variables 
# -----------------

@subscriptionId = REPLACEME
@resourceGroup = REPLACEME
@apimServiceName = REPLACEME
@apiName = REPLACEME

# -----------------
#    Rest Steps
# -----------------

### 1.Obtain trace credentials by calling the List debug credentials API.

# @name TraceCreds
POST https://management.azure.com/subscriptions/{{subscriptionId}}/resourceGroups/{{resourceGroup}}/providers/Microsoft.ApiManagement/service/{{apimServiceName}}/gateways/managed/listDebugCredentials
?api-version=2023-05-01-preview
Content-Type: application/json
Authorization: {{$aadToken}}

{
    "credentialsExpireAfter": "PT1H",
    "apiId": "/subscriptions/{{subscriptionId}}/resourceGroups/{{resourceGroup}}/providers/Microsoft.ApiManagement/service/{{apimServiceName}}/apis/{{apiName}}",
    "purposes": ["tracing"]
}

### 2.Call the API with the trace credentials.

# @name ApiOperationTracing
Author your API request here but make sure to keep the following Header in place.
Apim-Debug-Authorization: {{TraceCreds.response.body.$.token}}

### 3.Retrieve Trace Logs

# @name ObtainedTraceLogs
POST https://management.azure.com/subscriptions/{{subscriptionId}}/resourceGroups/{{resourceGroup}}/providers/Microsoft.ApiManagement/service/{{apimServiceName}}/gateways/managed/listTrace
?api-version=2023-05-01-preview
Content-Type: application/json
Authorization: {{$aadToken}}

{
    "traceId": "{{ApiOperationTracing.response.headers.Apim-Trace-Id}}"
}

Have a play and see if this extends your tracing capabilities.

I have recently been working on an API scoped policy within API Management, the policy ideally should not be impacted by any policies defined higher up in the hierarchy.

For reference, this means that any policies defined at the Product, Workspace, or Global level will not be inherited at the API scope for the given API Definition. See diagram below:

Ideally this means that my API traffic will start at my non-hierarchical policy definition, conduct any policy processing prior to being sent off to the backend, and then sent back to the calling outbound system.

Example Policy definition with the removal of <base /> which indicates inheritance:

 <policies>
   <inbound>
  		...
   </inbound>
   <backend>
  		...
   </backend>
   <outbound>
  		...
   </outbound>
   <on-error>
  		...
   </on-error>
 </policies>

In practice this is not what happened.

The API that I had defined should have successfully returned with a Status Code of 200 and a Json Payload. However, what I actually received was a successful API run with a return of Status Code 200 but no Json Payload.

Further investigation into this highlighted that the API request was never making it out to the Backend.

This being the case, a reviewal of the higher scoped policies showed that there was indeed a policy that is core to the APIM reverse proxy behaviour, and I had excluded this due to my lack of inheritance.

Offending Policy that was excluded due to my lack of inheritance

  <backend>
  	<forward-request />
  </backend>

For reference, the forward-request policy forwards the incoming request to the backend service specified in the request context. And more importantly the policy is included in the Global Backend Policy by default.

Conclusion

Two things came from this,

1. I fixed my problem by applying the forward-request policy in my API policy definition.
2. More importantly, Always review hierarchical policies when removing inheritance as there may be behaviour that you will require, even if you didn’t place it there.

In API Management, users and groups are a core aspect of the Developer Portal and are used to manage the visibility and access to respective products and their APIs.

One of the common questions that I often get asked is, “how do I appropriately govern the groups effectively so that I can ensure that the correct groups and users have access to the appropriate resources and those who don’t… well don’t?”.

Background

API Management has three immutable built in system groups used to govern access:

• Administrators | These are not consumers or customers of your APIs but are rather the users who build and manage the API Management service instances, creating the APIs, operations, and products that are used by developers. You can’t add users to this group.

  The group contains the admin email account provided at the time of service creation. Azure subscription administrators are members of this group.

• Developers | These are customers and consumers that build applications using your APIs. Developers are granted access to and are authenticated into the developer portal. Membership to this group is managed by the system and cannot be added to by you.

  Developers (Authenticated Users) invited, created, pulled from external groups, or otherwise signed up are all created as a developer and gain automatic membership to this group.

• Guests | These are users such as prospective customers and are able to visit the developer portal without having to be authenticated. APIs that guests can view are managed by yourself as well as the type of access such as read-only access (the ability to view APIs but not call them). As with the previous two groups, membership is managed by the system.

Then there is the ability to create custom groups and or use external groups in associated Microsoft Entra tenants. These custom groups can be used to further govern developer visibility and access to API products.

Note:

User accounts from MS Entra ID or B2C groups are still created as new developer user entities within APIM and associated with the Developers System group as well as the desired Entra ID group.

Governing Access

The key to understanding Group Access is to remember that all developers (no matter how they have been created) are associated with the Developers System group. In effect this means that any product that has been associated with the Developers system group will be accessible to all users (except Guests given association).

Therefore do not blindly associate the Developer System group to your products unless you expect all users past, present, and future to have access to them.

Rather in answer create custom groups/associate Entra ID groups of which can be used to provide access to specific products. Further to this, granularity is key for security, following the good practice of least privilege (A user or entity should only have access to the specific data, resources and applications needed to complete a required task.).

Have a play, and hope this helps.