Symptoms
There are two cases can occur when new subscription was created by the Microsoft365 application.
Case 1.
On attempt to increase limit for Microsoft365 subscription, a Change Order was failed. Meanwhile, a new Microsoft365 subscription was created from Microsoft.
M365 applicaton was reading a data from Cloud, but failed with an error in the Endpoint sitelog
2021-03-16 09:05:16,622 <5112> [ 41] ERROR flow : Forbidden Partner Exception: Error Category: Forbidden Service Error Payload: Error code: 403 Error message: Forbidden Error data: Context: Request Id: 44c1e213-e865-441f-9182-f92e34109c68, Correlation Id: aa334cad-9c44-412c-8516-738417fed047, Locale: en-US Base Description: Microsoft.Store.PartnerCenter.Exceptions.PartnerException: Forbidden
where the key point here is "Error Category: Forbidden". It means that the partner doesn't have an access to the subscription and the M365 endpoint deleted the subscription from the internal database.
2021-03-16 09:05:16,638 <5112> [ 41] INFO flow : The Subscription with id '52ef81bf-7d7b-494d-b7b1-a84b113fcf7f' (Csp Id '52ef81bf-7d7b-494d-b7b1-a84b113fcf7f') of customer with id '7h3d8101-726e-3b1f-8129-62b357ce0te5' does not exist and will be removed now. 2021-03-16 09:05:16,778 <5112> [ 41] INFO flow : Subscription provisioning is in progress: The subscription '52ef81bf-7d7b-494d-b7b1-a84b113fcf7f' (CSP Id '52ef81bf-7d7b-494d-b7b1-a84b113fcf7f', Offer Id 'bd938f12-058f-4927-bba3-ae36b1d2501c') cannot be updated yet.
Till now the error Forbidden returned by Microsoft for the discontinued subscriptions only, that's why the M365 endpoint considers such subscriptions as "Doesn't exist".
Failed order was re-submited and Microsoft365 application created a new subscription.
Case 2.
Renewal Order created a new subscription with the same behavior where "Forbidden" error was received from Microsoft side on attempt to read customer's subscription.
The failed task "Task Provisioning "AsyncOperation" for APS application Microsoft 365" exist in Task Log with output
500 Internal Server Error' received from 'POST https://10.120.2.40:443/Office365GatewayApplication/aps/asyncOperations': Use limit is exceeded for Offer id 'BD938F12-058F-4927-BBA3-AE36B1D2501C
and with error in the Endpoint sitelog
2021-05-13 17:10:05,343 <5684> [ 39] ERROR flow : Use limit is exceeded for Offer id 'BD938F12-058F-4927-BBA3-AE36B1D2501C' Partner Exception: Error Category: BadInput Service Error Payload: Error code: 27006 Error message: Use limit is exceeded for Offer id 'BD938F12-058F-4927-BBA3-AE36B1D2501C' Error data:
Cause
An internal issue from Microsoft side. Microsoft reported that the subscription is not accessible for Partner with "Forbidden" error.
The application behavior is recognized as a software bug APSA-23649, where Microsoft365 application creates a new subscription instead of mark an Order as "Failed".
Resolution
The issue has been fixed in Microsoft365 v 20.3.1. Please consider the application upgrade accordingly.
Case 2.
Please reach out CloudBlue Technical Support team to fix the issue with the failed task.