:: Cisco SD-WAN: stuck tasks in vManage

Hi there,

As I’m using my lab environment to testdrive new features, I stumbled upon an annoying behavior with Multi-Cloud Auditing. I’m not going te tell how this works, but more the issue that followed.

I was changing components in AWS without changing it via the vManage MultiCloud feature. This triggered the stuck task in vManage.

Note from author: Obviously this won’t work well, but we tend to break stuff in order to fix it again. That is how learning sometimes works.

The Multi-Cloud audit task hung for 5 weeks. I could operate vManage normally, but I needed to upgrade the environment. This failed due to this running task. Even a reboot did not kill that task. It seems it was re-started right after reboot. There was no option to delete the hung task from vManage. So, I started my adventure on the web in search for an option to kill the task. After having some trouble finding a solution for my problem, I found out that there was one tiny document from Cisco explaining how to cancel these type of tasks. And it’s pretty old. Let’s dust this very useful option off a bit and see how it works.

:: How it works

You initiate a API call to vManage. You can do this via various tools, but we will use the Chrome browser, as this works fine.

Initiate get call from the browser address bar:

Check the result:

{
“runningTasks”:
[
{“detailsURL”:”/dataservice/device/action/status”,
“userSessionUserName”:”system”
,”@rid”:1477,
“tenantName”:”DefaultTenant”,
“processId”:”7adf7476-3c9f-43e4-936b-222cedf3be68″,
“name”:”Multicloud – Audit”,
“userSessionIP”:”10.1.1.2″,
“tenantId”:”default”,
“action”:”multicloud_audit”,
“startTime”:1638530152903,
“endTime”:0,”status”:”in_progress”},
{“detailsURL”:”/dataservice/device/action/status”,
“userSessionUserName”:”admin”,
“@rid”:1516,
“tenantName”:”DefaultTenant”,
“processId”:”software_install-adea5361-0c0b-470d-97b6-e4cd5ff7cfe5″,
“userSessionIP”:”10.36.106.16″,
“name”:”Software Install”,
“tenantId”:”default”,
“action”:”software_install”,
“startTime”:1641406500245,
“endTime”:0,
“status”:”in_progress”}
]
}

As you may see here, we have two tasks running. The result is enclosed in {}. All tasks are enclosed by [] and each task is enclosed by an additional {}

  • MultiCloud Audit: {“runningTasks”:[{“detailsURL”:”/dataservice/device/action/status”,”userSessionUserName”:”system”,”@rid”:1477,”tenantName”:”DefaultTenant”,”processId”:”7adf7476-3c9f-43e4-936b-222cedf3be68″,”name”:”Multicloud – Audit”,”userSessionIP”:”10.1.1.2″,”tenantId”:”default”,”action”:”multicloud_audit”,”startTime”:1638530152903,”endTime”:0,”status”:”in_progress”},
  • Upgrade vManage: {“detailsURL”:”/dataservice/device/action/status”,”userSessionUserName”:”admin”,”@rid”:1516,”tenantName”:”DefaultTenant”,”processId”:”software_install-adea5361-0c0b-470d-97b6-e4cd5ff7cfe5″,”userSessionIP”:”10.36.106.16″,”name”:”Software Install”,”tenantId”:”default”,”action”:”software_install”,”startTime”:1641406500245,”endTime”:0,”status”:”in_progress”}]}

Both tasks have a processId, so it seems we can kill the processId to remove the task. Let initiate a clean with a reference to that particular processId in the browser’s address bar:

Over to vManage and this is what we see in the task-list. Status has changed to failed and the task is no longer running.

That’s it!

:: Resources

https://www.cisco.com/c/en/us/support/docs/routers/sd-wan/214660-vmanage-api-call-clearing-stuck-task.html