Cisco Prime License Manager inaccessible

When Cisco PLM is inaccessible the issue could show up after ipsec cert revalidation. Even when following process and service DRF was restarted – solution is to do it again.
Cisco DRF Master -> restart
Cisco DRF Local if required -> restart
On server when PLM works, usually on publisher

SIP call packets statistics

I have realized that useful option must be configured comparing ISR 4k to ISR G2.
By default in ISR G2 P-RTP-Stat field is present to collect media statistics.

BYE sip:343456678@10.7.0.20:5060 SIP/2.0
Via: SIP/2.0/UDP 10.7.0.10:5060;branch=z9hG4bK18670D1639
From: ;tag=D2838B18-126F
To: "John Doe" ;tag=1833309~ed2b7359-615c-4669-b217-c48cf84700a0-40060401
Date: Mon, 02 Sep 2018 14:22:15 GMT
Call-ID: 6fb9dd80-10001-3bb1a-93f492a@10.7.0.20
User-Agent: Cisco-SIPGateway/IOS-15.4.3.M7
Max-Forwards: 70
Timestamp: 1568054745
CSeq: 101 BYE
Reason: Q.850;cause=16
P-RTP-Stat: PS=927,OS=232720,PR=865,OR=230400,PL=0,JI=0,LA=0,DU=9
Content-Length: 0

If you need to migrate configuration to ISR 4k to have P-RTP-Stat header for IOS-XE the following commands need to be entered in order to collect and send the RTP stats inside of BYE messages:

voice service voip
media bulk-stats
media stats-disconnect

TMS and phonebook issue

When I wanted to upload new phonebook error was shown.

Unable to communicate with the Provisioning Extension. See log (log-web.txt) for details.

It looked like all is fine. File is uploaded and accepted but final version was not changed. When I checked directory on server, there was many files so uploading was OK. In logs I found that the problem can be with Cisco TMS Provisioning Extension service.

Fast check and what I found the the service wasn’t started, probably after last restart.

Restart done and nothing changed. Because in my case TMS is in cluster I checked secondary server. It was running but it is cluster I also restarted that service on secondary service.

Cluster-wide service restart solved the problem. New phonebook was uploaded and at the same moment visible at all video terminals.
Phonebooks local directory is C:\Program Files (x86)\TANDBERG\TMS\wwwtms\data\ExternalSourceFiles\

Debugging Q931 – one tip

When I started working at telephony technology first it was easy to debug calls using Q931. I analyzed small systems with small number of calls, so there was no problem to follow one call entire debug.
One day I had to debug a huge cluster with gateway with 4 E1 interfaces and huge amount of calls. Using filters loaded CPU too much so the unique solution was to grab all logs.
The question was “How should I know which line of logs I need to take regarding a call that is interesting for me.
Extracted part of log for one call below:
So first of all we need to catch first part of call with TX -> SETUP, Calling Party Number and Called Party Number. The key info is callref in our case equal 0x1517.

Oct 3 13:18:19.659: ISDN Se0/1/2:15 Q931: TX -> SETUP pd = 8 callref = 0x1517
Sending Complete
Bearer Capability i = 0x8090A3
Standard = CCITT
Transfer Capability = Speech
Transfer Mode = Circuit
Transfer Rate = 64 kbit/s
Channel ID i = 0xA98395
Exclusive, Channel 21
Calling Party Number i = 0x0083, '49928853291'
Plan:Unknown, Type:Unknown
Called Party Number i = 0x80, '44344112683'
Plan:Unknown, Type:Unknown
Oct 3 13:18:20.130: ISDN Se0/1/2:15 Q931: RX <- CALL_PROC pd = 8 callref = 0x9517
Channel ID i = 0xA98395
Exclusive, Channel 21
Oct 3 13:18:20.144: ISDN Se0/1/2:15 Q931: RX <- PROGRESS pd = 8 callref = 0x9517
Progress Ind i = 0x8188 - In-band info or appropriate now available
Oct 3 13:18:23.760: ISDN Se0/1/2:15 Q931: RX <- ALERTING pd = 8 callref = 0x9517
Oct 3 13:18:33.109: ISDN Se0/1/2:15 Q931: RX CONNECT_ACK pd = 8 callref = 0x1517
Oct 3 13:18:49.370: ISDN Se0/1/2:15 Q931: RX RELEASE pd = 8 callref = 0x1517
Oct 3 13:18:49.396: ISDN Se0/1/2:15 Q931: RX <- RELEASE_COMP pd = 8 callref = 0x9517

You can see callref is 64bits id in HEX. Within all logs our call transmitting part is with callref = 0x1517. First digit for receiving part is with 8 added. So RX callref is 0x9517.
That’s all.

How to audit CUCM changes performed by admins

Starting CUCM 10.x version as I remember there is new feature at RTMT.
It is possible to view all changes that has been done at CUCM. Not everything, not all details, but could be useful to be able see what was done by whom and when. There are no previous values neither post change values. Item type or name is good enough.

Go to RTMT->Tools->AuditLog Viewer.
Select a Node, cloud be Publisher is enough, of course.
AuditApp Logs and chooselog file.
That’s all and some example below.

09/1/2017 09:55:07.822, adminjoe, 10.1.2.3, Notice, GeneralConfigurationUpdate, CUCMAdmin, Success, No, AdministrativeEvent, Cisco CUCM Administration, record in table applicationuser with key field name = ctiphoneuser updated, EXTENSIONMOBILITY, , SERVER-CUCM, 243
09/11/2017 09:55:33.013, adminjoe, 10.1.2.3, Notice, UserRoleMembershipUpdate, CUCMAdmin, Success, No, AdministrativeEvent, Cisco CUCM Administration, Application user ctiphoneuser removed from usergroup Standard CTI Enabled, EXTENSIONMOBILITY, , SERVER-CUCM, 247
09/11/2017 09:55:33.017, adminjoe, 10.1.2.3, Notice, UserRoleMembershipUpdate, CUCMAdmin, Success, No, AdministrativeEvent, Cisco CUCM Administration, Application user ctiphoneuser removed from usergroup Standard CTI Allow Call Monitoring, EXTENSIONMOBILITY, , SERVER-CUCM, 248
09/11/2017 09:55:33.031, adminjoe, 10.1.2.3, Notice, UserRoleMembershipUpdate, CUCMAdmin, Success, No, AdministrativeEvent, Cisco CUCM Administration, Application user ctiphoneuser removed from usergroup Standard CTI Allow Call Recording, EXTENSIONMOBILITY, , SERVER-CUCM, 249
09/11/2017 09:55:33.053, adminjoe, 10.1.2.3, Notice, UserRoleMembershipUpdate, CUCMAdmin, Success, No, AdministrativeEvent, Cisco CUCM Administration, Application user ctiphoneuser removed from usergroup Standard CTI Allow Control of Phones supporting Connected Xfer and conf, EXTENSIONMOBILITY, , SERVER-CUCM, 250

UCCX RmCm subservice stuck initializaing

When issue occurs with stuck RmCm subservice there could be a few reasons but there is less points to check.
That service is responsible for communication between CUCM and UCCX using JTAPI protocol. Sometimes first step is to check JTAPI resynchronization or Route Points registration at CUCM.
If it looks fine but issue still exist don’t waste time, just check rmcm jtapi user at two points.
First – check or even change app user password from UCCX level. It will update app user at CUCM automatically.
Second – check user roles and rights.
Sounds strange but it can happen that someont will clear some role by mistake. Afer clearing all those small issues RmCm subservice shloud start to work properly. If not just restart CCX Engine service.

UCCX updates against CUCM

Sometimes the simplest solution can be done, sometime you don’t want to dig too deep or you have no enough time for such troubleshooting. When fast solution doesn’t work I bet you pretty often going to try “undone and done”, “unconfigure and configure” etc.
I found out this is good at one case. Phone and device profile were assigned to RM JTAPI application user as it should be. The issue was that phone behave as not being assigned to RM JTAPI.
I unassigned and assigned again, but it didn’t helped. After some time I talked to me colleague and the news was that UCCX updates settings with CUCM every 180s.
Remembering issue with UCCX agent when similar case happned I tried to unassigned phone from RM JTAPI user and asssign it back after time longer than mentioned 180s. Hurray – it helped.
Difference in solution time, in those two cases was huge.

Prime Collaboration Provisioning and many domains

Configuring Prime Collaboration Provisioning for centralized CUCM cluster and many domains there was an issue with access for specific users to specific domains.
The problem was delegation parameter value. Implementing a few versions of licenses didn’t change a lot. It was always equal 1.
At the end solutions was to delete standard.lic file.
After procedure as below license delegation value changed to 2.

Normally we will see delegation count as 1 when standard license is installed. With standard.lic file installed, this count will always be 1

even though other additional licenses are installed. So, if the count is showing as 1 , even after you install the advanced licenses, we need to delete the standard.lic file.

Steps for deleting the standard.lic:

Login to PCP using SSH as root

Go to path : /opt/cupm/license
cd /opt/cupm/license

List the contents of the folder to see the licenses files present.
ls -la

You may see other license files also including the standard.lic file.

Take a backup of standard.lic file by the command: cp standard.lic standard.lic.bak
cp standard.lic standard.lic.bak

List the contents again to ensure you have the copy(standard.lic.bak) saved.
ls -la

Remove the standard.lic file : rm standard.lic
rm standard.lic

Debug and less logs by filtering

To debug an issue it is always easier to focus on our source and destination. No one likes to dig into a lot of unwanted text.
I found out this by an accident and had forgotten about that a few time. Solution is simple. I show only an example.

configuration as below

call filter match-list 1 voice
incoming calling-number 442248123

and enable debugging (example)

debug condition match-list 1 exact-match
debug ccsip calls

what you will see is

show debug
CCSIP SPI: SIP Call Statistics tracing is enabled (filter is ON)

Just short, simple and so useful. You will see in logs only what you need. To match by other values just see documentation.

Matching dial-peer based on calling number for outgoing calls

I had problem matching IP phone with FXO assigned to it.

There is no short nor fast way to configure it. What I found is as below:

1. Match calling number with translation profile

dial-peer voice 37 voip
 translation-profile incoming FXO_17
 answer-address 918852217

2.  Profile uses translation for called number

voice translation-profile FXO_17
 translate called 27

3. Translation changes prefix to unique for this action purpose

voice translation-rule 27
 rule 1 /^0/ /57/

4.Last action is to match specific prefixed extension with port in that case FXO

dial-peer voice 47 pots
 destination-pattern 57T
 translate-outgoing called 27
 port 0/2/3