Home > Lync, UC > Centralized Logging Service not working in Skype for Business

Centralized Logging Service not working in Skype for Business

I haven’t written a new post in a while as I have not been doing as many deployments lately. I have been focused more on evangelism and speaking with different organizations that are thinking about deploying Skype for Business.

That said, recently I had the opportunity to get back in to the game and start a deployment for a large organization. We were experiencing some issues that necessitated us to do some logging on the Front-end servers. I turned to my favorite tool for this, the CLSLogger which takes advantage of the Centralized Logging Service (CLS).

I would start up CLSLogger and when I attempted to start the scenario, I would get an error like this:

WARNING: Failed on 1 agents
Agent - mediation1.domain.com, Reason - Error code - 20000, Message - Unknown error - 
Error calling agent mediation1.domain.com; Could not connect to 
net.tcp://mediation1.domain.com:50001/. The connection attempt lasted for 
a time span of 00:00:02.0228175. TCP error code 10061: 
No connection could be made because the target machine actively
 refused it . Please refer CLS logs for details.

When I went to the server and did a netstat, I saw that CLS was not actively listening. I should have seen the system listening on ports 50001-50003.

PS C:\Windows\system32> netstat -an | findstr 5000*
  UDP            *:*
  UDP           *:*
  UDP    [::]:500               *:*
  UDP    [::]:4500              *:*

I went round and round with this and ultimately had to open a case with Microsoft on it. After doing some traces and pulling the ETL logs, the Microsoft Engineer got back with and asked how we had generated the certificate. He was seeing the following message in the ETL log (which us mere mortals don’t have the ability to read and the real reason I’m writing this blog article):

29 TL_ERROR(TF_COMPONENT) [3]14CC.1728::06/08/2016-16:53:00.193.00000010 
(CLSAgent,CommandProcessor.Initialize:commandprocessor.cs(247))Exception - 
[System.ArgumentException: It is likely that certificate 
'CN=mediation.domain.com, OU=IT, O=Domain, C=US' 
may not have a private key that is capable of key exchange or the process 
may not have access rights for the private key. 
Please see inner exception for detail. ---> System.Security.Cryptography.CryptographicException: 
Invalid provider type specified.

We then looked at the certificates installed on the machine by running “certutil -store my”. (I’ve purposfully deleted identifying information and highlighted in red the Provider which is the key piece of information.)

PS C:\Windows\system32> certutil -store my
my "Personal"
================ Certificate 0 ================
Serial Number: 5xxxxxx
NotBefore: 4/15/2016 11:29 AM
NotAfter: 4/15/2019 11:59 AM
Subject: CN=, OU=IT, O=Domain, C=US
Non-root Certificate
Cert Hash(sha1): 39 2d..... 
  Key Container = le-360ab342
  Unique container name: 5d26bec417ed43b7840b7bf82c2fb363
  Provider = Microsoft Software Key Storage Provider
Encryption test passed
CertUtil: -store command completed successfully.

When we talked to the group that generated the certificate, we found that they used their own template instead of the Wizard in Skype for Business. While you certainly don’t have to use the Wizard, Skype for Business definitely has some requirements on the certs that will work with it. As an example, going to a key length longer that 256 usually doesn’t work out too well. In this case, the Provider was what was wrong.

I then turned to the Digicert Utility, one of our other favorite tools to generate the Certificate Request (CSR). This then utilized the correct Provider which is the Microsoft RSA SChannel Cryptographic Provider. After we issued the new cert and assigned it, we restarted the servers and checked the certs.

PS C:\Windows\system32> certutil -store my
my "Personal"
================ Certificate 0 ================
Serial Number: 5xxxxxx
 NotBefore: 6/9/2016 12:02 PM
 NotAfter: 6/9/2019 12:32 PM
Subject: CN=, OU=IT, O=Domain, C=US
Non-root Certificate
Cert Hash(sha1): 3c e2....
  Key Container = 70C232....
  Unique container name: 9ed77....
  Provider = Microsoft RSA SChannel Cryptographic Provider
Encryption test passed

We also started seeing CLS listening as expected:

PS C:\Windows\system32> netstat -an | findstr 5000*
  TCP              LISTENING
  TCP              LISTENING
  TCP              LISTENING
  TCP    [::]:50001             [::]:0                 LISTENING
  TCP    [::]:50002             [::]:0                 LISTENING
  TCP    [::]:50003             [::]:0                 LISTENING
  UDP            *:*
  UDP           *:*
  UDP    [::]:500               *:*
  UDP    [::]:4500              *:*

I hope this helps someone else who might end up in this situation.

Categories: Lync, UC
  1. soder
    July 26, 2016 at 12:11 pm

    “This then utilized the correct Provider which is the Microsoft RSA SChannel Cryptographic Provider”

    No, no, and no! 🙂 Dont spread misleading instructions and BAD PRACTICES on your blog please. Microsoft and all big IT companies (especially browser vendors like Google Chrome, Mozilla Forefox) remove the support for SHA-1 hash algorithm. I translate this: SHA-1 WONT WORK in your browser and in the base OS! In the coming weeks! Not sometime in the distant future, but literally in the coming weeks of 2016! So recommending anybody to still use SHA-1 based certificates is a very very irresponsible advice! How does all this connect to your post then?
    The MSFT Crypto Service Providers (CSP) -including the RSA SChannel CSP- dont support Crypto Next Generation. Only the newer Key Storage Provider (KSP) certificates support CNG. CNG contains the successor of SHA-1, namely the SHA-2 algorithm. It is STRONGLY ADVISED everybody to migrate to SHA-2 from SHA-1. For this, you must also migrate from the older CSP setup to the newer Key Storage Provider setup on the certificate server.

    So in short, dont recommend anybody to use CSP-based certificate templates, but recommend KSP-based templates only!

    Further reading: Just google for ” SHA-1 to SHA-2 migration” keywords, or check this:

    • July 26, 2016 at 1:16 pm

      Yes, people should only be using SHA-2. In this case, the organization’s internal PKI had not been completely upgraded yet.

      If this had been a public certificate, most of the vendors are automatically just replacing any SHA-1 request with a SHA-2 from what I am seeing.

      Lastly, the part of the post that I was trying to make was that if you were seeing this error, it could be a certificate issue. The error message doesn’t give you any hints towards that. This post was not an endorsement of any specific PKI component other than make sure you are using a cert that is supported by Skype for Business.

      Thanks for your comment and contribution. Always good!

      • soder
        July 26, 2016 at 3:27 pm

        I know this wasnt a PKI post. But when people find your blog regarding this issue, and they blindly apply the fix that worked for you, without translating the steps into their own environment, they will fall into the trap of generating sha1. Which simply wont work when Msft release the “kill switch” update on WU in any minute.
        Only reason i put so much effort into explaining this to you, that when next time I am stuck and find any random blog post about a config i am not expert in, I dont have to worry about the steps I will follow, as the ownet of the post doesnt write stupid things 🙂

      • soder
        July 26, 2016 at 3:33 pm

        Ah sorry 1 more damn important thing: SHA1 is still supported in Sfb, so technically speaking you dont break the supportability situation by applying sha1 hashing in the certificate. On the other hand it will be not Sfb but the base underlying OS that will bleed badly. So while your back is covered (as you can keep pointing fingers to the Sfb technet supportability docs) in reality it was a poor choice when you went the road of sha1.

  2. August 15, 2017 at 1:08 am

    I experienced the same issue with the same error message.
    The server was not listening on forts 50001-3
    I just restarted the Centralized logging service and confirmed with netstat that the server started listening on port 50001-3.
    CLS Logger worked as a charm after the restart of the service.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: