Insufficient bandwidth at the phone - not server

[rombrigs]

There is definitely an underlying issue that has not been identified.  I requested performance metrics from the service provider this morning, I have a feeling packet loss is occuring.

Cables to the phones was the first thing checked, they all passed.  Programming of the switch is correct; I rebooted last night.  QOS settings are good and frankly there is not enough traffic to cause the symptoms they are experiencing.  All settings have been triple checked and are identical to other sites that have no problems.  I may do a firmware upgrade and reset to factory defaults of the switch over the weekend.

I would like to try modifying the call configuration on a few phones, for test purposes and to see if it alleviates the disconnected/poor quality calls until I can diagnose the real problem.

Unfortunately, my Mitel vendor is competent enough to get the system running and do basic programming, but anything out of the box is "we will contact Mitel support" and nothing gets done.

Thanks again for your time.

[Tech Electronics]

Rombrigs,

Hopefully you get your issue resolved soon, but if you need any more information than just post your question back here.

Heritage02Rider,

Are you still on the straight and narrow or do you need some help still with your issue?

Thanks,

TE

[Heritage02Rider]

Heritage02Rider,

That will teach me to try and explain from memory, especially since I am not getting any younger, and you are correct it is Operations not Tools to get to the error logs; at least you found them and that is what matters.

Now, I don't know what kind of pipe you have at both ends between the US and Mexico, but let's just try a little experiment with your codecs before you start changing a lot of stuff around.

According to your Network Diagnostics File you are running G.711 for your remote phones so let's knock that down to G.729 and see if the frequency at which this happens either lessens or goes away all together; I will bet on it being reduced with some instances still occuring.

Go to System > IP-Related Information > IP Connections > Remote {or whatever your Mexico Phones are using}

Now, that I look at your Diagnostics again it looks as though you may have you Mexico Phones as local phones since it is showing 2 audio frames per packet. Hmm, you may just want to change your Mexico Phones over to the Remote IP Configuration or at least seperate them from the Local ones so you can change the Codec without affecting all the local phones as well. If you need help with this let me know and I can walk you through it.

Thanks,

TE

I did have them set as local.

I have moved them to the Remote and G.729.

I will monitor and get right back. I literally got a insufficient from the remote phone a moment before I changed the setting. So I will probably know today.

[Heritage02Rider]

Heritage02Rider,

That will teach me to try and explain from memory, especially since I am not getting any younger, and you are correct it is Operations not Tools to get to the error logs; at least you found them and that is what matters.

Now, I don't know what kind of pipe you have at both ends between the US and Mexico, but let's just try a little experiment with your codecs before you start changing a lot of stuff around.

According to your Network Diagnostics File you are running G.711 for your remote phones so let's knock that down to G.729 and see if the frequency at which this happens either lessens or goes away all together; I will bet on it being reduced with some instances still occuring.

Go to System > IP-Related Information > IP Connections > Remote {or whatever your Mexico Phones are using}

Now, that I look at your Diagnostics again it looks as though you may have you Mexico Phones as local phones since it is showing 2 audio frames per packet. Hmm, you may just want to change your Mexico Phones over to the Remote IP Configuration or at least seperate them from the Local ones so you can change the Codec without affecting all the local phones as well. If you need help with this let me know and I can walk you through it.

Thanks,

TE

You are correct - all my phones were listed as Local. I did move the affected phones to the Remote configuration. They should now be doing G.729. The lower bandwidth requirement should provide less dropouts. I will see if the call quality suffers, but I will assume it will be negligible.

This one is obviously a wait and see, so I will post back next week after we let it sit for a while.

As for the local - still have not see an INSUFF BANDWIDTH show up since the change to the switches.

Thank you for all the help! It has been detailed and informative.

Karl

[Tech Electronics]

Heritage02Rider,

Well I hope your problem with this issue is solved, but we will see. Anyway, I was glad to help out where I could.

Thanks,

TE

[rombrigs]

Still having problems, so to change these phones to remote/G729 is it as simple as changing the call configuration to "2" or is there more to it?

Thanks!  -R

[Tech Electronics]

Rombrigs,

No, that is all there is to it for your setup.

Thanks,

TE

[Heritage02Rider]

Follow up reply:

RE: Remote phones
Have not gotten a notice about bandwidth from the remote phone yesterday or today, but it appears they only made two calls. I will call them to see what is up - and perhaps a lengthy enough call to determine if we get a drop.

RE: Local phones
The INSUFF BANDWIDTH has been aggressive yesterday and today. Two phones today at exactly the same time today. So I checked the log and they were both receiving incoming calls. (not talking to each other) I am perplexed here. I cannot imagine that they are really having not enough bandwidth for a call. The one phone, which has had several since Friday had one occur at 5:17 pm. Basically 3/4 of the people had gone for the day. No possible bandwidth issue should be occurring.

To add to the frustration, I received an Insufficient Bandwidth notice from our SIP trunk as well. SO I decided to monitor the link and another occurred. We were utilizing about 1% of a 20 Mbps Ethernet MPLS circuit at the time the notice went out.

So... is it possible that the messages are phantom? The users are complaining that they get a brief busy signal, so something real is happening.

I will be contacting my phone vendor, but I already assume they will say its my network and that I need to call in someone else. I just cannot believe that it is the HP switches that are the weak link.

For reference - yesterdays Freeze - I see nothing indicating a problem:
Insufficient Bandwidth: 03-24-2014 at 11:45 AM
   IP Station Extension: '2167' at IP Address 192.168.200.21 on Board 7
   VoIP Configuration Information (Audio Session 110):
      VoIP DSP Number: 110
      Audio Frames Per Packet: 2
      In-Time Frame Percentage Threshold: 60%
      In-Time Frame Timeout: 5 secs
      Audio TX IP Address/Port: 192.168.200.21:50228
      Speech Vocoder Type: G711 Mu-Law
      Speech RTP Payload: G711 MU-Law
      Other Device System Echo Canceller Profile: No Echo
      Echo Suppression: FALSE
      Echo Saturation: TRUE
      Echo Suppression Sensitivity Level: 0
      Conn Dir: Two-Way; Call Session: 2167 (GW) 94072
   IP Audio Quality Snapshot Values:
      Reporting ICP IP address: 192.168.200.201
      Reporting IP Address: 192.168.200.21
      Reporting Signalling Port: 6993
      Reporting Voice Port: 50228
      Far End IP Address: 192.168.200.203
      Far End IP Port: 6824
      RX Codex: 1
      Delay: 0
      Jitter Instant: 0
      Jitter Average: 0
      Jitter Buffer Overflow: 0
      Jitter Buffer Underflow: 0
      Jitter Buffer Average Depth: 20
      Jitter Buffer Max Depth: 20
      Snapshot Packets Lost: 0
      Packet Loss Max Burst: 0
      Packets Out of Order: 0
   IP Audio Quality History:
      Packets Lost: 0
      Packets Received Total: 18250
      Jitter Buffer Histogram:
        10-19ms:   0
        20-39ms:   0
        40-59ms:   0
        60-99ms:   0
        100-200ms: 0
        200+ms:    0
      Packet Loss Histogram: consecutive packets
        1:    0
        2:    0
        3-5:  0
        6-10: 0
        10+:  0
   Audio Samplings:
   Rec:    Time   Jitter   Pck Lst   Pck Rx   InTime%
   1   5   0   0   250   100
   2   5   0   0   250   100
   3   5   0   0   250   100
   4   5   0   0   250   100
   5   5   0   0   250   100
   6   5   0   0   250   100
   7   5   0   0   250   100
   8   5   0   0   250   100
   9   5   0   0   250   100
   10   5   0   0   250   100
   11   5   0   0   250   100
   12   5   0   0   250   100

[Tech Electronics]

Heritage02Rider,

According to this it is not the phone that is having the issue as it is showing no lost packets or drops in service, but is 192.168.200.203 your SIP trunks?

Not to be nosey, but if your switch is an A-series 5120 are you sure that you set up QoS properly for your Voice VLAN?

Thanks,

TE

[Heritage02Rider]

Well, I am not going to discount that. I am not an HP expert, but I follow instructions fairly well. After pondering just that same idea, I can say that we have not had the problems prior to implementation of our SIP trunk. (Freud says that 'sometimes a cigar is just a cigar')

Regarding the connection between the MPLS and the core switches, I will investigate the QoS. To the end-user I believe it to be correct.

I am thinking if it were solely the SIP trunk to be the issue, I should be getting the insufficient bandwidth message at the MITEL switch rather than the end-user and it should be the connection to the SIP trunk in the alert; correct? I have had these messages, but they are never (or rarely) coincidental.

[Tech Electronics]

Heritage02Rider,

Since you do not have the Insufficient Bandwidth Alarm enabled on the System Flags it will not show at the switch it will always show at the phone where the issue is currently happening whether it is the phone or not.

At this point I would be looking at the SIP Trunk Configuration and downloading Error Logs to see if there is anything going on. Prior to that though I would make sure that the Expanded Message Print was turned on.

Turn on Online Monitor View > Online Monitor hit ok when it tells you to not do this.
Then go into the Maintenance and turn on Expanded Message Print. System > Maintenance > Message Print > Print Expanded Message Print > YES.
Now, turn off Online Monitor View > Online Monitor.
Now download Error Logs Operations > Error Information select both history and message print.

This is all the same stuff you have been downloading except instead of just looking at the .ndl file we want to look at the Message Print File.

Thanks,

TE

[WallIT]

Hi,

I don't wish to hijack this thread and will be happy to start a new one if asked, but have been reading with interest. I noticed this problem happens at our site rarely, but enough to bother me. We are 60 users, one site, using Mitel 5000 for a few years. Having just looked at the ndl logs we have had a number of 'insufficient bandwidth' issues affecting users over the past few months. This is not happening frequnetly enough that users are reporting it, but I am aware it's going on and would prefer to reduce this as much as possible.

Phones operate in their own VLAN (5).

I checked our network settings, which are:

DHCP
We are using option 43 only id:ipphone.mitel.com;sw_tftp=192.168.50.10;call_srv=192.168.50.10;vlan=5;l2p=6;dscp=46
We are not using option 125

Switches
Using Netgear FS728TP (hopefully being replaced soon). Under QOS menu...
QOS is enabled and using trust mode DSCP
COS (cost of service?) interface config shows each port as using a default COS of 0 (the options are 0-7)
Queue = strict priority (alternative option is WRR)

COS to Queue Mapping...
0 Low
1 Lowest
2 Lowest
3 Low
4 Normal
5 Normal
6 High
7 High

DSCP to Queue Mapping...

See attachment.

Mitel 5000
IP Connections...
P6000 = Audio RTP = 0, Data Type = 0
P6001 = Audio RTP = 0, Data Type = 0

From reading through this thread, the first thing I notice is my IP Connections settings are incorrect. Is it sufficient to just set this to 184 for both values? and for both P6000 and P6001?

Also, I'm confused about our Netgear switches. If DSCP is set to 46, but no 46 exists in the switch DSCP table, should I change the DSCP value the phones receive? If so, can I choose any number which has a DSCP value of 'high' (such as 63)?

Thanks in advance.

[Tech Electronics]

WallIT,

Your switch and phones are using DSCP which is a high priority, but your Class of Service should be either a 6 or 7; normally 6.

Option 34 and 125 are the same in this instance for your case as they provide the same information for the phones.

Thanks,

TE

[WallIT]

Thanks TE

Your switch and phones are using DSCP which is a high priority, but your Class of Service should be either a 6 or 7; normally 6.

Isn't it one or the other? In my switch, under the QOS menu, I have QOS mode set to enabled, and beneath that, trust mode = DSCP. The other choice is CoS, suggesting I need to use one or the other.

Also, should update my Mitel IP Settings as mention in my previous post? Or leave as is (both set to 0)?

Thanks. Much appreciated.

[Tech Electronics]

WallIT,

Since I do not know your switch I am not sure why it wouldn't allow for both since DSCP is normally for data packets leaving the switch and WAN side queueing. The CoS is for internal routing priority on the individual switch ports on that switch.

As for the setting for the P6000 and P6001 that is for the Controller talking on the WAN (DSCP), but if you want to set it then it will not hurt anything on your system.

Think CoS is Layer 2 and DSCP is more Layer 3, not a good analogy but it works in this instance.

Thanks,

TE