908e (2nd gen) loosing data connection

[Anonymous]

My 908e 2nd Generation appliance is loosing data connection but not the voice part. I have updated the firmware and it will stay up and running for about 9:00 am and then now at 3:38pm it is down again.

Any ideas?

[jayh]

Could you post the configuration with passwords redacted, please?

[Anonymous]

I have included the two adtran units.

REMOTE

MAIN

REMOTE - >>>>>

!

!

! ADTRAN, Inc. OS version A5.02.00.E

! Boot ROM version 14.05.00.SA

! Platform: Total Access 908e (2nd Gen), part number 4242908L1

! Serial number CFG0937346

!

!

hostname "Eng"

enable password ********

!

clock timezone -5-Eastern-Time

!

ip subnet-zero

ip classless

ip routing

!

!

!

!

no auto-config

!

event-history on

no logging forwarding

no logging email

!

no service password-encryption

!

username “***” password “***”

username “**” password “**”

!

banner motd #

*********************************************

*          PROPERTY OF DELTACOM!!!!        *

*    UNAUTHORIZED ACCESS IS PROHIBITED!!!!  *

*      VIOLATORS WILL BE PROSECUTED TO      *

*          THE FULLEST EXTENT!!!!          *

*            CALL 800-239-3000              *

*********************************************

#

!

!

no ip firewall alg msn

no ip firewall alg mszone

no ip firewall alg h323

!

aaa on

ftp authentication LoginUseLocalUsers

!

!

aaa authentication login LoginUseTacacs group tacacs+

aaa authentication login LoginUseRadius group radius

aaa authentication login LoginUseLocalUsers local

aaa authentication login LoginUseLinePass line

!

aaa authentication enable default enable

!

!

no dot11ap access-point-control

!

!

!

!

ip dhcp-server excluded-address 192.168.101.48

!

ip dhcp-server pool "ENGINEERING"

  network 192.168.101.0 255.255.255.0

  domain-name "bhi.local"

  dns-server 172.16.48.3 172.16.5.2

  netbios-name-server 172.16.5.3 172.16.5.2

  netbios-node-type h-node

  default-router 192.168.101.48

!

!

!

!

!

!

qos map voip 10

  match dscp 26

  match dscp 46

!

!

!

ip flow top-talkers

!

interface eth 0/1

  description Terminals

  ip address  192.168.101.48  255.255.255.0

  ip flow ingress

  no shutdown

!

!

interface eth 0/2

  description PHONES

  ip address  192.168.102.48  255.255.255.0

  ip flow ingress

  no shutdown

!

!

!

!

interface t1 0/1

  fdl none

  tdm-group 1 timeslots 1-24 speed 64

  no shutdown

!

interface t1 0/2

  shutdown

!

interface t1 0/3

  shutdown

!

interface t1 0/4

  shutdown

!

!

interface fxs 0/1

  no shutdown

!

interface fxs 0/2

  no shutdown

!

interface fxs 0/3

  no shutdown

!

interface fxs 0/4

  no shutdown

!

interface fxs 0/5

  no shutdown

!

interface fxs 0/6

  no shutdown

!

interface fxs 0/7

  no shutdown

!

interface fxs 0/8

  no shutdown

!

!

interface fxo 0/0

  shutdown

!

interface ppp 1

  ip address  172.16.28.49  255.255.255.252

  ip flow ingress

  ppp multilink

  qos-policy out voip

  no shutdown

  cross-connect 1 t1 0/1 1 ppp 1

!

!

!

!

timing-source t1 0/1

!

!

!

!

!

!

!

ip route 0.0.0.0 0.0.0.0 172.16.28.50

!

no ip tftp server

no ip tftp server overwrite

ip http authentication LoginUseLocalUsers

ip http server

no ip http secure-server

no ip snmp agent

no ip ftp server

no ip scp server

no ip sntp server

!

!

!

!

!

!

no ip sip

ip sip udp 5060

no ip sip tcp

!

!

!

voice feature-mode network

voice forward-mode network

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

line con 0

  login authentication LoginUseLinePass

!

line telnet 0 4

  login authentication LoginUseLocalUsers

  password

  no shutdown

line ssh 0 4

  login authentication LoginUseLocalUsers

  no shutdown

!

!

!

!

!

end

MAIN -- >>>

!

!

! ADTRAN, Inc. OS version A5.02.00.E

! Boot ROM version 14.05.00.SA

! Platform: Total Access 908e (2nd Gen), part number 4242908L1

! Serial number CFG0913002

!

!

hostname "Res"

enable password ****

!

clock timezone -5-Eastern-Time

!

ip subnet-zero

ip classless

ip default-gateway 172.16.48.1

ip routing

!

!

!

!

no auto-config

!

event-history on

no logging forwarding

no logging email

!

no service password-encryption

!

username "" password ""

username "" password ""

!

banner motd #

*********************************************

*          PROPERTY OF DELTACOM!!!!         *

*    UNAUTHORIZED ACCESS IS PROHIBITED!!!!  *

*      VIOLATORS WILL BE PROSECUTED TO      *

*           THE FULLEST EXTENT!!!!          *

*            CALL 800-239-3000              *

*********************************************

#

!

!

no ip firewall alg msn

no ip firewall alg mszone

no ip firewall alg h323

!

aaa on

radius-server key ItcdSplt

ftp authentication LoginUseLocalUsers

!

radius-server host 66.35.170.169

radius-server host 66.0.46.198

!

aaa authentication login mylist group radius local

aaa authentication login LoginUseTacacs group tacacs+

aaa authentication login LoginUseRadius group radius

aaa authentication login LoginUseLocalUsers local

aaa authentication login LoginUseLinePass line

!

aaa authentication enable default enable

!

!

no dot11ap access-point-control

!

!

!

!

!

!

!

!

!

!

qos map voip 10

  match dscp 26

  match dscp 46

!

!

!

!

interface eth 0/1

  no ip address

  no shutdown

!

!

interface eth 0/2

  ip address  172.16.48.246  255.255.255.0

  shutdown

!

!

!

!

interface t1 0/1

  fdl none

  tdm-group 1 timeslots 1-24 speed 64

  no shutdown

!

interface t1 0/2

  shutdown

!

interface t1 0/3

  shutdown

!

interface t1 0/4

  shutdown

!

!

interface fxs 0/1

  no shutdown

!

interface fxs 0/2

  no shutdown

!

interface fxs 0/3

  no shutdown

!

interface fxs 0/4

  no shutdown

!

interface fxs 0/5

  no shutdown

!

interface fxs 0/6

  no shutdown

!

interface fxs 0/7

  no shutdown

!

interface fxs 0/8

  no shutdown

!

!

interface fxo 0/0

  no shutdown

!

interface ppp 1

  description p2p to ENG

  ip address  172.16.28.50  255.255.255.252

  qos-policy out voip

  mtu 1520

  no shutdown

  cross-connect 1 t1 0/1 1 ppp 1

!

!

!

!

timing-source t1 0/1

!

!

!

!

!

!

!

ip route 0.0.0.0 0.0.0.0 172.16.48.1

ip route 172.16.0.0 255.255.0.0 172.16.48.1

ip route 172.16.5.0 255.255.255.0 172.16.48.1

ip route 172.16.48.0 255.255.255.0 172.16.48.1

ip route 172.16.49.0 255.255.255.0 172.16.48.1

ip route 192.168.101.0 255.255.255.0 172.16.28.49

ip route 192.168.102.0 255.255.255.0 172.16.28.49

!

no ip tftp server

no ip tftp server overwrite

ip http authentication LoginUseLocalUsers

ip http server

no ip http secure-server

no ip snmp agent

no ip ftp server

no ip scp server

no ip sntp server

!

!

!

!

!

!

ip sip

ip sip udp 5060

no ip sip tcp

!

!

!

voice feature-mode network

voice forward-mode network

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

line con 0

  login authentication LoginUseLinePass

!

line telnet 0 4

  login authentication LoginUseLocalUsers

  password

  no shutdown

line ssh 0 4

  login authentication LoginUseLocalUsers

  no shutdown

!

!

!

!

!

end

[jayh]

Because the phones are working this rules out the T1 and upstream devices. I would look at the eth 0/1 interface for errors, also check your switches for bridging loops and possible spanning-tree issues.

Other potential problems are a device failing to negotiate DHCP and filling up the binding table.

Lots of questions below to help narrow it down:

When the problem occurs, can the data devices ping each other? Can they ping the gateway? Are the activity lights on your ethernet switches going crazy?

What do you do to restore it?

Are the phones and data devices on completely separate physical networks and not collapsed somewhere on the LAN?

I don't see a DHCP pol for the phones. Are they statically assigned?

Possibly a rogue DHCP server on the data side? When it fails, do the workstations have IPs in the correct range with the correct gateway?

When it fails, do all of the terminals fail at the same time?

Does show ip dhcp binding reflect anything unusual?

How about show int eth 0/1 ?

[Anonymous]

One of the things that I did was to swap and reconfigure the 908e's.

Take the one from ENG and RSB and swapped them, reconfigured and went live. We were good for a few hours and then, we are now loosing connection. This time we are totally dropping the connection. The ENG side stays up and the  RSB drops. When this happens, I can still communication with the remote ADTRAN and the local ADTRAN.

I did the show int eh 0/1 and it did not show errors.

the show ip dhcp-server binding shows the clients and all correct ip addresses.

I believe it's the ADTRAN failing, as now, when I did the swap, I only use in the MAIN building eh 0/1 and wan and on the remote I use eth 0/1 and 0/2, 0/1 - data      0/2 voice.

Now, we are loosing connection for data and voice.

I am watching the ADTRAN on the remote side and I am now getting

BES 15 min threshold exceeded

PCV 15 min threshold exceeded

LES 15 min threshold exceeded

SEFS 15 min threshold exceeded

DM 15 min threshold exceeded

ES 15 min threshold exceeded

LCV 15 min threshold exceeded

I just lost connection when this appeared and I am still able to telnet into the unit.

To rectify the problem, I reboot the ADTRAN on one end  and that gets the connection back. I also, will reload the t1 and that will bring back the connection.

This last time, I waited 15 minutes and the connection came back online. I have the telephone company do a test on the T1 and they reported a clean line no errors.

Thanks...

[jayh]

You have "timing-source t1 0/1" on both units, with a point-to-point link between them. Each unit thinks the other side of the link is an accurate clock and tries to sync to it. Periodically they drift together to where the PLLs can't keep up and the T1 goes out of sync, loses frame, and blows up.

On one unit (not both!) configure "timing-source internal". This will cause it to use its internal crystal-controlled source to clock the T1 between them. The other unit will recover this clock from the T1 (timing-source t1 0/1) and you will be much happier. It makes no real difference which one is internal, unless there are other T1 links involved besides the one between the two units.

[Anonymous]

Thanks for the reply. I am going to adjust the main ADTRAN and see how it performs tomorrow.

Thanks..

[Anonymous]

Morning Jayh, I made the change that you suggested but unfortunately, the problem was still there. I then went ahead and started to research the routing and removed a couple of redundant routes and once I did that and saved the configuration it came back online. I am not able to browse the network at this time.

Researching the problem I discovered that the users were able to get to our INTRANET and outside WEBSITES, but anything that had to do with a network share and resource was not available to them.

1) Browsing

2) Printing

The other interesting thing that I found was that we were using the 908e to push voice not VOIP and that always stayed up as this was on a different subnet and did not require network authentication to access.

I will keep an eye on both of them.

Thanks for all your great suggestions and I have left the timing in place as you suggested.

[petetransitguy]

If the 908 supports this command, give it a shot (works great on the NetVanta)

"show interface t1 0/1 realtime"

An earlier post refers to the T1 timing & I would add to check that setting for all remote sites. It's a good idea to make your main site the source and the other sites derive T1 timing from there.

Headquarters side (first ensure that all other remotes are set to "line"):

configure terminal

interface t1 0/1
timing-source internal

Here are some troubleshooting suggestions for the remote side since it may be an IP issue. Excluding these addresses should prevent another device getting them via dhcp:

configure terminal

ip dhcp-server excluded-address 192.168.101.255
ip dhcp-server excluded-address 192.168.102.48
ip dhcp-server excluded-address 192.168.101.255

ip dhcp-server excluded-address 172.16.28.50

Next, set the dhcp lease to 1 day

ip dhcp-server pool "ENGINEERING"

lease 1

After verifying the HQ site is "timing-source internal," make this change:

interface t1 0/1
timing-source line
exit

Consider bullet-proofing the ppp connection by hard-setting the peer:

interface ppp 1
peer default ip address 172.16.28.50
exit

This last step is just another way to route outbound via the ppp.

no ip route 0.0.0.0 0.0.0.0 172.16.28.50

ip route 0.0.0.0 0.0.0.0 ppp1

HTH