OLAN PON Protection

Introduction

Document Number

ENG-010459

Purpose

The purpose of this document is to explain how GPON optical protection works and define the features and capabilities of the Tellabs Type B PON protection switching.  The GPON system and type B protection are defined by the G.984.1 document.

Definitions

PON - Passive Optical Networking.  PON is a point-to-multi-point architecture with passive infrastructure supporting distances of up to 20 km.  The infrastructure between the OLT and ONT is all optical and completely passive with no electronics in between the two.

Primary PON - The PON that by default carries all the traffic when no failures exist in the system in Revertive PON Protection groups(QOIU7 only).  This link has no special significance for non-revertive PPGs(OIU8).  There is a slight preference for the primary on startup in non-revertive systems, but that is all.

Secondary PON - The secondary PON is the PON that will carry the traffic in the event of a failure on the Primary PON for revertive PONs(QOIU7).   This link has no special significance for non-revertive PPGs(OIU8)

Active PON - The active PON is the PON which is currently carrying traffic.  

Standby PON - The standby PON is the PON which is not carrying traffic and in the normal case is providing standby protection for the Active PON.

Type B PON Protection - Type B PON protection is a part of the PON standards and allows the OLT, and OLT connections to be redundant and allows high reliability connections over the PON.

Revertive Switching - In revertive switching, there is a preferred PON and when the Primary PON failures clear, the traffic will switch back from the Secondary PON to the Primary PON.  In the normal state traffic will always be on the Primary PON.  The QOIU7 supports revertive switching only.

Non-Revertive Switching - When non revertive switching is employed, there is no preference for a particular link and the traffic will run on the Active link until a failure occurs and then will switch to the Standby link.  It will not switch back on the failure being cleared on the previously failed Link.  The OIU8 only supports non-revertive switching.

PPG - PON Protection Group. The Protection Group consists of a primary PON and secondary PON used to protect traffic in the event of an OLT failure.

Intra-OLT PON Protection - When type B PON protection is implemented within a single OLT for cost savings.  It is not the preferred configuration as some OLT hardware in the path is not redundant.  

Figure 1 Intra OLT Type B protection

Inter-OLT PON Protection - When type B PON protection is implemented between two OLTs which provides full redundancy of all OLT hardware, uplinks and PON.  This is the preferred configuration for reliability reasons and is the option recommended by Tellabs.

 

Figure 2 Inter OLT Type B protection

Applies To

This document applies to Type B PON protection on the Tellabs 1131, 1134, and 1150 OLTs and ONTs.

Tellabs Type B PON Protection

Type B protection makes use of a 2:N splitter which allows two inputs on the OLT side of the circuit(OLT to splitter side).  From the splitter to the ONT is a single fiber going to each of the ONTs on the PON.  This gives both equipment and facility protection for the OLT side of the PON. With Type B PON Protection one can accomplish OLT PON fiber path redundancy to the communication closet or zone box (e.g. 2:32 splitter).

PON Protection interfaces can be:

• Intra-OLT PON Protection: On two ports on different PON cards within the same OLT
• Inter-OLT PON Protection: On two ports on different OLTs.

This allows the user to control what level of redundancy they require based on what makes the most sense in terms of a tradeoff of cost/benefit, usually driven by the importance of the data flowing over the PON network

Tellabs supports the G.984.1 Type B PON protection on all of its OLTs and ONTs.  This allows the user to fully protect the PON side of the network in a cost effective manner.

Type B protection makes use of a 2:N splitter which allows two inputs on the OLT side of the circuit.  There is a single fiber going to each of the ONTs on the PON.  This gives both equipment and facility protection for the OLT side of the PON.  

Type B PON Protection Switching Timing

As of the 29.0 release, the system can detect the failure of the PON within 500 ms and switches the traffic in sub 5 seconds for the whole PON (up to 32 ONTs).

It should be noted that for the QOIU7, the protection switching is revertive meaning, when the primary link is returned to service, the system will switch back after the wait to restore timeout has expired. The default wait to restore time is 60 seconds. The revert time is used to provide hysteresis and avoid thrashing if the Primary PON is transitioning up and down.

The OIU8 has improved hardware that enables non-revertive switching.  When the OIU8 switches due to a PON failure, it will remain on that link as long as it is operational and will not switch back.  This is less disruptive than the revertive switching used on the QOIU7 as it does not need to switch back when the primary PON goes back into service.

MAC Table Updates After PON Protection Switch

When a PON protection switch occurs, for proper operation, the system must flush the MAC tables. MAC address re-learning will occur upon the next message sent upstream on the protected PON. In the case of an intra-OLT PPG, the ESU cards of the OLT will relearn the MAC addresses of the devices attached to the protected ONTs as coming from a different slot. In most cases, this learning is almost immediate actually done in the upstream direction because the devices connected to the ONTs are sending packets unaware that a change has occurred. This has the benefit of reducing the need for flooding. Any upstream switches are unaware of the change on the OLT. In the case of inter-OLT PPG switching, the same process occurs in the OLT, but the upstream switches must learn the new path to the client device since it is on a different OLT. Appropriately, it is typical in an environment with redundant PONs for the OLT to be connected to the core network via HSRP or VRRP, protocols that are designed for immediate response to path updates.

So, this behavior is normal and to be expected during a PON Protection Switch. Some flooding will occur downstream until the next message is sent upstream on the PON and the MAC is relearned.

Fast Failover via the Sync Channel

The Tellabs implementation is able to detect hard failures of the PON and failover even if the two PONs within the protection group have no communication.  If all ONTs on a PON stop communicating, the PON link will fail over to the secondary link.

While no messaging is required for PON protection switches to occur, it is beneficial to communicate status information from the primary to the secondary PON to speed up switches and minimize service disruption. Outages of up to 1 minute can occur if no sync channel is present and the system blindly switches as all services must be reconfigured. With the sync channel present and sufficient time to sync, the two sides will have exactly the same configuration and switches will be in the 1-5 second range.

For example, if a port has been authenticated via 802.1x, rather than force the 802.1x supplicant to re-authenticate when the PON protect switch occurs, the status can be shared with the protection PON. To do so, Tellabs has implemented a status update protocol between the primary and secondary PON. The status information is sent periodically, every few seconds to enable rapid protection switches. It is meant strictly to minimize the traffic disruptions in various scenarios. 802.1x and MAB can lengthen the time needed to sync the Primary and Secondary configurations as there is a good deal of status information that is exchanged.

In the case of Intra-OLT protection, the sync channel is configured by the EMS behind the scenes and requires no user intervention. In the case of Inter-OLT protection, the user will be prompted to configure the Sync Channel ID.

The Sync Channel VLAN ID configuration defines the VLAN the Active and Standby PON will use to communicate.  The Sync Channel VLAN ID will be used to exchange layer 2 Ethernet frames containing proprietary Tellabs messages used to synchronize state.  The VLAN used must be configured in the VLAN table and uplink NET interfaces on BOTH OLTs so that the protection group members can communicate. The user only needs to specify the Sync Channel VLAN when it is an Inter-OLT protection group.

After a switch the system will have a standing "PPG Sync Mismatch" Alarm indicating that the two PPG PON ports are syncing status.  Once sync is complete the alarm will clear.  PPGs should not be switched with a standing PPG Sync Mismatch alarm as it will always result in a slow switch.  Also if a "PPG Sync Mismatch" alarm remains standing for long periods of time, it may indicate that there is a problem communicating on the Synch channel VLAN between the two OLTs.  If this occurs verify that the sync channel VLAN is set up properly throughout the network between the two OLTs

PON Striping for Faster Switches

To ensure sub 5s speed of protection operations, it is recommended that Protected PONs be paired in such a way to stripe the pairs across multiple OIUs. This will minimize the time needed to recover the link by sharing the load across 4 OIU processors. The below figure illustrates an example with a QOIU7, but the scheme also applies to the OIU8.

Optical Network Setup and Budget

There are two key points associated with the optical budget of a protected PON network:

• Secondary PON must have a level of 5dB lower than the Primary PON when using QOIU7 cards by placing an additional 5dB of attenuation in the Secondary PON.   This additional budget is not required when using OIU8 cards as they do NOT have the 5dB requirement.
• 2:N PON Splitters typically add 3 dB loss to the optical budget versus a 1:N splitter.  Tellabs recommends deploying 2:N splitters at most installations to allow PPG to be added at a future date if needed.  The 2:N splitter does not typically add significantly to the cost of the system

This 8dB must be planned for within the optical budget of the system being deployed is using the QOIU7.  The OIU8 uses a slightly different hardware algorithm and does NOT require the 5dB pad and only needs to accommodate the  3dB losses of the 2:N splitter.

The goal should be, when using the QOIU7 to ensure that the system, when on the Primary PON has an optical power of 22-23dB or less, so that with the addition of the 5dB pad on QOIU7 cards, that the system stays below the 28dB recommended upper limit for receive power while on protect.

Since there can be variability in the launch power of lasers, it is very important that the actual outputs of the lasers be measured for QOIU7-based systems and ensure that there is a 5 dB differential in the signal levels.

The typical way this is accomplished is via insertion of a 5dB pad in the Standby PON optical path at the input to the 2nd splitter port.  Ensure that the correct attenuator type is utilized (UPC if inserted at the OLT, APC if inserted at the splitter).  The color of the attenuator stripe can be used to tell the connector type.

Importance of Testing PPGs Prior to Adding Service

It is critical that PON Protection be tested prior to adding any service to protected PONs.  A sample ONT should be placed onto the PON and traffic verified on both the Primary PON and the Secondary PON.  This is necessary to verify that the PPG configuration is correct, the Primary and Secondary PON are cabled correctly, and that both chassis have access to the uplink network.   Failure to do this can result in failures that are not discovered until the first protection event.  If the cabling and configuration of the secondary link do not agree, the PPG will fail to the wrong PON and likely all services will be lost until repairs can be made, or the configuration corrected.  See the section on striping to ensure the best failover performance when multiple PONs are impacted.

Forcing PON Protection by Disabling a Port

During Upgrades, System Maintenance and other scenarios, it is often useful to be able to force the protection over to the Secondary Link.  This can be done by disabling of the Primary PON from the Ports View->PON Tab.  Setting the context will show all relevant PONs within that context.  Simply select the drop down to set the PON to either Enabled or Disabled.

Software Download Strategies to Minimize Outages

During Upgrades, System Maintenance and other scenarios, PPGs should be used to minimize outage times during the time period of software switches. The general outline of a normal upgrade should be:

• Software Download to both the OLT containing the Primary Links, and the OLT containing the Secondary Links.  This step is not service affecting.
• Force all PONs onto a single OLT.  If all secondary links are on a single OLT, then this may or may not require disabling of PONs.
• Switch the OLT not carrying the traffic.  This step is not service affecting.
• Switch all the PONs to the OLT that has been switched to the new software release.
  • All ONTs that have an updated load will reboot and start the new load.  If the ONT load is the same, they will switch with a standard 1-5 second switch.
• Switch the 2nd OLT to the new software release.  This is not service affecting.

• Return PPGs to the normal state by enabling any disabled PPG PONs.

Configuration of Protection Switching

PON Protection is configured by clicking on the PON protection button on the top of the GUI(the shield icon).  Buttons exist to create, edit and delete PON Protection Groups.    The Create button will allow the creation of a new protection group.  The menu for creating PON Protection groups can be reached in one of two ways: 

• Right click on a specific PON and then selecting Protection and Add.
  
    
   
• Select the Protection Group icon(purple shield) and select Create.
  
  

For QOIU7 PPGs the prompts will be as follows: 

Protection Group Creation

The Add button allows the creation of a new protection group.

Protection Group Name: Name given to the protection group.

Admin State: Enables or Disables the protection group from operation.

Protection Type: The only supported type for protection is Type B 2:N splitter protection.  This type uses a 2:N splitter to allow a protection PON on the OLT side.

Sync Channel ID: Defines the VLAN used for synchronization of state between the primary and secondary PON of the protection group.  While this channel is not required, it speeds up protection actions by synchronizing state information between the Primary and Secondary PONs.  This Sync channel only needs to be configured for inter-OLT protection and will be auto-configured by the EMS for intra-OLT PPGs.

Wait to Restore Time: As noted before, the wait to restore time defines the time the Primary and Secondary must be active prior to switching back to the primary.  This provides hysteresis for the switching events to ensure the primary is stable, available and ready to take over.

OLT TID/PON AID: These two fields will define the PON ports that are connected to the 2:N splitter.  Any two PONs can be used that are of compatible types.  The only limitations are that the OLT for both ports must be managed by the same EMS, the PONs must be of the same type (XGS-PON/GPON), and must be of the same card type(QOIU7/OIU8 cannot be mixed).    The Primary and Secondary PON can be on the same OLT or two different OLTs.  If available, using a second OLT allows for full equipment and facility redundancy and is the best protection option.  In addition, the PON AIDs will be limited to PONs that are configured the same way and on the same card type.

Primary: Defines which of the two selected PONs is the primary PON that will carry the traffic under normal conditions.

The EMS will copy any configuration on the Primary PON to the secondary PON ONTs automatically so they do not need to be managed separately.  All changes must be made on the Primary PON ONTs.

One other thing to note is that the secondary PON for QOIU7 units should have a signal level 5 dB lower than the primary.  This is often accomplished using an SC-style single-mode fiber optic attenuator inserted into the optical path prior to the splitter.  This ensures that the Primary PON always wins at startup and is selected as the Primary PON.  This is not needed for OIU8 cards.

Once the PPG has been created and the secondary port enabled, the PPG configuration should be verified as follows:
 

1. Verify the ONTs can be seen on the Primary PON then pull the primary fiber or disable the primary PON port from the EMS.

2. Verify that all ONTs appear on the secondary PON. There should be ONT-OLT Link LOS alarms indicated for the ONTs previously active on the primary PON. The secondary PON should have an alarm indicating PON on Protect.

3. If traffic can be verified, do so. Traffic should restore during the switch in less than 5 seconds.

4. After a few minutes, switch the PON back by performing the reverse of step 1.

5. Verify traffic is restored and that alarms clear. In both cases, the switch should have taken less than 5 seconds.

It should also be noted that the OLT will suppress alarms on the Standby PON, such that generally speaking, all the alarms will only occur on the Active PON.

For best protection switching performance, it is recommended to spread the standby PON ports among QOIU7s. See the section on PON striping.

 Sync Channel VLAN

The system also supports Inter OLT protection where the Primary PON is on one OLT and the Secondary PON is on a different OLT.  Both OLTs must be managed by the same EMS.  When configuring Inter OLT protection, you are required to configure a Sync Channel VLAN.  This VLAN is not required but does speed up protection by allowing the two sides to exchange synchronization and state information to speed up the protection switching when a failure occurs.  Any Data VLAN that is present on the Primary PON can be used for this purpose as long as there is layer 2 connectivity between the two OLTs.  The protection messages are Ethernet frames but are NOT IP and will not be able to cross a routed network.  

The VLAN must be configured in the VLAN properties table as a dynamic VLAN and full bridged.

It should also be noted that when Intra OLT protection is used, the system picks the VLAN and the sync channel option will not be available. 

PON Protection Status

The system also includes a PON PPG status screen which indicates the current status of the protection group which is accessed via the purple shield icon on the Panorama button bar.

The protection group status for all PPGs managed by the EMS is shown in this display along with the ability to create, delete, or modify PPG groups.  The reload button on the left of the GUI dialog will allow the ability to read and update the status of the OLTs.

The status display will give the full status of the PPG via PON-A Status(Primary PON) and PON-B Status(Secondary PON), the status will have one of the following states: 

• Active: This PON link is the Active side of a protected PON and is carrying the user traffic.
• Standby: This PON link is the standby side and is in the normal state and ready to take over should a failure occur.
• Disconnected: This is a standby PON in an abnormal state.  The Active PON(the other PPG peer) is reported as working, but the standby PON cannot detect or take control of the link.  The most likely case is that the PON fiber is disconnected on this side.
• Disabled: This PON link has been administratively disabled or automatically disabled.  Path protection and other failures may automatically disable the port to force traffic to the other PON.
• Unknown: The EMS is unable to communicate with the OLT and the state of the PON is unknown.

PPGs can be shown in multiple colors on the PON Status screen: 

• White Indicates PON in the normal state and the active and standby link in a known good state.
• Red Indicates that neither side is reported as active and most likely the PON is out of service.  This can happen if both the Active and Standby PON are disabled, or in certain failure scenarios.
• Amber indicates that there is an Active PON but one of the two PONs is abnormal or the PPG has not completed sync with all the ONTs on the PON.  The Active PON is carrying the traffic, but the standby PON may not be available for protection at this time, or may switch slowly.

The PONs are also shown in the common tree with the PPG status information: 

The common tree display for PPGs can be interpreted as follows: 

• Primary PON is always green.  The Primary PON may or may not be be active, this just indicates whether it is primary or secondary.  This is only meaningful for QOIU7 PONs where PPGs will always revert back to the Primary PON when it is no longer failed.
• Secondary PON is always amber. 
• Checkbox: The checkbox indicates which of the PONs is the Active PON carrying traffic.  (ONLY get status from the Active PON).
• PPG Name: The PPG name will be shown for each PPG.
• Unprotected PONs: Unprotected PONs will not have any of these additional fields in the common tree.

Retrieving ONT and ONT UNI Port Status with PPGs

Introducing a PPG also means that you must be aware of the PPG when retrieving the status of an ONT or a ONT's UNI ports.  Just remember these rules.

• ONT Configuration and UNI port configuration can be seen on both the Primary and Secondary PON, but is best retrieved from the Primary side(green PON).
•   
• ONT Configuration and UNI port configuration can only be performed on the Primary PON(Green PON).  (Since the system always only copies configuration from the Primary to the Secondary).
• ONT Status is only valid when retrieved on the Active PON.

The Primary PON which should be used for all configuration actions, can be identified by the coloring on the GUI.  The Primary PON is colored green, only configure PPG PONs if they are green.

When retrieving status, ONLY the Active PON should be used. This is the PPG PON that has a check in the box that is shown prior to the PPG name. 

What PON Protection Can and Cannot Do

Simply described, the Tellabs OLAN PON protection mechanism allows for a protection PON port to detect when the primary port is no longer communicating with the ONTs and, upon such detection, take over control of the PON.

PON Protection does not protect individual ONT fibers since there is a single fiber going to the ONT.  Failures or breaks in this fiber cannot be protected with type B PON Protection.

Tellabs has also added additional PON protection mechanisms for handling some use cases where the Uplink fails in such a way as to isolate the OLT actively carrying traffic.  Also Tellabs has designed failover software to detect and fail on most hardware failures on the board.  Both use cases are detailed below.  At this time Tellabs is the only PON provider we are aware of with these features.

PON Protection with Spanning Tree Support

Tellabs has implemented an enhancement to PON Protection to couple it to the spanning tree which is available as a part of SR30.1 and above.  This feature enables the OLT to force a switch of the PON protection to the Standby link when the network access to the OLT is blocked on the OLT uplinks.  

The new features allow for addressing protection under certain network conditions when in an inter-OLT PON Protection scheme:

• Using Spanning Tree logic, an OIU (QOIU7 or OIU8 PON card) can detect when there is no longer a path to the designated root above the ESU uplink card(s). This allows the OIU to determine that it’s traffic will not egress the OLT and the primary PON can automatically disable itself and failover to protect the PON. Note that the failover times are a few seconds longer as the STP path failure must be detected, then the PON failover actions begin.
• Since PONs can contain traffic destined for multiple uplinks, the switch only occurs when the OLT is completely isolated and becomes the root of the CIST.  The logic is only really designed to support a single redundant path out of the OLT.  
• Also using spanning tree logic, the OIU can detect when the ESUs have been removed from the shelf or completely failed. If the spanning tree path is still good, but the ESUs management plane is out, the failover will take longer to occur. But, if the ESU(s) are pulled or lose power, the failover to the protect will occur immediately.

The  added logic is a huge improvement in the use cases that are addressed. There are still scenarios that may arise that do not result in the desired behavior:

• If the OLT is the designated root, the PON protection mechanisms that utilize STP will not work as it has no way to know whether or not the uplink Path is good.  It is always assumed that the root of the STP path is in the switches above the OLT.  Ensure that STP root bridge priorities are set to force the root to converge above the OLT.
• If all uplinks fail but a single 1G interface, and the OLT maintains the path to the core’s designated root, the PON will not switch to the protect.  The PPG will not switch on degraded bandwidth but only on complete failure of the uplinks.
• Another example would be with MSTP. If all VLANs are in one instance ID, and a single VLAN is in another, if the CIST’s path to root remains unchanged, PON protection will not engage, even if that means a majority of VLANs are not operational.  The PPG logic currently only supports switching when the STP logic shows the OLT as isolated from all uplinks.
• When the uplinks are failed, the primary PON does not have a status information as to the condition of the secondary PON. This means that the PON could failover and find that the secondary has poor optical conditions or that the uplinks to the network are no better than the primary link.  

The PON Path Protection configuration has been added to the PON Profile settings.  It can be enabled by checking the box shown below, "Enable Path Protection".

PON Protection on Hardware Failure Support

The PPG software has been updated in SR30.1 and above to detect failures of all major components on the card and switch the traffic to the secondary link when a hardware failure is detected.  The system will attempt to preserve unprotected links but, in some use cases, will reboot the PON line card.  Some of the conditions detected are:

• GPON Transmitter Failure OMCI failure Event - Whenever the OLT detects that it cannot get packets through the chipsets for OMCI transmissions, it will switch the PON to protection.  The PON will be alarmed with the GPON Transmitter Failure and have an Additional Text of "OMCI Failure".  If OMCI messages cannot be emitted, it cannot control the ONTs.
• Kernel OOPS Event - Kernel Oops occurs when Linux detects a fatal fault within one of the kernel routines.  It will issue a Kernel Oops then call panic which reboots the board.  Kernel Oops will now be logged as events.  All protected PONs are switched when the kernel oops occurs.
• Low Memory Alarm - No action will be taken on Low Memory, but this alarm was added to inform users to reboot the card in the next available maintenance window to ensure that it does not fail during normal business hours.
• Out of Memory Alarm - The system will now switch protected PONs to protect, then reboot when available memory gets catastrophically low.  Once the card has rebooted, it will switch the PONs back to the Primary Link.
• Traffic Outage Event - If the system detects zero packets coming from the ESU on the QOIU7 for a significant interval, it will switch all the Protected PONs to the Standby link.  If the ESU detects no packets from the QOIU7 for a significant interval, it will reboot the QOIU7 in an effort to recover traffic.  This will immediately force the spans over to the secondary link.  The Traffic Outage event will be seen in the Event Viewer on the EMS.
• Flash Failure Alarm - If the system detects that the card has an issue writing to the flash disk, it will switch all the protection PONs to the Standby, automatically disable all protected PONs and will signal a Flash Failure alarm.  The card will not reboot as in most instances where the flash fails it is unable to load the applications into ram and boot.  The card must be replaced in the next maintenance window.
• PPG Sync Mismatch Alarm - The PPG sync mismatch alarm has been added to indicate when the Primary and Secondary PONs are unable to sync config and status data to enable rapid switchover.  This typically indicates either a Network outage blocking the two OLTs from talking over the Sync channel VLAN, or misconfiguration or missing configuration for the Sync channel VLAN within the VLAN table of the OLT or its protection OLT.  It can also indicate that the OLT or some component of the OLT hosting the standby PON has failed.  PPG Sync mismatch will also occur after a switch to indicate that switching back to the other PON at this time will result in a longer outage time.

 Video