PPG Based ESU Upgrade

PON protection is implemented between two OLTs and will provide full redundancy of all OLT hardware, uplinks and PONs.  PPG protection makes use of a 2:N splitter which allows two inputs on the OLT side of the circuit (OLT to splitter side). From the splitter to the ONT is a single fiber going to each of the ONTs on the PON. This gives both equipment and facility protection for the OLT side of the PON. With PPG Protection, one can accomplish OLT PON fiber path redundancy. 

When PPG protection is implemented between two OLTs, it will provide full redundancy of all OLT hardware, uplinks and PON. This is the preferred configuration for reliability reasons and is the option recommended by Tellabs. 

PPG protection consists of two OLT systems, A Primary and a secondary.  These OLT systems should be in two separate locations with their own independent power supplies. This will protect from any hardware/software or facility problems. (i.e. Fire, water, power, etc.) 

 

Figure 1 PPG Protection

Pre-Upgrade

To Download and Print an Upgrade Checklist click Here.

• Verify Download Complete: Verify that all downloads are complete to the Primary and Secondary OLTs and that no auto downloads are in progress. If any are in progress, allow them to be completed. 
• Verify NO PPGs are in protect:  For QOIU7 PPGs, ensure that there are NO PPGs that are currently on protect. If they are, they should be repaired prior to upgrading as that PPG will fail during the upgrade and lose service for an extended outage time. 
• Repair any PPG Issues Prior to Upgrade: Since this is usually undetectable until you switch (due to the standby units laser being disabled), it is important to test all PPGs at the time of installation and periodically during maintenance windows to ensure no issues exist with the Standby PON link. 
• Ensure that all PPGs, on the primary, are properly switched and are not failing.
• Capture System State: Capture the current state of the system. In high value networks, this gives the ability to validate that all devices have returned to service and the system is fully operational after upgrade.
  1. Capture MACS: Capture all MAC addresses if needed to validate that all devices have returned to service by utilizing the Verifying MAC Counts Before/After Upgrade procedure.
  2. Capture Alarms: Capture all current alarms for the system so that they can be checked after the upgrade to ensure no new alarms are present by utilizing the Verifying Alarms Before/After Upgrade procedure.
  3. PPG In Sync: Ensure all PPG Sync alarms are cleared.

PPG Software Upgrade

WARNING: There will be outages during the upgrade where the impact might affect users. It is recommended to do the upgrade during a maintenance window.  In a properly maintained network, in which ONTs reboot, the outage can be as brief as 30 seconds and up to 2 minutes if dealing with 802.1x or VoIP phones where the phone needs to tag its voice traffic.

Note: In upgrades where the ONTs do not upgrade, resolving any PPG sync issues can limit outage times to 5 seconds or less, PPGs with sync issues can vary from 30 seconds to multiple minutes to get all users up and re-authenticated.

When instructed in the Upgrade EMS procedure, the EMS will stage the updated software into the secondary flash banks on the Primary and Secondary OLT ESUs. Then, with a Manual Switch in this procedure, the EMS will replace the software on the Secondary OLT ESU with the new revision and reboot the program. As the program reboots, the software will then, sequentially access each PON and OLT, collect its parameters and update the database. Because the update is conducted on a separate OLT, there will be no interruption of traffic flow during an update.

After the database has been updated in the OLT Primary ESU, the EMS will automatically (for QOIU7s only) switch traffic control from the Secondary OLT ESU to the Primary OLT ESU.

Initiating a software switch on the Secondary OLT will get the new software running on the Secondary OLT.

Note: When running a Manual Software Switch on the Secondary OLT, traffic is live on the Active Primary OLT, so there is NO downtime during the software switch.  When running a Manual Software Switch on the Primary OLT, traffic is live on the Active Secondary OLT, so there is NO downtime during the software switch.

1. Notify all parties of maintenance start.
2. Disable SNMP on Primary OLT and Protect OLTs by utilizing the Disable/Enable SNMP Agent procedure.
   1. Verify SNMP has been disabled on the Primary and Secondary OLTs if needed to prevent unnecessary trouble tickets during upgrade.

      Note: This is optional. There may be alarms and report tickets generated during a software update. To prevent this, SNMP should be disabled.

3. Enable Primary PONs: Ensure all PONs with a PPG are Enabled on the Primary OLT as traffic will be shifted to the OLT.

   Note: Verify all primary PONs are enabled and if there are any GPON PONs on protect, troubleshoot this condition and restore the Primary GPON PON to operation before continuing.

4. Disable Secondary PONs: Disable all PPG protected PONs on the Secondary or Protected OLT.
   1. Select Ports  Icon in the Application window, select the Secondary OLT in the Common Tree, and the PON tab. 
   2. In the Admin State column, select Disabled in the dropdown box.
   3. After all the OLTs have been disabled, click on the Apply button.

      Note: Traffic of the Disabled OLTs will automatically switch over to the Primary OLT.

      
      
       
   4. Ensure all PONs with a PPG are Disabled on the Secondary OLT to force traffic to the primary OLT.
5. Switch Secondary OLT: Initiate a Manual Software Switch on the Secondary OLT.
   1. Select the Download  Icon in Application window, the OLT in the Common Tree, and the Manual Switch tab.
   2. In the Switch to Version - pull-down menu, select the latest version software package from the dropdown list.
   3. Click on the Execute Switch button.
      
      
       

      Note: The System Switch Status window will show download status messages as the software is updating the OLTs. This can take from 1 to 2 hours, or less, to complete.

   4. System Switch Status will indicate Complete when switching is completed.

      Note: The STAT LED will be a solid red as the program is running a self-check and loading the OS. After the OS is loaded, the STAT LED will blink green at a one-sec interval as the program is downloading. After downloading, the STAT LED will blink green at one-half-sec intervals, as the program boots up and syncs with ONTs. When complete, the STAT LED will be a solid green.

   5. After the switching is complete, and the database is updated, the EMS will have switched the upgrade software to the Secondary PON.
   6. Verify all OLT cards switched (Software) successfully.
   7. Should a failure occur during the switching process, Contact Technical Support. 
6. Enable Secondary PONs: Enable all PONs on the Secondary OLT.
   1. Select Ports  Icon in the Application window, select the Secondary OLT in the Common Tree, and the PON tab. 
   2. In the Admin State column, select Enabled using the dropdown box.
   3. After all the OLTs have been enabled, click on the Apply button.
      
      
       
   4. Wait for all the PONs for PPG to Sync. 
7. Disable Primary PONs: Disable all PPG protected PONs on the Primary OLT. Wait for ONT Missing alarms to clear.

   Note: This will create a failover and the data traffic will transfer to the Secondary OLT and the ONTs to switch to the new software version.  Traffic will stop during the ONT reboot, but this should be the only outage during the upgrade.

8. ONTs on New Software: The ONTs will now be running the new software and should be on the Secondary OLT. Traffic should start returning in 1–2 minutes on the Secondary OLT.

   Note: At this point, the Primary OLT is still running the old release with the current database and can be switched back to in the event of any upgrade failures.

9. Validate Traffic on Secondary: Utilize the data captured in Pre-Upgrade to ensure that all devices have returned to service and all alarms have returned to their pre-upgrade state:
   • Capture MACS: Capture all MAC addresses to validate that all devices have returned to service by utilizing the Verifying MAC Counts Before/After Upgrade procedure.
   • Capture Alarms: Capture all current alarms for the system after the upgrade to ensure no new alarms are present by utilizing the Verifying Alarms Before/After Upgrade procedure.
10. Switch Primary OLT: If the traffic looks good on the Secondary OLT, then initiate a software switch on the Primary OLT to the new software release.
    1. Select the Download  Icon in Application window, the OLT in the Common Tree, and the Manual Switch tab.
    2. In the Switch to Version - pull-down menu, select the latest version software package from the dropdown list.
    3. Click on the Execute Switch button.
       
       
        

       Note: The System Switch Status window will show download status messages as the software is updating the OLTs. This can take from 1 to 2 hours, or less, to complete.

    4. System Switch Status will indicate Complete when switching is completed.

       Note: The STAT LED will be a solid red as the program is running a self check and loading the OS. After the OS is loaded, the STAT LED will blink green at a one sec interval as the program is downloading. After downloading, the STAT LED will blink green at one half sec intervals, as the program boots up and syncs with ONTs. When complete the STAT LED will be a solid green.

    5. After the switching is complete, and the database is updated, the EMS will have switched traffic from the Secondary OLT to the Primary OLT.
    6. Verify all cards switched (Software) successfully.

       Note: This will install the software upgrade to the Primary OLT.

    7. Verify all PON cards switched (Software) successfully. 
    8. Should a failure occur during the switching process, Contact Technical Support. 
11. Enable all Primary PONs: Enable all PONs on the Primary OLT and the system should now be back to a fully operational state with both at the new software release and fully upgraded (QOIU7 is auto revertive, OIU8 is not).  

    Note: If the OLTs are QOIU7s, they will auto-revert all traffic back to the Primary PONs. If the OLTs are OIU8s, they will need a manual failover by disabling the Secondary PONs. Once the OLTs are failed over to the Primary PONs, re-enable the Secondary OLT PONs.

    1. On OIU8s, disable all PON ports on the Secondary OLT.

       Note: While failing over to Primary OLT, ONTs will re-sync to Primary OLT.

12. Once all the PONs have failed over to the Primary OLT, they re-enable all the OLT PONs on the Secondary OLT.
13. Re-enable SNMP: Enable SNMP so that normal ticketing and system surveillance will return to normal by utilizing the Disable/Enable SNMP Agent procedure. (If disabled before software update).
14. Backup OLTs: Perform a backup of all OLTs. This action typically takes about 1–2 minutes per OLT. Backup the OLTS utilizing the Backup OLT Database procedure.

    Note: Ensure all the OLTs are backed up. It is good practice to ensure that backups are copied off of the machine to protect them in the event of any machine failure.

15. Validate Installation:
    1. Utilize available tools (device controllers, SNMP tools, ping, sweeps, etc.) to identify any network problems (Offline devices, uninstalled devices, PoE issues, physical plant problems, Etc.)
16. Notify parties that the upgrade is now complete.
     

    ---

    
      |  Next