Redundant Servers - ICX-AlphaCom
From Zenitel Wiki
This article describes how to configure redundant ICX-AlphaCom servers. For redundancy using AlphaCom XE servers, see Redundant_Servers_(AlphaCom_XE)
 |
| Redundant Servers - ICX-AlphaCom |
Contents
The Redundant Server concept
The concept is 1:1 redundancy of servers. The pair of servers has the same configuration, and only one of them is active at any time.
The High availability solution is maximizing uptime during:
- System level failures
- Environment-level failures (power outage, fire, earthquake, etc.)
- Planned outage for system maintenance and upgrades
Abbreviations
- HA: High Availability
- HA pair: A pair of servers, where one is operational, and the other is in standby with the same configuration.
- Primary Server (Configuration Master): The server in a HA pair that has the configuration as programmed from AlphaPro. This role is defined via the license. The role is fixed to a physical server.
- Secondary Server (Configuration slave): The server in a HA pair that fetches the configuration from the Primary Server. This role is fixed to a physical server.
- Operational IP address: A common IP address for a HA pair that stations and SIP devices connect to. The Operational IP Address is bound to the currently Operational server.
- Operational server: The server which is currently using the operational IP address and to which the stations and devices are registered to. This role is not fixed to a physical server.
- Standby server: The server which currently is not operational. It is not using the Operational IP Address. This role is not fixed to a physical server.
- Operational interface: The ethernet interface which the Operational IP address is bound to, eth0 or eth1.
Software requirements
All versions of ICX-AlphaCom software support the Redundancy Server feature. However, it is strongly recommended to always use the latest software to get the latest features, bug fixes and security updates.
Important notes:
|
|
Licenses
Three different licenses are available for the Redundancy feature:
- 1002606006 - ILR-U64 - Redundancy License 64 Users
- 1002606008 - ILR-U256 - Redundancy License 256 Users
- 1002606009 - ILR-U512 - Redundancy License 512 Users
These licenses can be stacked.
The system must have a Redundancy License installed covering all stations and devices used on the system. That is the sum of the following licenses:
- IP Station licenses
- SIP Station licenses
- SoftClient licenses
- Amplifier Channels licenses
- IP-ARIO licenses
The same license file is installed both in the Primary Server and in the Secondary Server. Information in the license file determines which server will be the Primary and which will be the Secondary.
 |
| Information in the license file decides which server is the Primary and Secondary server |
For test and demonstrations one can use the free license period. In general there must either be a free license in both servers, or valid licenses in both servers. When the servers are running in Free License period, which one to be the Configuration Master (Primary Server) is set by a check box in the Web Interface of the servers.
 |
| Configuration Master is set by a check box when running in the Free License period |
|
Before inserting the license file, the Configuration Master checkbox should be removed! |
Description
The principle
Two servers are set up as a redundant HA pair, sharing the same configuration, including node number. One of the servers is the operational server, the other one is in standby. The standby server monitors the operational server via the IP network and sends a request for a status update every three seconds. The standby server will try to take over if it looses the connection to the operational server, or if it receive a message that the operational server has made a reset.
Each time the operational server resets, the standby server will take over, i.e. the servers will swap roles at every reset. This also applies to a "small reset" (application reset).
The network interface
The network interfaces of the server is configured in the standard way with separate static IP addresses. The static addresses are used for maintenance, AlphaPro and Web interface. The operational address is an additional IP address, shared by the servers in the pair. Only one of them is active on that address at any time. All operational devices, like stations, SIP devices, other ICX-AlphaCom nodes (AlphaNet), must be set up to connect to the operational address.
- The operational address can be bound to eth0 or eth1.
- There must be configured a static IP address on the interface which will host operational address.
- The operational address must be within the subnet address range defined for the interface.
- A HA pair using only one ethernet port each will need a total of 3 IP addresses.
- A HA pair using both ethernet ports will need a total of 5 IP addresses.
The Intercom Application (AMCd) is running also while a server is in standby mode, but no stations will register to it because the operational IP address is used by the other (operational) server. If the operational server fails, the standby server will take over the operational IP address. Stations will then reconnect to the same IP address, but now to the new server.
Distribution of configuration
The Primary Server in the HA pair is designated as Configuration Master. AlphaPro updates are sent to the Primary Server. The Secondary Server will automatically be updated with copy of the configuration from the Primary Server. The Primary Server checks every 15 seconds if the configuration has been changed. If changed, the Secondary server is notified, and it will fetch and apply the configuration file via HTTP (port 80) or HTTPS (port 443). The operational server also updates the standby server with copy of the SIP registration database (because SIP registration times may be as long as an hour).
Configuration
 |
| Redundant Servers - Configuration example |
The configuration of the Redundant Server feature is done from the Web interface of the servers. This is best explained by an example. The below confiuration example is with reference to the drawing to the right.
Configure the IP address in both servers
- In System Configuration > Interfaces configure the Ethernet port 1 of the Primary Server with IP address 10.9.5.91
- In System Configuration > Interfaces configure the Ethernet port 1 of the Secondary Server with IP address 10.9.5.92
Install licenses
- In System Configuration > Licensing, install the licenses. The same license file is used on both servers.
- Check the license file and notice the MAC addresses, they tell which server will be the Primary Server and which will be the Secondary Server.
Configure the Redundancy feature in the Primary Server
On the Primary Server go to System Configuration > High Availability, click 'Create HA config'. Set:
- Operating interface: eth1 (10.9.5.91) (This is the local ethernet port used by this server)
- Peer exchange name: Any descriptive text (Typically you enter the name of the other server)
- Operational IP address: 10.9.5.90 (This is the common IP address used by both servers, and to which all stations and devices register to. Must be identical on both servers)
- Peer maintenance IP address: 10.9.5.92 (This is the IP address of the other server)
- Username: alpha (This is the "Read" user name as defined in System Configuration > User Management. Default is alpha/com. If the "Read" user is changed, this field must be updated accordingly)
- Password: com (This is the "Read" password name as defined in System Configuration > User Management. Default is alpha/com. If the "Read" user is changed, this field must be updated accordingly)
- HTTP or TLS/HTTPS: Here you choose if the communication between the servers should use HTTP (port 80) or HTTPS (port 443).
- Make sure HTTP (port 80) or HTTPS (port 443) is enabled in the Filters settings for Eth1
- Validate and Save
Configure the Redundancy feature in the Secondary Server
On the Secondary Server go to System Configuration > High Availability, click 'Create HA config'. Set:
- Operating interface: eth1 (10.9.5.92) (This is the local ethernet port used by this server)
- Peer exchange name: Any descriptive text (Typically you enter the name of the other server)
- Operational IP address: 10.9.5.90 (This is the common IP address used by both servers, and to which all stations and devices register to. Must be identical on both servers)
- Peer maintenance IP address: 10.9.5.91 (This is the IP address of the other server)
- Username: alpha (This is the "Read" user name as defined in System Configuration > User Management. Default is alpha/com. If the "Read" user is changed, this field must be updated accordingly)
- Password: com (This is the "Read" password name as defined in System Configuration > User Management. Default is alpha/com. If the "Read" user is changed, this field must be updated accordingly)
- HTTP or TLS/HTTPS: Here you choose if the communication between the servers should use HTTP (port 80) or HTTPS (port 443).
- Make sure HTTP (port 80) or HTTPS (port 443) is enabled in the Filters settings for Eth1
- Validate and Save
AlphaPro settings
In AlphaPro go to Exchange & System > System:
- In the first IP address field enter the local IP address of the Primary Server. Enable AlphaPro to Connect to this address (here 10.9.5.91). Uncheck "AlphaNet".
- In the second IP address field enter the Operational IP address of the HA pair. Check "AlphaNet".
(You can ignore the description in AlphaPro that the first IP address field is for Eth0 and the second field is for Eth1).
 |
| IP settings in AlphaPro |
|
|
By design the AlphaPro cannot connect to the Secondary Server. A connection attempt will be rejected by the server with the response "The remote server returned an error: (500) Internal Server Error". |
Status information and Test
Status information
The blue Active LED on the front and rear side of the ICX-500 is lit on the unit which is currently operational. This LED is off on the standby server.
 |
| The Active LED shows that this server is currently the operational server |
The current status is also displayed in the Web interface of the servers. In System Monitoring > Node Information:
 |
| Current status is shown in the web interface |
Explanation:
- Redundant Servers (IPHA): Shows if this server is either Config Master (= Primary Server) or Configuration Slave (= Secondary Server)
- Redundant Server State: Shows the current state of this server - Operational or Standby
The status is also displayed in System Configuration > High Availability:
 |
| Current status displayed in System Configuration > High Availability: |
- Haipd SW version: shows the current High Availability IP Daemon version
- server peer name: The name of the other server
- HA State: operational or standby
- Connected to <other server> since: shows the time and date since the two servers were paired
- Downloaded /api/haipd/amcd_conf.tar.gz: shows when the configuration file was last updated.
- Downloaded /api/haipd/state_data.tar.gz: shows when the state data file was last updated.
Test and verification
Force takeover: The standby server can be forced to take over the operation in several ways:
- Small Reset or Reboot from the Web interface of the Operational Server
- Disconnect the ethernet cable on the Operational Server
- Power off the Operational Server
Observe takeover: Observe that the standby server changes state to Operational and that:
- The Active LED becomes lit
- Status changes in the web interface of the servers
Verify that:
- Stations and Devices are registered and operational
- AlphaNet (if installed) is operational
- SIP trunks (if installed) are operational
- Any integrations to 3rd party systems are operational
Logging:
- Status information are logged in the System log
- Additional logging of status can be generated by adding some extra configuration. See IPHA - Monitoring server status
Synchronization of Data - Additional info
Configuration from AlphaPro
The configuration data sent from AlphaPro is written to the working (RAM) memory of the Primary Server. When the update is finished, the configuration data is copied to the flash memory for permanant storage. The secondary server will be notified about the change, and will fetch and apply the new configuration data.
After system reset or startup the working RAM memory is cleared. Configuration data is now loaded to the working memory from the flash.
Other configuration
Some configuration can also be modified by the users:
- Programming DAK keys on the station - code 784
- Changing the volume settings on the station
And some configuration can be modified by data commands, typically used within the Event Handler:
- $VOL - Modify station volume
- $GRM - Include/exclude a station from group
- $SSC - Modify the CoS (Class of Service) for a station
- $SNV - Modify values in the working RAM memory
These configuration changes are written to the working memory. However they are not immediately copied to the flash memory. This is to preserve the longevity of the flash memory itself. When a change is made, a 4 hour timer is started. At timeout the changes are written to the flash memory, and the configuration is syncronized with the Secondary server. When doing a software reset, any pending changes will be copied to flash before the reset is performed. (No copying if sudden power reset).
|
|
|
State data
State Data is temporarily user data. This data is stored in RAM memory, and is regularly synchronized to flash memory.
The following State Data is synchronized between the servers every 10 seconds (Note: requires firmware ICX version 1.2.3.7 or newer):
- Call Forwarding/Follow Me
- Call Request Forwarding
- Absence messages
- Call Request queue
- Mail messages
- UDD's (User Define Data)
The synchronization is from the operational server to the standby server.
The following State Data is not synchronized between the servers:
|
|
If it is important to preserve all state data, it should be considered to configure the system in such a way that the Primary Server is always operational when the HA pair is running. |
Set Primary Server to always be the operational server (optional)
To "force" the Primary Server to always be the operational server when the HA pair is up and running, add this event to the Event Handler of the system:
In AlphaPro, Exchange & System > Events, Insert a new event using Event Type 27, Subevent 231, When Change To = ON:
 |
| Primary Server set to always be the operational server |
Action Command when using software version 1.2.3.7 or later:
Action commands:
IF %syse(230) STOP ENDIF IF %op(%syse(232),=,0) $SMALL_RESET ENDIF
Action Command when using software version 1.2.3.6 or earlier:
Action commands:
IF %syse(230) STOP ENDIF IF %op(%syse(232),=,0) $RESET ENDIF
Explanation:
- 231 ON = The link to the other server is up again.
- 230 ON = I am in standby, -> do not reset.
- 232 OFF = I am the slave, so do a reset.
Troubleshooting
If one server is defined as Master using the checkbox available in Free License mode, and a license which defines the other as master is then inserted, the system will behave in an erratic manner (frequently restarting, cannot send database (error 503), etc).
To rectify:
- Do a Clean & Factory reset on both servers to remove the license
- Uncheck the Master Configuration checkbox in the IPHA settings, validate and Save and Apply. The server will reset.
- Insert the license again
- Upload the database
- Check and verify wanted operation