Pioneering the Age of AI in Electronic Data Interchange (EDI) and Managed File Transfer (MFT)
April 24, 2024
No Comments
End of Support Notification: IBM Sterling Data Exchange Select Products.
IBM Global Mailbox - components at a glance
A Global Mailbox system includes various components that operate together.
Global Mailbox components | Description | Required or optional | Operating system |
|---|---|---|---|
| Sterling B2B Integrator (application) | Processes the Global Mailbox messages. | Required | Windows, UNIX, AIX, or Linux® |
| Global Mailbox Management node | Manages the Global Mailbox data. | Required. Must be co-located with Sterling B2B Integrator. | Windows, UNIX, or Linux |
| Global load balancer | Distributes work load across data centers. | Required | Not applicable |
| Local load balancer | Distributes work load within a data center. | Required | Not applicable |
| Apache ZooKeeper | Creates distributed locks and coordinates actions across all Global Mailbox nodes. | Required | Linux |
| Apache Cassandra | Stores and replicates metadata across the servers in the Cassandra cluster in all the data centers. | Required | Linux |
| Cassandra Reaper | Used to schedule automatic repairs for the data in Cassandra. | Required | Linux |
| Shared file system, such as GPFS and NFS. | Stores payload data. | Required. Sterling B2B Integrator, Global Mailbox Management node, and replication server access the shared file system. | Windows, UNIX, or Linux |
| NFS Parameter actimeo | You must use actimeo=0 on your NFS mount. If the parameter is not specified, file segments of 0 KB or incorrect size will be displayed. Downloading of large files (in GBs) from the replicated data center may also fail. | Required | Linux |
| Replication server | Replicates payloads from shared disk of one data center to the shared disk to another data center. | Required. Must be installed on each Sterling B2B Integrator and Global Mailbox Management node. | Windows, UNIX, or Linux |
| WebSphere® MQ | Notifies applications about the events related to the Global Mailbox. | Required | Windows, UNIX, or Linux |
| Database cluster | Stores the application configuration, such as users, business processes, and protocol adapters. | Required | Windows, UNIX, or Linux |
| Perimeter server | Reduces network congestion issues and improves scalability for high volume environments through session and thread management. Enhances security by moving security threats further from your secure network and data. | Optional | Windows, UNIX, or Linux |
| Sterling File Gateway | Facilitates the exchange of file-based information securely, in virtually any format, protocol, and file size. | Optional | |
| Sterling Secure Proxy | Masks the IP address and HTTP details of the servers to secure the network and data centers. | Optional | Windows, UNIX, or Linux |
Conceptual diagram
The following diagram shows the relative relationship between the components of Global Mailbox
Sterling B2B Integrator and Sterling File Gateway add Global Mailbox capability for enhanced disaster recovery.
Companies in many industries are under pressure to sustain their operations on a 24 x 7 x 365 basis due to the nature of their business. Whether it is a global supply chain, hospitals providing 24-hour patient care, or the demands of consumer banking, the need to operate without interruption is no longer an option.
What is also common across all industries is a need to exchange a variety of business documents and in many use cases do so in real time. This complexity puts enormous pressure on IT to architect a business-to-business (B2B) integration platform that not only supports a just-in-time exchange of critical business documents but is always available. Complicating these complex challenges is that many companies have chosen to operate regionally dispersed data centers for both performance and disaster recovery purposes. Companies have gone to great lengths to ensure 24 x7 x 365 availability by deploying redundant hardware and software capabilities so they can operate in an active-active or active-passive mode or leverage database technology to minimize disaster recovery downtime.
Sterling B2B Integrator and Sterling File Gateway help companies overcome the challenges of operating without downtime by offering a highly available mailbox architecture for the exchange of files with trading partners. Active-active processing across globally distributed locations enables highly available business operations while built-in application disaster recovery satisfies data preservation requirements. IBM® Global High Availability Mailbox provides the following features for high availability and disaster recovery to Sterling B2B Integrator and Sterling File Gateway:
- Mailbox capabilities
- Access and support for the following protocols:
- File Transfer Protocol (FTP)
- Secure File Transfer Protocol (SFTP)
- IBM Sterling Connect:Direct®
- myFileGateway
- B2B architecture for active deployments across data centers
- Application level command and control to manage payloads across data centers
- Consistent view across data centers of what files get processed regardless of data center status
Global Mailbox is integrated with the business processes, protocol adapters, and file transfer capabilities of Sterling B2B Integrator and Sterling File Gateway.
The Global Mailbox management server and Sterling B2B Integrator must be installed together on each node.
In Sterling B2B Integrator, the Global Mailbox Client adapter (GMCA) is an adapter for managing access to the Global Mailbox system. The GMCA configuration and parameter values in the properties files control access to the Global Mailbox system and storage. This enables operation with other protocols in Global Mailbox.
The Global Mailbox management node hosts the Global Mailbox administration user interface. At least one Global Mailbox management node must be available on a data center for an administrator to use the administration user interface.
In the Global Mailbox management node, administrators create users, assign user permissions, assign virtual roots, create mailboxes, and manage event rules. If no Global Mailbox management node is operational in a data center, administrators must log in to the user interface from another data center.
Use the Global Mailbox management node to complete the following functions:
- Host the user interface for the Global Mailbox management tool for administrators to manage mailboxes, users, and events
- Run the scheduler to manage scheduled jobs
- Transport files between data centers for replication
- Coordinate replication between data centers as a polling agent
Load balancers distribute workload across the servers in the data centers. They redirect requests to other servers in case of server failures or network connectivity issues.
To handle the high volume of messaging requests from your trading partners, you can use a load balancer for a high availability deployment. The load balancer serves the following functions:
- Load balancing: To distribute the workload across a set of servers to ensure that servers are not overloaded.
- Failover: If a server fails, the user is directed to another server. Failover ensures that the system is still available when there is a failure of a server.
Load balancers verify connections between nodes in a cluster and between data centers. Select a load balancer that can monitor the status of the components in your system. If a node that is being checked does not respond within a specified timeout period, or the status of a node indicates that performance is degraded, the load balancer can redirect the traffic to another node. Any load balancer that meets your requirements is supported.
Global Mailbox uses the following types of load balancers:
- Global load balancer: Directs a trading partner to the closest available data center and monitors data centers for availability. Configure the global load balancer to send requests to the closest data center to ensure that network latency is not an issue (geographical load balancing). This load balancer typically uses DNS resolution to route users to the closest data center.
- Local load balancer: Manages local transactions to balance the workload across all servers within that data center and detect problems with a component. Within each data center, there is a local load balancer, which provides load balancing to the Sterling B2B Integrator and Sterling File Gateway services (protocols such as FTP, SFTP, Sterling Connect:Direct®, and myFileGateway). It spreads the load across several servers that are part of the Global Mailbox cluster in that data center.
You can have more advanced monitoring for the protocols that your system supports if this capability is supported by your load balancer. For example, all load balancers are capable of monitoring client connections to an FTP server (that is, the server is started and accepting connections). However, this capability might not be sufficient enough to validate that the FTP server is accepting new files or is able to serve files. You can do advanced monitoring with some load balancers to ensure successful FTP GET and FTP PUT operations.
If your load balancer supports monitoring FTP GET, create a file that never expires in Global Mailbox. Configure the load balancer to download the file at a regular interval. With this configuration, the load balancer continually validates that Cassandra and the shared disk are working on that node.
Apache ZooKeeper is an open source server which enables highly-reliable distributed coordination in distributed applications. You can deploy ZooKeeper with Global Mailbox for centralized services across a cluster.
ZooKeeper provides the following services:
- Maintains configuration information
- Naming
- Distributed synchronization
- Group services
- Metadata management
Apache Cassandra is an open source distributed database management system designed to handle large amounts of data across many commodity servers. It provides high availability with no single point of failure. Cassandra offers robust support for clusters spanning multiple data centers, with delayed (asynchronous) masterless replication allowing low latency operations for all clients.
Cassandra places a high value on performance. It achieves that highest throughput for the maximum number of nodes. Cassandra has the following features:
- Decentralized - every nodes has the same role. There is no master node.
- Supports multiple data center replication
- Scalability - read and write throughput increase linearly as new servers are added, with no downtime or interruption to applications
- Fault tolerant - failed nodes can be replaced with no downtime
- Tunable consistency - balance of replication and performance to meet business needs
- Reparability - schedule automatic repairs in the background for data consistency across nodes
The Cassandra Reaper tool is used to schedule automatic repairs for the data in Cassandra. The repairs are scheduled in the background. Reaper is a centralized, stateful, and highly configurable tool for running Apache Cassandra repairs against single or multi-site clusters.
The Reaper is installed automatically when Cassandra is installed. The directory for the reaper is <installation directory>/apache-cassandra/reaper. Reaper is started automatically when Cassandra is started. Reaper is pre-configured with a repair schedule.
You can view the repair status for Reaper on the UI. For more information, see Checking repair status. If repairs do not occur, you must restart Reaper for the automatic data repairs to start.
Global Mailbox stores message payloads in a shared file system, also called the storage repository.
Payloads are accessed from the storage repository. Sterling B2B Integrator, Global Mailbox Management node, and replication server access the shared file system across the data centers.
Based on your business requirements, you can use any shared, highly available file system such as General Parallel File System (GPFS) or an HA NFS server, for payload storage. GPFS is a scalable, highly-available, high performance file system optimized for multi-petabyte storage management, making it qualified and compatible for use with Global Mailbox. GPFS provides online storage management, scalable access, and integrated information lifecycle management tools capable of managing petabytes of data and billions of files.
In the Global Mailbox system, storage (file system) implementation is based on the concept of storage buckets. The buckets are containers (logical groups) in the file system, which are configured according to business requirements based on security and retention.
The Global Mailbox system uses a replication server for payload replication. The replication server transfers files at high speeds.
There is at least one replicator process in each data center that is responsible for replicating payloads. A table in Cassandra tracks the replication status of each payload file in each data center. The replicator processes use this table to find new payloads that exist in other data centers. The table provides a consolidated view of the state of all payloads in the system. This database table is also used when raising events. For example, the Message Add event is recorded when the payload exists in all data centers.
With IBM® Control Center V6.1 or later, you can monitor the status of Global Mailbox. For example, IBM Control Center can show you a down status of Global Mailbox if Global Mailbox stops sending events to it. You can configure IBM Control Center to monitor events based on your business needs.
IBM Control Center monitors the connections and availability of all components of the configured system across multiple data centers. IBM Control Center Web console dashboard monitors processes for Sterling B2B Integrator, Sterling File Gateway, and Global Mailbox. The alerts that are displayed in the Web console dashboard can be configured in Global Mailbox to meet your specific business requirements.
IBM Control Center provides enterprise-wide monitoring and management of Global Mailbox:
- Monitors external systems (but not individual servers), such as Cassandra, ZooKeeper, replication, storage, and WebSphere® MQ.
- Reports the health of processes within Global Mailbox, such as replication of payload, metadata, and events.
- Tracks heartbeat events from WebSphere Application Server Liberty Core, to confirm that the server is available, and to display a list of components and connection information, such as host, port, data center, and file system path.
The Global Mailbox events that are monitored by IBM Control Center are set in the global.properties file and in the mailbox.properties file. Configuration options for Global Mailbox when monitoring with IBM Control Center include:
- Enable or disable individual monitored event categories
- Enable or disable all monitored event categories
- Set the frequency of monitored event checks
IBM® WebSphere® MQ provides a communications layer for visibility and control of the flow of messages and data in Global Mailbox for security-rich message delivery.
WebSphere MQ notifies Sterling B2B Integrator about the events related to Global Mailbox, for example, when a message is added.
The WebSphere MQ configuration spans across the data centers. A mailbox message is processed in the data center where it is received.
Appropriate event rules for the messages must be configured for notifications to function. Sterling B2B Integrator processes the messages based on the event rule configuration. This is equivalent to the Routing Rule feature in the traditional Sterling B2B Integrator mailboxes. Global Mailbox raises events to notify Sterling B2B Integrator to run a business process through WebSphere MQ.
A supported database for each node of Sterling B2B Integrator is required for Global Mailbox. The databases store the application configuration, such as users, business processes, and protocol adapters.
Databases are not shared across data centers. Configuration such as users, business processes, contracts, protocol adapters, and any other required configuration, must be manually synced (through export and import functions of Sterling B2B Integrator) between the databases to ensure consistency for high availability.
A perimeter server is a software tool for communications management that can be installed in a DMZ. The perimeter server manages the communications flow between outer layers of your network and the TCP-based transport adapters. A perimeter server can solve problems with network congestion, security, and scalability, especially in high-volume, Internet-gateway environments.
You can use perimeter servers when you install Sterling B2B Integrator and Global Mailbox in a clustered environment. A cluster is two or more connected copies of Sterling B2B Integrator that share a database. A node is one copy of Sterling B2B Integrator in the cluster.
In a clustered environment, each node may have a perimeter server configured. You can have more than one perimeter server for each node, which enables you to increase the number of connections and improve processing times. However, each perimeter server can serve only one Sterling B2B Integrator node. You can also have many different services and adapters using the same perimeter server.
Sterling File Gateway facilitates the exchange of file-based information securely, in virtually any format, protocol, and file size. Sterling B2B Integrator and Sterling File Gateway add Global Mailbox capability for enhanced disaster recovery.
You can implement Global Mailboxfor your Sterling File Gateway system. Each partner is designated as either a Global Mailbox or a Sterling B2B Integrator traditional mailbox user. With Global Mailbox, messages and metadata are replicated on servers in multiple data centers. This replication enables another data center to continue work on transactions when another data center experiences an outage. Sterling File Gateway can operate with mailboxes in both Sterling B2B Integrator and Global Mailbox, depending on your configuration.
Sterling File Gateway is installed on an instance of Sterling B2B Integrator, and shares many of the resources with the latter, including:
- Communication Adapters
- Business Processes
- Security Services
- Perimeter Services
- Encryption
- Decryption
- Account Management
IBM® Sterling Secure Proxy acts as an application proxy between Global Mailbox and a Sterling B2B Integrator server. It provides a high level of data protection between external connections and your internal network. Create an inbound node definition for each trading partner connection from outside your company and an outbound node definition for every company server to which Sterling Secure Proxy connects.
Sterling Secure Proxy is an optional component that works with Global Mailbox by masking the IP address and HTTP details of the servers to secure the network and data centers.
Browse Categories
Share Blog Post
Pragma Edge (4.5/5)
4.5/5
Source: IBM
- Published on
The Vital Importance of Frequent Health Checks in B2B Electronic Data Interchange (EDI) and Managed File Transfer (MFT) Landscape
April 24, 2024
No Comments
End of Support Notification: IBM Sterling Data Exchange Select Products.
Discontinuance of Support for IBM Sterling Data Exchange Select Products
April 12, 2024
No Comments
End of Support Notification: IBM Sterling Data Exchange Select Products.
