The main components of Global Mailbox and the architecture for the components to work together
High availability in Global Mailbox is based on specific availability principles. There are some limitations to the high availability properties of Global Mailbox.
Global Mailbox is a facility for creating a directory of mailboxes with submailboxes and hosting them across multiple data centers. Messages are created by Global Mailbox for applications (on behalf of trading partners, as application users) and stored in mailboxes.
A mailbox is a secure document payload repository. Mailboxes are secure because there is a permission model that controls who can access which mailbox.
By creating a network of multiple data centers that host mailboxes, the mailboxes can be available even if a data center is not operational. Other data centers in the network continue to perform the transactions. Data, including metadata, is replicated between the data centers.
Mailboxes are have the following features:
A message consists of a payload and metadata.
Message metadata includes message name (file name), size of the file (that is the size of the payload), payload type (inline storage or shared file system storage), and depending on the payload type, the payload itself (inline) or a payload reference identifier. The metadata is stored in Cassandra. The payload is an actual business document or file. A message can have only one payload.
In Global Mailbox, you can configure a threshold for payload size. If the size of the payload is more than the threshold, the payload is stored in a shared file system storage. If the size is less than the threshold size, it is stored inline with the metadata in Cassandra, as a blob.
Message replication is the replication of both the message metadata and the message payload. Cassandra handles the metadata replication transparently. If the payload of a message is inline, replication of payload is handled by Cassandra. If the payload is not inline, its replication is performed by a replication server.
Global Mailbox administrators can manage messages from the Mailbox Explorer page. Global Mailbox administrators can delete messages or view all messages that a specific Global Mailbox contains.
Each user must be assigned a virtual root to access Global Mailbox.
The virtual root is the first level of the directory path for a user when they are navigating the mailbox navigation pane.
To support limited visibility into the mailbox hierarchy, mailboxes are visible to the user as a relative path, while administrators see the mailbox in an absolute or full path. This concept is referred to as the virtual root .
An administrator user always has a virtual root of / (slash). If a standard user is changed to an admin user, that user is assigned a virtual root value of / (slash). No default virtual root is assigned to users. An admin must assign a virtual root before users can access Global Mailbox.
A Dead Letter mailbox is a mailbox that contains AS2 messages that might not be routed for some reason. The Dead Letter mailbox is available by default in Global Mailbox under the root mailbox as /DeadLetter. A Dead Letter mailbox cannot be deleted or renamed.
If you already have a mailbox that is called Dead Letter in your setup, you must delete or rename the mailbox before you upgrade to v6.0.1 or later. Otherwise, the existing Dead Letter mailbox is overwritten with the new Dead Letter mailbox and your existing messages are lost.
You can create submailboxes under the Dead Letter mailbox. You can also move messages from the Dead Letter mailbox or delete them. While moving messages to a target mailbox, if the target mailbox has an event rule that matches the messages, events are generated and the files are routed.
An administrator can check the messages in the Dead Letter mailbox periodically for the messages that are not routed correctly and resolve them.
The Global Mailbox supports uploading and downloading of messages with same names (duplicate) through SFTP and FTP.
Duplicate message support is case insensitive. For example, if you have a message with name test, and another one with TEST or teST then the messages are considered as duplicate of each other. When you download a message with duplicate name, the newest extractable message is downloaded. If extraction count is configured for the newest file, and if the extraction count of the file reaches 0 remaining extractions, then the next download fetches the next newest duplicate file.
The listDuplicates property in the ftpserver.properties and sftpserver.properties files controls how the FTP Server adapter and the SFTP Server adapter handle presence of multiple files with same name in a mailbox. When the property is set to true, the FTP and SFTP Server adapters list all the duplicate messages. When the property is set to false, only the newest file is shown in the list.
Additionally, for FTP and SFTP transfers, you can configure the Support for concurrent duplicate named file transfers parameter, which is available in the FTP and SFTP Server adapter configuration wizards. Currently, Limited and Full (resume of file transfers not supported) options are supported for Global Mailbox. For more information about the parameter and supported options, see the FTP Server Adapter topic.
For FTP and SFTP transfers, if you configure the FTP or SFTP Server adapter to enable Global Mailbox, and also select the option Full, concatenate duplicate-named files on a GET (resume of file transfers not supported), the adapter ignores the concatenation request when operating on Global Mailbox. Instead, the adapter returns the newest message with the name that is specified in the request. If you enable Global Mailbox and concatenation, a warning message, stating that the concatenation setting is ignored, is printed in the adapter logs.
Multiple messages with the same message name are processed to present the correct message or messages to the user.
To support the feature of having messages with the same names, set the property com.ibm.mailbox.messages.allowDuplicates. This property is specified per application. Valid values are true and false (default). To specify a value for the property, use the command-line application registration utility. By default the property is set to false during installation. You can update the configuration later as required.
To update the property, enter the following command:
./appConfigUtility.sh updateAppConfig --appName=Sterling Integrator Instance 1 --Pcom.ibm.mailbox.messages.allowDuplicates=true(Linux® or UNIX)
appConfigUtility.bat updateAppConfig --appName=Sterling Integrator Instance 1 --Pcom.ibm.mailbox.messages.allowDuplicates=true
Global Mailbox administrators can manage mailboxes, messages, permissions, virtual roots, and event rules in the Global Mailbox management tool.
Mailbox Explorer
From the Mailbox Explorer page, Global Mailbox administrators can manage all messages, submailboxes, permissions, and event rules for all mailboxes in the Global Mailbox system. The mailbox navigation tree allows Global Mailbox administrators to view all mailboxes and submailboxes on the Mailbox Explorer page.
User Explorer
Event Explorer
With the Events tab on the Event Explorer page, Global Mailbox administrators can resend all events by application that match the filter criteria that is specified. For example, if one or more events were unsuccessfully processed, Global Mailbox administrators can resend events for processing by specifying filter criteria that matches the unsuccessfully processed events.
Administrative users in Sterling B2B Integrator (admins) are granted all permissions in Global Mailbox.
When a service or adapter is configured with a repository type of Global Mailbox, a user designated as a Sterling B2B Integrator administrator is granted all rights and permissions to Global Mailbox resources, without having to specifically add them. There is no scenario whereby a Sterling B2B Integrator administrator gets a permission denied or authorization exception type error because they have all permissions within Global Mailbox.
The virtual root must be set for each admin user in Global Mailbox. If an admin user without a Global Mailbox virtual root accesses an FTP or SFTP server adapter that is enabled for Global Mailbox, the user is directed to traditional mailboxes and not to Global Mailbox.
Admins can directly access the Global Mailbox management tool from Sterling B2B Integrator if single sign-on is configured. Admins must have the necessary permissions in Sterling B2B Integrator to access the Global Mailbox management tool by single sign-on.
In Global Mailbox, admins are not listed on pages where permissions are assigned. Permissions for admins cannot be restricted.
Global Mailbox administrators can create event rules to automatically initiate specific actions when a message is added to a Global Mailbox.
With event information, Global Mailbox administrators can monitor each individual message transfer in the Global Mailbox system. If one or more events are unsuccessfully processed, a Global Mailbox administrator can resend the events for processing from the Global Mailbox management tool.
An event rule is a specific action, or set of actions, that is performed by an application when a message is added to a Global Mailbox that the event rule applies to. Global Mailbox administrators can create event rules to automate event processing when messages are added to specified mailboxes. When a message is added to a Global Mailbox, all event rules that apply to the mailbox are automatically evaluated.
When an event is evaluated, this means that the event is sent to be processed by the application that it is configured to be processed by. An event rule ensures that the event is automatically evaluated at least once.
Event rules can be temporarily enabled or disabled from the Mailbox Explorer or Event Explorer pages.
When a file is uploaded, events are posted to the B2Bi application queue immediately. The file is uploaded to the local data center without waiting for being replicated to other data centers.
For event processing to be successful, Cassandra is required in the local data center.
In event processing, protocol traffic is automatically routed to a remote data center if a WebSphere MQ outage occurs on the local data center. Files are not processed more than once. In case of failures, all files are automatically processed.
If event processing fails because the queue manager failed or the WebSphere MQ service failed, the files are processed when the events are resent through the UI or through the eventUtility script. The files are resent to the data center from which the files originated, and the files are then processed.
Availability, performance, consistency, and durability of Global Mailbox are achieved by the replication feature.
Global Mailbox is, in essence, a distributed file system (DFS). Mailboxes correspond to directories in a DFS, and messages correspond to files within DFS directories. Files, much like messages, store two kinds of information:
In Global Mailbox, replication consists of two subsystems:
Unlike traditional Sterling B2B Integrator mailboxing, Global Mailbox can automatically replicate both the metadata and payloads associated with messages to remote data centers, thereby providing an increased degree of fault tolerance.
It is expected of a distributed system like Global Mailbox to be available most of the times, to perform at the best, to maintain data consistency, and to be durable. However, there are various trade-offs that must be considered by you, based on your business requirements, before configuring each functionality.
Availability
Availability refers to the proportion of time during which a particular service or a system in Global Mailbox is operating at functional capacity. For instance, we consider the payload replication service available only when users can successfully upload new payloads to Global Mailbox. The metadata replication service, on the other hand, is available only when sufficient number of Cassandra instances are online to service read and write requests.
Performance
Two primary concepts are involved in performance:
Minimizing response time and maximizing throughput are both desirable performance goals. In general, it is possible that some operations might have high response times but scale well across concurrent sessions, that is, be high-throughput. Similarly, low-throughput operations might complete quickly in the context of an individual session.
Consistency
Consistency defines the congruency of visible states within a computer system. A distributed system, comprised of different nodes can provide strong consistency only if every component can observe the same state in the same order. Weak consistency, on the other hand, does not provide this guarantee. Apache Cassandra supports a consistency model known as eventual consistency. There are chances, that an eventually consistent system might provide an outdated answer to a query. However, after a sufficient period of time passes, during which no component failures occur, all components in an eventually consistency system respond to a query with the same answer.
Though an eventually consistent system has potential benefits to performance and availability, it is suggested not to configure eventual consistency in Cassandra due to data integrity related issues in Global Mailbox.
Durability
Durability is the guarantee that any operation that completed successfully is not lost, rolled back, or changed due to a component failure. Durability is often compromised in distributed systems to improve performance and availability.
Configuration options
Payload replication and metadata replication provide different levels of availability, performance, consistency, and durability functionality. Availability, performance, consistency, and durability can be configured for payload replication. However for metadata replication, only availability, performance, and durability can be configured. You must retain the default consistency level for metadata replication.
In the Global Mailbox system, storage (file system) implementation is based on the concept of storage buckets. The buckets are containers (logical groups) in the file system, which are configured according to business requirements based on security and retention.
The storage system includes buckets, which store blobs. Buckets include variants, which are versions of the buckets. Blobs are stored in the variants. You must configure at least one bucket and variant for storage to operate.
A variant can have a different configuration (for example, encryption settings). Each variant within a bucket is identified by a unique variant identifier (0 - 63). Variants can be marked as retired after which a variant becomes read only. The data in blobs is distributed among the active variants of the bucket. During a READ or GET operation, the blob is retrieved from the variant it exists in.
File transfer process flow
Storage configuration
When you install the initial Global Mailbox node, storage buckets (1st_provisioned and global_mbx) and the first variant (0) are created. By default, Global Mailbox uses the global_mbx bucket to store message payload. When installing the initial Global Mailbox node, you must also specify the shared storage path for all other data centers. The configuration information (1st_provisioned and global_mbx buckets, global.properties and installinfo.properties files), is copied to the shared storage path specified for other data centers after installing the initial Global Mailbox node.
When you configure your storage system, you must decide how you want to use buckets and variants to store the different kinds of information that flow through the Global Mailbox.
Pause and resume
If resumption is configured for a file that is uploaded, the file is broken into segments.
If a file is broken into four segments, a total of five files, one for each segment, and a stub file, are created and stored on the disk. The stub file contains metadata with information to reassemble the segments.
If the upload is interrupted (either by pause or loss of network connection) in the middle of uploading a segment, the whole segment is removed, and upload is resumed when the error is corrected.
A scheduled job is a regularly performed task on the Global Mailbox system to automate routine operations and to maintain optimal system performance. You can modify the configuration of scheduled jobs with the schedulerConfigUtility script to meet your system requirements.
Scheduled jobs perform regular operations to maintain optimal Global Mailbox system performance. For example, the PayloadPurgeJob routinely identifies and removes orphaned payloads from the Global Mailbox system by deleting the orphan payload data from Cassandra. An orphaned payload is a message payload that is no longer referenced by a message.
You can set the frequency and schedule for jobs to meet your business requirements, with different schedules and frequencies for different jobs depending on your data patterns. A job can be scheduled for a specific time of day, such as 2 AM, with a frequency between daily and every several months.
Purging of data follows a regular, automated schedule to maintain Global Mailbox system performance. You can modify the configuration of purge jobs and purge job triggers with the schedulerConfigUtility script.
Removing unneeded items prevents allocation of resources and prevents performance deterioration.
Items that are purged cannot be recovered.
You can convert Sterling File Gateway partners and mailboxes to use Global Mailbox. You can create new accounts in Global Mailbox for Sterling B2B Integrator users and mailboxes and use adapters that are enabled for Global Mailbox. You cannot migrate users and mailboxes from one Global Mailbox system to another.
After Global Mailbox is installed and configured, Sterling File Gateway partners can be converted to enable Global Mailbox functionality. After a partner is converted in one data center, export the related resources and import them to the other data centers to take advantage of the features and architecture of Global Mailbox.
To use the features and advantages of Global Mailbox, you can convert the existing Sterling File Gateway partners (producer and consumer) to use Global Mailbox. You can only convert a single partner at a time. A Sterling File Gateway route provisioner, administrator, architect, or operator can convert a partner to use Global Mailbox. The conversion is irreversible. If for any reason you no longer want to use Global Mailbox, you must re-create the partner and routes in Sterling File Gateway.
When you convert the partners to use Global Mailbox, mailboxes that are equivalent to the existing traditional mailboxes are created in the Global Mailbox realm for the partner. User permissions and virtual roots are also created in the Global Mailbox realm. The user accounts associated with a Sterling File Gateway partner remain in Sterling B2B Integrator when the partner is converted to use Global Mailbox. Permissions for the user on certain mailboxes are granted within the Global Mailbox system, but the user account remains within Sterling B2B Integrator.
Traditional mailboxes continue to exist, but are no longer used. You can move the messages in traditional mailboxes to Global Mailbox at the time you convert the partner. After a partner is converted to use Global Mailbox, new messages for that partner are written to the mailboxes in Global Mailbox.
By creating your mailboxes and virtual roots in Global Mailbox, you can gain Global Mailbox advantages while continuing to use your existing Sterling B2B Integrator business processes. If you want to limit partner-facing impact, you may want to re-direct your load balancers to new Global Mailbox enabled adapters. Some downtime would be required while you adjust your adapters, but by re-using the externally facing ports, you may not need to have partners make additional adjustments.
With file transfer resumptions, active file transfers that are interrupted, disrupted, or stopped can be resumed because the Global Mailbox saves data prior to the occurrence of errors.
If an active file transfer is interrupted due to a network error, the protocol adapter getting disrupted or stopped, or the Sterling B2B Integrator node going down, the Global Mailbox saves the data that was uploaded before the error occurred.
To support transfer resumption, the server adapters in Sterling B2B Integrator store the incomplete files in a temporary document staging area. This allows FTP and SFTP clients to resume a transfer. File transfer is resumed on another server in the same data center.
With Load balancer, if an FTP server adapter from a specific data center goes down, an error occurs. When you resume the file transfer, a connection is established to the existing data center if the data center is not down and the file upload is successfully resumed.
However, if the data center is down, and a connection to the other data center is attempted, then resume fails. In this case, file resumption is possible only in the data center in which the file was initially uploaded.
The Global Mailbox implementation supports REST services to view and consume some resources in the Sterling B2B Integrator and Sterling File Gateway.
gmbx_user is shipped with permissions (Global Mailbox User Lookup REST API and Global Mailbox Event Property REST API permission) to the supported REST services. Each supported REST service is in its own servlet and shares a .WAR file with the other services.To establish a connection to the Global Mailbox system, Sterling B2B Integrator is registered with Global Mailbox by using the appConfigUtility during the installation. When registering Sterling B2B Integrator, the REST services are also registered with Global Mailbox. The registration is automated and does not require any manual intervention. However, if you need to change any parameters for user lookup or event data lookup, then you can run the following commands after installing Global Mailbox and Sterling B2B Integrator:
appConfigUtility invocation for user lookup:appConfigUtility invocation for event data lookup:To connect to Sterling B2B Integrator, and access the required resources through Global Mailbox REST Service adapter (of type HTTP Server adapter), Global Mailbox must have the following information. The installer configures the REST parameters during the installation.
Element | Description | Example |
|---|---|---|
| Request method | The REST request method. Only GET is supported. | GET |
| Host name | The host where the Global Mailbox REST Services adapter instance (HTTP Server adapter type) is running. It is a Sterling B2B Integrator host. | si.myco.com |
| Port | The listening port of the Global Mailbox REST Services adapter. An appropriate port is configured by the Sterling B2B Integrator administrator. | 9876 |
| Resource path | The REST resource path. It must start with /api/V1 | /api/V1/users
|
| User name | User name for basic authentication. A new Sterling B2B Integrator user is supported. Any other user with permission corresponding to the specific REST API can also be used. The gmbx_user user is granted the following permissions to execute the rest services:
| gmbx_user |
| Password | Password for basic authentication. The password is configured by the Sterling B2B Integrator administrator. | password |
| Server certificate | SSL certificate that represents the host Sterling B2B Integrator identity. Might be self-signed or signed by a certifying authority. The client (Global Mailbox) must be configured to trust this certificate as representing the server (Sterling B2B Integrator). | Certificate file that is downloaded from Sterling B2B Integrator and associated with HTTPS Server adapter. |
Note: If the auth password is modified using the command line in the setUserLookup utility, ensure you update the same in the setEventDataLookup utility.
For example:
./appConfigUtility.sh setUserLookup --authPassword="newpc4us" --appName="B2Bi" --adminUser="admin" --adminPassword="password" ./appConfigUtility.sh setEventDataLookup --authPassword="newpc4us" --appName="B2Bi" --adminUser="admin" --adminPassword="password"
User Lookup REST Service
The User Lookup REST Service is hosted in the Global Mailbox REST Services adapter, and can be accessed from the URI /api/V1/users/. After successfully authenticating the user, the request returns a list of users who are in the Active state in Sterling B2B Integrator. If required, a Global Mailbox administrator can optionally choose to view Sterling B2B Integrator administrator users also.
Event Properties REST Service
The Event Properties REST Service is hosted in the Global Mailbox REST Services adapter, and can be accessed from the URI /api/V1/event_properties/. After successfully authenticating the user, the request returns a list of Sterling B2B Integrator supported event properties, such as, business process name, contract name, and user. By default, the service returns information on mandatory and invalid event property combinations. If required, the Global Mailbox administrator can initiate a second call to view the allowed values, for example, /api/V1/event_properties/BPNAME/allowed_values.
The performance of your IBM® Global High Availability Mailbox system depends upon the interaction of the components, the communication paths between components, and the configuration settings. By monitoring your performance and tuning your system, you can balance file transfer speed with availability and protection from data loss. Your business requirements must be a consideration in designing and tuning performance.
Security for your Global Mailbox system must be planned and implemented as an integrated aspect of your deployment, and carefully monitored and administered to continue evolving to meet new risks as they form. By taking appropriate steps, your organization can minimize and mitigate threats.
Global Mailbox relies on the connecting applications, such as Sterling B2B Integrator and Sterling File Gateway to authorize end users. Multiple applications can access the same mailboxes, depending on permissions. Careful management of users in all applications is necessary to ensure security.
Administrators are authenticated with a user ID and password. Certain parts of the Global Mailbox management tool can only be accessed by administrators. Because of the extensive privileges granted to administrators, the user ID and password must be carefully controlled, changed frequently, and adhere to strict rules for complexity. Delete the default administrator after you have created a unique one for your system.
IBM Global Mailbox - components at a glance
IBM Global Mailbox - overview
Browse Categories
Share Blog Post
In today’s fast-paced world of data analytics and AI, optimizing your data infrastructure is key to unlocking valuable insights and driving innovation.
In today’s fast-paced world of data analytics and AI, optimizing your data infrastructure is key to unlocking valuable insights and driving innovation.
In today’s fast-paced world of data analytics and AI, optimizing your data infrastructure is key to unlocking valuable insights and driving innovation.
At Pragma Edge, we are a forward-thinking technology services provider dedicated to driving innovation and transformation across industries.
Thank you for submitting your details.
For more information, Download the PDF.
Thank you for the Registration Request, Our team will confirm your request shortly.
Invite and share the event with your colleagues
IBM Partner Engagement Manager Standard is the right solution
addressing the following business challenges
IBM Partner Engagement Manager Standard is the right solution
addressing the following business challenges
IBM Partner Engagement Manager Standard is the right solution
addressing the following business challenges