Blogs Cross Site Replication Improvements

Cross Site Replication Improvements

June 05, 2020 Tags: xsite cross site replication

By Pedro Ruivo

Infinispan introduced Cross Site Replication functionality in version 5.2 and Infinispan 7 extended it to support state transfer. With an increase of popularity, Infinispan 11 brings two major improvements to Cross Site Replication. Let’s take a look at them.

Support for multiple site masters

Infinispan uses JGroups' RELAY2 protocol to enable inter-site communication. Each site has Site Masters: these are nodes with special roles, and are responsible for the communication between sites.

RELAY2 can use more than one Site Master per site allowing load balancing of the inter-site requests. The new algorithm is now able to take advantage of multiple Site Masters.

The attribute max_site_masters configures the number of Site Masters, and it defaults to 1. To take advantage of the new algorithm, increase the number of Site Masters in RELAY2 configuration by changing max_site_masters to a value higher than 1. A number greater than the number of nodes can be used, and it enables the Site Master role in all nodes.

<relay.RELAY2 site="<LOCAL_SITE_NAME>" max_site_masters="<PUT_VALUE_HERE>"/>

More information about RELAY2 is available in JGroups' Manual.

Conflict detection and resolution for Asynchronous Cross-Site Replication

Infinispan is able to detect conflicts in asynchronous mode by taking advantage of vector clocks. A conflict happens when 2 or more sites update the same key at the same time. Let’s look at an example between 2 sites (LON and NYC):

            LON       NYC

k1=(n/a)    0,0       0,0

k1=2        1,0  -->  1,0   k1=2

k1=3        1,1  <--  1,1   k1=3

k1=5        2,1       1,2   k1=8

                 -->  2,1 (conflict)
(conflict)  1,2  <--

k1=5        2,1  <->  2,1   k1=5

LON puts k1=2, with vector clock 1,0, and replicates it to NYC.
NYC puts k1=3, with vector clock 1,1, and replicates it to LON.
However if LON puts k1=5 (with vector 2,1), and NYC puts k1=8 in NYC (with vector clock 1,2) at the same time, Infinispan detects the conflict since none of the vector clocks are greater than the other.

Infinispan resolves the conflicts by comparing using the site names in lexicographical order. The site’s name lower in lexicographical order takes priority. In the example above, both LON and NYC end up with k1=5 since LON < NYC.

You can choose the priority by prepending a number to the site name. For example, if you want updates from NYC to take priority over LON updates, you can prepend a number to the site name, example: 1NYC, 2LON and so on.

For more information check the Infinispan Documentation.

Get it, Use it, Ask us!

We’re hard at work on new features, improvements and fixes, so watch this space for more announcements!

Please, download and test the latest release.

The source code is hosted on GitHub. If you need to report a bug or request a new feature, look for a similar one on our JIRA issues tracker. If you don’t find any, create a new issue.

If you have questions, are experiencing a bug or want advice on using Infinispan, you can use GitHub discussions. We will do our best to answer you as soon as we can.

The Infinispan community uses Zulip for real-time communications. Join us using either a web-browser or a dedicated application on the Infinispan chat.