If you’re following Infinispan’s mailing lists you’ve probably caught a glimpse of the new developments in the Query land: a new DSL, remote querying via Hot Rod client, a new marshaller based on Google’s Protobuf. Time to unveil these properly!

Starting with version 6.0 Infinispan offers a new (experimental) way of running queries against your cached entities based on a simple filtering DSL. The aim of the new DSL is to simplify the way you write queries and to be agnostic of the underlying query mechanism(s) making it possible to provide alternative query engines in the future besides Lucene and still being able to use the same query language/API. The previous Hibernate Search & Lucene based approach is still in place and will continue to be supported and in fact the new DSL is currently implemented right on top of it. The future will surely bring index-less searching based on map-reduce and possibly other new cool search technologies.

Running DSL-based queries in embedded mode is almost identical to running the existing Lucene-based queries. All you need to do is have infinispan-query-dsl.jar and infinispan-query.jar in your classpath (besides Infinispan and its dependecies), enable indexing for your caches, annotate your POJO cache values and your’re ready.

__

____Alternatively, indexing (and everything else) can also be configured via XML configuration, as already described in the user guide, so we’ll not delve into details here.

Your Hibernate Search annotated entity might look like this.

__

___Running a DSL based query involves obtaining a _https://github.com/infinispan/infinispan/blob/6.0.0.Beta1/query-dsl/src/main/java/org/infinispan/query/dsl/QueryFactory.java[QueryFactory] from the (cache scoped) SearchManager and then constructing the query as follows:

__

___That’s it! I’m sure this raised your curiosity as to what the DSL is actually capable of so you might want to look at the list of supported filter operators in _https://github.com/infinispan/infinispan/blob/6.0.0.Beta1/query-dsl/src/main/java/org/infinispan/query/dsl/FilterConditionEndContext.java[FilterConditionEndContext]. Combining multiple conditions with boolean operators, including sub-conditions, is also possible:

The DSL is pretty nifty right now and will surely be expanded in the future based on your feedback. It also provides support for result pagination, sorting, projections, embedded objects, all demonstrated in QueryDslConditionsTest which I encourage you to look at until the proper user guide is published. Still, this is not a relational database, so keep in mind that all queries are written in the scope of the single targeted entity (and its embedded entities). There are no joins (yet), no correlated subqueries, no grouping or aggregations.

Moving further, probably the most exciting thing about the new DSL is using it remotely via the Hot Rod client. But to make this leap we first had to adopt a common format for storing our cache entries and marshalling them over the wire that would also be cross-language and robust enough to support evolving object schemas. But probably most of all, this format had to have a schema rather than just being an opaque blob otherwise indexing and searching are meaningless. Enter Protocol Buffers.

As mentioned by Mircea in the Infinispan 5.3.0.Beta2 release blog post, interoperability between embedded Infinispan and remote Infinispan modes, including Hot Rod, Memcached and REST is now here!

What this means is that you should be able to store data via one of the endpoints and retrieve data from a different one. So, I can store an Java object using the Java Hot Rod client, and I can retrieve it using the embedded interface.

Documentation for this new interoperability, or compatibility mode, can be found here, including the key aspects of this new functionality, configuration and links to some examples.

As we head towards the later part of the Infinispan 5.3 series, if you’re interested in accessing data in multiple ways, give it a go and let us know what you think!

Cheers, Galder

Dear Infinispan users,

This is hopefully the last CR release of the long expected infinispan 5.2 series. It contains some final touches and bug fixes, especially around the new non-blocking state transfer functionality but also a very useful enhancement to the HotRod protocol (and the Java client) which allows users to fetch the list of keys existing in the cluster - a big thanks to Ray Tsang for contributing this feature! For the complete list of features please refer to the release notes. You can download the distribution or the maven artifact. If you have any questions please check our forums, our mailing lists or ping us directly on IRC!

Cheers, Mircea

Thanks to a sustained effort from  Sunimal Rathnayake, the C# HotRod client has evolved quite a bit:

We’ve also set up a development setupiv document to speed you up in case you want to take a look at the code or contribute. 

You can download it from here - please give it a try and don’t hesitate to post your comments to our forums, the mailing list  or contact us directly on IRC for a chat!

Cheers,

Mircea

A while ago I was announcing that Sunimal Rathnayake would start the work for a C# Hot Rod  client for Infinispan as part of the Google Summer of Code. After 2 months of heavy work Sunimal delivered an intelligence-one (basic client, interested in neither cluster nor hash information) implementation. The release distribution can be downloaded from here. Besides the required binaries and doclets, is also contains a sample application showing how the client can be configured and illustrating the basic operations with the server. This and more are being described in the readme.txt file in the distribution root.

And there’s much more on the way! Sunimal is working on enhancing the client to the intelligence-two level: topology-aware client, interested in cluster information - stay tuned!

Please give it a try and don’t hesitate to post your comments to our forums, the mailing list  or contact us directly on IRC for a chat!

Cheers, Mircea

Infinispan 'Brahma' 5.1.5.FINAL has now been released fixing a whole bunch of issues around cache store preloading of distributed caches, Memcached server, tree module and Hot Rod client performance. We’ve also updated several libraries such as Netty (to 3.4.6) and JGroups (to 3.0.10).

Full details of what has been fixed can be found here, and if you have feedback, please visit our forums. Finally, as always, you can download the release from here.

Cheers, Galder

I’m very proud to announce yet another beta release in the 5.1 'Brahma' series. This time is the turn of Infinispan 5.1.0.BETA3 which apart from containing many small fixes, it comes with two major improvements:

Fine-grained Atomic Maps

Atomic Maps are special constructs that users can use to bundle data into the value side of a key/value pair. What’s special about them is that when the map changes, only the changes or deltas of that map are transfered, which makes Atomic Maps very efficient from a replication perspective when individual elements are modified.

Up until Infinispan 5.1.0.BETA2, the other interesting characteristic of these Atomic Maps was the fact that Atomic Map locking and isolation happened at the the level of the entire Atomic Map. So, if a single key/value pair in the Atomic Map was modified, the entire map was locked.

Starting with Infinispan 5.1.0.BETA3, thanks to Vladimir Blagojevic, Atomic Maps supporting fine-grained locking are available as well. What this means is that an Atomic Map’s key/value pairs can be modified in parallel thanks to the ability to lock individual map entries as opposed to the entire map.

This will be particularly handy for heavy Atomic Map users such as JBoss Application Server 7 which uses Atomic Maps for maintaining HTTP sessions, and Hibernate OGM which decomposes entities into Atomic Maps.

Hot Rod server topology improvements

When we originally designed Hot Rod protocol version 1.0, we decided that whenever a distributed cache wanted to send information about the topology of the backend servers to the clients, we’d send the hash ids of each of these nodes. At the time, this seemed like a good idea, until virtual nodes were implemented…​

With virtual nodes, each physical Hot Rod server can potentially represent tens, hundreds or even thousands of different virtual nodes. If we stuck with the original protocol, that would mean that we’d have to send each virtual node’s hash id back to the client. So, for a cluster of 8 nodes, and 1000 virtual nodes, that’d be at least 80kb of hash ids being transfered back to the client, on top of tons of redundant information about a node’s host and port, which is very inefficient.

So, after having some discussions, we decided to evolve the Hot Rod protocol to version 1.1 in order to address this issue. The end result is that now it’s the responsibility of the Hot Rod client to generate the hash ids of each of the physical nodes. We do that by sticking to a general formula to generate a Hot Rod server’s hash id which both the Hot Rod server and clients can implement.

This improvement has also lead to the significant decrease in memory consumption of the Hot Rod server because it does not need to cache those hash ids anymore.

So, if you are using Infinispan Hot Rod servers and in particular you’are configuring virtual nodes, you definitely should be upgrading your Hot Rod server and client libraries. From a client code perspective, no changes are necessary because starting with 5.1.0.BETA3, Hot Rod clients talk to servers using this latest protocol.

Finally, remember to use user forums to report back, grab the release here, enjoy and keep the feedback coming!!

Cheers, Galder

The first beta version of the Infinispan native python client to connect to the Infinispan Hot Rod servers is available now from Infinispan’s GitHub repository or PyPi. Please check the latest README information to find out more about installation instructions.

So, what’s in it? For this first beta release, we’ve implemented all the operations available in the Hot Rod specification and they’ve been tested with string inputs. This release does not include code to deal with client intelligence superior to 1, so clients cannot receive notifications of cluster changes nor can they locate data in an intelligent manner by applying consistent hash algorithms to the keys. These capabilities, plus further examples such as how to integrate with payloads from Google Protocol Buffers will come in next beta releases.

If you’re looking for code examples for the Infinispan Python client, you can find a very basic one in the README file itself, or check the testsuite within the source package. In next releases we’ll be creating a wiki with more detailed examples.

As always, if you have any doubt please use the user forums and for any bugs or feature requests, please log them via JIRA.

Cheers, Galder

I’m delighted to announce the release of Infinispan 4.1.0.BETA1. For this, our first beta release of the 4.1 series, we’ve finished Hot Rod and Memcached based server implementations and a Java-based Hot Rod client has been developed as a reference implementation. Starting with 4.1.0.BETA1 as well, thanks to help of Tom Fenelly, Infinispan caches can be exposed over a WebSocket.

A detailed change log is available and the release is downloadable from the usual place.

For the rest of the blog post, we’d like to share some of the objectives of Infinispan 4.1 with the community. Here at ‘chez Infinispan’ we’ve been repeating the same story over and over again: http://www.parleys.com/sl=1&st=5&id=1589[‘Memory is the new Disk, Disk is the new Tape’] and this release is yet another step to educate the community on this fact. Client/Server architectures based around [#SPELLING_ERROR_12 .blsp-spelling-error]#Infinispan data grids are key to enabling this reality but in case you might be wondering, why would someone use Infinispan in a client/server mode compared to using it as peer-to-peer (p2p) mode? How does the client/server architecture enable memory to become the new disk?

Broadly speaking, there three areas where a Infinispan client/server architecture might be chosen over p2p one:

1. Access to Infinispan from a non-JVM environment

Infinispan’s roots can be traced back to JBoss Cache, a caching library developed to provide J2[SPELLING_ERROR_19 .blsp-spelling-error]#EE application servers with data replication. As such, the primary way of accessing Infinispan or JBoss Cache has always been via direct calls coming from the same JVM. However, as we have repeated it before, Infinispan’s goal is to provide much more than that, it aims to provide data grid access to any software application that you can think of and this obviously requires Infinispan to enable access from non-Java environments.

Infinispan comes with a series of server modules that enable that precisely. All you have to do is decide which API suits your environment best. Do you want to enable access direct access to Infinispan via HTTP? Just use our REST or WebSocket modules. Or is it the case that you’re looking to expand the capabilities of your Memcached based applications? Start an Infinispan-backed and your existing Memcached clients will be able to talk to it immediately. Or maybe even you’re interested in accessing Infinispan via Hot Rod? Then, gives us a hand developing non-Java clients that can talk the Hot Rod protocol! :).

2. Infinispan as a dedicated data tier

Quite often applications running running a p2p environment have caching requirements larger than the heap size in which case it makes a lot of sense to separate caching into a separate dedicated tier.

It’s also very common to find businesses with varying work loads overtime where there’s a need to start business processing servers to deal with increased load, or stop them when work load is reduced to lower power consumption. When Infinispan data grid instances are deployed alongside business processing servers, starting/stopping these can be a slow process due to state transfer, or rehashing, particularly when large data sets are used. Separating Infinispan into a dedicated tier provides faster and more predictable server start/stop procedures – ideal for modern cloud-based deployments where elasticity in your application tier is important.

It’s common knowledge that optimizations for large memory usage systems compared to optimizations for CPU intensive systems are very different. If you mix both your data grid and business logic under the same roof, finding a balanced set of optimizations that keeps both sides happy is difficult. Once again, separating the data and business tiers can alleviate this problem.

You might be wondering that if Infinispan is moved to a separate tier, access to data now requires a network call and hence will hurt your performance in terms of time per call. However, separating tiers gives you a much more scalable architecture and your data is never more than 1 network call away. Even if the dedicated Infinispan data grid is configured with distribution, a Hot Rod smart-client implementation - such as the Java reference implementation shipped with Infinispan 4.1.0.BETA1 - can determine where a particular key is located and hit a server that contains it directly.

3. Data-as-a-Service (DaaS)

Increasingly, we see scenarios where environments host a multitude of applications that share the need for data storage, for example in Plattform-as-a-Service cloud-style environments (whether public or internal). In such configurations, you don’t want to be launching a data grid per each application since it’d be a nightmare to maintain – not to mention resource-wasteful. Instead you want deployments or applications to start processing as soon as possible. In these cases, it’d make a lot of sense to keep a pool of Infinispan data grid nodes acting as a shared storage tier. Isolated cache access could easily achieved by making sure each application uses a different cache name (i.e. the application name could be used as cache name). This can easily achieved with protocols such as Hot Rod where each operation requires a cache name to be provided.

Regardless of the scenarios explained above, there’re some common benefits to separating an Infinispan data grid from the business logic that accesses it. In fact, these are very similar to the benefits achieved when application servers and databases don’t run under the same roof. By separating the layers, you can manage each layer independently, which means that adding/removing nodes, maintenance, upgrades…​etc can be handled independently. In other words, if you wanna upgrade your application server or servlet container, you don’t need to bring down your data layer.

All of this is available to you now, but the story does not end here. Bearing in mind that these client/server modules are based around reliable TCP/IP, using Netty, they could also in the future form the base of new functionality. For example, client/server modules could be linked together to connect geographically separated Infinispan data grids and enable different disaster recovery strategies.

So, download Infinispan 4.1.0.BETA1 righty, give a try to these new modules and let us know your thoughts.

Finally, don’t forget that we’ll be talking about Hot Rod in Boston at the end of June for the first ever JUDCon. Don’t miss out!

Cheers,

Galder

We’ve just released Infinispan 4.1.0.Alpha2 with even more new functionality for the community to play with. Over the past few weeks we’ve been going backwards and forwards in the Infinispan development list discussing Infinispan’s binary client server protocol called Hot Rod and in 4.1.0.Alpha2 we’re proud to present the first versions of the Hot Rod server and java client implementations. Please visit this wiki to find out how to use Hot Rod’s java client client and server. Please note that certain functionality such as clients receiving topology and hashing information has not yet been implemented.

Besides, Infinispan 4.1.0.Alpha2 is the first release to feature the new LIRS eviction policy and the new eviction design that batches updates, which in combination should provide users with more efficient and accurate eviction functionality.

Another cool feature added in this release is GridFileSystem: a new, experimental API that exposes an Infinispan-backed data grid as a file system. Specifically, the API works as an extension to the JDK’s File, InputStream and OutputStream classes. You can read more on GridFileSystem here.

Finally, you can find the API docs for 4.1.0.Alpha2 here and again, please consider this an unstable release that is meant to gather feedback on the Hot Rod client/server modules and the new eviction design.

Cheers, Galder & Mircea