Tuesday, 23 February 2010

Infinispan 4.0.0.Final has landed!

imageIt is with great pleasure that I’d like to announce the availability of the final release of Infinispan 4.0.0. Infinispan is an open source, Java-based data grid platform that I first announced last April, and since then the codebase has been through a series of alpha and beta releases, and most recently 4 release candidates which generated a lot of community feedback.

It has been a long and wild ride, and the very active community has been critical to this release. A big thank you to everyone involved, you all know who you are.

Benchmarks

I recently published an article about running Infinispan in local mode - as a standalone cache - compared to JBoss Cache and EHCache. The article took readers through the ease of configuration and the simple API, and then demonstrated some performance benchmarks using the recently-announced Cache Benchmarking Framework. We’ve been making further use of this benchmarking framework in the recent weeks and months, extensively testing Infinispan on a large cluster.

Here are some simple charts, generated using the framework. The first set compare Infinispan against the latest and greatest JBoss Cache release (3.2.2.GA at this time), using both synchronous and asynchronous replication. But first, a little bit about the nodes in our test lab, comprising of a large number of nodes, each with the following configuration:

  • 2 x Intel Xeon E5530 2.40 GHz quad core, hyperthreaded processors (= 16 hardware threads per node)

  • 12GB memory per node, although the JVM heaps are limited at 2GB

  • RHEL 5.4 with Sun 64-bit JDK 1.6.0_18

  • InfiniBand connectivity between nodes

And a little bit about the way the benchmark framework was configured:

  • Run from 2 to 12 nodes in increments of 2

  • 25 worker threads per node

  • Writing 1kb of state (randomly generated Strings) each time, with a 20% write percentage

+ Reads Writes

Synchronous Replication

image

image

Asynchronous Replication

image

image

As you can see, Infinispan significantly outperforms JBoss Cache, even in replicated mode. The large gain in read performance, as well as asynchronous write performance, demonstrates the minimally locking data container and new marshalling techniques in Infinispan. But you also notice that with synchronous writes, performance starts to degrade as the cluster size increases. This is a characteristic of replicated caches, where you always have fast reads and all state available on each and every node, at the expense of ultimate scalability.

Enter Infinispan’s distributed mode. The goal of data distribution is to maintain enough copies of state in the cluster so it can be durable and fault tolerant, but not too many copies to prevent Infinispan from being scalable, with linear scalability being the ultimate prize. In the following runs, we benchmark Infinispan’s synchronous, distributed mode, comparing 2 different Infinispan configurations. The framework was configured with:

  • Run from 4 to 48 nodes, in increments of 4 (to better demonstrate linear scalability)

  • 25 worker threads per node

  • Writing 1kb of state (randomly generated Strings) each time, with a 20% write percentage

+ Reads Writes

Synchronous Distribution

image

image

As you can see, Infinispan scales linearly as the node count increases. The different configurations tested, lazy stands for enabling lazy unmarshalling, which allows for state to be stored in Infinispan as byte arrays rather than deserialized objects. This has certain advantages for certain access patterns, for example where remote lookups are very common and local lookups are rare.

How does Infinispan comparing against ${POPULAR_PROPRIETARY_DATAGRID_PRODUCT}?

Due to licensing restrictions on publishing benchmarks of such products, we are unfortunately not at liberty to make such comparisons public - although we are very pleased with how Infinispan compares against popular commercial offerings, and plan to push the performance envelope even further in 4.1.

And just because we cannot publish such results, that does not mean that you cannot run such comparisons yourself. The Cache Benchmark Framework has support for different data grid products, including Oracle Coherence, and more can be added easily.

Aren’t statistics just lies? We strongly recommend you running the benchmarks yourself. Not only does this prove things for yourself, but also allows you to benchmark behaviour on your specific hardware infrastructure, using the specific configurations you’d use in real-life, and with your specific access patterns.

So where do I get it?

Infinispan is available on the Infinispan downloads page. Please use the user forums to communicate with us about the release. A full change log of features in this release is on JIRA, and documentation is on our newly re-organised wiki. We have put together several articles, chapters and examples; feel free to suggest new sections for this user guide - topics you may find interesting or bits you feel we’ve left out or not addressed as fully.

What’s next?

We’re busy hacking away on Infinispan 4.1 features. Expect an announcement soon on this, including an early alpha release for folks to try out. If you’re looking for Infinispan’s roadmap for the future, look here.

Cheers, and enjoy!

Manik

Posted by Manik Surtani on 2010-02-23
Tags: release benchmarks final

Tuesday, 16 February 2010

Benchmarking Infinispan and other Data Grid software

Why benchmarking? Benchmarking is an important aspect for us: we want to monitor our performance improvements between releases and compare ourselves with other products as well. Benchmarking a data grid product such as Infinispan is not a trivial task: one needs to start multiple processes over multiple machines, coordinate between them to make sure everything runs at once and centralize reports. Then there is the question of what access patterns the benchmark should stress.

#*Introducing the cache benchmarking framework (CBF) *#What we’ve come up with is a tool to help us run our benchmarks and generate reports and charts. And more: - simple to configure (see config sample bellow) - simple to run. We supply a set of .sh scripts that connect to remote nodes and start cluster instances for you. - open source. Everybody can download it, read the code and run the benchmarks by themselves. Published results can be easily verified and validated. - extensible. It’s easy to extend the framework in order to benchmark against additional products. It’s also easy to write different data access patterns to be tested.

  • scalable. At this moment we’ve used CBF for benchmarking up to 62 nodes.

  • users can test products, configurations, and access patterns on their own hardware and network. This is crucial, since it means educated decisions can be made based on relevant and use-case specific statistics and measurements. Further, the benchmark can even be used to compare performance of different configurations and tuning parameters of a single data grid product, to help users choose a configuration that works best for them

Below is a sample configuration file and generated report.

<bench-config>

  <master bindAddress="${127.0.0.1:master.address}" port="${2103:master.port}"/>

  <benchmark initSize="2" maxSize="${4:slaves}" increment="1">
    <DestroyWrapper runOnAllSlaves="true"/>
    <StartCluster/>
    <ClusterValidation partialReplication="false"/>
    <Warmup operationCount="1000"/>
    <WebSessionBenchmark numberOfRequests="2500" numOfThreads="2"/>
    <CsvReportGeneration/>
  </benchmark>

  <products>
    <jbosscache3>
      <config name="mvcc/mvcc-repl-sync.xml"/>
    </jbosscache3>
    <infinispan4>
      <config name="repl-sync.xml"/>
      <config name="dist-sync.xml"/>
      <config name="dist-sync-l1.xml"/>
    </infinispan4>
  </products>

  <reports>
    <report name="Replicated">
      <item product="infinispan4" config="repl-sync.xml"/>
      <item product="jbosscache3" config="mvcc/mvcc-repl-sync.xml"/>
    </report>
    <report name="Distributed">
      <item product="infinispan4" config="dist-*"/>
    </report>
    <report name="All" includeAll="true"/>
  </reports>

</bench-config>

And this is what a generated charts look like:

image

Where can you find CBF? CBF can be found here. For a quick way of getting up to speed with it we recommend the 5 minutes tutorial.

Enjoy!

Mircea

Posted by Mircea Markus on 2010-02-16
Tags: benchmark

Friday, 12 February 2010

Infinispan/Jopr flash movies released

Back in December we announced the release of a screen cast showing how to monitor Infinispan with Jopr. Today we’ve just added 3 detailed flash movies on how to install Jopr and Infinispan Jopr plugin, and also how to monitor Infinispan instances that have been discovered automatically or have been added manually. You can find these flash movies in the Infinispan wiki.

Cheers, Galder

Posted by Galder Zamarreño on 2010-02-12
Tags: jopr videos

Friday, 12 February 2010

Poll: How do you interact with Infinispan?

While discussing the different ways to interact with Infinispan, we decided to open up a poll so that people tell us how they expect to be using Infinispan. Do you use Infinispan directly on the same VM? Or do you use REST? Are you planning to interact via memcached or Hot Rod interface?

The poll can be found here. Please make sure that if you vote, you add a comment indicating the reasons why you chosen that option.

Cheers, Galder

Posted by Galder Zamarreño on 2010-02-12
Tags: hotrod memcached rest

Thursday, 04 February 2010

Infinispan and storage in the cloud

image

I will be presenting on Infinispan and its role in cloud storage, at Red Hat’s Cloud Computing Forum on the 10th of February 2010.

This is a virtual event, where you get to attend from the comfort of your desk. And although it is free, you do need to register beforehand so I recommend your doing so.

Cheers

Manik

Posted by Manik Surtani on 2010-02-04
Tags: presentations cloud storage

News

Tags

JUGs alpha as7 asymmetric clusters asynchronous beta c++ cdi chat clustering community conference configuration console data grids data-as-a-service database devoxx distributed executors docker event functional grouping and aggregation hotrod infinispan java 8 jboss cache jcache jclouds jcp jdg jpa judcon kubernetes listeners meetup minor release off-heap openshift performance presentations product protostream radargun radegast recruit release release 8.2 9.0 final release candidate remote query replication queue rest query security spring streams transactions vert.x workshop 8.1.0 API DSL Hibernate-Search Ickle Infinispan Query JP-QL JSON JUGs JavaOne LGPL License NoSQL Open Source Protobuf SCM administration affinity algorithms alpha amazon anchored keys annotations announcement archetype archetypes as5 as7 asl2 asynchronous atomic maps atomic objects availability aws beer benchmark benchmarks berkeleydb beta beta release blogger book breizh camp buddy replication bugfix c# c++ c3p0 cache benchmark framework cache store cache stores cachestore cassandra cdi cep certification cli cloud storage clustered cache configuration clustered counters clustered locks codemotion codename colocation command line interface community comparison compose concurrency conference conferences configuration console counter cpp-client cpu creative cross site replication csharp custom commands daas data container data entry data grids data structures data-as-a-service deadlock detection demo deployment dev-preview development devnation devoxx distributed executors distributed queries distribution docker documentation domain mode dotnet-client dzone refcard ec2 ehcache embedded embedded query equivalence event eviction example externalizers failover faq final fine grained flags flink full-text functional future garbage collection geecon getAll gigaspaces git github gke google graalvm greach conf gsoc hackergarten hadoop hbase health hibernate hibernate ogm hibernate search hot rod hotrod hql http/2 ide index indexing india infinispan infinispan 8 infoq internationalization interoperability interview introduction iteration javascript jboss as 5 jboss asylum jboss cache jbossworld jbug jcache jclouds jcp jdbc jdg jgroups jopr jpa js-client jsr 107 jsr 347 jta judcon kafka kubernetes lambda language learning leveldb license listeners loader local mode lock striping locking logging lucene mac management map reduce marshalling maven memcached memory migration minikube minishift minor release modules mongodb monitoring multi-tenancy nashorn native near caching netty node.js nodejs non-blocking nosqlunit off-heap openshift operator oracle osgi overhead paas paid support partition handling partitioning performance persistence podcast presentation presentations protostream public speaking push api putAll python quarkus query quick start radargun radegast react reactive red hat redis rehashing releaase release release candidate remote remote events remote query replication rest rest query roadmap rocksdb ruby s3 scattered cache scripting second level cache provider security segmented server shell site snowcamp spark split brain spring spring boot spring-session stable standards state transfer statistics storage store store by reference store by value streams substratevm synchronization syntax highlighting tdc testing tomcat transactions tutorial uneven load user groups user guide vagrant versioning vert.x video videos virtual nodes vote voxxed voxxed days milano wallpaper websocket websockets wildfly workshop xsd xsite yarn zulip

back to top