Benchmarking Infinispan and other Data Grid software
Why benchmarking? Benchmarking is an important aspect for us: we want to monitor our performance improvements between releases and compare ourselves with other products as well. Benchmarking a data grid product such as Infinispan is not a trivial task: one needs to start multiple processes over multiple machines, coordinate between them to make sure everything runs at once and centralize reports. Then there is the question of what access patterns the benchmark should stress.
#*Introducing the cache benchmarking framework (CBF) *#What we’ve come up with is a tool to help us run our benchmarks and generate reports and charts. And more: - simple to configure (see config sample bellow) - simple to run. We supply a set of .sh scripts that connect to remote nodes and start cluster instances for you. - open source. Everybody can download it, read the code and run the benchmarks by themselves. Published results can be easily verified and validated. - extensible. It’s easy to extend the framework in order to benchmark against additional products. It’s also easy to write different data access patterns to be tested.
-
scalable. At this moment we’ve used CBF for benchmarking up to 62 nodes.
-
users can test products, configurations, and access patterns on their own hardware and network. This is crucial, since it means educated decisions can be made based on relevant and use-case specific statistics and measurements. Further, the benchmark can even be used to compare performance of different configurations and tuning parameters of a single data grid product, to help users choose a configuration that works best for them
Below is a sample configuration file and generated report.
<bench-config>
<master bindAddress="${127.0.0.1:master.address}" port="${2103:master.port}"/>
<benchmark initSize="2" maxSize="${4:slaves}" increment="1">
<DestroyWrapper runOnAllSlaves="true"/>
<StartCluster/>
<ClusterValidation partialReplication="false"/>
<Warmup operationCount="1000"/>
<WebSessionBenchmark numberOfRequests="2500" numOfThreads="2"/>
<CsvReportGeneration/>
</benchmark>
<products>
<jbosscache3>
<config name="mvcc/mvcc-repl-sync.xml"/>
</jbosscache3>
<infinispan4>
<config name="repl-sync.xml"/>
<config name="dist-sync.xml"/>
<config name="dist-sync-l1.xml"/>
</infinispan4>
</products>
<reports>
<report name="Replicated">
<item product="infinispan4" config="repl-sync.xml"/>
<item product="jbosscache3" config="mvcc/mvcc-repl-sync.xml"/>
</report>
<report name="Distributed">
<item product="infinispan4" config="dist-*"/>
</report>
<report name="All" includeAll="true"/>
</reports>
</bench-config>And this is what a generated charts look like:
Where can you find CBF? CBF can be found here. For a quick way of getting up to speed with it we recommend the 5 minutes tutorial.
Enjoy!
Mircea
Get it, Use it, Ask us!
We’re hard at work on new features, improvements and fixes, so watch this space for more announcements!Please, download and test the latest release.
The source code is hosted on GitHub. If you need to report a bug or request a new feature, look for a similar one on our JIRA issues tracker. If you don’t find any, create a new issue.
If you have questions, are experiencing a bug or want advice on using Infinispan, you can use GitHub discussions. We will do our best to answer you as soon as we can.
The Infinispan community uses Zulip for real-time communications. Join us using either a web-browser or a dedicated application on the Infinispan chat.
