Lithops Hot Rod storage backend
Lithops Hot Rod storage backend
We are very happy to announce a Hot Rod storage backend for Lithops, an open source framework for big data analytics and embarrassingly parallel jobs, that provides an universal API for building parallel applications in the cloud. The work was done in the context of the CloudButton project, which aims to build a serverless platform for quickly deploying workloads to the cloud, but the advantages are available to everyone using Infinispan!
Background
Lithops is a Python multi-cloud distributed computing framework. It allows you to run unmodified local python code at massive scale in the main serverless computing platforms. Lithops delivers the user’s code into the cloud without requiring knowledge of how it is deployed and run. Moreover, its multicloud-agnostic architecture ensures portability across cloud providers.
Lithops has multiple storage backends and, up until now, it was able to access an Infinispan Server via its REST API. While the REST API is functional, it doesn’t offer all of the advantages that Hot Rod, our native protocol, provides, such as topology awareness and client-side intelligence to automatically locate the owner of an entry, thus avoiding extra network hops.
The new Hot Rod Python client
We used to have a Hot Rod Python client that lied in semi-abandoned state for a long time. We decided it was time to get back on track. The Hot Rod Python client repository has been completely overhauled with our brand new implementation, which supports modern Infinispan features such as authentication, cache creation, etc. This is now installable using PIP, the Python Package Installer. The following is a short example that shows how to connect to a secure Infinispan server and manipulate an entry:
from Infinispan import Infinispan
conf=Infinispan.Configuration()
conf.addServer("localhost",11222)
conf.setProtocol("2.4")
conf.setSasl("PLAIN", "node0", "writer", "somePassword")
manager=Infinispan.RemoteCacheManager(conf)
manager.start()
key=Infinispan.UCharVector()
key.push_back(56)
value=Infinispan.UCharVector()
value.push_back(8)
cache=Infinispan.RemoteCache(manager)
cache.put(key,value)
res=cache.get(key)
print (res.pop())
manager.stop()The Lithops backend
Enabling the Lithops Hot Rod storage backend is very simple, as demonstrated by the code snippet below:
from lithops import Storage
if __name__ == "__main__":
st = Storage(backend='infinispan_hotrod')
st.put_object(bucket='mybucket',
key='test.txt',
body='Hello World')
print(st.get_object(bucket='mybucket',
key='test.txt'))Get it, Use it, Ask us!
We’re hard at work on new features, improvements and fixes, so watch this space for more announcements!Please, download and test the latest release.
The source code is hosted on GitHub. If you need to report a bug or request a new feature, look for a similar one on our GitHub issues tracker. If you don’t find any, create a new issue.
If you have questions, are experiencing a bug or want advice on using Infinispan, you can use GitHub discussions. We will do our best to answer you as soon as we can.
The Infinispan community uses Zulip for real-time communications. Join us using either a web-browser or a dedicated application on the Infinispan chat.
Tristan Tarrant
Tristan has been leading the Infinispan Engineering Team at Red Hat for quite a while now, as well as being Principal Architect for Red Hat Data Grid. He's been a passionate open-source advocate and contributor for over three decades.
