Nisshi Diskless Streaming for Apache Kafka® is picking up speed: what’s new in 0.7 and 0.8
Nisshi is an open-source streaming broker that is Apache Kafka® API compatible, so the Kafka clients, tools and applications you already use can connect to it unchanged. The difference is where your data lives. Classic Kafka keeps data on disks attached to its brokers; Nisshi is “diskless,” which means the brokers hold no data of their own and write it to shared storage you choose, such as S3, PostgreSQL or SQLite. It ships as a single binary, so you can start it, point a Kafka client at it and be producing and consuming in a few minutes.
Why go diskless? Kafka’s disk-based design means paying for replicated disks, cross-AZ traffic and unused headroom, then paying again for the people who tune and rebalance clusters. Take the disks and stateful brokers out of the picture and there is far less infrastructure to run, capacity can follow demand instead of peak load, and your data sits in storage that is ready for analytics (Apache Iceberg and Delta Lake are built in). The trade-off is a small amount of added latency, which rarely matters for logs, telemetry, retention and replay.
Not sure where to start? Check out all the helpful resources in the left panel. If you are ready to try it or contribute, “All releases on GitHub” has the downloads and source, and the release notes below show exactly what changed in these latest versions.
Mark Spielman, SVP of Product, talks with Nisshi creator Peter Morgan about where the project is going.
Picking up speed: what’s new in 0.7 and 0.8
Since Solace acquired the assets of Nisshi.io, the open-source project has moved quickly. Version 0.7 landed on 16 October. Version 0.8 followed on 27 October, with two quick updates (0.8.1 and 0.8.2) in November. That is four releases in under three weeks, all pointed the same way: a Kafka API compatible engine that is easier to run and easier to trust. This accelerated development is made possible because its founder has joined Solace and leads the team building on the project and we remain excited to continue its evolution.
Nisshi is Apache 2.0 and stays that way. It suits the workloads that belong on object storage, such as logs, telemetry, retention and replay. For those that are looking for a significantly more cost-effective way to integrate their Kafka clients than running full Kafka brokers, Nissi provides that solution. It is a solution that complements Solace Event Mesh by providing and easier alternative to running full Kafka clusters and a simple integration to Solace Event Broker and mesh.
Nisshi 0.7: a new name and a wider foundation
- Now called Nisshi. The project formerly known as Tansu has a new name throughout.
- Google Cloud Storage joins S3. Run on the object store you already use.
- Fewer, larger writes. Produce batching groups writes before they reach object storage.
- Closer to Kafka. Group rebalancing follows Kafka 3.9.1 timing, and request validation is stricter.
Nisshi 0.8: easier to run and operate
- Retention that behaves as Kafka teams expect, including keeping everything with retention.ms=-1.
- Kafka-style topic settings. New nisshi.* keys, with the old tansu.* keys still working as aliases.
- Clearer answers for clients and operators. More accurate offsets and group states, and better error codes.
- More control. Choose whether topics are auto-created and how many partitions they get (0.8.2), and a configurable PostgreSQL pool.
Before you upgrade
• Retention now applies to PostgreSQL and SQLite topics that have no cleanup policy, so older records can be deleted. Set retention to keep them before you upgrade.
• 0.7 changes the PostgreSQL, SQLite and Turso schema with no automatic migration.
• Transactional producers are not supported in 0.8.
The upgrade notes in the changelog walk through each step.