CLI design rationale
We chose a CLI-first approach because it keeps the interface simple, cross-platform, and easy to maintain while still enabling deep integration via the API.
Here we'd like to run through some of the design thinking that has gone into Hypermass. In particular the reasoning technical choices and how we got to where we are.
CLI, API or Library?
We've spent a lot of time at the whiteboard on this one. The short answer here is that API was always going to be necessary but somewhat hard to use, so we need a gentle "on-ramp" in terms of getting connected.
Why no Library (yet)
Software libraries are a common means of integrating to a service like Hypermass - but the more we thought about it the more we realised it was a poor fit for our specific application. We have two issues with building a library at this phase;
Dev effort
If we went the library route, we’d want everyone to be able to talk to Hypermass. However, even targeting the five most popular backend languages would only cover about 80% of developers. Even supporting this handful of major languages would mean maintaining multiple complex codebases — a heavy burden for our current team size. As usual, it's not just the development time, but the mix of keeping up with dependencies, documentation, feature requests, bug reports, fixes and consistency issues. Coupled with a collection of community managed libraries with their own needs, the support effort on this alone would disproportionately occupy our small dev team and ultimately stall other progress.
In short, the effort would swamp our small dev team taking engineering time from more impactful improvements.
Usability
Consider what a software client should ideally do; it needs to retain metadata, retry state, subscriptions lists, handle credentials and so on. A good library would expose these out to the user - there's a tension between simplicity and functionality that can be hard to get right. Getting this right 5 or more times, consistent between each implementation, again it's a big ask for a small team.
Files in a stream can be tiny or huge, but they all need to be available on a consistent interface. So the interface would have to accommodate small files easily processable in memory and large files best handled with stream processing — the lowest common denominator being the latter, which is also trickier to handle for the implementer.
Complexity, usability and consistency are trade-offs. Ultimately a library approach is not in-keeping with the "simple interface + strong decoupling" principle that we're aiming for.
CLI Client
So the solution we arrived at was; build a single program that users can run to subscribe and publish. The program obviously needs to be open source and should handle all the complexity behind the scenes.
Here's our requirements to ourselves for such a program;
- It must work basically everywhere (binaries for many platforms)
- It should be a small binary with a small memory- and disk- footprint.
- It should also be functional on developer boxes and servers, working equally well in both contexts.
- It must be simple to use
- It must support both simple and advanced users - config and options, but sensible defaults everywhere
Our solution to these design constraints was the current Hypermass CLI;
- a CLI tool
- written in Go
- extensive platform support
- excellent asynchronous support
- standalone (no extra steps to install JVM/NPM/etc)
- with a "config" file for subscription data
- "plain file" local storage method
- hot folder semantics
- "Sync" (pub/sub) mode and "command" mode for interaction
API
We of course want to support a lively community of software projects and integrations, so the API was always a "must". Coupled with the CLI we feel this strikes the right balance with constraints and accessibility for this stage of the platform - we'll of course keep an eye on how this evolves.
As the platform stabilises, we may revisit the idea of official language SDKs — ideally guided by real-world usage patterns and community feedback.