Building a client

Once the Hypermass CLI is syncing data to your local machine, the next step is to act on that data. Because Hypermass writes updates as standard files, you can build a client in any language without needing to implement complex networking or authentication logic yourself.

The "Filesystem Poller" Pattern

The most common way to build a client is the Filesystem Poller. Your application periodically checks the contents of a specific directory, filters and sorts the contents and processes them one by one in sequence from oldest to newest.

Please select your preferred language below for some sample code to get you started;

• Python - heavily used in data science
• C# .NET - Industry standard language for enterprise software
• Java - Industry standard language for enterprise software

Here's a general guide for implementing a subscription listener; General Implementation Guide

Pro Tip

The hypermass sync command writes each payload file to a hidden folder (here: ./.hypermass/temporary) then moves it to the subscription location. This means that the file is always "complete" by the time you read it - you don't need to check to see whether the file is still being written to.

Handling Different Writer Types

Files arrive one per payload by default. The writer-type setting on a subscription in hypermass-config.yaml can deliver a folder per payload with its metadata instead. Your client should be built to match:

Writer Type	Client Logic
file-per-payload	Your client should look for individual new files. Each file contains one update.
folder-with-metadata	Your client should look for new directories. Each directory contains the payload and a metadata.json file.

The metadata file

With folder-with-metadata, each payload's folder is named with its payload id and holds the payload file and a metadata.json:

{
  "formatVersion": 1,
  "payloadId": "p_001m4dh7t0hh8xpngcphahtex2g",
  "streamId": "_254gGE43g",
  "publishedTimestamp": "2026-10-08T14:02:11.123Z",
  "payloadFile": "payload.json",
  "bytesCount": 48213,
  "contentHashAlgorithm": "sha256",
  "contentHash": "47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=",
  "validationType": "schema"
}

• formatVersion: the version of this file's layout.
• payloadFile: the name of the payload file in the folder.
• bytesCount: the payload's size in bytes.
• contentHashAlgorithm and contentHash: the hash of the payload's bytes, base64 encoded. Both are null for payloads published before hashes were recorded.
• validationType: the checks the payload passed - basic, or schema for validation against the stream's schema.

Implementation Tips

When implementing your watcher, it's worth noting;

1. Sequence: always process files sorted by name - each is named with its payload id, and payload ids sort in publication order
2. Idempotency: each payload is written once. A payload already in the folder is left as it is, even when you replay it - deleting it is what marks it as consumed
3. Cleanup: Since Hypermass will continually writing files as they are published, ensure your client deletes files after processing or moves them to an "archive" location.
4. Error Handling: If your processing logic fails, stop processing and don't delete the original file, this gives you opportunity to fix the issue and restart the process.
5. replay: It's possible to reset the subscription to any point manually by following the replay process
6. Integrity: each payload is checked against its hash before it appears in your folder. If one doesn't match, the stream waits on it and downloads it again after 1 minute, then 10 minutes, then an hour, then once a day, so a persistent problem doesn't use up your allowance. Restarting hypermass sync retries it straight away.

Pro Tip

If you need access to payload metadata, use the folder-with-metadata writer type. This provides you with the original publication timestamp, payload Id and other publication metadata.

You will have to modify the code in here to open both the metadata file and the payload file.

Replay

It is possible to replay the history of a stream by using the replay command.