Subscription Listener Implementation Guide

If you are building a client in a language not covered by our specific examples, follow this standard logic flow. This "Scan-and-Process" pattern ensures data integrity and sequential processing.

info

Please note that we assume that the payloads published by most users are sequential. If that's not the case (or your logic simply doesn't need the payloads to be loaded in sequence) you can ignore the sorting and sequential processing steps in this guide and process any and all payloads you find.

The Core Algorithm

Regardless of your technology stack, your client should follow these five steps - ideally in a continuous loop with a wait. For efficiency and low latency, you may want to also listen for filesystem changes, but this comes with additional complexity in ensuring you process the elements in the right order.

1. Discovery & Filtering

List all contents of your configured subscription-target directory.

• Payloads: Every payload's name starts with p_, so you can use that to pick out the files and folders to process.
• Specifically: Always ignore the .hypermass directory, as this contains critical sync metadata.

2. Sequential Sorting

Our payload IDs are monotonic and lexicographically sortable. We use the payload ID as the file name, with the correct file extension for the type (or as the folder name, without an extension, for folder-with-metadata).

That means that if you see 5 payloads in the target directory, you can use an ascending text sort and that will give you the payloads in publication order (item one is the oldest, item two the second oldest, etc).

3. Processing each file

Take the first file in the list and execute your business logic (e.g., parsing JSON, updating a database, or triggering an alert).

4. Cleanup (The "Commit")

Once your processing logic has succeeded (specifically a "success"), delete the file or move it to an "archive" directory. This prevents you accidentally processing it twice or more.

5. Error Handling (The "Halt")

If Step 3 fails, do not delete the file and do not move to the next file.

• Stop the loop and log an error. This preserves the sequence and ensures no data is lost or processed out of order. Once you fix the issue and restart your client, it will naturally pick up exactly where it left off.

Implementation Checklist

Feature	Requirement
Hidden Files	Must be ignored (especially .hypermass).
Sorting	Must be by file name to ensure sequence.
Concurrency	Single-threaded processing is recommended to maintain sequence.
Atomic Moves	Trust the file; the CLI ensures files are complete, and match their hash, before they appear.

Why no "File Locking"?

You may notice we don't mention checking if a file is "in use." This is because the Hypermass CLI uses Atomic Renames. It writes the data to a temporary hidden folder and then "moves" it into your subscription folder in a single OS operation. By the time your code sees the file, it is 100% complete.