Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
157 changes: 137 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,15 +37,17 @@ EEG samples and the stimulus markers must share a **common clock domain** so tha
| Language | C++20 |
| Build system | CMake (≥ 3.25), Ninja-friendly |
| Experiment file format | Protocol Buffers (proto3) — see `protoFiles/neuronide.proto` |
| Device config format | JSON (`config.json`) parsed with [nlohmann/json](https://github.com/nlohmann/json) |
| EEG acquisition | [LSL — Lab Streaming Layer](https://github.com/sccn/liblsl) (`liblsl`) |
| Rendering / windowing | SDL2 (+ SDL2_image), with vsync |
| Inter-thread queues | [moodycamel ConcurrentQueue](https://github.com/cameron314/concurrentqueue) (lock-free) |
| Python scripting (planned)| pybind11 (`ScriptComponent`) |
| Testing | GoogleTest + CTest |
| Tooling | clang-format, clang-tidy, gcovr (coverage) |

`liblsl`, `concurrentqueue`, and `googletest` are fetched automatically by CMake
(`FetchContent`). SDL2 and Protobuf are expected to be installed on the system.
`liblsl`, `concurrentqueue`, `nlohmann/json`, and `googletest` are fetched
automatically by CMake (`FetchContent`). SDL2 and Protobuf are expected to be
installed on the system.

## 3. Architecture

Expand Down Expand Up @@ -86,7 +88,9 @@ flowchart LR
`lsl::local_clock()` right after `SDL_RenderPresent` and pushes `Marker`s onto
`markerQueue`.
- **LSLReader** resolves and subscribes to the EEG LSL stream and continuously
pushes `EEGData` samples onto `eegQueue`.
pushes `EEGData` samples onto `eegQueue`. It forwards **only the channels the
device config enables** (see [§5 Configuration boundary](#5-configuration-boundary-protobuf-vs-json)),
in config declaration order.
- **DataWriter** drains both queues and writes them to disk via a pluggable
formatting strategy (currently CSV).

Expand Down Expand Up @@ -141,7 +145,7 @@ the parser does not need to know about concrete component classes.
```cpp
struct EEGData { // one EEG sample
double timestamp; // in the local_clock() domain (see §4)
std::vector<double> channels;
std::vector<double> channels; // enabled channels only, in config order (see §5)
};

struct Marker { // one experiment event
Expand Down Expand Up @@ -169,7 +173,116 @@ are already mapped into the local `lsl::local_clock()` domain — the same clock
Renderer uses. This is the single most important correctness property of the data
path.

## 5. Thread lifecycle conventions
## 5. Configuration boundary: protobuf vs JSON

The runtime is fed by **two** config inputs and they are not interchangeable.
The split is deliberate and should be respected when adding new settings,
otherwise the same experiment stops being portable between labs.

> **The rule:** the **protobuf experiment file** describes *what the experiment
> does*; the **device `config.json`** describes *what the hardware is*.
> A new field belongs in protobuf if changing it changes the experiment's meaning
> for analysis, and in JSON if it only changes how this particular machine
> acquires or stores the data.

| Goes in the protobuf experiment file (`.neuroz`) | Goes in the device config (`config.json`) |
| --------------------------------------------------------- | ------------------------------------------------------------------ |
| Experiment name, scene objects, transforms, visibility | Device name, montage standard (`10-20`, …) |
| Components and their parameters (e.g. blink frequency) | LSL stream identity: `name`, `type`, `source_id` |
| Stimulus timing, trial structure, marker/event names | Expected stream shape: channel count, sample rate |
| Anything the editor authors and versions with the study | Channel table: index, label, enabled, unit |
| | Reference / ground electrodes, impedance check thresholds |
| | *(planned)* output file format for `DataWriter` |

Consequences of the split:

- The same experiment file runs on a different cap by swapping only
`config.json` — no re-export from the editor.
- The runtime can validate the incoming LSL stream (channel count, sample rate)
**before** the experiment starts, because expectations are declared per device.
- Electrode-level knowledge (which channel is `Oz`, which are enabled) lives in
one place, so `LSLReader` and, later, `DataWriter` agree on channel order.

The JSON is parsed **1:1** into `DeviceConfig`: every JSON key maps onto exactly
one field, and nesting in the file is the nesting in the struct. Keep it that
way — `channels` is a top-level key, so it is a top-level `DeviceConfig` field
(not tucked under `lsl`), even though `LSLReader` is its main consumer.

```jsonc
{
"config_version": "1.0", // "MAJOR.MINOR", checked first (see below)
"device_name": "OpenBCI Cyton 8ch",
"montage_standard": "10-20",
"lsl_stream": { // -> DeviceConfig::lsl (LSLConfig)
"name": "obci_eeg1",
"type": "EEG",
"source_id": "cyton-a1b2c3",
"expected_channel_count": 8, // must equal channels.size()
"expected_sample_rate_hz": 250
},
"reference": { "label": "linked_mastoids", "scheme": "physical" },
"ground": { "label": "Fpz" },
"channels": [ // -> DeviceConfig::channels
{ "index": 0, "label": "Fz", "enabled": true, "unit": "microvolts" },
{ "index": 1, "label": "Oz", "enabled": false, "unit": "microvolts" }
// ... one entry per expected_channel_count, indices unique and in range
],
"impedance_check": { "supported": true, "threshold_kohm": 5.0 }
}
```

`config_version`, `device_name`, `montage_standard`, `lsl_stream` and `channels`
are required; `reference`, `ground` and `impedance_check` default when absent.
Channels with `"enabled": false` stay in the config (they document the cap) but
are **not** acquired: `LSLReader` drops them from every sample.

### Validation contract

Mapping and validation are separate jobs. `ConfigParser` only turns JSON into
structs — presence of keys, types, array shape. Every *semantic* rule (non-empty
stream identity, positive rate, channel count matching the stream, unique
in-range indices) lives on the config types themselves as `validate()`, because
none of those rules are about JSON and they must hold for any producer:

```cpp
DeviceConfig config = ConfigParser::parse("config.json"); // already validated
```
```cpp
DeviceConfig config; // hand-assembled: no guarantees
config.lsl.name = ...;
config.validate(); // throws std::invalid_argument on the first broken rule
```

**A `DeviceConfig` returned by `ConfigParser` has passed `validate()`. One you
assemble yourself has not** — call it before handing the config to a consumer.
`LSLReader` validates in its constructor rather than trusting its caller, since
it indexes into raw samples with the configured channel offsets.

Rules that need more context than the config carries stay with the consumer, not
in `validate()`: "at least one channel is enabled" is an `LSLReader` precondition
(a fully disabled cap is a valid *config*, just nothing to acquire), and
"stream shape matches the live LSL stream" can only be checked against a resolved
stream at runtime.

### Schema versioning

`config_version` is `"MAJOR.MINOR"` and is the **first** thing `ConfigParser`
validates — on an unsupported schema every later complaint would be a misleading
"missing field" message instead of "your config is newer than this runtime".

- **MAJOR** — breaking change: a field moved, was renamed, or changed meaning.
A runtime rejects any major other than `ConfigParser::kSupportedConfigMajor`.
- **MINOR** — additive, backward-compatible change: new optional keys. Any minor
of the supported major is accepted, and unknown keys are ignored, so a `1.7`
file still runs on a runtime that only knows `1.0`.
- A version that cannot be compared (`"1"`, `"v1"`, `"1.2.3"`, empty) is rejected
rather than assumed — an unparseable version is worse than none.

Bump MINOR when adding optional keys, MAJOR when moving or renaming any existing
one, and raise `kSupportedConfigMajor` in the same commit that lands the breaking
parser change.

## 6. Thread lifecycle conventions

Threads use C++20 `std::jthread` + `std::stop_token` for cooperative cancellation.
Two ownership patterns are in use:
Expand All @@ -186,9 +299,14 @@ Two ownership patterns are in use:
`start()` returns immediately instead of blocking the runtime while waiting for the
cap), uses a **blocking pull with a finite timeout** (no busy-wait, low latency,
periodic stop-token checks), and catches `lsl::lost_error` to re-resolve a dropped
stream rather than letting an exception terminate the process.
stream rather than letting an exception terminate the process. Recovery is
deliberately *ours*: the inlet is created with liblsl's `recover` flag **off**, so
a lost cap surfaces as `lsl::lost_error` instead of being silently reconnected, and
the re-resolved stream is re-validated (channel count, sample rate) before
acquisition continues. Config errors (mismatched stream shape, no enabled channels)
stay fatal — they are logged and the worker exits instead of retrying forever.

## 6. Implementation status
## 7. Implementation status

| Area / class | Status | Notes |
| --------------------------- | ------------- | ------------------------------------------------------------ |
Expand All @@ -198,14 +316,16 @@ stream rather than letting an exception terminate the process.
| `ComponentRegistry` | Implemented | proto-type → factory, macro-based self-registration |
| `specifiic components` | **Planned** | defined in `neuronide.proto`, not yet implemented in C++ |
| `Renderer` | Implemented | SDL + vsync, marker timestamping |
| `LSLReader` | Implemented | LSL inlet → `eegQueue`, clock-synced (see §4) |
| `LSLReader` | Implemented | LSL inlet → `eegQueue`, clock-synced (see §4); driven by `DeviceConfig`, enabled channels only, re-resolves lost streams |
| `ConfigParser` | Implemented | `config.json` → `DeviceConfig` (1:1 mapping, major-version checked, see §5), nlohmann/json |
| Config `validate()` | Implemented | semantic rules on the config types themselves, independent of JSON (see §5) |
| `DataWriter` | Implemented | strategy-based; `CSVFormatStrategy` |
| `Runtime` orchestration | **Stub** | `Runtime::start()` currently only prints; wiring of Parser + the three threads is the next integration step |
| `Runtime` orchestration | **Stub** | currently does nothing |

The class diagram in older docs is partly aspirational; the table above reflects
the actual code.

## 7. Repository layout
## 8. Repository layout

```
Neuron-IDE-runtime/ # the C++ runtime (git repo)
Expand All @@ -217,6 +337,7 @@ Neuron-IDE-runtime/ # the C++ runtime (git repo)
│ └── tests/ # .pbtxt fixtures + compiled .pb
├── include/ # public headers, mirrored by src/
│ ├── data_structures/ # EEGData, Marker, Context
│ ├── config/ # ConfigParser + DeviceConfig / LSLConfig / ChannelConfig / ConfigVersion
│ ├── parser/ # Parser
│ ├── scene/ # Scene, SceneObject, components/
│ ├── renderer/ # Renderer
Expand All @@ -232,7 +353,7 @@ Neuron-IDE-runtime/ # the C++ runtime (git repo)
Each `src/<module>/` builds a static library; `runtime_core` links them together
and the `NeuronIDE` executable links `runtime_core`.

## 8. Build, test, and tooling
## 9. Build, test, and tooling

All commands are run from the `Neuron-IDE-runtime/` directory.

Expand All @@ -249,9 +370,12 @@ sudo apt install cmake clang-format clang-tidy libsdl2-dev protobuf-compiler gco
```bash
cmake -B build
cmake --build build
./build/src/NeuronIDE # run the (currently stub) executable
./build/src/NeuronIDE config.json experiment.neuroz # parses the device config, parses scene, starts LSLReader, DataWriter, Renderer.
```

`NeuronIDE` takes the path to a device `config.json` (defaults to `config.json` in
the working directory).

### Tests

```bash
Expand Down Expand Up @@ -293,17 +417,10 @@ protoc --encode=NeuronIDE.Scene protoFiles/neuronide.proto \
< protoFiles/tests/test_scene.pbtxt > protoFiles/tests/test_scene.pb
```

## 9. Contribution conventions
## 10. Contribution conventions

- **Branches:** `<type>/<description>`, e.g. `feat/setup-project`.
- **Commits:** `<type>(optional scope): description`, e.g.
`feat(parser): create Parser class`.
- **Types:** `feat`, `fix`, `style` (clang config), `test`, `ci` (`.github`).

## 10. Roadmap (next steps)

1. Implement `Runtime` orchestration: `Parser → Scene`, then run `Renderer`,
`LSLReader`, and `DataWriter` concurrently and shut them down cleanly.
2. Implement the remaining components: `SpriteRenderer`, `TextRenderer`,
`ScriptComponent` (pybind11), each self-registering with `ComponentRegistry`.
3. Validate `LSLReader` end-to-end against a real EEG headset.
4 changes: 3 additions & 1 deletion cmake/Coverage.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,9 @@ if(NEURON_IDE_ENABLE_COVERAGE)
--exclude ".*protoFiles.*"
--exclude ".*pb.*"
--exclude ".*\\.hpp"
--fail-under-line 60
--exclude-throw-branches
--exclude-unreachable-branches
--fail-under-line 90
--print-summary
--html-details ${COVERAGE_DIR}/index.html
--xml ${COVERAGE_DIR}/coverage.xml
Expand Down
14 changes: 11 additions & 3 deletions cmake/Dependencies.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,14 @@ FetchContent_Declare(
SYSTEM
)

# 4. nlohmann/json (header-only) - device config parsing
FetchContent_Declare(
nlohmann_json
URL https://github.com/nlohmann/json/releases/download/v3.11.3/json.tar.xz
SYSTEM
)
set(JSON_BuildTests OFF CACHE INTERNAL "")

# Suppress compiler warnings from third-party targets when compiling their source files
set(BACKUP_C_FLAGS "${CMAKE_C_FLAGS}")
set(BACKUP_CXX_FLAGS "${CMAKE_CXX_FLAGS}")
Expand All @@ -38,14 +46,14 @@ elseif(CMAKE_CXX_COMPILER_ID MATCHES "MSVC")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /w")
endif()

FetchContent_MakeAvailable(googletest liblsl concurrentqueue)
FetchContent_MakeAvailable(googletest liblsl concurrentqueue nlohmann_json)

# Restore compiler flags for our own project code
set(CMAKE_C_FLAGS "${BACKUP_C_FLAGS}")
set(CMAKE_CXX_FLAGS "${BACKUP_CXX_FLAGS}")

# 4. SDL2 (System installed)
# 5. SDL2 (System installed)
find_package(SDL2 REQUIRED)

# 5. Protobuf (System installed)
# 6. Protobuf (System installed)
find_package(Protobuf REQUIRED)
19 changes: 19 additions & 0 deletions include/config/ChannelConfig.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
#ifndef CHANNELCONFIG_HPP
#define CHANNELCONFIG_HPP

#include <string>

// Description of a single EEG channel as declared in the device config file.
struct ChannelConfig {
int index = 0;
std::string label;
bool enabled = true;
std::string unit;

// Throws std::invalid_argument if this channel breaks its own invariants.
// Rules that need the stream shape (index within range, uniqueness) belong
// to DeviceConfig::validate.
void validate() const;
};

#endif // CHANNELCONFIG_HPP
20 changes: 20 additions & 0 deletions include/config/ConfigParser.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
#ifndef CONFIGPARSER_HPP
#define CONFIGPARSER_HPP

#include <config/DeviceConfig.hpp>
#include <istream>
#include <string>

class ConfigParser {
public:
// Schema major version this runtime understands. Configs declaring another
// major are rejected; any minor of this major is accepted (see README §5).
static constexpr int kSupportedConfigMajor = 1;

ConfigParser() = default;

static DeviceConfig parse(const std::string& filePath);
static DeviceConfig parseStream(std::istream& stream);
};
Comment thread
MichalSzandar marked this conversation as resolved.

#endif // CONFIGPARSER_HPP
25 changes: 25 additions & 0 deletions include/config/ConfigVersion.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#ifndef CONFIGVERSION_HPP
#define CONFIGVERSION_HPP

#include <string>

// Schema version of a device config file, written as "MAJOR.MINOR".
// MAJOR changes are breaking (fields moved, renamed or removed) and are rejected
// by a runtime built for another major; MINOR changes are additive and
// backward-compatible, so any minor of a supported major is accepted.
struct ConfigVersion {
int major = 0;
int minor = 0;

bool operator==(const ConfigVersion&) const = default;

// Throws std::invalid_argument on a version that cannot be compared.
// Whether a valid version is *supported* is ConfigParser's decision.
void validate() const;

[[nodiscard]] std::string toString() const {
return std::to_string(major) + "." + std::to_string(minor);
}
};

#endif // CONFIGVERSION_HPP
Loading
Loading