Plugin System

NeXusCreator automatically discovers generator and parser plugins that live under plugins/. Each plugin can participate in generation (producing .nxd/YAML definitions) and/or parsing (building variable libraries from inputs).

Built-in Plugins

  • SPEC (plugins/spec_plugin.py) — generator + parser for SPEC scans.

  • DTA/DAT (plugins/dta_plugin.py) — generator + parser for batteries workflows (RAW, non-RAW, temperature datasets). Use -b batteries to activate the folder generator.

  • HDF5/NeXus (plugins/hdf5_plugin.py) — generate definitions and variable libraries from existing NeXus/HDF5 files (--hdf5-option links or extract).

  • TIFF (plugins/tiff_plugin.py) — create detector/data placeholders and NXdata links from TIFF images.

  • PEAXIS (plugins/peaxis_plugin.py) — folder generator and parsers for SIF/XAS/DAT inputs. Use -b peaxis to activate.

  • YAML (plugins/yaml_plugin.py) — generator + parser for YAML-format data files (.yaml, .yml). Supports SchemaPlacer-guided placement when --nxdl-root is provided.

  • MPES (plugins/mpes_plugin.py) — generator + parser for MPES HDF5 inputs. Activated by -b mpes; uses libraries/mpes_utils.py to locate the HDF5 file.

  • JSON-LD (plugins/jsonld_plugin.py) — generator + parser for fixed-width/delimited text files described via a CDI/Schema.org JSON-LD document (--jsonld-structure FILE). Pairs with SchemaPlacer for NXDL-guided placement.

  • Diamond B18 XAFS (plugins/diamond_ascii_plugin.py) — detects either the ASCII exports under data/cdi-ddi/.../ascii or the matching NeXus files under .../nexus, parses both into a single library, and emits an NXxas-compliant definition referencing every parsed variable.

Parser and Generator System

In addition to the plugin system, NeXusCreator includes a parser and generator system that provides a consistent and extensible way to handle different input formats and generate NeXus definitions.

Base Classes

  • BaseParser: Defines the interface for all parsers. Each parser must implement:

    • can_parse(input_path: str) -> bool: Checks if the parser can handle the given input path.

    • parse(input_path: str) -> Dict[str, object]: Parses the input file and returns a flat library.

  • BaseGenerator: Defines the interface for all generators. Each generator must implement:

    • can_generate(input_path: str) -> bool: Checks if the generator can handle the given input path.

    • generate(input_path: str) -> dict: Generates a NeXus-definition object from the input.

Managers

  • ParserManager: Manages the discovery and registration of parsers.

    • Discovers all parsers in the parsers package.

    • Provides a method to get a parser for a specific file type.

  • GeneratorManager: Manages the discovery and registration of generators.

    • Discovers all generators in the generators package.

    • Provides a method to get a generator for a specific file type.

Discovery and Registration

The system automatically discovers and registers parsers and generators by:

  1. Importing all modules in the parsers or generators package.

  2. Finding all classes that inherit from BaseParser or BaseGenerator.

  3. Instantiating these classes and sorting them by priority.

Usage

To use the parser and generator system:

from nexuscreator.parsers import get_parser_manager
from nexuscreator.generators import get_generator_manager

# Get a parser for a specific file type
parser_manager = get_parser_manager()
parser = parser_manager.get_parser("test.dta")
if parser:
    library = parser.parse("test.dta")

# Get a generator for a specific file type
generator_manager = get_generator_manager()
generator = generator_manager.get_generator("test.dta")
if generator:
    nexus_object = generator.generate("test.dta")

Creating a New Parser or Generator

To create a new parser or generator:

  1. Create a new parser:

from nexuscreator.parsers.base import BaseParser

class MyParser(BaseParser):
    id: str = 'my-parser'
    priority: int = 10

    def can_parse(self, input_path: str) -> bool:
        return input_path.lower().endswith('.myformat')

    def parse(self, input_path: str) -> Dict[str, object]:
        # Parse the file and return a flat library
        return {"key": "value"}

  1. Create a new generator:

from nexuscreator.generators.base import BaseGenerator

class MyGenerator(BaseGenerator):
    id: str = 'my-generator'
    priority: int = 10

    def can_generate(self, input_path: str) -> bool:
        return input_path.lower().endswith('.myformat')

    def generate(self, input_path: str) -> dict:
        # Generate a NeXus-definition object from the input
        return {"entry": {"@NX_class": "NXentry"}}

Benefits

  • Consistency: All parsers and generators follow a consistent interface.

  • Extensibility: New parsers and generators can be easily added by inheriting from the base classes.

  • Discoverability: The system automatically discovers and registers parsers and generators.

  • Testability: The new system is well-tested, ensuring reliability.

How Discovery Works

  • DefinitionGeneratorPlugin implements can_generate() and generate_definition().

  • DataParserPlugin implements can_parse() and parse_to_library().

  • Modules in plugins/ are imported automatically; no explicit registration is required.

  • Plugins declare an id and optional priority to influence selection order.

  • Optional external plugins can be imported from filesystem paths via NEXUSCREATOR_PLUGIN_PATHS (path-separated list of .py files or directories).

Create Your Own Plugin

  1. Subclass DefinitionGeneratorPlugin and/or DataParserPlugin.

  2. Implement can_* narrowly (extension, beamline, folder shape, etc.).

  3. Return:

    • generator: a valid nexus_object dict

    • parser: a flat library dict compatible with placeholder injection

  4. Set priority (lower value wins) when multiple plugins could match.

Minimal parser + generator

from nexuscreator.plugins.base import DefinitionGeneratorPlugin, DataParserPlugin


class MyGenerator(DefinitionGeneratorPlugin):
    id = "my-generator"
    priority = 40

    def can_generate(self, input_path, beamline, flags):
        return input_path.lower().endswith(".abc")

    def generate_definition(self, input_path, beamline, flags):
        return {
            "entry": {
                "@NX_class": "NXentry",
                "data": {
                    "@NX_class": "NXdata",
                    "@signal": "counts",
                    "@axes": "energy",
                    "energy": {"@dtype": "NX_FLOAT64[]", "@value": "energy"},
                    "counts": {"@dtype": "NX_FLOAT64[]", "@value": "counts"},
                },
            }
        }


class MyParser(DataParserPlugin):
    id = "my-parser"
    priority = 40

    def can_parse(self, input_path, beamline, flags):
        return input_path.lower().endswith(".abc")

    def parse_to_library(self, input_path, beamline, flags):
        return {"energy": [1.0, 2.0, 3.0], "counts": [10.0, 20.0, 30.0]}

External plugin loading

You can keep plugins outside this repository and point NeXusCreator to them:

export NEXUSCREATOR_PLUGIN_PATHS="/path/to/my_plugins:/path/to/custom_plugin.py"

For plugin import diagnostics:

export NEXUSCREATOR_PLUGIN_DEBUG=1

Minimal Example

from nexuscreator.plugins.base import DefinitionGeneratorPlugin


class MyGenerator(DefinitionGeneratorPlugin):
    id = "my-generator"

    def can_generate(self, input_path, beamline, flags):
        return input_path.endswith(".abc")

    def generate_definition(self, input_path, beamline, flags):
        # Return a NeXus definition object (dict-like)
        return {"entry": {"@NX_class": "NXentry"}}

Tip: combine your plugin with SchemaPlacer to map variables into canonical NXDL paths automatically.