Test Suite

Focus — verify parsing and NeXus conversion logic with realistic sample data.

Location — tests live in tests/ and rely on sample files under data/.

Quick Start

python -m pip install -r dev-requirements.txt
python -m pip install -r requirements.txt        # if runtime deps are missing
pytest -q                                        # run everything
pytest -q tests/test_spec_parser_data_files.py   # run a single file
pytest -q -k specparser                          # pattern filter

Common Environment Issue

  • Symptom — ImportError from shiboken6 during pytest collection.

  • Cause — an auto-loaded pytest plugin imports PySide6/shiboken6.

  • Workarounds

    • Disable plugin autoload: PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 pytest -q

    • Align/remove Qt packages: pip install -U --force-reinstall "PySide6==<ver>" "shiboken6==<ver>" or uninstall them entirely if not required.

Test File Inventory (tests/)

  • tests/test_auto_generate_nxd.py — validates dry-run auto-generation behavior for single files and directories, including summary output and no-file-creation guarantees.

  • tests/test_batteries_dta.py — validates batteries DTA/RAW parsing, generated .nxd placeholders, and resulting HDF5 group/dataset placement.

  • tests/test_batteries_ocp_temp.py — validates batteries OCP and temperature-log ingestion and mapping into /entry/experiments/....

  • tests/test_diamond_ascii_parser.py — validates Diamond B18 ASCII/NeXus detection, parsing, and NXxas object generation.

  • tests/test_direct_values.py — validates direct scalar/array literal handling from .nxd and YAML, including multidimensional values and placeholder expansion.

  • tests/test_directory_batch_dry_run.py — validates batch directory scanning behavior (recursive, grouping, globbing, summary-only) in dry-run mode.

  • tests/test_dta_pair_option.py — validates pair_dta_raw behavior for paired/non-paired DTA inputs and fallback behavior when counterparts are missing.

  • tests/test_hdf5_generator.py — validates HDF5 parser extraction of data/attributes and HDF5-to-NeXus generator external-link output.

  • tests/test_jsonld_parser.py — validates JSON-LD + XDI parsing and generated NeXus structure, including SchemaPlacer-assisted placement.

  • tests/test_mpes_support.py — validates MPES parser/generator behavior using external links and placeholder injection.

  • tests/test_schema_placer.py — validates SchemaPlacer ranking and placement decisions across base classes/app definitions with explicit overrides.

  • tests/test_spec_parser_data_files.py — validates real-data SPEC parsing for Femtospex and IKFT, including #L and #H derived variable keys.

  • tests/test_spec_per_scan_and_master.py — validates per-scan SPEC output plus master .nxs with external links.

  • tests/test_spec_template_option.py — validates generated SPEC templates (template_all_scans) preserve arrays when used for conversion.

  • tests/test_tif_generator.py — validates TIFF parser/generator integration with mocked image reading and NXdata linkage.

External Reference Test Suites

  • external_references/nexus/nexus_definitions/dev_tools/tests/ — upstream NXDL syntax/doc generation tests.

  • external_references/punx/punx/tests/ and external_references/punx/punx/validations/tests/ — upstream punx cache/validation/schema tests.

  • external_references/nomad/pynx/pynxtools-mpes/tests/ — upstream MPES reader example/compatibility tests.

  • These suites are vendored upstream tests and are not maintained as file-by-file project docs here.

Tips for Contributors

  • Prefer realistic inputs in data/ when feasible; otherwise create fixtures using tmp_path.

  • Derive assertions from file structure (for example SPEC headers) rather than hard-coded constants.

  • HDF5 checks work well with h5py.File(path, "r"); verify dtypes via simple type/shape checks.

  • Parsers often return (values, metadata). Normalise with:

    values, *_ = parsed if isinstance(parsed, tuple) else (parsed,)

  • Keep tests focused on behaviour relevant to the unit under test and avoid over-broad integrations.

testing_area .nxd Commands

  • Run from the repository root.

  • Use python3 NeXusCreator.py (or nexuscreator if installed).

# 01_minimal_literal.nxd
python3 NeXusCreator.py -n testing_area/01_minimal_literal.nxd -o testing_area/out_01.nxs

# 02_types_and_literals.nxd
python3 NeXusCreator.py -n testing_area/02_types_and_literals.nxd -o testing_area/out_02.nxs

# 03_nxdata_axes_signal.nxd
python3 NeXusCreator.py -n testing_area/03_nxdata_axes_signal.nxd -o testing_area/out_03.nxs

# 04_internal_soft_links.nxd
python3 NeXusCreator.py -n testing_area/04_internal_soft_links.nxd -o testing_area/out_04.nxs

# 05_external_links.nxd (requires out_01, out_03, out_12 to exist first)
python3 NeXusCreator.py -n testing_area/05_external_links.nxd -o testing_area/out_05.nxs

# 06_placeholder_expansion.nxd (SPEC input)
python3 NeXusCreator.py -n testing_area/06_placeholder_expansion.nxd -i data/spec/femtospex/2022-01-Schuessler-01_S1.spec -o testing_area/out_06.nxs

# 07_scan_template.nxd (SPEC multi-scan input)
python3 NeXusCreator.py -n testing_area/07_scan_template.nxd -i data/spec/femtospex/2022-01-Schuessler-01.spec -o testing_area/out_07.nxs

# 08_spec_per_scan_master.nxd (per-scan output + master file)
python3 NeXusCreator.py -n testing_area/08_spec_per_scan_master.nxd -i data/spec/femtospex/2022-01-Schuessler-01.spec -o testing_area/spec_master.nxs -f

# 09_batteries_workflow.nxd (batteries folder parser)
python3 NeXusCreator.py -n testing_area/09_batteries_workflow.nxd -i data/batteries/operando_eis -b batteries -o testing_area/out_09.nxs

# 10_pair_dta_raw_workflow.nxd (paired DTA/RAW parsing)
python3 NeXusCreator.py -n testing_area/10_pair_dta_raw_workflow.nxd -i data/batteries/operando_eis/EIS_CH_#1_#1.DTA -o testing_area/out_10.nxs --pair-dta-raw

# 11_jsonld_schema_placer_xas.nxd (XDI + JSON-LD structure)
python3 NeXusCreator.py -n testing_area/11_jsonld_schema_placer_xas.nxd -i data/cdi-ddi/Dagstuhl_example/Se_Na2SeO4_rt_01.xdi --jsonld-structure data/cdi-ddi/Dagstuhl_example/se_na2so4-testschemaorg-cdiv3.jsonLD -o testing_area/out_11.nxs

# 12_validation_focus.nxd
python3 NeXusCreator.py -n testing_area/12_validation_focus.nxd -o testing_area/out_12.nxs --validate

# 13_prompt_literals.nxd (interactive prompts)
python3 NeXusCreator.py -n testing_area/13_prompt_literals.nxd -o testing_area/out_13.nxs

# DTA_scan_template.nxd
python3 NeXusCreator.py -n testing_area/DTA_scan_template.nxd -i data/batteries/EIS_CH_#1_#3.DTA -o testing_area/out_dta_template.nxs