guide_build

tit.scene.guide_build

Generate a packaged guide scene (tit/scene/guide*/) from one head model.

This is a developer tool, not part of the running app: it needs SimNIBS, nibabel and scipy, it reads a 184 MB mesh, and it takes minutes. The server only ever reads what it wrote (:mod:tit.scene.guide).

Run it inside the toolbox container, where SimNIBS lives::

docker exec ti-toolbox-fad740e5-tit-1 \
    /root/SimNIBS-4.6/bin/simnibs_python -m tit.scene.guide_build \
    --project <bids> --subject ernie --out /ti-toolbox/tit/scene/guide

The head needs surfaces/{lh,rh}.central.gii, its cortical .annot files for a subject guide, and segmentation/labeling.nii.gz; each guide's PROVENANCE.md says where they came from.

What it does, and why in this order:

1. Builds the two surfaces and every cortical atlas' labels through the ordinary :mod:tit.scene.build pipeline, into the project's own scene cache. The guide is therefore built by the same code that builds a subject's pane, so a budget or winding fix cannot apply to one and not the other.
2. Copies those exact cached bytes into the package, renamed to fingerprint-free, stable file names — the packaged guide is immutable, so the fingerprint that keys a cache has no job here.
3. Reads every EEG net as JSON (a few hundred rows each; cheap and exact).
4. Writes manifest.json enumerating all of it with a sha256 and a byte count per file, so the gate test can prove that what the manifest advertises is what the package contains.

Never copied: the m2m_ directory, the head mesh, the label volume. Nothing here publishes anything: the guides are tracked files in this repository.

build_label_volume

build_label_volume(source: Path, lut: Path, out: Path, *, part_id: str, atlas_id: str, select_all: bool = False) -> tuple[dict, dict]

Freeze one label volume as a pickable surface plus its legend.

Only the developer build reads a label volume. Runtime requests reuse the same compact binary surface and legend API as cortical atlas requests.

select_all keeps every label the volume contains rather than the subcortical subset :func:tit.scene.volume_surfaces.default_visible would show. It is what a packaged MNI atlas wants: its whole point is that the user picks any of its regions, and the "is this name a deep structure" heuristic exists for a per-subject labeling.nii.gz, not for CIT168.

Source code in tit/scene/guide_build.py

def build_label_volume(
    source: Path,
    lut: Path,
    out: Path,
    *,
    part_id: str,
    atlas_id: str,
    select_all: bool = False,
) -> tuple[dict, dict]:
    """Freeze one label volume as a pickable surface plus its legend.

    Only the developer build reads a label volume.  Runtime requests reuse the
    same compact binary surface and legend API as cortical atlas requests.

    *select_all* keeps every label the volume contains rather than the
    subcortical subset :func:`tit.scene.volume_surfaces.default_visible` would
    show.  It is what a packaged MNI atlas wants: its whole point is that the
    user picks any of its regions, and the "is this name a deep structure"
    heuristic exists for a per-subject ``labeling.nii.gz``, not for CIT168.
    """
    import nibabel as nib
    import numpy as np

    from tit.scene import gifti
    from tit.scene.volume_surfaces import surfaces

    rows = build.parse_lut_text(lut.read_text())
    names = {row["id"]: row["name"] for row in rows}
    image = nib.load(str(source))
    selected: list[int] = []
    if select_all:
        data = image.get_fdata(dtype=np.float32)
        selected = [int(v) for v in np.unique(data) if v != 0]
    result = surfaces(image, names, selected, atlas_name=source.name)
    positions = np.asarray(result["positions"], dtype=np.float32).reshape(-1, 3)
    indices = np.asarray(result["indices"], dtype=np.uint32).reshape(-1, 3)
    labels = np.asarray(result["labels"], dtype=np.uint16)
    if not len(indices):
        raise ValueError(f"{source.name} has no regions to draw")
    kept = {entry["id"] for entry in result["entries"]}
    legend = [
        {**row, "label": row["id"], "hemi": "", "color": row["color"] or "#808080"}
        for row in rows
        if row["id"] in kept
    ]
    surface_blob = tvsc.encode(positions, indices)
    if len(surface_blob) > build.MAX_BYTES or len(indices) > build.MAX_TRIANGLES:
        raise ValueError(f"{source.name} surface exceeds the guide budget")

    def write_blobs(directory: str, name: str, blobs: dict) -> dict:
        files = {}
        for fmt, blob in blobs.items():
            rel = f"{directory}/{name}.{fmt}"
            path = out / rel
            path.parent.mkdir(parents=True, exist_ok=True)
            path.write_bytes(blob)
            files[fmt] = rel
            files[f"{fmt}_meta"] = {"bytes": len(blob), "sha256": sha256_of(path)}
        return files

    surface_files = write_blobs(
        "surfaces",
        part_id,
        {
            "tvsc": surface_blob,
            "gii": gifti.encode_surface(positions, indices),
        },
    )
    label_files = write_blobs(
        "labels",
        atlas_id,
        {
            "tvsc": tvsc.encode(positions, None, labels),
            "gii": gifti.encode_surface(
                positions, indices, labels, gifti.label_table_from_legend(legend)
            ),
        },
    )
    bbox = positions.min(axis=0).tolist() + positions.max(axis=0).tolist()
    part = {
        "id": part_id,
        "kind": "surface",
        "triangles": len(indices),
        "vertices": len(positions),
        "bytes": len(surface_blob),
        "fingerprint": f"guide-{guide.GUIDE_VERSION}-{part_id}",
        "url": f"/api/guide/surface?part={part_id}",
        "simplified": True,
        "within_budget": True,
        "bbox": bbox,
        "focus_bbox": bbox,
        "files": surface_files,
    }
    legend_rel = f"legends/{atlas_id}.json"
    legend_meta = _write_json(
        out / legend_rel,
        {
            "atlas": atlas_id,
            "space": guide.GUIDE_SPACE,
            "aligned_to": part_id,
            "vertices": len(positions),
            "radius_mm": 0,
            "labelled_fraction": 1,
            "legend": legend,
            "url": f"/api/guide/labels?atlas={atlas_id}",
        },
    )
    atlas = {
        "id": atlas_id,
        "kind": "subcortical",
        "hemispheres": [],
        "aligned_to": part_id,
        "regions": len(legend),
        "url": f"/api/guide/regions?atlas={atlas_id}",
        "files": label_files,
        "legend_file": legend_rel,
        "legend_meta": legend_meta,
        "source_sha256": sha256_of(source),
        "lut_sha256": sha256_of(lut),
    }
    return part, atlas

paint_on_cortex

paint_on_cortex(source: Path, lut: Path, out: Path, *, atlas_id: str, cortex: Any) -> dict

One cortical label volume as per-vertex labels on the whole central surfaces.

The subject path's transfer (:func:tit.scene.build.labels_from_nearest, then the mode filter) with labelled voxels in place of annotated vertices. A vertex looks only on its own side of the midline: the medial wall lies within the radius of the other hemisphere.

Source code in tit/scene/guide_build.py

def paint_on_cortex(
    source: Path, lut: Path, out: Path, *, atlas_id: str, cortex: Any
) -> dict:
    """One cortical label volume as per-vertex labels on the whole central surfaces.

    The subject path's transfer (:func:`tit.scene.build.labels_from_nearest`, then the mode
    filter) with labelled voxels in place of annotated vertices. A vertex looks only on its own
    side of the midline: the medial wall lies within the radius of the other hemisphere.
    """
    import nibabel as nib
    import numpy as np
    from scipy.spatial import cKDTree

    image = nib.load(str(source))
    values = np.asarray(image.dataobj).astype(np.int64)
    voxels = np.argwhere(values != 0)
    voxel_labels = values[tuple(voxels.T)]
    world = nib.affines.apply_affine(image.affine, voxels)
    points = np.asarray(cortex.positions, dtype=np.float64)
    raw = np.zeros(len(points), dtype=np.uint16)
    for voxel_side, vertex_side in (
        (world[:, 0] < 0, points[:, 0] < 0),
        (world[:, 0] >= 0, points[:, 0] >= 0),
    ):
        distance, index = cKDTree(world[voxel_side]).query(
            points[vertex_side], workers=-1
        )
        raw[vertex_side] = build.labels_from_nearest(
            index, distance, voxel_labels[voxel_side], build.LABEL_RADIUS_MM
        )
    labels = build.one_ring_mode_filter(raw, cortex.indices)

    present = set(np.unique(labels).tolist()) - {build.NO_REGION}
    legend = [
        {**row, "label": row["id"], "hemi": "", "color": row["color"] or "#808080"}
        for row in build.parse_lut_text(lut.read_text())
        if row["id"] in present
    ]
    rel = f"labels/{atlas_id}.tvsc"
    blob = tvsc.encode_labels(cortex.positions, labels)
    (out / rel).parent.mkdir(parents=True, exist_ok=True)
    (out / rel).write_bytes(blob)
    legend_rel = f"legends/{atlas_id}.json"
    legend_meta = _write_json(
        out / legend_rel,
        {
            "atlas": atlas_id,
            "space": guide.GUIDE_SPACE,
            "aligned_to": "gm",
            "vertices": len(labels),
            "radius_mm": build.LABEL_RADIUS_MM,
            "labelled_fraction": round(float((labels != build.NO_REGION).mean()), 4),
            "legend": legend,
            "url": f"/api/guide/labels?atlas={atlas_id}",
        },
    )
    return {
        "id": atlas_id,
        # The ROI route, not the shape: the row still targets this label volume.
        "kind": "subcortical",
        "hemispheres": [],
        "aligned_to": "gm",
        "regions": len(legend),
        "url": f"/api/guide/regions?atlas={atlas_id}",
        "files": {
            "tvsc": rel,
            "tvsc_meta": {"bytes": len(blob), "sha256": sha256_of(out / rel)},
        },
        "legend_file": legend_rel,
        "legend_meta": legend_meta,
        "source_sha256": sha256_of(source),
        "lut_sha256": sha256_of(lut),
    }

build_subcortical

build_subcortical(pm: Any, subject: str, out: Path) -> tuple[dict, dict]

The subject's own labeling.nii.gz as the guide's subcortical layer.

Source code in tit/scene/guide_build.py

def build_subcortical(pm: Any, subject: str, out: Path) -> tuple[dict, dict]:
    """The subject's own ``labeling.nii.gz`` as the guide's subcortical layer."""
    source = Path(pm.m2m(subject)) / "segmentation" / "labeling.nii.gz"
    return build_label_volume(
        source,
        source.with_name("labeling_LUT.txt"),
        out,
        part_id="subcortical",
        atlas_id=source.name,
    )

build_mni_atlases

build_mni_atlases(out: Path, cortex: Any) -> tuple[list[dict], list[dict]]

Every packaged MNI atlas: cortical ones on cortex, the rest as label surfaces.

Skipped with a printed reason rather than a failure: a volume with no colour table, or too large for its own surface, is not a reason to have no MNI guide at all. What it costs is that that atlas cannot be picked in the pane; it is still selectable in the ROI picker's list, which reads the catalog, not the guide.

Source code in tit/scene/guide_build.py

def build_mni_atlases(out: Path, cortex: Any) -> tuple[list[dict], list[dict]]:
    """Every packaged MNI atlas: cortical ones on *cortex*, the rest as label surfaces.

    Skipped with a printed reason rather than a failure: a volume with no colour
    table, or too large for its own surface, is not a reason to have no MNI guide
    at all.  What it costs is that that atlas cannot be *picked in the pane*; it is
    still selectable in the ROI picker's list, which reads the catalog, not the guide.
    """
    from tit.atlas.constants import MNI_ATLAS_FILES, mni_resources_dir
    from tit.opt.roi_spec import _find_volume_lut

    root = Path(mni_resources_dir())
    parts: list[dict] = []
    atlases: list[dict] = []
    first = {name: i for i, name in enumerate(MNI_CORTICAL_VOLUMES)}
    for name in sorted(MNI_ATLAS_FILES, key=lambda n: first.get(n, len(first))):
        source = root / name
        if not source.is_file():
            print(f"skip {name}: not packaged in {root}")
            continue
        lut = _find_volume_lut(str(source), "mni")
        if lut is None:
            print(f"skip {name}: no colour table beside it")
            continue
        if name in MNI_CORTICAL_VOLUMES:
            atlases.append(
                paint_on_cortex(source, Path(lut), out, atlas_id=name, cortex=cortex)
            )
            continue
        try:
            part, atlas = build_label_volume(
                source,
                Path(lut),
                out,
                part_id=f"atlas-{name}",
                atlas_id=name,
                select_all=True,
            )
        except ValueError as exc:
            print(f"skip {name}: {exc}")
            continue
        parts.append(part)
        atlases.append(atlas)
    return parts, atlases

generate

generate(project: str, subject: str, out: Path, label: str, *, atlas_source: str = 'subject', guide_id: str = 'default') -> dict[str, Any]

Build every guide asset and write out/manifest.json. Returns it.

atlas_source is "subject" for the anatomy guide (the subject's own cortical parcellations and labeling.nii.gz) or "mni" for the MNI152 guide, whose atlases are the packaged MNI volumes in resources/atlas/.

Source code in tit/scene/guide_build.py

def generate(
    project: str,
    subject: str,
    out: Path,
    label: str,
    *,
    atlas_source: str = "subject",
    guide_id: str = "default",
) -> dict[str, Any]:
    """Build every guide asset and write ``out/manifest.json``. Returns it.

    *atlas_source* is ``"subject"`` for the anatomy guide (the subject's own
    cortical parcellations and ``labeling.nii.gz``) or ``"mni"`` for the MNI152
    guide, whose atlases are the packaged MNI volumes in ``resources/atlas/``.
    """
    from tit import catalog
    from tit.paths import get_path_manager

    pm = get_path_manager(project_dir=project)
    started = time.perf_counter()

    surface_metas = build.build_surfaces(pm, subject, gii=True)
    if surface_metas["gm"]["source"] != "central surfaces":
        raise SystemExit(f"{subject} has no central surfaces for the guide's cortex")
    surface_fp = build.surface_fingerprint(pm, subject)

    out.mkdir(parents=True, exist_ok=True)
    parts: list[dict[str, Any]] = []
    for part in build.PART_TAGS:
        meta = surface_metas[part]
        files: dict[str, Any] = {}
        for fmt in CORTEX_FORMATS if part == "gm" else SURFACE_FORMATS:
            found = cache.find_cached(pm.project_dir, subject, part, surface_fp, fmt)
            if found is None:
                raise SystemExit(f"{part}.{fmt} was not published by build_surfaces")
            rel = f"surfaces/{part}.{fmt}"
            files[fmt] = rel
            files[f"{fmt}_meta"] = _copy(found.path, out / rel)
        parts.append(
            {
                "id": part,
                "kind": "surface",
                "triangles": meta["triangles"],
                "vertices": meta["vertices"],
                # `bytes` is the TVSC1 payload's size, as in the scene
                # manifest, so §S3's ≤3 MB budget reads off the same number.
                "bytes": files["tvsc_meta"]["bytes"],
                "fingerprint": f"guide-{guide.GUIDE_VERSION}-{part}",
                "url": f"/api/guide/surface?part={part}",
                "simplified": meta["simplified"],
                "within_budget": meta["within_budget"],
                "max_deviation_mm": meta["max_deviation_mm"],
                "bbox": meta["bbox"],
                "focus_bbox": meta["focus_bbox"],
                "files": files,
            }
        )

    atlases: list[dict[str, Any]] = []
    for entry in (
        []
        if atlas_source == "mni"
        else (catalog.atlases(pm, subject, kind="cortical") or [])
    ):
        atlas_id = str(entry["id"])
        hemispheres = sorted(build.annot_paths(pm, subject, atlas_id))
        if not hemispheres:
            continue
        meta = build.build_labels(pm, subject, atlas_id)
        fp = build.labels_fingerprint(pm, subject, atlas_id)
        key = build._labels_key(atlas_id)
        files = {}
        for fmt in CORTEX_FORMATS:
            found = cache.find_cached(pm.project_dir, subject, key, fp, fmt)
            if found is None:
                raise SystemExit(f"{key}.{fmt} was not published by build_labels")
            rel = f"labels/{atlas_id}.{fmt}"
            files[fmt] = rel
            files[f"{fmt}_meta"] = _copy(found.path, out / rel)
        legend_rel = f"legends/{atlas_id}.json"
        legend_body = {
            "atlas": atlas_id,
            "space": guide.GUIDE_SPACE,
            "aligned_to": meta.get("aligned_to", "gm"),
            "vertices": meta["vertices"],
            "radius_mm": meta["radius_mm"],
            "labelled_fraction": meta["labelled_fraction"],
            "legend": meta["legend"],
            "url": f"/api/guide/labels?atlas={atlas_id}",
        }
        legend_meta = _write_json(out / legend_rel, legend_body)
        atlases.append(
            {
                "id": atlas_id,
                "hemispheres": hemispheres,
                "regions": len(meta["legend"]),
                "url": f"/api/guide/regions?atlas={atlas_id}",
                "files": files,
                "legend_file": legend_rel,
                "legend_meta": legend_meta,
            }
        )

    if atlas_source == "mni":
        cortex = tvsc.decode((out / "surfaces" / "gm.tvsc").read_bytes())
        mni_parts, mni_atlases = build_mni_atlases(out, cortex)
        if not mni_atlases:
            raise SystemExit("No packaged MNI atlas could be built into the guide")
        parts.extend(mni_parts)
        atlases.extend(mni_atlases)
    else:
        subcortical, atlas = build_subcortical(pm, subject, out)
        parts.append(subcortical)
        atlases.append(atlas)

    nets: list[dict[str, Any]] = []
    for net_name in sorted(pm.list_eeg_caps(subject)):
        body = build.read_net(pm, subject, net_name)
        electrodes = body.get("electrodes", [])
        if not electrodes:
            continue
        rel = f"nets/{net_name}.json"
        net_body = {
            "net": net_name,
            "space": guide.GUIDE_SPACE,
            "electrodes": electrodes,
        }
        file_meta = _write_json(out / rel, net_body)
        nets.append(
            {
                "name": net_name,
                "electrodes": len(electrodes),
                "url": f"/api/guide/electrodes?net={net_name}",
                "file": rel,
                **file_meta,
            }
        )

    manifest = {
        "guide": {"id": subject, "label": label},
        "guide_version": guide.GUIDE_VERSION,
        "builder_version": build.BUILDER_VERSION,
        # Never "subject-ras": these millimetres belong to the guide head, and
        # writing them into a research subject's config would be wrong in a
        # way nothing downstream can detect (R4).
        "space": guide.GUIDE_SPACE,
        "bbox": _union([p["bbox"] for p in parts]),
        "focus_bbox": _union([p["focus_bbox"] for p in parts]),
        "parts": parts,
        "nets": nets,
        "atlases": atlases,
        # The label volume is deliberately not packaged: it is a ~10 MB NIfTI
        # whose only use in a pane is region picking, which the atlas payloads
        # already do on the surface the pane draws.
        "volumes": [],
        "cache": {"state": "ready", "built_ms": 0.0},
        "provenance": {
            "source": f"SimNIBS example dataset, subject {subject!r}",
            "source_url": "https://github.com/simnibs/example-dataset",
            "license": "GPL-3.0-or-later",
            "notes": "See tit/scene/guide/PROVENANCE.md.",
        },
        "generated": {
            "subject": subject,
            "generator": "tit.scene.guide_build",
            "seconds": round(time.perf_counter() - started, 1),
        },
    }
    # The legends were written before the manifest, so their own urls are
    # re-tagged on disk rather than only in memory.
    for legend_path in sorted((out / "legends").glob("*.json")):
        body = json.loads(legend_path.read_text(encoding="utf-8"))
        _write_json(legend_path, _tag_urls(body, guide_id))
    for atlas in manifest["atlases"]:
        atlas["legend_meta"] = {
            "bytes": (out / atlas["legend_file"]).stat().st_size,
            "sha256": sha256_of(out / atlas["legend_file"]),
        }
    _tag_urls(manifest, guide_id)
    _write_json(out / guide.MANIFEST_NAME, manifest)
    return manifest