Resources and writable workspaces

The installed library contains its Python modules and typing information. Application sources, notebooks and scientific inputs are separate downloads. These utilities accept explicit directories, URLs and catalogues; they contain no built-in case inventory.

download_resource acquires a file with a declared SHA256 and byte limit. It reuses a matching local copy and preserves that copy when acquisition fails. workspace_from_archive applies the same identity check to a ZIP, validates its paths and expanded size, then extracts regular files into a writable directory. Extraction preserves existing local files and does not execute code. Importing companion sources is an explicit step in the consuming notebook.

A workspace may register an external .pymhm-resources.json, or callers can pass a mapping to register_resources. For example:

from pymhm.io.workspace import register_resources, resource_file

register_resources(
    "run",
    {
        "resources": {
            "data/coefficients.csv": {
                "url": coefficient_url,
                "sha256": coefficient_sha256,
                "size_bytes": coefficient_size_bytes,
            },
        },
    },
)
coefficients = resource_file("data/coefficients.csv", directory="run")

Registration describes inputs without downloading them. resource_file checks immutable inputs against the catalogue and acquires only the requested file. materialize_resources copies selected inputs and declared attribution files into the workspace, preserving user edits. local_resource supports existing mutable local files. Unavailable original inputs retain explicit reasons and are not replaced with different data.

source_identity hashes the actual source files used by an application. A logical src/pymhm/... path identifies the installed library's corresponding module; application sources resolve in the declared workspace. Missing Git or lockfile metadata remain absent from provenance records.

For the notebook workflow and download catalogue, see the notebook guide and data downloads.

Bounded acquisition of immutable scientific resources with literal byte checks.

`download_resource(url: str, destination: str | Path, , sha256: str, maximum_bytes: int = 512 1024 ** 2, timeout: float = 60.0) -> Path`

Acquire immutable bytes with a size limit and an expected SHA256 digest.

maximum_bytes limits the complete payload in bytes, independently of HTTP headers. timeout is the positive finite network timeout in seconds. An existing file is reused only if its checksum matches; an unexpected existing file is preserved and raises ValueError. Failed, oversized or corrupt downloads cannot replace the destination. This function performs a network request only when explicitly called for a missing destination. PYMHM_RESOURCE_ORIGIN can select an HTTP mirror while retaining the source path and query; checksum and size verification remain unchanged.

Source code in src/pymhm/io/resources.py

def download_resource(
    url: str,
    destination: str | Path,
    *,
    sha256: str,
    maximum_bytes: int = 512 * 1024**2,
    timeout: float = 60.0,
) -> Path:
    """Acquire immutable bytes with a size limit and an expected SHA256 digest.

    ``maximum_bytes`` limits the complete payload in bytes, independently of
    HTTP headers. ``timeout`` is the positive finite network timeout in seconds.
    An existing file is reused only if its checksum matches; an unexpected
    existing file is preserved and raises ``ValueError``. Failed, oversized or
    corrupt downloads cannot replace the destination. This function performs
    a network request only when explicitly called for a missing destination.
    PYMHM_RESOURCE_ORIGIN can select an HTTP mirror while retaining the source
    path and query; checksum and size verification remain unchanged.
    """
    limit = positive_int(maximum_bytes, "maximum_bytes")
    if (
        not isinstance(sha256, str)
        or len(sha256) != 64
        or any(character not in "0123456789abcdef" for character in sha256)
    ):
        raise ValueError("sha256 must be a lowercase 64-character hexadecimal digest")
    if not math.isfinite(timeout) or timeout <= 0:
        raise ValueError("timeout must be positive and finite")
    target = Path(destination).expanduser()
    if target.exists():
        if file_digest(target) != sha256:
            raise ValueError(f"SHA256 mismatch for existing resource: {target}")
        return target
    target.parent.mkdir(parents=True, exist_ok=True)
    origin = os.environ.get("PYMHM_RESOURCE_ORIGIN")
    if origin:
        mirror, original = urlsplit(origin), urlsplit(url)
        url = urlunsplit((mirror.scheme, mirror.netloc, original.path, original.query, ""))
    with tempfile.TemporaryDirectory(prefix=".pymhm-resource-", dir=target.parent) as temporary:
        pending = Path(temporary) / target.name
        digest = hashlib.sha256()
        total = 0
        with urlopen(url, timeout=timeout) as response, pending.open("wb") as stream:
            while chunk := response.read(min(1024**2, limit + 1)):
                total += len(chunk)
                if total > limit:
                    raise ValueError(f"resource exceeds the {limit}-byte acquisition limit")
                digest.update(chunk)
                stream.write(chunk)
        if digest.hexdigest() != sha256:
            raise ValueError(f"SHA256 mismatch for downloaded resource: {url}")
        if target.exists():
            if file_digest(target) != sha256:
                raise ValueError(f"SHA256 mismatch for concurrently created resource: {target}")
        else:
            pending.replace(target)
    return target

Generic writable workspaces, external catalogues and literal source identities.

`register_resources(directory: str | Path, manifest: Mapping[str, Any] | str | Path | None = None) -> None`

Register external resources for one explicit directory without acquiring them.

A catalogue maps resources to records with sha256 and, for downloads, url and size_bytes. Records can declare related_resources such as attribution notices. An unavailable mapping provides reasons for absent original data. Pass a mapping or JSON path; the default is .pymhm-resources.json in the directory. The library contains no application catalogue or dataset.