# Development and typed data

The toolkit uses ordinary dictionaries for banks and questions, with shared types in `coursebanks.models` describing validated data and workflow results.
YAML remains the authoring format described in [the YAML schema](schema.md).

## Setup and checks

Create and activate the repository's `.venv` as described in the [getting-started guide](getting-started.md), then install the package and development tools:

```bash
python -m pip install -e . --group dev
./scripts/run_checks.sh
```

The Bash script runs from the repository root and stops at the first failed check.
It uses the active `python` interpreter; set `QBT_PYTHON` to another interpreter path if necessary.
For example, from the repository root without activating the environment:

```bash
QBT_PYTHON=.venv/bin/python ./scripts/run_checks.sh
```

On platforms without Bash, run the same commands individually from the root:

```bash
python -m pytest
python -m ruff check .
python -m ruff format . --check
python -m mypy
python -m basedpyright
python -c 'from pydoclint.main import main; main()' src
```

Ruff uses the repository's explicit rule set and an 80-character line length.
Mypy and BasedPyright check `src/` and `tests/` in strict mode against the minimum supported Python version, 3.14.
BasedPyright resolves dependencies from `.venv`; use that environment for the script to keep dependency resolution consistent across both checkers.
The development requirements include `types-PyYAML` because PyYAML does not supply complete inline annotations.
Pydoclint checks NumPy-style parameter documentation for source modules.
Targeted DOC502 and DOC503 suppressions document places where exceptions are propagated through called functions or implicit mapping access rather than direct `raise` statements.
The current examples are YAML, so they are validated by pytest rather than a Python type checker.
The check script also builds the Sphinx documentation with warnings treated as errors.

## Documentation site

Build the user guide and generated Python API reference locally with:

```bash
QBT_PYTHON=.venv/bin/python ./scripts/build_docs.sh
```

The generated site is written to `site/` by default.
Open `site/index.html` in a browser to review it.
Publishing the reviewed output to Codeberg's `pages` branch is described in the [release guide](releasing.md#publish-the-documentation-site).

## Distribution checks

Build the wheel and source distribution, inspect their required contents, install each into a separate temporary environment, and exercise the installed command with:

```bash
QBT_PYTHON=.venv/bin/python ./scripts/check_distribution.sh
```

The script runs the installed entry point from outside the checkout and removes its temporary environments and artifacts when it finishes.

## Release tooling

The development dependency group includes Towncrier, Bump My Version, Twine, Sphinx, MyST Parser, Furo, and ghp-import.
Run `scripts/check_release.sh` to verify the changelog configuration and confirm that the three version declarations agree without changing files.
For fragment naming, version increments, release preparation, review, commits, and tags, follow the [release guide](releasing.md).

## Continuous integration

The Woodpecker workflow in `.woodpecker/checks.yaml` uses Python 3.14 on Linux for pushes and pull requests handled by the configured Codeberg CI service.
It creates the same `.venv` layout used by local development, installs the development dependency group, invokes `scripts/run_checks.sh` as the single definition of code-quality checks, validates the release configuration, and then runs the distribution check.
The repository must be activated at Codeberg's Woodpecker service before the workflow will run remotely.
When adding Python examples, add their directory to both checker configurations.

Run `python -m ruff format .` to apply formatting and `python -m ruff check . --fix` for safe lint fixes.
Review changes before accepting any unsafe fixes.
Type-checker suppressions must be narrow and explain an external typing limitation; do not disable strict checks to accept an invalid application value.

## Untrusted and validated banks

`load_yaml` reads safe YAML and returns a recursive `YamlMapping` without requiring a complete bank.
`parse_bank` checks schema fields, accumulates diagnostics, and constructs a `Bank` and its `Question` dictionaries from checked values.
`load_bank` combines these operations and returns a validated `Bank` directly.

```python
from pathlib import Path

from coursebanks.core import build, load_bank
from coursebanks.models import Bank, BuildReport

bank: Bank = load_bank("examples/sample_course/questions.yaml")
archive: Path
report: BuildReport
archive, report = build(bank, "build")
print(archive, report["count"], report["sha256"])
```

For incomplete drafts, use `load_yaml` and `save_bank`; neither requires schema compliance.
`validate` remains a diagnostic entry point for arbitrary Python input and returns accumulated errors without modifying it.
`require_valid_bank` remains a compatibility wrapper for arbitrary safe data, but now constructs a copy rather than returning the original dictionary.
Existing code that used `load_bank` for drafts must switch to `load_yaml`.

Bank construction retains optional authoring metadata at the bank and question levels and leaves omitted defaults omitted.
The result is independent of the source draft, including nested choices and metadata.
Known fields are constructed explicitly; the parser does not cast dictionaries to `Bank` or `Question`.
Unknown string field names are copied separately without replacing checked fields.
Revalidate after editing a bank because Python types cannot enforce positive numbers, enum membership, unique IDs, or answer correctness.
`build` performs validation before creating outputs, and `import_qti` constructs a validated bank through the same parser.
Lower-level XML and QTI generation functions expect validated inputs.

`YamlValue` describes safe scalar values, lists, mappings, sets, and tuples used by YAML pairs.
Dates, timestamps, and binary metadata are supported alongside ordinary strings, numbers, booleans, and null.
Mapping keys and set members must be scalar values.
Bank and question field names must be strings; nested metadata mappings can retain other scalar keys.
Nonrecursive aliases retain their values as independent copies; recursive aliases are rejected with a diagnostic.
External Python container narrowing is confined to `yaml_data.py`, where members are checked before being admitted into this recursive type.

## Shared types

| Type | Contract |
| --- | --- |
| `Question` | Required single-answer question fields, ordered string choices, optional `format`, and an optional choice-shuffling override; validation enforces allowed values. |
| `Bank` | Bank identity, an ordered question list, and an optional choice-shuffling default; validation enforces schema version, nonemptiness, and uniqueness. |
| `QTIReport` | Structural question count, ordered question IDs, and the verified assessment ZIP member. |
| `BuildReport` | QTI report plus course/bank identity, difficulty counts, and archive SHA-256. |
| `UsageRecord` | Course, bank, term, assessment, and question ID for an administered question. |
| `PlannedUse` | Proposed usage with area and difficulty; generating it does not record administration. |

The package includes a `py.typed` marker so downstream type checkers can use these annotations.
Neither typed reports nor passing local checks certify Canvas rendering or import acceptance; those remain separate integration checks in the [cleanup plan](https://codeberg.org/ddmarshall/QuestionBankToolkit/src/branch/main/docs/cleanup-plan.md).

## External archive and XML parsing

`coursebanks.archive_io.validate_archive` inspects the ZIP central directory before any XML member is decompressed.
It rejects more than 100 entries, duplicate names, members larger than 16 MiB uncompressed, totals larger than 32 MiB uncompressed, and per-member compression ratios above 100:1.
`read_member` also bounds the observed decompressed bytes, and import repeats validation when it reopens the archive after structural verification.

All external XML parsing goes through `coursebanks.xml_io.parse_xml`, which uses `defusedxml` with DTDs, entities, and external references forbidden.
Both manifest and assessment members are protected, including the assessment reread during import.
Malformed XML and forbidden constructs become `ValueError` diagnostics handled by the CLI with exit code 2.
This policy rejects even benign DTD declarations; supported QTI packages must not depend on them.

Standard-library ElementTree remains in use for XML construction, serialization, element types, and parser exception types.
Those imports have narrowly documented S405 exemptions.
The two test modules that parse only locally generated XML have scoped S405/S314 exemptions; application parsing rules remain enabled.
The development dependencies include `types-defusedxml` for strict checking.
Archive limits reduce expansion risk but do not establish complete ZIP safety, QTI conformance, or Canvas compatibility.