libre-bible-data/CONVENTIONS.md

Libre Bible Data Conventions

LibreBible is the public Bible and study-resource data project. This technical repo, libre-bible-data, is the canonical source, normalization, and packaging repository for free-to-use Bible and Bible-study resources used by Libre Study, GracePress Bible Tooltip, and related projects.

Libre Study, LibreBible tooling, generated catalog code, and ChristIT-authored metadata are intended to be free to use, open source, free to own, and free to reuse. Third-party Bible texts and study resources may have narrower terms. When a resource is not public domain or open licensed, catalogs and manifests must describe it as a licensed/free-to-use resource and must preserve its redistribution, attribution, trademark, noncommercial, quotation, and app-linking requirements.

Repository Purpose

This repo should gather every Bible translation and study resource that we can legally redistribute and normalize into app-ready packages.

Priority order:

1. Free-to-use Bible translations tied to Strong's numbers, morphology, lemmas, or concordance data.
2. Free-to-use Bible translations with attached study notes.
3. Free-to-use lexicons, dictionaries, cross-reference sets, and translation helps.
4. Free-to-use commentaries that can be tied to specific verses, ranges, chapters, books, or biblical sections.
5. Free-to-use maps, media, timelines, outlines, and other study aids.

The long-term goal is a dynamic resource library that applications can search, install, update, and combine into a more complete Bible study experience.

Local and Remote Workflow

• Local development happens in the workspace repo on drive W:.
• The Christ Unscripted Gitea remote is the remote backup, collaboration, and publishing copy.
• Do not treat resource work as durable until it is committed locally and pushed to Gitea.
• Every named implementation phase should bump the repo version and be recorded in Git both locally and remotely.
• Phase completion means the version bump, code/data/docs changes, and validation result are committed locally and pushed to Gitea.
• The normal flow is local import/check, generated package verification, local commit, push to Gitea, then publish generated catalogs or releases from the pushed state.
• The local repo and the Gitea repo are both intentional copies. Either should be enough to recover the project if the other system fails.
• Do not publish generated resources or release artifacts from a dirty worktree unless Jason explicitly asks for that exact operation.
• Keep source-control bookkeeping quiet unless Jason asks about it or a source-control problem affects the work.
• Use the actual project scripts instead of ad hoc equivalents.
• If a command starts failing because of PowerShell, Bash, WSL, quoting, heredocs, pipes, regexes, $ variables, or nested shell layers, stop retrying the same command shape. Move the logic into a script file or use argv-style execution.
• Prefer PowerShell for normal Windows-local Node/import work in this repo. Use WSL Bash only when a tool or script is genuinely Linux-oriented.

Publishing Model

• Gitea should hold source manifests, importer scripts, generated packages, tags, release history, and issue/roadmap discussion.
• Public Gitea host: https://git.christit.com.
• Remote URL:

https://git.christit.com/libre-study/libre-bible-data.git
git@git.christit.com:libre-study/libre-bible-data.git

• A polished public front end should be generated from committed metadata, especially packages/json/catalog.json and per-resource package catalogs.
• Do not customize Gitea before proving the generated catalog/front-end approach. Gitea should remain the reliable Git and release system.
• Public catalog pages should show resource title, abbreviation, language, license, redistribution status, upstream source, last checked date, package checksums, counts, features, and download links.
• GracePress plugins, Libre Study, and any future apps should consume stable package URLs or release artifacts from the pushed repo/public catalog.
• Do not hand-maintain public catalog details that can be derived from manifests and generated package catalogs.
• Generated catalogs, package indexes, checksums, and format-specific outputs should be regenerated by scripts rather than hand-corrected after the fact.

Legal Boundary

• "Free online" is not enough. Redistribution and format conversion must be allowed.
• The software/tooling can be open source even when a bundled third-party text is only free-to-use under specific terms.
• Every committed text/resource package must have explicit license metadata.
• If redistribution is unclear, store source metadata and importer instructions only. Do not commit the resource content.
• Preserve upstream attribution and license notes in generated catalogs.
• Keep jurisdiction-specific restrictions visible, especially when a text is public domain in one country but restricted in another.

Source Manifests

Every source resource needs a manifest in sources/.

Each manifest should record:

• Stable resource id.
• Resource type, such as translation, study_notes, commentary, map, lexicon, dictionary, cross_reference, or timeline.
• Human title and abbreviation.
• Language code, language name, script, and dialect when relevant.
• Translation date, edition date, source text basis, tradition, and public description when the resource is a Bible translation.
• Contributors and maintainers with their roles.
• Upstream provider.
• Upstream human URL.
• Upstream download URL.
• Upstream format.
• License name and redistribution status.
• Jurisdiction notes.
• Expected source checksum.
• Last checked timestamp.
• Importer name and version.
• Generated package paths.
• Structured features and attachment metadata such as strongs, notes, morphology, commentary, maps, or cross-references.

Bible translation manifests should be broad enough to describe editions that include more than plain verse text. A translation may include or later connect to Hebrew, Greek, or Aramaic concordance entries, morphology, lemma links, study notes, commentaries, maps, timelines, media, and cross-reference resources. Do not encode KJV-only assumptions into the manifest shape.

Attachment metadata should identify:

• Attachment resource type.
• Whether the attachment is embedded in the source package or external.
• Relationship type, such as word-to-strongs, verse-to-note, range-to-commentary, place-to-map, or event-to-timeline.
• Anchor types, such as translation, book, chapter, verse, verse range, word/token, Strong's number, lemma, topic, place, or timeline event.
• Languages and systems involved, such as Hebrew, Greek, Aramaic, Strong's, morphology, or source-language lemmas.
• Generated package path when the attachment is packaged.

Metadata Truth

• Source manifests are the canonical metadata source for upstream provider, license, redistribution status, source checksum, importer, and generated package paths.
• Generated package catalogs must be produced from source manifests and package outputs, not manually corrected afterward.
• Build/import scripts should fail when required source, license, checksum, or package metadata is missing.
• The public catalog must not silently disagree with the source manifest.

Machine Catalogs

• packages/json/catalog.json is the machine-readable index for ChristIT.com, Libre Study, GracePress Bible Tooltip, and future consumers.
• Per-resource package catalogs, such as packages/json/<resource-id>/catalog.json, should carry the full resource metadata needed to render a public detail page without scraping Markdown.
• Machine catalogs should include schema version, project name, resource id, resource type, title, abbreviation, language, script, canon, translation/edition metadata, contributors, features, attachments, source, license, display summary, package paths, counts, file checksums, and source check metadata.
• Markdown docs are a human-readable projection of the same metadata. They must not become a separate truth source.
• Public ChristIT.com pages and app/plugin package discovery should read machine catalogs first, then link to Markdown docs for explanation.

Update Workflow

• Update checks must be mechanical and repeatable.
• The normal flow is:

npm.cmd run check
npm.cmd run build

• A source is considered unchanged when the upstream artifact checksum matches the manifest.
• If a checksum changes, inspect the upstream resource, license, and content before accepting the change.
• Update the manifest and generated packages in the same commit when accepting an upstream change.
• Do not silently overwrite generated packages without recording the source checksum that produced them.
• Keep update checks short and decisive: check source checksum, rebuild intended packages, inspect the relevant generated catalog, and stop unless Jason asks for broader verification.

Package Outputs

Generated packages should be app-friendly and stable.

Initial package targets:

• JSONL for simple streaming imports.
• SQLite-ready schemas for Libre Study and desktop apps.
• WordPress/plugin-ready packages for GracePress Bible Tooltip.

Future package targets may include:

• OSIS/USFM normalized exports.
• Search indexes.
• Static web catalogs.

Backup Copies

• Source control is the main recovery layer, but local .old backups are still useful before large, risky, or release-bound importer/schema changes.
• Put backups near the affected file or resource folder, not in random temp locations.
• Prefer backup filenames with timestamp plus short reason, for example import-usfm.js.20260712-parser-change.old.
• Do not create .old backups for every generated package refresh. Generated outputs should be reproducible from manifests and scripts.

Changelogs

• Keep changelogs newest-first when changelogs are introduced.
• Use a ## Current section when helpful, followed by dated release sections.
• Keep entries concrete and operational: source added, license verified, importer changed, package regenerated, checksum accepted, or behavior preserved.
• Mark reconstructed history clearly if it is built from old packages, upstream archives, or prior notes.

Data Model Direction

Resources should be able to attach to:

• Whole translation.
• Book.
• Chapter.
• Verse.
• Verse range.
• Word/token.
• Strong's number.
• Lemma.
• Topic/tag.
• Map location.
• Timeline event.

Do not flatten everything into verse text. Keep links, notes, lemmas, references, and resource relationships queryable.

Current First Resource

The first resource is KJV from eBible.org's eng-kjv2006 USFM package. It is treated as the first proving ground for:

• Source manifest discipline.
• Checksum-based update checks.
• USFM import.
• Verse normalization.
• Strong's-link extraction.
• Generated package catalogs.