Complete storage and search foundation

This commit is contained in:
2026-07-15 16:53:34 -05:00
parent 05ff4bcf9f
commit 2ece7bc36a
12 changed files with 288 additions and 27 deletions
+9 -6
View File
@@ -2,12 +2,15 @@
## Unreleased ## Unreleased
- Recorded the active development direction: use the local HotView/hot-reload setup for iterative app work and defer web/local builds and hosted deployment unless explicitly requested. - No unreleased changes.
- Started Phase 4 shared app core cleanup by adding an explicit `StudyDataClient` contract and browser/Tauri data adapters behind the existing study client API.
- Added explicit workspace/auth client contracts so local HotView storage and hosted workspace persistence sit behind the same app-core boundary. ## 0.1.32 - 2026-07-15
- Routed the app shell's hosted-login checks through the auth client instead of reading the hosted environment flag directly.
- Reframed the roadmap into seven sequential phases with a one-active-phase rule and marked hosted/deploy work as parked supporting infrastructure. - Completed Phase 5 storage/search foundation for the active HotView workflow.
- Added a versioned workspace snapshot with migration from legacy local session/settings keys. - Added a page-level data-client method so visible pages can hydrate through the shared app contract instead of assuming the reader owns whole-resource arrays.
- Changed browser/HotView Strong's loading to prefer KJV chapter shards and a compact Strong's search index, with whole-file loading kept only as a legacy package fallback.
- Added SQLite indexes and an FTS5 verse search table for the native data path.
- Updated LibreBible package consumption expectations around chapter-sharded verses, chapter-sharded Strong's links, and indexed Strong's lookups.
## 0.1.31 - 2026-07-15 ## 0.1.31 - 2026-07-15
+3 -1
View File
@@ -53,7 +53,7 @@ Work one phase at a time. A helper, spike, or supporting infrastructure task for
- Phase 2: LibreBible resource consumption. Complete for the first local study workspace. - Phase 2: LibreBible resource consumption. Complete for the first local study workspace.
- Phase 3: HotView development loop and Gitea sync discipline. Complete. - Phase 3: HotView development loop and Gitea sync discipline. Complete.
- Phase 4: Shared app core. Complete. - Phase 4: Shared app core. Complete.
- Phase 5: Storage and search foundation. - Phase 5: Storage and search foundation. Complete.
- Phase 6: Notes, exports, and optional sync. - Phase 6: Notes, exports, and optional sync.
- Phase 7: Study resource depth and polish. - Phase 7: Study resource depth and polish.
@@ -135,6 +135,8 @@ Goal: make hundreds of thousands or millions of entries feel normal.
Current foundation note: [Storage And Search Foundation](docs/storage-search-foundation.md). Current foundation note: [Storage And Search Foundation](docs/storage-search-foundation.md).
Current status: Complete for the active HotView workflow. Browser page loads now use a shared page-query method and hydrate visible KJV Strong's links from chapter shards. Strong's-number search uses a compact package index where available. The native SQLite path now has explicit reference/attachment indexes and an FTS5 verse-search table.
Tasks: Tasks:
- Keep SQLite for desktop and private/single-user workflows. - Keep SQLite for desktop and private/single-user workflows.
+6 -3
View File
@@ -2,12 +2,15 @@
Libre Study should query study resources through stable app-level search and resource interfaces. The UI should not care whether a result came from local JSON packages, browser indexes, Tauri SQLite, hosted SQLite, or a future server database. Libre Study should query study resources through stable app-level search and resource interfaces. The UI should not care whether a result came from local JSON packages, browser indexes, Tauri SQLite, hosted SQLite, or a future server database.
## Current Phase 5 Step ## Current Phase 5 Result
- Browser/local preview still loads LibreBible JSON/JSONL packages. - Browser/local preview still loads LibreBible JSON/JSONL packages.
- Visible page hydration now goes through a shared page-query method instead of the reader assuming every resource is already attached to the whole-library array.
- Verse search now goes through a cached verse search index instead of open-coded filtering. - Verse search now goes through a cached verse search index instead of open-coded filtering.
- The index supports normal text search, reference text, `tag:` search, and Strong's-number search. - KJV Strong's-number search uses a compact package index where available.
- The index is disposable and rebuildable from the loaded translation package, which keeps it compatible with later SQLite/server-backed search. - KJV page Strong's links load from `strongs-by-chapter` shards where available, with whole-file loading kept as a fallback for older packages.
- The native SQLite path has explicit reference/attachment indexes and an FTS5 verse-search table.
- Browser indexes are disposable and rebuildable from loaded package surfaces, which keeps them compatible with later SQLite/server-backed search.
## Resource Data vs User Data ## Resource Data vs User Data
+2 -2
View File
@@ -1,12 +1,12 @@
{ {
"name": "libre-study", "name": "libre-study",
"version": "0.1.31", "version": "0.1.32",
"lockfileVersion": 3, "lockfileVersion": 3,
"requires": true, "requires": true,
"packages": { "packages": {
"": { "": {
"name": "libre-study", "name": "libre-study",
"version": "0.1.31", "version": "0.1.32",
"dependencies": { "dependencies": {
"@tauri-apps/api": "^2.11.1", "@tauri-apps/api": "^2.11.1",
"jszip": "^3.10.1", "jszip": "^3.10.1",
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "libre-study", "name": "libre-study",
"private": true, "private": true,
"version": "0.1.31", "version": "0.1.32",
"type": "module", "type": "module",
"scripts": { "scripts": {
"dev": "vite --host 0.0.0.0", "dev": "vite --host 0.0.0.0",
+1 -1
View File
@@ -77,7 +77,7 @@ checksum = "2a4385e2e34eb35d6b3efe798b9eb88096925d87726c0798709bf56d9ed84af3"
[[package]] [[package]]
name = "app" name = "app"
version = "0.1.26" version = "0.1.32"
dependencies = [ dependencies = [
"log", "log",
"rusqlite", "rusqlite",
+1 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "app" name = "app"
version = "0.1.26" version = "0.1.32"
description = "A Tauri App" description = "A Tauri App"
authors = ["you"] authors = ["you"]
license = "" license = ""
+50 -3
View File
@@ -226,6 +226,7 @@ fn open_db(app: &AppHandle) -> Result<Connection, String> {
let mut conn = Connection::open(path).map_err(|err| err.to_string())?; let mut conn = Connection::open(path).map_err(|err| err.to_string())?;
migrate(&conn)?; migrate(&conn)?;
seed(&mut conn)?; seed(&mut conn)?;
sync_search_indexes(&conn)?;
Ok(conn) Ok(conn)
} }
@@ -297,6 +298,20 @@ fn migrate(conn: &Connection) -> Result<(), String> {
FOREIGN KEY (verse_id) REFERENCES verses(id), FOREIGN KEY (verse_id) REFERENCES verses(id),
FOREIGN KEY (tag_id) REFERENCES tags(id) FOREIGN KEY (tag_id) REFERENCES tags(id)
); );
CREATE VIRTUAL TABLE IF NOT EXISTS verses_fts USING fts5(
text,
reference,
content='verses',
content_rowid='id'
);
CREATE INDEX IF NOT EXISTS idx_verses_reference ON verses(book, chapter, verse);
CREATE INDEX IF NOT EXISTS idx_verse_strongs_strong ON verse_strongs(strong);
CREATE INDEX IF NOT EXISTS idx_verse_strongs_verse ON verse_strongs(verse_id);
CREATE INDEX IF NOT EXISTS idx_verse_footnotes_verse ON verse_footnotes(verse_id);
CREATE INDEX IF NOT EXISTS idx_verse_tags_tag ON verse_tags(tag_id);
CREATE INDEX IF NOT EXISTS idx_tags_label ON tags(label);
", ",
) )
.map_err(|err| err.to_string())?; .map_err(|err| err.to_string())?;
@@ -778,20 +793,28 @@ fn load_verses(conn: &Connection, query: Option<&str>) -> Result<Vec<Verse>, Str
match query { match query {
Some(query) => { Some(query) => {
let fts_query = search_query_for_fts(query);
if fts_query.is_empty() {
return Ok(Vec::new());
}
let pattern = format!("%{}%", query); let pattern = format!("%{}%", query);
let mut stmt = conn let mut stmt = conn
.prepare( .prepare(
" "
SELECT v.id, v.book, v.chapter, v.verse, v.text
FROM verses v
JOIN verses_fts fts ON fts.rowid = v.id
WHERE verses_fts MATCH ?1
UNION
SELECT id, book, chapter, verse, text SELECT id, book, chapter, verse, text
FROM verses FROM verses
WHERE text LIKE ?1 WHERE book || ' ' || chapter || ':' || verse LIKE ?2
OR book || ' ' || chapter || ':' || verse LIKE ?1
ORDER BY id ORDER BY id
", ",
) )
.map_err(|err| err.to_string())?; .map_err(|err| err.to_string())?;
let rows = stmt let rows = stmt
.query_map(params![pattern], row_to_verse_shell) .query_map(params![fts_query, pattern], row_to_verse_shell)
.map_err(|err| err.to_string())?; .map_err(|err| err.to_string())?;
for row in rows { for row in rows {
@@ -1110,6 +1133,30 @@ fn load_all_tags(conn: &Connection) -> Result<Vec<Tag>, String> {
Ok(tags) Ok(tags)
} }
fn sync_search_indexes(conn: &Connection) -> Result<(), String> {
let verse_count: i64 = conn
.query_row("SELECT COUNT(*) FROM verses", [], |row| row.get(0))
.map_err(|err| err.to_string())?;
let fts_count: i64 = conn
.query_row("SELECT COUNT(*) FROM verses_fts", [], |row| row.get(0))
.map_err(|err| err.to_string())?;
if verse_count != fts_count {
conn.execute("INSERT INTO verses_fts(verses_fts) VALUES('rebuild')", [])
.map_err(|err| err.to_string())?;
}
Ok(())
}
fn search_query_for_fts(query: &str) -> String {
query
.split(|char: char| !char.is_ascii_alphanumeric())
.filter(|token| !token.is_empty())
.collect::<Vec<_>>()
.join(" ")
}
fn looks_like_strongs(query: &str) -> bool { fn looks_like_strongs(query: &str) -> bool {
let mut chars = query.chars(); let mut chars = query.chars();
matches!(chars.next(), Some('G' | 'g' | 'H' | 'h')) && chars.all(|char| char.is_ascii_digit()) matches!(chars.next(), Some('G' | 'g' | 'H' | 'h')) && chars.all(|char| char.is_ascii_digit())
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"$schema": "../node_modules/@tauri-apps/cli/config.schema.json", "$schema": "../node_modules/@tauri-apps/cli/config.schema.json",
"productName": "Libre Study", "productName": "Libre Study",
"version": "0.1.26", "version": "0.1.32",
"identifier": "org.librestudy.desktop", "identifier": "org.librestudy.desktop",
"build": { "build": {
"frontendDist": "../dist", "frontendDist": "../dist",
+45 -2
View File
@@ -7,6 +7,7 @@ import {
getDictionaryEntriesForTerms, getDictionaryEntriesForTerms,
getLibraryState, getLibraryState,
getMapContextForVerses, getMapContextForVerses,
getPageVerses,
getTranslationNotesForVerses, getTranslationNotesForVerses,
getTranslations, getTranslations,
searchVerses, searchVerses,
@@ -62,6 +63,17 @@ function replaceChangedVerses(verses: Verse[], hydrate: (verse: Verse) => Verse)
return changed ? next : verses return changed ? next : verses
} }
function sameHydratedVerse(left: Verse | undefined, right: Verse) {
return Boolean(
left &&
left.text === right.text &&
(left.strongs?.length ?? 0) === (right.strongs?.length ?? 0) &&
(left.footnotes?.length ?? 0) === (right.footnotes?.length ?? 0) &&
(left.studyNotes?.length ?? 0) === (right.studyNotes?.length ?? 0) &&
left.tags.length === right.tags.length,
)
}
function formatTimelineTime(value: string) { function formatTimelineTime(value: string) {
return new Date(value).toLocaleTimeString([], { hour: 'numeric', minute: '2-digit' }) return new Date(value).toLocaleTimeString([], { hour: 'numeric', minute: '2-digit' })
} }
@@ -559,8 +571,8 @@ function App() {
const selectedVerse = useMemo( const selectedVerse = useMemo(
() => () =>
libraryVerses.find((verse) => verse.id === selectedId) ??
verses.find((verse) => verse.id === selectedId) ?? verses.find((verse) => verse.id === selectedId) ??
libraryVerses.find((verse) => verse.id === selectedId) ??
verses[0], verses[0],
[libraryVerses, selectedId, verses], [libraryVerses, selectedId, verses],
) )
@@ -639,7 +651,7 @@ function App() {
return anchors return anchors
}, [libraryVerses, verses]) }, [libraryVerses, verses])
const pageVerses = useMemo(() => { const pageShells = useMemo(() => {
if (!selectedVerse) return [] if (!selectedVerse) return []
const source = libraryVerses.length ? libraryVerses : verses const source = libraryVerses.length ? libraryVerses : verses
const pageStartIndex = source.findIndex((verse) => verse.id === pageStartId) const pageStartIndex = source.findIndex((verse) => verse.id === pageStartId)
@@ -650,6 +662,37 @@ function App() {
.slice(start, start + pageSize) .slice(start, start + pageSize)
.filter((verse) => verse.book === book) .filter((verse) => verse.book === book)
}, [libraryVerses, pageStartId, selectedVerse, verses]) }, [libraryVerses, pageStartId, selectedVerse, verses])
const pageVerses = useMemo(() => {
const hydratedVerses = new Map(verses.map((verse) => [verse.id, verse]))
return pageShells.map((verse) => hydratedVerses.get(verse.id) ?? verse)
}, [pageShells, verses])
useEffect(() => {
if (!activeSession?.baseTranslation || !pageStartId || pageShells.length === 0) return
let cancelled = false
getPageVerses(activeSession.baseTranslation, pageStartId, pageSize)
.then((nextPageVerses) => {
if (cancelled || nextPageVerses.length === 0) return
setVerses((current) => {
const merged = new Map(current.map((verse) => [verse.id, verse]))
let changed = false
for (const verse of nextPageVerses) {
const existing = merged.get(verse.id)
if (!sameHydratedVerse(existing, verse)) {
merged.set(verse.id, verse)
changed = true
}
}
return changed ? [...merged.values()].sort((left, right) => left.id - right.id) : current
})
})
.catch((reason) => setError(String(reason)))
return () => {
cancelled = true
}
}, [activeSession?.baseTranslation, pageStartId, pageShells.length])
useEffect(() => { useEffect(() => {
let cancelled = false let cancelled = false
+168 -6
View File
@@ -38,6 +38,8 @@ type ImportedVerse = {
type ImportedStrongLink = { type ImportedStrongLink = {
verse_id: string verse_id: string
book_id?: string
chapter?: number
position: number position: number
surface: string surface: string
strong: string strong: string
@@ -94,6 +96,12 @@ type ImportedStrongEntry = {
kjv_definition?: string kjv_definition?: string
} }
type ImportedStrongIndexEntry = {
strong: string
verse_ids: string[]
surfaces: string[]
}
type ImportedCommentaryEntry = { type ImportedCommentaryEntry = {
id: string id: string
resource_id: string resource_id: string
@@ -168,6 +176,9 @@ type BrowserStrongData = {
let browserCatalogCache: Promise<ImportedCatalog> | null = null let browserCatalogCache: Promise<ImportedCatalog> | null = null
const browserLibraryCache = new Map<string, Promise<Verse[]>>() const browserLibraryCache = new Map<string, Promise<Verse[]>>()
const browserStrongDataCache = new Map<string, Promise<BrowserStrongData>>() const browserStrongDataCache = new Map<string, Promise<BrowserStrongData>>()
const browserChapterStrongDataCache = new Map<string, Promise<BrowserStrongData>>()
const browserStrongSearchIndexCache = new Map<string, Promise<Map<string, string[]>>>()
let browserLexiconCache: Promise<Map<string, BrowserLexiconEntry>> | null = null
const browserFootnotesCache = new Map<string, Promise<Map<string, Footnote[]>>>() const browserFootnotesCache = new Map<string, Promise<Map<string, Footnote[]>>>()
const browserStudyNotesCache = new Map<string, Promise<Map<string, Footnote[]>>>() const browserStudyNotesCache = new Map<string, Promise<Map<string, Footnote[]>>>()
const browserChapterFootnotesCache = new Map<string, Promise<Map<string, Footnote[]>>>() const browserChapterFootnotesCache = new Map<string, Promise<Map<string, Footnote[]>>>()
@@ -179,6 +190,7 @@ let browserVersePlacesCache: Promise<Map<string, BiblePlace[]>> | null = null
let browserRoutesCache: Promise<BibleRoute[]> | null = null let browserRoutesCache: Promise<BibleRoute[]> | null = null
let browserAudioPlaylistCache: Promise<Map<string, ImportedAudioChapterPlaylist>> | null = null let browserAudioPlaylistCache: Promise<Map<string, ImportedAudioChapterPlaylist>> | null = null
const browserVerseSourceIds = new Map<number, string>() const browserVerseSourceIds = new Map<number, string>()
const browserVerseIdsBySource = new Map<string, number>()
const browserAudioZipCache = new Map<string, Promise<JSZip>>() const browserAudioZipCache = new Map<string, Promise<JSZip>>()
const browserAudioObjectUrlCache = new Map<string, Promise<string>>() const browserAudioObjectUrlCache = new Map<string, Promise<string>>()
@@ -329,20 +341,48 @@ async function loadBrowserVersesForTranslation(translationId = defaultTranslatio
} }
const verses = await browserLibraryCache.get(translationId)! const verses = await browserLibraryCache.get(translationId)!
const strongData = await loadBrowserStrongData(translationId).catch((error) => {
console.warn('Unable to load browser Strong data.', error)
return null
})
const tagsByVerse = readBrowserTags() const tagsByVerse = readBrowserTags()
return verses.map((verse) => ({ return verses.map((verse) => ({
...verse, ...verse,
strongs: strongData?.linksByVerse.get(browserVerseKey(verse)) ?? verse.strongs,
footnotes: verse.footnotes ?? [], footnotes: verse.footnotes ?? [],
studyNotes: verse.studyNotes ?? [], studyNotes: verse.studyNotes ?? [],
tags: tagsByVerse[verse.id] ?? [], tags: tagsByVerse[verse.id] ?? [],
})) }))
} }
async function hydrateBrowserVerses(translationId: string, verses: Verse[]) {
const chapterKeys = [
...new Set(
verses
.map((verse) => `${bookIdFromVerse(verse)}.${verse.chapter}`)
.filter((key) => !key.startsWith('unknown.')),
),
]
const strongDataMaps = await Promise.all(
chapterKeys.map((chapterKey) =>
loadBrowserChapterStrongData(translationId, chapterKey).catch((error) => {
console.warn(`Unable to load browser Strong data for ${chapterKey}.`, error)
return { linksByVerse: new Map<string, StrongLink[]>() }
}),
),
)
const tagsByVerse = readBrowserTags()
return verses.map((verse) => {
const verseKey = browserVerseKey(verse)
const strongs =
strongDataMaps.find((data) => data.linksByVerse.has(verseKey))?.linksByVerse.get(verseKey) ??
verse.strongs
return {
...verse,
strongs,
footnotes: verse.footnotes ?? [],
studyNotes: verse.studyNotes ?? [],
tags: tagsByVerse[verse.id] ?? [],
}
})
}
function allBrowserTags() { function allBrowserTags() {
const unique = new Map<string, Tag>() const unique = new Map<string, Tag>()
for (const tags of Object.values(readBrowserTags())) { for (const tags of Object.values(readBrowserTags())) {
@@ -375,6 +415,23 @@ async function getBrowserLibraryState(translationId = defaultTranslationId): Pro
} }
} }
async function getBrowserPageVerses(
translationId: string,
pageStartId: number,
pageSize: number,
): Promise<Verse[]> {
const verses = await loadBrowserVersesForTranslation(translationId)
const startIndex = verses.findIndex((verse) => verse.id === pageStartId)
if (startIndex < 0) return []
const book = verses[startIndex].book
const pageVerses = verses
.slice(startIndex, startIndex + pageSize)
.filter((verse) => verse.book === book)
return hydrateBrowserVerses(translationId, pageVerses)
}
async function getBrowserTranslationNotesForVerses( async function getBrowserTranslationNotesForVerses(
translationId: string, translationId: string,
verses: Verse[], verses: Verse[],
@@ -407,6 +464,14 @@ async function getBrowserTranslationNotesForVerses(
} }
async function searchBrowserVerses(query: string, translationId = defaultTranslationId): Promise<Verse[]> { async function searchBrowserVerses(query: string, translationId = defaultTranslationId): Promise<Verse[]> {
if (/^[gh]\d+$/i.test(query.trim())) {
const verses = await loadBrowserVersesForTranslation(translationId)
const strongIndex = await loadBrowserStrongSearchIndex(translationId)
const sourceIds = strongIndex.get(query.trim().toUpperCase()) ?? []
const verseIds = new Set(sourceIds.map((sourceId) => browserVerseIdsBySource.get(sourceId)))
return verses.filter((verse) => verseIds.has(verse.id))
}
const searchIndex = await loadBrowserSearchIndex(translationId) const searchIndex = await loadBrowserSearchIndex(translationId)
return searchIndex.search(query) return searchIndex.search(query)
} }
@@ -552,6 +617,7 @@ const emptyMapContext: MapContext = {
const browserStudyDataClient: StudyDataClient = { const browserStudyDataClient: StudyDataClient = {
getTranslations: getBrowserTranslations, getTranslations: getBrowserTranslations,
getLibraryState: getBrowserLibraryState, getLibraryState: getBrowserLibraryState,
getPageVerses: getBrowserPageVerses,
getTranslationNotesForVerses: getBrowserTranslationNotesForVerses, getTranslationNotesForVerses: getBrowserTranslationNotesForVerses,
searchVerses: searchBrowserVerses, searchVerses: searchBrowserVerses,
addTagToVerse: addBrowserTagToVerse, addTagToVerse: addBrowserTagToVerse,
@@ -565,6 +631,13 @@ const browserStudyDataClient: StudyDataClient = {
const tauriStudyDataClient: StudyDataClient = { const tauriStudyDataClient: StudyDataClient = {
getTranslations: async () => [], getTranslations: async () => [],
getLibraryState: () => invoke<LibraryState>('get_library_state'), getLibraryState: () => invoke<LibraryState>('get_library_state'),
getPageVerses: async (_translationId, pageStartId, pageSize) => {
const state = await invoke<LibraryState>('get_library_state')
const startIndex = state.verses.findIndex((verse) => verse.id === pageStartId)
if (startIndex < 0) return []
const book = state.verses[startIndex].book
return state.verses.slice(startIndex, startIndex + pageSize).filter((verse) => verse.book === book)
},
getTranslationNotesForVerses: async () => new Map(), getTranslationNotesForVerses: async () => new Map(),
searchVerses: (query) => invoke<Verse[]>('search_verses', { query }), searchVerses: (query) => invoke<Verse[]>('search_verses', { query }),
addTagToVerse: (verseId, label, color) => invoke<Verse>('add_tag_to_verse', { verseId, label, color }), addTagToVerse: (verseId, label, color) => invoke<Verse>('add_tag_to_verse', { verseId, label, color }),
@@ -589,6 +662,14 @@ export function getLibraryState(translationId = defaultTranslationId): Promise<L
return getStudyDataClient().getLibraryState(translationId) return getStudyDataClient().getLibraryState(translationId)
} }
export function getPageVerses(
translationId: string,
pageStartId: number,
pageSize: number,
): Promise<Verse[]> {
return getStudyDataClient().getPageVerses(translationId, pageStartId, pageSize)
}
export function getTranslationNotesForVerses( export function getTranslationNotesForVerses(
translationId: string, translationId: string,
verses: Verse[], verses: Verse[],
@@ -648,15 +729,42 @@ async function loadBrowserSearchIndex(translationId = defaultTranslationId) {
return browserSearchIndexCache.get(translationId)! return browserSearchIndexCache.get(translationId)!
} }
async function loadBrowserStrongSearchIndex(translationId = defaultTranslationId) {
if (!browserStrongSearchIndexCache.has(translationId)) {
browserStrongSearchIndexCache.set(
translationId,
fetchJsonl<ImportedStrongIndexEntry>(`${libreBibleDevRoot}/${translationId}/strongs-index.jsonl`)
.then((entries) => new Map(entries.map((entry) => [entry.strong.toUpperCase(), entry.verse_ids])))
.catch(async () => {
const strongData = await loadBrowserStrongData(translationId)
const index = new Map<string, string[]>()
for (const [verseId, links] of strongData.linksByVerse) {
for (const link of links) {
const key = link.strong.toUpperCase()
const current = index.get(key) ?? []
if (!current.includes(verseId)) current.push(verseId)
index.set(key, current)
}
}
return index
}),
)
}
return browserStrongSearchIndexCache.get(translationId)!
}
async function loadLibreBibleDevVerses(translationId = defaultTranslationId): Promise<Verse[]> { async function loadLibreBibleDevVerses(translationId = defaultTranslationId): Promise<Verse[]> {
const importedVerses = await fetchJsonl<ImportedVerse>( const importedVerses = await fetchJsonl<ImportedVerse>(
`${libreBibleDevRoot}/${translationId}/verses.jsonl`, `${libreBibleDevRoot}/${translationId}/verses.jsonl`,
) )
browserVerseSourceIds.clear() browserVerseSourceIds.clear()
browserVerseIdsBySource.clear()
return importedVerses.map((verse, index) => { return importedVerses.map((verse, index) => {
const id = index + 1 const id = index + 1
browserVerseSourceIds.set(id, verse.id) browserVerseSourceIds.set(id, verse.id)
browserVerseIdsBySource.set(verse.id, id)
return { return {
id, id,
book: verse.book, book: verse.book,
@@ -982,6 +1090,42 @@ function mapImportedNotes(importedFootnotes: ImportedFootnote[]): Map<string, Fo
return footnotes return footnotes
} }
async function loadBrowserChapterStrongData(
translationId: string,
chapterKey: string,
): Promise<BrowserStrongData> {
const cacheKey = `${translationId}:${chapterKey}`
if (!browserChapterStrongDataCache.has(cacheKey)) {
browserChapterStrongDataCache.set(
cacheKey,
loadLibreBibleDevChapterStrongData(translationId, chapterKey).catch(async () => {
const fullStrongData = await loadBrowserStrongData(translationId)
const linksByVerse = new Map<string, StrongLink[]>()
for (const [verseId, links] of fullStrongData.linksByVerse) {
if (verseId.startsWith(`${chapterKey}.`)) linksByVerse.set(verseId, links)
}
return { linksByVerse }
}),
)
}
return browserChapterStrongDataCache.get(cacheKey)!
}
async function loadLibreBibleDevChapterStrongData(
translationId: string,
chapterKey: string,
): Promise<BrowserStrongData> {
const [importedLinks, lexicon] = await Promise.all([
fetchJsonl<ImportedStrongLink>(
`${libreBibleDevRoot}/${translationId}/strongs-by-chapter/${chapterKey}.strongs.jsonl`,
),
loadBrowserLexicon(),
])
return mapStrongLinks(importedLinks, lexicon)
}
async function loadBrowserStrongData(translationId = defaultTranslationId): Promise<BrowserStrongData> { async function loadBrowserStrongData(translationId = defaultTranslationId): Promise<BrowserStrongData> {
if (!browserStrongDataCache.has(translationId)) { if (!browserStrongDataCache.has(translationId)) {
browserStrongDataCache.set(translationId, loadLibreBibleDevStrongData(translationId)) browserStrongDataCache.set(translationId, loadLibreBibleDevStrongData(translationId))
@@ -996,7 +1140,19 @@ async function loadLibreBibleDevStrongData(translationId = defaultTranslationId)
fetchJsonl<ImportedStrongEntry>(`${libreBibleDevRoot}/strongs-open-scriptures/entries.jsonl`), fetchJsonl<ImportedStrongEntry>(`${libreBibleDevRoot}/strongs-open-scriptures/entries.jsonl`),
]) ])
const lexicon = new Map<string, BrowserLexiconEntry>( return mapStrongLinks(importedLinks, mapBrowserLexicon(importedEntries))
}
async function loadBrowserLexicon(): Promise<Map<string, BrowserLexiconEntry>> {
browserLexiconCache ??= fetchJsonl<ImportedStrongEntry>(
`${libreBibleDevRoot}/strongs-open-scriptures/entries.jsonl`,
).then(mapBrowserLexicon)
return browserLexiconCache
}
function mapBrowserLexicon(importedEntries: ImportedStrongEntry[]) {
return new Map<string, BrowserLexiconEntry>(
importedEntries.map((entry) => [ importedEntries.map((entry) => [
entry.strongs_number, entry.strongs_number,
{ {
@@ -1007,6 +1163,12 @@ async function loadLibreBibleDevStrongData(translationId = defaultTranslationId)
}, },
]), ]),
) )
}
function mapStrongLinks(
importedLinks: ImportedStrongLink[],
lexicon: Map<string, BrowserLexiconEntry>,
): BrowserStrongData {
const linksByVerse = new Map<string, StrongLink[]>() const linksByVerse = new Map<string, StrongLink[]>()
for (const link of importedLinks) { for (const link of importedLinks) {
+1
View File
@@ -15,6 +15,7 @@ export type TranslationNotesByVerse = Map<string, { footnotes: Footnote[]; study
export type StudyDataClient = { export type StudyDataClient = {
getTranslations(): Promise<TranslationResource[]> getTranslations(): Promise<TranslationResource[]>
getLibraryState(translationId?: string): Promise<LibraryState> getLibraryState(translationId?: string): Promise<LibraryState>
getPageVerses(translationId: string, pageStartId: number, pageSize: number): Promise<Verse[]>
getTranslationNotesForVerses(translationId: string, verses: Verse[]): Promise<TranslationNotesByVerse> getTranslationNotesForVerses(translationId: string, verses: Verse[]): Promise<TranslationNotesByVerse>
searchVerses(query: string, translationId?: string): Promise<Verse[]> searchVerses(query: string, translationId?: string): Promise<Verse[]>
addTagToVerse(verseId: number, label: string, color: string, translationId?: string): Promise<Verse> addTagToVerse(verseId: number, label: string, color: string, translationId?: string): Promise<Verse>