Complete storage and search foundation
This commit is contained in:
+9
-6
@@ -2,12 +2,15 @@
|
|||||||
|
|
||||||
## Unreleased
|
## Unreleased
|
||||||
|
|
||||||
- Recorded the active development direction: use the local HotView/hot-reload setup for iterative app work and defer web/local builds and hosted deployment unless explicitly requested.
|
- No unreleased changes.
|
||||||
- Started Phase 4 shared app core cleanup by adding an explicit `StudyDataClient` contract and browser/Tauri data adapters behind the existing study client API.
|
|
||||||
- Added explicit workspace/auth client contracts so local HotView storage and hosted workspace persistence sit behind the same app-core boundary.
|
## 0.1.32 - 2026-07-15
|
||||||
- Routed the app shell's hosted-login checks through the auth client instead of reading the hosted environment flag directly.
|
|
||||||
- Reframed the roadmap into seven sequential phases with a one-active-phase rule and marked hosted/deploy work as parked supporting infrastructure.
|
- Completed Phase 5 storage/search foundation for the active HotView workflow.
|
||||||
- Added a versioned workspace snapshot with migration from legacy local session/settings keys.
|
- Added a page-level data-client method so visible pages can hydrate through the shared app contract instead of assuming the reader owns whole-resource arrays.
|
||||||
|
- Changed browser/HotView Strong's loading to prefer KJV chapter shards and a compact Strong's search index, with whole-file loading kept only as a legacy package fallback.
|
||||||
|
- Added SQLite indexes and an FTS5 verse search table for the native data path.
|
||||||
|
- Updated LibreBible package consumption expectations around chapter-sharded verses, chapter-sharded Strong's links, and indexed Strong's lookups.
|
||||||
|
|
||||||
## 0.1.31 - 2026-07-15
|
## 0.1.31 - 2026-07-15
|
||||||
|
|
||||||
|
|||||||
+3
-1
@@ -53,7 +53,7 @@ Work one phase at a time. A helper, spike, or supporting infrastructure task for
|
|||||||
- Phase 2: LibreBible resource consumption. Complete for the first local study workspace.
|
- Phase 2: LibreBible resource consumption. Complete for the first local study workspace.
|
||||||
- Phase 3: HotView development loop and Gitea sync discipline. Complete.
|
- Phase 3: HotView development loop and Gitea sync discipline. Complete.
|
||||||
- Phase 4: Shared app core. Complete.
|
- Phase 4: Shared app core. Complete.
|
||||||
- Phase 5: Storage and search foundation.
|
- Phase 5: Storage and search foundation. Complete.
|
||||||
- Phase 6: Notes, exports, and optional sync.
|
- Phase 6: Notes, exports, and optional sync.
|
||||||
- Phase 7: Study resource depth and polish.
|
- Phase 7: Study resource depth and polish.
|
||||||
|
|
||||||
@@ -135,6 +135,8 @@ Goal: make hundreds of thousands or millions of entries feel normal.
|
|||||||
|
|
||||||
Current foundation note: [Storage And Search Foundation](docs/storage-search-foundation.md).
|
Current foundation note: [Storage And Search Foundation](docs/storage-search-foundation.md).
|
||||||
|
|
||||||
|
Current status: Complete for the active HotView workflow. Browser page loads now use a shared page-query method and hydrate visible KJV Strong's links from chapter shards. Strong's-number search uses a compact package index where available. The native SQLite path now has explicit reference/attachment indexes and an FTS5 verse-search table.
|
||||||
|
|
||||||
Tasks:
|
Tasks:
|
||||||
|
|
||||||
- Keep SQLite for desktop and private/single-user workflows.
|
- Keep SQLite for desktop and private/single-user workflows.
|
||||||
|
|||||||
@@ -2,12 +2,15 @@
|
|||||||
|
|
||||||
Libre Study should query study resources through stable app-level search and resource interfaces. The UI should not care whether a result came from local JSON packages, browser indexes, Tauri SQLite, hosted SQLite, or a future server database.
|
Libre Study should query study resources through stable app-level search and resource interfaces. The UI should not care whether a result came from local JSON packages, browser indexes, Tauri SQLite, hosted SQLite, or a future server database.
|
||||||
|
|
||||||
## Current Phase 5 Step
|
## Current Phase 5 Result
|
||||||
|
|
||||||
- Browser/local preview still loads LibreBible JSON/JSONL packages.
|
- Browser/local preview still loads LibreBible JSON/JSONL packages.
|
||||||
|
- Visible page hydration now goes through a shared page-query method instead of the reader assuming every resource is already attached to the whole-library array.
|
||||||
- Verse search now goes through a cached verse search index instead of open-coded filtering.
|
- Verse search now goes through a cached verse search index instead of open-coded filtering.
|
||||||
- The index supports normal text search, reference text, `tag:` search, and Strong's-number search.
|
- KJV Strong's-number search uses a compact package index where available.
|
||||||
- The index is disposable and rebuildable from the loaded translation package, which keeps it compatible with later SQLite/server-backed search.
|
- KJV page Strong's links load from `strongs-by-chapter` shards where available, with whole-file loading kept as a fallback for older packages.
|
||||||
|
- The native SQLite path has explicit reference/attachment indexes and an FTS5 verse-search table.
|
||||||
|
- Browser indexes are disposable and rebuildable from loaded package surfaces, which keeps them compatible with later SQLite/server-backed search.
|
||||||
|
|
||||||
## Resource Data vs User Data
|
## Resource Data vs User Data
|
||||||
|
|
||||||
|
|||||||
Generated
+2
-2
@@ -1,12 +1,12 @@
|
|||||||
{
|
{
|
||||||
"name": "libre-study",
|
"name": "libre-study",
|
||||||
"version": "0.1.31",
|
"version": "0.1.32",
|
||||||
"lockfileVersion": 3,
|
"lockfileVersion": 3,
|
||||||
"requires": true,
|
"requires": true,
|
||||||
"packages": {
|
"packages": {
|
||||||
"": {
|
"": {
|
||||||
"name": "libre-study",
|
"name": "libre-study",
|
||||||
"version": "0.1.31",
|
"version": "0.1.32",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@tauri-apps/api": "^2.11.1",
|
"@tauri-apps/api": "^2.11.1",
|
||||||
"jszip": "^3.10.1",
|
"jszip": "^3.10.1",
|
||||||
|
|||||||
+1
-1
@@ -1,7 +1,7 @@
|
|||||||
{
|
{
|
||||||
"name": "libre-study",
|
"name": "libre-study",
|
||||||
"private": true,
|
"private": true,
|
||||||
"version": "0.1.31",
|
"version": "0.1.32",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"scripts": {
|
"scripts": {
|
||||||
"dev": "vite --host 0.0.0.0",
|
"dev": "vite --host 0.0.0.0",
|
||||||
|
|||||||
Generated
+1
-1
@@ -77,7 +77,7 @@ checksum = "2a4385e2e34eb35d6b3efe798b9eb88096925d87726c0798709bf56d9ed84af3"
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "app"
|
name = "app"
|
||||||
version = "0.1.26"
|
version = "0.1.32"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"log",
|
"log",
|
||||||
"rusqlite",
|
"rusqlite",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
[package]
|
[package]
|
||||||
name = "app"
|
name = "app"
|
||||||
version = "0.1.26"
|
version = "0.1.32"
|
||||||
description = "A Tauri App"
|
description = "A Tauri App"
|
||||||
authors = ["you"]
|
authors = ["you"]
|
||||||
license = ""
|
license = ""
|
||||||
|
|||||||
+50
-3
@@ -226,6 +226,7 @@ fn open_db(app: &AppHandle) -> Result<Connection, String> {
|
|||||||
let mut conn = Connection::open(path).map_err(|err| err.to_string())?;
|
let mut conn = Connection::open(path).map_err(|err| err.to_string())?;
|
||||||
migrate(&conn)?;
|
migrate(&conn)?;
|
||||||
seed(&mut conn)?;
|
seed(&mut conn)?;
|
||||||
|
sync_search_indexes(&conn)?;
|
||||||
Ok(conn)
|
Ok(conn)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -297,6 +298,20 @@ fn migrate(conn: &Connection) -> Result<(), String> {
|
|||||||
FOREIGN KEY (verse_id) REFERENCES verses(id),
|
FOREIGN KEY (verse_id) REFERENCES verses(id),
|
||||||
FOREIGN KEY (tag_id) REFERENCES tags(id)
|
FOREIGN KEY (tag_id) REFERENCES tags(id)
|
||||||
);
|
);
|
||||||
|
|
||||||
|
CREATE VIRTUAL TABLE IF NOT EXISTS verses_fts USING fts5(
|
||||||
|
text,
|
||||||
|
reference,
|
||||||
|
content='verses',
|
||||||
|
content_rowid='id'
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_verses_reference ON verses(book, chapter, verse);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_verse_strongs_strong ON verse_strongs(strong);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_verse_strongs_verse ON verse_strongs(verse_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_verse_footnotes_verse ON verse_footnotes(verse_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_verse_tags_tag ON verse_tags(tag_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_tags_label ON tags(label);
|
||||||
",
|
",
|
||||||
)
|
)
|
||||||
.map_err(|err| err.to_string())?;
|
.map_err(|err| err.to_string())?;
|
||||||
@@ -778,20 +793,28 @@ fn load_verses(conn: &Connection, query: Option<&str>) -> Result<Vec<Verse>, Str
|
|||||||
|
|
||||||
match query {
|
match query {
|
||||||
Some(query) => {
|
Some(query) => {
|
||||||
|
let fts_query = search_query_for_fts(query);
|
||||||
|
if fts_query.is_empty() {
|
||||||
|
return Ok(Vec::new());
|
||||||
|
}
|
||||||
let pattern = format!("%{}%", query);
|
let pattern = format!("%{}%", query);
|
||||||
let mut stmt = conn
|
let mut stmt = conn
|
||||||
.prepare(
|
.prepare(
|
||||||
"
|
"
|
||||||
|
SELECT v.id, v.book, v.chapter, v.verse, v.text
|
||||||
|
FROM verses v
|
||||||
|
JOIN verses_fts fts ON fts.rowid = v.id
|
||||||
|
WHERE verses_fts MATCH ?1
|
||||||
|
UNION
|
||||||
SELECT id, book, chapter, verse, text
|
SELECT id, book, chapter, verse, text
|
||||||
FROM verses
|
FROM verses
|
||||||
WHERE text LIKE ?1
|
WHERE book || ' ' || chapter || ':' || verse LIKE ?2
|
||||||
OR book || ' ' || chapter || ':' || verse LIKE ?1
|
|
||||||
ORDER BY id
|
ORDER BY id
|
||||||
",
|
",
|
||||||
)
|
)
|
||||||
.map_err(|err| err.to_string())?;
|
.map_err(|err| err.to_string())?;
|
||||||
let rows = stmt
|
let rows = stmt
|
||||||
.query_map(params![pattern], row_to_verse_shell)
|
.query_map(params![fts_query, pattern], row_to_verse_shell)
|
||||||
.map_err(|err| err.to_string())?;
|
.map_err(|err| err.to_string())?;
|
||||||
|
|
||||||
for row in rows {
|
for row in rows {
|
||||||
@@ -1110,6 +1133,30 @@ fn load_all_tags(conn: &Connection) -> Result<Vec<Tag>, String> {
|
|||||||
Ok(tags)
|
Ok(tags)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn sync_search_indexes(conn: &Connection) -> Result<(), String> {
|
||||||
|
let verse_count: i64 = conn
|
||||||
|
.query_row("SELECT COUNT(*) FROM verses", [], |row| row.get(0))
|
||||||
|
.map_err(|err| err.to_string())?;
|
||||||
|
let fts_count: i64 = conn
|
||||||
|
.query_row("SELECT COUNT(*) FROM verses_fts", [], |row| row.get(0))
|
||||||
|
.map_err(|err| err.to_string())?;
|
||||||
|
|
||||||
|
if verse_count != fts_count {
|
||||||
|
conn.execute("INSERT INTO verses_fts(verses_fts) VALUES('rebuild')", [])
|
||||||
|
.map_err(|err| err.to_string())?;
|
||||||
|
}
|
||||||
|
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn search_query_for_fts(query: &str) -> String {
|
||||||
|
query
|
||||||
|
.split(|char: char| !char.is_ascii_alphanumeric())
|
||||||
|
.filter(|token| !token.is_empty())
|
||||||
|
.collect::<Vec<_>>()
|
||||||
|
.join(" ")
|
||||||
|
}
|
||||||
|
|
||||||
fn looks_like_strongs(query: &str) -> bool {
|
fn looks_like_strongs(query: &str) -> bool {
|
||||||
let mut chars = query.chars();
|
let mut chars = query.chars();
|
||||||
matches!(chars.next(), Some('G' | 'g' | 'H' | 'h')) && chars.all(|char| char.is_ascii_digit())
|
matches!(chars.next(), Some('G' | 'g' | 'H' | 'h')) && chars.all(|char| char.is_ascii_digit())
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
{
|
{
|
||||||
"$schema": "../node_modules/@tauri-apps/cli/config.schema.json",
|
"$schema": "../node_modules/@tauri-apps/cli/config.schema.json",
|
||||||
"productName": "Libre Study",
|
"productName": "Libre Study",
|
||||||
"version": "0.1.26",
|
"version": "0.1.32",
|
||||||
"identifier": "org.librestudy.desktop",
|
"identifier": "org.librestudy.desktop",
|
||||||
"build": {
|
"build": {
|
||||||
"frontendDist": "../dist",
|
"frontendDist": "../dist",
|
||||||
|
|||||||
+45
-2
@@ -7,6 +7,7 @@ import {
|
|||||||
getDictionaryEntriesForTerms,
|
getDictionaryEntriesForTerms,
|
||||||
getLibraryState,
|
getLibraryState,
|
||||||
getMapContextForVerses,
|
getMapContextForVerses,
|
||||||
|
getPageVerses,
|
||||||
getTranslationNotesForVerses,
|
getTranslationNotesForVerses,
|
||||||
getTranslations,
|
getTranslations,
|
||||||
searchVerses,
|
searchVerses,
|
||||||
@@ -62,6 +63,17 @@ function replaceChangedVerses(verses: Verse[], hydrate: (verse: Verse) => Verse)
|
|||||||
return changed ? next : verses
|
return changed ? next : verses
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function sameHydratedVerse(left: Verse | undefined, right: Verse) {
|
||||||
|
return Boolean(
|
||||||
|
left &&
|
||||||
|
left.text === right.text &&
|
||||||
|
(left.strongs?.length ?? 0) === (right.strongs?.length ?? 0) &&
|
||||||
|
(left.footnotes?.length ?? 0) === (right.footnotes?.length ?? 0) &&
|
||||||
|
(left.studyNotes?.length ?? 0) === (right.studyNotes?.length ?? 0) &&
|
||||||
|
left.tags.length === right.tags.length,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
function formatTimelineTime(value: string) {
|
function formatTimelineTime(value: string) {
|
||||||
return new Date(value).toLocaleTimeString([], { hour: 'numeric', minute: '2-digit' })
|
return new Date(value).toLocaleTimeString([], { hour: 'numeric', minute: '2-digit' })
|
||||||
}
|
}
|
||||||
@@ -559,8 +571,8 @@ function App() {
|
|||||||
|
|
||||||
const selectedVerse = useMemo(
|
const selectedVerse = useMemo(
|
||||||
() =>
|
() =>
|
||||||
libraryVerses.find((verse) => verse.id === selectedId) ??
|
|
||||||
verses.find((verse) => verse.id === selectedId) ??
|
verses.find((verse) => verse.id === selectedId) ??
|
||||||
|
libraryVerses.find((verse) => verse.id === selectedId) ??
|
||||||
verses[0],
|
verses[0],
|
||||||
[libraryVerses, selectedId, verses],
|
[libraryVerses, selectedId, verses],
|
||||||
)
|
)
|
||||||
@@ -639,7 +651,7 @@ function App() {
|
|||||||
|
|
||||||
return anchors
|
return anchors
|
||||||
}, [libraryVerses, verses])
|
}, [libraryVerses, verses])
|
||||||
const pageVerses = useMemo(() => {
|
const pageShells = useMemo(() => {
|
||||||
if (!selectedVerse) return []
|
if (!selectedVerse) return []
|
||||||
const source = libraryVerses.length ? libraryVerses : verses
|
const source = libraryVerses.length ? libraryVerses : verses
|
||||||
const pageStartIndex = source.findIndex((verse) => verse.id === pageStartId)
|
const pageStartIndex = source.findIndex((verse) => verse.id === pageStartId)
|
||||||
@@ -650,6 +662,37 @@ function App() {
|
|||||||
.slice(start, start + pageSize)
|
.slice(start, start + pageSize)
|
||||||
.filter((verse) => verse.book === book)
|
.filter((verse) => verse.book === book)
|
||||||
}, [libraryVerses, pageStartId, selectedVerse, verses])
|
}, [libraryVerses, pageStartId, selectedVerse, verses])
|
||||||
|
const pageVerses = useMemo(() => {
|
||||||
|
const hydratedVerses = new Map(verses.map((verse) => [verse.id, verse]))
|
||||||
|
return pageShells.map((verse) => hydratedVerses.get(verse.id) ?? verse)
|
||||||
|
}, [pageShells, verses])
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
if (!activeSession?.baseTranslation || !pageStartId || pageShells.length === 0) return
|
||||||
|
|
||||||
|
let cancelled = false
|
||||||
|
getPageVerses(activeSession.baseTranslation, pageStartId, pageSize)
|
||||||
|
.then((nextPageVerses) => {
|
||||||
|
if (cancelled || nextPageVerses.length === 0) return
|
||||||
|
setVerses((current) => {
|
||||||
|
const merged = new Map(current.map((verse) => [verse.id, verse]))
|
||||||
|
let changed = false
|
||||||
|
for (const verse of nextPageVerses) {
|
||||||
|
const existing = merged.get(verse.id)
|
||||||
|
if (!sameHydratedVerse(existing, verse)) {
|
||||||
|
merged.set(verse.id, verse)
|
||||||
|
changed = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return changed ? [...merged.values()].sort((left, right) => left.id - right.id) : current
|
||||||
|
})
|
||||||
|
})
|
||||||
|
.catch((reason) => setError(String(reason)))
|
||||||
|
|
||||||
|
return () => {
|
||||||
|
cancelled = true
|
||||||
|
}
|
||||||
|
}, [activeSession?.baseTranslation, pageStartId, pageShells.length])
|
||||||
|
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
let cancelled = false
|
let cancelled = false
|
||||||
|
|||||||
+168
-6
@@ -38,6 +38,8 @@ type ImportedVerse = {
|
|||||||
|
|
||||||
type ImportedStrongLink = {
|
type ImportedStrongLink = {
|
||||||
verse_id: string
|
verse_id: string
|
||||||
|
book_id?: string
|
||||||
|
chapter?: number
|
||||||
position: number
|
position: number
|
||||||
surface: string
|
surface: string
|
||||||
strong: string
|
strong: string
|
||||||
@@ -94,6 +96,12 @@ type ImportedStrongEntry = {
|
|||||||
kjv_definition?: string
|
kjv_definition?: string
|
||||||
}
|
}
|
||||||
|
|
||||||
|
type ImportedStrongIndexEntry = {
|
||||||
|
strong: string
|
||||||
|
verse_ids: string[]
|
||||||
|
surfaces: string[]
|
||||||
|
}
|
||||||
|
|
||||||
type ImportedCommentaryEntry = {
|
type ImportedCommentaryEntry = {
|
||||||
id: string
|
id: string
|
||||||
resource_id: string
|
resource_id: string
|
||||||
@@ -168,6 +176,9 @@ type BrowserStrongData = {
|
|||||||
let browserCatalogCache: Promise<ImportedCatalog> | null = null
|
let browserCatalogCache: Promise<ImportedCatalog> | null = null
|
||||||
const browserLibraryCache = new Map<string, Promise<Verse[]>>()
|
const browserLibraryCache = new Map<string, Promise<Verse[]>>()
|
||||||
const browserStrongDataCache = new Map<string, Promise<BrowserStrongData>>()
|
const browserStrongDataCache = new Map<string, Promise<BrowserStrongData>>()
|
||||||
|
const browserChapterStrongDataCache = new Map<string, Promise<BrowserStrongData>>()
|
||||||
|
const browserStrongSearchIndexCache = new Map<string, Promise<Map<string, string[]>>>()
|
||||||
|
let browserLexiconCache: Promise<Map<string, BrowserLexiconEntry>> | null = null
|
||||||
const browserFootnotesCache = new Map<string, Promise<Map<string, Footnote[]>>>()
|
const browserFootnotesCache = new Map<string, Promise<Map<string, Footnote[]>>>()
|
||||||
const browserStudyNotesCache = new Map<string, Promise<Map<string, Footnote[]>>>()
|
const browserStudyNotesCache = new Map<string, Promise<Map<string, Footnote[]>>>()
|
||||||
const browserChapterFootnotesCache = new Map<string, Promise<Map<string, Footnote[]>>>()
|
const browserChapterFootnotesCache = new Map<string, Promise<Map<string, Footnote[]>>>()
|
||||||
@@ -179,6 +190,7 @@ let browserVersePlacesCache: Promise<Map<string, BiblePlace[]>> | null = null
|
|||||||
let browserRoutesCache: Promise<BibleRoute[]> | null = null
|
let browserRoutesCache: Promise<BibleRoute[]> | null = null
|
||||||
let browserAudioPlaylistCache: Promise<Map<string, ImportedAudioChapterPlaylist>> | null = null
|
let browserAudioPlaylistCache: Promise<Map<string, ImportedAudioChapterPlaylist>> | null = null
|
||||||
const browserVerseSourceIds = new Map<number, string>()
|
const browserVerseSourceIds = new Map<number, string>()
|
||||||
|
const browserVerseIdsBySource = new Map<string, number>()
|
||||||
const browserAudioZipCache = new Map<string, Promise<JSZip>>()
|
const browserAudioZipCache = new Map<string, Promise<JSZip>>()
|
||||||
const browserAudioObjectUrlCache = new Map<string, Promise<string>>()
|
const browserAudioObjectUrlCache = new Map<string, Promise<string>>()
|
||||||
|
|
||||||
@@ -329,20 +341,48 @@ async function loadBrowserVersesForTranslation(translationId = defaultTranslatio
|
|||||||
}
|
}
|
||||||
|
|
||||||
const verses = await browserLibraryCache.get(translationId)!
|
const verses = await browserLibraryCache.get(translationId)!
|
||||||
const strongData = await loadBrowserStrongData(translationId).catch((error) => {
|
|
||||||
console.warn('Unable to load browser Strong data.', error)
|
|
||||||
return null
|
|
||||||
})
|
|
||||||
const tagsByVerse = readBrowserTags()
|
const tagsByVerse = readBrowserTags()
|
||||||
return verses.map((verse) => ({
|
return verses.map((verse) => ({
|
||||||
...verse,
|
...verse,
|
||||||
strongs: strongData?.linksByVerse.get(browserVerseKey(verse)) ?? verse.strongs,
|
|
||||||
footnotes: verse.footnotes ?? [],
|
footnotes: verse.footnotes ?? [],
|
||||||
studyNotes: verse.studyNotes ?? [],
|
studyNotes: verse.studyNotes ?? [],
|
||||||
tags: tagsByVerse[verse.id] ?? [],
|
tags: tagsByVerse[verse.id] ?? [],
|
||||||
}))
|
}))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async function hydrateBrowserVerses(translationId: string, verses: Verse[]) {
|
||||||
|
const chapterKeys = [
|
||||||
|
...new Set(
|
||||||
|
verses
|
||||||
|
.map((verse) => `${bookIdFromVerse(verse)}.${verse.chapter}`)
|
||||||
|
.filter((key) => !key.startsWith('unknown.')),
|
||||||
|
),
|
||||||
|
]
|
||||||
|
const strongDataMaps = await Promise.all(
|
||||||
|
chapterKeys.map((chapterKey) =>
|
||||||
|
loadBrowserChapterStrongData(translationId, chapterKey).catch((error) => {
|
||||||
|
console.warn(`Unable to load browser Strong data for ${chapterKey}.`, error)
|
||||||
|
return { linksByVerse: new Map<string, StrongLink[]>() }
|
||||||
|
}),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
const tagsByVerse = readBrowserTags()
|
||||||
|
|
||||||
|
return verses.map((verse) => {
|
||||||
|
const verseKey = browserVerseKey(verse)
|
||||||
|
const strongs =
|
||||||
|
strongDataMaps.find((data) => data.linksByVerse.has(verseKey))?.linksByVerse.get(verseKey) ??
|
||||||
|
verse.strongs
|
||||||
|
return {
|
||||||
|
...verse,
|
||||||
|
strongs,
|
||||||
|
footnotes: verse.footnotes ?? [],
|
||||||
|
studyNotes: verse.studyNotes ?? [],
|
||||||
|
tags: tagsByVerse[verse.id] ?? [],
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
function allBrowserTags() {
|
function allBrowserTags() {
|
||||||
const unique = new Map<string, Tag>()
|
const unique = new Map<string, Tag>()
|
||||||
for (const tags of Object.values(readBrowserTags())) {
|
for (const tags of Object.values(readBrowserTags())) {
|
||||||
@@ -375,6 +415,23 @@ async function getBrowserLibraryState(translationId = defaultTranslationId): Pro
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async function getBrowserPageVerses(
|
||||||
|
translationId: string,
|
||||||
|
pageStartId: number,
|
||||||
|
pageSize: number,
|
||||||
|
): Promise<Verse[]> {
|
||||||
|
const verses = await loadBrowserVersesForTranslation(translationId)
|
||||||
|
const startIndex = verses.findIndex((verse) => verse.id === pageStartId)
|
||||||
|
if (startIndex < 0) return []
|
||||||
|
|
||||||
|
const book = verses[startIndex].book
|
||||||
|
const pageVerses = verses
|
||||||
|
.slice(startIndex, startIndex + pageSize)
|
||||||
|
.filter((verse) => verse.book === book)
|
||||||
|
|
||||||
|
return hydrateBrowserVerses(translationId, pageVerses)
|
||||||
|
}
|
||||||
|
|
||||||
async function getBrowserTranslationNotesForVerses(
|
async function getBrowserTranslationNotesForVerses(
|
||||||
translationId: string,
|
translationId: string,
|
||||||
verses: Verse[],
|
verses: Verse[],
|
||||||
@@ -407,6 +464,14 @@ async function getBrowserTranslationNotesForVerses(
|
|||||||
}
|
}
|
||||||
|
|
||||||
async function searchBrowserVerses(query: string, translationId = defaultTranslationId): Promise<Verse[]> {
|
async function searchBrowserVerses(query: string, translationId = defaultTranslationId): Promise<Verse[]> {
|
||||||
|
if (/^[gh]\d+$/i.test(query.trim())) {
|
||||||
|
const verses = await loadBrowserVersesForTranslation(translationId)
|
||||||
|
const strongIndex = await loadBrowserStrongSearchIndex(translationId)
|
||||||
|
const sourceIds = strongIndex.get(query.trim().toUpperCase()) ?? []
|
||||||
|
const verseIds = new Set(sourceIds.map((sourceId) => browserVerseIdsBySource.get(sourceId)))
|
||||||
|
return verses.filter((verse) => verseIds.has(verse.id))
|
||||||
|
}
|
||||||
|
|
||||||
const searchIndex = await loadBrowserSearchIndex(translationId)
|
const searchIndex = await loadBrowserSearchIndex(translationId)
|
||||||
return searchIndex.search(query)
|
return searchIndex.search(query)
|
||||||
}
|
}
|
||||||
@@ -552,6 +617,7 @@ const emptyMapContext: MapContext = {
|
|||||||
const browserStudyDataClient: StudyDataClient = {
|
const browserStudyDataClient: StudyDataClient = {
|
||||||
getTranslations: getBrowserTranslations,
|
getTranslations: getBrowserTranslations,
|
||||||
getLibraryState: getBrowserLibraryState,
|
getLibraryState: getBrowserLibraryState,
|
||||||
|
getPageVerses: getBrowserPageVerses,
|
||||||
getTranslationNotesForVerses: getBrowserTranslationNotesForVerses,
|
getTranslationNotesForVerses: getBrowserTranslationNotesForVerses,
|
||||||
searchVerses: searchBrowserVerses,
|
searchVerses: searchBrowserVerses,
|
||||||
addTagToVerse: addBrowserTagToVerse,
|
addTagToVerse: addBrowserTagToVerse,
|
||||||
@@ -565,6 +631,13 @@ const browserStudyDataClient: StudyDataClient = {
|
|||||||
const tauriStudyDataClient: StudyDataClient = {
|
const tauriStudyDataClient: StudyDataClient = {
|
||||||
getTranslations: async () => [],
|
getTranslations: async () => [],
|
||||||
getLibraryState: () => invoke<LibraryState>('get_library_state'),
|
getLibraryState: () => invoke<LibraryState>('get_library_state'),
|
||||||
|
getPageVerses: async (_translationId, pageStartId, pageSize) => {
|
||||||
|
const state = await invoke<LibraryState>('get_library_state')
|
||||||
|
const startIndex = state.verses.findIndex((verse) => verse.id === pageStartId)
|
||||||
|
if (startIndex < 0) return []
|
||||||
|
const book = state.verses[startIndex].book
|
||||||
|
return state.verses.slice(startIndex, startIndex + pageSize).filter((verse) => verse.book === book)
|
||||||
|
},
|
||||||
getTranslationNotesForVerses: async () => new Map(),
|
getTranslationNotesForVerses: async () => new Map(),
|
||||||
searchVerses: (query) => invoke<Verse[]>('search_verses', { query }),
|
searchVerses: (query) => invoke<Verse[]>('search_verses', { query }),
|
||||||
addTagToVerse: (verseId, label, color) => invoke<Verse>('add_tag_to_verse', { verseId, label, color }),
|
addTagToVerse: (verseId, label, color) => invoke<Verse>('add_tag_to_verse', { verseId, label, color }),
|
||||||
@@ -589,6 +662,14 @@ export function getLibraryState(translationId = defaultTranslationId): Promise<L
|
|||||||
return getStudyDataClient().getLibraryState(translationId)
|
return getStudyDataClient().getLibraryState(translationId)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
export function getPageVerses(
|
||||||
|
translationId: string,
|
||||||
|
pageStartId: number,
|
||||||
|
pageSize: number,
|
||||||
|
): Promise<Verse[]> {
|
||||||
|
return getStudyDataClient().getPageVerses(translationId, pageStartId, pageSize)
|
||||||
|
}
|
||||||
|
|
||||||
export function getTranslationNotesForVerses(
|
export function getTranslationNotesForVerses(
|
||||||
translationId: string,
|
translationId: string,
|
||||||
verses: Verse[],
|
verses: Verse[],
|
||||||
@@ -648,15 +729,42 @@ async function loadBrowserSearchIndex(translationId = defaultTranslationId) {
|
|||||||
return browserSearchIndexCache.get(translationId)!
|
return browserSearchIndexCache.get(translationId)!
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async function loadBrowserStrongSearchIndex(translationId = defaultTranslationId) {
|
||||||
|
if (!browserStrongSearchIndexCache.has(translationId)) {
|
||||||
|
browserStrongSearchIndexCache.set(
|
||||||
|
translationId,
|
||||||
|
fetchJsonl<ImportedStrongIndexEntry>(`${libreBibleDevRoot}/${translationId}/strongs-index.jsonl`)
|
||||||
|
.then((entries) => new Map(entries.map((entry) => [entry.strong.toUpperCase(), entry.verse_ids])))
|
||||||
|
.catch(async () => {
|
||||||
|
const strongData = await loadBrowserStrongData(translationId)
|
||||||
|
const index = new Map<string, string[]>()
|
||||||
|
for (const [verseId, links] of strongData.linksByVerse) {
|
||||||
|
for (const link of links) {
|
||||||
|
const key = link.strong.toUpperCase()
|
||||||
|
const current = index.get(key) ?? []
|
||||||
|
if (!current.includes(verseId)) current.push(verseId)
|
||||||
|
index.set(key, current)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return index
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
return browserStrongSearchIndexCache.get(translationId)!
|
||||||
|
}
|
||||||
|
|
||||||
async function loadLibreBibleDevVerses(translationId = defaultTranslationId): Promise<Verse[]> {
|
async function loadLibreBibleDevVerses(translationId = defaultTranslationId): Promise<Verse[]> {
|
||||||
const importedVerses = await fetchJsonl<ImportedVerse>(
|
const importedVerses = await fetchJsonl<ImportedVerse>(
|
||||||
`${libreBibleDevRoot}/${translationId}/verses.jsonl`,
|
`${libreBibleDevRoot}/${translationId}/verses.jsonl`,
|
||||||
)
|
)
|
||||||
|
|
||||||
browserVerseSourceIds.clear()
|
browserVerseSourceIds.clear()
|
||||||
|
browserVerseIdsBySource.clear()
|
||||||
return importedVerses.map((verse, index) => {
|
return importedVerses.map((verse, index) => {
|
||||||
const id = index + 1
|
const id = index + 1
|
||||||
browserVerseSourceIds.set(id, verse.id)
|
browserVerseSourceIds.set(id, verse.id)
|
||||||
|
browserVerseIdsBySource.set(verse.id, id)
|
||||||
return {
|
return {
|
||||||
id,
|
id,
|
||||||
book: verse.book,
|
book: verse.book,
|
||||||
@@ -982,6 +1090,42 @@ function mapImportedNotes(importedFootnotes: ImportedFootnote[]): Map<string, Fo
|
|||||||
return footnotes
|
return footnotes
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async function loadBrowserChapterStrongData(
|
||||||
|
translationId: string,
|
||||||
|
chapterKey: string,
|
||||||
|
): Promise<BrowserStrongData> {
|
||||||
|
const cacheKey = `${translationId}:${chapterKey}`
|
||||||
|
if (!browserChapterStrongDataCache.has(cacheKey)) {
|
||||||
|
browserChapterStrongDataCache.set(
|
||||||
|
cacheKey,
|
||||||
|
loadLibreBibleDevChapterStrongData(translationId, chapterKey).catch(async () => {
|
||||||
|
const fullStrongData = await loadBrowserStrongData(translationId)
|
||||||
|
const linksByVerse = new Map<string, StrongLink[]>()
|
||||||
|
for (const [verseId, links] of fullStrongData.linksByVerse) {
|
||||||
|
if (verseId.startsWith(`${chapterKey}.`)) linksByVerse.set(verseId, links)
|
||||||
|
}
|
||||||
|
return { linksByVerse }
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
return browserChapterStrongDataCache.get(cacheKey)!
|
||||||
|
}
|
||||||
|
|
||||||
|
async function loadLibreBibleDevChapterStrongData(
|
||||||
|
translationId: string,
|
||||||
|
chapterKey: string,
|
||||||
|
): Promise<BrowserStrongData> {
|
||||||
|
const [importedLinks, lexicon] = await Promise.all([
|
||||||
|
fetchJsonl<ImportedStrongLink>(
|
||||||
|
`${libreBibleDevRoot}/${translationId}/strongs-by-chapter/${chapterKey}.strongs.jsonl`,
|
||||||
|
),
|
||||||
|
loadBrowserLexicon(),
|
||||||
|
])
|
||||||
|
|
||||||
|
return mapStrongLinks(importedLinks, lexicon)
|
||||||
|
}
|
||||||
|
|
||||||
async function loadBrowserStrongData(translationId = defaultTranslationId): Promise<BrowserStrongData> {
|
async function loadBrowserStrongData(translationId = defaultTranslationId): Promise<BrowserStrongData> {
|
||||||
if (!browserStrongDataCache.has(translationId)) {
|
if (!browserStrongDataCache.has(translationId)) {
|
||||||
browserStrongDataCache.set(translationId, loadLibreBibleDevStrongData(translationId))
|
browserStrongDataCache.set(translationId, loadLibreBibleDevStrongData(translationId))
|
||||||
@@ -996,7 +1140,19 @@ async function loadLibreBibleDevStrongData(translationId = defaultTranslationId)
|
|||||||
fetchJsonl<ImportedStrongEntry>(`${libreBibleDevRoot}/strongs-open-scriptures/entries.jsonl`),
|
fetchJsonl<ImportedStrongEntry>(`${libreBibleDevRoot}/strongs-open-scriptures/entries.jsonl`),
|
||||||
])
|
])
|
||||||
|
|
||||||
const lexicon = new Map<string, BrowserLexiconEntry>(
|
return mapStrongLinks(importedLinks, mapBrowserLexicon(importedEntries))
|
||||||
|
}
|
||||||
|
|
||||||
|
async function loadBrowserLexicon(): Promise<Map<string, BrowserLexiconEntry>> {
|
||||||
|
browserLexiconCache ??= fetchJsonl<ImportedStrongEntry>(
|
||||||
|
`${libreBibleDevRoot}/strongs-open-scriptures/entries.jsonl`,
|
||||||
|
).then(mapBrowserLexicon)
|
||||||
|
|
||||||
|
return browserLexiconCache
|
||||||
|
}
|
||||||
|
|
||||||
|
function mapBrowserLexicon(importedEntries: ImportedStrongEntry[]) {
|
||||||
|
return new Map<string, BrowserLexiconEntry>(
|
||||||
importedEntries.map((entry) => [
|
importedEntries.map((entry) => [
|
||||||
entry.strongs_number,
|
entry.strongs_number,
|
||||||
{
|
{
|
||||||
@@ -1007,6 +1163,12 @@ async function loadLibreBibleDevStrongData(translationId = defaultTranslationId)
|
|||||||
},
|
},
|
||||||
]),
|
]),
|
||||||
)
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
function mapStrongLinks(
|
||||||
|
importedLinks: ImportedStrongLink[],
|
||||||
|
lexicon: Map<string, BrowserLexiconEntry>,
|
||||||
|
): BrowserStrongData {
|
||||||
const linksByVerse = new Map<string, StrongLink[]>()
|
const linksByVerse = new Map<string, StrongLink[]>()
|
||||||
|
|
||||||
for (const link of importedLinks) {
|
for (const link of importedLinks) {
|
||||||
|
|||||||
@@ -15,6 +15,7 @@ export type TranslationNotesByVerse = Map<string, { footnotes: Footnote[]; study
|
|||||||
export type StudyDataClient = {
|
export type StudyDataClient = {
|
||||||
getTranslations(): Promise<TranslationResource[]>
|
getTranslations(): Promise<TranslationResource[]>
|
||||||
getLibraryState(translationId?: string): Promise<LibraryState>
|
getLibraryState(translationId?: string): Promise<LibraryState>
|
||||||
|
getPageVerses(translationId: string, pageStartId: number, pageSize: number): Promise<Verse[]>
|
||||||
getTranslationNotesForVerses(translationId: string, verses: Verse[]): Promise<TranslationNotesByVerse>
|
getTranslationNotesForVerses(translationId: string, verses: Verse[]): Promise<TranslationNotesByVerse>
|
||||||
searchVerses(query: string, translationId?: string): Promise<Verse[]>
|
searchVerses(query: string, translationId?: string): Promise<Verse[]>
|
||||||
addTagToVerse(verseId: number, label: string, color: string, translationId?: string): Promise<Verse>
|
addTagToVerse(verseId: number, label: string, color: string, translationId?: string): Promise<Verse>
|
||||||
|
|||||||
Reference in New Issue
Block a user