Skip to content

Upsert SourceAttribution field into database #261

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Upsert SourceAttribution field into database #261

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7be1900
Upsert document sources as a DB query
chanmioh Jan 26, 2023
e2fb03f
Merge remote-tracking branch 'origin' into upsert-doc-sources
chanmioh Jan 26, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions sqlx-data.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,19 @@
},
"query": "insert into morpheme_gloss (document_id, gloss, example_shape, tag_id)\n values ($1, $2, $3, $4)\non conflict (coalesce(document_id, uuid_nil()), gloss)\n do update set example_shape = excluded.example_shape,\n tag_id = excluded.tag_id\nreturning id\n"
},
"16a151e6b4deeaf35fa60b209154e8ce8e1e7cbceac64fcc52b4422ced7ce236": {
"describe": {
"columns": [],
"nullable": [],
"parameters": {
"Left": [
"TextArray",
"TextArray"
]
}
},
"query": "insert into document_source (name, url)\nselect\n input_data.name,\n input_data.url\nfrom\n unnest(\n $1::text[], $2::text[]\n ) as input_data(name, url)\n-- If this document already has this source, move on.\non conflict do nothing"
},
"1dcec22261dae61c4404af7c630eb24e3cd2acc5b4717bcc9706f14c6dde7efc": {
"describe": {
"columns": [
Expand Down
10 changes: 10 additions & 0 deletions types/queries/upsert_document_sources.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
insert into document_source (name, url)
select
input_data.name,
input_data.url
from
unnest(
$1::text[], $2::text[]
) as input_data(name, url)
-- If this document already has this source, move on.
on conflict do nothing
16 changes: 16 additions & 0 deletions types/src/database_sql.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -580,6 +580,22 @@ impl Database {
.await?;
}

// Upsert document sources based on the SourceAttribution in a document's metadata.
{
let (name, url): (Vec<_>, Vec<_>) = meta
.sources
.iter()
.map(|source| (&*source.name, &*source.link))
.multiunzip();
query_file!(
"queries/upsert_document_sources.sql",
Comment on lines +588 to +591
Copy link
Collaborator

@loafofpiecrust loafofpiecrust Feb 9, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we've got a logic mismatch here -- might need schema changes. So a source link for a particular document points to the specific record in the source archive. What we want is a single row in document_source for the Yale Beinecke library and then a document_source_citation row for every document that all point to the same document_source row. Now document_source_citation doesn't have a link/url field right now, I think it needs one added. Does that make sense?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I kind of understand, but I'm wondering why does the document_source_citation need a link/url field? I'm looking at the docs for the current schema and I see document_source_citation has source_id to link to the corresponding document_source which also has the link/url field you're talking about I assume.

image

&*name as _,
&*url as _
)
.execute(&mut tx)
.await?;
}

tx.commit().await?;

Ok(DocumentId(document_uuid))
Expand Down
4 changes: 2 additions & 2 deletions website/src/graphql/wordpress/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6745,6 +6745,8 @@ export enum TimezoneEnum {
AmericaChicago = "AMERICA_CHICAGO",
/** Chihuahua */
AmericaChihuahua = "AMERICA_CHIHUAHUA",
/** Ciudad Juarez */
AmericaCiudadJuarez = "AMERICA_CIUDAD_JUAREZ",
/** Costa Rica */
AmericaCostaRica = "AMERICA_COSTA_RICA",
/** Creston */
Expand Down Expand Up @@ -6887,8 +6889,6 @@ export enum TimezoneEnum {
AmericaOjinaga = "AMERICA_OJINAGA",
/** Panama */
AmericaPanama = "AMERICA_PANAMA",
/** Pangnirtung */
AmericaPangnirtung = "AMERICA_PANGNIRTUNG",
/** Paramaribo */
AmericaParamaribo = "AMERICA_PARAMARIBO",
/** Phoenix */
Expand Down