PKP/OJS multi-source journal enrichment — simple version

This document performs Full-Cohort Journal Enrichment. It starts with every row in the PKP/OJS list. It looks for OpenAlex and Crossref records that share an exact, valid ISSN with each row. It does not use a title or publisher name to make a match.

This is Exploratory Analysis. It prepares evidence for review. It does not decide that two records describe the same journal when the evidence is uncertain.

The final Generated Artifact is a Parquet table with all input rows and all available top-level OpenAlex and Crossref fields. A second Parquet file contains the fixed ten-row Journal Enrichment Sample. Complex source values stay in JSON text so that the analysis does not discard them.

How to read the match result

An ISSN is the main journal identifier in this analysis. Identifier Availability tells us whether an input row has at least one valid ISSN. Each source can then give one of four match results:

Result	Plain-language meaning
not_attempted	The input row has no valid ISSN, so an exact-ISSN search is not possible.
unmatched	The row has a valid ISSN, but the source has no record with that ISSN.
unique	The source has one record with the ISSN. Its fields are expanded into columns.
ambiguous	The source has more than one record with the ISSN. All candidates are kept; the code does not select a winner.

The code can reuse saved data to avoid expensive API requests. A cache is only a saved copy of an earlier download or calculation. The complete download code remains in this document. Set reuse_shared_cache to FALSE and delete this document’s source-cache to request a fresh source snapshot. Missing OpenAlex batches require OPENALEX_API_KEY. A missing Crossref snapshot requires CROSSREF_MAILTO. Delete openalex-expanded.rds and crossref-expanded.rds after a change to the matching or field-expansion code.

Packages and paths

This block prepares the tools and file locations. All new files go to this analysis’s own artifact directory. reuse_shared_cache permits read-only reuse of earlier source data. The code still writes new data only to this analysis’s directory.

valid_pkp_file() compares the input file with the expected content fingerprint. This check makes sure that the analysis uses the intended PKP V7 release, not a different file with the same name.

library(dplyr)

Attaching package: 'dplyr'

The following objects are masked from 'package:stats':

    filter, lag

The following objects are masked from 'package:base':

    intersect, setdiff, setequal, union

library(httr2)
library(purrr)
library(readr)
library(tibble)

document_path <- knitr::current_input(dir = TRUE)
document_dir <- if (nzchar(document_path)) {
  dirname(normalizePath(document_path))
} else {
  normalizePath(getwd())
}
domain_dir <- normalizePath(file.path(document_dir, "..", ".."))
project_env_path <- file.path(domain_dir, "..", "..", ".env")
if (file.exists(project_env_path)) readRenviron(project_env_path)

artifact_dir <- file.path(
  domain_dir,
  "artifacts",
  "ojs_journal_enrichment_simple_ver"
)
shared_artifact_dir <- file.path(
  domain_dir,
  "artifacts",
  "ojs_journal_enrichment"
)
reuse_shared_cache <- TRUE
cache_dir <- file.path(artifact_dir, "source-cache")
shared_cache_dir <- file.path(shared_artifact_dir, "full-v7")
input_dir <- file.path(artifact_dir, "input")
input_path <- file.path(input_dir, "beacon.csv")
shared_input_path <- file.path(shared_artifact_dir, "pkp-v7", "beacon.csv")
openalex_batch_dir <- file.path(cache_dir, "openalex-batches")
shared_openalex_batch_dir <- file.path(shared_cache_dir, "openalex-batches")
crossref_cache_path <- file.path(cache_dir, "crossref-journals.rds")
shared_crossref_cache_path <- file.path(
  shared_cache_dir,
  "crossref-journals.rds"
)
output_path <- file.path(
  artifact_dir,
  "pkp-ojs-multisource-enriched-simple_ver.parquet"
)
sample_output_path <- file.path(
  artifact_dir,
  "pkp-ojs-multisource-enriched-simple_ver-sample.parquet"
)
openalex_expanded_path <- file.path(artifact_dir, "openalex-expanded.rds")
crossref_expanded_path <- file.path(artifact_dir, "crossref-expanded.rds")

dir.create(input_dir, recursive = TRUE, showWarnings = FALSE)
dir.create(openalex_batch_dir, recursive = TRUE, showWarnings = FALSE)

pkp_url <- paste0(
  "https://dataverse.harvard.edu/api/access/datafile/",
  "14084919?format=original"
)
pkp_md5 <- "3a4ad8ae1ebfcc2b991aaf55b2d82c92"

valid_pkp_file <- function(path) {
  file.exists(path) && unname(tools::md5sum(path)) == pkp_md5
}
if (!valid_pkp_file(input_path)) {
  if (valid_pkp_file(shared_input_path)) {
    invisible(file.copy(shared_input_path, input_path, overwrite = TRUE))
  } else {
    download.file(pkp_url, input_path, mode = "wb")
  }
  stopifnot(valid_pkp_file(input_path))
}

ISSN preparation

These functions prepare ISSNs before matching.

• normalize_issn() removes spaces and punctuation. It also checks the ISSN structure and check digit.
• parse_issns() returns every valid ISSN in one input cell. It returns an empty set when the cell is missing or contains no valid ISSN.
• openalex_issns() and crossref_issns() apply the same rule to source records.

This common preparation is important. It makes both sources use the same exact identifier rule.

normalize_issn <- function(value) {
  if (length(value) != 1L || is.na(value)) return(NA_character_)
  compact <- gsub("[^0-9X]", "", toupper(trimws(as.character(value))))
  if (!grepl("^[0-9]{7}[0-9X]$", compact)) return(NA_character_)

  digits <- as.integer(strsplit(substr(compact, 1, 7), "")[[1]])
  check_value <- (11 - sum(digits * 8:2) %% 11) %% 11
  expected <- if (check_value == 10) "X" else as.character(check_value)
  if (substr(compact, 8, 8) != expected) return(NA_character_)

  paste0(substr(compact, 1, 4), "-", substr(compact, 5, 8))
}

normalize_issn_set <- function(values) {
  normalized <- vapply(values, normalize_issn, character(1))
  unique(normalized[!is.na(normalized)])
}

is_missing_value <- function(value) {
  is.na(value) |
    !nzchar(trimws(value)) |
    toupper(trimws(value)) == "NA"
}

parse_issns <- function(value) {
  if (is_missing_value(value)) return(character())
  tokens <- unlist(strsplit(value, "[[:space:],;|]+"))
  normalize_issn_set(tokens[nzchar(tokens)])
}

openalex_issns <- function(source) {
  normalize_issn_set(unlist(source$issn, use.names = FALSE))
}

crossref_issns <- function(journal) {
  normalize_issn_set(journal$ISSN)
}

Source retrieval and caching

This block defines how to obtain source data when a usable cache is not available. read_cache() returns a saved R object when it can read one. Otherwise, the relevant fetch_*() function calls the source API.

OpenAlex accepts the input ISSNs in groups of 100. Crossref supplies its journal directory in pages of 1,000 records. A short timeout prevents a stalled request from waiting forever. Each request can make at most four attempts. The cursor checks stop the code if an API repeats the same page position.

read_cache <- function(path) {
  if (!file.exists(path)) return(NULL)
  tryCatch(readRDS(path), error = \(error) NULL)
}

fetch_openalex_page <- function(issns, cursor) {
  api_key <- Sys.getenv("OPENALEX_API_KEY")
  if (!nzchar(api_key)) {
    stop("Set OPENALEX_API_KEY to retrieve missing OpenAlex batches.")
  }

  body <- request("https://api.openalex.org/sources") |>
    req_url_query(
      filter = paste0("issn:", paste(issns, collapse = "|")),
      per_page = 100,
      cursor = cursor,
      api_key = api_key
    ) |>
    req_timeout(seconds = 30) |>
    req_retry(
      max_tries = 4L,
      retry_on_failure = TRUE
    ) |>
    req_perform() |>
    resp_body_json(simplifyVector = FALSE)

  stopifnot(
    is.list(body$results),
    is.list(body$meta),
    length(body$meta$count) == 1L,
    is.null(body$meta$next_cursor) ||
      length(body$meta$next_cursor) == 1L
  )
  body
}

fetch_openalex_batch <- function(issns) {
  stopifnot(length(issns) <= 100L)
  cursor <- "*"
  sources <- list()
  expected_count <- NULL

  repeat {
    page <- fetch_openalex_page(issns, cursor)
    if (is.null(expected_count)) expected_count <- as.integer(page$meta$count)
    sources <- c(sources, page$results)
    next_cursor <- page$meta$next_cursor
    if (identical(next_cursor, cursor)) stop("OpenAlex cursor did not advance.")
    cursor <- next_cursor
    if (is.null(cursor) || !nzchar(cursor)) break
  }

  stopifnot(length(sources) == expected_count)
  sources
}

fetch_crossref_page <- function(cursor) {
  mailto <- Sys.getenv("CROSSREF_MAILTO")
  if (!grepl("^[^@[:space:]]+@[^@[:space:]]+$", mailto)) {
    stop("Set CROSSREF_MAILTO to retrieve the missing Crossref directory.")
  }

  body <- request("https://api.crossref.org/journals") |>
    req_url_query(rows = 1000, cursor = cursor, mailto = mailto) |>
    req_user_agent(
      paste(
        "InvisibleResearch journal enrichment;",
        "https://github.com/YannJY02/InvisibleResearch"
      )
    ) |>
    req_timeout(seconds = 60) |>
    req_retry(
      max_tries = 4L,
      retry_on_failure = TRUE
    ) |>
    req_perform() |>
    resp_body_json(simplifyVector = FALSE) |>
    pluck("message")

  stopifnot(is.list(body$items))
  body
}

fetch_crossref_directory <- function() {
  cursor <- "*"
  journals <- list()

  repeat {
    page <- fetch_crossref_page(cursor)
    journals <- c(journals, page$items)
    if (length(page$items) < 1000L) break

    next_cursor <- page[["next-cursor"]]
    if (is.null(next_cursor) || !nzchar(next_cursor)) {
      stop("Crossref returned a full page without a next cursor.")
    }
    if (identical(next_cursor, cursor)) stop("Crossref cursor did not advance.")
    cursor <- next_cursor
    Sys.sleep(0.2)
  }

  journals
}

Load the complete PKP V7 cohort

This block reads the PKP V7 list. Each row is one OJS record. All 19 source fields are read as text so that R does not silently change an identifier, date, or special value.

row_identity() uses the OAI address, repository name, and OAI set identifier together as a stable row identity. The two checks confirm that the fields needed for matching exist and that every input row is an OJS row. input_issns then stores the valid ISSNs for each row. tokens is the distinct ISSN list sent to the source APIs.

pkp_rows <- read_csv(
  input_path,
  col_types = cols(.default = col_character()),
  na = character(),
  show_col_types = FALSE
)

row_identity <- function(data) {
  paste(data$oai_url, data$repository_name, data$set_spec, sep = "\r")
}

stopifnot(
  all(c("issn", "oai_url", "repository_name", "set_spec") %in% names(pkp_rows)),
  all(tolower(pkp_rows$application) == "ojs")
)

input_issns <- map(pkp_rows$issn, parse_issns)
tokens <- sort(unique(unlist(input_issns)))

Matching and field expansion helpers

These functions turn exact ISSN evidence into result columns.

• index_candidates() records which source objects contain each ISSN.
• match_candidates() compares one PKP row with that index and assigns one of the four match results defined above.
• expand_source_fields() creates one result column for every top-level source field. A simple source value keeps its number, true/false, or text type. A nested value becomes JSON text.

For a unique match, the source fields appear directly in the expanded columns. For an ambiguous match, the code leaves those single-record columns empty and stores every complete candidate in *_candidates__json. This keeps the evidence without making an unsupported choice.

index_candidates <- function(candidate_issns) {
  index <- new.env(hash = TRUE, parent = emptyenv())
  for (position in seq_along(candidate_issns)) {
    for (issn in candidate_issns[[position]]) {
      existing <- get0(
        issn,
        index,
        inherits = FALSE,
        ifnotfound = integer()
      )
      assign(issn, c(existing, position), index)
    }
  }
  index
}

match_candidates <- function(issns, index, candidate_issns) {
  positions <- unique(unlist(map(issns, \(issn) {
    get0(issn, index, inherits = FALSE, ifnotfound = integer())
  })))
  matched_issns <- base::intersect(
    issns,
    unique(unlist(candidate_issns[positions]))
  )
  status <- case_when(
    length(issns) == 0L ~ "not_attempted",
    length(positions) == 0L ~ "unmatched",
    length(positions) == 1L ~ "unique",
    TRUE ~ "ambiguous"
  )
  list(
    status = status,
    matched_issns = matched_issns,
    positions = positions
  )
}

serialize_json <- function(value) {
  as.character(jsonlite::toJSON(
    value,
    auto_unbox = TRUE,
    null = "null",
    na = "null",
    digits = NA
  ))
}

dataframe_value <- function(value) {
  if (is.null(value)) return(NA_character_)
  if (is.atomic(value) && length(value) == 1L) {
    return(as.character(value))
  }
  serialize_json(value)
}

scalar_column <- function(values) {
  present <- keep(values, \(value) !is.null(value) && length(value) == 1L)
  types <- unique(map_chr(present, typeof))
  missing <- \(value) is.null(value) || length(value) == 0L

  if (length(types) > 0L && all(types == "logical")) {
    return(map_lgl(values, \(value) if (missing(value)) NA else value))
  }
  if (length(types) > 0L && all(types == "integer")) {
    return(map_int(values, \(value) if (missing(value)) NA_integer_ else value))
  }
  if (length(types) > 0L && all(types %in% c("integer", "double"))) {
    return(map_dbl(values, \(value) if (missing(value)) NA_real_ else value))
  }
  map_chr(values, \(value) {
    if (missing(value)) NA_character_ else as.character(value)
  })
}

expand_source_fields <- function(matches, candidates, prefix) {
  fields <- sort(unique(unlist(map(candidates, names))))
  single_positions <- map_int(matches, \(match) {
    if (length(match$positions) == 1L) match$positions else NA_integer_
  })

  # ponytail: JSON conversion is single-process for this one-off; use Arrow
  # nested columns only if repeated full rebuilds make serialization a problem.
  columns <- set_names(
    map(fields, \(field) {
      values <- map(candidates, \(candidate) candidate[[field]])
      scalar_field <- all(map_lgl(values, \(value) {
        is.null(value) || (is.atomic(value) && length(value) <= 1L)
      }))
      candidate_values <- if (scalar_field) {
        scalar_column(values)
      } else {
        map_chr(values, dataframe_value)
      }
      candidate_values[single_positions]
    }),
    paste(prefix, fields, sep = "_")
  )
  columns[[paste0(prefix, "_candidates__json")]] <- map_chr(
    matches,
    \(match) {
      if (length(match$positions) > 1L) {
        serialize_json(candidates[match$positions])
      } else {
        NA_character_
      }
    }
  )
  as_tibble(columns, .name_repair = "check_unique")
}

OpenAlex enrichment

This block processes OpenAlex in four steps. It reuses or retrieves each 100-ISSN batch. It removes duplicate OpenAlex source IDs. It matches each PKP row by exact ISSN. It then expands all observed OpenAlex fields.

base_result also adds identifier_status, the ISSNs used for the search, the ISSNs that matched, the match result, and the candidate count. The expanded-data cache avoids repeating the most expensive field conversion. Reuse it only when the source cache and the matching code have not changed.

token_batches <- split(tokens, ceiling(seq_along(tokens) / 100L))
openalex_retrieved <- FALSE
openalex_batches <- imap(token_batches, \(batch, batch_number) {
  checkpoint_path <- file.path(
    openalex_batch_dir,
    sprintf("batch-%04d.rds", as.integer(batch_number))
  )
  checkpoint <- read_cache(checkpoint_path)
  reusable <- is.list(checkpoint) &&
    identical(checkpoint$issns, batch) &&
    is.list(checkpoint$sources)

  if (!reusable && reuse_shared_cache) {
    shared_checkpoint_path <- file.path(
      shared_openalex_batch_dir,
      basename(checkpoint_path)
    )
    checkpoint <- read_cache(shared_checkpoint_path)
    reusable <- is.list(checkpoint) &&
      identical(checkpoint$issns, batch) &&
      is.list(checkpoint$sources)
    if (reusable) checkpoint_path <- shared_checkpoint_path
  }

  if (!reusable) {
    openalex_retrieved <<- TRUE
    checkpoint <- list(
      issns = batch,
      sources = fetch_openalex_batch(batch)
    )
    saveRDS(checkpoint, checkpoint_path)
  }
  checkpoint$sources
})

openalex_sources <- unlist(openalex_batches, recursive = FALSE)
openalex_ids <- map_chr(openalex_sources, "id")
keep <- !duplicated(openalex_ids)
openalex_sources <- openalex_sources[keep]
openalex_ids <- openalex_ids[keep]
openalex_issns_by_candidate <- map(openalex_sources, openalex_issns)
relevant <- map_lgl(
  openalex_issns_by_candidate,
  \(issns) any(issns %in% tokens)
)
openalex_sources <- openalex_sources[relevant]
openalex_ids <- openalex_ids[relevant]
openalex_issns_by_candidate <- openalex_issns_by_candidate[relevant]
openalex_index <- index_candidates(openalex_issns_by_candidate)
openalex_matches <- map(
  input_issns,
  match_candidates,
  index = openalex_index,
  candidate_issns = openalex_issns_by_candidate
)

base_result <- pkp_rows |>
  mutate(
    identifier_status = case_when(
      is_missing_value(issn) ~ "missing",
      lengths(input_issns) == 0L ~ "invalid",
      TRUE ~ "valid"
    ),
    openalex_input_issns = map_chr(input_issns, paste, collapse = "|"),
    openalex_matched_issns = map_chr(
      openalex_matches,
      \(match) paste(match$matched_issns, collapse = "|")
    ),
    openalex_match_status = map_chr(openalex_matches, "status"),
    openalex_candidate_count = map_int(
      openalex_matches,
      \(match) length(match$positions)
    ),
    openalex_candidate_ids = map_chr(
      openalex_matches,
      \(match) paste(openalex_ids[match$positions], collapse = "|")
    )
  )

load_expanded <- function(path, source_retrieved) {
  # ponytail: this one-off cache has no version metadata; delete it after
  # changing matching or field-expansion logic.
  if (!reuse_shared_cache || source_retrieved) return(NULL)
  cached <- read_cache(path)
  if (is.list(cached) && is.data.frame(cached$data)) cached <- cached$data
  if (is.data.frame(cached) && nrow(cached) == nrow(pkp_rows)) cached else NULL
}

openalex_data <- load_expanded(openalex_expanded_path, openalex_retrieved)
if (is.null(openalex_data)) {
  openalex_data <- expand_source_fields(
    openalex_matches,
    openalex_sources,
    "openalex"
  )
  saveRDS(openalex_data, openalex_expanded_path)
}
rm(openalex_batches, openalex_index)
gc()

            used   (Mb) gc trigger   (Mb) limit (Mb)  max used   (Mb)
Ncells  62524857 3339.2  144509400 7717.7         NA 144509400 7717.7
Vcells 252579728 1927.1  369639819 2820.2      16384 294458067 2246.6

Crossref enrichment

This block applies the same exact-ISSN method to the Crossref journal directory. It first keeps only Crossref records that contain an ISSN used by the PKP list. It does not merge distinct Crossref objects merely because they have the same ISSN set. Therefore, a shared ISSN can correctly produce an ambiguous result.

The final result joins the original PKP fields, the match evidence, and every expanded OpenAlex and Crossref field. The original row order stays unchanged.

crossref_retrieved <- FALSE
crossref_journals <- read_cache(crossref_cache_path)
if (is.list(crossref_journals$journals)) {
  crossref_journals <- crossref_journals$journals
}
if (!is.list(crossref_journals) && reuse_shared_cache) {
  crossref_journals <- read_cache(shared_crossref_cache_path)
  if (is.list(crossref_journals$journals)) {
    crossref_journals <- crossref_journals$journals
  }
}
if (!is.list(crossref_journals)) {
  crossref_journals <- fetch_crossref_directory()
  saveRDS(crossref_journals, crossref_cache_path)
  crossref_retrieved <- TRUE
}
crossref_issns_by_candidate <- map(crossref_journals, crossref_issns)
relevant <- map_lgl(
  crossref_issns_by_candidate,
  \(issns) any(issns %in% tokens)
)
crossref_journals <- crossref_journals[relevant]
crossref_issns_by_candidate <- crossref_issns_by_candidate[relevant]
crossref_ids <- map_chr(
  crossref_issns_by_candidate,
  \(issns) paste(sort(issns), collapse = "|")
)
keep <- nzchar(crossref_ids)
crossref_journals <- crossref_journals[keep]
crossref_ids <- crossref_ids[keep]
crossref_issns_by_candidate <- crossref_issns_by_candidate[keep]
crossref_index <- index_candidates(crossref_issns_by_candidate)
crossref_matches <- map(
  input_issns,
  match_candidates,
  index = crossref_index,
  candidate_issns = crossref_issns_by_candidate
)

crossref_data <- load_expanded(crossref_expanded_path, crossref_retrieved)
if (is.null(crossref_data)) {
  crossref_data <- expand_source_fields(
    crossref_matches,
    crossref_journals,
    "crossref"
  )
  saveRDS(crossref_data, crossref_expanded_path)
}

result <- base_result |>
  mutate(
    crossref_input_issns = map_chr(input_issns, paste, collapse = "|"),
    crossref_matched_issns = map_chr(
      crossref_matches,
      \(match) paste(match$matched_issns, collapse = "|")
    ),
    crossref_match_status = map_chr(crossref_matches, "status"),
    crossref_candidate_count = map_int(
      crossref_matches,
      \(match) length(match$positions)
    ),
    crossref_candidate_issn_sets = map_chr(
      crossref_matches,
      \(match) paste(crossref_ids[match$positions], collapse = ";")
    )
  ) |>
  bind_cols(openalex_data, crossref_data)

Parquet outputs and lightweight validation

This block creates the two Generated Artifacts. sample_spec identifies ten specific rows by stable row identity. This Journal Enrichment Sample is not a random or representative sample. It is a visual review tool that shows several match situations and all expanded fields.

Before writing files, the checks confirm three points: the sensitive admin_email field is absent; the input rows and their order are unchanged; and each ambiguous JSON value contains the reported number of candidates.

Arrow writes each Parquet file to a .pending path with Zstandard compression. The code checks the full-table size and column names, and it reads the complete sample back. Only then does it give both files their final names. This short staging step protects an earlier valid result if writing stops partway through.

sample_spec <- tribble(
  ~context_name, ~oai_url, ~repository_name, ~set_spec,
  "Kuwait Journal of Science",
  "https://journalskuwait.org/kjs/index.php/index/oai",
  "Kuwait Journal of Science", "KJS",
  "Cakrawala",
  "https://cakrawalajournal.org/index.php/index/oai",
  "CAKRAWALA", "cakrawala",
  "Sintesa: Jurnal Ilmu Pendidikan",
  "https://sintesa.stkip-arrahmaniyah.ac.id/index.php/index/oai",
  "Sintesa: Jurnal Ilmu Pendidikan STKIP Arrahmaniyah Depok", "sintesa",
  "CARAKA: Jurnal Teologi Biblika dan Praktika",
  "https://ojs.sttibc.ac.id/index.php/index/oai",
  "CARAKA:Jurnal Teologi", "ibc",
  "Jurnal Tata Kelola dan Akuntabilitas Keuangan Negara",
  "https://jurnal.bpk.go.id/index.php/index/oai",
  "Open Journal Systems", "TAKEN",
  "JOEEL (Journal of English Education and Literature)",
  "https://journal.stkippamanetalino.ac.id/index.php/index/oai",
  "Rumah Jurnal STKIP Pamane Talino Ngabang", "bahasa-inggris",
  "Malaysian Journal of Paediatrics and Child Health",
  "https://mpaeds.my/journals/index.php/index/oai",
  "NA", "MJPCH",
  "Jami Scientific Research Quarterly Journal",
  "https://journals.jami.edu.af/index.php/index/oai",
  "Jami University Press", "jsrqj",
  "Journal of Polymer & Composites",
  "https://engineeringjournals.stmjournals.in/index.php/index/oai",
  "Engineering Journals", "JoPC",
  "JURNAL TEKNIK PERTAMBANGAN",
  "https://e-journal.upr.ac.id/index.php/index/oai",
  "Journal Online Universitas Palangka Raya", "JTP"
)
sample_positions <- match(row_identity(sample_spec), row_identity(result))
stopifnot(!anyNA(sample_positions))
sample_result <- result[sample_positions, ]

candidate_json_is_complete <- function(matches, values) {
  map2_lgl(matches, values, \(match, value) {
    candidate_count <- length(match$positions)
    if (candidate_count <= 1L) return(is.na(value))
    parsed <- tryCatch(
      jsonlite::fromJSON(value, simplifyVector = FALSE),
      error = \(error) NULL
    )
    is.list(parsed) && length(parsed) == candidate_count
  })
}

stopifnot(
  !"admin_email" %in% names(result),
  identical(row_identity(result), row_identity(pkp_rows)),
  all(candidate_json_is_complete(
    openalex_matches,
    result$openalex_candidates__json
  )),
  all(candidate_json_is_complete(
    crossref_matches,
    result$crossref_candidates__json
  ))
)

pending_output_path <- paste0(output_path, ".pending")
pending_sample_output_path <- paste0(sample_output_path, ".pending")
arrow::write_parquet(
  as.data.frame(result),
  pending_output_path,
  compression = "zstd"
)
arrow::write_parquet(
  as.data.frame(sample_result),
  pending_sample_output_path,
  compression = "zstd"
)

parquet_info <- nanoparquet::read_parquet_info(pending_output_path)
parquet_schema <- nanoparquet::read_parquet_schema(pending_output_path)
observed_sample <- nanoparquet::read_parquet(pending_sample_output_path)

stopifnot(
  parquet_info$num_rows == nrow(result),
  parquet_info$num_cols == ncol(result),
  identical(
    parquet_schema$name[!is.na(parquet_schema$r_col)],
    names(result)
  ),
  isTRUE(all.equal(
    as.data.frame(observed_sample),
    as.data.frame(sample_result),
    check.attributes = FALSE
  ))
)

stopifnot(
  file.rename(pending_output_path, output_path),
  file.rename(pending_sample_output_path, sample_output_path)
)

Ten-row visual review

The complete Generated Artifact contains 98,273 rows and 81 columns. The Journal Enrichment Sample contains 10 of those rows.

Journal Enrichment Sample

The interactive DT table below shows all result columns. Use its search boxes and horizontal scroll to inspect which expanded fields are useful.

This sample supports visual review. It does not represent the frequency of each match result in the complete table.

sample_result |>
  mutate(issn = gsub("[\\r\\n]+", " / ", issn, perl = TRUE)) |>
  DT::datatable(
    rownames = FALSE,
    filter = "top",
    options = list(
      scrollX = TRUE,
      pageLength = 10L,
      lengthChange = FALSE
    )
  )

Match-result counts

This table shows how many input rows received each match result from each source. Use the four definitions at the start of this document to interpret the counts.

bind_rows(
  count(result, openalex_match_status, name = "rows") |>
    rename(status = openalex_match_status) |>
    mutate(source = "OpenAlex"),
  count(result, crossref_match_status, name = "rows") |>
    rename(status = crossref_match_status) |>
    mutate(source = "Crossref")
) |>
  select(source, status, rows) |>
  knitr::kable()

source	status	rows
OpenAlex	ambiguous	198
OpenAlex	not_attempted	26189
OpenAlex	unique	54347
OpenAlex	unmatched	17539
Crossref	ambiguous	440
Crossref	not_attempted	26189
Crossref	unique	52630
Crossref	unmatched	19014

Complete field list

This list shows every column in the Generated Artifact. It helps reviewers find the original PKP fields, the match evidence, and the expanded source fields. The display does not replace the complete Parquet file.

tibble(column = names(result)) |>
  knitr::kable()

column
context_name
issn
oai_url
application
country
region
total_record_count
record_count_2025
earliest_datestamp
repository_name
set_spec
stats_id
best_doaj_url
last_completed_update
first_beacon
last_beacon
last_oai_response
unresponsive_endpoint
unresponsive_context
identifier_status
openalex_input_issns
openalex_matched_issns
openalex_match_status
openalex_candidate_count
openalex_candidate_ids
crossref_input_issns
crossref_matched_issns
crossref_match_status
crossref_candidate_count
crossref_candidate_issn_sets
openalex_alternate_titles
openalex_apc_prices
openalex_apc_usd
openalex_apc_usd_by_year
openalex_cited_by_count
openalex_country_code
openalex_counts_by_year
openalex_created_date
openalex_display_name
openalex_first_publication_year
openalex_homepage_url
openalex_host_organization
openalex_host_organization_lineage
openalex_host_organization_name
openalex_id
openalex_ids
openalex_is_core
openalex_is_high_oa_rate
openalex_is_high_oa_rate_since_year
openalex_is_in_doaj
openalex_is_in_doaj_since_year
openalex_is_in_scielo
openalex_is_oa
openalex_is_ojs
openalex_is_preprint_repository
openalex_issn
openalex_issn_l
openalex_last_publication_year
openalex_oa_flip_year
openalex_oa_works_count
openalex_societies
openalex_summary_stats
openalex_topic_share
openalex_topics
openalex_type
openalex_updated_date
openalex_works_api_url
openalex_works_count
openalex_candidates__json
crossref_breakdowns
crossref_counts
crossref_coverage
crossref_coverage-type
crossref_flags
crossref_ISSN
crossref_issn-type
crossref_last-status-check-time
crossref_publisher
crossref_subjects
crossref_title
crossref_candidates__json