OPEN DATA PUBLISHING GUIDE

How to publish a source-backed historical CSV archive

A downloadable CSV is not automatically a trustworthy public dataset. Historical records become reusable when a reader can understand every field, inspect where each row came from, reproduce basic checks, cite a stable version, and report a correction.

This guide describes a small-project workflow for publishing historical event data. It applies to archives such as public notices, observations, schedules, sports results, election records, or other dated facts gathered from multiple public pages. The worked example is the LottoLens PH public draw archive, but the controls are intentionally domain-independent.

1. Define the archive before collecting rows

Write down what one row represents, the fields that form a unique record, the time zone, the date range, and the sources you will accept. If those rules are implicit, two contributors may encode the same event differently or combine records that should remain separate.

2. Keep provenance at row level

A single “Sources” paragraph is too coarse when an archive combines many dates or providers. Keep a source name and source URL on every row whenever possible. That allows a user to audit a disputed record without guessing which page supported it.

Record provenance is not the same as claiming that every source is equally authoritative. Document the preferred authority and explain when a fallback archive is used. If a source page later disappears, preserve the original URL and mark the verification limitation instead of silently replacing the citation.

3. Publish a field dictionary beside the CSV

Column names rarely explain all important constraints. A field dictionary should state the data type, permitted format, null policy, example value, and meaning of each field. Preserve identifiers that can contain leading zeroes as strings. Use an unambiguous date format such as ISO 8601.

When a value contains several components, document their order. If the raw source order matters, do not sort it merely to simplify analysis. Derived columns should be visibly labelled and their calculation explained.

4. Make validation results reproducible

Publish the checks, not only a sentence saying that the data was “cleaned.” At minimum, a fixed tabular release should report:

• row and column counts;
• first and last dates;
• duplicate count for the declared key;
• null counts for required fields;
• invalid date, range, or enumeration counts;
• empty or malformed source URL counts;
• file size and a cryptographic checksum;
• the exact script or commands used for the checks.

A zero-error validation result proves only the rules tested. Publish the validator and name its limits so users can distinguish mechanical quality from full factual accuracy.

5. Separate a stable release from a changing live view

Researchers need a file that does not change after citation. Operational users may want a live archive that continues to receive new records. Serve both, but label them differently.

• Fixed release: version tag, date range, checksum, and immutable download.
• Live view: current coverage, update time, and a warning that totals can change.
• Change log: additions, corrections, removals, and the reason for each material revision.

Never update a file in place while leaving its version label unchanged. If a fixed snapshot is corrected, publish a new version and retain the old release with an explanatory note.

6. State missingness and coverage honestly

A date range does not prove that every expected event is present. Report coverage by relevant subgroup and identify known gaps. Explain whether a missing row means “no event occurred,” “the source was unavailable,” or “the archive has not verified this record.” Those states should not be collapsed into the same blank value.

7. Clarify ownership, licensing, and attribution

Apply a licence only to material you have the authority to license. A project may license its original compilation, documentation, schema, and validation code while underlying facts or source materials remain governed by their original rights and terms. State that boundary next to the licence, not in an obscure page.

Provide a preferred citation containing the dataset title, publisher, version, year, stable URL, and access date when appropriate. Machine-readable metadata such as CITATION.cff, Data Package metadata, or Schema.org Dataset markup improves discovery, but it should match the visible documentation.

8. Give users a correction and support path

Publish a clear way to report incorrect rows, broken sources, privacy concerns, or ambiguous fields. Ask reporters to include the record key, the disputed value, supporting evidence, and the requested correction. Document how corrections are reviewed and how quickly the project normally responds.

9. Verify the public release as a user would

After publishing, test the landing page, CSV, field dictionary, licence, citation file, and source links without relying on an authenticated session. Confirm HTTP status, canonical URL, indexability, sitemap presence, and that the downloaded file checksum matches the release record. A successful build or upload is not proof that the public artifact is usable.

Worked example: a fixed public-event archive

Release checklist

1. Define one row, the unique key, time zone, and coverage.
2. Attach source identity and URL to each record.
3. Publish a field dictionary and machine-readable metadata.
4. Run and save duplicate, null, range, source, and checksum checks.
5. Create an immutable version and distinguish it from live data.
6. Document known gaps and the meaning of missing values.
7. State licensing boundaries and provide a preferred citation.
8. Publish a correction path and versioned change log.
9. Verify every public artifact without a privileged session.

The goal is not to make a dataset look authoritative. It is to let another person inspect what was collected, understand what was not, reproduce the mechanical checks, and cite the exact version they used.