R Programming / Language and data foundations / Lesson 6

Text import, dates, and data contracts

Turn supplied text into typed observations without hiding missingness, identifiers, or time interpretation.

About 5 min readContent reviewed 2026-10-28

On this page

Lesson goals

Declare column types and missing-token rules.
Validate imported names, identifiers, and counts.
Separate calendar dates, timestamps, and text normalization.

Text is a representation with parsing assumptions

A CSV contains tokens separated under quoting and delimiter rules, not intrinsically typed measurements. A comma inside a quoted field is different from a separator, and decimal or encoding conventions can change interpretation. State the expected header, column order or names, units, missing tokens, and identifier format. Automatic inference can be useful for exploration but should not define a reviewed schema accidentally. Keep a small in-memory text fixture so parser behavior can be investigated without private records, external files, services, or credentials. A parser accepting input does not establish that the resulting values mean what the task expects.

Import with an explicit narrow schema

The first fixture uses read.csv on a supplied string, declaring character identifiers, numeric durations, and a character status. An empty duration token becomes missing under the chosen na.strings rule. A literal text status such as missing remains ordinary text in another column. The name assertion checks the expected schema before summarizing. Real validation should also inspect id uniqueness or intended multiplicity, row counts, range constraints, and consistency between status and value. Do not let a column’s convenient inferred type erase leading zeros from identifiers or convert an entire measurement field into character unnoticed.

r Illustrative base-R script; R runtime not available during review

csv <- "id,minutes,status\na,10,ok\nb,,missing\nc,30,ok"
records <- read.csv(text = csv, colClasses = c("character", "numeric", "character"),
                    na.strings = "", stringsAsFactors = FALSE)
stopifnot(identical(names(records), c("id", "minutes", "status")))
cat("rows=", nrow(records), "; missing=", sum(is.na(records$minutes)), "\n", sep = "")
cat("observed mean=", mean(records$minutes, na.rm = TRUE), "\n", sep = "")

Expected output

rows=3; missing=1
observed mean=20

The CSV is supplied as an in-memory string, with explicit column classes and a blank-token missingness rule; no file or service is accessed.

Missing-token rules can change the population

Treating empty, "NA", "null", or zero as missing are different policies. A token can be a valid category or identifier in one column and a missing marker in another. Count missingness after parsing and compare it with the supplied rows. Decide whether malformed numeric tokens are errors, quarantined rows, or documented missing values; do not silently drop them and then report an apparently cleaner average. Preserve original tokens for a diagnosis when permitted. The observed-only mean in the fixture uses two supplied durations and does not estimate the omitted value or justify a population conclusion.

Dates and timestamps carry different contracts

A Date represents a calendar date, while a timestamp represents a time point interpreted with timezone and calendar rules. The second fixture explicitly parses year-month-day labels and computes a three-day difference. It avoids daylight-saving and timezone-dependent timestamp behavior. For actual timestamps, state the source timezone, intended display timezone, ambiguous local-time policy, and units of differences. Formatting a value for display is not the same as changing its underlying meaning. A parse can return missing values for invalid text, so validate the result rather than assuming every input label matched the declared format.

r Illustrative base-R script; R runtime not available during review

labels <- c("2026-01-02", "2026-01-05")
dates <- as.Date(labels, format = "%Y-%m-%d")
stopifnot(!anyNA(dates))
cat("gap days=", as.integer(dates[2] - dates[1]), "\n", sep = "")
name <- "  sample-a  "
cat("trimmed=", trimws(name), "\n", sep = "")

Expected output

gap days=3
trimmed=sample-a

Calendar dates and whitespace cleanup are explicit transformations; the example does not interpret timezone-dependent timestamps.

Normalize text only under a justified identity rule

Trimming surrounding whitespace can repair accidental separators in identifiers, as the fixture shows. It can also collapse distinct raw labels, so check for collisions after normalization. Case folding, Unicode normalization, decimal replacement, and punctuation removal need their own policies; they are not universal cleaning operations. Keep transformations ordered and auditable so a later join can explain how a key was formed. If exporting text later, declare quoting, missing-value representation, encoding, and column order as well. The course’s examples stay in memory and have derived expected outputs; no R parser execution or locale-specific validation occurred during review.

Common errors and fixes

Identifiers lose their leading zeros

Why it happens: Type inference interpreted labels as numbers.
How to fix it: Declare the identifier column as character before import.
Prevent it next time: Assert representative ids and their required format.

A summary improves after import but many rows disappeared

Why it happens: Malformed or missing tokens were silently excluded.
How to fix it: Count every input, rejected, and retained row under an explicit policy.
Prevent it next time: Report parsing failures and missingness before aggregation.

A day difference is confused with elapsed timestamp hours

Why it happens: Calendar dates and timezone-dependent instants were mixed.
How to fix it: Choose the correct temporal representation and state units.
Prevent it next time: Test boundary dates and relevant timezone transitions separately.

Check your understanding

1. Why does the fixture retain three rows while averaging only two durations?

Show worked solution

The blank token becomes a missing numeric value under the explicit rule, and no row is deleted. mean with na.rm=TRUE uses ten and thirty, giving twenty. The row count remains three and the missing count is one; both should accompany the summary.

2. Two raw identifiers are "a" and " a ". What must follow trimming?