Transform JSON and CSV

This lesson converts a small CSV file into a JSON array of active users. We will preserve a name containing a comma, convert ages into numbers, and turn active flags into Booleans. Those details make this more than a file extension change. Open fixtures slash input dot CSV, then run the supplied converter from examples. The generated JSON and CSV files will appear inside the out directory.

hhy run convert.hhy

Run the complete example from its examples directory.

CSV allows quoted fields containing commas. Splitting every input line on a comma would damage Sam Junior's name by treating it as two fields. Use parse_csv to interpret records according to the format. With header set to true, the first record supplies field names. The parser returns a stream of Maps, so subsequent callbacks can access row dot name, row dot age, and row dot active explicitly.

name,age,active
Ada,36,true
Lin,28,false
"Sam, Jr",31,true

Run the complete example from its examples directory.

CSV does not infer a schema for us. The active field is a String containing the letters true, not a Boolean value yet. Our filter therefore compares it with a quoted string. This is an easy place to make a silent logic mistake by comparing unlike types. Confirm the representation at the input boundary, and decide whether filtering should happen before conversion or after a separate validation and conversion stage.

let users = read_lines(path("fixtures/input.csv"))
    |> parse_csv({ header: true })
    |> where { row -> row.active == "true" }

Run the complete example from its examples directory.

The projection constructs a fresh Map with exactly the fields we want to publish. To_int converts each retained age, and the equality expression produces a real Boolean. Collect materializes the stream as a List, which becomes a JSON array. This small lesson uses tiny fixtures, so that is appropriate. For a very large dataset, think about memory limits and whether your destination can accept records incrementally instead of one complete array.

    |> map { row -> {
        name: row.name,
        age: to_int(row.age),
        active: row.active == "true"
    } }
    |> collect

Run the complete example from its examples directory.

Encode_json turns ordinary HHY values into JSON text. Pretty mode adds readable formatting without changing the data types. Save_text writes that one complete String to disk. Do not send a lazy Stream or a filesystem object directly to the JSON encoder. First consume or project it into supported values. Our output is a List of ordinary Maps, and each Map contains only a String, an integer, and a Boolean.

users |> encode_json({ pretty: true })
    |> save_text(path("out/users.json"),
        { create_parents: true })

Run the complete example from its examples directory.

A round trip is a practical verification step. Read the file, parse it back, and inspect important properties. There should be two retained users. The first name is Ada, the age reports Int, and the second name still includes its comma. This checks filtering, conversion, encoding, and quoted CSV handling together. Merely seeing that a file exists would not prove any of those requirements were met correctly.

let decoded = read_text(path("out/users.json"))
    |> parse_json
print(length(decoded))
print(decoded[0].name)
print(type(decoded[0].age))
print(decoded[1].name)

2
Ada
Int
Sam, Jr

Run the complete example from its examples directory.

To encode CSV, convert our collected List back into a Stream explicitly. Encode_csv returns text records without line terminators, and save_lines supplies those terminators. Reusing the same users List is safe because we are constructing a fresh stream for this operation. Do not assume the generated CSV will preserve every formatting choice from the input. The purpose is to preserve selected data, with serialization handled by the encoder.

users |> stream
    |> encode_csv({ header: true })
    |> save_lines(path("out/users.csv"))

Run the complete example from its examples directory.

Malformed JSON should not quietly become a successful empty result. Here we deliberately pass invalid text and show a clear failure marker from catch. In a production converter, decide whether an invalid record should stop the job or become an explicit rejected record report. Never substitute a plausible default merely to make the script finish. The next lesson shows how modules and per operation Results help express those decisions cleanly.

try {
    parse_json("{broken")
} catch err {
    print("invalid JSON rejected")
}

invalid JSON rejected

Run the complete example from its examples directory.

For practice, add another active user whose quoted name contains a comma, then check the output count and preserved name. Next, in a copy of the fixture, replace an active user's age with nonnumeric text. Observe the conversion failure and consider where you would validate it. Keep the original fixture intact for repeatable comparison. You can now move between CSV records and JSON values while controlling selection, field types, and output structure.

# Add an active user with a quoted name.
# Run again and inspect both output formats.
hhy run convert.hhy

Run the complete example from its examples directory.