Core concepts

Larkspur has four ideas. Once they click, the rest of the API is obvious.

Frame

A Frame is a lazy sequence of rows, where each row is a plain dict. Frames are created from a source (read_csv, read_ndjson, or from_rows) and are not consumed until you iterate or call a writer.

Stream

Internally a Frame wraps a Stream — a pull-based iterator. Because rows are pulled on demand, a pipeline over a ten-gigabyte file uses the same memory as one over ten rows.

Transform

A Transform is any callable that receives one row and yields zero or more rows. The built-ins (listed here) are just conveniences around that contract.

def drop_empty(row):
    if any(v == "" for v in row.values()):
        return            # yields nothing -> row is skipped
    yield row

Sink

A sink is the end of a pipeline: write_csv, write_ndjson, or collect() to get a list. Sinks are what actually drive iteration.