Writing

Notes from the work.

Mostly the unglamorous part of finance data: reconciliation, validation, and the judgment calls that decide whether a number can be trusted. Written for the people who deal with these problems.

  1. The dead-letter pattern: never corrupting the client's view

    A record that fails validation should be set aside in a dead-letter store, not dropped and not forced in. How a borrowed messaging idea keeps a client's data clean.

  2. Reconciling two vendors when there's no ground truth

    When two data vendors disagree and there is no third source to settle it, picking a global winner is the wrong move. The useful work is classifying disagreements and routing the ambiguous ones to a person.

  3. What makes data “decision-grade”

    Most data is probably correct. The trouble is that nobody can tell without redoing the work, so they redo it. Three properties separate data you can act on from data you have to verify first.