Derive, Don't Store

The principle: Every fact you can recompute from data you already have is a fact you must not store, because a stored copy is a second source of truth that will eventually disagree with the first.

① First principles: the cost of a stored copy

You're building three features that feel like they obviously need their own storage.

First, read-state: which rooms has a user not yet caught up on? The obvious move is a table. Here's the version you'd vibe-code before the principle clicked:

class ReadReceipt < ApplicationRecord
  belongs_to :user
  belongs_to :message
end

# and then, to find unread rooms, something like:
def unread_rooms_for(user)
  user.rooms.select do |room|
    last = room.messages.maximum(:created_at)
    receipt = ReadReceipt.where(user: user, room: room).maximum(:read_at)
    last && (receipt.nil? || last > receipt)
  end
end

A row per user per message read. An N+1 that grows with the size of the app. And the moment you add a badge counter to make that loop cheaper — memberships.unread_count bumped on each new message — you've now got a second number that has to agree with the first, and the day a transaction half-fails, the badge says 3 and the room is empty.

Second, presence: who's online right now? The obvious move is a boolean.

# in the channel that opens when a user connects:
def subscribed
  current_user.update!(online: true)
end

def unsubscribed
  current_user.update!(online: false)
end

This works until the first laptop lid slams shut without a clean disconnect. Now online is stuck at true forever, because the unsubscribed event that was supposed to flip it back never fired. So you bolt on a sweeper cron to flip stale rows back to false, and a last_seen_at column to decide which rows are stale, and now you have two columns describing one fact, plus a background job whose entire job is to correct a value that lies.

Third, a capability: hand someone a URL that lets them load a user's avatar, or a one-time link to transfer their account to a new device. The obvious move is, again, a table.

class AvatarToken < ApplicationRecord
  belongs_to :user
  # columns: token, expires_at, purpose
end

token = AvatarToken.create!(user: user, token: SecureRandom.hex, expires_at: 1.hour.from_now)
# ...later, to verify:
t = AvatarToken.find_by(token: params[:token])
head :not_found unless t && t.expires_at > Time.current

A tokens table. A hand-checked expiry. A .destroy for one-time use. And — inevitably — a nightly cron to sweep the expired rows nobody cleaned up. Three of these features, three little storage subsystems, three things that drift.

Now look at what they have in common. In every case the fact you're storing is already implied by data you already have. Whether a room is unread is implied by when you last looked at it versus when the last message arrived. Whether you're online is implied by when you last touched the connection. Whether a URL is a valid capability is implied by a signature over the record id and a purpose — math, not a row. The stored copy isn't the source of truth; it's a cache of a truth that lives elsewhere, and a cache you have to keep in sync by hand is a bug waiting for a quiet afternoon.

So here is the principle, derived rather than decreed: derive, don't store. Replace a stored fact with a function of existing data. Read-state becomes one nullable timestamp. Presence becomes a question about a time window. A capability becomes a signed string where the credential is the URL. The unifying move is the same every time — and when you make it, the second table, the sweeper job, and an entire class of "the copy disagrees with the original" bugs all disappear together.

Fizzy, 37signals' Kanban board, pushes the same instinct somewhere you'd swear needs a stored answer: the order of cards in a column. There is no position column anywhere on a card — intra-column order is computed with scope :latest, -> { order last_active_at: :desc, id: :desc } (card.rb:24) plus a goldness join, so "reorder" has no stored value to renumber. That one decision deletes an entire acts_as_list-style ranking subsystem — the renumber-the-whole-column-on-every-drop, the off-by-ones when a middle card is deleted — and it's the same principle, just aimed at a fact you'd never guess was derivable.

There's even an HTTP-layer version of the same move, and it's worth seeing now because it's the one people forget is the same idea: a 304 Not Modified is just derive freshness from an etag instead of re-running the action — the full mechanics belong to see F6: Caching and see F2: Controllers & Routing; here it's only the derivation worldview. You don't store "is this response still fresh?" anywhere — you compute it from the data the response was built from. Same principle, three altitudes up.

"Isn't a stored count just faster than recomputing it every time?" Sometimes — and when it genuinely is, that's what see F6: Caching is for: a derived cache, keyed so it self-invalidates. The distinction is whether the stored value is the source of truth (a counter you bump by hand — drifts) or a derivable one keyed on the data it summarizes (a cache fragment keyed on updated_at — self-heals). Derivation isn't "never store a computed value." It's "never make the stored value the authority." A bumped counter is a second source of truth; a keyed cache is a function with a memo.

② The beauty in combination

This principle isn't a micro-optimization you apply line by line. Its real power shows up when you hold it together with the principles around it — that's when it stops deleting a column and starts deleting whole subsystems.

Hold it with P1 (the model owns the truth). see P1: The Model Owns Its Consequences says a record is the single source of truth about a fact. Derivation is the natural consequence: if the model already owns the truth, then any other representation of that truth must be computed from it, never stored beside it. One unread_at column on the membership owns "have you caught up?" — and so the unread badge, the sidebar dot, and the OS push count are all computed at their surface from that one column. There is no badge-says-3-but-the-room-is-empty bug, because there is no second number to disagree with the first. The badge isn't stored anywhere; it's user.memberships.unread.count, asked fresh at the moment something needs it.

This isn't a Campfire quirk — Fizzy reaches for the identical instinct one layer further out, on the client: its app-badge isn't a JS counter it increments and decrements, it's recomputed as the number of DOM nodes still carrying the unread class (badge_controller.js:34-36), so a wholesale repaint can never leave it stale. Two unrelated 37signals products, the same move — the count is a projection of truth, never a second copy of it.

Hold it with P3 (security is the shape of your data access). A derived credential is only safe if it can't be replayed somewhere it wasn't meant for. signed_id's purpose: scoping is exactly that guard, expressed as data instead of a check: an avatar token verified with purpose: :avatar simply will not verify against purpose: :transfer. The capability is derivable and unforgeable and un-replayable, all from one signed string — no table to leak, no expiry to forget. see P3: Security Is the Shape of Your Data Access shows how loading through the user makes the leak unwriteable; here, the same instinct makes the replay unwriteable.

Hold it with P4 (convention is leverage). There's one idea — "if the content changes, the key changes" — and derive, don't store makes it show up at three different altitudes of the same app. At the route layer, an avatar URL stamps v: user.updated_at so the URL itself changes when the avatar does. At the model layer, signed_id derives the whole credential from the record. At the HTTP layer, an immutable asset's freshness is derived from its etag. see P4: Convention Is Leverage is why all three can trust the framework to compute the same thing the same way; derivation is what they compute. One principle, three layers, zero hand-maintained version integers.

The reader's takeaway from this beat: derivation compounds. On its own it saves a column. Combined with "the model owns the truth," it deletes the drift-bug class. Combined with "security is a shape," it deletes the token table. Combined with "convention is leverage," it deletes the by-hand cache-busting. You don't apply these one at a time — they interlock, and the app gets smaller at every seam.

③ How 37signals did it

Now the evidence. Every claim below is a quoted line from Campfire, because the principle is only worth anything if it's load-bearing in real code.

Read-state: one nullable timestamp answers three questions

The entire read/unread/badge feature rides on a single column. Here's the whole of it in the model (membership.rb:15-23):

  scope :unread,  -> { where.not(unread_at: nil) }

  def read
    update!(unread_at: nil)
  end

  def unread?
    unread_at.present?
  end

That's it. unread_at is null when you've read the room and a timestamp when you haven't. Presence is read; absence is unread — the null is the data.

Fizzy reaches for the same silence-carries-the-meaning instinct, just spelled with a row instead of a column: a card is closed if and only if a satellite closure record exists (card/closeable.rb:7-8), where scope :closed, -> { joins(:closure) } and scope :open, -> { where.missing(:closure) } — and the closing user and timestamp come for free off that row (def closed_by; closure&.user; end), with no closed/closed_at/closed_by_id columns to keep in sync. Same derive-don't-store force, two surfaces: Campfire lets the absence of a timestamp mean caught-up; Fizzy lets the absence of a row mean open. No read_receipts table, no per-message rows, no unread_count integer to bump and watch go negative.

Watch the one column get derived into three different surfaces. Marking a room unread when a message lands is one bulk SQL update (room.rb:69, the in-band/out-of-band altitude of which belongs to see P9: Put Work at Its Right Altitude):

memberships.visible.disconnected.where.not(user: message.creator).update_all(unread_at: message.created_at, updated_at: Time.current)

The sidebar dot is a CSS boolean driven by the same column passed in as a local (users/sidebars/rooms/_shared.html.erb:3):

class: [ "align-center gap room btn txt-nowrap", "unread": local_assigns[:unread] ]

And the OS push badge is computed fresh at the moment a notification is built (push/subscription.rb:5):

badge: user.memberships.unread.count

One column. Three surfaces. The badge is never stored, so it can never lie. The contrast: the naive ReadReceipt model grows a row for every message every person reads, forces an N+1 to count unread rooms, and — the moment you add an unread_count cache to fix that — gives you a number maintained in two places that drifts the first time a write half-fails. Campfire derives all three surfaces from one nullable column.

State as a row's existence: Fizzy derives "closed" from a satellite, not a column

Campfire's read-state lets the absence of a value carry the meaning — unread_at IS NULL means caught-up. Fizzy takes that same silence-by-absence instinct one notch further: it lets the absence of an entire row carry the meaning, and the win compounds, because the row that does exist brings its own metadata along for free.

A Fizzy card has no closed, closed_at, or closed_by_id columns. "Closed" is the existence of a one-row satellite — a Closure. The whole derivation lives in a tiny concern (card/closeable.rb:4-21):

  included do
    has_one :closure, dependent: :destroy

    scope :closed, -> { joins(:closure) }
    scope :open, -> { where.missing(:closure) }

    scope :recently_closed_first, -> { closed.order(closures: { created_at: :desc }) }
    scope :closed_at_window, ->(window) { closed.where(closures: { created_at: window }) }
    scope :closed_by, ->(users) { closed.where(closures: { user_id: Array(users) }) }
  end

  def closed?
    closure.present?
  end

  def open?
    !closed?
  end

  def closed_by
    closure&.user
  end

  def closed_at
    closure&.created_at
  end

Read it as a derivation, not a relationship. scope :closed is joins(:closure) — a card is closed iff a closure row joins to it. scope :open is the exact inverse, where.missing(:closure) — open iff no closure row exists. There is no boolean to flip and no NULL-versus-false ambiguity to reason about, because the predicate is "does this row exist," which SQL answers with a join.

Now the part that makes "existence" beat "nullable timestamp" outright: the satellite carries its own metadata as ordinary columns, so "who closed it" and "when" cost nothing extra. The Closure model is four lines (closure.rb:1-5):

class Closure < ApplicationRecord
  belongs_to :account, default: -> { card.account }
  belongs_to :card, touch: true
  belongs_to :user, optional: true
end

That's the entire storage subsystem for closing. closed_by is closure&.user and closed_at is closure&.created_at (card/closeable.rb:23-29) — derived off a row that exists for one reason. The nullable-column approach would have forced you to add closed_at and closed_by_id to the cards table and remember to set both on close and null both on reopen; here the metadata rides along on the row whose presence already is the state.

And because the row's existence is the state, destroying the row is the reset — there is nothing to un-set. Reopen is closure&.destroy (card/closeable.rb:43-46); close is create_closure! (card/closeable.rb:35). That pairs exactly with [see P5: State Changes Are REST Resources] — closing a card is create on a Closure, reopening is destroy, so the data shape ("a row exists or it doesn't") and the HTTP shape ("POST the resource or DELETE it") are the same idea seen from two layers. The contrast: the naive Rails version adds closed/closed_at/closed_by_id to the cards table, then a four-column update! on close and a four-column null-out on reopen that you must never forget — and every next state (golden, not-now) adds two or three more columns to a table that keeps getting wider. Fizzy keeps the card skinny and lets each state be its own skinny satellite, queried with joins / where.missing, reset with destroy.

Presence: a question about time, not a flag

"Online" in Campfire is not a boolean you keep in sync. It's whether your connected_at falls inside a rolling window. The whole definition (connectable.rb:4-9):

  CONNECTION_TTL = 60.seconds

  included do
    scope :connected,    -> { where(connected_at: CONNECTION_TTL.ago..) }
    scope :disconnected, -> { where(connected_at: [ nil, ...CONNECTION_TTL.ago ]) }
  end

Presence is a question about time. Are you online? Only if your connected_at is newer than 60 seconds ago — expressed as a beginless range, CONNECTION_TTL.ago... And disconnected is the gorgeous inverse: an array folding never connected (nil) and seen too long ago (...CONNECTION_TTL.ago) into one where. The threshold is one named constant, and that same constant defines the in-memory predicate (connectable.rb:21-23), so the database and Ruby can never disagree on what "online" means:

  def connected?
    connected_at? && connected_at >= CONNECTION_TTL.ago
  end

"Why a time window instead of a status flag?" Because flags lie. A flag says "online" forever if the laptop slams shut without a clean disconnect event. A 60-second window self-heals: stop touching connected_at and you fade to disconnected automatically, with no sweeper job to correct a value that's wrong. The crashed client ages out on its own; the truth is derived from the last timestamp, not asserted by an event that might never arrive.

There's a quiet payoff in the derivation you'd miss if you weren't counting edge cases: the naive where("connected_at < ?", CONNECTION_TTL.ago) silently drops every row where connected_at is NULL, so never-connected users would miss notifications, while the beginless/endless-range form folds nil and stale into one where. (The range-as-query mechanics are see F1: The Rails Model & Active Record's; here it's evidence that one derived predicate absorbs the NULL bug.) And note the connect method nulls unread_at in the same write that sets connected_at (connectable.rb:16-18) — opening a room is marking it read, so there's no separate /read endpoint to call:

    def connect(membership, connections)
      where(id: membership.id).update_all(connections: connections, connected_at: Time.current, unread_at: nil)
    end

The contrast: the boolean version is stuck online after a crash, marks you offline while you're staring at the screen from a second tab, and needs a sweeper cron bolted on to half-fix it. The derived window needs none of that, because it never stores a fact it would have to correct later.

Capability links: the credential IS the URL

An avatar URL and an account-transfer link are capabilities — "whoever holds this may do this one thing." Campfire stores zero rows for them. The avatar concern, in full (user/avatar.rb:8-16):

module User::Avatar
  extend ActiveSupport::Concern

  included do
    has_one_attached :avatar
  end

  class_methods do
    def from_avatar_token(sid)
      find_signed!(sid, purpose: :avatar)
    end
  end

  def avatar_token
    signed_id(purpose: :avatar)
  end
end

The mint (avatar_token) and the verify (from_avatar_token) live a few lines apart in one tiny concern, with the identical purpose: :avatar, so they cannot drift. The transfer link is the same shape with one addition — an expiry derived into the signature itself (user/transferable.rb:4-14):

  TRANSFER_LINK_EXPIRY_DURATION = 4.hours

  class_methods do
    def find_by_transfer_id(id)
      find_signed(id, purpose: :transfer)
    end
  end

  def transfer_id
    signed_id(purpose: :transfer, expires_in: TRANSFER_LINK_EXPIRY_DURATION)
  end

The credential is the URL. No tokens table, no expires_at column to hand-check, no .destroy for one-time use, no nightly sweeper cron — and no cross-purpose replay, because purpose: :avatar and purpose: :transfer are different signatures over the same id.

Fizzy's passwordless login pushes the same don't-store-it instinct to its limit: a one-time login code isn't marked spent with a used boolean, it's deleted — consume destroys the row and returns it (magic_link.rb:18-19,27-30), and validity is a beginless/endless range scope (scope :active, -> { where(expires_at: Time.current...) }, magic_link.rb:9) so the database never even returns a stale code. A consumed credential isn't a flagged row; it's an absent one — the exact silence-by-absence move as read-state-as-null, applied to auth. (The mechanics of signed_id as a concern live in see F7: Concerns as a Mechanism; here it's evidence that a stored credential was never needed at all.)

Fizzy carries the same silence-by-absence instinct into auth, and lands it as deletion. A passwordless login code is never marked spent with a used boolean — it's destroyed. consume removes the row and hands it back so the caller can still read it, and the class method only ever finds a code that hasn't expired (magic_link.rb:9-10,17-30):

  scope :active, -> { where(expires_at: Time.current...) }
  scope :stale, -> { where(expires_at: ..Time.current) }

  class << self
    def consume(code)
      active.find_by(code: Code.sanitize(code))&.consume
    end

    def cleanup
      stale.delete_all
    end
  end

  def consume
    destroy
    self
  end

A consumed credential isn't a flagged row; it's an absent one — the exact same move as read-state-as-null, one layer down in the auth flow. And validity is derived, not stored: scope :active, -> { where(expires_at: Time.current...) } is an endless range, the same idiom as Campfire's Connectable TTL window (connectable.rb:7), so the database never even returns a stale code — a one-time link can't be replayed because after consume there is no row to find, and an expired one can't be found because it falls outside the range. No used column to set, no "is this code spent?" check to write, and the stale.delete_all cleanup just sweeps what the range scope already ignores.

Mentions: derive the User, don't store the name

The same instinct applies to text. An @mention is not stored as a parsed @token string. It's an ActionText attachable — a signed global id embedded in the rich-text body, rehydrated into a real User on the way out. Extracting the mentioned users is not a regex; it's a grep over the attachables already in the body (message/mentionee.rb:9-15):

    def mentioned_users
      if body.body
        body.body.attachables.grep(User).uniq
      else
        []
      end
    end

A User becomes mentionable just by including the attachable concern and declaring how it renders (user/mentionable.rb:1-15):

module User::Mentionable
  include ActionText::Attachable

  def to_attachable_partial_path
    "users/mention"
  end

  def to_trix_content_attachment_partial_path
    "users/mention"
  end

  def attachable_plain_text_representation(caption)
    "@#{name}"
  end
end

The mention round-trips losslessly: stored as a secure sgid, derived back into the live User (so a rename shows the new name everywhere), rendered as @name for push bodies. The contrast: store @nityesh as a literal token and you're back to body.scan(/@(\w+)/) then User.where(name: ...) — which breaks on duplicate names, can't survive a rename, can't render an avatar, and forces a re-parse anywhere the list is needed. Don't store the name string; derive the User. (Full unpack of attachables as a concern: see F7: Concerns as a Mechanism.)

HTTP: a 304 is derivation, three layers up

The principle reaches all the way to the wire. The avatar controller wraps its entire expensive body — WebP variant processing — in a freshness check derived from the record (avatars_controller.rb:9-10):

    if stale?(etag: @user)
      expires_in 30.minutes, public: true, stale_while_revalidate: 1.week

If the client's etag matches, stale? returns false and the image processing inside the block never runs — the fastest request is the one whose body never executes. The message index does the same with a collection (messages_controller.rb:13-17):

    if @messages.any?
      fresh_when @messages
    else
      head :no_content
    end

Nobody stores "is this response still fresh?" anywhere. It's derived from the data the response was built from — @user, @messages — exactly the same move as the unread_at timestamp, just at the HTTP layer. And the route that builds the avatar URL stamps the version straight onto it (routes.rb:58-60), so when the avatar changes, the URL changes:

  direct :fresh_user_avatar do |user, options|
    route_for :user_avatar, user.avatar_token, v: user.updated_at.to_fs(:number)
  end

That's "if the content changes, the key changes" at the route layer, signed_id at the model layer, and the etag at the HTTP layer — one idea, three altitudes, no hand-maintained version integer anywhere. Count the edge cases this one habit absorbs for free: the drifting badge, the stuck-online flag, the orphaned token row, the broken-by-rename mention, the recomputed-on-every-hit avatar. Each is a subsystem the naive version builds and Campfire simply derives away.

Key Takeaways — Patterns to Steal

• The instant you can recompute a fact from data you already hold, storing a copy of it is the bug — that copy is a second source of truth, and the day a write half-fails it disagrees with the original. Don't reach for a read_receipts table or an online boolean or a tokens table just because the feature "obviously needs storage"; ask first what existing data the fact is already implied by. The whole of P2 is Campfire choosing one nullable timestamp, one time window, one signed string where a naive build would have stood up three little storage subsystems that drift.
• Store read-state as a single nullable column and derive every view of it, instead of a ReadReceipt row per user per message plus an unread_count integer you bump by hand. The badge counter you'd add to make counting cheap is exactly the number that says 3 while the room is empty the first time a transaction rolls back half-finished. Campfire's membership.rb:15-23 is the entire feature — scope :unread, -> { where.not(unread_at: nil) } and a read that does update!(unread_at: nil) — and the OS push badge is just user.memberships.unread.count computed fresh in push/subscription.rb:5, never stored, so it cannot lie.
• Let null carry the meaning rather than adding a status enum or a read/unread boolean beside the timestamp. Presence of a value means unread-since-then, absence means caught up, and that one column answers "is it unread, since when, which rooms" in a single lookup. That's why unread_at in membership.rb needs no companion flag — the null is the data.
• Model "who's online" as a question about time, never a boolean you keep in sync. A flag stays stuck at true forever the first time a laptop lid slams shut without a clean disconnect, so you end up bolting on a last_seen_at column and a sweeper cron whose only job is correcting a value that lies. Campfire's membership/connectable.rb:7 writes scope :connected, -> { where(connected_at: CONNECTION_TTL.ago..) } — a 60-second beginless range that self-heals, because the crashed client simply ages out when nothing touches connected_at anymore.
• Make the credential be the URL with signed_id, instead of standing up a tokens table with an expires_at column you hand-check, a .destroy for one-time use, and the nightly cron that inevitably sweeps the rows nobody cleaned up. Campfire's user/avatar.rb:8-16 mints with signed_id(purpose: :avatar) and verifies with find_signed!(sid, purpose: :avatar) — zero rows stored — and crucially the mint and the verify sit a few lines apart in one tiny concern so the matching purpose: can't drift.
• For @mentions, derive the real User back out of the rich-text body instead of storing @nityesh as a literal token and re-parsing it with body.scan(/@(\w+)/) then User.where(name: ...). The string version breaks on duplicate names, can't survive a rename, and can't render an avatar. Campfire embeds a signed sgid as an ActionText attachable and extracts mentions with body.body.attachables.grep(User).uniq (message/mentionee.rb:9-15), so a rename shows the new name everywhere and the user round-trips losslessly.
• Treat a 304 Not Modified as the HTTP-layer version of the same derivation move — freshness computed from the record, never stored as an "is this still fresh?" field. Wrap the expensive work in a freshness check so it never runs on a cache hit: avatars_controller.rb:9 guards WebP variant processing with if stale?(etag: @user), and messages_controller.rb:14 does the collection version with fresh_when @messages. The fastest request is the one whose body never executes.
• Bake "if the content changes, the key changes" into the key at every altitude rather than maintaining a version integer by hand. The route stamps v: user.updated_at.to_fs(:number) straight onto the avatar URL (routes.rb:58-60), the model derives the whole credential with signed_id, and the response carries an etag — one idea, three layers, and not a single hand-bumped version number, which is what kills the "I changed the avatar but the old one is cached forever" class of bug.