Deterministic Rails for Non-Deterministic AI

Quality Engineering & Harness Design for Coding Agents

Early ideas and examples. An invitation to compare notes.

The One Person Framework

Rails and its ecosystem: a toolkit for building the whole product.

Idea: DHH, The One Person Framework. The toolkit shown here reflects Rails 8 and its ecosystem.

Pasabaya.app

Community carpooling in the Philippines

Rails 8 + Hotwire · Recurring rides · Private trip chat · pasabaya.app

CSMasters.dev

Interactive Computer Science learning for Cambridge students

AI Tutor, inside the lesson

A question about binary overflow:

Why does 200 + 80 overflow an unsigned 8-bit register?

The tutor recalls the limit of 255, then asks what happens to the extra bit.

Rails 8 + Hotwire · Pseudocode sandbox · AI Tutor · csmasters.dev

My Master's Capstone

How can project setup, executable checks and a clear workflow help coding agents deliver better changes?

For me, “better” needs evidence:

• Do changes meet the agreed requirements?
• Which defects reach review, and what is the total time and cost?

Rails vs Rust: The Cost of Change

After DHH's HEY/Rust discussion: what does it cost to add the next feature?

Campfire Bots: the same task and checks, Rails versus a specific Rust port.

Source: Evil Martians, October 21, 2026. The AI-generated Rust port contains roughly 13x more code.

Feature Tickets: What the Benchmark Measures

Agents on Rails: mean task scores with partial credit, October 15 data snapshot.

Stage 1: 19 tasks. Stage 2: 20 tasks. Three runs per task and configuration.

Sources: Rails Foundation / Evil Martians, fixed dataset.

Five Failure Modes to Check

Benchmark observations motivate checks for these Rails risks:

• 1. Migration mistakes: Handwritten migrations need checks for ordering, DDL safety, rollback behavior, and data preservation.
• 2. Broken interactions: An HTTP 200 check can pass while a Turbo Frame or Stimulus interaction fails.
• 3. Framework bypass: Custom logic can overlook existing APIs such as delegated_type or params.expect.
• 4. Runtime concerns: Job retries, persistent queues, authorization, and N+1 queries need explicit checks.
• 5. Weakened tests: A green suite can hide a changed assertion and a broken requirement.

Which Instructions Can Become Checks?

# Example project instructions
Please remember:
- Never write N+1 queries!
- Always use deliver_later, never deliver_now!
- Always make background jobs idempotent!
- Ensure your migrations are reversible!
- Use our design system, no custom CSS!

I'm exploring how repository instructions and executable checks can work together.

One Workflow to Explore

A workflow for non-trivial changes, from Kirill Mokevnin's Agentic Coding Design Patterns:

1. 01

   Explore

   Read the code,
   schema and tests.

2. 02

   Plan

   Agree on the outcome
   and acceptance checks.

3. 03

   Code

   Build a small slice
   across UI, API and DB.

4. 04

   Commit

   Verify the change.
   Review the evidence.

Four Layers of Project Checks

The agent writes a change. Each layer checks a different property.

1. 1. Schema contextAnnotateRb keeps schema metadata close to the model.
2. 2. Static rulesHerb checks templates. Archspec checks boundaries.
3. 3. Runtime scenariosExercise SQL, migrations and browser interactions.
4. 4. Verification loopReturn diagnostics, limit repairs and collect evidence.

“Deterministic” here: explicit pass/fail rules with controlled inputs. The combined effect still needs measurement.

AnnotateRb: Schema Context

AnnotateRb puts schema metadata beside the model. Shortened illustrative example:

# == Schema Information
# Table name: invoices
#  id           :bigint           not null, primary key
#  amount_cents :integer          default(0), not null
#  status       :string           default("draft"), not null
#  customer_id  :bigint           not null
class Invoice < ApplicationRecord
end

Check against a prepared verification database:

bundle exec annotaterb models --frozen
bundle exec annotaterb routes --frozen

Herb Demo: One Nested Form

broken.html.erb — five lines, no Rails app or database.

<%= form_with url: "/invoices" do |form| %>
  <%= form.text_field :title %>
  <%= form.submit "Save" %>
  <%= button_to "Delete", "/invoices/1", method: :delete %>
<% end %>

Herb 0.11.1: html-no-nested-forms at 4:2. Exit 1.

A successful HTTP response does not check this structure.

Herb Demo: Fix and Recheck

Move the delete button outside the outer form:

 <%= form_with url: "/invoices" do |form| %>
   <%= form.text_field :title %>
   <%= form.submit "Save" %>
-  <%= button_to "Delete", "/invoices/1", method: :delete %>
 <% end %>
+<%= button_to "Delete", "/invoices/1", method: :delete %>

Before

1 nested-form error · exit 1

After

0 errors for this rule · exit 0

cd examples/phrug-herb-demo
npm run demo

Reproduced fixtures, one rule. A small feedback cycle I'd like to explore at project scale.

Strict Locals: Enforcing Partial Contracts

Illustrative partial: app/views/invoices/_row.html.erb

<%# locals: (invoice:) %>

<tr>
  <td><%= invoice.number %></td>
</tr>

Caller with mismatched argument:

<%# WRONG: Agent passed order: instead of invoice: %>
<%= render "invoices/row", order: @invoice %>

Archspec: Contracts with 'because:'

Illustrative contracts for email delivery and a public billing API:

# Archspec.rb
component :controllers, in: "app/controllers/**/*.rb"
component :billing,     in: "app/domains/billing/**/*.rb"

# 1. Reject direct synchronous email delivery calls
controllers.cannot_call :deliver_now, :deliver_now!,
  because: "HTTP controllers must queue emails via deliver_later"

# 2. Expose the facade; keep other billing constants private
billing.public_api constants: "Billing::Facade",
  because: "Other components must use the billing facade"

Run the declared checks:

bundle exec archspec check --format json

rubocop-thread_safety: Puma & Solid Queue

Agent wants to disable a callback temporarily during an import:

# app/services/import_users.rb
User.skip_callback(:save, :after, :sync_profile)
user.save!
User.set_callback(:save, :after, :sync_profile)

# .rubocop.yml
plugins: [ rubocop-thread_safety ]
ThreadSafety/ActiveSupportCallbacks: { Enabled: true }

Prosopite: Runtime SQL Fingerprinting

Illustrative query loop outside association loading:

orders.each do |order|
  Customer.find(order.customer_id).name
end

Enable exceptions, then scan an integration test:

# config/environments/test.rb
config.after_initialize { Prosopite.raise = true }

# Inside an ActionDispatch::IntegrationTest subclass
test "orders list has no detected N+1" do
  Prosopite.scan { get orders_path }
end

DatabaseConsistency & Strong Migrations

Unique number per account. Prerequisite: account_id and number are already NOT NULL in the DB.

class Invoice < ApplicationRecord
  validates :number, presence: true, uniqueness: { scope: :account_id }
end

DatabaseConsistency reports missing indexes; Strong Migrations checks supported DDL risks.

# Existing PostgreSQL table; inside a migration class
disable_ddl_transaction!
def change
  add_index :invoices, [:account_id, :number], unique: true, algorithm: :concurrently
end

Rollback Is a Smoke Test

# config/initializers/strong_migrations.rb
StrongMigrations.check_down = true

# DATABASE_URL must point to an isolated, disposable DB.
RAILS_ENV=test bin/rails db:migrate &&
  RAILS_ENV=test bin/rails db:rollback STEP=1 &&
  RAILS_ENV=test bin/rails db:migrate

Hotwire: HTTP Green, Frame Error

Illustrative inline-editing response with a missing matching frame:

<%# app/views/invoices/show.html.erb %>
<%= turbo_frame_tag "invoice_details" do %>
  <%= link_to "Edit", edit_invoice_path(@invoice) %>
<% end %>

<%# app/views/invoices/edit.html.erb %>
<%# BUG: Agent forgot the matching turbo_frame_tag or misspelled ID! %>
<%= form_with model: @invoice do |f| %>
  <%= f.text_field :title %>
  <%= f.submit "Save" %>
<% end %>

Protect the Acceptance Checks

Illustrative failure: weakening an assertion can turn a test green.

# Illustrative weakened test:
# BEFORE: assert_equal 42, calculated_total
# AFTER:  assert true
# Or in RSpec: expect(calculated_total) # No matcher attached!

Proposed controls for a trusted verifier: 1. Assertion linting: explicitly enable Minitest/UselessAssertion; use RSpec/VoidExpect for RSpec. 2. Protected checks: compare tests, rules, and verifier code against a reviewed baseline; review intentional changes. 3. Patch coverage: example threshold after collecting fresh coverage: simplecov patch --base origin/main --minimum 100

A Proposed Agent Workflow

1. 1. Human + agentAgree on a ticket and acceptance checks.
2. 2. Author agentImplement the change in an isolated worktree.
3. 3. Project checksReturn diagnostics. Repair within time and attempt limits.
4. 4. Reviewer agentInspect the diff and evidence with fresh context.
5. 5. Human reviewAccept the outcome, permissions and data behavior.

Worktree & State Isolation

Parallel worktrees also need separate runtime resources:

A Proposed Interface: bin/verify

One project command can coordinate checks and collect their results:

# Proposed commands; this repository does not implement them yet
bin/verify --fast   # annotation freshness + template/architecture checks

bin/verify --full   # fast checks + DB/migration + SQL + system tests

What Each Check Establishes

For each result, I'm asking what it establishes and what remains outside the check:

Open Questions for the Rails Community

Questions I'd like to compare notes on:

1. Experience: Which checks have caught useful problems, and which mostly added friction?
2. Evidence: How do you tell whether an agent workflow improves delivery? What would change your mind?
3. Shared tools: What belongs in each project, and what could Rails provide by default?

Ideas I'm Taking Forward

• 1. Clear acceptance criteria: I want to make “done” explicit and try a workflow proportional to the change.
• 2. Feedback the agent can use: I want to connect project requirements to executable checks.
• 3. Measured outcomes: I want to learn which checks improve delivery, what they miss, and what they cost.

Let's Compare Notes

Salamat, PhRUG!

What has worked for you?
What would you try differently?

Stay in touch on LinkedIn

I write about Ruby and software development, and share what I'm learning.

linkedin.com/in/artem-blazhievskii

Slides, examples & sources on Ruby Overdrive

Artem Blazhievskii • GitHub: @ablzh