The model has spoken.
Who checks the verdict?
SIGNAL is an evidence-first evaluation studio exploring what happens between an automated decision and a judgment you can actually trust.
Quantitative criteria pass, but claims compliance was not evaluated.
A high score is not proof.
During creator and content operations, I saw why automated screening alone is not enough. Clear-looking scores can conceal gaps in evidence, criteria or judgment.
Missing ≠ failing
A creator’s highest observed video among 20 recent posts is not necessarily their all-time highest. Unknown evidence should stay unknown.
Different rules, different answers
A threshold-only rubric might approve a case that fails a content or compliance condition it never evaluated.
A verdict needs a reviewer
For ambiguous or high-impact decisions, the system needs a way to surface uncertainty and capture a human rationale.
Go inside the system.
Six checkpoints transform raw creator evidence into a traceable decision. Tap each step to see the corresponding SIGNAL interface and its role in the workflow.
Built with Cloudflare Workers, Google Apps Script and Google Sheets. Optional LLM and external metric providers are separate integrations.
Provide a creator profile, campaign requirements and a video transcript or caption. Imported metrics are available only with a connected provider; this screenshot uses a synthetic case.
ACTUAL PROTOTYPE SCREENSHOT · INPUT PANELOne case.
Two rubrics.
A different verdict.
Try the exact type of failure the prototype is designed to surface: the numeric requirements pass, but the first rubric forgets to check a risky product claim.
“This serum cures acne overnight for everyone — 100% guaranteed.”
✓ within sample tier
✓ above 50K threshold
✓ meets sample condition
✓ under $800 cap
Synthetic creator and campaign example. These fields are fabricated for evaluation testing, not scraped live from TikTok.
All active requirements pass. Missing claims gate is an untested quality risk.
Content-risk detection is not independent medical-claim fact checking. Both verdicts are deterministic prototype logic, not real model accuracy or API inference.
What changed —
and what didn't.
Two distinct kinds of evidence belong in this story. SIGNAL's own synthetic test results are kept separate from my professional operational experience.
One error fixed by adding the missing claims-compliance condition. Same five synthetic fixtures before and after; no held-out validation.
The initial rubric did not evaluate a potential unsupported product claim. A human reference label disagreed. The updated rubric changed that example's decision.
| Synthetic test | V1 | V2 | Human label |
|---|---|---|---|
| All criteria met | Approve | Approve | Approve |
| Missing views | Review | Review | Review |
| Fee exceeds cap | Reject | Reject | Reject |
| Claims oversight | Approve | Reject | Reject |
| Viral history unknown | Review | Review | Review |
Illustrative five-case matrix. A matched label is not proof of real-world model reliability.
The experience behind the experiment.
PROFESSIONAL WORK · NOT CAUSED BY SIGNALEvaluated TikTok/Instagram profiles using audience, relevance and historical performance criteria.
AI-assisted Airtable reporting reduced ~5 hours to 2 hours per ~30 creators.
Historical aftersales data across 16 dealerships to examine retention patterns.
Results from a 21-creator campaign; not a metric produced by SIGNAL.
Built to expose uncertainty,
not cover it up.
The design uses a lightweight web stack and deliberately keeps rule-based checks, optional AI assessment and reviewer judgment as separate layers.
Architecture at a glance
UI + API gateway
Workflow logic
Cases + audit
Rule-based criteria operate on recorded fields. Optional third-party providers supply public creator data when available; an optional external AI API can evaluate supplied text. Neither integration is required to show the offline synthetic case.
What remains unproven
The interesting part begins when automation is wrong.
I built SIGNAL to learn how to turn operational judgment into transparent evaluation criteria—and how to keep humans accountable for the final call.