building

Problem

QAD's production harness worked because every scenario ran as deterministic code, but that reliability came at a cost: each new scenario needed a developer, a requirements discussion with the SME who owns the business logic, and then a few days of iterating on the script until it behaved correctly. SMEs describe scenarios in loose clinical language and can't write the code themselves. Agentic QAD, a multi-agent system that turns an SME's plain-language description into a verified Python validation script.

Approach

Built a multi-agent system, it starts by Data Profiling Module which understands the context, parses study protocols for candidate checks and profile the study data. With these contents, Standardization Agent hold clarification rounds with the Data Managers via chat mode for data quality check scenario generation, using memory so no question is asked twice, until the scenario becomes a structured, code-ready specification. Before any code is written, a data contract and constrained pseudocode pin down exactly which columns exist, so agents can't invent them. The Code Generation Agent reuses helper functions from a closed set of known auxiliary cases, including one that calls an LLM for clinical judgment.

Verification is where the design matters most. A Testing Agent writes 10 strategic tests without ever seeing the code, everything runs in a sandbox after security checks, and a specialist 'Swarm Agent' repairs the script in case of failures. Scripts that fail more than 20% of tests are killed, not patched endlessly, and the time to add one scenario has dropped from days to a few hours. Simple and medium scenarios pass on the first attempt; complex ones still struggle, which is pushing us toward iterative code generation with Data Managers.

What i learnt: Code generation was quarter of the effort; Standardization: turning vague clinical text into a precise spec was major part of the effort. Every decision an agent has to guess becomes a bug, so the real work is closing those gaps before the first line of code is written.

Outcome

When an LLM writes code instead of making decisions, its output becomes something you can test before it touches data. But agents can only verify code against a spec. Only a human can confirm the spec is right.

Problem

The Subcat Classifier Model caught structural and simple clinical errors, but couldn't reason through complex clinical logic on real data, it reached only about 50% precision.

Approach

Generated training, validation data at scale with LLMs and refined by SMEs to mimic production data.

v1.0 · Llama 3 8B + LoRA: Passed each subject's chunked data as structured JSON so the model could see cross-domain anomalies. Date and value comparisons stayed unreliable on large inputs, even after more fine-tuning since it is 8B model.

v2.0 · DeepSeek-R1-Distill-Llama-70B + QLoRA: Scaled to a larger reasoning model for stronger clinical reasoning. Serving 70B at full precision needed far more GPU than the deployment could justify, so inference ran quantized in FP8. Fine-tuning and serving therefore happened at different numeric precisions, and accuracy collapsed in the served model.

v3.0 · Added chain-of-thought data: Trained the model to reason before each verdict. Accuracy improved, but the quantization drop remained.

v4.0 · Quantization-aware training + concise reasoning: Fixed the quantization drop, cut reasoning tokens by about 40%, and went to production. Math errors returned on large contexts, with high latency and cost on a single self-hosted GPU.

v5.0 · LLM harness: Python runs every deterministic check first, and an enterprise frontier model handles only clinical reasoning. Data managers approve, hold or reject each drafted query.

What i learnt: Fine-tuning, chain-of-thought and quantization work made the model better, not reliable at exact logic over large clinical contexts, the fix was architectural: compute everything computable in code, and leave the LLM only the judgment calls.

Outcome

The final hybrid system reached 97.5% accuracy, 96% precision and 95% recall in production. The best way to use a probabilistic model in production is to make the system around it as deterministic as possible.

Problem

Clinical data managers reviewed trial data by hand to catch discrepancies across domains such as Adverse Events, Concomitant Medications and Labs etc. Built-in EDC edit checks handled simple rules well enough, but not the complex, cross-domain logic a real discrepancy usually involves.

Approach

Built a synthetic data generator around 65 SME-defined discrepancy patterns since no labeled patient data was available due to PHI restrictions. Data generator was seeded with real value lists, producing 65,000 labeled samples that SMEs then reviewed. Each clinical domain got its own classifier, returning the specific pattern it found violated. I compared an MLP against XGBoost; XGBoost won on accuracy, stability across seeds, and explainability. Thresholds were tuned to favor recall, since a missed discrepancy costs far more than a false flag.

What i learnt: Strong offline metrics on synthetic data don't guarantee production performance.

Outcome

Validated at 93% precision and 97% recall on held-out synthetic data, but it failed at the proof-of-concept stage on real data: free text, abbreviations, partial dates and error types the generator had never produced dropped precision to about 50%.

Problem

A small local model does badly on specialist domains zero-shot, and the usual fix is hand-tuning the prompt: reword the instruction, add an example, try again. Prompting is fluid enough that there are always more avenues to refine, which makes it slow, unrepeatable, and impossible to attribute. Without a measured baseline there's no way to know whether a reworded instruction helped or not.

Approach

Used DSPy to treat prompting as a compilation problem: declared a class-based input/output signature instead of a prompt string, wrapped it in a ChainOfThought module so the model reasons before answering, and scored everything with an exact-match metric. Served the models locally with vLLM on a single GPU through an OpenAI-compatible endpoint, which made running a local model fast enough to iterate on without an API subscription.

Measured a zero-shot baseline first, so any gain was attributable, then compiled with BootstrapFewShotWithRandomSearch, it runs the model over the training set, keeps the traces where it answered correctly, and random-searches combinations of those as few-shot demonstrations, scoring each candidate against the metric. Ran the identical pipeline over Llama-3.2-3B-Instruct and Llama-3.1-8B-Instruct to separate the effect of optimisation from the effect of model size.

Outcome

Optimisation lifted the 3B model from 41.7% to 58.57% and the 8B model from 55% to 70%. The number that matters is across the two: the optimised 3B scored 58.57% against the un-optimised 8B's 55%, so compiling the prompt was worth more than the extra 5B parameters.

Problem

Paper application forms get typed into a system by hand which is slow, repetitive, and a quiet source of transcription errors. The same gap applies to digitising archives: an organisation holding years of paper records can't search or report on any of it until each form becomes a database row. OCR alone doesn't solve it: handwriting is unreliable to read, since one misread value makes it useless.

Approach

Exploited the fact that the form template is fixed, so field positions are known in advance and no layout detection is needed. Rendered the PDF, normalised each page to a known size, cropped the eight fields by coordinate, then routed each to the engine that handled it best. Pytesseract for handwritten text, EasyOCR for the digit fields, and a pixel-difference check for the checkbox.

Outcome

Extracts a filled-in form into structured values and writes them straight to the database, so a scan becomes a queryable record. The field positions are a template rather than special-cased logic, so supporting a new form means adding its template once instead of changing the pipeline making it scalable.

Problem

A raw waveform is just amplitude over time, two clips carrying the same emotion can have completely different sample values. The work is reducing each clip to a fixed-length vector that preserves both its frequency and time characteristics.

Approach

Extracted MFCCs per clip with librosa and mean-pooled them across time frames, making clips of different lengths directly comparable. Label-encoded the classes and trained a linear-kernel SVM on an 80/20 split. Also explored spectral centroid and zero-crossing rate as alternative features.

Outcome

The result is a working end-to-end pipeline from waveform to predicted class. It demonstrates the method rather than validating it, a first hands-on pass at audio feature engineering.

Problem

A Statistical Analysis Plan is written by hand from the protocol, and the analysis code is then written by hand from the SAP. Each hop is a fresh human interpretation, so statistical intent drifts from study design and the code drifts from both, with nothing in the chain a machine can check. The standards that would prevent it exist (CDISC USDM for the study, SADM for the analysis), but as long as the artifacts in between are prose, nothing connects them.

Approach

Built the chain as a sequence of governed models instead of documents: protocol to USDM v4.0 to SADM v6.0 to SAP to R/SAS scripts, with SADM as the custom computable blueprint tying protocol requirements to planned analyses to generated code. The extraction pipeline routes each SAP section to the SADM classes it populates and keeps every model call scoped to that section — roughly 25–35 bounded calls per protocol rather than full-document passes — then resolves cross-references deterministically against an ID table and validates cardinality straight from the schema. A React workspace gates each stage behind review, so a statistician approves or corrects the output before it feeds the next one.

Outcome

Won 1st place. The bet behind the design was that traceability has to be structural rather than documented after the fact: because every artifact is generated from SADM, the link from a protocol endpoint to the analysis that tests it survives into the generated code, instead of living in a spreadsheet someone maintains by hand.

Problem

A clinical trial protocol arrives as a hundred-page prose document, so every downstream system that needs it — study build, data collection, registry submission — re-keys the same facts by hand. CDISC USDM exists as the machine-readable standard for exactly this, but nothing turns a written protocol into one.

Approach

Built an LLM pipeline that parses the PDF into structure, maps each protocol section to the USDM schemas it populates, extracts content against those schemas, then connects the fragments into a single linked USDM graph. Put a React dashboard on top with human-in-the-loop review, so a domain expert approves or corrects each extracted field — the part that makes the output usable somewhere regulated rather than merely impressive in a demo.

Outcome

Won the Best Technical Implementation Award. The call that shaped the build was targeting CDISC USDM instead of inventing a bespoke schema: committing to a standard we didn't control made the output interoperable by construction, at the cost of having to satisfy a specification with thousands of defined attributes.