wall_01 quality
Not good enough.
The agent works in the demo and fails on your real cases. I build the harness around your domain (tools, memory, guardrails, evals) and fine-tune when prompting runs out.
→ Harnesses AI Systems Engineer AI from the tokenizer up Marbella, ES · Local time
When the API isn’t enough.
I build custom inference, agent harnesses and fine-tuned models, from the tokenizer up. I write the code myself.
30-minute technical call with the engineer who builds it. No sales team, no slides.
- years writing software
- 28
- years writing software
- open-source repos
- 7
- open-source repos
- where I train and benchmark
- 1× 4090
- where I train and benchmark
$ git clone github.com/jtrefon/llama.cpp-turboq-mtp# fork of llama.cpp-turboq · TBQ4 KV · MTP# SWA stability in Gated DeltaNet · hot model loadingcontext 262K tokensKV cache 4.125 bpv, TBQ4hardware 1× RTX 4090✓ peak decode 82 tok/s
Open-source proof
§00github.com/jtrefon
§01 Three walls n=3
Most teams start with an API. Some hit a wall.
wall_02 privacy
Not private enough.
The data can’t leave your building, your country, or your customer’s network. I build inference that runs where the data lives: on-prem, in your cloud account, or air-gapped.
→ Private LLM wall_03 cost
Not cheap or fast enough.
The token bill grows faster than revenue, or latency kills the experience. I tune the serving stack, quantise, or train a smaller model that does your job at a fraction of the cost.
→ InferenceIf the API works and the bill is fine, keep the API. I’ll tell you that on the call.
§02 What I build L0 → L3
Models. Inference. Harnesses.
Every layer, top to bottom. Most teams buy the top one. I build all four.
- L3
Harnesses
Agents built from the ground up for one job, done reliably.
Agents built from the ground up for one job, done reliably.
- tools
- memory
- provider routing
- guardrails
- human handoff
- evals
- L2
Inference
Serving stacks that are private, fast and cheap to run.
Serving stacks that are private, fast and cheap to run.
- serving
- quantisation
- KV cache
- MTP
- OpenAI-compatible API
- L1
Models
Fine-tuning, distillation, reinforcement learning, training from scratch.
Fine-tuning, distillation, reinforcement learning, training from scratch.
- tokenizer
- pre-training
- fine-tuning
- distillation
- RL
- L0
Hardware
Your GPUs, your cloud account, or an air-gapped rack.
Your GPUs, your cloud account, or an air-gapped rack.
- CUDA
- Metal
- on-prem
- air-gapped
§03 Built in your stack
Not everything is Python.
I’ve shipped agents in PHP inside a live app, a multi-provider agent platform in .NET with Angular and React, a harness in C++, an IDE in Swift, and tooling in Rust. The AI fits your architecture, not the other way round.
- C++ llama.cpp fork · amber
- Rust blt tokenizer · distiller
- Swift compass IDE
- Python training · llm-mini · lora
- C# .NET off-plan agent platform
- PHP inbound sales agents
- TypeScript React chat agent · Angular back office
§04 Selected work n=4
Built and handed over.
- Harness handed over
Off-plan real estate, Dubai
Agentic sales platform: .NET core and API, Angular back office, React chat agent, multiple LLM providers routed through OpenRouter.
- .NET
- Angular
- React
- OpenRouter
- Harness handed over
Inbound sales agents, several markets
Agents written in PHP and deployed inside the client’s own application, wired to local scripts, SOPs and domain knowledge, running on the Gemini API. Fast, cheap and reliable.
- PHP
- Gemini API
- Models ongoing · NDA
Document extraction, regulated accounting client
Vision-model extraction pipeline. Ongoing, under NDA.
- vision model
- Python
- Models training
compass model (in progress, open source)
A 4.5B agentic coding model: Qwen fine-tuned for terminal and agent work, then distilled and RL-tuned per language and stack, trained on one RTX 4090, to ship inside compass.
- Qwen
- RL
- distillation
- RTX 4090
§05 How I work pipeline
Teardown. Build. Hand over.
- 01 1–2 weeks
Teardown
One to two weeks. I profile your stack, run evals on your data, and hand you a written plan with numbers: what to build, what to keep, what to kill.
out
- profile
- evals on your data
- written plan
- 02 scoped
Build
I deliver the harness, inference stack or model, with evals, benchmarks, runbook and handover. We agree what success looks like in writing before I start.
out
- code
- evals
- benchmarks
- runbook
- 03 on request
Come back when needed
Fixes and extensions after handover, on request.
out
- fixes
- extensions
I build it, prove it, and hand it over. You own it, not a dependency on me.
Every build is different, so every quote is too. Fixed price or time and materials, scoped on the first call.
§06 Who builds it
I write the code.
I’m Jacek Trefon. 28 years writing software, years as CTO and Head of Engineering, and the last few building models, inference engines and harnesses. I work alone, on purpose: you talk to the person who writes the code.
- years writing software
- 28
- years writing software
- engineers led at CAFU
- 100+
- engineers led at CAFU
- regions, one platform at Logiswift
- 4
- regions, one platform at Logiswift
§07 articles ~/articles
Written from first principles.
- Harnesses10 minAgent context management: budgets, truncation, compaction, and knowing when to stopA 262K context isn't a licence to dump everything in. Each token of context costs speed on every step. How to budget context for an agent, truncate tool output without losing the signal, compact history without breaking the prefix cache, isolate work in sub-agents, and detect loops before they burn the budget.
- Harnesses9 minAgent eval statistics: is 816 vs 808 a real difference?Leaderboards rank models by a few points; the noise is usually bigger. Confidence intervals on pass rates, worked on my own 49-scenario benchmark, how many scenarios you'd need to separate two models, why paired tests are worth the effort, and why pass^k beats pass@k for agents.
- Inference10 minAttention from the kernel's side: FlashAttention, online softmax and fused dequantAttention is three lines of math and one big memory problem. How the online softmax makes tiling exact, what FlashAttention never writes to memory, why decode attention is a stream over the KV cache, and why a quantised cache only pays off when dequant is fused into the kernel.
- Architectures12 minAttention Is All You Need: the paper, the idea, and what it changedThe 2017 Google paper that introduced the transformer: the problem it set out to solve, what it actually proposed (scaled dot-product attention, multi-head attention, sinusoidal positions, an encoder-decoder for translation), the numbers it reported, what later models changed, and how a translation paper became the basis of modern AI.
Bring the problem the API couldn’t solve.
Describe your problem30-minute technical call with the engineer who builds it. No sales team, no slides.