What is a “virtual filesystem” for agents

The term “virtual filesystem” is traditionally used in the context of operating systems. There, a virtual filesystem is the layer that lets every program use the same open, read, and write calls whether it’s reading from an SSD, a USB stick, or a network share.

In the context of AI agents, a virtual filesystem describes a filesystem-shaped interface on top of persistent storage, such as a relational database, a vector store, or, as in this case, Elasticsearch. This lets the agent use ls, cat, find, and grep over the stored data. The agent running, for example, grep -r "access_token" /docs is searching a filesystem and doesn’t know it’s interacting with a search index. Thus, commands like grep become an interface, the implementation of which can make use of powerful search features, such as vector search or hybrid search.

Architecture Overview

The resulting implementation of the ElasticsearchFs virtual filesystem has four layers, similar to the Mintlify blog:

1. Agent layer: The LLM agent runs shell commands via tool calls.
   
2. Shell layer: just-bash is a TypeScript library that intercepts those shell command strings, parses flags, handles pipes, and dispatches to a IFileSystem interface.
   
3. ElasticsearchFs layer: Implements the IFileSystem interface against the underlying data layer.
4. Data layer: Data is stored in an Elasticsearch Serverless cluster.

In addition to the implementation described in the Mintlify blog, we also consider the following design guidelines for virtual filesystems described in the LangChain documentation:

• Absolute path handling and normalization. Paths are always absolute. normalizePath ensures any path the agent passes is canonicalized before any tree lookup or Elasticsearch call (e.g., auth/oauth becomes /auth/oauth).
  
• Implement ls and glob efficiently (server-side filtering where available, otherwise local filter). Both operations run in-memory after session boot without any Elasticsearch calls.
• Handle errors explicitly: LangChain’s guideline recommends structured result types with an error field for missing files or invalid patterns. Since our implementation builds on top of just-bash, we use POSIX-style ErrnoException errors (ENOENT, ENOTDIR, EROFS). Additionally, we follow Mintlify’s read-only design and thus every write operation throws EROFS.

Implementation details

The implementation splits into four areas: access control, filesystem navigation, file reading, and file search. You can find the full implementation in the elasticsearch-fs repository.

{
  "PUBLIC": {
    "cluster": [],
    "indices": [
      {
        "names": ["elasticsearchfs-chunks"],
        "privileges": ["read"],
        "query": {
          "bool": {
            "should": [
              { "prefix": { "slug": "auth/" } },
              { "prefix": { "slug": "api-reference/search-use-case/" } },
              { "term": { "slug": "api-reference/users" } }
            ],
            "minimum_should_match": 1
          }
        }
      },
      {
        "names": ["elasticsearchfs-meta"],
        "privileges": ["read"]
      }
    ]
  }
}

const doc = await client.get({
  index: 'elasticsearchfs-meta',
  id: '__path_tree__',
});

const json = Buffer.from(payload, 'base64').toString('utf8');
const pathTree = JSON.parse(json);

async readFile(path: string): Promise<string> {
  const slug = this.resolveReadFileSlug(path);

  const results = await this.client.search<FileHitSource>({
    index: ELASTICSEARCHFS_FILES_INDEX,
    size: 1,
    _source: ['content'],
    query: {
      bool: {
        filter: [{ term: { slug } }],
      },
    },
  });

  const hit = results.hits.hits[0];
  const content = hit?._source?.content;
  if (content === undefined) {
    throw enoent();
  }

  return content;
}

{
  "query": {
    "bool": {
      "filter": [{ "terms": { "slug": ["<slugs-in-scope>"] } }],
      "must": [{ "match_phrase": { "content": "<literal-pattern>" } }]
    }
  }
}

{
  "query": {
    "bool": {
      "filter": [{ "terms": { "slug": ["<slugs-in-scope>"] } }],
      "must": [
        {
          "regexp": {
            "content.pattern": {
              "value": ".*(<escaped-literal-pattern>).*",
              "case_insensitive": false
            }
          }
        }
      ]
    }
  }
}

{
  "query": {
    "bool": {
      "filter": [{ "terms": { "slug": ["<slugs-in-scope>"] } }],
      "must": [
        {
          "regexp": {
            "content.pattern": {
              "value": ".*(<regex-pattern>).*"
            }
          }
        }
      ]
    }
  }
}

  // Coarse Filter: Ask backing store for slugs matching the string/regex
  const matchedSlugs = await elasticsearchFs.findMatchingFiles(
      coarseFilter,
      slugsUnderDirs,
    );

  if (matchedSlugs.length === 0) return { stdout: '', stderr: '', exitCode: 1 };

  // Fine Filter: Narrow to resolved hit paths.
  const matchedPaths = matchedSlugs.map((slug) => slugToPath(slug));

  // Exec: Let the in-memory RegExp engine format the final output
  return execBuiltin(
    scannedArgs,
    matchedPaths,
    elasticsearchFs,
    shouldPrefixFilePath,
  );

Known Limitations

This implementation is a POC and has the following known limitations:

• Coarse grep can produce false negatives The coarse stage uses Elasticsearch/Lucene regexp syntax while the fine stage uses JavaScript RegExp. This mismatch can produce false negatives before validation.
  
• No bulk-prefetch. Mintlify’s grep coarse stage identifies candidate files in the data store, then bulk-fetches all matching chunks into a cache before the fine-filter pass, so large recursive queries complete in milliseconds. ElasticsearchFs reads each candidate file individually via readFile, with no in-process cache.
  
• stat returns placeholder metadata. size is always 0 and mtime is always epoch — stat makes no Elasticsearch call. Session boot loads only path-tree metadata (and not per-file timestamps/sizes); although updated_at exists on documents, this implementation does not materialize it into the in-memory tree. The practical consequence: ls -l, find -mtime, and find -size are not truthful.
  
• .mdx-only file type. The path tree is keyed on /<slug>.mdx exclusively. Files with any other extension cannot be represented in the tree and are invisible to the agent.

Trying it out

You can use the virtual filesystem over Elasticsearch as follows:

import { ElasticsearchFs } from '../src/core/elasticsearchfs.js';
import { runElasticGrep } from '../src/core/grep.js';
import { Bash, defineCommand } from 'just-bash';
import { createESClient } from '../src/es-adapter/client.js';
import { initSessionTree } from '../src/session.js';

// 1. Initialize ES client with the user profile
const profile = "PUBLIC"
const client = createESClient(profile);

// 2. Initialize the path tree 
const session_tree = await initSessionTree(client, profile);

// 3. Initialize the virtual filesystem
const elasticsearchFs = new ElasticsearchFs({
  client: client,
  files: session_tree.files,
  dirs: session_tree.dirs,
});

// 4. Define custom grep command
const grep = defineCommand('grep', async (args, ctx) =>
  runElasticGrep(args, ctx, elasticsearchFs),
);

// 5. Initialize Bash instance over Elasticsearch backend with custom grep command 
const bash = new Bash({
  fs: elasticsearchFs,
  cwd: '/',
  customCommands: [grep],
});

// 6. Usage examples
await bash.exec('grep -ri "OAuth" /auth');
await bash.exec('cat /auth/oauth.mdx');
await bash.exec('ls /api-reference');

Summary

This blog implements ElasticsearchFs, a virtual filesystem over Elasticsearch. It is built on top of just-bash and exposes cat, ls, cd, find, and grep for an agent while keeping access control close to the data. This POC has not been benchmarked. The goal is to explore the architecture and implementation, not to validate latency or cost claims.

The code is available in the repository. For a more production-ready implementation, the next upgrades could be an in-process/remote cache for repeated file reads during grep-heavy sessions or a semantic search alternative to grep. Note that the latter would require additional chunking on the files and reassembly at query time.

References

• Mintlify. How we built a virtual filesystem for our assistant
  
• Vercel Labs. just-bash GitHub repository
  
• LangChain. Design guidelines: Use a virtual filesystem

Prerequisites

To run this tutorial as a notebook, you will need:

• GPU for fine-tuning .If you don’t have one locally, you can run this notebook for free on Google Colab using a free NVIDIA T4 GPU instance or on Kaggle
• Optional: HF_TOKEN for faster downloads of the training dataset from Hugging Face.

Note: This notebook as trained on a free Colab notebook using a NVIDIA T4 GPU. (Torch: 2.10.0+cu128, CUDA: 7.5., CUDA Toolkit: 12.8., Triton: 3.6.0)

import torch

assert torch.cuda.is_available(), "No GPU detected. Make sure you've enabled GPU in Colab: Runtime > Change runtime type > T4 GPU"

Setup

Install the required packages (Unsloth v2026.4.8, Transformers v4.57.6, vLLM v0.19.1, trl v0.24.0).

!pip install -qU unsloth trl vllm transformers datasets matplotlib

Additionally, we will set UNSLOTH_VLLM_STANDBY, which enables a lower-VRAM standby mode and reduces GPU memory usage. We will also fix the random seed.

import os
os.environ['UNSLOTH_VLLM_STANDBY'] = "1" # Unsloth Standby reduces VRAM by 30%+

SEED = 42

Load Model and Tokenizer

We load LFM2.5-1.2B-Instruct as the base model, including the tokenizer. Then we apply LoRA adapters.

from unsloth import FastLanguageModel

MODEL_NAME     = "unsloth/LFM2.5-1.2B-Instruct"
max_seq_length = 4096   # Must be larger than prompt_length for max_new_tokens to be greater than 0
lora_rank      = 32#16    # # Choose any number > 0 ! Suggested 8, 16, 32, 64, 128 (higher rank = smarter, but slower)

# Load model
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name      = MODEL_NAME,
    max_seq_length  = max_seq_length,
    load_in_4bit = False,       # False for LoRA 16bit
    fast_inference = False,     # Enable vLLM fast inference
    max_lora_rank = lora_rank,
    load_in_fp8 = False,        # Float8 RL / GRPO!
)

# Inject LoRA adapters into attention and MLP layers
model = FastLanguageModel.get_peft_model(
    model,
    r = lora_rank,
    target_modules = [
        "q_proj", "k_proj", "v_proj", "out_proj",   # Attention layers
        "in_proj",                                  # Conv layers
        "w1", "w2", "w3",                           # FFN
    ],
    lora_alpha = lora_rank*2,              # Scaling factor for LoRA updates
    use_gradient_checkpointing = "unsloth",        # Reduces memory usage 
    random_state = SEED,
)

# Count trainable parameters
trainable = sum(p.numel() for p in model.parameters() if p.requires_grad)
total     = sum(p.numel() for p in model.parameters())
print(f"Trainable params : {trainable:,} ({100*trainable/total:.1f}% of total)")
print(f"Total params     : {total:,}")

🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.

2026-05-04 19:06:21.934230: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:467] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1777921582.153165      57 cuda_dnn.cc:8579] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1777921582.216471      57 cuda_blas.cc:1407] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
W0000 00:00:1777921582.738130      57 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.
W0000 00:00:1777921582.738168      57 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.
W0000 00:00:1777921582.738171      57 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.
W0000 00:00:1777921582.738173      57 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.

🦥 Unsloth Zoo will now patch everything to make training faster!
==((====))==  Unsloth 2026.4.8: Fast Lfm2 patching. Transformers: 4.57.6. vLLM: 0.19.1.
   \\   /|    Tesla T4. Num GPUs = 2. Max memory: 14.563 GB. Platform: Linux.
O^O/ \_/ \    Torch: 2.10.0+cu128. CUDA: 7.5. CUDA Toolkit: 12.8. Triton: 3.6.0
\        /    Bfloat16 = FALSE. FA [Xformers = 0.0.35. FA2 = False]
 "-____-"     Free license: http://github.com/unslothai/unsloth
Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored!
Unsloth: QLoRA and full finetuning all not selected. Switching to 16bit LoRA.

Trainable params : 22,216,704 (1.9% of total)
Total params     : 1,192,557,312

Data Preparation

For this tutorial, we use the Navneetkumar11/rvl-cdip-invoice-extracted dataset from Hugging Face. This is a OCR extraction dataset. Each example contains raw OCR text from a scanned invoice plus a normalized JSON extraction target.

Let’s load and preprocess the dataset. We train on a compact invoice header schema with only two keys invoice_date and total_amount.

from datasets import load_dataset
import json

dataset = load_dataset("Navneetkumar11/rvl-cdip-invoice-extracted", split="train")
dataset = dataset.filter(lambda ex: ex["extraction_confidence"] == "high")

def format_dataset(example):
    try:
        extracted = json.loads(example["extracted"])
    except (TypeError, ValueError, json.JSONDecodeError):
        extracted = None

    return {
        "text": example["raw_ocr_text"],
        "ground_truth_invoice_dates": str(extracted.get("invoice_date")).strip() if extracted.get("invoice_date") else None,
        "ground_truth_total_amounts": float(extracted.get("total_amount")) if extracted.get("total_amount") else None,
    }

dataset = dataset.map(format_dataset)

# Filter out entries with missing values
dataset = dataset.filter(lambda ex: all(ex[k] is not None for k in [ "ground_truth_invoice_dates", "ground_truth_total_amounts"]))

# Remove unused columns
dataset = dataset.remove_columns(
    [c for c in dataset.column_names if c not in [
        "text",
        "ground_truth_invoice_dates",
        "ground_truth_total_amounts",
    ]]
)

# Shuffle and split data into training and evaluation datasets
dataset = dataset.shuffle(seed=SEED)
train_ds = dataset.select(range(1500))
eval_ds = dataset.select(range(1500, 1600))

dataset

Dataset({
    features: ['text', 'ground_truth_invoice_dates', 'ground_truth_total_amounts'],
    num_rows: 2404
})

dataset[0]

{'text': 'P\nROMOTIONS\nON THE OCEAN INC.\n4205 Pleasant Valley Rd.\nSuite 167\nRaleigh, NC 27612\nAugust 1, 1995\nPROMOTIONS ON THE OCEAN, INC.\nINVOICE # LOR 3-95\nLORILLARD PO # 3119\nDate:\nJune 1-June 31, 1995\nEvent/Location: Newport Summer Nightclub Promotion/Virginia Beach, VA\nDescription:\nNightclub Sampling & Promotional Activities\nCasting, selection & training of models for sampling and activities.\n# Days:\n9\nAvg. duration of days:\n3.5 Hrs.\nAvg. # Supervisors:\n2\nAvg. # Supervisors day rate: $73.50\nTOTAL COSTS: $1,323.00\nPor 8/3\n18 stulas\nLegit\n93114516\nNorth Myrtle Beach, SC - Virginia Beach, VA - Myrtle Beach, SC :unselected:',
 'ground_truth_invoice_dates': '1995-08-01',
 'ground_truth_total_amounts': 1323.0}

Format Prompts (Chat Template)

Because GRPO’s trainer expects a dataset where each row has a "prompt" field, we will format the prompts using a chat template. LFM2.5 uses a ChatML-style format, where every conversation is wrapped in special tokens:

<|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
What is 2+2?<|im_end|>
<|im_start|>assistant

The model then generates from the assistant position onward.

In the system prompt we define how the model should the exact schema, how to handle missing fields, and what normalization to apply.

SYSTEM_PROMPT = """You extract structured invoice header data from OCR text.

Return a JSON object with exactly this kes and no others:
- "invoice_date": string in YYYY-MM-DD format or null
- "total_amount": JSON number or null

Rules:
- Output exactly one JSON object.
- Do not include markdown fences.
- Do not include explanations.
- Do not include any extra keys.
- Do not invent values.
- Do not use commas in numbers.
- total_amount must be a JSON number, not a string.
- Use null when a field is missing or unclear.
"""

def format_prompt(example):
    """Convert a dataset row into a ChatML-formatted prompt string."""
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user",   "content": example["text"]},
    ]

    prompt = tokenizer.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True
    )
    return {"prompt": prompt}

train_ds = train_ds.map(format_prompt)
eval_ds  = eval_ds.map(format_prompt)

Below you can see an example of the prompt field using the ChatML template:

ex = train_ds[0]
print("--- Raw OCR text (input) ---")
print(ex["text"][:300])

print("\n--- Formatted prompt (what the model sees) ---")
print(ex["prompt"][:500])

--- Raw OCR text (input) ---
P
ROMOTIONS
ON THE OCEAN INC.
4205 Pleasant Valley Rd.
Suite 167
Raleigh, NC 27612
August 1, 1995
PROMOTIONS ON THE OCEAN, INC.
INVOICE # LOR 3-95
LORILLARD PO # 3119
Date:
June 1-June 31, 1995
Event/Location: Newport Summer Nightclub Promotion/Virginia Beach, VA
Description:
Nightclub Sampling & Pr

--- Formatted prompt (what the model sees) ---
<|startoftext|><|im_start|>system
You extract structured invoice header data from OCR text.

Return a JSON object with exactly this kes and no others:
- "invoice_date": string in YYYY-MM-DD format or null
- "total_amount": JSON number or null

Rules:
- Output exactly one JSON object.
- Do not include markdown fences.
- Do not include explanations.
- Do not include any extra keys.
- Do not invent values.
- Do not use commas in numbers.
- total_amount must be a JSON number, not a string.
- Use nul

Define the Reward functions

Reward functions are the key ingredients for GRPO. They let us know if the model doing well or not.

For this example, we will define three reward functions, each with a distinct learning signal:

• JSON structure: did the model return the right schema?
• Field presence: did it contain the correct fields?
• Field quality: are the values correct? If not, are they close to the desired value?

General best practices for reward functions are:

• Provide clear signals with partial credit where possible
• Be deterministic and consistent
• Execute quickly for training efficiency
• Fail gracefully when the model emits malformed JSON

import json
import re

def _clean_json(text: str) -> str:
    text = text.strip()

    # Strip code fences
    if text.startswith("```"):
        text = re.sub(r"^```(?:json)?\s*", "", text, flags=re.IGNORECASE)
        text = re.sub(r"\s*```$", "", text)
    text = text.strip()

    # Extract JSON object by braces
    start, end = text.find("{"), text.rfind("}")
    if start != -1 and end != -1 and end > start:
        text = text[start:end+1]

    return text.strip()
    
def reward_valid_json(completions, **kwargs):
    scores = []
    for completion in completions:
        response = completion if isinstance(completion, str) else completion[0]["content"]

        try:
            json.loads(response)
            scores.append(1.0)        # clean valid JSON
        except json.JSONDecodeError:
            # Second pass: try light cleanup for common model/OCR numeric issues
            try:
                json.loads(_clean_json(response))
                scores.append(0.5)    # valid after cleaning (had ``` etc.)
            except json.JSONDecodeError:
                scores.append(0.0)    # invalid JSON
    return scores

EXPECTED_KEYS = {"invoice_date", "total_amount"}

def reward_correct_keys(completions, **kwargs):
    scores = []
    for completion in completions:
        response = completion if isinstance(completion, str) else completion[0]["content"]
        response = _clean_json(response)
        try:
            data = json.loads(response)
            keys = set(data.keys())
            if keys == EXPECTED_KEYS:
                scores.append(1.0)    # exact match
            elif EXPECTED_KEYS.issubset(keys):
                scores.append(0.5)    # correct keys + extra keys
            elif EXPECTED_KEYS & keys:
                scores.append(0.2)    # some correct keys
            else:
                scores.append(0.0)
        except (json.JSONDecodeError, AttributeError):
            scores.append(0.0)
    return scores

from datetime import datetime
from dateutil import parser

def reward_correct_values(completions, 
                          ground_truth_invoice_dates, 
                          ground_truth_total_amounts, 
                          **kwargs):
    scores = []
    for completion, date_gt, amount_gt in zip(
        completions, ground_truth_invoice_dates, ground_truth_total_amounts
    ):
        response = completion if isinstance(completion, str) else completion[0]["content"]
        response = _clean_json(response)

        try:
            data = json.loads(response)
        except (json.JSONDecodeError, AttributeError):
            scores.append(0.0)
            continue

        score = 0.0

        # Invoice date: ISO format + partial credit per matching component
        date_str = str(data.get("invoice_date", ""))
        gt_parsed = datetime.strptime(date_gt, "%Y-%m-%d")
        
        try:
            # Follows the correct format
            parsed_date = datetime.strptime(date_str, "%Y-%m-%d")
            score += 0.1
        except ValueError:
            try:
                # Doesn't follow the correct format but is a valid date
                parsed_date = parser.parse(date_str.strip(), fuzzy=False)
            except Exception:
                parsed_date = None

        if parsed_date == gt_parsed: # Exact match
            score +=0.9
        else:
            if parsed_date: # Approximate match
                if parsed_date.year  == gt_parsed.year:  score += 0.1
                if parsed_date.month == gt_parsed.month: score += 0.1
                if parsed_date.day   == gt_parsed.day:   score += 0.1
        
        # Total amount: scaled by closeness to ground truth
        try:
            amount = float(data.get("total_amount"))
            score += 0.1
            gt_amount = float(amount_gt)
            if gt_amount == 0: # div0 check
                amount_score = 1.0 if amount == 0 else 0.0
            else:
                deviation = abs(amount - gt_amount) / abs(gt_amount)
                amount_score = max(0.0, 1.0 - deviation)
            score += (0.2) * amount_score
            
            if amount == gt_amount: # Exact match
                score += 0.7
        except (TypeError, ValueError, AttributeError):
            pass  # not a valid number, no credit

        score /= 2
        scores.append(score)
    return scores

A(i) = (r(i) - mean(r)) / std(r)

Configure and Run GRPO Training

Now we configure the GRPO training hyperparameters with the following key configurations:

• num_generations: how many responses per prompt. Higher values lead to better advantage estimates, but linearly more VRAM and compute.
• beta: penalty for diverging from the reference policy. Higher → model stays closer to base model. Lower → more aggressive learning (risk of collapse).
• learning_rate: very low because the LoRA weights are already close to useful.
• max_steps: total number of training steps.

from vllm import SamplingParams
from trl import GRPOConfig
import numpy as np

def prompt_token_length(example):
    token_ids = tokenizer(example["prompt"], add_special_tokens=False)["input_ids"]
    return {"prompt_length_tokens": len(token_ids)}

tokenized = train_ds.map(prompt_token_length)

maximum_length = int(np.quantile(tokenized["prompt_length_tokens"], 0.9))
print("90th percentile prompt length (tokens) =", maximum_length)
print("Max completion length =", min(max_seq_length - maximum_length - 1, 128))

max_prompt_length = maximum_length + 1 # + 1 just in case!
max_completion_length = min(max_seq_length - max_prompt_length, 128)

vllm_sampling_params = SamplingParams(
    min_p = 0.1,
    top_p = 1.0,
    top_k = -1,
    seed = SEED,
    stop = [tokenizer.eos_token],
    include_stop_str_in_output = True,
)

training_args = GRPOConfig(
    vllm_sampling_params = vllm_sampling_params,
    temperature = 1.0,
    learning_rate = 5e-6,
    weight_decay = 0.01,
    warmup_steps = 15,
    lr_scheduler_type = "linear",
    optim = "adamw_8bit",
    logging_steps = 1,
    per_device_train_batch_size = 4,
    gradient_accumulation_steps = 2, # Increase to 4 for smoother training
    num_generations = 8, # Decrease if out of memory
    beta = 0.1,  # increase kl_coef from default 0.04 to penalize KL drift more
    max_prompt_length = max_prompt_length,
    max_completion_length = max_completion_length,
    # num_train_epochs = 1, # Set to 1 for a full training run
    max_steps = 250,
    save_steps = 100,
    report_to = "none", # Can use Weights & Biases
    output_dir = "outputs",

    # For optional training + evaluation
    # fp16_full_eval = True,
    # per_device_eval_batch_size = 4,
    # eval_accumulation_steps = 1,
    # eval_strategy = "steps",
    # eval_steps = 1,
)

90th percentile prompt length (tokens) = 730
Max completion length = 128

Now we wire everything together in the GRPOTrainer. During training the GRPO trainer handles sampling G completions per prompt, calling the reward function, computing advantages, computing the GRPO policy gradient loss and KL penalty, and updating the LoRA weights.

Instead of consolidating the three reward functions into a single reward_fn, we pass them separately for more transparent logging. We also add some weighting to the reward functions, since reward_valid_json and reward_correct_keys saturate quite quickly.

from trl import GRPOTrainer

trainer = GRPOTrainer(
    model = model,
    processing_class = tokenizer,
    reward_funcs = [reward_valid_json,
                    reward_correct_keys,
                    reward_correct_values],
    reward_weights=[0.5, 0.5, 2.0],  # upweight values
    args = training_args,
    train_dataset = train_ds,

    # For optional training + evaluation
    # train_dataset = new_dataset["train"],
    # eval_dataset = new_dataset["test"],
)

trainer.train()

What a healthy GRPO training curve looks like:

• Reward mean should rise steadily
• Reward std should stay non-zero (if it collapses to 0, all responses in every group are tied, the advantage = 0, and the gradient vanishes)
• KL divergence should grow slowly and stabilize. Spikes indicates the model has diverged too far from the reference policy. This means, either the learning rate is too high or kl_coef is too low

These curves show that the model learns the output format very quickly: reward_valid_json and reward_correct_keys stay close to 1, so most generations are valid JSON with the right schema. The noisier `reward_correct_values curve tells the more interesting story, since extracting the exact invoice date and amount from OCR is the hard part and remains the main source of variation throughout training. The KL curve rises with a few spikes, which is normal here and suggests the model is changing meaningfully from the base policy without becoming obviously unstable.

A natural next step would be to strengthen the reward function so it reflects real extraction quality more sharply, especially by rewarding exact value matches more and giving less credit to partially correct but still misleading dates or amounts. After that, I’d try a slightly longer run with a bit more data or a cleaner subset of the OCR examples, since this model already seems to have learned the JSON format and now mostly needs help improving value accuracy.

Save Checkpoint

Before evaluation, we will save the fine-tuned model. For this, we only have to save the LoRA adapter, as the base model weights remain unchanged and do not need to be saved.

FINAL_ADAPTER_PATH = "./grpo_lfm2.5_invoice_adapter"

# Save the final LoRA adapter from the current fine-tuned model
model.save_pretrained(FINAL_ADAPTER_PATH)
tokenizer.save_pretrained(FINAL_ADAPTER_PATH)

('./grpo_lfm2.5_invoice_adapter/tokenizer_config.json',
 './grpo_lfm2.5_invoice_adapter/special_tokens_map.json',
 './grpo_lfm2.5_invoice_adapter/chat_template.jinja',
 './grpo_lfm2.5_invoice_adapter/tokenizer.json')

Inference and Evaluation

Let’s compare the base model and the GRPO fine-tuned model on a few held-out examples from eval_ds to see if the fine-tuned model actually is an improvement over the base model.

For this, we first load both the base model and the fine-tuned model.

::: {#1f3a914c-f946-4d9c-8f13-af8925f29e3f .cell _kg_hide-output=‘true’ quarto-private-1=‘{“key”:“execution”,“value”:{“iopub.execute_input”:“2026-05-04T19:36:25.680873Z”,“iopub.status.busy”:“2026-05-04T19:36:25.679754Z”,“iopub.status.idle”:“2026-05-04T19:37:05.013941Z”,“shell.execute_reply”:“2026-05-04T19:37:05.013289Z”,“shell.execute_reply.started”:“2026-05-04T19:36:25.680807Z”}}’ trusted=‘true’}

# Load a fresh base model for comparison
base_model, base_tokenizer = FastLanguageModel.from_pretrained(
    model_name=MODEL_NAME,
    max_seq_length=max_seq_length,
    load_in_4bit=True,
)

FastLanguageModel.for_inference(base_model)

# Load the saved fine-tuned adapter as a fresh model for comparison
ft_model, ft_tokenizer = FastLanguageModel.from_pretrained(
    model_name=FINAL_ADAPTER_PATH,
    max_seq_length=max_seq_length,
    load_in_4bit=True,
)

FastLanguageModel.for_inference(ft_model)

:::

Let’s look at a few examples:

==================================================
Example 1

Ground truth:
"CUSTOMER NO.\nLENGER\nDTV.\nVENDOR NO.\nCREDIT DATE\n1276900000\n62\n0616\n0000013554\n01/20/95\nSOLD TO\nSHIP TO\nFAIRMONT WHOLESALE INC\nFAIRMONT WHOLESALE INC\nPO BOX 905\n000 NO NORTH AVE\nFAIRMONT\nMN56031\nFAIRMONT\nHN56031\nTHIS IS NOT A CREDIT MEMO - CHECK ATTACHED\nQUANTITY\n(IN THOUSANDS)\nBRAND NAME\nAMOUNT\n6\nTRUE HEN K\n335,79\n6\nTRUE MEN 1\n335,70\n6\nSTYLE LT I\n335,70\n6\nSTYLE MNLT\n335,70\n6\nSTYLE LT M\n335.70\n1.2\nSTYLE LT F\n671,40\nTOTALS-\n42\n2,349.90\nLORILLARD PLUS DISBURSEMENT IS\n$1.30 OF TOTAL QUANTITY - THANK YOU\nGROSS AMOUNT -\n54.60\nNET AMOUNT -\n54.60\nGROSS AMOUNT REFLECTS LIST PRICE AS OF PURCHASE DATE\n95602449"
{
  "invoice_dates": "1995-01-20",
  "total_amount": 2349.9
}

Base model output:
{
  "invoice_date": "2025-01-20",
  "total_amount": 2349000
}
R1=1.000 | R2=1.000 | R3=0.200 

Fine-tuned model output:
{
  "invoice_date": "1995-01-20",
  "total_amount": 2349.90
}
R1=1.000 | R2=1.000 | R3=1.000 

==================================================
Example 2

Ground truth:
"PRO BILLIARDS\nTOUR\nINVOICE\nDATE:\nMarch 20, 1997\nACCOUNT:\nSports Marketing Enterprises\nP.O. Box 2955\nWinston-Salem, N.C. 27102\nPRODUCT:\nBusiness License.\nDESCRIPTION:\n1/2 cost of license for Camel/RJR booth space at\nSands Regency XXIII (June 1996)\nAMOUNT:\n$300.00\nAMOUNT DUE:\n$300.00\nPAYABLE TO:\nSands Regency Hotel Casino\n345 N. Arlington Ave.\nReno, Nv. 89501\nDATE DUE:\nApril 15, 1997\nPRIVILEGED MATERIAL REDACTED\n51809 5560\n4412 Commercial Way . Spring Hill, Florida 34606 . (352) 596-7808 \u00b7 Fax (352) 596-7441\nVisit our website: www.propool.com"
{
  "invoice_dates": "1997-03-20",
  "total_amount": 300.0
}

Base model output:
{
  "invoice_date": "1997-03-20",
  "total_amount": null
}
R1=1.000 | R2=1.000 | R3=0.500 

Fine-tuned model output:
{
  "invoice_date": "1997-03-20",
  "total_amount": 300.00
}
R1=1.000 | R2=1.000 | R3=1.000 

==================================================
Example 3

Ground truth:
"ORIGINAL INVOICE\n\u0130\u0130TR\u0130\nHIT RESEARCH INSTITUTE\n50534\n#7622\nLorillard Research Center\n420 English St.\nGreensboro, NC 27405\nAttn: Dr. Thomas A. Vollmuth\nPLEASE REFER TO OUR INVOICE NUMBER\nAND REMIT TO:\nP. O. BOX 92003\nCHICAGO, ILLINOIS 60675\nDATE\nPROJECT No.\nACCOUNT NUMBER\nCONTRACT OR P. O. No.\nTERMS\n6/26/89\nL08170\n112-10 :selected:\ncr\n348B\nNET CASH\nAcute Toxicity Study in Rats\nRange Finding & LD50 Compound\nA262\n$ 4,150.00\nA268\n4,150.00\nA270\n4,150.00\nOK\n\u041d\u0410. \u0422\u0438\u043c\u0435\u043f\u0430\u0434\u0430\n7-5-890\nDept. 8700\nAcct. 4111 :selected:\nAMOUNT DUE $12,450.00\nFORM 583\n87148148 :unselected: :unselected: :unselected: :unselected:"
{
  "invoice_dates": "1989-06-26",
  "total_amount": 12450.0
}

Base model output:
{
  "invoice_date": "2029-06-26",
  "total_amount": 4120500
}
R1=1.000 | R2=1.000 | R3=0.200 

Fine-tuned model output:
{
  "invoice_date": "1990-06-26",
  "total_amount": 4150.00
}
R1=1.000 | R2=1.000 | R3=0.233 

As you can see, the fine-tuned model isn’t perfect (for example, it gets the total_amount in the last example wrong) but it is performing much better at many of the extraction tasks than the base model.

Push to Hugging Face Hub

To push your model to the Hugging Face Hub, uncomment the lies below and fill in your repository name.

# model.push_to_hub("your-username/lfm2.5-350m-grpo-invoice-extractor")
# tokenizer.push_to_hub("your-username/lfm2.5-350m-grpo-invoice-extractor")

Summary

In this notebook, we explored the key principles of a small GRPO fine-tuning pipeline to fine-tune LFM2.5-1.2B-Instruct to extract JSON from raw OCR text using Unsloth. The main goal was to teach the model to produce more accurate structured outputs, with rewards based on whether the response is valid JSON, contains the right keys, and predicts the correct values.

The training pipeline in this notebook is intended for learning purposes and thus the fine-tuned model still has room for improvement. For better results, I recommend revisiting the reward functions, training for at least 300 steps (which may take about 30 minutes on a T4), and experimenting with hyperparameter tuning.

References

• Unsloth Documentation. Liquid LFM2.5: How To Run & Fine-tune
• Unsloth Documentation. Tutorial: Train your own Reasoning model with GRPO

Agent Experience (AX) vs Developer Experience (DX) vs User Experience (UX)

Agents have entered the chat. Until now, software products have had two target audiences in mind when improving usability: End-users and developers. Now we have agents interacting with software products. Coding agents can build software using different software products, including databases or frameworks; Computer use agents can interact with different software products, including email or calendar apps on your behalf.

User Experience (UX) centers on how end-users successfully accomplish tasks with your product. Developer Experience (DX) focuses on how developers successfully build with your product.

Agent Experience (AX) is different: the “user” is an AI agent that discovers, evaluates, and operates your platform, often without a human in the loop. The central question becomes: How does an AI agent successfully use or build with your product? This also means the agent can be seen as both the end-user and a developer persona.

When designing for DX or UX, keeping the target audience’s emotions in mind is a key aspect: Are they frustrated, confused, delighted? For AI agents, this sentiment curve doesn’t exist. Instead, we have to replace emotions with failure modes (e.g., ambiguous error response, auth failure, etc.) and the sentiment curve with reliability curves (“How likely is the agent to succeed at each stage”).

Stages of the Agent Journey Map

Agent Experience has many different facets. Some aspects include answering questions like “How do you make an agent pick your software product?” or “How do you make your software product accessible for agents?”

This reminded me of a similar concept from UX and DX: In UX and DX, the target audience’s goals, questions, and answers are commonly mapped to so-called “user journey maps” or “developer journey maps”, which represent the end-to-end path the user or developer takes when interacting with your software product. In agent experience, the agent’s path is similar to the end-user’s or the developer’s, except that the human persona is now an AI agent.

In this section, I map the path an agent takes from discovery to success on an analogous “Agent Journey Map”. You can think of it as an adoption funnel for agents. The map I came up with follows the five stages of Discover, Evaluate, Onboard, Integrate, Advocate, as shown below.

(Inspired by Developer Journey Map)

(Note that in DX, you often have a “Scale” stage between “Integrate” and “Advocate”, which I couldn’t figure out how to properly manage in the AX case. If you have an idea how to improve this, please reach out.)

Summary

As AI agents establish themselves with use cases such as coding agents, considering Agent Experience when designing your software product becomes more important.

As we’ve seen, the industry proposes and experiments with different standards, such as the llms.txt or MCP, and already starts to discard them. All the aspects mentioned in this blog are a snapshot of the techniques practitioners are experimenting with right now and may be outdated in just a few months.

Whether you think “Agent Experience” is just another buzzword, the reality is that agents are now very real users of your product. If you are a big AI lab developing LLMs (which are the core part of an agent), Agent Experience might be less of a priority for you right now. However, if you have a developer tool or software product you want agents to use, Agent Experience should become a priority. People are already discussing whether Product-led growth (PLG) approaches will soon be replaced by “Agent-led growth” approaches.

“File interfaces are all you need” camp

Leaders like Anthropic, Letta, Langchain & LlamaIndex are leaning towards file interfaces because “files are surprisingly effective as agent memory”.

• Anthropic’s memory tool treats memory as a set of files (the storage implementation is left up to the developer)
• Langsmith’s agent builder also represents memory in as a set of files (the data is stored in a database and files are exposed to the agent as a filesystem)
• Letta found that simple filesystem tools like grep and ls outperformed specialized memory or retrieval tools in their benchmarks -LlamaIndex argues that for many use cases a well-organized filesystem with semantic search might be all you need

Agents are good at using filesystems because models are optimized for coding tasks (including CLI operations) duringpost-training.

That’s why we’re seeing a “virtual filesystem” pattern where the agent interface and the storage implementation are decoupled.

“Filesystems are just bad databases” camp

But then you have voices like Dax from OpenCode who rightly points out that “a filesystem is just the worst kind of database”.

swyx and colleagues in the database space warn about accidentally reinventing databases by solving the agent memory problem. Avoid writing worse versions of:

• search indexes,
• transaction logs,
• locking mechanisms,

Trade-offs

It’s important to match the complexity of your system to the complexity of your problem.