I wanted to know whether a small model, trained on one consumer graphics card, could learn to build web components the way I build them. Not a public corpus of HTML and CSS: my own conventions, from my own projects, collected over 25 years. Own projects only, no client work.
The short answer is yes, better than I expected, and the longer answer is that most of what I learned came from the measurement being wrong rather than from the training being right.
Nothing runs on it, nothing depends on it. It builds one component, not an application, and I am not putting it up for download. Why not is at the end.
What came out
The share of generated components that satisfy all my house rules went from 12.7 percent untrained to 74.6 percent, without a single rule mentioned in the prompt. Both figures come from 500 generations per model, 50 tasks with ten runs each, and the uncertainty is a 95 percent interval from a bootstrap over tasks rather than over runs (6.7 to 19.3 untrained, 67.8 to 81.2 trained). Tasks, not runs, because ten runs of the same task are ten correlated answers and treating them as independent understates the uncertainty badly. The rules cover semantic structure, ARIA attributes, contrast, visible focus, no console errors, no external dependencies.
I did not trust that number, so I also judged 360 components by hand, blind: opaque names, shuffled, origin unknown. 150 from run 13, 150 from run 1, 60 from the untrained model as a control. Run 13 is the one I judged, not run 14, which came later.
By eye the untrained model produced something usable in 33 percent of cases, run 13 in 77 percent, a secured gap of 41.7 points (interval 27.7 to 55.3). Two completely different yardsticks pointing the same direction. Not the same answer: the checker calls 12.7 percent of base-model components clean while I called 33 percent of them usable, so the two disagree substantially about the base model and agree only about the size and sign of the improvement. That much agreement still did more for my confidence than either number on its own.
Two things in that series are statistically secured and the rest is not. Secured: the gap to the untrained model, and run 1 to run 14, which is +8.8 points with an interval of +1.6 to +16.4. Not secured: any single step in between. Run 13 to run 14, for instance, is +2.4 points with an interval of −3.2 to +8.4, which includes zero. So the series as a whole went somewhere, assembled from steps none of which I could defend individually.
Fourteen training runs over four weeks, two and a half to three and a half hours each on an RTX 4070 Ti with 12 GB. In hindsight the first run already had almost all of the gain. Everything between run 1 and run 12 was me going sideways, and I could not see it at the time because I was measuring badly.
The part I got wrong for two weeks
For two weeks I measured the base model and thought I was measuring my fine-tune.
The adapter was loaded through a function that attaches trained weights to the base model by matching module names. The names did not match, by one segment in the path. The library attached what fitted, left the rest at zero, and said nothing. With those weights at zero the adapter contributes exactly nothing, so the model behaved token for token like an untrained one.
Every number I produced in those two weeks was real, reproducible, and about the wrong model. I built theories on top of it. "Fine-tuning a 4B model achieves little" fit my expectations, so I elaborated it instead of checking it.
What eventually broke it open was adding a second, unrelated way of running the model. The two disagreed. That is the only reason I found it.
Since then the service checks at startup whether the adapter changes anything at all, by comparing the raw output with and without it, and refuses to run if they are identical. It costs a tenth of a second.
The three lines I wish I had written first
If you load a LoRA adapter with PEFT, this is worth having in your own code. It compares the raw output with and without the adapter and stops if they are the same, which is the one symptom that silent mismatch produces:
import torch
from peft import PeftModel
# basis and tok are the already loaded base model and its tokenizer.
modell = PeftModel.from_pretrained(basis, "<adapter path>")
probe = tok("any fixed prompt", return_tensors="pt").to(modell.device)
with torch.no_grad():
with modell.disable_adapter():
ohne = modell(**probe).logits
mit = modell(**probe).logits
assert not torch.equal(mit, ohne), "adapter has no effect"
The usual cause, and mine, is multimodal base models. A checkpoint
loaded with AutoModelForImageTextToText carries an extra
language_model segment in its module paths that the same checkpoint
loaded with AutoModelForCausalLM does not. Train through one and
load through the other, and the names will not line up. On peft 0.20.0, which is
where I checked this, the mismatch does print one line about missing
adapter keys. It is one line among the loading output, the model runs
normally afterwards, and the generated text looks fine, which is why
the assertion is the part that actually saves you.
Three more of the same kind
Two adapters with the same name. One worked, one was a dud, both 41 MB, in two directories I had both used. One measurement came out at 7 percent before I noticed which file was being read.
A quantisation that inverted a classification. I built a small model to answer yes or no about component quality. In 8-bit it answered "no" to everything. In 16-bit it was fine. The fault was invisible in free text and total in a single-token answer.
A probe that searched for the wrong word. I spent a while preparing to release the adapter, and part of that was wanting to know whether it reproduces identifiers from my own projects. The tool that checks for this had itself been rewritten by the anonymisation pass, so it was searching for the replacement names, which the model cannot possibly have learned. It would have given me a clean bill of health. Twice more after fixing that, it flagged ordinary words as leaks, including the English word for a complaint form in a complaint form.
Once it worked, the answer was reassuring: across 120 generations aimed deliberately at the component kinds whose templates carry distinctive internal identifiers, not one appeared. By then I had already decided not to release the thing, so the probe ended up answering a question nobody asked. I would still run it again.
Two of those four agreed with what I expected, and those are exactly the two that survived. Measuring the base model produced the result I was expecting, that fine-tuning a 4B achieves little, so I explained it instead of checking it, for two weeks. The probe gave me an all-clear, which is what I wanted to hear, so I would have accepted it and published.
The other two contradicted me, and both were found almost at once. A 7 percent measurement was too bad to be true, and that was the only clue I needed. A judge answering "no" to every single question is obviously broken before you even look at the log probabilities.
So the mechanism is not that comfortable faults are more common. It is that a fault contradicting your expectation gets hunted until it is found, while one confirming it gets written up.
And one I caused while preparing that release. The tool that
strips project names walked the whole directory tree and
treated the model's tokenizer file as ordinary text, because it ends
in .json. It overwrote four entries of a 248044-entry vocabulary
across 17 files. I found it from a timestamp: the file had been
modified hours after training finished. A tokenizer is not a text
file, whatever its extension.
What the training actually taught me
A value can be moved, a structure cannot. A colour I use was picked up from 167 examples. A wrapper structure was not picked up from 117. Same data, same run. Unless the model's prior is too strong: Tailwind's default blue survived 346 counterexamples. That asymmetry decided most of what I did next, and I still do not fully understand it.
Small proportions come through. One broken template out of 152, 0.7 percent of the data, produced a 2.4 percent failure rate for three consecutive training runs. I repaired that one file and the failure rate went to exactly zero. I had been looking for the cause in the training parameters.
Training and prompting compete. Once the model knew the rules, putting those same rules in the prompt made the output measurably worse. On the untrained model the same text had been the single biggest improvement I ever measured. An aid helps while the capability is missing and gets in the way afterwards.
A rule without a concrete value does not get learned. The model reaches for
#777 on its own, which is a contrast ratio of 4.48 against a
threshold of 4.5. Telling it "ensure sufficient contrast" changed
nothing, because the problem is arithmetic nobody does in their head.
Giving it examples with correct values did.
Training on its own output buys uniformity, not quality. The last run, run 14, was trained on a sample of components it had generated and passed itself. By the rules it came out ahead of run 13, 74.6 against 72.2 percent, which is +2.4 points and inside the uncertainty, so not a secured improvement at all. Then I put the two side by side, 50 comparisons, and preferred run 13 nineteen times to eleven.
That split has a sign-test p of 0.20, meaning something that lopsided would turn up by chance about one time in five, so it does not settle anything either. What I am left with is two yardsticks that disagree about my last two models, one resting on 500 generations and one on 50 of my own judgements. My reading is that run 14 had learned to repeat its own best guess rather than to fit the task, and that the rule measurement cannot see the difference, because uniformity looks like consistency to a checker.
Where the measurement stops, and why that is the real finding
Across all 360 components I judged by hand, 211 passed every rule I had written, and of those 211 I was not satisfied with 45. That 211 is pooled across the three models in the sample, so it mixes a good model with a bad one. For the trained model on its own, run 13: 105 of its 150 components passed every rule, and 18 of those 105 did not satisfy me. About one in six.
So I wrote three more rules aimed at exactly those 45 pooled cases: overlapping content, wrong initial state, controls that do nothing. Together they caught about 7 percent of the 45.
The rest was spacing, proportion, alignment, and things that were technically present but did not work. I can see it in half a second and I could not write it down. Those remaining 93 percent of the 45 cases are the thing I would warn anyone about who plans to automate judgement of generated code. The wall arrives much earlier than you expect, and more rules do not close it.
What it cannot do
It builds one component. No routing, no state across pages, no data model. It builds to my conventions, so anyone else would get a model that is confidently wrong. It expects the task in German. One component in four still breaks a rule. Of the rule-clean ones, about one in six would not satisfy me, measured on run 13, which is the run I judged by hand; the last run, run 14, was never put through that test.
Why there is no download
I built this to find something out, and I found it out. The model itself solves no problem: it writes components in one person's house style, that person being me, which makes it useless to everyone else and unnecessary to me, since I can write them myself and faster. Putting a 41 MB file online so that it can sit there unused is not a result, it is housekeeping.
What the work produced is in this text. The measurement was wrong four times in a row, and the two cases that agreed with what I expected are the two that lasted, a single broken template out of 152 was visible in the output for three training runs, and the gap between "passes every check" and "is good" turned out to be mostly unwritable as rules. Those are the parts worth having, and they do not need a download.
I did prepare a release, far enough to find two more faults while doing it, including one I caused myself. I am keeping those in the account because they are the most useful thing that came out of the preparation.
The templates and the training data stay where they are in any case. They are 25 years of my own work and they encode the conventions this model exists to reproduce; releasing them would mean releasing the house style itself.