ALLSmartLLM is a routing + policy layer over open-weights LLMs on our own hardware. Everything below is what makes that a useful sentence.
If your code speaks the OpenAI API, it speaks ALLSmartLLM. Change one URL and one key — the SDK stays.
The gateway routes on aliases, not model names. An alias bundles three things:
A single string in your code — "model": "frontier" — decides which of 15 open-weights models runs, whether it stays on our hardware, and whether the prompt gets pseudonymised on the way in. Central control, zero code changes when policy shifts.
client = OpenAI( base_url="https://api.allsmartllm.com/v1", api_key="sk-your-key", ) resp = client.chat.completions.create( model="frontier", # alias, not a version privacy={"mode": "pseudonymize"}, messages=[...], )
Every prompt can opt into one of three levels of PII handling. On top of that, credentials get their own always-on scanner — even if you didn't ask for privacy.
The three modes for personal data:
Detected values are replaced by <<TYPE_n>> placeholders on the way to the model. The mapping lives in request memory only. Response placeholders are swapped back for the original values before you see them — streaming included.
Same detection + replacement, but the mapping is discarded immediately after the upstream call. The response the client sees still contains placeholders. Use this when the raw value should never round-trip — external providers, audit-heavy contexts.
Adds a second-pass LLM check on our own hardware. Catches indirect identifiers regex and NER miss — role+place mentions ("the only cardiologist in Neustadt"), chains of small attributes, unique-attribute descriptions. Additive only: never removes a deterministic detection. Fail-closed on any failure — a strict request that can't run gets 503, never a silent downgrade.
Prompts pick up secrets by accident all the time — a pasted .env, a curl example, a stack trace with an auth header. On the way to an external provider that would be a real leak. So we scan for them before the request leaves.
Recognised out of the box:
Values, placeholders, offsets never appear in a log line, metric label, or error message. The mapping is scoped to the request handler; garbage-collected the moment the response ships.
The gateway is one thing. The machines that answer are another. Both are in Austria.
Every request to a ALLSmartLLM alias that targets an open-weights model lands on our own hardware in Upper Austria. No sub-processors. No third-country hops.
External frontier models are still reachable — through the mask/pseudonymize layer, and only when you explicitly connected the provider with your own key. Your contract with them is your contract; we don't broker access. What we do is refuse to let a prompt leave the EU without the privacy layer having seen it first.
Throughput, detection recall, latency — everything with a number on it is benchmarked with a published method. If a number changes, it changes because a run said so.
The nightly bench pings every loaded model with a standard prompt, measures the streaming decode rate, records the median of three runs. Warm-up is discarded. That number is what shows up under "Measured" on the homepage and the model table.
Detection quality has its own eval set: ~50 labelled German-language examples covering direct PII (email/phone/IBAN/SVNR), context-dependent identifiers for the strict-mode LLM check, and realistic credentials for the secret scanner. Precision + recall per class, published in the repo. If we tell you "detects OpenAI keys", that's not marketing — it's a passing test.
10 million tokens free. No card, no subscription.