ALLSmartLLM is a routing + policy layer over open-weights LLMs on our own hardware. Everything below is what makes that a useful sentence.
If your code speaks the OpenAI API, it speaks ALLSmartLLM. Change one URL and one key, the SDK stays.
The gateway routes on aliases, not model names. An alias bundles three things:
A single string in your code, "model": "frontier", decides which of 18 open-weights models runs, whether it stays on our hardware, and whether the prompt gets pseudonymised on the way in. Central control, zero code changes when policy shifts.
client = OpenAI( base_url="https://api.allsmartllm.com/v1", api_key="sk-your-key", ) resp = client.chat.completions.create( model="frontier", # alias, not a version privacy={"mode": "pseudonymize"}, messages=[...], )
Every prompt can opt into one of three levels of PII handling. Every level also runs a dedicated scanner for credentials.
The three modes for personal data:
Detected values are replaced by <<TYPE_n>> placeholders on the way to the model. The mapping lives in request memory only. Response placeholders are swapped back for the original values before you see them, streaming included.
Same detection + replacement, but the mapping is discarded immediately after the upstream call. The response the client sees still contains placeholders. Use this when the raw value should never round-trip, external providers, audit-heavy contexts.
Adds a second-pass LLM check on our own hardware. Catches indirect identifiers regex and NER miss, role+place mentions ("the only cardiologist in Neustadt"), chains of small attributes, unique-attribute descriptions. Additive only: never removes a deterministic detection. Fail-closed on any failure, a strict request that can't run gets 503, never a silent downgrade.
Prompts pick up secrets by accident all the time, a pasted .env, a curl example, a stack trace with an auth header. On the way to an external provider that would be a real leak. As soon as a privacy level is set, we scan for them before the request leaves. Without a level, the prompt goes to the model unchanged.
Recognised out of the box:
Values, placeholders, offsets never appear in a log line, metric label, or error message. The mapping is scoped to the request handler; garbage-collected the moment the response ships.
The gateway is one thing. The machines that answer are another. Both are in Austria.
Every request to a ALLSmartLLM alias that targets an open-weights model lands on our own hardware in Upper Austria. No sub-processors. No third-country hops.
External frontier models are still reachable, through the mask/pseudonymize layer, and only when you explicitly connected the provider with your own key. Your contract with them is your contract; we don't broker access. What we do is refuse to let a prompt leave the EU without the privacy layer having seen it first.
Throughput, detection recall, latency, everything with a number on it is benchmarked with a published method. If a number changes, it changes because a run said so.
The nightly bench pings every loaded model with a standard prompt, measures the streaming decode rate, records the median of three runs. Warm-up is discarded. That number is what shows up under "Measured" on the homepage and the model table.
Detection quality has its own eval set: ~50 labelled German-language examples covering direct PII (email/phone/IBAN/SVNR), context-dependent identifiers for the strict-mode LLM check, and realistic credentials for the secret scanner. Precision + recall per class, published in the repo. If we tell you "detects OpenAI keys", that's not marketing, it's a passing test.
10 million tokens free. No card, no subscription.