Local LLM: analyze government text without data being sent to third parties

This guide is informational. Always verify how the rules apply to your specific case.

DocumentationEvidencePrivacyMeetings

Position in the flow: after recording and Whisper transcription, as part of In contact with authorities.

In short

Best privacy: run LLM entirely locally (Ollama / LM Studio) – the text never leaves your computer.
Second best: temporary, isolated GPU pod (e.g., RunPod Secure Cloud) that you start, load the model to, use, and then delete – not ChatGPT/Claude cloud chat.
Avoid for sensitive matters: consumer cloud where you paste the entire file into a chat that trains on or logs input data.

What a local LLM is good for in authority matters

A language model (LLM) does not replace legal advice. It is a work tool when you already have recordings, transcriptions, or requested documents. Typical tasks include:

Summarizing a long meeting transcript: decisions, promises, deadlines, unclear points.
Comparing your transcription with the authority's service note and listing differences point by point.
Suggesting a structure for corrections, responses, or follow-up emails (you rewrite and verify facts).
Translating into your language while retaining Swedish legal terms in parentheses.
Finding deadlines and “what should I request in writing?” in a complex text.

Never let the model “invent” quotes. Ask it to mark uncertain parts and always verify important sentences against the audio file or original document.

Privacy ladder: three levels

Completely local (recommended for children, social services, healthcare, migration, police): model + chat runs offline on your machine. No uploading.
Isolated GPU you rent briefly (RunPod Secure Cloud, similar): data leaves the computer but goes to a pod you control, not to a general chat service's training pipeline. Requires strict deletion routine (see below).
General cloud AI (ChatGPT, Claude.ai, Gemini, etc.): easiest – but sensitive personal data should be heavily masked or not sent at all. Read the terms; assume input data may be logged.

How to run LLM locally

Option A: Ollama (Mac, Windows, Linux) – easiest for many

Install Ollama from ollama.com.
Download a model, e.g., ollama pull llama3.1:8b or a Swedish/European fine-tuned model available in Ollama's catalog that fits in your memory.
Start the chat: ollama run llama3.1:8b or use Ollama's app/web interface.
Paste (or attach) the working copy of the transcription – not the entire audio file.
Work offline: disable unnecessary cloud sync for the folder with sensitive files.

Option B: LM Studio – graphical and good for beginners

Install LM Studio.
Download a GGUF model (start with 7B–8B quantized, e.g., Q4/Q5).
Load the model locally. Ensure “local server” / offline is selected and nothing is sent to the cloud.
Use the chat or the local API (often localhost) from your own computer.

Option C: GPT4All or llama.cpp

Works well on older computers. Same principle: download the model file once, run offline, keep sensitive texts out of cloud sync.

Prompts that work in authority matters

You are a legal language assistant, not a lawyer. Summarize the text in simple Swedish. Divide into: 1) decisions/notifications, 2) promises from the authority, 3) what I should do, 4) deadlines, 5) unclear points I should request in writing. Do not change quotes. Mark uncertain parts with [uncertain].

Compare TEXT A (my transcription) with TEXT B (the authority's note). List only concrete differences: dates, quotes, who said what, missing points. Do not invent differences. If something is unclear, write “cannot be determined”.

Write a short response/correction draft (max 200 words) based on the differences. I will verify each statement against the source before sending.

What the computer needs to perform

The figures are guidelines for quantized models (GGUF Q4/Q5). Full precision requires more. Swedish and long transcripts require more working memory (context) than short chats.

Goal	Model size	Memory / GPU	Expected feel
Minimal summary	~3B–4B	8 GB RAM, no GPU	Slow but usable on a laptop
Good everyday analysis (recommended starting point)	~7B–8B Q4/Q5	16 GB RAM, or 8 GB VRAM	Acceptable speed on modern laptop/Mac
Long files / better reasoning	~13B–14B Q4/Q5	32 GB RAM, or 12–16 GB VRAM	Noticeably better quality, heavier
Heavy models / many files	32B+ or 70B quantized	48 GB+ RAM or 24 GB+ VRAM	Often better on rented GPU than home computer

Apple Silicon (M1/M2/M3/M4): 16 GB unified memory is often sufficient for 7B–8B. 32 GB provides more margin for longer documents.
Windows/Linux with NVIDIA: CUDA support in Ollama/LM Studio provides a significant speed boost. 8 GB VRAM ≈ comfortably 7B–8B; 12–16 GB VRAM opens up 13B.
CPU only: works, but expect minutes per response for longer text. Divide the document into sections.
Disk: estimate 4–8 GB per model file; have 20+ GB free.
Whisper + LLM simultaneously: run them in sequence so the memory is sufficient.

If the computer cannot handle it: use a smaller model, shorter excerpts, or a temporary isolated GPU (next section) – not a general cloud chat with unmasked files.

RunPod and other isolated GPU environments – is it privacy secure?

Short answer: It is safer than pasting the entire file into ChatGPT, but not as safe as running entirely locally. Data leaves your computer and is processed in someone else's data center. For many cases, a well-managed Secure Cloud pod is sufficient. For protected personal data, serious healthcare/social service cases, or when you want to maximize control: stay local or mask heavily.

What “isolated pod” means in practice

You rent a GPU container (pod) where you start, for example, Ollama, text-generation-webui, or vLLM.
The model and your text run in that container – not in a shared consumer chat with millions of users' logs.
At RunPod, there is often a distinction between Community Cloud (shared host, cheaper) and Secure Cloud (dedicated/more isolated hardware at the provider). For sensitive material: choose Secure Cloud (or an equivalent dedicated option with another provider).
Ephemeral disk is often erased when the pod is stopped. Persistent “network volume” remains until you delete it – don't forget to erase.

When RunPod (or similar) might be reasonable

Your computer lacks memory/GPU for the model you need.
You need to analyze many pages over a short period (7–70B models).
You can follow the deletion routine below and avoid storing more than necessary in the pod.

When to opt out of rented GPU

Protected personal data, serious child cases, or when you don't want the material to leave home at all.
You are unsure about region/storage or want guaranteed “zero data retention” – you mainly achieve this by never uploading.
You only need simple summaries – then a 7B–8B model locally is usually sufficient.

Step-by-step: temporary analysis pod (RunPod-like)

Prepare at home: create a working copy where you have masked personal numbers, complete addresses, and unnecessary names if the case allows it. Save the original locally.
Choose Secure Cloud / dedicated instance in the EU region if available and suitable for you. Avoid Community/shared host for sensitive material.
Start a pod with a Linux image and sufficient VRAM (e.g., 16–24 GB for 13B–32B quantized).
Install only what you need (Ollama or a local web UI). Download the model in the pod.
Upload only the working text (transcript excerpts), not the entire home directory and not the original audio unnecessarily.
Run the analysis, copy the result to your own encrypted disk.
Delete: remove uploaded files, stop the pod, delete any network volumes/snapshots. Do not assume that “stopped” = “gone” if you have created persistent storage.
Turn off API keys and do not save them in insecure notes.

Other options in the same class

Vast.ai, Salad, TensorDock, etc.: similar idea – rent GPU. Integrity varies with whether the machine is community/shared or dedicated. Read terms and choose isolated capacity.
Own VPS with GPU in the EU: more control if you harden the server yourself, but more maintenance.
Not the same thing: “private mode” in consumer chats – it is still the provider's infrastructure and logging policy.

Checklist before sending text to any LLM

Do I need AI at all, or is manual marking sufficient?
Can I run locally with 7B–8B?
If no: isolated pod + Secure Cloud + deletion – not general cloud chat.
Have I masked what does not need to be analyzed?
Have I saved the result locally and cleared the upload?
Have I checked quotes against recording/document before sending an email to the authority?

Common mistakes

Pasting the entire social service file into ChatGPT “just to test”.
Trusting the model's quotes without listening again.
Leaving network volume with the file for months.
Using Community Cloud for sensitive material to save a few kronor.
Believing that “local model in the cloud” is the same as offline at home.

FAQ

Is RunPod as secure as my own computer?
No. It can be acceptable as isolated temporary computation if you use Secure Cloud and delete everything afterward. Own offline operation provides stronger control.

Do I need a graphics card?
No. CPU works for 3B–8B, but slower. Apple Silicon with 16 GB often works well without a discrete GPU.

Which model should I start with?
A quantized 7B–8B model locally. Upgrade only if the quality is insufficient.

Can I use the result in a JO complaint?
Yes, as your own working material – but attach actual quotes from recordings/documents, not AI hallucinations.

Connect to the rest of the flow

Record the Meeting
Transcribe with Whisper Locally
Analyze with Local LLM (this guide)
Request Correction / Write a Response
Basic Guide: In Contact with Authorities

/div>

For Those Reading Under Stress

This guide is intended as support when you already have a lot on your mind. You do not need to read everything at once: start with the section that matches your current situation and return when the next step becomes relevant.

Matters with authorities rarely improve if you handle everything alone. Seek help from a support person, lawyer, or relative if you can—especially before decisions with long-term effects.

Common Mistakes That Make the Case More Difficult

Relying solely on verbal promises without brief written confirmation or follow-up email.
Delaying the request for documents until the conflict is already locked—making it harder to demonstrate what was said.
Writing so extensively and emotionally that the authority stops responding to the substantive issue.
Missing deadlines for reconsideration or appeal because the decision was not read verbatim.

Mini-Checklist Before Leaving the Guide

Do I have the date, diary/case number, and the name of the person handling the case (if known)?
Do I know what the next step is: respond, request, remind, appeal, or report?
Have I saved emails and attachments in a place where I can find them in a week?