In the autumn of 2023 we wanted to show what a language model does when you tie it to a single source. Not “what does the AI think of this”, but: what does this party’s manifesto say, and where exactly? That is how Wijsstemmer came about, a project of our own by Eenvoud and Leto, in which you could start a conversation with the manifesto itself, party by party.
It was the shop window for Leto at the same time: our own service for chatbots on your own documents. Eleven parties, eleven bots, all on the same engine. Ten days after the first design it was online. Two weeks before the elections we took it down again ourselves.
The challenge
A party manifesto is dozens of pages long and almost nobody reads it. What does get read are summaries, and those belong to somebody else. We wanted the other way round: let someone ask a question themselves, and answer only with what the manifesto says.
That sets two hard requirements. The bot may not invent anything - if it is not in there, the answer is that it is not in there. And every answer has to be traceable back to the passage it rests on, otherwise it is just as much an opinion. There is something else on top of that: with a political subject the question itself is sensitive information. Who asks what must not be linkable to a person.
If it is not in there, the answer is that it is not in there.
View source
Our approach
We kept the content and the conversation strictly separate.
- Read in every party’s manifesto unchanged, cut into pieces per paragraph, with the source location attached to it
- Those pieces in a vector database, so that a question in ordinary words finds the right passage and not the right keyword
- Per question only the relevant passages plus the question to the language model, so that no answer can arise that is not in the manifesto
- Eleven party bots on one engine: our own service Leto, in which a bot is nothing more than a set of documents with a widget on top
- A front end in Laravel with Inertia and Vue, in which you start a conversation per party and switch to another party with a single tap
- Questions processed anonymously and passed on in bulk, so that a question cannot be linked to a visitor - explained on a page of its own instead of in the small print
The design went out of the door on 16 September, building started the Monday after, and ten days later the press release was with the newsrooms. Small team, short lines, every change to production through the same release pipeline.
The result
Wijsstemmer was online at the end of September and got picked up: Sargasso put it in the list of voting advice applications, and the press list did its work. Eleven parties could be questioned, in ordinary language, with the passage the answer rested on alongside it.
At the beginning of November a cease-and-desist letter arrived from ProDemos, the organisation behind StemWijzer: in their view the name Wijsstemmer infringed their trade mark. We did not think so - Wijsstemmer did not take over the role of StemWijzer and could perfectly well have existed alongside it - but trade mark proceedings two weeks before the elections did not seem a good idea to us. On 6 November the site was renamed Slimstemmer and shortly afterwards went offline. A week later Leto stopped as well: in a few months OpenAI had caught up with what we had built on, and that conclusion was already in our own papers in September.
What remains is what it demonstrated. A language model that is tied to one document and can name its source is a different machine from a language model that answers everything. That distinction is the heart of every AI project we have done for clients since.
A language model that is tied to one document is a different machine from a language model that answers everything.
Ten days of building and eleven parties given a voice - the technology was not the problem, the name was.
Let’s talk
Tell us about your challenge. We’ll think along.
A reply within one working day. No account managers, no sales pitch.
