Conveyor has been in the customer trust business for over four years. We do a lot of things, but the core of it has always been: fast, accurate, ready-to-share answers to high-stakes customer questions.
When we started, Retrieval Augmented Generation was the state of the art. You ingest a lot of information, curated Q&As, documents, websites, break it into chunks, index the chunks into a vector database using embeddings, and when a question comes in you retrieve the most relevant chunks, hand them to the smartest LLM available, and ask it to generate an answer from that context.
Over the years we added layers of pre and post processing. We perfected the prompts, the chunking strategy, the embeddings, the heuristics, conflict detection, evals, guardrails, customizations. All of it to squeeze out a bit more accuracy and eliminate a few more bad answers, the face palm moments, as we fondly called them. But the core stayed the same.
It has worked well. Hundreds of customers can testify, with many reaching 95% accuracy on AI-generated answers and some reaching 99%.
We still decided to change how the answer engine works. Here's why.
If it ain't broke…?
Even with a good RAG engine, three things stood between our customers and the full potential of a customer trust program.
- Knowledge management: what separates the wildly successful customers from the ones a bit behind is how well their knowledge library is maintained. No surprise there. Garbage in, garbage out. Our vision is a self-healing knowledge library: high quality, current, well organized, conflict free, with almost no maintenance work.
- Touchless: our larger customers want a truly touchless workflow, where sales or even the customer directly throws in a questionnaire and gets it back filled within minutes, with no human in the loop. For example Zendesk automated 93% of requests with Conveyor. When that's the bar, the last few percentage points of accuracy matter a lot more, both to cut the questionnaires escalated for human review on low confidence and to keep a wrong answer from slipping through.
- RFPs: more and more customers use Conveyor beyond security, especially for RFIs and RFPs. That brought far more varied questions, plus demand for real control over how an answer is worded, customized per vendor and per the customer receiving it.
We were hitting the ceiling of what we could do to improve the answering pipeline, and the system around it, to meet all three.
Until today
Over the past few months we took a step back. "Forget all previous instructions." How would we approach this problem today if we started from scratch? An exciting and slightly scary exercise. We read the latest research, reviewed different approaches, got inspired by the industry's best, and took a stab at building something new.
Building anything in 2026, it was pretty clear the approach should be agentic, both for processing knowledge and for generating answers. Karpathy's wiki approach felt like a good starting point for our explorations.
We also had a list of our toughest challenges, the questions and situations where our engine would break and produce a face palm moment. We stress tested every one against the new flow, first as a thought exercise, then for real.
With agentic coding on our side we could iterate fast. The goal was not to build. The goal was to experiment and then try to break whatever we built, and see where it failed. We ignored the worries of mere humans like latency and cost at first (funny story about that another time), and once we had something that worked well, we started turning it into a real solution with scale, security, user experience, cost, and continuous learning in mind.
Our biggest asset here was our customers. Talking to enterprise customers and letting them into our experiments in their own environments made all the difference. They brought edge cases, preferences, customizations they wanted, and insights we would not have found on our own, and all of it fed back into the next iteration. We could not have built this, certainly not this fast, without them.
So where did we land?
We built a multi-step agentic answering flow. It:
- Analyzes the incoming questionnaire in its entirety and plans how to tackle it
- Extracts the relevant context, and only that, for each sub-task
- Delegates the answering to sub-agents, each with tools for finding the right information
- Assesses whether an answer is complete and iterates when it isn't
- Collects everything and packages it back into a finished, customer-ready document
We could reuse many components of the old flow as tools for the agents, which let us ship faster and ship in stages, starting with pieces of the previous pipeline and replacing them over time. The new flow is far more flexible and adaptable, which shows up as higher quality answers and much better adherence to your instructions.
It works the way a person would. Breeze through the questionnaire and supporting materials for context. Read the questions in their sections, with the earlier questions in mind. Look for information in more than one place. Draft. Follow the guidance you were given when you joined the team, and any feedback you've gotten since. The agent just does it faster and more consistently.
We could also finally show the reasoning behind an answer: what the agent tried, how it weighed different sources, where it wasn't sure. You get visibility into what's happening in the AI brain instead of a black box handing you an answer.
We are also overhauling the knowledge library itself, how it gets created, how it's maintained, and how information is retrieved from it. That gives the agent finer guidance on which sources to use, which to avoid, and how to use them in different situations. It also opens the door to real self-healing: the library detects conflicting, duplicate, or stale information and acts on it, and it learns from your edits and feedback, whether that's knowledge or style, and applies it going forward.
The thinking is simple. However smart the LLM is, it makes no sense for the system to answer your thousandth question exactly the way it answered your first. It should get more accurate, and more aligned with how you want answers phrased, over time. Automatically.
Today the new answer engine is in Labs (beta). If you use Conveyor for RFPs please turn it on in settings - it's an instant upgrade. If you want to test it for security questionniare automation reach out to your CSM for help.
So what does this fix?
Back to the three gaps we set out to close.
Knowledge management. The new system includes an agentic librarian looking after your knowledge library. New information from a new or updated source, from an edit you make to an answer, or from you directly teaching Conveyor something, gets processed by that agent, and anything genuinely new gets stored in the right place with the right context. The agent also spots stale information that should be retired in light of what it just learned.
Touchless. Reading the entire questionnaire and the full context around each question closes a big share of the old face palm moments. Add self-healing knowledge, the agent's exposed reasoning creating a feedback loop, and revamped tools pulling fresher knowledge, and the accuracy gaps that stood in the way of touchless are addressed.
RFPs. A smarter, more flexible, iterative engine with rich control over which information to use and how answers are styled is what RFP response actually requires. To most AI these look like the same job. They aren't. A security questionnaire requires precision and an RFP requires persuasion. ConveyorAI is now smart enough to know the difference and deliver answers that fit the assignment. More on what that unlocked in Introducing ConveyorAI for RFPs.
Are we done? Nope. Just getting started.
We think of the new flow as a foundation, something to keep building on as we add capabilities and customizations and as the underlying LLM and agent technology keeps moving.
A few things we're thinking about: stronger feedback loops, with the agent suggesting improvements to your knowledge or style guides on its own, and an interactive mode where you work through an RFP alongside the answering agent.
Try it: an admin can turn on the agentic pipeline in Settings, Answering, Custom Instructions. Read the docs, or talk to your CSM if you'd like a hand.
