An AI assistant at a large company: system access, knowledge bases and tickets

Most of the work is reaching the systems and their APIs

Marek Grabarz: “at the end of the day it turns out that 90% of the problems, or rather of the work, is about breaking through all these barriers and doing the integration, I would say, from start to finish” (translated) 08:13

That is why the project starts with discovery: which systems the assistant integrates with, whether they have an API, what kind of API it is and what scope of knowledge it gives. The assistant connects to systems through APIs, without a user interface, so the team must first find the API and learn what it can do.

Existing systems are often maintained with the frontend in mind. For example, the knowledge base team in ServiceNow keeps the content up to date and has a documented way to write it, but nobody has asked about access through the API before, so nobody knows the answer. Maintenance teams spread across Asia, the Americas and Europe do not know either.

API availability is often selective: whether an API exists depends on the deployed modules and the license plans. The APIs of individual modules can be completely different from each other. It can also turn out that there is no API at all, or that it supports only a service account or impersonation.

Discovery also covers licenses, access terms and rate limits, which the system architecture must then handle. One of the integrations in the project leads to SAP. The recording was made at the very start of May 2026. Shortly before, SAP announced that the use of its API by AI agents would be charged for or forbidden, and the analysis of the impact was still in progress. The speakers suppose that SAP wants to push everyone to use its own assistant, Joule, this way.

A service account lets a prompt injection reach other people’s data

A service account has very limited use when the assistant’s access must be narrowed. In the project described here, access was one of the first problems. In most organizations the intuitive choice is a technical account with administrative access to the source system, for example SAP. The assistant uses it to fetch the data of one employee, and then of any other.

Such an account is very easy to manipulate. A successful prompt injection on the chatbot side is enough to convince the assistant to hand over, for example, the data of the company’s CEO: their plans, and maybe also their salary and benefits.

Impersonation is adding the user’s ID or full name to the technical account’s request, for example in a header or in the query string. It is still easy to manipulate, so it can leave a vulnerability in the integration and expose sensitive data.

The assistant queries systems with the signed-in user’s token

The assistant should use the user’s real token and the user’s permissions in the target system. Delegated access is a model based on delegated OAuth flows: the user signs in as themselves, gets an access token that represents them to the target system, and the assistant queries the system with that token. The assistant then generally cannot go beyond the user’s permissions. It does not re-create RBAC (Role-Based Access Control) and does not restrict access with system prompts or parameters: it uses full permissions, but only those of the signed-in person.

This model requires identity federation, and its design needs knowledge from several fields. SAP, SAP SuccessFactors and every other large platform in the organization have their own user identity. Federation means that the agent’s user does not get a sign-in window for every system the assistant uses. SAP may also expect a completely different identity. Then a SAML federation is needed between Microsoft Entra ID and the source system, and in large organizations its configuration is not always simple.

If access must be impersonated, you need an API Gateway that translates one model into the other. The assistant reaches the gateway with delegated access. The gateway reads the user’s identity from the token, and this path cannot be spoofed. Only then does the gateway add impersonation inside the system, based on its own deterministic logic.

The assistant reads only selected SharePoint sites

When you connect SharePoint to the assistant, you must extract the knowledge it should show, limit what it sees and defend it against injected bad content. SharePoint Online connects through Microsoft Graph API, and technically this works. The speakers see, however, that clients tend to turn on the integration for all of SharePoint. The assistant then gets access to many sites, including ones it should not read, even with delegated access.

A user has access to the sites of their projects. These can also be sites where someone once turned on public access, uploaded very sensitive data and forgot about it. The access existed before, but a person does not go through sites one by one unless they do it on purpose. The assistant synthesizes everything it finds, so a question about budgets can surface projects that the user in theory had no access to. In the project described here, the assistant reads only selected, moderated sites with HR knowledge.

Content poisoning is injecting bad content into the sources that the assistant reads. A person with write access to such a site can generate, for example with an LLM, a correct-looking HR procedure that says they are entitled to a Porsche as a company benefit, and the assistant starts to repeat it in its answers. They can also replace the form for reporting misconduct or workplace bullying with their own and collect the reports, even outside the organization.

Graph API works only with SharePoint Online. SharePoint on-premises, which is also common, requires scraping and a vector database, and delegated access does not work there. The whole content of the selected site then goes into the database as vectors.

ServiceNow knowledge base content needs cleanup before the model

In the project described here, most of the knowledge sat in knowledge bases that the HR departments built in ServiceNow. Authors write the articles in a WYSIWYG editor, similar to the old Pajączek tool, in which you built a web page by clicking, so the API returns them as HTML with inline styles, divs and nested tables. Loading such content without extracting the text made token use 5 times higher and can end in a serious hallucination.

Converting HTML to Markdown or plain text seems like a solved problem. The articles, however, often contain images, diagrams, attachments and links to articles in SharePoint. An agent that fetches only text does not know, for example, which procedure flow an attached image shows. So the model must be multimodal and handle images, text, PDFs and Word files, and the fetching of these elements must be orchestrated.

Adding all images to the context causes other problems. Someone on the editorial team used icons with a green check mark or a red X instead of bullet points. One fetched article had 20 image attachments, and 19 of them, or maybe all of them, added nothing: they were icons or a banner with the company logo at the bottom of the page. You need exception lists that block the fetching of such attachments, for example at the API level, in the integration or in the agent’s instructions. Without them, token use grows.

Knowledge base search needs rules that the business describes

Knowledge base search needs priorities, because local procedures exist next to global ones. An employee from Poland who asks about leave wants an answer under Polish law, not under the global policy. ServiceNow searches through the API by keyword: a query for “company car” does not find a Polish procedure that uses the Polish term “samochód służbowy”, so such differences need a description. Nested cases are harder, for example an HR employee with access to everything, or a manager who has people in five locations and asks how much leave their employee in France has. The permissions and the query then need proper orchestration.

Search alone does not give 100% certainty that the answer is correct. The assistant also needs text understanding. The way to search and summarize the knowledge base must be described in user stories that the business and the analysts prepare.

The agent gets ticket form rules from a custom API

The goal of the project is fewer tickets, so the assistant first answers from the knowledge bases. When it cannot help, it falls back to a ticket. If the user says from the start that they are sick and want to file a ticket, the assistant starts the request at once.

A ticket in ServiceNow is usually a form with drop-down lists, where the choice of one option shows more fields. There are many ticket types. Marek Grabarz: “We try, and I have said this before, to follow ‘orchestrate, do not replicate.’ So we try to orchestrate the handling of such a ticket, and not necessarily every ticket type.” (translated) 18:00 The agent fetches the categories, matches a category to the user’s request, fetches the ticket types in that category and then the template: the fields, their types and whether they are required.

This is not enough, because a lot of logic sits in the ticket form in ServiceNow, not in the backend: validators, lookups to tables (for example City ID) and dependencies between fields. ServiceNow refers to tables through SYS_ID identifiers, separate for the country table, the user table and every other table, and this is a big problem. Even if these dependencies were in the API, the agent would not understand them, because nobody has described them. There is no documentation: an external company built the ticket forms in the ServiceNow UI 10 years ago, and the HR staff know roughly how they work.

An example: in a ticket that requests a training course, the training cannot take place earlier than 2 weeks from now. This rule is not in the API entity; it exists only in the form logic. The team solved this with a custom API in the middle that adds new properties to the ticket fields on the fly, especially to the dynamic ones: validation rules, descriptions and dependencies. The agent’s instructions tell it to fetch the template, check which fields are required and read the descriptive and validation rules. The rules are plain sentences, without JSON Logic, for example: if field A has a given value, field B is required.

An LLM handles such a description very well, because it is not technical jargon or an IF this then THAT rule. The topic owner, for example the HR department, can also edit the description. The source of the rules for the custom API can be an attachment or another set of information, and the API applies the fields, validations and descriptions on the fly as text elements. The agent gets the information it needs dynamically, without a rewrite of the frontend or a step-by-step description in the prompt.

The team needs integration experience and soft skills

The challenges of such a project lie mainly in integrations, security and authentication. The team needs people with long experience in IT who understand authentication and authorization, how APIs work and how to call them, networking, protocols and secure access to APIs over the network.

Integration also means meetings with many stakeholders: the network people and the platform owners. Sometimes you have to push support to get information, and maybe also escalate the case to the system vendor. This work requires leadership, soft skills and the ability to translate needs into the language of different teams.

Maintaining the agent requires automated answer tests

An eval strategy is an automated assessment of the agent’s answers against an expected answer, prepared from business cases and customer stories. The team is still working on it. LLM-based and non-LLM mechanisms automatically run a set of use cases and assess whether the agent answered sensibly and reliably, and whether it was proactive and friendly or curt and unhelpful.