Adding Sources
How to add web pages, Discord channels, files, and GitHub repositories as knowledge base sources, and keep them fresh
Sources are the content your bot answers from. This page walks through each kind of source, the fields you fill in, what the status next to a source means, how refreshing works, and what to do when a source cannot be read. It is written for the admin who maintains the documentation.
Pro allows 2 sources per server and Enterprise 15. GitHub repository sources need Enterprise. File and GitHub repository sources can only be managed by the bot owner; dashboard staff can manage web page and Discord channel sources.
Adding a source
Open Sources
Go to AI assistant → Atlas → Sources, the first card on the Help content tab. Atlas answers from your sources once AI assistant → Atlas → Answer questions in tickets is on.
Choose what to add
In the Add a source card, under What do you want to add?, pick Web page, Discord channel, File, or GitHub repository and fill in the fields listed below.
Click Add source
The button stays disabled until the required fields are filled in. At your plan's limit the card shows Source limit reached instead. Remove a source or upgrade to add more.
Save
Click Save changes in the bar at the bottom of the page. Until then the source is listed under Your sources as Not saved yet. A file skips this step: it is saved as soon as you add it. After saving, the first read starts by itself, and the source is ready to use when its status reads Ready. When reading it would use more than a few AI tokens, the source first shows Needs your OK with an estimate; see Approving the AI cost.
Source types
Web page
For a single page, or a page and the pages it links to.
| Field | Values | Default |
|---|---|---|
| Page address | A complete https:// URL. Private addresses, localhost, and URLs with a username or password are rejected | Required |
| Name | Any name, for example Help center. Shown with answers that use this source | Required |
| How many pages to read | Just this page, Up to 5, 10, 15, 20, 25, 50, 75, or 100 pages; Up to 500 pages (Enterprise) | Just this page |
| Check for changes | Only when I ask, Every day, Every week, Every month | Only when I ask |
| Description (optional) | A note about what the source covers | Empty |
With more than one page, the bot starts at your address, follows the links it finds, and reads pages until it reaches the limit. This is a maximum, not a target: a small site may have fewer pages. Linked pages on the same site, including its documentation subdomains, policies and public account help, are included. Duplicate pages count once. The status shows Finding pages, then Reading 12 of 40 pages, then (for a larger read) Needs your OK until you start it, then Ready, 40 pages.
Point the bot at a pricing page, an FAQ, or a guide rather than your home page. Start with a small page count and raise it if answers are missing.
Google Docs, Sheets, and Slides
Paste the file's share link as the Page address. The bot reads the file itself, not the editor page around it, so How many pages to read does not matter: one file counts as one page.
- In Google, choose Share and set General access to Anyone with the link (Viewer is enough). A private file shows Google file not shared.
- A Google Doc is read with its headings, lists, and tables. A Google Sheet is read as a table, one tab only: the tab open in the link you copied, or the first tab when the link names none. Slides are read as text; images and speaker notes are not.
- Check for changes reads the file again, so edits in Google reach the bot on the schedule you pick.
Discord channel
For content that already lives in your server, such as an FAQ or announcements channel.
| Field | Values | Default |
|---|---|---|
| Channel | A text or announcement channel the bot can see and read the history of. Don't see it? Reload from Discord at the bottom of the list picks up new channels | Required |
| Name | Any name; choosing a channel fills in #channel-name | Required |
| Include bot messages | On, Off | Off |
| Description (optional) | A note about what the channel covers | Empty |
Only the newest 20 messages are used on Pro and the newest 100 on Enterprise. Very short messages and Discord's own notices (a new thread, a pinned message, a followed channel) are skipped. When the limit is reached, each new message replaces the oldest ones, so the source always holds the latest messages. A curated FAQ channel works far better than a busy chat channel.
A message's text is read along with the text of its embeds (title, description, and fields) and of its text blocks when it uses Discord's newer message layout. Link previews that Discord adds under a pasted link are not read. An FAQ posted by a bot or a webhook, which is how most embeds are sent, is only read while Include bot messages is on. The status shows Ready, 18 messages when done, and Updated on the line under the source's name shows the last time a channel message was saved.
The bot only reads channels it has permission to view. Do not add channels that contain private information; anything the bot reads can be quoted in answers.
File
For a FAQ, a policy, a product guide, or notes you keep in a document. In What do you want to add? this is File. Only the bot owner can add or remove files.
| Field | Values |
|---|---|
| File | A .txt, .md, or .docx file with at most 50,000 characters (256 KB) of text. A Word file can be up to 10 MB. Older .doc files, PDFs, and macro-enabled .docm files are not accepted |
| Name | Any name; choosing a file fills in its file name |
| Description (optional) | A note about what the file covers |
Any layout works. A well organised file with one topic per section gives the best entries, but headings, numbering, and formatting are optional. When you add the file:
- Your browser reads the file and sends only its text. The file itself is never uploaded or opened on TicketCord's servers, and nothing in it is run. From a Word file, headings, lists, tables, and links are kept as text; images and embedded objects are ignored.
- The full text becomes searchable, so the bot can answer from any part of it. The source shows Ready, N sections.
- AI then reads the file and turns it into question-and-answer entries under Staff Q&A, using only what the file says. It never follows instructions written inside the file, and an entry that would contain instructions for an AI is discarded. While it works, the source shows Creating Q&A entries.... A long file takes a few minutes.
- Entries go live right away, like entries you add with Add an answer. An entry whose answer mentions a link, email address, or number that is not in the file waits as Waiting for your review instead, with a note listing what was not found.
- An entry that asks the same question as one already in your knowledge base, or says nearly the same thing, is skipped and counted as already there. One file adds at most 100 entries.
When it finishes, the line under the file's name shows the result, for example 12 Q&A entries added · 3 waiting for review · 2 already in your knowledge base. Under Staff Q&A, entries from a file show the file's name. See Q&A Entries for reviewing and editing them.
Answers that come from the file, or from the entries made from it, name the file's source title. Keep the title short and plain, for example Refunds: a title of three or more words, or one containing #, /, or _, is never shown to customers, so entries from that file are labelled Staff Q&A instead.
When a file gives no entries
If the file has nothing to turn into entries, the source says why instead of a count. Its text stays searchable either way.
| The source says | What it means | What to do |
|---|---|---|
| this file has nothing customers would ask support about | The text is readable but holds nothing a customer would ask, such as a list of names or raw data | Nothing, or add a file with your FAQ, policies, or guides |
| this file looks like internal notes for your team | The file is written for your team, such as internal procedures or staff schedules | Remove it if it is private: its text can still be quoted to customers |
| this file gives instructions to the AI instead of information | The file is a prompt or a set of rules for how the bot should reply | Remove it and put the rules in Instructions for Atlas on the About your server card |
| this file is a template or placeholder text | Empty headings, TODO notes, or sample text | Fill it in and add it again |
| the AI could not read the text in this file | The text is garbled or encoded, for example after a broken export | Save it again as plain text or a Word document and add it again |
| this file mainly holds passwords, keys or other secrets | Secrets are never turned into entries | Remove the file: its text can still be quoted in answers |
| this file has too little text to answer a question | Only a title or a few words | Add the full content |
| the file only hints at answers without stating them | The answers are implied rather than written out | Write each answer out plainly and add the file again |
When it does not finish
- The last attempt did not finish, so it will try again shortly.: the AI service did not respond, and the source tries again by itself a few minutes later. After three failed attempts it shows Could not create Q&A entries because the AI service did not respond. and Try again.
- The Q&A entries could not be saved. or Could not create Q&A entries.: click Try again.
- The AI could not read N of M parts of the file: the other parts were turned into entries. Try again reads only the missing parts and adds what it finds to the counts. Entries that already exist are not added twice.
- No Q&A entries were created because you are out of AI tokens. or Stopped early because you ran out of AI tokens.: creating entries uses AI tokens from the bot owner's allowance. Add tokens, then click Try again to read the parts it did not reach.
- Your current plan does not include Q&A entries from files.: file entries need a plan with knowledge base sources. After upgrading, click Try again.
- Q&A entries will be created once the file is ready and turned on.: the file is still being read, or it is switched off. Switching it on starts it.
Try again only appears while the file is switched on and ready.
Files cannot be edited after upload. To update one, remove it and add the new version. Removing a file also removes the Q&A entries it created, except entries you edited, which stay. Switching a file off stops answers from both its text and the Q&A entries it created, except entries you edited.
A file that shows Added before files became Q&A entries has no entries. Remove it and add it again to create them.
GitHub repository (Enterprise)
For documentation kept as Markdown in a repository.
Connect GitHub
In Add a source, choose GitHub repository and click Connect GitHub. Only the bot owner can connect or disconnect GitHub. Grant the GitHub App access to the repositories you want to use.
Choose the repository
Pick the Repository. Branch starts on the repository's default branch and Name is filled in from the repository name; change either if you like. Add an optional Folder such as docs/ to use only the Markdown files in that folder, or leave it empty to use every Markdown file. Click Add source, then Save changes to start the first read.
Let it sync
The source updates by itself on every push to the chosen branch. The status shows Ready, 24 files, and the number of saved files under the source opens the list of file paths.
To change the branch or folder later, open the source's ⋯ menu and choose Change branch or folder. Changing either clears what the bot read, so save and then choose Read it again now. Each repository and branch pair can be added once.
Disconnect removes every GitHub repository source on the server. If the dashboard says GitHub paused the connection, click Reconnect to install the GitHub App again.
Source statuses
Each source under Your sources shows its status next to it:
| Status | Meaning |
|---|---|
| Not saved yet | Added in the card but not saved. Click Save changes |
| Waiting to be read | Saved but not read yet |
| Finding pages / Reading 12 of 40 pages / Preparing 30 of 40 pages / Reading it now | The bot is reading it. The ring fills as it goes |
| Needs your OK | The pages were read, and preparing them for answers waits for you to approve the AI cost. See Approving the AI cost |
| Paused | The bot owner ran out of AI tokens partway. The finished pages are kept. See When the tokens run out |
| Not read yet | Someone chose Not now on the estimate. Choose Read it again now for a new estimate |
| Ready, N pages / messages / files / sections | Ready to answer from |
| Ready, 12 of 40 pages | Only some of the pages are ready, for example after a read stopped when the tokens ran out. The bot answers from those pages |
| Could not read it | The last read did not finish. Hover the info icon next to it for the reason |
| Domain not found | The website's domain does not exist (see below) |
| Redirect detected | The address sends visitors to a different website (see below) |
| Blocked by bot protection | The website's bot protection, such as a challenge page, captcha, or firewall, refused the bot. See Let TicketCordBot through bot protection |
| Discord link | The address is a Discord invite or channel link, which cannot be indexed |
| Google file not shared | The Google Docs, Sheets, or Slides file is private or was deleted. See Google Docs, Sheets, and Slides |
| Reconnect GitHub | GitHub access to the repository must be renewed |
| Not in your plan | Your current plan no longer includes this source |
Each source also has an on/off switch. A switched-off source keeps its content but is not used for answers until you switch it on again. For a file, that includes the Q&A entries it created, except entries you edited. The line under the name shows where the source comes from, how often it is checked for changes, and when it was last updated.
A source that fails keeps answering from the content it had before, so a temporary outage does not empty your knowledge base. Redirect detected, Discord link, Google file not shared, and Reconnect GitHub pause Check for changes for that source until you edit it or choose Read it again now. Domain not found stops it if the domain stays unreachable across several attempts.
Approving the AI cost
Reading a web page or GitHub source happens in two steps. First the bot fetches the pages and splits them into sections, which costs nothing. Then AI prepares each page so the bot can answer from it, which uses AI tokens from the bot owner's allowance. The AI usage page lists these tokens under By feature as Reading knowledge sources, separate from Knowledge base, which is what answering questions used.
When the second step would use more than a few tokens, the source stops at Needs your OK and shows one sentence with the estimate, for example Reading 40 pages will use about 1.2M AI tokens. When some pages have not changed since the last read, the sentence adds how many of them are free. When the estimate is more than the bot owner has left, it says so and how many pages fit: the read then stops after about that many pages and keeps them.
- Start begins the read. If the estimate changed since you opened the page, the dashboard says The estimate changed and shows the new number. Check it and click Start again.
- Not now spends nothing. A source that was already Ready keeps answering from its last read; a new source shows Not read yet. Choose Read it again now from the ⋯ menu when you want a new estimate. Automatic reads leave a declined source alone for a while: scheduled checks skip one round, and GitHub pushes wait a week. A new source someone declined is only read again when you ask.
Start and Not now (and Continue on a paused source) appear only while Atlas and Answer questions in tickets are both on and saved, because Atlas uses sources only then. While either is off, the line under the source says which one to turn on, with a link to it.
The bot owner and dashboard staff can both answer, except on file and GitHub sources, which only the bot owner manages. AI tools connected to your account can see the estimate but cannot start the read.
Automatic reads, from a Check for changes schedule or a GitHub push, go ahead on their own only when both are true: the cost is within the allowance for reads nobody is watching, which is the largest of 25,000 tokens, 2% of the bot owner's tokens for the month (bought tokens included), and a quarter above the last estimate someone approved for that source; and it is at most half of the tokens the bot owner has left. Changing what a source reads (its address, page limit, rendering, or GitHub repository, branch or folder) means earlier approvals no longer count; renaming it does not. Otherwise they wait at Needs your OK like a first read. A push does not read a source that is waiting for an answer, and continues a paused one only once the bot owner has tokens again. When an update to the bot improves how pages are prepared, every source is read again the same way, over the following days; during a trial it is not, and the last read keeps answering.
When the tokens run out
Pages are saved and paid for one at a time. If the bot owner's AI tokens run out partway through a read, the read stops and the source shows Paused with Out of AI tokens. 12 of 40 pages are ready.
- The finished pages are kept. On a first read the bot already answers from them.
- The read continues by itself when the bot owner has tokens again, or straight away with Continue (the bot owner also sees a link to buy more tokens).
- Pages that were already finished are never paid for twice.
Refreshing a source
- Read it again now in the source's ⋯ menu reads it again straight away. You see Reading it again and the status returns to the reading states. You can do this once every 8 hours after the source last finished reading.
- Check for changes in the same menu reads web pages and Discord channels again Every day, Every week, or Every month. Only when I ask turns the schedule off.
- GitHub sources refresh on every push. Discord channel sources save new messages as they arrive, and edited or deleted messages are updated within a few seconds.
- Reading a web page or GitHub source uses AI tokens from the bot owner's allowance. A refresh skips the web pages and GitHub files that have not changed since the last read, and the skipped pages cost nothing. A larger refresh can wait for approval; see Approving the AI cost.
Removing a source
Open the source's ⋯ menu, choose Remove this source, and confirm. The bot stops answering from the source right away, and this cannot be undone.
What the bot read from a web page source, including Google Docs, Sheets and Slides, is kept for 7 days and then deleted for good. If you add the same source to the same server again within those 7 days, the pages that have not changed cost no AI tokens. Everything read from a GitHub repository, a file or a Discord channel is deleted at once.
Removing a file also deletes the Q&A entries created from it, except entries you edited. Answers already posted in tickets stay in Discord.
Limits and gotchas
- A web page source's address and page limit cannot be edited in the dashboard after adding it. To change them, remove it and add a replacement. Pages the replacement shares with the removed source cost no AI tokens for 7 days; see Removing a source.
- Sources cost nothing to keep, but every answer costs tokens from the bot owner's allowance. The one exception: a question the bot already answered for your server in the last day, asked again as the first message of a ticket, is answered from memory at no cost. Any change to your sources, Q&As or settings clears that memory.
- A web page that needs a login, or that only shows content after running scripts, may be read with little or no text. Check the Ready count after the first read.
- A web page larger than 10 MB, as the site sends it or once it has finished loading, is not read. In a read of several pages it is left out; a source with Just this page shows Could not read it.
- A read of several pages stops after 90 minutes, or sooner when the site stops answering in time. A source with a complete read keeps that content and shows Could not read it. A source's first read keeps the pages it got and shows Ready, 12 of 40 pages; a later read, scheduled or started with Read it again now, goes through the site again and adds the missing pages.
- A read that gets interrupted again and again stops after three retries and shows Could not read it. Choose Read it again now to start over.
- A Google Docs, Sheets, or Slides file is read only while it is shared with Anyone with the link. A Google Drive link to an uploaded file, such as a PDF, is not read: download the file and add it as a File.
- The links in your pages, files, and channels are kept, so an answer can send the customer straight to the page or dashboard setting your content links to, placeholders such as
:serverincluded.
Let TicketCordBot through bot protection
The bot reads web pages with a browser user agent that ends in this token:
TicketCordBot/1.0 (+https://ticketcord.com/docs/ai/knowledge-base-sources#allow-ticketcordbot)If your website sits behind bot protection (a challenge page, a captcha, or a firewall that blocks automated visitors), the source can show Blocked by bot protection. Add a rule in your firewall or CDN that allows requests whose user agent contains TicketCordBot, then choose Read it again now. A source with Check for changes set to a schedule also picks the change up by itself on its next check.
On Cloudflare:
Create a custom rule
Go to Security → Security rules, choose Create rule → Custom rules, and give it a name such as Allow TicketCordBot.
Match the user agent
Set Field to User Agent, Operator to contains, and Value to TicketCordBot.
Skip the protections
Set the action to Skip and select All remaining custom rules, All rate limiting rules, All managed rules, and All Super Bot Fight Mode rules. Deploy the rule and move it above any rule that blocks or challenges visitors.
Cloudflare's free Bot Fight Mode ignores custom rules, so no rule can let the bot through while it is on. Turn it off under Security → Settings, or use Super Bot Fight Mode (Pro plan and up), which the rule above can skip.
Anyone can send a user agent, so this rule lets through every request that claims to be TicketCordBot. Only allow it on a site whose content is public anyway, which is the content a knowledge base should hold.
Troubleshooting
| Symptom | Cause | Fix |
|---|---|---|
| Invalid URL when adding | The address is missing https://, uses http://, points at a private address, or contains a username or password | Use the full public HTTPS address |
| Source limit reached | Your plan's source limit is used up | Remove a source you no longer need, or upgrade |
| The source shows Could not read it | The page requires a login, blocks automated readers, or was temporarily unavailable | Open the address in a private browser window to confirm it loads without signing in, then choose Read it again now. If the site uses bot protection, let TicketCordBot through |
| The source shows Blocked by bot protection | The website's challenge page, captcha, or firewall refused the bot | Let TicketCordBot through, then choose Read it again now |
| The source shows Domain not found | The domain expired or the address has a typo | For a typo, remove the source and add a new Web page with the correct Page address. For an expired domain restored at the same address, choose Read it again now |
| The source shows Redirect detected | The address forwards to a different website, for example a docs site that moved to a new host. For your security, only the website you entered is read | Remove the source and add a new Web page with the final address in Page address |
| Discord links can't be indexed when adding, or the source shows Discord link | The address is a Discord invite or channel link | Add the channel as a Discord channel source instead |
| The source shows Google file not shared | The Google Docs, Sheets, or Slides file is private, or it was deleted | In Google, choose Share and set General access to Anyone with the link, then choose Read it again now. If it already is, open the settings (gear icon) in the same dialog and turn on Viewers and commenters can see the option to download, print, and copy |
| The source shows Reconnect GitHub | GitHub access for the repository changed or predates the current connection | Reconnect GitHub under Add a source → GitHub repository, then choose Read it again now. If you uninstalled and reinstalled the GitHub App, remove the source and add the repository again |
| The source shows Not in your plan | Your current plan no longer includes this source | Upgrade to a plan that includes it to read it again |
| Ready, 0 pages or a very small count | The page has little visible text, or How many pages to read is Just this page for a site with many pages | Remove the source and add a new Web page with an address that has real text, or a larger How many pages to read choice |
| Read it again now is greyed out | The source is not saved yet, is being read right now, or Atlas or Answer questions in tickets is off or was turned on but not saved | Turn on Atlas and Answer questions in tickets and click Save changes, or wait for the current read to finish |
| Could not read it again | The source finished reading within the last 8 hours | Wait for the cooldown to end |
| The source shows Needs your OK | Preparing the pages would use more than a few AI tokens, or an automatic read would cost more than it may spend without asking | Check the estimate under the source, then click Start or Not now. See Approving the AI cost |
| The source shows Needs your OK but no Start button | Atlas or Answer questions in tickets is off, or was turned on but not saved | Follow the link under the estimate, turn the switch on and click Save changes. Start and Not now then appear |
| The estimate changed after clicking Start | The source was read again since you opened the page, so it has a new estimate | Check the new number under the source and click Start again |
| The source shows Paused and Out of AI tokens | The bot owner's AI token allowance ran out during the read | Add tokens. The read continues by itself, or click Continue. Finished pages are kept and not paid for again |
| Answers leave out a link that a web page contains | The page was read before the bot kept the links in web pages | Choose Read it again now |
| Unsupported file | The file is not a .txt, .md, or .docx file | Save it as one of those formats, for example Save As → Word Document (.docx) for an old .doc file |
| File is too large | The file has more than 50,000 characters (256 KB) of text, or a Word file is over 10 MB | Split it into several files, one per topic area |
| Could not read the file | The file is damaged, password protected, or not really the type its name says | Save it again as .txt, .md, or .docx without a password and choose it again |
| The file has no text | The file is empty, or a Word file contains only images | Add the content as text |
| A file source says No Q&A entries: followed by a reason | The AI found nothing in the file to turn into entries | Follow the advice for that reason in When a file gives no entries. The text is still searchable |
| No Q&A entries were created because you are out of AI tokens. | The bot owner's AI token allowance ran out | Add tokens, then click Try again on the source |
| Only the server owner can change this source on a file or GitHub source | You are a dashboard staff member | Ask the bot owner to add or change it |
| Duplicate source | That repository and branch already exist as a source | Change the existing source's branch or folder instead |
| No repositories found | The GitHub App was not given access to the repository | Update the app's repository access in GitHub, then reload the page |
Common questions
How many sources can I add? 2 on Pro and 15 on Enterprise, per server.
How do I read a whole documentation site? Add it as a Web page and set How many pages to read to the number of pages you need. Pro reads up to 100 pages per source and Enterprise up to 500.
Does the bot read my whole Discord channel? No. It uses the newest 20 messages on Pro and 100 on Enterprise, and skips very short ones. As new messages arrive, the oldest ones drop out. Keep the channel curated.
How often is a source updated? Whenever you choose Read it again now (once every 8 hours), on the Check for changes schedule for web pages and channels, on every push for GitHub repositories, and as new messages arrive for Discord channels. A scheduled or push update that would cost more than it may spend without asking waits for Start first; see Approving the AI cost.
Can a read use up all of the bot owner's tokens without asking? No. A read that uses more than a few tokens shows its estimate and waits for Start, and an automatic read goes ahead on its own only when it costs no more than the allowance for unwatched reads (the largest of 25,000 tokens, 2% of the month's tokens, and a quarter above the last approved estimate) and at most half of the tokens left. A read that does run out stops and keeps the pages it finished.
Can I edit a file source? No. Remove it and upload the corrected file. Its Q&A entries are removed with it, except ones you edited, and the new file creates new entries.
Is my file uploaded to TicketCord? No. Your browser reads the file and sends only its text, which is stored for the bot to answer from. The file itself is never uploaded or opened on TicketCord's servers, and nothing in it is run.
Does my file need headings or a special format? No. Any layout works, including plain notes and Word documents. AI reads the content and turns it into Q&A entries, and the full text is searchable as well.
Why is a Q&A entry from my file waiting for review? Its answer mentions a link, email address, or number that the file does not contain. The entry shows which ones. Check the answer, edit it if needed, and approve it.
Why can't I add a GitHub repository? GitHub sources need an Enterprise plan, and only the bot owner can connect GitHub.
Will removing a source delete answers the bot already posted? No. Messages in Discord stay; only the saved content is deleted, so the bot stops using it for new answers.
Join our Discord server for support
Was this page helpful?
Last updated on