Test Atlas

Ask Atlas a question in the dashboard and see what it would do in a ticket, with its reply, its sources, and the tokens it used, or replay your latest closed tickets next to your staff's replies, before members see it

Test Atlas lets you try Atlas before it talks to your members. You type a question the way a member would, and Atlas runs it through the same answer steps as a real ticket, with your saved settings and help content. Instead of posting anything, it shows you what it would do: answer, ask a question first, offer Talk to staff, or stay quiet, with the reply, the sources it used, and why. You can also replay your latest closed tickets and compare what Atlas would answer with what your staff replied. It works while Atlas and Answer questions in tickets are still off, so you can check your help content first and turn answers on when you are happy with them.

Test Atlas needs a Pro or Enterprise plan on the bot owner's account. Only the bot owner can run tests, because each test uses the owner's AI tokens. It also needs something to answer from: at least one source under Help content that is on and shows Ready. Until then the tab links to Help content instead of asking, and replays are hidden. It is not available yet for servers in ModMail mode.

Go to AI assistant → Atlas → Test Atlas

Run a test

Open the Test tab

Go to AI assistant → Atlas → Test Atlas. It is the Test tab of the AI assistant page.

Type a question

Type a question in Question, the way a member would write it in a ticket, up to 2,000 characters.

Ask Atlas

Click Ask Atlas, or press Ctrl+Enter (Cmd+Enter on a Mac). Atlas usually answers within a few seconds. The result appears in the same card, under the question, and stays there while you switch to another tab and back.

Adjust and try again

If the result is not what you want, change your help content or settings, click Save changes, and ask again. Tests use your saved settings, so unsaved changes are not tested; the tab tells you when the page has some.

What the result shows

The result appears under the question box, like a short chat: your question first, then what Atlas would do.

PartWhat it shows
You askedYour question
OutcomeNext to the Atlas mark: what Atlas would do in a ticket. See the table below
WhyThe reason for the outcome, for example "Nothing in your help content answers this question." A plain answer from your help content has no reason, because its reply and sources show it
ReplyThe text of the reply in a box, formatted as in Discord. In a ticket it is posted with its Source: line, the Generated by AI notice, and vote buttons when Let members rate answers is on. For Atlas would ask a question first, it is the question Atlas asks the member
Atlas's noteWhen Atlas cannot answer, what it found missing. When it answers and hands the ticket to staff anyway, why it hands over
SourcesUp to 5 sources the reply was built from, on one line. Web sources link to the page
TokensHow many of the bot owner's AI tokens the test used

Outcomes

OutcomeWhat happens in a real ticket
Atlas would answerAtlas replies from your help content
Atlas would repeat an earlier answerAtlas answered the same question in a ticket recently and sends that answer again. Repeats use no tokens
Atlas would answer from general knowledgeYour help content does not cover the question, and Answer general tech questions is on under AI assistant → Atlas → Where else Atlas may look, so Atlas answers from what it knows in general
Atlas would ask a question firstThe question is unclear, so Atlas asks the member to explain before it answers
Atlas would offer Talk to staffAtlas cannot answer, so it offers the member the Talk to staff button
Atlas would hand the ticket to staffAtlas passes the ticket to your staff right away, for example when the member asks for a person
Atlas would stay quietAtlas does not reply, for example to small talk or a thank-you after an answer
Atlas would not answerAtlas does not look for an answer at all, for example because the message is too short or your server is open while Atlas is set to answer only outside your business hours

When Why says Atlas sends the answer and then hands the ticket to your staff, the reply goes out and the ticket is handed over after it, because the reply itself says staff need to step in.

Replay recent tickets

The Replay recent tickets card on the same tab runs your latest closed tickets through Atlas, one after another, and shows for each ticket what the member asked, what Atlas would answer today, and the first reply your staff gave.

Choose how many tickets

Go to AI assistant → Atlas → Test Atlas and pick Last 5 tickets or Last 10 tickets on the Replay recent tickets card.

Replay

Click Replay. Before you start, the card estimates the tokens from your last test when you ran one. Each ticket shows its result as soon as it is done, with the tokens used so far.

Stop if you need to

Click Stop to end the replay. The ticket in progress still finishes and uses its tokens.

Which tickets are replayed:

  • The server's latest closed channel and thread tickets, newest first, with at least two messages.
  • Not ModMail tickets, and not tickets whose messages were removed by your retention settings.

What each ticket shows:

PartWhat it shows
Member askedWhat the ticket was opened with (the message or the form the member submitted), else the member's first message with text. Questions longer than 2,000 characters are shortened
Atlas todayThe outcome, why, and the reply. Sources are shown only for single tests
Staff repliedThe first reply staff gave after the question, including replies sent with /reply or from the dashboard. When nobody replied, it says so, and says when Atlas answered the ticket
TokensHow many tokens the replay of this ticket used

How Atlas answers a replayed ticket:

  • It uses your help content and saved settings as they are now, and the ticket's category, like a new ticket in that category.
  • It leaves out what it learned from that ticket: the learned answers that came from it (the ticket shows how many), and the ticket's own conversation when it searches past Discord conversations. It never repeats a saved answer.
  • A learned answer from the ticket that was later merged into another answer, when Atlas tidies up learned answers, can still be used.

A ticket shows "The member did not write a question to replay" when the member never wrote anything, and "This ticket can no longer be replayed" when it was reopened, deleted, or its messages were removed in the meantime.

What a test does and does not do

  • Nothing is posted. A test posts nothing in Discord, creates no ticket, pings no one, and does not post the hand-over card.
  • The same answer steps and settings. A test uses your saved help content, About your server (including Topics to leave to staff), Reply length, Where else Atlas may look, and the Outside hours choice of When staff are around, like a real ticket.
  • It works before answers are on. A test runs while the Atlas switch or Answer questions in tickets is off. Members get answers only once both are on; the tab reminds you while they are off, with a link to the switch.
  • Each test is a new ticket's first message. There is no earlier conversation and no staff in the ticket, so a test never waits for staff (Staff first) and Stay quiet once staff reply does not come into play.
  • Member settings do not apply. Limit per member, Keep Atlas away from, and panel buttons that turn Atlas off are about real members and their tickets, so tests ignore them.
  • It leaves no trace. Tests do not count in Stats, do not become learned answers in Staff Q&A, and are not remembered: asking a follow-up starts over.

Tokens

Each test uses the bot owner's AI tokens like a real answer, billed the same way. The result shows how many the test used, and the line next to Ask Atlas shows how many tokens are left this month, with a Buy more link. A repeated answer uses no tokens. On AI usage, tests count as knowledge base usage.

Limits and gotchas

Tests use the saved settings. After changing help content or settings, click Save changes before testing them. A new web or Discord channel source is used once its first read finishes and it shows Ready under Your sources.

  • You can run up to 10 tests a minute. A replay is counted separately: up to 20 replayed tickets a minute, and up to 10 tickets in one replay.
  • Search past Discord conversations under Where else Atlas may look needs the bot to be online. While it is offline, a test answers without searching your Discord.
  • ModMail servers answer in DMs in a different way, so Test Atlas is not available for servers in ModMail mode yet.
  • Answers can vary a little between runs of the same question, as they do in tickets.

Troubleshooting

SymptomCauseFix
There is no question box, only a note that only the bot owner can run testsYou are dashboard staff, not the bot ownerAsk the bot owner to run the test
"Atlas has no help content on this server yet"Atlas has never been set up on this serverAdd a source under AI assistant → Atlas → Sources, then test again
"Testing is not available yet for servers in ModMail mode"The server takes tickets by DMTest Atlas works only for servers that use panels
"Too many tests. Wait a minute and try again."More than 10 tests, or more than 20 replayed tickets, in a minuteWait a minute
"There are no closed tickets to replay yet."The server has no closed channel or thread tickets with a conversation, or their messages were removedClose a few tickets first, or test with your own questions
The result says There aren't enough AI tokens left this monthThe bot owner's monthly tokens are used upCheck AI usage; wait for the reset or add a token package
The result says Nothing in your help content answers this question for a question you documentedThe source is still being read, or the page that answers it is not in your sourcesWait until the source shows Ready, or add the page that answers it as a source
The result ignores a change you just madeThe change is not savedClick Save changes, then ask again
Go to AI assistant → Atlas → Sources

Common questions

Can I test Atlas before turning it on? Yes. Test Atlas works while the Atlas switch and Answer questions in tickets are off. Members get nothing until you turn both on.

Does a test post anything in my Discord server? No. A test posts nothing, creates no ticket, and pings no one. It only shows the result in the dashboard.

Do tests cost tokens? Yes. Each test uses the bot owner's AI tokens like a real answer, and the result shows how many. A repeated answer uses no tokens.

Why does a test not show my latest change? Tests use your saved settings. Click Save changes, then ask again.

Who can run tests? Only the bot owner, on a Pro or Enterprise plan, because tests use the owner's AI tokens. Dashboard staff see the tab without the question box.

Do tests show up in Stats or teach Atlas anything? No. Tests do not count in Stats, do not create learned answers, and are not remembered between questions.

Can I see what Atlas would have answered in tickets we already handled? Yes. Replay recent tickets on the Test tab replays your last 5 or 10 closed tickets and shows Atlas's answer next to your staff's first reply. Atlas answers without what it learned from each ticket, so the comparison is fair.

How much does a replay cost? Each replayed ticket uses about as many tokens as one test. The card estimates the total from your last test, and shows the tokens used as it runs. Stopping still finishes the ticket in progress.

Can I test a ModMail server? Not yet. Test Atlas is not available for servers in ModMail mode.

Last updated on

On this page