🔁Adding & syncing your website content

Your AI agent will ensure your content is always automatically up to date from your website and help documentation.

Each fortnight (2 weeks) we re-scrape your webpages (looking for any new ones in the process) and add them to your AI agent to ensure the AI's content is always up to date.

How many websites can you add?

On all paid plans, you can add a maximum of 3 websites to crawl, however each website can contain thousands of pages.

How many pages do you add?

On all paid plans there is no limit (subject to fair use) to the number of pages that are added and by default we will look for up to 2,500 pages per site.

Plan	Page Limit
Pro	Unlimited
Enterprise	Unlimited

Can I add more subdomains or sites?

Yes, all paid plans allow you to add additional sub-domains and sites.

Just enter your site URL and click "Add new site".

How do I see which pages have been added to my AI support agent?

To see the pages added, just click on each URL in your Add Website Content section and you'll see a pop-up with a list of all the webpages from that URL

Can I add individual webpages to my AI support agent?

Yes, you can add individual webpages. Just go to your Dashboard > Content > Knowledge > Add website content then click on the domain you want to add the site to, and at the top of the pop-up you will see an option to add a new page (we'll check for duplicates too so you don't add multiple versions of the same page).

The webpage(s) will then be included in any future site syncs.

Can I refresh or update webpages before the fortnightly sync?

Yes, you can update webpages at any time, just go to your Dashboard > Content > Knowledge > Add website content then click on the domain that has the individual page you want to refresh, then from the pop-up, find the page you want to refresh and click the refresh icon next to the page, we'll then force a manual re-sync of that page.

Can I hide or delete individual webpage(s) from the AI agent to prevent it from training on them?

Yes, you can hide or delete individual webpage(s) from the AI agent to prevent it from training on them. To do this, go to your Dashboard > Content > Knowledge > Add website content, then click on the domain that has the individual page/URL you want to delete, then from the pop-up, find the page you want to delete and click the "Visible" toggle button, it will then move to "Hidden".

This will mean that the page/URL will no longer be used to answer customer/user questions.

If you have removed the page from your site permanently, then use the trash can icon to delete the page.

How do I know when a website (or other content) was last synced?

To see when a piece of content was last synced (i.e. when the content was last added to your AI agent), go to your Dashboard > Content > Knowledge, then select your Knowledge source (either website domain or knowledge connection).

When you click into this, next to each piece of knowledge (file or webpage) you should see a "last sync" date.

Can I include or exclude certain sites, sub-folders, directories or pages?

Yes, you can exclude certain sites, sub-folders, directories or pages.

If you want to exclude certain sites or pages from your crawl, please ask via live chat by asking to "Talk to a person" and share the sub-folders to be included or excluded e.g. Exclude: company.com/blog/* OR Include: company.com/products/*.

How can I change or remove the website it has crawled?

To change or delete the website that has been added to your AI agent, click the trash can icon next to the site you wish to remove.