Build a Free Bot That Tracks Whether AI Recommends You

Two things happened this month.

ChatGPT stopped citing Reddit as much. It dropped around 86% in a single day from 3.83% to 0.5% of citations.

But what is more interesting.

Automated traffic passed human traffic on the web. Read that gain.

The internet is now being read more by machines than by people.

This has major impact on how businesses handle their marketing and sales on digitally. Companies are now prioritising their digital strategy for bots over people.

This tutorial helps your figure out is your business being found by these AI bots.

You’ll learn how to set up a bot that runs your buyer questions through AI search every Monday. It will record if your business gets mentioned or which competitor got recommended instead.

We’ll be using Bot Mode in the Hermes desktop app.

It is a new feature that is similar to Grok bot but allows you to use any AI model instead of just grok.

This bot will be designed to be a specialist. It uses its own model, has it’s own memory, and its own recurring schedule.

You can build it once and it will keep on running for you in the background.

I’ll be using Ox Alpha as the example AI model. It’s (currently free & anonymous) model that landed on OpenRouter last week and is free while its preview window is open.

If it’s not available you change one setting and choose an alternative AI model (sidenote: this is the benefit of using open source software).

Free, anonymous, and nobody has claimed it yet

❝

A privacy note. The provider running Ox Alpha is anonymous. They say prompts are not used for training, but they are retained. I’d just make sure this bot is using public marketing questions and not client data.

What You'll Need

The Hermes desktop app on Mac or Windows
A free OpenRouter account
30 minutes for the setup, then it runs itself

You do not need any credit on the OpenRouter account to follow this. The model is free right now.

Step 1: Install Hermes Desktop

Download the installer from hermes-agent.nousresearch.com and run it. Mac and Windows both have one.

If you followed last week's Hermes and OpenRouter setup tutorial you already have this and can skip to Step 3.

Step 2: Connect OpenRouter

Full instructions are above but, quickly.

You will need an OpenRouter API key first. This is where we get access to lots of different AI models. Like ChatGPT, Claude, Grok & free models too. It’s an AI buffet where you can choose what you want.

Sign in at OpenRouter.ai, open Keys in settings, click Create Key, name it something obvious, and copy it into a password manager. You only see it once.

Then in Hermes, open settings, go to Providers, then Accounts, scroll to OpenRouter and paste the key in.

Step 3: Create the Bot

Open the Bots tab in the left sidebar. This sits next to Sessions.

Hit New Bot. The quick path is three fields:

Name: AI Visibility Tracker
Title: AI Visibility Tracker
Description: Checks weekly whether we get recommended in AI search, and who beats us.

That description helps. When you run several bots they can see each other's roles, so write it like you would a job title.

Next open the Advanced and select Provider OpenRouter & Model stealth/ox-alpha. This is what makes the bot free to run. But with each bot you own can choose whatever model you prefer, which is the best part of Hermes.

You create can create specific AI model per bot, not global. So if it’s an “easy” task pick a cheaper fast AI model. Or if choose a smarter model for complex automations.

Once saved you can chat to your bot. Say hello and let it introduce itself. Now you have a permanent teammate vs a throwaway chat conversation.

Step 4: Give It Specific Instructions

I’d recommend this next step. It will save you repeating yourself to your Bot. Hermes calls it a “Soul” you can think about it as a personality and information you would give to a new hire for a task.

In Advanced, find Custom SOUL.md. This is the bot's persona and instructions. It helps the bot run the task and job properly every week and so you don’t have to re-explain yourself to every time.

Paste this in, swapping in your own company name and questions (I’ve made this one specific to digital marketing for a construction company):

# Role

You track whether [YOUR COMPANY] gets recommended when buyers ask AI for help.
You are blunt. You do not soften gaps. A comfortable report is a useless report.

# The questions

1. who are the best commercial fit-out contractors in London
2. my office refurb is running over budget, what do I do
3. should I use a main contractor or manage trades myself
4. what should I look for in a commercial fit-out quote
5. how much does an office fit-out cost per square metre in the UK
...twenty in total

# Method

For each question:
- Search the web for it exactly as written.
- Read the top results properly. Do not skim titles.
- Answer it the way an AI assistant would, using only what you found.
- Record: was [YOUR COMPANY] mentioned, which companies were, in what order,
  and which pages were the sources.

# Report format

- A table: question, mentioned yes/no, companies that appeared
- The three questions where we are most visible
- The three questions where we are absent but a competitor appears
- For each of those three gaps: the page currently winning, and what it does
  that ours does not
- One specific piece of content to publish this week to close the biggest gap

If you are unsure what to put in the above. Just use your new bot to help you come up with the instructions.

Once saved this will live inside the bot. If you ever want to change it you can always just edit your bot details.

Step 5: Write the 20 Questions

This is the part that decides whether the whole thing is useful so don’t rush it.

You’re not writing questions about your company.

Because nobody will types your company name into ChatGPT, unless they already know you.

So your goal is to write what someone asks before they know who to call.

Here’s some example types:

Direct need: "who are the best commercial fit-out contractors in London"
Problem first: "my office refurb is running over budget, what do I do"
Comparison: "should I use a main contractor or manage trades myself"
Qualifying: "what should I look for in a commercial fit-out quote"
Cost: "how much does an office fit-out cost per square metre in the UK"

The most important thing to remember is if it does not sound like something a real buyer would ask then you shouldn’t use it.

Include questions where you would expect a named competitor to come up as well.

Those can be the most useful information as you will find out what is working for them and how you can compete.

Step 6: Run It Once by Hand

The goal of these bots are that they will take a task off your hands and run it on repeat so you don’t have to micro manage.

But before you schedule an automation to run. It’s best to watch it work through the task to catch any issues.

To do this, just chat to your bot and say:

Run the visibility check now.

It will take a few minutes as it will have multiple questions to run through.

When it’s done you can read the output. If the report is vague you can adjust your questions to make it more specific. These will be specific to your business and goals so you may have to tweak and test a few times.

But the more effort you put upfront here the more useful it will be for you in the long run.

To edit your questions just click your bot and choose Edit Profile to get back into the SOUL.md any time.

A real use case example while running the steps above with SlideDrift I found it appeared in 2 out of 20 questions. And I’ve now got ideas of how I can close the gap with competitors.

Step 7: Put It on a Schedule

Here is where Bot Mode starts feeling like an extra pair of hands. With the Bots tab open, the routines panel sits on the right of the conversation. Depending on your build it is labelled Routines or Cronjobs.

Click Create, and the schedule picker asks for frequency first, then only the detail that matters. Set it to weekly, Monday, 8am.

For the prompt, you can use:

Run the visibility check.

Since the instructions are already in the bot's SOUL.md, the routine will just run that on repeat at your scheduled time.

Your results will be in the bot's own chat history, so the report is sitting exactly where you would go to talk to it.

Without getting too technical routines (cronjob) are ordinary Hermes cron jobs underneath, namespaced to the bot, so hermes cron list in a terminal shows them if you ever want to check. (if this doesn’t make sense, you can ask your AI to explain)

Step 8: Read It Like a Marketer

I wouldn’t treat the report as a good or bad signal. It’s data that is there to help you improve your business. Here are things you can look out for:

Where a competitor appears and you do not: Is there a page they have and you do not? Is that one better than yours. This is your sign to add a this task to your marketing for this month.
Which pages get cited: AI answers tend to pull from pages that answer one question directly. They like clear structure and real numbers in them. You can see what AI is finding important and add it to your site.
What changes week to week: This is the whole point of running this weekly. A single report is a good reference point. But 8 reports will show you trends. You can see sudden drops (like Reddit citations mentioned earlier) as a early warning that something changed.

Because everything stays in the bot's chat, the history builds itself. You can scroll back through every week an ask your AI questions as you continually improve.

Step 9: When the Free Model Ends

This is specific to this tutorial but it’s important to state. Since Ox Alpha is a preview. It will either become a named paid model or disappear.

But this isn’t a problem and the above instructions will still work.

To change AI models just right-click the bot, choose Edit Profile, and change the model pin to something like DeepSeek, Grok, GPT Luna Max or if there is another model that is on a free promotion you can use that.

The idea here is that you are free to use whatever model suits your budget and business. You are not locked in.

The workflow stays the same. The instructions, questions, schedule and the whole report history will stay exactly as they are.

This is why this method is something I’m confident will work today and if you are setting it up in 6 months time.

The bot and system is the asset and the model underneath is just a setting that can be changed.

Troubleshooting

Ox Alpha not in the model list: Check the OpenRouter key saved correctly in Providers, then restart Hermes. Models appear grouped under OpenRouter.
Rate limited: The free window is popular. Wait and retry, or split the run into two batches of ten questions.
Report says you appear everywhere: Your questions are too niche and probably include your company name. Rewrite them as problems, not brands.
Report says you appear nowhere: That is likely accurate, and it is the useful answer. Start with the gap the bot flags as biggest.
Routine did not run: Your machine was asleep. Either leave it awake Monday morning, or run Hermes somewhere always on.

A final important note

This bot searches the web and reasons about what it finds.

It is a close for how AI answers work but it isn’t exactly what ChatGPT, Claude etc told a specific person on a specific day. This is constantly changing and evolving day by day. Based on their history, previous chat responses, personalisation, location and history. These all effect what answer will get shown. (kind of like everyones Google/YouTube search results are different and curated to that specific person.)

This is for tracking direction and spotting gaps. It’s free to run and you will get genuine insights. If you find value you can invest in specific software which will run more specific analyse. But start here and you can upgrade when your business is ready.

If you want help with AI integrations like this you can read more about me and how to work together with metalabs.global.

Found this helpful? Forward it to someone who still thinks Google rankings are the whole picture.