This tutorial will cover deploying the bot to Chai, a platform for creating and interacting with conversational AI's. It will allow us to chat with our bot through a mobile app. We will also be able to see performance stats and watch it climb the Chai bot leaderboard.

By the end of this tutorial you will have your very own chatbot, like the one pictured above 😎

I've also made a notebook containing all the code in this blog which is much more convenient than copying and pasting all the code snippets by hand. Go check it out if you're interested!

Finding a chatbot

We can use huggingface 🤗 for this. It's home to hundreds of state-of-the-art NLP models. We're going to use Facebook's Blenderbot for this tutorial as it's one of the best around at the moment. It learns from the conversation and gets smarter as you chat!

Trying the bot out

Here's an example conversation: If you'd like to try the bot out yourself, before deploying, you can use HuggingFace's Transformers module (otherwise skip to the next section):

pip -q install transformers

Run the following python code:

from transformers import BlenderbotTokenizer, BlenderbotForConditionalGeneration
import torch

model = BlenderbotForConditionalGeneration.from_pretrained("facebook/blenderbot-400M-distill")
tokenizer = BlenderbotTokenizer.from_pretrained("facebook/blenderbot-400M-distill")

print("Type \"q\" to quit")
while True:
    message = input("MESSAGE: ")
    if message in ["", "q", "quit"]:
        break
    inputs = tokenizer([message], return_tensors='pt')
    reply_ids = model.generate(**inputs)
    print(f"BOT:     {tokenizer.batch_decode(reply_ids, skip_special_tokens=True)[0]}")

Pretty cool right?

Deploying the chat bot to Chai

Chai's mobile app allows us to show off the bot really easily, no need to whip out your laptop and fire up a terminal instance!

There's also a bot leaderboard to climb. We can see how our new bot compares to others on the platform:

Install the required python module:

pip -q install --upgrade chaipy

Directory structure

We want a directory structure like this:

- bot/
    - bot.py
- dist.py

The bot code

This is the code that will run whenever a user messages our bot.

Any model on HuggingFace can be used. To change our bot to another model, change the self.model url. You can also add to the past_user_inputs to change the bot's starting context. This changes how the bot will act when the conversation starts.

Before we run this, head over to huggingface.co and create an account. Go to the top right corner to access your settings, where you will find your API Tokens. Copy this and paste it in the line that says <YOUR API_TOKEN GOES HERE>.

bot/bot.py:

import json, requests, time
from chai_py import ChaiBot, Update


class Bot(ChaiBot):
    def setup(self): 
        self.apiToken = "<YOUR API_TOKEN GOES HERE>"  # put your huggingface API token here
        self.model = "facebook/blenderbot-400M-distill" # You can change this

        self.ENDPOINT = f"https://api-inference.huggingface.co/models/{self.model}"
        self.headers = {"Authorization": f"Bearer {self.apiToken}"}
        self.first_response = "Hey!"

    async def on_message(self, update: Update) -> str:
        if update.latest_message.text == self.FIRST_MESSAGE_STRING:
            return self.first_response
        payload = await self.get_payload(update)
        return self.query(payload)

    def query(self, payload):
        data = json.dumps(payload)
        response = requests.post(self.ENDPOINT, headers=self.headers, data=data)
        if response.status_code == 503: # This means we need to wait for the model to load 😴.
            estimated_time = response.json()['estimated_time']
            time.sleep(estimated_time)
            self.logger.info(f"Sleeping for model to load: {estimated_time}")
            data = json.loads(data)
            data["options"] = {"use_cache": False, "wait_for_model": True}
            data = json.dumps(data)
            response = requests.post(
                self.ENDPOINT,
                headers=self.headers,
                data=data
            )
        return json.loads(response.content.decode("utf-8"))["generated_text"]

    async def get_payload(self, update):
        messages = await self.get_messages(update.conversation_id)
        past_user_inputs = ["Hey"] # You can change this!
        generated_responses = [self.first_response]
        for message in messages:
            content = message.content
            if content == self.FIRST_MESSAGE_STRING:
                continue
            if message.sender_uid == self.uid:
                # Put the user's messages into past_user_inputs
                past_user_inputs.append(content)
            else:
                # Put the model generated messages into here
                generated_responses.append(content) 
        return {
            "inputs": {
                "past_user_inputs": past_user_inputs,
                "generated_responses": generated_responses,
                "text": update.latest_message.text,
            },
        }

Time to deploy the bot!

Head over to the Chai Dev Platform to set up your developer account. This allows us to deploy the bot under our own account. Your developer ID and keys can be found at the bottom of the dev page. Paste these into the DEV_UID and DEV_KEY below.

You can change the name, image_url and description to personalise how the bot will appear on the platform.

dist.py:

from chai_py import package, Metadata, upload_and_deploy, wait_for_deployment, share_bot
from chai_py.auth import set_auth

# you can find these on the chai dev page
DEV_UID = ""  
DEV_KEY = ""
set_auth(DEV_UID, DEV_KEY)

package(
    Metadata(
        name="Blenderbot Tutorial",
        image_url="https://i.imgur.com/vs0uUfD.png",
        color="0000ff",
        description="Hey! Come chat with me",
        input_class=Bot,
        developer_uid=DEV_UID,
        memory=3000,
    )
)

bot_uid = upload_and_deploy("bot/_package.zip")
wait_for_deployment(bot_uid)
share_bot(bot_uid)

Time to run dist.py

Success 🎉

Scan the QR code above with your phone and you will be taken to a chat screen with your brand new bot, how cool is that?!

(Make sure you have the Chai app installed on your phone, get it on the App store or Google Play)

Conclusion

And thats it! You now have your very own chatbot 😄

More tutorials:

• Chai docs creating your first bot
• Google Colab tutorial on training a Rick & Morty chatbot on custom dialogue
• Making a Rock Paper Scissors bot