Social Squared Chatbot

Git Repository: https://github.com/Lightning-Tools/Social-Squared-Bot

Description Page: https://lightning-tools.github.io/Social-Squared-Bot/description/

This is the Lightning Tools Social Squared forum https://lightningtools.com/product/social-squared chatbot which is based on Dialogflow powered by Google's machine

learning and natural language processing technologies.

How To Install

Figure 1

Go to GitHub page and download SocialSquared.zip file. https://github.com/Lightning-Tools/Social-Squared-Bot/blob/master/SocialSquared.zip

The zip file contains all files related to the agent, Including intents, training phrases and contexts.

Figure 2

Go to https://dialogflow.com/ and login with your Gmail account. Click on Go to console link.

Figure 3

Select Create Agent option and specify a name. I specified SocialSquared and clicked on CREATE button.

Figure 4

Select gear icon then Export and Import tab then IMPORT FROM ZIP option.

Figure 5

Select SocialSquared.zip file you downloaded from GitHub (Figure 1).

Figure 6

Type IMPORT then click IMPORT button to import the file.

Figure 7

Once imported click DONE.

Figure 8

Click Intents and you should see some entries already specified.

Figure 9

Click on Small Talk and make sure that it is enabled (It is enabled in SocialSquared.zip file just to make sure).

The next step is to integrate DialogFlow with Facebook Messenger, Skype and Microsoft Teams. You can integrate Dialogflow with other apps such as Telegram, Spark,

Twilio, Twitter, Viber, Slack etc.

Facebook Messenger

Figure 10

Login to your Facebook and go to https://www.facebook.com/pages/creation/ to create a page. I selected Business and Brand option then pressed Get Started.

Figure 11

Specify Page Name and Category and click on Continue. Your page will be created.

Figure 12

I already created Social Squared Chatbot page. Now, it is time to create a Facebook application.

Figure 13

Figure 14

Specify Display name for your app and your email then click on Create App ID button. Your Facebook app will be created. I already created Social Squared Facebook app

Figure 15

Select Social Squared agent then select Integrations tab. Enable Facebook Messenger option.

Figure 16

Facebook Messenger dialog will open. Verify Token and Page Access Token input boxes will be empty (I already specified) in your case. You will only see Callback URL

which is the URL that Facebook application will call each time when somebody speaks to the bot. It remains to connect Dialogflow to Facebook.

Figure 17

Go to Facebook app page, go to Messenger -> Settings and select the page you created. In my case it is Social Squared Chatbot.

Figure 18

You will see the access token in Page Access Token area, after selecting the page.

Figure 19

Copy the access token and paste to DialogFlow Page Access Token field.

Figure 20

It remains to add a verify token which should be a long unique password. Both Dialogflow and Facebook have to have the same verify Token. This it the token they will use

to verify the request.

Figure 21

Click the Start button. Now, fallback URL is active. We should go back to Facebook and finish webhook setup.

Copy the Verify token to clipboard and go back to Facebook app.

Figure 22

Instead of Edit Events button you will see Setup Webhooks button. Click it.

Figure 23

First, paste Verify Token from the clip board. Next, copy Callback URL from Dialogflow (Figure 21) to Callback URL field. This is where the communication will take place.

Facebook will call this URL and get the response from Dialogflow. Now, we subscribe webhook to messages and messaging_postbacks. We do not need any other events.

Click Verify and Save. Facebook will call the Callback URL and see if the URL response is OK to verify tokens matching.

Figure 24

It remains one more step to finish up. We need to select a page to subscribe the webhook to the page events.

In you case they will be Subscribe button to subscribe.

Figure 25

The connection between Dialogflow agent and Facebook app is now made.

Figure 26

Go to the page you created and select View as Page Visitor option.

Figure 27

Click Send Messages to talk to the bot. You will see three options. Click one of them to start. I click Get Started.

Figure 28

The bot does not behave as expected. You can see after Get Started there is no bot response but when I type hi there is a response. I type can you help me? and

again, no response. The reason is that text responses specified on DialogFlow UI can be retrieved and displayed (more about it you can read on

https://lightning-tools.github.io/Social-Squared-Bot/description/ description page). This is the hi case.

The response Greetings! How can I assist? is a Dialogflow text response so it can retrieve and displayed, but the response to can you help me? is not specified on

Dialogflow UI, so no response from the bot. In this case we use so called Dialogflow fulfilment. What it actually means is that we send response via our custom

back end app. Dialogflow will make an HTTP POST request to our backend code and get a response from there. This means that we should host our backend app

somewhere and connect DialogFlow to our app to make requests and get responses. Our back-end app is Node JS app.

We will host it on Heroku https://www.heroku.com/. Heroku is a cloud-based platform and most Heroku deployments are performed with Git (also Docker-based

developments). Let's see how to connect Dialogflow to our local server for development then Heroku option for deployment.

Figure 29

Either download the project from the Git Hub or Clone it.

Local Server

Figure 30

Run npm run dev to run the project. You see nodemon started. nodemon is a tool that helps develop node.js based applications by automatically restarting

the node application when file changes in the directory are detected. The app runs on port 5000.

We run our server locally and Dialogflow somehow to reach our server. As mentioned, we do not want to always deploy the code on Heroku to test it out.

As a solution we will use ngrok https://ngrok.com/ which provides public URL for exposing your local server. It will create a subdomain on ngrok domain and will redirect

all the traffic on that subdomain to your local server.

Figure 31

Code is running on our local server which is the same computer ngrok is running. We have ngrok.io that creates subdomain for us. We use that subdomain to set it as a webhook

address for DialogFlow. When an intent with fulfillment enabled will be triggered the Dialogflow will make a request to that subdomain and ngrok will redirect that traffic to

our local server. Local server will do what it is supposed to do and return the response. Ngrok will take the response and just forward it to subdomain and from there to

Dialogflow.

Figure 32

Go to https://ngrok.com/ and LOGIN or SIGN UP.

Figure 33

Download ngrok executable for your platform unzip and run it.

Figure 34

Connect your account with auth token.

Figure 35

I ran the command and auth token was saved to configuration file.

Figure 36

It remains to do the last step; connect our app.

Figure 37

You remember our app was running on port 5000 (Figure 30). We specify http and port our app is running for fulfillment to go to backend app.

Figure 38

You see ngrok is forwarding from https://e0 … to localhost 5000.

Figure 39

If you go to http://localhost:5000 you will see GET request.

Figure 40

I specified ngrok address https://e0b93d24.ngrok.io and it is working!

Figure 41

We go to Dialogflow select Fulfillment tab, enable Webhook and specify ngrok address with https.

Figure 42

Do not forget to save it.

Figure 43

Now you see bot's response to Get Started via fulfilment. The response is coming from our node.js back end app via ngrok hosted locally.

Figure 44

You may notice that there is Web Interface address. Let's go to that page.

Figure 45

Here you track requests coming to this tunnel (you can read more about the request/response structure on description

page https://lightning-tools.github.io/Social-Squared-Bot/description/).

Each time you run ngrok command, the new address will be created and each time you have to go to Dialogflow (Figure 41) to replace

the address for fulfillment with the new address. You can get the payed ngrok version to set up a permanent address.

Heroku

We host our backend code to Heroku when ready.

You should go to https://signup.heroku.com/ to sign up and create a free account.

Next, go to https://devcenter.heroku.com/articles/heroku-cli and download and install Heroku Command Line Interface (CLI) which makes it

easy to create and manage your Heroku apps directly from the terminal.

Figure 46

It is time to create an app in Heroku. I already created socialsquared app.

Figure 47

We already mentioned that Heroku is a cloud-based platform and most Heroku deployments are performed with Git. Here you can see the commands

to deploy changed to Heroku using Git.

Our backend code also sends emails to customer support. Emails on feedbacks, feature requests, bugs etc. For that we use SendGrid https://sendgrid.com/

which is a cloud-based SMTP provider that allows you to send email without having to maintain email servers. SendGrid code used for our local server and Heroku one

are different.

Figure 48

sendgrid package is for Heroku and @sendgrid/mail is for local development.

You should enable SendGrid on Heroku to send emails https://devcenter.heroku.com/articles/sendgrid

Figure 49

I chose Starter (Free) option that allows to send up to 12000 emails per month which is more than enough for our app.

Our app not only sends emails but also keeps records in database. Based on database records we can generate some statistic or built an admin pages to track data.

Figure 50

The app uses mlab https://mlab.com/ which is a database-as-a-Service for MongoDB.

Figure 51

Currently, we use Sandbox plan which provides a single database with 0.5 GB of storage.

Figure 52

We signed up, selected Sandbox plan and created social squared database.

Figure 53

Here is Bugs collection and issues sent by clients.

We use some variables in our app, and publishing it to GIT is a mistake, security issue. The best thing to do is to define all the variables as Config Vars on Heroku.

Figure 54

In config folder we created keys.js file. If code runs on Heroku then process.env.NODE_ENV is production and variables are read from prod.js otherwise (local development)

variables are read from dev.js

Figure 55

Let's explore dev.js used in local development. Key is the SendGrid key. SendGrid recommends API keys because they are a secure way to talk to the SendGrid API

that is separate from your username and password. Here is the key generated by us. The second one in mongoURI to connect to mlab. The path includes dbuser

and dbpassword. We have emailFrom and EmailTo which are required to send emails. The last line is email sending code email_local used in local environment for development.

dev.js must not be published and must be defined in .gitignore file.

Figure 56

prod.js is deployed to Heroku and variables are read from Heroku Config Vars.

Figure 57

I go to our app's console in Heroku, select Settings tab and click Reveal Config Vars button.

Figure 58

You see Config Vars specified in Heroku which are obtained by the app via prod.js (Figure 56).

Figure 59

You will find the project URL under Settings tab.

Figure 60

I navigated to that URL and could see the page.

Figure 61

You can also view your GIT commits and view logs too on your Heroku page.

Figure 62

You remember that our webhook URL was pointing to ngrok for the local development.

Figure 63

I change it to Heroku URL, and save it. Now, webhook points to Heroku.

Figure 64

It remains to make our bot public, available for everyone. Your Facebook page can be publicly available, but the only person that the bot will currently

corresponds to is you, because you are the admin. In order to make this bot correspond to everyone you have to get your Facebook App approved.

Approval may take a few days.

Figure 65

I go to Messenger -> Settings then click Add to Submission button for pages_messaging option.

Figure 66

Scroll down the page and you will see that it has been added to current submission list. Click on Add Details link.

Figure 67

This will open a dialog. First select you page.

Figure 68

Now, you should tell Facebook team what is you bot about, what experience the bot offers. You need to describe the example conversation. Click Save.

Figure 69.

It remains to add App Icon, specify Private Policy URL, Select a category etc. Once you specified all the info Submit For Review button will turn to active and

you can submit it for Review.

Figure 70

As mentioned, Dialogflow can be integrated with Skype https://dialogflow.com/docs/integrations/skype and other platforms.

Figure 71

I will not go into detail. It is quite simple. Just follow the instructions.

Figure 72

Here is Social Squared bot. You see that both Skype and Microsoft Teams are running.

Figure 73

Here is I am talking to Social Squared Chatbot via Skype.

Figure 74

Talking to Social Squared Chatbot via Microsoft Teams.