Seminar Copenhagen 2019

 

 

 

 

 

 

 

Telegram

 

 

Telegram is a social networking app to communicate between people and groups. The app is available on the 2 main platform: Android and iOS. This application is very similar to Viber, WhatsApp and other chat apps.

 

There are plenty differences between Telegram and the other apps, such as security, upload and download speed, stickers, any file can be sent, on many devices, etc... but the main difference is that it is possible to make powerful bots to help us solve problems. These bots are mostly used in groups but can also be of personal use. There are by simplification AIs.

 

So, what is exactly a Telegram bot? A bot is like a robot that can automatically or manually do some things. Let’s imagine that you want to create a small game and you want the people in a group to share their scores and have a ranking between the people of your group, a bot can be used. Another use could be for instance that you want to know what is near you to do and you want the people of the group to know what you’re searching for and maybe where you’re going, well this can be another use for the bot.

 

Quite common bots are bots that help do polls. These bots answer the following questions, which usually are source of arguments within the group: What are we going to do as a group? Where should we go?

 

During this workshop, we are going to show you how to make 2 telegram bots which can be quite useful:

1.       A translation bot that translates all messages to English

2.       A bot searching what is near you depending on what you are looking for

 

(Examples)

 

 

 

 

 

Prepare the environment

 

There are several ways to create Telegram bots. Server side languages such as PHP or script languages like Python can be used. This is very nice because it allows a friendliness of use and of coding, as well as expandability.

In order to make it easy for as many people as possible, we are going to make these bots in Python. This script language is very easy and no prior knowledge is required.

 

 

 

 

Since not all computers have Python installed on their OS, we need to install it. Unix systems should already have Python 2.7. So that all computers have exactly the same Python version, we suggest installing the latest Python which is Python 3.7.

 

If, for some reason, the python installation or any other libraries’ installation doesn’t work you can skip the section Run your Code and try again the installations. It should work as it is on a virtual Linux environment.

 

MacOS

If you have home-brew on your computer, open a terminal and enter the command:

 

Else you can download from the following link the pkg file:

https://www.python.org/downloads/mac-osx/

 

Linux

 

For Linux Fedora, open a terminal a type the command:

For Linux Ubuntu, the command is the following:

 

Windows

 To download and install Python visit the official website of Python https://www.python.org/downloads/ and choose your version.

 

 

Once the download is complete, run the exe to install Python and click on Install Now.

 

 

 

Don’t forget to check Add Python to your PATH

 

You can see the installation is running:

 

 

When it finishes, you will see a screen that says the Setup was successful. Now click on "Close".

 

 

 

If you can’t launch any python command, you will need to add it to the Windows Environment Variables and to do that open up Control Panel > System and Security > System and select Advanced system settings from the left-hand side, a pop-up will appear like this:

 

 

Next select Environment Variables and under System variables locate the Path variable, then double click on it.

 

 

A new window will open, select New and type the of your python installation folder.

 

Finally click OK on all the opened windows.

 

Because Python is a script language, you don’t really need an IDE. Writing your script on Notepad++, is a way of coding your script.

 

But we love fancy things such as a colorful ide, integrated unit-testing or coding assistance, that’s why we also provide you the installation of PyCharm community, which is the python-specific IDE by the Czech company JetBrains: https://www.jetbrains.com/pycharm/download/ .

 

Just click on the download button on the community tab

 

 

When your download is finish, launch the installer.

 

Click on next, then choose your installation folder

 

On the next screen, you configure as you want.

And then your star menu folder. You can leave JetBrains.

 

You might need to reboot your computer.

 

Now, you can configure different settings of pycharm.

If you want to install different plugin, you are free. We decline any responsibility, if you install Vim.

 

 

A nice feature about PyCharm, is that when you create a new project, you can also create a virtual environment inside your project.

 

What is a virtual environment?

Basically, a virtual environment is a system to keep dependencies required by separated projects, by creating isolated python virtual environments for them.

 

Why a virtual Environment?

 

Python has its own way of downloading, storing and resolving modules.

The different location where the modules can be installed by third party package installers can cause many problems.

 

It’s important to know this because, by default, every project on your system will use these same directories to store and retrieve site packages (third party libraries). At first glance, this may not seem like a big deal, and it isn’t really, for system packages (packages that are part of the standard Python library), but it does matter for site packages.

 

Consider the following scenario where you have two projects: ProjectA and ProjectB, both of which have a dependency on the same library, ProjectC. The problem becomes apparent when we start requiring different versions of ProjectC. Maybe ProjectA needs v1.0.0, while ProjectB requires the newer v2.0.0, for example.

 

This is a real problem for Python since it can’t differentiate between versions in the site-packages directory. So, both v1.0.0 and v2.0.0 would reside in the same directory with the same name:

 

/System/Library/Frameworks/Python.framework/Versions/3.5/Extras/lib/python/ProjectC

 

Since projects are stored according to their name, there is no differentiation between versions. Thus, both projects, ProjectA and ProjectB, would be required to use the same version, which is not wanted in many cases.

This is where virtual environments and the virtualenv/venv tools come in handy.

 

 

 

Now that Python has successfully been installed on your computer, we can now install the libraries required to access the Telegram API. You will see, it is extremely easy.

 

We will add 2 libraries that we will use in the 1^st tutorial. The second tutorial will show you how to use environments and install libraries in it.  

These libraries are pyTelegramBotAPI and py-translator

If you want to look at the source code of these APIs you can find them here:

 

pyTelegramBotAPI: https://github.com/eternnoir/pyTelegramBotAPI

 

To install them, open a terminal or command prompt and enter the commands below:

 

•    For pyTelegramBotAPI, enter the following command:

 

•    The second library to install is py-translator

 

There, all the required libraries have been successfully installed!

Now, we want to create, on Telegram, a new bot associated to your account.

 

 

Since the bots that you are going to create are linked to your account, the first step is to connect on Telegram with your account. It doesn’t matter if you connect on your phone, tablet, computer or browser.

 

Usually, to call a bot, we use the symbol “/” followed by a word and the bot will know that it is being called. It is possible to have the bot running in the background and check all the messages.

The second step is to search, in the Chat tab, for BotFather. (The bot responsible for all bots)

 

 

 

Open the discussion with BotFather and type the command:

You will see all the commands that you can enter using BotFather.

 

Examples of commands are:

 

/newBot                             Create a new bot

/myBots                              See your bots

/setName                           Change or set the name of your bot

/setDescription Change or set the description of your bot

/token                                 Retrieve the token necessary to ID your bot

...

 

And so on.

 

Let’s create a new bot by entering the command:

Follow the steps required to finish creating the bot. It will ask for the name of the bot, the displayed name / surname.  You will get the bot information once you’re done with these steps.

 

 

Make sure to keep this message as you need the message token to access your bot through your code.

 

 

First of all, we need to understand how messages are sent to telegram. When we send something to our bot, it is sent as a JSON object

 

For exemple:

 

If you go on the getUpdates() function from the bot api, you will be able to see the objet that have been sent with all the metadata.

To see your bot objects, enter this URL in a browser. Firebox displays JSON objects the best:

 https://api.telegram.org/bot<YOURTOKEN> /getUpdates

 

 

For the bot to be run continuously, we have decided to work with a python script that we will run somewhere over the rainbow, on the website https://www.pythonanywhere.com.

 

But first, we need to create the python file. You can create a virtual environment inside your
project
folder, by running this command line (left for Linux/OSX, right for windows):

And then activate your venv with your command shell

If you want to exit your virtual environment just put “deactivate “

 

 

Let’s start coding! First things first, we need to import the libraries installed previously. Here is the code:

 

import telebot

from py_translator import Translator

 

then we need to instantiate our bot with your token (we saved the message), carefully inside the bot:

 

#initialization

bot_token = '<YOURTOKEN>'

bot = telebot.TeleBot(token=bot_token)

 

 

Now, we can access the bot and do what we want.

 

The syntax to get a command is really simple and intuitive. We will begin with an answer to the command /start. That specific command is used to start the bot inside a conversation.

 

To handle the command, we use

 

@bot.message_handler(commands=['start'])

 

Then we declare the function and get the message:

 

def send_welcome(message):

 

And we implement the logic to answer the message:

 

bot.reply_to(message, 'Welcome :)')

 

So, let’s try it with a more complicated one:

 

This function will scan every message receive and try to find an “@” inside a message text.

 

# reacts as a function if something happens in the condition

@bot.message_handler(func=lambda msg: msg.text is not None and '@' in msg.text)

def at_answer(message):

    texts = message.text.split()

    at_text = find_at(texts)

    bot.reply_to(message, 'https://instagram.com/{}'.format(at_text[1:]))

 

#go through the list to see which work has an @

def find_at(msg):

    for text in msg:

        if '@' in text:

            return text

 

As you can see, it’s pretty simple to code some answers for the bot.

 

A good practice for bots, is to have a help command, that will give all command with a short explanation

 

@bot.message_handler(commands=['help'])

def send_welcome(message):

    bot.reply_to(message, "Type a @ followed by a name to search intagram")

 

Finally, to make your bot available and online you will need this line:

 

#start the bot

bot.polling()

Because the second decade of the 21^st century is all about API, we have decided to make our bot connected to API.

 

We had many experiences with some people that had trouble with English. So, we have tried to help them with a translator to English.

 

After installing (previously installed) the API inside your environment, import the following:

 

from py_translator import Translator

 

Try by yourself with the documentation of the api:

 

https://pypi.org/project/py-translator/

 

 

Solution :

@bot.message_handler(commands=['trans']

def traduction(message):

    to_language = 'EN'

    translation = Translator().translate(message.text, to_language).text

    bot.reply_to(message, translation[8:])

 

 

 

 

Bonus: we also tried to implement an innovative way of communicating with young people. So, it will be a command to translate your message into gangsta.

 

So the implementation need an another file that we have call gizzogle:

 

import re

import bs4

import requests

 

 

def text(input_text):

    params = {"translatetext": input_text}

    target_url = "http://www.gizoogle.net/textilizer.php"

    resp = requests.post(target_url, data=params)

    # the html returned is in poor form normally.

    soup_input = re.sub(

        "/name=translatetext[^>]*>/", 'name="translatetext" >', resp.text)

    soup = bs4.BeautifulSoup(soup_input, "lxml")

    giz = soup.find_all(text=True)

    giz_text = giz[39].strip("\r\n")  # Hacky, but consistent.

    return giz_text

 

Now it’s your turn to make it work!

 

Solution:

#Convert text into gangsta text

@bot.message_handler(commands=['fun'])

def translate_yoda(message):

    modified = make_reply(message.text)

    # get the message without the command

    reply = modified[4:]

    bot.reply_to(message, reply)

 

#Creates the reply with the new text

def make_reply(msg):

    reply = None

    if msg is not None:

        reply = gizoogle.text(msg)

    return reply

 

This bot will be asking the user to input a type of “point of interest” to search. Once the user has entered the kind of point of interest, the bot will ask the user to send his current location. It will then return a list of locations that match the searching criteria near the user, thanks to the TomTom API.

 

 

The first thing to do is register a free developer account on TomTom’s website : https://developer.tomtom.com/

While we wait for our confirmation mail, let’s start with the code.

 

Create a new repository and start a new virtual environment :

mkdir searchbot

cd searchbot

pipenv install

 

This will create a pipfile and a pipfile.lock to manage our dependencies

 

Add the pyTelegramBotApi dependency :

pipenv install pyTelegramBotApi

 

Create a new file called bot.py

This file will contain all the code of our bot.

 

Inside bot.py, start with the import statements :

import telebot

import urllib.request

import json

 

The first and most important import here is telebot from the pyTelegramBotApi module. This will allow us to handle messages between the user and the bot.

urllib.request and json are used to communicate with the TomTom API later on.

 

At this point you should have gotten your confirmation email from TomTom. Let’s activate your account and add a new app to your account.

 

Give it a name and check the “Search API” option.

Expand your newly created app, and you should see your unique API key. Don’t ever share it publicly.

 

We’ll also need a new bot token from the BotFather. If you haven’t already added him, add @botfather on Telegram.

 

Ask him nicely to create a new bot, give him a name and a username.

 

Now that we have setup this, let’s go back to our code.

 

The next thing we have to do is define our bot token that we got from the BotFather as well as the API key we got from the TomTom developer page.

bot_token = 'YOUR_BOT_TOKEN'

tomtom_api_key = 'YOUR_TOMTOM_API_KEY'

 

Then, we instantiate our bot

bot = telebot.TeleBot(token=bot_token)

 

That’s all that is needed to start the bot. Now we have just have to handle the different commands the user can send. We’ll start with an easy one, the “/start” command that is used to start interacting with every bot.

 

@bot.message_handler(commands=['start'])

def send_welcome(message):

    bot.reply_to(message, ['Welcome !'])

 

Nothing fancy here. We filter for any messages that are a commands (starting with a slash “/”) containing the “start” directive. The we define the function that is executed when a message passes this filter. In this case, we simply return a “Welcome !” message.

The method contains a message parameter that contains the reference to the message that triggered the function.

The next line calls the “reply” method of the bot. This method needs a message argument to which it sends a reply, contained in the second argument of the method.

 

At this point we have instantiated our bot, defined a message handler for the “/start” command, but we still need to start the bot so that it can handle incoming messages.

At the end of the file, add the line :

bot.polling()

 

You can now start your bot and try it out. In your console :

python bot.py

 

On telegram, find your bot with it’s username : @it_seminar_2019_yourname_bot

And click on the Start button

 

 

Your bot should answer you with your own crafted message.

 

 

Stop your bot by pressing CTRL+C in your console.

 

Let’s attack our search function !

The way our bot will handle the search function requires us to store the state of each user interacting with our bot. Let’s start with initiating two state array for the current step and search interest of the users, as well as helper methods to retrieve those steps and search interests.

 

Write this just between the token definitions and the first message handler.

 

userStep = {}

userSearchInterest = {}

 

 

def get_user_step(uid):

    if uid in userStep:

        return userStep[uid]

 

 

def get_user_interest(uid):

    if uid in userSearchInterest:

        return userSearchInterest[uid]

 

Then we write the first function that starts the search. We will define a message handler for the “/search” command.

 

@bot.message_handler(commands=['search'])

def search_ask_poi(message):

    cid = message.chat.id

    bot.reply_to(message, text='please enter your search interest')

    userStep[cid] = 1

 

The first line of the function will retrieve the unique id of the chat from where the message originates. Then we send a reply asking for a search interest. We also set the current step for this chat to 1. Why we need this will be clear in a few seconds.

 

Now we write a second message handler for the search interest answer.

 

@bot.message_handler(func=lambda message: get_user_step(message.chat.id) == 1)

def search_ask_location(message):

    cid = message.chat.id

    bot.reply_to(message, text="please send your current location")

    userStep[cid] = 2

    userSearchInterest[cid] = message.text

 

As you can see, we filter any message from chats that have “1” as their current step, which means they have initiated the search but not given a search interest yet.

Again, we retrieve the chat id and reply by asking for a location in which to search. We advance the step for this chat to 2, and store the search interest for this chat.

 

Finally, we define a message handler for the location answer.

 

@bot.message_handler(func=lambda message: get_user_step(message.chat.id) == 2, content_types=['location'])

def search_poi(message):

    cid = message.chat.id

    long = message.location.longitude

    lat = message.location.latitude

 

    poi = userSearchInterest[cid]

 

    query = "https://api.tomtom.com/search/2/poiSearch/{0}.json?typeahead=true&lat={1}&lon={2}&radius=2000&key={3}".format(poi, lat, long, tomtom_api_key)

 

    with urllib.request.urlopen(query) as url:

        data = json.loads(url.read().decode())

 

        if data['summary']['numResults'] == 0:

            bot.reply_to(message, text='Sorry, no results')

            return

 

        text = ""

        for result in data['results']:

            name = result['poi']['name']

 

            distance = int(result['dist'])

 

            city = result['address']['municipalitySubdivision']

            postalcode = result['address']['postalCode']

            streetname = result['address']['streetName']

            streetnumber = ""

            if 'streetNumber' in result['address']:

                streetnumber = result['address']['streetNumber']

            country = result['address']['country']

            address = "{}, {} {}, {} {}, {}".format(name, streetnumber, streetname, postalcode, city, country)

 

            googlequery = "https://maps.google.com/?q={}".format(address)

 

            text += "<a href=\"{}\">{} - {}m</a>\n".format(googlequery, name, distance)

 

        bot.reply_to(message, text=text.rstrip(), parse_mode="HTML", disable_web_page_preview=True)

 

This one is a little longer, but the important part is the message handler. We filter for message that are of type “location” and where the chat’s step is set to 2, meaning the have initiated the search, given a search interest but no location yet.

 

We then get the latitude and longitude of the location, and call the TomTom API with the search interest and location from the chat. You can check the use of the API here : https://developer.tomtom.com/content/search-api-explorer#/Search/get_search__versionNumber__poiSearch__query___ext_

 

The rest of the code handles the JSON response from the API, extracts the relevant information and formats it into a list of clickable links before sending it back to the chat.

 

Restart your bot and try it out !

 

You may notice that the API is pretty limited and can send you some outdated data. That is because TomTom is firstly designed as a navigation service, and while their Point of Interest service works well, it definitely is not a priority for their business. The Google Maps API could probably yield better results, but sadly they require a form of payment method even if they offer up to 200$ worth of use of their service for free each month. If you are lucky enough to have access to such an API key, feel free implement it into your code.

 

 

 

 

Run the code

 

Now that the bot works well, we want to have it continuously running. Of course, we don’t want to have to leave our computer running all day long. Normally, we would set up a server running from home, or we could buy a virtual server on one of the many companies offering this kind of job.

In our case, since we are geeks are we don’t want to pay anything, we will use a free virtual Linux environment.

 

PythonAnyhwere allows us to do just that. It is also great as we can code python directly there on the cloud. So, let’s create an account.

 

 

You will have the following dashboard.

 

 

On the left, the consoles represent the terminals running. The Files part, are all the files that you have created. Here we can see that I have 1 console running called Firstbot console, and 2 files called bots.py and gizoogle.py.

 

Let’s create the files. We are going to run the first bot tutorial. For that create a first file by pressing on the +Open another file button.

 

Enter the path with the new file name like the following:

 

 

You can copy paste the code of the main bot in there and save it using the top right Save button.

 

Now, we’re going to create the 2^nd needed file. Press on your username:

 

Enter the new file name with the extension .py and click on “New file”. Repeat the same process as with the previous file and copy the right code.

 

Now that all the files are there, we need to install the missing libraries to the virtual environment.

 

Go back to the main dashboard by clicking on the logo

 

 

and create a new bash console. Be careful, we don’t want to create a python console!!

 

 

We’re going to install the missing libraries exactly the same way as we did on our computer. There will be a small change however. Because we aren’t admin on these virtual
environments, we need to specify that we want to install on our current user.

 

Once the installations are done, you can go to your files via the terminal and launch the python script.

 

 

If there are no errors, you’re good to go and you can close the browser should you wish to. Your bot should still be running! J