How to create your own bot from scratch

Apr 10, 201818 min read

Deviation Actions

Published: Apr 10, 2018

7.1K Views

Greetings! – v1.2 (added some more info)

Someone inspired by dADroid-bot recently asked me to help them in the right direction of creating their own bot, and I wanted to write a complete guide on how a bot is made. Since this seemed to be something that other people might want to read in the future, I decided to write and share a comprehensive guide, listing all the required tools and skills to make a new bot from scratch without any past experience.

Now, I should mention that there are in fact third-party tools and services that can be used to create and/or replace any part of this guide, but considering that those services do or use more or less the same things explained here, I'd recommend to be at least familiar with the concepts explained here, and the basics of what is happening behind the scenes to run a bot.

I'd like to point out a few essential tools & skillsets required to start a new project like this, and I hope you'll make an amazing bot here!

With that said, let's get started!

Introduction – What is a bot?

a robot is simply a piece of software designed to work continuously behind the scenes to do what real humans do, automatically. Some bots are unfortunately made with malicious intent, but most bots are created to help people easily & quickly do what they'd otherwise regularly do by hand.

On DeviantArt, this generally means reading & writing comments, posts, journals and interacting with people.
_{(For more information about Good vs. Bad bots, have a look at here.)}

⚠ There's also another category of completely different Deviant Art bots which happen to interact with the Deviant Art Messaging Network, which BotLab has had covered very well. Those bots are specific to chat system this site runs. This category is out of the scope of this tutorial.

Step 1 – Programming language

The first step to create a bot (or a software of any kind) is to be able to code.
Do you have any past experience with coding? If so, you might be able to focus writing your bot with that. If not, no worries, you'll need to start learning a new Programming Language. Which programming language to choose is not that important as long as it's capable of connecting to DeviantArt.

You'll just need to understand the basics of computer logic and should probably be comfortable to use the language on a regular basis. As an example for the programming language, dADroid-bot is coded in PHP, a standard web-development language. Other bots, such as dAbot (created by my friend Kishan-Bagaria) use Python, and some like dAhub I believe use Java (not to be confused with JavaScript).

I'll try to give a short description for each language, and I hope you find the right amd the most suitable one to use for you.

(Remember: memorizing it is not required, instead you should learn the logic behind it.)

Ruby: a beautiful, simple to understand and & easy to use but powerful tool to start programming.
Try it here: tryruby.org
Python: a very popular scripting language, has a great, powerful and moderately easy to master syntax.
Try it at: learnpython.org
Lua: a very flexible and easy scripting language. Here's an excellent YouTube video: www.youtube.com/watch?v=S4eNl1…

Once you've familiarized yourself with the concept, you can try some more advanced/complex languages in the future:

JavaScript (not to be confused with Java): the defacto standard scripting language. it's powerful yet easy, and originated from designing web pages and currently focused for modern web-apps.
Try it at: www.javascript.com/try
(I highly recommend learning this in the long run!)
PHP: the most-used web programming language, which only runs in the back-end (like servers, though it's possible to run on Desktop computers and your laptop too). It's powerful but some consider it to be ancient, and most modern developers try to avoid it. A very dangerous tool if handled uncarefully. A powerful, sharp tool if handled with care.
dADroid is written mostly in PHP.
Java (not JavaScript): a powerful, moderately easy to understand and object-oriented language, which is somehow similar to JavaScript, yet completely different. Developed by Oracle Corporation and it's in wide use by many businesses. You don't need to bother yourself with it, at least for now. I believe dAhub was written in Java.
C#: developed by Microsoft, originally based on C and C++ syntax. Much better than ancient and unmaintained counterpart VB (visual basic) language, which was also designed by Microsoft. Until recently, it could only run on Windows operating systems (which meant not being able to run on Linux servers), though a minimalistic version for Linux has been recently released.
It has a somehow similar syntax to JScript (Microsoft's fork of JavaScript.)
Used widely in business environments, not exactly the most suitable for DIY projects like this, but has great potential, and as such is widespread, and hence has great support on websites like StackOverflow.com.
Again, you do not bother yourself with it, just now. Might be a handy tool if you've picked up programming languages in academic classes (like college and university.)

Step 2 – API (connecting to DeviantArt)

Now that you've worked with some languages and picked up a general syntax (and hopefully chosen one of them to use), it's time to write an interface for your bot.

Using an API (application programming interface) your bot can talk to DeviantArt servers, and vice versa. If you imagine two pieces of code (one is your bot and the other one is DeviantArt server) are commuting with each other, the method they send and receive data is called an API. Kinda like what texting is for humans, if you will.

Wikipedia has a great (and in-depth) article on APIs, if you want to read more. (Not bad reading material!)

DeviantArt uses two distinct set of APIs:

a public one which DeviantArt provides here:
www.deviantart.com/developers/
an internal one called DiFi (DeviantArt Interactive Fragmented Interface) which they use themselves for calls between their site and their server.

some kind souls have documented DiFi here:
github.com/danopia/deviantart-…

a bot can use either (or even both) of these APIs to interface their bot with DeviantArt.

The public APIs are somehow limited, and you're required to register a key for your application to work with. However, it's free, documented regularly, and super-easy to use, and less prone to suddenly breaking.

The DiFi API is much more powerful, doesn't require signing up for a key, and is more popular with the DeviantArt modding community. However, DeviantArt site owners meant it to be only used for their own internal stuff, so it's a bit less reliable, can change at any given time without prior warning, and thus could potentially break your program. However, since DeviantArt hasn't really changed for several years, it's rare and unlikely that it completely remove this API.
(That's not to say instances of a bot completely breaking down in the past haven't occurred at all (like dAhub), but usually, developers are able to fix them and bring them back online completely, usually at a short time.)

Another concept which you should be familiar with is HTML scraping, which is the practice of fetching and loading the HTML code that is meant for the browsers from the DeviantArt servers into the bot and parsing it.

Professionals most often recommend using the official and public APIs, and you don't hear one recommending you to use the internal APIs. Against their warnings, however, dADroid, dAbot, (and dAhub to my knowledge) all use the DiFi internal API with the addition of some HTML scraping, because it's more powerful and portable.

Basically, anything you can do with the website can be done using DiFi, but they might try to block you (or your bot) – especially if your bot is made with malicious intent.

dADroid has a great HTML scraping and DiFi parsing utility built right into it.

The unofficial route gives you much more power but is harder to maintain. You may, however, choose to use the official way, which is a bit less powerful but easier to work with.

So what to use?

If you choose public APIs, everything is documented, but you need to register a key, and fewer functions are available to you.

If you prefer to have more functions available to use at the cost of unofficial support, you can go the DiFi way. If so, you can either write the code for DiFi yourself, use the already made samples, or politely ask someone to write it for you.

Kishan-Bagaria had even put his Python code on GitHub: github.com/KishanBagaria/dAbot
danopia has put an old (untested) Ruby code on GitHub: github.com/danopia/deviantart-…

You can use these as samples to write your own. The one for dAhub is closed-source, meaning that you can not obtain the source code from him.

dADroid-bot (the one that I've created) is open-source, and actively maintained, but at the time of creating it I didn't have any intentions of releasing its source code to the public, so it's a bit messy.

Nonetheless, if you'd like to use PHP, you can use dADroid's source in your project. You can see a list of version history here: dadroid.ir/Changelog.txt

Step 3 – means of storage

So, at this point you've created a bot which can connect to DeviantArt, read its content, and post some results onto it.

Now, as the APIs are eyes and ears of your bot, you need a brain to process and store the content your bot receives.

What use would a bot have if it forgets everything whenever you close it?

You can use two primary methods of storing the content of your bot for later retrieval and use: a) as files on your disk and b) using a Database software.

For file-based storage, have a look at XML, YAML , and JSON. Don't get scared by lines of text! They're merely written to help your bot remember what's going on after each startup.

However, modern and complex bots can't rely on text files to accurately and efficiently store their data. So you need to use another tool to handle your data.

Nowadays, the SQL (read SEQUEL) which stand for Structured Query Language is the standard way of writing and accessing data in Databases.

The most popular SQL servers (which you can use in your project) are MySQL and MariaDB at the moment. They are sister databases and you can use them interchangeably. The first one is currently developed by Oracle, the same company who maintains Java (not to be confused by JavaScript).

If you've ever heard of WordPress, it uses one of these Databases to hold its data, and I'm pretty sure that DeviantArt also uses something similar.

There's a lighter version of SQL which doesn't need a server and resides on disk, called SQLite. You can use it for your starting project. Fun Fact: Google uses SQLite in their Android OS all the time. So if you currently have a phone with Android as its OS, there's a pretty good chance that your phone uses SQLite!

Microsoft also uses their own propriety and non-free fork of SQL server, which is focused on businesses and you don't have to worry about.

All of these SQL variants (with the exception of Microsoft's) are free and open-source, you can download the software right away and use them in your projects.

Here are some great resources to learn SQL:

Step 4 – logic, user settings & GUI

Now that your bot can connect to DeviantArt, get data, write to it, and remember what to do; you need to write the logic for what it actually needs to do.

The best way to do this is to write down your requirements for a bot, organize it into smaller block diagrams, and write the code for each block of your diagram. It's easier to follow (and debug) your bot's logic that way as it gets bigger, especially if you comment your work.

You also need to devise a means of controlling and "configuring" your bot. You can write an external web-page for its settings, or use DeviantArt comments. It's up to you! Once the command is received, the result will be saved in the Database.

At one point, you'll also probably need to implement timers in your logic, which on Linux servers are achieved by using cron jobs. Best idea to familiarize yourself with the concept, but no need to worry about it, for now.

When coding, it's best to always follow DRY practice: Don't Repeat Yourself. This means avoid copy-pasting code, which trust me, makes everything a hundred times harder. Instead, split your code into different meaningful and re-usable parts.

You can read more about on how to follow this pattern in a carefully designed fashion called MVC:

I'm not going to get deep into this, as there are already a lot of good articles written. Just remember, keep everything organized, and keep your code clean and well-commented, even if you're the only one working on the project, you wouldn't know if someone else would work on it tomorrow. Otherwise, you'll get into some unpredicted issues soon, like I have with the dADroid's code getting so messy that I need a full rewrite.

By the way, if you're going to version your bot (and you should), it's best to follow the Semantic Versioning practices. Remember: You probably won't be the only one working on the code, forever.

Step 5 – running the bot on a server

The last step is to run your bot somewhere which can it keep running in the background, as long as you wish.

For this task, generally, a server is used to keep your bot running. You can host your own server (which definitely isn't recommended, but I personally do anyways), or you can go the easier route, and buy a hosting plan (renting a server dedicated to your project.)

Generally, VPS hosting services are really cheap, and can run your code in the cloud. I recommend using them. Or, you can also choose to run your bot from your own personal computer, which means you need to keep your PC always on to keep your bot continuously running. If your computer (or a server for that matter) is turned off, it equals your bot not running.

You should choose your server plan very carefully according to your needs, especially if you intend to keep the bot running for a long time.

If you develop your bot in PHP (like I did) you can buy even cheaper plans meant for website hosting services, which could also host your bot program.

Final Notes – social side of the bot

Please try to keep your bot polite, and avoid spamming people. That's a very bad thing to do!

Don't do things that DeviantArt staff wouldn't like, they'd block you for no sensible reason, faster than what you'd think. I've learned this the hard way.

And it has become a standard convention on DeviantArt to mark your bot's generated messages with "I'm a bot" or similar notes. It's especially important if your bot generates replies to somehow identify it as a bot. (Like any instance of both dAhub and dADroid-bot. All of the popular bots on DeviantArt do this, to my knowledge.)

Conclusion

And there you go, a list of all essential points I could think of for a working and practical Deviant Art bot! I hope you learn new things along the way and have a great experience writing your own bot.

Be sure to check DeviousBots for some examples of useful bots around DeviantArt, and submit your own bot there if you made a great one!

You are encouraged to comment any questions you have or any problems that might occur in the future. I'll try my best to answer them when I can!

I tried to list everything that came to my mind, but I might update this journal in the future if something else also needs to be mentioned.

Have a great day.

Comments27

Join the community to add your comment. Already a deviant? Log In

YuriTheFoxprincess

Jul 9, 2019

Is there a Bot that can Send a Note or reply To a Comment with a Spacific message Telling People that i may not respond due to the fact maybe is asleep?