From zero to hero: Creating a chatbot with Rasa NLU and Rasa Core

AI assistants are a hot topic these days. Chances are that you have already had an encounter with at least one of them, as a user or as a developer. In this post, I would like to talk about a stack of software called Rasa, which you should definitely include in your toolbox if you would like to build conversational assistants yourself.

In short, Rasa NLU and Rasa Core are two open source Python libraries for development of conversational AI. They are packed with Machine Learning and handle natural language understanding and dialogue management tasks. Most importantly, Rasa stack is easy to use, you don’t need massive amounts of training data to get started and it is perfectly suited for production.

I have been building chatbots with Rasa stack for almost a year now and it is safe to say, that it has been a tool that I have been the most excited about throughout that time. And here is why:

  • It is open source. You own your data and you can hack things.
  •  It is developer friendly. You don’t even need to know Python to use it.
  •  It has an awesome community and highly involved developers. If you have any issues, just post a message on Gitter and someone from Rasa team or other developers will help you out.
  •  It is a great example of how applied research can be shipped to practice and empower thousands of developers around the world.

So… With all that in mind, I decided to make a tutorial on how to create a chatbot using Rasa stack completely from scratch. It is going to be an exhaustive tutorial, with a deep dive into Python (if you don’t code in Python, don’t get discouraged – check the Rasa documentation of how you can do it all without any Python whatsoever). I am going to build a simple Slack integrated weather bot, called Frank. I highly encourage you to follow along so grab a cup of tea and let’s build some chatbots! 🙂

Update: The devs of Rasa NLU and Rasa Core are doing an amazing job updating and improving these libraries. Since I recorded this tutorial there were quite a few things introduced to Rasa NLU and Rasa Core which brought some changes in how some things should be coded. To keep this video consistent with the code, I updated requirements.txt file so that it would install Rasa NLU 0.11.4 and Rasa Core 0.8.2 – the versions which I used when I recorded the tutorial. If you want to use the latest releases of NLU and Core, you can find this directory which contains the tutorial code, compatible with the latest releases of these libraries (keep in mind, that the code will slightly differ from the one shown in the video).

Useful links:

Data files used in this tutorial
Full code of this tutorial
Full code of this tutorial [Latest release of Rasa NLU and Rasa Core]

Rasa NLU documentation
Rasa Core documentation
Rasa: Open Source Language Understanding and Dialogue Management (paper)
RasaHQ Gitter

54 thoughts on “From zero to hero: Creating a chatbot with Rasa NLU and Rasa Core”

  1. Thanks for sharing, I see you you said you have around 1 year of experience. I wonder if you have any examples of a bot you built online already? Would be great to see it. Thanks.

    1. Hey, thanks for checking it out! 🙂 The biggest chatbot I have built so far is for internal use of the company which I currently work for, so I can’t share it online, unfortunately. If you are interested in what it is about, you can check this interview where I spoke a little bit about it https://medium.com/rasa-blog/building-bots-with-rasa-shiba-slack-hosted-interface-for-business-analytics-ed47f346a16b . Apart from that, I am currently in a process of developing another chatbot which I think will be worth sharing online. I will post about it here once it’s done 🙂

  2. Hi Justina,

    I really appreciate the effort you have taken inorder to put this really nice example out. I am currently also working on a Rasa_NLU , but its nice to see how i can integrate with Rasa_core .

  3. Hi there thanks for all the guidance. However, i am getting this error “Unable to find vcvarsall.bat”
    Installed python 3.5.2. Im stuck can you please help?

    1. Hey 🙂 I am pretty sure it is your Visual Studio complaining. Can you share some more details on your current setup? What VS version do you have and can you share a full error log? If you want to, you can drop me a direct message with those details.

  4. hey nice tutorial
    I tried on different data but even after giving around 40 stories next_action is always action_listen, can you help me?

    1. Hey, swaps. Thanks for checking it out! 🙂 Would you mind sharing your domain and some of your stories and nlu data with me so I could reproduce it? You can shoot me a DM on Twitter, Linkedin or send me an email.

    1. Hey Alisa! Thanks for checking it out. I hope you will continue using Rasa and creating cool chatbots 🙂

  5. Hi Justina,

    Thank you so much for creating this awesome content. It has got me started in chatbots. I have one question. How to give multiple choices to the user and then proceed with the user’s response?

    1. Hi Palash,

      Thanks for checking out the tutorial, I’m glad you found it helpful 🙂

      Do you want to implement buttons or do you want to keep it fully conversational? If you want to go the button route, then I would suggest you checking out Slack connector on Rasa Core https://core.rasa.com/connectors.html#slack-setup , it has a button functionality already implemented (if you need any help with using it, just shoot me a message). If you don’t want to use buttons, then in your domain file you should simply create a template or an action which will send a message with multiple options to your user and the user’s response should be parsed using NLU model and saved as a slot. Let me know if you need help with anything 🙂

      1. Can I keep it both conversational and buttons whenever needed? I think the button route is sometimes very useful when I want to avoid any user mistakes. And what if I want to integrate the chatbot with a website and not with the existing messenger platform?

      2. Hey. Yes, your bot can use buttons and conversational responses depending on which one is needed. You will only need separate actions for those cases. Speaking of custom integration – it shouldn’t be a problem to integrate your chatbot with a custom website, for this, you can run Rasa as an http server (more on this here: https://core.rasa.com/http.html) or you can create your custom connector (here is an example for a custom connector from Rasa https://github.com/RasaHQ/rasa_core/blob/master/rasa_core/channels/custom.py).

  6. HI Justina, Thanks for the awesome work. I followed it and it worked. But couple of queries. How do i not end the conversation until, i say good bye? The bot automatically goes to BYE intent after it answers the weather results? could i keep asking weather for multiple times and not end the conversation , until i say bye?

    Thanks
    Roshan

    1. Hi Roshan. I am glad you gave it a try! 🙂 I think your problem related to the lack of training data. In my example I had only a few stories and it is likely that the bot will make a lot of mistakes. Try adding more stories to stories.md file and see if that improves the performance 🙂

  7. Hi,

    I am running this on Python 3.6 /Win 10 and this is happening while training the model:

    PS C:\developerfolder\weatherbot> python nlu_model.py
    Traceback (most recent call last):
    File “nlu_model.py”, line 1, in
    from rasa_nlu.converters import load_data
    File “C:\Users\renji\AppData\Local\Programs\Python\Python36\lib\site-packages\rasa_nlu\__init__.py”, line 5, in
    import version
    ModuleNotFoundError: No module named ‘version’

    1. Hey R.Phil 🙂 I believe the issue is due to the updates which were introduced to Rasa NLU after I recorded the tutorial. I have made an update recently where in requirements.txt I specified which Rasa NLU and Core version I used in a tutorial. If you prefer, you can use the latest versions of the libraries, but then check out the new repo which contains the code which should run with the latest releases of Rasa NLU and Core. Let me know if the issue persists 🙂

  8. Hi Justina,

    First of all thank you for the wonderful tutorial. I am getting the below error when I run the train_online.py, bot is not able to move to the actions.

    D:\Python36\lib\site-packages\sklearn\preprocessing\label.py:151: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use `array.size > 0` to check that an array is not empty.

    I don’t find solution anywhere, need your help

    1. Hi Lawrence. Thanks for checking out the tutorial! 🙂 The error you get is not really an error, it’s a deprecation warning coming from sklearn. This shouldn’t affect your models at all, it’s just an annoying message which gets printed every time an response is provided by your bot. You try muting them by following the instructions here https://docs.python.org/2/library/warnings.html (just add those lines to you train_online.py code. Let me know if that helps.

  9. Please help!I have installed the nlu trainer just like you said.But when I run “rasa-nlu-trainer” in the same directory as my JSON file, I get the following error :-

    Devanshs-MacBook-Pro:data devanshsingh$ rasa-nlu-trainer
    searching for the training examples…
    (node:32128) UnhandledPromiseRejectionWarning: Error: Can’t find training file, please try to specify it with the –source option
    at checkDone (/usr/local/lib/node_modules/rasa-nlu-trainer/server.js:98:15)
    at readData.then.catch.then (/usr/local/lib/node_modules/rasa-nlu-trainer/server.js:128:11)
    at
    (node:32128) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
    (node:32128) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

    1. Hey Devansh. That’s odd. Have you tried specifying the file using –source option?

  10. Thanks a lot Justina for the share. It would be really helpful if you can suggest me how can I train my model from data which is contained in pdf form. The chat bot should be able to answer all the FAQ related to the pdf file……Is there any way…?
    Thanks again and keep sharing.

    1. Hey Rahul. Thanks for checking out the tutorial! Could you give me the example of what queries users would have and how the answer of the bot would look like? The things is, Rasa is awesome for building goal-oriented bots, but I am not sure if it is the best choice for FAQ bots. An example of possible conversation would help me to give a more proper advice. If you prefer, you can send it to me as a private message or reach out to me using any social media channels you can find on my blog 🙂

  11. Hi Justina Petraityte,

    Hope you’re doing great!!! Your video about rasa core and rasa nlu is really awesome. You thought in such a way so that even a beginner in rasa /python can understand each and every line of it.

    Could u have a video about the latest rasa core/nlu changes?

    1. Hi Revathy. Thank you so much for your kind words, I really appreciate it! The goal of this tutorial was exactly that – to help people of all levels of programming expertise to get started with Rasa! Based on your awesome feedback I can assume that the goal was achieved 🙂

      I should post more content on Rasa quite soon so stay tuned!

  12. Hi Justina first of all thanks for this video. can you tell me how to create a context based chatbot? not stateless

    1. Hey Pankaj. Thanks for checking out the tutorial. You can use Rasa Core for context-based chatbots – slots are used to keep the context and track what the conversation is about. Can you give me an example of what do you mean by a context-based chatbot, not stateless?

      1. If you familiar with Dialogflow then they have follow–up intent I am taking about this only see one seneario here In my chatbot I have two follow 1) registration 2) troubleshooting so if user ask I want to register as new customer then my bot Will response tell me your name when user enter name then bot will response tell me your mail I’d ….so on… same I have some flow-up intent for troubleshooting how can I do this??

      2. So, in Rasa Core you would simply have an intent for that (in my tutorial I used an intent called ‘inform, which is a generic intent to provide the location information’. For the model to pick up different conversation patterns (one for when user provides all details immediately; another one for the case when a user just starts a registration and a bot has to ask all the details, etc.) you have to create training stories which would cover those scenarios. In my tutorial, I have a case when a user immediately told what location he is asking about and another one when a bot has to clarify those details.

      3. So this what Rasa Core is all about. You should train your dialogue management model using stories where bot asks all of those questions and the information which gets extracted from user’s responses should be save as slots to keep the context. Just like in my video – I trained the bot to follow up with a question about the location if a user didn’t provide the location immediately when asking about the weather.

  13. hi i am facing the entities extraction issue like my bot response enter your name then user will enter Michael Jackson but it will not extracted the value if i give like my name is Michael Jackson than it will extracted but i want to give only Michale Jackson
    here is the config file

    {
    “pipeline”:”spacy_sklearn”,
    “path”:”./models/nlu”,
    “data”:”./data/data.json”

    }

  14. Hi Justina! Great work done by you, tnx! Just loved this tutorial, easy to follow!

    For a followup of my bot I would like to be able to extract Dutch names as an entity. The default tagger doesn’t get Dutch names correctly very often. I have a list of all the Dutch names available; do you have a suggestion where to start / apply this functionality?

    Again, tnx and keep up the good work!

    Regards,
    Ramon

    1. Hey. Thanks for checking out the tutorial 🙂 You should be able to train the model in Dutch by using spacy dutch model https://spacy.io/usage/models (the change should be made inside the configuration file). Check it out and let me know if you need more help with it.

  15. Hi Justina Petraityte,

    Hope you are doing well!!!…
    Getting below error while installing the requirement.txt using pip :

    bleach 1.5.0 has requirement html5lib!=0.9999,!=0.99999,=0.999, but you’ll have html5lib 1.0.1 which is incompatible.
    tensorboard 1.8.0 has requirement html5lib==0.9999999, but you’ll have html5lib 1.0.1 which is incompatible.
    tensorflow 1.8.0 has requirement numpy>=1.13.3, but you’ll have numpy 1.13.1 which is incompatible.
    rasa-core 0.9.6 has requirement coloredlogs~=10.0, but you’ll have coloredlogs 7.3 which is incompatible.
    rasa-core 0.9.6 has requirement ConfigArgParse~=0.13.0, but you’ll have configargparse 0.12.0 which is incompatible.
    rasa-core 0.9.6 has requirement fakeredis~=0.10.0, but you’ll have fakeredis 0.8.2 which is incompatible.
    rasa-core 0.9.6 has requirement fbmessenger~=5.0, but you’ll have fbmessenger 4.3.1 which is incompatible.
    rasa-core 0.9.6 has requirement flask~=1.0, but you’ll have flask 0.12 which is incompatible.
    rasa-core 0.9.6 has requirement graphviz~=0.8.0, but you’ll have graphviz 0.7.1 which is incompatible.
    rasa-core 0.9.6 has requirement networkx~=2.0, but you’ll have networkx 1.11 which is incompatible.
    rasa-core 0.9.6 has requirement requests~=2.15, but you’ll have requests 2.14.2 which is incompatible.

    but still the installation goes on, is it fine? or do i need to change any version?

  16. Hi Justina,

    Thank you for providing such good and useful information on Rasa NLU project.

    I have an issue while running ‘action.py ‘ file at ‘apixu’ package/library.
    error message: ModuleNotFoundError: No module named ‘apixu’.

    please help me out how to install ‘apixu’ module in anaconda-spyder for python.

    Thank you.

  17. Hi Justina,

    Enjoying your tutorial immensely and the quality is top-notch. Thank you for all your efforts! Not sure if this is the forum for a technical question but I ran into an error that is not making sense regarding the api call: Error code 1003: “Parameter q is missing.

    I’ve checked the code against your github and it all checks out. Any ideas?

    1. Hey Sean. Thanks a lot for your kind words, I am super happy you found my tutorial helpful! 🙂 I think what is happening is that it is not extracting the entity ‘location’ correctly. That’s why the parameter q is empty and throws you an error, when it tries to make an api call. A good solution for this would be to add more training data for NLU model (I had a really small sample for it in my tutorial) so that the model would be more confident in extracting entities.

      1. Worked brilliantly…thank you!! I finished the tutorial and I’m extremely interested in learning more about Rasa.

        Great work!!!

      2. Awesome! Glad it worked and happy to hear you are going to continue learning about Rasa!

      3. Awesome! Glad to hear it worked. Keep me updated with your progress 🙂

  18. Hey justina, thank you so much for ur awesome video. I have one small request, could u explain about the policies in rasa and about tensorflow? It ll be really helpful fr deep diving.. but as i said earlier your video is a blessing fr everyone who wants to learn rasa.

  19. Hello Justina, really nice and helpful tutorial. I have a question hopefully you can give me some tips. In your example their can be mutliple locations with the same name. So for example you ask “how is the weather in Neustadt”. And the API-Response gives you a list of different locations with this name. How would the user interaction look like. Would you save the response in a slot with type list and let the user choose one. What is the best solution for this with Rasa-Core. Thank you very much for your answer.

    1. Hey Jan. Thanks for checking out the tutorial! 🙂 How you handle the cases when an API returns more than one possible answer depends on how you want your chatbot to operate. A naive approach could be to select the random one and send it back to the user or a lot better one would be to save the returned options in a slot as a list and train the bot so that when the list slot is populated, a bot would ask a follow up question so that a user could provide more info of what specifically they want to see and then a bot could make a more accurate suggestions by picking the best option from the saved slot.

  20. Hi Justina, thank you very much for your tutorial. I have got a question. How would you program a user interaction when they are different locations from a Api-Response. For example their are a lot of cities with the Name Neustadt in Germany. How would you let the user choose out of a list, which one he means. Iam struggling with the interreupt of my first intenet to connect the two actions. I hope you can give me some tips to solve this problem. Sinercerly Jan

Leave a Reply