Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Migrates a Rasa tracker store (Redis, Mongo, SQL) to Rasa X.
# This gist is deprecated.
# Please follow the guide here
# https://rasa.com/docs/rasa-x/installation-and-setup/existing-deployment/#import-existing-conversations-from-rasa-open-source
# to migrate from Rasa Open Source to Rasa X
@EmannuelRoque

This comment has been minimized.

Copy link

@EmannuelRoque EmannuelRoque commented Jul 2, 2019

Hi, I'm getting this error:

Traceback (most recent call last):
File "migrate_tracker_store_to_rasa_x.py", line 172, in
_migrate_tracker_store_to_rasa_x(path_to_endpoints_file, args.max_trackers)
File "migrate_tracker_store_to_rasa_x.py", line 66, in _migrate_tracker_store_to_rasa_x
db_session = db_utils.get_database_session(True, create_tables=True)
TypeError: get_database_session() got an unexpected keyword argument 'create_tables'

Thanks
Emannuel Roque

@Btibert3

This comment has been minimized.

Copy link

@Btibert3 Btibert3 commented Jul 4, 2019

im getting the same error

@Btibert3

This comment has been minimized.

Copy link

@Btibert3 Btibert3 commented Jul 4, 2019

In addition to above, I attempted to run this against my running Mongo tracking store for the bot and got the following error: [Errno -3] Temporary failure in name resolution

The mongo server is running and I confirmed via Mongo Express that my conversations are being logged.

@wochinge

This comment has been minimized.

Copy link
Owner Author

@wochinge wochinge commented Jul 5, 2019

TypeError: get_database_session() got an unexpected keyword argument 'create_tables'

will work on fixing this asap

Temporary failure in name resolution

seems you are using a host name, which can't be resolved by your dns server. What's your setup?

@wochinge

This comment has been minimized.

Copy link
Owner Author

@wochinge wochinge commented Jul 5, 2019

@Btibert3 @EmannuelRoque Fixed the script. Please let me know if you run into any more errors.

@EmannuelRoque

This comment has been minimized.

Copy link

@EmannuelRoque EmannuelRoque commented Jul 5, 2019

@wochinge It's working! Perfect! Thanks....

@Btibert3

This comment has been minimized.

Copy link

@Btibert3 Btibert3 commented Jul 8, 2019

Getting this when using a sqlite backend.

(rasabot) Brocks-MBP-2:pqe-chatbot btibert$ python migrate_tracker_store_to_rasa_x.py 
INFO:apscheduler.scheduler:Scheduler started
Welcome to Rasa X 🚀 

This script will migrate your old tracker store to the new SQL based Rasa X tracker store.
Let's start!

? Please provide the path to your endpoints configuration which specified the credentials for your old tracker store endpoints.y
ml                                                                                                                              
Start migrating 0 trackers.
0it [00:00, ?it/s]
None
Traceback (most recent call last):
  File "migrate_tracker_store_to_rasa_x.py", line 172, in <module>
    _migrate_tracker_store_to_rasa_x(path_to_endpoints_file, args.max_trackers)
  File "migrate_tracker_store_to_rasa_x.py", line 95, in _migrate_tracker_store_to_rasa_x
    set_latest_event_id(db_session, rasa_x_tracker_store)
  File "migrate_tracker_store_to_rasa_x.py", line 154, in set_latest_event_id
    existing.latest_event_id = max_event_id
AttributeError: 'NoneType' object has no attribute 'latest_event_id'
@wochinge

This comment has been minimized.

Copy link
Owner Author

@wochinge wochinge commented Jul 9, 2019

@Btibert3 This seems weird. That could only happen if zero events were migrated. Maybe check your database logs if anything indicates errors there.

@Btibert3

This comment has been minimized.

Copy link

@Btibert3 Btibert3 commented Jul 9, 2019

That was it, but strange that my local testing was creating the database, I was talking to it, but no dice. Grabbed my dev db and it worked great. Thanks!

@FrancisThibaultNRC

This comment has been minimized.

Copy link

@FrancisThibaultNRC FrancisThibaultNRC commented Oct 24, 2019

I have a question regarding the use of this python script:
I run a mongdb docker container using the following command on Win10:
docker run -p 27017:27017 -v ./mongodb-clone:/data/db -d mongo

The mongodb-clone directory exists, it is a copy of the directory coming from the production server:
scp username@host:/my/own/datadir mongodb-clone

when I ran this migrating script:
python migrate_tracker_store_to_rasa_x.py

and got the following error:
image

Any hints???

Thanks.

@DiegoProtec

This comment has been minimized.

Copy link

@DiegoProtec DiegoProtec commented Oct 25, 2019

Hello,

When I run the script, I receive this:

trace

My new endpoint is like this:

tracker_store:
  type: sql
  dialect: "oracle"
  url: host
  port: 5555
  username: user
  password: pass
  db: db

What does this message mean, is it an error, warning, bug? The tables were created, but I don't know whether the process was correctly finished.
Can anybody help me?

@DiegoProtec

This comment has been minimized.

Copy link

@DiegoProtec DiegoProtec commented Nov 4, 2019

Finally I managed to run "migrate_tracker_store_to_rasa_x.py", passing "new_endpoints.yml" like this:

tracker_store:
  type: "sql"
  dialect: oracle
  url: "oracle://admin:teste@127.0.0.1:1521/rasa"

But I take this:

sqlalchemy.exc.DatabaseError: (cx_Oracle.DatabaseError) ORA-00906: left parenthese not found
[SQL:
CREATE TABLE entity_synonym_value (
        id INTEGER NOT NULL,
        entity_synonym_id INTEGER,
        name VARCHAR2,
        PRIMARY KEY (id),
        FOREIGN KEY(entity_synonym_id) REFERENCES entity_synonym (id) ON DELETE cascade
)

]
(Background on this error at: http://sqlalche.me/e/4xp6)
@wochinge

This comment has been minimized.

Copy link
Owner Author

@wochinge wochinge commented Nov 6, 2019

@DiegoProtec Your error is a bug in Rasa X which makes in incompatible with Oracle. We already fixed it, and it will be part of the next release (some other things have to go in this as well, which is why I can't do that right now). Sorry for the inconvenience!

@FrancisThibaultNRC Fixed your error. The migration should have worked anyway, but this step is not longer necessary so I removed it.

@DiegoProtec

This comment has been minimized.

Copy link

@DiegoProtec DiegoProtec commented Nov 6, 2019

@wochinge thank you so much, now after logger printed, I can understand the flow. I'm passing a new tracker store for converstaions at environment variable "DB_URL", it migrate my events and conversations but can't migrate the values to the tables, the tables stay empties.

@FrancisThibaultNRC

This comment has been minimized.

Copy link

@FrancisThibaultNRC FrancisThibaultNRC commented Nov 6, 2019

I retried the new script and even if I specified the username and password in the endpoints.yml file, I got the following message while running the following command (sudo docker run -p 27017:27017 -v ~/git/aida/agents/agents/cybersickness/mongodb/data:/data/db -d mongo) on linux:
SASL SCRAM-SHA-1 authentication failed for rasa on admin from client 172.17.0.1:48740 ; UserNotFound: Could not find user "rasa" for db "admin"

This is the right username but not the right database name as specified in the following endpoints.yml:

tracker_store:
type: mongod
url: mongodb://localhost:27017
db: cybersickness
username: rasa
password: example

Any ideas?

@FrancisThibaultNRC

This comment has been minimized.

Copy link

@FrancisThibaultNRC FrancisThibaultNRC commented Nov 6, 2019

@wochinge when you said this step is no longer necessary and you removed it, so which step were you referring to?

Thanks for your collaboration.

@DiegoProtec

This comment has been minimized.

Copy link

@DiegoProtec DiegoProtec commented Nov 11, 2019

Hello!

When I run the "migrate.py", with environment variable "DB_URL":

DB_URL=oracle://admin:admin@localhost:1521/db
occurs this:

sqlalchemy.exc.DatabaseError: (cx_Oracle.DatabaseError) ORA-00936: expression not found
[SQL: select name, synonym from entity_synonym]
(Background on this error at: http://sqlalche.me/e/4xp6)
@wochinge

This comment has been minimized.

Copy link
Owner Author

@wochinge wochinge commented Nov 12, 2019

@DiegoProtec That's an issue with Rasa X and compatible. Can you please create an issue in the rasa repository (I believe you have already done that, didn't you)?

@DiegoProtec

This comment has been minimized.

Copy link

@DiegoProtec DiegoProtec commented Nov 12, 2019

Hi @wochinge, here is there, issue.
Ty!

@FrancisThibaultNRC

This comment has been minimized.

Copy link

@FrancisThibaultNRC FrancisThibaultNRC commented Nov 18, 2019

Hi @wochinge,

I have been able to migrate conversations from rasa tracker store into sql databases, thank you so much for your script, worked nicely. Could it be possible to extract conversations by given a range (456-700). I know that now we can extract a maximum number of conversations (--max-trackers 10), but it could be nice to extract based on a given range.

Thanks.

@nicolasfarina

This comment has been minimized.

Copy link

@nicolasfarina nicolasfarina commented Jan 13, 2020

the script doesn't work in rasa-x 0.24.1

@wochinge

This comment has been minimized.

Copy link
Owner Author

@wochinge wochinge commented Jan 30, 2020

@nicolasfarina Which error are you getting?

@nicolasfarina

This comment has been minimized.

Copy link

@nicolasfarina nicolasfarina commented Jan 30, 2020

@nicolasfarina Which error are you getting?

Version of Rasa X is 24.6 in Rasa x 23.5 it is work.

image

@ortsaCniveK

This comment has been minimized.

Copy link

@ortsaCniveK ortsaCniveK commented Feb 3, 2020

@nicolasfarina I believe that's because you have an older version of Rasa you're running the migration with, but you have a conversation on the database that has a document from Rasa 1.6 and up. I removed the document with the event session_started from my Mongo database and was able to run the migrations fine.

@nicolasfarina

This comment has been minimized.

Copy link

@nicolasfarina nicolasfarina commented Feb 4, 2020

@wochinge Any solution or should we delete the event from the tracker.db?

@bardx

This comment has been minimized.

Copy link

@bardx bardx commented Feb 25, 2020

I'm using Rasa 1.7.0 with Redis 5.0.7.
I get this error:

File "migrate_tracker_store_to_rasa_x.py", line 259, in
args.max_trackers,
File "migrate_tracker_store_to_rasa_x.py", line 89, in _migrate_tracker_store_to_rasa_x
old_tracker_store = _get_tracker_store_from_endpoints_config(old_endpoints_file)
File "migrate_tracker_store_to_rasa_x.py", line 170, in _get_tracker_store_from_endpoints_config
tracker_store = TrackerStore.find_tracker_store(
AttributeError: type object 'TrackerStore' has no attribute 'find_tracker_store'

@wochinge

This comment has been minimized.

Copy link
Owner Author

@wochinge wochinge commented Mar 11, 2020

Please use rasa export now which is officially maintained and doesn't break between releases :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.