Skip to content

Instantly share code, notes, and snippets.

@wochinge
Last active March 11, 2020 16:06
  • Star 1 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save wochinge/547a844f9851bb031c129e0416489068 to your computer and use it in GitHub Desktop.
Migrates a Rasa tracker store (Redis, Mongo, SQL) to Rasa X.
# This gist is deprecated.
# Please follow the guide here
# https://rasa.com/docs/rasa-x/installation-and-setup/existing-deployment/#import-existing-conversations-from-rasa-open-source
# to migrate from Rasa Open Source to Rasa X
@Btibert3
Copy link

Btibert3 commented Jul 9, 2019

That was it, but strange that my local testing was creating the database, I was talking to it, but no dice. Grabbed my dev db and it worked great. Thanks!

@FrancisThibaultNRC
Copy link

I have a question regarding the use of this python script:
I run a mongdb docker container using the following command on Win10:
docker run -p 27017:27017 -v ./mongodb-clone:/data/db -d mongo

The mongodb-clone directory exists, it is a copy of the directory coming from the production server:
scp username@host:/my/own/datadir mongodb-clone

when I ran this migrating script:
python migrate_tracker_store_to_rasa_x.py

and got the following error:
image

Any hints???

Thanks.

@DiegoProtec
Copy link

DiegoProtec commented Oct 25, 2019

Hello,

When I run the script, I receive this:

trace

My new endpoint is like this:

tracker_store:
  type: sql
  dialect: "oracle"
  url: host
  port: 5555
  username: user
  password: pass
  db: db

What does this message mean, is it an error, warning, bug? The tables were created, but I don't know whether the process was correctly finished.
Can anybody help me?

@DiegoProtec
Copy link

DiegoProtec commented Nov 4, 2019

Finally I managed to run "migrate_tracker_store_to_rasa_x.py", passing "new_endpoints.yml" like this:

tracker_store:
  type: "sql"
  dialect: oracle
  url: "oracle://admin:teste@127.0.0.1:1521/rasa"

But I take this:

sqlalchemy.exc.DatabaseError: (cx_Oracle.DatabaseError) ORA-00906: left parenthese not found
[SQL:
CREATE TABLE entity_synonym_value (
        id INTEGER NOT NULL,
        entity_synonym_id INTEGER,
        name VARCHAR2,
        PRIMARY KEY (id),
        FOREIGN KEY(entity_synonym_id) REFERENCES entity_synonym (id) ON DELETE cascade
)

]
(Background on this error at: http://sqlalche.me/e/4xp6)

@wochinge
Copy link
Author

wochinge commented Nov 6, 2019

@DiegoProtec Your error is a bug in Rasa X which makes in incompatible with Oracle. We already fixed it, and it will be part of the next release (some other things have to go in this as well, which is why I can't do that right now). Sorry for the inconvenience!

@FrancisThibaultNRC Fixed your error. The migration should have worked anyway, but this step is not longer necessary so I removed it.

@DiegoProtec
Copy link

DiegoProtec commented Nov 6, 2019

@wochinge thank you so much, now after logger printed, I can understand the flow. I'm passing a new tracker store for converstaions at environment variable "DB_URL", it migrate my events and conversations but can't migrate the values to the tables, the tables stay empties.

@FrancisThibaultNRC
Copy link

FrancisThibaultNRC commented Nov 6, 2019

I retried the new script and even if I specified the username and password in the endpoints.yml file, I got the following message while running the following command (sudo docker run -p 27017:27017 -v ~/git/aida/agents/agents/cybersickness/mongodb/data:/data/db -d mongo) on linux:
SASL SCRAM-SHA-1 authentication failed for rasa on admin from client 172.17.0.1:48740 ; UserNotFound: Could not find user "rasa" for db "admin"

This is the right username but not the right database name as specified in the following endpoints.yml:

tracker_store:
type: mongod
url: mongodb://localhost:27017
db: cybersickness
username: rasa
password: example

Any ideas?

@FrancisThibaultNRC
Copy link

@wochinge when you said this step is no longer necessary and you removed it, so which step were you referring to?

Thanks for your collaboration.

@DiegoProtec
Copy link

DiegoProtec commented Nov 11, 2019

Hello!

When I run the "migrate.py", with environment variable "DB_URL":

DB_URL=oracle://admin:admin@localhost:1521/db
occurs this:

sqlalchemy.exc.DatabaseError: (cx_Oracle.DatabaseError) ORA-00936: expression not found
[SQL: select name, synonym from entity_synonym]
(Background on this error at: http://sqlalche.me/e/4xp6)

@wochinge
Copy link
Author

@DiegoProtec That's an issue with Rasa X and compatible. Can you please create an issue in the rasa repository (I believe you have already done that, didn't you)?

@DiegoProtec
Copy link

Hi @wochinge, here is there, issue.
Ty!

@FrancisThibaultNRC
Copy link

Hi @wochinge,

I have been able to migrate conversations from rasa tracker store into sql databases, thank you so much for your script, worked nicely. Could it be possible to extract conversations by given a range (456-700). I know that now we can extract a maximum number of conversations (--max-trackers 10), but it could be nice to extract based on a given range.

Thanks.

@nicolasfarina
Copy link

the script doesn't work in rasa-x 0.24.1

@wochinge
Copy link
Author

@nicolasfarina Which error are you getting?

@nicolasfarina
Copy link

@nicolasfarina Which error are you getting?

Version of Rasa X is 24.6 in Rasa x 23.5 it is work.

image

@kevinthenet
Copy link

@nicolasfarina I believe that's because you have an older version of Rasa you're running the migration with, but you have a conversation on the database that has a document from Rasa 1.6 and up. I removed the document with the event session_started from my Mongo database and was able to run the migrations fine.

@nicolasfarina
Copy link

@wochinge Any solution or should we delete the event from the tracker.db?

@bardx
Copy link

bardx commented Feb 25, 2020

I'm using Rasa 1.7.0 with Redis 5.0.7.
I get this error:

File "migrate_tracker_store_to_rasa_x.py", line 259, in
args.max_trackers,
File "migrate_tracker_store_to_rasa_x.py", line 89, in _migrate_tracker_store_to_rasa_x
old_tracker_store = _get_tracker_store_from_endpoints_config(old_endpoints_file)
File "migrate_tracker_store_to_rasa_x.py", line 170, in _get_tracker_store_from_endpoints_config
tracker_store = TrackerStore.find_tracker_store(
AttributeError: type object 'TrackerStore' has no attribute 'find_tracker_store'

@wochinge
Copy link
Author

Please use rasa export now which is officially maintained and doesn't break between releases :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment