Skip to content

Instantly share code, notes, and snippets.

@sidferreira
Last active June 14, 2023 14:50
Show Gist options
  • Star 9 You must be signed in to star a gist
  • Fork 4 You must be signed in to fork a gist
  • Save sidferreira/2b6b640e25544d601f311e35e430bce0 to your computer and use it in GitHub Desktop.
Save sidferreira/2b6b640e25544d601f311e35e430bce0 to your computer and use it in GitHub Desktop.
WatermelonDB preloaded db

This is the basic structure I'm using in an app.

getDatabasePath will handle any detail on unzip, copy, etc,

index will do all the usual dirty work, including withDatabase that won't explode if the database is missing

WatermelonProvider to cover original one til the DB is ready

import RNFS from "react-native-fs";
// Currently android will add `.db` to file name...
async function getDatabasePath(dbName) {
let dbPath = `${RNFS.DocumentDirectoryPath}/${dbName}`;
const exists = await RNFS.exists(dbPath.concat(".db"));
if (!exists) {
try {
await RNFS.copyFileAssets(dbName.concat(".db"), dbPath.concat(".db"));
} catch (e) {
console.log(">>> getDatabasePath error", e);
}
}
return dbPath;
}
export default getDatabasePath;
import RNFS from 'react-native-fs';
import * as RNZIP from 'react-native-zip-archive
async function getDatabasePath(dbName) {
let sanitizedDbName = dbName;
if (!sanitizedDbName.endsWith(".db")) {
sanitizedDbName = sanitizedDbName.concat(".db");
}
let dbPath = `${RNFS.DocumentDirectoryPath}/${sanitizedDbName}`;
const exists = await RNFS.exists(dbPath);
if (!exists) {
const assetPath = `${RNFS.MainBundlePath}/${sanitizedDbName.replace(
".db",
".zip"
)}`;
const existsZip = await RNFS.exists(assetPath);
if (existsZip) {
const tmpPath = `${RNFS.DocumentDirectoryPath}/tmp`;
await RNZIP.unzip(assetPath, tmpPath);
await RNFS.moveFile(`${tmpPath}/${sanitizedDbName}`, dbPath);
await RNFS.unlink(tmpPath);
}
}
return dbPath;
}
export default getDatabasePath;
async function getDatabasePath(dbName) {
let fullDbName = dbName;
if (!fullDbName.endsWith('.db') && !fullDbName.includes('?')) {
fullDbName = fullDbName.concat('.db');
}
let dbPath = dbName.includes(process.cwd())
? fullDbName
: `${process.cwd()}/${fullDbName}`;
return dbPath;
}
export default getDatabasePath;
import getDatabasePath from './getDatabasePath';
const DEFAULT_DATABASE_NAME = 'watermelon';
const promises: { [key: string]: Promise<Database> } = {};
const instances: { [key: string]: Database } = {};
async function init(dbName: string = DEFAULT_DATABASE_NAME) {
const dbPath = await getDatabasePath(dbName);
const adapter = new SQLiteAdapter({
schema,
migrations,
dbName: dbPath,
});
const instance = new Database({
adapter,
modelClasses,
actionsEnabled: true,
});
instances[dbName] = instance;
// sync();
return db;
}
export const getDatabase = async (
dbName: string = DEFAULT_DATABASE_NAME,
) => {
if (!promises[dbName]) {
promises[dbName] = init(dbName);
}
const instance = await promises[dbName];
return instance;
};
// For _very_ specific uses
export const getDatabaseSync = (altDbName: string = DEFAULT_DATABASE_NAME) => {
return instances[altDbName];
};
// custom withDatabase to handle the possibility of not having the database just yet...
export type Omit<T, K> = Pick<T, Exclude<keyof T, K>>;
export const withDatabase = <T extends DatabaseProp>(
Wrapped: React.ComponentType<T>,
) => {
const WrappedComponent: React.ComponentType<Omit<T, DatabaseProp>> = (
props,
) => {
const database = useDatabase();
if (!database) {
return <View />;
}
return <Wrapped {...(props as T)} database={database} />;
};
return WrappedComponent;
};
import React, { ReactChild } from 'react';
import { Database } from '@nozbe/watermelondb';
import { getWatermelon } from 'app/utils/DBWatermelonOld/DBWatermelonOld';
import DatabaseProvider from '@nozbe/watermelondb/DatabaseProvider';
const WatermelonProvider = (props) => {
const watermelonRef = React.useRef();
const [database, setDatabase] = React.useState();
if (!watermelonRef.current) {
const promise = getWatermelon();
if (promise) {
watermelonRef.current = promise;
watermelonRef.current.then(setDatabase);
}
}
return database ? <DatabaseProvider database={database} {...props} /> : null;
};
export default WatermelonProvider;
@sidferreira
Copy link
Author

@Stophface I would need to check the code

@Stophface
Copy link

@sidferreira I can add you to my repo if you want. Might be the easiest.

@sidferreira
Copy link
Author

@Stophface go fo it

@Stophface
Copy link

@sidferreira I added you to the repository. I created an Issue describing the problem https://github.com/Stophface/MapStar/issues/4

@Stophface
Copy link

@sidferreira Any news on this?

@sidferreira
Copy link
Author

@Stophface I'll try to check it later today, sorry...

@Stophface
Copy link

@sidferreira Alrighty.

@sidferreira
Copy link
Author

@Stophface checking it rn

@fullstackNRJ
Copy link

Hey, @sidferreira I landed here after looking for ways to prepopulate one of the watermelondb table with the data from a asset, a sqlite db file. I am not able to make it work.

  • So, the scenario is I ship a static some_data.db file inside www/assets in my react-native app.
  • This file has 200,000+ records basically product data.
  • I also use watermelondb for other data like users, sales etc.
  • Now, I have a use case where I need to keep updating this static database file with new products and it's pain to maintain two dbs.
  • I use react-native-sqlite-storage to read and write into the product database file.

what I want is to add the content of this static database file into one of the table in watermelondb on app launch, I can't figure out how to do so yet. I tried the trick mentioned above but nothing happens it just loads the other modelClasses and tables.
IDK what I'm doing wrong, any help will be much appreciated.

@sidferreira
Copy link
Author

@devnrj07 We have a node command line that generates the database and we ship it ready. We then copy from assets to a working folder.

@fullstackNRJ
Copy link

Hey @sidferreira , thanks for the quick reply. Quick question, after you copy it from assets to a working folder were you able to pre-populate watermelondb tables with the database that you shipped ?

@sidferreira
Copy link
Author

@devnrj07 that's what I meant.. The db file already has everything in it

@fullstackNRJ
Copy link

Ah, I get your point now. I will see if that's possible in my case. Thanks a lot.

@Stophface
Copy link

@devnrj07 If you solve that, can you share (code) how? I banged my head on this for a while and started using https://github.com/andpor/react-native-sqlite-storage because I was not able to use a pre-populated database.

@fullstackNRJ
Copy link

@Stophface nope mate not yet! I'm few hours away from giving up 🤞. Even, I use the same package and it's a pain to update data in both dbs.

  • In case of watermelon, I just use sync feature but for other db, I have to do it using that rn-sqlite-package, this is causing lot of bugs!
  • I'm probably going to take the naive approach, adding stuff in js runtime i.e if turbo sync works!
    will keep you updated!
    btw, how are you managing it right now ?

@Stophface
Copy link

@devnrj07 Yeah, I cannot believe that this such a pain in the ... to work with pre-populated databases... I included expo-sqlite to my project, no luck. Then I tried it with watermelonDB, no luck. I wasted so much time of my life on this.

The way I am doing it right now is with https://github.com/andpor/react-native-sqlite-storage (make sure you follow the installation instructions closely).

For iOS I put the database into ios/APP_NAME/www/some_database.db, for Android I put it into this folder android/app/src/main/assets/www/some_database.db. Make sure you add the DB file to the project in iOS as stated in the instructions of the Library https://github.com/andpor/react-native-sqlite-storage#setting-up-your-project-to-import-a-pre-populated-sqlite-database-from-application-for-ios.

And then its this open database call, using the name of my databasefile (some_database.db): https://github.com/andpor/react-native-sqlite-storage#setting-up-your-project-to-import-a-pre-populated-sqlite-database-from-application-for-ios

SQLite.openDatabase({name : "some_database.db", createFromLocation : 1}, okCallback,errorCallback);

However, I'd prefer to do this with watermelonDB since this comes with a few advantages. But, no change that I get this working. I think what is happening when I use watermelonDB: I try to open the pre-populated database, but its overwriting my content of the database by creating a database/table with the exact same name... And I do not know how to get around this.

@sidferreira
Copy link
Author

@Stophface @devnrj07 long story short, I created a node script that includes the same classes of the database and run all the import commands. After populating the DB, it already copy the files to the assets folders.

As I use WatermelonSync, I just use the sync method, but is just like batch prepareCreate statements.

Note that I create the DB from scratch every time.

@fullstackNRJ
Copy link

fullstackNRJ commented Feb 18, 2022

@Stophface I'm doing pretty much same what you just described, relying on react-native-sqlite-storage just to have access to my products catalog.

  • Rest of the data is in watermelondb and it works great. But, I want the product catalog data also in watermelondb's table, same as your requirement.
  • I've reached to the same conclusion that it is overwriting it somehow instead of using that .db file as the source for creating/populating table.

@sidferreira Thanks for more information. I think I'm definitely doing something wrong and since I'm running out of time, skipping this method for now.

Guys, let's stay connected on twitter @devnrj07 ! @Stophface we'll sort it out soon.

@sidferreira
Copy link
Author

@devnrj07 @Stophface to me it was really straightforward. If you want, we can meet online to find out what's going on...

@fullstackNRJ
Copy link

@sidferreira that would be really great!

@Stophface
Copy link

@sidferreira @devnrj07 Awesome, that is really very nice. Which timezone are you living in? I am UTC+1.

@fullstackNRJ
Copy link

@Stophface I am UTC + 5:30. Let us see what is @sidferreira 's availability, possible today or tomorrow

@sidferreira
Copy link
Author

@Stophface @devnrj07 reach me on Twitter https://twitter.com/sidferreiraz

@gabriel-almeida250
Copy link

gabriel-almeida250 commented Jun 7, 2023

Hello, @sidferreira @Stophface

Is this solution still functional? I am currently trying to implement this method but I am having some difficulties.

@Stophface
Copy link

@gabriel-almeida250 you probably need this in addition Nozbe/WatermelonDB#774 (comment)

@sidferreira
Copy link
Author

@gabriel-almeida250 Fala Gabriel! So, I moved to another company and I'm not using it anymore, but should be...

@gabriel-almeida250
Copy link

gabriel-almeida250 commented Jun 9, 2023

Thanks for your attention, guys @sidferreira @Stophface! I'm going to try to implement it now.

Fala, @sidferreira! Beleza, agradeço pelo rápido retorno!

@KosolapovR
Copy link

@gabriel-almeida250
Please let us know if you would be able to implement it, I'm also trying to prepopulate data using this method.

@KosolapovR
Copy link

KosolapovR commented Jun 14, 2023

I can say that this solution still work.
My personal struggle was that I misunderstood how I should create watermelon.db file, some sql statement packed into text file with .db extension not worked for some reason and I was able to solve it only by creating db file via "DB browser for sqlite".
I'm not sure if it will be difficult to prepopuate db later after multiple consecutive migrations.

@gabriel-almeida250
Copy link

Ok, @KosolapovR. I haven't had time to implement the solution yet, I had to prioritize other activities, but thanks for the feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment