https://www.moosejs.com/ Docs: https://docs.moosejs.com/
The create-moose-app is a NodeJS script that creates (seeds) the a moosejs application.
Issuing the following command creates an initial project folder.
npx create-moose-app my-moose-app
$ tree
├── app
│ ├── datamodels
│ ├── flows
│ └── insights
│ ├── charts
│ └── metrics
└── package.json
7 directories, 1 file
The package.json
contains a key development dependency:
"devDependencies": {
"@514labs/moose-cli": "latest"
}
When the user runs npm i
or npm install
the devDependency for moose-cli is pulled from NPM.
That results in the creation of two additional modules under the node_modules/@514labs directory branch: moose-cli
and moose-cli-darwin-arm64
.
$ tree
├── app
│ ├── datamodels
│ ├── flows
│ └── insights
│ ├── charts
│ └── metrics
├── node_modules
│ └── @514labs
│ ├── moose-cli
│ │ ├── LICENSE
│ │ ├── README.md
│ │ ├── dist
│ │ │ └── index.js
│ │ └── package.json
│ └── moose-cli-darwin-arm64
│ ├── bin
│ │ └── moose-cli
│ └── package.json
├── package-lock.json
└── package.json
The first dependency is @514labs/moose-cli
which is executed by npx and runs the dist/index.js
script. That script in-turn runs (spawns) the @514labs/moose-cli-darwin-arm64/moose-cli
application.
It wasn't clear to me is how the @514labs/moose-cli-darwin-arm64
folder is created. After speaking with callicles I learned that the correct version is chosen from the list below using this line in the create-moose-app/src/index.ts
script
require.resolve(
`@514labs/moose-cli-${os}-${arch}/bin/moose-cli${extension}`,
)
from a list of these optional dependencies defined in the package.json file.
"optionalDependencies": {
"@514labs/moose-cli-darwin-arm64": "0.3.83",
"@514labs/moose-cli-darwin-x64": "0.3.83",
"@514labs/moose-cli-linux-arm64": "0.3.83",
"@514labs/moose-cli-linux-x64": "0.3.83"
},
The moose-cli is a command-line executable with an entry point at main.rs
. In main.rs a config is checked to determine whether the executable is built a development
or production
executable.
The Tokio crate is used to initialize a new multi-threaded runtime builder - which I suspect is a container for async functions. Per the source documentation it's necessary to avoid having main() be an async function because the initialization of sentry instrumentation needs to happen before Tokio takes over the main process thread.
Tokio blocks the current thread until the future returned by cli::cli_run()
completes. This prevents the process main from terminating until other async calls are completed.
The cli_run
function is defined in the cli.rs
create and performs the following tasks upon start:
- setup_user_directory()
- defined in settings.rs
- obtains the user directory and then uses
std::fs::create_dir_all()
to create the parent folders leading to the full path provided.
- init_config_file()
- defined in settings.rs
- checks whether there is a config file path exists and if so creates a TOML file with initial hardcoded content.
- read_settings()
- defined in settings.rs
- calls
config_path()
to retrieve the path to the config file.- calls
user_directory()
to retrieve the path to the user's directory- calls home_dir() from home crate
- https://crates.io/crates/home
- Required because: "The definition of
home_dir
provided by the standard library is incorrect because it considers theHOME
environment variable on Windows."
- calls home_dir() from home crate
- calls
- Uses the cargo
config
crate to support config related concerns
- setup_logging()
- defined in settings.rs
- Configures logging file and logging format
- Defines a closure for handling incoming log messages
- Cli::parse()
- Parses the command line arguments
- top_command_handler()
- Top-level command dispatcher
In the cli_run
function in cli.rs
the above functions are used to:
- Determine the path location of the user's home directory
- Create an initial configuration file if one doesn't exists
- Use that path location to load the (
config.toml
) configuration file - Setup logging using the path specified in
config.logger
- The Cargo
log
crate is used. - LoggerSettings are created in the logger.rs file.
- A
cli.log
file is used to hold log messages - Uses the cargo
fern
crate for log entry formatting.- https://docs.rs/fern/latest/fern/
- Uses a closure with the fern.format function to handle the output display / logging format for incoming log messages.
- The Cargo
- The process command line arguments are parsed using
Cli::parse()
- The
struct Cli
incli.rs
is derived fromclap::Parser
which comes from theclap
crate.- https://docs.rs/clap/latest/clap/
- Clap is a command line argument parser
- So
Cli::Parse()
invoked the derived Parse function from clap. - After parsing a
cli
is returned which contains a.command
value.
- The
- Both the config and the cli.command is passed to the
top_command_handler()
- Execution is awaited until top_command_handler completes.
top_command_handler()
- First checks whether the settings (config)
features.coming_soon_wall
is not set to true.- If true then a message is displayed to the user saying coming soon and offers an invitation to join the MooseJS community.
- The
cli.rs
crate references aCommands
crate which maintains an enum with command variants:Init
,Dev
,Update
,Stop
,Clean
- The
Commands
enum is derived from the clips::Subcommand trait.
- The
- First checks whether the settings (config)
The top_command_handler()
functions as a command dispatcher and is inspired by the Rust command design pattern The implementation uses RoutineController
defined in routines.rs
to hold a Vec collection of routines. The RoutineController's
impl defines an add_routine
method for pushing routines to the routines collection and a run_routines
method for iterating through the collection of routines an calling the routine.run() method on each.
The routines.rs
module defines a Routine
trait which can be used by command handlers to implement run_xxxx() methods for execution. This is where the meat of a command's operations are triggered.
Each command handler in top_command_handler()
creates a RoutineController
and called one or more controller.add_routine()
methods before calling the controller.run_routines()
method.
-
Commands::Init:
- Determines the path for a new project
- If path is in user's home directory then an error message is displayed saying "You cannot create a project in your home directory" and the process exists.
- A new
InitalizeProject
struct is created and passed into thecontroller.add_routine()
and later executed via thecontroller.run_routines()
method.- A new project struct is created using
from_dir
defined in theproject.rs
crate
- A new project struct is created using
- The code operations required to implement a project initialization is stored in
initialize.rs
there the Route trait is implemented for theInitalizeProject
struct. - Afterwards, the project file is initialized with default parameters from:
- RedpandaConfig
- ClickhouseConfig
- LocalWebserverConfig
- ConsoleConfig
- The
serde
crate is used to serialize and deserialize Rust data structures.- https://docs.rs/serde/latest/serde/
- As far as I can tell it seems that the JSON format and TOML format is used with
serde
serialization.
- The Init method concludes by validating that the project repo path is indeed a Git repository.
- This is accomplished using the
git
crate inside of the utilities folder.- is_git_repo
- The repo's first commit is accomplished using the
crate::utilities::git::create_init_commit
function - Git functionality is made possible with the use of the
git2
crate.
- This is accomplished using the
-
Commands.Dev:
- Attempts to load a project file that should have been created during the
Commands.Init
phase. - Panics if "No project found, please run
moose init
to create a project" - Otherwise proceeds to create a
RoutineController
and adds two routines for execution:- RunLocalInfrastructure
- ValidateRedPandaCluster
- The
RunLocalInfrastructure
struct is defined instart.rs
- A
Routine
is implemented for it with arun_silent
function. - A long list of steps are defined:
- Create Internal Temp Directory Tree
- Validate Mount Volumes
- Create Docker Network
- Validate Panda House Network
- Run RedPanda Container
- Validate RedPanda Run
- Run Clickhouse Container
- Validate Clickhouse Run
- Run Console Container
- Validate Console Run
- If nothing panics then a success message is displayed the user stating that "Successfully ran local infrastructure"
- Each of the above steps are implemented / marked with the
Routine
trait and implements arun_slient()
handler.
- A
- The above routines are executed and lastly, the
routines::start_development_mode
method is invoked with a reference to the loaded project.- One of the first things that happens is that the
initialize_project_state
function is invoked with the directory of the project's schemas, a project struct and a mutable reference to a route_table. - The
process_schemas_in_dir
function recursively traverses the schema directory and uses theprocess_schema_file
method to process actual files.- The
process_schema_file
function is defined in theschema.rs
file. - process_objects() is utilized to create a topic in RedPanda and creates or replaces tables and kafka triggers which matches a table name.
- The
create_language_objects
function is called to create a TypeScript interface.- This creates an interface that web developers can use to avoid having to create an interface for their data models.
- The
process_schema_file
function continues to generate a TypeScript SDK. The SDK is then built using npm install, the package is then run, then the npm link --global command is executed to ensure the package is available for local dev use.- The process creates a symbolic link between a local package and a project. This is especially useful during development when you want to test changes to a package without having to publish it to npm first.
- The information is used to create a route entry which will later be passed to and used by the webserver.
- The
- One of the first things that happens is that the
- Attempts to load a project file that should have been created during the
-
Commands.Update:
- TBD
-
Commands.Stop:
- TBD
-
Commands.Clean:
- TBD
{in progress}