0xbe7a/envs.md

## envs.md

      
    Raw
  

              envs.md
            
          
    Proposal: Introduction of Feature Sets

Objective

The aim is to introduce a feature set mechanism in the pixi package manager. This mechanism will enable clear, conflict-free management of dependencies tailored to specific environments, while also maintaining the integrity of fixed lockfiles.
Motivating Example: Test Dependencies and Multiple Python Versions

Consider a scenario where a project needs to be tested across multiple Python versions, each requiring a different set of dependencies. In this case, defining separate feature sets for each Python version (like py39, py310, etc.) allows for easy switching between environments without conflicts. Similarly, for development purposes, a test feature set can include dependencies necessary for testing and linting, which are not required in the production environment.
Design Considerations


Non-Combinatorial: To ensure the dependency resolution process remains manageable, the solution should avoid a combinatorial explosion of dependency sets.
Single Feature Activation: The design should allow only one feature set to be active at any given time, simplifying the resolution process and preventing conflicts.
Fixed Lockfiles: It's crucial to preserve fixed lockfiles for consistency and predictability. Solutions must ensure reliability not just for authors but also for end-users, particularly at the time of lockfile creation.

Proposed Solution

Feature Set Definitions

Introduce feature sets in the pixi.toml configuration file, with each set comprising dependencies specific to a given environment or use case. For instance, a test feature set may include dependencies like pytest and pre-commit, essential for development but not for production.
pixi.toml Example:

[features]
test = ["pytest", "pre-commit"]
py39 = ["py39"]
py310 = ["py310"]

[dependencies]
requests = "*"
pytest = { version = ">= 1.2", optional = true }
pre-commit = { version = ">= 2", optional = true }
py39 = { package = "python", version = "3.9", optional = true}
py310 = { package = "python", version = "3.10", optional = true}
Lockfile Structure

Within the pixi.lock file, a package may now include an additional feature field, specifying the feature set to which it belongs. This structure ensures clarity and prevents unnecessary duplication of dependencies across different environments.
Feature Set Activation

Users can manually activate the desired feature set via command line or configuration. This approach guarantees a conflict-free environment by allowing only one feature set to be active at a time.
Command Configuration

Commands defined in pixi.toml can specify the supported feature sets. For example, a testing command can be linked to the test feature set, ensuring it runs with the correct dependencies.
Command Configuration Examples:

[tasks.test]
cmd = "pytest"
feature_set = ["test"]
[tasks.test]
cmd = "pytest"
feature_set = ["py39", "py310"]
Benefits


Simplicity: Clear and straightforward dependency management is achieved by making each feature set mutually exclusive.
Consistency: The solution upholds the principle of fixed lockfiles, ensuring stable and predictable dependency management across different project stages.

Drawbacks

In the proposed feature set mechanism, users can activate only one feature set at a time. This design decision simplifies the dependency resolution process and prevents conflicts. However, it can be limiting in certain scenarios:
Scenario with Orthogonal Feature Sets


Orthogonal feature sets are sets that could theoretically be combined because they don't interfere with each other. For example, one set might pertain to linting tools, while another might specify a Python version.
The limitation arises when a user wants to activate multiple such orthogonal sets simultaneously. For instance, they might want to run linting tools (lint feature set) under a specific Python version (py39 feature set).
In the current design, they would have to define a new feature set that combines both sets of dependencies. This becomes cumbersome if there are many such orthogonal sets, as it requires defining every possible combination.