Skip to content

Instantly share code, notes, and snippets.

@tdelabro

tdelabro/no_std-guide.md

Last active June 14, 2024 02:58
Show Gist options
  • Save tdelabro/b2d1f2a0f94ceba72b718b92f9a7ad7b to your computer and use it in GitHub Desktop.
Save tdelabro/b2d1f2a0f94ceba72b718b92f9a7ad7b to your computer and use it in GitHub Desktop.
Download ZIP
How to easely port a crate to `no_std`?
Raw
no_std-guide.md

What is Rust's standard library?

One of the Rust programming language promises is "zero-cost abstraction". Therefore the language itself is pretty conservative about what it incorporates. Types that are basics in other languages, such as string, or features, such as async, cannot be part of the language itself because they are costly abstractions. The user may not need them at all, or he may prefer other alternative implementations. To make those types and functions available nonetheless, they are available as part of the Rust standard library, known as std. Part of this library, known as the prelude is imported by default in each .rs file so you don't have to repeat yourself too much.

Why would you want a no_std version of your crate?

Most of the time having this standard library available is not a problem because you run your code on a pretty standard environment: your own computer or a Linux server somewhere in the cloud. However, sometimes, you have to build for a more exotic environment, maybe a microcontroller, or web assembly. For those cases, you have to remove the standard library from your program. Indeed, because this library is designed for a more "classic" execution environment it assumes the existence of things such as a filesystem or a memory allocator. Therefore we want to get rid of std entirely and then manually import the parts we actually need and support.

How?

Examples

Here are two PR I did on existing repositories to add no_std support:

Give them a look to find everything explained bellow illustrated.

#![no_std]

If you add the #![no_std] crate directive at the top of your lib.rs or main.rs file, it will instruct the compiler to get rid of the std crate when it compiles. Instead, we can use the core crate, which contains some of the symbols present in std, but not all. Only the ones that are common to every compilation target.

Unfortunately, this is not enough. If we stopped there, our crate will only ever compile without the standard library. Yet we want to retain the ability to compile our code with the std lib if it's supported by the execution environment. We want to be able to compile both with and without the standard library. To do so we have to use Rust features.

Features

In your Cargo.toml create a feature section as follows:

[features]
default = ["std"]
std = []

And instead of using #![no_std] you will use:

#![cfg_attr(not(feature = "std"), no_std)]

This way, the std library is only removed from your binary if you are not compiling with the std feature. Otherwise, it will stay exactly the same.

Now, your crate compiles in two ways, with and without std. Without std:

cargo build --no-default-features

With std (leveraging the implicit default features):

cargo build

or if you want to be more explicit about the exact mix of features you are using:

cargo build --no-default-features --features std

Pro tip: By default, your IDE is probably configurated to use the default feature while running rust-analyzer. So you won't see anything changed or red at this point. In order to see what is broken when compiling without std, just remove std from the default feature.

At this point, trying to compile your crate without the std feature will probably fail and raise a ton of errors. But, before we fix your code, we frist need to fix the dependencies your code relies upon.

Dependencies

In your Cargo.toml you will have to express your intention not to use the std version of the crate you depend on. To do so, you will disable the default feature

[dependencies]
serde = "1.0.152"

will become:

[dependencies]
serde = { version = "1.0.152", default-features = false }

This way, you are only importing the slimmest version of your dependency, with everything accessory stripped away. And you have to hope that the crate maintainers have got the same idea as us: making their crate no_std compatible by putting everything relying on the standard library behind an std feature flag. If they didn't you have two options:

  • don't use this crate at all
  • fork it and do the whole process described in this article again, this time on a crate that is not yours (if you do so, please open a PR on the original repo so everybody can profit). It gets really tedious quite rapidly, that's why we should all be really consistent about always providing a no_std version of our crate ourselves, from the get-go.

Pro tip: features are not really well supported by rustdoc, so it's not always easy to know which features a crate offers if the maintainers didn't manually list them. The best way to find out is to read its Cargo.toml file.

At this point, you will have each one of your dependencies imported without std. Nonetheless, when you compile with the std feature, you may want to put those crates' std-guarded content back in place. If so, edit your std feature to look something like this:

std = ["serde/std"]

This way, when compiling with the std feature flag, the compiler will use the version of the dependency crates also compiled with the std flag.

Fixing your code

The last thing in order to be able to compile your crate in no_std, is to fix your crate code. You will face two main types of problems: broken imports and incompatible functionalities

Imports

When compiling without the std feature your code imports will break in two manners:

  • everything that was previously implicitly imported by the prelude won't be in scope anymore
  • any import that relies on the std crate won't be a valid import path anymore

Naive approach

If your crate is only a few files, you can manually fix every import.

Import, behind a feature flag, the symbols that were previously imported by the prelude:

#[cfg(not(feature = "std"))]
use core::boxed::Box;

And replace imports using std with imports using core.

use core::ops::BitOr;

Pro tip: here are three lints you can add to your crate to highlight the problematic imports:

Crate level approach

The naive approach has drawbacks. It is verbose and error-prone but also tedious to maintain if you have many files and an evolving codebase. To avoid those problems a good solution is to centralize the imports at the crate root level and then use everything from there. Nonetheless, it's not a silver bullet, and a lot of small and medium crates would be better off keeping it simple. It goes as follows.

Create a with_std.rs file where you re-export the stuff you need, from the standard library:

use std::{ops, any};

Create a without_std.rs file where you re-export the exact same stuff, but from the core crate this time:

use core::{ops, any};

In your crate root (lib.rs or main.rs) add the following lines:

#[cfg(feature = "std")]
include!("./with_std.rs");

#[cfg(not(feature = "std"))]
include!("./without_std.rs");

Now you can use those items anywhere in your crate, like this:

use crate::any::Any;

It will compile in both std and no_std mode, without having to use a cfg feature flag anywhere else.

If you feel like it, you can customize it to be a bit more expressive by doing something like this:

// without_std.rs
pub mod without_std {
	pub use core::{ops, any};
}

// with_std.rs
pub mod with_std {
	pub use std::{ops, any};
}

// lib.rs
mod stdlib {
    #[cfg(feature = "std")]
    pub use crate::with_std::*;
    #[cfg(not(feature = "std"))]
    pub use crate::without_std::*;
}

// any_other_file.rs
use crate::stdlib::any::Any;

Incompatible functionalities

There are things that you simply cannot do on no_std, such as interacting with the file system. Therefore anything that is related to the file system, is not part of the core crate and does not exist when compiling with no_std. Symbols like std::io::Reader or std::fs::read are simply not accessible.

There are two things to do: First, get rid of every no_std-incompatible symbol in the core of your crate logic. Then, add back no_std-incompatible functionalities behind the #[cfg(feature = "std")] flag.

Thus, anything io-related can be added back as optional, but convenient, functions, that can be used only in std mode, but are not part of the core crate logic.

alloc

Not all hardware comes with a memory allocator, but some do ( and web assembly does). In Rust, in order to use a heap, you will have to define a #[global_allocator]. It's a global data structure responsible for managing your program memory usage safely. It will give memory to your program when needed and take it back when not needed anymore, allowing you to create run-time growable data structures such as Vec, String, and HashMap.

There are multiple implementations of such memory allocators (tcmalloc, jemalloc, ...) and often there is a default one installed on the system which is common to all processes. If the target you are building for has such a thing, you can import the crate alloc in your codebase and benefit from all the cool things it offers. Here is how.

Vec, String, and co.

In Cargo.toml, add a new alloc feature:

[features]
default = ["std"]
std = []
alloc = []

If some of your dependencies also have an alloc feature you can include it too:

[features]
default = ["std"]
std = ["serde/std"]
alloc = ["serde/alloc"]

Create a with_alloc.rs file:

#[macro_use]
extern crate alloc;

use alloc::{string, vec, boxed};

Include it in without_std.rs, behind a feature flag:

#[cfg(feature = "alloc")]
include!("./with_alloc.rs");

Because Vec, String, etc are already present in the standard library we don't want to import them again from alloc in case we are also compiling with the std feature. Therefore if compiled with both std and alloc, the alloc feature will just be ignored.

HashMap

HashMap is not part of the alloc crate. You can use other collections instead such as alloc::collections::BTreeMap to achieve the same role, but if you really want to use HashMap there is still a way: the hashbrown crate.

Just add it to your dependencies in your Cargo.toml:

[dependencies]
hashbrown = "0.13.2"

This crate is by default no_std compatible, so there is no obligation to get rid of its default features.

Then edit your with_alloc.rs file:

#[macro_use]
extern crate alloc;

use alloc::{string, vec, rc};

pub mod collections {
	pub use hashbrown::{Hashmap, HashSet};
}

Along with your with_std.rs file:

use std::{ops, boxed};

pub mod collections {
	pub use std::collections::{HashMap, HashSet};
}

Now you can do the following in any rust file of your crate:

use crate::collections::HashMap;

Common problems

lib format

If you use the alloc crate and try to compile your crate as a staticlib, an error will be raised asking you to define a global allocator and a bunch of other symbols. My advice is the following: "don't". Compile to rlib, lib or dylib instead and everything will be fine. More on the subject of Rust code linkage here.

thiserror

thiserror is a great crate to derive the Error trait onto your own types. Our problem is that the Error trait is defined in std::error, which is not part of the core crate.

You can use thiserror_no_std instead, it's a wrapper crate over thiserror, but it sometimes has some strange behaviours.

In Cargo.toml:

[features]
std = ["thiserror-no-std/std"]

[dependencies]
# thiserror = "1.0.38" <- remove thiserror
thiserror-no-std = "2.0.2"

Use it in any Rust file:

use thiserror_no_std::Error;

snafu is another alternative that offers native no_std support.

But if you don't want any trouble, the best way is still to implement everything yourself. There are three you will have to implement on your Error type:

Pro tip: you can expand the macro to copy-paste the generated code directly and then adapt it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment