Skip to content

Instantly share code, notes, and snippets.

@KodrAus
Created January 25, 2019 01:09
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save KodrAus/a2add147a06624441c727e460b9c52a1 to your computer and use it in GitHub Desktop.
Save KodrAus/a2add147a06624441c727e460b9c52a1 to your computer and use it in GitHub Desktop.
Using cargo features to replace a set of concrete impls with a blanket one

Using cargo features to replace a set of concrete trait impls with a blanket one

There's subtlety involved in doing this so I've just dumped this out from another document in case I ever need to remember what they were in the future.

value::Visit

The Visit trait can be treated like a lightweight subset of serde::Serialize that can interoperate with serde, without necessarily depending on it. It can't be implemented manually:

/// A type that can be converted into a borrowed value.
pub trait Visit: private::Sealed {
    /// Visit this value.
    fn visit(&self, visitor: &mut dyn Visitor) -> Result<(), Error>;
    /// Convert a reference to this value into an erased `Value`.
    fn to_value(&self) -> Value
    where
        Self: Sized,
    {
        Value::new(self)
    }
}
mod private {
    #[cfg(not(feature = "kv_serde"))]
    pub trait Sealed: Debug {}
    #[cfg(feature = "kv_serde")]
    pub trait Sealed: Debug + Serialize {}
}

We'll look at the Visitor trait in more detail later.

Visit is the trait bound that structured values need to satisfy before they can be logged. The trait can't be implemented outside of the log crate, because it uses blanket implementations depending on Cargo features. If a crate defines a datastructure that users might want to log, instead of trying to implement Visit, it should implement the serde::Serialize and std::fmt::Debug traits. It shouldn't need to depend on the log crate at all. This means that Visit can piggyback off serde::Serialize as the pervasive public dependency, so that Visit itself doesn't need to be one.

The trait bounds on private::Sealed ensure that any generic T: Visit carries some additional traits that are needed for the blanket implementation of Serialize. As an example, any Option<T: Visit> can also be treated as Option<T: Serialize> and therefore implement Serialize itself. The Visit trait is responsible for a lot of type system mischief.

With default features, the types that implement Visit are a subset of T: Debug + Serialize:

  • Standard formats: Arguments
  • Primitives: bool, char
  • Unsigned integers: u8, u16, u32, u64, u128
  • Signed integers: i8, i16, i32, i64, i128
  • Strings: &str, String
  • Bytes: &[u8], Vec<u8>
  • Paths: &Path, PathBuf
  • Special types: Option<T>, &T, and ().

Enabling the kv_serde feature expands the set of types that implement Visit from this subset to all T: Debug + Serialize.

-------- feature = "kv_serde" --------
|                                    |
|        T: Debug + Serialize        |
|                                    |
|                                    |
|   - not(feature = "kv_serde") -    |
|   |                           |    |
|   | u8, u16, u32, u64, u128   |    |
|   | i8, i16, i32, i64, i128   |    |
|   | bool, char, &str, String  |    |
|   | &[u8], Vec<u8>            |    |
|   | &Path, PathBuf, Arguments |    |
|   | Option<T>, &T, ()         |    |
|   |                           |    |
|   -----------------------------    |
|                                    |
|                                    |
--------------------------------------

Visit without the kv_serde feature

Without the kv_serde feature, the Visit trait is implemented for a fixed set of fundamental types from the standard library:

impl Visit for u8 {
    fn visit(&self, visitor: &mut dyn Visitor) -> Result<(), Error> {
        visitor.visit_u64(*self as u64)
    }
}
impl Visit for u16 {
    fn visit(&self, visitor: &mut dyn Visitor) -> Result<(), Error> {
        visitor.visit_u64(*self as u64)
    }
}
impl Visit for u32 {
    fn visit(&self, visitor: &mut dyn Visitor) -> Result<(), Error> {
        visitor.visit_u64(*self as u64)
    }
}
impl Visit for u64 {
    fn visit(&self, visitor: &mut dyn Visitor) -> Result<(), Error> {
        visitor.visit_u64(*self)
    }
}
impl Visit for i8 {
    fn visit(&self, visitor: &mut dyn Visitor) -> Result<(), Error> {
        visitor.visit_i64(*self as i64)
    }
}
impl Visit for i16 {
    fn visit(&self, visitor: &mut dyn Visitor) -> Result<(), Error> {
        visitor.visit_i64(*self as i64)
    }
}
impl Visit for i32 {
    fn visit(&self, visitor: &mut dyn Visitor) -> Result<(), Error> {
        visitor.visit_i64(*self as i64)
    }
}
impl Visit for i64 {
    fn visit(&self, visitor: &mut dyn Visitor) -> Result<(), Error> {
        visitor.visit_i64(*self)
    }
}
impl Visit for f32 {
    fn visit(&self, visitor: &mut dyn Visitor) -> Result<(), Error> {
        visitor.visit_f64(*self as f64)
    }
}
impl Visit for f64 {
    fn visit(&self, visitor: &mut dyn Visitor) -> Result<(), Error> {
        visitor.visit_f64(*self)
    }
}
impl Visit for char {
    fn visit(&self, visitor: &mut dyn Visitor) -> Result<(), Error> {
        visitor.visit_char(*self)
    }
}
impl Visit for bool {
    fn visit(&self, visitor: &mut dyn Visitor) -> Result<(), Error> {
        visitor.visit_bool(*self)
    }
}
impl Visit for () {
    fn visit(&self, visitor: &mut dyn Visitor) -> Result<(), Error> {
        visitor.visit_none()
    }
}
#[cfg(feature = "i128")]
impl Visit for u128 {
    fn visit(&self, visitor: &mut dyn Visitor) -> Result<(), Error> {
        visitor.visit_u128(*self)
    }
}
#[cfg(feature = "i128")]
impl Visit for i128 {
    fn visit(&self, visitor: &mut dyn Visitor) -> Result<(), Error> {
        visitor.visit_i128(*self)
    }
}
impl<T> Visit for Option<T>
where
    T: Visit,
{
    fn visit(&self, visitor: &mut dyn Visitor) -> Result<(), Error> {
        match self {
            Some(v) => v.visit(visitor),
            None => visitor.visit_none(),
        }
    }
}
impl<'a> Visit for fmt::Arguments<'a> {
    fn visit(&self, visitor: &mut dyn Visitor) -> Result<(), Error> {
        visitor.visit_fmt(self)
    }
}
impl<'a> Visit for &'a str {
    fn visit(&self, visitor: &mut dyn Visitor) -> Result<(), Error> {
        visitor.visit_str(self)
    }
}
impl<'a> Visit for &'a [u8] {
    fn visit(&self, visitor: &mut dyn Visitor) -> Result<(), Error> {
        visitor.visit_bytes(self)
    }
}
impl<'v> Visit for Value<'v> {
    fn visit(&self, visitor: &mut dyn Visitor) -> Result<(), Error> {
        self.visit(visitor)
    }
}
#[cfg(feature = "std")]
impl<T: ?Sized> Visit for Box<T>
where
    T: Visit
{
    fn visit(&self, visitor: &mut dyn Visitor) -> Result<(), Error> {
        (**self).visit(visitor)
    }
}
#[cfg(feature = "std")]
impl<'a> Visit for &'a Path {
    fn visit(&self, visitor: &mut dyn Visitor) -> Result<(), Error> {
        match self.to_str() {
            Some(s) => visitor.visit_str(s),
            None => visitor.visit_fmt(&format_args!("{:?}", self)),
        }
    }
}
#[cfg(feature = "std")]
impl Visit for String {
    fn visit(&self, visitor: &mut dyn Visitor) -> Result<(), Error> {
        visitor.visit_str(&*self)
    }
}
#[cfg(feature = "std")]
impl Visit for Vec<u8> {
    fn visit(&self, visitor: &mut dyn Visitor) -> Result<(), Error> {
        visitor.visit_bytes(&*self)
    }
}
#[cfg(feature = "std")]
impl Visit for PathBuf {
    fn visit(&self, visitor: &mut dyn Visitor) -> Result<(), Error> {
        self.as_path().visit(visitor)
    }
}

Visit with the kv_serde feature

With the kv_serde feature, the Visit trait is implemented for any type that is Debug + Serialize:

#[cfg(feature = "kv_serde")]
impl<T: ?Sized> Visit for T
where
    T: Debug + Serialize {}

Ensuring the fixed set is a subset of the blanket implementation

Changing trait implementations based on Cargo features is a dangerous game. Cargo features are additive, so any observable changes to trait implementations must also be purely additive, otherwise you can end up with libraries that can't compile if a feature is active. This can be very subtle when references and generics are involved.

When the kv_serde feature is active, the implementation of Visit changes from a fixed set to an open one. We have to guarantee that the open set is a superset of the fixed one. That means any valid T: Visit without the kv_serde feature remains a valid T: Visit with the kv_serde feature.

There are a few ways we could achieve this, depending on the quality of the docs we want to produce.

For more readable documentation at the risk of incorrectly implementing Visit, we can use a private trait like EnsureVisit: Visit that is implemented alongside the concrete Visit trait regardless of any blanket implementations of Visit:

// The blanket implementaˀtion of `Visit` when `kv_serde` is enabled
#[cfg(feature = "kv_serde")]
impl<T: ?Sized> Visit for T where T: Debug + Serialize {}
/// This trait is a private implementation detail for testing.
/// 
/// All it does is make sure that our set of concrete types
/// that implement `Visit` always implement the `Visit` trait,
/// regardless of crate features and blanket implementations.
trait EnsureVisit: Visit {}
// Ensure any reference to a `Visit` implements `Visit`
impl<'a, T> EnsureVisit for &'a T where T: Visit {}
// These impl blocks always exists
impl<T> EnsureVisit for Option<T> where T: Visit {}
// This impl block only exists if the `kv_serde` isn't active
#[cfg(not(feature = "kv_serde"))]
impl<T> private::Sealed for Option<T> where T: Visit {}
#[cfg(not(feature = "kv_serde"))]
impl<T> Visit for Option<T> where T: Visit {
    fn visit(&self, visitor: &mut dyn Visitor) -> Result<(), Error> {
    }
}

In the above example, we can ensure that Option<T: Visit> always implements the Visit trait, whether it's done manually or as part of a blanket implementation. All types that implement Visit manually with any #[cfg] must also always implement EnsureVisit manually (with no #[cfg]) with the exact same type bounds. It's pretty subtle, but the subtlety can be localized to a single module within the log crate so it can be managed.

Using a trait for this type checking means the impl Visit for Option<T> and impl EnsureVisit for Option<T> can be wrapped up in a macro so that we never miss adding them. The below macro is an example of a (not very pretty) one that can add the needed implementations of EnsureVisit along with the regular Visit:

macro_rules! impl_to_value {
    () => {};
    (
        impl: { $($params:tt)* }
        where: { $($where:tt)* }
        $ty:ty: { $($serialize:tt)* }
        $($rest:tt)*
    ) => {
        impl<$($params)*> EnsureVisit for $ty
        where
            $($where)* {}
        
        #[cfg(not(feature = "kv_serde"))]
        impl<$($params)*> private::Sealed for $ty
        where
            $($where)* {}
        #[cfg(not(feature = "kv_serde"))]
        impl<$($params)*> Visit for $ty
        where
            $($where)*
        {
            $($serialize)*
        }
        impl_to_value!($($rest)*);
    };
    (
        impl: { $($params:tt)* }
        $ty:ty: { $($serialize:tt)* } 
        $($rest:tt)*
    ) => {
        impl_to_value! {
            impl: {$($params)*} where: {} $ty: { $($serialize)* } $($rest)*
        }
    };
    (
        $ty:ty: { $($serialize:tt)* } 
        $($rest:tt)*
    ) => {
        impl_to_value! {
            impl: {} where: {} $ty: { $($serialize)* } $($rest)*
        }
    }
}
// Ensure any reference to a `Visit` is also `Visit`
impl<'a, T> EnsureVisit for &'a T where T: Visit {}
impl_to_value! {
    u8: {
        fn visit(&self, visitor: &mut dyn Visitor) -> Result<(), Error> {
            visitor.visit_u64(*self as u64)
        }
    }
    impl: { T: Visit } Option<T>: {
        fn visit(&self, visitor: &mut dyn Visitor) -> Result<(), Error> {
            match self {
                Some(v) => v.to_value().visit(visitor),
                None => visitor.visit_none(),
            }
        }
    }
    ...
}

We don't necessarily need a macro to make new implementations accessible for new contributors safely though.

What about specialization?

In a future Rust with specialization we might be able to avoid all the machinery needed to keep the manual impls consistent with the blanket one, and allow consumers to implement Visit without needing serde. The specifics of specialization are still up in the air though. Under the proposed always applicable rule, manual implementations like impl<T> Visit for Option<T> where T: Visit wouldn't be allowed. The where specialize(T: Visit) scheme might make it possible though, although this would probably be a breaking change in any case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment