alexcrichton/foo.md Secret

## foo.md

      
    Raw
  

              foo.md
            
          
    @rfcbot fcp merge
I'd like to propose a subset of macros 2.0 story be stabilized as macros 1.2 for the Rust 1.28 release. Rust 1.28 enters nightly on May 10, 2018 (~2.5 weeks from this writing) and will become stable on August 2, 2018. I think that FCP may finish before the May 10 cutoff for 1.27 entering beta, but I'd like to hold off any stabilizations here until after that cutoff has happened and delay this to the 1.28 release.
This has been discussed on internals recently along with a number of issues registered by @petrochenkov and which should be fixed now (but not quite yet released in nightly). I think it'd be helpful to recap here so this is concretely the subset which would be stabilized.
Remember though that this is a subset. Functionality missing here does not mean it will never be stabilized or removed from the compiler. Rather this functionality will remain unstable after this proposed pass of stabilization to be stabilized at a later date.
Macros and the module system

Primarily covered by rust-lang/rust#35896 and now
having finished its FCP, the main idea is that you can use use statements to
import macros. For example code like this:
use some_proc_macro_crate::bar;

#[bar]
fn baz() {}
or
use some_proc_macro_crate::bar;
bar!();
or even
pub use some_proc_macro_crate::bar; // reexport an attribute or macro
This introduces a third namespace in Rust (in addition to the value/type
namespaces), the macro namespace. Attributes, macro_rules, and procedural
macros all reside in the maro namespace.
The difference from the full-blown module system is that only one-element
paths will be allowed to invoke
macros.
For example #[foo::bar] or ::bar::baz!() will be disallowed. This
restriction may be lifted one day but this is a good conservative route to
start with.
Where can expansion happen?

Attributes can only be applied to non-module
items.
"items" here includes things like trait items, impl items, and foreign module
items. Module expansion will not be stable yet due to the hygeine and
implementation ramifications. It's left to specify and stabilize this at a
later date.
Statements and expression attribute macros will not be stable yet. This is
primarily due to the real necessity for hygiene at the expression level (as
opposed to the item level). This is left to stabilize at a later date.
Finally, attribute macros must have arguments inside
delimiters.
For example #[foo], #[foo(bar)], and #[foo { bar baz ... @ | ^ hello }]
are valid invocations. Invocations like #[foo = "baz"], #[foo bar], or
#[foo ...  = ( baz )] will not be initially stable.
What do procedural macros look like?

Like custom derive, they're defined in proc-macro crate-type crates.
Procedural macros and attributes are defined like so:
extern crate proc_macro;
use proc_macro::TokenStream;

/// Invoked as `foo!()`
///
/// When invoked as `foo!(a b ( c ))` then the `TokenStream`
/// here will be `a b ( c )`.
///
/// The invocation is replaced with the `TokenStream` returned
#[proc_macro]
pub fn foo(a: TokenStream) -> TokenStream {
    // ...
}

/// Invoked as `#[bar]`
///
/// The first argument, `attr`, is the token stream inside of the attribute
/// itself. The second argument, `item`, is the token stream corresponding to
/// the item the attribute is attached to.
///
/// An attribute of the form `#[bar ( a b [ c ] )]` will have the `attr`
/// argument look like `a b [ c ]`. Note the lack of delimiters passed to
/// `attr`! An API may later be added to learn what delimiter a macro was
/// invoked with.
///
/// The `item` here is a tokenified version of the original item.
///
/// The return value here will contain all non-expanded attributes as well for
/// this attribute to inspect. The return value replaces the original item.
#[proc_macro]
pub fn bar(attr: TokenStream, item: TokenStream) -> TokenStream {
    // ...
}
What about hygiene?

Above it was seen that custom attributes and macros can only be expanded in
item contexts, notably only generating new AST nodes that are items. This
means that we only have to worry about the hygiene of generating new AST item
nodes.
New items will have the same hygiene as macro_rules! does today. They will
not be hygienic. New items added to the AST will enter the same namespace as
other items in the module.
The proc_macro API.

In order to enable all this the following surface area will be stabilized for
the proc_macro crate:
pub struct TokenStream(_);

impl TokenStream {
    pub fn empty() -> TokenStream;
    pub fn is_empty(&self) -> bool;
}

impl Clone for TokenStream { ... }
impl Debug for TokenStream { ... }
impl Display for TokenStream { ... }
impl FromStr for TokenStream { ... }
impl From<TokenTree> for TokenStream { ... }
impl FromIterator<TokenTree> for TokenStream { ... }
impl FromIterator<TokenStream> for TokenStream { ... }
impl !Send for TokenStream { ... }
impl !Sync for TokenStream { ... }

impl IntoIterator for TokenStream {
    type Item = TokenTree;
    type Iter = token_stream::IntoIter;
}

pub mod token_stream {
    pub struct IntoIter(_);

    impl Iterator for IntoIter {
        type Item = ::TokenTree;
    }
}

pub enum TokenTree {
    Op(Op),
    Term(Term),
    Literal(Literal),
    Group(Group),
}

impl TokenTree {
    pub fn span(&self) -> Span;
    pub fn set_span(&mut self, span: Span);
}

impl Clone for TokenTree { ... }
impl Debug for TokenTree { ... }
impl Display for TokenTree { ... }
impl From<Op> for TokenTree { ... }
impl From<Term> for TokenTree { ... }
impl From<Literal> for TokenTree { ... }
impl From<Group> for TokenTree { ... }
impl !Send for TokenTree { ... }
impl !Sync for TokenTree { ... }

pub struct Span(_);

impl Span {
    pub fn call_site() -> Span;
}

impl Clone for Span { ... }
impl Copy for Span { ... }
impl Debug for Span { ... }
impl !Send for Span { ... }
impl !Sync for Span { ... }

pub struct Group(_);

pub enum Delimiter {
    Parenthesis,
    Brace,
    Bracket,
    None,
}

impl Group {
    pub fn new(delimiter: Delimiter, stream: TokenStream) -> Group;
    pub fn stream(&self) -> TokenStream;
    pub fn delimiter(&self) -> Delimiter;
    pub fn span(&self) -> Span;
    pub fn set_span(&mut self, span: Span);
}

impl Clone for Group { ... }
impl Debug for Group { ... }
impl Display for Group { ... }
impl !Send for Group { ... }
impl !Sync for Group { ... }

impl Copy for Delimiter { ... }
impl Clone for Delimiter { ... }
impl Debug for Delimiter { ... }
impl PartialEq for Delimiter { ... }
impl Eq for Delimeter { ... }

pub struct Term(_);

impl Term {
    pub fn new(s: &str, span: Span) -> Term;
    pub fn span(&self) -> Span;
    pub fn set_span(&mut self, span: Span);
}

impl Copy for Term { ... }
impl Clone for Term { ... }
impl Debug for Term { ... }
impl Display for Term { ... }
impl !Send for Term { ... }
impl !Sync for Term { ... }

pub struct Op(_);

pub enum Spacing {
   Alone,
   Joint,
}

impl Op {
    pub fn new(op: char, spacing: Spacing) -> Op;
    pub fn op(&self) -> char;
    pub fn spacing(&self) -> Spacing;
    pub fn span(&self) -> Span;
    pub fn set_span(&mut self, span: Span);
}

impl Debug for Op { ... }
impl Display for Op { ... }
impl Clone for Op { ... }
impl Copy for Op { ... }
impl !Send for Op { ... }
impl !Sync for Op { ... }

impl Copy for Spacing { ... }
impl Clone for Spacing { ... }
impl Debug for Spacing { ... }
impl PartialEq for Spacing { ... }
impl Eq for Spacing { ... }

pub struct Literal(_);

impl Literal {
  // panic on infinity and NaN
  pub fn f{32,64}_{un,}suffixed(f: f{32,64}) -> Literal;

  pub fn i{8,16,32,64,128,size}_{un,}suffixed(n: i{8,16,32,64,128,size}) -> Literal;
  pub fn u{8,16,32,64,128,size}_{un,}suffixed(n: u{8,16,32,64,128,size}) -> Literal;

  pub fn string(s: &str) -> Literal;
  pub fn character(c: char) -> Literal;
  pub fn byte_string(b: &[u8]) -> Literal;

  pub fn span(&self) -> Span;
  pub fn set_span(&mut self, span: Span) -> Span;
}

impl Clone for Literal { ... }
impl Debug for Literal { ... }
impl Display for Literal { ... }
impl !Send for Literal { ... }
impl !Sync for Literal { ... }

More information about this API can be found online or in the original
PR
Testing strategy

The macros 1.1 and macros 2.0 systems have been extensively dogfooded throughout
the ecosystem for quite some time now. Notably this entire proposal is also
extensively tested through the 0.3 release of the proc-macro2 crate as
well as the syn crate. Throughout testing a number of bugs have been
identified and fixed and the current system feels solid enough to begin to
stabilized. (not to say it's bug-free!)