Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
THIS DOCUMENT MOVED TO http://chneukirchen.github.com/rps/
AND http://github.com/chneukirchen/rps
= Ruby Packaging Standard
The aim of this document is two-fold. First, to specify a common
structure of how a Ruby package distributed as source (that is, but
not limited to, development directories, version-controlled
repositories, .tar.gz, Gems, ...) should conform to.
Second, to document common and proven ways to structure Ruby packages,
and to point out certain anti-patterns that sneaked into common use.
It is by intent not to innovate.
(See RFC 2119 for use of MUST, SHOULD, SHALL.)
== Library files
Library code MUST reside in lib/.
Libraries SHOULD use a directory as namespace, e.g. lib/foo.rb and lib/foo/**.
Libraries SHOULD NOT require code of the project that are outside of lib/.
Libraries MUST not require 'rubygems' or modify the $LOAD_PATH.
Ruby library files MUST end with .rb.
Library files SHOULD be installed with mode 0644.
== Executables
Executables MUST reside in bin/.
Ruby executables SHOULD have a shebang line using env:
#!/usr/bin/env ruby
Executables SHOULD NOT require code of the project that are outside of lib/.
Executables SHOULD NOT require 'rubygems' or modify the $LOAD_PATH,
unless they are specifically made for doing that (e.g. package managers).
Executable files SHOULD NOT end with .rb.
Executable files SHOULD be installed with mode 0755.
== Extensions
Extensions are directories which contain a extconf.rb.
Extensions SHOULD reside in ext/.
Extensions SHOULD be buildable with "ruby extconf.rb; make".
XXX How to install, how to find out what was built.
== Data files
Data files and resources of the project belong to data/.
XXX Or data/<projectname>?
XXX Where to install to, how to find out that place from the program?
== Tests
Tests SHOULD reside in test/ or spec/.
== History
09apr2010: First initial draft.
10apr2010: Fix binary permissions.
10apr2010: Add data files.
@postmodern
Copy link

postmodern commented Apr 10, 2010

@drbrain That naming convention fits nicely with the current practice of prefixing libraries that have FFI bindings with ffi-.

@riffraff
Copy link

riffraff commented Apr 10, 2010

a few things

  • no mention of ancillary files like README, NEWS, LICENSE ? Maybe a tiny little SHOULD clause?
  • where do config files go for an executable?
  • what is the preferred directory for sample code? I've seen plenty of /(ex|s)amples?/
  • test data goes in test/data/, test/fixtures or just fixtures/?

But good work thanks for sharing :)

@jamesarosen
Copy link

jamesarosen commented Apr 10, 2010

@riffraff good point on the doc files

Which readme formats are allowed? README? README.md? README.txt? GitHub seems to be pushing us towards ones that allow some markup.

And what about the doc/ folder? My convention so far has been to put hand-written documentation in doc/ and generated documentation in doc/rdoc/, but I don't love it.

@josh
Copy link

josh commented Apr 10, 2010

GNU Conventions:

COPYING http://www.gnu.org/prep/standards/standards.html#Releases
README http://www.gnu.org/prep/standards/standards.html#index-g_t_0040file_007bREADME_007d-file-144
NEWS http://www.gnu.org/prep/standards/standards.html#NEWS-File
ChangeLog http://www.gnu.org/prep/standards/standards.html#Change-Logs

The GNU packaging conventions have been long established (http://www.gnu.org/prep/standards/standards.html). Ruby core uses many of these conventions. We should be following the packaging model of ruby core itself not gems.

Checkout ruby's own file structure http://github.com/ruby/ruby

@jbarnette
Copy link

jbarnette commented Apr 10, 2010

This is very good stuff, especially because your first draft is tightly focused. While there's a ton of stuff that could be added, I encourage you to avoid spending too much time talking about meta-files (README, ChangeLog, etc): They're not nearly as important as the basic library and extension guidelines you've already hit.

@josh
Copy link

josh commented Apr 10, 2010

@jbarnette I agree. Though READMEs are a great thing, they don't fall under the "MUST" category. Its really annoying that autoconf barks at you if these files are missing.

@zimbatm
Copy link

zimbatm commented Apr 10, 2010

Regarding setup.rb:

  • data/ is copied in #{PREFIX_DIR}/share (like /usr/share). Why not name it "share" then ?
  • conf/ is copied in /etc. Why not name it "etc" then ?
  • setup.rb multi-project layout is probably a bad idea. Package management should handle this

@josh
Copy link

josh commented Apr 10, 2010

@zimbatm I agree about the multi-project layout thing :P

data/ is already an established convention:

>> Config.datadir("foo")
=> "/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/share/foo"

If the lib is a gem installed by rubygems, it returns the gemdir.

>> Config.datadir("rake")
=> "/Users/josh/.gem/ruby/1.8/gems/rake-0.8.7/data/rake"

@leahneukirchen
Copy link
Author

leahneukirchen commented Apr 10, 2010

@josh

Where is that defined?

@zimbatm
Copy link

zimbatm commented Apr 10, 2010

@josh: good to know

I also propose the following sections:

== Metadata

The project SHOULD include it's ruby and system compile and runtime dependencies in a metadata file.

(whom format is to be defined, maybe .gemspec or something more general)

== setup.rb

The project SHOULD include setup.rb at the root if it's hierarchy

(ideally, setup.rb should just contain "require 'setup.rb'" and additional hooks)

@josh
Copy link

josh commented Apr 10, 2010

@chneukirchen rubygems monkey patches rbconfig.rb :P

@zimbatm I don't think rubygem specific config such as "gemspec" belongs in this document.

@jbarnette
Copy link

jbarnette commented Apr 10, 2010

@chneukirchen We do it in http://github.com/jbarnette/rubygems/blob/master/lib/rbconfig/datadir.rb

@zimbatm's suggestions are understandable, but I think they're feature creep. Pretty please keep this lean and pragmatic.

@josh
Copy link

josh commented Apr 10, 2010

@jbarnette I'm curious how you think we can make "data/" and Config.datadir more standardized.

@dubek
Copy link

dubek commented Apr 10, 2010

styling: s/Libraries MUST not/Libraries MUST NOT/

@rtomayko
Copy link

rtomayko commented Apr 10, 2010

There's RbConfig::CONFIG['datadir'].

@leahneukirchen
Copy link
Author

leahneukirchen commented Apr 10, 2010

Please think of ways future package managers can control the datadir.

Personally, I'm ok with overriding Config.datadir.

Another option would be a fixed path relative to the lib/, but I think that's messy.

@edward
Copy link

edward commented Apr 11, 2010

I’d like to see a one-liner clarification mentioning that vendor/ belongs under lib/ and maybe a comment about vendoring practices in general.

I also totally agree with @drbrain about having a standard library naming convention; it still drives me up the wall whenever I see libraries that CamelCase or hyphenate their names such that figuring out how to require them involves guesswork.

@sunaku
Copy link

sunaku commented Apr 17, 2010

Regarding this rule:

Executables SHOULD NOT require 'rubygems' or modify the $LOAD_PATH,

What's wrong with modifying $LOAD_PATH in the executable? In my executables, I use a line at the top like this:

$LOAD_PATH.unshift File.expand_path('../../lib', __FILE__)
require 'my_library'

Otherwise, I cannot rely on the executable's shebang to run the executable because I need to specify ruby -I some_path/lib/ some_path/bin/my_executable or I need to remember to set $RUBYLIB to some_path/lib.

In my mind, when you run an executable, you know which executable you're running, and which library that executable belongs to. So why not make the computer understand your intention? It should Just Work! :-)

Also, I think of executables as a top-level entry point into your library's code. Nobody requires an executable, so there's no harm done.

@rtomayko
Copy link

rtomayko commented Apr 17, 2010

sunaku: the issue with modifying the load path in that way is that you're assuming the library files are in a location relative to your executable. When your program is installed by a package manager like apt/dpkg, Rip, or setup.rb, the lib directory may be in an entirely different place. When installed with Rubygems, that relative path location is correct but rubygems installs wrapper executables that setup the load path for you, so you're just duplicating that effort.

All that being said, modifying the load path in the way you've described is mostly harmless in my estimation. Worst case is you end up with a duplicate or invalid load path entry. It's also nice to have executables work automatically when developing out of a checkout / working copy. I prefer not modifying the load path in executables, or using a slightly more stringent set of checks to make sure I'm modifying it correctly.

I think the SHOULD NOT language is the right thing in this case. From RFC 2119:

SHOULD NOT This phrase, or the phrase "NOT RECOMMENDED" mean that there may exist valid reasons in particular circumstances when the particular behavior is acceptable or even useful, but the full implications should be understood and the case carefully weighed before implementing any behavior described with this label.

The key is understanding how the code you gave as an example will act in various scenarios and then making a good judgement based on that understanding and the specific case.

@postmodern
Copy link

postmodern commented Apr 18, 2010

The alteration of the $LOAD_PATH in order to exercise a project's command-line utilities seems like it's a problem that needs a better solution. If only there was something like bundle exec but for enabling lib/ or even ext/?

@leahneukirchen
Copy link
Author

leahneukirchen commented Apr 18, 2010

@jbarnette
Copy link

jbarnette commented Apr 18, 2010

@postmodern, @sunaku: This is a problem that's very easily solved with aliases. It's not completely transparent, but the disadvantages are outweighed by the fact that your executables aren't making any assumptions about the relative directory structure. When I'm developing, I'm generally executing scripts from the root of the project directory, so I use something like this:

$ alias rb='ruby -rubygems -Ilib:test'
$ rb bin/johnson

It's also pretty darn easy to make a project-specific alias (using absolute paths) for any given executable as well.

@josh
Copy link

josh commented Apr 18, 2010

@jbarnette I have this in my env

export RUBYOPT="-Ilib:test"

or

export RUBYOPT="-rubygems -Ilib:test"

@jbarnette
Copy link

jbarnette commented Apr 18, 2010

@josh Absolutely, the only reason I don't have that turned on all the time is because I'm dumb and it's deeply confused me a couple of times when I've been in the root directory of a project that contributes, say, RubyGems plugins via the load path. :)

@sunaku
Copy link

sunaku commented Apr 18, 2010

@jbarnette Aliases seem like a good compromise, but they don't solve the problem completely for me: I like to use the development versions of my projects right from my source area (e.g. I run ~/src/my_project/bin/my_project directly from anywhere in the file system).

I had to write a little wrapper script (could also be a shell function if you prefer) to address this:

> cat ~/bin/rubin
#!/bin/sh -x
dir="$(dirname "$(dirname "$1")")"
ruby -rubygems -I "$dir/lib:$dir/test" "$@"

And then run it this way:

rubin ~/src/my_project/bin/my_project

I guess I can live with this. :-/

@sunaku
Copy link

sunaku commented Apr 19, 2010

@chneukirchen Your ruby-wrapper script is awesome! Totally solves my problem. Thank you.

@sunaku
Copy link

sunaku commented Apr 19, 2010

I've only ever used RubyGems for packaging my projects, so I'm kind of scared of setup.rb's style of separating my project's directories (i.e. lib/ is no longer next to bin/ and so on). In particular, I am confused about the following:

I want to run the man executable (pointing it to my project's man/ directory) from my project's executable in bin/. With RubyGems, I can rely on bin/ being next to man/. But with setup.rb I don't know how to find my project's man/ directory from my project's executable in bin/!

Any suggestions? Thanks.

@rtomayko
Copy link

rtomayko commented Apr 19, 2010

Similar to josh and jbarnette, I use this rbdev shell function to activate bin, lib, and ext dirs when I'm working on a project:

# develop out of the current directory.
rbdev() {
    local useful dir
    test -d ./bin && {
        useful=yes
        PATH="$(pwd)/bin:$PATH"
        echo "./bin on \$PATH"
    }
    test -d ./ext && {
        useful=yes
        for dir in ./ext/*
        do
            test -d "$dir" || continue
            RUBYLIB="$(pwd)/$dir:$RUBYLIB"
            export RUBYLIB
            echo "$dir on \$RUBYLIB"
        done
    }
    test -d ./lib && {
        useful=yes
        RUBYLIB="$(pwd)/lib:$RUBYLIB"
        export RUBYLIB
        echo "./lib on \$RUBYLIB"
    }
    test -z "$useful" && {
        echo "no ./lib or ./bin detected. help me help you."
        return 1
    }
}

When I'm working on e.g., RDiscount, I do something like:

$ cd ~/git/rdiscount
$ rbdev
./bin on $PATH
./ext on $RUBYLIB
./lib on $RUBYLIB

Not setting up the load path from executables has been a problem, though. People assume everything is broken when they get a LoadError + backtrace after checking out a project for the first time and running an executable. I really wish these techniques for setting up development environments would become common practice. I'm thinking a template HACKING file that explained basic environment prep would probably go a long way.

@sunaku
Copy link

sunaku commented Apr 20, 2010

@chneukirchen Would you mind moving this discussion to ruby-talk? It's hard to quote and inline-reply to the various tangents here. And it's easy for questions to get lost in the noise. Also, I'm sure the vast number of Rubyists on ruby-talk would have valuable feedback on this spec. Thanks.

@leahneukirchen
Copy link
Author

leahneukirchen commented Apr 20, 2010

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment