public
Last active

  • Download Gist
RPS-SPEC
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74
THIS DOCUMENT MOVED TO http://chneukirchen.github.com/rps/
AND http://github.com/chneukirchen/rps
 
= Ruby Packaging Standard
 
The aim of this document is two-fold. First, to specify a common
structure of how a Ruby package distributed as source (that is, but
not limited to, development directories, version-controlled
repositories, .tar.gz, Gems, ...) should conform to.
 
Second, to document common and proven ways to structure Ruby packages,
and to point out certain anti-patterns that sneaked into common use.
It is by intent not to innovate.
 
(See RFC 2119 for use of MUST, SHOULD, SHALL.)
 
== Library files
 
Library code MUST reside in lib/.
 
Libraries SHOULD use a directory as namespace, e.g. lib/foo.rb and lib/foo/**.
 
Libraries SHOULD NOT require code of the project that are outside of lib/.
 
Libraries MUST not require 'rubygems' or modify the $LOAD_PATH.
 
Ruby library files MUST end with .rb.
 
Library files SHOULD be installed with mode 0644.
 
== Executables
 
Executables MUST reside in bin/.
 
Ruby executables SHOULD have a shebang line using env:
 
#!/usr/bin/env ruby
 
Executables SHOULD NOT require code of the project that are outside of lib/.
 
Executables SHOULD NOT require 'rubygems' or modify the $LOAD_PATH,
unless they are specifically made for doing that (e.g. package managers).
 
Executable files SHOULD NOT end with .rb.
 
Executable files SHOULD be installed with mode 0755.
 
== Extensions
 
Extensions are directories which contain a extconf.rb.
 
Extensions SHOULD reside in ext/.
 
Extensions SHOULD be buildable with "ruby extconf.rb; make".
 
XXX How to install, how to find out what was built.
 
== Data files
 
Data files and resources of the project belong to data/.
 
XXX Or data/<projectname>?
 
XXX Where to install to, how to find out that place from the program?
 
== Tests
 
Tests SHOULD reside in test/ or spec/.
 
== History
 
09apr2010: First initial draft.
10apr2010: Fix binary permissions.
10apr2010: Add data files.

Nice work.

Checkout the original setup.rb packaging recommendations. http://i.loveruby.net/en/projects/setup/doc/devel.html

I'd add a section for "man" and "data" too.

"Libraries MUST not require 'rubygems' or modify the $LOAD_PATH."

What about in "non-library" files like ./init.rb or /rails/init.rb that are not normally required except when treating the library as a plugin to a system?

Cool. Among other things, a standard document would be nice to have to hand off to developers new to Ruby.

How about something that correlates the library name with the lib directory structure?

Library: foo-bar
Directory: lib/foo/bar
Namespace: Foo::Bar

Library: foo_bar
Directory: lib/foo_bar
Namespace: FooBar

Is the permission inconsistency between lib/ being g-w and bin/ being g+w intentional?

This is great. Is the "XXX" under extensions about how extensions should be installed? make install is probably best but I've found problems with that because I can't tell what files are installed unless I do some kind of DESTDIR thing.

Can this specification also cover the packaging of static resources, such as templates. Many libraries place .erb templates within lib/, which to me is a big no-no. I always place static content in the static/ directory.

Libraries MUST not require 'rubygems' or modify the $LOAD_PATH.

This statement should be clarified just a little bit. What if the library has the explicit purpose of providing convenience methods for manipulating the $LOAD_PATH variable?

@jamesarosen Rails' init.rb are a bad idea. I'd just avoid them.

@rtomayko setup.rb called "ruby extconf.rb && make" in ext/ then copied those files to lib/yourarchdir. Rubygems sets RUBYARCHDIR and RUBYLIBDIR to lib/, then "make && make install" so the files are copied into the gem dir. I think thats a fair approach too. I'd recommend supporting "make" and "make install". The makefile extconf already provides both.

@postmodern Thats what "data/" is for.

+1

If everybody namespaced their libraries, the world would be a better place.

Signed, a person who wanted to name a model Twitter and use a Twitter library in the same project.

A project naming recommendation:

Projects should only contain underscores in their names like "sexp_processor".

If a project is an enhancement, plugin, extension, etc. for some other project it should contain a dash in the name between the original name and the project's name like "hoe-git" or a hypothetical "sexp_processor-pretty_print".

Why 0775 for bin files? This is incongruous with lib permissions of 0644. Bin files should be 0755.

See also: http://www.debian.org/doc/debian-policy/ch-files.html#s10.9

@drbrain That naming convention fits nicely with the current practice of prefixing libraries that have FFI bindings with ffi-.

a few things

  • no mention of ancillary files like README, NEWS, LICENSE ? Maybe a tiny little SHOULD clause?
  • where do config files go for an executable?
  • what is the preferred directory for sample code? I've seen plenty of /(ex|s)amples?/
  • test data goes in test/data/, test/fixtures or just fixtures/?

But good work thanks for sharing :)

@riffraff good point on the doc files

Which readme formats are allowed? README? README.md? README.txt? GitHub seems to be pushing us towards ones that allow some markup.

And what about the doc/ folder? My convention so far has been to put hand-written documentation in doc/ and generated documentation in doc/rdoc/, but I don't love it.

GNU Conventions:

COPYING http://www.gnu.org/prep/standards/standards.html#Releases
README http://www.gnu.org/prep/standards/standards.html#index-g_t_0040file_007bREADME_007d-file-144
NEWS http://www.gnu.org/prep/standards/standards.html#NEWS-File
ChangeLog http://www.gnu.org/prep/standards/standards.html#Change-Logs

The GNU packaging conventions have been long established (http://www.gnu.org/prep/standards/standards.html). Ruby core uses many of these conventions. We should be following the packaging model of ruby core itself not gems.

Checkout ruby's own file structure http://github.com/ruby/ruby

This is very good stuff, especially because your first draft is tightly focused. While there's a ton of stuff that could be added, I encourage you to avoid spending too much time talking about meta-files (README, ChangeLog, etc): They're not nearly as important as the basic library and extension guidelines you've already hit.

@jbarnette I agree. Though READMEs are a great thing, they don't fall under the "MUST" category. Its really annoying that autoconf barks at you if these files are missing.

Regarding setup.rb:

  • data/ is copied in #{PREFIX_DIR}/share (like /usr/share). Why not name it "share" then ?
  • conf/ is copied in /etc. Why not name it "etc" then ?
  • setup.rb multi-project layout is probably a bad idea. Package management should handle this

@zimbatm I agree about the multi-project layout thing :P

data/ is already an established convention:

>> Config.datadir("foo")
=> "/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/share/foo"

If the lib is a gem installed by rubygems, it returns the gemdir.

>> Config.datadir("rake")
=> "/Users/josh/.gem/ruby/1.8/gems/rake-0.8.7/data/rake"

@josh

Where is that defined?

@josh: good to know

I also propose the following sections:

== Metadata

The project SHOULD include it's ruby and system compile and runtime dependencies in a metadata file.

(whom format is to be defined, maybe .gemspec or something more general)

== setup.rb

The project SHOULD include setup.rb at the root if it's hierarchy

(ideally, setup.rb should just contain "require 'setup.rb'" and additional hooks)

@chneukirchen rubygems monkey patches rbconfig.rb :P

@zimbatm I don't think rubygem specific config such as "gemspec" belongs in this document.

@chneukirchen We do it in http://github.com/jbarnette/rubygems/blob/master/lib/rbconfig/datadir.rb

@zimbatm's suggestions are understandable, but I think they're feature creep. Pretty please keep this lean and pragmatic.

@jbarnette I'm curious how you think we can make "data/" and Config.datadir more standardized.

styling: s/Libraries MUST not/Libraries MUST NOT/

There's RbConfig::CONFIG['datadir'].

Please think of ways future package managers can control the datadir.

Personally, I'm ok with overriding Config.datadir.

Another option would be a fixed path relative to the lib/, but I think that's messy.

I’d like to see a one-liner clarification mentioning that vendor/ belongs under lib/ and maybe a comment about vendoring practices in general.

I also totally agree with @drbrain about having a standard library naming convention; it still drives me up the wall whenever I see libraries that CamelCase or hyphenate their names such that figuring out how to require them involves guesswork.

Regarding this rule:

Executables SHOULD NOT require 'rubygems' or modify the $LOAD_PATH,

What's wrong with modifying $LOAD_PATH in the executable? In my executables, I use a line at the top like this:

$LOAD_PATH.unshift File.expand_path('../../lib', __FILE__)
require 'my_library'

Otherwise, I cannot rely on the executable's shebang to run the executable because I need to specify ruby -I some_path/lib/ some_path/bin/my_executable or I need to remember to set $RUBYLIB to some_path/lib.

In my mind, when you run an executable, you know which executable you're running, and which library that executable belongs to. So why not make the computer understand your intention? It should Just Work! :-)

Also, I think of executables as a top-level entry point into your library's code. Nobody requires an executable, so there's no harm done.

sunaku: the issue with modifying the load path in that way is that you're assuming the library files are in a location relative to your executable. When your program is installed by a package manager like apt/dpkg, Rip, or setup.rb, the lib directory may be in an entirely different place. When installed with Rubygems, that relative path location is correct but rubygems installs wrapper executables that setup the load path for you, so you're just duplicating that effort.

All that being said, modifying the load path in the way you've described is mostly harmless in my estimation. Worst case is you end up with a duplicate or invalid load path entry. It's also nice to have executables work automatically when developing out of a checkout / working copy. I prefer not modifying the load path in executables, or using a slightly more stringent set of checks to make sure I'm modifying it correctly.

I think the SHOULD NOT language is the right thing in this case. From RFC 2119:

SHOULD NOT This phrase, or the phrase "NOT RECOMMENDED" mean that there may exist valid reasons in particular circumstances when the particular behavior is acceptable or even useful, but the full implications should be understood and the case carefully weighed before implementing any behavior described with this label.

The key is understanding how the code you gave as an example will act in various scenarios and then making a good judgement based on that understanding and the specific case.

The alteration of the $LOAD_PATH in order to exercise a project's command-line utilities seems like it's a problem that needs a better solution. If only there was something like bundle exec but for enabling lib/ or even ext/?

@postmodern, @sunaku: This is a problem that's very easily solved with aliases. It's not completely transparent, but the disadvantages are outweighed by the fact that your executables aren't making any assumptions about the relative directory structure. When I'm developing, I'm generally executing scripts from the root of the project directory, so I use something like this:

$ alias rb='ruby -rubygems -Ilib:test'
$ rb bin/johnson

It's also pretty darn easy to make a project-specific alias (using absolute paths) for any given executable as well.

@jbarnette I have this in my env

export RUBYOPT="-Ilib:test"

or

export RUBYOPT="-rubygems -Ilib:test"

@josh Absolutely, the only reason I don't have that turned on all the time is because I'm dumb and it's deeply confused me a couple of times when I've been in the root directory of a project that contributes, say, RubyGems plugins via the load path. :)

@jbarnette Aliases seem like a good compromise, but they don't solve the problem completely for me: I like to use the development versions of my projects right from my source area (e.g. I run ~/src/my_project/bin/my_project directly from anywhere in the file system).

I had to write a little wrapper script (could also be a shell function if you prefer) to address this:

> cat ~/bin/rubin
#!/bin/sh -x
dir="$(dirname "$(dirname "$1")")"
ruby -rubygems -I "$dir/lib:$dir/test" "$@"

And then run it this way:

rubin ~/src/my_project/bin/my_project

I guess I can live with this. :-/

@chneukirchen Your ruby-wrapper script is awesome! Totally solves my problem. Thank you.

I've only ever used RubyGems for packaging my projects, so I'm kind of scared of setup.rb's style of separating my project's directories (i.e. lib/ is no longer next to bin/ and so on). In particular, I am confused about the following:

I want to run the man executable (pointing it to my project's man/ directory) from my project's executable in bin/. With RubyGems, I can rely on bin/ being next to man/. But with setup.rb I don't know how to find my project's man/ directory from my project's executable in bin/!

Any suggestions? Thanks.

Similar to josh and jbarnette, I use this rbdev shell function to activate bin, lib, and ext dirs when I'm working on a project:

# develop out of the current directory.
rbdev() {
    local useful dir
    test -d ./bin && {
        useful=yes
        PATH="$(pwd)/bin:$PATH"
        echo "./bin on \$PATH"
    }
    test -d ./ext && {
        useful=yes
        for dir in ./ext/*
        do
            test -d "$dir" || continue
            RUBYLIB="$(pwd)/$dir:$RUBYLIB"
            export RUBYLIB
            echo "$dir on \$RUBYLIB"
        done
    }
    test -d ./lib && {
        useful=yes
        RUBYLIB="$(pwd)/lib:$RUBYLIB"
        export RUBYLIB
        echo "./lib on \$RUBYLIB"
    }
    test -z "$useful" && {
        echo "no ./lib or ./bin detected. help me help you."
        return 1
    }
}

When I'm working on e.g., RDiscount, I do something like:

$ cd ~/git/rdiscount
$ rbdev
./bin on $PATH
./ext on $RUBYLIB
./lib on $RUBYLIB

Not setting up the load path from executables has been a problem, though. People assume everything is broken when they get a LoadError + backtrace after checking out a project for the first time and running an executable. I really wish these techniques for setting up development environments would become common practice. I'm thinking a template HACKING file that explained basic environment prep would probably go a long way.

@chneukirchen Would you mind moving this discussion to ruby-talk? It's hard to quote and inline-reply to the various tangents here. And it's easy for questions to get lost in the noise. Also, I'm sure the vast number of Rubyists on ruby-talk would have valuable feedback on this spec. Thanks.

Please sign in to comment on this gist.

Something went wrong with that request. Please try again.