new developer wiki

New Developer Setup
READING

    You can find the old wiki dump, of which some of these pages are based off of, attached as a pdf at the bottom of this page.
    [new dev pages start around pg 170, additional tips at the end of the pdf... some info is outdated, so look here first]

    The Pragmatic Programmers have a few solid books that are considered essential reading within the ruby community:

        Programming Ruby 1.9 "The Pickaxe Book" (we aren't using 1.9 yet but moving our code from 1.8 to 1.9 is a priority) [ http://pragprog.com/titles/ruby3/programming-ruby-1-9 ]
        Agile Web Dev w/ Rails 4.0 (in beta right now & covers Rails 3.0 exclusively, which we aren't on.  The 3rd ed. is recommended for now.)

            [ http://pragprog.com/titles/rails3/agile-web-development-with-rails-third-edition ]

        Metaprogramming Ruby [ http://pragprog.com/titles/ppmetr/metaprogramming-ruby ]

    Other:

        All of the Addison-Wesley Professional Series come highly recommended but haven't gotten around to reading any myself.

            In Particular, The Ruby Way 2nd ed. has received high praise.

            [ http://www.pearsonhighered.com/educator/series/AddisonWesley-Professional-Ruby-Series/10582.page ]

        And finally for fun and to keep your toolset sharpened:

            Best of Ruby Quiz (few years old, so most of these tricks might be ruby 1.8 only) [ http://oreilly.com/catalog/9780976694076 ]

                Original Free & in Full Online version can be found at: [ http://rubyquiz.com/ ]


IDE Suggestions

    [OS X] Textmate [ http://macromates.com/ ]

        Additional Textmate Ruby Bundles [ http://code.leadmediapartners.com/ ] && [ http://adventuresincoding.com/2010/05/10-textmate-bundlesplugins-to-boost-your-ruby-on-rails-development-productivity/ ]

    [Unix] Gedit (recommended): 
    Making Gedit more like Textmate [ http://rbjl.net/22-rubybuntu-4-make-gedit-better-than-any-ide ]

        vim / 
        emacs (only recommended with prior experience)


Support Suggestions

    IRC    Getting Started: Dated but still true... http://oreilly.com/pub/a/javascript/2004/08/27/IRCtips.html, on OS X, Colloquy is nice

        Few useful rooms, rvm, rubberec2, passenger

        Performance Tip:  Type a letter or two and hit tab, to scroll through members in the room who start with those letters.

    Useful MAILING GROUPS

        WMLProgramming / Mongrel-Unicorn


Installation / Setup
0) Install Ruby / Required Gems / And Setup your PATH Variable

    [Ubuntu 9.10+10.4] [ http://rbjl.net/19-rubybuntu-1-installing-ruby-and-rails-on-ubuntu ]
    $ sudo gem install rubygems-update
    $ sudo gem update --system
    $ sudo gem sources --add http://gems.github.com
    $ sudo gem install 
    rails nettica mongrel mongrel_cluster fiveruns-memcache-client amazon-ec2 capistrano
      ruby-debug-ide RubyInline hoe ruby-debug open4 production_log_analyzer sml-log4r daemons mms2r
      sml-aws-s3 s3sync ruby-aws RedCloth libxml-ruby json oauth diff-lcs spicycode-rcov syntax exifr
      newrelic_rpm fakeweb mocha thoughtbot-shoulda rspec rspec-rails 3scale-3scale_ws_api_for_ruby
      sml-rest-client


1) Install the latest version of MySQL

    Binaries are available for most platforms but building it yourself is advised
    [OS X] The following page outlines all the steps of building MySQL properly [ http://hivelogic.com/articles/installing-mysql-on-mac-os-x ]

        Manual Start & Stop Commands

        sudo launchctl unload -w /Library/LaunchDaemons/com.mysql.mysqld.plist

        sudo launchctl load -w /Library/LaunchDaemons/com.mysql.mysqld.plist

    [Ubuntu 9.10]

        sudo apt-get install mysql-server libmysqlclient15-dev


2) Install RMagick

[Ubuntu]
sudo apt-get install imagemagick libmagickcore-dev libmagickwand-dev
sudo gem install rmagick
[Snow Leopard]
cd ~/src git clone git://github.com/masterkain/ImageMagick-sl.git cd ImageMagick-sl sh install_im.sh *sometimes the download fails so retry again sudo gem install rmagick

3) Create /etc/my.cnf file to set mysql settings... copy the following into the blank file

    [mysqld]
    bind-address = 127.0.0.1
    socket = /var/run/mysqld/mysqld.sock
    default-character-set = utf8
    max_allowed_packet = 16M
    log-slow-queries = /var/log/mysql-slow.log
    ; log queries taking longer than 5 seconds
    long_query_time = 5
    ; log queries that don't use indexes even if they take less than long_query_time
    log-queries-not-using-indexes

    [client]
    socket = /var/run/mysqld/mysqld.sock
    default-character-set = utf8


4) Install the MySQL Connector

    $ sudo mkdir -p /var/run/mysqld sudo
    $ sudo chown -R mysql:mysql /var/run/mysqld
    put the following line into ~/.bash_login create the file if necessary
    export PATH="/usr/local/bin:/usr/local/sbin:/usr/local/mysql/bin:$PATH"
    "touch" the file or reboot for it to take affect
    $ . ~/.bash_login
    Install the MySQL ruby gem
    [OSX <= Leopard]   $ sudo env ARCHFLAGS=”-arch i386″ gem install mysql — –with-mysql-config=/usr/local/mysql/bin/mysql_config
    [OSX Snow Leopard] $ sudo env ARCHFLAGS="-arch x86_64" gem install mysql — –with-mysql-config=/usr/local/mysql/bin/mysql_config


5) Create tables mtagd_test & mtagd_development

    $ mysql -uroot
    mysql> create database mtagd_test;
    mysql> create database mtagd_development;
    mysql> quit


6) Get & Install Keys for AWS

    Install the aws keys found attached below in your home directory.  The keys & source code are setup and expected to be found in your home dir.
    * NOTE:  The keys extract into hidden folders.  The dev keys folder is named   .ec2_smldev
    $ mv aws_keys.tgz ~/aws_keys.tgz    # move the archive to your home_dir
    $ cd ~    # move to your home_dir
    $ tar xzvf aws_keys.tgz    # un-tar the archive creating 3 hidden folders(directory name starts with a '.') 1 for each AWS account
    $ ls -al    # this command will display the directory contents including hidden files/directories and permission settings as well

    Install the ec2 utilities, also to your home directory [ http://developer.amazonwebservices.com/connect/entry.jspa?externalID=351&categoryID=88 ]
    Copy the following to your ~/.bash_profile file
    export EC2_HOME=~/ec2
    PATH=$PATH:$EC2_HOME/bin
    function ec2env {
        env=$1
        export EC2_KEYS_HOME=~/.ec2${env:+"_${env}"}
        . $EC2_KEYS_HOME/profile
    }
    ec2env smldev
    ssh-add -t 0 ~/.ec2_sml/mobyec2 &> /dev/null
    ssh-add -t 0 ~/.ec2_smldev/smldev &> /dev/null
    ssh-add -t 0 ~/.ec2_moby/moby &> /dev/null
    ssh-add -t 0 &> /dev/null

    6.1) Confirm the keys work by SSHing into each instance

    $ ssh root@staging.snap2twitter.com


7) Install Git

    [Ubuntu 9.10]    $ apt-get install git-core

    [OS X]    [ http://code.google.com/p/git-osx-installer/ ]

    7.1) Create Account on GitHub / Create & Register a Public Key with them / Contact the current admin of the SML account on GitHub to add you to the private repositories

        GitHub [ http://github.com ]
        SML @ GitHub [ http://github.com/sml ]

        [UNIX] http://help.github.com/linux-key-setup/

        [OS X] http://help.github.com/mac-key-setup/


8) Retrieve the Source from Git 

    * Note: The SML respository is the old codebase and thus should be used for educational purposes only
    *          Moving Forward the Silhouette (backend) & Fiji (frontend) are the new codebase for SML
    *          make sure to clone from the correct branch
       git clone -b <branch> <remote_repo>

      Example :  $ git clone -b new_wave git@github.com:sml/fiji.git ~/fiji

    Following commands install the SML repository into your home directory under a folder labeled 'sml', then recursively installs & updates each submodule listed within the hidden git file

    $ git clone git@github.com:sml/sml.git ~/sml
    $ cd ~/sml
    $ git submodule init
    $ git submodule update

Following commands install the Silhouette repository into your home directory.  If you only want SML, skip to step 9.

        $ cd ~
        $ git clone git@github.com:sml/silhouette.git silhouette

        $ git clone git@github.com:sml/silhouette-gatekeeper.git silhouette-gatekeeper
        ** edit the ~/silhouette-gatekeeper/database.yml
        replace silhouette_gatekeeper_development with silhouette_development
        replace silhouette_gatekeeper_test with silhouette_test

        $ git clone git@github.com:sml/silhouette-moderation-service.git silhouette-moderation-service

        $ git clone git@github.com:sml/silhouette-photo-service.git silhouette-photo-service

        $ git clone git@github.com:sml/silhouette-postmaster-general.git silhouette-postmaster-general

        $ git clone git@github.com:sml/silhouette-user-service.git silhouette-user-service
        $ git clone git@github.com:sml/silhouette-adapter-service.git silhouette-adapter-service
        $ git clone git@github.com:sml/silhouette-tiny-url-service.git silhouette-tiny-url-service

    In each silhoutte-* directory run the following commands
    [Ubuntu] may need to install one or more of the following packages for webrat to install properly

    $ apt-get install libxml2-dev libxslt1-dev libxml-ruby libxslt-ruby

    [ALL] not all will actually do anything

        $ git submodule init
        $ git submodule update

        $ sudo gem install haruska-ninja-decorators thoughtbot-factory_girl cucumber webrat

        $ sudo gem install sml-rest-client

        $ sudo gem install wr0ngway-rubber
        $ sudo gem install lumber oauth

    Setup the databases for S2T
    Make sure mysql server is running (check mysql widget in os x system preferences)

        $ mysql -u root -e "create database silhouette_development; create database silhouette_test"

        $ mysql -u root -e "create database silhouette_gatekeeper_development; create database silhouette_gatekeeper_test"

        $ mysql -u root -e "create database silhouette_user_development; create database silhouette_user_test"

    Install Sphinx & Thinking Sphinx, the following two pages give concise instructions on installing each on various platforms

        Installing Sphinx [ http://freelancing-god.github.com/ts/en/installing_sphinx.html ]

        Installing Thinking Sphinx [ http://freelancing-god.github.com/ts/en/installing_thinking_sphinx.html ]

    Run the following commands inside each silhouette-* directory, you may need to install a few additional gems

        $ mkdir log    # if directory isn't already present
        $ rake db:migrate

        $ rake

    Following commands install Fiji (aka S2T) to your home directory.

        $ git clone git@github.com:sml/fiji.git ~/fiji
        $ cd ~/fiji
        $ git submodule init
        $ git submodule update
        $ mysql -uroot -e "create database fiji_test; create database fiji_development;"

        **check for the presence of database.yml in fiji/config. Otherwise create it by  
                  adding
        # SQLite version 3.x
        # gem install sqlite3-ruby (not necessary on OS X Leopard)
        development:
        adapter: mysql
        encoding: utf8
        database: fiji_development
        reconnect: false
        pool: 5
        username: root
        password:
        socket: /var/run/mysqld/mysqld.sock

        # Warning: The database defined as "test" will be erased and
        # re-generated from your development database when you run "rake".
        # Do not set this db to the same as development or production.
        test: &TEST
        adapter: mysql
        database: fiji_test
        reconnect: false
        pool: 5
        username: root
        password:
        socket: /var/run/mysqld/mysqld.sock

        cucumber:
        <<: *TEST

        $ rake db:schema:load
        $ rake --trace
        < install whatever missing packages are listed >
        $ rake --trace     # wash and repeat


9) Setup your local database

    * Note: These scripts were for the original SML codebase only

    Load the latest DB Schema, !important! do not run rake db:migrate, the first time you setup the db

    $ cd ~/sml
    $ rake db:schema:load

    Run the following script from your project root (
    wait - this script takes quite a while to pull data)
    $ ./script/reset_dev_data.rb
    Setup your hosts file, run the following script from your project root as well
    $ sudo ./script/create_dev_hosts.rb


10) Test Your Setup

    10.1) Run the following script, from your project root, to start up a simple web server hosting the site

        $ ./script/server

        Then hit, [ http://localhost:3000 ] from any web browser

    10.2) Run rake to verify all the tests pass

        [ 4-29-2010, not all tests are passing ]
        [ integrations: all pass | units: 34 fail, 7 err | functionals: 290 fail, 0 err ]
        I suggest getting the latest 'gem list' from one of the running web server instances as a baseline for what you should have installed locally.  Then start rolling the gems back to find what upgrade is causing the issues.  Over 90% of the functional failures appear to be linked to the same issue.

        $ rake


11) Celebrate!