IslamAzab

I'm hunting for the best solution on how to handle keeping large sets of DB records "sorted" in a performant manner.

Problem Description

Most of us have work on projects at some point where we have needed to have ordered lists of objects. Whether it be a to-do list sorted by priority, or a list of documents that a user can sort in whatever order they want.

A traditional approach for this on a Rails project is to use something like the acts_as_list gem, or something similar. These systems typically add some sort of "postion" or "sort order" column to each record, which is then used when querying out the records in a traditional order by position SQL query.

This approach seems to work fine for smaller datasets, but can be hard to manage on large data sets with hundreds (or thousands) of records needing to be sorted. Changing the sort position of even a single object will require updating every single record in the database that is in the same sort group. This requires potentially thousands of wri

IslamAzab

# http://guides.rubygems.org/command-reference/#gem-environment
--- 
gem: --no-document --verbose --backtrace
update_sources: true
backtrace: true

IslamAzab

-- POSTGRESQL check for duplicate primary keys due to
-- BUG #11141: Duplicate primary key values corruption
-- http://www.postgresql.org/message-id/20140811083748.2536.10437@wrigleys.postgresql.org

-- Select all tables (excluding schema_migrations table)
-- SELECT TABLE_NAME AS tablename
-- FROM information_schema.tables
-- WHERE table_schema = 'public'
--  AND TABLE_NAME != 'schema_migrations'
-- ORDER BY TABLE_NAME;

IslamAzab

[user]
        name =
        email =
[color]
        ui = true
[merge]
        log = true
        tool = meld
[alias]
        st = status -s -uno

IslamAzab

[1] pry(#<#<Class:0x000000068defe0>>)> quit-program
/home/myuser/.rvm/gems/ruby-2.0.0-p451@mygemset/gems/activesupport-3.2.18/lib/active_support/notifications.rb:125: [BUG] Segmentation fault
ruby 2.0.0p451 (2014-02-24 revision 45167) [x86_64-linux]

-- Control frame information -----------------------------------------------
c:0062 p:0002 s:0343 e:000342 EVAL   /home/myuser/.rvm/gems/ruby-2.0.0-p451@mygemset/gems/activesupport-3.2.18/lib/active_support/notifications.rb:125 [FINISH]
c:0061 p:---- s:0341 e:000340 CFUNC  :eval
c:0060 p:0010 s:0337 E:002598 BLOCK  /home/myuser/.rvm/gems/ruby-2.0.0-p451@mygemset/gems/better_errors-2.1.1/lib/better_errors/raised_exception.rb:37 [FINISH]
c:0059 p:---- s:0331 e:000330 CFUNC  :map

IslamAzab

sudo -i
apt-get update
apt-get install -y python-pip python-dev build-essential 
pip install psycopg2 
pip install psutil
pip install setuptools
git clone https://github.com/julmon/pg_activity.git 
cd pg_activity/
python setup.py install --with-man

IslamAzab

group :development do
  gem 'rspec-rails'
  gem 'factory_girl_rails'
  gem 'better_errors'
  gem 'binding_of_caller'
  gem 'pry-byebug'
  gem 'rack-mini-profiler'
  gem 'thin'
  gem 'marginalia'
  gem 'pry-rails'

IslamAzab

class DebugUtils
  def self.run_in_sql_transaction
    ActiveRecord::Base.transaction do
      yield
    end
  end

  def self.time_some_code
    time = Benchmark.realtime do
      yield

IslamAzab

#!/bin/bash

echo -e "\nbenchmark.sh -n<number of requests> -c<number of concurrency> <URL1> <URL2> ..."

echo -e "\nEx: benchmark.sh -n100 -c10 http://www.google.com/ http://www.bing.com/ \n"

## Gnuplot settings
echo "set terminal jpeg
set output 'benchmark_${1}_${2}.jpeg'
set title 'Benchmark: ${1} ${2}'

IslamAzab

# remove all merged branches except "develop" branch
git branch -r --merged | grep -v master | grep -v '^ develop$' | sed 's/origin\///' | xargs -n 1 git push --delete origin

# to remove them
git branch -r --merged | grep -v master | grep -v '^ development$' | sed -e 's/origin\///' | xargs -n 1 git branch --delete

# track all remote branches locally
for remote in `git branch -r | grep -v /HEAD`; do git checkout --track $remote ; done