Skip to content

Instantly share code, notes, and snippets.

@TylerRick
Last active August 29, 2015 13:57
Show Gist options
  • Save TylerRick/9811465 to your computer and use it in GitHub Desktop.
Save TylerRick/9811465 to your computer and use it in GitHub Desktop.
Canvas installation errors
Following https://github.com/instructure/canvas-lms/wiki/Quick-Start I ran into some problems. Here's what I did to work around them.
I am installing on my development computer, running Ubuntu 13.10.
dpkg --get-selections | egrep -i "postgresql"
postgresql install
postgresql-9.1 install
postgresql-9.1-postgis deinstall
postgresql-client-9.1 install
postgresql-client-common install
postgresql-common install
Minor problem: Errors from Gemfile
==================================
> cd canvas
RVM used your Gemfile for selecting Ruby, it is all fine - Heroku does that too,
you can ignore these warnings with 'rvm rvmrc warning ignore canvas/Gemfile'.
To ignore the warning for all files run 'rvm rvmrc warning ignore allGemfiles'.
sed: -e expression #1, char 45: unterminated `s' command
ruby-1.9.3,:engine=>ruby,:engine_version=>1.9.3 is not installed.
To install do: 'rvm install ruby-1.9.3,:engine=>ruby,:engine_version=>1.9.3'
> rvm install ruby-1.9.3
> rvm use ruby-1.9.3-p545
Looks like it's already on the stable branch by default:
> git rev-parse origin/stable
6058f6b8d4826ded72b2e5894ff0974761cb476f
> git rev-parse HEAD
6058f6b8d4826ded72b2e5894ff0974761cb476f
> bundle install --without mysql
> for config in amazon_s3 delayed_jobs domain file_store outgoing_mail security scribd external_migration; do cp config/$config.yml.example config/$config.yml; done
> cp config/database.yml.example config/database.yml
> createdb canvas_development
> createdb canvas_queue_development
Problem: rake db:initial_setup fails, apparently because remove_index in migrations is not having any effect?!
==============================================================================================================
> bundle exec rake db:initial_setup
== InitCanvasDb: migrating ===================================================
== AddStickyXlisting: migrating ==============================================
-- add_column(:course_sections, :sticky_xlist, :boolean)
-> 0.0008s
== AddStickyXlisting: migrated (0.0010s) =====================================
== OptimizeDelayedJobs: migrating ============================================
-- create_table(:failed_jobs)
NOTICE: CREATE TABLE will create implicit sequence "failed_jobs_id_seq" for serial column "failed_jobs.id"
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "failed_jobs_pkey" for table "failed_jobs"
-> 0.0177s
-- remove_index(:delayed_jobs, {:name=>"get_delayed_jobs_index"})
-> 0.0020s
-- remove_index(:delayed_jobs, [:strand])
-> 0.0015s
-- add_index(:delayed_jobs, ["run_at", "queue", "locked_at", "strand", "priority"], {:name=>"index_delayed_jobs_for_get_next"})
-> 0.0074s
-- add_index(:delayed_jobs, ["strand", "id"], {:name=>"index_delayed_jobs_on_strand"})
rake aborted!
An error has occurred, this and all later migrations canceled:
PG::Error: ERROR: relation "index_delayed_jobs_on_strand" already exists
: CREATE INDEX "index_delayed_jobs_on_strand" ON "delayed_jobs" ("strand", "id")/home/tyler/.rvm/gems/ruby-1.9.3-p545/bundler/gems/rails-e86daf8ff727/activerecord/lib/active_record/connection_adapters/abstract_adapter.rb:227:in `rescue in log'
/home/tyler/.rvm/gems/ruby-1.9.3-p545/bundler/gems/rails-e86daf8ff727/activerecord/lib/active_record/connection_adapters/abstract_adapter.rb:204:in `log'
/home/tyler/.rvm/gems/ruby-1.9.3-p545/bundler/gems/rails-e86daf8ff727/activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb:520:in `execute'
canvas/config/initializers/active_record.rb:1359:in `add_index'
/home/tyler/.rvm/gems/ruby-1.9.3-p545/bundler/gems/rails-e86daf8ff727/activerecord/lib/active_record/migration.rb:352:in `block in method_missing'
/home/tyler/.rvm/gems/ruby-1.9.3-p545/bundler/gems/rails-e86daf8ff727/activerecord/lib/active_record/migration.rb:328:in `block in say_with_time'
/home/tyler/.rvm/gems/ruby-1.9.3-p545/bundler/gems/rails-e86daf8ff727/activerecord/lib/active_record/migration.rb:328:in `say_with_time'
/home/tyler/.rvm/gems/ruby-1.9.3-p545/bundler/gems/rails-e86daf8ff727/activerecord/lib/active_record/migration.rb:348:in `method_missing'
canvas/db/migrate/20110610213249_optimize_delayed_jobs.rb:29:in `up'
canvas (stable) > bundle exec rake db:initial_setup
rake aborted!
PG::Error: ERROR: relation "schema_migrations" already exists
: CREATE TABLE "schema_migrations" ("version" character varying(255) NOT NULL)
/home/tyler/.rvm/gems/ruby-1.9.3-p545/bundler/gems/rails-e86daf8ff727/activerecord/lib/active_record/connection_adapters/abstract_adapter.rb:227:in `rescue in log'
/home/tyler/.rvm/gems/ruby-1.9.3-p545/bundler/gems/rails-e86daf8ff727/activerecord/lib/active_record/connection_adapters/abstract_adapter.rb:204:in `log'
/home/tyler/.rvm/gems/ruby-1.9.3-p545/bundler/gems/rails-e86daf8ff727/activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb:520:in `execute'
/home/tyler/.rvm/gems/ruby-1.9.3-p545/bundler/gems/rails-e86daf8ff727/activerecord/lib/active_record/connection_adapters/abstract/schema_statements.rb:109:in `create_table'
/home/tyler/.rvm/gems/ruby-1.9.3-p545/bundler/gems/rails-e86daf8ff727/activerecord/lib/active_record/connection_adapters/abstract/schema_statements.rb:371:in `initialize_schema_migrations_table'
/home/tyler/.rvm/gems/ruby-1.9.3-p545/bundler/gems/rails-e86daf8ff727/activerecord/lib/active_record/migration.rb:441:in `initialize'
/home/tyler/.rvm/gems/ruby-1.9.3-p545/bundler/gems/rails-e86daf8ff727/activerecord/lib/active_record/migration.rb:401:in `new'
/home/tyler/.rvm/gems/ruby-1.9.3-p545/bundler/gems/rails-e86daf8ff727/activerecord/lib/active_record/migration.rb:401:in `up'
/home/tyler/.rvm/gems/ruby-1.9.3-p545/bundler/gems/rails-e86daf8ff727/activerecord/lib/active_record/migration.rb:383:in `migrate'
/home/tyler/.rvm/gems/ruby-1.9.3-p545/bundler/gems/rails-e86daf8ff727/railties/lib/tasks/databases.rake:112:in `block (2 levels) in <top (required)>'
/home/tyler/.rvm/gems/ruby-1.9.3-p545/bin/ruby_executable_hooks:15:in `eval'
/home/tyler/.rvm/gems/ruby-1.9.3-p545/bin/ruby_executable_hooks:15:in `<main>'
Tasks: TOP => db:initial_setup => db:migrate
(See full trace by running task with --trace)
Yet schema_migrations table *does* exist:
> ./script/dbconsole
canvas_development=# select count(version) from schema_migrations;
count
-------
60
(1 row)
> \d
shows that it is in the 'public' schema.
I edited activerecord/lib/active_record/connection_adapters/abstract/schema_statements.rb:371 and added some debugging info:
--- a/activerecord/lib/active_record/connection_adapters/abstract/schema_statements.rb
+++ b/activerecord/lib/active_record/connection_adapters/abstract/schema_statements.rb
@@ -367,6 +367,7 @@ module ActiveRecord
def initialize_schema_migrations_table
sm_table = ActiveRecord::Migrator.schema_migrations_table_name
+ puts %(tables=#{(tables).inspect})
unless tables.detect { |t| t == sm_table }
create_table(sm_table, :id => false) do |schema_migrations_table|
schema_migrations_table.column :version, :string, :null => false
And it outputs this:
tables=[]
http://www.redmine.org/boards/2/topics/6051 suggested adding a schema_search_path: public to my config/database.yml.
development:
adapter: postgresql
encoding: utf8
database: canvas_development
schema_search_path: public
When I did that, it was able to see/list all the tables, and found the schema_migrations table.
But now I'm back to getting this error:
> bundle exec rake db:initial_setup
== OptimizeDelayedJobs: migrating ============================================
-- create_table(:failed_jobs)
NOTICE: CREATE TABLE will create implicit sequence "failed_jobs_id_seq" for serial column "failed_jobs.id"
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "failed_jobs_pkey" for table "failed_jobs"
-> 0.0312s
-- remove_index(:delayed_jobs, {:name=>"get_delayed_jobs_index"})
-> 0.0058s
-- remove_index(:delayed_jobs, [:strand])
-> 0.0023s
-- add_index(:delayed_jobs, ["run_at", "queue", "locked_at", "strand", "priority"], {:name=>"index_delayed_jobs_for_get_next"})
-> 0.0098s
-- add_index(:delayed_jobs, ["strand", "id"], {:name=>"index_delayed_jobs_on_strand"})
rake aborted!
An error has occurred, this and all later migrations canceled:
PG::Error: ERROR: relation "index_delayed_jobs_on_strand" already exists
: CREATE INDEX "index_delayed_jobs_on_strand" ON "delayed_jobs" ("strand", "id")/home/tyler/.rvm/gems/ruby-1.9.3-p545/bundler/gems/rails-e86daf8ff727/activerecord/lib/active_record/connection_adapters/abstract_adapter.rb:227:in `rescue in log'
> psql canvas_queue_development
canvas_queue_development=# \d delayed_jobs
Table "public.delayed_jobs"
Column | Type | Modifiers
--------------+-----------------------------+-----------------------------------------------------------
id | bigint | not null default nextval('delayed_jobs_id_seq'::regclass)
strand | character varying(255) |
Indexes:
"delayed_jobs_pkey" PRIMARY KEY, btree (id)
"delayed_jobs_priority" btree (priority, run_at)
"delayed_jobs_queue" btree (queue)
"get_delayed_jobs_index" btree (priority, run_at, failed_at NULLS FIRST, locked_at NULLS FIRST, queue)
"index_delayed_jobs_on_strand" btree (strand)
"index_delayed_jobs_on_tag" btree (tag)
Maybe this line was supposed to remove that index?
remove_index :delayed_jobs, [:strand]
--- a/db/migrate/20110610213249_optimize_delayed_jobs.rb
+++ b/db/migrate/20110610213249_optimize_delayed_jobs.rb
@@ -23,7 +23,7 @@ class OptimizeDelayedJobs < ActiveRecord::Migration
end
remove_index :delayed_jobs, :name => 'get_delayed_jobs_index'
- remove_index :delayed_jobs, [:strand]
+ remove_index :delayed_jobs, :name => 'index_delayed_jobs_on_strand'
add_index :delayed_jobs, %w(run_at queue locked_at strand priority), :name => 'index_delayed_jobs_for_get_next'
add_index :delayed_jobs, %w(strand id), :name => 'index_delayed_jobs_on_strand'
> bundle exec rake db:initial_setup
== OptimizeDelayedJobs: migrating ============================================
-- create_table(:failed_jobs)
NOTICE: CREATE TABLE will create implicit sequence "failed_jobs_id_seq" for serial column "failed_jobs.id"
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "failed_jobs_pkey" for table "failed_jobs"
-> 0.0219s
-- remove_index(:delayed_jobs, {:name=>"get_delayed_jobs_index"})
-> 0.0045s
-- remove_index(:delayed_jobs, {:name=>"index_delayed_jobs_on_strand"})
-> 0.0015s
-- add_index(:delayed_jobs, ["run_at", "queue", "locked_at", "strand", "priority"], {:name=>"index_delayed_jobs_for_get_next"})
-> 0.0080s
-- add_index(:delayed_jobs, ["strand", "id"], {:name=>"index_delayed_jobs_on_strand"})
rake aborted!
An error has occurred, this and all later migrations canceled:
PG::Error: ERROR: relation "index_delayed_jobs_on_strand" already exists
: CREATE INDEX "index_delayed_jobs_on_strand" ON "delayed_jobs" ("strand", "id")/home/tyler/.rvm/gems/ruby-1.9.3-p545/bundler/gems/rails-e86daf8ff727/activerecord/lib/active_record/connection_adapters/abstract_adapter.rb:227:in `rescue in log'
I worked around this error by dropping the entire table and re-creating it (not a problem since I don't have any data yet):
--- a/db/migrate/20110610213249_optimize_delayed_jobs.rb
+++ b/db/migrate/20110610213249_optimize_delayed_jobs.rb
@@ -25,8 +25,30 @@ class OptimizeDelayedJobs < ActiveRecord::Migration
remove_index :delayed_jobs, :name => 'get_delayed_jobs_index'
remove_index :delayed_jobs, [:strand]
+ drop_table :delayed_jobs
+ create_table :delayed_jobs, :force => true do |t|
+ t.integer "priority", :default => 0
+ t.integer "attempts", :default => 0
+ t.string "handler", :limit => 512000
+ t.text "last_error"
+ t.string "queue"
+ t.datetime "run_at"
+ t.datetime "locked_at"
+ t.datetime "failed_at"
+ t.string "locked_by"
+ t.datetime "created_at"
+ t.datetime "updated_at"
+ t.string "tag"
+ t.integer "max_attempts"
+ t.string "strand"
+ end
+
add_index :delayed_jobs, %w(run_at queue locked_at strand priority), :name => 'index_delayed_jobs_for_get_next'
+ add_index "delayed_jobs", ["priority", "run_at", "failed_at", "locked_at", "queue"], :name => "get_delayed_jobs_index"
+ add_index "delayed_jobs", ["priority", "run_at"], :name => "delayed_jobs_priority"
+ add_index "delayed_jobs", ["queue"], :name => "delayed_jobs_queue"
add_index :delayed_jobs, %w(strand id), :name => 'index_delayed_jobs_on_strand'
+ add_index "delayed_jobs", ["tag"], :name => "index_delayed_jobs_on_tag"
# move all failed jobs to the new failed table
Delayed::Backend::ActiveRecord::Job.where("failed_at IS NOT NULL").find_each do |job|
But I got a similar error when it got to the *next* migration with a `remove_index` followed by a `add_index` for the same index name:
> bundle exec rake db:initial_setup
== AddDelayedJobsNextInStrand: migrating =====================================
-- remove_index(:delayed_jobs, {:name=>"index_delayed_jobs_for_get_next"})
-> 0.0019s
-- add_column(:delayed_jobs, :next_in_strand, :boolean, {:default=>true, :null=>false})
-> 0.0030s
rake aborted!
An error has occurred, this and all later migrations canceled:
PG::Error: ERROR: relation "get_delayed_jobs_index" already exists
: CREATE INDEX get_delayed_jobs_index ON delayed_jobs (priority, run_at) WHERE locked_at IS NULL AND queue = 'canvas_queue' AND next_in_strand = 't'/home/tyler/.rvm/gems/ruby-1.9.3-p545/bundler/gems/rails-e86daf8ff727/activerecord/lib/active_record/connection_adapters/abstract_adapter.rb:227:in `rescue in log'
Tried this workaround (from http://stackoverflow.com/questions/1395672/how-do-i-run-a-migration-without-starting-a-transaction-in-rails):
--- a/db/migrate/20110831210257_add_delayed_jobs_next_in_strand.rb
+++ b/db/migrate/20110831210257_add_delayed_jobs_next_in_strand.rb
@@ -3,6 +3,11 @@ class AddDelayedJobsNextInStrand < ActiveRecord::Migration
Delayed::Backend::ActiveRecord::Job.connection
end
+ def ddl_transaction(&block)
+ puts 'Tried to run ddl_transaction'
+ block.call
+ end
+
def self.up
raise("#{connection.adapter_name} is not supported for delayed jobs queue") unless %w(PostgreSQL MySQL Mysql2 SQLite).include?(connection.adapter_name)
I'm not sure if it had any effect since I didn't see the output but I got a different result nonetheless:
> bundle exec rake db:initial_setup
== AddDelayedJobsNextInStrand: migrating =====================================
-- remove_index(:delayed_jobs, {:name=>"index_delayed_jobs_for_get_next"})
-> 0.0023s
-- add_column(:delayed_jobs, :next_in_strand, :boolean, {:default=>true, :null=>false})
rake aborted!
An error has occurred, this and all later migrations canceled:
PG::Error: ERROR: column "next_in_strand" of relation "delayed_jobs" already exists
: ALTER TABLE "delayed_jobs" ADD COLUMN "next_in_strand" boolean/home/tyler/.rvm/gems/ruby-1.9.3-p545/bundler/gems/rails-e86daf8ff727/activerecord/lib/active_record/connection_adapters/abstract_adapter.rb:227:in `rescue in log'
> psql canvas_queue_development
psql (9.1.11)
Type "help" for help.
canvas_queue_development=# \d delayed_jobs
Table "public.delayed_jobs"
Column | Type | Modifiers
----------------+-----------------------------+-----------------------------------------------------------
id | bigint | not null default nextval('delayed_jobs_id_seq'::regclass)
priority | integer | default 0
attempts | integer | default 0
handler | character varying(512000) |
last_error | text |
queue | character varying(255) |
run_at | timestamp without time zone |
locked_at | timestamp without time zone |
failed_at | timestamp without time zone |
locked_by | character varying(255) |
created_at | timestamp without time zone |
updated_at | timestamp without time zone |
tag | character varying(255) |
max_attempts | integer |
strand | character varying(255) |
next_in_strand | boolean | not null default true
Indexes:
"delayed_jobs_pkey" PRIMARY KEY, btree (id)
"delayed_jobs_priority" btree (priority, run_at)
"delayed_jobs_queue" btree (queue)
"get_delayed_jobs_index" btree (priority, run_at, failed_at, locked_at, queue)
"index_delayed_jobs_for_get_next" btree (run_at, queue, locked_at, strand, priority)
"index_delayed_jobs_on_strand" btree (strand, id)
"index_delayed_jobs_on_tag" btree (tag)
Discovered the `transactional` option in config/initializers/active_record.rb and referenced by https://github.com/instructure/canvas-lms/commit/41279b925192402df4214ea6f595e6280100bae0.
When I tried this:
class AddDelayedJobsNextInStrand < ActiveRecord::Migration
def self.connection
Delayed::Backend::ActiveRecord::Job.connection
end
self.transactional = false
I got this:
> -- transactional=(false)
rake aborted!
undefined method `transactional=' for #<ActiveRecord::ConnectionAdapters::PostgreSQLAdapter:0x000000125350b8>
Trying this now:
--- a/db/migrate/20110831210257_add_delayed_jobs_next_in_strand.rb
+++ b/db/migrate/20110831210257_add_delayed_jobs_next_in_strand.rb
@@ -3,12 +3,19 @@ class AddDelayedJobsNextInStrand < ActiveRecord::Migration
Delayed::Backend::ActiveRecord::Job.connection
end
+ class << self
+ def transactional?
+ false
+ end
+ end
+
def self.up
raise("#{connection.adapter_name} is not supported for delayed jobs queue") unless %w(PostgreSQL MySQL Mysql2 SQLite).include?(connection.adapter_name)
remove_index :delayed_jobs, :name => 'index_delayed_jobs_for_get_next'
+ remove_index :delayed_jobs, :name => 'get_delayed_jobs_index'
- add_column :delayed_jobs, :next_in_strand, :boolean, :default => true, :null => false
+ #add_column :delayed_jobs, :next_in_strand, :boolean, :default => true, :null => false
# create the new index
case connection.adapter_name
> bundle exec rake db:initial_setup
== AddDelayedJobsNextInStrand: migrating =====================================
-- remove_index(:delayed_jobs, {:name=>"index_delayed_jobs_for_get_next"})
-> 0.0047s
-- remove_index(:delayed_jobs, {:name=>"get_delayed_jobs_index"})
-> 0.0014s
rake aborted!
An error has occurred, all later migrations canceled:
PG::Error: ERROR: relation "get_delayed_jobs_index" already exists
: CREATE INDEX get_delayed_jobs_index ON delayed_jobs (priority, run_at) WHERE locked_at IS NULL AND queue = 'canvas_queue' AND next_in_strand = 't'
It looks like the remove_index just isn't having any effect!
I guess I'll try just executing the DDL directly:
--- a/db/migrate/20110831210257_add_delayed_jobs_next_in_strand.rb
+++ b/db/migrate/20110831210257_add_delayed_jobs_next_in_strand.rb
@@ -7,8 +7,10 @@ class AddDelayedJobsNextInStrand < ActiveRecord::Migration
raise("#{connection.adapter_name} is not supported for delayed jobs queue") unless %w(PostgreSQL MySQL Mysql2 SQLite).include?(connection.adapter_name)
remove_index :delayed_jobs, :name => 'index_delayed_jobs_for_get_next'
+ remove_index :delayed_jobs, :name => 'get_delayed_jobs_index'
+ connection.execute("DROP INDEX get_delayed_jobs_index")
- add_column :delayed_jobs, :next_in_strand, :boolean, :default => true, :null => false
+ #add_column :delayed_jobs, :next_in_strand, :boolean, :default => true, :null => false
# create the new index
case connection.adapter_name
That worked...
> bundle exec rake db:initial_setup
== AddDelayedJobsNextInStrand: migrating =====================================
-- remove_index(:delayed_jobs, {:name=>"index_delayed_jobs_for_get_next"})
-> 0.0049s
-- remove_index(:delayed_jobs, {:name=>"get_delayed_jobs_index"})
-> 0.0015s
-- execute(" CREATE FUNCTION delayed_jobs_before_insert_row_tr_fn () RETURNS trigger AS $$\n BEGIN\n LOCK delayed_jobs IN SHARE ROW EXCLUSIVE MODE;\n IF (SELECT 1 FROM delayed_jobs WHERE strand = NEW.strand LIMIT 1) = 1 THEN\n NEW.next_in_strand := 'f';\n END IF;\n RETURN NEW;\n END;\n $$ LANGUAGE plpgsql;\n")
-> 0.0050s
-- execute("CREATE TRIGGER delayed_jobs_before_insert_row_tr BEFORE INSERT ON delayed_jobs FOR EACH ROW WHEN (NEW.strand IS NOT NULL) EXECUTE PROCEDURE delayed_jobs_before_insert_row_tr_fn()")
-> 0.0029s
-- execute(" CREATE FUNCTION delayed_jobs_after_delete_row_tr_fn () RETURNS trigger AS $$\n BEGIN\n UPDATE delayed_jobs SET next_in_strand = 't' WHERE id = (SELECT id FROM delayed_jobs j2 WHERE j2.strand = OLD.strand ORDER BY j2.strand, j2.id ASC LIMIT 1);\n RETURN OLD;\n END;\n $$ LANGUAGE plpgsql;\n")
-> 0.0006s
-- execute("CREATE TRIGGER delayed_jobs_after_delete_row_tr AFTER DELETE ON delayed_jobs FOR EACH ROW WHEN (OLD.strand IS NOT NULL AND OLD.next_in_strand = 't') EXECUTE PROCEDURE delayed_jobs_after_delete_row_tr_fn()")
-> 0.0007s
-- execute("UPDATE delayed_jobs SET next_in_strand = 'f' WHERE strand IS NOT NULL AND id <> (SELECT id FROM delayed_jobs j2 WHERE j2.strand = delayed_jobs.strand ORDER BY j2.strand, j2.id ASC LIMIT 1)")
-> 0.0023s
== AddDelayedJobsNextInStrand: migrated (0.0364s) ============================
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment