Journal for OpenStack Ceilometer work -- 5 Jul 2013

diary-5jul2013.md

5 Jul 2013

Bug I posted on Launchpad about GnomeKeyring errors when installing Devstack

• Reference: https://bugs.launchpad.net/devstack/+bug/1193164
• Noticed that someone posted a fix to this problem. Running $ sudo apt-get install gir1.2-gnomekeyring-1.0 removes the error.

Pulled new changes and ran tests

• Pulled down new changes in /opt/stack/ceilometer with git pull origin master

• Rebuilt test environment but got error message about

  ceilometer.conf.sample is not up to date, please run 
  tools/conf/generate_sample.sh
  

• Had to run $ ./tools/conf/generate_sample.sh, then the command

  $ tox -r -e py27,pep8
  

  works and all tests pass -- yay!

How to run a single test

• By trial and error, it seems like you can run a single test by doing something like

  $ tox -e py27 -- 
  tests.storage.test_impl_mongodb.MeterTest.test_get_meters_by_metaquery
  

• Interestingly enough if I do this command

  $ tox -e py27 -- tests.storage.test_impl_mongodb.MeterTest.test_get_meters
  

  It matches all the tests that start with test_get_meters, so it runs test_get_meters, test_get_meters_by_user, test_get_meters_by_project, test_get_meters_by_metaquery, test_get_meters_by_empty metaquery

The bug I'm working on

• jd wants me to focus on this bug "unable to sort data with MongoDB": https://bugs.launchpad.net/ceilometer/+bug/1193906

• Notice there are already some compound indexes for MongoDB in ceilometer/storage/impl_mongodb.py

  for primary in ['user_id', 'project_id']:
      self.db.resource.ensure_index([
          (primary, pymongo.ASCENDING),
          ('source', pymongo.ASCENDING),
      ], name='resource_idx')
      self.db.meter.ensure_index([
          ('resource_id', pymongo.ASCENDING),
          (primary, pymongo.ASCENDING),
          ('counter_name', pymongo.ASCENDING),
          ('timestamp', pymongo.ASCENDING),
          ('source', pymongo.ASCENDING),
      ], name='meter_idx')
  

• My guess on how to fix the bug: add a simple index only for descending time stamps in class Connection of the file ceilometer/storage/impl_mongodb.py, something like

  self.db.meter.ensure_index(('timestamp', pymongo.DESCENDING))
  

• But first I want to reproduce the error jd mentioned in the bug report

  OperationFailure: database error: too much data for sort() with no index. add an index or specify a smaller limit
  

• Made test set 1000 times larger by hacking tests/storage/base.py with a line

  timestamps_for_test_samples_default_order = timestamps_for_test_samples_default_order*1000
  

  then ran a single test with the command

  $ tox -e py27 -- tests.storage.test_impl_mongodb.RawSampleTest.test_get_samples_in_default_order
  

  The runtime was 1510 seconds.

• Also created a test that only gets the samples and doesn't check time order

  def test_get_samples_index(self):
      f = storage.SampleFilter()
      results = list(self.conn.get_samples(f))
  

  then I ran the test

  $ tox -e py27 -- tests.storage.test_impl_mongodb.RawSampleTest.test_get_samples_index
  

  and the run time was 1468 sec, not much faster. So very little time is spent checking the time ordering of the samples.

• I then tried increasing the database size by an order of magnitude:

  timestamps_for_test_samples_default_order = timestamps_for_test_samples_default_order*10000

  but the test takes forever to run. There was no result after 12 hours, so I stopped the test with a keyboard interrupt.

• If I try a ridiculously large database size

  timestamps_for_test_samples_default_order = timestamps_for_test_samples_default_order*10000000

  Python throws a MemoryError probably because the list is too large for memory.