Skip to content

Instantly share code, notes, and snippets.

@varunpalekar

varunpalekar/Chef_basics.md

Last active November 14, 2018 06:12
Show Gist options
  • Save varunpalekar/65603410b605fe17d06a to your computer and use it in GitHub Desktop.
Save varunpalekar/65603410b605fe17d06a to your computer and use it in GitHub Desktop.
Download ZIP
Chef
Raw
Chef_basics.md

This document includes details about chef-dk tool set.

Chef-dk tool set consists of following tools:

  • chef
  • kitchen
  • berks
  • chef-apply
  • chef-client
  • chef-shell
  • chef-service-manager
  • chef-solo
  • chef-vault
  • fauxhai
  • foodcritic
  • kitchen
  • knife
  • ohai
  • rubocop
  • shef

The best practice of using chef

There are various tools which are used to automate tasks related to chef.

  1. Chef

    • chef exec

      Runs the command in context of the embedded ruby.

    • chef gem

      Runs the gem command in context of the embedded ruby. e.g.,

        chef gem install gem_package

    • chef generate

      Generate a new app, cookbook, or component.

      • chef app Generate an application repo

         chef gemerate app test

        Above command will do the following things

        • Creating directory name test
          • Creating kitchen file .kitchen.yml]
          • File test/README.md
          • Folder cookbooks
            • Folder test
              • File metadata.rb
              • File chefignore
              • File Berksfile
              • Folder recipes
                • file default.rb
        • Now run git init .

      • chef cookbook Generate a single cookbook

      • chef recipe Generate a new recipe

      • chef attribute nerate an attributes file

      • chef template Generate a file template

      • chef file Generate a cookbook file

      • chef lwrp Generate a lightweight resource/provider

      • cehf repo Generate a Chef policy repository

    • shell-init

      Initialize your shell to use ChefDK as your primary ruby

    • install

      Install cookbooks from a Policyfile and generate a locked cookbook set

    • push

      Push a local policy lock to a policy group on the server

  2. berkshelf

    Please note that berkshelf is only for the cookbook, not for the whole application.

    • Source code management tool.

    • A package Manager .

        berks install	# install all cookbook dependencies written in bershelf file

    • Replaces some portion of knife

      • cookbook generator, cookbook uploader, cookbook downloader.

        • berks cookbook NAME : to Genrate cookbook

        • berks init : to convert an existing cookbook

          • Berksfile - berks file
          • Thorfile -
          • chefignore
          • .gitignore
          • Gemfile
          • .kitchen.yml
          • test/integration/default

    Please follow this pattern to make a cookbook.

    • Each cookbook contain each services like loadbalancer, app_proxy server, caching server, worker_pool, database_server, etc. 

        cookbook::default
  cookbook:load_balancer
  cookbook::app_proxy
  cookbook::caching_server
  cookbook::worker_pool
  cookbook::db_server

  3. ChefSpec

    It is unit test tool for cookbook. ChefSpec makes it easy to write examples and get fast feedback on cookbook changes without the need for virtual machines or cloud servers.

    ChefSpec runs your cookbook(s) locally with Chef Solo or chefserver without actually converging a node. This has two primary benefits:

    • It's really fast!
    • Your tests can vary node attributes, operating systems, and search results to assert behavior under varying conditions.

    Testing with chefspecs

    The associated ChefSpec test might look like:

    require 'chefspec'

describe 'example::default' do
  let(:chef_run) { ChefSpec::SoloRunner.converge(described_recipe) }

  it 'installs foo' do
    expect(chef_run).to install_package('foo')
  end
end

    Folder and file structure

     /spec - folder for containing unit testcases

    In General we divide our files in two :

    1. Configuration file

      • /spec/spec_helper.rb : the default file to define general configuration of all tests

         require 'chefspec'	#to include chefspec in rspec
 require 'chefspec/berkshelf'	#to include bershelf which is responsible for downloding external cookbook dependencies.
 # require 'chefspec/cacher'		# The cacher module allows for ultra-fast tests by caching the results of a CCR in memory across an example group
 # require 'chefspec/librarian'	#

 # Dir[File.dirname(__FILE__) + '/support/**/*.rb'].each { |f| require f }

 RSpec.configure do |config|
   # Specify the path for Chef Solo to find cookbooks (default: [inferred from
   # the location of the calling spec file])
   # config.cookbook_path = '/var/cookbooks'

   # Specify the path for Chef Solo to find roles (default: [ascending search])
   # config.role_path = '/var/roles'

   # Specify the path for Chef Solo to find environments (default: [ascending search])
   # config.environment_path = '/var/environments'

   # Specify the path to a local JSON file with Ohai data (default: nil)
   # config.path = 'ohai.json'

   config.platform = 'ubuntu'
   config.version = '12.04'

   config.log_level = :error
 end

 at_exit { ChefSpec::Coverage.report! }

      • Values specified at the initialization of the Runner merge and take precedence over the global settings:

        Here Runner is ServerRunner or SoloRunner

         # Override only the operating system version (platform is still "ubuntu" from above)
 ChefSpec::Runner.new(version: '10.04')

 # Use a different operating system platform and version
 ChefSpec::Runner.new(platform: 'centos', version: '5.4')

 # Specify a different cookbook_path
 ChefSpec::Runner.new(cookbook_path: '/var/my/other/path', role_path: '/var/my/roles')

 # Add debug log output
 ChefSpec::Runner.new(log_level: :debug).converge(described_recipe)

    2. Test file

      • /spec/unit/recipe/default_spec.rb : In this file we write the test cases.

         describe 'example::default' do
   let(:chef_run) { ChefSpec::SoloRunner.converge(described_recipe) }

   it 'does something' do
     expect(chef_run).to ACTION_RESOURCE(NAME)
   end
 end

        Where:

        • ACTION - the action on the resource (e.g. install)
        • RESOURCE - the name of the resource (e.g. package)
        • NAME - the name attribute for the resource (e.g. apache2)

      • Describe : It specifies which recipe you want to test, For e.g., memsql::default

    Require

    The common plugin/gems for chefspec which is used to enhance or modify its functionality are as:

    1. require 'chefspec'

      Create a temporary working directory and set ChefSpec's cookbook_path to the temporary directory.

    2. require 'chefspec/berkshelf'

      Download all the dependencies listed in your Berksfile into the temporary directory.

    3. require 'chefspec/cacher'

      The cacher module allows for ultra-fast tests by caching the results of a CCR in memory across an example group

    Runner

    In this we specify what to use in testing, chefsolo or chefzero. This can be defined in let

    For e.g.,

    1. CHEFSOLO

       let(:chef_run) { ChefSpec::SoloRunner.converge(described_recipe) }

    2. CHEFZERO

       let(:chef_run) { ChefSpec::ServerRunner.converge(described_recipe) }

      When we using ServerRunner we have to upload some data like client, databags, environment, etc to chefzero server, then we using following methods for that:

      • Create a client:

         ChefSpec::ServerRunner.new do |node, server|
   server.create_client('my_client', { admin: true })
 end

      • Create a data bag (and items):

         ChefSpec::ServerRunner.new do |node, server|
   server.create_data_bag('my_data_bag', {
     'item_1' => {
       'password' => 'abc123'
     },
     'item_2' => {
       'password' => 'def456'
     }
   })
 end

      • Create an environment:

         ChefSpec::ServerRunner.new do |node, server|
   server.create_environment('my_environment', { description: '...' })
 end

      • Create a node:

         ChefSpec::ServerRunner.new do |node, server|
   server.create_node('my_node', { run_list: ['...'] })
 end

        At the time of creation of node also define some ohai value, Known as Fauxhai objects.

        First Create and attribute object like,

         www = stub_node(platform: 'ubuntu', version: '12.04') do |node|
     node.set['attribute'] = 'value'
   end

 # `www` is now a local Chef::Node object you can use in your test. To publish
 # this node to the server, call `create_node`:

 ChefSpec::ServerRunner.new do |node, server|
   server.create_node(www)
 end

      • Create a role:

         ChefSpec::ServerRunner.new do |node, server|
   server.create_role('my_role', { default_attributes: {} })
 end

      NOTE The ChefSpec server is empty at the start of each example to avoid interdependent tests.

    • ChefSpec asserts that resource actions have been performed. In general, ChefSpec follows the following pattern:

    • Action and resource includes following helpful matchers.

      • include_recipe

        Assert that the Chef run included a recipe from another cookbook

          expect(chef_run).to include_recipe('other_cookbook::recipe')

      • notify

        Assert that a resource notifies another in the Chef run

          resource = chef_run.template('/etc/foo')
  expect(resource).to notify('service[apache2]').to(:restart).immediately

      • subscribes

        Assert that a resource subscribes to another in the Chef run

          resource = chef_run.service('apache2')
  expect(resource).to subscribe_to('template[/etc/foo]').on(:create).delayed

      • render_file

        Assert that the Chef run renders a file (with optional content); this will match cookbook_file, file, and template resources and can also check the resulting content

         expect(chef_run).to render_file('/etc/foo')
 expect(chef_run).to render_file('/etc/foo').with_content('This is content')
 expect(chef_run).to render_file('/etc/foo').with_content(/regex works too.+/)

        You can use any RSpec content matcher inside of the with_content predicate:

          expect(chef_run).to render_file('/etc/foo').with_content(start_with('# First line'))

      • package

          expect(chef_run).to install_package('apache2')

        If you want to specify package('apache2').run_action(:install)

          expect(chef_run).to install_package('apache2').at_compile_time

        Similarly, you can assert that a resource is executed during convergence time:

          expect(chef_run).to install_package('apache2').at_converge_time

        Other forms

         expect(chef_run).to install_package('apache2').with_version('1.2.3')
 expect(chef_run).to install_package('apache2').with(version: '1.2.3')
 expect(chef_run).to_not install_package('apache2')

 expect(chef_run).to purge_package('apache2')
 expect(chef_run).to purge_package('apache2').with(version: '1.2.3')
 expect(chef_run).to_not purge_package('apache2')
 expect(chef_run).to reconfig_package('apache2').with(version: '1.2.3')
 expect(chef_run).to remove_package('apache2').with(version: '1.2.3')
 expect(chef_run).to upgrade_package('apache2').with(version: '1.2.3')

      • do_nothing

        Assert that a resource performs no action

          resource = chef_run.execute('install')
  expect(resource).to do_nothing

    Stubbing

    Stubbing means creating a fake return from any asking things. Like if in my recipe if I used not_if with command then how chef_spec provide this in fake environment. So for this we have to make a stub command which define what to return if this ask.

    For e.g., If I am using not_if in any template like

    template '/tmp/foo.txt' do
  not_if 'grep text /tmp/foo.txt'
end

    Then we can stub the command what to return with grep text /tmp/foo.txt.

    before do
  stub_command("grep text /tmp/foo.txt").and_return(true)
end

    There are various types of stubbing. Some of them are:

    1. Command

      not_if 'grep text /tmp/foo.txt'

#We can stub this by
before do
  stub_command("grep text /tmp/foo.txt").and_return(true)
end

    2. Data Bag & Data Bag Item

      Generally this is not required if you are using ChefSpec server, since you can upload databags directly in it.

      Given a recipe that executes a data_bag method:

      data_bag('users').each do |user|
  data_bag_item('users', user['id'])
end

      You can create a stub like this

      before do
    stub_data_bag('users').and_return([])
  end

      You can also define multiple stubs in defore like:

      describe 'example::default' do
  let(:chef_run) { ChefSpec::SoloRunner.new }

  before do
    stub_data_bag('users').and_return(['svargo', 'francis'])

    stub_data_bag_item('users', 'svargo').and_return({ ... })
    stub_data_bag_item('users', 'francis') { (ruby code) }
  end
end

      If you are using encypted databags then

      before do
    allow(Chef::EncryptedDataBagItem).to receive(:load).with('users', 'svargo').and_return(...)
  end

    3. Search

      Similar to the above it also not required in chefspec server.

      before do
    stub_search(:node, 'name:hello').and_return([])
  end

before do
    stub_search(:node, 'name:hello') { (ruby_code) }
  end

    4. Environment

      let(:chef_run) do
  ChefSpec::SoloRunner.new do |node|
    # Create a new environment (you could also use a different :let block or :before block)
    env = Chef::Environment.new
    env.name 'staging'

    # Stub the node to return this environment
    allow(node).to receive(:chef_environment).and_return(env.name)

    # Stub any calls to Environment.load to return this environment
    allow(Chef::Environment).to receive(:load).and_return(env)
  end.converge('cookbook::recipe')
end

  4. busser-ServerSpec

    Serverspec is another test platform that tests your actual servers via ssh. With Serverspec you can write RSpec test for checking your servers and configured correctly.

    Similar to ChefSpec it has decribe in which we describe the whole test cases.

    describe "Git Daemon" do

  it "is listening on port 3306" do
    expect(port(3306)).to be_listening
  end

  it "has a running service of memsql" do
    expect(service("memsql")).to be_running
  end

end

  5. Kitchen

    It is a QA tool for chef cookbooks which uses virtual instances for testing like vmware, virtualbox, EC2, etc. Its uses driver to connect these virtual instance service. To use kitchen we have the following steps:

    • initialize

        kitchen init --driver=kitchen-vagrant
        create  .kitchen.yml
        create  test/integration/default
        create  .gitignore
        append  .gitignore
        append  .gitignore

    • .kitchen.yml file

       ---
 driver:
   name: vagrant

 provisioner:
   name: chef_solo

 platforms:
   - name: ubuntu-12.04
   	driver_config:
       network:
         - ['private_network', {ip: '192.168.1.1'}]
   - name: centos-6.4
   	driver_config:
       network:
         - ['private_network', {ip: '192.168.1.2'}]

 suites:
   - name: default
     run_list:
       - recipe[git::default]
     attributes:

      • driver: This is where we configure the behaviour of the Kitchen Driver - the component that is responsible for creating a machine that we'll use to test our cookbook. Here we set up basics like credentials, ssh usernames, sudo requirements, etc. Each Driver is reponsible for requiring and using the configuration here.

      • provisioner: This tells Test Kitchen how to run Chef, to apply the code in our cookbook to the machine under test. The default and simplest approach is to use chef-solo, but other options are available, and ultimately Test Kitchen doesn't care how the infrastructure is built - it could theoretically be with Puppet, Ansible, or Perl for all it cares.

      • platforms: This is a list of operation systems on which we want to run our code. Note that the operating system's version, architecture, cloud environment, etc. might be relevant to what Test Kitchen considers a Platform.

      • suites: This section defines what we want to test. It includes the Chef run-list and any node attribute setups that we want run on each Platform above. For example, we might want to test the MySQL client cookbook code seperately from the server cookbook code for maximum isolation.

    • List all machines

        $ kitchen list

  Instance             Driver   Provisioner  Last Action
  default-ubuntu-1204  Vagrant  ChefZero     <Not Created>
  default-centos-64    Vagrant  ChefZero     <Not Created>

    • Create a instance

        $ kitchen create default-ubuntu-1204

    • upload cookbook and run chef client

        kitchen converge default-ubuntu-1204

    • Kitchen verify it manually first

      • you can login to instance and manually check

          kitchen login default-ubuntu-1204

    • Writing first integration testing

      The component that helps facilitate testing on your instances is called Busser. To keep things simple we're going to use the busser-bats runner plugin which uses the Bash Automated Testing System also known as bats. You can also use any automated testing tool for this like

      • make file in cookbook's folder test/integration/default/bats/verify_installed.bat

         @test "tmux is installed and in the path" {
   which tmux
 }

 @test "tmux configuration exists" {
   cat /etc/tmux.conf | grep "map" # this could be a more complex test
 }

      • As a reminder, the format for the path to a test is:

          test/integration/SUITE/BUSSER/TEST

    • General structure of folders in kitchen

      • test : shows it contains all test cases
        • integration : contains all scripts of intehration testing
          • SUITE : which suite name you use in .kitchen.yml file
            • BUSSER : this tells kitchen which busser runner plugin is used. For e.g., bats
              • Test : Testing scripts file

    • kitchen test automatically

        kitchen verify default-ubuntu-1204

    • you can also run all in one command

        kitchen test

    There are few basic ways also for dynamic configuring Test Kitchen

    • A local configuration will be looked for in .kitchen.local.yml which could be used for development purposes. This is a file that is not typically checked into version control.
    • A global configuration file will also be looked for in $HOME/.kitchen/config.yml to set preferred defaults for all your projects.

    Testing in detail by mode of example:

    • Integration testing using serverspec

      • Create a file test/integration/server/serverspec/git_daemon_spec.rb

         require 'serverspec'

 # Required by serverspec
 set :backend, :exec

 describe "Git Daemon" do

   it "is listening on port 9418" do
     expect(port(9418)).to be_listening
   end

   it "has a running service of git-daemon" do
     expect(service("git-daemon")).to be_running
   end

 end

  6. Foodcritic

    Foodcritic has two goals:

    • To make it easier to flag problems in your Chef cookbooks that will cause Chef to blow up when you attempt to converge. This is about faster feedback. If you automate checks for common problems you can save a lot of time.

    • To encourage discussion within the Chef community on the more subjective stuff - what does a good cookbook look like? Opscode have avoided being overly prescriptive which by and large I think is a good thing. Having a set of rules to base discussion on helps drive out what we as a community think is good style.

    • To simple check the code :

        foodcritic

  7. Guard

    It is used to automate test cookbooks. Guard monitors file system modifications and then takes actions based on what's defined in your Guardfile. And guard-kitchen is a plugin for Guard that auto-generates Test-Kitchen matchers for your Guardfile.

    • Guard init repo2/cookbooks/hello

        guard init

    • Guard start

        guard start

    Chef-Guard is a feature rich Chef add-on that protects your Chef server from untested and uncommitted cookbooks by running several validations and checks during the cookbook upload process. In addition Chef-Guard will also monitor, audit, save and email (including any difference with the actual change) all configuration changes, and is even capable of validating certain changes before passing them through to Chef.

    Generally Guard is used to automate other tools like kitchen, foodcritic, etc. which automatically launch the script on the basis of file system change specified in watchers.

    For e.g.,

    • Automate kitchen testing

       # A sample Guardfile
 # More info at https://github.com/guard/guard#readme

 guard 'kitchen' do  
   watch(%r{test/.+})
   watch(%r{^recipes/(.+)\.rb$})
   watch(%r{^attributes/(.+)\.rb$})
   watch(%r{^files/(.+)})
   watch(%r{^templates/(.+)})
   watch(%r{^providers/(.+)\.rb})
   watch(%r{^resources/(.+)\.rb})
 end

    • Automate foodcritic

       # A sample Guardfile
 # More info at https://github.com/guard/guard#readme

 guard "foodcritic" do  
   watch(%r{attributes/.+\.rb$})
   watch(%r{providers/.+\.rb$})
   watch(%r{recipes/.+\.rb$})
   watch(%r{resources/.+\.rb$})
   watch(%r{^templates/(.+)})
   watch('metadata.rb')
 end
Raw
Chef_experience
MemSQL Cookbook
-----------------------
* In development of collectd recipe in which I experience a unusual behavior. When I created a template of collectd daemon to run, then the service is not running and giving me error.
* Issues with collectd config file. Run '/var/lib/collectd/collectd -T -C "/etc/collectd.conf"' for more information.
This type of issue come due to wrong configuration file provided by memsql site. Sample configuration file is as:
```bash
BaseDir "/var/lib/collectd"
PluginDir "lib/collectd/lib/collectd"
TypesDB "lib/collectd/share/collectd/types.db"
Interval 1
LoadPlugin cpu
LoadPlugin memory
LoadPlugin df
LoadPlugin interface
LoadPlugin syslog
LoadPlugin logfile
<Plugin logfile>
LogLevel "info"
File "stdout"
Timestamp true
PrintSeverity true
</Plugin>
<LoadPlugin python>
Globals true
</LoadPlugin>
<Plugin python>
ModulePath "python/lib/python2.7/dist-packages/"
Import "memsql_collectd.plugin"
<Module "memsql_collectd.plugin">
Host "<%=node["memsql"]["ops"]["ip_address"]%>"
User "root"
Port "3306"
TypesDB "lib/collectd/share/collectd/types.db"
</Module>
</Plugin>
```
* After correcting that manually and collectd daemon running properly, but when I ran chef-client, then it again giving me the same error as above as.
```bash
Parse error in file `/etc/collectd.conf', line 36 near `': syntax error, unexpected $end, expecting EOL
yyparse returned error #1
configfile: Cannot read file `/etc/collectd.conf'.
Unable to read config file /etc/collectd.conf.
Error: Reading the config file failed!
Read the syslog for details.
```
After that I again login and just edit file with nothing and write file again. My collectd daemon running fine. I am not getting how to fix this with chef. Then after adding an extra enter to template file fix this issue.
Git Sync issue
--------------------
I am using git sync with chef with following code
```ruby
git "/home/app/app" do
repository "git@github.com:decisive/bidder.git"
revision deploy_revision
action :sync
enable_submodules true
user "app"
group "app"
notifies :run, "bash[app_requirements]", :immediately
end
```
If there is per folder of sync like `/home/app/app` then git is not sync. Thats' the problem I am facing.
Raw
MemSQL_cookbook_readme
# memsql-cookbook
This cookbook is used to install MemSQL server and set as a Leaf Node, Master Aggregator or child Aggregator.
**See MemSql.md for MemSQL**
####NOTE : Please see the attributes briefly. Since at some places recipe uses search node which depends on roles and environment of your chef nodes.
## Supported Platforms
* Amazon AMI 2012.03
* CentOS 6.0
* Debian 6.0
* Fedora 14
* OpenSUSE 11.0
* Red Hat 6.0
* Ubuntu 8.04
## Attributes
| Key | | Type | Description | Default value |
|---------------------------------------------------|---------------------------------|-----------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------|
| ['memsql'] | ['install_method'] | String | Used to define what to use "package" or "binary" method of installation | "package" |
| | ['licence'] | String | MemSQL licence to download | "92a4284b277a47d28c2ba3226e260d07" |
| | ['version'] | String | Version of MemSQL to install according to licence no. | '3.1' |
| ['memsql']['config'] | ['port'] | String | port acquire by memsql | '3306' |
| | ['bind_address'] | String | home/installation directory for redis | "0.0.0.0" |
| | ['durability'] | String | Default shell for redis user | "on" |
| | ['transaction_buffer'] | String | Is system user? | "64m" |
| | ['snapshot_trigger_size'] | String | | "256m" |
| | ['basedir'] | String | | '.' |
| | ['tmpdir'] | String | | '.' |
| | ['lc_messages_dir'] | String | | './share' |
| | ['socket'] | String | | 'memsql.sock' |
| | ['lock_wait_timeout'] | String | | '60' |
| | ['max_connections'] | String | | '8192' |
| | ["redundency"] | String | Redundancy used by MemSQL Cluster 1 or 2 | 1 |
| ["memsql"]["custom_url"] | ['use'] | Boolean | Use custom download link or not for memsql | false |
| | ['link'] | String | Link of memsql file | "http://192.168.94.100/memsql-3.1.x86_64.deb" |
| ["memsql"] | ["checksum"] | Boolean | Use cheksum to check memsql download is valid or not.SHA256 checksum is used | false |
| | ["file_check_sum"] | String | Checksum value | "6b534d0d24173637dbfa703b62b097423c6ddf8cbc5d13427b4bf0307edc1526" |
| ["memsql"]["aggregator"] | ["enable"] | Boolean | Is note is aggregator or not. If not aggregator then by default it is leaf node. | true |
| | ["type"] | String | If aggregator, then child or master aggregator | "master" |
| ["memsql"] | ["use_search"] | Boolean | Use chef search command to search mater agg, child agg, and leaf node or use attribute value | false |
| ["memsql"]["search"] | ["parameter"] | Integer | If above is true then, Which paramter is used of ohai for domain, 'fqdn' , 'ipaddress', '["network"]["interfaces"]["eth1"]["addresses"][1]'# 1 => 'fqdn' ; 2=> 'ipaddress' ; 3=> ["network"]["interfaces"]["eth1"]["addresses"][1] | 3 |
| ["memsql"] | ["master_aggregator_ipaddress"] | String | If it is not nil then it is used as IP Address of master Aggregator. Else it useses search(:node, "role:memsql_leaf AND chef_environment:production") | |
| | ["master_aggregator_port"] | String | Master aggregator port, used only if above has a value. | "3306" |
| | ["leaf_nodes"] | Array of String | If it is not nil then it is uses this array as IP Addresses of Lead node of MemSQL. Else it uses search(:node, "role:memsql_leaf AND chef_environment:production") | ['127.0.0.1'] |
| ["memsql"]["password"] | ["enable"] | Boolean | Enable MemSQL password or not | |
| | ["databag_name"] | String | Databag name in which password is stored | |
| | ["databag_key"] | String | | |
| | | | | |
| ["memsql"]["ops"] | ["version"] | String | Specify version of MemSQL Ops | "3.1.3" |
| ["memsql"]["ops"]["custom_url"] | ["use"] | Boolean | Use MemSQL Ops custom download link or not | false |
| | ["link"] | String | Custom link of MemSQL Ops | "http://192.168.94.100/memsql-ops-3.1.1.x86_64.deb" |
| ["memsql"]["ops"] | ["checksum"] | Boolean | For MemSQL ops wheather to use checksum to check downloaded file. SHA256 checksum is used | |
| | ["file_check_sum"] | String | Checksum value for MemSQL Ops | "6283ff998f3aa52407b1f4c6619fa1f027dc26ba849a771a5554d626308abe49" |
| ["memsql"]["ops"]["config"]["memsqlops_db"] | ["db_host"] | String | | "" |
| | ["db_port"] | String | | "3306" |
| | ["db_name"] | String | | "dashboard" |
| | ["db_user"] | String | | "root" |
| | ["db_pass"] | String | | "" |
| ["memsql"]["ops"]["config"]["analytics"] | ["retention"] | String | | ".5" |
| | ["events_retention"] | String | | ".5" |
| | ["facts_retention"] | String | | ".5" |
| | ["collectd_prefix"]["on"] | Boolean | | false |
| | ["collectd_prefix"]["value"] | String | | "system" |
| ["memsql"]["ops"]["config"]["healthchecks"] | ["memory_max"] | String | | "90" |
| ["memsql"]["ops"]["config"] | ["skew_max"] | String | | "10" |
| | ["skew_min_rows"] | String | | "10000000" |
| | ["skew_min_bytes"] | String | | "10000000000" |
| | ["http"]["host"] | String | | "0.0.0.0" |
| | ["http"]["port"] | String | | "9000" |
| ["memsql"]["ops"]["config"]["global"] | ["debug"] | String | | "false" |
| | ["verbose_errors"] | String | | "false" |
| ["memsql"]["ops"]["config"]["features"] | ["stats_scan_level"] | String | | "0" |
| | ["stats_scan_interval"] | String | | "60" |
| | ["tablestats_charting"] | String | | "true" |
| | ["querystats_charting"] | String | | "true" |
| ["memsql"]["ops"]["config"]["garbage_collection"] | ["gc_interval"] | String | | "3" |
| | ["gc_batch_size"] | String | | "10000" |
| | ["queries_retention"] | String | | "24" |
| | ["query_stats_retention"] | String | | "6" |
| | ["log_retention"] | String | | "24" |
| | | | | |
| ["memsql"]["collectd"] | ["enable"] | Boolean | Install collectd on each node of cluster, used by MemSQL-Ops for monitoring Disk, memory, cpu, memsql, etc | true |
| | | | | |
| | | | | |
| | | | | |
## Usage
### memesql::default
Include `memesql` in your node's `run_list`:
```json
{
"run_list": [
"recipe[memesql::default]"
],
ov
}
```
### Role
#### Development
##### MemSQL in Development and Staging
* node1:
Act as Master Aggregator and also a leaf node too
* Role File
```json
{
"name" : "memsql",
"default_attributes": {
"memsql" : {
"licence" : "92a4284b277a47d28c2ba3226e260d07",
"version" : "3.1",
"checksum" : true,
"file_check_sum" : "6b534d0d24173637dbfa703b62b097423c6ddf8cbc5d13427b4bf0307edc1526",
"leaf_nodes" : ["127.0.0.1"],
"custom_url" : {
"use" : false
},
"aggregator" : {
"enable" : true,
"type" : "master"
}
}
},
"json_class": "Chef::Role",
"run_list": ["recipe[memsql::default]"
],
"description": "For setting MemSQL",
"chef_type": "role"
}
```
##### MemSQL in Production
* Node : master_aggregator
* Role : `memsql_master_aggregator`
* Role json file
```json
{
"name" : "memsql_master_aggregator",
"override_attributes": {
"memsql" : {
"use_search" : true ,
"licence" : "92a4284b277a47d28c2ba3226e260d07",
"version" : "3.1",
"checksum" : true,
"file_check_sum" : "6b534d0d24173637dbfa703b62b097423c6ddf8cbc5d13427b4bf0307edc1526",
"leaf_nodes" : null,
"custom_url" : {
"use" : true,
"link" : "http://192.168.94.100/memsql-3.1.x86_64.deb"
},
"aggregator" : {
"enable" : true,
"type" : "master"
}
}
},
"json_class": "Chef::Role",
"run_list": ["recipe[memsql::default]"],
"description": "For setting memsql_master_aggregator",
"chef_type": "role"
}
```
* Node : Child aggregator
* Role : `memsql_child_aggregator`
* Role json file
```json
{
"name" : "memsql_child_aggregator",
"override_attributes": {
"memsql" : {
"use_search" : true ,
"licence" : "92a4284b277a47d28c2ba3226e260d07",
"version" : "3.1",
"checksum" : true,
"file_check_sum" : "6b534d0d24173637dbfa703b62b097423c6ddf8cbc5d13427b4bf0307edc1526",
"leaf_nodes" : null,
"custom_url" : {
"use" : true,
"link" : "http://192.168.94.100/memsql-3.1.x86_64.deb"
},
"aggregator" : {
"enable" : true,
"type" : "child"
},
"master_aggregator_ipaddress" : null
}
},
"json_class": "Chef::Role",
"run_list": ["recipe[memsql::default]"
],
"description": "It sets memsql child aggregator",
"chef_type": "role"
}
```
* Node : lead node
* Role : memsql_leaf
* Role json file
```json
{
"name" : "memsql_leaf",
"override_attributes": {
"memsql" : {
"use_search" : true ,
"licence" : "92a4284b277a47d28c2ba3226e260d07",
"version" : "3.1",
"checksum" : true,
"file_check_sum" : "6b534d0d24173637dbfa703b62b097423c6ddf8cbc5d13427b4bf0307edc1526",
"leaf_nodes" : null,
"custom_url" : {
"use" : true,
"link" : "http://192.168.94.100/memsql-3.1.x86_64.deb"
},
"aggregator" : {
"enable" : false
},
"master_aggregator_ipaddress" : null
}
},
"json_class": "Chef::Role",
"run_list": ["recipe[memsql::default]"],
"description": "It sets memsql child aggregator",
"chef_type": "role"
}
```
* MemSQL- Ops
* Role : `memsql-ops`
* Json file:
```json
{
"name" : "memsql-ops",
"override_attributes": {
"memsql" : {
"ops" : {
"config" : {
"memsqlops_db" : {
"db_host" : "192.168.94.55"
}
}
},
"master_aggregator_ipaddress" : "192.168.94.50"
}
},
"json_class": "Chef::Role",
"run_list": ["recipe[memsql::memsqlops]"
],
"description": "It sets memsql child aggregator",
"chef_type": "role"
}
```
## License and Authors
Author:: stackexpress (<varun.palekar@stackexpress.com>)
----------
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment