Goal
Define a standard for how configuration files similar to hadoop
*-env.sh should be defined in chef-bach
.
Requirement
- Should be able to dynamically add/remove properties to the configuration file
- Should be able to override default values defined in
chef-bach
through a wrapper cookbook - Should be able to conditionally add/remove properties to the configuration file
Proposal
- Define the property/value pairs to be stored in the configuration file as
chef
node attributes in the relevant attribute file- relevant -> if the config file is for the
hadoop
componentyarn
, the attribute need to be defined in theyarn.rb
attribute file inbcpc-hadoop
cookbook. - The attribute should be defined using the standard
default["bach"]["PRODUCT_GROUP"]["COMPONENT_NAME"]["env_sh"]["property"] = value
- For e.g.
default["bach"]["hadoop"]["yarn"]["env_sh"]["YARN_OPTS"] = "-Dcom.sun.management.jmxremote.ssl=false"
where PRODUCT_GROUP is hadoop, the hadoop component to which this property is applicable is yarn and YARN_OPTS is one of the property which need to be added to theenv.sh
file
- For e.g.
- If the component is not part of a "PRODUCT_GROUP" for e.g. kafka or spark then the standard which need to be followed is
default["bach"]["COMPONENT_NAME"]["env_sh"]["property"] = value
- For e.g. ``default["bach"]["kafka"]["env_sh"]["KAFKA_OPTS"] = "-Dcom.sun.management.jmxremote.ssl=false"
- relevant -> if the config file is for the
- Define a recipe in the cookbook where the attribute was added with the name
COMPONENT_NAME_config_filename.rb
. For e.g. foryarn
env.sh
config file the recipe name will beyarn_env.rb
- For any property which need to be included or updated in the config file based on certain conditions, add the logic to this recipe
- for e.g. using the
.tap
ruby method
if node[:bcpc][:hadoop][:kerberos][:enable] == true then default["bach"]["hadoop"]["yarn"]["env_sh"].tap do |env| env[:HBASE_OPTS] = '"$HBASE_OPTS -Djava.security.auth.login.config=/etc/hbase/conf/hbase-client.jaas"' env[:HBASE_MASTER_OPTS] = '$HBASE_MASTER_OPTS -Djava.security.auth.login.config=/etc/hbase/conf/hbase-server.jaas' env[:HBASE_REGIONSERVER_OPTS] += ' -Djava.security.auth.login.config=/etc/hbase/conf/regionserver.jaas' end end
- for e.g. using the
- At the end of the recipe use a template to generate the config file using generic_env.sh.erb
chef-bach
template resource.- Note generic_env.sh.erb will need to be updated to handle different key/value seperators like = or spaces
- For any property which need to be included or updated in the config file based on certain conditions, add the logic to this recipe
- Include this recipe in the recipe which installs the component and before the service gets started
Thanks @pu239ppy for the comments. The idea is to have one recipe for each config file. As a start we are trying to define the standard for
-env.sh
file and as mentioned in the write-up it will have its own recipe. Hope we are on the same page.