Goal
Define a standard for how configuration files similar to hadoop
*-env.sh should be defined in chef-bach
.
Requirement
- Should be able to dynamically add/remove properties to the configuration file
- Should be able to override default values defined in
chef-bach
through a wrapper cookbook - Should be able to conditionally add/remove properties to the configuration file
Proposal
- Define the property/value pairs to be stored in the configuration file as
chef
node attributes in the relevant attribute file- relevant -> if the config file is for the
hadoop
componentyarn
, the attribute need to be defined in theyarn.rb
attribute file inbcpc-hadoop
cookbook. - The attribute should be defined using the standard
default["bach"]["PRODUCT_GROUP"]["COMPONENT_NAME"]["env_sh"]["property"] = value
- For e.g.
default["bach"]["hadoop"]["yarn"]["env_sh"]["YARN_OPTS"] = "-Dcom.sun.management.jmxremote.ssl=false"
where PRODUCT_GROUP is hadoop, the hadoop component to which this property is applicable is yarn and YARN_OPTS is one of the property which need to be added to theenv.sh
file
- For e.g.
- If the component is not part of a "PRODUCT_GROUP" for e.g. kafka or spark then the standard which need to be followed is
default["bach"]["COMPONENT_NAME"]["env_sh"]["property"] = value
- For e.g. ``default["bach"]["kafka"]["env_sh"]["KAFKA_OPTS"] = "-Dcom.sun.management.jmxremote.ssl=false"
- relevant -> if the config file is for the
- Define a recipe in the cookbook where the attribute was added with the name
COMPONENT_NAME_config_filename.rb
. For e.g. foryarn
env.sh
config file the recipe name will beyarn_env.rb
- For any property which need to be included or updated in the config file based on certain conditions, add the logic to this recipe
- for e.g. using the
.tap
ruby method
if node[:bcpc][:hadoop][:kerberos][:enable] == true then default["bach"]["hadoop"]["yarn"]["env_sh"].tap do |env| env[:HBASE_OPTS] = '"$HBASE_OPTS -Djava.security.auth.login.config=/etc/hbase/conf/hbase-client.jaas"' env[:HBASE_MASTER_OPTS] = '$HBASE_MASTER_OPTS -Djava.security.auth.login.config=/etc/hbase/conf/hbase-server.jaas' env[:HBASE_REGIONSERVER_OPTS] += ' -Djava.security.auth.login.config=/etc/hbase/conf/regionserver.jaas' end end
- for e.g. using the
- At the end of the recipe use a template to generate the config file using generic_env.sh.erb
chef-bach
template resource.- Note generic_env.sh.erb will need to be updated to handle different key/value seperators like = or spaces
- For any property which need to be included or updated in the config file based on certain conditions, add the logic to this recipe
- Include this recipe in the recipe which installs the component and before the service gets started
Went with how the code is currently structured to minimize change to bring the code to common standard. It is not a must we need to have it since the same component existing under different product groups is
nil
.