OSDN Git Service

adds the zookeeper-grid cookbook. zookeeper-grid-0.1.1
authorwhitestar <whitestar@gaea.test>
Sun, 29 Jan 2017 05:17:19 +0000 (14:17 +0900)
committerwhitestar <whitestar@gaea.test>
Sun, 29 Jan 2017 05:17:19 +0000 (14:17 +0900)
17 files changed:
DEPRECATED.md
cookbooks/zookeeper-grid/.foodcritic [new file with mode: 0644]
cookbooks/zookeeper-grid/.rubocop.yml [new file with mode: 0644]
cookbooks/zookeeper-grid/CHANGELOG.md [new file with mode: 0644]
cookbooks/zookeeper-grid/Gemfile [new file with mode: 0644]
cookbooks/zookeeper-grid/README.md [new file with mode: 0644]
cookbooks/zookeeper-grid/Rakefile [new file with mode: 0644]
cookbooks/zookeeper-grid/attributes/default.rb [new file with mode: 0644]
cookbooks/zookeeper-grid/metadata.rb [new file with mode: 0644]
cookbooks/zookeeper-grid/recipes/default.rb [new file with mode: 0644]
cookbooks/zookeeper-grid/templates/default/conf/configuration.xsl [new file with mode: 0644]
cookbooks/zookeeper-grid/templates/default/conf/jaas.conf [new file with mode: 0644]
cookbooks/zookeeper-grid/templates/default/conf/java.env [new file with mode: 0644]
cookbooks/zookeeper-grid/templates/default/conf/log4j.properties [new file with mode: 0644]
cookbooks/zookeeper-grid/templates/default/conf/zoo.cfg [new file with mode: 0644]
cookbooks/zookeeper-grid/templates/default/conf/zoo_sample.cfg [new file with mode: 0644]
cookbooks/zookeeper-grid/templates/default/conf/zookeeper-env.sh [new file with mode: 0644]

index d8c0a73..b1c8249 100644 (file)
@@ -3,7 +3,8 @@ DEPRECATED Resources
 
 ## Cookbooks
 
-- `commons`
-- `nagios`
-- `node_commons`
-- `openldap`
+- `commons` => `apt_utils`, `platform_utils` and `yum_utils`
+- `nagios` => `nagios-grid`
+- `node_commons` => `commons`
+- `openldap` => `openldap-grid`
+- `zookeeper` => `zookeeper-grid`
diff --git a/cookbooks/zookeeper-grid/.foodcritic b/cookbooks/zookeeper-grid/.foodcritic
new file mode 100644 (file)
index 0000000..3907cff
--- /dev/null
@@ -0,0 +1 @@
+~FC001
diff --git a/cookbooks/zookeeper-grid/.rubocop.yml b/cookbooks/zookeeper-grid/.rubocop.yml
new file mode 100644 (file)
index 0000000..1874b9a
--- /dev/null
@@ -0,0 +1,41 @@
+AllCops:
+  Exclude:
+    - vendor/**/*
+
+AlignParameters:
+  Enabled: false
+LineLength:
+  Enabled: false
+Lint/UnusedBlockArgument:
+  Enabled: false
+Metrics/AbcSize:
+  Enabled: false
+Style/BlockComments:
+  Enabled: false
+Style/BlockDelimiters:
+  Enabled: false
+Style/ExtraSpacing:
+  Enabled: false
+Style/FileName:
+  Enabled: false
+Style/LeadingCommentSpace:
+  Enabled: false
+Style/RescueModifier:
+  Enabled: false
+Style/SpaceBeforeFirstArg:
+  Enabled: false
+Style/TrailingCommaInLiteral:
+  EnforcedStyleForMultiline: comma
+Style/WordArray:
+  Enabled: false
+  
+Lint/ShadowingOuterLocalVariable:
+  Enabled: false
+#Style/AccessorMethodName:
+#  Enabled: false
+#Style/MethodLength:
+#  Max: 30
+#Style/ModuleLength:
+#  Max: 150
+Style/PerlBackrefs:
+  Enabled: false
diff --git a/cookbooks/zookeeper-grid/CHANGELOG.md b/cookbooks/zookeeper-grid/CHANGELOG.md
new file mode 100644 (file)
index 0000000..2b05797
--- /dev/null
@@ -0,0 +1,9 @@
+# CHANGELOG for zookeeper
+
+0.1.1
+-----
+- refactoring.
+
+0.1.0
+-----
+- Initial release of zookeeper
diff --git a/cookbooks/zookeeper-grid/Gemfile b/cookbooks/zookeeper-grid/Gemfile
new file mode 100644 (file)
index 0000000..907b097
--- /dev/null
@@ -0,0 +1,3 @@
+source 'https://rubygems.org'
+
+#gem 'foodcritic'
diff --git a/cookbooks/zookeeper-grid/README.md b/cookbooks/zookeeper-grid/README.md
new file mode 100644 (file)
index 0000000..1ccb017
--- /dev/null
@@ -0,0 +1,194 @@
+zookeeper Cookbook
+==================
+
+This cookbook sets up Zookeeper ensemble (cluster).
+
+## Contents
+
+- [Requirements](#requirements)
+    - [cookbooks](#cookbooks)
+    - [packages](#packages)
+- [Attributes](#attributes)
+- [Usage](#usage)
+    - [Recipes](#recipes)
+        - [zookeeper::default](#zookeeperdefault)
+    - [Role examples](#role-examples)
+- [License and Authors](#license-and-authors)
+
+## Requirements
+
+### cookbooks
+- grid
+- java
+
+### packages
+none.
+
+## Attributes
+
+|Key|Type|Description (with examples)|Default|
+|:--|:--|:--|:--|
+|`['zookeeper']['install_flavor']`|String|distribution.|`'apache'`|
+|`['zookeeper']['version']`|String|Zookeeper's version.|`'3.4.5'`|
+|`['zookeeper']['archive_url']`|String|archive URL.|`'http://archive.apache.org/dist/zookeeper'`|
+|`['zookeeper']['colo_name']`|String|colocation name|`'localhost'`|
+|`['zookeeper']['member_of_hadoop']`|Boolean|whether a member of hadoop or not.|`false`|
+|`['zookeeper']['run_mode']`|String|`'standalone'`, `'pseudo-replicated'`, `'full-replicated'`|`'standalone'`|
+|`['zookeeper']['with_security']`|Boolean|work with security.|`false`|
+|`['zookeeper']['realm']`|String|Kerberos realm.|`'LOCALDOMAIN'`|
+|`['zookeeper']['keytab_dir']`|String|keytab stored directory.|`"#{node['grid']['etc_root']}/keytabs/#{node['zookeeper']['colo_name']}"`|
+|`['zookeeper']['ZOOKEEPER_PREFIX']`|String|property in the zookeeper-env.sh file.|`"#{node['grid']['app_root']}/zookeeper"`|
+|`['zookeeper']['ZOO_LOG_DIR_PREFIX']`|String||`"#{node['grid']['vol_root']}/0/var/log/zookeeper"`|
+|`['zookeeper']['dataDirPrefix']`|String|property in the zoo.cfg file.|`"#{node['grid']['vol_root']}/0/var/lib/zookeeper"`|
+|`['zookeeper']['dataLogDir']`|String||`nil`|
+|`['zookeeper']['clientPort']`|String||`'2181'`|
+|`['zookeeper']['ensemble']`|Hash|see attributes/default.rb|`{}` (empty)|
+|`['zookeeper']['tickTime']`|String||`'2000'`|
+|`['zookeeper']['initLimit']`|String||`'10'`|
+|`['zookeeper']['syncLimit']`|String||`'5'`|
+|`['zookeeper']['authProvider']['0']`|String|if it works with_security, the following properties are activated.|`'org.apache.zookeeper.server.auth.SASLAuthenticationProvider'`|
+|`['zookeeper']['jaasLoginRenew']`|String||`'3600000'`|
+|`['zookeeper']['kerberos.removeHostFromPrincipal']`|String||`'true'`|
+|`['zookeeper']['kerberos.removeRealmFromPrincipal']`|String||`'true'`|
+|`['zookeeper']['extra_configs']['zookeeper-env.sh']`|Hash|extra properties. key and value pairs|`{}` (empty)|
+|`['zookeeper']['extra_configs']['zoo.cfg']`|Hash|extra properties. key and value pairs|`{}` (empty)|
+|`['zookeeper']['extra_configs']['java.env']`|Hash|extra properties. key and value pairs|`{}` (empty)|
+
+## Usage
+
+### Recipes
+
+#### zookeeper::default
+- installs Zookeeper and sets up ensemble's nodes.
+
+### Role examples
+
+- `roles/zookeeper.rb`
+
+```ruby
+name 'zookeeper'
+description 'ZooKeeper node'
+
+run_list(
+  'recipe[grid]',
+  'recipe[java]',
+  'recipe[zookeeper]'
+)
+
+default_attributes(
+  'java' => {
+    'install_flavor' => 'oracle',
+    'jdk_version' => '7',
+    'java_home' => '/usr/local/jvm/java-7-ora',
+    'oracle' => {
+      'accept_oracle_download_terms' => true,
+    },
+  }
+)
+
+override_attributes(
+  'java' => {
+    'jdk' => {
+      '7' => {
+        # no update-alternatives
+        'bin_cmds' => [],
+      },
+    },
+  }
+)
+```
+
+- `roles/zookeeper-standalone.rb`
+
+```ruby
+name 'zookeeper-standalone'
+description 'ZooKeeper standalone mode'
+
+run_list(
+  'role[zookeeper]'
+)
+
+default_attributes(
+  'zookeeper' => {
+    'run_mode' => 'standalone',
+    'member_of_hadoop' => true,
+  }
+)
+```
+
+- `roles/zookeeper-pseudo-replicated.rb`
+
+```ruby
+name 'zookeeper-pseudo-replicated'
+description 'ZooKeeper pseudo replicated mode'
+
+run_list(
+  'role[zookeeper]'
+)
+
+default_attributes(
+  'zookeeper' => {
+    'run_mode' => 'pseudo-replicated',
+    'member_of_hadoop' => true,
+    'clientPort' => '2180',
+  }
+)
+```
+
+- `roles/zookeeper-colo00.rb`
+
+```ruby
+name 'zookeeper-colo00'
+description 'ZooKeeper colo00 node'
+
+run_list(
+  'role[zookeeper]'
+)
+
+ensemble = {
+  '0' => {
+    :hostname => 'zk00.grid.example.com',
+    :leader_port => '2888',
+    :election_port => '3888',
+  },
+  '1' => {
+    :hostname => 'zk01.grid.example.com',
+    :leader_port => '2888',
+    :election_port => '3888',
+  },
+  '2' => {
+    :hostname => 'zk02.grid.example.com',
+    :leader_port => '2888',
+    :election_port => '3888',
+  }
+}
+
+default_attributes(
+  'zookeeper' => {
+    'run_mode' => 'full-replicated',
+    'colo_name' => 'colo00',
+    'dataLogDir' => '/grid/vol/1/var/lib/zookeeper',
+    'ensemble' => ensemble,
+  }
+)
+```
+
+## License and Authors
+
+- Author:: whitestar at osdn.jp
+
+```text
+Copyright 2013-2017, whitestar
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+```
diff --git a/cookbooks/zookeeper-grid/Rakefile b/cookbooks/zookeeper-grid/Rakefile
new file mode 100644 (file)
index 0000000..513cecc
--- /dev/null
@@ -0,0 +1,23 @@
+require 'rspec/core/rake_task'
+require 'rubocop/rake_task'
+require 'foodcritic'
+
+namespace :style do
+  desc 'Run Ruby style checks'
+  RuboCop::RakeTask.new(:ruby)
+
+  desc 'Run Chef style checks'
+  FoodCritic::Rake::LintTask.new(:chef) do |t|
+    t.options = {
+      fail_tags: ['any'],
+    }
+  end
+end
+
+desc 'Run all style checks'
+task style: ['style:chef', 'style:ruby']
+
+desc 'Run ChefSpec examples'
+RSpec::Core::RakeTask.new(:spec)
+
+task default: ['style', 'spec']
diff --git a/cookbooks/zookeeper-grid/attributes/default.rb b/cookbooks/zookeeper-grid/attributes/default.rb
new file mode 100644 (file)
index 0000000..d65400d
--- /dev/null
@@ -0,0 +1,85 @@
+#
+# Cookbook Name:: zookeeper-grid
+# Attributes:: default
+#
+# Copyright 2013-2017, whitestar
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+# default: for pseudo-replicated
+default['zookeeper']['install_flavor'] = 'apache'
+default['zookeeper']['version'] = '3.4.5'
+default['zookeeper']['archive_url'] = 'http://archive.apache.org/dist/zookeeper'
+default['zookeeper']['colo_name'] = 'localhost'
+default['zookeeper']['member_of_hadoop'] = false
+## run mode: standalone, pseudo-replicated, full-replicated
+default['zookeeper']['run_mode'] = 'standalone'
+default['zookeeper']['with_security'] = false
+default['zookeeper']['realm'] = 'LOCALDOMAIN'
+default['zookeeper']['keytab_dir'] = "#{node['grid']['etc_root']}/keytabs/#{node['zookeeper']['colo_name']}"
+## zookeeper-env.sh
+default['zookeeper']['ZOOKEEPER_PREFIX'] = "#{node['grid']['app_root']}/zookeeper"
+default['zookeeper']['ZOO_LOG_DIR_PREFIX'] = "#{node['grid']['vol_root']}/0/var/log/zookeeper"
+## zoo.cfg
+default['zookeeper']['dataDirPrefix'] = "#{node['grid']['vol_root']}/0/var/lib/zookeeper"
+default['zookeeper']['dataLogDir'] = nil
+default['zookeeper']['clientPort'] = '2181'
+default['zookeeper']['ensemble'] = {
+}
+default['zookeeper']['tickTime'] = '2000'
+default['zookeeper']['initLimit'] = '10'
+default['zookeeper']['syncLimit'] = '5'
+### if with_security
+default['zookeeper']['authProvider'] = {
+  '0' => 'org.apache.zookeeper.server.auth.SASLAuthenticationProvider',
+}
+default['zookeeper']['jaasLoginRenew'] = '3600000'
+default['zookeeper']['kerberos.removeHostFromPrincipal'] = 'true'
+default['zookeeper']['kerberos.removeRealmFromPrincipal'] = 'true'
+## extra settings
+default['zookeeper']['extra_configs'] = {
+  # e.g. 'zoo.cfg' => {'k1' => 'v1', 'k2' => 'v2'},
+  'zookeeper-env.sh' => {},
+  'zoo.cfg' => {},
+  'java.env' => {},
+}
+#default['zookeeper'][''] =
+
+=begin
+# e.g. for pseudo-replicated
+default['zookeeper']['run_mode'] = 'pseudo-replicated'
+default['zookeeper']['clientPort'] = '2180'
+
+# e.g. for full-replicated
+default['zookeeper']['run_mode'] = 'full-replicated'
+default['zookeeper']['colo_name'] = 'colo00'
+default['zookeeper']['realm'] = 'GRID.EXAMPLE.COM'
+default['zookeeper']['ensemble'] = {
+  '0' => {
+    :hostname => 'zk00.grid.example.com',
+    :leader_port => '2888',
+    :election_port => '3888'
+  },
+  '1' => {
+    :hostname => 'zk01.grid.example.com',
+    :leader_port => '2888',
+    :election_port => '3888'
+  },
+  '2' => {
+    :hostname => 'zk02.grid.example.com',
+    :leader_port => '2888',
+    :election_port => '3888'
+  }
+}
+=end
diff --git a/cookbooks/zookeeper-grid/metadata.rb b/cookbooks/zookeeper-grid/metadata.rb
new file mode 100644 (file)
index 0000000..6353ad2
--- /dev/null
@@ -0,0 +1,17 @@
+# $ knife supermarket share zookeeper-grid "Databases"
+name             'zookeeper-grid'
+maintainer       'whitestar'
+maintainer_email ''
+license          'Apache 2.0'
+description      'Installs/Configures Apache Zookeeper'
+long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
+version          '0.1.1'
+source_url       'http://scm.osdn.jp/gitroot/metasearch/grid-chef-repo.git'
+issues_url       'https://osdn.jp/projects/metasearch/ticket'
+
+%w( debian ubuntu centos redhat fedora ).each do |os|
+  supports os
+end
+
+depends          'grid'
+depends          'java'
diff --git a/cookbooks/zookeeper-grid/recipes/default.rb b/cookbooks/zookeeper-grid/recipes/default.rb
new file mode 100644 (file)
index 0000000..bdb717a
--- /dev/null
@@ -0,0 +1,316 @@
+#
+# Cookbook Name:: zookeeper-grid
+# Recipe:: default
+#
+# Copyright 2013-2017, whitestar
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+require 'digest/md5'
+
+users = {
+  hadoop:    { name: 'hadoop',    uid: 10_001 },
+  zookeeper: { name: 'zookeeper', uid: 10_050 },
+}
+
+conf_files = [
+  'configuration.xsl',
+  'jaas.conf',
+  'java.env',
+  'log4j.properties',
+  'zoo.cfg',
+  'zoo_sample.cfg',
+  'zookeeper-env.sh',
+]
+
+def conf_template(conf_dir, conf_files, tpl_vars)
+  conf_files.each {|conf_file|
+    template "#{conf_dir}/#{conf_file}" do
+      source "conf/#{conf_file}"
+      #source "conf-#{middle_version}/#{conf_file}"
+      owner 'root'
+      group 'root'
+      mode '0644'
+      variables(tpl_vars)
+    end
+  }
+end
+
+version = node['zookeeper']['version']
+
+if /^(\d+)\.(\d+)\.(\d+)$/ =~ version
+  major_version = $1
+  middle_version = "#{$1}.#{$2}"
+  Chef::Application.info("ZooKeeper version: major (#{major_version}), middle (#{middle_version})")
+else
+  Chef::Application.fatal!("Invalid ZooKeeper version: #{version}")
+end
+
+users.each {|key, user|
+  next if key == :hadoop
+
+  group user[:name] do
+    gid user[:uid]
+    members []
+    action :create
+    not_if "getent group #{user[:name]}"
+  end
+
+  user user[:name] do
+    uid user[:uid]
+    gid user[:uid]
+    home "/home/#{user[:name]}"
+    shell '/bin/sh'
+    password nil
+    supports manage_home: false
+    not_if "getent passwd #{user[:name]}"
+  end
+}
+
+if node['zookeeper']['member_of_hadoop']
+  g_id = nil
+  act = nil
+  if shell_out("getent group #{users[:hadoop][:name]}").exitstatus == 0
+    g_id = nil
+    act = :modify
+  else
+    g_id = users[:hadoop][:uid]
+    act = :create
+  end
+
+  group 'add_zookeeper_to_hadoop' do
+    group_name users[:hadoop][:name]
+    gid g_id
+    members ['zookeeper']
+    append true
+    action act
+  end
+end
+
+file_cache_path = Chef::Config[:file_cache_path]
+install_root = "#{node['grid']['app_root']}/zookeeper-#{version}"
+tarball = "zookeeper-#{version}.tar.gz"
+tarball_md5 = "#{tarball}.md5"
+#tarball_mds = "#{tarball}.mds"
+downloaded_tarball = "#{file_cache_path}/#{tarball}"
+downloaded_tarball_md5 = "#{file_cache_path}/#{tarball_md5}"
+#downloaded_tarball_mds = "#{file_cache_path}/#{tarball_mds}"
+
+archive_url = node['zookeeper']['archive_url']
+unless FileTest.directory? install_root
+  remote_file downloaded_tarball_md5 do
+    source "#{archive_url}/zookeeper-#{version}/#{tarball_md5}"
+    action :create_if_missing
+  end
+=begin
+  remote_file downloaded_tarball_mds do
+    source "#{archive_url}/zookeeper-#{version}/#{tarball_mds}"
+    action :create_if_missing
+  end
+=end
+
+  remote_file downloaded_tarball do
+    source "#{archive_url}/zookeeper-#{version}/#{tarball}"
+    action :create_if_missing
+  end
+
+  ruby_block "checksum #{downloaded_tarball}" do
+    block do
+      # e.g. md file format 'f64fef86c0bf2e5e0484d19425b22dcb  zookeeper-3.4.5.tar.gz'
+      /^(\w+)\s+.*$/ =~ File.read(downloaded_tarball_md5)
+      checksum = $1
+      Chef::Log.info "#{tarball}: MD5 = #{checksum}"
+      actual_checksum = Digest::MD5.file(downloaded_tarball).to_s
+      Chef::Log.info "#{tarball}: actual MD5 = #{actual_checksum}"
+      unless checksum.casecmp(actual_checksum)
+        Chef::Application.fatal!("Invalid MD5 checksum of #{downloaded_tarball}, expected: #{checksum}")
+      end
+=begin
+      checksum = File.read(downloaded_tarball_mds).
+        gsub(/(\s)+/, '').
+        scan(/#{tarball}:(.+?)=([0-9A-Z]+)/).
+        assoc('SHA256')[1]
+      Chef::Log.info "#{tarball}: SHA256 = #{checksum}"
+      if ! Digest::SHA256.file(downloaded_tarball).to_s.casecmp(checksum) then
+        Chef::Application.fatal!("Invalid SHA256 checksum of #{downloaded_tarball}, expected: #{checksum}")
+      end
+=end
+    end
+    action :create
+  end
+
+  pkg = 'tar'
+  resources(package: pkg) rescue package pkg do
+    action :install
+  end
+
+  bash 'install_zookeeper' do
+    code <<-EOC
+      tar xvzf #{downloaded_tarball} -C #{node['grid']['app_root']}
+    EOC
+    creates install_root
+  end
+end
+
+zk_prefix = node['zookeeper']['ZOOKEEPER_PREFIX']
+link zk_prefix do
+  to install_root
+  action [:delete, :create]
+end
+
+case node['zookeeper']['run_mode']
+when 'standalone'
+  node_id = 'standalone'
+  conf_dir = "#{install_root}/conf.#{node_id}"
+  directory conf_dir do
+    owner 'root'
+    group 'root'
+    mode '0755'
+    action :create
+    recursive true
+  end
+
+  %w( lib log ).each {|dir|
+    directory "#{node['grid']['vol_root']}/0/var/#{dir}/zookeeper/#{node_id}" do
+      owner 'zookeeper'
+      group 'zookeeper'
+      mode '0755'
+      action :create
+      recursive true
+    end
+  }
+
+  tpl_vars = {
+    node_id: node_id,
+    conf_dir: conf_dir,
+    clientPort: node['zookeeper']['clientPort'],
+  }
+  conf_template(conf_dir, conf_files, tpl_vars)
+
+  log <<-EOM
+Note:
+Start command:
+$ cd $ZOOKEEPER_PREFIX
+$ sudo -u zookeeper sh -c 'export ZOOCFGDIR=/grid/usr/zookeeper/conf.standalone;./bin/zkServer.sh start'
+  EOM
+when 'pseudo-replicated'
+  client_port = node['zookeeper']['clientPort'].to_i
+  3.times {|node_id|
+    conf_dir = "#{install_root}/conf.#{node_id}"
+    directory conf_dir do
+      owner 'root'
+      group 'root'
+      mode '0755'
+      action :create
+      recursive true
+    end
+
+    %w( lib log ).each {|dir|
+      directory "#{node['grid']['vol_root']}/0/var/#{dir}/zookeeper/#{node_id}" do
+        owner 'zookeeper'
+        group 'zookeeper'
+        mode '0755'
+        action :create
+        recursive true
+      end
+    }
+
+    myid_file = "#{node['grid']['vol_root']}/0/var/lib/zookeeper/#{node_id}/myid"
+    bash "setup_myid_#{node_id}" do
+      code <<-EOC
+        echo #{node_id} > #{myid_file}
+      EOC
+      creates myid_file
+    end
+
+    tpl_vars = {
+      node_id: node_id,
+      conf_dir: conf_dir,
+      clientPort: client_port,
+    }
+    conf_template(conf_dir, conf_files, tpl_vars)
+
+    client_port += 1
+  }
+
+  log <<-EOM
+Note:
+Start command:
+$ cd $ZOOKEEPER_PREFIX
+$ sudo -u zookeeper sh -c 'export ZOOCFGDIR=/grid/usr/zookeeper/conf.0;./bin/zkServer.sh start'
+$ sudo -u zookeeper sh -c 'export ZOOCFGDIR=/grid/usr/zookeeper/conf.1;./bin/zkServer.sh start'
+$ sudo -u zookeeper sh -c 'export ZOOCFGDIR=/grid/usr/zookeeper/conf.2;./bin/zkServer.sh start'
+  EOM
+when 'full-replicated'
+  conf_dir = "#{install_root}/conf"
+  %w( lib log ).each {|dir|
+    directory "#{node['grid']['vol_root']}/0/var/#{dir}/zookeeper" do
+      owner 'zookeeper'
+      group 'zookeeper'
+      mode '0755'
+      action :create
+      recursive true
+    end
+  }
+
+  data_log_dir = node['zookeeper']['dataLogDir']
+  directory data_log_dir do
+    owner 'zookeeper'
+    group 'zookeeper'
+    mode '0755'
+    action :create
+    recursive true
+    only_if { !data_log_dir.nil? && data_log_dir != '' }
+  end
+
+  tpl_vars = {
+    node_id: '',
+    conf_dir: conf_dir,
+    clientPort: node['zookeeper']['clientPort'],
+  }
+  conf_template(conf_dir, conf_files, tpl_vars)
+
+  node_id = ''
+  node['zookeeper']['ensemble'].each {|id, server|
+    if server['hostname'] == node['fqdn']
+      node_id = id
+      break
+    end
+  }
+  myid_file = "#{node['grid']['vol_root']}/0/var/lib/zookeeper/myid"
+  bash "setup_myid_#{node_id}" do
+    code <<-EOC
+      echo #{node_id} > #{myid_file}
+    EOC
+    creates myid_file
+  end
+
+  log <<-EOM
+Note:
+Start command:
+$ cd $ZOOKEEPER_PREFIX
+$ sudo -u zookeeper ./bin/zkServer.sh start
+  EOM
+end
+
+# with security
+directory "#{node['zookeeper']['keytab_dir']} for zookeeper" do
+  path node['zookeeper']['keytab_dir']
+  owner 'root'
+  group 'root'
+  mode '0755'
+  action :create
+  recursive true
+  only_if { node['zookeeper']['with_security'] }
+end
diff --git a/cookbooks/zookeeper-grid/templates/default/conf/configuration.xsl b/cookbooks/zookeeper-grid/templates/default/conf/configuration.xsl
new file mode 100644 (file)
index 0000000..377cdbe
--- /dev/null
@@ -0,0 +1,24 @@
+<?xml version="1.0"?>
+<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
+<xsl:output method="html"/>
+<xsl:template match="configuration">
+<html>
+<body>
+<table border="1">
+<tr>
+ <td>name</td>
+ <td>value</td>
+ <td>description</td>
+</tr>
+<xsl:for-each select="property">
+<tr>
+  <td><a name="{name}"><xsl:value-of select="name"/></a></td>
+  <td><xsl:value-of select="value"/></td>
+  <td><xsl:value-of select="description"/></td>
+</tr>
+</xsl:for-each>
+</table>
+</body>
+</html>
+</xsl:template>
+</xsl:stylesheet>
diff --git a/cookbooks/zookeeper-grid/templates/default/conf/jaas.conf b/cookbooks/zookeeper-grid/templates/default/conf/jaas.conf
new file mode 100644 (file)
index 0000000..5aba56b
--- /dev/null
@@ -0,0 +1,25 @@
+Server {
+    com.sun.security.auth.module.Krb5LoginModule required
+        useKeyTab=true
+        keyTab="<%= node['zookeeper']['keytab_dir'] %>/zk.keytab"
+        storeKey=true
+        useTicketCache=false
+<%
+case node['zookeeper']['run_mode']
+  when 'standalone', 'pseudo-replicated'
+%>
+        principal="zookeeper/localhost@<%= node['zookeeper']['realm'] %>";
+<%
+  when 'full-replicated'
+%>
+        principal="zookeeper/${this.fqdn}@<%= node['zookeeper']['realm'] %>";
+<%
+end
+%>
+};
+Client {
+    com.sun.security.auth.module.Krb5LoginModule required
+        useKeyTab=false
+        useTicketCache=true
+        debug=true;
+};
diff --git a/cookbooks/zookeeper-grid/templates/default/conf/java.env b/cookbooks/zookeeper-grid/templates/default/conf/java.env
new file mode 100644 (file)
index 0000000..b426aa3
--- /dev/null
@@ -0,0 +1,17 @@
+<% if node['zookeeper']['with_security'] then %>
+FQDN=`hostname --fqdn`
+
+export JVMFLAGS="-Dthis.fqdn=${FQDN} -Djava.security.auth.login.config=<%= @conf_dir %>/jaas.conf"
+<% end %>
+
+<%
+this_file = 'java.env'
+if defined? node['zookeeper']['extra_configs'][this_file] \
+  && node['zookeeper']['extra_configs'][this_file] != nil then
+  node['zookeeper']['extra_configs'][this_file].each do |key, value|
+%>
+export <%= key %>=<%= value %>
+<%
+  end
+end
+%>
diff --git a/cookbooks/zookeeper-grid/templates/default/conf/log4j.properties b/cookbooks/zookeeper-grid/templates/default/conf/log4j.properties
new file mode 100644 (file)
index 0000000..9825970
--- /dev/null
@@ -0,0 +1,58 @@
+# Define some default values that can be overridden by system properties
+zookeeper.root.logger=INFO, CONSOLE
+zookeeper.console.threshold=INFO
+zookeeper.log.dir=.
+zookeeper.log.file=zookeeper.log
+zookeeper.log.threshold=DEBUG
+zookeeper.tracelog.dir=.
+zookeeper.tracelog.file=zookeeper_trace.log
+
+#
+# ZooKeeper Logging Configuration
+#
+
+# Format is "<default threshold> (, <appender>)+
+
+# DEFAULT: console appender only
+log4j.rootLogger=${zookeeper.root.logger}
+
+# Example with rolling log file
+#log4j.rootLogger=DEBUG, CONSOLE, ROLLINGFILE
+
+# Example with rolling log file and tracing
+#log4j.rootLogger=TRACE, CONSOLE, ROLLINGFILE, TRACEFILE
+
+#
+# Log INFO level and above messages to the console
+#
+log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
+log4j.appender.CONSOLE.Threshold=${zookeeper.console.threshold}
+log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout
+log4j.appender.CONSOLE.layout.ConversionPattern=%d{ISO8601} [myid:%X{myid}] - %-5p [%t:%C{1}@%L] - %m%n
+
+#
+# Add ROLLINGFILE to rootLogger to get log file output
+#    Log DEBUG level and above messages to a log file
+log4j.appender.ROLLINGFILE=org.apache.log4j.RollingFileAppender
+log4j.appender.ROLLINGFILE.Threshold=${zookeeper.log.threshold}
+log4j.appender.ROLLINGFILE.File=${zookeeper.log.dir}/${zookeeper.log.file}
+
+# Max log file size of 10MB
+log4j.appender.ROLLINGFILE.MaxFileSize=10MB
+# uncomment the next line to limit number of backup files
+#log4j.appender.ROLLINGFILE.MaxBackupIndex=10
+
+log4j.appender.ROLLINGFILE.layout=org.apache.log4j.PatternLayout
+log4j.appender.ROLLINGFILE.layout.ConversionPattern=%d{ISO8601} [myid:%X{myid}] - %-5p [%t:%C{1}@%L] - %m%n
+
+
+#
+# Add TRACEFILE to rootLogger to get log file output
+#    Log DEBUG level and above messages to a log file
+log4j.appender.TRACEFILE=org.apache.log4j.FileAppender
+log4j.appender.TRACEFILE.Threshold=TRACE
+log4j.appender.TRACEFILE.File=${zookeeper.tracelog.dir}/${zookeeper.tracelog.file}
+
+log4j.appender.TRACEFILE.layout=org.apache.log4j.PatternLayout
+### Notice we are including log4j's NDC here (%x)
+log4j.appender.TRACEFILE.layout.ConversionPattern=%d{ISO8601} [myid:%X{myid}] - %-5p [%t:%C{1}@%L][%x] - %m%n
diff --git a/cookbooks/zookeeper-grid/templates/default/conf/zoo.cfg b/cookbooks/zookeeper-grid/templates/default/conf/zoo.cfg
new file mode 100644 (file)
index 0000000..136cd50
--- /dev/null
@@ -0,0 +1,78 @@
+# The number of milliseconds of each tick
+tickTime=<%= node['zookeeper']['tickTime'] %>
+# The number of ticks that the initial 
+# synchronization phase can take
+initLimit=<%= node['zookeeper']['initLimit'] %>
+# The number of ticks that can pass between 
+# sending a request and getting an acknowledgement
+syncLimit=<%= node['zookeeper']['syncLimit'] %>
+# the directory where the snapshot is stored.
+# do not use /tmp for storage, /tmp here is just 
+# example sakes.
+#dataDir=/tmp/zookeeper
+dataDir=<%= node['zookeeper']['dataDirPrefix'] %><%= @node_id == '' ? '' : "/#{@node_id}"%>
+<%
+dataLogDir = node['zookeeper']['dataLogDir']
+if dataLogDir != nil && dataLogDir != '' then
+%>
+dataLogDir=<%= dataLogDir %>
+<% end %>
+# the port at which the clients will connect
+clientPort=<%= @clientPort %>
+<%
+if node['zookeeper']['with_security'] then
+  node['zookeeper']['authProvider'].each do |id, class_name|
+%>
+authProvider.<%= id %>=<%= class_name %>
+<%
+  end
+%>
+jaasLoginRenew=<%= node['zookeeper']['jaasLoginRenew'] %>
+kerberos.removeHostFromPrincipal=<%= node['zookeeper']['kerberos.removeHostFromPrincipal'] %>
+kerberos.removeRealmFromPrincipal=<%= node['zookeeper']['kerberos.removeRealmFromPrincipal'] %>
+# available?!
+#requireClientAuthScheme=sasl
+#allowSaslFailedClients=false
+#maintain_connection_despite_sasl_failure=false
+<% end %>
+#
+# Be sure to read the maintenance section of the 
+# administrator guide before turning on autopurge.
+#
+# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
+#
+# The number of snapshots to retain in dataDir
+#autopurge.snapRetainCount=3
+# Purge task interval in hours
+# Set to "0" to disable auto purge feature
+#autopurge.purgeInterval=1
+
+<%
+this_file = 'zoo.cfg'
+if defined? node['zookeeper']['extra_configs'][this_file] \
+  && node['zookeeper']['extra_configs'][this_file] != nil then
+  node['zookeeper']['extra_configs'][this_file].each do |key, value|
+%>
+<%= key %>=<%= value %>
+<%
+  end
+end
+%>
+
+<%
+case node['zookeeper']['run_mode']
+  when 'pseudo-replicated'
+%>
+# Pseudo ensemble mode on single host
+server.0=localhost:2888:3888
+server.1=localhost:2889:3889
+server.2=localhost:2890:3890
+<%
+  when 'full-replicated'
+    node['zookeeper']['ensemble'].each do |id, server|
+%>
+server.<%= id %>=<%= server[:hostname] %>:<%= server[:leader_port] %>:<%= server[:election_port] %>
+<%
+    end
+end
+%>
diff --git a/cookbooks/zookeeper-grid/templates/default/conf/zoo_sample.cfg b/cookbooks/zookeeper-grid/templates/default/conf/zoo_sample.cfg
new file mode 100644 (file)
index 0000000..aafb324
--- /dev/null
@@ -0,0 +1,25 @@
+# The number of milliseconds of each tick
+tickTime=2000
+# The number of ticks that the initial 
+# synchronization phase can take
+initLimit=10
+# The number of ticks that can pass between 
+# sending a request and getting an acknowledgement
+syncLimit=5
+# the directory where the snapshot is stored.
+# do not use /tmp for storage, /tmp here is just 
+# example sakes.
+dataDir=/tmp/zookeeper
+# the port at which the clients will connect
+clientPort=2181
+#
+# Be sure to read the maintenance section of the 
+# administrator guide before turning on autopurge.
+#
+# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
+#
+# The number of snapshots to retain in dataDir
+#autopurge.snapRetainCount=3
+# Purge task interval in hours
+# Set to "0" to disable auto purge feature
+#autopurge.purgeInterval=1
diff --git a/cookbooks/zookeeper-grid/templates/default/conf/zookeeper-env.sh b/cookbooks/zookeeper-grid/templates/default/conf/zookeeper-env.sh
new file mode 100644 (file)
index 0000000..42b1766
--- /dev/null
@@ -0,0 +1,15 @@
+export JAVA_HOME=<%= node['java']['java_home'] %>
+
+export ZOO_LOG_DIR=<%= node['zookeeper']['ZOO_LOG_DIR_PREFIX'] %><%= @node_id == '' ? '' : "/#{@node_id}"%>
+
+<%
+this_file = 'zookeeper-env.sh'
+if defined? node['zookeeper']['extra_configs'][this_file] \
+  && node['zookeeper']['extra_configs'][this_file] != nil then
+  node['zookeeper']['extra_configs'][this_file].each do |key, value|
+%>
+export <%= key %>=<%= value %>
+<%
+  end
+end
+%>