cookbooks/screwdriver/README.md

   1 screwdriver Cookbook
   2 ====================
   3
   4 This cookbook sets up a Screwdriver CI/CD service by Docker Compose.
   5
   6 ## Contents
   7
   8 - [Requirements](#requirements)
   9     - [platforms](#platforms)
  10     - [packages](#packages)
  11     - [cookbooks](#cookbooks)
  12 - [Attributes](#attributes)
  13 - [Usage](#usage)
  14     - [Recipes](#recipes)
  15         - [screwdriver::default](#screwdriverdefault)
  16         - [screwdriver::docker-compose](#screwdriverdocker-compose)
  17     - [Role Examples](#role-examples)
  18     - [SSL server keys and certificates management by ssl_cert cookbook](#ssl-server-keys-and-certificates-management-by-ssl_cert-cookbook)
  19     - [JWT private and public keys management by Chef Vault](#jwt-private-and-public-keys-management-by-chef-vault)
  20     - [Cookie password management by Chef Vault](#cookie-password-management-by-chef-vault)
  21     - [Secrets encryption password management by Chef Vault](#secrets-encryption-password-management-by-chef-vault)
  22     - [OAuth client ID and secret management by Chef Vault](#oauth-client-id-and-secret-management-by-chef-vault)
  23     - [GitHub webhook secret management by Chef Vault](#github-webhook-secret-management-by-chef-vault)
  24 - [License and Authors](#license-and-authors)
  25
  26 ## Requirements
  27
  28 ### platforms
  29 - Debian >= 9.0
  30 - Ubuntu >= 14.04
  31 - CentOS, RHEL >= 7.3
  32
  33 ### packages
  34 - none.
  35
  36 ### cookbooks
  37 - `docker-grid`
  38 - `ssl_cert`
  39
  40 ## Attributes
  41
  42 |Key|Type|Description, example|Default|
  43 |:--|:--|:--|:--|
  44 |`['screwdriver']['with_ssl_cert_cookbook']`|Boolean|See `attributes/default.rb`|`false`|
  45 |`['screwdriver']['ssl_cert']['ca_names']`|Array|Internal CA names that are imported by the ssl_cert cookbook.|`[]`|
  46 |`['screwdriver']['ssl_cert']['common_name']`|String|Server common name for TLS|`node['fqdn']`|
  47 |`['screwdriver']['jwt_private_key_vault_item']`|Hash|Optional, Sets a JWT private key from Chef Vault. See `attributes/default.rb`|`{}`|
  48 |`['screwdriver']['jwt_public_key_vault_item']`|Hash|Optional, Sets a JWT public key from Chef Vault. See `attributes/default.rb`|`{}`|
  49 |`['screwdriver']['cookie_password_vault_item']`|Hash|Optional, Sets a session cookie password from Chef Vault. See `attributes/default.rb`|`{}`|
  50 |`['screwdriver']['password_vault_item']`|Hash|Optional, Sets a password for secrets encryption from Chef Vault. See `attributes/default.rb`|`{}`|
  51 |`['screwdriver']['ui']['tls_setup_mode']`|String|`'reverseproxy'` only. Note: [_Add TLS support to UI docker container #377_](https://github.com/screwdriver-cd/screwdriver/issues/377)|`'reverseproxy'`|
  52 |`['screwdriver']['api']['config']`|Hash|This hash object is expanded to a `/config/local.yaml` file in the API Docker container.|See `attributes/default.rb`|
  53 |`['screwdriver']['api']['scms_vault_items']`|Hash|This hash contains Chef Vault item definitions of SCM's secrets.|See `attributes/default.rb`|
  54 |`['screwdriver']['store']['config']`|Hash|This hash object is expanded to a `/config/local.yaml` file in the Store Docker container.|See `attributes/default.rb`|
  55 |`['screwdriver']['docker-compose']['import_ca']`|Boolean|whether import internal CA certificates or not.|`false`|
  56 |`['screwdriver']['docker-compose']['app_dir']`|String|Path string.|`"#{node['docker-grid']['compose']['app_dir']}/screwdriver"`|
  57 |`['screwdriver']['docker-compose']['bin_dir']`|String|Path string.|`"#{node['screwdriver']['docker-compose']['app_dir']}/bin"`|
  58 |`['screwdriver']['docker-compose']['config_dir']`|String|Path string.|`"#{node['screwdriver']['docker-compose']['app_dir']}/config"`|
  59 |`['screwdriver']['docker-compose']['data_dir']`|String|Path string.|`"#{node['screwdriver']['docker-compose']['app_dir']}/data"`|
  60 |`['screwdriver']['docker-compose']['etc_dir']`|String|Path string.|`"#{node['screwdriver']['docker-compose']['app_dir']}/etc"`|
  61 |`['screwdriver']['docker-compose']['jwt_private_key_reset']`|Boolean|Only available if the JWT key pair is automatically generated by Chef.|`false`|
  62 |`['screwdriver']['docker-compose']['jwt_private_key_vault_item']`|Hash|**DEPRECATED**: use `['screwdriver']['jwt_private_key_vault_item']`. Optional, Sets a JWT private key from Chef Vault. See `attributes/default.rb`|`{}`|
  63 |`['screwdriver']['docker-compose']['jwt_public_key_vault_item']`|Hash|**DEPRECATED**: use `['screwdriver']['jwt_public_key_vault_item']`. Optional, Sets a JWT public key from Chef Vault. See `attributes/default.rb`|`{}`|
  64 |`['screwdriver']['docker-compose']['cookie_password_vault_item']`|Hash|**DEPRECATED**: use `['screwdriver']['cookie_password_vault_item']`. Optional, Sets a session cookie password from Chef Vault. See `attributes/default.rb`|`{}`|
  65 |`['screwdriver']['docker-compose']['password_vault_item']`|Hash|**DEPRECATED**: use `['screwdriver']['password_vault_item']`. Optional, Sets a password for secrets encryption from Chef Vault. See `attributes/default.rb`|`{}`|
  66 |`['screwdriver']['docker-compose']['oauth_client_id_vault_item']`|Hash|**DEPRECATED**: use `['screwdriver']['api']['scms_vault_items']`. Required, Sets a OAuth client ID for SCM from Chef Vault. See `attributes/default.rb`|`{}`|
  67 |`['screwdriver']['docker-compose']['oauth_client_secret_vault_item']`|Hash|**DEPRECATED**: use `['screwdriver']['api']['scms_vault_items']`. Required, Sets a OAuth secret for SCM from Chef Vault. See `attributes/default.rb`|`{}`|
  68 |`['screwdriver']['docker-compose']['webhook_github_secret_vault_item']`|Hash|**DEPRECATED**: use `['screwdriver']['api']['scms_vault_items']`. Required for GitHub, Sets a secret for GitHub webhook from Chef Vault. See `attributes/default.rb`|`{}`|
  69 |`['screwdriver']['docker-compose']['config']`|Hash|`docker-compose.yml` configurations.|See `attributes/default.rb`|
  70
  71 ## Usage
  72
  73 ### Recipes
  74
  75 #### screwdriver::default
  76
  77 This recipe does nothing.
  78
  79 #### screwdriver::docker-compose
  80
  81 This recipe generates JWT key pair and a `docker-compose.yml` file for the Screwdriver CI/CD service.
  82
  83 ### Role Examples
  84
  85 - `roles/screwdriver.rb`
  86
  87 ```ruby
  88 name 'screwdriver'
  89 description 'screwdriver'
  90
  91 ui_port     = '9000'
  92 api_port    = '9001'
  93 store_port  = '9002'
  94
  95 run_list(
  96   'role[docker]',
  97   'recipe[screwdriver::docker-compose]',
  98 )
  99
 100 override_attributes(
 101   'screwdriver' => {
 102     'api' => {
 103       'config' => {
 104         'executor' => {
 105           'plugin' => 'docker',
 106           'docker' => {
 107             'options' => {
 108               'docker' => {
 109                 'socketPath' => '/var/run/docker.sock',
 110               },
 111               'launchVersion' => 'stable',
 112             },
 113           },
 114         },
 115         'scms' => {
 116           'github.com' => {
 117             'plugin' => 'github',
 118             'config' => {
 119               # OAuth Callback URL: "http://#{node['fqdn']}:9001/v4/auth/login/web"
 120               'username' => 'ci-tool',
 121               'email' => 'citool@mail.example.com',
 122               'privateRepo' => false,
 123             },
 124           },
 125         },
 126       },
 127       'scms_vault_items' => {
 128         'github.com' => {
 129           'oauthClientId' => {
 130             'vault' => 'screwdriver',
 131             'name' => 'github',
 132             'env_context' => false,
 133             'key' => 'oauthClientId',  # real hash path: "/oauthClientId"
 134           },
 135           'oauthClientSecret' => {
 136             'vault' => 'screwdriver',
 137             'name' => 'github',
 138             'env_context' => false,
 139             'key' => 'oauthClientSecret',  # real hash path: "/oauthClientSecret"
 140           },
 141           'secret' => {
 142             'vault' => 'screwdriver',
 143             'name' => 'github',
 144             'env_context' => false,
 145             'key' => 'secret',  # real hash path: "/secret"
 146           },
 147         },
 148       },
 149     },
 150     'docker-compose' => {
 151       'config' => {
 152         'services' => {
 153           'api' => {
 154             'ports' => [
 155               "#{api_port}:80",
 156             ],
 157             'environment' => {
 158               'NODE_TLS_REJECT_UNAUTHORIZED' => '0',  # for self-signed cetificates
 159               # The following variables will be set by the screwdriver::docker-compose recipe automatically.
 160               #'ECOSYSTEM_UI' => "http://#{node['fqdn']}:#{ui_port}",
 161               #'ECOSYSTEM_STORE' => "http://#{node['fqdn']}:#{store_port}",
 162             },
 163           },
 164           'ui' => {
 165             'ports' => [
 166               "#{ui_port}:80",
 167             ],
 168             'environment' => {
 169               # These variables will be set by the screwdriver::docker-compose recipe automatically.
 170               #'ECOSYSTEM_API' => "http://#{node['fqdn']}:#{api_port}",
 171               #'ECOSYSTEM_STORE' => "http://#{node['fqdn']}:#{store_port}",
 172             },
 173           },
 174           'store' => {
 175             'ports' => [
 176               "#{store_port}:80",
 177             ],
 178             'environment' => {
 179               # This variable will be set by the screwdriver::docker-compose recipe automatically.
 180               #'ECOSYSTEM_UI' => "http://#{node['fqdn']}:#{ui_port}",
 181             },
 182           },
 183         },
 184       },
 185     },
 186   },
 187 )
 188 ```
 189
 190 - `roles/screwdriver-with-ssl.rb`
 191
 192 ```ruby
 193 name 'screwdriver-with-ssl'
 194 description 'screwdriver with SSL'
 195
 196 cn = 'screwdriver.io.example.com'
 197 ui_port     = '9000'
 198 api_port    = '9001'
 199 store_port  = '9002'
 200
 201 run_list(
 202   'role[docker]',
 203   'recipe[screwdriver::docker-compose]',
 204 )
 205
 206 override_attributes(
 207   'ssl_cert' => {
 208     #'common_names' => [
 209     #  cn,  # screwdriver cookbook < 0.2.2
 210     #],
 211   },
 212   'screwdriver' => {
 213     'with_ssl_cert_cookbook' => true,
 214     'ssl_cert' => {
 215       'common_name' => cn,
 216     },
 217     'api' => {
 218       'config' => {
 219         'executor' => {
 220           'plugin' => 'docker',
 221           'docker' => {
 222             'options' => {
 223               'docker' => {
 224                 'socketPath' => '/var/run/docker.sock',
 225               },
 226               'launchVersion' => 'stable',
 227             },
 228           },
 229         },
 230         'scms' => {
 231           'github.com' => {
 232             'plugin' => 'github',
 233             'config' => {
 234               # OAuth Callback URL: "http://#{node['fqdn']}:9001/v4/auth/login/web"
 235               'username' => 'ci-tool',
 236               'email' => 'citool@mail.example.com',
 237               'privateRepo' => false,
 238             },
 239           },
 240         },
 241       },
 242       'scms_vault_items' => {
 243         'github.com' => {
 244           'oauthClientId' => {
 245             'vault' => 'screwdriver',
 246             'name' => 'github',
 247             'env_context' => false,
 248             'key' => 'oauthClientId',  # real hash path: "/oauthClientId"
 249           },
 250           'oauthClientSecret' => {
 251             'vault' => 'screwdriver',
 252             'name' => 'github',
 253             'env_context' => false,
 254             'key' => 'oauthClientSecret',  # real hash path: "/oauthClientSecret"
 255           },
 256           'secret' => {
 257             'vault' => 'screwdriver',
 258             'name' => 'github',
 259             'env_context' => false,
 260             'key' => 'secret',  # real hash path: "/secret"
 261           },
 262         },
 263       },
 264     },
 265     'docker-compose' => {
 266       'config' => {
 267         'services' => {
 268           'reverseproxy' => {
 269             'ports' => [
 270               "#{ui_port}:9000",
 271             ],
 272             'environment' => {
 273             },
 274           },
 275           'api' => {
 276             'ports' => [
 277               "#{api_port}:80",
 278             ],
 279             'environment' => {
 280               'NODE_TLS_REJECT_UNAUTHORIZED' => '0',  # for self-signed cetificates
 281               # The following variables will be set by the screwdriver::docker-compose recipe automatically.
 282               #'ECOSYSTEM_UI' => "http://#{node['fqdn']}:#{ui_port}",
 283               #'ECOSYSTEM_STORE' => "http://#{node['fqdn']}:#{store_port}",
 284             },
 285           },
 286           'ui' => {
 287             #'ports' => [
 288             #  "#{ui_port}:80",
 289             #],
 290             'environment' => {
 291               # These variables will be set by the screwdriver::docker-compose recipe automatically.
 292               #'ECOSYSTEM_API' => "http://#{node['fqdn']}:#{api_port}",
 293               #'ECOSYSTEM_STORE' => "http://#{node['fqdn']}:#{store_port}",
 294             },
 295           },
 296           'store' => {
 297             'ports' => [
 298               "#{store_port}:80",
 299             ],
 300             'environment' => {
 301               # These variables will be set by the screwdriver::docker-compose recipe automatically.
 302               #'ECOSYSTEM_UI' => "http://#{node['fqdn']}:#{ui_port}",
 303             },
 304           },
 305         },
 306       },
 307     },
 308   },
 309 )
 310 ```
 311
 312 ### SSL server keys and certificates management by ssl_cert cookbook
 313
 314 - create vault items.
 315
 316 ```text
 317 $ ruby -rjson -e 'puts JSON.generate({"private" => File.read("screwdriver.io.example.com.prod.key")})' \
 318 > > ~/tmp/screwdriver.io.example.com.prod.key.json
 319
 320 $ ruby -rjson -e 'puts JSON.generate({"public" => File.read("screwdriver.io.example.com.prod.crt")})' \
 321 > > ~/tmp/screwdriver.io.example.com.prod.crt.json
 322
 323 $ cd $CHEF_REPO_PATH
 324
 325 $ knife vault create ssl_server_keys screwdriver.io.example.com.prod \
 326 > --json ~/tmp/screwdriver.io.example.com.prod.key.json
 327
 328 $ knife vault create ssl_server_certs screwdriver.io.example.com.prod \
 329 > --json ~/tmp/screwdriver.io.example.com.prod.crt.json
 330 ```
 331
 332 - grant reference permission to the screwdriver host
 333
 334 ```text
 335 $ knife vault update ssl_server_keys  screwdriver.io.example.com.prod -S 'name:screwdriver-host.example.com'
 336 $ knife vault update ssl_server_certs screwdriver.io.example.com.prod -S 'name:screwdriver-host.example.com'
 337 ```
 338
 339 - modify attributes
 340
 341 ```ruby
 342 override_attributes(
 343   'ssl_cert' => {
 344     #'common_names' => [
 345     #  'screwdriver.io.example.com',  # screwdriver cookbook < 0.2.2
 346     #],
 347   },
 348   'screwdriver' => {
 349     'with_ssl_cert_cookbook' => true,
 350     'ssl_cert' => {
 351       'common_name' => 'screwdriver.io.example.com',
 352     },
 353     # ...
 354   },
 355 )
 356 ```
 357
 358 ### JWT private and public keys management by Chef Vault
 359
 360 - create vault items.
 361
 362 ```text
 363 $ ruby -rjson -e 'puts JSON.generate({"private" => File.read("screwdriver_jwt_private.key")})' \
 364 > > ~/sec/tmp/screwdriver_jwt_private.key.json
 365
 366 $ ruby -rjson -e 'puts JSON.generate({"public" => File.read("screwdriver_jwt_public.key")})' \
 367 > > ~/sec/tmp/screwdriver_jwt_public.key.json
 368
 369 $ cd $CHEF_REPO_PATH
 370
 371 $ knife vault create screwdriver jwt_private_key \
 372 > --json ~/sec/tmp/screwdriver_jwt_private.key.json
 373
 374 $ knife vault create screwdriver screwdriver_jwt_public \
 375 > --json ~/sec/tmp/screwdriver_jwt_public.key.json
 376 ```
 377
 378 - grant reference permission to the screwdriver host
 379
 380 ```text
 381 $ knife vault update screwdriver jwt_private_key -S 'name:screwdriver-host.example.com'
 382 $ knife vault update screwdriver jwt_public_key  -S 'name:screwdriver-host.example.com'
 383 ```
 384
 385 - modify attributes
 386
 387 ```ruby
 388 override_attributes(
 389   'screwdriver' => {
 390     # ...
 391     'jwt_private_key_vault_item' => {
 392       'vault' => 'screwdriver',
 393       'name' => 'jwt_private_key',
 394       'env_context' => false,
 395       'key' => 'private',
 396     },
 397     'jwt_public_key_vault_item' => {
 398       'vault' => 'screwdriver',
 399       'name' => 'jwt_public_key',
 400       'env_context' => false,
 401       'key' => 'public',
 402     },
 403     # ...
 404   },
 405 )
 406 ```
 407
 408 ### Cookie password management by Chef Vault
 409
 410 - create vault items.
 411
 412 ```text
 413 # A password used for encrypting session data. Needs to be minimum 32 characters
 414 $ cat ~/tmp/screwdriver_cookie_password.json
 415 {"password":"********************************"}
 416
 417 $ cd $CHEF_REPO_PATH
 418 $ knife vault create screwdriver cookie_password --json ~/sec/tmp/screwdriver_cookie_password.json
 419 ```
 420
 421 - grant reference permission to the screwdriver host
 422
 423 ```text
 424 $ knife vault update screwdriver cookie_password -S 'name:screwdriver-host.example.com'
 425 ```
 426
 427 - modify attributes
 428
 429 ```ruby
 430 override_attributes(
 431   'screwdriver' => {
 432     # ...
 433     'cookie_password_vault_item' => {
 434       'vault' => 'screwdriver',
 435       'name' => 'cookie_password',
 436       'env_context' => false,
 437       'key' => 'password',
 438     },
 439     # ...
 440   },
 441 )
 442 ```
 443
 444 ### Secrets encryption password management by Chef Vault
 445
 446 - create vault items.
 447
 448 ```text
 449 # A password used for encrypting stored secrets. Needs to be minimum 32 characters
 450 $ cat ~/tmp/screwdriver_password.json
 451 {"password":"********************************"}
 452
 453 $ cd $CHEF_REPO_PATH
 454 $ knife vault create screwdriver password --json ~/sec/tmp/screwdriver_password.json
 455 ```
 456
 457 - grant reference permission to the screwdriver host
 458
 459 ```text
 460 $ knife vault update screwdriver password -S 'name:screwdriver-host.example.com'
 461 ```
 462
 463 - modify attributes
 464
 465 ```ruby
 466 override_attributes(
 467   'screwdriver' => {
 468     # ...
 469     'password_vault_item' => {
 470       'vault' => 'screwdriver',
 471       'name' => 'password',
 472       'env_context' => false,
 473       'key' => 'password',
 474     },
 475     # ...
 476   },
 477 )
 478 ```
 479
 480 ### OAuth client ID, secret and GitHub webhook secret management by Chef Vault
 481
 482 - create vault items.
 483
 484 ```text
 485 $ cat ~/sec/tmp/screwdriver_github_secrets.json
 486 {
 487   "oauthClientId": "***************************************************************",
 488   "oauthClientSecret": "***************************************************************",
 489   "secret": "**************************"
 490 }
 491 ```
 492
 493 $ cd $CHEF_REPO_PATH
 494
 495 ```text
 496 $ knife vault create screwdriver github --json ~/sec/tmp/screwdriver_github_secrets.json
 497 ```
 498
 499 - grant reference permission to the screwdriver host
 500
 501 ```text
 502 $ knife vault update screwdriver github -S 'name:screwdriver-host.example.com'
 503 ```
 504
 505 - modify attributes
 506
 507 ```ruby
 508 override_attributes(
 509   'screwdriver' => {
 510     # ...
 511     'api' => {
 512       # ...
 513       'scms_vault_items' => {
 514         'github.com' => {
 515           'oauthClientId' => {
 516             'vault' => 'screwdriver',
 517             'name' => 'github',
 518             'env_context' => false,
 519             'key' => 'oauthClientId',  # real hash path: "/oauthClientId"
 520           },
 521           'oauthClientSecret' => {
 522             'vault' => 'screwdriver',
 523             'name' => 'github',
 524             'env_context' => false,
 525             'key' => 'oauthClientSecret',  # real hash path: "/oauthClientSecret"
 526           },
 527           'secret' => {
 528             'vault' => 'screwdriver',
 529             'name' => 'github',
 530             'env_context' => false,
 531             'key' => 'secret',  # real hash path: "/secret"
 532           },
 533         },
 534       },
 535     },
 536     # ...
 537   },
 538 )
 539 ```
 540
 541 ## License and Authors
 542
 543 - Author:: whitestar at osdn.jp
 544
 545 ```text
 546 Copyright 2017, whitestar
 547
 548 Licensed under the Apache License, Version 2.0 (the "License");
 549 you may not use this file except in compliance with the License.
 550 You may obtain a copy of the License at
 551
 552     http://www.apache.org/licenses/LICENSE-2.0
 553
 554 Unless required by applicable law or agreed to in writing, software
 555 distributed under the License is distributed on an "AS IS" BASIS,
 556 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 557 See the License for the specific language governing permissions and
 558 limitations under the License.
 559 ```