Windows Worker and Concourse Setup

Windows Worker Setup

Instructions for setting up a Windows worker for building vSphere stemcells and optionally registering it with Concourse.

Host machine requirements

Windows Server 2012R2 or Windows Server 2016
8GB RAM minimum / 12GB recommended
6 CPU cores minimum / 8 recommended
- If using 8 or less cores you will want to set the NUM_VCPUS environment variable when invoking the build rake task.
200GB of disk space minimum / 500GB recommended
If the host is a virtual machine it must have hardware assisted virtualization:
- VMware vSphere/ESXi: VM Hardware => Virtual Hardware => CPU => Expose hardware assisted virtualization to the guest OS
- VMware Fusion: Settings => Processors & Memory => Advanced options => Enable hypervisor applications in this virual machine
- VMware Workstation: Virtual Machine Settings => Processors => Virtualize Intel VT-x/EPT or AMD-V/RVI

Software Requirements

Golang (latest)
Ruby 2.3.X
tar.exe
Packer: Download the Windows binary from the linked release.
VMware Workstation 12.5 Pro: A free trial is available, but will expire in 30 days

Concourse specific

Installation

Installers

Golang: use defaults
Ruby: in the Installation Destination and Optional Tasks section (immediately after license agreement) select: "Add Ruby executables to your PATH" and "Associate .rb and .rbw files with Ruby installation".
- Install bundler gem install bundler
VMware Workstation: use defaults, you can select the "Enhanced Keyboard Driver" if desired.
- If VT-x (Intel hardware virtualization) pass-through is not enabled you may get an error during the install or when trying to run Workstation. Instructions are available in the Host machine requirements section.

Add ovftool to the System path using the below PowerShell script:

$ovftool="C:\Program Files (x86)\VMware\VMware Workstation\OVFTool"
[Environment]::SetEnvironmentVariable("Path", "$env:PATH;$ovftool", [System.EnvironmentVariableTarget]::Machine)

Executables

Create a C:\bin directory and add it to the System PATH:

New-Item -ItemType directory -Path C:\bin -Force
[Environment]::SetEnvironmentVariable("Path", "$env:PATH;C:\bin", [System.EnvironmentVariableTarget]::Machine)

Tar: Copy the downloaded tar executable to C:\bin\tar.exe - you may have to rename the downloaded file.
Packer: Extract the downloaded packer zip and copy packer.exe to C:\bin\packer.exe

Check that everything is installed

Run the below script to make sure everything is installed:

$RequiredExes=@(
    "tar.exe",
    "packer.exe",
    "ovftool.exe",
    "go.exe",
    "ruby.exe"
)
foreach ($exe in $RequiredExes) {
    Get-Command $exe
}

Setup VMX Directory and Initial VM

The initial VM that will be used by the build process needs to be created by hand.

Follow the documentation for creating a base VM: here.
- Note:
  1. Follow only the steps listed in: 'Step 1: Create base VM for stemcell'
  2. Name the VM vmx-1 and note where it is saved
When done shutdown the VM.
Find the directory where the VM is saved
- The default location is %USERPROFILE%\Documents\Virtual Machines
- If you named your VM vmx-1 it is likely located at %USERPROFILE%\Documents\Virtual Machines\vmx-1
Create the directory: C:\vmx-data

Create a gzip'd tar archive of the base VM and save it to C:\vmx-data\vmx-1.tgz, below is a helper script:

# Stop on error
$ErrorActionPreference = "Stop";

# Directory where you VM is saved
$VMDir="$env:USERPROFILE\Documents\Virtual Machines"

# Name of your virtual machine - must be vmx-1
$VMName="vmx-1"

# Tarball we will create
$Tarball="C:\vmx-data\vmx-1.tgz"

if ($VMName -ne "vmx-1") {
    Write-Error "Invalid VM name: $VMName - rename it's directory to vmx-1"
}
if (-Not (Test-Path "$VMDir\$VMName")) {
    Write-Error "Invalid VM directory and name: $VMDir\$VMName"
}
if (Test-Path $Tarball) {
    Write-Error "Error: tarball already exists - refusing to overwrite: $Tarball"
}
if (-Not (Test-Path "C:\vmx-data")) {
    New-Item -ItemType directory -Path "C:\vmx-data"
}
Push-Location $VMDir
    tar.exe czvf "$Tarball" "$VMName"
    if ($LASTEXITCODE -ne 0) {
        Write-Error "Tar exited with: $LASTEXITCODE"
    }
Pop-Location

Note: If you are not using Concourse the setup is complete.

Install Concourse Worker

This section documents how to configure the worker for Concourse. If you are not using Concourse the setup of your worker is complete.

Concourse Setup (assumes Concourse is deployed with BOSH):

Make sure your Concourse deployment is configured to support external workers: Supporting external workers
- This requires manually configuring the TSA'a host key
Generate the worker's RSA keys: ssh-keygen -t rsa -b 4096 -N "" -f tsa-worker-private-key
- ssh-keygen is available on macOS, Linux and can be installed on Windows with Git for Windows
Add the workers public key tsa-worker-private-key.pub to the authorized_keys property of the tsa job in your Concourse deployment manifest.

Worker Setup:

Create the following directories: C:\containers, C:\concourse and C:\vmx-data

foreach ($name in @("containers", "concourse", "vmx-data")) {
    if (-Not (Test-Path "C:\$name")) {
        New-Item -ItemType directory -Path "C:\$name" -Force
    }
}

Move concourse_windows_amd64.exe to C:\concourse\concourse_windows_amd64.exe
Move WinSW.NET4.exe to C:\concourse\concourse.exe (renaming to it concourse.exe)
Save the generated tsa-worker-private-key to C:\concourse\tsa-worker-private-key
Save the TSA hosts public key to C:\concourse\tsa-public-key.pub

Save below configuration as C:\concourse\concourse.xml replacing the TSA_HOST_ADDRESS and WORKER_TAG_NAME fields

TSA_HOST_ADDRESS: The address of your TSA host: 10.0.0.6 or my-ci.my-app.com
WORKER_TAG_NAME: The tag you will use to specify this worker in your pipeline, for example "windows-worker"

<service>
  <id>concourse</id>
  <name>Concourse</name>
  <description>Concourse Windows worker.</description>
  <startmode>Automatic</startmode>
  <executable>C:\concourse\concourse_windows_amd64.exe</executable>
  <argument>worker</argument>
  <argument>/work-dir</argument>
  <argument>C:\containers</argument>
  <argument>/tsa-worker-private-key</argument>
  <argument>C:\concourse\tsa-worker-private-key</argument>
  <argument>/tsa-public-key</argument>
  <argument>C:\concourse\tsa-public-key.pub</argument>
  <argument>/tsa-host</argument>
  <argument>TSA_HOST_ADDRESS</argument> <!-- REPLACE ME -->
  <argument>/tag</argument>
  <argument>WORKER_TAG_NAME</argument> <!-- REPLACE ME -->
  <onfailure action="restart" delay="10 sec"/>
  <onfailure action="restart" delay="20 sec"/>
  <logmode>rotate</logmode>
</service>

Make sure everything is in the right place:

# Required directories
$Failed=$false
foreach ($dir in @("C:\containers", "C:\concourse", "C:\vmx-data")) {
    if (-Not (Test-Path $dir)) {
        Write-Host -ForegroundColor Red "Error: Missing required directory: $dir"
        $Failed=$true
    }
}
$RequiredFiles=@(
    "concourse_windows_amd64.exe",
    "concourse.exe",
    "concourse.xml",
    "tsa-worker-private-key",
    "tsa-public-key.pub"
)
foreach ($name in $RequiredFiles) {
    $path = "C:\concourse\$name"
    if (-Not (Test-Path $path)) {
        Write-Host -ForegroundColor Red "Error: missing required file: $path"
        $Failed=$true
    }
}
# Make sure TSA_HOST_ADDRESS and WORKER_TAG_NAME were replaced
$Config="C:\concourse\concourse.xml"
if (Test-Path $Config) {
    $Content=(Get-Content -Raw -Path $Config)
    if ($Content.Contains("TSA_HOST_ADDRESS")) {
        Write-Host -ForegroundColor Red "Error: '$Config' contains: TSA_HOST_ADDRESS"
        $Failed=$true
    }
    if ($Content.Contains("WORKER_TAG_NAME")) {
        Write-Host -ForegroundColor Red "Error: '$Config' contains: WORKER_TAG_NAME"
        $Failed=$true
    }
}
if ($Failed) {
    Write-Host -ForegroundColor Red "Error: check failed"
} else {
    Write-Host -ForegroundColor Green "Success: check passed"
}

Install concourse service: C:\concourse\concourse.exe install
Start the concourse service C:\concourse\concourse.exe start
- This will also register the worker with your Concourse deployment
Make sure the service is running:
- Service wrapper: C:\concourse\concourse.exe status
- PowerShell: (Get-Service -Name concourse).Status
- Use the Task Manager or Services GUIs
Make sure the the worker is registered with your Concourse: fly -t TARGET workers

Note: You must restart the service for changes to concourse.xml to be applied: C:\concourse\concourse.exe restart

Debugging Concourse Worker

Check The service logs:
- Stderr: C:\concourse\concourse.err.log
- Stdout: C:\concourse\concourse.out.log
- Service Wrapper: C:\concourse\concourse.wrapper.log
Make sure the workers public key has been added to the TSA hosts authorized_keys

Provide feedback

Saved searches

Use saved searches to filter your results more quickly