diff --git a/.gitignore b/.gitignore index a2082370b..3c4c506c8 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,8 @@ .rspec results .*.sw[op] +*.wixobj +downloads/* +pkg/* +stagedir/* +wix/fragments/* diff --git a/README_DEVELOPER.md b/README_DEVELOPER.md index aa6fb2bf4..f38067a0e 100644 --- a/README_DEVELOPER.md +++ b/README_DEVELOPER.md @@ -1,91 +1,98 @@ # Developer README # This file is intended to provide a place for developers and contributors to document what other developers need to know about changes made to Puppet. # UTF-8 Handling # As Ruby 1.9 becomes more commonly used with Puppet, developers should be aware of major changes to the way Strings and Regexp objects are handled. Specifically, every instance of these two classes will have an encoding attribute determined in a number of ways. * If the source file has an encoding specified in the magic comment at the top, the instance will take on that encoding. * Otherwise, the encoding will be determined by the LC\_LANG or LANG environment variables. * Otherwise, the encoding will default to ASCII-8BIT ## References ## Excellent information about the differences between encodings in Ruby 1.8 and Ruby 1.9 is published in this blog series: [Understanding M17n](http://links.puppetlabs.com/understanding_m17n) ## Encodings of Regexp and String instances ## In general, please be aware that Ruby 1.9 regular expressions need to be compatible with the encoding of a string being used to match them. If they are not compatible you can expect to receive and error such as: Encoding::CompatibilityError: incompatible encoding regexp match (ASCII-8BIT regexp with UTF-8 string) In addition, some escape sequences were valid in Ruby 1.8 are no longer valid in 1.9 if the regular expression is not marked as an ASCII-8BIT object. You may expect errors like this in this situation: SyntaxError: (irb):7: invalid multibyte escape: /\xFF/ This error is particularly common when serializing a string to other representations like JSON or YAML. To resolve the problem you can explicitly mark the regular expression as ASCII-8BIT using the /n flag: "a" =~ /\342\230\203/n Finally, any time you're thinking of a string as an array of bytes rather than an array of characters, common when escaping a string, you should work with everything in ASCII-8BIT. Changing the encoding will not change the data itself and allow the Regexp and the String to deal with bytes rather than characters. Puppet provides a monkey patch to String which returns an encoding suitable for byte manipulations: # Example of how to escape non ASCII printable characters for YAML. >> snowman = "☃" >> snowman.to_ascii8bit.gsub(/([\x80-\xFF])/n) { |x| "\\x#{x.unpack("C")[0].to_s(16)} } => "\\xe2\\x98\\x83" If the Regexp is not marked as ASCII-8BIT using /n, then you can expect the SyntaxError, invalid multibyte escape as mentioned above. # Windows # If you'd like to run Puppet from source on Windows platforms, the include `ext/envpuppet.bat` will help. All file paths in the Puppet code base should use a path separator of / regardless of Windows or Unix filesystem. To quickly run Puppet from source, assuming you already have Ruby installed from [rubyinstaller.org](http://rubyinstaller.org). gem install sys-admin win32-process win32-dir win32-taskscheduler --no-rdoc --no-ri gem install win32-service --platform=mswin32 --no-rdoc --no-ri --version 0.7.1 net use Z: "\\vmware-host\Shared Folders" /persistent:yes Z: cd set PATH=%PATH%;Z:\\ext envpuppet puppet --version 2.7.9 Some spec tests are known to fail on Windows, e.g. no mount provider on Windows, so use the following rspec exclude filter: cd envpuppet rspec --tag ~fails_on_windows spec This will give you a shared filesystem with your Mac and allow you to run Puppet directly from source without using install.rb or copying files around. +## Building Windows Packages ## + +Please see the README files in `tasks/windows/` for more information about +building MSI packages of Puppet for Windows. Please see also +[#11205](http://projects.puppetlabs.com/issues/11205) for up to date progress +on this project. + EOF diff --git a/tasks/windows/README.markdown b/tasks/windows/README.markdown new file mode 100644 index 000000000..bae55d340 --- /dev/null +++ b/tasks/windows/README.markdown @@ -0,0 +1,50 @@ +# For the Win # + +This project is a small set of Rake tasks to automate the process of building +MSI packages for Puppet on Windows systems. + +This is a separate repository because it is meant to build MSI packages for +arbitrary versions of Puppet, Facter and other related tools. + +This project is meant to be checked out into a special Puppet Windows Dev Kit +directory structure. This Dev Kit will provide the tooling necessary to +actually build the packages. + +This project requires these tools from the `puppetbuilder` Dev Kit for Windows +systems. + + * Ruby + * Rake + * Git + * 7zip + * WiX + +# Getting Started # + +Given a basic Windows 2003 R2 x64 system with the [Puppet Win +Builder](http://links.puppetlabs.com/puppetwinbuilder) archive unpacked into +`C:/puppetwinbuilder/` the following are all that is required to build the MSI +packages. + + C:\>cd puppetwinbuilder + C:\puppetwinbuilder\> build + ... + +(REVISIT - This is the thing we're working to. Make sure this is accurate once +implemented) + +# Making Changes # + +The [Puppet Win Builder](http://links.puppetlabs.com/puppetwinbuilder) archive +should remain relatively static. The purpose of this archive is simply to +bootstrap the tools required for the build process. + +Changes to the build process itself should happen in the [Puppet For the +Win](https://github.com/puppetlabs/puppet_for_the_win) repository on Github. + +# Continuous Integration # + +The `build.bat` build script _should_ work just fine with a build system like +Jenkins. If it does not, please let us know. + +EOF diff --git a/tasks/windows/README_DEVELOPER.markdown b/tasks/windows/README_DEVELOPER.markdown new file mode 100644 index 000000000..89db77ef3 --- /dev/null +++ b/tasks/windows/README_DEVELOPER.markdown @@ -0,0 +1,20 @@ +# Setup Tips # + +To get a shared filesystem: + + net use Z: "\\vmware-host\Shared Folders" /persistent:yes + +# Common Issues # + +I seem to be getting this a lot downloading files: + + undefined method `zero?' for nil:NilClass + +This appears to be from the call to the progress bar method having nil content +from the response.content\_length here: + + % ack with_progress_bar + rake/contrib/uri_ext.rb + 161: # with_progress_bar(enable, file_name, size) { |progress| ... } + 171: def with_progress_bar(enable, file_name, size) #:nodoc: + 254: with_progress_bar options[:progress], path.split('/').last, response.content_length do |progress| diff --git a/tasks/windows/windows.rake b/tasks/windows/windows.rake new file mode 100644 index 000000000..ed554bc14 --- /dev/null +++ b/tasks/windows/windows.rake @@ -0,0 +1,149 @@ +#! /usr/bin/env ruby + +# This rakefile is meant to be run from within the [Puppet Win +# Builder](http://links.puppetlabs.com/puppetwinbuilder) tree. + +# Load Rake +begin + require 'rake' +rescue LoadError + require 'rubygems' + require 'rake' +end + +require 'rake/clean' + +# Added download task from buildr +require 'rake/downloadtask' + +# Where we're situated in the filesystem relative to the Rakefile +TOPDIR=File.expand_path(File.join(File.dirname(__FILE__), "..", "..")) + +# Produce a wixobj from a wxs file. +def candle(wxs_file, basedir) + Dir.chdir File.join(TOPDIR, File.dirname(wxs_file)) do + sh "candle -dStageDir=#{basedir} #{File.basename(wxs_file)}" + end +end + +# Produce a wxs file from a directory in the stagedir +# e.g. heat('wxs/fragments/foo.wxs', 'stagedir/sys/foo') +def heat(wxs_file, stage_dir) + Dir.chdir TOPDIR do + cg_name = File.basename(wxs_file.ext('')) + dir_ref = File.basename(File.dirname(stage_dir)) + sh "heat dir #{stage_dir} -v -ke -indent 2 -cg #{cg_name} -gg -dr #{dir_ref} -var var.StageDir -out #{wxs_file}" + end +end + +def unzip(zip_file, dir) + Dir.chdir TOPDIR do + Dir.chdir dir do + sh "7za -y x #{File.join(TOPDIR, zip_file)}" + end + end +end + +def gitclone(target, uri) + Dir.chdir(File.dirname(target)) do + sh "git clone #{uri} #{File.basename(target)}" + end +end + +CLOBBER.include('downloads/*') +CLEAN.include('stagedir/*') +CLEAN.include('wix/fragments/*.wxs') +CLEAN.include('wix/**/*.wixobj') +CLEAN.include('pkg/*') + +namespace :windows do + + # These are file tasks that behave like mkdir -p + directory 'pkg' + directory 'downloads' + directory 'stagedir/sys' + directory 'wix/fragments' + + ## File Lists + + TARGETS = FileList['pkg/puppet.msi'] + + # These translate to ZIP files we'll download + # FEATURES = %w{ ruby git wix misc } + FEATURES = %w{ ruby } + DOWNLOADS = FEATURES.collect { |fn| File.join("downloads", fn.ext('zip')) } + + # There is a 1:1 mapping between a wxs file and a wixobj file + + # These files should be committed to VCS + WXSFILES = FileList['wix/*.wxs'] + # These files should be auto-generated by heat + WXSFRAGMENTS = FEATURES.collect { |fn| File.join("wix", "fragments", fn.ext('wxs')) } + # All of the objects we need to create + WIXOBJS = (WXSFILES + WXSFRAGMENTS).ext('wixobj') + + # These directories should be unpacked into stagedir/sys + SYSTOOLS = FEATURES.collect { |fn| File.join("stagedir", "sys", fn) } + + task :default => :build + # High Level Tasks. Other tasks will add themselves to these tasks + # dependencies. + + # This is also called from the build script in the Puppet Win Builder archive. + # This will be called AFTER the update task in a new process. + desc "Build puppet.msi" + task :build => "pkg/puppet.msi" + + desc "Download example" + task :download => DOWNLOADS + + desc "Unzip and stage sys tools" + task :unzip => SYSTOOLS + + desc "List available rake tasks" + task :help do + sh 'rake -T' + end + + # The update task is always called from the build script + # This gives the repository an opportunity to update itself + # and manage how it updates itself. + desc "Update the build scripts" + task :update do + sh 'git pull' + end + + # Tasks to unpack the zip files + SYSTOOLS.each do |systool| + zip_file = File.join("downloads", File.basename(systool).ext('zip')) + file systool => [ zip_file, File.dirname(systool) ] do + unzip(zip_file, File.dirname(systool)) + end + end + + DOWNLOADS.each do |fn| + file fn => [ File.dirname(fn) ] do |t| + download t.name => "http://downloads.puppetlabs.com/development/ftw/#{File.basename(t.name)}" + end + end + + WIXOBJS.each do |wixobj| + stagedir = File.join(TOPDIR, 'stagedir', 'sys') + file wixobj => [ wixobj.ext('wxs'), File.dirname(wixobj) ] do |t| + candle t.name.ext('wxs'), File.join(stagedir, File.basename(t.name.ext(''))).gsub('/', File::SEPARATOR) + end + end + + WXSFRAGMENTS.each do |wxs_frag| + source_dir = File.join('stagedir', 'sys', File.basename(wxs_frag).ext('')) + file wxs_frag => [ source_dir, File.dirname(wxs_frag) ] do |t| + heat t.name, source_dir + end + end + + ####### REVISIT + + file 'pkg/puppet.msi' => WIXOBJS do |t| + sh "light #{t.prerequisites.join(' ')} -out #{t.name}" + end +end diff --git a/wix/include/puppet.wxi b/wix/include/puppet.wxi new file mode 100644 index 000000000..f1af4de54 --- /dev/null +++ b/wix/include/puppet.wxi @@ -0,0 +1,31 @@ + + + + + + + + + + + + + + + + + + + + diff --git a/wix/puppet.wxs b/wix/puppet.wxs new file mode 100644 index 000000000..48e801245 --- /dev/null +++ b/wix/puppet.wxs @@ -0,0 +1,61 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + +