diff --git a/CHANGELOG b/CHANGELOG index f51e1f3c6..27aa0fa5e 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -1,775 +1,777 @@ 0.23.2 (misspiggy) + Added shortname support to config.rb and refactored addargs + Fixed the problem in cron jobs where environment settings tended to multiple. (#749) Collection of resources now correctly only collects exported resources again. This was broken in 0.23.0. (#731) 'gen_config' now generates a configuration with all parameters under a heading that matches the process name, rather than keeping section headings. Refactored how the parser and interpreter relate, so parsing is now effectively an atomic process (thus fixing #314 and #729). This makes the interpreter less prone to error and less prone to show the error to the clients. Note that this means that if a configuration fails to parse, then the previous, parseable configuration will be used instead, so the client will not know that the configuration failed to parse. Added support for managing interfaces, thanks to work by Paul Rose. Fixed #652, thanks to a patch by emerose; --fqdn again works with puppetd. Added an extra check to the Mongrel support so that Apache can be used with optional cert checking, instead of mandatory, thus allowing Mongrel to function as the CA. This is thanks to work done by Marcin Owsiany. 0.23.1 (beaker) You can now specify relationships to classes, which work exactly like relationships to defined types: require => Class[myclass] This works with qualified classes, too. You can now do simple queries in a collection of exported resources. You still cannot do multi-condition queries, though. (#703) puppetca now exits with a non-zero code if it cannot find any host certificates to clean. (Patch by Dean Wilson.) Fully-qualified resources can now have defaults. (#589) Resource references can now be fully-qualified names, meaning you can list definitions with a namespace as dependencies. (#468) Files modified using a FileType instance, as ParsedFile does, will now automatically get backed up to the filebucket named "puppet". Added a 'maillist' type for managing mailing lists. Added a 'mailalias' type for managing mail aliases. Added patch by Valentin Vidic that adds the '+>' syntax to resources, so parameter values can be added to. The configuration client now pulls libraries down to $libdir, and all autoloading is done from there with full support for any reloadable file, such as types and providers. (#621) Note that this is not backward compatible -- if you're using pluginsync right now, you'll need to disable it on your clients until you can upgrade them. The Rails log level can now be set via (shockingly!) the 'rails_loglevel' parameter (#710). Note that this isn't exactly the feature asked for, but I could not find a way to directly copy ActiveRecord's concept of an environment. External node sources can now return undefined classes (#687). Puppet clients now have http proxy support (#701). The parser now throws an error when a resource reference is created for an unknown type. Also, resource references look up defined types and translate their type accordingly. (#706) Hostnames can now be double quoted. Adding module autoloading (#596) -- you can now 'include' classes from modules without ever needing to specifically load them. Class names and node names now conflict (#620). 0.23.0 Modified the fileserver to cache file information, so that each file isn't being read on every connection. Also, added londo's patch from #678 to avoid reading entire files into memory. Fixed environment handling in the crontab provider (#669). Added patch by trombik in #572, supporting old-style freebsd init scripts with '.sh' endings. Added fink package provider (#642), as provided by 'do'. Marked the dpkg package provider as versionable (#647). Applied patches by trombik to fix FreeBSD ports (#624 and #628). Fixed the CA server so that it refuses to send back a certificate whose public key doesn't match the CSR. Instead, it tells the user to run 'puppetca --clean'. Invalid certificates are no longer written to disk (#578). Added a package provider (appdmg) able to install .app packages on .dmg files on OS X (#641). Applied the patch from #667 to hopefully kill the client hanging problems (permanently, this time). Fixed functions so that they accept most other rvalues as valid values (#548). COMPATIBILITY ALERT: Significantly reworked external node support, in a way that's NOT backward-compatible: Only ONE node source can be used -- you can use LDAP, code, or an external node program, but not more than one. LDAP node support has two changes: First, the "ldapattrs" attribute is now used for setting the attributes to retrieve from the server (in addition to required attriutes), and second, all retrieved attributes are set as variables in the top scope. This means you can set attributes on your LDAP nodes and they will automatically appear as variables in your configurations. External node support has been completely rewritten. These programs must now generate a YAML dump of a hash, with "classes" and "parameters" keys. The classes should be an array, and the parameters should be a hash. The external node program has no support for parent nodes -- the script must handle that on its own. Reworked the database schema used to store configurations with the storeconfigs option. Replaced the obsolete RRD ruby library with the maintained RubyRRDtool library (which requires rrdtool2) (#659). The Portage package provider now calls eix-update automatically when eix's database is absent or out of sync (#666). Mounts now correctly handle existing fstabs with no pass or dump values (#550). Mounts now default to 0 for pass and dump (#112). Added urpmi support (#592). Finishing up the type => provider interface work. Basically, package providers now return lists of provider instances. In the proces, I rewrote the interface between package types and providers, and also enabled prefetching on all packages. This should significantly speed up most package operations. Hopefully fixing the file descriptor/open port problems, with patches from Valentin Vidic. Significantly reworked the type => provider interface with respect to listing existing provider instances. The class method on both class heirarchies has been renamed to 'instances', to start. Providers are now expected to return provider instances, instead of creating resources, and the resource's 'instances' method is expected to find the matching resource, if any, and set the resource's provider appropriately. This *significantly* reduces the reliance on effectively global state (resource references in the resource classes). This global state will go away soon. Along with this change, the 'prefetch' class method on providers now accepts the list of resources for prefetching. This again reduces reliance on global state, and makes the execution path much easier to follow. Fixed #532 -- reparsing config files now longer throws an exception. Added some warnings and logs to the service type so users will be encouraged to specify either "ensure" or "enabled" and added debugging to indicate why restarting is skipped when it is. Changed the location of the classes.txt to the state directory. Added better error reporting on unmatched brackets. Moved puppetd and puppetmasterd to sbin in svn and fixed install.rb to copy them into sbin on the local system appropriately. (#323) Added a splay option (#501). It's disabled when running under --test in puppetd. The value is random but cached. It defaults to the runinterval but can be tuned with --splaylimit Changing the notify type so that it always uses the loglevel. Fixing #568 - nodes can inherit from quoted node names. Tags (and thus definitions and classes) can now be a single character. (#566) Added an 'undef' keyword (#629), which will evaluate to "" within strings but when used as a resource parameter value will cause that parameter to be evaluated as undefined. Changed the topological sort algorithm (#507) so it will always fail on cycles. Added a 'dynamicfacts' configuration option; any facts in that comma-separated list will be ignored when comparing facts to see if they have changed and thus whether a recompile is necessary. Renamed some poorly named internal variables: @models in providers are now either @resource or @resource_type (#605). @children is no longer used except by components (#606). @parent is now @resource within parameters (#607). The old variables are still set for backward compatibility. Significantly reworking configuration parsing. Executables all now look for 'puppet.conf' (#206), although they will parse the old-style configuration files if they are present, although they throw a deprecation warning. Also, file parameters (owner, mode, group) are now set on the same line as the parameter, in brackets. (#422) Added transaction summaries (available with the --summarize option), useful for getting a quick idea of what happened in a transaction. Currently only useful on the client or with the puppet interpreter. Changed the interal workings for retrieve and removed the :is attribute from Property. The retrieve methods now return the current value of the property for the system. Removed acts_as_taggable from the rails models. 0.22.4 Execs now autorequire the user they run as, as long as the user is specified by name. (#430) Files on the local machine but not on the remote server during a source copy are now purged if purge => true. (#594) Providers can now specify that some commands are optional (#585). Also, the 'command' method returns nil on missing commands, rather than throwing an error, so the presence of commands be tested. The 'useradd' provider for Users can now manage passwords. No other providers can, at this point. Parameters can now declare a dependency on specific features, and parameters that require missing features will not be instantiated. This is most useful for properties. FileParsing classes can now use instance_eval to add many methods at once to a record type. Modules no longer return directories in the list of found manifests (#588). The crontab provider now defaults to root when there is no USER set in the environment. Puppetd once again correctly responds to HUP. Added a syntax for referring to variables defined in other classes (e.g., $puppet::server). STDIN, STDOUT, STDERR are now redirected to /dev/null in service providers descending from base. Certificates are now valid starting one day before they are created, to help handle small amounts of clock skew. Files are no longer considered out of sync if some properties are out of sync but they have no properties that can create the file. 0.22.3 Fixed backward compatibility for logs and metrics from older clients. Fixed the location of the authconfig parameters so there aren't loading order issues. Enabling attribute validation on the providers that subclass 'nameservice', so we can verify that an integer is passed to UID and GID. Added a stand-alone filebucket client, named 'filebucket'. Fixed the new nested paths for filebuckets; the entire md5 sum was not being stored. Fixing #553; -M is no longer added when home directories are being managed on Red Hat. 0.22.2 (grover) Users can now manage their home directories, using the managehome parameter, partially using patches provided by Tim Stoop and Matt Palmer. (#432) Added 'ralsh' (formerly x2puppet) to the svn tree. When possible it should be added to the packages. The 'notify' type now defaults to its message being the same as its name. Reopening $stdin to read from /dev/null during execution, in hopes that init scripts will stop hanging. Changed the 'servername' fact set on the server to use the server's fqdn, instead of the short-name. Changing the location of the configuration cache. It now defaults to being in the state directory, rather than in the configuration directory. All parameter instances are stored in a single @parameters instance variable hash within resource type instances. We used to use separate hashes for each parameter type. Added the concept of provider features. Eventually these should be able to express the full range of provider functionality, but for now they can test a provider to see what methods it has set and determine what features it provides as a result. These features are integrated into the doc generation system so that you get feature documentation automatically. Switched apt/aptitide to using "apt-cache policy" instead of "apt-cache showpkg" for determining the latest available version. (#487) FileBuckets now use a deeply nested structure for storing files, so you do not end up with hundreds or thousands of files in the same directory. (#447) Facts are now cached in the state file, and when they change the configuration is always recompiled. (#519) Added 'ignoreimport' setting for use in commit hooks. This causes the parser to ignore import statements so a single file can be parse-checked. (#544) Import statements can now specify multiple comma-separated arguments. Definitions now support both 'name' and 'title', just like any other resource type. (#539) Added a generate() command, which sets values to the result of an external command. (#541) Added a file() command to read in files with no interpolation. The first found file has its content returned. puppetd now exits if no cert is present in onetime mode. (#533) The client configuration cache can be safely removed and the client will correctly realize the client is not in sync. Resources can now be freely deleted, thus fixing many problems introduced when deletion of required resources was forbidden when purging was introduced. Only resources being purged will not be deleted. Facts and plugins now download even in noop mode (#540). Resources in noop mode now log when they would have responded to an event (#542). Refactored cron support entirely. Cron now uses providers, and there is a single 'crontab' provider that handles user crontabs. While this refactor does not include providers for /etc/crontab or cron.d, it should now be straightforward to write those providers. Changed the parameter sorting so that the provider parameter comes right after name, so the provider is available when the other parameters and properties are being created. Redid some of the internals of the ParsedFile provider base class. It now passes a FileRecord around instead of a hash. Fixing a bug related to link recursion that caused link directories to always be considered out of sync. The bind address for puppetmasterd can now be specified with --bindaddress. Added (probably experimental) mongrel support. At this point you're still responsible for starting each individual process, and you have to set up a proxy in front of it. Redesigned the 'network' tree to support multiple web servers, including refactoring most of the structural code so it's much clearer and more reusable now. Set up the CA client to default to ca_server and ca_port, so you can easily run a separate CA. Supporting hosts with no domain name, thanks to a patch from Dennis Jacobfeuerborn. Added an 'ignorecache' option to tell puppetd to force a recompile, thanks to a patch by Chris McEniry. Made up2date the default for RHEL < 4 and yum the default for the rest. The yum provider now supports versions. Case statements correctly match when multiple values are provided, thanks to a patch by David Schmitt. Functions can now be called with no arguments. String escapes parse correctly in all cases now, thanks to a patch by cstorey. Subclasses again search parent classes for defaults. You can now purge apt and dpkg packages. When doing file recursion, 'ensure' only affects the top-level directory. States have been renamed to Properties. 0.22.1 (kermit) -- Mostly a bugfix release Compile times now persist between restarts of puppetd. Timeouts have been added to many parts of Puppet, reducing the likelihood if it hanging forever on broken scripts or servers. All of the documentation and recipes have been moved to the wiki by Peter Abrahamsen and Ben Kite has moved the FAQ to the wiki. Explicit relationships now override automatic relationships, allowing you to manually specify deletion order when removing resources. Resources with dependencies can now be deleted as long as all of their dependencies are also being deleted. Namespaces for both classes and definitions now work much more consistently. You should now be able to specify a class or definition with a namespace everywhere you would normally expect to be able to specify one without. Downcasing of facts can be selectively disabled. Cyclic dependency graphs are now checked for and forbidden. The netinfo mounts provider was commented out, because it really doesn't work at all. Stupid NetInfo stores mount information with the device as the key, which doesn't work with my current NetInfo code. Otherwise, lots and lots of bugfixes. Check the tickets associated with the 'kermit' milestone. 0.22.0 Integrated the GRATR graph library into Puppet, for handling resource relationships. Lots of bug-fixes (see bugs tickets associated with the 'minor' milestone). Added new 'resources' metatype, which currently only includes the ability to purge unmanaged resources. Added better ability to generate new resource objects during transactions (using 'generate' and 'eval_generate' methods). Rewrote all Rails support with a much better database design. Export/collect now works, although the database is incompatible with previous versions. Removed downcasing of facts and made most of the language case-insensitive. Added support for printing the graphs built during transactions. Reworked how paths are built for logging. Switched all providers to directly executing commands instead of going through a subshell, which removes the need to quote or escape arguments. 0.20.1 Mostly a bug-fix release, with the most important fix being the multiple-definition error. Completely rewrote the ParsedFile system; each provider is now much shorter and much more maintainable. However, fundamental problems were found with the 'port' type, so it was disabled. Also, added a NetInfo provider for 'host' and an experimental NetInfo provider for 'mount'. Made the RRDGraph report *much* better and added reference generation for reports and functions. 0.20.0 Significantly refactored the parser. Resource overrides now consistently work anywhere in a class hierarchy. The language was also modified somewhat. The previous export/collect syntax is now used for handling virtual objects, and export/collect (which is still experimental) now uses double sigils (@@ and <<| |>>). Resource references (e.g., File["/etc/passwd"]) now have to be capitalized, in fitting in with capitalizing type operations. As usual, lots of other smaller fixes, but most of the work was in the language. 0.19.3 Fixing a bug in server/master.rb that causes the hostname not to be available in locally-executed manifests. 0.19.2 Fixing a few smaller bugs, notably in the reports system. Refreshed objects now generate an event, which can result in further refreshes of other objects. 0.19.1 Fixing two critical bugs: User management works again and cron jobs are no longer added to all user accounts. 0.19.0 Added provider support. Added support for %h, %H, and %d expansion in fileserver.conf. Added Certificate Revocation support. Made dynamic loading pervasive -- nearly every aspect of Puppet will now automatically load new instances (e.g., types, providers, and reports). Added support for automatic distribution of facts and plugins (custom types). 0.18.4 Another bug-fix release. The most import bug fixed is that cronjobs again work even with initially empty crontabs. 0.18.3 Mostly a bug-fix release; fixed small bugs in the functionality added in 0.18.2. 0.18.2 Added templating support. Added reporting. Added gem and blastwave packaging support. 0.18.1 Added signal handlers for HUP, so both client and server deal correctly with it. Added signal handler for USR1, which triggers a run on the client. As usual, fixed many bugs. Significant fixes to puppetrun -- it should behave much more correctly now. Added "fail" function which throws a syntax error if it's encountered. Added plugin downloading from the central server to the client. It must be enabled with --pluginsync. Added support for FreeBSD's special "@daily" cron schedules. Correctly handling spaces in file sources. Moved documentation into svn tree. 0.18.0 Added support for a "default" node. When multiple nodes are specified, they must now be comma-separated (this introduces a language incompatibility). Failed dependencies cause dependent objects within the same transaction not to run. Many updates to puppetrun Many bug fixes Function names are no longer reserved words. Links can now replace files. 0.17.2 Added "puppetrun" application and associated runner server and client classes. Fixed cron support so it better supports valid values and environment settings. 0.17.1 Fixing a bug requiring rails on all Debian boxes Fixing a couple of other small bugs 0.17.0 Adding ActiveRecord integration on the server Adding export/collect functionality Fixing many bugs 0.16.5 Fixing a critical bug in importing classes from other files Fixing nodename handling to actually allow dashes 0.16.4 Fixing a critical bug in puppetd when acquiring a certificate for the first time 0.16.3 Some significant bug fixes Modified puppetd so that it can now function as an agent independent of a puppetmasterd process, e.g., using the PuppetShow web application. 0.16.2 Modified some of the AST classes so that class names, definition names, and node names are all set within the code being evaluated, so 'tagged(name)' returns true while evaluating 'name', for instance. Added '--clean' argument to puppetca to remove all traces of a given client. 0.16.1 Added 'tagged' and 'defined' functions. Moved all functions to a general framework that makes it very easy to add new functions. 0.16.0 Added 'tag' keyword/function. Added FreeBSD Ports support Added 'pelement' server for sending or receiving Puppet objects, although none of the executables use it yet. 0.15.3 Fixed many bugs in :exec, including adding support for arrays of checks Added autoloading for types and service variants (e.g., you can now just create a new type in the appropriate location and use it in Puppet, without modifying the core Puppet libs). 0.15.2 Added darwinport, Apple .pkg, and freebsd package types Added 'mount type Host facts are now set at the top scope (Bug #103) Added -e (inline exection) flag to 'puppet' executable Many small bug fixes 0.15.1 Fixed 'yum' installs so that they successfully upgrade packages. Fixed puppetmasterd.conf file so group settings take. 0.15.0 Upped the minor release because the File server is incompatible with 0.14, because it now handles links. The 'symlink' type is deprecated (but still present), in favor of using files with the 'target' parameter. Unset variables no longer throw an error, they just return an empty string You can now specify tags to restrict which objects run during a given run. You can also specify to skip running against the cached copy when there's a failure, which is useful for testing new configurations. RPMs and Sun packages can now install, as long as they specify a package location, and they'll automatically upgrade if you point them to a new file with an upgrade. Multiple bug fixes. 0.14.1 Fixed a couple of small logging bugs Fixed a bug with handling group ownership of links 0.14.0 Added some ability to selectively manage symlinks when doing file management Many bug fixes Variables can now be used as the test values in case statements and selectors Bumping a minor release number because 0.13.4 introduced a protocol incompatibility and should have had a minor rev bump 0.13.6 Many, many small bug fixes FreeBSD user/group support has been added The configuration system has been rewritten so that daemons can now generate and repair the files and directories they need. (Fixed bug #68.) Fixed the element override issues; now only subclasses can override values. 0.13.5 Fixed packages so types can be specified Added 'enable' state to services, although it does not work everywhere yet 0.13.4 A few important bug fixes, mostly in the parser. 0.13.3 Changed transactions to be one-stage instead of two Changed all types to use self[:name] instead of self.name, to support the symbolic naming implemented in 0.13.1 0.13.2 Changed package[answerfile] to package[adminfile], and added package[responsefile] Fixed a bunch of internal functions to behave more consistently and usefully 0.13.1 Fixed RPM spec files to create puppet user and group (lutter) Fixed crontab reading and writing (luke) Added symbolic naming in the language (luke) 0.13.0 Added support for configuration files. Even more bug fixes, including the infamous 'frozen object' bug, which was a problem with 'waitforcert'. David Lutterkort got RPM into good shape. 0.12.0 Added Scheduling, and many bug fixes, of course. 0.11.2 Fixed bugs related to specifying arrays of requirements Fixed a key bug in retrieving checksums Fixed lots of usability bugs Added 'fail' methods that automatically add file and line info when possible, and converted many errors to use that method 0.11.1 Fixed bug with recursive copying with 'ignore' set. Added OpenBSD package support. 0.11.0 Added 'ensure' state to many elements. Modified puppetdoc to correctly handle indentation and such. Significantly rewrote much of the builtin documentation to take advantage of the new features in puppetdoc, including many examples. 0.10.2 Added SMF support Added autorequire functionality, with specific support for exec and file Exec elements autorequire any mentioned files, including the scripts, along with their CWDs. Files autorequire any parent directories. Added 'alias' metaparam. Fixed dependencies so they don't depend on file order. 0.10.1 Added Solaris package support and changed puppetmasterd to run as a non-root user. 0.10.0 Significant refactoring of how types, states, and parameters work, including breaking out parameters into a separate class. This refactoring did not introduce much new functionality, but made extension of Puppet significantly easier Also, fixed the bug with 'waitforcert' in puppetd. 0.9.4 Small fix to wrap the StatusServer class in the checks for required classes. 0.9.3 Fixed some significant bugs in cron job management. 0.9.2 Second Public Beta 0.9.0 First Public Beta diff --git a/lib/puppet/util/config.rb b/lib/puppet/util/config.rb index 6d42f0ea6..932314215 100644 --- a/lib/puppet/util/config.rb +++ b/lib/puppet/util/config.rb @@ -1,1116 +1,1153 @@ require 'puppet' require 'sync' require 'puppet/transportable' +require 'getoptlong' + # The class for handling configuration files. class Puppet::Util::Config include Enumerable include Puppet::Util @@sync = Sync.new attr_reader :file, :timer # Retrieve a config value def [](param) param = symbolize(param) # Yay, recursion. self.reparse() unless param == :filetimeout # Cache the returned values; this method was taking close to # 10% of the compile time. unless @returned[param] if @config.include?(param) if @config[param] @returned[param] = @config[param].value end else raise ArgumentError, "Undefined configuration parameter '%s'" % param end end return @returned[param] end # Set a config value. This doesn't set the defaults, it sets the value itself. def []=(param, value) @@sync.synchronize do # yay, thread-safe param = symbolize(param) unless @config.include?(param) raise Puppet::Error, "Attempt to assign a value to unknown configuration parameter %s" % param.inspect end unless @order.include?(param) @order << param end @config[param].value = value if @returned.include?(param) @returned.delete(param) end end return value end # A simplified equality operator. def ==(other) self.each { |myname, myobj| unless other[myname] == myobj.value return false end } return true end # Generate the list of valid arguments, in a format that GetoptLong can # understand, and add them to the passed option list. def addargs(options) - require 'getoptlong' - # Hackish, but acceptable. Copy the current ARGV for restarting. Puppet.args = ARGV.dup # Add all of the config parameters as valid options. - self.each { |param, obj| - if self.boolean?(param) - options << ["--#{param}", GetoptLong::NO_ARGUMENT] - options << ["--no-#{param}", GetoptLong::NO_ARGUMENT] - else - options << ["--#{param}", GetoptLong::REQUIRED_ARGUMENT] - end + self.each { |name, element| + element.getopt_args.each { |args| options << args } } return options end # Turn the config into a transaction and apply it def apply trans = self.to_transportable begin comp = trans.to_type trans = comp.evaluate trans.evaluate comp.remove rescue => detail if Puppet[:trace] puts detail.backtrace end Puppet.err "Could not configure myself: %s" % detail end end # Is our parameter a boolean parameter? def boolean?(param) param = symbolize(param) if @config.include?(param) and @config[param].kind_of? CBoolean return true else return false end end # Remove all set values, potentially skipping cli values. def clear(exceptcli = false) @config.each { |name, obj| unless exceptcli and obj.setbycli obj.clear end } @returned.clear # Don't clear the 'used' in this case, since it's a config file reparse, # and we want to retain this info. unless exceptcli @used = [] end end # This is mostly just used for testing. def clearused @returned.clear @used = [] end def each @order.each { |name| if @config.include?(name) yield name, @config[name] else raise Puppet::DevError, "%s is in the order but does not exist" % name end } end # Iterate over each section name. def eachsection yielded = [] @order.each { |name| if @config.include?(name) section = @config[name].section unless yielded.include? section yield section yielded << section end else raise Puppet::DevError, "%s is in the order but does not exist" % name end } end # Return an object by name. def element(param) param = symbolize(param) @config[param] end # Handle a command-line argument. def handlearg(opt, value = nil) value = munge_value(value) if value str = opt.sub(/^--/,'') bool = true newstr = str.sub(/^no-/, '') if newstr != str str = newstr bool = false end if self.valid?(str) if self.boolean?(str) self[str] = bool else self[str] = value end # Mark that this was set on the cli, so it's not overridden if the # config gets reread. @config[str.intern].setbycli = true else raise ArgumentError, "Invalid argument %s" % opt end end def include?(name) name = name.intern if name.is_a? String @config.include?(name) end + # check to see if a short name is already defined + def shortinclude?(short) + short = short.intern if name.is_a? String + @shortnames.include?(short) + end + # Create a new config object def initialize @order = [] @config = {} + @shortnames = {} @created = [] @returned = {} end # Make a directory with the appropriate user, group, and mode def mkdir(default) obj = nil unless obj = @config[default] raise ArgumentError, "Unknown default %s" % default end unless obj.is_a? CFile raise ArgumentError, "Default %s is not a file" % default end Puppet::Util::SUIDManager.asuser(obj.owner, obj.group) do mode = obj.mode || 0750 Dir.mkdir(obj.value, mode) end end # Return all of the parameters associated with a given section. def params(section) section = section.intern if section.is_a? String @config.find_all { |name, obj| obj.section == section }.collect { |name, obj| name } end # Parse the configuration file. def parse(file) configmap = parse_file(file) # We know we want the 'main' section if main = configmap[:main] set_parameter_hash(main) end # Otherwise, we only want our named section if @config.include?(:name) and named = configmap[symbolize(self[:name])] set_parameter_hash(named) end end # Parse the configuration file. As of May 2007, this is a backward-compatibility method and # will be deprecated soon. def old_parse(file) text = nil if file.is_a? Puppet::Util::LoadedFile @file = file else @file = Puppet::Util::LoadedFile.new(file) end # Don't create a timer for the old style parsing. # settimer() begin text = File.read(@file.file) rescue Errno::ENOENT raise Puppet::Error, "No such file %s" % file rescue Errno::EACCES raise Puppet::Error, "Permission denied to file %s" % file end @values = Hash.new { |names, name| names[name] = {} } # Get rid of the values set by the file, keeping cli values. self.clear(true) section = "puppet" metas = %w{owner group mode} values = Hash.new { |hash, key| hash[key] = {} } text.split(/\n/).each { |line| case line when /^\[(\w+)\]$/: section = $1 # Section names when /^\s*#/: next # Skip comments when /^\s*$/: next # Skip blanks when /^\s*(\w+)\s*=\s*(.+)$/: # settings var = $1.intern if var == :mode value = $2 else value = munge_value($2) end # Only warn if we don't know what this config var is. This # prevents exceptions later on. unless @config.include?(var) or metas.include?(var.to_s) Puppet.warning "Discarded unknown configuration parameter %s" % var.inspect next # Skip this line. end # Mmm, "special" attributes if metas.include?(var.to_s) unless values.include?(section) values[section] = {} end values[section][var.to_s] = value # If the parameter is valid, then set it. if section == Puppet[:name] and @config.include?(var) @config[var].value = value end next end # Don't override set parameters, since the file is parsed # after cli arguments are handled. unless @config.include?(var) and @config[var].setbycli Puppet.debug "%s: Setting %s to '%s'" % [section, var, value] self[var] = value end @config[var].section = symbolize(section) metas.each { |meta| if values[section][meta] if @config[var].respond_to?(meta + "=") @config[var].send(meta + "=", values[section][meta]) end end } else raise Puppet::Error, "Could not match line %s" % line end } end # Create a new element. The value is passed in because it's used to determine # what kind of element we're creating, but the value itself might be either # a default or a value, so we can't actually assign it. def newelement(hash) value = hash[:value] || hash[:default] klass = nil if hash[:section] hash[:section] = symbolize(hash[:section]) end case value when true, false, "true", "false": klass = CBoolean when /^\$\w+\//, /^\//: klass = CFile when String, Integer, Float: # nothing klass = CElement else raise Puppet::Error, "Invalid value '%s' for %s" % [value.inspect, hash[:name]] end hash[:parent] = self element = klass.new(hash) @order << element.name return element end # Iterate across all of the objects in a given section. def persection(section) section = symbolize(section) self.each { |name, obj| if obj.section == section yield obj end } end # Reparse our config file, if necessary. def reparse if defined? @file and @file.changed? Puppet.notice "Reparsing %s" % @file.file @@sync.synchronize do parse(@file) end reuse() end end def reuse return unless defined? @used @@sync.synchronize do # yay, thread-safe @used.each do |section| @used.delete(section) self.use(section) end end end # Get a list of objects per section def sectionlist sectionlist = [] self.each { |name, obj| section = obj.section || "puppet" sections[section] ||= [] unless sectionlist.include?(section) sectionlist << section end sections[section] << obj } return sectionlist, sections end # Convert a single section into transportable objects. def section_to_transportable(section, done = nil, includefiles = true) done ||= Hash.new { |hash, key| hash[key] = {} } objects = [] persection(section) do |obj| if @config[:mkusers] and @config[:mkusers].value [:owner, :group].each do |attr| type = nil if attr == :owner type = :user else type = attr end # If a user and/or group is set, then make sure we're # managing that object if obj.respond_to? attr and name = obj.send(attr) # Skip root or wheel next if %w{root wheel}.include?(name.to_s) # Skip owners and groups we've already done, but tag # them with our section if necessary if done[type].include?(name) tags = done[type][name].tags unless tags.include?(section) done[type][name].tags = tags << section end elsif newobj = Puppet::Type.type(type)[name] unless newobj.property(:ensure) newobj[:ensure] = "present" end newobj.tag(section) if type == :user newobj[:comment] ||= "%s user" % name end else newobj = Puppet::TransObject.new(name, type.to_s) newobj.tags = ["puppet", "configuration", section] newobj[:ensure] = "present" if type == :user newobj[:comment] ||= "%s user" % name end # Set the group appropriately for the user if type == :user newobj[:gid] = Puppet[:group] end done[type][name] = newobj objects << newobj end end end end if obj.respond_to? :to_transportable next if obj.value =~ /^\/dev/ transobjects = obj.to_transportable transobjects = [transobjects] unless transobjects.is_a? Array transobjects.each do |trans| # transportable could return nil next unless trans unless done[:file].include? trans.name @created << trans.name objects << trans done[:file][trans.name] = trans end end end end bucket = Puppet::TransBucket.new bucket.type = section bucket.push(*objects) bucket.keyword = "class" return bucket end # Set a bunch of defaults in a given section. The sections are actually pretty # pointless, but they help break things up a bit, anyway. def setdefaults(section, defs) section = symbolize(section) defs.each { |name, hash| if hash.is_a? Array tmp = hash hash = {} [:default, :desc].zip(tmp).each { |p,v| hash[p] = v } end name = symbolize(name) hash[:name] = name hash[:section] = section name = hash[:name] if @config.include?(name) raise Puppet::Error, "Parameter %s is already defined" % name end - @config[name] = newelement(hash) + tryconfig = newelement(hash) + if short = tryconfig.short + if other = @shortnames[short] + raise ArgumentError, "Parameter %s is already using short name '%s'" % [other.name, short] + end + @shortnames[short] = tryconfig + end + @config[name] = tryconfig } end # Create a timer to check whether the file should be reparsed. def settimer if Puppet[:filetimeout] > 0 @timer = Puppet.newtimer( :interval => Puppet[:filetimeout], :tolerance => 1, :start? => true ) do self.reparse() end end end # Convert our list of objects into a component that can be applied. def to_component transport = self.to_transportable return transport.to_type end # Convert our list of objects into a configuration file. def to_config str = %{The configuration file for #{Puppet[:name]}. Note that this file is likely to have unused configuration parameters in it; any parameter that's valid anywhere in Puppet can be in any config file, even if it's not used. Every section can specify three special parameters: owner, group, and mode. These parameters affect the required permissions of any files specified after their specification. Puppet will sometimes use these parameters to check its own configured state, so they can be used to make Puppet a bit more self-managing. Note also that the section names are entirely for human-level organizational purposes; they don't provide separate namespaces. All parameters are in a single namespace. Generated on #{Time.now}. }.gsub(/^/, "# ") # Add a section heading that matches our name. if @config.include?(:name) str += "[%s]\n" % self[:name] end eachsection do |section| persection(section) do |obj| str += obj.to_config + "\n" end end return str end # Convert our configuration into a list of transportable objects. def to_transportable done = Hash.new { |hash, key| hash[key] = {} } topbucket = Puppet::TransBucket.new if defined? @file.file and @file.file topbucket.name = @file.file else topbucket.name = "configtop" end topbucket.type = "puppetconfig" topbucket.top = true # Now iterate over each section eachsection do |section| topbucket.push section_to_transportable(section, done) end topbucket end # Convert to a parseable manifest def to_manifest transport = self.to_transportable manifest = transport.to_manifest + "\n" eachsection { |section| manifest += "include #{section}\n" } return manifest end # Create the necessary objects to use a section. This is idempotent; # you can 'use' a section as many times as you want. def use(*sections) @@sync.synchronize do # yay, thread-safe unless defined? @used @used = [] end runners = sections.collect { |s| symbolize(s) }.find_all { |s| ! @used.include? s } return if runners.empty? bucket = Puppet::TransBucket.new bucket.type = "puppetconfig" bucket.top = true # Create a hash to keep track of what we've done so far. @done = Hash.new { |hash, key| hash[key] = {} } runners.each do |section| bucket.push section_to_transportable(section, @done, false) end objects = bucket.to_type objects.finalize tags = nil if Puppet[:tags] tags = Puppet[:tags] Puppet[:tags] = "" end trans = objects.evaluate trans.ignoretags = true trans.configurator = true trans.evaluate if tags Puppet[:tags] = tags end # Remove is a recursive process, so it's sufficient to just call # it on the component. objects.remove(true) objects = nil runners.each { |s| @used << s } end end def valid?(param) param = symbolize(param) @config.has_key?(param) end # Open a file with the appropriate user, group, and mode def write(default, *args) obj = nil unless obj = @config[default] raise ArgumentError, "Unknown default %s" % default end unless obj.is_a? CFile raise ArgumentError, "Default %s is not a file" % default end chown = nil if Puppet::Util::SUIDManager.uid == 0 chown = [obj.owner, obj.group] else chown = [nil, nil] end Puppet::Util::SUIDManager.asuser(*chown) do mode = obj.mode || 0640 if args.empty? args << "w" end args << mode File.open(obj.value, *args) do |file| yield file end end end # Open a non-default file under a default dir with the appropriate user, # group, and mode def writesub(default, file, *args) obj = nil unless obj = @config[default] raise ArgumentError, "Unknown default %s" % default end unless obj.is_a? CFile raise ArgumentError, "Default %s is not a file" % default end chown = nil if Puppet::Util::SUIDManager.uid == 0 chown = [obj.owner, obj.group] else chown = [nil, nil] end Puppet::Util::SUIDManager.asuser(*chown) do mode = obj.mode || 0640 if args.empty? args << "w" end args << mode # Update the umask to make non-executable files Puppet::Util.withumask(File.umask ^ 0111) do File.open(file, *args) do |file| yield file end end end end private # Extra extra setting information for files. def extract_fileinfo(string) paramregex = %r{(\w+)\s*=\s*([\w\d]+)} result = {} string.scan(/\{\s*([^}]+)\s*\}/) do params = $1 params.split(/\s*,\s*/).each do |str| if str =~ /^\s*(\w+)\s*=\s*([\w\w]+)\s*$/ param, value = $1.intern, $2 result[param] = value unless [:owner, :mode, :group].include?(param) raise Puppet::Error, "Invalid file option '%s'" % param end if param == :mode and value !~ /^\d+$/ raise Puppet::Error, "File modes must be numbers" end else raise Puppet::Error, "Could not parse '%s'" % string end end return result end return nil end # Convert arguments into booleans, integers, or whatever. def munge_value(value) # Handle different data types correctly return case value when /^false$/i: false when /^true$/i: true when /^\d+$/i: Integer(value) else value.gsub(/^["']|["']$/,'').sub(/\s+$/, '') end end # This is an abstract method that just turns a file in to a hash of hashes. # We mostly need this for backward compatibility -- as of May 2007 we need to # support parsing old files with any section, or new files with just two # valid sections. def parse_file(file) text = nil if file.is_a? Puppet::Util::LoadedFile @file = file else @file = Puppet::Util::LoadedFile.new(file) end # Create a timer so that this file will get checked automatically # and reparsed if necessary. settimer() begin text = File.read(@file.file) rescue Errno::ENOENT raise Puppet::Error, "No such file %s" % file rescue Errno::EACCES raise Puppet::Error, "Permission denied to file %s" % file end result = Hash.new { |names, name| names[name] = {} } count = 0 # Default to 'main' for the section. section = :main result[section][:_meta] = {} text.split(/\n/).each { |line| count += 1 case line when /^\[(\w+)\]$/: section = $1.intern # Section names # Add a meta section result[section][:_meta] ||= {} when /^\s*#/: next # Skip comments when /^\s*$/: next # Skip blanks when /^\s*(\w+)\s*=\s*(.+)$/: # settings var = $1.intern # We don't want to munge modes, because they're specified in octal, so we'll # just leave them as a String, since Puppet handles that case correctly. if var == :mode value = $2 else value = munge_value($2) end # Check to see if this is a file argument and it has extra options begin if value.is_a?(String) and options = extract_fileinfo(value) result[section][:_meta][var] = options end result[section][var] = value rescue Puppet::Error => detail detail.file = file detail.line = line raise end else error = Puppet::Error.new("Could not match line %s" % line) error.file = file error.line = line raise error end } return result end # Take all members of a hash and assign their values appropriately. def set_parameter_hash(params) params.each do |param, value| next if param == :_meta unless @config.include?(param) Puppet.warning "Discarded unknown configuration parameter %s" % param next end if @config[param].setbycli Puppet.debug "Ignoring %s set by config file; overridden by cli" % param else self[param] = value end end if meta = params[:_meta] meta.each do |var, values| values.each do |param, value| @config[var].send(param.to_s + "=", value) end end end end # The base element type. class CElement attr_accessor :name, :section, :default, :parent, :setbycli - attr_reader :desc + attr_reader :desc, :short # Unset any set value. def clear @value = nil end # Do variable interpolation on the value. def convert(value) return value unless value return value unless value.is_a? String newval = value.gsub(/\$(\w+)|\$\{(\w+)\}/) do |value| varname = $2 || $1 if pval = @parent[varname] pval else raise Puppet::DevError, "Could not find value for %s" % parent end end return newval end def desc=(value) @desc = value.gsub(/^\s*/, '') end + # get the arguments in getopt format + def getopt_args + if short + [["--#{name}", "-#{short}", GetoptLong::REQUIRED_ARGUMENT]] + else + [["--#{name}", GetoptLong::REQUIRED_ARGUMENT]] + end + end + def hook=(block) meta_def :handle, &block end # Create the new element. Pretty much just sets the name. def initialize(args = {}) if args.include?(:parent) self.parent = args[:parent] args.delete(:parent) end args.each do |param, value| method = param.to_s + "=" unless self.respond_to? method raise ArgumentError, "%s does not accept %s" % [self.class, param] end self.send(method, value) end unless self.desc raise ArgumentError, "You must provide a description for the %s config option" % self.name end end def iscreated @iscreated = true end def iscreated? if defined? @iscreated return @iscreated else return false end end def set? if defined? @value and ! @value.nil? return true else return false end end + # short name for the celement + def short=(value) + if value.to_s.length != 1 + raise ArgumentError, "Short names can only be one character." + end + @short = value.to_s + end + # Convert the object to a config statement. def to_config str = @desc.gsub(/^/, "# ") + "\n" # Add in a statement about the default. if defined? @default and @default str += "# The default value is '%s'.\n" % @default end line = "%s = %s" % [@name, self.value] # If the value has not been overridden, then print it out commented # and unconverted, so it's clear that that's the default and how it # works. if defined? @value and ! @value.nil? line = "%s = %s" % [@name, self.value] else line = "# %s = %s" % [@name, @default] end str += line + "\n" str.gsub(/^/, " ") end # Retrieves the value, or if it's not set, retrieves the default. def value retval = nil if defined? @value and ! @value.nil? retval = @value elsif defined? @default retval = @default else return nil end if retval.is_a? String return convert(retval) else return retval end end # Set the value. def value=(value) if respond_to?(:validate) validate(value) end if respond_to?(:munge) @value = munge(value) else @value = value end if respond_to?(:handle) handle(@value) end end end # A file. class CFile < CElement attr_writer :owner, :group attr_accessor :mode, :create def group if defined? @group return convert(@group) else return nil end end def owner if defined? @owner return convert(@owner) else return nil end end # Set the type appropriately. Yep, a hack. This supports either naming # the variable 'dir', or adding a slash at the end. def munge(value) if value.to_s =~ /\/$/ @type = :directory return value.sub(/\/$/, '') end return value end # Return the appropriate type. def type value = self.value if @name.to_s =~ /dir/ return :directory elsif value.to_s =~ /\/$/ return :directory elsif value.is_a? String return :file else return nil end end # Convert the object to a TransObject instance. # FIXME There's no dependency system in place right now; if you use # a section that requires another section, there's nothing done to # correct that for you, at the moment. def to_transportable type = self.type return nil unless type path = self.value.split(File::SEPARATOR) path.shift # remove the leading nil objects = [] obj = Puppet::TransObject.new(self.value, "file") # Only create directories, or files that are specifically marked to # create. if type == :directory or self.create obj[:ensure] = type end [:mode].each { |var| if value = self.send(var) # Don't both converting the mode, since the file type # can handle it any old way. obj[var] = value end } # Only chown or chgrp when root if Puppet::Util::SUIDManager.uid == 0 [:group, :owner].each { |var| if value = self.send(var) obj[var] = value end } end # And set the loglevel to debug for everything obj[:loglevel] = "debug" # We're not actually modifying any files here, and if we allow a # filebucket to get used here we get into an infinite recursion # trying to set the filebucket up. obj[:backup] = false if self.section obj.tags += ["puppet", "configuration", self.section, self.name] end objects << obj objects end # Make sure any provided variables look up to something. def validate(value) return true unless value.is_a? String value.scan(/\$(\w+)/) { |name| name = $1 unless @parent.include?(name) raise ArgumentError, "Configuration parameter '%s' is undefined" % name end } end end # A simple boolean. class CBoolean < CElement + # get the arguments in getopt format + def getopt_args + if short + [["--#{name}", "-#{short}", GetoptLong::NO_ARGUMENT], + ["--no-#{name}", GetoptLong::NO_ARGUMENT]] + else + [["--#{name}", GetoptLong::NO_ARGUMENT], + ["--no-#{name}", GetoptLong::NO_ARGUMENT]] + end + end + def munge(value) case value when true, "true": return true when false, "false": return false else raise Puppet::Error, "Invalid value '%s' for %s" % [value.inspect, @name] end end end end # $Id$ diff --git a/lib/puppet/util/suidmanager.rb b/lib/puppet/util/suidmanager.rb index e15656d77..c1ff66248 100644 --- a/lib/puppet/util/suidmanager.rb +++ b/lib/puppet/util/suidmanager.rb @@ -1,71 +1,72 @@ require 'facter' require 'puppet/util/warnings' +require 'forwardable' module Puppet::Util::SUIDManager include Puppet::Util::Warnings extend Forwardable to_delegate_to_process = [ :euid=, :euid, :egid=, :egid, :uid=, :uid, :gid=, :gid ] to_delegate_to_process.each do |method| def_delegator Process, method module_function method end if Facter['kernel'].value == 'Darwin' # Cannot change real UID on Darwin so we set euid alias :uid :euid alias :gid :egid end # Runs block setting uid and gid if provided then restoring original ids def asuser new_uid=nil, new_gid=nil # We set both because some programs like to drop privs, i.e. bash. old_uid, old_gid = self.uid, self.gid old_euid, old_egid = self.euid, self.egid begin self.uid = convert_xid :uid, new_uid if new_uid self.gid = convert_xid :gid, new_gid if new_gid self.euid = convert_xid :uid, new_uid if new_uid self.egid = convert_xid :gid, new_gid if new_gid yield ensure self.uid, self.gid = old_uid, old_gid self.euid, self.egid = old_euid, old_egid end end module_function :asuser # Make sure the passed argument is a number. def convert_xid(type, id) map = {:gid => :group, :uid => :user} raise ArgumentError, "Invalid id type %s" % type unless map.include?(type) ret = Puppet::Util.send(type, id) if ret == nil raise Puppet::Error, "Invalid %s: %s" % [map[type], id] end return ret end module_function :convert_xid def run_and_capture(command, new_uid=nil, new_gid=nil) output = Puppet::Util.execute(command, :failonfail => false, :uid => new_uid, :gid => new_gid) [output, $?.dup] end module_function :run_and_capture def system(command, new_uid=nil, new_gid=nil) status = nil asuser(new_uid, new_gid) do Kernel.system(command) status = $?.dup end status end module_function :system end # $Id$ diff --git a/test/util/config.rb b/test/util/config.rb index 71343556f..9a1017058 100755 --- a/test/util/config.rb +++ b/test/util/config.rb @@ -1,1248 +1,1311 @@ #!/usr/bin/env ruby $:.unshift("../lib").unshift("../../lib") if __FILE__ =~ /\.rb$/ require 'mocha' require 'puppettest' require 'puppet/util/config' require 'puppettest/parsertesting' class TestConfig < Test::Unit::TestCase include PuppetTest include PuppetTest::ParserTesting + CElement = Puppet::Util::Config::CElement + CBoolean = Puppet::Util::Config::CBoolean def setup super @config = mkconfig end def set_configs(config = nil) config ||= @config config.setdefaults("main", :one => ["a", "one"], :two => ["a", "two"], :yay => ["/default/path", "boo"], :mkusers => [true, "uh, yeah"], :name => ["testing", "a"] ) config.setdefaults("section1", :attr => ["a", "one"], :attrdir => ["/another/dir", "two"], :attr3 => ["$attrdir/maybe", "boo"] ) end def check_for_users count = Puppet::Type.type(:user).inject(0) { |c,o| c + 1 } assert(count > 0, "Found no users") end def test_to_transportable set_configs trans = nil assert_nothing_raised("Could not convert to a transportable") { trans = @config.to_transportable } comp = nil assert_nothing_raised("Could not convert transportable to component") { comp = trans.to_type } assert_nothing_raised("Could not retrieve transported config") { comp.retrieve } end def test_to_manifest set_configs manifest = nil assert_nothing_raised("Could not convert to a manifest") { manifest = @config.to_manifest } Puppet[:parseonly] = true interp = nil assert_nothing_raised do interp = mkinterp :Code => manifest, :UseNodes => false end trans = nil assert_nothing_raised do trans = interp.evaluate(nil, {}) end assert_nothing_raised("Could not instantiate objects") { trans.to_type } end def test_to_comp set_configs comp = nil assert_nothing_raised("Could not convert to a component") { comp = @config.to_component } assert_nothing_raised("Could not retrieve component") { comp.retrieve } end def test_to_config set_configs newc = mkconfig set_configs(newc) # Reset all of the values, so we know they're changing. newc.each do |name, obj| next if name == :name newc[name] = true end newfile = tempfile() File.open(newfile, "w") { |f| @config.to_config.split("\n").each do |line| # Uncomment the settings, so they actually take. if line =~ / = / f.puts line.sub(/^\s*#/, '') else f.puts line end end } assert_nothing_raised("Could not parse generated configuration") { newc.parse(newfile) } @config.each do |name, object| assert_equal(@config[name], newc[name], "Parameter %s is not the same" % name) end end def mkconfig c = nil assert_nothing_raised { c = Puppet::Util::Config.new } return c end def test_addbools assert_nothing_raised { @config.setdefaults(:testing, :booltest => [true, "testing"]) } assert(@config[:booltest]) @config = mkconfig assert_nothing_raised { @config.setdefaults(:testing, :booltest => ["true", "testing"]) } assert(@config[:booltest]) assert_nothing_raised { @config[:booltest] = false } assert(! @config[:booltest], "Booltest is not false") assert_nothing_raised { @config[:booltest] = "false" } assert(! @config[:booltest], "Booltest is not false") assert_raise(Puppet::Error) { @config[:booltest] = "yayness" } assert_raise(Puppet::Error) { @config[:booltest] = "/some/file" } end def test_strings val = "this is a string" assert_nothing_raised { @config.setdefaults(:testing, :strtest => [val, "testing"]) } assert_equal(val, @config[:strtest]) # Verify that variables are interpolated assert_nothing_raised { @config.setdefaults(:testing, :another => ["another $strtest", "testing"]) } assert_equal("another #{val}", @config[:another]) end def test_files c = mkconfig parent = "/puppet" assert_nothing_raised { @config.setdefaults(:testing, :parentdir => [parent, "booh"]) } assert_nothing_raised { @config.setdefaults(:testing, :child => ["$parent/child", "rah"]) } assert_equal(parent, @config[:parentdir]) assert_equal("/puppet/child", File.join(@config[:parentdir], "child")) end def test_getset initial = "an initial value" assert_raise(Puppet::Error) { @config[:yayness] = initial } default = "this is a default" assert_nothing_raised { @config.setdefaults(:testing, :yayness => [default, "rah"]) } assert_equal(default, @config[:yayness]) assert_nothing_raised { @config[:yayness] = initial } assert_equal(initial, @config[:yayness]) assert_nothing_raised { @config.clear } assert_equal(default, @config[:yayness]) assert_nothing_raised { @config[:yayness] = "not default" } assert_equal("not default", @config[:yayness]) end def test_parse_file text = %{ one = this is a test two = another test owner = root group = root yay = /a/path [main] four = five six = seven [section1] attr = value owner = puppet group = puppet attrdir = /some/dir attr3 = $attrdir/other } file = tempfile() File.open(file, "w") { |f| f.puts text } @config.expects(:settimer) result = nil assert_nothing_raised { result = @config.send(:parse_file, file) } main = result[:main] assert(main, "Did not get section for main") { :one => "this is a test", :two => "another test", :owner => "root", :group => "root", :yay => "/a/path", :four => "five", :six => "seven" }.each do |param, value| assert_equal(value, main[param], "Param %s was not set correctly in main" % param) end section1 = result[:section1] assert(section1, "Did not get section1") { :attr => "value", :owner => "puppet", :group => "puppet", :attrdir => "/some/dir", :attr3 => "$attrdir/other" }.each do |param, value| assert_equal(value, section1[param], "Param %s was not set correctly in section1" % param) end end def test_old_parse text = %{ one = this is a test two = another test owner = root group = root yay = /a/path [section1] attr = value owner = puppet group = puppet attrdir = /some/dir attr3 = $attrdir/other } file = tempfile() File.open(file, "w") { |f| f.puts text } assert_nothing_raised { @config.setdefaults("puppet", :one => ["a", "one"], :two => ["a", "two"], :yay => ["/default/path", "boo"], :mkusers => [true, "uh, yeah"] ) } assert_nothing_raised { @config.setdefaults("section1", :attr => ["a", "one"], :attrdir => ["/another/dir", "two"], :attr3 => ["$attrdir/maybe", "boo"] ) } assert_nothing_raised { @config.old_parse(file) } assert_equal("value", @config[:attr]) assert_equal("/some/dir", @config[:attrdir]) assert_equal(:directory, @config.element(:attrdir).type) assert_equal("/some/dir/other", @config[:attr3]) elem = nil assert_nothing_raised { elem = @config.element(:attr3) } assert(elem) assert_equal("puppet", elem.owner) config = nil assert_nothing_raised { config = @config.to_config } assert_nothing_raised("Could not create transportable config") { @config.to_transportable } end def test_parse result = { :main => {:main => "main", :bad => "invalid", :cliparam => "reset"}, :puppet => {:other => "puppet", :cliparam => "reset"}, :puppetd => {:other => "puppetd", :cliparam => "reset"} } # Set our defaults, so they're valid. Don't define 'bad', since we want to test for failures. @config.setdefaults(:main, :main => ["whatever", "a"], :cliparam => ["default", "y"], :other => ["a", "b"], :name => ["puppet", "b"] # our default name ) @config.setdefaults(:other, :one => ["whatever", "a"], :two => ["default", "y"], :apple => ["a", "b"], :shoe => ["puppet", "b"] # our default name ) @config.handlearg("--cliparam", "changed") @config.expects(:parse_file).returns(result).times(2) # First do it with our name being 'puppet' assert_nothing_raised("Could not handle parse results") do @config.parse(tempfile) end assert_logged(:warning, /unknown configuration parameter bad/, "Did not log invalid config param") assert_equal("main", @config[:main], "Did not get main value") assert_equal("puppet", @config[:other], "Did not get name value") assert_equal("changed", @config[:cliparam], "CLI values were overridden by config") # Now switch names and make sure the parsing switches, too. @config.clear(true) @config[:name] = :puppetd assert_nothing_raised("Could not handle parse results") do @config.parse(tempfile) end assert_logged(:warning, /unknown configuration parameter bad/, "Did not log invalid config param") assert_equal("main", @config[:main], "Did not get main value") assert_equal("puppetd", @config[:other], "Did not get name value") assert_equal("changed", @config[:cliparam], "CLI values were overridden by config") end # Make sure we can extract file options correctly. def test_parsing_file_options @config.setdefaults(:whev, :file => { :desc => "whev", :default => "/default", :owner => "me", :group => "me", :mode => "755" } ) file = tempfile { :pass => { " {owner = you}" => {:owner => "you"}, " {owner = you, group = you}" => {:owner => "you", :group => "you"}, " {owner = you, group = you, mode = 755}" => {:owner => "you", :group => "you", :mode => "755"}, " { owner = you, group = you } " => {:owner => "you", :group => "you"}, "{owner=you,group=you} " => {:owner => "you", :group => "you"}, "{owner=you,} " => {:owner => "you"} }, :fail => [ %{{owner = you group = you}}, %{{owner => you, group => you}}, %{{user => you}}, %{{random => you}}, %{{mode => you}}, # make sure modes are numbers %{{owner => you}} ] }.each do |type, list| list.each do |value| if type == :pass value, should = value[0], value[1] end # Write our file out File.open(file, "w") do |f| f.puts %{[main]\nfile = /other%s} % value end if type == :fail assert_raise(Puppet::Error, "Did not fail on %s" % value.inspect) do @config.send(:parse_file, file) end else result = nil assert_nothing_raised("Failed to parse %s" % value.inspect) do result = @config.send(:parse_file, file) end assert_equal(should, result[:main][:_meta][:file], "Got incorrect return for %s" % value.inspect) end end end end # Make sure file options returned from parse_file are handled correctly. def test_parsed_file_options @config.setdefaults(:whev, :file => { :desc => "whev", :default => "/default", :owner => "me", :group => "me", :mode => "755" } ) result = { :main => { :file => "/other", :_meta => { :file => { :owner => "you", :group => "you", :mode => "644" } } } } @config.expects(:parse_file).returns(result) assert_nothing_raised("Could not handle file options") do @config.parse("/whatever") end # Get the actual object, so we can verify metadata file = @config.element(:file) assert_equal("you", file.owner, "Did not pass on user") assert_equal("you", file.group, "Did not pass on group") assert_equal("644", file.mode, "Did not pass on mode") end def test_arghandling c = mkconfig assert_nothing_raised { @config.setdefaults("testing", :onboolean => [true, "An on bool"], :offboolean => [false, "An off bool"], :string => ["a string", "A string arg"], :file => ["/path/to/file", "A file arg"] ) } data = { :onboolean => [true, false], :offboolean => [true, false], :string => ["one string", "another string"], :file => %w{/a/file /another/file} } data.each { |param, values| values.each { |val| opt = nil arg = nil if @config.boolean?(param) if val opt = "--%s" % param else opt = "--no-%s" % param end else opt = "--%s" % param arg = val end assert_nothing_raised("Could not handle arg %s with value %s" % [opt, val]) { @config.handlearg(opt, arg) } } } end - def test_argadding - c = mkconfig - - assert_nothing_raised { - @config.setdefaults("testing", - :onboolean => [true, "An on bool"], - :offboolean => [false, "An off bool"], - :string => ["a string", "A string arg"], - :file => ["/path/to/file", "A file arg"] - ) - } - options = [] - - @config.addargs(options) + def test_addargs + @config.setdefaults("testing", + :onboolean => [true, "An on bool"], + :offboolean => [false, "An off bool"], + :string => ["a string", "A string arg"], + :file => ["/path/to/file", "A file arg"] + ) + + should = [] + @config.each { |name, element| + element.expects(:getopt_args).returns([name]) + should << name + } + result = [] + assert_nothing_raised("Add args failed") do + @config.addargs(result) + end + assert_equal(should, result, "Did not call addargs correctly.") - @config.each { |param, obj| - opt = "--%s" % param - assert(options.find { |ary| - ary[0] == opt - }, "Argument %s was not added" % opt) + end - if @config.boolean?(param) - o = "--no-%s" % param - assert(options.find { |ary| - ary[0] == o - }, "Boolean off %s was not added" % o) + def test_addargs_functional + @config.setdefaults("testing", + :onboolean => [true, "An on bool"], + :string => ["a string", "A string arg"] + ) + result = [] + should = [] + assert_nothing_raised("Add args failed") do + @config.addargs(result) + end + @config.each do |name, element| + if name == :onboolean + should << ["--onboolean", GetoptLong::NO_ARGUMENT] + should << ["--no-onboolean", GetoptLong::NO_ARGUMENT] + elsif name == :string + should << ["--string", GetoptLong::REQUIRED_ARGUMENT] end - } + end + assert_equal(should, result, "Add args functional test failed") end def test_usesection # We want to make sure that config processes do not result in graphing. Puppet[:graphdir] = tempfile() Puppet[:graph] = true Dir.mkdir(Puppet[:graphdir]) c = mkconfig dir = tempfile() file = "$mydir/myfile" realfile = File.join(dir, "myfile") otherfile = File.join(dir, "otherfile") section = "testing" assert_nothing_raised { @config.setdefaults(section, :mydir => [dir, "A dir arg"], :otherfile => { :default => "$mydir/otherfile", :create => true, :desc => "A file arg" }, :myfile => [file, "A file arg"] ) } assert_nothing_raised("Could not use a section") { @config.use(section) } assert_nothing_raised("Could not reuse a section") { @config.use(section) } # Make sure it didn't graph anything, which is the only real way # to test that the transaction was marked as a configurator. assert(Dir.entries(Puppet[:graphdir]).reject { |f| f =~ /^\.\.?$/ }.empty?, "Graphed config process") assert(FileTest.directory?(dir), "Did not create directory") assert(FileTest.exists?(otherfile), "Did not create file") assert(!FileTest.exists?(realfile), "Created file") end def test_setdefaultsarray c = mkconfig assert_nothing_raised { @config.setdefaults("yay", :a => [false, "some value"], :b => ["/my/file", "a file"] ) } assert_equal(false, @config[:a], "Values are not equal") assert_equal("/my/file", @config[:b], "Values are not equal") end def test_setdefaultshash c = mkconfig assert_nothing_raised { @config.setdefaults("yay", :a => {:default => false, :desc => "some value"}, :b => {:default => "/my/file", :desc => "a file"} ) } assert_equal(false, @config[:a], "Values are not equal") assert_equal("/my/file", @config[:b], "Values are not equal") end def test_reuse c = mkconfig file = tempfile() section = "testing" assert_nothing_raised { @config.setdefaults(section, :myfile => {:default => file, :create => true, :desc => "yay"} ) } assert_nothing_raised("Could not use a section") { @config.use(section) } assert(FileTest.exists?(file), "Did not create file") assert(! Puppet::Type.type(:file)[file], "File obj still exists") File.unlink(file) @config.reuse assert(FileTest.exists?(file), "Did not create file") end def test_mkusers c = mkconfig file = tempfile() section = "testing" assert_nothing_raised { @config.setdefaults(section, :mkusers => [false, "yay"], :myfile => { :default => file, :owner => "pptest", :group => "pptest", :desc => "yay", :create => true } ) } comp = nil assert_nothing_raised { comp = @config.to_component } [:user, :group].each do |type| # The objects might get created internally by Puppet::Util; just # make sure they're not being managed if obj = Puppet.type(type)["pptest"] assert(! obj.managed?, "%s objectis managed" % type) end end comp.each { |o| o.remove } @config[:mkusers] = true assert_nothing_raised { @config.to_component } user = Puppet.type(:user)["pptest"] assert(user, "User object did not get created") assert(user.managed?, "User object is not managed.") assert(user.should(:comment), "user does not have a comment set") group = Puppet.type(:group)["pptest"] assert(group, "Group object did not get created") assert(group.managed?, "Group object is not managed." ) if Process.uid == 0 cleanup do user[:ensure] = :absent group[:ensure] = :absent assert_apply(user, group) end assert_apply(user, group) end end def test_notmanagingdev c = mkconfig path = "/dev/testing" @config.setdefaults(:test, :file => { :default => path, :mode => 0640, :desc => 'yay' } ) assert_nothing_raised { @config.to_component } assert(! Puppet.type(:file)["/dev/testing"], "Created dev file") end def test_groupsetting cfile = tempfile() group = "yayness" File.open(cfile, "w") do |f| f.puts "[main] group = #{group} " end config = mkconfig config.setdefaults(Puppet[:name], :group => ["puppet", "a group"]) assert_nothing_raised { config.parse(cfile) } assert_equal(group, config[:group], "Group did not take") end # provide a method to modify and create files w/out specifying the info # already stored in a config def test_writingfiles File.umask(0022) path = tempfile() mode = 0644 config = mkconfig args = { :default => path, :mode => mode, :desc => "yay" } user = nonrootuser() group = nonrootgroup() if Puppet::Util::SUIDManager.uid == 0 args[:owner] = user.name args[:group] = group.name end config.setdefaults(:testing, :myfile => args) assert_nothing_raised { config.write(:myfile) do |file| file.puts "yay" end } assert_equal(mode, filemode(path), "Modes are not equal") # OS X is broken in how it chgrps files if Puppet::Util::SUIDManager.uid == 0 assert_equal(user.uid, File.stat(path).uid, "UIDS are not equal") case Facter["operatingsystem"].value when /BSD/, "Darwin": # nothing else assert_equal(group.gid, File.stat(path).gid, "GIDS are not equal") end end end def test_mkdir File.umask(0022) path = tempfile() mode = 0755 config = mkconfig args = { :default => path, :mode => mode, :desc => "a file" } user = nonrootuser() group = nonrootgroup() if Puppet::Util::SUIDManager.uid == 0 args[:owner] = user.name args[:group] = group.name end config.setdefaults(:testing, :mydir => args) assert_nothing_raised { config.mkdir(:mydir) } assert_equal(mode, filemode(path), "Modes are not equal") # OS X and *BSD is broken in how it chgrps files if Puppet::Util::SUIDManager.uid == 0 assert_equal(user.uid, File.stat(path).uid, "UIDS are not equal") case Facter["operatingsystem"].value when /BSD/, "Darwin": # nothing else assert_equal(group.gid, File.stat(path).gid, "GIDS are not equal") end end end def test_booleans_and_integers config = mkconfig config.setdefaults(:mysection, :booltest => [false, "yay"], :inttest => [14, "yay"] ) file = tempfile() File.open(file, "w") do |f| f.puts %{ [main] booltest = true inttest = 27 } end assert_nothing_raised { config.parse(file) } assert_equal(true, config[:booltest], "Boolean was not converted") assert_equal(27, config[:inttest], "Integer was not converted") # Now make sure that they get converted through handlearg config.handlearg("--inttest", "true") assert_equal(true, config[:inttest], "Boolean was not converted") config.handlearg("--no-booltest", "false") assert_equal(false, config[:booltest], "Boolean was not converted") end # Make sure that tags are ignored when configuring def test_configs_ignore_tags config = mkconfig file = tempfile() config.setdefaults(:mysection, :mydir => [file, "a file"] ) Puppet[:tags] = "yayness" assert_nothing_raised { config.use(:mysection) } assert(FileTest.directory?(file), "Directory did not get created") assert_equal("yayness", Puppet[:tags], "Tags got changed during config") end def test_configs_replace_in_url config = mkconfig config.setdefaults(:mysection, :name => ["yayness", "yay"]) config.setdefaults(:mysection, :url => ["http://$name/rahness", "yay"]) val = nil assert_nothing_raised { val = config[:url] } assert_equal("http://yayness/rahness", val, "Config got messed up") end def test_correct_type_assumptions config = mkconfig file = Puppet::Util::Config::CFile element = Puppet::Util::Config::CElement bool = Puppet::Util::Config::CBoolean # We have to keep these ordered, unfortunately. [ ["/this/is/a/file", file], ["true", bool], [true, bool], ["false", bool], ["server", element], ["http://$server/yay", element], ["$server/yayness", file], ["$server/yayness.conf", file] ].each do |ary| value, type = ary elem = nil assert_nothing_raised { elem = config.newelement( :name => value, :default => value, :desc => name.to_s, :section => :yayness ) } assert_instance_of(type, elem, "%s got created as wrong type" % value.inspect) end end # Make sure we correctly reparse our config files but don't lose CLI values. def test_reparse Puppet[:filetimeout] = 0 config = mkconfig() config.setdefaults(:mysection, :default => ["default", "yay"]) config.setdefaults(:mysection, :clichange => ["clichange", "yay"]) config.setdefaults(:mysection, :filechange => ["filechange", "yay"]) file = tempfile() # Set one parameter in the file File.open(file, "w") { |f| f.puts %{[main]\nfilechange = filevalue} } assert_nothing_raised { config.parse(file) } # Set another "from the cli" assert_nothing_raised { config.handlearg("clichange", "clivalue") } # And leave the other unset assert_equal("default", config[:default]) assert_equal("filevalue", config[:filechange]) assert_equal("clivalue", config[:clichange]) # Now rewrite the file File.open(file, "w") { |f| f.puts %{[main]\nfilechange = newvalue} } cfile = config.file cfile.send("tstamp=".intern, Time.now - 50) # And check all of the values assert_equal("default", config[:default]) assert_equal("clivalue", config[:clichange]) assert_equal("newvalue", config[:filechange]) end def test_parse_removes_quotes config = mkconfig() config.setdefaults(:mysection, :singleq => ["single", "yay"]) config.setdefaults(:mysection, :doubleq => ["double", "yay"]) config.setdefaults(:mysection, :none => ["noquote", "yay"]) config.setdefaults(:mysection, :middle => ["midquote", "yay"]) file = tempfile() # Set one parameter in the file File.open(file, "w") { |f| f.puts %{[main]\n singleq = 'one' doubleq = "one" none = one middle = mid"quote } } assert_nothing_raised { config.parse(file) } %w{singleq doubleq none}.each do |p| assert_equal("one", config[p], "%s did not match" % p) end assert_equal('mid"quote', config["middle"], "middle did not match") end def test_timer Puppet[:filetimeout] = 0.1 origpath = tempfile() config = mkconfig() config.setdefaults(:mysection, :paramdir => [tempfile(), "yay"]) file = tempfile() # Set one parameter in the file File.open(file, "w") { |f| f.puts %{[main]\n paramdir = #{origpath} } } assert_nothing_raised { config.parse(file) config.use(:mysection) } assert(FileTest.directory?(origpath), "dir did not get created") # Now start the timer assert_nothing_raised { EventLoop.current.monitor_timer config.timer } newpath = tempfile() File.open(file, "w") { |f| f.puts %{[main]\n paramdir = #{newpath} } } config.file.send("tstamp=".intern, Time.now - 50) sleep 1 assert_equal(newpath, config["paramdir"], "File did not get reparsed from timer") assert(FileTest.directory?(newpath), "new dir did not get created") end # Test that config parameters correctly call passed-in blocks when the value # is set. def test_paramblocks config = mkconfig() testing = nil elem = nil assert_nothing_raised do elem = config.newelement :default => "yay", :name => :blocktest, :desc => "boo", :section => :test, :hook => proc { |value| testing = value } end assert_nothing_raised do assert_equal("yay", elem.value) end assert_nothing_raised do elem.value = "yaytest" end assert_nothing_raised do assert_equal("yaytest", elem.value) end assert_equal("yaytest", testing) assert_nothing_raised do elem.value = "another" end assert_nothing_raised do assert_equal("another", elem.value) end assert_equal("another", testing) # Now verify it works from setdefault assert_nothing_raised do config.setdefaults :test, :blocktest2 => { :default => "yay", :desc => "yay", :hook => proc { |v| testing = v } } end assert_equal("yay", config[:blocktest2]) assert_nothing_raised do config[:blocktest2] = "footest" end assert_equal("footest", config[:blocktest2]) assert_equal("footest", testing) end def test_no_modify_root config = mkconfig config.setdefaults(:yay, :mydir => {:default => tempfile(), :mode => 0644, :owner => "root", :group => "root", :desc => "yay" }, :mkusers => [false, "yay"] ) assert_nothing_raised do config.use(:yay) end # Now enable it so they'll be added config[:mkusers] = true comp = config.to_component Puppet::Type.type(:user).each do |u| assert(u.name != "root", "Tried to manage root user") end Puppet::Type.type(:group).each do |u| assert(u.name != "root", "Tried to manage root group") assert(u.name != "wheel", "Tried to manage wheel group") end # assert(yay, "Did not find yay component") # yay.each do |c| # puts @config.ref # end # assert(! yay.find { |o| o.class.name == :user and o.name == "root" }, # "Found root user") # assert(! yay.find { |o| o.class.name == :group and o.name == "root" }, # "Found root group") end # #415 def test_remove_trailing_spaces config = mkconfig() config.setdefaults(:yay, :rah => ["testing", "a desc"]) file = tempfile() File.open(file, "w") { |f| f.puts "rah = something " } assert_nothing_raised { config.parse(file) } assert_equal("something", config[:rah], "did not remove trailing whitespace in parsing") end # #484 def test_parsing_unknown_variables logstore() config = mkconfig() config.setdefaults(:mysection, :one => ["yay", "yay"]) file = tempfile() File.open(file, "w") { |f| f.puts %{[main]\n one = one two = yay } } assert_nothing_raised("Unknown parameter threw an exception") do config.parse(file) end assert(@logs.detect { |l| l.message =~ /unknown configuration/ and l.level == :warning }, "Did not generate warning message") end def test_multiple_interpolations @config.setdefaults(:section, :one => ["oneval", "yay"], :two => ["twoval", "yay"], :three => ["$one/$two", "yay"] ) assert_equal("oneval/twoval", @config[:three], "Did not interpolate multiple variables") end # Make sure we can replace ${style} var names def test_curly_replacements @config.setdefaults(:section, :one => ["oneval", "yay"], :two => ["twoval", "yay"], :three => ["$one/${two}/${one}/$two", "yay"] ) assert_equal("oneval/twoval/oneval/twoval", @config[:three], "Did not interpolate curlied variables") end # Discovered from #734 def test_set_parameter_hash @config.setdefaults(:section, :unchanged => ["unval", "yay"], :normal => ["normalval", "yay"], :cliparam => ["clival", "yay"], :file => ["/my/file", "yay"] ) # Set the cli param using the cli method @config.handlearg("--cliparam", "other") # Make sure missing params just throw warnings, not errors assert_nothing_raised("Could not call set_parameter_hash with an invalid option") do @config.send(:set_parameter_hash, :missing => "something") end # Make sure normal values get set assert_nothing_raised("Could not call set_parameter_hash with a normal value") do @config.send(:set_parameter_hash, :normal => "abnormal") end assert_equal("abnormal", @config[:normal], "Value did not get set") # Make sure cli-set values don't get overridden assert_nothing_raised("Could not call set_parameter_hash with an override of a cli value") do @config.send(:set_parameter_hash, :cliparam => "something else") end assert_equal("other", @config[:cliparam], "CLI value was overridden by config value") # Make sure the meta stuff works assert_nothing_raised("Could not call set_parameter_hash with meta info") do @config.send(:set_parameter_hash, :file => "/other/file", :_meta => {:file => {:mode => "0755"}}) end assert_equal("/other/file", @config[:file], "value with meta info was overridden by config value") assert_equal("0755", @config.element(:file).mode, "Did not set mode from meta info") # And make sure other params are unchanged assert_equal("unval", @config[:unchanged], "Unchanged value has somehow changed") end + + # Test to make sure that we can set and get a short name + def test_celement_short_name + element = nil + assert_nothing_raised("Could not create celement") do + element = CElement.new :short => "n", :desc => "anything" + end + assert_equal("n", element.short, "Short value is not retained") + + assert_raise(ArgumentError,"Allowed multicharactered short names.") do + element = CElement.new :short => "no", :desc => "anything" + end + end + + # Test to make sure that no two celements have the same short name + def test_celement_short_name_not_duplicated + config = mkconfig + assert_nothing_raised("Could not create celement with short name.") do + config.setdefaults(:main, + :one => { :default => "blah", :desc => "anything", :short => "o" }) + end + assert_nothing_raised("Could not create second celement with short name.") do + config.setdefaults(:main, + :two => { :default => "blah", :desc => "anything", :short => "i" }) + end + assert_raise(ArgumentError, "Could create second celement with duplicate short name.") do + config.setdefaults(:main, + :three => { :default => "blah", :desc => "anything", :short => "i" }) + end + # make sure that when the above raises an expection that the config is not included + assert(!config.include?(:three), "Invalid configuration item was retained") + end + + # Tell getopt which arguments are valid + def test_get_getopt_args + element = CElement.new :name => "foo", :desc => "anything" + assert_equal([["--foo", GetoptLong::REQUIRED_ARGUMENT]], element.getopt_args, "Did not produce appropriate getopt args") + + element.short = "n" + assert_equal([["--foo", "-n", GetoptLong::REQUIRED_ARGUMENT]], element.getopt_args, "Did not produce appropriate getopt args") + + element = CBoolean.new :name => "foo", :desc => "anything" + assert_equal([["--foo", GetoptLong::NO_ARGUMENT], ["--no-foo", GetoptLong::NO_ARGUMENT]], + element.getopt_args, "Did not produce appropriate getopt args") + + element.short = "n" + assert_equal([["--foo", "-n", GetoptLong::NO_ARGUMENT],["--no-foo", GetoptLong::NO_ARGUMENT]], + element.getopt_args, "Did not produce appropriate getopt args") + end end # $Id$