diff --git a/CHANGELOG b/CHANGELOG index 2b398419b..234b24393 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -1,979 +1,984 @@ + Somewhat refactored fileserving so that it no longer caches + any objects, nor does it use Puppet's RAL resources. In the + process, I fixed #894 (you can now copy links) and refactored + other classes as necessary. Mostly it was fixing tests. + Hopefully partially fixed #1010 -- clients should now fail to install files whose checksums do not match the checksum from the server. Fixed #1018 -- resources now have their namevars added as aliases in the resource catalog, just like they were added in the resource classes. Fixed #1037 -- remote unreadable files no longer have the permission denied exceptions caught, thus forbidding them from being replaced with 'nil'. Fixed #1043 -- autoloading now searches the plugins directory in each module, in addition to the lib directory. The 'lib' directory is also deprecated, but supported for now to give people a chance to convert. Fixed #1003 -- Applying DavidS's patch to fix searching for tags in sql. Fixed #992 -- Puppet is now compatible with gems 1.0.1. Fixed #968 again, this time with tests -- parseonly works, including not compiling the configurations, and also storeconfigs is no longer required during parse-testing. Fixed #1021 -- the problem was that my method of determining the in-degree sometimes resulted in a lower number than the number of in-edges. Fixed #997 -- virtual defined types are no longer evaluated. NOTE: This introduces a behaviour change, in that you previously could realize a resource within a virtual defined resource, and now you must realize the entire defined resource, rather than just the contained resource. Fixed #1030 - class and definition evaluation has been significantly refactored, fixing this problem and making the whole interplay between the classes, definitions, and nodes, and the Compile class much cleaner. - Exec resources must now have unique names. This is easily accomplished by - just specifying a unique name with whatever (unique or otherwise) command - you need. + Exec resources must now have unique names, although the commands can still + be duplicated. This is easily accomplished by just specifying a unique + name with whatever (unique or otherwise) command you need. Fixed #989 -- missing CRL files are correctly ignored, and the value should be set to 'false' to explicitly not look for these files. Fixed #1017 -- environment-specific modulepath is no longer ignored. Fixing #794 -- consolidating the gentoo configuration files. Fixing #976 -- both the full name of qualified classes and the class parts are now added as tags. I've also created a Tagging module that we should push throughout the rest of the system that uses tags. Fixing #995 -- puppetd no longer dies at startup if the server is not running. Fixing #977 -- the rundir is again set to 1777. Fixed #971 -- classes can once again be included multiple times. Added builtin support for Nagios types using Naginator to parse and generate the files. 0.24.1 Updated vim filetype detection. (#900 and #963) Default resources like schedules no longer conflict with managed resources. (#965) Removing the ability to disable http keep-alive, since it didn't really work anyway and it should no longer be necessary. Refactored http keep-alive so it actually works again. This should be sufficient enough that we no longer need the ability to disable keep-alive. There is now a central module responsible for managing HTTP instances, along with all certificates in those instances. Fixed a backward compatibility issue when running 0.23.x clients against 0.24.0 servers -- relationships would consistently not work. (#967) Closing existing http connections when opening a new one, and closing all connections after each run. (#961) Removed warning about deprecated explicit plugins mounts. 0.24.0 (misspiggy) Modifying the behaviour of the certdnsnames setting. It now defaults to an empty string, and will only be used if it is set to something else. If it is set, then the host's FQDN will also be added as an alias. The default behaviour is now to add 'puppet' and 'puppet.$domain' as DNS aliases when the name for the cert being signed is equal to the signing machine's name, which will only be the case for CA servers. This should result in servers always having the alias set up and no one else, but you can still override the aliases if you want. External node support now requires that you set the 'node_terminus' setting to 'exec'. See the IndirectionReference on the wiki for more information. http_enable_post_connection_check added as a configuration option for puppetd. This defaults to true, which validates the server SSL certificate against the requested host name in new versions of ruby. See #896 for more information. Mounts no longer remount swap filesystems. Slightly modifying how services manage their list of paths (and adding documention for it). Services now default to the paths specified by the provider classes. Removed 'type' as a valid attribute for services, since it's been deprecated since the creation of providers. Removed 'running' as a valid attribute for services, since it's been deprecated since February 2006. Added modified patch by Matt Palmer which adds a 'plugins' mount, fixing #891. See PluginsInModules on the wiki for information on usage. Empty dbserver and dbpassword settings will now be ignored when initializing Rails connections (patch by womble). Configuration settings can now be blank (patch by womble). Added calls to endpwent/endgrent when searching for user and group IDs, which fixes #791. Obviated 'target' in interfaces, as all file paths were automatically calculated anyway. The parameter is still there, but it's not used and just generates a warning. Fixing some of the problems with interface management on Red Hat. Puppet now uses the :netmask property and does not try to set the bootproto (#762). You now must specify an environment and you are required to specify the valid environments for your site. (#911) Certificates now always specify a subjectAltName, but it defaults to '*', meaning that it doesn't require DNS names to match. You can override that behaviour by specifying a value for 'certdnsnames', which will then require that hostname as a match (#896). Relationship metaparams (:notify, :require, :subscribe, and :before) now stack when they are collecting metaparam values from their containers (#446). For instance, if a resource inside a definition has a value set for 'require', and you call the definition with 'require', the resource gets both requires, where before it would only retain its initial value. Changed the behavior of --debug to include Mongrel client debugging information. Mongrel output will be written to the terminal only, not to the puppet debug log. This should help anyone working with reverse HTTP SSL proxies. (#905) Fixed #800 -- invalid configurations are no longer cached. This was done partially by adding a relationship validation step once the entire configuration is created, but it also required the previously-mentioned changes to how the configuration retrieval process works. Removed some functionality from the Master client, since the local functionality has been replaced with the Indirector already, and rearranging how configuration retrieval is done to fix ordering and caching bugs. The node scope is now above all other scopes besides the 'main' scope, which should help make its variables visible to other classes, assuming those classes were not included in the node's parent. Replaced GRATR::Digraph with Puppet::SimpleGraph as the base class for Puppet's graphing. Functionality should be equivalent but with dramatically better performance. The --use-nodes and --no-nodes options are now obsolete. Puppet automatically detects when nodes are defined, and if they are defined it will require that a node be found, else it will not look for a node nor will it fail if it fails to find one. Fixed #832. Added the '--no-daemonize' option to puppetd and puppetmasterd. NOTE: The default behavior of 'verbose' and 'debug' no longer cause puppetd and puppetmasterd to not daemonize. Added k5login type. (#759) Fixed CA race condition. (#693) Added shortname support to config.rb and refactored addargs 0.23.2 Fixed the problem in cron jobs where environment settings tended to multiple. (#749) Collection of resources now correctly only collects exported resources again. This was broken in 0.23.0. (#731) 'gen_config' now generates a configuration with all parameters under a heading that matches the process name, rather than keeping section headings. Refactored how the parser and interpreter relate, so parsing is now effectively an atomic process (thus fixing #314 and #729). This makes the interpreter less prone to error and less prone to show the error to the clients. Note that this means that if a configuration fails to parse, then the previous, parseable configuration will be used instead, so the client will not know that the configuration failed to parse. Added support for managing interfaces, thanks to work by Paul Rose. Fixed #652, thanks to a patch by emerose; --fqdn again works with puppetd. Added an extra check to the Mongrel support so that Apache can be used with optional cert checking, instead of mandatory, thus allowing Mongrel to function as the CA. This is thanks to work done by Marcin Owsiany. 0.23.1 (beaker) You can now specify relationships to classes, which work exactly like relationships to defined types: require => Class[myclass] This works with qualified classes, too. You can now do simple queries in a collection of exported resources. You still cannot do multi-condition queries, though. (#703) puppetca now exits with a non-zero code if it cannot find any host certificates to clean. (Patch by Dean Wilson.) Fully-qualified resources can now have defaults. (#589) Resource references can now be fully-qualified names, meaning you can list definitions with a namespace as dependencies. (#468) Files modified using a FileType instance, as ParsedFile does, will now automatically get backed up to the filebucket named "puppet". Added a 'maillist' type for managing mailing lists. Added a 'mailalias' type for managing mail aliases. Added patch by Valentin Vidic that adds the '+>' syntax to resources, so parameter values can be added to. The configuration client now pulls libraries down to $libdir, and all autoloading is done from there with full support for any reloadable file, such as types and providers. (#621) Note that this is not backward compatible -- if you're using pluginsync right now, you'll need to disable it on your clients until you can upgrade them. The Rails log level can now be set via (shockingly!) the 'rails_loglevel' parameter (#710). Note that this isn't exactly the feature asked for, but I could not find a way to directly copy ActiveRecord's concept of an environment. External node sources can now return undefined classes (#687). Puppet clients now have http proxy support (#701). The parser now throws an error when a resource reference is created for an unknown type. Also, resource references look up defined types and translate their type accordingly. (#706) Hostnames can now be double quoted. Adding module autoloading (#596) -- you can now 'include' classes from modules without ever needing to specifically load them. Class names and node names now conflict (#620). 0.23.0 Modified the fileserver to cache file information, so that each file isn't being read on every connection. Also, added londo's patch from #678 to avoid reading entire files into memory. Fixed environment handling in the crontab provider (#669). Added patch by trombik in #572, supporting old-style freebsd init scripts with '.sh' endings. Added fink package provider (#642), as provided by 'do'. Marked the dpkg package provider as versionable (#647). Applied patches by trombik to fix FreeBSD ports (#624 and #628). Fixed the CA server so that it refuses to send back a certificate whose public key doesn't match the CSR. Instead, it tells the user to run 'puppetca --clean'. Invalid certificates are no longer written to disk (#578). Added a package provider (appdmg) able to install .app packages on .dmg files on OS X (#641). Applied the patch from #667 to hopefully kill the client hanging problems (permanently, this time). Fixed functions so that they accept most other rvalues as valid values (#548). COMPATIBILITY ALERT: Significantly reworked external node support, in a way that's NOT backward-compatible: Only ONE node source can be used -- you can use LDAP, code, or an external node program, but not more than one. LDAP node support has two changes: First, the "ldapattrs" attribute is now used for setting the attributes to retrieve from the server (in addition to required attriutes), and second, all retrieved attributes are set as variables in the top scope. This means you can set attributes on your LDAP nodes and they will automatically appear as variables in your configurations. External node support has been completely rewritten. These programs must now generate a YAML dump of a hash, with "classes" and "parameters" keys. The classes should be an array, and the parameters should be a hash. The external node program has no support for parent nodes -- the script must handle that on its own. Reworked the database schema used to store configurations with the storeconfigs option. Replaced the obsolete RRD ruby library with the maintained RubyRRDtool library (which requires rrdtool2) (#659). The Portage package provider now calls eix-update automatically when eix's database is absent or out of sync (#666). Mounts now correctly handle existing fstabs with no pass or dump values (#550). Mounts now default to 0 for pass and dump (#112). Added urpmi support (#592). Finishing up the type => provider interface work. Basically, package providers now return lists of provider instances. In the proces, I rewrote the interface between package types and providers, and also enabled prefetching on all packages. This should significantly speed up most package operations. Hopefully fixing the file descriptor/open port problems, with patches from Valentin Vidic. Significantly reworked the type => provider interface with respect to listing existing provider instances. The class method on both class heirarchies has been renamed to 'instances', to start. Providers are now expected to return provider instances, instead of creating resources, and the resource's 'instances' method is expected to find the matching resource, if any, and set the resource's provider appropriately. This *significantly* reduces the reliance on effectively global state (resource references in the resource classes). This global state will go away soon. Along with this change, the 'prefetch' class method on providers now accepts the list of resources for prefetching. This again reduces reliance on global state, and makes the execution path much easier to follow. Fixed #532 -- reparsing config files now longer throws an exception. Added some warnings and logs to the service type so users will be encouraged to specify either "ensure" or "enabled" and added debugging to indicate why restarting is skipped when it is. Changed the location of the classes.txt to the state directory. Added better error reporting on unmatched brackets. Moved puppetd and puppetmasterd to sbin in svn and fixed install.rb to copy them into sbin on the local system appropriately. (#323) Added a splay option (#501). It's disabled when running under --test in puppetd. The value is random but cached. It defaults to the runinterval but can be tuned with --splaylimit Changing the notify type so that it always uses the loglevel. Fixing #568 - nodes can inherit from quoted node names. Tags (and thus definitions and classes) can now be a single character. (#566) Added an 'undef' keyword (#629), which will evaluate to "" within strings but when used as a resource parameter value will cause that parameter to be evaluated as undefined. Changed the topological sort algorithm (#507) so it will always fail on cycles. Added a 'dynamicfacts' configuration option; any facts in that comma-separated list will be ignored when comparing facts to see if they have changed and thus whether a recompile is necessary. Renamed some poorly named internal variables: @models in providers are now either @resource or @resource_type (#605). @children is no longer used except by components (#606). @parent is now @resource within parameters (#607). The old variables are still set for backward compatibility. Significantly reworking configuration parsing. Executables all now look for 'puppet.conf' (#206), although they will parse the old-style configuration files if they are present, although they throw a deprecation warning. Also, file parameters (owner, mode, group) are now set on the same line as the parameter, in brackets. (#422) Added transaction summaries (available with the --summarize option), useful for getting a quick idea of what happened in a transaction. Currently only useful on the client or with the puppet interpreter. Changed the interal workings for retrieve and removed the :is attribute from Property. The retrieve methods now return the current value of the property for the system. Removed acts_as_taggable from the rails models. 0.22.4 Execs now autorequire the user they run as, as long as the user is specified by name. (#430) Files on the local machine but not on the remote server during a source copy are now purged if purge => true. (#594) Providers can now specify that some commands are optional (#585). Also, the 'command' method returns nil on missing commands, rather than throwing an error, so the presence of commands be tested. The 'useradd' provider for Users can now manage passwords. No other providers can, at this point. Parameters can now declare a dependency on specific features, and parameters that require missing features will not be instantiated. This is most useful for properties. FileParsing classes can now use instance_eval to add many methods at once to a record type. Modules no longer return directories in the list of found manifests (#588). The crontab provider now defaults to root when there is no USER set in the environment. Puppetd once again correctly responds to HUP. Added a syntax for referring to variables defined in other classes (e.g., $puppet::server). STDIN, STDOUT, STDERR are now redirected to /dev/null in service providers descending from base. Certificates are now valid starting one day before they are created, to help handle small amounts of clock skew. Files are no longer considered out of sync if some properties are out of sync but they have no properties that can create the file. 0.22.3 Fixed backward compatibility for logs and metrics from older clients. Fixed the location of the authconfig parameters so there aren't loading order issues. Enabling attribute validation on the providers that subclass 'nameservice', so we can verify that an integer is passed to UID and GID. Added a stand-alone filebucket client, named 'filebucket'. Fixed the new nested paths for filebuckets; the entire md5 sum was not being stored. Fixing #553; -M is no longer added when home directories are being managed on Red Hat. 0.22.2 (grover) Users can now manage their home directories, using the managehome parameter, partially using patches provided by Tim Stoop and Matt Palmer. (#432) Added 'ralsh' (formerly x2puppet) to the svn tree. When possible it should be added to the packages. The 'notify' type now defaults to its message being the same as its name. Reopening $stdin to read from /dev/null during execution, in hopes that init scripts will stop hanging. Changed the 'servername' fact set on the server to use the server's fqdn, instead of the short-name. Changing the location of the configuration cache. It now defaults to being in the state directory, rather than in the configuration directory. All parameter instances are stored in a single @parameters instance variable hash within resource type instances. We used to use separate hashes for each parameter type. Added the concept of provider features. Eventually these should be able to express the full range of provider functionality, but for now they can test a provider to see what methods it has set and determine what features it provides as a result. These features are integrated into the doc generation system so that you get feature documentation automatically. Switched apt/aptitide to using "apt-cache policy" instead of "apt-cache showpkg" for determining the latest available version. (#487) FileBuckets now use a deeply nested structure for storing files, so you do not end up with hundreds or thousands of files in the same directory. (#447) Facts are now cached in the state file, and when they change the configuration is always recompiled. (#519) Added 'ignoreimport' setting for use in commit hooks. This causes the parser to ignore import statements so a single file can be parse-checked. (#544) Import statements can now specify multiple comma-separated arguments. Definitions now support both 'name' and 'title', just like any other resource type. (#539) Added a generate() command, which sets values to the result of an external command. (#541) Added a file() command to read in files with no interpolation. The first found file has its content returned. puppetd now exits if no cert is present in onetime mode. (#533) The client configuration cache can be safely removed and the client will correctly realize the client is not in sync. Resources can now be freely deleted, thus fixing many problems introduced when deletion of required resources was forbidden when purging was introduced. Only resources being purged will not be deleted. Facts and plugins now download even in noop mode (#540). Resources in noop mode now log when they would have responded to an event (#542). Refactored cron support entirely. Cron now uses providers, and there is a single 'crontab' provider that handles user crontabs. While this refactor does not include providers for /etc/crontab or cron.d, it should now be straightforward to write those providers. Changed the parameter sorting so that the provider parameter comes right after name, so the provider is available when the other parameters and properties are being created. Redid some of the internals of the ParsedFile provider base class. It now passes a FileRecord around instead of a hash. Fixing a bug related to link recursion that caused link directories to always be considered out of sync. The bind address for puppetmasterd can now be specified with --bindaddress. Added (probably experimental) mongrel support. At this point you're still responsible for starting each individual process, and you have to set up a proxy in front of it. Redesigned the 'network' tree to support multiple web servers, including refactoring most of the structural code so it's much clearer and more reusable now. Set up the CA client to default to ca_server and ca_port, so you can easily run a separate CA. Supporting hosts with no domain name, thanks to a patch from Dennis Jacobfeuerborn. Added an 'ignorecache' option to tell puppetd to force a recompile, thanks to a patch by Chris McEniry. Made up2date the default for RHEL < 4 and yum the default for the rest. The yum provider now supports versions. Case statements correctly match when multiple values are provided, thanks to a patch by David Schmitt. Functions can now be called with no arguments. String escapes parse correctly in all cases now, thanks to a patch by cstorey. Subclasses again search parent classes for defaults. You can now purge apt and dpkg packages. When doing file recursion, 'ensure' only affects the top-level directory. States have been renamed to Properties. 0.22.1 (kermit) -- Mostly a bugfix release Compile times now persist between restarts of puppetd. Timeouts have been added to many parts of Puppet, reducing the likelihood if it hanging forever on broken scripts or servers. All of the documentation and recipes have been moved to the wiki by Peter Abrahamsen and Ben Kite has moved the FAQ to the wiki. Explicit relationships now override automatic relationships, allowing you to manually specify deletion order when removing resources. Resources with dependencies can now be deleted as long as all of their dependencies are also being deleted. Namespaces for both classes and definitions now work much more consistently. You should now be able to specify a class or definition with a namespace everywhere you would normally expect to be able to specify one without. Downcasing of facts can be selectively disabled. Cyclic dependency graphs are now checked for and forbidden. The netinfo mounts provider was commented out, because it really doesn't work at all. Stupid NetInfo stores mount information with the device as the key, which doesn't work with my current NetInfo code. Otherwise, lots and lots of bugfixes. Check the tickets associated with the 'kermit' milestone. 0.22.0 Integrated the GRATR graph library into Puppet, for handling resource relationships. Lots of bug-fixes (see bugs tickets associated with the 'minor' milestone). Added new 'resources' metatype, which currently only includes the ability to purge unmanaged resources. Added better ability to generate new resource objects during transactions (using 'generate' and 'eval_generate' methods). Rewrote all Rails support with a much better database design. Export/collect now works, although the database is incompatible with previous versions. Removed downcasing of facts and made most of the language case-insensitive. Added support for printing the graphs built during transactions. Reworked how paths are built for logging. Switched all providers to directly executing commands instead of going through a subshell, which removes the need to quote or escape arguments. 0.20.1 Mostly a bug-fix release, with the most important fix being the multiple-definition error. Completely rewrote the ParsedFile system; each provider is now much shorter and much more maintainable. However, fundamental problems were found with the 'port' type, so it was disabled. Also, added a NetInfo provider for 'host' and an experimental NetInfo provider for 'mount'. Made the RRDGraph report *much* better and added reference generation for reports and functions. 0.20.0 Significantly refactored the parser. Resource overrides now consistently work anywhere in a class hierarchy. The language was also modified somewhat. The previous export/collect syntax is now used for handling virtual objects, and export/collect (which is still experimental) now uses double sigils (@@ and <<| |>>). Resource references (e.g., File["/etc/passwd"]) now have to be capitalized, in fitting in with capitalizing type operations. As usual, lots of other smaller fixes, but most of the work was in the language. 0.19.3 Fixing a bug in server/master.rb that causes the hostname not to be available in locally-executed manifests. 0.19.2 Fixing a few smaller bugs, notably in the reports system. Refreshed objects now generate an event, which can result in further refreshes of other objects. 0.19.1 Fixing two critical bugs: User management works again and cron jobs are no longer added to all user accounts. 0.19.0 Added provider support. Added support for %h, %H, and %d expansion in fileserver.conf. Added Certificate Revocation support. Made dynamic loading pervasive -- nearly every aspect of Puppet will now automatically load new instances (e.g., types, providers, and reports). Added support for automatic distribution of facts and plugins (custom types). 0.18.4 Another bug-fix release. The most import bug fixed is that cronjobs again work even with initially empty crontabs. 0.18.3 Mostly a bug-fix release; fixed small bugs in the functionality added in 0.18.2. 0.18.2 Added templating support. Added reporting. Added gem and blastwave packaging support. 0.18.1 Added signal handlers for HUP, so both client and server deal correctly with it. Added signal handler for USR1, which triggers a run on the client. As usual, fixed many bugs. Significant fixes to puppetrun -- it should behave much more correctly now. Added "fail" function which throws a syntax error if it's encountered. Added plugin downloading from the central server to the client. It must be enabled with --pluginsync. Added support for FreeBSD's special "@daily" cron schedules. Correctly handling spaces in file sources. Moved documentation into svn tree. 0.18.0 Added support for a "default" node. When multiple nodes are specified, they must now be comma-separated (this introduces a language incompatibility). Failed dependencies cause dependent objects within the same transaction not to run. Many updates to puppetrun Many bug fixes Function names are no longer reserved words. Links can now replace files. 0.17.2 Added "puppetrun" application and associated runner server and client classes. Fixed cron support so it better supports valid values and environment settings. 0.17.1 Fixing a bug requiring rails on all Debian boxes Fixing a couple of other small bugs 0.17.0 Adding ActiveRecord integration on the server Adding export/collect functionality Fixing many bugs 0.16.5 Fixing a critical bug in importing classes from other files Fixing nodename handling to actually allow dashes 0.16.4 Fixing a critical bug in puppetd when acquiring a certificate for the first time 0.16.3 Some significant bug fixes Modified puppetd so that it can now function as an agent independent of a puppetmasterd process, e.g., using the PuppetShow web application. 0.16.2 Modified some of the AST classes so that class names, definition names, and node names are all set within the code being evaluated, so 'tagged(name)' returns true while evaluating 'name', for instance. Added '--clean' argument to puppetca to remove all traces of a given client. 0.16.1 Added 'tagged' and 'defined' functions. Moved all functions to a general framework that makes it very easy to add new functions. 0.16.0 Added 'tag' keyword/function. Added FreeBSD Ports support Added 'pelement' server for sending or receiving Puppet objects, although none of the executables use it yet. 0.15.3 Fixed many bugs in :exec, including adding support for arrays of checks Added autoloading for types and service variants (e.g., you can now just create a new type in the appropriate location and use it in Puppet, without modifying the core Puppet libs). 0.15.2 Added darwinport, Apple .pkg, and freebsd package types Added 'mount type Host facts are now set at the top scope (Bug #103) Added -e (inline exection) flag to 'puppet' executable Many small bug fixes 0.15.1 Fixed 'yum' installs so that they successfully upgrade packages. Fixed puppetmasterd.conf file so group settings take. 0.15.0 Upped the minor release because the File server is incompatible with 0.14, because it now handles links. The 'symlink' type is deprecated (but still present), in favor of using files with the 'target' parameter. Unset variables no longer throw an error, they just return an empty string You can now specify tags to restrict which objects run during a given run. You can also specify to skip running against the cached copy when there's a failure, which is useful for testing new configurations. RPMs and Sun packages can now install, as long as they specify a package location, and they'll automatically upgrade if you point them to a new file with an upgrade. Multiple bug fixes. 0.14.1 Fixed a couple of small logging bugs Fixed a bug with handling group ownership of links 0.14.0 Added some ability to selectively manage symlinks when doing file management Many bug fixes Variables can now be used as the test values in case statements and selectors Bumping a minor release number because 0.13.4 introduced a protocol incompatibility and should have had a minor rev bump 0.13.6 Many, many small bug fixes FreeBSD user/group support has been added The configuration system has been rewritten so that daemons can now generate and repair the files and directories they need. (Fixed bug #68.) Fixed the element override issues; now only subclasses can override values. 0.13.5 Fixed packages so types can be specified Added 'enable' state to services, although it does not work everywhere yet 0.13.4 A few important bug fixes, mostly in the parser. 0.13.3 Changed transactions to be one-stage instead of two Changed all types to use self[:name] instead of self.name, to support the symbolic naming implemented in 0.13.1 0.13.2 Changed package[answerfile] to package[adminfile], and added package[responsefile] Fixed a bunch of internal functions to behave more consistently and usefully 0.13.1 Fixed RPM spec files to create puppet user and group (lutter) Fixed crontab reading and writing (luke) Added symbolic naming in the language (luke) 0.13.0 Added support for configuration files. Even more bug fixes, including the infamous 'frozen object' bug, which was a problem with 'waitforcert'. David Lutterkort got RPM into good shape. 0.12.0 Added Scheduling, and many bug fixes, of course. 0.11.2 Fixed bugs related to specifying arrays of requirements Fixed a key bug in retrieving checksums Fixed lots of usability bugs Added 'fail' methods that automatically add file and line info when possible, and converted many errors to use that method 0.11.1 Fixed bug with recursive copying with 'ignore' set. Added OpenBSD package support. 0.11.0 Added 'ensure' state to many elements. Modified puppetdoc to correctly handle indentation and such. Significantly rewrote much of the builtin documentation to take advantage of the new features in puppetdoc, including many examples. 0.10.2 Added SMF support Added autorequire functionality, with specific support for exec and file Exec elements autorequire any mentioned files, including the scripts, along with their CWDs. Files autorequire any parent directories. Added 'alias' metaparam. Fixed dependencies so they don't depend on file order. 0.10.1 Added Solaris package support and changed puppetmasterd to run as a non-root user. 0.10.0 Significant refactoring of how types, states, and parameters work, including breaking out parameters into a separate class. This refactoring did not introduce much new functionality, but made extension of Puppet significantly easier Also, fixed the bug with 'waitforcert' in puppetd. 0.9.4 Small fix to wrap the StatusServer class in the checks for required classes. 0.9.3 Fixed some significant bugs in cron job management. 0.9.2 Second Public Beta 0.9.0 First Public Beta diff --git a/lib/puppet/dsl.rb b/lib/puppet/dsl.rb index 4fbce556c..966feaf9b 100644 --- a/lib/puppet/dsl.rb +++ b/lib/puppet/dsl.rb @@ -1,277 +1,275 @@ # Just quick mess-around to see what a DSL would look like. # # This is what the executable could look like: ##!/usr/bin/ruby # #require 'puppet' #require 'puppet/dsl' # #Puppet::DSL.import(ARGV[0]) # #bucket = Puppet::TransBucket.new #bucket.type = "top" #bucket.keyword = "class" # #Puppet::DSL.find_all do |name, sub| # sub.included #end.each do |name, sub| # bucket.push sub.export #end # #puts bucket.to_manifest # # And here's what an example config could look like: # ##!/usr/bin/ruby # # # require 'puppet' # require 'puppet/dsl' # # include Puppet::DSL # init() # # aspect :webserver do # file "/tmp/testone", :content => "yaytest" # # exec "testing", :command => "/bin/echo this is a test" # end # # aspect :other, :inherits => :webserver do # file "/tmp/testone", :mode => "755" # end # # acquire :other # # apply -module Puppet - # Provide the actual commands for acting like a language. - module DSL - def aspect(name, options = {}, &block) - Puppet::Aspect.new(name, options, &block) - end +require 'puppet' - def acquire(*names) - names.each do |name| - if aspect = Puppet::Aspect[name] - unless aspect.evaluated? - aspect.evaluate - end - else - raise "Could not find aspect %s" % name +# Provide the actual commands for acting like a language. +module Puppet::DSL + def aspect(name, options = {}, &block) + Puppet::DSL::Aspect.new(name, options, &block) + end + + def acquire(*names) + names.each do |name| + if aspect = Puppet::DSL::Aspect[name] + unless aspect.evaluated? + aspect.evaluate end + else + raise "Could not find aspect %s" % name end end + end - def apply - bucket = export() - catalog = bucket.to_catalog - catalog.apply - end + def apply + bucket = export() + catalog = bucket.to_catalog + catalog.apply + end - def export - objects = Puppet::Aspect.collect do |name, aspect| - if aspect.evaluated? - aspect.export - end - end.reject { |a| a.nil? }.flatten.collect do |obj| - obj.to_trans + def export + objects = Puppet::DSL::Aspect.collect do |name, aspect| + if aspect.evaluated? + aspect.export end - bucket = Puppet::TransBucket.new(objects) - bucket.name = "top" - bucket.type = "class" - - return bucket + end.reject { |a| a.nil? }.flatten.collect do |obj| + obj.to_trans end + bucket = Puppet::TransBucket.new(objects) + bucket.name = "top" + bucket.type = "class" - def init - unless Process.uid == 0 - Puppet[:confdir] = File.expand_path("~/.puppet") - Puppet[:vardir] = File.expand_path("~/.puppet/var") - end - Puppet[:user] = Process.uid - Puppet[:group] = Process.gid - Puppet::Util::Log.newdestination(:console) - Puppet::Util::Log.level = :info - end + return bucket + end - private + def init + unless Process.uid == 0 + Puppet[:confdir] = File.expand_path("~/.puppet") + Puppet[:vardir] = File.expand_path("~/.puppet/var") + end + Puppet[:user] = Process.uid + Puppet[:group] = Process.gid + Puppet::Util::Log.newdestination(:console) + Puppet::Util::Log.level = :info end class Aspect Resource = Puppet::Parser::Resource include Puppet::Util include Puppet::DSL extend Puppet::Util extend Enumerable attr_accessor :parent, :name, :evaluated @aspects = {} @@objects = Hash.new do |hash, key| hash[key] = {} end # Create an instance method for every type Puppet::Type.loadall Puppet::Type.eachtype do |type| define_method(type.name) do |*args| newresource(type, *args) end end def self.[]=(name, aspect) name = symbolize(name) @aspects[name] = aspect end def self.[](name) name = symbolize(name) # Make sure there's always a main. This can get deleted in testing. if name == :main and ! @aspects[name] new(:main) {} end @aspects[name] end def self.clear @aspects.clear @@objects.clear end def self.delete(name) name = symbolize(name) if @aspects.has_key?(name) @aspects.delete(name) end end def self.each @aspects.each do |name, a| yield name, a end end def child_of?(aspect) unless aspect.is_a?(self.class) obj = self.class[aspect] unless obj raise "Could not find aspect %s" % aspect end aspect = obj end if self.parent if self.parent == aspect return true elsif self.parent.child_of?(aspect) return true else return false end else return false end end def evaluate if self.parent and ! self.parent.evaluated? self.parent.evaluate end unless evaluated? if defined? @block instance_eval(&@block) end @evaluated = true end end def evaluated? if self.evaluated true else false end end def export @resources.dup end def initialize(name, options = {}, &block) name = symbolize(name) @name = name if block @block = block end if pname = options[:inherits] if pname.is_a?(self.class) @parent = pname elsif parent = self.class[pname] @parent = parent else raise "Could not find parent aspect %s" % pname end end @resources = [] self.class[name] = self end def newresource(type, name, params = {}) - if self.is_a?(Puppet::Aspect) + if self.is_a?(Puppet::DSL::Aspect) source = self else - source = Puppet::Aspect[:main] + source = Puppet::DSL::Aspect[:main] end unless obj = @@objects[type][name] obj = Resource.new :title => name, :type => type.name, :source => source, :scope => scope @@objects[type][name] = obj @resources << obj end params.each do |name, value| param = Resource::Param.new( :name => name, :value => value, :source => source ) obj.send(:set_parameter, param) end obj end def scope unless defined?(@scope) # Set the code to something innocuous; we just need the # scopes, not the interpreter. Hackish, but true. Puppet[:code] = " " @interp = Puppet::Parser::Interpreter.new require 'puppet/node' @node = Puppet::Node.new(Facter.value(:hostname)) if env = Puppet[:environment] and env == "" env = nil end @node.parameters = Facter.to_hash - @compile = Puppet::Parser::Compile.new(@node, @interp.send(:parser, env)) + @compile = Puppet::Parser::Compiler.new(@node, @interp.send(:parser, env)) @scope = @compile.topscope end @scope end def type self.name end end end @aspects = {} diff --git a/lib/puppet/file_serving/file_base.rb b/lib/puppet/file_serving/file_base.rb index 7f169d1ea..06b3ad9ef 100644 --- a/lib/puppet/file_serving/file_base.rb +++ b/lib/puppet/file_serving/file_base.rb @@ -1,63 +1,75 @@ # # Created by Luke Kanies on 2007-10-22. # Copyright (c) 2007. All rights reserved. require 'puppet/file_serving' # The base class for Content and Metadata; provides common # functionality like the behaviour around links. class Puppet::FileServing::FileBase attr_accessor :key + # Does our file exist? + def exist? + begin + stat + return true + rescue => detail + return false + end + end + # Return the full path to our file. Fails if there's no path set. def full_path raise(ArgumentError, "You must set a path to get a file's path") unless self.path - relative_path ? File.join(path, relative_path) : path + if relative_path.nil? or relative_path == "" + path + else + File.join(path, relative_path) + end end def initialize(key, options = {}) - raise ArgumentError.new("Files must not be fully qualified") if path =~ /^#{::File::SEPARATOR}/ - @key = key @links = :manage options.each do |param, value| begin send param.to_s + "=", value rescue NoMethodError raise ArgumentError, "Invalid option %s for %s" % [param, self.class] end end end # Determine how we deal with links. attr_reader :links def links=(value) raise(ArgumentError, ":links can only be set to :manage or :follow") unless [:manage, :follow].include?(value) @links = value end # Set our base path. attr_reader :path def path=(path) raise ArgumentError.new("Paths must be fully qualified") unless path =~ /^#{::File::SEPARATOR}/ @path = path end # Set a relative path; this is used for recursion, and sets # the file's path relative to the initial recursion point. attr_reader :relative_path def relative_path=(path) raise ArgumentError.new("Relative paths must not be fully qualified") if path =~ /^#{::File::SEPARATOR}/ @relative_path = path end # Stat our file, using the appropriate link-sensitive method. def stat unless defined?(@stat_method) @stat_method = self.links == :manage ? :lstat : :stat end File.send(@stat_method, full_path()) end end diff --git a/lib/puppet/file_serving/metadata.rb b/lib/puppet/file_serving/metadata.rb index e26e75844..56712122c 100644 --- a/lib/puppet/file_serving/metadata.rb +++ b/lib/puppet/file_serving/metadata.rb @@ -1,69 +1,87 @@ # # Created by Luke Kanies on 2007-10-16. # Copyright (c) 2007. All rights reserved. require 'puppet' require 'puppet/indirector' require 'puppet/file_serving' require 'puppet/file_serving/file_base' require 'puppet/util/checksums' require 'puppet/file_serving/indirection_hooks' # A class that handles retrieving file metadata. class Puppet::FileServing::Metadata < Puppet::FileServing::FileBase module MetadataHelper include Puppet::FileServing::IndirectionHooks def post_find(instance) end def post_search(key, options = {}) end end include Puppet::Util::Checksums extend Puppet::Indirector indirects :file_metadata, :extend => Puppet::FileServing::IndirectionHooks attr_reader :path, :owner, :group, :mode, :checksum_type, :checksum, :ftype, :destination + PARAM_ORDER = [:mode, :ftype, :owner, :group] + + def attributes_with_tabs + desc = [] + PARAM_ORDER.each { |check| + check = :ftype if check == :type + desc << send(check) + } + + case ftype + when "file", "directory": desc << checksum + when "link": desc << @destination + else + raise ArgumentError, "Cannot manage files of type %s" % ftype + end + + return desc.join("\t") + end + def checksum_type=(type) raise(ArgumentError, "Unsupported checksum type %s" % type) unless respond_to?("%s_file" % type) @checksum_type = type end # Retrieve the attributes for this file, relative to a base directory. # Note that File.stat raises Errno::ENOENT if the file is absent and this # method does not catch that exception. def collect_attributes real_path = full_path() stat = stat() @owner = stat.uid @group = stat.gid @ftype = stat.ftype - # Set the octal mode, but as a string. - @mode = "%o" % (stat.mode & 007777) + # We have to mask the mode, yay. + @mode = stat.mode & 007777 - if stat.ftype == "symlink" + case stat.ftype + when "file": + @checksum = ("{%s}" % @checksum_type) + send("%s_file" % @checksum_type, real_path) + when "directory": # Always just timestamp the directory. + sumtype = @checksum_type.to_s =~ /time/ ? @checksum_type : "ctime" + @checksum = ("{%s}" % sumtype) + send("%s_file" % sumtype, path).to_s + when "link": @destination = File.readlink(real_path) else - @checksum = get_checksum(real_path) + raise ArgumentError, "Cannot manage files of type %s" % stat.ftype end end def initialize(*args) @checksum_type = "md5" super end - - private - - # Retrieve our checksum. - def get_checksum(path) - ("{%s}" % @checksum_type) + send("%s_file" % @checksum_type, path) - end end diff --git a/lib/puppet/metatype/evaluation.rb b/lib/puppet/metatype/evaluation.rb index b3b6570b2..08756e988 100644 --- a/lib/puppet/metatype/evaluation.rb +++ b/lib/puppet/metatype/evaluation.rb @@ -1,164 +1,149 @@ class Puppet::Type # This method is responsible for collecting property changes we always # descend into the children before we evaluate our current properties. # This returns any changes resulting from testing, thus 'collect' rather # than 'each'. def evaluate #Puppet.err "Evaluating %s" % self.path.join(":") unless defined? @evalcount self.err "No evalcount defined on '%s' of type '%s'" % [self.title,self.class] @evalcount = 0 end @evalcount += 1 if p = self.provider and p.respond_to?(:prefetch) p.prefetch end # this only operates on properties, not properties + children # it's important that we call retrieve() on the type instance, # not directly on the property, because it allows the type to override # the method, like pfile does currentvalues = self.retrieve changes = propertychanges(currentvalues).flatten # now record how many changes we've resulted in if changes.length > 0 self.debug "%s change(s)" % [changes.length] end # If we're in noop mode, we don't want to store the checked time, # because it will result in the resource not getting scheduled if # someone were to apply the catalog in non-noop mode. # We're going to go ahead and record that we checked if there were # no changes, since it's unlikely it will affect the scheduling. noop = noop? if ! noop or (noop && changes.length == 0) self.cache(:checked, Time.now) end return changes.flatten end # Flush the provider, if it supports it. This is called by the # transaction. def flush if self.provider and self.provider.respond_to?(:flush) self.provider.flush end end # if all contained objects are in sync, then we're in sync # FIXME I don't think this is used on the type instances any more, # it's really only used for testing def insync?(is) insync = true if property = @parameters[:ensure] unless is.include? property raise Puppet::DevError, "The is value is not in the is array for '%s'" % [property.name] end ensureis = is[property] if property.insync?(ensureis) and property.should == :absent return true end end properties.each { |property| unless is.include? property raise Puppet::DevError, "The is value is not in the is array for '%s'" % [property.name] end propis = is[property] unless property.insync?(propis) property.debug("Not in sync: %s vs %s" % [propis.inspect, property.should.inspect]) insync = false #else # property.debug("In sync") end } #self.debug("%s sync status is %s" % [self,insync]) return insync end # retrieve the current value of all contained properties def retrieve return currentpropvalues end # get a hash of the current properties. def currentpropvalues(override_value = nil) # it's important to use the method here, as it follows the order # in which they're defined in the object return properties().inject({}) { | prophash, property| prophash[property] = override_value.nil? ? property.retrieve : override_value prophash } end # Are we running in noop mode? def noop? @noop || Puppet[:noop] end def noop noop? end # Retrieve the changes associated with all of the properties. def propertychanges(currentvalues) # If we are changing the existence of the object, then none of # the other properties matter. changes = [] ensureparam = @parameters[:ensure] if @parameters.include?(:ensure) && !currentvalues.include?(ensureparam) raise Puppet::DevError, "Parameter ensure defined but missing from current values" end if @parameters.include?(:ensure) and ! ensureparam.insync?(currentvalues[ensureparam]) -# self.info "ensuring %s from %s" % -# [@parameters[:ensure].should, @parameters[:ensure].is] changes << Puppet::PropertyChange.new(ensureparam, currentvalues[ensureparam]) # Else, if the 'ensure' property is correctly absent, then do # nothing elsif @parameters.include?(:ensure) and currentvalues[ensureparam] == :absent - # self.info "Object is correctly absent" return [] else -# if @parameters.include?(:ensure) -# self.info "ensure: Is: %s, Should: %s" % -# [@parameters[:ensure].is, @parameters[:ensure].should] -# else -# self.info "no ensure property" -# end changes = properties().find_all { |property| - unless currentvalues.include?(property) - raise Puppet::DevError, "Property %s does not have a current value", - [property.name] - end + currentvalues[property] ||= :absent ! property.insync?(currentvalues[property]) }.collect { |property| Puppet::PropertyChange.new(property, currentvalues[property]) } end if Puppet[:debug] and changes.length > 0 - self.debug("Changing " + changes.collect { |ch| - ch.property.name - }.join(",") - ) + self.debug("Changing " + changes.collect { |ch| ch.property.name }.join(",")) end changes end end diff --git a/lib/puppet/network/client.rb b/lib/puppet/network/client.rb index 283436e95..0a0a72345 100644 --- a/lib/puppet/network/client.rb +++ b/lib/puppet/network/client.rb @@ -1,192 +1,189 @@ # the available clients require 'puppet' require 'puppet/daemon' require 'puppet/network/xmlrpc/client' require 'puppet/util/subclass_loader' require 'puppet/util/methodhelper' require 'puppet/sslcertificates/support' require 'net/http' # Some versions of ruby don't have this method defined, which basically causes # us to never use ssl. Yay. class Net::HTTP def use_ssl? if defined? @use_ssl @use_ssl else false end end # JJM: This is a "backport" of sorts to older ruby versions which # do not have this accessor. See #896 for more information. unless Net::HTTP.instance_methods.include? "enable_post_connection_check" attr_accessor :enable_post_connection_check end end # The base class for all of the clients. Many clients just directly # call methods, but some of them need to do some extra work or # provide a different interface. class Puppet::Network::Client Client = self include Puppet::Daemon include Puppet::Util extend Puppet::Util::SubclassLoader include Puppet::Util::MethodHelper # This handles reading in the key and such-like. include Puppet::SSLCertificates::Support attr_accessor :schedule, :lastrun, :local, :stopping attr_reader :driver # Set up subclass loading handle_subclasses :client, "puppet/network/client" # Determine what clients look for when being passed an object for local # client/server stuff. E.g., you could call Client::CA.new(:CA => ca). def self.drivername unless defined? @drivername @drivername = self.name end @drivername end # Figure out the handler for our client. def self.handler unless defined? @handler @handler = Puppet::Network::Handler.handler(self.name) end @handler end # The class that handles xmlrpc interaction for us. def self.xmlrpc_client unless defined? @xmlrpc_client @xmlrpc_client = Puppet::Network::XMLRPCClient.handler_class(self.handler) end @xmlrpc_client end # Create our client. def initialize(hash) # to whom do we connect? @server = nil if hash.include?(:Cache) @cache = hash[:Cache] else @cache = true end driverparam = self.class.drivername if hash.include?(:Server) args = {:Server => hash[:Server]} @server = hash[:Server] args[:Port] = hash[:Port] || Puppet[:masterport] @driver = self.class.xmlrpc_client.new(args) self.read_cert # We have to start the HTTP connection manually before we start # sending it requests or keep-alive won't work. - if @driver.respond_to? :start - @driver.start - end + @driver.start if @driver.respond_to? :start @local = false elsif hash.include?(driverparam) @driver = hash[driverparam] if @driver == true @driver = self.class.handler.new end @local = true else - raise Puppet::Network::ClientError, "%s must be passed a Server or %s" % - [self.class, driverparam] + raise Puppet::Network::ClientError, "%s must be passed a Server or %s" % [self.class, driverparam] end end # Are we a local client? def local? if defined? @local and @local true else false end end # Make sure we set the driver up when we read the cert in. def recycle_connection @driver.recycle_connection if @driver.respond_to?(:recycle_connection) end # A wrapper method to run and then store the last run time def runnow if self.stopping Puppet.notice "In shutdown progress; skipping run" return end begin self.run self.lastrun = Time.now.to_i rescue => detail puts detail.backtrace if Puppet[:trace] Puppet.err "Could not run %s: %s" % [self.class, detail] end end def run raise Puppet::DevError, "Client type %s did not override run" % self.class end def scheduled? if sched = self.schedule return sched.match?(self.lastrun) else return true end end def shutdown if self.stopping Puppet.notice "Already in shutdown" else self.stopping = true if self.respond_to? :running? and self.running? Puppet::Util::Storage.store end rmpidfile() end end # Start listening for events. We're pretty much just listening for # timer events here. def start # Create our timer. Puppet will handle observing it and such. timer = Puppet.newtimer( :interval => Puppet[:runinterval], :tolerance => 1, :start? => true ) do begin self.runnow if self.scheduled? rescue => detail puts detail.backtrace if Puppet[:trace] Puppet.err "Could not run client; got otherwise uncaught exception: %s" % detail end end # Run once before we start following the timer self.runnow end require 'puppet/network/client/proxy' end diff --git a/lib/puppet/network/handler/fileserver.rb b/lib/puppet/network/handler/fileserver.rb index e6378bf01..a9a95bcfe 100755 --- a/lib/puppet/network/handler/fileserver.rb +++ b/lib/puppet/network/handler/fileserver.rb @@ -1,828 +1,805 @@ require 'puppet' require 'puppet/network/authstore' require 'webrick/httpstatus' require 'cgi' require 'delegate' require 'sync' +require 'puppet/file_serving' +require 'puppet/file_serving/metadata' + class Puppet::Network::Handler AuthStoreError = Puppet::AuthStoreError class FileServerError < Puppet::Error; end class FileServer < Handler desc "The interface to Puppet's fileserving abilities." attr_accessor :local CHECKPARAMS = [:mode, :type, :owner, :group, :checksum] # Special filserver module for puppet's module system MODULES = "modules" PLUGINS = "plugins" @interface = XMLRPC::Service::Interface.new("fileserver") { |iface| iface.add_method("string describe(string, string)") iface.add_method("string list(string, string, boolean, array)") iface.add_method("string retrieve(string, string)") } def self.params CHECKPARAMS.dup end # If the configuration file exists, then create (if necessary) a LoadedFile # object to manage it; else, return nil. def configuration # Short-circuit the default case. return @configuration if defined?(@configuration) config_path = @passed_configuration_path || Puppet[:fileserverconfig] return nil unless FileTest.exist?(config_path) # The file exists but we don't have a LoadedFile instance for it. @configuration = Puppet::Util::LoadedFile.new(config_path) end # Create our default mounts for modules and plugins. This is duplicated code, # but I'm not really worried about that. def create_default_mounts @mounts = {} Puppet.debug "No file server configuration file; autocreating #{MODULES} mount with default permissions" mount = Mount.new(MODULES) mount.allow("*") @mounts[MODULES] = mount Puppet.debug "No file server configuration file; autocreating #{PLUGINS} mount with default permissions" mount = PluginMount.new(PLUGINS) mount.allow("*") @mounts[PLUGINS] = mount end # Describe a given file. This returns all of the manageable aspects # of that file. - def describe(url, links = :ignore, client = nil, clientip = nil) + def describe(url, links = :follow, client = nil, clientip = nil) links = links.intern if links.is_a? String - if links == :manage - raise Puppet::Network::Handler::FileServerError, "Cannot currently copy links" - end - mount, path = convert(url, client, clientip) - if client - mount.debug "Describing %s for %s" % [url, client] - end + mount.debug("Describing %s for %s" % [url, client]) if client + + # Remove any leading slashes, since Metadata doesn't like them, yo. + metadata = Puppet::FileServing::Metadata.new(url, :path => mount.path(client), :relative_path => path.sub(/^\//, ''), :links => links) - obj = nil - unless obj = mount.getfileobject(path, links, client) + return "" unless metadata.exist? + + begin + metadata.collect_attributes + rescue => detail + puts detail.backtrace if Puppet[:trace] + Puppet.err detail return "" end - currentvalues = mount.check(obj) - - desc = [] - CHECKPARAMS.each { |check| - if value = currentvalues[check] - desc << value - else - if check == "checksum" and currentvalues[:type] == "file" - mount.notice "File %s does not have data for %s" % - [obj.name, check] - end - desc << nil - end - } - - return desc.join("\t") + return metadata.attributes_with_tabs end # Create a new fileserving module. def initialize(hash = {}) @mounts = {} @files = {} if hash[:Local] @local = hash[:Local] else @local = false end if hash[:Config] == false @noreadconfig = true end @passed_configuration_path = hash[:Config] if hash.include?(:Mount) @passedconfig = true unless hash[:Mount].is_a?(Hash) raise Puppet::DevError, "Invalid mount hash %s" % hash[:Mount].inspect end hash[:Mount].each { |dir, name| if FileTest.exists?(dir) self.mount(dir, name) end } self.mount(nil, MODULES) self.mount(nil, PLUGINS) else @passedconfig = false if configuration readconfig(false) # don't check the file the first time. else create_default_mounts() end end end # List a specific directory's contents. def list(url, links = :ignore, recurse = false, ignore = false, client = nil, clientip = nil) mount, path = convert(url, client, clientip) - if client - mount.debug "Listing %s for %s" % [url, client] - end + mount.debug "Listing %s for %s" % [url, client] if client - obj = nil - unless mount.path_exists?(path, client) - return "" - end + return "" unless mount.path_exists?(path, client) desc = mount.list(path, recurse, ignore, client) if desc.length == 0 - mount.notice "Got no information on //%s/%s" % - [mount, path] + mount.notice "Got no information on //%s/%s" % [mount, path] return "" end - - desc.collect { |sub| - sub.join("\t") - }.join("\n") + + desc.collect { |sub| sub.join("\t") }.join("\n") end def local? self.local end # Is a given mount available? def mounted?(name) @mounts.include?(name) end # Mount a new directory with a name. def mount(path, name) if @mounts.include?(name) if @mounts[name] != path raise FileServerError, "%s is already mounted at %s" % [@mounts[name].path, name] else # it's already mounted; no problem return end end # Let the mounts do their own error-checking. @mounts[name] = Mount.new(name, path) @mounts[name].info "Mounted %s" % path return @mounts[name] end # Retrieve a file from the local disk and pass it to the remote # client. def retrieve(url, links = :ignore, client = nil, clientip = nil) links = links.intern if links.is_a? String mount, path = convert(url, client, clientip) if client mount.info "Sending %s to %s" % [url, client] end unless mount.path_exists?(path, client) mount.debug "#{mount} reported that #{path} does not exist" return "" end links = links.intern if links.is_a? String if links == :ignore and FileTest.symlink?(path) mount.debug "I think that #{path} is a symlink and we're ignoring them" return "" end - str = nil - if links == :manage - raise Puppet::Error, "Cannot copy links yet." - else - str = mount.read_file(path, client) - end + str = mount.read_file(path, client) if @local return str else return CGI.escape(str) end end def umount(name) @mounts.delete(name) if @mounts.include? name end private def authcheck(file, mount, client, clientip) # If we're local, don't bother passing in information. if local? client = nil clientip = nil end unless mount.allowed?(client, clientip) mount.warning "%s cannot access %s" % [client, file] raise Puppet::AuthorizationError, "Cannot access %s" % mount end end # Take a URL and some client info and return a mount and relative # path pair. # def convert(url, client, clientip) readconfig url = URI.unescape(url) mount, stub = splitpath(url, client) authcheck(url, mount, client, clientip) return mount, stub end # Return the mount for the Puppet modules; allows file copying from # the modules. def modules_mount(module_name, client) # Find our environment, if we have one. unless hostname = (client || Facter.value("hostname")) raise ArgumentError, "Could not find hostname" end if node = Puppet::Node.find(hostname) env = node.environment else env = nil end # And use the environment to look up the module. mod = Puppet::Module::find(module_name, env) if mod return @mounts[MODULES].copy(mod.name, mod.files) else return nil end end # Read the configuration file. def readconfig(check = true) return if @noreadconfig return unless configuration if check and ! @configuration.changed? return end newmounts = {} begin File.open(@configuration.file) { |f| mount = nil count = 1 f.each { |line| case line when /^\s*#/: next # skip comments when /^\s*$/: next # skip blank lines when /\[([-\w]+)\]/: name = $1 if newmounts.include?(name) raise FileServerError, "%s is already mounted at %s" % [newmounts[name], name], count, @configuration.file end mount = Mount.new(name) newmounts[name] = mount when /^\s*(\w+)\s+(.+)$/: var = $1 value = $2 case var when "path": if mount.name == MODULES Puppet.warning "The '#{mount.name}' module can not have a path. Ignoring attempt to set it" else begin mount.path = value rescue FileServerError => detail Puppet.err "Removing mount %s: %s" % [mount.name, detail] newmounts.delete(mount.name) end end when "allow": value.split(/\s*,\s*/).each { |val| begin mount.info "allowing %s access" % val mount.allow(val) rescue AuthStoreError => detail raise FileServerError.new(detail.to_s, count, @configuration.file) end } when "deny": value.split(/\s*,\s*/).each { |val| begin mount.info "denying %s access" % val mount.deny(val) rescue AuthStoreError => detail raise FileServerError.new(detail.to_s, count, @configuration.file) end } else raise FileServerError.new("Invalid argument '%s'" % var, count, @configuration.file) end else raise FileServerError.new("Invalid line '%s'" % line.chomp, count, @configuration.file) end count += 1 } } rescue Errno::EACCES => detail Puppet.err "FileServer error: Cannot read %s; cannot serve" % @configuration #raise Puppet::Error, "Cannot read %s" % @configuration rescue Errno::ENOENT => detail Puppet.err "FileServer error: '%s' does not exist; cannot serve" % @configuration end unless newmounts[MODULES] Puppet.debug "No #{MODULES} mount given; autocreating with default permissions" mount = Mount.new(MODULES) mount.allow("*") newmounts[MODULES] = mount end unless newmounts[PLUGINS] Puppet.debug "No #{PLUGINS} mount given; autocreating with default permissions" mount = PluginMount.new(PLUGINS) mount.allow("*") newmounts[PLUGINS] = mount end unless newmounts[PLUGINS].valid? Puppet.debug "No path given for #{PLUGINS} mount; creating a special PluginMount" # We end up here if the user has specified access rules for # the plugins mount, without specifying a path (which means # they want to have the default behaviour for the mount, but # special access control). So we need to move all the # user-specified access controls into the new PluginMount # object... mount = PluginMount.new(PLUGINS) # Yes, you're allowed to hate me for this. mount.instance_variable_set(:@declarations, newmounts[PLUGINS].instance_variable_get(:@declarations) ) newmounts[PLUGINS] = mount end # Verify each of the mounts are valid. # We let the check raise an error, so that it can raise an error # pointing to the specific problem. newmounts.each { |name, mount| unless mount.valid? raise FileServerError, "Invalid mount %s" % name end } @mounts = newmounts end # Split the path into the separate mount point and path. def splitpath(dir, client) # the dir is based on one of the mounts # so first retrieve the mount path mount = nil path = nil if dir =~ %r{/([-\w]+)} # Strip off the mount name. mount_name, path = dir.sub(%r{^/}, '').split(File::Separator, 2) unless mount = modules_mount(mount_name, client) unless mount = @mounts[mount_name] raise FileServerError, "Fileserver module '%s' not mounted" % mount_name end end else raise FileServerError, "Fileserver error: Invalid path '%s'" % dir end if path.nil? or path == '' path = '/' elsif path # Remove any double slashes that might have occurred path = URI.unescape(path.gsub(/\/\//, "/")) end return mount, path end def to_s "fileserver" end # A simple class for wrapping mount points. Instances of this class # don't know about the enclosing object; they're mainly just used for # authorization. class Mount < Puppet::Network::AuthStore attr_reader :name @@syncs = {} @@files = {} Puppet::Util.logmethods(self, true) def getfileobject(dir, links, client = nil) unless path_exists?(dir, client) self.debug "File source %s does not exist" % dir return nil end return fileobj(dir, links, client) end # Run 'retrieve' on a file. This gets the actual parameters, so # we can pass them to the client. def check(obj) # Retrieval is enough here, because we don't want to cache # any information in the state file, and we don't want to generate # any state changes or anything. We don't even need to sync # the checksum, because we're always going to hit the disk # directly. # We're now caching file data, using the LoadedFile to check the # disk no more frequently than the :filetimeout. path = obj[:path] sync = sync(path) unless data = @@files[path] data = {} sync.synchronize(Sync::EX) do @@files[path] = data data[:loaded_obj] = Puppet::Util::LoadedFile.new(path) data[:values] = properties(obj) return data[:values] end end changed = nil sync.synchronize(Sync::SH) do changed = data[:loaded_obj].changed? end if changed sync.synchronize(Sync::EX) do data[:values] = properties(obj) return data[:values] end else sync.synchronize(Sync::SH) do return data[:values] end end end # Create a map for a specific client. def clientmap(client) { "h" => client.sub(/\..*$/, ""), "H" => client, "d" => client.sub(/[^.]+\./, "") # domain name } end # Replace % patterns as appropriate. def expand(path, client = nil) # This map should probably be moved into a method. map = nil if client map = clientmap(client) else Puppet.notice "No client; expanding '%s' with local host" % path # Else, use the local information map = localmap() end path.gsub(/%(.)/) do |v| key = $1 if key == "%" "%" else map[key] || v end end end # Do we have any patterns in our path, yo? def expandable? if defined? @expandable @expandable else false end end # Return a fully qualified path, given a short path and # possibly a client name. def file_path(relative_path, node = nil) full_path = path(node) raise ArgumentError.new("Mounts without paths are not usable") unless full_path # If there's no relative path name, then we're serving the mount itself. return full_path unless relative_path and relative_path != "/" return File.join(full_path, relative_path) end # Create out object. It must have a name. def initialize(name, path = nil) unless name =~ %r{^[-\w]+$} raise FileServerError, "Invalid name format '%s'" % name end @name = name if path self.path = path else @path = nil end super() end def fileobj(path, links, client) obj = nil if obj = Puppet.type(:file)[file_path(path, client)] # This can only happen in local fileserving, but it's an # important one. It'd be nice if we didn't just set # the check params every time, but I'm not sure it's worth # the effort. obj[:check] = CHECKPARAMS else obj = Puppet.type(:file).create( :name => file_path(path, client), :check => CHECKPARAMS ) end if links == :manage links = :follow end # This, ah, might be completely redundant unless obj[:links] == links obj[:links] = links end return obj end # Read the contents of the file at the relative path given. def read_file(relpath, client) File.read(file_path(relpath, client)) end # Cache this manufactured map, since if it's used it's likely # to get used a lot. def localmap unless defined? @@localmap @@localmap = { "h" => Facter.value("hostname"), "H" => [Facter.value("hostname"), Facter.value("domain")].join("."), "d" => Facter.value("domain") } end @@localmap end # Return the path as appropriate, expanding as necessary. def path(client = nil) if expandable? return expand(@path, client) else return @path end end # Set the path. def path=(path) # FIXME: For now, just don't validate paths with replacement # patterns in them. if path =~ /%./ # Mark that we're expandable. @expandable = true else unless FileTest.exists?(path) raise FileServerError, "%s does not exist" % path end unless FileTest.directory?(path) raise FileServerError, "%s is not a directory" % path end unless FileTest.readable?(path) raise FileServerError, "%s is not readable" % path end @expandable = false end @path = path end # Verify that the path given exists within this mount's subtree. # def path_exists?(relpath, client = nil) File.exists?(file_path(relpath, client)) end # Return the current values for the object. def properties(obj) obj.retrieve.inject({}) { |props, ary| props[ary[0].name] = ary[1]; props } end # Retrieve a specific directory relative to a mount point. # If they pass in a client, then expand as necessary. def subdir(dir = nil, client = nil) basedir = self.path(client) dirname = if dir File.join(basedir, *dir.split("/")) else basedir end dirname end def sync(path) @@syncs[path] ||= Sync.new @@syncs[path] end def to_s "mount[%s]" % @name end # Verify our configuration is valid. This should really check to # make sure at least someone will be allowed, but, eh. def valid? if name == MODULES return @path.nil? else return ! @path.nil? end end # Return a new mount with the same properties as +self+, except # with a different name and path. def copy(name, path) result = self.clone result.path = path result.instance_variable_set(:@name, name) return result end # List the contents of the relative path +relpath+ of this mount. # # +recurse+ is the number of levels to recurse into the tree, # or false to provide no recursion or true if you just want to # go for broke. # # +ignore+ is an array of filenames to ignore when traversing # the list. # # The return value of this method is a complex nest of arrays, # which describes a directory tree. Each file or directory is # represented by an array, where the first element is the path # of the file (relative to the root of the mount), and the # second element is the type. A directory is represented by an # array as well, where the first element is a "directory" array, # while the remaining elements are other file or directory # arrays. Confusing? Hell yes. As an added bonus, all names # must start with a slash, because... well, I'm fairly certain # a complete explanation would involve the words "crack pipe" # and "bad batch". # def list(relpath, recurse, ignore, client = nil) reclist(file_path(relpath, client), nil, recurse, ignore) end # Recursively list the files in this tree. def reclist(basepath, abspath, recurse, ignore) abspath = basepath if abspath.nil? relpath = abspath.sub(%r{^#{basepath}}, '') relpath = "/#{relpath}" if relpath[0] != ?/ #/ return unless FileTest.exists?(abspath) desc = [relpath] ftype = File.stat(abspath).ftype desc << ftype if recurse.is_a?(Integer) recurse -= 1 end ary = [desc] if recurse == true or (recurse.is_a?(Integer) and recurse > -1) if ftype == "directory" children = Dir.entries(abspath) if ignore children = handleignore(children, abspath, ignore) end children.each { |child| next if child =~ /^\.\.?$/ reclist(basepath, File.join(abspath, child), recurse, ignore).each { |cobj| ary << cobj } } end end return ary.compact end # Deal with ignore parameters. def handleignore(files, path, ignore_patterns) ignore_patterns.each do |ignore| files.delete_if do |entry| File.fnmatch(ignore, entry, File::FNM_DOTMATCH) end end return files end end # A special mount class specifically for the plugins mount -- just # has some magic to effectively do a union mount of the 'plugins' # directory of all modules. # class PluginMount < Mount def path(client) '' end def mod_path_exists?(mod, relpath, client = nil) File.exists?(File.join(mod, PLUGINS, relpath)) end def path_exists?(relpath, client = nil) !valid_modules.find { |m| mod_path_exists?(m, relpath, client) }.nil? end def valid? true end def mod_file_path(mod, relpath, client = nil) File.join(mod, PLUGINS, relpath) end def file_path(relpath, client = nil) mod = valid_modules.map { |m| mod_path_exists?(m, relpath, client) ? m : nil }.compact.first mod_file_path(mod, relpath, client) end # create a list of files by merging all modules def list(relpath, recurse, ignore, client = nil) result = [] valid_modules.each do |m| ary = reclist(mod_file_path(m, relpath, client), nil, recurse, ignore) ary = [] if ary.nil? result += ary end result end private def valid_modules Puppet::Module.all.find_all { |m| File.directory?(File.join(m, PLUGINS)) } end def add_to_filetree(f, filetree) first, rest = f.split(File::SEPARATOR, 2) end end end end diff --git a/lib/puppet/network/http_server/webrick.rb b/lib/puppet/network/http_server/webrick.rb index e4f00dd73..568b4e798 100644 --- a/lib/puppet/network/http_server/webrick.rb +++ b/lib/puppet/network/http_server/webrick.rb @@ -1,168 +1,169 @@ require 'puppet' require 'puppet/daemon' require 'webrick' require 'webrick/https' require 'fcntl' require 'puppet/sslcertificates/support' require 'puppet/network/xmlrpc/webrick_servlet' require 'puppet/network/http_server' require 'puppet/network/client' +require 'puppet/network/handler' module Puppet class ServerError < RuntimeError; end module Network # The old-school, pure ruby webrick server, which is the default serving # mechanism. class HTTPServer::WEBrick < WEBrick::HTTPServer include Puppet::Daemon include Puppet::SSLCertificates::Support # Read the CA cert and CRL and populate an OpenSSL::X509::Store # with them, with flags appropriate for checking client # certificates for revocation def x509store if Puppet[:cacrl] == 'false' # No CRL, no store needed return nil end unless File.exist?(Puppet[:cacrl]) raise Puppet::Error, "Could not find CRL; set 'cacrl' to 'false' to disable CRL usage" end crl = OpenSSL::X509::CRL.new(File.read(Puppet[:cacrl])) store = OpenSSL::X509::Store.new store.purpose = OpenSSL::X509::PURPOSE_ANY store.flags = OpenSSL::X509::V_FLAG_CRL_CHECK_ALL|OpenSSL::X509::V_FLAG_CRL_CHECK unless self.ca_cert raise Puppet::Error, "Could not find CA certificate" end store.add_file(Puppet[:localcacert]) store.add_crl(crl) return store end # Set up the http log. def httplog args = [] # yuck; separate http logs file = nil Puppet.settings.use(:main, :ssl, Puppet[:name]) if Puppet[:name] == "puppetmasterd" file = Puppet[:masterhttplog] else file = Puppet[:httplog] end # open the log manually to prevent file descriptor leak file_io = open(file, "a+") file_io.sync file_io.fcntl(Fcntl::F_SETFD, Fcntl::FD_CLOEXEC) args << file_io if Puppet[:debug] args << WEBrick::Log::DEBUG end log = WEBrick::Log.new(*args) return log end # Create our server, yo. def initialize(hash = {}) Puppet.info "Starting server for Puppet version %s" % Puppet.version if handlers = hash[:Handlers] handler_instances = setup_handlers(handlers) else raise ServerError, "A server must have handlers" end unless self.read_cert if ca = handler_instances.find { |handler| handler.is_a?(Puppet::Network::Handler.ca) } request_cert(ca) else raise Puppet::Error, "No certificate and no CA; cannot get cert" end end setup_webrick(hash) begin super(hash) rescue => detail puts detail.backtrace if Puppet[:trace] raise Puppet::Error, "Could not start WEBrick: %s" % detail end # make sure children don't inherit the sockets listeners.each { |sock| sock.fcntl(Fcntl::F_SETFD, Fcntl::FD_CLOEXEC) } Puppet.info "Listening on port %s" % hash[:Port] # this creates a new servlet for every connection, # but all servlets have the same list of handlers # thus, the servlets can have their own state -- passing # around the requests and such -- but the handlers # have a global state # mount has to be called after the server is initialized servlet = Puppet::Network::XMLRPC::WEBrickServlet.new( handler_instances) self.mount("/RPC2", servlet) end # Create a ca client to set up our cert for us. def request_cert(ca) client = Puppet::Network::Client.ca.new(:CA => ca) unless client.request_cert raise Puppet::Error, "Could get certificate" end end # Create all of our handler instances. def setup_handlers(handlers) unless handlers.is_a?(Hash) raise ServerError, "Handlers must have arguments" end handlers.collect { |handler, args| hclass = nil - unless hclass = Handler.handler(handler) + unless hclass = Puppet::Network::Handler.handler(handler) raise ServerError, "Invalid handler %s" % handler end hclass.new(args) } end # Handle all of the many webrick arguments. def setup_webrick(hash) hash[:Port] ||= Puppet[:masterport] hash[:Logger] ||= self.httplog hash[:AccessLog] ||= [ [ self.httplog, WEBrick::AccessLog::COMMON_LOG_FORMAT ], [ self.httplog, WEBrick::AccessLog::REFERER_LOG_FORMAT ] ] hash[:SSLCertificateStore] = x509store hash[:SSLCertificate] = self.cert hash[:SSLPrivateKey] = self.key hash[:SSLStartImmediately] = true hash[:SSLEnable] = true hash[:SSLCACertificateFile] = Puppet[:localcacert] hash[:SSLVerifyClient] = OpenSSL::SSL::VERIFY_PEER hash[:SSLCertName] = nil if addr = Puppet[:bindaddress] and addr != "" hash[:BindAddress] = addr end end end end end diff --git a/lib/puppet/node/catalog.rb b/lib/puppet/node/catalog.rb index ee4cedd4b..d680de9a0 100644 --- a/lib/puppet/node/catalog.rb +++ b/lib/puppet/node/catalog.rb @@ -1,507 +1,512 @@ require 'puppet/indirector' +require 'puppet/pgraph' +require 'puppet/transaction' require 'puppet/util/tagging' # This class models a node catalog. It is the thing # meant to be passed from server to client, and it contains all # of the information in the catalog, including the resources # and the relationships between them. class Puppet::Node::Catalog < Puppet::PGraph extend Puppet::Indirector indirects :catalog, :terminus_class => :compiler include Puppet::Util::Tagging # The host name this is a catalog for. attr_accessor :name # The catalog version. Used for testing whether a catalog # is up to date. attr_accessor :version # How long this catalog took to retrieve. Used for reporting stats. attr_accessor :retrieval_duration # How we should extract the catalog for sending to the client. attr_reader :extraction_format # We need the ability to set this externally, so we can yaml-dump the # catalog. attr_accessor :edgelist_class # Whether this is a host catalog, which behaves very differently. # In particular, reports are sent, graphs are made, and state is # stored in the state database. If this is set incorrectly, then you often # end up in infinite loops, because catalogs are used to make things # that the host catalog needs. attr_accessor :host_config # Whether this graph is another catalog's relationship graph. # We don't want to accidentally create a relationship graph for another # relationship graph. attr_accessor :is_relationship_graph # Whether this catalog was retrieved from the cache, which affects # whether it is written back out again. attr_accessor :from_cache # Add classes to our class list. def add_class(*classes) classes.each do |klass| @classes << klass end # Add the class names as tags, too. tag(*classes) end # Add one or more resources to our graph and to our resource table. def add_resource(*resources) resources.each do |resource| unless resource.respond_to?(:ref) raise ArgumentError, "Can only add objects that respond to :ref" end fail_unless_unique(resource) ref = resource.ref @resource_table[ref] = resource # If the name and title differ, set up an alias - self.alias(resource, resource.name) if resource.respond_to?(:name) and resource.respond_to?(:title) and resource.name != resource.title + #self.alias(resource, resource.name) if resource.respond_to?(:name) and resource.respond_to?(:title) and resource.name != resource.title + if resource.respond_to?(:name) and resource.respond_to?(:title) and resource.name != resource.title + self.alias(resource, resource.name) if resource.class.isomorphic? + end resource.catalog = self if resource.respond_to?(:catalog=) and ! is_relationship_graph add_vertex(resource) end end # Create an alias for a resource. def alias(resource, name) resource.ref =~ /^(.+)\[/ newref = "%s[%s]" % [$1 || resource.class.name, name] if existing = @resource_table[newref] return if existing == resource raise(ArgumentError, "Cannot alias %s to %s; resource %s already exists" % [resource.ref, name, newref]) end @resource_table[newref] = resource @aliases[resource.ref] << newref end # Apply our catalog to the local host. Valid options # are: # :tags - set the tags that restrict what resources run # during the transaction # :ignoreschedules - tell the transaction to ignore schedules # when determining the resources to run def apply(options = {}) @applying = true Puppet::Util::Storage.load if host_config? transaction = Puppet::Transaction.new(self) transaction.tags = options[:tags] if options[:tags] transaction.ignoreschedules = true if options[:ignoreschedules] transaction.addtimes :config_retrieval => @retrieval_duration begin transaction.evaluate rescue Puppet::Error => detail Puppet.err "Could not apply complete catalog: %s" % detail rescue => detail puts detail.backtrace if Puppet[:trace] Puppet.err "Got an uncaught exception of type %s: %s" % [detail.class, detail] ensure # Don't try to store state unless we're a host config # too recursive. Puppet::Util::Storage.store if host_config? end yield transaction if block_given? transaction.send_report if host_config and (Puppet[:report] or Puppet[:summarize]) return transaction ensure @applying = false cleanup() transaction.cleanup if defined? transaction and transaction end # Are we in the middle of applying the catalog? def applying? @applying end def clear(remove_resources = true) super() # We have to do this so that the resources clean themselves up. @resource_table.values.each { |resource| resource.remove } if remove_resources @resource_table.clear if defined?(@relationship_graph) and @relationship_graph @relationship_graph.clear(false) @relationship_graph = nil end end def classes @classes.dup end # Create an implicit resource, meaning that it will lose out # to any explicitly defined resources. This method often returns # nil. # The quirk of this method is that it's not possible to create # an implicit resource before an explicit resource of the same name, # because all explicit resources are created before any generate() # methods are called on the individual resources. Thus, this # method can safely just check if an explicit resource already exists # and toss this implicit resource if so. def create_implicit_resource(type, options) unless options.include?(:implicit) options[:implicit] = true end # This will return nil if an equivalent explicit resource already exists. # When resource classes no longer retain references to resource instances, # this will need to be modified to catch that conflict and discard # implicit resources. if resource = create_resource(type, options) resource.implicit = true return resource else return nil end end # Create a new resource and register it in the catalog. def create_resource(type, options) unless klass = Puppet::Type.type(type) raise ArgumentError, "Unknown resource type %s" % type end return unless resource = klass.create(options) @transient_resources << resource if applying? add_resource(resource) if @relationship_graph @relationship_graph.add_resource(resource) unless @relationship_graph.resource(resource.ref) end resource end # Make sure we support the requested extraction format. def extraction_format=(value) unless respond_to?("extract_to_%s" % value) raise ArgumentError, "Invalid extraction format %s" % value end @extraction_format = value end # Turn our catalog graph into whatever the client is expecting. def extract send("extract_to_%s" % extraction_format) end # Create the traditional TransBuckets and TransObjects from our catalog # graph. This will hopefully be deprecated soon. def extract_to_transportable top = nil current = nil buckets = {} unless main = vertices.find { |res| res.type == "Class" and res.title == :main } raise Puppet::DevError, "Could not find 'main' class; cannot generate catalog" end # Create a proc for examining edges, which we'll use to build our tree # of TransBuckets and TransObjects. bucket = nil walk(main, :out) do |source, target| # The sources are always non-builtins. unless tmp = buckets[source.to_s] if tmp = buckets[source.to_s] = source.to_trans bucket = tmp else # This is because virtual resources return nil. If a virtual # container resource contains realized resources, we still need to get # to them. So, we keep a reference to the last valid bucket # we returned and use that if the container resource is virtual. end end bucket = tmp || bucket if child = target.to_trans unless bucket raise "No bucket created for %s" % source end bucket.push child # It's important that we keep a reference to any TransBuckets we've created, so # we don't create multiple buckets for children. unless target.builtin? buckets[target.to_s] = child end end end # Retrieve the bucket for the top-level scope and set the appropriate metadata. unless result = buckets[main.to_s] # This only happens when the catalog is entirely empty. result = buckets[main.to_s] = main.to_trans end result.classes = classes # Clear the cache to encourage the GC buckets.clear return result end # Make sure all of our resources are "finished". def finalize make_default_resources @resource_table.values.each { |resource| resource.finish } write_graph(:resources) end def host_config? host_config || false end def initialize(name = nil) super() @name = name if name @extraction_format ||= :transportable @classes = [] @resource_table = {} @transient_resources = [] @applying = false @relationship_graph = nil @aliases = Hash.new { |hash, key| hash[key] = [] } if block_given? yield(self) finalize() end end # Make the default objects necessary for function. def make_default_resources # We have to add the resources to the catalog, or else they won't get cleaned up after # the transaction. # First create the default scheduling objects Puppet::Type.type(:schedule).mkdefaultschedules.each { |res| add_resource(res) unless resource(res.ref) } # And filebuckets if bucket = Puppet::Type.type(:filebucket).mkdefaultbucket add_resource(bucket) end end # Create a graph of all of the relationships in our catalog. def relationship_graph raise(Puppet::DevError, "Tried get a relationship graph for a relationship graph") if self.is_relationship_graph unless defined? @relationship_graph and @relationship_graph # It's important that we assign the graph immediately, because # the debug messages below use the relationships in the # relationship graph to determine the path to the resources # spitting out the messages. If this is not set, # then we get into an infinite loop. @relationship_graph = Puppet::Node::Catalog.new @relationship_graph.host_config = host_config? @relationship_graph.is_relationship_graph = true # First create the dependency graph self.vertices.each do |vertex| @relationship_graph.add_vertex vertex vertex.builddepends.each do |edge| @relationship_graph.add_edge(edge) end end # Lastly, add in any autorequires @relationship_graph.vertices.each do |vertex| vertex.autorequire.each do |edge| unless @relationship_graph.edge?(edge.source, edge.target) # don't let automatic relationships conflict with manual ones. unless @relationship_graph.edge?(edge.target, edge.source) vertex.debug "Autorequiring %s" % [edge.source] @relationship_graph.add_edge(edge) else vertex.debug "Skipping automatic relationship with %s" % (edge.source == vertex ? edge.target : edge.source) end end end end @relationship_graph.write_graph(:relationships) # Then splice in the container information @relationship_graph.splice!(self, Puppet::Type::Component) @relationship_graph.write_graph(:expanded_relationships) end @relationship_graph end # Remove the resource from our catalog. Notice that we also call # 'remove' on the resource, at least until resource classes no longer maintain # references to the resource instances. def remove_resource(*resources) resources.each do |resource| @resource_table.delete(resource.ref) @aliases[resource.ref].each { |res_alias| @resource_table.delete(res_alias) } @aliases[resource.ref].clear remove_vertex!(resource) if vertex?(resource) @relationship_graph.remove_vertex!(resource) if @relationship_graph and @relationship_graph.vertex?(resource) resource.remove end end # Look a resource up by its reference (e.g., File[/etc/passwd]). def resource(type, title = nil) # Always create a resource reference, so that it always canonizes how we # are referring to them. if title ref = Puppet::ResourceReference.new(type, title).to_s else # If they didn't provide a title, then we expect the first # argument to be of the form 'Class[name]', which our # Reference class canonizes for us. ref = Puppet::ResourceReference.new(nil, type).to_s end if resource = @resource_table[ref] return resource elsif defined?(@relationship_graph) and @relationship_graph @relationship_graph.resource(ref) end end # Return an array of all resources. def resources @resource_table.keys end # Convert our catalog into a RAL catalog. def to_ral to_catalog :to_type end # Turn our parser catalog into a transportable catalog. def to_transportable to_catalog :to_transobject end # Produce the graph files if requested. def write_graph(name) # We only want to graph the main host catalog. return unless host_config? return unless Puppet[:graph] Puppet.settings.use(:graphing) file = File.join(Puppet[:graphdir], "%s.dot" % name.to_s) File.open(file, "w") { |f| f.puts to_dot("name" => name.to_s.capitalize) } end # LAK:NOTE We cannot yaml-dump the class in the edgelist_class, because classes cannot be # dumped by default, nor does yaml-dumping # the edge-labels work at this point (I don't # know why). # Neither of these matters right now, but I suppose it could at some point. # We also have to have the vertex_dict dumped after the resource table, because yaml can't # seem to handle the output of yaml-dumping the vertex_dict. def to_yaml_properties props = instance_variables.reject { |v| %w{@edgelist_class @edge_labels @vertex_dict}.include?(v) } props << "@vertex_dict" props end private def cleanup unless @transient_resources.empty? remove_resource(*@transient_resources) @transient_resources.clear @relationship_graph = nil end end # Verify that the given resource isn't defined elsewhere. def fail_unless_unique(resource) # Short-curcuit the common case, return unless existing_resource = @resource_table[resource.ref] # Either it's a defined type, which are never # isomorphic, or it's a non-isomorphic type, so # we should throw an exception. msg = "Duplicate definition: %s is already defined" % resource.ref if existing_resource.file and existing_resource.line msg << " in file %s at line %s" % [existing_resource.file, existing_resource.line] end if resource.line or resource.file msg << "; cannot redefine" end raise ArgumentError.new(msg) end # An abstracted method for converting one catalog into another type of catalog. # This pretty much just converts all of the resources from one class to another, using # a conversion method. def to_catalog(convert) result = self.class.new(self.name) map = {} vertices.each do |resource| next if resource.respond_to?(:virtual?) and resource.virtual? newres = resource.send(convert) # We can't guarantee that resources don't munge their names # (like files do with trailing slashes), so we have to keep track # of what a resource got converted to. map[resource.ref] = newres result.add_resource newres end message = convert.to_s.gsub "_", " " edges.each do |edge| # Skip edges between virtual resources. next if edge.source.respond_to?(:virtual?) and edge.source.virtual? next if edge.target.respond_to?(:virtual?) and edge.target.virtual? unless source = map[edge.source.ref] raise Puppet::DevError, "Could not find resource %s when converting %s resources" % [edge.source.ref, message] end unless target = map[edge.target.ref] raise Puppet::DevError, "Could not find resource %s when converting %s resources" % [edge.target.ref, message] end result.add_edge(source, target, edge.label) end map.clear - result.add_class *self.classes + result.add_class(*self.classes) result.tag(*self.tags) return result end end diff --git a/lib/puppet/provider/package/gem.rb b/lib/puppet/provider/package/gem.rb index f73694779..bb09bc5b0 100755 --- a/lib/puppet/provider/package/gem.rb +++ b/lib/puppet/provider/package/gem.rb @@ -1,103 +1,107 @@ require 'puppet/provider/package' # Ruby gems support. Puppet::Type.type(:package).provide :gem, :parent => Puppet::Provider::Package do desc "Ruby Gem support. By default uses remote gems, but you can specify the path to a local gem via ``source``." has_feature :versionable commands :gemcmd => "gem" def self.gemlist(hash) command = [command(:gemcmd), "list"] if hash[:local] command << "--local" else command << "--remote" end if name = hash[:justme] command << name end begin list = execute(command).split("\n").collect do |set| if gemhash = gemsplit(set) gemhash[:provider] = :gem gemhash else nil end end.compact rescue Puppet::ExecutionFailure => detail raise Puppet::Error, "Could not list gems: %s" % detail end if hash[:justme] return list.shift else return list end end def self.gemsplit(desc) case desc when /^\*\*\*/, /^\s*$/, /^\s+/; return nil when /^(\S+)\s+\((.+)\)/ name = $1 version = $2.split(/,\s*/)[0] return { :name => name, :ensure => version } else Puppet.warning "Could not match %s" % desc nil end end def self.instances(justme = false) gemlist(:local => true).collect do |hash| new(hash) end end def install(useversion = true) command = ["install"] if (! @resource.should(:ensure).is_a? Symbol) and useversion command << "-v" << @resource.should(:ensure) end # Always include dependencies command << "--include-dependencies" if source = @resource[:source] command << source else command << @resource[:name] end - gemcmd(*command) + output = gemcmd(*command) + # Apparently some stupid gem versions don't exit non-0 on failure + if output.include?("ERROR") + self.fail "Could not install: %s" % output.chomp + end end def latest # This always gets the latest version available. hash = self.class.gemlist(:justme => @resource[:name]) return hash[:ensure] end def query self.class.gemlist(:justme => @resource[:name], :local => true) end def uninstall gemcmd "uninstall", "-x", "-a", @resource[:name] end def update self.install(false) end end diff --git a/lib/puppet/type.rb b/lib/puppet/type.rb index c4e154d47..09003e8f5 100644 --- a/lib/puppet/type.rb +++ b/lib/puppet/type.rb @@ -1,431 +1,424 @@ require 'puppet' require 'puppet/util/log' require 'puppet/event' require 'puppet/util/metric' require 'puppet/property' require 'puppet/parameter' require 'puppet/util' require 'puppet/util/autoload' require 'puppet/metatype/manager' require 'puppet/util/errors' require 'puppet/util/log_paths' require 'puppet/util/logging' require 'puppet/resource_reference' # see the bottom of the file for the rest of the inclusions module Puppet class Type include Puppet::Util include Puppet::Util::Errors include Puppet::Util::LogPaths include Puppet::Util::Logging # Nearly all of the code in this class is stored in files in the # metatype/ directory. This is a temporary measure until I get a chance # to refactor this class entirely. There's still more simplification to # do, but this works for now. require 'puppet/metatype/attributes' require 'puppet/metatype/closure' require 'puppet/metatype/container' require 'puppet/metatype/evaluation' require 'puppet/metatype/instances' require 'puppet/metatype/metaparams' require 'puppet/metatype/providers' require 'puppet/metatype/relationships' require 'puppet/metatype/schedules' require 'puppet/metatype/tags' # Types (which map to resources in the languages) are entirely composed of # attribute value pairs. Generally, Puppet calls any of these things an # 'attribute', but these attributes always take one of three specific # forms: parameters, metaparams, or properties. # In naming methods, I have tried to consistently name the method so # that it is clear whether it operates on all attributes (thus has 'attr' in # the method name, or whether it operates on a specific type of attributes. attr_accessor :file, :line attr_writer :title attr_writer :noop include Enumerable # class methods dealing with Type management public # the Type class attribute accessors class << self attr_reader :name attr_accessor :self_refresh include Enumerable, Puppet::Util::ClassGen include Puppet::MetaType::Manager include Puppet::Util include Puppet::Util::Logging end # all of the variables that must be initialized for each subclass def self.initvars # all of the instances of this class @objects = Hash.new @aliases = Hash.new @providers = Hash.new @defaults = {} unless defined? @parameters @parameters = [] end @validproperties = {} @properties = [] @parameters = [] @paramhash = {} @attr_aliases = {} @paramdoc = Hash.new { |hash,key| if key.is_a?(String) key = key.intern end if hash.include?(key) hash[key] else "Param Documentation for %s not found" % key end } unless defined? @doc @doc = "" end end def self.to_s if defined? @name "Puppet::Type::" + @name.to_s.capitalize else super end end # Create a block to validate that our object is set up entirely. This will # be run before the object is operated on. def self.validate(&block) define_method(:validate, &block) #@validate = block end # The catalog that this resource is stored in. attr_accessor :catalog # create a log at specified level def log(msg) Puppet::Util::Log.create( :level => @parameters[:loglevel].value, :message => msg, :source => self ) end # instance methods related to instance intrinsics # e.g., initialize() and name() public def initvars @evalcount = 0 @tags = [] # callbacks are per object and event @callbacks = Hash.new { |chash, key| chash[key] = {} } # properties and parameters are treated equivalently from the outside: # as name-value pairs (using [] and []=) # internally, however, parameters are merely a hash, while properties # point to Property objects # further, the lists of valid properties and parameters are defined # at the class level unless defined? @parameters @parameters = {} end # set defalts @noop = false # keeping stats for the total number of changes, and how many were # completely sync'ed # this isn't really sufficient either, because it adds lots of special # cases such as failed changes # it also doesn't distinguish between changes from the current transaction # vs. changes over the process lifetime @totalchanges = 0 @syncedchanges = 0 @failedchanges = 0 @inited = true end # initialize the type instance def initialize(hash) unless defined? @inited self.initvars end namevar = self.class.namevar orighash = hash # If we got passed a transportable object, we just pull a bunch of info # directly from it. This is the main object instantiation mechanism. if hash.is_a?(Puppet::TransObject) # XXX This will need to change when transobjects change to titles. self.title = hash.name #self[:name] = hash[:name] [:file, :line, :tags, :catalog].each { |getter| if hash.respond_to?(getter) setter = getter.to_s + "=" if val = hash.send(getter) self.send(setter, val) end end } hash = hash.to_hash else if hash[:title] @title = hash[:title] hash.delete(:title) end end # Before anything else, set our parent if it was included if hash.include?(:parent) @parent = hash[:parent] hash.delete(:parent) end # Munge up the namevar stuff so we only have one value. hash = self.argclean(hash) # Let's do the name first, because some things need to happen once # we have the name but before anything else attrs = self.class.allattrs if hash.include?(namevar) #self.send(namevar.to_s + "=", hash[namevar]) self[namevar] = hash[namevar] hash.delete(namevar) if attrs.include?(namevar) attrs.delete(namevar) else self.devfail "My namevar isn't a valid attribute...?" end else self.devfail "I was not passed a namevar" end # If the name and title differ, set up an alias if self.name != self.title if obj = self.class[self.name] if self.class.isomorphic? raise Puppet::Error, "%s already exists with name %s" % [obj.title, self.name] end else self.class.alias(self.name, self) end end if hash.include?(:provider) self[:provider] = hash[:provider] hash.delete(:provider) else setdefaults(:provider) end # This is all of our attributes except the namevar. attrs.each { |attr| if hash.include?(attr) begin self[attr] = hash[attr] rescue ArgumentError, Puppet::Error, TypeError raise rescue => detail - error = Puppet::DevError.new( - "Could not set %s on %s: %s" % - [attr, self.class.name, detail] - ) + error = Puppet::DevError.new( "Could not set %s on %s: %s" % [attr, self.class.name, detail]) error.set_backtrace(detail.backtrace) raise error end hash.delete attr end } # Set all default values. self.setdefaults if hash.length > 0 self.debug hash.inspect self.fail("Class %s does not accept argument(s) %s" % [self.class.name, hash.keys.join(" ")]) end if self.respond_to?(:validate) self.validate end end # Set up all of our autorequires. def finish # Scheduling has to be done when the whole config is instantiated, so # that file order doesn't matter in finding them. self.schedule # Make sure all of our relationships are valid. Again, must be done # when the entire catalog is instantiated. self.class.relationship_params.collect do |klass| if param = @parameters[klass.name] param.validate_relationship end end.flatten.reject { |r| r.nil? } end # Return a cached value def cached(name) Puppet::Util::Storage.cache(self)[name] #@cache[name] ||= nil end # Cache a value def cache(name, value) Puppet::Util::Storage.cache(self)[name] = value #@cache[name] = value end # def set(name, value) # send(name.to_s + "=", value) # end # # def get(name) # send(name) # end # For now, leave the 'name' method functioning like it used to. Once 'title' # works everywhere, I'll switch it. def name return self[:name] end # Look up our parent in the catalog, if we have one. def parent return nil unless catalog unless defined?(@parent) # This is kinda weird. if implicit? parents = catalog.relationship_graph.adjacent(self, :direction => :in) else parents = catalog.adjacent(self, :direction => :in) end if parents # We should never have more than one parent, so let's just ignore # it if we happen to. @parent = parents.shift else @parent = nil end end @parent end # Return the "type[name]" style reference. def ref "%s[%s]" % [self.class.name.to_s.capitalize, self.title] end def self_refresh? self.class.self_refresh end # Mark that we're purging. def purging @purging = true end # Is this resource being purged? Used by transactions to forbid # deletion when there are dependencies. def purging? if defined? @purging @purging else false end end # Retrieve the title of an object. If no title was set separately, # then use the object's name. def title unless defined? @title and @title namevar = self.class.namevar if self.class.validparameter?(namevar) @title = self[:name] elsif self.class.validproperty?(namevar) @title = self.should(namevar) else self.devfail "Could not find namevar %s for %s" % [namevar, self.class.name] end end return @title end # convert to a string def to_s self.ref end # Convert to a transportable object def to_trans(ret = true) trans = TransObject.new(self.title, self.class.name) values = retrieve() values.each do |name, value| trans[name.name] = value end @parameters.each do |name, param| # Avoid adding each instance name as both the name and the namevar next if param.class.isnamevar? and param.value == self.title # We've already got property values next if param.is_a?(Puppet::Property) trans[name] = param.value end trans.tags = self.tags # FIXME I'm currently ignoring 'parent' and 'path' return trans end end # Puppet::Type end require 'puppet/propertychange' require 'puppet/provider' -require 'puppet/type/component' -require 'puppet/type/file' -require 'puppet/type/filebucket' -require 'puppet/type/tidy' - - +# Always load these types. +require 'puppet/type/component' diff --git a/lib/puppet/type/file.rb b/lib/puppet/type/file.rb index f093543d7..3518e8bb3 100644 --- a/lib/puppet/type/file.rb +++ b/lib/puppet/type/file.rb @@ -1,1156 +1,1151 @@ require 'digest/md5' require 'cgi' require 'etc' require 'uri' require 'fileutils' require 'puppet/network/handler' require 'puppet/util/diff' require 'puppet/util/checksums' module Puppet newtype(:file) do include Puppet::Util::MethodHelper include Puppet::Util::Checksums @doc = "Manages local files, including setting ownership and permissions, creation of both files and directories, and retrieving entire files from remote servers. As Puppet matures, it expected that the ``file`` resource will be used less and less to manage content, and instead native resources will be used to do so. If you find that you are often copying files in from a central location, rather than using native resources, please contact Reductive Labs and we can hopefully work with you to develop a native resource to support what you are doing." newparam(:path) do desc "The path to the file to manage. Must be fully qualified." isnamevar validate do |value| unless value =~ /^#{File::SEPARATOR}/ raise Puppet::Error, "File paths must be fully qualified" end end end newparam(:backup) do desc "Whether files should be backed up before being replaced. The preferred method of backing files up is via a ``filebucket``, which stores files by their MD5 sums and allows easy retrieval without littering directories with backups. You can specify a local filebucket or a network-accessible server-based filebucket by setting ``backup => bucket-name``. Alternatively, if you specify any value that begins with a ``.`` (e.g., ``.puppet-bak``), then Puppet will use copy the file in the same directory with that value as the extension of the backup. Setting ``backup => false`` disables all backups of the file in question. Puppet automatically creates a local filebucket named ``puppet`` and defaults to backing up there. To use a server-based filebucket, you must specify one in your configuration:: filebucket { main: server => puppet } The ``puppetmasterd`` daemon creates a filebucket by default, so you can usually back up to your main server with this configuration. Once you've described the bucket in your configuration, you can use it in any file:: file { \"/my/file\": source => \"/path/in/nfs/or/something\", backup => main } This will back the file up to the central server. At this point, the benefits of using a filebucket are that you do not have backup files lying around on each of your machines, a given version of a file is only backed up once, and you can restore any given file manually, no matter how old. Eventually, transactional support will be able to automatically restore filebucketed files. " defaultto { "puppet" } munge do |value| # I don't really know how this is happening. if value.is_a?(Array) value = value.shift end case value when false, "false", :false: false when true, "true", ".puppet-bak", :true: ".puppet-bak" when /^\./ value when String: # We can't depend on looking this up right now, # we have to do it after all of the objects # have been instantiated. if bucketobj = Puppet::Type.type(:filebucket)[value] @resource.bucket = bucketobj.bucket bucketobj.title else # Set it to the string; finish() turns it into a # filebucket. @resource.bucket = value value end when Puppet::Network::Client.client(:Dipper): @resource.bucket = value value.name else self.fail "Invalid backup type %s" % value.inspect end end end newparam(:recurse) do desc "Whether and how deeply to do recursive management." newvalues(:true, :false, :inf, /^[0-9]+$/) # Replace the validation so that we allow numbers in # addition to string representations of them. validate { |arg| } munge do |value| newval = super(value) case newval when :true, :inf: true when :false: false when Integer, Fixnum, Bignum: value when /^\d+$/: Integer(value) else raise ArgumentError, "Invalid recurse value %s" % value.inspect end end end newparam(:replace, :boolean => true) do desc "Whether or not to replace a file that is sourced but exists. This is useful for using file sources purely for initialization." newvalues(:true, :false) aliasvalue(:yes, :true) aliasvalue(:no, :false) defaultto :true end newparam(:force, :boolean => true) do desc "Force the file operation. Currently only used when replacing directories with links." newvalues(:true, :false) defaultto false end newparam(:ignore) do desc "A parameter which omits action on files matching specified patterns during recursion. Uses Ruby's builtin globbing engine, so shell metacharacters are fully supported, e.g. ``[a-z]*``. Matches that would descend into the directory structure are ignored, e.g., ``*/*``." defaultto false validate do |value| unless value.is_a?(Array) or value.is_a?(String) or value == false self.devfail "Ignore must be a string or an Array" end end end newparam(:links) do desc "How to handle links during file actions. During file copying, ``follow`` will copy the target file instead of the link, ``manage`` will copy the link itself, and ``ignore`` will just pass it by. When not copying, ``manage`` and ``ignore`` behave equivalently (because you cannot really ignore links entirely during local recursion), and ``follow`` will manage the file to which the link points." - newvalues(:follow, :manage, :ignore) + newvalues(:follow, :manage) - # :ignore and :manage behave equivalently on local files, - # but don't copy remote links - defaultto :ignore + defaultto :manage end newparam(:purge, :boolean => true) do desc "Whether unmanaged files should be purged. If you have a filebucket configured the purged files will be uploaded, but if you do not, this will destroy data. Only use this option for generated files unless you really know what you are doing. This option only makes sense when recursively managing directories. Note that when using ``purge`` with ``source``, Puppet will purge any files that are not on the remote system." defaultto :false newvalues(:true, :false) end newparam(:sourceselect) do desc "Whether to copy all valid sources, or just the first one. This parameter is only used in recursive copies; by default, the first valid source is the only one used as a recursive source, but if this parameter is set to ``all``, then all valid sources will have all of their contents copied to the local host, and for sources that have the same file, the source earlier in the list will be used." defaultto :first newvalues(:first, :all) end attr_accessor :bucket # Autorequire any parent directories. autorequire(:file) do if self[:path] File.dirname(self[:path]) else Puppet.err "no path for %s, somehow; cannot setup autorequires" % self.ref nil end end # Autorequire the owner and group of the file. {:user => :owner, :group => :group}.each do |type, property| autorequire(type) do if @parameters.include?(property) # The user/group property automatically converts to IDs next unless should = @parameters[property].shouldorig val = should[0] if val.is_a?(Integer) or val =~ /^\d+$/ nil else val end end end end CREATORS = [:content, :source, :target] validate do count = 0 CREATORS.each do |param| count += 1 if self.should(param) end if count > 1 self.fail "You cannot specify more than one of %s" % CREATORS.collect { |p| p.to_s}.join(", ") end end def self.[](path) return nil unless path super(path.gsub(/\/+/, '/').sub(/\/$/, '')) end # List files, but only one level deep. def self.instances(base = "/") unless FileTest.directory?(base) return [] end files = [] Dir.entries(base).reject { |e| e == "." or e == ".." }.each do |name| path = File.join(base, name) if obj = self[path] obj[:check] = :all files << obj else files << self.create( :name => path, :check => :all ) end end files end @depthfirst = false def argument?(arg) @arghash.include?(arg) end # Determine the user to write files as. def asuser if self.should(:owner) and ! self.should(:owner).is_a?(Symbol) writeable = Puppet::Util::SUIDManager.asuser(self.should(:owner)) { FileTest.writable?(File.dirname(self[:path])) } # If the parent directory is writeable, then we execute # as the user in question. Otherwise we'll rely on # the 'owner' property to do things. if writeable asuser = self.should(:owner) end end return asuser end # We have to do some extra finishing, to retrieve our bucket if # there is one. def finish # Let's cache these values, since there should really only be # a couple of these buckets @@filebuckets ||= {} # Look up our bucket, if there is one if bucket = self.bucket case bucket when String: if obj = @@filebuckets[bucket] # This sets the @value on :backup, too self.bucket = obj elsif bucket == "puppet" obj = Puppet::Network::Client.client(:Dipper).new( :Path => Puppet[:clientbucketdir] ) self.bucket = obj @@filebuckets[bucket] = obj elsif obj = Puppet::Type.type(:filebucket).bucket(bucket) @@filebuckets[bucket] = obj self.bucket = obj else self.fail "Could not find filebucket %s" % bucket end when Puppet::Network::Client.client(:Dipper): # things are hunky-dorey else self.fail "Invalid bucket type %s" % bucket.class end end super end # Create any children via recursion or whatever. def eval_generate recurse() end # Deal with backups. def handlebackup(file = nil) # let the path be specified file ||= self[:path] # if they specifically don't want a backup, then just say # we're good unless FileTest.exists?(file) return true end unless self[:backup] return true end case File.stat(file).ftype when "directory": if self[:recurse] # we don't need to backup directories when recurse is on return true else backup = self.bucket || self[:backup] case backup when Puppet::Network::Client.client(:Dipper): notice "Recursively backing up to filebucket" require 'find' Find.find(self[:path]) do |f| if File.file?(f) sum = backup.backup(f) self.info "Filebucketed %s to %s with sum %s" % [f, backup.name, sum] end end return true when String: newfile = file + backup # Just move it, since it's a directory. if FileTest.exists?(newfile) remove_backup(newfile) end begin bfile = file + backup # Ruby 1.8.1 requires the 'preserve' addition, but # later versions do not appear to require it. FileUtils.cp_r(file, bfile, :preserve => true) return true rescue => detail # since they said they want a backup, let's error out # if we couldn't make one self.fail "Could not back %s up: %s" % [file, detail.message] end else self.err "Invalid backup type %s" % backup.inspect return false end end when "file": backup = self.bucket || self[:backup] case backup when Puppet::Network::Client.client(:Dipper): sum = backup.backup(file) self.info "Filebucketed to %s with sum %s" % [backup.name, sum] return true when String: newfile = file + backup if FileTest.exists?(newfile) remove_backup(newfile) end begin # FIXME Shouldn't this just use a Puppet object with # 'source' specified? bfile = file + backup # Ruby 1.8.1 requires the 'preserve' addition, but # later versions do not appear to require it. FileUtils.cp(file, bfile, :preserve => true) return true rescue => detail # since they said they want a backup, let's error out # if we couldn't make one self.fail "Could not back %s up: %s" % [file, detail.message] end else self.err "Invalid backup type %s" % backup.inspect return false end when "link": return true else self.notice "Cannot backup files of type %s" % File.stat(file).ftype return false end end def handleignore(children) return children unless self[:ignore] self[:ignore].each { |ignore| ignored = [] Dir.glob(File.join(self[:path],ignore), File::FNM_DOTMATCH) { |match| ignored.push(File.basename(match)) } children = children - ignored } return children end def initialize(hash) # Store a copy of the arguments for later. tmphash = hash.to_hash # Used for caching clients @clients = {} super # Get rid of any duplicate slashes, and remove any trailing slashes. @title = @title.gsub(/\/+/, "/") @title.sub!(/\/$/, "") unless @title == "/" # Clean out as many references to any file paths as possible. # This was the source of many, many bugs. @arghash = tmphash @arghash.delete(self.class.namevar) [:source, :parent].each do |param| if @arghash.include?(param) @arghash.delete(param) end end @stat = nil end # Build a recursive map of a link source def linkrecurse(recurse) target = @parameters[:target].should method = :lstat if self[:links] == :follow method = :stat end targetstat = nil unless FileTest.exist?(target) return end # Now stat our target targetstat = File.send(method, target) unless targetstat.ftype == "directory" return end # Now that we know our corresponding target is a directory, # change our type self[:ensure] = :directory unless FileTest.readable? target self.notice "Cannot manage %s: permission denied" % self.name return end children = Dir.entries(target).reject { |d| d =~ /^\.+$/ } # Get rid of ignored children if @parameters.include?(:ignore) children = handleignore(children) end added = [] children.each do |file| Dir.chdir(target) do longname = File.join(target, file) # Files know to create directories when recursion # is enabled and we're making links args = { :recurse => recurse, :ensure => longname } if child = self.newchild(file, true, args) added << child end end end added end # Build up a recursive map of what's around right now def localrecurse(recurse) unless FileTest.exist?(self[:path]) and self.stat.directory? #self.info "%s is not a directory; not recursing" % # self[:path] return end unless FileTest.readable? self[:path] self.notice "Cannot manage %s: permission denied" % self.name return end children = Dir.entries(self[:path]) #Get rid of ignored children if @parameters.include?(:ignore) children = handleignore(children) end added = [] children.each { |file| file = File.basename(file) next if file =~ /^\.\.?$/ # skip . and .. options = {:recurse => recurse} if child = self.newchild(file, true, options) added << child end } added end # Create a new file or directory object as a child to the current # object. def newchild(path, local, hash = {}) raise(Puppet::DevError, "File recursion cannot happen without a catalog") unless catalog # make local copy of arguments args = symbolize_options(@arghash) # There's probably a better way to do this, but we don't want # to pass this info on. if v = args[:ensure] v = symbolize(v) args.delete(:ensure) end if path =~ %r{^#{File::SEPARATOR}} self.devfail( "Must pass relative paths to PFile#newchild()" ) else path = File.join(self[:path], path) end args[:path] = path unless hash.include?(:recurse) if args.include?(:recurse) if args[:recurse].is_a?(Integer) args[:recurse] -= 1 # reduce the level of recursion end end end hash.each { |key,value| args[key] = value } child = nil # The child might already exist because 'localrecurse' runs # before 'sourcerecurse'. I could push the override stuff into # a separate method or something, but the work is the same other # than this last bit, so it doesn't really make sense. if child = catalog.resource(:file, path) unless child.parent.object_id == self.object_id self.debug "Not managing more explicit file %s" % path return nil end # This is only necessary for sourcerecurse, because we might have # created the object with different 'should' values than are # set remotely. unless local args.each { |var,value| next if var == :path next if var == :name # behave idempotently unless child.should(var) == value child[var] = value end } end return nil else # create it anew #notice "Creating new file with args %s" % args.inspect args[:parent] = self begin # This method is used by subclasses of :file, so use the class name rather than hard-coding # :file. return nil unless child = catalog.create_implicit_resource(self.class.name, args) rescue => detail self.notice "Cannot manage: %s" % [detail] return nil end end # LAK:FIXME This shouldn't be necessary, but as long as we're # modeling the relationship graph specifically, it is. catalog.relationship_graph.add_edge self, child return child end # Files handle paths specially, because they just lengthen their # path names, rather than including the full parent's title each # time. def pathbuilder # We specifically need to call the method here, so it looks # up our parent in the catalog graph. if parent = parent() # We only need to behave specially when our parent is also # a file if parent.is_a?(self.class) # Remove the parent file name list = parent.pathbuilder list.pop # remove the parent's path info return list << self.ref else return super end else return [self.ref] end end # Should we be purging? def purge? @parameters.include?(:purge) and (self[:purge] == :true or self[:purge] == "true") end # Recurse into the directory. This basically just calls 'localrecurse' # and maybe 'sourcerecurse', returning the collection of generated # files. def recurse # are we at the end of the recursion? return unless self.recurse? recurse = self[:recurse] # we might have a string, rather than a number if recurse.is_a?(String) if recurse =~ /^[0-9]+$/ recurse = Integer(recurse) else # anything else is infinite recursion recurse = true end end if recurse.is_a?(Integer) recurse -= 1 end children = [] # We want to do link-recursing before normal recursion so that all # of the target stuff gets copied over correctly. if @parameters.include? :target and ret = self.linkrecurse(recurse) children += ret end if ret = self.localrecurse(recurse) children += ret end # These will be files pulled in by the file source sourced = false if @parameters.include?(:source) ret, sourced = self.sourcerecurse(recurse) if ret children += ret end end # The purge check needs to happen after all of the other recursion. if self.purge? children.each do |child| if (sourced and ! sourced.include?(child[:path])) or ! child.managed? child[:ensure] = :absent end end end children end # A simple method for determining whether we should be recursing. def recurse? return false unless @parameters.include?(:recurse) val = @parameters[:recurse].value if val and (val == true or val > 0) return true else return false end end # Remove the old backup. def remove_backup(newfile) if self.class.name == :file and self[:links] != :follow method = :lstat else method = :stat end old = File.send(method, newfile).ftype if old == "directory" raise Puppet::Error, "Will not remove directory backup %s; use a filebucket" % newfile end info "Removing old backup of type %s" % File.send(method, newfile).ftype begin File.unlink(newfile) rescue => detail puts detail.backtrace if Puppet[:trace] self.err "Could not remove old backup: %s" % detail return false end end # Remove any existing data. This is only used when dealing with # links or directories. def remove_existing(should) return unless s = stat(true) - unless handlebackup - self.fail "Could not back up; will not replace" - end + self.fail "Could not back up; will not replace" unless handlebackup unless should.to_s == "link" return if s.ftype.to_s == should.to_s end case s.ftype when "directory": if self[:force] == :true - debug "Removing existing directory for replacement with %s" % - should + debug "Removing existing directory for replacement with %s" % should FileUtils.rmtree(self[:path]) else notice "Not removing directory; use 'force' to override" end when "link", "file": debug "Removing existing %s for replacement with %s" % [s.ftype, should] File.unlink(self[:path]) else self.fail "Could not back up files of type %s" % s.ftype end end # a wrapper method to make sure the file exists before doing anything def retrieve unless stat = self.stat(true) self.debug "File does not exist" # If the file doesn't exist but we have a source, then call # retrieve on that property propertyvalues = properties().inject({}) { |hash, property| hash[property] = :absent hash } if @parameters.include?(:source) propertyvalues[:source] = @parameters[:source].retrieve end return propertyvalues end return currentpropvalues() end # This recurses against the remote source and makes sure the local # and remote structures match. It's run after 'localrecurse'. This # method only does anything when its corresponding remote entry is # a directory; in that case, this method creates file objects that # correspond to any contained remote files. def sourcerecurse(recurse) # we'll set this manually as necessary if @arghash.include?(:ensure) @arghash.delete(:ensure) end r = false if recurse unless recurse == 0 r = 1 end end ignore = self[:ignore] result = [] found = [] # Keep track of all the files we found in the source, so we can purge # appropriately. sourced = [] @parameters[:source].should.each do |source| sourceobj, path = uri2obj(source) # okay, we've got our source object; now we need to # build up a local file structure to match the remote # one server = sourceobj.server desc = server.list(path, self[:links], r, ignore) if desc == "" next end # Now create a new child for every file returned in the list. result += desc.split("\n").collect { |line| file, type = line.split("\t") next if file == "/" # skip the listing object name = file.sub(/^\//, '') # This makes sure that the first source *always* wins # for conflicting files. next if found.include?(name) # For directories, keep all of the sources, so that # sourceselect still works as planned. if type == "directory" newsource = @parameters[:source].should.collect do |tmpsource| tmpsource + file end else newsource = source + file end args = {:source => newsource} if type == file args[:recurse] = nil end found << name sourced << File.join(self[:path], name) self.newchild(name, false, args) }.reject {|c| c.nil? } if self[:sourceselect] == :first return [result, sourced] end end return [result, sourced] end # Set the checksum, from another property. There are multiple # properties that modify the contents of a file, and they need the # ability to make sure that the checksum value is in sync. def setchecksum(sum = nil) if @parameters.include? :checksum if sum @parameters[:checksum].checksum = sum else # If they didn't pass in a sum, then tell checksum to # figure it out. currentvalue = @parameters[:checksum].retrieve @parameters[:checksum].checksum = currentvalue end end end # Stat our file. Depending on the value of the 'links' attribute, we # use either 'stat' or 'lstat', and we expect the properties to use the # resulting stat object accordingly (mostly by testing the 'ftype' # value). def stat(refresh = false) method = :stat # Files are the only types that support links if (self.class.name == :file and self[:links] != :follow) or self.class.name == :tidy method = :lstat end path = self[:path] # Just skip them when they don't exist at all. unless FileTest.exists?(path) or FileTest.symlink?(path) @stat = nil return @stat end if @stat.nil? or refresh == true begin @stat = File.send(method, self[:path]) rescue Errno::ENOENT => error @stat = nil rescue Errno::EACCES => error self.warning "Could not stat; permission denied" @stat = nil end end return @stat end # We have to hack this just a little bit, because otherwise we'll get # an error when the target and the contents are created as properties on # the far side. def to_trans(retrieve = true) obj = super if obj[:target] == :notlink obj.delete(:target) end obj end def localfileserver unless defined? @@localfileserver args = { :Local => true, :Mount => { "/" => "localhost" }, :Config => false } @@localfileserver = Puppet::Network::Handler.handler(:fileserver).new(args) end @@localfileserver end def uri2obj(source) sourceobj = Puppet::Type::File::FileSource.new path = nil unless source devfail "Got a nil source" end if source =~ /^\// source = "file://localhost/%s" % URI.escape(source) sourceobj.mount = "localhost" sourceobj.local = true end begin uri = URI.parse(URI.escape(source)) rescue => detail self.fail "Could not understand source %s: %s" % [source, detail.to_s] end case uri.scheme when "file": sourceobj.server = localfileserver path = "/localhost" + uri.path when "puppet": # FIXME: We should cache clients by uri.host + uri.port # not by the full source path unless @clients.include?(source) host = uri.host host ||= Puppet[:server] unless Puppet[:name] == "puppet" if host.nil? server = localfileserver else args = { :Server => host } if uri.port args[:Port] = uri.port end server = Puppet::Network::Client.file.new(args) end @clients[source] = server end sourceobj.server = @clients[source] tmp = uri.path if tmp =~ %r{^/(\w+)} sourceobj.mount = $1 path = tmp #path = tmp.sub(%r{^/\w+},'') || "/" else self.fail "Invalid source path %s" % tmp end else self.fail "Got other URL type '%s' from %s" % [uri.scheme, source] end return [sourceobj, path.sub(/\/\//, '/')] end # Write out the file. Requires the content to be written, # the property name for logging, and the checksum for validation. def write(content, property, checksum = nil) if validate = validate_checksum? # Use the appropriate checksum type -- md5, md5lite, etc. sumtype = property(:checksum).checktype checksum ||= "{#{sumtype}}" + property(:checksum).send(sumtype, content) end remove_existing(:file) use_temporary_file = (content.length != 0) path = self[:path] path += ".puppettmp" if use_temporary_file mode = self.should(:mode) # might be nil umask = mode ? 000 : 022 Puppet::Util.withumask(umask) do File.open(path, File::CREAT|File::WRONLY|File::TRUNC, mode) { |f| f.print content } end # And put our new file in place if use_temporary_file # This is only not true when our file is empty. begin fail_if_checksum_is_wrong(path, checksum) if validate File.rename(path, self[:path]) rescue => detail self.err "Could not rename tmp %s for replacing: %s" % [self[:path], detail] ensure # Make sure the created file gets removed File.unlink(path) if FileTest.exists?(path) end end # make sure all of the modes are actually correct property_fix # And then update our checksum, so the next run doesn't find it. self.setchecksum(checksum) end # Should we validate the checksum of the file we're writing? def validate_checksum? if sumparam = @parameters[:checksum] return sumparam.checktype.to_s !~ /time/ else return false end end private # Make sure the file we wrote out is what we think it is. def fail_if_checksum_is_wrong(path, checksum) if checksum =~ /^\{(\w+)\}.+/ sumtype = $1 else # This shouldn't happen, but if it happens to, it's nicer # to just use a default sumtype than fail. sumtype = "md5" end newsum = property(:checksum).getsum(sumtype, path) return if newsum == checksum self.fail "File written to disk did not match checksum; discarding changes (%s vs %s)" % [checksum, newsum] end # Override the parent method, because we don't want to generate changes # when the file is missing and there is no 'ensure' state. def propertychanges(currentvalues) unless self.stat found = false ([:ensure] + CREATORS).each do |prop| if @parameters.include?(prop) found = true break end end unless found return [] end end super end # There are some cases where all of the work does not get done on # file creation/modification, so we have to do some extra checking. def property_fix properties.each do |thing| next unless [:mode, :owner, :group].include?(thing.name) # Make sure we get a new stat objct self.stat(true) currentvalue = thing.retrieve unless thing.insync?(currentvalue) thing.sync end end end end # Puppet.type(:pfile) # the filesource class can't include the path, because the path # changes for every file instance class ::Puppet::Type::File::FileSource attr_accessor :mount, :root, :server, :local end # We put all of the properties in separate files, because there are so many # of them. The order these are loaded is important, because it determines # the order they are in the property lit. require 'puppet/type/file/checksum' require 'puppet/type/file/content' # can create the file require 'puppet/type/file/source' # can create the file require 'puppet/type/file/target' # creates a different type of file require 'puppet/type/file/ensure' # can create the file require 'puppet/type/file/owner' require 'puppet/type/file/group' require 'puppet/type/file/mode' require 'puppet/type/file/type' end diff --git a/lib/puppet/type/file/content.rb b/lib/puppet/type/file/content.rb index 687a83f14..1eb1423aa 100755 --- a/lib/puppet/type/file/content.rb +++ b/lib/puppet/type/file/content.rb @@ -1,92 +1,82 @@ module Puppet - Puppet.type(:file).newproperty(:content) do + Puppet::Type.type(:file).newproperty(:content) do include Puppet::Util::Diff desc "Specify the contents of a file as a string. Newlines, tabs, and spaces can be specified using the escaped syntax (e.g., \\n for a newline). The primary purpose of this parameter is to provide a kind of limited templating:: define resolve(nameserver1, nameserver2, domain, search) { $str = \"search $search domain $domain nameserver $nameserver1 nameserver $nameserver2 \" file { \"/etc/resolv.conf\": content => $str } } This attribute is especially useful when used with `PuppetTemplating templating`:trac:." def change_to_s(currentvalue, newvalue) newvalue = "{md5}" + Digest::MD5.hexdigest(newvalue) if currentvalue == :absent return "created file with contents %s" % newvalue else currentvalue = "{md5}" + Digest::MD5.hexdigest(currentvalue) return "changed file contents from %s to %s" % [currentvalue, newvalue] end end # Override this method to provide diffs if asked for. # Also, fix #872: when content is used, and replace is true, the file # should be insync when it exists def insync?(is) if ! @resource.replace? and File.exists?(@resource[:path]) return true end result = super if ! result and Puppet[:show_diff] and File.exists?(@resource[:path]) string_file_diff(@resource[:path], self.should) end return result end - # We should probably take advantage of existing md5 sums if they're there, - # but I really don't feel like dealing with the complexity right now. def retrieve - stat = nil - unless stat = @resource.stat - return :absent - end + return :absent unless stat = @resource.stat - if stat.ftype == "link" and @resource[:links] == :ignore - return self.should - end + return self.should if stat.ftype == "link" and @resource[:links] == :ignore # Don't even try to manage the content on directories - if stat.ftype == "directory" and @resource[:links] == :ignore - @resource.delete(:content) - return nil - end + return nil if stat.ftype == "directory" begin currentvalue = File.read(@resource[:path]) return currentvalue rescue => detail raise Puppet::Error, "Could not read %s: %s" % [@resource.title, detail] end end # Make sure we're also managing the checksum property. def should=(value) super @resource.newattr(:checksum) unless @resource.property(:checksum) end # Just write our content out to disk. def sync return_event = @resource.stat ? :file_changed : :file_created @resource.write(self.should, :content) return return_event end end end diff --git a/lib/puppet/type/file/ensure.rb b/lib/puppet/type/file/ensure.rb index 028a7083d..0d2171216 100755 --- a/lib/puppet/type/file/ensure.rb +++ b/lib/puppet/type/file/ensure.rb @@ -1,177 +1,181 @@ module Puppet Puppet.type(:file).ensurable do require 'etc' desc "Whether to create files that don't currently exist. Possible values are *absent*, *present* (will match any form of file existence, and if the file is missing will create an empty file), *file*, and *directory*. Specifying ``absent`` will delete the file, although currently this will not recursively delete directories. Anything other than those values will be considered to be a symlink. For instance, the following text creates a link:: # Useful on solaris file { \"/etc/inetd.conf\": ensure => \"/etc/inet/inetd.conf\" } You can make relative links:: # Useful on solaris file { \"/etc/inetd.conf\": ensure => \"inet/inetd.conf\" } If you need to make a relative link to a file named the same as one of the valid values, you must prefix it with ``./`` or something similar. You can also make recursive symlinks, which will create a directory structure that maps to the target directory, with directories corresponding to each directory and links corresponding to each file." # Most 'ensure' properties have a default, but with files we, um, don't. nodefault newvalue(:absent) do File.unlink(@resource[:path]) end aliasvalue(:false, :absent) newvalue(:file) do # Make sure we're not managing the content some other way if property = (@resource.property(:content) || @resource.property(:source)) property.sync else @resource.write("", :ensure) mode = @resource.should(:mode) end return :file_created end #aliasvalue(:present, :file) newvalue(:present) do # Make a file if they want something, but this will match almost # anything. set_file end newvalue(:directory) do mode = @resource.should(:mode) parent = File.dirname(@resource[:path]) unless FileTest.exists? parent raise Puppet::Error, "Cannot create %s; parent directory %s does not exist" % [@resource[:path], parent] end if mode Puppet::Util.withumask(000) do Dir.mkdir(@resource[:path],mode) end else Dir.mkdir(@resource[:path]) end @resource.send(:property_fix) @resource.setchecksum return :directory_created end newvalue(:link) do if property = @resource.property(:target) property.retrieve return property.mklink else self.fail "Cannot create a symlink without a target" end end # Symlinks. newvalue(/./) do # This code never gets executed. We need the regex to support # specifying it, but the work is done in the 'symlink' code block. end munge do |value| value = super(value) + # It doesn't make sense to try to manage links unless, well, + # we're managing links. + resource[:links] = :manage if value == :link return value if value.is_a? Symbol @resource[:target] = value + resource[:links] = :manage return :link end def change_to_s(currentvalue, newvalue) if property = (@resource.property(:content) || @resource.property(:source)) and ! property.insync?(currentvalue) currentvalue = property.retrieve return property.change_to_s(property.retrieve, property.should) else super(currentvalue, newvalue) end end # Check that we can actually create anything def check basedir = File.dirname(@resource[:path]) if ! FileTest.exists?(basedir) raise Puppet::Error, "Can not create %s; parent directory does not exist" % @resource.title elsif ! FileTest.directory?(basedir) raise Puppet::Error, "Can not create %s; %s is not a directory" % [@resource.title, dirname] end end # We have to treat :present specially, because it works with any # type of file. def insync?(currentvalue) if property = @resource.property(:source) and ! property.described? warning "No specified sources exist" return true end if self.should == :present if currentvalue.nil? or currentvalue == :absent return false else return true end else return super(currentvalue) end end def retrieve if stat = @resource.stat(false) return stat.ftype.intern else if self.should == :false return :false else return :absent end end end def sync @resource.remove_existing(self.should) if self.should == :absent return :file_removed end event = super return event end end end diff --git a/lib/puppet/type/file/source.rb b/lib/puppet/type/file/source.rb index 7fa5eb1a9..1b0dd3141 100755 --- a/lib/puppet/type/file/source.rb +++ b/lib/puppet/type/file/source.rb @@ -1,277 +1,283 @@ module Puppet # Copy files from a local or remote source. This state *only* does any work # when the remote file is an actual file; in that case, this state copies # the file down. If the remote file is a dir or a link or whatever, then # this state, during retrieval, modifies the appropriate other states # so that things get taken care of appropriately. Puppet.type(:file).newproperty(:source) do include Puppet::Util::Diff attr_accessor :source, :local desc "Copy a file over the current file. Uses ``checksum`` to determine when a file should be copied. Valid values are either fully qualified paths to files, or URIs. Currently supported URI types are *puppet* and *file*. This is one of the primary mechanisms for getting content into applications that Puppet does not directly support and is very useful for those configuration files that don't change much across sytems. For instance:: class sendmail { file { \"/etc/mail/sendmail.cf\": source => \"puppet://server/module/sendmail.cf\" } } You can also leave out the server name, in which case ``puppetd`` will fill in the name of its configuration server and ``puppet`` will use the local filesystem. This makes it easy to use the same configuration in both local and centralized forms. Currently, only the ``puppet`` scheme is supported for source URL's. Puppet will connect to the file server running on ``server`` to retrieve the contents of the file. If the ``server`` part is empty, the behavior of the command-line interpreter (``puppet``) and the client demon (``puppetd``) differs slightly: ``puppet`` will look such a file up on the module path on the local host, whereas ``puppetd`` will connect to the puppet server that it received the manifest from. See the `FileServingConfiguration fileserver configuration documentation`:trac: for information on how to configure and use file services within Puppet. If you specify multiple file sources for a file, then the first source that exists will be used. This allows you to specify what amount to search paths for files:: file { \"/path/to/my/file\": source => [ \"/nfs/files/file.$host\", \"/nfs/files/file.$operatingsystem\", \"/nfs/files/file\" ] } This will use the first found file as the source. You cannot currently copy links using this mechanism; set ``links`` to ``follow`` if any remote sources are links. " uncheckable validate do |source| unless @resource.uri2obj(source) raise Puppet::Error, "Invalid source %s" % source end end munge do |source| # if source.is_a? Symbol # return source # end # Remove any trailing slashes source.sub(/\/$/, '') end def change_to_s(currentvalue, newvalue) # newvalue = "{md5}" + @stats[:checksum] if @resource.property(:ensure).retrieve == :absent return "creating from source %s with contents %s" % [@source, @stats[:checksum]] else return "replacing from source %s with contents %s" % [@source, @stats[:checksum]] end end def checksum if defined?(@stats) @stats[:checksum] else nil end end # Ask the file server to describe our file. def describe(source) sourceobj, path = @resource.uri2obj(source) server = sourceobj.server begin desc = server.describe(path, @resource[:links]) rescue Puppet::Network::XMLRPCClientError => detail self.err "Could not describe %s: %s" % [path, detail] return nil end + return nil if desc == "" + + # Collect everything except the checksum + values = desc.split("\t") + other = values.pop args = {} - pinparams.zip( - desc.split("\t") - ).each { |param, value| + pinparams.zip(values).each { |param, value| if value =~ /^[0-9]+$/ value = value.to_i end unless value.nil? args[param] = value end } - # we can't manage ownership as root, so don't even try - unless Puppet::Util::SUIDManager.uid == 0 - args.delete(:owner) + # Now decide whether we're doing checksums or symlinks + if args[:type] == "link" + args[:target] = other + else + args[:checksum] = other end - if args.empty? or (args[:type] == "link" and @resource[:links] == :ignore) - return nil - else - return args + # we can't manage ownership unless we're root, so don't even try + unless Puppet::Util::SUIDManager.uid == 0 + args.delete(:owner) end + + return args end # Have we successfully described the remote source? def described? ! @stats.nil? and ! @stats[:type].nil? #and @is != :notdescribed end # Use the info we get from describe() to check if we're in sync. def insync?(currentvalue) unless described? warning "No specified sources exist" return true end if currentvalue == :nocopy return true end # the only thing this actual state can do is copy files around. Therefore, # only pay attention if the remote is a file. unless @stats[:type] == "file" return true end #FIXARB: Inefficient? Needed to call retrieve on parent's ensure and checksum parentensure = @resource.property(:ensure).retrieve if parentensure != :absent and ! @resource.replace? return true end # Now, we just check to see if the checksums are the same parentchecksum = @resource.property(:checksum).retrieve result = (!parentchecksum.nil? and (parentchecksum == @stats[:checksum])) # Diff the contents if they ask it. This is quite annoying -- we need to do this in # 'insync?' because they might be in noop mode, but we don't want to do the file # retrieval twice, so we cache the value. if ! result and Puppet[:show_diff] and File.exists?(@resource[:path]) and ! @stats[:_diffed] @stats[:_remote_content] = get_remote_content string_file_diff(@resource[:path], @stats[:_remote_content]) @stats[:_diffed] = true end return result end def pinparams - Puppet::Network::Handler.handler(:fileserver).params + [:mode, :type, :owner, :group] end # This basically calls describe() on our file, and then sets all # of the local states appropriately. If the remote file is a normal # file then we set it to copy; if it's a directory, then we just mark # that the local directory should be created. def retrieve(remote = true) sum = nil @source = nil # This is set to false by the File#retrieve function on the second # retrieve, so that we do not do two describes. if remote # Find the first source that exists. @shouldorig contains # the sources as specified by the user. @should.each { |source| if @stats = self.describe(source) @source = source break end } end if @stats.nil? or @stats[:type].nil? return nil # :notdescribed end case @stats[:type] - when "directory", "file": + when "directory", "file", "link": @resource[:ensure] = @stats[:type] unless @resource.deleting? else self.info @stats.inspect self.err "Cannot use files of type %s as sources" % @stats[:type] return :nocopy end # Take each of the stats and set them as states on the local file # if a value has not already been provided. @stats.each { |stat, value| next if stat == :checksum next if stat == :type # was the stat already specified, or should the value # be inherited from the source? @resource[stat] = value unless @resource.argument?(stat) } return @stats[:checksum] end def should @should end # Make sure we're also checking the checksum def should=(value) super checks = (pinparams + [:ensure]) checks.delete(:checksum) @resource[:check] = checks @resource[:checksum] = :md5 unless @resource.property(:checksum) end def sync contents = @stats[:_remote_content] || get_remote_content() exists = File.exists?(@resource[:path]) @resource.write(contents, :source, @stats[:checksum]) if exists return :file_changed else return :file_created end end private def get_remote_content raise Puppet::DevError, "Got told to copy non-file %s" % @resource[:path] unless @stats[:type] == "file" sourceobj, path = @resource.uri2obj(@source) begin contents = sourceobj.server.retrieve(path, @resource[:links]) rescue => detail self.fail "Could not retrieve %s: %s" % [path, detail] end contents = CGI.unescape(contents) unless sourceobj.server.local if contents == "" self.notice "Could not retrieve contents for %s" % @source end return contents end end end diff --git a/lib/puppet/util/filetype.rb b/lib/puppet/util/filetype.rb index 1c7734cc4..bbc837e75 100755 --- a/lib/puppet/util/filetype.rb +++ b/lib/puppet/util/filetype.rb @@ -1,330 +1,330 @@ # Basic classes for reading, writing, and emptying files. Not much # to see here. class Puppet::Util::FileType attr_accessor :loaded, :path, :synced class << self attr_accessor :name include Puppet::Util::ClassGen end # Create a new filetype. def self.newfiletype(name, &block) @filetypes ||= {} klass = genclass(name, :block => block, :prefix => "FileType", :hash => @filetypes ) # Rename the read and write methods, so that we're sure they # maintain the stats. klass.class_eval do # Rename the read method define_method(:real_read, instance_method(:read)) define_method(:read) do begin val = real_read() @loaded = Time.now if val return val.gsub(/# HEADER.*\n/,'') else return "" end rescue Puppet::Error => detail raise rescue => detail if Puppet[:trace] puts detail.backtrace end raise Puppet::Error, "%s could not read %s: %s" % [self.class, @path, detail] end end # And then the write method define_method(:real_write, instance_method(:write)) define_method(:write) do |text| begin val = real_write(text) @synced = Time.now return val rescue Puppet::Error => detail raise rescue => detail if Puppet[:debug] puts detail.backtrace end raise Puppet::Error, "%s could not write %s: %s" % [self.class, @path, detail] end end end end def self.filetype(type) @filetypes[type] end # Back the file up before replacing it. def backup bucket.backup(@path) if FileTest.exists?(@path) end # Pick or create a filebucket to use. def bucket filebucket = Puppet::Type.type(:filebucket) (filebucket["puppet"] || filebucket.mkdefaultbucket).bucket end def initialize(path) raise ArgumentError.new("Path is nil") if path.nil? @path = path end # Operate on plain files. newfiletype(:flat) do # Read the file. def read if File.exists?(@path) File.read(@path) else return nil end end # Remove the file. def remove if File.exists?(@path) File.unlink(@path) end end # Overwrite the file. def write(text) backup() File.open(@path, "w") { |f| f.print text; f.flush } end end # Operate on plain files. newfiletype(:ram) do @@tabs = {} def self.clear @@tabs.clear end def initialize(path) super @@tabs[@path] ||= "" end # Read the file. def read Puppet.info "Reading %s from RAM" % @path @@tabs[@path] end # Remove the file. def remove Puppet.info "Removing %s from RAM" % @path @@tabs[@path] = "" end # Overwrite the file. def write(text) Puppet.info "Writing %s to RAM" % @path @@tabs[@path] = text end end # Handle Linux-style cron tabs. newfiletype(:crontab) do def initialize(user) self.path = user end def path=(user) begin @uid = Puppet::Util.uid(user) rescue Puppet::Error => detail raise Puppet::Error, "Could not retrieve user %s" % user end # XXX We have to have the user name, not the uid, because some # systems *cough*linux*cough* require it that way @path = user end # Read a specific @path's cron tab. def read %x{#{cmdbase()} -l 2>/dev/null} end # Remove a specific @path's cron tab. def remove - if Facter.value("operatingsystem") == "FreeBSD" + if %w{Darwin FreeBSD}.include?(Facter.value("operatingsystem")) %x{/bin/echo yes | #{cmdbase()} -r 2>/dev/null} else %x{#{cmdbase()} -r 2>/dev/null} end end # Overwrite a specific @path's cron tab; must be passed the @path name # and the text with which to create the cron tab. def write(text) IO.popen("#{cmdbase()} -", "w") { |p| p.print text } end private # Only add the -u flag when the @path is different. Fedora apparently # does not think I should be allowed to set the @path to my own user name def cmdbase cmd = nil if @uid == Puppet::Util::SUIDManager.uid return "crontab" else return "crontab -u #{@path}" end end end # SunOS has completely different cron commands; this class implements # its versions. newfiletype(:suntab) do # Read a specific @path's cron tab. def read begin output = Puppet::Util.execute(%w{crontab -l}, :uid => @path) return "" if output.include?("can't open your crontab") return output rescue # If there's a failure, treat it like an empty file. return "" end end # Remove a specific @path's cron tab. def remove begin Puppet::Util.execute(%w{crontab -r}, :uid => @path) rescue => detail raise Puppet::Error, "Could not remove crontab for %s: %s" % [@path, detail] end end # Overwrite a specific @path's cron tab; must be passed the @path name # and the text with which to create the cron tab. def write(text) puts text require "tempfile" output_file = Tempfile.new("puppet") fh = output_file.open fh.print text fh.close # We have to chown the stupid file to the user. File.chown(Puppet::Util.uid(@path), nil, output_file.path) begin Puppet::Util.execute(["crontab", output_file.path], :uid => @path) rescue => detail raise Puppet::Error, "Could not write crontab for %s: %s" % [@path, detail] end output_file.delete end end # Treat netinfo tables as a single file, just for simplicity of certain # types newfiletype(:netinfo) do class << self attr_accessor :format end def read %x{nidump -r /#{@path} /} end # This really only makes sense for cron tabs. def remove %x{nireport / /#{@path} name}.split("\n").each do |name| newname = name.gsub(/\//, '\/').sub(/\s+$/, '') output = %x{niutil -destroy / '/#{@path}/#{newname}'} unless $? == 0 raise Puppet::Error, "Could not remove %s from %s" % [name, @path] end end end # Convert our table to an array of hashes. This only works for # handling one table at a time. def to_array(text = nil) unless text text = read end hash = nil # Initialize it with the first record records = [] text.split("\n").each do |line| next if line =~ /^[{}]$/ # Skip the wrapping lines next if line =~ /"name" = \( "#{@path}" \)/ # Skip the table name next if line =~ /CHILDREN = \(/ # Skip this header next if line =~ /^ \)/ # and its closer # Now we should have nothing but records, wrapped in braces case line when /^\s+\{/: hash = {} when /^\s+\}/: records << hash when /\s+"(\w+)" = \( (.+) \)/ field = $1 values = $2 # Always use an array hash[field] = [] values.split(/, /).each do |value| if value =~ /^"(.*)"$/ hash[field] << $1 else raise ArgumentError, "Could not match value %s" % value end end else raise ArgumentError, "Could not match line %s" % line end end records end def write(text) text.gsub!(/^#.*\n/,'') text.gsub!(/^$/,'') if text == "" or text == "\n" self.remove return end unless format = self.class.format raise Puppe::DevError, "You must define the NetInfo format to inport" end IO.popen("niload -d #{format} . 1>/dev/null 2>/dev/null", "w") { |p| p.print text } unless $? == 0 raise ArgumentError, "Failed to write %s" % @path end end end end diff --git a/lib/puppet/util/settings.rb b/lib/puppet/util/settings.rb index cf15d3194..a91b0d6b1 100644 --- a/lib/puppet/util/settings.rb +++ b/lib/puppet/util/settings.rb @@ -1,1243 +1,1243 @@ require 'puppet' require 'sync' require 'puppet/transportable' require 'getoptlong' # The class for handling configuration files. class Puppet::Util::Settings include Enumerable include Puppet::Util @@sync = Sync.new attr_accessor :file attr_reader :timer # Retrieve a config value def [](param) value(param) end # Set a config value. This doesn't set the defaults, it sets the value itself. def []=(param, value) @@sync.synchronize do # yay, thread-safe param = symbolize(param) unless element = @config[param] raise ArgumentError, "Attempt to assign a value to unknown configuration parameter %s" % param.inspect end if element.respond_to?(:munge) value = element.munge(value) end if element.respond_to?(:handle) element.handle(value) end # Reset the name, so it's looked up again. if param == :name @name = nil end @values[:memory][param] = value @cache.clear end return value end # A simplified equality operator. # LAK: For some reason, this causes mocha to not be able to mock # the 'value' method, and it's not used anywhere. # def ==(other) # self.each { |myname, myobj| # unless other[myname] == value(myname) # return false # end # } # # return true # end # Generate the list of valid arguments, in a format that GetoptLong can # understand, and add them to the passed option list. def addargs(options) # Hackish, but acceptable. Copy the current ARGV for restarting. Puppet.args = ARGV.dup # Add all of the config parameters as valid options. self.each { |name, element| element.getopt_args.each { |args| options << args } } return options end # Turn the config into a Puppet configuration and apply it def apply trans = self.to_transportable begin config = trans.to_catalog config.store_state = false config.apply config.clear rescue => detail if Puppet[:trace] puts detail.backtrace end Puppet.err "Could not configure myself: %s" % detail end end # Is our parameter a boolean parameter? def boolean?(param) param = symbolize(param) if @config.include?(param) and @config[param].kind_of? CBoolean return true else return false end end # Remove all set values, potentially skipping cli values. def clear(exceptcli = false) @config.each { |name, obj| unless exceptcli and obj.setbycli obj.clear end } @values.each do |name, values| next if name == :cli and exceptcli @values.delete(name) end # Don't clear the 'used' in this case, since it's a config file reparse, # and we want to retain this info. unless exceptcli @used = [] end @cache.clear @name = nil end # This is mostly just used for testing. def clearused @cache.clear @used = [] end # Do variable interpolation on the value. def convert(value) return value unless value return value unless value.is_a? String newval = value.gsub(/\$(\w+)|\$\{(\w+)\}/) do |value| varname = $2 || $1 if pval = self.value(varname) pval else raise Puppet::DevError, "Could not find value for %s" % parent end end return newval end # Return a value's description. def description(name) if obj = @config[symbolize(name)] obj.desc else nil end end def each @config.each { |name, object| yield name, object } end # Iterate over each section name. def eachsection yielded = [] @config.each do |name, object| section = object.section unless yielded.include? section yield section yielded << section end end end # Return an object by name. def element(param) param = symbolize(param) @config[param] end # Handle a command-line argument. def handlearg(opt, value = nil) @cache.clear value = munge_value(value) if value str = opt.sub(/^--/,'') bool = true newstr = str.sub(/^no-/, '') if newstr != str str = newstr bool = false end str = str.intern if self.valid?(str) if self.boolean?(str) @values[:cli][str] = bool else @values[:cli][str] = value end else raise ArgumentError, "Invalid argument %s" % opt end end def include?(name) name = name.intern if name.is_a? String @config.include?(name) end # check to see if a short name is already defined def shortinclude?(short) short = short.intern if name.is_a? String @shortnames.include?(short) end # Create a new config object def initialize @config = {} @shortnames = {} @created = [] @searchpath = nil # Keep track of set values. @values = Hash.new { |hash, key| hash[key] = {} } # And keep a per-environment cache @cache = Hash.new { |hash, key| hash[key] = {} } # A central concept of a name. @name = nil end # Return a given object's file metadata. def metadata(param) if obj = @config[symbolize(param)] and obj.is_a?(CFile) return [:owner, :group, :mode].inject({}) do |meta, p| if v = obj.send(p) meta[p] = v end meta end else nil end end # Make a directory with the appropriate user, group, and mode def mkdir(default) obj = nil unless obj = @config[default] raise ArgumentError, "Unknown default %s" % default end unless obj.is_a? CFile raise ArgumentError, "Default %s is not a file" % default end Puppet::Util::SUIDManager.asuser(obj.owner, obj.group) do mode = obj.mode || 0750 Dir.mkdir(obj.value, mode) end end # Figure out our name. def name unless @name unless @config[:name] return nil end searchpath.each do |source| next if source == :name break if @name = @values[source][:name] end unless @name @name = convert(@config[:name].default).intern end end @name end # Return all of the parameters associated with a given section. def params(section = nil) if section section = section.intern if section.is_a? String @config.find_all { |name, obj| obj.section == section }.collect { |name, obj| name } else @config.keys end end # Parse the configuration file. def parse(file) clear(true) parse_file(file).each do |area, values| @values[area] = values end # Determine our environment, if we have one. if @config[:environment] env = self.value(:environment).to_sym else env = "none" end # Call any hooks we should be calling. settings_with_hooks.each do |setting| each_source(env) do |source| if value = @values[source][setting.name] # We still have to use value() to retrieve the value, since # we want the fully interpolated value, not $vardir/lib or whatever. # This results in extra work, but so few of the settings # will have associated hooks that it ends up being less work this # way overall. setting.handle(self.value(setting.name, env)) break end end end # We have to do it in the reverse of the search path, # because multiple sections could set the same value # and I'm too lazy to only set the metadata once. searchpath.reverse.each do |source| if meta = @values[source][:_meta] set_metadata(meta) end end end # Parse the configuration file. As of May 2007, this is a backward-compatibility method and # will be deprecated soon. def old_parse(file) text = nil if file.is_a? Puppet::Util::LoadedFile @file = file else @file = Puppet::Util::LoadedFile.new(file) end # Don't create a timer for the old style parsing. # settimer() begin text = File.read(@file.file) rescue Errno::ENOENT raise Puppet::Error, "No such file %s" % file rescue Errno::EACCES raise Puppet::Error, "Permission denied to file %s" % file end @values = Hash.new { |names, name| names[name] = {} } # Get rid of the values set by the file, keeping cli values. self.clear(true) section = "puppet" metas = %w{owner group mode} values = Hash.new { |hash, key| hash[key] = {} } text.split(/\n/).each { |line| case line when /^\[(\w+)\]$/: section = $1 # Section names when /^\s*#/: next # Skip comments when /^\s*$/: next # Skip blanks when /^\s*(\w+)\s*=\s*(.+)$/: # settings var = $1.intern if var == :mode value = $2 else value = munge_value($2) end # Only warn if we don't know what this config var is. This # prevents exceptions later on. unless @config.include?(var) or metas.include?(var.to_s) Puppet.warning "Discarded unknown configuration parameter %s" % var.inspect next # Skip this line. end # Mmm, "special" attributes if metas.include?(var.to_s) unless values.include?(section) values[section] = {} end values[section][var.to_s] = value # If the parameter is valid, then set it. if section == Puppet[:name] and @config.include?(var) #@config[var].value = value @values[:main][var] = value end next end # Don't override set parameters, since the file is parsed # after cli arguments are handled. unless @config.include?(var) and @config[var].setbycli Puppet.debug "%s: Setting %s to '%s'" % [section, var, value] @values[:main][var] = value end @config[var].section = symbolize(section) metas.each { |meta| if values[section][meta] if @config[var].respond_to?(meta + "=") @config[var].send(meta + "=", values[section][meta]) end end } else raise Puppet::Error, "Could not match line %s" % line end } end # Create a new element. The value is passed in because it's used to determine # what kind of element we're creating, but the value itself might be either # a default or a value, so we can't actually assign it. def newelement(hash) value = hash[:value] || hash[:default] klass = nil if hash[:section] hash[:section] = symbolize(hash[:section]) end case value when true, false, "true", "false": klass = CBoolean when /^\$\w+\//, /^\//: klass = CFile when String, Integer, Float: # nothing klass = CElement else raise Puppet::Error, "Invalid value '%s' for %s" % [value.inspect, hash[:name]] end hash[:parent] = self element = klass.new(hash) return element end # This has to be private, because it doesn't add the elements to @config private :newelement # Iterate across all of the objects in a given section. def persection(section) section = symbolize(section) self.each { |name, obj| if obj.section == section yield obj end } end # Reparse our config file, if necessary. def reparse if defined? @file and @file.changed? Puppet.notice "Reparsing %s" % @file.file @@sync.synchronize do parse(@file) end reuse() end end def reuse return unless defined? @used @@sync.synchronize do # yay, thread-safe @used.each do |section| @used.delete(section) self.use(section) end end end # The order in which to search for values. def searchpath(environment = nil) if environment [:cli, :memory, environment, :name, :main] else [:cli, :memory, :name, :main] end end # Get a list of objects per section def sectionlist sectionlist = [] self.each { |name, obj| section = obj.section || "puppet" sections[section] ||= [] unless sectionlist.include?(section) sectionlist << section end sections[section] << obj } return sectionlist, sections end # Convert a single section into transportable objects. def section_to_transportable(section, done = nil) done ||= Hash.new { |hash, key| hash[key] = {} } objects = [] persection(section) do |obj| if @config[:mkusers] and value(:mkusers) objects += add_user_resources(section, obj, done) end value = obj.value # Only files are convertable to transportable resources. next unless obj.respond_to? :to_transportable and transobjects = obj.to_transportable transobjects = [transobjects] unless transobjects.is_a? Array transobjects.each do |trans| # transportable could return nil next unless trans unless done[:file].include? trans.name @created << trans.name objects << trans done[:file][trans.name] = trans end end end bucket = Puppet::TransBucket.new bucket.type = "Settings" bucket.name = section bucket.push(*objects) bucket.keyword = "class" return bucket end # Set a bunch of defaults in a given section. The sections are actually pretty # pointless, but they help break things up a bit, anyway. def setdefaults(section, defs) section = symbolize(section) call = [] defs.each { |name, hash| if hash.is_a? Array unless hash.length == 2 raise ArgumentError, "Defaults specified as an array must contain only the default value and the decription" end tmp = hash hash = {} [:default, :desc].zip(tmp).each { |p,v| hash[p] = v } end name = symbolize(name) hash[:name] = name hash[:section] = section name = hash[:name] if @config.include?(name) raise ArgumentError, "Parameter %s is already defined" % name end tryconfig = newelement(hash) if short = tryconfig.short if other = @shortnames[short] raise ArgumentError, "Parameter %s is already using short name '%s'" % [other.name, short] end @shortnames[short] = tryconfig end @config[name] = tryconfig # Collect the settings that need to have their hooks called immediately. # We have to collect them so that we can be sure we're fully initialized before # the hook is called. call << tryconfig if tryconfig.call_on_define } call.each { |setting| setting.handle(self.value(setting.name)) } end # Create a timer to check whether the file should be reparsed. def settimer if Puppet[:filetimeout] > 0 @timer = Puppet.newtimer( :interval => Puppet[:filetimeout], :tolerance => 1, :start? => true ) do self.reparse() end end end # Convert our list of objects into a component that can be applied. def to_configuration transport = self.to_transportable return transport.to_catalog end # Convert our list of config elements into a configuration file. def to_config str = %{The configuration file for #{Puppet[:name]}. Note that this file is likely to have unused configuration parameters in it; any parameter that's valid anywhere in Puppet can be in any config file, even if it's not used. Every section can specify three special parameters: owner, group, and mode. These parameters affect the required permissions of any files specified after their specification. Puppet will sometimes use these parameters to check its own configured state, so they can be used to make Puppet a bit more self-managing. Note also that the section names are entirely for human-level organizational purposes; they don't provide separate namespaces. All parameters are in a single namespace. Generated on #{Time.now}. }.gsub(/^/, "# ") # Add a section heading that matches our name. if @config.include?(:name) str += "[%s]\n" % self[:name] end eachsection do |section| persection(section) do |obj| str += obj.to_config + "\n" end end return str end # Convert our configuration into a list of transportable objects. def to_transportable(*sections) done = Hash.new { |hash, key| hash[key] = {} } topbucket = Puppet::TransBucket.new if defined? @file.file and @file.file topbucket.name = @file.file else topbucket.name = "top" end topbucket.type = "Settings" topbucket.top = true # Now iterate over each section if sections.empty? eachsection do |section| sections << section end end sections.each do |section| obj = section_to_transportable(section, done) topbucket.push obj end topbucket end # Convert to a parseable manifest def to_manifest transport = self.to_transportable manifest = transport.to_manifest + "\n" eachsection { |section| manifest += "include #{section}\n" } return manifest end # Create the necessary objects to use a section. This is idempotent; # you can 'use' a section as many times as you want. def use(*sections) @@sync.synchronize do # yay, thread-safe unless defined? @used @used = [] end bucket = to_transportable(*sections) config = bucket.to_catalog config.host_config = false config.apply do |transaction| if failures = transaction.any_failed? raise "Could not configure for running; got %s failure(s)" % failures end end config.clear sections.each { |s| @used << s } @used.uniq end end def valid?(param) param = symbolize(param) @config.has_key?(param) end # Find the correct value using our search path. Optionally accept an environment # in which to search before the other configuration sections. def value(param, environment = nil) param = symbolize(param) environment = symbolize(environment) if environment # Short circuit to nil for undefined parameters. return nil unless @config.include?(param) # Yay, recursion. self.reparse() unless param == :filetimeout # Check the cache first. It needs to be a per-environment # cache so that we don't spread values from one env # to another. if cached = @cache[environment||"none"][param] return cached end # See if we can find it within our searchable list of values val = nil each_source(environment) do |source| # Look for the value. We have to test the hash for whether # it exists, because the value might be false. if @values[source].include?(param) val = @values[source][param] break end end # If we didn't get a value, use the default val = @config[param].default if val.nil? # Convert it if necessary val = convert(val) # And cache it @cache[environment||"none"][param] = val return val end # Open a file with the appropriate user, group, and mode def write(default, *args) obj = nil unless obj = @config[default] raise ArgumentError, "Unknown default %s" % default end unless obj.is_a? CFile raise ArgumentError, "Default %s is not a file" % default end chown = nil if Puppet::Util::SUIDManager.uid == 0 chown = [obj.owner, obj.group] else chown = [nil, nil] end Puppet::Util::SUIDManager.asuser(*chown) do mode = obj.mode || 0640 if args.empty? args << "w" end args << mode File.open(value(obj.name), *args) do |file| yield file end end end # Open a non-default file under a default dir with the appropriate user, # group, and mode def writesub(default, file, *args) obj = nil unless obj = @config[default] raise ArgumentError, "Unknown default %s" % default end unless obj.is_a? CFile raise ArgumentError, "Default %s is not a file" % default end chown = nil if Puppet::Util::SUIDManager.uid == 0 chown = [obj.owner, obj.group] else chown = [nil, nil] end Puppet::Util::SUIDManager.asuser(*chown) do mode = obj.mode || 0640 if args.empty? args << "w" end args << mode # Update the umask to make non-executable files Puppet::Util.withumask(File.umask ^ 0111) do File.open(file, *args) do |file| yield file end end end end private # Create the transportable objects for users and groups. def add_user_resources(section, obj, done) resources = [] [:owner, :group].each do |attr| type = nil if attr == :owner type = :user else type = attr end # If a user and/or group is set, then make sure we're # managing that object if obj.respond_to? attr and name = obj.send(attr) # Skip root or wheel next if %w{root wheel}.include?(name.to_s) # Skip owners and groups we've already done, but tag # them with our section if necessary if done[type].include?(name) tags = done[type][name].tags unless tags.include?(section) done[type][name].tags = tags << section end else newobj = Puppet::TransObject.new(name, type.to_s) newobj.tags = ["puppet", "configuration", section] newobj[:ensure] = :present if type == :user newobj[:comment] ||= "%s user" % name end # Set the group appropriately for the user if type == :user newobj[:gid] = Puppet[:group] end done[type][name] = newobj resources << newobj end end end resources end # Yield each search source in turn. def each_source(environment) searchpath(environment).each do |source| # Modify the source as necessary. source = self.name if source == :name yield source end end # Return all elements that have associated hooks; this is so # we can call them after parsing the configuration file. def settings_with_hooks @config.values.find_all { |setting| setting.respond_to?(:handle) } end # Extract extra setting information for files. def extract_fileinfo(string) result = {} value = string.sub(/\{\s*([^}]+)\s*\}/) do params = $1 params.split(/\s*,\s*/).each do |str| if str =~ /^\s*(\w+)\s*=\s*([\w\d]+)\s*$/ param, value = $1.intern, $2 result[param] = value unless [:owner, :mode, :group].include?(param) raise ArgumentError, "Invalid file option '%s'" % param end if param == :mode and value !~ /^\d+$/ raise ArgumentError, "File modes must be numbers" end else raise ArgumentError, "Could not parse '%s'" % string end end '' end result[:value] = value.sub(/\s*$/, '') return result return nil end # Convert arguments into booleans, integers, or whatever. def munge_value(value) # Handle different data types correctly return case value when /^false$/i: false when /^true$/i: true when /^\d+$/i: Integer(value) else value.gsub(/^["']|["']$/,'').sub(/\s+$/, '') end end # This is an abstract method that just turns a file in to a hash of hashes. # We mostly need this for backward compatibility -- as of May 2007 we need to # support parsing old files with any section, or new files with just two # valid sections. def parse_file(file) text = read_file(file) # Create a timer so that this file will get checked automatically # and reparsed if necessary. settimer() result = Hash.new { |names, name| names[name] = {} } count = 0 # Default to 'main' for the section. section = :main result[section][:_meta] = {} text.split(/\n/).each { |line| count += 1 case line when /^\s*\[(\w+)\]$/: section = $1.intern # Section names # Add a meta section result[section][:_meta] ||= {} when /^\s*#/: next # Skip comments when /^\s*$/: next # Skip blanks when /^\s*(\w+)\s*=\s*(.*)$/: # settings var = $1.intern # We don't want to munge modes, because they're specified in octal, so we'll # just leave them as a String, since Puppet handles that case correctly. if var == :mode value = $2 else value = munge_value($2) end # Check to see if this is a file argument and it has extra options begin if value.is_a?(String) and options = extract_fileinfo(value) value = options[:value] options.delete(:value) result[section][:_meta][var] = options end result[section][var] = value rescue Puppet::Error => detail detail.file = file detail.line = line raise end else error = Puppet::Error.new("Could not match line %s" % line) error.file = file error.line = line raise error end } return result end # Read the file in. def read_file(file) if file.is_a? Puppet::Util::LoadedFile @file = file else @file = Puppet::Util::LoadedFile.new(file) end begin return File.read(@file.file) rescue Errno::ENOENT raise ArgumentError, "No such file %s" % file rescue Errno::EACCES raise ArgumentError, "Permission denied to file %s" % file end end # Set file metadata. def set_metadata(meta) meta.each do |var, values| values.each do |param, value| @config[var].send(param.to_s + "=", value) end end end # The base element type. class CElement attr_accessor :name, :section, :default, :parent, :setbycli, :call_on_define attr_reader :desc, :short # Unset any set value. def clear @value = nil end def desc=(value) @desc = value.gsub(/^\s*/, '') end # get the arguments in getopt format def getopt_args if short [["--#{name}", "-#{short}", GetoptLong::REQUIRED_ARGUMENT]] else [["--#{name}", GetoptLong::REQUIRED_ARGUMENT]] end end def hook=(block) meta_def :handle, &block end # Create the new element. Pretty much just sets the name. def initialize(args = {}) if args.include?(:parent) self.parent = args[:parent] args.delete(:parent) end args.each do |param, value| method = param.to_s + "=" unless self.respond_to? method raise ArgumentError, "%s does not accept %s" % [self.class, param] end self.send(method, value) end unless self.desc raise ArgumentError, "You must provide a description for the %s config option" % self.name end end def iscreated @iscreated = true end def iscreated? if defined? @iscreated return @iscreated else return false end end def set? if defined? @value and ! @value.nil? return true else return false end end # short name for the celement def short=(value) if value.to_s.length != 1 raise ArgumentError, "Short names can only be one character." end @short = value.to_s end # Convert the object to a config statement. def to_config str = @desc.gsub(/^/, "# ") + "\n" # Add in a statement about the default. if defined? @default and @default str += "# The default value is '%s'.\n" % @default end # If the value has not been overridden, then print it out commented # and unconverted, so it's clear that that's the default and how it # works. value = @parent.value(self.name) if value != @default line = "%s = %s" % [@name, value] else line = "# %s = %s" % [@name, @default] end str += line + "\n" str.gsub(/^/, " ") end # Retrieves the value, or if it's not set, retrieves the default. def value @parent.value(self.name) end end # A file. class CFile < CElement attr_writer :owner, :group attr_accessor :mode, :create def group if defined? @group return @parent.convert(@group) else return nil end end def owner if defined? @owner return @parent.convert(@owner) else return nil end end # Set the type appropriately. Yep, a hack. This supports either naming # the variable 'dir', or adding a slash at the end. def munge(value) # If it's not a fully qualified path... if value.is_a?(String) and value !~ /^\$/ and value !~ /^\// and value != 'false' # Make it one value = File.join(Dir.getwd, value) end if value.to_s =~ /\/$/ @type = :directory return value.sub(/\/$/, '') end return value end # Return the appropriate type. def type value = @parent.value(self.name) if @name.to_s =~ /dir/ return :directory elsif value.to_s =~ /\/$/ return :directory elsif value.is_a? String return :file else return nil end end # Convert the object to a TransObject instance. def to_transportable type = self.type return nil unless type path = self.value return nil unless path.is_a?(String) return nil if path =~ /^\/dev/ - return nil if Puppet::Type::File[path] # skip files that are in our global resource list. + return nil if Puppet::Type.type(:file)[path] # skip files that are in our global resource list. objects = [] # Skip plain files that don't exist, since we won't be managing them anyway. return nil unless self.name.to_s =~ /dir$/ or File.exist?(path) or self.create obj = Puppet::TransObject.new(path, "file") # Only create directories, or files that are specifically marked to # create. if type == :directory or self.create obj[:ensure] = type end [:mode].each { |var| if value = self.send(var) # Don't bother converting the mode, since the file type # can handle it any old way. obj[var] = value end } # Only chown or chgrp when root if Puppet.features.root? [:group, :owner].each { |var| if value = self.send(var) obj[var] = value end } end # And set the loglevel to debug for everything obj[:loglevel] = "debug" # We're not actually modifying any files here, and if we allow a # filebucket to get used here we get into an infinite recursion # trying to set the filebucket up. obj[:backup] = false if self.section obj.tags += ["puppet", "configuration", self.section, self.name] end objects << obj objects end # Make sure any provided variables look up to something. def validate(value) return true unless value.is_a? String value.scan(/\$(\w+)/) { |name| name = $1 unless @parent.include?(name) raise ArgumentError, "Settings parameter '%s' is undefined" % name end } end end # A simple boolean. class CBoolean < CElement # get the arguments in getopt format def getopt_args if short [["--#{name}", "-#{short}", GetoptLong::NO_ARGUMENT], ["--no-#{name}", GetoptLong::NO_ARGUMENT]] else [["--#{name}", GetoptLong::NO_ARGUMENT], ["--no-#{name}", GetoptLong::NO_ARGUMENT]] end end def munge(value) case value when true, "true": return true when false, "false": return false else raise ArgumentError, "Invalid value '%s' for %s" % [value.inspect, @name] end end end end diff --git a/spec/unit/file_serving/file_base.rb b/spec/unit/file_serving/file_base.rb index 4c7724f7c..e1a61cd65 100755 --- a/spec/unit/file_serving/file_base.rb +++ b/spec/unit/file_serving/file_base.rb @@ -1,101 +1,120 @@ #!/usr/bin/env ruby require File.dirname(__FILE__) + '/../../spec_helper' require 'puppet/file_serving/file_base' -describe Puppet::FileServing::FileBase, " when initializing" do +describe Puppet::FileServing::FileBase do it "should accept a key in the form of a URI" do Puppet::FileServing::FileBase.new("puppet://host/module/dir/file").key.should == "puppet://host/module/dir/file" end it "should allow specification of whether links should be managed" do Puppet::FileServing::FileBase.new("puppet://host/module/dir/file", :links => :manage).links.should == :manage end it "should fail if :links is set to anything other than :manage or :follow" do proc { Puppet::FileServing::FileBase.new("puppet://host/module/dir/file", :links => :else) }.should raise_error(ArgumentError) end it "should default to :manage for :links" do Puppet::FileServing::FileBase.new("puppet://host/module/dir/file").links.should == :manage end it "should allow specification of a path" do FileTest.stubs(:exists?).returns(true) Puppet::FileServing::FileBase.new("puppet://host/module/dir/file", :path => "/my/file").path.should == "/my/file" end it "should allow specification of a relative path" do FileTest.stubs(:exists?).returns(true) Puppet::FileServing::FileBase.new("puppet://host/module/dir/file", :relative_path => "my/file").relative_path.should == "my/file" end -end -describe Puppet::FileServing::FileBase, " when setting the base path" do - before do - @file = Puppet::FileServing::FileBase.new("puppet://host/module/dir/file") + it "should have a means of determining if the file exists" do + Puppet::FileServing::FileBase.new("blah").should respond_to(:exist?) end - it "should require that the base path be fully qualified" do - FileTest.stubs(:exists?).returns(true) - proc { @file.path = "unqualified/file" }.should raise_error(ArgumentError) + it "should correctly indicate if the file is present" do + File.expects(:lstat).with("/my/file").returns(mock("stat")) + Puppet::FileServing::FileBase.new("blah", :path => "/my/file").exist?.should be_true end -end -describe Puppet::FileServing::FileBase, " when setting the relative path" do - it "should require that the relative path be unqualified" do - @file = Puppet::FileServing::FileBase.new("puppet://host/module/dir/file") - FileTest.stubs(:exists?).returns(true) - proc { @file.relative_path = "/qualified/file" }.should raise_error(ArgumentError) + it "should correctly indicate if the file is asbsent" do + File.expects(:lstat).with("/my/file").raises RuntimeError + Puppet::FileServing::FileBase.new("blah", :path => "/my/file").exist?.should be_false end -end -describe Puppet::FileServing::FileBase, " when determining the full file path" do - before do - @file = Puppet::FileServing::FileBase.new("mykey", :path => "/this/file") - end + describe "when setting the base path" do + before do + @file = Puppet::FileServing::FileBase.new("puppet://host/module/dir/file") + end - it "should return the path if there is no relative path" do - @file.full_path.should == "/this/file" + it "should require that the base path be fully qualified" do + FileTest.stubs(:exists?).returns(true) + proc { @file.path = "unqualified/file" }.should raise_error(ArgumentError) + end end - it "should return the path joined with the relative path if there is a relative path" do - @file.relative_path = "not/qualified" - @file.full_path.should == "/this/file/not/qualified" + describe "when setting the relative path" do + it "should require that the relative path be unqualified" do + @file = Puppet::FileServing::FileBase.new("puppet://host/module/dir/file") + FileTest.stubs(:exists?).returns(true) + proc { @file.relative_path = "/qualified/file" }.should raise_error(ArgumentError) + end end - it "should should fail if there is no path set" do - @file = Puppet::FileServing::FileBase.new("not/qualified") - proc { @file.full_path }.should raise_error(ArgumentError) - end -end + describe "when determining the full file path" do + before do + @file = Puppet::FileServing::FileBase.new("mykey", :path => "/this/file") + end -describe Puppet::FileServing::FileBase, " when stat'ing files" do - before do - @file = Puppet::FileServing::FileBase.new("mykey", :path => "/this/file") - end + it "should return the path if there is no relative path" do + @file.full_path.should == "/this/file" + end - it "should stat the file's full path" do - @file.stubs(:full_path).returns("/this/file") - File.expects(:lstat).with("/this/file").returns stub("stat", :ftype => "file") - @file.stat - end + it "should return the path if the relative_path is set to ''" do + @file.relative_path = "" + @file.full_path.should == "/this/file" + end - it "should fail if the file does not exist" do - @file.stubs(:full_path).returns("/this/file") - File.expects(:lstat).with("/this/file").raises(Errno::ENOENT) - proc { @file.stat }.should raise_error(Errno::ENOENT) - end + it "should return the path joined with the relative path if there is a relative path and it is not set to '/' or ''" do + @file.relative_path = "not/qualified" + @file.full_path.should == "/this/file/not/qualified" + end - it "should use :lstat if :links is set to :manage" do - File.expects(:lstat).with("/this/file").returns stub("stat", :ftype => "file") - @file.stat + it "should should fail if there is no path set" do + @file = Puppet::FileServing::FileBase.new("not/qualified") + proc { @file.full_path }.should raise_error(ArgumentError) + end end - it "should use :stat if :links is set to :follow" do - File.expects(:stat).with("/this/file").returns stub("stat", :ftype => "file") - @file.links = :follow - @file.stat + describe "when stat'ing files" do + before do + @file = Puppet::FileServing::FileBase.new("mykey", :path => "/this/file") + end + + it "should stat the file's full path" do + @file.stubs(:full_path).returns("/this/file") + File.expects(:lstat).with("/this/file").returns stub("stat", :ftype => "file") + @file.stat + end + + it "should fail if the file does not exist" do + @file.stubs(:full_path).returns("/this/file") + File.expects(:lstat).with("/this/file").raises(Errno::ENOENT) + proc { @file.stat }.should raise_error(Errno::ENOENT) + end + + it "should use :lstat if :links is set to :manage" do + File.expects(:lstat).with("/this/file").returns stub("stat", :ftype => "file") + @file.stat + end + + it "should use :stat if :links is set to :follow" do + File.expects(:stat).with("/this/file").returns stub("stat", :ftype => "file") + @file.links = :follow + @file.stat + end end end diff --git a/spec/unit/file_serving/metadata.rb b/spec/unit/file_serving/metadata.rb index d31dd21f0..9743370c1 100755 --- a/spec/unit/file_serving/metadata.rb +++ b/spec/unit/file_serving/metadata.rb @@ -1,134 +1,171 @@ #!/usr/bin/env ruby require File.dirname(__FILE__) + '/../../spec_helper' require 'puppet/file_serving/metadata' describe Puppet::FileServing::Metadata do it "should should be a subclass of FileBase" do Puppet::FileServing::Metadata.superclass.should equal(Puppet::FileServing::FileBase) end it "should indirect file_metadata" do Puppet::FileServing::Metadata.indirection.name.should == :file_metadata end it "should should include the IndirectionHooks module in its indirection" do Puppet::FileServing::Metadata.indirection.metaclass.included_modules.should include(Puppet::FileServing::IndirectionHooks) end end describe Puppet::FileServing::Metadata, " when finding the file to use for setting attributes" do before do @metadata = Puppet::FileServing::Metadata.new("my/path") @full = "/base/path/my/path" @metadata.path = @full - # Use a symlink because it's easier to test -- no checksumming - @stat = stub "stat", :uid => 10, :gid => 20, :mode => 0755, :ftype => "symlink" + # Use a link because it's easier to test -- no checksumming + @stat = stub "stat", :uid => 10, :gid => 20, :mode => 0755, :ftype => "link" end it "should accept a base path path to which the file should be relative" do File.expects(:lstat).with(@full).returns @stat File.expects(:readlink).with(@full).returns "/what/ever" @metadata.collect_attributes end it "should use the set base path if one is not provided" do File.expects(:lstat).with(@full).returns @stat File.expects(:readlink).with(@full).returns "/what/ever" @metadata.collect_attributes() end it "should fail if a base path is neither set nor provided" do proc { @metadata.collect_attributes() }.should raise_error(Errno::ENOENT) end it "should raise an exception if the file does not exist" do File.expects(:lstat).with(@full).raises(Errno::ENOENT) proc { @metadata.collect_attributes()}.should raise_error(Errno::ENOENT) end end describe Puppet::FileServing::Metadata, " when collecting attributes" do before do @path = "/my/file" - @stat = stub 'stat', :uid => 10, :gid => 20, :mode => 0755, :ftype => "file" + # Use a real file mode, so we can validate the masking is done. + @stat = stub 'stat', :uid => 10, :gid => 20, :mode => 33261, :ftype => "file" File.stubs(:lstat).returns(@stat) - @filehandle = mock 'filehandle' - #@filehandle.expects(:read).with(512).returns("some content\n").then.returns(nil) - File.stubs(:open).with(@path, 'r').yields(@filehandle) @checksum = Digest::MD5.hexdigest("some content\n") @metadata = Puppet::FileServing::Metadata.new("file", :path => "/my/file") - @metadata.expects(:md5_file).returns(@checksum) + @metadata.stubs(:md5_file).returns(@checksum) @metadata.collect_attributes end + it "should be able to produce xmlrpc-style attribute information" do + @metadata.should respond_to(:attributes_with_tabs) + end + # LAK:FIXME This should actually change at some point it "should set the owner by id" do @metadata.owner.should be_instance_of(Fixnum) end # LAK:FIXME This should actually change at some point it "should set the group by id" do @metadata.group.should be_instance_of(Fixnum) end it "should set the owner to the file's current owner" do @metadata.owner.should == 10 end it "should set the group to the file's current group" do @metadata.group.should == 20 end - it "should set the mode to a string version of the mode in octal" do - @metadata.mode.should == "755" - end - - it "should set the mode to the file's current mode" do - @metadata.mode.should == "755" + it "should set the mode to the file's masked mode" do + @metadata.mode.should == 0755 end it "should set the checksum to the file's current checksum" do @metadata.checksum.should == "{md5}" + @checksum end - it "should default to a checksum of type MD5" do - @metadata.checksum.should == "{md5}" + @checksum + describe "when managing files" do + it "should default to a checksum of type MD5" do + @metadata.checksum.should == "{md5}" + @checksum + end + + it "should produce tab-separated mode, type, owner, group, and checksum for xmlrpc" do + @metadata.attributes_with_tabs.should == "#{0755.to_s}\tfile\t10\t20\t{md5}#{@checksum}" + end + end + + describe "when managing directories" do + before do + @stat.stubs(:ftype).returns("directory") + @time = Time.now + @metadata.expects(:ctime_file).returns(@time) + @metadata.collect_attributes + end + + it "should only use checksums of type 'ctime' for directories" do + @metadata.checksum.should == "{ctime}" + @time.to_s + end + + it "should produce tab-separated mode, type, owner, group, and checksum for xmlrpc" do + @metadata.attributes_with_tabs.should == "#{0755.to_s}\tdirectory\t10\t20\t{ctime}#{@time.to_s}" + end + end + + describe "when managing links" do + before do + @stat.stubs(:ftype).returns("link") + File.expects(:readlink).with("/my/file").returns("/path/to/link") + @metadata.collect_attributes + end + + it "should read links instead of returning their checksums" do + @metadata.destination.should == "/path/to/link" + end + + it "should produce tab-separated mode, type, owner, group, and destination for xmlrpc" do + @metadata.attributes_with_tabs.should == "#{0755.to_s}\tlink\t10\t20\t/path/to/link" + end end end -describe Puppet::FileServing::Metadata, " when pointing to a symlink" do - it "should store the destination of the symlink in :destination if links are :manage" do +describe Puppet::FileServing::Metadata, " when pointing to a link" do + it "should store the destination of the link in :destination if links are :manage" do file = Puppet::FileServing::Metadata.new("mykey", :links => :manage, :path => "/base/path/my/file") - File.expects(:lstat).with("/base/path/my/file").returns stub("stat", :uid => 1, :gid => 2, :ftype => "symlink", :mode => 0755) + File.expects(:lstat).with("/base/path/my/file").returns stub("stat", :uid => 1, :gid => 2, :ftype => "link", :mode => 0755) File.expects(:readlink).with("/base/path/my/file").returns "/some/other/path" file.collect_attributes file.destination.should == "/some/other/path" end it "should not collect the checksum" do file = Puppet::FileServing::Metadata.new("my/file", :links => :manage, :path => "/base/path/my/file") - File.expects(:lstat).with("/base/path/my/file").returns stub("stat", :uid => 1, :gid => 2, :ftype => "symlink", :mode => 0755) + File.expects(:lstat).with("/base/path/my/file").returns stub("stat", :uid => 1, :gid => 2, :ftype => "link", :mode => 0755) File.expects(:readlink).with("/base/path/my/file").returns "/some/other/path" file.collect_attributes file.checksum.should be_nil end end describe Puppet::FileServing::Metadata, " when converting from yaml" do # LAK:FIXME This isn't in the right place, but we need some kind of # control somewhere that requires that all REST connections only pull # from the file-server, thus guaranteeing they go through our authorization # hook. it "should set the URI scheme to 'puppetmounts'" do pending "We need to figure out where this should be" end end diff --git a/spec/unit/node/catalog.rb b/spec/unit/node/catalog.rb index ecbd20487..b1bf5abaa 100755 --- a/spec/unit/node/catalog.rb +++ b/spec/unit/node/catalog.rb @@ -1,813 +1,829 @@ #!/usr/bin/env ruby require File.dirname(__FILE__) + '/../../spec_helper' describe Puppet::Node::Catalog, " when compiling" do it "should accept tags" do config = Puppet::Node::Catalog.new("mynode") config.tag("one") config.tags.should == %w{one} end it "should accept multiple tags at once" do config = Puppet::Node::Catalog.new("mynode") config.tag("one", "two") config.tags.should == %w{one two} end it "should convert all tags to strings" do config = Puppet::Node::Catalog.new("mynode") config.tag("one", :two) config.tags.should == %w{one two} end it "should tag with both the qualified name and the split name" do config = Puppet::Node::Catalog.new("mynode") config.tag("one::two") config.tags.include?("one").should be_true config.tags.include?("one::two").should be_true end it "should accept classes" do config = Puppet::Node::Catalog.new("mynode") config.add_class("one") config.classes.should == %w{one} config.add_class("two", "three") config.classes.should == %w{one two three} end it "should tag itself with passed class names" do config = Puppet::Node::Catalog.new("mynode") config.add_class("one") config.tags.should == %w{one} end end describe Puppet::Node::Catalog, " when extracting" do it "should return extraction result as the method result" do config = Puppet::Node::Catalog.new("mynode") config.expects(:extraction_format).returns(:whatever) config.expects(:extract_to_whatever).returns(:result) config.extract.should == :result end end describe Puppet::Node::Catalog, " when extracting transobjects" do def mkscope @parser = Puppet::Parser::Parser.new :Code => "" @node = Puppet::Node.new("mynode") @compiler = Puppet::Parser::Compiler.new(@node, @parser) # XXX This is ridiculous. @compiler.send(:evaluate_main) @scope = @compiler.topscope end def mkresource(type, name) Puppet::Parser::Resource.new(:type => type, :title => name, :source => @source, :scope => @scope) end it "should always create a TransBucket for the 'main' class" do config = Puppet::Node::Catalog.new("mynode") @scope = mkscope @source = mock 'source' main = mkresource("class", :main) config.add_vertex(main) bucket = mock 'bucket' bucket.expects(:classes=).with(config.classes) main.stubs(:builtin?).returns(false) main.expects(:to_transbucket).returns(bucket) config.extract_to_transportable.should equal(bucket) end # This isn't really a spec-style test, but I don't know how better to do it. it "should transform the resource graph into a tree of TransBuckets and TransObjects" do config = Puppet::Node::Catalog.new("mynode") @scope = mkscope @source = mock 'source' defined = mkresource("class", :main) builtin = mkresource("file", "/yay") config.add_edge(defined, builtin) bucket = [] bucket.expects(:classes=).with(config.classes) defined.stubs(:builtin?).returns(false) defined.expects(:to_transbucket).returns(bucket) builtin.expects(:to_transobject).returns(:builtin) config.extract_to_transportable.should == [:builtin] end # Now try it with a more complicated graph -- a three tier graph, each tier it "should transform arbitrarily deep graphs into isomorphic trees" do config = Puppet::Node::Catalog.new("mynode") @scope = mkscope @scope.stubs(:tags).returns([]) @source = mock 'source' # Create our scopes. top = mkresource "class", :main topbucket = [] topbucket.expects(:classes=).with([]) top.expects(:to_trans).returns(topbucket) topres = mkresource "file", "/top" topres.expects(:to_trans).returns(:topres) config.add_edge top, topres middle = mkresource "class", "middle" middle.expects(:to_trans).returns([]) config.add_edge top, middle midres = mkresource "file", "/mid" midres.expects(:to_trans).returns(:midres) config.add_edge middle, midres bottom = mkresource "class", "bottom" bottom.expects(:to_trans).returns([]) config.add_edge middle, bottom botres = mkresource "file", "/bot" botres.expects(:to_trans).returns(:botres) config.add_edge bottom, botres toparray = config.extract_to_transportable # This is annoying; it should look like: # [[[:botres], :midres], :topres] # but we can't guarantee sort order. toparray.include?(:topres).should be_true midarray = toparray.find { |t| t.is_a?(Array) } midarray.include?(:midres).should be_true botarray = midarray.find { |t| t.is_a?(Array) } botarray.include?(:botres).should be_true end end describe Puppet::Node::Catalog, " when converting to a transobject catalog" do class TestResource attr_accessor :name, :virtual, :builtin def initialize(name, options = {}) @name = name options.each { |p,v| send(p.to_s + "=", v) } end def ref if builtin? "File[%s]" % name else "Class[%s]" % name end end def virtual? virtual end def builtin? builtin end def to_transobject Puppet::TransObject.new(name, builtin? ? "file" : "class") end end before do @original = Puppet::Node::Catalog.new("mynode") @original.tag(*%w{one two three}) @original.add_class *%w{four five six} @top = TestResource.new 'top' @topobject = TestResource.new 'topobject', :builtin => true @virtual = TestResource.new 'virtual', :virtual => true @virtualobject = TestResource.new 'virtualobject', :builtin => true, :virtual => true @middle = TestResource.new 'middle' @middleobject = TestResource.new 'middleobject', :builtin => true @bottom = TestResource.new 'bottom' @bottomobject = TestResource.new 'bottomobject', :builtin => true @resources = [@top, @topobject, @middle, @middleobject, @bottom, @bottomobject] @original.add_edge(@top, @topobject) @original.add_edge(@top, @virtual) @original.add_edge(@virtual, @virtualobject) @original.add_edge(@top, @middle) @original.add_edge(@middle, @middleobject) @original.add_edge(@middle, @bottom) @original.add_edge(@bottom, @bottomobject) @catalog = @original.to_transportable end it "should add all resources as TransObjects" do @resources.each { |resource| @catalog.resource(resource.ref).should be_instance_of(Puppet::TransObject) } end it "should not extract defined virtual resources" do @catalog.vertices.find { |v| v.name == "virtual" }.should be_nil end it "should not extract builtin virtual resources" do @catalog.vertices.find { |v| v.name == "virtualobject" }.should be_nil end it "should copy the tag list to the new catalog" do @catalog.tags.sort.should == @original.tags.sort end it "should copy the class list to the new catalog" do @catalog.classes.should == @original.classes end it "should duplicate the original edges" do @original.edges.each do |edge| next if edge.source.virtual? or edge.target.virtual? source = @catalog.resource(edge.source.ref) target = @catalog.resource(edge.target.ref) source.should_not be_nil target.should_not be_nil @catalog.edge?(source, target).should be_true end end it "should set itself as the catalog for each converted resource" do @catalog.vertices.each { |v| v.catalog.object_id.should equal(@catalog.object_id) } end end describe Puppet::Node::Catalog, " when converting to a RAL catalog" do before do @original = Puppet::Node::Catalog.new("mynode") @original.tag(*%w{one two three}) @original.add_class *%w{four five six} @top = Puppet::TransObject.new 'top', "class" @topobject = Puppet::TransObject.new '/topobject', "file" @middle = Puppet::TransObject.new 'middle', "class" @middleobject = Puppet::TransObject.new '/middleobject', "file" @bottom = Puppet::TransObject.new 'bottom', "class" @bottomobject = Puppet::TransObject.new '/bottomobject', "file" @resources = [@top, @topobject, @middle, @middleobject, @bottom, @bottomobject] @original.add_resource(*@resources) @original.add_edge(@top, @topobject) @original.add_edge(@top, @middle) @original.add_edge(@middle, @middleobject) @original.add_edge(@middle, @bottom) @original.add_edge(@bottom, @bottomobject) @catalog = @original.to_ral end it "should add all resources as RAL instances" do @resources.each { |resource| @catalog.resource(resource.ref).should be_instance_of(Puppet::Type) } end it "should copy the tag list to the new catalog" do @catalog.tags.sort.should == @original.tags.sort end it "should copy the class list to the new catalog" do @catalog.classes.should == @original.classes end it "should duplicate the original edges" do @original.edges.each do |edge| @catalog.edge?(@catalog.resource(edge.source.ref), @catalog.resource(edge.target.ref)).should be_true end end it "should set itself as the catalog for each converted resource" do @catalog.vertices.each { |v| v.catalog.object_id.should equal(@catalog.object_id) } end # This tests #931. it "should not lose track of resources whose names vary" do changer = Puppet::TransObject.new 'changer', 'test' config = Puppet::Node::Catalog.new('test') config.add_resource(changer) config.add_resource(@top) config.add_edge(@top, changer) resource = stub 'resource', :name => "changer2", :title => "changer2", :ref => "Test[changer2]", :catalog= => nil, :remove => nil changer.expects(:to_type).returns(resource) newconfig = nil Puppet::Type.allclear proc { @catalog = config.to_ral }.should_not raise_error @catalog.resource("Test[changer2]").should equal(resource) end after do # Remove all resource instances. @catalog.clear(true) end end describe Puppet::Node::Catalog, " when functioning as a resource container" do before do @catalog = Puppet::Node::Catalog.new("host") @one = stub 'resource1', :ref => "Me[one]", :catalog= => nil @two = stub 'resource2', :ref => "Me[two]", :catalog= => nil @dupe = stub 'resource3', :ref => "Me[one]", :catalog= => nil end it "should provide a method to add one or more resources" do @catalog.add_resource @one, @two @catalog.resource(@one.ref).should equal(@one) @catalog.resource(@two.ref).should equal(@two) end it "should set itself as the resource's catalog if it is not a relationship graph" do @one.expects(:catalog=).with(@catalog) @catalog.add_resource @one end it "should not set itself as the resource's catalog if it is a relationship graph" do @one.expects(:catalog=).never @catalog.is_relationship_graph = true @catalog.add_resource @one end it "should make all vertices available by resource reference" do @catalog.add_resource(@one) @catalog.resource(@one.ref).should equal(@one) @catalog.vertices.find { |r| r.ref == @one.ref }.should equal(@one) end it "should canonize how resources are referred to during retrieval when both type and title are provided" do @catalog.add_resource(@one) @catalog.resource("me", "one").should equal(@one) end it "should canonize how resources are referred to during retrieval when just the title is provided" do @catalog.add_resource(@one) @catalog.resource("me[one]", nil).should equal(@one) end it "should not allow two resources with the same resource reference" do @catalog.add_resource(@one) # These are used to build the failure @dupe.stubs(:file) @dupe.stubs(:line) @one.stubs(:file) @one.stubs(:line) proc { @catalog.add_resource(@dupe) }.should raise_error(ArgumentError) end it "should not store objects that do not respond to :ref" do proc { @catalog.add_resource("thing") }.should raise_error(ArgumentError) end it "should remove all resources when asked" do @catalog.add_resource @one @catalog.add_resource @two @one.expects :remove @two.expects :remove @catalog.clear(true) end it "should support a mechanism for finishing resources" do @one.expects :finish @two.expects :finish @catalog.add_resource @one @catalog.add_resource @two @catalog.finalize end it "should make default resources when finalizing" do @catalog.expects(:make_default_resources) @catalog.finalize end it "should add default resources to the catalog upon creation" do @catalog.make_default_resources @catalog.resource(:schedule, "daily").should_not be_nil end it "should optionally support an initialization block and should finalize after such blocks" do @one.expects :finish @two.expects :finish config = Puppet::Node::Catalog.new("host") do |conf| conf.add_resource @one conf.add_resource @two end end it "should inform the resource that it is the resource's catalog" do @one.expects(:catalog=).with(@catalog) @catalog.add_resource @one end it "should be able to find resources by reference" do @catalog.add_resource @one @catalog.resource(@one.ref).should equal(@one) end it "should be able to find resources by reference or by type/title tuple" do @catalog.add_resource @one @catalog.resource("me", "one").should equal(@one) end it "should have a mechanism for removing resources" do @catalog.add_resource @one @one.expects :remove @catalog.remove_resource(@one) @catalog.resource(@one.ref).should be_nil @catalog.vertex?(@one).should be_false end it "should have a method for creating aliases for resources" do @catalog.add_resource @one @catalog.alias(@one, "other") @catalog.resource("me", "other").should equal(@one) end # This test is the same as the previous, but the behaviour should be explicit. it "should alias using the class name from the resource reference, not the resource class name" do @catalog.add_resource @one @catalog.alias(@one, "other") @catalog.resource("me", "other").should equal(@one) end it "should fail to add an alias if the aliased name already exists" do @catalog.add_resource @one proc { @catalog.alias @two, "one" }.should raise_error(ArgumentError) end it "should not fail when a resource has duplicate aliases created" do @catalog.add_resource @one proc { @catalog.alias @one, "one" }.should_not raise_error end + it "should be able to look resources up by their aliases" do + @catalog.add_resource @one + @catalog.alias @one, "two" + @catalog.resource(:me, "two").should equal(@one) + end + it "should remove resource aliases when the target resource is removed" do @catalog.add_resource @one @catalog.alias(@one, "other") @one.expects :remove @catalog.remove_resource(@one) @catalog.resource("me", "other").should be_nil end - it "should add an alias for the namevar when the title and name differ" do - @one.stubs(:name).returns "other" + it "should add an alias for the namevar when the title and name differ on isomorphic resource types" do resource = Puppet::Type.type(:file).create :path => "/something", :title => "other", :content => "blah" @catalog.add_resource(resource) @catalog.resource(:file, "other").should equal(resource) - @catalog.resource(:file, "/something").should equal(resource) + @catalog.resource(:file, "/something").ref.should == resource.ref + end + + it "should not add an alias for the namevar when the title and name differ on non-isomorphic resource types" do + resource = Puppet::Type.type(:exec).create :command => "/bin/true", :title => "other" + @catalog.add_resource(resource) + @catalog.resource(:exec, resource.title).should equal(resource) + # We can't use .should here, because the resources respond to that method. + if @catalog.resource(:exec, resource.name) + raise "Aliased non-isomorphic resource" + end end after do Puppet::Type.allclear end end describe Puppet::Node::Catalog do before :each do @catalog = Puppet::Node::Catalog.new("host") @catalog.retrieval_duration = Time.now @transaction = mock 'transaction' Puppet::Transaction.stubs(:new).returns(@transaction) @transaction.stubs(:evaluate) @transaction.stubs(:cleanup) @transaction.stubs(:addtimes) end describe Puppet::Node::Catalog, " when applying" do it "should create and evaluate a transaction" do @transaction.expects(:evaluate) @catalog.apply end it "should provide the catalog time to the transaction" do @transaction.expects(:addtimes).with do |arg| arg[:config_retrieval].should be_instance_of(Time) true end @catalog.apply end it "should clean up the transaction" do @transaction.expects :cleanup @catalog.apply end it "should return the transaction" do @catalog.apply.should equal(@transaction) end it "should yield the transaction if a block is provided" do @catalog.apply do |trans| trans.should equal(@transaction) end end it "should default to not being a host catalog" do @catalog.host_config.should be_nil end it "should pass supplied tags on to the transaction" do @transaction.expects(:tags=).with(%w{one two}) @catalog.apply(:tags => %w{one two}) end it "should set ignoreschedules on the transaction if specified in apply()" do @transaction.expects(:ignoreschedules=).with(true) @catalog.apply(:ignoreschedules => true) end end describe Puppet::Node::Catalog, " when applying host catalogs" do # super() doesn't work in the setup method for some reason before do @catalog.host_config = true end it "should send a report if reporting is enabled" do Puppet[:report] = true @transaction.expects :send_report @transaction.stubs :any_failed? => false @catalog.apply end it "should send a report if report summaries are enabled" do Puppet[:summarize] = true @transaction.expects :send_report @transaction.stubs :any_failed? => false @catalog.apply end it "should initialize the state database before applying a catalog" do Puppet::Util::Storage.expects(:load) # Short-circuit the apply, so we know we're loading before the transaction Puppet::Transaction.expects(:new).raises ArgumentError proc { @catalog.apply }.should raise_error(ArgumentError) end it "should sync the state database after applying" do Puppet::Util::Storage.expects(:store) @transaction.stubs :any_failed? => false @catalog.apply end after { Puppet.settings.clear } end describe Puppet::Node::Catalog, " when applying non-host catalogs" do before do @catalog.host_config = false end it "should never send reports" do Puppet[:report] = true Puppet[:summarize] = true @transaction.expects(:send_report).never @catalog.apply end it "should never modify the state database" do Puppet::Util::Storage.expects(:load).never Puppet::Util::Storage.expects(:store).never @catalog.apply end after { Puppet.settings.clear } end end describe Puppet::Node::Catalog, " when creating a relationship graph" do before do + Puppet::Type.type(:component) @catalog = Puppet::Node::Catalog.new("host") @compone = Puppet::Type::Component.create :name => "one" @comptwo = Puppet::Type::Component.create :name => "two", :require => ["class", "one"] @file = Puppet::Type.type(:file) @one = @file.create :path => "/one" @two = @file.create :path => "/two" @catalog.add_edge @compone, @one @catalog.add_edge @comptwo, @two @three = @file.create :path => "/three" @four = @file.create :path => "/four", :require => ["file", "/three"] @five = @file.create :path => "/five" @catalog.add_resource @compone, @comptwo, @one, @two, @three, @four, @five @relationships = @catalog.relationship_graph end it "should fail when trying to create a relationship graph for a relationship graph" do proc { @relationships.relationship_graph }.should raise_error(Puppet::DevError) end it "should be able to create a relationship graph" do @relationships.should be_instance_of(Puppet::Node::Catalog) end it "should copy its host_config setting to the relationship graph" do config = Puppet::Node::Catalog.new config.host_config = true config.relationship_graph.host_config.should be_true end it "should not have any components" do @relationships.vertices.find { |r| r.instance_of?(Puppet::Type::Component) }.should be_nil end it "should have all non-component resources from the catalog" do # The failures print out too much info, so i just do a class comparison @relationships.vertex?(@five).should be_true end it "should have all resource relationships set as edges" do @relationships.edge?(@three, @four).should be_true end it "should copy component relationships to all contained resources" do @relationships.edge?(@one, @two).should be_true end it "should get removed when the catalog is cleaned up" do @relationships.expects(:clear).with(false) @catalog.clear @catalog.instance_variable_get("@relationship_graph").should be_nil end it "should create a new relationship graph after clearing the old one" do @relationships.expects(:clear).with(false) @catalog.clear @catalog.relationship_graph.should be_instance_of(Puppet::Node::Catalog) end it "should look up resources in the relationship graph if not found in the main catalog" do five = stub 'five', :ref => "File[five]", :catalog= => nil @relationships.add_resource five @catalog.resource(five.ref).should equal(five) end it "should provide a method to create additional resources that also registers the resource" do args = {:name => "/yay", :ensure => :file} resource = stub 'file', :ref => "File[/yay]", :catalog= => @catalog Puppet::Type.type(:file).expects(:create).with(args).returns(resource) @catalog.create_resource :file, args @catalog.resource("File[/yay]").should equal(resource) end it "should provide a mechanism for creating implicit resources" do args = {:name => "/yay", :ensure => :file} resource = stub 'file', :ref => "File[/yay]", :catalog= => @catalog Puppet::Type.type(:file).expects(:create).with(args).returns(resource) resource.expects(:implicit=).with(true) @catalog.create_implicit_resource :file, args @catalog.resource("File[/yay]").should equal(resource) end it "should add implicit resources to the relationship graph if there is one" do args = {:name => "/yay", :ensure => :file} resource = stub 'file', :ref => "File[/yay]", :catalog= => @catalog resource.expects(:implicit=).with(true) Puppet::Type.type(:file).expects(:create).with(args).returns(resource) # build the graph relgraph = @catalog.relationship_graph @catalog.create_implicit_resource :file, args relgraph.resource("File[/yay]").should equal(resource) end it "should remove resources created mid-transaction" do args = {:name => "/yay", :ensure => :file} resource = stub 'file', :ref => "File[/yay]", :catalog= => @catalog @transaction = mock 'transaction' Puppet::Transaction.stubs(:new).returns(@transaction) @transaction.stubs(:evaluate) @transaction.stubs(:cleanup) @transaction.stubs(:addtimes) Puppet::Type.type(:file).expects(:create).with(args).returns(resource) resource.expects :remove @catalog.apply do |trans| @catalog.create_resource :file, args @catalog.resource("File[/yay]").should equal(resource) end @catalog.resource("File[/yay]").should be_nil end it "should remove resources from the relationship graph if it exists" do @catalog.remove_resource(@one) @catalog.relationship_graph.vertex?(@one).should be_false end after do Puppet::Type.allclear end end describe Puppet::Node::Catalog, " when writing dot files" do before do @catalog = Puppet::Node::Catalog.new("host") @name = :test @file = File.join(Puppet[:graphdir], @name.to_s + ".dot") end it "should only write when it is a host catalog" do File.expects(:open).with(@file).never @catalog.host_config = false Puppet[:graph] = true @catalog.write_graph(@name) end it "should only write when graphing is enabled" do File.expects(:open).with(@file).never @catalog.host_config = true Puppet[:graph] = false @catalog.write_graph(@name) end it "should write a dot file based on the passed name" do File.expects(:open).with(@file, "w").yields(stub("file", :puts => nil)) @catalog.expects(:to_dot).with("name" => @name.to_s.capitalize) @catalog.host_config = true Puppet[:graph] = true @catalog.write_graph(@name) end after do Puppet.settings.clear end end describe Puppet::Node::Catalog, " when indirecting" do before do @indirection = mock 'indirection' Puppet::Indirector::Indirection.clear_cache end it "should redirect to the indirection for retrieval" do Puppet::Node::Catalog.stubs(:indirection).returns(@indirection) @indirection.expects(:find).with(:myconfig) Puppet::Node::Catalog.find(:myconfig) end it "should default to the 'compiler' terminus" do Puppet::Node::Catalog.indirection.terminus_class.should == :compiler end after do mocha_verify Puppet::Indirector::Indirection.clear_cache end end describe Puppet::Node::Catalog, " when converting to yaml" do before do @catalog = Puppet::Node::Catalog.new("me") @catalog.add_edge("one", "two") end it "should be able to be dumped to yaml" do YAML.dump(@catalog).should be_instance_of(String) end end describe Puppet::Node::Catalog, " when converting from yaml" do before do @catalog = Puppet::Node::Catalog.new("me") @catalog.add_edge("one", "two") text = YAML.dump(@catalog) @newcatalog = YAML.load(text) end it "should get converted back to a catalog" do @newcatalog.should be_instance_of(Puppet::Node::Catalog) end it "should have all vertices" do @newcatalog.vertex?("one").should be_true @newcatalog.vertex?("two").should be_true end it "should have all edges" do @newcatalog.edge?("one", "two").should be_true end end diff --git a/spec/unit/other/pgraph.rb b/spec/unit/other/pgraph.rb index 7d66ae331..10ab934a6 100755 --- a/spec/unit/other/pgraph.rb +++ b/spec/unit/other/pgraph.rb @@ -1,209 +1,210 @@ #!/usr/bin/env ruby # # Created by Luke Kanies on 2007-9-12. # Copyright (c) 2006. All rights reserved. require File.dirname(__FILE__) + '/../../spec_helper' +require 'puppet/pgraph' require 'puppet/util/graph' class Container include Puppet::Util::Graph include Enumerable attr_accessor :name def each @children.each do |c| yield c end end def initialize(name, ary) @name = name @children = ary end def push(*ary) ary.each { |c| @children.push(c)} end def to_s @name end end describe Puppet::PGraph do before do @graph = Puppet::PGraph.new end it "should correctly clear vertices and edges when asked" do @graph.add_edge("a", "b") @graph.add_vertex "c" @graph.clear @graph.vertices.should be_empty @graph.edges.should be_empty end end describe Puppet::PGraph, " when matching edges" do before do @graph = Puppet::PGraph.new @event = Puppet::Event.new(:source => "a", :event => :yay) @none = Puppet::Event.new(:source => "a", :event => :NONE) @edges = {} @edges["a/b"] = Puppet::Relationship.new("a", "b", {:event => :yay, :callback => :refresh}) @edges["a/c"] = Puppet::Relationship.new("a", "c", {:event => :yay, :callback => :refresh}) @graph.add_edge(@edges["a/b"]) end it "should match edges whose source matches the source of the event" do @graph.matching_edges([@event]).should == [@edges["a/b"]] end it "should match always match nothing when the event is :NONE" do @graph.matching_edges([@none]).should be_empty end it "should match multiple edges" do @graph.add_edge(@edges["a/c"]) edges = @graph.matching_edges([@event]) edges.should be_include(@edges["a/b"]) edges.should be_include(@edges["a/c"]) end end describe Puppet::PGraph, " when determining dependencies" do before do @graph = Puppet::PGraph.new @graph.add_edge("a", "b") @graph.add_edge("a", "c") @graph.add_edge("b", "d") end it "should find all dependents when they are on multiple levels" do @graph.dependents("a").sort.should == %w{b c d}.sort end it "should find single dependents" do @graph.dependents("b").sort.should == %w{d}.sort end it "should return an empty array when there are no dependents" do @graph.dependents("c").sort.should == [].sort end it "should find all dependencies when they are on multiple levels" do @graph.dependencies("d").sort.should == %w{a b} end it "should find single dependencies" do @graph.dependencies("c").sort.should == %w{a} end it "should return an empty array when there are no dependencies" do @graph.dependencies("a").sort.should == [] end end describe Puppet::PGraph, " when splicing the relationship graph" do def container_graph @one = Container.new("one", %w{a b}) @two = Container.new("two", ["c", "d"]) @three = Container.new("three", ["i", "j"]) @middle = Container.new("middle", ["e", "f", @two]) @top = Container.new("top", ["g", "h", @middle, @one, @three]) @empty = Container.new("empty", []) @contgraph = @top.to_graph # We have to add the container to the main graph, else it won't # be spliced in the dependency graph. @contgraph.add_vertex(@empty) end def dependency_graph @depgraph = Puppet::PGraph.new @contgraph.vertices.each do |v| @depgraph.add_vertex(v) end # We have to specify a relationship to our empty container, else it # never makes it into the dep graph in the first place. {@one => @two, "f" => "c", "h" => @middle, "c" => @empty}.each do |source, target| @depgraph.add_edge(source, target, :callback => :refresh) end end def splice @depgraph.splice!(@contgraph, Container) end before do container_graph dependency_graph splice end # This is the real heart of splicing -- replacing all containers in # our relationship and exploding their relationships so that each # relationship to a container gets copied to all of its children. it "should remove all Container objects from the dependency graph" do @depgraph.vertices.find_all { |v| v.is_a?(Container) }.should be_empty end it "should add container relationships to contained objects" do @contgraph.leaves(@middle).each do |leaf| @depgraph.should be_edge("h", leaf) end end it "should explode container-to-container relationships, making edges between all respective contained objects" do @one.each do |oobj| @two.each do |tobj| @depgraph.should be_edge(oobj, tobj) end end end it "should no longer contain anything but the non-container objects" do @depgraph.vertices.find_all { |v| ! v.is_a?(String) }.should be_empty end it "should copy labels" do @depgraph.edges.each do |edge| edge.label.should == {:callback => :refresh} end end it "should not add labels to edges that have none" do @depgraph.add_edge(@two, @three) splice @depgraph.edge_label("c", "i").should == {} end it "should copy labels over edges that have none" do @depgraph.add_edge("c", @three, {:callback => :refresh}) splice # And make sure the label got copied. @depgraph.edge_label("c", "i").should == {:callback => :refresh} end it "should not replace a label with a nil label" do # Lastly, add some new label-less edges and make sure the label stays. @depgraph.add_edge(@middle, @three) @depgraph.add_edge("c", @three, {:callback => :refresh}) splice @depgraph.edge_label("c", "i").should == {:callback => :refresh} end it "should copy labels to all created edges" do @depgraph.add_edge(@middle, @three) @depgraph.add_edge("c", @three, {:callback => :refresh}) splice @three.each do |child| edge = Puppet::Relationship.new("c", child) @depgraph.should be_edge(edge.source, edge.target) @depgraph.edge_label(edge.source, edge.target).should == {:callback => :refresh} end end end diff --git a/spec/unit/other/transaction.rb b/spec/unit/other/transaction.rb index d88f03005..e277a24c0 100755 --- a/spec/unit/other/transaction.rb +++ b/spec/unit/other/transaction.rb @@ -1,26 +1,28 @@ #!/usr/bin/env ruby require File.dirname(__FILE__) + '/../../spec_helper' +require 'puppet/transaction' + describe Puppet::Transaction, " when determining tags" do before do @config = Puppet::Node::Catalog.new @transaction = Puppet::Transaction.new(@config) end it "should default to the tags specified in the :tags setting" do Puppet.expects(:[]).with(:tags).returns("one") @transaction.tags.should == %w{one} end it "should split tags based on ','" do Puppet.expects(:[]).with(:tags).returns("one,two") @transaction.tags.should == %w{one two} end it "should use any tags set after creation" do Puppet.expects(:[]).with(:tags).never @transaction.tags = %w{one two} @transaction.tags.should == %w{one two} end end diff --git a/spec/unit/ral/types/file.rb b/spec/unit/ral/types/file.rb index 62fe2f677..b213987bb 100755 --- a/spec/unit/ral/types/file.rb +++ b/spec/unit/ral/types/file.rb @@ -1,62 +1,99 @@ #!/usr/bin/env ruby require File.dirname(__FILE__) + '/../../../spec_helper' require 'puppet/type/file' describe Puppet::Type::File do before do @path = Tempfile.new("puppetspec") @path.close!() @path = @path.path @file = Puppet::Type::File.create(:name => @path) end describe "when used with content and replace=>false" do before do @file[:content] = "foo" @file[:replace] = false end it "should be insync if the file exists and the content is different" do File.open(@path, "w") do |f| f.puts "bar" end @file.property(:content).insync?("bar").should be_true end it "should be insync if the file exists and the content is right" do File.open(@path, "w") do |f| f.puts "foo" end @file.property(:content).insync?("foo").should be_true end it "should not be insync if the file does not exist" do @file.property(:content).insync?(:nil).should be_false end end describe "when retrieving remote files" do before do @filesource = Puppet::Type::File::FileSource.new @filesource.server = mock 'fileserver' @file.stubs(:uri2obj).returns(@filesource) @file[:source] = "puppet:///test" end it "should fail without writing if it cannot retrieve remote contents" do # create the file, because we only get the problem when it starts # out absent. File.open(@file[:path], "w") { |f| f.puts "a" } @file.expects(:write).never @filesource.server.stubs(:describe).returns("493\tfile\t100\t0\t{md5}3f5fef3bddbc4398c46a7bd7ba7b3af7") @filesource.server.stubs(:retrieve).raises(RuntimeError) @file.property(:source).retrieve lambda { @file.property(:source).sync }.should raise_error(Puppet::Error) end end + describe "when managing links" do + require 'puppettest/support/assertions' + include PuppetTest + + before do + @basedir = tempfile() + Dir.mkdir(@basedir) + @file = File.join(@basedir, "file") + @link = File.join(@basedir, "link") + + File.open(@file, "w", 0644) { |f| f.puts "yayness"; f.flush } + File.symlink(@file, @link) + + @resource = Puppet.type(:file).create( + :path => @link, + :mode => "755" + ) + end + + after do + teardown + end + + it "should default to managing the link" do + assert_events([], @resource) + # I convert them to strings so they display correctly if there's an error. + ("%o" % (File.stat(@file).mode & 007777)).should == "%o" % 0644 + end + + it "should be able to follow links" do + @resource[:links] = :follow + assert_events([:file_changed], @resource) + + ("%o" % (File.stat(@file).mode & 007777)).should == "%o" % 0755 + end + end + after do Puppet::Type::File.clear end end diff --git a/test/lib/puppettest.rb b/test/lib/puppettest.rb index 902831e68..e276bdf0f 100755 --- a/test/lib/puppettest.rb +++ b/test/lib/puppettest.rb @@ -1,335 +1,339 @@ # Add .../test/lib testlib = File.expand_path(File.dirname(__FILE__)) $LOAD_PATH.unshift(testlib) unless $LOAD_PATH.include?(testlib) # Add .../lib mainlib = File.expand_path(File.join(File.dirname(__FILE__), '../../lib')) $LOAD_PATH.unshift(mainlib) unless $LOAD_PATH.include?(mainlib) require 'puppet' # include any gems in vendor/gems Dir["#{mainlib}/../vendor/gems/**"].each do |path| libpath = File.join(path, "lib") if File.directory?(libpath) $LOAD_PATH.unshift(libpath) else $LOAD_PATH.unshift(path) end end require 'mocha' # Only load the test/unit class if we're not in the spec directory. # Else we get the bogus 'no tests, no failures' message. unless Dir.getwd =~ /spec/ require 'test/unit' end # Yay; hackish but it works if ARGV.include?("-d") ARGV.delete("-d") $console = true end # Some monkey-patching to allow us to test private methods. class Class def publicize_methods(*methods) saved_private_instance_methods = methods.empty? ? self.private_instance_methods : methods - self.class_eval { public *saved_private_instance_methods } + self.class_eval { public(*saved_private_instance_methods) } yield - self.class_eval { private *saved_private_instance_methods } + self.class_eval { private(*saved_private_instance_methods) } end end module PuppetTest # Munge cli arguments, so we can enable debugging if we want # and so we can run just specific methods. def self.munge_argv require 'getoptlong' result = GetoptLong.new( [ "--debug", "-d", GetoptLong::NO_ARGUMENT ], [ "--resolve", "-r", GetoptLong::REQUIRED_ARGUMENT ], [ "-n", GetoptLong::REQUIRED_ARGUMENT ], [ "--help", "-h", GetoptLong::NO_ARGUMENT ] ) usage = "USAGE: TESTOPTS='[-n -n ...] [-d]' rake [target] [target] ..." opts = [] dir = method = nil result.each { |opt,arg| case opt when "--resolve" dir, method = arg.split(",") when "--debug" $puppet_debug = true Puppet::Util::Log.level = :debug Puppet::Util::Log.newdestination(:console) when "--help" puts usage exit else opts << opt << arg end } suites = nil args = ARGV.dup # Reset the options, so the test suite can deal with them (this is # what makes things like '-n' work). opts.each { |o| ARGV << o } return args end # Find the root of the Puppet tree; this is not the test directory, but # the parent of that dir. def basedir(*list) unless defined? @@basedir Dir.chdir(File.dirname(__FILE__)) do @@basedir = File.dirname(File.dirname(Dir.getwd)) end end if list.empty? @@basedir else File.join(@@basedir, *list) end end def datadir(*list) File.join(basedir, "test", "data", *list) end def exampledir(*args) unless defined? @@exampledir @@exampledir = File.join(basedir, "examples") end if args.empty? return @@exampledir else return File.join(@@exampledir, *args) end end module_function :basedir, :datadir, :exampledir def cleanup(&block) @@cleaners << block end # Rails clobbers RUBYLIB, thanks def libsetup curlibs = ENV["RUBYLIB"].split(":") $:.reject do |dir| dir =~ /^\/usr/ end.each do |dir| unless curlibs.include?(dir) curlibs << dir end end ENV["RUBYLIB"] = curlibs.join(":") end def logcollector collector = [] Puppet::Util::Log.newdestination(collector) cleanup do Puppet::Util::Log.close(collector) end collector end def rake? $0 =~ /test_loader/ end # Redirect stdout and stderr def redirect @stderr = tempfile @stdout = tempfile $stderr = File.open(@stderr, "w") $stdout = File.open(@stdout, "w") cleanup do $stderr = STDERR $stdout = STDOUT end end def setup @memoryatstart = Puppet::Util.memory if defined? @@testcount @@testcount += 1 else @@testcount = 0 end @configpath = File.join(tmpdir, "configdir" + @@testcount.to_s + "/" ) unless defined? $user and $group $user = nonrootuser().uid.to_s $group = nonrootgroup().gid.to_s end Puppet.settings.clear Puppet[:user] = $user Puppet[:group] = $group Puppet[:confdir] = @configpath Puppet[:vardir] = @configpath unless File.exists?(@configpath) Dir.mkdir(@configpath) end @@tmpfiles = [@configpath, tmpdir()] @@tmppids = [] @@cleaners = [] @logs = [] # If we're running under rake, then disable debugging and such. #if rake? or ! Puppet[:debug] - if defined?($puppet_debug) or ! rake? + #if defined?($puppet_debug) or ! rake? if textmate? Puppet[:color] = false end Puppet::Util::Log.newdestination(@logs) if defined? $console Puppet.info @method_name Puppet::Util::Log.newdestination(:console) Puppet[:trace] = true end Puppet::Util::Log.level = :debug #$VERBOSE = 1 - else - Puppet::Util::Log.close - Puppet::Util::Log.newdestination(@logs) - Puppet[:httplog] = tempfile() - end + #else + # Puppet::Util::Log.close + # Puppet::Util::Log.newdestination(@logs) + # Puppet[:httplog] = tempfile() + #end Puppet[:ignoreschedules] = true #@start = Time.now end def tempfile if defined? @@tmpfilenum @@tmpfilenum += 1 else @@tmpfilenum = 1 end f = File.join(self.tmpdir(), "tempfile_" + @@tmpfilenum.to_s) @@tmpfiles << f return f end def textmate? if ENV["TM_FILENAME"] return true else return false end end def tstdir dir = tempfile() Dir.mkdir(dir) return dir end def tmpdir unless defined? @tmpdir and @tmpdir @tmpdir = case Facter["operatingsystem"].value when "Darwin": "/private/tmp" when "SunOS": "/var/tmp" else "/tmp" end @tmpdir = File.join(@tmpdir, "puppettesting" + Process.pid.to_s) unless File.exists?(@tmpdir) FileUtils.mkdir_p(@tmpdir) File.chmod(01777, @tmpdir) end end @tmpdir end - def teardown - #@stop = Time.now - #File.open("/tmp/test_times.log", ::File::WRONLY|::File::CREAT|::File::APPEND) { |f| f.puts "%0.4f %s %s" % [@stop - @start, @method_name, self.class] } - @@cleaners.each { |cleaner| cleaner.call() } - + def remove_tmp_files @@tmpfiles.each { |file| unless file =~ /tmp/ puts "Not deleting tmpfile %s" % file next end if FileTest.exists?(file) system("chmod -R 755 %s" % file) system("rm -rf %s" % file) end } @@tmpfiles.clear + end + + def teardown + #@stop = Time.now + #File.open("/tmp/test_times.log", ::File::WRONLY|::File::CREAT|::File::APPEND) { |f| f.puts "%0.4f %s %s" % [@stop - @start, @method_name, self.class] } + @@cleaners.each { |cleaner| cleaner.call() } + + remove_tmp_files @@tmppids.each { |pid| %x{kill -INT #{pid} 2>/dev/null} } @@tmppids.clear Puppet::Type.allclear Puppet::Util::Storage.clear Puppet.clear Puppet.settings.clear Puppet::Indirector::Indirection.clear_cache @memoryatend = Puppet::Util.memory diff = @memoryatend - @memoryatstart if diff > 1000 Puppet.info "%s#%s memory growth (%s to %s): %s" % [self.class, @method_name, @memoryatstart, @memoryatend, diff] end # reset all of the logs Puppet::Util::Log.close @logs.clear # Just in case there are processes waiting to die... require 'timeout' begin Timeout::timeout(5) do Process.waitall end rescue Timeout::Error # just move on end end def logstore @logs = [] Puppet::Util::Log.newdestination(@logs) end end require 'puppettest/support' require 'puppettest/filetesting' require 'puppettest/fakes' require 'puppettest/exetest' require 'puppettest/parsertesting' require 'puppettest/servertest' require 'puppettest/testcase' diff --git a/test/network/handler/fileserver.rb b/test/network/handler/fileserver.rb index 233e705c6..a705dbf4b 100755 --- a/test/network/handler/fileserver.rb +++ b/test/network/handler/fileserver.rb @@ -1,1173 +1,1175 @@ #!/usr/bin/env ruby require File.dirname(__FILE__) + '/../../lib/puppettest' require 'puppettest' require 'puppet/network/handler/fileserver' class TestFileServer < Test::Unit::TestCase include PuppetTest def mkmount(path = nil) mount = nil name = "yaytest" base = path || tempfile() unless FileTest.exists?(base) Dir.mkdir(base) end # Create a test file File.open(File.join(base, "file"), "w") { |f| f.puts "bazoo" } assert_nothing_raised { mount = Puppet::Network::Handler.fileserver::Mount.new(name, base) } return mount end # make a simple file source def mktestdir testdir = File.join(tmpdir(), "remotefilecopytesting") @@tmpfiles << testdir # create a tmpfile pattern = "tmpfile" tmpfile = File.join(testdir, pattern) assert_nothing_raised { Dir.mkdir(testdir) File.open(tmpfile, "w") { |f| 3.times { f.puts rand(100) } } } return [testdir, %r{#{pattern}}, tmpfile] end # make a bunch of random test files def mktestfiles(testdir) @@tmpfiles << testdir assert_nothing_raised { files = %w{a b c d e}.collect { |l| name = File.join(testdir, "file%s" % l) File.open(name, "w") { |f| f.puts rand(100) } name } return files } end def assert_describe(base, file, server) file = File.basename(file) assert_nothing_raised { desc = server.describe(base + file) assert(desc, "Got no description for %s" % file) assert(desc != "", "Got no description for %s" % file) assert_match(/^\d+/, desc, "Got invalid description %s" % desc) } end # test for invalid names def test_namefailures server = nil assert_nothing_raised { server = Puppet::Network::Handler.fileserver.new( :Local => true, :Config => false ) } [" ", "=" "+", "&", "#", "*"].each do |char| assert_raise(Puppet::Network::Handler::FileServerError, "'%s' did not throw a failure in fileserver module names" % char) { server.mount("/tmp", "invalid%sname" % char) } end end # verify that listing the root behaves as expected def test_listroot server = nil testdir, pattern, tmpfile = mktestdir() file = nil checks = Puppet::Network::Handler.fileserver::CHECKPARAMS # and make our fileserver assert_nothing_raised { server = Puppet::Network::Handler.fileserver.new( :Local => true, :Config => false ) } # mount the testdir assert_nothing_raised { server.mount(testdir, "test") } # and verify different iterations of 'root' return the same value list = nil assert_nothing_raised { - list = server.list("/test/", :ignore, true, false) + list = server.list("/test/", :manage, true, false) } assert(list =~ pattern) assert_nothing_raised { - list = server.list("/test", :ignore, true, false) + list = server.list("/test", :manage, true, false) } assert(list =~ pattern) end # test listing individual files def test_getfilelist server = nil testdir, pattern, tmpfile = mktestdir() file = nil assert_nothing_raised { server = Puppet::Network::Handler.fileserver.new( :Local => true, :Config => false ) } assert_nothing_raised { server.mount(testdir, "test") } # get our listing list = nil sfile = "/test/tmpfile" assert_nothing_raised { - list = server.list(sfile, :ignore, true, false) + list = server.list(sfile, :manage, true, false) } assert_nothing_raised { file = Puppet.type(:file)[tmpfile] } output = "/\tfile" # verify it got listed as a file assert_equal(output, list) # verify we got all fields assert(list !~ /\t\t/) # verify that we didn't get the directory itself list.split("\n").each { |line| assert(line !~ %r{remotefile}) } # and then verify that the contents match contents = File.read(tmpfile) ret = nil assert_nothing_raised { ret = server.retrieve(sfile) } assert_equal(contents, ret) end # check that the fileserver is seeing newly created files def test_seenewfiles server = nil testdir, pattern, tmpfile = mktestdir() newfile = File.join(testdir, "newfile") # go through the whole schtick again... file = nil checks = Puppet::Network::Handler.fileserver::CHECKPARAMS assert_nothing_raised { server = Puppet::Network::Handler.fileserver.new( :Local => true, :Config => false ) } assert_nothing_raised { server.mount(testdir, "test") } list = nil sfile = "/test/" assert_nothing_raised { - list = server.list(sfile, :ignore, true, false) + list = server.list(sfile, :manage, true, false) } # create the new file File.open(newfile, "w") { |f| 3.times { f.puts rand(100) } } newlist = nil assert_nothing_raised { - newlist = server.list(sfile, :ignore, true, false) + newlist = server.list(sfile, :manage, true, false) } # verify the list has changed assert(list != newlist) # and verify that we are specifically seeing the new file assert(newlist =~ /newfile/) end # verify we can mount /, which is what local file servers will # normally do def test_mountroot server = nil assert_nothing_raised { server = Puppet::Network::Handler.fileserver.new( :Local => true, :Config => false ) } assert_nothing_raised { server.mount("/", "root") } testdir, pattern, tmpfile = mktestdir() list = nil assert_nothing_raised { - list = server.list("/root/" + testdir, :ignore, true, false) + list = server.list("/root/" + testdir, :manage, true, false) } assert(list =~ pattern) assert_nothing_raised { - list = server.list("/root" + testdir, :ignore, true, false) + list = server.list("/root" + testdir, :manage, true, false) } assert(list =~ pattern) end # verify that we're correctly recursing the right number of levels def test_recursionlevels server = nil assert_nothing_raised { server = Puppet::Network::Handler.fileserver.new( :Local => true, :Config => false ) } # make our deep recursion basedir = File.join(tmpdir(), "recurseremotetesting") testdir = "%s/with/some/sub/directories/for/the/purposes/of/testing" % basedir oldfile = File.join(testdir, "oldfile") assert_nothing_raised { system("mkdir -p %s" % testdir) File.open(oldfile, "w") { |f| 3.times { f.puts rand(100) } } @@tmpfiles << basedir } assert_nothing_raised { server.mount(basedir, "test") } # get our list list = nil assert_nothing_raised { - list = server.list("/test/with", :ignore, false, false) + list = server.list("/test/with", :manage, false, false) } # make sure we only got one line, since we're not recursing assert(list !~ /\n/) # for each level of recursion, make sure we get the right list [0, 1, 2].each { |num| assert_nothing_raised { - list = server.list("/test/with", :ignore, num, false) + list = server.list("/test/with", :manage, num, false) } count = 0 while list =~ /\n/ list.sub!(/\n/, '') count += 1 end assert_equal(num, count) } end # verify that we're not seeing the dir we ask for; i.e., that our # list is relative to that dir, not it's parent dir def test_listedpath server = nil assert_nothing_raised { server = Puppet::Network::Handler.fileserver.new( :Local => true, :Config => false ) } # create a deep dir basedir = tempfile() testdir = "%s/with/some/sub/directories/for/testing" % basedir oldfile = File.join(testdir, "oldfile") assert_nothing_raised { system("mkdir -p %s" % testdir) File.open(oldfile, "w") { |f| 3.times { f.puts rand(100) } } @@tmpfiles << basedir } # mounty mounty assert_nothing_raised { server.mount(basedir, "localhost") } list = nil # and then check a few dirs assert_nothing_raised { - list = server.list("/localhost/with", :ignore, false, false) + list = server.list("/localhost/with", :manage, false, false) } assert(list !~ /with/) assert_nothing_raised { - list = server.list("/localhost/with/some/sub", :ignore, true, false) + list = server.list("/localhost/with/some/sub", :manage, true, false) } assert(list !~ /sub/) end # test many dirs, not necessarily very deep def test_widelists server = nil assert_nothing_raised { server = Puppet::Network::Handler.fileserver.new( :Local => true, :Config => false ) } basedir = tempfile() dirs = %w{a set of directories} assert_nothing_raised { Dir.mkdir(basedir) dirs.each { |dir| Dir.mkdir(File.join(basedir, dir)) } @@tmpfiles << basedir } assert_nothing_raised { server.mount(basedir, "localhost") } list = nil assert_nothing_raised { - list = server.list("/localhost/", :ignore, 1, false) + list = server.list("/localhost/", :manage, 1, false) } assert_instance_of(String, list, "Server returned %s instead of string") list = list.split("\n") assert_equal(dirs.length + 1, list.length) end # verify that 'describe' works as advertised def test_describe server = nil testdir = tstdir() files = mktestfiles(testdir) file = nil checks = Puppet::Network::Handler.fileserver::CHECKPARAMS assert_nothing_raised { server = Puppet::Network::Handler.fileserver.new( :Local => true, :Config => false ) } assert_nothing_raised { server.mount(testdir, "test") } # get our list list = nil sfile = "/test/" assert_nothing_raised { - list = server.list(sfile, :ignore, true, false) + list = server.list(sfile, :manage, true, false) } # and describe each file in the list assert_nothing_raised { list.split("\n").each { |line| file, type = line.split("\t") desc = server.describe(sfile + file) } } # and then make sure we can describe everything that we know is there files.each { |file| assert_describe(sfile, file, server) } # And then describe some files that we know aren't there retval = nil assert_nothing_raised("Describing non-existent files raised an error") { retval = server.describe(sfile + "noexisties") } assert_equal("", retval, "Description of non-existent files returned a value") # Now try to describe some sources that don't even exist retval = nil assert_raise(Puppet::Network::Handler::FileServerError, "Describing non-existent mount did not raise an error") { retval = server.describe("/notmounted/" + "noexisties") } assert_nil(retval, "Description of non-existent mounts returned a value") end # test that our config file is parsing and working as planned def test_configfile server = nil basedir = File.join(tmpdir, "fileserverconfigfiletesting") @@tmpfiles << basedir # make some dirs for mounting Dir.mkdir(basedir) mounts = {} %w{thing thus the-se those}.each { |dir| path = File.join(basedir, dir) Dir.mkdir(path) mounts[dir] = mktestfiles(path) } # create an example file with each of them conffile = tempfile @@tmpfiles << conffile File.open(conffile, "w") { |f| f.print "# a test config file [thing] path #{basedir}/thing allow 192.168.0.* [thus] path #{basedir}/thus allow *.madstop.com, *.kanies.com deny *.sub.madstop.com [the-se] path #{basedir}/the-se [those] path #{basedir}/those " } # create a server with the file assert_nothing_raised { server = Puppet::Network::Handler.fileserver.new( :Local => false, :Config => conffile ) } list = nil # run through once with no host/ip info, to verify everything is working mounts.each { |mount, files| mount = "/#{mount}/" assert_nothing_raised { - list = server.list(mount, :ignore, true, false) + list = server.list(mount, :manage, true, false) } assert_nothing_raised { list.split("\n").each { |line| file, type = line.split("\t") desc = server.describe(mount + file) } } files.each { |f| assert_describe(mount, f, server) } } # now let's check that things are being correctly forbidden # this is just a map of names and expected results { "thing" => { :deny => [ ["hostname.com", "192.168.1.0"], ["hostname.com", "192.158.0.0"] ], :allow => [ ["hostname.com", "192.168.0.0"], ["hostname.com", "192.168.0.245"], ] }, "thus" => { :deny => [ ["hostname.com", "192.168.1.0"], ["name.sub.madstop.com", "192.158.0.0"] ], :allow => [ ["luke.kanies.com", "192.168.0.0"], ["luke.madstop.com", "192.168.0.245"], ] } }.each { |mount, hash| mount = "/#{mount}/" # run through the map hash.each { |type, ary| ary.each { |sub| host, ip = sub case type when :deny: assert_raise(Puppet::AuthorizationError, "Host %s, ip %s, allowed %s" % [host, ip, mount]) { - list = server.list(mount, :ignore, true, false, host, ip) + list = server.list(mount, :manage, true, false, host, ip) } when :allow: assert_nothing_raised("Host %s, ip %s, denied %s" % [host, ip, mount]) { - list = server.list(mount, :ignore, true, false, host, ip) + list = server.list(mount, :manage, true, false, host, ip) } end } } } end # Test that we smoothly handle invalid config files def test_configfailures # create an example file with each of them conffile = tempfile() invalidmounts = { "noexist" => "[noexist] path /this/path/does/not/exist allow 192.168.0.* " } invalidconfigs = [ "[not valid] path /this/path/does/not/exist allow 192.168.0.* ", "[valid] invalidstatement path /etc allow 192.168.0.* ", "[valid] allow 192.168.0.* " ] invalidmounts.each { |mount, text| File.open(conffile, "w") { |f| f.print text } # create a server with the file server = nil assert_nothing_raised { server = Puppet::Network::Handler.fileserver.new( :Local => true, :Config => conffile ) } assert_raise(Puppet::Network::Handler::FileServerError, "Invalid mount was mounted") { - server.list(mount, :ignore) + server.list(mount, :manage) } } invalidconfigs.each_with_index { |text, i| File.open(conffile, "w") { |f| f.print text } # create a server with the file server = nil assert_raise(Puppet::Network::Handler::FileServerError, "Invalid config %s did not raise error" % i) { server = Puppet::Network::Handler.fileserver.new( :Local => true, :Config => conffile ) } } end # verify we reread the config file when it changes def test_filereread server = nil conffile = tempfile() dir = tstdir() files = mktestfiles(dir) File.open(conffile, "w") { |f| f.print "# a test config file [thing] path #{dir} allow test1.domain.com " } # Reset the timeout, so we reload faster Puppet[:filetimeout] = 0.5 # start our server with a fast timeout assert_nothing_raised { server = Puppet::Network::Handler.fileserver.new( :Local => false, :Config => conffile ) } list = nil assert_nothing_raised { - list = server.list("/thing/", :ignore, false, false, + list = server.list("/thing/", :manage, false, false, "test1.domain.com", "127.0.0.1") } assert(list != "", "List returned nothing in rereard test") assert_raise(Puppet::AuthorizationError, "List allowed invalid host") { - list = server.list("/thing/", :ignore, false, false, + list = server.list("/thing/", :manage, false, false, "test2.domain.com", "127.0.0.1") } sleep 1 File.open(conffile, "w") { |f| f.print "# a test config file [thing] path #{dir} allow test2.domain.com " } assert_raise(Puppet::AuthorizationError, "List allowed invalid host") { - list = server.list("/thing/", :ignore, false, false, + list = server.list("/thing/", :manage, false, false, "test1.domain.com", "127.0.0.1") } assert_nothing_raised { - list = server.list("/thing/", :ignore, false, false, + list = server.list("/thing/", :manage, false, false, "test2.domain.com", "127.0.0.1") } assert(list != "", "List returned nothing in rereard test") list = nil end # Verify that we get converted to the right kind of string def test_mountstring mount = nil name = "yaytest" path = tmpdir() assert_nothing_raised { mount = Puppet::Network::Handler.fileserver::Mount.new(name, path) } assert_equal("mount[#{name}]", mount.to_s) end def test_servinglinks # Disable the checking, so changes propagate immediately. Puppet[:filetimeout] = -5 server = nil source = tempfile() file = File.join(source, "file") link = File.join(source, "link") Dir.mkdir(source) File.open(file, "w") { |f| f.puts "yay" } File.symlink(file, link) assert_nothing_raised { server = Puppet::Network::Handler.fileserver.new( :Local => true, :Config => false ) } assert_nothing_raised { server.mount(source, "mount") } # First describe the link when following results = {} assert_nothing_raised { server.describe("/mount/link", :follow).split("\t").zip( Puppet::Network::Handler.fileserver::CHECKPARAMS ).each { |v,p| results[p] = v } } assert_equal("file", results[:type]) # Then not results = {} assert_nothing_raised { - server.describe("/mount/link", :ignore).split("\t").zip( + server.describe("/mount/link", :manage).split("\t").zip( Puppet::Network::Handler.fileserver::CHECKPARAMS ).each { |v,p| results[p] = v } } assert_equal("link", results[:type]) results.each { |p,v| assert(v, "%s has no value" % p) assert(v != "", "%s has no value" % p) } end # Test that substitution patterns in the path are exapanded # properly. Disabled, because it was testing too much of the process # and in a non-portable way. This is a thorough enough test that it should # be kept, but it should be done in a way that is clearly portable (e.g., # no md5 sums of file paths). def test_host_specific client1 = "client1.example.com" client2 = "client2.example.com" ip = "127.0.0.1" # Setup a directory hierarchy for the tests fsdir = File.join(tmpdir(), "host-specific") @@tmpfiles << fsdir hostdir = File.join(fsdir, "host") fqdndir = File.join(fsdir, "fqdn") client1_hostdir = File.join(hostdir, "client1") client2_fqdndir = File.join(fqdndir, client2) contents = { client1_hostdir => "client1\n", client2_fqdndir => client2 + "\n" } [fsdir, hostdir, fqdndir, client1_hostdir, client2_fqdndir].each { |d| Dir.mkdir(d) } [client1_hostdir, client2_fqdndir].each do |d| File.open(File.join(d, "file.txt"), "w") do |f| f.print contents[d] end end conffile = tempfile() File.open(conffile, "w") do |f| f.print(" [host] path #{hostdir}/%h allow * [fqdn] path #{fqdndir}/%H allow * ") end server = nil assert_nothing_raised { server = Puppet::Network::Handler.fileserver.new( :Local => true, :Config => conffile ) } # check that list returns the correct thing for the two clients list = nil sfile = "/host/file.txt" assert_nothing_raised { - list = server.list(sfile, :ignore, true, false, client1, ip) + list = server.list(sfile, :manage, true, false, client1, ip) } assert_equal("/\tfile", list) assert_nothing_raised { - list = server.list(sfile, :ignore, true, false, client2, ip) + list = server.list(sfile, :manage, true, false, client2, ip) } assert_equal("", list) sfile = "/fqdn/file.txt" assert_nothing_raised { - list = server.list(sfile, :ignore, true, false, client1, ip) + list = server.list(sfile, :manage, true, false, client1, ip) } assert_equal("", list) assert_nothing_raised { - list = server.list(sfile, :ignore, true, false, client2, ip) + list = server.list(sfile, :manage, true, false, client2, ip) } assert_equal("/\tfile", list) # check describe sfile = "/host/file.txt" assert_nothing_raised { - list = server.describe(sfile, :ignore, client1, ip).split("\t") + list = server.describe(sfile, :manage, client1, ip).split("\t") } assert_equal(5, list.size) assert_equal("file", list[1]) md5 = Digest::MD5.hexdigest(contents[client1_hostdir]) assert_equal("{md5}#{md5}", list[4]) assert_nothing_raised { - list = server.describe(sfile, :ignore, client2, ip).split("\t") + list = server.describe(sfile, :manage, client2, ip).split("\t") } assert_equal([], list) sfile = "/fqdn/file.txt" assert_nothing_raised { - list = server.describe(sfile, :ignore, client1, ip).split("\t") + list = server.describe(sfile, :manage, client1, ip).split("\t") } assert_equal([], list) assert_nothing_raised { - list = server.describe(sfile, :ignore, client2, ip).split("\t") + list = server.describe(sfile, :manage, client2, ip).split("\t") } assert_equal(5, list.size) assert_equal("file", list[1]) md5 = Digest::MD5.hexdigest(contents[client2_fqdndir]) assert_equal("{md5}#{md5}", list[4]) # Check retrieve sfile = "/host/file.txt" assert_nothing_raised { - list = server.retrieve(sfile, :ignore, client1, ip).chomp + list = server.retrieve(sfile, :manage, client1, ip).chomp } assert_equal(contents[client1_hostdir].chomp, list) assert_nothing_raised { - list = server.retrieve(sfile, :ignore, client2, ip).chomp + list = server.retrieve(sfile, :manage, client2, ip).chomp } assert_equal("", list) sfile = "/fqdn/file.txt" assert_nothing_raised { - list = server.retrieve(sfile, :ignore, client1, ip).chomp + list = server.retrieve(sfile, :manage, client1, ip).chomp } assert_equal("", list) assert_nothing_raised { - list = server.retrieve(sfile, :ignore, client2, ip).chomp + list = server.retrieve(sfile, :manage, client2, ip).chomp } assert_equal(contents[client2_fqdndir].chomp, list) end # Make sure the 'subdir' method in Mount works. def test_mount_subdir mount = nil base = tempfile() Dir.mkdir(base) subdir = File.join(base, "subdir") Dir.mkdir(subdir) [base, subdir].each do |d| File.open(File.join(d, "file"), "w") { |f| f.puts "bazoo" } end mount = mkmount(base) assert_equal(base, mount.subdir(), "Did not default to base path") assert_equal(subdir, mount.subdir("subdir"), "Did not default to base path") end # Make sure mounts get correctly marked expandable or not, depending on # the path. def test_expandable name = "yaytest" dir = tempfile() Dir.mkdir(dir) mount = mkmount() assert_nothing_raised { mount.path = dir } assert(! mount.expandable?, "Mount incorrectly called expandable") assert_nothing_raised { mount.path = "/dir/a%a" } assert(mount.expandable?, "Mount not called expandable") # This isn't a valid replacement pattern, so it should throw an error # because the dir doesn't exist assert_raise(Puppet::Network::Handler::FileServerError) { mount.path = "/dir/a%" } # Now send it back to a normal path assert_nothing_raised { mount.path = dir } # Make sure it got reverted assert(! mount.expandable?, "Mount incorrectly called expandable") end def test_mount_expand mount = mkmount() check = proc do |client, pattern, repl| path = "/my/#{pattern}/file" assert_equal("/my/#{repl}/file", mount.expand(path, client)) end # Do a round of checks with a fake client client = "host.domain.com" {"%h" => "host", # Short name "%H" => client, # Full name "%d" => "domain.com", # domain "%%" => "%", # escape "%o" => "%o" # other }.each do |pat, repl| result = check.call(client, pat, repl) end # Now, check that they use Facter info Puppet.notice "The following messages are normal" client = nil - local = Facter["hostname"].value - domain = Facter["domain"].value - fqdn = [local, domain].join(".") - {"%h" => local, # Short name - "%H" => fqdn, # Full name - "%d" => domain, # domain + Facter.stubs(:value).with(:ipaddress).returns("127.0.0.1") + Facter.stubs(:value).with { |v| v.to_s == "hostname" }.returns("myhost") + Facter.stubs(:value).with { |v| v.to_s == "domain" }.returns("mydomain.com") + Facter.stubs(:value).with(:domain).returns("mydomain.com") + + {"%h" => "myhost", # Short name + "%H" => "myhost.mydomain.com", # Full name + "%d" => "mydomain.com", # domain "%%" => "%", # escape "%o" => "%o" # other }.each do |pat, repl| check.call(client, pat, repl) end end # Test that the fileserver expands the %h and %d things. def test_fileserver_expansion server = nil assert_nothing_raised { server = Puppet::Network::Handler.fileserver.new( :Local => true, :Config => false ) } dir = tempfile() Facter.stubs(:value).with(:ipaddress).returns("127.0.0.1") Facter.stubs(:value).with { |v| v.to_s == "hostname" }.returns("myhost") Facter.stubs(:value).with { |v| v.to_s == "domain" }.returns("mydomain.com") Facter.stubs(:value).with(:domain).returns("mydomain.com") ip = Facter.value(:ipaddress) Dir.mkdir(dir) host = "myhost.mydomain.com" { "%H" => "myhost.mydomain.com", "%h" => "myhost", "%d" => "mydomain.com" }.each do |pattern, string| file = File.join(dir, string) mount = File.join(dir, pattern) File.open(file, "w") do |f| f.puts "yayness: %s" % string end name = "name" obj = nil assert_nothing_raised { obj = server.mount(mount, name) } obj.allow "*" ret = nil assert_nothing_raised do - ret = server.list("/name", :ignore, false, false, host, ip) + ret = server.list("/name", :manage, false, false, host, ip) end assert_equal("/\tfile", ret) assert_nothing_raised do - ret = server.describe("/name", :ignore, host, ip) + ret = server.describe("/name", :manage, host, ip) end - assert(ret =~ /\tfile\t/, "Did not get valid a description") + assert(ret =~ /\tfile\t/, "Did not get valid a description (#{ret.inspect})") assert_nothing_raised do - ret = server.retrieve("/name", :ignore, host, ip) + ret = server.retrieve("/name", :manage, host, ip) end assert_equal(ret, File.read(file)) server.umount(name) File.unlink(file) end end # Test the default modules fileserving def test_modules_default moddir = tempfile Dir.mkdir(moddir) mounts = {} Puppet[:modulepath] = moddir mods = %w{green red}.collect do |name| path = File::join(moddir, name, Puppet::Module::FILES) FileUtils::mkdir_p(path) if name == "green" file = File::join(path, "test.txt") File::open(file, "w") { |f| f.print name } end Puppet::Module::find(name) end conffile = tempfile File.open(conffile, "w") { |f| f.puts "# a test config file" } # create a server with the file server = nil assert_nothing_raised { server = Puppet::Network::Handler::FileServer.new( :Local => false , :Config => conffile ) } mods.each do |mod| mount = "/#{mod.name}/" list = nil assert_nothing_raised { - list = server.list(mount, :ignore, true, false) + list = server.list(mount, :manage, true, false) } list = list.split("\n") if mod.name == "green" assert_equal(2, list.size) assert_equal("/\tdirectory", list[0]) assert_equal("/test.txt\tfile", list[1]) else assert_equal(1, list.size) assert_equal("/\tdirectory", list[0]) end assert_nothing_raised("Host 'allow' denied #{mount}") { - server.list(mount, :ignore, true, false, + server.list(mount, :manage, true, false, 'allow.example.com', "192.168.0.1") } end end # Test that configuring deny/allow for modules works def test_modules_config moddir = tempfile Dir.mkdir(moddir) mounts = {} Puppet[:modulepath] = moddir path = File::join(moddir, "amod", Puppet::Module::FILES) file = File::join(path, "test.txt") FileUtils::mkdir_p(path) File::open(file, "w") { |f| f.print "Howdy" } mod = Puppet::Module::find("amod") conffile = tempfile @@tmpfiles << conffile File.open(conffile, "w") { |f| f.print "# a test config file [modules] path #{basedir}/thing allow 192.168.0.* " } # create a server with the file server = nil assert_nothing_raised { server = Puppet::Network::Handler::FileServer.new( :Local => false, :Config => conffile ) } list = nil mount = "/#{mod.name}/" assert_nothing_raised { - list = server.list(mount, :ignore, true, false) + list = server.list(mount, :manage, true, false) } assert_nothing_raised { list.split("\n").each { |line| file, type = line.split("\t") server.describe(mount + file) } } assert_describe(mount, file, server) # now let's check that things are being correctly forbidden assert_raise(Puppet::AuthorizationError, "Host 'deny' allowed #{mount}") { - server.list(mount, :ignore, true, false, + server.list(mount, :manage, true, false, 'deny.example.com', "192.168.1.1") } assert_nothing_raised("Host 'allow' denied #{mount}") { - server.list(mount, :ignore, true, false, + server.list(mount, :manage, true, false, 'allow.example.com', "192.168.0.1") } end # Make sure we successfully throw errors -- someone ran into this with # 0.22.4. def test_failures # create a server with the file server = nil config = tempfile [ "[this is invalid]\nallow one.two.com", # invalid name "[valid]\nallow *.testing something.com", # invalid allow "[valid]\nallow one.two.com\ndeny *.testing something.com", # invalid deny ].each do |failer| File.open(config, "w") { |f| f.puts failer } assert_raise(Puppet::Network::Handler::FileServerError, "Did not fail on %s" % failer.inspect) { server = Puppet::Network::Handler::FileServer.new( :Local => false, :Config => config ) } end end def test_can_start_without_configuration Puppet[:fileserverconfig] = tempfile assert_nothing_raised("Could not create fileserver when configuration is absent") do server = Puppet::Network::Handler::FileServer.new( :Local => false ) end end def test_creates_default_mounts_when_no_configuration_is_available Puppet[:fileserverconfig] = tempfile server = Puppet::Network::Handler::FileServer.new(:Local => false) assert(server.mounted?("plugins"), "Did not create default plugins mount when missing configuration file") assert(server.mounted?("modules"), "Did not create default modules mount when missing configuration file") end end diff --git a/test/network/server/webrick.rb b/test/network/server/webrick.rb index d3408c166..fe6d69ade 100755 --- a/test/network/server/webrick.rb +++ b/test/network/server/webrick.rb @@ -1,152 +1,152 @@ #!/usr/bin/env ruby require File.dirname(__FILE__) + '/../../lib/puppettest' require 'puppettest' require 'puppet/network/http_server/webrick' require 'mocha' class TestWebrickServer < Test::Unit::TestCase include PuppetTest::ServerTest def setup Puppet::Util::SUIDManager.stubs(:asuser).yields super end def teardown super Puppet::Network::HttpPool.clear_http_instances end # Make sure we can create a server, and that it knows how to create its # certs by default. def test_basics server = nil assert_raise(Puppet::Error, "server succeeded with no cert") do server = Puppet::Network::HTTPServer::WEBrick.new( :Port => @@port, :Handlers => { :Status => nil } ) end assert_nothing_raised("Could not create simple server") do server = Puppet::Network::HTTPServer::WEBrick.new( :Port => @@port, :Handlers => { :CA => {}, # so that certs autogenerate :Status => nil } ) end assert(server, "did not create server") assert(server.cert, "did not retrieve cert") end # test that we can connect to the server # we have to use fork here, because we apparently can't use threads # to talk to other threads def test_connect_with_fork Puppet[:autosign] = true serverpid, server = mk_status_server # create a status client, and verify it can talk client = mk_status_client assert(client.cert, "did not get cert for client") retval = nil assert_nothing_raised("Could not connect to server") { retval = client.status } assert_equal(1, retval) end # Test that a client whose cert has been revoked really can't connect def test_certificate_revocation Puppet[:autosign] = true serverpid, server = mk_status_server client = mk_status_client status = nil assert_nothing_raised() { status = client.status } assert_equal(1, status) client.shutdown # Revoke the client's cert ca = Puppet::SSLCertificates::CA.new() ca.revoke(ca.getclientcert(Puppet[:certname])[0].serial) # Restart the server @@port += 1 Puppet[:autosign] = false kill_and_wait(serverpid, server.pidfile) serverpid, server = mk_status_server # This time the client should be denied. With keep-alive, # the client starts its connection immediately, thus throwing # the error. assert_raise(OpenSSL::SSL::SSLError) { - client = Puppet::Network::Client.status.new(:Server => "localhost", :Port => @@port) + Puppet::Network::HttpPool.http_instance("localhost", @@port).start } end def mk_status_client client = nil # Otherwise, the client initalization will trip over itself # since elements created in the last run are still around Puppet::Type::allclear assert_nothing_raised() { client = Puppet::Network::Client.status.new( :Server => "localhost", :Port => @@port ) } client end def mk_status_server server = nil Puppet[:certdnsnames] = "localhost" assert_nothing_raised() { server = Puppet::Network::HTTPServer::WEBrick.new( :Port => @@port, :Handlers => { :CA => {}, # so that certs autogenerate :Status => nil } ) } pid = fork { Puppet[:name] = "puppetmasterd" assert_nothing_raised() { trap(:INT) { server.shutdown } server.start } } @@tmppids << pid [pid, server] end def kill_and_wait(pid, file) %x{kill -INT #{pid} 2>/dev/null} count = 0 while count < 30 && File::exist?(file) count += 1 sleep(1) end assert(count < 30, "Killing server #{pid} failed") end end diff --git a/test/other/dsl.rb b/test/other/dsl.rb index b4dd0659b..45b51982d 100755 --- a/test/other/dsl.rb +++ b/test/other/dsl.rb @@ -1,216 +1,215 @@ #!/usr/bin/env ruby require File.dirname(__FILE__) + '/../lib/puppettest' require 'puppet' require 'puppet/dsl' require 'puppettest' class TestDSL < Test::Unit::TestCase include PuppetTest include Puppet::DSL def teardown super - Puppet::Aspect.clear + Puppet::DSL::Aspect.clear end def test_aspect a = nil assert_nothing_raised do a = aspect :yaytest do end end - assert_equal(a, Puppet::Aspect[:yaytest]) + assert_equal(a, Puppet::DSL::Aspect[:yaytest]) # Now make a child aspect b = nil assert_nothing_raised do b = aspect :child, :inherits => :yaytest do end end assert(b.child_of?(a), "Parentage not set up correctly") assert(b.child_of?(:yaytest), "Parentage not set up for symbols") # Now make another subclass c = nil assert_nothing_raised do c = aspect :kid, :inherits => :child do end end assert(c.child_of?(b), "Parentage not set up correctly") assert(c.child_of?(a), "Parentage is not inherited") # Lastly, make a separate aspect x = nil assert_nothing_raised do x = aspect :other do end end assert(! x.child_of?(a), "Parentage came from nowhere") assert(! x.child_of?(b), "Parentage came from nowhere") assert(! x.child_of?(c), "Parentage came from nowhere") # Make sure we can specify the name or the aspect y = nil assert_nothing_raised do x = aspect :naming, :inherits => a do end end assert(x.child_of?(a), "Parentage not set up correctly") # And make sure the parent must exist z = nil assert_raise(RuntimeError) do z = aspect :noparent, :inherits => :nosuchaspect do end end assert(x.child_of?(a), "Parentage not set up correctly") end def test_evaluate parent = child = nil parenteval = childeval = nil assert_nothing_raised do parent = aspect :parent do if parenteval raise "parent already evaluated" end parenteval = true end child = aspect :child, :inherits => parent do if childeval raise "child already evaluated" end childeval = true end end assert_nothing_raised do parent.evaluate() end assert(parenteval, "Parent was not evaluated") assert(parent.evaluated?, "parent was not considered evaluated") # Make sure evaluating twice silently does nothing assert_nothing_raised do parent.evaluate() end # Now evaluate the child assert_nothing_raised do child.evaluate end assert(childeval, "child was not evaluated") assert(child.evaluated?, "child was not considered evaluated") # Now reset them both parenteval = childeval = nil parent.evaluated = false child.evaluated = false # evaluate the child assert_nothing_raised do child.evaluate end # and make sure both get evaluated assert(parenteval, "Parent was not evaluated") assert(parent.evaluated?, "parent was not considered evaluated") assert(childeval, "child was not evaluated") assert(child.evaluated?, "child was not considered evaluated") end def test_acquire evalled = false a = aspect :test do evalled = true end assert_nothing_raised do acquire :test end assert(evalled, "Did not evaluate aspect") assert_nothing_raised do acquire :test end end def test_newresource filetype = Puppet::Type.type(:file) path = tempfile() a = aspect :testing resource = nil assert_nothing_raised do - resource = a.newresource filetype, path, - :content => "yay", :mode => "640" + resource = a.newresource filetype, path, :content => "yay", :mode => "640" end assert_instance_of(Puppet::Parser::Resource, resource) assert_equal("yay", resource[:content]) assert_equal("640", resource[:mode]) assert_equal(:testing, resource.source.name) # Now try exporting our aspect assert_nothing_raised do a.evaluate end result = nil assert_nothing_raised do result = a.export end assert_equal([resource], result) # Then try the DSL export assert_nothing_raised do result = export end assert_instance_of(Puppet::TransBucket, result) # And just for kicks, test applying everything assert_nothing_raised do apply() end assert(FileTest.exists?(path), "File did not get created") assert_equal("yay", File.read(path)) end def test_typemethods Puppet::Type.loadall filetype = Puppet::Type.type(:file) path = tempfile() a = aspect :testing Puppet::Type.eachtype do |type| next if type.name.to_s =~ /test/ assert(a.respond_to?(type.name), "Aspects do not have a %s method" % type.name) end file = nil assert_nothing_raised do file = a.file path, :content => "yay", :mode => "640" end assert_instance_of(Puppet::Parser::Resource, file) end end diff --git a/test/other/transactions.rb b/test/other/transactions.rb index 105698da1..ce2d0d52b 100755 --- a/test/other/transactions.rb +++ b/test/other/transactions.rb @@ -1,1109 +1,1111 @@ #!/usr/bin/env ruby require File.dirname(__FILE__) + '/../lib/puppettest' require 'puppet' require 'puppettest' require 'mocha' require 'puppettest/support/resources' +require 'puppettest/support/utils' class TestTransactions < Test::Unit::TestCase include PuppetTest::FileTesting include PuppetTest::Support::Resources + include PuppetTest::Support::Utils class Fakeprop true) def finish $finished << self.name end end if block type.class_eval(&block) end cleanup do Puppet::Type.rmtype(:generator) end return type end # Create a new type that generates instances with shorter names. def mkreducer(&block) type = mkgenerator() do def eval_generate ret = [] if title.length > 1 ret << self.class.create(:title => title[0..-2]) else return nil end ret end end if block type.class_eval(&block) end return type end def test_reports path1 = tempfile() path2 = tempfile() objects = [] objects << Puppet::Type.newfile( :path => path1, :content => "yayness" ) objects << Puppet::Type.newfile( :path => path2, :content => "booness" ) trans = assert_events([:file_created, :file_created], *objects) report = nil assert_nothing_raised { report = trans.generate_report } # First test the report logs assert(report.logs.length > 0, "Did not get any report logs") report.logs.each do |obj| assert_instance_of(Puppet::Util::Log, obj) end # Then test the metrics metrics = report.metrics assert(metrics, "Did not get any metrics") assert(metrics.length > 0, "Did not get any metrics") assert(metrics.has_key?("resources"), "Did not get object metrics") assert(metrics.has_key?("changes"), "Did not get change metrics") metrics.each do |name, metric| assert_instance_of(Puppet::Util::Metric, metric) end end def test_prefetch # Create a type just for testing prefetch name = :prefetchtesting $prefetched = false type = Puppet::Type.newtype(name) do newparam(:name) {} end cleanup do Puppet::Type.rmtype(name) end # Now create a provider type.provide(:prefetch) do def self.prefetch(resources) $prefetched = resources end end # Now create an instance inst = type.create :name => "yay" # Create a transaction trans = Puppet::Transaction.new(mk_catalog(inst)) # Make sure prefetch works assert_nothing_raised do trans.prefetch end assert_equal({inst.title => inst}, $prefetched, "type prefetch was not called") # Now make sure it gets called from within evaluate() $prefetched = false assert_nothing_raised do trans.evaluate end assert_equal({inst.title => inst}, $prefetched, "evaluate did not call prefetch") end def test_refreshes_generate_events path = tempfile() firstpath = tempfile() secondpath = tempfile() file = Puppet::Type.newfile(:title => "file", :path => path, :content => "yayness") first = Puppet::Type.newexec(:title => "first", :command => "/bin/echo first > #{firstpath}", :subscribe => [:file, path], :refreshonly => true ) second = Puppet::Type.newexec(:title => "second", :command => "/bin/echo second > #{secondpath}", :subscribe => [:exec, "first"], :refreshonly => true ) assert_apply(file, first, second) assert(FileTest.exists?(secondpath), "Refresh did not generate an event") end unless %x{groups}.chomp.split(/ /).length > 1 $stderr.puts "You must be a member of more than one group to test transactions" else def ingroup(gid) require 'etc' begin group = Etc.getgrgid(gid) rescue => detail puts "Could not retrieve info for group %s: %s" % [gid, detail] return nil end return @groups.include?(group.name) end def setup super @groups = %x{groups}.chomp.split(/ /) unless @groups.length > 1 p @groups raise "You must be a member of more than one group to test this" end end def newfile(hash = {}) tmpfile = tempfile() File.open(tmpfile, "w") { |f| f.puts rand(100) } # XXX now, because os x apparently somehow allows me to make a file # owned by a group i'm not a member of, i have to verify that # the file i just created is owned by one of my groups # grrr unless ingroup(File.stat(tmpfile).gid) Puppet.info "Somehow created file in non-member group %s; fixing" % File.stat(tmpfile).gid require 'etc' firstgr = @groups[0] unless firstgr.is_a?(Integer) str = Etc.getgrnam(firstgr) firstgr = str.gid end File.chown(nil, firstgr, tmpfile) end hash[:name] = tmpfile assert_nothing_raised() { return Puppet.type(:file).create(hash) } end def newexec(file) assert_nothing_raised() { return Puppet.type(:exec).create( :name => "touch %s" % file, :path => "/bin:/usr/bin:/sbin:/usr/sbin", :returns => 0 ) } end # modify a file and then roll the modifications back def test_filerollback transaction = nil file = newfile() properties = {} check = [:group,:mode] file[:check] = check assert_nothing_raised() { file.retrieve } assert_nothing_raised() { check.each { |property| value = file.value(property) assert(value) properties[property] = value } } component = mk_catalog("file",file) require 'etc' groupname = Etc.getgrgid(File.stat(file.name).gid).name assert_nothing_raised() { # Find a group that it's not set to group = @groups.find { |group| group != groupname } unless group raise "Could not find suitable group" end file[:group] = group file[:mode] = "755" } trans = assert_events([:file_changed, :file_changed], component) file.retrieve assert_rollback_events(trans, [:file_changed, :file_changed], "file") assert_nothing_raised() { file.retrieve } properties.each { |property,value| assert_equal( value, file.value(property), "File %s remained %s" % [property, file.value(property)] ) } end # test that services are correctly restarted and that work is done # in the right order def test_refreshing transaction = nil file = newfile() execfile = File.join(tmpdir(), "exectestingness") exec = newexec(execfile) properties = {} check = [:group,:mode] file[:check] = check file[:group] = @groups[0] config = mk_catalog(file) config.apply @@tmpfiles << execfile # 'subscribe' expects an array of arrays exec[:subscribe] = [[file.class.name,file.name]] exec[:refreshonly] = true assert_nothing_raised() { file.retrieve exec.retrieve } check.each { |property| properties[property] = file.value(property) } assert_nothing_raised() { file[:mode] = "755" } # Make a new catalog so the resource relationships get # set up. config = mk_catalog(file, exec) trans = assert_events([:file_changed, :triggered], config) assert(FileTest.exists?(execfile), "Execfile does not exist") File.unlink(execfile) assert_nothing_raised() { file[:group] = @groups[1] } trans = assert_events([:file_changed, :triggered], config) assert(FileTest.exists?(execfile), "Execfile does not exist") end # Verify that one component requiring another causes the contained # resources in the requiring component to get refreshed. def test_refresh_across_two_components transaction = nil file = newfile() execfile = File.join(tmpdir(), "exectestingness2") @@tmpfiles << execfile exec = newexec(execfile) properties = {} check = [:group,:mode] file[:check] = check file[:group] = @groups[0] assert_apply(file) config = Puppet::Node::Catalog.new fcomp = Puppet::Type.type(:component).create(:name => "file") config.add_resource fcomp config.add_resource file config.add_edge(fcomp, file) ecomp = Puppet::Type.type(:component).create(:name => "exec") config.add_resource ecomp config.add_resource exec config.add_edge(ecomp, exec) # 'subscribe' expects an array of arrays #component[:require] = [[file.class.name,file.name]] ecomp[:subscribe] = fcomp exec[:refreshonly] = true trans = assert_events([], config) assert_nothing_raised() { file[:group] = @groups[1] file[:mode] = "755" } trans = assert_events([:file_changed, :file_changed, :triggered], config) end # Make sure that multiple subscriptions get triggered. def test_multisubs path = tempfile() file1 = tempfile() file2 = tempfile() file = Puppet.type(:file).create( :path => path, :ensure => "file" ) exec1 = Puppet.type(:exec).create( :path => ENV["PATH"], :command => "touch %s" % file1, :refreshonly => true, :subscribe => [:file, path] ) exec2 = Puppet.type(:exec).create( :path => ENV["PATH"], :command => "touch %s" % file2, :refreshonly => true, :subscribe => [:file, path] ) assert_apply(file, exec1, exec2) assert(FileTest.exists?(file1), "File 1 did not get created") assert(FileTest.exists?(file2), "File 2 did not get created") end # Make sure that a failed trigger doesn't result in other events not # getting triggered. def test_failedrefreshes path = tempfile() newfile = tempfile() file = Puppet.type(:file).create( :path => path, :ensure => "file" ) exec1 = Puppet.type(:exec).create( :path => ENV["PATH"], :command => "touch /this/cannot/possibly/exist", :logoutput => true, :refreshonly => true, :subscribe => file, :title => "one" ) exec2 = Puppet.type(:exec).create( :path => ENV["PATH"], :command => "touch %s" % newfile, :logoutput => true, :refreshonly => true, :subscribe => [file, exec1], :title => "two" ) assert_apply(file, exec1, exec2) assert(FileTest.exists?(newfile), "Refresh file did not get created") end # Make sure that unscheduled and untagged objects still respond to events def test_unscheduled_and_untagged_response Puppet::Type.type(:schedule).mkdefaultschedules Puppet[:ignoreschedules] = false file = Puppet.type(:file).create( :name => tempfile(), :ensure => "file", :backup => false ) fname = tempfile() exec = Puppet.type(:exec).create( :name => "touch %s" % fname, :path => "/usr/bin:/bin", :schedule => "monthly", :subscribe => ["file", file.name] ) config = mk_catalog(file, exec) # Run it once assert_apply(config) assert(FileTest.exists?(fname), "File did not get created") assert(!exec.scheduled?, "Exec is somehow scheduled") # Now remove it, so it can get created again File.unlink(fname) file[:content] = "some content" assert_events([:file_changed, :triggered], config) assert(FileTest.exists?(fname), "File did not get recreated") # Now remove it, so it can get created again File.unlink(fname) # And tag our exec exec.tag("testrun") # And our file, so it runs file.tag("norun") Puppet[:tags] = "norun" file[:content] = "totally different content" assert(! file.insync?(file.retrieve), "Uh, file is in sync?") assert_events([:file_changed, :triggered], config) assert(FileTest.exists?(fname), "File did not get recreated") end def test_failed_reqs_mean_no_run exec = Puppet::Type.type(:exec).create( :command => "/bin/mkdir /this/path/cannot/possibly/exit", :title => "mkdir" ) file1 = Puppet::Type.type(:file).create( :title => "file1", :path => tempfile(), :require => exec, :ensure => :file ) file2 = Puppet::Type.type(:file).create( :title => "file2", :path => tempfile(), :require => file1, :ensure => :file ) config = mk_catalog(exec, file1, file2) assert_apply(config) assert(! FileTest.exists?(file1[:path]), "File got created even tho its dependency failed") assert(! FileTest.exists?(file2[:path]), "File got created even tho its deep dependency failed") end end def test_relationship_graph config = mktree config.meta_def(:f) do |name| self.resource("File[%s]" % name) end {"one" => "two", "File[f]" => "File[c]", "File[h]" => "middle"}.each do |source_ref, target_ref| source = config.resource(source_ref) or raise "Missing %s" % source_ref target = config.resource(target_ref) or raise "Missing %s" % target_ref target[:require] = source end trans = Puppet::Transaction.new(config) graph = nil assert_nothing_raised do graph = trans.relationship_graph end assert_instance_of(Puppet::Node::Catalog, graph, "Did not get relationship graph") # Make sure all of the components are gone comps = graph.vertices.find_all { |v| v.is_a?(Puppet::Type::Component)} assert(comps.empty?, "Deps graph still contains components %s" % comps.collect { |c| c.ref }.join(",")) assert_equal([], comps, "Deps graph still contains components") # It must be reversed because of how topsort works sorted = graph.topsort.reverse # Now make sure the appropriate edges are there and are in the right order assert(graph.dependents(config.f(:f)).include?(config.f(:c)), "c not marked a dep of f") assert(sorted.index(config.f(:c)) < sorted.index(config.f(:f)), "c is not before f") config.resource("one").each do |o| config.resource("two").each do |t| assert(graph.dependents(o).include?(t), "%s not marked a dep of %s" % [t.ref, o.ref]) assert(sorted.index(t) < sorted.index(o), "%s is not before %s" % [t.ref, o.ref]) end end trans.catalog.leaves(config.resource("middle")).each do |child| assert(graph.dependents(config.f(:h)).include?(child), "%s not marked a dep of h" % [child.ref]) assert(sorted.index(child) < sorted.index(config.f(:h)), "%s is not before h" % child.ref) end # Lastly, make sure our 'g' vertex made it into the relationship # graph, since it's not involved in any relationships. assert(graph.vertex?(config.f(:g)), "Lost vertexes with no relations") # Now make the reversal graph and make sure all of the vertices made it into that reverse = graph.reversal %w{a b c d e f g h}.each do |letter| file = config.f(letter) assert(reverse.vertex?(file), "%s did not make it into reversal" % letter) end end # Test pre-evaluation generation def test_generate mkgenerator() do def generate ret = [] if title.length > 1 ret << self.class.create(:title => title[0..-2]) else return nil end ret end end yay = Puppet::Type.newgenerator :title => "yay" rah = Puppet::Type.newgenerator :title => "rah" config = mk_catalog(yay, rah) trans = Puppet::Transaction.new(config) assert_nothing_raised do trans.generate end %w{ya ra y r}.each do |name| assert(trans.catalog.vertex?(Puppet::Type.type(:generator)[name]), "Generated %s was not a vertex" % name) assert($finished.include?(name), "%s was not finished" % name) end # Now make sure that cleanup gets rid of those generated types. assert_nothing_raised do trans.cleanup end %w{ya ra y r}.each do |name| assert(!trans.catalog.vertex?(Puppet::Type.type(:generator)[name]), "Generated vertex %s was not removed from graph" % name) assert_nil(Puppet::Type.type(:generator)[name], "Generated vertex %s was not removed from class" % name) end end # Test mid-evaluation generation. def test_eval_generate $evaluated = [] cleanup { $evaluated = nil } type = mkreducer() do def evaluate $evaluated << self.title return [] end end yay = Puppet::Type.newgenerator :title => "yay" rah = Puppet::Type.newgenerator :title => "rah", :subscribe => yay config = mk_catalog(yay, rah) trans = Puppet::Transaction.new(config) trans.prepare # Now apply the resources, and make sure they appropriately generate # things. assert_nothing_raised("failed to apply yay") do trans.eval_resource(yay) end ya = type["ya"] assert(ya, "Did not generate ya") assert(trans.relationship_graph.vertex?(ya), "Did not add ya to rel_graph") # Now make sure the appropriate relationships were added assert(trans.relationship_graph.edge?(yay, ya), "parent was not required by child") assert(! trans.relationship_graph.edge?(ya, rah), "generated child ya inherited depencency on rah") # Now make sure it in turn eval_generates appropriately assert_nothing_raised("failed to apply yay") do trans.eval_resource(type["ya"]) end %w{y}.each do |name| res = type[name] assert(res, "Did not generate %s" % name) assert(trans.relationship_graph.vertex?(res), "Did not add %s to rel_graph" % name) assert($finished.include?("y"), "y was not finished") end assert_nothing_raised("failed to eval_generate with nil response") do trans.eval_resource(type["y"]) end assert(trans.relationship_graph.edge?(yay, ya), "no edge was created for ya => yay") assert_nothing_raised("failed to apply rah") do trans.eval_resource(rah) end ra = type["ra"] assert(ra, "Did not generate ra") assert(trans.relationship_graph.vertex?(ra), "Did not add ra to rel_graph" % name) assert($finished.include?("ra"), "y was not finished") # Now make sure this generated resource has the same relationships as # the generating resource assert(! trans.relationship_graph.edge?(yay, ra), "rah passed its dependencies on to its children") assert(! trans.relationship_graph.edge?(ya, ra), "children have a direct relationship") # Now make sure that cleanup gets rid of those generated types. assert_nothing_raised do trans.cleanup end %w{ya ra y r}.each do |name| assert(!trans.relationship_graph.vertex?(type[name]), "Generated vertex %s was not removed from graph" % name) assert_nil(type[name], "Generated vertex %s was not removed from class" % name) end # Now, start over and make sure that everything gets evaluated. trans = Puppet::Transaction.new(config) $evaluated.clear assert_nothing_raised do trans.evaluate end assert_equal(%w{yay ya y rah ra r}, $evaluated, "Not all resources were evaluated or not in the right order") end def test_ignore_tags? config = Puppet::Node::Catalog.new config.host_config = true transaction = Puppet::Transaction.new(config) assert(! transaction.ignore_tags?, "Ignoring tags when applying a host catalog") config.host_config = false transaction = Puppet::Transaction.new(config) assert(transaction.ignore_tags?, "Not ignoring tags when applying a non-host catalog") end def test_missing_tags? resource = stub 'resource', :tagged? => true config = Puppet::Node::Catalog.new # Mark it as a host config so we don't care which test is first config.host_config = true transaction = Puppet::Transaction.new(config) assert(! transaction.missing_tags?(resource), "Considered a resource to be missing tags when none are set") # host catalogs pay attention to tags, no one else does. Puppet[:tags] = "three,four" config.host_config = false transaction = Puppet::Transaction.new(config) assert(! transaction.missing_tags?(resource), "Considered a resource to be missing tags when not running a host catalog") # config.host_config = true transaction = Puppet::Transaction.new(config) assert(! transaction.missing_tags?(resource), "Considered a resource to be missing tags when running a host catalog and all tags are present") transaction = Puppet::Transaction.new(config) resource.stubs :tagged? => false assert(transaction.missing_tags?(resource), "Considered a resource not to be missing tags when running a host catalog and tags are missing") end # Make sure changes generated by eval_generated resources have proxies # set to the top-level resource. def test_proxy_resources type = mkreducer do def evaluate return Puppet::PropertyChange.new(Fakeprop.new( :path => :path, :is => :is, :should => :should, :name => self.name, :resource => "a parent"), :is) end end resource = type.create :name => "test" config = mk_catalog(resource) trans = Puppet::Transaction.new(config) trans.prepare assert_nothing_raised do trans.eval_resource(resource) end changes = trans.instance_variable_get("@changes") assert(changes.length > 0, "did not get any changes") changes.each do |change| assert_equal(resource, change.source, "change did not get proxy set correctly") end end # Make sure changes in contained files still generate callback events. def test_generated_callbacks dir = tempfile() maker = tempfile() Dir.mkdir(dir) file = File.join(dir, "file") File.open(file, "w") { |f| f.puts "" } File.chmod(0644, file) File.chmod(0755, dir) # So only the child file causes a change dirobj = Puppet::Type.type(:file).create :mode => "755", :recurse => true, :path => dir exec = Puppet::Type.type(:exec).create :title => "make", :command => "touch #{maker}", :path => ENV['PATH'], :refreshonly => true, :subscribe => dirobj assert_apply(dirobj, exec) assert(FileTest.exists?(maker), "Did not make callback file") end # Yay, this out to be fun. def test_trigger $triggered = [] cleanup { $triggered = nil } trigger = Class.new do attr_accessor :name include Puppet::Util::Logging def initialize(name) @name = name end def ref self.name end def refresh $triggered << self.name end def to_s self.name end end # Make a graph with some stuff in it. graph = Puppet::Node::Catalog.new # Add a non-triggering edge. a = trigger.new(:a) b = trigger.new(:b) c = trigger.new(:c) nope = Puppet::Relationship.new(a, b) yep = Puppet::Relationship.new(a, c, {:callback => :refresh}) graph.add_edge(nope) # And a triggering one. graph.add_edge(yep) # Create our transaction trans = Puppet::Transaction.new(graph) # Set the non-triggering on assert_nothing_raised do trans.set_trigger(nope) end assert(! trans.targeted?(b), "b is incorrectly targeted") # Now set the other assert_nothing_raised do trans.set_trigger(yep) end assert(trans.targeted?(c), "c is not targeted") # Now trigger our three resources assert_nothing_raised do assert_nil(trans.trigger(a), "a somehow triggered something") end assert_nothing_raised do assert_nil(trans.trigger(b), "b somehow triggered something") end assert_equal([], $triggered,"got something in triggered") result = nil assert_nothing_raised do result = trans.trigger(c) end assert(result, "c did not trigger anything") assert_instance_of(Array, result) event = result.shift assert_instance_of(Puppet::Event, event) assert_equal(:triggered, event.event, "event was not set correctly") assert_equal(c, event.source, "source was not set correctly") assert_equal(trans, event.transaction, "transaction was not set correctly") assert(trans.triggered?(c, :refresh), "Transaction did not store the trigger") end def test_set_target file = Puppet::Type.newfile(:path => tempfile(), :content => "yay") exec1 = Puppet::Type.type(:exec).create :command => "/bin/echo exec1" exec2 = Puppet::Type.type(:exec).create :command => "/bin/echo exec2" trans = Puppet::Transaction.new(mk_catalog(file, exec1, exec2)) # First try it with an edge that has no callback edge = Puppet::Relationship.new(file, exec1) assert_nothing_raised { trans.set_trigger(edge) } assert(! trans.targeted?(exec1), "edge with no callback resulted in a target") # Now with an edge that has an unsupported callback edge = Puppet::Relationship.new(file, exec1, :callback => :nosuchmethod, :event => :ALL_EVENTS) assert_nothing_raised { trans.set_trigger(edge) } assert(! trans.targeted?(exec1), "edge with invalid callback resulted in a target") # Lastly, with an edge with a supported callback edge = Puppet::Relationship.new(file, exec1, :callback => :refresh, :event => :ALL_EVENTS) assert_nothing_raised { trans.set_trigger(edge) } assert(trans.targeted?(exec1), "edge with valid callback did not result in a target") end # Testing #401 -- transactions are calling refresh() on classes that don't support it. def test_callback_availability $called = [] klass = Puppet::Type.newtype(:norefresh) do newparam(:name, :namevar => true) {} def method_missing(method, *args) $called << method end end cleanup do $called = nil Puppet::Type.rmtype(:norefresh) end file = Puppet::Type.newfile :path => tempfile(), :content => "yay" one = klass.create :name => "one", :subscribe => file assert_apply(file, one) assert(! $called.include?(:refresh), "Called refresh when it wasn't set as a method") end # Testing #437 - cyclic graphs should throw failures. def test_fail_on_cycle one = Puppet::Type.type(:exec).create(:name => "/bin/echo one") two = Puppet::Type.type(:exec).create(:name => "/bin/echo two") one[:require] = two two[:require] = one config = mk_catalog(one, two) trans = Puppet::Transaction.new(config) assert_raise(Puppet::Error) do trans.prepare end end def test_errors_during_generation type = Puppet::Type.newtype(:failer) do newparam(:name) {} def eval_generate raise ArgumentError, "Invalid value" end def generate raise ArgumentError, "Invalid value" end end cleanup { Puppet::Type.rmtype(:failer) } obj = type.create(:name => "testing") assert_apply(obj) end def test_self_refresh_causes_triggering type = Puppet::Type.newtype(:refresher, :self_refresh => true) do attr_accessor :refreshed, :testing newparam(:name) {} newproperty(:testing) do def sync self.is = self.should :ran_testing end end def refresh @refreshed = true end end cleanup { Puppet::Type.rmtype(:refresher)} obj = type.create(:name => "yay", :testing => "cool") assert(! obj.insync?(obj.retrieve), "fake object is already in sync") # Now make sure it gets refreshed when the change happens assert_apply(obj) assert(obj.refreshed, "object was not refreshed during transaction") end # Testing #433 def test_explicit_dependencies_beat_automatic # Create a couple of different resource sets that have automatic relationships and make sure the manual relationships win rels = {} # First users and groups group = Puppet::Type.type(:group).create(:name => nonrootgroup.name, :ensure => :present) user = Puppet::Type.type(:user).create(:name => nonrootuser.name, :ensure => :present, :gid => group.title) # Now add the explicit relationship group[:require] = user rels[group] = user # Now files d = tempfile() f = File.join(d, "file") file = Puppet::Type.newfile(:path => f, :content => "yay") dir = Puppet::Type.newfile(:path => d, :ensure => :directory, :require => file) rels[dir] = file rels.each do |after, before| config = mk_catalog(before, after) trans = Puppet::Transaction.new(config) str = "from %s to %s" % [before, after] assert_nothing_raised("Failed to create graph %s" % str) do trans.prepare end graph = trans.relationship_graph assert(graph.edge?(before, after), "did not create manual relationship %s" % str) assert(! graph.edge?(after, before), "created automatic relationship %s" % str) end end # #542 - make sure resources in noop mode still notify their resources, # so that users know if a service will get restarted. def test_noop_with_notify path = tempfile epath = tempfile spath = tempfile file = Puppet::Type.newfile(:path => path, :ensure => :file, :title => "file") exec = Puppet::Type.type(:exec).create(:command => "touch %s" % epath, :path => ENV["PATH"], :subscribe => file, :refreshonly => true, :title => 'exec1') exec2 = Puppet::Type.type(:exec).create(:command => "touch %s" % spath, :path => ENV["PATH"], :subscribe => exec, :refreshonly => true, :title => 'exec2') Puppet[:noop] = true assert(file.noop, "file not in noop") assert(exec.noop, "exec not in noop") @logs.clear assert_apply(file, exec, exec2) assert(! FileTest.exists?(path), "Created file in noop") assert(! FileTest.exists?(epath), "Executed exec in noop") assert(! FileTest.exists?(spath), "Executed second exec in noop") assert(@logs.detect { |l| l.message =~ /should be/ and l.source == file.property(:ensure).path}, "did not log file change") assert(@logs.detect { |l| l.message =~ /Would have/ and l.source == exec.path }, "did not log first exec trigger") assert(@logs.detect { |l| l.message =~ /Would have/ and l.source == exec2.path }, "did not log second exec trigger") end def test_only_stop_purging_with_relations files = [] paths = [] 3.times do |i| path = tempfile paths << path file = Puppet::Type.newfile(:path => path, :ensure => :absent, :backup => false, :title => "file%s" % i) File.open(path, "w") { |f| f.puts "" } files << file end files[0][:ensure] = :file files[0][:require] = files[1..2] # Mark the second as purging files[1].purging assert_apply(*files) assert(FileTest.exists?(paths[1]), "Deleted required purging file") assert(! FileTest.exists?(paths[2]), "Did not delete non-purged file") end def test_flush $state = "absent" $flushed = 0 type = Puppet::Type.newtype(:flushtest) do newparam(:name) newproperty(:ensure) do def retrieve $state end def set(value) $state = value :thing_changed end end def flush $flushed += 1 end end cleanup { Puppet::Type.rmtype(:flushtest) } obj = type.create(:name => "test", :ensure => "present") # first make sure it runs through and flushes assert_apply(obj) assert_equal("present", $state, "Object did not make a change") assert_equal(1, $flushed, "object was not flushed") # Now run a noop and make sure we don't flush obj[:ensure] = "other" obj[:noop] = true assert_apply(obj) assert_equal("present", $state, "Object made a change in noop") assert_equal(1, $flushed, "object was flushed in noop") end end diff --git a/test/rails/ast.rb b/test/rails/ast.rb index e51fa6cf7..1deaec0f4 100755 --- a/test/rails/ast.rb +++ b/test/rails/ast.rb @@ -1,73 +1,73 @@ #!/usr/bin/env ruby require File.dirname(__FILE__) + '/../lib/puppettest' require 'puppettest' require 'puppet/rails' require 'puppet/parser/parser' require 'puppettest/resourcetesting' require 'puppettest/parsertesting' require 'puppettest/railstesting' require 'puppettest/support/collection' class TestRailsAST < PuppetTest::TestCase confine "Missing rails" => Puppet.features.rails? include PuppetTest::RailsTesting include PuppetTest::ParserTesting include PuppetTest::ResourceTesting include PuppetTest::Support::Collection def test_exported_collexp railsinit Puppet[:storeconfigs] = true @scope = mkscope # make a rails resource railsresource "file", "/tmp/testing", :owner => "root", :group => "bin", :mode => "644" run_collection_queries(:exported) do |string, result, query| code = nil str = nil # We don't support more than one search criteria at the moment. retval = nil bad = false # Figure out if the search is for anything rails will ignore if string =~ /\band\b|\bor\b/ bad = true else bad = false end # And if it is, make sure we throw an error. if bad assert_raise(Puppet::ParseError, "Evaluated '#{string}'") do - str, code = query.evaluate :scope => @scope + str, code = query.evaluate @scope end next else assert_nothing_raised("Could not evaluate '#{string}'") do - str, code = query.evaluate :scope => @scope + str, code = query.evaluate @scope end end assert_nothing_raised("Could not find resource") do retval = Puppet::Rails::Resource.find(:all, :include => {:param_values => :param_name}, :conditions => str) end if result assert_equal(1, retval.length, "Did not find resource with '#{string}'") res = retval.shift assert_equal("File", res.restype) assert_equal("/tmp/testing", res.title) else assert_equal(0, retval.length, "found a resource with '#{string}'") end end end end diff --git a/test/rails/configuration.rb b/test/rails/configuration.rb index 9e2ddfedd..a878d1381 100755 --- a/test/rails/configuration.rb +++ b/test/rails/configuration.rb @@ -1,71 +1,71 @@ #!/usr/bin/env ruby require File.dirname(__FILE__) + '/../lib/puppettest' require 'puppettest' require 'puppet/parser/parser' require 'puppet/network/client' require 'puppet/rails' require 'puppettest/resourcetesting' require 'puppettest/parsertesting' require 'puppettest/servertest' require 'puppettest/railstesting' class ConfigurationRailsTests < PuppetTest::TestCase include PuppetTest include PuppetTest::ServerTest include PuppetTest::ParserTesting include PuppetTest::ResourceTesting include PuppetTest::RailsTesting AST = Puppet::Parser::AST confine "No rails support" => Puppet.features.rails? # We need to make sure finished objects are stored in the db. def test_finish_before_store railsinit - compile = mkcompile + compile = mkcompiler parser = compile.parser node = parser.newnode [compile.node.name], :code => AST::ASTArray.new(:children => [ resourcedef("file", "/tmp/yay", :group => "root"), defaultobj("file", :owner => "root") ]) # Now do the rails crap Puppet[:storeconfigs] = true Puppet::Rails::Host.expects(:store).with do |node, resources| if res = resources.find { |r| r.type == "File" and r.title == "/tmp/yay" } assert_equal("root", res["owner"], "Did not set default on resource") true else raise "Resource was not passed to store()" end end compile.compile end def test_hoststorage assert_nothing_raised { Puppet[:storeconfigs] = true } Puppet[:code] = "file { \"/etc\": owner => root }" interp = Puppet::Parser::Interpreter.new facts = {} Facter.each { |fact, val| facts[fact] = val } node = mknode(facts["hostname"]) node.parameters = facts objects = nil assert_nothing_raised { objects = interp.compile(node) } obj = Puppet::Rails::Host.find_by_name(node.name) assert(obj, "Could not find host object") end end diff --git a/test/ral/manager/type.rb b/test/ral/manager/type.rb index 6c5587ddd..324550acb 100755 --- a/test/ral/manager/type.rb +++ b/test/ral/manager/type.rb @@ -1,816 +1,816 @@ #!/usr/bin/env ruby require File.dirname(__FILE__) + '/../../lib/puppettest' require 'puppettest' require 'mocha' class TestType < Test::Unit::TestCase include PuppetTest def test_typemethods Puppet::Type.eachtype { |type| name = nil assert_nothing_raised("Searching for name for %s caused failure" % type.to_s) { name = type.name } assert(name, "Could not find name for %s" % type.to_s) assert_equal( type, Puppet::Type.type(name), "Failed to retrieve %s by name" % name ) # Skip types with no parameters or valid properties #unless ! type.parameters.empty? or ! type.validproperties.empty? # next #end assert_nothing_raised { assert( type.namevar, "Failed to retrieve namevar for %s" % name ) assert_not_nil( type.properties, "Properties for %s are nil" % name ) assert_not_nil( type.validproperties, "Valid properties for %s are nil" % name ) } } end def test_stringvssymbols file = nil path = tempfile() assert_nothing_raised() { system("rm -f %s" % path) file = Puppet.type(:file).create( :path => path, :ensure => "file", :recurse => true, :checksum => "md5" ) } assert_nothing_raised() { file.retrieve } assert_nothing_raised() { file.evaluate } Puppet.type(:file).clear assert_nothing_raised() { system("rm -f %s" % path) file = Puppet.type(:file).create( "path" => path, "ensure" => "file", "recurse" => true, "checksum" => "md5" ) } assert_nothing_raised() { file.retrieve } assert_nothing_raised() { file[:path] } assert_nothing_raised() { file["path"] } assert_nothing_raised() { file[:recurse] } assert_nothing_raised() { file["recurse"] } assert_nothing_raised() { file.evaluate } end # This was supposed to test objects whose name was a property, but that # fundamentally doesn't make much sense, and we now don't have any such # types. def disabled_test_nameasproperty # currently groups are the only objects with the namevar as a property group = nil assert_nothing_raised { group = Puppet.type(:group).create( :name => "testing" ) } assert_equal("testing", group.name, "Could not retrieve name") end # Verify that values get merged correctly def test_mergepropertyvalues file = tempfile() # Create the first version assert_nothing_raised { Puppet.type(:file).create( :path => file, :owner => ["root", "bin"] ) } # Make sure no other statements are allowed assert_raise(Puppet::Error) { Puppet.type(:file).create( :path => file, :group => "root" ) } end def test_aliases_to_self_are_not_failures resource = Puppet.type(:file).create( :name => "/path/to/some/missing/file", :ensure => "file" ) resource.stubs(:path).returns("") catalog = stub 'catalog' catalog.expects(:resource).with(:file, "/path/to/some/missing/file").returns(resource) resource.catalog = catalog # Verify our adding ourselves as an alias isn't an error. assert_nothing_raised("Could not add alias") { resource[:alias] = "/path/to/some/missing/file" } assert_equal(resource.object_id, Puppet.type(:file)["/path/to/some/missing/file"].object_id, "Could not retrieve alias to self") end def test_aliases_are_added_to_class_and_catalog resource = Puppet.type(:file).create( :name => "/path/to/some/missing/file", :ensure => "file" ) resource.stubs(:path).returns("") catalog = stub 'catalog' catalog.stubs(:resource).returns(nil) catalog.expects(:alias).with(resource, "funtest") resource.catalog = catalog assert_nothing_raised("Could not add alias") { resource[:alias] = "funtest" } assert_equal(resource.object_id, Puppet.type(:file)["funtest"].object_id, "Could not retrieve alias") end def test_aliasing_fails_without_a_catalog resource = Puppet.type(:file).create( :name => "/no/such/file", :ensure => "file" ) assert_raise(Puppet::Error, "Did not fail to alias when no catalog was available") { resource[:alias] = "funtest" } end def test_catalogs_are_set_during_initialization_if_present_on_the_transobject trans = Puppet::TransObject.new("/path/to/some/file", :file) trans.catalog = :my_config resource = trans.to_type assert_equal(resource.catalog, trans.catalog, "Did not set catalog on initialization") end # Verify that requirements don't depend on file order def test_prereqorder one = tempfile() two = tempfile() twoobj = nil oneobj = nil assert_nothing_raised("Could not create prereq that doesn't exist yet") { twoobj = Puppet.type(:file).create( :name => two, :require => [:file, one] ) } assert_nothing_raised { oneobj = Puppet.type(:file).create( :name => one ) } comp = mk_catalog(twoobj, oneobj) assert_nothing_raised { comp.finalize } assert(twoobj.requires?(oneobj), "Requirement was not created") end # Verify that names are aliases, not equivalents def test_nameasalias file = nil # Create the parent dir, so we make sure autorequiring the parent dir works parentdir = tempfile() dir = Puppet.type(:file).create( :name => parentdir, :ensure => "directory" ) assert_apply(dir) path = File.join(parentdir, "subdir") name = "a test file" transport = Puppet::TransObject.new(name, "file") transport[:path] = path transport[:ensure] = "file" assert_nothing_raised { file = transport.to_type } assert_equal(path, file[:path]) assert_equal(name, file.title) assert_nothing_raised { file.retrieve } assert_apply(file) assert(Puppet.type(:file)[name], "Could not look up object by name") end def test_ensuredefault user = nil assert_nothing_raised { user = Puppet.type(:user).create( :name => "pptestAA", :check => [:uid] ) } # make sure we don't get :ensure for unmanaged files assert(! user.property(:ensure), "User got an ensure property") assert_nothing_raised { user = Puppet.type(:user).create( :name => "pptestAB", :comment => "Testingness" ) } # but make sure it gets added once we manage them assert(user.property(:ensure), "User did not add ensure property") assert_nothing_raised { user = Puppet.type(:user).create( :name => "pptestBC", :comment => "A fake user" ) } # and make sure managed objects start with them assert(user.property(:ensure), "User did not get an ensure property") end def test_newtype_methods assert_nothing_raised { Puppet::Type.newtype(:mytype) do newparam(:wow) do isnamevar end end } assert(Puppet::Type.respond_to?(:newmytype), "new method did not get created") obj = nil assert_nothing_raised { obj = Puppet::Type.newmytype(:wow => "yay") } assert(obj.is_a?(Puppet::Type.type(:mytype)), "Obj is not the correct type") # Now make the type again, just to make sure it works on refreshing. assert_nothing_raised { Puppet::Type.newtype(:mytype) do newparam(:yay) do isnamevar end end } obj = nil # Make sure the old class was thrown away and only the new one is sitting # around. assert_raise(Puppet::Error) { obj = Puppet::Type.newmytype(:wow => "yay") } assert_nothing_raised { obj = Puppet::Type.newmytype(:yay => "yay") } # Now make sure that we don't replace existing, non-type methods parammethod = Puppet::Type.method(:newparam) assert_nothing_raised { Puppet::Type.newtype(:param) do newparam(:rah) do isnamevar end end } assert_equal(parammethod, Puppet::Type.method(:newparam), "newparam method got replaced by newtype") end def test_newproperty_options # Create a type with a fake provider providerclass = Class.new do def self.supports_parameter?(prop) return true end def method_missing(method, *args) return method end end self.class.const_set("ProviderClass", providerclass) type = Puppet::Type.newtype(:mytype) do newparam(:name) do isnamevar end def provider @provider ||= ProviderClass.new @provider end end # Now make a property with no options. property = nil assert_nothing_raised do property = type.newproperty(:noopts) do end end # Now create an instance obj = type.create(:name => :myobj) inst = property.new(:resource => obj) # And make sure it's correctly setting @is ret = nil assert_nothing_raised { ret = inst.retrieve } assert_equal(:noopts, inst.retrieve) # Now create a property with a different way of doing it property = nil assert_nothing_raised do property = type.newproperty(:setretrieve, :retrieve => :yayness) end inst = property.new(:resource => obj) # And make sure it's correctly setting @is ret = nil assert_nothing_raised { ret = inst.retrieve } assert_equal(:yayness, ret) end def test_name_vs_title path = tempfile() trans = nil assert_nothing_raised { trans = Puppet::TransObject.new(path, :file) } file = nil assert_nothing_raised { file = Puppet::Type.newfile(trans) } assert(file.respond_to?(:title), "No 'title' method") assert(file.respond_to?(:name), "No 'name' method") assert_equal(file.title, file.name, "Name and title were not marked equal") assert_nothing_raised { file.title = "My file" } assert_equal("My file", file.title) assert_equal(path, file.name) end # Make sure the title is sufficiently differentiated from the namevar. def test_title_at_creation_with_hash file = nil fileclass = Puppet::Type.type(:file) path = tempfile() assert_nothing_raised do file = fileclass.create( :title => "Myfile", :path => path ) end assert_equal("Myfile", file.title, "Did not get correct title") assert_equal(path, file[:name], "Did not get correct name") file = nil Puppet::Type.type(:file).clear # Now make sure we can specify both and still get the right answers assert_nothing_raised do file = fileclass.create( :title => "Myfile", :name => path ) end assert_instance_of(fileclass, file) assert_equal("Myfile", file.title, "Did not get correct title") assert_equal(path, file[:name], "Did not get correct name") end # Make sure the "create" class method behaves appropriately. def test_class_create title = "Myfile" validate = proc do |element| assert(element, "Did not create file") assert_instance_of(Puppet::Type.type(:file), element) assert_equal(title, element.title, "Title is not correct") end type = :file args = {:path => tempfile(), :owner => "root"} trans = Puppet::TransObject.new(title, type) args.each do |name, val| trans[name] = val end # First call it on the appropriate typeclass obj = nil assert_nothing_raised do obj = Puppet::Type.type(:file).create(trans) end validate.call(obj) # Now try it using the class method on Type oldid = obj.object_id obj = nil Puppet::Type.type(:file).clear assert_nothing_raised { obj = Puppet::Type.create(trans) } validate.call(obj) assert(oldid != obj.object_id, "Got same object back") # Now try the same things with hashes instead of a transobject oldid = obj.object_id obj = nil Puppet::Type.type(:file).clear hash = { :type => :file, :title => "Myfile", :path => tempfile(), :owner => "root" } # First call it on the appropriate typeclass obj = nil assert_nothing_raised do obj = Puppet::Type.type(:file).create(hash) end validate.call(obj) assert_equal(:file, obj.should(:type), "Type param did not pass through") assert(oldid != obj.object_id, "Got same object back") # Now try it using the class method on Type oldid = obj.object_id obj = nil Puppet::Type.type(:file).clear assert_nothing_raised { obj = Puppet::Type.create(hash) } validate.call(obj) assert(oldid != obj.object_id, "Got same object back") assert_nil(obj.should(:type), "Type param passed through") end def test_multiplenames obj = nil path = tempfile() assert_raise ArgumentError do obj = Puppet::Type.type(:file).create( :name => path, :path => path ) end end def test_title_and_name obj = nil path = tempfile() fileobj = Puppet::Type.type(:file) assert_nothing_raised do obj = fileobj.create( :title => "myfile", :path => path ) end assert_equal(obj, fileobj["myfile"], "Could not retrieve obj by title") assert_equal(obj, fileobj[path], "Could not retrieve obj by name") end # Make sure default providers behave correctly def test_defaultproviders # Make a fake type type = Puppet::Type.newtype(:defaultprovidertest) do newparam(:name) do end end basic = type.provide(:basic) do defaultfor :operatingsystem => :somethingelse, :operatingsystemrelease => :yayness end assert_equal(basic, type.defaultprovider) type.defaultprovider = nil greater = type.provide(:greater) do defaultfor :operatingsystem => Facter.value("operatingsystem") end assert_equal(greater, type.defaultprovider) end - # Make sure that we can have multiple isomorphic objects with the same name, - # but not with non-isomorphic objects. + # Make sure that we can have multiple non-isomorphic objects with the same name, + # but not with isomorphic objects. def test_isomorphic_names # First do execs, since they're not isomorphic. echo = Puppet::Util.binary "echo" exec1 = exec2 = nil assert_nothing_raised do exec1 = Puppet::Type.type(:exec).create( :title => "exec1", :command => "#{echo} funtest" ) end assert_nothing_raised do exec2 = Puppet::Type.type(:exec).create( :title => "exec2", :command => "#{echo} funtest" ) end assert_apply(exec1, exec2) # Now do files, since they are. This should fail. file1 = file2 = nil path = tempfile() assert_nothing_raised do file1 = Puppet::Type.type(:file).create( :title => "file1", :path => path, :content => "yayness" ) end # This will fail, but earlier systems will catch it. assert_raise(Puppet::Error) do file2 = Puppet::Type.type(:file).create( :title => "file2", :path => path, :content => "rahness" ) end assert(file1, "Did not create first file") assert_nil(file2, "Incorrectly created second file") end def test_tags obj = Puppet::Type.type(:file).create(:path => tempfile()) tags = [:some, :test, :tags] obj.tags = tags assert_equal(tags + [:file], obj.tags) end def disabled_test_list Puppet::Type.loadall Puppet::Type.eachtype do |type| next if type.name == :symlink next if type.name == :component next if type.name == :tidy assert(type.respond_to?(:list), "%s does not respond to list" % type.name) end end def test_to_hash file = Puppet::Type.newfile :path => tempfile(), :owner => "luke", :recurse => true, :loglevel => "warning" hash = nil assert_nothing_raised do hash = file.to_hash end [:path, :owner, :recurse, :loglevel].each do |param| assert(hash[param], "Hash did not include %s" % param) end end # Make sure that classes behave like hashes. def test_class_hash_behaviour path = tempfile() filetype = Puppet::Type.type(:file) one = Puppet::Type.newfile :path => path assert_equal(one, filetype[path], "Did not get file back") assert_raise(Puppet::Error) do filetype[path] = one end end def test_ref path = tempfile() Puppet::Type.type(:exec) # uggh, the methods need to load the types file = Puppet::Type.newfile(:path => path) assert_equal("File[#{path}]", file.ref) exec = Puppet::Type.newexec(:title => "yay", :command => "/bin/echo yay") assert_equal("Exec[yay]", exec.ref) end def test_noop_metaparam file = Puppet::Type.newfile :path => tempfile assert(!file.noop, "file incorrectly in noop") assert_nothing_raised do file[:noop] = true end assert(file.noop, "file should be in noop") # Now set the main one Puppet[:noop] = true assert(file.noop, "file should be in noop") file[:noop] = false assert(file.noop, "file should be in noop") end def test_path config = mk_catalog # Check that our paths are built correctly. Just pick a random, "normal" type. type = Puppet::Type.type(:exec) mk = Proc.new do |i, hash| hash[:title] = "exec%s" % i hash[:command] = "/bin/echo" if parent = hash[:parent] hash.delete(:parent) end res = type.create(hash) config.add_resource res if parent config.add_edge(parent, res) end res end exec = mk.call(1, {}) assert_equal("/Exec[exec1]", exec.path) comp = Puppet::Type.newcomponent :title => "My[component]", :type => "Yay" config.add_resource comp exec = mk.call(2, :parent => comp) assert_equal("/My[component]/Exec[exec2]", exec.path) comp = Puppet::Type.newcomponent :name => "Other[thing]" config.add_resource comp exec = mk.call(3, :parent => comp) assert_equal("/Other[thing]/Exec[exec3]", exec.path) comp = Puppet::Type.newcomponent :type => "server", :name => "server" config.add_resource comp exec = mk.call(4, :parent => comp) assert_equal("/server/Exec[exec4]", exec.path) comp = Puppet::Type.newcomponent :type => "whatever", :name => "class[main]" config.add_resource comp exec = mk.call(5, :parent => comp) assert_equal("//Exec[exec5]", exec.path) newcomp = Puppet::Type.newcomponent :type => "yay", :name => "Good[bad]" config.add_resource newcomp config.add_edge comp, newcomp exec = mk.call(6, :parent => newcomp) assert_equal("//Good[bad]/Exec[exec6]", exec.path) end def test_evaluate faketype = Puppet::Type.newtype(:faketype) do newparam(:name) {} end cleanup { Puppet::Type.rmtype(:faketype) } faketype.provide(:fake) do def prefetch end end obj = faketype.create :name => "yayness", :provider => :fake assert(obj, "did not create object") obj.provider.expects(:prefetch) obj.expects(:retrieve) obj.expects(:propertychanges).returns([]) obj.expects(:cache) obj.evaluate end # Partially test #704, but also cover the rest of the schedule management bases. def test_schedule Puppet::Type.type(:schedule).create(:name => "maint") {"maint" => true, nil => false, :fail => :fail}.each do |name, should| args = {:name => tempfile, :ensure => :file} if name args[:schedule] = name end resource = Puppet::Type.type(:file).create(args) if should == :fail assert_raise(Puppet::Error, "Did not fail on missing schedule") do resource.schedule end else sched = nil assert_nothing_raised("Failed when schedule was %s" % sched) do sched = resource.schedule end if should assert_equal(name, sched.name, "did not get correct schedule back") end end end end # #801 -- resources only checked in noop should be rescheduled immediately. def test_reschedule_when_noop Puppet::Type.type(:schedule).mkdefaultschedules file = Puppet::Type.type(:file).create(:path => "/tmp/whatever", :mode => "755", :noop => true, :schedule => :daily, :ensure => :file) assert(file.noop?, "File not considered in noop") assert(file.scheduled?, "File is not considered scheduled") file.evaluate assert_nil(file.cached(:checked), "Stored a checked time when running in noop mode when there were changes") file.cache(:checked, nil) file.stubs(:propertychanges).returns([]) file.evaluate assert_instance_of(Time, file.cached(:checked), "Did not store a checked time when running in noop mode when there were no changes") end end diff --git a/test/ral/providers/cron/crontab.rb b/test/ral/providers/cron/crontab.rb index 2da4b1b57..1ff1e34ef 100755 --- a/test/ral/providers/cron/crontab.rb +++ b/test/ral/providers/cron/crontab.rb @@ -1,602 +1,604 @@ #!/usr/bin/env ruby require File.dirname(__FILE__) + '/../../../lib/puppettest' require 'puppettest' require 'mocha' require 'puppettest/fileparsing' require 'puppet/type/cron' class TestCronParsedProvider < Test::Unit::TestCase include PuppetTest include PuppetTest::FileParsing FIELDS = { :crontab => %w{command minute hour month monthday weekday}.collect { |o| o.intern }, :freebsd_special => %w{special command}.collect { |o| o.intern }, :environment => [:line], :blank => [:line], :comment => [:line], } # These are potentially multi-line records; there's no one-to-one map, but they model # a full cron job. These tests assume individual record types will always be correctly # parsed, so all they def sample_crons unless defined? @sample_crons @sample_crons = YAML.load(File.read(File.join(@crondir, "crontab_collections.yaml"))) end @sample_crons end # These are simple lines that can appear in the files; there is a one to one # mapping between records and lines. We have plenty of redundancy here because # we use these records to build up our complex, multi-line cron jobs below. def sample_records unless defined? @sample_records @sample_records = YAML.load(File.read(File.join(@crondir, "crontab_sample_records.yaml"))) end @sample_records end def setup super @type = Puppet::Type.type(:cron) @provider = @type.provider(:crontab) @provider.initvars @crondir = datadir(File.join(%w{providers cron})) @oldfiletype = @provider.filetype end def teardown Puppet::Util::FileType.filetype(:ram).clear @provider.clear super end # Make sure a cron job matches up. Any non-passed fields are considered absent. def assert_cron_equal(msg, cron, options) assert_instance_of(@provider, cron, "not an instance of provider in %s" % msg) options.each do |param, value| assert_equal(value, cron.send(param), "%s was not equal in %s" % [param, msg]) end %w{command environment minute hour month monthday weekday}.each do |var| unless options.include?(var.intern) assert_equal(:absent, cron.send(var), "%s was not parsed absent in %s" % [var, msg]) end end end # Make sure a cron record matches. This only works for crontab records. def assert_record_equal(msg, record, options) unless options.include?(:record_type) raise ArgumentError, "You must pass the required record type" end assert_instance_of(Hash, record, "not an instance of a hash in %s" % msg) options.each do |param, value| assert_equal(value, record[param], "%s was not equal in %s" % [param, msg]) end FIELDS[record[:record_type]].each do |var| unless options.include?(var) assert_equal(:absent, record[var], "%s was not parsed absent in %s" % [var, msg]) end end end def assert_header(file) header = [] file.gsub! /^(# HEADER: .+$)\n/ do header << $1 '' end assert_equal(4, header.length, "Did not get four header lines") end # This handles parsing every possible iteration of cron records. Note that this is only # single-line stuff and doesn't include multi-line values (e.g., with names and/or envs). # Those have separate tests. def test_parse_line # First just do each sample record one by one sample_records.each do |name, options| result = nil assert_nothing_raised("Could not parse %s: '%s'" % [name, options[:text]]) do result = @provider.parse_line(options[:text]) end assert_record_equal("record for %s" % name, result, options[:record]) end # Then do them all at once. records = [] text = "" sample_records.each do |name, options| records << options[:record] text += options[:text] + "\n" end result = nil assert_nothing_raised("Could not match all records in one file") do result = @provider.parse(text) end records.zip(result).each do |should, record| assert_record_equal("record for %s in full match" % should.inspect, record, should) end end # Here we test that each record generates to the correct text. def test_generate_line # First just do each sample record one by one sample_records.each do |name, options| result = nil assert_nothing_raised("Could not generate %s: '%s'" % [name, options[:record]]) do result = @provider.to_line(options[:record]) end assert_equal(options[:text], result, "Did not generate correct text for %s" % name) end # Then do them all at once. records = [] text = "" sample_records.each do |name, options| records << options[:record] text += options[:text] + "\n" end result = nil assert_nothing_raised("Could not match all records in one file") do result = @provider.to_file(records) end assert_header(result) assert_equal(text, result, "Did not generate correct full crontab") end # Test cronjobs that are made up from multiple records. def test_multi_line_cronjobs fulltext = "" all_records = [] sample_crons.each do |name, record_names| records = record_names.collect do |record_name| unless record = sample_records[record_name] raise "Could not find sample record %s" % record_name end record end text = records.collect { |r| r[:text] }.join("\n") + "\n" record_list = records.collect { |r| r[:record] } # Add it to our full collection all_records += record_list fulltext += text # First make sure we generate each one correctly result = nil assert_nothing_raised("Could not generate multi-line cronjob %s" % [name]) do result = @provider.to_file(record_list) end assert_header(result) assert_equal(text, result, "Did not generate correct text for multi-line cronjob %s" % name) # Now make sure we parse each one correctly assert_nothing_raised("Could not parse multi-line cronjob %s" % [name]) do result = @provider.parse(text) end record_list.zip(result).each do |should, record| assert_record_equal("multiline cronjob %s" % name, record, should) end end # Make sure we can generate it all correctly result = nil assert_nothing_raised("Could not generate all multi-line cronjobs") do result = @provider.to_file(all_records) end assert_header(result) assert_equal(fulltext, result, "Did not generate correct text for all multi-line cronjobs") # Now make sure we parse them all correctly assert_nothing_raised("Could not parse multi-line cronjobs") do result = @provider.parse(fulltext) end all_records.zip(result).each do |should, record| assert_record_equal("multiline cronjob %s", record, should) end end # Take our sample files, and make sure we can entirely parse them, # then that we can generate them again and we get the same data. def test_parse_and_generate_sample_files @provider.stubs(:filetype).returns(Puppet::Util::FileType.filetype(:ram)) crondir = datadir(File.join(%w{providers cron})) files = Dir.glob("%s/crontab.*" % crondir) setme @provider.default_target = @me target = @provider.target_object(@me) files.each do |file| str = args = nil assert_nothing_raised("could not load %s" % file) do str, args = YAML.load(File.read(file)) end # Stupid old yaml args.each do |hash| hash.each do |param, value| if param.is_a?(String) and param =~ /^:/ hash.delete(param) param = param.sub(/^:/,'').intern hash[param] = value end if value.is_a?(String) and value =~ /^:/ value = value.sub(/^:/,'').intern hash[param] = value end end end target.write(str) assert_nothing_raised("could not parse %s" % file) do @provider.prefetch end records = @provider.send(:instance_variable_get, "@records") args.zip(records) do |should, sis| # Make the values a bit more equal. should[:target] = @me should[:ensure] = :present #should[:environment] ||= [] should[:on_disk] = true is = sis.dup sis.dup.each do |p,v| is.delete(p) if v == :absent end assert_equal(should, is, "Did not parse %s correctly" % file) end assert_nothing_raised("could not generate %s" % file) do @provider.flush_target(@me) end assert_equal(str, target.read, "%s changed" % file) @provider.clear end end # A simple test to see if we can load the cron from disk. def test_load setme() records = nil assert_nothing_raised { records = @provider.retrieve(@me) } assert_instance_of(Array, records, "did not get correct response") end # Test that a cron job turns out as expected, by creating one and generating # it directly def test_simple_to_cron # make the cron setme() name = "yaytest" args = {:name => name, :command => "date > /dev/null", :minute => "30", :user => @me, :record_type => :crontab } # generate the text str = nil assert_nothing_raised { str = @provider.to_line(args) } assert_equal("# Puppet Name: #{name}\n30 * * * * date > /dev/null", str, "Cron did not generate correctly") end # Test that comments are correctly retained def test_retain_comments str = "# this is a comment\n#and another comment\n" user = "fakeuser" records = nil @provider.stubs(:filetype).returns(Puppet::Util::FileType.filetype(:ram)) target = @provider.target_object(user) target.write(str) assert_nothing_raised { @provider.prefetch } assert_nothing_raised { newstr = @provider.flush_target(user) assert(target.read.include?(str), "Comments were lost") } end def test_simpleparsing @provider.stubs(:filetype).returns(Puppet::Util::FileType.filetype(:ram)) text = "5 1,2 * 1 0 /bin/echo funtest" records = nil assert_nothing_raised { records = @provider.parse(text) } should = { :minute => %w{5}, :hour => %w{1 2}, :monthday => :absent, :month => %w{1}, :weekday => %w{0}, :command => "/bin/echo funtest" } is = records.shift assert(is, "Did not get record") should.each do |p, v| assert_equal(v, is[p], "did not parse %s correctly" % p) end end - # Make sure we can create a cron in an empty tab + # Make sure we can create a cron in an empty tab. + # LAK:FIXME This actually modifies the user's crontab, + # which is pretty heinous. def test_mkcron_if_empty setme @provider.filetype = @oldfiletype records = @provider.retrieve(@me) target = @provider.target_object(@me) cleanup do if records.length == 0 target.remove else target.write(@provider.to_file(records)) end end # Now get rid of it assert_nothing_raised("Could not remove cron tab") do target.remove end @provider.flush :target => @me, :command => "/do/something", :record_type => :crontab created = @provider.retrieve(@me) assert(created.detect { |r| r[:command] == "/do/something" }, "Did not create cron tab") end # Make sure we correctly bidirectionally parse things. def test_records_and_strings @provider.stubs(:filetype).returns(Puppet::Util::FileType.filetype(:ram)) setme target = @provider.target_object(@me) [ "* * * * * /some/command", "0,30 * * * * /some/command", "0-30 * * * * /some/command", "# Puppet Name: name\n0-30 * * * * /some/command", "# Puppet Name: name\nVAR=VALUE\n0-30 * * * * /some/command", "# Puppet Name: name\nVAR=VALUE\nC=D\n0-30 * * * * /some/command", "0 * * * * /some/command" ].each do |str| @provider.initvars str += "\n" target.write(str) assert_equal(str, target.read, "Did not write correctly") assert_nothing_raised("Could not prefetch with %s" % str.inspect) do @provider.prefetch end assert_nothing_raised("Could not flush with %s" % str.inspect) do @provider.flush_target(@me) end assert_equal(str, target.read, "Changed in read/write") @provider.clear end end # Test that a specified cron job will be matched against an existing job # with no name, as long as all fields match def test_matchcron mecron = "0,30 * * * * date * * * * * funtest # a comment 0,30 * * 1 * date " youcron = "0,30 * * * * date * * * * * yaytest # a comment 0,30 * * 1 * fooness " setme @provider.stubs(:filetype).returns(Puppet::Util::FileType.filetype(:ram)) you = "you" # Write the same tab to multiple targets @provider.target_object(@me).write(mecron.gsub(/^\s+/, '')) @provider.target_object(you).write(youcron.gsub(/^\s+/, '')) # Now make some crons that should match matchers = [ @type.create( :name => "yaycron", :minute => [0, 30], :command => "date", :user => @me ), @type.create( :name => "youtest", :command => "yaytest", :user => you ) ] nonmatchers = [ @type.create( :name => "footest", :minute => [0, 30], :hour => 1, :command => "fooness", :user => @me # wrong target ), @type.create( :name => "funtest2", :command => "funtest", :user => you # wrong target for this cron ) ] # Create another cron so we prefetch two of them @type.create(:name => "testing", :minute => 30, :command => "whatever", :user => "you") assert_nothing_raised("Could not prefetch cron") do @provider.prefetch([matchers, nonmatchers].flatten.inject({}) { |crons, cron| crons[cron.name] = cron; crons }) end matchers.each do |cron| assert_equal(:present, cron.provider.ensure, "Cron %s was not matched" % cron.name) if value = cron.value(:minute) and value == "*" value = :absent end assert_equal(value, cron.provider.minute, "Minutes were not retrieved, so cron was not matched") assert_equal(cron.value(:target), cron.provider.target, "Cron %s was matched from the wrong target" % cron.name) end nonmatchers.each do |cron| assert_equal(:absent, cron.provider.ensure, "Cron %s was incorrectly matched" % cron.name) end end def test_data setme @provider.stubs(:filetype).returns(Puppet::Util::FileType.filetype(:ram)) target = @provider.target_object(@me) fakedata("data/providers/cron/examples").each do |file| text = File.read(file) target.write(text) assert_nothing_raised("Could not parse %s" % file) do @provider.prefetch end # mark the provider modified @provider.modified(@me) # and zero the text target.write("") result = nil assert_nothing_raised("Could not generate %s" % file) do @provider.flush_target(@me) end # Ignore whitespace differences, since those don't affect function. modtext = text.gsub(/[ \t]+/, " ") modtarget = target.read.gsub(/[ \t]+/, " ") assert_equal(modtext, modtarget, "File was not rewritten the same") @provider.clear end end # Match freebsd's annoying @daily stuff. def test_match_freebsd_special @provider.stubs(:filetype).returns(Puppet::Util::FileType.filetype(:ram)) setme target = @provider.target_object(@me) [ "@daily /some/command", "@daily /some/command more" ].each do |str| @provider.initvars str += "\n" target.write(str) assert_equal(str, target.read, "Did not write correctly") assert_nothing_raised("Could not prefetch with %s" % str.inspect) do @provider.prefetch end records = @provider.send(:instance_variable_get, "@records") records.each do |r| assert_equal(:freebsd_special, r[:record_type], "Did not create lines as freebsd lines") end assert_nothing_raised("Could not flush with %s" % str.inspect) do @provider.flush_target(@me) end assert_equal(str, target.read, "Changed in read/write") @provider.clear end end def test_prefetch cron = @type.create :command => "/bin/echo yay", :name => "test", :hour => 4 assert_nothing_raised("Could not prefetch cron") do cron.provider.class.prefetch("test" => cron) end end # Testing #669. def test_environment_settings @provider.stubs(:filetype).returns(Puppet::Util::FileType.filetype(:ram)) setme target = @provider.target_object(@me) # First with no env settings resource = @type.create :command => "/bin/echo yay", :name => "test", :hour => 4 cron = resource.provider cron.ensure = :present cron.command = "/bin/echo yay" cron.hour = %w{4} cron.flush result = target.read assert_equal("# Puppet Name: test\n* 4 * * * /bin/echo yay\n", result, "Did not write cron out correctly") # Now set the env cron.environment = "TEST=foo" cron.flush result = target.read assert_equal("# Puppet Name: test\nTEST=foo\n* 4 * * * /bin/echo yay\n", result, "Did not write out environment setting") # Modify it cron.environment = ["TEST=foo", "BLAH=yay"] cron.flush result = target.read assert_equal("# Puppet Name: test\nTEST=foo\nBLAH=yay\n* 4 * * * /bin/echo yay\n", result, "Did not write out environment setting") # And remove it cron.environment = :absent cron.flush result = target.read assert_equal("# Puppet Name: test\n* 4 * * * /bin/echo yay\n", result, "Did not write out environment setting") end end diff --git a/test/ral/types/file.rb b/test/ral/types/file.rb index c7872ccea..cbbe818ae 100755 --- a/test/ral/types/file.rb +++ b/test/ral/types/file.rb @@ -1,1843 +1,1782 @@ #!/usr/bin/env ruby require File.dirname(__FILE__) + '/../../lib/puppettest' require 'puppettest' require 'puppettest/support/utils' require 'fileutils' class TestFile < Test::Unit::TestCase include PuppetTest::Support::Utils include PuppetTest::FileTesting def mkfile(hash) file = nil assert_nothing_raised { file = Puppet.type(:file).create(hash) } return file end def mktestfile tmpfile = tempfile() File.open(tmpfile, "w") { |f| f.puts rand(100) } @@tmpfiles.push tmpfile mkfile(:name => tmpfile) end def setup super @file = Puppet::Type.type(:file) $method = @method_name Puppet[:filetimeout] = -1 end def teardown Puppet::Util::Storage.clear system("rm -rf %s" % Puppet[:statefile]) super end def initstorage Puppet::Util::Storage.init Puppet::Util::Storage.load end def clearstorage Puppet::Util::Storage.store Puppet::Util::Storage.clear end def test_owner file = mktestfile() users = {} count = 0 # collect five users Etc.passwd { |passwd| if count > 5 break else count += 1 end users[passwd.uid] = passwd.name } fake = {} # find a fake user while true a = rand(1000) begin Etc.getpwuid(a) rescue fake[a] = "fakeuser" break end end uid, name = users.shift us = {} us[uid] = name users.each { |uid, name| assert_apply(file) assert_nothing_raised() { file[:owner] = name } assert_nothing_raised() { file.retrieve } assert_apply(file) } end def test_group file = mktestfile() [%x{groups}.chomp.split(/ /), Process.groups].flatten.each { |group| assert_nothing_raised() { file[:group] = group } assert(file.property(:group)) assert(file.property(:group).should) } end def test_groups_fails_when_invalid assert_raise(Puppet::Error, "did not fail when the group was empty") do Puppet::Type.type(:file).create :path => "/some/file", :group => "" end end if Puppet::Util::SUIDManager.uid == 0 def test_createasuser dir = tmpdir() user = nonrootuser() path = File.join(tmpdir, "createusertesting") @@tmpfiles << path file = nil assert_nothing_raised { file = Puppet.type(:file).create( :path => path, :owner => user.name, :ensure => "file", :mode => "755" ) } comp = mk_catalog("createusertest", file) assert_events([:file_created], comp) end def test_nofollowlinks basedir = tempfile() Dir.mkdir(basedir) file = File.join(basedir, "file") link = File.join(basedir, "link") File.open(file, "w", 0644) { |f| f.puts "yayness"; f.flush } File.symlink(file, link) # First test 'user' user = nonrootuser() inituser = File.lstat(link).uid File.lchown(inituser, nil, link) obj = nil assert_nothing_raised { obj = Puppet.type(:file).create( :title => link, :owner => user.name ) } obj.retrieve # Make sure it defaults to managing the link assert_events([:file_changed], obj) assert_equal(user.uid, File.lstat(link).uid) assert_equal(inituser, File.stat(file).uid) File.chown(inituser, nil, file) File.lchown(inituser, nil, link) # Try following obj[:links] = :follow assert_events([:file_changed], obj) assert_equal(user.uid, File.stat(file).uid) assert_equal(inituser, File.lstat(link).uid) # And then explicitly managing File.chown(inituser, nil, file) File.lchown(inituser, nil, link) obj[:links] = :manage assert_events([:file_changed], obj) assert_equal(user.uid, File.lstat(link).uid) assert_equal(inituser, File.stat(file).uid) obj.delete(:owner) - obj[:links] = :ignore + obj[:links] = :follow # And then test 'group' group = nonrootgroup initgroup = File.stat(file).gid obj[:group] = group.name assert_events([:file_changed], obj) assert_equal(initgroup, File.stat(file).gid) assert_equal(group.gid, File.lstat(link).gid) File.chown(nil, initgroup, file) File.lchown(nil, initgroup, link) obj[:links] = :follow assert_events([:file_changed], obj) assert_equal(group.gid, File.stat(file).gid) File.chown(nil, initgroup, file) File.lchown(nil, initgroup, link) obj[:links] = :manage assert_events([:file_changed], obj) assert_equal(group.gid, File.lstat(link).gid) assert_equal(initgroup, File.stat(file).gid) end def test_ownerasroot file = mktestfile() users = {} count = 0 # collect five users Etc.passwd { |passwd| if count > 5 break else count += 1 end next if passwd.uid < 0 users[passwd.uid] = passwd.name } fake = {} # find a fake user while true a = rand(1000) begin Etc.getpwuid(a) rescue fake[a] = "fakeuser" break end end users.each { |uid, name| assert_nothing_raised() { file[:owner] = name } changes = [] assert_nothing_raised() { changes << file.evaluate } assert(changes.length > 0) assert_apply(file) currentvalue = file.retrieve assert(file.insync?(currentvalue)) assert_nothing_raised() { file[:owner] = uid } assert_apply(file) currentvalue = file.retrieve # make sure changing to number doesn't cause a sync assert(file.insync?(currentvalue)) } # We no longer raise an error here, because we check at run time #fake.each { |uid, name| # assert_raise(Puppet::Error) { # file[:owner] = name # } # assert_raise(Puppet::Error) { # file[:owner] = uid # } #} end def test_groupasroot file = mktestfile() [%x{groups}.chomp.split(/ /), Process.groups].flatten.each { |group| next unless Puppet::Util.gid(group) # grr. assert_nothing_raised() { file[:group] = group } assert(file.property(:group)) assert(file.property(:group).should) assert_apply(file) currentvalue = file.retrieve assert(file.insync?(currentvalue)) assert_nothing_raised() { file.delete(:group) } } end if Facter.value(:operatingsystem) == "Darwin" def test_sillyowner file = tempfile() File.open(file, "w") { |f| f.puts "" } File.chown(-2, nil, file) assert(File.stat(file).uid > 120000, "eh?") user = nonrootuser obj = Puppet::Type.newfile( :path => file, :owner => user.name ) assert_apply(obj) assert_equal(user.uid, File.stat(file).uid) end end else $stderr.puts "Run as root for complete owner and group testing" end def test_create %w{a b c d}.collect { |name| tempfile() + name.to_s }.each { |path| file =nil assert_nothing_raised() { file = Puppet.type(:file).create( :name => path, :ensure => "file" ) } assert_events([:file_created], file) assert_events([], file) assert(FileTest.file?(path), "File does not exist") assert(file.insync?(file.retrieve)) @@tmpfiles.push path } end def test_create_dir basedir = tempfile() Dir.mkdir(basedir) %w{a b c d}.collect { |name| "#{basedir}/%s" % name }.each { |path| file = nil assert_nothing_raised() { file = Puppet.type(:file).create( :name => path, :ensure => "directory" ) } assert(! FileTest.directory?(path), "Directory %s already exists" % [path]) assert_events([:directory_created], file) assert_events([], file) assert(file.insync?(file.retrieve)) assert(FileTest.directory?(path)) @@tmpfiles.push path } end def test_modes file = mktestfile # Set it to something else initially File.chmod(0775, file.title) [0644,0755,0777,0641].each { |mode| assert_nothing_raised() { file[:mode] = mode } assert_events([:file_changed], file) assert_events([], file) assert(file.insync?(file.retrieve)) assert_nothing_raised() { file.delete(:mode) } } end def test_checksums types = %w{md5 md5lite timestamp time} exists = "/tmp/sumtest-exists" nonexists = "/tmp/sumtest-nonexists" @@tmpfiles << exists @@tmpfiles << nonexists # try it both with files that exist and ones that don't files = [exists, nonexists] initstorage File.open(exists,File::CREAT|File::TRUNC|File::WRONLY) { |of| of.puts "initial text" } types.each { |type| files.each { |path| if Puppet[:debug] Puppet.warning "Testing %s on %s" % [type,path] end file = nil events = nil # okay, we now know that we have a file... assert_nothing_raised() { file = Puppet.type(:file).create( :name => path, :ensure => "file", :checksum => type ) } trans = nil currentvalues = file.retrieve if file.title !~ /nonexists/ sum = file.property(:checksum) assert(sum.insync?(currentvalues[sum]), "file is not in sync") end events = assert_apply(file) assert(events) assert(! events.include?(:file_changed), "File incorrectly changed") assert_events([], file) # We have to sleep because the time resolution of the time-based # mechanisms is greater than one second sleep 1 if type =~ /time/ assert_nothing_raised() { File.open(path,File::CREAT|File::TRUNC|File::WRONLY) { |of| of.puts "some more text, yo" } } Puppet.type(:file).clear # now recreate the file assert_nothing_raised() { file = Puppet.type(:file).create( :name => path, :checksum => type ) } trans = nil assert_events([:file_changed], file) # Run it a few times to make sure we aren't getting # spurious changes. sum = nil assert_nothing_raised do sum = file.property(:checksum).retrieve end assert(file.property(:checksum).insync?(sum), "checksum is not in sync") sleep 1.1 if type =~ /time/ assert_nothing_raised() { File.unlink(path) File.open(path,File::CREAT|File::TRUNC|File::WRONLY) { |of| # We have to put a certain amount of text in here or # the md5-lite test fails 2.times { of.puts rand(100) } of.flush } } assert_events([:file_changed], file) # verify that we're actually getting notified when a file changes assert_nothing_raised() { Puppet.type(:file).clear } if path =~ /nonexists/ File.unlink(path) end } } end def cyclefile(path) # i had problems with using :name instead of :path [:name,:path].each { |param| file = nil changes = nil comp = nil trans = nil initstorage assert_nothing_raised { file = Puppet.type(:file).create( param => path, :recurse => true, :checksum => "md5" ) } comp = Puppet.type(:component).create( :name => "component" ) comp.push file assert_nothing_raised { trans = comp.evaluate } assert_nothing_raised { trans.evaluate } clearstorage Puppet::Type.allclear } end def test_localrecurse # Create a test directory path = tempfile() dir = @file.create :path => path, :mode => 0755, :recurse => true config = mk_catalog(dir) Dir.mkdir(path) # Make sure we return nothing when there are no children ret = nil assert_nothing_raised() { ret = dir.localrecurse(true) } assert_equal([], ret, "empty dir returned children") # Now make a file and make sure we get it test = File.join(path, "file") File.open(test, "w") { |f| f.puts "yay" } assert_nothing_raised() { ret = dir.localrecurse(true) } fileobj = @file[test] assert(fileobj, "child object was not created") assert_equal([fileobj], ret, "child object was not returned") # And that it inherited our recurse setting assert_equal(true, fileobj[:recurse], "file did not inherit recurse") # Make sure it's not returned again assert_nothing_raised() { ret = dir.localrecurse(true) } assert_equal([], ret, "child object was returned twice") # Now just for completion, make sure we will return many files files = [] 10.times do |i| f = File.join(path, i.to_s) files << f File.open(f, "w") do |o| o.puts "" end end assert_nothing_raised() { ret = dir.localrecurse(true) } assert_equal(files.sort, ret.collect { |f| f.title }.sort, "child object was returned twice") # Clean everything up and start over files << test files.each do |f| File.unlink(f) end # Now make sure we correctly ignore things dir[:ignore] = "*.out" bad = File.join(path, "test.out") good = File.join(path, "yayness") [good, bad].each do |f| File.open(f, "w") { |o| o.puts "" } end assert_nothing_raised() { ret = dir.localrecurse(true) } assert_equal([good], ret.collect { |f| f.title }, "ignore failed") # Now make sure purging works dir[:purge] = true dir[:ignore] = "svn" assert_nothing_raised() { ret = dir.localrecurse(true) } assert_equal([bad], ret.collect { |f| f.title }, "purge failed") badobj = @file[bad] assert(badobj, "did not create bad object") end def test_recurse basedir = tempfile() FileUtils.mkdir_p(basedir) # Create our file dir = nil assert_nothing_raised { dir = Puppet.type(:file).create( :path => basedir, :check => %w{owner mode group} ) } return_nil = false # and monkey-patch it [:localrecurse, :linkrecurse].each do |m| dir.meta_def(m) do |recurse| if return_nil # for testing nil return, of course return nil else return [recurse] end end end # We have to special-case this, because it returns a list of # found files. dir.meta_def(:sourcerecurse) do |recurse| if return_nil # for testing nil return, of course return nil else return [recurse], [] end end # First try it with recurse set to false dir[:recurse] = false assert_nothing_raised do assert_nil(dir.recurse) end # Now try it with the different valid positive values [true, "true", "inf", 50].each do |value| assert_nothing_raised { dir[:recurse] = value} # Now make sure the methods are called appropriately ret = nil assert_nothing_raised do ret = dir.recurse end # We should only call the localrecurse method, so make sure # that's the case if value == 50 # Make sure our counter got decremented assert_equal([49], ret, "did not call localrecurse") else assert_equal([true], ret, "did not call localrecurse") end end # Make sure it doesn't recurse when we've set recurse to false [false, "false"].each do |value| assert_nothing_raised { dir[:recurse] = value } ret = nil assert_nothing_raised() { ret = dir.recurse } assert_nil(ret) end dir[:recurse] = true # Now add a target, so we do the linking thing dir[:target] = tempfile() ret = nil assert_nothing_raised { ret = dir.recurse } assert_equal([true, true], ret, "did not call linkrecurse") # And add a source, and make sure we call that dir[:source] = tempfile() assert_nothing_raised { ret = dir.recurse } assert_equal([true, true, true], ret, "did not call linkrecurse") # Lastly, make sure we correctly handle returning nil return_nil = true assert_nothing_raised { ret = dir.recurse } end def test_recurse? file = Puppet::Type.type(:file).create :path => tempfile # Make sure we default to false assert(! file.recurse?, "Recurse defaulted to true") [true, "true", 10, "inf"].each do |value| file[:recurse] = value assert(file.recurse?, "%s did not cause recursion" % value) end [false, "false", 0].each do |value| file[:recurse] = value assert(! file.recurse?, "%s caused recursion" % value) end end def test_recursion basedir = tempfile() subdir = File.join(basedir, "subdir") tmpfile = File.join(basedir,"testing") FileUtils.mkdir_p(subdir) dir = nil [true, "true", "inf", 50].each do |value| assert_nothing_raised { dir = Puppet.type(:file).create( :path => basedir, :recurse => value, :check => %w{owner mode group} ) } config = mk_catalog dir children = nil assert_nothing_raised { children = dir.eval_generate } assert_equal([subdir], children.collect {|c| c.title }, "Incorrect generated children") # Remove our subdir resource, subdir_resource = config.resource(:file, subdir) config.remove_resource(subdir_resource) # Create the test file File.open(tmpfile, "w") { |f| f.puts "yayness" } assert_nothing_raised { children = dir.eval_generate } # And make sure we get both resources back. assert_equal([subdir, tmpfile].sort, children.collect {|c| c.title }.sort, "Incorrect generated children when recurse == %s" % value.inspect) File.unlink(tmpfile) Puppet.type(:file).clear end end def test_filetype_retrieval file = nil # Verify it retrieves files of type directory assert_nothing_raised { file = Puppet.type(:file).create( :name => tmpdir(), :check => :type ) } assert_nothing_raised { file.evaluate } assert_equal("directory", file.property(:type).retrieve) # And then check files assert_nothing_raised { file = Puppet.type(:file).create( :name => tempfile(), :ensure => "file" ) } assert_apply(file) file[:check] = "type" assert_apply(file) assert_equal("file", file.property(:type).retrieve) file[:type] = "directory" currentvalues = {} assert_nothing_raised { currentvalues = file.retrieve } # The 'retrieve' method sets @should to @is, so they're never # out of sync. It's a read-only class. assert(file.insync?(currentvalues)) end def test_remove basedir = tempfile() subdir = File.join(basedir, "this") FileUtils.mkdir_p(subdir) dir = nil assert_nothing_raised { dir = Puppet.type(:file).create( :path => basedir, :recurse => true, :check => %w{owner mode group} ) } mk_catalog dir assert_nothing_raised { dir.eval_generate } obj = nil assert_nothing_raised { obj = Puppet.type(:file)[subdir] } assert(obj, "Could not retrieve subdir object") assert_nothing_raised { obj.remove(true) } assert_nothing_raised { obj = Puppet.type(:file)[subdir] } assert_nil(obj, "Retrieved removed object") end def test_path dir = tempfile() path = File.join(dir, "subdir") assert_nothing_raised("Could not make file") { FileUtils.mkdir_p(File.dirname(path)) File.open(path, "w") { |f| f.puts "yayness" } } file = nil dirobj = nil assert_nothing_raised("Could not make file object") { dirobj = Puppet.type(:file).create( :path => dir, :recurse => true, :check => %w{mode owner group} ) } mk_catalog dirobj assert_nothing_raised { dirobj.eval_generate } assert_nothing_raised { file = dirobj.class[path] } assert(file, "Could not retrieve file object") assert_equal("/%s" % file.ref, file.path) end def test_autorequire basedir = tempfile() subfile = File.join(basedir, "subfile") baseobj = Puppet.type(:file).create( :name => basedir, :ensure => "directory" ) subobj = Puppet.type(:file).create( :name => subfile, :ensure => "file" ) edge = nil assert_nothing_raised do edge = subobj.autorequire.shift end assert_equal(baseobj, edge.source, "file did not require its parent dir") assert_equal(subobj, edge.target, "file did not require its parent dir") end def test_content file = tempfile() str = "This is some content" obj = nil assert_nothing_raised { obj = Puppet.type(:file).create( :name => file, :content => str ) } assert(!obj.insync?(obj.retrieve), "Object is incorrectly in sync") assert_events([:file_created], obj) currentvalues = obj.retrieve assert(obj.insync?(currentvalues), "Object is not in sync") text = File.read(file) assert_equal(str, text, "Content did not copy correctly") newstr = "Another string, yo" obj[:content] = newstr assert(!obj.insync?(obj.retrieve), "Object is incorrectly in sync") assert_events([:file_changed], obj) text = File.read(file) assert_equal(newstr, text, "Content did not copy correctly") currentvalues = obj.retrieve assert(obj.insync?(currentvalues), "Object is not in sync") end # Unfortunately, I know this fails def disabled_test_recursivemkdir path = tempfile() subpath = File.join(path, "this", "is", "a", "dir") file = nil assert_nothing_raised { file = Puppet.type(:file).create( :name => subpath, :ensure => "directory", :recurse => true ) } comp = mk_catalog("yay", file) comp.finalize assert_apply(comp) #assert_events([:directory_created], comp) assert(FileTest.directory?(subpath), "Did not create directory") end # Make sure that content updates the checksum on the same run def test_checksumchange_for_content dest = tempfile() File.open(dest, "w") { |f| f.puts "yayness" } file = nil assert_nothing_raised { file = Puppet.type(:file).create( :name => dest, :checksum => "md5", :content => "This is some content" ) } file.retrieve assert_events([:file_changed], file) file.retrieve assert_events([], file) end # Make sure that content updates the checksum on the same run def test_checksumchange_for_ensure dest = tempfile() file = nil assert_nothing_raised { file = Puppet.type(:file).create( :name => dest, :checksum => "md5", :ensure => "file" ) } file.retrieve assert_events([:file_created], file) file.retrieve assert_events([], file) end # Make sure that content gets used before ensure def test_contentbeatsensure dest = tempfile() file = nil assert_nothing_raised { file = Puppet.type(:file).create( :name => dest, :ensure => "file", :content => "this is some content, yo" ) } currentvalues = file.retrieve assert_events([:file_created], file) file.retrieve assert_events([], file) assert_events([], file) end # Make sure that content gets used before ensure def test_deletion_beats_source dest = tempfile() source = tempfile() File.open(source, "w") { |f| f.puts "yay" } file = nil assert_nothing_raised { file = Puppet.type(:file).create( :name => dest, :ensure => :absent, :source => source ) } file.retrieve assert_events([], file) assert(! FileTest.exists?(dest), "file was copied during deletion") # Now create the dest, and make sure it gets deleted File.open(dest, "w") { |f| f.puts "boo" } assert_events([:file_removed], file) assert(! FileTest.exists?(dest), "file was not deleted during deletion") end def test_nameandpath path = tempfile() file = nil assert_nothing_raised { file = Puppet.type(:file).create( :title => "fileness", :path => path, :content => "this is some content" ) } assert_apply(file) assert(FileTest.exists?(path)) end # Make sure that a missing group isn't fatal at object instantiation time. def test_missinggroup file = nil assert_nothing_raised { file = Puppet.type(:file).create( :path => tempfile(), :group => "fakegroup" ) } assert(file.property(:group), "Group property failed") end def test_modecreation path = tempfile() file = Puppet.type(:file).create( :path => path, :ensure => "file", :mode => "0777" ) assert_equal(0777, file.should(:mode), "Mode did not get set correctly") assert_apply(file) assert_equal(0777, File.stat(path).mode & 007777, "file mode is incorrect") File.unlink(path) file[:ensure] = "directory" assert_apply(file) assert_equal(0777, File.stat(path).mode & 007777, "directory mode is incorrect") end - def test_followlinks - File.umask(0022) - - basedir = tempfile() - Dir.mkdir(basedir) - file = File.join(basedir, "file") - link = File.join(basedir, "link") - - File.open(file, "w", 0644) { |f| f.puts "yayness"; f.flush } - File.symlink(file, link) - - obj = nil - assert_nothing_raised { - obj = Puppet.type(:file).create( - :path => link, - :mode => "755" - ) - } - obj.retrieve - - assert_events([], obj) - - # Assert that we default to not following links - assert_equal("%o" % 0644, "%o" % (File.stat(file).mode & 007777)) - - # Assert that we can manage the link directly, but modes still don't change - obj[:links] = :manage - assert_events([], obj) - - assert_equal("%o" % 0644, "%o" % (File.stat(file).mode & 007777)) - - obj[:links] = :follow - assert_events([:file_changed], obj) - - assert_equal("%o" % 0755, "%o" % (File.stat(file).mode & 007777)) - - # Now verify that content and checksum don't update, either - obj.delete(:mode) - obj[:checksum] = "md5" - obj[:links] = :ignore - - assert_events([], obj) - File.open(file, "w") { |f| f.puts "more text" } - assert_events([], obj) - obj[:links] = :follow - assert_events([], obj) - File.open(file, "w") { |f| f.puts "even more text" } - assert_events([:file_changed], obj) - - obj.delete(:checksum) - obj[:content] = "this is some content" - obj[:links] = :ignore - - assert_events([], obj) - File.open(file, "w") { |f| f.puts "more text" } - assert_events([], obj) - obj[:links] = :follow - assert_events([:file_changed, :file_changed], obj) - end - # If both 'ensure' and 'content' are used, make sure that all of the other # properties are handled correctly. def test_contentwithmode path = tempfile() file = nil assert_nothing_raised { file = Puppet.type(:file).create( :path => path, :ensure => "file", :content => "some text\n", :mode => 0755 ) } assert_apply(file) assert_equal("%o" % 0755, "%o" % (File.stat(path).mode & 007777)) end def test_backupmodes File.umask(0022) file = tempfile() newfile = tempfile() File.open(file, "w", 0411) { |f| f.puts "yayness" } obj = nil assert_nothing_raised { obj = Puppet::Type.type(:file).create( :path => file, :content => "rahness\n", :backup => ".puppet-bak" ) } assert_apply(obj) backupfile = file + obj[:backup] @@tmpfiles << backupfile assert(FileTest.exists?(backupfile), "Backup file %s does not exist" % backupfile) assert_equal(0411, filemode(backupfile), "File mode is wrong for backupfile") bucket = "bucket" bpath = tempfile() Dir.mkdir(bpath) Puppet::Type.type(:filebucket).create( :title => bucket, :path => bpath ) obj[:backup] = bucket obj[:content] = "New content" assert_apply(obj) md5 = "18cc17fa3047fcc691fdf49c0a7f539a" dir, file, pathfile = Puppet::Network::Handler.filebucket.paths(bpath, md5) assert_equal(0440, filemode(file)) end def test_largefilechanges source = tempfile() dest = tempfile() # Now make a large file File.open(source, "w") { |f| 500.times { |i| f.puts "line %s" % i } } obj = Puppet::Type.type(:file).create( :title => dest, :source => source ) assert_events([:file_created], obj) File.open(source, File::APPEND|File::WRONLY) { |f| f.puts "another line" } assert_events([:file_changed], obj) # Now modify the dest file File.open(dest, File::APPEND|File::WRONLY) { |f| f.puts "one more line" } assert_events([:file_changed, :file_changed], obj) end def test_replacefilewithlink path = tempfile() link = tempfile() File.open(path, "w") { |f| f.puts "yay" } File.open(link, "w") { |f| f.puts "a file" } file = nil assert_nothing_raised { file = Puppet.type(:file).create( :ensure => path, :path => link ) } assert_events([:link_created], file) assert(FileTest.symlink?(link), "Link was not created") assert_equal(path, File.readlink(link), "Link was created incorrectly") end def test_file_with_spaces dir = tempfile() Dir.mkdir(dir) source = File.join(dir, "file spaces") dest = File.join(dir, "another space") File.open(source, "w") { |f| f.puts :yay } obj = Puppet::Type.type(:file).create( :path => dest, :source => source ) assert(obj, "Did not create file") assert_apply(obj) assert(FileTest.exists?(dest), "File did not get created") end def test_present_matches_anything path = tempfile() file = Puppet::Type.newfile(:path => path, :ensure => :present) currentvalues = file.retrieve assert(! file.insync?(currentvalues), "File incorrectly in sync") # Now make a file File.open(path, "w") { |f| f.puts "yay" } currentvalues = file.retrieve assert(file.insync?(currentvalues), "File not in sync") # Now make a directory File.unlink(path) Dir.mkdir(path) currentvalues = file.retrieve assert(file.insync?(currentvalues), "Directory not considered 'present'") Dir.rmdir(path) # Now make a link file[:links] = :manage otherfile = tempfile() File.symlink(otherfile, path) currentvalues = file.retrieve assert(file.insync?(currentvalues), "Symlink not considered 'present'") File.unlink(path) # Now set some content, and make sure it works file[:content] = "yayness" assert_apply(file) assert_equal("yayness", File.read(path), "Content did not get set correctly") end # Make sure unmanaged files are purged. def test_purge sourcedir = tempfile() destdir = tempfile() Dir.mkdir(sourcedir) Dir.mkdir(destdir) sourcefile = File.join(sourcedir, "sourcefile") dsourcefile = File.join(destdir, "sourcefile") localfile = File.join(destdir, "localfile") purgee = File.join(destdir, "to_be_purged") File.open(sourcefile, "w") { |f| f.puts "funtest" } # this file should get removed File.open(purgee, "w") { |f| f.puts "footest" } lfobj = Puppet::Type.newfile( :title => "localfile", :path => localfile, :content => "rahtest", :ensure => :file, :backup => false ) destobj = Puppet::Type.newfile(:title => "destdir", :path => destdir, :source => sourcedir, :backup => false, :recurse => true) config = mk_catalog(lfobj, destobj) config.apply assert(FileTest.exists?(dsourcefile), "File did not get copied") assert(FileTest.exists?(localfile), "Local file did not get created") assert(FileTest.exists?(purgee), "Purge target got prematurely purged") assert_nothing_raised { destobj[:purge] = true } config.apply assert(FileTest.exists?(localfile), "Local file got purged") assert(FileTest.exists?(dsourcefile), "Source file got purged") assert(! FileTest.exists?(purgee), "File did not get purged") end # Testing #274. Make sure target can be used without 'ensure'. def test_target_without_ensure source = tempfile() dest = tempfile() File.open(source, "w") { |f| f.puts "funtest" } obj = nil assert_nothing_raised { obj = Puppet::Type.newfile(:path => dest, :target => source) } assert_apply(obj) end def test_autorequire_owner_and_group file = tempfile() comp = nil user = nil group =nil home = nil ogroup = nil assert_nothing_raised { user = Puppet.type(:user).create( :name => "pptestu", :home => file, :gid => "pptestg" ) home = Puppet.type(:file).create( :path => file, :owner => "pptestu", :group => "pptestg", :ensure => "directory" ) group = Puppet.type(:group).create( :name => "pptestg" ) comp = mk_catalog(user, group, home) } # Now make sure we get a relationship for each of these rels = nil assert_nothing_raised { rels = home.autorequire } assert(rels.detect { |e| e.source == user }, "owner was not autorequired") assert(rels.detect { |e| e.source == group }, "group was not autorequired") end # Testing #309 -- //my/file => /my/file def test_slash_deduplication ["/my/////file/for//testing", "//my/file/for/testing///", "/my/file/for/testing"].each do |path| file = nil assert_nothing_raised do file = Puppet::Type.newfile(:path => path) end assert_equal("/my/file/for/testing", file.title) assert_equal(file, Puppet::Type.type(:file)["/my/file/for/testing"]) Puppet::Type.type(:file).clear end end # Testing #304 def test_links_to_directories link = tempfile() file = tempfile() dir = tempfile() Dir.mkdir(dir) bucket = Puppet::Type.newfilebucket :name => "main" File.symlink(dir, link) File.open(file, "w") { |f| f.puts "" } assert_equal(dir, File.readlink(link)) - obj = Puppet::Type.newfile :path => link, :ensure => :link, - :target => file, :recurse => false, :backup => "main" + obj = Puppet::Type.newfile :path => link, :ensure => :link, :target => file, :recurse => false, :backup => "main" assert_apply(obj) assert_equal(file, File.readlink(link)) end # Testing #303 def test_nobackups_with_links link = tempfile() new = tempfile() File.open(link, "w") { |f| f.puts "old" } File.open(new, "w") { |f| f.puts "new" } obj = Puppet::Type.newfile :path => link, :ensure => :link, :target => new, :recurse => true, :backup => false assert_nothing_raised do obj.handlebackup end bfile = [link, "puppet-bak"].join(".") assert(! FileTest.exists?(bfile), "Backed up when told not to") assert_apply(obj) assert(! FileTest.exists?(bfile), "Backed up when told not to") end # Make sure we consistently handle backups for all cases. def test_ensure_with_backups # We've got three file types, so make sure we can replace any type # with the other type and that backups are done correctly. types = [:file, :directory, :link] dir = tempfile() path = File.join(dir, "test") linkdest = tempfile() creators = { :file => proc { File.open(path, "w") { |f| f.puts "initial" } }, :directory => proc { Dir.mkdir(path) }, :link => proc { File.symlink(linkdest, path) } } bucket = Puppet::Type.newfilebucket :name => "main", :path => tempfile() obj = Puppet::Type.newfile :path => path, :force => true, :links => :manage Puppet[:trace] = true ["main", false].each do |backup| obj[:backup] = backup obj.finish types.each do |should| types.each do |is| # It makes no sense to replace a directory with a directory # next if should == :directory and is == :directory Dir.mkdir(dir) # Make the thing creators[is].call obj[:ensure] = should if should == :link obj[:target] = linkdest else if obj.property(:target) obj.delete(:target) end end # First try just removing the initial data assert_nothing_raised do obj.remove_existing(should) end unless is == should # Make sure the original is gone assert(! FileTest.exists?(obj[:path]), "remove_existing did not work: " + "did not remove %s with %s" % [is, should]) end FileUtils.rmtree(obj[:path]) # Now make it again creators[is].call property = obj.property(:ensure) currentvalue = property.retrieve unless property.insync?(currentvalue) assert_nothing_raised do property.sync end end FileUtils.rmtree(dir) end end end end if Process.uid == 0 # Testing #364. def test_writing_in_directories_with_no_write_access # Make a directory that our user does not have access to dir = tempfile() Dir.mkdir(dir) # Get a fake user user = nonrootuser # and group group = nonrootgroup # First try putting a file in there path = File.join(dir, "file") file = Puppet::Type.newfile :path => path, :owner => user.name, :group => group.name, :content => "testing" # Make sure we can create it assert_apply(file) assert(FileTest.exists?(path), "File did not get created") # And that it's owned correctly assert_equal(user.uid, File.stat(path).uid, "File has the wrong owner") assert_equal(group.gid, File.stat(path).gid, "File has the wrong group") assert_equal("testing", File.read(path), "file has the wrong content") # Now make a dir subpath = File.join(dir, "subdir") subdir = Puppet::Type.newfile :path => subpath, :owner => user.name, :group => group.name, :ensure => :directory # Make sure we can create it assert_apply(subdir) assert(FileTest.directory?(subpath), "File did not get created") # And that it's owned correctly assert_equal(user.uid, File.stat(subpath).uid, "File has the wrong owner") assert_equal(group.gid, File.stat(subpath).gid, "File has the wrong group") assert_equal("testing", File.read(path), "file has the wrong content") end end # #366 def test_replace_aliases file = Puppet::Type.newfile :path => tempfile() file[:replace] = :yes assert_equal(:true, file[:replace], ":replace did not alias :true to :yes") file[:replace] = :no assert_equal(:false, file[:replace], ":replace did not alias :false to :no") end # #365 -- make sure generated files also use filebuckets. def test_recursive_filebuckets source = tempfile() dest = tempfile() s1 = File.join(source, "1") sdir = File.join(source, "dir") s2 = File.join(sdir, "2") Dir.mkdir(source) Dir.mkdir(sdir) [s1, s2].each { |file| File.open(file, "w") { |f| f.puts "yay: %s" % File.basename(file) } } sums = {} [s1, s2].each do |f| sums[File.basename(f)] = Digest::MD5.hexdigest(File.read(f)) end dfiles = [File.join(dest, "1"), File.join(dest, "dir", "2")] bpath = tempfile bucket = Puppet::Type.type(:filebucket).create :name => "rtest", :path => bpath dipper = bucket.bucket dipper = Puppet::Network::Handler.filebucket.new( :Path => bpath ) assert(dipper, "did not receive bucket client") file = Puppet::Type.newfile :path => dest, :source => source, :recurse => true, :backup => "rtest" assert_apply(file) dfiles.each do |f| assert(FileTest.exists?(f), "destfile %s was not created" % f) end # Now modify the source files to make sure things get backed up correctly [s1, s2].each { |sf| File.open(sf, "w") { |f| f.puts "boo: %s" % File.basename(sf) } } assert_apply(file) dfiles.each do |f| assert_equal("boo: %s\n" % File.basename(f), File.read(f), "file was not copied correctly") end # Make sure we didn't just copy the files over to backup locations dfiles.each do |f| assert(! FileTest.exists?(f + "rtest"), "file %s was copied for backup instead of bucketed" % File.basename(f)) end # Now make sure we can get the source sums from the bucket sums.each do |f, sum| result = nil assert_nothing_raised do result = dipper.getfile(sum) end assert(result, "file %s was not backed to filebucket" % f) assert_equal("yay: %s\n" % f, result, "file backup was not correct") end end def test_backup path = tempfile() file = Puppet::Type.newfile :path => path, :content => "yay" [false, :false, "false"].each do |val| assert_nothing_raised do file[:backup] = val end assert_equal(false, file[:backup], "%s did not translate" % val.inspect) end [true, :true, "true", ".puppet-bak"].each do |val| assert_nothing_raised do file[:backup] = val end assert_equal(".puppet-bak", file[:backup], "%s did not translate" % val.inspect) end # Now try a non-bucket string assert_nothing_raised do file[:backup] = ".bak" end assert_equal(".bak", file[:backup], ".bak did not translate") # Now try a non-existent bucket assert_nothing_raised do file[:backup] = "main" end assert_equal("main", file[:backup], "bucket name was not retained") assert_equal("main", file.bucket, "file's bucket was not set") # And then an existing bucket obj = Puppet::Type.type(:filebucket).create :name => "testing" bucket = obj.bucket assert_nothing_raised do file[:backup] = "testing" end assert_equal("testing", file[:backup], "backup value was reset") assert_equal(obj.bucket, file.bucket, "file's bucket was not set") end def test_pathbuilder dir = tempfile() Dir.mkdir(dir) file = File.join(dir, "file") File.open(file, "w") { |f| f.puts "" } obj = Puppet::Type.newfile :path => dir, :recurse => true, :mode => 0755 mk_catalog obj assert_equal("/%s" % obj.ref, obj.path) list = obj.eval_generate fileobj = obj.class[file] assert(fileobj, "did not generate file object") assert_equal("/%s" % fileobj.ref, fileobj.path, "did not generate correct subfile path") end # Testing #403 def test_removal_with_content_set path = tempfile() File.open(path, "w") { |f| f.puts "yay" } file = Puppet::Type.newfile(:name => path, :ensure => :absent, :content => "foo") assert_apply(file) assert(! FileTest.exists?(path), "File was not removed") end # Testing #434 def test_stripping_extra_slashes_during_lookup file = Puppet::Type.newfile(:path => "/one/two") %w{/one/two/ /one/two /one//two //one//two//}.each do |path| assert(Puppet::Type.type(:file)[path], "could not look up file via path %s" % path) end end # Testing #438 def test_creating_properties_conflict file = tempfile() first = tempfile() second = tempfile() params = [:content, :source, :target] params.each do |param| assert_nothing_raised("%s conflicted with ensure" % [param]) do Puppet::Type.newfile(:path => file, param => first, :ensure => :file) end Puppet::Type.type(:file).clear params.each do |other| next if other == param assert_raise(Puppet::Error, "%s and %s did not conflict" % [param, other]) do Puppet::Type.newfile(:path => file, other => first, param => second) end end end end # Testing #508 if Process.uid == 0 def test_files_replace_with_right_attrs source = tempfile() File.open(source, "w") { |f| f.puts "some text" } File.chmod(0755, source) user = nonrootuser group = nonrootgroup path = tempfile() good = {:uid => user.uid, :gid => group.gid, :mode => 0640} run = Proc.new do |obj, msg| assert_apply(obj) stat = File.stat(obj[:path]) good.each do |should, sval| if should == :mode current = filemode(obj[:path]) else current = stat.send(should) end assert_equal(sval, current, "Attr %s was not correct %s" % [should, msg]) end end file = Puppet::Type.newfile(:path => path, :owner => user.name, :group => group.name, :mode => 0640, :backup => false) {:source => source, :content => "some content"}.each do |attr, value| file[attr] = value # First create the file run.call(file, "upon creation with %s" % attr) # Now change something so that we replace the file case attr when :source: File.open(source, "w") { |f| f.puts "some different text" } when :content: file[:content] = "something completely different" else raise "invalid attr %s" % attr end # Run it again run.call(file, "after modification with %s" % attr) # Now remove the file and the attr file.delete(attr) File.unlink(path) end end end # #505 def test_numeric_recurse dir = tempfile() subdir = File.join(dir, "subdir") other = File.join(subdir, "deeper") file = File.join(other, "file") [dir, subdir, other].each { |d| Dir.mkdir(d) } File.open(file, "w") { |f| f.puts "yay" } File.chmod(0644, file) obj = Puppet::Type.newfile(:path => dir, :mode => 0750, :recurse => "2") config = mk_catalog(obj) children = nil assert_nothing_raised("Failure when recursing") do children = obj.eval_generate end assert(obj.class[subdir], "did not create subdir object") children.each do |c| assert_nothing_raised("Failure when recursing on %s" % c) do c.catalog = config others = c.eval_generate end end oobj = obj.class[other] assert(oobj, "did not create other object") assert_nothing_raised do assert_nil(oobj.eval_generate, "recursed too far") end end # Make sure we default to the "puppet" filebucket, rather than a string def test_backup_defaults_to_bucket path = tempfile file = Puppet::Type.newfile(:path => path, :content => 'some content') file.finish assert_instance_of(Puppet::Network::Client::Dipper, file.bucket, "did not default to a filebucket for backups") end # #515 - make sure 'ensure' other than "link" is deleted during recursion def test_ensure_deleted_during_recursion dir = tempfile() Dir.mkdir(dir) file = File.join(dir, "file") File.open(file, "w") { |f| f.puts "asdfasdf" } obj = Puppet::Type.newfile(:path => dir, :ensure => :directory, :recurse => true) config = mk_catalog(obj) children = nil assert_nothing_raised do children = obj.eval_generate end fobj = obj.class[file] assert(fobj, "did not create file object") assert(fobj.should(:ensure) != :directory, "ensure was passed to child") end # #567 def test_missing_files_are_in_sync file = tempfile obj = Puppet::Type.newfile(:path => file, :mode => 0755) changes = obj.evaluate assert(changes.empty?, "Missing file with no ensure resulted in changes") end def test_root_dir_is_named_correctly obj = Puppet::Type.newfile(:path => '/', :mode => 0755) assert_equal("/", obj.title, "/ directory was changed to empty string") end # #1010 and #1037 -- write should fail if the written checksum does not # match the file we thought we were writing. def test_write_validates_checksum file = tempfile inst = Puppet::Type.newfile(:path => file, :content => "something") tmpfile = file + ".puppettmp" wh = mock 'writehandle', :print => nil rh = mock 'readhandle' rh.expects(:read).with(512).times(2).returns("other").then.returns(nil) File.expects(:open).with { |*args| args[0] == tmpfile and args[1] != "r" }.yields(wh) File.expects(:open).with { |*args| args[0] == tmpfile and args[1] == "r" }.yields(rh) File.stubs(:rename) FileTest.stubs(:exist?).returns(true) FileTest.stubs(:file?).returns(true) inst.expects(:fail) inst.write("something", :whatever) end end diff --git a/test/ral/types/filesources.rb b/test/ral/types/filesources.rb index 02bf8a5b3..a7bb6fefa 100755 --- a/test/ral/types/filesources.rb +++ b/test/ral/types/filesources.rb @@ -1,1016 +1,997 @@ #!/usr/bin/env ruby require File.dirname(__FILE__) + '/../../lib/puppettest' require 'puppettest' require 'puppettest/support/utils' require 'cgi' require 'fileutils' require 'mocha' class TestFileSources < Test::Unit::TestCase include PuppetTest::Support::Utils include PuppetTest::FileTesting def setup super if defined? @port @port += 1 else @port = 12345 end @file = Puppet::Type.type(:file) Puppet[:filetimeout] = -1 Puppet::Util::SUIDManager.stubs(:asuser).yields end def teardown super Puppet::Network::HttpPool.clear_http_instances end def use_storage begin initstorage rescue system("rm -rf %s" % Puppet[:statefile]) end end def initstorage Puppet::Util::Storage.init Puppet::Util::Storage.load end # Make a simple recursive tree. def mk_sourcetree source = tempfile() sourcefile = File.join(source, "file") Dir.mkdir source File.open(sourcefile, "w") { |f| f.puts "yay" } dest = tempfile() destfile = File.join(dest, "file") return source, dest, sourcefile, destfile end def test_newchild path = tempfile() @@tmpfiles.push path FileUtils.mkdir_p path File.open(File.join(path,"childtest"), "w") { |of| of.puts "yayness" } file = nil comp = nil trans = nil assert_nothing_raised { file = Puppet.type(:file).create( :name => path ) } config = mk_catalog(file) child = nil assert_nothing_raised { child = file.newchild("childtest", true) } assert(child) assert_raise(Puppet::DevError) { file.newchild(File.join(path,"childtest"), true) } end def test_describe source = tempfile() dest = tempfile() - file = Puppet::Type.newfile :path => dest, :source => source, - :title => "copier" + file = Puppet::Type.newfile :path => dest, :source => source, :title => "copier" property = file.property(:source) # First try describing with a normal source result = nil assert_nothing_raised do result = property.describe(source) end assert_nil(result, "Got a result back when source is missing") # Now make a remote directory Dir.mkdir(source) assert_nothing_raised do result = property.describe(source) end assert_equal("directory", result[:type]) # And as a file Dir.rmdir(source) File.open(source, "w") { |f| f.puts "yay" } assert_nothing_raised do result = property.describe(source) end assert_equal("file", result[:type]) assert(result[:checksum], "did not get value for checksum") if Puppet::Util::SUIDManager.uid == 0 assert(result.has_key?(:owner), "Lost owner in describe") else assert(! result.has_key?(:owner), "Kept owner in describe even tho not root") end # Now let's do the various link things File.unlink(source) target = tempfile() File.open(target, "w") { |f| f.puts "yay" } File.symlink(target, source) - file[:links] = :ignore - assert_nil(property.describe(source), - "Links were not ignored") - file[:links] = :manage - # We can't manage links at this point - assert_raise(Puppet::Network::Handler::FileServerError) do - property.describe(source) - end + assert_equal("link", property.describe(source)[:type]) - # And then make sure links get followed, otherwise + # And then make sure links get followed file[:links] = :follow assert_equal("file", property.describe(source)[:type]) end def test_source_retrieve source = tempfile() dest = tempfile() file = Puppet::Type.newfile :path => dest, :source => source, :title => "copier" assert(file.property(:checksum), "source property did not create checksum property") property = file.property(:source) assert(property, "did not get source property") # Make sure the munge didn't actually change the source assert_equal([source], property.should, "munging changed the source") # First try it with a missing source currentvalue = nil assert_nothing_raised do currentvalue = property.retrieve end # And make sure the property considers itself in sync, since there's nothing # to do assert(property.insync?(currentvalue), "source thinks there's work to do with no file or dest") # Now make the dest a directory, and make sure the object sets :ensure # up to create a directory Dir.mkdir(source) assert_nothing_raised do currentvalue = property.retrieve end assert_equal(:directory, file.should(:ensure), "Did not set to create directory") # And make sure the source property won't try to do anything with a # remote dir assert(property.insync?(currentvalue), "Source was out of sync even tho remote is dir") # Now remove the source, and make sure :ensure was not modified Dir.rmdir(source) assert_nothing_raised do property.retrieve end assert_equal(:directory, file.should(:ensure), "Did not keep :ensure setting") # Now have a remote file and make sure things work correctly File.open(source, "w") { |f| f.puts "yay" } File.chmod(0755, source) assert_nothing_raised do property.retrieve end assert_equal(:file, file.should(:ensure), "Did not make correct :ensure setting") assert_equal(0755, file.should(:mode), "Mode was not copied over") # Now let's make sure that we get the first found source fake = tempfile() property.should = [fake, source] assert_nothing_raised do property.retrieve end assert_equal(Digest::MD5.hexdigest(File.read(source)), property.checksum.sub(/^\{\w+\}/, ''), "Did not catch later source") end def test_insync source = tempfile() dest = tempfile() file = Puppet::Type.newfile :path => dest, :source => source, :title => "copier" property = file.property(:source) assert(property, "did not get source property") # Try it with no source at all currentvalues = file.retrieve assert(property.insync?(currentvalues[property]), "source property not in sync with missing source") # with a directory Dir.mkdir(source) currentvalues = file.retrieve assert(property.insync?(currentvalues[property]), "source property not in sync with directory as source") Dir.rmdir(source) # with a file File.open(source, "w") { |f| f.puts "yay" } currentvalues = file.retrieve assert(!property.insync?(currentvalues[property]), "source property was in sync when file was missing") # With a different file File.open(dest, "w") { |f| f.puts "foo" } currentvalues = file.retrieve assert(!property.insync?(currentvalues[property]), "source property was in sync with different file") # with matching files File.open(dest, "w") { |f| f.puts "yay" } currentvalues = file.retrieve assert(property.insync?(currentvalues[property]), "source property was not in sync with matching file") end def test_source_sync source = tempfile() dest = tempfile() file = Puppet::Type.newfile :path => dest, :source => source, :title => "copier" property = file.property(:source) File.open(source, "w") { |f| f.puts "yay" } currentvalues = file.retrieve assert(! property.insync?(currentvalues[property]), "source thinks it's in sync") event = nil assert_nothing_raised do event = property.sync end assert_equal(:file_created, event) assert_equal(File.read(source), File.read(dest), "File was not copied correctly") # Now write something different File.open(source, "w") { |f| f.puts "rah" } currentvalues = file.retrieve assert(! property.insync?(currentvalues[property]), "source should be out of sync") assert_nothing_raised do event = property.sync end assert_equal(:file_changed, event) assert_equal(File.read(source), File.read(dest), "File was not copied correctly") end # XXX This test doesn't cover everything. Specifically, # it doesn't handle 'ignore' and 'links'. def test_sourcerecurse source, dest, sourcefile, destfile = mk_sourcetree # The sourcerecurse method will only ever get called when we're # recursing, so we go ahead and set it. obj = Puppet::Type.newfile :source => source, :path => dest, :recurse => true config = mk_catalog(obj) result = nil sourced = nil assert_nothing_raised do result, sourced = obj.sourcerecurse(true) end assert_equal([destfile], sourced, "Did not get correct list of sourced objects") dfileobj = @file[destfile] assert(dfileobj, "Did not create destfile object") assert_equal([dfileobj], result) # Clean this up so it can be recreated config.remove_resource(dfileobj) # Make sure we correctly iterate over the sources nosource = tempfile() obj[:source] = [nosource, source] result = nil assert_nothing_raised do result, sourced = obj.sourcerecurse(true) end assert_equal([destfile], sourced, "Did not get correct list of sourced objects") dfileobj = @file[destfile] assert(dfileobj, "Did not create destfile object with a missing source") assert_equal([dfileobj], result) dfileobj.remove # Lastly, make sure we return an empty array when no sources are there obj[:source] = [nosource, tempfile()] assert_nothing_raised do result, sourced = obj.sourcerecurse(true) end assert_equal([], sourced, "Did not get correct list of sourced objects") assert_equal([], result, "Sourcerecurse failed when all sources are missing") end def test_simplelocalsource path = tempfile() FileUtils.mkdir_p path frompath = File.join(path,"source") topath = File.join(path,"dest") fromfile = nil tofile = nil trans = nil File.open(frompath, File::WRONLY|File::CREAT|File::APPEND) { |of| of.puts "yayness" } assert_nothing_raised { tofile = Puppet.type(:file).create( :name => topath, :source => frompath ) } assert_apply(tofile) assert(FileTest.exists?(topath), "File #{topath} is missing") from = File.open(frompath) { |o| o.read } to = File.open(topath) { |o| o.read } assert_equal(from,to) end # Make sure a simple recursive copy works def test_simple_recursive_source source, dest, sourcefile, destfile = mk_sourcetree file = Puppet::Type.newfile :path => dest, :source => source, :recurse => true assert_events([:directory_created, :file_created], file) assert(FileTest.directory?(dest), "Dest dir was not created") assert(FileTest.file?(destfile), "dest file was not created") assert_equal("yay\n", File.read(destfile), "dest file was not copied correctly") end def recursive_source_test(fromdir, todir) Puppet::Type.allclear initstorage tofile = nil trans = nil assert_nothing_raised { tofile = Puppet.type(:file).create( :path => todir, :recurse => true, :backup => false, :source => fromdir ) } assert_apply(tofile) assert(FileTest.exists?(todir), "Created dir %s does not exist" % todir) Puppet::Type.allclear end def run_complex_sources(networked = false) path = tempfile() # first create the source directory FileUtils.mkdir_p path # okay, let's create a directory structure fromdir = File.join(path,"fromdir") Dir.mkdir(fromdir) FileUtils.cd(fromdir) { File.open("one", "w") { |f| f.puts "onefile"} File.open("two", "w") { |f| f.puts "twofile"} } todir = File.join(path, "todir") source = fromdir if networked source = "puppet://localhost/%s%s" % [networked, fromdir] end recursive_source_test(source, todir) return [fromdir,todir, File.join(todir, "one"), File.join(todir, "two")] end def test_complex_sources_twice fromdir, todir, one, two = run_complex_sources assert_trees_equal(fromdir,todir) recursive_source_test(fromdir, todir) assert_trees_equal(fromdir,todir) # Now remove the whole tree and try it again. [one, two].each do |f| File.unlink(f) end Dir.rmdir(todir) recursive_source_test(fromdir, todir) assert_trees_equal(fromdir,todir) end def test_sources_with_deleted_destfiles fromdir, todir, one, two = run_complex_sources assert(FileTest.exists?(todir)) # We shouldn't have a 'two' file object in memory assert_nil(@file[two], "object for 'two' is still in memory") # then delete a file File.unlink(two) # and run recursive_source_test(fromdir, todir) assert(FileTest.exists?(two), "Deleted file was not recopied") # and make sure they're still equal assert_trees_equal(fromdir,todir) end def test_sources_with_readonly_destfiles fromdir, todir, one, two = run_complex_sources assert(FileTest.exists?(todir)) File.chmod(0600, one) recursive_source_test(fromdir, todir) # and make sure they're still equal assert_trees_equal(fromdir,todir) # Now try it with the directory being read-only File.chmod(0111, todir) recursive_source_test(fromdir, todir) # and make sure they're still equal assert_trees_equal(fromdir,todir) end def test_sources_with_modified_dest_files fromdir, todir, one, two = run_complex_sources assert(FileTest.exists?(todir)) # Modify a dest file File.open(two, "w") { |f| f.puts "something else" } recursive_source_test(fromdir, todir) # and make sure they're still equal assert_trees_equal(fromdir,todir) end def test_sources_with_added_destfiles fromdir, todir = run_complex_sources assert(FileTest.exists?(todir)) # and finally, add some new files add_random_files(todir) recursive_source_test(fromdir, todir) fromtree = file_list(fromdir) totree = file_list(todir) assert(fromtree != totree, "Trees are incorrectly equal") # then remove our new files FileUtils.cd(todir) { %x{find . 2>/dev/null}.chomp.split(/\n/).each { |file| if file =~ /file[0-9]+/ File.unlink(file) end } } # and make sure they're still equal assert_trees_equal(fromdir,todir) end # Make sure added files get correctly caught during recursion def test_RecursionWithAddedFiles basedir = tempfile() Dir.mkdir(basedir) @@tmpfiles << basedir file1 = File.join(basedir, "file1") file2 = File.join(basedir, "file2") subdir1 = File.join(basedir, "subdir1") file3 = File.join(subdir1, "file") File.open(file1, "w") { |f| f.puts "yay" } rootobj = nil assert_nothing_raised { rootobj = Puppet.type(:file).create( :name => basedir, :recurse => true, :check => %w{type owner}, :mode => 0755 ) } assert_apply(rootobj) assert_equal(0755, filemode(file1)) File.open(file2, "w") { |f| f.puts "rah" } assert_apply(rootobj) assert_equal(0755, filemode(file2)) Dir.mkdir(subdir1) File.open(file3, "w") { |f| f.puts "foo" } assert_apply(rootobj) assert_equal(0755, filemode(file3)) end def mkfileserverconf(mounts) file = tempfile() File.open(file, "w") { |f| mounts.each { |path, name| f.puts "[#{name}]\n\tpath #{path}\n\tallow *\n" } } @@tmpfiles << file return file end def test_NetworkSources server = nil mounts = { "/" => "root" } fileserverconf = mkfileserverconf(mounts) Puppet[:autosign] = true Puppet[:masterport] = 8762 Puppet[:name] = "puppetmasterd" Puppet[:certdnsnames] = "localhost" serverpid = nil assert_nothing_raised() { server = Puppet::Network::HTTPServer::WEBrick.new( :Handlers => { :CA => {}, # so that certs autogenerate :FileServer => { :Config => fileserverconf } } ) } serverpid = fork { assert_nothing_raised() { #trap(:INT) { server.shutdown; Kernel.exit! } trap(:INT) { server.shutdown } server.start } } @@tmppids << serverpid sleep(1) fromdir, todir = run_complex_sources("root") assert_trees_equal(fromdir,todir) recursive_source_test(fromdir, todir) assert_trees_equal(fromdir,todir) assert_nothing_raised { system("kill -INT %s" % serverpid) } end def test_unmountedNetworkSources server = nil mounts = { "/" => "root", "/noexistokay" => "noexist" } fileserverconf = mkfileserverconf(mounts) Puppet[:autosign] = true Puppet[:masterport] = @port Puppet[:certdnsnames] = "localhost" serverpid = nil assert_nothing_raised("Could not start on port %s" % @port) { server = Puppet::Network::HTTPServer::WEBrick.new( :Port => @port, :Handlers => { :CA => {}, # so that certs autogenerate :FileServer => { :Config => fileserverconf } } ) } serverpid = fork { assert_nothing_raised() { #trap(:INT) { server.shutdown; Kernel.exit! } trap(:INT) { server.shutdown } server.start } } @@tmppids << serverpid sleep(1) name = File.join(tmpdir(), "nosourcefile") file = Puppet.type(:file).create( :source => "puppet://localhost/noexist/file", :name => name ) assert_nothing_raised { file.retrieve } comp = mk_catalog(file) comp.apply assert(!FileTest.exists?(name), "File with no source exists anyway") end def test_alwayschecksum from = tempfile() to = tempfile() File.open(from, "w") { |f| f.puts "yayness" } File.open(to, "w") { |f| f.puts "yayness" } file = nil # Now the files should be exactly the same, so we should not see attempts # at copying assert_nothing_raised { file = Puppet.type(:file).create( :path => to, :source => from ) } currentvalue = file.retrieve assert(currentvalue[file.property(:checksum)], "File does not have a checksum property") assert_equal(0, file.evaluate.length, "File produced changes") end def test_sourcepaths files = [] 3.times { files << tempfile() } to = tempfile() File.open(files[-1], "w") { |f| f.puts "yee-haw" } file = nil assert_nothing_raised { file = Puppet.type(:file).create( :name => to, :source => files ) } comp = mk_catalog(file) assert_events([:file_created], comp) assert(File.exists?(to), "File does not exist") txt = nil File.open(to) { |f| txt = f.read.chomp } assert_equal("yee-haw", txt, "Contents do not match") end # Make sure that source-copying updates the checksum on the same run def test_checksumchange source = tempfile() dest = tempfile() File.open(dest, "w") { |f| f.puts "boo" } File.open(source, "w") { |f| f.puts "yay" } file = nil assert_nothing_raised { file = Puppet.type(:file).create( :name => dest, :source => source ) } file.retrieve assert_events([:file_changed], file) file.retrieve assert_events([], file) end # Make sure that source-copying updates the checksum on the same run def test_sourcebeatsensure source = tempfile() dest = tempfile() File.open(source, "w") { |f| f.puts "yay" } file = nil assert_nothing_raised { file = Puppet.type(:file).create( :name => dest, :ensure => "file", :source => source ) } file.retrieve assert_events([:file_created], file) file.retrieve assert_events([], file) assert_events([], file) end def test_sourcewithlinks source = tempfile() link = tempfile() dest = tempfile() File.open(source, "w") { |f| f.puts "yay" } File.symlink(source, link) file = nil assert_nothing_raised { file = Puppet.type(:file).create( :name => dest, - :source => link + :source => link, + :links => :follow ) } - # Default to skipping links - assert_events([], file) - assert(! FileTest.exists?(dest), "Created link") - - # Now follow the links - file[:links] = :follow assert_events([:file_created], file) assert(FileTest.file?(dest), "Destination is not a file") # Now copy the links - #assert_raise(Puppet::Network::Handler::FileServerError) { - trans = nil - assert_nothing_raised { - file[:links] = :manage - comp = mk_catalog(file) - trans = comp.apply - } - - assert(trans.failed?(file), "Object did not fail to copy links") + file[:links] = :manage + assert_events([:link_created], file) + assert(FileTest.symlink?(dest), "Destination is not a link") end def test_changes source = tempfile() dest = tempfile() File.open(source, "w") { |f| f.puts "yay" } obj = nil assert_nothing_raised { obj = Puppet.type(:file).create( :name => dest, :source => source ) } assert_events([:file_created], obj) assert_equal(File.read(source), File.read(dest), "Files are not equal") assert_events([], obj) File.open(source, "w") { |f| f.puts "boo" } assert_events([:file_changed], obj) assert_equal(File.read(source), File.read(dest), "Files are not equal") assert_events([], obj) File.open(dest, "w") { |f| f.puts "kaboom" } # There are two changes, because first the checksum is noticed, and # then the source causes a change assert_events([:file_changed, :file_changed], obj) assert_equal(File.read(source), File.read(dest), "Files are not equal") assert_events([], obj) end def test_file_source_with_space dir = tempfile() source = File.join(dir, "file with spaces") Dir.mkdir(dir) File.open(source, "w") { |f| f.puts "yayness" } newdir = tempfile() newpath = File.join(newdir, "file with spaces") file = Puppet::Type.newfile( :path => newdir, :source => dir, :recurse => true ) assert_apply(file) assert(FileTest.exists?(newpath), "Did not create file") assert_equal("yayness\n", File.read(newpath)) end # Make sure files aren't replaced when replace is false, but otherwise # are. def test_replace source = tempfile() File.open(source, "w") { |f| f.puts "yayness" } dest = tempfile() file = Puppet::Type.newfile( :path => dest, :source => source, :recurse => true ) assert_apply(file) assert(FileTest.exists?(dest), "Did not create file") assert_equal("yayness\n", File.read(dest)) # Now set :replace assert_nothing_raised { file[:replace] = false } File.open(source, "w") { |f| f.puts "funtest" } assert_apply(file) # Make sure it doesn't change. assert_equal("yayness\n", File.read(dest), "File got replaced when :replace was false") # Now set it to true and make sure it does change. assert_nothing_raised { file[:replace] = true } assert_apply(file) # Make sure it doesn't change. assert_equal("funtest\n", File.read(dest), "File was not replaced when :replace was true") end # Testing #285. This just makes sure that URI parsing works correctly. def test_fileswithpoundsigns dir = tstdir() subdir = File.join(dir, "#dir") Dir.mkdir(subdir) file = File.join(subdir, "file") File.open(file, "w") { |f| f.puts "yayness" } dest = tempfile() source = "file://localhost#{dir}" obj = Puppet::Type.newfile( :path => dest, :source => source, :recurse => true ) newfile = File.join(dest, "#dir", "file") poundsource = "file://localhost#{subdir}" sourceobj = path = nil assert_nothing_raised { sourceobj, path = obj.uri2obj(poundsource) } assert_equal("/localhost" + URI.escape(subdir), path) assert_apply(obj) assert(FileTest.exists?(newfile), "File did not get created") assert_equal("yayness\n", File.read(newfile)) end def test_sourceselect dest = tempfile() sources = [] 2.times { |i| i = i + 1 source = tempfile() sources << source file = File.join(source, "file%s" % i) Dir.mkdir(source) File.open(file, "w") { |f| f.print "yay" } } file1 = File.join(dest, "file1") file2 = File.join(dest, "file2") file3 = File.join(dest, "file3") # Now make different files with the same name in each source dir sources.each_with_index do |source, i| File.open(File.join(source, "file3"), "w") { |f| f.print i.to_s } end obj = Puppet::Type.newfile(:path => dest, :recurse => true, :source => sources) assert_equal(:first, obj[:sourceselect], "sourceselect has the wrong default") # First, make sure we default to just copying file1 assert_apply(obj) assert(FileTest.exists?(file1), "File from source 1 was not copied") assert(! FileTest.exists?(file2), "File from source 2 was copied") assert(FileTest.exists?(file3), "File from source 1 was not copied") assert_equal("0", File.read(file3), "file3 got wrong contents") # Now reset sourceselect assert_nothing_raised do obj[:sourceselect] = :all end File.unlink(file1) File.unlink(file3) Puppet.err :yay assert_apply(obj) assert(FileTest.exists?(file1), "File from source 1 was not copied") assert(FileTest.exists?(file2), "File from source 2 was copied") assert(FileTest.exists?(file3), "File from source 1 was not copied") assert_equal("0", File.read(file3), "file3 got wrong contents") end def test_recursive_sourceselect dest = tempfile() source1 = tempfile() source2 = tempfile() files = [] [source1, source2, File.join(source1, "subdir"), File.join(source2, "subdir")].each_with_index do |dir, i| Dir.mkdir(dir) # Make a single file in each directory file = File.join(dir, "file%s" % i) File.open(file, "w") { |f| f.puts "yay%s" % i} # Now make a second one in each directory file = File.join(dir, "second-file%s" % i) File.open(file, "w") { |f| f.puts "yaysecond-%s" % i} files << file end obj = Puppet::Type.newfile(:path => dest, :source => [source1, source2], :sourceselect => :all, :recurse => true) assert_apply(obj) ["file0", "file1", "second-file0", "second-file1", "subdir/file2", "subdir/second-file2", "subdir/file3", "subdir/second-file3"].each do |file| path = File.join(dest, file) assert(FileTest.exists?(path), "did not create %s" % file) assert_equal("yay%s\n" % File.basename(file).sub("file", ''), File.read(path), "file was not copied correctly") end end # #594 def test_purging_missing_remote_files source = tempfile() dest = tempfile() s1 = File.join(source, "file1") s2 = File.join(source, "file2") d1 = File.join(dest, "file1") d2 = File.join(dest, "file2") Dir.mkdir(source) [s1, s2].each { |name| File.open(name, "w") { |file| file.puts "something" } } # We have to add a second parameter, because that's the only way to expose the "bug". file = Puppet::Type.newfile(:path => dest, :source => source, :recurse => true, :purge => true, :mode => "755") assert_apply(file) assert(FileTest.exists?(d1), "File1 was not copied") assert(FileTest.exists?(d2), "File2 was not copied") File.unlink(s2) assert_apply(file) assert(FileTest.exists?(d1), "File1 was not kept") assert(! FileTest.exists?(d2), "File2 was not purged") end end