No OneTemporary
Actions

Size

50 KB

Referenced Files

None

Subscribers

None

View Options

	diff --git a/lib/puppet/pops/loader/loader.rb b/lib/puppet/pops/loader/loader.rb
	index f162604f5..256fc373e 100644
	--- a/lib/puppet/pops/loader/loader.rb
	+++ b/lib/puppet/pops/loader/loader.rb
	@@ -1,172 +1,180 @@
	# Loader
	# ===
	# A Loader is responsible for loading "entities" ("instantiable and executable objects in the puppet language" which
	# are type, hostclass, definition, function, and bindings.
	#
	# The main method for users of a Loader is the `load` or `load_typed methods`, which returns a previously loaded entity
	# of a given type/name, and searches and loads the entity if not already loaded.
	#
	# private entities
	# ---
	# TODO: handle loading of entities that are private. Suggest that all calls pass an origin_loader (the loader
	# where request originated (or symbol :public). A module loader has one (or possibly a list) of what is
	# considered to represent private loader - i.e. the dependency loader for a module. If an entity is private
	# it should be stored with this status, and an error should be raised if the origin_loader is not on the list
	# of accepted "private" loaders.
	# The private loaders can not be given at creation time (they are parented by the loader in question). Another
	# alternative is to check if the origin_loader is a child loader, but this requires bidirectional links
	# between loaders or a search if loader with private entity is a parent of the origin_loader).
	#
	# @api public
	#
	class Puppet::Pops::Loader::Loader

	# Produces the value associated with the given name if already loaded, or available for loading
	# by this loader, one of its parents, or other loaders visible to this loader.
	# This is the method an external party should use to "get" the named element.
	#
	# An implementor of this method should first check if the given name is already loaded by self, or a parent
	# loader, and if so return that result. If not, it should call `find` to perform the loading.
	#
	# @param type [:Symbol] the type to load
	# @param name [String, Symbol] the name of the entity to load
	# @return [Object, nil] the value or nil if not found
	#
	# @api public
	#
	def load(type, name)
	if result = load_typed(TypedName.new(type, name.to_s))
	result.value
	end
	end

	# Loads the given typed name, and returns a NamedEntry if found, else returns nil.
	# This the same a `load`, but returns a NamedEntry with origin/value information.
	#
	# @param typed_name [TypedName] - the type, name combination to lookup
	# @return [NamedEntry, nil] the entry containing the loaded value, or nil if not found
	#
	# @api public
	#
	def load_typed(typed_name)
	raise NotImplementedError.new
	end

	# Produces the value associated with the given name if defined in this loader, or nil if not defined.
	# This lookup does not trigger any loading, or search of the given name.
	# An implementor of this method may not search or look up in any other loader, and it may not
	# define the name.
	#
	# @param typed_name [TypedName] - the type, name combination to lookup
	#
	# @api private
	#
	def [] (typed_name)
	if found = get_entry(typed_name)
	found.value
	else
	nil
	end
	end

	# Searches for the given name in this loader's context (parents should already have searched their context(s) without
	# producing a result when this method is called).
	# An implementation of find typically caches the result.
	#
	# @param typed_name [TypedName] the type, name combination to lookup
	# @return [NamedEntry, nil] the entry for the loaded entry, or nil if not found
	#
	# @api private
	#
	def find(typed_name)
	raise NotImplementedError.new
	end

	# Returns the parent of the loader, or nil, if this is the top most loader. This implementation returns nil.
	def parent
	nil
	end

	+ # Produces the private loader for loaders that have a one (the visibility given to loaded entities).
	+ # For loaders that does not provide a private loader, self is returned.
	+ #
	+ # @api private
	+ def private_loader
	+ self
	+ end
	+
	# Binds a value to a name. The name should not start with '::', but may contain multiple segments.
	#
	# @param type [:Symbol] the type of the entity being set
	# @param name [String, Symbol] the name of the entity being set
	# @param origin [URI, #uri, String] the origin of the set entity, a URI, or provider of URI, or URI in string form
	# @return [NamedEntry, nil] the created entry
	#
	# @api private
	#
	def set_entry(type, name, value, origin = nil)
	raise NotImplementedError.new
	end

	# Produces a NamedEntry if a value is bound to the given name, or nil if nothing is bound.
	#
	# @param typed_name [TypedName] the type, name combination to lookup
	# @return [NamedEntry, nil] the value bound in an entry
	#
	# @api private
	#
	def get_entry(typed_name)
	raise NotImplementedError.new
	end

	# An entry for one entity loaded by the loader.
	#
	class NamedEntry
	attr_reader :typed_name
	attr_reader :value
	attr_reader :origin

	def initialize(typed_name, value, origin)
	@name = typed_name
	@value = value
	@origin = origin
	freeze()
	end
	end

	# A name/type combination that can be used as a compound hash key
	#
	class TypedName
	attr_reader :type
	attr_reader :name
	attr_reader :name_parts

	# True if name is qualified (more than a single segment)
	attr_reader :qualified

	def initialize(type, name)
	@type = type
	# relativize the name (get rid of leading ::), and make the split string available
	@name_parts = name.to_s.split(/::/)
	@name_parts.shift if name_parts[0].empty?
	@name = name_parts.join('::')
	@qualified = name_parts.size > 1
	# precompute hash - the name is frozen, so this is safe to do
	@hash = [self.class, type, @name].hash

	# Not allowed to have numeric names - 0, 010, 0x10, 1.2 etc
	if Puppet::Pops::Utils.is_numeric?(@name)
	raise ArgumentError, "Illegal attempt to use a numeric name '#{name}' at #{origin_label(origin)}."
	end

	freeze()
	end

	def hash
	@hash
	end

	def ==(o)
	o.class == self.class && type == o.type && name == o.name
	end

	alias eql? ==

	def to_s
	"#{type}/#{name}"
	end
	end
	end

	diff --git a/lib/puppet/pops/loader/module_loaders.rb b/lib/puppet/pops/loader/module_loaders.rb
	index fb54df70e..a36204dc2 100644
	--- a/lib/puppet/pops/loader/module_loaders.rb
	+++ b/lib/puppet/pops/loader/module_loaders.rb
	@@ -1,228 +1,241 @@

	# =ModuleLoaders
	# A ModuleLoader loads items from a single module.
	# The ModuleLoaders (ruby) module contains various such loaders. There is currently one concrete
	# implementation, ModuleLoaders::FileBased that loads content from the file system.
	# Other implementations can be created - if they are based on name to path mapping where the path
	# is relative to a root path, they can derive the base behavior from the ModuleLoaders::AbstractPathBasedModuleLoader class.
	#
	# Examples of such extensions could be a zip/jar/compressed file base loader.
	#
	# Notably, a ModuleLoader does not configure itself - it is given the information it needs (the root, its name etc.)
	# Logic higher up in the loader hierarchy of things makes decisions based on the "shape of modules", and "available
	# modules" to determine which module loader to use for each individual module. (There could be differences in
	# internal layout etc.)
	#
	# A module loader is also not aware of the mapping of name to relative paths - this is performed by the
	# included module Puppet::Pops::Loader::PathBasedInstantatorConfig which knows about the map from type/name to
	# relative path, and the logic that can instantiate what is expected to be found in the content of that path.
	#
	# @api private
	#
	module Puppet::Pops::Loader::ModuleLoaders
	class AbstractPathBasedModuleLoader < Puppet::Pops::Loader::BaseLoader

	# The name of the module, or nil, if this is a global "component"
	attr_reader :module_name

	# The path to the location of the module/component - semantics determined by subclass
	attr_reader :path

	# A map of type to smart-paths that help with minimizing the number of paths to scan
	attr_reader :smart_paths

	+ # A Module Loader has a private loader, it is lazily obtained on request to provide the visibility
	+ # for entities contained in the module. Since a ModuleLoader also represents an environment and it is
	+ # created a different way, this loader can be set explicitly by the loaders bootstrap logic.
	+ #
	+ # @api private
	+ attr_accessor :private_loader
	+
	# Initialize a kind of ModuleLoader for one module
	# @param parent_loader [Puppet::Pops::Loader] loader with higher priority
	# @param module_name [String] the name of the module (non qualified name), may be nil for a global "component"
	# @param path [String] the path to the root of the module (semantics defined by subclass)
	# @param loader_name [String] a name that is used for human identification (useful when module_name is nil)
	#
	def initialize(parent_loader, module_name, path, loader_name)
	super parent_loader, loader_name

	# Irrespective of the path referencing a directory or file, the path must exist.
	unless Puppet::FileSystem.exist?(path)
	raise ArgumentError, "The given path '#{path}' does not exist!"
	end

	@module_name = module_name
	@path = path
	@smart_paths = Puppet::Pops::Loader::LoaderPaths::SmartPaths.new(self)
	end

	# Finds typed/named entity in this module
	# @param typed_name [Puppet::Pops::Loader::TypedName] the type/name to find
	# @return [Puppet::Pops::Loader::Loader::NamedEntry, nil found/created entry, or nil if not found
	#
	def find(typed_name)
	# Assume it is a global name, and that all parts of the name should be used when looking up
	name_part_index = 0
	name_parts = typed_name.name_parts

	# Certain types and names can be disqualified up front
	if name_parts.size > 1
	# The name is in a name space.

	# Then entity cannot possible be in this module unless the name starts with the module name.
	# Note: If "module" represents a "global component", the module_name is nil and cannot match which is
	# ok since such a "module" cannot have namespaced content).
	#
	return nil unless name_parts[0] == module_name

	# Skip the first part of the name when computing the path since the path already contains the name of the
	# module
	name_part_index = 1
	else
	# The name is in the global name space.

	# The only globally name-spaced elements that may be loaded from modules are functions and resource types
	case typed_name.type
	when :function
	when :resource_type
	else
	# anything else cannot possibly be in this module
	# TODO: should not be allowed anyway... may have to revisit this decision
	return nil
	end
	end

	# Get the paths that actually exist in this module (they are lazily processed once and cached).
	# The result is an array (that may be empty).
	# Find the file to instantiate, and instantiate the entity if file is found
	origin = nil
	if (smart_path = smart_paths.effective_paths(typed_name.type).find do \|sp\|
	origin = sp.effective_path(typed_name, name_part_index)
	existing_path(origin)
	end)
	value = smart_path.instantiator.create(self, typed_name, origin, get_contents(origin))
	# cache the entry and return it
	set_entry(typed_name, value, origin)
	else
	nil
	end
	end

	# Abstract method that subclasses override that checks if it is meaningful to search using a generic smart path.
	# This optimization is performed to not be tricked into searching an empty directory over and over again.
	# The implementation may perform a deep search for file content other than directories and cache this in
	# and index. It is guaranteed that a call to meaningful_to_search? takes place before checking any other
	# path with relative_path_exists?.
	#
	# This optimization exists because many modules have been created from a template and they have
	# empty directories for functions, types, etc. (It is also the place to create a cached index of the content).
	#
	# @param relative_path [String] a path relative to the module's root
	# @return [Boolean] true if there is content in the directory appointed by the relative path
	#
	def meaningful_to_search?(smart_path)
	raise NotImplementedError.new
	end

	# Abstract method that subclasses override to answer if the given relative path exists, and if so returns that path
	#
	# @param relative_path [String] a path resolved by a smart path against the loader's root (if it has one)
	# @return [Boolean] true if the file exists
	#
	def existing_path(resolved_path)
	raise NotImplementedError.new
	end

	# Abstract method that subclasses override to produce the content of the effective path.
	# It should either succeed and return a String or fail with an exception.
	#
	# @param relative_path [String] a path as resolved by a smart path
	# @return [String] the content of the file
	#
	def get_contents(effective_path)
	raise NotImplementedError.new
	end

	# Abstract method that subclasses override to produce a source reference String used to identify the
	# system resource (resource in the URI sense).
	#
	# @param relative_path [String] a path relative to the module's root
	# @return [String] a reference to the source file (in file system, zip file, or elsewhere).
	#
	def get_source_ref(relative_path)
	raise NotImplementedError.new
	end
	+
	+ # Produces the private loader for the module. If this module is not already resolved, this will trigger resolution
	+ #
	+ def private_loader
	+ @private_loader \|\|= Puppet.lookup(:loaders).private_loader_for_module(module_name)
	+ end
	end

	# @api private
	#
	class FileBased < AbstractPathBasedModuleLoader

	attr_reader :smart_paths
	attr_reader :path_index

	# Create a kind of ModuleLoader for one module (Puppet Module, or module like)
	#
	# @param parent_loader [Puppet::Pops::Loader::Loader] typically the loader for the environment or root
	# @param module_name [String] the name of the module (non qualified name), may be nil for "modules" only containing globals
	# @param path [String] the path to the root of the module (semantics defined by subclass)
	# @param loader_name [String] a name that identifies the loader
	#
	def initialize(parent_loader, module_name, path, loader_name)
	super
	unless Puppet::FileSystem.directory?(path)
	raise ArgumentError, "The given module root path '#{path}' is not a directory (required for file system based module path entry)"
	end
	@path_index = Set.new()
	end

	def existing_path(effective_path)
	# Optimized, checks index instead of visiting file system
	@path_index.include?(effective_path) ? effective_path : nil
	end

	def meaningful_to_search?(smart_path)
	! add_to_index(smart_path).empty?
	end

	def to_s()
	"(ModuleLoader::FileBased '#{loader_name()}' '#{module_name()}')"
	end

	def add_to_index(smart_path)
	found = Dir.glob(File.join(smart_path.generic_path, '*', "#{smart_path.extension}"))
	@path_index.merge(found)
	found
	end

	def get_contents(effective_path)
	Puppet::FileSystem.read(effective_path)
	end
	end

	# Loads from a gem specified as a URI, gem://gemname/optional/path/in/gem, or just a String gemname.
	# The source reference (shown in errors etc.) is the expanded path of the gem as this is believed to be more
	# helpful - given the location it should be quite obvious which gem it is, without the location, the user would
	# need to go on a hunt for where the file actually is located.
	#
	# TODO: How does this get instantiated? Does the gemname refelect the name of the module (the namespace)
	# or is that specified a different way? Can a gem be the container of multiple modules?
	#
	# @api private
	#
	class GemBased < FileBased
	include Puppet::Pops::Loader::GemSupport

	attr_reader :gem_ref

	# Create a kind of ModuleLoader for one module
	# The parameters are:
	# * parent_loader - typically the loader for the root
	# * module_name - the name of the module (non qualified name)
	# * gem_ref - [URI, String] gem reference to the root of the module (URI, gem://gemname/optional/path/in/gem), or
	# just the gem's name as a String.
	#
	def initialize(parent_loader, module_name, gem_ref, loader_name)
	@gem_ref = gem_ref
	super parent_loader, module_name, gem_dir(gem_ref), loader_name
	end

	def to_s()
	"(ModuleLoader::GemBased '#{loader_name()}' '#{@gem_ref}' [#{module_name()}])"
	end
	end
	end
	diff --git a/lib/puppet/pops/loader/ruby_function_instantiator.rb b/lib/puppet/pops/loader/ruby_function_instantiator.rb
	index d298eec2e..6c2b716cf 100644
	--- a/lib/puppet/pops/loader/ruby_function_instantiator.rb
	+++ b/lib/puppet/pops/loader/ruby_function_instantiator.rb
	@@ -1,34 +1,34 @@
	# The RubyFunctionInstantiator instantiates a Puppet::Functions::Function given the ruby source
	# that calls Puppet::Functions.create_function.
	#
	class Puppet::Pops::Loader::RubyFunctionInstantiator
	# Produces an instance of the Function class with the given typed_name, or fails with an error if the
	# given ruby source does not produce this instance when evaluated.
	#
	# @param loader [Puppet::Pops::Loader::Loader] The loader the function is associated with
	# @param typed_name [Puppet::Pops::Loader::TypedName] the type / name of the function to load
	# @param source_ref [URI, String] a reference to the source / origin of the ruby code to evaluate
	# @param ruby_code_string [String] ruby code in a string
	#
	# @return [Puppet::Pops::Functions.Function] - an instantiated function with global scope closure associated with the given loader
	#
	def self.create(loader, typed_name, source_ref, ruby_code_string)
	unless ruby_code_string.is_a?(String) && ruby_code_string =~ /Puppet\:\:Functions\.create_function/
	raise ArgumentError, "The code loaded from #{source_ref} does not seem to be a Puppet 4x API function - no create_function call."
	end
	created = eval(ruby_code_string)
	unless created.is_a?(Class)
	raise ArgumentError, "The code loaded from #{source_ref} did not produce a Function class when evaluated. Got '#{created.class}'"
	end
	unless created.name.to_s == typed_name.name()
	raise ArgumentError, "The code loaded from #{source_ref} produced mis-matched name, expected '#{typed_name.name}', got #{created.name}"
	end
	# create the function instance - it needs closure (scope), and loader (i.e. where it should start searching for things
	# when calling functions etc.
	# It should be bound to global scope

	# TODO: Cheating wrt. scope - assuming it is found in the context
	closure_scope = Puppet.lookup(:global_scope) { {} }
	- created.new(closure_scope, loader)
	+ created.new(closure_scope, loader.private_loader)
	end
	end
	diff --git a/lib/puppet/pops/loader/simple_environment_loader.rb b/lib/puppet/pops/loader/simple_environment_loader.rb
	index ef9a5994f..488aea73d 100644
	--- a/lib/puppet/pops/loader/simple_environment_loader.rb
	+++ b/lib/puppet/pops/loader/simple_environment_loader.rb
	@@ -1,18 +1,20 @@
	# SimpleEnvironmentLoader
	# ===
	# This loader does not load anything and it is populated by the bootstrapping logic that loads
	# the site.pp or equivalent for an environment. It does not restrict the names of what it may contain,
	# and what is loaded here overrides any child loaders (modules).
	#
	class Puppet::Pops::Loader::SimpleEnvironmentLoader < Puppet::Pops::Loader::BaseLoader

	+ attr_accessor :private_loader
	+
	# Never finds anything, everything "loaded" is set externally
	def find(typed_name)
	nil
	end

	def to_s()
	"(SimpleEnvironmentLoader '#{loader_name}')"
	end

	-end
	\ No newline at end of file
	+end
	\ No newline at end of file
	diff --git a/lib/puppet/pops/loaders.rb b/lib/puppet/pops/loaders.rb
	index 076d66926..290fd739e 100644
	--- a/lib/puppet/pops/loaders.rb
	+++ b/lib/puppet/pops/loaders.rb
	@@ -1,234 +1,251 @@
	class Puppet::Pops::Loaders
	class LoaderError < Puppet::Error; end

	attr_reader :static_loader
	attr_reader :puppet_system_loader
	attr_reader :public_environment_loader
	attr_reader :private_environment_loader

	def initialize()
	# The static loader can only be changed after a reboot
	@@static_loader \|\|= Puppet::Pops::Loader::StaticLoader.new()

	# Create the set of loaders
	# 1. Puppet, loads from the "running" puppet - i.e. bundled functions, types, extension points and extensions
	# Does not change without rebooting the service running puppet.
	#
	@@puppet_system_loader \|\|= create_puppet_system_loader()

	# 2. Environment loader - i.e. what is bound across the environment, may change for each setup
	# TODO: loaders need to work when also running in an agent doing catalog application. There is no
	# concept of environment the same way as when running as a master (except when doing apply).
	# The creation mechanisms should probably differ between the two.
	#
	@private_environment_loader = create_environment_loader()

	# 3. module loaders are set up from the create_environment_loader, they register themselves
	end

	# Clears the cached static and puppet_system loaders (to enable testing)
	#
	def self.clear
	@@static_loader = nil
	@@puppet_system_loader = nil
	end

	def static_loader
	@@static_loader
	end

	def puppet_system_loader
	@@puppet_system_loader
	end

	def self.create_loaders()
	self.new()
	end

	def public_loader_for_module(module_name)
	md = @module_resolver[module_name] \|\| (return nil)
	- # Note, this loader is not resolved until it is asked to load something it may contain
	+ # Note, this loader is not resolved until there is interest in the visibility of entities from the
	+ # perspective of something contained in the module. (Many request may pass through a module loader
	+ # without it loading anything.
	+ # See {#private_loader_for_module}, and not in {#configure_loaders_for_modules}
	md.public_loader
	end

	def private_loader_for_module(module_name)
	md = @module_resolver[module_name] \|\| (return nil)
	+ # Since there is interest in the visibility from the perspective of entities contained in the
	+ # module, it must be resolved (to provide this visibility).
	+ # See {#configure_loaders_for_modules}
	unless md.resolved?
	@module_resolver.resolve(md)
	end
	md.private_loader
	end

	private

	def create_puppet_system_loader()
	module_name = nil
	loader_name = 'puppet_system'

	# Puppet system may be installed in a fixed location via RPM, installed as a Gem, via source etc.
	# The only way to find this across the different ways puppet can be installed is
	# to search up the path from this source file's __FILE__ location until it finds the parent of
	# lib/puppet... e.g.. dirname(__FILE__)/../../.. (i.e. <somewhere>/lib/puppet/pops/loaders.rb).
	#
	puppet_lib = File.join(File.dirname(__FILE__), '../../..')
	Puppet::Pops::Loader::ModuleLoaders::FileBased.new(static_loader, module_name, puppet_lib, loader_name)
	end

	def create_environment_loader()
	# This defines where to start parsing/evaluating - the "initial import" (to use 3x terminology)
	# Is either a reference to a single .pp file, or a directory of manifests. If the environment becomes
	# a module and can hold functions, types etc. then these are available across all other modules without
	# them declaring this dependency - it is however valuable to be able to treat it the same way
	# bindings and other such system related configuration.

	# This is further complicated by the many options available:
	# - The environment may not have a directory, the code comes from one appointed 'manifest' (site.pp)
	# - The environment may have a directory and also point to a 'manifest'
	# - The code to run may be set in settings (code)

	# Further complication is that there is nothing specifying what the visibility is into
	# available modules. (3x is everyone sees everything).
	# Puppet binder currently reads confdir/bindings - that is bad, it should be using the new environment support.

	current_environment = Puppet.lookup(:current_environment)
	# The environment is not a namespace, so give it a nil "module_name"
	module_name = nil
	loader_name = "environment:#{current_environment.name}"
	env_dir = Puppet[:environmentdir]
	if env_dir.nil?
	# Use an environment loader that can be populated externally
	loader = Puppet::Pops::Loader::SimpleEnvironmentLoader.new(puppet_system_loader, loader_name)
	else
	envdir_path = File.join(env_dir, current_environment.name.to_s)
	# TODO: Representing Environment as a Module - needs something different (not all types are supported),
	# and it must be able to import .pp code from 3x manifest setting, or from code setting as well as from
	# a manifests directory under the environment's root. The below is cheating...
	#
	loader = Puppet::Pops::Loader::ModuleLoaders::FileBased(puppet_system_loader, module_name, envdir_path, loader_name)
	end
	# An environment has a module path even if it has a null loader
	configure_loaders_for_modules(loader, current_environment)
	# modules should see this loader
	@public_environment_loader = loader

	# Code in the environment gets to see all modules (since there is no metadata for the environment)
	# but since this is not given to the module loaders, they can not load global code (since they can not
	# have prior knowledge about this
	loader = Puppet::Pops::Loader::DependencyLoader.new(loader, "environment", @module_resolver.all_module_loaders())

	+ # The module loader gets the private loader via a lazy operation to look up the module's private loader.
	+ # This does not work for an environment since it is not resolved the same way.
	+ # TODO: The EnvironmentLoader could be a specialized loader instead of using a ModuleLoader to do the work.
	+ # This is subject to future design - an Environment may move more in the direction of a Module.
	+ @public_environment_loader.private_loader = loader
	loader
	end

	def configure_loaders_for_modules(parent_loader, current_environment)
	@module_resolver = mr = ModuleResolver.new()
	current_environment.modules.each do \|puppet_module\|
	# Create data about this module
	md = LoaderModuleData.new(puppet_module)
	mr[puppet_module.name] = md
	md.public_loader = Puppet::Pops::Loader::ModuleLoaders::FileBased.new(parent_loader, md.name, md.path, md.name)
	end
	+ # NOTE: Do not resolve all modules here - this is wasteful if only a subset of modules / functions are used
	+ # The resolution is triggered by asking for a module's private loader, since this means there is interest
	+ # in the visibility from that perspective.
	+ # If later, it is wanted that all resolutions should be made up-front (to capture errors eagerly, this
	+ # can be introduced (better for production), but may be irritating in development mode.
	end

	# =LoaderModuleData
	# Information about a Module and its loaders.
	# TODO: should have reference to real model element containing all module data; this is faking it
	# TODO: Should use Puppet::Module to get the metadata (as a hash) - a somewhat blunt instrument, but that is
	# what is available with a reasonable API.
	#
	class LoaderModuleData

	attr_accessor :state
	attr_accessor :public_loader
	attr_accessor :private_loader
	attr_accessor :resolutions

	# The Puppet::Module this LoaderModuleData represents in the loader configuration
	attr_reader :puppet_module

	# @param puppet_module [Puppet::Module] the module instance for the module being represented
	#
	def initialize(puppet_module)
	@state = :initial
	@puppet_module = puppet_module
	@resolutions = []
	@public_loader = nil
	@private_loader = nil
	end

	def name
	@puppet_module.name
	end

	def version
	@puppet_module.version
	end

	def path
	@puppet_module.path
	end

	+ # TODO: UNUSED? Remove this
	def requirements
	nil # FAKE: this says "wants to see everything"
	end

	def resolved?
	@state == :resolved
	end
	end

	# Resolves module loaders - resolution of model dependencies is done by Puppet::Module
	#
	class ModuleResolver

	def initialize()
	@index = {}
	@all_module_loaders = nil
	end

	def [](name)
	@index[name]
	end

	def []=(name, module_data)
	@index[name] = module_data
	end

	def all_module_loaders
	@all_module_loaders \|\|= @index.values.map {\|md\| md.public_loader }
	end

	def resolve(module_data)
	return if module_data.resolved?
	pm = module_data.puppet_module
	# Resolution rules
	# If dependencies.nil? means "see all other modules" (This to make older modules work, and modules w/o metadata)
	# TODO: Control via flag/feature ?
	module_data.private_loader =
	if pm.dependencies.nil?
	# see everything
	if Puppet::Util::Log.level == :debug
	Puppet.debug("ModuleLoader: module '#{module_data.name}' has unknown dependencies - it will have all other modules visible")
	end

	- Puppet::Pops::Loader::DependencyLoader.new(module_data.loader, module_data.name, all_module_loaders())
	+ Puppet::Pops::Loader::DependencyLoader.new(module_data.public_loader, module_data.name, all_module_loaders())
	else
	# If module has resolutions they must resolve - it will not see into other modules otherwise
	# TODO: possible give errors if there are unresolved references
	# i.e. !pm.unmet_dependencies.empty? (if module lacks metadata it is considered to have met all).
	# The face "module" can display error information.
	# Here, we are just giving up without explaining - the user can check with the module face (or console)
	#
	unless pm.unmet_dependencies.empty?
	# TODO: Exception or just warning?
	Puppet.warning("ModuleLoader: module '#{module_data.name}' has unresolved dependencies"+
	" - it will only see those that are resolved."+
	" Use 'puppet module list --tree' to see information about modules")
	# raise Puppet::Pops::Loader::Loader::Error, "Loader Error: Module '#{module_data.name}' has unresolved dependencies - use 'puppet module list --tree' to see information"
	end
	dependency_loaders = pm.dependencies_as_modules.map { \|dep\| @index[dep.name].loader }
	- Puppet::Pops::Loader::DependencyLoader.new(module_data.loader, module_data.name, dependency_loaders)
	+ Puppet::Pops::Loader::DependencyLoader.new(module_data.public_loader, module_data.name, dependency_loaders)
	end

	end
	end
	end
	\ No newline at end of file
	diff --git a/spec/fixtures/unit/pops/loaders/loaders/wo_metadata_module/modules/moduleb/lib/puppet/functions/moduleb/rb_func_b.rb b/spec/fixtures/unit/pops/loaders/loaders/wo_metadata_module/modules/moduleb/lib/puppet/functions/moduleb/rb_func_b.rb
	new file mode 100644
	index 000000000..785b2f4dc
	--- /dev/null
	+++ b/spec/fixtures/unit/pops/loaders/loaders/wo_metadata_module/modules/moduleb/lib/puppet/functions/moduleb/rb_func_b.rb
	@@ -0,0 +1,6 @@
	+Puppet::Functions.create_function(:'moduleb::rb_func_b') do
	+ def rb_func_b()
	+ # Should be able to call modulea::rb_func_a()
	+ call_function('modulea::rb_func_a') + " + I am moduleb::rb_func_b()"
	+ end
	+end
	\ No newline at end of file
	diff --git a/spec/fixtures/unit/pops/loaders/loaders/wo_metadata_module/modules/moduleb/manifests/init.pp b/spec/fixtures/unit/pops/loaders/loaders/wo_metadata_module/modules/moduleb/manifests/init.pp
	new file mode 100644
	index 000000000..4cd7ab7b7
	--- /dev/null
	+++ b/spec/fixtures/unit/pops/loaders/loaders/wo_metadata_module/modules/moduleb/manifests/init.pp
	@@ -0,0 +1,3 @@
	+class moduleb {
	+
	+}
	diff --git a/spec/unit/functions/assert_type_spec.rb b/spec/unit/functions/assert_type_spec.rb
	index 7fb84d651..f4bd72162 100644
	--- a/spec/unit/functions/assert_type_spec.rb
	+++ b/spec/unit/functions/assert_type_spec.rb
	@@ -1,52 +1,59 @@
	require 'spec_helper'
	require 'puppet/pops'
	require 'puppet/loaders'

	describe 'the assert_type function' do

	after(:all) { Puppet::Pops::Loaders.clear }
	- let(:func) do
	+
	+ around(:each) do \|example\|
	loaders = Puppet::Pops::Loaders.new()
	- loaders.puppet_system_loader.load(:function, 'assert_type')
	+ Puppet.override({:loaders => loaders}, "test-example") do
	+ example.run
	+ end
	+ end
	+
	+ let(:func) do
	+ Puppet.lookup(:loaders).puppet_system_loader.load(:function, 'assert_type')
	end

	it 'asserts compliant type by returning the value' do
	expect(func.call({}, type(String), 'hello world')).to eql('hello world')
	end

	it 'accepts type given as a String' do
	expect(func.call({}, 'String', 'hello world')).to eql('hello world')
	end

	it 'asserts non compliant type by raising an error' do
	expect do
	func.call({}, type(Integer), 'hello world')
	end.to raise_error(Puppet::ParseError, /does not match actual/)
	end

	it 'checks that first argument is a type' do
	expect do
	func.call({}, 10, 10)
	end.to raise_error(ArgumentError, Regexp.new(Regexp.escape(
	"function 'assert_type' called with mis-matched arguments
	expected one of:
	assert_type(Type type, Optional[Object] value) - arg count {2}
	assert_type(String type_string, Optional[Object] value) - arg count {2}
	actual:
	assert_type(Integer, Integer) - arg count {2}")))
	end

	it 'allows the second arg to be undef/nil)' do
	expect do
	func.call({}, optional(String), nil)
	end.to_not raise_error(ArgumentError)
	end

	def optional(type_ref)
	Puppet::Pops::Types::TypeFactory.optional(type(type_ref))
	end

	def type(type_ref)
	Puppet::Pops::Types::TypeFactory.type_of(type_ref)
	end
	end
	diff --git a/spec/unit/pops/loaders/dependency_loader_spec.rb b/spec/unit/pops/loaders/dependency_loader_spec.rb
	index 5f12528a2..9f6dc90fd 100644
	--- a/spec/unit/pops/loaders/dependency_loader_spec.rb
	+++ b/spec/unit/pops/loaders/dependency_loader_spec.rb
	@@ -1,43 +1,51 @@
	require 'spec_helper'
	require 'puppet_spec/files'
	require 'puppet/pops'
	require 'puppet/loaders'

	describe 'dependency loader' do
	include PuppetSpec::Files

	let(:static_loader) { Puppet::Pops::Loader::StaticLoader.new() }

	+
	describe 'FileBased module loader' do
	+ around(:each) do \|example\|
	+ loaders = Puppet::Pops::Loaders.new()
	+ Puppet.override({:loaders => loaders}, "test-example") do
	+ example.run
	+ end
	+ end
	+
	it 'load something in global name space raises an error' do
	module_dir = dir_containing('testmodule', {
	'lib' => { 'puppet' => { 'functions' => { 'testmodule' => {
	'foo.rb' => 'Puppet::Functions.create_function("foo") { def foo; end; }'
	}}}}})

	module_loader = Puppet::Pops::Loader::ModuleLoaders::FileBased.new(static_loader, 'testmodule', module_dir, 'test1')
	dep_loader = Puppet::Pops::Loader::DependencyLoader.new(static_loader, 'test-dep', [module_loader])
	expect do
	dep_loader.load_typed(typed_name(:function, 'testmodule::foo')).value
	end.to raise_error(ArgumentError, /produced mis-matched name, expected 'testmodule::foo', got foo/)
	end

	it 'can load something in a qualified name space' do
	module_dir = dir_containing('testmodule', {
	'lib' => { 'puppet' => { 'functions' => { 'testmodule' => {
	'foo.rb' => 'Puppet::Functions.create_function("testmodule::foo") { def foo; end; }'
	}}}}})
	module_loader = Puppet::Pops::Loader::ModuleLoaders::FileBased.new(static_loader, 'testmodule', module_dir, 'test1')
	dep_loader = Puppet::Pops::Loader::DependencyLoader.new(static_loader, 'test-dep', [module_loader])

	function = dep_loader.load_typed(typed_name(:function, 'testmodule::foo')).value

	expect(function.class.name).to eq('testmodule::foo')
	expect(function.is_a?(Puppet::Functions::Function)).to eq(true)
	end
	end

	def typed_name(type, name)
	Puppet::Pops::Loader::Loader::TypedName.new(type, name)
	end
	end
	diff --git a/spec/unit/pops/loaders/loaders_spec.rb b/spec/unit/pops/loaders/loaders_spec.rb
	index a3ee91ac3..810bb59dd 100644
	--- a/spec/unit/pops/loaders/loaders_spec.rb
	+++ b/spec/unit/pops/loaders/loaders_spec.rb
	@@ -1,78 +1,133 @@
	require 'spec_helper'
	require 'puppet_spec/files'

	require 'puppet/pops'
	require 'puppet/loaders'

	describe 'loaders' do
	include PuppetSpec::Files

	def config_dir(config_name)
	my_fixture(config_name)
	end

	# Loaders caches the puppet_system_loader, must reset between tests
	#
	before(:each) { Puppet::Pops::Loaders.clear() }

	it 'creates a puppet_system loader' do
	loaders = Puppet::Pops::Loaders.new()
	expect(loaders.puppet_system_loader().class).to be(Puppet::Pops::Loader::ModuleLoaders::FileBased)
	end

	it 'creates an environment loader' do
	loaders = Puppet::Pops::Loaders.new()
	# When this test is running, there is no environments dir configured, and a NullLoader is therefore used a.t.m
	expect(loaders.public_environment_loader().class).to be(Puppet::Pops::Loader::SimpleEnvironmentLoader)
	# The default name of the enironment is 'root', and the loader should identify itself that way
	expect(loaders.public_environment_loader().to_s).to eql("(SimpleEnvironmentLoader 'environment:root')")

	expect(loaders.private_environment_loader().class).to be(Puppet::Pops::Loader::DependencyLoader)
	expect(loaders.private_environment_loader().to_s).to eql("(DependencyLoader 'environment' [])")
	end

	context 'when delegating 3x to 4x' do
	before(:each) { Puppet[:biff] = true }

	it 'the puppet system loader can load 3x functions' do
	loaders = Puppet::Pops::Loaders.new()
	puppet_loader = loaders.puppet_system_loader()
	function = puppet_loader.load_typed(typed_name(:function, 'sprintf')).value
	expect(function.class.name).to eq('sprintf')
	expect(function.is_a?(Puppet::Functions::Function)).to eq(true)
	end
	end

	- # TODO: LOADING OF MODULES ON MODULEPATH
	context 'loading from path with single module' do
	before do
	env = Puppet::Node::Environment.create(:'test', [File.join(config_dir('single_module'), 'modules')], '')
	overrides = {
	:current_environment => env
	}
	Puppet.push_context(overrides, "single-module-test-loaders")
	end

	after do
	Puppet.pop_context()
	end

	it 'can load from a module path' do
	loaders = Puppet::Pops::Loaders.new()
	- modulea_loader = loaders.public_loader_for_module('modulea')
	- expect(modulea_loader.class).to eql(Puppet::Pops::Loader::ModuleLoaders::FileBased)
	+ Puppet.override({:loaders => loaders}, 'testcase') do
	+ modulea_loader = loaders.public_loader_for_module('modulea')
	+ expect(modulea_loader.class).to eql(Puppet::Pops::Loader::ModuleLoaders::FileBased)

	- function = modulea_loader.load_typed(typed_name(:function, 'rb_func_a')).value
	- expect(function.is_a?(Puppet::Functions::Function)).to eq(true)
	- expect(function.class.name).to eq('rb_func_a')
	+ function = modulea_loader.load_typed(typed_name(:function, 'rb_func_a')).value
	+ expect(function.is_a?(Puppet::Functions::Function)).to eq(true)
	+ expect(function.class.name).to eq('rb_func_a')

	- function = modulea_loader.load_typed(typed_name(:function, 'modulea::rb_func_a')).value
	- expect(function.is_a?(Puppet::Functions::Function)).to eq(true)
	- expect(function.class.name).to eq('modulea::rb_func_a')
	+ function = modulea_loader.load_typed(typed_name(:function, 'modulea::rb_func_a')).value
	+ expect(function.is_a?(Puppet::Functions::Function)).to eq(true)
	+ expect(function.class.name).to eq('modulea::rb_func_a')
	+ end
	+ end
	+ end
	+
	+ context 'loading from path with two module, one without meta-data' do
	+ before do
	+ module_path = [File.join(config_dir('single_module'), 'modules'), File.join(config_dir('wo_metadata_module'), 'modules')]
	+ env = Puppet::Node::Environment.create(:'test', module_path, '')
	+ overrides = {
	+ :current_environment => env,
	+ }
	+ Puppet.push_context(overrides, "two-modules-test-loaders")
	+ end
	+
	+ after do
	+ Puppet.pop_context()
	+ end
	+
	+ it 'can load from module with metadata' do
	+ loaders = Puppet::Pops::Loaders.new
	+ Puppet.override({:loaders => loaders}, 'testcase') do
	+ modulea_loader = loaders.public_loader_for_module('modulea')
	+ expect(modulea_loader.class).to eql(Puppet::Pops::Loader::ModuleLoaders::FileBased)
	+
	+ function = modulea_loader.load_typed(typed_name(:function, 'rb_func_a')).value
	+ expect(function.is_a?(Puppet::Functions::Function)).to eq(true)
	+ expect(function.class.name).to eq('rb_func_a')
	+
	+ function = modulea_loader.load_typed(typed_name(:function, 'modulea::rb_func_a')).value
	+ expect(function.is_a?(Puppet::Functions::Function)).to eq(true)
	+ expect(function.class.name).to eq('modulea::rb_func_a')
	+ end
	+ end
	+
	+ it 'can load from module with metadata' do
	+ loaders = Puppet::Pops::Loaders.new
	+ Puppet.override({:loaders => loaders}, 'testcase') do
	+ moduleb_loader = loaders.public_loader_for_module('moduleb')
	+ expect(moduleb_loader.class).to eql(Puppet::Pops::Loader::ModuleLoaders::FileBased)
	+
	+ function = moduleb_loader.load_typed(typed_name(:function, 'moduleb::rb_func_b')).value
	+ expect(function.is_a?(Puppet::Functions::Function)).to eq(true)
	+ expect(function.class.name).to eq('moduleb::rb_func_b')
	+ end
	+ end
	+
	+ it 'module without metadata has all modules visible' do
	+ loaders = Puppet::Pops::Loaders.new
	+ Puppet.override({:loaders => loaders}, 'testcase') do
	+ moduleb_loader = loaders.private_loader_for_module('moduleb')
	+
	+ function = moduleb_loader.load_typed(typed_name(:function, 'moduleb::rb_func_b')).value
	+ result = function.call({})
	+ expect(result).to eql("I am modulea::rb_func_a() + I am moduleb::rb_func_b()")
	+ end
	end
	end

	def typed_name(type, name)
	Puppet::Pops::Loader::Loader::TypedName.new(type, name)
	end
	end
	diff --git a/spec/unit/pops/loaders/module_loaders_spec.rb b/spec/unit/pops/loaders/module_loaders_spec.rb
	index 24bad0cf4..a5eba200c 100644
	--- a/spec/unit/pops/loaders/module_loaders_spec.rb
	+++ b/spec/unit/pops/loaders/module_loaders_spec.rb
	@@ -1,122 +1,124 @@
	require 'spec_helper'
	require 'puppet_spec/files'
	require 'puppet/pops'
	require 'puppet/loaders'

	-describe 'module loaders' do
	+describe 'FileBased module loader' do
	include PuppetSpec::Files

	let(:static_loader) { Puppet::Pops::Loader::StaticLoader.new() }
	+ around(:each) do \|example\|
	+ loaders = Puppet::Pops::Loaders.new()
	+ Puppet.override({:loaders => loaders}, "test-example") do
	+ example.run
	+ end
	+ end

	- describe 'FileBased module loader' do
	- it 'can load a 4x function API ruby function in global name space' do
	- module_dir = dir_containing('testmodule', {
	- 'lib' => {
	- 'puppet' => {
	- 'functions' => {
	+ it 'can load a 4x function API ruby function in global name space' do
	+ module_dir = dir_containing('testmodule', {
	+ 'lib' => {
	+ 'puppet' => {
	+ 'functions' => {
	+ 'foo4x.rb' => <<-CODE
	+ Puppet::Functions.create_function(:foo4x) do
	+ def foo4x()
	+ 'yay'
	+ end
	+ end
	+ CODE
	+ }
	+ }
	+ }
	+ })
	+
	+ module_loader = Puppet::Pops::Loader::ModuleLoaders::FileBased.new(static_loader, 'testmodule', module_dir, 'test1')
	+ function = module_loader.load_typed(typed_name(:function, 'foo4x')).value
	+ expect(function.class.name).to eq('foo4x')
	+ expect(function.is_a?(Puppet::Functions::Function)).to eq(true)
	+ end
	+
	+ it 'can load a 4x function API ruby function in qualified name space' do
	+ module_dir = dir_containing('testmodule', {
	+ 'lib' => {
	+ 'puppet' => {
	+ 'functions' => {
	+ 'testmodule' => {
	'foo4x.rb' => <<-CODE
	- Puppet::Functions.create_function(:foo4x) do
	+ Puppet::Functions.create_function('testmodule::foo4x') do
	def foo4x()
	'yay'
	end
	end
	CODE
	- }
	}
	}
	- })
	+ }
	+ }})

	- module_loader = Puppet::Pops::Loader::ModuleLoaders::FileBased.new(static_loader, 'testmodule', module_dir, 'test1')
	- function = module_loader.load_typed(typed_name(:function, 'foo4x')).value
	- expect(function.class.name).to eq('foo4x')
	- expect(function.is_a?(Puppet::Functions::Function)).to eq(true)
	- end
	+ module_loader = Puppet::Pops::Loader::ModuleLoaders::FileBased.new(static_loader, 'testmodule', module_dir, 'test1')
	+ function = module_loader.load_typed(typed_name(:function, 'testmodule::foo4x')).value
	+ expect(function.class.name).to eq('testmodule::foo4x')
	+ expect(function.is_a?(Puppet::Functions::Function)).to eq(true)
	+ end
	+
	+ it 'makes parent loader win over entries in child' do
	+ module_dir = dir_containing('testmodule', {
	+ 'lib' => { 'puppet' => { 'functions' => { 'testmodule' => {
	+ 'foo.rb' => <<-CODE
	+ Puppet::Functions.create_function('testmodule::foo') do
	+ def foo()
	+ 'yay'
	+ end
	+ end
	+ CODE
	+ }}}}})
	+ module_loader = Puppet::Pops::Loader::ModuleLoaders::FileBased.new(static_loader, 'testmodule', module_dir, 'test1')
	+
	+ module_dir2 = dir_containing('testmodule2', {
	+ 'lib' => { 'puppet' => { 'functions' => { 'testmodule2' => {
	+ 'foo.rb' => <<-CODE
	+ raise "should not get here"
	+ CODE
	+ }}}}})
	+ module_loader2 = Puppet::Pops::Loader::ModuleLoaders::FileBased.new(module_loader, 'testmodule2', module_dir2, 'test2')
	+
	+ function = module_loader2.load_typed(typed_name(:function, 'testmodule::foo')).value
	+
	+ expect(function.class.name).to eq('testmodule::foo')
	+ expect(function.is_a?(Puppet::Functions::Function)).to eq(true)
	+ end
	+
	+ context 'when delegating 3x to 4x' do
	+ before(:each) { Puppet[:biff] = true }

	- it 'can load a 4x function API ruby function in qualified name space' do
	+ it 'can load a 3x function API ruby function in global name space' do
	module_dir = dir_containing('testmodule', {
	'lib' => {
	'puppet' => {
	- 'functions' => {
	- 'testmodule' => {
	- 'foo4x.rb' => <<-CODE
	- Puppet::Functions.create_function('testmodule::foo4x') do
	- def foo4x()
	- 'yay'
	- end
	- end
	+ 'parser' => {
	+ 'functions' => {
	+ 'foo3x.rb' => <<-CODE
	+ Puppet::Parser::Functions::newfunction(
	+ :foo3x, :type => :rvalue,
	+ :arity => 1
	+ ) do \|args\|
	+ args[0]
	+ end
	CODE
	- }
	+ }
	}
	}
	}})

	- module_loader = Puppet::Pops::Loader::ModuleLoaders::FileBased.new(static_loader, 'testmodule', module_dir, 'test1')
	- function = module_loader.load_typed(typed_name(:function, 'testmodule::foo4x')).value
	- expect(function.class.name).to eq('testmodule::foo4x')
	- expect(function.is_a?(Puppet::Functions::Function)).to eq(true)
	- end
	-
	- it 'makes parent loader win over entries in child' do
	- module_dir = dir_containing('testmodule', {
	- 'lib' => { 'puppet' => { 'functions' => { 'testmodule' => {
	- 'foo.rb' => <<-CODE
	- Puppet::Functions.create_function('testmodule::foo') do
	- def foo()
	- 'yay'
	- end
	- end
	- CODE
	- }}}}})
	- module_loader = Puppet::Pops::Loader::ModuleLoaders::FileBased.new(static_loader, 'testmodule', module_dir, 'test1')
	-
	- module_dir2 = dir_containing('testmodule2', {
	- 'lib' => { 'puppet' => { 'functions' => { 'testmodule2' => {
	- 'foo.rb' => <<-CODE
	- raise "should not get here"
	- CODE
	- }}}}})
	- module_loader2 = Puppet::Pops::Loader::ModuleLoaders::FileBased.new(module_loader, 'testmodule2', module_dir2, 'test2')
	-
	- function = module_loader2.load_typed(typed_name(:function, 'testmodule::foo')).value
	-
	- expect(function.class.name).to eq('testmodule::foo')
	- expect(function.is_a?(Puppet::Functions::Function)).to eq(true)
	+ module_loader = Puppet::Pops::Loader::ModuleLoaders::FileBased.new(static_loader, 'testmodule', module_dir, 'test1')
	+ function = module_loader.load_typed(typed_name(:function, 'foo3x')).value
	+ expect(function.class.name).to eq('foo3x')
	+ expect(function.is_a?(Puppet::Functions::Function)).to eq(true)
	end
	-
	- context 'when delegating 3x to 4x' do
	- before(:each) { Puppet[:biff] = true }
	-
	- it 'can load a 3x function API ruby function in global name space' do
	- module_dir = dir_containing('testmodule', {
	- 'lib' => {
	- 'puppet' => {
	- 'parser' => {
	- 'functions' => {
	- 'foo3x.rb' => <<-CODE
	- Puppet::Parser::Functions::newfunction(
	- :foo3x, :type => :rvalue,
	- :arity => 1
	- ) do \|args\|
	- args[0]
	- end
	- CODE
	- }
	- }
	- }
	- }})
	-
	- module_loader = Puppet::Pops::Loader::ModuleLoaders::FileBased.new(static_loader, 'testmodule', module_dir, 'test1')
	- function = module_loader.load_typed(typed_name(:function, 'foo3x')).value
	- expect(function.class.name).to eq('foo3x')
	- expect(function.is_a?(Puppet::Functions::Function)).to eq(true)
	- end
	- end
	-
	- # Gives error when loading something with mismatched name
	-
	end

	+
	def typed_name(type, name)
	Puppet::Pops::Loader::Loader::TypedName.new(type, name)
	end
	end

File Metadata

Mime Type: text/x-diff
Expires: Fri, Nov 1, 8:14 AM (1 d, 3 h)
Storage Engine: blob
Storage Format: Raw Data
Storage Handle: 10074846
Default Alt Text: (50 KB)

No OneTemporaryActions

View Options

File Metadata

Event Timeline

No OneTemporary
Actions