HomePhorge

(PUP-4461) Turn off caching of loaded system functions
3685befc9ae9Unpublished

Unpublished Commit ยท Learn More

Repository Importing: This repository is still importing.

Description

(PUP-4461) Turn off caching of loaded system functions

Before this, the configured system loader (loading functions supplied in
puppet core) would cache all loaded functions for the lifetime of the
process. This was done to reduce loading time since hot-loading of the
function code is not required (requires a restart of the process, just
like loading any other puppet runtime code).

Since functions are not only loaded code-wise, but are also tied to a
closure scope, and this scope is associated with a compiler, all loaded
types, a catalog etc. this lead to binding a more or less complete
environment in memory. Worst case, if each request called a unique
function there would be as many environment copies in memory as there
are functions.

In addition to the memory leak, functions such as hiera_include would
get the old information when navigating to the catalog. This would not
be the case for calls to 3.x functions. The hiera_include function is
the only function that was bit by this problem, since all other funcions
that operate on scope (using state rather than given arguments) operate
on the calling scope as opposed to the functions's closure scope.

The fix for this is simply to not cache the loaders data structures for
system loading. Instead, functions are loaded fresh for each environment
irrespective of if they are loaded from the puppet core system, from an
environment, or from a module.

There is opportunity to optiize this should the loading become a
bottleneck by making the static loader cache loaded system function
classs, and the system loader simply looking them up and instantiating
them (binding them to a closure). That would trade loading and parsing
of ruby code for some lookup in the loader's hierarchy. In practice, the
loading of these functions are probably not noticable - it has no effect
at all on a single run with puppet apply.

Event Timeline

Henrik Lindberg <henrik.lindberg@cloudsmith.com> committed rPU3685befc9ae9: (PUP-4461) Turn off caching of loaded system functions (authored by Henrik Lindberg <henrik.lindberg@cloudsmith.com>).Apr 27 2015, 2:56 PM