retools.cache

Caching

Cache regions are used to simplify common expirations and group function caches.

To indicate functions should use cache regions, apply the decorator:

from retools.cache import cache_region

@cache_region('short_term')
def myfunction(arg1):
    return arg1

To configure the cache regions, setup the CacheRegion object:

from retools.cache import CacheRegion

CacheRegion.add_region("short_term", expires=60)

Constants

retools.cache.NoneMarker

A module global returned to indicate no value is present in Redis rather than a None object.

Functions

retools.cache.cache_region(region, *deco_args, **kwargs)[source]

Decorate a function such that its return result is cached, using a “region” to indicate the cache arguments.

Parameters:
  • region (string) – Name of the region to cache to
  • *deco_args – Optional str()-compatible arguments which will uniquely identify the key used by this decorated function, in addition to the positional arguments passed to the function itself at call time. This is recommended as it is needed to distinguish between any two functions or methods that have the same name (regardless of parent class or not).

Note

The function being decorated must only be called with positional arguments, and the arguments must support being stringified with str(). The concatenation of the str() version of each argument, combined with that of the *args sent to the decorator, forms the unique cache key.

Example:

from retools.cache import cache_region

@cache_region('short_term', 'load_things')
def load(search_term, limit, offset):
    '''Load from a database given a search term, limit, offset.'''
    return database.query(search_term)[offset:offset + limit]

The decorator can also be used with object methods. The self argument is not part of the cache key. This is based on the actual string name self being in the first argument position:

class MyThing(object):
    @cache_region('short_term', 'load_things')
    def load(self, search_term, limit, offset):
        '''Load from a database given a search term, limit, offset.'''
        return database.query(search_term)[offset:offset + limit]

Classmethods work as well - use cls as the name of the class argument, and place the decorator around the function underneath @classmethod:

class MyThing(object):
    @classmethod
    @cache_region('short_term', 'load_things')
    def load(cls, search_term, limit, offset):
        '''Load from a database given a search term, limit, offset.'''
        return database.query(search_term)[offset:offset + limit]

Note

When a method on a class is decorated, the self or cls argument in the first position is not included in the “key” used for caching.

retools.cache.invalidate_region(region)[source]

Invalidate all the namespace’s in a given region

Note

This does not actually clear the region of data, but just sets the value to expire on next access.

Parameters:region (string) – Region name
retools.cache.invalidate_function(callable, *args)

Invalidate the cache for a callable

Parameters:
  • callable (callable object) – The callable that was cached
  • *args – Arguments the function was called with that should be invalidated. If the args is just the differentiator for the function, or not present, then all values for the function will be invalidated.

Example:

@cache_region('short_term', 'small_engine')
def local_search(search_term):
    # do search and return it

@cache_region('long_term')
def lookup_folks():
    # look them up and return them

# To clear local_search for search_term = 'fred'
invalidate_function(local_search, 'fred')

# To clear all cached variations of the local_search function
invalidate_function(local_search)

# To clear out lookup_folks
invalidate_function(lookup_folks)

Classes

class retools.cache.CacheKey(region, namespace, key, today=None)[source]

Cache Key object

Generator of cache keys for a variety of purposes once provided with a region, namespace, and key (args).

class retools.cache.CacheRegion[source]

CacheRegion manager and configuration object

For organization sake, the CacheRegion object is used to configure the available cache regions, query regions for currently cached keys, and set batches of keys by region for immediate expiration.

Caching can be turned off globally by setting enabled to False:

CacheRegion.enabled = False

Statistics should also be turned on or off globally:

CacheRegion.statistics = False

However, if only some namespaces should have statistics recorded, then this should be used directly.

classmethod add_region(name, expires, redis_expiration=604800)[source]

Add a cache region to the current configuration

Parameters:
  • name (string) – The name of the cache region
  • expires (integer) – The expiration in seconds.
  • redis_expiration (integer) – How long the Redis key expiration is set for. Defaults to 1 week.
classmethod invalidate(region)[source]

Invalidate an entire region

Note

This does not actually clear the region of data, but just sets the value to expire on next access.

Parameters:region (string) – Region name
classmethod load(region, namespace, key, regenerate=True, callable=None, statistics=None)[source]

Load a value from Redis, and possibly recreate it

This method is used to load a value from Redis, and usually regenerates the value using the callable when provided.

If regenerate is False and a callable is not passed in, then NoneMarker will be returned.

Parameters:
  • region (string) – Region name
  • namespace (string) – Namespace for the value
  • key (string) – Key for this value under the namespace
  • regenerate (bool) – If False, then existing keys will always be returned regardless of cache expiration. In the event that there is no existing key and no callable was provided, then a NoneMarker will be returned.
  • callable – A callable to use when the cached value needs to be created
  • statistics (bool) – Whether or not hit/miss statistics should be updated