3. The Recipe interface¶
-
class
cpl.
Recipe
(name, filename=None, version=None, threaded=False)¶ Pluggable Data Reduction Module (PDRM) from a ESO pipeline.
Recipes are loaded from shared libraries that are provided with the pipeline library of the instrument. The module does not need to be linked to the same library version as the one used for the compilation of python-cpl. Currently, recipes compiled with CPL versions from 4.0 are supported. The list of supported versions is stored as
cpl.cpl_versions
.The libraries are searched in the directories specified by the class attribute
Recipe.path
or its subdirectories. The search path is automatically set to the esorex path whencpl.esorex.init()
is called.
3.1. Static members¶
-
Recipe.
path
= ['.']¶ Search path for the recipes. It may be set to either a string, or to a list of strings. All shared libraries in the search path and their subdirectories are searched for CPL recipes. On default, the path is set to the current directory.
The search path is automatically set to the esorex path when
cpl.esorex.init()
is called.
-
Recipe.
memory_mode
= 0¶ CPL memory management mode. The valid values are
- 0
- Use the default system functions for memory handling
- 1
- Exit if a memory-allocation fails, provide checking for memory leaks, limited reporting of memory allocation and limited protection on deallocation of invalid pointers.
- 2
- Exit if a memory-allocation fails, provide checking for memory leaks, extended reporting of memory allocation and protection on deallocation of invalid pointers.
Note
This variable is only effective before the CPL library was initialized. Even
cpl.Recipe.list()
initializes the library. Therefore it is highly recommended to set this as the first action after importingcpl
.
-
static
Recipe.
list
()¶ Return a list of recipes.
Searches for all recipes in in the directory specified by the class attribute
Recipe.path
or its subdirectories.
-
static
Recipe.
set_maxthreads
(n)¶ Set the maximal number of threads to be executed in parallel.
Note
This affects only threads that are started afterwards with the
threaded = True
flag.See also
3.2. Constructor¶
-
Recipe.
__init__
(name, filename=None, version=None, threaded=False)¶ Try to load a recipe with the specified name in the directory specified by the class attribute
Recipe.path
or its subdirectories.Parameters: - name (
str
) – Name of the recipe. Required. Usecpl.Recipe.list()
to get a list of available recipes. - filename (
str
) – Name of the shared library. Optional. If not set,Recipe.path
is searched for the library file. - version (
int
orstr
) – Version number. Optional. If not set, the newest version is loaded. - threaded (
bool
) – Run the recipe in the background, returning immediately after calling it. Default isFalse
. This may be also set as an attribute or specified as a parameter when calling the recipe.
- name (
3.3. Common attributes and methods¶
These attributes and methods are available for all recipes.
-
Recipe.
__name__
¶ Recipe name.
-
Recipe.
__file__
= None¶ Shared library file name.
Author name
-
Recipe.
__email__
¶ Author email
-
Recipe.
__copyright__
¶ Copyright string of the recipe
-
Recipe.
description
¶ Pair (synopsis, description) of two strings.
-
Recipe.
version
¶ Pair (versionnumber, versionstring) of an integer and a string. The integer will be increased on development progress.
-
Recipe.
cpl_version
¶ Version of the CPL library that is linked to the recipe, as a string
-
Recipe.
cpl_description
¶ Version numbers of CPL and its libraries that were linked to the recipe, as a string.
-
Recipe.
output_dir
¶ Output directory if specified, or
None
. The recipe will write the output files into this directory and return their file names. If the directory does not exist, it will be created before the recipe is executed. Output files within the output directory will be silently overwritten. If no output directory is set, the recipe call will result inastropy.io.fits.HDUList
result objects. The output directory may be also set as parameter in the recipe call.
-
Recipe.
temp_dir
¶ Base directory for temporary directories where the recipe is executed. The working dir is created as a subdir with a random file name. If set to
None
, the system temp dir is used. Defaults to'.'
.
-
Recipe.
threaded
¶ Specify whether the recipe should be executed synchroniously or as an extra process in the background.
See also
-
Recipe.
tag
¶ Default tag when the recipe is called. This is set automatically only if the recipe provided the information about input tags. Otherwise this tag has to be set manually.
Possible tags for the raw input frames, or ‘
None
if this information is not provided by the recipe.
-
Recipe.
output
¶ Return a dictionary of output frame tags.
Keys are the tag names, values are the corresponding list of output tags. If the recipe does not provide this information, an exception is raised.
-
Recipe.
memory_dump
¶ If set to 1, a memory dump is issued to stdout if the memory was not totally freed after the execution. If set to 2, the dump is always issued. Standard is 0: nothing dumped.
3.4. Recipe parameters¶
Recipe parameters may be set either via the Recipe.param
attribute or
as named keywords on the run execution. A value set in the recipe call will
overwrite any value that was set previously in the Recipe.param
attribute for that specific call.
-
Recipe.
param
¶ This attribute contains all recipe parameters. It is iteratable and then returns all individual parameters:
>>> for p in muse_scibasic.param: ... print p.name, p.value, p.default ... nifu None 99 cr None dcr xbox None 15 ybox None 40 passes None 2 thres None 4.5 sample None False dlambda None 1.2
On interactive sessions, all parameter settings can be easily printed by printing the
param
attribute of the recipe:>>> print muse_scibasic.param [Parameter('nifu', default=99), Parameter('cr', default=dcr), Parameter('xbox', default=15), Parameter('ybox', default=40), Parameter('passes', default=2), Parameter('thres', default=4.5), Parameter('sample', default=False), Parameter('dlambda', default=1.2)]
To set the value of a recipe parameter, the value can be assigned to the according attribute:
>>> muse_scibasic.param.nifu = 1
The new value is checked against parameter type, and possible value limitations provided by the recipe. Hyphens in parameter names are converted to underscores. In a recipe call, the same parameter can be specified as
dict
:>>> res = muse_scibasic( ..., param = {'nifu':1})
To reset a value to its default, it is either deleted, or set to
None
. The following two lines:>>> muse_scibasic.param.nifu = None >>> del muse_scibasic.param.nifu
will both reset the parameter to its default value.
All parameters can be set in one step by assigning a
dict
to the parameters. In this case, all values that are not in the map are reset to default, and unknown parameter names are ignored. The keys of the map may contain contain the name or the fullname with context:>>> muse_scibasic.param = { 'nifu':1, 'xbox':11, 'resample':True }
See also
3.5. Recipe frames¶
There are three groups of frames: calibration (“calib”) frames, input (“raw”)
frames, and result (“product”) frames. Calibration frames may be set either
via the Recipe.calib
attribute or as named keywords on the run
execution. A value set in the recipe call will overwrite any value that was
set previously in the Recipe.calib
attribute for that specific
call. Input frames are always set in the recipe call. If their tag name was
not given, the tag name from Recipe.tag
is used if the recipe provides
it.
-
Recipe.
calib
¶ This attribute contains the calibration frames for the recipe. It is iterable and then returns all calibration frames:
>>> for f in muse_scibasic.calib: ... print f.tag, f.min, f.max, f.frames TRACE_TABLE 1 1 None WAVECAL_TABLE 1 1 None MASTER_BIAS 1 1 master_bias_0.fits MASTER_DARK None 1 None GEOMETRY_TABLE 1 1 None BADPIX_TABLE None None ['badpix_1.fits', 'badpix_2.fits'] MASTER_FLAT None 1 None
Note
Only MUSE recipes are able to provide the full list of calibration frames and the minimal/maximal number of calibration frames. For other recipes, only frames that were set by the users are returned here. Their minimum and maximum value will be set to
None
.In order to assing a FITS file to a tag, the file name or the
astropy.io.fits.HDUList
is assigned to the calibration attribute:>>> muse_scibasic.calib.MASTER_BIAS = 'MASTER_BIAS_0.fits'
Using
astropy.io.fits.HDUList
is useful when it needs to be patched before fed into the recipe.>>> master_bias = astropy.io.fits.open('MASTER_BIAS_0.fits') >>> master_bias[0].header['HIERARCH ESO DET CHIP1 OUT1 GAIN'] = 2.5 >>> muse_scibasic.calib.MASTER_BIAS = master_bias
Note that
astropy.io.fits.HDUList
objects are stored in temporary files before the recipe is called which may produce some overhead. Also, the CPL then assigns the random temporary file names to the FITS keywordsHIERARCH ESO PRO RECm RAWn NAME
which should be corrected afterwards if needed.To assign more than one frame, put them into a list:
>>> muse_scibasic.calib.BADPIX_TABLE = [ 'badpix1.fits', 'badpix2.fits' ]
All calibration frames can be set in one step by assigning a
dict
to the parameters. In this case, frame that are not in the map are set are removed from the list, and unknown frame tags are silently ignored. The key of the map is the tag name; the values are either a string, or a list of strings, containing the file name(s) or theastropy.io.fits.HDUList
objects.>>> muse_scibasic.calib = { 'MASTER_BIAS':'master_bias_0.fits', ... 'BADPIX_TABLE':[ 'badpix_1.fits', 'badpix_2.fits' ] }
In a recipe call, the calibration frame lists may be overwritten by specifying them in a
dict
:>>> res = muse_scibasic( ..., calib = {'MASTER_BIAS':'master_bias_1.fits'})
See also
3.6. Runtime environment¶
For debugging purposes, the runtime environment of the recipe may be
changed. The change may be either done by specifying the Recipe.env
attribute of as a parameter on the recipe invocation. The change will have no
influence on the environment of the framework itself.
Note
Some variables are only read on startup
(like MALLOC_CHECK_
), changing or deleting them will have
no effect.
-
Recipe.
env
= None¶ Environment changes for the recipe. This is a
dict
with the name of the environment variable as the key and the content as the value. It is possible to overwrite a specific environment variable. SpecifyingNone
as value will remove the variable:>>> muse_flat.env['MUSE_RESAMPLE_LAMBDA_LOG'] = '1' >>> muse_flat.env['MUSE_TIMA_FILENAME'] = 'tima.fits'
In a recipe call, the runtime environment may be overwritten as well:
>>> res = muse_flat( ..., env = {'MUSE_PLOT_TRACE':'true'})
3.7. Recipe invocation¶
-
Recipe.
__call__
(*data, **ndata)¶ Call the recipes execution with a certain input frame.
Parameters: - raw (
astropy.io.fits.HDUlist
orstr
or alist
of them, ordict
) – Data input frames. - tag (
str
) – Overwrite thetag
attribute (optional). - threaded (
bool
) – overwrite thethreaded
attribute (optional). - loglevel (
int
) – set the log level for pythonlogging
(optional). - logname (
str
) – set the log name for the pythonlogging.Logger
(optional, default is ‘cpl.’ + recipename). - output_dir (
str
) – Set or overwrite theoutput_dir
attribute. (optional) - param (
dict
) – overwrite the CPL parameters of the recipe specified as keys with their dictionary values (optional). - calib (
dict
) – Overwrite the calibration frame lists for the tags specified as keys with their dictionary values (optional). - env (
dict
) – overwrite environment variables for the recipe call (optional).
Returns: The object with the return frames as
astropy.io.fits.HDUList
objectsReturn type: Raise: exceptions.ValueError
If the invocation parameters are incorrect.Raise: exceptions.IOError
If the temporary directory could not be built, the recipe could not start or the files could not be read/written.Raise: cpl.CplError
If the recipe returns an error.Raise: cpl.RecipeCrash
If the CPL recipe crashes with a SIGSEV or a SIGBUSNote
If the recipe is executed in the background (
threaded = True
) and an exception occurs, this exception is raised whenever result fields are accessed.- raw (
See also