Understanding the Variable Metadata

Understanding the Variable Metadata#

How Variable Metadata Works#

Every variable in an Aviary variable hierarchy must have metadata associated with it. This metadata is used for setting initial values, setting Aviary inputs and outputs, and various other functionalities throughout the code. It is also helpful information for the user to have regarding each variable and the metadata dictionary allows all that information to live in one organized location. Unlike variable hierarchies, which are broken up into different categories based on the type of information they contain, the variable metadata all lives in the same dictionary, regardless of which variable hierarchy its variables come from.

The variable metadata dictionary is exactly what it sounds like: a Python dictionary, or more explicitly a Python dictionary of dictionaries. The entire metadata is one dictionary, and within that metadata dictionary each variable (the key) has its own sub-dictionary including all the information relevant to that variable. The information included in each sub-dictionary is:

Key Name in Metadata	Default Value	Information
key	`None`	Name
units	`unitless`	Units
default_value	`0.0`	Default Value
types	`None`	Type Restrictions
multivalue	`False`	Can variable be vectorized?
option	`False`	Is Option?
desc	`None`	Description
historical_name	`None`	Historical Variable Name(s)

Many of these variables are self-explanatory, but many require additional discussion.

key is the variable name, which must be a string compatible with OpenMDAO’s variable name rules. Units must be a string, also compatible with OpenMDAO’s list of supported units.

default_value is what Aviary will use if the variable is not provided by the user as an input or does not come from another part of the problem (computed by another component, provided by Dymos as a state/control/timeseries , etc.)

types is a Python type or tuple of types that this variable is allowed to be. If not provided, types defaults to the type of default_value (which in turn is defaulted to a float).

multivalue is a boolean flag that tells Aviary if this variable can be a iterable (typically a list, numpy array, or tuple). If this flag is True, those iterable types are also allowable types in addition to whatever is listed in types. When doing type checks, Aviary will check the value of each index in a provided iterable against types. So if your variable is expected to be a list of floats, then types should be set to float, and multivalue should be True. If you provide an iterable type in types while multivalue is True, then you are telling Aviary that you can have a multidimensional array (e.g. a list that contains lists). Expected array size is not set in metadata, but instead in OpenMDAO system definition, when inputs/outputs/options are added to the system.

If multivalue is False and an iterable type is given in types, then Aviary will not know how to enforce type checks for values inside the iterable! Any iterable that matches types will pass, regardless of what it contains - this is technically fine, but opens you up to an accidental TypeError later down the line. We don’t reccomend setting up variables like this. In general, only add iterable types to types if you are working with multidimensional arrays.

option is a boolean flag if your variable is used as an OpenMDAO option, rather than a component input or output. Set this flag to True to ensure your variable correctly gets connected to any components that ask for it through options.

desc is a string that should describe what the variable represents, how it is used, and any other information that would be helpful for an aircraft designer setting a value for that variable in their input file.

historical_name is a dictionary that connects this variable with any potential matching variables in legacy codes Aviary inherrited from. Variable names under the keys FLOPS or GASP will be used by the fortran_to_aviary input file conversion utility to attempt to match legacy input files with Aviary variables.

The information in the metadata dictionary is accessed just like information in any other Python dictionary. For example, if you wanted to know the units of the Aircraft.Wing.SPAN variable from the Aviary-core Aircraft variable hierarchy along with whether or not the variable was an option, you would access those units using the following code:

import aviary.api as av

AviaryAircraft = av.Aircraft

wingspan_units = av.CoreMetaData[AviaryAircraft.Wing.SPAN]['units']
wingspan_is_option = av.CoreMetaData[AviaryAircraft.Wing.SPAN]['option']

print(wingspan_units)
print(wingspan_is_option)

In this example we use the variable hierarchy to provide the name of the variable we are seeking to Aviary’s CoreMetaData, and we use the keys from the metadata dictionary to provide the specific information that we would like to know. This would return

ft
False

which tells you that the units of the variable Aircraft.Wing.SPAN from the Aviary-core Aircraft variable hierarchy are feet, and that Aircraft.Wing.SPAN is not an option.

Note

Many of the weight and aerodynamic estimating relationships in Aviary originated from historical codes called GASP and FLOPS. For engineers who are familiar with GASP and FLOPS it is helpful to know what an Aviary variable was called in those historical codes.

The historical variable name portion of the metadata allows us to associate any names that an Aviary variable may have had in a previous code. This piece of the metadata is actually a dictionary within each subdictionary belonging to each variable. This dictionary is used by adding an entry for each historical code, where the key is the name of the historical code, and the value for that key is a string or list of strings illustrating the name(s) that variable held in the historic code. This is an optional feature, but can be helpful for users who are porting old codes into new formats. If a tilde (~) is attached to a historical variable, it is a local variable or parameter in GASP or FLOPS. More details about the naming convention is described in utils/develop_metadata.py.

The Aviary-core Metadata#

The Aviary code provides metadata for every variable in the Aviary-core variable hierarchies. As noted above, the metadata is not broken up into multiple dictionaries like the variable hierarchy, but instead the metadata for every variable lives in the same dictionary. As such there is only one Aviary-core metadata dictionary, which can be viewed here and accessed in the following way:

import aviary.api as av

MetaData = av.CoreMetaData

In the Aviary-core metadata dictionary, due to the size of the data we have adopted a structure of organizing the metadata alphabetically by variable hierarchy. Thus, while all the variables are part of the same metadata dictionary, you will notice that the organization structure of the file is the same alphabetical hierarchal organization structure as the Aviary-core variable hierarchy. This is a convention that we encourage to improve the cleanliness of code, but it is not strictly required.

Building Your Own Metadata#

Unlike the variable hierarchies, which are separated out into different hierarchies for different types of data, there is only one metadata dictionary for all variables in every variable hierarchy. Technically the user may build a metadata dictionary from scratch instead of extending the Aviary-core metadata, however, there is no real value to this as you will eventually have to merge back in the Aviary-core metadata anyway, so there are no normal circumstances under which this is the recommended practice. However, just like with variable hierarchies, you can have several different metadata dictionaries which will eventually be merged together. This may be necessary when there are multiple people developing different external subsystems in different locations.

There are two different ways to change the metadata in a metadata dictionary. The first is to add a new variable to the dictionary, and add that variable’s metadata along with it. This makes use of the add_meta_data() function. This function takes in the variable name of the variable to be added to the metadata dictionary be provided, as well as the dictionary itself that the variable should be added to. It also optionally takes in all of the metadata information listed at the beginning of this page. The function returns nothing, but it internally updates the provided metadata dictionary so that dictionary will contain the new variable and its metadata.

The second way to change the metadata in a metadata dictionary is by updating the metadata associated with a variable that is already in the dictionary. This is accomplished using the update_meta_data() function. This function behaves almost identically to the add_meta_data() function, the only difference being that instead of adding a new variable to the dictionary, it will take the input of metadata information that you provide and overwrite the old metadata of the given variable with the new metadata.

There are two pitfalls that may occur when using these functions. The first pitfall is attempting to call the add_meta_data() function for a variable that already exists in the metadata. This will throw an error, because the add_meta_data() function is only for new variables to the metadata. Conversely, attempting to update the metadata of a variable that is not in the metadata dictionary via update_meta_data() will throw an error because that function is only for variables that already exist in the metadata.

The methods outlined above for updating and adding to the variable metadata are the crux of how the variable metadata can be extended for new variables. The user will simply import the existing Aviary-core metadata and add to it as they see fit.

Note

The variable metadata dictionary that is imported from the Aviary API is actually a copy of the original Aviary metadata dictionary to avoid mutating the original dictionary. That being said, it functions just as a metadata dictionary that you would input to an Aviary model and you can extend it or input it to a model as-is depending on your needs.

Lets examine how we would extend the variable metadata dictionary in practice. Say we have just extended the Aviary-core Aircraft variable hierarchy to add some center of gravity, flap, and jury strut information using the extension below:

import aviary.api as av

AviaryAircraft = av.Aircraft

class ExtendedAircraft(AviaryAircraft):

    CG = 'aircraft:center_of_gravity'

    class Wing(AviaryAircraft.Wing):
        class Flap:
            AREA = 'aircraft:wing:flap:area'
            ROOT_CHORD = 'aircraft:wing:flap:root_chord'
            SPAN = 'aircraft:wing:flap:span'

    class Jury:
        MASS = 'aircraft:jury:mass'

Now we want to extend the Aviary-core metadata into our own metadata that includes metadata for each one of these variables in the same code:

ExtendedMetaData = av.CoreMetaData

av.add_meta_data(
    ExtendedAircraft.CG,
    meta_data=av.CoreMetaData,
    units='ft',
    desc='Center of gravity',
    default_value=0,
    option=False,
)

av.add_meta_data(
    ExtendedAircraft.Wing.Flap.AREA,
    meta_data=ExtendedMetaData,
    units='ft**2',
    desc='planform area of flap',
    default_value=10,
    option=False
)

av.add_meta_data(
    ExtendedAircraft.Wing.Flap.ROOT_CHORD,
    meta_data=ExtendedMetaData,
    units='ft',
    desc='chord of flap at root of wing',
    default_value=1,
    option=False
)

av.add_meta_data(
    ExtendedAircraft.Wing.Flap.SPAN,
    meta_data=ExtendedMetaData,
    units='ft',
    desc='span of flap',
    default_value=60,
    option=False
)

av.add_meta_data(
    ExtendedAircraft.Jury.MASS,
    meta_data=ExtendedMetaData,
    units='kg',
    desc='mass of jury strut',
    default_value=50,
    option=False
)

ExtendedMetaData now contains the metadata of all the Aviary-core variables along with the metadata information that we just added.

Merging Independent Metadata#

Extending the metadata is great, but sometimes users will end up with multiple metadata dictionaries because different subsystem developers extended the metadata (and created associated variable hierarchies) to suit their own needs. Aviary needs to be given one single metadata dictionary which contains metadata of all the variables it has been given, so we need to be able to merge together multiple metadata dictionaries into one. The merge_meta_data() function has been provided to combine all the different metadata into one. The merge_meta_data() function behaves quite similarly to the merge_hierarchies() function. It takes in a string of metadata dictionaries that need to be merged together, and it returns a single metadata dictionary containing the metadata from all the individual dictionaries.

Let’s say that we have created our ExtendedAircraft and ExtendedMetaData from above, and that elsewhere we have a subsystem that requires information about engine cooling system mass as well as whether the aircraft has winglets. Below is the buildup of the Aircraft type hierarchy and the metadata for our new subsystem:

import aviary.api as av

class ExtendedAircraft2(av.Aircraft):

    class Engine(av.Aircraft.Engine):

        class Cooling:
            MASS = 'aircraft:engine:cooling:mass'

    class Wing(av.Aircraft.Wing):
        WINGLETS = 'aircraft:wing:winglets'

ExtendedMetaData2 = av.CoreMetaData

av.add_meta_data(
    ExtendedAircraft2.Engine.Cooling.MASS,
    units='kg',
    desc='mass of cooling system for one engine',
    default_value=100,
    meta_data=ExtendedMetaData2,
    historical_name=None,
)

av.add_meta_data(
    ExtendedAircraft2.Wing.WINGLETS,
    units=None,
    desc='Tells whether the aircraft has winglets',
    default_value=True,
    option=True,
    types=bool,
    meta_data=ExtendedMetaData2,
    historical_name=None,
)

# Testing Cell
glue_variable(get_variable_name(ExtendedAircraft2), md_code=True)
glue_variable(get_variable_name(ExtendedMetaData2), md_code=True)

We can see from the above code that we have an Aircraft type variable hierarchy named ExtendedAircraft2 and that we have created our own metadata dictionary ExtendedMetaData2 which is an extension of the CoreMetaData dictionary in Aviary-core. Now we have two different Aircraft type variable hierarchy extensions, ExtendedAircraft and ExtendedAircraft2. We also have two different metadata extensions, ExtendedMetaData and ExtendedMetaData2. We need a single Aircraft type variable hierarchy, and single metadata dictionary. Thus, we will use the merging functions built into Aviary:

FinalAircraft = av.merge_hierarchies([ExtendedAircraft, ExtendedAircraft2])
FinalMetaData = av.merge_meta_data([ExtendedMetaData, ExtendedMetaData2])

Above we merged together our hierarchy and metadata extensions, and now we have one single Aircraft type hierarchy FinalAircraft and one single metadata dictionary FinalMetaData which we can provide to the Aviary model.

There is one situation when an attempt to merge together multiple metadata dictionaries will cause errors, and that situation is if more than one metadata dictionary contains the same variable with different metadata. If multiple dictionaries contain the same variable with identical metadata the merge will proceed, but if the metadata differs at all the merge will halt and force the user to rectify the discrepancy.

More syntactical data on the merging functions can be found here.

Providing Aviary with Necessary Variable Metadata#

This section is under development.

Searchable Metadata Table#

The table below contains all the metadata for every variable in the Aviary-core variable hierarchies. The table is searchable and sortable and is created automatically from the Aviary core metadata dictionary.

init_notebook_modetrusted

variable name	units	desc	option	default_value	types	multivalue	historical_name
Loading ITables v2.3.0 from the `init_notebook_mode` cell... (need help?)