DocTAPE Examples

DocTAPE Examples#

DocTAPE (Documentation Testing and Automated Placement of Expressions) is a collection of utility functions (and wrappers for Glue) that are useful for automating the process of building and testing documentation to ensure that documentation doesn’t get stale.

Custom Classes#

expected_error#

Functions that raise an error provide the option to specify an error type to use instead of the default. This allows users to change the error type that is raised which can be useful in try/except blocks, especially when combined with the expected_error class.

from aviary.utils.doctape import expected_error, check_value
try:
    check_value(int('1'), 2, error_type=expected_error)
except expected_error:
    print('we expected that to fail (1 is not equal to 2),')
print('but this will still run')

we expected that to fail (1 is not equal to 2),
but this will still run

If we just used ValueError in the except branch, we might miss errors that we actually do want to catch.

from aviary.utils.doctape import expected_error, check_value

try:
    check_value(int('1)'), 2)
except ValueError:
    print('1 is not equal to 2')
print("we mistyped '1', so we should have failed")

try:
    check_value(int('1)'), 2, error_type=expected_error)
except expected_error:
    print('1 is not equal to 2')
print("something unnexpected happened (we mistyped '1'), and we won't reach this")

1 is not equal to 2
we mistyped '1', so we should have failed

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[3], line 10
      7 print("we mistyped '1', so we should have failed")
      9 try:
---> 10     check_value(int('1)'), 2, error_type=expected_error)
     11 except expected_error:
     12     print('1 is not equal to 2')

ValueError: invalid literal for int() with base 10: '1)'

Testing Functions#

The testing functions provide code that will raise errors when the documentation is built if the results don’t match what is expected. These can be used in places where it would be too difficult to glue portions of the documentation, or it is preferable to have a more uninterupted flow in the markdown cells.

However, it is important to note that it is possible to notice an error when the documentation builds and fix the code in the testing cell without updating the text in the markdown cell. For this reason, it is recommended to use a combination of testing and glueing functions in documentation.

check_value#

is a simple function for comparing two values.

from aviary.utils.doctape import check_value
from aviary.examples.reserve_missions.run_reserve_mission_fixedrange import phase_info

user_opts = phase_info['reserve_cruise']['user_options']
check_value(user_opts['target_distance'],(200, 'km'))
check_value(user_opts['reserve'],True)

/usr/share/miniconda/envs/test/lib/python3.12/site-packages/pyoptsparse/pyOpt_MPI.py:68: UserWarning: mpi4py could not be imported. mpi4py is required to use the parallel gradient analysis and parallel objective analysis for non-gradient based optimizers. Continuing using a dummy MPI module from pyOptSparse.
  warnings.warn(warn)

check_contains#

confirms that all the elements of one iterable are contained in the other

from aviary.utils.doctape import check_contains
import aviary.api as av
import os

off_design_examples = av.get_path(os.path.join('examples'))
check_contains(
    ('run_off_design_example.py'),
    os.listdir(off_design_examples),
    error_string="{var} not in "+str(off_design_examples),
    error_type=FileNotFoundError)
print('This file exists and does not raise any errors')
check_contains(
    ('made_up_file.py'),
    os.listdir(off_design_examples),
    error_string="{var} not in "+str(off_design_examples),
    error_type=FileNotFoundError)
print('This file does not exist, so we will not reach this point')

This file exists and does not raise any errors

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
Cell In[5], line 12
      6 check_contains(
      7     ('run_off_design_example.py'),
      8     os.listdir(off_design_examples),
      9     error_string="{var} not in "+str(off_design_examples),
     10     error_type=FileNotFoundError)
     11 print('This file exists and does not raise any errors')
---> 12 check_contains(
     13     ('made_up_file.py'),
     14     os.listdir(off_design_examples),
     15     error_string="{var} not in "+str(off_design_examples),
     16     error_type=FileNotFoundError)
     17 print('This file does not exist, so we will not reach this point')

File ~/work/Aviary/Aviary/aviary/utils/doctape.py:174, in check_contains(expected_values, actual_values, error_string, error_type)
    172 for var in expected_values:
    173     if var not in actual_values:
--> 174         raise error_type(error_string.format(var=var, actual_values=actual_values))

FileNotFoundError: made_up_file.py not in /home/runner/work/Aviary/Aviary/aviary/examples

Here we are checking that a certain file exists in a folder and specify a more useful error type than the default RuntimeError

check_args#

gets the signature of a function and compares it to the arguments you are expecting.

# Testing Cell
from aviary.utils.doctape import check_args, check_contains, glue_variable

default_error = RuntimeError
check_args(check_contains, {'error_type':default_error}, exact=False)
glue_variable('default_error', default_error.__name__)

exact_arg = 'exact'
check_args(check_args, exact_arg)
glue_variable(exact_arg, md_code=True)

Setting the exact argument to False means that we don’t need to exactly match the signature of the function and instead just want to make sure that all of the arguments are valid and possibly that their default values are correct.

run_command_no_file_error#

executes a CLI command but won’t fail if a FileNotFoundError is raised.

# Testing Cell
from aviary.utils.doctape import run_command_no_file_error
command = """
    aviary run_mission --optimizer IPOPT --phase_info outputted_phase_info.py 
    validation_cases/benchmark_tests/test_aircraft/aircraft_for_bench_FwFm.csv
    --max_iter 0
"""
run_command_no_file_error(command)

This allows the command syntax and setup to be tested without requiring all of the files that command will use.

Glue Functions#

The glue functions provide a wrapper for the myst_nb glue function that simplifies the interface.

After a variable has been glued in a Python cell, it can be accessed from a markdown cell with the {glue:md}`variable name` notation. Note that glue won’t access the value of the glued variable until the documentation is built.

glue_variable#

allows users to specify a value that is not the same as what is displayed, but defaults to using the name of the variable if nothing is specified. This makes adapting old documentation easier, because users can just wrap the entire phrase they want to replace.

Glued text can either be plain text or can be formatted as inline code

# Testing Cell
from aviary.utils.doctape import glue_variable

glue_variable('plain text')
glue_variable('inline code', md_code=True)
glue_variable('something different than','not the same as')
glue_variable('the entire phrase they want to replace')

glue_keys#

combines get_all_keys and glue_variable to glue all of the unique keys from a dict of dicts for later use.

# Testing Cell
from aviary.utils.doctape import glue_keys

simplified_dict = {
    'phase1':{'altitude':{'val':30,'units':'kft'},'mach':.4},
    'phase2':{'altitude':{'val':10,'units':'km'},'mach':.5}
    }
glue_keys(simplified_dict)

This allows us to ensure that altitude and mach do exist in the dictionary.

Utility Functions#

Utility functions are provided that the user may find useful for generating or testing their documentation.

gramatical_list#

is a simple function that forms a string that can be used in a sentence using a list of items.

from aviary.utils.doctape import gramatical_list

single_element = gramatical_list([1])
two_elements = gramatical_list(['apples','bananas'])
three_elements_with_or = gramatical_list(['apples','bananas', 'strawberries'],'or')

print(f"I would like to order {single_element} smoothie.")
print(f"Do you want {three_elements_with_or} in your smoothie?")
print(f"I only want {two_elements}.")

I would like to order 1 smoothie.
Do you want apples, bananas, or strawberries in your smoothie?
I only want apples and bananas.

get_variable_name#

is a function that just returns the name of the variable passed to it as a string.

The contents of the variable can be of any type, as the variable isn’t used in the function, but rather the inspect functionality is used to retrieve the line of code itself.

get_variable_name can even accept multiple arguments, in which case a list of the names will be returned.

from aviary.utils.doctape import get_variable_name, glue_variable
from aviary.api import AviaryProblem

glue_variable('function_name', get_variable_name(get_variable_name))
glue_variable(get_variable_name(print))

some_string = 'that contains important information'
simple_variable_name = get_variable_name(some_string)
phrase = simple_variable_name + ' is a variable ' + some_string
print(phrase)

complex_object_name = get_variable_name(AviaryProblem)
print(complex_object_name)

multiple = 2
arguments = str
print(get_variable_name(multiple, arguments))
Hide code cell output

get_variable_name

print

some_string is a variable that contains important information
AviaryProblem
['multiple', 'arguments']

get_variable_name can be called directly in functions like print or glue_variable or the results can be saved.

get_previous_line#

returns the previous line of code as a string, which allows users to grab individual lines of code from Python cells to use as inline code in markdown cells.

# Testing Cell
from aviary.api import Mission
from aviary.utils.doctape import glue_variable, get_previous_line, get_variable_name

glue_variable('value', Mission.Design.MACH, md_code=True)
glue_variable('var_value_code', get_previous_line(), md_code=True)
glue_variable(get_variable_name(Mission.Design.MACH), md_code=True)
glue_variable('var_name_code', get_previous_line(), md_code=True)

If you want to glue the name of a variable, instead of the value that variable holds, you can use the get_variable_name to extract it.

For example: Using glue_variable('value', Mission.Design.MACH, md_code=True) will result in mission:design:mach, whereas using glue_variable(get_variable_name(Mission.Design.MACH), md_code=True) will result in Mission.Design.MACH

get_attribute_name#

allows users to get the name of object attributes in order to glue them into documentation. This works well for Enums or Class Variables that have unique values.

from aviary.api import LegacyCode
from aviary.utils.doctape import get_attribute_name, glue_variable
import aviary.api as av

some_custom_alias = av.LegacyCode

gasp_name = get_attribute_name(some_custom_alias, LegacyCode.GASP)
glue_variable(gasp_name)
brief_name = get_attribute_name(av.Verbosity, 1)
glue_variable(brief_name)
verbosity = get_attribute_name(av.Settings, av.Settings.VERBOSITY)
glue_variable(verbosity)
glue_variable(av.Settings.VERBOSITY)
Hide code cell output

GASP

use_args

VERBOSITY

settings:verbosity

get_all_keys and get_value#

are intended to be used together for getting keys from nested dictionaries and then getting values back from those nested dictionaries, respectively. They were originally added for complex dictionaries, like the phase_info.

from aviary.utils.doctape import get_all_keys, get_value

simplified_dict = {
    'phase1':{'altitude':{'val':30,'units':'kft'},'mach':.4},
    'phase2':{'altitude':{'val':10,'units':'km'},'mach':.5}
    }
unique_keys_only = get_all_keys(simplified_dict)
all_keys = get_all_keys(simplified_dict, track_layers=True)
print(unique_keys_only)
print(all_keys)

p1_alt = get_value(simplified_dict, 'phase1.altitude.val')
print(p1_alt)

['phase1', 'altitude', 'val', 'units', 'mach', 'phase2']
['phase1', 'phase1.altitude', 'phase1.altitude.val', 'phase1.altitude.units', 'phase1.mach', 'phase2', 'phase2.altitude', 'phase2.altitude.val', 'phase2.altitude.units', 'phase2.mach']
30