Aviary’s AI Policy#
Use of generative artificial intelligence (LLMs) to produce code for Aviary is allowed, but contributions must follow the following guidelines. If you wish to use AI to assist in contributing code to Aviary, we recommend sticking to limited, targeted uses to speed up development time for small-scale tasks, or to limit AI use to a purely advisory role and not for writing code. Extensive use of AI-generated code leads to large, fragmented, and difficult to maintain codebases.
However, this advice is based on experience using current-generation AI models, and as these tool’s capabilities improve we may revise our recommendations.
1. Disclose any use of AI#
Any use of AI tools should be clearly disclosed. It is helpful to separate AI-generated code into their own commits whenever possible. Mention that AI was used in the commit messages. Give additional details in your pull request descriptions about how AI was used. Include a brief explanation of what you used the AI for (debugging, generating new code, etc.), which model you used, and which files were affected.
2. Thoroughly test all AI-generated code#
Like all code contributions, it is important to run all unittests and benchmark tests on as many different OS and environment configurations as you reasonably can - this goes double for any changes that used AI assistance. If any code generated by AI is not explicitly tested by a unittest, you should write a new test to cover the AI generated code. If AI modifies existing code, especially if new functionality is added, existing unittests should be extended to test that new functionality.
3. AI should never be used as a source for data#
Generative AI, as the name implies, is not a tool that is good at “fetching” information - it generates a best guess for what you asked for. Therefore, it is extremely unreliable as a source of “truth” data. AI should not be used as the source of numerical data, such as default values, constants/conversion factors, aircraft performance data, or similar.
AI should also not be used to write unittests. While writing unittests can be a tedious step that may be tempting to automate, they are extremely critical to get right. The test setup and which values are compared against must be carefully chosen to ensure the code is properly being verified (and potentially validated) including potential edge cases.
4. Do not use AI for documentation#
AI tools should not be used for documentation. Current AI tools struggle with holding large codebases like Aviary in context, and hallucinations or general mistakes are harder to catch in text then they are in code - even with Aviary’s documentation checking utilities (DocTAPE). Any AI written documentation pages will be rejected.
5. Use common sense when using AI tools#
In general, you should never just accept and copy/paste code an AI tool gives you. All AI-generated code should be reviewed and understood before you commit it.
The key takeaway is that you should always fully understand your code before submitting it to Aviary, no matter how it was created.