Regular code analysis forms an essential part of any software quality assurance program. LArSoft is introducing a regime of code analyses and fostering a shift in culture that seeks third-party reviews. The ultimate goal is to form a strong foundation for building increasingly sophisticated algorithms via compliance with LArSoft and art design principles. Good software engineering and low-level coding practices should be followed across the entire LArSoft suite.
The involvement and cooperation of the author (or the active maintainer of the code in question) with the process is critical to the success of analysis.
When having an analysis meeting, a clear charge and a well-defined scope are developed.
Profiling results collected prior to code analysis meetings are performed by the code authors with the assistance of a single expert from the analysis team.
The authors provide an overview of the code either prior to or at the beginning of the code analysis meetings.
The authors and reviewers actively analyze the same code elements as a group. It is important to have a balance between having enough people to stimulate discussion and ideas, but not too many so as to make decision making overly difficult.
One aim is to examine code as it comes into the repository. Code compliance and C best practice code analysis can be done on immature code with considerably less effort than that required for a full code analysis, and can head off some major problems before they become embedded. Code analysis in other less formal settings such as software meetings or collaboration meetings will also help to foster a culture of code analysis.
Since experts will often find issues that more inexperienced users will not immediately see when using profiling tools, using such tools during a code analysis meeting can aid both the analysis of the code and provide learning for code authors.
The preparatory work requires a significant time investment from each of the team members as does follow-up consulting or documentation. This is taken into account and charged to the code analysis when planning and scheduling meetings.
The following types of code analysis meetings differ in the target, and therefore in the preparatory work needed, the tools used, and the level of reading of the code that might be expected:
- In general, such reviews are left to the experiments, and are not considered in this document
Although a single “code analysis” may include more than one of the above, each additional type may require added preparatory work, experts and time during the meeting.
A light-weight process focused on a single type at a time - the specific type agreed upon in advance.
Code analysis meetings are initiated on a request from any of the following:
The type of code analysis and the scope of code involved are decided at the time of the request in consultation with the requestor, the author, and the core LArSoft team. Whenever possible, the specific charge, code analysis objectives and any metrics of success are defined and agreed upon at this time.
Several considerations can drive a code analysis:
Each code analysis meeting includes at a minimum the following people:
Additional people can be added as deemed necessary or helpful. In general, the process should be completely open, so observers are welcome.
Each meeting needs people in the following roles to be present:
The audience for the results can include several entities, depending upon the targets identified:
Most code analysis, however, will require most or all of the following in advance of any formal meetings:
- An initial examination of the output prior to the meetings can identify and possibly address easy to fix issues early in the process.
- The results can also be used to help understand the control flow of the code.
- Typically tests for bitwise identical output are the most useful, since physics validation and changes that result in changes to the physics are outside the scope of these code analysis meetings.
- To the extent feasible, each expert should preview the code on his or her own.
Meetings will broadly perform or cover the following items.
- Any major concept and abstractions used by the code should be included in this.
- The intent of this presentation is to discuss what the code does, so need not go into details such as the class structure, for instance.
Previous experience suggests that sessions of about half a day in duration are the most effective. Multiple such sessions can be organized as needed. Care should be exercised in scheduling and throughout the planning, however, so as to keep the entire process as light-weight as possible. Single session meetings, for instance, should possibly be the norm. In all cases, the time to be committed should be agreed upon in advance.
Meetings should allow brief discussion regarding topics of opportunity noted by the participants during the course of the meeting (for example, noting a use of call-by-value function arguments when const references would be substantially more efficient).
The direct product of the preparatory work and meetings should include the following:
- Recommended changes;
- Problems or other issues that require further investigation, thought or discussion;
- Conclusions regarding any targets of opportunity discussed;
- Any findings that are not reflected among the items above, including discussion of potential trade-offs in any areas addressed by the code analysis;
- The metrics of success for the code analysis, when they exist;
- Any comments on the process offered.
- A reference to comments in GitHub can be included in any of the relevant sections.
A strong follow-up effort from a code analysis meeting is as critical to the success of the overall code analysis as are the steps leading to the report itself.
We assume that the primary responsibility for implementing changes falls to the author or experiment that owns the code. We further assume that this work will be carried out in consultation with the experts participating in the code analysis, the core LArSoft team, and the offline management of the relevant experiment.
The details of the follow-up work, as for the code analysis meetings, should be tailored to the particular code analysis meeting, the types of recommendations, and the effort available. The following tasks will ensure that the follow-up work is properly specified and tracked, and that any more generally useful conclusions will be disseminated.
- All such lessons learned should be accumulated in a single location.
- A brief report of such lessons learned may be appropriate in cases when they are or broad interest.
The above process is based on the recommendations at https://cd-docdb.fnal.gov/cgi-bin/ShowDocument?docid=5765 which modified the original process discussed at
Original proposal for code analysis is as given in slides at the steering meeting.