specification_and_documentation
Last updated: 2024-09-27 10:48:03.193024 File source: link on GitLab
Specification and Documentation
This document explains the current framework or structure for specification of platform components. It also outlines the process through which updates to platform specification and corresponding documentation are managed.
Table of Contents
Specification Framework
The specification for each component / package / sub-package is described in the README
file situated in the same folder.
The README
file consists of following elements:
Static section: This section contains the package name and some links to information about the project. Note that each
README
file has the same set of links in this section. The links are followed by a Table of Contents which again is common across allREADMEs
.Description: This section should have a brief description of what the package is about and its core functionality.
Structure and organisation: Here we give an high level overview of the contents of the package. This includes any file or folder created within the directory.
Class Diagram: Each package/sub-package is represented by a class diagram created in
plantuml
. This file namedclass_diagram.puml
is present in thespecs
folder present in each directory. Note that the class diagram needs to be detailed at the sub-pacakge level. The package level diagram automatically gets created by integrating the diagrams of each sub-package into a single file. Similarly, the global class diagram at the component level gets created by integrating diagrams of all packages into a single file.Functionality: This section typically explains the interfaces and methods that define the functionality of the package. Developers can choose to link documentation auto generated from the code as long as it clearly explains the package functionality. Alternatively, they can also follow the structures/templates prescribed in Interfaces & Methods section of this document. It is recommended to specify any additional information (as applicable) to enhance the clarity and understanding of the reader.
Data Types: This sections enlists the various data models used by the package. As a default
Go
structs have been used to describe the data types. However, developer may choose to use an equivalent structure as per the language (ex. Python) used in the component/package. The conventions to be followed for specifying data types are further explained in Data Models section of this document.Testing: This section is to be used to explain the reader how to test the functionality. This may cover unit tests, functional tests or anything else as required.
Proposed Functionality / Requirements: This section allows developers to capture the functionality that is not yet built but is in the pipeline. This requirement could be coming from other packages needing a functionality or it could be features that from the roadmap of the said package. This section essentially serves two roles:
a. Give an idea to the reader what modifications are expected to the current package. Refer to the list of issues that are referenced in this section to access work being done/planned for the package.
b. This is the place where request for new functionality can be specified. The process for doing the same is outlined in this section.
Note: All future functionality should have a
proposed
tag in the heading to make it clear to the user. See below for an illustration.References: Any additional links or content relevant for the reader can be mentioned here. For example, Nunet research blog is referenced in several places to give an idea of the background research prior to development of the functionality for those who are interested.
Interfaces & Methods
Interfaces can be written directly in the README
file using the code block, which mostly applies to proposed interfaces. It is best to explain the purpose of the interface and its methods in plain English.
Template / Structure for method description
A recommended (but not mandatory) structure for describing methods is as follows:
signature:
<function_signature>
input #1:
<explanation of first input parameter>
input #2:
<explanation of second input parameter>
output (success):
<Expected output data type>
output (error):
<Output in case of any error>
<Function_name>
function <function_description>
See below for an illustration
Note: It is recommended to specify only the main methods that describe the core functionality. Helper functions need not be described in the README
file.
Naming Convention
It is recommended to use camelCase
for function names with first word in lower case and following words having the first letter capitalized. For example - sendMessage
or publishBidRequest
.
Data Models
All data structures required or used in the functionality should be specified. See below for an example data model.
Naming Conventions
The data models are specified using a standard code block. Following naming convention need to be utilised.
For example dms.orchestrator.BidRequest
implies that a data type by the name BidRequest
is defined in the orchestrator
sub-pacakge of dms
package. It is important to adhere to this convention for discoverability and readability of the specifications.
Another applicable convention is to capitalize the first letter of the word in the name of data models. For example - BidRequest
or PriceBid
.
Note: Only the data models defined in the current package need to be specified using the code block. For data models from other packages, only the name should be mentioned along with a brief explanation of what role it is playing in the package functionality.
For example, see below a illustrative screenshot from executor
package.
Note that the data types defined in types
package only explained here in context of the package functionality. It does not specify its parameters which have alredy been defined in the README
of the types
package. However, LogStreamRequest
data type is fully described as it is being defined in the executor
package itself.
API End Points
Below is the recommended structure for describing API endpoints. However, developers can modify this or use alternate tools like Swagger if that is more beneficial.
<Explanation of the endpoint functionality>
See below for an illustrative example of onboard
end point
Sequence Diagrams
Sequence Diagrams can be a useful tool to describe a functionality in the initial or design phase. Developers are recommended to make use of this wherever they see fit.
We suggest to use PlantUML for creating sequence diagrams. Few reasons for this choice is because
PlantUML files support the naming convention we follow. See naming conventions for data types
It is very easy to insert a
.puml
file in the README. In fact class diagrams of all the packages in DMS (Device Management Service) are made using PlantUML.PlantUML allows for both constructing the whole diagram as well as parts of it as well. This allows us to divide specs to each package and store close to the code
Alternatively one can also use Mermaid.
See below for an illustrative example of search and match
operation where DMS tries to find elibile compute providers from the network.
The important aspects that typically should be covered in the sequence diagram are:
It should show the entities involved in the functionality. Ex, User, Compute Provider DMS, Elasticsearch etc
Define routines and subroutines. In the above example, the loop
Compute provider decision to accept/reject
covers the functionality where Compute Provider is assessing the job opportunity. Within this loop we have two subroutines covering the two possible scenarios - decision to bid or not bid.Add description, comments etc. It is particularly useful to make the sequences more clear to the reader.
Gherkin Feature File
Gherkin feature files are another useful tool that can be used by developers to specify a functionality. Gherkin syntax allows us to describe the functionality of a component using natural language.
This means we develop a granular list of steps that should be executed for each functionality that is being offered by the component or package. This also covers different scenarios of interaction which can lead to different outcomes within the same functionality.
The steps written in Gherkin are saved in a file with an extension .feature
. An example of such a feature file with a single scenario is shown below.
The important aspects that should be covered in the feature file are:
The applicable function and data models should be referenced.
Define endpoints if applicable.
Explain the precondition that should exist using the
Given
keyword.Define the different scenarios than can occur within this functionality
Note the naming convention which specifies the package in which the said function/data model will be located -
createBid
function inorchestrator
package of DMS component.
Update Procedure
All documentation at Nunet should be considered as Living Documentation
which essentially means that it gets updated along with code and evolution of the project.
The below steps describe the typical process that is followed for making any change or update to the platform functionality. Which means, that contributions to documentation are considered contributions to the code base and should follow the NuNet contributing guidelines (see document).
Steps to suggest or propose a change
Open the list of issues link in the
Proposed Functionality / Requirements
section in theREADME
file.Check if a similar issue has already been created. If the answer is
Yes
, add your comments on the issue to facilitate discussion. Use sequence diagrams, Gherkin files as required. Skip step 3. If the answer isNo
continue to step 3.Create a new issue explaining the proposed functionality. Use sequence diagrams, Gherkin files as required.
Tag developers who are maintaining the code/package on the issue / comment.
Create a new feature branch from
main
branch. If a feature alredy exisits with related updates, it may be useful to clone your feature from there.Update the
Proposed Functionality / Requirements
section in theREADME
with proposed interfaces, methods, data types etc.Created a Merge Request (MR) for review. Assign maintainers of the code as reviewers.
Link the MR created in the previous step on the issue created earlier.
The initial review Process
The assignees will either review the merge request themselves or assign a responsible core team member (who is knowledgeable about the functionality).
The issue and linked MR will be reviewed by the assigned person. If it is found acceptable, we move to the discussion stage. If the proposal is simple, the reviewers can accept or reject it immediately.
In-depth Discussion Process
The issue will be discussed further as needed with core team members. The discussion has to be coordinated by the assigned reviewer.
The free discussion can happen on the issue as comments broken into topics if needed.
The discussion is closed when the reviewer accepts the proposed changes to specifications and the
README
file on the feature branch adequately described the proposed functionality in the relevant section.
Upon acceptance, the feature branch will be merged to the
main
branches. Note that here onlyREADME
file has been updated.New Issue will be created for development of the proposed changes and placed into
Open
and described as needed in order to place intokb::backlog
, considering the flow of team work at the moment.The specification of the feature will be placed in the description of the issue so that developers and contributors can easily pick it up and implement.
The specifications will serve as acceptance criteria for the issue to be accepted for merge into
main
branch (after implementation is done).
Implementation
Create a feature branch from the
main
branch.Implement the changes on the feature branch to incorporate the proposed functionality.
Update the
README
file so that the specification matches the state of implementation.Create a merge request to the
main
branch.Link the MR on the relevant issue.
Implementation Review and Acceptance
The linked MR will be reviewed by the core team.
The reviewer will check if the implemented functionality meets the specification and all specified tests pass as defined in Test Management process and makes the decision to merge / close it.
Upon acceptance, the feature branch will be merged to the
main
branch.The issue will be marked as
kb::done
.
Last updated