nunet_test_process_and_environments

Last updated: 2024-09-17 21:09:16.700906 File source: link on GitLab

NuNet is running an extensive testing program and framework in order to reduce possibility of errors on its production network, where the NTX-backed global economy of decentralized computing will be operating. This page explains the NuNet test environments, architecture of network and the process of constructing, maintaining and growing the network -- integrated into the NuNet overall development process and CI/CD pipeline.

Related documentation:

• CI/CD Pipeline

• Team Process

• GIT Workflows and Branching Strategy

Contents of this page

[TOC]

Summary

Similarly to other decentralized computing projects (as blockchains), the network is running on the hardware provisioned via independent devices. In NuNet case, there is an additional complexity due to the fact that test networks have to resemble heterogeneity of the population of devices, operating systems and setups. Therefore, large portion of the NuNet CI/CD pipeline have to run not on centralized servers (e.g. in our case, via gitlab-ci runners), but on the geographically dispersed network. In order to manage the full life-cycle of the platform, including testing of separate features and iterations of the network components, NuNet is using isolated channels categorized into three environments:

• feature environment runs the ci/cd pipeline on the main branch of the NuNet repositories;

• staging environment runs extensive pre-release testing on the frozen features in the release branch;

• production environment runs the final releases of NuNet network, exposed to end users.

Environments

Feature environment

Purpose and audience

The feature environment is composed of a network of heterogeneous devices sourced from the community. Since NuNet, as a decentralized network, will not have control of the devices sourced from community, the feature environment will encompass communication channels with the community members who will participate in NuNet testers programs.

CI/CD stages

Branch: main branch

Feature environment is used to run the following CI/CD pipeline stages according to the pre-defined schedule, to be communicated to community testers:

• static analysis

• unit tests

• static security tests

• build

• functional tests / API tests

• security tests

• automatic regression tests

Architecture

The feature environment contains:

• virtual machines and containers hosted in NuNet cloud servers;

• machines owned by NuNet team members;

• machines provided by the community members on constant basis via NuNet Network private testers programs.

More details about the architecture supporting the current implementation of the feature environment can be found here.

Triggering schedule

The CI/CD pipeline in feature environment is triggered in two cases:

1. automatically when a merge request is merged into the main branch;

2. according to the pre-determined schedule for running stages that are more heavy on compute requirements -- which ideally may include the more advanced stages; depending on the speed of development, NuNet may be schedule weekly or nightly builds and runs of the platform with the full pipeline (possibly including the latest stages of the CI/CD pipeline normally reserved for staging environment only). In principle, feature environment should be able to run all automatic tests.

Staging environment

Purpose and public

Testnet is this network and is used by developers, QA/security engineers and community testers. Manged by the Product Owner.

CI/CD stages

Branch: release branch, created from main by freezing features scheduled for release;

CI/CD pipeline runs the following stages automatically as well as manually where required:

• static analysis

• unit tests

• static security tests

• build

• functional tests / API tests

• security tests

• regression tests

• performance and load tests

• live security tests

Architecture

The staging environment contains:

• virtual machines and containers hosted in NuNet cloud servers

• machines owned by NuNet team members

• extensive network of community testers' machines/devices provided via NuNet Network private testers, covering all possible configurations of the network and most closely resembling the actual NuNet network in production:

  • with different hardware devices

  • owned by separate individuals and entities

  • connected to internet by different means:

    • having IP addresses

    • behind different NAT types

    • having different internet speeds

    • having different stability of connection

    • etc

Triggering schedule

Testing on staging environment is triggered manually as per platform life-cycle and release schedule. When the staging branch is created from main branch with the frozen features ready for release, the following actions are performed:

1. The staging environment / testnet is constructed by inviting community testers to join their machines in order to cover architecture described above;

2. All applications are deployed on the network (as needed) in preparation for automatic and manual regression testing and load testing;

3. Manual testing schedule is released and communicated to community testers;

4. CI/CD pipeline is triggered with all automatic tests and manual tests;

5. Bug reports are collected and resolved;

6. Manual tests resulting in bugs are automated and included into CI/CD pipeline;

The above cycle is repeated until no bugs are observed. When this happens, the staging branch is marked for release into production environment.

Production environment

Purpose and public

This is the live environment used by the community to onboard machines/devices or to use the computational resources available on the NuNet platform.

CI/CD pipeline

No CI/CD pipeline stages are running on production environment. However, all users are provided with tools and are encouraged to report any bugs or file feature requests following the Contributing guidelines.

Architecture

The Production environment contains all community machines/devices connected to production network.

Production Release

When the tests in the Testnet (staging environment) are finished with success and approved by the testers, the module(s)/API(s) should be released to production. The following processes are being defined:

• versioning process: versioning of modules and APIs;

• compatibility/deprecation process: releasing modules/APIs that do not have compatibility with others modules/APIs currently running on the platform should be avoided since NuNet is a highly decentralized network; however old versions should be deprecated so maintaining the compatibility will not create other problems related to security, performance, code readability, etc.

• communication process: how the community is notified of modules updates, bugs, security issues

• updating process: how the modules/APIs are updated.