nunet_test_process_and_environments
Last updated: 2024-09-27 10:48:02.688551 File source: link on GitLab
NuNet is running an extensive testing program and framework in order to reduce possibility of errors on its production network, where the NTX-backed global economy of decentralized computing will be operating. This page explains the NuNet test environments, architecture of network and the process of constructing, maintaining and growing the network -- integrated into the NuNet overall development process and CI/CD pipeline.
Related documentation:
Contents of this page
Summary
Similarly to other decentralized computing projects (as blockchains), the network is running on the hardware provisioned via independent devices. In NuNet case, there is an additional complexity due to the fact that test networks have to resemble heterogeneity of the population of devices, operating systems and setups. Therefore, large portion of the NuNet CI/CD pipeline have to run not on centralized servers (e.g. in our case, via gitlab-ci runners), but on the geographically dispersed network. In order to manage the full life-cycle of the platform, including testing of separate features and iterations of the network components, NuNet is using isolated channels categorized into three environments:
feature environment runs the ci/cd pipeline on the
main
branch of the NuNet repositories;staging environment runs extensive pre-release testing on the frozen features in the
release
branch;production environment runs the final releases of NuNet network, exposed to end users.
Environments
Feature environment
Purpose and audience
The feature environment is composed of a network of heterogeneous devices sourced from the community. Since NuNet, as a decentralized network, will not have control of the devices sourced from community, the feature environment will encompass communication channels with the community members who will participate in NuNet testers programs.
CI/CD stages
Branch: main
branch
Feature environment is used to run the following CI/CD pipeline stages according to the pre-defined schedule, to be communicated to community testers:
static analysis
unit tests
static security tests
build
functional tests / API tests
security tests
automatic regression tests
Architecture
The feature environment contains:
virtual machines and containers hosted in NuNet cloud servers;
machines owned by NuNet team members;
machines provided by the community members on constant basis via NuNet Network private testers programs.
More details about the architecture supporting the current implementation of the feature environment can be found here.
Triggering schedule
The CI/CD pipeline in feature environment is triggered in two cases:
automatically when a merge request is merged into the
main
branch;according to the pre-determined schedule for running stages that are more heavy on compute requirements -- which ideally may include the more advanced stages; depending on the speed of development, NuNet may be schedule weekly or nightly builds and runs of the platform with the full pipeline (possibly including the latest stages of the CI/CD pipeline normally reserved for staging environment only). In principle, feature environment should be able to run all automatic tests.
Staging environment
Purpose and public
Testnet is this network and is used by developers, QA/security engineers and community testers. Manged by the Product Owner.
CI/CD stages
Branch: release
branch, created from main
by freezing features scheduled for release;
CI/CD pipeline runs the following stages automatically as well as manually where required:
static analysis
unit tests
static security tests
build
functional tests / API tests
security tests
regression tests
performance and load tests
live security tests
Architecture
The staging environment contains:
virtual machines and containers hosted in NuNet cloud servers
machines owned by NuNet team members
extensive network of community testers' machines/devices provided via NuNet Network private testers, covering all possible configurations of the network and most closely resembling the actual NuNet network in production:
with different hardware devices
owned by separate individuals and entities
connected to internet by different means:
having IP addresses
behind different NAT types
having different internet speeds
having different stability of connection
etc
Triggering schedule
Testing on staging environment is triggered manually as per platform life-cycle and release schedule. When the staging branch is created from main
branch with the frozen features ready for release, the following actions are performed:
The staging environment / testnet is constructed by inviting community testers to join their machines in order to cover architecture described above;
All applications are deployed on the network (as needed) in preparation for automatic and manual regression testing and load testing;
Manual testing schedule is released and communicated to community testers;
CI/CD pipeline is triggered with all automatic tests and manual tests;
Bug reports are collected and resolved;
Manual tests resulting in bugs are automated and included into CI/CD pipeline;
The above cycle is repeated until no bugs are observed. When this happens, the staging branch is marked for release into production environment.
Production environment
Purpose and public
This is the live environment used by the community to onboard machines/devices or to use the computational resources available on the NuNet platform.
CI/CD pipeline
No CI/CD pipeline stages are running on production environment. However, all users are provided with tools and are encouraged to report any bugs or file feature requests following the Contributing guidelines.
Architecture
The Production environment contains all community machines/devices connected to production network.
Production Release
When the tests in the Testnet (staging environment) are finished with success and approved by the testers, the module(s)/API(s) should be released to production. The following processes are being defined:
versioning process: versioning of modules and APIs;
compatibility/deprecation process: releasing modules/APIs that do not have compatibility with others modules/APIs currently running on the platform should be avoided since NuNet is a highly decentralized network; however old versions should be deprecated so maintaining the compatibility will not create other problems related to security, performance, code readability, etc.
communication process: how the community is notified of modules updates, bugs, security issues
updating process: how the modules/APIs are updated.
Last updated