Last updated: 2024-11-07 21:04:46.195281 File source: link on GitLab

Introduction

The Nunet Command-Line Interface (CLI) serves as a powerful tool for interacting with the Nunet ecosystem, enabling you to manage network configurations, control capabilities, and handle cryptographic keys. It provides a comprehensive set of commands to streamline various tasks and operations within the Nunet network.

Top-Level Commands

Actor System

nunet actor

This command provides a suite of operations tailored for interacting with the Nunet Actor System. It enables you to communicate with actors within the network, facilitating actions like sending messages, invoking specific behaviors, and broadcasting information to multiple actors simultaneously.

Detailed documentation can be found here.

Capability Management

nunet cap

This command focuses on capability management within the Nunet ecosystem. It allows you to define, delegate, and control the permissions and authorizations granted to different entities, ensuring secure and controlled interactions within the network.

Detailed documentation can be found here.

Configuration Management

nunet config

The config command allows to interact with and manage your configuration file directly from the command line. This allows you to view existing settings, modify them as needed, and ensure Nunet DMS is tailored to your preferences.

Usage

nunet config COMMAND

Available Commands

• edit: Opens the configuration file in your default text editor for manual adjustments.

• get: Retrieve and display the current value associated with a specific configuration key.

• set: Modify the configuration file by assigning a new value to a specified key.

Flags

• -h, --help: Display help information for the config command and its subcommands.

Key Management

nunet key

The Nunet Key Management CLI allows generating new keypairs and retrieve the Decentralized Identifier (DID) associated with a specific key.

Main Command

• nunet key

  • Description: The primary command to manage keys within the Nunet DMS

  • Usage: nunet key COMMAND

  • Available Commands:

    • did: Retrieve the DID for a specified key

    • new: Generate a new keypair

  • Flags:

    • -h, --help: Display help information for the main key command.

Subcommands

• nunet key did

  • Description: Retrieves and displays the DID associated with a specified key. This DID uniquely identifies the key within the Nunet network.

  • Usage: nunet key did <key-name> [flags]

  • Arguments:

    • <key-name>: The name of the key for which the DID is to be retrieved

  • Flags:

    • -h, --help: Display help information for the did command

• nunet key new

  • Description: Generates a new keypair and securely stores the private key in the user's local keystore. The corresponding public key can be used for various cryptographic operations within the Nunet DMS

  • Usage: nunet key new <name> [flags]

  • Arguments:

    • <name>: A name to identify the newly generated keypair

  • Flags:

    • -h, --help: Display help information for the new command.

Important Considerations:

• Keep your private keys secure, as they provide access to your identity and associated capabilities within the Nunet DMS

• Choose descriptive names for your keypairs to easily identify their purpose or associated devices.

Run Command

nunet run

Starts the Nunet Device Management Service (DMS) process, responsible for handling network operations and device management.

Usage:

nunet run [flags]

Flags:

• -c, --context string: Specifies the key and capability context to use (default: "dms").

• -h, --help: Displays help information for the run command.

Example:

nunet run

This starts the Nunet DMS with the default "dms" context.

TAP Command

nunet tap

Purpose: Creates a TAP (network tap) interface to bridge the host network with a virtual machine (VM). It also configures network settings like IP forwarding and iptables rules.

Key Points:

• Root Privileges Required: This command necessitates root or administrator privileges for execution due to its manipulation of network interfaces and system-level settings.

Usage:

nunet tap [main_interface] [vm_interface] [CIDR] [flags]

Arguments:

• main_interface: (e.g., eth0) The name of the existing network interface on your host machine that you want to bridge with the TAP interface.

• vm_interface: (e.g., tap0) The name you want to assign to the newly created TAP interface.

• CIDR: (e.g., 172.16.0.1/24) The Classless Inter-Domain Routing (CIDR) notation specifying the IP address range and subnet mask for the TAP interface. This ensures that the VM or container connected to the TAP has its own IP address within the specified network.

Flags:

• -h, --help: Displays help information for the tap command.

Example:

sudo nunet tap eth0 tap0 172.16.0.1/24

This command will create a TAP interface named 'tap0' bridged to your host's 'eth0' interface. The 'tap0' interface will be assigned an IP address of '172.16.0.1' with a subnet mask of '/24'. This configuration allows a VM connected to 'tap0' to access the network through your host's 'eth0' interface.

Important Notes:

• Ensure you have the necessary permissions to execute this command.

• Be cautious when configuring network settings, as incorrect configurations can disrupt your network connectivity.

GPU Command

nunet gpu

Purpose: The nunet gpu command provides gpu related apis.

Usage:

nunet gpu <operation>

Available Operations:

• list: List all the available GPUs on the system.

• test: Test the GPU deployment on the system using docker.

Flags:

• -h, --help: Display help information for the gpu command and its subcommands.

Example:

nunet gpu list

This command will list all the available GPUs on the system.

nunet gpu test

This command will test the GPU deployment on the system using docker.

This documentation portal is your gateway to understanding and leveraging NuNet’s unique platform capabilities. Here, you’ll find detailed guides, technical specifications, and best practices to help you get started, dive deep into our architecture, and contribute to our vibrant community.

Device Management Service (DMS)

The Device Management Service (DMS) is a critical component of the NuNet platform, orchestrating the management of all devices within our decentralized network. Designed for flexibility, security, and performance, the DMS handles everything from decentralized identity management and private network creation to interoperability with mainstream frameworks like Kubernetes. Whether you are managing a single node or a network of devices, the DMS ensures seamless and secure operations, optimizing resource allocation and performance across the board.

NuNet Test Suite

Quality and reliability are foundational to our platform, and the NuNet Test Suite is our comprehensive framework to ensure both. It automates and streamlines testing processes across all NuNet components, from unit tests to end-to-end scenarios, guaranteeing that every update meets our high standards for security, functionality, and performance. Integrated with our CI/CD pipelines, the test suite supports continuous testing and validation, helping to deliver a robust and secure platform ready for real-world applications.

Team Processes and Guidelines

At NuNet, we foster a culture of innovation and collaboration driven by transparent and efficient processes. Our Team Processes and Guidelines provide a framework for how we work together, from agile development practices and clear communication protocols to rigorous security standards and inclusive contribution guidelines. These processes ensure that all team members and contributors are aligned, empowered, and working towards a shared vision of advancing decentralized computing.

As you explore the documentation, you’ll find comprehensive guides and detailed technical specifications covering all aspects of the NuNet platform, from the Device Management Service (DMS) to our testing framework and team processes. Whether you're a developer, researcher, or enthusiast, we encourage you to dive in, contribute, and help shape the future of decentralized computing with us.

Last updated: 2024-11-07 21:04:45.427177 File source: link on GitLab

Device Management Service (DMS)

• Project README

• Release/Build Status

• Changelog

• License

• Contribution Guidelines

• Code of Conduct

• Secure Coding Guidelines

Table of Contents

• Device Management Service (DMS)

  • Table of Contents

  • About

    • Payment

  • Installation

    • Binary releases

      • Ubuntu/Debian

    • Building from source

      • Dependencies

    • Installation on VMs

    • Installation on WSL

    • Permissions and features (for compute providers)

      • Net-admin permission and IP over libp2p

      • Firecracker (for compute providers)

    • System Requirements

      • CPU-only machines

        • Minimum System Requirements

        • Recommended System Requirements

      • GPU Machines

        • Minimum System Requirements

        • Recommended System Requirements

    • GPU Driver Installation

  • Usage

    • Quick Start

      • Creating identities

      • Using a Ledger Wallet

      • Setting up Capabilities

        • Create capability contexts

          • Add a root anchor for your DMS context

        • Setup your DMS for the public testnet

      • Running DMS

    • Onboarding

    • REST Endpoints

  • Configuration

    • Config file

      • Run Two DMS Instances Side by Side

  • Tests

  • Specification

    • Description

    • Design and Architecture

      • Conceptual Basis

      • Ontology

      • Architecture

      • Research

    • Functionality

    • Data Types

    • References

    • Class Diagram

      • Source File

About

Device Management Service or DMS enables a machine to join the decentralized NuNet network both as a compute provider, offering its resources to the network, or to leverage the compute power of other machines in the network for processing tasks. Eventually users with available hardware resources will get compensated whenever their machine is utilized for a computational job by other users in the network. The ultimate goal of the platorm is to create a decentralized compute economy that is able to sustain itself.

Payment

All transactions on the Nunet network are expected to be conducted using the platform's utility token NTX. However, DMS is currently in development, and payment isn't part of v0.5.0-boot release. NTX payments are expected to be implemented in the Public Alpha Mainnet milestone within later release cycles.

Note: If you are a developer, please check out the DMS specifications and Building from Source sections of this document.

Installation

You can install Device Management Service (DMS) via binary releases or building it from source.

Binary releases

You can find all binary releases here and other builds in-between releases in the package registry. We currently support ARM and AMD64 architectures. You may check your architecture with appropriate command (uname -p for linux) and refer to the architecture name mapping e.g. here for figuring correct package to download.

Note: If you intalled the binary from a release and you would like to act as compute provider, you may need to check permissions and features to enable some required and optional features.

Ubuntu/Debian

1. Download the latest .deb package from the package registry

2. Install the Debian package with apt or dpkg:

sudo apt update
sudo apt install ./nunet-dms_<latest>.deb -y

1. Some dependencies such as docker and libsystemd-dev might be missing so it's recommended to fix install by running:

sudo apt -f install

Building from source

We currently support Linux and MacOS (Darwin).

Dependencies

• iproute2 (linux only)

• build-essential (linux only)

• libsystemd-dev (linux only)

• go (v1.21.7 or later)

Clone the repository:

git clone https://gitlab.com/nunet/device-management-service.git

Build the CLI:

cd device-management-service
make

This will result in a binary file in builds/ folder named as dms_linux_amd64 or dms_darwin_arm64 depending on the platform.

Note: If you built from source and would like to act as a compute provider, you may need to check permissions and features to enable some required and optional features.

To cross compile to arm, cross compilers need to be installed. In particular arm-linux-gnueabihf and aarch64-linux-gnu. For debian systems, install with:

apt install gcc-arm-linux-gnueabihf gcc-aarch64-linux-gnu

You can add the compiled binary to a directory in your $PATH. See the Usage section for more information.

Permissions and features (for compute providers)

The following applies only for compute providers. If you're running a client/orchestrator, you do not need to set any permissions.

If you built DMS from source or installed the binary from one of our releases, you may need to set some permissions to the binary to enable some features.

Darwin users: unfortunately, the DMS can't work with granular permissions on Mac. So, for now, if running a compute provider, you will have to run the nunet daemon (nunet run) as root.

For Linux users, granular permissions will have to be set to the binary (optionally, but NOT recommended, you can run the binary as root).

Net-admin permission and IP over libp2p

Note: cap_net_admin is a required capability for compute providers.

Setting the permission enables IP over libp2p which is a feature that enhances the capabilities of compute providers, allowing them to participate in a wider range of jobs. One capability enabled with this feature is to do port forwarding which it won't be possible without setting the right unix permissions.

One of the reasons for requiring this permission is because this feature depends on creating and managing tun interfaces.

To enable this feature, the nunet binary requires network-admin capabilities. These capabilities allow the application to perform network configuration tasks without needing to run the entire application as root, which is a more secure approach.

To set the necessary capabilities, run the following command:

sudo setcap cap_net_admin+ep /usr/bin/nunet

The above command depends on: libcap2-bin (Debian/Ubuntu) or libcap (CentOS/RHEL/Arch...)

Firecracker (for compute providers)

Note: Linux only.

To act as a compute provider capable of receiving and running jobs with Firecracker, ensure your user is part of the kvm group. You can do this by running:

sudo usermod -aG kvm $USER

Installation on VMs

• Skip doing an unattended installation for the new Ubuntu VM as it might not add the user with administrative privileges.

• Enable Guest Additions when installing the VM (VirtualBox only).

• Always change the default NAT network setting to Bridged before booting the VM.

• Install Extension Pack if using VirtualBox (recommended).

• Install VMware Tools if using VMware (recommended).

• ML on GPU jobs on VMs are not supported.

Installation on WSL

• Install WSL through the Windows Store.

• Install the Update KB5020030 (Windows 10 only).

• Install Ubuntu 20.04 through WSL.

• Enable systemd on Ubuntu WSL.

• ML Jobs deployed on Linux cannot be resumed on WSL.

Though it is possible to run ML jobs on Windows machines with WSL, using Ubuntu 20.04 natively is highly recommended to avoid unpredictability and performance losses.

If you are using a dual-boot machine, make sure to use the wsl --shutdown command before shutting down Windows and running Linux for ML jobs. Also, ensure your Windows machine is not in a hibernated state when you reboot into Linux.

System Requirements

CPU-only machines

Minimum System Requirements

We require you to specify CPU (MHz x no. of cores) and RAM, but your system must meet at least the following requirements before you decide to onboard it:

• CPU: 2 GHz

• RAM: 4 GB

• Free Disk Space: 10 GB

• Internet Download/Upload Speed: 4 Mbps / 0.5 MBps

If the above CPU has 4 cores, your available CPU would be around 8000 MHz. So if you want to onboard half your CPU and RAM on NuNet, you can specify 4000 MHz CPU and 2000 MB RAM.

Recommended System Requirements

• CPU: 3.5 GHz

• RAM: 8-16 GB

• Free Disk Space: 20 GB

• Internet Download/Upload Speed: 10 Mbps / 1.25 MBps

GPU Machines

Minimum System Requirements

• CPU: 3 GHz

• RAM: 8 GB

• GPU: 4 GB VRAM (NVIDIA, AMD, or Intel discrete GPU with manually installed drivers)

• Free Disk Space: 50 GB

• Internet Download/Upload Speed: 50 Mbps

Note: For AMD64 platforms, we recommend using HiveOS as it comes with all necessary drivers pre-installed. For other setups, proper GPU drivers must be manually installed. See the GPU Driver Installation section for instructions.

Recommended System Requirements

• CPU: 4 GHz

• RAM: 16-32 GB

• GPU: 8-12 GB VRAM (NVIDIA, AMD, or Intel discrete GPU with manually installed drivers)

• Free Disk Space: 100 GB

• Internet Download/Upload Speed: 100 Mbps

GPU Driver Installation

NuNet DMS requires properly installed GPU drivers to function correctly. We do not automatically install drivers to ensure compatibility and flexibility across different user setups.

For AMD64 Platforms:

We recommend using the Ubuntu-based HiveOS for the easiest setup.

If you prefer to use a different operating system or need to install drivers manually, please follow these steps:

NVIDIA GPUs:

1. Visit the NVIDIA Official Driver Downloads page.

2. Select your GPU model and operating system.

3. Download and install the recommended driver.

4. Reboot your system after installation.

AMD GPUs:

1. Visit the AMD Drivers and Support for Processors and Graphics page.

2. Select your GPU model and operating system.

3. Download and install the recommended driver.

4. Reboot your system after installation.

Intel Discrete GPUs:

1. Visit the Intel® software for general purpose GPU capabilities documentation page.

2. Select your GPU model and operating system.

3. Download and install the recommended driver.

4. Reboot your system after installation.

For detailed instructions specific to your operating system, please refer to the documentation provided by NVIDIA, AMD, or Intel.

Note: Ensure that you have the correct permissions to install drivers on your system. On Linux systems, you may need to use sudo or log in as root to install drivers.

Usage

Quick Start

Before starting, ensure that you have properly installed GPU drivers if you're using a GPU-enabled machine. For AMD64 platforms, we recommend using HiveOS for the easiest setup. For other configurations, refer to the GPU Driver Installation section for instructions.

This quick start guide will walk you through the process of setting up a Device Management Service (DMS) instance for the first time and getting it running. We'll cover creating identities, setting up capabilities, and running the DMS.

The NuNet CLI

The NuNet CLI is the command-line interface for interacting with the Nunet Device Management Service (DMS). It provides commands for managing keys, capabilities, configuration, running the DMS, and more. It's essential for setting up and administering your DMS instance.

Key Concepts

• Actor: An independent entity in the Nunet system capable of performing actions and communicating with other actors.

• Capability: Defines the permissions and restrictions granted to actors within the system.

• Key: A cryptographic key pair used for authentication and authorization within the DMS.

You can find a detailed documentation here.

Creating identities

The first step is to generate identities/keys and capability contexts. It is recommended that two keys are setup: one for the user (default name user) and another for the dms (default name dms)

A capability context is created with the nunet cap new <context> command and it is anchored on a key with the context name.

To set up a new identity/create a new key, run the command:

$ nunet key new <identity>

Then, to initialize its capability context:

$ nunet cap new <identity>

In this example, we are going to set up two identities:

First for the user

$ nunet key new user

then for the dms instance.

$ nunet key new dms

We then setup the capability contexts for each identity:

First the user

$ nunet cap new user

then the dms instance.

$ nunet cap new dms

If you use a ledger wallet for your personal key, you can create the user context as follows:

Create a new key for the user

$ nunet key did ledger

Then create the capability context for the user

$ nunet cap new ledger:user

You can create as many identities as you want, specially if you want to manage multiple DMS instances.

The key new command returns a DID key for the specified identity. Remember to secure your keys and capability contexts, as they control access to your NuNet resources. They are encrypted and stored under $HOME/.nunet by default.

Each time a new identity is generated it will prompt the user for a passphrase. The passphrase is associated with the created identity, thus a different passphrase can be set up for each identity. If you prefer, it's possible to set a DMS_PASSPHRASE environment variable to avoid the command prompt.

Using a Ledger Wallet

It is also possible to use a Ledger Wallet instead of creating a new key; this is recommended for user contexts, but you should not use it for the dms context as it needs the key to sign capability tokens.

To set up a user context with a Ledger Wallet, you need the ledger-cli script from NuNet's ledger wallet tool. The tool uses the Eth application and specifically the first Eth account with signing of personal messages. Everything that needs to be signed (namely capability tokens) will be presented on your Nano's screen in plaintext so that you can inspect it.

You can get your Ledger wallet's DID with:

$ nunet key did ledger

Setting up Capabilities

NuNet's network communication is powered by the NuActor System, a zero-trust system that utilizes fine-grained capabilities, anchored on DIDs, following the UCAN model.

Once both identities are created, you'll need to set up capabilities. Specifically:

1. Create capability contexts for both the user and each of your DMS instances.

2. Add the user's DID as a root anchor for the DMS capability context. This ensures that the DMS instance fully trusts the user, granting complete control over the DMS (the root capability).

3. If you want your DMS to participate in the public NuNet testnet (and eventually the mainnet), you'll need to set up capability anchors for your DMS:

   1. Create a capability anchor to allow your DMS to accept public behavior invocations from authorized users and DMSs in the NuNet ecosystem.

   2. Add this token to your DMS as a require anchor.

   3. Request a capability token from NuNet to invoke public behaviors on the network.

   4. Add the token as a provide anchor in your personal capability context.

   5. Delegate to your DMS the ability to make public invocations using your token.

   6. Add the delegation token as a provide anchor in your DMS.

Add a root anchor for your DMS context

You can do this by invoking the dms cap anchor command:

$ nunet cap anchor --context dms --root <user-did>

Where <user-did> is the user did created above in Creating identities and can be obtained by:

$ nunet key did <user>

or if you are using a Ledger Wallet

$ nunet key did ledger

Setup your DMS for the public testnet

1. The NuNet DID

did:key:zzCHUybNYmK8QsttZwXqUX8aDLoBGHnMCakDX2RpsGwmXmYHEW

1. Create a capability anchor for public behaviors

Create the grant

$ nunet cap grant --context user --cap /public --cap /broadcast --topic /nunet --expiry 2024-12-31 <nunet-did>

or if you are using a Ledger Wallet

$ nunet cap grant --context ledger:user --cap /public --cap /broadcast --topic /nunet --expiry 2024-12-31 <nunet-did>

And the granted token as a require anchor

$ nunet cap anchor --context dms --require <the-grant-output>

The first command grants nunet authorized users the capability to invoke public behaviors until December 31, 2024, and outputs a token.

The second command consumes the token and adds the require anchor for your DMS

1. Ask NuNet for a public network capability token

To request tokens for participating in the testnet, please go to did.nunet.io and submit the did you generated along with your gitlab username and an email address to receive the token. It's highly recommended that you use a Ledger hardware wallet for your keys.

1. Use the NuNet granted token to authorize public behavior invocations in the public network

3.1 Add the provide anchor to your personal context

$ nunet cap anchor --context user --provide <the-token-you-got-from-nunet>

or if you are using a Ledger Wallet

$ nunet cap anchor --context ledger:user --provide <the-token-you-got-from-nunet>

3.2 Delegate to your DMS

$ nunet cap delegate --context user --cap /public --cap /broadcast --topic /nunet --expiry 2024-12-31 <your-dms-did>

or if you are using a Ledger Wallet

$ nunet cap delegate --context ledger:user --cap /public --cap /broadcast --topic /nunet --expiry 2024-12-31 <your-dms-did>

3.3 Add the delegation token as a provide anchor in your DMS

$ nunet cap anchor --context dms --provide <the-delegate-output>

The first command ingests the NuNet provided token and the last two commands use this token to delegate the public behavior capabilities to your DMS.

Running DMS

If everything was setup properly, you should be able to run:

$ nunet run

Darwin users: If you plan to onboard your computer power to the network, You may need to run with sudo. See the optional features and permissions section for more information.

By default, DMS runs on port 9999.

Onboarding

You don't necessarily need to onboard for development, but that depends on which part you're working on. To onboard during development, /etc/nunet needs to be manually created since it is created with the package during installation.

Refer to dms/onboarding package README for details of onboarding functionality for compute provider users.

REST Endpoints

Refer to the api package README for the list of all endpoints. Head over to project's issue section and create an issue with your question.

Configuration

Config file

The DMS searches for a configuration file dms_config.json in the following locations, in order of priority whenever it's started:

1. The current directory (.)

2. $HOME/.nunet

3. /etc/nunet

The configuration file must be in JSON format and it does not support comments. It's recommended that only the parameters that need to be changed are included in the config file so that other parameters can retain their default values.

It's possible to manage configuration using the config subcommand as well. nunet config set allows setting each parameter individually and nunet config edit will open the config file in the default editor from $EDITOR

Run Two DMS Instances Side by Side

As a developer, you might find yourself needing to run two DMS instances, one acting as an SP (Service Provider) and the other as a CP (Compute Provider).

Step 1:

Clone the repository to two different directories. You might want to use descriptive directory names to avoid confusion.

Step 2:

You need to modify some configurations so that both DMS instances do not end up trying to listen on the same port and use the same path for storage. For example, ports on p2p.listen_address, rest.port, general.user_dir etc... neeed to be different for two instances on the same host.

The dms_config.json file can be used to modify these settings. Here is a sample config file that can be modified to your preference:

{
  "p2p": {
    "listen_address": ["/ip4/0.0.0.0/tcp/9100", "/ip4/0.0.0.0/udp/9100/quic-v1"]
  },
  "general": {
    "user_dir": "/home/user/.config/nunet/dms/",
    "debug": true
  },
  "rest": {
    "port": 10000
  }
}

Prefer to use absolute paths and have a look at the config structure for more info.

Tests

Some packages contain tests, and it is always best to run them to ensure there are no broken tests before submitting any changes. Before running the tests, the Firecracker executor requires some test data, such as a kernel file, which can be downloaded with:

make testdata

After the download is complete, all unit tests can be run with the following command. It's necessary to include the unit tag due to the existence of files that contain functional and integration tests.

go test --tags unit ./...

Help in contributing tests is always appreciated :)

Specification

Description

NuNet is a computing platform that provides globally distributed and optimized computing power and storage for decentralized networks, by connecting data owners and computing resources with computational processes in demand of these resources. NuNet provides a layer of intelligent interoperability between computational processes and physical computing infrastructures in an ecosystem which intelligently harnesses latent computing resources of the community into the global network of computations.

Detailed information about the NuNet platform, concepts, architecture, models, stakeholders can be found in these two papers:

• White Paper

• Yellow Paper

DMS (Device Management Service) acts as the foundation of the NuNet platform, orchestrating the complex interactions between various computing resources and users. DMS implementation is structured into packages, creating a more maintainable, scalable, and robust codebase that is easier to understand, test, and collaborate on. Here are the existing packages in DMS and their purposes:

• dms: Responsible for starting the whole application and core DMS functionality such as onboarding, job orchestration, job and resource management, etc.

• internal: Code that will not be imported by any other packages and is used only on the running instance of DMS. This includes all configuration-related code, background tasks, etc.

• db: Database used by the DMS.

• storage: Disk storage management on each DMS for data related to DMS and jobs deployed by DMS. It also acts as an adapter to external storage services.

• api: All API functionality (including REST API, etc.) to interact with the DMS.

• cmd: Command line functionality and tools.

• network: All network-related code such as p2p communication, IP over Libp2p, and other networks that might be needed in the future.

• executor: Responsible for executing the jobs received by the DMS. Interface to various executors such as Docker, Firecracker, etc.

• telemetry: Logs, traces, and everything related to telemetry.

• plugins: Defined entry points and specs for third-party plugins, registration, and execution of plugin code.

• types: Contains data models imported by various packages.

• utils: Utility tools and functionalities.

• tokenomics: Interaction with blockchain for the crypto-micropayments layer of the platform.

Design and Architecture

Conceptual Basis

Main concepts of the architecture of DMS, the main component of the NuNet platform, can be found in the Yellow Paper.

Ontology

The Nunet Ontology, which forms the basis of the design, is explained in the articles below:

• NuNet Job Orchestration I: Ontology and Nomenclature

• NuNet Job Orchestration II: Scheduling

• NuNet Job Orchestration III: Mapping Ontology to Scheduling

Architecture

Refer to the following items to understand DMS architecture at a high level.

• DMS Architecture -- Understanding I

• Entity Diagram - DMS High Level

Research

Relevant research work that has informed the design of DMS can be found below:

• Detailed Job Orchestration Sequences I

• Detailed Job Orchestration Sequences II

• Gossipsub, DHT, and Push/Pull Mechanisms

• Parent-Child Hierarchy, Allocations, and Failure Tolerance

• Kubernetes Integration Specs

Functionality

DMS is currently being refactored and new functionality will be added.

Data Types

Refer to the DMS global class diagram in this section and various packages for data models.

References

In addition to the relevant links added in the sections above, you can also find useful links here: NuNet Links.

Class Diagram

The global class diagram for the DMS is shown below.

Source File

Global Class Diagram

Last updated: 2024-11-07 21:04:45.674781 File source: link on GitLab

NuActor: Secure Actor Programming in Decentralized Systems

NuActor is a framework designed for secure actor oriented programming in decentralized systems. The framework utilizes zero trust interactions, whereby every message is authenticated individually at the point of interaction. The system supports fine-grained capabilities, anchored in decentralized identifiers (see DID) and effected with user controlled authorization networks (see UCAN).

Secure Interactions in Decentralized Systems

Decentralized systems are distributed systems where there are different stake holders and controlling entities who are mutually distrustful. Actors are ideally suited for modeling and programming such systems, as they are able to express concurrency, distribution, and agency on behalf of their controllers.

However, given the open ended computing nature of decentralized systems, there is a fundamental problem in securing interactions. Because the system is open, there is effectively no perimeter; the messages are coming from the Internet, and can potentially originate in malicious or hostile actors.

NuActor takes the following approach:

• The only entity an actor can fully trust is itself and its controller.

• All messages invoking a behavior carry with them capability tokens that authorize them to perform the invocation.

• Invocations are checked at dispatch so that it is always verified whether an invocation is allowed, anchored on the entities the actor trusts for the required capabilities.

• There is no central authority; every entity (identified by a DID) can issue their own capability tokens and anchor trust wherever they want.

• There are certain entities in the open public networks that may be marginally trusted to vet users (KYC) for invoking public behaviors. The set of such entities is open, and everyone is free to trust whoever they want. The creators of the network at bootstrap are good candidates for such entities.

• Trust is ephemeral and can be revoked at all times.

• In effect, users are in control of authorization in the network (UCAN!)

Capabilities

Behaviors and the Capability Namespace

Capabilities are defined in a hierarchical namespace, akin to the UNIX file system structure. The root capability, which implicitly has all other capabilities, is /. Every other capability extends this path, separating the namespace with additional /s. A capability is narrower than another if it is a subpath in the UNIX sense. So /A imples /A/B and so on, but /A does not imply /B.

Behaviors have names that directly map to capabilities. So the behavior namespace is also hierarchical, allowing for easy automated matching of behaviors to capabilities.

Capability Tokens

Capabilities are expressed with a token, which is a structured object signed by the private key of the issuer. The issuer is in the token as a DID, which allows any entity inspecting the token to verify by retrieving the public key associated with the DID. Typically these are key DIDs, which embed the public key directly.

The structure of the token is as following:

type Token struct {
	// DMS tokens
	DMS *DMSToken `json:"dms,omitempty"`
}

type DMSToken struct {
	Action     Action       `json:"act"`
	Issuer     did.DID      `json:"iss"`
	Subject    did.DID      `json:"sub"`
	Audience   did.DID      `json:"aud"`
	Topic      []Capability `json:"topic,omitempty"`
	Capability []Capability `json:"cap"`
	Nonce      []byte       `json:"nonce"`
	Expire     uint64       `json:"exp"`
	Depth      uint64       `json:"depth,omitempty"`
	Chain      *Token       `json:"chain,omitempty"`
	Signature  []byte       `json:"sig,omitempty"`
}

The Subject is the DID of the entity to which the Issuer grants (if the chain is empty) or delegates the capabilities listed in the Capability field and the broadcast topics listed in the Topic field. The audience may be empty, but when present it restricts the receiver of invocations to a specified entity.

The Action can be any of Delegate, Invoke or Broadcast, with revocations to be added in the very near future.

If the Action is Delegate then the Issuer confers to the Subject the ability to further create new tokens, chained on this one.

If the Action is Invoke or Broadcast, then the token confers to the Subject the capability to make an invocation or broadcast to a behavior. Such tokens are terminal and cannot be chained further.

The Chain field of the token inlines the chain of tokens (could be a single one) on which the capability transfer is anchored on.

Note that the delegation spread can be restricted by the issuer of a token using the Depth field. If set, it is the maximum chain depth at which a token can appear. If it appears deeper in the chain, the token chain fails verification.

Finally, all capabilities have an expiration time (in UNIX nanoseconds). An expired token cannot be used any more and fails verification.

Anchors of Trust

In order to sign and verify token chains, the receiver needs to install some trust anchors. Specifically, we distinguish 3 types of anchors:

• root anchors which are DIDs that are fully trusted for input with implicit root capability. Any valid chain anchored on one of our roots will be admissible.

• require anchors which are tokens that act as side chains for marginal input trust. These tokens admit a chain anchored in their subject, as long as the capability and depth constraints are satisfied.

• provide anchors which are tokens that anchour the actor's output invocation and broadcast tokens. These are delegations which the actor can use to prove that it has the required capabilities, beside self-signing.

Token Chain Verification

The token chain is verified with strict rules:

• The entire chain must not have expired.

• Each token in the chain cannot expire before its chain.

• Each token must match the Issuer with the Subject of its chain.

• Each token in the chain can only narrow (attenuate) the capabilities of its chain.

• Each token in the chain can only narrow the audience; an empty audience ("to whom it may concern") can only be narrowed once to an audience DID and all chains build on top must concern the same audience.

• The chain of a token can only delegate.

• The signature must verify.

• The whole chain must recursively verify.

Programming NuActors

The actor package

The Go implementation of NuActor lives in the actor package of DMS.

To use it:

import "gitlab.com/nunet/device-management-service/actor"

The network substrate for NuActor is currently implemented with libp2p, with broadcast using gossipsub.

The Security Context

Each actor has a key pair for signing its messages; the actor's id is the public key itself and is embedded in every message it sends. The private key for the actor lives inside the actor's SecurityContext.

In general:

• each actor has its own SecurityContext; however, if the actor wants to create multiple subactors and act as an ensemble, it can share it.

• the key pair is ephemeral; however, the root actor in the process has a persistent key pair, which matches the libp2p key and Peer ID. This makes the actor reachable by default given its Peer ID or DID.

• every actor in the process shares a DID, which is the ID of the root actor.

Each Security Context is anchored in a process wide CapabilityContext, which stores anchors of trust and ephemeral tokes consumed during actor interactions.

The CapabilityContext itself is anchored on a TrustCotext, which contains the private key for the root actor and the process itself.

Actor Handles

type Handle struct {
	ID      ID      `json:"id"`
	DID     DID     `json:"did"`
	Address Address `json:"addr"`
}

type Address struct {
	HostID       string `json:"host,omitempty"`
	InboxAddress string `json:"inbox,omitempty"`
}

Messages

type Envelope struct {
	To         Handle          `json:"to"`
	Behavior   string          `json:"be"`
	From       Handle          `json:"from"`
	Nonce      uint64          `json:"nonce"`
	Options    EnvelopeOptions `json:"opt"`
	Message    []byte          `json:"msg"`
	Capability []byte          `json:"cap,omitempty"`
	Signature  []byte          `json:"sig,omitempty"`

	Discard func() `json:"-"`
}

type EnvelopeOptions struct {
	Expire  uint64 `json:"exp"`
	ReplyTo string `json:"cont,omitempty"`
	Topic   string `json:"topic,omitempty"`
}

The Actor Interfaces

type Actor interface {
	Context() context.Context
	Handle() Handle
	Security() SecurityContext

	AddBehavior(behavior string, continuation Behavior, opt ...BehaviorOption) error
	RemoveBehavior(behavior string)

	Receive(msg Envelope) error
	Send(msg Envelope) error
	Invoke(msg Envelope) (<-chan Envelope, error)

	Publish(msg Envelope) error
	Subscribe(topic string, setup ...BroadcastSetup) error

	Start() error
	Stop() error

	Limiter() RateLimiter
}

Sending Messages

The following code shows how to send a message at the call site:

	msg, _ := actor.Message(
		myActor.Handle(),
		destinationHandle,
		"/some/behavior",
		MyMessage{ /*...*/ },
	)
	_ = myActor.Send(msg)

At the receiver this is how we can react to the message:

	_ = myActor.AddBehavior("/some/behavior", func(msg Envelope) {
		defer msg.Discard()
		// process message
	})

Notice the _ for errors, please don't do this in production.

Interactive Invocations

Interactive invocations are a combinations of a synchronous send and wait for a reply.

At the call site:

	msg, _ := actor.Message(
		myActor.Handle(),
		destinationHandle,
		"/some/behavior",
		MyMessage{ /*...*/ },
	)
	replyChan, _ := myActorInvoke(msg)
	reply := <-replyChan
	defer reply.Discard()
	// process reply ...

At the receiver this is how we can create an interactive behavior:

	_ = myActor.AddBehavior("/some/behavior", func(msg Envelope) {
		defer msg.Discard()
		reply, _ := actor.ReplyTo(
			msg
			MyReply{ /*...*/ },
		)
		_ = mayActor.Send(MyReply)
	})

Again, notice the _ for errors, please don't do this in production.

Broadcast

We can easily broadcast messages to all interested parties in a topic.

At the broadcast site:

	msg, _ := actor.Message(
		myActor.Handle(),
		destinationHandle,
		"/some/broadcast/behavior",
		MyMessage{ /*...*/ },
		actor.WithMessageTopic("/some/topic"),
	)
	_ = actor.Publish(msg)

At the receiver:

	_ = myActor.Subscribe("/some/topic")
	_ = myActor.AddBehavior("/some/broadcast/behavior", func(msg Envelope) {
		defer msg.Discard()
		// process the message
	}, actor.WithBehaviorTopic("/some/topic"),
	)

Discarding Capability Tokens

Notice all these defer msg.Discard() in the examples above; this is necessary to ensure deterministic cleanup of tokens exchanged during the interaction. Please do not forget that.

Behind the Scenes

Sending a Message

Invoking a Behavior

Verifying Capabilities

NuNet is a computing platform that provides globally distributed and optimized computing power and storage for decentralized networks, by connecting data owners and computing resources with computational processes in demand of these resources. NuNet provides a layer of intelligent interoperability between computational processes and physical computing infrastructures in an ecosystem which intelligently harnesses latent computing resources of the community into the global network of computations.

Detailed information about the NuNet platform, concepts, architecture, models, stakeholders can be found in these two papers:

Device Management Service (DMS) is a core component of the NuNet platform that orchestrates the management of all devices contributing to our decentralized network. The DMS is designed to enable secure and efficient communication and coordination across devices of varying capabilities, ensuring they can work together in a trustless, decentralized environment.

As you explore the documentation, you will find in-depth guides and technical specifications to help you navigate the features and functionalities of the DMS and other components of the NuNet platform.

Last updated: 2024-11-07 21:04:46.460092 File source:

Nunet Actor System CLI

The Nunet Actor System CLI provides a set of commands for interacting with the Nunet actor system, enabling you to send messages to actors, invoke behaviors, and broadcast messages across the network.

Basic Commands

• nunet actor msg: Constructs a message for an actor.

• nunet actor send: Sends a constructed message to an actor.

• nunet actor invoke: Invokes a behavior in an actor and returns the result.

• nunet actor broadcast: Broadcasts a message to a topic, potentially reaching multiple actors.

• nunet actor cmd: Invokes a predefined public behavior on an actor.

nunet actor msg

This command is used to create a message that can be sent to an actor. It encapsulates the behavior to be invoked and the associated payload data.

Usage

Arguments

• <behavior>: The specific behavior you want the actor to perform upon receiving the message

• <payload>: The data accompanying the message, providing context or input for the behavior

Flags

• -b, --broadcast string: Designates the topic for broadcasting the message.

• -c, --context string: Specifies the capability context name

• -d, --dest string: Identifies the destination handle for the message.

• -e, --expiry time: Sets an expiration time for the message.

• -h, --help: Displays help information for the msg command.

• -i, --invoke: Marks the message as an invocation, requesting a response from the actor.

• -t, --timeout duration: Sets a timeout for awaiting a response after invoking a behavior.

nunet actor send

This command delivers a previously constructed message to an actor.

Usage

Arguments

• <msg>: The message, created using the nunet actor msg command, to be sent.

Flags

• -h, --help: Displays help information for the send command.

nunet actor invoke

This command directly invokes a specific behavior on an actor and expects a response.

Usage

Arguments

• <msg>: The message, crafted with nunet actor msg, containing the behavior and payload

Flags

• -h, --help: Displays help information for the invoke command

nunet actor broadcast

This command disseminates a message to a designated topic, potentially reaching multiple actors who have subscribed to that topic.

Usage

Arguments

• <msg>: The message to be broadcasted

Flags

• -h, --help: Displays help information for the broadcast command.

Please let me know if you have any other questions.

nunet actor cmd

This command invokes a behavior on an actor.

Usage

Available Commands

• /broadcast/hello: Invoke /broadcast/hello behavior on an actor.

• /dms/node/onboarding/offboard: Invoke /dms/node/onboarding/offboard behavior on an actor.

• /dms/node/onboarding/onboard: Invoke /dms/node/onboarding/onboard behavior on an actor.

• /dms/node/onboarding/resource: Invoke /dms/node/onboarding/resource behavior on an actor.

• /dms/node/onboarding/status: Invoke /dms/node/onboarding/status behavior on an actor.

• /dms/node/peers/connect: Invoke /dms/node/peers/connect behavior on an actor.

• /dms/node/peers/dht: Invoke /dms/node/peers/dht behavior on an actor.

• /dms/node/peers/list: Invoke /dms/node/peers/list behavior on an actor.

• /dms/node/peers/ping: Invoke /dms/node/peers/ping behavior on an actor.

• /dms/node/peers/score: Invoke /dms/node/peers/score behavior on an actor.

• /dms/node/peers/self: Invoke /dms/node/peers/self behavior on an actor.

• /dms/node/vm/list: Invoke /dms/node/vm/list behavior on an actor.

• /dms/node/vm/start/custom: Invoke /dms/node/vm/start/custom behavior on an actor.

• /dms/node/vm/stop: Invoke /dms/node/vm/stop behavior on an actor.

• /public/hello: Invoke /public/hello behavior on an actor

• /public/status: Invoke /public/status behavior on an actor

Flags

• -c, --context string: Capability context name.

• -d, --dest string: Destination DMS DID, peer ID or handle.

• -e, --expiry time: Expiration time.

• -h, --help: Help for the cmd command.

• -t, --timeout duration: Timeout duration.

Broadcast Commands

• /broadcast/hello

  • Description: Invokes the /broadcast/hello behavior on an actor. This sends a "hello" message to a broadcast topic for polite introduction.

  • Usage: nunet actor cmd /broadcast/hello [<param> ...] [flags]

  • Flags:

    • -h, --help: Display help information for the /broadcast/hello command

DMS Node Commands

• /dms/node/onboarding/offboard

  • Description: Invokes the /dms/node/onboarding/offboard behavior on an actor. This is used to offboard a node from the DMS (Device Management Service).

  • Usage: nunet actor cmd /dms/node/onboarding/offboard [<param> ...] [flags]

  • Flags:

    • -f, --force: Force the offboarding process overriding any safety checks and invalid states.

    • -h, --help: Display help information for the /dms/node/onboarding/offboard command.

• /dms/node/onboarding/onboard

  • Description: Invokes the /dms/node/onboarding/onboard behavior on an actor. This is used to onboard a node to the DMS, making its resources available for use.

  • Usage: nunet actor cmd /dms/node/onboarding/onboard [<param> ...] [flags]

  • Flags:

    • -a, --available: Set the node as unavailable for job deployment (default: false).

    • -z, --cpu int: Set number of CPU cores.

    • -h, --help: Display help information for the /dms/node/onboarding/onboard command.

    • -l, --local-enable: Set server mode (enable for local) (default: true).

    • -m, --memory uint: Set the value for memory usage in GB.

    • -x, --ntx-price float: Set the price in NTX per minute for the onboarded compute resource.

    • -w, --wallet string: Set the wallet address.

• /dms/node/onboarding/resource

  • Description: Invokes the /dms/node/onboarding/resource behavior on an actor. This retrieves or manages resource information related to the onboarding process.

  • Usage: nunet actor cmd /dms/node/onboarding/resource [<param> ...] [flags]

  • Flags:

    • -h, --help: Display help information for the /dms/node/onboarding/resource command.

• /dms/node/onboarding/status

  • Description: Invokes the /dms/node/onboarding/status behavior on an actor. This is used to check the onboarding status of a node.

  • Usage: nunet actor cmd /dms/node/onboarding/status [<param> ...] [flags]

  • Flags:

    • -h, --help: Display help information for the /dms/node/onboarding/status command

• /dms/node/peers/connect

  • Description: Invokes the /dms/node/peers/connect behavior on an actor. This initiates a connection to a specified peer.

  • Usage: nunet actor cmd /dms/node/peers/connect [<param> ...] [flags]

  • Flags:

    • -a, --address string: The peer address to connect to

    • -h, --help: Display help information for the /dms/node/peers/connect command.

• /dms/node/peers/dht

  • Description: Invokes the /dms/node/peers/dht behavior on an actor. This interacts with the Distributed Hash Table (DHT) used for peer discovery and content routing

  • Usage: nunet actor cmd /dms/node/peers/dht [<param> ...] [flags]

  • Flags:

    • -h, --help: Display help information for the /dms/node/peers/dht command.

• /dms/node/peers/list

  • Description: Invokes the /dms/node/peers/list behavior on an actor. This retrieves a list of connected peers

  • Usage: nunet actor cmd /dms/node/peers/list [<param> ...] [flags]

  • Flags:

    • -h, --help: Display help information for the /dms/node/peers/list command.

• /dms/node/peers/ping

  • Description: Invokes the /dms/node/peers/ping behavior on an actor. This sends a ping message to a specified host to check its reachability

  • Usage: nunet actor cmd /dms/node/peers/ping [<param> ...] [flags]

  • Flags:

    • -h, --help: Display help information for the /dms/node/peers/ping command

    • -H, --host string: The host address to ping

• /dms/node/peers/score

  • Description: Invokes the /dms/node/peers/score behavior on an actor. This retrieves a snapshot of the peer's gossipsub broadcast score.

  • Usage: nunet actor cmd /dms/node/peers/score [<param> ...] [flags]

  • Flags:

    • -h, --help: Display help information for the /dms/node/peers/score command

• /dms/node/peers/self

  • Description: Invokes the /dms/node/peers/self behavior on an actor. This retrieves information about the node itself, such as its ID or addresses

  • Usage: nunet actor cmd /dms/node/peers/self [<param> ...] [flags]

  • Flags:

    • -h, --help: Display help information for the /dms/node/peers/self command.

• /dms/node/vm/list

  • Description: Invokes the /dms/node/vm/list behavior on an actor. This retrieves a list of virtual machines (VMs) running on the node

  • Usage: nunet actor cmd /dms/node/vm/list [<param> ...] [flags]

  • Flags:

    • -h, --help: Display help information for the /dms/node/vm/list command.

• /dms/node/vm/start/custom

  • Description: Invokes the /dms/node/vm/start/custom behavior on an actor. This starts a new VM with custom configurations.

  • Usage: nunet actor cmd /dms/node/vm/start/custom [<param> ...] [flags]

  • Flags:

    • -a, --args string: Arguments to pass to the kernel

    • -z, --cpu float32: CPU cores to allocate (default 1)

    • -h, --help: Display help information for the /dms/node/vm/start/custom command.

    • -i, --initrd string: Path to initial ram disk

    • -k, --kernel string: Path to kernel image file.

    • -m, --memory uint: Memory to allocate (default 1024)

    • -r, --rootfs string: Path to root fs image file

• /dms/node/vm/stop

  • Description: Invokes the /dms/node/vm/stop behavior on an actor. This stops a running VM

  • Usage: nunet actor cmd /dms/node/vm/stop [<param> ...] [flags]

  • Flags:

    • -h, --help: Display help information for the /dms/node/vm/stop command

    • -i, --id string: Execution id of the VM

Public Commands

• /public/hello

  • Description: Invokes the /public/hello behavior on an actor. This broadcasts a "hello" for a polite introduction.

  • Usage: nunet actor cmd /public/hello [<param> ...] [flags]

  • Flags:

    • -h, --help: Display help information for the /public/hello command

• /public/status

  • Description: Invokes the /public/status behavior on an actor. This retrieves the status or health information of the actor or system

  • Flags:

    • -h, --help: Display help information for the /public/status command

Global Flags

These flags can be used with any of the above commands:

• -c, --context string: Specifies the capability context name. This is used for authorization or access control.

• -d, --dest string: Specifies the destination for the command. This can be a DMS DID (Decentralized Identifier), a peer ID, or a handle.

• -e, --expiry time: Sets an expiration time for the message or command.

• -t, --timeout duration: Sets a timeout duration for the command. If the command does not complete within the specified duration, it will time out.

• -h, --help: Display help information for the commands

Last updated: 2024-11-07 21:04:45.945848 File source:

api

Specification

Description

The api package contains all API functionality of Device Management Service (DMS). DMS exposes various endpoints through which its different functionalities can be accessed.

Structure and Organisation

Here is quick overview of the contents of this directory:

• : Current file which is aimed towards developers who wish to use and modify the api functionality.

• : This file contains router setup using Gin framework. It also applies Cross-Origin Resource Sharing (CORS) middleware and OpenTelemetry middleware for tracing. Further it lists down the endpoint URLs and the associated handler functions.

• : Contains endpoints for actor interaction.

Functionality

Configuration

The REST server by default binds to 127.0.0.1 on port 9999. The configuration file dms_config.json can be used to change to a different address and port.

The parameters rest.port and rest.addr define the port and the address respectively.

You can use the following format to construct the URL for accessing API endpoints

Currently, all endpoints are under the /actor path

/actor/handle

Retrieve actor handle with ID, DID, and inbox address

Response:

/actor/send

Send a message to actor

The request should be an enveloped message

/actor/invoke

Invoke actor with message

The request should be an enveloped message

/actor/broadcast

Broadcast message to actors

The request should be an enveloped message

Last updated: 2024-11-07 21:04:47.304305 File source:

db

Table of Contents

Specification

Description

The db package contains the configuration and functionality of database used by the DMS

Structure and Organisation

Here is quick overview of the contents of this pacakge:

Files

Subpackages

Class Diagram

The class diagram for the db package is shown below.

Source file

Rendered from source file

Functionality

There are two types of interfaces defined to cover database operations:

1. GenericRepository

2. GenericEntityRepository

These interfaces are described below.

GenericRepository Interface

GenericRepository interface defines basic CRUD operations and standard querying methods. It is defined with generic data types. This allows it to be used for any data type.

• interface definition: type GenericRepository[T ModelType] interface

The methods of GenericRepository are as follows:

Create

• signature: Create(ctx context.Context, data T) -> (T, error)

• input #1: Go context

• input #2: Data to be added to the database. It should be of type used to initialize the repository

• output (success): Data type used to initialize the repository

• output (error): error message

Create function adds a new record to the database.

Get

• signature: Get(ctx context.Context, id interface{}) -> (T, error)

• input #1: Go context

• input #2: Identifier of the record. Can be any data type

• output (success): Data with the identifier provided. It is of type used to initialize the repository

• output (error): error message

Get function retrieves a record from the database by its identifier.

Update

• signature: Update(ctx context.Context, id interface{}, data T) -> (T, error)

• input #1: Go context

• input #2: Identifier of the record. Can be any data type

• input #3: New data of type used to initialize the repository

• output (success): Updated record of type used to initialize the repository

• output (error): error message

Update function modifies an existing record in the database using its identifier.

Delete

• signature: Delete(ctx context.Context, id interface{}) -> error

• input #1: Go context

• input #2: Identifier of the record. Can be any data type

• output (success): None

• output (error): error message

Delete function deletes an existing record in the database using its identifier.

Find

• signature: Find(ctx context.Context, query Query[T]) -> (T, error)

• input #1: Go context

• input #2: Query of type db.query

• output (success): Result of query having the data type used to initialize the repository

• output (error): error message

Find function retrieves a single record from the database based on a query.

FindAll

• signature: FindAll(ctx context.Context, query Query[T]) -> ([]T, error)

• input #1: Go context

• input #2: Query of type db.query

• output (success): Lists of records based on query result. The data type of each record will be what was used to initialize the repository

• output (error): error message

FindAll function retrieves multiple records from the database based on a query.

GetQuery

• signature: GetQuery() -> Query[T]

• input: None

• output: Query of type db.query

GetQuery function returns an empty query instance for the repository's type.

GenericEntityRepository Interface

GenericEntityRepository defines basic CRUD operations for repositories handling a single record. It is defined with generic data types. This allows it to be used for any data type.

• interface definition: type GenericEntityRepository[T ModelType] interface

The methods of GenericEntityRepository are as follows:

Save

• signature: Save(ctx context.Context, data T) -> (T, error)

• input #1: Go context

• input #2: Data to be saved of type used to initialize the database

• output (success): Updated record of type used to initialize the repository

• output (error): error message

Save function adds or updates a single record in the repository

Get

• signature: Get(ctx context.Context) -> (T, error)

• input: Go context

• output (success): Record of type used to initialize the repository

• output (error): error message

Get function retrieves the single record from the database.

Clear

• signature: Clear(ctx context.Context) -> error

• input: Go context

• output (success): None

• output (error): error

Clear function removes the record and its history from the repository.

History

• signature: History(ctx context.Context, qiery Query[T]) -> ([]T, error)

• input #1: Go context

• input #2: query of type db.query

• output (success):List of records of repository's type

• output (error): error

History function retrieves previous records from the repository which satisfy the query conditions.

GetQuery

• signature: GetQuery() -> Query[T]

• input: None

• output: New query of type db.query

GetQuery function returns an empty query instance for the repository's type.

Data Types

• db.Query: This contains parameters related to a query that is passed to the database.

• db.QueryCondition: This contains parameters defining a query condition.

GenericRepository has been initialised for the following data types:

• types.DeploymentRequestFlat

• types.VirtualMachine

• types.PeerInfo

• types.Machine

• types.Services

• types.ServiceResourceRequirements

• types.Connection

• types.ElasticToken

GenericEntityRepository has been initialised for the following data types:

• types.FreeResources

• types.AvailableResources

• types.Libp2pInfo

• types.MachineUUID

Testing

The unit tests for utility functions are defined in utils_test.go. Refer to *_test.go files for unit tests of various implementations covered in subpackages.

Proposed Functionality / Requirements

List of issues

All issues that are related to the implementation of db package can be found below. These include any proposals for modifications to the package or new functionality needed to cover the requirements of other packages.

References

Last updated: 2024-11-07 21:04:47.848116 File source:

gorm

Table of Contents

Specification

Description

This sub package contains Gorm implementation of the database interfaces.

Structure and Organisation

Here is quick overview of the contents of this pacakge:

All files with *_test.go naming convention contain unit tests with respect to the specific implementation.

Class Diagram

The class diagram for the gorm package is shown below.

Source file

Rendered from source file

Functionality

GenericRepository

NewGenericRepository

• signature: NewGenericRepository[T repositories.ModelType](db *gorm.DB) -> repositories.GenericRepository[T]

• input: Gorm Database object

• output: Repository of type db.gorm.GenericRepositoryGORM

NewGenericRepository function creates a new instance of GenericRepositoryGORM struct. It initializes and returns a repository with the provided GORM database.

Interface Methods

GenericEntityRepository

NewGenericEntityRepository

• signature: NewGenericEntityRepository[T repositories.ModelType](db *gorm.DB) -> repositories.GenericEntityRepository[T]

• input #1: Gorm Database object

• output: Repository of type db.gorm.GenericEntityRepositoryGORM

NewGenericEntityRepository creates a new instance of GenericEntityRepositoryGORM struct. It initializes and returns a repository with the provided GORM database.

Interface Methods

Data Types

• db.gorm.GenericRepositoryGORM: This is a generic repository implementation using GORM as an ORM.

• db.gorm.GenericEntityRepositoryGORM: This is a generic single entity repository implementation using GORM as an ORM

For other data types refer to db package readme.

Testing

Refer to *_test.go files for unit tests of different functionalities.

Proposed Functionality / Requirements

List of issues

All issues that are related to the implementation of db package can be found below. These include any proposals for modifications to the package or new functionality needed to cover the requirements of other packages.

References

Last updated: 2024-11-07 21:04:47.589794 File source:

clover

Table of Contents

Specification

Description

This sub package contains CloverDB implementation of the database interfaces.

Structure and Organisation

Here is quick overview of the contents of this pacakge:

All files with *_test.go naming convention contain unit tests with respect to the specific implementation.

Class Diagram

The class diagram for the clover package is shown below.

Source file

Rendered from source file

Functionality

GenericRepository

NewGenericRepository

• signature: NewGenericRepository[T repositories.ModelType](db *clover.DB) -> repositories.GenericRepository[T]

• input: clover Database object

• output: Repository of type db.clover.GenericRepositoryclover

NewGenericRepository function creates a new instance of GenericRepositoryclover struct. It initializes and returns a repository with the provided clover database.

Interface Methods

query

• signature: query(includeDeleted bool) -> *clover_q.Query

• input: boolean value to choose whether to include deleted records

queryWithID

• signature: queryWithID(id interface{}, includeDeleted bool) -> *clover_q.Query

• input #1: identifier

• input #2: boolean value to choose whether to include deleted records

Providing includeDeleted as False will add a condition to exclude the deleted records.

GenericEntityRepository

NewGenericEntityRepository

• signature: NewGenericEntityRepository[T repositories.ModelType](db *clover.DB) repositories.GenericEntityRepository[T]

• input: clover Database object

• output: Repository of type db.clover.GenericEntityRepositoryclover

NewGenericEntityRepository creates a new instance of GenericEntityRepositoryclover struct. It initializes and returns a repository with the provided clover database instance and name of the collection in the database.

Interface Methods

query

• signature: query() -> *clover_q.Query

• input: None

Data Types

• db.clover.GenericRepositoryClover: This is a generic repository implementation using clover as an ORM

• db.clover.GenericEntityRepositoryClover: This is a generic single entity repository implementation using clover as an ORM

For other data types refer to db package readme.

Testing

Refer to *_test.go files for unit tests of different functionalities.

Proposed Functionality / Requirements

List of issues

All issues that are related to the implementation of db package can be found below. These include any proposals for modifications to the package or new functionality needed to cover the requirements of other packages.

References

Last updated: 2024-11-07 21:04:48.114546 File source:

dms

Table of Contents

Specification

Description

This package is responsible for starting the whole application. It also contains various core functionality of DMS:

• Onboarding compute provider devices

• Job orchestration and management

• Resource management

• Actor implementation for each node

Structure and Organisation

Here is quick overview of the contents of this pacakge:

Subpackages

proposed: All files with *_test.go naming convention contain unit tests with respect to the specific implementation.

Class Diagram

The class diagram for the dms package is shown below.

Source file

Rendered from source file

Functionality

TBD

Data Types

TBD

Testing

proposed Refer to *_test.go files for unit tests of different functionalities.

Proposed Functionality / Requirements

List of issues

All issues that are related to the implementation of dms package can be found below. These include any proposals for modifications to the package or new functionality needed to cover the requirements of other packages.

Interfaces & Methods

proposed Capability_interface

add method will combine capabilities of two nodes. Example usage - When two jobs have to be run on a single machine, the capability requirements of each will need to be combined.

subtract method will subtract two capabilities. Example usage - When resources are locked for a job, the available capability of a machine will need to be reduced.

Data types

proposed dms.Capability

The Capability struct will capture all the relevant data that defines the capability of a node to perform the job. At the same time this will be used to define capability requirements that a job requires from a node.

An initial data model for Capability is defined below.

proposed dms.Connectivity

type Connectivity struct {

}

proposed dms.PriceInformation

proposed dms.TimeInformation

type TimeInformation struct { // Units holds the units of time ex - hours, days, weeks Units string

}

References

Last updated: 2024-11-07 21:04:47.036333 File source:

db

Table of Contents

Specification

Description

This package defines the local database functionality for the Device Management Service (DMS). Currently two repository structures have been implemented:

• clover: which is a NoSQL or document oriented database implementation.

Structure and Organisation

Here is quick overview of the contents of this pacakge:

Class Diagram

The class diagram for the db package is shown below.

Source file

Rendered from source file

Package Specification

Last updated: 2024-11-07 21:04:46.736000 File source:

Nunet Capability Management CLI Documentation

The Nunet Capability Management CLI provides commands to manage capabilities within the Nunet ecosystem. Capabilities define the permissions and authorizations granted to different entities within the system. These commands allow you to create, modify, delegate, and list capabilities.

Commands

• nunet cap anchor: Add or modify capability anchors in a capability context.

• nunet cap delegate: Delegate capabilities for a subject.

• nunet cap grant: Grant (delegate) capabilities as anchors and side chains from a capability context.

• nunet cap list: List all capability anchors in a capability context

• nunet cap new: Create a new persistent capability context for DMS or personal usage

• nunet cap remove: Remove capability anchors in a capability context.

nunet cap anchor

This command is used to add new or modify existing capability anchors within a specific capability context. Anchors serve as the foundation for defining capabilities, establishing the core permissions and restrictions

Usage

Flags

• -c, --context string: Specifies the operation context, defining the key and capability context to use (defaults to "user").

• -h, --help: Displays help information for the anchor command

• --provide string: Adds tokens as a "provide" anchor in JSON format, defining capabilities the context can offer

• --require string: Adds tokens as a "require" anchor in JSON format, defining capabilities the context demands

• --root string: Adds a DID as a "root" anchor, establishing the root authority for the context

nunet cap delegate

This command delegates specific capabilities to a subject, granting them permissions within the system

Usage

Arguments

• <subjectDID>: The Decentralized Identifier (DID) of the entity receiving the delegated capabilities

Flags

• -a, --audience string: (Optional) Specifies the audience DID, restricting the delegation to a specific recipient

• --cap strings: Defines the capabilities to be granted or delegated (can be specified multiple times)

• -c, --context string: Specifies the operation context (defaults to "user")

• -d, --depth uint: (Optional) Sets the delegation depth, controlling how many times the capabilities can be further delegated (default 0)

• --duration duration: Sets the duration for which the delegation is valid

• -e, --expiry time: Sets an expiration time for the delegation

• -h, --help: Displays help information for the delegate command

• --self-sign string: Specifies self-signing options: 'no' (default), 'also', or 'only'

• -t, --topic strings: Defines the topics for which capabilities are granted or delegated (can be specified multiple times)

nunet cap grant

This command grants (delegates) capabilities as anchors and side chains from a specified capability context

Usage

Arguments

• <subjectDID>: The Decentralized Identifier (DID) of the entity receiving the granted capabilities

Flags

• -a, --audience string: (Optional) Specifies the audience DID

• --cap strings: Defines the capabilities to be granted or delegated

• -c, --context string: Specifies the operation context (defaults to "user")

• -d, --depth uint: (Optional) Sets the delegation depth

• --duration duration: Sets the duration for which the grant is valid

• -e, --expiry time: Sets an expiration time for the grant

• -h, --help: Displays help information for the grant command

• -t, --topic strings: Defines the topics for which capabilities are granted

nunet cap list

This command lists all capability anchors within a specified capability context

Usage

Flags

• -c, --context string: Specifies the operation context (defaults to "user")

• -h, --help: Displays help information for the list command

nunet cap new

This command creates a new persistent capability context, which can be used for DMS or personal purposes

Usage

Arguments

• <name>: The name for the new capability context

Flags

• -h, --help: Displays help information for the new command

nunet cap remove

This command removes capability anchors from a specified capability context

Usage

Flags

• -c, --context string: Specifies the operation context (defaults to "user")

• -h, --help: Displays help information for the remove command

• --provide string: Removes tokens from the "provide" anchor in JSON format

• --require string: Removes tokens from the "require" anchor in JSON format

• --root string: Removes a DID from the "root" anchor

Last updated: 2024-11-07 21:04:48.407065 File source:

dms

Table of Contents

Specification

Description

The hardware package is responsible for handling the hardware related functionalities of the DMS.

Structure and Organisation

Here is quick overview of the contents of this package:

Functionality

GetMachineResources()

• signature: GetMachineResources() (types.MachineResources, error)

• input: None

• output: types.MachineResources

• output(error): error

GetCPU()

• signature: GetCPU() (types.CPU, error)

• input: None

• output: types.CPU

• output(error): error

GetRAM()

• signature: GetRAM() (types.RAM, error)

• input: None

• output: types.RAM

• output(error): error

GetDisk()

• signature: GetDisk() (types.Disk, error)

• input: None

• output: types.Disk

• output(error): error

Data Types

Testing

The tests can be found in the *_test.go files in the respective packages.

References