c_observability-framework

Last updated: 2024-09-06 01:09:56.786919 File source: link on GitLab

Observability framework

The nature of open-ended decentralized computing dictates that computing processes cannot be directly observed or controlled from a single point in software architecture. The absence of the requirement of persistent global state and synchronous execution, while allowing for richer and more scalable models of computation, is also difficult to reason about and creates a great deal of confusion as it contrasts with the usual practices of software development, which has possibly contributed to complications in the adoption of the actor model in practical computing and software development. It would be probably fair to say that this confusion is created by the very principle of open-ended decentralized computing, which posits that there is no global observer understanding the 'true and only nature' of the system. However that does not prevent to posit local observers which can reason about some properties of the system and the nature of selected aspects of it. For that purpose, we are building an observability framework.

Therefore, in order to provide observability over computational processes, the NuNet computing model considers two fairly separate subsystems – implementation of the B.1 Computing model and Observability framework. This section considers Observability framework in the context of the overall model.

• Computing model enables implementation of all aspects of open-ended de-centralized computing: (1) heterogeneous agent’s logic, (2) actor framework, (3) graph computing engine and persistent network topology (distributed or otherwise);

• Observability framework collects information of events happening in the network and performs real-time or post-mortem analysis or visualizations as required by software design requirements and use-cases.

Observability framework is currently implemented by:

• device-management-service as a telemetry package, which defines and implements interfaces that are then used by each Node in the network to observe internal events and send them to registered data Collectors (which can be external or internal to the network);

• External to the platform, but part of NuNet infrastructure Elasticsearch cluster, which will be by default configured to serve as the main collector of events in the whole network. Initially, in the development environment, the NuNet-run Elasticsearch cluster will be used for collecting traces, metrics and logs for the debugging purposes. Eventually in the production environment, it will be used for network telemetry dashboard as well as data source for reputation system's infrastructure.

Observability framework implemented in NuNet is loosely based on adapting the following prior research Offer networks: a model of decentralized exchange/Software architecture/Monitoring and analysis engine done by SingularityNET as well as learnings from further NuNet's efforts.

Decentralized indexing

Observability framework is related to the notion of decentralized indexing, which is the most abstract conceptual framework to think about that. Indexing is an an essential technical aspect of graph computing and the operation of decentralized networks in general.

Indexing therefore can be seen as a way to introduce specialized supernodes which effectively reduce the diameter of a graph and make information propagation between nodes more efficient. Another aspect of indices in general and search engines in particular is that they provide a mapping between node content and address for routing purposes. Actually, modern search engines introduce a kind of content addressing into the web – often one does not even notice the actual URL of a server or webpage one follows from a search engine’s page.

Qualitatively speaking, search engines and reputation systems and their indexes are also nodes in the web graph, alongside other servers, programs and devices. Yet they have two notable quantitative differences: (1) a much larger degree of connectivity to other nodes (which is the whole point of search engines/reputation systems – i.e. providing a gateway to other nodes that are needed for some business use-case / purpose) and (2) more computing resources (storage space and processing power). These are the supernodes. In network science and graph theory, a supernode is a vertex of a graph that has a disproportionally large number of connections to other vertices.

Graph computing allows any Actor in the network (so in NuNet case Node or Allocation) to construct any use-case specific index it needs for its goals and by that become a 'supernode' in the network, if network participants consider that index important for their own goals. Observability framework shall provide low level technical implementation allowing for any user/stakeholder/actor to create such a supernode in the NuNet platform that would serve purposes of the network efficiency of business needs of one or more class of its users. This is related to the NuNet reputation system infrastructure architecture.