6079 Proof of Inference Protocol
Probabilistic Verification of AI Inference on Decentralized Physical Infrastructure Ai Layer | February 23, 2024
Abstract: We introduce 6079 Proof of Inference Protocol (PoIP), a system for enabling effective peer-to-peer AI compute within an open and decentralized AI ecosystem.
The surge of large language models (LLMs) raises significant challenges regarding verification of distributed compute integrity due to their non-deterministic nature. PoIP overcomes this challenge by measuring hardware node capability and transaction compliance through layers of cryptoeconomic security, paving the way for widespread adoption of Decentralized Physical Infrastructure (DePIn) approaches.
We present a strategy governing effective behavior across diverse infrastructure and outline the two core components of PoIP:
Inference Engine Standard: This Service and Control Layer defines standardized compute patterns for PoIP.
Proof of Inference Protocol: This Transaction, Proof, and Economic Layer incentivizes productive behavior and discourages malicious activity through a combination of cryptographic, diagnostic, game-theoretic, and economic mechanisms.
The Problem: AI Inference on Decentralized Physical Infrastructure Lacks Robust Security at Scale
Bitcoin popularized blockchain technology as a method for decentralized trust. The process of adding new bitcoins to the blockchain, known as Proof of Work, involves finding a specific numerical value (“nonce”) that generates a hash value below a predetermined difficulty target. The design of Bitcoin enables robust security through a probabilistic proof demonstrating that it is uneconomical for an attacker to corrupt the blockchain.
Ethereum pioneered the concept of smart contracts, self-executing code stored on the blockchain that can automate agreements and financial transactions without intermediaries. This enabled the creation of Decentralized Finance (DeFi) applications, Non-Fungible Tokens (NFTs), Decentralized Autonomous Organizations (DAOs), and Decentralized Physical Infrastructure (DePIn).
Ethereum also led the shift from energy-intensive "proof of work" to "proof of stake" consensus, which replaces computationally expensive proof of work mining with a system where users "stake" a large number of existing coins to validate transactions, securing the network and earning rewards proportionally to their stake. However if a validator demonstrates bad behavior their stake is slashed with a probabilistic mechanism based around the same time, which is known as the correlation penalty. The design of Ethereum’s proof of stake consensus mechanism enables robust cryptoeconomic security by making the cost of attacking the system greater than any expected reward.
The rise in capability of Large Language Models (LLMs) and other generative AI has increased demand for powerful Graphics Processing Units (GPUs) which are necessary for performant compute on such large models. In response, a number of DePIn providers including Render Network, Akash, and IOnet have made GPUs and other hardware available to rent by the hour on-chain instead of through traditional payment options like a credit card. Bringing distributed hardware on-chain is a first step in building a decentralized AI platform—yet the market lacks a robust security protocol that would assign a useful probabilistic value to the question of whether a swarm of distributed GPUs has returned the correct inference computation.
Our introduction of Proof of Inference Protocol, described herein, enables large swarms of GPUs to coordinate in a trustless manner.
The Solution: Proof of Inference
We propose 6079 Proof of Inference Protocol (PoIP), enabling effective verification and security for decentralized AI inference across DePIn networks. Many popular large AI models are non-deterministic and therefore the outputs of an inference system cannot be easily proven in practice.
PoIP leverages a cryptoeconomic security approach to incentivize desired behavior and penalize malicious actors. This approach quantifies the cost an adversary must incur to disrupt the protocol. This makes it effectively resistant to corruption.
The optimization function is to ensure that the Cost-of-Corruption (CoC) remains significantly higher than the Profit-from-Corruption (PfC).
Interactions between AI and blockchains have been described by Ethereum co-founder Vitalik Buterin as a platform for creating "games." PoIP can be thought of as a game mechanic for coordinating a series of actors (Nodes and Agents) toward economic alignment. We are creating a literal game on top of the geometric structure of the physical node network.
The initial competitive game, which is one feature of the Proof of Inference Protocol, enables humans who interact with LLMs to identify non-compliant GPUs (Bad Actors). We are able to deterministically value the lowest probability that the system identifies a bad actor based on results of a sample of successful identification of bad actors. The optimization function translates into an adaptive incentive system that adjusts the penalty and reward system for actors in the network. Thus providing the system robust cryptoeconomic security.
6079 PoIP adopts concepts from the fields of blockchain, artificial intelligence, game theory, and complex adaptive systems.
We anticipate that at scale the game will evolve into a competitive game between AI agents that must demonstrate fitness in competition against other agents running models on arbitrary GPU swarms.
Proposed Model for Decentralized Inference
We adopt a similar approach to the SAKSHI Decentralized AI platform architecture and introduce PoIP as a missing critical step for probabilistic verification of non-deterministic models which maximize the CoC and minimize PfC.
Service Layer: Model inference is computed on physical hardware and sent to the Control Layer. Control Layer: API endpoints are hosted for query/response. Model weights, which are the instructions for computing AI/ML models, are sent to the Service Layer, load balancing is coordinated, and–where diagnostic tests are executed and relevant–metadata is sent to the Transaction Layer. Transaction Layer: On this layer, secure transaction metadata is stored. The primary object of the Transaction Layer is a distributed hash table hosted by all nodes–acting as short term memory for the system. Metadata is sent to the Probabilistic Proof Layer for validation. Probabilistic Proof Layer: This layer incentivizes productive behavior and discourages malicious activity through cryptographic, diagnostic, game-theoretic, and economic mechanisms. Economic Layer: This is the path for payment, staking/slashing, security, governance, and public good funding mechanisms.
Inference Engine Architecture
The Inference Engine is open-source software for deploying and serving AI/ML models as serverless microservices via a common API hosted by a Gateway Node (Control Layer) across a peer-to-peer GPU network (Service Layer).
The Agent is the actor making a standard API call for an inference task to a Gateway Node.
The Gateway Node selects one or more GPU nodes to be utilized, sends each node the task requirements, including: the model to use, the specific portions of the model to compute, any user preferences, and inputs.
The GPU nodes each compute their tasks, coordinate with other nodes through a Distributed Hash Table (DHT, defined and detailed below), and return the output to the Gateway Node.
The Gateway Node then compiles the responses and returns an output to the Agent.
The modular design enables a scalable and composable compute standard (without PoIP, this pattern would only work in a trusted compute environment). There are two layers of peer to peer relationships:
Gateway Node : GPU Node
Agent : Gateway Node
The role definitions and separation of interests in the Inference Engine enable the key functionality of the Probabilistic Proof Layer.
Proof of Inference Protocol
Transaction Layer
The modular design of the Inference Engine leads to emergent complexity. A single inference job on a Large Language Model (LLM) could be computed across many GPUs which can each be associated with a separate wallet. Using a blockchain to track every microtransaction in the process is—currently—prohibitively slow and expensive. Instead of a blockchain, the Transaction Layer uses a distributed hash table (DHT) to synchronize transaction data across nodes.
Every node in the system hosts the DHT, which acts as a handshake for establishing shared truth on a per transaction basis. If data from one or more nodes deviate, it is an immediate sign that at least one node is non-compliant and the transaction will be sent to the Probabilistic Proof Layer.
The DHT has the added benefit of giving each party verification that their data is in sync with the others.
In addition to every node in the transaction hosting a DHT with data related to the transaction, Validator Nodes—whose primary role is to secure the Economic Layer—can also host DHTs of transaction data to act as a third party whistleblower for identifying non-compliant nodes.
Proving Transaction Input Across All Nodes
Using the following steps, Merkle tree validation in the protocol can ensure that the correct data was sent to each node in the system.
Input Preprocessing: The Gateway Node splits the inference task into subtasks for each GPU. To create an immutable fingerprint, a Merkle tree is constructed: each subtask's input is hashed with a cryptographic function. Hashes are paired and re-hashed, creating higher-level hashes. This continues until a single root hash representing the entire input data emerges.
Distribution and Verification: The Gateway Node sends subtasks and their corresponding hashes from the Merkle tree to the GPUs and the DHT. Each GPU computes its subtask using the provided input. It returns the result along with the original hash.
Merkle Tree Reconstruction: The Gateway Node collects results and hashes from all GPUs and verifies against the DHT. It reconstructs the Merkle tree using the received hashes. It compares the newly constructed root hash with the original root hash.
Integrity Validation: Matching roots confirm the input data's integrity, ensuring it reaches the GPUs unaltered. Mismatched roots indicate potential corruption, triggering investigation.
The Merkle tree structure efficiently handles large datasets and multiple subtasks, and only a few hashes are needed for verification, regardless of data size. Any modification to the input data is detected and properly routed. Additionally, it can be combined with digital signatures for enhanced security.
Transaction Flagging and Data Storage
Any node can flag a transaction as potentially non-compliant—this can happen automatically through a failed test in the Probabilistic Proof Layer, or it can happen manually. Flagging a transaction routes the transaction to the Economic Layer for deliberation. Additionally, a flagged transaction triggers copying hashed transaction data from the DHT to long-term storage on-chain for use in the Economic Layer.
Probabilistic Proof Layer
6079 PoIP employs multiple layers of cryptoeconomic security designed to increase the Cost-of-Corruption (CoC) and decrease the Profit-from-Corruption (PfC). Each test is modular in design and open source—allowing contributors to adapt and improve the system in order to increase their economic rewards. Gateway Nodes compete in an open marketplace for compute—agents are incentivized to select Gateway Nodes whose reputation, capabilities, and price demonstrate maximum value for their interest. Similarly, GPU Nodes compete in an open marketplace to be selected by Gateway Nodes. The feedback loops of economic policy between nodes incentivizes behavior that leads to an increase in Agents selecting a specific Gateway, and for each Gateway to select a GPU. This layer provides common information across the network that enables each node to make independent economic decisions in a competitive environment.
Onboarding Diagnostics and Performance Tests: When a Gateway Node engages a GPU Node, diagnostic tests are conducted to evaluate the hardware capabilities of the GPUs including network speed, benchmarking workloads, and performance benchmarks against claimed hardware specs. This metadata is used by the Gateway Node to coordinate load balancing and swarm optimization. By default, failed GPU diagnostics cycle a GPU out of the Gateway Nodes’ active network.
At this stage, Gateway Nodes are able to identify new nodes by an Identity Token. This enables a node operator to claim many nodes and configure settings for their distributed infrastructure.
Reputation Building on Network: Canary networks function as isolated testbeds, offering a secure environment to evaluate novel features and functionalities prior to their deployment on the mainnet. This controlled setting mirrors the historical practice of employing canaries in coal mines to detect hazardous gas concentrations, safeguarding the primary system from unforeseen risks.
In the specific context of user onboarding and reputation score establishment, the canary network is a distinct blockchain, equipped with its own token economy that is focused on identity-based reputation—for this paper we are calling it the “Test Network.” This segregation shields the mainnet from potential vulnerabilities associated with untested nodes and features.
****New users are initially introduced to the canary network instead of directly entering the mainnet. This staged approach allows them to acclimate to the system and contribute to the nascent reputation ecosystem within the canary network.
The Test Network fosters its own reputation scoring system that evaluates users based on their interactions and activities. Upon transitioning to the mainnet, this established score can be ported or serve as a reference point.
New users gain valuable experience and build their reputation within the Test Network before integrating into the main ecosystem. A well-functioning canary network fosters trust in the integrity and robustness of the mainnet's reputation system. Additionally, developers can harness user behavior data from the Test Network to refine the reputation scoring system prior to mainnet deployment.
Market-Based Reputation Building: Successful inference tasks build the reputation of nodes through a token on the Economic Layer. The more tasks that accumulate, the more attractive the node becomes for selection, and the more non-compliant behavior is disincentivized. When a transaction is flagged as potentially non-compliant, it is routed to the Economic Layer where reputation and stake can be slashed.
Staking: Gateway and GPU Nodes may stake tokens to demonstrate to Agents their level of “skin in the game” and signal to the network that they have a higher CoC and a lower PfC. The competitive nature of this system enables the market to direct the proper stake required to reach a local equilibrium.
Competitive Game Theoretic Mechanisms: The protocol incentivizes cooperation and discourages cheating through a combination of economic rewards and penalties. The initial game theoretic mechanism is the architecture of the PoIP itself with nodes/agents cooperating and competing in a complex adaptive system of well-defined signals and boundaries, leading to constant evolutionary pressure toward fitness.
Agent nodes seek compliant inference returned at a competitive price.
Nodes are incentivized to maximize individual return on investment and thus nodes compete against one another to set effective policy and pricing. All Agent behavior in this system is a signal for nodes to maximize return on investment for nodes.
The open data of supply and demand across the network determines competitive and transparent market-based pricing and reputation/staking requirements.
Simulating Non-compliant Transaction Behavior and Rewarding Effective Identification: Nodes in the network generally have agency in choosing which other nodes they interact with. In order to be fully PoIP compliant and to receive staking and mining rewards from the 6079 protocol, all active Gateway Node and GPU nodes must connect with a DHT hosted by all Gateway Nodes on the network and respond to any request within a predefined SLA (service level agreement).
The settlement contract initiates simulated transactions through the root node which acts as a mining “pulse” of the system. Gateway Nodes receive real transaction requests from the system, and the authorized tokens are a bounty that, when verified by PoIP, are “mined” by the network and allocated to the appropriate nodes when compliant transactions are settled.
Economic Layer
Figure 3: The Economic Layer.
This layer is the substrate of the network, grounding transaction and reputation data in trustless on-chain record of truth by employing an application-specific layer on the Ethereum blockchain. The Economic Layer broadcasts a signal to the network when an Agent has authorized tokens, and the DHT keeps track of the data associated with the transaction and enables a swarm to keep track of the requests from the Agent and the ledger of how tokens transact across nodes.
On a regular rhythm, the settlement contract runs and settled transactions are committed on-chain. 6079 Ethereum ERC-20 Token: This token is optimized for low gas fees, secures the 6079 Proof of Inference Protocol (PoIP), and serves as a governance vote. 6079 On-chain Settlement Contract: This contract collects and posts transaction metadata, authorizes funds, verifies transactions, and enforces staking/slashing mechanisms.
Agents sign to authorize a number of tokens that can be used to initiate transactions. Under the settlement contract, holds are authorized so that they cannot be transferred between initiation and settling.
The settlement contract executes on a regular basis to settle transactions, initiate compliance disputes, and govern staking and slashing behavior. Transactions are batched in order to keep transaction costs minimized.
Staking and Slashing: All nodes can stake tokens to demonstrate an increased CoC for that node to the network, and decrease PfC network-wide. We take a market-based economic approach to staking. Gateway and GPU nodes choose the amount of tokens staked as a way to signal reputation to the network. Compliant inference jobs deliver tokens on both the Test Network and the 6079 chain so that a node can demonstrate reputation without having to pay a fee to any centralized authority.
When transactions are routed through the Probabilistic Proof Layer, there are two vectors of attack to identify. First is a Gateway Node that selects GPU nodes that behave in a non-compliant manner. Merkle tree validation allows us to prove inputs. The Gateway Node is responsible for selecting compliant GPU nodes.
When a transaction is flagged as non-compliant the system will increase the number of simulated transactions required by all Gateway Nodes, which increases the CoC.
A ledger of transactions maintains lists of flagged non-compliant nodes along with hashed metadata. Validators are able to propose and vote on slashing of stakes.
Identity Tokens: These tokens enable user preferences, ownership verification, and network-wide reputation building. A single node operator can configure the nodes and agents they operate using an Identity Token to prove ownership of nodes. The operator may also set policy, direct economic transactions and build reputation across nodes.
Identity tokens are a primitive for enabling composability of decentralized physical infrastructure networks, which enables operators to build decentralized tools on top of the 6079 PoIP and the Inference Engine. Gateway Nodes are Validators: Gateway Nodes play a role in the Economic Layer as validators of the 6079 blockchain through the Ethereum Proof of Stake consensus mechanism. Additionally, Gateway Nodes play a critical role in hosting nodes of DHTs used across many transactions. Validators enable consensus across long term memory (on-chain) and short term memory (DHT). These services secure the network and are valuable. Thus, staking rewards are earned by Gateway Nodes/Validators.
Governance: All holders and stakers of the token are eligible to cast votes on primary network parameters such as staking thresholds for gateways and GPUs, minting rate of new tokens, allocation of token treasury.
Transaction Layer: DHT synchronization establishes a baseline of data consistency, making corrupted inputs detectable.
Probabilistic Proof Layer: Compliance tests raise the CoC, as detection of cheating triggers Economic Layer action. Reputation systems further increase CoC; bad behavior has lasting consequences.
Economic Layer: Staking means non-compliant nodes lose capital (high CoC). Slashing mechanisms directly punish corruption.
Conclusion
6079 Proof of Inference Protocol (PoIP) is an enabling technology that allows protocols, AI agents, and applications to run inference on large language models and other AI/ML models. We envision an open distributed network forming around the periphery of the 6079 chain that leverages PoIP, extending the benefits of decentralized AI to a broader audience and fostering a new era of innovation and accessibility.
This network, built to be a positive-sum economy under the principles of transparency, security, and win-win collaboration, aims to transform the AI landscape. By distributing the computational load across a global network of nodes, 6079 PoIP not only democratizes access to cutting-edge AI capabilities but also ensures that these technologies are used ethically and responsibly. The 6079 protocol, underpinned by PoIP, stands as a testament to the power of collective effort and open-source philosophy, offering a scalable and sustainable model for AI inference that respects user privacy and data sovereignty.
As we move forward, the 6079 protocol aims to catalyze the growth of decentralized AI applications, empowering developers, businesses, and individuals to harness the full potential of AI technologies without the barriers imposed by centralized infrastructures. We invite the global community—developers, researchers, entrepreneurs, and enthusiasts—to join us in this endeavor. Together, we can build a decentralized AI ecosystem that is more inclusive, efficient, and aligned with the needs of society at large.
The journey ahead is filled with challenges and opportunities. Yet, with the foundational technology of 6079 PoIP and the collective ambition of the 6079 community, we are poised to redefine the boundaries of what is possible in the realm of AI. Our commitment to openness, innovation, and community will guide us as we explore new frontiers in AI and blockchain integration.
In conclusion, the 6079 Proof of Inference Protocol (PoIP) is not just a technological innovation; it is a vision for a future where AI is accessible to all and capturable by none, empowering humanity with additional tools to help solve our most pressing challenges.
Last updated
