TallyBox Tutorial - P2P Network Flow

P2P Networking in TallyBox DAG

This tutorial guides developers through the peer-to-peer (P2P) networking flow in a TallyBox application, which operates on a Directed Acyclic Graph (DAG) structure for scalable and efficient transaction processing. The tutorial emphasizes the sender's role in recomposing transactions from the database into XML messages using the main_recomposer function (with supporting functions transaction_recomposer, group_transaction_recomposer, and minting_transaction_recomposer), followed by the receiver's role in decomposing incoming XML messages using transaction_decomposer.

Unlike most blockchain systems that rely on bidirectional synchronization, where peers mutually exchange and reconcile transaction data to maintain a consistent ledger, TallyBox employs unidirectional synchronization. In this approach, the sender peer pushes transaction data to the receiver, which processes and validates it, returning a SHA256 hash to confirm successful integration into the DAG without sending back its own transaction data. This unidirectional flow enhances efficiency in high-throughput scenarios by reducing network overhead and simplifying conflict resolution.

This tutorial covers data flow, messaging formats, and the hash-based synchronization mechanism where the receiver generates a SHA256 hash that the sender verifies to confirm synchronization before sending subsequent records. The implementation is in Python, using a SQL Server database for ledger storage and ECDSA (secp256r1) for cryptographic validation.

Step 1: Overview of P2P Networking in TallyBox DAG

TallyBox operates as a decentralized P2P network where nodes (peers) exchange transaction records using a DAG structure, allowing transactions to reference multiple previous transactions for scalability. The sender peer uses main_recomposer to construct XML messages in the sync_transaction format from database records and broadcasts them to other peers. The receiver peer processes these messages with transaction_decomposer, validates them, stores them in the database, and generates a final SHA256 hash. This hash is sent back to the sender, who verifies it to ensure successful processing before sending the next record. The system supports three transaction types:

• Single Transactions: Transfers between two wallets with a numeric order_id.
• Group Transactions: Transfers to multiple recipients, identified by order_id starting with '#', with a hashed wallet_to field.
• Minting Transactions: Treasury-based transactions, identified by order_id starting with '$'.

FUNCTION sender_broadcast_and_sync(tnx_id, the_tnx_md5):
    xml_string = main_recomposer(tnx_id, the_tnx_md5)
    IF xml_string IS NOT NULL THEN
        log_message("Broadcasting transaction: " + tnx_id)
        broadcast_to_peers(xml_string)
        RETURN xml_string
    ELSE
        log_message("Failed to recompose transaction")
        RETURN NULL
    END IF

FUNCTION receiver_process(xml_string):
    sha256_hash = transaction_decomposer(xml_string)
    IF sha256_hash IS NOT NULL THEN
        log_message("Processed transaction, hash: " + sha256_hash)
        send_to_sender(sha256_hash)  // Notify sender for synchronization
        broadcast_to_peers(xml_string)
        RETURN sha256_hash
    ELSE
        log_message("Failed to process transaction")
        RETURN NULL
    END IF
                

References:
Directed Acyclic Graph (Wikipedia)
PyODBC (Microsoft Docs)

Step 2: Sender - Transaction Recomposition

The sender peer uses the main_recomposer function to retrieve transaction data from the database and construct XML messages in the sync_transaction format for broadcasting. It delegates to one of three functions based on the order_id prefix:

• transaction_recomposer: For single transactions (numeric order_id).
• group_transaction_recomposer: For group transactions (order_id starts with '#').
• minting_transaction_recomposer: For minting transactions (order_id starts with '$').

FUNCTION main_recomposer(tnx_id, the_tnx_md5):
    order_id = sql_find_record("tbl_tallybox_sign", "order_id", "tnx_id", tnx_id, "the_tnx_md5", the_tnx_md5)
    IF order_id == "no_record" THEN
        log_error("Transaction not found")
        RETURN NULL
    END IF
    IF order_id.starts_with("$") THEN
        RETURN minting_transaction_recomposer(tnx_id, the_tnx_md5)
    ELSE IF order_id.starts_with("#") THEN
        RETURN group_transaction_recomposer(tnx_id, the_tnx_md5)
    ELSE
        RETURN transaction_recomposer(tnx_id, the_tnx_md5)
    END IF

FUNCTION transaction_recomposer(tnx_id, the_tnx_md5):
    sign_data = sql_query("tbl_tallybox_sign", tnx_id, the_tnx_md5)
    sender_data = sql_query("tbl_tallybox_book", tnx_type="1", tnx_id)
    wallet_from = sql_query("tbl_tallybox_wallet", sender_data.wallet_id)
    public_key = sql_query("tbl_tallybox_wallet_pubkey", sender_data.wallet_id)
    order_currency = sql_query("tbl_system_currency", sender_data.currency_id)
    graph_from = sql_query("tbl_system_graph", sender_data.graph_id)
    recipient_data = sql_query("tbl_tallybox_book", tnx_type="2", tnx_id)
    wallet_to = sql_query("tbl_tallybox_wallet", recipient_data.wallet_id)
    graph_to = sql_query("tbl_system_graph", recipient_data.graph_id)
    order_csv = join_fields(["tallybox", "parcel_of_transaction", "graph_from", graph_from, ..., "publicKey_xy_compressed", public_key], "~")
    xml = create_xml(order_csv, "", tnx_id, the_tnx_md5)
    RETURN xml
                

References:
MiniDOM XML (Python Docs)
XML (Wikipedia)

Step 3: Sender - Broadcasting Transactions

After recomposing the transaction into an XML message, the sender peer broadcasts it to other peers in the TallyBox network. The process involves:

• Retrieve tnx_id and the_tnx_md5 from tbl_tallybox_sign to identify transactions for broadcasting.
• Use main_recomposer to generate the XML message.
• Send the XML to connected peers via a socket or other P2P protocol, awaiting a SHA256 hash from each receiver to confirm successful processing.

FUNCTION broadcast_transactions():
    transactions = sql_query("tbl_tallybox_sign", "tnx_id, the_tnx_md5")
    FOR tnx_id, the_tnx_md5 IN transactions:
        xml_string = main_recomposer(tnx_id, the_tnx_md5)
        IF xml_string IS NOT NULL THEN
            broadcast_to_peers(xml_string)
            expected_hash = calculate_expected_hash(tnx_id)
            received_hash = await_receiver_hash()
            IF received_hash == expected_hash THEN
                log_message("Synchronization confirmed, sending next record")
                continue
            ELSE
                log_error("Synchronization failed, retrying")
                request_resend(xml_string)
            END IF
        END IF
    END FOR
                

References:
Peer-to-Peer Networking (Wikipedia)
Socket Programming (Python Docs)

Step 4: Receiver - Parsing XML Messages

The receiver peer processes incoming XML messages using the transaction_decomposer function, extracting fields from the sync_transaction structure. The process involves:

• Parse the XML to extract order_csv, order_csv_multiple, enforce_tnx_id, and checksum.
• Split order_csv by '~' to obtain fields like graph_from, wallet_from, order_currency, and the_sign.
• For group transactions, parse order_csv_multiple to extract multiple wallet_to and order_amount pairs.
• Extract tree_id and branch_id from enforce_tnx_id (format: tree_id.branch_id_reverse).

FUNCTION transaction_decomposer(xml_string):
    TRY:
        root = parse_xml(xml_string)
        order_csv = root.find("order_csv").text
        order_csv_multiple = root.find("order_csv_multiple").text OR ""
        enforce_tnx_id = root.find("enforce_tnx_id").text
        checksum = root.find("checksum").text
        csv_parts = split(order_csv, "~")
        is_group = csv_parts[2] == "number_of_transactions"
        offset = 2 IF is_group ELSE 0
        graph_from = csv_parts[3 + offset]
        graph_to = csv_parts[5 + offset]
        wallet_from = csv_parts[7 + offset]
        wallet_to = csv_parts[9 + offset]
        order_currency = csv_parts[11 + offset]
        order_amount = Decimal(csv_parts[13 + offset])
        order_id = csv_parts[15 + offset]
        order_utc_unix = csv_parts[17 + offset]
        the_sign = csv_parts[19 + offset]
        public_key = csv_parts[21 + offset]
        IF is_group THEN
            multiple_fields = split(order_csv_multiple, "~")
            wallet_to_arr, order_amount_arr = parse_multiple_fields(multiple_fields)
        ELSE
            wallet_to_arr = [wallet_to]
            order_amount_arr = [order_amount]
        END IF
        tree_id, branch_id_reverse = split(enforce_tnx_id, ".")
        branch_id = reverse(branch_id_reverse)
        RETURN parsed_data
    CATCH Exception e:
        log_error("XML parsing error: " + e)
        RETURN NULL
END FUNCTION
                

References:
ElementTree XML API (Python Docs)
XML (Wikipedia)

Step 5: Receiver - Validation and Hash Generation

The receiver peer validates the parsed transaction and generates a final SHA256 hash for synchronization. The validation and hash generation process includes:

• Format Validation: Ensure graph_from and graph_to contain dots, wallets start with 'tjb' and have a valid MD5 checksum, and amounts/timestamps are numeric.
• Database Validation: Verify graph_from, graph_to, and order_currency exist in tbl_system_graph and tbl_system_currency.
• Treasury Transactions ($): Validate treasury ID, check for double-spending, and ensure currency, amount, and wallet match treasury records.
• Group Transactions (#): Verify the SHA256 hash of wallet_to and order_amount pairs, check for duplicates, and ensure total amount matches.
• Balance Checks: For non-minting transactions, ensure sufficient IRR for fees (2 * 250.0 + num_tnxs * 250.0) and sufficient currency balance for the sender.
• Signature Verification: Use ECDSA (secp256r1) to verify the signature against the SHA256 digest of the order string.
• Hash Generation: Generate ledger entries in tbl_tallybox_book (fee, sender, recipient(s)) with tally_hash fields linking to previous transactions. Concatenate these hashes (e.g., tally_hash_fee~tally_hash_from~total_tally_hash_to for non-minting, or total_tally_hash_to for minting), compute a SHA256 hash, and verify its MD5 against the input checksum. Send the SHA256 hash to the sender for synchronization.

FUNCTION validate_and_generate_hash(parsed_data):
    IF NOT validate_formats(parsed_data) THEN
        log_error("Invalid format")
        RETURN NULL
    END IF
    IF NOT validate_database_records(parsed_data) THEN
        log_error("Invalid database records")
        RETURN NULL
    END IF
    IF parsed_data.order_id.starts_with("$") THEN
        IF NOT validate_treasury(parsed_data) THEN
            log_error("Treasury validation failed")
            RETURN NULL
        END IF
    ELSE IF parsed_data.order_id.starts_with("#") THEN
        IF NOT validate_group_transaction(parsed_data) THEN
            log_error("Group transaction validation failed")
            RETURN NULL
        END IF
    END IF
    IF NOT parsed_data.order_id.starts_with("$") THEN
        IF NOT validate_balances(parsed_data) THEN
            log_error("Insufficient balance")
            RETURN NULL
        END IF
    END IF
    order_string = join_fields(parsed_data, ["graph_from", "graph_to", "wallet_from", "wallet_to", "order_currency", "order_amount", "order_id", "order_utc_unix"])
    digest = hash_sha256(order_string)
    IF NOT verify_digest(digest, parsed_data.the_sign, parsed_data.public_key) THEN
        log_error("Signature verification failed")
        RETURN NULL
    END IF
    ledger_entries = generate_ledger_entries(parsed_data)
    IF NOT parsed_data.order_id.starts_with("$") THEN
        total_tally_hash = ledger_entries.tally_hash_fee + "~" + ledger_entries.tally_hash_from + "~" + join(ledger_entries.tally_hash_to_arr, "~")
    ELSE
        total_tally_hash = join(ledger_entries.tally_hash_to_arr, "~")
    END IF
    sha256_output = hash_sha256(total_tally_hash)
    computed_tnx_md5 = hash_md5(sha256_output)
    IF computed_tnx_md5 != parsed_data.checksum THEN
        log_error("Checksum mismatch")
        RETURN NULL
    END IF
    store_ledger_entries(ledger_entries)
    store_signature(parsed_data, computed_tnx_md5)
    RETURN sha256_output
END FUNCTION
                

References:
ECDSA (Wikipedia)
SHA-256 (Wikipedia)

Step 6: Receiver - Database Storage

After validation, the receiver peer stores the transaction data in the SQL Server database, maintaining the DAG structure. The process involves:

• Register wallet_from and wallet_to in tbl_tallybox_wallet and public_key in tbl_tallybox_wallet_pubkey.
• Store fee record (tnx_type='0') in tbl_tallybox_book for non-minting transactions, with a tally_hash linking to the previous fee record.
• Store sender record (tnx_type='1') and recipient record(s) (tnx_type='2' or higher for group transactions) in tbl_tallybox_book, each with a tally_hash linking to the previous record for that wallet and currency.
• Store the signature in tbl_tallybox_sign with the transaction’s MD5 checksum.
• The DAG structure is maintained by tally_hash_dag fields, which reference previous transactions, forming a graph of dependencies.

FUNCTION store_transaction(parsed_data):
    session_wallet_id = sql_max_field_id("tbl_tallybox_wallet", "wallet_id") + 1
    wallet_from_id = register_wallet(parsed_data.wallet_from, session_wallet_id)
    register_public_key(parsed_data.public_key, wallet_from_id)
    wallet_to_id_arr = []
    FOR wallet_to IN parsed_data.wallet_to_arr:
        wallet_to_id = register_wallet(wallet_to, session_wallet_id)
        wallet_to_id_arr.append(wallet_to_id)
    END FOR
    ledger_entries = []
    IF NOT parsed_data.order_id.starts_with("$") THEN
        fee_entry = create_fee_entry(parsed_data, wallet_from_id)
        ledger_entries.append(fee_entry)
    END IF
    from_entry = create_sender_entry(parsed_data, wallet_from_id)
    ledger_entries.append(from_entry)
    FOR i = 0 TO len(parsed_data.wallet_to_arr) - 1:
        to_entry = create_recipient_entry(parsed_data, wallet_to_id_arr[i], i)
        ledger_entries.append(to_entry)
    END FOR
    FOR entry IN ledger_entries:
        sql_insert("tbl_tallybox_book", entry)
    END FOR
    signature_entry = create_signature_entry(parsed_data, computed_tnx_md5)
    sql_insert("tbl_tallybox_sign", signature_entry)
END FUNCTION
                

References:
SQL INSERT Statement (Microsoft Docs)
DAG Structure (Wikipedia)

Step 7: Synchronization via Hash Confirmation

The TallyBox P2P network relies on a hash-based synchronization mechanism to ensure reliable data exchange. After the receiver processes a transaction with transaction_decomposer, it generates a final SHA256 hash from the concatenated tally_hash fields of ledger entries. This hash is sent back to the sender peer, which compares it to its expected hash to confirm successful processing. Only then does the sender proceed to send the next transaction record. The process ensures that the DAG remains consistent across peers, as each transaction’s hash depends on previous transactions in the graph. The data flow is:

• Sender Peer: Reconstructs XML with main_recomposer, broadcasts to receiver, and awaits SHA256 hash.
• Receiver Peer: Processes XML, stores in database, generates SHA256 hash of tally_hash_fee~tally_hash_from~total_tally_hash_to (or total_tally_hash_to for minting), and sends it to the sender.
• Sender Peer: Verifies the received hash matches its expected value, confirming synchronization, and sends the next record.
• Receiver Peer: Broadcasts the XML to other peers, who repeat the process.

FUNCTION synchronize_transaction(xml_string):
    sha256_hash = transaction_decomposer(xml_string)
    IF sha256_hash IS NOT NULL THEN
        send_to_sender(sha256_hash)
        broadcast_to_peers(xml_string)
        RETURN true
    ELSE
        send_to_sender(NULL)
        RETURN false
    END IF

FUNCTION sender_verify_hash(received_hash, expected_hash):
    IF received_hash == expected_hash THEN
        log_message("Synchronization confirmed, sending next record")
        send_next_record()
    ELSE
        log_error("Synchronization failed, hash mismatch")
        request_resend()
    END IF
                

References:
SHA-256 Algorithm (Wikipedia)
Socket Programming (Python Docs)

Acknowledgments

Special thanks to Grok, created by xAI, for its invaluable assistance in creating this TallyBox P2P DAG networking tutorial.