zkTLS Technology: A New Direction to Break the Data Silos in Web3

Recently, I have been exploring new project directions. During the product design process, I came across a previously uncharted technical field - zkTLS. After in-depth research and study, I would like to share my insights with everyone.

zkTLS is a new technology that combines Zero-Knowledge Proofs ( ZKP ) and the Transport Layer Security protocol ( TLS ). In the Web3 space, it is mainly used in on-chain virtual machine environments, allowing the verification of the authenticity of off-chain HTTPS data without trusting a third party. Here, authenticity includes three aspects: the data source indeed comes from a certain HTTPS resource, the returned data has not been tampered with, and the timeliness of the data is guaranteed. Through this cryptographic implementation mechanism, on-chain smart contracts gain the ability to reliably access off-chain Web2 HTTPS resources, thereby breaking data silos.

Introduction to TLS Protocol

To gain a deeper understanding of the value of zkTLS technology, it is necessary to provide a brief overview of the TLS protocol. TLS is used to provide encryption, authentication, and data integrity in network communications, ensuring the security of data transmission between clients like browsers ( and servers like websites ).

Non-network developers may notice that some website domain names are prefixed with https, while others are prefixed with http. When accessing the latter, mainstream browsers typically warn of insecurity. When accessing the former, you may encounter messages such as "Your connection is not private" or HTTPS certificate errors. These messages are related to the availability of the TLS protocol.

The HTTPS protocol is actually based on the HTTP protocol and uses the TLS protocol to ensure the privacy and integrity of information transmission, as well as to make the authenticity of the server verifiable. The HTTP protocol is a plaintext transmission network protocol and cannot verify the authenticity of the server, which can lead to several security issues:

1. The information transmitted to the server may be monitored by third parties, leading to privacy leaks.
2. Unable to verify the authenticity of the server side, requests may be hijacked by malicious nodes and return malicious information.
3. Unable to verify the integrity of the returned information, which may cause data loss due to network issues.

The TLS protocol was designed to address these issues. It is worth mentioning that the TLS protocol is developed based on SSL version 3.1, and the two can be used interchangeably in certain contexts.

The main idea of the TLS protocol to solve the above problems is:

1. Encrypted communication: Use symmetric encryption ( such as AES, ChaCha20) to protect data and prevent eavesdropping.
2. Identity Authentication: Verify server identity through a digital certificate ( issued by a third party to a designated institution, such as an X.509 certificate ), to prevent man-in-the-middle attacks.
3. Data Integrity: Use HMAC( hash-based message authentication code) or AEAD( authenticated encryption) to ensure that the data has not been tampered with.

The HTTPS protocol based on the TLS protocol is divided into two stages during data exchange: the handshake phase and the data transmission phase. The specific process includes four steps:

1. The client sends ClientHello
2. The server sends ServerHello
3. Client-side validation of the server
4. Start encrypted communication

This foundational technology, widely used in Web2 networks, poses challenges for Web3 application development. Especially when on-chain smart contracts need to access off-chain data, the on-chain virtual machine does not open external data call capabilities due to data availability issues, ensuring the traceability of all data and thereby guaranteeing the security of the consensus mechanism.

However, as development progressed, developers found that DApps have a demand for off-chain data. Thus, the oracle (Oracle) projects emerged, such as Chainlink and Pyth. These projects act as a relay bridge between on-chain data and off-chain data, breaking the data island phenomenon. To ensure the availability of the relayed data, these oracles usually adopt a PoS consensus mechanism, which makes the cost of malicious behavior for relay nodes higher than the benefits, economically preventing the provision of incorrect information on-chain.

Advantages of zkTLS

However, there are two issues with the data acquisition solution based on Oracle:

1. High costs: To ensure the authenticity of data transmitted to the chain by the Oracle, a PoS consensus mechanism is required, which incurs high maintenance costs. Typically, Oracle projects only maintain the most mainstream data for free, such as the prices of major assets like BTC. For exclusive requirements, payment is necessary, which hinders application innovation, especially for long-tail and customized needs.

2. Low efficiency: The consensus of the PoS mechanism requires time, leading to data lag on the chain, which is unfavorable for high-frequency access scenarios.

The zkTLS technology was born to solve these problems. By introducing the ZKP (Zero-Knowledge Proof) algorithm, it allows on-chain smart contracts to directly verify that the data provided by a certain node indeed comes from a specific HTTPS resource and has not been tampered with, thereby avoiding the high usage costs caused by consensus algorithms in traditional oracles.

zkTLS uses cryptographic protection to replace the high costs associated with traditional Oracle-based consensus mechanisms for data availability. Specifically, by introducing ZKP (Zero-Knowledge Proof), it calculates and generates Proof from the data obtained from off-chain relay node requests for HTTPS resources, relevant CA certificate verification information, timestamp proofs, and data integrity proofs based on HMAC or AEAD, while maintaining necessary verification information and algorithms on-chain. This allows smart contracts to verify the authenticity, timeliness, and reliability of the data source without exposing key information.

The biggest advantage of this technical solution is the reduction of the cost of achieving availability for Web2 HTTPS resources, which has stimulated many new demands. In particular, it optimizes the technical architecture design of DID and Web3 games by lowering the on-chain price acquisition of long-tail assets and utilizing authoritative Web2 websites for on-chain KYC.

Of course, zkTLS has also brought impacts to existing Web3 enterprises, especially for the current mainstream oracle projects. In response to this impact, industry giants like Chainlink and Pyth are actively following up research in related directions, attempting to maintain their leading position during the technological iteration process, while also giving rise to new business models, such as transitioning from time-based charging to usage-based charging, and Compute as a Service, etc. The main difficulty is similar to most ZK projects, which lies in how to reduce computing costs to make it commercially viable.

In summary, during product design, paying attention to the development trends of zkTLS and appropriately integrating this technology stack may lead to new directions in business innovation and technical architecture.