Alogrand团队Gorbunov等人2020年论文《Pointproofs: Aggregating Proofs for Multiple Vector Commitments》,配套的代码实现参见:https://github.com/algorand/pointproofs
在该论文中,实现了:
Vector commitment可用于减少存储空间:instead of storing a vector of values, one can store only the commitment and receive the values together with their proofs as needed。
Vector commitment可让application 在storage of all value和 bandwidth taken up by revealed values and proofs 之间进行取舍平衡。
为了在减少存储空间的同时尽可能减少带宽,需要 reduce the proof size。但是,由于需要满足cryptographically hard to forge的要求,单个proof的size cannot be reduced too far。改进的方式可为:
Boneh等人2019年论文《Batching techniques for accumulators with applications to IOPs and stateless blockchains》可实现dynamic aggregation for proofs in a single (the same) commitment —— aggregate proofs for elements of a vector into a single proof for a subvector。(参见博客 密码学累加器cryptographic accumulator)
而在本论文中,Gorbunov等人实现了跨多个commitments的aggregate proofs。
具体的各方案对比为:
在本论文中,Gorbunov等人的主要贡献为:
实现了proofs for individual elements of a single vector commitment can be aggregated by any third party into a single proof for the subvector;
实现了proofs for subvectors of multiple commitments can be aggregated by any third party into a single proof for the multiple subvectors。
在实现aggregation的同时,也提供了hiding属性。
在Libert和Yung 2010年论文《Concise mercurial vector commitments and independent zero-knowledge sets with short proofs》构建的vector commitment基础上,增加了same-commitment aggregation和cross-commitment aggregation,从而实现了Pointproofs。
1)与此类似,Pointproofs的public parameter size is linear in the size of the committed vector(可将long vector切分为多个短的vectors,多个短的vectors的proofs可以aggregate,但是commitments不能aggregate,从而缩短了public parameter size,但是增加了total size of the commitments);
2)与此类似,Pointproofs也基于 q q q-type assumption。In order to prove security of aggregation, we have to work in the algebraic group model and the random oracle model. We can reduce these assumptions by lowering efficiency and/or security requirements.
Pointproofs生成的proof为single point on a pairing-friendly curve (48 bytes at 128-bit security),无论是single value for a single commitment,subvector of values for a single commitment,还是set of subvectors for multiple commitments。
Pointproofs中实现了支持aggregation的hiding属性,仅需增加an additional exponentiation,commitment size和proof size均无需增加。而Dario Catalano 和 Dario Fiore 2013年论文《Vector Commitments and their Applications》中提到的给Vector commitment加hiding属性的方法为:add an inner layer of hiding commitments to individual values —— first commit to each message separately using a standard commitment scheme, then apply the VC to the obtained sequence of commitments。但是该方式无法automatically extend to aggregatable vector commitments,因为proofs for the inner layer are not automatically aggregatable。
Pointproofs可用于reduce storage requirements for blockchains。主要针对smart contract智能合约场景。假设一个智能合约有多个变量,所有变量当前值 ( m 1 , ⋯ , m N ) (m_1,\cdots, m_N) (m1,⋯,mN) are committed to a single vector commitment C C C,每个智能合约有一个commitment。
为了与智能合约交互,one provides a 480byte proof $ \hat{\pi}$ of the current values of the variables needed for the transaction,transaction成功执行后可能会更新这些变量值。当存在多个智能合约时,cross-commitment aggregation允许compress multiple proofs π ^ 1 , ⋯ , π ^ l \hat{\pi}_1,\cdots,\hat{\pi}_l π^1,⋯,π^l into a single 48-byte proof π \pi π。从而可从根本上消除the bandwidth overhead due to proofs in a proposed block。
将Pointproofs用于智能合约存储时,针对 1 0 8 10^8 108千万级accounts,可将validators’ storage requirements降为4.5GB,assuming one open value per transaction 的情况下,仅需增加31KB per block overhead for 1000 transactions。
Pointproofs 代码实现https://github.com/algorand/pointproofs 中的实际性能表现为:针对a commitment for 1000 variables of a smart contract at 128-bit security level,生成任意subvector proof的时间为54~123ms;a block proposer对所有proofs进行cross-commitment aggregate的时间约为0.07ms per proof;存储了commitments 的 validator verify the aggregated proofs in a block的时间约为 0.7~1.9ms per value verified;为表示变量值的更新(通过交易执行),需要update commitments的时间约为0.2~0.3ms per variable updated。
cross-commitment aggregation of proofs可用于很多场景,如:
vector commitment的相关工作:
The Algebraic Group Model(AGM) :即adversary输出的group element值应基于其收到的group element进行有效的group operation计算得出的,而不是随意创造的。
Suppose adversary is given group elements X 1 , ⋯ , X N ∈ G 1 X_1,\cdots,X_N\in\mathbb{G}_1 X1,⋯,XN∈G1. Then, for every group element Z ∈ G 1 Z\in\mathbb{G}_1 Z∈G1 that the adversary outputs, it must also ouput z 1 , ⋯ , z N ∈ Z p z_1,\cdots,z_N\in\mathbb{Z}_p z1,⋯,zN∈Zp such that Z = ∏ i = 1 N X i z i Z=\prod_{i=1}^{N}X_i^{z_i} Z=∏i=1NXizi.
security assumption:在bilinear pairing group中求解 l l l-wBDHE(weak bilinear Diffie-Hellman exponent problem)很难,即对任意的 α ← Z p \alpha\leftarrow \mathbb{Z}_p α←Zp已知 g 1 α , g 1 ( α 2 ) , ⋯ , g 1 ( α l ) , g 1 ( α l + 2 ) , ⋯ , g 1 ( α 3 l ) , g 2 α , g 2 ( α 2 ) , ⋯ , g 2 ( α l ) g_1^{\alpha},g_1^{(\alpha^2)},\cdots,g_1^{(\alpha^l)},g_1^{(\alpha^{l+2})},\cdots,g_1^{(\alpha^{3l})},g_2^{\alpha},g_2^{(\alpha^2)},\cdots,g_2^{(\alpha^l)} g1α,g1(α2),⋯,g1(αl),g1(αl+2),⋯,g1(α3l),g2α,g2(α2),⋯,g2(αl)
求解 g 1 ( α l + 1 ) g_1^{(\alpha^{l+1})} g1(αl+1)很难。
对于BLS12-381 pairing-friendly curve with l = 32 l=32 l=32,当前best attack has complexity 2 112 2^{112} 2112。
The Random Oracle Model(ROM):本文的security proofs are in the random oracle model。在本文model a cryptographic hash function as a truly random function, accessible to all parties only via oracle queries。本文使用了两个random oracles H H H和 H ′ H' H′,both with output space Z p \mathbb{Z}_p Zp。
采用与Libert和Yung 2010年论文《Concise mercurial vector commitments and independent zero-knowledge sets with short proofs》类似的思路,基于非对称bilinear pairing group,相应的实现细节为:
Setup: Let ( G 1 , G 2 , G T ) (\mathbb{G}_1,\mathbb{G}_2,\mathbb{G}_T) (G1,G2,GT) be a group of prime order p p p,along with pairing e : G 1 × G 2 → G T e:\mathbb{G}_1\times\mathbb{G}_2\rightarrow \mathbb{G}_T e:G1×G2→GT and generators g 1 , g 2 , g T = e ( g 1 , g 2 ) g_1,g_2,g_T=e(g_1,g_2) g1,g2,gT=e(g1,g2) for G 1 , G 2 , G T \mathbb{G}_1,\mathbb{G}_2,\mathbb{G}_T G1,G2,GT respectively. Let α ∈ Z p \alpha\in \mathbb{Z}_p α∈Zp be a secret value (known to no one after the initial generation of public parameters). The pulic parameters are given by 2 N − 1 2N-1 2N−1 values in G 1 \mathbb{G}_1 G1, N N N values in G 2 \mathbb{G}_2 G2, and one value in G T \mathbb{G}_T GT(该值易于计算获得,如 g T α N + 1 = e ( g 1 α , g 2 α N ) = e ( g 1 , g 2 ) α N + 1 g_T^{\alpha^{N+1}}=e(g_1^{\alpha},g_2^{\alpha^{N}})=e(g_1,g_2)^{\alpha^{N+1}} gTαN+1=e(g1α,g2αN)=e(g1,g2)αN+1):【注意 g 1 α N + 1 g_1^{\alpha^{N+1}} g1αN+1不应包含在public parameters中,否则Prover可伪造证明。】
g 1 α , ⋯ , g 1 α N , g 1 α N + 2 , ⋯ , g 1 α 2 N ; g 2 α , ⋯ , g 2 α N ; g T α N + 1 g_1^{\alpha},\cdots,g_1^{\alpha^{N}},g_1^{\alpha^{N+2}},\cdots,g_1^{\alpha^{2N}};g_2^{\alpha},\cdots,g_2^{\alpha^N};g_T^{\alpha^{N+1}} g1α,⋯,g1αN,g1αN+2,⋯,g1α2N;g2α,⋯,g2αN;gTαN+1
Commit:对vector m ⃗ = ( m 1 , ⋯ , m N ) ∈ Z p N \vec{m}=(m_1,\cdots,m_N)\in\mathbb{Z}_p^N m=(m1,⋯,mN)∈ZpN,
C = g 1 ∑ i = 1 N m i α i C=g_1^{\sum_{i=1}^{N}m_i\alpha^i} C=g1∑i=1Nmiαi
Prove:reveal m i m_i mi,
π i = g 1 ∑ j ≠ i m j α N + 1 − i + j = ( C / g 1 m i α i ) α N + 1 − i \pi_i=g_1^{\sum_{j\neq i}m_j\alpha^{N+1-i+j}}=(C/g_1^{m_i\alpha^i})^{\alpha^{N+1-i}} πi=g1∑j=imjαN+1−i+j=(C/g1miαi)αN+1−i
Verify:
e ( C , g 2 α N + 1 − i ) = e ( π i , g 2 ) ⋅ g T m i α N + 1 e(C,g_2^{\alpha^{N+1-i}})=e(\pi_i,g_2)\cdot g_T^{m_i\alpha^{N+1}} e(C,g2αN+1−i)=e(πi,g2)⋅gTmiαN+1
为了实现reveal multiple values m i : i ∈ S m_i:i\in S mi:i∈S (其中 S ⊆ [ N ] S\subseteq [N] S⊆[N]) for a single commitment C C C via a very short proof π S \pi_S πS。
思路一:
直接计算 π S = ∏ i ∈ S π i \pi_S=\prod_{i\in S}\pi_i πS=∏i∈Sπi,然后验证 e ( C , ∏ i ∈ S g 2 α N + 1 − i ) = e ( π S , g 2 ) ⋅ g T α N + 1 ∑ i ∈ S m i e(C,\prod_{i\in S}g_2^{\alpha^{N+1-i}})=e(\pi_S,g_2)\cdot g_T^{\alpha^{N+1}\sum_{i\in S}m_i} e(C,∏i∈Sg2αN+1−i)=e(πS,g2)⋅gTαN+1∑i∈Smi。
该方式不安全,若open S = 1 , 2 S={1,2} S=1,2,可commit to ( m 1 , m 2 ) = ( 1 , 3 ) (m_1,m_2)=(1,3) (m1,m2)=(1,3)而open为 ( m 1 , m 2 ) = ( 2 , 2 ) (m_1,m_2)=(2,2) (m1,m2)=(2,2),违反了binding属性(只bound to ∑ i ∈ S m i \sum_{i\in S}m_i ∑i∈Smi,而不是 { m i : i ∈ S } \{m_i:i\in S\} {mi:i∈S}中的每一个值。)。
同时,还需要防止inconsistnent reveals for possibly two different sets,如分别open ( m 1 , m 2 ) (m_1,m_2) (m1,m2)为 ( 1 , 3 ) (1,3) (1,3), ( m 2 , m 3 ) (m_2,m_3) (m2,m3)为 ( 2 , 1 ) (2,1) (2,1)的情况是不允许的。
思路二:实现same-commitment aggregation
在verification方程式中引入“随机”scalars t i t_i ti,
e ( C , ∏ i ∈ S g 2 α N + 1 − i t i ) = e ( π S , g 2 ) ⋅ g T α N + 1 ∑ i ∈ S m i t i e(C,\prod_{i\in S}g_2^{\alpha^{N+1-i}t_i})=e(\pi_S,g_2)\cdot g_T^{\alpha^{N+1}\sum_{i\in S}m_it_i} e(C,∏i∈Sg2αN+1−iti)=e(πS,g2)⋅gTαN+1∑i∈Smiti
aggregated proof π S = ∏ i ∈ S π i t i \pi_S=\prod_{i\in S}\pi_i^{t_i} πS=∏i∈Sπiti
scalars t i t_i ti的值可通过applying a hash function H H H on some carefully chosen inputs depending on C , S , { m i : i ∈ S } C,S,\{m_i:i\in S\} C,S,{mi:i∈S}。类似的思路在Boneh等人2018年论文《Compact multi-signatures for smaller blockchains》的aggregating signatures中有提及。
怎样选择 t i t_i ti来保证binding属性呢?若 t i ← Z p t_i\leftarrow \mathbb{Z}_p ti←Zp为indeed random,则可保证 P r [ ∑ i ∈ S m i t i = ∑ i ∈ S m i ′ t i ′ ] = 1 / p Pr[\sum_{i\in S}m_it_i=\sum_{i\in S}m_i't_i']=1/p Pr[∑i∈Smiti=∑i∈Smi′ti′]=1/p,即对同一位置open为两个不同值的概率可忽略。
可将hash function H H H 看成是a random oracle。同时,还需要restrict the adversary to the so-called algebraic group model,以便可express adversarially generated commitments C C C in terms of public parameters。
思路三:实现cross-commitment aggregation
对多个不同的vector进行commit,第 j j j个vector 可表示为 m ⃗ j = ( m j , 1 , ⋯ , m j , N ) \vec{m}_j=(m_{j,1},\cdots,m_{j,N}) mj=(mj,1,⋯,mj,N),对应的commitment为 C j C_j Cj,对set S j S_j Sj的open proof为 π ^ j \hat{\pi}_j π^j,则满足:
e ( C j , ∏ i ∈ S j g 2 α N + 1 − i t j , i ) = e ( π ^ j , g 2 ) ⋅ g T α N + 1 ∑ i ∈ S j m j , i t j , i e(C_j,\prod_{i\in S_j}g_2^{\alpha^{N+1-i}t_{j,i}})=e(\hat{\pi}_j,g_2)\cdot g_T^{\alpha^{N+1}\sum_{i\in S_j}m_{j,i}t_{j,i}} e(Cj,∏i∈Sjg2αN+1−itj,i)=e(π^j,g2)⋅gTαN+1∑i∈Sjmj,itj,i
若直接将多个vector对应的verification equation都一起相乘,则有:
∏ j e ( C j , ∏ i ∈ S j g 2 α N + 1 − i t j , i ) = e ( ∏ j π ^ j , g 2 ) ⋅ g T α N + 1 ∑ j ∑ i ∈ S j m j , i t j , i \prod_{j}e(C_j,\prod_{i\in S_j}g_2^{\alpha^{N+1-i}t_{j,i}})=e(\prod_{j}\hat{\pi}_j,g_2)\cdot g_T^{\alpha^{N+1}\sum_{j}\sum_{i\in S_j}m_{j,i}t_{j,i}} ∏je(Cj,∏i∈Sjg2αN+1−itj,i)=e(∏jπ^j,g2)⋅gTαN+1∑j∑i∈Sjmj,itj,i
与思路一类似,上述方式是不安全的,需要再引入额外的random scalars t j ′ t_j' tj′,相应的aggregated proof为 π = ∏ j π j ^ t j ′ \pi=\prod_{j}\hat{\pi_j}^{t_j'} π=∏jπj^tj′,对应的verification equation调整为:
∏ j e ( C j , ∏ i ∈ S j g 2 α N + 1 − i t j , i ) t j ′ = e ( π , g 2 ) ⋅ g T α N + 1 ∑ j ∑ i ∈ S j m j , i t j , i t j ′ \prod_{j}e(C_j,\prod_{i\in S_j}g_2^{\alpha^{N+1-i}t_{j,i}})^{t_j'}=e(\pi,g_2)\cdot g_T^{\alpha^{N+1}\sum_{j}\sum_{i\in S_j}m_{j,i}t_{j,i}t_j'} ∏je(Cj,∏i∈Sjg2αN+1−itj,i)tj′=e(π,g2)⋅gTαN+1∑j∑i∈Sjmj,itj,itj′
首先考虑的是aggregation of proofs for a single commitment。
基本的算法包括Setup
, Commit
, UpdateCommit
, Aggregate
, Verify
:
本论文中,为了具有通用性,定义的Verify
算法总是针对的aggregated proof,哪怕仅仅open了1个位置。同时,上述定义是调用多次Prove
算法生成单个位置的proof然后调用一次Aggregated
算法生成aggregated proof,可能存在其它算法可直接生成aggregated proof从而提升performance,但是并不影响定义。上述定义中,若commitment updated了,需调用Prove
重新生成新的proof,可能存在效率更高的updateproof算法直接update existing proof。(如博客 Vector Commitments and their Applications学习笔记 中提到的ProofUpdate
算法)
整个流程应关注如下属性:
上述具体实现中,可从如下维度优化:
Commit
运算需要 n z ( m ⃗ ) nz(\vec{m}) nz(m)次 G 1 \mathbb{G}_1 G1 exponentiation计算,Prove
比Commit
运算少1次。同时,可通过Pippenger等算法,计算products of exponentiations的效率要高于分别计算exponentiations。Prove
生成 π i \pi_i πi再Aggregate
,可将两个算法合并直接生成aggregated proof,效率更高:Aggregate
运算,直接设置 π ^ = π i \hat{\pi}=\pi_i π^=πi。考虑的是aggregation of proofs across l l l commitments。
在Same-commitment aggregation的基础上,增加了两组算法AggregateAcross
和VerifyAcross
:
Aggregate
算法获得),输出为跨commitment的aggregated proof π \pi π。与Same-commitment aggregation类似,也需要满足Correctness of opening属性。
Cross-commitment aggregation的binding属性以实际场景举例:【具体见论文4.4节证明——Proof of binding for cross-commitment aggregation。】
存在两组commitments,第一组(第一次)有 l 0 l^0 l0个不同的commitments,如包含了vectors x ⃗ , z ⃗ \vec{x},\vec{z} x,z;第二组(第二次)有 l 1 l^1 l1个不同的commitments,如包含了vectors y ⃗ , x ⃗ . z ⃗ \vec{y},\vec{x}.\vec{z} y,x.z。
所谓的Cross-commitment aggregation的binding属性是指,若第一次cross-commitment-aggregated proof π 0 \pi^0 π0 和 第二次的 π 1 \pi^1 π1均VerifyAcross
验证成功,以 x ⃗ \vec{x} x为例,第一次对应的commitment为 C j 0 0 C_{j^0}^0 Cj00,open位置集合为 S j 0 0 S_{j^0}^0 Sj00;第二次对用的为 C j 1 1 , S j 1 1 C_{j^1}^1,S_{j^1}^1 Cj11,Sj11,binding属性要求两次open的交集应具有一致性。
若an opening is inconsistent with a same-commitment opening,则将其和其它commitment openings 聚合在一起的cross-commitment aggregated proof应验证失败。
?即cross-commitment aggregation的binding属性要求要强于same-commitment aggregation 中的binding属性。若cross-commitment中的任一commitment的binding属性有问题,则相应的cross-commitment aggregation proof应验证失败。
特殊地,当 l 0 = l 1 = 1 l^0=l^1=1 l0=l1=1时,其实就是same-commitment。
具体的实现可为:
借助2.2节中的优化思路,可做如下调整:
本文考虑的是simulation-based statistical security——存在efficient randomized simulator ( S e t u p ∗ , C o m m i t ∗ , P r o v e ∗ ) (Setup*,Commit*,Prove*) (Setup∗,Commit∗,Prove∗),其中:
其核心思想为以上算法生成的fake proofs应statistically indistinguishable from real ones, even given the commitments, and even if the committed messages are chosen adversarially。
fake commitment和fake proof中除了要open的信息外,应leak no information about other messages。
举例为:
Hiding属性要求基于真实的stateful oracle O − r e a l ( p p ) O-real(pp) O−real(pp)和仿真的stateful oracle O − s i m ( p p , t d ) O-sim(pp,td) O−sim(pp,td)的输出分布的差异性应可忽略。
当进行update操作时,由于 C ′ ← U p d a t e C o m m i t ( C , S , m ⃗ [ S ] , m ⃗ ′ [ S ] ) C'\leftarrow UpdateCommit(C,S,\vec{m}[S],\vec{m}'[S]) C′←UpdateCommit(C,S,m[S],m′[S])是确定性的,若使用相同的randomness,通过 C C C与 C ′ C' C′之间的关系会reveal m ⃗ \vec{m} m与 m ⃗ ′ \vec{m}' m′之间的关系。可以在UpdateCommit
之后再增加rerandomization处理来hide this relationship。本文方案支持rerandomization操作。【commitment can be rerandomized via multiplication by ( g 1 α N ) r ′ (g_1^{\alpha^N})^{r'} (g1αN)r′.】
实际hiding属性的实现采用与Benoît Libert和Moti Yung 2010年论文《Concise Mercurial Vector Commitments and Independent Zero-Knowledge Sets with Short Proofs》中类似的方式——在Commit
时引入随机值 g γ g^{\gamma} gγ。
加入hiding元素后,具体的算法调整如下:
Blockchain:is an append-only public ledger that consists of blocks, with every block containing some number of transactions.
区块链中最根本的问题是对新区块达成共识,共识的过程包括:
传统的方式validator需要维护整个state账本,存在存储压力问题。
在[ST99, Mil12, Whi15, Tod16,But17, Dra17, RMCI17, CPZ18, BBF19]等很多论文中,都提议validator store commitments to vectors of relevant values instead of the values themselves。基于此提议实现的可称为stateless client或者stateless blockchain。相应的,transactions中需要包含:
该模式需要在storage,bandwidth和computation之间做取舍平衡。理想的情况应该是具有small commitments and proof sizes and add little computation and bandwidth overheads to the validators。
简单的transaction,类似于比特币账号之间的转账;
复杂的transaction,是基于智能合约的。
现有各方案在解决storage方面的表现对比:【若validator直接存储明文,当需存储 1 0 8 10^8 108个account的状态信息时,需要的存储空间将近3TB;Pointproofs中若引入central entity,对整个系统维护一个commitment,而不是为每个account维护一个commitment,但是没有意义,当 1 0 8 10^8 108个account时,仅需4.5GB已经足够小了,没必要再引入中心化的机制。】
当由1000个256-bit message时,各vector commitment方案的参数:
将Pointproofs用于smart-contract-based的transaction:
Pointproofs Prove
、Aggregate
、Verify
、AggregateAcross
等基础算法的性能表现为: