【密码学】传统密码：统计分析

    传统密码学中，单表代换密码用统计分析的方法很容易破译，因为在单表代换下，字母出现频率和重复模式都得到了保留，也是比较不安全的。而多表代换密码破译难度则相对大一些，因为明文的统计信息在多个表转换之后已经变得很弱了。而Hill密码如果只是拿到密文，破译难度是很大的，除非得知部分密文和明文，则容易获取密钥，这过程类似于矩阵求解。

    用统计分析的方法，我们以英文为例。事实上，英文是比较容易破解的语言，因为英语只有26种字母，忽略大小写。而我们汉字一级常用汉字就有3755个。下面我们用一个程序来统计英文中各个字母出现的概率。

/***********************************************************/      
// 程序目的：统计英文字母出现概率    
// 日期：    2014-12-20      
// 作者：    spencer_chong      
// 邮箱：    [email protected]      
/***********************************************************/   
#include 
void main()
{
	float number[26]={0};

	FILE *fp;
	int total=0;
	int i;
	fp=fopen("English.txt","r");
	if(fp!=NULL)
		while(!feof(fp)){
			c=fgetc(fp);
			if(c>='A'&&c<='Z'){
				number[c-'A']++;
				total++;
			}
			else if(c>='a'&&c<='z'){
				number[c-'a']++;
				total++;
			}
		}
	else
		printf("fail to open1\n");
	fclose(fp);

	for(i=0;i<26;i++){
		//rate=number[i]/total;
		printf("%c,%c  :  %0.3f\n",'a'+i,'A'+i,number[i]/total);
	}
}

    今天是澳门回归15周年，我在chinadaily上找了一篇文章放到English.txt里面，然后统计每个字母出现的概率。

MACAO - Chinese President Xi Jinping said here Saturday morning that Macao's return to the motherland was an event of great historic significance in the development of the Chinese nation and marked a new chapter in Macao's development.
"We are glad to see that the principles of 'one country, two systems', 'Macao people administering Macao', and a high degree of autonomy as well the Basic Law of the Macao SAR have won massive support from the people in Macao and have been implemented in real earnest," Xi said at a celebration gathering marking the 15th anniversary of Macao's return. 
President Xi said that Macao has enjoyed orderly progress in democracy, fast economic growth, rising living standards, and social harmony and stability since its return to the motherland 15 years ago.
Xi added that the close national bond of "blood being thicker than water" is becoming ever stronger.
"Love for the motherland and love for Macao has become a prevalent value in Macao society," he said.
In the meantime, he said, Macao, as a historical city where Chinese and Western cultures meet, displays a unique charm that belongs to a culture typical of Southern China, yet with a distinctive European touch.
"Here, communities from various ethnic backgrounds live in harmony and help and learn from each other," said Xi.
"Together, they present to the world a dynamic Macao beaming with vitality." 
Noting rapid economic and social progress, Xi said, "Certain deep-seated problems formed over the years have surfaced, and development risks have built up to some extent."
"It is important for Macao to adopt a global, nationwide, future-oriented and long-term perspective, formulate appropriate plans and blueprints for its development and promote sound economic and social development," he said in a speech delivered at the Macao East Asian Games Dome.
The president stressed the necessity to make long-term planing, seize the opportunity of the national efforts to comprehensively deepen reform, and promote appropriately diversified and sustainable economic development in Macao based on its positioning as a global tourism and leisure hub and a service platform for economic and trade cooperation between China and Portuguese-speaking countries. 
Xi also urged continued efforts to enhance social harmony and stability in the Macao Special Administrative Region (SAR).
"Both the SAR government and people from all walks of life in Macao must doubly cherish and do everything they can to safeguard harmony and stability in Macao," Xi said at a celebration gathering marking the 15th anniversary of Macao's return to China.
The president also urged the SAR government to "put people first," properly respond to diverse demands in the society, balance the interests of various sides and foster a more equitable and just social environment.
The SAR government needs to make sure that the Macao people have better access to the fruits of development, improve their quality of life and increase their happiness index, he said.
Meanwhile, the president called on the Macao people to continue to love Macao and the motherland, support the work of the chief executive and the SAR government in governing Macao in accordance with law, strengthen social cohesion and positive energy, and work jointly for long-term prosperity and stability of Macao.
Xi emphasized the need to guard against external infiltration and interference, so as to maintain the sound atmosphere of stability and unity in Macao.

统计结果如下：

    由于文本内容比较短，而且本文中Xi，Macao等关键字出现的频率特别大，所以这个统计结果和实际情况会有所偏差。要获得比较准确的概率分布，可以通过大量文本统计来实现，这里只是举个例子。

一、单表代换密码分析

    不妨假设加密函数为e(x)=ax+b(mod26),为一个二元一次方程，那么就必须确定两种密文和明文的对应。理论上将，穷举法来选择也可以求解出a和b，但是我们有每个字母出现概率的先验知识，不烦假设密文中出现概率最高的就是明文中出现概率最高的，然后验证解出来的明文是否有意义来确定，这样的效率更高。

二、多表代换密码分析

    多表代换密码的分析方法分为三步：（1）确定密钥的长度（2）确定密钥（3）根据第二步确定的密钥恢复出明文，并且验证明文。

    确定密钥的长度，常用的方法有Kasiski法和重合指数法。

    Kasiski测试法思路是对一份用周期性多表密码加密的密文，确定其中所有的重复出现的字母串，计算他们之间的距离，并对这些距离进行因子分解，出现频率较高的因子很可能是密钥的长度。

    重复指数法利用随机文本和英文文本的统计概率差别来分析密钥长度。重复指数指的是两个字母相等的概率。在一串随机文本中，我们随意抽取两个字母，由于每个字母被抽到的概率相同，那么抽取的两个字母相同的概率为26*（1/26）^2=0.038。如果密文是从一篇文章或者一句完整的话中抽取的，每个字母出现的概率并不是随机的，根据前面对每个字母出现概率的统计，我们抽出的两个字母相等的概率为P(a)^2+P(b)^2+….+p(z)^2=0.065。

    但是事实上，我们的密文的长度是很有限的，和大量数据统计出来的而结果有较大偏差，所以我们采用无偏估计值来评估。重合指数的无偏差计算公式为IC=sum/(X*(X-1))。为什么是IC=sum/(X*(X-1))这个公式呢？假设密文为abcdefg...，先统计出各个字母出现的频数，f(a),f(b),……,f(z)，然后定义f(a)+f(b)+……+f(z)=X，也就是说X就是密文的长度。然后把所有的 f(i)*(f(i)-1) 的和求出来（这里的i代表某个字母），f(a)*(f(a)-1)+f(b)*(f(b)-1)+...+f(z)*(f(z)-1)=sum。已知从X个密文中抽取两个字母的方式有CX^2=X(X-1)/2种，如果X个密文中有f(a)个字母a，则可以组成Cf(a)^2=f(a)(f(a)-1)/2种，共有26个字母，所以重复指数为[f(a)*(f(a)-1)+f(b)*(f(b)-1)+...+f(z)*(f(z)-1)]/X(X-1)。下面我们通过一个例子加以说明。

明文：

the birth of cars have made an enormous change to our life.in the past,we travel from one place to another only by foot,nowaday,cars can do it .its goes withour saying that the invention of cars bring great benefit to all of us.but as proverb goes:no garden without weeds.car is not exception.
owing a car has a lot of advantages.for one thing,car provide us the most convient way of transportation.we can get around freely without spenting a lot of time.emotionally,i always found driving is so exciting.for another,its the comfortable to drive a car.In winter.drivers always can stay warm and dry even in rainy whether,in addition,drivers are usually safe in their cars when they are out at night.
Cars bring the human merits,their side-effects graudually come to the surface.firstly,to run a car need a lot of oil,which is getting less and less.the increasing number of cars contribute the lacking of energy.secondlly,as  more and more cars are used,the traffic ecpecially in big cities is getting heaver and heavier,which lead to the serious social problem--traffic jam.in addition,the inceasing numbers of cars ,which excaust sent a huge quantities of carbon monoxide into atmosphere.it make the air of cities unbreathabe,it strip people contact with frensh air.
therefore,the new energy should be explored to replace the oil so that our envionmental pollution can be avioded .and the strick law should be issued to keep the numbers of cars under control.thus,our heavier traffic can be solluted!

统计结果如下：

Aa	Bb	Cc	Dd	Ee	Ff	Gg	Hh	Ii	Jj	Kk	Ll	Mm	Nn
0.087	0.017	0.042	0.031	0.116	0.029	0.023	0.045	0.078	0.001	0.003	0.036	0.018	0.074
Oo	Pp	Qq	Rr	Ss	Tt	Uu	Vv	Ww	Xx	Yy	Zz	IC
0.08	0.014	0.001	0.074	0.059	0.091	0.03	0.014	0.017	0.004	0.016	0	0.065

为了方便，我们将其化为大写字母来表示，

THEBIRTHOFCARSHAVEMADEANENORMOUSCHANGETOOURLIFEINTHEPASTWETRAVELFROMONEPLACETOANOTHERONLYBYFOOTNOWADAYCARSCANDOITITSGOESWITHOURSAYINGTHATTHEINVENTIONOFCARSBRINGGREATBENEFITTOALLOFUSBUTASPROVERBGOESNOGARDENWITHOUTWEEDSCARISNOTEXCEPTIONOWINGACARHASALOTOFADVANTAGESFORONETHINGCARPROVIDEUSTHEMOSTCONVIENTWAYOFTRANSPORTATIONWECANGETAROUNDFREELYWITHOUTSPENTINGALOTOFTIMEEMOTIONALLYIALWAYSFOUNDDRIVINGISSOEXCITINGFORANOTHERITSTHECOMFORTABLETODRIVEACARINWINTERDRIVERSALWAYSCANSTAYWARMANDDRYEVENINRAINYWHETHERINADDITIONDRIVERSAREUSUALLYSAFEINTHEIRCARSWHENTHEYAREOUTATNIGHTCARSBRINGTHEHUMANMERITSTHEIRSIDEEFFECTSGRAUDUALLYCOMETOTHESURFACEFIRSTLYTORUNACARNEEDALOTOFOILWHICHISGETTINGLESSANDLESSTHEINCREASINGNUMBEROFCARSCONTRIBUTETHELACKINGOFENERGYSECONDLLYASMOREANDMORECARSAREUSEDTHETRAFFICECPECIALLYINBIGCITIESISGETTINGHEAVERANDHEAVIERWHICHLEADTOTHESERIOUSSOCIALPROBLEMTRAFFICJAMINADDITIONTHEINCEASINGNUMBERSOFCARSWHICHEXCAUSTSENTAHUGEQUANTITIESOFCARBONMONOXIDEINTOATMOSPHEREITMAKETHEAIROFCITIESUNBREATHABEITSTRIPPEOPLECONTACTWITHFRENSHAIRTHEREFORETHENEWENERGYSHOULDBEEXPLOREDTOREPLACETHEOILSOTHATOURENVIONMENTALPOLLUTIONCANBEAVIODEDANDTHESTRICKLAWSHOULDBEISSUEDTOKEEPTHENUMBERSOFCARSUNDERCONTROLTHUSOURHEAVIERTRAFFICCANBESOLLUTED

我们是使用密钥key=SCUT对其尽心加密，加密程序如下：

/***********************************************************/      
// 程序目的：多表代换加密    
// 日期：    2014-12-21      
// 作者：    spencer_chong      
// 邮箱：    [email protected]      
/***********************************************************/   
#include 
void main()
{

	FILE *fp;
	FILE *fp2;
	char c;
	char text[10000]={0};
	int i=0;
	int number=0;

	char key[4]={'S','C','U','T'};
	fp=fopen("English.txt","r");

	if(fp!=NULL)
		while(!feof(fp)){
			c=fgetc(fp);
			if(c>='A'&&c<='Z'){
				text[number]=c;
				number++;
			}
			else if(c>='a'&&c<='z'){
				text[number]=c-'a'+'A';
				number++;
			}
		}
	else
		printf("fail to open1\n");
	fclose(fp);

//多表代换加密	
	for(i=0;i

加密结果如下，我们将其放入English.txt文本中，一次按照密钥长度为1,2,3,4,5...分析子串的IC值。

LJYUATNAGHWTJUBTNGGTVGUGWPIKEQOLUJUGYGNHGWLEAHYBFVBXHCMMOGNKSXYEXTIFGPYIDCWXLQUGGVBXJQHEQDSYGQNGGYUWSAWTJUWTFFIBLKNLYQYLOKNAGWLLSACGYVBTLVBXAPPXFVCHFQZVSTMUJKHZYTYTLDYGWHCMLQUEDQZNKDOMSUJKGXYKTIIXKPIZSTXXFYCMZQOMOGYWKEUKAUHHLGRVWRNBGPIPAPATUCLASUUEGVIYSFPTFVUZWUZHJQHXLJCGYEUKHTIOAFYNKVBXEQMMUQHOAGHMOCSHXVLTFUJHJVUMAQHPWEUGYGNTJQOGVHLXWNSPAVBHMVMIWPNBFIUEGVIYLKGXWOIMAQHTDNSBSNQTQUZHMPXWJKPBFICLKQYQUKNBFIZHJCHHLJYKAVMMZGWHEHIKLCVEWVIWJKPXSEUKAPQBFVYKVTCOWTMTDYURKEUGKVUROCLFSPXWJAYOWPCGJCCGQYBXLJYKAPUWVKNBGPXKAXYKKCLXMUOTDNSLSHYBFVBXATWTJUQAWPNAWAUKWQOMSVHBYJNVSTMUJKHZLJYAMOUGEGLBLUNAWKLLAFYXXHYVLUAKSWXNSNFRUQGXLQNAWUOKXCWXXKLLLNSMGTOGSEUKFGYWSNIMGHIBDYBBUJCLYGNMAPAEWUMTFFFXKUNAWKHVJGULAPAGMOVXJQZVSTMVGPNKADOMWVBXDCWDAPAHXGHXJISLWEIGVNFRSUGHJGUGVOIKWEUKKCLXMUYWLJYMJCZYAEYVHGWBSNFRAPVBYECMAGMBKIYMLKHZZGUOWTUGVJYTNKYKOJCVZNYTVVIMZGMXJKINKUIVACFIJQVEWONKSHZBULUFAPUWVKNBGPNAWKHVWCMBFIHNEDYKKQZVSTMPZKWAWZWTMUNLWPNTZWAXIWUGLKNBWUIYUCLUGPGHFQRBVGCGLQUMEQMIZGLXAVGTCGNAWCCKGHWBLKYLMPVKWCNASDYBLUNKARJXGRFXUQHMSENPAVBYJGHLZCCKLJYKWHIKWVBXFGQXFGLZQUBHMNXUWGRIDQLXVVIKWRFTUGNAWQCEKQNASVINJGHOAQHFWPNTDRIEDWNBGPWTFDYTNKIWWFUGVVBXKVLBUMFTOUBHMNXUWKMLMGXMGMYXHVBXFWGUWTMHXEUKKWHWWTWHFVLHDVBNKQOKZGUOAGLMJCZYAEWTFDYLGNFNLGX

（1）当密钥长度为1时，

key： 1

密文：L

      J

      Y

      U

      ....

子串1：

Aa	Bb	Cc	Dd	Ee	Ff	Gg	Hh	Ii	Jj	Kk	Ll	Mm	Nn
0.046	0.036	0.03	0.016	0.025	0.034	0.067	0.045	0.031	0.032	0.058	0.049	0.043	0.046
Oo	Pp	Qq	Rr	Ss	Tt	Uu	Vv	Ww	Xx	Yy	Zz	IC
0.024	0.029	0.033	0.011	0.027	0.037	0.06	0.049	0.061	0.043	0.046	0.022	0.043

（2）当密钥长度为2时：

key： 1  2

密文：L  J

      Y  U

      ....

子串1（LY...）：

Aa	Bb	Cc	Dd	Ee	Ff	Gg	Hh	Ii	Jj	Kk	Ll	Mm	Nn
0.061	0.03	0.028	0.018	0.01	0.051	0.048	0.041	0.043	0.044	0.028	0.07	0.044	0.057
Oo	Pp	Qq	Rr	Ss	Tt	Uu	Vv	Ww	Xx	Yy	Zz	IC
0.028	0.007	0.013	0.005	0.054	0.002	0.07	0.028	0.09	0.026	0.072	0.033	0.05

子串2(JU...)：

Aa	Bb	Cc	Dd	Ee	Ff	Gg	Hh	Ii	Jj	Kk	Ll	Mm	Nn
0.031	0.043	0.033	0.015	0.039	0.016	0.087	0.049	0.02	0.02	0.089	0.028	0.041	0.034
Oo	Pp	Qq	Rr	Ss	Tt	Uu	Vv	Ww	Xx	Yy	Zz	IC
0.02	0.051	0.053	0.016	0	0.072	0.049	0.071	0.031	0.061	0.02	0.011	0.05

（2）当密钥长度为3时：

子串1：

Aa	Bb	Cc	Dd	Ee	Ff	Gg	Hh	Ii	Jj	Kk	Ll	Mm	Nn
0.025	0.039	0.017	0.022	0.022	0.027	0.064	0.052	0.037	0.049	0.049	0.049	0.047	0.054
Oo	Pp	Qq	Rr	Ss	Tt	Uu	Vv	Ww	Xx	Yy	Zz	IC
0.025	0.029	0.027	0.007	0.02	0.039	0.086	0.037	0.074	0.044	0.034	0.025	0.044

子串2：

Aa	Bb	Cc	Dd	Ee	Ff	Gg	Hh	Ii	Jj	Kk	Ll	Mm	Nn
0.059	0.039	0.039	0.007	0.027	0.037	0.062	0.039	0.022	0.032	0.059	0.049	0.039	0.044
Oo	Pp	Qq	Rr	Ss	Tt	Uu	Vv	Ww	Xx	Yy	Zz	IC
0.017	0.027	0.034	0.015	0.03	0.047	0.049	0.034	0.047	0.047	0.062	0.034	0.041

子串3：

Aa	Bb	Cc	Dd	Ee	Ff	Gg	Hh	Ii	Jj	Kk	Ll	Mm	Nn
0.054	0.03	0.034	0.02	0.025	0.037	0.076	0.044	0.034	0.015	0.067	0.049	0.042	0.039
Oo	Pp	Qq	Rr	Ss	Tt	Uu	Vv	Ww	Xx	Yy	Zz	IC
0.03	0.03	0.037	0.01	0.032	0.025	0.044	0.076	0.062	0.039	0.042	0.007	0.044

（2）当密钥长度为4时：

子串1：

Aa	Bb	Cc	Dd	Ee	Ff	Gg	Hh	Ii	Jj	Kk	Ll	Mm	Nn
0.102	0	0.003	0.036	0.02	0.072	0.072	0.013	0.003	0.079	0.056	0.082	0.036	0.01
Oo	Pp	Qq	Rr	Ss	Tt	Uu	Vv	Ww	Xx	Yy	Zz	IC
0.023	0	0.013	0	0.085	0.003	0.036	0.039	0.131	0.023	0.03	0.033	0.067

子串2：

Aa	Bb	Cc	Dd	Ee	Ff	Gg	Hh	Ii	Jj	Kk	Ll	Mm	Nn
0.013	0	0.066	0.026	0.043	0.02	0.105	0.036	0.023	0.039	0.072	0.003	0.007	0.043
Oo	Pp	Qq	Rr	Ss	Tt	Uu	Vv	Ww	Xx	Yy	Zz	IC
0.016	0.085	0.102	0.016	0	0.052	0.075	0.102	0.026	0.01	0.016	0.003	0.064

子串3：

Aa	Bb	Cc	Dd	Ee	Ff	Gg	Hh	Ii	Jj	Kk	Ll	Mm	Nn
0.02	0.059	0.052	0	0	0.03	0.023	0.069	0.082	0.01	0	0.059	0.052	0.105
Oo	Pp	Qq	Rr	Ss	Tt	Uu	Vv	Ww	Xx	Yy	Zz	IC
0.033	0.013	0.013	0.01	0.023	0	0.105	0.016	0.049	0.03	0.115	0.033	0.065

子串4：

Aa	Bb	Cc	Dd	Ee	Ff	Gg	Hh	Ii	Jj	Kk	Ll	Mm	Nn
0.049	0.086	0	0.003	0.036	0.013	0.069	0.063	0.016	0	0.105	0.053	0.076	0.026
Oo	Pp	Qq	Rr	Ss	Tt	Uu	Vv	Ww	Xx	Yy	Zz	IC
0.023	0.016	0.003	0.016	0	0.092	0.023	0.039	0.036	0.112	0.023	0.02	0.064

    由此可见，当密钥长度为4时，IC稳定在理论值0.065附近。

    知道密钥长度之后，采用单表代换的方法可以纠结密钥，从而实现密码破解。