用python写语言的解释器
我花了一下午的时间完成了一个简单语言的解释器，我会在最后帖出所有代码，但是今天不打算详细解释程序的每一个步骤，最近考虑找实习、做论文，要花一些时间。有时间了我会解释每一部分，在这里只提醒一下读者，程序的写作过程和它呈现出来的不一样，总体来说我的写作过程是先写一个只包含一条指令的解释器，然后逐渐加入其他指令。ps：我是多么的想看看高手们写程序的过程，而不只是结果，但是就像graham说的“写的过程往往显得冗长”，所以这方面的书籍不多。我觉得比较接近的书包括《clean code》、《paip》、《software tools》。只可惜现在只看完了一本半。
想法的来源
昨天圣诞节，坐在不能上网的实验室，觉得好无聊。然后想起了学习《可计算性与计算复杂性》的时候，里面用到了一种含有5条指令的原语言，老师也要求我们用这个语言写程序，所以我就想写一个解释器，来执行用该语言，并把这个解释器作为自己的圣诞礼物。
原语言描述
书中说它是一个fortran式的简单语言，所以我就给它起了个简单的名字Metafor，表示meta fortran。下面来看一下它的具体描述，然后贴上我在代码里更详细的描述，我为每一种指令都起了个名字（注意其中把赋值语句命名为setf是由于我最近在写common lisp多一些）。
+-------------------------------------------------------------------------------------------+
指令 描述
x = x + 1 变量x的值加1
x = x - 1 变元x的值减1。若x的值为0，则结果仍为0
TO A IF x≠0 若x≠0，则转标号为A的指令；否则执行下一条指令
TO A 无条件转到标号为A的指令
y=x 把x的值赋给变元y，x值保持不变
+-------------------------------------------------------------------------------------------+
1. Metafor has five instructions:
name instruction description
inc: x = x + 1, Increase by 1 the value of the variable x.
dec: x = x - 1, If the value of x is 0, leave it unchanged; otherwise
decrease by 1 the value of x.
con_goto: TO A IF x != 0, If the value of x is nonzero, perform the instruction
with the label A next; otherwise proceed to the next
instruction in the list.
goto: TO A, Perform the instruction with the label A next.
setf: y = x, Change the value variable y to the value of variable x.
2. Some specification:
(1) Input variables are represented as:
x1, x2, x3, ...
(2) Local variables are represented as:
z1, z2, z3, ...
(3) Output variable is represented as:
y
note: the num 1 is often omitted(i.e., x stands for x1 and z stands
for z1).
3. Labeled instructions:
Instructions may or may not have labels. When an instruction is labeled,
the label is written to its left in square brackets. For example,
[B] Z = Z - l
4. A smaple program:
"Program to compute y = x1+ x2"
y = x1
[B] TO A IF x2 != 0
TO E
[A] x2 = x2 - 1
y = y + 1
TO B
4. For more information, please refer to 《可计算性与计算复杂性》,
周长林、李占山著.由于该语言中含有goto语句，所以我不能一句一句的解释执行，我需要把整个程序读进来，转换成一种方面的内部格式后，作为整体执行。例如，这是计算y = x + x2的语言程序：
y = x
[B] TO A IF x2 != 0
TO E
[A] x2 = x2 - 1
y = y + 1
TO B它转换为内部格式后，为：
[['setf', 'y', 'x'],
['labeled_exp', 'B', ['con_goto', 'A', 'x2']],
['goto', 'E'],
['labeled_exp', 'A', ['dec', 'x2']],
['inc', 'y'],
['goto', 'B']]还是前面那个y = x + x2的程序，首先我需要为两个输入参数（x, x2）给定值，如果没有为其赋值，那么它们的值默认为0。另外，我们还可以查看程序的内部表示结构，以及简单调试（就查看其他环境变量的值）。其演示结果如下：
======================================================
*Metafor 1.0, Welecome to Metafor shell environment. *
*Author: Zhu zhaolong([email protected]) *
======================================================
Metafor> 2 3
5
Metafor> 5 6
11
Metafor> code
[['setf', 'y', 'x'],
['labeled_exp', 'B', ['con_goto', 'A', 'x2']],
['goto', 'E'],
['labeled_exp', 'A', ['dec', 'x2']],
['inc', 'y'],
['goto', 'B']]
>>> ========================================== RESTART ==========================================
>>>
======================================================
*Metafor 1.0, Welecome to Metafor shell environment. *
*Author: Zhu zhaolong([email protected]) *
======================================================
Metafor> 234 5
239
Metafor> debug
debug> x
234
debug> x2
0
debug> y
239
debug> exit
>>> 我写了一个简单的repl交互环境，这样我们可以方便的测试简单的原语言程序，例如下面给出了一个y = x + 1的程序，示例如下：
>>> repl()
======================================================
*Metafor 1.0, Welecome to Metafor shell environment. *
*Author: Zhu zhaolong([email protected]) *
======================================================
Input your program:
x = x + 1
y = x
inputs> 5
=>6
Input your program:
在这里我贴出所有python代码：
""" Metafor: meta fortran, is a small language used in
"Computabilit and Complexity" course in Jilin University.
@Author: Zhu Zhaolong([email protected])
@Date: 2011.12.25
1. Metafor has five instructions:
name instruction description
inc: x = x + 1, Increase by 1 the value of the variable x.
dec: x = x - 1, If the value of x is 0, leave it unchanged; otherwise
decrease by 1 the value of x.
con_goto: TO A IF x != 0, If the value of x is nonzero, perform the instruction
with the label A next; otherwise proceed to the next
instruction in the list.
goto: TO A, Perform the instruction with the label A next.
setf: y = x, Change the value variable y to the value of variable x.
2. Some specification:
(1) Input variables are represented as:
x1, x2, x3, ...
(2) Local variables are represented as:
z1, z2, z3, ...
(3) Output variable is represented as:
y
note: the num 1 is often omitted(i.e., x stands for x1 and z stands
for z1).
3. Labeled instructions:
Instructions may or may not have labels. When an instruction is labeled,
the label is written to its left in square brackets. For example,
[B] Z = Z - l
4. A smaple program:
"Program to compute y = x1+ x2"
y = x1
[B] TO A IF x2 != 0
TO E
[A] x2 = x2 - 1
y = y + 1
TO B
4. For more information, please refer to 《可计算性与计算复杂性》,
周长林、李占山著.
"""
###========================================================
import re
from pprint import pprint
###===========================parse========================
def parse(program):
return [read(e) for e in program]
def read(s):
"""Read a metafor insctruction from a string. and change
it to internal represtation."""
return read_from(tokenize(s))
def tokenize(s):
"Convert a string into alist of tokens."
return s.split()
def read_from(tokens):
"Read an exp from a sequence of tokens."
if len(tokens) == 0:
raise SyntaxError("uncexpected EOF while reading.")
if is_setf(tokens):
return make_setf(tokens[0], tokens[2])
elif is_inc(tokens):
return make_inc(tokens[0])
elif is_dec(tokens):
return make_dec(tokens[0])
elif is_goto(tokens):
return make_goto(tokens[1])
elif is_con_goto(tokens):
return make_con_goto(tokens[1], tokens[3])
elif is_labeled_exp(tokens):
return make_labeled_exp(get_label(tokens[0]), read_from(tokens[1:]))
else:
raise SyntaxError("unexpected instructions: %s" % " ".join(tokens))
#=======================================================
def is_variable(token):
if token.startswith("x") or \
token.startswith("z") or \
token == "y":
return True
else:
return False
def get_label(token):
"Token is like [A1]. we want get A1."
return token.replace('[', '').replace(']', '')
## setf
def is_setf(tokens):
if tokens[1] == "=" and len(tokens) == 3:
if is_variable(tokens[0]) and is_variable(tokens[2]):
return True
else:
raise SyntaxError("unexpected setf instruction: %s" % " ".join(tokens))
else:
return False
def make_setf(des_var, src_var):
return ["setf", des_var, src_var]
## inc
def is_inc(tokens):
if len(tokens) == 5 and tokens[1] == "=" \
and tokens[3] == "+" and tokens[4] == "1":
if tokens[0] == tokens[2] and is_variable(tokens[0]):
return True
else:
raise SyntaxError("unexpected inc instruction: %s" % " ".join(tokens))
else:
return False
def make_inc(var):
return ["inc", var]
## dec
def is_dec(tokens):
if len(tokens) == 5 and tokens[1] == "=" \
and tokens[3] == "-" and tokens[4] == "1":
if tokens[0] == tokens[2] and is_variable(tokens[0]):
return True
else:
raise SyntaxError("unexpected dec instruction: %s" % " ".join(tokens))
else:
return False
def make_dec(var):
return ["dec", var]
## goto
def is_goto(tokens):
if len(tokens) == 2 and tokens[0] == "TO":
return True
else:
return False
def make_goto(lable):
return ["goto", lable]
## con_goto
def is_con_goto(tokens):
if len(tokens) == 6 and tokens[0] == "TO" \
and tokens[2] == "IF" and is_variable(tokens[3]) \
and tokens[4] == "!=" and tokens[5] == "0":
return True
else:
return False
def make_con_goto(lable, var):
return ["con_goto", lable, var]
## labeled exp
def is_labeled_exp(tokens):
return tokens[0].startswith('[')
def make_labeled_exp(label, exp):
return ["labeled_exp", label, exp]
###==========================CodeSeg======================
def is_label(s):
return s == "labeled_exp"
class CodeSeg:
""" To store internal instructions (exps).
N : the number of instructions.
exps: the list of instructions.
label_2_exps: a dict of {'label': num} pairs."""
def __init__(self, exps):
self.N = len(exps)
self.exps = []
self.label_2_exp={}
for (i, e) in enumerate(exps):
if is_label(e[0]):
(_, label, exp) = e
self.label_2_exp[label] = i
self.exps.append(exp)
else:
self.exps.append(e)
def get_instruction_num(self, label):
"Return instruction num with the label."
if label in self.label_2_exp.keys():
return self.label_2_exp[label]
# IF there is no instruction with that label,
# return a instruction num that doesn't exit.
else:
return self.N
def get_instruction(self, pc):
return self.exps[pc]
###=============================Env======================
class Env():
"An enviromnet: a dict of {'var': val} paris, var's default value is 0."
def __init__(self, args):
self.pairs = {}
for (i, v) in enumerate(args):
if i == 0:
self.pairs['x'] = int(v)
else:
self.pairs['x'+str(i+1)] = int(v)
def get_v(self, var):
if var in self.pairs.keys():
return self.pairs[var]
else:
return 0
def set_v(self, var, value):
self.pairs[var] = value
###=========================mainloop=======================
def mainloop(codeseg, env):
"pc refers to the current instruction."
pc = 0
while pc < codeseg.N:
e = codeseg.get_instruction(pc)
if e[0] == 'inc':
(_, v) = e
env.set_v(v, env.get_v(v) + 1)
pc += 1
elif e[0] == 'dec':
(_, v) = e
env.set_v(v, max(env.get_v(v) - 1, 0))
pc += 1
elif e[0] == 'setf':
(_, x, y) = e
env.set_v(x, env.get_v(y))
pc += 1
elif e[0] == 'goto':
(_, label) = e
pc = codeseg.get_instruction_num(label)
elif e[0] == "con_goto":
(_, label, var) = e
val = env.get_v(var)
if val != 0:
pc = codeseg.get_instruction_num(label)
else:
pc += 1
else:
raise SyntaxError("unexpected instructions: %s" % " ".join(e))
return env.get_v('y')
###========================repl================================
def repl(input_prompt='inputs> '):
"A prompt-read-eval-print loop."
print_info()
while True:
program = get_prog_from_shell()
if program == []:
return "Return successfully!"
codes = CodeSeg(parse(program))
inputs = raw_input(input_prompt)
env = Env(inputs.split())
print("=>" + str(mainloop(codes, env)))
def get_prog_from_shell():
program = []
exp = raw_input("\n\nInput your program:\n")
while exp:
program.append(exp)
exp = raw_input()
return program
def print_info():
print("======================================================\n"
"*Metafor 1.0, Welecome to Metafor shell environment. *\n"
"*Author: Zhu zhaolong([email protected]) *\n"
"======================================================\n")
###========================run=============================
def run(filepath, prompt='Metafor> '):
"This function can run Metafor program from file."
print_info()
program = get_prog_from_file(filepath)
parsed_prog = parse(program)
codes = CodeSeg(parsed_prog)
inputs = raw_input(prompt)
while True:
env = Env(inputs.split())
print(mainloop(codes, env))
inputs = raw_input(prompt)
if inputs == "exit":
return "Return successfully!"
if inputs == "debug":
while True:
var = raw_input('debug> ')
if var == "exit":
return None
print(env.get_v(var))
if inputs == "code":
pprint(parsed_prog)
return None
def get_prog_from_file(filepath):
f = open(filepath, "r")
program = f.readlines()
f.close()
return program
###===========================test==================================
# y = x + x2
test_file_1 = "metafor_test.mf"
# y = 2 * x
test_file_2 = "metafor_test2.mf"
if __name__ == "__main__":
run(test_file_1)