chapter 17 Weak Tables and Finalizers

17 Weak Tables and Finalizers

Lua does automatic memory management. Programs create objects (tables,
threads, etc.), but there is no function to delete objects. Lua automatically
deletes objects that become garbage, using garbage collection. This frees you
from most of the burden of memory management and, more importantly, frees
you from most of the bugs related to this activity, such as dangling pointers and
memory leaks.

The use of a real garbage collector means that Lua has no problems with
cycles. You do not need to take any special action when using cyclic data
structures; (也就是哪怕最后这些对象之间形成一个引用环，还是可以collect 到的)

Weak tables and finalizers are the mechanisms that you can use in Lua
to help the garbage collector. Weak tables allow the collection of Lua objects
that are still accessible to the program, while finalizers allow the collection of
external objects that are not directly under control of the garbage collector.

17.1 Weak Tables

A garbage collector can collect only what it can be sure is garbage; it cannot
guess what you consider garbage.

A typical example is a stack, implemented  with an array and an index to the top. You know that the valid part of the
array goes only up to the top, but Lua does not. If you pop an element by  simply decrementing the top, the object left in the array is not garbage to Lua. see blow :

(Stack)

Topindex-> [SomeElement1]

                   [SomeElemen2t]

                   [SomeElement3]

                    ..........

                  [SomeElementN]

when you decrement the top to pop the top element

                       [SomeElement1]   (This still in memory, But Lua do not think it's garbage)

Topindex->    [SomeElemen2t]

                      [SomeElement3]

                       ..........

                     [SomeElementN]

Similarly, any object stored in a global variable is not garbage to Lua, even if
your program will never use it again.

In both cases, it is up to you (i.e., your
program) to assign nil to these positions (让Lua认为是null)so that they do not lock an otherwise
disposable object.

However, simply cleaning your references is not always enough. Some constructions
need extra collaboration between the program and the collector. A
typical example happens when you want to keep a collection of all live objects
of some kind (e.g., files) in your program. This task seems simple: all you have
to do is to insert each new object into the collection. However, once the object is
part of the collection, it will never be collected! Even if no one else points to it,
the collection does. Lua cannot know that this reference should not prevent the
reclamation of the object, unless you tell Lua about this fact.

a="mya";b="myb";c={a,b}; a,b=nil,nil;
for k,v in pairs(c) do
 print (k,v)
end

dofile("lession16.lua");
1       mya
2       myb   -- eventhough I have put a,b to nil, we c the collection hold the object. and what's more, Lua do not know

                     this reference should not prevent the reclamation, 也就是虽然a,b=nil,但是否因该回收的，还不行。because table c point these objects.

Weak tables are the mechanism that you use to tell Lua that a reference
should not prevent the reclamation of an object. (读到这里我已经猜到时什么了，其实跟Object C 里面的weak 引用一样的)

A weak reference is a reference
to an object that is not considered by the garbage collector. (被申明为weak 的引用不被collector 考虑，) If all references  pointing to an object are weak,the object is collected and somehow these weak
references are deleted. Lua implements weak references as weak tables: a weak
table is a table whose entries are weak. This means that, if an object is held only
inside weak tables, Lua will eventually collect the object.

Tables have keys and values, and both can contain any kind of object. Under
normal circumstances, the garbage collector does not collect objects that appear
as keys or as values of an accessible table. That is, both keys and values are
strong references,(是Object C 抄Lua or Lua 仿Object C, who knows) as they prevent the reclamation of objects they refer to.

In a
weak table, both keys and values can be weak. This means that there are three
kinds of weak tables: tables with weak keys, tables with weak values, and fully
weak tables, where both keys and values are weak. Irrespective of the table
kind, when a key or a value is collected the whole entry disappears from the
table 也就是不管是哪种类型的table, Ie. weak Key ,strong value, 如果再没有 strong referece refer 住这个对象，那么整个entry will be collect.

The weakness of a table is givenby the field __mode of its metatable. The
value of this field, when present, should be a string: if this string is “k”, the keys
in the table are weak; if this string is “v”, the values in the table are weak; if this
string is “kv”, both keys and values are weak

Notice thatonly objects can be collected from a weak table. Values, such as
numbers and booleans, are not collectible. 如果这个non-object key/value  对应的value/key 是一对象，而这对象

被回收那么整个entry will be collect.

Strings present a subtlety here: although strings are collectible, from an implementation
point of view, they are not like other collectible objects. Other
objects, such as tables and threads, are created explicitly.For instance, whenever
Lua evaluates the expression {}, it creates a new table. However, does
Lua create a new string when it evaluates "a".."b"? What if there is already a
string “ab” in the system? Does Lua create a new one? Can the compiler create
this string before running the program? It does not matter: these are implementation
details. From the programmer’s point of view, strings are values, not
objects. Therefore, like a number or a boolean, a string is not removed from
weak tables (unless its associated value is collected).

一句搞定，booleans, numbers strings are value,not object ,they can't not be collectiable. unless it's assciate value is

an object like table, thread,function are collected

a="mya";b="myb";c="myc"
keyObj1={};
keyObj2={};
keyObj3={};
collect={keyObj1=a};  --注意先放了keyObj1=a 之后才set weaktable,那么keyObj1=a 还是strong reference
m={__mode="kv"};
setmetatable(collect,m);
collect[keyObj2]=b;
collect[c]=keyObj3;
 a,b,c=nil,nil,nil;
 keyObj1,keyObj2,keyObj3=nil,nil,nil;
 collectgarbage(); --forge a garbage collection cycle
for k,v in pairs(collect) do
 print (k,v)  --only have :  keyObj1 mya
end

17.2 Memoize Functions

A common programming technique is to trade space for time. You can speed up
a function by memorizing its results so that, later, when you call the function
with the same argument, the function can reuse that result.

local results = {}
function mem_loadstring (s)
   local res = results[s]
     if res == nil then -- result not available?
           res = assert(load(s)) -- compute new result,load 的话是很expensive to create a new chunk. so better cache it
           results[s] = res -- save for later reuse
   end

  return res;   --找到就重用回上次的结果，就不用再计算多一次了

end

但这种做法会有可能over memory, 比如有的command only use once, some command use frequently. we need to

collect the 那些不常用的,weak table will be the good choice.

local results = {}
setmetatable(results, {__mode = "v"}) -- make values weak
function mem_loadstring (s)

..................

其实return res. 如果我们 用完后res=nil. 而其他地方又没有strong reference,then the whole entry keyresults[s] = res

将会被回收.

另外一个example:

local results = {}
setmetatable(results, {__mode = "v"}) -- make values weak
function createRGB (r, g, b)
    local key = r .. "-" .. g .. "-" .. b
    local color = results[key]
   if color == nil then
       color = {red = r, green = g, blue = b}
       results[key] = color
    end
return color  --这样相同的color 就不会被create 2 次,当这个color no strong reference 的时候又可以被回收
end

17.3 Object Attributes

17.4 Revisiting Tables with Default Values

-------

local default={};
setmetatable(default,{__mode="k"});
local mt={__index=function(t)
   return default[t];
end
};

function setDefault(tb,defaultVal)
  default[tb]=defaultVal; --tb is table object. 而default is a key-weak table.  so after tb is nil, the entry will be collected
  setmetatable(tb,mt);
end
test={};
setDefault(test,"100");
print(test.bbgb); --100
for k,v in pairs(default) do
 print(k,v);
end
test=nil; --
collectgarbage(); --其实default 里面的以test table 为key的 就会被回收.
print("after collectgarbage");
for k,v in pairs(default) do
 print(k,v);  -- will print nothing.
end

----------second approch

local default={};
setmetatable(default,{__mode="v"});
function setDefault(tb,defaultVal)
   local mt=default[defaultVal];
   if mt==nil then
      mt={__index=function() return defaultVal end} -- create a closure to enclose the defaultVal
      default[defaultVal]=mt;  --相同的default value share the same metatable.
   end
   setmetatable(tb,mt);
end

test1={};
test2={};
setDefault(test1,100);
setDefault(test2,100);

print(test2.abc);
for k,v in pairs(default) do
 print(k,v);
end
print("after collect garbage..");
test1=nil;
test2=nil;
collectgarbage();  --when test1,test2 are nil, their same metable will get nil. and the table is value-weak, so collected
for k,v in pairs(default) do
 print(k,v);
end

depends. Both have similar complexity and similar performance.The first
implementation needs a few memory words for each table with a default value
(an entry in defaults). The second implementation needs a few dozen memory
words for each distinct default value (a new table, a new closure, plus an entry in
metas). So, if your application has thousands of tables with a few distinct default
values, the second implementation is clearly superior. On the other hand, if few
tables share common defaults, then you should favor the first implementation.

17.5 Ephemeron Tables

local cache={};
setmetatable(cache,{__mode="k"});

function factory(o)
    local res=cache[o];
       if res==nil then
          res=function () return o end;  -- res 是strong function object,而关键是他的closure refer 住了weak 的o.
          cache[o]=res;
       end
    return res;
end

p={};
f=factory(p);
for k,v in pairs(cache) do
 print(k,v);
end
p=nil;
collectgarbage();
print("after garbage collect..");
for k,v in pairs(cache) do
 print(k,v);
end

Lua 5.2 solves the above problem with the concept of ephemeron tables. In
Lua 5.2, a table with weak keys and strong values is an ephemeron table. In
an ephemeron table, the accessibility of a key controls the accessibility of its
corresponding value. More specifically, consider an entry (k; v) in an ephemeron
table. The reference to v is only strong if there is some strong reference to
k. Otherwise, the entry is eventually removed from the table, even if v refers
(directly or indirectly) to k.

但打印的结果来看， 并没有这个效果， 按他的意思， p=nil 后，就算 f still strong and reference the p,ephemeron table  will collect the enetry.

table: 006CA1B0 function: 0193C858
after garbage collect..
table: 006CA1B0 function: 0193C858  --还是在的，除非p=nil, f=nil, 才会被回收,,,,

17.6 Finalizers

A finalizer is a function associated
with an object that is called when that object is about to be collected.

Lua implements finalizers through the metamethod __gc. See the following
example:
o = {x = "hi"}
setmetatable(o, {__gc = function (o) print(o.x) end})
o = nil
collectgarbage() --> hi

A subtleness of finalizers in Lua is the concept ofmarking an object for
finalization. We mark an object for finalization when we set a metatable for
it with a non-null __gc metamethod. If we do not mark the object, it will not be
finalized. Most code we write works naturally, but some strange cases can occur,
like here:
o = {x = "hi"}
mt = {}
setmetatable(o, mt)
mt.__gc = function (o) print(o.x) end  -- 必须要在setmetatable 的时候赋一个non-null __gc mothod
o = nil
collectgarbage() --> (prints nothing)

In this example, the metatable we set for o does not have a __gc metamethod,
so the object is not marked for finalization. Even if we later set a __gc field to
the metatable, Lua does not detect that assignment as something special, so it
will not mark the object. 

If you really need to set the metamethod later, you can provide any value for
the __gc field, as a placeholder:
o = {x = "hi"}
mt = {__gc = true}
setmetatable(o, mt)
mt.__gc = function (o) print(o.x) end
o = nil
collectgarbage() --> hi

When the collector finalizes several objects in the same cycle, it calls their
finalizers in the reverse order that the objects were marked for finalization.
Consider the next example, which creates a linked list of objects with finalizers: (书上的例子还要花时间)

o1={x="o1"};
o2={x="o2"};
o3={x="o3"};
o4={x="o4"};

setmetatable(o1,{__gc=function(o) print (o.x)end});
setmetatable(o4,{__gc=function(o) print (o.x)end});
setmetatable(o2,{__gc=function(o) print (o.x)end});
setmetatable(o3,{__gc=function(o) print (o.x)end});

o1,o2,o3,o4=nil,nil,nil,nil;
collectgarbage();

o3
o2
o4
o1

---由此更加可以明白这个mark 的作用因为每一次setmetatable 都是把这些对象mark 了finalizer 的order.

然后回收的时候就到过来回收.

==============下面的是书上的例子:

mt = {__gc = function (o) print(o[1]) end}
list = nil
for i = 1, 3 do
list = setmetatable({i, link = list}, mt) -- setmetatable 原来是返回了第一个parameter.
end
list = nil
collectgarbage()
--> 3
--> 2
--> 1

----------由此我们也可以推断出GC   过程的某一部分，就是setmetatable 的时候，会把这个对象所定义的__gc 方法 放到一个LinkList 里面， 当需要回收的时候我们会从ListList 的尾部往前回收，如果__gc 不为nil,  那么调用__gc method.

Another tricky point about finalizers is resurrection. When a finalizer is
called, it gets the object being finalized as a parameter. So, the object becomes
alive again, at least during the finalization. (这么说如果这时候我把这个object assign to a global message, 那这个对象就复活了) I call this a transient resurrection.  While the finalizer runs, nothing stops it from storing the object in a global   variable, for instance, so that it remains accessible after the finalizer returns. I
call this a permanent resurrection.

Resurrection must be transitive. Consider the following piece of code:
A = {x = "this is A"}
B = {f = A}
setmetatable(B, {__gc = function (o) print(o.f.x) end})
A, B = nil --虽然A,B 都变成nil, A，B 都会被回收，但当call __gc of B 的时候，B 复活了，而B 有A 的reference, so A 也要相应的复活 ，，，
collectgarbage() --> this is A
The finalizer for B accesses A, so A cannot be collected before the finalization of
B. Lua must resurrect both B and A before running that finalizer.

这一部分值得反复看，因为他谈到了GC 的关键部分，，平时我们只知道 obj=nil, collectgarbage, 但还需要阅读这部分的内容加深理解。。。。。。

Because of resurrection, objectswith finalizers are collected in two phases.
The first time the collector detects that an object with a finalizer is not reachable,
the collector resurrects the object and queues it to be finalized. Once its finalizer
runs, Lua marks the object as finalized. The next time the collector detects
that the object is not reachable, it deletes the object. If you want to ensure
that all garbage in your program has been actually released, you must call
collectgarbage twice; the second call will delete the objects that were finalized
during the first call.

有set finalizer 的对象的回收是分两步的， 1, 当collector 发现这个对象已经unreachable ,也就是再没什么strong reference 了，那么collector 需要让这个对象复活，也就是把它放到一个queue 里面， 也就是这个queue

strong reference 了这个复活的对象，然后collector  run 这个queue 里面的 finalizer, 把这个复活的对象传入，，运行完这个finalizer 后，mark 住这个对象已经finalized.... 当下一次GC 时候的，发现这个对象需要回收而且已经finalized ,那么

这个时候才是真正delete  。。。。so  if you want to ensure the objects with finalizers had been fully release, you must call collectgarbage twice.,,,哈哈，，不细心看怎么能发现这些奥秘，，，，

The finalizer for each object runs exactly once, due to the mark that Lua puts
on finalized objects. If an object is not collected until the end of a program, Lua
will call its finalizer when the entire Lua state is closed. This last feature allows
a form of [at exit] functions in Lua, that is, functions that will run immediately
before the program terminates. All we have to do is to create a table with a
finalizer and anchors it somewhere, for instance in a global variable:

onExit={
__gc=function()
 print("this is the last call when program exit");
end
};
setmetatable(onExit,onExit);

c:\Trevor\lua\bin>lua
Lua 5.2.2  Copyright (C) 1994-2013 Lua.org, PUC-Rio
> dofile("lession16.lua")
>
this is the last call when program exit^C 在程序退出时run theonExit, and mark _G.onExit
c:\Trevor\lua\bin>lua
Lua 5.2.2  Copyright (C) 1994-2013 Lua.org, PUC-Rio
> dofile("lession16.lua")
> ^C
c:\Trevor\lua\bin>  --太恐怖了吧，在同一个命令行在此运行的时候，上次退出时候的对象回收marked 的finalizer 还在，，看我下面的test:

c:\Trevor\lua\bin>lua
Lua 5.2.2  Copyright (C) 1994-2013 Lua.org, PUC-Rio
> =_G.onExit
nil  --虽然_G 已经没有了onExit
> dofile("lession16.lua")
> =_G.onExit
table: 0017A368
> 这时候在退出就没再调用 onExit 了，因为上一次退出的时候marked.

Another interesting technique allows a program to call a given function every
time Lua completes a collection cycle. As a finalizer runs only once, the trick here
is to make the finalizer creates a new object to run the next finalizer:
do
local mt = {__gc = function (o)
-- whatever you want to do
print("new cycle")
-- creates new object for next cycle
setmetatable({}, getmetatable(o))
end}
-- creates first object
setmetatable({}, mt)
end
collectgarbage() --> new cycle
collectgarbage() --> new cycle
collectgarbage() --> new cycle

The interaction of objects with finalizers and weak tables also has a subtlety.
The collector clears the values in weak tables before the resurrection, while
the keys are cleared after resurrection. 也就是weak-value table 在复活前 key-value 会被clear

but weak-key table 则在after resurrection, 具体点应该是call 完finalizer 之后拉，，，

The following fragment illustrates this
behavior:
-- a table with weak keys
wk = setmetatable({}, {__mode = "k"})
-- a table with weak values
wv = setmetatable({}, {__mode = "v"})
o = {} -- an object
wv[1] = o; wk[o] = 10 -- add it to both tables
setmetatable(o, {__gc = function (o)
print(wk[o], wv[1]) -- 虽然o 被复活了，但是wv 里面的key-value entry 已经在call finalizer 之前被清调了
end})
o = nil; collectgarbage() --> 10 nil