chapter 11 Data Structures

11.1 Arrays

We implement arrays in Lua simply by indexing tables with integers. Therefore,
arrays do not have a fixed size, but grow as needed. Usually, when we initialize
the array we define its size indirectly.

a={};
for i=1,100 do
a[i]=i;
end
print(#a);

a1={};
for i=-5,5 do  -- can start the index from any value you want, but lua suggeust from 1 as conversional
a1[i]=i;
end
for i=-5,#a1 do
print(a1[i]);
end

a2={"a","b",1,3,5}; --create and initialize arrays in a single expression
for i=1,#a2 do
print(a2[i]);
end

11.2 Matrices and Multi-Dimensional Arrays

There are two main ways to represent matrices in Lua. The first one is to
use an array of arrays, that is, a table wherein each element is another table.
For instance, you can create a matrix of zeros with dimensions N by M with the
following code:
mt = {} -- create the matrix

N,M=5,6;

for i = 1, N do
mt[i] = {} -- create a new row
for j = 1, M do
mt[i][j] = 0
end
end

for i=1,N do
   
   for j=1,M do
    io.write(mat[i][j])
    io.write(" ");
   end
   io.write("\n");
   
end

-------------lower triangle matrix

mat={};
N,M=5,6
for i=1,N do
 mat[i]={};
 for j=1,i do
   mat[i][j]=0;
 end
end

for i=1,N do
   
   for j=1,M do
    if mat[i][j] then
    io.write(mat[i][j])
    io.write(" ");
    end
   end
   io.write("\n");
   
end

0
0 0
0 0 0
0 0 0 0
0 0 0 0 0

The second way to represent a matrix in Lua is by composing the two indices
into a single one. If the two indices are integers, you can multiply the first one
by a suitable constant and then add the second index. With this approach, the
following code would create our matrix of zeros with dimensions N by M:
mt = {} -- create the matrix
for i = 1, N do
for j = 1, M do
mt[(i - 1)*M + j] = 0  --one table to represent the matrix, 其实跟 openCV  Mat 是一样拉，，，
end
end

if Multiple dimensions then the same as below:

P=5;
matM={};
for i=1,N do
  for j=1,M do
     for k=1,P do
       index=(i-1)*M*P +(j-1)*P + k;
       print(index);
       matM[index]=0;
     end
  end

end

k是最小的维度，让他先增，增到这个纬度的最大value P 时，2nd 维度再增，that's (j-1)P, 一次是增P, after the 2nd get

to the max of its demension, the 3rd the most outside need to increment. increase (i-1)*M*P

我猜OpenCv 里面的Mat 也是怎么实现的因为他的Mat.data member is a uchar*, one demesion array.

If the indices are strings, you can create a single index concatenating both
indices with a character in between to separate them. For instance, you can
index a matrix m with string indices s and t using the code m[s..":"..t],
provided that both s and t do not contain colons; otherwise, pairs like (“a:”,
“b”) and (“a”, “:b”) would collapse into a single index, “a::b”. When in doubt, you
can use a control character like ‘\0’ to separate the indices

string indices 有什么用呢？

Quite often, applications use a sparse matrix, a matrix wherein most elements
are zero or nil. For instance, you can represent a graph by its adjacency
matrix, which has the value x in position m,n when there is a connection with
cost x between nodes m and n. When these nodes are not connected, the value in
position m,n is nil. To represent a graph with ten thousand nodes, where each
node has about five neighbors, you will need a matrix with a hundred million entries
(a square matrix with 10000 columns and 10000 rows), but approximately
only fifty thousand of them will not be nil (five non-nil columns for each row,
corresponding to the five neighbors of each node). Many books on data structures
discuss at length how to implement such sparse matrices without wasting
400 MB of memory, but you seldom need these techniques when programming in
Lua. Because arrays are represented by tables, they are naturally sparse. With
our first representation (tables of tables), you will need ten thousand tables,
each one with about five elements, with a grand total of fifty thousand entries.
With the second representation, you will have a single table, with fifty thousand
entries in it. Whatever the representation, you need space only for the non-nil
elements.
We cannot use the length operator over sparse matrices, because of the holes
(nil values) between active entries. This is not a big loss; even if we could use
it, we should not. For most operations, it would be quite inefficient to traverse
all these empty entries. Instead, we can use pairs to traverse only the non-nil
elements. For instance, to multiply a row by a constant, we can use the following
code:

function mult (a, rowindex, k)  -- this function only for' a' table of table  才适用，否则 a[rowindex] 得到的是number.
local row = a[rowindex]
for i, v in pairs(row) do
row[i] = v * k
end
end

==========prove

do
local N,M=5,6
matK={};
for i=1,N do
   for j=1,M do
      local index=(i-1)*M + j;
      if index%2==0 or index==1 then
        print (index);
       matK[index]=1;
      end
   end
end

print (#matK); --will print 0, if we do not add [or index==1] condition in the if statement  to set the first element not nill

matP={};
for i=1,N do
    matP[i]={};
    for j=1,M do
      if j==1 or j%2==0 then
         matP[i][j]=1;
      end
    
    end
   
end


k=matK[2];
p=matP[2];
print(type(k)); --number
print(type(p));---table

for k,v in pairs(matK) do
      
   print (k,v);-- same as openCV's SparseMat,保留非零元素的order.
end

end

11.3 Linked Lists

Each node is represented by a table and links are simply table fields that contain
references to other tables. For instance, let us implement a basic list, where each
node has two fields, next and value. A simple variable is the list root:
list = nil

To insert an element at the beginning of the list, with a value v, we do:
list = {next = list, value = v}
To traverse the list, we write:
local l = list
while l do
<visit l.value>
l = l.next
end

========correct version should be this

list=nil;
list={value="a",last=list}; --不应该用next ,should be last
print (list,list.last);
list={value="b",last=list};
print (list,list.last);
list={value="c",last=list};
print (list,list.last);
local l=list;
while l do
     print(l.value);
     l=l.last;
end

table: 00D234E8 nil
table: 00D23560 table: 00D234E8 --last 保存的是上一item.
table: 003EA830 table: 00D23560
c
b
a

===========

11.4 Queues and Double Queues

A simple way to implement queues in Lua is with functions insert and remove
from the table library. These functions insert and remove elements in any
position of an array, moving other elements to accommodate the operation.
However, these moves can be expensive for large structures. A more efficient
implementation uses two indices, one for the first element and another for the
last:
function ListNew ()
return {first = 0, last = -1}
end

a=table;  --table libray, i got a way to print his function:
for k,v in pairs(table) do
 print (k,v);
end

---result:

insert  function: 011A38C0
concat  function: 011A3D10
remove  function: 011A3A60
sort    function: 011A4E50
unpack  function: 011A41A0
pack    function: 011A4010

To avoid polluting the global space, we will define all list operations inside a
table, properly called List (that is, we will create a module). Therefore, we
rewrite our last example like this:

首先我们需要注意一下table 的:

a={first=0,last=1};
print(a["last"]);
print(a.last);
last=1;
first="first";
print(a[last]);
print(a[first]);  dot . 语法是refer key-value 特性的 but [] 是order,如果里面是number,but if strring ,just like dot . 语法。

=============

implement version

List = {}
function List.new ()  -- List 里面再定义function. 几乎是对象的encapsulation lar,,
return {first = 0, last = -1}
end

这个queue 只是用了一个table, 靠的是order  number, 然后first, last key 记录的第一与最后一element 的order number.

还有一点就是order number can be any interger number. include negative,so the implement is so easy and intuitve.

Now, we can insert or remove an element at both ends in constant time:
function List.pushfirst (list, value)
local first = list.first - 1
list.first = first
list[first] = value
end
function List.pushlast (list, value)
local last = list.last + 1
list.last = last
list[last] = value
end
function List.popfirst (list)
local first = list.first
if first > list.last then error("list is empty") end
local value = list[first]
list[first] = nil -- to allow garbage collection
list.first = first + 1
return value
end
function List.poplast (list)
local last = list.last
if list.first > last then error("list is empty") end
local value = list[last]
list[last] = nil -- to allow garbage collection
list.last = last - 1
return value
end

11.5 Sets and Bags

Suppose you want to list all identifiers used in a program source; somehow you
need to filter the reserved words out of your listing. Some C programmers could
be tempted to represent the set of reserved words as an array of strings, and

then to search this array to know whether a given word is in the set. To speed
up the search, they could even use a binary tree to represent the set.

In Lua, an efficient and simple way to represent such sets is to put the set
elements as indices in a table. Then, instead of searching the table for a given
element, you just index the table and test whether the result is nil or not. In our
example, we could write the next code:
reserved = {
["while"] = true, ["end"] = true,
["function"] = true, ["local"] = true,

while=true, -- error ,because while is a key word reserve by Lua. we need write: ["while"]=true

jerico="abc" -- it's ok.

}
for w in allwords() do
if not reserved[w] then
<do something with ’w’> -- 'w' is not a reserved word
end
end

----------my version

reserverWords={
["while"]=true,
["for"]=true,
["function"]=true,
jerico=true,
["do"]=true,
["end"]=true
};
kb={"while",
"for","do","end","asdf","sdfasfkde"
};
print("==============--------");
for k,v in pairs(kb) do
  if reserverWords[v] then
   kb[k]=nil;  -- filter the reserver words.
  end

end
for k,v in pairs(kb) do
   print (k,v);
end

----------

You can have a clearer initialization using an auxiliary function to build the
set:
function Set (list)
local set = {}
for _, l in ipairs(list) do set[l] = true end   --  _ 其实就一变量名称
return set
end
reserved = Set{"while", "end", "function", "local", }  --其实不觉得这样有什么好，，，

但可以看到的事返回来的set 每一key 是唯一的，也就是没有重复的key, that why below bag appear.

-----------------

Bags, also called multisets, differ from regular sets in that each element can
appear multiple times. An easy representation for bags in Lua is similar to the
previous representation for sets, but with a counter associated to each key. To
insert an element we increment its counter:
function insert (bag, element)
bag[element] = (bag[element] or 0) + 1
end
To remove an element we decrement its counter:
function remove (bag, element)
local count = bag[element]
bag[element] = (count and count > 1) and count - 1 or nil
end
We only keep the counter if it already exists and it is still greater than zero.其实也是没重复的key,只是用这个key 对应的value 保存了这个key 出现的次数而已。。。。

11.6 String Buffers

Suppose you are building a string piecemeal, for instance reading a file line by
line. Your typical code would look like this:
local buff = ""
for line in io.lines() do
buff = buff .. line .. "\n"
end

Despite its innocent look, this code in Lua can cause a huge performance penalty
for large files ,性能不高，

Why is that? To understand what happens, let us assume that we are
in the middle of the read loop; each line has 20 bytes and we have already
read some 2500 lines, so buff is a string with 50 kB. When Lua concatenates
buff..line.."\n", it allocates a new string with 50020 bytes and copies the
50000 bytes from buff into this new string. That is, for each new line, Lua
moves around 50 kB of memory, and growing. More specifically, the algorithm
is quadratic. After reading 100 new lines (only 2 kB), Lua has already moved
more than 5 MB of memory. When Lua finishes reading 350 kB, it has moved
around more than 50 GB. (This problem is not peculiar to Lua: other languages
wherein strings are immutable values present a similar behavior, Java being
the most famous example.)

To read  an entire file, Lua provides the io.read("*a") option, which reads the file at
once. 

In Lua, we can use a table as the string  buffer.The key to this approach is the table.concat function, which returns the   concatenation of all the strings of a given list. Using concat, we can write our  previous loop as follows:
local t = {}
for line in io.lines() do
t[#t + 1] = line .. "\n" --一行就是一个element
end
local s = table.concat(t) --concat all the element

-------------------my test version

print("===================");
start=os.time();
strbuff="";
for v in io.lines("test.txt") do
strbuff=strbuff ..v.. "\n";
end
--print (strbuff);
print(os.difftime(os.time(),start));

tblebuff={};
start=os.time();
for v in io.lines("test.txt") do
tblebuff[#tblebuff+1]=v..'\n'
end
strbff=table.concat(tblebuff);
--print(strbff);
print(os.difftime(os.time(),start));

This algorithm takes less than 0.5 seconds to read the same file that took almost
a minute to read with the original code. (Nevertheless, for reading a whole file
it is still better to use io.read with the “*a” option.)
We can do even better. The concat function accepts an optional second
argument, which is a separator to be inserted between the strings. Using this
separator, we do not need to insert a newline after each line:

local t = {}
for line in io.lines() do
t[#t + 1] = line
end
s = table.concat(t, "\n") .. "\n"  --这里还需要copy ,reallocate buffer 一次 for concat result

Function concat inserts the separator between the strings,but we still have to
add the last newline. This last concatenation duplicates the resulting string,
which can be quite long. There is no option to make concat insert this extra
separator, but we can deceive it, inserting an extra empty string in t:
t[#t + 1] = ""
s = table.concat(t, "\n")       ---finally this is the best soluation
The extra newline that concat adds before this empty string is at the end of the
resulting string, as we wanted.

11.7 Graphs

这里面讨论的应该是离散数学里面图论中的图graph 的表示问题。。。以后再看吧，，，