[Erlang 0070] Erlang Queue

    Queue 是Erlang的队列,它的内部实现充分考虑到了效率,值得学习.估计"如何用链表高效实现Queue"这个也会在面试题目中频繁出现吧.queue模块中除了len/1, join/2, split/2, filter/2 and member/2复杂度是O(n)之外所有的操作的复杂度都在O(1).怎么做到的呢?

 

巧妙的实现

   queue使用两个list来实现,这两个List为{RearList,FrontList} 即{尾端,前端},queue的第一个元素出现在FrontList的Head;queue的最后一个元素出现在RearList的Head.下面创建,检查,转换的方法的执行效率是比较明显的,凡是涉及到遍历的,复杂度都在O(L).

%% O(1)
new() -> {[],[]}. %{RearList,FrontList}

%% O(1)
is_queue({R,F}) when is_list(R), is_list(F) ->
    true;
is_queue(_) ->
    false.

%% O(1)
is_empty({[],[]}) ->
    true;
is_empty({In,Out}) when is_list(In), is_list(Out) ->
    false;
is_empty(Q) ->
    erlang:error(badarg, [Q]).

%% O(len(Q))
len({R,F}) when is_list(R), is_list(F) ->
    length(R)+length(F);
len(Q) ->
    erlang:error(badarg, [Q]).

%% O(len(Q))
to_list({In,Out}) when is_list(In), is_list(Out) ->
    Out++lists:reverse(In, []);
to_list(Q) ->
    erlang:error(badarg, [Q]).

%% O(length(L))
from_list(L) when is_list(L) ->
    f2r(L);
from_list(L) ->
    erlang:error(badarg, [L]).

  

    queue模块所有的方法都会进行数据结构检查,不符合Queue的特征就会抛出异常.由于内部数据结构使用的是List,improper list会引发内部崩溃.索引超出界限会抛出badarg的错误.为了减少queue操作的复杂性,队列本身并没维护长度信息,所以len/1要经过遍历得到所以复杂度是O(n).如果对长度信息敏感,调用者很容易维护queue的长度信息.

 

   list转换到queue的过程使用的f2r方法的作用是:至少有三个元素的情况下,将两个数据元素从尾端RearList移动到前段FrontList,r2f是其逆过程;对应的还有注意下面两个方法是做了inline编译优化的,这个之前已经提到过:[Erlang 0029] Erlang Inline编译  

%% Internal workers

-compile({inline, [{r2f,1},{f2r,1}]}).

%%至少有三个元素的情况下,将两个数据元素从尾端RearList移动到前段FrontList
%% Move all but two from R to F, if there are at least three
r2f([]) ->
    {[],[]};
r2f([_]=R) ->
    {[],R};
r2f([X,Y]) ->
    {[X],[Y]};
r2f([X,Y|R]) ->
    {[X,Y],lists:reverse(R, [])}.

%% Move all but two from F to R, if there are enough
f2r([]) ->
    {[],[]};
f2r([_]=F) ->
    {F,[]};
f2r([X,Y]) ->
    {[Y],[X]};
f2r([X,Y|F]) ->
    {lists:reverse(F, []),[X,Y]}.

%%%%queue:f2r的测试代码

1> q:f2r([a]).
{[a],[]}
2> q:f2r([a,b]).
{[b],[a]}
3> q:f2r([a,b,c]).
{[c],[a,b]}
4> q:f2r([a,b,c,d]).
{[d,c],[a,b]}

10> q:f2r([a,b,c,d,e,f,g,h,i,j]).
{[j,i,h,g,f,e,d,c],[a,b]}
11> q:f2r([a,b,c,d,e,f,g,h,i,j,k]).
{[k,j,i,h,g,f,e,d,c],[a,b]}
12> q:f2r(lists:seq(1,100)).       
{[100,99,98,97,96,95,94,93,92,91,90,89,88,87,86,85,84,83,82,
  81,80,79,78,77,76,75,74,73|...],
[1,2]}

13> q:r2f(lists:seq(1,100)).
{[1,2],
[100,99,98,97,96,95,94,93,92,91,90,89,88,87,86,85,84,83,82,
  81,80,79,78,77,76,75,74|...]}
14> q:r2f([a,b]).          
{[a],[b]}
15> q:r2f([a,b,c]).
{[a,b],[c]}
16> q:r2f([a,b,c,d]).
{[a,b],[d,c]}
17>
 

%% Return true or false depending on if element is in queue
%%
%% O(length(Q)) worst case
-spec member(Item :: term(), Q :: queue()) -> boolean().
member(X, {R,F}) when is_list(R), is_list(F) ->
    lists:member(X, R) orelse lists:member(X, F);
member(X, Q) ->
    erlang:error(badarg, [X,Q]).

  

  入队列操作实际上就是不断的在RearList头部追加元素,所以这里的复杂度也是O(1).

%% O(1)
in(X, {[_]=In,[]}) ->
    {[X], In};
in(X, {In,Out}) when is_list(In), is_list(Out) ->
    {[X|In],Out};
in(X, Q) ->
    erlang:error(badarg, [X,Q]).

  测试代码:

Eshell V5.9  (abort with ^G)
1> queue:new().
{[],[]}
2> queue:in(a,v(1)).
{[a],[]}
3> queue:in(b,v(2)).
{[b],[a]}
4> queue:in(c,v(3)).
{[c,b],[a]}
5> queue:in(d,v(4)).
{[d,c,b],[a]}
6> queue:in(e,v(5)).
{[e,d,c,b],[a]}
7> queue:in(f,v(6)).
{[f,e,d,c,b],[a]}
8> queue:in(g,v(7)).
{[g,f,e,d,c,b],[a]}
9> 

  

  出队列通常复杂度也是咋O(1),最差的情况是O(len(Q));对于RearList和FrontList都有数据的情况下,取出一个数据元素仅仅是从FrontList中取出头元素,所以时间复杂度也是1.如果恰好取出了FrontList的最后一个元素,就要做前后端数据元素的转移.

%% O(1) amortized, O(len(Q)) worst case
out({[],[]}=Q) ->
    {empty,Q};
out({[V],[]}) ->
    {{value,V},{[],[]}};
out({[Y|In],[]}) ->
    [V|Out] = lists:reverse(In, []),
    {{value,V},{[Y],Out}};
out({In,[V]}) when is_list(In) ->
    {{value,V},r2f(In)};
out({In,[V|Out]}) when is_list(In) ->
    {{value,V},{In,Out}};
out(Q) ->
    erlang:error(badarg, [Q]).

10> queue:out(v(8)).  
{{value,a},{[g,f],[b,c,d,e]}}
11> {_,R}=queue:out(v(8)).
{{value,a},{[g,f],[b,c,d,e]}}
12> {_,R2}=queue:out(R).   
{{value,b},{[g,f],[c,d,e]}}
13> {_,R3}=queue:out(R2).
{{value,c},{[g,f],[d,e]}}
14> {_,R4}=queue:out(R3).
{{value,d},{[g,f],[e]}}
15> 

 

  类似的情况还有get方法,大多数情况下复杂度是O(1),最差的情况下需要去RearList的最后一个元素,时间复杂度是O(len(Q)).

%% O(1) since the queue is supposed to be well formed
get({[],[]}=Q) ->
    erlang:error(empty, [Q]);
get({R,F}) when is_list(R), is_list(F) ->
    get(R, F);
get(Q) ->
    erlang:error(badarg, [Q]).

get(R, [H|_]) when is_list(R) ->
    H;
get([H], []) ->
    H;
get([_|R], []) -> % malformed queue -> O(len(Q))
    lists:last(R).

  

 逆转整个队列这样的操作时间复杂度也是O(1),因为只需要把前后段两个List互换就可以了;

%% O(1)
reverse({R,F}) when is_list(R), is_list(F) ->
    {F,R};
reverse(Q) ->
    erlang:error(badarg, [Q]).

 

三种API

    queue是双端数据结构.从front端进入,从rear端出.模块的接口比较丰富,分为: "Original API", the "Extended API" and the "Okasaki API".Original API 和 Extended API 接口都可以用队列进出模型去理解,其中进行了reverse操作都会添加"_r"的后缀.

 

官方文档中的说法:

       地址:http://www.erlang.org/doc/man/queue.html

The "Original API" item removal functions return compound terms with both the removed item and the resulting queue. The "Extended API" contain alternative functions that build less garbage as well as functions for just inspecting the queue ends. Also the "Okasaki API" functions build less garbage.

The "Okasaki API" is inspired by "Purely Functional Data structures" by Chris Okasaki. It regards queues as lists. The API is by many regarded as strange and avoidable. For example many reverse operations have lexically reversed names, some with more readable but perhaps less understandable aliases.

Learn you some Erlang的解释:

        地址:http://learnyousomeerlang.com/a-short-visit-to-common-data-structures#queues

Original API

The original API contains the functions at the base of the queue concept, including: new/0, for creating empty queues, in/2, for inserting new elements, out/1, for removing elements, and then functions to convert to lists, reverse the queue, look if a particular value is part of it, etc.

Extended API

The extended API mainly adds some introspection power and flexibility: it lets you do things such as looking at the front of the queue without removing the first element (see get/1 orpeek/1), removing elements without caring about them (drop/1), etc. These functions are not essential to the concept of queues, but they're still useful in general.

Okasaki API

The Okasaki API is a bit weird. It's derived from Chris Okasaki's Purely Functional Data Structures. The API provides operations similar to what was available in the two previous APIs, but some of the function names are written backwards and the whole thing is relatively peculiar. Unless you do know you want this API, I wouldn't bother with it.

 晚安!