《APUE》chapter 14 Advanced I/O 学习笔记(加上自己的代码)

Advanced I/O

Nonblocking  I/O

The  slow  system  calls  are those  that  can  block  forever. They include

•Reads that can block the caller forever if data isn’t present with certain file types  (pipes, terminal devices, and network devices)

•Writes that can block the caller forever if the data can’t be accepted immediately  by these same file types (e.g., no room in the pipe, network flow control)

•Opens  that  block  until  some  condition  occurs  on  certain  file  types  (such  as  an  open  of  a  terminal  device  that  waits  until  an  attached  modem  answers  the  phone,  or  an  open  of  a  FIFO  for  writing  only,when  no  other  process  has  the  FIFO open for reading)

•Reads and writes of files that have mandatory record locking enabled

•Certain ioctl operations

•Some of the interprocess communication functions (Chapter 15)

      We  also said that system calls related to disk I/O are not considered slow, even though the read or write of a disk file can block the caller temporarily. Nonblocking I/O lets us issue an I/O operation, such as an open, read,or write, and  not  have  it  block  forever. If the  operation  cannot  be  completed,  the  call  returns immediately with an error noting that the operation would have blocked.

There are two ways to specify nonblocking I/O for a given descriptor.

1.  If we  call open to  get  the  descriptor, we can  specify  the O_NONBLOCK flag

2.  For a descriptor that is already open, we call fcntl to turn on the O_NONBLOCK file  status  flag  (Section  3.14). Figure3.12  shows  a  function  that  we  can  call  to turn on any of the file status flags for a descriptor.

#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <errno.h>
#include <unistd.h>
#include <myerr.h>

#define BUFFSIZE 50

void set_fl(int fd,int flags)
{
        int val;


        if((val = fcntl(fd,F_GETFL,0)) < 0)
        {
                err_sys("fcntl F_GETFL error\n");
        }


        val |= flags;


        if(fcntl(fd,F_SETFL,val) < 0)
        {
                err_sys("fcntl F_SETEL error\n");
        }
}


void clr_fl(int fd,int flags)
{
        int val;


        if((val = fcntl(fd,F_GETFL,0)) < 0)
        {
                err_sys("fcntl F_GETFL error\n");
        }


        val &= ~flags;

        if(fcntl(fd,F_SETFL,val) < 0)
        {
                err_sys("fcntl F_SETEL error\n");
        }
}

char buf[BUFFSIZE];

int main()
{
        int ntowrite,nwrite;
        char    *ptr;


        ntowrite = read(STDIN_FILENO,buf,sizeof(buf));
        fprintf(stderr,"read %d bytes\n",ntowrite);

        set_fl(STDOUT_FILENO,O_NONBLOCK);

        ptr = buf;

        while(ntowrite > 0)
        {
                errno = 0;
                nwrite = write(STDOUT_FILENO,ptr,ntowrite);
                fprintf(stderr,"nwrite = %d,errno %d\n",nwrite,errno);

                if(nwrite > 0)
                {
                        ptr += nwrite;
                        ntowrite -= nwrite;
                }
        }

        clr_fl(STDOUT_FILENO,O_NONBLOCK);

        exit(0);
}

jasonleaster@ubuntu:/Ad_Pro_in_Unix/chapter_14$ ./a.out < ./temp.txt 2>temp_output.txt 
Hello world!
Hello world!
Hello world!
Hello world!
Hello world!

我在linux下测试的时候没有发生书上的那种nwrite == -1的情况。。。

Record Locking

            

                  Record  locking is  the  term  normally  used  to  describe  the  ability  of  a  process  to prevent  other  processes  from  modifying  a  region  of  a  file  while  the  first  process  is reading  or  modifying  that  portion  of  the  file.

fcntl Record Locking
 

#include <fcntl.h>
int fcntl(int fd ,int cmd ,... /* struct flock *flockptr */ );
Returns: depends oncmd if OK (see following), −1 on error

linux下面找到的flock定义

struct flock
  {
   short int l_type;   /* Type oflock: F_RDLCK, F_WRLCK, or F_UNLCK.  */
   short int l_whence; /* Where `l_start' is relative to (like`lseek').  */
#ifndef __USE_FILE_OFFSET64
   __off_t l_start;    /* Offsetwhere the lock begins.  */
   __off_t l_len;      /* Size of thelocked area; zero means until EOF.  */
#else
   __off64_t l_start;  /* Offsetwhere the lock begins.  */
   __off64_t l_len;    /* Size of thelocked area; zero means until EOF.  */
#endif
   __pid_t l_pid;      /* Processholding the lock.  */
  };

Numerous rules apply to the specification of the region to be locked or unlocked.

•The two elements that specify the starting offset of the region are similar to the last two arguments of the lseek function (Section 3.6). Indeed, the l_whence member is specified as SEEK_SET , SEEK_CUR ,or SEEK_END .

•Locks  can  start  and  extend  beyond  the  current  end  of  file,  but  cannot  start  or extend before the beginning of the file.

•If l_lenis 0, it means that the lock extends to the largest possible offset of the file.  This allows us to lock a region starting anywhere in the file, up through and including  any  data  that  is  appended  to  the  file. (We don’t  have  to  try  to  guess how many bytes might be appended to the file.)

•To lock the entire file, we set l_start and l_whence to point to the beginning of the file and specify a length (l_len) of 0 .(There are  several ways to specify the  beginning  of  the  file,  but  most  applications  specify l_start as  0  and l_whence as SEEK_SET .)

 

          The basic rule is that any number of processes can have a shared read lock on a given byte, but only one process can have an exclusive  write  lock  on  a  given  byte. Further more,  if  there are  one  or  more read  locks
on a byte, there can’t be any write locks on that byte; 

         If a process has an existing lock on a range  of  a  file,  a  subsequent  attempt  to  place  a  lock  on  the  same range  by  the  same process  will  replace  the  existing  lock  with  the  new  one.

F_GETLK 

Determine whether the lock described by flockptr is blocked by some other  lock.  If a lock  exists  that  would  prevent  ours  from  being  created,  the  information on that existing lock overwrites the information pointed to by  flockptr.If no lock exists that would prevent ours from being created, the  structure pointed  to  by flockptr is left  unchanged  except  for  the l_type  member,which is set to F_UNLCK.

F_SETLK 

Set  the  lock  described  by flockptr.If we a re  t rying  to  obtain  a  read  lock  (l_type of F_RDLCK) or a write  lock  ( l_type of F_WRLCK)and  the  compatibility  rule  prevents  the  system  from  giving  us  the  lock

(Figur e14.3), fcntl re turns  immediately  with errno set  to  either  EACCES or EAGAIN

F_SETLKW 

This command is a blocking version of F_SETLK.(The W in the command  name  means wait .)  If the requested  read  lock  or  write  lock  cannot  be  granted because another process currently has some part of the requested  region  locked,  the  calling  process  is  put  to  sleep. The  process  wakes  up  either when the lock becomes available or when interrupted by a signal.

deadlock demo:

/***********************************************************************************************
code write : EOF
code date : 2014.04.08
e-mail: [email protected]


code purpose :
        Just a demo for deadlock
************************************************************************************************/
#include <stdio.h>
#include <fcntl.h>
#include <stdlib.h>
#include <myerr.h>
#include <signal.h>




int lock_reg(int fd,int cmd,int type,off_t offset,int whence,off_t len)
{
        struct flock lock;


        lock.l_type = type;
        lock.l_start = offset;
        lock.l_whence = whence;
        lock.l_len = len;


        return fcntl(fd,cmd,&lock);
}


static void lockabyte(const char *name,int fd,off_t offset)
{
        if(lock_reg(fd,F_SETLKW,F_WRLCK,offset,SEEK_SET,1) < 0)
        {
                err_sys("%s: lock_reg error\n",name);
        }


        printf("%s got the lock,byte %ld\n",name,offset);
}

void parent_handler(int signo)
{
        if(waitpid(-1,NULL,0) < 0)
        {
                printf("waitpid error\n");
        }
}

void child_handler(int signo)
{
/* 
Nothing need to be done....child should not to wait for parent's termination     
*/
}

int main()
{
        int fd;
        pid_t pid;
//-----------------------initialize the signal handler-------------------------------------     
        struct sigaction sa_parent,sa_child;

        sigemptyset(&sa_parent.sa_mask);
        sa_parent.sa_flags = 0;
        sa_parent.sa_handler = parent_handler;

        sigemptyset(&sa_child.sa_mask);
        sa_child.sa_flags = 0;
        sa_child.sa_handler = child_handler;

        if(sigaction(SIGUSR1,&sa_parent,NULL) < 0)
        {
                printf("parent sigaction error\n");
                return 0;
        }

        if(sigaction(SIGUSR2,&sa_child,NULL) < 0)
        {
                printf("child sigaction error\n");
                return 0;
        }

//-----------------------------------------------------------------------------------------     
        if((fd = creat("./templock",(S_IRUSR|S_IWUSR|S_IXUSR))) < 0)
        {
                err_sys("creat error\n");
        }

        if(write(fd,"ab",2) != 2)
        {
                err_sys("write error\n");
        }

        if((pid = fork()) < 0)
        {
                err_sys("fork error\n");
        }
        else if(pid == 0)
        {
                lockabyte("child",fd,0);
                kill(getppid(),SIGUSR1);//      TELL_PARENT( );

                lockabyte("child",fd,1);
        }
        else
        {
                lockabyte("parent",fd,1);
                kill(pid,SIGUSR2);//    TELL_CHILD(pid);

                lockabyte("parent",fd,0);
        }

        exit(0);
}

jasonleaster@ubuntu:/Ad_Pro_in_Unix/chapter_14$ ./a.out
parent got the lock,byte 1
child: lock_reg error
: Resource deadlock avoided parent got the lock,byte 0

Implied Inheritance and Release of Locks

•  Locks are associated with a process and a file.
•  Locks are never  inherited  by  the  child  across  a fork.
•  Locks are inherited  by  a  new  program  across  an exec

demo:

/****************************************************************************
code writer : EOF
code date : 2014.04.08
e-mail : [email protected]
code purpose:
	Just a demonstartion for mandatory locking
If there is something wrong with my code, please touch me by e-mail.

****************************************************************************/
#include <stdio.h>
#include <errno.h>
#include <fcntl.h>
#include <sys/wait.h>
#include <stdlib.h>
#include <myerr.h>

void set_fl(int fd,int flags)
{
	int val;
	
	if((val = fcntl(fd,F_GETFL,0)) < 0)
	{
		err_sys("fcntl F_GETFL error\n");
	}

	val |= flags;
	
	if(fcntl(fd,F_SETFL,val) < 0)
	{
		err_sys("fcntl F_SETEL error\n");
	}
}

int lock_reg(int fd,int cmd,int type,off_t offset,int whence,off_t len)
{
	struct flock lock;
	
	lock.l_type = type;
	lock.l_start = offset;
	lock.l_whence = whence;
	lock.l_len = len;

	return fcntl(fd,cmd,&lock);
}

void child_handler(int signo)
{
	//Nothing to be done. Just a empty handler
}

int main(int argc, char* argv[])
{
	int 		fd;
	pid_t		pid;
	char	 	buf[5];
	struct stat	statbuf;

	if(argc != 2)
	{
		fprintf(stderr,"usage :%s filename\n",argv[0]);
		exit(1);
	}

	if((fd = open(argv[1],O_RDWR | O_CREAT | O_TRUNC,S_IRUSR | S_IWUSR | S_IXUSR)) < 0)
	{
		err_sys("open error\n");
		return 0;
	}

	if(write(fd,"abcdef",6) != 6)
	{
		err_sys("write error\n");
		return 0;
	}

	if(fstat(fd,&statbuf) < 0)
	{
		err_sys("fstat error\n");
	}

	if(fchmod(fd,(statbuf.st_mode & ~S_IXGRP) | S_ISGID) < 0)
	{
		err_sys("fchmod error\n");
		return 0;
	}

	if((pid = fork()) < 0)
	{
		err_sys("for error\n");
	}
	else if(pid > 0)
	{
		//write lock
		if(lock_reg(fd,F_SETLK,F_WRLCK,0,SEEK_SET,6) < 0)
		{
			printf("lock_reg error\n");
		}
		sleep(3);
		//It's neccessay to sleep for a few second to 
		//let the child install the signal handler first.
		kill(pid,SIGUSR1);
		
		if(waitpid(pid,NULL,0) < 0 )
		{
			err_sys("waitpid error\n");
		}
	}
	else
	{
		struct sigaction sa;

		sigemptyset(&sa.sa_mask);
		sa.sa_flags = 0;
		sa.sa_handler = child_handler;
		sigaddset(&sa.sa_mask,SIGUSR1);
	
		if(sigaction(SIGUSR1,&sa,NULL) < 0)
		{
			printf("sigaction error\n");
			return 0;
		}

		pause();//wait to caught the signal
	
		set_fl(fd,O_NONBLOCK);
	
		//read lock
		if(lock_reg(fd,F_SETLK,F_RDLCK,0,SEEK_SET,6) < 0)
		{
			printf("lock_reg error\n");
		}
		
		printf("read lock of alredy-locked region returns %d \n",errno);
	
		if(lseek(fd,0,SEEK_SET) < 0)
		{
			err_sys("lseek error\n");
		}

		if(read(fd,buf,2) < 0)
		{
			err_ret("read failed (mandatory locking works)\n");
		}
		else
		{
			printf("read OK (no mandatory locking),buf = %2.2s\n",buf);
		}

		return 0;
	}
	exit(0);
}

jasonleaster@ubuntu:/Ad_Pro_in_Unix/chapter_14$ ./a.out ./temp.lock 
read lock of alredy-locked region returns 11 
read OK (no mandatory locking),buf = ab

         linux下面支持mandatory locking貌似要个mount -o 命令，倒腾半天。。。没明白。留下以后慢慢探究吧，不想再去问大牛了，总要有问题要学会放下，学会自己慢慢的解决，或许以后就明白了。blog我还是会反复看的。。。

STREAMS

我只能说操蛋。。。linux根本就不支持STREAMS。。。。这是第三版和第二版最大的不同！

APUE preface的原话：

•Linux doesn’tinclude STREAMS in its base system, although packages (LiS and OpenSS7 ) areavailable to add this functionality.

         The STREAMS mechanism is provided by System V as a general way to interface communication drivers into the kernel.  The implementation of STREAMS-based pipes and named pipes.

关于full-duplex的好处的理解：

                 The full-duplex nature of streams and to emphasize that the processing in one direction is separate from the processing in the other direction.

              Data that we write to a stream head is sent downstream. Data read by the device driver is sent upstream.

STREAMS Messages

 

好吧下面是我已经做好的笔记。。。

      All input and output under STREAMS is based on messages. Thestream head and the user process exchange messages using read, write,ioctl, getmsg, getpmsg, putmsg, and putpmsg. Messages are also passed up and down a stream between thestream head, the processing modules, and the device driver.

 

linux下面找到的strbuf的定义

struct strbuf
  {
    int maxlen;         /* Maximum buffer length. */
    int len;            /* Length ofdata.  */
    char *buf;          /* Pointer tobuffer.  */
  };

       When we send a message with putmsg or putpmsg, lens pecifies the number of bytes of data in the buffer. When we receive amessage with getmsg or getpmsg, maxlen specifies the size of the buffer(so the kernel won't over flow the buffer), and len is set by the kernel to the amount of data stored in the buffer. We'll see that a zero-length message is OK and that a len of 1 can specify that there is no control or data.

 

       We'll encounter only three of these message types with the functions we use (read, write, getmsg, getpmsg, putmsg, and putpmsg):

• M_DATA (userdata for I/O)
• M_PROTO(protocol control information)
• M_PCPROTO(high-priority protocol control information)

Every message on a stream has a queueing priority:

• High-priority messages (highest priority)
• Priority band messages
• Ordinary messages (lowest priority)

         Ordinary messages are simply priority band messages with a band of 0. Priority band messages have a band of 1255,with a higher band specifying a higher priority. High-priority messages are special in that only one is queued by the stream head at a time. Additional high-priority messages are discarded when one is already on the stream head's read queue. Each STREAMS module has two input queues.

 

 

putmsg and putpmsg Functions

#include <stropts.h>
int putmsg(intfiledes, const struct strbuf *ctlptr, const struct strbuf *dataptr, int flag);
int putpmsg(intfiledes, const struct strbuf *ctlptr, const struct strbuf *dataptr, int band,int flag);
Both return: 0if OK, 1 on error

    We can also write to a stream, which is equivalent to a putmsg without any control informationand with a flag of 0.

 

STREAMS ioctl Operations

        In Section 3.15,we said that the ioctl function is the catchall for anything that can't be done with the other I/O functions. The STREAMS system continues this tradition.

 

Isastream 函数就是使用ioctl的例子

#include<stropts.h>
intisastream(int filedes);

Returns: 1(true) if STREAMS device, 0 (false) otherwise

Like isatty,this is usually a trivial function that merely tries an ioctl that is validonly on a STREAMS device.

 反正linux下面不管怎么测试都是个坑。。。不支持

 

 

 I/O Multiplexing

       then try to read from the first descriptor again. This type of loop is called polling. Polling should be avoided on a multitasking system.

select and pselect Functions

The select function  lets  us  do  I/O  multiplexing  under  all  POSIX-compatible platforms. 

#include <sys/select.h>
int select(int maxfdp1,fd_set *restrictre adfds, fd_set *restrict writefds,fd_set *restrictexceptfds , struct timeval *restrict tvptr);
Returns: count of ready descriptors, 0 on timeout, −1 on error

There are three conditions.

tvptr == NULL

            Wait forever.This infinite wait can be interrupted if we catch a signal. Return is  made  when  one  of  the  specified descriptors  is  ready  or  when  a  signal  is caught.  If a signal is caught, select re turns −1 with errno set to EINTR.

tvptr !> tv_sec == 0 && tvptr !> tv_usec == 0

              Don’t  wait  at  all. All  the  specified  descriptors  are tested,  and  return  is  made immediately .This is a way to poll the system to find out the status of multiple descriptors without blocking in the select function.

tvptr!>tv_sec != 0 || tvptr!>tv_usec != 0

              Wait the specified number of seconds and microseconds.  Return is made when one  of  the  specified descriptors  is  ready  or  when  the  timeout  value  expires.  If the timeout expires before any of the descriptors is ready,t he return value is 0. (If the system doesn’t provide microsecond resolution, the tvptr!>tv_usec value  is rounded up to the nearest supported value.) As with the first condition, this  wait can also be interrupted by a caught signal.

use one of the following four functions on a variable of this type.

#include <sys/select.h>
int FD_ISSET(int fd ,fd_set *fdset );
Returns: nonzeroif fd is in set, 0 otherwise
void FD_CLR(intfd ,fd_set *fdset );
void FD_SET(intfd ,fd_set *fdset );
void FD_ZERO(fd_set *fdset );

      Any (or all) of the middle three arguments to select (the pointers to the descriptor sets)  can  be  null  pointers  if we’r e not  interested  in  that  condition. If  all  three  pointers are NULL,t hen  we  have  a  higher-precision  timer than  is  provided  by sleep.

As an example, Figure14.16 shows what two descriptor sets look like if we write fd_set  readset, writeset;

FD_ZERO(&readset);
FD_ZERO(&writeset);
FD_SET(0, &readset);
FD_SET(3, &readset);
FD_SET(1, &writeset);
FD_SET(2, &writeset);
select(4, &readset, &writeset, NULL, NULL)

1.  A return  value  of −1 means  that  an  error  occurred.  This can  happen,  for example,  if  a  signal  is  caught before any  of  the  specified  descriptors  are ready. In this case, none of the descriptor sets will be modified.

2.  A re turn  value  of  0  means  that  no  descriptors  are ready.This  happens  if  the time  limit  expires  before any  of  the  descriptors  are ready.When  this  happens, all the descriptor sets will be zeroed out.

3.  A positive return value specifies the number of descriptors that are ready.This value  is  the  sum  of  the descriptors  ready  in  all  three  sets,  so  if  the  same descriptor is ready to be read and written, it will be counted twice in the return value.  The only  bits  left  on  in  the  three  descriptor  sets  are the  bits corresponding to the descriptors that are ready.

readv and writev Functions

The readv and writev functions  let  us  read  into  and  write  from  multiple noncontiguous buffers in a single function call. These operations are called scatter read and gather write .

#include <sys/uio.h>
ssize_t readv(int fd ,const struct iovec *iov,int iovcnt );
ssize_t writev(int fd ,const struct iovec *iov,int iovcnt );
Both return: number of bytes read or written, −1 on error

The second argument to both functions is a pointer to an array of iovec structures:

struct iovec {
void  *iov_base; /* starting address of buffer */
size_t  iov_len; /* size of buffer */
};

Readn and writenFunctions

 

1.  A read operation  may return  less  than asked  for,even  though we  have  not encountered the end of file. This is not an error,and we should simply continue reading from the device.

2.  A writeoperation  can return  less  than we  specified. This  may be  caused  by kernel  output buffers  becoming  full, for  example. Again,  it’s not  an  error,and we  should continue  writing  the remainder  of  the data. (Normally,this  short return from a write occurs only with a nonblocking descriptor or if a signal is

caught.)

        We’ll  never see  this  happen when  reading  or writing  a  disk file,  except  when the  file system  runs out  of  space or  we  hit our  quota  limit and  we  can’t write  all  that we requested.这个可以解释为什么pro_14_1.c的输出为什么没有和书上最后一个显示一样。

 

#include"apue.h"
ssize_treadn(int fd ,void *buf,size_tnbytes );
ssize_twriten(int fd ,void *buf,size_tnbytes );
Both return:number of bytes read or written, −1 on error

Memory-Mapped  I/O

             when we store data in the buffer, the corresponding bytes are automatically written to the file.This lets us perform I/O without using read or write

 

#include<sys/mman.h>
void *mmap(void*addr ,size_tlen,int prot ,int flag,int fd ,off_t off);
Returns:starting address of mapped region if OK,MAP_FAILED on error

             The addr argument lets us specify the address where we want the mapped region to start.  We normally set this value to 0 to allow thesystem to choose the starting address

 

prot                                Description

PROT_READ                Region can be read.

PROT_WRITE              Region can be written.

PROT_EXEC                Region can be executed.

PROT_NONE               Region cannot be accessed.

 

         The  protection specified for  a region  can’t  allow more access  than  the open mode of  the  file. For example,  we can’t specifyPROT_WRITE if the file was opened read-only.

 

 

 

       If we then try to access the memory-mapped region corresponding to the end portion of the file that was truncated, we’ll receive SIGBUS.

 

 

        A memory-mapped  region is  inherited  by  a  child across  a fork (since  it’s part  of the  parent’s address  space),  but for  the  same reason,  is  not  inherited  by the  new program across an exec.

 

 

          A memory-mapped    region is automatically unmapped when the process terminates or  we can  unmap  a region  directly  by calling  the munmap function.  Closing the file descriptor used when we mapped the region does not unmap the region.

#include<sys/mman.h>
int munmap(void*addr ,size_tlen);
Returns: 0 ifOK,−1 on error