SQLite杂念

1.2.8.5故障恢复

the journal keeps an account of all write operations carried out by transaction on the database

日志保存数据库上事务执行的所有写入操作的帐户

Each log record has a header that is a descriptor for the
record, and a payload containing redo-undo information
for the operation that has produced this
log record. The log manager prepares the descriptor, and the log client the payload.

每条日志记录有一个头部(它是该这条记录的描述信息)，还有一个payload(保存的是这条记录的redo和undo信息)。

为了提高效率，日志的一部分会保存在稳定的存储中，剩余的部分保存到不稳定(volatile)的主存中。这样会带来一些问题，为了避免出现系统错误，DBMS通常遵循两条原则来客服这个问题：

(1)在对数据库文件进行更改之前，DBMS创建一个稳定的日志记录(undo)来记录那些已经改变的内容，这条规则被称作预写日志原则(or WAL,in short)

(2)当事务提交时，DBMS确保该事务的所有(redo)日志记录都被保存在稳定的日志存储中，这条规则被成为刷新提交时日志协议。

可知，事务开始时，原始信息保存到WAL中(以便于undo)，事务提交时，更改后的日志记录保存在稳定的日志存储中(以便于redo)。

日志记录一般只是用来写入的，而不是用来读取的，但是当事务出现了不正常的终止，或者出现了系统错误，这时日志记录将会被读取以使得数据库恢复到一个合适的一致性状态。

在数据库崩溃之后，DBMS会重启数据库，首先执行恢复操作，在这个操作(恢复)的过程中，未完成执行的事务(事务已经开始执行，但是还没有被提交)将会被undo，已提交的事务将会被进行redo操作(利用的是刷新提交时日志协议)。

1.2.8.6检查点

A checkpoint establishes a relatively recent database state
and journal content and these can be used as a basis for recovery at future restarts.】Checkpoints
are taken solely to cut down restart processing time.In essence, a checkpoint helps eliminating
some old log records from the journal, and thereby, it helps speeding up restart processing upon
system failures.

检查点提供了相对最近的数据库状态和日志内容，这些是未来恢复数据库的基础。检查点用于减少重新启动处理时间。从本质上讲,检查点有助于从日志中消除一些旧的日志记录,从而有助于加快系统故障时的重新启动处理。

It uses a lock based concurrency control scheme and a journal
based recovery scheme.
SQLite使用锁来进行并发控制，使用日志来进行数据恢复。

SQLite还支持基于UTF-8和UTF-16标准Unicode文本的编码。

The latest release is SQLite 3.7.8 as of September 19, 2011 as of finalizing the
writing of this book. This book is based on this particular release.
到2011年9月19号，最新的版本是3.7.8，这本书正是基于这一特定的版本进行编写的。

Embeddable: You do not need a separate server process dedicated to SQLite. The SQLite
library is embeddable in your own applications. The applications do not need to include any
interprocess communications schemes to interact with SQLite.
对于SQLite，你不需要单独的服务进程，SQLite源码编译出来之后，会形成一个库，这个库可以嵌入到你的应用程序当中。应用程序不需要引入任何进程间通信方案和SQLite交互。

Thread-safe: SQLite is a thread-safe library, and many threads in an application process
can access the same or different databases concurrently. SQLite takes care of database-level
thread concurrency
SQLite是线层安全的库，在应用进程中的很多线程能够同步地访问数据库中相同的或者不同的内容。SQLite从数据库水平保证线程安全。

Single database file: Each database is stored entirely in a single native file; user data and
metadata are stored in the same file. The single file approach eases moving/ copying a database
from one place to another. (SQLite though uses many temporary files while manipulating a
database.)
尽管SQLite数据库会使用许多临时的文件，但是一个数据库使用一个文件进行表示，使用单个文件来表示数据库可以令复制变得轻而易举。

keep it simple and splendid.

2.2.2 SQLiet APIs

SQLite接口定义了一套API。这些接口用于应用程序和SQLite库之前的交互。

1.sqlite3_open: 

这个方法有两个参数，一个输入一个输出。输入：数据库文件名。在数据库连接的内部，该方法打开了数据库文件，如果数据库文件被成功打开了，那么就会返回SQLITE_OK，否则，将会是一个error code。

Lazy File Opening: When opening or creating a database file, SQLite follows a lazy approachthe actual opening or creation is deferred until the file is accessed for reading. If the database file
does exist, SQLite automatically recovers the database into a consistent state if there is a need. The
lazy file creation gives you an opportunity to (re)define values for various database setting parameters
using pragma commands. (These setting parameters will be discussed in Chapter 3

The open function returns a connection handle (a pointer to an object of type sqli te3) via
the output parameter (db, in the preceding example), and the handle is used to apply further
operations on the library connection (for this open SQLite session). The handle represents
the complete state of this library connection.
open方法通过输出参数:db,返回一个连接句柄(这是一个指向sqlite3对象的指针)。这个句柄可以被用于后续的操作。这个句柄代表了这个库连接的完整的状态。

图2.3展示了一个典型的脚本，一个应用程序对于同一个数据库打开了两个数据库连接。这两个连接中的任意一个对应另外一个都是独立的，它们任意一个都代表了一个sqlite3对象。一个简单的sqlite3对象代表了一个管理SQLite库的连接。如图中所示，一个连接包含三个prepared statements，但是另外一个连接拥有0个prepared statements

下面将会对prepared statements进行讨论。

Newer APls: Of late, th e SQLite development t ea m discourages using this open fun ct ion; th ey
recommend t o use t he sqlite3 _open _v2 function instead . There are man y other *_v2 API funct ions.
In order to maintain the simplicity in the presentation of applications , I avoid using these newer API
func t ions in this book.
SQLite开发团队推荐使用sqliet3_open_v2方法。也存在很多其他的*_v2 API方法。但是，为了进行简单性地表示，在本书中不使用这些新的API function。

2.sqlite3_prepare: This function compiles an SQL statement, and produces an equivalent
int erna l object (of typ e sqli te3_stmt ). This obj ect is common ly referr ed to as a prepared
statement in databas e literature , and is implem ented as a bytecod e program in SQLite. A
byt ecode program is an abstract repr esentation of an SQL stat ement tha t is execut ed by a
database engin e. I will discuss the bytecode programming langu age in Section 7.2 on page 175.
I use the terms bytecode program and prep ared sta tement interchangeably in this book to
mean the same. This function returns SQLITE_OK on success, and an appropriate error code
on failure.

这个方法对SQL参数进行编译，之后产生一个对等的内部对象：sqlite3_stmt。从数据库文化来看，这个对象经常会和prepared statement有着千丝万缕的联系。在SQLite中，这个对象将会以字节码程序的方式存在。字节码程序是SQL statement的一个抽象的表示。在7.2节，第175页将会讨论字节码程序原因。使用字节码程序和技术性地称呼：prepated statement这两种表示形式描述的都是同一个事物。

3. sqlite3_step: After an SQL statement has been prepared with a call to the sqli te3_prepare
function, the sqli te3_step function must be called one or more times to execute the prepared statement. Each call to the step function executes the bytecode program until it hits
a break point (because it has produced a new output row), or until it halts (because there
are no more rows). The function returns the caller SQLITE_.ROW in the former case, and
SQLITE_DONE in the latter case. In the former case, the application can read the column
values of the row using the appropriate sqli te3_column_* API functions. (See the next item
on the list.) The step function is called again to retrieve the next row. The step function
moves the position of the cursor for the results of a SELECT statement. Initially, the cursor
points before the first row of the output rowset. Every execution of the step function moves
the cursor pointer to the next row of the rowset. The cursor moves only in the forward direction. For SQL statements that do not return rows (such as UPDATE, INSERT, DELETE,
CREATE, and DROP), the step function always returns SQLITE_DONE because there is
no row to process. Eventually the step function returns SQLITEJ)ONE. (The step function
should not be called again on this statement handle without first calling the sqli te3_reset
function to reset the program execution back to its initial state. I discuss the reset function
shortly.
在sqliet3_prepare方法被调用之后，sqlite3_step方法将会被调用一次或者多次，这是为了执行prepared statement(字节码程序).每次调用step方法都会执行字节码程序，直到遇到break point(因为此时产生了一个新的输出row)或者直到没有更多的row为止。对应前者，该方法返回SQLITE_ROW,对于后者,该方法返回SQLITE_DONE.同样，对于前者，应用程序可以读取该行的某列的值(使用sqlite3_column_*API 方法)，为了读取下一行的数据，step方法将再次被调用。step方法会移动到cursor所在的位置。 Initially, the cursor
points before the first row of the output rowset. Every execution of the step function moves
the cursor pointer to the next row of the rowset. The cursor moves only in the forward direction

4 sqli e3_column_*: If the sqli te_step function returns SQLITE_ROW, you can retrieve the
value of each column of the row by executing one of the sqli te3_column_* API functions.
The datatype mismatch between SQL/SQLite and the C language is handled automatically
by the engine: the column functions convert data between the two languages and from storage
types to requested types. (For example, if the internal representation of a value is FLOAT
and the application requests a text output value, SQ Lite uses sprint£() internally to do the
value conversion.)

该方法返回列的值。如果数据类型在C语言和SQLite之间不一致，那么么engine会自动进行处理。例如：如果内部的表示是FLOAT类型，应用程序请求的是text类型的输出，那么SQLite在内部会使用sprintf方法进行这种转换。
5. sqlite3_finalize: This function closes and destroys a statement handle and the associated
prepared statement. That is, it erases the bytecode program, and frees all resources allocated
to the statement handle. The statement handle becomes invalid and must not be reused.
If the statement was executed successfully or not executed at all, then the finalize function
returns SQLITE_OK. If the previous execution of the statement failed, then the function
returns an error code. The finalize function can be called at any point during the execution
of the prepared statement. If the engine has not completed the statement execution when this
routine is called, it is like encountering an error or an interrupt in the execution. Incomplete
updates are rolled back and the execution is aborted, and the result code returned will be
SQLITE_ABO RT.
这个方法会破坏statement句柄以及相关联的prepared statement，也就是说，擦除字节码程序，释放分配给statement句柄的所有的资源，然后statement句柄将不会再次被使用。

6. sqlite3_close: This function closes a library connection, and frees all resources allocated to
the connection. The connection handle becomes invalid. This function returns SQLITE_OK
on success, and other error codes on failure. If there are prepared statements that have not
been finalized, then SQLITE_BUSY is returned and the connection remains open
这个方法关闭对数据库的连接，释放分配给这个连接的所有资源。这个连接句柄也会变得无效。

2.2.3 Direct SQL execution
It first opens the database file, then applies the statement
to the database by executing the sqli te3_exec API function, and finally closes the database
file. The exec function executes the SQL statement directly, without the need by the application
manually to go through the prepare, step, and finalize API functions, as was done in the previous
example application. In case the statement produces output, the exec function executes the callback
function for each output row and lets the application to process the row further. You must have
read permission on the given database file, and depending on the query type, you may need to have
a write permission on the file and the directory it is contained in.
首先打开数据库，然后通过执行sqlite3_exec API方法来执行statement参数，接着关闭数据库文件。如果语句产生输出，那么exec方法将会执行回调函数，然后应用程序可以继续执行。

sqlite3_exec: This function executes one or more SQL statements directly. (Two consecutive
SQL statements are separated by a semicolon.) Internally, it compiles and executes the statements
one after another, in the left to the right order of input. If any statement execution results in
an error, the remaining statements are not executed. If a statement has SQL parameter markers,
the SQL NULL values are taken. If a statement produces output, the exec function calls a user
specified callback function for each output row. The signature of the callback function can be found
in Fig. 2.4. The exec function is a convenient wrapper for prepare, step, ( column), and finalize
functions. Nonetheless, the SQLite development team discourages in using the function because
they may remove it in future releases.
这个方法执行一个或者更多个SQL语句，当前一个SQL语句执行出错之后，后一个(后面所有的)SQL语句就不执行了。

./app2 MyDB "insert into Students values(lO0)"
./app2 MyDB "insert into Students values(lO)"
./app2 MyDB "insert into Students values(lO00)
 

多线程

Threading Modes: Threading mode is controlled by the SQLITE_THREADSAFE preprocessor macro.
In order to be thread-safe, the SQLite source code must be compiled with the macro set to 1 for serialized
and 2 for normal multithread. If the macro is set to 0, the library is in the single-threading mode. What this
means is that multiple threads in a single process can use the same SQ Lite library, but SQLite ( connection
and statement) handles created by one thread cannot be safely used by another thread; also it is unsafe
to use SQLite by multiple threads simultaneously. In the former two cases, this restriction is relaxed and
the library is said to be 'thread-safe'. In the normal multithreading mode (safeness value 2), although
multiple threads can use the same library connection, they cannot use it simultaneously; they can use the
connection mutually exclusively; they can though use different connections simultaneously. In the serialized
multithreading mode, there is no such restriction. The default is the serialized mode. You can invoke the
sqli te3_threadsafe API function to find out whether the SQLite library you are using is thread-safe or
not. If the compile time option is multithread or serialized, you can change this option at library start time
or runtime using sqlite3_open_v2 or sqlite3_config API function. Figure 2.5 presents a very simple multithreaded application. The application creates 10 threads,
and each of them tries to insert one row in the Students table in the same My DB database. SQLite
implements a lock-based concurrency scheme, so some of the INSERT statements may fail due to
lock conflict. The application does not need to worry about concurrency control and database
consistency issues; it cannot corrupt the database. SQLite takes care of concurrency control and
consistency issues. However, you will need to check for failures, and handle them appropriately
within the code (for example, you might retry the statement that failed, or inform the user that it
failed and let her 3 decide what to do next)

线程安全被SQLITE_THREADSAFE预处理器宏来控制的。为了实现线程安全，必须设置编译SQLite源码，这样有两种作用：设置为1时：实现序列化，设置为2时：实现多线程。如果设置为0：:SQLite库将被设置为单线程模式，

What this
means is that multiple threads in a single process can use the same SQ Lite library, but SQLite ( connection
and statement) handles created by one thread cannot be safely used by another thread;
also it is unsafe
to use SQLite by multiple threads simultaneously.
这意味着在一个进程中的多个线程可以使用相同的SQLite库，但是被某个线程创建的SQLite句柄(连接句柄和语句句柄)不能够被其他的线程使用。

In the normal multithreading mode (safeness value 2), although
multiple threads can use the same library connection, they cannot use it simultaneously; they can use the
connection mutually exclusively; they can though use different connections simultaneously.
尽管多个线程可以使用相同的数据库连接，但是在多线程模式，他们不能同时使用这个连接。他们能够使用独占地使用数据库连接。他们能够同时使用不同的数据库连接。

In the serialized
multithreading mode, there is no such restriction. The default is the serialized mode. You can invoke the
sqli te3_threadsafe API function to find out whether the SQLite library you are using is thread-safe or
not. If the compile time option is multithread or serialized, you can change this option at library start time
or runtime using sqlite3_open_v2 or sqlite3_config API function.

在序列化的多线程模式，没有这个限制。默认的是序列化模式。你通过调用sqlite3_threadsafe API方法能够找出你正在使用的SQLite库是否是线程安全或者线程不安全的。如果编译时选项是多线程或者序列化，那么在库开始运行或者正在运行时你能够通过分别调用sqlite3_open_v2或者sqlite3_config 的API方法来改变这个选项。

Figure 2.5 presents a very simple multithreaded application. The application creates 10 threads,
and each of them tries to insert one row in the Students table in the same My DB database. SQLite
implements a lock-based concurrency scheme, so some of the INSERT statements may fail due to
lock conflict. The application does not need to worry about concurrency control and database
consistency issues; it cannot corrupt the database. SQLite takes care of concurrency control and
consistency issues. However, you will need to check for failures, and handle them appropriately
within the code (for example, you might retry the statement that failed, or inform the user that it
failed and let her 3 decide what to do next).

图2.5提供了一个非常简单的多线程应用程序。这个应用程序创建了10个线程，每个线程都尝试着向MyDb数据库的Studengs表总插入一条数据。SQLite实现了一个基于并发模式的锁，所以一些INSERT语句会由于锁征用而执行失败。但是应用程序不需要关注并发控制和一致性问题。可是，你需要去检查是否发生了失败，并且在代码的内部对这些失败进行处理(比如，你可以重新执行那些实行失败的语句或者告知用户这是一次失败的执行，然后让用户决定下一步该怎样做)。

2.2.5 Working with multiple databases

Figure 2.6 shows a typical SQLite application that works on two databases. (I have simplified
the code by not including error checks for function calls.) The application first opens the MyDB
database, and then attaches the MyDBExtn database to the current library connection. After
completion of the attach command execution, the single library connection has two database connections and the application can now access all tables in the two databases via the same library
connection. I assume that the MyDB database has a Students(SID) table, and the MyDBExtn
database a Courses(name, SID) table. The application executes an SQL select statement that
accesses the two tables from the two databases.
图2.6展示了数据库应用程序使用到了两个数据库。应用程序首先打开MyDB数据库，接下来向当前库连接中附加MyDBExtn数据库。在附加命令执行完成之后，这个单独的库连接拥有两个数据库连接，并且应用程序现在可以通过这个库连接来访问这两个数据库中的所有的表。我假定MyDB数据库拥有Students(SID)表，MyDBExten数据库拥有Courses(name,SID)表。应用程序执行数据库查找语句来从这两个数据库中访问这两个表。

Library Connection vs. Database Connection Confusion: A connection to the SQLite library
can have more than one database associated with it. See Fig. 2.7. In the figure, the single library connection
has three database connections, each to a different database file. The application can access all the databases
via the same library connection. Although the application sees only one library connection, internally SQLite
opens up multiple database connections, one for each database. If a library connection has only one database,we naively call it also a database connection . You have been warned about the connection confusion.

库连接 和 数据库连接的困惑：对SQLite库的连接可以和许多个数据库相关联。在图2.7中，单个库连接有三个数据库连接，每个连接的都是不同的数据库文件。应用程序可以通过这个库连接来访问所有的数据库。尽管对于应用程序来说，只存在一个库连接，我们通常称之为数据库连接。