bitvec与脏页

摘自：http://sqlite.1065341.n5.nabble.com/Is-performance-of-v3-5-7-improved-with-new-bitvec-td36830.html

位图是一个非常有趣的话题，在SQLite中通过位图记录下，当前脏的页面，方便事务的回滚，当然了，我们已经记录下该结构对于journal的基本使用方式，大家可能不知道，如果事务发生了回滚，就会产生journal文件，那么情况来了，SQLite是如何记录，到底记录什么信息，保存在journal文件当中，

下面我们查看一个简单的讨论：

有一句话目前翻译还是很有意思：

以前采用位图的方式来处理脏页面的技术受限与SQLite的最大控制数量GB范围（10sof GBs），而不是理论值TB，因为SQLite数据库在处理事务的过程中，强制给数据库每1Mb的数据分配256个字节。

原文如下：

（The "old" way of dealing with dirty pages with bitmapslimited SQLite to an approximate maximal capacity of 10s of GBs, as opposed totherical TBs, because it imposed to malloc 256 bytes for every 1Mb of database duringeach transaction）

 

现在采用bitvec结构体的方式来处理脏页（在SQLite v3.5.7版本中引入），她允许稀疏的位图，并且突破了10s of GBs的限制。

> - The "old" way of dealing with dirty pages with bitmaps limited SQLite 
> to an approximate maximal capacity of 10s of GBs, as opposed to therical 
> TBs, because it imposed to malloc 256 bytes for every 1Mb of database 
> during each transaction. 
> 
> - The "new" way of dealing with dirty pages with a bitvec structure 
> (introduced in SQLite v3.5.7) allows for sparse bitmaps and is then 
> supposed to push away the "10s of GBs" limit.

  Just to be clear, the bitvec stuff can greatly reduce memory use for the 
  average-case, but doesn't change the worst-case.  If you have a 
  transaction that touches a lot of pages (especially if they're spread 
  out in the file) the bitvec can still grow to be quite large. 

> Now the questions are: 
> 1) What are the new practical limits with SQLite v3.5.7? 

  Depends on your environment.  A full-blown desktop with 4GB of RAM is 
  going to have much different practical limits than an iPhone. 

  It also depends on what you're doing.  None of this really matters if 
  you're using the database read-only. 

> 2) Does somebody have any real-life experience (or home-made tests and 
> figures) on SQLite v3.5.7 and really big tables? (say 100 000 000 lines). 

  Personally, I've only gotten to about five or six million rows in a 
  ~6GB db.  That was pre-3.5.7 anyways. 

> 3) Does the new "bitvec" algorithm really help with such a big table? 

  The bitvec stuff has nothing directly to do with table size, only the 
  total database size.  That said, if a single table makes up most of a 
  database, it might be easier to dirty a larger number of pages with a 
  single transaction.  I'm less clear on that aspect, however. 

> I am mainly interested in performance of INSERTs 

  If you mean "speed" when you use the word "performance", the bitvec 
  changes aren't likely to have any significant impact unless the old 
  bit-vector was getting so huge it was forcing the VM system to page 
  things out to disk. 

   -j